You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-28 - 15:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot3s.osadl.org (updated Thu May 28, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
485991138,64cyclictest0-21swapper/008:10:100
177752820,2sleep0101ktimersoftd/009:55:030
48599781,41cyclictest0-21swapper/007:25:150
485997727,10cyclictest0-21swapper/011:55:140
267102770,4sleep09-21ksoftirqd/012:39:460
48599755,11cyclictest0-21swapper/007:55:160
105882730,2sleep20-21swapper/207:35:292
48599711,38cyclictest0-21swapper/010:30:160
485997034,6cyclictest7950irq/56-dwc_otg_10:15:040
157222700,2sleep30-21swapper/311:07:583
48599691,42cyclictest0-21swapper/007:30:180
485996820,24cyclictest21133-21unixbench-2d10:35:320
485996629,22cyclictest0-21swapper/007:15:160
48599652,27cyclictest0-21swapper/010:00:150
38132650,1sleep30-21swapper/311:32:173
48599644,11cyclictest0-21swapper/011:50:160
48599641,49cyclictest26032-21grep11:20:230
488996321,13cyclictest26984-21fschecks_count10:05:203
488996321,13cyclictest26984-21fschecks_count10:05:203
485996336,7cyclictest7950irq/56-dwc_otg_09:25:050
48899626,8cyclictest7950irq/56-dwc_otg_08:05:173
485996240,5cyclictest1318-21ssh09:35:210
48599621,35cyclictest21780-21fschecks_count11:15:230
126002620,3sleep10-21swapper/112:21:431
488996116,26cyclictest8050irq/56-dwc_otg_10:30:223
485996130,5cyclictest10802-21switchtime11:40:310
48599612,33cyclictest0-21swapper/011:05:220
486996011,12cyclictest19922-21ssh10:35:141
485996037,6cyclictest141-21systemd-journal09:20:040
485996035,7cyclictest7950irq/56-dwc_otg_10:20:050
485996014,27cyclictest27708-21basename08:25:050
485996014,27cyclictest27708-21basename08:25:050
155332600,3sleep121-21rcuc/111:45:561
155332600,3sleep121-21rcuc/111:45:561
92502590,19sleep28050irq/56-dwc_otg_11:39:372
485995943,7cyclictest0-21swapper/010:25:060
48599592,25cyclictest682-21apt09:35:050
48599582,27cyclictest0-21swapper/011:45:310
48599582,27cyclictest0-21swapper/011:45:310
48599581,37cyclictest0-21swapper/010:40:310
48599581,29cyclictest6596-21wget12:15:190
48599581,29cyclictest6596-21wget12:15:190
485995737,6cyclictest7950irq/56-dwc_otg_09:30:050
485995726,7cyclictest21708-21grep08:05:330
48599571,30cyclictest5827-21unixbench_multi08:50:330
488995621,20cyclictest18781-21ssh09:17:273
488995621,15cyclictest12551-21ssh09:48:233
488995618,13cyclictest16196-21sort09:15:063
488995618,13cyclictest16196-21sort09:15:063
488995617,20cyclictest8050irq/56-dwc_otg_08:55:183
488995614,23cyclictest8050irq/56-dwc_otg_10:55:113
485995641,3cyclictest0-21swapper/007:50:210
485995630,5cyclictest6624-21unixbench_singl09:40:340
485995626,14cyclictest17881-21missed_timers11:10:260
485995626,14cyclictest17881-21missed_timers11:10:260
485995615,5cyclictest5021-21fschecks_count10:55:220
312652560,2sleep10-21swapper/111:25:471
48899553,8cyclictest0-21swapper/312:05:153
488995521,18cyclictest23745-21rm11:55:463
488995521,11cyclictest4562-21latency_hist10:55:063
488995518,21cyclictest24866-21ssh10:02:303
48799557,28cyclictest0-21swapper/212:35:182
48599552,33cyclictest0-21swapper/008:55:320
48599552,29cyclictest0-21swapper/008:15:180
48599552,26cyclictest0-21swapper/011:00:240
488995423,18cyclictest0-21swapper/309:20:183
488995417,11cyclictest4535-21date08:50:063
488995417,11cyclictest4535-21date08:50:063
488995416,8cyclictest3618-21users12:10:333
485995434,5cyclictest9397-21grep09:00:320
48599541,32cyclictest11235-21unixbench_multi09:05:320
48599541,28cyclictest1092-21ntp_kernel_pll_10:50:270
488995317,21cyclictest14580-21taskset07:46:513
488995311,12cyclictest18792-21/usr/sbin/munin08:00:253
48599539,20cyclictest0-21swapper/012:00:340
485995328,12cyclictest14402-21grep07:45:320
48599532,28cyclictest30396-21grep11:25:240
48599532,28cyclictest0-21swapper/008:40:040
48599531,32cyclictest13-21rcuc/009:45:160
277282530,2sleep30-21swapper/309:28:093
488995217,11cyclictest8050irq/56-dwc_otg_12:30:243
485995228,7cyclictest5359-21unixbench_multi07:20:340
485995227,5cyclictest2477-21latency_hist08:45:050
485995226,5cyclictest7950irq/56-dwc_otg_10:10:050
485995226,5cyclictest7950irq/56-dwc_otg_10:10:050
48599522,28cyclictest0-21swapper/007:10:270
485995220,7cyclictest18611-21munin-run12:30:050
48599521,34cyclictest6305-21ssh11:35:250
488995124,19cyclictest12173-21ssh12:20:573
488995120,20cyclictest3193-21rm09:37:003
488995117,16cyclictest23961-21ssh11:17:363
488995115,19cyclictest374-21ssh09:34:363
487995114,11cyclictest8050irq/56-dwc_otg_09:45:342
48599512,30cyclictest11257-21chrt12:20:250
488995019,16cyclictest14108-21df09:50:203
488995016,18cyclictest27760-21taskset10:44:233
488995016,11cyclictest14004-21ssh11:45:173
488995016,11cyclictest14004-21ssh11:45:173
488995014,12cyclictest7867-21rm11:37:073
488995013,9cyclictest0-21swapper/308:50:073
488995013,12cyclictest26447-21ntpq11:20:283
48799502,3cyclictest7950irq/56-dwc_otg_07:30:182
48599509,21cyclictest0-21swapper/011:30:260
48599503,37cyclictest2306-21apt-get12:10:150
48599502,27cyclictest0-21swapper/007:35:230
48599501,26cyclictest18891-21ntp_kernel_pll_09:55:260
48599501,24cyclictest29320-21/usr/sbin/munin10:45:280
488994921,12cyclictest5204-21/usr/sbin/munin07:20:323
488994916,12cyclictest17327-21ssh12:27:463
488994910,22cyclictest0-21swapper/312:19:013
488994910,22cyclictest0-21swapper/312:19:013
48799496,19cyclictest0-21swapper/209:40:172
48599492,29cyclictest31385-21latency_hist08:35:050
326212493,12sleep00-21swapper/007:09:230
488994819,18cyclictest5244-21ssh09:40:103
488994819,14cyclictest24621-21ssh12:35:573
488994817,17cyclictest15163-21ssh10:29:493
488994817,13cyclictest982350/usr/sbin/munin11:40:303
488994816,11cyclictest32453-21ssh10:10:503
488994815,16cyclictest22228-21ssh10:37:133
487994833,4cyclictest8050irq/56-dwc_otg_11:03:282
487994815,15cyclictest8050irq/56-dwc_otg_12:10:332
48699487,28cyclictest0-21swapper/108:10:101
48599482,24cyclictest0-21swapper/012:35:050
48599481,6cyclictest29498-21munin-run08:30:040
488994724,11cyclictest7075-21/usr/sbin/munin07:25:313
488994714,11cyclictest31160-21taskset08:33:303
488994713,11cyclictest0-21swapper/308:35:583
48799478,24cyclictest0-21swapper/211:30:172
48799476,24cyclictest15663-21apt-get10:30:182
487994715,16cyclictest8050irq/56-dwc_otg_12:30:192
48599476,23cyclictest0-21swapper/008:45:110
48599476,23cyclictest0-21swapper/008:45:110
485994729,6cyclictest7850irq/56-dwc_otg07:40:310
48899467,25cyclictest0-21swapper/307:15:103
488994617,11cyclictest10346-21rm11:00:363
488994612,20cyclictest7850irq/56-dwc_otg10:24:483
488994612,20cyclictest19424-21ssh11:12:083
488994612,20cyclictest19424-21ssh11:12:083
48799463,15cyclictest8050irq/56-dwc_otg_11:00:052
487994627,6cyclictest7850irq/56-dwc_otg09:20:182
487994615,16cyclictest8050irq/56-dwc_otg_12:21:512
48799461,17cyclictest7950irq/56-dwc_otg_08:05:152
486994622,6cyclictest2503450wget09:25:181
485994632,4cyclictest8050irq/56-dwc_otg_12:10:050
48599462,22cyclictest0-21swapper/008:00:060
48599461,28cyclictest14034-21ssh09:11:400
48599461,28cyclictest14034-21ssh09:11:400
488994515,13cyclictest0-21swapper/307:55:183
488994514,16cyclictest8050irq/56-dwc_otg_11:50:513
48799459,16cyclictest8050irq/56-dwc_otg_10:05:192
48799459,16cyclictest8050irq/56-dwc_otg_10:05:192
48899449,20cyclictest8050irq/56-dwc_otg_11:28:323
488994415,20cyclictest8050irq/56-dwc_otg_12:00:313
488994415,11cyclictest4674-21ssh10:16:283
488994414,11cyclictest1889-21taskset07:12:393
487994415,14cyclictest8050irq/56-dwc_otg_09:25:542
487994414,8cyclictest4262-21apt-get07:20:172
487994410,5cyclictest26294-21/usr/sbin/munin11:20:262
487994410,12cyclictest32638-21rm09:34:152
48699447,23cyclictest0-21swapper/108:45:111
48699447,23cyclictest0-21swapper/108:45:111
486994421,13cyclictest2554-21latency_hist10:15:051
48699442,10cyclictest0-21swapper/108:00:181
48599441,22cyclictest12590-21unixbench_multi10:25:310
48899438,23cyclictest0-21swapper/308:10:173
488994318,11cyclictest20416-21ssh09:57:123
488994317,10cyclictest0-21swapper/310:45:163
488994315,11cyclictest12746-21chrt07:41:303
488994312,11cyclictest0-21swapper/308:22:083
488994312,11cyclictest0-21swapper/308:22:083
48799438,14cyclictest6594-21aten_rbpower_en08:55:182
48799437,24cyclictest0-21swapper/211:15:122
488994214,11cyclictest8050irq/56-dwc_otg_08:27:403
48799427,25cyclictest0-21swapper/207:15:112
487994216,13cyclictest8050irq/56-dwc_otg_10:46:532
487994213,15cyclictest8050irq/56-dwc_otg_11:45:152
487994213,15cyclictest8050irq/56-dwc_otg_11:45:152
486994216,10cyclictest11716-21diskmemload12:10:051
48699421,17cyclictest11154-21users11:40:341
488994114,12cyclictest1165-21snmpd08:42:593
48799418,15cyclictest9250-21ssh10:21:582
48799417,25cyclictest8050irq/56-dwc_otg_12:05:162
48799412,14cyclictest19082-21fschecks_count08:00:222
488994013,10cyclictest16630-21chrt07:54:583
48799409,19cyclictest18360-21rm09:16:452
48799408,17cyclictest2716-21ssh10:52:222
48799407,23cyclictest0-21swapper/207:45:182
48799406,12cyclictest1822-21ssh10:13:572
48799404,24cyclictest0-21swapper/211:10:182
48799404,24cyclictest0-21swapper/211:10:182
48799403,20cyclictest31224-21ssh11:25:432
487994020,10cyclictest8050irq/56-dwc_otg_10:38:552
487994016,10cyclictest6796-21/usr/sbin/munin07:25:282
487994013,10cyclictest8050irq/56-dwc_otg_12:02:052
48799401,14cyclictest16121-21ssh09:15:062
48799401,14cyclictest16121-21ssh09:15:062
48699407,23cyclictest0-21swapper/108:55:171
486994019,4cyclictest18427-21/usr/sbin/munin11:50:211
48799397,21cyclictest0-21swapper/207:40:182
48799397,20cyclictest0-21swapper/212:25:192
487993922,10cyclictest8050irq/56-dwc_otg_11:59:382
48799392,13cyclictest4640-21apt-config08:50:062
486993924,12cyclictest23-21ksoftirqd/107:50:151
486993915,17cyclictest23-21ksoftirqd/110:30:161
48899389,9cyclictest0-21swapper/307:35:313
488993810,8cyclictest0-21swapper/309:05:163
488993810,13cyclictest9782-21chrt09:03:373
48799388,12cyclictest4665-21ssh10:16:272
48799387,5cyclictest14609-21ssh11:05:542
48799384,24cyclictest14034-21ssh10:27:502
48799383,18cyclictest30316-21cat08:30:252
48799382,21cyclictest8050irq/56-dwc_otg_11:50:522
48799381,26cyclictest26330-21ls08:20:212
48799381,26cyclictest26330-21ls08:20:212
48799381,24cyclictest0-21swapper/208:15:192
487993812,13cyclictest8050irq/56-dwc_otg_12:16:332
487993812,13cyclictest8050irq/56-dwc_otg_12:16:332
487993810,13cyclictest7950irq/56-dwc_otg_08:50:062
48699386,5cyclictest0-21swapper/111:09:471
486993820,4cyclictest11716-21diskmemload12:25:201
486993818,4cyclictest15357-21latency_hist10:30:051
48699381,6cyclictest2278250ssh10:00:221
486993810,7cyclictest0-21swapper/111:30:181
48899379,15cyclictest8050irq/56-dwc_otg_08:16:433
488993726,8cyclictest8782-21ls07:30:303
48799377,20cyclictest0-21swapper/208:10:102
487993716,6cyclictest301ktimersoftd/210:02:282
487993714,4cyclictest7950irq/56-dwc_otg_08:50:062
48799371,10cyclictest8050irq/56-dwc_otg_10:40:152
48699379,5cyclictest18141-21ssh09:55:161
48699378,7cyclictest18461-21unixbench_multi11:10:331
48699378,7cyclictest18461-21unixbench_multi11:10:331
48699378,11cyclictest8050irq/56-dwc_otg_09:05:051
48699371,22cyclictest11206-21ls10:25:061
486993711,12cyclictest0-21swapper/109:45:381
111552370,1sleep20-21swapper/211:40:342
48799368,4cyclictest8050irq/56-dwc_otg_08:25:182
48799361,19cyclictest0-21swapper/208:40:192
48699369,6cyclictest7950irq/56-dwc_otg_12:30:151
48699368,11cyclictest0-21swapper/112:38:151
48699367,23cyclictest0-21swapper/107:55:181
48699367,12cyclictest8050irq/56-dwc_otg_12:10:211
48699365,9cyclictest8050irq/56-dwc_otg_08:20:061
48699363,12cyclictest29011-21unixbench-2d08:25:321
48699361,9cyclictest16014-21idleruntime-cro09:15:051
48699361,9cyclictest16014-21idleruntime-cro09:15:051
486993611,11cyclictest0-21swapper/110:05:071
486993611,11cyclictest0-21swapper/110:05:071
48799358,13cyclictest8050irq/56-dwc_otg_08:35:202
48799353,18cyclictest0-21swapper/209:37:272
48699359,4cyclictest0-21swapper/110:17:371
486993528,4cyclictest23-21ksoftirqd/112:15:241
486993528,4cyclictest23-21ksoftirqd/112:15:241
486993527,5cyclictest26561-21grep12:00:181
486993518,4cyclictest23-21ksoftirqd/110:55:201
48699351,20cyclictest29895-21ls09:30:261
48799346,3cyclictest0-21swapper/207:55:192
48799341,24cyclictest0-21swapper/207:50:212
48799341,23cyclictest18776-21missed_timers09:55:252
48699346,6cyclictest21-21rcuc/107:25:191
48699344,8cyclictest7950irq/56-dwc_otg_09:20:201
486993417,4cyclictest2670-21latency_hist08:45:061
486993412,4cyclictest0-21swapper/109:50:181
487993313,6cyclictest7850irq/56-dwc_otg09:51:322
48699336,6cyclictest12604-21sed07:40:331
48699336,14cyclictest8050irq/56-dwc_otg_09:35:331
48699333,4cyclictest21070-21apt-get11:15:101
48699332,7cyclictest6681-21wc11:35:311
48699332,7cyclictest20359-21cat09:20:051
486993319,4cyclictest8050irq/56-dwc_otg_11:22:091
48699331,6cyclictest1545-21grep10:50:321
486993313,5cyclictest1252-21/usr/sbin/munin07:10:291
486993312,5cyclictest256350wget07:15:191
48799327,7cyclictest11313-21grep09:05:322
487993226,4cyclictest301ktimersoftd/209:00:172
48699329,6cyclictest7850irq/56-dwc_otg08:55:051
48699324,12cyclictest26085-21ssh12:00:011
48699323,6cyclictest8050irq/56-dwc_otg_07:38:071
48699323,5cyclictest29817-21sh10:45:341
48699322,7cyclictest14403-21tail07:45:321
486993226,4cyclictest23-21ksoftirqd/109:40:221
486993212,5cyclictest8390-21wc07:30:241
48699313,5cyclictest24611-21grep10:40:221
486993110,5cyclictest0-21swapper/111:00:151
48699304,6cyclictest0-21swapper/108:35:301
48699304,16cyclictest0-21swapper/108:32:441
48699302,19cyclictest0-21swapper/108:20:181
48699302,19cyclictest0-21swapper/108:20:181
311372306,11sleep30-21swapper/307:05:173
48799291,5cyclictest640-21tr07:10:192
48699294,6cyclictest10604-21irqstats09:05:231
48699291,5cyclictest4580-21fschecks_time07:20:231
48699281,4cyclictest20611-21apt-get08:05:151
321132284,16sleep132195-21grep07:05:311
324802263,14sleep20-21swapper/207:07:392
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional