You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-14 - 00:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Mon Jul 13, 2026 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2896622980,1sleep10-21swapper/107:10:171
291769913624,29cyclictest12103-21/usr/sbin/munin09:15:450
29176991175,84cyclictest0-21swapper/011:10:410
29176991142,22cyclictest8050irq/56-dwc_otg_10:25:330
29176991057,72cyclictest0-21swapper/008:40:230
2917699962,77cyclictest8746-21ps09:10:440
2917699953,65cyclictest0-21swapper/010:40:210
29176999256,12cyclictest0-21swapper/010:35:410
29176998652,28cyclictest0-21swapper/008:45:320
2917699816,67cyclictest0-21swapper/009:05:300
29179997723,26cyclictest8050irq/56-dwc_otg_10:25:333
94012730,2sleep30-21swapper/311:09:313
2917699715,30cyclictest0-21swapper/008:20:310
59502700,6sleep12917799cyclictest10:25:411
58122700,3sleep123-21ksoftirqd/111:44:161
156162700,3sleep10-21swapper/111:55:391
2917699692,32cyclictest0-21swapper/010:00:290
29176996921,22cyclictest0-21swapper/010:10:290
125492670,2sleep112548-21ssh12:30:461
40902660,4sleep14089-21wc09:00:181
2917799662,28cyclictest7850irq/56-dwc_otg10:35:431
28062660,2sleep30-21swapper/309:42:473
49372650,2sleep30-21swapper/311:42:413
189712650,2sleep30-21swapper/308:10:373
2917999643,43cyclictest17690-21ssh10:40:193
29178996314,28cyclictest0-21swapper/210:35:412
29176996336,6cyclictest8050irq/56-dwc_otg_12:10:170
2917899612,53cyclictest11157-21ls11:10:422
29177996116,30cyclictest13048-21ssh09:15:461
2917999602,4cyclictest0-21swapper/309:50:283
29178996016,26cyclictest518-21rs:main2
2917999584,26cyclictest0-21swapper/309:15:453
29178995813,24cyclictest7950irq/56-dwc_otg_10:40:202
29176995836,5cyclictest7950irq/56-dwc_otg_09:45:290
2917699581,33cyclictest5115-21unixbench_multi07:30:460
29176995813,23cyclictest12898-21latency_hist07:55:170
2917699581,23cyclictest0-21swapper/011:00:310
29179995714,20cyclictest8782-21processes09:10:453
29177995716,17cyclictest7950irq/56-dwc_otg_09:10:431
2917699579,24cyclictest7021-21cron12:25:170
2917699571,26cyclictest16067-21grep12:35:360
2917999563,35cyclictest11263-21grep11:10:443
29179995629,13cyclictest0-21swapper/310:55:313
2917699567,28cyclictest3401-21unixbench-2d08:55:460
29176995632,7cyclictest7950irq/56-dwc_otg_09:30:460
29176995628,16cyclictest22274-21apt11:25:170
29176995628,16cyclictest22274-21apt11:25:170
2917699561,5cyclictest8710-21grep07:40:450
2917699555,47cyclictest0-21swapper/009:25:170
29176995528,15cyclictest15617-21grep11:15:470
2917699551,28cyclictest14116-21unixbench_multi07:55:460
29176995425,14cyclictest3762-21modprobe07:30:170
2917899537,34cyclictest0-21swapper/210:25:332
2917699533,28cyclictest0-21swapper/008:00:470
29176995323,6cyclictest7527-21diskmemload09:40:170
2917699532,30cyclictest2454-21sed10:20:460
2917699531,44cyclictest0-21swapper/011:55:440
2917699527,24cyclictest1550-21modprobe11:00:170
2917699523,18cyclictest0-21swapper/007:15:300
2917699521,33cyclictest7646-21grep12:25:340
2917699521,25cyclictest6957-21grep07:35:450
2917799514,9cyclictest7850irq/56-dwc_otg10:40:211
2917799513,10cyclictest0-21swapper/111:10:441
29176995131,5cyclictest7950irq/56-dwc_otg_11:35:170
29176995127,6cyclictest10493-21grep07:45:460
29176995122,16cyclictest1716-21taskset11:39:210
2917699511,30cyclictest0-21swapper/010:54:470
2917699511,30cyclictest0-21swapper/010:54:470
288222512,42sleep00-21swapper/007:09:120
29179995010,13cyclictest0-21swapper/310:35:423
29178995014,8cyclictest0-21swapper/209:15:442
2917699509,21cyclictest0-21swapper/012:10:420
2917699502,26cyclictest6781-21rm11:45:330
2917699501,16cyclictest0-21swapper/009:05:180
2917699497,19cyclictest20030-21ssh12:00:420
2917699493,27cyclictest0-21swapper/011:30:170
2917699492,36cyclictest0-21swapper/011:55:020
2917699492,36cyclictest0-21swapper/011:55:020
2917699492,24cyclictest0-21swapper/009:50:390
29176994915,23cyclictest0-21swapper/012:30:320
2917699491,27cyclictest0-21swapper/010:31:460
2917699491,26cyclictest0-21swapper/009:55:440
2917699491,26cyclictest0-21swapper/009:55:440
2917999486,27cyclictest16979-21ssh10:00:323
29179994810,15cyclictest39-21ksoftirqd/307:50:333
2917799489,9cyclictest7604-21runrttasks07:40:281
29176994825,3cyclictest0-21swapper/011:05:310
29176994824,14cyclictest21037-21modprobe10:05:180
2917699481,26cyclictest2689-21aten_rbpower_po11:40:320
2917699481,23cyclictest0-21swapper/008:10:380
29176994810,30cyclictest0-21swapper/010:15:400
2917999477,25cyclictest0-21swapper/308:00:323
29176994727,6cyclictest7850irq/56-dwc_otg10:45:480
2917699471,26cyclictest468-21aten_rbpower_vo07:20:310
2917699471,26cyclictest0-21swapper/009:42:540
2917999467,24cyclictest0-21swapper/308:35:303
2917799457,26cyclictest0-21swapper/109:30:301
2917699453,7cyclictest31527-21ssh12:15:290
2917699453,7cyclictest31527-21ssh12:15:290
29176994520,5cyclictest29143-21mktemp08:40:170
29176994520,11cyclictest0-21swapper/009:27:290
29176994520,11cyclictest0-21swapper/009:27:290
29176994517,11cyclictest2112-21idleruntime-cro08:55:170
2917999447,23cyclictest0-21swapper/308:40:293
2917999442,30cyclictest0-21swapper/307:22:423
2917899447,22cyclictest0-21swapper/207:15:332
2917799446,4cyclictest8050irq/56-dwc_otg_12:10:301
2917799445,27cyclictest7850irq/56-dwc_otg12:05:351
2917699441,33cyclictest0-21swapper/008:30:350
280572446,23sleep327351-21/usr/sbin/munin07:05:433
29179994315,11cyclictest381ktimersoftd/307:35:353
2917899437,22cyclictest0-21swapper/208:15:322
2917799434,16cyclictest13450-21ssh09:55:391
2917799434,16cyclictest13450-21ssh09:55:391
2917799433,5cyclictest8050irq/56-dwc_otg_11:30:501
2917699431,29cyclictest0-21swapper/008:05:320
40272420,4sleep121-21rcuc/110:23:351
2917999428,24cyclictest0-21swapper/311:45:323
2917999427,24cyclictest37-21rcuc/310:10:473
2917799423,5cyclictest8050irq/56-dwc_otg_11:15:291
2917999417,26cyclictest0-21swapper/312:25:303
2917999417,22cyclictest0-21swapper/307:25:323
29178994118,6cyclictest27984-21sh10:51:532
29178994118,6cyclictest27984-21sh10:51:532
29176994124,5cyclictest7850irq/56-dwc_otg08:25:330
29176994124,5cyclictest7850irq/56-dwc_otg08:25:330
2917699412,27cyclictest0-21swapper/008:15:230
29176994111,21cyclictest0-21swapper/007:10:520
2917999404,18cyclictest37-21rcuc/310:22:143
2917999401,6cyclictest8050irq/56-dwc_otg_08:25:303
2917999401,6cyclictest8050irq/56-dwc_otg_08:25:303
2917999401,24cyclictest0-21swapper/308:20:313
29179994010,18cyclictest37-21rcuc/309:20:423
2917899407,21cyclictest0-21swapper/209:05:302
2917899407,21cyclictest0-21swapper/208:05:312
29178994025,5cyclictest28899-21rm09:35:322
2917799401,3cyclictest7850irq/56-dwc_otg11:35:271
2917999396,2cyclictest0-21swapper/311:20:363
2917999396,2cyclictest0-21swapper/311:20:363
2917999394,17cyclictest39-21ksoftirqd/308:05:393
2917999391,7cyclictest8050irq/56-dwc_otg_08:55:313
2917899397,20cyclictest0-21swapper/207:35:302
2917899394,20cyclictest0-21swapper/207:40:282
2917799391,8cyclictest8050irq/56-dwc_otg_07:30:291
29177993914,4cyclictest0-21swapper/110:01:001
148962390,3sleep337-21rcuc/311:55:213
2917999387,19cyclictest37-21rcuc/312:04:483
2917999386,18cyclictest39-21ksoftirqd/310:16:313
2917999384,11cyclictest8050irq/56-dwc_otg_07:30:343
2917999383,6cyclictest0-21swapper/310:45:303
2917999382,18cyclictest39-21ksoftirqd/307:45:333
29178993820,7cyclictest8050irq/56-dwc_otg_12:35:302
2917799389,15cyclictest1285-21ssh09:40:441
2917799388,3cyclictest0-21swapper/110:12:591
29177993818,6cyclictest18730-21sh12:38:501
153762380,2sleep20-21swapper/211:55:362
2917999378,18cyclictest39-21ksoftirqd/311:35:353
2917999377,17cyclictest39-21ksoftirqd/312:10:383
2917999376,16cyclictest37-21rcuc/312:16:233
2917999376,16cyclictest37-21rcuc/312:16:233
2917999375,14cyclictest8050irq/56-dwc_otg_11:00:353
29179993726,4cyclictest8050irq/56-dwc_otg_09:35:433
2917899374,23cyclictest21210-21ssh10:05:322
2917799379,6cyclictest7850irq/56-dwc_otg12:00:331
2917799376,5cyclictest0-21swapper/112:25:291
29177993719,4cyclictest30523-21fschecks_time10:55:351
2917799370,3cyclictest8050irq/56-dwc_otg_09:25:291
2917799370,3cyclictest8050irq/56-dwc_otg_09:25:291
23882370,1sleep10-21swapper/112:19:221
23882370,1sleep10-21swapper/112:19:221
219932370,1sleep20-21swapper/211:24:482
219932370,1sleep20-21swapper/211:24:482
2917999366,6cyclictest39-21ksoftirqd/308:50:403
2917999363,6cyclictest26759-21ssh11:30:313
2917999363,15cyclictest7950irq/56-dwc_otg_07:20:183
2917999363,12cyclictest8050irq/56-dwc_otg_08:45:383
2917899365,9cyclictest8050irq/56-dwc_otg_09:42:482
2917899361,6cyclictest7850irq/56-dwc_otg10:15:302
29178993616,4cyclictest7112-21fschecks_time11:45:372
29178993612,7cyclictest8682-1kworker/u9:211:50:412
29178993612,7cyclictest8682-1kworker/u9:211:50:412
2917899361,24cyclictest23695-21ls12:05:402
29177993612,4cyclictest0-21swapper/110:45:351
2917999357,6cyclictest381ktimersoftd/310:05:233
2917999357,14cyclictest39-21ksoftirqd/309:04:153
2917999354,6cyclictest39-21ksoftirqd/310:30:453
2917999353,19cyclictest37-21rcuc/309:31:393
2917999352,28cyclictest8050irq/56-dwc_otg_09:05:293
2917899357,16cyclictest0-21swapper/208:40:232
2917899357,10cyclictest7950irq/56-dwc_otg_11:40:172
29178993519,7cyclictest8050irq/56-dwc_otg_12:00:282
29178993518,4cyclictest8419-21latency_hist09:50:172
2917799359,5cyclictest0-21swapper/110:30:351
2917799356,5cyclictest22655-21aten_rbpower_cu11:25:311
2917799355,15cyclictest8050irq/56-dwc_otg_08:10:171
2917799352,5cyclictest9085-21taskset09:50:351
2917799351,23cyclictest11170-21cut11:50:361
2917799351,23cyclictest11170-21cut11:50:361
289472351,2sleep20-21swapper/210:15:002
2917999346,14cyclictest7950irq/56-dwc_otg_07:13:323
2917999345,9cyclictest39-21ksoftirqd/312:05:443
2917999342,6cyclictest8050irq/56-dwc_otg_10:50:293
2917999342,6cyclictest8050irq/56-dwc_otg_10:50:293
29179993413,10cyclictest7527-21diskmemload09:25:203
29179993413,10cyclictest7527-21diskmemload09:25:203
29179993410,11cyclictest7850irq/56-dwc_otg12:30:483
2917899349,10cyclictest0-21swapper/211:15:462
2917899347,7cyclictest7950irq/56-dwc_otg_10:05:162
2917899342,16cyclictest3042950ssh10:55:342
29178993414,6cyclictest3447-21unixbench_multi08:55:462
29178993411,10cyclictest0-21swapper/209:59:322
29178993411,10cyclictest0-21swapper/209:59:322
29178993410,4cyclictest0-21swapper/211:00:352
29178993410,11cyclictest29-21rcuc/208:30:452
2917799349,4cyclictest2312-21aten_rbpower_vo07:25:331
2917799346,5cyclictest20313-21aten_rbpower_po08:15:331
29177993417,6cyclictest0-21swapper/110:05:231
29177993414,6cyclictest31495-21/usr/sbin/munin08:45:361
2917999339,13cyclictest381ktimersoftd/311:26:473
2917999337,14cyclictest0-21swapper/312:20:313
2917999337,13cyclictest7950irq/56-dwc_otg_07:40:333
2917999335,12cyclictest381ktimersoftd/312:39:333
2917999334,18cyclictest37-21rcuc/307:55:343
2917999334,16cyclictest0-21swapper/309:55:313
2917999334,16cyclictest0-21swapper/309:55:313
2917899338,14cyclictest8050irq/56-dwc_otg_09:30:172
2917899338,14cyclictest8050irq/56-dwc_otg_09:30:172
2917899336,8cyclictest2881-21ssh11:40:352
2917899333,17cyclictest7850irq/56-dwc_otg12:18:532
2917899333,17cyclictest7850irq/56-dwc_otg12:18:532
2917899332,18cyclictest0-21swapper/207:24:462
2917899331,15cyclictest7850irq/56-dwc_otg10:30:342
2917799337,5cyclictest8050irq/56-dwc_otg_09:05:461
2917799336,5cyclictest0-21swapper/110:19:531
2917799334,19cyclictest0-21swapper/109:46:091
2917799332,7cyclictest28002-21load08:35:371
2917799331,20cyclictest15305-21ls08:00:391
2917999326,9cyclictest37-21rcuc/311:50:373
2917999326,9cyclictest37-21rcuc/311:50:373
29179993210,10cyclictest37-21rcuc/311:17:513
2917899329,5cyclictest0-21swapper/212:30:172
2917899329,10cyclictest0-21swapper/209:52:582
2917899326,11cyclictest7950irq/56-dwc_otg_11:05:432
2917899325,13cyclictest8050irq/56-dwc_otg_07:30:402
2917899321,18cyclictest16515-21ls08:05:182
2917899321,17cyclictest8050irq/56-dwc_otg_10:45:462
2917799328,13cyclictest0-21swapper/111:10:041
2917799324,5cyclictest7850irq/56-dwc_otg12:20:511
2917799322,5cyclictest0-21swapper/108:25:321
2917799322,5cyclictest0-21swapper/108:25:321
29177993218,8cyclictest12743-21cron07:55:171
29177993211,4cyclictest7926-21sed11:45:481
2917899319,9cyclictest8050irq/56-dwc_otg_08:28:052
2917899319,9cyclictest8050irq/56-dwc_otg_08:28:052
2917899317,10cyclictest7950irq/56-dwc_otg_12:30:352
2917899317,10cyclictest7950irq/56-dwc_otg_12:20:452
2917899316,8cyclictest7850irq/56-dwc_otg07:45:412
2917899316,4cyclictest0-21swapper/211:25:322
2917899315,7cyclictest29525-21ssh12:12:162
2917899311,6cyclictest30523-21ssh11:35:172
2917899311,13cyclictest7850irq/56-dwc_otg07:10:442
2917799318,3cyclictest0-21swapper/111:00:301
2917799312,7cyclictest585-21aten_rbpower_en08:50:311
29177993115,5cyclictest29443-21basename08:40:301
2917799311,20cyclictest29235-21ls09:35:361
2917799311,16cyclictest7850irq/56-dwc_otg10:50:391
2917799311,16cyclictest7850irq/56-dwc_otg10:50:391
2917999304,4cyclictest37-21rcuc/309:45:333
2917899307,14cyclictest0-21swapper/209:30:302
2917899302,8cyclictest16653-21ssh09:20:362
2917899302,14cyclictest0-21swapper/208:20:442
29178993012,6cyclictest8050irq/56-dwc_otg_10:22:092
2917799309,3cyclictest0-21swapper/111:23:431
2917799309,3cyclictest0-21swapper/111:23:431
2917799307,14cyclictest0-21swapper/109:20:311
2917799302,6cyclictest7850irq/56-dwc_otg08:30:361
2917799301,7cyclictest9386-21/usr/sbin/munin07:45:291
2917799301,5cyclictest29702-21sort07:10:361
2917999298,4cyclictest8050irq/56-dwc_otg_08:15:393
2917999297,19cyclictest0-21swapper/308:30:313
2917899296,8cyclictest7950irq/56-dwc_otg_08:50:462
2917899293,5cyclictest7950irq/56-dwc_otg_08:10:432
2917799299,8cyclictest0-21swapper/108:10:431
2917799293,5cyclictest7850irq/56-dwc_otg07:20:171
2917799293,13cyclictest8050irq/56-dwc_otg_07:20:401
2917899285,4cyclictest8050irq/56-dwc_otg_08:45:362
2917899283,10cyclictest8050irq/56-dwc_otg_09:00:462
2917799284,5cyclictest22279-21cut08:20:341
2917799283,5cyclictest0-21swapper/107:55:291
2917799282,2cyclictest0-21swapper/107:35:281
2917899272,5cyclictest8050irq/56-dwc_otg_07:50:382
2917899271,10cyclictest7850irq/56-dwc_otg08:39:572
2917799271,10cyclictest7850irq/56-dwc_otg09:00:371
2917899261,9cyclictest13754-21grep07:55:432
2917899261,12cyclictest8050irq/56-dwc_otg_07:25:452
286742252,14sleep20-21swapper/207:07:232
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional