You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-21 - 17:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Tue Jul 21, 2026 12:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
57609911071,18cyclictest8050irq/56-dwc_otg_11:05:530
57629910810,28cyclictest8050irq/56-dwc_otg_10:40:412
57619910619,64cyclictest7950irq/56-dwc_otg_11:05:541
57619910532,33cyclictest0-21swapper/110:50:381
5763991043,75cyclictest0-21swapper/311:00:383
57609910130,46cyclictest0-21swapper/010:40:430
5763999810,66cyclictest7850irq/56-dwc_otg11:05:523
576199987,61cyclictest0-21swapper/111:10:351
576199983,72cyclictest0-21swapper/112:30:481
576199983,72cyclictest0-21swapper/112:30:481
576399964,63cyclictest8050irq/56-dwc_otg_12:30:483
576399964,63cyclictest8050irq/56-dwc_otg_12:30:483
40952960,2sleep00-21swapper/012:15:280
576099953,57cyclictest887-21unixbench-2d10:50:520
576199943,18cyclictest7850irq/56-dwc_otg12:10:461
576399923,13cyclictest8855-21grep11:40:523
5760999132,31cyclictest14547-1kworker/u9:112:30:480
5760999132,31cyclictest14547-1kworker/u9:112:30:480
103792910,6sleep1576199cyclictest08:50:431
576399903,67cyclictest8050irq/56-dwc_otg_11:10:353
576399903,12cyclictest989-21ssh10:50:533
5763998912,61cyclictest7950irq/56-dwc_otg_12:25:533
576199896,12cyclictest0-21swapper/111:45:411
576099896,45cyclictest0-21swapper/011:45:360
576099885,50cyclictest17693-21kworker/0:211:10:370
5760998822,10cyclictest31829-21apt-get11:30:340
576199868,12cyclictest7950irq/56-dwc_otg_12:25:541
576199865,11cyclictest0-21swapper/111:00:391
576199864,15cyclictest24550-21missed_timers10:40:441
576199863,15cyclictest31829-21apt-get11:30:351
297502850,2sleep30-21swapper/312:06:093
576199843,9cyclictest8050irq/56-dwc_otg_10:45:511
576199843,10cyclictest31176-21ssh11:29:471
576199843,10cyclictest31176-21ssh11:29:471
53302830,5sleep2576299cyclictest10:56:122
576399823,15cyclictest24556-21wc10:40:443
94212810,5sleep2576299cyclictest07:20:352
5763998114,52cyclictest0-21swapper/311:30:353
5760998121,36cyclictest0-21swapper/007:45:360
576399804,69cyclictest12104-21sort11:45:433
576199795,67cyclictest0-21swapper/111:40:521
5760997926,6cyclictest0-21swapper/012:10:440
576399783,11cyclictest29336-21ssh10:45:503
576099774,46cyclictest0-21swapper/012:25:550
5760997713,32cyclictest0-21swapper/010:45:500
5761997616,4cyclictest4228-21cat08:35:221
576399753,10cyclictest125-21systemd-journal11:29:453
576399753,10cyclictest125-21systemd-journal11:29:453
5760997548,12cyclictest13-21rcuc/008:05:350
576199742,15cyclictest0-21swapper/108:40:511
576199742,15cyclictest0-21swapper/108:40:511
576099743,56cyclictest11-21rcu_preempt11:00:400
576399732,8cyclictest5692sleep312:10:443
5762997316,35cyclictest7850irq/56-dwc_otg10:50:522
576099724,18cyclictest23912-21ssh11:20:330
576199715,6cyclictest15798-21/usr/sbin/munin10:30:381
576099713,7cyclictest0-21swapper/011:29:470
576099713,7cyclictest0-21swapper/011:29:470
54332710,2sleep10-21swapper/111:36:561
576099702,36cyclictest9-21ksoftirqd/011:40:530
576299693,12cyclictest11797-21ssh11:45:352
576099693,9cyclictest0-21swapper/010:15:330
576399681,7cyclictest8050irq/56-dwc_otg_09:35:233
576199688,48cyclictest0-21swapper/109:51:511
576199688,48cyclictest0-21swapper/109:51:511
185972680,2sleep10-21swapper/111:53:271
576399677,45cyclictest7850irq/56-dwc_otg09:15:483
5763996714,41cyclictest8590-21runrttasks09:40:323
576099669,48cyclictest727-21runrttasks10:22:350
5760996614,31cyclictest0-21swapper/008:30:220
5760996614,31cyclictest0-21swapper/008:30:220
250592660,6sleep1576199cyclictest10:00:441
5761996522,40cyclictest5020-21/usr/sbin/munin12:15:501
576099657,13cyclictest9-21ksoftirqd/012:35:340
576399647,43cyclictest8050irq/56-dwc_otg_10:05:223
576399641,5cyclictest24260-21fschecks_time08:00:403
5762996413,29cyclictest7850irq/56-dwc_otg11:30:362
576299633,32cyclictest31-21ksoftirqd/212:30:492
576299633,32cyclictest31-21ksoftirqd/212:30:492
576199622,7cyclictest20635-21missed_timers11:15:451
576199621,10cyclictest7950irq/56-dwc_otg_12:05:441
5760996240,3cyclictest0-21swapper/010:00:330
5760996237,17cyclictest2817-21kworker/0:009:40:090
5760996237,17cyclictest2817-21kworker/0:009:40:090
5760996210,7cyclictest0-21swapper/007:50:300
576399611,5cyclictest7850irq/56-dwc_otg10:35:493
576199616,11cyclictest8050irq/56-dwc_otg_10:27:561
576199612,10cyclictest25315-21ssh11:21:011
5760996133,14cyclictest8050irq/56-dwc_otg_09:20:520
5760996126,25cyclictest0-21swapper/008:10:330
227082610,3sleep30-21swapper/309:57:403
576399603,4cyclictest7850irq/56-dwc_otg08:40:513
576399603,4cyclictest7850irq/56-dwc_otg08:40:513
5763996012,6cyclictest29486-21ssh09:25:403
576299606,18cyclictest8940-21head11:40:532
576199605,9cyclictest8050irq/56-dwc_otg_07:50:361
576199605,12cyclictest30809-21chrt09:26:391
576199601,8cyclictest8584-21dpkg09:40:321
576099602,40cyclictest0-21swapper/009:27:180
5760996011,24cyclictest13112-21unixbench_multi10:25:530
576399591,4cyclictest8050irq/56-dwc_otg_09:20:353
576399591,4cyclictest8050irq/56-dwc_otg_09:10:493
576299594,13cyclictest0-21swapper/210:45:502
576299593,40cyclictest0-21swapper/211:00:422
576199593,6cyclictest0-21swapper/109:15:481
576199593,5cyclictest7850irq/56-dwc_otg08:38:361
576199593,5cyclictest28980-21cat10:05:431
576199592,5cyclictest5151-21unixbench_multi10:15:521
576199591,6cyclictest26713-21timerandwakeup08:05:511
5760995914,6cyclictest0-21swapper/007:40:340
5760995914,6cyclictest0-21swapper/007:40:340
237522590,2sleep10-21swapper/112:39:171
576399586,5cyclictest7950irq/56-dwc_otg_11:15:463
576399583,4cyclictest0-21swapper/310:20:513
5763995818,37cyclictest4603-21/usr/sbin/munin09:35:473
5763995818,37cyclictest4603-21/usr/sbin/munin09:35:473
576399581,4cyclictest7950irq/56-dwc_otg_10:30:203
5762995813,24cyclictest13282-21tail11:05:532
576199587,10cyclictest8050irq/56-dwc_otg_09:05:211
576199583,9cyclictest8050irq/56-dwc_otg_10:38:511
576199581,9cyclictest8050irq/56-dwc_otg_09:37:481
576199581,9cyclictest8050irq/56-dwc_otg_09:37:481
576199581,7cyclictest22936-21ps07:55:471
576199581,5cyclictest17938-21/usr/sbin/munin09:10:451
576199581,5cyclictest0-21swapper/108:18:471
5760995817,38cyclictest61-21kswapd010:05:330
576399573,4cyclictest7950irq/56-dwc_otg_12:15:333
576399572,6cyclictest23916-21apt-get11:20:343
576399571,5cyclictest0-21swapper/310:08:523
576399571,4cyclictest0-21swapper/310:18:073
5763995712,5cyclictest31272-21/usr/sbin/munin08:20:413
576299573,10cyclictest16040-21apt-get11:10:362
576199578,45cyclictest9650irq/80-mmc010:10:331
576199572,7cyclictest0-21swapper/109:55:381
576199572,4cyclictest0-21swapper/110:24:491
576199571,7cyclictest20952-21ssh11:55:491
5761995714,5cyclictest1381-21sh09:30:461
576099573,40cyclictest0-21swapper/010:10:340
576099571,28cyclictest0-21swapper/007:30:340
576399562,4cyclictest0-21swapper/309:54:173
576399562,4cyclictest0-21swapper/309:54:173
576399561,7cyclictest1253-21/usr/sbin/munin10:10:553
576299568,25cyclictest7850irq/56-dwc_otg12:10:462
576199564,4cyclictest18469-21latency_hist07:45:211
576199564,4cyclictest18469-21latency_hist07:45:211
576199561,6cyclictest24228-21tune2fs08:00:401
576199561,5cyclictest28638-21taskset09:25:081
576199561,10cyclictest8050irq/56-dwc_otg_09:49:111
576099562,40cyclictest0-21swapper/010:39:430
576099561,52cyclictest24227-21cut12:00:360
576399551,3cyclictest0-21swapper/312:35:233
576399551,11cyclictest7950irq/56-dwc_otg_10:32:523
5762995510,31cyclictest13640-21ssh12:25:542
576199551,3cyclictest7850irq/56-dwc_otg12:02:561
5760995514,16cyclictest16543-21aten_rbpower_en09:50:360
5760995514,16cyclictest16543-21aten_rbpower_en09:50:360
576099551,30cyclictest0-21swapper/009:35:220
576399541,5cyclictest26659-21cat08:05:503
576399541,5cyclictest20535-21/usr/sbin/munin07:50:343
576199541,4cyclictest0-21swapper/112:22:251
576099546,26cyclictest0-21swapper/011:55:520
5760995425,14cyclictest20510-21taskset09:55:360
576399532,5cyclictest8050irq/56-dwc_otg_08:38:363
576399531,7cyclictest22839-21ntp_states07:55:463
576399531,3cyclictest0-21swapper/309:50:193
5763995312,4cyclictest15363-21aten_rbpower_po09:05:353
576299534,12cyclictest31171-21rm11:29:452
576299534,12cyclictest31171-21rm11:29:452
576099533,16cyclictest0-21swapper/008:30:390
5760995324,7cyclictest23726-21ssh11:20:220
576099532,29cyclictest17275-21grep09:10:350
576099532,23cyclictest12912-21fschecks_time09:45:410
5760995314,19cyclictest0-21swapper/011:50:330
51712533,42sleep00-21swapper/007:06:190
576399527,4cyclictest7950irq/56-dwc_otg_12:00:363
576399527,4cyclictest7950irq/56-dwc_otg_10:57:073
576399523,4cyclictest7950irq/56-dwc_otg_12:20:353
576399522,9cyclictest14244-21sed07:30:503
576199527,4cyclictest0-21swapper/107:35:361
576199521,6cyclictest9650irq/80-mmc007:30:211
5760995223,10cyclictest8050irq/56-dwc_otg_10:55:440
5760995220,15cyclictest6643-21ls07:10:470
5760995219,18cyclictest0-21swapper/007:15:330
5760995219,18cyclictest0-21swapper/007:15:330
576099521,20cyclictest4931-21missed_timers08:35:430
576399511,9cyclictest8050irq/56-dwc_otg_07:17:203
576399511,9cyclictest8050irq/56-dwc_otg_07:17:203
5763995111,4cyclictest157412chrt11:50:233
576399511,13cyclictest8050irq/56-dwc_otg_08:35:063
576199511,10cyclictest7850irq/56-dwc_otg07:45:431
5760995129,4cyclictest24967-21timerwakeupswit08:00:510
5760995127,7cyclictest15965-21tr10:30:360
576099512,28cyclictest9152-21ssh09:40:420
576099512,28cyclictest14552-21grep09:00:500
576399502,6cyclictest0-21swapper/309:02:153
576399501,3cyclictest8050irq/56-dwc_otg_11:37:103
576399501,3cyclictest0-21swapper/311:55:283
576399501,3cyclictest0-21swapper/308:59:323
5763995010,4cyclictest10510-21unixbench_multi07:20:503
576199503,5cyclictest7850irq/56-dwc_otg09:05:381
576199502,5cyclictest27776-21df_inode08:10:371
576099501,26cyclictest21241-21ssh09:15:380
576399495,6cyclictest57942sleep307:10:343
576399491,2cyclictest0-21swapper/308:45:353
5760994925,5cyclictest8050irq/56-dwc_otg_08:00:210
576099491,28cyclictest9473-21cut07:20:360
576099491,27cyclictest30607-21sed08:15:530
576099491,26cyclictest15972-21grep09:05:430
5762994822,11cyclictest7950irq/56-dwc_otg_09:45:422
576199485,4cyclictest8050irq/56-dwc_otg_07:18:551
576199485,4cyclictest8050irq/56-dwc_otg_07:18:551
576199482,6cyclictest0-21swapper/110:57:411
576199481,5cyclictest7979-21apt-get08:45:351
576199481,10cyclictest7950irq/56-dwc_otg_07:32:091
576099483,24cyclictest0-21swapper/011:35:420
576399472,4cyclictest0-21swapper/308:51:083
576399472,3cyclictest8050irq/56-dwc_otg_08:18:473
5763994711,4cyclictest1406-21processes08:25:473
5763994711,4cyclictest1406-21processes08:25:473
576099479,18cyclictest0-21swapper/012:05:450
5760994723,6cyclictest8011-21cut08:45:360
576099472,25cyclictest125-21systemd-journal08:25:210
576099462,24cyclictest0-21swapper/007:25:380
576299455,4cyclictest0-21swapper/209:20:352
576099454,21cyclictest727-21runrttasks07:35:430
576399442,9cyclictest16492-21munin-run07:40:213
5760994423,4cyclictest6688-21/usr/sbin/munin08:40:430
5760994423,4cyclictest6688-21/usr/sbin/munin08:40:430
576099442,27cyclictest0-21swapper/012:20:440
5760994324,6cyclictest7950irq/56-dwc_otg_08:50:280
5760994215,12cyclictest8050irq/56-dwc_otg_09:00:210
576299413,33cyclictest8050irq/56-dwc_otg_10:05:332
576299412,8cyclictest0-21swapper/210:20:312
5762994126,5cyclictest6106-21fschecks_time07:10:392
5762994112,15cyclictest0-21swapper/209:25:532
576299408,21cyclictest0-21swapper/209:35:282
576299408,21cyclictest0-21swapper/209:35:282
576299407,19cyclictest7850irq/56-dwc_otg09:30:392
5762994028,7cyclictest8050irq/56-dwc_otg_08:05:352
576299401,28cyclictest8050irq/56-dwc_otg_09:40:392
576299401,26cyclictest12873-21ls10:25:512
5762994012,12cyclictest0-21swapper/207:45:332
576299397,4cyclictest0-21swapper/212:09:012
576299392,29cyclictest0-21swapper/211:21:262
5762993915,5cyclictest3255350wget10:10:372
576199391,28cyclictest13327-21ls09:00:221
39532394,22sleep23981-21aten_rbpower_en07:05:362
576299389,18cyclictest0-21swapper/209:05:342
576299383,25cyclictest7950irq/56-dwc_otg_09:50:362
576299383,25cyclictest7950irq/56-dwc_otg_09:50:362
576299381,4cyclictest0-21swapper/207:25:352
576399371,11cyclictest0-21swapper/307:42:463
576399371,11cyclictest0-21swapper/307:42:463
576299376,7cyclictest5371-21grep12:15:532
576299372,6cyclictest0-21swapper/210:15:332
576299372,17cyclictest7950irq/56-dwc_otg_11:55:522
576299371,7cyclictest571950ssh11:37:272
576299371,23cyclictest0-21swapper/208:45:372
5761993720,5cyclictest32429-21grep08:20:521
576399361,4cyclictest0-21swapper/308:13:163
576299368,15cyclictest8050irq/56-dwc_otg_11:15:412
576299365,21cyclictest7950irq/56-dwc_otg_12:20:502
576299365,15cyclictest8050irq/56-dwc_otg_11:55:212
576299364,24cyclictest7950irq/56-dwc_otg_08:30:412
576299363,12cyclictest7950irq/56-dwc_otg_08:00:352
5762993611,4cyclictest0-21swapper/212:00:442
576299356,19cyclictest61-21kswapd007:30:272
576299355,20cyclictest8050irq/56-dwc_otg_10:35:432
576299354,14cyclictest7850irq/56-dwc_otg10:00:492
576299351,8cyclictest0-21swapper/207:55:342
576299351,14cyclictest7950irq/56-dwc_otg_08:15:522
576399341,8cyclictest0-21swapper/307:45:223
576399341,12cyclictest0-21swapper/307:26:593
576299345,17cyclictest14922-21/usr/sbin/munin07:35:502
576299344,23cyclictest0-21swapper/207:50:302
576299343,15cyclictest7850irq/56-dwc_otg12:40:212
5762993412,6cyclictest0-21swapper/209:15:382
5762993410,4cyclictest0-21swapper/209:55:382
576299332,13cyclictest7850irq/56-dwc_otg10:32:262
5762993313,9cyclictest7950irq/56-dwc_otg_09:10:422
5761993311,5cyclictest587450wget07:10:361
576299324,16cyclictest379-21cron08:25:212
576299322,8cyclictest7950irq/56-dwc_otg_09:00:502
576299321,28cyclictest996-21ls08:25:402
576299321,28cyclictest996-21ls08:25:402
576299321,14cyclictest7950irq/56-dwc_otg_08:15:212
576199324,5cyclictest7850irq/56-dwc_otg07:23:561
576299318,4cyclictest6454-21fschecks_time08:40:392
576299318,4cyclictest6454-21fschecks_time08:40:392
5761993110,4cyclictest1151-21sed08:25:421
5761993110,4cyclictest1151-21sed08:25:421
80132300,1sleep20-21swapper/207:15:402
80132300,1sleep20-21swapper/207:15:402
576299304,4cyclictest7950irq/56-dwc_otg_08:55:522
5762993014,5cyclictest7950irq/56-dwc_otg_08:50:452
39822304,16sleep34111-21cpuspeed07:05:373
576299285,13cyclictest7950irq/56-dwc_otg_08:35:512
576299283,7cyclictest9650irq/80-mmc007:40:342
576299283,7cyclictest9650irq/80-mmc007:40:342
46482274,14sleep14694-21proc_pri07:05:481
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional