You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-03 - 21:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Mon Aug 03, 2026 12:43:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
235079911315,65cyclictest13069-21ssh10:45:300
235079911315,65cyclictest13069-21ssh10:45:300
23510991094,15cyclictest13058-21latency_hist10:45:293
23510991094,15cyclictest13058-21latency_hist10:45:293
23507991065,85cyclictest0-21swapper/007:10:310
1095321050,3sleep10-21swapper/111:22:501
23508991003,63cyclictest13053-21cat10:45:291
23508991003,63cyclictest13053-21cat10:45:291
2350799985,79cyclictest0-21swapper/010:45:320
23507999817,59cyclictest26116-21aten_rbpower_vo10:20:440
23510999610,65cyclictest7850irq/56-dwc_otg10:45:313
79492920,2sleep00-21swapper/010:38:210
2350899906,55cyclictest8050irq/56-dwc_otg_10:45:311
2350799888,76cyclictest0-21swapper/008:20:410
2350799888,76cyclictest0-21swapper/008:20:410
2351099873,14cyclictest8050irq/56-dwc_otg_10:20:443
2351099862,10cyclictest4785-21ssh09:53:303
2350899843,74cyclictest26111-21ssh10:20:441
114232840,2sleep20-21swapper/209:20:552
58392830,2sleep20-21swapper/209:55:282
2350799809,15cyclictest20800-21ssh11:35:410
66742780,6sleep12350899cyclictest11:58:121
23507997843,16cyclictest8050irq/56-dwc_otg_09:10:390
77502770,1sleep30-21swapper/309:16:093
2351099756,16cyclictest25191-21ssh11:00:413
2350799755,13cyclictest9-21ksoftirqd/010:50:410
2351099743,11cyclictest0-21swapper/307:10:303
2350899744,55cyclictest22133-21/usr/sbin/munin10:55:571
2350899744,55cyclictest22133-21/usr/sbin/munin10:55:571
2350899742,44cyclictest8050irq/56-dwc_otg_12:00:591
2350899742,44cyclictest8050irq/56-dwc_otg_12:00:591
2350899733,61cyclictest14845-21ssh12:08:411
23507997329,11cyclictest0-21swapper/008:10:400
23509997232,10cyclictest13100-21cat10:45:292
23509997232,10cyclictest13100-21cat10:45:292
2350899723,62cyclictest0-21swapper/107:10:301
2350899723,47cyclictest0-21swapper/111:05:551
2350899721,60cyclictest850-21/usr/sbin/munin11:50:491
2350899714,60cyclictest4778-21ssh09:53:281
2350899712,53cyclictest1955-21diskmemload10:11:241
2350899712,46cyclictest0-21swapper/108:45:461
2350899711,59cyclictest0-21swapper/110:05:521
2351099707,12cyclictest7950irq/56-dwc_otg_12:00:553
2351099707,12cyclictest7950irq/56-dwc_otg_12:00:553
2350999708,31cyclictest0-21swapper/208:20:412
2350999708,31cyclictest0-21swapper/208:20:412
2350899703,40cyclictest0-21swapper/110:17:181
2350899701,52cyclictest0-21swapper/110:00:501
23507997024,30cyclictest0-21swapper/007:20:430
23507997024,30cyclictest0-21swapper/007:20:430
23507997020,10cyclictest0-21swapper/011:50:410
2350799701,57cyclictest24914-21ps12:20:540
275102690,2sleep00-21swapper/009:40:550
2351099692,7cyclictest8050irq/56-dwc_otg_10:00:553
2350899695,53cyclictest9849-21missed_timers08:00:511
2350899695,52cyclictest7659-21ssh11:19:221
2350899695,50cyclictest26841-21diskstats09:40:451
2350899681,55cyclictest21647-21ssh12:16:281
2350899681,46cyclictest7278-21latency_hist12:40:281
2350799688,9cyclictest0-21swapper/008:05:390
23507996832,17cyclictest7950irq/56-dwc_otg_11:55:580
2350799682,53cyclictest5896-21unixbench_singl11:15:590
2350799681,51cyclictest19091-21ssh10:10:570
228362680,4sleep339-21ksoftirqd/309:35:473
2350999678,52cyclictest0-21swapper/209:45:352
2350899672,56cyclictest21144-21fschecks_time11:35:471
2350899672,51cyclictest7850irq/56-dwc_otg11:45:381
2350899672,49cyclictest0-21swapper/110:00:281
2350899671,57cyclictest19798-21ssh10:53:211
2350899671,54cyclictest5681-21/usr/sbin/munin07:50:431
2350799671,6cyclictest24855-21apt11:00:280
2350799671,6cyclictest24855-21apt11:00:280
2350799671,54cyclictest0-21swapper/011:10:500
2350799671,54cyclictest0-21swapper/011:10:500
2350899667,47cyclictest8921-21ssh10:40:181
2350899663,47cyclictest7950irq/56-dwc_otg_11:27:081
2350899663,41cyclictest17183-21timerandwakeup12:10:571
2350899662,50cyclictest31399-21ssh12:29:561
2350899662,49cyclictest5198-21ssh10:35:291
2350899662,44cyclictest0-21swapper/109:07:061
23507996632,9cyclictest0-21swapper/012:25:400
23507996618,15cyclictest0-21swapper/009:53:290
23507996612,36cyclictest0-21swapper/010:25:430
2351099658,5cyclictest7950irq/56-dwc_otg_10:55:583
2351099658,5cyclictest7950irq/56-dwc_otg_10:55:583
2351099652,8cyclictest14570-21grep09:25:423
2351099652,8cyclictest14570-21grep09:25:423
2350899655,46cyclictest17711-21kworker/u8:012:22:111
2350899653,49cyclictest3728-21ssh11:14:251
2350899653,49cyclictest3728-21ssh11:14:251
2350899651,54cyclictest26589-21ssh11:01:091
2350799652,48cyclictest9587-21fschecks_time08:00:470
23507996516,34cyclictest2883650kworker/u9:209:45:300
2350799651,47cyclictest1496-21sed09:05:590
2351099643,57cyclictest4963-21cut11:15:473
2350899649,40cyclictest8050irq/56-dwc_otg_07:50:251
2350899644,45cyclictest15741-21/usr/sbin/munin09:25:591
2350899644,45cyclictest15741-21/usr/sbin/munin09:25:591
23508996410,35cyclictest727-21runrttasks09:47:041
2350799646,24cyclictest0-21swapper/012:15:420
2350799644,46cyclictest0-21swapper/009:00:410
2350799641,50cyclictest26172-21munin-run09:40:280
2350799641,44cyclictest18733-21grep09:30:460
231912643,12sleep20-21swapper/207:09:572
2351099631,6cyclictest7850irq/56-dwc_otg11:35:173
2351099631,6cyclictest7850irq/56-dwc_otg11:35:173
23510996311,5cyclictest3090-21ssh09:10:533
2350899631,50cyclictest5699-21runrttasks09:14:531
2350899631,47cyclictest19822-21ssh09:31:041
2350799636,42cyclictest9-21ksoftirqd/011:00:520
2350799632,47cyclictest0-21swapper/012:10:590
2350799632,3cyclictest0-21swapper/011:45:400
2350799631,33cyclictest0-21swapper/009:15:400
2351099622,5cyclictest0-21swapper/312:20:543
2351099621,8cyclictest7850irq/56-dwc_otg11:55:473
23510996212,40cyclictest8050irq/56-dwc_otg_10:05:553
2350899623,49cyclictest0-21swapper/109:21:361
2350899622,42cyclictest0-21swapper/111:31:271
2350899622,42cyclictest0-21swapper/111:31:271
2350899621,47cyclictest0-21swapper/111:40:371
2350899621,30cyclictest8050irq/56-dwc_otg_09:35:421
41602610,2sleep20-21swapper/211:15:182
41602610,2sleep20-21swapper/211:15:182
2351099618,9cyclictest8050irq/56-dwc_otg_07:50:283
2351099612,7cyclictest7850irq/56-dwc_otg11:50:523
2350899611,42cyclictest9974-21ssh09:20:171
2350799614,41cyclictest0-21swapper/012:30:560
23507996130,15cyclictest8050irq/56-dwc_otg_09:56:000
2350799612,29cyclictest0-21swapper/008:35:390
23507996114,29cyclictest32010-21grep07:30:580
23507996112,36cyclictest9-21ksoftirqd/012:05:490
58872600,2sleep20-21swapper/209:15:152
2351099601,8cyclictest28060-21grep08:50:533
2351099601,8cyclictest28060-21grep08:50:533
23510996015,6cyclictest31141-21unixbench_multi10:25:583
2350899603,46cyclictest1040-21ssh10:30:211
2350899602,4cyclictest0-21swapper/112:30:451
2350899602,42cyclictest750-21snmpd07:55:521
2350899601,45cyclictest27987-21ntpq08:50:511
2350899601,45cyclictest27987-21ntpq08:50:511
2350799601,35cyclictest25829-21uniq08:45:480
2351099591,9cyclictest17411-21ssh10:50:433
23509995941,10cyclictest16117-21sh12:10:402
2350899594,43cyclictest30424-21taskset08:56:141
23507995934,6cyclictest0-21swapper/010:15:460
2351099588,8cyclictest8050irq/56-dwc_otg_10:37:423
2351099583,5cyclictest7950irq/56-dwc_otg_09:30:563
23510995818,37cyclictest12425-21fschecks_count12:05:463
2351099581,7cyclictest9921-21tr08:00:523
23510995813,5cyclictest2102-21missed_timers10:30:523
2350899586,31cyclictest7850irq/56-dwc_otg08:20:501
2350899586,31cyclictest7850irq/56-dwc_otg08:20:501
2350799582,6cyclictest8616-21apt-get11:20:350
2351099576,10cyclictest8050irq/56-dwc_otg_09:47:323
2351099574,4cyclictest8050irq/56-dwc_otg_12:35:543
2351099573,4cyclictest7850irq/56-dwc_otg09:44:303
2351099571,4cyclictest0-21swapper/309:05:293
23510995711,5cyclictest0-21swapper/309:20:513
2350799574,49cyclictest11-21rcu_preempt08:55:540
2350799571,46cyclictest28677-21kworker/0:210:07:530
2351099562,6cyclictest8050irq/56-dwc_otg_10:19:153
2351099561,13cyclictest0-21swapper/309:57:543
2350899565,40cyclictest8050irq/56-dwc_otg_07:40:201
2350899562,36cyclictest0-21swapper/107:22:171
2350899562,36cyclictest0-21swapper/107:22:171
23507995631,4cyclictest8050irq/56-dwc_otg_11:25:410
23507995613,28cyclictest9-21ksoftirqd/009:25:420
23507995613,28cyclictest9-21ksoftirqd/009:25:420
2350799561,21cyclictest0-21swapper/012:03:440
2350799561,21cyclictest0-21swapper/012:03:440
2350799561,12cyclictest25364-21aten_rbpower_cu07:15:430
230472563,46sleep10-21swapper/107:08:101
88412550,2sleep20-21swapper/210:40:082
2351099554,8cyclictest8050irq/56-dwc_otg_11:27:123
2351099553,9cyclictest8050irq/56-dwc_otg_12:11:183
2351099552,7cyclictest32551-21ssh11:50:303
2351099552,11cyclictest0-21swapper/312:16:303
2351099551,13cyclictest0-21swapper/311:24:343
2350899552,38cyclictest0-21swapper/108:16:261
23507995523,4cyclictest8050irq/56-dwc_otg_11:05:560
23507995522,9cyclictest0-21swapper/008:25:420
2351099543,8cyclictest0-21swapper/311:10:593
2351099543,8cyclictest0-21swapper/311:10:593
2351099542,10cyclictest0-21swapper/312:28:093
2351099541,5cyclictest21496-21ls11:35:513
2351099541,4cyclictest8050irq/56-dwc_otg_10:13:163
2351099541,11cyclictest7950irq/56-dwc_otg_08:45:343
2351099541,11cyclictest0-21swapper/311:42:513
2350999543,18cyclictest7850irq/56-dwc_otg10:20:432
2350799544,27cyclictest30762-21latency_hist07:30:290
2350799542,39cyclictest0-21swapper/011:40:490
2351099536,4cyclictest8050irq/56-dwc_otg_12:34:553
2351099534,5cyclictest32020-21sh11:09:023
2351099532,12cyclictest7950irq/56-dwc_otg_09:06:503
2351099531,6cyclictest0-21swapper/308:55:453
23509995312,22cyclictest0-21swapper/210:45:582
2350899532,5cyclictest0-21swapper/107:15:441
2350899531,41cyclictest7950irq/56-dwc_otg_08:27:181
2350899531,41cyclictest122542sleep108:06:031
2350799537,34cyclictest0-21swapper/010:00:430
2350799532,31cyclictest14719-21aten_rbpower_en08:15:420
2350799531,7cyclictest3597-21latency_hist07:45:270
2350799531,7cyclictest3597-21latency_hist07:45:270
2350999528,27cyclictest0-21swapper/209:05:432
2350999528,27cyclictest0-21swapper/207:20:432
2350999528,27cyclictest0-21swapper/207:20:432
2350899522,39cyclictest0-21swapper/108:40:441
2350899521,37cyclictest32715-21grep09:05:291
2350799521,17cyclictest0-21swapper/011:31:130
2350799521,17cyclictest0-21swapper/011:31:130
23510995114,5cyclictest9108-21cat08:00:283
23509995110,11cyclictest0-21swapper/207:10:302
2350899511,40cyclictest0-21swapper/108:11:331
2350899511,33cyclictest0-21swapper/108:35:111
2350799514,25cyclictest0-21swapper/007:55:530
2350799514,17cyclictest0-21swapper/010:30:430
2351099501,4cyclictest0-21swapper/307:20:283
2351099501,10cyclictest8050irq/56-dwc_otg_08:10:403
2350899501,5cyclictest21978-21aten_rbpower_po08:35:401
2350899501,38cyclictest0-21swapper/107:45:131
2350899501,38cyclictest0-21swapper/107:45:131
2350799504,25cyclictest13-21rcuc/008:50:540
2350799504,25cyclictest13-21rcuc/008:50:540
2350799501,9cyclictest3554-21ssh12:35:390
2351099495,8cyclictest7950irq/56-dwc_otg_07:32:193
2351099492,8cyclictest0-21swapper/307:37:283
2351099492,4cyclictest8050irq/56-dwc_otg_07:50:483
2350899492,39cyclictest8050irq/56-dwc_otg_07:32:171
2350899491,45cyclictest29335-21tune2fs07:25:471
2350799498,20cyclictest1955-21diskmemload09:24:410
2350799493,6cyclictest0-21swapper/007:45:470
23507994915,7cyclictest0-21swapper/007:50:420
2351099481,4cyclictest0-21swapper/307:25:353
2351099481,11cyclictest0-21swapper/308:19:283
2350999487,29cyclictest750-21snmpd09:15:412
2350999486,27cyclictest26530-21ssh09:40:352
2350999478,26cyclictest0-21swapper/208:55:342
2350999473,11cyclictest0-21swapper/208:10:402
2350799472,5cyclictest20837-21grep08:30:530
2351099461,6cyclictest0-21swapper/308:37:343
2350799467,21cyclictest0-21swapper/007:35:410
2350999458,26cyclictest0-21swapper/211:35:422
2350999455,7cyclictest7950irq/56-dwc_otg_11:25:412
23509994420,19cyclictest8050irq/56-dwc_otg_12:30:412
23509994418,5cyclictest8050irq/56-dwc_otg_12:25:402
23509994415,5cyclictest12474-21fschecks_time12:05:462
2351099431,9cyclictest0-21swapper/307:24:573
2351099431,9cyclictest0-21swapper/307:24:573
2350999431,34cyclictest0-21swapper/210:57:482
2350999431,34cyclictest0-21swapper/210:57:482
23507994214,16cyclictest0-21swapper/008:44:010
229292422,33sleep00-21swapper/007:06:350
23510994114,4cyclictest18272-21/usr/sbin/munin08:25:443
2351099411,27cyclictest16740-21cut08:20:453
2351099411,27cyclictest16740-21cut08:20:453
2350999417,22cyclictest0-21swapper/207:25:432
2350999417,21cyclictest0-21swapper/209:35:422
2350999416,11cyclictest8050irq/56-dwc_otg_10:30:422
23509994130,8cyclictest8050irq/56-dwc_otg_12:15:432
23509994124,6cyclictest27763-21rm12:25:282
2350999407,4cyclictest7950irq/56-dwc_otg_10:00:422
2350999406,24cyclictest0-21swapper/207:55:442
23509994025,7cyclictest1034350ssh11:21:382
2350999402,26cyclictest0-21swapper/211:30:452
2350999402,26cyclictest0-21swapper/211:30:452
23509994016,15cyclictest6759-21sh12:39:402
2350999401,27cyclictest18467-21ls10:10:472
2351099391,27cyclictest12049-21ls08:05:563
23509993916,11cyclictest4961-21fschecks_time11:15:472
2350999391,26cyclictest0-21swapper/211:00:432
2350999391,26cyclictest0-21swapper/208:45:442
49532380,4sleep20-21swapper/211:55:512
2350999387,21cyclictest0-21swapper/207:50:432
23509993818,10cyclictest25988-21/usr/sbin/munin11:40:592
23509993817,8cyclictest3777-21latency_hist07:45:282
23509993817,8cyclictest3777-21latency_hist07:45:282
23509993813,16cyclictest301ktimersoftd/211:50:442
154512381,3sleep20-21swapper/210:06:052
2351099374,8cyclictest0-21swapper/308:30:403
2350999377,17cyclictest0-21swapper/207:15:432
2350999375,6cyclictest31-21ksoftirqd/210:16:202
23509993713,14cyclictest1955-21diskmemload10:26:392
2350999368,18cyclictest0-21swapper/211:10:202
2350999367,17cyclictest0-21swapper/209:30:422
23509993522,5cyclictest0-21swapper/208:05:392
23509993516,11cyclictest14764-21ssh09:25:462
23509993516,11cyclictest14764-21ssh09:25:462
23509993514,5cyclictest18271-21switchtime10:50:572
2350999351,27cyclictest15185-21ls08:15:492
219072357,13sleep30-21swapper/307:05:453
2350999346,7cyclictest7950irq/56-dwc_otg_08:25:422
2350999345,16cyclictest31-21ksoftirqd/212:00:442
2350999345,16cyclictest31-21ksoftirqd/212:00:442
2350999342,7cyclictest8050irq/56-dwc_otg_07:35:432
23509993424,7cyclictest9783-21sh10:00:282
23509993413,5cyclictest31943-21unixbench_multi07:30:582
23509993412,10cyclictest29515-21ntpq11:45:532
2351099331,26cyclictest23967-21cut08:40:453
2350999339,6cyclictest8050irq/56-dwc_otg_09:05:282
2350999331,8cyclictest0-21swapper/208:30:422
23509993316,10cyclictest23482-21latency_hist08:40:282
2350999331,15cyclictest8050irq/56-dwc_otg_08:50:412
2350999331,15cyclictest8050irq/56-dwc_otg_08:50:412
23509993213,4cyclictest387250wget07:45:422
2351099291,23cyclictest2314-21cut07:40:433
2351099291,23cyclictest2314-21cut07:40:433
2350999291,8cyclictest0-21swapper/208:42:252
2350999281,10cyclictest8050irq/56-dwc_otg_08:00:412
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional