You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-12 - 15:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sat Sep 12, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
415521650,7sleep2301ktimersoftd/207:05:402
54119914918,25cyclictest0-21swapper/009:20:380
5414991188,75cyclictest25261-21diskstats09:20:393
622221170,5sleep0541199cyclictest11:35:070
5411991093,26cyclictest7950irq/56-dwc_otg_09:25:440
1124821040,6sleep0541199cyclictest07:25:370
5411999612,57cyclictest0-21swapper/009:40:390
269442950,2sleep10-21swapper/112:39:091
541199937,56cyclictest16601-21ssh09:10:210
5411999243,28cyclictest1925-21timerandwakeup09:30:490
5411999243,28cyclictest1925-21timerandwakeup09:30:490
541499854,74cyclictest17479-21sshd09:10:433
5411998446,8cyclictest7750irq/56-dwc_otg09:10:440
541499832,61cyclictest7950irq/56-dwc_otg_09:10:203
541499814,50cyclictest29890-21ntp_states09:25:443
541499764,65cyclictest9594-21if_wlan009:40:403
541499753,64cyclictest1935-21timerandwakeup09:30:493
541499753,64cyclictest1935-21timerandwakeup09:30:493
541399746,10cyclictest7300-21ssh12:15:302
541499728,47cyclictest0-21swapper/311:57:103
541499723,10cyclictest0-21swapper/308:10:323
288092720,2sleep00-21swapper/010:04:170
5413997119,7cyclictest7950irq/56-dwc_otg_10:25:312
304622700,7sleep0541199cyclictest11:25:130
233852690,3sleep123386-21unixbench_multi10:35:501
541299685,14cyclictest1922-21timerandwakeup09:30:491
541299685,14cyclictest1922-21timerandwakeup09:30:491
5412996813,22cyclictest16553-21latency_hist09:10:201
5412996812,13cyclictest23-21ksoftirqd/109:25:461
541199673,8cyclictest20868-21ssh09:15:320
541199673,8cyclictest20868-21ssh09:15:320
160502670,3sleep216013-21ssh11:45:482
71722660,1sleep30-21swapper/311:35:383
541299663,31cyclictest0-21swapper/109:10:451
541499657,5cyclictest7950irq/56-dwc_otg_12:35:193
128212650,2sleep00-21swapper/012:20:570
541499643,16cyclictest7950irq/56-dwc_otg_11:15:333
5413996421,10cyclictest7750irq/56-dwc_otg09:25:452
5412996412,39cyclictest19498-21ls11:10:461
5412996412,39cyclictest19498-21ls11:10:461
541399634,12cyclictest0-21swapper/208:55:322
541399634,12cyclictest0-21swapper/208:55:322
5411996320,24cyclictest21659-21latency_hist07:55:200
541499629,5cyclictest7850irq/56-dwc_otg_10:40:433
541499629,5cyclictest7850irq/56-dwc_otg_10:40:433
5414996212,38cyclictest7950irq/56-dwc_otg_11:50:313
541399622,26cyclictest7750irq/56-dwc_otg09:10:192
541499617,4cyclictest7850irq/56-dwc_otg_08:20:193
541499611,5cyclictest1390-21timerwakeupswit08:25:483
541399615,9cyclictest0-21swapper/210:00:322
541399615,10cyclictest7750irq/56-dwc_otg11:00:342
541499601,6cyclictest0-21swapper/310:20:413
541499597,3cyclictest0-21swapper/310:35:353
541499591,5cyclictest19425-21/usr/sbin/munin11:10:453
541499591,5cyclictest19425-21/usr/sbin/munin11:10:453
541399594,10cyclictest7850irq/56-dwc_otg_09:40:382
541499583,41cyclictest7950irq/56-dwc_otg_10:15:473
541499581,7cyclictest8246-21grep12:15:453
541299584,24cyclictest25257-21ssh09:20:381
541299583,8cyclictest7750irq/56-dwc_otg08:55:321
541299583,8cyclictest7750irq/56-dwc_otg08:55:321
541299583,13cyclictest9486-21fschecks_time09:40:381
541199582,30cyclictest18569-21cut07:45:360
541499576,3cyclictest7950irq/56-dwc_otg_07:10:463
541499571,7cyclictest9307-21latency_hist08:50:203
5413995712,24cyclictest7850irq/56-dwc_otg_09:10:442
541299572,23cyclictest0-21swapper/111:15:331
5411995731,11cyclictest7950irq/56-dwc_otg_08:00:490
541199571,32cyclictest6562-21grep07:10:510
541499565,5cyclictest16118-21ssh11:45:493
541499564,4cyclictest7850irq/56-dwc_otg_12:05:383
541499563,42cyclictest0-21swapper/310:53:103
541499562,46cyclictest5005-21ssh09:35:343
541499561,3cyclictest0-21swapper/311:30:313
541399566,25cyclictest7850irq/56-dwc_otg_09:20:372
541399562,7cyclictest7950irq/56-dwc_otg_08:25:322
541199568,26cyclictest518-21in:imuxsock12:35:190
5411995611,24cyclictest0-21swapper/010:05:330
541499552,3cyclictest7750irq/56-dwc_otg10:55:363
541199552,34cyclictest27261-21/usr/sbin/munin12:00:470
5411995518,24cyclictest3319-21unixbench_multi08:30:490
541499544,4cyclictest7950irq/56-dwc_otg_12:25:403
541499543,8cyclictest7950irq/56-dwc_otg_10:46:193
541499543,4cyclictest7750irq/56-dwc_otg09:17:413
541499543,4cyclictest7750irq/56-dwc_otg09:17:413
541499541,4cyclictest0-21swapper/311:44:333
541499541,4cyclictest0-21swapper/311:44:333
541399548,14cyclictest0-21swapper/212:30:342
541199546,15cyclictest0-21swapper/010:55:340
541199545,6cyclictest0-21swapper/011:10:210
5411995436,3cyclictest0-21swapper/009:55:200
541199542,45cyclictest0-21swapper/011:25:330
541199541,28cyclictest16203-21systemd-journal12:30:200
5411995410,24cyclictest0-21swapper/011:05:200
541499533,8cyclictest7950irq/56-dwc_otg_08:20:443
541499533,10cyclictest7750irq/56-dwc_otg08:00:323
541499531,6cyclictest12357-21unixbench_multi08:55:483
541499531,6cyclictest12357-21unixbench_multi08:55:483
5414995314,36cyclictest12835-21ssh12:20:583
541499531,10cyclictest7850irq/56-dwc_otg_12:13:203
541399535,7cyclictest7850irq/56-dwc_otg_10:50:312
541199533,29cyclictest0-21swapper/007:20:340
541199532,32cyclictest11110-21grep11:40:390
541199532,32cyclictest11110-21grep11:40:390
541199532,29cyclictest27767-21latency_hist12:40:200
5411995316,6cyclictest14543-21latency_hist07:35:200
5411995315,19cyclictest23933-21/usr/sbin/munin11:55:420
541199531,44cyclictest0-21swapper/009:50:200
541199531,31cyclictest27478-21unixbench_multi10:40:480
541199531,31cyclictest27478-21unixbench_multi10:40:480
541499523,3cyclictest7950irq/56-dwc_otg_10:10:133
541499521,6cyclictest14277-21ssh10:25:383
541499521,5cyclictest31718-21/usr/sbin/munin11:25:463
541499521,5cyclictest27247-21ssh10:01:243
541499521,4cyclictest0-21swapper/311:07:583
541499521,4cyclictest0-21swapper/309:55:053
541499521,12cyclictest7950irq/56-dwc_otg_09:48:023
541299529,24cyclictest30880-21ssh11:25:331
5411995233,11cyclictest10185-21ls10:20:400
541199522,26cyclictest0-21swapper/008:10:350
5411995220,5cyclictest0-21swapper/008:15:310
5411995214,21cyclictest22987-21proc_pri10:35:460
541199521,31cyclictest5887-21latency_hist10:55:200
541499517,4cyclictest7850irq/56-dwc_otg_12:38:013
541499512,5cyclictest0-21swapper/310:32:243
5413995113,33cyclictest2003-21ls09:30:512
5413995113,33cyclictest2003-21ls09:30:512
5411995132,6cyclictest7750irq/56-dwc_otg10:50:190
541199512,19cyclictest0-21swapper/008:00:200
5411995117,20cyclictest18810-21fschecks_count11:10:370
5411995117,20cyclictest18810-21fschecks_count11:10:370
541199511,4cyclictest7750irq/56-dwc_otg10:30:330
541499508,38cyclictest0-21swapper/308:35:363
541499502,7cyclictest7950irq/56-dwc_otg_12:01:493
541499501,3cyclictest0-21swapper/311:00:333
541499501,3cyclictest0-21swapper/310:10:293
541499501,3cyclictest0-21swapper/309:56:013
541399506,9cyclictest0-21swapper/210:15:302
541299507,27cyclictest5633-21ssh10:15:301
5411995030,5cyclictest7950irq/56-dwc_otg_09:00:480
5411995024,14cyclictest32685-21latency_hist08:25:200
541199501,29cyclictest7006-21sort11:35:370
541199501,28cyclictest4468-21sed12:10:490
541499491,5cyclictest10010-21/usr/sbin/munin08:50:403
5414994911,35cyclictest234852chrt07:59:373
541299493,7cyclictest7850irq/56-dwc_otg_09:45:301
5411994929,4cyclictest7750irq/56-dwc_otg11:45:360
5411994927,11cyclictest0-21swapper/010:14:540
5411994924,5cyclictest25254-21cron10:00:200
541199492,28cyclictest6072-21fschecks_count08:40:370
5411994921,15cyclictest11146-21wc08:55:210
5411994918,18cyclictest11258-21latency_hist12:20:200
541499483,7cyclictest13359-21kernelversion07:30:413
541499482,7cyclictest19376-21unixbench_multi07:45:473
5414994812,4cyclictest27306-21latency11:20:403
5413994827,15cyclictest31-21ksoftirqd/211:55:302
5411994825,5cyclictest20004-21switchtime11:50:440
541199482,43cyclictest11261-21kworker/0:108:55:390
541199482,43cyclictest11261-21kworker/0:108:55:390
541199482,27cyclictest0-21swapper/008:05:410
541199482,27cyclictest0-21swapper/008:05:410
541499472,8cyclictest0-21swapper/309:00:483
541499472,7cyclictest9112-21aten_rbpower_en07:20:343
541499472,7cyclictest6788-21grep08:40:463
541499471,7cyclictest17599-21grep07:40:483
541499471,4cyclictest0-21swapper/307:50:443
541199472,6cyclictest4865-21kworker/0:210:15:350
541199472,30cyclictest0-21swapper/010:25:480
5411994721,11cyclictest0-21swapper/008:45:310
5411994715,2cyclictest0-21swapper/008:35:340
541499467,24cyclictest0-21swapper/307:25:253
541499461,8cyclictest8342-21grep07:15:483
5414994612,4cyclictest27380-21latency_hist08:10:213
5414994612,4cyclictest27380-21latency_hist08:10:213
541399462,8cyclictest11441-21ssh12:20:312
541199469,4cyclictest9550irq/80-mmc009:40:190
541199467,22cyclictest32416-21timerandwakeup12:05:470
5411994629,4cyclictest16808-21grep07:40:370
541499458,4cyclictest16012-21taskset07:37:393
541499458,4cyclictest16012-21taskset07:37:393
541499451,8cyclictest3376-21sed08:30:493
541399454,3cyclictest0-21swapper/210:10:302
541299457,25cyclictest0-21swapper/112:20:331
541299451,11cyclictest7750irq/56-dwc_otg07:35:341
541299451,11cyclictest7750irq/56-dwc_otg07:35:341
5411994517,21cyclictest0-21swapper/011:19:550
541399447,21cyclictest7950irq/56-dwc_otg_12:25:322
541299447,11cyclictest7950irq/56-dwc_otg_12:15:301
541299443,31cyclictest7950irq/56-dwc_otg_10:50:311
541299437,22cyclictest0-21swapper/112:05:331
5412994320,9cyclictest0-21swapper/107:40:481
541299431,15cyclictest0-21swapper/108:50:201
5411994324,11cyclictest0-21swapper/008:25:320
541199432,24cyclictest0-21swapper/007:18:300
541299421,26cyclictest0-21swapper/108:35:351
541399417,23cyclictest17904-21rm09:50:442
5412994112,6cyclictest0-21swapper/108:15:311
541299411,25cyclictest0-21swapper/108:25:341
5411994132,5cyclictest101ktimersoftd/007:35:330
5411994132,5cyclictest101ktimersoftd/007:35:330
241522410,2sleep10-21swapper/109:19:251
241522410,2sleep10-21swapper/109:19:251
541399409,27cyclictest0-21swapper/211:25:352
5412994017,4cyclictest6194-21aten_rbpower_cu10:55:341
5413993923,10cyclictest31-21ksoftirqd/207:25:322
5412993917,5cyclictest2637650wget10:40:351
5412993917,5cyclictest2637650wget10:40:351
541299391,25cyclictest19521-21ls12:30:211
541399386,5cyclictest7950irq/56-dwc_otg_08:20:372
541399381,4cyclictest0-21swapper/210:05:352
541399381,24cyclictest14852-21ls07:35:362
541399381,24cyclictest14852-21ls07:35:362
541299387,27cyclictest7950irq/56-dwc_otg_10:10:301
541299381,27cyclictest10368-21ssh10:20:421
541399372,17cyclictest0-21swapper/211:15:382
5413993720,13cyclictest0-21swapper/208:15:312
5413993716,13cyclictest11-21rcu_preempt10:30:352
541399371,25cyclictest0-21swapper/210:45:452
541399371,15cyclictest242632sh09:19:372
541399371,15cyclictest242632sh09:19:372
541299379,6cyclictest0-21swapper/110:25:311
541299371,5cyclictest7938-21ntp_states07:15:431
541299371,19cyclictest0-21swapper/111:45:331
541399361,21cyclictest9008-21ls07:20:202
541299366,15cyclictest9450irq/48-DMA08:56:171
5412993621,8cyclictest0-21swapper/111:55:301
541299362,18cyclictest0-21swapper/110:47:231
5412993612,4cyclictest0-21swapper/110:00:331
313822360,1sleep10-21swapper/110:06:151
541399353,17cyclictest0-21swapper/207:58:492
541399351,6cyclictest0-21swapper/209:55:332
5413993516,5cyclictest26322-21aten_rbpower_po10:40:342
5413993516,5cyclictest26322-21aten_rbpower_po10:40:342
541399351,17cyclictest25494-21latency_hist08:05:202
541299357,15cyclictest0-21swapper/109:50:441
541299351,21cyclictest11032-21ls08:55:201
541299351,15cyclictest0-21swapper/111:10:181
541299351,14cyclictest7850irq/56-dwc_otg_08:20:331
541399347,15cyclictest0-21swapper/209:00:342
541399342,6cyclictest2564-21ssh11:30:312
541399342,6cyclictest11-21rcu_preempt11:05:342
5413993411,6cyclictest22251-21aten_rbpower_vo10:35:362
541299347,14cyclictest0-21swapper/107:10:371
541299345,20cyclictest2550-21ssh11:30:311
5412993417,5cyclictest23385-21sh12:35:111
5412993412,8cyclictest216862sleep111:53:161
541299341,23cyclictest6849-21timerwakeupswit08:40:471
541299341,23cyclictest0-21swapper/107:50:331
541399337,15cyclictest0-21swapper/207:30:352
541399336,24cyclictest0-21swapper/212:35:342
541399333,9cyclictest7850irq/56-dwc_otg_08:45:322
5413993315,7cyclictest21703-21cat07:55:202
541299332,13cyclictest0-21swapper/109:55:471
5412993320,5cyclictest31002-21ssh12:05:151
5412993316,10cyclictest11540-21unixbench_singl11:00:501
541299331,15cyclictest25915-21fschecks_time08:05:371
541299331,15cyclictest25915-21fschecks_time08:05:371
5452320,5sleep2541399cyclictest12:06:212
541399327,23cyclictest0-21swapper/211:20:332
541399324,5cyclictest13207-21apt-get09:45:302
5413993225,5cyclictest301ktimersoftd/211:35:312
541399321,20cyclictest19954-21ls07:50:202
541399321,19cyclictest10702-21ls07:25:192
541299326,10cyclictest0-21swapper/107:20:321
541299322,5cyclictest10913-21aten_rbpower_po11:40:361
541299322,5cyclictest10913-21aten_rbpower_po11:40:361
5412993218,4cyclictest7578-21grep11:35:451
5412993215,4cyclictest1180-21runrttasks09:00:401
5412993213,12cyclictest0-21swapper/109:36:571
5412993212,6cyclictest3779-21cron08:35:191
5412993210,4cyclictest0-21swapper/111:20:471
541399319,15cyclictest0-21swapper/212:10:252
541399318,13cyclictest7950irq/56-dwc_otg_12:03:322
541399317,15cyclictest0-21swapper/207:10:372
541399316,5cyclictest7950irq/56-dwc_otg_09:35:382
541399316,5cyclictest7950irq/56-dwc_otg_08:50:372
541399315,7cyclictest7850irq/56-dwc_otg_08:10:472
541399311,7cyclictest10881-21aten_rbpower_en11:40:352
541399311,7cyclictest10881-21aten_rbpower_en11:40:352
5413993110,10cyclictest0-21swapper/210:56:472
5412993115,9cyclictest7950irq/56-dwc_otg_10:30:431
5412993113,6cyclictest13084-21sh07:30:371
541299311,12cyclictest0-21swapper/107:45:331
40552316,14sleep10-21swapper/107:05:401
541399301,6cyclictest6908-21unixbench-2d08:40:472
541399301,19cyclictest0-21swapper/210:20:402
5412993014,5cyclictest3978-21open_files12:10:451
541399297,5cyclictest7750irq/56-dwc_otg11:50:412
541399292,13cyclictest7950irq/56-dwc_otg_11:10:362
541399292,13cyclictest7950irq/56-dwc_otg_11:10:362
541399291,4cyclictest7950irq/56-dwc_otg_07:45:192
5412992912,5cyclictest25158-21seq08:02:521
541399281,12cyclictest7850irq/56-dwc_otg_08:30:462
541299287,14cyclictest27843-21if_wlan008:10:391
541299281,15cyclictest0-21swapper/107:55:251
541399274,5cyclictest26964-21runrttasks08:07:512
541399274,5cyclictest26964-21runrttasks08:07:512
541399261,12cyclictest7850irq/56-dwc_otg_08:35:412
48632264,14sleep00-21swapper/007:06:480
35822264,14sleep30-21swapper/307:05:303
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional