You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-26 - 19:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sun Jul 26, 2026 12:43:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
22749915787,11cyclictest0-21swapper/012:25:300
22749915787,11cyclictest0-21swapper/012:25:300
22749912967,31cyclictest9533-21grep12:20:480
269521250,10sleep2227699cyclictest10:15:422
2275991225,66cyclictest8700-21/usr/sbin/munin11:40:511
22749912111,32cyclictest13668-21ssh09:10:430
22749911842,52cyclictest19999-21fschecks_count11:15:440
2274991173,83cyclictest31355-21runrttasks10:10:450
22749911245,35cyclictest0-21swapper/009:45:550
2274991123,74cyclictest13357-21sed09:10:250
2274991095,82cyclictest0-21swapper/010:00:590
2275991064,13cyclictest19679-21/usr/sbin/munin11:15:461
2275991052,84cyclictest8050irq/56-dwc_otg_10:43:421
2275991037,70cyclictest8050irq/56-dwc_otg_10:30:501
2275991034,9cyclictest8050irq/56-dwc_otg_12:15:541
2275991034,9cyclictest8050irq/56-dwc_otg_12:15:541
22749910234,51cyclictest0-21swapper/010:25:460
22749910115,63cyclictest0-21swapper/012:05:500
2275991006,66cyclictest7850irq/56-dwc_otg10:00:551
227599987,69cyclictest11399-21ssh09:45:551
2274999815,38cyclictest27480-21processes10:05:510
227599974,71cyclictest27418-21ssh10:05:491
227499976,68cyclictest0-21swapper/012:15:310
227499976,68cyclictest0-21swapper/012:15:310
2274999736,40cyclictest13264-21diskmemload09:30:560
227499964,67cyclictest32003-21missed_timers10:50:460
227599952,9cyclictest8050irq/56-dwc_otg_09:10:411
2274999517,52cyclictest0-21swapper/010:15:400
227599943,77cyclictest8756-21/usr/sbin/munin12:20:461
2274999361,17cyclictest7850irq/56-dwc_otg12:00:530
227499933,65cyclictest0-21swapper/009:20:450
227499933,65cyclictest0-21swapper/009:20:450
2274999322,41cyclictest13-21rcuc/011:20:510
227499923,19cyclictest8050irq/56-dwc_otg_10:30:520
227599914,70cyclictest8050irq/56-dwc_otg_11:10:551
227599914,70cyclictest8050irq/56-dwc_otg_11:10:551
2275999112,70cyclictest7727-21timerwakeupswit10:20:531
2274999038,27cyclictest0-21swapper/009:25:440
227499902,65cyclictest0-21swapper/011:40:380
271962890,1sleep10-21swapper/111:24:421
227599893,8cyclictest0-21swapper/110:25:471
37212880,3sleep13697-21unixbench_singl08:40:521
37212880,3sleep13697-21unixbench_singl08:40:521
2274998854,16cyclictest0-21swapper/010:20:510
227499882,65cyclictest0-21swapper/010:43:420
227599873,75cyclictest25874-21/usr/sbin/munin12:00:551
227699863,13cyclictest0-21swapper/209:10:432
227599863,76cyclictest13386-21grep09:10:251
2274998647,20cyclictest0-21swapper/012:30:460
227699858,50cyclictest301ktimersoftd/212:15:532
227699858,50cyclictest301ktimersoftd/212:15:532
2274998543,19cyclictest13-21rcuc/011:00:490
227599843,9cyclictest8050irq/56-dwc_otg_10:10:441
227599843,11cyclictest12525-21apt-get12:25:291
227599843,11cyclictest12525-21apt-get12:25:291
227699832,51cyclictest24922-21ls11:20:532
227599835,71cyclictest2674-21/usr/sbin/munin10:15:401
227499833,51cyclictest0-21swapper/011:10:550
227499833,51cyclictest0-21swapper/011:10:550
227599823,57cyclictest15227-1kworker/u9:209:20:421
227599823,57cyclictest15227-1kworker/u9:209:20:421
227599822,73cyclictest7935-21ntp_kernel_err11:00:481
2277998120,9cyclictest39-21ksoftirqd/312:20:463
227599813,71cyclictest0-21swapper/109:25:471
227599813,10cyclictest29483-21ntp_states12:05:491
227799804,54cyclictest16696-21ls11:10:553
227799804,54cyclictest16696-21ls11:10:553
148072800,2sleep30-21swapper/311:09:513
227599798,53cyclictest7950irq/56-dwc_otg_12:30:441
227599793,12cyclictest32142-21ntp_states10:50:491
2277997714,32cyclictest37-21rcuc/309:45:523
2276997725,27cyclictest301ktimersoftd/212:20:482
227699768,48cyclictest8050irq/56-dwc_otg_12:30:472
227599767,62cyclictest0-21swapper/109:30:561
227699743,51cyclictest13389-21ls09:10:252
2277997340,10cyclictest7850irq/56-dwc_otg09:10:253
2275997319,34cyclictest14115-21timerwakeupswit07:40:521
227499730,48cyclictest0-21swapper/010:40:240
227699729,33cyclictest11330-21sed09:45:532
227499722,30cyclictest0-21swapper/007:20:360
227799713,21cyclictest0-21swapper/309:10:433
2274997127,41cyclictest101ktimersoftd/008:20:340
173462710,1sleep20-21swapper/209:54:122
173462710,1sleep20-21swapper/209:54:122
6752700,3sleep10-21swapper/107:05:421
2277997024,32cyclictest381ktimersoftd/311:15:473
227699703,50cyclictest0-21swapper/212:05:492
227499701,28cyclictest0-21swapper/009:50:350
227499701,28cyclictest0-21swapper/009:50:350
2277996921,21cyclictest8050irq/56-dwc_otg_12:30:453
2276996925,28cyclictest0-21swapper/209:25:472
227499692,29cyclictest0-21swapper/007:40:360
227799683,44cyclictest0-21swapper/312:00:543
2277996814,17cyclictest39-21ksoftirqd/310:10:453
227699684,12cyclictest8050irq/56-dwc_otg_10:50:452
227699683,45cyclictest0-21swapper/211:15:432
227699676,40cyclictest7850irq/56-dwc_otg10:10:452
2276996713,32cyclictest8050irq/56-dwc_otg_10:30:512
2276996710,32cyclictest8050irq/56-dwc_otg_12:00:552
227499671,53cyclictest11120-21cat07:35:240
227499671,53cyclictest11120-21cat07:35:240
227499670,38cyclictest750-21snmpd08:12:580
288712660,4sleep123-21ksoftirqd/111:25:511
2277996621,15cyclictest0-21swapper/311:20:533
227499661,47cyclictest25873-21aten_rbpower_vo08:15:380
227499661,47cyclictest25873-21aten_rbpower_vo08:15:380
2277996521,14cyclictest381ktimersoftd/312:15:553
2277996521,14cyclictest381ktimersoftd/312:15:553
2277996511,37cyclictest8582-21ssh11:40:493
2276996519,19cyclictest7950irq/56-dwc_otg_11:00:472
2276996513,35cyclictest0-21swapper/210:43:422
227599651,3cyclictest0-21swapper/109:35:331
227499653,34cyclictest0-21swapper/007:15:360
2274996531,24cyclictest8050irq/56-dwc_otg_08:40:240
227499650,40cyclictest17134-21sensors11:50:520
227799644,33cyclictest0-21swapper/310:30:513
227699648,23cyclictest27410-21ntp_states10:05:482
227699643,38cyclictest7950irq/56-dwc_otg_11:40:512
2274996427,15cyclictest0-21swapper/007:25:530
227499640,48cyclictest0-21swapper/008:45:470
149782640,1sleep30-21swapper/309:50:473
149782640,1sleep30-21swapper/309:50:473
227799633,43cyclictest0-21swapper/310:50:493
2277996312,24cyclictest8050irq/56-dwc_otg_12:05:503
227699633,40cyclictest0-21swapper/209:30:572
227499631,44cyclictest18554-21ls09:55:390
227499630,60cyclictest5741-21latency_hist09:40:240
227499630,50cyclictest28451-21/usr/sbin/munin10:45:510
227499630,50cyclictest28451-21/usr/sbin/munin10:45:510
2274996227,18cyclictest4221-21http12:15:240
2274996215,20cyclictest0-21swapper/011:35:360
227499621,49cyclictest0-21swapper/008:51:140
227499620,58cyclictest18910-21unixbench-2d09:15:530
227499611,50cyclictest0-21swapper/010:56:400
227499610,49cyclictest0-21swapper/011:45:480
2277996036,10cyclictest37-21rcuc/312:25:373
2277996036,10cyclictest37-21rcuc/312:25:373
2277996016,22cyclictest0-21swapper/310:00:563
227599607,7cyclictest7950irq/56-dwc_otg_11:10:251
227499600,43cyclictest12008-21/usr/sbin/munin07:35:460
227799599,32cyclictest0-21swapper/309:20:443
227799599,32cyclictest0-21swapper/309:20:443
227799598,25cyclictest7950irq/56-dwc_otg_11:00:473
2276995916,24cyclictest8050irq/56-dwc_otg_12:25:302
2276995916,24cyclictest8050irq/56-dwc_otg_12:25:302
227499591,49cyclictest6370-21ssh09:40:430
227499590,46cyclictest0-21swapper/011:10:130
227799583,23cyclictest0-21swapper/310:15:423
227599588,3cyclictest0-21swapper/110:38:441
227599585,5cyclictest12045-21ssh11:45:351
227599584,4cyclictest7850irq/56-dwc_otg07:25:241
227599583,5cyclictest0-21swapper/107:35:351
227599581,13cyclictest6163-21ssh09:40:401
227499581,32cyclictest10888-21grep09:00:530
227499581,31cyclictest0-21swapper/011:55:490
227499580,41cyclictest24853-21ssh12:40:060
227799573,31cyclictest27478-21grep10:05:503
2276995730,16cyclictest29-21rcuc/207:50:352
2276995724,18cyclictest7950irq/56-dwc_otg_10:00:542
2276995712,40cyclictest16684-21ssh11:10:552
2276995712,40cyclictest16684-21ssh11:10:552
227599575,11cyclictest7850irq/56-dwc_otg08:21:391
2275995710,4cyclictest7850irq/56-dwc_otg09:05:241
227499572,30cyclictest32047-21proc_pri08:30:500
2274995714,26cyclictest2966-21/usr/sbin/munin08:40:420
2274995714,26cyclictest2966-21/usr/sbin/munin08:40:420
227799563,33cyclictest7713-21ssh10:20:533
227599563,11cyclictest0-21swapper/108:35:381
227599562,9cyclictest6923-21ssh11:00:241
227599562,14cyclictest0-21swapper/108:10:361
227599562,13cyclictest7850irq/56-dwc_otg12:40:151
227599561,5cyclictest7850irq/56-dwc_otg08:05:241
227599551,9cyclictest7850irq/56-dwc_otg11:40:111
227599551,4cyclictest30261-21ssh10:49:021
227599551,4cyclictest30261-21ssh10:49:021
2274995529,14cyclictest2827-21ls07:10:430
227499551,46cyclictest0-21swapper/008:29:240
227699548,21cyclictest7950irq/56-dwc_otg_09:20:452
227699548,21cyclictest7950irq/56-dwc_otg_09:20:452
227599544,10cyclictest0-21swapper/111:35:231
227599542,42cyclictest23731-21modprobe08:10:241
227599541,6cyclictest26516-21wc08:15:481
227599541,6cyclictest26516-21wc08:15:481
227599541,6cyclictest17770-21ssh11:51:211
227599541,4cyclictest0-21swapper/112:00:241
2275995411,18cyclictest7950irq/56-dwc_otg_07:50:381
227499541,42cyclictest0-21swapper/011:29:110
2274995412,20cyclictest563-21switchtime11:30:500
227499540,42cyclictest0-21swapper/008:59:270
227599532,5cyclictest20771-21sh09:19:011
227599532,4cyclictest0-21swapper/112:13:051
227599531,5cyclictest6082-21latency_hist08:50:241
2275995311,4cyclictest7567-21taskset08:53:191
15022534,33sleep00-21swapper/007:05:540
2277995229,15cyclictest381ktimersoftd/309:30:563
2277995218,12cyclictest7950irq/56-dwc_otg_10:25:453
227599527,4cyclictest7950irq/56-dwc_otg_07:30:381
227599527,4cyclictest7950irq/56-dwc_otg_07:30:381
227599522,9cyclictest31347-21wget08:30:401
227599521,8cyclictest7850irq/56-dwc_otg09:51:221
227599521,8cyclictest7850irq/56-dwc_otg09:51:221
227599521,3cyclictest7850irq/56-dwc_otg09:57:151
227599521,14cyclictest0-21swapper/108:29:321
227499521,37cyclictest0-21swapper/008:00:500
2277995022,8cyclictest7950irq/56-dwc_otg_10:43:423
2276995012,18cyclictest0-21swapper/210:25:452
227499500,27cyclictest22300-21cpu08:05:400
227799491,7cyclictest0-21swapper/308:15:353
227799491,7cyclictest0-21swapper/308:15:353
227699498,19cyclictest7606-21ssh10:20:512
227599491,5cyclictest3405-21unixbench_multi07:10:511
227499490,41cyclictest0-21swapper/007:46:070
2277994811,8cyclictest7850irq/56-dwc_otg09:25:453
227599481,9cyclictest18638-21wget07:55:401
227599481,11cyclictest0-21swapper/107:15:521
227499481,39cyclictest0-21swapper/007:51:260
227499480,35cyclictest0-21swapper/007:57:060
2275994712,5cyclictest7718-21tr07:25:351
38882460,6sleep2227699cyclictest12:14:522
227599447,24cyclictest0-21swapper/108:55:411
227599441,5cyclictest16190-21taskset07:46:341
227699422,5cyclictest7850irq/56-dwc_otg11:25:512
28712390,1sleep30-21swapper/311:34:233
2276993818,4cyclictest727-21runrttasks08:45:392
2276993815,5cyclictest16481-21fschecks_count11:50:432
227699381,4cyclictest4043-21ssh11:35:422
2276993810,6cyclictest125-21systemd-journal07:27:492
229732371,4sleep223068-21ls08:05:522
227799377,21cyclictest0-21swapper/309:00:383
227799374,4cyclictest0-21swapper/312:10:533
227699371,4cyclictest28529-21ls08:20:502
2277993617,5cyclictest22963-21sh11:58:183
227699367,16cyclictest20344-21/usr/sbin/munin08:00:462
227699364,11cyclictest0-21swapper/212:35:462
227799357,6cyclictest17320-21timerwakeupswit11:50:543
227799354,5cyclictest7850irq/56-dwc_otg07:35:513
2277993514,4cyclictest31043-21idleruntime-cro10:50:243
2277993514,4cyclictest31043-21idleruntime-cro10:50:243
227699353,23cyclictest0-21swapper/208:10:382
227699353,20cyclictest20337-21ssh11:55:362
227799349,3cyclictest0-21swapper/310:56:113
227799347,4cyclictest7950irq/56-dwc_otg_10:38:403
227799345,14cyclictest8050irq/56-dwc_otg_08:30:243
227799342,17cyclictest0-21swapper/309:38:543
2277993410,9cyclictest7850irq/56-dwc_otg07:25:533
227699346,17cyclictest7950irq/56-dwc_otg_08:30:542
2276993414,5cyclictest22391-21latency_hist10:00:242
2276993411,4cyclictest0-21swapper/209:35:452
227799339,3cyclictest0-21swapper/311:38:423
227799337,12cyclictest0-21swapper/311:47:043
227799336,13cyclictest8050irq/56-dwc_otg_07:10:493
227799335,15cyclictest39-21ksoftirqd/307:50:353
227799334,16cyclictest8050irq/56-dwc_otg_08:50:373
227799331,4cyclictest5940-21apt-get09:40:373
2277993314,5cyclictest39-21ksoftirqd/309:15:413
227699339,9cyclictest2962-21ntp_kernel_err07:10:452
227699335,15cyclictest0-21swapper/210:59:372
2276993321,7cyclictest301ktimersoftd/211:30:432
2276993319,5cyclictest5880-21sh09:40:312
227699331,4cyclictest8050irq/56-dwc_otg_09:15:362
227699331,13cyclictest0-21swapper/210:35:542
227799329,4cyclictest0-21swapper/309:58:433
227799327,5cyclictest8050irq/56-dwc_otg_08:35:533
2277993217,5cyclictest9553-21latency_hist09:00:243
227799321,23cyclictest6167-21uniq07:20:413
227799321,15cyclictest7850irq/56-dwc_otg08:23:493
2277993210,9cyclictest8050irq/56-dwc_otg_08:45:483
227699327,5cyclictest12212-21grep07:35:502
227699324,19cyclictest20114-21latency_hist08:00:252
2276993223,5cyclictest301ktimersoftd/208:55:392
227699321,7cyclictest0-21swapper/208:40:352
227699321,7cyclictest0-21swapper/208:40:352
227699321,10cyclictest7950irq/56-dwc_otg_08:35:502
227799317,15cyclictest0-21swapper/308:30:393
227799317,15cyclictest0-21swapper/308:10:383
227699316,14cyclictest0-21swapper/209:00:382
227699314,4cyclictest7950irq/56-dwc_otg_11:45:412
227699312,16cyclictest8050irq/56-dwc_otg_08:25:462
227699311,3cyclictest7850irq/56-dwc_otg08:15:352
227699311,3cyclictest7850irq/56-dwc_otg08:15:352
2276993113,10cyclictest52992chrt07:15:562
227799309,11cyclictest0-21swapper/311:25:363
227799308,5cyclictest4571-21kernelversion07:15:433
227699302,16cyclictest0-21swapper/211:08:592
227699301,21cyclictest15956-21timerwakeupswit07:45:492
227699301,14cyclictest0-21swapper/207:34:332
227699301,14cyclictest0-21swapper/207:34:332
227699301,13cyclictest0-21swapper/210:45:362
227699301,13cyclictest0-21swapper/210:45:362
227799296,14cyclictest0-21swapper/308:40:373
227799296,14cyclictest0-21swapper/308:40:373
227799292,12cyclictest0-21swapper/312:35:263
227799285,6cyclictest39-21ksoftirqd/307:40:413
227799283,5cyclictest8050irq/56-dwc_otg_07:55:393
227799282,14cyclictest8050irq/56-dwc_otg_08:04:123
227799281,13cyclictest7950irq/56-dwc_otg_08:10:253
227699282,7cyclictest5961-21wget07:20:382
227699281,14cyclictest0-21swapper/208:50:352
18942289,12sleep20-21swapper/207:09:042
227799272,16cyclictest0-21swapper/307:49:323
227699277,5cyclictest7950irq/56-dwc_otg_07:40:372
227799265,4cyclictest7950irq/56-dwc_otg_07:30:363
227799265,4cyclictest7950irq/56-dwc_otg_07:30:363
19382263,14sleep30-21swapper/307:09:333
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional