You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-02 - 00:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Wed Jul 01, 2026 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
978521870,5sleep01087299cyclictest12:07:470
108212920,3sleep10-21swapper/107:10:111
267822890,2sleep0101ktimersoftd/009:52:490
31022870,2sleep00-21swapper/010:40:560
10872998322,11cyclictest8050irq/56-dwc_otg_12:00:220
148212810,2sleep00-21swapper/010:55:330
10872998112,8cyclictest31128-21aten_rbpower_cu09:20:250
10872998028,5cyclictest8050irq/56-dwc_otg_07:35:390
10872998016,17cyclictest32429-21ls10:00:110
262462790,3sleep20-21swapper/209:51:542
1087299781,27cyclictest22409-21tr10:25:350
76012770,2sleep20-21swapper/208:30:262
1087299761,28cyclictest0-21swapper/007:55:290
10872997430,5cyclictest8050irq/56-dwc_otg_12:00:100
10872997420,2cyclictest0-21swapper/011:10:350
10872997420,2cyclictest0-21swapper/011:10:350
92302730,2sleep10-21swapper/110:49:341
1088699733,52cyclictest8050irq/56-dwc_otg_09:20:382
1088699701,56cyclictest16536-21ssh11:36:142
1087299702,18cyclictest0-21swapper/011:41:410
1087299701,23cyclictest20071-21cut12:20:290
1088699691,54cyclictest0-21swapper/212:05:582
1088699691,51cyclictest14877-21sh12:14:312
66202680,6sleep11088099cyclictest10:06:551
1088699661,39cyclictest17101-21latency_hist10:20:112
1087299661,38cyclictest0-21swapper/012:14:440
252612650,2sleep30-21swapper/309:13:183
152482650,2sleep20-21swapper/207:20:352
1087299652,24cyclictest727-21runrttasks11:24:430
10872996518,31cyclictest30628-21idleruntime-cro09:20:100
199592640,4sleep219949-21ssh11:40:362
1088699641,51cyclictest11184-21apt-get08:40:252
1088699641,45cyclictest14454-21sshd08:49:142
1088699641,42cyclictest1427-21ssh11:57:402
1088699631,43cyclictest29216-21ssh09:17:442
10891996224,18cyclictest21647-21/usr/sbin/munin10:25:233
10891996224,12cyclictest11471-21ls07:10:313
1088699621,52cyclictest9232-21ssh09:31:582
1088699621,46cyclictest11257-21ssh10:50:412
1088699621,44cyclictest32750-21apt-get10:00:222
1088699621,44cyclictest23127-21ssh10:25:582
1088699621,41cyclictest4406-21load08:20:322
1088699621,38cyclictest0-21swapper/212:35:112
1088699621,34cyclictest1535550ssh09:40:102
1088699614,43cyclictest8050irq/56-dwc_otg_09:55:462
1088699611,46cyclictest27257-21ssh12:29:242
1088699601,44cyclictest23182-21ssh09:48:592
1088699601,44cyclictest18586-21/usr/sbin/munin09:00:412
1088699601,38cyclictest4449-21sh11:22:032
10891995922,19cyclictest27315-21rm10:31:023
1088699597,42cyclictest21977-21diskmemload11:25:322
1088699592,40cyclictest8050irq/56-dwc_otg_07:35:352
1088699591,48cyclictest5410-21ssh10:45:092
1088699591,47cyclictest19732-21ssh12:20:222
1088699591,44cyclictest6167-21ssh10:45:292
1088699591,42cyclictest6080-21ssh10:05:582
1088699591,40cyclictest13562-21ssh11:33:242
1088699591,39cyclictest0-21swapper/211:50:022
1088699591,39cyclictest0-21swapper/211:50:022
10872995953,4cyclictest13-21rcuc/009:35:210
10891995821,13cyclictest8050irq/56-dwc_otg_08:15:183
1088699583,42cyclictest11578-21/usr/sbin/munin07:10:332
1088699581,49cyclictest29298-21ssh10:34:382
1088699581,48cyclictest1095-21ssh10:39:502
1088699581,48cyclictest1095-21ssh10:39:502
1088699581,40cyclictest3922-21ssh12:00:322
1088699581,39cyclictest26487-21apt-get11:10:232
1088699581,39cyclictest26487-21apt-get11:10:232
125712570,2sleep30-21swapper/310:14:503
10891995725,17cyclictest4368-21latency08:20:313
10891995723,16cyclictest29687-21/usr/sbin/munin08:00:373
10891995720,19cyclictest18321-21ssh09:42:563
1088699571,42cyclictest15958-21ssh10:56:422
1088699571,39cyclictest29306-21rm11:51:542
1088699571,37cyclictest2368-21ssh12:38:032
1088699571,34cyclictest4339-21ssh09:25:522
1088699571,34cyclictest4339-21ssh09:25:522
10872995718,21cyclictest2986-21aten_rbpower_po09:25:260
10872995718,21cyclictest2986-21aten_rbpower_po09:25:260
293152560,4sleep1221ktimersoftd/110:34:411
10891995613,25cyclictest8050irq/56-dwc_otg_12:20:243
1088699561,52cyclictest317-21cron11:17:102
1088699561,44cyclictest25405-21ssh09:13:332
1088699561,42cyclictest10797-21ssh10:11:452
1088699561,42cyclictest0-21swapper/208:00:392
1087299561,45cyclictest7552-21timerandwakeup11:25:370
10872995613,24cyclictest19261-21grep07:30:370
10891995523,17cyclictest3824-21ssh10:42:153
10891995523,12cyclictest19010-21/usr/sbin/munin11:40:363
1088699551,45cyclictest22066-21latency_hist11:05:112
1088699551,43cyclictest6887-21sshd08:26:142
1088699551,42cyclictest32260-21seq08:09:312
1087299551,30cyclictest20041-21aten_rbpower_po09:45:240
1087299551,28cyclictest24715-21grep07:45:390
1087299551,28cyclictest24715-21grep07:45:390
10891995421,23cyclictest20544-21missed_timers07:35:333
10891995421,19cyclictest8050irq/56-dwc_otg_07:55:253
1088699544,29cyclictest0-21swapper/210:20:312
1088699541,50cyclictest25551-21/usr/sbin/munin07:50:262
1088699541,50cyclictest22700-21/usr/sbin/munin07:40:382
1088699541,43cyclictest0-21swapper/207:58:102
1088699541,41cyclictest24748-21sh11:07:372
10891995328,14cyclictest6394-21ls10:45:323
10872995326,8cyclictest11164-21latency_hist09:35:110
1087299532,30cyclictest18568-21sort11:00:280
10891995221,19cyclictest1263-21sh11:18:503
10891995221,17cyclictest30672-21ssh11:54:223
10891995213,27cyclictest16447-21ssh11:36:053
1088699523,31cyclictest7950irq/56-dwc_otg_07:25:332
1088699521,39cyclictest16296-21ssh09:40:322
1088699521,38cyclictest8050irq/56-dwc_otg_08:15:182
10872995233,5cyclictest7850irq/56-dwc_otg10:15:320
10872995224,5cyclictest15294-21ls11:35:280
10872995215,19cyclictest11735-21/usr/sbin/munin08:40:320
1087299521,33cyclictest26539-21latency_hist09:15:120
10891995119,18cyclictest30622-21apt-get09:20:163
10891995113,19cyclictest8050irq/56-dwc_otg_12:00:163
1088699511,34cyclictest18108-21ssh12:17:492
1087299516,7cyclictest0-21swapper/010:00:230
1087299515,28cyclictest8681-21unixbench_multi08:30:400
10872995125,7cyclictest18919-21cut09:00:280
1087299512,22cyclictest0-21swapper/008:35:390
1087299512,22cyclictest0-21swapper/008:35:390
10872995122,11cyclictest8050irq/56-dwc_otg_09:10:110
10891995025,18cyclictest32750-21apt-get10:00:243
10891995023,18cyclictest10377-21ssh12:08:523
10891995022,12cyclictest17874-21ssh12:17:233
10891995021,12cyclictest15390-21ssh10:17:173
10891995020,11cyclictest26960-21if_eth011:10:293
10891995020,11cyclictest26960-21if_eth011:10:293
10891995013,19cyclictest8050irq/56-dwc_otg_11:02:273
1087299502,28cyclictest1319-21taskset10:40:110
1087299502,28cyclictest1319-21taskset10:40:110
1087299502,26cyclictest0-21swapper/010:10:310
10872995022,15cyclictest31099-21cron11:55:100
10891994922,17cyclictest29034-21ssh09:17:253
10891994919,7cyclictest19643-21ls09:00:383
10891994919,21cyclictest17352-21ssh10:59:153
10891994915,18cyclictest14176-21taskset07:18:513
1088699491,45cyclictest1513-21sshd08:13:142
1088699491,44cyclictest15432-21sed08:50:322
1088699491,32cyclictest727-21runrttasks07:18:482
1087299493,26cyclictest0-21swapper/010:06:030
10872994926,4cyclictest10779-21aten_rbpower_cu11:30:230
10872994924,6cyclictest13740-21grep07:15:380
10872994918,7cyclictest27092-21unixbench_singl10:30:400
1087299491,24cyclictest0-21swapper/010:50:300
1089199489,28cyclictest0-21swapper/309:48:433
10891994822,15cyclictest14524-21/usr/sbin/munin07:20:363
10891994822,10cyclictest10307-21apt-get11:30:143
10891994819,21cyclictest15546-21cut09:40:113
10891994819,19cyclictest25642-21ssh09:50:503
10891994819,15cyclictest25451-21ssh12:26:063
10891994819,12cyclictest31670-21ssh12:34:543
10891994819,11cyclictest3244-21ssh12:39:403
10891994818,11cyclictest14222-21seq08:46:223
10891994813,22cyclictest8050irq/56-dwc_otg_10:36:493
10891994813,22cyclictest8050irq/56-dwc_otg_10:36:493
10891994813,11cyclictest7950irq/56-dwc_otg_08:35:113
1088699481,44cyclictest23541-21munin-node07:45:222
1088699481,44cyclictest23541-21munin-node07:45:222
1088699481,36cyclictest0-21swapper/208:57:532
1087299484,23cyclictest23448-21ssh11:05:390
10872994822,4cyclictest22896-21unixbench_singl07:40:400
10872994816,17cyclictest23165-21sh11:45:220
10872994816,17cyclictest23165-21sh11:45:220
1087299481,44cyclictest14074-21tail08:45:390
10872994814,15cyclictest29334-21grep12:30:400
1087299481,34cyclictest2108-21latency_hist11:20:110
1087299481,34cyclictest0-21swapper/008:19:530
90562473,10sleep00-21swapper/007:05:170
10891994721,15cyclictest15091-21sh12:14:553
10891994721,13cyclictest13003-21ssh10:53:523
10891994719,15cyclictest23750-21ssh11:05:483
10891994717,14cyclictest5507-21ssh10:05:373
10891994715,12cyclictest20121-21ssh10:23:023
10891994711,22cyclictest0-21swapper/311:49:233
10891994711,22cyclictest0-21swapper/311:49:233
1088699471,43cyclictest9371-21apt-get08:35:232
1088699471,43cyclictest9371-21apt-get08:35:232
1088699471,43cyclictest20859-21irqstats09:05:302
1088699471,42cyclictest18147-21apt-get07:30:222
10872994721,14cyclictest18634-21sh12:18:470
1087299471,4cyclictest1688-21apt-get08:15:110
1087299471,31cyclictest0-21swapper/009:45:110
1087299471,27cyclictest0-21swapper/012:39:340
10891994621,13cyclictest17963-21cat07:30:113
10891994619,20cyclictest8802-21ssh11:27:273
10891994618,16cyclictest8050irq/56-dwc_otg_09:30:443
10891994615,11cyclictest4536-21ssh09:26:133
10891994615,11cyclictest4536-21ssh09:26:133
10891994614,18cyclictest8050irq/56-dwc_otg_11:25:013
1088099467,26cyclictest0-21swapper/108:10:251
1088099467,25cyclictest0-21swapper/110:55:241
1087299469,18cyclictest5809-21aten_rbpower_cu10:45:250
10891994515,14cyclictest30855-21ssh09:57:283
10880994533,8cyclictest221ktimersoftd/109:15:221
1088099452,13cyclictest21-21rcuc/111:10:251
1088099452,13cyclictest21-21rcuc/111:10:251
1087299452,33cyclictest0-21swapper/008:00:350
10872994520,5cyclictest0-21swapper/007:25:370
10872994518,18cyclictest5410-21sh08:25:110
101112450,2sleep110113-21rm09:33:291
10891994416,10cyclictest8050irq/56-dwc_otg_11:57:253
1088099448,22cyclictest0-21swapper/108:25:251
10872994421,5cyclictest7950irq/56-dwc_otg_10:20:340
10872994416,14cyclictest0-21swapper/007:20:350
10891994323,7cyclictest11184-21apt-get08:40:253
1088099437,24cyclictest0-21swapper/108:30:161
1088099437,24cyclictest0-21swapper/108:05:271
1088099437,23cyclictest0-21swapper/111:15:241
10880994318,6cyclictest221ktimersoftd/111:30:111
1087299434,29cyclictest25595-21uniq07:50:280
10872994321,3cyclictest0-21swapper/008:30:110
1087299431,30cyclictest31876-21unixbench_multi08:05:410
10872994312,16cyclictest24365-21if_eth012:25:300
1089199426,23cyclictest0-21swapper/308:25:253
10891994229,11cyclictest18695-21ls07:30:303
10891994228,11cyclictest21311-21ls09:05:373
1088099427,22cyclictest0-21swapper/108:55:161
10880994214,6cyclictest12785-21latency_hist10:15:111
10872994222,5cyclictest7850irq/56-dwc_otg08:50:130
1087299421,7cyclictest12082-21grep07:10:390
1089199416,23cyclictest0-21swapper/307:40:253
10891994124,14cyclictest24298-21proc_pri07:45:353
10891994124,14cyclictest24298-21proc_pri07:45:353
10891994116,12cyclictest0-21swapper/308:12:033
1088099417,24cyclictest0-21swapper/109:10:161
1088099417,21cyclictest0-21swapper/108:40:251
1088099416,22cyclictest10927-21runrttasks07:10:241
1088099416,19cyclictest0-21swapper/112:25:111
10891994014,11cyclictest10712-21taskset08:37:383
10891994014,11cyclictest10712-21taskset08:37:383
1088099409,4cyclictest0-21swapper/112:34:121
1088099407,24cyclictest0-21swapper/109:55:241
1088099407,24cyclictest0-21swapper/108:15:171
1088099406,24cyclictest0-21swapper/109:20:161
1088099405,20cyclictest221ktimersoftd/110:25:241
10880994016,4cyclictest221ktimersoftd/111:50:251
10880994013,14cyclictest221ktimersoftd/110:20:121
10880994010,11cyclictest7850irq/56-dwc_otg09:00:221
10872994010,13cyclictest17531-21/usr/sbin/munin08:55:400
10891993925,12cyclictest31499-21pluginstate08:05:373
10891993925,11cyclictest16844-21aten_rbpower_en08:55:273
10891993924,11cyclictest14863-21apt-get08:50:173
1088099397,20cyclictest0-21swapper/107:45:251
1088099397,20cyclictest0-21swapper/107:45:251
1088099395,24cyclictest0-21swapper/112:05:241
1088099395,17cyclictest0-21swapper/109:35:211
10880993916,10cyclictest221ktimersoftd/111:57:151
1088699381,35cyclictest10885-21cut07:10:112
1088099389,6cyclictest23-21ksoftirqd/111:35:241
1088099385,4cyclictest7850irq/56-dwc_otg12:10:291
1088099385,24cyclictest10673-21ssh11:30:161
1088099381,7cyclictest0-21swapper/107:15:241
8402370,2sleep10-21swapper/112:35:381
10891993721,13cyclictest25610-21fschecks_count07:50:283
1088099377,7cyclictest7950irq/56-dwc_otg_10:55:111
1088099377,19cyclictest0-21swapper/108:45:251
1088099376,9cyclictest0-21swapper/107:30:221
1088099375,20cyclictest23195-21ssh11:45:251
1088099375,20cyclictest23195-21ssh11:45:251
10880993718,5cyclictest23-21ksoftirqd/109:40:241
10880993714,6cyclictest7850irq/56-dwc_otg08:00:341
1088099371,26cyclictest16205-21ls07:25:111
1088099365,6cyclictest7950irq/56-dwc_otg_12:25:401
10880993616,10cyclictest31043-21ssh10:35:391
10880993616,10cyclictest31043-21ssh10:35:391
1088099361,21cyclictest221ktimersoftd/112:00:301
1088099358,5cyclictest7850irq/56-dwc_otg10:40:381
1088099352,4cyclictest0-21swapper/109:05:211
10880993519,5cyclictest727-21runrttasks07:55:291
10880993515,5cyclictest0-21swapper/111:20:271
1088099351,15cyclictest221ktimersoftd/108:20:291
1088099349,8cyclictest7850irq/56-dwc_otg07:35:291
1088099344,5cyclictest16776-21grep12:15:401
1088099336,7cyclictest7850irq/56-dwc_otg09:25:401
1088099336,7cyclictest7850irq/56-dwc_otg09:25:401
1088099334,12cyclictest23-21ksoftirqd/110:22:331
1088099331,17cyclictest7850irq/56-dwc_otg10:00:321
10880993311,4cyclictest0-21swapper/109:45:221
1088099326,5cyclictest8050irq/56-dwc_otg_11:05:401
10880993223,6cyclictest28292-21latency_hist09:55:121
1088099321,19cyclictest0-21swapper/107:50:251
1088099312,6cyclictest0-21swapper/111:40:371
10880993124,5cyclictest217582sleep107:40:231
1088099312,19cyclictest19264-21ls11:00:371
1088099306,14cyclictest0-21swapper/107:25:261
1088099302,11cyclictest21-21rcuc/108:35:281
1088099302,11cyclictest21-21rcuc/108:35:281
90542294,16sleep30-21swapper/307:05:163
10880992912,4cyclictest15308-21cat08:50:311
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional