You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-22 - 16:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sat Aug 22, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
931822250,3sleep20-21swapper/207:10:102
9427991177,90cyclictest7950irq/56-dwc_otg_09:40:331
94219911527,21cyclictest0-21swapper/010:03:080
9421991095,72cyclictest0-21swapper/009:45:390
94219910822,19cyclictest0-21swapper/009:10:240
9421991075,69cyclictest0-21swapper/009:35:240
94219910710,65cyclictest0-21swapper/010:05:270
9436991053,78cyclictest20159-21diskmemload10:05:382
9427991015,63cyclictest7850irq/56-dwc_otg_10:13:091
943999928,25cyclictest0-21swapper/310:03:093
942799908,59cyclictest7850irq/56-dwc_otg_09:45:341
942199893,62cyclictest0-21swapper/010:35:330
266912890,1sleep30-21swapper/309:16:503
266912890,1sleep30-21swapper/309:16:503
942799883,60cyclictest24660-21ssh09:15:151
942799883,60cyclictest24660-21ssh09:15:151
942199875,61cyclictest0-21swapper/009:15:140
942199875,61cyclictest0-21swapper/009:15:140
942799864,56cyclictest0-21swapper/110:05:261
942199868,21cyclictest2194-21taskset11:25:230
942199863,62cyclictest0-21swapper/010:13:100
232182860,3sleep121-21rcuc/110:31:041
182692860,5sleep1942799cyclictest11:05:261
942199852,60cyclictest27291-21kworker/0:109:40:330
942799838,53cyclictest7950irq/56-dwc_otg_10:35:331
138542820,2sleep3381ktimersoftd/309:40:373
942799812,72cyclictest9042-21ssh09:35:261
942799812,72cyclictest32072-21ssh10:03:101
942799803,56cyclictest0-21swapper/109:10:241
200762800,6sleep0942199cyclictest12:25:360
9436997833,22cyclictest7850irq/56-dwc_otg_09:40:352
35592760,3sleep229-21rcuc/209:28:452
140882760,4sleep1942799cyclictest08:50:351
942799692,4cyclictest13397-21/usr/sbin/munin10:20:271
942799692,49cyclictest7950irq/56-dwc_otg_11:34:161
9436996837,25cyclictest26442-21sed10:35:322
942799684,4cyclictest7750irq/56-dwc_otg10:30:091
9427996818,37cyclictest1180-21runrttasks11:29:591
942199682,53cyclictest0-21swapper/009:25:290
943999672,40cyclictest1191-21snmpd10:35:323
942199676,45cyclictest0-21swapper/008:25:240
18612670,6sleep3943999cyclictest12:04:223
18092670,4sleep31807-21ssh09:25:403
942799662,49cyclictest7750irq/56-dwc_otg10:15:211
9427996612,35cyclictest0-21swapper/111:10:351
943699653,43cyclictest0-21swapper/209:35:262
942799653,51cyclictest29780-21ssh11:19:401
942799652,5cyclictest7950irq/56-dwc_otg_12:24:561
942799652,5cyclictest7850irq/56-dwc_otg_07:50:331
942799651,4cyclictest16777-21apt-get09:00:201
169252650,2sleep00-21swapper/010:24:390
942799646,42cyclictest7750irq/56-dwc_otg10:40:331
942799642,4cyclictest0-21swapper/109:50:191
9421996420,22cyclictest14681-21idleruntime-cro07:25:090
943699631,30cyclictest0-21swapper/211:15:242
942799633,44cyclictest7950irq/56-dwc_otg_11:48:061
942799633,44cyclictest7950irq/56-dwc_otg_11:48:061
942799631,8cyclictest28139-21ntpq12:35:321
942799631,5cyclictest0-21swapper/112:30:241
942199631,46cyclictest13165-21latency_hist10:20:090
942799622,46cyclictest0-21swapper/110:57:281
942799622,46cyclictest0-21swapper/110:57:281
942199621,49cyclictest7051-21fschecks_count12:10:270
943699614,29cyclictest0-21swapper/209:45:342
9436996133,22cyclictest20756-21ssh09:10:262
943699613,18cyclictest7850irq/56-dwc_otg_11:30:242
942799616,41cyclictest0-21swapper/110:52:201
942799611,6cyclictest2241-21latency_hist08:20:101
942199618,42cyclictest0-21swapper/012:00:280
9436996032,23cyclictest7874-21sh10:13:122
9436996011,35cyclictest0-21swapper/210:03:092
942799602,4cyclictest0-21swapper/109:25:091
942799601,5cyclictest10569-21grep07:10:351
942799601,4cyclictest28811-21ssh11:56:491
942199601,47cyclictest10639-21unixbench_singl07:10:360
942799592,42cyclictest0-21swapper/107:45:091
942799592,3cyclictest0-21swapper/112:25:311
942799591,4cyclictest7750irq/56-dwc_otg09:30:231
9421995917,6cyclictest0-21swapper/011:10:190
943699583,13cyclictest5697-21ssh10:50:242
942799586,37cyclictest7950irq/56-dwc_otg_08:25:391
942799584,4cyclictest12365-21unixbench_multi07:15:371
942799581,5cyclictest0-21swapper/110:48:211
942799581,3cyclictest7950irq/56-dwc_otg_12:05:051
943699575,4cyclictest0-21swapper/209:55:212
942799575,37cyclictest7950irq/56-dwc_otg_11:50:551
9427995719,35cyclictest20135-21cron09:10:091
9421995729,15cyclictest31312-21latency_hist12:40:100
943699567,25cyclictest0-21swapper/207:45:222
943699567,25cyclictest0-21swapper/207:45:222
9436995618,22cyclictest9897-21/usr/sbin/munin07:10:262
942799566,39cyclictest7950irq/56-dwc_otg_12:13:551
942799563,3cyclictest0-21swapper/109:26:211
942799561,6cyclictest2657-21ssh12:05:221
942799561,5cyclictest7850irq/56-dwc_otg_11:39:361
9421995614,24cyclictest6034-21fschecks_time10:50:280
942199561,40cyclictest14935-21ssh11:00:360
9421995613,36cyclictest20978-21ntp_kernel_err07:40:320
943999553,23cyclictest0-21swapper/309:45:383
9439995510,18cyclictest39-21ksoftirqd/309:35:263
9436995525,24cyclictest24649-21sh09:15:142
9436995525,24cyclictest24649-21sh09:15:142
9436995518,11cyclictest18157-21/usr/sbin/munin11:05:242
942799552,4cyclictest0-21swapper/111:44:311
942799551,4cyclictest12748-21ssh12:16:381
942799551,4cyclictest12748-21ssh12:16:381
9421995529,6cyclictest7850irq/56-dwc_otg_08:05:370
9421995522,4cyclictest18609-21/usr/sbin/munin11:45:200
9421995522,4cyclictest18609-21/usr/sbin/munin11:45:200
942199551,51cyclictest0-21swapper/010:45:250
942199551,30cyclictest19242-21unixbench_multi11:05:380
9421995510,37cyclictest0-21swapper/011:20:090
943999543,10cyclictest1191-21snmpd10:13:093
942799546,4cyclictest7750irq/56-dwc_otg11:22:301
942799543,48cyclictest7950irq/56-dwc_otg_09:55:211
942799541,5cyclictest7750irq/56-dwc_otg08:30:231
942799541,5cyclictest7750irq/56-dwc_otg08:30:231
942799541,4cyclictest7850irq/56-dwc_otg_07:39:541
942199547,41cyclictest9-21ksoftirqd/012:35:090
942199545,38cyclictest15151-21cpuspeed08:55:250
9421995427,13cyclictest7950irq/56-dwc_otg_08:20:360
942199542,48cyclictest21482-21grep10:30:100
942199542,44cyclictest0-21swapper/009:32:390
942199541,45cyclictest10099-21latency_hist11:35:090
942199541,42cyclictest0-21swapper/010:57:130
942199541,42cyclictest0-21swapper/010:57:130
942199541,31cyclictest30084-21grep09:20:380
942799532,4cyclictest0-21swapper/111:00:261
942199533,42cyclictest0-21swapper/011:36:050
9421995327,5cyclictest7950irq/56-dwc_otg_09:55:270
9421995319,17cyclictest7850irq/56-dwc_otg_07:30:300
942199531,35cyclictest0-21swapper/012:15:230
942199531,35cyclictest0-21swapper/012:15:230
943999523,9cyclictest7950irq/56-dwc_otg_10:05:243
942799524,36cyclictest7950irq/56-dwc_otg_07:45:321
942799524,36cyclictest7950irq/56-dwc_otg_07:45:321
942799522,46cyclictest0-21swapper/108:15:091
9421995233,12cyclictest0-21swapper/012:20:240
9421995215,20cyclictest17210-21if_wlan009:00:270
55012520,2sleep20-21swapper/211:29:222
9436995122,12cyclictest17821-21ls09:00:352
942799513,44cyclictest0-21swapper/107:30:261
9421995138,4cyclictest7950irq/56-dwc_otg_11:40:280
942199512,43cyclictest0-21swapper/009:51:520
942199512,42cyclictest0-21swapper/010:30:350
9421995118,17cyclictest8011-21fschecks_count08:35:280
942199511,31cyclictest26052-21fschecks_time07:55:260
942199511,29cyclictest12033-21proc_pri07:15:330
9421995112,23cyclictest7511-21idleruntime-cro08:35:090
9421995112,23cyclictest7511-21idleruntime-cro08:35:090
90772513,12sleep00-21swapper/007:10:090
943699508,23cyclictest7950irq/56-dwc_otg_10:23:362
9436995014,21cyclictest21895-21munin-run07:45:092
942799502,44cyclictest16035-21unixbench_singl07:25:361
942799501,6cyclictest27613-21aten_rbpower_po08:00:231
942799501,4cyclictest0-21swapper/108:06:331
942199504,34cyclictest0-21swapper/012:05:380
9421995026,5cyclictest23259-21ssh11:50:290
942199501,34cyclictest27758-21/usr/sbin/munin11:55:340
943699499,26cyclictest0-21swapper/212:10:232
942799491,5cyclictest0-21swapper/108:55:211
942799491,4cyclictest9644-21aten_rbpower_po08:40:231
9421994923,11cyclictest7950irq/56-dwc_otg_10:40:220
942199491,30cyclictest11458-21aten_rbpower_po08:45:220
9436994812,11cyclictest21650-21/usr/sbin/munin10:30:272
9436994810,23cyclictest21424-21ssh11:48:232
9436994810,23cyclictest21424-21ssh11:48:232
942799481,7cyclictest26752-21timerwakeupswit07:55:361
942199483,36cyclictest12741-21kworker/0:007:50:250
942199481,8cyclictest20348-21latency_hist09:10:100
943999473,10cyclictest20738-21df09:10:243
943699476,27cyclictest21167-21ssh09:50:222
9436994715,12cyclictest7950irq/56-dwc_otg_09:30:212
9436994714,18cyclictest17066-21ssh12:21:592
9436994714,12cyclictest32663-21basename09:25:082
9436994712,19cyclictest7850irq/56-dwc_otg_10:40:322
9436994710,22cyclictest29156-21sh12:36:282
942799472,5cyclictest0-21swapper/108:20:271
942799471,5cyclictest7869-21tr08:35:251
942799471,4cyclictest0-21swapper/107:20:211
9421994729,5cyclictest7850irq/56-dwc_otg_11:20:240
943999462,5cyclictest7950irq/56-dwc_otg_08:00:203
943699467,24cyclictest0-21swapper/207:30:232
9436994617,20cyclictest644-21sh12:02:082
9436994611,17cyclictest7850irq/56-dwc_otg_11:22:292
9421994623,5cyclictest7850irq/56-dwc_otg_07:50:090
9421994623,5cyclictest7850irq/56-dwc_otg_07:50:090
942199462,25cyclictest0-21swapper/008:40:300
9421994612,19cyclictest29076-21sshd08:05:030
943999452,11cyclictest7750irq/56-dwc_otg11:10:193
943699457,24cyclictest0-21swapper/207:15:222
943699457,23cyclictest0-21swapper/208:05:242
9436994515,20cyclictest24705-21ssh12:30:522
9436994510,21cyclictest2682-21/usr/sbin/munin12:05:232
9436994510,12cyclictest11871-21taskset10:17:552
942799452,5cyclictest0-21swapper/108:45:211
9421994518,14cyclictest0-21swapper/008:10:250
942199451,23cyclictest119150snmpd08:52:540
943999442,4cyclictest7950irq/56-dwc_otg_10:50:213
9436994411,12cyclictest25916-21rm11:54:042
942199442,33cyclictest0-21swapper/007:38:430
942199441,35cyclictest0-21swapper/007:25:320
943999432,6cyclictest7950irq/56-dwc_otg_09:05:223
9439994315,5cyclictest0-21swapper/311:35:343
943699436,22cyclictest7850irq/56-dwc_otg_11:11:292
9436994312,19cyclictest12460-21ssh11:36:592
9436994312,12cyclictest24227-21grep07:50:262
9436994312,11cyclictest7850irq/56-dwc_otg_08:25:372
9436994310,13cyclictest3156-21ssh10:46:092
252342430,1sleep30-21swapper/312:31:513
943699427,24cyclictest0-21swapper/211:00:222
943699426,32cyclictest1209050ssh10:57:252
943699426,32cyclictest1209050ssh10:57:252
9436994111,19cyclictest20268-21date07:40:102
9436994111,16cyclictest28660-21ssh11:56:332
942199411,30cyclictest0-21swapper/008:17:100
943699407,20cyclictest0-21swapper/208:00:232
9436994021,4cyclictest1180-21runrttasks07:25:312
9436994016,11cyclictest14888-21cat08:55:092
9436994013,17cyclictest21760-21seq12:28:022
9436994010,20cyclictest20807-21ssh10:29:122
156602401,2sleep30-21swapper/311:01:263
9436993918,14cyclictest7750irq/56-dwc_otg11:42:202
9436993915,13cyclictest16579-21sed09:00:092
943699391,25cyclictest0-21swapper/207:55:222
9436993911,14cyclictest7120-21/usr/sbin/munin08:30:382
9436993911,14cyclictest7120-21/usr/sbin/munin08:30:382
943699385,20cyclictest1867-21taskset08:17:322
9436993810,12cyclictest19452-21sed09:05:352
9436993810,10cyclictest7750irq/56-dwc_otg08:14:532
9439993714,13cyclictest2897950ssh10:00:023
9439993710,9cyclictest7950irq/56-dwc_otg_07:20:213
943699379,19cyclictest11057-21runrttasks08:44:242
943699377,17cyclictest12868-21chrt08:49:282
943699376,23cyclictest0-21swapper/212:16:402
943699376,23cyclictest0-21swapper/212:16:402
943999367,14cyclictest7950irq/56-dwc_otg_08:25:243
943999365,14cyclictest7850irq/56-dwc_otg_11:20:243
943699367,9cyclictest7950irq/56-dwc_otg_08:36:052
319202360,3sleep311-21rcu_preempt09:23:503
943999353,10cyclictest7950irq/56-dwc_otg_10:20:093
9439993525,6cyclictest381ktimersoftd/311:25:233
9439993516,11cyclictest24215-21fschecks_time07:50:263
943699356,10cyclictest7950irq/56-dwc_otg_07:24:022
9436993511,11cyclictest0-21swapper/208:20:132
943999348,14cyclictest7850irq/56-dwc_otg_10:55:333
943999348,14cyclictest7850irq/56-dwc_otg_10:55:333
943999348,14cyclictest2685050ssh11:55:213
943999347,24cyclictest0-21swapper/308:10:223
943999347,24cyclictest0-21swapper/307:30:233
943999345,20cyclictest2668-21ssh12:05:233
943999342,8cyclictest7850irq/56-dwc_otg_12:10:213
943999342,21cyclictest14366-21switchtime10:20:353
9439993416,10cyclictest37-21rcuc/308:50:333
9439993414,13cyclictest381ktimersoftd/312:16:393
9439993414,13cyclictest381ktimersoftd/312:16:393
943999341,17cyclictest21333-21latency_hist10:30:103
943999337,23cyclictest0-21swapper/308:30:233
943999337,23cyclictest0-21swapper/308:30:233
943999332,6cyclictest7950irq/56-dwc_otg_07:15:373
9439993310,4cyclictest0-21swapper/311:30:333
943999323,20cyclictest0-21swapper/311:43:303
9439993214,11cyclictest0-21swapper/311:16:063
943999321,21cyclictest7850irq/56-dwc_otg_10:45:473
9439993210,8cyclictest0-21swapper/311:05:313
943999318,4cyclictest0-21swapper/312:25:253
943999317,14cyclictest0-21swapper/308:45:213
943999317,14cyclictest0-21swapper/307:35:233
943999316,20cyclictest15899-21ls07:25:353
943999315,4cyclictest7950irq/56-dwc_otg_11:50:263
943999315,4cyclictest0-21swapper/310:40:233
943999313,16cyclictest7950irq/56-dwc_otg_12:20:323
9439993116,5cyclictest1549-21date08:15:363
9439993115,7cyclictest22380-21/usr/sbin/munin10:30:323
943999311,23cyclictest28348-21cat12:35:363
943999311,22cyclictest22180-21sh09:50:353
9439993111,4cyclictest0-21swapper/309:32:583
9439993016,4cyclictest11130-21latency_hist08:45:093
943999301,5cyclictest2569-21cut08:20:253
943999301,19cyclictest18895-21fschecks_count11:45:263
943999301,19cyclictest18895-21fschecks_count11:45:263
943999301,19cyclictest0-21swapper/309:00:223
943999291,25cyclictest23074-21timerandwakeup07:45:353
943999291,25cyclictest23074-21timerandwakeup07:45:353
943999284,6cyclictest7850irq/56-dwc_otg_07:40:223
943999282,8cyclictest8842-21grep08:35:383
9439992822,4cyclictest27436-21latency_hist08:00:093
943999281,6cyclictest7950irq/56-dwc_otg_07:15:083
943999271,8cyclictest29351-21/usr/sbin/munin08:05:243
88742274,15sleep30-21swapper/307:07:333
82362276,13sleep10-21swapper/107:05:321
943999261,4cyclictest9550irq/80-mmc009:00:093
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional