You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-05 - 16:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Sun Jul 05, 2026 12:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1961423100,2sleep10-21swapper/107:10:131
19833991539,108cyclictest727-21runrttasks10:45:310
19833991513,118cyclictest0-21swapper/012:35:290
19833991383,109cyclictest12322-21grep09:25:430
19833991324,105cyclictest29472-21ssh10:25:410
19833991295,100cyclictest0-21swapper/009:10:440
19833991277,69cyclictest0-21swapper/012:15:200
19833991222,7cyclictest0-21swapper/010:50:440
19833991215,96cyclictest0-21swapper/012:20:260
19833991183,92cyclictest0-21swapper/011:20:310
198339911834,63cyclictest14878-21/usr/sbin/munin11:25:420
19833991152,10cyclictest19899-21users12:10:440
19833991152,10cyclictest19899-21users12:10:440
2823621140,5sleep01983399cyclictest09:00:410
198339911426,68cyclictest0-21swapper/009:21:300
198339911281,8cyclictest7850irq/56-dwc_otg09:15:380
19833991105,86cyclictest0-21swapper/011:45:430
19833991102,11cyclictest11863-21sh10:05:260
19833991093,85cyclictest11580-21grep12:00:420
19833991083,81cyclictest0-21swapper/009:30:380
19841991052,78cyclictest13598-21taskset10:45:402
198419910511,66cyclictest26134-21ssh11:00:412
198419910511,66cyclictest26134-21ssh11:00:412
198339910412,72cyclictest0-21swapper/011:00:460
198339910412,72cyclictest0-21swapper/011:00:460
1151421010,3sleep10-21swapper/108:15:341
198419910014,61cyclictest7312-21ssh12:35:292
198419910010,66cyclictest7950irq/56-dwc_otg_10:50:362
1983399993,79cyclictest0-21swapper/009:35:420
19837999717,64cyclictest0-21swapper/111:45:281
185112960,5sleep20-21swapper/207:05:322
1984199953,66cyclictest8050irq/56-dwc_otg_11:25:402
19833999515,55cyclictest0-21swapper/011:55:360
100102940,6sleep31984299cyclictest12:00:133
1984199939,58cyclictest8050irq/56-dwc_otg_12:10:462
1984199939,58cyclictest8050irq/56-dwc_otg_12:10:462
1984199933,10cyclictest12253-21ssh09:25:422
19841999315,59cyclictest29527-21proc_pri10:25:432
1983799937,15cyclictest8050irq/56-dwc_otg_09:10:431
1983799913,11cyclictest13586-21ssh10:45:391
1983799912,17cyclictest7850irq/56-dwc_otg12:20:241
1983799882,17cyclictest8050irq/56-dwc_otg_12:35:311
19837998815,9cyclictest7850irq/56-dwc_otg09:15:371
1984199875,75cyclictest11535-21grep12:00:422
19841998611,52cyclictest7850irq/56-dwc_otg11:45:372
1984199853,57cyclictest0-21swapper/209:10:442
1983799853,14cyclictest9744-21ssh11:20:311
1983799833,8cyclictest0-21swapper/110:50:281
19837998323,9cyclictest6818-21missed_timers11:55:351
19837998312,15cyclictest17997-21kworker/u8:109:25:411
19833998330,41cyclictest7850irq/56-dwc_otg08:00:300
1984199823,72cyclictest6877-21ssh09:20:162
1983799827,8cyclictest7850irq/56-dwc_otg12:00:411
1983799826,8cyclictest7950irq/56-dwc_otg_11:00:421
1983799826,8cyclictest7950irq/56-dwc_otg_11:00:421
1983399822,70cyclictest30575-21grep11:05:420
1984199813,12cyclictest22735-21ssh12:15:182
1983799819,8cyclictest0-21swapper/109:20:161
1984199809,64cyclictest8050irq/56-dwc_otg_09:15:372
1984199804,70cyclictest20536-21cat09:35:422
19833998017,47cyclictest17993-21ssh11:30:280
19842997910,48cyclictest31015-21ntp_states11:45:403
1984199793,6cyclictest0-21swapper/212:20:232
1984199792,11cyclictest6915-21awk11:55:372
1983399791,65cyclictest7364-21modprobe10:00:130
1983399791,62cyclictest0-21swapper/011:10:390
1984199784,67cyclictest15226-21/usr/sbin/munin09:30:362
19842997721,33cyclictest381ktimersoftd/310:25:433
1983799767,17cyclictest8050irq/56-dwc_otg_09:35:401
19833997620,41cyclictest0-21swapper/010:57:520
1983399761,57cyclictest28267-21unixbench_singl07:30:410
1984199754,11cyclictest9712-21sh11:20:292
1983799754,10cyclictest22376-21apt-get12:15:201
1983799753,11cyclictest19925-21taskset12:10:461
1983799753,11cyclictest19925-21taskset12:10:461
1983399754,60cyclictest0-21swapper/010:15:260
19842997411,40cyclictest7950irq/56-dwc_otg_11:25:413
1984199744,12cyclictest11849-21apt-get10:05:252
1983399741,62cyclictest11879-21ssh10:44:120
1983399741,5cyclictest3463-21modprobe07:55:120
265962730,2sleep30-21swapper/310:22:093
19842997325,26cyclictest11578-21unixbench_singl12:00:423
1983399731,62cyclictest0-21swapper/012:25:390
300942720,2sleep30-21swapper/309:47:423
1984199725,53cyclictest14749-21cut08:25:262
1983399721,53cyclictest0-21swapper/010:36:560
19833997211,51cyclictest518-21rs:main0
19833997211,51cyclictest518-21rs:main0
19842997113,35cyclictest381ktimersoftd/310:45:293
1983399711,58cyclictest0-21swapper/009:52:150
19842997026,27cyclictest7850irq/56-dwc_otg10:05:263
1983399702,58cyclictest727-21runrttasks10:03:040
1983399701,55cyclictest0-21swapper/011:35:410
1983399701,14cyclictest30654-21diskmemload12:07:310
80952690,1sleep20-21swapper/208:05:362
1984299697,37cyclictest39-21ksoftirqd/311:00:263
1984299697,37cyclictest39-21ksoftirqd/311:00:263
1984299697,35cyclictest37-21rcuc/312:35:303
19842996831,19cyclictest7950irq/56-dwc_otg_09:10:423
19842996818,33cyclictest0-21swapper/312:15:203
19842996813,34cyclictest8050irq/56-dwc_otg_10:50:263
1983399681,52cyclictest0-21swapper/010:21:040
1983399681,13cyclictest2780-21ntpq11:50:360
19842996751,11cyclictest12194-21ssh09:25:413
1983399672,49cyclictest0-21swapper/010:10:250
1983399671,55cyclictest0-21swapper/011:19:090
1983399671,53cyclictest0-21swapper/007:25:400
1983399671,52cyclictest0-21swapper/007:46:480
1984299668,37cyclictest7950irq/56-dwc_otg_12:20:233
19833996644,5cyclictest7950irq/56-dwc_otg_07:45:130
1983399661,55cyclictest28994-21runrttasks09:45:420
1983399661,54cyclictest0-21swapper/012:33:130
19833996612,43cyclictest0-21swapper/009:40:230
19833996610,47cyclictest0-21swapper/008:45:240
19842996523,10cyclictest15964-21netstat09:30:363
1984199659,6cyclictest7950irq/56-dwc_otg_11:30:302
19833996522,31cyclictest7850irq/56-dwc_otg08:50:360
1983399651,53cyclictest0-21swapper/010:30:290
1983399651,48cyclictest0-21swapper/007:10:260
1983399641,55cyclictest0-21swapper/007:17:410
19833996415,35cyclictest14620-21/usr/sbin/munin08:25:230
19841996310,3cyclictest0-21swapper/211:40:042
1983799635,8cyclictest8050irq/56-dwc_otg_11:50:241
1983399632,47cyclictest3683-21/usr/sbin/munin07:55:330
326762610,1sleep10-21swapper/112:26:001
19841996113,4cyclictest5654-21grep08:00:282
19833996112,41cyclictest213722sleep008:42:130
1983399601,50cyclictest0-21swapper/008:39:220
1983399601,48cyclictest0-21swapper/008:55:290
1984199595,5cyclictest12554-21sed08:20:132
1984199592,15cyclictest0-21swapper/210:18:432
1983399592,49cyclictest0-21swapper/008:11:530
1984199587,4cyclictest7850irq/56-dwc_otg11:44:332
1984199587,4cyclictest7850irq/56-dwc_otg11:44:332
180012582,14sleep00-21swapper/007:05:150
1984299573,30cyclictest3452-21rm09:15:353
1984199577,7cyclictest125-21systemd-journal09:59:522
1983399576,11cyclictest0-21swapper/007:36:030
1983399576,11cyclictest0-21swapper/007:36:030
19833995732,9cyclictest29118-21aten_rbpower_cu09:05:270
1983399571,48cyclictest0-21swapper/008:17:240
1984299564,17cyclictest23511-21ssh09:40:233
1984299563,40cyclictest0-21swapper/309:21:313
1984199568,7cyclictest7850irq/56-dwc_otg10:34:202
1984199567,5cyclictest0-21swapper/209:41:532
1984199562,4cyclictest0-21swapper/212:26:342
1984199561,8cyclictest25383-21aten_rbpower_po07:25:272
1984199561,12cyclictest8782-21grep10:40:292
1983799563,15cyclictest0-21swapper/110:25:441
1984199555,4cyclictest7950irq/56-dwc_otg_10:14:032
1984199551,11cyclictest7850irq/56-dwc_otg07:22:432
1983399551,3cyclictest0-21swapper/008:22:330
1984199543,40cyclictest3145-21timerwakeupswit11:50:402
1984199541,5cyclictest8050irq/56-dwc_otg_12:07:342
19841995414,5cyclictest1831-21sh09:52:452
19841995414,5cyclictest11533-21sh10:05:122
1983399541,43cyclictest0-21swapper/008:06:210
1984299533,11cyclictest7950irq/56-dwc_otg_07:50:233
1984199533,7cyclictest8050irq/56-dwc_otg_08:50:482
1984199533,10cyclictest0-21swapper/210:24:152
1984199532,8cyclictest7850irq/56-dwc_otg09:45:482
1984199531,9cyclictest0-21swapper/212:33:032
19833995310,6cyclictest0-21swapper/008:31:150
19833995310,6cyclictest0-21swapper/008:31:150
1984199521,8cyclictest0-21swapper/210:59:452
1984199521,5cyclictest0-21swapper/211:10:402
1983799527,24cyclictest23-21ksoftirqd/110:05:241
1983799523,10cyclictest14928-21users11:25:431
1983799521,6cyclictest8050irq/56-dwc_otg_12:05:241
1983399522,5cyclictest0-21swapper/007:20:270
107602520,23sleep38050irq/56-dwc_otg_10:42:103
1984299512,10cyclictest0-21swapper/312:05:233
1984199513,9cyclictest232292chrt07:20:082
1984199511,5cyclictest4875-21ssh10:35:322
19841995115,4cyclictest28843-21latency_hist09:05:132
1984299504,20cyclictest9458-21/usr/sbin/munin11:20:293
1984199501,5cyclictest6972-21ssh11:16:202
19841995014,4cyclictest1869-21/usr/sbin/munin07:50:362
19841995011,4cyclictest31886-21uptime07:40:422
1984199493,5cyclictest8050irq/56-dwc_otg_11:05:242
1984199492,7cyclictest8050irq/56-dwc_otg_08:45:072
1984199492,4cyclictest21868-21/usr/sbin/munin08:45:242
19841994913,4cyclictest1349-21dhclient-script07:46:422
1984299486,15cyclictest0-21swapper/307:35:273
1984299486,15cyclictest0-21swapper/307:35:273
1984299484,7cyclictest0-21swapper/311:05:253
1984299483,30cyclictest0-21swapper/312:10:453
1984299483,30cyclictest0-21swapper/312:10:453
1984299483,20cyclictest8050irq/56-dwc_otg_09:35:403
1984199482,4cyclictest0-21swapper/207:38:482
1984199482,4cyclictest0-21swapper/207:38:482
1984199482,3cyclictest0-21swapper/208:34:072
1984199482,3cyclictest0-21swapper/208:34:072
19837994810,24cyclictest0-21swapper/108:20:181
1984299477,25cyclictest0-21swapper/308:55:263
1983799473,6cyclictest0-21swapper/111:35:271
1984199462,9cyclictest0-21swapper/208:36:382
19837994616,4cyclictest2173050wget07:15:261
1983799457,23cyclictest0-21swapper/110:20:261
1983799454,11cyclictest7850irq/56-dwc_otg07:40:271
1983799452,10cyclictest7850irq/56-dwc_otg08:35:311
1984299447,24cyclictest0-21swapper/308:50:273
1984199446,25cyclictest0-21swapper/208:20:182
1984199441,5cyclictest0-21swapper/208:59:152
1984199441,4cyclictest10394-21runrttasks08:11:582
1983799447,24cyclictest0-21swapper/107:10:261
1983799443,11cyclictest16030-21ntp_states09:30:381
1984299437,22cyclictest0-21swapper/311:40:263
1984299437,22cyclictest0-21swapper/311:40:263
1984299437,22cyclictest0-21swapper/308:15:253
1984299437,21cyclictest0-21swapper/307:15:263
1984199437,24cyclictest0-21swapper/207:10:262
1984299427,21cyclictest0-21swapper/308:20:203
1983799421,4cyclictest7850irq/56-dwc_otg08:40:261
19842994121,11cyclictest18279-21sh11:30:323
1984199419,5cyclictest301ktimersoftd/209:08:492
25902400,2sleep30-21swapper/310:32:233
239102401,35sleep10-21swapper/110:58:321
1984299407,23cyclictest0-21swapper/307:55:263
1984299407,23cyclictest0-21swapper/307:20:273
1984299392,29cyclictest8050irq/56-dwc_otg_12:25:263
1983799398,18cyclictest0-21swapper/110:15:191
1984299383,10cyclictest7850irq/56-dwc_otg10:35:253
1984199381,13cyclictest0-21swapper/207:30:432
1983799387,17cyclictest0-21swapper/110:10:301
1984299374,3cyclictest0-21swapper/312:30:253
19842993718,5cyclictest32553-21netstat09:50:353
1984299371,25cyclictest1460-21ssh11:10:303
1983799378,8cyclictest0-21swapper/110:00:261
1983799372,20cyclictest0-21swapper/107:52:361
19837993714,7cyclictest32753-21ssh09:50:391
1984299367,7cyclictest7950irq/56-dwc_otg_08:25:403
19837993627,6cyclictest221ktimersoftd/107:30:251
19837993617,5cyclictest3952-21sshd07:55:291
1983799361,25cyclictest0-21swapper/109:42:451
1983799361,11cyclictest0-21swapper/108:25:251
1983799359,6cyclictest7850irq/56-dwc_otg10:40:131
1983799356,12cyclictest0-21swapper/111:10:301
1983799355,6cyclictest7850irq/56-dwc_otg09:00:281
1983799355,21cyclictest17778-21ssh11:30:261
1983799353,15cyclictest1294-21grep10:30:401
1984299347,15cyclictest0-21swapper/308:30:263
1984299347,15cyclictest0-21swapper/308:30:263
1984299342,12cyclictest7950irq/56-dwc_otg_10:15:293
1984299341,15cyclictest7950irq/56-dwc_otg_08:49:543
1983799345,8cyclictest0-21swapper/107:25:251
1983799342,8cyclictest29890-21munin-run11:45:131
1983799342,8cyclictest29890-21munin-run11:45:131
19837993428,4cyclictest3396-21if_wlan012:30:321
1983799342,6cyclictest0-21swapper/111:05:401
19837993410,3cyclictest0-21swapper/108:55:401
1984299337,14cyclictest0-21swapper/309:00:283
1984299332,15cyclictest8050irq/56-dwc_otg_10:00:133
19842993314,6cyclictest30654-21diskmemload11:35:363
1984199331,7cyclictest0-21swapper/207:55:182
1983799338,12cyclictest6618-21vmstat11:15:421
1983799334,6cyclictest7850irq/56-dwc_otg09:55:281
19837993311,4cyclictest29338-21fschecks_count09:05:311
1984299329,11cyclictest7950irq/56-dwc_otg_11:50:333
1984299325,10cyclictest8050irq/56-dwc_otg_08:05:133
1984299324,13cyclictest8050irq/56-dwc_otg_11:15:303
1984299324,12cyclictest8050irq/56-dwc_otg_07:30:303
19842993214,12cyclictest24866-21apt11:00:133
1984299321,20cyclictest0-21swapper/308:10:443
1983799326,4cyclictest7950irq/56-dwc_otg_07:35:361
1983799326,4cyclictest7950irq/56-dwc_otg_07:35:361
1983799321,5cyclictest32551-21wget07:45:271
1984299311,6cyclictest31439-21/usr/sbin/munin07:40:383
1983799317,14cyclictest0-21swapper/108:45:271
19837993120,5cyclictest29033-21ssh09:45:461
1984299305,12cyclictest7950irq/56-dwc_otg_07:10:453
1984299302,12cyclictest735550wget08:05:263
1984299301,6cyclictest32757-21fschecks_count07:45:303
1984299301,5cyclictest21630-21munin-run08:45:123
19842993014,4cyclictest37-21rcuc/307:25:253
19842993013,11cyclictest17937-21ssh10:11:493
1984299301,18cyclictest0-21swapper/310:03:573
1983799307,4cyclictest30654-21diskmemload10:42:321
1983799305,5cyclictest8050irq/56-dwc_otg_08:10:331
1983799304,6cyclictest8050irq/56-dwc_otg_08:30:271
1983799304,6cyclictest8050irq/56-dwc_otg_08:30:271
19837993010,5cyclictest0-21swapper/107:20:391
19842992918,8cyclictest18487-21fschecks_count08:35:313
19842992910,13cyclictest30573-21taskset09:09:283
19837992922,5cyclictest221ktimersoftd/108:50:251
1983799292,18cyclictest0-21swapper/108:03:401
1983799291,6cyclictest7894-21missed_timers08:05:331
188302283,15sleep318895-21/usr/sbin/munin07:05:383
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional