You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-14 - 16:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Sat Mar 14, 2026 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3166525580,5sleep0101ktimersoftd/007:05:300
1859621100,2sleep00-21swapper/010:35:040
326479910854,9cyclictest7850irq/56-dwc_otg_12:20:150
3264799997,72cyclictest0-21swapper/007:55:180
3264799982,69cyclictest366-21fschecks_time12:10:200
3264799981,79cyclictest0-21swapper/010:15:150
3264799962,74cyclictest20528-21timerwakeupswit11:15:300
3264799961,83cyclictest8635-21ls11:40:290
3264799961,79cyclictest6536-21fschecks_count10:20:200
3264799961,77cyclictest20556-21ssh09:20:240
3264799961,76cyclictest13017-21ssh09:50:130
3264799941,82cyclictest15286-21apt-config09:15:040
32647999241,9cyclictest7750irq/56-dwc_otg10:55:160
32647999126,48cyclictest7750irq/56-dwc_otg09:25:310
3264799911,78cyclictest29276-21/usr/sbin/munin12:05:280
3264799901,74cyclictest13528-21rm10:28:380
3264799891,80cyclictest29549-21sh11:26:360
3264799891,76cyclictest17843-21seq11:51:330
3264799891,75cyclictest0-21swapper/007:40:220
3264799891,72cyclictest20845-21rm10:36:550
3264799881,69cyclictest0-21swapper/011:32:060
3264799881,68cyclictest14850-21ssh11:09:560
3264799871,70cyclictest27005-21ssh10:45:130
3264799871,68cyclictest18079-21ssh12:30:490
3264799871,67cyclictest29781-21ls08:30:240
3264799871,65cyclictest15760-21cut09:15:170
3264799861,71cyclictest6200-21ssh12:16:470
3264799861,70cyclictest3720-21taskset09:39:150
3264799861,70cyclictest15124-21ssh12:27:590
3264799861,35cyclictest1486-21grep08:40:310
3264799851,72cyclictest30141-21ssh09:31:090
3264799851,71cyclictest0-21swapper/008:24:550
3264799851,70cyclictest15482-21/usr/sbin/munin11:10:170
3264799851,69cyclictest0-21swapper/011:46:040
3264799851,68cyclictest25256-21ssh10:42:250
3264799841,71cyclictest0-21swapper/011:01:430
3264799841,69cyclictest0-21swapper/007:14:370
3264799841,68cyclictest23290-21taskset08:11:540
3264799841,60cyclictest21631-21awk12:35:290
3264799841,43cyclictest0-21swapper/007:35:030
3264799831,71cyclictest0-21swapper/011:59:560
3264799821,68cyclictest0-21swapper/009:47:270
3264799821,66cyclictest15345-21chrt07:50:240
3264799821,64cyclictest6654-21ssh09:41:560
3264799821,62cyclictest3987-21/usr/sbin/munin08:50:300
3264799821,62cyclictest3987-21/usr/sbin/munin08:50:300
32647998120,39cyclictest25049-21sh10:04:110
3264799811,66cyclictest0-21swapper/008:35:170
3264799811,61cyclictest0-21swapper/010:06:390
3264799811,49cyclictest3483-21apt-get11:35:160
3264799811,49cyclictest3483-21apt-get11:35:160
3264799802,66cyclictest0-21swapper/007:35:270
3264799801,70cyclictest0-21swapper/011:21:020
3264799801,64cyclictest5430-21latency_hist07:25:050
3264799801,61cyclictest0-21swapper/010:50:320
3264799792,67cyclictest0-21swapper/008:20:030
3264799791,66cyclictest0-21swapper/007:19:320
3264799791,64cyclictest21667-21runrttasks08:09:160
3264799791,64cyclictest21667-21runrttasks08:09:160
32647997827,11cyclictest0-21swapper/009:55:160
3264799782,52cyclictest0-21swapper/007:29:570
3264799781,66cyclictest0-21swapper/009:07:170
3264799771,66cyclictest0-21swapper/010:14:540
3264799771,61cyclictest0-21swapper/008:48:220
3264799771,57cyclictest0-21swapper/007:45:330
3264799761,64cyclictest24456-21ls12:00:200
3264799761,57cyclictest0-21swapper/008:03:590
3264799761,35cyclictest28991-21munin-run08:30:030
3264799751,61cyclictest6126-21grep08:55:200
3264799701,55cyclictest0-21swapper/009:04:360
321112660,2sleep30-21swapper/307:06:423
312812660,3sleep121-21rcuc/109:33:091
100372660,2sleep20-21swapper/212:21:142
157392630,2sleep229-21rcuc/212:29:072
182182580,4sleep218219-21rt-features09:55:292
152252570,2sleep30-21swapper/310:30:243
3264899554,20cyclictest0-21swapper/107:15:151
249932530,2sleep20-21swapper/211:20:482
3265099503,31cyclictest7850irq/56-dwc_otg_12:25:223
3265099493,26cyclictest0-21swapper/308:25:153
3265099448,24cyclictest0-21swapper/310:50:173
32648994413,6cyclictest20970-21ssh09:20:301
3264999432,10cyclictest11506-21fschecks_time11:05:222
3264899434,5cyclictest7850irq/56-dwc_otg_11:15:131
3265099424,23cyclictest0-21swapper/308:55:163
3265099424,21cyclictest27630-21/usr/sbin/munin11:25:193
3264899429,5cyclictest7750irq/56-dwc_otg10:45:041
32648994219,8cyclictest7850irq/56-dwc_otg_07:30:261
32648994214,3cyclictest0-21swapper/109:18:541
107832420,4sleep315918-21kworker/3:012:22:363
3265099411,25cyclictest0-21swapper/312:15:153
3265099411,25cyclictest0-21swapper/309:35:193
32649994110,12cyclictest1053950wget10:25:172
3264899415,6cyclictest24376-21cut12:00:191
32648994122,4cyclictest4030-21wget08:50:161
32648994122,4cyclictest4030-21wget08:50:161
32650994025,6cyclictest5835-21rm09:40:323
3265099401,29cyclictest0-21swapper/308:20:223
3265099401,24cyclictest0-21swapper/310:00:103
3264999407,25cyclictest0-21swapper/207:50:172
3264899406,5cyclictest27453-21wget08:25:171
3264899405,8cyclictest3746-21ssh11:35:201
3264899405,8cyclictest3746-21ssh11:35:201
3264899404,6cyclictest845550ssh12:20:181
3264899404,5cyclictest4072-21grep10:55:321
32648994016,5cyclictest4496-21/usr/sbin/munin07:20:251
32648994014,3cyclictest5626-21ssh09:40:301
32648994012,5cyclictest23-21ksoftirqd/112:25:211
32648994010,4cyclictest0-21swapper/110:15:211
290482400,1sleep20-21swapper/210:08:452
3265099396,16cyclictest17237-21missed_timers12:30:243
3265099395,18cyclictest23922-21latency_hist09:25:043
3265099391,24cyclictest0-21swapper/311:40:173
3265099391,21cyclictest11766-21latency11:05:243
32649993913,4cyclictest7513-21unixbench_singl10:20:312
32648993920,4cyclictest16567-21/usr/sbin/munin07:55:211
32648993919,5cyclictest1312850wget09:50:161
32648993914,13cyclictest0-21swapper/110:50:201
3264999386,18cyclictest12242-21ssh09:10:282
32649993831,5cyclictest2886-21sh10:55:182
32649993810,15cyclictest0-21swapper/212:30:222
3264899387,5cyclictest7750irq/56-dwc_otg10:00:301
3264899384,6cyclictest6715-21ls08:55:291
3264899384,5cyclictest7850irq/56-dwc_otg_09:45:291
32648993812,11cyclictest25891-21fschecks_count08:20:211
3265099377,13cyclictest3722-21ps10:55:293
3265099372,13cyclictest0-21swapper/308:20:043
3265099371,25cyclictest3263-21ls11:35:043
3265099371,21cyclictest0-21swapper/308:34:253
3265099371,12cyclictest1233350ssh09:10:293
3264999379,4cyclictest0-21swapper/210:35:042
32649993722,3cyclictest7850irq/56-dwc_otg_07:40:182
32649993720,7cyclictest7950irq/56-dwc_otg_10:00:162
3264999371,26cyclictest0-21swapper/212:37:212
3264899378,7cyclictest24365-21missed_timers08:15:251
3264899378,5cyclictest7750irq/56-dwc_otg07:10:301
3264899376,6cyclictest7850irq/56-dwc_otg_10:45:331
3264899376,4cyclictest0-21swapper/111:50:101
3264899375,6cyclictest11367-21aten_rbpower_cu09:10:161
3264899374,4cyclictest0-21swapper/112:08:491
3264899373,6cyclictest32714-21/usr/sbin/munin09:35:301
32648993719,8cyclictest221ktimersoftd/111:00:241
32648993711,4cyclictest7750irq/56-dwc_otg11:05:241
313882370,1sleep30-21swapper/312:08:393
3265099367,17cyclictest3840-21fschecks_time11:35:213
3265099367,17cyclictest3840-21fschecks_time11:35:213
3265099367,14cyclictest7669-21basename07:30:173
3265099366,20cyclictest19973-21grep11:15:223
3265099361,23cyclictest29519-21ls09:30:293
3265099361,19cyclictest0-21swapper/307:10:273
3264999362,19cyclictest7950irq/56-dwc_otg_09:55:032
32649993620,4cyclictest16244-21fschecks_count11:50:202
32649993617,5cyclictest10978-21diskmemload12:10:212
32649993615,5cyclictest28624-21/usr/sbin/munin12:05:202
3264999361,4cyclictest7950irq/56-dwc_otg_10:10:222
32649993612,5cyclictest20114-21aten_rbpower_cu11:55:162
3264999361,17cyclictest7950irq/56-dwc_otg_09:44:242
3264899369,4cyclictest0-21swapper/110:30:351
3264899369,12cyclictest0-21swapper/110:10:101
3264899367,4cyclictest7750irq/56-dwc_otg12:15:291
3264899366,6cyclictest0-21swapper/109:29:531
3264899366,4cyclictest0-21swapper/112:10:251
3264899364,5cyclictest21382-21ssh11:55:301
32648993616,5cyclictest923-21sshd09:00:211
32648993610,6cyclictest9550irq/80-mmc010:05:231
200652360,3sleep320068-21latency_hist08:05:043
3265099358,18cyclictest24416-21ssh09:25:183
3265099353,19cyclictest0-21swapper/310:50:043
32650993519,9cyclictest23661-21sh12:38:293
32650993516,5cyclictest8917-21sh10:22:533
32650993515,13cyclictest908-21runrttasks11:56:553
32650993514,6cyclictest19083-21fschecks_count10:35:223
3265099351,20cyclictest7062-21ssh11:00:183
3265099351,20cyclictest0-21swapper/307:45:233
3265099351,19cyclictest27471-21ls11:25:043
3265099351,19cyclictest0-21swapper/312:10:223
3265099351,14cyclictest0-21swapper/307:40:293
3264999357,7cyclictest0-21swapper/209:30:212
32649993516,5cyclictest27084-21seq08:24:012
32649993511,7cyclictest0-21swapper/209:15:192
32649993511,11cyclictest7409-21latency_hist07:30:042
3264899358,4cyclictest0-21swapper/110:39:551
3264899354,4cyclictest7750irq/56-dwc_otg11:25:281
3264899354,4cyclictest0-21swapper/109:58:351
3264899352,4cyclictest10729-21taskset07:37:371
3264899351,6cyclictest15549-21ssh11:10:191
3264899351,5cyclictest0-21swapper/110:29:381
3264899351,24cyclictest18669-21cut08:00:201
3265099349,4cyclictest0-21swapper/311:10:303
3265099346,20cyclictest7750irq/56-dwc_otg12:00:183
32650993415,11cyclictest11702-21sh10:25:313
3265099341,24cyclictest0-21swapper/307:25:173
32650993411,13cyclictest0-21swapper/309:55:573
3264999345,3cyclictest0-21swapper/209:29:012
3264999345,21cyclictest7950irq/56-dwc_otg_10:16:542
3264999345,16cyclictest0-21swapper/211:42:512
3264999342,4cyclictest8765-21/usr/sbin/munin09:45:202
32649993412,14cyclictest0-21swapper/210:42:182
3264999341,15cyclictest7950irq/56-dwc_otg_09:20:182
3264899347,3cyclictest7750irq/56-dwc_otg10:20:311
3264899345,4cyclictest0-21swapper/111:45:181
3264899344,5cyclictest9444-21sh11:41:281
3264899342,8cyclictest32140-21fschecks_time11:30:201
3264899342,5cyclictest24044-21/usr/sbin/munin11:20:231
3264899342,5cyclictest20733-21ssh12:35:161
32648993415,11cyclictest221ktimersoftd/108:32:521
3265099338,11cyclictest5559-21modprobe08:55:043
3265099338,11cyclictest5559-21modprobe08:55:043
3265099336,15cyclictest55612sleep310:19:293
3265099332,19cyclictest7850irq/56-dwc_otg_09:45:263
3265099332,14cyclictest0-21swapper/310:06:283
32650993318,8cyclictest7850irq/56-dwc_otg_07:35:283
3264999335,16cyclictest0-21swapper/211:34:392
32649993315,10cyclictest4252-21basename12:15:162
3264999331,10cyclictest0-21swapper/208:05:212
3264999331,10cyclictest0-21swapper/208:05:212
3264899334,5cyclictest7850irq/56-dwc_otg_08:40:311
3265099328,12cyclictest22743-21proc_pri08:10:283
3265099327,17cyclictest16934-21idleruntime07:55:223
3265099327,15cyclictest0-21swapper/309:50:163
3265099325,16cyclictest3896-21wget07:20:173
3265099323,12cyclictest1973-21modprobe08:45:033
3265099321,9cyclictest7750irq/56-dwc_otg10:40:243
3265099321,3cyclictest0-21swapper/311:50:103
3264999325,3cyclictest0-21swapper/212:00:112
3264999321,5cyclictest12004-21cut11:45:182
32649993214,10cyclictest75472sleep207:30:062
32649993213,4cyclictest8048-21cat09:00:212
3264999321,16cyclictest32731-21apt-get09:35:182
32649993211,12cyclictest0-21swapper/211:01:052
3264899325,4cyclictest7750irq/56-dwc_otg07:50:201
3264899323,3cyclictest0-21swapper/108:10:201
32648993221,5cyclictest18468-21sh12:31:321
3264899321,5cyclictest11175-21/usr/sbin/munin07:40:221
3264899321,24cyclictest31306-21cut08:35:211
3265099318,5cyclictest15605-21grep11:50:043
3265099317,21cyclictest0-21swapper/307:50:173
3265099317,12cyclictest3034-21cat07:15:313
3265099312,15cyclictest15982-21grep09:15:213
3264999319,4cyclictest0-21swapper/211:16:452
3264999318,4cyclictest0-21swapper/211:37:182
3264999318,4cyclictest0-21swapper/211:37:182
3264999317,9cyclictest0-21swapper/210:38:572
3264999317,4cyclictest0-21swapper/211:14:372
3264999312,3cyclictest0-21swapper/210:50:292
32649993123,5cyclictest10565-21grep09:05:332
3264999311,4cyclictest0-21swapper/210:45:432
32649993113,5cyclictest13515-21sed07:45:242
32649993113,10cyclictest29767-21ssh11:27:002
32649993110,11cyclictest168092chrt07:55:212
3264899311,6cyclictest11-21rcu_preempt07:45:261
3264899311,18cyclictest2720-21sed08:45:231
3264899311,12cyclictest0-21swapper/107:30:041
3265099307,9cyclictest20483-21fschecks_time08:05:213
3265099307,9cyclictest20483-21fschecks_time08:05:213
3264999303,4cyclictest7850irq/56-dwc_otg_08:30:182
32649993010,5cyclictest4345-21fw_packets08:50:202
32649993010,5cyclictest4345-21fw_packets08:50:202
32649993010,12cyclictest382-21cat07:10:172
3264899305,6cyclictest10089-21grep09:05:271
3265099294,18cyclictest0-21swapper/308:36:123
3265099291,19cyclictest0-21swapper/310:10:543
3264999292,8cyclictest25357-21cron08:20:032
3264999291,5cyclictest7750irq/56-dwc_otg08:40:232
3264999291,4cyclictest7850irq/56-dwc_otg_08:25:182
32649992914,5cyclictest7149-21seq08:57:302
3264999291,14cyclictest9359-21/usr/sbin/munin07:35:302
3264899295,5cyclictest20180-21/usr/sbin/munin08:05:241
3264899295,5cyclictest20180-21/usr/sbin/munin08:05:241
3265099288,10cyclictest7850irq/56-dwc_otg_08:47:573
3265099281,5cyclictest7950irq/56-dwc_otg_09:00:253
3265099281,3cyclictest0-21swapper/309:05:153
3264999283,14cyclictest0-21swapper/207:20:152
3264999282,9cyclictest0-21swapper/207:15:292
32649992810,9cyclictest2864-21ntp_kernel_err08:45:242
312292284,8sleep10-21swapper/107:05:231
32649992712,4cyclictest301ktimersoftd/208:00:242
323422273,15sleep20-21swapper/207:09:372
32649992610,7cyclictest29-21rcuc/208:35:292
3264999251,4cyclictest7850irq/56-dwc_otg_08:10:142
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional