You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-02 - 22:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Tue Jun 02, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
655921200319,11969sleep23259-21kworker/u8:007:33:412
16940222469,2224sleep026894-21kworker/0:211:32:450
99402221914,2188sleep09942-21ntp_states11:08:430
8511269138,640sleep18519-21ntpq07:38:491
6477254315,510sleep36478-21/usr/sbin/munin07:33:383
30637238414,356sleep030682-21ntpq07:08:440
30637238414,356sleep030682-21ntpq07:08:440
13131236813,341sleep113170-21idleruntime-cro09:38:181
14972235512,327sleep314980-21/usr/sbin/munin09:43:293
14972235512,327sleep314980-21/usr/sbin/munin09:43:293
8773227992,143sleep18825-21cpuspeed09:23:331
19810224847,183sleep119934-21timerwakeupswit08:13:491
4760223137,38sleep14781-21modprobe12:38:171
295722250,16sleep130026-21cyclictest09:03:411
584422230,30sleep130026-21cyclictest09:13:391
663222216,143sleep1816-21unixbench_singl10:38:531
2702222262,120sleep12815-21unixbench_multi12:29:051
29259221816,181sleep029338-21mailstats08:43:510
5995221755,144sleep10-21swapper/107:33:191
24412221410,190sleep124494-21fschecks_time10:13:381
24412221410,190sleep124494-21fschecks_time10:13:381
31777220318,169sleep331811-21ps12:18:493
15848220114,170sleep115702-21/usr/sbin/munin11:28:381
14134220146,112sleep112-21rcu_sched09:38:511
197122000,8sleep30-21swapper/310:43:463
175421970,7sleep20-21swapper/207:18:422
3261921829,21sleep10-21swapper/112:23:191
312421779,153sleep12618-21apt-get07:23:311
7592217610,154sleep17610-21netstat09:18:401
18922216717,96sleep10-21swapper/108:13:201
31072216410,117sleep131148-21latency_hist12:18:191
58521618,115sleep1649-21cat12:23:361
32125216048,23sleep10-21swapper/108:53:411
2403421599,94sleep124061-21idleruntime-cro10:13:181
1027215694,45sleep11075-21readlink07:18:181
21833215510,130sleep321911-21find10:03:513
24479215419,121sleep224517-21ntpq08:28:442
2131421489,82sleep10-21swapper/108:18:451
18560214813,121sleep318592-21proc_pri09:53:463
17641214815,93sleep10-21swapper/111:33:431
1377221489,128sleep1363-21snmpd07:56:041
32539214735,93sleep20-21swapper/208:53:502
1426421477,15sleep10-21swapper/107:58:311
31360214644,20sleep10-21swapper/110:33:481
1143121416,93sleep10-21swapper/107:48:421
1454221398,31sleep10-21swapper/111:23:421
4964213549,74sleep14788-21/usr/sbin/munin10:53:381
8856213317,84sleep00-21swapper/009:23:360
2939213328,83sleep03117-21latency_hist10:48:200
23786213137,80sleep123825-21modprobe08:28:181
20285213044,42sleep10-21swapper/109:58:491
1785921248,101sleep118008-21cat09:53:201
3638212275,22sleep10-21swapper/110:48:461
24749212122,73sleep224795-21munin-plugin-st11:58:182
28770211916,79sleep128847-21gpgv10:28:191
28770211916,79sleep128847-21gpgv10:28:191
18218211915,92sleep00-21swapper/011:34:140
2800211816,86sleep22505-21/usr/sbin/munin09:03:392
16804211827,54sleep10-21swapper/108:04:121
1902621165,11sleep10-21swapper/109:54:041
24697211523,78sleep224737-21grep10:13:432
24697211523,78sleep224737-21grep10:13:432
2301121145,12sleep10-21swapper/111:50:501
7920211227,69sleep10-21swapper/111:03:311
1017521128,13sleep10-21swapper/111:08:481
645821118,45sleep10-21swapper/110:58:351
2582121098,61sleep10-21swapper/108:33:391
2930621087,18sleep10-21swapper/110:29:161
223921078,16sleep10-21swapper/109:03:181
22036210715,75sleep222145-21fschecks_time11:48:372
20478210615,70sleep220549-21fschecks_time11:43:362
20397210618,70sleep220547-21unixbench_singl09:58:502
1475121060,5sleep30-21swapper/311:23:473
1227521056,12sleep10-21swapper/111:17:521
30877210422,67sleep230951-21timerandwakeup07:08:502
30877210422,67sleep230951-21timerandwakeup07:08:502
30013210446,30sleep10-21swapper/107:08:211
26403210410,56sleep10-21swapper/110:18:471
20661210443,49sleep120734-21sed11:43:391
2867121039,20sleep1284-21dbus-daemon12:08:461
2206210213,47sleep10-21swapper/107:18:501
28837210110,74sleep228857-21grep08:43:412
26382210116,67sleep326526-21latency_hist12:03:193
26382210116,67sleep326526-21latency_hist12:03:193
22862210123,60sleep022900-21grep10:08:370
31656210043,40sleep10-21swapper/107:13:211
29235210015,67sleep228734-21apt-get10:28:292
274512996,14sleep10-21swapper/108:38:441
127142998,19sleep112739-21grep11:18:371
3048129634,46sleep10-21swapper/107:08:411
2781829618,62sleep227931-21timerandwakeup08:38:532
1259829538,30sleep00-21swapper/009:33:490
2904829428,44sleep229100-21grep07:03:422
2787729424,47sleep027895-21modprobe12:08:170
2787729424,47sleep027895-21modprobe12:08:170
147229440,14sleep10-21swapper/110:43:331
1040829421,55sleep010463-21fschecks_count09:28:360
2792929322,49sleep228016-21grep12:08:192
2792929322,49sleep228016-21grep12:08:192
212022937,53sleep121221-21apt-get10:03:291
105492925,76sleep10-21swapper/107:44:261
240742918,45sleep10-21swapper/108:28:301
200592915,74sleep10-21swapper/111:42:431
326329013,61sleep03412-21latency_hist12:33:190
289529013,60sleep32918-21/usr/sbin/munin09:03:403
64332897,45sleep10-21swapper/107:33:241
2315528914,57sleep223206-21awk10:08:432
1894328817,57sleep019037-21fschecks_time11:38:380
1230828818,45sleep212338-21munin-run11:18:182
111732887,44sleep10-21swapper/109:28:501
3197528721,41sleep20-21swapper/212:18:522
1092928719,56sleep311022-21timerandwakeup09:28:473
2688228622,41sleep026907-21munin-run08:38:180
1707328616,51sleep317228-21latency_hist11:33:203
2569828516,54sleep025825-21timerwakeupswit11:58:480
2569828516,54sleep025825-21timerwakeupswit11:58:480
1322028515,44sleep312665-21/usr/sbin/munin07:53:483
1993728418,46sleep220059-21/usr/sbin/munin08:13:502
1636528422,46sleep016400-21latency08:04:030
1603128420,48sleep215550-21apt-get08:03:282
366428317,46sleep23103-21/usr/sbin/munin07:24:412
3099228319,47sleep331011-21idleruntime-cro12:18:173
2086928216,47sleep320934-21grep11:43:463
1818128213,54sleep218213-21tune2fs09:53:372
2825228121,45sleep328262-21/usr/sbin/munin12:08:363
1923528110,49sleep019133-21/usr/sbin/munin08:13:360
149632815,41sleep10-21swapper/109:43:231
149632815,41sleep10-21swapper/109:43:231
1436328111,55sleep014472-21perl07:58:380
1124828110,16sleep00-21swapper/011:13:390
771528014,48sleep37800-21mailstats09:18:463
313212806,40sleep10-21swapper/108:50:101
260762805,42sleep10-21swapper/111:59:581
260762805,42sleep10-21swapper/111:59:581
245912806,40sleep10-21swapper/111:56:151
1425828028,31sleep00-21swapper/011:23:360
52812795,39sleep10-21swapper/109:12:261
3206027916,48sleep2662-21cron10:38:172
3156427925,30sleep20-21swapper/208:53:182
205527915,40sleep02201-21unixbench_singl07:18:490
920127820,25sleep30-21swapper/311:08:183
278292785,40sleep10-21swapper/112:07:411
278292785,40sleep10-21swapper/112:07:411
1141027822,30sleep00-21swapper/007:48:410
769027715,29sleep20-21swapper/209:18:452
73827721,36sleep0791-21latency_hist08:58:200
647327716,43sleep36539-21fschecks_time10:58:373
457427714,48sleep34711-21cat10:53:203
2738727713,49sleep00-21swapper/008:38:430
744527610,54sleep0363-21snmpd07:34:400
2886427615,43sleep229076-21apt-key10:28:202
2886427615,43sleep229076-21apt-key10:28:202
2324427615,46sleep023274-21cut11:53:190
137832765,16sleep20-21swapper/211:21:572
3233227519,27sleep30-21swapper/308:53:473
297602755,57sleep00-21swapper/007:06:020
288302757,15sleep10-21swapper/108:43:401
2482927520,26sleep00-21swapper/011:58:200
2021627512,48sleep020284-21swap09:58:480
1391627522,31sleep20-21swapper/209:38:472
1830527417,23sleep00-21swapper/009:53:390
2949027320,31sleep229640-21latency_hist12:13:192
1816827320,24sleep20-21swapper/211:33:522
1329527319,32sleep00-21swapper/007:53:490
780227224,24sleep00-21swapper/009:18:470
2994027219,30sleep329969-21munin-run07:08:193
2871727230,26sleep00-21swapper/010:28:180
2871727230,26sleep00-21swapper/010:28:180
2826527219,30sleep00-21swapper/012:08:370
2720527222,30sleep30-21swapper/308:38:373
2637727212,42sleep026402-21/usr/sbin/munin10:18:460
1244227215,42sleep312473-21latency_hist07:53:203
1085227218,31sleep20-21swapper/207:48:182
842127112,40sleep28025-21/usr/sbin/munin07:38:462
1305027112,35sleep013082-21ntp_states11:18:440
110727118,31sleep30-21swapper/307:18:203
831327020,30sleep20-21swapper/211:03:442
54227020,20sleep00-21swapper/010:38:520
376927013,40sleep23840-21grep10:48:512
274227011,16sleep30-21swapper/307:23:203
2541927014,39sleep325569-21latency_hist08:33:213
134527024,29sleep30-21swapper/308:58:443
625626916,36sleep36377-21sed09:13:503
3044426920,29sleep20-21swapper/212:13:482
2256926912,39sleep322602-21df08:23:363
1240926911,16sleep20-21swapper/209:33:472
3096426815,27sleep30-21swapper/307:08:513
3096426815,27sleep30-21swapper/307:08:513
1884826816,39sleep318855-21wc11:38:333
183522688,45sleep00-21swapper/008:08:470
171426818,24sleep00-21swapper/012:28:190
1090526823,22sleep00-21swapper/007:48:200
1007526818,30sleep30-21swapper/309:28:193
478426720,24sleep20-21swapper/209:08:522
474526710,42sleep2363-21snmpd12:38:062
3257426717,25sleep20-21swapper/210:38:422
3051826710,16sleep00-21swapper/008:48:400
2812526715,29sleep30-21swapper/310:23:493
2812526715,29sleep30-21swapper/310:23:493
1481426724,23sleep20-21swapper/207:58:482
1385126712,40sleep313895-21awk09:38:443
579426617,24sleep20-21swapper/210:54:442
526226614,29sleep30-21swapper/310:53:443
2534926611,40sleep325351-21modprobe10:15:183
2534926611,40sleep325351-21modprobe10:15:183
1404126620,27sleep00-21swapper/009:38:500
103922668,14sleep20-21swapper/209:28:342
686626517,26sleep00-21swapper/010:58:460
416826516,21sleep30-21swapper/309:08:373
378426523,27sleep03797-21modprobe09:08:180
3129226511,16sleep00-21swapper/010:33:470
2385526517,25sleep30-21swapper/311:53:433
2200026517,28sleep222046-21apt-config08:23:182
1694826518,30sleep316991-21/usr/sbin/munin09:48:423
1283626512,37sleep312877-21if_eth011:18:393
589426415,32sleep30-21swapper/307:32:453
2408426417,28sleep30-21swapper/310:13:203
232526417,26sleep20-21swapper/212:28:452
2289926418,20sleep00-21swapper/008:23:450
1631026417,23sleep20-21swapper/209:48:202
1631026417,23sleep20-21swapper/209:48:202
2213726317,25sleep00-21swapper/008:23:210
925926210,22sleep20-21swapper/209:23:462
605726217,27sleep26095-21sshd09:13:452
324526211,35sleep33249-21diskstats07:24:013
2204726219,26sleep322092-21munin-run08:23:193
1699426218,25sleep00-21swapper/009:48:430
1140526218,28sleep311441-21ntpq11:13:433
539426119,21sleep00-21swapper/007:28:510
489826125,15sleep00-21swapper/010:53:370
3004926119,20sleep30-21swapper/310:29:513
2599526113,26sleep30-21swapper/310:18:373
185192619,39sleep20-21swapper/208:08:492
1787826118,27sleep317937-21/usr/sbin/munin08:08:383
16032615,35sleep30-21swapper/312:27:333
1151026117,30sleep211541-21/usr/sbin/munin07:48:432
3244826018,26sleep232482-21/usr/sbin/munin07:13:392
323026014,23sleep00-21swapper/010:48:290
244432609,33sleep00-21swapper/008:28:420
2193826018,25sleep321935-21apt-get11:48:303
2163426020,21sleep20-21swapper/210:03:462
1942226016,19sleep20-21swapper/211:38:482
1305826016,27sleep213092-21/usr/sbin/munin07:53:432
93482599,36sleep09323-21latency_hist07:43:200
609325918,19sleep20-21swapper/207:33:202
3186625919,21sleep00-21swapper/007:13:240
292392597,21sleep00-21swapper/010:28:310
2546025920,23sleep225498-21ntpq11:58:422
2546025920,23sleep225498-21ntpq11:58:422
185032595,12sleep30-21swapper/311:37:563
1467425913,28sleep214750-21/usr/sbin/munin11:23:472
292172587,12sleep30-21swapper/308:43:503
276925815,26sleep22744-21apt-get10:48:182
2696125819,22sleep20-21swapper/208:38:202
2685125820,17sleep30-21swapper/312:03:403
2685125820,17sleep30-21swapper/312:03:403
3138725712,18sleep00-21swapper/012:18:380
310225713,28sleep03121-21apt-get07:23:300
15872576,38sleep20-21swapper/212:27:202
125225715,24sleep21344-21memory08:58:432
930525617,15sleep30-21swapper/307:43:193
108302565,11sleep30-21swapper/307:48:033
989025513,25sleep30-21swapper/311:08:423
2253125514,23sleep222532-21cpu08:23:352
637925416,17sleep00-21swapper/009:13:510
463225413,23sleep04702-21sensors_temp09:08:500
3212925415,24sleep332138-21munin-run10:38:183
29552546,35sleep30-21swapper/312:29:593
2191525421,17sleep00-21swapper/010:03:520
1936525415,21sleep30-21swapper/308:13:393
1488325413,21sleep30-21swapper/307:58:493
949125310,16sleep20-21swapper/211:08:332
684125313,23sleep26847-21ntp_states10:58:442
3012925315,22sleep030174-21ntp_states12:13:420
1655125312,23sleep216621-21df09:48:322
160442539,18sleep30-21swapper/308:03:293
1500025311,18sleep00-21swapper/009:43:340
1500025311,18sleep00-21swapper/009:43:340
1572425213,23sleep215721-21apt-get11:28:312
2039125112,23sleep020386-21apt-get11:43:300
3029825012,21sleep330301-21apt-get08:48:303
2570824911,22sleep30-21swapper/308:33:373
821424811,15sleep00-21swapper/011:03:420
376724819,16sleep00-21swapper/012:33:400
263432488,17sleep20-21swapper/210:18:442
122924714,15sleep30-21swapper/310:43:193
1248624614,16sleep30-21swapper/309:33:483
118624616,16sleep00-21swapper/008:58:410
2097624510,20sleep320986-21munin-run10:03:193
255902449,19sleep025649-21munin-run10:18:170
255902449,19sleep025649-21munin-run10:18:170
37082418,15sleep30-21swapper/312:33:393
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional