You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-01 - 22:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Mon Jun 01, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30717216276,1610sleep125749-21kworker/1:207:36:061
24344216046,1553sleep121209-21kworker/1:210:42:431
44892145610,1434sleep21466-21kworker/2:007:56:052
20084248817,452sleep120093-21ntpq07:03:491
32439246625,424sleep232445-21/usr/sbin/munin09:23:532
1090623467,290sleep1363-21snmpd08:18:201
32167228024,207sleep1363-21snmpd11:05:111
14723227071,182sleep114927-21apt-config11:53:231
19092226417,181sleep119185-21fschecks_time08:43:431
2959922599,47sleep10-21swapper/110:58:371
13077224845,181sleep113199-21latency_hist11:48:231
26480224542,51sleep10-21swapper/110:48:401
26480224542,51sleep10-21swapper/110:48:401
2541223544,176sleep12695-21unixbench_multi07:48:531
2541223544,176sleep12695-21unixbench_multi07:48:531
15400221641,124sleep115457-21fschecks_count10:13:411
24748220746,84sleep10-21swapper/107:18:341
1558220711,77sleep11658-21latency_hist11:13:231
32528220517,126sleep10-21swapper/107:43:241
27335220054,24sleep10-21swapper/112:33:241
11304220063,61sleep111367-21apt-key11:43:221
4952219510,170sleep05069-21cat11:23:230
18321219347,97sleep118397-21fschecks_time12:03:411
23443219025,143sleep023575-21latency_hist08:58:230
23443219025,143sleep023575-21latency_hist08:58:230
13995218045,80sleep10-21swapper/110:08:401
17068217817,147sleep217096-21ntpq11:58:452
8188216919,125sleep38284-21fschecks_time08:08:413
21056216119,124sleep321132-21sendmail_mailtr08:48:503
24818216032,53sleep10-21swapper/109:01:391
16534215911,132sleep0225-21systemd-journal10:17:430
298721558,23sleep10-21swapper/109:33:491
26499215414,83sleep126548-21modprobe09:08:211
26949215310,100sleep127007-21store09:08:251
5387215212,94sleep15432-21ntp_offset07:58:461
1248621527,15sleep10-21swapper/110:03:441
27052215016,118sleep227036-21apt-get09:08:392
3270021496,129sleep00-21swapper/011:08:330
1091921478,126sleep110932-21irqstats09:58:431
9405214639,61sleep10-21swapper/108:13:231
29168214513,73sleep10-21swapper/107:31:051
24655214518,83sleep10-21swapper/110:43:321
15225214534,66sleep115231-21fschecks_count11:53:441
3041821427,17sleep10-21swapper/109:18:351
14929214115,112sleep215007-21find08:28:552
28033214026,87sleep10-21swapper/112:33:481
7836213811,108sleep07746-21apt11:30:090
27801213520,89sleep027832-21modprobe07:28:220
22687213520,58sleep10-21swapper/112:18:221
3145213423,50sleep10-21swapper/111:18:231
3145213423,50sleep10-21swapper/111:18:231
21937213327,56sleep121978-21ntpq10:33:471
391213237,48sleep10-21swapper/107:43:411
3776213037,73sleep10-21swapper/107:53:461
30310212811,101sleep230431-21timerwakeupswit07:33:532
28668212819,69sleep10-21swapper/110:53:531
28668212819,69sleep10-21swapper/110:53:531
3243721227,93sleep132438-21ntp_states09:23:521
673821218,41sleep10-21swapper/111:28:371
26509212121,79sleep326026-21apt-get07:23:323
25786211843,59sleep125929-21date12:28:241
21532211632,47sleep021551-21apt-get10:33:340
15583211635,23sleep115586-21cron08:33:221
28124211521,74sleep328195-21ls12:33:503
26007211521,68sleep026105-21apt-config10:48:230
17419211436,22sleep10-21swapper/111:58:521
467821136,57sleep10-21swapper/111:20:291
1660211314,79sleep01744-21/usr/sbin/munin09:28:510
16441211344,55sleep016545-21latency_hist11:58:230
9171211126,69sleep19266-21grep09:53:431
9171211126,69sleep19266-21grep09:53:431
8798211146,51sleep11-21systemd09:53:221
24226211115,78sleep024293-21/usr/sbin/munin08:58:520
5600211012,81sleep05645-21/usr/sbin/munin11:23:470
308621097,16sleep30-21swapper/309:33:513
25516210933,58sleep025608-21/usr/sbin/munin09:03:440
19856210959,35sleep10-21swapper/112:08:371
1507210814,78sleep31538-21awk11:13:213
19603210729,55sleep219622-21apt-get07:03:342
18773210515,71sleep00-21swapper/010:23:450
18773210515,71sleep00-21swapper/010:23:450
15804210515,72sleep315289-21/usr/sbin/munin10:13:513
24942210216,68sleep025002-21/usr/sbin/munin12:23:500
27015210014,70sleep326531-21apt-get09:08:313
2529721008,14sleep20-21swapper/207:18:512
2162521000,8sleep00-21swapper/007:08:470
16884210020,62sleep10-21swapper/110:18:331
12860210035,48sleep10-21swapper/108:23:411
78482997,80sleep17869-21latency_hist08:08:231
71722995,19sleep10-21swapper/109:47:291
2846729914,68sleep228480-21/usr/sbin/munin12:33:542
167992995,16sleep10-21swapper/108:34:061
2880429819,60sleep328336-21apt-get09:13:323
1954229815,70sleep10-21swapper/110:26:121
1954229815,70sleep10-21swapper/110:26:121
1745029812,17sleep20-21swapper/208:38:352
212322978,72sleep120517-21/usr/sbin/munin08:48:521
1015729715,66sleep010205-21cat11:38:410
879229619,56sleep08796-21modprobe09:53:210
2594429617,60sleep326005-21apt-config10:48:223
2900129518,28sleep10-21swapper/109:13:551
265422949,70sleep126575-21grep07:23:381
2391429413,17sleep20-21swapper/207:13:592
2391429413,17sleep20-21swapper/207:13:592
146829413,60sleep31039-21/usr/sbin/munin09:28:463
1328429417,51sleep013372-21grep08:23:520
1131629416,65sleep211365-21fschecks_count08:18:412
1131629416,65sleep211365-21fschecks_count08:18:412
183142930,11sleep00-21swapper/008:38:530
654929210,69sleep2363-21snmpd08:03:402
54482925,19sleep10-21swapper/109:40:101
3249029217,59sleep232640-21latency_hist11:08:232
241432925,14sleep10-21swapper/107:14:461
241432925,14sleep10-21swapper/107:14:461
230792925,12sleep10-21swapper/108:54:411
230792925,12sleep10-21swapper/108:54:411
750329110,12sleep20-21swapper/211:28:542
106429119,54sleep10-21swapper/109:28:361
123032905,17sleep10-21swapper/108:21:021
123032905,17sleep10-21swapper/108:21:021
498828925,39sleep34983-21apt-get07:58:363
2484428814,53sleep024931-21fschecks_time07:18:410
2368228814,14sleep20-21swapper/208:58:372
2111028713,58sleep321264-21latency_hist12:13:243
2285028621,39sleep323001-21head08:53:543
2285028621,39sleep323001-21head08:53:543
2143328617,50sleep321540-21fschecks_time12:13:403
1444928620,43sleep314604-21grep10:08:503
88702855,11sleep10-21swapper/111:33:541
88702855,11sleep10-21swapper/111:33:541
2121428517,37sleep30-21swapper/307:08:353
2048728418,45sleep220554-21awk10:28:522
955528314,54sleep29615-21/usr/sbin/munin09:53:522
955528314,54sleep29615-21/usr/sbin/munin09:53:522
649728313,45sleep36550-21sort11:28:223
2130228316,21sleep121311-21forks07:08:411
2515928216,48sleep225225-21sendmail_mailqu10:43:492
849228119,49sleep08527-21ntp_kernel_pll_08:08:460
2620628116,21sleep30-21swapper/307:23:243
1948628117,45sleep219547-21mailstats08:43:522
1235828113,54sleep312394-21sshd11:44:053
1758728011,56sleep017723-21awk10:18:530
150252795,14sleep20-21swapper/210:13:212
187022785,61sleep10-21swapper/108:43:191
3032827718,35sleep330462-21head10:58:563
2313627715,12sleep10-21swapper/112:18:431
1997427715,49sleep019993-21apt-get10:28:330
347227625,29sleep03565-21timerandwakeup09:34:000
2573227612,50sleep225758-21idleruntime-cro12:28:202
232727621,25sleep20-21swapper/207:48:502
232727621,25sleep20-21swapper/207:48:502
1741627611,28sleep20-21swapper/210:18:502
3170227521,23sleep00-21swapper/007:38:510
1451427516,20sleep00-21swapper/008:28:440
1176427513,32sleep011750-21apt-get11:43:380
1009127524,23sleep20-21swapper/208:13:492
2117527419,10sleep10-21swapper/110:32:091
1852927411,49sleep318533-21diskstats10:23:403
1852927411,49sleep318533-21diskstats10:23:403
1295027412,39sleep213096-21grep10:03:532
77802738,12sleep00-21swapper/008:08:060
753027311,27sleep20-21swapper/209:48:352
319642738,14sleep00-21swapper/011:03:530
2939027326,25sleep30-21swapper/310:58:233
17912737,16sleep30-21swapper/311:13:333
17912737,16sleep30-21swapper/311:13:333
127902738,16sleep00-21swapper/010:03:510
525727223,27sleep20-21swapper/207:58:432
293272727,12sleep30-21swapper/307:33:053
2654927215,43sleep226620-21cmp09:08:222
1032427213,40sleep310452-21/usr/sbin/munin08:13:543
54622719,50sleep0363-21snmpd09:40:220
143627121,22sleep20-21swapper/209:28:452
2941427014,25sleep20-21swapper/207:33:242
2859827016,23sleep00-21swapper/010:53:520
2859827016,23sleep00-21swapper/010:53:520
2855327018,33sleep028591-21grep07:28:500
1995627011,42sleep020012-21basename12:08:400
1199927027,26sleep30-21swapper/308:18:553
1199927027,26sleep30-21swapper/308:18:553
1142227011,28sleep20-21swapper/211:43:232
1072127017,19sleep30-21swapper/311:38:523
666726916,35sleep06817-21sed09:43:530
516326915,10sleep10-21swapper/111:23:321
2361826924,27sleep30-21swapper/310:38:503
1816726911,25sleep30-21swapper/308:38:533
1407726921,35sleep314098-21ls08:28:233
692726812,28sleep30-21swapper/311:28:433
3018226816,37sleep230277-21cut09:18:242
2167826817,29sleep20-21swapper/207:08:502
213842689,12sleep20-21swapper/210:33:242
1865226813,39sleep318689-21ntp_states12:03:473
1636226821,25sleep20-21swapper/211:58:212
1345426818,22sleep20-21swapper/208:23:542
1309926820,27sleep30-21swapper/310:03:543
84626714,19sleep20-21swapper/211:08:522
729326728,21sleep30-21swapper/309:48:243
288172678,12sleep00-21swapper/012:38:110
2848626722,26sleep20-21swapper/210:53:502
2848626722,26sleep20-21swapper/210:53:502
229012678,38sleep30-21swapper/312:18:333
2285426722,27sleep30-21swapper/307:13:273
2285426722,27sleep30-21swapper/307:13:273
1568626713,31sleep215289-21/usr/sbin/munin10:13:462
1550726721,29sleep315557-21grep11:53:503
4412665,49sleep00-21swapper/009:24:070
349026617,26sleep30-21swapper/307:53:403
317026616,25sleep00-21swapper/007:53:240
317026616,25sleep00-21swapper/007:53:240
3086426616,29sleep030893-21awk11:03:210
2653826617,27sleep326462-21apt-get10:49:233
2653826617,27sleep326462-21apt-get10:49:233
2499726618,28sleep30-21swapper/309:03:233
1433126618,33sleep30-21swapper/308:28:393
1001326515,22sleep00-21swapper/008:13:470
3078326415,31sleep230793-21/usr/sbin/munin09:18:442
2038926418,20sleep20-21swapper/212:08:512
1114826426,21sleep00-21swapper/009:58:500
1033926416,34sleep310367-21munin-run09:58:213
1033926416,34sleep310367-21munin-run09:58:213
886026313,23sleep00-21swapper/011:33:530
886026313,23sleep00-21swapper/011:33:530
3090726315,30sleep330964-21find09:18:493
2726426317,26sleep00-21swapper/009:09:130
2662126318,22sleep30-21swapper/309:08:233
2384126314,33sleep222902-21/usr/sbin/munin12:18:562
1897226316,29sleep018976-21find08:43:340
188126316,24sleep20-21swapper/211:13:402
188126316,24sleep20-21swapper/211:13:402
1463726315,30sleep014722-21apt-config11:53:220
1457426327,21sleep30-21swapper/311:53:213
127892639,10sleep20-21swapper/211:45:172
644826217,28sleep21-21systemd11:28:212
640126213,35sleep36507-21sshd08:03:243
2027026213,30sleep320285-21uname10:28:443
132012625,13sleep20-21swapper/211:48:242
96442618,41sleep20-21swapper/211:38:112
96442618,41sleep20-21swapper/211:38:112
2023426124,20sleep00-21swapper/007:03:520
1871126112,20sleep20-21swapper/212:03:502
1128926118,24sleep20-21swapper/209:58:522
300052607,12sleep30-21swapper/307:33:463
2654026011,25sleep00-21swapper/010:49:240
2654026011,25sleep00-21swapper/010:49:240
241572606,10sleep20-21swapper/210:40:182
19312605,14sleep30-21swapper/307:48:393
19312605,14sleep30-21swapper/307:48:393
383925918,25sleep33841-21ntp_states11:18:503
3237925921,21sleep30-21swapper/309:23:503
3161025916,29sleep331617-21ntpq11:03:483
2968525916,27sleep229694-21forks10:58:432
1665125911,36sleep016657-21cat11:58:310
1584625916,24sleep00-21swapper/008:33:330
81525815,20sleep30-21swapper/307:43:513
56725813,29sleep0586-21uname07:43:440
2662925816,21sleep00-21swapper/012:28:510
1394625815,17sleep00-21swapper/010:08:390
908325715,26sleep09077-21apt-get09:53:380
908325715,26sleep09077-21apt-get09:53:380
443025714,26sleep34425-21apt-get09:38:353
322732579,37sleep30-21swapper/307:40:483
1850425713,17sleep00-21swapper/012:03:440
640125611,17sleep20-21swapper/209:43:472
464625616,24sleep24704-21irqstats09:38:422
2732525613,25sleep227443-21users10:49:422
2732525613,25sleep227443-21users10:49:422
2728225614,24sleep227283-21cron12:33:212
2210625614,25sleep222098-21apt-get08:53:372
2210625614,25sleep222098-21apt-get08:53:372
866325519,17sleep30-21swapper/311:33:483
866325519,17sleep30-21swapper/311:33:483
44052555,11sleep20-21swapper/211:20:142
2785425518,18sleep20-21swapper/207:28:242
2601925512,25sleep326015-21apt-get12:28:343
1066025513,24sleep310727-21df09:58:373
179325414,23sleep01788-21apt-get11:13:340
179325414,23sleep01788-21apt-get11:13:340
1611025414,24sleep30-21swapper/308:33:443
3130325316,19sleep20-21swapper/207:38:412
3003425316,22sleep030070-21ntpq07:33:470
244482539,18sleep30-21swapper/312:23:333
2014925314,23sleep320231-21sensors_temp07:03:513
2368625217,16sleep00-21swapper/010:38:510
2091925112,19sleep20-21swapper/208:48:442
2080625114,16sleep00-21swapper/008:48:420
1697925116,19sleep30-21swapper/310:18:393
123362516,36sleep00-21swapper/008:21:260
123362516,36sleep00-21swapper/008:21:260
34972509,14sleep00-21swapper/007:53:410
281725011,23sleep22336-21apt-get09:33:292
2324925013,22sleep023255-21diskstats07:13:440
2324925013,22sleep023255-21diskstats07:13:440
2291925014,14sleep00-21swapper/012:18:340
1942825011,20sleep319452-21ntp_states08:43:493
49802498,18sleep00-21swapper/007:58:330
3041624914,19sleep00-21swapper/009:18:340
2867424913,18sleep00-21swapper/009:13:250
2160824810,14sleep20-21swapper/212:13:432
1633624816,20sleep216365-21proc_pri08:33:512
2133524711,21sleep021331-21apt-get12:13:330
1893924711,15sleep20-21swapper/210:23:512
1893924711,15sleep20-21swapper/210:23:512
2851124614,17sleep30-21swapper/310:53:513
1709824610,20sleep317130-21ntp_states11:58:463
157124610,15sleep20-21swapper/207:48:222
794724410,16sleep30-21swapper/309:48:483
83682429,16sleep20-21swapper/208:08:442
2164224210,16sleep30-21swapper/310:33:413
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional