You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-25 - 11:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Thu Dec 25, 2025 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
318432185314,1823sleep231706-21kworker/2:021:49:352
1610263912,612sleep01654-21ntpq23:39:430
5953239419,362sleep16032-21fschecks_time23:54:381
936623277,305sleep19403-21ntpq00:04:431
12614223744,22sleep10-21swapper/120:49:191
1001223023,188sleep11076-21latency_hist23:39:191
29544222310,200sleep029623-21systemctl23:24:490
16392221423,40sleep10-21swapper/122:44:331
27387221145,38sleep127424-21latency21:34:411
10300220822,169sleep210278-21latency_hist00:09:202
15788220739,148sleep10-21swapper/120:59:201
17420220431,148sleep10-21swapper/100:29:501
17420220431,148sleep10-21swapper/100:29:501
6797220140,148sleep16829-21tune2fs20:29:411
58921960,35sleep1594-21munin-plugin-st21:54:181
21770219414,168sleep121855-21cut22:59:471
16295219015,156sleep116327-21latency20:59:401
19623218512,118sleep119657-21latency_hist00:39:191
9945218015,150sleep29968-21latency_hist22:24:192
13592217814,22sleep10-21swapper/120:49:551
13592217814,22sleep10-21swapper/120:49:551
9109217613,149sleep19153-21processes22:19:461
20551217211,146sleep020596-21grep19:29:320
5350216714,63sleep10-21swapper/120:24:401
15122216617,78sleep10-21swapper/122:39:411
13483216060,38sleep113245-21/usr/sbin/munin22:34:401
11843215910,93sleep111965-21latency_hist00:14:201
16467215411,125sleep116492-21munin-run00:29:171
7454215115,114sleep00-21swapper/023:59:370
2308621449,13sleep10-21swapper/121:19:471
21374214312,51sleep10-21swapper/121:15:031
21374214312,51sleep10-21swapper/121:15:031
1432421438,20sleep10-21swapper/119:11:081
958214214,76sleep10-21swapper/120:09:471
958214214,76sleep10-21swapper/120:09:471
17743214229,56sleep10-21swapper/119:19:441
28050213913,68sleep128095-21latency_hist19:54:201
20536213813,102sleep020747-21date21:14:210
26878213714,108sleep127022-21cat21:34:201
18759213712,109sleep218766-21open_files00:34:432
3327212815,55sleep10-21swapper/121:59:521
855212314,93sleep30-21swapper/321:54:333
2701212324,75sleep32766-21grep21:59:423
1754821228,16sleep00-21swapper/019:19:390
10343212111,95sleep310497-21latency_hist00:09:213
1298621200,10sleep012981-21cron22:34:180
1298621200,10sleep012981-21cron22:34:180
32318211620,76sleep331746-21/usr/sbin/munin21:49:493
31812211661,25sleep10-21swapper/120:04:461
24373211311,43sleep10-21swapper/123:09:321
24373211311,43sleep10-21swapper/123:09:321
23638211342,55sleep123657-21apt-get19:39:311
17574211335,16sleep10-21swapper/121:04:331
26457211224,63sleep326488-21munin-run19:49:183
30291210945,51sleep10-21swapper/123:29:181
25977210911,14sleep00-21swapper/019:44:510
11448210821,67sleep0369-21snmpd20:44:380
917210725,55sleep3927-21modprobe23:39:173
31120210712,81sleep231164-21systemd-run20:04:182
22552210710,44sleep122561-21awk19:34:461
22552210710,44sleep122561-21awk19:34:461
15986210729,57sleep10-21swapper/119:14:401
14530210715,73sleep214547-21sh22:39:172
19688210436,51sleep10-21swapper/122:54:301
3950210337,48sleep10-21swapper/122:04:281
19475210328,51sleep30-21swapper/322:54:203
1912821039,37sleep10-21swapper/121:09:341
31281210237,51sleep10-21swapper/121:46:401
12085210237,45sleep10-21swapper/100:14:361
12085210237,45sleep10-21swapper/100:14:361
598121019,80sleep16015-21ps22:09:451
23168210122,59sleep223314-21unixbench_multi21:19:482
22806210120,55sleep322825-21apt-get23:04:293
21046210118,62sleep321075-21latency_hist22:59:213
2094210137,47sleep10-21swapper/120:14:351
19130210136,49sleep10-21swapper/119:24:371
8720210016,32sleep38726-21diskstats22:19:363
13371210027,55sleep113424-21gpgv19:09:191
13371210027,55sleep113424-21gpgv19:09:191
892729911,25sleep00-21swapper/022:19:410
30542996,79sleep10-21swapper/123:44:401
2403129913,71sleep10-21swapper/123:08:351
1075529953,26sleep10-21swapper/122:24:471
1037029821,19sleep10-21swapper/120:39:461
525129723,53sleep35268-21fschecks_count20:24:383
296529713,68sleep33027-21cat23:44:383
2043629720,56sleep320530-21sort21:14:203
1801529733,48sleep10-21swapper/122:49:211
2777729514,67sleep00-21swapper/019:51:210
144562957,11sleep30-21swapper/320:54:343
1303029558,25sleep113029-21/usr/sbin/munin22:34:201
1303029558,25sleep113029-21/usr/sbin/munin22:34:201
165629432,48sleep10-21swapper/123:39:441
2967929332,12sleep10-21swapper/121:40:541
225452939,70sleep00-21swapper/019:34:450
225452939,70sleep00-21swapper/019:34:450
918429231,12sleep10-21swapper/120:35:101
3155829231,12sleep10-21swapper/123:30:441
287972929,67sleep128890-21mailstats19:54:491
287972929,67sleep128890-21mailstats19:54:491
2767329217,58sleep127724-21idleruntime23:19:411
193529234,10sleep10-21swapper/121:56:571
184692925,13sleep30-21swapper/321:04:513
1257129213,24sleep20-21swapper/200:14:492
1257129213,24sleep20-21swapper/200:14:492
848829131,13sleep10-21swapper/100:01:281
2615829132,11sleep10-21swapper/119:45:241
1460029131,12sleep10-21swapper/100:20:051
1054829112,51sleep30-21swapper/322:24:413
803129032,11sleep10-21swapper/122:15:341
803129032,11sleep10-21swapper/122:15:341
524029031,11sleep10-21swapper/123:49:531
2129929013,62sleep321329-21awk21:15:013
2129929013,62sleep321329-21awk21:15:013
2017628913,58sleep220198-21/usr/sbin/munin22:54:472
720628818,48sleep37141-21latency_hist23:59:203
3103728831,11sleep10-21swapper/120:03:121
2525228831,11sleep10-21swapper/121:28:511
2162828831,11sleep10-21swapper/119:32:001
268462878,60sleep30-21swapper/319:49:383
2684628730,12sleep10-21swapper/123:15:051
52312869,15sleep30-21swapper/322:09:183
2375228622,36sleep223791-21fschecks_count19:39:402
1871928625,38sleep20-21swapper/219:24:182
467328528,10sleep10-21swapper/120:21:371
74502848,16sleep30-21swapper/323:59:363
1148828425,28sleep20-21swapper/222:29:202
840728328,28sleep30-21swapper/322:19:203
840728328,28sleep30-21swapper/322:19:203
2971728316,51sleep229843-21/usr/sbin/munin23:24:512
2449428314,50sleep024005-21/usr/sbin/munin21:24:470
3153328220,37sleep00-21swapper/021:49:210
1766228221,45sleep217730-21fschecks_time21:04:382
214028113,52sleep02203-21fschecks_time20:14:380
1715128121,25sleep30-21swapper/300:29:443
1715128121,25sleep30-21swapper/300:29:443
80122809,43sleep00-21swapper/020:34:200
2185628013,46sleep221926-21/usr/sbin/munin22:59:482
2836927920,43sleep228400-21idleruntime-cro21:39:182
2445627910,53sleep224485-21ntp_states21:24:452
2225127914,49sleep222347-21latency_hist21:19:202
2225127914,49sleep222347-21latency_hist21:19:202
1535927915,48sleep215424-21/usr/sbin/munin00:24:382
374127814,18sleep20-21swapper/222:04:212
3096227814,45sleep031007-21grep23:29:460
2913927817,27sleep30-21swapper/323:24:403
1278527824,23sleep20-21swapper/219:04:482
1278527824,23sleep20-21swapper/219:04:482
2285527714,47sleep322884-21nfs4_client21:19:413
1424727713,47sleep314293-21/usr/sbin/munin19:11:063
1220027731,26sleep30-21swapper/319:04:373
1220027731,26sleep30-21swapper/319:04:373
1185627717,42sleep211918-21/usr/sbin/munin20:44:562
230162768,49sleep00-21swapper/023:04:390
562727521,22sleep30-21swapper/323:54:193
342027517,42sleep33504-21latency_hist20:19:213
2718727512,46sleep0372-21sshd21:34:360
2192727520,40sleep322060-21wc22:59:493
1312327418,34sleep30-21swapper/320:49:453
1312327418,34sleep30-21swapper/320:49:453
1211127417,27sleep00-21swapper/019:04:300
1211127417,27sleep00-21swapper/019:04:300
1173327418,40sleep311803-21/usr/sbin/munin22:29:373
1173327418,40sleep311803-21/usr/sbin/munin22:29:373
559627310,46sleep20-21swapper/220:24:452
1733527316,44sleep017419-21cat00:29:490
1733527316,44sleep017419-21cat00:29:490
1217027324,26sleep00-21swapper/000:14:390
1217027324,26sleep00-21swapper/000:14:390
2880027218,26sleep30-21swapper/323:24:203
1770427217,37sleep317796-21apt-config22:49:193
1334327214,24sleep30-21swapper/300:19:183
1334327214,24sleep30-21swapper/300:19:183
1303127123,34sleep013040-21uniq20:49:420
1303127123,34sleep013040-21uniq20:49:420
2567527013,21sleep00-21swapper/021:29:390
2116527022,26sleep30-21swapper/319:29:483
161872707,55sleep312735-21kworker/3:200:25:403
717326818,26sleep20-21swapper/220:29:502
2254426810,39sleep022568-21munin-run23:04:180
1615426817,23sleep30-21swapper/320:59:383
908426718,24sleep00-21swapper/000:04:380
648126717,23sleep00-21swapper/023:54:490
3152126719,35sleep231530-21unin-run21:49:192
1817426717,18sleep20-21swapper/222:49:352
1337926721,28sleep213419-21grep20:49:522
1337926721,28sleep213419-21grep20:49:522
2760026620,28sleep00-21swapper/023:19:400
2434926617,28sleep324368-21apt-get23:09:303
2434926617,28sleep324368-21apt-get23:09:303
1610226619,21sleep20-21swapper/222:44:182
133426612,21sleep00-21swapper/021:54:460
1145326610,13sleep30-21swapper/300:09:543
560326514,34sleep05634-21fschecks_count22:09:360
458126522,27sleep00-21swapper/022:04:470
3002426521,19sleep00-21swapper/021:44:210
2619026519,29sleep20-21swapper/223:14:422
2269626518,21sleep00-21swapper/021:19:380
1846826525,20sleep00-21swapper/022:50:010
1628126517,19sleep20-21swapper/219:14:452
1524126519,20sleep20-21swapper/219:14:202
1143926514,30sleep20-21swapper/200:09:532
89926418,24sleep30-21swapper/320:09:443
89926418,24sleep30-21swapper/320:09:443
219326419,21sleep20-21swapper/221:59:202
218032648,13sleep20-21swapper/219:34:192
2107826416,22sleep00-21swapper/022:59:220
2026226416,25sleep30-21swapper/319:29:193
1519026419,21sleep00-21swapper/019:14:180
1457026418,29sleep014676-21tune2fs20:54:510
1181726418,23sleep20-21swapper/222:29:402
1181726418,23sleep20-21swapper/222:29:402
1008726412,32sleep29792-21/usr/sbin/munin20:39:392
3271526316,20sleep30-21swapper/320:09:203
1623726315,20sleep00-21swapper/020:59:390
3058326217,26sleep230584-21sshd23:29:352
132226215,31sleep21326-21diskstats23:39:372
577726123,25sleep35790-21irqstats22:09:393
281962616,11sleep30-21swapper/321:37:013
2712626117,20sleep00-21swapper/023:18:350
1898126118,27sleep018985-21find19:24:280
193292608,39sleep0369-21snmpd00:36:230
116632607,18sleep00-21swapper/000:12:370
902825915,26sleep29150-21users20:34:512
468725922,16sleep20-21swapper/223:49:422
444425912,28sleep24505-21processes22:04:452
408525917,23sleep00-21swapper/023:49:200
3151125913,22sleep00-21swapper/020:04:410
2555025912,28sleep325567-21/usr/sbin/munin21:29:303
2346225916,22sleep30-21swapper/319:39:203
94925816,24sleep00-21swapper/020:09:450
94925816,24sleep00-21swapper/020:09:450
56625819,26sleep0587-21latency_hist21:54:180
492725814,28sleep20-21swapper/220:24:212
271022585,41sleep30-21swapper/323:18:183
2651125817,15sleep20-21swapper/219:49:212
2217125816,27sleep022214-21munin-run21:19:180
2217125816,27sleep022214-21munin-run21:19:180
1830025813,28sleep318370-21/usr/sbin/munin00:34:323
1611225824,18sleep20-21swapper/220:59:372
1491525817,18sleep20-21swapper/220:54:572
96725717,21sleep20-21swapper/220:09:482
96725717,21sleep20-21swapper/220:09:482
824625717,24sleep38243-21apt-get20:34:313
3243025712,25sleep20-21swapper/220:06:162
3013325715,22sleep30-21swapper/319:59:433
254902576,39sleep00-21swapper/023:12:300
254902576,39sleep00-21swapper/023:12:300
1615325716,24sleep016255-21sed22:44:200
1560825712,18sleep00-21swapper/019:14:260
1373725710,33sleep213744-21df_inode00:19:332
693925618,22sleep27001-21sed22:14:222
2446825615,23sleep324567-21users19:39:533
2416625616,24sleep024245-21awk19:39:500
1366725612,26sleep313663-21apt-get00:19:303
1191925614,24sleep30-21swapper/320:44:573
646225514,22sleep00-21swapper/020:29:190
33862555,34sleep20-21swapper/220:19:202
3221825510,29sleep032314-21fschecks_time23:34:380
3060525517,21sleep030749-21sed19:59:520
432325413,25sleep04318-21apt-get23:49:320
198125412,23sleep30-21swapper/323:39:503
1521125413,24sleep015205-21apt-get00:24:320
990225312,17sleep30-21swapper/320:39:363
92512535,36sleep00-21swapper/020:36:020
69912535,11sleep20-21swapper/223:57:372
2943825310,31sleep0369-21snmpd19:57:550
2943825310,31sleep0369-21snmpd19:57:550
2757925311,17sleep20-21swapper/223:19:382
275725313,23sleep22775-21apt-get23:44:292
2216125313,24sleep322172-21df_inode19:34:363
1795625312,17sleep30-21swapper/319:19:503
998225216,21sleep00-21swapper/020:39:370
774225216,18sleep20-21swapper/222:14:512
774225216,18sleep20-21swapper/222:14:512
411225216,15sleep30-21swapper/320:19:503
2933725216,18sleep30-21swapper/319:56:363
2933725216,18sleep30-21swapper/319:56:363
237572528,33sleep20-21swapper/223:05:012
1355625213,24sleep313646-21memory22:34:423
1327225211,17sleep00-21swapper/022:34:350
2868925115,21sleep328759-21df21:39:353
286622519,23sleep228683-21dpkg21:39:302
267472519,20sleep20-21swapper/219:49:332
2411725116,21sleep30-21swapper/321:24:373
1780525111,22sleep017662-21latency_hist22:49:200
1687725111,14sleep20-21swapper/200:29:392
1687725111,14sleep20-21swapper/200:29:392
1503325117,16sleep30-21swapper/322:39:383
113592515,35sleep00-21swapper/022:28:170
744925012,23sleep27445-21apt-get23:59:352
362525015,18sleep00-21swapper/020:19:340
240625010,22sleep02425-21apt-get21:59:310
1921525011,18sleep20-21swapper/219:24:382
1889025015,18sleep00-21swapper/021:09:200
678124912,21sleep36788-21df_inode20:29:393
2208724910,16sleep20-21swapper/219:34:312
2208724910,16sleep20-21swapper/219:34:312
158612499,15sleep30-21swapper/319:14:303
2557524813,20sleep225571-21apt-get21:29:332
2038124814,21sleep21-21systemd21:14:192
1764424813,17sleep20-21swapper/222:49:182
3068024710,14sleep30-21swapper/321:44:443
706324514,17sleep00-21swapper/022:14:320
706324514,17sleep00-21swapper/022:14:320
2829024411,17sleep20-21swapper/219:54:332
2829024411,17sleep20-21swapper/219:54:332
321202439,17sleep20-21swapper/223:34:322
3067724312,15sleep30-21swapper/323:29:413
243742439,16sleep20-21swapper/223:09:332
243742439,16sleep20-21swapper/223:09:332
1366924311,16sleep00-21swapper/000:19:310
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional