You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-16 - 11:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sat May 16, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
203412497512,4949sleep019865-21apt-get22:09:250
4766235706,3552sleep11219-21kworker/1:119:39:141
2169212975,1279sleep122942-1kworker/1:2H22:54:321
3523211496,1130sleep331841-1kworker/3:2H19:34:443
270372594558,22sleep327039-21sshd20:45:143
270372594558,22sleep327039-21sshd20:45:143
11788248089,20sleep10-21swapper/119:59:331
785224516,434sleep1363-21snmpd23:10:511
7767237213,343sleep27773-21cat21:29:382
898923388,255sleep19078-21cat23:14:511
898923388,255sleep19078-21cat23:14:511
308732333107,63sleep130904-21fschecks_count20:59:411
23435225323,213sleep323448-21/usr/sbin/munin20:34:483
27099224891,145sleep127170-21mailstats22:29:491
26452236165,55sleep12708-21modprobe00:39:201
10090222697,59sleep110104-21fschecks_count23:19:391
6132221858,23sleep10-21swapper/121:24:321
11354221214,113sleep111427-21switchtime21:39:511
1753022048,183sleep117519-21grep23:44:211
8194219534,141sleep10-21swapper/119:49:231
2762219011,164sleep12906-21latency_hist21:14:231
2463218915,156sleep12615-21grep19:29:531
459721880,38sleep04602-21unixbench_multi22:59:510
459721880,38sleep04602-21unixbench_multi22:59:510
19945218711,75sleep120029-21grep20:24:401
12330217918,67sleep10-21swapper/121:44:311
17529217616,136sleep317007-21/usr/sbin/munin21:59:473
30841217579,37sleep130940-21latency_hist19:19:231
182902172104,23sleep10-21swapper/122:04:131
24775217017,55sleep10-21swapper/122:24:201
24775217017,55sleep10-21swapper/122:24:201
13560217019,50sleep10-21swapper/123:29:481
1215216516,125sleep21261-21cat21:09:232
2754521629,67sleep127553-21wc19:09:381
26826215814,128sleep126859-21awk00:14:201
11638215729,111sleep111698-21fschecks_time23:24:391
3260215382,20sleep10-21swapper/121:14:431
24105215319,119sleep324133-21fschecks_count00:04:423
963321529,71sleep19680-21ntpq21:34:461
31288215013,123sleep031338-21if_eth019:19:420
16453214714,46sleep10-21swapper/120:14:231
9953214510,123sleep19963-21find19:54:321
3231421435,11sleep10-21swapper/122:46:171
3231421435,11sleep10-21swapper/122:46:171
1337321415,123sleep00-21swapper/023:29:430
10559213718,34sleep00-21swapper/023:20:350
1805321369,44sleep10-21swapper/120:19:231
5921213512,105sleep10-21swapper/123:04:501
5921213512,105sleep10-21swapper/123:04:501
14319213421,88sleep314460-21unixbench_singl20:04:563
13256213414,104sleep313471-21latency_hist20:04:233
536021308,41sleep10-21swapper/119:39:541
26073213022,36sleep00-21swapper/020:44:300
26073213022,36sleep00-21swapper/020:44:300
32627212834,75sleep10-21swapper/119:24:361
2401821278,48sleep124010-21apt-get00:04:361
888621248,33sleep10-21swapper/119:49:481
578021225,38sleep10-21swapper/121:23:271
32048212210,18sleep00-21swapper/000:29:340
22831212214,91sleep322863-21awk23:59:453
19944212110,88sleep120081-21sort23:49:501
25518212011,19sleep00-21swapper/022:24:480
25518212011,19sleep00-21swapper/022:24:480
16717212020,84sleep10-21swapper/121:59:211
27931211910,94sleep127996-21latency20:49:441
18894211810,80sleep10-21swapper/120:19:521
31816211710,91sleep331962-21awk19:19:543
23017211727,73sleep10-21swapper/123:59:511
19905211766,26sleep10-21swapper/122:09:221
30498211635,45sleep00-21swapper/000:24:330
10341211413,81sleep210350-21/usr/sbin/munin19:54:452
547211321,77sleep3591-21ntpq19:24:463
5324211314,81sleep34797-21apt-get19:39:323
28437211311,86sleep228510-21sed22:34:432
28437211311,86sleep228510-21sed22:34:432
21373211214,78sleep020852-21/usr/sbin/munin23:54:500
14923211211,85sleep214992-21sort23:34:422
21792211010,13sleep10-21swapper/120:29:451
21792211010,13sleep10-21swapper/120:29:451
28712210823,66sleep10-21swapper/122:34:501
28712210823,66sleep10-21swapper/122:34:501
25040210816,70sleep10-21swapper/122:24:321
25040210816,70sleep10-21swapper/122:24:321
2328121089,83sleep123424-21latency_hist22:19:231
3166221076,14sleep00-21swapper/022:44:460
3166221076,14sleep00-21swapper/022:44:460
14873210722,69sleep10-21swapper/123:34:411
4263210522,65sleep10-21swapper/122:59:491
4263210522,65sleep10-21swapper/122:59:491
2578521055,89sleep00-21swapper/020:44:010
1912210510,80sleep31935-21/usr/sbin/munin21:09:493
31073210311,36sleep131226-21latency_hist22:44:231
436421027,12sleep20-21swapper/222:59:502
436421027,12sleep20-21swapper/222:59:502
3098821029,78sleep131039-21find00:24:501
21716210231,16sleep10-21swapper/122:14:231
2147210229,28sleep00-21swapper/022:54:310
32259210120,11sleep10-21swapper/119:23:191
2934321017,13sleep30-21swapper/320:54:423
4936210014,68sleep25091-21cat19:39:232
29190210013,13sleep10-21swapper/100:19:421
3015729919,11sleep10-21swapper/120:55:211
2112829918,66sleep10-21swapper/123:54:431
1407829911,65sleep013567-21/usr/sbin/munin20:04:530
360129813,72sleep03622-21cpu19:35:410
264429816,67sleep32666-21pluginstate22:54:513
177772988,14sleep10-21swapper/123:44:331
1591329827,48sleep015939-21basename23:39:200
1533129817,63sleep30-21swapper/320:09:403
1490829818,11sleep10-21swapper/121:51:011
38829710,14sleep10-21swapper/122:49:361
2793129712,69sleep228041-21latency_hist22:34:222
2201729720,60sleep322092-21fschecks_time22:14:413
2415329618,10sleep10-21swapper/120:38:171
323412957,14sleep10-21swapper/121:04:341
320442958,13sleep10-21swapper/100:29:311
2837729518,10sleep10-21swapper/100:16:101
2550329515,10sleep10-21swapper/120:40:231
2716829415,10sleep10-21swapper/120:46:531
2716829415,10sleep10-21swapper/120:46:531
2345129414,19sleep00-21swapper/020:34:490
2230229215,58sleep022347-21/usr/sbin/munin22:14:460
2983329130,29sleep30-21swapper/322:39:403
2398029116,58sleep324002-21/usr/sbin/munin22:19:503
2398029116,58sleep324002-21/usr/sbin/munin22:19:503
744728925,27sleep30-21swapper/321:29:233
686128924,39sleep36858-21apt-get23:09:323
2372828921,50sleep223752-21modprobe00:04:202
421428710,61sleep34221-21ntpq22:59:473
421428710,61sleep34221-21ntpq22:59:473
2689328713,60sleep0369-21sshd19:05:130
2476328616,48sleep2369-21sshd22:24:132
2476328616,48sleep2369-21sshd22:24:132
1906028617,55sleep01-21systemd23:49:210
289652857,13sleep00-21swapper/000:19:360
86742845,12sleep10-21swapper/121:30:541
4662848,49sleep30-21swapper/322:49:383
1830128416,51sleep018316-21modprobe22:04:200
1184228414,34sleep20-21swapper/220:00:102
1184028414,36sleep011786-21apt-get20:00:080
496028312,57sleep05004-21ntpq21:19:470
323128321,36sleep00-21swapper/019:34:240
1090028319,49sleep310975-21tune2fs21:39:403
376527813,49sleep33897-21uptime21:14:533
2213527823,38sleep222155-21cron23:59:202
2106227815,26sleep00-21swapper/020:29:030
139027820,29sleep20-21swapper/200:34:362
135892785,11sleep10-21swapper/120:04:341
1282427816,46sleep212888-21ps21:44:482
250892777,60sleep00-21swapper/000:06:570
916127617,35sleep29302-21unixbench_singl19:49:532
3262127612,50sleep332701-21switchtime00:29:493
2660027628,29sleep30-21swapper/319:04:523
1987627619,42sleep019943-21/usr/sbin/munin23:49:490
1662727619,22sleep00-21swapper/023:39:440
148627619,39sleep00-21swapper/019:29:230
3078927521,27sleep20-21swapper/220:59:382
2805627516,43sleep228178-21timerwakeupswit00:15:062
207922750,6sleep20-21swapper/223:54:222
2193127417,15sleep20-21swapper/220:29:502
2193127417,15sleep20-21swapper/220:29:502
1689127413,46sleep016933-21if_err_eth020:14:420
1143027415,24sleep20-21swapper/221:39:522
894427321,28sleep00-21swapper/021:34:200
878227220,33sleep30-21swapper/323:14:453
878227220,33sleep30-21swapper/323:14:453
68527212,27sleep00-21swapper/022:49:510
3076027220,32sleep330819-21modprobe19:19:213
1882727218,23sleep00-21swapper/020:19:510
1737327216,39sleep317521-21grep20:14:533
1712627213,42sleep217214-21fschecks_time21:59:382
486027111,24sleep20-21swapper/221:19:452
2890627111,47sleep329055-21unixbench_singl22:34:523
2890627111,47sleep329055-21unixbench_singl22:34:523
2782527124,27sleep30-21swapper/320:49:423
198122717,14sleep30-21swapper/323:49:483
1827827122,28sleep20-21swapper/220:19:372
3196727019,27sleep30-21swapper/322:44:533
3196727019,27sleep30-21swapper/322:44:533
2305927017,25sleep20-21swapper/220:34:392
1053327013,28sleep00-21swapper/019:54:510
1999126920,29sleep20-21swapper/222:09:232
931726816,27sleep30-21swapper/321:34:403
761526817,21sleep20-21swapper/223:09:512
2891226822,26sleep20-21swapper/219:14:232
2805026817,29sleep30-21swapper/319:09:523
165326817,37sleep01666-21irqstats00:34:470
259352678,13sleep00-21swapper/000:09:430
2043126717,23sleep20-21swapper/222:10:122
1876126720,32sleep318824-21/usr/sbin/munin20:19:503
159626713,15sleep30-21swapper/300:34:463
1133926721,20sleep30-21swapper/319:59:233
3105526616,28sleep00-21swapper/022:44:220
2863426621,21sleep00-21swapper/000:19:200
2726726619,28sleep127285-21munin-run19:09:221
261826615,31sleep20-21swapper/219:29:542
2559926617,32sleep325669-21/usr/sbin/munin22:24:513
2559926617,32sleep325669-21/usr/sbin/munin22:24:513
1448026618,27sleep30-21swapper/321:49:503
907926518,20sleep20-21swapper/223:14:522
907926518,20sleep20-21swapper/223:14:522
1202626512,20sleep20-21swapper/223:24:492
668126416,27sleep20-21swapper/219:44:242
426726418,28sleep2225-21systemd-journal21:19:202
2733426420,28sleep327351-21apt00:14:313
2686226416,29sleep226896-21munin-run00:14:212
193726411,22sleep00-21swapper/021:09:500
1421226419,23sleep20-21swapper/220:04:552
1073326422,19sleep30-21swapper/323:20:463
2781426315,33sleep027820-21uname19:09:450
115026314,29sleep01193-21munin-run21:09:210
3187026212,37sleep231966-21sed22:44:522
3187026212,37sleep231966-21sed22:44:522
2931626216,30sleep329324-21needreboot00:19:443
2898726222,20sleep00-21swapper/020:54:230
2895326212,30sleep228947-21apt-get00:19:342
2812226211,35sleep228136-21awk20:49:492
2565226221,22sleep30-21swapper/300:09:383
1632626216,22sleep30-21swapper/323:39:393
1572126216,28sleep015818-21threads20:09:500
1474026219,24sleep30-21swapper/323:34:383
1361926221,21sleep30-21swapper/321:49:213
890226114,33sleep08988-21/usr/sbin/munin23:14:500
890226114,33sleep08988-21/usr/sbin/munin23:14:500
33926118,28sleep3383-21grep21:04:483
2210626119,19sleep00-21swapper/020:29:520
2210626119,19sleep00-21swapper/020:29:520
1436226115,27sleep014368-21ntp_states21:49:470
1086026114,27sleep010885-21/usr/sbin/munin21:39:370
657526019,26sleep36604-21munin-run23:09:203
657526019,26sleep36604-21munin-run23:09:203
529126017,27sleep25310-21apt-get23:04:302
529126017,27sleep25310-21apt-get23:04:302
2452726011,27sleep20-21swapper/220:39:312
2246126016,27sleep022454-21apt-get23:59:340
900425917,19sleep30-21swapper/321:34:233
3172325915,24sleep20-21swapper/219:19:532
3128925920,19sleep30-21swapper/320:59:513
923025816,26sleep29224-21apt-get21:34:352
750025812,30sleep27623-21timerwakeupswit19:44:522
656425814,27sleep26588-21ntp_states21:24:452
60425816,28sleep2664-21/usr/sbin/munin22:49:492
3001625818,17sleep20-21swapper/222:39:442
2964025813,24sleep00-21swapper/019:14:420
2021625811,11sleep20-21swapper/220:24:452
969825717,25sleep09758-21/usr/sbin/munin21:34:490
743825712,29sleep07491-21sort23:09:490
304722577,10sleep20-21swapper/219:15:452
2320025718,24sleep223224-21munin-run22:19:202
2167925715,24sleep321777-21users23:54:533
1985125720,22sleep00-21swapper/022:09:210
1754325713,19sleep20-21swapper/223:44:222
1511725715,25sleep215136-21apt-get20:09:312
974125616,20sleep00-21swapper/019:54:230
3111425614,27sleep331222-21timerandwakeup00:24:523
2637925613,16sleep20-21swapper/219:04:472
1296225614,18sleep00-21swapper/021:44:500
1208025618,18sleep30-21swapper/323:24:503
734325519,18sleep30-21swapper/319:44:493
2910525515,21sleep00-21swapper/022:35:130
2910525515,21sleep00-21swapper/022:35:130
2736425512,17sleep00-21swapper/022:29:520
2688625516,22sleep326979-21memory22:29:433
2273825516,15sleep30-21swapper/320:34:233
2273825516,15sleep30-21swapper/320:34:233
1714625516,23sleep217152-21ntp_states20:14:482
1679925515,17sleep00-21swapper/021:59:240
1441725513,20sleep30-21swapper/323:34:223
584925411,29sleep05920-21sensors23:04:490
584925411,29sleep05920-21sensors23:04:490
3212125413,25sleep232127-21df_inode00:29:362
2921025415,24sleep029204-21apt-get20:54:350
1930225411,17sleep30-21swapper/322:04:523
1325125415,23sleep213321-21idleruntime23:29:412
78132536,37sleep00-21swapper/019:45:100
629425313,27sleep06295-21sshd19:40:300
2455425312,24sleep324622-21df20:39:363
1786425314,24sleep217908-21grep23:44:392
166732535,11sleep20-21swapper/221:58:492
1516825313,23sleep215198-21munin-run21:54:212
145125312,24sleep21445-21apt-get21:09:352
613625213,18sleep00-21swapper/021:24:350
531425214,16sleep30-21swapper/321:19:543
3050025212,23sleep230494-21apt-get00:24:352
2482825214,16sleep00-21swapper/022:24:230
2482825214,16sleep00-21swapper/022:24:230
1990225213,16sleep30-21swapper/320:24:383
1545325212,24sleep315449-21apt-get21:54:343
2929825115,22sleep229342-21grep20:54:412
2660425115,16sleep20-21swapper/220:44:482
2660425115,16sleep20-21swapper/220:44:482
2037325112,24sleep320370-21apt-get22:09:323
313625014,23sleep33228-21sed19:34:233
3078725012,22sleep030782-21apt-get20:59:360
435124914,17sleep00-21swapper/021:19:220
282524915,18sleep20-21swapper/222:54:542
1200624912,17sleep00-21swapper/023:24:470
1051824911,23sleep210527-21ntpq23:20:332
250792479,20sleep225138-21df22:24:372
250792479,20sleep225138-21df22:24:372
1903924710,20sleep30-21swapper/323:49:203
2459524514,16sleep20-21swapper/200:04:532
93052417,13sleep30-21swapper/319:49:543
143323911,15sleep20-21swapper/219:29:212
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional