You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-29 - 05:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sun Mar 29, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2015121115615,11120sleep022005-21kworker/u8:221:52:450
51842655227,6513sleep35167-21kworker/3:221:05:073
14122297413,925sleep314133-21ntpq21:32:453
32339283220,794sleep232389-21/usr/sbin/munin19:07:342
53526568,613sleep3584-21ntpq22:32:433
18686259513,566sleep218699-21/usr/sbin/munin23:27:512
18897253416,457sleep318925-21idleruntime-cro20:07:203
18897253416,457sleep318925-21idleruntime-cro20:07:203
939240721,366sleep1933-21apt-get00:17:341
12956237515,344sleep213005-21/usr/sbin/munin19:47:362
5143232121,212sleep30-21swapper/322:47:413
10081229944,214sleep310118-21unin-run21:22:193
16761224236,117sleep316916-21unixbench_multi19:57:513
25745223919,197sleep025826-21fschecks_time20:27:450
25745223919,197sleep025826-21fschecks_time20:27:450
24816223818,202sleep024807-21apt-get22:07:360
3858223441,19sleep30-21swapper/321:02:203
14968223310,131sleep315026-21sed23:17:423
9541223054,99sleep39521-21munin-run23:02:193
9541223054,99sleep39521-21munin-run23:02:193
12891222447,108sleep313034-21latency_hist23:12:213
25927222035,76sleep30-21swapper/320:27:483
25927222035,76sleep30-21swapper/320:27:483
30146221231,93sleep30-21swapper/322:22:493
3101221033,134sleep30-21swapper/300:22:493
7295220926,163sleep17356-21fschecks_time21:12:401
17721220725,157sleep117801-21fschecks_time20:02:381
17721220725,157sleep117801-21fschecks_time20:02:381
2538922069,136sleep324788-21/usr/sbin/munin22:07:513
28963220131,85sleep30-21swapper/320:37:423
14379219710,119sleep314441-21sendmail-msp23:17:183
32083219618,158sleep132146-21latency_hist19:07:211
24452218786,56sleep30-21swapper/323:47:433
18714217736,127sleep318320-21/usr/sbin/munin21:47:433
31505217630,88sleep331568-21processes22:27:463
5669216817,65sleep35703-21fschecks_count00:32:393
19099216870,46sleep30-21swapper/323:32:203
30517216724,126sleep130530-21uname00:07:401
2642921628,31sleep30-21swapper/322:12:353
26936215413,96sleep326882-40mandb23:57:203
26936215413,96sleep326882-40mandb23:57:203
4719215267,63sleep30-21swapper/300:27:503
2996521508,104sleep31-21systemd00:07:183
2996521508,104sleep31-21systemd00:07:183
27455214729,106sleep327469-21grep20:32:413
32679214321,100sleep132721-21sort22:32:371
7088214211,68sleep37236-21grep22:52:503
12032214115,106sleep012067-21proc_pri23:07:460
134321408,99sleep31390-21ntpq00:17:443
2094121398,106sleep30-21swapper/320:12:423
2326821388,112sleep00-21swapper/022:02:350
9251213722,99sleep29257-21ntp_states21:17:502
1905213714,98sleep21998-21fschecks_time19:12:392
495221364,12sleep20-21swapper/200:27:532
20100213517,75sleep30-21swapper/321:52:283
406121338,22sleep00-21swapper/000:27:350
11891213314,101sleep032020-21kworker/u8:321:27:210
6959212911,59sleep36993-21ps19:27:463
11601212731,16sleep30-21swapper/319:42:453
3090021269,104sleep330291-21/usr/sbin/munin20:42:503
3090021269,104sleep330291-21/usr/sbin/munin20:42:503
14356212611,98sleep014492-21unixbench_multi21:32:500
6765212413,95sleep16781-21uname19:27:401
30717212439,53sleep330751-21ps00:07:473
3073212328,81sleep33177-21date22:42:203
5929212226,61sleep35984-21cat21:07:413
10971212129,80sleep311012-21ntpq21:22:553
17166211910,93sleep317211-21awk21:42:433
17166211910,93sleep317211-21awk21:42:433
12674211932,45sleep30-21swapper/321:27:483
1448211814,25sleep00-21swapper/000:17:490
16217211711,88sleep116236-21apt-get23:22:291
1132521179,88sleep311452-21latency_hist23:07:203
29348211613,90sleep029425-21switchtime20:37:510
29687211516,77sleep10-21swapper/122:22:391
13369211530,42sleep30-21swapper/319:47:583
4361211418,29sleep30-21swapper/319:18:293
28026211434,18sleep30-21swapper/322:17:363
230221138,70sleep32366-21sendmail_mailst19:12:483
10037211226,39sleep30-21swapper/319:37:443
368211118,30sleep30-21swapper/320:48:283
368211118,30sleep30-21swapper/320:48:283
30474211119,74sleep130291-21/usr/sbin/munin20:42:401
30474211119,74sleep130291-21/usr/sbin/munin20:42:401
2890211019,76sleep12891-21sshd22:40:371
28859210914,78sleep128869-21forks20:37:391
7718210813,57sleep37808-21mailstats21:12:503
23026210830,57sleep123075-21latency_hist22:02:201
23026210830,57sleep123075-21latency_hist22:02:201
21792210812,82sleep30-21swapper/321:57:373
21792210812,82sleep30-21swapper/321:57:373
822821079,16sleep30-21swapper/319:32:393
27028210711,80sleep127144-21sed22:12:501
1965921067,64sleep30-21swapper/320:09:063
6489210319,68sleep16497-21vmstat00:32:531
1572721039,76sleep00-21swapper/021:37:470
24690210020,60sleep024828-21grep23:47:480
94129917,15sleep30-21swapper/320:52:313
1493529822,38sleep30-21swapper/319:52:463
2588029719,58sleep20-21swapper/223:52:432
2588029719,58sleep20-21swapper/223:52:432
467729310,70sleep14760-21switchtime21:02:511
251672939,66sleep325143-21munin-run20:27:203
251672939,66sleep325143-21munin-run20:27:203
2454129327,43sleep224689-21grep20:22:512
2454129327,43sleep224689-21grep20:22:512
1997029121,49sleep219032-21/usr/sbin/munin20:09:132
2800929014,15sleep30-21swapper/323:57:533
2800929014,15sleep30-21swapper/323:57:533
1650828914,30sleep3222-21systemd-journal21:42:183
2063928813,18sleep10-21swapper/121:53:071
312862866,11sleep20-21swapper/220:43:382
312862866,11sleep20-21swapper/220:43:382
223092868,37sleep30-21swapper/320:17:303
207322867,18sleep20-21swapper/220:12:372
1162928620,50sleep111653-21/usr/sbin/munin19:42:471
2887028519,46sleep228958-21fschecks_time20:37:402
167592856,41sleep316844-21mailstats23:22:473
1125128519,48sleep011289-21grep19:42:380
863328410,21sleep10-21swapper/122:58:181
863328410,21sleep10-21swapper/122:58:181
324182848,36sleep30-21swapper/300:12:513
3214828416,48sleep232037-21latency_hist19:07:222
3201328419,49sleep032082-21unin-run19:07:200
285228419,33sleep00-21swapper/000:22:420
2373028414,52sleep123753-21/usr/sbin/munin22:02:481
1496728416,16sleep10-21swapper/119:52:481
1177328419,16sleep10-21swapper/123:07:391
874828327,37sleep18901-21tail19:32:571
2910928215,49sleep129154-21/usr/sbin/munin00:02:441
2910928215,49sleep129154-21/usr/sbin/munin00:02:441
105828217,32sleep30-21swapper/319:11:143
691728115,50sleep06943-21munin-run21:12:180
2727128116,45sleep127284-21fschecks_count23:57:391
2727128116,45sleep127284-21fschecks_count23:57:391
2462028116,48sleep224770-21cut22:07:212
1552128115,52sleep215606-21latency21:37:412
58422807,14sleep10-21swapper/121:07:391
272202799,52sleep20-21swapper/223:57:362
272202799,52sleep20-21swapper/223:57:362
2304727925,30sleep123192-21grep20:17:531
2221527921,42sleep122240-21latency_hist23:42:201
2221527921,42sleep122240-21latency_hist23:42:201
2002727910,57sleep220093-21store21:52:232
684327825,36sleep26855-21modprobe00:37:182
3171927820,39sleep231869-21sed22:27:492
252132780,9sleep20-21swapper/222:07:452
1501227817,45sleep215121-21latency_hist21:37:202
866027713,46sleep08745-21timerandwakeup19:32:560
419027716,27sleep10-21swapper/119:17:511
3250027716,30sleep00-21swapper/000:12:520
3198727714,50sleep132041-21cat00:12:411
3022827715,40sleep230229-21/usr/sbin/munin00:07:292
2693627720,26sleep00-21swapper/022:12:490
232352775,35sleep30-21swapper/322:02:263
2044027726,33sleep020464-21modprobe20:12:180
1328927713,50sleep013346-21idleruntime23:12:390
775527617,45sleep07803-21systemd-run19:32:200
457027620,39sleep04580-21modprobe22:47:180
3096227626,32sleep331020-21/usr/sbin/munin19:02:393
224362767,32sleep30-21swapper/323:42:303
224362767,32sleep30-21swapper/323:42:303
2168427615,46sleep021703-21apt-get21:57:300
2168427615,46sleep021703-21apt-get21:57:300
198332769,15sleep20-21swapper/223:32:482
177292760,7sleep20-21swapper/223:27:222
692227517,38sleep06983-21latency_hist00:37:200
1972727516,38sleep019869-21latency_hist21:52:210
1815227514,41sleep118160-21/usr/sbin/munin23:27:421
1026427524,25sleep10-21swapper/121:22:211
948527419,23sleep20-21swapper/223:02:182
948527419,23sleep20-21swapper/223:02:182
855627419,23sleep20-21swapper/221:17:212
218482746,28sleep30-21swapper/323:38:103
218482746,28sleep30-21swapper/323:38:103
189482737,51sleep20-21swapper/220:07:242
189482737,51sleep20-21swapper/220:07:242
476127219,31sleep24883-21timerwakeupswit21:02:522
224927222,26sleep20-21swapper/222:37:472
3001027119,27sleep00-21swapper/000:07:190
3001027119,27sleep00-21swapper/000:07:190
2831227116,33sleep00-21swapper/022:17:410
2186127110,49sleep00-21swapper/020:15:330
2582827017,24sleep10-21swapper/123:52:421
2582827017,24sleep10-21swapper/123:52:421
1810827021,21sleep10-21swapper/121:47:211
1810827021,21sleep10-21swapper/121:47:211
1787327018,21sleep00-21swapper/023:27:370
1414227012,15sleep10-21swapper/121:32:471
1120927015,42sleep111335-21cat21:23:021
104927014,41sleep21058-21forks20:52:382
131672697,11sleep10-21swapper/121:29:321
462026821,20sleep20-21swapper/222:47:202
2563226816,33sleep125132-21apt-get20:27:301
2563226816,33sleep125132-21apt-get20:27:301
2088726813,37sleep220905-21/usr/sbin/munin23:37:282
2088726813,37sleep220905-21/usr/sbin/munin23:37:282
1960226811,25sleep10-21swapper/121:52:181
1941626820,32sleep30-21swapper/323:32:373
520026720,32sleep30-21swapper/319:22:413
495126721,25sleep10-21swapper/119:22:351
35492676,11sleep00-21swapper/020:58:580
319626712,39sleep03110-21latency_hist19:17:210
785526620,25sleep20-21swapper/219:32:222
42442665,28sleep30-21swapper/322:43:093
1942326612,20sleep00-21swapper/023:32:380
1275126617,30sleep00-21swapper/021:27:490
8542656,11sleep00-21swapper/019:08:370
847626521,23sleep00-21swapper/021:17:190
680926516,35sleep16854-21/usr/sbin/munin22:52:441
2897526529,16sleep20-21swapper/200:02:412
2897526529,16sleep20-21swapper/200:02:412
1467326518,32sleep014692-21apt-get23:17:310
100432650,10sleep20-21swapper/223:02:342
873326414,35sleep08747-21awk22:58:210
873326414,35sleep08747-21awk22:58:210
513026412,34sleep25142-21/usr/sbin/munin22:47:402
340326416,26sleep00-21swapper/019:17:330
2946726421,27sleep029486-21apt-get22:22:300
2794026416,29sleep227934-21apt-get22:17:312
2451826411,39sleep224610-21mailstats23:47:462
2377226422,19sleep10-21swapper/123:47:211
2377226422,19sleep10-21swapper/123:47:211
211202648,19sleep00-21swapper/023:37:380
211202648,19sleep00-21swapper/023:37:380
1790526414,36sleep017964-21latency20:02:410
1790526414,36sleep017964-21latency20:02:410
152726414,32sleep01656-21latency_hist22:37:200
97526311,31sleep20-21swapper/222:32:502
930226314,22sleep00-21swapper/021:17:520
534626315,30sleep15455-21latency_hist00:32:201
3040526316,22sleep00-21swapper/020:42:390
3040526316,22sleep00-21swapper/020:42:390
606326216,26sleep20-21swapper/221:07:442
377426219,22sleep10-21swapper/122:42:471
2141426221,19sleep10-21swapper/120:12:521
161726213,35sleep21731-21sed00:17:512
154026220,17sleep10-21swapper/120:52:501
375526120,19sleep10-21swapper/121:01:381
1734426122,25sleep20-21swapper/220:02:192
1622726125,19sleep00-21swapper/019:57:400
1323326118,21sleep00-21swapper/019:47:550
3261026015,19sleep10-21swapper/119:07:411
2765726023,20sleep10-21swapper/120:32:471
2747126017,18sleep00-21swapper/020:32:420
2231126019,26sleep022306-21apt-get20:17:310
1755226016,21sleep00-21swapper/023:27:200
145426011,33sleep01539-21mailstats20:52:490
1069126016,19sleep20-21swapper/221:22:502
543825915,20sleep20-21swapper/219:22:482
503425917,18sleep00-21swapper/019:22:380
2394225918,21sleep10-21swapper/120:22:371
2394225918,21sleep10-21swapper/120:22:371
2192725917,26sleep221978-21irqstats21:57:402
2192725917,26sleep221978-21irqstats21:57:402
1679725920,22sleep20-21swapper/221:42:332
1679725920,22sleep20-21swapper/221:42:332
1532525914,30sleep115332-21diskstats21:37:361
1425725917,21sleep10-21swapper/119:52:201
3235725818,17sleep10-21swapper/122:32:211
2847925813,26sleep128550-21/usr/sbin/munin22:17:471
2246425815,18sleep00-21swapper/023:42:350
2246425815,18sleep00-21swapper/023:42:350
1009925817,23sleep00-21swapper/023:02:370
970525718,22sleep10-21swapper/119:37:381
2725925719,23sleep227269-21df_inode20:32:362
2607425717,23sleep226111-21ntp_states20:27:512
2607425717,23sleep226111-21ntp_states20:27:512
2518425713,29sleep125212-21/usr/sbin/munin22:07:451
1661425712,24sleep10-21swapper/119:57:491
1450125715,25sleep214498-21apt-get19:52:332
143382574,9sleep20-21swapper/223:16:442
1254425714,17sleep20-21swapper/219:47:202
103292574,13sleep20-21swapper/219:37:512
72625614,20sleep00-21swapper/020:52:200
72625614,20sleep00-21swapper/020:52:200
3682565,11sleep20-21swapper/200:14:162
314772565,16sleep00-21swapper/000:11:510
2577825614,19sleep00-21swapper/023:52:410
2577825614,19sleep00-21swapper/023:52:410
2525625614,27sleep025350-21apt-config20:27:210
2525625614,27sleep025350-21apt-config20:27:210
1898525620,19sleep20-21swapper/221:47:512
165025615,17sleep10-21swapper/119:12:261
1459025617,24sleep00-21swapper/019:52:390
998725513,25sleep19502-21apt-get23:02:301
37112555,15sleep20-21swapper/221:01:022
243772555,35sleep10-21swapper/123:47:411
989025412,26sleep09962-21cat19:37:410
67425415,24sleep2713-21modprobe20:52:182
67425415,24sleep2713-21modprobe20:52:182
611725422,17sleep00-21swapper/022:52:190
3103525418,18sleep00-21swapper/022:27:310
2231325419,22sleep222362-21/usr/sbin/munin20:17:332
203825413,17sleep00-21swapper/022:37:410
197422549,19sleep10-21swapper/120:09:081
128662549,23sleep20-21swapper/223:12:202
418625314,21sleep04279-21perl21:02:380
248625317,21sleep22482-21apt-get00:22:322
1342525314,19sleep20-21swapper/221:32:192
667625211,21sleep20-21swapper/222:52:412
640125213,23sleep06399-21apt-get22:52:310
1334725214,24sleep113381-21irqstats23:12:401
3079925110,18sleep20-21swapper/222:27:212
2715125015,18sleep00-21swapper/023:57:340
2715125015,18sleep00-21swapper/023:57:340
1932525013,21sleep119332-21apt-get23:32:301
1491125014,20sleep214967-21irqstats23:17:412
1484625010,20sleep10-21swapper/123:17:401
940824911,17sleep20-21swapper/219:37:222
236842497,31sleep20-21swapper/223:46:492
236842497,31sleep20-21swapper/223:46:492
340024610,20sleep23397-21apt-get19:17:312
33502469,21sleep13391-21users00:22:511
493224412,14sleep10-21swapper/122:47:351
264812449,18sleep226567-21perl22:12:392
1687224411,18sleep00-21swapper/021:42:350
1687224411,18sleep00-21swapper/021:42:350
318622417,16sleep10-21swapper/120:47:331
318622417,16sleep10-21swapper/120:47:331
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional