You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-06 - 20:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Fri Mar 06, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
32159236406,3599sleep128993-21kworker/1:108:10:171
1411826656,646sleep114154-21ntpq10:39:001
868924560,14sleep20-21swapper/208:39:452
868924560,14sleep20-21swapper/208:39:452
15222427102,217sleep11545-21idleruntime-cro08:18:371
10023241327,367sleep110090-21/usr/sbin/munin07:04:001
2389023956,377sleep120517-21kworker/u8:009:28:461
14331222754,145sleep10-21swapper/108:58:381
21308222411,195sleep121372-21mailstats09:19:051
11098222014,188sleep111246-21latency_hist12:13:401
11098222014,188sleep111246-21latency_hist12:13:401
1720622149,194sleep013441-21snmpd09:06:170
12886220118,167sleep012903-21apt12:18:480
2205219829,48sleep10-21swapper/108:19:011
6796219730,155sleep16832-21ntpq11:59:021
12046219120,81sleep112198-21unixbench_singl12:14:081
15454217919,145sleep215482-21users08:59:122
5121217813,153sleep15200-21cat11:54:021
5121217813,153sleep15200-21cat11:54:021
31184217635,119sleep131274-21grep11:33:551
21606217411,151sleep121635-21cpu07:38:521
3151217012,145sleep13221-21switchtime10:04:071
27909216917,131sleep027950-21uniq07:58:570
806321687,143sleep10-21swapper/112:03:551
806321687,143sleep10-21swapper/112:03:551
2669321645,147sleep10-21swapper/109:38:151
3085421639,141sleep130923-21sed11:33:391
20042216277,48sleep10-21swapper/110:58:551
17960216212,132sleep117981-21kernelversion09:09:001
16558216183,63sleep116567-21modprobe10:48:371
27504215721,45sleep027515-21modprobe07:58:370
6615215417,123sleep26666-21fschecks_count08:33:562
6615215417,123sleep26666-21fschecks_count08:33:562
1978221548,51sleep119790-21munin-run07:33:391
12264215414,119sleep112354-21fschecks_time10:34:001
12264215414,119sleep112354-21fschecks_time10:34:001
15895215112,49sleep10-21swapper/109:03:391
12981214814,109sleep20-21swapper/212:18:542
11552214628,104sleep111733-21modprobe07:08:401
18899214519,16sleep10-21swapper/112:38:391
5081214415,103sleep10-21swapper/108:28:591
27707214482,14sleep10-21swapper/111:23:371
14149214229,93sleep10-21swapper/112:23:211
3092721409,55sleep130991-21cat08:08:401
29302213920,95sleep10-21swapper/109:44:051
18194213872,24sleep1281-21dbus-daemon07:28:381
702621375,32sleep10-21swapper/110:16:231
6981213488,30sleep10-21swapper/108:34:031
6981213488,30sleep10-21swapper/108:34:031
2438321345,61sleep10-21swapper/107:48:171
32667213316,94sleep10-21swapper/111:38:511
345021327,113sleep13519-21basename11:48:571
345021327,113sleep13519-21basename11:48:571
32419212711,18sleep00-21swapper/011:38:380
30178212716,23sleep30-21swapper/309:48:433
26644212731,84sleep126688-21ntpq07:54:021
2292521268,20sleep10-21swapper/111:08:401
2292521268,20sleep10-21swapper/111:08:401
12355212613,100sleep212410-21basename10:34:012
12355212613,100sleep212410-21basename10:34:012
19388212422,77sleep00-21swapper/009:14:000
12313212315,88sleep212338-21/usr/sbin/munin07:09:182
13618212217,77sleep30-21swapper/308:54:083
19775212111,86sleep10-21swapper/109:14:091
19722212027,80sleep11-21systemd10:58:381
2827521195,11sleep10-21swapper/109:40:131
22829211911,92sleep022837-21ntpq09:24:050
1027821185,11sleep00-21swapper/010:28:010
797211716,63sleep10-21swapper/109:58:381
797211716,63sleep10-21swapper/109:58:381
789021178,97sleep17926-21ntpq10:19:011
26130211528,70sleep126191-21sendmail-msp11:18:371
28737211418,72sleep028888-21head11:24:090
16645211422,15sleep10-21swapper/107:23:381
14478211426,11sleep10-21swapper/107:14:091
23739211222,15sleep10-21swapper/111:09:161
11442211219,52sleep10-21swapper/108:48:491
11442211219,52sleep10-21swapper/108:48:491
4375211121,70sleep23872-21apt-get10:08:492
25958211123,13sleep10-21swapper/107:53:381
14438211110,84sleep10-21swapper/112:23:491
28890210929,47sleep10-21swapper/111:24:101
10687210921,15sleep10-21swapper/110:28:551
9973210822,15sleep10-21swapper/108:43:571
9973210822,15sleep10-21swapper/108:43:571
8672210826,67sleep10-21swapper/108:39:451
8672210826,67sleep10-21swapper/108:39:451
7693210721,58sleep37714-21awk12:03:363
630621078,86sleep00-21swapper/010:14:040
4355210710,81sleep00-21swapper/010:08:470
22020210713,75sleep022099-21/usr/sbin/munin11:04:070
22020210713,75sleep022099-21/usr/sbin/munin11:04:070
17395210721,65sleep216907-21/usr/sbin/munin07:24:072
1016321075,67sleep10-21swapper/110:26:321
490721057,16sleep10-21swapper/110:09:451
4742210413,76sleep24774-21latency_hist08:28:392
3158521045,52sleep10-21swapper/109:53:011
3158521045,52sleep10-21swapper/109:53:011
1789021044,88sleep00-21swapper/010:50:540
14223210421,55sleep314307-21latency_hist12:23:393
11029210422,64sleep211056-21munin-plugin-st12:13:372
11029210422,64sleep211056-21munin-plugin-st12:13:372
31686210324,61sleep331780-21latency_hist09:53:403
31686210324,61sleep331780-21latency_hist09:53:403
16848210312,17sleep10-21swapper/110:48:531
17644210215,66sleep317734-21fschecks_time12:33:573
17310210214,66sleep317345-21proc_pri10:49:083
1008121026,77sleep00-21swapper/008:44:000
1008121026,77sleep00-21swapper/008:44:000
2921929921,61sleep229108-21latency_hist08:03:402
2913329911,16sleep129218-21gpgv08:03:391
1907329920,54sleep319220-21grep10:54:083
3080229826,51sleep330816-21sh11:33:373
2273929767,14sleep10-21swapper/109:24:021
184929714,60sleep21733-21/usr/sbin/munin11:43:572
184929714,60sleep21733-21/usr/sbin/munin11:43:572
112442964,11sleep30-21swapper/307:08:183
326172950,7sleep20-21swapper/208:13:402
2546529514,65sleep225475-21cpu09:33:522
493829410,71sleep25011-21fschecks_time11:53:592
493829410,71sleep25011-21fschecks_time11:53:592
2938429420,54sleep329560-21apt-key08:03:413
2635729326,45sleep325440-21/usr/sbin/munin09:34:093
2219329322,39sleep20-21swapper/211:04:092
2219329322,39sleep20-21swapper/211:04:092
1139929310,64sleep011551-21sed07:08:390
385729219,55sleep33887-21awk10:08:383
3047429016,59sleep030574-21fschecks_time09:48:560
3047429016,59sleep030574-21fschecks_time09:48:560
2953728918,45sleep029687-21sed09:44:080
2050528716,54sleep020567-21wc07:34:070
518228614,59sleep25239-21if_eth008:29:002
833928520,48sleep37824-21apt-get08:38:503
833928520,48sleep37824-21apt-get08:38:503
553828516,54sleep30-21swapper/310:12:173
277912850,6sleep30-21swapper/311:23:383
1749128514,16sleep10-21swapper/109:08:401
157128513,15sleep00-21swapper/009:59:060
437828412,57sleep34379-21sshd11:51:173
437828412,57sleep34379-21sshd11:51:173
2325128414,54sleep223360-21perl07:43:582
188928412,55sleep21914-21df_inode08:18:542
1466528418,21sleep00-21swapper/008:58:580
393828313,14sleep10-21swapper/108:24:061
306628326,27sleep30-21swapper/311:48:383
306628326,27sleep30-21swapper/311:48:383
3001128312,56sleep030067-21processes11:29:050
2600828337,24sleep30-21swapper/307:53:403
1645128311,57sleep216452-21sshd10:47:172
19732825,43sleep10-21swapper/109:59:231
141128212,53sleep31018-21/usr/sbin/munin09:59:013
51828111,54sleep0617-21memory08:14:000
169912817,42sleep10-21swapper/112:30:031
169912817,42sleep10-21swapper/112:30:031
26712805,42sleep10-21swapper/111:44:131
26712805,42sleep10-21swapper/111:44:131
1038028022,26sleep30-21swapper/310:28:403
999027916,46sleep310078-21fschecks_time08:43:593
999027916,46sleep310078-21fschecks_time08:43:593
2692427913,16sleep30-21swapper/307:54:083
2123627919,46sleep321267-21awk11:03:373
211752785,40sleep10-21swapper/107:37:141
316927717,37sleep03285-21apt-config08:23:410
2522727717,44sleep225324-21latency_hist09:33:402
2446027712,52sleep324493-21latency_hist07:48:393
127612779,56sleep212832-21latency_hist08:53:392
127612779,56sleep212832-21latency_hist08:53:392
46072757,55sleep00-21swapper/008:27:320
2341827521,31sleep323474-21fschecks_count11:09:103
1970127514,29sleep30-21swapper/309:14:073
1792727411,33sleep00-21swapper/012:34:020
166002747,56sleep00-21swapper/007:23:040
1059027414,40sleep310592-21/usr/sbin/munin10:28:493
3200927323,26sleep30-21swapper/309:53:563
3200927323,26sleep30-21swapper/309:53:563
2486327317,28sleep20-21swapper/211:13:542
82927215,28sleep20-21swapper/209:58:392
82927215,28sleep20-21swapper/209:58:392
760727217,34sleep07687-21tune2fs10:18:560
148162728,13sleep00-21swapper/010:41:090
146392728,15sleep20-21swapper/212:23:592
85627112,36sleep0980-21timerwakeupswit11:39:070
84922715,54sleep30-21swapper/310:19:563
1876127113,43sleep018793-21/usr/sbin/munin10:54:010
779027017,32sleep37816-21awk08:38:373
779027017,32sleep37816-21awk08:38:373
327827017,28sleep00-21swapper/011:48:480
327827017,28sleep00-21swapper/011:48:480
322427017,36sleep23349-21ls10:04:092
2367927011,45sleep223718-21cat09:28:392
2854026922,27sleep20-21swapper/209:43:372
2478426923,23sleep00-21swapper/007:48:560
1355026917,32sleep20-21swapper/208:54:072
1059226920,35sleep210676-21switchtime07:04:132
2286426815,37sleep222897-21idleruntime-cro07:43:372
225962688,47sleep00-21swapper/007:40:120
1547726817,36sleep215531-21fschecks_count07:19:032
1518626821,29sleep00-21swapper/007:18:400
1267326822,19sleep30-21swapper/312:18:393
1119726817,29sleep20-21swapper/210:29:082
370826716,23sleep20-21swapper/211:49:022
370826716,23sleep20-21swapper/211:49:022
3208926712,39sleep2222-21systemd-journal11:35:172
186932678,12sleep20-21swapper/209:10:182
1613326717,24sleep30-21swapper/309:03:533
727026616,25sleep20-21swapper/210:18:382
1690626611,15sleep30-21swapper/307:23:483
146926621,27sleep20-21swapper/211:43:372
1241126611,40sleep312447-21uname10:34:023
1241126611,40sleep312447-21uname10:34:023
1193726621,21sleep00-21swapper/012:14:070
1061926616,23sleep00-21swapper/010:28:540
389226522,24sleep00-21swapper/010:08:390
2330026518,27sleep223302-21apt-get11:08:492
1900126520,18sleep00-21swapper/009:13:390
1669626520,23sleep30-21swapper/307:23:403
1647326521,18sleep00-21swapper/012:29:050
1647326521,18sleep00-21swapper/012:29:050
1401826524,21sleep30-21swapper/310:38:583
696126422,19sleep20-21swapper/211:59:072
2594426419,26sleep30-21swapper/311:16:173
180622645,12sleep30-21swapper/309:09:023
1419726418,17sleep30-21swapper/307:14:033
1396026412,35sleep214017-21irqstats10:38:572
2070926316,20sleep30-21swapper/307:34:103
1660926317,26sleep30-21swapper/310:48:393
1646526313,20sleep30-21swapper/312:29:043
1646526313,20sleep30-21swapper/312:29:043
1107326315,28sleep00-21swapper/012:13:390
1107326315,28sleep00-21swapper/012:13:390
3116326217,28sleep331172-21diskstats11:33:533
2452226218,22sleep30-21swapper/309:29:093
2373726235,16sleep023825-21sed07:44:080
1855526214,34sleep218556-21sshd12:35:172
1314926218,27sleep00-21swapper/007:13:400
129652626,46sleep00-21swapper/008:53:410
1271026216,31sleep012713-21cron08:53:370
1271026216,31sleep012713-21cron08:53:370
79492618,17sleep00-21swapper/012:03:420
79492618,17sleep00-21swapper/012:03:420
307902615,44sleep00-21swapper/008:07:340
189402616,13sleep00-21swapper/007:29:060
1391926114,28sleep214005-21perl07:13:582
94952606,42sleep00-21swapper/008:42:140
94952606,42sleep00-21swapper/008:42:140
574326015,27sleep35756-21/usr/sbin/munin11:54:123
574326015,27sleep35756-21/usr/sbin/munin11:54:123
388726018,21sleep30-21swapper/308:24:043
3208426022,19sleep00-21swapper/009:53:570
3208426022,19sleep00-21swapper/009:53:570
2681626016,20sleep00-21swapper/009:38:390
236626012,25sleep20-21swapper/210:03:402
2106126014,28sleep221095-21/usr/sbin/munin09:18:582
1658626020,19sleep20-21swapper/212:29:072
1658626020,19sleep20-21swapper/212:29:072
1513326026,20sleep30-21swapper/308:59:093
812825920,21sleep30-21swapper/312:03:573
812825920,21sleep30-21swapper/312:03:573
655025917,25sleep36671-21sed10:14:113
640325920,23sleep06422-21apt-get11:58:490
2839725923,20sleep30-21swapper/307:59:093
2520325913,19sleep20-21swapper/207:49:062
1172125919,21sleep30-21swapper/308:48:593
1172125919,21sleep30-21swapper/308:48:593
1105825913,30sleep311071-21systemd-run12:13:383
1105825913,30sleep311071-21systemd-run12:13:383
652825816,25sleep06524-21apt-get08:33:500
652825816,25sleep06524-21apt-get08:33:500
63225812,31sleep3661-21ntp_states11:39:023
3116225714,26sleep031157-21apt-get08:08:510
1874825718,19sleep30-21swapper/307:29:013
564225617,21sleep00-21swapper/008:29:100
2753525615,24sleep327592-21irqstats09:39:313
2727725615,21sleep00-21swapper/009:38:510
2711925611,21sleep00-21swapper/011:19:090
2544525617,20sleep00-21swapper/011:14:080
2203825615,23sleep322057-21ntp_states07:39:023
1446225617,23sleep014456-21apt-get12:23:520
906525515,24sleep29062-21apt-get10:23:512
3123125514,22sleep331238-21df_inode08:08:543
3036625513,26sleep230511-21/usr/sbin/munin08:04:412
2775225517,19sleep20-21swapper/211:23:382
2437925514,19sleep00-21swapper/009:29:060
1621425515,23sleep216218-21diskstats09:03:562
989625415,23sleep29890-21apt-get08:43:542
989625415,23sleep29890-21apt-get08:43:542
68025414,24sleep3703-21ntp_states08:14:033
2954425412,25sleep229537-21apt-get11:28:512
2669025413,27sleep226700-21ntpq07:54:032
231125411,17sleep00-21swapper/011:44:070
231125411,17sleep00-21swapper/011:44:070
2213525413,18sleep30-21swapper/309:23:403
1866025418,17sleep20-21swapper/210:53:592
1819025415,24sleep018191-21cron07:28:370
85432536,38sleep20-21swapper/210:20:362
1979125311,17sleep20-21swapper/207:33:402
183725314,23sleep31833-21apt-get08:18:513
1684925313,22sleep216843-21apt-get10:48:542
2462325214,18sleep00-21swapper/011:13:390
226372527,11sleep30-21swapper/309:24:003
1924025214,23sleep219234-21apt-get09:13:542
1132225212,22sleep211389-21df12:13:532
50752519,18sleep00-21swapper/011:54:010
50752519,18sleep00-21swapper/011:54:010
1203425116,16sleep30-21swapper/307:09:113
798325015,22sleep27994-21grep12:03:522
798325015,22sleep27994-21grep12:03:522
2930625016,17sleep20-21swapper/211:28:392
2127125011,22sleep021286-21munin-run11:03:380
2005125013,22sleep220055-21diskstats10:58:562
1538325012,17sleep00-21swapper/010:43:560
2091224912,16sleep00-21swapper/009:18:560
2048224913,20sleep220488-21ntp_states07:34:052
1486824911,17sleep30-21swapper/312:24:043
2232524811,20sleep322425-21users11:04:103
2232524811,20sleep322425-21users11:04:103
541424611,21sleep35437-21ntp_states08:29:053
2923024610,17sleep20-21swapper/209:44:022
2826424613,17sleep30-21swapper/311:24:003
304562449,20sleep230460-21diskstats09:48:542
304562449,20sleep230460-21diskstats09:48:542
1225924413,15sleep00-21swapper/010:33:580
1225924413,15sleep00-21swapper/010:33:580
15512419,15sleep20-21swapper/208:18:382
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional