You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-15 - 20:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Wed Jul 15, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
48342114713,1120sleep211610-1kworker/2:1H09:08:542
48342114713,1120sleep211610-1kworker/2:1H09:08:542
907282221,784sleep1964-21/usr/sbin/munin10:39:111
907282221,784sleep1964-21/usr/sbin/munin10:39:111
30291256420,528sleep230330-21ntpq07:04:082
19909246020,423sleep319902-21apt-get08:13:573
19909246020,423sleep319902-21apt-get08:13:573
86824000,113sleep030124-1kworker/0:1H12:23:400
3048238228,340sleep03101-21ls09:03:590
15850234319,308sleep215851-21sshd11:24:332
2461233114,301sleep32523-21/usr/sbin/munin10:44:093
32008226247,199sleep032084-21awk07:09:100
32008226247,199sleep032084-21awk07:09:100
3368225146,180sleep03574-21latency_hist10:48:420
30005224575,156sleep030007-21diskstats07:04:010
22905223429,11sleep00-21swapper/010:04:300
19628223125,195sleep019668-21unin-run08:13:410
19628223125,195sleep019668-21unin-run08:13:410
4124222615,199sleep04130-21grep07:23:550
16629221517,143sleep016660-21latency_hist08:03:440
16629221517,143sleep016660-21latency_hist08:03:440
27623221312,186sleep027666-21latency12:04:030
19314221345,115sleep019374-21sendmail_mailqu09:54:090
6283220738,110sleep00-21swapper/007:29:080
6283220738,110sleep00-21swapper/007:29:080
17238220738,119sleep017342-21grep11:29:150
2828422059,179sleep00-21swapper/010:24:000
24371220240,121sleep024322-21latency_hist08:28:430
17266219542,64sleep00-21swapper/009:48:520
2897121938,171sleep029018-21systemd-run08:43:410
18112219011,166sleep118101-21grep08:08:421
1567218610,83sleep01598-21fschecks_count08:58:590
3257321797,71sleep00-21swapper/008:54:020
12331217715,145sleep312378-21systemd-run09:33:413
2719821636,142sleep00-21swapper/010:19:120
19267216212,104sleep019328-21sendmail-msp11:38:400
19267216212,104sleep019328-21sendmail-msp11:38:400
1500721628,112sleep015101-21fschecks_time11:23:580
1987216115,102sleep00-21swapper/007:15:190
1323215933,21sleep00-21swapper/012:24:010
30255215730,19sleep00-21swapper/010:29:090
24435215218,22sleep00-21swapper/011:54:020
24435215218,22sleep00-21swapper/011:54:020
14923215226,105sleep115062-21grep09:39:121
24816215017,118sleep324824-21/usr/sbin/munin10:13:423
31306214831,63sleep031357-21grep10:33:570
31423214511,122sleep031505-21grep08:49:140
21870214510,118sleep021918-21ntp_kernel_pll_08:19:060
21870214510,118sleep021918-21ntp_kernel_pll_08:19:060
16813214527,67sleep00-21swapper/008:03:570
8215214221,99sleep28325-21latency_hist11:03:422
7434214114,68sleep00-21swapper/007:33:530
7434214114,68sleep00-21swapper/007:33:530
12476213317,97sleep212620-21grep11:14:122
12476213317,97sleep212620-21grep11:14:122
22171213121,85sleep322300-21unixbench_multi08:19:143
22171213121,85sleep322300-21unixbench_multi08:19:143
1983321300,12sleep30-21swapper/308:13:433
1983321300,12sleep30-21swapper/308:13:433
12597213013,102sleep312662-21awk07:49:103
2604212924,82sleep12725-21grep10:44:111
1153321288,42sleep011543-21munin-run11:13:410
3659212413,94sleep33776-21timerandwakeup09:04:143
27392212331,30sleep0364-21snmpd08:38:400
27392212331,30sleep0364-21snmpd08:38:400
19678212251,53sleep019754-21grep11:39:000
147221200,6sleep30-21swapper/312:24:033
29448211921,24sleep10-21swapper/112:09:091
827621187,58sleep00-21swapper/009:19:050
6392211811,56sleep00-21swapper/009:13:560
25152211823,77sleep225192-21sort10:14:012
9593211747,16sleep00-21swapper/009:23:570
1106921179,18sleep00-21swapper/009:28:540
136921169,18sleep10-21swapper/112:24:021
21252211410,90sleep021329-21fschecks_time11:44:010
15504211416,83sleep315482-21latency_hist09:43:423
14081211317,77sleep114226-21/usr/sbin/munin11:19:151
20277211222,55sleep020320-21tr08:14:060
20277211222,55sleep020320-21tr08:14:060
5520211119,74sleep35728-21gpgv07:28:433
21106211116,79sleep321108-21acpi11:43:523
1361421116,61sleep00-21swapper/009:35:200
2036521100,8sleep30-21swapper/309:58:433
13443211044,54sleep013519-21sed07:53:430
25433210924,66sleep325042-21/usr/sbin/munin10:14:063
19347210914,20sleep2278-21dbus-daemon11:38:422
19347210914,20sleep2278-21dbus-daemon11:38:422
30468210719,72sleep330552-21switchtime07:04:133
2349521076,86sleep00-21swapper/011:49:140
2349521076,86sleep00-21swapper/011:49:140
6654210613,77sleep36589-21latency_hist10:58:433
31487210514,78sleep131524-21basename10:34:021
2711821049,57sleep027260-21cat12:03:430
23212210414,50sleep10-21swapper/111:49:101
23212210414,50sleep10-21swapper/111:49:101
2548521035,40sleep00-21swapper/008:29:120
632821025,53sleep00-21swapper/010:55:330
31383210020,65sleep231389-21df_inode10:33:592
23614210012,73sleep323616-21sshd11:49:333
23614210012,73sleep323616-21sshd11:49:333
61012997,53sleep031300-21cyclictest09:13:400
61012997,53sleep031300-21cyclictest09:13:400
41382994,14sleep10-21swapper/112:33:431
86462989,16sleep10-21swapper/111:03:451
288982988,46sleep00-21swapper/012:08:510
103402989,51sleep010387-21latency_hist07:43:420
103402989,51sleep010387-21latency_hist07:43:420
879529719,60sleep18742-21latency_hist07:38:421
879529719,60sleep18742-21latency_hist07:38:421
399429713,58sleep00-21swapper/012:33:420
325929719,58sleep23406-21grep07:19:132
2360429729,46sleep123643-21fschecks_count10:08:591
1261929711,17sleep20-21swapper/209:33:572
1213229618,64sleep212133-21cpu07:48:572
213952956,19sleep00-21swapper/010:00:020
430829434,17sleep00-21swapper/010:49:150
994629237,22sleep09975-21munin-run11:08:410
2673729217,52sleep126753-21sort10:19:001
1697229212,65sleep117052-21tr11:29:121
245762915,13sleep00-21swapper/010:11:080
73512909,22sleep00-21swapper/010:59:080
3147929019,54sleep231480-21/usr/sbin/munin07:08:512
3147929019,54sleep231480-21/usr/sbin/munin07:08:512
1935029013,61sleep319498-21latency_hist11:38:433
1935029013,61sleep319498-21latency_hist11:38:433
1295928915,56sleep012-21rcu_sched11:17:330
1295928915,56sleep012-21rcu_sched11:17:330
997528810,62sleep29982-21ntp_states09:24:082
1886328820,23sleep10-21swapper/108:09:101
1886328820,23sleep10-21swapper/108:09:101
459928727,40sleep14677-21latency_hist09:08:441
225928726,40sleep22288-21munin-run07:18:402
175228717,53sleep21856-21timerandwakeup12:24:112
832628642,24sleep30-21swapper/311:03:433
2117328627,38sleep221197-21modprobe08:18:402
2117328627,38sleep221197-21modprobe08:18:402
169082865,68sleep00-21swapper/009:47:310
15092866,18sleep00-21swapper/010:41:240
15092866,18sleep00-21swapper/010:41:240
2838228527,43sleep228445-21cat10:24:022
3053828412,56sleep030627-21fschecks_time12:14:000
145428416,52sleep31491-21ps07:14:093
1427328434,36sleep00-21swapper/009:38:590
944528322,41sleep29587-21grep11:04:272
484228312,56sleep14830-21apt-get09:08:571
484228312,56sleep14830-21apt-get09:08:571
1800528211,55sleep118134-21unixbench_multi09:49:131
148572827,64sleep00-21swapper/007:57:390
1418828213,53sleep214186-21apt-get09:38:532
449128118,29sleep10-21swapper/107:24:041
2084228116,49sleep320855-21idleruntime-cro11:43:393
871528026,33sleep38748-21munin-run07:38:403
871528026,33sleep38748-21munin-run07:38:403
1812428017,46sleep218264-21cat08:08:432
105792808,17sleep00-21swapper/007:43:540
2706627819,41sleep127098-21munin-run12:03:401
777827720,29sleep20-21swapper/209:18:432
351227730,12sleep00-21swapper/007:19:310
3100827720,34sleep3225-21systemd-journal10:33:403
1673627713,18sleep20-21swapper/211:29:052
51582765,60sleep00-21swapper/012:34:230
1777427621,38sleep217833-21sendmail_mailqu09:49:102
377127517,21sleep30-21swapper/310:48:583
2173427526,28sleep10-21swapper/110:03:421
1499527517,40sleep315005-21/usr/sbin/munin11:23:573
1435327530,11sleep00-21swapper/011:20:110
3023627429,12sleep00-21swapper/008:45:060
2630427430,11sleep00-21swapper/010:18:190
540327323,29sleep20-21swapper/210:54:012
3213027212,31sleep20-21swapper/212:18:592
3189527219,28sleep30-21swapper/310:34:123
1076927218,28sleep210831-21grep11:09:112
2782427119,34sleep227859-21ps12:04:092
1055527115,16sleep20-21swapper/207:43:522
953527014,39sleep19611-21awk07:39:111
953527014,39sleep19611-21awk07:39:111
66327012,37sleep332339-21/usr/sbin/munin08:54:133
2936026914,25sleep30-21swapper/312:09:073
1937726917,29sleep10-21swapper/109:54:101
1600026916,33sleep116149-21/usr/sbin/munin07:59:131
1600026916,33sleep116149-21/usr/sbin/munin07:59:131
1501626917,35sleep315157-21latency_hist07:58:433
1423526917,26sleep10-21swapper/107:54:131
1078626919,21sleep30-21swapper/307:44:013
91872689,16sleep30-21swapper/311:04:243
3089526824,22sleep10-21swapper/108:49:031
3000826828,21sleep10-21swapper/107:04:031
2167926820,23sleep30-21swapper/310:03:403
1968626820,26sleep20-21swapper/208:13:432
1968626820,26sleep20-21swapper/208:13:432
1539726813,33sleep115455-21/usr/sbin/munin11:24:081
1376726819,21sleep20-21swapper/207:54:012
721026710,45sleep27227-21latency_hist07:33:432
721026710,45sleep27227-21latency_hist07:33:432
3177026713,24sleep10-21swapper/107:09:031
3177026713,24sleep10-21swapper/107:09:031
3062926723,21sleep10-21swapper/112:14:011
2661526717,28sleep226679-21mailstats08:34:102
2661526717,28sleep226679-21mailstats08:34:102
234126715,29sleep10-21swapper/107:18:441
207526722,18sleep20-21swapper/210:44:002
2008526710,13sleep20-21swapper/211:39:102
125026717,27sleep10-21swapper/108:58:431
1121726718,28sleep20-21swapper/209:29:002
824326619,30sleep38273-21kernelversion09:19:043
50426611,40sleep2509-21diskstats10:39:002
50426611,40sleep2509-21diskstats10:39:002
313512666,11sleep30-21swapper/308:49:133
295426621,23sleep30-21swapper/307:19:063
240662669,12sleep30-21swapper/308:25:433
1262326612,29sleep30-21swapper/311:14:133
1262326612,29sleep30-21swapper/311:14:133
314426511,41sleep13166-21/usr/sbin/munin12:29:071
2983526518,28sleep229913-21grep10:28:592
1573726514,24sleep20-21swapper/207:59:102
1573726514,24sleep20-21swapper/207:59:102
1083326519,30sleep110946-21cat09:28:431
947926416,19sleep20-21swapper/207:39:082
947926416,19sleep20-21swapper/207:39:082
84926419,29sleep3862-21modprobe12:23:403
766826411,37sleep27725-21awk07:34:032
766826411,37sleep27725-21awk07:34:032
2819426416,21sleep10-21swapper/108:39:101
2205226420,28sleep222056-21diskstats10:04:002
543426323,19sleep20-21swapper/207:28:422
502526321,22sleep10-21swapper/110:53:401
430026318,27sleep24373-21apt-config09:08:412
2794726312,35sleep227981-21irqstats08:39:032
2368826313,33sleep123013-21/usr/sbin/munin08:24:121
1944526315,25sleep20-21swapper/209:54:112
460126216,17sleep30-21swapper/312:34:053
447126214,25sleep20-21swapper/210:49:172
325922627,14sleep10-21swapper/112:19:101
2503626217,27sleep125043-21/usr/sbin/munin08:29:051
2431126217,23sleep20-21swapper/211:54:002
2431126217,23sleep20-21swapper/211:54:002
1580626220,19sleep30-21swapper/307:59:113
1580626220,19sleep30-21swapper/307:59:113
116826216,29sleep21169-21cron08:58:402
1021226212,23sleep30-21swapper/309:24:133
3276626117,28sleep3318-21modprobe07:13:413
3276626117,28sleep3318-21modprobe07:13:413
2455926113,24sleep20-21swapper/208:28:492
223702617,12sleep10-21swapper/110:04:071
88426014,30sleep3890-21ntp_states10:39:093
88426014,30sleep3890-21ntp_states10:39:093
875826018,18sleep20-21swapper/209:19:152
2675526018,20sleep20-21swapper/210:19:012
2610826013,18sleep10-21swapper/108:33:531
2610826013,18sleep10-21swapper/108:33:531
903425917,24sleep39101-21df07:38:563
903425917,24sleep39101-21df07:38:563
797825919,21sleep10-21swapper/109:18:581
722825911,24sleep30-21swapper/307:33:443
722825911,24sleep30-21swapper/307:33:443
681425920,21sleep10-21swapper/107:29:181
681425920,21sleep10-21swapper/107:29:181
394225917,24sleep23968-21munin-run12:33:402
3057025913,23sleep30-21swapper/308:48:443
2466125920,23sleep124680-21ntp_states11:54:071
2466125920,23sleep124680-21ntp_states11:54:071
2037625914,17sleep10-21swapper/111:39:131
1705325921,20sleep20-21swapper/209:48:422
1181125913,25sleep10-21swapper/107:48:191
1104425916,26sleep111063-21apt-get09:28:511
2637925816,25sleep326402-21irqstats08:34:033
2637925816,25sleep326402-21irqstats08:34:033
2161825810,33sleep121697-21grep11:44:101
304192577,10sleep20-21swapper/212:13:462
3019225715,18sleep10-21swapper/110:29:061
2827625718,23sleep328282-21diskstats10:23:593
2590725714,24sleep226007-21perl11:59:002
2564125717,18sleep10-21swapper/110:14:121
246682577,18sleep20-21swapper/210:12:192
2294925720,18sleep20-21swapper/211:49:032
2294925720,18sleep20-21swapper/211:49:032
2049825718,22sleep20-21swapper/209:58:592
187725718,24sleep21900-21ntpq08:59:052
1837625716,19sleep10-21swapper/111:34:051
1837625716,19sleep10-21swapper/111:34:051
2832525620,23sleep328439-21sed08:39:123
2744025622,21sleep127513-21fschecks_time12:04:001
2539125611,17sleep30-21swapper/308:29:113
2225625616,17sleep30-21swapper/310:04:053
2169825619,19sleep20-21swapper/211:44:112
2146525613,20sleep20-21swapper/208:18:522
2146525613,20sleep20-21swapper/208:18:522
1707925615,18sleep30-21swapper/308:04:043
126962565,15sleep10-21swapper/111:14:151
126962565,15sleep10-21swapper/111:14:151
688725513,26sleep16881-21apt-get10:58:531
646125513,17sleep10-21swapper/109:13:571
275642558,17sleep30-21swapper/312:04:023
2149025513,25sleep121485-21apt-get08:18:551
2149025513,25sleep121485-21apt-get08:18:551
1771425513,24sleep317743-21ntp_states09:49:073
423525412,25sleep24229-21apt-get12:33:562
3166625412,22sleep231719-21users08:49:162
2944125421,17sleep10-21swapper/108:44:021
2933225414,15sleep20-21swapper/208:43:592
26022547,34sleep30-21swapper/309:01:483
257825412,25sleep12572-21apt-get07:18:561
1646825418,23sleep316531-21cpuspeed11:28:583
1306525417,23sleep113076-21ntpq09:34:081
2432425315,20sleep324433-21perl11:54:013
2432425315,20sleep324433-21perl11:54:013
1221525315,24sleep112220-21entropy07:49:001
1334225219,18sleep213344-21acpi11:18:522
1068925213,24sleep310695-21ntp_states11:09:083
1028725213,21sleep10-21swapper/111:08:581
639025112,23sleep36385-21apt-get09:13:553
2684925115,18sleep30-21swapper/310:19:023
1571925110,23sleep10-21swapper/109:43:531
539825014,21sleep15402-21diskstats10:54:001
492925014,15sleep30-21swapper/309:09:153
492925014,15sleep30-21swapper/309:09:153
3045125011,22sleep330446-21apt-get12:13:553
210222506,34sleep10-21swapper/108:16:451
210222506,34sleep10-21swapper/108:16:451
2935324910,23sleep229359-21ntp_states12:09:062
1574224912,22sleep315738-21apt-get09:43:553
1274024911,16sleep30-21swapper/309:34:023
2308224811,22sleep223109-21df08:23:572
2817524715,16sleep10-21swapper/110:23:531
2618024611,17sleep30-21swapper/311:59:053
14842459,19sleep11551-21df08:58:561
276024412,16sleep30-21swapper/312:28:573
133482449,18sleep30-21swapper/311:18:533
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional