You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-12 - 16:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Fri Jun 12, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
23486288213,852sleep223478-21apt-get08:08:002
14540240613,379sleep214581-21ntpq07:38:122
28428235719,323sleep128432-21ntpq11:48:111
22472230018,265sleep022542-21/usr/sbin/munin08:03:130
302972270127,21sleep30-21swapper/310:12:453
262972269168,23sleep3284-21dbus-daemon08:17:423
46802259109,29sleep30-21swapper/310:33:073
46802259109,29sleep30-21swapper/310:33:073
5862224915,220sleep05864-21/usr/sbin/munin10:37:570
27155222819,187sleep327305-21unixbench_singl11:43:253
31635220278,52sleep331682-21ntp_kernel_pll_08:33:083
6366218612,153sleep36397-21munin-run08:57:433
12852218018,139sleep30-21swapper/307:33:093
11852217420,34sleep311881-21munin-run10:57:423
14187217316,26sleep30-21swapper/311:03:073
861221709,20sleep30-21swapper/310:46:323
715821708,123sleep37306-21grep12:23:133
11534217028,72sleep311613-21sendmail_mailtr07:28:163
6092166122,24sleep30-21swapper/308:38:053
10785216625,116sleep30-21swapper/307:27:443
31072216310,140sleep331146-21switchtime10:13:133
31072216310,140sleep331146-21switchtime10:13:133
287132158119,19sleep30-21swapper/310:07:443
85332157118,17sleep30-21swapper/312:28:103
8809215514,26sleep20-21swapper/210:47:442
30766215210,106sleep330827-21sendmail-msp11:57:423
5609215010,123sleep05742-21unixbench_singl12:18:140
19408214921,19sleep30-21swapper/307:53:113
1901521485,128sleep30-21swapper/311:18:133
2871214710,122sleep3225-21systemd-journal12:11:063
2871214710,122sleep3225-21systemd-journal12:11:063
1839821468,125sleep318522-21timerwakeupswit09:33:123
5944214510,118sleep30-21swapper/310:38:003
1507721458,12sleep00-21swapper/007:38:220
12819214419,18sleep00-21swapper/009:17:430
12819214419,18sleep00-21swapper/009:17:430
9760214312,113sleep39755-21apt-get12:32:543
2815921397,116sleep30-21swapper/308:22:553
2734421384,12sleep20-21swapper/211:43:282
21588213622,88sleep221676-21tune2fs11:28:012
1129021359,109sleep311438-21sed09:12:453
5588213293,28sleep30-21swapper/307:11:453
21213213112,17sleep30-21swapper/311:27:423
1716121317,10sleep00-21swapper/009:28:450
1716121317,10sleep00-21swapper/009:28:450
32250212812,14sleep30-21swapper/310:18:013
32250212812,14sleep30-21swapper/310:18:013
397821265,17sleep30-21swapper/310:32:393
26939212450,10sleep30-21swapper/310:00:293
1656212411,98sleep21718-21/usr/sbin/munin10:23:092
1656212411,98sleep21718-21/usr/sbin/munin10:23:092
588521239,75sleep35986-21tune2fs07:12:463
25956212364,43sleep326057-21apt-config11:42:443
2275321237,86sleep30-21swapper/311:32:393
12695212349,11sleep30-21swapper/309:16:243
12695212349,11sleep30-21swapper/309:16:243
4566212218,33sleep04593-21munin-run12:17:410
3226421226,104sleep30-21swapper/312:01:333
3226421226,104sleep30-21swapper/312:01:333
4823212113,92sleep14778-21latency_hist08:52:441
4823212113,92sleep14778-21latency_hist08:52:441
254021215,11sleep30-21swapper/308:43:143
2355821218,48sleep323650-21mailstats11:33:113
1677621217,80sleep30-21swapper/307:45:203
6417212018,86sleep26426-21df_inode07:13:012
28863212048,11sleep30-21swapper/311:48:173
1531421195,81sleep30-21swapper/307:41:283
23609211849,11sleep30-21swapper/309:48:343
1819221178,71sleep318345-21latency_hist11:17:453
17418211752,54sleep30-21swapper/309:32:073
17418211752,54sleep30-21swapper/309:32:073
1337421175,81sleep30-21swapper/311:01:543
9466211548,10sleep30-21swapper/309:05:023
1824821156,77sleep30-21swapper/307:49:203
6104211472,11sleep30-21swapper/312:22:123
15906211474,29sleep30-21swapper/309:27:313
24232211170,29sleep30-21swapper/309:52:463
141921116,93sleep30-21swapper/312:07:193
141921116,93sleep30-21swapper/312:07:193
4572211047,52sleep30-21swapper/308:50:243
4572211047,52sleep30-21swapper/308:50:243
7887210946,52sleep30-21swapper/308:59:373
2348821087,60sleep30-21swapper/308:08:013
2214221075,68sleep30-21swapper/309:44:393
640210513,75sleep0647-21ntp_states12:03:080
640210513,75sleep0647-21ntp_states12:03:080
2829221050,8sleep20-21swapper/210:03:142
23420210515,73sleep223452-21sshd11:33:062
18162210513,76sleep218191-21latency_hist11:17:442
21675210423,23sleep00-21swapper/009:43:140
16229210355,37sleep30-21swapper/311:08:493
2901210234,45sleep02915-21grep07:03:020
20645210263,27sleep30-21swapper/309:40:183
1956521028,74sleep10-21swapper/109:37:541
8898210163,11sleep30-21swapper/307:18:593
18509210117,67sleep218532-21latency_hist07:52:422
2363210058,10sleep30-21swapper/310:26:503
2363210058,10sleep30-21swapper/310:26:503
12383210014,26sleep112440-21irqstats10:58:051
3273529917,56sleep1370-21fschecks_time12:03:011
3273529917,56sleep1370-21fschecks_time12:03:011
2173429916,67sleep021887-21latency_hist08:02:450
1406829960,28sleep30-21swapper/309:18:473
1406829960,28sleep30-21swapper/309:18:473
321462986,83sleep00-21swapper/012:00:010
321462986,83sleep00-21swapper/012:00:010
1037629814,70sleep110410-21ntp_kernel_pll_09:08:071
310112967,17sleep30-21swapper/308:32:443
1490529616,62sleep114133-21/usr/sbin/munin07:38:191
2142929558,27sleep30-21swapper/307:59:573
2929529417,56sleep00-21swapper/008:26:070
2670429314,65sleep30-21swapper/308:18:023
2190429347,12sleep30-21swapper/308:02:523
94092928,13sleep30-21swapper/310:48:053
261629221,48sleep02773-21tail08:43:150
1547529211,63sleep015567-21users09:23:160
923828917,50sleep29239-21cron12:32:412
3251928826,40sleep132535-21modprobe08:37:421
3235028825,40sleep132056-21/usr/sbin/munin10:18:031
3235028825,40sleep132056-21/usr/sbin/munin10:18:031
398228715,57sleep24026-21systemd-run10:32:422
39032879,62sleep30-21swapper/312:13:123
3099928714,56sleep231070-21grep10:13:122
3099928714,56sleep231070-21grep10:13:122
2638628626,37sleep026405-21apt-get11:42:530
2483628521,45sleep224870-21/usr/sbin/munin11:38:032
661828415,32sleep00-21swapper/012:23:020
2588428424,41sleep125889-21modprobe11:42:421
1516928415,52sleep115248-21systemctl09:23:131
1380128423,30sleep00-21swapper/009:18:150
1380128423,30sleep00-21swapper/009:18:150
3176528314,51sleep031832-21modprobe10:17:420
3176528314,51sleep031832-21modprobe10:17:420
841928216,18sleep10-21swapper/109:02:581
2958528214,52sleep029592-21diskstats11:52:580
2267328219,35sleep10-21swapper/109:48:001
930728020,40sleep09417-21apt-config12:32:430
2794128014,52sleep027960-21apt-get11:47:540
3200727921,35sleep20-21swapper/208:33:152
318427925,23sleep10-21swapper/107:03:071
28752795,14sleep00-21swapper/012:11:070
28752795,14sleep00-21swapper/012:11:070
200372798,55sleep10-21swapper/111:23:001
1952627929,29sleep00-21swapper/007:53:120
1443827916,29sleep10-21swapper/109:22:441
1443827916,29sleep10-21swapper/109:22:441
942027821,38sleep19623-21latency_hist12:32:441
3013727817,15sleep10-21swapper/111:53:121
2853027813,46sleep228558-21/usr/sbin/munin08:23:062
2941827716,17sleep00-21swapper/010:08:090
279127716,45sleep12797-21df_inode10:27:571
2003327714,40sleep00-21swapper/011:22:590
3025527623,27sleep20-21swapper/210:12:432
2637827611,52sleep126398-21cat08:17:441
2281427613,16sleep10-21swapper/111:32:451
1254527624,31sleep00-21swapper/010:58:080
112127626,25sleep10-21swapper/108:38:151
374927517,40sleep23815-21latency08:48:052
374927517,40sleep23815-21latency08:48:052
2204727518,26sleep00-21swapper/011:28:120
1988127514,44sleep219888-21kernelversion09:38:322
1666027516,31sleep016645-21apt-get11:13:010
156432755,11sleep10-21swapper/111:08:061
479527429,19sleep00-21swapper/008:52:430
479527429,19sleep00-21swapper/008:52:430
2652127412,46sleep226634-21cat09:58:142
259682747,55sleep00-21swapper/008:13:350
2688727321,26sleep20-21swapper/208:18:052
1075327319,29sleep00-21swapper/009:08:140
406727217,33sleep00-21swapper/010:32:440
2438527221,26sleep00-21swapper/008:08:180
1795227213,46sleep118010-21cat09:33:021
1162627221,22sleep20-21swapper/209:13:042
1162627221,22sleep20-21swapper/209:13:042
630227124,30sleep26331-21munin-run08:57:422
2193527111,44sleep221984-21awk08:02:592
2136527112,45sleep121366-21sshd07:59:071
120062714,57sleep20-21swapper/207:29:082
74632707,15sleep00-21swapper/007:15:500
3102327013,40sleep031151-21ls08:32:450
1235427015,39sleep00-21swapper/007:32:450
1971526911,40sleep119832-21latency_hist11:22:431
643426816,32sleep06594-21cat08:57:440
55026812,16sleep20-21swapper/208:38:042
2814326814,39sleep128291-21awk10:03:141
1818826820,30sleep218207-21ntp_states09:33:072
1582826820,29sleep20-21swapper/211:08:112
1287726816,25sleep10-21swapper/109:17:441
1287726816,25sleep10-21swapper/109:17:441
956626715,38sleep19571-21entropy07:23:051
2581926716,25sleep10-21swapper/109:57:531
1307826720,23sleep20-21swapper/207:33:162
105002678,18sleep00-21swapper/007:24:420
86632669,21sleep10-21swapper/110:46:381
813026618,31sleep21-21systemd09:02:432
659526617,23sleep10-21swapper/108:57:451
436926623,27sleep14402-21grep10:33:011
436926623,27sleep14402-21grep10:33:011
350726613,39sleep23556-21irqstats12:13:032
336026629,20sleep20-21swapper/207:03:122
3177026620,28sleep231933-21grep11:58:132
3177026620,28sleep231933-21grep11:58:132
2987526621,20sleep20-21swapper/211:53:052
2928326619,26sleep20-21swapper/211:52:442
56192659,43sleep00-21swapper/007:12:060
35126512,40sleep0352-21munin-node08:37:540
2979726518,29sleep129800-21entropy08:28:011
1911826513,23sleep00-21swapper/011:18:140
1884526514,34sleep118873-21df07:52:571
1619326512,36sleep016255-21/usr/sbin/munin07:43:150
1016726514,22sleep20-21swapper/212:33:262
53862648,43sleep00-21swapper/010:35:430
53862648,43sleep00-21swapper/010:35:430
2436226419,20sleep20-21swapper/209:53:012
2365326412,33sleep023731-21threads11:33:120
902626313,23sleep00-21swapper/009:03:130
80982636,11sleep00-21swapper/009:02:160
3096226317,32sleep130996-21ps10:13:111
3096226317,32sleep130996-21ps10:13:111
2950226321,21sleep10-21swapper/108:27:451
2460326316,24sleep00-21swapper/011:37:520
182026311,36sleep21824-21diskstats12:07:582
182026311,36sleep21824-21diskstats12:07:582
1383626318,27sleep10-21swapper/107:37:431
904426216,29sleep00-21swapper/010:47:540
78762629,37sleep07940-21ps10:43:100
308226223,20sleep30-21swapper/307:03:053
230126220,23sleep10-21swapper/108:43:071
2121326214,23sleep20-21swapper/209:43:042
1685626216,19sleep10-21swapper/109:28:131
1685626216,19sleep10-21swapper/109:28:131
967726119,20sleep20-21swapper/210:48:122
858826114,31sleep28646-21cat09:03:032
800526114,29sleep18014-21/usr/sbin/munin07:18:011
716926122,16sleep20-21swapper/210:42:432
704726117,26sleep20-21swapper/212:23:122
2716426113,27sleep00-21swapper/010:02:440
11742619,16sleep10-21swapper/110:22:561
11742619,16sleep10-21swapper/110:22:561
978926016,25sleep10-21swapper/112:33:091
973526010,21sleep00-21swapper/012:32:510
643426017,26sleep16428-21apt-get12:22:551
502726017,27sleep25179-21sed10:33:152
502726017,27sleep25179-21sed10:33:152
322172609,39sleep00-21swapper/008:33:500
35025913,16sleep20-21swapper/210:18:132
35025913,16sleep20-21swapper/210:18:132
1196225918,18sleep20-21swapper/210:57:452
997925814,26sleep29975-21apt-get09:07:552
2351325816,19sleep10-21swapper/111:33:091
1217225815,26sleep212166-21apt-get10:57:572
1112325816,26sleep111215-21perl07:28:061
793025716,24sleep27924-21apt-get07:17:572
769125718,17sleep10-21swapper/107:17:441
619125717,17sleep20-21swapper/212:22:442
1772525714,30sleep117774-21processes07:48:131
161862579,20sleep20-21swapper/209:27:542
161862579,20sleep20-21swapper/209:27:542
811425613,21sleep00-21swapper/007:18:040
440725612,26sleep24401-21apt-get07:07:562
2740025615,25sleep027394-21apt-get10:02:560
2375525610,30sleep123800-21latency08:08:071
233725616,24sleep22386-21ntp_states08:43:082
2267225616,24sleep022666-21apt-get09:47:590
2006125614,19sleep00-21swapper/009:38:380
1725725615,25sleep017249-21apt-get07:47:590
1701625617,22sleep217156-21sed07:47:452
1068025615,19sleep00-21swapper/010:53:030
570725517,18sleep10-21swapper/107:12:441
278682555,39sleep10-21swapper/108:22:361
2426925516,23sleep024264-21apt-get09:52:550
2006025514,24sleep20-21swapper/207:57:422
1786425515,19sleep00-21swapper/009:33:010
173825515,24sleep11733-21apt-get12:07:531
173825515,24sleep11733-21apt-get12:07:531
1474725512,19sleep20-21swapper/209:23:012
696125420,16sleep10-21swapper/108:58:091
2971425412,26sleep229768-21ls08:27:582
1369725412,18sleep20-21swapper/211:02:502
112902547,13sleep10-21swapper/110:53:161
103462549,22sleep10-21swapper/110:52:441
2678225319,17sleep00-21swapper/008:18:030
1498525317,21sleep115015-21munin-run11:07:411
1257325316,21sleep012575-21apt-get07:32:570
325825211,23sleep10-21swapper/112:12:501
2642225214,23sleep126405-21apt-get11:43:021
2316125214,22sleep223247-21mailstats09:48:132
230022526,34sleep10-21swapper/108:05:161
209422525,33sleep20-21swapper/211:24:142
1578725213,24sleep115790-21diskstats07:43:041
1366225215,18sleep20-21swapper/209:18:132
1366225215,18sleep20-21swapper/209:18:132
947725115,20sleep29474-21apt-get07:22:582
37325111,25sleep2438-21if_eth012:03:022
37325111,25sleep2438-21if_eth012:03:022
2948425015,19sleep10-21swapper/110:08:101
1863825014,15sleep20-21swapper/211:18:042
178925013,16sleep00-21swapper/010:23:110
178925013,16sleep00-21swapper/010:23:110
623024916,16sleep10-21swapper/110:38:061
2640024911,16sleep20-21swapper/208:17:452
2120824913,21sleep121212-21diskstats09:43:031
205172497,18sleep20-21swapper/207:58:032
1764224911,22sleep017732-21users11:13:220
1707124919,16sleep10-21swapper/111:13:111
137942499,17sleep00-21swapper/011:02:560
92724813,18sleep00-21swapper/010:22:430
92724813,18sleep00-21swapper/010:22:430
3240524811,21sleep132533-21latency_hist12:02:451
3240524811,21sleep132533-21latency_hist12:02:451
245422449,17sleep10-21swapper/109:53:041
50692428,15sleep10-21swapper/112:18:031
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional