You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-10 - 04:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Wed Jun 10, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
32446271699,599sleep332547-21unixbench_multi23:18:373
20040270055,441sleep320064-21latency_hist22:42:513
647326596,641sleep36509-21ntpq21:58:163
28034247911,451sleep028075-21ntpq21:23:160
27236233271,216sleep327275-21fschecks_count19:38:113
25142229716,264sleep125144-21sshd19:31:021
25142229716,264sleep125144-21sshd19:31:021
15640227721,216sleep315719-21mailstats20:43:213
6095227251,202sleep36136-21latency20:13:293
2227226313,234sleep22306-21latency_hist20:02:532
23785223319,198sleep323886-21latency_hist19:27:533
9022222947,74sleep39040-21fschecks_count20:23:103
13706220521,162sleep213801-21fschecks_time20:38:112
3155022009,178sleep331598-21latency_hist19:52:533
108321917,54sleep3786-21/usr/sbin/munin23:23:133
4528218940,133sleep34623-21timerandwakeup23:33:233
8938218512,53sleep30-21swapper/322:07:533
10929218212,154sleep010449-21apt-get22:12:590
26175217710,152sleep326237-21grep21:18:103
449921747,11sleep30-21swapper/321:53:073
17530217141,90sleep317577-21ntp_kernel_pll_19:08:183
17530217141,90sleep317577-21ntp_kernel_pll_19:08:183
3616216910,104sleep33634-21idleruntime-cro23:32:503
3616216910,104sleep33634-21idleruntime-cro23:32:503
17175216528,125sleep317195-21dpkg22:33:013
2124216317,126sleep02260-21unixbench_singl21:43:230
30353215910,101sleep330364-21open_files23:13:203
30353215910,101sleep330364-21open_files23:13:203
29681215972,39sleep30-21swapper/321:28:283
2856621587,103sleep328600-21ps23:08:193
2093121578,137sleep320923-21apt19:18:083
2093121578,137sleep320923-21apt19:18:083
11741215341,72sleep311774-21grep20:32:513
22571215213,52sleep30-21swapper/300:33:003
757921519,93sleep30-21swapper/323:43:213
2826214913,119sleep30-21swapper/320:03:153
9447214641,57sleep30-21swapper/322:08:143
1854921456,36sleep30-21swapper/320:53:143
1160214214,54sleep30-21swapper/321:42:543
18018214112,25sleep30-21swapper/320:52:543
20563213974,20sleep30-21swapper/320:58:233
6507213624,91sleep06652-21sed20:13:380
1959221358,114sleep319623-21fschecks_count00:23:103
15302213417,29sleep20-21swapper/200:08:192
12978213418,98sleep2225-21systemd-journal00:02:502
32068213211,48sleep30-21swapper/321:37:533
22559213214,101sleep122700-21unixbench_multi22:48:221
2242121327,112sleep322481-21sensors_temp22:48:203
30912213113,80sleep20-21swapper/221:33:112
29935212812,102sleep230054-21unixbench_singl21:28:312
15403212746,21sleep30-21swapper/322:27:533
2576121248,92sleep00-21swapper/022:58:230
2576121248,92sleep00-21swapper/022:58:230
2563421249,99sleep325759-21sed22:58:223
2563421249,99sleep325759-21sed22:58:223
17955212111,98sleep317979-21cpu00:18:063
3206021199,71sleep332125-21grep19:53:133
2578721197,44sleep30-21swapper/319:33:143
1160621199,95sleep311452-40mandb23:57:543
16060211818,77sleep30-21swapper/322:28:173
27814211623,38sleep30-21swapper/323:07:513
24593211615,85sleep30-21swapper/321:13:133
22151211510,45sleep30-21swapper/321:03:243
2105321147,68sleep321046-21apt-get00:28:073
19300211414,31sleep219443-21cat00:22:532
486721139,15sleep00-21swapper/021:53:170
8691211216,82sleep28725-21latency_hist20:22:522
2827921115,94sleep00-21swapper/019:41:110
8585211013,83sleep18649-21cpuspeed23:48:071
11148211018,75sleep111204-21grep22:14:011
24763210913,80sleep224840-21cut22:57:522
24763210913,80sleep224840-21cut22:57:522
18763210817,78sleep118846-21tr20:53:211
14916210713,80sleep014921-21wc20:42:540
1380421069,40sleep30-21swapper/320:38:123
23226210516,71sleep023320-21/usr/sbin/munin21:08:140
1612421058,39sleep30-21swapper/300:12:533
9547210414,71sleep19586-21ntpq22:08:161
1422521045,51sleep30-21swapper/300:04:003
1356721047,11sleep30-21swapper/322:19:453
10435210424,62sleep010487-21true22:12:510
2291921035,51sleep30-21swapper/321:07:563
3775210217,61sleep13824-21latency_hist20:07:531
23083210212,75sleep123143-21sendmail_mailst00:33:181
20370210211,53sleep320419-21munin-run19:17:513
15545210217,71sleep015557-21acpi19:03:030
16676210115,69sleep016689-21sort20:48:020
5792210012,73sleep15289-21apt-get20:13:021
31334210014,70sleep031415-21timerandwakeup21:33:220
1277521008,82sleep0194-21jbd2/mmcblk2p2-00:00:140
572429927,55sleep05758-21/usr/sbin/munin23:38:130
572429927,55sleep05758-21/usr/sbin/munin23:38:130
2800229911,38sleep30-21swapper/321:23:153
2150929915,66sleep221578-21/usr/sbin/munin19:18:272
2150929915,66sleep221578-21/usr/sbin/munin19:18:272
155429921,60sleep01695-21grep23:23:230
297562986,10sleep30-21swapper/319:45:183
2361129842,34sleep00-21swapper/022:53:140
2361129842,34sleep00-21swapper/022:53:140
1095429818,26sleep30-21swapper/322:13:023
254222967,12sleep00-21swapper/019:32:550
585729526,57sleep35893-21ntpq23:38:163
585729526,57sleep35893-21ntpq23:38:163
1594029433,37sleep115547-21/usr/sbin/munin19:03:161
87262929,45sleep38875-21latency_hist20:22:543
2763729226,49sleep127716-21mailstats19:38:211
2332129218,51sleep023455-21unixbench_singl00:33:210
95932917,29sleep30-21swapper/323:49:433
132042918,71sleep0363-21snmpd20:36:440
1057329121,53sleep010635-21grep20:28:120
1138729025,44sleep011452-40mandb23:57:510
3115928923,48sleep231226-21http23:17:512
3115928923,48sleep231226-21http23:17:512
3033128916,47sleep230395-21fschecks_time19:48:112
250782895,34sleep30-21swapper/319:30:113
250782895,34sleep30-21swapper/319:30:113
177192879,17sleep30-21swapper/300:17:533
1747328711,62sleep117607-21uptime20:48:241
557728624,42sleep25666-21tune2fs23:38:112
557728624,42sleep25666-21tune2fs23:38:112
2995428622,39sleep229983-21awk19:47:512
2548928510,58sleep125557-21mailstats22:58:201
2548928510,58sleep125557-21mailstats22:58:201
2095628512,61sleep0363-21snmpd21:00:590
808428414,55sleep18090-21ntpq22:03:191
988728228,29sleep20-21swapper/223:52:532
2957528224,32sleep129755-21sort23:12:531
236692829,45sleep10-21swapper/122:53:151
236692829,45sleep10-21swapper/122:53:151
1200928245,20sleep30-21swapper/320:33:033
464928111,55sleep04769-21unixbench_singl23:33:240
2038228114,45sleep119812-21/usr/sbin/munin20:58:211
1147628120,32sleep20-21swapper/223:57:532
3179228012,53sleep010142-1kworker/0:1H19:53:060
315528030,34sleep30-21swapper/321:48:113
1020228020,45sleep30-21swapper/323:53:103
609327918,35sleep16178-21timerandwakeup23:38:221
609327918,35sleep16178-21timerandwakeup23:38:221
279112790,6sleep20-21swapper/221:23:132
201127914,47sleep32120-21timerandwakeup21:43:223
201082797,13sleep00-21swapper/019:14:370
2645827816,46sleep026497-21ntpq21:18:150
96252770,8sleep00-21swapper/020:23:220
2692927626,22sleep20-21swapper/219:37:532
2166527618,41sleep221752-21ls00:28:272
1492227621,28sleep10-21swapper/120:42:551
2388927515,17sleep30-21swapper/322:53:213
2388927515,17sleep30-21swapper/322:53:213
813327423,29sleep18266-21grep20:18:231
702927416,30sleep20-21swapper/223:43:032
2936027424,32sleep229381-21munin-run23:12:502
193672745,52sleep10-21swapper/119:13:131
56802736,15sleep10-21swapper/121:56:531
352227316,36sleep03613-21switchtime21:48:200
3198827320,33sleep132012-21munin-run21:37:511
1767027323,30sleep00-21swapper/022:33:190
52752727,46sleep30-21swapper/320:12:523
296227210,48sleep23037-21find23:28:212
296227210,48sleep23037-21find23:28:212
1976827214,40sleep219787-21uname00:23:142
190727220,41sleep30-21swapper/319:59:233
1579727216,40sleep115913-21timerwakeupswit20:43:231
128227223,23sleep10-21swapper/119:58:161
101027221,33sleep01104-21fschecks_time19:58:110
2487527117,40sleep024483-21/usr/sbin/munin21:13:190
207912718,46sleep30-21swapper/322:43:193
1673127115,38sleep216741-21/usr/sbin/munin00:13:202
452427022,29sleep00-21swapper/020:08:210
294927014,40sleep23016-21df21:48:062
1885727013,35sleep10-21swapper/122:38:081
2318726922,26sleep00-21swapper/022:52:530
158402695,43sleep20-21swapper/222:28:122
1381026920,32sleep213823-21apt22:22:512
969126812,44sleep00-21swapper/023:50:570
748826814,38sleep07494-21ntp_states23:43:170
516826820,31sleep25195-21munin-run23:37:502
1525126819,27sleep20-21swapper/220:43:122
1316326813,33sleep013238-21timerandwakeup22:18:250
575526719,22sleep20-21swapper/221:57:512
3270226719,22sleep20-21swapper/221:38:162
18812678,40sleep10-21swapper/121:43:201
1632526722,23sleep10-21swapper/122:28:231
1544026717,25sleep00-21swapper/022:27:540
1535726710,42sleep215402-21systemd-run22:27:522
3084926623,23sleep10-21swapper/119:48:231
3000626618,30sleep030091-21latency_hist19:47:530
1690126621,30sleep317027-21latency_hist19:07:543
129626625,24sleep10-21swapper/123:23:191
326732658,47sleep00-21swapper/023:18:580
310882658,49sleep00-21swapper/023:16:560
310882658,49sleep00-21swapper/023:16:560
2476826517,20sleep20-21swapper/221:13:172
2273826516,27sleep00-21swapper/021:07:530
2247526520,31sleep022476-21cpu19:23:050
1543326521,26sleep00-21swapper/000:08:210
1363026514,24sleep20-21swapper/200:03:192
1213426514,32sleep112241-21fschecks_time20:33:131
2555226417,25sleep10-21swapper/119:33:041
2305126422,22sleep10-21swapper/121:08:111
112882645,14sleep10-21swapper/123:56:351
900426313,34sleep19014-21df_inode20:23:081
469326314,29sleep20-21swapper/220:08:232
428326314,33sleep14298-21kernelversion20:08:141
2817526311,24sleep00-21swapper/023:08:080
1716226317,33sleep217171-21cpu19:08:092
1716226317,33sleep217171-21cpu19:08:092
1554026319,23sleep10-21swapper/100:08:221
1458426316,24sleep00-21swapper/000:07:530
1100626321,22sleep20-21swapper/222:13:382
35132626,46sleep00-21swapper/020:05:060
3226726221,22sleep10-21swapper/119:53:191
1592226220,22sleep00-21swapper/022:28:140
1024726216,20sleep20-21swapper/220:27:532
425326116,17sleep10-21swapper/121:52:531
382626118,22sleep20-21swapper/220:07:542
2896626121,21sleep10-21swapper/119:43:141
2402826116,27sleep024021-21apt-get19:28:060
2402826116,27sleep024021-21apt-get19:28:060
2235426115,30sleep222419-21grep22:48:192
1841326112,29sleep018414-21cron19:12:510
1841326112,29sleep018414-21cron19:12:510
1832626116,26sleep018360-21/usr/sbin/munin00:18:150
3147626010,34sleep131622-21dpkg23:17:541
2264926016,29sleep222684-21/usr/sbin/munin21:07:512
2132726020,17sleep10-21swapper/100:28:171
1802826013,20sleep10-21swapper/100:18:081
1354526014,28sleep10-21swapper/100:03:181
809125920,18sleep20-21swapper/222:03:202
275112599,38sleep20-21swapper/223:03:582
2631925921,16sleep20-21swapper/223:02:532
2631925921,16sleep20-21swapper/223:02:532
2500725915,28sleep125081-21/usr/sbin/munin21:13:241
198022595,18sleep00-21swapper/022:39:490
1688125919,20sleep20-21swapper/219:07:532
3169325812,25sleep231188-21apt-get23:18:032
267692587,12sleep10-21swapper/123:03:121
2322625824,19sleep10-21swapper/119:23:431
1910225817,25sleep219096-21apt-get19:13:062
634325716,26sleep06348-21/usr/sbin/munin21:58:130
616225716,28sleep16221-21fschecks_time21:58:101
2294225717,22sleep20-21swapper/219:23:242
411525616,23sleep14155-21/usr/sbin/munin23:33:131
2948925614,27sleep129561-21memory21:28:221
2702625619,18sleep00-21swapper/023:03:190
2041525614,26sleep20-21swapper/222:43:102
2740025515,24sleep227422-21irqstats19:38:142
2315625511,18sleep20-21swapper/221:08:122
2259625515,24sleep222592-21apt-get00:33:032
2139325516,23sleep121388-21apt-get21:03:061
2091225520,17sleep10-21swapper/122:43:211
2080525512,25sleep020814-21munin-run00:27:520
1886525514,24sleep218869-21diskstats22:38:092
1825525515,23sleep218251-21apt-get20:53:052
1515425513,26sleep015146-21apt-get20:43:060
1217825513,24sleep10-21swapper/123:58:211
972125419,18sleep20-21swapper/222:08:212
2918325411,26sleep029176-21apt-get21:28:050
2669525415,24sleep20-21swapper/221:18:212
2616325414,24sleep126169-21df_inode21:18:081
211125410,30sleep22121-21munin-run23:27:532
1699225417,16sleep10-21swapper/100:13:271
1410225415,18sleep10-21swapper/122:23:061
1292525424,16sleep10-21swapper/122:18:171
607025314,25sleep26067-21apt-get21:58:032
3220125312,22sleep232243-21ntp_states19:53:162
3160225315,18sleep00-21swapper/019:52:540
288425319,14sleep10-21swapper/123:28:201
288425319,14sleep10-21swapper/123:28:201
274525315,16sleep20-21swapper/221:47:532
2127525314,25sleep021326-21irqstats00:28:160
2092925313,24sleep120924-21apt-get19:18:051
2092925313,24sleep120924-21apt-get19:18:051
700725213,19sleep10-21swapper/123:43:021
250922526,33sleep20-21swapper/219:30:222
250922526,33sleep20-21swapper/219:30:222
2081525212,24sleep10-21swapper/100:27:531
1771225213,21sleep10-21swapper/122:33:201
1719825213,17sleep20-21swapper/220:48:212
739225110,23sleep00-21swapper/020:18:060
580325114,22sleep25787-21apt-get20:13:072
3230925112,22sleep132376-21df21:38:071
3042625115,17sleep10-21swapper/123:13:221
3042625115,17sleep10-21swapper/123:13:221
2763525112,23sleep127628-21apt-get21:23:051
235525114,22sleep02351-21apt-get23:28:030
235525114,22sleep02351-21apt-get23:28:030
745425020,13sleep00-21swapper/022:02:540
3258025011,23sleep032673-21memory21:38:140
1838325012,22sleep018464-21fw_packets20:53:110
2130424911,23sleep021348-21ntpq19:18:210
2130424911,23sleep021348-21ntpq19:18:210
2083524917,19sleep020911-21systemctl22:43:200
1854524913,14sleep20-21swapper/222:37:532
2044924812,15sleep20-21swapper/220:58:222
1241324813,17sleep20-21swapper/220:33:182
1761424712,18sleep117620-21ntp_states19:08:201
1761424712,18sleep117620-21ntp_states19:08:201
105602478,16sleep20-21swapper/220:28:102
1048424610,21sleep110481-21apt-get20:28:041
1636324413,16sleep00-21swapper/000:13:060
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional