You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-08 - 19:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Mon Jun 08, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27853210068,982sleep327940-21munin-node07:13:003
27853210068,982sleep327940-21munin-node07:13:003
1189326999,676sleep311938-21ntpq11:28:223
31977251916,489sleep131970-21apt-get12:33:111
31977251916,489sleep131970-21apt-get12:33:111
21485242610,404sleep321526-21awk11:58:213
21485242610,404sleep321526-21awk11:58:213
1546023864,371sleep325556-1kworker/3:0H09:58:313
30689237013,255sleep330846-21tail07:18:293
1441223379,313sleep314484-21cat08:12:593
5410232726,152sleep35497-21fschecks_time07:43:153
5410232726,152sleep35497-21fschecks_time07:43:153
5930232113,296sleep36017-21perl09:28:163
27016228619,243sleep027057-21awk12:17:560
1426226512,131sleep31447-21latency_hist10:57:583
8406223212,15sleep20-21swapper/211:18:192
11641221333,100sleep311703-21fschecks_time08:03:183
25516219911,171sleep325669-21sort12:12:593
27398219617,167sleep00-21swapper/010:35:300
1022218934,64sleep31059-21ntp_kernel_err07:28:223
1677218115,143sleep30-21swapper/310:58:143
20222218022,60sleep30-21swapper/311:53:333
28505217714,141sleep30-21swapper/307:13:163
13373217610,148sleep313422-21latency_hist09:52:583
28581217319,52sleep30-21swapper/310:38:283
1363921730,11sleep30-21swapper/309:53:133
30068217111,63sleep330165-21timerandwakeup10:43:283
31681216910,142sleep00-21swapper/012:32:500
9464216515,57sleep30-21swapper/309:38:253
331721606,142sleep33350-21grep11:03:183
1533221605,13sleep10-21swapper/109:58:001
27091215911,129sleep327102-21sort08:53:073
25889215723,110sleep125944-21gpgv07:07:571
437121549,61sleep30-21swapper/309:23:153
437121549,61sleep30-21swapper/309:23:153
31050214818,112sleep231178-21cat12:28:282
22969214516,114sleep30-21swapper/310:22:573
16988214322,105sleep30-21swapper/310:03:113
204221406,13sleep30-21swapper/309:13:433
3197421386,34sleep30-21swapper/312:33:093
3197421386,34sleep30-21swapper/312:33:093
17494213644,37sleep317508-21modprobe08:22:563
6582213127,81sleep06608-21awk07:47:560
6582213127,81sleep06608-21awk07:47:560
31690213110,87sleep331732-21true12:32:573
665212932,81sleep3636-21/usr/sbin/munin09:12:583
23952212910,102sleep324059-21latency_hist12:07:583
27291212213,87sleep30-21swapper/310:34:083
1979421227,98sleep30-21swapper/310:12:563
29742212118,47sleep30-21swapper/309:00:063
32654211818,11sleep30-21swapper/307:25:553
29893211717,12sleep30-21swapper/312:24:453
23445211715,50sleep30-21swapper/308:39:233
26856211631,76sleep327860-21munin-node12:15:533
24224211618,47sleep30-21swapper/310:24:393
3223021155,12sleep30-21swapper/310:52:333
17546211522,27sleep20-21swapper/208:22:592
10595211411,15sleep30-21swapper/311:24:223
18766211312,20sleep10-21swapper/110:08:181
15916211336,11sleep30-21swapper/308:17:553
735211115,79sleep2830-21cut07:28:162
16240211117,68sleep00-21swapper/008:18:130
30984210813,78sleep131049-21switchtime12:28:271
19020210813,77sleep019082-21/usr/sbin/munin10:08:250
3862210718,76sleep03893-21fschecks_count07:38:200
2409421079,43sleep323984-21/usr/sbin/munin08:43:143
12812210724,62sleep112834-21modprobe08:07:561
816321060,10sleep08155-21munin-run07:52:560
14994210611,74sleep115040-21/usr/sbin/munin11:38:251
30413210529,40sleep30-21swapper/309:03:173
32553210322,29sleep20-21swapper/210:53:122
9934210221,63sleep19439-21apt-get11:23:071
9001210222,44sleep30-21swapper/307:53:293
8792210210,15sleep30-21swapper/311:18:283
2608221028,46sleep326182-21sed10:32:583
25233210222,62sleep125269-21wc10:28:241
15023210118,61sleep114624-21/usr/sbin/munin08:13:211
16168210022,42sleep30-21swapper/311:43:113
808829918,13sleep30-21swapper/307:52:243
782629918,11sleep30-21swapper/311:17:483
782629918,11sleep30-21swapper/311:17:483
37752998,78sleep33766-21apt-get07:38:153
1566729919,11sleep30-21swapper/311:40:123
130482998,79sleep313038-21apt-get11:33:143
714329816,68sleep17226-21awk11:14:231
714329816,68sleep17226-21awk11:14:231
363129811,72sleep13655-21ntp_kernel_pll_11:03:241
2666529818,12sleep30-21swapper/308:50:533
1334029820,62sleep20-21swapper/208:08:172
835429718,12sleep30-21swapper/309:34:133
584029718,11sleep30-21swapper/311:10:213
584029718,11sleep30-21swapper/311:10:213
2836729718,11sleep30-21swapper/312:20:013
2771329719,62sleep027737-21latency_hist07:12:560
2771329719,62sleep027737-21latency_hist07:12:560
240429720,44sleep30-21swapper/307:33:183
190432978,17sleep30-21swapper/308:27:563
1831829717,54sleep20-21swapper/210:07:582
1794729728,51sleep017442-21apt-get11:48:090
1161229718,11sleep30-21swapper/309:45:553
999629625,42sleep09988-21apt-get07:58:100
2116629618,11sleep30-21swapper/310:15:303
2087729621,44sleep30-21swapper/308:33:093
146422968,59sleep00-21swapper/008:13:080
1120129615,13sleep30-21swapper/308:02:093
394229522,11sleep30-21swapper/309:22:183
2035729515,11sleep30-21swapper/308:29:543
114829516,62sleep11186-21grep09:13:121
864129414,63sleep18710-21systemctl11:18:261
1697729413,65sleep117069-21users11:43:291
994629313,63sleep29931-21apt-get11:23:122
1999829320,54sleep120126-21timerwakeupswit08:28:271
1899129315,11sleep30-21swapper/311:50:013
1543929320,53sleep215420-21apt-get09:58:182
558129214,64sleep15586-21systemd-run09:27:571
558129214,64sleep15586-21systemd-run09:27:571
166072926,11sleep10-21swapper/108:18:221
2930629119,56sleep129412-21timerandwakeup08:58:261
2641629021,50sleep126456-21sort10:33:171
2594629028,37sleep226045-21apt-config07:07:582
131102900,8sleep10-21swapper/108:08:111
3226028929,36sleep20-21swapper/210:52:562
1047528914,59sleep010481-21ntp_states11:24:190
1346028713,55sleep013473-21/usr/sbin/munin08:08:200
825028616,46sleep08345-21fschecks_time11:18:170
2811528614,58sleep128166-21if_eth010:38:171
111428631,35sleep01131-21/usr/sbin/munin09:13:070
777528416,51sleep10-21swapper/107:48:301
321028414,52sleep12708-21/usr/sbin/munin09:18:251
2527028419,25sleep20-21swapper/210:28:252
131352846,12sleep00-21swapper/011:33:190
377828314,54sleep13843-21cpuspeed07:38:171
96128222,33sleep2626-21apt-get09:13:012
11872827,48sleep20-21swapper/209:13:132
78228121,43sleep2812-21munin-run12:37:562
78228121,43sleep2812-21munin-run12:37:562
2709528121,35sleep20-21swapper/212:17:582
1837128117,47sleep117924-21/usr/sbin/munin11:48:541
3076528017,45sleep130898-21latency_hist10:47:581
436327917,30sleep20-21swapper/209:23:142
436327917,30sleep20-21swapper/209:23:142
2941327925,31sleep20-21swapper/208:58:272
289672799,56sleep221001-1kworker/2:1H12:23:152
2438927918,46sleep124426-21ntp_kernel_pll_08:43:211
2427927911,52sleep124370-21fschecks_time12:08:161
1941227919,21sleep00-21swapper/008:28:140
3130727817,16sleep10-21swapper/107:22:591
2525827832,27sleep325406-21grep07:03:273
2495027829,29sleep00-21swapper/007:03:200
246427822,28sleep00-21swapper/007:33:190
2149627822,30sleep20-21swapper/208:33:282
1625527815,42sleep016288-21grep11:43:160
2895727720,29sleep00-21swapper/008:58:170
263762770,12sleep20-21swapper/210:33:132
2261527715,43sleep222633-21apt12:03:092
1121527719,30sleep10-21swapper/109:43:271
2749327618,31sleep10-21swapper/108:53:221
2076527621,41sleep220766-21cron11:57:562
1342527617,16sleep00-21swapper/009:52:590
821927518,24sleep20-21swapper/207:52:592
2241527516,35sleep122564-21unixbench_singl10:18:271
1027027511,50sleep010370-21latency_hist09:42:580
604027424,25sleep00-21swapper/011:12:560
604027424,25sleep00-21swapper/011:12:560
2921327411,48sleep229254-21cut10:42:592
2895227413,36sleep00-21swapper/012:23:130
259272745,61sleep00-21swapper/010:31:340
71127314,40sleep1796-21timerandwakeup10:53:271
611727322,25sleep20-21swapper/211:12:582
611727322,25sleep20-21swapper/211:12:582
219727312,45sleep22264-21df07:33:132
1203227214,42sleep012050-21/usr/sbin/munin09:48:060
1493627121,30sleep214979-21munin-run09:57:572
554327026,22sleep00-21swapper/009:27:560
554327026,22sleep00-21swapper/009:27:560
2084627023,26sleep00-21swapper/011:57:580
195627014,30sleep00-21swapper/007:32:590
1437127016,41sleep21-21systemd11:37:572
1373127017,31sleep113880-21tail11:33:321
589626920,27sleep20-21swapper/207:43:262
589626920,27sleep20-21swapper/207:43:262
2045126916,29sleep20-21swapper/210:13:192
606826817,34sleep16108-21unin-run11:12:571
606826817,34sleep16108-21unin-run11:12:571
57772677,41sleep00-21swapper/009:28:000
30602678,13sleep00-21swapper/011:02:590
2285126713,24sleep00-21swapper/012:03:190
2118826710,10sleep00-21swapper/010:15:480
35026622,26sleep2378-21munin-run07:27:562
3130126618,30sleep031305-21systemd-run07:22:580
1217126610,40sleep112266-21perl09:48:161
1175626624,20sleep00-21swapper/011:28:180
315222657,46sleep00-21swapper/007:23:060
3078826513,35sleep030859-21sensors_temp09:03:260
2654426516,29sleep10-21swapper/107:08:171
2654426516,29sleep10-21swapper/107:08:171
2465926513,36sleep224719-21/usr/sbin/munin12:08:252
1182426521,24sleep20-21swapper/208:03:212
2946326415,34sleep128855-21/usr/sbin/munin12:23:261
2869126414,21sleep20-21swapper/207:13:192
2842226424,24sleep00-21swapper/007:13:130
2568326420,27sleep225769-21/usr/sbin/munin08:48:182
2128026415,30sleep021324-21ntpq08:33:220
1182126419,22sleep00-21swapper/009:47:580
896326321,24sleep20-21swapper/209:38:112
2836926310,38sleep00-21swapper/010:38:240
2358626321,27sleep123609-21netstat10:23:201
2254626322,23sleep10-21swapper/108:38:161
2145326317,23sleep20-21swapper/211:58:202
2145326317,23sleep20-21swapper/211:58:202
1756626317,27sleep217766-21dpkg11:47:582
1587726319,28sleep015906-21munin-run11:42:560
879126220,22sleep10-21swapper/107:53:231
61072627,12sleep20-21swapper/209:28:192
41892629,10sleep20-21swapper/211:04:062
2863526215,24sleep10-21swapper/112:22:581
280432626,44sleep20-21swapper/208:53:392
1701926211,35sleep117062-21df10:03:131
1286526218,23sleep00-21swapper/008:07:590
12432628,10sleep00-21swapper/010:56:290
3095926113,29sleep231102-21/usr/sbin/munin09:03:282
2758226115,13sleep00-21swapper/008:53:250
206792616,11sleep10-21swapper/111:56:501
2035826113,21sleep10-21swapper/110:13:181
191002616,11sleep20-21swapper/211:51:262
1815526119,22sleep10-21swapper/108:23:221
656226022,20sleep00-21swapper/011:13:080
656226022,20sleep00-21swapper/011:13:080
653926015,28sleep26065-21apt-get11:13:062
653926015,28sleep26065-21apt-get11:13:062
315352609,34sleep031607-21sensors_temp10:48:260
220492607,12sleep20-21swapper/210:18:202
2096426018,19sleep10-21swapper/108:33:151
187026020,21sleep00-21swapper/010:58:190
1696226013,19sleep00-21swapper/010:03:080
1512926016,26sleep015329-21apt-config09:57:590
90725915,27sleep01061-21apt-config12:37:580
90725915,27sleep01061-21apt-config12:37:580
326925916,19sleep20-21swapper/209:18:262
3133925913,21sleep20-21swapper/207:23:002
1908425919,25sleep019094-21munin-run08:27:570
1850825914,31sleep218526-21/usr/sbin/munin10:08:072
1487425918,22sleep10-21swapper/109:57:561
1155925912,32sleep211570-21df_inode11:28:122
1017925913,29sleep210232-21/usr/sbin/munin07:58:162
871425817,20sleep00-21swapper/009:37:570
3238925817,17sleep00-21swapper/009:08:270
2168325817,25sleep021762-21/usr/sbin/munin11:58:260
2168325817,25sleep021762-21/usr/sbin/munin11:58:260
2108425811,32sleep121077-21apt-get11:58:111
2108425811,32sleep121077-21apt-get11:58:111
1504125714,27sleep215077-21ntp_states11:38:262
896225614,24sleep18953-21apt-get09:38:101
88225616,19sleep10-21swapper/109:13:001
415825620,19sleep20-21swapper/207:38:262
2890425616,21sleep10-21swapper/107:13:261
2779025613,28sleep027796-21ntp_states12:18:250
194825610,23sleep20-21swapper/210:58:212
1195225613,29sleep111958-21ntpq11:28:241
1008825611,17sleep10-21swapper/107:58:151
842425514,20sleep00-21swapper/007:53:130
2776325516,22sleep227735-21latency_hist07:12:582
2776325516,22sleep227735-21latency_hist07:12:582
2410125513,22sleep024192-21perl08:43:150
2290525514,22sleep10-21swapper/112:03:201
228242557,39sleep00-21swapper/010:21:330
64525413,24sleep1636-21apt-get07:28:111
581225412,18sleep10-21swapper/107:43:251
581225412,18sleep10-21swapper/107:43:251
3160825422,18sleep10-21swapper/110:48:271
2945425414,23sleep229446-21apt-get10:43:112
1332425413,19sleep20-21swapper/209:52:572
3086025314,16sleep10-21swapper/109:03:271
2621125315,19sleep10-21swapper/112:13:251
2321925314,23sleep023211-21apt-get10:23:100
2245025313,15sleep20-21swapper/208:38:092
1599925320,17sleep20-21swapper/208:17:582
842125213,23sleep28417-21apt-get07:53:112
738225212,25sleep27378-21apt-get09:33:112
2770625216,18sleep20-21swapper/212:18:232
211025213,22sleep12129-21ntp_states10:58:241
1068525217,18sleep20-21swapper/209:43:162
83122516,32sleep10-21swapper/109:33:431
3075425112,22sleep030764-21munin-run10:47:570
3162725013,21sleep131634-21df_inode07:23:151
270725013,20sleep02726-21apt-get09:18:080
2682325014,21sleep026825-21cron08:52:560
265825010,20sleep10-21swapper/107:33:241
2556125014,18sleep10-21swapper/108:48:131
245412509,20sleep20-21swapper/208:43:262
2419025012,22sleep024183-21apt-get12:08:110
2246025011,23sleep022453-21apt-get08:38:110
1932324912,21sleep219326-21apt-get08:28:082
2370324812,21sleep223767-21mailstats10:23:252
1785724511,22sleep217866-21df_inode08:23:152
714324313,14sleep10-21swapper/109:32:581
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional