You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-29 - 11:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Sat Aug 29, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
115372141525,1375sleep210500-21kworker/2:119:04:262
9755280514,776sleep39757-21ntp_states20:39:403
9580255211,524sleep19615-21ntpq00:04:221
3032237985,219sleep03046-21fschecks_count23:44:150
1053923279,305sleep010542-21cron20:43:570
198262281104,154sleep019890-21mailstats22:54:260
313202241104,107sleep00-21swapper/021:48:590
29130221947,110sleep00-21swapper/019:59:170
28652221613,143sleep028683-21idleruntime-cro23:23:570
28652221613,143sleep028683-21idleruntime-cro23:23:570
32645221251,107sleep00-21swapper/020:09:260
32645221251,107sleep00-21swapper/020:09:260
14168220937,114sleep014288-21munin-node19:14:000
3058219112,162sleep018611-1kworker/0:0H20:19:110
23365216421,115sleep323440-21grep19:39:263
21315216438,102sleep00-21swapper/022:59:230
1391321648,61sleep00-21swapper/020:54:100
27024216239,105sleep027247-21latency_hist23:19:000
27024216239,105sleep027247-21latency_hist23:19:000
6242216019,32sleep012483-21cyclictest20:29:160
2507215710,130sleep32587-21dpkg20:18:583
24451215118,106sleep00-21swapper/019:44:160
12881215133,101sleep013002-21latency_hist22:34:000
12881215133,101sleep013002-21latency_hist22:34:000
12578215164,35sleep012630-21if_err_eth020:49:180
2589215012,121sleep02680-21apt-config20:18:590
1126214921,108sleep11182-21awk21:54:251
32175214641,17sleep00-21swapper/021:49:280
7558214522,92sleep20-21swapper/222:14:292
24130214454,75sleep024195-21date19:43:590
12470213920,97sleep112689-21apt-config19:09:001
6623213621,90sleep16661-21sshd20:29:251
19169213659,34sleep019199-21idleruntime-cro19:28:570
26671213518,95sleep226820-21latency_hist21:34:002
13092213510,106sleep013112-21apt-get22:34:080
13092213510,106sleep013112-21apt-get22:34:080
18368213360,54sleep018525-21latency_hist00:34:000
7401213258,56sleep07544-21latency_hist23:59:000
7387213212,105sleep07479-21mailstats22:14:260
20503213161,25sleep020594-21fschecks_time21:14:160
17882213010,105sleep017960-21grep22:49:180
17912212614,97sleep117982-21/usr/sbin/munin19:24:141
1645021268,16sleep00-21swapper/019:19:170
5241212524,81sleep15390-21grep23:49:291
20254212542,64sleep00-21swapper/019:29:300
25340212441,66sleep00-21swapper/021:29:140
736212312,88sleep00-21swapper/021:54:160
5592212230,27sleep05610-21cut22:09:180
16122212126,67sleep00-21swapper/020:59:270
3050211811,27sleep00-21swapper/021:59:290
22253211830,21sleep022229-21apt-get21:19:210
15228211813,89sleep00-21swapper/000:23:590
18475211724,67sleep118685-21latency_hist21:09:001
18475211724,67sleep118685-21latency_hist21:09:001
19230211626,18sleep00-21swapper/021:10:490
16330211633,67sleep00-21swapper/022:44:170
2406121149,90sleep324154-21cut21:24:203
2406121149,90sleep324154-21cut21:24:203
16782211432,19sleep016787-21munin-run00:28:590
23792211313,78sleep00-21swapper/021:24:130
23792211313,78sleep00-21swapper/021:24:130
3786211222,16sleep00-21swapper/022:04:130
23782211211,82sleep023917-21latency_hist23:08:590
7380211020,74sleep37398-21ls23:58:593
29454211077,16sleep00-21swapper/023:24:210
12818211017,76sleep212283-21apt-get19:09:102
17019210918,67sleep117149-21/usr/sbin/munin22:44:311
3315210720,63sleep12923-21/usr/sbin/munin23:44:201
2976221059,79sleep029771-21munin-run21:43:590
17421210511,79sleep217462-21ntpq00:29:212
1235021030,7sleep012-21rcu_sched19:08:580
14623210229,48sleep314772-21unixbench_multi20:54:333
19782999,20sleep00-21swapper/023:39:270
476829811,50sleep04857-21cat23:49:190
246772988,77sleep024809-21cat23:09:280
941429712,71sleep19512-21fschecks_time20:39:341
1966629719,53sleep319667-21munin-node19:29:093
1081329714,64sleep310818-21/usr/sbin/munin00:09:163
1028429717,68sleep110343-21latency22:24:201
271229619,57sleep12835-21wc23:44:001
1454629610,69sleep214620-21find20:54:322
813429516,62sleep00-21swapper/023:59:290
3203829529,36sleep132162-21cat23:33:591
151792959,44sleep015243-21sendmail_mailst22:39:250
1109629515,69sleep010708-21/usr/sbin/munin00:09:210
1857029415,16sleep00-21swapper/000:34:120
2792929318,54sleep127995-21processes19:54:261
2168529320,55sleep121742-21sendmail-msp21:18:571
1131129322,51sleep211398-21cat00:09:272
895029117,27sleep30-21swapper/322:19:263
1832529013,63sleep318365-21latency_hist00:33:593
1733428953,19sleep00-21swapper/021:04:190
1733428953,19sleep00-21swapper/021:04:190
1368528915,57sleep113762-21latency_hist00:18:591
268228825,44sleep12779-21latency_hist20:19:001
2589128813,37sleep10-21swapper/119:49:091
1527528821,44sleep115325-21latency_hist20:58:591
1111228819,58sleep00-21swapper/022:26:540
855028719,47sleep18583-21fschecks_count22:19:161
2722728710,60sleep00-21swapper/019:54:000
1369828722,33sleep20-21swapper/220:54:002
904328617,53sleep39190-21sed20:39:003
532928620,28sleep30-21swapper/322:09:093
2318728618,30sleep20-21swapper/219:39:212
124002868,13sleep00-21swapper/000:14:140
1121828614,55sleep111310-21apt.systemd.dai00:09:261
592028549,19sleep00-21swapper/020:29:000
269262858,62sleep00-21swapper/021:34:140
1301928525,32sleep313122-21fschecks_time19:10:283
274642847,61sleep00-21swapper/019:54:120
2664728416,54sleep126670-21sed21:33:591
2290528413,51sleep122795-21/usr/sbin/munin19:39:161
2179428426,32sleep30-21swapper/321:18:593
147782848,61sleep00-21swapper/019:14:120
140328411,19sleep20-21swapper/223:39:122
3121428314,49sleep331295-21mailstats23:29:253
2943328317,45sleep129466-21ntp_kernel_pll_19:59:221
290828322,37sleep33046-21grep21:59:283
109002837,60sleep00-21swapper/020:44:130
50782820,6sleep30-21swapper/320:24:263
1233128214,46sleep312398-21/usr/sbin/munin00:14:133
1035228215,56sleep00-21swapper/000:07:480
82842810,6sleep20-21swapper/220:34:262
2814428121,38sleep328174-21modprobe21:38:573
221542815,11sleep00-21swapper/019:34:310
94902805,41sleep00-21swapper/022:20:560
319462805,11sleep00-21swapper/023:33:250
1009728013,53sleep210099-21sshd00:04:352
2224127912,13sleep30-21swapper/321:19:123
1017327917,41sleep39989-21/usr/sbin/munin22:24:183
3236127828,27sleep20-21swapper/223:34:162
322472775,11sleep00-21swapper/023:34:060
318422776,11sleep00-21swapper/020:08:500
234582777,10sleep10-21swapper/121:23:251
1130927714,27sleep30-21swapper/320:44:243
86202765,10sleep00-21swapper/020:34:330
509527620,39sleep25170-21latency_hist22:08:592
2354227518,44sleep223561-21latency_hist21:23:592
199127525,29sleep30-21swapper/321:58:593
158942757,10sleep10-21swapper/119:17:001
734227420,36sleep37382-21sshd22:14:253
613127426,22sleep20-21swapper/223:54:182
486827413,13sleep20-21swapper/223:49:212
3095727420,39sleep230984-21awk20:04:202
2250427425,18sleep30-21swapper/319:38:583
1466827417,27sleep30-21swapper/322:39:083
350427315,41sleep33540-21ntp_kernel_pll_20:20:073
2868527322,22sleep10-21swapper/123:23:581
2868527322,22sleep10-21swapper/123:23:581
1845527315,32sleep20-21swapper/219:24:272
3240727219,30sleep10-21swapper/120:09:191
3240727219,30sleep10-21swapper/120:09:191
2958927210,52sleep326277-21kworker/3:221:42:163
278127215,37sleep22964-21sed20:19:012
253122728,45sleep30-21swapper/321:29:093
3155727117,23sleep10-21swapper/121:49:131
17242716,12sleep20-21swapper/221:56:322
39627021,23sleep10-21swapper/123:34:271
3109627016,41sleep231106-21netstat23:29:202
3049127014,16sleep10-21swapper/123:28:581
2977227020,30sleep10-21swapper/121:44:001
2692427019,24sleep30-21swapper/323:18:593
2692427019,24sleep30-21swapper/323:18:593
1796127018,32sleep10-21swapper/122:49:191
741626911,39sleep20-21swapper/220:33:582
1401026922,22sleep10-21swapper/100:19:151
789826814,24sleep10-21swapper/123:59:211
469626815,39sleep24740-21grep20:24:182
3035426819,31sleep230402-21cat20:03:592
2067826822,21sleep10-21swapper/122:58:591
3050426712,41sleep230585-21sshd21:44:252
2610426719,29sleep226156-21/usr/sbin/munin19:49:182
1979926713,24sleep30-21swapper/322:54:253
1107126716,34sleep311077-21diskstats19:04:153
3067626625,22sleep10-21swapper/120:04:151
2683726618,25sleep20-21swapper/223:18:582
2683726618,25sleep20-21swapper/223:18:582
1817826623,20sleep30-21swapper/319:24:203
1673226615,20sleep20-21swapper/200:28:572
1358326615,32sleep113624-21/usr/sbin/munin22:34:251
1358326615,32sleep113624-21/usr/sbin/munin22:34:251
1351626622,25sleep20-21swapper/222:34:222
1351626622,25sleep20-21swapper/222:34:222
31452658,13sleep20-21swapper/223:44:172
2100626516,27sleep20-21swapper/219:34:002
1886326519,29sleep218860-21apt-get21:09:092
1091326519,25sleep20-21swapper/220:44:152
631426429,18sleep10-21swapper/123:54:211
2822526423,18sleep10-21swapper/121:38:591
2806826419,22sleep30-21swapper/319:54:283
2436426416,29sleep20-21swapper/221:24:272
2436426416,29sleep20-21swapper/221:24:272
2151126417,21sleep10-21swapper/119:34:201
2100126415,21sleep20-21swapper/222:59:172
1995726419,23sleep20-21swapper/222:54:282
1592726414,36sleep115933-21ntp_states00:24:241
1211526417,30sleep312177-21/usr/sbin/munin22:29:273
1211526417,30sleep312177-21/usr/sbin/munin22:29:273
349426316,26sleep10-21swapper/122:03:581
319322637,13sleep20-21swapper/221:49:222
2409826312,36sleep324102-21diskstats23:09:153
1591626318,22sleep30-21swapper/320:59:223
809926220,25sleep10-21swapper/120:34:221
3179126215,20sleep30-21swapper/321:49:193
271572626,12sleep20-21swapper/221:34:202
2017526223,19sleep30-21swapper/321:13:593
1636626217,25sleep10-21swapper/119:19:141
1548726220,21sleep10-21swapper/120:59:071
1414226215,29sleep314191-21irqstats00:19:173
73192617,11sleep30-21swapper/320:32:423
405326112,34sleep34122-21sort22:04:203
1693926119,14sleep30-21swapper/319:19:283
106826119,18sleep20-21swapper/220:14:002
106826119,18sleep20-21swapper/220:14:002
894126018,18sleep20-21swapper/200:03:582
1904926014,30sleep319053-21modprobe22:53:573
1311726015,28sleep313112-21apt-get22:34:113
1311726015,28sleep313112-21apt-get22:34:113
1164926013,32sleep111538-21/usr/sbin/munin22:29:171
1164926013,32sleep111538-21/usr/sbin/munin22:29:171
3109125914,25sleep30-21swapper/320:04:253
2414625914,18sleep10-21swapper/123:09:171
209832599,14sleep20-21swapper/221:14:262
1978525914,14sleep10-21swapper/119:29:191
1884125915,20sleep30-21swapper/300:34:193
1743325919,24sleep20-21swapper/221:04:212
1743325919,24sleep20-21swapper/221:04:212
1713825915,29sleep317145-21sh21:04:143
1713825915,29sleep317145-21sh21:04:143
1317125912,31sleep113193-21irqstats19:10:301
461025814,26sleep14605-21apt-get20:24:131
354525821,20sleep20-21swapper/222:03:592
2510225821,16sleep10-21swapper/119:44:291
2444025821,20sleep30-21swapper/319:44:153
185392584,41sleep10-21swapper/100:34:051
1755525817,19sleep20-21swapper/222:48:592
1282925815,27sleep2289-21dbus-daemon22:33:582
1282925815,27sleep2289-21dbus-daemon22:33:582
49172576,16sleep10-21swapper/122:07:211
2915125716,25sleep20-21swapper/223:24:072
2912125711,32sleep329012-21/usr/sbin/munin19:59:163
1944325713,17sleep10-21swapper/121:10:541
40962565,41sleep20-21swapper/220:21:032
352325616,27sleep33621-21cat23:44:263
3237925612,18sleep30-21swapper/323:34:173
3037325614,24sleep130400-21awk21:44:201
247562569,19sleep20-21swapper/219:44:222
2247225615,24sleep222466-21apt-get23:04:132
1010025620,16sleep20-21swapper/222:24:172
472125514,25sleep34766-21irqstats23:49:183
275052558,31sleep327512-21diskstats23:19:403
275052558,31sleep327512-21diskstats23:19:403
2273325516,20sleep10-21swapper/123:04:201
1546925515,24sleep315466-21apt-get00:24:103
1532625514,18sleep20-21swapper/220:59:002
1496725516,19sleep20-21swapper/222:39:192
666325411,19sleep20-21swapper/222:14:002
663625420,19sleep10-21swapper/122:13:591
1912225415,24sleep20-21swapper/200:34:262
339625314,22sleep10-21swapper/120:20:051
3250925312,17sleep30-21swapper/320:09:213
3250925312,17sleep30-21swapper/320:09:213
2379525313,24sleep123863-21df21:24:141
2379525313,24sleep123863-21df21:24:141
1739025316,17sleep10-21swapper/100:29:201
1450525313,22sleep314642-21users19:14:033
1247425315,17sleep30-21swapper/320:49:173
1238225312,24sleep112377-21apt-get20:49:111
2739425211,23sleep227384-21apt-get23:19:142
2739425211,23sleep227384-21apt-get23:19:142
1606925216,18sleep20-21swapper/219:18:582
846725117,19sleep20-21swapper/222:19:132
2591425112,23sleep325910-21apt-get19:49:113
2283925117,15sleep30-21swapper/323:04:223
1802625116,16sleep20-21swapper/222:49:202
1082725113,20sleep10-21swapper/120:44:101
822825017,16sleep10-21swapper/122:18:591
3025325017,16sleep30-21swapper/321:44:183
2973625018,19sleep329814-21sed23:24:283
2556925013,24sleep125585-21cpu23:14:131
2556925013,24sleep125585-21cpu23:14:131
2099425012,23sleep121000-21diskstats22:59:161
139925012,21sleep31396-21apt-get23:39:093
1233025011,22sleep212323-21apt-get00:14:122
1022725014,21sleep20-21swapper/220:39:552
2090324910,23sleep320910-21apt-get22:59:103
2845824812,20sleep228455-21apt-get21:39:092
1657424814,16sleep20-21swapper/219:19:192
2799724713,16sleep20-21swapper/219:54:272
2441124717,16sleep20-21swapper/223:09:222
159292476,33sleep10-21swapper/122:43:381
946224510,19sleep39546-21memory00:04:203
2121724311,18sleep20-21swapper/219:34:082
809024112,15sleep30-21swapper/323:59:273
162452418,15sleep30-21swapper/322:44:123
148202389,16sleep20-21swapper/200:19:292
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional