You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-03 - 10:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Sat Jan 03, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
243502793413,7908sleep24431-21kworker/2:119:14:272
18893280656,733sleep018870-21/usr/sbin/munin00:08:480
22723264619,611sleep222777-21/usr/sbin/munin20:54:092
21368233213,303sleep021412-21ntpq20:49:140
266382288105,20sleep10-21swapper/119:24:071
4416227146,210sleep14505-21fschecks_time21:39:061
24503224896,76sleep124650-21grep00:24:191
10583223310,207sleep210732-21awk23:39:192
24836221648,71sleep124942-21fschecks_time22:44:061
23266221136,107sleep10-21swapper/120:54:201
24837220922,167sleep224980-21grep20:59:222
22247218013,70sleep122250-21cron20:53:471
12321218035,110sleep10-21swapper/122:04:011
2054321785,154sleep00-21swapper/022:29:190
7109217661,22sleep10-21swapper/123:29:121
1707321757,15sleep10-21swapper/122:19:071
9606217441,96sleep19686-21latency_hist23:38:491
9606217441,96sleep19686-21latency_hist23:38:491
21995217414,30sleep20-21swapper/219:09:182
15172217339,94sleep115305-21latency_hist22:13:501
19354216113,132sleep219412-21irqstats00:09:102
16161215844,16sleep10-21swapper/122:14:191
30019215739,103sleep121260-21cyclictest19:34:131
19213215727,18sleep10-21swapper/120:43:501
23533215610,132sleep123571-21modprobe00:23:471
1077021548,132sleep110760-21apt-get21:59:001
1077021548,132sleep110760-21apt-get21:59:001
27492215012,52sleep10-21swapper/121:09:101
27159215011,123sleep027254-21sed19:24:180
25058215027,26sleep30-21swapper/320:59:473
17077215023,102sleep217212-21grep20:34:262
3913214928,107sleep13993-21latency23:19:111
26982214740,23sleep10-21swapper/121:08:491
26982214740,23sleep10-21swapper/121:08:491
19611214727,108sleep119670-21processes00:09:171
32127214615,115sleep132135-21wc21:24:041
32127214615,115sleep132135-21wc21:24:041
15867214526,25sleep10-21swapper/123:58:521
25189214234,35sleep10-21swapper/121:01:231
5189214151,46sleep10-21swapper/123:24:041
28466214110,119sleep128491-21awk19:29:121
28525213713,109sleep028630-21cat21:13:490
2413213713,35sleep32459-21sed23:15:133
2413213713,35sleep32459-21sed23:15:133
18619213712,108sleep018711-21tune2fs22:24:070
25482213517,96sleep30-21swapper/321:03:513
25482213517,96sleep30-21swapper/321:03:513
4814213315,95sleep20-21swapper/221:39:152
339213317,73sleep3424-21timerandwakeup21:25:153
339213317,73sleep3424-21timerandwakeup21:25:153
15800213224,85sleep015830-21awk20:33:480
15800213224,85sleep015830-21awk20:33:480
4946213018,92sleep24954-21latency_hist23:23:482
10456212828,17sleep10-21swapper/123:39:181
9975212631,64sleep10-21swapper/120:14:101
2007421266,14sleep10-21swapper/122:29:001
23767212466,16sleep10-21swapper/122:39:181
29779212328,19sleep10-21swapper/122:59:121
31049212219,53sleep10-21swapper/121:19:191
2620421215,10sleep10-21swapper/119:23:021
7385212015,90sleep37501-21timerwakeupswit23:29:183
23793211927,71sleep023821-21munin-plugin-st20:58:470
14777211713,95sleep00-21swapper/022:09:210
27830211533,60sleep10-21swapper/122:54:041
27830211533,60sleep10-21swapper/122:54:041
15679211514,81sleep015771-21/usr/sbin/munin22:14:090
4121211312,84sleep24155-21/usr/sbin/munin19:55:462
4303211211,85sleep04322-21apt-get21:38:560
1014321115,10sleep10-21swapper/121:57:511
18665211034,30sleep10-21swapper/100:06:231
164921109,78sleep20-21swapper/221:29:132
1198821106,13sleep10-21swapper/123:44:181
32731210912,80sleep10-21swapper/123:09:011
14553210819,74sleep314563-21forks23:54:053
11852210816,78sleep311891-21processes23:44:163
22800210631,27sleep10-21swapper/122:37:441
19443210612,23sleep00-21swapper/020:44:010
1737321065,11sleep10-21swapper/120:36:031
1320021068,84sleep113259-21latency23:49:111
1320021068,84sleep113259-21latency23:49:111
980210415,39sleep11108-21cat19:44:221
2537821048,14sleep10-21swapper/100:29:081
2537821048,14sleep10-21swapper/100:29:081
23681210410,16sleep30-21swapper/319:14:123
4022210322,60sleep04060-21wc19:55:440
2618210311,42sleep10-21swapper/123:15:191
2618210311,42sleep10-21swapper/123:15:191
20505210332,27sleep10-21swapper/120:46:161
2918021028,15sleep10-21swapper/121:14:151
1455121027,21sleep114544-21apt-get20:29:041
1455121027,21sleep114544-21apt-get20:29:041
302721015,10sleep10-21swapper/119:52:501
32318210016,68sleep032336-21fschecks_count21:25:030
32318210016,68sleep032336-21fschecks_count21:25:030
651529916,67sleep36662-21latency_hist20:03:503
2688429916,32sleep30-21swapper/300:33:583
55692977,13sleep10-21swapper/119:59:121
456629717,59sleep34600-21/usr/sbin/munin21:39:083
2521529715,63sleep225274-21sendmail_mailqu22:44:152
2279329628,19sleep10-21swapper/119:13:491
1606329630,16sleep10-21swapper/123:59:041
157629612,69sleep31663-21apt-config23:13:483
2319929512,26sleep30-21swapper/322:39:043
2997329421,51sleep230049-21grep22:59:172
2501729417,63sleep325065-21grep19:19:083
70802935,76sleep10-21swapper/121:48:101
252229332,22sleep12603-21latency_hist21:33:491
233529318,59sleep22402-21systemctl19:49:202
1707829320,53sleep217117-21cut22:19:082
114652937,36sleep10-21swapper/120:19:031
800429220,57sleep38040-21awk20:08:473
2068729232,15sleep10-21swapper/100:14:021
1217629211,17sleep20-21swapper/220:19:202
488329116,50sleep10-21swapper/119:58:481
76212909,62sleep10-21swapper/120:04:211
2635229015,60sleep026502-21latency_hist19:23:500
1987529021,49sleep219397-21apt-get19:03:582
38002895,37sleep10-21swapper/121:35:591
84192886,29sleep10-21swapper/120:09:091
3071228817,47sleep230318-21/usr/sbin/munin21:19:122
2876428813,59sleep228738-21/usr/sbin/munin21:14:032
1382428819,52sleep312903-21/usr/sbin/munin23:49:213
1382428819,52sleep312903-21/usr/sbin/munin23:49:213
242242870,6sleep30-21swapper/300:24:123
2317428616,20sleep10-21swapper/119:13:531
3109928525,26sleep00-21swapper/023:04:080
2885928512,50sleep328943-21tune2fs21:14:083
285328515,51sleep03001-21unixbench_singl23:15:220
285328515,51sleep03001-21unixbench_singl23:15:220
2587728520,47sleep225952-21mailstats00:29:212
2587728520,47sleep225952-21mailstats00:29:212
649328430,34sleep20-21swapper/223:28:492
1999228434,29sleep30-21swapper/319:04:073
3210028330,41sleep10-21swapper/123:07:241
2840328314,52sleep328494-21sed22:54:183
2840328314,52sleep328494-21sed22:54:183
280962836,28sleep10-21swapper/100:37:521
263542837,14sleep20-21swapper/222:49:042
1259328326,31sleep30-21swapper/322:04:113
78802825,29sleep10-21swapper/121:49:151
78802825,29sleep10-21swapper/121:49:151
2340728216,51sleep023423-21awk19:14:010
1367128220,38sleep213823-21sed23:49:202
1367128220,38sleep213823-21sed23:49:202
864228116,41sleep08251-21/usr/sbin/munin23:34:120
864228116,41sleep08251-21/usr/sbin/munin23:34:120
3087628018,44sleep330947-21sensors_temp21:19:173
195672805,29sleep10-21swapper/122:25:571
1425828021,42sleep014276-21awk20:28:470
325182798,50sleep10-21swapper/119:43:501
3061327914,49sleep030644-21latency21:19:100
262472795,28sleep10-21swapper/122:48:511
1839027916,51sleep218454-21run-parts20:39:192
247027822,37sleep32472-21cron21:33:473
2338227814,48sleep023441-21cat22:39:090
877827742,17sleep00-21swapper/021:53:500
3066227722,29sleep30-21swapper/323:03:473
2507927715,45sleep025096-21latency_hist00:28:460
2394427716,30sleep20-21swapper/200:24:082
2126327720,34sleep221335-21latency20:49:122
160727712,51sleep01648-21cut19:48:490
139422775,27sleep10-21swapper/120:24:401
41892767,12sleep00-21swapper/023:19:180
3148427619,28sleep20-21swapper/219:39:102
2611627523,22sleep00-21swapper/022:48:500
2307627515,27sleep30-21swapper/320:54:183
192827515,45sleep31934-21diskstats19:49:093
872227421,30sleep30-21swapper/321:53:493
872227421,30sleep30-21swapper/321:53:493
2908327422,28sleep00-21swapper/022:58:470
2908327422,28sleep00-21swapper/022:58:470
897927314,33sleep09122-21grep20:09:210
1504427314,42sleep215142-21/usr/sbin/munin23:54:172
1477427322,31sleep213837-21/usr/sbin/munin22:09:192
1456627320,25sleep00-21swapper/023:54:060
952627218,35sleep39554-21awk23:38:473
952627218,35sleep39554-21awk23:38:473
2173327219,39sleep321852-21unixbench_singl20:49:213
1759927225,22sleep30-21swapper/300:04:033
105232728,13sleep30-21swapper/321:58:503
105232728,13sleep30-21swapper/321:58:503
3276127119,21sleep20-21swapper/219:44:072
2626927121,29sleep00-21swapper/021:04:200
2626927121,29sleep00-21swapper/021:04:200
179982718,11sleep00-21swapper/022:20:350
123552717,54sleep00-21swapper/023:45:060
856427024,26sleep30-21swapper/320:09:113
2758527023,25sleep20-21swapper/221:09:112
2277227010,13sleep30-21swapper/300:19:173
2052527026,24sleep10-21swapper/119:05:141
1559027017,29sleep20-21swapper/222:14:072
97026922,26sleep00-21swapper/023:10:260
2952126921,24sleep00-21swapper/022:59:070
193626917,35sleep01970-21fschecks_count19:49:100
1876826916,32sleep218508-21/usr/sbin/munin22:24:092
1653026921,29sleep30-21swapper/320:34:153
893226816,26sleep30-21swapper/321:53:573
682726820,25sleep20-21swapper/220:04:062
3130126822,26sleep30-21swapper/319:39:073
2760026819,24sleep30-21swapper/322:53:493
2004926812,31sleep20-21swapper/222:28:572
174862685,11sleep00-21swapper/020:37:310
747726721,27sleep00-21swapper/020:04:200
587426718,25sleep20-21swapper/221:44:032
412426711,12sleep30-21swapper/323:19:173
2133726713,40sleep321361-21modprobe22:33:473
1596126718,35sleep316026-21switchtime22:14:163
322326618,25sleep00-21swapper/021:34:160
2534626617,28sleep025455-21timerandwakeup22:44:170
2203126624,19sleep30-21swapper/300:18:483
1383626618,22sleep30-21swapper/322:08:553
1138626611,28sleep20-21swapper/223:44:032
1114726614,32sleep211149-21cron20:18:472
717026516,19sleep30-21swapper/321:48:493
1306526518,27sleep00-21swapper/022:04:200
69626413,22sleep30-21swapper/319:44:163
673326425,21sleep00-21swapper/023:29:030
337326415,33sleep33476-21grep21:34:193
3158426420,23sleep00-21swapper/019:39:120
3103826419,21sleep30-21swapper/323:04:073
2193526418,24sleep30-21swapper/319:09:153
173602646,12sleep30-21swapper/322:19:133
646426318,28sleep06500-21basename20:03:470
2045026316,22sleep00-21swapper/000:13:490
129832638,12sleep30-21swapper/320:24:013
3010326211,22sleep00-21swapper/019:34:160
2264626218,26sleep222673-21awk00:19:122
1953126221,19sleep20-21swapper/220:44:072
186162627,11sleep20-21swapper/200:05:432
1680626216,23sleep20-21swapper/223:59:292
61632616,11sleep30-21swapper/323:25:113
2935326119,28sleep029380-21cron19:33:470
2604526118,17sleep30-21swapper/322:48:493
2254326126,17sleep20-21swapper/222:34:262
2116226115,27sleep021196-21munin-run19:08:480
1992526116,21sleep30-21swapper/320:44:163
2546226022,23sleep025527-21uniq00:29:110
2546226022,23sleep025527-21uniq00:29:110
673025916,24sleep26726-21apt-get23:29:012
326002595,11sleep20-21swapper/221:25:102
326002595,11sleep20-21swapper/221:25:102
1085425914,32sleep010882-21cpu22:00:000
1085425914,32sleep010882-21cpu22:00:000
644225817,26sleep06444-21cron23:28:470
2916325813,31sleep229179-21sed22:58:492
2916325813,31sleep229179-21sed22:58:492
2709325812,32sleep027121-21/usr/sbin/munin00:34:070
2695325814,18sleep30-21swapper/319:24:133
2465125815,26sleep024091-21/usr/sbin/munin20:59:200
988825716,24sleep09882-21apt-get20:14:050
633325717,18sleep00-21swapper/021:44:170
2876225716,24sleep028757-21apt-get21:14:010
1476125720,18sleep20-21swapper/220:29:132
1476125720,18sleep20-21swapper/220:29:132
1362025720,18sleep20-21swapper/220:24:212
884725620,16sleep20-21swapper/223:34:182
884725620,16sleep20-21swapper/223:34:182
2814825612,20sleep20-21swapper/219:29:052
2537425614,26sleep325369-21apt-get00:29:063
2537425614,26sleep325369-21apt-get00:29:063
1845525614,18sleep30-21swapper/320:39:203
1663625620,16sleep30-21swapper/323:59:263
1030325614,24sleep310340-21ntp_states20:14:163
2851125518,18sleep30-21swapper/321:13:483
266122555,10sleep20-21swapper/221:04:512
266122555,10sleep20-21swapper/221:04:512
2117425518,18sleep00-21swapper/000:14:170
1167825513,24sleep011734-21irqstats20:19:100
1026525513,20sleep00-21swapper/023:39:130
769225413,23sleep07785-21memory21:49:100
769225413,23sleep07785-21memory21:49:100
73325412,19sleep20-21swapper/223:10:202
276025414,23sleep22756-21apt-get21:34:022
2227825415,23sleep022273-21apt-get00:19:010
2171225420,18sleep30-21swapper/322:34:073
206625415,18sleep20-21swapper/223:14:032
206625415,18sleep20-21swapper/223:14:032
1511825415,18sleep30-21swapper/320:29:223
1511825415,18sleep30-21swapper/320:29:223
1107725414,15sleep20-21swapper/222:00:182
1107725414,15sleep20-21swapper/222:00:182
1073325412,26sleep310747-21vmstat23:39:203
428525316,21sleep30-21swapper/319:55:513
3095125314,23sleep230945-21apt-get23:04:022
2690925314,22sleep226903-21apt-get00:34:012
2565825312,25sleep225784-21sed19:19:242
2068825311,24sleep220682-21apt-get00:14:032
202562538,34sleep30-21swapper/300:11:583
1602525312,17sleep00-21swapper/023:58:590
88072527,11sleep20-21swapper/220:09:192
2194625214,24sleep021954-21ntp_states19:09:160
895825114,21sleep28953-21apt-get21:54:002
2384925111,23sleep223963-21sed22:39:192
2162925112,22sleep021696-21df22:34:040
1797125114,19sleep00-21swapper/020:39:050
2971625017,18sleep20-21swapper/219:34:052
19902509,24sleep02119-21awk21:29:200
1183425011,14sleep30-21swapper/320:19:133
1853124911,22sleep318526-21apt-get22:24:023
3009524812,21sleep330101-21ntp_states19:34:153
2736224812,19sleep227454-21users19:24:202
1334724613,16sleep00-21swapper/023:49:140
1334724613,16sleep00-21swapper/023:49:140
2809724510,18sleep30-21swapper/319:29:023
741524311,19sleep20-21swapper/221:49:002
741524311,19sleep20-21swapper/221:49:002
538624311,12sleep20-21swapper/219:59:092
123982428,18sleep20-21swapper/222:04:062
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional