You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-26 - 12:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Fri Dec 26, 2025 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2054626496,632sleep320587-21ntpq19:25:153
2054626496,632sleep320587-21ntpq19:25:153
30067264215,611sleep030019-21apt-get21:39:570
28667252315,493sleep128676-21ntpq21:34:411
28667252315,493sleep128676-21ntpq21:34:411
416247320,434sleep0473-21/usr/sbin/munin23:29:390
416247320,434sleep0473-21/usr/sbin/munin23:29:390
24884238616,356sleep224893-21systemd-run21:24:162
24884238616,356sleep224893-21systemd-run21:24:162
7799236430,314sleep37941-21grep22:09:533
16364234113,313sleep116437-21unin-run22:39:151
307052318200,95sleep330832-21gpgv23:24:163
307052318200,95sleep330832-21gpgv23:24:163
17851227670,144sleep317866-21fschecks_count00:24:343
19590227472,185sleep319622-21cut19:24:143
19590227472,185sleep319622-21cut19:24:143
24305225311,198sleep324487-21gpgv23:04:173
16910222411,193sleep317057-21latency_hist20:59:173
12387221741,130sleep312389-21munin-run00:09:163
4177221314,184sleep14181-21modprobe20:16:141
18389221311,164sleep318422-21awk21:04:143
24500221114,178sleep024149-21apt-get23:04:180
5465221011,126sleep35597-21unixbench_singl20:19:483
740218642,20sleep30-21swapper/321:49:343
2909121828,158sleep329149-21date23:19:163
12233217975,22sleep30-21swapper/322:24:433
1993216851,101sleep32038-21ntp_kernel_pll_20:09:423
1993216851,101sleep32038-21ntp_kernel_pll_20:09:423
13719216818,133sleep213732-21sh20:49:152
5827216749,70sleep30-21swapper/322:04:483
15505216071,25sleep30-21swapper/322:34:413
1681721599,137sleep316885-21fw_packets22:39:343
13548215352,79sleep312823-21/usr/sbin/munin00:10:053
13548215352,79sleep312823-21/usr/sbin/munin00:10:053
26400215241,92sleep326494-21fschecks_time19:44:343
31759215031,79sleep331086-21/usr/sbin/munin23:26:103
32308214073,51sleep30-21swapper/321:44:493
3458213928,95sleep33525-21modprobe21:59:153
19536213816,104sleep319596-21switchtime21:05:193
12178213828,90sleep312211-21grep20:44:153
15072213310,108sleep315086-21grep19:09:363
13734213311,110sleep313770-21unin-run20:49:163
9191213032,82sleep39240-21modprobe23:59:143
31346212812,93sleep00-21swapper/023:25:510
10204212673,39sleep30-21swapper/322:19:263
13564212511,90sleep30-21swapper/322:29:363
26482212430,56sleep326505-21kernelversion23:09:433
1075321247,83sleep3369-21snmpd00:03:553
22709212327,18sleep30-21swapper/319:31:333
22709212327,18sleep30-21swapper/319:31:333
24976212212,25sleep30-21swapper/319:39:373
24632212240,61sleep30-21swapper/323:04:253
19302212268,12sleep30-21swapper/300:29:273
15235212127,34sleep315319-21cat00:15:013
21836212027,81sleep321897-21tr22:54:483
9363211825,45sleep30-21swapper/320:34:343
7820211810,19sleep30-21swapper/320:29:333
28499211825,69sleep228643-21sed23:14:502
2413921186,79sleep3369-21snmpd19:35:053
19710211519,72sleep119854-21unixbench_singl21:05:211
108221136,12sleep30-21swapper/320:06:063
108221136,12sleep30-21swapper/320:06:063
10766211316,73sleep210205-21/usr/sbin/munin22:19:462
481421127,71sleep3369-21snmpd22:01:213
481421127,71sleep3369-21snmpd22:01:213
32284211210,64sleep332424-21latency_hist20:04:173
32284211210,64sleep332424-21latency_hist20:04:173
14981211218,80sleep214991-21latency_hist22:34:202
5327211145,35sleep30-21swapper/323:44:483
15320210939,53sleep30-21swapper/320:54:173
2186621086,12sleep30-21swapper/300:36:143
2186621086,12sleep30-21swapper/300:36:143
9131210610,56sleep39191-21sendmail_mailqu22:14:463
31085210615,75sleep131101-21awk23:24:271
2412721068,21sleep30-21swapper/321:19:453
2412721068,21sleep30-21swapper/321:19:453
31102210523,62sleep230596-21apt-get23:24:282
29812210516,65sleep129306-21/usr/sbin/munin23:19:541
29812210516,65sleep129306-21/usr/sbin/munin23:19:541
1828210516,72sleep21834-21diskstats23:34:332
445421046,11sleep30-21swapper/323:42:463
2931921046,13sleep30-21swapper/321:37:183
2931921046,13sleep30-21swapper/321:37:183
1799721045,12sleep30-21swapper/319:18:533
1799721045,12sleep30-21swapper/319:18:533
634210315,72sleep0653-21apt-get21:49:270
21012210322,19sleep30-21swapper/321:09:483
19852210214,69sleep119281-21/usr/sbin/munin00:29:451
17860210013,63sleep218001-21unixbench_singl20:59:492
1105021005,65sleep10-21swapper/120:39:371
2063029915,66sleep20-21swapper/219:25:182
2063029915,66sleep20-21swapper/219:25:182
2262929817,66sleep00-21swapper/019:30:290
2262929817,66sleep00-21swapper/019:30:290
1622129819,17sleep30-21swapper/300:19:303
2386329713,66sleep023930-21ls21:19:380
2386329713,66sleep023930-21ls21:19:380
2108429417,61sleep021178-21timerandwakeup21:09:490
1712229416,63sleep117132-21ntpq22:39:401
63932927,24sleep30-21swapper/320:24:363
1248329222,48sleep012641-21latency_hist00:09:170
291132918,67sleep3369-21snmpd19:54:153
277052909,69sleep3369-21snmpd21:31:203
2343129012,62sleep123507-21cat22:59:471
2208829011,60sleep20-21swapper/221:14:352
560228919,52sleep05622-21kernelversion22:04:410
625228813,51sleep26271-21apt-get20:24:272
1597328814,52sleep10-21swapper/100:19:151
402628710,52sleep20-21swapper/221:59:382
402628710,52sleep20-21swapper/221:59:382
424128617,48sleep03752-21/usr/sbin/munin21:59:440
424128617,48sleep03752-21/usr/sbin/munin21:59:440
307828618,52sleep03176-21cat23:39:160
3062328615,52sleep230704-21cp23:24:152
3062328615,52sleep230704-21cp23:24:152
2072128611,63sleep020791-21switchtime19:25:200
2072128611,63sleep020791-21switchtime19:25:200
1612928624,27sleep00-21swapper/019:14:180
1362628626,31sleep30-21swapper/319:04:403
324128514,58sleep0369-21snmpd21:56:250
1773028510,63sleep1369-21snmpd19:15:321
1773028510,63sleep1369-21snmpd19:15:321
2963128423,42sleep029726-21cat21:39:170
2963128423,42sleep029726-21cat21:39:170
1822028430,13sleep30-21swapper/322:44:303
1226128416,52sleep012341-21mailstats22:24:440
957528316,47sleep19632-21irqstats20:34:401
906928321,47sleep09083-21apt20:34:140
381928315,53sleep03896-21awk23:39:440
324528324,36sleep20-21swapper/220:14:382
298642836,65sleep329898-21ps19:54:443
1419928311,55sleep014318-21sed22:30:100
1174128336,25sleep20-21swapper/200:04:502
116428321,42sleep21240-21/usr/sbin/munin21:49:452
263532826,65sleep30-21swapper/321:28:503
2599428223,30sleep20-21swapper/223:09:172
2201728220,34sleep00-21swapper/021:14:330
998328119,45sleep110004-21latency_hist22:19:161
44172817,15sleep3287-21dbus-daemon20:19:153
1846328124,40sleep218494-21sort19:19:372
1846328124,40sleep218494-21sort19:19:372
1537528119,26sleep00-21swapper/022:34:380
1406828116,50sleep214090-21df_inode20:49:352
52212809,57sleep05287-21run-parts20:19:450
311392805,64sleep30-21swapper/321:42:233
2209828022,35sleep11-21systemd00:39:141
2209828022,35sleep11-21systemd00:39:141
1851928020,36sleep118529-21store21:04:161
1593128015,30sleep00-21swapper/020:54:420
2720927918,45sleep227281-21cut21:29:462
1564627913,42sleep215536-21/usr/sbin/munin20:54:362
1377327919,36sleep10-21swapper/120:49:181
63772788,52sleep30-21swapper/323:49:293
3274827811,50sleep2332-21cut23:29:372
3274827811,50sleep2332-21cut23:29:372
304427821,16sleep30-21swapper/323:39:153
2255727825,33sleep122620-21sendmail-msp22:59:141
222592786,13sleep30-21swapper/321:14:403
2094627817,43sleep021034-21/usr/sbin/munin00:34:370
2094627817,43sleep021034-21/usr/sbin/munin00:34:370
1521627810,53sleep015217-21sshd20:53:330
2368427728,26sleep10-21swapper/121:19:351
2368427728,26sleep10-21swapper/121:19:351
799127621,28sleep10-21swapper/123:54:311
662727518,35sleep26659-21munin-run22:09:152
3133627518,43sleep231344-21needreboot19:59:392
3133627518,43sleep231344-21needreboot19:59:392
233627516,41sleep22475-21sed20:09:492
233627516,41sleep22475-21sed20:09:492
2090027511,50sleep0369-21snmpd22:52:430
1223027520,30sleep20-21swapper/220:44:182
2980227413,34sleep00-21swapper/019:54:400
2901227426,32sleep129018-21modprobe23:19:141
1643827413,19sleep20-21swapper/222:39:162
711627321,31sleep10-21swapper/123:49:511
498227326,23sleep00-21swapper/023:44:370
2262627315,31sleep10-21swapper/121:14:501
13412737,12sleep30-21swapper/323:32:453
13412737,12sleep30-21swapper/323:32:453
550827233,16sleep20-21swapper/223:44:512
252302729,38sleep00-21swapper/021:24:390
2350827215,37sleep222850-21/usr/sbin/munin22:59:482
208732727,12sleep30-21swapper/322:52:263
2047827213,42sleep120545-21df21:09:331
188442726,16sleep00-21swapper/019:19:470
188442726,16sleep00-21swapper/019:19:470
1718527217,29sleep00-21swapper/022:39:430
752827120,33sleep17536-21modprobe20:29:151
251352716,45sleep20-21swapper/221:24:332
2050827117,35sleep120639-21sed22:49:501
757127021,33sleep27578-21/usr/sbin/munin22:09:482
3084627020,27sleep00-21swapper/023:24:170
3084627020,27sleep00-21swapper/023:24:170
2477927025,29sleep224842-21/usr/sbin/munin19:39:322
2412627023,28sleep00-21swapper/019:34:540
1032427017,22sleep00-21swapper/022:19:350
815026920,29sleep20-21swapper/223:54:372
66126912,45sleep1635-21/usr/sbin/munin21:49:321
3219126820,23sleep20-21swapper/221:44:482
83352676,50sleep00-21swapper/022:13:540
340326719,20sleep20-21swapper/223:39:342
2821926711,40sleep20-21swapper/221:34:302
2821926711,40sleep20-21swapper/221:34:302
1787026715,36sleep017950-21/usr/sbin/munin00:24:350
1231926722,28sleep112326-21modprobe00:09:141
11512676,18sleep00-21swapper/020:07:000
11512676,18sleep00-21swapper/020:07:000
3065726622,29sleep230680-21idleruntime-cro19:59:152
2832326612,26sleep20-21swapper/219:49:422
200402666,48sleep00-21swapper/021:07:300
1000526616,24sleep20-21swapper/222:19:172
2999926513,19sleep10-21swapper/121:39:261
2533226512,39sleep025401-21expr19:39:460
2172426516,31sleep221752-21munin-run21:14:152
1590326518,30sleep0305-21systemd-logind00:19:140
1401426512,18sleep10-21swapper/120:49:341
751926414,27sleep00-21swapper/023:52:330
256512645,16sleep10-21swapper/123:05:451
1830826419,25sleep00-21swapper/022:44:350
1301826415,31sleep013082-21/usr/sbin/munin00:09:530
1301826415,31sleep013082-21/usr/sbin/munin00:09:530
46532635,47sleep20-21swapper/220:19:182
1952526310,26sleep20-21swapper/222:49:182
1481426318,29sleep114842-21munin-run22:34:141
1410126320,28sleep114129-21munin-run00:14:141
1410126320,28sleep114129-21munin-run00:14:141
2847226211,23sleep10-21swapper/119:49:451
2177626222,21sleep20-21swapper/222:54:472
2175426220,25sleep221842-21tune2fs19:29:352
2175426220,25sleep221842-21tune2fs19:29:352
9612617,38sleep20-21swapper/220:04:482
9612617,38sleep20-21swapper/220:04:482
81126115,23sleep10-21swapper/120:04:471
81126115,23sleep10-21swapper/120:04:471
758126110,29sleep07691-21latency_hist20:29:180
3031626117,24sleep129375-21/usr/sbin/munin19:54:501
2999526110,35sleep229494-21apt-get21:39:232
2661326114,31sleep226655-21ntpq23:09:462
1978426117,24sleep00-21swapper/000:29:440
1842726120,23sleep018518-21sed21:04:150
1675926119,19sleep20-21swapper/222:39:332
1606226119,26sleep116098-21modprobe19:14:151
983226019,21sleep00-21swapper/020:34:470
2788226023,28sleep00-21swapper/021:33:350
2630126017,19sleep00-21swapper/023:09:390
1330326012,32sleep013312-21diskstats19:04:330
131982605,11sleep10-21swapper/119:04:221
1302326018,26sleep213038-21apt-get22:29:152
1028526014,27sleep010421-21/usr/sbin/munin23:59:520
912025915,25sleep20-21swapper/220:34:162
794125914,27sleep07990-21/usr/sbin/munin23:54:300
647725910,33sleep26483-21df_inode23:49:362
1832925921,20sleep20-21swapper/200:24:462
1752625919,17sleep10-21swapper/100:24:161
156032594,18sleep20-21swapper/200:15:292
1531625919,20sleep20-21swapper/219:09:432
599625814,21sleep10-21swapper/122:04:501
543825812,22sleep10-21swapper/123:44:501
473425815,26sleep14728-21apt-get20:19:291
2757225817,21sleep00-21swapper/019:49:150
2680225810,19sleep10-21swapper/119:44:441
2398325813,17sleep20-21swapper/219:34:522
132332589,21sleep10-21swapper/122:29:181
1069025818,21sleep20-21swapper/220:39:172
808925717,19sleep10-21swapper/120:29:391
707425715,25sleep16594-21apt-get22:09:231
3157225715,26sleep031567-21apt-get21:44:300
2287425714,27sleep022869-21apt-get22:59:300
1673925716,24sleep216758-21apt-get19:14:282
1673925716,24sleep216758-21apt-get19:14:282
1622025715,25sleep216217-21apt-get00:19:282
1332225711,16sleep20-21swapper/219:04:352
1148325717,25sleep011545-21awk20:39:480
907025611,18sleep00-21swapper/022:14:430
3266025614,25sleep132654-21apt-get23:29:321
3266025614,25sleep132654-21apt-get23:29:321
2801625612,17sleep10-21swapper/121:34:171
2775525614,26sleep027746-21apt-get23:14:320
2751425612,22sleep10-21swapper/123:14:171
1991625614,20sleep20-21swapper/200:29:462
178152566,40sleep00-21swapper/019:16:320
178152566,40sleep00-21swapper/019:16:320
161525615,25sleep11608-21apt-get20:09:321
161525615,25sleep11608-21apt-get20:09:321
1314025612,24sleep213154-21kernelversion00:09:552
1314025612,24sleep213154-21kernelversion00:09:552
951725514,25sleep29511-21apt-get23:59:292
2840625517,16sleep00-21swapper/019:49:440
40925415,20sleep10-21swapper/121:49:151
375925414,24sleep13770-21ntp_states23:39:411
364325417,17sleep00-21swapper/020:14:480
2321225416,23sleep123282-21df19:34:341
1966425416,15sleep10-21swapper/119:24:161
1966425416,15sleep10-21swapper/119:24:161
1945025414,19sleep20-21swapper/221:05:182
1776325414,23sleep117760-21apt-get00:24:281
385025315,23sleep13855-21df_inode21:59:331
385025315,23sleep13855-21df_inode21:59:331
3094825313,20sleep10-21swapper/119:59:261
3094825313,20sleep10-21swapper/119:59:261
269225311,25sleep10-21swapper/121:54:451
2598625214,21sleep126039-21users21:24:521
204025212,23sleep02069-21ntp_states20:09:430
204025212,23sleep02069-21ntp_states20:09:430
1596625215,19sleep10-21swapper/120:54:431
1246925214,20sleep00-21swapper/020:44:320
1083025220,16sleep10-21swapper/100:04:171
2687825115,20sleep20-21swapper/219:44:452
2166625113,22sleep121660-21apt-get19:29:291
2166625113,22sleep121660-21apt-get19:29:291
1630925113,21sleep00-21swapper/000:19:350
2062325012,22sleep120629-21ntp_states19:25:171
2062325012,22sleep120629-21ntp_states19:25:171
1890425015,19sleep10-21swapper/122:44:491
1153325017,15sleep10-21swapper/122:24:171
310512499,15sleep00-21swapper/019:59:330
310512499,15sleep00-21swapper/019:59:330
213932499,27sleep00-21swapper/022:54:370
2109824910,15sleep20-21swapper/200:34:392
2109824910,15sleep20-21swapper/200:34:392
174624712,22sleep11741-21apt-get23:34:281
2605224612,18sleep10-21swapper/119:44:161
22832447,13sleep20-21swapper/221:54:332
2014324213,15sleep20-21swapper/222:49:422
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional