You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-07 - 08:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Mon Sep 07, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
327562149582,1396sleep032757-21ntp_states20:13:550
327562149582,1396sleep032757-21ntp_states20:13:550
16856288220,848sleep216859-21cron00:33:242
9434246963,316sleep20-21swapper/220:43:422
1886823907,370sleep218904-21fschecks_count21:13:452
24712236629,324sleep224772-21if_eth019:48:462
328231938,155sleep2341-21latency_hist21:58:272
3548229610,204sleep23634-21switchtime20:23:552
219142296168,61sleep20-21swapper/223:03:572
8222227629,203sleep2367-21snmpd00:03:522
3647224762,136sleep23816-21latency_hist22:08:292
3647224762,136sleep23816-21latency_hist22:08:292
6841223520,193sleep36984-21latency_hist22:18:273
6841223520,193sleep36984-21latency_hist22:18:273
11949223329,139sleep20-21swapper/219:08:492
4629222834,71sleep211471-21cyclictest22:09:092
11447222678,55sleep20-21swapper/220:48:542
20909220544,118sleep220340-21/usr/sbin/munin21:18:542
32469220456,134sleep232505-21cat23:38:492
15360220115,167sleep115457-21latency_hist00:28:271
1806421956,173sleep218126-21latency_hist19:28:272
1253821839,139sleep2367-21snmpd20:53:392
12504218117,145sleep012498-21apt-get00:18:410
19507217932,64sleep219528-21latency_hist22:58:272
329221768,93sleep00-21swapper/023:48:520
8462217310,142sleep20-21swapper/220:39:132
4875217182,50sleep24938-21mailstats23:53:552
4875217182,50sleep24938-21mailstats23:53:552
9365216820,45sleep20-21swapper/200:08:412
7966216812,19sleep10-21swapper/100:03:471
4415216823,127sleep34475-21/usr/sbin/munin22:09:063
14312216832,59sleep20-21swapper/200:23:472
15246216618,34sleep20-21swapper/221:01:382
756821657,142sleep27713-21latency_hist00:03:282
29422216334,19sleep20-21swapper/221:48:262
8367216032,111sleep28482-21latency_hist22:23:272
2637721588,135sleep226413-21ntpq23:18:512
12871215827,115sleep112904-21ntp_kernel_err00:18:501
32506215648,43sleep20-21swapper/220:13:492
32506215648,43sleep20-21swapper/220:13:492
1963121567,134sleep219782-21latency_hist19:33:282
16836215612,117sleep20-21swapper/222:48:512
13720215613,83sleep20-21swapper/219:13:462
20447215114,124sleep320456-21forks21:18:433
2439821497,128sleep224446-21awk23:13:422
1567121498,44sleep20-21swapper/221:03:402
14593214716,116sleep314659-21latency_hist22:43:263
24140214537,84sleep20-21swapper/221:29:002
10401214518,110sleep20-21swapper/222:28:482
56321436,122sleep2582-21apt-get21:58:352
2192214317,84sleep22207-21awk23:43:582
2047121438,72sleep220607-21grep22:59:002
15805214284,42sleep20-21swapper/219:18:542
28873213021,92sleep329015-21grep21:43:583
31537212952,66sleep231539-21awk23:34:012
24106212942,51sleep21-21systemd23:13:262
27386212812,99sleep127485-21users21:39:001
27386212812,99sleep127485-21users21:39:001
26636212117,81sleep026757-21timerwakeupswit23:18:570
6190211920,42sleep10-21swapper/123:58:381
5451211818,35sleep20-21swapper/220:29:122
29888211521,74sleep029405-21apt-get21:48:360
29888211521,74sleep029405-21apt-get21:48:360
29774211525,38sleep20-21swapper/220:03:562
1047211416,26sleep31056-21munin-run23:43:263
5964211319,45sleep20-21swapper/222:13:542
5964211319,45sleep20-21swapper/222:13:542
5645211328,67sleep35665-21systemd23:58:243
5645211328,67sleep35665-21systemd23:58:243
2240721118,42sleep221889-21/usr/sbin/munin21:23:552
14514211018,38sleep20-21swapper/222:43:012
11709211024,50sleep211703-21apt-get22:33:402
7310210918,36sleep20-21swapper/220:38:122
28838210910,84sleep228982-21latency_hist23:28:272
22684210911,83sleep322728-21systemd-run19:43:263
553210712,67sleep00-21swapper/023:38:590
25268210512,37sleep20-21swapper/221:33:482
25268210512,37sleep20-21swapper/221:33:482
751921029,80sleep07526-21systemctl00:03:260
13630210215,34sleep20-21swapper/200:21:452
1905210118,66sleep11985-21latency_hist22:03:271
17619210126,55sleep117657-21ntp_kernel_pll_21:08:521
10616210119,65sleep110646-21modprobe00:13:241
22019210020,57sleep322160-21head23:03:583
39192996,57sleep20-21swapper/223:51:042
1982629922,27sleep30-21swapper/319:33:373
2191529812,66sleep20-21swapper/219:38:572
1659029821,56sleep316105-21apt-get22:48:353
2896029736,17sleep20-21swapper/220:03:282
2896029736,17sleep20-21swapper/220:03:282
260429716,66sleep32675-21cut23:48:273
1848729626,56sleep218490-21cron21:13:252
973429413,66sleep09773-21ntpq00:08:510
143272930,7sleep30-21swapper/300:23:473
2861629212,23sleep00-21swapper/021:43:550
3042329119,32sleep20-21swapper/221:49:102
3042329119,32sleep20-21swapper/221:49:102
1230329116,53sleep012431-21latency_hist00:18:280
919228917,53sleep09163-21latency_hist20:43:270
354628911,65sleep13646-21cat22:08:271
354628911,65sleep13646-21cat22:08:271
1920428926,47sleep319215-21vmstat19:29:013
1675828918,35sleep00-21swapper/019:23:410
853528821,46sleep37857-21/usr/sbin/munin20:39:143
2698228826,50sleep227017-21ntpq21:38:532
2698228826,50sleep227017-21ntpq21:38:532
2248528820,29sleep30-21swapper/321:23:563
1479828816,55sleep014956-21sed20:58:580
52102870,9sleep00-21swapper/020:28:570
3003128720,52sleep130033-21sshd23:29:311
232928716,35sleep00-21swapper/022:03:460
232928716,35sleep00-21swapper/022:03:460
1078128716,50sleep310130-21/usr/sbin/munin22:28:573
295052867,15sleep20-21swapper/223:28:522
153272868,36sleep20-21swapper/222:43:542
204672859,63sleep220604-21timerwakeupswit19:33:562
1896428518,52sleep119067-21timerwakeupswit19:28:591
1419528517,51sleep314202-21diskstats20:58:443
1183428510,61sleep00-21swapper/019:08:470
865428421,13sleep20-21swapper/222:23:432
2643928425,46sleep226482-21tr19:53:512
2436328413,58sleep0314-21rs:main0
109072848,64sleep210904-21apt-get00:13:382
68028316,45sleep1748-21/usr/sbin/munin20:14:011
68028316,45sleep1748-21/usr/sbin/munin20:14:011
2145828314,53sleep021521-21idleruntime23:03:470
1077428323,22sleep20-21swapper/219:03:572
1077428323,22sleep20-21swapper/219:03:572
518628228,32sleep15204-21munin-run22:13:241
2459328212,56sleep324643-21if_eth023:13:473
229702828,34sleep20-21swapper/219:43:372
1515428213,54sleep1231-21systemd-journal21:00:321
272928122,16sleep20-21swapper/220:23:262
272928122,16sleep20-21swapper/220:23:262
2327728125,27sleep30-21swapper/321:28:283
1451528026,31sleep00-21swapper/000:23:540
238952798,18sleep00-21swapper/021:28:540
1309827912,51sleep013169-21sensors_temp20:54:130
1178827915,51sleep311795-21diskstats22:33:453
2956727812,51sleep029591-21netstat20:03:500
2788027724,36sleep327898-21latency_hist21:43:263
2788027724,36sleep327898-21latency_hist21:43:263
1567427737,21sleep00-21swapper/019:18:520
1918327617,31sleep00-21swapper/021:13:510
180072767,53sleep00-21swapper/000:33:590
1874427516,36sleep10-21swapper/122:53:541
1497227510,51sleep015027-21sed19:18:280
3062927413,45sleep030673-21/usr/sbin/munin23:33:420
3053227427,21sleep10-21swapper/120:08:271
2429727420,26sleep10-21swapper/119:48:291
2109427415,44sleep021230-21unixbench_singl21:18:560
1742027418,37sleep20-21swapper/200:33:492
1540727416,31sleep30-21swapper/322:43:553
1229927424,23sleep30-21swapper/320:53:283
512827314,42sleep35207-21timerandwakeup20:28:573
3074827321,12sleep20-21swapper/220:08:412
3074827321,12sleep20-21swapper/220:08:412
1135127325,19sleep30-21swapper/319:08:253
1135127325,19sleep30-21swapper/319:08:253
10722734,59sleep30-21swapper/320:17:543
10722734,59sleep30-21swapper/320:17:543
574527221,32sleep15686-40mandb23:58:261
574527221,32sleep15686-40mandb23:58:261
2652427213,27sleep00-21swapper/019:53:530
2607227217,29sleep10-21swapper/119:53:421
1304627220,25sleep10-21swapper/122:38:281
297627119,37sleep03029-21cpuspeed20:23:410
1575727114,37sleep315849-21tune2fs21:03:453
346827015,32sleep13547-21/usr/sbin/munin20:23:541
991526914,23sleep30-21swapper/320:43:553
945126922,24sleep30-21swapper/300:08:463
3033426920,31sleep3231-21systemd-journal23:33:253
293226921,24sleep10-21swapper/123:48:451
2550626923,26sleep10-21swapper/121:33:551
2550626923,26sleep10-21swapper/121:33:551
217752696,12sleep00-21swapper/021:23:280
966926816,23sleep00-21swapper/020:43:480
749726814,36sleep17693-21sort20:38:271
613126816,22sleep00-21swapper/022:13:560
613126816,22sleep00-21swapper/022:13:560
2785326813,36sleep327906-21/usr/sbin/munin19:58:483
2785326813,36sleep327906-21/usr/sbin/munin19:58:483
2773126826,20sleep10-21swapper/119:58:461
2773126826,20sleep10-21swapper/119:58:461
274622680,7sleep30-21swapper/319:58:273
260626819,22sleep30-21swapper/322:03:533
260626819,22sleep30-21swapper/322:03:533
2276626818,25sleep10-21swapper/119:43:281
2947126719,31sleep329565-21memory20:03:493
2944426716,20sleep00-21swapper/023:28:500
2162126717,28sleep10-21swapper/121:23:251
1932826725,27sleep30-21swapper/321:14:113
1520826714,23sleep30-21swapper/319:18:393
1015526710,17sleep00-21swapper/022:28:420
2568226618,34sleep30-21swapper/321:33:573
2568226618,34sleep30-21swapper/321:33:573
740826517,28sleep00-21swapper/020:38:260
318372659,32sleep30-21swapper/321:53:523
2772426511,38sleep027730-21diskstats19:58:450
2772426511,38sleep027730-21diskstats19:58:450
2655526525,21sleep30-21swapper/323:18:563
1690726523,23sleep00-21swapper/000:33:270
1383726517,18sleep30-21swapper/300:23:283
1141926521,30sleep011491-21cat19:08:260
1141926521,30sleep011491-21cat19:08:260
1063726521,28sleep110704-21awk22:28:551
3205526426,21sleep10-21swapper/120:13:281
3205526426,21sleep10-21swapper/120:13:281
3121026419,25sleep00-21swapper/020:08:550
3121026419,25sleep00-21swapper/020:08:550
2762826412,35sleep027727-21grep23:23:450
2306026418,20sleep10-21swapper/123:08:481
185326416,32sleep3231-21systemd-journal22:03:253
152926424,20sleep10-21swapper/120:18:491
152926424,20sleep10-21swapper/120:18:491
1502826417,24sleep10-21swapper/119:18:291
775826313,36sleep07903-21awk22:18:560
615626321,28sleep36216-21/usr/sbin/munin20:33:453
3152326312,22sleep10-21swapper/123:34:001
255426316,30sleep12565-21sh23:48:251
209226310,38sleep02204-21timerwakeupswit20:19:000
209226310,38sleep02204-21timerwakeupswit20:19:000
1787226316,20sleep30-21swapper/300:33:583
1550126316,24sleep00-21swapper/022:43:560
2040126215,23sleep10-21swapper/119:33:551
928126120,21sleep00-21swapper/022:23:570
593926113,32sleep15963-21/usr/sbin/munin22:13:531
593926113,32sleep15963-21/usr/sbin/munin22:13:531
470226116,20sleep10-21swapper/120:28:461
29162617,12sleep30-21swapper/323:48:433
276226111,35sleep32909-21cut20:23:273
276226111,35sleep32909-21cut20:23:273
1723426117,27sleep317229-21apt-get21:08:423
1596626118,29sleep316011-21awk00:28:493
1106426112,34sleep011068-21diskstats20:48:440
775826017,27sleep37755-21apt-get00:03:333
2418726015,29sleep024336-21sed23:13:280
2002126014,21sleep00-21swapper/022:58:500
1149226014,20sleep10-21swapper/119:08:281
790425913,17sleep10-21swapper/122:18:581
523825918,25sleep05346-21sed22:13:270
2526225919,19sleep10-21swapper/119:48:571
23602594,45sleep00-21swapper/023:45:530
136232597,24sleep30-21swapper/319:13:383
1005325921,20sleep10-21swapper/120:43:571
628825816,27sleep36297-21diskstats23:58:443
567125818,25sleep05744-21apt-config23:58:250
567125818,25sleep05744-21apt-config23:58:250
2354725818,18sleep30-21swapper/319:43:553
2023225810,12sleep30-21swapper/322:58:573
1856125816,19sleep00-21swapper/022:53:490
177952586,17sleep00-21swapper/022:52:200
132532585,11sleep00-21swapper/022:38:310
1236625814,30sleep012482-21timerwakeupswit22:33:590
1098425816,25sleep110977-21apt-get20:48:411
324092576,38sleep00-21swapper/021:54:250
3101825719,20sleep30-21swapper/320:08:503
3101825719,20sleep30-21swapper/320:08:503
3020725717,21sleep30-21swapper/321:49:033
3020725717,21sleep30-21swapper/321:49:033
2938625717,23sleep10-21swapper/121:48:251
2725725717,20sleep00-21swapper/021:38:590
2725725717,20sleep00-21swapper/021:38:590
2452425715,25sleep024518-21apt-get19:48:400
2724125619,20sleep10-21swapper/123:23:261
2297325614,25sleep022967-21apt-get19:43:390
2207225617,19sleep00-21swapper/019:38:590
2074725615,26sleep00-21swapper/019:33:590
297425514,25sleep32969-21apt-get20:23:393
2820425513,20sleep30-21swapper/323:23:563
1733125512,27sleep317398-21/usr/sbin/munin19:23:563
870525418,16sleep10-21swapper/122:23:461
2501025411,25sleep025077-21df21:33:420
2501025411,25sleep025077-21df21:33:420
2285325414,17sleep30-21swapper/323:08:423
1948125411,18sleep10-21swapper/121:14:131
1454325415,24sleep114552-21modprobe22:43:241
1052825414,22sleep110547-21ntp_states19:03:521
860725312,24sleep38600-21apt-get22:23:403
732925318,18sleep30-21swapper/322:18:473
2882725314,22sleep128837-21munin-run23:28:261
2346525312,24sleep123459-21apt-get21:28:421
1953025314,17sleep30-21swapper/322:58:293
1839025314,25sleep018391-21diskstats19:28:450
1714825312,24sleep117143-21apt-get00:33:401
1176625316,19sleep30-21swapper/319:08:463
1101325315,17sleep10-21swapper/100:13:491
59225217,16sleep00-21swapper/021:58:410
3241125211,21sleep10-21swapper/123:38:481
2466825211,15sleep10-21swapper/123:13:491
1741925213,22sleep117528-21users22:49:021
1175425216,21sleep111760-21df_inode19:08:441
1566925113,23sleep015665-21apt-get21:03:380
1481525112,23sleep114827-21apt-get22:43:371
1809325011,23sleep018146-21users21:09:070
814824911,21sleep08188-21memory00:03:500
2189624911,21sleep121908-21apt-get21:23:381
167592498,15sleep10-21swapper/119:23:421
1608624914,14sleep10-21swapper/121:03:511
2506724817,14sleep30-21swapper/319:48:553
2036324814,20sleep120468-21timerandwakeup22:58:591
1605224813,19sleep116058-21ntp_states00:28:511
298822476,15sleep10-21swapper/121:48:331
298822476,15sleep10-21swapper/121:48:331
127852478,17sleep10-21swapper/120:53:501
1254424713,15sleep30-21swapper/320:53:443
828224612,19sleep10-21swapper/120:39:081
3173724612,14sleep10-21swapper/121:53:501
1520624610,21sleep115201-21apt-get19:18:371
1226324612,15sleep30-21swapper/300:18:273
3126224313,15sleep30-21swapper/323:33:583
2137524214,15sleep10-21swapper/123:03:441
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional