You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-01 - 23:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Wed Jul 01, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
242422353511,3511sleep011922-21kworker/0:111:35:400
2835329368,854sleep328388-21ntpq11:49:573
30920238434,325sleep34330-21cyclictest11:59:343
30920238434,325sleep34330-21cyclictest11:59:343
568228697,170sleep3720-21latency_hist08:39:343
22339226525,218sleep122465-21latency_hist09:49:341
32743225114,220sleep2492-21cat12:04:332
13662223411,208sleep013704-21ntpq09:19:570
3521223010,203sleep03638-21cat12:14:330
4377221712,191sleep04443-21sort10:34:470
21831220955,22sleep30-21swapper/311:29:503
21831220955,22sleep30-21swapper/311:29:503
11927220417,163sleep30-21swapper/310:59:333
21505220316,167sleep321566-21sendmail_mailqu09:44:593
28670220215,168sleep328756-21fschecks_time08:24:523
16967219211,166sleep117042-21find09:30:081
16967219211,166sleep117042-21find09:30:081
29009219015,159sleep128533-21apt-get10:09:411
17536219044,96sleep30-21swapper/311:15:003
21223218218,137sleep321362-21grep08:00:053
15979218119,140sleep30-21swapper/311:10:033
32750218014,145sleep0363-21switchtime10:20:010
1198218011,135sleep31233-21ps12:05:523
8074217922,20sleep30-21swapper/312:28:293
30966217914,87sleep30-21swapper/310:14:563
30966217914,87sleep30-21swapper/310:14:563
23451217917,146sleep023453-21sshd08:07:410
13384217911,18sleep30-21swapper/309:19:523
5110217812,24sleep30-21swapper/312:19:333
23652217720,143sleep34330-21cyclictest11:34:553
17667217610,152sleep317758-21latency_hist09:34:343
17667217610,152sleep317758-21latency_hist09:34:343
32152216739,82sleep20-21swapper/210:19:432
11254216431,49sleep311325-21true12:39:313
16514216317,103sleep30-21swapper/307:45:033
13931216244,31sleep30-21swapper/311:04:423
13931216244,31sleep30-21swapper/311:04:423
930421607,14sleep00-21swapper/010:49:530
4319215715,91sleep30-21swapper/307:09:333
1029215612,100sleep31090-21latency_hist10:24:333
10349215215,112sleep30-21swapper/310:54:333
8106214612,49sleep30-21swapper/307:19:473
9706214012,46sleep3225-21systemd-journal12:34:313
63021395,11sleep10-21swapper/110:20:061
22283213580,37sleep122350-21/usr/sbin/munin11:30:001
22283213580,37sleep122350-21/usr/sbin/munin11:30:001
1198321357,21sleep34330-21cyclictest09:14:553
8327213312,85sleep38471-21cat09:04:343
11602213310,49sleep311649-21ntpq07:29:573
25143213226,85sleep025147-21modprobe08:14:310
4836213013,41sleep30-21swapper/307:09:543
17881212911,101sleep017905-21proc_pri07:50:010
17881212911,101sleep017905-21proc_pri07:50:010
644721258,36sleep30-21swapper/310:39:593
29537212551,31sleep329621-21mailstats10:10:073
20591212317,90sleep120636-21grep11:24:551
20591212317,90sleep120636-21grep11:24:551
127221228,78sleep31339-21df10:24:493
13127212134,51sleep313168-21ntpq07:34:573
2332212082,27sleep32347-21grep08:44:473
796021188,76sleep37998-21ps10:45:003
30832211818,75sleep230093-21/usr/sbin/munin08:30:042
17325211816,93sleep00-21swapper/009:30:470
17325211816,93sleep00-21swapper/009:30:470
28139211710,68sleep30-21swapper/310:05:063
15652211713,90sleep315798-21wc07:44:343
26304211615,87sleep326407-21timerandwakeup10:00:013
26304211615,87sleep326407-21timerandwakeup10:00:013
31902211516,85sleep131973-21switchtime12:00:121
1986211521,68sleep22135-21latency_hist12:09:342
30299211314,20sleep30-21swapper/308:29:533
14088211319,20sleep00-21swapper/007:39:350
636121129,92sleep3364-21snmpd08:55:173
26007211113,82sleep226076-21switchtime08:15:022
26007211113,82sleep226076-21switchtime08:15:022
22949211112,26sleep322986-21sed09:49:563
2607721109,18sleep30-21swapper/308:15:033
2607721109,18sleep30-21swapper/308:15:033
7916210911,22sleep38062-21latency_hist07:19:353
1790621096,85sleep30-21swapper/309:34:463
1790621096,85sleep30-21swapper/309:34:463
13169210911,87sleep00-21swapper/011:01:080
3130921088,17sleep30-21swapper/311:59:423
30724210820,71sleep130829-21timerandwakeup08:30:031
20120210812,82sleep00-21swapper/007:58:120
18407210812,78sleep018476-21latency_hist11:19:330
15054210718,65sleep015199-21grep07:40:070
671921060,7sleep10-21swapper/107:14:521
15083210616,76sleep315104-21kernelversion09:24:573
25194210510,80sleep225265-21cut08:14:342
14805210512,75sleep114863-21sendmail_mailqu07:40:041
1994321049,46sleep319952-21munin-run11:24:333
1994321049,46sleep319952-21munin-run11:24:333
7291210315,70sleep07419-21uptime07:15:040
22640210312,37sleep30-21swapper/308:05:183
19325210313,66sleep019461-21unixbench_multi11:20:020
19325210313,66sleep019461-21unixbench_multi11:20:020
29482210214,72sleep029499-21apt11:54:420
29482210214,72sleep029499-21apt11:54:420
4844210110,44sleep30-21swapper/308:50:413
896121006,17sleep20-21swapper/209:04:572
17420210010,13sleep30-21swapper/307:49:473
17420210010,13sleep30-21swapper/307:49:473
2865929922,55sleep228780-21latency_hist10:09:332
2851429912,17sleep00-21swapper/010:09:310
281629911,34sleep30-21swapper/310:29:453
2111729822,58sleep221159-21fschecks_count09:44:502
117342987,78sleep00-21swapper/010:58:050
101312989,46sleep30-21swapper/309:09:463
296529711,48sleep30-21swapper/312:10:043
982529627,42sleep09863-21munin-run09:09:310
437529610,13sleep30-21swapper/310:34:463
2641129612,31sleep30-21swapper/311:44:483
1022629615,31sleep00-21swapper/009:09:520
2921629516,62sleep129218-21cron11:54:311
2304129511,70sleep223048-21awk09:49:592
2066629518,43sleep30-21swapper/309:43:343
964329419,40sleep30-21swapper/309:07:123
66622948,44sleep30-21swapper/312:24:333
1777129418,50sleep217919-21head11:15:032
690329311,65sleep16966-21cpuspeed12:24:481
148642930,10sleep20-21swapper/207:40:042
100632937,43sleep30-21swapper/307:25:003
195612927,14sleep20-21swapper/209:39:502
1322929218,41sleep30-21swapper/311:01:563
1349629115,56sleep213494-21/usr/sbin/munin11:04:332
270202906,43sleep30-21swapper/308:19:473
248582907,12sleep30-21swapper/311:39:433
8542896,32sleep30-21swapper/308:39:503
8542896,32sleep30-21swapper/308:39:503
1343428920,52sleep113495-21apt-key11:04:321
2498328817,54sleep225043-21fschecks_time11:39:502
67502879,17sleep20-21swapper/210:40:022
2211428711,64sleep022159-21grep11:29:550
2211428711,64sleep022159-21grep11:29:550
1531428716,58sleep215376-21mailstats09:25:042
245532867,31sleep30-21swapper/309:54:573
2554028522,42sleep125691-21sed11:40:011
1560228524,29sleep10-21swapper/111:09:531
2677928412,57sleep126916-21latency_hist08:19:331
2677928412,57sleep126916-21latency_hist08:19:331
1272628423,25sleep20-21swapper/207:34:442
912828319,43sleep19141-21fschecks_count10:49:501
582828319,40sleep20-21swapper/212:20:002
3185528323,39sleep231880-21munin-run10:19:312
3185528323,39sleep231880-21munin-run10:19:312
2449128312,56sleep124514-21netstat09:54:551
581228216,51sleep15827-21/usr/sbin/munin12:19:591
201642826,28sleep30-21swapper/311:24:413
201642826,28sleep30-21swapper/311:24:413
1527728215,50sleep115313-21wc09:25:031
617528119,42sleep16323-21grep08:55:071
2771728111,54sleep027826-21sed11:49:330
249362815,27sleep30-21swapper/308:11:523
1645228017,47sleep216513-21switchtime07:45:022
758027915,45sleep07710-21grep12:25:030
198912795,26sleep30-21swapper/307:55:173
1645227811,18sleep20-21swapper/209:29:562
1645227811,18sleep20-21swapper/209:29:562
1633227814,48sleep016369-21ps07:45:000
571027726,27sleep25788-21cut10:39:342
1091127719,36sleep210937-21awk07:29:312
771727610,12sleep10-21swapper/109:00:031
2701427626,32sleep227101-21wc10:04:332
2701427626,32sleep227101-21wc10:04:332
2376427611,52sleep0364-21snmpd09:53:360
519627524,23sleep10-21swapper/108:54:341
1848027517,29sleep20-21swapper/209:35:012
1848027517,29sleep20-21swapper/209:35:012
1023127412,45sleep010324-21sed07:25:050
893127323,35sleep18989-21find12:30:001
570827315,25sleep00-21swapper/008:54:580
2054227218,33sleep020537-21apt-get07:59:460
18482728,49sleep21915-21switchtime10:25:042
91692710,8sleep00-21swapper/012:30:030
1958227110,15sleep00-21swapper/009:39:520
430027024,28sleep24318-21munin-run07:09:322
414927014,40sleep24181-21/usr/sbin/munin08:49:552
2919827013,26sleep20-21swapper/210:09:582
2309627018,29sleep20-21swapper/211:34:332
208827019,28sleep22173-21latency_hist08:44:342
208827019,28sleep22173-21latency_hist08:44:342
950426912,40sleep29526-21/usr/sbin/munin10:49:592
279826923,26sleep00-21swapper/008:45:000
1905226917,32sleep219086-21fschecks_count07:54:512
309226815,24sleep00-21swapper/010:29:550
2388926818,34sleep123959-21df08:09:481
965826716,35sleep19693-21cpu07:24:501
344326712,37sleep03450-21ntp_states07:04:590
3227026717,24sleep00-21swapper/008:35:030
3193626720,26sleep00-21swapper/010:19:330
3193626720,26sleep00-21swapper/010:19:330
27382667,12sleep10-21swapper/108:44:571
2668026616,20sleep10-21swapper/111:44:541
1490226617,25sleep00-21swapper/009:24:540
134826612,35sleep10-21swapper/110:24:511
1168226617,22sleep00-21swapper/009:14:480
800226510,32sleep00-21swapper/010:45:010
3116626523,21sleep10-21swapper/110:15:021
3116626523,21sleep10-21swapper/110:15:021
2989126522,23sleep10-21swapper/108:29:351
1184726520,28sleep111879-21munin-run10:59:311
1032926517,24sleep10-21swapper/109:09:531
2055926414,22sleep00-21swapper/011:24:540
2055926414,22sleep00-21swapper/011:24:540
1150126413,35sleep111568-21memory07:29:551
902926325,22sleep10-21swapper/109:05:001
831226317,25sleep00-21swapper/007:19:520
78326317,27sleep0777-21apt-get08:39:470
78326317,27sleep0777-21apt-get08:39:470
775726319,26sleep10-21swapper/110:44:541
74742637,12sleep20-21swapper/207:15:382
2180826314,27sleep20-21swapper/208:04:322
2075426317,23sleep10-21swapper/107:59:551
190132636,13sleep10-21swapper/111:19:541
190132636,13sleep10-21swapper/111:19:541
173902639,41sleep117391-21sshd07:49:411
173902639,41sleep117391-21sshd07:49:411
2807726222,23sleep20-21swapper/211:49:522
277226221,24sleep10-21swapper/112:10:021
253632626,43sleep00-21swapper/009:59:180
1395026218,22sleep00-21swapper/011:04:430
1395026218,22sleep00-21swapper/011:04:430
45826112,34sleep1530-21systemd-run08:39:321
3083226118,25sleep20-21swapper/211:59:332
3083226118,25sleep20-21swapper/211:59:332
1836826117,30sleep017880-21/usr/sbin/munin09:34:590
1836826117,30sleep017880-21/usr/sbin/munin09:34:590
588426016,30sleep25970-21latency_hist07:14:332
2827526022,20sleep20-21swapper/208:24:312
282026017,26sleep10-21swapper/110:29:471
260026018,20sleep20-21swapper/208:44:542
2586426016,19sleep10-21swapper/109:59:511
2586426016,19sleep10-21swapper/109:59:511
1180526019,18sleep00-21swapper/007:30:020
761825916,21sleep00-21swapper/009:00:020
66325915,22sleep10-21swapper/112:04:491
435725914,22sleep00-21swapper/008:50:010
3019125914,20sleep00-21swapper/008:29:500
2883925914,21sleep20-21swapper/208:24:552
2544825916,26sleep225458-21/usr/sbin/munin09:59:322
248225915,25sleep22575-21memory12:09:552
1954825913,20sleep10-21swapper/107:55:021
837525815,19sleep20-21swapper/207:19:542
689925817,24sleep26896-21apt-get12:24:442
3166625814,26sleep231736-21/usr/sbin/munin08:34:492
2285025818,21sleep10-21swapper/109:49:541
1588425819,21sleep20-21swapper/211:09:582
384425724,19sleep10-21swapper/112:14:501
2734125715,20sleep10-21swapper/110:04:521
2649525712,28sleep20-21swapper/211:44:512
226502575,11sleep20-21swapper/211:30:052
226502575,11sleep20-21swapper/211:30:052
2231125713,31sleep222322-21basename08:04:442
999725612,26sleep29991-21apt-get12:34:462
444425610,30sleep14451-21df_inode10:34:481
299822566,13sleep10-21swapper/111:54:591
299822566,13sleep10-21swapper/111:54:591
299125614,24sleep12985-21apt-get07:04:481
2543625619,22sleep00-21swapper/008:14:480
2543625619,22sleep00-21swapper/008:14:480
2149925614,27sleep221595-21latency_hist11:29:322
2149925614,27sleep221595-21latency_hist11:29:322
941425519,17sleep10-21swapper/107:24:341
1677225514,26sleep116775-21/usr/sbin/munin07:45:051
1205025510,24sleep212102-21users07:30:042
1135725515,23sleep111506-21sed12:39:331
886825414,24sleep28900-21ntp_states12:29:572
419325412,17sleep00-21swapper/012:14:570
2723025415,24sleep127287-21irqstats08:19:541
2570025416,19sleep00-21swapper/009:59:460
2570025416,19sleep00-21swapper/009:59:460
1283525412,24sleep10-21swapper/107:34:511
1149025421,18sleep20-21swapper/209:14:342
621625315,19sleep00-21swapper/010:39:520
3131225311,20sleep00-21swapper/011:59:430
301125315,19sleep20-21swapper/210:29:532
2748725312,18sleep00-21swapper/008:20:010
2232825313,24sleep022338-21munin-run09:49:330
1132725311,28sleep011354-21latency_hist12:39:320
719525215,17sleep20-21swapper/208:59:522
662325216,17sleep10-21swapper/110:40:011
51792528,30sleep00-21swapper/012:19:340
2898225212,24sleep028533-21apt-get10:09:350
2215825210,22sleep10-21swapper/108:04:351
139282528,17sleep20-21swapper/211:04:412
139282528,17sleep20-21swapper/211:04:412
1066425212,16sleep20-21swapper/210:54:482
1013025215,19sleep20-21swapper/209:09:452
48582519,14sleep00-21swapper/007:09:550
2856625116,18sleep00-21swapper/008:24:450
2371525111,24sleep123738-21ntp_states11:34:571
670225012,19sleep20-21swapper/208:59:322
469825012,21sleep14782-21perl07:09:521
290182505,15sleep00-21swapper/011:52:000
1954925012,22sleep119560-21df_inode09:39:491
1392225011,23sleep113462-21apt-get11:04:361
1392225011,23sleep113462-21apt-get11:04:361
1923724912,17sleep10-21swapper/109:39:321
1923724912,17sleep10-21swapper/109:39:321
206824811,21sleep02173-21users10:25:060
1864324810,22sleep218639-21apt-get11:19:432
1864324810,22sleep218639-21apt-get11:19:432
1733124813,18sleep10-21swapper/111:14:541
1334224811,21sleep113349-21diskstats09:19:501
777224711,20sleep27862-21memory10:44:552
70124611,17sleep00-21swapper/012:05:160
456324610,19sleep24668-21users12:15:032
1559524614,16sleep00-21swapper/011:09:520
2616924513,14sleep00-21swapper/011:44:330
1302424514,17sleep10-21swapper/109:19:341
1308124311,16sleep20-21swapper/209:19:352
7652429,17sleep10-21swapper/108:39:441
7652429,17sleep10-21swapper/108:39:441
174192366,14sleep20-21swapper/207:49:462
174192366,14sleep20-21swapper/207:49:462
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional