You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-21 - 08:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Fri Aug 21, 2026 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
107812260910,2587sleep16138-21kworker/1:122:18:211
2278825087,489sleep022832-21ntpq19:29:490
7052246721,429sleep17064-21/usr/sbin/munin23:49:471
2096239513,366sleep12190-21/usr/sbin/munin21:49:501
17355236929,325sleep017485-21awk00:20:230
20995230368,217sleep021032-21latency_hist00:34:280
13481227953,204sleep013542-21cat00:09:490
13481227953,204sleep013542-21cat00:09:490
13681227518,221sleep0781-21sendmail-mta22:25:270
13681227518,221sleep0781-21sendmail-mta22:25:270
25318227215,240sleep225259-21latency_hist19:39:292
15614226910,216sleep0314-21rs:main0
14937226350,192sleep015090-21cmp20:49:280
8601226113,196sleep08677-21fschecks_time23:54:470
21456224148,176sleep021574-21grep22:49:580
4951223831,139sleep00-21swapper/021:59:420
666822299,207sleep1367-21snmpd20:22:441
9364222110,197sleep09512-21latency_hist22:14:280
19738222133,171sleep019786-21apt-config21:04:260
29689222019,91sleep00-21swapper/021:34:470
29689222019,91sleep00-21swapper/021:34:470
15701221616,188sleep314431-21kworker/3:000:15:213
20949221059,29sleep00-21swapper/019:24:450
7180220616,143sleep07272-21fschecks_time20:24:450
3634220514,137sleep03719-21latency_hist23:39:280
3634220514,137sleep03719-21latency_hist23:39:280
957220173,68sleep0724-21/usr/sbin/munin23:29:470
29022219158,83sleep00-21swapper/023:14:570
5307219056,80sleep00-21swapper/020:19:280
1561218335,29sleep00-21swapper/020:04:590
15811218212,155sleep115831-21open_inodes20:49:581
289842176109,47sleep329016-21proc_pri23:14:563
1757421699,146sleep00-21swapper/020:54:570
2399216828,128sleep02491-21cat21:49:570
11747216819,134sleep00-21swapper/000:04:480
45421628,143sleep01-21systemd23:29:270
26774215720,119sleep226790-21kernelversion21:24:512
3175221569,91sleep00-21swapper/021:39:560
3175221569,91sleep00-21swapper/021:39:560
2590021548,97sleep025902-21sshd21:20:210
22292215422,19sleep00-21swapper/022:54:380
2376021539,91sleep023800-21ntpq21:14:520
10226215330,78sleep00-21swapper/022:14:570
3045321468,126sleep030523-21cat19:54:490
3045321468,126sleep030523-21cat19:54:490
2697214522,108sleep12736-21ntp_kernel_pll_23:34:531
2697214522,108sleep12736-21ntp_kernel_pll_23:34:531
1810821437,79sleep00-21swapper/022:39:520
20618213916,108sleep320636-21unin-run19:24:273
2039221386,119sleep00-21swapper/021:04:510
6969213618,99sleep27022-21cpuspeed23:49:442
3738213542,32sleep03782-21ntp_kernel_err21:54:500
3715213511,111sleep03741-21latency_hist20:14:270
17723213548,47sleep017735-21apt-get19:14:390
2954121308,111sleep0367-21snmpd19:50:010
23126212775,41sleep01-21systemd21:14:280
5514211928,67sleep25615-21fschecks_time23:44:472
932021169,69sleep09485-21unixbench_multi20:29:580
13304211618,29sleep00-21swapper/020:44:280
11446211652,17sleep00-21swapper/020:39:270
83721159,55sleep00-21swapper/021:44:560
83721159,55sleep00-21swapper/021:44:560
23347211313,76sleep223348-21/usr/sbin/munin21:14:382
24337211229,35sleep024372-21ps22:59:550
873721117,14sleep20-21swapper/223:54:492
2173211117,72sleep12235-21latency_hist20:09:281
31362211011,83sleep031397-21awk23:24:260
26980210957,16sleep00-21swapper/021:24:580
32439210739,51sleep032580-21unixbench_multi19:59:580
28348210722,70sleep228349-21cron19:49:262
16339210746,18sleep00-21swapper/019:09:550
1629321065,31sleep00-21swapper/020:50:250
28106210555,15sleep00-21swapper/019:46:210
24139210410,80sleep224175-21latency22:59:492
30519210311,37sleep10-21swapper/123:19:511
28142210350,13sleep00-21swapper/021:29:460
28142210350,13sleep00-21swapper/021:29:460
1941021038,58sleep019428-21fschecks_count19:19:440
25802210121,20sleep00-21swapper/019:39:470
8778210016,66sleep38916-21awk22:09:593
23035210022,55sleep323183-21unixbench_singl22:54:573
11738210022,61sleep311637-21/usr/sbin/munin00:04:473
74929815,35sleep20-21swapper/223:29:412
274872979,66sleep327529-21ntp_kernel_pll_19:44:513
2074729612,20sleep3367-21snmpd00:31:563
1979029619,51sleep119881-21ls21:04:271
1518529621,16sleep00-21swapper/000:14:530
3015929519,50sleep130178-21apt-get19:54:381
3015929519,50sleep130178-21apt-get19:54:381
1490829325,45sleep315073-21unixbench_singl19:04:583
1028429340,39sleep010345-21cat23:59:470
448329218,57sleep00-21swapper/023:39:570
1607529235,42sleep00-21swapper/022:34:390
541829136,40sleep00-21swapper/023:44:410
1210029028,49sleep011162-21/usr/sbin/munin22:20:020
2095528920,54sleep321084-21unixbench_multi21:05:063
1812928925,50sleep018131-21apt-get00:24:390
1451728917,56sleep314488-21latency_hist00:14:283
1451728917,56sleep314488-21latency_hist00:14:283
68942889,27sleep07049-21latency_hist20:24:290
1901628810,21sleep00-21swapper/022:44:280
648228734,38sleep00-21swapper/022:04:390
3191828613,58sleep131973-21latency23:24:471
2855528527,26sleep20-21swapper/221:29:562
2855528527,26sleep20-21swapper/221:29:562
2719328518,45sleep327340-21grep21:25:013
2638028519,55sleep00-21swapper/023:05:320
1411628524,37sleep214278-21sort22:29:282
1411628524,37sleep214278-21sort22:29:282
1045228531,25sleep20-21swapper/220:34:492
3037228425,31sleep20-21swapper/223:19:482
1655828421,44sleep316051-21apt-get00:19:373
1200528419,48sleep211990-21apt-get20:39:452
2724428331,35sleep00-21swapper/023:09:480
2383428318,44sleep323348-21/usr/sbin/munin21:14:553
1938628216,48sleep319444-21/usr/sbin/munin22:44:483
890628115,20sleep30-21swapper/320:29:493
621328125,36sleep36221-21modprobe22:04:263
2103328123,31sleep10-21swapper/100:34:291
1057428115,51sleep10-21swapper/120:34:521
459128012,52sleep14733-21awk23:39:581
3201728019,41sleep332057-21ntpq23:24:493
2842428016,29sleep10-21swapper/121:29:511
2842428016,29sleep10-21swapper/121:29:511
513227917,33sleep10-21swapper/121:59:471
2464327919,39sleep124791-21grep19:34:551
1953227916,27sleep20-21swapper/219:19:462
237902789,16sleep023932-21modprobe19:34:300
2130927821,14sleep00-21swapper/000:34:440
1480527817,42sleep20-21swapper/220:49:262
56182770,6sleep30-21swapper/323:44:473
3165027722,28sleep20-21swapper/223:24:402
2200027725,29sleep30-21swapper/321:09:463
822427619,35sleep38231-21modprobe23:54:253
228527620,42sleep22321-21proc_pri21:49:552
209592760,6sleep10-21swapper/119:24:451
1689227622,38sleep316887-21apt-get20:54:403
1619027625,26sleep20-21swapper/200:19:282
374227517,38sleep13885-21latency_hist20:14:291
2823127523,32sleep128247-21apt23:14:261
2554327526,26sleep20-21swapper/219:39:412
1786127519,22sleep20-21swapper/222:39:462
1514027527,19sleep10-21swapper/122:30:281
78812748,52sleep06944-21/usr/sbin/munin23:50:020
2308427422,30sleep223103-21modprobe21:14:262
1710327415,25sleep20-21swapper/220:54:472
404527116,38sleep14140-21perl20:14:451
2786527118,23sleep20-21swapper/223:09:582
1252627121,34sleep312676-21latency_hist22:24:293
67127022,24sleep30-21swapper/321:44:513
67127022,24sleep30-21swapper/321:44:513
54927015,30sleep10-21swapper/121:44:491
54927015,30sleep10-21swapper/121:44:491
449327016,30sleep30-21swapper/320:14:553
2209727016,39sleep122184-21sed21:09:481
1821127019,33sleep218276-21ps19:14:572
1557427014,39sleep30-21swapper/320:49:523
1334027018,23sleep10-21swapper/120:44:291
1925026910,17sleep20-21swapper/200:27:312
1551926925,26sleep10-21swapper/119:09:271
2520726824,22sleep10-21swapper/123:04:281
2205826824,19sleep20-21swapper/222:54:282
147126819,23sleep30-21swapper/323:29:583
1269826819,26sleep30-21swapper/320:41:203
1178326819,25sleep10-21swapper/100:04:491
1028026821,23sleep30-21swapper/320:34:463
1015226822,22sleep30-21swapper/322:14:563
916526719,31sleep20-21swapper/220:29:562
502326715,31sleep25032-21/usr/sbin/munin21:59:442
313112674,16sleep30-21swapper/323:23:483
2878926723,29sleep128803-21uname23:14:501
2551026719,26sleep10-21swapper/123:04:451
1755426715,22sleep30-21swapper/319:14:343
1493326719,24sleep30-21swapper/322:30:223
3152326623,19sleep30-21swapper/319:59:283
3152326623,19sleep30-21swapper/319:59:283
2994426620,21sleep30-21swapper/319:54:283
600426518,27sleep26026-21/usr/sbin/munin20:19:552
2258026524,19sleep30-21swapper/300:39:283
522726415,34sleep25268-21awk20:19:262
3010326416,25sleep30-21swapper/321:34:583
3010326416,25sleep30-21swapper/321:34:583
2890426412,33sleep128920-21/usr/sbin/munin19:49:491
1988526416,28sleep219751-21apt-get21:04:282
1587026417,31sleep215936-21fschecks_time19:09:452
1412126424,18sleep10-21swapper/120:44:571
13082647,45sleep20-21swapper/221:46:322
13082647,45sleep20-21swapper/221:46:322
1273826423,22sleep112807-21df22:24:431
1273826423,22sleep112807-21df22:24:431
355626318,26sleep23618-21sendmail-msp23:39:262
355626318,26sleep23618-21sendmail-msp23:39:262
3157126321,24sleep30-21swapper/321:39:513
3157126321,24sleep30-21swapper/321:39:513
296062636,11sleep10-21swapper/121:34:441
296062636,11sleep10-21swapper/121:34:441
1466526329,18sleep00-21swapper/019:04:550
977326219,29sleep39800-21munin-run23:59:263
81812626,16sleep20-21swapper/220:27:182
717126221,23sleep30-21swapper/320:24:443
397326221,19sleep20-21swapper/221:54:562
2496226212,35sleep125029-21df21:19:421
2454426221,17sleep30-21swapper/322:59:583
2133526212,19sleep20-21swapper/219:24:552
2123326216,20sleep20-21swapper/200:34:422
1354826210,34sleep313615-21df20:44:433
910526110,13sleep10-21swapper/120:29:551
31202617,12sleep20-21swapper/220:09:572
2632026112,23sleep30-21swapper/323:04:593
2574826122,24sleep30-21swapper/319:39:463
202742617,12sleep10-21swapper/119:20:261
1926626110,38sleep3367-21snmpd00:27:453
1702126114,18sleep10-21swapper/120:54:461
1610326116,25sleep116189-21gpgv00:19:271
113226119,22sleep30-21swapper/320:04:493
832426012,24sleep20-21swapper/222:09:502
757226021,18sleep10-21swapper/120:24:541
406526018,24sleep34216-21unixbench_multi21:54:573
2621926017,26sleep2289-21dbus-daemon21:24:272
2373026014,22sleep30-21swapper/319:34:293
2159026019,25sleep221721-21latency_hist21:09:292
199022607,12sleep20-21swapper/222:44:592
386825916,25sleep33864-21apt-get23:39:393
3258125913,31sleep131736-21/usr/sbin/munin19:59:591
2828225918,19sleep30-21swapper/323:14:283
2691325921,22sleep10-21swapper/121:24:541
2618225913,29sleep226319-21uptime23:04:582
1920325917,25sleep119197-21apt-get22:44:421
1454425920,16sleep20-21swapper/222:29:572
129525914,19sleep10-21swapper/123:29:561
650525816,26sleep36500-21apt-get22:04:423
3107525817,25sleep231086-21/usr/sbin/munin19:54:592
3107525817,25sleep231086-21/usr/sbin/munin19:54:592
229925817,24sleep32294-21apt-get23:34:433
229925817,24sleep32294-21apt-get23:34:433
1896225824,20sleep312-21rcu_sched22:44:273
1822425818,18sleep10-21swapper/100:24:481
137542586,10sleep30-21swapper/322:26:223
137542586,10sleep30-21swapper/322:26:223
107425819,22sleep21131-21irqstats20:04:482
959725714,25sleep29593-21apt-get22:14:382
95252575,39sleep10-21swapper/123:56:171
3126225718,21sleep10-21swapper/121:39:441
3126225718,21sleep10-21swapper/121:39:441
2880525714,25sleep228889-21memory23:14:512
1741825714,17sleep10-21swapper/122:39:291
858925614,18sleep10-21swapper/122:09:571
62425617,19sleep20-21swapper/220:04:282
241325615,18sleep30-21swapper/320:09:423
2368025613,26sleep123712-21/usr/sbin/munin19:34:271
2102025615,18sleep30-21swapper/322:49:493
1705325612,17sleep20-21swapper/200:20:172
1578625620,22sleep315853-21df19:09:423
1578425613,26sleep115778-21apt-get19:09:401
121122566,39sleep20-21swapper/222:20:122
1060425614,26sleep110677-21/usr/sbin/munin23:59:561
1016825611,19sleep30-21swapper/323:59:423
3115325513,20sleep20-21swapper/221:39:372
3115325513,20sleep20-21swapper/221:39:372
2861325512,24sleep228632-21apt-get19:49:382
2717425511,15sleep10-21swapper/119:44:451
2464225516,24sleep124643-21cron21:19:251
2229425514,24sleep222288-21apt-get22:54:402
1802125518,17sleep10-21swapper/122:39:491
1450725516,23sleep214516-21munin-run00:14:272
1450725516,23sleep214516-21munin-run00:14:272
405325417,20sleep20-21swapper/223:39:462
2449525413,25sleep324564-21sensors_temp19:34:533
2173025414,24sleep321776-21users19:25:003
2131625420,15sleep10-21swapper/100:34:451
2093425418,20sleep10-21swapper/122:49:461
1464425417,18sleep20-21swapper/219:04:532
1008225414,24sleep210094-21apt-get23:59:372
2960225312,24sleep229599-21apt-get21:34:412
2960225312,24sleep229599-21apt-get21:34:412
1205925316,18sleep20-21swapper/200:04:542
1628325215,20sleep316328-21if_eth022:34:463
572625116,15sleep30-21swapper/320:19:473
378425114,23sleep13821-21ntp_states21:54:511
2503825119,17sleep30-21swapper/321:19:443
1658725115,20sleep116581-21apt-get00:19:391
1551425116,15sleep20-21swapper/220:49:512
1164225114,21sleep111648-21ntp_states22:19:541
687425012,24sleep16917-21tr22:04:521
541925011,22sleep15413-21apt-get23:44:421
3235625013,20sleep332435-21find19:59:573
1321025015,21sleep113280-21df00:09:421
1321025015,21sleep113280-21df00:09:421
2287024911,23sleep222872-21ntp_states19:29:512
1966024913,19sleep20-21swapper/200:29:372
2253924815,20sleep122565-21irqstats22:54:471
189324813,20sleep31900-21df_inode21:49:443
258152478,13sleep10-21swapper/119:39:481
1883024717,17sleep30-21swapper/320:59:493
1744324711,21sleep117453-21ntp_states19:14:321
160782478,14sleep20-21swapper/222:34:412
2805624613,15sleep30-21swapper/321:29:413
2805624613,15sleep30-21swapper/321:29:413
207822428,14sleep10-21swapper/121:05:041
65042407,15sleep20-21swapper/222:04:412
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional