You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-10 - 05:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Sun May 10, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
139192193837,19341sleep013887-21apt-get00:00:110
19211210708,1049sleep019255-21ntpq20:50:070
17423259822,558sleep317509-21/usr/sbin/munin00:11:043
1571025328,424sleep015715-21find20:39:530
3027225007,438sleep030365-21latency_hist23:09:430
22595247715,448sleep322597-21sshd19:18:053
10223239214,268sleep010253-21grep23:49:410
19969236878,270sleep020012-21wc19:10:190
3268523588,334sleep0313-21tune2fs19:50:010
3268523588,334sleep0313-21tune2fs19:50:010
2779223348,290sleep027841-21awk23:00:110
2779223348,290sleep027841-21awk23:00:110
177562316120,178sleep0225-21systemd-journal22:29:410
2807231513,288sleep12814-21df_inode23:24:591
2807231513,288sleep12814-21df_inode23:24:591
20488229637,245sleep020585-21fschecks_time00:20:020
13325229518,260sleep313373-40logrotate23:59:423
23195226310,238sleep223199-21ntpq22:45:102
16846225718,140sleep00-21swapper/000:09:440
8702223532,61sleep00-21swapper/021:59:580
25776223210,203sleep025935-21grep21:10:140
4402221721,182sleep24513-21sed21:45:132
29446221622,159sleep0194-21jbd2/mmcblk2p2-21:24:110
25805221422,18sleep00-21swapper/022:54:530
7622220134,126sleep07686-21mailstats21:55:210
559322016,28sleep00-21swapper/021:50:040
20209218910,164sleep020293-21expr22:35:110
20209218910,164sleep020293-21expr22:35:110
31529218114,46sleep00-21swapper/021:29:530
28672217831,125sleep328701-21munin-plugin-st23:04:423
28672217831,125sleep328701-21munin-plugin-st23:04:423
9340217713,150sleep19383-21ntpq23:45:061
22564217714,142sleep222638-21cat00:25:132
475621747,137sleep0363-21snmpd21:46:000
30508217432,106sleep00-21swapper/023:09:560
21699217418,138sleep321734-21awk00:24:413
16467217430,115sleep316486-21apt-get22:24:523
13574217118,136sleep213770-21cut23:59:452
580821706,29sleep00-21swapper/020:06:040
18909216732,119sleep018989-21fschecks_time00:15:010
1438521608,141sleep014394-21modprobe22:16:410
13450215521,24sleep00-21swapper/020:30:160
26345215334,77sleep026348-21cron00:39:410
12551215027,100sleep312691-21unixbench_singl22:10:143
21782214710,122sleep021930-21date20:59:440
4042214612,122sleep04073-21latency_hist23:29:420
4042214612,122sleep04073-21latency_hist23:29:420
846121457,127sleep0363-21snmpd23:41:580
19362214310,121sleep019399-21date22:34:420
23992213913,107sleep00-21swapper/000:30:080
28825213610,113sleep028838-21awk19:35:150
221021357,113sleep00-21swapper/021:39:540
30604213423,45sleep00-21swapper/021:25:120
2741721335,30sleep00-21swapper/019:31:560
2350021295,30sleep00-21swapper/021:04:530
2350021295,30sleep00-21swapper/021:04:530
20870212923,41sleep00-21swapper/020:55:120
1916212711,22sleep00-21swapper/021:39:420
24794212624,87sleep1225-21systemd-journal00:34:411
1698212656,59sleep01740-21grep19:54:580
1698212656,59sleep01740-21grep19:54:580
2280221257,82sleep022891-21ls19:19:440
14666212516,83sleep114770-21unixbench_multi00:00:321
2583521246,30sleep00-21swapper/019:26:300
2675921227,96sleep00-21swapper/021:15:020
2675921227,96sleep00-21swapper/021:15:020
1781421219,76sleep017887-21mailstats00:11:140
2524521205,30sleep00-21swapper/022:50:540
15069211813,90sleep115155-21fw_packets22:20:011
1310721185,52sleep00-21swapper/023:56:560
2249521178,94sleep022621-21latency_hist22:44:440
1047721165,28sleep00-21swapper/023:49:450
19357211419,78sleep219426-21grep00:15:112
10669211316,83sleep110232-21/usr/sbin/munin22:05:071
10669211316,83sleep110232-21/usr/sbin/munin22:05:071
2953211128,58sleep32979-21modprobe19:59:413
2953211128,58sleep32979-21modprobe19:59:413
23131211124,70sleep123220-21cut19:20:011
20756210926,61sleep320788-21cut19:14:423
882921089,60sleep00-21swapper/020:15:170
7879210817,76sleep38019-21unixbench_singl21:55:243
3419210817,69sleep33558-21tail23:25:133
3419210817,69sleep33558-21tail23:25:133
583721078,59sleep00-21swapper/023:34:550
3081121078,65sleep030809-21munin-run19:44:440
13123210723,16sleep00-21swapper/022:14:430
12883210712,76sleep112942-21/usr/sbin/munin20:30:041
31022210621,73sleep00-21swapper/019:44:520
1416021068,40sleep00-21swapper/020:34:560
2276121036,81sleep00-21swapper/000:25:150
28675210225,18sleep10-21swapper/121:20:081
18314210212,76sleep218348-21wc22:30:042
15622210265,16sleep00-21swapper/022:20:130
1236321027,33sleep00-21swapper/020:29:430
9590210118,64sleep39712-21wc23:45:123
11392210130,33sleep00-21swapper/022:08:050
11392210130,33sleep00-21swapper/022:08:050
5335210025,51sleep15327-21apt-get21:49:571
32513210018,17sleep00-21swapper/023:15:080
24476210034,50sleep024684-21ls19:24:450
18274210026,47sleep30-21swapper/319:05:053
235492996,17sleep30-21swapper/322:45:153
229702996,31sleep00-21swapper/022:45:040
216129915,69sleep12184-21pluginstate19:55:111
216129915,69sleep12184-21pluginstate19:55:111
581229814,68sleep15831-21apt-get23:34:521
103712988,35sleep00-21swapper/020:20:140
426029724,49sleep34279-21apt-get23:29:533
2109129728,46sleep321243-21tail20:55:153
2109129728,46sleep321243-21tail20:55:153
793229611,50sleep20-21swapper/220:14:592
34812966,31sleep00-21swapper/020:00:030
2535829624,53sleep325422-21cat21:10:053
2205929624,51sleep122098-21cut21:00:011
2205929624,51sleep122098-21cut21:00:011
830229519,61sleep38339-21sed20:15:083
2204329517,62sleep322052-21/usr/sbin/munin20:59:593
2204329517,62sleep322052-21/usr/sbin/munin20:59:593
3111929411,64sleep30-21swapper/319:44:593
1960429414,60sleep219672-21df22:34:572
1960429414,60sleep219672-21df22:34:572
11892948,17sleep00-21swapper/023:19:570
745829333,27sleep00-21swapper/020:12:220
60929212,65sleep1676-21processes19:50:101
60929212,65sleep1676-21processes19:50:101
2663529221,49sleep326630-21apt-get21:14:573
2663529221,49sleep326630-21apt-get21:14:573
2264129013,60sleep322759-21sed00:25:143
1845029014,57sleep10-21swapper/122:30:071
1521329021,39sleep315232-21uname22:20:043
1329829018,47sleep313446-21grep20:30:153
1115029025,45sleep311210-21fschecks_time20:25:023
2639728918,43sleep226521-21latency_hist21:14:442
2399128817,52sleep1225-21systemd-journal22:49:421
491928720,51sleep14974-21fschecks_count20:05:001
235828716,35sleep10-21swapper/121:40:031
2942528616,55sleep329449-21proc_pri23:05:103
2244328517,54sleep122457-21idleruntime-cro22:44:411
41128421,18sleep20-21swapper/221:34:432
693028327,29sleep20-21swapper/221:54:472
3151228317,48sleep231527-21/usr/sbin/munin19:45:102
2819828315,55sleep228217-21apt21:19:532
2000428314,55sleep320055-21cat22:35:063
2000428314,55sleep320055-21cat22:35:063
1270028316,50sleep212760-21/usr/sbin/munin20:30:012
447328212,24sleep00-21swapper/023:30:030
1363128017,43sleep113697-21wc22:15:051
2303827918,29sleep00-21swapper/021:02:060
2303827918,29sleep00-21swapper/021:02:060
1831427815,28sleep00-21swapper/020:46:470
2642327723,28sleep20-21swapper/200:39:432
1161727721,34sleep111685-21grep20:25:121
3071627619,41sleep230763-21cat23:10:022
2721127615,48sleep227290-21sort21:15:132
2721127615,48sleep227290-21sort21:15:132
2090827621,37sleep221032-21munin-node19:14:452
1723527516,42sleep116732-21apt-get00:09:531
573227414,42sleep35768-21/usr/sbin/munin21:50:073
3116527423,28sleep30-21swapper/321:29:443
2872427415,42sleep128804-21latency_hist23:04:441
2872427415,42sleep128804-21latency_hist23:04:441
2792527418,30sleep30-21swapper/319:34:573
1976727419,36sleep219785-21kernelversion19:10:132
1541827424,27sleep10-21swapper/120:39:421
1052627415,23sleep20-21swapper/222:05:042
1052627415,23sleep20-21swapper/222:05:042
37562735,10sleep10-21swapper/121:44:521
2869427317,18sleep30-21swapper/321:20:103
2100127314,35sleep20-21swapper/220:55:142
2100127314,35sleep20-21swapper/220:55:142
1924027318,24sleep20-21swapper/219:09:442
1872827322,31sleep30-21swapper/322:30:133
1707027317,41sleep217153-21cat22:25:112
1703827318,27sleep20-21swapper/220:44:432
908027215,36sleep18677-21/usr/sbin/munin22:00:071
3081227217,29sleep10-21swapper/119:44:451
2905027220,22sleep20-21swapper/223:05:012
1945427224,23sleep20-21swapper/220:50:132
1051227220,23sleep30-21swapper/323:49:563
1163227126,29sleep1225-21systemd-journal22:09:441
3230927018,36sleep132320-21modprobe19:49:421
3223527019,25sleep20-21swapper/223:15:022
2856927028,23sleep20-21swapper/219:35:132
191312705,18sleep00-21swapper/019:09:140
1628627019,25sleep30-21swapper/320:40:123
1214527015,42sleep112173-21fschecks_count23:54:591
395426930,17sleep20-21swapper/220:00:132
2629926911,41sleep126334-21ps22:55:111
2464626917,29sleep20-21swapper/222:50:032
2430126916,36sleep224363-21apt-config19:24:432
1554126918,29sleep10-21swapper/100:05:011
1081626923,25sleep10-21swapper/120:24:431
933726828,22sleep30-21swapper/322:00:133
58322689,16sleep20-21swapper/223:34:532
2528626812,40sleep325292-21ntp_states19:25:123
2243926815,37sleep222479-21/usr/sbin/munin21:00:102
2243926815,37sleep222479-21/usr/sbin/munin21:00:102
81932679,43sleep10-21swapper/121:57:161
32426721,28sleep1471-21grep23:15:141
2219826715,21sleep10-21swapper/100:25:041
218526711,34sleep20-21swapper/219:55:122
218526711,34sleep20-21swapper/219:55:122
2607526610,43sleep10-21swapper/121:11:121
216622668,12sleep10-21swapper/119:15:041
1388926611,26sleep30-21swapper/323:59:543
1242726615,37sleep212550-21cat22:10:132
768926515,35sleep37740-21ls20:14:443
2531826518,32sleep125400-21mailstats19:25:141
2436426519,23sleep30-21swapper/319:24:443
2010426512,35sleep30-21swapper/320:54:423
1796726524,21sleep20-21swapper/200:11:162
820126420,21sleep10-21swapper/120:15:061
1922126416,29sleep119239-21munin-run19:09:431
1114526410,32sleep20-21swapper/220:25:012
959826315,25sleep20-21swapper/220:20:002
938526311,36sleep29417-21ntp_states23:45:072
490326314,27sleep30-21swapper/320:04:583
349226319,27sleep31-21systemd21:44:423
3024026324,17sleep30-21swapper/319:40:153
2015526321,17sleep10-21swapper/120:54:441
741026218,29sleep27502-21/usr/sbin/munin21:55:152
451426220,18sleep30-21swapper/321:45:143
318926212,36sleep23191-21ntp_states23:25:082
318926212,36sleep23191-21ntp_states23:25:082
2693126218,19sleep10-21swapper/119:30:111
2673526215,32sleep326744-21needreboot19:30:053
2390726223,19sleep20-21swapper/200:30:062
1034326218,28sleep310403-21/usr/sbin/munin22:05:013
1034326218,28sleep310403-21/usr/sbin/munin22:05:013
353926118,17sleep10-21swapper/120:00:041
2407126122,18sleep30-21swapper/300:30:113
1957726118,23sleep10-21swapper/122:34:521
1957726118,23sleep10-21swapper/122:34:521
159326115,30sleep31641-21grep23:20:083
702726015,27sleep17153-21sed20:10:131
2473726011,36sleep124738-21sshd21:09:061
2473726011,36sleep124738-21sshd21:09:061
1470226018,26sleep30-21swapper/322:19:433
1415826015,27sleep314154-21apt-get20:34:553
127626015,30sleep11323-21fschecks_count23:20:021
1091126014,20sleep10-21swapper/123:50:071
621725920,21sleep30-21swapper/323:35:063
2995925912,31sleep229965-21ntp_states19:40:092
274562598,40sleep2369-21sshd19:32:262
2568825919,23sleep30-21swapper/300:35:123
2138825915,21sleep20-21swapper/222:40:012
197972596,41sleep3363-21snmpd00:16:173
1899025916,26sleep10-21swapper/100:15:021
363925816,28sleep33675-21ntpq20:00:063
2609925813,29sleep326130-21latency22:55:053
1182425818,19sleep20-21swapper/223:54:442
74925716,23sleep3835-21perl21:35:013
474525712,26sleep24751-21ntp_states23:30:092
3260325716,25sleep332670-21swap23:15:113
2391525712,21sleep20-21swapper/221:05:072
2391525712,21sleep20-21swapper/221:05:072
645225618,18sleep20-21swapper/220:09:582
3107925619,21sleep21-21systemd21:29:422
199625614,17sleep20-21swapper/221:39:442
3169225512,19sleep10-21swapper/121:30:101
3127925519,15sleep10-21swapper/123:10:131
2953725517,18sleep10-21swapper/123:05:121
2087725517,22sleep11-21systemd00:20:111
2080725515,25sleep120889-21munin-node19:14:441
174282558,17sleep10-21swapper/122:25:311
1728625519,21sleep20-21swapper/220:45:002
948825413,20sleep30-21swapper/320:19:533
872725417,21sleep30-21swapper/323:44:433
64825412,25sleep1642-21apt-get21:34:551
2352425413,24sleep323520-21apt-get21:04:563
2352425413,24sleep323520-21apt-get21:04:563
1598925414,23sleep316060-21sensors_temp00:05:113
877525314,24sleep28784-21diskstats22:00:002
560125316,17sleep10-21swapper/123:34:441
2997925314,22sleep130036-21irqstats21:25:011
2792325313,23sleep127918-21apt-get19:34:551
2719225315,15sleep20-21swapper/222:59:442
2613825316,22sleep226233-21latency_hist19:29:432
1573825317,22sleep115802-21cpuspeed20:39:581
1544425316,23sleep215496-21munin-run20:39:432
1501625314,24sleep215023-21df_inode22:19:582
1333625311,23sleep213355-21apt-get22:14:542
3050625212,24sleep330502-21apt-get23:09:553
2802725218,15sleep20-21swapper/223:00:172
2802725218,15sleep20-21swapper/223:00:172
263825218,18sleep20-21swapper/221:40:092
1545125213,23sleep215448-21apt-get00:04:532
790225114,15sleep10-21swapper/123:40:091
3259725113,22sleep332592-21apt-get19:49:563
3259725113,22sleep332592-21apt-get19:49:563
3163325112,25sleep331639-21df_inode21:30:003
2427625112,22sleep324272-21apt-get22:49:523
1386825110,20sleep20-21swapper/223:59:532
60382507,32sleep20-21swapper/220:09:012
213242506,35sleep10-21swapper/120:55:291
213242506,35sleep10-21swapper/120:55:291
11092506,35sleep20-21swapper/219:50:562
11092506,35sleep20-21swapper/219:50:562
2555324915,19sleep10-21swapper/100:35:101
216982495,35sleep20-21swapper/200:24:402
182242486,30sleep30-21swapper/320:45:373
1755924812,21sleep117648-21/usr/sbin/munin20:45:071
871524715,17sleep20-21swapper/223:44:422
2670724718,14sleep10-21swapper/121:14:591
2670724718,14sleep10-21swapper/121:14:591
2125724710,14sleep30-21swapper/322:39:583
245522468,15sleep10-21swapper/122:50:021
118124611,20sleep21184-21apt-get23:19:552
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional