You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-14 - 16:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Tue Jul 14, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2157278925,726sleep32201-21ntpq10:04:103
25439268021,641sleep125530-21/usr/sbin/munin09:34:151
25439268021,641sleep125530-21/usr/sbin/munin09:34:151
151325569,538sleep026729-1kworker/0:1H11:44:170
14555242743,229sleep314592-21latency_hist07:18:463
16575242270,204sleep316599-21fschecks_count10:49:043
12012241046,203sleep312041-21fschecks_time10:35:033
12012241046,203sleep312041-21fschecks_time10:35:033
5120240144,340sleep15188-21/usr/sbin/munin10:14:171
13637234315,310sleep013762-21latency_hist12:23:460
202922299117,53sleep30-21swapper/309:19:033
290262286113,54sleep30-21swapper/311:29:013
4762226872,175sleep34882-21timerwakeupswit08:29:163
20071226514,236sleep120077-21ntpq10:59:111
3813226389,63sleep33894-21mailstats10:09:153
10456225235,200sleep310653-21cat08:48:473
9276223219,183sleep09340-21fschecks_time12:09:040
22327222216,145sleep322410-21latency_hist07:43:463
18383221752,65sleep318472-21latency_hist09:13:463
30753221045,87sleep30-21swapper/308:09:103
32347220414,170sleep332050-21/usr/sbin/munin11:39:073
23873220415,173sleep223927-21latency_hist07:48:452
28040220233,19sleep30-21swapper/311:24:173
28040220233,19sleep30-21swapper/311:24:173
12086219110,153sleep30-21swapper/308:53:433
12086219110,153sleep30-21swapper/308:53:433
125721876,169sleep3892-21apt-get11:43:473
25038218645,82sleep324827-21/usr/sbin/munin09:34:053
25038218645,82sleep324827-21/usr/sbin/munin09:34:053
10821218326,71sleep30-21swapper/312:14:043
9809217817,142sleep19953-21cat10:28:471
6138217513,48sleep30-21swapper/308:35:103
12474217318,75sleep30-21swapper/307:13:003
26921216462,27sleep326925-21cron07:58:443
15226215931,89sleep315270-21unin-run09:03:453
1518321579,89sleep315280-21latency_hist12:28:463
30828215313,19sleep00-21swapper/009:53:450
1396621499,106sleep314049-21sed10:39:153
1396621499,106sleep314049-21sed10:39:153
1233214815,116sleep01289-21/usr/sbin/munin08:19:040
843221458,81sleep3364-21snmpd10:23:523
23400214518,45sleep30-21swapper/309:29:043
22715214511,72sleep30-21swapper/311:08:593
18619214215,66sleep311061-21cyclictest09:13:583
18311214211,97sleep318375-21cat10:54:093
21755214110,42sleep30-21swapper/309:24:003
9762214014,100sleep30-21swapper/312:09:153
21574214011,116sleep021645-21ls07:39:140
5891213512,109sleep35997-21grep10:14:363
5064213524,88sleep05072-21/usr/sbin/munin10:13:560
1237721348,40sleep30-21swapper/308:54:003
14896213216,91sleep214986-21fschecks_time07:19:052
23496212913,30sleep00-21swapper/009:29:050
1767621279,104sleep017750-21find09:09:150
31164212315,36sleep30-21swapper/309:54:043
304521239,99sleep33101-21tune2fs11:49:043
304521239,99sleep33101-21tune2fs11:49:043
253521225,15sleep00-21swapper/008:23:480
15599212123,76sleep30-21swapper/309:04:053
1145212136,24sleep30-21swapper/308:19:033
26964212010,41sleep30-21swapper/309:39:143
26964212010,41sleep30-21swapper/309:39:143
24113212010,61sleep30-21swapper/307:48:593
24113212010,61sleep30-21swapper/307:48:593
19116212027,81sleep3788-21sendmail-mta07:33:163
3135211742,32sleep33210-21switchtime08:24:163
747821167,68sleep30-21swapper/308:38:583
29991211626,79sleep330025-21ps09:49:113
7145211514,89sleep37188-21processes10:19:123
24727211412,44sleep30-21swapper/311:14:113
24727211412,44sleep30-21swapper/311:14:113
1498821147,17sleep30-21swapper/307:19:063
763621139,44sleep37683-21grep12:03:583
32264211326,29sleep30-21swapper/308:14:103
15856211324,37sleep30-21swapper/310:44:203
4573211122,31sleep30-21swapper/311:54:013
30574210919,32sleep30-21swapper/311:34:023
25936210920,30sleep30-21swapper/311:19:043
19003210910,38sleep119004-21sshd10:55:001
9929210819,70sleep110063-21unixbench_singl12:09:171
17798210821,60sleep017883-21cat10:53:450
1694221088,35sleep30-21swapper/312:33:553
1694221088,35sleep30-21swapper/312:33:553
15132210817,73sleep115162-21basename12:28:441
27394210723,28sleep30-21swapper/307:59:043
19609210612,37sleep30-21swapper/310:58:573
1944321068,39sleep30-21swapper/307:34:003
1534121069,13sleep00-21swapper/010:44:080
15027210512,75sleep215083-21fschecks_count10:44:022
9362210313,61sleep10-21swapper/108:44:081
3227210316,74sleep23248-21kernelversion11:49:072
3227210316,74sleep23248-21kernelversion11:49:072
458621027,12sleep00-21swapper/008:29:130
27979210118,70sleep30-21swapper/309:43:583
5712998,55sleep3612-21ntpq09:59:093
2585029922,42sleep10-21swapper/111:18:581
141929922,55sleep21395-21apt-get11:44:072
889529822,26sleep20-21swapper/210:24:322
1956329810,75sleep119654-21fschecks_time07:34:061
2137829717,58sleep121142-21/usr/sbin/munin11:04:041
2137829717,58sleep121142-21/usr/sbin/munin11:04:041
1863829726,49sleep018788-21grep07:29:170
587229611,68sleep15394-21apt-get08:33:511
1595829416,42sleep316023-21mailstats12:29:143
1010229416,59sleep110123-21fschecks_count10:29:031
2769029322,51sleep027699-21modprobe09:43:430
2769029322,51sleep027699-21modprobe09:43:430
1899129314,63sleep119024-21/usr/sbin/munin09:14:081
138872938,12sleep20-21swapper/210:39:142
138872938,12sleep20-21swapper/210:39:142
1232929349,26sleep30-21swapper/312:19:013
2843329212,63sleep228439-21ntp_states09:44:102
3094929111,64sleep031023-21find08:09:150
211672918,41sleep30-21swapper/311:03:583
211672918,41sleep30-21swapper/311:03:583
1044029029,37sleep110571-21head07:04:151
299692895,29sleep30-21swapper/308:07:403
2542428925,46sleep225548-21cat07:53:462
2542428925,46sleep225548-21cat07:53:462
1643028812,59sleep00-21swapper/007:24:040
2456228621,42sleep224588-21awk09:33:442
1405128610,61sleep014160-21timerwakeupswit10:39:160
1405128610,61sleep014160-21timerwakeupswit10:39:160
2176328520,49sleep221914-21sed07:39:162
551128415,54sleep15703-21latency_hist08:33:471
2409028415,55sleep124091-21munin-node07:48:571
2409028415,55sleep124091-21munin-node07:48:571
163562849,63sleep2364-21snmpd12:29:342
1485128411,48sleep20-21swapper/212:25:052
1568728313,54sleep015741-21cat09:04:060
118472837,17sleep20-21swapper/210:34:252
118472837,17sleep20-21swapper/210:34:252
1008228322,45sleep30-21swapper/310:29:013
973628225,30sleep20-21swapper/207:03:592
716528217,43sleep27187-21modprobe08:38:442
2293628210,54sleep30-21swapper/307:44:103
1411528214,52sleep014124-21vmstat07:14:190
1085728212,43sleep210804-21apt-get08:49:302
1085728212,43sleep210804-21apt-get08:49:302
80672806,15sleep20-21swapper/212:04:092
1578228015,50sleep215822-21expr09:04:082
461727911,18sleep20-21swapper/210:13:462
2560427923,28sleep10-21swapper/111:18:461
2560427923,28sleep10-21swapper/111:18:461
1030127922,32sleep00-21swapper/007:04:140
725227720,26sleep30-21swapper/312:02:333
2901027717,22sleep00-21swapper/008:04:060
222427718,29sleep20-21swapper/210:04:132
188227714,27sleep10-21swapper/111:44:261
184427725,23sleep10-21swapper/110:04:041
1774527721,38sleep217772-21idleruntime-cro10:53:432
1737827718,27sleep30-21swapper/307:25:533
3247027623,26sleep10-21swapper/108:14:161
2636127610,53sleep226436-21run-parts11:19:152
1184627616,40sleep111801-21apt-get10:34:241
1184627616,40sleep111801-21apt-get10:34:241
1035227621,29sleep20-21swapper/208:48:462
805927511,50sleep08142-21cat08:39:150
2049127511,50sleep0364-21snmpd07:36:030
607627414,32sleep00-21swapper/008:35:070
3217027412,44sleep232263-21memory08:14:092
2153727411,45sleep00-21swapper/011:04:070
2153727411,45sleep00-21swapper/011:04:070
834427321,35sleep08452-21sed12:04:160
171202730,7sleep20-21swapper/207:24:172
806927211,45sleep18189-21cmp10:23:461
388227217,20sleep20-21swapper/208:28:472
3116127212,43sleep231246-21users08:09:172
2587327213,43sleep025925-21/usr/sbin/munin07:54:040
1579127218,28sleep10-21swapper/112:29:081
3262227120,34sleep20-21swapper/211:39:142
3145727114,43sleep131494-21awk09:54:101
3058727115,26sleep10-21swapper/111:34:041
2895127112,46sleep128953-21cpu11:28:591
105712710,7sleep20-21swapper/212:13:462
456727014,41sleep14615-21systemd-run10:13:451
2423827015,40sleep224255-21apt11:13:542
2423827015,40sleep224255-21apt11:13:542
11092706,44sleep20-21swapper/208:19:022
2945326919,20sleep20-21swapper/208:04:162
1416326911,17sleep10-21swapper/110:39:171
1416326911,17sleep10-21swapper/110:39:171
1028626916,34sleep010299-21idleruntime-cro08:48:440
3025426823,23sleep20-21swapper/211:33:452
2839626813,39sleep128432-21ntp_states09:44:091
2777426813,39sleep027802-21/usr/sbin/munin11:24:100
2777426813,39sleep027802-21/usr/sbin/munin11:24:100
2306626822,25sleep20-21swapper/209:28:472
2227726822,29sleep122297-21modprobe07:43:441
1258326814,28sleep00-21swapper/007:13:460
2816326716,27sleep00-21swapper/009:44:040
1521026715,37sleep015252-21ntp_states07:19:110
1330026716,36sleep213369-21fschecks_time07:14:052
1195426720,22sleep10-21swapper/107:09:151
3241526620,25sleep00-21swapper/009:58:450
2790026620,19sleep10-21swapper/111:24:151
2790026620,19sleep10-21swapper/111:24:151
2680026616,24sleep20-21swapper/209:39:092
2680026616,24sleep20-21swapper/209:39:092
1871726622,22sleep00-21swapper/009:14:030
1247726611,33sleep00-21swapper/008:54:050
3255926515,33sleep132621-21/usr/sbin/munin11:39:131
257982655,12sleep00-21swapper/009:34:430
257982655,12sleep00-21swapper/009:34:430
1198426523,20sleep00-21swapper/012:18:250
80826413,37sleep1909-21timerandwakeup09:59:151
3034326412,24sleep10-21swapper/108:08:591
1164226420,25sleep20-21swapper/207:09:082
989726314,22sleep30-21swapper/307:04:053
3161426315,33sleep231685-21sensors_temp09:54:152
2632826314,35sleep226448-21cat07:54:152
1803726325,24sleep2225-21systemd-journal10:54:002
1348326314,34sleep113549-21wc07:14:081
1305626323,21sleep10-21swapper/112:19:191
61526211,21sleep00-21swapper/009:59:100
3121926220,23sleep20-21swapper/211:34:172
294042628,13sleep00-21swapper/011:29:110
1988326213,34sleep019933-21sed10:59:060
78672615,47sleep00-21swapper/010:22:520
375526118,26sleep00-21swapper/010:09:120
266926115,28sleep12757-21/usr/sbin/munin08:24:061
2594226110,38sleep026003-21fschecks_time11:19:050
2553126118,20sleep20-21swapper/209:34:162
2553126118,20sleep20-21swapper/209:34:162
1724026114,30sleep217313-21sed12:34:062
1724026114,30sleep217313-21sed12:34:062
150442619,43sleep20-21swapper/209:01:302
814326017,27sleep18262-21ls08:39:161
1984226017,25sleep20-21swapper/207:34:122
1371626016,21sleep10-21swapper/108:58:461
1230226018,24sleep20-21swapper/212:18:552
1226226013,33sleep012302-21ntp_kernel_err10:35:080
1226226013,33sleep012302-21ntp_kernel_err10:35:080
1113426016,27sleep111146-21/usr/sbin/munin08:49:421
1113426016,27sleep111146-21/usr/sbin/munin08:49:421
669825911,18sleep20-21swapper/211:59:172
469325914,23sleep20-21swapper/208:29:152
2256925916,26sleep122563-21apt-get07:44:001
2240225911,30sleep022498-21sed09:24:170
172925917,26sleep00-21swapper/010:03:540
119322595,42sleep00-21swapper/008:51:470
119322595,42sleep00-21swapper/008:51:470
2397725816,25sleep01-21systemd11:13:440
1962725815,19sleep20-21swapper/210:59:002
1830625821,20sleep10-21swapper/112:38:451
1830625821,20sleep10-21swapper/112:38:451
1521025817,23sleep10-21swapper/110:44:051
1125225814,24sleep20-21swapper/212:14:152
846025715,25sleep08455-21apt-get10:23:570
802925718,24sleep10-21swapper/112:04:081
429325716,17sleep00-21swapper/011:53:460
429325716,17sleep00-21swapper/011:53:460
2866025714,26sleep228710-21munin-run11:28:442
2866025714,26sleep228710-21munin-run11:28:442
2770125711,31sleep127739-21unin-run09:43:441
2770125711,31sleep127739-21unin-run09:43:441
2041725715,23sleep120474-21irqstats09:19:051
1131825711,30sleep211362-21readlink10:33:452
608325618,23sleep10-21swapper/111:59:001
3265325612,27sleep232700-21/usr/sbin/munin09:59:002
2516125610,31sleep025168-21users11:14:160
2516125610,31sleep025168-21users11:14:160
2376025617,20sleep20-21swapper/211:10:592
2320225613,27sleep123264-21/usr/sbin/munin11:09:131
2249925617,23sleep10-21swapper/109:24:181
1734525613,27sleep117392-21users10:49:181
1680325612,27sleep116809-21ntp_states07:24:131
1507125617,19sleep10-21swapper/107:19:081
1245725615,26sleep212463-21diskstats08:54:032
87825520,21sleep00-21swapper/011:43:440
458625515,24sleep04592-21diskstats11:54:020
3014525517,20sleep10-21swapper/108:08:471
2952825513,25sleep229521-21apt-get09:48:582
1550925517,24sleep015533-21fschecks_count12:29:030
735225416,17sleep20-21swapper/210:19:152
648225415,17sleep20-21swapper/210:18:472
335925421,17sleep10-21swapper/111:49:111
335925421,17sleep10-21swapper/111:49:111
3083925412,17sleep10-21swapper/109:53:461
1801525411,19sleep00-21swapper/010:53:580
1789225413,24sleep217885-21apt-get07:28:592
1715425413,21sleep00-21swapper/012:34:040
1715425413,21sleep00-21swapper/012:34:040
748325312,24sleep27552-21df08:39:012
450625312,24sleep14500-21apt-get11:53:591
3093225313,24sleep030955-21ntp_states11:34:110
230825314,23sleep12331-21munin-run08:23:451
1883325315,23sleep218890-21irqstats09:14:052
1446025312,23sleep014553-21mailstats08:59:140
2723525215,20sleep00-21swapper/007:58:570
2020725211,24sleep020279-21df09:19:000
179892526,36sleep20-21swapper/209:09:212
1403625213,23sleep114040-21diskstats08:59:021
2721125113,22sleep227230-21apt-get07:58:552
2295425114,22sleep00-21swapper/007:44:110
1725425116,17sleep10-21swapper/109:09:041
1395825111,16sleep00-21swapper/012:24:020
1001425113,22sleep010009-21apt-get10:28:580
608225013,19sleep00-21swapper/011:58:590
487525014,17sleep20-21swapper/211:54:082
2781425012,25sleep127896-21switchtime07:59:141
2355425012,22sleep123576-21irqstats09:29:061
3229424816,18sleep00-21swapper/008:14:110
2699824813,16sleep10-21swapper/107:58:461
2124724810,14sleep20-21swapper/211:04:012
2124724810,14sleep20-21swapper/211:04:012
1811424811,21sleep118163-21irqstats07:29:061
1211424810,13sleep10-21swapper/112:18:461
2419624711,21sleep024203-21diskstats07:49:040
2419624711,21sleep024203-21diskstats07:49:040
155092468,20sleep10-21swapper/109:03:591
2047524515,17sleep20-21swapper/209:19:062
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional