You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-13 - 11:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Mon Jul 13, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2256027607,742sleep316613-1kworker/3:0H00:04:093
2256027607,742sleep316613-1kworker/3:0H00:04:093
30203268934,637sleep230197-21/usr/sbin/munin00:28:512
7578257611,550sleep27541-21latency_hist21:33:522
13279241815,381sleep213428-21unixbench_multi20:09:242
20430240920,367sleep020488-40mandb23:58:490
28551240312,254sleep214218-21kworker/u8:300:23:502
28551240312,254sleep214218-21kworker/u8:300:23:502
16924234296,182sleep217050-21latency_hist22:03:512
308222274138,23sleep20-21swapper/219:23:532
32353225024,203sleep232444-21fschecks_time21:09:102
1517223830,191sleep21571-21idleruntime-cro22:58:492
4890223026,186sleep21-21systemd23:08:502
27295220620,161sleep227375-21munin-node19:13:522
28112220513,175sleep228243-21head20:54:412
26499220223,158sleep226473-21/usr/sbin/munin22:34:072
25715219014,155sleep225814-21latency_hist19:08:522
20330219014,37sleep220233-21/usr/sbin/munin22:14:052
26271218914,158sleep226407-21unixbench_singl20:49:322
28199218522,20sleep20-21swapper/219:14:132
28368218417,151sleep028370-21sshd00:22:070
28368218417,151sleep028370-21sshd00:22:070
5183218212,33sleep25234-21sort23:09:062
22596218111,152sleep222597-21cpu00:04:372
22596218111,152sleep222597-21cpu00:04:372
17761218111,158sleep2364-21snmpd22:04:202
11601218113,151sleep011670-21wc20:04:210
5894217515,140sleep20-21swapper/221:28:522
3023021757,156sleep230350-21sed22:44:202
9172216916,75sleep29238-21modprobe19:58:502
344321677,14sleep10-21swapper/123:03:531
31097216718,135sleep131133-21ntpq21:04:171
15507216313,135sleep215657-21cat20:18:522
19674216026,26sleep20-21swapper/220:29:222
18930216027,113sleep20-21swapper/223:53:512
16542215817,126sleep216581-21wc23:44:182
1504021587,127sleep20-21swapper/223:39:222
3058921548,17sleep20-21swapper/200:29:122
9027215218,121sleep39028-21sshd19:56:573
25064215298,34sleep225073-21munin-run20:48:512
1982121505,28sleep20-21swapper/222:11:552
1752521506,29sleep20-21swapper/220:24:092
1812214711,118sleep20-21swapper/222:59:012
224862143105,18sleep20-21swapper/222:19:212
32315214215,112sleep232344-21munin-run19:28:502
3143521398,37sleep2364-21snmpd22:49:122
14577213914,104sleep314009-21/usr/sbin/munin21:54:203
9391213812,51sleep20-21swapper/221:39:052
1676521386,29sleep20-21swapper/220:20:582
3096721378,116sleep231001-21wc21:04:142
2913421379,20sleep20-21swapper/220:59:012
13225213614,22sleep213254-21cut23:34:132
1201021357,116sleep2364-21snmpd20:05:042
2533221345,29sleep20-21swapper/200:13:292
2124621339,18sleep20-21swapper/223:59:242
22757213211,99sleep20-21swapper/220:39:232
19398212912,14sleep20-21swapper/223:54:112
10753212917,77sleep20-21swapper/223:24:242
7237212810,100sleep20-21swapper/223:14:232
7237212810,100sleep20-21swapper/223:14:232
679221287,15sleep20-21swapper/219:49:172
3260421289,14sleep20-21swapper/219:29:022
3260421289,14sleep20-21swapper/219:29:022
1093121258,105sleep210932-21apt-get21:44:022
29192212429,76sleep329230-21modprobe19:18:503
2814921228,35sleep20-21swapper/222:39:072
1491821178,43sleep20-21swapper/221:54:232
1296621178,35sleep20-21swapper/221:49:182
9210211614,30sleep20-21swapper/223:19:242
9210211614,30sleep20-21swapper/223:19:242
430721167,12sleep20-21swapper/219:40:572
27166211523,75sleep026805-21apt-get20:54:020
263121149,11sleep20-21swapper/219:34:392
25644211336,53sleep025677-21grep19:08:500
2624121127,90sleep20-21swapper/222:33:502
132121126,12sleep20-21swapper/221:13:582
881821119,29sleep20-21swapper/221:35:342
68721115,42sleep20-21swapper/200:35:462
558521115,29sleep20-21swapper/219:44:212
2450221118,30sleep20-21swapper/222:26:562
1241521115,30sleep20-21swapper/223:30:382
318221106,67sleep20-21swapper/221:19:112
2670821105,11sleep20-21swapper/200:16:032
2670821105,11sleep20-21swapper/200:16:032
2181821086,10sleep20-21swapper/220:38:062
1041121076,30sleep20-21swapper/219:59:312
24608210618,16sleep10-21swapper/100:09:201
15837210617,75sleep315954-21latency_hist23:43:513
1094221066,78sleep00-21swapper/021:44:080
26246210416,68sleep125951-21/usr/sbin/munin19:09:151
1866021049,82sleep018665-21sshd23:50:580
17488210115,68sleep117489-21munin-node20:24:001
31652210011,72sleep131720-21systemctl22:49:201
25737210013,18sleep10-21swapper/100:14:081
25737210013,18sleep10-21swapper/100:14:081
10832210014,70sleep010975-21latency_hist20:03:520
2122429915,17sleep10-21swapper/120:34:211
1499029912,72sleep014997-21sh23:39:200
1762229821,57sleep317138-21/usr/sbin/munin22:04:173
1871629713,66sleep018717-21cpu22:09:060
1271629719,60sleep112778-21latency_hist23:33:521
710129516,54sleep07237-21unixbench_singl19:49:230
303529416,18sleep30-21swapper/319:38:523
1477229220,51sleep114916-21unixbench_singl21:54:221
2254629121,53sleep00-21swapper/000:04:020
2254629121,53sleep00-21swapper/000:04:020
133029110,69sleep3364-21snmpd22:56:243
2475029028,30sleep30-21swapper/319:04:133
2228229016,56sleep322342-21fschecks_time20:39:123
1616628912,64sleep116235-21fschecks_time23:44:091
398028818,49sleep03250-21/usr/sbin/munin19:39:220
323828820,43sleep03440-21latency_hist23:03:520
2085928815,57sleep320945-21fschecks_time23:59:133
2549528720,47sleep325010-21apt-get20:49:013
1706528724,30sleep10-21swapper/120:23:521
167428711,60sleep01693-21apt-get19:34:010
545128615,48sleep15158-21/usr/sbin/munin23:09:131
846228421,42sleep18523-21/usr/sbin/munin23:19:101
846228421,42sleep18523-21/usr/sbin/munin23:19:101
3169428411,59sleep131695-21sshd00:32:011
915228311,17sleep10-21swapper/121:38:521
2703028317,49sleep327101-21/usr/sbin/munin22:34:203
860728218,25sleep00-21swapper/023:19:130
860728218,25sleep00-21swapper/023:19:130
2731628117,48sleep327327-21diskstats00:19:073
2731628117,48sleep327327-21diskstats00:19:073
2374328116,50sleep123758-21grep20:44:061
1919028118,43sleep119253-21sendmail_mailqu22:09:191
305412806,15sleep30-21swapper/300:28:563
216628014,31sleep10-21swapper/119:34:181
3004727932,27sleep030104-21apt-config00:28:490
2724427912,15sleep10-21swapper/100:19:051
2724427912,15sleep10-21swapper/100:19:051
2043927927,28sleep10-21swapper/122:14:081
997427720,28sleep10-21swapper/121:39:201
1809227712,46sleep018114-21/usr/sbin/munin20:25:400
41932767,17sleep00-21swapper/021:23:490
2734827620,36sleep327260-21/usr/sbin/munin20:54:213
2223827616,29sleep00-21swapper/022:19:150
1965127622,33sleep10-21swapper/123:54:181
494027518,37sleep34907-21latency_hist23:08:513
3422758,54sleep00-21swapper/000:34:230
2939027520,24sleep30-21swapper/322:43:533
2726627513,30sleep10-21swapper/120:54:031
2568627525,32sleep125714-21munin-run19:08:511
2400127517,41sleep324127-21cat22:24:193
62427410,15sleep00-21swapper/022:54:150
500427416,26sleep00-21swapper/023:08:520
1628027425,24sleep30-21swapper/321:59:203
600627315,26sleep30-21swapper/321:28:533
3252927324,22sleep10-21swapper/121:09:131
607827223,26sleep30-21swapper/319:48:503
1975427216,30sleep30-21swapper/320:29:233
1698627216,33sleep317009-21munin-run20:23:503
1343027213,39sleep313500-21/usr/sbin/munin20:09:253
648727121,28sleep30-21swapper/323:13:523
2685827119,19sleep10-21swapper/120:53:511
2052127117,29sleep30-21swapper/323:58:523
2003827111,46sleep0364-21snmpd20:30:090
1748327112,44sleep317000-21apt-get20:23:583
1230427115,28sleep10-21swapper/120:08:501
1161827119,24sleep30-21swapper/321:44:233
2950227014,30sleep10-21swapper/100:24:221
2345827016,25sleep10-21swapper/122:24:051
230327021,27sleep30-21swapper/322:59:183
3106126913,32sleep00-21swapper/019:24:040
1528726913,40sleep015289-21cron21:58:490
807826822,22sleep30-21swapper/323:18:523
807826822,22sleep30-21swapper/323:18:523
710126811,43sleep17129-21nfs4_client23:14:181
710126811,43sleep17129-21nfs4_client23:14:181
57926813,38sleep3598-21ntp_states19:29:163
57926813,38sleep3598-21ntp_states19:29:163
2881626815,28sleep10-21swapper/122:39:231
2412926811,13sleep00-21swapper/022:24:200
781726720,27sleep30-21swapper/321:34:063
1248526716,21sleep10-21swapper/121:49:051
272526612,37sleep02875-21latency_hist21:18:530
207532669,15sleep10-21swapper/123:59:021
1578726611,36sleep10-21swapper/123:43:491
882626525,22sleep00-21swapper/021:35:400
664326517,29sleep30-21swapper/319:49:143
3230726515,30sleep30-21swapper/321:09:073
2979226518,27sleep00-21swapper/022:44:100
1442726516,27sleep10-21swapper/123:38:521
956926421,25sleep39584-21apt23:23:493
956926421,25sleep39584-21apt23:23:493
490426415,35sleep14931-21fschecks_count19:44:071
3242426420,28sleep31-21systemd22:53:493
2930326418,17sleep00-21swapper/022:43:290
1023326412,20sleep10-21swapper/123:24:151
3250626314,32sleep132545-21/usr/sbin/munin00:34:151
3069926316,32sleep330734-21grep21:04:063
302382636,44sleep00-21swapper/021:01:370
2516926318,19sleep30-21swapper/320:48:523
2052526314,32sleep320579-21sed22:14:103
2025826313,20sleep00-21swapper/022:14:030
1656426318,24sleep00-21swapper/020:19:230
132726315,29sleep11346-21apt-get21:14:011
1285826316,30sleep012870-21/usr/sbin/munin20:09:130
624026219,21sleep30-21swapper/321:29:013
2625526217,20sleep30-21swapper/322:33:513
2619326216,22sleep00-21swapper/000:14:200
2619326216,22sleep00-21swapper/000:14:200
200022627,11sleep10-21swapper/120:29:411
1847226218,24sleep10-21swapper/122:08:511
179332628,39sleep00-21swapper/022:04:220
1204226221,23sleep30-21swapper/323:29:223
674926119,21sleep10-21swapper/119:49:161
3260726115,24sleep032601-21apt-get19:29:040
3260726115,24sleep032601-21apt-get19:29:040
2969026116,24sleep30-21swapper/321:00:193
2615926112,20sleep30-21swapper/319:09:133
2540926117,22sleep00-21swapper/000:13:510
2254826115,27sleep122542-21apt-get00:04:031
2254826115,27sleep122542-21apt-get00:04:031
2216926119,25sleep10-21swapper/120:39:011
1811226119,22sleep10-21swapper/123:49:181
1481426113,32sleep014931-21cat20:14:220
77812605,11sleep00-21swapper/023:16:030
77812605,11sleep00-21swapper/023:16:030
3087426014,30sleep330894-21munin-run22:48:493
3016226013,29sleep130229-21swap22:44:191
1155526018,22sleep00-21swapper/023:29:110
975825916,19sleep30-21swapper/321:39:143
360325917,22sleep00-21swapper/023:04:000
308512599,19sleep00-21swapper/021:04:120
270542595,12sleep00-21swapper/019:11:320
269725916,22sleep30-21swapper/321:18:523
2695825918,20sleep10-21swapper/122:34:181
2533125919,22sleep00-21swapper/020:48:530
2137725913,26sleep00-21swapper/023:59:260
1789925912,21sleep30-21swapper/323:49:123
1295825915,27sleep312953-21apt-get23:34:063
1091225911,28sleep10-21swapper/121:44:011
862925812,31sleep08769-21unixbench_singl19:54:240
140925816,25sleep11446-21modprobe19:33:501
140925816,25sleep11446-21modprobe19:33:501
2749725710,31sleep027515-21ntp_states19:13:540
191672579,20sleep30-21swapper/322:09:173
183725716,25sleep11831-21apt-get22:59:041
1711525713,30sleep017117-21sshd23:45:580
1309825615,27sleep013222-21sed21:49:200
1069825616,18sleep00-21swapper/021:43:510
3192425513,17sleep00-21swapper/022:49:230
3049025517,19sleep10-21swapper/121:03:521
2879025515,23sleep328785-21apt-get00:24:043
2864025515,26sleep028690-21munin-run20:58:500
261342556,11sleep00-21swapper/022:32:560
2418925520,19sleep30-21swapper/300:09:103
169822556,38sleep30-21swapper/323:44:233
1429125515,17sleep00-21swapper/023:38:510
3239525413,16sleep10-21swapper/119:28:531
2989025414,20sleep00-21swapper/019:19:150
2792825414,24sleep127923-21apt-get19:14:041
2620425413,23sleep10-21swapper/122:33:491
2146625414,16sleep30-21swapper/320:34:233
2115325414,19sleep00-21swapper/020:34:200
1413425414,23sleep114140-21apt-get20:14:031
485225316,21sleep14891-21ntpq21:24:151
320525313,24sleep30-21swapper/321:19:123
258412536,11sleep30-21swapper/300:14:113
258412536,11sleep30-21swapper/300:14:113
2183625314,23sleep321839-21apt-get22:19:023
1422425313,16sleep30-21swapper/320:14:093
293325217,18sleep10-21swapper/121:19:021
2192225212,16sleep10-21swapper/122:19:081
1611025214,18sleep10-21swapper/121:59:181
445925111,24sleep04478-21apt-get21:24:010
297602518,17sleep10-21swapper/119:19:121
1259725113,19sleep10-21swapper/120:09:061
828925013,21sleep18334-21/usr/sbin/munin19:54:171
623525013,22sleep05757-21apt-get21:28:580
3235425015,16sleep30-21swapper/319:28:513
3233525012,21sleep032342-21diskstats21:09:080
164062508,15sleep00-21swapper/021:59:210
1302925012,21sleep313096-21awk21:49:193
1042925017,16sleep30-21swapper/323:24:213
985124912,22sleep09858-21apt-get23:24:020
3212824912,21sleep332119-21apt-get00:34:053
123552499,25sleep312376-21latency_hist20:08:523
1080024918,15sleep30-21swapper/320:03:513
95602489,17sleep10-21swapper/119:59:051
626924812,20sleep16260-21apt-get21:29:031
481824811,21sleep04813-21apt-get19:44:020
3059424812,19sleep00-21swapper/000:29:140
375024713,16sleep10-21swapper/119:39:191
221992468,14sleep00-21swapper/020:39:090
269502459,16sleep00-21swapper/022:34:170
7662439,14sleep10-21swapper/122:54:201
37272429,13sleep30-21swapper/319:39:173
294882427,17sleep329507-21apt-get19:19:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional