You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-15 - 05:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Sun Mar 15, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
7380243316,4312sleep027993-21kworker/0:120:04:290
20777230985,3081sleep119100-21kworker/1:122:32:301
26577280521,764sleep326622-21/usr/sbin/munin19:23:343
9926277617,742sleep09959-21/usr/sbin/munin20:13:340
9926277617,742sleep09959-21/usr/sbin/munin20:13:340
14496276915,738sleep214503-21/usr/sbin/munin20:28:262
24893264115,607sleep324924-21/usr/sbin/munin19:18:293
1821725068,484sleep118242-21modprobe00:08:071
162242409330,64sleep116240-21sshd20:33:291
648923408,295sleep16522-21grep20:03:251
30608233527,295sleep130705-21date19:38:101
4229228712,261sleep34240-21ntpq21:38:333
14992227091,99sleep115016-21apt-key23:58:101
22112226910,243sleep022152-21ps00:18:360
5038224063,155sleep15048-21modprobe21:43:071
26264222816,189sleep126325-21processes22:48:371
26264222816,189sleep126325-21processes22:48:371
30846222415,191sleep230877-21nfs_client23:03:322
1521219520,94sleep10-21swapper/123:13:361
1521219520,94sleep10-21swapper/123:13:361
1155821858,158sleep111691-21latency_hist22:03:101
30649218382,49sleep10-21swapper/121:18:361
2273121808,160sleep313441-21snmpd00:20:533
15565218010,68sleep10-21swapper/122:13:401
24515217819,142sleep224576-21/usr/sbin/munin22:43:292
18104217717,35sleep20-21swapper/222:23:262
21209216717,134sleep121243-21fschecks_count22:33:261
19332216743,91sleep119406-21latency_hist22:28:101
24303216672,77sleep10-21swapper/120:58:371
10956216310,141sleep111028-21mailstats23:43:361
28891216214,40sleep10-21swapper/122:58:181
12970216130,84sleep113062-21fschecks_time20:23:281
31491216059,63sleep10-21swapper/119:38:381
12371215451,64sleep10-21swapper/122:03:381
1634821519,131sleep116353-21run-parts23:59:071
25940214818,106sleep10-21swapper/121:03:381
8154214224,100sleep38193-21/usr/sbin/munin20:08:283
315921417,120sleep00-21swapper/021:34:140
22700213913,112sleep022769-21df22:38:220
22275213917,97sleep30-21swapper/320:53:263
7064213715,39sleep10-21swapper/121:48:281
30938213715,99sleep231029-21cut19:38:272
2024121370,24sleep120861-21cyclictest00:13:271
2145121360,5sleep30-21swapper/319:08:313
26382213414,106sleep026456-21fschecks_time00:33:260
2647621277,56sleep10-21swapper/119:23:321
25036212515,24sleep025066-21munin-run21:03:080
20917212510,89sleep10-21swapper/120:48:321
30450212431,81sleep130470-21dpkg23:03:191
30394212214,92sleep330440-21ntpq21:18:303
15729212211,48sleep115781-21sort20:33:091
22659212110,94sleep022165-21/usr/sbin/munin20:53:350
8132211911,90sleep08176-21systemd-run21:53:070
26098211911,75sleep00-21swapper/022:48:320
26098211911,75sleep00-21swapper/022:48:320
23405211615,85sleep323414-21needreboot19:13:323
4478211511,17sleep00-21swapper/023:23:290
2474821158,17sleep10-21swapper/100:28:221
9976211323,67sleep310121-21grep21:58:113
9976211323,67sleep310121-21grep21:58:113
25183211312,83sleep025202-21ntp_states00:28:330
17519211332,51sleep217538-21apt-get20:38:192
15968211114,81sleep316018-21cpuspeed20:33:233
28519211011,72sleep10-21swapper/121:13:271
15496211016,79sleep315502-21diskstats23:58:373
10249210917,74sleep210224-21apt-get21:58:312
29018210812,32sleep129036-21latency_hist19:33:091
2073921085,11sleep00-21swapper/019:07:400
23959210715,73sleep123977-21awk22:43:081
32266210527,66sleep132298-21irqstats23:08:321
32153210514,78sleep032193-21cut19:43:090
29285210515,76sleep229315-21awk22:58:312
892721029,80sleep19006-21mailstats21:53:371
892721029,80sleep19006-21mailstats21:53:371
5243210218,10sleep10-21swapper/123:24:431
26666210210,19sleep10-21swapper/100:33:311
24184210014,10sleep10-21swapper/100:24:391
13328210031,55sleep113417-21ls23:53:091
42632995,11sleep10-21swapper/119:57:441
3266229929,55sleep132718-21sed19:43:311
109332998,13sleep00-21swapper/023:43:350
1023029717,67sleep210334-21cat23:43:092
2246429413,64sleep022593-21latency_hist22:38:100
224122947,24sleep122481-21latency_hist19:13:101
935129315,62sleep19540-21dpkg20:13:101
195502928,17sleep10-21swapper/120:43:351
482029114,54sleep34317-21apt-get19:58:193
3156929120,15sleep10-21swapper/121:23:241
185229116,20sleep00-21swapper/019:48:300
185229116,20sleep00-21swapper/019:48:300
2559629031,37sleep30-21swapper/322:48:103
2889628924,48sleep228920-21proc_pri21:13:362
274528951,24sleep12775-21munin-run19:53:081
274528951,24sleep12775-21munin-run19:53:081
2499528916,18sleep10-21swapper/119:18:311
2737128826,29sleep30-21swapper/321:08:363
2072328821,46sleep220787-21fschecks_time20:48:282
1865428810,62sleep10-21swapper/100:08:281
792328736,31sleep10-21swapper/123:33:391
579928719,14sleep10-21swapper/123:28:221
2446528721,49sleep10-21swapper/122:43:281
2396728716,53sleep30-21swapper/300:23:403
2935128632,31sleep30-21swapper/319:33:273
230412869,38sleep10-21swapper/119:13:221
179932869,15sleep10-21swapper/120:38:341
818228525,33sleep20-21swapper/221:53:092
3202828513,59sleep332047-21grep23:08:253
2245628513,57sleep222477-21sed20:53:292
996028421,47sleep10-21swapper/120:13:351
996028421,47sleep10-21swapper/120:13:351
486828410,34sleep10-21swapper/119:58:331
1503628414,51sleep315158-21timerandwakeup20:28:393
1383228415,55sleep013849-21/usr/sbin/munin23:53:290
891628319,12sleep10-21swapper/123:38:191
1851628312,51sleep018595-21sendmail_mailtr22:23:360
841528216,46sleep28435-21/usr/sbin/munin20:08:352
484828222,41sleep24839-21apt-get19:58:222
229572828,56sleep10-21swapper/100:23:091
2271928218,14sleep10-21swapper/120:53:361
169212828,40sleep116927-21ntp_states22:18:321
10912829,15sleep10-21swapper/121:28:331
868528120,32sleep30-21swapper/323:38:093
304752818,14sleep00-21swapper/023:03:220
238428120,42sleep32396-21modprobe23:18:063
238428120,42sleep32396-21modprobe23:18:063
195728121,30sleep00-21swapper/021:33:090
115332816,12sleep10-21swapper/120:18:271
112912819,62sleep3245-21systemd-udevd22:00:253
2794228018,11sleep10-21swapper/121:10:321
2190928018,41sleep221923-21kernelversion00:18:302
2044428016,23sleep00-21swapper/020:48:100
654527818,11sleep10-21swapper/121:47:351
334527830,28sleep120861-21cyclictest23:18:371
334527830,28sleep120861-21cyclictest23:18:371
2124427814,47sleep221331-21fschecks_time22:33:272
2005627814,48sleep020101-21cpu00:13:230
1537327722,36sleep215392-21apt-get23:58:192
1482527719,38sleep314893-40mandb23:58:083
1331027718,11sleep10-21swapper/122:08:151
766527615,45sleep37699-21awk20:08:073
430927613,44sleep24384-21/usr/sbin/munin21:38:362
1464627624,30sleep30-21swapper/322:13:093
982527516,24sleep20-21swapper/220:13:272
982527516,24sleep20-21swapper/220:13:272
627527513,46sleep36334-21/usr/sbin/munin23:28:363
443427521,34sleep24552-21apt-key19:58:092
2856627518,10sleep10-21swapper/122:57:411
2856627518,10sleep10-21swapper/122:57:411
3133327428,24sleep30-21swapper/319:38:363
2918427417,23sleep00-21swapper/022:58:290
2084727417,40sleep221011-21latency_hist19:08:102
2975227322,24sleep00-21swapper/019:33:360
272012737,11sleep00-21swapper/019:25:120
2459927313,38sleep024603-21find19:18:180
1871727311,29sleep20-21swapper/200:08:292
2762727217,40sleep227663-21latency_hist00:38:082
1885127227,24sleep00-21swapper/020:43:090
850127115,22sleep00-21swapper/020:08:370
499327125,23sleep20-21swapper/223:23:392
774127016,23sleep00-21swapper/023:33:340
3247927015,38sleep232567-21tune2fs19:43:282
2792627012,34sleep20-21swapper/219:28:302
2060827015,26sleep20-21swapper/200:13:382
1791427024,26sleep30-21swapper/320:38:323
2664126919,25sleep00-21swapper/021:08:090
2272026912,19sleep00-21swapper/019:13:130
2039526914,26sleep30-21swapper/320:48:093
316426814,23sleep20-21swapper/223:18:342
316426814,23sleep20-21swapper/223:18:342
2337826814,29sleep30-21swapper/322:38:373
1990926822,29sleep319954-21awk19:03:313
130692686,14sleep00-21swapper/023:50:300
2664226720,21sleep20-21swapper/219:23:372
2087826722,21sleep00-21swapper/022:33:090
1413226714,39sleep014167-21awk20:28:070
346326614,35sleep23485-21munin-run21:38:072
3101026614,26sleep30-21swapper/323:03:373
2347026615,29sleep20-21swapper/200:23:312
220426633,16sleep10-21swapper/121:33:221
2082626626,23sleep10-21swapper/119:08:091
2068626616,35sleep30-21swapper/300:13:393
1327526617,31sleep313309-21awk23:53:073
455626517,26sleep34724-21latency_hist19:58:103
2437526514,34sleep224465-21sed20:58:382
2430826514,34sleep224372-21sendmail-msp19:18:082
1827026516,25sleep00-21swapper/000:08:100
304432646,13sleep00-21swapper/021:18:310
2713226415,19sleep00-21swapper/021:08:290
256952647,13sleep30-21swapper/321:03:323
1310326420,22sleep00-21swapper/022:08:100
1110226415,32sleep311222-21ls23:43:383
974226317,21sleep30-21swapper/320:13:243
974226317,21sleep30-21swapper/320:13:243
6002636,48sleep00-21swapper/023:10:120
1263526318,16sleep30-21swapper/320:23:103
1018026314,24sleep00-21swapper/023:43:070
67382627,10sleep00-21swapper/023:29:020
429226223,24sleep00-21swapper/019:58:070
400226214,27sleep00-21swapper/023:23:080
400226214,27sleep00-21swapper/023:23:080
2698126216,19sleep20-21swapper/221:08:272
217562625,16sleep00-21swapper/020:51:130
1802126212,25sleep30-21swapper/322:23:233
1154826212,36sleep011557-21munin-run22:03:090
663726118,20sleep30-21swapper/321:48:103
205626119,26sleep21461-21/usr/sbin/munin19:48:362
205626119,26sleep21461-21/usr/sbin/munin19:48:362
1644626117,24sleep20-21swapper/222:18:172
1368726118,24sleep20-21swapper/223:53:272
1210126114,31sleep212133-21tune2fs23:48:262
743326012,32sleep27465-21tune2fs23:33:282
294302608,40sleep00-21swapper/021:14:470
2769426011,22sleep00-21swapper/019:28:190
266826018,21sleep30-21swapper/321:33:363
2495426016,24sleep30-21swapper/300:28:283
240526013,30sleep01-21systemd23:18:070
240526013,30sleep01-21systemd23:18:070
2358026016,26sleep023626-21awk00:23:330
441925913,33sleep34225-21/usr/sbin/munin23:23:283
3224925910,19sleep00-21swapper/021:23:400
2999225914,28sleep230011-21apt-get21:18:172
2855825910,10sleep30-21swapper/322:57:353
2855825910,10sleep30-21swapper/322:57:353
2770425921,23sleep00-21swapper/022:53:280
2770425921,23sleep00-21swapper/022:53:280
2137025919,23sleep20-21swapper/219:08:312
1659425913,30sleep316680-21fw_packets22:18:263
120025915,27sleep21207-21sshd23:13:292
120025915,27sleep21207-21sshd23:13:292
1145125914,21sleep00-21swapper/020:18:260
74225821,18sleep20-21swapper/221:28:262
1744225812,25sleep20-21swapper/200:03:362
1596525814,28sleep015961-21apt-get20:33:200
1578225815,18sleep20-21swapper/220:33:102
1179525813,27sleep211787-21apt-get22:03:232
1102925813,17sleep20-21swapper/223:43:372
3183525717,19sleep30-21swapper/321:23:313
2766425715,21sleep30-21swapper/300:38:093
2532525716,21sleep20-21swapper/221:03:222
2483425720,19sleep20-21swapper/219:18:282
2422925716,25sleep30-21swapper/320:58:353
1334125716,24sleep013335-21apt-get22:08:220
112225714,31sleep31146-21proc_pri21:28:353
2865125613,26sleep30-21swapper/321:13:293
2771825613,25sleep327714-21apt-get19:28:223
2147225616,19sleep00-21swapper/019:08:330
1976625617,17sleep00-21swapper/000:13:070
1439625614,24sleep014415-21apt-get20:28:190
937725516,17sleep30-21swapper/323:38:343
541325516,20sleep00-21swapper/021:43:260
318825514,25sleep33257-21mailstats23:18:353
318825514,25sleep33257-21mailstats23:18:353
2953725519,18sleep20-21swapper/219:33:302
2267125513,18sleep20-21swapper/222:38:162
647625417,18sleep20-21swapper/223:28:392
386625418,20sleep00-21swapper/021:38:260
2587525413,25sleep325880-21diskstats22:48:273
2587525413,25sleep325880-21diskstats22:48:273
148525412,25sleep31480-21apt-get19:48:213
148525412,25sleep31480-21apt-get19:48:213
1341025412,25sleep213419-21df_inode22:08:242
2931625311,24sleep329358-21awk22:58:323
165162536,34sleep00-21swapper/000:01:120
1488625313,23sleep314883-21apt-get22:13:223
652425215,22sleep26610-21perl20:03:262
3177625215,20sleep20-21swapper/221:23:302
2083025212,21sleep320872-21systemd-run22:33:083
1956925212,22sleep219636-21df22:28:232
1877525216,17sleep30-21swapper/300:08:303
1695425212,23sleep316950-21apt-get00:03:213
1287225212,23sleep312868-21apt-get20:23:223
1262525213,23sleep212634-21munin-run20:23:092
839725112,24sleep08415-21apt-get21:53:200
839725112,24sleep08415-21apt-get21:53:200
529725113,23sleep05321-21ntp_states19:58:440
2604025114,22sleep226055-21munin-run00:33:082
211932519,27sleep321200-21df_inode22:33:243
1488825116,21sleep00-21swapper/022:13:230
1259625112,22sleep312695-21users22:03:403
46425011,23sleep0538-21/usr/sbin/munin19:43:380
3239525012,24sleep332465-21df19:43:243
2195325012,22sleep322094-21sed20:53:103
2791324912,19sleep20-21swapper/222:53:342
2791324912,19sleep20-21swapper/222:53:342
3211124811,22sleep232118-21sshd23:08:282
199112488,15sleep20-21swapper/220:43:402
1500824811,24sleep215071-21fschecks_time22:13:292
1418224811,21sleep20-21swapper/220:28:092
985524713,21sleep29975-21cmp21:58:102
985524713,21sleep29975-21cmp21:58:102
75182479,15sleep30-21swapper/321:48:383
2546624710,28sleep00-21swapper/022:47:350
1647124511,19sleep016467-21apt-get22:18:190
2619924311,19sleep226248-21ntp_offset22:48:342
2619924311,19sleep226248-21ntp_offset22:48:342
53242418,16sleep20-21swapper/221:43:202
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional