You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-20 - 04:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Mon Apr 20, 2026 00:43:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1128027048,637sleep211329-21unin-run23:31:012
11470266416,632sleep111506-21/usr/sbin/munin20:06:221
11470266416,632sleep111506-21/usr/sbin/munin20:06:221
3110423837,319sleep231147-21unin-run19:26:022
2365123229,300sleep223713-21perl22:26:262
2365123229,300sleep223713-21perl22:26:262
31996231115,206sleep20-21swapper/219:26:342
16508229512,269sleep216523-21kernelversion23:46:232
16508229512,269sleep216523-21kernelversion23:46:232
18680228515,254sleep218717-21fschecks_count22:11:202
31021226976,173sleep231041-21grep22:51:002
31021226976,173sleep231041-21grep22:51:002
31532226063,180sleep231556-21munin-run00:36:012
27412224615,217sleep227193-21/usr/sbin/munin20:56:242
27412224615,217sleep227193-21/usr/sbin/munin20:56:242
29424224561,160sleep229548-21timerwakeupswit21:01:332
29134224454,176sleep229145-21awk22:41:522
28944223686,95sleep229074-21timerwakeupswit19:16:582
16049223264,22sleep20-21swapper/223:46:032
11185223213,128sleep20-21swapper/221:46:262
11185223213,128sleep20-21swapper/221:46:262
25800223069,63sleep20-21swapper/220:51:362
25800223069,63sleep20-21swapper/220:51:362
32606221423,130sleep232670-21mailstats21:11:302
16328221331,168sleep216042-21/usr/sbin/munin20:21:242
16328221331,168sleep216042-21/usr/sbin/munin20:21:242
10402220815,177sleep310429-21awk23:26:343
24521220212,169sleep224588-21wc00:11:302
4111219673,38sleep24084-21latency_hist21:26:032
1083219610,171sleep2446-21/usr/sbin/munin19:31:312
3687218617,152sleep23777-21cut19:41:202
4016218420,92sleep20-21swapper/223:06:392
4016218420,92sleep20-21swapper/223:06:392
16758218422,142sleep316778-21awk22:06:003
16758218422,142sleep316778-21awk22:06:003
2646921809,18sleep20-21swapper/219:11:212
2646921809,18sleep20-21swapper/219:11:212
21025217915,86sleep20-21swapper/200:01:212
1758421779,149sleep117654-21sensors_temp22:06:311
13215217319,79sleep20-21swapper/223:36:182
30056217214,101sleep10-21swapper/100:31:031
16811216915,138sleep216951-21latency_hist22:06:042
16811216915,138sleep216951-21latency_hist22:06:042
11143216851,102sleep211197-21latency_hist20:06:022
11143216851,102sleep211197-21latency_hist20:06:022
8442216663,22sleep20-21swapper/223:21:122
8442216663,22sleep20-21swapper/223:21:122
26138216520,89sleep226229-21cut00:16:312
15945216412,81sleep20-21swapper/223:45:552
1256321609,139sleep013441-21snmpd23:32:360
9358215646,94sleep29397-21fschecks_count21:41:252
9358215646,94sleep29397-21fschecks_count21:41:252
7940215053,82sleep27944-21modprobe23:21:002
7940215053,82sleep27944-21modprobe23:21:002
32589214951,27sleep2291-21systemd-logind22:56:012
1862214745,19sleep20-21swapper/221:16:352
141492143103,19sleep20-21swapper/221:56:272
991621375,109sleep00-21swapper/020:01:200
991621375,109sleep00-21swapper/020:01:200
618021309,70sleep26130-21apt-get21:31:482
22347213013,103sleep322355-21needreboot22:21:353
2204021307,72sleep222027-21latency_hist20:41:022
11571213053,65sleep211583-21grep23:31:172
6807212816,98sleep36810-21fschecks_count23:16:253
6807212816,98sleep36810-21fschecks_count23:16:253
5214212720,78sleep15242-21fschecks_time23:11:221
17556212517,88sleep117689-21latency_hist23:51:021
17556212517,88sleep117689-21latency_hist23:51:021
30252212415,94sleep130313-21grep22:46:401
30252212415,94sleep130313-21grep22:46:401
31361212133,43sleep10-21swapper/119:26:131
30706212110,85sleep00-21swapper/021:06:270
2520212129,69sleep22560-21ps19:36:322
24216212113,23sleep20-21swapper/220:46:272
2250521219,99sleep20-21swapper/200:06:172
15290212014,39sleep20-21swapper/220:16:362
25206211811,91sleep225217-21fschecks_count22:31:232
25206211811,91sleep225217-21fschecks_count22:31:232
17886211645,53sleep20-21swapper/220:26:252
9079211314,82sleep19148-21switchtime23:21:561
9079211314,82sleep19148-21switchtime23:21:561
28007211376,18sleep20-21swapper/222:41:032
28007211376,18sleep20-21swapper/222:41:032
1502621138,21sleep00-21swapper/023:41:250
1502621138,21sleep00-21swapper/023:41:250
5559211237,55sleep20-21swapper/223:11:312
22461211219,35sleep122480-21apt-get00:06:121
32210211119,69sleep132292-21fschecks_time21:11:211
11960211015,25sleep00-21swapper/020:06:330
11960211015,25sleep00-21swapper/020:06:330
2522521096,90sleep00-21swapper/000:14:550
20587210719,73sleep120636-21ntp_states22:16:281
23233210621,64sleep123268-21grep22:26:011
32694210512,78sleep3334-21sed19:31:043
28557210519,25sleep20-21swapper/200:26:032
24222210523,60sleep324391-21grep22:26:443
24222210523,60sleep324391-21grep22:26:443
21480210513,62sleep121633-21unixbench_singl20:36:331
18341210517,73sleep118411-21wc23:51:311
6325210416,74sleep125884-21systemd-journal19:49:551
6325210416,74sleep125884-21systemd-journal19:49:551
28377210418,64sleep328381-21grep22:41:373
22919210424,41sleep00-21swapper/019:01:140
22919210424,41sleep00-21swapper/019:01:140
472921018,16sleep20-21swapper/221:26:282
8364210012,64sleep20-21swapper/219:56:222
8364210012,64sleep20-21swapper/219:56:222
31833210023,54sleep125884-21systemd-journal21:11:001
15171210017,67sleep115231-21sendmail_mailqu23:41:301
15171210017,67sleep115231-21sendmail_mailqu23:41:301
873129911,73sleep38732-21sshd21:37:553
677729930,58sleep26783-21df_inode19:51:212
677729930,58sleep26783-21df_inode19:51:212
226629921,26sleep30-21swapper/323:01:273
63332986,47sleep222-21ksoftirqd/219:50:002
63332986,47sleep222-21ksoftirqd/219:50:002
1916929812,71sleep119252-21cat22:11:321
247842979,76sleep224852-21ls19:06:282
335329612,63sleep00-21swapper/019:41:020
115829415,19sleep20-21swapper/222:56:462
2927429217,65sleep00-21swapper/019:17:320
2244029222,15sleep20-21swapper/222:21:382
2047329219,14sleep20-21swapper/220:36:022
3114829117,18sleep30-21swapper/319:26:033
2144229113,16sleep20-21swapper/222:21:012
1184629121,51sleep20-21swapper/220:06:312
1184629121,51sleep20-21swapper/220:06:312
3116529049,20sleep20-21swapper/221:06:382
2806929023,39sleep328140-21latency_hist22:41:043
2806929023,39sleep328140-21latency_hist22:41:043
2065129012,64sleep020692-21cut22:16:310
169929026,40sleep11725-21modprobe23:01:011
1320029010,66sleep013225-21kernelversion20:11:250
635628911,64sleep00-21swapper/019:50:180
635628911,64sleep00-21swapper/019:50:180
2708328916,56sleep127160-21/usr/sbin/munin19:11:441
2708328916,56sleep127160-21/usr/sbin/munin19:11:441
271832889,15sleep20-21swapper/200:21:192
131192888,17sleep20-21swapper/221:51:342
2237428620,46sleep122461-21fschecks_time20:41:221
19962867,14sleep20-21swapper/223:01:182
1731828612,36sleep20-21swapper/222:06:242
980728528,31sleep10-21swapper/120:01:131
980728528,31sleep10-21swapper/120:01:131
1407128520,42sleep114147-21fschecks_time21:56:261
129412858,14sleep20-21swapper/220:11:192
818928414,49sleep18297-21/usr/sbin/munin21:36:361
679028420,37sleep06854-21sort19:51:230
679028420,37sleep06854-21sort19:51:230
2878128417,31sleep30-21swapper/319:16:553
2631328421,41sleep026452-21grep00:16:330
235922848,17sleep223626-21sed20:46:032
289022830,6sleep00-21swapper/022:41:490
2589828339,28sleep30-21swapper/300:16:243
2219728318,27sleep10-21swapper/100:06:001
2570228213,51sleep325778-21threads22:31:443
2570228213,51sleep325778-21threads22:31:443
3038728117,47sleep330493-21ls22:46:423
3038728117,47sleep330493-21ls22:46:423
194602819,13sleep20-21swapper/223:56:202
177812819,49sleep00-21swapper/020:26:230
199482800,6sleep30-21swapper/320:31:343
182482807,16sleep20-21swapper/223:51:282
3050027914,45sleep030643-21sed22:46:430
3050027914,45sleep030643-21sed22:46:430
235252798,54sleep00-21swapper/022:26:160
235252798,54sleep00-21swapper/022:26:160
954427810,16sleep10-21swapper/120:01:011
954427810,16sleep10-21swapper/120:01:011
3257027817,43sleep132588-21modprobe22:56:001
1362327828,30sleep00-21swapper/021:56:000
2996127719,37sleep130046-21fschecks_time19:21:251
2300627625,28sleep20-21swapper/219:01:202
2300627625,28sleep20-21swapper/219:01:202
2131927614,44sleep021369-21awk00:01:270
1946727611,47sleep30-21swapper/323:56:213
812327514,45sleep18257-21latency_hist23:21:031
812327514,45sleep18257-21latency_hist23:21:031
2599927517,25sleep10-21swapper/100:16:261
1090327520,28sleep10-21swapper/121:46:211
1090327520,28sleep10-21swapper/121:46:211
47272748,10sleep10-21swapper/119:44:181
2721827418,31sleep00-21swapper/020:56:180
2721827418,31sleep00-21swapper/020:56:180
2510327420,33sleep025116-21apt-get20:51:010
124322740,6sleep30-21swapper/323:31:343
408427316,42sleep34147-21mailstats19:41:293
2497527316,38sleep324990-21/usr/sbin/munin19:06:333
2339827329,27sleep10-21swapper/119:01:391
2339827329,27sleep10-21swapper/119:01:391
1788927321,26sleep30-21swapper/323:51:213
2870127214,42sleep128746-21grep19:16:361
2149927221,23sleep30-21swapper/322:21:023
205312726,10sleep10-21swapper/123:59:301
1314327211,32sleep30-21swapper/323:36:153
88402717,51sleep20-21swapper/221:39:152
50327121,21sleep00-21swapper/022:56:190
3064727113,45sleep130669-21netstat00:31:281
2773327116,38sleep327805-21gpgv19:16:023
2773327116,38sleep327805-21gpgv19:16:023
2042127111,42sleep120471-21sendmail20:36:011
1835527116,34sleep018406-21latency_hist22:11:020
1395027118,26sleep00-21swapper/021:56:210
134427119,35sleep31380-21/usr/sbin/munin21:16:233
636327024,27sleep36394-21munin-run23:16:003
566127018,25sleep10-21swapper/121:31:031
1051827017,28sleep00-21swapper/023:26:390
2933426910,42sleep029409-21find00:26:340
2385926911,42sleep323997-21modprobe00:11:033
2255726911,39sleep022616-21/usr/sbin/munin20:41:250
1736626912,34sleep317416-21memory22:06:253
94652689,47sleep00-21swapper/023:22:350
94652689,47sleep00-21swapper/023:22:350
845126811,38sleep37420-21/usr/sbin/munin21:36:383
366626818,17sleep30-21swapper/323:06:243
366626818,17sleep30-21swapper/323:06:243
2870326815,39sleep328719-21awk00:26:133
2869626820,32sleep128731-21grep22:41:471
2450726818,26sleep10-21swapper/100:11:291
216862687,12sleep00-21swapper/020:37:010
1670926817,31sleep30-21swapper/320:21:333
1670926817,31sleep30-21swapper/320:21:333
1460626811,39sleep114642-21/usr/sbin/munin20:16:231
351726714,21sleep00-21swapper/023:06:120
351726714,21sleep00-21swapper/023:06:120
245626722,22sleep30-21swapper/319:36:293
2128326715,36sleep321351-21/usr/sbin/munin20:36:313
1737826718,19sleep10-21swapper/120:26:041
1737826718,19sleep10-21swapper/120:26:041
101526711,40sleep31142-21wc21:16:033
2686626614,29sleep026900-21ntp_kernel_pll_19:11:310
2686626614,29sleep026900-21ntp_kernel_pll_19:11:310
2508526615,31sleep00-21swapper/022:31:170
2508526615,31sleep00-21swapper/022:31:170
2289126612,21sleep30-21swapper/300:06:273
1339126619,27sleep113392-21ntp_states20:11:301
55726519,23sleep30-21swapper/319:31:203
3091726529,17sleep00-21swapper/000:31:350
2520026526,22sleep10-21swapper/119:06:471
2380126521,24sleep10-21swapper/122:26:291
2380126521,24sleep10-21swapper/122:26:291
92382647,45sleep05532-1kworker/0:2H21:41:110
92382647,45sleep05532-1kworker/0:2H21:41:110
3027526416,32sleep30-21swapper/300:31:173
2247026413,23sleep00-21swapper/022:21:400
215526417,30sleep12240-21/usr/sbin/munin19:36:221
211726415,20sleep00-21swapper/023:01:240
1624926411,24sleep00-21swapper/023:46:170
1624926411,24sleep00-21swapper/023:46:170
1245226418,20sleep10-21swapper/121:51:211
2604826320,25sleep326085-21/usr/sbin/munin20:51:413
2604826320,25sleep326085-21/usr/sbin/munin20:51:413
150342636,44sleep30-21swapper/321:58:463
1300526316,25sleep30-21swapper/320:11:203
8452626,44sleep00-21swapper/021:14:510
3049326213,27sleep00-21swapper/019:21:360
288826223,21sleep10-21swapper/121:21:221
2313526220,26sleep323193-21irqstats19:01:253
2313526220,26sleep323193-21irqstats19:01:253
705926115,16sleep00-21swapper/023:16:300
705926115,16sleep00-21swapper/023:16:300
561226115,29sleep05633-21munin-run21:31:020
307126117,20sleep00-21swapper/021:21:250
847226015,26sleep38463-21apt-get23:21:173
847226015,26sleep38463-21apt-get23:21:173
415126015,26sleep00-21swapper/019:41:300
24532605,44sleep00-21swapper/021:20:160
2094126021,16sleep30-21swapper/300:01:183
1633226014,25sleep30-21swapper/323:46:203
1633226014,25sleep30-21swapper/323:46:203
576025918,19sleep00-21swapper/023:11:370
1577625916,19sleep30-21swapper/320:21:013
1214425918,18sleep30-21swapper/321:51:033
1214425918,18sleep30-21swapper/321:51:033
954125815,19sleep10-21swapper/121:41:281
954125815,19sleep10-21swapper/121:41:281
3176425815,26sleep331770-21ntp_states22:51:303
2758425817,18sleep10-21swapper/120:56:271
2758425817,18sleep10-21swapper/120:56:271
2531025817,20sleep30-21swapper/320:51:043
1994225815,23sleep00-21swapper/022:16:030
751725713,25sleep07438-21apt-get21:36:190
2849025711,17sleep10-21swapper/100:26:021
2641825722,19sleep10-21swapper/122:36:021
2641825722,19sleep10-21swapper/122:36:021
1011925711,18sleep30-21swapper/321:41:383
1011925711,18sleep30-21swapper/321:41:383
3009025614,23sleep330177-21gpgv21:06:023
2884025612,18sleep00-21swapper/021:01:190
2472125611,28sleep024726-21idleruntime19:06:260
1711325611,32sleep013441-21snmpd20:23:160
1711325611,32sleep013441-21snmpd20:23:160
1314525615,24sleep013138-21apt-get23:36:160
515425516,18sleep30-21swapper/319:46:173
515425516,18sleep30-21swapper/319:46:173
31622556,11sleep10-21swapper/123:04:521
2006225517,20sleep00-21swapper/023:56:340
1965825512,18sleep00-21swapper/020:31:320
1779225515,23sleep017788-21apt-get23:51:130
1326225513,23sleep30-21swapper/321:51:353
1218425519,21sleep10-21swapper/123:31:321
670025413,20sleep10-21swapper/119:51:161
670025413,20sleep10-21swapper/119:51:161
3138625414,23sleep031380-21apt-get19:26:160
3125825413,22sleep30-21swapper/321:06:393
287525412,27sleep32881-21df_inode21:21:203
2560525410,18sleep00-21swapper/020:51:140
2560525410,18sleep00-21swapper/020:51:140
2383625413,18sleep10-21swapper/120:46:181
2717925311,25sleep327176-21apt-get00:21:163
2611125317,21sleep126117-21ntp_states20:51:431
2611125317,21sleep126117-21ntp_states20:51:431
234572537,33sleep30-21swapper/320:44:553
1794625314,22sleep317984-21memory20:26:263
115912539,24sleep30-21swapper/320:06:243
115912539,24sleep30-21swapper/320:06:243
851425211,26sleep08548-21irqstats19:56:240
851425211,26sleep08548-21irqstats19:56:240
477525213,22sleep34804-21ntp_states21:26:293
3131225214,20sleep00-21swapper/022:51:150
1588225213,18sleep00-21swapper/022:01:250
1588225213,18sleep00-21swapper/022:01:250
129825213,23sleep31350-21users22:56:473
697225114,19sleep30-21swapper/319:51:263
697225114,19sleep30-21swapper/319:51:263
2792125112,23sleep027946-21/usr/sbin/munin22:41:010
2792125112,23sleep027946-21/usr/sbin/munin22:41:010
2695825111,21sleep127052-21memory22:36:431
2695825111,21sleep127052-21memory22:36:431
1009525112,24sleep110110-21diskstats23:26:201
1896125014,21sleep318569-21/usr/sbin/munin22:11:253
1509425013,19sleep00-21swapper/020:16:340
2920424911,22sleep129227-21ntp_states21:01:281
2746624912,17sleep30-21swapper/320:56:253
2746624912,17sleep30-21swapper/320:56:253
274242499,16sleep00-21swapper/000:21:250
1761524911,20sleep117610-21apt-get20:26:161
1557424912,23sleep115581-21diskstats22:01:181
1557424912,23sleep115581-21diskstats22:01:181
108162499,23sleep00-21swapper/021:46:160
108162499,23sleep00-21swapper/021:46:160
915124814,19sleep39179-21users19:56:353
915124814,19sleep39179-21users19:56:353
54524812,20sleep1551-21df_inode19:31:181
47424811,19sleep0544-21df19:31:170
144032487,32sleep10-21swapper/123:40:571
2248724712,17sleep00-21swapper/000:06:150
132924710,21sleep11335-21df_inode21:16:211
990624610,22sleep39914-21df20:01:193
990624610,22sleep39914-21df20:01:193
90282439,19sleep18991-21latency_hist21:41:041
35462439,19sleep13616-21df23:06:171
35462439,19sleep13616-21df23:06:171
151502426,18sleep30-21swapper/323:41:283
151502426,18sleep30-21swapper/323:41:283
90272356,15sleep00-21swapper/021:41:030
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional