You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-07 - 17:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Tue Apr 07, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
220712471156,181sleep10-21swapper/110:26:481
292352442107,316sleep129286-21sort12:32:001
28895234048,188sleep128985-21latency_hist07:21:491
28895234048,188sleep128985-21latency_hist07:21:491
6356227815,249sleep06367-21ntpq11:17:100
13353226811,243sleep313397-21ntpq09:57:263
30111224244,87sleep130260-21unixbench_singl09:07:171
28065222113,194sleep228079-21irqstats09:02:052
27452221935,139sleep127591-21latency_hist12:26:471
1905022179,110sleep10-21swapper/108:32:161
27940221424,171sleep128074-21unixbench_singl10:42:191
27940221424,171sleep128074-21unixbench_singl10:42:191
2904221134,95sleep11-21systemd09:26:471
17921221114,179sleep1367-21sshd11:55:261
31051220871,27sleep10-21swapper/107:27:101
2711422049,176sleep127240-21latency_hist07:16:481
31974219938,108sleep10-21swapper/110:57:041
5995219869,81sleep16028-21fschecks_count11:17:031
17500219613,18sleep10-21swapper/108:27:151
17500219613,18sleep10-21swapper/108:27:151
2820321857,104sleep10-21swapper/109:02:081
14519218318,149sleep014513-21apt-get10:01:590
14519218318,149sleep014513-21apt-get10:01:590
17983218195,21sleep10-21swapper/110:12:101
15777217917,100sleep10-21swapper/110:06:451
15777217917,100sleep10-21swapper/110:06:451
13891217758,98sleep10-21swapper/108:17:051
21919217642,22sleep121982-21processes12:07:131
25251217412,140sleep125323-21switchtime08:52:141
16470217214,101sleep10-21swapper/111:51:391
1835217140,116sleep11899-21mailstats07:37:151
10460216448,16sleep10-21swapper/108:06:471
10460216448,16sleep10-21swapper/108:06:471
9090216256,93sleep19107-21fschecks_count11:27:041
8518216149,96sleep18583-21awk09:42:141
28370215219,24sleep10-21swapper/112:27:161
23941215256,44sleep10-21swapper/110:32:051
4046214913,18sleep10-21swapper/107:45:231
12983214010,17sleep10-21swapper/109:57:161
32590213925,90sleep00-21swapper/009:17:050
26670213833,89sleep00-21swapper/010:38:260
1222221377,80sleep10-21swapper/111:37:051
11296213614,24sleep10-21swapper/108:07:161
25325213226,86sleep225455-21timerwakeupswit08:52:152
3374213023,87sleep32873-21apt-get09:26:573
1600821299,46sleep30-21swapper/311:47:173
16571212665,17sleep10-21swapper/110:07:131
18426212527,85sleep118446-21grep11:57:041
20385212325,23sleep10-21swapper/108:37:081
12587212111,30sleep112733-21gpgv09:56:471
4036212013,18sleep00-21swapper/011:11:450
24808212010,40sleep024809-21sshd07:08:230
9684211713,86sleep39711-21fschecks_count09:47:043
9684211713,86sleep39711-21fschecks_count09:47:043
16091211718,78sleep016235-21unixbench_singl08:22:170
1359211623,71sleep31427-21latency_hist09:21:483
9402211519,82sleep39445-21/usr/sbin/munin11:27:103
1686221130,7sleep00-21swapper/008:26:480
1686221130,7sleep00-21swapper/008:26:480
11133211310,86sleep211155-21/usr/sbin/munin08:07:132
376721096,27sleep10-21swapper/111:08:191
18971210917,65sleep00-21swapper/008:32:150
12490210921,71sleep012584-21latency_hist09:56:460
5533210716,38sleep123603-21cyclictest09:32:161
29141210715,36sleep10-21swapper/107:22:051
21576210748,16sleep10-21swapper/108:42:011
26715210612,74sleep226735-21/usr/sbin/munin08:57:122
2587221060,15sleep025873-21munin-plugin-st12:21:450
10085210611,82sleep110169-21grep09:47:141
10085210611,82sleep110169-21grep09:47:141
4814210513,38sleep10-21swapper/107:47:091
2339521058,44sleep123436-21ls08:47:101
1181021057,22sleep111867-21sendmail-msp11:36:451
8886210415,38sleep10-21swapper/108:01:471
8886210415,38sleep10-21swapper/108:01:471
7763210449,43sleep17793-21munin-run09:41:451
19007210420,24sleep20-21swapper/210:16:472
1113210414,36sleep10-21swapper/107:36:471
7873210316,63sleep37546-21/usr/sbin/munin07:57:333
7873210316,63sleep37546-21/usr/sbin/munin07:57:333
4931210325,52sleep24924-21apt-get09:32:012
432721038,35sleep10-21swapper/111:12:021
2566521028,14sleep10-21swapper/110:36:571
28768210112,72sleep10-21swapper/110:46:591
25260210113,72sleep325388-21sed12:17:163
2248210045,43sleep12298-21systemd-run11:06:451
16745210018,55sleep316874-21unixbench_multi10:07:153
599129811,71sleep10-21swapper/107:51:571
3060029816,65sleep030745-21latency_hist12:36:470
2427829813,60sleep30-21swapper/307:07:133
2036129817,57sleep10-21swapper/112:02:121
305332969,32sleep10-21swapper/110:52:081
305332969,32sleep10-21swapper/110:52:081
153482967,35sleep10-21swapper/108:21:581
76829510,70sleep0769-21sshd07:33:230
324992958,12sleep10-21swapper/109:16:581
3218729532,18sleep10-21swapper/109:16:461
2147929518,53sleep321622-21unixbench_singl10:22:183
936329411,69sleep39469-21latency_hist09:46:483
2660729417,20sleep00-21swapper/012:22:140
1944229435,16sleep10-21swapper/110:17:071
408229326,26sleep123603-21cyclictest09:28:291
246632937,32sleep10-21swapper/112:17:021
235592937,75sleep123589-21munin-run10:31:441
201072937,13sleep20-21swapper/212:02:052
1834129311,19sleep00-21swapper/011:56:590
1448029311,31sleep10-21swapper/111:42:181
2644429229,38sleep226601-21unixbench_multi07:12:172
840029128,51sleep10-21swapper/111:22:441
3068229022,47sleep030764-21latency_hist09:11:480
286012908,69sleep013441-21snmpd12:27:190
493228911,62sleep34999-21df09:32:023
2633928818,47sleep10-21swapper/108:57:031
2385428813,40sleep30-21swapper/310:31:593
1477828818,49sleep314494-21/usr/sbin/munin10:02:073
1477828818,49sleep314494-21/usr/sbin/munin10:02:073
1293328820,49sleep212439-21apt-get09:56:572
2590428723,31sleep20-21swapper/212:21:482
28992866,24sleep20-21swapper/207:42:012
2894528624,39sleep228967-21awk12:31:452
1356828612,59sleep013680-21cat08:16:480
2512228510,62sleep125137-21latency_hist07:11:471
2290028519,47sleep123033-21unixbench_multi07:02:151
319852840,6sleep20-21swapper/210:57:042
2514028428,29sleep20-21swapper/207:11:482
2913428313,53sleep029200-21latency_hist09:06:470
20652836,65sleep00-21swapper/011:04:260
816028215,47sleep28303-21/usr/sbin/munin11:22:172
441228220,45sleep04493-21grep11:12:050
1912828216,46sleep219268-21sed08:32:172
131852826,27sleep10-21swapper/108:13:011
2057628113,55sleep320637-21switchtime08:37:143
1375128111,52sleep30-21swapper/311:42:043
75128013,44sleep00-21swapper/011:01:480
2941628021,38sleep229454-21ls12:32:052
1506228025,35sleep015074-21sh08:21:450
28742798,51sleep30-21swapper/307:41:583
239762795,28sleep10-21swapper/112:13:021
2545627812,15sleep30-21swapper/308:52:173
2535827814,46sleep025135-21apt-get10:36:480
3164927719,32sleep30-21swapper/310:56:463
3164927719,32sleep30-21swapper/310:56:463
2136227614,46sleep221478-21cat10:22:172
3095627520,23sleep00-21swapper/007:27:090
1920127515,45sleep319251-21cpuspeed10:17:013
103327523,34sleep31058-21modprobe07:36:453
876527417,28sleep00-21swapper/011:26:470
802627412,47sleep08034-21/usr/sbin/munin09:41:560
1841127414,46sleep318417-21df_inode11:57:023
642427318,25sleep30-21swapper/311:17:133
464127317,35sleep24669-21munin-run09:31:452
2948527317,24sleep20-21swapper/210:47:162
2213427314,40sleep022269-21awk12:07:160
2055527320,26sleep00-21swapper/012:02:160
401127111,43sleep04081-21sensors09:28:280
3056727117,41sleep330530-21latency_hist12:36:463
2680027111,44sleep026867-21awk08:57:140
2507727114,42sleep025119-21awk08:52:090
2303527122,30sleep223125-21/usr/sbin/munin07:02:162
1770027121,23sleep00-21swapper/010:12:050
1736527120,24sleep30-21swapper/308:27:133
1736527120,24sleep30-21swapper/308:27:133
1222027120,21sleep20-21swapper/208:12:012
1085527117,25sleep30-21swapper/308:07:063
579227016,26sleep00-21swapper/007:51:480
2309827017,20sleep00-21swapper/008:46:560
1961327016,33sleep219643-21awk12:01:452
1589527017,32sleep20-21swapper/208:22:152
119122706,13sleep00-21swapper/008:11:310
817426919,18sleep20-21swapper/207:57:402
817426919,18sleep20-21swapper/207:57:402
500126917,35sleep05010-21/usr/sbin/munin09:32:030
497826916,27sleep20-21swapper/207:47:142
455726912,43sleep24594-21irqstats11:12:072
2877026915,22sleep30-21swapper/310:47:013
210226915,16sleep00-21swapper/007:37:190
172522699,49sleep00-21swapper/010:10:520
3131426812,38sleep231352-21grep09:12:102
2276326822,24sleep30-21swapper/312:11:473
2198426814,28sleep20-21swapper/212:07:142
2157726825,22sleep20-21swapper/208:42:022
1891426815,26sleep30-21swapper/308:32:133
3199226715,26sleep20-21swapper/207:31:472
2858526720,19sleep20-21swapper/210:46:482
2858526720,19sleep20-21swapper/210:46:482
1511326714,30sleep20-21swapper/208:21:472
1328726722,33sleep020638-1kworker/0:2H11:40:310
1124026722,21sleep20-21swapper/209:52:042
101522678,47sleep00-21swapper/008:03:480
101522678,47sleep00-21swapper/008:03:480
3271326623,28sleep2301-21/usr/sbin/munin09:17:072
3170326616,26sleep00-21swapper/010:56:470
2774226619,27sleep20-21swapper/207:17:092
2774226619,27sleep20-21swapper/207:17:092
2308526619,21sleep00-21swapper/012:12:050
219926613,23sleep20-21swapper/209:22:162
2065626611,25sleep00-21swapper/008:37:150
2359926518,29sleep21-21systemd10:31:462
2050726517,33sleep21-21systemd08:37:132
168026516,21sleep30-21swapper/309:22:053
926526415,25sleep00-21swapper/011:27:070
769726419,19sleep00-21swapper/007:57:300
769726419,19sleep00-21swapper/007:57:300
3044626411,40sleep330503-21cat07:26:483
1968526414,32sleep00-21swapper/010:17:140
1588826420,29sleep216007-21cat11:47:162
3772638,15sleep20-21swapper/207:32:162
2960626318,27sleep329640-21/usr/sbin/munin09:07:063
2618226318,27sleep30-21swapper/310:37:483
2616226323,23sleep00-21swapper/007:12:110
637526224,19sleep20-21swapper/211:17:122
2735026214,30sleep027344-21apt-get07:16:590
2735026214,30sleep027344-21apt-get07:16:590
2682626218,21sleep30-21swapper/312:22:173
162326220,25sleep31712-21load07:37:093
1099626217,27sleep20-21swapper/211:32:102
761926120,25sleep30-21swapper/311:22:053
665926118,17sleep20-21swapper/207:52:152
2871726114,26sleep328814-21gpgv07:21:473
2871726114,26sleep328814-21gpgv07:21:473
2848026118,20sleep30-21swapper/309:02:143
1757926115,25sleep20-21swapper/208:27:162
1757926115,25sleep20-21swapper/208:27:162
295026018,20sleep00-21swapper/007:42:030
2783326024,22sleep00-21swapper/010:42:180
2783326024,22sleep00-21swapper/010:42:180
2643726010,13sleep30-21swapper/308:57:053
233362606,12sleep00-21swapper/010:28:570
1860926018,23sleep30-21swapper/310:13:253
1745026019,24sleep20-21swapper/211:52:162
1002826012,31sleep210052-21ntp_states09:47:112
1002826012,31sleep210052-21ntp_states09:47:112
1677025912,32sleep316837-21/usr/sbin/munin11:52:013
1269925912,18sleep30-21swapper/308:12:153
1101625918,23sleep00-21swapper/011:32:120
658825817,23sleep00-21swapper/009:37:040
471425818,20sleep30-21swapper/307:47:073
39202586,40sleep20-21swapper/211:10:182
3228325816,18sleep30-21swapper/309:16:483
285625813,27sleep02903-21true09:26:460
2775825817,24sleep327767-21df_inode12:27:013
2659125821,21sleep20-21swapper/212:22:122
2323625818,24sleep323270-21/usr/sbin/munin12:12:083
230125818,25sleep22392-21dpkg11:06:462
1521925816,25sleep015215-21apt-get11:46:570
1498425813,32sleep215001-21cut11:46:472
1262725816,19sleep20-21swapper/211:37:142
1188825815,15sleep30-21swapper/311:36:473
3039825713,29sleep230400-21/usr/sbin/munin10:52:052
3039825713,29sleep230400-21/usr/sbin/munin10:52:052
2348625716,20sleep00-21swapper/007:06:460
1697625711,17sleep00-21swapper/011:52:060
1153825715,28sleep311575-21sed09:52:093
2385825614,24sleep223911-21users12:12:192
2358925616,19sleep20-21swapper/208:47:152
2093225624,18sleep00-21swapper/010:22:060
1420025614,25sleep30-21swapper/308:17:113
1375925624,18sleep00-21swapper/011:42:050
3039925516,22sleep20-21swapper/207:26:472
2876725513,25sleep028761-21apt-get10:46:580
2612925519,16sleep30-21swapper/307:12:103
971425412,23sleep09806-21perl09:47:050
971425412,23sleep09806-21perl09:47:050
781425414,25sleep37890-21latency_hist09:41:473
678525413,25sleep36890-21users07:52:163
459525420,18sleep30-21swapper/311:12:083
2222025413,24sleep322219-21apt-get07:01:573
2157525413,24sleep021570-21apt-get08:42:000
899825312,23sleep28996-21apt-get11:26:572
3006625311,19sleep00-21swapper/007:22:330
274425310,19sleep30-21swapper/311:06:563
2422325317,20sleep20-21swapper/210:32:102
2360425316,18sleep30-21swapper/310:31:473
2288525318,17sleep30-21swapper/308:46:473
2047325315,18sleep30-21swapper/312:02:143
920525214,22sleep39210-21diskstats08:02:053
920525214,22sleep39210-21diskstats08:02:053
1632725210,23sleep20-21swapper/210:07:062
3076125117,18sleep30-21swapper/307:27:043
3068425113,16sleep00-21swapper/010:52:110
2936825114,21sleep229365-21apt-get09:06:572
2697925116,20sleep227074-21latency_hist10:41:472
239325113,20sleep30-21swapper/311:06:473
2312825113,22sleep323195-21df08:47:043
190125119,17sleep20-21swapper/207:37:162
1270725114,15sleep30-21swapper/311:37:153
927925014,20sleep29280-21sshd09:46:232
599024912,22sleep05986-21apt-get07:51:560
1296624811,22sleep012952-21apt-get09:57:030
93362479,18sleep20-21swapper/208:02:082
93362479,18sleep20-21swapper/208:02:082
1670824711,13sleep30-21swapper/308:26:483
1491624613,15sleep20-21swapper/210:02:102
1491624613,15sleep20-21swapper/210:02:102
3091824512,16sleep00-21swapper/009:12:000
176862459,20sleep217693-21df_inode10:12:032
3188724412,19sleep30-21swapper/310:56:583
687224311,15sleep20-21swapper/209:37:102
276722426,13sleep30-21swapper/310:42:123
276722426,13sleep30-21swapper/310:42:123
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional