You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-16 - 07:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Wed Sep 16, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
82012981410,9787sleep02776-21kworker/u8:323:18:240
128962216715,2139sleep0997-1kworker/0:2H20:08:100
128962216715,2139sleep0997-1kworker/0:2H20:08:100
7978212629,1237sleep212872-1kworker/2:2H19:53:102
31518268748,475sleep131548-21modprobe21:07:521
31518268748,475sleep131548-21modprobe21:07:521
30637252312,494sleep230685-21ntpq21:03:172
30637252312,494sleep230685-21ntpq21:03:172
3103625146,495sleep12244-21kworker/u8:122:48:121
4894246454,393sleep14952-21cut23:08:111
9654237267,217sleep19757-21fschecks_time19:58:141
2679222909,179sleep126912-21sed22:33:221
16632227518,240sleep316648-21sort22:03:113
7629227411,203sleep17689-21sendmail_mailqu21:33:251
10783224795,25sleep10-21swapper/121:43:221
10783224795,25sleep10-21swapper/121:43:221
3060224218,209sleep33140-21sed21:18:253
3409223917,208sleep03475-21sort23:03:140
26862223530,187sleep226981-21cat19:12:562
26862223530,187sleep226981-21cat19:12:562
1007222545,114sleep10-21swapper/121:13:141
82042218107,50sleep10-21swapper/119:53:291
24002221833,171sleep124012-21acpi20:43:051
24652221417,135sleep10-21swapper/122:28:131
7925221122,170sleep08104-21latency_hist23:17:560
22487221148,119sleep122506-21modprobe22:22:521
30499220911,183sleep130526-21munin-run22:47:521
27957220931,115sleep10-21swapper/122:38:151
27957220931,115sleep10-21swapper/122:38:151
2740622069,26sleep127477-21latency20:53:191
2740622069,26sleep127477-21latency20:53:191
16946218832,97sleep116981-21ntpq22:03:171
2913921850,35sleep125295-21cyclictest00:23:231
3149421839,94sleep131496-21apt-get00:33:051
859021818,154sleep18615-21awk23:18:501
3591218116,113sleep13669-21mailstats19:38:231
32556218144,97sleep00-21swapper/022:53:080
3789217012,113sleep10-21swapper/123:03:231
6342216730,115sleep26488-21unixbench_singl21:28:252
28676216111,132sleep028740-21cat00:23:140
5947215711,131sleep05965-21latency_hist19:47:550
5947215711,131sleep05965-21latency_hist19:47:550
29764215714,27sleep129909-21unixbench_multi19:18:281
302221567,101sleep13021-21munin-run23:02:541
2043215344,92sleep10-21swapper/119:33:431
13039215028,96sleep10-21swapper/120:08:161
13039215028,96sleep10-21swapper/120:08:161
1247421509,126sleep112603-21apt-key20:07:551
3121221499,85sleep131249-21modprobe00:32:531
18484214616,24sleep10-21swapper/120:23:301
183521465,11sleep00-21swapper/021:13:290
5682214510,115sleep10-21swapper/121:28:121
31814214517,20sleep20-21swapper/222:48:502
19738214318,16sleep10-21swapper/120:28:191
194042143110,15sleep10-21swapper/122:12:551
23379214217,22sleep10-21swapper/100:07:121
1767721399,51sleep117738-21if_eth023:48:141
30896213813,109sleep130940-21latency19:23:171
30896213813,109sleep130940-21latency19:23:171
2662421328,111sleep126771-21cat00:17:541
2662421328,111sleep126771-21cat00:17:541
26117213087,20sleep10-21swapper/120:48:221
21117213017,16sleep10-21swapper/123:58:241
26833212751,64sleep126815-21latency_hist19:12:551
26833212751,64sleep126815-21latency_hist19:12:551
12330212713,97sleep311842-21/usr/sbin/munin21:48:213
15565212665,23sleep115635-21switchtime21:58:251
4661212516,96sleep34745-21expr21:23:253
4661212516,96sleep34745-21expr21:23:253
3002321229,93sleep10-21swapper/121:02:561
16114212120,68sleep30-21swapper/320:18:153
1187821209,89sleep111958-21timerandwakeup23:28:231
27088211911,94sleep127237-21cat19:12:591
10137211712,28sleep20-21swapper/221:42:552
10137211712,28sleep20-21swapper/221:42:552
1285521168,75sleep1367-21snmpd21:49:241
574721137,96sleep1367-21snmpd19:45:581
574721137,96sleep1367-21snmpd19:45:581
7444211231,21sleep17474-21munin-run19:52:531
7444211231,21sleep17474-21munin-run19:52:531
2403521129,87sleep124088-21systemd-run22:27:531
8534211016,79sleep38629-21cut21:37:553
28375211012,83sleep228511-21awk22:38:242
28375211012,83sleep228511-21awk22:38:242
635621097,17sleep10-21swapper/123:13:081
30048210913,78sleep030123-21fschecks_time00:28:150
1476221088,86sleep114801-21ntpq23:38:161
1476221088,86sleep114801-21ntpq23:38:161
2685921056,87sleep126854-21apt-get00:18:061
2685921056,87sleep126854-21apt-get00:18:061
2043121058,82sleep120572-21latency_hist23:57:551
17584210516,69sleep017673-21fschecks_time23:48:130
7573210319,66sleep37659-21apt-config19:52:553
7573210319,66sleep37659-21apt-config19:52:553
250721036,17sleep12574-21df21:18:101
29530210211,78sleep029540-21ntp_states19:18:230
1964321027,35sleep10-21swapper/122:13:071
2414021017,13sleep10-21swapper/100:08:171
2880021009,58sleep20-21swapper/220:58:162
144132998,51sleep10-21swapper/120:13:041
923329817,61sleep09265-21grep19:57:530
67729819,56sleep2641-21latency_hist21:12:542
67729819,56sleep2641-21latency_hist21:12:542
2408629818,61sleep324089-21needreboot00:08:153
1286129821,59sleep212387-21apt-get20:08:042
1286129821,59sleep212387-21apt-get20:08:042
159402976,32sleep10-21swapper/123:43:091
1505329711,65sleep00-21swapper/020:13:240
6532966,13sleep10-21swapper/121:12:531
6532966,13sleep10-21swapper/121:12:531
4992968,51sleep10-21swapper/122:53:211
1539229614,63sleep015426-21ntp_kernel_pll_21:58:200
1443529627,43sleep214432-21apt-get20:13:052
130692965,32sleep10-21swapper/123:33:161
130692965,32sleep10-21swapper/123:33:161
201062948,13sleep30-21swapper/322:13:213
1235529415,63sleep012435-21grep21:48:220
1007829430,19sleep10-21swapper/121:42:541
1007829430,19sleep10-21swapper/121:42:541
29082938,66sleep20-21swapper/221:18:192
2453329319,54sleep024051-21apt-get22:28:040
108082936,32sleep10-21swapper/123:25:361
227522928,53sleep10-21swapper/120:38:161
171782925,11sleep10-21swapper/120:21:321
1655029212,67sleep016601-21sort22:03:080
53082897,31sleep10-21swapper/121:27:521
53082897,31sleep10-21swapper/121:27:521
147628917,51sleep21492-21awk19:33:032
138712896,29sleep10-21swapper/121:53:201
1196128916,49sleep20-21swapper/223:28:242
1137828917,45sleep311081-21/usr/sbin/munin20:03:173
8812886,28sleep10-21swapper/119:31:361
766128814,57sleep07864-21latency_hist19:52:560
766128814,57sleep07864-21latency_hist19:52:560
2497728825,43sleep0231-21systemd-journal00:12:520
218812885,29sleep10-21swapper/120:34:161
2590528718,50sleep226057-21date22:32:552
2332228621,49sleep223389-21/usr/sbin/munin22:23:222
39428516,50sleep2485-21ls00:37:542
2726928514,53sleep027314-21/usr/sbin/munin00:18:220
2726928514,53sleep027314-21/usr/sbin/munin00:18:220
1946228517,53sleep219601-21cut22:12:562
128928518,17sleep20-21swapper/221:13:192
3105728420,36sleep00-21swapper/022:48:280
3274328315,52sleep3356-21/usr/sbin/munin19:28:233
2785628317,47sleep228000-21grep20:53:282
2785628317,47sleep228000-21grep20:53:282
2536128326,23sleep30-21swapper/300:13:123
2536128326,23sleep30-21swapper/300:13:123
884128211,57sleep28849-21df21:38:102
884128211,57sleep28849-21df21:38:102
3023028235,29sleep330295-21latency00:28:183
567328110,53sleep00-21swapper/021:28:110
226428119,20sleep30-21swapper/322:58:203
1578528012,51sleep315728-21latency_hist20:17:553
2421827923,32sleep324276-21/usr/sbin/munin20:43:153
451727823,32sleep24545-21modprobe23:07:522
405527812,17sleep00-21swapper/021:23:100
405527812,17sleep00-21swapper/021:23:100
2463427812,49sleep324641-21df_inode22:28:113
2029127821,35sleep220313-40install23:57:522
724027727,26sleep00-21swapper/021:33:160
2683327616,43sleep226852-21/usr/sbin/munin00:18:032
2683327616,43sleep226852-21/usr/sbin/munin00:18:032
259127619,26sleep00-21swapper/021:18:130
1394427619,42sleep014019-21cut21:53:230
3062627518,39sleep230620-21apt-get19:23:092
3062627518,39sleep230620-21apt-get19:23:092
2733327516,43sleep327383-21if_eth020:53:173
2733327516,43sleep327383-21if_eth020:53:173
196022750,7sleep30-21swapper/322:12:563
113027516,41sleep31188-21/usr/sbin/munin21:13:163
3582736,46sleep00-21swapper/019:28:240
30227322,31sleep00-21swapper/000:37:520
2776627313,40sleep327858-21/usr/sbin/munin19:13:173
3239127218,28sleep30-21swapper/321:08:233
3239127218,28sleep30-21swapper/321:08:233
306227215,39sleep33063-21/usr/sbin/munin19:38:053
2955027214,18sleep20-21swapper/219:18:252
2932627120,21sleep20-21swapper/222:43:132
2566727118,28sleep30-21swapper/320:48:123
239022710,6sleep00-21swapper/020:42:550
2099127118,27sleep00-21swapper/022:17:540
1879827124,29sleep20-21swapper/223:52:552
1803027114,41sleep318087-21irqstats20:23:203
507927012,44sleep05131-21latency23:08:150
268532708,16sleep30-21swapper/300:18:043
268532708,16sleep30-21swapper/300:18:043
2304627024,21sleep30-21swapper/322:23:153
193582708,46sleep30-21swapper/320:28:093
1147227018,29sleep00-21swapper/020:03:180
2420926913,13sleep10-21swapper/119:03:141
1734826916,33sleep217451-21apt-config20:22:552
1650326910,42sleep016562-21/usr/sbin/munin20:18:240
1439326917,14sleep00-21swapper/023:38:070
1439326917,14sleep00-21swapper/023:38:070
1419226919,18sleep30-21swapper/320:12:543
1419226919,18sleep30-21swapper/320:12:543
871326818,24sleep20-21swapper/223:18:552
37212689,15sleep30-21swapper/323:03:213
3194126820,24sleep00-21swapper/019:27:560
3194126820,24sleep00-21swapper/019:27:560
2895026812,39sleep329018-21/usr/sbin/munin00:23:223
2378926815,36sleep323898-21cut20:42:553
1913526815,35sleep019242-21grep23:53:140
3150626718,33sleep331573-21df00:33:113
264252678,50sleep00-21swapper/000:16:200
264252678,50sleep00-21swapper/000:16:200
2236126714,35sleep222149-21/usr/sbin/munin00:03:132
200326726,28sleep00-21swapper/019:33:420
1991626618,31sleep20-21swapper/222:13:162
748526520,32sleep27572-21apt-config19:52:542
748526520,32sleep27572-21apt-config19:52:542
655526513,37sleep36612-21cat23:13:143
3195926518,25sleep30-21swapper/322:48:513
2535126524,23sleep30-21swapper/319:07:563
2256626521,24sleep20-21swapper/220:38:132
2115126514,35sleep021176-21irqstats20:33:140
2108726518,25sleep30-21swapper/320:33:133
165826516,21sleep30-21swapper/319:33:323
681826414,26sleep20-21swapper/223:13:212
62726417,29sleep0648-21apt21:12:520
62726417,29sleep0648-21apt21:12:520
2661626414,12sleep30-21swapper/322:33:203
2348126411,35sleep323582-21sed00:07:553
2244626410,23sleep00-21swapper/000:03:150
2130926424,21sleep30-21swapper/322:18:103
1031926420,24sleep30-21swapper/321:43:083
1031926420,24sleep30-21swapper/321:43:083
689926314,22sleep30-21swapper/321:32:543
536426321,23sleep30-21swapper/319:43:263
536426321,23sleep30-21swapper/319:43:263
207826311,36sleep22122-21ntpq22:58:152
109726323,25sleep20-21swapper/219:32:552
1033826320,24sleep30-21swapper/323:23:243
1591326222,20sleep00-21swapper/023:43:040
1291426215,21sleep30-21swapper/323:33:143
1291426215,21sleep30-21swapper/323:33:143
123732628,42sleep30-21swapper/323:30:513
945426111,28sleep00-21swapper/021:38:250
945426111,28sleep00-21swapper/021:38:250
2912826110,37sleep029165-21ntp_states20:58:220
2146726113,31sleep221497-21irqstats22:18:132
135652616,12sleep30-21swapper/321:53:143
1165926111,34sleep211732-21sort20:03:242
3158326019,18sleep00-21swapper/021:07:550
3158326019,18sleep00-21swapper/021:07:550
2959026016,26sleep329684-21mailstats19:18:263
2684226014,27sleep026861-21munin-run20:52:530
2433326020,16sleep20-21swapper/200:08:222
2122826017,24sleep00-21swapper/022:18:070
2032326017,26sleep31-21systemd23:57:533
192832607,40sleep00-21swapper/022:11:570
1746126017,28sleep317479-21apt23:48:043
530925917,26sleep25320-21modprobe21:27:532
530925917,26sleep25320-21modprobe21:27:532
2434825923,19sleep00-21swapper/019:03:170
2129025914,29sleep321415-21timerwakeupswit23:58:263
1839325919,21sleep00-21swapper/020:23:290
146425915,20sleep20-21swapper/222:57:552
1086725918,28sleep210870-21latency_hist20:02:552
438825820,18sleep00-21swapper/019:42:540
318212587,42sleep30-21swapper/319:27:063
318212587,42sleep30-21swapper/319:27:063
281652588,17sleep00-21swapper/020:54:590
281652588,17sleep00-21swapper/020:54:590
2061725813,27sleep020613-21apt-get23:58:060
814325713,23sleep20-21swapper/221:34:152
463925710,23sleep20-21swapper/219:43:092
463925710,23sleep20-21swapper/219:43:092
2168025715,19sleep20-21swapper/220:33:252
1631225712,21sleep30-21swapper/322:02:553
1508025721,19sleep20-21swapper/221:58:142
1439025711,25sleep214386-21apt-get23:38:052
1439025711,25sleep214386-21apt-get23:38:052
611625610,19sleep30-21swapper/323:12:543
3068725615,23sleep330711-21ntp_states21:03:183
3068725615,23sleep330711-21ntp_states21:03:183
2883525619,23sleep00-21swapper/019:17:540
2776825616,26sleep027774-21/usr/sbin/munin22:38:100
2776825616,26sleep027774-21/usr/sbin/munin22:38:100
2770325618,23sleep327766-21df22:38:093
2770325618,23sleep327766-21df22:38:093
273702565,38sleep00-21swapper/022:37:450
261892564,11sleep20-21swapper/200:13:272
261892564,11sleep20-21swapper/200:13:272
2580825618,20sleep30-21swapper/319:08:163
2580825618,20sleep30-21swapper/319:08:163
1935725615,24sleep219353-21apt-get20:28:082
1828125618,17sleep20-21swapper/220:23:272
1724925617,22sleep217312-21cut23:47:552
1180725614,23sleep00-21swapper/023:28:220
998325515,24sleep310005-21ntp_states19:58:203
3258025517,22sleep332575-21apt-get22:53:113
1786025516,25sleep217939-21latency_hist22:07:552
981225416,19sleep00-21swapper/023:23:130
955025411,25sleep09544-21apt-get19:58:090
797925414,24sleep37968-21apt-get19:53:113
480225416,22sleep24803-21apt-get23:08:042
308925414,24sleep23083-21apt-get19:38:082
3062125414,23sleep330682-21munin-run22:47:543
2995825412,26sleep229954-21apt-get00:28:092
262592549,23sleep00-21swapper/019:08:250
262592549,23sleep00-21swapper/019:08:250
2410425414,16sleep00-21swapper/020:43:120
234025416,20sleep00-21swapper/022:58:210
201042547,13sleep00-21swapper/020:28:260
1341825413,25sleep213410-21apt-get21:53:092
2528025315,21sleep20-21swapper/219:07:552
1014225314,24sleep210179-21ntp_states23:23:192
814725213,24sleep37639-21apt-get23:18:033
2756725215,18sleep20-21swapper/219:13:122
1250825221,17sleep20-21swapper/221:48:242
3001025117,22sleep31-21systemd21:02:543
2689325112,25sleep226988-21sed20:52:552
2609325114,22sleep226111-21apt-get22:33:032
1321625110,21sleep00-21swapper/023:33:190
1321625110,21sleep00-21swapper/023:33:190
1309225114,21sleep213186-21memory23:33:172
1309225114,21sleep213186-21memory23:33:172
2945325011,23sleep029510-21irqstats22:43:150
1653224913,19sleep30-21swapper/323:43:233
1162224917,16sleep00-21swapper/021:47:540
1162224917,16sleep00-21swapper/021:47:540
2323724810,21sleep023258-21awk22:23:200
2106824810,18sleep20-21swapper/223:58:212
1447524811,17sleep30-21swapper/323:38:103
1447524811,17sleep30-21swapper/323:38:103
877024711,21sleep38766-21apt-get21:38:073
877024711,21sleep38766-21apt-get21:38:073
2449124717,16sleep20-21swapper/220:43:222
2928024410,20sleep229308-21users00:23:252
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional