You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-13 - 09:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Sun Sep 13, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
58052539611,5372sleep3799-21kworker/3:121:12:313
4118212375,1204sleep212872-1kworker/2:2H00:33:262
17536266811,641sleep317537-21sendmail-mta20:04:313
17536266811,641sleep317537-21sendmail-mta20:04:313
113072490459,18sleep311310-21sshd19:44:323
2910323637,345sleep2367-21snmpd22:26:532
1433122837,222sleep214367-21ntpq21:38:272
3263228210,191sleep23342-21latency_hist22:48:052
7402226510,197sleep27448-21idleruntime-cro21:18:042
1044922490,8sleep30-21swapper/323:08:373
854221611,140sleep20-21swapper/200:23:242
23352221357,141sleep223446-21fschecks_time22:08:242
3661221254,134sleep23782-21dpkg19:23:062
3661221254,134sleep23782-21dpkg19:23:062
6305220410,150sleep26309-21cron22:58:032
2497922039,154sleep225021-21basename22:13:242
21885220113,146sleep221903-21idleruntime-cro23:48:032
25405219716,130sleep225450-21fschecks_count23:58:252
26871219215,163sleep026936-21awk22:18:310
3082218939,20sleep20-21swapper/221:03:222
2469218010,153sleep02537-21fw_packets19:18:280
2469218010,153sleep02537-21fw_packets19:18:280
21735217878,43sleep20-21swapper/220:18:352
17213216919,135sleep11-21systemd23:33:031
17213216919,135sleep11-21systemd23:33:031
23811216663,47sleep230997-21cyclictest23:53:242
3615216310,138sleep23824-21latency_hist00:33:062
27698216219,117sleep20-21swapper/222:23:062
20636215945,79sleep220670-21ps21:58:472
1655621587,139sleep216633-21find23:28:322
1655621587,139sleep216633-21find23:28:322
11028215712,94sleep211083-21interrupts21:28:302
1468221559,132sleep214746-21modprobe19:58:032
8646215215,77sleep20-21swapper/223:03:322
18852214910,125sleep219001-21latency_hist23:38:052
7722214618,69sleep20-21swapper/221:18:202
9265214545,42sleep20-21swapper/221:23:172
662821456,104sleep20-21swapper/219:29:242
662821456,104sleep20-21swapper/219:29:242
31649214329,58sleep231754-21timerandwakeup22:33:362
10003214011,85sleep20-21swapper/223:08:292
29568213829,48sleep229654-21fschecks_time22:28:232
17974213716,102sleep118040-21awk23:33:321
12730213614,106sleep212766-21ntpq21:33:312
2304621359,85sleep223060-21kernelversion20:23:282
10940213418,46sleep111003-21fschecks_time21:28:291
22255213310,78sleep20-21swapper/223:48:232
22255213310,78sleep20-21swapper/223:48:232
17216213162,24sleep20-21swapper/221:48:252
5903213010,58sleep20-21swapper/221:13:072
10330212914,57sleep210381-21sort19:43:212
7291212813,96sleep17383-21tune2fs19:33:231
18471212314,93sleep118482-21/usr/sbin/munin20:08:281
12640212333,76sleep212785-21latency_hist23:18:062
458721215,11sleep20-21swapper/222:50:522
15486212117,42sleep20-21swapper/221:43:202
1951421205,104sleep20-21swapper/221:53:542
815821196,12sleep20-21swapper/219:34:092
30741211911,90sleep30-21swapper/322:33:043
17134211722,81sleep017226-21cat20:03:350
17134211722,81sleep017226-21cat20:03:350
548321168,46sleep20-21swapper/219:28:072
548321168,46sleep20-21swapper/219:28:072
4960211611,89sleep04991-21nfs_client21:08:290
28406211615,32sleep028418-21apt-get00:08:200
28406211615,32sleep028418-21apt-get00:08:200
11366211512,76sleep10-21swapper/123:13:251
69921135,29sleep20-21swapper/220:55:182
69921135,29sleep20-21swapper/220:55:182
32269211353,38sleep20-21swapper/200:18:362
2818021126,40sleep228316-21latency_hist00:08:062
2818021126,40sleep228316-21latency_hist00:08:062
31585211114,79sleep131648-21/usr/sbin/munin22:33:351
6222210913,81sleep36246-21fschecks_count21:13:243
32456210915,70sleep00-21swapper/022:38:060
4755210822,68sleep30-21swapper/300:33:433
2721421078,44sleep227265-21ls20:38:062
13205210746,40sleep20-21swapper/219:53:072
13205210746,40sleep20-21swapper/219:53:072
359210612,79sleep332317-21apt-get22:38:093
1151210551,36sleep20-21swapper/222:39:012
1933210454,35sleep20-21swapper/220:58:372
1003321038,84sleep00-21swapper/019:42:590
7819210217,69sleep07935-21sed19:33:340
30446210215,18sleep30-21swapper/319:03:333
30018210247,14sleep20-21swapper/200:13:232
18190210217,62sleep018080-21/usr/sbin/munin20:08:230
895210116,64sleep10-21swapper/122:38:521
3168121017,39sleep20-21swapper/219:08:322
28422210111,72sleep028504-21mailstats22:23:320
5541210022,63sleep35589-21latency_hist19:28:083
5541210022,63sleep35589-21latency_hist19:28:083
3124229745,13sleep20-21swapper/220:48:362
1008229746,39sleep210088-21munin-run19:43:052
351929611,69sleep13614-21date00:33:051
31162967,13sleep00-21swapper/021:03:240
2963829623,44sleep20-21swapper/219:03:162
34929534,41sleep20-21swapper/219:13:092
2764629518,55sleep327653-21modprobe22:23:033
1848329534,48sleep218531-21ntp_kernel_pll_20:08:292
2596429319,54sleep126025-21fschecks_time20:33:241
67042927,69sleep20-21swapper/222:58:232
300182929,68sleep030089-21sensors22:28:330
2793229234,18sleep20-21swapper/220:38:302
2608829214,57sleep325853-21/usr/sbin/munin20:33:263
239829116,61sleep12407-21df_inode19:18:251
239829116,61sleep12407-21df_inode19:18:251
1703529113,62sleep117036-21/usr/sbin/munin21:48:201
124129014,62sleep31314-21sendmail_mailtr19:13:343
1158629014,60sleep011628-21systemd-run19:48:040
2562828916,55sleep20-21swapper/220:33:052
2033628924,44sleep120468-21tail20:13:371
2033628924,44sleep120468-21tail20:13:371
1439728913,62sleep10-21swapper/121:38:301
91828825,40sleep1941-21apt-get20:58:041
91828825,40sleep1941-21apt-get20:58:041
543128816,51sleep14567-21/usr/sbin/munin21:08:381
148042887,22sleep212-21rcu_sched23:23:272
148042887,22sleep212-21rcu_sched23:23:272
3038628710,16sleep30-21swapper/300:13:323
150032878,66sleep214994-21apt-get19:58:202
1285428723,17sleep20-21swapper/223:18:142
2967228610,59sleep20-21swapper/200:13:032
2967228610,59sleep20-21swapper/200:13:032
2262328512,20sleep00-21swapper/022:03:370
2208528511,58sleep322129-21ntpq22:03:283
2147128519,28sleep10-21swapper/122:03:061
41572840,6sleep10-21swapper/122:48:401
1748928416,12sleep20-21swapper/220:03:532
1748928416,12sleep20-21swapper/220:03:532
77742830,8sleep00-21swapper/021:18:210
2292228317,40sleep023072-21grep23:48:370
2292228317,40sleep023072-21grep23:48:370
2017728318,11sleep20-21swapper/223:41:002
2919428217,46sleep329196-21cron22:28:033
2497128212,55sleep125078-21timerwakeupswit20:28:361
241328218,20sleep30-21swapper/319:18:273
241328218,20sleep30-21swapper/319:18:273
2285828219,11sleep20-21swapper/222:06:382
147728224,33sleep10-21swapper/120:58:201
1330128214,52sleep013312-21ntp_states23:18:280
12032820,9sleep00-21swapper/020:58:060
12032820,9sleep00-21swapper/020:58:060
748328114,54sleep07572-21latency_hist21:18:060
399628013,51sleep04065-21df00:33:200
3030928020,39sleep030358-21latency_hist20:48:050
144942807,37sleep00-21swapper/023:23:210
144942807,37sleep00-21swapper/023:23:210
2002327910,16sleep20-21swapper/220:13:302
2002327910,16sleep20-21swapper/220:13:302
1244627923,27sleep00-21swapper/021:33:240
946627816,41sleep38759-21/usr/sbin/munin19:38:383
41527811,15sleep30-21swapper/300:23:053
390927818,44sleep33951-21awk22:48:353
829727727,31sleep10-21swapper/121:18:351
299432779,47sleep10-21swapper/100:13:181
874227622,25sleep00-21swapper/023:03:340
3196127514,26sleep00-21swapper/000:18:300
2799427529,27sleep30-21swapper/320:38:313
279002755,61sleep00-21swapper/000:05:090
279002755,61sleep00-21swapper/000:05:090
2498127516,39sleep125025-21munin-run23:58:031
1603227412,48sleep016099-21swap21:43:340
918427315,40sleep19228-21/usr/sbin/munin19:38:321
235562730,7sleep10-21swapper/123:53:061
235562730,7sleep10-21swapper/123:53:061
2016127320,16sleep30-21swapper/320:13:353
2016127320,16sleep30-21swapper/320:13:353
294012728,48sleep20-21swapper/220:43:312
2812727224,21sleep30-21swapper/300:08:033
2812727224,21sleep30-21swapper/300:08:033
826927115,41sleep18326-21grep23:03:231
346727116,41sleep01-21systemd00:33:040
2987427019,33sleep129908-21irqstats19:03:231
2440827019,26sleep00-21swapper/020:28:220
1285927015,35sleep112832-21/usr/sbin/munin23:18:171
1172627012,44sleep011738-21ntpq23:13:320
1162927012,42sleep111653-21cut19:48:051
105827031,21sleep30-21swapper/320:58:063
105827031,21sleep30-21swapper/320:58:063
3186926918,21sleep00-21swapper/020:53:060
3102526915,37sleep331087-21/usr/sbin/munin20:48:323
2386626911,40sleep123968-21/usr/sbin/munin22:08:351
1663626925,20sleep30-21swapper/323:28:333
1663626925,20sleep30-21swapper/323:28:333
102626913,38sleep11053-21awk00:23:281
3079326828,18sleep10-21swapper/122:33:061
30532687,12sleep10-21swapper/122:46:031
902826724,24sleep30-21swapper/321:23:053
408426723,25sleep00-21swapper/022:48:400
2807226718,22sleep00-21swapper/020:38:320
2574326710,12sleep00-21swapper/020:33:070
2563826716,25sleep30-21swapper/320:33:063
2372726715,36sleep323794-21df23:53:213
191826717,20sleep00-21swapper/000:28:030
132026718,21sleep00-21swapper/019:13:350
2572126623,23sleep00-21swapper/023:58:310
2234926618,27sleep10-21swapper/122:03:341
1361826621,21sleep30-21swapper/321:38:043
117592669,12sleep30-21swapper/321:29:033
47572658,41sleep14881-21timerwakeupswit19:23:541
47572658,41sleep14881-21timerwakeupswit19:23:541
3112626520,27sleep30-21swapper/322:33:243
3097326522,25sleep00-21swapper/019:08:060
1487026528,17sleep10-21swapper/123:23:301
1487026528,17sleep10-21swapper/123:23:301
3261726423,21sleep00-21swapper/019:13:070
2926226413,19sleep30-21swapper/320:43:283
245926420,23sleep30-21swapper/322:43:343
2278326418,28sleep122780-21apt-get20:23:181
1922626416,25sleep10-21swapper/121:53:371
1750726413,35sleep017508-21/usr/sbin/munin23:33:190
1664226413,28sleep10-21swapper/120:03:231
1664226413,28sleep10-21swapper/120:03:231
1566726416,21sleep30-21swapper/321:43:253
664426316,33sleep16720-21systemctl21:13:341
505326316,22sleep30-21swapper/322:53:233
2075026311,14sleep00-21swapper/021:58:490
1248426311,21sleep00-21swapper/019:48:360
1248426311,21sleep00-21swapper/019:48:360
839326218,26sleep38450-21irqstats23:03:253
671126222,24sleep30-21swapper/322:58:243
582226214,26sleep00-21swapper/019:28:250
582226214,26sleep00-21swapper/019:28:250
3193826220,25sleep30-21swapper/319:08:373
2903726214,26sleep30-21swapper/300:08:393
2903726214,26sleep30-21swapper/300:08:393
2063126220,21sleep00-21swapper/023:43:200
1811226213,34sleep318188-21/usr/sbin/munin23:33:343
1730326220,19sleep00-21swapper/021:48:270
1595826215,20sleep00-21swapper/023:28:180
1595826215,20sleep00-21swapper/023:28:180
1241226223,26sleep312481-21sensors_temp19:48:353
1241226223,26sleep312481-21sensors_temp19:48:353
1098726221,25sleep011010-21modprobe23:13:030
793926121,21sleep00-21swapper/023:03:060
260252616,17sleep10-21swapper/122:17:271
2039026129,15sleep10-21swapper/123:43:051
1992426119,18sleep00-21swapper/020:13:280
1992426119,18sleep00-21swapper/020:13:280
1556826120,18sleep10-21swapper/121:43:231
1342826114,28sleep013422-21apt-get19:53:210
1336326116,30sleep30-21swapper/323:18:323
944826019,23sleep00-21swapper/021:23:230
68882605,11sleep10-21swapper/119:32:421
68882605,11sleep10-21swapper/119:32:421
636826015,20sleep30-21swapper/319:28:363
636826015,20sleep30-21swapper/319:28:363
3138226016,24sleep10-21swapper/100:18:061
248426014,34sleep32575-21/usr/sbin/munin00:28:303
161592605,45sleep10-21swapper/120:01:381
702525916,24sleep17071-21grep22:58:301
303025916,26sleep33026-21apt-get21:03:193
2705925911,23sleep10-21swapper/100:03:241
2705925911,23sleep10-21swapper/100:03:241
222122596,42sleep00-21swapper/020:19:260
2121525913,25sleep121234-21apt-get20:18:181
2033925921,24sleep320364-21munin-run23:43:033
1256125921,18sleep10-21swapper/119:48:381
1256125921,18sleep10-21swapper/119:48:381
2545325822,20sleep30-21swapper/323:58:263
2480625815,26sleep024800-21apt-get22:13:180
2052625810,27sleep30-21swapper/321:58:313
200725815,29sleep12033-21munin-run19:18:041
349625711,23sleep10-21swapper/121:03:331
2793425715,25sleep127929-21apt-get22:23:171
1571825720,17sleep30-21swapper/323:28:053
1571825720,17sleep30-21swapper/323:28:053
1524825717,22sleep00-21swapper/021:43:060
147025717,23sleep31472-21apt-get20:58:183
1407225714,20sleep10-21swapper/119:53:381
1057725713,23sleep30-21swapper/321:28:063
238632565,39sleep30-21swapper/320:25:473
1858025616,24sleep318576-21apt-get21:53:183
995325515,22sleep110002-21irqstats23:08:281
406625516,17sleep10-21swapper/100:33:211
31022555,39sleep00-21swapper/022:46:380
3036125511,15sleep10-21swapper/120:48:061
2345125513,26sleep323513-21if_eth022:08:253
1947725510,18sleep00-21swapper/023:38:280
1090525523,16sleep00-21swapper/021:28:280
3054925415,22sleep130544-21apt-get20:48:181
2615325415,23sleep326271-21date22:18:063
2576925414,24sleep125774-21ntp_states23:58:321
2368725312,25sleep023710-21ntp_states22:08:300
1631925315,23sleep016389-21sed20:03:060
162925315,20sleep00-21swapper/020:58:310
91425215,21sleep3929-21irqstats00:23:253
720225212,24sleep37197-21apt-get19:33:173
419425214,23sleep04200-21diskstats19:23:290
419425214,23sleep04200-21diskstats19:23:290
2288425213,22sleep022992-21perl20:23:260
1818225213,24sleep318188-21diskstats20:08:223
1234425214,22sleep112356-21apt-get21:33:171
562625117,18sleep10-21swapper/122:53:381
505025112,25sleep05046-21apt-get22:53:200
3131825113,19sleep00-21swapper/000:18:050
2818225113,23sleep128322-21head20:38:331
2122925115,18sleep30-21swapper/323:43:363
316352509,15sleep10-21swapper/119:08:311
2646225014,18sleep10-21swapper/122:18:201
2131725012,23sleep321323-21diskstats20:18:243
1855625012,23sleep018574-21apt21:53:150
1725225013,21sleep317301-21irqstats21:48:263
206592497,21sleep10-21swapper/123:43:211
105162497,31sleep10-21swapper/121:27:561
4072468,14sleep00-21swapper/022:38:220
1363824511,18sleep30-21swapper/319:53:283
1401324410,20sleep314019-21diskstats21:38:203
242322438,12sleep00-21swapper/023:53:340
154482439,18sleep315459-21ntp_states19:58:333
1179024213,14sleep30-21swapper/323:13:353
292942416,18sleep10-21swapper/100:08:411
292942416,18sleep10-21swapper/100:08:411
2245024013,12sleep30-21swapper/323:48:283
2245024013,12sleep30-21swapper/323:48:283
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional