You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-02 - 22:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Thu Apr 02, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
29749238317,3808sleep029718-21apt-get09:37:310
215722377126,3728sleep316126-21kworker/u8:410:52:213
2774928246,805sleep327792-21unin-run11:12:033
2774928246,805sleep327792-21unin-run11:12:033
3262241130,340sleep33377-21cat09:57:043
3165722737,228sleep331703-21awk09:42:273
7172226762,147sleep37266-21fschecks_time10:07:223
21164224563,128sleep321295-21latency_hist10:52:053
21164224563,128sleep321295-21latency_hist10:52:053
19908223852,20sleep30-21swapper/309:07:053
6105223414,203sleep26122-21apt08:22:142
22833223437,139sleep322983-21latency_hist07:22:063
13393223451,82sleep30-21swapper/310:27:213
22957222941,174sleep322974-21modprobe09:17:033
29257222311,126sleep329235-21latency_hist07:47:053
29257222311,126sleep329235-21latency_hist07:47:053
16111222142,91sleep316148-21idleruntime-cro10:37:023
16111222142,91sleep316148-21idleruntime-cro10:37:023
1215022168,192sleep310490-21kworker/u8:208:42:053
18932214136,22sleep31783-21/usr/sbin/munin11:32:223
25889221232,85sleep325932-21awk07:32:243
25889221232,85sleep325932-21awk07:32:243
13031220977,86sleep313180-21unixbench_multi08:42:343
21878220219,120sleep30-21swapper/312:37:053
1884122019,137sleep318962-21timerwakeupswit10:42:363
6595219849,125sleep30-21swapper/308:22:313
12514219663,90sleep312598-21latency_hist12:07:053
3357218647,61sleep30-21swapper/311:37:143
3154721850,8sleep30-21swapper/307:57:063
13899217744,53sleep30-21swapper/308:47:183
13899217744,53sleep30-21swapper/308:47:183
9412216847,64sleep39498-21latency_hist11:57:043
15520216037,57sleep30-21swapper/308:52:233
24992215741,54sleep30-21swapper/309:22:243
24992215741,54sleep30-21swapper/309:22:243
983121569,134sleep39916-21timerandwakeup08:32:323
32508215383,52sleep30-21swapper/311:27:053
24660215312,119sleep324833-21dpkg11:02:063
29381214910,65sleep329522-21gpgv09:37:063
28602214946,66sleep328677-21switchtime11:12:323
5306214818,45sleep30-21swapper/311:42:273
5306214818,45sleep30-21swapper/311:42:273
30076214120,107sleep130084-21ntp_states11:17:301
4675213661,56sleep34559-21/usr/sbin/munin08:17:223
2761213243,70sleep30-21swapper/308:12:053
7787213160,54sleep37802-21apt-get11:52:043
12781212614,53sleep30-21swapper/310:24:023
31040212414,94sleep10-21swapper/109:42:041
17515212342,62sleep316995-21/usr/sbin/munin08:57:323
30509212216,86sleep230660-21head09:38:022
24829212226,24sleep30-21swapper/307:27:153
5260212115,85sleep35439-21apt-key10:02:053
22473211812,92sleep222486-21vmstat10:52:452
19686211819,81sleep20-21swapper/209:05:022
2192721146,34sleep30-21swapper/307:15:033
1706121145,33sleep30-21swapper/312:20:583
3810211312,39sleep13834-21ntp_states11:37:281
20298211347,15sleep30-21swapper/312:32:043
24909211027,70sleep324928-21apt-get11:02:143
21533211013,82sleep021050-21apt-get10:52:100
16685211016,77sleep216658-21apt-get10:37:302
14582211025,66sleep014715-21sed08:47:360
14582211025,66sleep014715-21sed08:47:360
4395210927,16sleep30-21swapper/309:58:063
7762210817,74sleep17787-21fschecks_count08:27:211
21303210816,76sleep021488-21apt-key10:52:060
3365210512,80sleep33406-21ntp_kernel_err08:12:253
27679210534,15sleep30-21swapper/309:32:053
11569210513,71sleep211578-21/usr/sbin/munin12:02:282
16693210446,44sleep316658-21apt-get10:37:353
2806221030,12sleep00-21swapper/009:32:250
18307210211,76sleep018399-21tune2fs10:42:250
8967210129,59sleep38985-21munin-run08:32:043
24280210119,63sleep30-21swapper/307:26:303
27139210014,67sleep027267-21/usr/sbin/munin09:27:340
2490929722,56sleep224991-21fschecks_time09:22:232
2490929722,56sleep224991-21fschecks_time09:22:232
2057729736,15sleep30-21swapper/309:07:303
567029620,55sleep25820-21unixbench_singl11:42:342
567029620,55sleep25820-21unixbench_singl11:42:342
1816129627,49sleep117657-21apt-get07:02:141
939529538,38sleep30-21swapper/310:12:353
939529538,38sleep30-21swapper/310:12:353
931129518,19sleep00-21swapper/008:32:210
512029418,25sleep20-21swapper/210:02:042
2557229316,62sleep025576-21diskstats07:32:160
2557229316,62sleep025576-21diskstats07:32:160
2298629318,56sleep00-21swapper/009:17:040
316852926,37sleep30-21swapper/307:57:153
15812929,68sleep31724-21latency_hist11:32:053
15812929,68sleep31724-21latency_hist11:32:053
1047729230,43sleep30-21swapper/310:17:243
1523029118,58sleep015279-21ntpq10:32:270
1523029118,58sleep015279-21ntpq10:32:270
1283529114,61sleep112926-21fschecks_time12:07:201
153642907,36sleep30-21swapper/312:14:243
1401229017,26sleep01-21systemd12:12:030
3164328912,64sleep231656-21netstat09:42:262
3008528913,61sleep230117-21grep11:17:312
237812897,64sleep323876-21cut09:17:313
2289528913,62sleep122905-21systemd-run10:57:031
317628834,31sleep13197-21idleruntime-cro09:57:021
308942875,35sleep30-21swapper/309:40:493
274592875,35sleep30-21swapper/307:39:303
2528328712,58sleep125434-21latency_hist07:32:051
2445728619,47sleep224574-21apt-key07:27:062
227428619,44sleep00-21swapper/011:32:310
2021828612,21sleep220224-21df_inode09:07:202
1022628620,11sleep30-21swapper/311:57:333
402128510,63sleep14022-21sshd08:14:011
390028521,40sleep03966-21cut09:57:540
88332847,65sleep08601-21/usr/sbin/munin10:12:240
88332847,65sleep08601-21/usr/sbin/munin10:12:240
434628411,59sleep24289-21latency_hist08:17:052
1564928426,27sleep10-21swapper/112:17:041
63828325,25sleep00-21swapper/009:47:240
124528322,30sleep10-21swapper/108:07:061
1095628313,55sleep210964-21systemd-run12:02:042
2340628211,58sleep223482-21fschecks_time09:17:222
2169128221,37sleep221685-21apt-get09:12:172
2445028125,31sleep10-21swapper/111:02:041
1766928130,33sleep30-21swapper/312:22:233
1748228117,42sleep217514-21/usr/sbin/munin08:57:312
1634728120,47sleep216359-21vmstat08:52:382
1144528113,31sleep00-21swapper/012:02:260
1048128124,36sleep110517-21munin-run08:37:031
314628017,29sleep20-21swapper/211:37:052
2625028016,48sleep126393-21cut11:07:051
893327924,27sleep20-21swapper/208:32:032
3143627922,33sleep00-21swapper/011:22:260
3212327812,16sleep10-21swapper/108:02:061
2395627815,42sleep124097-21sed09:17:331
989227719,34sleep29921-21grep10:17:022
989227719,34sleep29921-21grep10:17:022
633827713,48sleep06388-21if_eth008:22:240
2839827739,19sleep10-21swapper/111:12:251
180512777,15sleep20-21swapper/207:02:072
1175827612,50sleep011803-21grep10:22:170
2868427515,28sleep00-21swapper/011:12:330
1608427518,44sleep016206-21sed08:52:360
309272748,17sleep30-21swapper/311:22:043
2452027422,25sleep20-21swapper/211:02:052
338127321,22sleep00-21swapper/009:57:050
294692725,36sleep30-21swapper/307:47:143
2919427224,29sleep10-21swapper/109:37:041
2685527219,38sleep126941-21mailstats09:27:311
1096827219,27sleep00-21swapper/012:02:060
2862627114,43sleep028785-21unixbench_multi07:42:300
2862627114,43sleep028785-21unixbench_multi07:42:300
2017427111,24sleep31-21systemd07:12:043
1743327118,29sleep10-21swapper/112:22:171
1222527110,15sleep20-21swapper/210:22:322
999027021,28sleep10-21swapper/110:17:061
999027021,28sleep10-21swapper/110:17:061
90627011,16sleep30-21swapper/309:47:313
633227012,28sleep20-21swapper/210:02:492
593127018,24sleep20-21swapper/208:22:052
1798227021,24sleep10-21swapper/110:42:051
1350727016,22sleep20-21swapper/212:07:332
815226910,17sleep00-21swapper/011:52:210
548026913,37sleep04559-21/usr/sbin/munin08:17:350
1869226920,32sleep218712-21modprobe12:27:032
694326826,24sleep10-21swapper/111:47:291
624226822,23sleep10-21swapper/111:47:051
624226822,23sleep10-21swapper/111:47:051
348426816,34sleep03546-21sendmail_mailst08:12:300
2011626821,22sleep10-21swapper/112:30:211
1816026817,29sleep018154-21apt-get07:02:130
1523226826,19sleep20-21swapper/208:52:062
1523226826,19sleep20-21swapper/208:52:062
127472687,45sleep30-21swapper/312:07:133
108422685,50sleep10-21swapper/112:01:061
2304026714,27sleep10-21swapper/109:17:051
224972675,13sleep20-21swapper/207:18:252
1821326717,24sleep30-21swapper/307:02:523
1767926713,32sleep117803-21timerwakeupswit08:57:341
1318326717,29sleep00-21swapper/008:42:350
933126617,32sleep19388-21basename11:57:021
670926618,19sleep10-21swapper/108:22:331
535026610,13sleep00-21swapper/011:42:280
535026610,13sleep00-21swapper/011:42:280
376026611,34sleep13723-21apt-get09:57:351
2704726613,23sleep10-21swapper/107:37:061
2704726613,23sleep10-21swapper/107:37:061
2561026617,34sleep225731-21cat11:02:542
1631126618,21sleep20-21swapper/210:37:052
1631126618,21sleep20-21swapper/210:37:052
3116926514,36sleep231236-21df11:22:192
2930226518,31sleep21-21systemd11:17:032
2128026519,20sleep00-21swapper/012:32:340
2108126520,23sleep10-21swapper/107:12:231
210652658,14sleep10-21swapper/109:07:401
2050726518,28sleep120648-21grep10:47:351
2050726518,28sleep120648-21grep10:47:351
365326420,19sleep20-21swapper/211:37:252
266242646,12sleep20-21swapper/211:07:222
266242646,12sleep20-21swapper/211:07:222
2545226417,17sleep00-21swapper/011:02:520
2107626416,23sleep20-21swapper/210:52:042
2107626416,23sleep20-21swapper/210:52:042
2060826414,25sleep20-21swapper/212:32:192
1731526422,20sleep00-21swapper/010:38:000
120562649,16sleep10-21swapper/108:42:031
44926317,32sleep1520-21cut09:47:211
2716826318,30sleep127305-21unixbench_multi11:07:331
2716826318,30sleep127305-21unixbench_multi11:07:331
1502326316,22sleep00-21swapper/012:12:310
1379326322,21sleep10-21swapper/110:27:311
133912637,41sleep20-21swapper/210:27:202
11452639,44sleep30-21swapper/308:06:523
1089226317,30sleep011049-21sed10:17:330
86362629,41sleep00-21swapper/008:28:090
3136226216,33sleep131403-21uniq09:42:211
287862620,9sleep10-21swapper/107:42:301
287862620,9sleep10-21swapper/107:42:301
2352526214,32sleep123545-21/usr/sbin/munin07:22:281
1986326221,21sleep00-21swapper/010:47:210
1146226222,25sleep00-21swapper/008:37:340
785526121,21sleep00-21swapper/011:52:050
765226119,20sleep10-21swapper/110:07:331
2019626112,30sleep020287-21sort07:12:060
1838826116,24sleep00-21swapper/009:02:050
55322608,36sleep00-21swapper/010:02:190
3154226020,22sleep00-21swapper/009:42:240
2989726011,32sleep229970-21cat07:52:052
2984526019,24sleep00-21swapper/007:52:030
2585826017,22sleep20-21swapper/207:32:232
2585826017,22sleep20-21swapper/207:32:232
2524826016,18sleep00-21swapper/007:32:040
2333626023,18sleep00-21swapper/007:22:230
2053226015,28sleep220197-21munin-node07:12:082
2012126013,32sleep020129-21/usr/sbin/munin09:07:140
1941426015,24sleep219439-21ntp_states12:27:312
1845626016,28sleep218490-21irqstats10:42:272
1588726013,29sleep20-21swapper/212:17:182
1581226020,20sleep10-21swapper/108:52:291
233625919,19sleep20-21swapper/208:07:362
155925916,26sleep10-21swapper/111:32:031
155925916,26sleep10-21swapper/111:32:031
1520025919,19sleep10-21swapper/108:52:051
1520025919,19sleep10-21swapper/108:52:051
3235225816,26sleep032346-21apt-get08:02:130
3116825815,26sleep131161-21apt-get11:22:181
206725813,21sleep00-21swapper/008:07:340
1366225812,27sleep20-21swapper/208:47:052
712925714,20sleep00-21swapper/010:07:190
3139325723,18sleep00-21swapper/007:57:040
2728125710,23sleep20-21swapper/209:27:362
2607925714,21sleep10-21swapper/109:27:021
2607925714,21sleep10-21swapper/109:27:021
1103525715,18sleep20-21swapper/208:37:242
560025615,24sleep15606-21df_inode10:02:201
386725617,20sleep00-21swapper/011:37:310
647525516,23sleep26472-21apt-get11:47:142
1961625518,16sleep10-21swapper/107:07:061
1719825516,17sleep10-21swapper/112:22:051
1621925512,23sleep10-21swapper/110:37:041
1621925512,23sleep10-21swapper/110:37:041
1596225515,21sleep00-21swapper/012:17:200
142725513,29sleep11448-21dpkg08:07:141
3129725415,24sleep131298-21sshd07:55:521
2962125414,25sleep029614-21apt-get11:17:170
2327525413,23sleep023269-21apt-get09:17:160
224272545,38sleep00-21swapper/007:17:310
1105025414,23sleep111111-21users10:17:341
939025312,25sleep29410-21munin-run11:57:032
85725310,29sleep2936-21find11:27:322
85725310,29sleep2936-21find11:27:322
249412535,12sleep20-21swapper/207:28:392
2482125314,24sleep024816-21apt-get09:22:170
2482125314,24sleep024816-21apt-get09:22:170
2088325312,23sleep020892-21df_inode07:12:180
1994825315,17sleep00-21swapper/009:07:060
1874425313,18sleep00-21swapper/012:27:050
1255725314,24sleep112614-21irqstats08:42:241
1020725313,23sleep210203-21apt-get10:17:152
929525214,23sleep29301-21df_inode08:32:192
707825213,22sleep27072-21apt-get10:07:172
1858325213,21sleep118650-21df09:02:231
1450725212,23sleep214565-21irqstats12:12:212
12962525,14sleep20-21swapper/209:47:432
3237325113,23sleep232424-21df08:02:152
2924725112,22sleep229256-21munin-run07:47:042
2924725112,22sleep229256-21munin-run07:47:042
275242518,34sleep00-21swapper/007:40:200
2709025117,15sleep00-21swapper/011:07:320
2709025117,15sleep00-21swapper/011:07:320
2441125116,19sleep00-21swapper/011:02:030
589025015,16sleep10-21swapper/108:22:041
289102505,33sleep20-21swapper/209:33:372
180625011,23sleep11803-21apt-get11:32:161
37524911,22sleep0461-21fw_packets11:27:220
37524911,22sleep0461-21fw_packets11:27:220
2053224912,22sleep120529-21apt-get12:32:141
1456524916,20sleep014571-21modprobe10:32:030
2727524813,20sleep126772-21apt-get07:37:101
1895524813,18sleep00-21swapper/012:27:130
1221824812,18sleep10-21swapper/110:22:311
320524715,17sleep20-21swapper/209:57:032
3058424713,19sleep230590-21ntp_states07:52:252
2765724713,18sleep20-21swapper/207:42:042
1821824510,20sleep10-21swapper/110:42:171
320312445,29sleep20-21swapper/208:01:342
199542445,27sleep20-21swapper/207:09:452
227792427,18sleep10-21swapper/107:22:041
1120724110,15sleep10-21swapper/112:02:191
3199924013,15sleep00-21swapper/008:01:080
298902357,15sleep10-21swapper/107:52:041
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional