You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-10 - 18:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Fri Jul 10, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
222528317,811sleep22233-21systemd-run10:03:592
222528317,811sleep22233-21systemd-run10:03:592
1485227947,775sleep214863-21ntpq08:59:262
7210278519,748sleep37227-21/usr/sbin/munin10:19:153
7210278519,748sleep37227-21/usr/sbin/munin10:19:153
30002255931,504sleep230062-21sshd08:04:232
13745232821,291sleep313773-21/usr/sbin/munin10:39:223
29206232431,180sleep20-21swapper/211:29:242
31891231421,272sleep331965-21latency_hist11:39:023
3616230227,213sleep23695-21fschecks_time08:24:212
21720229895,53sleep20-21swapper/207:39:152
14309226813,212sleep214435-21timerwakeupswit07:14:322
17606225944,194sleep217668-21fschecks_time09:09:202
16879225312,224sleep116926-21ntpq10:49:251
55212250135,100sleep20-21swapper/208:29:302
55212250135,100sleep20-21swapper/208:29:302
146892248104,118sleep20-21swapper/210:44:012
21401224593,61sleep221431-21munin-run07:38:592
15503223554,81sleep215556-21cpuspeed07:19:162
18295220839,156sleep217997-21/usr/sbin/munin10:54:232
18967220016,62sleep20-21swapper/207:29:262
6063219724,156sleep26064-21cpu11:59:122
6063219724,156sleep26064-21cpu11:59:122
28419219326,53sleep20-21swapper/207:59:222
11815219114,163sleep011881-21fschecks_time07:09:210
12986218820,153sleep21-21systemd07:13:592
1694721879,84sleep216948-21cpu12:34:132
10329218410,158sleep010400-21find08:44:300
10329218410,158sleep010400-21find08:44:300
27101217923,61sleep20-21swapper/209:39:202
26364217711,51sleep20-21swapper/207:54:012
3257217214,146sleep23327-21switchtime11:49:312
32537216786,63sleep21-21systemd08:14:002
1645321669,71sleep216466-21apt-get10:49:102
8262216120,28sleep28317-21fschecks_count08:39:202
8262216120,28sleep28317-21fschecks_count08:39:202
988216086,25sleep20-21swapper/211:44:022
3777215337,102sleep23814-21latency_hist10:09:002
24148215318,114sleep20-21swapper/207:44:562
23101215213,126sleep123120-21ntp_kernel_pll_11:09:241
13856214814,59sleep20-21swapper/212:24:162
2843621428,53sleep228444-21acpi09:44:092
1656221398,120sleep0364-21snmpd12:32:460
9532213824,102sleep29568-21ntpq12:09:222
9532213824,102sleep29568-21ntpq12:09:222
11178213864,28sleep20-21swapper/212:14:272
11178213864,28sleep20-21swapper/212:14:272
13061213721,94sleep013099-21idleruntime-cro10:38:580
5827213416,102sleep35973-21latency_hist11:59:013
23386213316,25sleep00-21swapper/011:09:310
7248213032,84sleep27354-21tune2fs10:19:182
7248213032,84sleep27354-21tune2fs10:19:182
2105621288,17sleep20-21swapper/209:19:252
13775212825,82sleep013811-21ntp_kernel_pll_10:39:230
31166212720,83sleep131320-21grep11:34:321
10248212710,104sleep00-21swapper/010:29:130
14046212514,91sleep314075-21/usr/sbin/munin07:14:253
4501212434,71sleep04477-21apt-get11:54:260
31369212426,77sleep031434-21uniq08:09:210
16586212414,89sleep116702-21/usr/sbin/munin09:04:301
25292212218,85sleep20-21swapper/209:34:152
17139212010,93sleep217216-21tune2fs07:24:212
17158211925,71sleep317301-21grep10:49:313
32400211829,14sleep20-21swapper/209:54:312
29918211814,25sleep230109-21apt-key09:49:012
26413211713,18sleep20-21swapper/211:19:312
26413211713,18sleep20-21swapper/211:19:312
21767211713,21sleep20-21swapper/211:04:282
11881211522,15sleep20-21swapper/210:34:162
22961211416,24sleep222981-21kernelversion11:09:212
2849211317,84sleep02917-21idleruntime11:49:210
1702621136,43sleep20-21swapper/209:06:332
6835211225,66sleep36854-21sh10:18:593
22955211211,17sleep20-21swapper/209:24:342
19315211224,14sleep20-21swapper/209:14:212
19315211224,14sleep20-21swapper/209:14:212
18829211217,75sleep318958-21latency_hist09:14:013
9957211114,84sleep19972-21idleruntime-cro10:28:581
25514211176,23sleep225517-21cron11:18:582
12572211117,74sleep312618-21latency12:19:203
13085211024,16sleep20-21swapper/208:54:222
23585210717,51sleep123591-21diskstats07:44:191
21894210718,70sleep322038-21sed11:04:293
12611210776,17sleep212762-21latency_hist08:54:012
2770621069,17sleep20-21swapper/211:24:232
2770621069,17sleep20-21swapper/211:24:232
19866210623,52sleep20-21swapper/210:59:202
19534210618,72sleep119593-21processes09:14:281
19534210618,72sleep119593-21processes09:14:281
8680210518,24sleep00-21swapper/008:39:300
8680210518,24sleep00-21swapper/008:39:300
7668210514,83sleep00-21swapper/008:36:350
7668210514,83sleep00-21swapper/008:36:350
7618210510,16sleep20-21swapper/212:04:132
7618210510,16sleep20-21swapper/212:04:132
1103021058,16sleep20-21swapper/208:49:012
1103021058,16sleep20-21swapper/208:49:012
2455221048,17sleep20-21swapper/209:29:312
476121039,76sleep04809-21systemd-run08:29:000
409221038,15sleep20-21swapper/210:09:162
3266210310,17sleep20-21swapper/208:24:012
12302210310,15sleep20-21swapper/212:19:112
30852210210,14sleep20-21swapper/211:34:252
2227210115,70sleep02311-21sed08:19:230
16658210130,48sleep316671-21idleruntime-cro12:33:583
13564210026,49sleep013581-21modprobe12:23:570
892529910,74sleep08972-21latency_hist12:08:590
892529910,74sleep08972-21latency_hist12:08:590
106642998,15sleep20-21swapper/210:29:232
1768729815,57sleep30-21swapper/312:34:313
149002987,17sleep214919-21apt-get10:44:092
2311029722,55sleep023216-21sed07:44:010
1903929723,61sleep319041-21acpi09:14:093
1903929723,61sleep319041-21acpi09:14:093
493929513,68sleep14977-21ntp_kernel_pll_11:55:151
2163929412,65sleep021697-21sendmail_mailst11:04:260
1990929419,16sleep10-21swapper/107:34:011
1494429417,59sleep1225-21systemd-journal07:18:591
2509029211,66sleep025239-21sed09:34:010
2219529210,39sleep10-21swapper/109:24:131
168552917,12sleep30-21swapper/309:04:323
6262890,14sleep3628-21modprobe09:58:593
56012896,11sleep20-21swapper/211:57:062
291828910,16sleep10-21swapper/111:49:221
187728811,19sleep20-21swapper/211:44:422
96262879,58sleep10-21swapper/112:09:251
96262879,58sleep10-21swapper/112:09:251
65292875,29sleep20-21swapper/210:15:102
3025228714,37sleep30-21swapper/309:49:143
1827928712,62sleep018312-21latency_hist12:39:000
97402865,10sleep20-21swapper/210:26:122
3230728614,54sleep132399-21mailstats09:54:291
275682865,28sleep20-21swapper/207:55:142
1303528620,32sleep30-21swapper/307:14:003
75652857,10sleep20-21swapper/208:35:162
75652857,10sleep20-21swapper/208:35:162
1416528514,18sleep10-21swapper/108:59:011
79128415,17sleep00-21swapper/008:14:260
659628423,37sleep36593-21apt-get08:34:123
659628423,37sleep36593-21apt-get08:34:123
164182845,11sleep20-21swapper/212:30:542
1561128418,44sleep015753-21unixbench_multi10:44:320
798428312,54sleep38010-21netstat12:04:223
798428312,54sleep38010-21netstat12:04:223
1046228313,57sleep3278-21dbus-daemon12:13:593
1046228313,57sleep3278-21dbus-daemon12:13:593
532528223,23sleep00-21swapper/008:29:240
532528223,23sleep00-21swapper/008:29:240
13452825,10sleep20-21swapper/208:15:212
3108928111,30sleep00-21swapper/011:34:310
2817428122,27sleep00-21swapper/009:43:580
1198628120,44sleep112019-21awk12:18:581
1198628120,44sleep112019-21awk12:18:581
359028023,39sleep13608-21fschecks_count08:24:201
2908028023,28sleep30-21swapper/307:59:353
2822728022,26sleep30-21swapper/309:44:013
2591828015,28sleep00-21swapper/011:19:160
2591828015,28sleep00-21swapper/011:19:160
645327913,50sleep06491-21sshd11:59:220
645327913,50sleep06491-21sshd11:59:220
3165327917,27sleep10-21swapper/108:09:261
2596827917,33sleep10-21swapper/109:34:311
1147127920,36sleep00-21swapper/007:09:000
1147127920,36sleep00-21swapper/007:09:000
1067527923,37sleep310746-21/usr/sbin/munin07:04:283
1067527923,37sleep310746-21/usr/sbin/munin07:04:283
369727816,42sleep33481-21/usr/sbin/munin08:24:223
2493427820,37sleep00-21swapper/011:14:320
2425427813,47sleep324399-21head07:44:573
15722779,17sleep00-21swapper/011:44:360
2981927516,26sleep30-21swapper/308:04:203
2537127512,49sleep025375-21diskstats09:34:170
2022727516,26sleep10-21swapper/107:34:171
1831327516,42sleep118225-21latency_hist12:39:011
1000827522,28sleep30-21swapper/310:29:003
685927412,40sleep06914-21munin-run10:19:000
2321827422,25sleep10-21swapper/107:44:021
1099027422,24sleep20-21swapper/207:04:312
1099027422,24sleep20-21swapper/207:04:312
2734827321,28sleep10-21swapper/109:39:271
2721427313,29sleep00-21swapper/007:54:310
125227320,30sleep10-21swapper/111:44:291
779527215,39sleep17918-21sed10:19:291
779527215,39sleep17918-21sed10:19:291
569727212,45sleep15796-21timerwakeupswit08:29:331
569727212,45sleep15796-21timerwakeupswit08:29:331
3274627218,24sleep00-21swapper/011:39:400
2931427210,36sleep00-21swapper/009:44:310
262427216,39sleep32664-21date11:49:033
3185727114,38sleep331936-21sort08:09:333
197032710,10sleep012-21rcu_sched09:14:300
197032710,10sleep012-21rcu_sched09:14:300
2738327016,41sleep027391-21cpu11:24:130
2738327016,41sleep027391-21cpu11:24:130
1811827022,29sleep318228-21cut10:54:203
1440427015,22sleep30-21swapper/308:59:153
1367527010,44sleep013670-21apt-get07:14:130
1267127013,44sleep112703-21sshd12:19:221
2893226924,25sleep00-21swapper/011:29:140
2574126812,17sleep00-21swapper/007:49:390
2051326814,34sleep320548-21ntp_kernel_err07:34:233
188602688,43sleep00-21swapper/007:29:240
171926811,16sleep30-21swapper/308:19:013
2972026717,33sleep329737-21awk09:48:583
1880726723,26sleep30-21swapper/307:29:233
2424826615,28sleep324327-21mailstats09:29:283
738126512,38sleep17465-21latency_hist12:04:001
738126512,38sleep17465-21latency_hist12:04:001
2765326517,25sleep10-21swapper/111:24:221
2765326517,25sleep10-21swapper/111:24:221
186426515,37sleep1225-21systemd-journal09:59:591
186426515,37sleep1225-21systemd-journal09:59:591
1547026519,24sleep00-21swapper/007:19:100
1408326516,30sleep314151-21modprobe08:58:593
1177626517,25sleep10-21swapper/108:49:281
2190726414,34sleep3278-21dbus-daemon09:23:593
2073726420,19sleep00-21swapper/009:19:190
204326420,30sleep10-21swapper/108:19:201
1577826414,34sleep115832-21sed07:19:271
147742646,39sleep00-21swapper/008:59:240
977026312,24sleep00-21swapper/012:09:280
977026312,24sleep00-21swapper/012:09:280
691526318,23sleep10-21swapper/110:19:011
2886326315,28sleep10-21swapper/107:59:331
1830726311,37sleep018354-21systemd-run07:28:590
1175326317,23sleep10-21swapper/107:09:181
817626216,22sleep10-21swapper/108:39:141
817626216,22sleep10-21swapper/108:39:141
439026221,24sleep10-21swapper/110:09:241
268962625,12sleep10-21swapper/111:21:271
268962625,12sleep10-21swapper/111:21:271
319826115,21sleep00-21swapper/010:04:300
2528826121,24sleep325284-21apt-get09:34:123
149326114,32sleep31552-21sed09:59:283
149326114,32sleep31552-21sed09:59:283
732926016,28sleep37344-21modprobe12:03:583
732926016,28sleep37344-21modprobe12:03:583
231192607,42sleep00-21swapper/009:24:400
2301926013,33sleep323108-21gpgv07:44:003
2021726018,23sleep00-21swapper/007:34:160
179712605,11sleep10-21swapper/107:24:401
1501226014,25sleep30-21swapper/310:44:183
1220526014,25sleep10-21swapper/110:34:231
1188826017,28sleep311969-21grep10:34:173
953025915,19sleep30-21swapper/308:44:013
953025915,19sleep30-21swapper/308:44:013
291002595,12sleep00-21swapper/007:59:400
2374325920,22sleep10-21swapper/109:29:141
1850225911,33sleep018560-21/usr/sbin/munin10:54:290
1694425917,23sleep00-21swapper/010:49:280
1568325917,28sleep315719-21idleruntime07:19:253
65992587,36sleep10-21swapper/108:34:141
65992587,36sleep10-21swapper/108:34:141
464725819,21sleep00-21swapper/010:09:310
381525815,20sleep30-21swapper/310:09:013
2980725811,31sleep129813-21df_inode08:04:181
2195825821,23sleep10-21swapper/109:24:011
1959425815,20sleep30-21swapper/310:59:123
1721925818,19sleep30-21swapper/307:24:223
1399325815,20sleep00-21swapper/012:24:220
878625718,17sleep00-21swapper/010:24:170
45225715,24sleep1462-21/usr/sbin/munin08:14:191
2905625717,22sleep30-21swapper/311:29:213
2553225715,15sleep10-21swapper/107:49:361
2309925715,24sleep30-21swapper/309:24:353
115082576,37sleep10-21swapper/110:33:591
787525617,23sleep17888-21irqstats12:04:201
787525617,23sleep17888-21irqstats12:04:201
5802566,11sleep30-21swapper/311:40:273
402125617,24sleep34016-21apt-get10:09:133
3156125616,22sleep10-21swapper/109:54:011
2731625612,28sleep127423-21awk07:54:321
1651325620,18sleep00-21swapper/009:04:290
1394225618,20sleep30-21swapper/312:24:213
1126725615,24sleep311260-21apt-get08:49:143
897925514,24sleep39077-21users08:39:333
897925514,24sleep39077-21users08:39:333
3104325517,18sleep00-21swapper/008:09:010
2796425513,26sleep30-21swapper/311:24:323
2796425513,26sleep30-21swapper/311:24:323
2583225514,25sleep325828-21apt-get11:19:113
2583225514,25sleep325828-21apt-get11:19:113
67725412,20sleep00-21swapper/009:59:010
2424725412,25sleep124241-21apt-get11:14:131
155525412,18sleep00-21swapper/009:59:290
155525412,18sleep00-21swapper/009:59:290
1041325413,17sleep10-21swapper/107:04:221
1041325413,17sleep10-21swapper/107:04:221
3023025316,22sleep130215-21/usr/sbin/munin11:33:591
247825315,17sleep10-21swapper/110:04:151
2315725312,23sleep323163-21ntp_states11:09:263
1282025314,23sleep012818-21apt-get08:54:110
529225213,23sleep35323-21irqstats08:29:233
529225213,23sleep35323-21irqstats08:29:233
2115025212,23sleep121145-21apt-get11:04:111
1826525213,23sleep118294-21irqstats10:54:221
1138925211,23sleep011478-21fw_packets08:49:190
1024625212,23sleep310241-21apt-get10:29:123
758325113,16sleep00-21swapper/010:19:240
758325113,16sleep00-21swapper/010:19:240
1206825113,23sleep012193-21grep12:19:010
1206825113,23sleep012193-21grep12:19:010
2564625013,17sleep30-21swapper/307:49:383
1568025011,23sleep115763-21memory12:29:201
1070825014,20sleep310705-21apt-get12:14:113
1070825014,20sleep310705-21apt-get12:14:113
7832495,33sleep10-21swapper/111:43:001
2899924913,19sleep10-21swapper/109:44:281
1877724914,20sleep118809-21munin-run09:13:591
175932497,16sleep00-21swapper/009:09:180
1440224914,20sleep114397-21apt-get08:59:131
143282495,33sleep10-21swapper/110:40:001
970824811,21sleep19715-21apt-get08:44:111
970824811,21sleep19715-21apt-get08:44:111
2497624811,17sleep00-21swapper/007:49:220
219224814,15sleep30-21swapper/308:19:223
1406624816,16sleep10-21swapper/112:24:231
3025424713,16sleep00-21swapper/009:49:150
2701624711,20sleep327058-21ntp_states07:54:263
1959224712,21sleep119587-21apt-get10:59:101
2485624612,18sleep30-21swapper/311:14:313
1980124613,16sleep00-21swapper/010:59:180
124024610,12sleep30-21swapper/311:44:273
149252398,17sleep10-21swapper/110:44:121
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional