You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-24 - 17:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Fri Apr 24, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6980259914,566sleep37074-21/usr/sbin/munin08:21:183
11966232783,63sleep10-21swapper/110:20:481
11966232783,63sleep10-21swapper/110:20:481
3230229913,272sleep23306-21systemctl09:51:122
12626229414,239sleep112689-21latency10:21:331
12626229414,239sleep112689-21latency10:21:331
16759226979,146sleep116793-21latency_hist10:35:461
2471322617,215sleep124858-21latency_hist07:35:481
13655225634,203sleep213737-21fschecks_time07:01:042
1432622559,60sleep114360-21latency_hist12:10:461
28442225036,57sleep10-21swapper/107:46:471
28442225036,57sleep10-21swapper/107:46:471
17274224744,120sleep10-21swapper/110:36:071
25549223791,81sleep10-21swapper/107:36:171
25549223791,81sleep10-21swapper/107:36:171
1165022268,206sleep111664-21kernelversion08:36:191
11243222283,122sleep111319-21latency_hist08:35:501
24275221474,58sleep10-21swapper/109:16:051
5230220611,143sleep14953-21/usr/sbin/munin08:16:031
15242220317,131sleep10-21swapper/107:06:041
15242220317,131sleep10-21swapper/107:06:041
11790219521,157sleep211795-21modprobe10:20:452
11790219521,157sleep211795-21modprobe10:20:452
9128218922,149sleep29156-21/usr/sbin/munin10:11:032
26576218532,112sleep126730-21latency_hist11:05:481
2694021829,130sleep127015-21latency_hist09:25:471
8703217632,46sleep10-21swapper/110:10:461
2594217238,77sleep10-21swapper/108:06:181
2594217238,77sleep10-21swapper/108:06:181
7170216510,140sleep07204-21ps08:21:240
665216512,22sleep10-21swapper/111:26:031
9033216432,89sleep19126-21fschecks_time10:11:021
14308216410,140sleep214190-21latency_hist08:45:492
14308216410,140sleep214190-21latency_hist08:45:492
18424216325,31sleep30-21swapper/308:57:463
16137216312,114sleep116147-21find12:15:551
8801216222,39sleep10-21swapper/111:51:111
4095216254,93sleep14246-21latency_hist09:55:481
22336216124,66sleep116804-1kworker/1:1H07:26:151
23179215931,113sleep123327-21grep09:11:171
23179215931,113sleep123327-21grep09:11:171
2771421589,135sleep027154-21/usr/sbin/munin09:26:140
18076215616,81sleep10-21swapper/112:21:081
15730215620,65sleep115741-21modprobe08:50:451
7110215028,21sleep10-21swapper/110:05:451
24166214620,106sleep324198-21proc_pri10:56:123
19951214375,18sleep10-21swapper/109:01:151
1934521398,92sleep119469-21ls07:16:181
870213613,110sleep2435-21/usr/sbin/munin08:01:142
870213613,110sleep2435-21/usr/sbin/munin08:01:142
93521347,116sleep1972-21ps08:01:171
93521347,116sleep1972-21ps08:01:171
26780213172,22sleep112-21rcu_sched11:05:561
2807213011,83sleep12916-21cat11:31:161
13191212632,81sleep113233-21sort08:41:121
13191212632,81sleep113233-21sort08:41:121
4720212530,60sleep113441-21snmpd11:37:381
4720212530,60sleep113441-21snmpd11:37:381
20711212428,61sleep120771-21find07:21:141
10320212329,83sleep110363-21ntpq11:56:121
26538212213,88sleep326430-21/usr/sbin/munin07:41:033
21180212237,47sleep121215-21sshd12:31:101
18372212236,74sleep118394-21idleruntime-cro07:15:451
461221216,15sleep00-21swapper/008:12:590
461221216,15sleep00-21swapper/008:12:590
30474212114,64sleep130621-21unixbench_multi07:51:191
2946921216,100sleep10-21swapper/109:31:161
572021207,65sleep15785-21awk11:41:141
15619212017,85sleep215651-21/usr/sbin/munin07:06:132
15619212017,85sleep215651-21/usr/sbin/munin07:06:132
22095211920,78sleep321613-21apt-get10:50:553
5764211823,78sleep35846-21grep08:16:263
4300211819,39sleep10-21swapper/109:56:001
18784211720,77sleep218279-21apt-get10:40:562
8926211623,69sleep39076-21unixbench_multi08:26:193
3264421157,70sleep12-21kthreadd11:24:261
1734211346,53sleep11808-21switchtime09:46:141
2159721109,64sleep121644-21apt-config10:50:451
20648211019,71sleep30-21swapper/309:05:453
20602211018,72sleep00-21swapper/007:21:090
14902210928,66sleep115056-21date07:05:471
13028210913,80sleep013132-21wc10:21:440
13028210913,80sleep013132-21wc10:21:440
26949210836,53sleep10-21swapper/107:41:141
720721068,83sleep16669-21/usr/sbin/munin08:21:251
20898210620,66sleep020986-21tune2fs12:31:050
7039210523,13sleep10-21swapper/111:46:061
5051210526,41sleep30-21swapper/308:15:573
5051210526,41sleep30-21swapper/308:15:573
27447210511,47sleep10-21swapper/109:26:071
19632210518,63sleep10-21swapper/112:26:091
2077210212,74sleep02090-21/usr/sbin/munin09:46:170
32663210025,59sleep232749-21switchtime09:41:152
19194210014,71sleep319267-21mailstats07:16:163
1592629927,60sleep115927-21munin-run12:15:471
872229817,65sleep08784-21wc08:26:160
3268529840,42sleep10-21swapper/108:00:491
3268529840,42sleep10-21swapper/108:00:491
1768929826,45sleep217833-21grep10:36:162
1749829814,66sleep317532-21/usr/sbin/munin07:11:113
1499929818,51sleep10-21swapper/110:29:001
2525829717,39sleep00-21swapper/011:00:580
1890129718,67sleep10-21swapper/108:58:121
116822967,15sleep20-21swapper/208:36:202
1090529613,69sleep210906-21sshd11:57:102
1992829531,50sleep10-21swapper/110:44:111
283592948,72sleep128348-21apt-get11:11:011
3104529322,48sleep231074-21idleruntime-cro07:55:452
2832929331,32sleep00-21swapper/011:10:550
167142938,70sleep116860-21sed08:51:251
167142938,70sleep116860-21sed08:51:251
533429219,46sleep05402-21fschecks_time11:41:050
2304129217,56sleep323068-21basename07:30:453
249729114,57sleep22528-21/usr/sbin/munin11:31:092
2280729110,37sleep10-21swapper/110:51:181
585429021,43sleep05999-21sed08:16:270
209502908,39sleep10-21swapper/109:05:591
1262229019,50sleep312642-21awk08:40:453
2207028821,48sleep322096-21idleruntime-cro12:35:443
2085428813,60sleep320942-21mailstats10:46:153
958428734,34sleep10-21swapper/108:30:501
3080828611,61sleep330839-21grep09:36:073
1422128619,24sleep00-21swapper/008:45:470
1422128619,24sleep00-21swapper/008:45:470
412128526,23sleep14191-21ls08:11:191
412128526,23sleep14191-21ls08:11:191
2909528518,43sleep229231-21grep11:11:182
93128428,41sleep1940-21munin-run09:45:461
3123828430,38sleep10-21swapper/111:20:461
2225028421,31sleep20-21swapper/210:51:062
585528315,50sleep05904-21/usr/sbin/munin10:01:000
1563828319,48sleep215687-21if_eth010:31:072
233972817,54sleep30-21swapper/307:31:023
1578028114,52sleep315822-21latency_hist08:50:473
3195328014,18sleep30-21swapper/311:21:123
2084228010,20sleep30-21swapper/307:21:163
2991327914,31sleep30-21swapper/311:15:553
2899727918,43sleep029143-21sed07:46:580
2899727918,43sleep029143-21sed07:46:580
2682127917,28sleep00-21swapper/007:41:090
1592827915,48sleep215999-21cat12:15:482
968027816,41sleep09826-21sed11:55:470
812627714,46sleep38265-21awk10:06:163
2738627713,34sleep00-21swapper/011:06:150
2370327722,38sleep10-21swapper/110:55:581
2590027615,38sleep325601-21/usr/sbin/munin09:21:083
141542765,62sleep00-21swapper/012:09:080
1252327613,42sleep312291-21/usr/sbin/munin10:21:313
1252327613,42sleep312291-21/usr/sbin/munin10:21:313
1987727524,29sleep00-21swapper/009:01:140
785427417,30sleep20-21swapper/210:06:112
3157827412,46sleep231630-21/usr/sbin/munin07:56:092
3157827412,46sleep231630-21/usr/sbin/munin07:56:092
3022527420,32sleep20-21swapper/209:35:452
1194027421,13sleep10-21swapper/112:01:131
811527318,21sleep20-21swapper/208:25:502
552027315,36sleep35554-21wc11:41:083
40827319,28sleep00-21swapper/009:41:170
3053727320,13sleep10-21swapper/109:35:561
2961327312,46sleep329730-21sed09:31:183
2637327327,22sleep00-21swapper/009:21:170
2270927313,45sleep022776-21switchtime10:51:170
1019427316,31sleep30-21swapper/311:56:103
2466527216,34sleep00-21swapper/007:35:470
1656427217,37sleep30-21swapper/312:16:103
1186127215,37sleep011965-21dpkg10:20:470
1186127215,37sleep011965-21dpkg10:20:470
872227120,35sleep38761-21ntp_kernel_pll_11:51:093
2500827130,26sleep20-21swapper/207:36:062
2500827130,26sleep20-21swapper/207:36:062
335827011,30sleep30-21swapper/308:10:483
335827011,30sleep30-21swapper/308:10:483
3206027018,37sleep332068-21acpi09:40:553
2872727023,27sleep328461-21apt-get09:30:493
2529527014,27sleep30-21swapper/307:36:113
2529527014,27sleep30-21swapper/307:36:113
1407827014,26sleep00-21swapper/010:26:040
209442699,47sleep021022-21find10:46:160
810226820,29sleep00-21swapper/011:50:460
377426817,24sleep20-21swapper/211:36:022
377426817,24sleep20-21swapper/211:36:022
311712689,47sleep00-21swapper/011:20:250
247252687,52sleep00-21swapper/010:57:350
164226810,30sleep30-21swapper/309:46:123
787926724,21sleep00-21swapper/010:06:130
345426724,19sleep30-21swapper/311:35:463
3062726720,26sleep00-21swapper/009:36:040
1813126715,39sleep318169-21ntp_states12:21:103
1771226719,27sleep217709-21apt-get12:20:572
1747826723,19sleep00-21swapper/012:20:470
155522678,45sleep315592-21ntp_kernel_pll_07:06:103
155522678,45sleep315592-21ntp_kernel_pll_07:06:103
1387126718,31sleep30-21swapper/310:25:593
1338926715,31sleep313455-21awk08:41:183
1338926715,31sleep313455-21awk08:41:183
1112826718,25sleep30-21swapper/308:35:483
612526620,27sleep30-21swapper/310:01:073
578626611,27sleep20-21swapper/211:41:152
2943226619,28sleep00-21swapper/009:31:150
280926614,15sleep00-21swapper/009:51:020
23922658,14sleep00-21swapper/011:31:070
188726522,21sleep00-21swapper/011:30:470
1540426513,27sleep00-21swapper/010:30:570
1179726518,19sleep30-21swapper/310:20:463
1179726518,19sleep30-21swapper/310:20:463
496926421,26sleep24998-21apt-get08:15:562
496926421,26sleep24998-21apt-get08:15:562
2546726416,33sleep025547-21switchtime07:36:160
2546726416,33sleep025547-21switchtime07:36:160
2003326417,23sleep20-21swapper/209:01:162
1935826415,23sleep00-21swapper/012:26:040
3019526313,21sleep30-21swapper/307:51:133
2720426316,27sleep30-21swapper/311:06:103
2690726317,28sleep227010-21fschecks_time11:06:052
181126322,22sleep20-21swapper/209:46:152
2841326216,28sleep228430-21fschecks_count07:46:442
2841326216,28sleep228430-21fschecks_count07:46:442
2784326220,27sleep227992-21gpgv07:45:472
2534626211,33sleep225358-21/usr/sbin/munin11:01:042
2241826213,32sleep222535-21ls07:26:162
1940426221,21sleep30-21swapper/310:41:593
1573226218,26sleep00-21swapper/007:06:150
1573226218,26sleep00-21swapper/007:06:150
1278326219,26sleep20-21swapper/212:05:462
282702618,16sleep20-21swapper/209:28:382
2799426115,19sleep30-21swapper/307:45:483
2799426115,19sleep30-21swapper/307:45:483
251426111,21sleep00-21swapper/008:06:170
251426111,21sleep00-21swapper/008:06:170
2506326120,24sleep20-21swapper/211:00:482
2379126113,33sleep223826-21grep07:31:132
1923626116,29sleep319239-21/usr/sbin/munin12:25:553
1328926123,21sleep30-21swapper/312:06:083
498126014,24sleep00-21swapper/009:56:150
3157426018,23sleep20-21swapper/211:21:032
191322609,35sleep019143-21ntp_states07:16:130
1835926016,19sleep00-21swapper/008:57:420
1767826012,25sleep00-21swapper/007:11:160
172742607,12sleep00-21swapper/008:55:400
172742607,12sleep00-21swapper/008:55:400
1488226012,25sleep014901-21munin-run07:05:460
135526014,28sleep31483-21uptime11:26:173
104426019,20sleep30-21swapper/308:01:193
104426019,20sleep30-21swapper/308:01:193
1273225921,24sleep012741-21modprobe12:05:440
1059825913,19sleep20-21swapper/208:31:192
3148425816,25sleep031480-21apt-get11:20:570
268525816,20sleep20-21swapper/208:06:192
268525816,20sleep20-21swapper/208:06:192
2651925814,26sleep30-21swapper/311:05:453
2499325813,17sleep20-21swapper/209:16:192
1638825819,20sleep30-21swapper/308:51:113
1638825819,20sleep30-21swapper/308:51:113
147982587,10sleep30-21swapper/307:05:213
966425716,26sleep29741-21sed08:30:512
627025715,25sleep26306-21ntp_states10:01:102
37922576,13sleep00-21swapper/011:36:040
37922576,13sleep00-21swapper/011:36:040
2226025719,19sleep00-21swapper/009:10:470
2109625714,20sleep20-21swapper/209:06:052
299125610,32sleep30-21swapper/309:51:053
235662568,36sleep00-21swapper/009:13:130
235662568,36sleep00-21swapper/009:13:130
1879625611,19sleep00-21swapper/010:41:030
1191625614,18sleep00-21swapper/008:36:260
1021225620,21sleep310238-21munin-run10:15:453
560625515,23sleep35616-21munin-run10:00:463
480625515,17sleep20-21swapper/209:56:132
2407925512,25sleep324073-21apt-get09:15:593
2178025514,24sleep021776-21apt-get07:25:580
1710525519,19sleep20-21swapper/207:11:012
1707925513,25sleep00-21swapper/010:36:020
1454425513,16sleep20-21swapper/210:26:142
1005825512,16sleep00-21swapper/008:31:090
1600825416,22sleep30-21swapper/310:31:163
640325312,18sleep20-21swapper/208:20:452
2220925321,16sleep30-21swapper/309:10:463
2081025311,24sleep220806-21apt-get12:30:592
1823525316,22sleep218329-21memory08:57:402
1737325312,26sleep317417-21ntpq10:36:093
92525217,17sleep00-21swapper/008:01:160
92525217,17sleep00-21swapper/008:01:160
831725212,24sleep28336-21apt-get11:50:562
3136625214,22sleep031361-21apt-get07:55:590
3136625214,22sleep031361-21apt-get07:55:590
3021225213,24sleep229914-21/usr/sbin/munin11:16:072
2408125212,16sleep00-21swapper/009:16:000
2268525216,20sleep20-21swapper/209:11:052
2268525216,20sleep20-21swapper/209:11:052
2057425213,18sleep20-21swapper/212:30:472
1833425218,17sleep00-21swapper/012:21:150
1446025212,23sleep214454-21apt-get08:45:572
1436225215,20sleep20-21swapper/212:10:472
318392516,24sleep20-21swapper/209:40:462
3039725113,24sleep030473-21grep07:51:180
295582516,9sleep30-21swapper/311:14:543
2257325115,21sleep322579-21diskstats09:11:023
2257325115,21sleep322579-21diskstats09:11:023
169592515,34sleep20-21swapper/208:51:392
169592515,34sleep20-21swapper/208:51:392
1217225113,19sleep211807-21apt-get10:20:492
1217225113,19sleep211807-21apt-get10:20:492
679625016,19sleep30-21swapper/311:46:003
2041525012,23sleep220418-21df_inode10:46:012
1508825012,17sleep00-21swapper/012:11:140
725224911,21sleep27259-21ntp_states11:46:122
265782485,34sleep20-21swapper/209:21:482
169712488,14sleep20-21swapper/212:16:172
1123824711,15sleep00-21swapper/012:00:480
72824613,15sleep20-21swapper/211:26:042
3143624611,20sleep331442-21df_inode07:56:023
3143624611,20sleep331442-21df_inode07:56:023
2050324610,22sleep220591-21cat07:21:072
1905524613,18sleep20-21swapper/207:16:102
146242468,13sleep00-21swapper/008:46:150
153852458,19sleep315484-21users12:11:173
1191824512,19sleep311929-21ntp_states12:01:113
89472449,17sleep00-21swapper/010:10:570
413824410,19sleep34157-21ntp_states11:36:113
413824410,19sleep34157-21ntp_states11:36:113
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional