You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-26 - 23:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sun Apr 26, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
8251298823,917sleep231214-21kworker/u8:311:50:492
20920256612,539sleep120921-21sshd10:48:571
20920256612,539sleep120921-21sshd10:48:571
7715255210,526sleep27821-21ls10:06:092
173642294127,150sleep217391-21awk12:20:372
15024229336,214sleep215127-21timerandwakeup08:46:112
22286228629,216sleep222352-21fschecks_time09:10:592
4768226787,166sleep24785-21kworker/u8:108:15:382
10419226317,119sleep20-21swapper/210:15:582
11618225764,72sleep20-21swapper/212:00:592
6893224034,189sleep27033-21latency_hist10:05:402
29818222036,56sleep20-21swapper/207:50:582
7007221278,25sleep20-21swapper/208:21:072
7007221278,25sleep20-21swapper/208:21:072
657822028,133sleep215670-21sshd11:45:572
657822028,133sleep215670-21sshd11:45:572
29441220210,144sleep229459-21modprobe07:50:382
22982219912,74sleep223000-21cron07:30:382
25411219836,121sleep225468-21fschecks_count09:20:572
27354219315,114sleep227420-21sendmail_mailqu09:26:062
1811921908,172sleep218153-21ps12:21:062
24126218829,121sleep224127-21sshd10:59:572
24126218829,121sleep224127-21sshd10:59:572
418021879,13sleep00-21swapper/011:36:090
31228218612,112sleep20-21swapper/211:21:252
19034218340,130sleep219081-21ntp_offset07:16:042
19034218340,130sleep219081-21ntp_offset07:16:042
20413217616,143sleep020384-21latency_hist09:05:400
29156217439,23sleep20-21swapper/211:15:402
2133217418,78sleep22159-21latency_hist09:50:382
2133217418,78sleep22159-21latency_hist09:50:382
19699217232,124sleep219796-21timerandwakeup09:01:102
5847217025,108sleep25860-21awk11:41:122
17172217017,139sleep2758-21sendmail-mta08:55:042
2220216812,144sleep22230-21kernelversion08:06:012
17621216824,123sleep117686-21fschecks_time08:55:561
24152216649,79sleep20-21swapper/209:16:042
824021656,13sleep30-21swapper/311:50:423
30371215555,47sleep20-21swapper/209:36:042
30371215555,47sleep20-21swapper/209:36:042
279221517,26sleep20-21swapper/209:51:022
16104214912,124sleep216153-21apt-config07:10:392
1075214931,79sleep21188-21timerwakeupswit08:01:122
9553214815,48sleep20-21swapper/208:30:402
25569214717,115sleep125591-21/usr/sbin/munin07:36:231
27327214611,65sleep20-21swapper/211:10:392
19507214522,95sleep119540-21cut10:45:381
15669214511,23sleep10-21swapper/108:50:391
17998214411,76sleep20-21swapper/208:56:052
4704214310,118sleep20-21swapper/211:40:402
1139121425,11sleep30-21swapper/308:35:593
2572821399,118sleep225776-21systemd-run11:05:382
2572821399,118sleep225776-21systemd-run11:05:382
24588213850,21sleep20-21swapper/207:35:392
15788213413,106sleep315856-21sh12:15:383
15788213413,106sleep315856-21sh12:15:383
31581213112,88sleep10-21swapper/111:22:171
32616213011,68sleep20-21swapper/208:00:392
21907212919,48sleep121919-21sh09:10:381
306221277,109sleep213441-21snmpd11:35:232
306221277,109sleep213441-21snmpd11:35:232
22568212414,92sleep322613-21awk09:11:043
16804212332,50sleep216926-21munin-node07:10:472
16804212332,50sleep216926-21munin-node07:10:472
23026212216,86sleep023046-21latency_hist07:30:400
13337211810,73sleep213380-21ntpq12:06:042
27948211713,66sleep227922-21latency_hist07:45:412
27948211713,66sleep227922-21latency_hist07:45:412
965211421,67sleep31026-21fschecks_time09:45:583
965211421,67sleep31026-21fschecks_time09:45:583
6718211413,43sleep00-21swapper/008:21:010
6718211413,43sleep00-21swapper/008:21:010
2055021125,39sleep20-21swapper/207:21:052
18126211123,45sleep20-21swapper/210:40:422
13884211013,79sleep313952-21wc10:26:093
13884211013,79sleep313952-21wc10:26:093
864821099,58sleep20-21swapper/210:10:532
28458210813,80sleep328547-21load07:46:023
20927210810,35sleep20-21swapper/209:06:032
4414210715,52sleep20-21swapper/209:56:032
29906210710,39sleep229974-21mailstats11:16:082
17003210712,78sleep317069-21/usr/sbin/munin10:36:073
13698210721,69sleep10-21swapper/110:26:031
13698210721,69sleep10-21swapper/110:26:031
8399210612,78sleep08496-21fschecks_time11:51:040
14589210615,30sleep00-21swapper/007:05:400
2851621048,32sleep20-21swapper/209:30:592
2163210413,23sleep10-21swapper/108:06:001
23382210313,74sleep023414-21/usr/sbin/munin10:56:070
23382210313,74sleep023414-21/usr/sbin/munin10:56:070
10261210216,71sleep010303-21ntp_kernel_pll_11:56:040
30988210122,61sleep131020-21kworker/u8:107:55:381
16342210124,58sleep116539-21ls07:10:421
1138521018,81sleep211389-21diskstats08:35:582
3215329923,40sleep232171-21memory09:41:172
1187729915,58sleep20-21swapper/210:20:592
1187729915,58sleep20-21swapper/210:20:592
919429813,69sleep19258-21swap10:11:081
2514629817,65sleep125265-21timerwakeupswit11:01:091
2514629817,65sleep125265-21timerwakeupswit11:01:091
1935829836,43sleep20-21swapper/212:25:582
2212429712,69sleep322274-21latency_hist12:35:403
1309029716,64sleep313200-21cat10:25:403
1309029716,64sleep313200-21cat10:25:403
2072329613,69sleep020731-21acpi12:30:490
157772967,80sleep213441-21snmpd12:15:282
157772967,80sleep213441-21snmpd12:15:282
254972959,47sleep225533-21ntpq07:36:202
1351129519,53sleep312936-21/usr/sbin/munin12:06:093
986129410,42sleep20-21swapper/211:55:542
258972946,46sleep10-21swapper/109:21:081
115392947,14sleep30-21swapper/310:20:403
321029223,50sleep33234-21idleruntime-cro08:10:383
2115329111,64sleep20-21swapper/212:31:032
2268029016,49sleep122776-21cat10:55:391
2268029016,49sleep122776-21cat10:55:391
128252908,70sleep212838-21acpi08:40:502
836628931,19sleep28428-21cat08:26:002
836628931,19sleep28428-21cat08:26:002
2855128911,63sleep028563-21grep07:46:030
2198528911,62sleep022103-21sed10:51:100
2198528911,62sleep022103-21sed10:51:100
2183128912,63sleep20-21swapper/210:51:082
2183128912,63sleep20-21swapper/210:51:082
190902888,15sleep10-21swapper/109:00:531
148552885,16sleep20-21swapper/210:30:482
148552885,16sleep20-21swapper/210:30:482
205812879,35sleep20-21swapper/210:46:112
205812879,35sleep20-21swapper/210:46:112
1718228719,48sleep017236-21fschecks_time07:11:100
1718228719,48sleep017236-21fschecks_time07:11:100
2741628621,42sleep327610-21grep11:10:403
2091328618,50sleep121053-21unixbench_singl07:21:121
1178328622,45sleep111848-21grep08:36:091
625328520,30sleep10-21swapper/111:45:391
1774128528,36sleep117765-21fschecks_count12:20:561
1671228513,19sleep10-21swapper/107:10:461
1671228513,19sleep10-21swapper/107:10:461
1351428526,44sleep213576-21if_err_eth010:26:002
1351428526,44sleep213576-21if_err_eth010:26:002
360128416,17sleep00-21swapper/008:10:590
1988628423,41sleep319911-21munin-run07:20:383
1988628423,41sleep319911-21munin-run07:20:383
594328211,53sleep00-21swapper/008:16:130
2260028214,50sleep322658-21modprobe10:55:373
2260028214,50sleep322658-21modprobe10:55:373
1744028219,28sleep20-21swapper/210:37:022
1019028216,50sleep310214-21proc_pri08:31:063
712728122,31sleep37209-21grep08:21:123
712728122,31sleep37209-21grep08:21:123
3422816,12sleep20-21swapper/211:25:582
3178928120,16sleep30-21swapper/311:22:233
943928014,48sleep09488-21unin-run08:30:380
943928014,48sleep09488-21unin-run08:30:380
854927912,54sleep18560-21netstat08:26:031
854927912,54sleep18560-21netstat08:26:031
657327913,45sleep10-21swapper/111:45:561
657327913,45sleep10-21swapper/111:45:561
385327911,41sleep30-21swapper/311:36:043
139232790,6sleep20-21swapper/207:01:102
604827715,30sleep20-21swapper/208:16:512
2577827721,27sleep30-21swapper/311:05:393
2577827721,27sleep30-21swapper/311:05:393
2110527722,27sleep30-21swapper/310:50:413
2110527722,27sleep30-21swapper/310:50:413
1416427712,50sleep314231-21latency_hist08:45:403
660827610,54sleep16611-21acpi08:20:501
660827610,54sleep16611-21acpi08:20:501
63727624,30sleep10-21swapper/109:45:401
2896127613,15sleep10-21swapper/107:46:131
1794327617,40sleep117740-21latency_hist10:40:411
171527619,43sleep31828-21cat08:05:413
2465027515,41sleep324804-21apt-key07:35:403
1741527519,34sleep00-21swapper/012:20:390
2341527414,42sleep123488-21mailstats10:56:081
2341527414,42sleep123488-21mailstats10:56:081
2112127413,45sleep121152-21nfs_client12:31:021
518027310,44sleep30-21swapper/308:15:593
1587227325,22sleep10-21swapper/112:15:401
1587227325,22sleep10-21swapper/112:15:401
1383327323,32sleep113922-21sed07:01:101
2907727218,33sleep029080-21cron11:15:380
2105227220,30sleep00-21swapper/009:06:050
1955327218,40sleep319567-21awk09:01:073
1839527215,16sleep30-21swapper/307:15:413
1839527215,16sleep30-21swapper/307:15:413
3226527119,30sleep30-21swapper/309:41:223
2976627122,21sleep10-21swapper/111:16:031
2789427121,27sleep30-21swapper/307:45:383
2789427121,27sleep30-21swapper/307:45:383
1841227127,27sleep30-21swapper/310:41:113
1059427116,37sleep110603-21/usr/sbin/munin10:16:011
64127016,39sleep0656-21needreboot11:26:040
2446027013,39sleep124586-21/usr/sbin/munin09:16:111
1993727019,27sleep10-21swapper/107:20:401
1993727019,27sleep10-21swapper/107:20:401
192722707,19sleep00-21swapper/010:42:380
1604227013,41sleep316049-21/usr/sbin/munin08:50:573
1560727016,30sleep00-21swapper/010:31:110
1560727016,30sleep00-21swapper/010:31:110
426826921,26sleep30-21swapper/309:56:003
3125626916,37sleep031272-21apt09:40:380
3125626916,37sleep031272-21apt09:40:380
2908326917,35sleep11-21systemd11:15:391
279426917,31sleep32795-21sshd11:31:563
279426917,31sleep32795-21sshd11:31:563
494826819,25sleep00-21swapper/011:40:540
1451826828,21sleep30-21swapper/307:05:393
564126718,25sleep00-21swapper/010:00:580
294252675,50sleep00-21swapper/009:32:220
2452926719,31sleep124585-21sendmail07:35:381
1619626715,29sleep316277-21tune2fs12:15:583
609026612,40sleep36144-21tr10:01:093
2758726624,22sleep10-21swapper/109:26:091
2622726613,23sleep00-21swapper/009:21:110
1797126613,37sleep017990-21ntp_states08:56:030
1605126614,33sleep016087-21fschecks_count08:50:580
911126514,23sleep30-21swapper/310:11:063
81026521,22sleep00-21swapper/008:01:060
685626518,24sleep10-21swapper/110:05:391
293542658,11sleep30-21swapper/309:31:283
250822657,13sleep30-21swapper/307:35:533
1632026518,25sleep10-21swapper/112:16:001
1151426517,28sleep10-21swapper/112:00:581
3000126421,31sleep130047-21cpu09:35:541
3000126421,31sleep130047-21cpu09:35:541
2572026419,27sleep00-21swapper/007:36:260
1965326414,23sleep00-21swapper/012:26:040
1960726418,25sleep30-21swapper/312:26:033
1330226419,20sleep00-21swapper/010:25:500
1330226419,20sleep00-21swapper/010:25:500
689726310,38sleep06944-21grep11:46:030
689726310,38sleep06944-21grep11:46:030
2041626316,29sleep320449-21/usr/sbin/munin07:21:023
1863126310,39sleep018659-21cpu07:15:540
1863126310,39sleep018659-21cpu07:15:540
1500426313,27sleep00-21swapper/012:11:060
1500426313,27sleep00-21swapper/012:11:060
2727726216,29sleep127301-21munin-run11:10:381
2714926214,25sleep00-21swapper/009:26:000
233612625,45sleep00-21swapper/009:14:210
1624626218,29sleep016272-21munin-node07:10:410
160326210,10sleep10-21swapper/108:04:551
825026112,21sleep10-21swapper/111:50:481
3187426114,18sleep00-21swapper/011:22:240
2787126111,39sleep00-21swapper/007:45:190
2787126111,39sleep00-21swapper/007:45:190
209772616,15sleep00-21swapper/010:49:410
209772616,15sleep00-21swapper/010:49:410
2046026115,29sleep120492-21munin-run12:30:371
155512618,16sleep00-21swapper/008:49:080
1549526121,19sleep30-21swapper/310:31:103
1549526121,19sleep30-21swapper/310:31:103
1296426117,27sleep012955-21apt-get12:05:540
129282619,37sleep112936-21diskstats08:40:571
3139026018,22sleep10-21swapper/107:55:581
3077526018,26sleep330776-21sshd07:52:573
1233626013,33sleep012397-21/usr/sbin/munin10:21:090
1233626013,33sleep012397-21/usr/sbin/munin10:21:090
436825917,27sleep14413-21awk09:56:021
350225916,27sleep33497-21apt-get08:10:543
282625912,21sleep30-21swapper/309:51:033
257882598,12sleep30-21swapper/309:21:063
782325812,26sleep37957-21/usr/sbin/munin10:06:103
3065125814,20sleep00-21swapper/011:20:380
2102125817,21sleep30-21swapper/309:06:043
1117225820,21sleep30-21swapper/312:00:403
1112125810,24sleep10-21swapper/112:00:381
7332579,34sleep3739-21ntp_states11:26:073
2842325717,22sleep128417-21apt-get09:30:541
269542576,42sleep00-21swapper/011:06:250
269325711,33sleep02733-21latency09:51:000
2447125715,26sleep024466-21apt-get11:00:510
2447125715,26sleep024466-21apt-get11:00:510
1342825717,19sleep10-21swapper/112:06:071
250825617,23sleep12565-21fschecks_count09:50:571
2209825618,19sleep30-21swapper/307:26:033
1205525614,22sleep10-21swapper/110:21:021
1205525614,22sleep10-21swapper/110:21:021
114912566,12sleep30-21swapper/312:00:563
999625515,18sleep00-21swapper/010:15:400
370425516,21sleep10-21swapper/108:11:011
3138025515,16sleep30-21swapper/307:55:563
1895525517,23sleep30-21swapper/312:25:393
1012525515,17sleep00-21swapper/008:31:030
972625413,25sleep19721-21apt-get08:30:521
775025417,21sleep30-21swapper/311:50:383
775025417,21sleep30-21swapper/311:50:383
43832546,39sleep10-21swapper/111:37:071
309922548,33sleep30-21swapper/309:37:143
309922548,33sleep30-21swapper/309:37:143
2354625413,22sleep023639-21memory07:31:010
140625414,19sleep10-21swapper/109:46:081
140625414,19sleep10-21swapper/109:46:081
1292125414,16sleep00-21swapper/008:40:560
408425315,23sleep00-21swapper/009:55:570
2939525311,24sleep329390-21apt-get11:15:533
2695025311,16sleep30-21swapper/309:25:553
2291825312,24sleep322913-21apt-get10:55:543
2291825312,24sleep322913-21apt-get10:55:543
2246925315,20sleep10-21swapper/109:11:021
1785625313,24sleep017926-21munin-run10:40:400
65725215,20sleep10-21swapper/111:26:051
182525214,22sleep01820-21apt-get11:30:520
182525214,22sleep01820-21apt-get11:30:520
1773425212,18sleep00-21swapper/012:20:550
1505125210,27sleep115111-21idleruntime10:31:001
1505125210,27sleep115111-21idleruntime10:31:001
8492516,36sleep00-21swapper/009:45:470
8492516,36sleep00-21swapper/009:45:470
712425112,15sleep00-21swapper/010:05:540
563225117,20sleep15695-21swap08:16:101
494625112,22sleep34941-21apt-get11:40:533
3130425112,23sleep031298-21apt-get07:55:530
302262519,17sleep00-21swapper/007:51:080
29872517,34sleep10-21swapper/111:34:251
29872517,34sleep10-21swapper/111:34:251
2304825112,17sleep10-21swapper/107:30:411
1676425112,14sleep10-21swapper/108:51:111
1854025010,11sleep30-21swapper/308:57:153
1440625012,23sleep314473-21df08:45:563
1168825013,16sleep00-21swapper/012:01:010
2921624918,14sleep30-21swapper/311:15:413
1603824913,20sleep30-21swapper/307:09:513
751924811,16sleep10-21swapper/110:06:041
242322489,17sleep00-21swapper/009:16:060
2400224812,21sleep324051-21irqstats09:16:013
2026424814,15sleep00-21swapper/007:20:590
1261324812,21sleep012722-21latency_hist08:40:400
1157624813,21sleep011713-21latency_hist10:20:410
3166624710,21sleep031272-21apt-get09:40:420
241624611,21sleep02453-21ps08:06:070
826124315,14sleep30-21swapper/308:25:573
826124315,14sleep30-21swapper/308:25:573
2700724312,16sleep10-21swapper/107:41:041
2700724312,16sleep10-21swapper/107:41:041
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional