You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-04 - 03:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Thu Jun 04, 2026 00:43:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
311122607015,6043sleep315435-21kworker/3:123:00:153
189382128226,1240sleep318926-21/usr/sbin/munin22:23:133
15829210888,1066sleep112596-21kworker/1:123:56:071
8773274518,709sleep38824-21/usr/sbin/munin19:38:303
8773274518,709sleep38824-21/usr/sbin/munin19:38:303
30686261780,516sleep030790-21grep22:58:420
2211245211,426sleep32253-21ntpq21:28:363
11353236213,336sleep211394-21unin-run23:43:122
11353236213,336sleep211394-21unin-run23:43:122
524123016,282sleep00-21swapper/019:20:470
524123016,282sleep00-21swapper/019:20:470
2754222687,248sleep027632-21grep00:33:320
2754222687,248sleep027632-21grep00:33:320
19138225621,208sleep319158-21awk00:08:123
29991225334,207sleep00-21swapper/021:10:330
12001223515,29sleep00-21swapper/023:43:350
12001223515,29sleep00-21swapper/023:43:350
1991223452,163sleep02017-21modprobe23:13:110
13662222818,191sleep113722-21sendmail_mailqu23:48:401
13662222818,191sleep113722-21sendmail_mailqu23:48:401
24480222220,134sleep00-21swapper/000:23:390
24480222220,134sleep00-21swapper/000:23:390
28068221240,115sleep00-21swapper/021:03:150
23137220711,176sleep00-21swapper/000:18:420
11842220615,170sleep011983-21unixbench_multi21:58:440
11842220615,170sleep011983-21unixbench_multi21:58:440
13272220343,68sleep00-21swapper/023:48:310
13272220343,68sleep00-21swapper/023:48:310
7229220229,109sleep00-21swapper/021:43:430
2824322028,56sleep01-21systemd22:53:130
17921220218,130sleep017952-21kernelversion22:18:460
9586219810,176sleep09589-21grep21:53:280
10756218621,108sleep00-21swapper/019:48:240
10756218621,108sleep00-21swapper/019:48:240
1494221825,157sleep00-21swapper/023:53:320
25821217030,116sleep00-21swapper/020:53:140
5621216540,111sleep00-21swapper/021:38:420
2429216510,103sleep00-21swapper/023:13:320
830721648,117sleep08380-21sed21:48:350
1268216433,106sleep21295-21ls19:08:142
19103216211,69sleep019104-21gpgv20:23:150
12980216114,134sleep213057-21sed19:58:152
20820215950,92sleep00-21swapper/022:28:310
7472215667,18sleep00-21swapper/023:28:410
7472215667,18sleep00-21swapper/023:28:410
2796215641,99sleep00-21swapper/019:13:120
28702215432,108sleep028843-21sed21:08:150
20710214711,121sleep220788-21find20:28:422
10695214786,41sleep00-21swapper/023:38:420
10695214786,41sleep00-21swapper/023:38:420
32424214617,24sleep00-21swapper/021:23:120
12901214511,118sleep012928-21munin-plugin-st19:58:130
307921437,119sleep03124-21unin-run21:33:130
6111214211,86sleep00-21swapper/019:28:140
6111214211,86sleep00-21swapper/019:28:140
347214127,101sleep0449-21grep21:23:330
32091213946,44sleep032125-21proc_pri23:03:390
32091213946,44sleep032125-21proc_pri23:03:390
19581213913,110sleep119619-21ntpq22:23:361
27843213812,114sleep127879-21ntpq00:33:371
27843213812,114sleep127879-21ntpq00:33:371
6151213410,109sleep16300-21cut19:28:151
6151213410,109sleep16300-21cut19:28:151
6752213335,84sleep06797-21ntpq19:28:340
6752213335,84sleep06797-21ntpq19:28:340
23842213315,101sleep223938-21apt-key00:23:122
24360213123,17sleep00-21swapper/022:38:400
2233521309,106sleep022480-21sed00:18:140
16320213010,35sleep00-21swapper/023:58:240
16815212772,19sleep0363-21snmpd22:14:550
16815212772,19sleep0363-21snmpd22:14:550
59702126100,13sleep00-21swapper/023:23:410
1891321265,89sleep00-21swapper/000:05:220
29615212518,80sleep229747-21tail21:08:422
22806212430,22sleep022880-21mailstats20:38:370
26320212216,40sleep00-21swapper/022:43:490
916921197,101sleep09243-21find19:38:390
916921197,101sleep09243-21find19:38:390
28635211423,70sleep128665-21awk21:08:121
23538211312,24sleep023541-21cron22:38:120
29176211210,82sleep00-21swapper/022:53:420
2108421125,40sleep00-21swapper/020:29:330
8124211121,65sleep18188-21fschecks_time21:48:321
15411210621,66sleep215545-21sed23:53:422
1964621048,84sleep019683-21ps22:23:390
18364210418,66sleep118513-21unixbench_singl22:18:561
31714210121,27sleep30-21swapper/323:03:303
31714210121,27sleep30-21swapper/323:03:303
15552210118,70sleep315620-21basename20:08:293
19191210011,64sleep119285-21latency_hist00:08:141
265812997,77sleep026734-21awk00:28:430
265812997,77sleep026734-21awk00:28:430
258329610,61sleep21790-21/usr/sbin/munin21:28:432
1526929562,22sleep0363-21snmpd22:09:540
2622329411,71sleep326225-21sshd20:55:473
1181729410,71sleep111880-21if_err_eth023:43:321
1181729410,71sleep111880-21if_err_eth023:43:321
1215529323,52sleep312354-21latency_hist19:53:153
1215529323,52sleep312354-21latency_hist19:53:153
3125929138,37sleep131345-21fschecks_time21:18:291
854129018,56sleep18542-21/usr/sbin/munin19:38:201
854129018,56sleep18542-21/usr/sbin/munin19:38:201
835529016,52sleep00-21swapper/019:38:150
835529016,52sleep00-21swapper/019:38:150
2742129016,53sleep327422-21/usr/sbin/munin00:33:233
2742129016,53sleep327422-21/usr/sbin/munin00:33:233
1708129021,52sleep217141-21modprobe22:18:122
1708129021,52sleep217141-21modprobe22:18:122
1402929018,26sleep10-21swapper/122:08:141
1402929018,26sleep10-21swapper/122:08:141
85122896,12sleep00-21swapper/023:33:260
85122896,12sleep00-21swapper/023:33:260
2097628817,51sleep320969-21apt-get00:13:293
197012885,11sleep00-21swapper/020:26:380
197012885,11sleep00-21swapper/020:26:380
139692886,70sleep00-21swapper/022:08:050
139692886,70sleep00-21swapper/022:08:050
321572875,12sleep30-21swapper/321:19:483
285928711,19sleep30-21swapper/319:13:153
215592876,11sleep00-21swapper/020:33:160
165042876,11sleep00-21swapper/020:11:040
150982876,12sleep00-21swapper/020:07:530
144432875,11sleep00-21swapper/020:02:410
635328613,17sleep20-21swapper/219:28:242
635328613,17sleep20-21swapper/219:28:242
62528630,38sleep0783-21sed19:03:440
263882866,11sleep00-21swapper/020:57:540
2565628617,23sleep20-21swapper/200:28:132
2565628617,23sleep20-21swapper/200:28:132
1731928617,22sleep10-21swapper/122:18:151
1731928617,22sleep10-21swapper/122:18:151
551028514,54sleep25600-21fschecks_time23:23:312
373128514,52sleep33339-21/usr/sbin/munin21:33:373
373128514,52sleep33339-21/usr/sbin/munin21:33:373
34652855,10sleep00-21swapper/019:13:250
34652855,10sleep00-21swapper/019:13:250
305972857,11sleep00-21swapper/021:15:050
2857728515,46sleep328472-21/usr/sbin/munin22:53:293
2724028522,18sleep30-21swapper/320:58:383
241812855,11sleep00-21swapper/020:47:410
241812855,11sleep00-21swapper/020:47:410
172622856,11sleep00-21swapper/020:17:300
172622856,11sleep00-21swapper/020:17:300
100942856,10sleep00-21swapper/019:43:230
2641128419,42sleep226437-21munin-run20:58:122
204232847,10sleep00-21swapper/000:09:500
204232847,10sleep00-21swapper/000:09:500
202528426,32sleep20-21swapper/219:08:422
50372836,12sleep00-21swapper/023:21:590
1889428316,50sleep118920-21awk20:23:121
45552826,11sleep00-21swapper/021:37:170
45552826,11sleep00-21swapper/021:37:170
3007728214,49sleep230144-21df22:58:282
2570028219,24sleep30-21swapper/300:28:143
2570028219,24sleep30-21swapper/300:28:143
99528011,15sleep30-21swapper/323:08:343
299542800,7sleep10-21swapper/122:58:141
2634728013,52sleep326358-21ntpq00:28:383
2634728013,52sleep326358-21ntpq00:28:383
1092528025,23sleep30-21swapper/321:58:153
2704827815,47sleep327144-21grep22:48:323
3016127727,28sleep10-21swapper/122:58:311
75822766,19sleep30-21swapper/321:46:043
47892768,53sleep24790-21sshd23:18:512
2448327618,33sleep20-21swapper/220:48:192
404427521,29sleep24200-21grep21:33:442
404427521,29sleep24200-21grep21:33:442
2586627518,28sleep10-21swapper/120:53:151
1657527511,22sleep30-21swapper/322:13:453
1657527511,22sleep30-21swapper/322:13:453
2825927417,30sleep10-21swapper/122:53:141
2201127419,28sleep10-21swapper/120:38:131
1468827410,14sleep10-21swapper/122:08:381
167752738,48sleep10-21swapper/123:58:371
1671127321,38sleep316744-21ntp_kernel_pll_23:58:353
863327221,26sleep30-21swapper/323:33:333
863327221,26sleep30-21swapper/323:33:333
2548927215,21sleep20-21swapper/222:43:312
254842729,47sleep125488-21diskstats22:43:301
2204427214,35sleep322020-21latency_hist22:33:143
1503627220,31sleep30-21swapper/322:08:443
1134527223,24sleep10-21swapper/123:43:111
1134527223,24sleep10-21swapper/123:43:111
2636727119,24sleep10-21swapper/100:28:401
2636727119,24sleep10-21swapper/100:28:401
1771927121,27sleep20-21swapper/220:18:292
1367127114,40sleep213672-21ntp_states19:58:352
827527010,19sleep10-21swapper/123:33:141
827527010,19sleep10-21swapper/123:33:141
2975727022,30sleep229821-21sendmail-msp22:58:122
2411627019,30sleep10-21swapper/122:38:331
1811727019,29sleep30-21swapper/320:18:383
1449627021,27sleep20-21swapper/223:53:132
1449627021,27sleep20-21swapper/223:53:132
2005426926,25sleep10-21swapper/100:08:421
2005426926,25sleep10-21swapper/100:08:421
848826817,20sleep20-21swapper/223:33:242
848826817,20sleep20-21swapper/223:33:242
392826821,19sleep20-21swapper/219:18:152
392826821,19sleep20-21swapper/219:18:152
1269526819,23sleep30-21swapper/322:03:243
1269526819,23sleep30-21swapper/322:03:243
2720826716,22sleep20-21swapper/200:33:132
2720826716,22sleep20-21swapper/200:33:132
337726612,36sleep13439-21ntp_states19:13:211
337726612,36sleep13439-21ntp_states19:13:211
255126612,25sleep20-21swapper/223:13:342
1979726610,12sleep30-21swapper/300:08:363
1979726610,12sleep30-21swapper/300:08:363
159512669,36sleep10-21swapper/120:08:391
934626523,22sleep30-21swapper/321:53:143
521026515,27sleep30-21swapper/321:38:333
43026514,34sleep1467-21basename23:08:121
43026514,34sleep1467-21basename23:08:121
2802026515,29sleep228102-21switchtime00:33:422
2802026515,29sleep228102-21switchtime00:33:422
2680926518,33sleep126829-21grep20:58:271
2427726521,20sleep20-21swapper/220:48:152
2427726521,20sleep20-21swapper/220:48:152
2382226512,32sleep123841-21idleruntime-cro00:23:111
14182656,10sleep10-21swapper/121:26:491
840626414,26sleep28014-21/usr/sbin/munin21:48:372
3267626424,21sleep30-21swapper/319:03:353
2719026416,25sleep20-21swapper/220:58:372
1139526420,22sleep30-21swapper/323:43:133
1139526420,22sleep30-21swapper/323:43:133
470826316,23sleep10-21swapper/121:38:151
470826316,23sleep10-21swapper/121:38:151
2275026313,20sleep30-21swapper/300:18:333
515126212,32sleep20-21swapper/221:38:322
424826219,21sleep30-21swapper/319:18:283
424826219,21sleep30-21swapper/319:18:283
3132625,11sleep10-21swapper/119:03:371
2029326216,19sleep30-21swapper/320:28:313
147332626,14sleep30-21swapper/323:53:273
1345626215,31sleep313545-21/usr/sbin/munin23:48:343
1345626215,31sleep313545-21/usr/sbin/munin23:48:343
1218926219,23sleep20-21swapper/222:00:082
1218926219,23sleep20-21swapper/222:00:082
32526111,34sleep2331-21diskstats21:23:312
2240826118,26sleep322498-21cut20:38:283
2147926116,22sleep10-21swapper/100:13:431
202326111,34sleep12041-21munin-run23:13:131
148282617,13sleep20-21swapper/220:04:282
679926017,20sleep10-21swapper/119:28:351
679926017,20sleep10-21swapper/119:28:351
2208926016,23sleep20-21swapper/220:38:142
203632606,11sleep20-21swapper/200:09:042
203632606,11sleep20-21swapper/200:09:042
1926926023,17sleep30-21swapper/322:23:303
176052606,35sleep20-21swapper/222:18:202
152932607,10sleep20-21swapper/220:08:162
129926022,21sleep20-21swapper/223:08:412
997425917,27sleep110012-21/usr/sbin/munin21:53:371
69372596,16sleep20-21swapper/223:28:192
69372596,16sleep20-21swapper/223:28:192
606625918,25sleep3284-21dbus-daemon19:28:133
606625918,25sleep3284-21dbus-daemon19:28:133
2254825915,27sleep20-21swapper/222:33:342
1190525912,18sleep30-21swapper/323:43:343
1190525912,18sleep30-21swapper/323:43:343
1052125917,23sleep30-21swapper/319:48:153
832525816,26sleep30-21swapper/319:38:143
832525816,26sleep30-21swapper/319:38:143
822425818,20sleep30-21swapper/323:33:123
822425818,20sleep30-21swapper/323:33:123
560125815,20sleep30-21swapper/323:23:323
401225821,18sleep30-21swapper/323:18:313
3094825813,28sleep331093-21cat21:18:153
193982585,12sleep20-21swapper/220:23:252
193982585,12sleep20-21swapper/220:23:252
1365925817,27sleep113665-21ntp_states19:58:341
1142925817,25sleep311460-21kernelversion21:58:353
1142925817,25sleep311460-21kernelversion21:58:353
1081125818,22sleep10-21swapper/119:48:251
1081125818,22sleep10-21swapper/119:48:251
2068325713,22sleep10-21swapper/120:28:411
1293725714,25sleep20-21swapper/223:48:132
1293725714,25sleep20-21swapper/223:48:132
648525615,25sleep16482-21apt-get21:43:241
413525615,28sleep14169-21wc23:18:331
3268625615,25sleep332704-21/usr/sbin/munin21:23:243
3135225614,19sleep20-21swapper/221:18:302
2758425615,18sleep20-21swapper/222:48:432
2481525618,22sleep10-21swapper/100:23:481
2481525618,22sleep10-21swapper/100:23:481
2124325615,26sleep221259-21/usr/sbin/munin00:13:362
2203225521,19sleep20-21swapper/222:33:132
1075325513,25sleep210749-21apt-get19:48:222
1075325513,25sleep210749-21apt-get19:48:222
673025416,15sleep10-21swapper/123:28:141
480725416,18sleep20-21swapper/219:18:392
480725416,18sleep20-21swapper/219:18:392
471825414,20sleep10-21swapper/119:18:381
471825414,20sleep10-21swapper/119:18:381
3170725413,23sleep231713-21diskstats23:03:292
3170725413,23sleep231713-21diskstats23:03:292
2539725412,16sleep30-21swapper/322:43:243
225922547,22sleep20-21swapper/220:38:312
2227625413,25sleep122271-21apt-get22:33:251
1517425418,18sleep10-21swapper/120:08:151
300225320,17sleep30-21swapper/323:13:433
2394025317,16sleep20-21swapper/222:38:302
1938825313,22sleep118909-21apt-get20:23:201
1938825313,22sleep118909-21apt-get20:23:201
1416325318,21sleep30-21swapper/319:59:133
58492525,35sleep20-21swapper/219:25:252
58492525,35sleep20-21swapper/219:25:252
2893525211,24sleep128929-21apt-get21:08:241
2199225211,26sleep122022-21munin-run22:33:121
1115925212,23sleep111156-21apt-get21:58:251
1115925212,23sleep111156-21apt-get21:58:251
1051125212,21sleep210520-21munin-run19:48:142
715425112,22sleep37252-21users19:28:413
715425112,22sleep37252-21users19:28:413
398125112,18sleep30-21swapper/319:18:163
2385725116,15sleep30-21swapper/322:38:273
200125112,18sleep10-21swapper/119:08:411
1636725111,24sleep216393-21df23:58:262
1251725113,23sleep112032-21apt-get19:53:211
35932505,32sleep30-21swapper/319:15:013
3092125015,21sleep230945-21cat21:18:142
2577325011,23sleep31-21systemd20:53:133
1918425011,23sleep219180-21apt-get22:23:252
911724915,17sleep10-21swapper/123:33:431
911724915,17sleep10-21swapper/123:33:431
2239924911,15sleep10-21swapper/120:38:261
1738724916,20sleep3284-21dbus-daemon20:18:133
1738724916,20sleep3284-21dbus-daemon20:18:133
1707724910,23sleep10-21swapper/120:15:131
1707724910,23sleep10-21swapper/120:15:131
1015624914,22sleep210249-21sed21:53:422
90322489,25sleep29078-21processes19:38:372
90322489,25sleep29078-21processes19:38:372
77042485,33sleep20-21swapper/221:47:402
49442488,15sleep10-21swapper/121:38:271
2907124811,16sleep30-21swapper/321:08:303
282524813,20sleep12258-21/usr/sbin/munin23:13:411
280524711,22sleep12806-21sshd21:29:471
2427324712,21sleep124276-21munin-run20:48:141
2427324712,21sleep124276-21munin-run20:48:141
2225224710,19sleep30-21swapper/322:33:233
1490924713,18sleep20-21swapper/222:08:432
1989424612,14sleep30-21swapper/320:28:143
1989424612,14sleep30-21swapper/320:28:143
825424514,19sleep21-21systemd19:38:132
825424514,19sleep21-21systemd19:38:132
80112456,30sleep10-21swapper/119:35:051
80112456,30sleep10-21swapper/119:35:051
1796224511,19sleep20-21swapper/200:03:312
1363224514,16sleep20-21swapper/222:03:472
1363224514,16sleep20-21swapper/222:03:472
3085024415,15sleep10-21swapper/121:18:131
99992439,13sleep10-21swapper/119:43:161
2844524113,14sleep20-21swapper/221:05:472
2396923910,15sleep30-21swapper/320:44:573
2396923910,15sleep30-21swapper/320:44:573
125192396,18sleep30-21swapper/319:53:233
173182387,14sleep20-21swapper/220:18:122
173182387,14sleep20-21swapper/220:18:122
2635623510,15sleep10-21swapper/120:57:301
146022358,13sleep30-21swapper/320:03:163
286902347,13sleep30-21swapper/321:08:143
218972287,14sleep30-21swapper/320:36:463
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional