You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-08 - 07:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Wed Apr 08, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
17512210888,1066sleep217543-21/usr/sbin/munin21:17:122
16899267119,634sleep316930-21/usr/sbin/munin19:32:073
1655325729,546sleep016592-21ntpq22:57:080
1655325729,546sleep016592-21ntpq22:57:080
2677524639,438sleep026821-21unin-run23:31:440
24799242812,401sleep324836-21cat21:41:443
11223241760,276sleep2222-21systemd-journal22:41:432
16721236323,322sleep116765-21awk21:16:431
2250234718,312sleep22367-21latency_hist23:56:452
187912309113,130sleep218946-21grep19:37:152
23179229017,259sleep2222-21systemd-journal21:36:432
2082224917,215sleep02128-40ogrotate23:56:430
8029221812,194sleep28073-21ntp_kernel_err20:47:082
12926220320,121sleep20-21swapper/221:02:152
28731220214,167sleep228819-21fschecks_time23:37:032
25874220122,81sleep20-21swapper/223:27:062
968321969,76sleep20-21swapper/220:52:132
19290219310,44sleep20-21swapper/223:06:592
2552521919,170sleep225605-21grep21:42:122
12442218316,150sleep012546-21fschecks_time21:02:040
13150217638,120sleep21-21systemd19:21:452
11836216979,69sleep212002-21latency_hist00:26:452
11836216979,69sleep212002-21latency_hist00:26:452
3420216517,59sleep20-21swapper/220:32:142
3420216517,59sleep20-21swapper/220:32:142
479021638,89sleep24852-21idleruntime20:37:522
479021638,89sleep24852-21idleruntime20:37:522
28978216148,97sleep20-21swapper/220:11:592
24073215913,129sleep024169-21sed21:37:130
31268215767,31sleep20-21swapper/222:01:582
21017215729,116sleep221050-21basename23:12:042
30934215627,117sleep230981-21ntpq20:17:082
238021560,13sleep30-21swapper/323:56:453
24010215523,109sleep024030-21idleruntime-cro19:56:430
22494215530,105sleep20-21swapper/223:17:002
11574215435,102sleep211631-21grep19:16:442
31842214311,116sleep131934-21cat22:02:131
1621214113,111sleep21648-21munin-plugin-st22:11:432
210132140102,18sleep20-21swapper/221:27:142
25644213910,114sleep225793-21latency_hist20:01:462
980321375,12sleep10-21swapper/100:17:151
20123213729,17sleep20-21swapper/219:42:122
19432213461,34sleep219522-21switchtime21:22:382
7912213232,85sleep28058-21latency_hist22:31:462
30450213232,85sleep230597-21sed21:57:142
18366213111,17sleep20-21swapper/223:02:142
13266212512,21sleep10-21swapper/122:47:031
2430421246,12sleep00-21swapper/019:56:550
21136212411,84sleep30-21swapper/321:27:153
6259212311,98sleep26325-21idleruntime20:42:042
6259212311,98sleep26325-21idleruntime20:42:042
20086212321,82sleep020121-21basename21:26:430
3899212229,77sleep20-21swapper/222:17:082
3211121228,45sleep20-21swapper/220:22:012
6719212117,89sleep06831-21timerandwakeup20:42:140
6719212117,89sleep06831-21timerandwakeup20:42:140
2452021208,100sleep024593-21cat23:22:150
26859211912,94sleep326860-21journal-offline20:03:083
837521188,97sleep28423-21awk19:07:032
31728211723,16sleep20-21swapper/223:46:562
11072211612,21sleep20-21swapper/220:57:062
2933621157,98sleep213441-21snmpd21:55:572
12812211530,19sleep212821-21munin-run22:46:442
3179211314,80sleep20-21swapper/222:16:442
26153211218,14sleep20-21swapper/220:02:062
20964211218,79sleep020999-21latency_hist19:46:460
5509210962,26sleep00-21swapper/000:06:450
13778210833,40sleep00-21swapper/000:32:000
863121066,81sleep00-21swapper/019:07:090
22395210613,24sleep00-21swapper/023:16:540
16795210610,15sleep20-21swapper/222:57:142
16795210610,15sleep20-21swapper/222:57:142
14282210671,23sleep214354-21mailstats00:32:142
22388210516,17sleep30-21swapper/321:32:113
13921210415,26sleep20-21swapper/219:22:132
25684210311,77sleep025782-21cat21:42:140
21181210313,74sleep021200-21apt-get19:46:540
3114210212,74sleep33141-21/usr/sbin/munin20:32:073
3114210212,74sleep33141-21/usr/sbin/munin20:32:073
2509210219,16sleep20-21swapper/220:31:462
10199210112,18sleep20-21swapper/200:21:452
3989210015,20sleep20-21swapper/200:01:462
3989210015,20sleep20-21swapper/200:01:462
745629911,73sleep17521-21idleruntime00:12:031
2033729922,53sleep120500-21grep19:42:151
411229816,66sleep14235-21latency_hist20:36:461
411229816,66sleep14235-21latency_hist20:36:461
305782987,75sleep230615-21sed23:42:082
2882329614,19sleep30-21swapper/323:37:043
1270429610,41sleep212768-21find00:27:142
554929514,24sleep30-21swapper/322:22:143
3246229516,60sleep132477-21/usr/sbin/munin20:22:081
274312947,67sleep20-21swapper/220:06:592
236082948,18sleep223703-21perl21:37:032
380829312,62sleep03864-21memory22:17:060
270612938,15sleep20-21swapper/223:31:582
1624529317,57sleep016284-21kworker/u8:119:31:440
1691328915,58sleep317063-21unixbench_singl22:57:153
1691328915,58sleep317063-21unixbench_singl22:57:153
1388128911,14sleep20-21swapper/221:06:552
1388128911,14sleep20-21swapper/221:06:552
2965628820,40sleep129804-21grep20:12:151
2840128826,26sleep10-21swapper/123:36:461
51792878,12sleep20-21swapper/200:03:072
1439028714,57sleep314455-21awk21:07:123
1439028714,57sleep314455-21awk21:07:123
1374028712,48sleep313905-21sed22:47:133
919128616,55sleep19193-21/usr/sbin/munin20:51:561
82372866,10sleep20-21swapper/200:12:322
81152867,64sleep20-21swapper/222:31:542
81152867,64sleep20-21swapper/222:31:542
2251328615,51sleep322641-21latency_hist19:51:463
216202867,63sleep20-21swapper/219:47:082
1445728614,51sleep012187-1kworker/0:2H21:07:130
1445728614,51sleep012187-1kworker/0:2H21:07:130
140102868,12sleep20-21swapper/222:47:452
2321328411,61sleep123223-21open_files19:52:121
176492845,11sleep20-21swapper/219:34:532
158162845,11sleep20-21swapper/222:56:052
92932830,6sleep00-21swapper/000:17:050
70082837,61sleep20-21swapper/200:11:432
60842835,12sleep20-21swapper/222:23:552
46032838,17sleep00-21swapper/020:37:290
46032838,17sleep00-21swapper/020:37:290
278342836,11sleep20-21swapper/221:51:322
239692838,12sleep20-21swapper/223:22:012
227492837,60sleep20-21swapper/219:51:572
164542835,10sleep20-21swapper/221:13:252
159642835,10sleep20-21swapper/219:28:072
1827228212,56sleep318273-21sshd21:21:253
1406028224,41sleep014192-21timerwakeupswit19:22:150
705928113,18sleep00-21swapper/000:11:450
327928011,53sleep03429-21cat22:16:460
308352809,52sleep00-21swapper/020:17:060
2344328012,53sleep323446-21sshd23:18:433
1030527921,35sleep010299-21apt-get19:11:570
238027812,17sleep30-21swapper/323:56:463
238027812,17sleep30-21swapper/323:56:463
1125527816,41sleep310764-21/usr/sbin/munin20:57:113
3163427711,51sleep331660-21netstat22:02:073
2138827724,28sleep10-21swapper/119:47:031
1732627621,20sleep00-21swapper/021:17:060
940727520,36sleep39435-21munin-run22:36:433
940727520,36sleep39435-21munin-run22:36:433
705227518,43sleep37082-21proc_pri22:27:113
3076227516,40sleep330160-21/usr/sbin/munin23:42:133
3147527413,41sleep031515-21/usr/sbin/munin22:02:040
2666127411,23sleep00-21swapper/021:46:580
2252627415,40sleep122624-21timerandwakeup21:32:131
686727325,26sleep10-21swapper/119:02:021
508527316,24sleep10-21swapper/120:37:591
508527316,24sleep10-21swapper/120:37:591
1197727323,20sleep00-21swapper/019:17:020
696627216,42sleep37038-21cat19:02:043
3092427216,37sleep130539-21/usr/sbin/munin20:17:071
1285027216,43sleep112922-21mailstats21:02:141
1052627218,38sleep010541-21sort00:22:030
1052627218,38sleep010541-21sort00:22:030
1822627119,29sleep30-21swapper/319:37:043
1343927114,40sleep113480-21cpu19:22:001
598627014,39sleep36019-21irqstats00:07:043
3102927021,26sleep10-21swapper/122:01:451
3018527012,42sleep130234-21/usr/sbin/munin23:41:581
2086027019,26sleep00-21swapper/021:27:120
1942627016,28sleep10-21swapper/119:41:461
1525527015,22sleep30-21swapper/319:27:073
742126914,16sleep30-21swapper/320:46:453
742126914,16sleep30-21swapper/320:46:453
738826912,38sleep07499-21ls19:02:130
687326821,27sleep26965-21fschecks_time19:02:032
2717826825,23sleep00-21swapper/020:06:450
53826722,24sleep30-21swapper/320:22:183
2516026719,26sleep10-21swapper/119:57:171
2367826712,39sleep123706-21munin-run23:21:441
170626721,20sleep10-21swapper/120:27:131
480626620,24sleep30-21swapper/322:21:463
2347126613,36sleep123473-21/usr/sbin/munin21:36:541
1828726611,42sleep118364-21switchtime23:02:131
128726615,31sleep30-21swapper/320:27:033
1212826620,26sleep30-21swapper/319:17:053
120042667,20sleep30-21swapper/300:26:463
2328526517,35sleep323356-21/usr/sbin/munin19:52:143
455326416,29sleep34025-21apt-get20:36:563
455326416,29sleep34025-21apt-get20:36:563
3187026411,40sleep331947-21latency_hist20:21:463
2559726415,19sleep10-21swapper/123:27:011
2326026417,23sleep00-21swapper/021:36:450
22452647,12sleep00-21swapper/020:28:570
1127526418,28sleep00-21swapper/022:41:450
879926321,26sleep10-21swapper/122:32:131
879926321,26sleep10-21swapper/122:32:131
2722926319,23sleep10-21swapper/121:47:111
2093526310,38sleep121012-21switchtime21:27:131
2059926320,20sleep00-21swapper/023:11:450
1475226316,20sleep30-21swapper/319:26:463
308582629,37sleep00-21swapper/023:42:140
1703726214,29sleep017115-21mailstats19:32:120
1453526213,33sleep114657-21expr21:07:141
1453526213,33sleep114657-21expr21:07:141
142922626,44sleep00-21swapper/022:51:220
1295326218,20sleep10-21swapper/100:27:171
945326122,23sleep00-21swapper/022:36:450
945326122,23sleep00-21swapper/022:36:450
929526114,25sleep00-21swapper/020:52:030
575126116,24sleep10-21swapper/100:06:581
2873826111,34sleep028878-21latency_hist20:11:450
2743826118,23sleep00-21swapper/023:32:080
1829726125,22sleep1281-21dbus-daemon21:21:441
1500726114,23sleep10-21swapper/122:52:081
973726013,27sleep19742-21munin-node19:11:461
807626015,29sleep313441-21snmpd20:47:093
737026019,26sleep17407-21modprobe20:46:431
737026019,26sleep17407-21modprobe20:46:431
215926016,27sleep12225-21munin-run23:56:441
2058726016,25sleep320597-21munin-run23:11:443
1773926012,21sleep30-21swapper/323:01:593
1403726019,21sleep30-21swapper/300:32:073
124672606,12sleep30-21swapper/322:42:493
630525911,29sleep06307-21cron22:26:430
262012596,41sleep10-21swapper/121:45:261
230125915,27sleep12335-21/usr/sbin/munin22:12:071
764925813,29sleep07663-21/usr/sbin/munin00:12:060
702125816,25sleep17044-21ntp_states22:27:091
59922589,11sleep10-21swapper/122:22:451
52825814,27sleep11-21systemd23:51:441
361925817,24sleep113441-21snmpd22:17:031
3210925814,26sleep032103-21apt-get20:21:590
2459425817,19sleep10-21swapper/123:22:161
171412584,10sleep10-21swapper/122:57:451
171412584,10sleep10-21swapper/122:57:451
1542525816,26sleep015444-21apt-get21:11:540
1376925816,23sleep113772-21apt-get00:31:571
110552589,22sleep10-21swapper/120:57:051
756325720,21sleep30-21swapper/300:12:053
403825717,26sleep00-21swapper/020:36:450
403825717,26sleep00-21swapper/020:36:450
289412575,41sleep30-21swapper/320:11:483
2218125718,23sleep30-21swapper/323:16:453
1811525713,28sleep018156-21ntpq23:02:080
1506325716,23sleep315070-21ntpq22:52:103
828125620,19sleep00-21swapper/020:47:150
2994725617,19sleep10-21swapper/123:41:461
2821425614,17sleep10-21swapper/121:52:001
2527225617,22sleep30-21swapper/319:57:433
2504025614,25sleep325036-21apt-get21:41:573
235925616,23sleep32365-21ntp_states22:12:093
1965525614,25sleep319652-21apt-get19:41:573
1222125615,25sleep012216-21apt-get00:26:550
98272556,37sleep30-21swapper/300:17:343
942525512,22sleep30-21swapper/320:52:063
3217225516,23sleep332183-21ntp_states23:47:093
306452556,11sleep30-21swapper/321:57:313
2992925512,25sleep029980-21irqstats21:57:040
264425517,19sleep10-21swapper/123:56:561
264425517,19sleep10-21swapper/123:56:561
2128825513,18sleep00-21swapper/023:12:100
1129225514,22sleep10-21swapper/122:41:461
1013025516,24sleep310261-21open_inodes19:11:523
305725415,18sleep00-21swapper/023:57:320
305725415,18sleep00-21swapper/023:57:320
2957925413,27sleep029653-21find20:12:140
1039125418,18sleep10-21swapper/119:12:031
995625313,25sleep39953-21apt-get22:36:563
798925315,22sleep18028-21memory20:47:071
2599725312,18sleep30-21swapper/323:27:113
1498925310,25sleep115059-21df19:27:011
785225212,16sleep00-21swapper/022:31:440
32125214,21sleep1341-21apt-get22:06:541
2831925215,22sleep328347-21munin-run23:36:443
2742725214,21sleep327424-21apt-get20:06:563
2595825210,17sleep10-21swapper/120:02:011
1931325214,23sleep319319-21ntp_states21:22:353
1471925212,21sleep014814-21perl22:52:030
1277225214,16sleep10-21swapper/122:46:431
1261825214,21sleep312711-21memory21:02:073
1189425212,27sleep111919-21cpu19:16:591
93225110,23sleep31028-21sed22:07:123
466325112,20sleep30-21swapper/300:02:113
295472519,17sleep00-21swapper/023:37:170
202252515,36sleep10-21swapper/123:07:341
1863725112,18sleep00-21swapper/019:37:130
1205625111,22sleep012082-21munin-run21:01:440
113922517,13sleep10-21swapper/100:22:491
113922517,13sleep10-21swapper/100:22:491
320042508,17sleep00-21swapper/023:47:060
2813325011,22sleep028129-21apt-get21:51:550
46312498,19sleep10-21swapper/100:02:091
292222497,32sleep30-21swapper/321:54:283
276452495,34sleep30-21swapper/321:49:073
1159724914,19sleep00-21swapper/022:42:020
208102479,19sleep10-21swapper/123:11:541
1885524712,20sleep118898-21df21:22:001
1787124713,16sleep10-21swapper/119:36:461
1590424711,20sleep315910-21ntp_states21:12:093
962724615,18sleep30-21swapper/319:11:443
278832469,14sleep10-21swapper/120:07:101
1051924612,17sleep30-21swapper/300:22:023
1051924612,17sleep30-21swapper/300:22:023
167224512,21sleep01643-21latency_hist22:11:450
241862438,17sleep30-21swapper/323:22:063
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional