You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-06 - 12:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Sun Sep 06, 2026 00:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12282241167,22sleep20-21swapper/219:05:342
319862238163,24sleep10-21swapper/119:05:051
13132233161,59sleep00-21swapper/019:06:400
12642204168,24sleep30-21swapper/319:06:043
2175221650,5sleep3169199cyclictest22:20:153
322672890,4sleep20-21swapper/223:55:262
243152510,2sleep30-21swapper/322:25:173
60202500,2sleep30-21swapper/322:55:273
104912480,1sleep10-21swapper/120:40:171
104912480,1sleep10-21swapper/120:40:171
286352410,1sleep30-21swapper/323:50:133
167499405,25cyclictest26842-21kworker/u8:021:24:060
167499400,29cyclictest0-21swapper/023:15:200
167499392,6cyclictest24949-21munin-run21:15:020
167499390,29cyclictest0-21swapper/020:00:220
167499382,6cyclictest27149-21munin-run21:20:010
167499381,28cyclictest546-21/usr/sbin/munin00:00:210
1674993710,16cyclictest11441-21awk00:25:000
167499369,12cyclictest2008-21idleruntime-cro22:50:010
167499361,7cyclictest21161-21sh22:20:020
167499361,4cyclictest29324-21grep22:35:270
167499361,26cyclictest808-21snmpd23:00:310
167499361,26cyclictest808-21snmpd23:00:310
169199351,27cyclictest0-21swapper/300:35:563
167499353,3cyclictest87550irq/128-enp1s0-22:40:270
167499351,7cyclictest0-21swapper/020:25:220
167499351,4cyclictest22743-21diskmemload21:55:190
167499343,4cyclictest26844-21ntpq22:30:240
167499341,5cyclictest30799-21/usr/sbin/munin23:55:180
168799331,14cyclictest0-21swapper/220:11:412
167499336,3cyclictest17615-21/usr/sbin/munin23:25:270
167499336,3cyclictest15759-21/usr/sbin/munin19:40:140
167499331,6cyclictest22884-21latency23:35:180
167499331,4cyclictest810-21sshd19:18:050
167499331,23cyclictest27330-21perl20:05:150
167499331,22cyclictest13237-21smart_sda20:45:240
167499331,20cyclictest2974-21awk20:25:010
168799328,3cyclictest0-21swapper/220:42:122
168799328,3cyclictest0-21swapper/220:42:122
168799321,14cyclictest0-21swapper/223:48:012
168799321,13cyclictest0-21swapper/220:28:052
167499329,20cyclictest29048-21cut23:50:150
167499323,21cyclictest0-21swapper/023:44:000
167499322,3cyclictest87350irq/126-enp1s0-21:10:020
167499321,3cyclictest11745-21latency_hist20:45:020
167499321,2cyclictest0-21swapper/000:20:010
1674993210,19cyclictest22350-21/usr/sbin/munin19:55:230
167499320,21cyclictest0-21swapper/023:20:210
169199319,3cyclictest0-21swapper/320:07:013
169199319,3cyclictest0-21swapper/319:31:453
169199319,3cyclictest0-21swapper/319:31:453
169199311,13cyclictest0-21swapper/319:56:453
168799311,15cyclictest0-21swapper/219:12:012
167499313,2cyclictest12050irq/123-ahci[0021:35:000
167499312,3cyclictest827-21runrttasks22:01:130
167499311,6cyclictest808-21snmpd20:40:080
167499311,5cyclictest5069-21/usr/sbin/munin00:10:230
167499311,5cyclictest16743-21hwlatdetect00:35:190
167499311,5cyclictest14179-21/usr/sbin/munin20:50:170
167499311,5cyclictest11365-21/usr/sbin/munin19:30:150
167499311,5cyclictest11365-21/usr/sbin/munin19:30:150
169199301,13cyclictest0-21swapper/321:16:293
1691993010,3cyclictest0-21swapper/320:51:533
168799307,3cyclictest0-21swapper/221:09:532
168799306,3cyclictest0-21swapper/220:06:492
168799305,3cyclictest0-21swapper/222:37:412
168799301,14cyclictest0-21swapper/223:08:562
1687993011,3cyclictest0-21swapper/220:56:162
167499308,19cyclictest1-21systemd23:00:010
167499307,20cyclictest30392-21uname21:25:180
167499307,19cyclictest9640-21fschecks_time19:25:160
167499303,3cyclictest87350irq/126-enp1s0-22:50:240
167499301,5cyclictest9736-21load23:05:200
167499301,5cyclictest0-21swapper/023:33:400
167499301,5cyclictest0-21swapper/021:44:260
167499301,5cyclictest0-21swapper/020:30:060
167499301,3cyclictest11664-21/usr/sbin/munin00:25:150
167499301,26cyclictest31146-21/usr/sbin/munin20:15:260
167499300,23cyclictest0-21swapper/021:45:230
169199299,3cyclictest0-21swapper/320:29:173
169199297,3cyclictest0-21swapper/320:49:133
1691992911,3cyclictest0-21swapper/320:23:573
169199291,12cyclictest0-21swapper/320:39:373
1691992910,3cyclictest0-21swapper/323:30:483
168799299,3cyclictest0-21swapper/220:19:052
168799297,3cyclictest0-21swapper/219:46:572
168799296,3cyclictest0-21swapper/223:04:562
168799296,3cyclictest0-21swapper/223:04:562
168799296,3cyclictest0-21swapper/200:16:252
168799295,3cyclictest0-21swapper/220:31:252
1687992915,3cyclictest4399-21grep21:40:172
168799291,13cyclictest0-21swapper/221:14:012
167499299,17cyclictest1884-21/usr/sbin/munin19:10:160
167499299,17cyclictest16876-21sort22:10:010
167499299,17cyclictest14327-21cat19:35:170
167499298,19cyclictest14399-21grep00:30:190
167499297,19cyclictest27932-21unixbench_multi23:45:230
167499297,19cyclictest22049-21sed22:20:170
167499293,3cyclictest18677-21cut21:00:130
167499293,3cyclictest12050irq/123-ahci[0021:40:000
167499291,4cyclictest8612-21/usr/sbin/munin20:35:210
167499291,4cyclictest24358-21/usr/sbin/munin22:25:170
167499291,4cyclictest19063-21/usr/sbin/munin19:45:240
167499291,3cyclictest12057-21/usr/sbin/munin23:10:210
1674992912,14cyclictest16370-21/usr/sbin/munin20:55:230
169199289,3cyclictest0-21swapper/323:47:583
169199288,3cyclictest0-21swapper/320:34:013
1691992811,3cyclictest0-21swapper/322:19:533
1691992811,3cyclictest0-21swapper/320:56:083
1691992811,3cyclictest0-21swapper/320:01:173
169199281,13cyclictest0-21swapper/319:44:253
169199281,12cyclictest0-21swapper/322:35:333
169199281,12cyclictest0-21swapper/321:55:293
169199281,12cyclictest0-21swapper/321:31:493
169199281,11cyclictest0-21swapper/322:11:253
169199281,11cyclictest0-21swapper/320:41:173
169199281,11cyclictest0-21swapper/320:41:173
1691992810,3cyclictest0-21swapper/319:11:013
168799288,9cyclictest0-21swapper/223:20:172
168799288,3cyclictest0-21swapper/222:26:252
168799288,3cyclictest0-21swapper/220:03:532
168799288,3cyclictest0-21swapper/200:10:222
168799285,3cyclictest0-21swapper/221:03:252
168799283,3cyclictest0-21swapper/221:33:332
1687992810,8cyclictest0-21swapper/219:40:492
168799280,17cyclictest0-21swapper/222:20:282
168099287,3cyclictest0-21swapper/122:24:191
167499289,16cyclictest827-21runrttasks21:50:070
167499288,17cyclictest2925-21ls00:05:120
167499288,17cyclictest2925-21ls00:05:120
167499281,3cyclictest29951-21/usr/sbin/munin20:10:210
167499281,3cyclictest20302-21tr19:50:130
167499281,3cyclictest18000-21needreboot22:10:210
169199279,3cyclictest8935-21latency_hist19:25:003
169199279,3cyclictest0-21swapper/321:38:453
169199279,3cyclictest0-21swapper/321:03:323
169199279,3cyclictest0-21swapper/320:10:493
169199276,3cyclictest0-21swapper/320:16:133
1691992711,3cyclictest0-21swapper/319:49:093
169199271,12cyclictest0-21swapper/322:44:013
169199271,11cyclictest0-21swapper/321:07:563
169199271,11cyclictest0-21swapper/319:36:253
169199271,10cyclictest10354-21latency00:20:203
169199271,10cyclictest0-21swapper/319:25:443
1691992710,3cyclictest0-21swapper/323:23:363
1691992710,3cyclictest0-21swapper/321:40:093
1691992710,3cyclictest0-21swapper/300:34:333
168799279,9cyclictest0-21swapper/221:58:052
168799279,3cyclictest0-21swapper/222:19:572
168799275,3cyclictest0-21swapper/220:37:522
168799275,3cyclictest0-21swapper/219:25:412
168799274,3cyclictest0-21swapper/222:13:372
168799272,3cyclictest0-21swapper/219:56:132
1687992712,3cyclictest29667-21/usr/sbin/munin23:50:222
1687992712,3cyclictest0-21swapper/200:36:302
168799271,15cyclictest0-21swapper/220:45:082
168799271,12cyclictest0-21swapper/221:35:132
168799271,11cyclictest0-21swapper/200:09:212
168799271,11cyclictest0-21swapper/200:09:212
1687992710,7cyclictest5484-21ntp_states22:55:202
1687992710,3cyclictest0-21swapper/220:24:012
168799270,12cyclictest0-21swapper/223:10:562
167499271,3cyclictest6976-21/usr/sbin/munin19:20:130
169199269,3cyclictest0-21swapper/323:26:083
169199269,3cyclictest0-21swapper/322:33:493
169199269,3cyclictest0-21swapper/322:04:493
169199269,3cyclictest0-21swapper/319:15:053
169199269,3cyclictest0-21swapper/300:18:173
169199268,3cyclictest0-21swapper/323:58:543
169199267,3cyclictest0-21swapper/300:28:303
169199261,9cyclictest0-21swapper/321:46:133
1691992610,3cyclictest0-21swapper/321:12:043
1691992610,3cyclictest0-21swapper/300:04:013
168799269,4cyclictest0-21swapper/200:00:252
168799268,3cyclictest10531-40dnf00:20:252
168799264,3cyclictest0-21swapper/221:15:492
168799263,3cyclictest0-21swapper/221:29:172
1687992611,3cyclictest0-21swapper/221:52:172
168799261,12cyclictest0-21swapper/223:40:012
168799261,11cyclictest0-21swapper/219:18:162
1687992610,4cyclictest12434-21ntp_states19:30:212
1687992610,4cyclictest12434-21ntp_states19:30:212
168099261,9cyclictest0-21swapper/121:56:221
168099261,9cyclictest0-21swapper/100:09:511
168099261,9cyclictest0-21swapper/100:09:511
169199259,3cyclictest0-21swapper/300:08:493
169199259,3cyclictest0-21swapper/300:08:493
169199258,3cyclictest0-21swapper/323:02:213
169199258,3cyclictest0-21swapper/323:02:203
169199257,3cyclictest0-21swapper/321:22:213
169199256,3cyclictest0-21swapper/323:15:083
169199256,3cyclictest0-21swapper/322:06:373
168799258,3cyclictest15148-21diskstats22:05:132
168799257,3cyclictest0-21swapper/200:34:222
168799257,3cyclictest0-21swapper/200:28:062
168799256,3cyclictest0-21swapper/219:21:292
168799255,3cyclictest0-21swapper/223:29:532
168799255,3cyclictest0-21swapper/222:51:082
168799255,3cyclictest0-21swapper/222:40:482
168799255,3cyclictest0-21swapper/222:31:332
168799255,3cyclictest0-21swapper/219:38:132
1687992511,3cyclictest0-21swapper/222:45:202
168799250,13cyclictest0-21swapper/221:49:292
1680992517,2cyclictest18031-21awk00:40:011
168099251,6cyclictest0-21swapper/121:29:351
168099250,19cyclictest0-21swapper/122:08:551
169199249,3cyclictest0-21swapper/322:52:373
169199248,3cyclictest0-21swapper/321:28:013
169199246,3cyclictest0-21swapper/323:38:483
169199245,3cyclictest0-21swapper/300:14:493
169199241,13cyclictest0-21swapper/323:08:083
168799246,3cyclictest0-21swapper/222:01:332
168799241,3cyclictest15093-21/usr/sbin/munin20:50:202
1687992411,3cyclictest0-21swapper/223:41:582
168099241,6cyclictest0-21swapper/122:46:011
168099241,16cyclictest22743-21diskmemload21:52:571
168099241,12cyclictest0-21swapper/100:21:231
168099240,5cyclictest0-21swapper/121:33:281
169199235,3cyclictest0-21swapper/322:47:093
169199231,14cyclictest0-21swapper/323:40:403
168799239,2cyclictest22743-21diskmemload23:32:542
168799237,3cyclictest0-21swapper/221:22:292
168099233,10cyclictest27632-21expr21:20:141
168099232,5cyclictest21-21rcuc/123:10:161
168099231,6cyclictest0-21swapper/123:38:521
168099231,6cyclictest0-21swapper/122:57:121
168099231,16cyclictest0-21swapper/120:51:351
168099231,13cyclictest0-21swapper/122:04:111
168099231,11cyclictest7765-21cstates20:35:131
168099231,11cyclictest0-21swapper/121:12:431
169199229,3cyclictest0-21swapper/321:54:533
169199226,3cyclictest0-21swapper/323:13:283
168799224,3cyclictest0-21swapper/223:15:492
168099224,3cyclictest0-21swapper/122:32:281
168099222,11cyclictest28334-21diskstats22:35:151
168099221,6cyclictest0-21swapper/100:17:381
1680992216,3cyclictest29089-21cut23:50:161
168099221,5cyclictest0-21swapper/121:47:331
168099221,4cyclictest11689-21cpu00:25:131
168099221,2cyclictest0-21swapper/119:35:201
168099221,19cyclictest0-21swapper/123:26:591
168099221,16cyclictest0-21swapper/121:08:201
168099221,15cyclictest0-21swapper/123:45:161
168099221,13cyclictest0-21swapper/123:32:011
1680992210,3cyclictest27079-40dnf21:19:231
168099220,4cyclictest12292-21kworker/u8:122:13:571
168099220,16cyclictest0-21swapper/122:44:471
168799217,3cyclictest0-21swapper/219:50:292
168099216,4cyclictest0-21swapper/121:41:531
168099211,9cyclictest31849-21ntpq23:55:211
168099211,5cyclictest0-21swapper/123:18:321
168099211,4cyclictest32590-21kworker/u8:000:10:591
1680992114,2cyclictest0-21swapper/122:16:461
168099211,3cyclictest0-21swapper/100:31:451
1680992113,2cyclictest0-21swapper/119:12:151
168099211,19cyclictest0-21swapper/123:20:401
1680992110,3cyclictest8209-21users23:00:271
1680992110,3cyclictest8209-21users23:00:271
168099210,16cyclictest0-21swapper/121:02:321
168099210,15cyclictest0-21swapper/120:30:071
31942200,2sleep10-21swapper/121:35:271
168099208,3cyclictest4188-21sh22:55:011
168099201,3cyclictest0-21swapper/120:57:061
1680992013,2cyclictest0-21swapper/122:25:311
168099201,14cyclictest0-21swapper/120:45:531
168099201,14cyclictest0-21swapper/119:43:421
168099201,14cyclictest0-21swapper/119:34:001
168099201,14cyclictest0-21swapper/119:34:001
168099201,13cyclictest0-21swapper/123:40:521
168099201,11cyclictest1908-21memory20:20:181
168099201,10cyclictest9000-21ls23:05:141
168099201,10cyclictest790-21in:imjournal19:55:011
168099201,10cyclictest763-21dbus-daemon20:10:041
168099200,3cyclictest0-21swapper/119:21:071
168099200,14cyclictest0-21swapper/120:05:091
168099198,3cyclictest790-21in:imjournal00:01:021
168099191,9cyclictest6651-21seq19:19:011
1680991915,3cyclictest0-21swapper/120:19:541
168099191,14cyclictest0-21swapper/120:02:471
168099191,14cyclictest0-21swapper/119:46:241
168099191,11cyclictest0-21swapper/120:25:141
168099191,10cyclictest22620-21expr19:55:131
168099191,10cyclictest10162-21cut19:25:201
169199181,2cyclictest0-21swapper/319:50:133
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional