You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-02 - 13:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Wed Sep 02, 2026 00:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
149462252193,20sleep20-21swapper/219:06:222
150752232162,23sleep10-21swapper/119:08:021
149762205176,20sleep00-21swapper/019:06:440
151922202165,24sleep30-21swapper/319:09:333
323482950,1sleep00-21swapper/023:25:210
15320996057,2cyclictest0-21swapper/222:33:192
213902500,2sleep10-21swapper/119:20:181
15320994945,3cyclictest0-21swapper/221:56:532
142212440,1sleep10-21swapper/100:00:011
226592430,2sleep20-21swapper/223:05:122
1532599410,30cyclictest0-21swapper/323:07:013
1531399411,36cyclictest4867-21/usr/sbin/munin22:25:191
1532599391,28cyclictest0-21swapper/321:59:013
1531399380,32cyclictest0-21swapper/121:20:191
1532599371,27cyclictest0-21swapper/321:39:533
1532599370,30cyclictest0-21swapper/323:41:453
1532099370,23cyclictest0-21swapper/200:38:092
1531399377,14cyclictest19593-21sort00:10:191
1531399371,32cyclictest28920-21fschecks_time22:05:151
1531399371,23cyclictest21509-21grep00:15:171
1531399371,23cyclictest15859-21/usr/sbin/munin22:50:191
1531399371,21cyclictest26857-21/usr/sbin/munin23:15:191
15320993633,2cyclictest0-21swapper/221:04:392
1531399361,26cyclictest8027-21/usr/sbin/munin20:05:161
1532599351,27cyclictest0-21swapper/321:22:453
1532099351,25cyclictest0-21swapper/220:57:192
1531399352,29cyclictest30349-21grep00:35:161
1531399352,23cyclictest12050irq/123-ahci[0019:35:021
1531399351,31cyclictest6168-21/usr/sbin/munin20:00:141
1531399351,31cyclictest17498-21idleruntime-cro19:15:011
1531399351,31cyclictest13437-21awk22:45:011
1531399351,31cyclictest13437-21awk22:45:011
1531399351,30cyclictest16852-21/usr/sbin/munin00:05:151
1531399351,22cyclictest11869-21sleep23:55:011
1531399351,22cyclictest11869-21sleep23:55:011
1531399351,20cyclictest31431-21cstates23:25:121
1530499358,12cyclictest24450-21perf23:10:010
1532599341,21cyclictest0-21swapper/322:02:053
1532599341,20cyclictest0-21swapper/323:13:413
1532599341,20cyclictest0-21swapper/321:40:133
1532599341,19cyclictest0-21swapper/320:15:213
1532099341,25cyclictest0-21swapper/219:44:512
1531399341,31cyclictest19423-21awk21:45:011
1531399341,30cyclictest0-21swapper/122:01:011
1531399341,29cyclictest4015-21latency_hist21:10:011
1531399341,29cyclictest25909-21mailstats23:10:241
1531399341,23cyclictest29318-21chrt00:33:161
1531399341,23cyclictest1070-21/usr/sbin/munin23:30:121
1531399341,20cyclictest14659-21/usr/sbin/munin22:45:221
1531399341,19cyclictest29052-21/usr/sbin/munin23:20:241
1531399341,17cyclictest25068-21ntpq21:55:221
1531399340,24cyclictest0-21swapper/121:50:251
152782340,5sleep015282-21gltestperf20:20:160
81042330,4sleep01530499cyclictest21:16:110
1532599331,24cyclictest0-21swapper/300:21:473
1532599331,22cyclictest0-21swapper/323:25:013
1532599331,19cyclictest0-21swapper/322:28:253
1532099331,14cyclictest4018-21diskmemload21:50:152
1531399338,18cyclictest0-21swapper/120:25:141
1531399336,16cyclictest7621-21cat23:45:011
1531399332,27cyclictest23448-21diskstats00:20:141
1531399331,28cyclictest766-21ntpd19:46:151
1531399331,28cyclictest24436-21munin-run23:10:011
1531399331,21cyclictest361-21/usr/sbin/munin22:15:161
1531399331,20cyclictest30732-21expr22:10:131
1531399331,20cyclictest15975-21awk21:35:161
1531399331,16cyclictest15171-21latency_hist21:35:011
1531399331,16cyclictest15171-21latency_hist21:35:011
1531399330,21cyclictest0-21swapper/119:55:161
1531399330,19cyclictest0-21swapper/120:40:201
1530499334,14cyclictest22242-21munin-run23:05:010
1532599321,25cyclictest0-21swapper/321:00:053
1532599321,21cyclictest0-21swapper/300:32:513
1532599321,13cyclictest0-21swapper/320:40:453
1532599320,20cyclictest0-21swapper/323:50:293
1532599320,20cyclictest0-21swapper/323:50:293
1531399328,11cyclictest0-21swapper/123:45:131
1531399326,17cyclictest0-21swapper/123:40:011
1531399323,24cyclictest27810-21/usr/sbin/munin20:50:271
1531399321,28cyclictest7128-21cpuspeed_turbos22:30:131
1531399321,28cyclictest27263-21latency_hist00:30:021
1531399321,27cyclictest4665-21sort22:25:011
1531399321,27cyclictest19830-21interrupts20:30:191
1531399321,27cyclictest19567-21unixbench_multi22:55:251
1531399321,21cyclictest6707-21cstates21:15:161
1531399321,19cyclictest12409-21unixbench_singl21:25:251
15325993118,3cyclictest0-21swapper/320:50:293
1532599311,24cyclictest0-21swapper/300:15:513
1532599311,22cyclictest0-21swapper/322:06:373
1532599311,20cyclictest0-21swapper/319:53:333
1532599311,18cyclictest0-21swapper/322:37:333
1532599311,18cyclictest0-21swapper/321:51:213
1532599311,14cyclictest0-21swapper/321:49:173
1532599310,18cyclictest0-21swapper/300:29:193
1532099311,12cyclictest0-21swapper/220:45:152
1532099310,25cyclictest0-21swapper/222:27:352
1531399317,9cyclictest0-21swapper/119:20:011
1531399311,27cyclictest3394-21latency_hist19:55:011
1531399311,27cyclictest21623-21timerandwakeup23:00:231
1531399311,27cyclictest1414-21chrt21:00:261
1531399311,26cyclictest15891-21smart_sda20:20:241
1531399311,19cyclictest774-21avahi-daemon20:50:011
1531399311,19cyclictest579-21systemd-journal20:16:211
1531399311,18cyclictest4249-21/usr/sbin/munin21:10:211
1531399310,29cyclictest0-21swapper/119:40:201
1530499311,14cyclictest23805-21munin-run21:55:010
180052300,4sleep01530499cyclictest19:15:030
1532599301,20cyclictest0-21swapper/322:41:533
1532599301,20cyclictest0-21swapper/322:41:533
1532599301,13cyclictest0-21swapper/323:58:153
1532599301,13cyclictest0-21swapper/319:11:333
1532599301,12cyclictest26878-21smart_sda20:45:253
1532599300,23cyclictest0-21swapper/321:11:453
1532599300,14cyclictest0-21swapper/323:34:093
1532599300,12cyclictest0-21swapper/319:58:573
15320993023,2cyclictest0-21swapper/200:13:592
1532099301,13cyclictest0-21swapper/220:02:472
1532099300,18cyclictest0-21swapper/222:48:072
1532099300,12cyclictest0-21swapper/221:11:192
1531399301,26cyclictest15100-21/usr/sbin/munin00:00:191
1531399300,27cyclictest10004-21/usr/sbin/munin22:35:171
15325992917,3cyclictest0-21swapper/300:07:513
1532599291,19cyclictest0-21swapper/300:37:113
1532599291,13cyclictest0-21swapper/321:29:373
1532599291,11cyclictest0-21swapper/319:28:053
1532599291,11cyclictest0-21swapper/300:12:593
1532599290,15cyclictest0-21swapper/321:34:373
1532599290,15cyclictest0-21swapper/321:34:373
1532599290,13cyclictest0-21swapper/323:38:413
1532599290,12cyclictest0-21swapper/323:17:173
1532099299,11cyclictest0-21swapper/222:40:152
1532099299,11cyclictest0-21swapper/222:40:152
1532099291,9cyclictest11892-21ntpq21:25:192
1532099291,12cyclictest0-21swapper/220:52:512
1532099290,21cyclictest0-21swapper/222:01:152
1532099290,12cyclictest0-21swapper/223:32:432
1531399291,25cyclictest22800-21/usr/sbin/munin19:25:221
1531399291,25cyclictest21218-21/usr/sbin/munin20:35:121
1531399291,25cyclictest19638-21/usr/sbin/munin21:45:151
1531399291,25cyclictest0-21swapper/120:55:201
1531399291,19cyclictest0-21swapper/119:39:291
1531399291,14cyclictest10227-21/usr/sbin/munin20:10:211
1532599281,18cyclictest0-21swapper/320:12:253
1532599281,12cyclictest0-21swapper/319:17:493
1532599281,11cyclictest0-21swapper/322:58:213
1532599281,11cyclictest0-21swapper/322:20:213
1532599280,19cyclictest0-21swapper/323:23:373
1532099281,17cyclictest0-21swapper/200:02:332
1532099281,12cyclictest0-21swapper/219:30:512
1532099281,11cyclictest8865-21cut23:45:232
1532099281,10cyclictest0-21swapper/221:15:232
1532099280,11cyclictest0-21swapper/221:21:032
1532099280,11cyclictest0-21swapper/220:22:552
1532099280,10cyclictest0-21swapper/221:43:032
15304992810,15cyclictest10615-21munin-run21:25:010
1532599271,2cyclictest0-21swapper/320:37:533
1532599271,2cyclictest0-21swapper/319:41:453
1532599271,11cyclictest0-21swapper/323:00:213
1532599271,11cyclictest0-21swapper/322:50:093
1532599271,11cyclictest0-21swapper/322:32:013
1532599270,12cyclictest0-21swapper/320:57:093
1532599270,10cyclictest0-21swapper/322:10:273
1532099271,9cyclictest0-21swapper/219:26:312
1532099271,12cyclictest0-21swapper/219:23:592
1532099271,12cyclictest0-21swapper/200:16:212
1532099271,11cyclictest0-21swapper/219:17:512
1532099271,11cyclictest0-21swapper/219:14:272
1532099271,10cyclictest27694-21expr00:30:132
1532099271,10cyclictest0-21swapper/223:03:192
1532099271,10cyclictest0-21swapper/219:45:512
1532099270,9cyclictest0-21swapper/223:52:392
1532099270,9cyclictest0-21swapper/223:52:392
1532099270,8cyclictest0-21swapper/222:55:192
1532099270,13cyclictest0-21swapper/222:36:072
1532099270,10cyclictest0-21swapper/220:12:072
1530499278,16cyclictest25089-21basename00:25:010
15304992712,13cyclictest0-21swapper/022:05:010
272122260,2sleep20-21swapper/219:35:122
1532599261,13cyclictest0-21swapper/321:07:293
1532599261,13cyclictest0-21swapper/300:01:353
1532599261,12cyclictest0-21swapper/320:22:493
1532599261,12cyclictest0-21swapper/320:07:373
1532599261,12cyclictest0-21swapper/319:46:013
1532599261,12cyclictest0-21swapper/319:24:093
1532599261,11cyclictest0-21swapper/321:16:213
1532599260,13cyclictest0-21swapper/322:49:373
1532599260,12cyclictest0-21swapper/322:16:143
1532099263,10cyclictest20082-40dnf20:30:242
1532099261,11cyclictest0-21swapper/223:58:022
1532099261,11cyclictest0-21swapper/219:50:472
1532099261,10cyclictest0-21swapper/220:41:092
1532099260,13cyclictest0-21swapper/222:14:072
1532099260,11cyclictest0-21swapper/223:40:272
1532099260,11cyclictest0-21swapper/223:28:112
1532099260,10cyclictest0-21swapper/220:27:032
1532599251,12cyclictest0-21swapper/320:32:213
1532599251,10cyclictest0-21swapper/323:46:213
1532599250,11cyclictest0-21swapper/320:01:193
1532599250,10cyclictest0-21swapper/319:30:313
1532099258,12cyclictest0-21swapper/200:24:032
1532099251,8cyclictest0-21swapper/220:06:432
1532099251,11cyclictest0-21swapper/222:18:352
1532099251,11cyclictest0-21swapper/220:37:432
1532099250,12cyclictest0-21swapper/223:16:472
1532099250,11cyclictest0-21swapper/221:48:072
1532099250,11cyclictest0-21swapper/200:25:132
1532099250,10cyclictest0-21swapper/223:38:512
1532099250,10cyclictest0-21swapper/223:23:272
1532099250,10cyclictest0-21swapper/223:10:312
1530499256,16cyclictest0-21swapper/019:30:010
1532599241,2cyclictest0-21swapper/320:25:173
1532099248,10cyclictest0-21swapper/222:53:132
1532099241,11cyclictest0-21swapper/221:30:112
1532099241,11cyclictest0-21swapper/221:30:112
1532099240,9cyclictest0-21swapper/222:21:432
97802230,4sleep011-21rcuc/023:50:010
1532099236,3cyclictest0-21swapper/219:58:072
1532099230,11cyclictest0-21swapper/220:18:312
1532099230,10cyclictest0-21swapper/221:39:232
1532099227,3cyclictest0-21swapper/221:09:392
1532099227,3cyclictest0-21swapper/200:09:412
1530499221,5cyclictest0-21swapper/022:30:080
1530499221,4cyclictest0-21swapper/000:35:080
1532599211,13cyclictest0-21swapper/319:35:093
1530499216,14cyclictest0-21swapper/019:40:010
1530499211,4cyclictest0-21swapper/022:40:160
1530499211,4cyclictest0-21swapper/022:40:160
1530499211,17cyclictest5599-21munin-run20:00:010
1532099201,12cyclictest0-21swapper/222:06:062
1530499209,3cyclictest13296-21sshd23:55:220
1530499202,16cyclictest3062-21munin-run23:35:010
1530499201,7cyclictest18777-21/usr/sbin/munin21:40:240
1530499201,6cyclictest0-21swapper/023:16:350
1530499201,6cyclictest0-21swapper/000:09:560
15304992013,4cyclictest1708-21find22:15:250
1530499201,12cyclictest25000-21/usr/sbin/munin19:30:280
1530499201,11cyclictest13038-21/usr/sbin/munin21:30:120
1530499201,11cyclictest13038-21/usr/sbin/munin21:30:120
1530499201,11cyclictest10838-21/usr/sbin/munin21:25:160
1530499201,11cyclictest0-21swapper/023:42:500
33862190,15sleep00-21swapper/019:55:010
1530499196,4cyclictest0-21swapper/022:22:420
1530499195,3cyclictest0-21swapper/022:55:480
1530499191,6cyclictest0-21swapper/000:27:250
1530499191,4cyclictest15234-21/usr/sbin/munin21:35:210
1530499191,11cyclictest10240-21forks23:50:150
1530499191,11cyclictest10240-21forks23:50:150
1530499191,11cyclictest0-21swapper/000:10:200
1530499186,5cyclictest5911-21ntp_kernel_pll_22:25:210
1530499185,4cyclictest0-21swapper/021:46:230
1530499185,3cyclictest14650-21needreboot22:45:210
1530499183,3cyclictest0-21swapper/022:53:410
1530499181,5cyclictest0-21swapper/023:10:510
15304991813,3cyclictest5240-21needreboot21:10:180
1530499181,14cyclictest31292-21sed22:10:170
1530499181,11cyclictest0-21swapper/022:09:210
1530499181,11cyclictest0-21swapper/000:00:240
1530499181,10cyclictest22511-21sshd19:24:220
1530499176,3cyclictest0-21swapper/023:24:450
1530499174,4cyclictest866-21ntp_states21:00:210
1530499172,3cyclictest0-21swapper/023:36:420
1530499171,9cyclictest0-21swapper/022:35:170
1530499171,9cyclictest0-21swapper/021:55:020
1530499171,6cyclictest26220-21head20:45:180
15304991712,3cyclictest30978-21uptime19:40:250
15304991711,4cyclictest29773-21crond20:55:010
1530499171,13cyclictest12422-21/usr/sbin/munin20:15:260
1530499171,10cyclictest0-21swapper/000:33:180
1530499163,3cyclictest8631-21tail20:05:150
1530499161,4cyclictest15515-21/usr/sbin/munin19:10:130
1530499161,3cyclictest0-21swapper/020:04:420
1530499161,14cyclictest0-21swapper/019:48:170
15304991611,3cyclictest31594-21uptime20:55:270
15304991611,3cyclictest23416-21/usr/sbin/munin20:40:250
15304991611,3cyclictest21498-21grep00:15:170
15304991611,3cyclictest19024-21/usr/sbin/munin20:30:200
15304991611,3cyclictest11290-21ntp_offset20:10:220
15304991512,2cyclictest0-21swapper/020:28:130
15304991510,3cyclictest2350-21cat21:05:120
15304991510,3cyclictest21565-21diskstats20:35:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional