You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-01 - 20:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Tue Sep 01, 2026 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
212212721240,3sleep03056-21PK-Backend07:55:160
214841921180,3sleep342-21ksoftirqd/309:10:183
221931721140,4sleep30-21swapper/312:31:323
210442127538,14sleep10-21swapper/107:06:251
210464227252,14sleep30-21swapper/307:09:163
210467326341,16sleep20-21swapper/207:09:402
210444626244,13sleep00-21swapper/007:06:440
21707322450,5sleep13056-21PK-Backend10:15:041
21205692440,4sleep20-21swapper/207:50:212
22112242430,3sleep30-21swapper/312:10:183
22069352410,5chrt0-21swapper/012:00:010
21712662410,2sleep20-21swapper/210:15:172
210484899399,28cyclictest2152836-21apt-key09:25:013
210484899285,14cyclictest0-21swapper/309:57:123
21667902260,4sleep02166784-21awk10:05:000
210484899269,15cyclictest2139573-21sed08:45:153
210484499261,5cyclictest2115984-21munin-plugin-st07:40:002
210484499261,5cyclictest2115984-21munin-plugin-st07:40:002
210483999264,6cyclictest2130121-21apt08:20:000
210484899258,15cyclictest2156364-21cron09:35:003
210484899258,15cyclictest2115254-21wc07:35:223
210484899258,15cyclictest2115254-21wc07:35:213
210484899258,15cyclictest2109722-21cat07:20:173
210484199256,17cyclictest2155826-21grep09:30:211
22146472240,3sleep13056-21PK-Backend12:20:111
21512822240,10sleep013-21ksoftirqd/009:20:130
210484899248,14cyclictest2127302-21hddtemp_smartct08:10:153
210484899245,18cyclictest2163333-21basename09:55:003
2104848992410,12cyclictest0-21swapper/312:00:183
2104844992419,3cyclictest2174973-21ntp_states10:25:202
210484899239,12cyclictest2195010-21/usr/sbin/munin11:25:153
210484899238,13cyclictest2214729-21diskstats12:20:123
210484899238,13cyclictest2169511-21ntp_states10:10:193
210484899238,13cyclictest2138064-21ntp_states08:40:203
210484899232,12cyclictest0-21swapper/310:20:183
2104848992319,3cyclictest0-21swapper/308:07:253
2104848992316,3cyclictest2167043-21apt10:05:123
210484899231,19cyclictest2219895-21apt12:35:123
2104844992317,4cyclictest2122942-21systemd08:00:002
2104844992314,6cyclictest2165056-21ls10:00:002
2104841992312,10cyclictest3056-21PK-Backend11:13:461
210484199230,22cyclictest3056-21PK-Backend12:10:491
139099230,4rtkit-daemon1389-21rtkit-daemon11:05:480
139099230,4rtkit-daemon1389-21rtkit-daemon09:19:300
139099230,4rtkit-daemon1389-21rtkit-daemon07:34:560
139099230,4rtkit-daemon1389-21rtkit-daemon07:14:440
210484899228,13cyclictest2200508-21cstates11:40:143
210484899228,12cyclictest2205026-21sshd11:53:273
210484899228,12cyclictest2119771-21ls07:50:123
210484899227,13cyclictest2185742-21runrttasks10:55:373
2104848992214,5cyclictest2205406-21/usr/sbin/munin11:55:253
2104844992220,1cyclictest0-21swapper/209:40:522
2104844992220,1cyclictest0-21swapper/209:36:072
2104844992220,1cyclictest0-21swapper/208:32:182
2104844992220,1cyclictest0-21swapper/207:33:372
2104844992220,1cyclictest0-21swapper/207:16:272
210484499220,20cyclictest2129932-21taskset08:17:522
210484499220,20cyclictest0-21swapper/208:51:582
210484499220,20cyclictest0-21swapper/208:10:092
210484199227,14cyclictest3056-21PK-Backend08:02:351
210484199226,14cyclictest0-21swapper/109:50:261
210484199225,15cyclictest2200075-21sed11:40:011
210484199225,15cyclictest2125611-21awk08:05:171
2104841992220,1cyclictest3056-21PK-Backend11:17:561
2104841992212,9cyclictest3056-21PK-Backend10:11:351
2104841992212,9cyclictest3056-21PK-Backend08:15:021
210484199221,19cyclictest2146824-21cut09:05:221
210484199220,21cyclictest3056-21PK-Backend11:50:211
210484199220,1cyclictest3056-21PK-Backend08:13:431
21322252210,4sleep13056-21PK-Backend08:25:121
210484899217,12cyclictest2192487-21switchtime11:15:223
210484899217,12cyclictest2160978-21smartctl09:45:223
210484899215,14cyclictest2186368-21apt11:00:123
2104848992113,6cyclictest2193430-21expr11:20:123
2104848992112,6cyclictest436-20systemd-journal12:15:043
2104848992111,9cyclictest2178569-21python310:35:233
2104848992111,8cyclictest2145723-21awk09:05:003
2104848992110,9cyclictest2177441-21munin-plugin-st10:35:003
2104844992119,1cyclictest0-21swapper/212:15:182
2104844992119,1cyclictest0-21swapper/212:06:122
2104844992119,1cyclictest0-21swapper/212:00:452
2104844992119,1cyclictest0-21swapper/211:44:082
2104844992119,1cyclictest0-21swapper/211:31:522
2104844992119,1cyclictest0-21swapper/211:03:392
2104844992119,1cyclictest0-21swapper/210:06:342
2104844992119,1cyclictest0-21swapper/209:32:072
2104844992119,1cyclictest0-21swapper/209:02:332
2104844992119,1cyclictest0-21swapper/208:46:242
210484499210,20cyclictest0-21swapper/212:23:222
210484499210,20cyclictest0-21swapper/211:37:532
210484499210,20cyclictest0-21swapper/211:22:122
210484499210,20cyclictest0-21swapper/211:12:322
210484199219,10cyclictest3056-21PK-Backend08:54:041
210484199217,12cyclictest3056-21PK-Backend09:04:461
210484199216,14cyclictest3056-21PK-Backend08:40:481
2104841992119,1cyclictest3056-21PK-Backend12:38:401
2104841992119,1cyclictest3056-21PK-Backend12:31:511
2104841992119,1cyclictest3056-21PK-Backend11:47:441
2104841992119,1cyclictest3056-21PK-Backend11:01:201
2104841992119,1cyclictest3056-21PK-Backend10:24:431
2104841992119,1cyclictest3056-21PK-Backend09:25:351
2104841992118,2cyclictest3056-21PK-Backend07:56:331
2104841992118,2cyclictest3056-21PK-Backend07:45:241
2104839992112,7cyclictest2192148-21memory11:15:180
210483999210,1cyclictest0-21swapper/010:54:090
210483999210,19cyclictest2172226-21cron10:20:000
139099210,3rtkit-daemon1389-21rtkit-daemon10:33:520
139099210,18rtkit-daemon1997-21gvfs-afc-volume08:12:110
139099210,17rtkit-daemon2104833-21cyclictest07:53:460
210484899204,15cyclictest0-21swapper/311:05:223
2104848992017,2cyclictest2210190-21runrttasks12:08:033
2104848992017,2cyclictest0-21swapper/310:40:293
2104848992017,2cyclictest0-21swapper/308:15:373
2104848992014,5cyclictest2121615-21cpuspeed_turbos07:55:113
2104848992012,5cyclictest2112728-21apt07:30:123
210484899201,17cyclictest436-20systemd-journal07:20:013
2104848992011,7cyclictest2142271-21basename08:55:003
2104848992011,7cyclictest2130389-21/usr/sbin/munin08:20:123
2104848992011,7cyclictest2112066-21users07:25:223
2104848992010,8cyclictest2217582-21basename12:30:003
2104848992010,8cyclictest2198231-21ls11:35:003
210484899200,1cyclictest0-21swapper/308:32:003
210484899200,19cyclictest0-21swapper/310:50:293
210484899200,19cyclictest0-21swapper/310:47:053
210484899200,19cyclictest0-21swapper/309:37:293
210484899200,19cyclictest0-21swapper/309:27:223
210484899200,19cyclictest0-21swapper/307:44:203
210484499205,13cyclictest2124629-21cron08:05:002
2104844992019,1cyclictest0-21swapper/209:20:242
2104844992018,1cyclictest0-21swapper/211:05:292
2104844992018,1cyclictest0-21swapper/210:55:492
2104844992018,1cyclictest0-21swapper/210:41:092
2104844992018,1cyclictest0-21swapper/210:34:192
2104844992018,1cyclictest0-21swapper/209:46:272
2104844992018,1cyclictest0-21swapper/209:07:212
2104844992018,1cyclictest0-21swapper/208:43:132
2104844992018,1cyclictest0-21swapper/208:27:042
2104844992018,1cyclictest0-21swapper/208:06:292
2104844992018,1cyclictest0-21swapper/207:29:392
2104844992014,5cyclictest2178349-21needreboot10:35:202
2104844992013,5cyclictest2172628-21expr10:20:102
2104844992011,7cyclictest2170240-21apt-get10:15:002
210484499200,1cyclictest0-21swapper/212:14:172
210484499200,1cyclictest0-21swapper/209:18:012
210484499200,19cyclictest0-21swapper/212:33:162
210484499200,19cyclictest0-21swapper/211:26:382
210484499200,19cyclictest0-21swapper/210:45:212
210484499200,19cyclictest0-21swapper/209:50:262
210484499200,19cyclictest0-21swapper/208:24:342
210484499200,19cyclictest0-21swapper/207:12:492
210484199206,13cyclictest3056-21PK-Backend11:44:061
210484199203,15cyclictest0-21swapper/107:10:021
2104841992018,1cyclictest3056-21PK-Backend11:57:101
2104841992018,1cyclictest3056-21PK-Backend11:06:201
2104841992018,1cyclictest3056-21PK-Backend10:26:181
2104841992018,1cyclictest3056-21PK-Backend08:35:531
210484199201,7cyclictest2131245-21python308:20:211
210484199201,6cyclictest0-21swapper/112:25:131
2104841992016,3cyclictest3056-21PK-Backend09:22:381
210484199200,1cyclictest3056-21PK-Backend08:32:081
2104839992018,1cyclictest0-21swapper/011:12:270
2104839992018,1cyclictest0-21swapper/010:06:020
2104839992018,1cyclictest0-21swapper/009:49:200
210483999201,6cyclictest2124650-21perf08:05:010
210483999200,2cyclictest0-21swapper/008:38:290
210483999200,1cyclictest0-21swapper/009:35:130
210483999200,19cyclictest0-21swapper/012:15:540
210483999200,19cyclictest0-21swapper/012:07:500
210483999200,19cyclictest0-21swapper/010:46:310
210483999200,19cyclictest0-21swapper/009:55:420
210483999200,19cyclictest0-21swapper/007:41:580
210483999200,19cyclictest0-21swapper/007:35:370
210483999200,19cyclictest0-21swapper/007:35:360
210483999200,18cyclictest0-21swapper/012:35:140
139099200,16rtkit-daemon2104833-21cyclictest10:12:020
139099200,16rtkit-daemon1788643-21eadedCompositor11:38:400
21311042190,3sleep013-21ksoftirqd/008:20:180
21120142190,4sleep13056-21PK-Backend07:25:231
210484899199,8cyclictest2202844-21/usr/sbin/munin11:45:223
210484899199,8cyclictest2190329-21kernelversion11:10:183
210484899199,7cyclictest2170766-21/usr/sbin/munin10:15:113
2104848991917,1cyclictest0-21swapper/311:37:343
2104848991917,1cyclictest0-21swapper/308:27:263
2104848991912,5cyclictest2143381-21python308:55:213
2104848991911,6cyclictest2118103-21grep07:45:133
2104848991911,5cyclictest2158351-21/usr/sbin/munin09:40:213
2104848991910,7cyclictest2147601-21latency_hist09:10:013
2104848991910,7cyclictest2123323-21expr08:00:133
2104844991917,1cyclictest0-21swapper/211:45:102
2104844991917,1cyclictest0-21swapper/210:51:232
2104844991917,1cyclictest0-21swapper/209:29:442
2104844991911,6cyclictest2116615-21df_inode07:40:142
210484499190,1cyclictest0-21swapper/208:39:152
210484499190,1cyclictest0-21swapper/207:21:082
210484199192,15cyclictest0-21swapper/110:43:151
2104841991917,1cyclictest3056-21PK-Backend12:00:351
2104841991917,1cyclictest3056-21PK-Backend10:53:561
2104841991917,1cyclictest3056-21PK-Backend08:46:561
2104841991917,1cyclictest3056-21PK-Backend07:50:591
210484199190,8cyclictest2177042-21users10:30:221
210484199190,6cyclictest2181713-21grep10:45:161
210484199190,5cyclictest2108153-21switchtime07:15:221
210484199190,1cyclictest3056-21PK-Backend11:30:201
210484199190,1cyclictest3056-21PK-Backend10:01:451
2104839991917,1cyclictest0-21swapper/012:14:050
2104839991917,1cyclictest0-21swapper/012:03:240
2104839991917,1cyclictest0-21swapper/011:53:300
2104839991917,1cyclictest0-21swapper/011:31:330
2104839991917,1cyclictest0-21swapper/008:07:120
2104839991913,5cyclictest2195736-21awk11:25:180
210483999190,1cyclictest3056-21PK-Backend07:28:260
210483999190,1cyclictest0-21swapper/011:49:520
210483999190,1cyclictest0-21swapper/008:53:230
210483999190,1cyclictest0-21swapper/007:20:080
210483999190,18cyclictest3056-21PK-Backend08:58:100
210483999190,18cyclictest0-21swapper/012:33:250
210483999190,18cyclictest0-21swapper/011:22:550
210483999190,18cyclictest0-21swapper/011:01:590
210483999190,18cyclictest0-21swapper/009:51:440
210483999190,18cyclictest0-21swapper/009:11:270
210483999190,18cyclictest0-21swapper/009:06:550
139099190,15rtkit-daemon2104833-21cyclictest12:27:250
210484899189,7cyclictest2166169-21/usr/sbin/munin10:00:213
210484899189,7cyclictest2149845-21smartctl09:15:163
210484899189,7cyclictest2105902-21python307:10:203
210484899188,9cyclictest2174407-21tr10:25:133
2104848991810,6cyclictest2135587-21/usr/sbin/munin08:35:133
210484499189,7cyclictest2192029-21cat11:15:172
2104844991816,1cyclictest0-21swapper/207:49:132
2104844991814,3cyclictest2142493-21find08:55:112
210484499181,13cyclictest2205432-21apt11:55:112
2104844991810,6cyclictest2220474-21memory12:35:182
210484199186,10cyclictest22100122chrt12:05:431
210484199182,3cyclictest3056-21PK-Backend07:30:561
2104841991816,1cyclictest3056-21PK-Backend11:23:441
2104841991816,1cyclictest3056-21PK-Backend10:07:531
2104841991816,1cyclictest3056-21PK-Backend07:43:481
210484199181,4cyclictest2179167-21apt10:40:001
210484199181,3cyclictest21649092chrt09:58:321
210484199180,4cyclictest2109971-21awk07:20:191
2104839991816,1cyclictest3056-21PK-Backend11:40:120
2104839991816,1cyclictest0-21swapper/012:22:570
2104839991813,3cyclictest2173956-21cron10:25:000
2104839991811,5cyclictest2178192-21cut10:35:190
2104839991811,5cyclictest2133701-21grep08:30:000
2104839991810,6cyclictest2155576-21grep09:30:170
2104839991810,6cyclictest2145005-21awk09:00:180
210483999180,1cyclictest0-21swapper/009:25:550
139099180,14rtkit-daemon1389-21rtkit-daemon09:43:520
210484499179,6cyclictest2148057-21expr09:10:122
2104844991711,5cyclictest2203897-21ls11:50:132
2104841991715,1cyclictest3056-21PK-Backend09:14:331
210484199170,8cyclictest2213721-21chrt12:15:301
210484199170,4cyclictest2159767-21chrt09:44:161
210484199170,4cyclictest2114458-21chrt07:35:121
210484199170,4cyclictest2114458-21chrt07:35:121
210484199170,3cyclictest21612382sleep109:45:521
210484199170,2cyclictest21859862sleep110:58:471
210484199170,2cyclictest21437082sleep108:56:441
210484199170,1cyclictest3056-21PK-Backend09:16:431
210484199170,15cyclictest21961332sleep111:25:301
2104839991715,1cyclictest0-21swapper/010:59:220
2104839991715,1cyclictest0-21swapper/008:41:400
2104839991711,4cyclictest2156-21gnome-shell08:33:090
210484499169,5cyclictest2165479-21expr10:00:142
2104844991610,4cyclictest2216245-21grep12:25:132
210484199160,2cyclictest21575152sleep109:35:221
210483999168,6cyclictest883-21dbus-daemon10:30:000
2104839991510,3cyclictest2179820-21forks10:40:150
2104839991510,3cyclictest2108928-21cat07:20:010
210483999147,5cyclictest2139311-21expr08:45:110
2104839991410,3cyclictest2118093-21cpuspeed_turbos07:45:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional