You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-21 - 06:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackfslot7s.osadl.org (updated Tue Apr 21, 2026 00:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
408517999314913,3131cyclictest4163086-21kworker/1:0+events23:15:181
408517499284210,2828cyclictest4160847-21kworker/0:1+events23:20:160
408518599283914,2822cyclictest4130616-21kworker/3:2+events21:35:013
408517499280311,2788cyclictest4129429-21kworker/0:1+events21:27:020
408517499280113,2785cyclictest4153166-21kworker/0:1+events22:45:110
408517499270313,2686cyclictest4112811-21kworker/0:0+events20:35:170
408517999267113,2655cyclictest4121076-21kworker/1:1+events21:20:181
408518599266611,2651cyclictest4091603-21kworker/3:2+events19:30:123
408517999263713,2619cyclictest4177389-21kworker/1:0+events00:00:121
408518599263113,2614cyclictest4183511-21kworker/3:1+events00:15:143
408517999259512,2579cyclictest4114321-21kworker/1:0+events20:40:151
408517999257812,2562cyclictest4090305-21kworker/1:0+events19:35:131
408518599257512,2559cyclictest4102023-21kworker/3:1+events20:03:433
408518599255110,2537cyclictest4100223-21kworker/3:0+events19:56:063
408518599254612,2531cyclictest4154851-21kworker/3:1+events22:56:133
408517999254412,2528cyclictest4132541-21kworker/1:1+events21:35:451
40851859925419,2528cyclictest4124886-21kworker/3:1+events21:20:403
40851859925355,2526cyclictest4161221-21kworker/3:0+events23:34:453
408517499253211,2517cyclictest4108283-21kworker/0:1+events20:45:130
40851749925120,2509cyclictest4132000-21kworker/0:0+events21:44:060
408517999250212,2486cyclictest4145127-21kworker/1:1+events22:30:151
408517999250114,2484cyclictest4135851-21kworker/1:0+events22:03:331
408517999249512,2479cyclictest4098527-21kworker/1:1+events19:56:281
408518599249312,2477cyclictest4096961-21kworker/3:2+events19:49:173
408518599249212,2478cyclictest4179090-21kworker/3:3+events00:05:053
408517999248912,2474cyclictest4088677-21kworker/1:2+events19:19:571
408518599248812,2473cyclictest4091603-21kworker/3:2+events19:27:323
408518599248812,2472cyclictest4106380-21kworker/3:1+events20:59:393
408518599248612,2471cyclictest4106380-21kworker/3:1+events21:00:473
408518599248612,2470cyclictest4134106-21kworker/3:0+events21:54:443
408517999248412,2469cyclictest4167531-21kworker/1:1+events23:21:011
408517999248312,2468cyclictest4135851-21kworker/1:0+events22:07:001
408517999248312,2467cyclictest4100131-21kworker/1:2+events20:06:241
408518599248212,2467cyclictest4183511-21kworker/3:1+events00:39:143
408518599248212,2466cyclictest4142250-21kworker/3:0+events22:11:363
408517999248211,2468cyclictest4145127-21kworker/1:1+events23:11:041
408517999248012,2464cyclictest4090305-21kworker/1:0+events19:28:171
408517999248010,2466cyclictest4172726-21kworker/1:2+events00:16:011
408518599247910,2467cyclictest4124886-21kworker/3:1+events21:25:343
408517999247912,2465cyclictest4159771-21kworker/1:0+events23:00:051
408517999247912,2463cyclictest4189830-21kworker/1:1+events00:38:511
408518599247712,2462cyclictest4142250-21kworker/3:0+events22:28:413
408517999247612,2460cyclictest4124578-21kworker/1:0+events21:12:121
408517999247512,2461cyclictest4112721-21kworker/1:1+events20:36:241
408517999247512,2459cyclictest4186985-21kworker/1:2+events00:28:091
408517999247511,2460cyclictest4128063-21kworker/1:0+events21:28:111
408517999247411,2459cyclictest4145127-21kworker/1:1+events22:40:591
408517999247310,2461cyclictest4145127-21kworker/1:1+events22:28:181
408518599246710,2454cyclictest4122578-21kworker/3:0+events21:13:223
408517999246611,2452cyclictest4140684-21kworker/1:2+events22:14:351
408518599246512,2450cyclictest4183511-21kworker/3:1+events00:29:433
408518599246511,2450cyclictest4134106-21kworker/3:0+events21:40:163
408517999246511,2450cyclictest4146725-21kworker/1:0+events22:19:351
408518599246112,2445cyclictest4135728-21kworker/3:2+events21:45:373
408517999245712,2443cyclictest4163086-21kworker/1:0+events23:35:071
408517499245711,2443cyclictest4122673-21kworker/0:2+events21:10:190
408517499245613,2438cyclictest4088463-21kworker/0:0+events19:35:370
408518599245511,2440cyclictest4181912-21kworker/3:0+events00:14:513
40851819924549,2442cyclictest4093781-21kworker/2:1+events19:54:142
408518599245312,2438cyclictest4161221-21kworker/3:0+events23:36:403
408518599245312,2437cyclictest4142250-21kworker/3:0+events22:30:373
408518599245311,2439cyclictest4140568-21kworker/3:2+events22:06:353
408518599245212,2436cyclictest4106380-21kworker/3:1+events20:18:163
408518599245112,2436cyclictest4153231-21kworker/3:0+events22:38:413
408518599243014,2412cyclictest4140568-21kworker/3:2+events21:58:563
408518599243010,2417cyclictest4146695-21kworker/3:2+events22:16:293
408518599242112,2405cyclictest4158633-21kworker/3:2+events23:00:523
408517499242012,2405cyclictest4173843-21kworker/0:2+events23:55:150
408518599241912,2404cyclictest4111268-21kworker/3:0+events20:30:443
408517499241812,2404cyclictest4173843-21kworker/0:2+events23:50:210
40851859924179,2405cyclictest4172585-21kworker/3:2+events00:03:563
408517499241710,2403cyclictest4178785-21kworker/0:0+events00:03:360
408517999241512,2399cyclictest4145127-21kworker/1:1+events23:09:331
408517499241312,2399cyclictest4106678-21kworker/0:0+events20:25:030
408517499240612,2390cyclictest4093269-21kworker/0:2+events19:34:480
408518199240312,2389cyclictest4125952-21kworker/2:1+events22:20:192
408517499240212,2386cyclictest4108283-21kworker/0:1+events20:24:170
408517499238711,2372cyclictest4167767-21kworker/0:2+events23:25:110
408517999237910,2366cyclictest4148352-21kworker/1:2+events22:23:431
408517499237911,2364cyclictest4138872-21kworker/0:2+events22:02:220
40851859923759,2363cyclictest4086466-21kworker/3:1+events19:14:583
408518599236713,2351cyclictest4109634-21kworker/3:2+events20:24:403
408517499236511,2350cyclictest4146974-21kworker/0:1+events22:29:050
408517499236012,2344cyclictest4169404-21kworker/0:1+events23:39:230
408517499236010,2347cyclictest4188374-21kworker/0:2+events00:31:160
408518599235912,2343cyclictest4106380-21kworker/3:1+events20:49:463
408518599235211,2337cyclictest4161221-21kworker/3:0+events23:09:103
408517499234811,2334cyclictest4105080-21kworker/0:2+events20:14:470
408518199234112,2327cyclictest4153078-21kworker/2:0+events22:45:332
408517999234111,2327cyclictest4122732-21kworker/1:2+events21:05:001
408517999234110,2328cyclictest4183690-21kworker/1:0+events00:32:471
408517499233512,2321cyclictest4132000-21kworker/0:0+events21:50:100
408517999233411,2321cyclictest4163086-21kworker/1:0+events23:30:091
408518599233311,2319cyclictest4177017-21kworker/3:1+events23:54:303
408518199232912,2314cyclictest4100313-21kworker/2:2+events20:19:232
408517499232912,2313cyclictest4119124-21kworker/0:1+events21:02:440
408517499232812,2312cyclictest4182179-21kworker/0:1+events00:23:140
408517499232712,2311cyclictest4164508-21kworker/0:2+events23:17:120
408517499232712,2311cyclictest4154820-21kworker/0:0+events22:56:370
408517499232711,2312cyclictest4182179-21kworker/0:1+events00:11:280
408517499232612,2311cyclictest4180557-21kworker/0:2+events00:08:080
408517499232612,2311cyclictest4148591-21kworker/0:2+events22:39:510
408517499232612,2310cyclictest4132000-21kworker/0:0+events21:47:550
408517499232611,2311cyclictest4138872-21kworker/0:2+events22:11:130
408518199232512,2309cyclictest4117869-21kworker/2:0+events21:11:492
408517499232512,2310cyclictest4171083-21kworker/0:0+events23:42:030
408517999232210,2309cyclictest4177389-21kworker/1:0+events23:57:571
408517499232010,2306cyclictest4182179-21kworker/0:1+events00:15:380
408517499231711,2303cyclictest4132000-21kworker/0:0+events21:33:520
408518199231312,2299cyclictest4088901-21kworker/2:2+events19:26:452
408517499230910,2295cyclictest4119124-21kworker/0:1+events20:56:380
408517999230412,2289cyclictest4112721-21kworker/1:1+events20:45:581
408517499230412,2288cyclictest4148591-21kworker/0:2+events22:24:520
40851749922959,2282cyclictest4188374-21kworker/0:2+events00:35:480
408518599229011,2275cyclictest4186649-21kworker/3:2+events00:22:053
408518599228910,2276cyclictest4096961-21kworker/3:2+events19:54:363
408518199228811,2274cyclictest4125952-21kworker/2:1+events22:31:022
408517999227311,2259cyclictest4172726-21kworker/1:2+events23:48:031
408517999227311,2258cyclictest4108255-21kworker/1:0+events20:27:211
408517999227311,2258cyclictest4090305-21kworker/1:0+events19:44:171
408518599227211,2257cyclictest4161221-21kworker/3:0+events23:12:133
408517999227211,2257cyclictest4130632-21kworker/1:2+events21:33:291
408517999227111,2257cyclictest4098527-21kworker/1:1+events20:19:451
408517999227110,2258cyclictest4092004-21kworker/1:2+events19:34:251
408517999227011,2255cyclictest4126192-21kworker/1:2+events21:19:311
408517999227010,2256cyclictest4145127-21kworker/1:1+events22:45:531
408517999226811,2253cyclictest4090305-21kworker/1:0+events19:46:361
408517999226810,2255cyclictest4119218-21kworker/1:2+events20:54:001
408517999226810,2254cyclictest4145127-21kworker/1:1+events22:59:201
408517499226811,2255cyclictest4108283-21kworker/0:1+events20:41:010
408518599226711,2253cyclictest4142250-21kworker/3:0+events22:22:133
408518599226511,2251cyclictest4161221-21kworker/3:0+events23:44:173
408518599226510,2251cyclictest4161221-21kworker/3:0+events23:46:323
408517999226511,2251cyclictest4112721-21kworker/1:1+events20:31:551
408517999226510,2251cyclictest4185352-21kworker/1:1+events00:24:001
408518599226410,2250cyclictest4096961-21kworker/3:2+events19:42:233
408517999226410,2251cyclictest4100131-21kworker/1:2+events20:03:211
408518599226211,2248cyclictest4166113-21kworker/3:2+events23:18:423
408518599226211,2247cyclictest4154851-21kworker/3:1+events22:50:023
408518599226210,2249cyclictest4089055-21kworker/3:0+events19:23:463
408518599226111,2246cyclictest4151532-21kworker/3:2+events22:46:183
40851859922604,2253cyclictest4153231-21kworker/3:0+events22:42:273
408518599225910,2245cyclictest4104777-21kworker/3:0+events20:07:563
408517999225811,2243cyclictest4172726-21kworker/1:2+events23:43:101
408517999225810,2244cyclictest4183690-21kworker/1:0+events00:14:081
408518599225711,2243cyclictest4140568-21kworker/3:2+events22:00:493
408518599225710,2244cyclictest4124886-21kworker/3:1+events21:17:113
40851799922578,2246cyclictest4145127-21kworker/1:1+events22:51:581
408518599225611,2242cyclictest4106380-21kworker/3:1+events20:51:413
40851859922559,2243cyclictest4167797-21kworker/3:1+events23:28:123
408518599225510,2244cyclictest4124886-21kworker/3:1+events21:36:293
408517999225310,2240cyclictest4169218-21kworker/1:2+events23:27:511
40851799922528,2241cyclictest4135851-21kworker/1:0+events21:46:001
408517999225211,2238cyclictest4135851-21kworker/1:0+events21:59:401
408517999225110,2237cyclictest4145127-21kworker/1:1+events22:37:091
40851749922449,2231cyclictest4158375-21kworker/0:2+events23:02:440
40851799922438,2233cyclictest4117583-21kworker/1:0+events21:00:031
408518599223610,2223cyclictest4117759-21kworker/3:2+events21:06:323
408518599223413,2219cyclictest4106380-21kworker/3:1+events20:40:383
408518599223311,2219cyclictest4161221-21kworker/3:0+events23:21:473
408518599223310,2220cyclictest4183511-21kworker/3:1+events00:34:423
408518599223210,2219cyclictest4106380-21kworker/3:1+events20:13:413
408518199222913,2212cyclictest4115918-21kworker/2:1+events20:42:322
408518599222811,2215cyclictest4093241-21kworker/3:1+events19:36:213
408518199221510,2202cyclictest4147321-21kworker/2:2+events22:36:242
40851799922126,2203cyclictest4098527-21kworker/1:1+events20:14:041
408517499221110,2197cyclictest4112811-21kworker/0:0+events20:33:460
40851859922089,2195cyclictest4114533-21kworker/3:0+events20:39:533
408517999220711,2194cyclictest4098527-21kworker/1:1+events19:50:021
408517499220410,2191cyclictest4103475-21kworker/0:0+events20:05:170
408518599220310,2190cyclictest4106380-21kworker/3:1+events20:27:423
408517499219911,2184cyclictest4173843-21kworker/0:2+events23:47:170
408517999218810,2174cyclictest4086373-21kworker/1:1+events19:24:101
408518199217011,2156cyclictest4164418-21kworker/2:0+events23:45:232
408517499216611,2151cyclictest4100192-21kworker/0:0+events19:56:510
408517499215912,2143cyclictest4095473-21kworker/0:3+events19:45:480
408518199215012,2135cyclictest4114264-21kworker/2:0+events20:38:462
408517999214610,2132cyclictest4182062-21kworker/1:1+events00:08:301
408517499214411,2129cyclictest4146974-21kworker/0:1+events22:19:560
408517499214311,2129cyclictest4100192-21kworker/0:0+events19:54:580
40851819921428,2131cyclictest4125952-21kworker/2:1+events21:29:432
408517999214111,2127cyclictest4130632-21kworker/1:2+events21:43:201
408517499213911,2125cyclictest4153166-21kworker/0:1+events22:40:350
408517499213810,2124cyclictest4121015-21kworker/0:0+events21:05:220
408517499213410,2120cyclictest4119124-21kworker/0:1+events20:52:510
40851859921329,2120cyclictest4178753-21kworker/3:0+events23:59:513
408517499213211,2118cyclictest4127771-21kworker/0:2+events21:23:150
408518199213110,2118cyclictest4091691-21kworker/2:0+events19:44:402
408517499213011,2115cyclictest4095473-21kworker/0:3+events19:43:530
408518199212911,2115cyclictest4182348-21kworker/2:2+events00:16:232
408517999212911,2115cyclictest4172726-21kworker/1:2+events23:54:501
408517499212911,2115cyclictest4169404-21kworker/0:1+events23:30:320
408517499212910,2116cyclictest4153166-21kworker/0:1+events22:51:350
408517499212811,2114cyclictest4105080-21kworker/0:2+events20:17:530
408517499212811,2113cyclictest4132000-21kworker/0:0+events21:39:310
408517499212611,2112cyclictest4162814-21kworker/0:0+events23:07:360
408517499212610,2113cyclictest4142098-21kworker/0:0+events22:05:510
408517499212511,2111cyclictest4126132-21kworker/0:0+events21:19:550
408518199211811,2104cyclictest4088901-21kworker/2:2+events19:20:432
408517499211710,2104cyclictest4090095-21kworker/0:1+events19:25:590
408518599211410,2101cyclictest4076366-21kworker/3:2+events19:15:433
408518199211411,2100cyclictest4185109-21kworker/2:1+events00:28:562
408518199211411,2100cyclictest4108226-21kworker/2:0+events20:21:122
408518199211411,2099cyclictest4153078-21kworker/2:0+events22:57:242
408517999211110,2098cyclictest4079850-21kworker/1:0+events19:13:051
40851799921109,2098cyclictest4108255-21kworker/1:0+events20:20:271
408517999211010,2096cyclictest4130632-21kworker/1:2+events21:50:561
408518199210710,2093cyclictest4125952-21kworker/2:1+events22:13:052
408517499210411,2090cyclictest4101852-21kworker/0:1+events20:01:030
408518199209611,2082cyclictest4147321-21kworker/2:2+events22:27:072
40851819920878,2076cyclictest4088901-21kworker/2:2+events19:30:342
40851819920827,2072cyclictest4137650-21kworker/2:2+events21:51:192
408517499206710,2053cyclictest4076806-21kworker/0:3+events19:11:340
40851799920516,2042cyclictest4117583-21kworker/1:0+events20:58:321
40851749920348,2022cyclictest4132000-21kworker/0:0+events21:56:150
408517499198810,1974cyclictest4088463-21kworker/0:0+events19:19:100
408517499198210,1970cyclictest4148591-21kworker/0:2+events22:31:240
408517499197710,1964cyclictest4188374-21kworker/0:2+events00:29:190
40851749919687,1958cyclictest4088463-21kworker/0:0+events19:21:280
408518199195810,1945cyclictest4164418-21kworker/2:0+events00:12:592
40851749919587,1948cyclictest4162814-21kworker/0:0+events23:10:410
408518199195310,1940cyclictest4158462-21kworker/2:2+events23:11:272
40851819915397,1531cyclictest4063656-21kworker/2:0+events19:10:012
4084003214793,50sleep00-21swapper/019:05:160
40848302145102,24sleep10-21swapper/119:09:221
40845782143114,24sleep30-21swapper/319:06:123
40848112141113,23sleep20-21swapper/219:09:072
408518199288,19cyclictest4100839-21ls19:55:122
408518199281,14cyclictest4121229-21basename21:00:002
408518199266,10cyclictest4179899-21collect200:00:172
408518199251,18cyclictest4119379-21munin-run20:54:592
408518199244,17cyclictest0-21swapper/219:15:012
408518199241,20cyclictest4172645-21perf23:40:002
4085181992316,4cyclictest607-21rs:main2
408518199231,19cyclictest622-21gdbus22:00:002
408518199231,19cyclictest4191771-21perf00:40:002
408518199231,18cyclictest4168759-21expr23:25:202
408518199231,10cyclictest0-21swapper/222:15:192
408518199230,20cyclictest4105414-21apt-get20:10:132
408518199228,12cyclictest70450irq/124-enp2s0-TxRx-023:59:592
408518199223,16cyclictest4143922-21basename22:10:012
408518199222,17cyclictest0-21swapper/221:19:592
4085181992214,5cyclictest4112310-21ldconfig20:30:212
408518199221,18cyclictest4177427-21idleruntime-cro23:55:002
408518199221,18cyclictest4137643-21cat21:50:012
408518199221,18cyclictest0-21swapper/221:39:592
408518199221,16cyclictest4182274-21perf00:10:002
4085181992112,7cyclictest4142343-21awk22:05:002
408518199211,17cyclictest4132848-21idleruntime-cro21:35:002
408518199211,17cyclictest4111487-21basename20:29:592
408518199211,17cyclictest4098229-21python319:45:192
408518199211,17cyclictest4094313-21smartctl19:35:142
408518199210,20cyclictest4170474-21/usr/sbin/munin23:30:212
408518199201,17cyclictest4122826-21awk21:05:002
408518199201,16cyclictest4117809-21munin-run20:49:592
408518199201,16cyclictest4105162-21idleruntime-cro20:10:012
408518199201,15cyclictest4136012-21awk21:45:002
408518199200,18cyclictest4173598-21ps23:40:202
408518199200,17cyclictest4166320-21awk23:20:002
408518199200,11cyclictest4161331-21seq23:02:072
408518199200,11cyclictest4128060-21taskset21:20:052
408518199196,11cyclictest4157584-21ntpq22:50:162
408518199191,15cyclictest4154987-21munin-run22:45:002
408518199191,15cyclictest4124658-21munin-run21:10:002
408518199191,13cyclictest4188871-21apt-get00:30:122
4085181991811,6cyclictest4102878-21/usr/sbin/munin20:00:192
408518199171,14cyclictest4167109-21ntpq23:20:172
408518199169,5cyclictest4163007-21chrt23:08:222
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional