You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-22 - 07:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackfslot7s.osadl.org (updated Fri May 22, 2026 00:44:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
204632099304215,3022cyclictest2128657-21kworker/0:2+events23:40:130
204632099302814,3009cyclictest2085228-21kworker/0:0+events21:10:010
204632299289213,2875cyclictest2125619-21kworker/1:0+events23:15:131
204632299283913,2822cyclictest2095169-21kworker/1:0+events21:45:181
204632799280113,2784cyclictest2122223-21kworker/2:1+events23:13:412
204633099279013,2774cyclictest2047583-21kworker/3:2+events19:15:023
204632299278813,2771cyclictest2087260-21kworker/1:3+events21:20:181
204632099278714,2771cyclictest2141714-21kworker/0:2+events00:05:000
204633099276213,2744cyclictest2059272-21kworker/3:1+events19:55:143
204633099275912,2742cyclictest2088450-21kworker/3:0+events21:25:123
204633099273912,2722cyclictest2076931-21kworker/3:1+events20:45:163
204632299273913,2722cyclictest2087260-21kworker/1:3+events21:15:221
204632299271713,2700cyclictest2056022-21kworker/1:1+events19:54:511
204633099270913,2692cyclictest2090337-21kworker/3:1+events21:33:343
204632099267512,2659cyclictest2151922-21kworker/0:2+events00:39:300
20463229926726,2662cyclictest2067194-21kworker/1:0+events20:23:081
204633099264812,2632cyclictest2052762-21kworker/3:1+events19:30:373
204632099257812,2562cyclictest2143345-21kworker/0:1+events00:23:330
204632099257712,2561cyclictest2110536-21kworker/0:0+events23:20:480
204632099257610,2564cyclictest2098294-21kworker/0:1+events22:00:070
204632099257512,2559cyclictest2127037-21kworker/0:1+events23:18:110
204632099257512,2559cyclictest2096670-21kworker/0:2+events21:41:550
204632099257412,2559cyclictest2103215-21kworker/0:2+events22:25:200
204632099256312,2547cyclictest2057698-21kworker/0:0+events19:49:310
204632099255612,2540cyclictest2149819-21kworker/0:1+events00:31:300
204633099250212,2487cyclictest2104697-21kworker/3:1+events22:18:013
204632299249812,2482cyclictest2070473-21kworker/1:1+events20:27:411
204632099249712,2481cyclictest2047641-21kworker/0:0+events19:15:240
204633099249511,2480cyclictest2150089-21kworker/3:0+events00:39:533
204632099249412,2479cyclictest2103215-21kworker/0:2+events22:17:160
204632299249312,2478cyclictest2136613-21kworker/1:1+events23:50:391
204632099249312,2477cyclictest2052640-21kworker/0:1+events19:33:180
204632099248912,2473cyclictest2098294-21kworker/0:1+events22:06:140
204632099248811,2474cyclictest2112471-21kworker/0:1+events22:31:500
204632099248611,2472cyclictest2085228-21kworker/0:0+events21:32:460
204632099248611,2471cyclictest2052640-21kworker/0:1+events20:01:170
204632099248412,2469cyclictest2119469-21kworker/0:1+events22:55:390
204632099248312,2468cyclictest2098294-21kworker/0:1+events22:10:450
204632099248312,2467cyclictest2075426-21kworker/0:2+events20:47:050
204633099248212,2466cyclictest2080160-21kworker/3:2+events20:55:283
204632799248212,2467cyclictest2051461-21kworker/2:2+events19:36:192
204632799248212,2467cyclictest2051461-21kworker/2:2+events19:36:182
204632099248211,2467cyclictest2098294-21kworker/0:1+events21:57:030
204632299248112,2465cyclictest2117792-21kworker/1:2+events22:52:121
20463309924809,2467cyclictest2114338-21kworker/3:1+events22:42:273
204632799248011,2466cyclictest2051461-21kworker/2:2+events20:17:552
20463209924809,2468cyclictest2128657-21kworker/0:2+events23:36:020
204632099248011,2466cyclictest2052640-21kworker/0:1+events19:29:070
204633099247712,2461cyclictest2080160-21kworker/3:2+events21:03:323
20463309924769,2464cyclictest2140362-21kworker/3:3+events00:04:383
204633099247611,2462cyclictest2135287-21kworker/3:0+events23:46:023
204633099247511,2461cyclictest2128688-21kworker/3:1+events23:33:453
204633099247211,2457cyclictest2083419-21kworker/3:0+events21:07:473
204633099247211,2457cyclictest2068974-21kworker/3:0+events20:30:453
204633099247112,2455cyclictest2063826-21kworker/3:2+events20:05:483
204632299247110,2457cyclictest2146369-21kworker/1:2+events00:18:171
204632099246812,2452cyclictest2052640-21kworker/0:1+events19:44:570
204632299246412,2448cyclictest2130137-21kworker/1:0+events23:27:591
204632299246412,2448cyclictest2130137-21kworker/1:0+events23:27:581
204633099246311,2448cyclictest2125562-21kworker/3:0+events23:24:353
204632299246312,2447cyclictest2080066-21kworker/1:1+events20:53:341
204633099246212,2446cyclictest2078525-21kworker/3:0+events20:52:253
204632299246012,2444cyclictest2122310-21kworker/1:2+events23:02:461
204632099245512,2439cyclictest2131853-21kworker/0:0+events23:34:530
204632299245412,2440cyclictest2093510-21kworker/1:2+events21:35:061
204632299245312,2438cyclictest2060840-21kworker/1:0+events20:02:021
204632099245212,2436cyclictest2088719-21kworker/0:1+events21:26:200
204632099245012,2434cyclictest2135069-21kworker/0:1+events23:57:280
204633099244912,2433cyclictest2093688-21kworker/3:0+events21:40:473
204633099244812,2432cyclictest2128688-21kworker/3:1+events23:26:293
204633099244812,2432cyclictest2128688-21kworker/3:1+events23:26:283
204633099244712,2431cyclictest2143221-21kworker/3:0+events00:12:153
204633099244612,2430cyclictest2059272-21kworker/3:1+events19:53:183
204633099244412,2428cyclictest2099835-21kworker/3:0+events22:14:133
204632299243912,2425cyclictest2114193-21kworker/1:0+events22:40:081
204632299243412,2418cyclictest2075893-21kworker/1:2+events20:41:041
20463229924306,2420cyclictest2075893-21kworker/1:2+events20:58:131
204632099243011,2415cyclictest2110536-21kworker/0:0+events23:26:500
204632099243011,2415cyclictest2110536-21kworker/0:0+events23:26:490
20463209924246,2414cyclictest2065780-21kworker/0:1+events20:15:590
204633099241112,2397cyclictest2111107-21kworker/3:0+events22:36:243
204633099241011,2395cyclictest2146282-21kworker/3:2+events00:22:013
204632299241012,2394cyclictest2144741-21kworker/1:1+events00:11:531
204632099241012,2395cyclictest2117704-21kworker/0:2+events22:53:430
20463309924099,2396cyclictest2111107-21kworker/3:0+events22:34:053
204632099240812,2392cyclictest2139862-21kworker/0:0+events00:28:500
204632099240712,2392cyclictest2088719-21kworker/0:1+events21:24:050
204632299240512,2390cyclictest2098420-21kworker/1:2+events21:50:331
204632299240512,2389cyclictest2093510-21kworker/1:2+events21:33:091
204633099240312,2387cyclictest2123968-21kworker/3:1+events23:12:333
204633099240311,2388cyclictest2143221-21kworker/3:0+events00:07:183
20463209924019,2388cyclictest2080308-21kworker/0:2+events20:56:390
204632299240011,2385cyclictest2149638-21kworker/1:2+events00:36:261
204633099239912,2384cyclictest2050638-21kworker/3:0+events19:25:183
204633099239911,2385cyclictest2122280-21kworker/3:0+events23:05:453
204632099239913,2382cyclictest2123811-21kworker/0:1+events23:06:540
204633099239811,2384cyclictest2130378-21kworker/3:2+events23:41:483
204632299239811,2383cyclictest2107718-21kworker/1:1+events22:26:511
204632299239810,2384cyclictest2067194-21kworker/1:0+events20:36:511
204632299239712,2382cyclictest2067194-21kworker/1:0+events20:32:411
204633099239612,2380cyclictest2085447-21kworker/3:3+events21:11:103
204633099239610,2382cyclictest2098209-21kworker/3:2+events21:53:593
204633099239511,2380cyclictest2150089-21kworker/3:0+events00:30:433
204632299239512,2379cyclictest2107718-21kworker/1:1+events22:16:521
204632299239511,2380cyclictest2142377-21kworker/1:0+events00:29:361
204632099239512,2379cyclictest2110536-21kworker/0:0+events23:12:560
204632299239412,2380cyclictest2122310-21kworker/1:2+events23:05:211
204632299239412,2380cyclictest2060840-21kworker/1:0+events20:05:031
204633099239312,2377cyclictest2085447-21kworker/3:3+events21:18:013
204632299239312,2377cyclictest2051098-21kworker/1:0+events19:24:091
204632299239310,2381cyclictest2049469-21kworker/1:2+events19:25:421
204633099239212,2377cyclictest2103101-21kworker/3:2+events22:03:343
204633099239212,2377cyclictest2059272-21kworker/3:1+events20:02:263
204633099239211,2377cyclictest2114338-21kworker/3:1+events22:56:013
204633099239210,2378cyclictest2067342-21kworker/3:2+events20:14:293
204633099239210,2378cyclictest2059272-21kworker/3:1+events20:28:053
204632299239211,2379cyclictest2107718-21kworker/1:1+events22:20:201
204632099239210,2379cyclictest2071866-21kworker/0:1+events20:40:180
204633099239112,2375cyclictest2107872-21kworker/3:2+events22:21:323
204632299239112,2375cyclictest2054364-21kworker/1:2+events19:49:081
204633099239012,2375cyclictest2130378-21kworker/3:2+events23:39:533
204633099239012,2375cyclictest2099835-21kworker/3:0+events21:56:413
204633099239012,2374cyclictest2047583-21kworker/3:2+events19:36:423
204633099239012,2374cyclictest2047583-21kworker/3:2+events19:36:413
204633099239011,2375cyclictest2125562-21kworker/3:0+events23:19:403
204632299239012,2375cyclictest2098420-21kworker/1:2+events21:56:171
204632299239012,2374cyclictest2123902-21kworker/1:1+events23:31:521
204632299239012,2374cyclictest2104871-21kworker/1:0+events22:11:091
204632299239012,2374cyclictest2103246-21kworker/1:2+events22:07:221
204633099238912,2375cyclictest2090337-21kworker/3:1+events21:35:513
204633099238912,2374cyclictest2073441-21kworker/3:2+events20:35:213
204632299238912,2373cyclictest2142377-21kworker/1:0+events00:32:381
204632299238912,2373cyclictest2095169-21kworker/1:0+events21:40:241
204633099238812,2372cyclictest2068974-21kworker/3:0+events20:22:013
204632299238812,2372cyclictest2056022-21kworker/1:1+events19:55:591
204632299238812,2372cyclictest2054364-21kworker/1:2+events19:38:571
204632299238812,2372cyclictest2054364-21kworker/1:2+events19:38:561
204632299238812,2372cyclictest2054364-21kworker/1:2+events19:34:031
204633099238712,2372cyclictest2088450-21kworker/3:0+events21:21:033
204632299238712,2371cyclictest2120661-21kworker/1:0+events22:57:551
204632299238712,2371cyclictest2075893-21kworker/1:2+events21:00:301
204632299238712,2371cyclictest2056022-21kworker/1:1+events19:43:511
204632299238711,2373cyclictest2096793-21kworker/1:1+events22:04:181
204632299238613,2370cyclictest2063702-21kworker/1:2+events20:10:451
204632299238612,2371cyclictest2091876-21kworker/1:1+events21:28:371
204632099238512,2369cyclictest2052640-21kworker/0:1+events19:51:470
204632299238312,2368cyclictest2140137-21kworker/1:1+events00:02:251
204633099238212,2367cyclictest2090337-21kworker/3:1+events21:47:323
204633099237612,2360cyclictest2117125-21kworker/3:0+events22:54:273
204632299237110,2357cyclictest2148028-21kworker/1:1+events00:23:551
20463229923649,2352cyclictest2123902-21kworker/1:1+events23:11:071
204633099236212,2346cyclictest2135287-21kworker/3:0+events23:50:593
204632299236212,2346cyclictest2130137-21kworker/1:0+events23:42:571
204632099235712,2341cyclictest2114130-21kworker/0:2+events22:38:170
204632799235111,2337cyclictest2148090-21kworker/2:0+events00:31:072
204632299235112,2335cyclictest2114193-21kworker/1:0+events22:39:031
204633099234810,2335cyclictest2057641-21kworker/3:0+events19:49:543
204632799234812,2332cyclictest2145040-21kworker/2:1+events00:23:112
204632799234611,2331cyclictest2072286-21kworker/2:1+events21:13:042
204632299234212,2326cyclictest2067194-21kworker/1:0+events20:15:141
204632099233812,2323cyclictest2085228-21kworker/0:0+events21:50:550
204632299233112,2315cyclictest2130137-21kworker/1:0+events23:47:561
204632099233110,2317cyclictest2047641-21kworker/0:0+events19:24:310
204632799232712,2311cyclictest2095314-21kworker/2:1+events22:19:582
204632099232712,2312cyclictest2065780-21kworker/0:1+events20:12:150
204632099232511,2310cyclictest2096670-21kworker/0:2+events21:46:010
204632099232412,2308cyclictest2082012-21kworker/0:1+events21:08:080
20463209923239,2310cyclictest2062259-21kworker/0:0+events20:06:130
204632099232311,2309cyclictest2085228-21kworker/0:0+events21:37:440
204632099232212,2306cyclictest2122192-21kworker/0:2+events23:01:180
204632099232210,2309cyclictest2110536-21kworker/0:0+events22:40:310
204632099232111,2307cyclictest2078618-21kworker/0:1+events20:51:400
204632099232111,2306cyclictest2071866-21kworker/0:1+events20:28:520
204632099232011,2306cyclictest2088719-21kworker/0:1+events21:19:550
204632799231812,2303cyclictest2051461-21kworker/2:2+events20:24:402
204632799231712,2302cyclictest2125591-21kworker/2:2+events23:17:482
204632799231712,2302cyclictest2070592-21kworker/2:0+events20:25:242
20463309923149,2301cyclictest2122280-21kworker/3:0+events23:04:153
204632099231312,2298cyclictest2052640-21kworker/0:1+events19:37:030
204632099231312,2298cyclictest2052640-21kworker/0:1+events19:37:030
204632299230211,2288cyclictest2080066-21kworker/1:1+events21:09:151
204632799228910,2276cyclictest2051461-21kworker/2:2+events19:26:262
204633099226811,2253cyclictest2104697-21kworker/3:1+events22:09:343
204632099226710,2254cyclictest2082012-21kworker/0:1+events21:01:390
204632099226610,2253cyclictest2135069-21kworker/0:1+events23:54:220
20463229922558,2244cyclictest2130137-21kworker/1:0+events23:37:341
204632099225511,2241cyclictest2139862-21kworker/0:0+events00:14:340
204632799225411,2240cyclictest2093451-21kworker/2:0+events21:39:132
204632099224810,2235cyclictest2067372-21kworker/0:0+events20:24:170
204632099224411,2229cyclictest2139862-21kworker/0:0+events00:16:020
204633099223312,2217cyclictest2047583-21kworker/3:2+events19:24:553
204633099223310,2219cyclictest2114338-21kworker/3:1+events22:46:563
20463309922329,2219cyclictest2139954-21kworker/3:2+events23:57:483
204633099222311,2208cyclictest2146282-21kworker/3:2+events00:16:243
204632799221812,2203cyclictest2095314-21kworker/2:1+events22:33:192
204632299220410,2191cyclictest2104871-21kworker/1:0+events22:31:291
204633099219811,2184cyclictest2047583-21kworker/3:2+events19:41:573
204633099219711,2182cyclictest2073441-21kworker/3:2+events20:41:283
20463209921952,2190cyclictest2071866-21kworker/0:1+events20:37:370
204632099219311,2181cyclictest2139862-21kworker/0:0+events00:08:260
204632299219210,2178cyclictest2083331-21kworker/1:0+events21:12:411
204633099218911,2174cyclictest2146282-21kworker/3:2+events00:25:483
204632299218810,2174cyclictest2049469-21kworker/1:2+events19:18:001
204632299218710,2174cyclictest2127350-21kworker/1:2+events23:24:581
204632299218410,2170cyclictest2114193-21kworker/1:0+events22:45:501
204632099218311,2169cyclictest2071866-21kworker/0:1+events20:31:320
204632299218211,2168cyclictest2130137-21kworker/1:0+events23:58:131
204632299217810,2164cyclictest2138212-21kworker/1:2+events00:07:401
204632299217111,2156cyclictest2075893-21kworker/1:2+events20:45:371
204633099216711,2153cyclictest2059272-21kworker/3:1+events20:19:233
204632799213011,2115cyclictest2051461-21kworker/2:2+events20:34:342
204633099212410,2111cyclictest2047583-21kworker/3:2+events19:13:293
204632799211911,2104cyclictest2112650-21kworker/2:0+events22:42:042
204632799211711,2102cyclictest2123937-21kworker/2:0+events23:08:252
204632799211611,2102cyclictest2047487-21kworker/2:0+events19:18:242
204632799211610,2102cyclictest2051461-21kworker/2:2+events19:47:582
20463309921147,2104cyclictest2104697-21kworker/3:1+events22:28:463
204632799211411,2099cyclictest2072286-21kworker/2:1+events20:42:122
204632099211411,2100cyclictest2108105-21kworker/0:1+events22:24:340
204632799211311,2099cyclictest2133710-21kworker/2:2+events00:14:552
20463209921049,2091cyclictest2047641-21kworker/0:0+events19:13:520
20463209920988,2087cyclictest2060602-21kworker/0:2+events19:56:210
204632799209211,2078cyclictest2087056-21kworker/2:2+events21:30:332
20463209920929,2080cyclictest2115813-21kworker/0:1+events22:48:040
204632099209110,2077cyclictest2131853-21kworker/0:0+events23:46:470
204632799209011,2076cyclictest2054475-21kworker/2:0+events19:43:292
204632799208711,2073cyclictest2076986-21kworker/2:0+events21:03:562
20463229919637,1953cyclictest2044684-21kworker/1:0+events19:12:441
20463279919606,1951cyclictest2122223-21kworker/2:1+events23:40:382
20463279919496,1940cyclictest2115724-21kworker/2:2+events23:00:572
20463279918937,1883cyclictest2059120-21kworker/2:1+events19:50:402
20463279918159,1803cyclictest2044607-21kworker/2:2+events19:11:122
20458982156115,25sleep30-21swapper/319:06:523
20458512142109,28sleep00-21swapper/019:06:180
20459352141111,24sleep20-21swapper/219:07:192
20460482139108,26sleep10-21swapper/119:08:471
204632799298,10cyclictest709-21mta-sts-daemon21:05:182
204632799269,14cyclictest2136596-21sshd23:47:352
204632799261,22cyclictest2128907-21awk23:25:002
204632799260,22cyclictest2097234-21sshd21:45:062
204632799248,13cyclictest2109023-21grep22:20:182
2046327992419,3cyclictest0-21swapper/223:00:002
204632799231,19cyclictest2133638-21perf23:40:002
204632799231,19cyclictest2078818-21munin-run20:50:002
204632799230,15cyclictest0-21swapper/220:55:132
204632799222,17cyclictest0-21swapper/200:40:002
2046327992218,2cyclictest2140234-21cron00:00:012
2046327992218,2cyclictest0-21swapper/200:30:012
2046327992212,8cyclictest0-21swapper/222:15:012
204632799220,10cyclictest2138552-21awk23:55:002
204632799211,17cyclictest2110504-21nvmesmart_nvme022:25:182
204632799211,17cyclictest2067482-21awk20:15:002
204632799210,19cyclictest2053786-21ldconfig19:30:172
204632799210,10cyclictest2100645-21hddtemp_smartct21:55:152
2046327992015,4cyclictest865-10wireplumber21:20:002
204632799201,17cyclictest2062449-21munin-run20:00:012
204632799201,16cyclictest0-21swapper/221:20:122
204632799200,19cyclictest0-21swapper/221:25:202
204632799200,18cyclictest2075481-21runrttasks20:38:092
204632799200,11cyclictest2101991-21/usr/sbin/munin22:00:142
204632799200,10cyclictest2118136-21grep22:50:112
204632799191,16cyclictest2146057-21python300:15:212
204632799191,15cyclictest2100161-21awk21:55:012
2046327991910,6cyclictest2116545-21grep22:45:112
204632799181,15cyclictest2065910-21basename20:10:002
2046327991811,5cyclictest2063919-21chrt20:03:272
204632799181,15cyclictest0-21swapper/221:40:102
204632799181,14cyclictest2130471-21idleruntime-cro23:30:002
204632799181,14cyclictest2130471-21idleruntime-cro23:29:592
204632799181,14cyclictest2080394-21perf20:54:592
204632799180,16cyclictest0-21swapper/222:35:192
204632799180,14cyclictest2142411-21smartctl00:05:142
204632799180,10cyclictest2141744-21seq00:02:582
204632799171,13cyclictest2104905-21munin-run22:10:012
204632799170,10cyclictest2131704-21taskset23:30:452
204632799150,11cyclictest2051110-21taskset19:21:262
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional