You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-19 - 10:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackfslot7s.osadl.org (updated Sat Sep 19, 2026 00:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
16477899359915,3581cyclictest267308-21kworker/3:0+events00:30:183
16477899298514,2967cyclictest223661-21kworker/3:2+events22:15:133
16477899297015,2951cyclictest238354-21kworker/3:2+events23:00:003
16477899296314,2944cyclictest180875-21kworker/3:0+events20:00:193
16477299292513,2909cyclictest254848-21kworker/1:1+events23:55:011
16477299290914,2890cyclictest235006-21kworker/1:0+events22:50:161
16477899280912,2792cyclictest240004-21kworker/3:1+events23:10:193
16477799274912,2733cyclictest169753-21kworker/2:0+events19:30:412
16477299274413,2726cyclictest172851-21kworker/1:0+events19:35:191
16476999266213,2646cyclictest206173-21kworker/0:2+events21:16:260
16476999266213,2646cyclictest206173-21kworker/0:2+events21:16:260
16477299262113,2605cyclictest204611-21kworker/1:1+events21:23:131
16477899260710,2593cyclictest198870-21kworker/3:2+events21:02:113
16477899258610,2572cyclictest209398-21kworker/3:2+events21:40:203
16477799255010,2537cyclictest214134-21kworker/2:0+events21:51:392
16476999254912,2533cyclictest204251-21kworker/0:0+events21:21:010
16477799254711,2532cyclictest239787-21kworker/2:0+events23:09:102
1647779925437,2533cyclictest188804-21kworker/2:2+events20:28:422
16477799253912,2523cyclictest223810-21kworker/2:1+events22:49:522
16477799250112,2485cyclictest185563-21kworker/2:1+events20:23:032
16477799249912,2483cyclictest222170-21kworker/2:2+events22:14:052
16477799249612,2480cyclictest169753-21kworker/2:0+events19:55:452
16477799249512,2479cyclictest217348-21kworker/2:1+events21:58:032
16477899249312,2478cyclictest247924-21kworker/3:0+events00:17:453
16477799249312,2479cyclictest259987-21kworker/2:1+events00:15:062
16477799249312,2477cyclictest257921-21kworker/2:3+events00:02:592
16477299249312,2477cyclictest207863-21kworker/1:0+events21:35:011
16477899249212,2476cyclictest167860-21kworker/3:2+events19:42:553
16477799249212,2476cyclictest249439-21kworker/2:0+events23:41:112
16477299249212,2476cyclictest188776-21kworker/1:0+events20:27:321
16477799249112,2476cyclictest166118-21kworker/2:2+events19:19:162
16477799248912,2474cyclictest220579-21kworker/2:0+events22:07:352
16477799248911,2475cyclictest263892-21kworker/2:2+events00:21:342
16477799248910,2475cyclictest190400-21kworker/2:0+events20:32:062
16476999248911,2475cyclictest232018-21kworker/0:0+events22:56:160
16477799248812,2474cyclictest242991-21kworker/2:2+events23:21:012
16477799248812,2473cyclictest191992-21kworker/2:2+events20:44:372
16477799248712,2472cyclictest209678-21kworker/2:1+events21:43:232
16477799248712,2471cyclictest252919-21kworker/2:2+events23:48:102
16477799248212,2466cyclictest196653-21kworker/2:1+events20:46:352
1647699924807,2470cyclictest232018-21kworker/0:0+events22:42:420
16477799247911,2464cyclictest214134-21kworker/2:0+events21:47:542
16476999247811,2463cyclictest211261-21kworker/0:1+events22:05:170
16477299247612,2460cyclictest236671-21kworker/1:1+events23:08:021
16477299247610,2462cyclictest206204-21kworker/1:2+events21:18:191
16477299247610,2462cyclictest206204-21kworker/1:2+events21:18:191
1647729924756,2466cyclictest177678-21kworker/1:0+events19:53:051
16477899247411,2461cyclictest211021-21kworker/3:0+events21:50:103
16477799247311,2458cyclictest223810-21kworker/2:1+events22:39:372
16477799247210,2458cyclictest208035-21kworker/2:2+events21:26:392
16477299247210,2458cyclictest239974-21kworker/1:2+events23:13:211
16477299246911,2455cyclictest190677-21kworker/1:2+events20:47:441
16477299246811,2453cyclictest227062-21kworker/1:2+events22:26:401
16477299246612,2450cyclictest222151-21kworker/1:2+events22:17:531
16477299246612,2450cyclictest171242-21kworker/1:1+events19:46:391
16477799245012,2434cyclictest198287-21kworker/2:0+events20:55:432
1647779924459,2433cyclictest263892-21kworker/2:2+events00:25:472
16477299244211,2427cyclictest172851-21kworker/1:0+events19:40:391
1647779924419,2428cyclictest236554-21kworker/2:0+events22:51:482
16476999243512,2420cyclictest211261-21kworker/0:1+events21:35:210
16476999243512,2419cyclictest190198-21kworker/0:0+events20:53:500
16477299243412,2418cyclictest264110-21kworker/1:2+events00:31:261
16477799243312,2418cyclictest198287-21kworker/2:0+events21:01:472
16476999243212,2416cyclictest211261-21kworker/0:1+events21:53:320
16476999243112,2415cyclictest222328-21kworker/0:2+events22:32:220
16477299242812,2412cyclictest238326-21kworker/1:0+events23:01:291
16476999242812,2412cyclictest171068-21kworker/0:1+events19:30:170
16477299242312,2408cyclictest223799-21kworker/1:0+events22:34:591
16477299242210,2408cyclictest233406-21kworker/1:1+events22:44:571
16477899242110,2409cyclictest217531-21kworker/3:2+events21:55:013
16477899242012,2404cyclictest247924-21kworker/3:0+events00:01:513
16476999241912,2403cyclictest190198-21kworker/0:0+events20:28:190
16476999241212,2396cyclictest243067-21kworker/0:0+events23:15:170
16477899240812,2393cyclictest236754-21kworker/3:0+events23:00:213
16477899240812,2393cyclictest162757-21kworker/3:0+events19:20:273
16477299240312,2387cyclictest166055-21kworker/1:2+events19:14:181
16477299240212,2386cyclictest259173-21kworker/1:1+events00:06:211
16477299240012,2384cyclictest223799-21kworker/1:0+events22:37:201
16477899239912,2383cyclictest265552-21kworker/3:2+events00:29:143
16477299239612,2381cyclictest179328-21kworker/1:2+events19:59:551
16477299239611,2381cyclictest182324-21kworker/1:1+events20:07:301
16477899239410,2380cyclictest198870-21kworker/3:2+events21:12:183
16477299239312,2377cyclictest177678-21kworker/1:0+events20:14:441
16477799239211,2377cyclictest173005-21kworker/2:2+events19:36:502
16477299239212,2376cyclictest239974-21kworker/1:2+events23:27:251
16477799239112,2376cyclictest223810-21kworker/2:1+events22:34:352
16477299239111,2376cyclictest166055-21kworker/1:2+events19:23:071
16477299239012,2374cyclictest214393-21kworker/1:0+events22:04:091
16476999239012,2376cyclictest177413-21kworker/0:1+events19:50:030
16477299238912,2374cyclictest225441-21kworker/1:1+events22:23:131
16477299238812,2372cyclictest247776-21kworker/1:1+events23:41:341
16477299238811,2374cyclictest214393-21kworker/1:0+events21:51:161
16477299238712,2374cyclictest253015-21kworker/1:0+events23:58:041
16477299238712,2371cyclictest235006-21kworker/1:0+events22:49:071
16477299238711,2373cyclictest260800-21kworker/1:0+events00:20:501
16477299238711,2373cyclictest219493-21kworker/1:1+events22:07:131
16477299238612,2371cyclictest253015-21kworker/1:0+events00:00:451
16476999238611,2372cyclictest259408-21kworker/0:1+events00:09:230
16477299238412,2369cyclictest187156-21kworker/1:2+events20:19:381
16477299238410,2371cyclictest222151-21kworker/1:2+events22:11:001
16477299238312,2368cyclictest204611-21kworker/1:1+events21:11:321
16477299238311,2369cyclictest166378-21kworker/1:3+events19:16:571
16477299238212,2367cyclictest166055-21kworker/1:2+events19:32:351
16477799238110,2367cyclictest198287-21kworker/2:0+events21:05:552
16477299237810,2364cyclictest169494-21kworker/1:0+events19:27:161
16476999237811,2363cyclictest255958-21kworker/0:2+events00:33:190
16477899237711,2363cyclictest246238-21kworker/3:2+events23:29:183
16476999237712,2361cyclictest248073-21kworker/0:2+events23:38:070
16476999237311,2359cyclictest246414-21kworker/0:1+events23:28:120
16477899237212,2356cyclictest185316-21kworker/3:2+events20:23:243
1647699923716,2361cyclictest214158-21kworker/0:2+events21:41:290
16477299236212,2346cyclictest251369-21kworker/1:2+events23:46:141
16477299236112,2346cyclictest260800-21kworker/1:0+events00:18:541
16476999236112,2346cyclictest241463-21kworker/0:1+events23:13:000
16477299235412,2338cyclictest200085-21kworker/1:2+events21:01:011
16476999235212,2336cyclictest257655-21kworker/0:0+events23:59:580
16477299235110,2338cyclictest260800-21kworker/1:0+events00:26:561
16476999235110,2338cyclictest222328-21kworker/0:2+events22:11:460
16476999235011,2335cyclictest222328-21kworker/0:2+events22:29:410
1647789923499,2336cyclictest252033-21kworker/3:2+events23:43:323
16477299234812,2332cyclictest190677-21kworker/1:2+events20:41:591
16476999234712,2332cyclictest217387-21kworker/0:2+events21:56:310
16476999234712,2332cyclictest196897-21kworker/0:1+events21:07:480
16476999234712,2332cyclictest196897-21kworker/0:1+events20:59:510
1647699923439,2331cyclictest182170-21kworker/0:1+events20:01:510
16477299234112,2325cyclictest270261-21kworker/1:0+events00:37:051
16477899233911,2324cyclictest247924-21kworker/3:0+events23:56:333
16477899233910,2325cyclictest209398-21kworker/3:2+events21:31:363
16476999233912,2323cyclictest165995-21kworker/0:2+events19:22:430
16476999233911,2325cyclictest239825-21kworker/0:2+events23:07:390
1647699923389,2325cyclictest211261-21kworker/0:1+events21:48:410
16477899233611,2322cyclictest177561-21kworker/3:1+events20:09:033
16476999233612,2322cyclictest171068-21kworker/0:1+events19:35:430
16476999233611,2322cyclictest200175-21kworker/0:2+events21:04:030
16477899233412,2319cyclictest180875-21kworker/3:0+events20:10:343
16477899233312,2317cyclictest177561-21kworker/3:1+events19:49:193
16477899232812,2313cyclictest198870-21kworker/3:2+events21:07:253
16477899232511,2312cyclictest180875-21kworker/3:0+events20:15:083
16477799232412,2308cyclictest223810-21kworker/2:1+events22:20:562
16477799232112,2306cyclictest244572-21kworker/2:1+events23:28:352
16476999231912,2304cyclictest177413-21kworker/0:1+events19:47:020
16476999231512,2300cyclictest230091-21kworker/0:1+events22:38:290
16477899231412,2299cyclictest230571-21kworker/3:1+events22:53:153
16477799231412,2299cyclictest208035-21kworker/2:2+events21:38:252
16477899231311,2299cyclictest219134-21kworker/3:0+events22:11:223
16477899231311,2299cyclictest188898-21kworker/3:0+events20:26:263
16476999231210,2299cyclictest259408-21kworker/0:1+events00:10:520
16477799231011,2295cyclictest268705-21kworker/2:2+events00:38:132
16477899230912,2294cyclictest247924-21kworker/3:0+events23:45:273
16477899230911,2295cyclictest209398-21kworker/3:2+events21:27:013
16476999230612,2292cyclictest183921-21kworker/0:0+events20:10:560
16477899230512,2289cyclictest222059-21kworker/3:1+events22:09:043
16477799229512,2280cyclictest254549-21kworker/2:0+events00:07:312
16477799229110,2277cyclictest183905-21kworker/2:2+events20:09:262
16477799228313,2268cyclictest201532-21kworker/2:1+events21:10:062
16477299228212,2267cyclictest198471-21kworker/1:0+events20:58:441
16477899227710,2264cyclictest191683-21kworker/3:2+events20:31:423
16477799227511,2261cyclictest168133-21kworker/2:1+events19:27:412
16477299227510,2262cyclictest185535-21kworker/1:1+events20:35:291
16477799227411,2260cyclictest179009-21kworker/2:1+events19:51:102
16477799227410,2260cyclictest169753-21kworker/2:0+events19:42:092
16477799227211,2258cyclictest198287-21kworker/2:0+events20:52:412
16477799227210,2259cyclictest223810-21kworker/2:1+events22:18:152
16477299227210,2258cyclictest204611-21kworker/1:1+events21:25:551
16477799227110,2258cyclictest208035-21kworker/2:2+events21:32:212
16477799227110,2258cyclictest204767-21kworker/2:0+events21:18:422
16477799227110,2258cyclictest204767-21kworker/2:0+events21:18:422
16477799226910,2256cyclictest244572-21kworker/2:1+events23:54:372
16477799226910,2256cyclictest244572-21kworker/2:1+events23:19:302
1647779922678,2256cyclictest249439-21kworker/2:0+events23:35:262
16477799226710,2254cyclictest244572-21kworker/2:1+events23:59:142
16477799226710,2253cyclictest259987-21kworker/2:1+events00:13:332
1647779922669,2254cyclictest168133-21kworker/2:1+events19:21:342
16477799226610,2253cyclictest268705-21kworker/2:2+events00:32:572
16477299226611,2252cyclictest246175-21kworker/1:0+events23:24:461
16477799226511,2252cyclictest236554-21kworker/2:0+events22:55:112
16477799226511,2250cyclictest247846-21kworker/2:2+events23:32:262
16477799226510,2252cyclictest220579-21kworker/2:0+events22:03:002
1647779922649,2252cyclictest204767-21kworker/2:0+events21:20:372
16476999226412,2249cyclictest239825-21kworker/0:2+events23:03:260
16477899226212,2247cyclictest196989-21kworker/3:0+events20:55:203
1647729922618,2249cyclictest214393-21kworker/1:0+events21:49:481
16477899225710,2243cyclictest177561-21kworker/3:1+events19:54:133
16476999225210,2239cyclictest232018-21kworker/0:0+events22:53:380
1647699922507,2240cyclictest219104-21kworker/0:0+events22:04:310
16476999223711,2224cyclictest204251-21kworker/0:0+events21:25:070
16477299223510,2221cyclictest185535-21kworker/1:1+events20:21:551
1647779922349,2222cyclictest185563-21kworker/2:1+events20:38:062
1647699922278,2216cyclictest191990-21kworker/0:1+events20:39:390
1647699922179,2204cyclictest171068-21kworker/0:1+events19:29:080
16477299221110,2198cyclictest190677-21kworker/1:2+events20:51:561
16477299220410,2190cyclictest209550-21kworker/1:2+events21:36:531
16476999220310,2189cyclictest257655-21kworker/0:0+events00:03:430
16477299220210,2189cyclictest239974-21kworker/1:2+events23:31:401
16477299219111,2176cyclictest177678-21kworker/1:0+events20:04:541
1647789921878,2175cyclictest247924-21kworker/3:0+events23:32:033
16477299218611,2171cyclictest238326-21kworker/1:0+events22:59:161
16477299218513,2169cyclictest209550-21kworker/1:2+events21:40:421
16477299218510,2172cyclictest239974-21kworker/1:2+events23:16:031
16477299218410,2171cyclictest190677-21kworker/1:2+events20:32:301
1647729921835,2174cyclictest198471-21kworker/1:0+events21:07:021
16477299218310,2171cyclictest247776-21kworker/1:1+events23:35:041
16477299218310,2169cyclictest257442-21kworker/1:2+events00:12:021
16477799218110,2168cyclictest169753-21kworker/2:0+events19:48:342
16477299217811,2164cyclictest217671-21kworker/1:2+events21:59:351
16476999217612,2160cyclictest251096-21kworker/0:1+events23:47:460
16476999216211,2148cyclictest185556-21kworker/0:1+events20:16:170
1647699921579,2145cyclictest179386-21kworker/0:3+events20:07:520
16476999215711,2142cyclictest175817-21kworker/0:2+events19:41:460
16476999215510,2141cyclictest251096-21kworker/0:1+events23:41:580
16476999215410,2140cyclictest255958-21kworker/0:2+events00:22:200
16476999215311,2139cyclictest244734-21kworker/0:2+events23:23:380
16476999215311,2138cyclictest263288-21kworker/0:0+events00:19:180
16476999215211,2138cyclictest179386-21kworker/0:3+events19:57:170
16476999215111,2137cyclictest185556-21kworker/0:1+events20:21:110
16477899214411,2130cyclictest169789-21kworker/3:1+events19:25:233
16476999214310,2130cyclictest255958-21kworker/0:2+events00:26:100
16476999214210,2128cyclictest222328-21kworker/0:2+events22:22:510
16476999214010,2127cyclictest232018-21kworker/0:0+events22:46:510
16476999214010,2127cyclictest219104-21kworker/0:0+events22:16:430
16476999214010,2127cyclictest188544-21kworker/0:2+events20:31:190
1647779921389,2125cyclictest223810-21kworker/2:1+events22:44:342
16476999213811,2124cyclictest243067-21kworker/0:0+events23:34:410
1647699921377,2127cyclictest195141-21kworker/0:2+events20:44:140
16477899213211,2117cyclictest230571-21kworker/3:1+events22:41:563
1647779921309,2118cyclictest179009-21kworker/2:1+events20:01:282
1647699921288,2117cyclictest204251-21kworker/0:0+events21:13:030
16476999212411,2110cyclictest204251-21kworker/0:0+events21:30:500
16477899212310,2110cyclictest240004-21kworker/3:1+events23:20:163
16477899212110,2108cyclictest195292-21kworker/3:1+events20:52:193
1647779921218,2109cyclictest223810-21kworker/2:1+events22:28:552
16477899212010,2107cyclictest211021-21kworker/3:0+events21:46:003
16477899211811,2104cyclictest177561-21kworker/3:1+events19:59:313
16477899211811,2104cyclictest167860-21kworker/3:2+events19:38:433
16477899211611,2102cyclictest235129-21kworker/3:2+events22:48:453
16477899211110,2097cyclictest219134-21kworker/3:0+events22:01:073
1647789921088,2096cyclictest203160-21kworker/3:0+events21:17:113
1647789921088,2096cyclictest203160-21kworker/3:0+events21:17:113
16477899210711,2092cyclictest247924-21kworker/3:0+events00:08:383
16477899210611,2091cyclictest247924-21kworker/3:0+events23:36:573
1647789921059,2093cyclictest247924-21kworker/3:0+events23:51:163
16477899210411,2090cyclictest209398-21kworker/3:2+events21:36:313
16477899210410,2091cyclictest262659-21kworker/3:1+events00:21:573
16477899210211,2088cyclictest169789-21kworker/3:1+events19:34:563
16477899210111,2088cyclictest267308-21kworker/3:0+events00:35:113
1647789920909,2078cyclictest236754-21kworker/3:0+events23:08:473
16477899208610,2073cyclictest227092-21kworker/3:2+events22:35:223
1647779920804,2073cyclictest164426-21kworker/2:1+events19:12:462
1647789920576,2047cyclictest193314-21kworker/3:0+events20:40:493
16477899205610,2043cyclictest191683-21kworker/3:2+events20:36:383
1647699920557,2045cyclictest255958-21kworker/0:2+events23:53:300
16477799205010,2037cyclictest183905-21kworker/2:2+events20:15:292
1647789920487,2038cyclictest260922-21kworker/3:2+events00:11:163
16477899201210,1998cyclictest219134-21kworker/3:0+events22:24:233
1647699920058,1993cyclictest190198-21kworker/0:0+events20:48:280
1647699919973,1991cyclictest255958-21kworker/0:2+events00:38:580
1647699919728,1961cyclictest161076-21kworker/0:1+events19:12:230
1647699919627,1952cyclictest161076-21kworker/0:1+events19:17:200
1647789919599,1947cyclictest240004-21kworker/3:1+events23:17:133
16477899195610,1943cyclictest162757-21kworker/3:0+events19:17:423
1647789919518,1940cyclictest195292-21kworker/3:1+events20:49:383
1647789919466,1937cyclictest162757-21kworker/3:0+events19:12:013
1641992157127,25sleep30-21swapper/319:05:403
1644622146110,31sleep00-21swapper/019:08:480
1632022146106,34sleep10-21swapper/119:05:011
1641932138109,25sleep20-21swapper/219:05:352
164778992011,2cyclictest191rcu_preempt22:35:013
16477799181,14cyclictest243251-21munin-run23:15:002
16477899161,7cyclictest52-21ksoftirqd/322:25:183
164777991612,3cyclictest0-21swapper/220:15:012
164777991610,4cyclictest238830-21/usr/sbin/munin23:00:122
16477899150,9cyclictest52-21ksoftirqd/321:20:173
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional