You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-12 - 03:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackfslot7s.osadl.org (updated Tue Nov 11, 2025 00:44:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
302795699281813,2799cyclictest3124050-21kworker/3:1+events00:10:203
302795399277713,2762cyclictest3102949-21kworker/2:0+events23:00:222
302795399277612,2761cyclictest3081989-21kworker/2:2+events22:00:182
302795699277513,2759cyclictest3117599-21kworker/3:0+events23:45:183
302795199276313,2746cyclictest3080237-21kworker/1:0+events21:50:221
302795199275713,2740cyclictest3050791-21kworker/1:2+events20:30:161
302795699275613,2739cyclictest3101383-21kworker/3:2+events23:05:183
302795399275514,2737cyclictest3046188-21kworker/2:2+events20:20:192
302795699271613,2699cyclictest3070588-21kworker/3:2+events21:20:203
302795199269611,2683cyclictest3032968-21kworker/1:0+events19:25:181
302795399260313,2586cyclictest3088322-21kworker/2:2+events22:25:252
302795399255411,2539cyclictest3046188-21kworker/2:2+events20:15:002
302795199249512,2480cyclictest3085512-21kworker/1:1+events22:07:341
302795199249212,2477cyclictest3111347-21kworker/1:2+events23:27:031
302795199248312,2467cyclictest3099719-21kworker/1:1+events23:24:001
302795199247810,2464cyclictest3059623-21kworker/1:2+events20:53:281
302795199247712,2462cyclictest3083853-21kworker/1:0+events22:02:131
302795199247512,2460cyclictest3057277-21kworker/1:0+events20:48:341
302795199246011,2445cyclictest3095192-21kworker/1:2+events22:54:171
302795399245911,2446cyclictest3095460-21kworker/2:2+events22:40:092
302795199245311,2439cyclictest3131985-21kworker/1:2+events00:38:191
302795199245310,2440cyclictest3114673-21kworker/1:2+events23:41:031
302795199244712,2433cyclictest3099719-21kworker/1:1+events22:55:021
302795699244412,2428cyclictest3073786-21kworker/3:1+events21:30:153
302795199243911,2424cyclictest3091868-21kworker/1:2+events22:24:181
302795199243810,2426cyclictest3088261-21kworker/1:2+events22:15:051
302795699242012,2404cyclictest3085210-21kworker/3:2+events22:03:213
302795199241712,2402cyclictest3037535-21kworker/1:1+events19:50:311
302795199241711,2403cyclictest3072212-21kworker/1:1+events21:31:001
302795399241012,2393cyclictest3055708-21kworker/2:2+events20:35:132
302794699241012,2395cyclictest3049392-21kworker/0:1+events20:25:190
302795399240512,2390cyclictest3101322-21kworker/2:2+events22:59:592
302795199239711,2383cyclictest3085512-21kworker/1:1+events22:29:571
302795699239212,2377cyclictest3075385-21kworker/3:0+events21:37:003
302795399239112,2375cyclictest3046188-21kworker/2:2+events20:19:342
302795399238611,2371cyclictest3086692-21kworker/2:0+events22:16:132
302795399238412,2368cyclictest3099687-21kworker/2:0+events22:47:482
302795399238412,2368cyclictest3099687-21kworker/2:0+events22:47:482
302795399238312,2368cyclictest3049127-21kworker/2:1+events20:51:362
302795399238312,2367cyclictest3125595-21kworker/2:0+events00:17:422
302795399238312,2367cyclictest3080389-21kworker/2:1+events22:09:482
302795399238212,2366cyclictest3080389-21kworker/2:1+events21:57:382
302795399238011,2365cyclictest3034417-21kworker/2:2+events19:32:532
302795699237812,2363cyclictest3099750-21kworker/3:0+events22:57:403
302795399237814,2361cyclictest3063040-21kworker/2:0+events20:59:092
302795699237312,2358cyclictest3102981-21kworker/3:1+events23:01:043
302795399237012,2355cyclictest3107599-21kworker/2:2+events23:17:312
302795699236412,2348cyclictest3114036-21kworker/3:0+events23:35:223
30279519923644,2357cyclictest3072212-21kworker/1:1+events21:44:191
302795699236212,2348cyclictest3070588-21kworker/3:2+events21:40:063
302795699236212,2346cyclictest3109677-21kworker/3:1+events23:28:333
302795699235310,2340cyclictest3033102-21kworker/3:1+events19:25:413
302795699235212,2337cyclictest3053990-21kworker/3:0+events20:34:063
302795699235112,2336cyclictest3077019-21kworker/3:1+events21:52:163
302795699235111,2337cyclictest3094865-21kworker/3:1+events22:37:313
302795699235012,2336cyclictest3133408-21kworker/3:2+events00:36:473
302795399234810,2335cyclictest3116216-21kworker/2:1+events23:47:352
302795399234611,2331cyclictest3122347-21kworker/2:2+events00:35:162
302795199234312,2328cyclictest3078314-21kworker/1:2+events21:45:511
302795199234311,2328cyclictest3107839-21kworker/1:0+events23:15:341
302794699233812,2324cyclictest3098175-21kworker/0:1+events23:09:300
302795699233712,2322cyclictest3122403-21kworker/3:0+events00:06:353
302795699232910,2316cyclictest3127347-21kworker/3:0+events00:15:223
302794699232510,2312cyclictest3114554-21kworker/0:0+events23:44:070
302795699232311,2309cyclictest3102981-21kworker/3:1+events23:13:403
302795699232211,2308cyclictest3059377-21kworker/3:0+events21:05:333
302795199231011,2295cyclictest3128636-21kworker/1:2+events00:20:501
302794699230411,2291cyclictest3098175-21kworker/0:1+events22:45:080
302794699230411,2291cyclictest3098175-21kworker/0:1+events22:45:080
302795199230312,2288cyclictest3104366-21kworker/1:0+events23:02:381
302795699230211,2288cyclictest3042778-21kworker/3:1+events20:16:293
30279569922999,2286cyclictest3106176-21kworker/3:0+events23:18:163
302795199229312,2278cyclictest3106027-21kworker/1:2+events23:11:011
302795199227111,2257cyclictest3035930-21kworker/1:2+events19:32:291
30279569922689,2256cyclictest3059377-21kworker/3:0+events20:57:163
302795199226611,2252cyclictest3113075-21kworker/1:0+events23:31:571
30279519922659,2252cyclictest3024261-21kworker/1:3+events19:23:471
302795199226410,2251cyclictest3099719-21kworker/1:1+events23:06:491
302795199226410,2251cyclictest3035930-21kworker/1:2+events19:35:541
302795199226111,2247cyclictest3120839-21kworker/1:2+events23:56:191
302795199226110,2248cyclictest3073815-21kworker/1:2+events21:29:501
302795199226110,2248cyclictest3044595-21kworker/1:2+events20:01:351
30279519922569,2244cyclictest3085512-21kworker/1:1+events22:13:141
30279519922559,2243cyclictest3065665-21kworker/1:1+events21:04:001
302795199225510,2241cyclictest3117569-21kworker/1:1+events23:48:211
302795399225012,2235cyclictest3059314-21kworker/2:2+events21:02:542
302795199224711,2232cyclictest3044595-21kworker/1:2+events20:06:361
302795399224210,2228cyclictest3053749-21kworker/2:0+events20:27:112
302795199223810,2225cyclictest3041001-21kworker/1:2+events19:49:451
302794699223811,2223cyclictest3093723-21kworker/0:2+events22:42:260
30279519922379,2225cyclictest3099719-21kworker/1:1+events22:48:581
30279519922379,2225cyclictest3099719-21kworker/1:1+events22:48:581
302795199223111,2217cyclictest3057277-21kworker/1:0+events20:55:421
30279519922308,2218cyclictest3122433-21kworker/1:1+events00:27:591
30279519922308,2218cyclictest3122433-21kworker/1:1+events00:27:591
30279519922299,2217cyclictest3128636-21kworker/1:2+events00:18:051
302795199222911,2215cyclictest3068922-21kworker/1:1+events21:17:371
302795199222911,2215cyclictest3052473-21kworker/1:0+events20:26:041
302795199222811,2214cyclictest3085512-21kworker/1:1+events22:37:091
30279519922279,2214cyclictest3081836-21kworker/1:2+events21:56:311
302795199222711,2213cyclictest3122433-21kworker/1:1+events00:30:381
302795199222711,2213cyclictest3119173-21kworker/1:0+events23:52:331
302795199222611,2212cyclictest3054612-21kworker/1:1+events20:37:091
30279519922259,2213cyclictest3027538-21kworker/1:0+events19:15:361
302795199222411,2209cyclictest3042957-21kworker/1:0+events19:58:121
30279519922239,2211cyclictest3122701-21kworker/1:3+events00:09:351
302795199222311,2209cyclictest3075415-21kworker/1:0+events21:38:341
302795199222310,2210cyclictest3098031-21kworker/1:0+events22:42:481
302794699222210,2209cyclictest3091809-21kworker/0:1+events22:23:080
302795199222111,2207cyclictest3049097-21kworker/1:1+events20:18:231
302795399222012,2205cyclictest3044536-21kworker/2:0+events20:09:162
302795699221911,2205cyclictest3042778-21kworker/3:1+events20:06:133
302795699221910,2206cyclictest3088443-21kworker/3:2+events22:20:493
302795199220711,2192cyclictest3114673-21kworker/1:2+events23:36:091
302795699220010,2187cyclictest3068953-21kworker/3:0+events21:18:483
302795399219610,2183cyclictest3070524-21kworker/2:1+events21:24:532
302795399219012,2175cyclictest3088322-21kworker/2:2+events22:20:242
302795699218410,2171cyclictest3035813-21kworker/3:0+events19:35:343
30279539921849,2171cyclictest3026241-21kworker/2:0+events19:21:322
302795399218411,2170cyclictest3072271-21kworker/2:2+events21:32:302
302795199218411,2170cyclictest3044595-21kworker/1:2+events20:11:121
30279539921839,2171cyclictest3059314-21kworker/2:2+events21:06:182
302795399218311,2169cyclictest3037626-21kworker/2:0+events19:53:132
302795399218110,2167cyclictest3119512-21kworker/2:0+events23:56:432
302795399218013,2163cyclictest3041094-21kworker/2:1+events19:57:262
302795399218011,2166cyclictest3072271-21kworker/2:2+events21:35:322
302795399218011,2165cyclictest3114615-21kworker/2:0+events23:37:182
302795399218010,2167cyclictest3122347-21kworker/2:2+events00:26:052
302795399218010,2167cyclictest3122347-21kworker/2:2+events00:26:052
302795699217912,2164cyclictest3101383-21kworker/3:2+events22:54:393
302795399217911,2164cyclictest3093589-21kworker/2:1+events22:39:242
302795399217911,2164cyclictest3078524-21kworker/2:2+events21:48:082
302795399217910,2166cyclictest3102949-21kworker/2:0+events23:05:412
302795699217611,2161cyclictest3059377-21kworker/3:0+events20:50:493
302795399217611,2162cyclictest3059314-21kworker/2:2+events21:15:452
302795699217511,2162cyclictest3035813-21kworker/3:0+events19:55:073
30279539921759,2163cyclictest3102949-21kworker/2:0+events23:13:182
302795399217510,2162cyclictest3111411-21kworker/2:0+events23:27:262
302795399217510,2162cyclictest3031169-21kworker/2:1+events19:36:412
302795699217410,2161cyclictest3052260-21kworker/3:2+events20:28:443
302795399217411,2160cyclictest3133436-21kworker/2:1+events00:32:562
302795399217411,2160cyclictest3093589-21kworker/2:1+events22:52:232
30279539921738,2162cyclictest3075236-21kworker/2:0+events21:51:542
302795699217011,2156cyclictest3035813-21kworker/3:0+events19:53:373
302795699216811,2154cyclictest3052260-21kworker/3:2+events20:24:093
302795699216811,2153cyclictest3094865-21kworker/3:1+events22:32:393
302794699216810,2155cyclictest3117811-21kworker/0:1+events23:49:540
302795699216710,2154cyclictest3092051-21kworker/3:0+events22:27:163
302795699216710,2154cyclictest3024460-21kworker/3:0+events19:24:093
302795399216710,2154cyclictest3091963-21kworker/2:0+events22:33:232
302795399216611,2152cyclictest3037626-21kworker/2:0+events19:40:532
302795699216411,2150cyclictest3077019-21kworker/3:1+events21:59:323
302795699216410,2151cyclictest3120809-21kworker/3:2+events23:54:293
30279539921649,2152cyclictest3076865-21kworker/2:1+events21:42:262
302795399216410,2150cyclictest3107599-21kworker/2:2+events23:23:122
30279569921638,2151cyclictest3094865-21kworker/3:1+events22:48:343
30279569921638,2151cyclictest3094865-21kworker/3:1+events22:48:333
302795699216310,2150cyclictest3128964-21kworker/3:1+events00:26:513
302795699216310,2150cyclictest3128964-21kworker/3:1+events00:26:513
302795699216310,2150cyclictest3120809-21kworker/3:2+events23:59:193
30279569921628,2151cyclictest3120809-21kworker/3:2+events00:01:133
302795699216211,2148cyclictest3114036-21kworker/3:0+events23:32:433
30279539921619,2148cyclictest3086692-21kworker/2:0+events22:11:202
30279569921609,2147cyclictest3035813-21kworker/3:0+events19:41:163
302795699216010,2146cyclictest3077019-21kworker/3:1+events21:47:443
30279569921599,2147cyclictest3068953-21kworker/3:0+events21:28:003
302795399215811,2143cyclictest3125595-21kworker/2:0+events00:11:302
302795699215710,2144cyclictest3109677-21kworker/3:1+events23:22:033
302795399215511,2141cyclictest3026241-21kworker/2:0+events19:17:032
302795399215311,2139cyclictest3114615-21kworker/2:0+events23:42:342
302795399215311,2139cyclictest3072271-21kworker/2:2+events21:26:502
302795699215011,2136cyclictest3035813-21kworker/3:0+events19:34:023
30279569921498,2138cyclictest3039305-21kworker/3:2+events19:47:273
302795699214211,2127cyclictest3055859-21kworker/3:1+events20:39:253
302795699214110,2128cyclictest3128964-21kworker/3:1+events00:30:163
30279569921408,2129cyclictest3086813-21kworker/3:1+events22:14:433
302795699214010,2129cyclictest3042778-21kworker/3:1+events20:10:033
30279469921376,2128cyclictest3041240-21kworker/0:2+events20:10:470
302795699213611,2122cyclictest3128964-21kworker/3:1+events00:24:113
302795399213111,2117cyclictest3057400-21kworker/2:0+events20:43:182
302795199212711,2114cyclictest3057277-21kworker/1:0+events21:05:091
30279519921259,2113cyclictest3050791-21kworker/1:2+events20:21:271
302794699211910,2106cyclictest3073665-21kworker/0:1+events21:40:520
302794699211811,2104cyclictest3130344-21kworker/0:0+events00:33:430
302794699211711,2103cyclictest3098175-21kworker/0:1+events23:15:580
302795699211610,2103cyclictest3029373-21kworker/3:1+events19:18:323
302794699211611,2102cyclictest3098175-21kworker/0:1+events23:01:510
302794699211610,2102cyclictest3047551-21kworker/0:0+events20:16:060
30279539921157,2104cyclictest3041094-21kworker/2:1+events20:02:212
302794699211510,2102cyclictest3111379-21kworker/0:2+events23:33:060
302795699211310,2100cyclictest3085210-21kworker/3:2+events22:06:023
30279519921137,2103cyclictest3122701-21kworker/1:3+events00:03:541
302794699211311,2099cyclictest3065812-21kworker/0:1+events21:04:450
302794699211310,2099cyclictest3062688-21kworker/0:0+events21:11:350
30279519921129,2100cyclictest3068922-21kworker/1:1+events21:13:501
302794699211211,2098cyclictest3062688-21kworker/0:0+events21:32:100
30279539921117,2101cyclictest3128932-21kworker/2:1+events00:23:262
302795199211010,2096cyclictest3032968-21kworker/1:0+events19:44:211
302794699210810,2095cyclictest3130344-21kworker/0:0+events00:23:460
30279469921079,2094cyclictest3032880-21kworker/0:1+events19:26:470
302794699210711,2092cyclictest3127225-21kworker/0:2+events00:38:430
302794699210710,2094cyclictest3098175-21kworker/0:1+events23:22:500
30279469921048,2093cyclictest3127225-21kworker/0:2+events00:14:120
302794699210411,2090cyclictest3117811-21kworker/0:1+events00:16:550
302794699210410,2091cyclictest3119449-21kworker/0:2+events23:51:470
302795699210311,2089cyclictest3094865-21kworker/3:1+events22:43:583
302795399210310,2090cyclictest3067236-21kworker/2:1+events21:10:522
302794699210311,2089cyclictest3049392-21kworker/0:1+events20:40:340
30279469920947,2084cyclictest3059016-21kworker/0:2+events20:53:520
302794699209210,2079cyclictest3059016-21kworker/0:2+events20:57:590
302795199209010,2077cyclictest3024261-21kworker/1:3+events19:14:491
30279519920799,2067cyclictest3127045-21kworker/1:0+events00:11:541
30279569920745,2066cyclictest3088443-21kworker/3:2+events22:15:283
302795199206910,2055cyclictest3057277-21kworker/1:0+events21:22:161
30279469920697,2059cyclictest3062688-21kworker/0:0+events21:27:370
302795199206510,2051cyclictest3054612-21kworker/1:1+events20:42:071
302794699206210,2049cyclictest3036215-21kworker/0:1+events19:44:440
302795199205110,2038cyclictest3093497-21kworker/1:0+events22:33:001
30279539920479,2035cyclictest3116216-21kworker/2:1+events23:54:522
302795399204710,2033cyclictest3053749-21kworker/2:0+events20:30:382
302795399204610,2033cyclictest3031169-21kworker/2:1+events19:27:562
302795699203011,2016cyclictest3044506-21kworker/3:2+events20:02:443
302795699202910,2016cyclictest3059377-21kworker/3:0+events20:45:323
30279569920287,2018cyclictest3117599-21kworker/3:0+events23:43:443
30279539920289,2016cyclictest3123993-21kworker/2:1+events00:04:402
302795399202410,2011cyclictest3049127-21kworker/2:1+events20:48:132
302795399202010,2007cyclictest3125595-21kworker/2:0+events00:08:502
302795699201610,2002cyclictest3065695-21kworker/3:1+events21:02:333
302794699200910,1996cyclictest3062688-21kworker/0:0+events21:08:360
302795699200710,1993cyclictest3067325-21kworker/3:2+events21:11:573
30279569920007,1990cyclictest3057458-21kworker/3:2+events20:43:393
302795399198910,1976cyclictest3026241-21kworker/2:0+events19:14:022
302795699198810,1975cyclictest3019753-21kworker/3:2+events19:13:403
30279539919868,1975cyclictest3037626-21kworker/2:0+events19:48:582
30279539919688,1957cyclictest3114615-21kworker/2:0+events23:34:592
302794699196510,1952cyclictest3076688-21kworker/0:2+events21:39:190
302794699196510,1951cyclictest3019513-21kworker/0:2+events19:18:540
30279469919639,1951cyclictest3093723-21kworker/0:2+events22:36:220
30279469919612,1956cyclictest3123961-21kworker/0:0+events00:03:310
30279469919587,1948cyclictest3039574-21kworker/0:0+events19:51:180
302794699195810,1945cyclictest3083795-21kworker/0:0+events22:10:570
302794699195310,1940cyclictest3049392-21kworker/0:1+events20:31:250
302794699195210,1938cyclictest3080534-21kworker/0:1+events21:59:100
30279469919498,1938cyclictest3029472-21kworker/0:0+events19:23:250
302794699194910,1936cyclictest3106238-21kworker/0:2+events23:11:230
30279469919482,1943cyclictest3041240-21kworker/0:2+events20:07:230
30279469919479,1935cyclictest3041240-21kworker/0:2+events19:55:280
30279469919469,1933cyclictest3067418-21kworker/0:2+events21:19:100
30279469919468,1935cyclictest3055646-21kworker/0:0+events20:35:350
30279469919457,1935cyclictest3080534-21kworker/0:1+events22:07:570
30279469919438,1932cyclictest3083795-21kworker/0:0+events22:25:490
302794699192810,1915cyclictest3083795-21kworker/0:0+events22:04:090
30279469919259,1913cyclictest3076688-21kworker/0:2+events21:51:080
30277152150100,30sleep00-21swapper/019:09:140
30276352141108,28sleep30-21swapper/319:08:133
30277162139110,24sleep10-21swapper/119:09:161
30275672138103,30sleep20-21swapper/219:07:242
3027946999088,1cyclictest3034537-21kworker/0:2+pm19:39:570
3027946992316,4cyclictest3041213-21awk19:50:000
302794699229,9cyclictest3036193-21munin-run19:35:000
302794699226,5cyclictest873-10wireplumber22:55:000
302794699201,3cyclictest16-21ksoftirqd/023:35:180
302794699191,16cyclictest744-21snmpd22:19:510
3027946991911,3cyclictest3112073-21smartctl23:25:180
3027946991812,4cyclictest3125007-21/usr/sbin/munin00:05:180
302794699178,6cyclictest3103146-21perf23:00:000
302794699166,7cyclictest3122729-21date00:00:000
302794699161,9cyclictest3059947-21irqstats20:45:160
3027946991511,3cyclictest3044880-21/usr/sbin/munin20:00:130
302794699148,4cyclictest3072239-21taskset21:24:190
302794699146,6cyclictest3052549-21awk20:25:000
302794699140,3cyclictest3028859-21/usr/sbin/munin19:10:210
302794699137,4cyclictest171ktimers/021:45:160
302794699131,11cyclictest3130860-21apt-get00:25:130
302794699131,11cyclictest3130860-21apt-get00:25:130
302794699130,12cyclictest3094677-21/usr/sbin/munin22:30:220
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional