You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-10 - 09:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackfslot7s.osadl.org (updated Fri Apr 10, 2026 00:44:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
303376699303014,3012cyclictest3137962-21kworker/1:1+events00:35:011
303376699288814,2870cyclictest3059500-21kworker/1:2+events20:55:151
303376999280612,2790cyclictest3109561-21kworker/2:0+events23:11:072
303376999279714,2781cyclictest3037726-21kworker/2:0+events19:30:152
303376999275213,2733cyclictest3093222-21kworker/2:0+events22:15:122
303376999274714,2729cyclictest3106303-21kworker/2:2+events22:55:112
303375999274213,2726cyclictest3112589-21kworker/0:2+events23:15:130
303376699273610,2722cyclictest3117520-21kworker/1:1+events23:35:121
303375999271812,2704cyclictest3062836-21kworker/0:1+events20:45:210
303375999271412,2698cyclictest3093428-21kworker/0:3+events22:22:490
303376699270613,2689cyclictest3112647-21kworker/1:2+events23:58:121
30337669927029,2689cyclictest3059500-21kworker/1:2+events20:50:181
303376999269313,2676cyclictest3078888-21kworker/2:2+events21:34:132
303376999269111,2676cyclictest3073058-21kworker/2:1+events21:15:172
303375999267711,2662cyclictest3083652-21kworker/0:0+events21:45:190
303377499266713,2650cyclictest3046764-21kworker/3:2+events21:00:293
303375999264012,2623cyclictest3044935-21kworker/0:1+events19:50:180
303376699263513,2619cyclictest3049746-21kworker/1:0+events20:10:181
303375999260713,2591cyclictest3099645-21kworker/0:0+events22:39:540
303376999257712,2561cyclictest3120519-21kworker/2:2+events23:54:282
303376999257011,2556cyclictest3083807-21kworker/2:0+events21:58:372
303376999256912,2554cyclictest3037726-21kworker/2:0+events19:44:212
303376999256712,2551cyclictest3111223-21kworker/2:1+events23:09:582
303376999256512,2549cyclictest3111223-21kworker/2:1+events23:16:212
30337599925628,2550cyclictest3086887-21kworker/0:0+events22:00:290
303376999255811,2544cyclictest3133269-21kworker/2:1+events00:22:172
303376999255612,2540cyclictest3133269-21kworker/2:1+events00:30:132
303376999255012,2535cyclictest3120519-21kworker/2:2+events23:58:342
303376999254710,2533cyclictest3077258-21kworker/2:0+events21:37:172
303376999254411,2530cyclictest3130069-21kworker/2:0+events00:12:422
303375999253911,2524cyclictest3127117-21kworker/0:2+events00:15:210
30337599925369,2522cyclictest3044935-21kworker/0:1+events19:55:130
303376999253412,2518cyclictest3067682-21kworker/2:0+events21:07:172
303375999253311,2518cyclictest3120663-21kworker/0:2+events23:54:050
303375999253212,2516cyclictest3061179-21kworker/0:0+events20:33:510
303376999252212,2508cyclictest3083807-21kworker/2:0+events22:00:072
303375999251811,2503cyclictest3109713-21kworker/0:0+events23:05:260
303375999251412,2498cyclictest3066121-21kworker/0:0+events21:18:450
303375999250813,2492cyclictest3038444-21kworker/0:0+events19:24:380
303375999250812,2493cyclictest3041702-21kworker/0:2+events19:43:120
303376999250712,2491cyclictest3051774-21kworker/2:1+events20:11:242
30337699925003,2494cyclictest3067682-21kworker/2:0+events20:54:512
303376999249912,2483cyclictest3055129-21kworker/2:2+events20:17:252
303376999249810,2484cyclictest3033367-21kworker/2:1+events19:20:242
303376999249712,2481cyclictest3093222-21kworker/2:0+events22:47:522
303375999249711,2483cyclictest3101333-21kworker/0:2+events22:43:170
303376999249612,2481cyclictest3120519-21kworker/2:2+events23:42:022
303376999249512,2480cyclictest3133269-21kworker/2:1+events00:26:002
303376999249412,2479cyclictest3091468-21kworker/2:2+events22:08:482
303376699249412,2479cyclictest3072788-21kworker/1:1+events21:17:131
303376999249312,2478cyclictest3093222-21kworker/2:0+events22:24:442
303376999249312,2478cyclictest3071188-21kworker/2:2+events21:10:182
303376999249312,2478cyclictest3053437-21kworker/2:0+events20:25:252
303376999249312,2477cyclictest3098353-21kworker/2:2+events22:34:162
303376999249311,2478cyclictest3130069-21kworker/2:0+events00:07:262
303376999249212,2477cyclictest3053437-21kworker/2:0+events20:23:352
303376999249212,2477cyclictest3053437-21kworker/2:0+events20:23:342
303376999249212,2477cyclictest3046021-21kworker/2:1+events19:56:212
303376999249212,2476cyclictest3120519-21kworker/2:2+events23:49:102
303376999249212,2476cyclictest3046021-21kworker/2:1+events19:48:522
303376999249112,2476cyclictest3046021-21kworker/2:1+events19:51:492
303376999249010,2475cyclictest3101244-21kworker/2:1+events22:54:242
303376699248812,2472cyclictest3034888-21kworker/1:0+events19:20:461
303376999248712,2472cyclictest3049833-21kworker/2:2+events20:03:052
30337669924878,2475cyclictest3067621-21kworker/1:0+events21:09:551
303376699248511,2470cyclictest3114241-21kworker/1:0+events23:18:141
303376999248312,2467cyclictest3131662-21kworker/2:2+events00:18:042
303376699248312,2468cyclictest3104704-21kworker/1:1+events22:55:551
303376699248312,2467cyclictest3085504-21kworker/1:0+events21:52:511
303376699248312,2467cyclictest3072788-21kworker/1:1+events21:28:351
303376699248312,2467cyclictest3043458-21kworker/1:2+events19:43:581
303376999248212,2467cyclictest3077258-21kworker/2:0+events21:23:402
303376999248211,2468cyclictest3118883-21kworker/2:0+events23:34:022
303376999248211,2467cyclictest3101244-21kworker/2:1+events22:43:382
303376699248112,2465cyclictest3067621-21kworker/1:0+events21:11:531
303376699248110,2467cyclictest3093282-21kworker/1:1+events22:14:501
303376699248011,2465cyclictest3062739-21kworker/1:1+events20:41:301
303376999247812,2462cyclictest3062801-21kworker/2:1+events20:41:092
303376999247612,2461cyclictest3037726-21kworker/2:0+events19:38:342
303376999247512,2459cyclictest3109561-21kworker/2:0+events23:22:412
303376699247110,2457cyclictest3049746-21kworker/1:0+events20:23:571
303376699247110,2457cyclictest3049746-21kworker/1:0+events20:23:561
303376999246811,2453cyclictest3106303-21kworker/2:2+events23:01:132
303376999246512,2450cyclictest3083807-21kworker/2:0+events21:49:492
303376999246412,2449cyclictest3114520-21kworker/2:2+events23:25:182
303375999246312,2448cyclictest3133517-21kworker/0:1+events00:20:240
303376699246011,2446cyclictest3104704-21kworker/1:1+events22:54:001
303375999246012,2445cyclictest3104556-21kworker/0:1+events22:50:290
303375999246011,2445cyclictest3041702-21kworker/0:2+events19:47:010
30337699924599,2447cyclictest3062801-21kworker/2:1+events20:39:392
30337669924578,2446cyclictest3122379-21kworker/1:1+events23:49:541
303375999245710,2444cyclictest3109713-21kworker/0:0+events23:27:110
303376699245612,2440cyclictest3104704-21kworker/1:1+events23:14:531
303376999245512,2439cyclictest3138278-21kworker/2:0+events00:39:312
303375999245512,2440cyclictest3086887-21kworker/0:0+events22:11:040
303375999245411,2440cyclictest3056333-21kworker/0:0+events20:21:380
303375999245411,2440cyclictest3056333-21kworker/0:0+events20:21:380
303376999244812,2433cyclictest3069544-21kworker/2:1+events21:03:072
303376699244811,2434cyclictest3038571-21kworker/1:2+events19:27:591
303375999244812,2433cyclictest3127117-21kworker/0:2+events00:07:040
303375999244811,2433cyclictest3091501-21kworker/0:1+events22:15:370
303375999244711,2432cyclictest3098041-21kworker/0:1+events22:28:080
303375999244711,2432cyclictest3030176-21kworker/0:1+events19:19:140
303375999244612,2432cyclictest3127117-21kworker/0:2+events00:26:470
303375999244612,2431cyclictest3066121-21kworker/0:0+events20:59:440
303375999244512,2430cyclictest3109713-21kworker/0:0+events23:38:330
303375999244412,2428cyclictest3109713-21kworker/0:0+events23:11:300
303375999244312,2428cyclictest3083652-21kworker/0:0+events21:43:000
303375999244312,2427cyclictest3061179-21kworker/0:0+events20:38:050
303375999244311,2429cyclictest3131752-21kworker/0:0+events00:13:050
303375999244210,2429cyclictest3082030-21kworker/0:1+events21:53:150
303375999244012,2425cyclictest3127117-21kworker/0:2+events00:02:510
303375999243912,2423cyclictest3077633-21kworker/0:2+events21:36:560
303375999243912,2423cyclictest3062836-21kworker/0:1+events20:54:290
303375999243711,2422cyclictest3041702-21kworker/0:2+events19:35:570
303375999243710,2423cyclictest3067854-21kworker/0:2+events21:03:290
303376999243612,2420cyclictest3093222-21kworker/2:0+events22:26:152
303376999243510,2422cyclictest3101244-21kworker/2:1+events22:37:162
303376999243310,2419cyclictest3061209-21kworker/2:0+events20:32:412
303375999243312,2417cyclictest3101333-21kworker/0:2+events22:45:580
303376699243213,2415cyclictest3112647-21kworker/1:2+events23:32:541
303375999243111,2417cyclictest3053468-21kworker/0:2+events20:10:400
303376699243012,2414cyclictest3062739-21kworker/1:1+events20:40:001
303375999243012,2414cyclictest3119003-21kworker/0:1+events23:34:500
303376699242812,2413cyclictest3109446-21kworker/1:0+events23:08:241
303375999242712,2412cyclictest3138074-21kworker/0:1+events00:32:470
303376699242612,2411cyclictest3135077-21kworker/1:0+events00:24:551
303375999242612,2411cyclictest3064519-21kworker/0:2+events20:44:360
303376699242412,2409cyclictest3112647-21kworker/1:2+events00:11:321
303375999242311,2408cyclictest3038444-21kworker/0:0+events19:32:280
303376699242112,2405cyclictest3072788-21kworker/1:1+events21:37:411
303376699242012,2405cyclictest3114241-21kworker/1:0+events23:22:191
303376699242012,2404cyclictest3109446-21kworker/1:0+events23:04:171
303376699241210,2399cyclictest3088401-21kworker/1:1+events22:07:151
303376699241012,2394cyclictest3112647-21kworker/1:2+events23:41:381
303376999240310,2390cyclictest3037726-21kworker/2:0+events19:29:052
303376699240112,2386cyclictest3098254-21kworker/1:1+events22:29:171
303376699240112,2386cyclictest3048151-21kworker/1:2+events19:51:061
303375999240112,2386cyclictest3077633-21kworker/0:2+events21:33:300
303376699240011,2385cyclictest3067621-21kworker/1:0+events21:03:511
303376699239911,2384cyclictest3085504-21kworker/1:0+events21:57:291
303376699239712,2383cyclictest3049746-21kworker/1:0+events20:25:041
303376699239712,2382cyclictest3072788-21kworker/1:1+events21:22:301
303376699239712,2381cyclictest3098254-21kworker/1:1+events22:42:541
303376699239612,2381cyclictest3093282-21kworker/1:1+events22:17:291
303376699239612,2380cyclictest3131845-21kworker/1:1+events00:18:511
303376699239612,2380cyclictest3103101-21kworker/1:0+events22:45:351
303376699239611,2382cyclictest3098254-21kworker/1:1+events22:37:591
303376699239611,2381cyclictest3048151-21kworker/1:2+events19:55:571
303376699239512,2380cyclictest3040167-21kworker/1:1+events19:47:441
303376699239412,2378cyclictest3056731-21kworker/1:1+events20:30:471
303376699238812,2373cyclictest3126097-21kworker/1:1+events00:04:001
303376699236612,2350cyclictest3085504-21kworker/1:0+events22:23:591
30337669923658,2354cyclictest3034888-21kworker/1:0+events19:18:501
303376699236212,2347cyclictest3133455-21kworker/1:2+events00:39:091
303376699236012,2345cyclictest3059500-21kworker/1:2+events20:48:011
303377499234212,2327cyclictest3046764-21kworker/3:2+events20:18:113
30337599923427,2332cyclictest3109713-21kworker/0:0+events23:21:340
303377499233211,2317cyclictest3070492-21kworker/3:1+events21:05:213
303376999233012,2314cyclictest3120519-21kworker/2:2+events23:38:552
30337699923288,2317cyclictest3073058-21kworker/2:1+events21:25:572
303377499232512,2310cyclictest3115719-21kworker/3:2+events23:26:253
303377499232312,2308cyclictest3062868-21kworker/3:1+events20:39:153
303377499232113,2304cyclictest3080782-21kworker/3:1+events21:53:373
303376699232012,2304cyclictest3124049-21kworker/1:0+events23:53:421
303376999231812,2303cyclictest3059619-21kworker/2:2+events20:48:232
303376999231412,2299cyclictest3083807-21kworker/2:0+events21:54:452
303376699231411,2300cyclictest3040167-21kworker/1:1+events19:39:441
303375999231111,2297cyclictest3106427-21kworker/0:2+events23:02:210
303376999230510,2291cyclictest3053437-21kworker/2:0+events20:09:552
303375999230211,2288cyclictest3044935-21kworker/0:1+events20:02:200
303375999229812,2283cyclictest3120663-21kworker/0:2+events23:47:420
303376699229310,2279cyclictest3095204-21kworker/1:2+events22:31:351
303375999229310,2279cyclictest3066121-21kworker/0:0+events21:12:590
303376699228811,2274cyclictest3040167-21kworker/1:1+events19:30:371
303376699227911,2265cyclictest3049746-21kworker/1:0+events20:02:421
303375999227611,2262cyclictest3072549-21kworker/0:1+events21:07:410
303376999226911,2254cyclictest3035323-21kworker/2:3+events19:19:372
303376999226710,2254cyclictest3088608-21kworker/2:1+events22:13:212
303376999226511,2251cyclictest3125488-21kworker/2:1+events00:04:222
303376999226510,2252cyclictest3078888-21kworker/2:2+events21:40:202
303375999226211,2247cyclictest3082030-21kworker/0:1+events21:55:560
303375999226110,2247cyclictest3044935-21kworker/0:1+events20:06:070
303376999225711,2243cyclictest3067682-21kworker/2:0+events20:59:012
303375999225412,2239cyclictest3136475-21kworker/0:0+events00:36:070
303375999224311,2229cyclictest3120663-21kworker/0:2+events23:44:160
303376699224111,2227cyclictest3112647-21kworker/1:2+events23:28:401
303375999224110,2227cyclictest3075852-21kworker/0:1+events21:27:510
303375999224010,2226cyclictest3122319-21kworker/0:1+events23:59:210
303375999223810,2225cyclictest3086887-21kworker/0:0+events22:06:300
303375999223610,2222cyclictest3059589-21kworker/0:1+events20:27:430
303375999223410,2220cyclictest3044935-21kworker/0:1+events20:17:480
30337599922339,2221cyclictest3098041-21kworker/0:1+events22:31:120
303375999223310,2220cyclictest3038444-21kworker/0:0+events19:26:500
303377499222011,2206cyclictest3086917-21kworker/3:0+events22:10:423
303376699221610,2202cyclictest3112647-21kworker/1:2+events00:06:421
30337669922149,2202cyclictest3083839-21kworker/1:2+events21:46:051
303376699221210,2198cyclictest3053314-21kworker/1:1+events20:08:251
303376699221210,2197cyclictest3085504-21kworker/1:0+events22:01:391
303376699220410,2191cyclictest3072788-21kworker/1:1+events21:40:441
303375999220111,2186cyclictest3066121-21kworker/0:0+events21:20:360
303377499219911,2185cyclictest3134900-21kworker/3:1+events00:38:003
303376699219911,2185cyclictest3133455-21kworker/1:2+events00:27:341
303376699219511,2181cyclictest3055056-21kworker/1:2+events20:15:341
303376699219311,2179cyclictest3072788-21kworker/1:1+events21:32:241
303375999217810,2165cyclictest3099645-21kworker/0:0+events22:58:320
303377499215211,2138cyclictest3041826-21kworker/3:1+events19:48:053
303377499211911,2104cyclictest3093014-21kworker/3:1+events22:15:593
303377499211711,2103cyclictest3099704-21kworker/3:2+events22:34:383
303377499211611,2102cyclictest3117340-21kworker/3:1+events23:52:573
303377499211610,2102cyclictest3046764-21kworker/3:2+events20:01:133
30337699921138,2102cyclictest3033367-21kworker/2:1+events19:13:042
303377499211210,2098cyclictest3093014-21kworker/3:1+events23:10:203
30337749921119,2097cyclictest3031589-21kworker/3:0+events19:34:003
303377499211110,2098cyclictest3130480-21kworker/3:0+events00:21:563
303377499210810,2095cyclictest3058337-21kworker/3:0+events20:28:273
303377499210111,2087cyclictest3050336-21kworker/3:0+events20:05:193
303377499209111,2077cyclictest3078982-21kworker/3:0+events21:35:463
303375999208210,2068cyclictest3033397-21kworker/0:0+events19:13:270
303376699203810,2024cyclictest3031618-21kworker/1:2+events19:14:121
30337749919705,1962cyclictest3111374-21kworker/3:0+events23:20:523
30337749919687,1958cyclictest3046764-21kworker/3:2+events21:28:133
30337749919645,1956cyclictest3086917-21kworker/3:0+events22:03:533
30337749919149,1902cyclictest3090113-21kworker/3:2+events22:05:233
30337749918968,1885cyclictest3023896-21kworker/3:1+events19:12:183
30334152161120,25sleep30-21swapper/319:08:433
30332662143109,29sleep00-21swapper/019:06:510
30331492143116,22sleep10-21swapper/119:05:201
30333882142107,30sleep20-21swapper/219:08:232
3033774993113,16cyclictest0-21swapper/321:30:183
303377499268,6cyclictest3113955-21grep23:15:223
3033774992615,8cyclictest3101568-21plymouth22:40:013
3033774992613,10cyclictest3136767-21idleruntime-cro00:30:003
3033774992515,7cyclictest3040362-21awk19:30:003
303377499247,13cyclictest3126047-21hddtemp_smartct23:55:143
303377499246,9cyclictest3097583-21ld22:25:173
3033774992416,5cyclictest3088631-21perf21:59:593
3033774992413,8cyclictest3108161-21idleruntime-cro23:00:013
303377499239,10cyclictest3095512-21sshd22:20:123
3033774992314,6cyclictest3123991-21awk23:50:003
303377499228,5cyclictest3066449-21apt20:50:123
3033774992215,5cyclictest3066200-21sleep20:50:003
3033774992212,7cyclictest3120832-21awk23:40:003
303377499219,2cyclictest355-21systemd-journal20:00:003
303377499215,6cyclictest3044511-21python319:40:193
303377499215,10cyclictest3063809-21ntpq20:40:173
3033774992117,3cyclictest3133470-21chrt00:18:533
3033774992115,2cyclictest3043518-21munin-run19:40:013
3033774992113,8cyclictest0-21swapper/323:40:173
3033774992113,5cyclictest0-21swapper/323:02:353
303377499206,9cyclictest3060609-21ntpq20:30:173
303377499206,7cyclictest3036583-21smartctl19:15:193
303377499206,5cyclictest0-21swapper/321:40:143
303377499204,13cyclictest3129102-21apt-get00:05:103
3033774992017,1cyclictest3137243-21hddtemp_smartct00:30:133
3033774992014,4cyclictest3131995-21munin-run00:15:003
3033774992010,8cyclictest3104925-21latency_hist22:50:003
303377499196,4cyclictest3076138-21date21:20:003
303377499195,9cyclictest3084885-21python321:45:223
303377499195,9cyclictest3054286-21ntpq20:10:143
303377499194,14cyclictest739-21runrttasks20:23:213
303377499194,14cyclictest739-21runrttasks20:23:213
3033774991913,4cyclictest739-21runrttasks22:41:123
3033774991913,2cyclictest0-21swapper/300:00:113
3033774991911,6cyclictest3038788-21awk19:25:003
303377499189,8cyclictest0-21swapper/323:35:013
303377499186,7cyclictest3076458-21df21:20:153
303377499184,7cyclictest3047596-21memory19:50:143
3033774991812,4cyclictest3069661-21awk21:00:003
303377499176,6cyclictest3110490-21ntpq23:05:163
303377499166,4cyclictest3073755-10awk21:10:213
3033774991611,3cyclictest3106559-21munin-plugin-st22:55:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional