You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-27 - 16:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackfslot7s.osadl.org (updated Mon Apr 27, 2026 00:44:18)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
275908699338314,3365cyclictest2855753-21kworker/0:1+events00:25:010
275910099304914,3030cyclictest2764216-21kworker/3:1+events19:30:133
275910099298214,2962cyclictest2767453-21kworker/3:2+events19:35:123
275908699290410,2889cyclictest2855753-21kworker/0:1+events00:15:140
275910099281713,2798cyclictest2812282-21kworker/3:0+events21:55:183
275909599274213,2727cyclictest2831255-21kworker/2:1+events23:45:192
275909599272613,2711cyclictest2782065-21kworker/2:3+events20:30:022
27590909926999,2685cyclictest2850595-21kworker/1:1+events00:00:021
27590909926999,2685cyclictest2850595-21kworker/1:1+events00:00:011
275910099269213,2676cyclictest2758607-21kworker/3:1+events19:15:003
275910099265813,2643cyclictest2821557-21kworker/3:0+events22:50:183
275910099262913,2613cyclictest2786725-21kworker/3:1+events20:38:083
275910099253512,2519cyclictest2775584-21kworker/3:1+events20:05:513
275908699253112,2515cyclictest2815303-21kworker/0:0+events22:07:100
275910099252611,2512cyclictest2809078-21kworker/3:0+events21:44:383
275910099252010,2507cyclictest2799380-21kworker/3:0+events21:30:313
275908699251910,2505cyclictest2788152-21kworker/0:1+events20:52:240
275910099251012,2494cyclictest2862595-21kworker/3:0+events00:29:353
275908699249612,2480cyclictest2807488-21kworker/0:0+events21:42:210
275908699249512,2479cyclictest2809105-21kworker/0:1+events21:48:510
275909599249412,2478cyclictest2831255-21kworker/2:1+events23:38:462
275908699249312,2478cyclictest2775380-21kworker/0:2+events20:04:180
275908699249012,2475cyclictest2838774-21kworker/0:1+events23:24:180
275910099248712,2471cyclictest2799380-21kworker/3:0+events21:27:513
275910099248512,2469cyclictest2852827-21kworker/3:4+events00:21:593
275909599247012,2456cyclictest2781794-21kworker/2:0+events20:25:052
275909599247012,2455cyclictest2823130-21kworker/2:2+events22:35:532
275910099246112,2446cyclictest2852827-21kworker/3:4+events00:30:223
275909599245612,2440cyclictest2823130-21kworker/2:2+events22:45:202
27590959924535,2444cyclictest2831255-21kworker/2:1+events22:55:142
275910099245112,2436cyclictest2831225-21kworker/3:2+events22:58:383
275910099245012,2434cyclictest2784892-21kworker/3:2+events20:47:023
275910099244911,2434cyclictest2741554-21kworker/3:2+events19:27:093
275908699244812,2433cyclictest2784861-21kworker/0:2+events20:33:290
275910099244712,2432cyclictest2821557-21kworker/3:0+events22:40:473
275910099244511,2431cyclictest2817319-21kworker/3:0+events22:12:313
275910099244511,2431cyclictest2817319-21kworker/3:0+events22:12:313
275909099244412,2429cyclictest2812252-21kworker/1:0+events21:55:431
275910099244112,2426cyclictest2807339-21kworker/3:1+events22:21:423
275910099244111,2426cyclictest2821557-21kworker/3:0+events22:47:593
275908699243812,2423cyclictest2775380-21kworker/0:2+events20:08:060
275910099243711,2423cyclictest2810682-21kworker/3:2+events21:50:463
27590959924364,2428cyclictest2781794-21kworker/2:0+events20:21:422
275910099243412,2419cyclictest2807339-21kworker/3:1+events21:38:353
275910099243412,2418cyclictest2789171-21kworker/3:0+events20:42:003
275910099243412,2418cyclictest2783296-21kworker/3:0+events20:31:103
275910099243312,2418cyclictest2773994-21kworker/3:2+events19:55:203
275908699243311,2418cyclictest2838774-21kworker/0:1+events23:36:050
275910099243212,2417cyclictest2794509-21kworker/3:2+events20:58:313
275910099243012,2415cyclictest2783296-21kworker/3:0+events20:23:313
275909599242913,2413cyclictest2776967-21kworker/2:0+events20:03:562
275910099242712,2411cyclictest2775584-21kworker/3:1+events20:12:163
275910099242611,2411cyclictest2837337-21kworker/3:1+events23:12:303
275909599242612,2410cyclictest2816175-21kworker/2:0+events22:26:412
275910099242512,2410cyclictest2765820-21kworker/3:0+events19:48:053
275910099242412,2409cyclictest2862595-21kworker/3:0+events00:36:233
275910099242412,2409cyclictest2783296-21kworker/3:0+events20:28:083
275910099242411,2410cyclictest2796137-21kworker/3:1+events21:23:353
275910099242311,2409cyclictest2849180-21kworker/3:0+events00:02:403
275910099242311,2409cyclictest2849180-21kworker/3:0+events00:02:403
275910099242211,2408cyclictest2844212-21kworker/3:0+events23:35:443
275910099242210,2409cyclictest2807339-21kworker/3:1+events22:17:503
275908699242212,2407cyclictest2813612-21kworker/0:1+events22:19:000
275910099242112,2406cyclictest2812282-21kworker/3:0+events22:03:443
275910099242112,2405cyclictest2840595-21kworker/3:2+events23:44:113
275910099242111,2406cyclictest2852827-21kworker/3:4+events00:14:523
275909599242112,2405cyclictest2782065-21kworker/2:3+events20:36:372
275910099242012,2405cyclictest2773994-21kworker/3:2+events20:00:553
275910099241912,2403cyclictest2799380-21kworker/3:0+events21:15:553
275908699241914,2402cyclictest2784861-21kworker/0:2+events20:56:380
275910099241611,2402cyclictest2791270-21kworker/3:1+events20:52:483
275909599241512,2400cyclictest2765669-21kworker/2:0+events19:34:492
275910099241312,2398cyclictest2775584-21kworker/3:1+events20:19:023
275909099241212,2396cyclictest2834574-21kworker/1:2+events23:14:441
275910099241011,2396cyclictest2807339-21kworker/3:1+events21:46:563
275910099240911,2394cyclictest2762593-21kworker/3:0+events19:22:393
275908699240910,2396cyclictest2827938-21kworker/0:0+events22:46:490
275909599240412,2388cyclictest2838878-21kworker/2:2+events23:25:262
275910099240312,2388cyclictest2792904-21kworker/3:0+events21:02:483
275908699240212,2386cyclictest2860848-21kworker/0:2+events00:27:410
275910099240112,2386cyclictest2821557-21kworker/3:0+events22:35:293
275908699240012,2386cyclictest2807488-21kworker/0:0+events21:50:230
275909599239812,2382cyclictest2795512-21kworker/2:1+events21:07:492
275909599239711,2382cyclictest2823130-21kworker/2:2+events22:41:332
275909599239710,2384cyclictest2856084-21kworker/2:0+events00:10:432
275909599239312,2378cyclictest2823130-21kworker/2:2+events22:33:342
275909599239012,2375cyclictest2831255-21kworker/2:1+events23:15:532
275909599239012,2374cyclictest2819925-21kworker/2:1+events22:21:192
275909599239012,2374cyclictest2805883-21kworker/2:2+events21:35:552
275908699239012,2375cyclictest2860848-21kworker/0:2+events00:32:570
275908699238912,2374cyclictest2794568-21kworker/0:0+events21:22:490
275909599238810,2375cyclictest2810470-21kworker/2:0+events21:56:512
27590959923879,2374cyclictest2800747-21kworker/2:0+events21:18:132
275909599238712,2371cyclictest2823130-21kworker/2:2+events22:51:242
275909599238712,2371cyclictest2782065-21kworker/2:3+events20:41:142
275909599238711,2372cyclictest2769133-21kworker/2:2+events19:38:592
27590959923869,2373cyclictest2844153-21kworker/2:2+events23:42:382
275909599238612,2371cyclictest2802376-21kworker/2:1+events21:26:402
275909599238612,2370cyclictest2831255-21kworker/2:1+events23:51:192
275909599238512,2370cyclictest2765669-21kworker/2:0+events19:27:552
275909099238512,2370cyclictest2765573-21kworker/1:2+events19:32:341
275909599238412,2369cyclictest2831255-21kworker/2:1+events23:59:372
275909599238412,2369cyclictest2831255-21kworker/2:1+events00:17:292
275909599238412,2369cyclictest2831255-21kworker/2:1+events00:03:282
275909599238412,2369cyclictest2831255-21kworker/2:1+events00:03:272
275909599238412,2369cyclictest2767277-21kworker/2:1+events19:40:532
275909599238412,2368cyclictest2831255-21kworker/2:1+events23:01:222
275909599238312,2368cyclictest2807577-21kworker/2:1+events21:42:442
275909599237910,2366cyclictest2810470-21kworker/2:0+events21:52:172
275909599237811,2364cyclictest2802376-21kworker/2:1+events21:30:552
275909599237711,2363cyclictest2831255-21kworker/2:1+events00:28:262
275909599237512,2360cyclictest2813970-21kworker/2:2+events22:02:132
275909599237512,2359cyclictest2795512-21kworker/2:1+events21:12:022
275909599237412,2359cyclictest2863982-21kworker/2:2+events00:35:592
275910099237211,2357cyclictest2852827-21kworker/3:4+events00:18:143
275908699236212,2347cyclictest2834337-21kworker/0:2+events23:04:020
275908699236112,2345cyclictest2765606-21kworker/0:2+events19:37:470
275908699236111,2347cyclictest2794568-21kworker/0:0+events21:19:220
275909599235811,2343cyclictest2795512-21kworker/2:1+events21:01:132
27590869923557,2344cyclictest2813612-21kworker/0:1+events22:04:310
275909599235212,2336cyclictest2793055-21kworker/2:2+events20:55:282
275908699235111,2336cyclictest2847552-21kworker/0:0+events23:59:140
275908699235110,2338cyclictest2784861-21kworker/0:2+events20:42:240
275908699235012,2336cyclictest2824369-21kworker/0:1+events22:41:120
275908699235012,2335cyclictest2842136-21kworker/0:2+events23:27:430
275908699235012,2334cyclictest2811010-21kworker/0:2+events21:57:590
275908699234912,2334cyclictest2824369-21kworker/0:1+events22:36:390
27591009923489,2336cyclictest2832829-21kworker/3:0+events23:25:483
275908699234712,2332cyclictest2802408-21kworker/0:2+events21:25:540
275908699234512,2331cyclictest2769316-21kworker/0:1+events19:40:070
275909599234411,2329cyclictest2786606-21kworker/2:0+events20:46:172
275909099234211,2329cyclictest2767307-21kworker/1:0+events19:49:161
275908699234112,2325cyclictest2775380-21kworker/0:2+events20:17:120
275908699234012,2324cyclictest2780341-21kworker/0:1+events20:23:090
275908699234012,2323cyclictest2845925-21kworker/0:2+events23:41:290
275909099233811,2324cyclictest2767307-21kworker/1:0+events20:25:491
275908699233712,2322cyclictest2796199-21kworker/0:1+events21:08:590
275908699233612,2321cyclictest2824369-21kworker/0:1+events22:31:160
275908699233512,2321cyclictest2834337-21kworker/0:2+events23:11:430
275908699232912,2313cyclictest2805853-21kworker/0:1+events21:37:480
275908699232911,2314cyclictest2860848-21kworker/0:2+events00:39:240
275908699232712,2311cyclictest2786517-21kworker/0:0+events20:35:510
275909599232411,2310cyclictest2760194-21kworker/2:0+events19:19:332
275908699232110,2307cyclictest2788152-21kworker/0:1+events20:49:420
275908699231912,2303cyclictest2813612-21kworker/0:1+events22:20:550
275908699231311,2299cyclictest2767340-21kworker/0:0+events19:58:370
275908699231112,2296cyclictest2802408-21kworker/0:2+events21:33:120
275909599230512,2290cyclictest2767277-21kworker/2:1+events19:57:332
275910099228910,2276cyclictest2796137-21kworker/3:1+events21:14:443
275909599227512,2259cyclictest2831255-21kworker/2:1+events23:31:352
27590869922618,2250cyclictest2849149-21kworker/0:2+events23:53:330
275910099225211,2237cyclictest2832829-21kworker/3:0+events23:09:013
275908699225012,2234cyclictest2831136-21kworker/0:0+events23:05:300
275910099224810,2234cyclictest2849180-21kworker/3:0+events23:53:113
275909599224012,2224cyclictest2831255-21kworker/2:1+events00:22:212
275909599223412,2218cyclictest2813970-21kworker/2:2+events22:14:252
275909599223412,2218cyclictest2813970-21kworker/2:2+events22:14:252
275910099223310,2221cyclictest2821871-21kworker/3:3+events22:30:053
275910099223211,2218cyclictest2741554-21kworker/3:2+events19:18:023
275910099223210,2218cyclictest2849180-21kworker/3:0+events00:08:063
275910099223110,2218cyclictest2840595-21kworker/3:2+events23:34:573
275908699223111,2216cyclictest2784861-21kworker/0:2+events20:29:400
275910099223011,2216cyclictest2821871-21kworker/3:3+events22:27:293
275910099223010,2217cyclictest2850874-21kworker/3:2+events23:55:273
275910099222910,2217cyclictest2794509-21kworker/3:2+events21:05:083
275910099222411,2209cyclictest2815483-21kworker/3:2+events22:07:563
275909599222311,2208cyclictest2832860-21kworker/2:2+events23:09:252
275910099222111,2207cyclictest2840595-21kworker/3:2+events23:23:063
275910099221510,2202cyclictest2770247-21kworker/3:1+events19:41:383
275910099220511,2191cyclictest2847582-21kworker/3:1+events23:46:273
275910099220410,2191cyclictest2837337-21kworker/3:1+events23:18:323
27590959922039,2190cyclictest2778548-21kworker/2:1+events20:15:402
275908699220312,2188cyclictest2758637-21kworker/0:0+events19:15:460
275910099220111,2186cyclictest2832829-21kworker/3:0+events23:02:053
275908699219611,2182cyclictest2767340-21kworker/0:0+events19:47:420
275908699219211,2177cyclictest2853229-21kworker/0:2+events00:01:320
275908699219211,2177cyclictest2853229-21kworker/0:2+events00:01:320
275909599219010,2177cyclictest2837275-21kworker/2:0+events23:10:562
275909599218911,2174cyclictest2819925-21kworker/2:1+events22:18:132
27590959921867,2176cyclictest2772602-21kworker/2:0+events19:54:112
27590959921867,2176cyclictest2772602-21kworker/2:0+events19:54:112
275909599218610,2172cyclictest2769133-21kworker/2:2+events19:47:192
275908699218510,2172cyclictest2818479-21kworker/0:2+events22:14:030
275908699218510,2172cyclictest2818479-21kworker/0:2+events22:14:030
27590959921839,2171cyclictest2778548-21kworker/2:1+events20:10:462
275909599218111,2167cyclictest2816175-21kworker/2:0+events22:07:342
275909599218011,2166cyclictest2776967-21kworker/2:0+events20:05:282
275909599217911,2165cyclictest2831255-21kworker/2:1+events00:06:102
275909599217511,2161cyclictest2802376-21kworker/2:1+events21:24:442
275908699217110,2158cyclictest2838774-21kworker/0:1+events23:31:110
275910099216910,2157cyclictest2770247-21kworker/3:1+events19:50:013
275910099216910,2157cyclictest2770247-21kworker/3:1+events19:50:013
275908699216611,2152cyclictest2796199-21kworker/0:1+events21:12:470
275909599216310,2150cyclictest2831255-21kworker/2:1+events23:20:282
275909599215810,2145cyclictest2863982-21kworker/2:2+events00:32:132
275908699214411,2129cyclictest2775380-21kworker/0:2+events20:11:090
275908699213812,2123cyclictest2855753-21kworker/0:1+events00:06:310
275909099213011,2115cyclictest2809019-21kworker/1:1+events22:00:191
275909099213011,2115cyclictest2809019-21kworker/1:1+events21:54:101
275908699212912,2114cyclictest2765606-21kworker/0:2+events19:27:330
275908699212311,2110cyclictest2821767-21kworker/0:2+events22:25:320
275908699212311,2109cyclictest2773757-21kworker/0:1+events19:54:360
275908699212311,2109cyclictest2773757-21kworker/0:1+events19:54:350
275909099212011,2106cyclictest2844096-21kworker/1:2+events23:52:251
275909099212011,2105cyclictest2762821-21kworker/1:0+events19:24:091
275908699211910,2106cyclictest2794568-21kworker/0:0+events21:04:220
275909099211811,2104cyclictest2856197-21kworker/1:2+events00:12:341
275908699211811,2104cyclictest2838774-21kworker/0:1+events23:45:420
275909099211711,2102cyclictest2814179-21kworker/1:2+events22:33:571
275908699211711,2103cyclictest2829533-21kworker/0:1+events22:54:500
275909099211611,2102cyclictest2842074-21kworker/1:0+events23:30:481
275908699211611,2102cyclictest2767340-21kworker/0:0+events19:34:030
275909599211510,2102cyclictest2745370-21kworker/2:2+events19:23:462
27590869921139,2100cyclictest2758637-21kworker/0:0+events19:20:190
27590869921037,2093cyclictest2831136-21kworker/0:0+events22:56:010
275908699209711,2084cyclictest2834337-21kworker/0:2+events23:15:310
27590959920747,2063cyclictest2810470-21kworker/2:0+events21:49:592
27590909920739,2061cyclictest2814179-21kworker/1:2+events22:28:361
275909099205710,2044cyclictest2824892-21kworker/1:0+events22:49:541
27590909920468,2035cyclictest2852633-21kworker/1:0+events00:34:281
27590959920264,2019cyclictest2786606-21kworker/2:0+events20:54:432
27590959920129,2000cyclictest2757441-21kworker/2:1+events19:11:112
275909099199610,1983cyclictest2767307-21kworker/1:0+events20:13:471
27590909919873,1981cyclictest2767307-21kworker/1:0+events21:06:191
275908699196210,1949cyclictest2857563-21kworker/0:2+events00:14:270
27590909919617,1951cyclictest2831354-21kworker/1:1+events22:56:221
27590909919609,1948cyclictest2797534-21kworker/1:2+events21:43:071
27590869919587,1948cyclictest2750396-21kworker/0:2+events19:13:300
275909099195210,1938cyclictest2837602-21kworker/1:1+events23:26:121
275909099194110,1928cyclictest2772452-21kworker/1:2+events19:52:421
275909099194110,1928cyclictest2772452-21kworker/1:2+events19:52:411
27590909919369,1924cyclictest2753506-21kworker/1:1+events19:25:381
27590909918927,1882cyclictest2844096-21kworker/1:2+events23:46:051
275909099157153,2cyclictest2850595-21kworker/1:1+pm23:59:501
27585862155111,23sleep30-21swapper/319:06:373
275909099143139,2cyclictest2824892-21kworker/1:0+pm23:09:591
27586652141110,26sleep20-21swapper/219:07:362
27586942139108,27sleep00-21swapper/019:07:580
27586302139108,26sleep10-21swapper/119:07:111
2759090992923,4cyclictest2826578-21idleruntime-cro22:40:001
2759090992517,6cyclictest2799440-21awk21:15:001
2759090992412,7cyclictest355-21systemd-journal23:22:111
275909099241,20cyclictest0-21swapper/100:20:121
275909099240,1cyclictest0-21swapper/123:15:131
275909099231,19cyclictest2810787-21basename21:50:001
275909099231,16cyclictest2816309-21ntpq22:05:171
275909099226,9cyclictest609-21NetworkManager20:05:121
275909099221,18cyclictest2806032-21awk21:35:001
275909099221,18cyclictest2801018-21perf21:19:591
275909099221,18cyclictest2788001-21taskset20:35:391
275909099211,17cyclictest2769380-21idleruntime-cro19:40:001
2759090992015,3cyclictest2794631-21seq20:59:341
275909099201,16cyclictest2802892-21cat21:25:001
275909099201,15cyclictest2820257-21perf22:20:011
275909099201,15cyclictest2807606-21munin-run21:39:591
275909099201,14cyclictest2818676-21perf22:15:001
275909099201,14cyclictest2818676-21perf22:15:001
275909099200,16cyclictest2785469-21apt-get20:30:121
275909099197,10cyclictest2774877-21ld19:55:171
275909099191,15cyclictest593-21dbus-daemon00:20:001
275909099191,15cyclictest2856120-21munin-run00:09:591
275909099191,15cyclictest2846089-21run-parts23:39:591
275909099191,15cyclictest2827088-21hddtemp_smartct22:40:161
275909099191,15cyclictest2760536-21dpkg19:15:001
275909099191,14cyclictest2865880-21basename00:40:001
275909099191,14cyclictest2783614-21munin-run20:25:001
275909099191,12cyclictest2802605-21apt-get21:25:011
275909099190,17cyclictest739-21runrttasks20:18:011
2759090991813,3cyclictest607-21rs:main1
275909099181,14cyclictest0-21swapper/122:50:131
275909099181,13cyclictest2795164-21df21:00:141
2759090991810,6cyclictest2846569-21diskstats23:40:141
275909099180,16cyclictest2777273-21cron20:05:001
275909099171,13cyclictest2861904-21ntpq00:25:191
275909099171,13cyclictest2833870-21ps23:00:201
275909099171,13cyclictest2821086-21ntpq22:20:191
275909099171,13cyclictest2790414-21df_inode20:45:131
275909099171,13cyclictest2762018-21ntpq19:15:181
275909099171,12cyclictest2792186-21memory20:50:171
275909099160,13cyclictest2769619-21runrttasks19:40:061
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional