You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-01 - 05:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackfslot7s.osadl.org (updated Mon Jun 01, 2026 00:44:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
257259299326116,3241cyclictest2597249-21kworker/0:1+events20:35:100
257259299317215,3152cyclictest2655566-21kworker/0:2+events23:35:000
257259499315113,3133cyclictest2637570-21kworker/1:0+events22:35:011
257260099307615,3059cyclictest2627693-21kworker/3:0+events22:10:013
257259299298413,2966cyclictest2577405-21kworker/0:0+events19:30:120
257259499295312,2934cyclictest2595521-21kworker/1:1+events20:20:141
257260099289614,2879cyclictest2587303-21kworker/3:0+events20:00:003
257259499289213,2875cyclictest2576112-21kworker/1:0+events19:25:161
257259299282912,2810cyclictest2595490-21kworker/0:0+events20:25:160
257260099279911,2784cyclictest2590643-21kworker/3:2+events20:43:083
257259299279411,2778cyclictest2622850-21kworker/0:1+events21:58:310
257259299278313,2766cyclictest2642198-21kworker/0:1+events22:45:170
257260099276914,2752cyclictest2615103-21kworker/3:0+events21:35:153
257259299276813,2751cyclictest2603199-21kworker/0:1+events20:47:450
257259299276311,2748cyclictest2671679-21kworker/0:2+events00:22:050
257259299276013,2743cyclictest2663634-21kworker/0:0+events23:52:580
257259299275813,2741cyclictest2639330-21kworker/0:0+events22:51:230
257260099275013,2733cyclictest2639774-21kworker/3:2+events22:40:183
257259299272713,2710cyclictest2590584-21kworker/0:2+events20:07:390
25725989927110,2707cyclictest2650595-21kworker/2:0+events23:16:322
257259899271013,2693cyclictest2656826-21kworker/2:0+events23:45:202
257259899271013,2693cyclictest2656826-21kworker/2:0+events23:45:192
257259499270712,2692cyclictest2642436-21kworker/1:1+events22:59:251
257259499269813,2681cyclictest2587334-21kworker/1:2+events19:51:421
257259499269113,2675cyclictest2637570-21kworker/1:0+events22:43:431
257259299268513,2670cyclictest2639330-21kworker/0:0+events22:36:100
257259899267912,2662cyclictest2583216-21kworker/2:0+events20:00:212
257260099267312,2657cyclictest2637750-21kworker/3:0+events22:30:233
257260099267311,2658cyclictest2655434-21kworker/3:1+events23:37:433
257260099265312,2637cyclictest2642258-21kworker/3:1+events22:53:173
257259899265213,2635cyclictest2593767-21kworker/2:2+events20:15:172
257259499264712,2631cyclictest2590734-21kworker/1:2+events20:12:591
25725989926351,2630cyclictest2609595-21kworker/2:1+events21:12:242
257259899263113,2614cyclictest2607976-21kworker/2:0+events21:37:092
257259899262713,2610cyclictest2589542-21kworker/2:1+events20:26:462
257259499262711,2613cyclictest2642436-21kworker/1:1+events23:03:591
257259299262012,2606cyclictest2622850-21kworker/0:1+events21:45:060
257260099260313,2588cyclictest2619357-21kworker/3:2+events22:15:203
257259499260310,2589cyclictest2661580-21kworker/1:2+events23:57:591
257259299260012,2584cyclictest2586031-21kworker/0:1+events19:52:040
257260099259513,2578cyclictest2664882-21kworker/3:0+events23:55:183
257259499257912,2563cyclictest2634331-21kworker/1:1+events22:20:371
257259899257312,2557cyclictest2632700-21kworker/2:2+events22:49:292
257259899256912,2553cyclictest2632700-21kworker/2:2+events22:53:422
257260099255910,2545cyclictest2619357-21kworker/3:2+events22:25:493
257259499255312,2538cyclictest2576112-21kworker/1:0+events19:24:291
257259899254912,2534cyclictest2626147-21kworker/2:0+events21:55:522
257259899254813,2532cyclictest2652252-21kworker/2:1+events23:24:152
257260099254611,2532cyclictest2590643-21kworker/3:2+events20:46:573
257259299254312,2527cyclictest2588445-21kworker/0:0+events19:57:450
257259299254212,2526cyclictest2671679-21kworker/0:2+events00:18:130
25725929925419,2528cyclictest2603199-21kworker/0:1+events20:52:200
257259299253912,2524cyclictest2599969-21kworker/0:2+events20:33:360
25726009925388,2526cyclictest2645558-21kworker/3:0+events22:59:033
257259299253710,2523cyclictest2637723-21kworker/0:2+events22:34:360
257259299253511,2520cyclictest2614491-21kworker/0:0+events21:31:030
257259299253511,2520cyclictest2614491-21kworker/0:0+events21:31:020
257259299253312,2517cyclictest2590584-21kworker/0:2+events20:03:010
257259299253212,2516cyclictest2642198-21kworker/0:1+events22:41:050
257259499252812,2512cyclictest2659923-21kworker/1:1+events23:40:481
25725929925288,2516cyclictest2639330-21kworker/0:0+events23:12:440
257259299252812,2512cyclictest2608125-21kworker/0:0+events21:08:550
25726009925278,2515cyclictest2655434-21kworker/3:1+events23:25:253
257259299252712,2511cyclictest2579013-21kworker/0:2+events19:49:250
257259299252612,2510cyclictest2650042-21kworker/0:1+events23:06:150
257259299252511,2510cyclictest2639330-21kworker/0:0+events23:02:020
257259299252412,2508cyclictest2659985-21kworker/0:1+events00:08:390
257259299252012,2504cyclictest2608125-21kworker/0:0+events20:59:280
25725949925195,2510cyclictest2577854-21kworker/1:2+events19:39:591
257259299251812,2502cyclictest2666245-21kworker/0:2+events23:58:450
257259299251712,2502cyclictest2622850-21kworker/0:1+events22:04:200
257259299251511,2500cyclictest2622850-21kworker/0:1+events21:52:250
257260099251213,2497cyclictest2664882-21kworker/3:0+events00:10:113
257259499251113,2494cyclictest2619570-21kworker/1:1+events21:37:551
257259299250912,2493cyclictest2608125-21kworker/0:0+events21:04:430
25725929925069,2493cyclictest2659985-21kworker/0:1+events00:10:340
25725949925053,2499cyclictest2632727-21kworker/1:0+events22:18:441
257259499250511,2492cyclictest2597214-21kworker/1:2+events20:40:061
257259299250212,2486cyclictest2673315-21kworker/0:1+events00:26:590
25725929925018,2489cyclictest2661704-21kworker/0:2+events23:47:380
25725929925018,2489cyclictest2661704-21kworker/0:2+events23:47:380
257260099250012,2485cyclictest2663273-21kworker/3:1+events23:54:073
257259499249713,2480cyclictest2653529-21kworker/1:2+events23:34:171
257259299249612,2480cyclictest2621162-21kworker/0:2+events21:42:510
257259899249511,2480cyclictest2596909-21kworker/2:0+events20:38:572
257260099249412,2478cyclictest2639774-21kworker/3:2+events22:36:333
257259299249312,2477cyclictest2658359-21kworker/0:0+events23:41:530
25725949924847,2473cyclictest2653529-21kworker/1:2+events23:18:281
25725929924848,2473cyclictest2630654-21kworker/0:2+events22:17:370
257259299248312,2469cyclictest2666245-21kworker/0:2+events00:01:030
257260099248113,2464cyclictest2663273-21kworker/3:1+events00:09:023
257259499248112,2465cyclictest2616238-21kworker/1:1+events21:24:111
25725989924790,2475cyclictest2601812-21kworker/2:1+events20:44:172
25725949924797,2467cyclictest2678012-21kworker/1:2+events00:37:231
257259499247911,2465cyclictest2582897-21kworker/1:1+events19:45:201
25725929924779,2464cyclictest2579013-21kworker/0:2+events19:44:110
257259899247610,2463cyclictest2676970-21kworker/2:2+events00:37:462
257259499247612,2460cyclictest2653529-21kworker/1:2+events23:22:221
257260099247411,2459cyclictest2578888-21kworker/3:0+events19:38:513
25725989924746,2464cyclictest2574679-21kworker/2:1+events19:23:202
257259499247412,2458cyclictest2606616-21kworker/1:1+events20:58:011
257260099247313,2456cyclictest2584313-21kworker/3:2+events19:43:483
257259499247312,2457cyclictest2661580-21kworker/1:2+events23:52:341
257259499247212,2456cyclictest2604985-21kworker/1:2+events21:01:201
257259499247212,2456cyclictest2588960-21kworker/1:0+events19:58:071
257259499247111,2456cyclictest2582897-21kworker/1:1+events19:41:541
257259899246912,2454cyclictest2583216-21kworker/2:0+events20:10:222
257259499246911,2454cyclictest2655164-21kworker/1:0+events23:47:591
257259499246911,2454cyclictest2655164-21kworker/1:0+events23:47:581
257259499246812,2452cyclictest2670075-21kworker/1:1+events00:12:291
257259499246812,2452cyclictest2661580-21kworker/1:2+events00:25:491
257259499246811,2450cyclictest2614612-21kworker/1:0+events21:18:061
257259499246711,2452cyclictest2642436-21kworker/1:1+events22:49:071
257259299246611,2451cyclictest2653716-21kworker/0:1+events23:28:040
257259499246511,2450cyclictest2670075-21kworker/1:1+events00:18:371
257259499246312,2447cyclictest2609831-21kworker/1:0+events21:07:001
257260099246212,2447cyclictest2655434-21kworker/3:1+events23:31:553
257259499246012,2445cyclictest2617930-21kworker/1:2+events21:32:341
257259499246012,2445cyclictest2617930-21kworker/1:2+events21:32:341
257259299246012,2444cyclictest2630654-21kworker/0:2+events22:10:470
257260099245912,2443cyclictest2619357-21kworker/3:2+events22:23:373
257260099245810,2444cyclictest2606469-21kworker/3:1+events20:55:213
257259499245611,2441cyclictest2601604-21kworker/1:0+events20:46:121
257259299245211,2437cyclictest2622850-21kworker/0:1+events22:08:070
257260099245012,2434cyclictest2647515-21kworker/3:2+events23:16:553
257259299244812,2431cyclictest2573730-21kworker/0:2+events19:14:350
257259499244712,2431cyclictest2597214-21kworker/1:2+events20:34:241
257259499244710,2433cyclictest2617930-21kworker/1:2+events21:45:551
257260099244612,2430cyclictest2658379-21kworker/3:2+events23:48:433
257260099244612,2430cyclictest2658379-21kworker/3:2+events23:48:433
257259499244612,2431cyclictest2597214-21kworker/1:2+events20:27:311
257259299244612,2430cyclictest2658359-21kworker/0:0+events23:37:190
257259299244612,2430cyclictest2614491-21kworker/0:0+events21:21:530
257259299244513,2428cyclictest2645683-21kworker/0:1+events22:55:350
25726009924448,2432cyclictest2650386-21kworker/3:0+events23:10:483
257260099244411,2429cyclictest2676691-21kworker/3:2+events00:37:013
257259899244413,2427cyclictest2658589-21kworker/2:2+events23:35:242
257259299244410,2430cyclictest2675237-21kworker/0:0+events00:38:100
257260099244312,2428cyclictest2655434-21kworker/3:1+events23:41:323
257259299244312,2427cyclictest2609800-21kworker/0:1+events21:14:200
257259299244311,2428cyclictest2609800-21kworker/0:1+events21:17:430
257259499244212,2427cyclictest2627604-21kworker/1:1+events22:13:271
257259499244211,2428cyclictest2637570-21kworker/1:0+events22:52:561
257259499244211,2428cyclictest2590734-21kworker/1:2+events20:04:571
257259299244112,2425cyclictest2599969-21kworker/0:2+events20:41:150
25726009924408,2429cyclictest2573673-21kworker/3:2+events19:26:263
257259299244012,2424cyclictest2590584-21kworker/0:2+events20:22:590
257259299243812,2423cyclictest2639330-21kworker/0:0+events23:20:250
257259299243810,2425cyclictest2634153-21kworker/0:1+events22:29:160
257259299243712,2421cyclictest2653716-21kworker/0:1+events23:19:390
257259299243711,2423cyclictest2621162-21kworker/0:2+events21:37:330
25726009924369,2423cyclictest2609921-21kworker/3:2+events21:06:153
25726009924368,2424cyclictest2590643-21kworker/3:2+events20:06:283
257260099243612,2422cyclictest2619357-21kworker/3:2+events22:05:263
257260099243512,2419cyclictest2571850-21kworker/3:1+events19:20:393
25726009924349,2422cyclictest2619357-21kworker/3:2+events21:46:183
257259499243412,2418cyclictest2627604-21kworker/1:1+events22:09:141
257259299243412,2418cyclictest2580954-21kworker/0:1+events19:36:590
257259299243411,2420cyclictest2590584-21kworker/0:2+events20:16:250
257260099243212,2416cyclictest2592283-21kworker/3:1+events20:12:393
257259299243110,2417cyclictest2573730-21kworker/0:2+events19:21:460
257260099243012,2415cyclictest2612228-21kworker/3:0+events21:11:153
257259299242812,2413cyclictest2675237-21kworker/0:0+events00:33:320
257259899242712,2411cyclictest2603438-21kworker/2:2+events20:49:162
257260099242612,2411cyclictest2584313-21kworker/3:2+events19:54:413
257260099242512,2409cyclictest2590643-21kworker/3:2+events20:54:133
257259499242511,2411cyclictest2617930-21kworker/1:2+events21:58:071
257259899242112,2405cyclictest2572313-21kworker/2:0+events19:15:212
257260099241412,2398cyclictest2642258-21kworker/3:1+events23:03:363
257259899241412,2398cyclictest2630836-21kworker/2:1+events22:16:512
257260099241212,2396cyclictest2627693-21kworker/3:0+events22:00:523
25725989924119,2398cyclictest2596909-21kworker/2:0+events20:24:312
25725989924109,2397cyclictest2583216-21kworker/2:0+events20:05:202
257260099240811,2393cyclictest2663273-21kworker/3:1+events00:26:123
257259499240812,2392cyclictest2577854-21kworker/1:2+events19:31:411
25726009924059,2393cyclictest2647515-21kworker/3:2+events23:05:533
25726009924039,2390cyclictest2608098-21kworker/3:0+events21:00:373
25725989924032,2397cyclictest2607976-21kworker/2:0+events21:16:572
257259899240312,2387cyclictest2622668-21kworker/2:2+events21:44:442
25726009924029,2389cyclictest2598723-21kworker/3:0+events20:35:323
25725989924019,2388cyclictest2664639-21kworker/2:2+events00:08:162
257259499240012,2384cyclictest2627604-21kworker/1:1+events22:01:161
257259899239912,2384cyclictest2632700-21kworker/2:2+events22:35:472
257259299239912,2384cyclictest2592220-21kworker/0:0+events20:11:300
257259899239812,2383cyclictest2656826-21kworker/2:0+events23:51:022
257259899239810,2384cyclictest2656826-21kworker/2:0+events23:28:502
257259899239611,2382cyclictest2630836-21kworker/2:1+events22:24:212
257260099239312,2377cyclictest2619357-21kworker/3:2+events21:51:163
25725989923938,2382cyclictest2675205-21kworker/2:1+events00:30:012
257259499239212,2378cyclictest2649113-21kworker/1:0+events23:06:591
257259499239012,2375cyclictest2616238-21kworker/1:1+events21:27:361
257259499239012,2374cyclictest2601604-21kworker/1:0+events20:37:251
257260099238912,2373cyclictest2617717-21kworker/3:1+events21:28:213
257260099238911,2375cyclictest2615103-21kworker/3:0+events21:32:123
257260099238911,2375cyclictest2615103-21kworker/3:0+events21:32:123
257259899238812,2372cyclictest2616208-21kworker/2:1+events21:32:592
257259899238812,2372cyclictest2616208-21kworker/2:1+events21:32:582
257259499238812,2372cyclictest2637570-21kworker/1:0+events22:28:071
257259499238812,2372cyclictest2574001-21kworker/1:2+events19:16:291
257259499238811,2373cyclictest2637570-21kworker/1:0+events22:38:011
257260099238712,2371cyclictest2590643-21kworker/3:2+events20:34:003
257259499238712,2371cyclictest2673464-21kworker/1:0+events00:31:591
257259499238711,2374cyclictest2649113-21kworker/1:0+events23:10:261
257260099238511,2370cyclictest2670198-21kworker/3:2+events00:20:553
257259499238512,2369cyclictest2653529-21kworker/1:2+events23:26:101
257259499238410,2371cyclictest2655164-21kworker/1:0+events23:39:391
257260099238312,2367cyclictest2670198-21kworker/3:2+events00:15:543
257260099238312,2367cyclictest2573673-21kworker/3:2+events19:30:363
257259499238312,2368cyclictest2617930-21kworker/1:2+events21:52:021
257259499238312,2368cyclictest2590734-21kworker/1:2+events20:09:361
257259499238311,2368cyclictest2621495-21kworker/1:0+events21:42:061
257259899238212,2366cyclictest2583216-21kworker/2:0+events19:41:092
257259499237511,2361cyclictest2604985-21kworker/1:2+events20:51:101
257260099237311,2358cyclictest2675083-21kworker/3:0+events00:30:483
257259899237312,2357cyclictest2632700-21kworker/2:2+events22:25:272
257259899237011,2355cyclictest2607976-21kworker/2:0+events21:02:062
257259899236912,2353cyclictest2574679-21kworker/2:1+events19:36:372
257260099236712,2351cyclictest2642258-21kworker/3:1+events22:46:503
25725989923658,2354cyclictest2626147-21kworker/2:0+events22:01:392
257259899236010,2347cyclictest2607976-21kworker/2:0+events21:24:572
257259499235711,2344cyclictest2606616-21kworker/1:1+events21:10:511
257260099235012,2334cyclictest2590643-21kworker/3:2+events20:29:493
257260099234912,2333cyclictest2626088-21kworker/3:1+events21:57:013
257260099234812,2332cyclictest2580837-21kworker/3:1+events19:47:583
257260099234811,2334cyclictest2571850-21kworker/3:1+events19:18:413
257260099234810,2336cyclictest2619357-21kworker/3:2+events21:40:093
257260099234712,2333cyclictest2593884-21kworker/3:0+events20:18:213
257259899234712,2331cyclictest2583216-21kworker/2:0+events19:49:482
257259899234312,2327cyclictest2648783-21kworker/2:2+events23:11:112
257260099233912,2323cyclictest2609921-21kworker/3:2+events21:20:453
257259899233711,2323cyclictest2668496-21kworker/2:1+events00:17:032
257259899233612,2320cyclictest2640621-21kworker/2:0+events22:44:312
257259899233512,2320cyclictest2656826-21kworker/2:0+events00:01:252
257259899233512,2320cyclictest2607976-21kworker/2:0+events21:28:442
257259899233512,2319cyclictest2574679-21kworker/2:1+events19:26:472
257259899233312,2318cyclictest2596909-21kworker/2:0+events20:32:512
25725989923286,2319cyclictest2640621-21kworker/2:0+events22:58:412
257259899232811,2313cyclictest2668496-21kworker/2:1+events00:12:052
257259899232811,2313cyclictest2645943-21kworker/2:1+events23:00:332
257259899232412,2308cyclictest2656826-21kworker/2:0+events23:40:242
257259899232312,2308cyclictest2673344-21kworker/2:2+events00:26:352
257259899231412,2299cyclictest2629084-21kworker/2:2+events22:05:502
257259899231012,2294cyclictest2603438-21kworker/2:2+events20:51:322
257259899230812,2293cyclictest2586061-21kworker/2:1+events19:54:172
25725989923068,2294cyclictest2639028-21kworker/2:1+events22:31:332
257259899230611,2291cyclictest2670133-21kworker/2:0+events00:22:282
257260099229710,2283cyclictest2647515-21kworker/3:2+events23:23:303
257259499229611,2282cyclictest2590734-21kworker/1:2+events20:17:351
25725989922740,2271cyclictest2650595-21kworker/2:0+events23:09:402
257260099227212,2259cyclictest2590643-21kworker/3:2+events20:04:103
257259299226611,2252cyclictest2630654-21kworker/0:2+events22:21:000
257259299226111,2247cyclictest2575748-21kworker/0:1+events19:25:400
257260099225712,2242cyclictest2609921-21kworker/3:2+events21:15:263
257259499225410,2240cyclictest2666370-21kworker/1:1+events00:00:171
257259299223311,2218cyclictest2573730-21kworker/0:2+events19:16:520
257259499218311,2169cyclictest2661580-21kworker/1:2+events00:23:581
257260099216711,2153cyclictest2590643-21kworker/3:2+events20:24:533
257259499216211,2147cyclictest2570716-21kworker/1:0+events19:12:151
257259899216111,2147cyclictest2616208-21kworker/2:1+events21:54:432
25725949921609,2149cyclictest2668022-21kworker/1:0+events00:05:091
25726009921549,2142cyclictest2663273-21kworker/3:1+events00:00:403
257259899211411,2100cyclictest2656826-21kworker/2:0+events23:30:472
257259899210810,2094cyclictest2558002-21kworker/2:2+events19:13:232
257259899210511,2091cyclictest2630836-21kworker/2:1+events22:14:132
257259899208511,2071cyclictest2603438-21kworker/2:2+events20:56:522
257260099200610,1992cyclictest2573673-21kworker/3:2+events19:14:103
25723442151107,24sleep10-21swapper/119:09:471
25722312148113,30sleep00-21swapper/019:08:230
25723412142112,25sleep30-21swapper/319:09:453
25722282142109,28sleep20-21swapper/219:08:202
2572598992315,3cyclictest191rcu_preempt21:10:012
257259899195,12cyclictest191rcu_preempt19:55:142
2572598991712,4cyclictest2665850-21/usr/sbin/munin23:55:202
2572598991710,2cyclictest191rcu_preempt19:30:162
257259299170,15cyclictest2616688-21/usr/sbin/munin21:25:120
2572598991611,1cyclictest191rcu_preempt21:45:162
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional