You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-07 - 19:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackfslot7s.osadl.org (updated Tue Jul 07, 2026 00:44:18)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
288250099371014,3691cyclictest2904342-21kworker/1:1+events20:35:181
288249599341916,3389cyclictest2969893-21kworker/0:0+events23:50:000
288249599334414,3325cyclictest2884945-21kworker/0:2+events20:00:210
288250099333717,3317cyclictest2980069-21kworker/1:1+events00:25:201
288249599310215,3083cyclictest2920368-21kworker/0:3+events21:15:120
288249599305514,3037cyclictest2924833-21kworker/0:1+events21:30:200
288250099302515,3006cyclictest2969655-21kworker/1:0+events00:20:011
288249599301915,3000cyclictest2961822-21kworker/0:1+events23:35:170
288250099299914,2979cyclictest2907593-21kworker/1:2+events20:45:121
288249599294710,2932cyclictest2933403-21kworker/0:1+events21:55:180
288250099292115,2902cyclictest2980069-21kworker/1:1+events00:30:131
288250099289613,2877cyclictest2961941-21kworker/1:1+events23:20:131
288250899284714,2828cyclictest2971308-21kworker/3:1+events00:00:113
288249599284314,2824cyclictest2957279-21kworker/0:2+events23:15:170
288250099282014,2802cyclictest2933782-21kworker/1:0+events22:00:181
288250899281913,2802cyclictest2929985-21kworker/3:0+events22:05:153
288250899280613,2789cyclictest2946406-21kworker/3:0+events22:42:473
288250899280310,2791cyclictest2897882-21kworker/3:2+events20:15:073
288250899279813,2781cyclictest2929985-21kworker/3:0+events21:45:513
288250899278913,2772cyclictest2915299-21kworker/3:1+events20:52:503
288250499277813,2761cyclictest2945869-21kworker/2:1+events23:37:572
288250499277713,2760cyclictest2973556-21kworker/2:1+events00:04:452
288250099277613,2759cyclictest2912065-21kworker/1:0+events21:14:491
28825009927740,2770cyclictest2896249-21kworker/1:2+events20:04:061
288250499276713,2750cyclictest2904188-21kworker/2:1+events20:17:272
288250499276613,2749cyclictest2927997-21kworker/2:2+events21:37:572
288250499276613,2749cyclictest2889570-21kworker/2:0+events19:33:502
288250899276513,2748cyclictest2902909-21kworker/3:0+events20:32:123
288250899276413,2747cyclictest2952268-21kworker/3:2+events23:22:063
288250499276413,2748cyclictest2986231-21kworker/2:0+events00:39:442
288250899276313,2746cyclictest2973258-21kworker/3:2+events23:59:263
288250899276112,2747cyclictest2915299-21kworker/3:1+events21:00:023
288250499276112,2746cyclictest2955490-21kworker/2:2+events23:03:102
288250499276012,2746cyclictest2973556-21kworker/2:1+events00:13:052
288250499275813,2741cyclictest2907474-21kworker/2:2+events20:28:262
288250899275713,2740cyclictest2888060-21kworker/3:0+events19:58:293
288250899275712,2741cyclictest2909014-21kworker/3:1+events20:37:573
28825009927578,2745cyclictest2960309-21kworker/1:0+events23:15:401
288250499275613,2739cyclictest2912461-21kworker/2:3+events20:46:452
288250899275513,2738cyclictest2921977-21kworker/3:2+events21:19:023
288250499275413,2737cyclictest2926309-21kworker/2:1+events21:28:502
288250099273911,2724cyclictest2920306-21kworker/1:2+events21:25:261
288250499273813,2721cyclictest2945869-21kworker/2:1+events23:43:152
288250499273613,2719cyclictest2945869-21kworker/2:1+events22:51:292
288250099272713,2710cyclictest2926397-21kworker/1:1+events21:47:221
288250099272513,2709cyclictest2930012-21kworker/1:0+events21:41:441
288250099272213,2705cyclictest2920306-21kworker/1:2+events21:18:391
288250099272113,2705cyclictest2920306-21kworker/1:2+events21:21:201
288250099272113,2704cyclictest2891285-21kworker/1:1+events19:49:421
288250099271913,2702cyclictest2891397-21kworker/1:3+events19:41:211
288250899271513,2699cyclictest2882667-21kworker/3:1+events19:25:263
28825049927139,2701cyclictest2945869-21kworker/2:1+events22:49:592
288250099271312,2697cyclictest2969655-21kworker/1:0+events23:49:371
288250499270413,2687cyclictest2912461-21kworker/2:3+events20:51:182
288250099270312,2687cyclictest2969655-21kworker/1:0+events00:08:101
288250899270013,2683cyclictest2882667-21kworker/3:1+events19:18:083
288249599269912,2683cyclictest2968272-21kworker/0:2+events23:52:400
288250099269112,2675cyclictest2888092-21kworker/1:2+events19:32:421
288250499268212,2666cyclictest2934821-21kworker/2:1+events21:53:012
288250499267312,2657cyclictest2945869-21kworker/2:1+events23:18:412
288249599267013,2653cyclictest2884945-21kworker/0:2+events19:37:320
288249599267013,2653cyclictest2884945-21kworker/0:2+events19:37:320
288250899266913,2652cyclictest2950563-21kworker/3:1+events22:47:223
288249599265313,2636cyclictest2924833-21kworker/0:1+events21:23:340
288249599265213,2635cyclictest2891106-21kworker/0:1+events19:40:580
288249599265113,2634cyclictest2904155-21kworker/0:0+events20:34:080
288249599263812,2622cyclictest2910766-21kworker/0:2+events21:04:140
288249599263713,2620cyclictest2965100-21kworker/0:0+events23:31:570
288249599262813,2611cyclictest2931459-21kworker/0:0+events21:43:360
288250899262513,2608cyclictest2888060-21kworker/3:0+events19:52:463
288249599262211,2608cyclictest2924833-21kworker/0:1+events21:27:190
288250899261612,2600cyclictest2924863-21kworker/3:0+events21:32:353
288250899261112,2595cyclictest2889808-21kworker/3:2+events19:40:103
288250499260612,2592cyclictest2918404-21kworker/2:2+events21:06:102
288250499260413,2587cyclictest2899836-21kworker/2:0+events20:14:462
28825009926009,2589cyclictest2958558-21kworker/1:2+events23:25:061
288250099259312,2577cyclictest2933782-21kworker/1:0+events21:51:061
288249599258612,2570cyclictest2904155-21kworker/0:0+events20:24:170
28824959925831,2578cyclictest2954038-21kworker/0:1+events23:03:540
288250099258012,2565cyclictest2882789-21kworker/1:0+events19:35:211
288250099258012,2565cyclictest2882789-21kworker/1:0+events19:35:211
288250499257813,2561cyclictest2945869-21kworker/2:1+events22:31:462
288250499257812,2562cyclictest2945869-21kworker/2:1+events23:34:122
288250499257512,2561cyclictest2916427-21kworker/2:1+events21:00:472
288249599256812,2552cyclictest2944542-21kworker/0:2+events22:29:090
288250499256711,2550cyclictest2947470-21kworker/2:0+events22:36:212
288249599256211,2547cyclictest2928180-21kworker/0:2+events21:36:270
288250899256013,2543cyclictest2941044-21kworker/3:2+events22:12:053
288250499255812,2542cyclictest2914247-21kworker/2:0+events20:55:302
288249599255610,2542cyclictest2979380-21kworker/0:0+events00:20:240
288250899254312,2528cyclictest2965187-21kworker/3:3+events23:41:443
288250499254312,2526cyclictest2891197-21kworker/2:1+events20:05:142
28825089925409,2527cyclictest2965187-21kworker/3:3+events23:31:333
28825049925406,2530cyclictest2981169-21kworker/2:0+events00:24:572
288250499253812,2523cyclictest2921699-21kworker/2:1+events21:16:442
288250899253610,2522cyclictest2902909-21kworker/3:0+events20:20:333
28825049925367,2525cyclictest2945869-21kworker/2:1+events22:27:362
288250499253512,2520cyclictest2941134-21kworker/2:0+events22:20:502
288249599253512,2520cyclictest2936607-21kworker/0:0+events22:16:420
28825049925348,2523cyclictest2886269-21kworker/2:0+events19:25:492
288250499253312,2517cyclictest2875426-21kworker/2:2+events19:15:542
288250099253312,2517cyclictest2938112-21kworker/1:2+events22:10:131
288250899253212,2516cyclictest2899563-21kworker/3:1+events20:12:303
288250899253112,2515cyclictest2929985-21kworker/3:0+events22:03:433
288250499253112,2515cyclictest2921699-21kworker/2:1+events21:13:412
288250499253012,2514cyclictest2976269-21kworker/2:0+events00:05:522
288250499253010,2517cyclictest2941134-21kworker/2:0+events22:14:222
288250499252912,2515cyclictest2949102-21kworker/2:2+events22:40:102
288250499252911,2514cyclictest2920008-21kworker/2:0+events21:20:352
288250499252812,2512cyclictest2902584-21kworker/2:2+events20:20:552
288250899252712,2511cyclictest2915299-21kworker/3:1+events21:09:103
288250899252711,2512cyclictest2915299-21kworker/3:1+events21:26:113
28824959925278,2515cyclictest2918675-21kworker/0:1+events21:05:450
288250899252612,2510cyclictest2984721-21kworker/3:0+events00:32:303
288250499252611,2511cyclictest2927997-21kworker/2:2+events21:46:132
288250899252412,2509cyclictest2948952-21kworker/3:2+events22:39:493
288250899252412,2508cyclictest2910650-21kworker/3:2+events20:49:483
288250099252213,2506cyclictest2899471-21kworker/1:0+events20:06:011
288250099252212,2506cyclictest2949257-21kworker/1:1+events22:50:221
288250899252112,2506cyclictest2952268-21kworker/3:2+events23:16:483
288250899252111,2507cyclictest2917019-21kworker/3:2+events20:59:143
288250099252112,2505cyclictest2971272-21kworker/1:2+events23:50:451
288250499251911,2504cyclictest2945869-21kworker/2:1+events23:05:262
288250899251812,2502cyclictest2889808-21kworker/3:2+events19:37:553
288250899251812,2502cyclictest2889808-21kworker/3:2+events19:37:543
288250099251812,2502cyclictest2946021-21kworker/1:0+events22:34:251
288250899251612,2500cyclictest2942892-21kworker/3:0+events22:21:363
288250899251511,2500cyclictest2920244-21kworker/3:0+events21:10:193
288250899251412,2498cyclictest2928090-21kworker/3:1+events21:43:563
28825049925145,2506cyclictest2927997-21kworker/2:2+events21:42:082
288249599251013,2493cyclictest2973107-21kworker/0:1+events23:57:360
288250899250712,2491cyclictest2928090-21kworker/3:1+events21:39:043
288250099250611,2491cyclictest2949257-21kworker/1:1+events22:58:351
288250899250412,2488cyclictest2982850-21kworker/3:2+events00:35:123
288250499250412,2488cyclictest2934821-21kworker/2:1+events22:07:092
288250899250112,2486cyclictest2971308-21kworker/3:1+events00:11:343
288250499250012,2484cyclictest2945869-21kworker/2:1+events23:22:492
288250099249812,2482cyclictest2944392-21kworker/1:1+events22:27:141
28825049924979,2485cyclictest2891197-21kworker/2:1+events19:58:042
288250499249710,2483cyclictest2907474-21kworker/2:2+events20:37:132
288250099249712,2481cyclictest2912065-21kworker/1:0+events21:01:091
288250099249612,2481cyclictest2904342-21kworker/1:1+events20:31:501
288250899249411,2480cyclictest2902909-21kworker/3:0+events20:26:553
288250099249412,2478cyclictest2946021-21kworker/1:0+events22:39:251
288250099249410,2480cyclictest2958558-21kworker/1:2+events23:14:101
288250099249312,2477cyclictest2912065-21kworker/1:0+events20:43:421
288250899249112,2475cyclictest2924863-21kworker/3:0+events21:23:113
28825009924919,2479cyclictest2958558-21kworker/1:2+events23:07:411
288250099248912,2473cyclictest2891285-21kworker/1:1+events19:50:491
288250099248812,2474cyclictest2913698-21kworker/1:1+events20:55:071
288250099248812,2472cyclictest2896249-21kworker/1:2+events20:15:291
288250099248712,2472cyclictest2938112-21kworker/1:2+events22:17:051
288250099248711,2473cyclictest2969655-21kworker/1:0+events23:42:531
288250899248611,2472cyclictest2971308-21kworker/3:1+events00:21:113
288250099248612,2470cyclictest2974796-21kworker/1:2+events23:59:481
288250099248612,2470cyclictest2949257-21kworker/1:1+events22:46:591
28825009924860,2482cyclictest2938112-21kworker/1:2+events22:24:181
288249599248612,2470cyclictest2884945-21kworker/0:2+events19:49:180
288250099248512,2471cyclictest2966374-21kworker/1:1+events23:36:271
288250099248512,2471cyclictest2882789-21kworker/1:0+events19:20:031
288250899248411,2470cyclictest2910650-21kworker/3:2+events20:44:503
288250099248412,2469cyclictest2902734-21kworker/1:0+events20:23:091
288250899248312,2469cyclictest2955606-21kworker/3:1+events23:00:283
288250099248310,2469cyclictest2904342-21kworker/1:1+events20:29:121
288250099248212,2467cyclictest2980069-21kworker/1:1+events00:20:471
288250899247911,2465cyclictest2884829-21kworker/3:2+events19:21:353
288250499247712,2461cyclictest2955490-21kworker/2:2+events22:59:212
288250499247612,2460cyclictest2979558-21kworker/2:2+events00:25:412
288250099247612,2460cyclictest2913698-21kworker/1:1+events20:53:351
288250499247312,2457cyclictest2926309-21kworker/2:1+events21:30:422
28825009924726,2462cyclictest2963560-21kworker/1:0+events23:33:491
288250499247112,2456cyclictest2889570-21kworker/2:0+events19:38:402
288250499247112,2456cyclictest2889570-21kworker/2:0+events19:38:402
288250099247110,2457cyclictest2882789-21kworker/1:0+events19:29:181
288250099247012,2454cyclictest2949257-21kworker/1:1+events23:01:151
288250899246812,2452cyclictest2952268-21kworker/3:2+events22:59:433
288250499246811,2453cyclictest2971174-21kworker/2:2+events23:55:192
288250099246410,2450cyclictest2926397-21kworker/1:1+events21:37:351
288250499246312,2448cyclictest2934821-21kworker/2:1+events21:58:462
288250499246312,2447cyclictest2910797-21kworker/2:0+events20:44:272
288250899246111,2448cyclictest2971308-21kworker/3:1+events00:26:043
288250899246111,2446cyclictest2934852-21kworker/3:1+events22:27:593
288249599245912,2444cyclictest2899620-21kworker/0:1+events20:15:530
288250499245512,2441cyclictest2891197-21kworker/2:1+events19:45:072
288249599245411,2439cyclictest2952418-21kworker/0:2+events22:57:500
288250499245112,2436cyclictest2971174-21kworker/2:2+events23:49:132
288249599245112,2435cyclictest2928307-21kworker/0:3+events21:51:290
288250899244912,2433cyclictest2882667-21kworker/3:1+events19:12:513
288250899244911,2434cyclictest2934852-21kworker/3:1+events22:30:143
288250499244810,2434cyclictest2945869-21kworker/2:1+events23:25:522
288250899244711,2433cyclictest2971308-21kworker/3:1+events23:50:243
288250499244712,2431cyclictest2945869-21kworker/2:1+events23:13:472
288250499244611,2431cyclictest2907474-21kworker/2:2+events20:32:572
288250099244612,2431cyclictest2946021-21kworker/1:0+events22:44:401
288250499244512,2430cyclictest2969594-21kworker/2:0+events23:51:072
288250499244510,2432cyclictest2934821-21kworker/2:1+events22:15:552
288249599244512,2429cyclictest2884945-21kworker/0:2+events19:28:310
288250499244412,2429cyclictest2981169-21kworker/2:0+events00:19:392
288250499244412,2428cyclictest2891197-21kworker/2:1+events19:51:372
288250899244311,2430cyclictest2971308-21kworker/3:1+events00:15:013
288249599244311,2429cyclictest2910766-21kworker/0:2+events20:43:180
288250899244212,2427cyclictest2958734-21kworker/3:0+events23:13:013
288250899243710,2424cyclictest2952268-21kworker/3:2+events23:25:293
288250499243611,2422cyclictest2886269-21kworker/2:0+events19:22:212
288249599243612,2422cyclictest2974737-21kworker/0:0+events00:00:350
288250899243512,2420cyclictest2899563-21kworker/3:1+events20:09:493
288250899243511,2421cyclictest2929985-21kworker/3:0+events21:52:163
288249599243513,2419cyclictest2982944-21kworker/0:1+events00:28:450
288250899243211,2417cyclictest2899563-21kworker/3:1+events20:04:283
288249599243112,2415cyclictest2904155-21kworker/0:0+events20:28:040
288250099243012,2414cyclictest2933782-21kworker/1:0+events21:57:371
288249599243012,2414cyclictest2896374-21kworker/0:1+events19:53:090
288250899242810,2415cyclictest2977956-21kworker/3:2+events00:08:333
288249599242712,2411cyclictest2954038-21kworker/0:1+events23:05:490
288249599242612,2410cyclictest2910766-21kworker/0:2+events20:54:430
288250899242512,2410cyclictest2952268-21kworker/3:2+events22:52:583
288249599242512,2409cyclictest2936607-21kworker/0:0+events22:21:130
288249599242512,2409cyclictest2904155-21kworker/0:0+events20:35:400
288249599242512,2409cyclictest2884945-21kworker/0:2+events19:23:300
28825089924248,2412cyclictest2934852-21kworker/3:1+events22:18:593
28824959924234,2415cyclictest2884945-21kworker/0:2+events19:31:360
288249599242312,2408cyclictest2944542-21kworker/0:2+events22:40:540
288249599242211,2408cyclictest2896374-21kworker/0:1+events19:55:260
288249599242112,2405cyclictest2928307-21kworker/0:3+events22:01:270
288249599242111,2407cyclictest2944542-21kworker/0:2+events22:34:010
288249599242111,2406cyclictest2984539-21kworker/0:0+events00:38:110
28825009924209,2408cyclictest2982817-21kworker/1:2+events00:37:061
288249599242011,2405cyclictest2916987-21kworker/0:0+events20:59:390
288249599241412,2399cyclictest2910766-21kworker/0:2+events21:10:430
288250099241212,2397cyclictest2884796-21kworker/1:2+events19:18:331
288250099241210,2399cyclictest2938112-21kworker/1:2+events22:06:241
288249599241112,2395cyclictest2910766-21kworker/0:2+events20:46:000
288250099240912,2393cyclictest2974796-21kworker/1:2+events00:03:141
28824959924097,2399cyclictest2882728-21kworker/0:1+events19:16:390
288249599240912,2393cyclictest2936607-21kworker/0:0+events22:10:580
288250099240610,2393cyclictest2901097-21kworker/1:1+events20:12:071
288250899240112,2385cyclictest2882667-21kworker/3:1+events19:32:203
288250099240112,2386cyclictest2896249-21kworker/1:2+events19:59:341
288249599240112,2386cyclictest2897975-21kworker/0:0+events20:07:540
288250099240012,2384cyclictest2918571-21kworker/1:1+events21:06:331
288250099239612,2382cyclictest2969655-21kworker/1:0+events00:10:041
28824959923929,2379cyclictest2961822-21kworker/0:1+events23:23:570
288249599238612,2371cyclictest2982944-21kworker/0:1+events00:32:090
288249599238011,2365cyclictest2944542-21kworker/0:2+events22:36:430
288249599236512,2349cyclictest2977416-21kworker/0:1+events00:06:140
288249599236211,2349cyclictest2950668-21kworker/0:0+events22:45:040
288249599236012,2344cyclictest2952418-21kworker/0:2+events22:53:190
288249599234711,2333cyclictest2963470-21kworker/0:2+events23:28:540
288249599234313,2326cyclictest2936607-21kworker/0:0+events22:07:550
288249599234112,2325cyclictest2968272-21kworker/0:2+events00:17:210
288249599234112,2325cyclictest2931459-21kworker/0:0+events21:46:360
288249599233911,2324cyclictest2968272-21kworker/0:2+events00:10:490
288249599233712,2321cyclictest2960094-21kworker/0:0+events23:14:330
288250899227611,2262cyclictest2968398-21kworker/3:0+events23:48:053
288250899224612,2231cyclictest2952268-21kworker/3:2+events23:09:113
288250499224311,2228cyclictest2869589-21kworker/2:0+events19:12:052
288250099223611,2222cyclictest2882052-21kworker/1:1+events19:14:231
288250499222111,2207cyclictest2934821-21kworker/2:1+events22:04:302
288250499220011,2186cyclictest2984393-21kworker/2:1+events00:34:032
28824959921568,2144cyclictest2882728-21kworker/0:1+events19:13:150
288249599212911,2115cyclictest2901252-21kworker/0:2+events20:13:380
288250899211311,2099cyclictest2892968-21kworker/3:1+events19:46:173
288250099209811,2084cyclictest2926397-21kworker/1:1+events21:33:441
28822642143109,29sleep30-21swapper/319:09:453
28821252143111,27sleep20-21swapper/219:07:552
28821162142112,23sleep10-21swapper/119:07:511
28819872142110,27sleep00-21swapper/019:06:130
288249599130126,2cyclictest2968272-21kworker/0:2+pm23:45:000
2882504994238,3cyclictest2891197-21kworker/2:1+pm19:44:572
288250499201,15cyclictest2899498-21taskset20:01:392
288250899166,6cyclictest558333-21systemd-journal23:36:363
288250899150,4cyclictest2935335-21/usr/sbin/munin21:55:183
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional