You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-20 - 14:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackfslot7s.osadl.org (updated Sat Jun 20, 2026 00:44:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
320651199423422,4206cyclictest3249814-21kworker/1:1+events21:25:181
320651199369717,3669cyclictest3236947-21kworker/1:2+events20:55:131
320651499358716,3566cyclictest3265891-21kworker/2:2+events22:15:112
320651199340415,3383cyclictest3260847-21kworker/1:1+events22:05:151
320651499329310,3279cyclictest3268740-21kworker/2:1+events22:35:102
32065179931814,3171cyclictest3279877-21kworker/3:2+events23:10:163
320651199303714,3016cyclictest3253112-21kworker/1:1+events21:35:131
32065149930356,3024cyclictest3251265-21kworker/2:2+events21:30:172
320651499300415,2987cyclictest3222532-21kworker/2:0+events20:00:142
320651799299814,2979cyclictest3276674-21kworker/3:1+events22:53:523
320651799297414,2956cyclictest3290687-21kworker/3:2+events00:16:223
320651799295212,2936cyclictest3210027-21kworker/3:1+events19:25:123
320651199294615,2928cyclictest3296556-21kworker/1:1+events23:51:471
320651499294111,2926cyclictest3288456-21kworker/2:1+events23:35:172
320651199286413,2847cyclictest3275193-21kworker/1:0+events22:58:031
320651199286314,2846cyclictest3273568-21kworker/1:1+events22:46:371
320651799286213,2845cyclictest3264486-21kworker/3:2+events22:13:173
320651499285613,2839cyclictest3216466-21kworker/2:2+events20:16:172
320651199284613,2829cyclictest3265785-21kworker/1:0+events22:34:051
320651499284513,2828cyclictest3235416-21kworker/2:0+events20:42:512
320651499284010,2826cyclictest3284898-21kworker/2:1+events23:16:002
320651199283613,2819cyclictest3253112-21kworker/1:1+events21:44:391
320651499283513,2818cyclictest3244904-21kworker/2:1+events21:49:392
320651799283313,2816cyclictest3210027-21kworker/3:1+events19:30:133
320651499283313,2816cyclictest3216466-21kworker/2:2+events19:46:142
320651499283313,2816cyclictest3216466-21kworker/2:2+events19:46:142
320651199283211,2817cyclictest3228898-21kworker/1:0+events20:28:461
320651799283014,2812cyclictest3268827-21kworker/3:2+events22:25:213
320651799282813,2811cyclictest3253210-21kworker/3:2+events21:54:173
320651799282713,2811cyclictest3210027-21kworker/3:1+events19:20:203
320651499282413,2807cyclictest3268740-21kworker/2:1+events22:31:252
320651499282413,2807cyclictest3259276-21kworker/2:2+events21:53:102
320651499282411,2809cyclictest3264125-21kworker/2:0+events22:26:282
320651499282312,2807cyclictest3305842-21kworker/2:2+events00:37:232
320651499282213,2805cyclictest3278218-21kworker/2:1+events22:59:572
320651499282212,2806cyclictest3216466-21kworker/2:2+events19:52:422
320651199282213,2805cyclictest3255007-21kworker/1:2+events21:50:261
320651499282013,2803cyclictest3244904-21kworker/2:1+events21:15:232
320651499281913,2801cyclictest3275316-21kworker/2:0+events23:05:402
320651499281813,2801cyclictest3262493-21kworker/2:2+events22:06:012
320651799281713,2800cyclictest3233970-21kworker/3:1+events20:38:153
320651499281713,2800cyclictest3243273-21kworker/2:0+events21:08:572
320651499281712,2801cyclictest3232151-21kworker/2:0+events20:32:112
320651499281613,2797cyclictest3281573-21kworker/2:2+events23:03:482
32065119928139,2800cyclictest3275193-21kworker/1:0+events22:50:251
320651799281213,2795cyclictest3235568-21kworker/3:3+events20:59:493
320651799281213,2795cyclictest3219692-21kworker/3:2+events19:51:573
320651199281113,2794cyclictest3236947-21kworker/1:2+events20:48:271
320651799280811,2793cyclictest3225786-21kworker/3:1+events20:11:413
320651499280813,2791cyclictest3224150-21kworker/2:1+events20:05:332
320651199280813,2791cyclictest3230522-21kworker/1:1+events20:33:431
320651799280713,2790cyclictest3227447-21kworker/3:0+events20:15:533
320651799280613,2790cyclictest3229081-21kworker/3:1+events20:28:013
320651799280613,2790cyclictest3219692-21kworker/3:2+events20:04:223
320651799280612,2790cyclictest3206063-21kworker/3:0+events19:45:513
320651799280612,2790cyclictest3206063-21kworker/3:0+events19:45:513
320651199280613,2789cyclictest3275193-21kworker/1:0+events23:07:111
320651199280613,2789cyclictest3255007-21kworker/1:2+events21:57:211
320651199280613,2789cyclictest3221224-21kworker/1:2+events19:59:081
320651199280513,2788cyclictest3236947-21kworker/1:2+events21:06:151
320651799280413,2787cyclictest3276674-21kworker/3:1+events22:56:553
320651799280412,2788cyclictest3233970-21kworker/3:1+events20:40:593
320651499280413,2787cyclictest3275316-21kworker/2:0+events23:27:132
320651499280413,2787cyclictest3275316-21kworker/2:0+events23:27:132
320651199280413,2787cyclictest3271915-21kworker/1:2+events22:42:461
320651199280413,2787cyclictest3236947-21kworker/1:2+events20:51:501
320651199280313,2786cyclictest3283327-21kworker/1:1+events23:10:381
320651199280313,2786cyclictest3211477-21kworker/1:0+events19:24:261
320651199280311,2788cyclictest3247926-21kworker/1:0+events21:18:011
320651799280213,2785cyclictest3219692-21kworker/3:2+events20:07:073
320651499280212,2786cyclictest3216466-21kworker/2:2+events20:12:032
320651199280113,2784cyclictest3233607-21kworker/1:2+events20:35:341
320651499280013,2783cyclictest3213284-21kworker/2:0+events19:42:042
320651499280013,2783cyclictest3213284-21kworker/2:0+events19:42:032
320651199280013,2783cyclictest3228898-21kworker/1:0+events20:19:191
32065179927997,2788cyclictest3296407-21kworker/3:0+events23:51:233
320651499279913,2782cyclictest3207779-21kworker/2:1+events19:36:192
320651199279913,2782cyclictest3217969-21kworker/1:0+events19:51:331
320651199279812,2782cyclictest3290128-21kworker/1:2+events23:42:091
320651499279613,2779cyclictest3260878-21kworker/2:1+events22:02:572
320651199279613,2781cyclictest3214663-21kworker/1:2+events19:40:101
320651199279613,2781cyclictest3214663-21kworker/1:2+events19:40:091
320651199279613,2779cyclictest3273568-21kworker/1:1+events22:39:181
320651499279412,2780cyclictest3237033-21kworker/2:1+events20:45:062
320651799279214,2774cyclictest3232299-21kworker/3:2+events20:30:153
320651799279212,2776cyclictest3210027-21kworker/3:1+events19:19:113
320651199278913,2772cyclictest3302660-21kworker/1:1+events00:20:121
32065079927896,2779cyclictest3244722-21kworker/0:2+events21:24:310
320651799278813,2771cyclictest3253210-21kworker/3:2+events21:46:363
320651199278112,2765cyclictest3281697-21kworker/1:2+events23:19:041
320651799277913,2762cyclictest3250053-21kworker/3:1+events21:29:533
320651499277813,2761cyclictest3244904-21kworker/2:1+events21:39:482
320651499277413,2758cyclictest3239963-21kworker/2:2+events20:55:362
320651499277313,2756cyclictest3210089-21kworker/2:0+events19:22:352
320651499277213,2755cyclictest3293036-21kworker/2:0+events00:07:302
320651499277213,2755cyclictest3244904-21kworker/2:1+events21:29:302
320651499277013,2753cyclictest3296589-21kworker/2:2+events23:53:412
320651499277012,2754cyclictest3244904-21kworker/2:1+events21:12:442
320651499276813,2751cyclictest3230552-21kworker/2:1+events20:27:382
320651499276813,2751cyclictest3230552-21kworker/2:1+events20:23:042
320651499276813,2751cyclictest3207779-21kworker/2:1+events19:18:472
320651799276713,2750cyclictest3270429-21kworker/3:1+events22:35:323
320651499276713,2751cyclictest3304235-21kworker/2:1+events00:11:432
320651499276713,2750cyclictest3275316-21kworker/2:0+events22:46:122
320651799276612,2750cyclictest3275072-21kworker/3:0+events23:00:213
320651799276513,2748cyclictest3275072-21kworker/3:0+events22:45:273
320651799276412,2748cyclictest3235568-21kworker/3:3+events20:47:193
320651499276413,2747cyclictest3239963-21kworker/2:2+events20:52:122
320651799276213,2745cyclictest3206063-21kworker/3:0+events19:39:013
320651799276212,2746cyclictest3267212-21kworker/3:0+events22:32:583
320651799276113,2744cyclictest3259245-21kworker/3:1+events22:00:443
320651799276013,2743cyclictest3290687-21kworker/3:2+events00:14:023
320651799276013,2743cyclictest3246499-21kworker/3:1+events21:16:323
320651799275913,2742cyclictest3235568-21kworker/3:3+events21:09:443
320651799275813,2743cyclictest3219692-21kworker/3:2+events20:20:273
320651799275813,2741cyclictest3284928-21kworker/3:0+events23:19:273
320651499275813,2741cyclictest3207779-21kworker/2:1+events19:31:002
320651799275713,2740cyclictest3267212-21kworker/3:0+events22:18:313
320651799275711,2742cyclictest3272222-21kworker/3:2+events22:44:193
320651799275711,2742cyclictest3253210-21kworker/3:2+events21:56:133
320651799275612,2742cyclictest3235568-21kworker/3:3+events21:35:373
320651799275513,2738cyclictest3235568-21kworker/3:3+events21:14:143
320651799275412,2738cyclictest3284928-21kworker/3:0+events23:30:143
320651499275213,2735cyclictest3239963-21kworker/2:2+events21:03:152
320651799275112,2735cyclictest3262582-21kworker/3:0+events22:09:043
320651199274713,2730cyclictest3302660-21kworker/1:1+events00:06:211
320651499274611,2730cyclictest3311029-21kworker/2:1+events00:34:192
320651199274612,2730cyclictest3290128-21kworker/1:2+events23:48:411
32065179927459,2732cyclictest3292882-21kworker/3:1+events23:39:513
32065149927449,2731cyclictest3299897-21kworker/2:3+events00:00:142
320651199274313,2726cyclictest3213073-21kworker/1:1+events19:28:421
320651799274212,2726cyclictest3275072-21kworker/3:0+events23:06:503
320651499274212,2726cyclictest3222532-21kworker/2:0+events19:56:502
320651199273813,2721cyclictest3216254-21kworker/1:1+events19:37:521
320651799273413,2717cyclictest3305967-21kworker/3:1+events00:20:353
320651199273013,2713cyclictest3286564-21kworker/1:1+events23:37:111
320651199272913,2712cyclictest3241769-21kworker/1:1+events21:02:511
320651499271513,2699cyclictest3257697-21kworker/2:0+events21:57:442
320651499271012,2696cyclictest3305842-21kworker/2:2+events00:25:072
320651799270811,2693cyclictest3286593-21kworker/3:1+events23:29:283
320651799270811,2693cyclictest3286593-21kworker/3:1+events23:29:283
320650799270612,2690cyclictest3273718-21kworker/0:2+events23:20:130
320651199270511,2691cyclictest3247926-21kworker/1:0+events21:22:371
320651499270412,2688cyclictest3243273-21kworker/2:0+events21:21:512
320651499269813,2681cyclictest3273685-21kworker/2:2+events22:40:292
320651199269713,2680cyclictest3247926-21kworker/1:0+events21:34:281
320651199268912,2673cyclictest3219631-21kworker/1:1+events20:02:061
320651799268512,2669cyclictest3307852-21kworker/3:2+events00:36:593
320651799268510,2671cyclictest3267212-21kworker/3:0+events22:23:033
320651799268413,2667cyclictest3290687-21kworker/3:2+events23:46:453
320651199267313,2656cyclictest3298160-21kworker/1:0+events00:02:321
320651499266612,2650cyclictest3244904-21kworker/2:1+events21:44:152
320651199266612,2650cyclictest3257009-21kworker/1:0+events21:48:301
320651199266513,2648cyclictest3286564-21kworker/1:1+events23:34:301
320651199266313,2646cyclictest3265785-21kworker/1:0+events22:18:101
320651199266313,2646cyclictest3260847-21kworker/1:1+events22:14:471
320651499266213,2645cyclictest3294665-21kworker/2:1+events23:45:132
320651199265913,2642cyclictest3270310-21kworker/1:1+events22:27:371
320651499265711,2642cyclictest3293036-21kworker/2:0+events23:40:142
320651199265213,2636cyclictest3267459-21kworker/1:2+events22:24:591
320651499265013,2633cyclictest3305842-21kworker/2:2+events00:21:422
320650799264413,2627cyclictest3210289-21kworker/0:2+events19:30:370
320651199264213,2625cyclictest3302660-21kworker/1:1+events00:19:041
320651199264213,2625cyclictest3205625-21kworker/1:1+events19:15:441
320651499264013,2623cyclictest3293036-21kworker/2:0+events23:55:152
320651199263913,2622cyclictest3230522-21kworker/1:1+events20:41:441
320651199263613,2620cyclictest3298160-21kworker/1:0+events23:57:581
320651199263513,2618cyclictest3255007-21kworker/1:2+events22:02:121
320651799263310,2618cyclictest3235568-21kworker/3:3+events21:43:293
320651199263013,2612cyclictest3214663-21kworker/1:2+events19:33:151
320651799262913,2613cyclictest3299798-21kworker/3:0+events00:02:563
320651799262611,2611cyclictest3239934-21kworker/3:1+events20:52:343
320651199262513,2608cyclictest3281697-21kworker/1:2+events23:03:011
320650799262512,2609cyclictest3254683-21kworker/0:2+events21:49:170
320651199262313,2607cyclictest3302660-21kworker/1:1+events00:10:571
320651199262113,2604cyclictest3236947-21kworker/1:2+events21:13:511
32065149926166,2606cyclictest3207779-21kworker/2:1+events19:27:102
32065119926168,2604cyclictest3221224-21kworker/1:2+events20:13:571
320650799260312,2587cyclictest3216285-21kworker/0:1+events19:43:340
320650799260312,2587cyclictest3216285-21kworker/0:1+events19:43:340
320651199259813,2581cyclictest3305934-21kworker/1:2+events00:26:161
320651499258912,2574cyclictest3291307-21kworker/2:2+events23:31:482
320651199257912,2564cyclictest3224520-21kworker/1:0+events20:09:021
320651799257811,2564cyclictest3219692-21kworker/3:2+events19:55:423
320651199257612,2561cyclictest3286564-21kworker/1:1+events23:27:581
320651199257612,2561cyclictest3286564-21kworker/1:1+events23:27:571
320651199257612,2561cyclictest3228898-21kworker/1:0+events20:21:561
320651199257512,2559cyclictest3281697-21kworker/1:2+events23:23:431
320651199257512,2559cyclictest3207599-21kworker/1:2+events19:14:571
320651199257112,2555cyclictest3309513-21kworker/1:1+events00:36:131
320651499256512,2550cyclictest3264125-21kworker/2:0+events22:23:262
320651199256512,2549cyclictest3214663-21kworker/1:2+events19:47:011
320651199256512,2549cyclictest3214663-21kworker/1:2+events19:47:011
320651799256112,2546cyclictest3252811-21kworker/3:0+events21:33:423
320651799255812,2542cyclictest3290687-21kworker/3:2+events00:09:023
320651799255210,2538cyclictest3309481-21kworker/3:1+events00:29:183
320651499255111,2536cyclictest3230552-21kworker/2:1+events20:35:132
320651499255012,2534cyclictest3275316-21kworker/2:0+events23:14:532
320651499254811,2533cyclictest3278218-21kworker/2:1+events22:53:292
320651499254511,2530cyclictest3275316-21kworker/2:0+events23:21:002
320651499254111,2526cyclictest3260878-21kworker/2:1+events22:13:392
320651799253711,2523cyclictest3290687-21kworker/3:2+events23:40:373
320651799253612,2520cyclictest3290687-21kworker/3:2+events23:59:523
320651799253611,2521cyclictest3239934-21kworker/3:1+events21:00:573
320651499253612,2520cyclictest3293036-21kworker/2:0+events00:15:592
320651799253510,2521cyclictest3207748-21kworker/3:2+events19:13:493
320651799253412,2518cyclictest3286593-21kworker/3:1+events23:21:253
320651799253410,2520cyclictest3248104-21kworker/3:0+events21:22:143
320651499253412,2518cyclictest3190071-21kworker/2:2+events19:14:122
320651799252911,2514cyclictest3307852-21kworker/3:2+events00:34:403
32065179924998,2487cyclictest3206063-21kworker/3:0+events19:43:573
32065179924998,2487cyclictest3206063-21kworker/3:0+events19:43:573
320651199249512,2481cyclictest3309513-21kworker/1:1+events00:30:051
320650799244812,2432cyclictest3213452-21kworker/0:0+events19:37:040
32065079924209,2409cyclictest3272117-21kworker/0:0+events22:40:060
320650799241312,2397cyclictest3235624-21kworker/0:0+events21:06:390
320650799240912,2394cyclictest3217996-21kworker/0:2+events20:27:150
320650799240612,2391cyclictest3235624-21kworker/0:0+events21:39:250
320650799232612,2310cyclictest3279698-21kworker/0:1+events22:59:340
320650799231912,2303cyclictest3288756-21kworker/0:1+events00:32:480
320650799231812,2303cyclictest3195136-21kworker/0:1+events19:20:420
32065079921019,2089cyclictest3262706-21kworker/0:1+events22:34:480
320650799196110,1948cyclictest3206242-21kworker/0:0+events19:10:210
32059872191163,23sleep30-21swapper/319:06:113
32060012155110,24sleep20-21swapper/219:06:222
32060642146111,28sleep10-21swapper/119:07:091
32062392145109,31sleep00-21swapper/019:09:240
3206507998178,2cyclictest3272117-21kworker/0:0+pm22:39:560
3206507993523,8cyclictest865-10wireplumber23:40:000
320650799335,12cyclictest739-21runrttasks23:52:460
320650799311,17cyclictest3238707-21awk20:50:010
3206507993016,11cyclictest3294847-21idleruntime-cro23:45:000
3206507993014,14cyclictest3270598-21awk22:30:000
320650799301,25cyclictest3286682-21dpkg23:20:000
320650799271,22cyclictest3235524-21munin-run20:40:000
3206507992619,5cyclictest740-21snmpd22:52:320
3206507992617,7cyclictest865-10wireplumber00:20:000
320650799261,21cyclictest3241867-21sendmail21:00:010
320650799261,21cyclictest3229217-21idleruntime-cro20:20:000
3206507992519,4cyclictest3264250-21idleruntime-cro22:10:000
320650799251,20cyclictest865-10wireplumber19:20:000
320650799251,20cyclictest3222899-21awk20:00:000
320650799251,19cyclictest3261096-21awk22:00:000
320650799251,19cyclictest3248205-21sendmail-msp21:20:000
320650799251,19cyclictest3213391-21munin-run19:30:000
320650799251,16cyclictest3235805-21apt-get20:40:120
320650799250,22cyclictest3242510-21/usr/sbin/munin21:00:170
320650799249,12cyclictest3267449-21lockfile-remove22:20:000
3206507992413,9cyclictest3233934-21cron20:35:000
320650799241,20cyclictest3308064-21idleruntime-cro00:25:000
320650799241,19cyclictest3299858-21awk00:00:000
320650799241,19cyclictest3253186-21munin-run21:35:000
320650799240,10cyclictest0-21swapper/022:10:180
320650799232,7cyclictest3261368-21apt-get22:00:110
320650799231,20cyclictest0-21swapper/020:10:000
320650799231,18cyclictest3269001-21munin-run22:25:000
320650799230,22cyclictest0-21swapper/019:50:150
320650799230,19cyclictest3227814-21latency_hist20:15:010
320650799221,18cyclictest3285090-21munin-run23:15:000
320650799221,17cyclictest3303042-21perf00:10:000
320650799221,17cyclictest3251606-21perf21:30:000
3206507992210,7cyclictest3219703-21cron19:50:000
3206507992210,7cyclictest3219703-21cron19:50:000
320650799211,17cyclictest3295374-21/usr/sbin/munin23:45:110
320650799211,17cyclictest3290115-21idleruntime-cro23:30:010
320650799211,17cyclictest3290115-21idleruntime-cro23:30:000
320650799211,17cyclictest3283518-21grep23:10:010
320650799211,17cyclictest3259510-21munin-run21:55:000
320650799211,16cyclictest3312786-21munin-run00:39:590
320650799210,20cyclictest3246596-21awk21:15:000
320650799208,9cyclictest3223150-21apt-get20:00:110
320650799207,11cyclictest3301517-21date00:05:010
3206507992016,3cyclictest3230797-21cut20:25:010
320650799191,15cyclictest3309633-21perf00:30:000
320650799191,15cyclictest32916952sleep023:35:000
320650799190,9cyclictest3238927-21/usr/sbin/munin20:50:130
320650799190,16cyclictest3280585-21dpkg23:00:020
320650799181,14cyclictest3304119-21sed00:10:220
320650799181,13cyclictest3256356-21munin-run21:45:000
3206507991710,3cyclictest967-40gmain22:45:190
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional