You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-30 - 05:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackfslot7s.osadl.org (updated Tue Jun 30, 2026 00:44:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
376645799450718,4482cyclictest3867688-21kworker/1:0+events00:30:131
376645799450718,4482cyclictest3867688-21kworker/1:0+events00:30:121
376645799405021,4023cyclictest3762543-21kworker/1:2+events19:35:001
376645799360515,3585cyclictest3813045-21kworker/1:0+events21:40:001
376646199355915,3539cyclictest3798844-21kworker/2:2+events20:51:412
376645799351918,3497cyclictest3861221-21kworker/1:0+events00:10:251
376646199348812,3473cyclictest3845036-21kworker/2:0+events23:25:022
376645799334115,3322cyclictest3838516-21kworker/1:2+events22:55:121
376646199307015,3050cyclictest3811604-21kworker/2:1+events21:45:182
376645799293614,2918cyclictest3827316-21kworker/1:0+events22:24:421
376646199288812,2872cyclictest3817627-21kworker/2:2+events22:00:282
376645799288314,2861cyclictest3867688-21kworker/1:0+events00:25:181
376646399287411,2859cyclictest3792445-21kworker/3:0+events20:30:213
376646199287114,2854cyclictest3806514-21kworker/2:2+events21:15:142
376645799284611,2831cyclictest3769670-21kworker/1:1+events19:20:171
376645799283713,2820cyclictest3853406-21kworker/1:2+events23:54:501
376646199283313,2818cyclictest3861365-21kworker/2:1+events00:10:012
376646199281313,2796cyclictest3845036-21kworker/2:0+events23:44:522
376646199280913,2792cyclictest3782378-21kworker/2:2+events19:59:052
376645799280510,2791cyclictest3806636-21kworker/1:0+events21:20:141
376646199279513,2778cyclictest3854732-21kworker/2:2+events23:59:252
376645799279213,2774cyclictest3846548-21kworker/1:1+events23:25:241
376645799278112,2765cyclictest3862875-21kworker/1:1+events00:24:101
376645799278012,2766cyclictest3825681-21kworker/1:2+events22:30:491
376645799277513,2760cyclictest3795643-21kworker/1:1+events20:45:091
376646199277112,2755cyclictest3862965-21kworker/2:0+events00:33:382
376646199277112,2755cyclictest3862965-21kworker/2:0+events00:33:382
376645299277111,2756cyclictest3809629-21kworker/0:0+events21:29:210
376645799276813,2751cyclictest3853406-21kworker/1:2+events23:40:201
376645799275313,2736cyclictest3786001-21kworker/1:2+events20:18:111
376645799274313,2727cyclictest3790728-21kworker/1:0+events20:23:101
376646399274113,2725cyclictest3797087-21kworker/3:2+events20:55:503
37664579927419,2728cyclictest3851803-21kworker/1:1+events23:34:581
376645799273913,2722cyclictest3779280-21kworker/1:2+events19:53:491
376645799273710,2723cyclictest3803375-21kworker/1:1+events21:02:431
376645799273610,2722cyclictest3790728-21kworker/1:0+events20:31:511
376645799273413,2718cyclictest3833634-21kworker/1:0+events22:46:521
37664579927328,2720cyclictest3853406-21kworker/1:2+events23:45:391
376645799273213,2715cyclictest3853406-21kworker/1:2+events00:02:501
376645799272913,2713cyclictest3811439-21kworker/1:2+events21:29:451
376645799272613,2710cyclictest3814754-21kworker/1:3+events21:52:081
376645799272613,2709cyclictest3772796-21kworker/1:0+events20:01:411
376645799272413,2707cyclictest3838516-21kworker/1:2+events23:06:151
376645799272413,2707cyclictest3822316-21kworker/1:1+events22:06:411
376645799272313,2707cyclictest3825681-21kworker/1:2+events22:17:461
376645799272213,2706cyclictest3803375-21kworker/1:1+events21:13:211
376646199272013,2703cyclictest3766188-21kworker/2:2+events19:19:102
376645799272012,2704cyclictest3840410-21kworker/1:0+events23:03:121
376645799271912,2703cyclictest3790728-21kworker/1:0+events20:43:591
376645799271712,2701cyclictest3833634-21kworker/1:0+events22:51:231
376645799271613,2699cyclictest3838516-21kworker/1:2+events23:20:261
376645799271513,2698cyclictest3867688-21kworker/1:0+events00:37:481
376645799271513,2698cyclictest3822316-21kworker/1:1+events22:03:341
37664579927149,2701cyclictest3772796-21kworker/1:0+events20:05:071
376645799271413,2697cyclictest3814754-21kworker/1:3+events21:48:401
376645799271213,2695cyclictest3851803-21kworker/1:1+events23:36:531
376645799271213,2695cyclictest3786001-21kworker/1:2+events20:28:061
376645799270913,2692cyclictest3795643-21kworker/1:1+events20:54:431
376646399270513,2688cyclictest3861040-21kworker/3:2+events00:01:193
376646399270512,2689cyclictest3864419-21kworker/3:1+events00:20:213
376645799270512,2690cyclictest3825681-21kworker/1:2+events22:37:461
376645299270013,2683cyclictest3866221-21kworker/0:0+events00:26:520
376645299270013,2683cyclictest3798510-21kworker/0:2+events20:46:180
376646399269813,2680cyclictest3809779-21kworker/3:2+events21:40:223
376646199269813,2681cyclictest3827069-21kworker/2:2+events22:40:242
376645299269011,2676cyclictest3854879-21kworker/0:1+events23:45:160
376645299268613,2670cyclictest3838634-21kworker/0:2+events23:00:290
376646199268413,2667cyclictest3827069-21kworker/2:2+events22:45:192
376646199268413,2667cyclictest3792326-21kworker/2:0+events21:06:292
376645299268413,2667cyclictest3816088-21kworker/0:0+events21:58:090
376645299268413,2667cyclictest3801868-21kworker/0:0+events20:59:160
376646199268313,2666cyclictest3854732-21kworker/2:2+events23:54:282
376646199268313,2666cyclictest3784026-21kworker/2:1+events20:06:422
376645299268313,2666cyclictest3787440-21kworker/0:1+events20:34:310
376646199268212,2666cyclictest3792326-21kworker/2:0+events20:33:222
376645299268213,2665cyclictest3798510-21kworker/0:2+events20:50:530
376645299267613,2659cyclictest3862668-21kworker/0:0+events00:08:080
376646199266413,2647cyclictest3846643-21kworker/2:1+events23:18:102
376645299266311,2648cyclictest3769573-21kworker/0:1+events19:22:310
376645299266211,2647cyclictest3787440-21kworker/0:1+events20:21:140
376646199266113,2644cyclictest3862965-21kworker/2:0+events00:16:552
376646199266113,2644cyclictest3792326-21kworker/2:0+events20:36:012
376645299266013,2643cyclictest3853162-21kworker/0:2+events23:44:290
376646199265713,2641cyclictest3862965-21kworker/2:0+events00:24:542
376646399265513,2638cyclictest3789437-21kworker/3:2+events20:23:573
376646199265313,2636cyclictest3848270-21kworker/2:2+events23:24:152
376646199265313,2636cyclictest3808153-21kworker/2:1+events21:21:222
376646199265212,2636cyclictest3815969-21kworker/2:0+events21:44:312
376646399265112,2635cyclictest3864419-21kworker/3:1+events00:29:033
376646199265110,2639cyclictest3792326-21kworker/2:0+events20:55:062
376646399265013,2634cyclictest3809779-21kworker/3:2+events22:17:233
376646199265013,2633cyclictest3869110-21kworker/2:1+events00:28:192
376645799264913,2632cyclictest3772796-21kworker/1:0+events20:10:581
376645799264912,2633cyclictest3772796-21kworker/1:0+events19:41:211
376646399264713,2630cyclictest3821017-21kworker/3:1+events22:04:443
376646399264613,2629cyclictest3797087-21kworker/3:2+events21:10:173
376646199264613,2629cyclictest3774576-21kworker/2:0+events19:42:302
376645799264613,2631cyclictest3833634-21kworker/1:0+events22:40:021
376646199264513,2628cyclictest3841853-21kworker/2:1+events23:11:162
376645799264312,2627cyclictest3825681-21kworker/1:2+events22:28:061
376646399264211,2626cyclictest3832209-21kworker/3:0+events22:36:373
376645299264213,2625cyclictest3816088-21kworker/0:0+events22:17:000
376646399264113,2624cyclictest3787500-21kworker/3:0+events20:19:203
376646199264113,2624cyclictest3774576-21kworker/2:0+events19:52:422
376646199264112,2625cyclictest3830537-21kworker/2:0+events22:35:262
376646199264112,2625cyclictest3784026-21kworker/2:1+events20:15:542
376645299264113,2624cyclictest3827259-21kworker/0:1+events22:24:180
376646399264013,2623cyclictest3850105-21kworker/3:0+events23:33:073
376646199264012,2624cyclictest3792326-21kworker/2:0+events20:42:032
376646199264011,2626cyclictest3797237-21kworker/2:1+events20:48:142
376646399263915,2620cyclictest3797087-21kworker/3:2+events20:43:353
376646399263913,2624cyclictest3809779-21kworker/3:2+events22:12:273
376646399263913,2623cyclictest3853377-21kworker/3:2+events23:40:433
376646399263913,2622cyclictest3783877-21kworker/3:1+events20:36:233
376646399263912,2624cyclictest3819285-21kworker/3:0+events21:53:593
376646199263913,2622cyclictest3827069-21kworker/2:2+events22:26:572
376646199263913,2622cyclictest3821823-21kworker/2:1+events22:22:452
376646199263912,2624cyclictest3854732-21kworker/2:2+events00:04:442
376646399263813,2621cyclictest3803698-21kworker/3:0+events21:09:323
376646199263813,2621cyclictest3813781-21kworker/2:2+events21:36:342
376646199263813,2621cyclictest3790693-21kworker/2:2+events20:27:202
376646399263713,2620cyclictest3843367-21kworker/3:2+events23:27:443
376646399263713,2620cyclictest3813073-21kworker/3:0+events21:39:373
376646399263712,2621cyclictest3825563-21kworker/3:1+events22:44:353
376645299263713,2622cyclictest3845128-21kworker/0:0+events23:21:100
376645299263713,2621cyclictest3869052-21kworker/0:2+events00:36:410
376646199263613,2619cyclictest3821823-21kworker/2:1+events22:14:422
376645299263613,2619cyclictest3794074-21kworker/0:2+events20:39:030
376646199263513,2618cyclictest3792326-21kworker/2:0+events21:11:502
376645299263513,2618cyclictest3811469-21kworker/0:2+events21:30:530
376646199263213,2615cyclictest3767870-21kworker/2:0+events19:25:562
376645799263212,2616cyclictest3822316-21kworker/1:1+events22:13:571
376645299263112,2615cyclictest3777646-21kworker/0:1+events19:48:120
376646399262813,2612cyclictest3767994-21kworker/3:3+events19:26:423
376646199262613,2609cyclictest3835792-21kworker/2:0+events22:57:292
376645799261912,2603cyclictest3762543-21kworker/1:2+events19:38:191
376646399261811,2603cyclictest3834042-21kworker/3:2+events22:52:293
37664579926078,2595cyclictest3861221-21kworker/1:0+events00:18:031
376646399259912,2583cyclictest3843367-21kworker/3:2+events23:14:423
376646399259613,2579cyclictest3851716-21kworker/3:1+events23:59:023
376645799259310,2579cyclictest3772796-21kworker/1:0+events19:45:321
376645799258711,2572cyclictest3861221-21kworker/1:0+events00:08:521
376646199258613,2569cyclictest3771140-21kworker/2:1+events19:32:232
376645799257512,2559cyclictest3762543-21kworker/1:2+events19:15:271
376645299257512,2560cyclictest3801868-21kworker/0:0+events21:09:540
376646199257011,2555cyclictest3774576-21kworker/2:0+events20:00:112
376646199256812,2552cyclictest3821823-21kworker/2:1+events22:16:372
376646199256812,2552cyclictest3792326-21kworker/2:0+events21:00:252
376645299256612,2550cyclictest3777646-21kworker/0:1+events19:51:360
376645299256213,2545cyclictest3845128-21kworker/0:0+events23:16:380
376646199256112,2546cyclictest3835792-21kworker/2:0+events22:50:592
376646399255812,2543cyclictest3850105-21kworker/3:0+events23:35:453
376646199255612,2541cyclictest3784026-21kworker/2:1+events20:12:042
37664619925509,2537cyclictest3861365-21kworker/2:1+events00:05:302
376646399254912,2534cyclictest3767994-21kworker/3:3+events19:33:303
376646399254712,2532cyclictest3845244-21kworker/3:1+events23:21:343
376645299254513,2528cyclictest3836888-21kworker/0:1+events22:58:130
376645799253912,2523cyclictest3801717-21kworker/1:0+events21:06:071
376646199253213,2516cyclictest3840230-21kworker/2:2+events23:03:582
376645799252812,2513cyclictest3814676-21kworker/1:2+events21:44:551
376646399252712,2511cyclictest3767994-21kworker/3:3+events19:52:203
37664639925249,2511cyclictest3837161-21kworker/3:0+events23:08:123
376646399252012,2504cyclictest3783877-21kworker/3:1+events20:09:243
376645799251812,2504cyclictest3838516-21kworker/1:2+events23:10:061
376645799251712,2501cyclictest3801717-21kworker/1:0+events20:56:581
376645799250412,2488cyclictest3806636-21kworker/1:0+events21:16:011
376646399250212,2486cyclictest3783877-21kworker/3:1+events20:14:453
376646199250112,2485cyclictest3774576-21kworker/2:0+events19:48:352
376645799250112,2485cyclictest3859539-21kworker/1:1+events23:57:541
376646399249912,2485cyclictest3809779-21kworker/3:2+events21:55:073
376645799249612,2480cyclictest3838516-21kworker/1:2+events23:16:131
376646199249512,2480cyclictest3771140-21kworker/2:1+events19:22:102
376646399249312,2478cyclictest3861040-21kworker/3:2+events00:15:213
376646399248812,2472cyclictest3767994-21kworker/3:3+events19:44:023
376645799248712,2471cyclictest3803375-21kworker/1:1+events21:30:301
376645799248612,2470cyclictest3769670-21kworker/1:1+events19:26:201
376645799248611,2471cyclictest3780868-21kworker/1:1+events19:57:581
37664579924849,2472cyclictest3814754-21kworker/1:3+events21:55:291
376645299248313,2467cyclictest3836888-21kworker/0:1+events22:54:250
376645299248111,2466cyclictest3845128-21kworker/0:0+events23:39:580
376645299247912,2464cyclictest3755958-21kworker/0:2+events19:19:540
376646399247812,2462cyclictest3774488-21kworker/3:0+events19:39:283
376646199247711,2462cyclictest3817627-21kworker/2:2+events21:58:562
376645799246910,2456cyclictest3767663-21kworker/1:0+events19:14:191
376645299246710,2453cyclictest3853162-21kworker/0:2+events00:13:050
376645299246513,2448cyclictest3809629-21kworker/0:0+events21:21:460
376646199245912,2445cyclictest3870768-21kworker/2:2+events00:35:102
376646199245512,2439cyclictest3806514-21kworker/2:2+events21:26:392
376646399244812,2433cyclictest3837161-21kworker/3:0+events22:58:593
376646399244812,2432cyclictest3861040-21kworker/3:2+events00:13:283
376645299243812,2423cyclictest3777646-21kworker/0:1+events19:58:410
376646199243612,2421cyclictest3817627-21kworker/2:2+events21:54:452
376645299243612,2420cyclictest3798510-21kworker/0:2+events21:01:360
376646399243513,2418cyclictest3809779-21kworker/3:2+events21:34:413
376646399243512,2421cyclictest3809779-21kworker/3:2+events21:46:253
376646199243512,2420cyclictest3821823-21kworker/2:1+events22:09:232
376645299243512,2420cyclictest3814346-21kworker/0:1+events21:36:560
376646199243110,2418cyclictest3854732-21kworker/2:2+events23:48:462
37664529924298,2417cyclictest3787440-21kworker/0:1+events20:40:110
37664529924237,2412cyclictest3829145-21kworker/0:0+events22:38:070
376646399242211,2407cyclictest3864419-21kworker/3:1+events00:35:543
376646199242211,2407cyclictest3790693-21kworker/2:2+events20:22:472
376646399242112,2405cyclictest3851716-21kworker/3:1+events23:51:243
376645299242112,2405cyclictest3787440-21kworker/0:1+events20:28:290
376646399241611,2401cyclictest3843367-21kworker/3:2+events23:19:173
376646399241312,2397cyclictest3798696-21kworker/3:0+events20:48:593
376646399241312,2397cyclictest3767994-21kworker/3:3+events20:00:343
376645299241212,2396cyclictest3829145-21kworker/0:0+events22:31:350
376645299241210,2399cyclictest3848147-21kworker/0:1+events23:30:510
376645299241210,2398cyclictest3824019-21kworker/0:1+events22:11:180
376646399241112,2395cyclictest3805452-21kworker/3:1+events21:16:223
376646399241111,2396cyclictest3767994-21kworker/3:3+events19:56:053
376646199241111,2397cyclictest3850167-21kworker/2:1+events23:33:282
376645299241112,2395cyclictest3816088-21kworker/0:0+events21:50:340
376645299241110,2397cyclictest3817689-21kworker/0:1+events21:49:490
376646399241012,2394cyclictest3840442-21kworker/3:1+events23:02:253
376645299241012,2394cyclictest3853162-21kworker/0:2+events23:52:320
376646199240912,2393cyclictest3840230-21kworker/2:2+events23:06:392
376645299240912,2393cyclictest3859860-21kworker/0:3+events00:02:270
376645299240912,2393cyclictest3816088-21kworker/0:0+events22:07:500
376646399240812,2392cyclictest3851716-21kworker/3:1+events23:47:363
376646199240712,2393cyclictest3845036-21kworker/2:0+events23:37:382
376646199240712,2391cyclictest3830537-21kworker/2:0+events22:30:252
376646199240712,2391cyclictest3772914-21kworker/2:2+events19:39:052
376645299240712,2392cyclictest3771256-21kworker/0:0+events19:36:070
376645299240611,2391cyclictest3836888-21kworker/0:1+events22:49:490
376646199240512,2389cyclictest3809806-21kworker/2:0+events21:31:382
37664529924059,2392cyclictest3805182-21kworker/0:2+events21:15:370
376646399240310,2389cyclictest3767694-21kworker/3:0+events19:13:333
376645799240311,2389cyclictest3786001-21kworker/1:2+events20:36:471
376646399240112,2387cyclictest3829286-21kworker/3:2+events22:30:023
376645299240112,2385cyclictest3833786-21kworker/0:2+events22:42:180
376645299239910,2385cyclictest3769573-21kworker/0:1+events19:30:300
376646399239512,2379cyclictest3856424-21kworker/3:0+events00:08:283
376645299238812,2372cyclictest3843520-21kworker/0:1+events23:09:430
376646399238412,2368cyclictest3869227-21kworker/3:0+events00:32:513
376646399238412,2368cyclictest3869227-21kworker/3:0+events00:32:513
376646399238212,2366cyclictest3825563-21kworker/3:1+events22:28:523
376645299236911,2354cyclictest3838634-21kworker/0:2+events23:13:110
376645299236712,2351cyclictest3771256-21kworker/0:0+events19:43:160
376645299236012,2344cyclictest3869052-21kworker/0:2+events00:34:260
376645299236012,2344cyclictest3869052-21kworker/0:2+events00:34:250
376645299235212,2336cyclictest3829145-21kworker/0:0+events22:26:360
376646399234912,2334cyclictest3809779-21kworker/3:2+events21:27:493
376645299234312,2328cyclictest3771256-21kworker/0:0+events20:08:150
376645299233911,2324cyclictest3811469-21kworker/0:2+events21:42:380
376646199232112,2306cyclictest3761031-21kworker/2:1+events19:14:422
376646399232012,2305cyclictest3767994-21kworker/3:3+events19:20:413
376646399231912,2303cyclictest3808057-21kworker/3:0+events21:22:533
376646399230412,2288cyclictest3767694-21kworker/3:0+events19:18:253
376646399229011,2276cyclictest3783877-21kworker/3:1+events20:28:513
376645299211710,2104cyclictest3785625-21kworker/0:2+events20:13:590
37661762154112,24sleep00-21swapper/019:09:280
37660272142111,26sleep10-21swapper/119:07:321
37660922141107,29sleep30-21swapper/319:08:233
37659922140108,27sleep20-21swapper/219:07:042
3766452993021,6cyclictest3767919-21systemd19:15:000
3766463992712,7cyclictest3837129-10sh22:50:003
376646399241,20cyclictest3824228-21awk22:10:003
376646399241,20cyclictest0-21swapper/322:20:023
376646399221,16cyclictest3803653-21sleep21:05:003
376646399210,8cyclictest3799659-21ntpq20:50:193
376646399210,2cyclictest3778046-21/usr/sbin/munin19:45:183
376645299209,7cyclictest3866148-21perf00:19:590
376645299201,10cyclictest3859801-21munin-run00:00:000
3766452991914,3cyclictest3789423-21sendmail20:20:000
3766452991911,3cyclictest3806798-21munin-run21:15:000
3766452991910,6cyclictest3784149-21sleep20:04:590
3766452991812,4cyclictest3848482-21perf23:29:550
3766452991810,6cyclictest558333-21systemd-journal00:21:010
376645299171,10cyclictest3773047-21munin-run19:30:000
376645299166,7cyclictest3822651-21awk22:05:000
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional