You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-03 - 15:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackfslot7s.osadl.org (updated Wed Jun 03, 2026 00:44:20)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
352976999532622,5294cyclictest3591390-21kworker/3:2+events22:30:143
352976299359717,3575cyclictest3585069-21kworker/1:2+events22:15:221
352976999291114,2892cyclictest3599531-21kworker/3:0+events23:00:213
352976999279611,2780cyclictest3616350-21kworker/3:0+events23:50:213
352976599278613,2770cyclictest3536782-21kworker/2:2+events20:00:172
352976999277811,2762cyclictest3612604-21kworker/3:2+events23:35:123
352975499274413,2726cyclictest3597971-21kworker/0:1+events23:15:110
352976299272013,2703cyclictest3572415-21kworker/1:2+events21:35:201
352976999271813,2701cyclictest3584919-21kworker/3:1+events22:10:463
352976999271512,2699cyclictest3588125-21kworker/3:1+events22:35:163
352976999271213,2694cyclictest3589752-21kworker/3:0+events22:23:213
352976299271212,2695cyclictest3602981-21kworker/1:2+events23:10:121
352976599270511,2689cyclictest3621009-21kworker/2:3+events00:15:142
352976999269913,2683cyclictest3561473-21kworker/3:1+events20:57:343
35297629926769,2662cyclictest3580262-21kworker/1:0+events22:20:171
352976999267113,2656cyclictest3540078-21kworker/3:1+events19:55:143
352976599266813,2651cyclictest3615644-21kworker/2:2+events23:44:132
352976599266813,2651cyclictest3615644-21kworker/2:2+events23:44:132
352976599266413,2647cyclictest3573809-21kworker/2:1+events21:34:142
352976299265113,2634cyclictest3620507-21kworker/1:0+events00:10:131
352975499265113,2636cyclictest3573866-21kworker/0:1+events21:40:180
352976599265010,2635cyclictest3597535-21kworker/2:2+events22:58:502
352975499263313,2616cyclictest3584328-21kworker/0:0+events22:25:170
35297629926231,2616cyclictest3576976-21kworker/1:1+events21:41:241
352976999259414,2577cyclictest3616350-21kworker/3:0+events23:55:173
352976999258212,2566cyclictest3618752-21kworker/3:2+events00:00:173
352976299257913,2562cyclictest3601120-21kworker/1:0+events22:59:571
352975499257812,2562cyclictest3625910-21kworker/0:1+events00:25:340
352976599257312,2557cyclictest3578754-21kworker/2:1+events21:52:502
352976599257312,2557cyclictest3578754-21kworker/2:1+events21:52:502
352976999255112,2535cyclictest3582122-21kworker/3:2+events22:02:253
352976299253113,2514cyclictest3543343-21kworker/1:2+events20:06:461
35297699924995,2490cyclictest3571035-21kworker/3:3+events21:46:483
352976999249712,2481cyclictest3540078-21kworker/3:1+events19:50:163
352976999249712,2481cyclictest3540078-21kworker/3:1+events19:50:153
352976999249612,2481cyclictest3601589-21kworker/3:2+events23:15:573
352976599249612,2481cyclictest3563040-21kworker/2:2+events21:11:422
352976299249212,2476cyclictest3536813-21kworker/1:1+events19:38:441
352976299249112,2476cyclictest3595964-21kworker/1:2+events22:43:391
352976999248711,2472cyclictest3551478-21kworker/3:2+events20:33:403
352976999248612,2470cyclictest3522993-21kworker/3:2+events19:25:253
352976999248012,2464cyclictest3605737-21kworker/3:0+events23:11:463
35297699924799,2466cyclictest3596292-21kworker/3:2+events22:54:413
352976999247911,2465cyclictest3589752-21kworker/3:0+events22:25:413
352976999247712,2461cyclictest3565942-21kworker/3:0+events21:08:393
35297549924776,2467cyclictest3541920-21kworker/0:2+events20:18:030
352976999247412,2458cyclictest3556264-21kworker/3:1+events20:41:173
352976999247410,2460cyclictest3536156-21kworker/3:0+events19:44:063
352976999247212,2457cyclictest3624147-21kworker/3:1+events00:22:553
352976999246810,2454cyclictest3588125-21kworker/3:1+events22:55:503
35297699924679,2455cyclictest3610948-21kworker/3:1+events23:29:123
352976999246712,2451cyclictest3570899-21kworker/3:2+events21:24:433
352976999246612,2451cyclictest3601589-21kworker/3:2+events23:24:393
352976999246512,2450cyclictest3630605-21kworker/3:2+events00:37:293
35297699924606,2451cyclictest3582122-21kworker/3:2+events22:16:503
352976999245112,2435cyclictest3614237-21kworker/3:1+events23:45:203
352976999245112,2435cyclictest3614237-21kworker/3:1+events23:45:193
35297629924508,2438cyclictest3559520-21kworker/1:2+events20:51:331
352976299244811,2434cyclictest3624053-21kworker/1:2+events00:19:531
352976999244711,2432cyclictest3618752-21kworker/3:2+events00:08:173
352976299244612,2431cyclictest3536813-21kworker/1:1+events20:15:051
352976299243610,2422cyclictest3580262-21kworker/1:0+events22:06:341
352976599243512,2421cyclictest3581990-21kworker/2:1+events22:05:052
352976599243512,2419cyclictest3617542-21kworker/2:0+events23:54:082
352976999243212,2415cyclictest3565942-21kworker/3:0+events21:16:193
352976299243212,2416cyclictest3610855-21kworker/1:1+events23:34:041
352976999242612,2411cyclictest3549796-21kworker/3:0+events20:20:413
352976299242612,2410cyclictest3597626-21kworker/1:0+events22:51:131
352976299242511,2412cyclictest3629135-21kworker/1:0+events00:30:081
352975499242512,2409cyclictest3563170-21kworker/0:0+events21:01:250
352976299242312,2407cyclictest3533830-21kworker/1:1+events19:27:191
352976299242212,2407cyclictest3536813-21kworker/1:1+events19:59:301
352976299242211,2407cyclictest3597626-21kworker/1:0+events22:49:181
352976999242112,2407cyclictest3578841-21kworker/3:0+events21:55:063
352976299241810,2405cyclictest3576976-21kworker/1:1+events21:45:141
352976599241712,2402cyclictest3625582-21kworker/2:1+events00:22:102
352976599241312,2397cyclictest3578754-21kworker/2:1+events21:49:022
352976999241112,2395cyclictest3618752-21kworker/3:2+events00:19:083
352976999241110,2397cyclictest3584919-21kworker/3:1+events22:09:133
352975499241111,2396cyclictest3630572-21kworker/0:0+events00:30:530
352976999240911,2394cyclictest3543527-21kworker/3:0+events20:02:103
352976999240811,2395cyclictest3578841-21kworker/3:0+events21:50:083
352976999240811,2395cyclictest3578841-21kworker/3:0+events21:50:083
352976999240712,2391cyclictest3551478-21kworker/3:2+events20:29:543
352976999240712,2391cyclictest3522993-21kworker/3:2+events19:31:513
352976999240512,2390cyclictest3588125-21kworker/3:1+events22:47:493
352975499240511,2390cyclictest3554968-21kworker/0:2+events21:13:140
352976299240311,2389cyclictest3572415-21kworker/1:2+events21:30:261
35297699924028,2390cyclictest3551478-21kworker/3:2+events20:39:473
352976299240212,2386cyclictest3591483-21kworker/1:0+events22:32:101
352976599240112,2385cyclictest3563040-21kworker/2:2+events21:18:142
352976599240111,2386cyclictest3610980-21kworker/2:2+events23:33:222
352976299240112,2386cyclictest3581869-21kworker/1:1+events21:58:551
352976299240112,2385cyclictest3607278-21kworker/1:0+events23:19:191
352976999240012,2384cyclictest3618752-21kworker/3:2+events00:13:193
352976299240011,2385cyclictest3580262-21kworker/1:0+events22:04:411
352976599239911,2385cyclictest3609023-21kworker/2:1+events23:38:362
352976299239811,2383cyclictest3539861-21kworker/1:0+events19:54:511
352976299239811,2383cyclictest3539861-21kworker/1:0+events19:54:511
352976999239712,2381cyclictest3536874-21kworker/3:2+events19:47:563
35297659923969,2383cyclictest3539986-21kworker/2:0+events19:52:122
35297659923969,2383cyclictest3539986-21kworker/2:0+events19:52:122
352976599239612,2381cyclictest3594699-21kworker/2:0+events22:45:332
352976999239512,2380cyclictest3588125-21kworker/3:1+events22:44:483
352976999239512,2379cyclictest3629002-21kworker/3:0+events00:31:413
352976999239511,2380cyclictest3546765-21kworker/3:3+events20:14:423
352976999239510,2382cyclictest3533860-21kworker/3:1+events19:37:113
35297659923959,2382cyclictest3539986-21kworker/2:0+events19:49:532
352976599239511,2380cyclictest3591422-21kworker/2:2+events22:35:402
352976299239512,2379cyclictest3602981-21kworker/1:2+events23:09:051
352976299239410,2380cyclictest3627275-21kworker/1:1+events00:36:421
352976999239312,2378cyclictest3557950-21kworker/3:0+events20:48:563
352976299239311,2378cyclictest3585069-21kworker/1:2+events22:13:051
352976999239212,2376cyclictest3629002-21kworker/3:0+events00:26:203
352975499239211,2378cyclictest3541920-21kworker/0:2+events20:08:390
352976599239111,2376cyclictest3543189-21kworker/2:1+events20:30:592
352976299239112,2376cyclictest3553563-21kworker/1:2+events20:31:221
352976999239012,2374cyclictest3571035-21kworker/3:3+events21:26:593
352976999239011,2375cyclictest3571035-21kworker/3:3+events21:34:583
352976999238912,2373cyclictest3575530-21kworker/3:0+events21:42:093
352976999238912,2373cyclictest3565942-21kworker/3:0+events21:12:063
352976299238910,2376cyclictest3546575-21kworker/1:0+events20:26:491
352976299238812,2372cyclictest3563128-21kworker/1:2+events21:16:421
352976999238712,2372cyclictest3557950-21kworker/3:0+events20:53:263
352976999238711,2373cyclictest3522993-21kworker/3:2+events19:21:593
352976999238512,2370cyclictest3549796-21kworker/3:0+events20:16:323
352976999238412,2369cyclictest3540078-21kworker/3:1+events20:05:143
35297699923838,2371cyclictest3612604-21kworker/3:2+events23:43:053
35297699923838,2371cyclictest3612604-21kworker/3:2+events23:43:053
352975499238012,2365cyclictest3624239-21kworker/0:2+events00:11:450
352975499237612,2361cyclictest3617756-21kworker/0:0+events00:09:280
352975499237512,2361cyclictest3584328-21kworker/0:0+events22:40:590
352975499236912,2354cyclictest3524735-21kworker/0:0+events19:36:490
352975499236812,2352cyclictest3580414-21kworker/0:2+events21:59:420
352976599236312,2347cyclictest3575562-21kworker/2:2+events21:59:202
352976299236111,2346cyclictest3585069-21kworker/1:2+events22:27:321
352976299235711,2342cyclictest3567608-21kworker/1:1+events21:29:401
352975499235612,2340cyclictest3559641-21kworker/0:1+events20:51:570
352976599235512,2340cyclictest3561411-21kworker/2:0+events20:56:262
352975499235312,2337cyclictest3617756-21kworker/0:0+events00:17:350
352976599235212,2337cyclictest3531249-21kworker/2:1+events19:32:142
352976299235212,2336cyclictest3536813-21kworker/1:1+events19:42:131
352975499235212,2337cyclictest3586585-21kworker/0:1+events22:20:400
352976299235011,2336cyclictest3591483-21kworker/1:0+events22:38:431
352975499235012,2334cyclictest3573866-21kworker/0:1+events21:36:530
352976999234812,2333cyclictest3610948-21kworker/3:1+events23:30:423
35297659923479,2334cyclictest3556172-21kworker/2:0+events20:37:072
352976599234712,2332cyclictest3607399-21kworker/2:0+events23:26:322
352976299234412,2329cyclictest3563128-21kworker/1:2+events21:05:151
352976299234312,2328cyclictest3550015-21kworker/1:2+events20:21:281
352975499234212,2327cyclictest3557737-21kworker/0:0+events20:44:460
352976599234112,2325cyclictest3599405-21kworker/2:1+events23:12:082
352976599234112,2325cyclictest3534099-21kworker/2:2+events19:25:462
352975499234110,2328cyclictest3553233-21kworker/0:0+events20:31:460
352976599234012,2325cyclictest3621009-21kworker/2:3+events00:06:222
352976299234012,2324cyclictest3576976-21kworker/1:1+events21:51:171
352976299234012,2324cyclictest3576976-21kworker/1:1+events21:51:171
352976299234010,2326cyclictest3617661-21kworker/1:1+events23:54:531
352976599233912,2323cyclictest3619172-21kworker/2:2+events00:04:032
352976599233912,2323cyclictest3551418-21kworker/2:2+events20:44:012
352976599233911,2325cyclictest3601376-21kworker/2:0+events23:02:372
352976299233911,2324cyclictest3531885-21kworker/1:2+events19:20:291
352976599233812,2322cyclictest3563040-21kworker/2:2+events21:05:592
352976599233812,2322cyclictest3536782-21kworker/2:2+events19:57:572
352976299233812,2322cyclictest3553563-21kworker/1:2+events20:40:321
352975499233812,2322cyclictest3524735-21kworker/0:0+events19:26:090
352976599233612,2321cyclictest3531249-21kworker/2:1+events19:42:572
352976599233610,2322cyclictest3617542-21kworker/2:0+events23:56:502
352976299233612,2321cyclictest3589878-21kworker/1:1+events23:00:441
352976299233611,2322cyclictest3608933-21kworker/1:2+events23:27:191
352976299233510,2322cyclictest3556142-21kworker/1:0+events20:57:571
352976999233211,2317cyclictest3522993-21kworker/3:2+events19:18:343
352976599233212,2316cyclictest3575562-21kworker/2:2+events22:04:182
352976299233112,2315cyclictest3610855-21kworker/1:1+events23:36:191
352975499233112,2316cyclictest3597971-21kworker/0:1+events23:05:140
352976299232910,2316cyclictest3516808-21kworker/1:0+events19:32:361
35297659923289,2316cyclictest3609023-21kworker/2:1+events23:24:152
352976299232612,2311cyclictest3561441-21kworker/1:1+events21:03:441
352975499232512,2309cyclictest3541920-21kworker/0:2+events20:12:280
352976599232411,2309cyclictest3588218-21kworker/2:1+events22:18:432
352976599232410,2309cyclictest3633904-21kworker/2:0+events00:39:262
352975499232412,2309cyclictest3524735-21kworker/0:0+events19:20:520
352975499232412,2308cyclictest3554968-21kworker/0:2+events21:22:030
352976599232312,2308cyclictest3588218-21kworker/2:1+events22:28:192
352975499232311,2309cyclictest3582021-21kworker/0:1+events22:03:340
352976599232212,2306cyclictest3628971-21kworker/2:2+events00:27:052
35297549923219,2309cyclictest3541920-21kworker/0:2+events20:26:020
352975499232112,2305cyclictest3554968-21kworker/0:2+events21:05:380
352976599232012,2304cyclictest3551418-21kworker/2:2+events20:51:122
352975499232011,2305cyclictest3539891-21kworker/0:0+events19:56:230
352975499231912,2304cyclictest3580414-21kworker/0:2+events22:05:270
352976599231710,2304cyclictest3559384-21kworker/2:1+events21:04:302
352976299231612,2300cyclictest3614175-21kworker/1:2+events23:44:371
352976299231612,2300cyclictest3614175-21kworker/1:2+events23:44:361
352976299231611,2301cyclictest3563128-21kworker/1:2+events21:21:401
352976599231412,2298cyclictest3563040-21kworker/2:2+events21:29:182
352976999231312,2298cyclictest3530844-21kworker/3:1+events19:14:253
352976599231311,2299cyclictest3575562-21kworker/2:2+events21:41:472
352976599231311,2298cyclictest3575562-21kworker/2:2+events21:38:482
352976299231312,2298cyclictest3629135-21kworker/1:0+events00:27:271
352975499231212,2297cyclictest3597971-21kworker/0:1+events23:10:350
352976299231012,2295cyclictest3607278-21kworker/1:0+events23:21:121
35297659923089,2296cyclictest3591422-21kworker/2:2+events22:33:202
352976599230812,2293cyclictest3607399-21kworker/2:0+events23:18:572
352976599230810,2295cyclictest3597535-21kworker/2:2+events22:54:172
352976299230812,2293cyclictest3546575-21kworker/1:0+events20:12:061
352976299230712,2292cyclictest3624053-21kworker/1:2+events00:22:331
352976299230510,2292cyclictest3539861-21kworker/1:0+events19:47:331
352976599230011,2286cyclictest3621009-21kworker/2:3+events00:11:212
352976299228611,2271cyclictest3615793-21kworker/1:0+events23:47:401
352976299228611,2271cyclictest3615793-21kworker/1:0+events23:47:401
352975499226812,2253cyclictest3628698-21kworker/0:2+events00:23:170
35297629922619,2249cyclictest3543343-21kworker/1:2+events20:01:471
352975499224612,2231cyclictest3628698-21kworker/0:2+events00:38:400
352976599224110,2227cyclictest3559384-21kworker/2:1+events20:48:092
352976299223010,2216cyclictest3556142-21kworker/1:0+events20:39:031
352975499222411,2210cyclictest3551572-21kworker/0:1+events20:24:300
35297699922219,2209cyclictest3561473-21kworker/3:1+events21:00:173
352976299221712,2202cyclictest3529763-21kworker/1:1+events19:13:151
352975499221311,2199cyclictest3590236-21kworker/0:2+events22:50:500
352976299219912,2184cyclictest3620507-21kworker/1:0+events00:01:241
35297699921938,2182cyclictest3604132-21kworker/3:1+events23:08:423
352975499219211,2178cyclictest3554968-21kworker/0:2+events21:31:330
352975499218312,2167cyclictest3554968-21kworker/0:2+events21:27:220
352976999218110,2168cyclictest3573934-21kworker/3:1+events21:36:303
352976599214111,2127cyclictest3599405-21kworker/2:1+events23:07:342
35297659921348,2123cyclictest3546424-21kworker/2:0+events20:07:082
352975499212912,2113cyclictest3575470-21kworker/0:0+events21:54:200
352975499212912,2113cyclictest3575470-21kworker/0:0+events21:54:200
352975499212711,2113cyclictest3605706-21kworker/0:2+events23:27:410
352975499212511,2111cyclictest3605706-21kworker/0:2+events23:23:310
35297549921219,2108cyclictest3536079-21kworker/0:1+events19:47:090
352976599212011,2106cyclictest3583648-21kworker/2:0+events22:22:352
352976299211811,2104cyclictest3614175-21kworker/1:2+events23:59:551
352975499211811,2104cyclictest3536079-21kworker/0:1+events19:40:390
35297659921168,2105cyclictest3531249-21kworker/2:1+events19:16:412
352976599211611,2101cyclictest3543189-21kworker/2:1+events20:21:052
352975499211611,2102cyclictest3586585-21kworker/0:1+events22:13:510
352975499211611,2101cyclictest3584328-21kworker/0:0+events22:16:290
35297659921158,2104cyclictest3531249-21kworker/2:1+events19:37:572
352975499211511,2101cyclictest3617756-21kworker/0:0+events00:00:400
352976599211311,2099cyclictest3563040-21kworker/2:2+events21:24:212
35297659921059,2093cyclictest3628971-21kworker/2:2+events00:32:262
352976599210510,2092cyclictest3583648-21kworker/2:0+events22:11:092
35297629921058,2094cyclictest3620507-21kworker/1:0+events00:08:411
35297629921039,2091cyclictest3516808-21kworker/1:0+events19:15:331
352976299210111,2087cyclictest3563128-21kworker/1:2+events21:12:291
352976299209811,2084cyclictest3556142-21kworker/1:0+events20:48:321
35297659920729,2060cyclictest3609023-21kworker/2:1+events23:48:272
35297659920729,2060cyclictest3609023-21kworker/2:1+events23:48:262
352976599204211,2028cyclictest3526156-21kworker/2:2+events19:12:292
35297549918819,1868cyclictest3519802-21kworker/0:2+events19:10:540
35291712196168,23sleep10-21swapper/119:05:491
352975499156151,3cyclictest3551572-21kworker/0:1+pm20:39:590
3528424214692,48sleep00-21swapper/019:05:160
35294662144113,25sleep30-21swapper/319:08:553
35294852142112,25sleep20-21swapper/219:09:112
3529754996966,2cyclictest3554968-21kworker/0:2+pm20:49:540
3529765993635,1cyclictest3531249-21kworker/2:1+pm19:24:512
3529754993022,6cyclictest1165-21gmain19:55:000
3529754993022,6cyclictest1165-21gmain19:55:000
352975499287,16cyclictest3615850-21apt-get23:45:010
352975499287,16cyclictest3615850-21apt-get23:45:000
352975499255,12cyclictest3611354-21cat23:30:120
352975499251,20cyclictest3594759-21readlink22:40:010
3529765992310,8cyclictest3548350-21systemctl20:15:002
3529754992311,10cyclictest1-21systemd21:50:000
352975499221,18cyclictest3620856-21seq00:00:010
352976599219,8cyclictest0-21swapper/220:15:022
352975499211,17cyclictest3602938-21runrttasks23:05:000
352975499211,17cyclictest3569419-21munin-run21:20:000
352975499211,16cyclictest593-21dbus-daemon19:20:010
352975499211,16cyclictest3536354-21idleruntime-cro19:35:000
352975499211,15cyclictest0-21swapper/023:45:020
352975499211,15cyclictest0-21swapper/023:45:020
352975499201,15cyclictest3563088-21munin-run21:00:000
352975499200,18cyclictest3613493-21ntpq23:35:170
352975499191,15cyclictest3597913-21dump-pmu-power22:50:000
352975499190,1cyclictest0-21swapper/023:50:220
352975499190,16cyclictest3601346-21readlink23:00:010
352976599181,12cyclictest3596340-21munin-run22:45:002
352975499181,14cyclictest3545170-21awk20:05:000
352976599172,5cyclictest3552541-21ld20:25:172
352975499171,13cyclictest3592050-21df_inode22:30:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional