You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-09 - 21:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackfslot7s.osadl.org (updated Tue Dec 09, 2025 00:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
369084999295115,2931cyclictest3754342-21kworker/2:0+events22:35:162
369085299281011,2793cyclictest3756064-21kworker/3:1+events22:45:123
369084299280513,2788cyclictest3702192-21kworker/0:2+events20:05:190
369085299279411,2779cyclictest3746859-21kworker/3:1+events22:10:183
369084799274913,2731cyclictest3735780-21kworker/1:1+events21:55:201
369084999270012,2684cyclictest3703585-21kworker/2:2+events20:00:202
369084999265413,2638cyclictest3733844-21kworker/2:1+events21:30:172
369084999254811,2534cyclictest3733844-21kworker/2:1+events21:25:002
369084999254412,2530cyclictest3728053-21kworker/2:0+events21:10:062
369084999254412,2530cyclictest3728053-21kworker/2:0+events21:10:052
369085299253011,2515cyclictest3685457-21kworker/3:0+events19:20:213
369085299252912,2513cyclictest3694576-21kworker/3:1+events19:25:173
369084299247211,2456cyclictest3737133-21kworker/0:2+events21:35:350
369084299246112,2446cyclictest3738739-21kworker/0:1+events21:41:430
369084299246012,2445cyclictest3722432-21kworker/0:2+events20:46:230
369084999245112,2436cyclictest3788954-21kworker/2:2+events00:20:212
369084999245112,2435cyclictest3748335-21kworker/2:2+events22:15:142
369084299244712,2432cyclictest3702192-21kworker/0:2+events19:52:200
369084299244511,2431cyclictest3702192-21kworker/0:2+events19:56:080
369084999244310,2429cyclictest3741899-21kworker/2:0+events22:12:142
369085299244212,2426cyclictest3727358-21kworker/3:1+events21:06:163
369085299244111,2427cyclictest3710014-21kworker/3:1+events20:19:513
369084999244011,2426cyclictest3719481-21kworker/2:2+events20:39:132
369085299243912,2423cyclictest3685457-21kworker/3:0+events19:18:053
369085299243911,2424cyclictest3790525-21kworker/3:2+events00:35:333
369084999243712,2421cyclictest3695939-21kworker/2:1+events19:38:352
369085299243612,2420cyclictest3746859-21kworker/3:1+events22:26:113
36908529924349,2421cyclictest3729279-21kworker/3:0+events21:13:113
36908529924349,2421cyclictest3729279-21kworker/3:0+events21:13:113
369084999242811,2414cyclictest3757541-21kworker/2:2+events22:40:152
369085299241012,2394cyclictest3730884-21kworker/3:2+events21:17:473
369085299241010,2397cyclictest3746859-21kworker/3:1+events22:16:463
369084999241012,2395cyclictest3788954-21kworker/2:2+events00:25:152
369084999240711,2393cyclictest3695939-21kworker/2:1+events19:40:302
369084299239912,2381cyclictest3793641-21kworker/0:1+events00:31:000
369084299238710,2373cyclictest3732190-21kworker/0:0+events21:17:010
369084299237210,2359cyclictest3788661-21kworker/0:2+events00:14:000
369084999236510,2352cyclictest3738586-21kworker/2:0+events21:39:252
369084999235910,2345cyclictest3741899-21kworker/2:0+events21:52:382
369085299234812,2334cyclictest3775748-21kworker/3:1+events23:45:243
369085299234112,2326cyclictest3720951-21kworker/3:2+events20:50:593
369084999232811,2314cyclictest3714526-21kworker/2:2+events20:30:522
369084299232712,2312cyclictest3749760-21kworker/0:0+events22:36:010
369084299232111,2307cyclictest3749760-21kworker/0:0+events22:13:460
36908529923199,2306cyclictest3735381-21kworker/3:0+events21:26:533
369084799230712,2293cyclictest3700294-21kworker/1:0+events19:40:061
36908499922897,2278cyclictest3728053-21kworker/2:0+events21:05:532
369084999228611,2272cyclictest3757541-21kworker/2:2+events22:48:382
36908529922775,2269cyclictest3714913-21kworker/3:0+events20:37:423
369084299227011,2256cyclictest3697302-21kworker/0:2+events19:32:090
369084999226510,2253cyclictest3787357-21kworker/2:1+events00:15:052
369084999226012,2245cyclictest3745135-21kworker/2:1+events22:09:092
369084999225811,2244cyclictest3763822-21kworker/2:2+events22:58:552
369085299225711,2243cyclictest3741930-21kworker/3:0+events21:51:073
369084999225710,2244cyclictest3719481-21kworker/2:2+events20:52:552
36908499922545,2246cyclictest3785721-21kworker/2:2+events00:09:022
369084999225411,2240cyclictest3714526-21kworker/2:2+events20:22:072
369084999225410,2240cyclictest3750174-21kworker/2:1+events22:34:302
369085299225011,2236cyclictest3702221-21kworker/3:0+events19:55:203
369084299225010,2237cyclictest3763940-21kworker/0:1+events23:06:060
369085299224911,2235cyclictest3765533-21kworker/3:2+events23:21:453
369084299224910,2235cyclictest3698991-21kworker/0:1+events19:40:530
369085299224811,2234cyclictest3720951-21kworker/3:2+events20:45:593
369084999224711,2233cyclictest3783778-21kworker/2:0+events00:14:202
369084999224512,2230cyclictest3714526-21kworker/2:2+events20:27:492
369084999224511,2231cyclictest3772582-21kworker/2:2+events23:56:522
36908529922399,2227cyclictest3730884-21kworker/3:2+events21:38:143
369084999223910,2226cyclictest3790818-21kworker/2:0+events00:35:542
36908499922389,2226cyclictest3721130-21kworker/2:1+events20:44:522
369084299223811,2223cyclictest3722432-21kworker/0:2+events20:50:350
369085299223611,2222cyclictest3793610-21kworker/3:0+events00:30:353
369084299223610,2223cyclictest3774383-21kworker/0:0+events23:48:280
36908529922349,2222cyclictest3775748-21kworker/3:1+events23:33:543
369084299223410,2222cyclictest3774383-21kworker/0:0+events23:40:050
369085299223311,2219cyclictest3782210-21kworker/3:0+events00:09:233
369084299223311,2218cyclictest3749760-21kworker/0:0+events22:21:400
369085299223211,2217cyclictest3745168-21kworker/3:0+events22:08:243
369085299223211,2217cyclictest3743557-21kworker/3:1+events21:55:433
369084999223210,2218cyclictest3772582-21kworker/2:2+events23:50:472
36908429922327,2222cyclictest3714820-21kworker/0:2+events20:27:270
369085299223110,2218cyclictest3756064-21kworker/3:1+events22:39:063
369085299223010,2217cyclictest3788924-21kworker/3:1+events00:22:353
369085299223010,2217cyclictest3780611-21kworker/3:2+events23:53:253
369085299223010,2217cyclictest3756064-21kworker/3:1+events23:00:243
369084299223011,2216cyclictest3729206-21kworker/0:2+events21:08:350
369085299222911,2215cyclictest3752804-21kworker/3:2+events22:24:413
369085299222911,2215cyclictest3717305-21kworker/3:2+events20:33:093
369085299222910,2216cyclictest3756064-21kworker/3:1+events22:41:233
369085299222910,2216cyclictest3752804-21kworker/3:2+events22:31:093
369084999222910,2216cyclictest3765504-21kworker/2:1+events23:06:292
369084999222910,2216cyclictest3763822-21kworker/2:2+events23:00:472
369084999222910,2216cyclictest3724334-21kworker/2:1+events20:55:342
369084299222910,2216cyclictest3735471-21kworker/0:0+events21:49:380
369085299222811,2214cyclictest3767129-21kworker/3:0+events23:17:343
36908429922289,2216cyclictest3772523-21kworker/0:1+events23:26:430
369084299222810,2214cyclictest3711532-21kworker/0:0+events20:31:380
369085299222711,2213cyclictest3759513-21kworker/3:2+events22:54:453
369084999222711,2213cyclictest3776064-21kworker/2:0+events23:36:362
369084999222710,2214cyclictest3703585-21kworker/2:2+events19:56:312
369084299222711,2213cyclictest3711532-21kworker/0:0+events20:24:240
369084299222710,2214cyclictest3749760-21kworker/0:0+events22:47:040
369084299222710,2214cyclictest3733813-21kworker/0:1+events21:27:170
369084299222710,2214cyclictest3718078-21kworker/0:1+events21:01:430
369085299222511,2212cyclictest3730884-21kworker/3:2+events21:45:073
369084299222510,2212cyclictest3751416-21kworker/0:2+events22:28:530
36908529922249,2212cyclictest3788924-21kworker/3:1+events00:18:253
36908529922248,2213cyclictest3730884-21kworker/3:2+events21:31:023
36908529922246,2214cyclictest3765533-21kworker/3:2+events23:07:373
369085299222410,2211cyclictest3732517-21kworker/3:1+events21:22:433
369084299222411,2211cyclictest3698991-21kworker/0:1+events19:35:090
369084299222410,2211cyclictest3749760-21kworker/0:0+events22:57:230
36908499922235,2214cyclictest3710077-21kworker/2:2+events20:14:052
369084999222311,2208cyclictest3769025-21kworker/2:1+events23:24:482
369084999222310,2210cyclictest3772582-21kworker/2:2+events23:40:512
369084299222310,2210cyclictest3711532-21kworker/0:0+events20:14:510
36908499922227,2211cyclictest3769025-21kworker/2:1+events23:18:202
369084999222211,2207cyclictest3750174-21kworker/2:1+events22:22:462
369084299222211,2207cyclictest3769058-21kworker/0:2+events23:21:000
36908499922218,2212cyclictest3732218-21kworker/2:0+events21:26:292
36908429922219,2209cyclictest3769058-21kworker/0:2+events23:30:320
369084299222110,2208cyclictest3732190-21kworker/0:0+events21:20:240
36908529922207,2210cyclictest3702221-21kworker/3:0+events19:49:183
369085299222011,2206cyclictest3710014-21kworker/3:1+events20:29:423
369084999222010,2207cyclictest3733844-21kworker/2:1+events21:44:442
369085299221910,2205cyclictest3698847-21kworker/3:1+events19:42:253
36908499922188,2207cyclictest3705187-21kworker/2:1+events20:07:132
369084299221810,2205cyclictest3749760-21kworker/0:0+events22:44:280
369084299221810,2205cyclictest3708324-21kworker/0:0+events20:03:240
369084299221610,2203cyclictest3718078-21kworker/0:1+events20:59:490
369084299221310,2200cyclictest3769058-21kworker/0:2+events23:18:410
369085299221110,2197cyclictest3714913-21kworker/3:0+events20:40:423
369085299220810,2196cyclictest3735381-21kworker/3:0+events21:40:123
369084999220810,2195cyclictest3695939-21kworker/2:1+events19:45:512
369085299220711,2193cyclictest3724157-21kworker/3:0+events20:59:033
369085299220511,2191cyclictest3787326-21kworker/3:2+events00:13:353
369084999220410,2191cyclictest3759406-21kworker/2:0+events23:11:042
36908529922039,2191cyclictest3765533-21kworker/3:2+events23:14:523
369084299220310,2190cyclictest3748006-21kworker/0:1+events22:08:460
369084299220310,2190cyclictest3741991-21kworker/0:1+events21:59:100
369084299220310,2190cyclictest3729418-21kworker/0:3+events21:14:430
369084299220310,2190cyclictest3729418-21kworker/0:3+events21:14:430
36908529922029,2191cyclictest3702221-21kworker/3:0+events20:01:053
369085299220210,2189cyclictest3774082-21kworker/3:0+events23:27:273
369085299220111,2187cyclictest3720951-21kworker/3:2+events21:02:533
369084299219811,2184cyclictest3702192-21kworker/0:2+events19:46:140
36908529921928,2181cyclictest3685457-21kworker/3:0+events19:32:313
369084299218710,2174cyclictest3745247-21kworker/0:0+events22:01:290
369085299218410,2171cyclictest3691228-21kworker/3:2+events19:14:393
36908429921798,2168cyclictest3749760-21kworker/0:0+events22:19:000
369084999215710,2143cyclictest3730615-21kworker/2:2+events21:19:162
369084999214610,2133cyclictest3741899-21kworker/2:0+events21:57:382
36908429921396,2130cyclictest3685517-21kworker/0:0+events19:16:560
369084999213810,2125cyclictest3785721-21kworker/2:2+events00:04:462
369084299213610,2123cyclictest3711532-21kworker/0:0+events20:40:190
36908499921328,2120cyclictest3719481-21kworker/2:2+events21:00:362
36908499921317,2121cyclictest3750174-21kworker/2:1+events22:25:502
36908529921199,2107cyclictest3700447-21kworker/3:2+events19:54:363
369085299211810,2104cyclictest3782210-21kworker/3:0+events00:03:403
369084999211810,2105cyclictest3703585-21kworker/2:2+events19:54:582
369084799211711,2103cyclictest3708205-21kworker/1:1+events20:36:551
36908429921179,2105cyclictest3783896-21kworker/0:1+events00:06:420
369084999211310,2100cyclictest3721130-21kworker/2:1+events20:48:412
36908429921139,2101cyclictest3685517-21kworker/0:0+events19:21:520
36908429921129,2100cyclictest3776010-21kworker/0:1+events23:35:470
369084999211010,2096cyclictest3688847-21kworker/2:0+events19:27:352
36908429921088,2097cyclictest3782298-21kworker/0:2+events23:56:270
36908429921058,2094cyclictest3783896-21kworker/0:1+events00:17:160
36908429921040,2101cyclictest3774383-21kworker/0:0+events23:54:320
369084999210010,2087cyclictest3695939-21kworker/2:1+events19:31:012
369084999210010,2087cyclictest3688847-21kworker/2:0+events19:13:092
36908429921007,2090cyclictest3791999-21kworker/0:2+events00:38:310
36908499920926,2083cyclictest3733844-21kworker/2:1+events21:46:592
36908479920919,2079cyclictest3767099-21kworker/1:1+events23:10:411
36908529920899,2077cyclictest3777376-21kworker/3:0+events23:38:313
369084299208910,2076cyclictest3685517-21kworker/0:0+events19:26:020
369084799208810,2075cyclictest3719358-21kworker/1:0+events21:30:391
36908529920859,2073cyclictest3782210-21kworker/3:0+events23:56:063
36908529920859,2073cyclictest3702221-21kworker/3:0+events20:13:423
369084999208410,2071cyclictest3795001-21kworker/2:1+events00:34:462
369084999208210,2069cyclictest3780516-21kworker/2:0+events23:49:362
369084999208210,2069cyclictest3759406-21kworker/2:0+events22:50:322
36908429920826,2073cyclictest3741991-21kworker/0:1+events21:54:550
36908529920818,2070cyclictest3775748-21kworker/3:1+events23:40:273
36908499920816,2072cyclictest3745135-21kworker/2:1+events22:00:452
36908529920806,2071cyclictest3698847-21kworker/3:1+events19:39:443
369084299207910,2066cyclictest3749760-21kworker/0:0+events22:52:500
369084299207910,2066cyclictest3711532-21kworker/0:0+events20:37:190
36908479920770,2074cyclictest3708205-21kworker/1:1+events20:17:091
369084299207610,2063cyclictest3783896-21kworker/0:1+events00:00:560
369085299207510,2062cyclictest3710014-21kworker/3:1+events20:09:043
369084299207510,2061cyclictest3790313-21kworker/0:0+events00:21:040
369084299207310,2060cyclictest3791999-21kworker/0:2+events00:27:330
369084299206910,2056cyclictest3754641-21kworker/0:1+events22:32:160
36908479920688,2057cyclictest3748442-21kworker/1:2+events23:07:151
36908529920677,2057cyclictest3763913-21kworker/3:0+events22:58:083
36908499920679,2055cyclictest3769025-21kworker/2:1+events23:27:052
36908429920679,2055cyclictest3767196-21kworker/0:0+events23:13:410
36908429920678,2056cyclictest3709926-21kworker/0:1+events20:15:380
369084999206610,2053cyclictest3772582-21kworker/2:2+events23:32:272
369084999206610,2053cyclictest3705187-21kworker/2:1+events20:18:202
36908529920609,2048cyclictest3710014-21kworker/3:1+events20:23:173
36908429920567,2047cyclictest3763940-21kworker/0:1+events23:00:030
36908499920498,2037cyclictest3688847-21kworker/2:0+events19:16:342
36908429920379,2025cyclictest3735471-21kworker/0:0+events21:33:170
36908429920239,2011cyclictest3692006-21kworker/0:1+events19:14:170
36908499920229,2010cyclictest3688847-21kworker/2:0+events19:22:162
369085299201310,2000cyclictest3745168-21kworker/3:0+events22:01:083
36908529920007,1990cyclictest3793610-21kworker/3:0+events00:29:503
36908479919956,1986cyclictest3774693-21kworker/1:2+events23:35:241
36908479919619,1949cyclictest3675508-21kworker/1:1+events19:35:541
369084799195910,1946cyclictest3702072-21kworker/1:2+events19:48:101
369084799195410,1941cyclictest3675508-21kworker/1:1+events19:50:261
369084799195410,1940cyclictest3713046-21kworker/1:0+events20:21:201
369084799194910,1936cyclictest3784136-21kworker/1:2+events00:05:321
36908479919269,1914cyclictest3748442-21kworker/1:2+events23:15:141
36908479919049,1892cyclictest3767099-21kworker/1:1+events23:33:331
36908479918837,1873cyclictest3705455-21kworker/1:0+events20:00:421
36908479918832,1878cyclictest3787011-21kworker/1:0+events00:15:511
36908479918528,1841cyclictest3743858-21kworker/1:0+events22:11:501
36905242160121,33sleep30-21swapper/319:08:193
36905592147112,29sleep10-21swapper/119:08:441
36905482145111,29sleep00-21swapper/019:08:350
36905082143107,30sleep20-21swapper/219:08:052
3690847992614,6cyclictest3756279-10awk22:34:591
369084799203,7cyclictest3761776-21ntpq22:50:191
369084799201,6cyclictest3758426-21irqstats22:40:161
369084799200,16cyclictest3725140-21ntpq20:55:211
369084799199,6cyclictest3738876-21idleruntime-cro21:40:001
369084799193,8cyclictest3734926-21gcc21:25:181
369084799193,6cyclictest3747610-21ntpq22:05:171
3690847991916,1cyclictest0-21swapper/120:00:001
3690847991911,6cyclictest3764169-21sh22:59:591
3690847991910,6cyclictest3783951-21dpkg00:00:001
369084799188,6cyclictest3773674-21ntpq23:25:191
369084799181,14cyclictest3782371-21awk23:55:011
3690847991811,4cyclictest3739361-21sh21:40:141
369084799181,13cyclictest3724380-21munin-run20:55:001
369084799180,16cyclictest355-21systemd-journal00:27:511
369084799178,7cyclictest3720839-21/usr/sbin/munin20:40:211
369084799178,5cyclictest3796904-21run-parts00:40:001
369084799177,8cyclictest3792159-21idleruntime-cro00:25:001
369084799177,7cyclictest3726736-21ntpq21:00:191
369084799175,7cyclictest0-21swapper/119:30:141
3690847991714,2cyclictest0-21swapper/123:00:141
3690847991711,4cyclictest3729304-21chrt21:09:041
369084799171,13cyclictest3718065-21perf-exec20:35:001
369084799170,4cyclictest3794958-21chrt00:30:371
369084799169,5cyclictest3756930-21meminfo22:35:181
369084799169,5cyclictest3731205-21/usr/sbin/munin21:15:141
369084799167,6cyclictest3734149-21perf21:25:001
369084799165,5cyclictest3785639-10sshd00:02:501
3690847991611,3cyclictest3695089-21ntpq19:20:161
3690847991610,4cyclictest3730861-21runrttasks21:13:531
3690847991610,4cyclictest3730861-21runrttasks21:13:521
3690847991610,1cyclictest3715140-21/usr/sbin/munin20:25:211
369084799160,15cyclictest3691599-21/usr/sbin/munin19:10:211
369084799160,13cyclictest3777877-21/usr/sbin/munin23:40:151
369084799160,13cyclictest3760062-21/usr/sbin/munin22:45:161
369084799159,4cyclictest3741917-21taskset21:48:321
369084799156,6cyclictest3742822-21ntpq21:50:171
369084799156,6cyclictest3696704-21ntpq19:25:171
369084799154,10cyclictest3694339-21awk19:20:001
369084799153,4cyclictest3711335-21users20:10:211
369084799153,3cyclictest3779864-21meminfo23:45:171
369084799153,3cyclictest3753904-21ntpq22:25:181
3690847991511,2cyclictest602-21polkitd23:20:001
369084799150,4cyclictest3708802-21taskset20:05:071
369084799146,5cyclictest3751532-21grep22:19:591
369084799144,6cyclictest3787667-21apt-get00:10:141
3690847991411,1cyclictest3751786-21/usr/sbin/munin22:20:131
369084799140,12cyclictest0-21swapper/122:00:071
369084799136,5cyclictest3722811-21perf20:50:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional