You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-28 - 07:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackfslot7s.osadl.org (updated Sun Dec 28, 2025 00:44:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
311911799589127,5860cyclictest3115364-21kworker/2:2+events19:11:502
311911399574826,5719cyclictest3118958-21kworker/1:2+events19:11:271
311910899381718,3796cyclictest3113806-21kworker/0:2+events19:11:050
311911399333618,3314cyclictest3146289-21kworker/1:0+events20:35:191
311911399294411,2928cyclictest3170519-21kworker/1:1+events22:00:171
311911999282613,2810cyclictest3118735-21kworker/3:0+events19:12:143
311911999258011,2566cyclictest3159115-21kworker/3:2+events21:15:013
311910899251010,2496cyclictest3168830-21kworker/0:2+events21:55:200
311911399245610,2444cyclictest3202971-21kworker/1:2+events23:50:041
311911399244212,2427cyclictest3139835-21kworker/1:0+events20:20:161
311911799236612,2351cyclictest3146204-21kworker/2:1+events20:40:152
311910899236412,2349cyclictest3160893-21kworker/0:1+events21:34:450
311911999234912,2334cyclictest3196544-21kworker/3:1+events23:10:133
311910899233412,2318cyclictest3136671-21kworker/0:1+events20:15:190
311911799231211,2298cyclictest3219161-21kworker/2:1+events00:36:052
311911399230411,2290cyclictest3223823-21kworker/1:0+events00:37:341
311911799227510,2261cyclictest3120416-21kworker/2:1+events19:15:192
311911399227512,2260cyclictest3215818-21kworker/1:0+events00:09:231
311911399227512,2260cyclictest3215818-21kworker/1:0+events00:09:231
311911399227210,2258cyclictest3163681-21kworker/1:0+events21:44:141
311911399227111,2257cyclictest3183819-21kworker/1:0+events22:43:141
311911399226711,2253cyclictest3157631-21kworker/1:0+events21:10:481
31191139922669,2254cyclictest3217710-21kworker/1:1+events00:17:431
311911399226410,2251cyclictest3170519-21kworker/1:1+events22:10:531
311911799225711,2243cyclictest3166887-21kworker/2:0+events21:50:212
311911399224810,2235cyclictest3128581-21kworker/1:2+events20:07:181
31191179922415,2233cyclictest3170278-21kworker/2:1+events21:59:572
311911799223910,2227cyclictest3214333-21kworker/2:2+events00:15:052
31191089922376,2228cyclictest3194801-21kworker/0:1+events23:05:190
311910899223711,2222cyclictest3165342-21kworker/0:0+events21:45:230
311911799223611,2221cyclictest3170278-21kworker/2:1+events22:02:142
311910899222911,2215cyclictest3196396-21kworker/0:2+events23:11:210
31191179922268,2215cyclictest3199750-21kworker/2:0+events23:22:212
311910899221711,2203cyclictest3168830-21kworker/0:2+events22:04:540
311911399219011,2177cyclictest3202971-21kworker/1:2+events23:30:201
31191139921769,2164cyclictest3117253-21kworker/1:0+events19:35:191
31191089921538,2142cyclictest3160893-21kworker/0:1+events21:37:050
311911999214611,2132cyclictest3190033-21kworker/3:1+events22:54:203
311911999213611,2122cyclictest3139715-21kworker/3:2+events20:22:093
31191089921199,2107cyclictest3204846-21kworker/0:1+events23:40:140
311911399211211,2098cyclictest3170519-21kworker/1:1+events21:54:121
311911799211010,2097cyclictest3210416-21kworker/2:2+events23:51:342
31191089921085,2100cyclictest3199688-21kworker/0:2+events23:19:440
31191139921079,2095cyclictest3152749-21kworker/1:1+events21:06:591
311910899210611,2092cyclictest3201373-21kworker/0:0+events23:47:010
31191139921039,2091cyclictest3181929-21kworker/1:2+events22:27:001
311911399210310,2089cyclictest3157631-21kworker/1:0+events21:15:211
311911799210210,2088cyclictest3148179-21kworker/2:0+events21:14:162
31191139921028,2091cyclictest3180153-21kworker/1:1+events22:31:561
31191139921027,2091cyclictest3146289-21kworker/1:0+events20:43:211
311911399210110,2087cyclictest3199659-21kworker/1:0+events23:25:251
31191139920997,2090cyclictest3202971-21kworker/1:2+events23:45:091
31191139920988,2087cyclictest3214153-21kworker/1:2+events00:04:481
31191139920986,2089cyclictest3117253-21kworker/1:0+events19:28:031
311910899209710,2084cyclictest3142915-21kworker/0:2+events20:23:400
311911799209610,2082cyclictest3180310-21kworker/2:1+events22:23:572
31191199920955,2088cyclictest3219255-21kworker/3:0+events00:20:043
31191139920909,2078cyclictest3163681-21kworker/1:0+events21:26:511
31191089920849,2072cyclictest3168830-21kworker/0:2+events22:18:360
31191089920849,2072cyclictest3142915-21kworker/0:2+events20:31:110
311911399208310,2070cyclictest3183819-21kworker/1:0+events22:58:311
31191139920798,2068cyclictest3199659-21kworker/1:0+events23:18:331
31191139920783,2072cyclictest3163681-21kworker/1:0+events21:48:061
31191179920777,2067cyclictest3124187-21kworker/2:0+events19:29:142
31191179920767,2066cyclictest3133565-21kworker/2:0+events20:01:352
311910899207610,2063cyclictest3168830-21kworker/0:2+events22:06:470
311910899207610,2063cyclictest3168830-21kworker/0:2+events22:06:470
311911799207510,2062cyclictest3182000-21kworker/2:0+events22:28:292
31191089920757,2066cyclictest3160893-21kworker/0:1+events21:40:060
311911799207410,2061cyclictest3152686-21kworker/2:1+events20:56:592
31191179920734,2065cyclictest3148179-21kworker/2:0+events21:02:212
311911399207310,2060cyclictest3180153-21kworker/1:1+events22:18:121
311910899207310,2060cyclictest3168830-21kworker/0:2+events22:24:420
31191139920726,2063cyclictest3183819-21kworker/1:0+events22:47:271
31191179920719,2058cyclictest3128612-21kworker/2:1+events19:55:562
311911799207110,2058cyclictest3160809-21kworker/2:2+events21:31:222
31191089920715,2063cyclictest3220966-21kworker/0:2+events00:28:260
311910899207110,2058cyclictest3212678-21kworker/0:2+events00:13:330
311911799207010,2057cyclictest3204311-21kworker/2:1+events23:34:322
311911399207010,2057cyclictest3202971-21kworker/1:2+events23:43:161
31191089920708,2059cyclictest3201373-21kworker/0:0+events23:57:140
311911999206910,2055cyclictest3159115-21kworker/3:2+events21:30:153
31191139920699,2056cyclictest3214153-21kworker/1:2+events00:33:231
311911399206910,2056cyclictest3161099-21kworker/1:1+events21:21:041
31191179920689,2056cyclictest3204311-21kworker/2:1+events23:45:322
31191179920687,2058cyclictest3160809-21kworker/2:2+events21:36:422
31191179920684,2061cyclictest3144574-21kworker/2:2+events20:37:592
31191179920675,2059cyclictest3130232-21kworker/2:0+events19:42:502
31191179920674,2060cyclictest3126987-21kworker/2:2+events19:36:482
311911799206710,2055cyclictest3128612-21kworker/2:1+events19:50:322
311911799206710,2054cyclictest3136521-21kworker/2:2+events20:16:272
311911799206710,2054cyclictest3133565-21kworker/2:0+events20:27:512
311911399206710,2053cyclictest3117253-21kworker/1:0+events19:24:361
311910899206710,2054cyclictest3222612-21kworker/0:1+events00:32:590
31191179920669,2054cyclictest3191580-21kworker/2:0+events22:57:432
31191179920664,2059cyclictest3196481-21kworker/2:2+events23:13:382
31191179920658,2054cyclictest3219161-21kworker/2:1+events00:29:322
31191179920649,2052cyclictest3133565-21kworker/2:0+events20:11:502
311910899206410,2051cyclictest3209100-21kworker/0:1+events23:53:270
31191139920637,2053cyclictest3152749-21kworker/1:1+events20:54:221
311911399206310,2050cyclictest3144689-21kworker/1:2+events20:34:351
31191089920637,2053cyclictest3135018-21kworker/0:0+events20:08:480
31191089920636,2054cyclictest3130112-21kworker/0:2+events19:52:510
31191139920626,2053cyclictest3128581-21kworker/1:2+events20:16:041
31191089920617,2051cyclictest3147800-21kworker/0:1+events20:54:450
31191199920603,2054cyclictest3141315-21kworker/3:1+events20:18:443
31191179920608,2049cyclictest3175647-21kworker/2:2+events22:13:352
31191179920607,2050cyclictest3142974-21kworker/2:1+events20:24:022
311910899206010,2047cyclictest3138388-21kworker/0:3+events20:12:370
311910899206010,2047cyclictest3125674-21kworker/0:2+events19:34:330
311911799205910,2046cyclictest3175647-21kworker/2:2+events22:37:112
31191179920584,2051cyclictest3126987-21kworker/2:2+events19:33:002
31191139920586,2049cyclictest3141438-21kworker/1:1+events20:29:201
31191179920568,2045cyclictest3191580-21kworker/2:0+events22:51:202
31191089920569,2044cyclictest3181900-21kworker/0:0+events22:28:530
311910899205510,2042cyclictest3219346-21kworker/0:1+events00:22:220
31191089920545,2046cyclictest3204846-21kworker/0:1+events23:36:480
31191139920529,2040cyclictest3126950-21kworker/1:1+events19:33:461
311910899205210,2039cyclictest3222612-21kworker/0:1+events00:35:190
31191089920517,2041cyclictest3124037-21kworker/0:1+events19:29:360
31191089920508,2039cyclictest3201373-21kworker/0:0+events23:25:470
31191179920498,2038cyclictest3148179-21kworker/2:0+events20:52:082
31191179920488,2037cyclictest3194887-21kworker/2:1+events23:04:562
31191179920483,2042cyclictest3204311-21kworker/2:1+events23:37:112
311911799204810,2035cyclictest3120416-21kworker/2:1+events19:23:262
311910899204810,2035cyclictest3155813-21kworker/0:1+events21:06:120
31191089920478,2036cyclictest3217678-21kworker/0:0+events00:19:390
31191199920469,2034cyclictest3144543-21kworker/3:1+events20:44:303
31191199920464,2039cyclictest3159115-21kworker/3:2+events22:31:323
311911399204610,2033cyclictest3183819-21kworker/1:0+events23:06:041
31191089920465,2038cyclictest3190001-21kworker/0:0+events22:49:000
31191199920448,2032cyclictest3217588-21kworker/3:2+events00:13:103
31191139920436,2034cyclictest3128581-21kworker/1:2+events19:53:131
31191139920426,2033cyclictest3163681-21kworker/1:0+events21:36:181
311910899204210,2029cyclictest3168830-21kworker/0:2+events22:12:470
31191179920419,2029cyclictest3212621-21kworker/2:0+events23:59:272
31191179920408,2029cyclictest3136521-21kworker/2:2+events20:06:352
31191139920403,2034cyclictest3154367-21kworker/1:2+events20:59:391
31191199920399,2026cyclictest3209251-21kworker/3:0+events23:55:223
311911799203710,2024cyclictest3196481-21kworker/2:2+events23:15:552
31191139920378,2026cyclictest3221035-21kworker/1:1+events00:25:481
311911399203510,2022cyclictest3151138-21kworker/1:2+events20:48:391
31191179920333,2027cyclictest3175647-21kworker/2:2+events22:44:242
31191139920326,2023cyclictest3152749-21kworker/1:1+events21:01:581
311910899203110,2019cyclictest3168830-21kworker/0:2+events22:30:020
31191139920299,2017cyclictest3170519-21kworker/1:1+events22:09:231
31191139920299,2017cyclictest3170519-21kworker/1:1+events22:09:221
31191089920278,2016cyclictest3151589-21kworker/0:0+events21:24:330
311910899202710,2014cyclictest3154191-21kworker/0:2+events20:58:080
311911399202610,2013cyclictest3217710-21kworker/1:1+events00:14:441
311910899202410,2011cyclictest3199688-21kworker/0:2+events23:24:150
311910899202110,2008cyclictest3187053-21kworker/0:2+events22:41:210
31191179920197,2009cyclictest3220859-21kworker/2:0+events00:24:152
311911399201810,2005cyclictest3206239-21kworker/1:0+events23:58:421
31191179920157,2005cyclictest3175647-21kworker/2:2+events22:48:132
31191179920119,1999cyclictest3160809-21kworker/2:2+events21:27:592
311911999201010,1997cyclictest3211439-21kworker/3:2+events00:02:293
31191089920109,1999cyclictest3185279-21kworker/0:1+events22:35:180
31191179920097,1998cyclictest3146204-21kworker/2:1+events20:34:132
31191139920099,1997cyclictest3219315-21kworker/1:0+events00:23:051
31191139920099,1997cyclictest3183819-21kworker/1:0+events22:36:051
31191139920089,1996cyclictest3128581-21kworker/1:2+events20:01:131
31191139920083,2002cyclictest3197936-21kworker/1:1+events23:11:421
31191199920074,2000cyclictest3185217-21kworker/3:0+events22:45:113
31191089920079,1995cyclictest3190001-21kworker/0:0+events23:04:100
31191179920069,1994cyclictest3204311-21kworker/2:1+events00:09:012
31191179920069,1994cyclictest3204311-21kworker/2:1+events00:09:012
31191179920065,1998cyclictest3175647-21kworker/2:2+events22:19:482
311911799200610,1993cyclictest3185010-21kworker/2:1+events22:32:172
31191139920069,1995cyclictest3195207-21kworker/1:2+events23:21:591
31191089920064,1999cyclictest3142915-21kworker/0:2+events20:35:410
31191139920059,1993cyclictest3167064-21kworker/1:2+events21:58:011
311911399200510,1992cyclictest3126950-21kworker/1:1+events19:41:001
31191179920049,1992cyclictest3191580-21kworker/2:0+events23:07:322
31191089920045,1996cyclictest3147800-21kworker/0:1+events20:46:230
311911999200310,1990cyclictest3151319-21kworker/3:0+events20:56:363
31191179920029,1990cyclictest3204311-21kworker/2:1+events00:04:252
31191179920029,1990cyclictest3148179-21kworker/2:0+events21:17:172
31191139920029,1990cyclictest3204577-21kworker/1:1+events23:36:021
31191139920029,1990cyclictest3163681-21kworker/1:0+events21:32:291
31191089920023,1996cyclictest3212678-21kworker/0:2+events00:07:290
31191089920023,1996cyclictest3212678-21kworker/0:2+events00:07:280
31191089920023,1996cyclictest3170459-21kworker/0:1+events21:52:160
31191179920018,1990cyclictest3199750-21kworker/2:0+events23:29:112
311911799200110,1988cyclictest3170278-21kworker/2:1+events21:47:422
31191139920019,1989cyclictest3117253-21kworker/1:0+events19:16:291
31191089920014,1994cyclictest3110970-21kworker/0:0+events19:17:590
31191199920008,1989cyclictest3205997-21kworker/3:0+events23:38:433
31191179920006,1991cyclictest3217559-21kworker/2:0+events00:14:202
311911799200010,1987cyclictest3160809-21kworker/2:2+events21:40:492
311911999199910,1985cyclictest3159115-21kworker/3:2+events22:05:183
311911999199910,1985cyclictest3159115-21kworker/3:2+events22:05:183
31191139919999,1987cyclictest3178514-21kworker/1:0+events22:22:481
31191139919999,1987cyclictest3128581-21kworker/1:2+events19:58:111
31191139919994,1992cyclictest3183819-21kworker/1:0+events22:53:571
31191089919999,1988cyclictest3124037-21kworker/0:1+events19:45:080
311911999199710,1984cyclictest3159115-21kworker/3:2+events22:16:423
31191179919979,1985cyclictest3131868-21kworker/2:2+events19:49:242
311911799199610,1983cyclictest3222519-21kworker/2:2+events00:33:472
31191089919936,1984cyclictest3199688-21kworker/0:2+events23:33:450
31191089919934,1986cyclictest3130112-21kworker/0:2+events19:41:200
31191089919927,1982cyclictest3135018-21kworker/0:0+events20:00:480
311910899199210,1979cyclictest3154191-21kworker/0:2+events21:00:480
31191199919909,1978cyclictest3124130-21kworker/3:0+events20:08:043
31191199919895,1981cyclictest3115483-21kworker/3:2+events19:16:513
311910899198710,1974cyclictest3190001-21kworker/0:0+events22:58:080
311911999198610,1973cyclictest3146144-21kworker/3:2+events20:33:523
31191179919866,1977cyclictest3148179-21kworker/2:0+events20:46:462
31191089919855,1977cyclictest3110970-21kworker/0:0+events19:23:480
311911999198410,1971cyclictest3162673-21kworker/3:0+events21:29:273
31191199919835,1975cyclictest3183585-21kworker/3:1+events22:35:423
31191199919829,1970cyclictest3124130-21kworker/3:0+events19:43:133
31191199919819,1969cyclictest3170491-21kworker/3:1+events21:53:243
31191199919819,1969cyclictest3160719-21kworker/3:1+events21:21:523
31191199919814,1974cyclictest3151319-21kworker/3:0+events21:06:363
31191199919809,1969cyclictest3151319-21kworker/3:0+events21:18:023
31191089919779,1965cyclictest3163711-21kworker/0:2+events21:27:120
311911999197510,1962cyclictest3193265-21kworker/3:0+events22:58:523
31191199919747,1964cyclictest3149456-21kworker/3:2+events20:54:013
31191089919749,1963cyclictest3212678-21kworker/0:2+events00:01:220
31191199919736,1964cyclictest3131925-21kworker/3:1+events19:59:193
31191199919718,1960cyclictest3217588-21kworker/3:2+events00:18:313
31191199919718,1960cyclictest3209251-21kworker/3:0+events23:51:113
31191199919718,1960cyclictest3185217-21kworker/3:0+events22:40:133
31191199919718,1960cyclictest3170491-21kworker/3:1+events21:49:353
31191199919715,1963cyclictest3159115-21kworker/3:2+events21:41:593
311911999197110,1958cyclictest3130322-21kworker/3:2+events19:47:293
31191089919719,1958cyclictest3147800-21kworker/0:1+events20:43:430
311911999197010,1957cyclictest3186993-21kworker/3:2+events23:17:043
31191199919679,1955cyclictest3222489-21kworker/3:0+events00:34:323
31191089919659,1953cyclictest3142915-21kworker/0:2+events20:27:040
31191199919639,1950cyclictest3159115-21kworker/3:2+events22:14:203
31191089919635,1955cyclictest3135018-21kworker/0:0+events19:58:570
31191139919616,1952cyclictest3128581-21kworker/1:2+events19:49:001
31191199919609,1948cyclictest3144543-21kworker/3:1+events20:36:043
31191199919609,1948cyclictest3142943-21kworker/3:0+events20:27:263
31191199919574,1950cyclictest3214063-21kworker/3:1+events00:06:413
31191199919574,1950cyclictest3214063-21kworker/3:1+events00:06:413
311911999195610,1943cyclictest3201103-21kworker/3:1+events23:43:383
31191199919466,1937cyclictest3124130-21kworker/3:0+events19:34:093
311911999193710,1924cyclictest3198238-21kworker/3:0+events23:20:483
31191139919369,1924cyclictest3193386-21kworker/1:1+events23:00:241
31191199919346,1925cyclictest3159115-21kworker/3:2+events22:21:433
31191199919329,1920cyclictest3131925-21kworker/3:1+events19:50:103
311911999193210,1920cyclictest3126980-21kworker/3:1+events19:37:313
311911999193210,1919cyclictest3224093-21kworker/3:1+events00:37:593
31191139919329,1921cyclictest3128581-21kworker/1:2+events20:13:001
31191089919328,1922cyclictest3157600-21kworker/0:2+events21:10:010
31191089919302,1925cyclictest3124037-21kworker/0:1+events19:38:410
31191199919298,1918cyclictest3186993-21kworker/3:2+events23:02:403
31191179919279,1915cyclictest3170278-21kworker/2:1+events22:08:152
31191179919279,1915cyclictest3170278-21kworker/2:1+events22:08:152
311911999192610,1914cyclictest3124130-21kworker/3:0+events20:11:303
31191199919199,1906cyclictest3159115-21kworker/3:2+events21:57:403
31191199919194,1912cyclictest3159115-21kworker/3:2+events22:27:453
311911999191910,1907cyclictest3162673-21kworker/3:0+events21:35:103
311911999191810,1904cyclictest3124130-21kworker/3:0+events20:00:253
311911999191710,1904cyclictest3202855-21kworker/3:2+events23:29:583
311911999191710,1904cyclictest3186993-21kworker/3:2+events23:08:193
31191199919169,1904cyclictest3175316-21kworker/3:0+events22:02:383
31191199919169,1904cyclictest3115483-21kworker/3:2+events19:26:343
31191199919167,1906cyclictest3149456-21kworker/3:2+events20:45:163
31191199919165,1908cyclictest3115483-21kworker/3:2+events19:20:423
31191199919159,1903cyclictest3201103-21kworker/3:1+events23:48:553
31191199919089,1895cyclictest3155910-21kworker/3:2+events21:03:083
311911999190410,1891cyclictest3217588-21kworker/3:2+events00:29:093
31187452149104,25sleep30-21swapper/319:08:023
311910899145142,2cyclictest3190001-21kworker/0:0+pm22:54:540
31186202143108,29sleep10-21swapper/119:06:271
31186382140106,29sleep20-21swapper/219:06:402
31185732140100,27sleep00-21swapper/019:05:500
311911999140,5cyclictest3203306-21runrttasks23:30:143
311910899147,3cyclictest3159501-21/usr/sbin/munin21:15:150
311911799130,12cyclictest3207081-21/usr/sbin/munin23:40:182
311911799121,8cyclictest3162439-21munin-run21:24:592
311911799120,12cyclictest0-21swapper/221:05:212
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional