You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-23 - 00:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackfslot7s.osadl.org (updated Wed Apr 22, 2026 00:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
36831599280613,2790cyclictest471978-21kworker/0:1+events00:35:190
36831999279413,2777cyclictest442140-21kworker/1:2+events23:05:141
36831999278110,2766cyclictest465644-21kworker/1:0+events00:20:131
36831599277511,2760cyclictest413139-21kworker/0:1+events21:42:280
36831999276513,2747cyclictest459541-21kworker/1:1+events00:05:181
36831999272511,2712cyclictest424382-21kworker/1:0+events22:15:001
3683199927149,2703cyclictest442140-21kworker/1:2+events23:20:271
36832399265912,2642cyclictest400441-21kworker/2:0+events20:50:162
36831999257412,2558cyclictest421143-21kworker/1:2+events22:00:131
36831599256912,2553cyclictest400411-21kworker/0:2+events20:55:170
3683199925688,2556cyclictest463060-21kworker/1:2+events00:15:161
36831599255711,2542cyclictest405523-21kworker/0:2+events21:10:170
36832399254312,2528cyclictest387706-21kworker/2:2+events20:35:172
36831599253211,2518cyclictest450164-21kworker/0:0+events23:50:110
36831999250112,2485cyclictest405315-21kworker/1:2+events21:05:191
36831599249912,2484cyclictest467523-21kworker/0:1+events00:20:370
36831599249812,2482cyclictest450164-21kworker/0:0+events23:34:410
36832399249211,2477cyclictest416588-21kworker/2:2+events21:44:242
36832399249112,2476cyclictest465768-21kworker/2:1+events00:28:542
36832399249111,2477cyclictest378420-21kworker/2:0+events19:47:232
36831999249012,2474cyclictest442140-21kworker/1:2+events23:29:391
36832399248912,2474cyclictest470632-21kworker/2:2+events00:34:332
36832399248912,2474cyclictest432250-21kworker/2:2+events22:37:072
36831599248912,2474cyclictest445277-21kworker/0:0+events23:16:390
36831599248912,2473cyclictest419721-21kworker/0:1+events21:58:430
36831599248810,2474cyclictest409911-21kworker/0:0+events21:26:110
36832399248712,2472cyclictest451511-21kworker/2:1+events23:42:172
36831599248511,2471cyclictest384367-21kworker/0:0+events20:09:430
36831599248511,2470cyclictest416408-21kworker/0:2+events21:46:180
36831599248412,2469cyclictest390940-21kworker/0:1+events20:24:320
36831599248312,2469cyclictest450164-21kworker/0:0+events00:05:400
36832399248212,2467cyclictest390702-21kworker/2:0+events20:20:192
36831999247912,2465cyclictest421143-21kworker/1:2+events21:55:461
36832399247812,2463cyclictest427974-21kworker/2:0+events22:18:282
36831599247312,2457cyclictest445277-21kworker/0:0+events23:09:470
36832399247011,2455cyclictest451511-21kworker/2:1+events23:39:592
36831599246811,2454cyclictest448517-21kworker/0:1+events23:23:050
36831599246712,2451cyclictest382765-21kworker/0:1+events20:00:140
36832399246612,2451cyclictest413410-21kworker/2:1+events21:30:192
36831599246511,2451cyclictest427413-21kworker/0:1+events22:19:340
36831599246412,2450cyclictest424206-21kworker/0:0+events22:10:030
36831599246411,2449cyclictest424206-21kworker/0:0+events22:06:140
36831999246312,2447cyclictest438938-21kworker/1:2+events22:53:461
36831599246310,2449cyclictest392564-21kworker/0:2+events20:28:020
36832399246112,2446cyclictest448183-21kworker/2:0+events23:51:412
36831999246112,2447cyclictest406903-21kworker/1:1+events21:25:041
36831599246112,2447cyclictest405523-21kworker/0:2+events21:15:340
36832399246012,2444cyclictest442020-21kworker/2:2+events23:02:092
36832399245711,2443cyclictest427974-21kworker/2:0+events22:21:282
36832399245414,2435cyclictest413410-21kworker/2:1+events21:35:192
3683199924535,2444cyclictest414998-21kworker/1:2+events21:37:581
36832399244112,2426cyclictest459597-21kworker/2:1+events00:04:322
36831999243911,2425cyclictest395660-21kworker/1:0+events20:59:111
36832399242312,2405cyclictest398621-21kworker/2:1+events20:45:372
36831999242112,2405cyclictest442140-21kworker/1:2+events23:48:431
36831999241810,2405cyclictest424382-21kworker/1:0+events22:34:021
36831999241012,2395cyclictest395660-21kworker/1:0+events20:49:291
36831599240512,2389cyclictest366188-21kworker/0:1+events19:24:200
36831999240212,2387cyclictest371413-21kworker/1:1+events19:36:181
36831999240112,2386cyclictest446640-21kworker/1:1+events23:19:421
36831999239910,2386cyclictest383181-21kworker/1:1+events20:04:231
36831999239910,2385cyclictest383181-21kworker/1:1+events20:06:181
36831999239812,2382cyclictest440535-21kworker/1:0+events22:58:201
3683199923979,2384cyclictest390796-21kworker/1:0+events20:17:411
36831999239712,2381cyclictest465644-21kworker/1:0+events00:25:331
36831999239712,2381cyclictest449905-21kworker/1:0+events23:36:111
36831999239612,2381cyclictest424382-21kworker/1:0+events22:35:581
36831999239611,2382cyclictest433728-21kworker/1:1+events22:44:201
36831999239512,2380cyclictest374983-21kworker/1:2+events19:32:011
36831999239512,2379cyclictest405315-21kworker/1:2+events21:02:361
36831999239512,2379cyclictest405315-21kworker/1:2+events21:02:361
36831999239511,2381cyclictest459541-21kworker/1:1+events00:00:231
36831999239511,2381cyclictest378295-21kworker/1:2+events19:44:221
36831599239411,2380cyclictest435242-21kworker/0:1+events22:46:150
3683199923939,2381cyclictest465644-21kworker/1:0+events00:11:281
36831599239312,2377cyclictest403896-21kworker/0:1+events21:03:240
36831599239312,2377cyclictest403896-21kworker/0:1+events21:03:230
36831999239212,2377cyclictest433728-21kworker/1:1+events23:01:461
36832399239012,2375cyclictest408410-21kworker/2:0+events21:15:562
36831599239011,2376cyclictest384367-21kworker/0:0+events20:19:100
36831999238711,2373cyclictest421143-21kworker/1:2+events21:53:521
36831999238412,2369cyclictest457918-21kworker/1:0+events23:54:191
36831999237112,2356cyclictest392476-21kworker/1:2+events20:39:511
36832399234612,2331cyclictest448183-21kworker/2:0+events23:21:582
36831599234612,2331cyclictest414776-21kworker/0:0+events21:34:100
36831599234610,2332cyclictest450164-21kworker/0:0+events23:35:480
36832399234112,2326cyclictest440358-21kworker/2:0+events22:52:382
36832399233611,2321cyclictest437101-21kworker/2:1+events22:55:192
36832399233410,2321cyclictest400441-21kworker/2:0+events20:57:382
36831999233410,2320cyclictest395660-21kworker/1:0+events20:34:541
36831999232812,2312cyclictest472142-21kworker/1:2+events00:34:551
36831999232812,2312cyclictest413380-21kworker/1:0+events21:32:141
36832399232512,2310cyclictest434065-21kworker/2:0+events22:40:332
36832399232512,2310cyclictest395600-21kworker/2:0+events20:40:132
36832399232512,2310cyclictest371629-21kworker/2:0+events19:23:582
36831599231912,2304cyclictest427413-21kworker/0:1+events22:20:200
3683269923098,2298cyclictest402013-21kworker/3:1+events21:03:463
3683269923098,2298cyclictest402013-21kworker/3:1+events21:03:463
36831599230812,2292cyclictest427413-21kworker/0:1+events22:28:440
36831999230611,2292cyclictest424382-21kworker/1:0+events22:26:041
36832399230412,2289cyclictest426075-21kworker/2:1+events22:12:462
3683199923016,2291cyclictest395660-21kworker/1:0+events20:42:071
36832399229812,2282cyclictest451511-21kworker/2:1+events23:47:352
36831599229711,2283cyclictest413139-21kworker/0:1+events21:39:060
36832399229611,2282cyclictest384399-21kworker/2:2+events19:58:422
36831599229512,2280cyclictest435242-21kworker/0:1+events22:38:180
36831999229311,2279cyclictest470547-21kworker/1:1+events00:39:541
36832399229212,2276cyclictest356580-21kworker/2:1+events19:18:392
36831999229112,2276cyclictest400473-21kworker/1:2+events20:52:571
36832699227812,2262cyclictest443735-21kworker/3:0+events23:10:543
36831599227512,2260cyclictest392564-21kworker/0:2+events20:39:260
36831599227411,2260cyclictest366188-21kworker/0:1+events19:33:350
36831599227210,2258cyclictest437310-21kworker/0:2+events23:02:550
36831599227011,2257cyclictest435242-21kworker/0:1+events22:43:140
36831599227010,2256cyclictest405523-21kworker/0:2+events21:21:410
36831599226912,2253cyclictest378326-21kworker/0:2+events19:48:290
36832399226810,2255cyclictest376716-21kworker/2:1+events19:35:542
3683159922685,2261cyclictest379949-21kworker/0:0+events19:50:010
36831599226811,2254cyclictest437310-21kworker/0:2+events22:51:060
36831599226811,2254cyclictest405523-21kworker/0:2+events21:05:420
36831599226810,2255cyclictest384367-21kworker/0:0+events19:57:140
36831599226711,2253cyclictest441988-21kworker/0:0+events22:58:440
36831599226710,2253cyclictest366188-21kworker/0:1+events19:37:280
36832399226510,2252cyclictest413410-21kworker/2:1+events21:45:322
36831599226511,2250cyclictest457767-21kworker/0:2+events23:49:060
36831599226511,2250cyclictest366188-21kworker/0:1+events19:26:380
36831599226311,2249cyclictest470398-21kworker/0:0+events00:27:040
36831599226311,2248cyclictest395813-21kworker/0:0+events20:47:340
36831599226210,2249cyclictest450164-21kworker/0:0+events23:44:340
36831599226210,2249cyclictest450164-21kworker/0:0+events23:26:110
36831599226110,2248cyclictest387589-21kworker/0:1+events20:10:520
36831599225911,2245cyclictest450164-21kworker/0:0+events23:57:210
36832699225812,2243cyclictest458118-21kworker/3:1+events00:17:113
3683159922589,2247cyclictest465705-21kworker/0:2+events00:30:010
36831599225710,2243cyclictest400411-21kworker/0:2+events20:54:310
3683239922537,2243cyclictest465768-21kworker/2:1+events00:15:402
36832399225311,2238cyclictest413410-21kworker/2:1+events21:51:572
36832399225311,2237cyclictest440358-21kworker/2:0+events23:08:152
36831599225311,2239cyclictest432490-21kworker/0:2+events22:31:470
36831599223513,2220cyclictest378326-21kworker/0:2+events19:42:250
36832699222911,2214cyclictest440567-21kworker/3:1+events23:04:263
3683239922286,2218cyclictest373250-21kworker/2:2+events19:27:012
36831999222210,2209cyclictest405315-21kworker/1:2+events21:11:041
36832399221711,2203cyclictest424414-21kworker/2:0+events22:08:302
36831999221610,2202cyclictest413380-21kworker/1:0+events21:41:181
36831999220911,2195cyclictest419543-21kworker/1:1+events21:48:111
36831999220512,2190cyclictest381502-21kworker/1:0+events19:59:501
36831999220310,2189cyclictest442140-21kworker/1:2+events23:44:111
36831999220210,2189cyclictest373045-21kworker/1:0+events19:22:251
3683199922016,2192cyclictest378295-21kworker/1:2+events19:46:391
3683199922008,2189cyclictest387559-21kworker/1:0+events20:11:141
36831999220011,2186cyclictest459541-21kworker/1:1+events23:58:271
3683199921999,2187cyclictest406903-21kworker/1:1+events21:22:021
3683199921999,2187cyclictest369406-21kworker/1:2+events19:15:591
36831999219911,2187cyclictest424382-21kworker/1:0+events22:18:501
36831999219911,2185cyclictest449905-21kworker/1:0+events23:34:171
36831999219710,2183cyclictest437280-21kworker/1:0+events22:45:511
36831999219510,2182cyclictest383181-21kworker/1:1+events20:21:301
36832399219111,2177cyclictest387706-21kworker/2:2+events20:32:592
36831999218811,2174cyclictest371413-21kworker/1:1+events19:27:241
36831999218611,2172cyclictest383181-21kworker/1:1+events20:27:151
36831999218611,2171cyclictest430520-21kworker/1:1+events22:21:511
36831999218610,2173cyclictest408552-21kworker/1:0+events21:16:431
3683239921788,2166cyclictest430578-21kworker/2:1+events22:27:382
36831599217810,2164cyclictest465705-21kworker/0:2+events00:14:540
3683199921759,2162cyclictest443833-21kworker/1:0+events23:10:111
36832399217110,2157cyclictest387706-21kworker/2:2+events20:08:562
3683159921709,2158cyclictest390940-21kworker/0:1+events20:34:080
36831599216911,2155cyclictest419721-21kworker/0:1+events21:51:130
36831599216513,2149cyclictest398832-21kworker/0:1+events20:44:280
3683159921647,2154cyclictest450164-21kworker/0:0+events00:16:270
36832399215411,2140cyclictest437101-21kworker/2:1+events22:49:382
36832399214611,2131cyclictest446584-21kworker/2:2+events23:15:532
36832399214311,2129cyclictest411690-21kworker/2:0+events21:26:342
36832699212910,2116cyclictest437709-21kworker/3:2+events22:54:323
3683159921279,2115cyclictest443616-21kworker/0:1+events23:13:570
36831999212010,2106cyclictest424382-21kworker/1:0+events22:06:371
36832399211911,2107cyclictest386076-21kworker/2:1+events20:10:072
36831599211910,2106cyclictest419721-21kworker/0:1+events22:00:560
36832399211810,2105cyclictest448183-21kworker/2:0+events23:27:432
36832699211711,2103cyclictest464064-21kworker/3:2+events00:09:333
36832399211610,2103cyclictest443649-21kworker/2:1+events23:12:492
3683269921159,2103cyclictest445408-21kworker/3:2+events23:25:483
36832399210711,2092cyclictest376716-21kworker/2:1+events19:42:022
3683159921079,2095cyclictest461266-21kworker/0:3+events00:00:460
3683239921067,2096cyclictest384399-21kworker/2:2+events20:00:342
36832399210611,2091cyclictest413410-21kworker/2:1+events21:57:582
36832399210610,2092cyclictest465768-21kworker/2:1+events00:21:212
36832699210410,2091cyclictest416350-21kworker/3:2+events22:06:593
3683239921049,2092cyclictest406796-21kworker/2:2+events21:22:482
36832399210211,2088cyclictest404034-21kworker/2:1+events21:07:572
36832399210210,2089cyclictest459597-21kworker/2:1+events23:58:512
3683239920997,2089cyclictest378420-21kworker/2:0+events19:54:142
36832399209910,2086cyclictest472259-21kworker/2:0+events00:36:292
3683239920989,2086cyclictest464126-21kworker/2:2+events00:10:432
36832399209810,2085cyclictest421112-21kworker/2:2+events22:03:342
36832399209711,2083cyclictest387706-21kworker/2:2+events20:19:342
36831599209510,2082cyclictest371520-21kworker/0:0+events19:19:030
36832399209411,2080cyclictest373250-21kworker/2:2+events19:34:222
36831999207710,2064cyclictest378295-21kworker/1:2+events19:54:361
3683269920698,2057cyclictest412633-21kworker/3:1+events21:36:503
3683239920657,2055cyclictest404034-21kworker/2:1+events21:04:102
3683239920657,2055cyclictest404034-21kworker/2:1+events21:04:102
3683159920477,2037cyclictest367815-21kworker/0:2+events19:13:410
3683239920456,2036cyclictest461244-21kworker/2:0+events00:06:502
3683269920379,2025cyclictest451357-21kworker/3:0+events23:52:043
3683199920248,2013cyclictest369406-21kworker/1:2+events19:14:471
36832699198910,1976cyclictest398988-21kworker/3:2+events20:46:013
36832399198311,1969cyclictest408410-21kworker/2:0+events21:14:032
36832699195810,1945cyclictest405433-21kworker/3:0+events21:25:273
3683269919516,1942cyclictest375195-21kworker/3:1+events20:26:273
36832699194910,1937cyclictest359893-21kworker/3:2+events19:30:063
36832699194910,1936cyclictest398988-21kworker/3:2+events20:50:383
3683239919479,1935cyclictest432250-21kworker/2:2+events22:31:002
36832399194710,1934cyclictest451511-21kworker/2:1+events23:30:252
3683239919469,1934cyclictest387706-21kworker/2:2+events20:27:372
36832399194010,1927cyclictest369613-21kworker/2:2+events19:14:022
3679112174143,28sleep00-21swapper/019:07:340
3677992168140,23sleep30-21swapper/319:06:103
3679132149103,27sleep20-21swapper/219:07:372
36832699141138,2cyclictest359893-21kworker/3:2+pm19:29:563
3677732140101,26sleep10-21swapper/119:05:511
368326992616,7cyclictest593-21dbus-daemon22:40:003
368326992613,9cyclictest437487-21awk22:44:593
368326992519,4cyclictest586-21avahi-daemon20:00:013
368326992314,6cyclictest429274-21cron22:19:593
368326992215,6cyclictest0-21swapper/320:40:003
36832699219,2cyclictest511ktimers/319:35:163
36832699209,8cyclictest473911-21awk00:40:003
36832699209,8cyclictest454885-21basename23:40:003
36832699208,1cyclictest391524-21df_inode20:20:143
36832699207,8cyclictest388741-21tr20:10:193
36832699203,10cyclictest431703-21ntpq22:25:193
36832699194,5cyclictest470958-21sed00:30:003
36832699193,7cyclictest385541-21ntpq20:00:183
36832699193,13cyclictest471189-21expr00:30:123
368326991911,4cyclictest391031-21munin-run20:20:003
368326991910,6cyclictest415057-21basename21:34:593
368326991910,5cyclictest407140-21perf21:10:003
36832699189,6cyclictest403962-21idleruntime-cro21:00:003
36832699188,7cyclictest0-21swapper/321:45:013
36832699186,9cyclictest421389-21sleep21:55:003
36832699184,5cyclictest419070-21ntpq21:45:193
36832699183,6cyclictest422201-21ld21:55:193
36832699183,6cyclictest397657-21http20:40:123
36832699182,8cyclictest432935-21diskstats22:30:133
36832699181,7cyclictest0-21swapper/319:50:183
368326991814,3cyclictest465153-21ntp_kernel_err00:10:173
368326991810,5cyclictest410285-21munin-run21:20:003
36832699175,5cyclictest444996-21users23:05:223
368326991716,1cyclictest26450irq/143-i91523:41:533
368326991714,2cyclictest593-21dbus-daemon19:25:003
368326991712,3cyclictest0-21swapper/300:00:013
368326991711,4cyclictest451958-21/usr/sbin/munin23:30:153
36832699168,6cyclictest371777-21systemctl19:20:003
36832699167,6cyclictest468565-21python300:20:203
36832699166,5cyclictest355-21systemd-journal19:46:053
36832699164,6cyclictest426882-21ntpq22:10:163
36832699163,5cyclictest447772-21ntpq23:15:193
36832699162,1cyclictest0-21swapper/321:20:203
368326991615,1cyclictest26450irq/143-i91500:03:233
368326991613,2cyclictest395548-21taskset20:32:083
368326991613,2cyclictest26450irq/143-i91522:55:193
368326991612,3cyclictest407379-21/usr/sbin/munin21:10:153
36832699160,15cyclictest0-21swapper/319:10:123
36832699158,4cyclictest430888-21munin-run22:25:003
36832699154,3cyclictest379050-21meminfo19:40:153
368326991514,1cyclictest26450irq/143-i91520:07:253
368326991511,2cyclictest0-21swapper/322:05:013
36832699148,4cyclictest449777-21chrt23:20:343
36832699143,6cyclictest456954-21hddtemp_smartct23:45:143
36832699142,4cyclictest438132-21meminfo22:45:163
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional