You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-29 - 22:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackfslot7s.osadl.org (updated Tue Apr 29, 2025 00:44:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
51492999316516,3144cyclictest548881-21kworker/0:0+events21:10:100
51493199298314,2964cyclictest534279-21kworker/1:2+events21:00:171
51493699282814,2809cyclictest604789-21kworker/2:1+events00:20:182
51493199281013,2792cyclictest534279-21kworker/1:2+events20:20:241
51493199279513,2778cyclictest534279-21kworker/1:2+events20:25:181
51493199278912,2775cyclictest553933-21kworker/1:2+events21:20:231
51492999276013,2743cyclictest516127-21kworker/0:0+events19:25:190
51493699270112,2686cyclictest557330-21kworker/2:1+events22:00:222
51492999269613,2676cyclictest595601-21kworker/0:2+events23:50:120
51492999264013,2623cyclictest516127-21kworker/0:0+events19:20:210
51493699258310,2569cyclictest576092-21kworker/2:0+events22:40:222
51492999256212,2547cyclictest557209-21kworker/0:0+events21:40:160
5149319925279,2514cyclictest606702-21kworker/1:1+events00:24:091
51494399251912,2503cyclictest607342-21kworker/3:1+events00:30:123
5149319925169,2503cyclictest603600-21kworker/1:2+events00:33:381
51492999250612,2491cyclictest566575-21kworker/0:2+events22:00:000
51493699250212,2485cyclictest527888-21kworker/2:2+events19:50:172
5149439924975,2489cyclictest560884-21kworker/3:0+events21:36:483
51492999249411,2480cyclictest564714-21kworker/0:1+events22:00:460
51492999249312,2477cyclictest568967-21kworker/0:2+events22:11:200
51494399247711,2462cyclictest576040-21kworker/3:1+events22:44:343
51492999247712,2461cyclictest519762-21kworker/0:2+events19:45:450
51493199245211,2439cyclictest524790-21kworker/1:2+events19:40:041
51492999245111,2437cyclictest529744-21kworker/0:2+events19:59:250
51493199244912,2434cyclictest584970-21kworker/1:2+events23:08:221
51492999244511,2431cyclictest589738-21kworker/0:1+events23:22:520
51493199244111,2427cyclictest527799-21kworker/1:0+events19:51:501
51493199243810,2424cyclictest529401-21kworker/1:1+events19:55:361
51492999242110,2407cyclictest590451-21kworker/0:0+events23:35:150
51494399242012,2404cyclictest579481-21kworker/3:0+events23:00:213
51492999241712,2401cyclictest588115-21kworker/0:0+events23:19:260
51493199241412,2398cyclictest534279-21kworker/1:2+events20:58:471
51492999239910,2386cyclictest512821-21kworker/0:1+events19:15:260
51494399239611,2381cyclictest582656-21kworker/3:2+events22:56:513
51493199239512,2379cyclictest594999-21kworker/1:0+events23:41:461
51492999238312,2367cyclictest547037-21kworker/0:2+events20:53:290
51492999237612,2360cyclictest583104-21kworker/0:1+events23:05:400
51492999236211,2347cyclictest548881-21kworker/0:0+events20:56:100
51493199235712,2341cyclictest568286-21kworker/1:0+events22:03:461
51492999235710,2345cyclictest601383-21kworker/0:1+events00:10:030
51492999235111,2337cyclictest557209-21kworker/0:0+events21:25:210
51493699234212,2326cyclictest546975-21kworker/2:1+events20:50:242
51493699233812,2322cyclictest567124-21kworker/2:2+events22:10:102
51493199233212,2317cyclictest542153-21kworker/1:0+events20:36:441
51492999233012,2315cyclictest557209-21kworker/0:0+events21:30:150
51493699232912,2313cyclictest559587-21kworker/2:2+events21:37:572
51493699232712,2312cyclictest588236-21kworker/2:0+events23:27:032
51494399232612,2310cyclictest529430-21kworker/3:1+events19:54:513
51493699232412,2309cyclictest588236-21kworker/2:0+events23:24:232
51494399231611,2302cyclictest555749-21kworker/3:2+events21:20:003
51492999231410,2300cyclictest516127-21kworker/0:0+events19:32:510
51492999231310,2300cyclictest601383-21kworker/0:1+events00:17:590
51494399230712,2292cyclictest552300-21kworker/3:1+events21:24:113
51494399230611,2291cyclictest595929-21kworker/3:0+events00:19:093
51492999230610,2293cyclictest595601-21kworker/0:2+events23:57:470
51492999228110,2267cyclictest583104-21kworker/0:1+events23:01:530
51492999227610,2263cyclictest557209-21kworker/0:0+events21:53:300
51494399227311,2258cyclictest576040-21kworker/3:1+events22:47:403
51492999227311,2259cyclictest528066-21kworker/0:1+events20:02:080
51494399227211,2258cyclictest574401-21kworker/3:0+events22:30:253
51492999227211,2257cyclictest608869-21kworker/0:0+events00:35:540
51494399227011,2257cyclictest573752-21kworker/3:2+events22:25:063
51493699227012,2255cyclictest542124-21kworker/2:0+events20:44:002
51492999227011,2256cyclictest590451-21kworker/0:0+events23:45:130
51492999227011,2255cyclictest608869-21kworker/0:0+events00:31:460
51493699226710,2254cyclictest589918-21kworker/2:1+events23:39:272
51492999226510,2251cyclictest552096-21kworker/0:2+events21:21:090
51493199226311,2248cyclictest589889-21kworker/1:2+events23:25:531
51492999226311,2249cyclictest595601-21kworker/0:2+events00:03:340
51492999226111,2248cyclictest568253-21kworker/0:0+events22:20:070
51492999226111,2246cyclictest579719-21kworker/0:2+events22:59:120
51492999226111,2246cyclictest534102-21kworker/0:2+events20:07:030
51492999225811,2243cyclictest579034-21kworker/0:1+events22:48:260
5149299922579,2245cyclictest523247-21kworker/0:1+events19:40:260
51492999225611,2242cyclictest534102-21kworker/0:2+events20:13:330
51492999225510,2241cyclictest568253-21kworker/0:0+events22:15:540
51492999225411,2239cyclictest606643-21kworker/0:2+events00:23:460
51492999225411,2239cyclictest541940-21kworker/0:1+events20:34:010
51492999225211,2237cyclictest555558-21kworker/0:1+events21:19:400
51492999225210,2239cyclictest579034-21kworker/0:1+events22:53:260
51492999225210,2238cyclictest595601-21kworker/0:2+events00:05:520
51492999225111,2236cyclictest579034-21kworker/0:1+events22:44:100
51492999224911,2235cyclictest552096-21kworker/0:2+events21:09:000
51492999224811,2234cyclictest589738-21kworker/0:1+events23:29:380
51492999224710,2234cyclictest571914-21kworker/0:1+events22:27:440
51492999224611,2232cyclictest584878-21kworker/0:2+events23:12:590
51493699223710,2224cyclictest557983-21kworker/2:0+events21:29:532
5149319922379,2224cyclictest582954-21kworker/1:1+events23:01:311
5149319922377,2227cyclictest581323-21kworker/1:2+events22:55:201
51493199223711,2223cyclictest601646-21kworker/1:0+events00:11:111
51493199223710,2223cyclictest578971-21kworker/1:1+events22:45:431
51493199223510,2222cyclictest568286-21kworker/1:0+events22:12:511
51492999223510,2222cyclictest526468-21kworker/0:0+events19:53:420
5149319922349,2222cyclictest534279-21kworker/1:2+events20:16:341
5149319922348,2222cyclictest594999-21kworker/1:0+events23:52:071
51493199223010,2217cyclictest563440-21kworker/1:2+events21:52:461
51493199222911,2215cyclictest527799-21kworker/1:0+events19:49:331
51493199222810,2215cyclictest578971-21kworker/1:1+events22:43:061
51493199222710,2214cyclictest573482-21kworker/1:2+events22:36:311
51493199222511,2212cyclictest529401-21kworker/1:1+events20:05:081
51493199222411,2210cyclictest529401-21kworker/1:1+events20:00:111
51493199222410,2211cyclictest516335-21kworker/1:0+events19:32:301
5149319922239,2209cyclictest594999-21kworker/1:0+events23:46:451
51493199222111,2207cyclictest568286-21kworker/1:0+events22:15:331
5149319922199,2207cyclictest566545-21kworker/1:1+events22:07:111
5149319922189,2205cyclictest561103-21kworker/1:0+events21:40:371
51493199221811,2205cyclictest551592-21kworker/1:1+events21:07:051
51493199221811,2204cyclictest542153-21kworker/1:0+events20:34:251
5149319922178,2206cyclictest568286-21kworker/1:0+events22:20:311
51493199221711,2203cyclictest553933-21kworker/1:2+events21:27:381
51493199221711,2202cyclictest534279-21kworker/1:2+events20:48:061
51493199221711,2202cyclictest534279-21kworker/1:2+events20:48:061
51493199221411,2200cyclictest578971-21kworker/1:1+events22:51:511
51493199221410,2201cyclictest573482-21kworker/1:2+events22:34:381
5149319922137,2203cyclictest573482-21kworker/1:2+events22:25:521
51493199221310,2200cyclictest521152-21kworker/1:2+events19:28:211
51493199221310,2199cyclictest601646-21kworker/1:0+events00:17:381
51493199221211,2197cyclictest586666-21kworker/1:1+events23:15:401
5149319922088,2197cyclictest592166-21kworker/1:2+events23:36:231
51492999220511,2191cyclictest548881-21kworker/0:0+events21:02:110
51493699220311,2188cyclictest532800-21kworker/2:0+events20:07:262
51493199219910,2187cyclictest584970-21kworker/1:2+events23:10:411
51493699219611,2181cyclictest593411-21kworker/2:0+events00:08:532
5149319921939,2180cyclictest553933-21kworker/1:2+events21:38:421
5149369921899,2177cyclictest557330-21kworker/2:1+events21:43:382
51493699218711,2173cyclictest557330-21kworker/2:1+events21:49:442
51493699218711,2172cyclictest570607-21kworker/2:1+events22:17:512
51494399218011,2165cyclictest542346-21kworker/3:1+events20:35:583
51492999217910,2165cyclictest593733-21kworker/0:1+events23:43:190
51493199217710,2164cyclictest562784-21kworker/1:1+events21:47:031
51493199217710,2163cyclictest529401-21kworker/1:1+events20:13:091
51493199217410,2160cyclictest548282-21kworker/1:0+events21:19:151
51492999217211,2158cyclictest537457-21kworker/0:1+events20:21:310
51494399216611,2152cyclictest539100-21kworker/3:2+events20:33:383
51493199216411,2150cyclictest598922-21kworker/1:2+events00:07:231
51494399216111,2147cyclictest599390-21kworker/3:1+events00:03:563
51492999215910,2146cyclictest563115-21kworker/0:2+events21:46:410
51494399215311,2139cyclictest539100-21kworker/3:2+events20:29:273
51493199215311,2139cyclictest542153-21kworker/1:0+events20:40:561
51492999215211,2137cyclictest557895-21kworker/0:2+events21:36:000
51493199214911,2135cyclictest563440-21kworker/1:2+events21:56:121
51493699214311,2129cyclictest576092-21kworker/2:0+events22:50:422
51493199214211,2127cyclictest589889-21kworker/1:2+events23:24:451
51493699213811,2123cyclictest567124-21kworker/2:2+events22:06:482
51493199213310,2119cyclictest603600-21kworker/1:2+events00:37:461
5149319921326,2123cyclictest512960-21kworker/1:1+events19:22:381
51493699213011,2115cyclictest593411-21kworker/2:0+events00:15:462
51493699213011,2115cyclictest542124-21kworker/2:0+events20:37:062
51493699212911,2115cyclictest589918-21kworker/2:1+events23:44:032
51493699212911,2114cyclictest606735-21kworker/2:2+events00:26:482
51493699212910,2115cyclictest576092-21kworker/2:0+events22:36:082
51493699212811,2114cyclictest527888-21kworker/2:2+events19:59:022
51493699212810,2116cyclictest536247-21kworker/2:2+events20:20:022
51493699212711,2113cyclictest557330-21kworker/2:1+events21:24:342
51493699212710,2114cyclictest526173-21kworker/2:1+events19:47:372
5149369921269,2113cyclictest598983-21kworker/2:1+events00:02:012
51493699212611,2113cyclictest574169-21kworker/2:2+events22:30:022
51493699212611,2111cyclictest590598-21kworker/2:2+events23:32:182
51493699212511,2111cyclictest593411-21kworker/2:0+events00:12:212
51492999212310,2110cyclictest575978-21kworker/0:2+events22:38:260
5149319921226,2113cyclictest586666-21kworker/1:1+events23:34:531
51493699212010,2107cyclictest542124-21kworker/2:0+events21:06:222
51494399211910,2105cyclictest592062-21kworker/3:0+events23:40:123
51494399211810,2105cyclictest608992-21kworker/3:2+events00:35:333
51493699211811,2103cyclictest610468-21kworker/2:2+events00:37:242
51493699211811,2103cyclictest593411-21kworker/2:0+events23:52:522
51492999211810,2105cyclictest545426-21kworker/0:1+events20:45:100
51492999211810,2105cyclictest545426-21kworker/0:1+events20:45:100
51493699211711,2104cyclictest593411-21kworker/2:0+events23:55:092
51493699211711,2104cyclictest532800-21kworker/2:0+events20:11:152
51492999211710,2104cyclictest535852-21kworker/0:0+events20:17:440
51494399211610,2103cyclictest548391-21kworker/3:1+events20:57:393
51492999211610,2103cyclictest590451-21kworker/0:0+events23:31:540
5149319921159,2103cyclictest523130-21kworker/1:1+events19:35:291
51494399211411,2101cyclictest568225-21kworker/3:0+events22:10:553
5149369921149,2102cyclictest557330-21kworker/2:1+events21:53:542
51492999211410,2101cyclictest606643-21kworker/0:2+events00:26:280
51493699211111,2097cyclictest542124-21kworker/2:0+events20:56:542
5149299921119,2099cyclictest523247-21kworker/0:1+events19:38:540
51494399211011,2096cyclictest593654-21kworker/3:2+events23:48:393
51493699211011,2096cyclictest582775-21kworker/2:1+events22:59:582
5149299921098,2098cyclictest543815-21kworker/0:0+events20:44:250
51494399210711,2092cyclictest576040-21kworker/3:1+events22:38:013
51494399210710,2094cyclictest589829-21kworker/3:1+events23:30:473
51494399210611,2092cyclictest546804-21kworker/3:0+events20:47:223
51494399210611,2092cyclictest546804-21kworker/3:0+events20:47:213
51494399210611,2092cyclictest519760-21kworker/3:1+events19:24:113
51492999210610,2093cyclictest568253-21kworker/0:0+events22:08:430
51494399210511,2091cyclictest542346-21kworker/3:1+events20:44:463
51494399210510,2092cyclictest562510-21kworker/3:2+events21:55:503
51494399210510,2092cyclictest523217-21kworker/3:1+events19:41:583
51494399210510,2091cyclictest534257-21kworker/3:2+events20:08:113
51494399210411,2090cyclictest576040-21kworker/3:1+events22:54:353
51494399210411,2090cyclictest516152-21kworker/3:0+events19:31:433
51494399210410,2092cyclictest589829-21kworker/3:1+events23:25:293
51494399210410,2090cyclictest531049-21kworker/3:2+events19:58:163
51493699210410,2090cyclictest576092-21kworker/2:0+events23:11:052
51494399210311,2089cyclictest607342-21kworker/3:1+events00:28:203
51493699210311,2090cyclictest593411-21kworker/2:0+events23:45:362
51493699210310,2090cyclictest574169-21kworker/2:2+events22:28:072
5149439921029,2089cyclictest566604-21kworker/3:1+events22:02:163
51494399210211,2088cyclictest595929-21kworker/3:0+events00:21:043
51494399210211,2088cyclictest595929-21kworker/3:0+events00:05:063
51494399210211,2088cyclictest568225-21kworker/3:0+events22:07:563
51494399210211,2088cyclictest545180-21kworker/3:2+events20:52:203
51493699210211,2088cyclictest527888-21kworker/2:2+events20:04:472
51494399210111,2088cyclictest537485-21kworker/3:1+events20:20:453
51494399210111,2087cyclictest572149-21kworker/3:0+events22:19:443
51494399210110,2088cyclictest529430-21kworker/3:1+events20:10:293
51493699209810,2084cyclictest557330-21kworker/2:1+events21:34:292
5149369920978,2086cyclictest553829-21kworker/2:0+events21:13:582
51493699209511,2080cyclictest584639-21kworker/2:2+events23:18:172
51494399209310,2080cyclictest562510-21kworker/3:2+events21:42:303
5149369920927,2081cyclictest538898-21kworker/2:1+events20:27:342
51494399208811,2073cyclictest552300-21kworker/3:1+events21:09:243
5149439920846,2074cyclictest586527-21kworker/3:0+events23:19:023
5149319920848,2073cyclictest559380-21kworker/1:1+events21:34:511
51493199208410,2070cyclictest512960-21kworker/1:1+events19:19:151
51494399208011,2066cyclictest529430-21kworker/3:1+events20:02:543
5149319920795,2071cyclictest597325-21kworker/1:1+events00:01:141
51494399207811,2064cyclictest595929-21kworker/3:0+events00:14:593
51494399207710,2063cyclictest570541-21kworker/3:1+events22:20:553
51493199207610,2063cyclictest606702-21kworker/1:1+events00:29:051
51493199206810,2055cyclictest597325-21kworker/1:1+events23:57:241
5149319920675,2059cyclictest511310-21kworker/1:2+events19:12:461
51493699206511,2051cyclictest577902-21kworker/2:1+events22:46:072
51493199206410,2050cyclictest551592-21kworker/1:1+events21:13:141
5149439920612,2055cyclictest584262-21kworker/3:1+events23:07:593
51492999205811,2045cyclictest534102-21kworker/0:2+events20:25:410
51494399205511,2041cyclictest584262-21kworker/3:1+events23:11:523
5149369920547,2043cyclictest519792-21kworker/2:2+events19:28:422
51494399205110,2038cyclictest557359-21kworker/3:0+events21:26:523
51494399204711,2033cyclictest521349-21kworker/3:2+events19:39:173
5149299920469,2034cyclictest534102-21kworker/0:2+events20:39:230
5149299920469,2034cyclictest516127-21kworker/0:0+events19:13:100
5149299920368,2025cyclictest571914-21kworker/0:1+events22:32:440
51494399203010,2017cyclictest516152-21kworker/3:0+events19:26:273
5149369920188,2006cyclictest576092-21kworker/2:0+events23:05:192
5149319920105,2002cyclictest548282-21kworker/1:0+events20:51:581
51493699199810,1985cyclictest570607-21kworker/2:1+events22:21:412
51493699199610,1983cyclictest532800-21kworker/2:0+events20:15:252
51493699199410,1981cyclictest542124-21kworker/2:0+events20:34:472
5149369919887,1978cyclictest545384-21kworker/2:2+events20:48:302
5149369919887,1978cyclictest545384-21kworker/2:2+events20:48:302
51493699198710,1974cyclictest551500-21kworker/2:1+events21:15:282
5149369919846,1974cyclictest519792-21kworker/2:2+events19:34:442
5149439919804,1972cyclictest550003-21kworker/3:0+events21:11:453
51494399197410,1961cyclictest526225-21kworker/3:0+events19:46:083
5149369919737,1963cyclictest551500-21kworker/2:1+events21:03:212
5149369919728,1961cyclictest593411-21kworker/2:0+events00:34:012
51493699197210,1959cyclictest564920-21kworker/2:0+events21:55:262
51493699197210,1959cyclictest519792-21kworker/2:2+events19:24:332
5149369919709,1958cyclictest523308-21kworker/2:0+events19:44:132
5149369919697,1959cyclictest514598-21kworker/2:0+events19:16:332
5149439919627,1952cyclictest597519-21kworker/3:2+events23:55:543
5149439919534,1945cyclictest550003-21kworker/3:0+events21:03:443
51494399195310,1940cyclictest563169-21kworker/3:0+events21:50:533
5149439919518,1940cyclictest558061-21kworker/3:2+events21:30:403
5149439919507,1939cyclictest588153-21kworker/3:2+events23:21:413
51494399194810,1935cyclictest516152-21kworker/3:0+events19:17:423
5149439919468,1935cyclictest592062-21kworker/3:0+events23:36:013
5149439919227,1912cyclictest535883-21kworker/3:0+events20:16:563
5149369919109,1897cyclictest512972-21kworker/2:2+events19:11:372
51494399187910,1866cyclictest514415-21kworker/3:2+events19:14:183
5149439918779,1865cyclictest595929-21kworker/3:0+events23:51:453
5146682143109,29sleep30-21swapper/319:09:443
5143542142109,28sleep20-21swapper/219:05:432
5145442140110,24sleep00-21swapper/019:08:090
5146382139110,24sleep10-21swapper/119:09:231
51494399190,12cyclictest563459-21latency_hist21:50:003
514936991814,2cyclictest769-21mta-sts-daemon23:00:182
51493699160,14cyclictest524980-21munin-run19:40:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional