You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-23 - 06:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot3s.osadl.org (updated Tue Dec 23, 2025 00:43:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2033644212153,63sleep30-21swapper/319:09:463
220589221170,2sleep00-21swapper/021:50:180
203335529060,9sleep10-21swapper/119:06:071
202875628154,19sleep00-21swapper/019:05:100
203354527863,9sleep20-21swapper/219:08:292
2033786992824,2cyclictest0-21swapper/321:35:003
2033786992815,10cyclictest2151959-21sleep20:59:593
203378699249,11cyclictest2289041-21turbostat23:10:003
2033786992421,1cyclictest0-21swapper/320:15:003
2033786992218,2cyclictest0-21swapper/300:15:003
203378699216,11cyclictest2273119-21sleep22:54:593
2033786992118,1cyclictest0-21swapper/323:25:013
2033786992118,1cyclictest0-21swapper/321:45:273
2033786992118,1cyclictest0-21swapper/321:15:013
2033786992118,1cyclictest0-21swapper/300:01:563
2033786992117,2cyclictest0-21swapper/300:25:013
21056722201,7sleep02107370-21kthreadcore20:15:160
2033786992017,1cyclictest0-21swapper/323:40:203
2033786992017,1cyclictest0-21swapper/323:15:293
2033786992016,2cyclictest0-21swapper/320:45:263
2033786992016,2cyclictest0-21swapper/320:19:593
2033786992014,3cyclictest2038988-21turbostat19:14:593
203378699197,6cyclictest2331069-21taskset23:46:433
2033786991917,1cyclictest0-21swapper/322:20:193
2033786991916,1cyclictest0-21swapper/323:55:173
2033786991916,1cyclictest0-21swapper/322:49:013
2033786991916,1cyclictest0-21swapper/322:49:003
2033786991916,1cyclictest0-21swapper/322:30:143
2033786991916,1cyclictest0-21swapper/322:25:193
2033786991916,1cyclictest0-21swapper/321:28:403
2033786991916,1cyclictest0-21swapper/320:05:213
2033786991916,1cyclictest0-21swapper/319:15:023
2033786991916,1cyclictest0-21swapper/300:25:173
2033786991915,2cyclictest0-21swapper/321:40:123
203378699191,14cyclictest2173142-21chrt21:20:113
203378699189,6cyclictest2125263-21taskset20:32:503
203378699187,8cyclictest2157189-21turbostat21:09:543
203378699183,12cyclictest0-21swapper/322:10:213
2033786991815,1cyclictest0-21swapper/323:25:153
2033786991815,1cyclictest0-21swapper/319:35:193
2033786991814,2cyclictest0-21swapper/319:55:193
203378699181,13cyclictest2357829-21taskset00:15:023
203378699181,13cyclictest2347418-21chrt00:05:123
203378699181,13cyclictest0-21swapper/320:20:203
203378699181,11cyclictest2126393-21taskset20:35:143
203378699181,11cyclictest2126393-21taskset20:35:143
203378699177,6cyclictest2252400-21taskset22:35:023
2033786991715,1cyclictest0-21swapper/319:35:013
2033786991714,1cyclictest0-21swapper/323:50:203
2033786991714,1cyclictest0-21swapper/322:15:023
2033786991714,1cyclictest0-21swapper/319:45:113
2033786991714,1cyclictest0-21swapper/319:40:113
2033786991714,1cyclictest0-21swapper/300:30:143
2033786991713,2cyclictest0-21swapper/322:08:113
2033786991713,2cyclictest0-21swapper/320:01:373
2033786991713,2cyclictest0-21swapper/319:50:273
2033786991713,2cyclictest0-21swapper/300:36:013
2033786991713,1cyclictest0-21swapper/323:10:353
2033786991711,3cyclictest2219467-21taskset22:00:223
203378699171,13cyclictest0-21swapper/321:15:213
203378699171,10cyclictest2118866-21taskset20:25:163
203378699170,14cyclictest0-21swapper/322:40:183
203378699170,14cyclictest0-21swapper/319:25:233
203378699170,14cyclictest0-21swapper/319:25:003
23407952161,10sleep22340841-21ntpq23:55:222
23357292161,4sleep12335873-21mailstats23:50:241
23197272161,10sleep02319773-21ntpq23:35:200
23103182161,9sleep12310368-21munin-run23:29:591
203378699162,11cyclictest0-21swapper/321:50:183
203378699161,9cyclictest2152198-21taskset21:00:073
2033786991613,1cyclictest0-21swapper/323:35:233
2033786991613,1cyclictest0-21swapper/323:30:123
2033786991613,1cyclictest0-21swapper/323:00:243
2033786991613,1cyclictest0-21swapper/321:35:163
2033786991613,1cyclictest0-21swapper/320:50:283
203378699161,10cyclictest2278207-21taskset22:55:303
203378699160,13cyclictest0-21swapper/321:55:163
203378699160,13cyclictest0-21swapper/320:40:173
21898342151,3sleep02191493-21grep21:35:160
23216892141,8sleep12323314-21tr23:40:181
23200502141,9sleep12320055-21grep23:35:251
22720392140,9sleep12272096-21ntpq22:50:201
22626372141,4sleep22262640-21cron22:45:002
22348502141,9sleep12235361-21grep22:15:171
21148762140,9sleep22114924-21munin-run20:25:002
23043732130,8sleep22304444-21find23:20:272
22803712131,8sleep12282139-21cut23:00:181
22273772131,7sleep22229151-21cut22:10:172
23063232121,8sleep20-21swapper/223:25:162
22900762121,7sleep22290227-21kthreadcore23:10:172
22736202122,4sleep12273639-21apt-get22:55:131
21459762120,8sleep02145978-21grep20:50:270
21300332121,6sleep12130088-21ntpq20:35:211
21300332121,6sleep12130088-21ntpq20:35:201
20827062121,7sleep02082962-21unixbench_singl19:50:270
23616812110,2sleep00-21swapper/000:15:200
23588532111,6sleep22360548-21kthreadcore00:15:182
23526082112,4sleep12352627-21apt-get00:10:121
23260292111,7sleep2800-21snmpd23:44:482
22524092111,5sleep22252428-21apt-get22:35:092
22255682110,7sleep22225797-21sed22:10:012
21562052111,5sleep02156254-21ntpq21:00:210
20823512111,6sleep22082405-21modprobe19:50:212
23658182101,5sleep22366963-21memory00:20:192
23499212101,4sleep02351187-21memory00:05:180
23429302100,6sleep22344188-21kthreadcore00:00:192
23406532101,4sleep12340794-21ntpq23:55:211
23321362101,4sleep22332499-21cut23:50:172
22948852101,4sleep02295103-21df_inode23:15:160
22883422101,5sleep12288445-21/usr/sbin/munin23:05:271
22416792101,4sleep22241727-21munin-run22:25:002
22308482101,5sleep12231119-21cat22:15:011
22100552101,4sleep12210311-21df21:55:131
21843362101,4sleep02184407-21irqstats21:30:150
21780702101,4sleep22178142-21munin-run21:25:002
21718762101,4sleep12171922-21ntpq21:15:211
21299822101,4sleep02130031-21ntpq20:35:200
21299822101,4sleep02130031-21ntpq20:35:200
21223722100,6sleep22124194-21kthreadcore20:30:172
21191012101,4sleep02119149-21ntpq20:25:180
20876322101,4sleep22087672-21ntpq19:55:212
20771402101,4sleep22077196-21ntp_states19:45:192
20663252101,4sleep22066369-21ntpq19:35:222
20620462101,5sleep02062065-21munin-run19:35:000
20464892101,3sleep02046508-21apt-get19:20:130
2378861291,4sleep02379115-21df_inode00:35:160
2363126291,2sleep00-21swapper/000:20:130
2358389291,4sleep12358851-21kthreadcore00:15:171
2347396291,2sleep20-21swapper/200:05:122
2284079291,4sleep02284224-21cstates23:05:130
2279415290,2sleep00-21swapper/023:00:170
2273587291,3sleep22273092-21apt-get22:55:022
2257715291,4sleep02257911-21expr22:40:130
2251179291,3sleep22251228-21ntpq22:30:212
2242524291,4sleep02244004-21tr22:25:180
2203693291,4sleep12203781-21ps21:45:231
2193586290,1sleep20-21swapper/221:35:262
2182527290,1sleep20-21swapper/221:25:202
2173446291,4sleep12173654-21if_enp1s021:20:131
2163222291,4sleep12164444-21cut21:10:161
2162735290,1sleep20-21swapper/221:10:132
2157456291,4sleep22157683-21cstates21:05:122
2157434291,3sleep12157453-21apt-get21:05:111
2151629290,2sleep2171rcu_preempt20:56:212
2136788291,4sleep02136935-21grep20:45:140
2110179291,3sleep12110388-21hddtemp_smartct20:20:141
2104375291,4sleep02104476-21cat20:15:000
2099412290,3sleep22099431-21apt-get20:10:132
2098202291,3sleep02098259-21ntpq20:05:220
2098054291,4sleep22098144-21gcc20:05:192
2088915290,4sleep02088933-21apt-get20:00:130
2056049291,4sleep22056059-21awk19:25:242
2051520291,4sleep21-21systemd19:25:002
2377556281,2sleep12377597-21ntpq00:30:191
2374828281,4sleep22376554-21kthreadcore00:30:172
2367150281,3sleep12367199-21ntpq00:20:211
2339706281,3sleep02340652-21/usr/sbin/munin23:55:200
2315353280,1sleep20-21swapper/223:33:062
2309159280,1sleep00-21swapper/023:25:180
2268402281,3sleep02268611-21hddtemp_smartct22:50:150
2267573281,4sleep1800-21snmpd22:46:311
2267573281,4sleep1800-21snmpd22:46:311
2242235281,4sleep20-21swapper/222:25:162
2236710281,2sleep10-21swapper/122:20:141
2235726281,4sleep02235785-21sed22:15:230
2218227281,3sleep22219169-21sed22:00:192
2216496281,3sleep12218225-21cut22:00:181
2214955280,1sleep20-21swapper/221:59:232
2171930281,3sleep22171999-21sshd21:15:222
2156683280,1sleep20-21swapper/221:00:272
2146924281,2sleep10-21swapper/120:55:131
2145754280,1sleep10-21swapper/120:50:241
2124389281,3sleep02124437-21ntpq20:30:180
2083909281,4sleep02084079-21sed19:55:160
2082331281,3sleep12082350-21ntpq19:50:201
2073062280,2sleep10-21swapper/119:45:021
2062855281,3sleep12063279-21cat19:35:161
2056082281,3sleep02051781-21/usr/sbin/munin19:25:250
2035579281,3sleep22037376-21tr19:10:172
2034528281,3sleep12035577-21kthreadcore19:10:161
2383527270,1sleep10-21swapper/100:36:331
2378829271,3sleep22378844-21acpi00:35:142
2378383270,1sleep00-21swapper/000:32:430
2353539271,2sleep20-21swapper/200:10:172
2342119270,2sleep10-21swapper/100:00:141
2330983270,1sleep10-21swapper/123:45:391
2326877271,2sleep22328089-21cut23:45:162
2315504270,1sleep10-21swapper/123:34:571
2295104271,3sleep12295262-21head23:15:161
2293686271,3sleep10-21swapper/123:10:241
2284347270,3sleep22284560-21hddtemp_smartct23:05:162
2283144271,3sleep20-21swapper/223:00:272
2268684271,3sleep22270081-21cut22:50:172
2262903270,2sleep20-21swapper/222:45:162
2262903270,2sleep20-21swapper/222:45:152
2247178270,2sleep10-21swapper/122:30:121
2246786270,4sleep10-21swapper/122:28:231
2225212270,1sleep00-21swapper/022:05:290
2210033270,2sleep00-21swapper/021:55:120
2208617271,3sleep22033780-21cyclictest21:50:212
2199634270,2sleep20-21swapper/221:45:122
2198243270,1sleep00-21swapper/021:40:170
2194422270,2sleep10-21swapper/121:40:101
2167884270,2sleep00-21swapper/021:15:120
2147250271,2sleep02147469-21cat20:55:160
2135553271,3sleep12135757-21unixbench-2d20:40:261
2125951270,2sleep20-21swapper/220:35:022
2125951270,2sleep20-21swapper/220:35:012
2109596270,1sleep20-21swapper/220:19:552
2109502270,1sleep10-21swapper/120:18:421
2091957270,3sleep22092772-21/usr/sbin/munin20:00:192
2090223270,4sleep10-21swapper/120:00:181
2067599271,3sleep20-21swapper/219:40:112
2057606271,2sleep20-21swapper/219:30:172
2057034270,3sleep12057053-21apt-get19:30:121
2046511271,2sleep10-21swapper/119:20:141
2041787271,3sleep02043254-21kthreadcore19:15:150
2369371261,2sleep00-21swapper/000:25:180
2368661260,3sleep10-21swapper/100:25:151
2352267260,4sleep10-21swapper/100:09:001
2329890261,2sleep02330274-21strings23:45:180
2321030261,1sleep00-21swapper/023:40:130
2320641260,1sleep20-21swapper/223:38:422
2300670261,2sleep10-21swapper/123:20:181
2299506260,1sleep20-21swapper/223:16:102
2267554260,1sleep00-21swapper/022:46:180
2267554260,1sleep00-21swapper/022:46:180
2257229260,4sleep10-21swapper/122:37:461
2256715261,2sleep00-21swapper/022:35:240
2251882260,1sleep00-21swapper/022:31:350
2241339260,4sleep00-21swapper/022:20:410
2220860260,1sleep10-21swapper/122:05:151
2215293260,1sleep02215312-21apt-get22:00:130
2205126260,2sleep10-21swapper/121:50:161
2203643261,1sleep00-21swapper/021:45:220
2195562261,2sleep20-21swapper/221:40:142
2189135260,1sleep10-21swapper/121:35:051
2188143260,2sleep20-21swapper/221:30:242
2184408260,2sleep10-21swapper/121:30:151
2180329261,2sleep00-21swapper/021:25:180
2162226260,4sleep00-21swapper/021:07:590
2137068261,2sleep20-21swapper/220:45:152
2136058260,1sleep20-21swapper/220:42:442
2132305260,2sleep00-21swapper/020:40:170
2120988261,1sleep10-21swapper/120:30:151
2120141260,1sleep10-21swapper/120:29:521
2120039260,1sleep20-21swapper/220:28:332
2113935261,1sleep00-21swapper/020:20:200
2088407260,1sleep10-21swapper/119:57:141
2078047260,1sleep00-21swapper/019:49:310
2067577260,2sleep10-21swapper/119:40:101
2067037260,1sleep00-21swapper/019:36:430
2051780260,2sleep10-21swapper/119:25:111
2046132260,4sleep20-21swapper/219:19:072
2038707260,1sleep00-21swapper/019:11:350
2373027250,3sleep20-21swapper/200:25:522
2352934250,2sleep00-21swapper/000:10:140
2300593250,1sleep00-21swapper/023:20:170
2294200251,2sleep00-21swapper/023:10:310
2277694250,1sleep00-21swapper/022:55:240
2262288250,1sleep10-21swapper/122:40:341
2236280250,2sleep20-21swapper/222:18:152
2183573250,1sleep10-21swapper/121:30:001
2163150251,1sleep00-21swapper/021:10:140
2154907250,1sleep10-21swapper/121:00:181
2146336250,3sleep20-21swapper/220:50:532
2141303250,3sleep10-21swapper/120:48:241
2099713250,2sleep10-21swapper/120:10:161
2072213250,3sleep00-21swapper/019:40:490
2039038251,1sleep10-21swapper/119:15:011
2341809240,1sleep00-21swapper/000:00:010
2331869240,1sleep00-21swapper/023:50:150
2310861240,1sleep00-21swapper/023:30:150
2225828240,1sleep00-21swapper/022:10:110
2173397240,1sleep00-21swapper/021:20:120
2094160240,2sleep10-21swapper/120:05:111
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional