You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-17 - 12:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot8s.osadl.org (updated Mon Nov 17, 2025 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
18452229910010,996cyclictest1910042-21sleep20:00:003
1845222993160,313cyclictest1992209-21turbostat21:09:563
1845222992219,1cyclictest0-21swapper/320:40:183
1845222992219,1cyclictest0-21swapper/320:40:173
1845222992118,1cyclictest0-21swapper/321:40:003
1845222992017,1cyclictest0-21swapper/321:00:203
184522299177,7cyclictest2192931-21turbostat23:10:003
184522299176,7cyclictest22634642sleep323:50:233
1845222991714,1cyclictest0-21swapper/321:30:003
1845222991613,1cyclictest0-21swapper/323:32:183
1845222991613,1cyclictest0-21swapper/322:25:133
1845222991613,1cyclictest0-21swapper/300:30:283
22661602150,5sleep22266271-21entropy23:55:162
184522299152,9cyclictest1979925-21chrt20:55:113
1845222991513,1cyclictest0-21swapper/320:33:503
1845222991513,1cyclictest0-21swapper/300:25:263
1845222991513,0cyclictest0-21swapper/321:54:513
1845222991512,1cyclictest0-21swapper/322:35:163
1845222991512,1cyclictest0-21swapper/322:14:093
1845222991512,1cyclictest0-21swapper/319:30:193
1845222991512,1cyclictest0-21swapper/300:36:493
184522299151,11cyclictest2157931-21taskset22:45:243
184522299151,10cyclictest2239334-21taskset23:35:233
184522299151,10cyclictest1859811-21sleep19:20:013
19545042141,3sleep01954537-21apt-get20:35:150
190898140,4rtkit-daemon2014889-21ssh21:20:030
190898140,3rtkit-daemon2297594-21ssh00:14:102
184522299145,7cyclictest2005248-21taskset21:11:543
1845222991413,0cyclictest0-21swapper/322:07:573
1845222991413,0cyclictest0-21swapper/320:27:583
1845222991412,1cyclictest0-21swapper/322:31:593
1845222991412,1cyclictest0-21swapper/321:42:573
1845222991412,1cyclictest0-21swapper/321:22:543
1845222991412,1cyclictest0-21swapper/319:35:183
1845222991412,0cyclictest0-21swapper/319:10:543
1845222991411,1cyclictest0-21swapper/323:55:183
1845222991411,1cyclictest0-21swapper/323:26:173
1845222991411,1cyclictest0-21swapper/323:15:173
1845222991411,1cyclictest0-21swapper/320:17:563
184522299141,10cyclictest2295336-21taskset00:10:233
184522299141,10cyclictest2289053-21taskset00:07:493
184522299141,10cyclictest2086178-21chrt22:02:233
184522299140,11cyclictest2201159-21chrt23:15:003
184522299140,11cyclictest1891443-21taskset19:45:003
190898130,9rtkit-daemon667-21snmpd20:17:400
184522299135,6cyclictest20312502sleep321:30:153
184522299131,9cyclictest2306711-21taskset00:20:163
184522299131,9cyclictest2298512-21chrt00:15:023
184522299131,9cyclictest2209762-21taskset23:20:133
184522299131,9cyclictest1866276-21chrt19:25:013
184522299131,8cyclictest19102882sleep320:00:153
1845222991312,0cyclictest0-21swapper/319:40:483
1845222991311,1cyclictest0-21swapper/322:20:193
1845222991311,1cyclictest0-21swapper/320:50:023
1845222991311,1cyclictest0-21swapper/320:50:023
1845222991311,1cyclictest0-21swapper/320:47:573
184522299131,10cyclictest2160741-21taskset22:50:123
184522299131,10cyclictest2111653-21turbostat22:19:593
1845222991310,1cyclictest0-21swapper/320:25:003
184522299130,10cyclictest2253632-21taskset23:45:223
184522299130,10cyclictest2242025-21taskset23:40:133
184522299130,10cyclictest2174217-21taskset22:55:243
184522299130,10cyclictest2071510-21chrt21:55:153
184522299130,10cyclictest2012863-21taskset21:15:563
22335042120,4sleep01998506-21diskmemload23:34:500
21186152121,4sleep12118623-21ssh22:22:261
20606552120,1sleep12060662-21/usr/sbin/munin21:45:241
20133492121,8sleep22013357-21ssh21:17:082
190898120,8rtkit-daemon2280126-21ssh00:00:500
184522299129,1cyclictest0-21swapper/300:00:163
184522299124,6cyclictest2055377-21taskset21:45:093
184522299123,7cyclictest1916525-21taskset20:05:043
184522299121,9cyclictest2150975-21chrt22:42:303
184522299121,9cyclictest1960222-21taskset20:35:533
184522299121,8cyclictest1922826-21taskset20:10:043
184522299120,9cyclictest2177358-21chrt23:00:143
184522299120,9cyclictest1897703-21taskset19:50:063
184522299120,10cyclictest1872102-21taskset19:26:503
18449822120,3sleep30-21swapper/319:08:593
23040382110,8sleep02304241-21unixbench-2d00:15:250
21546852110,5sleep12155500-21kthreadcore22:45:181
20252132111,7sleep12027262-21cut21:25:191
19668442110,8sleep01966880-21http20:45:000
19668442110,8sleep01966880-21http20:45:000
19345432111,7sleep21934760-21users20:15:252
190898110,4rtkit-daemon667-21snmpd21:18:410
190898110,3rtkit-daemon1858969-21ps19:15:222
190898110,3rtkit-daemon1851619-21munin-node00:23:490
18974552110,8sleep11897623-21cat19:50:001
22477482101,6sleep22247756-21ssh23:40:292
21204482100,3sleep22120533-21ssh22:25:162
20959642100,5sleep10-21swapper/122:10:161
190898100,6rtkit-daemon674-21lldpd21:42:470
190898100,6rtkit-daemon2079069-21ssh21:59:330
190898100,6rtkit-daemon1928-21sendmail-mta00:30:062
190898100,5rtkit-daemon1947717-21apt-get20:30:010
190898100,5rtkit-daemon1935443-21apt-get20:20:140
190898100,5rtkit-daemon1902874-21ntpq19:50:210
190898100,4rtkit-daemon2184269-21ssh23:03:350
190898100,4rtkit-daemon1928-21sendmail-mta00:25:272
190898100,3rtkit-daemon2094219-21ssh22:07:140
190898100,3rtkit-daemon2062751-21pool-udisksd21:49:200
190898100,3rtkit-daemon1928-21sendmail-mta23:52:460
2150387291,5sleep12150395-21ssh22:41:101
2135592290,6sleep12135600-21ssh22:34:161
2111375290,6sleep02111382-21sh22:19:290
2048802290,6sleep22050775-21grep21:40:202
2015452290,6sleep22016200-21cut21:20:212
1946811291,4sleep21946866-21ntp_states20:25:222
19089890,6rtkit-daemon81364-21packagekitd19:58:080
19089890,5rtkit-daemon667-21snmpd20:25:190
19089890,5rtkit-daemon2036351-21ntp_states21:30:220
19089890,5rtkit-daemon1922214-21users20:05:270
19089890,5rtkit-daemon1884218-21swap19:35:242
19089890,4rtkit-daemon2239882-21ssh23:36:020
19089890,4rtkit-daemon2168033-21chrt22:54:002
19089890,4rtkit-daemon2126211-21ssh22:26:190
19089890,4rtkit-daemon2046425-21taskset21:38:550
19089890,4rtkit-daemon2030433-21ssh21:29:160
19089890,4rtkit-daemon1998285-21chrt21:07:270
19089890,3rtkit-daemon473-21gmain23:26:410
19089890,1rtkit-daemon0-21swapper/200:18:282
2298590280,4sleep12298624-21apt-get00:15:141
2295483281,4sleep02295545-21ntp_states00:10:250
2282299281,4sleep02282460-21diskstats00:05:150
2241326281,4sleep22241334-21ssh23:39:282
2209351281,4sleep12209384-21/usr/sbin/munin23:20:011
2184343281,5sleep22184350-21diskmemload23:03:472
2169790281,5sleep02170695-21kthreadcore22:55:190
2139301280,5sleep02141232-21latency22:35:190
2134951281,4sleep02134959-21ssh22:32:460
2126651280,5sleep12126659-21ssh22:27:151
2104852281,4sleep12105792-21ssh22:15:171
2011180280,5sleep12011949-21sed21:15:231
1991290281,4sleep01991343-21ntp_states21:00:220
1936241280,5sleep21937110-21kthreadcore20:20:192
19089880,4rtkit-daemon81364-21packagekitd23:05:580
19089880,4rtkit-daemon674-21lldpd23:17:030
19089880,4rtkit-daemon674-21lldpd22:42:040
19089880,4rtkit-daemon674-21lldpd20:00:150
19089880,4rtkit-daemon667-21snmpd23:22:320
19089880,4rtkit-daemon667-21snmpd20:53:240
19089880,4rtkit-daemon667-21snmpd20:53:230
19089880,4rtkit-daemon473-21gmain22:12:340
19089880,4rtkit-daemon438-21wpa_supplicant23:55:550
19089880,4rtkit-daemon438-21wpa_supplicant20:48:260
19089880,4rtkit-daemon2074564-21kworker/0:0-events23:47:190
19089880,4rtkit-daemon2068740-21ntp_states21:50:210
19089880,4rtkit-daemon1922873-21chrt20:10:140
19089880,4rtkit-daemon1868113-21grep19:25:172
19089880,4rtkit-daemon1851573-21date19:15:012
19089880,3rtkit-daemon1985040-21ntp_states20:55:210
1891658281,4sleep21891857-21hddtemp_smartct19:45:162
1840181280,5sleep21841350-21grep19:05:172
2332631270,4sleep02334506-21cut00:35:200
2279584271,4sleep22279785-21ssh00:00:252
2266110270,5sleep12266159-21/usr/sbin/munin23:55:151
2231888270,4sleep12231896-21ssh23:31:111
2231317270,4sleep22231570-21grep23:30:282
2214895270,4sleep22214974-21ntp_states23:20:222
2191208271,4sleep12191216-21ssh23:05:581
2187844270,5sleep22189955-21grep23:05:232
2162236271,3sleep12163978-21cut22:50:191
2158645270,1sleep00-21swapper/022:46:330
2136215271,2sleep20-21swapper/222:35:142
2028336271,3sleep22028462-21smartctl21:25:242
2022488270,5sleep12022494-21ssh21:24:361
1998894270,4sleep01999057-21diskstats21:10:160
1987662270,4sleep11989699-21tr21:00:191
1985888270,4sleep21986084-21latency_hist21:00:002
1984461270,5sleep11984895-21sed20:55:191
1975771270,4sleep21977910-21cut20:50:202
1975771270,4sleep21977910-21cut20:50:192
1954213270,1sleep10-21swapper/120:34:351
1924880270,4sleep11927007-21grep20:10:201
19089870,4rtkit-daemon671-21ntpd22:20:480
19089870,4rtkit-daemon667-21snmpd00:35:292
19089870,4rtkit-daemon473-21gmain22:56:181
19089870,4rtkit-daemon473-21gmain22:48:332
19089870,4rtkit-daemon1851619-21munin-node19:43:390
19089870,3rtkit-daemon667-21snmpd00:25:080
19089870,3rtkit-daemon299-21systemd-journal19:32:302
19089870,3rtkit-daemon2248292-21ssh23:41:460
19089870,3rtkit-daemon1845217-21cyclictest23:12:140
19089870,1rtkit-daemon0-21swapper/022:00:540
1874806270,5sleep01876968-21kthreadcore19:30:200
1872351270,4sleep11872396-21munin-run19:30:011
1860089271,3sleep21860238-21df_inode19:20:122
2330391260,3sleep02330399-21ssh00:34:160
2323559260,3sleep12324554-21irqstats00:30:201
2297896260,1sleep10-21swapper/100:14:501
2289097260,1sleep10-21swapper/100:07:561
2264489260,3sleep12264497-21ssh23:52:011
2258238260,3sleep22258361-21grep23:50:142
2255241260,4sleep22255282-21proc_pri23:45:262
2248667260,3sleep12248675-21ssh23:42:441
2174798260,1sleep20-21swapper/222:55:562
2151619260,1sleep20-21swapper/222:44:032
2141515260,3sleep12141589-21processes22:35:251
2105799260,4sleep22107834-21tr22:15:172
2095748260,3sleep22095783-21apt-get22:10:132
2079655261,2sleep20-21swapper/222:00:162
2077641260,1sleep20-21swapper/221:56:112
2070619260,1sleep22070623-21ssh21:53:332
2063898261,2sleep12064009-21cut21:50:161
2052511262,2sleep1321rcuc/121:40:221
2045140260,4sleep22045148-21ssh21:35:542
2037905260,4sleep12037913-21ssh21:32:541
2036990260,1sleep20-21swapper/221:30:422
2005637260,4sleep10-21swapper/121:12:531
1998739260,1sleep20-21swapper/221:10:022
1967397261,3sleep11967543-21df20:45:141
19089860,3rtkit-daemon674-21lldpd19:43:012
19089860,3rtkit-daemon667-21snmpd19:46:520
19089860,3rtkit-daemon473-21gmain00:20:022
19089860,2rtkit-daemon2338078-21ssh00:38:251
19089860,2rtkit-daemon2289705-21ssh00:09:212
1898111261,3sleep20-21swapper/219:50:182
1884879260,1sleep10-21swapper/119:39:421
1845001260,1sleep00-21swapper/019:09:130
1839537260,3sleep11840179-21cut19:05:161
2313287250,3sleep10-21swapper/100:22:231
2274095250,3sleep12274103-21ssh00:00:121
2250003251,2sleep10-21swapper/123:45:171
2225197250,3sleep12225205-21ssh23:29:251
2206930251,2sleep20-21swapper/223:15:252
2199225250,2sleep20-21swapper/223:10:552
2193197250,1sleep10-21swapper/123:10:061
2177162250,2sleep10-21swapper/123:00:121
2160655250,3sleep02160664-21ssh22:50:020
2135539250,3sleep22135546-21diskmemload22:34:092
2088771250,2sleep20-21swapper/222:05:192
2087639251,1sleep12087678-21apt-get22:05:141
2087305250,3sleep12087312-21sh22:04:551
2077312250,1sleep10-21swapper/121:55:301
2046674250,3sleep10-21swapper/121:39:271
1992841250,2sleep20-21swapper/221:05:182
1978720250,3sleep11978791-21ntp_states20:50:231
1978720250,3sleep11978791-21ntp_states20:50:221
1961012250,2sleep10-21swapper/120:40:171
1961012250,2sleep10-21swapper/120:40:161
1960423250,1sleep10-21swapper/120:38:291
1954246250,2sleep21954482-21sed20:35:012
1928530250,1sleep20-21swapper/220:10:332
1922519250,1sleep20-21swapper/220:09:172
1922419250,1sleep10-21swapper/120:08:001
1910480250,3sleep11910618-21grep20:00:171
1909934250,1sleep10-21swapper/119:59:061
19089850,1rtkit-daemon0-21swapper/123:38:101
1872832251,2sleep11873014-21idleruntime19:30:171
1866017250,1sleep00-21swapper/019:24:340
1860053250,2sleep10-21swapper/119:20:111
1851388250,1sleep10-21swapper/119:14:181
1851164250,1sleep00-21swapper/019:11:240
2321892240,2sleep12321900-21ssh00:28:451
2219160240,2sleep22220339-21kthreadcore23:25:182
2216086241,1sleep10-21swapper/123:22:011
2117889240,2sleep22117897-21ssh22:20:472
2061835240,2sleep2421rcuc/221:47:012
1992020240,1sleep20-21swapper/221:02:542
1969764240,2sleep20-21swapper/220:45:192
1961128240,2sleep21960827-21/usr/sbin/munin20:40:182
1961128240,2sleep21960827-21/usr/sbin/munin20:40:172
1942085240,1sleep10-21swapper/120:25:181
1935809240,2sleep10-21swapper/120:20:181
1903999240,1sleep20-21swapper/219:55:142
1886647240,2sleep10-21swapper/119:40:231
1884752240,3sleep00-21swapper/019:38:050
1872124240,3sleep00-21swapper/019:27:070
1853959240,2sleep10-21swapper/119:15:151
1851446241,1sleep00-21swapper/019:15:020
1996383230,1sleep10-21swapper/121:05:211
1959739230,1sleep20-21swapper/220:35:252
1929360230,1sleep10-21swapper/120:15:161
1915243230,1sleep21915355-21strings20:00:222
1897701230,1sleep10-21swapper/119:50:041
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional