You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-23 - 05:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot8s.osadl.org (updated Tue Dec 23, 2025 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
17589139912180,1214cyclictest18966962sleep320:54:563
17589139911970,1194cyclictest20005702sleep322:15:003
1758913999161,910cyclictest2071828-21sleep23:10:003
1758913992810,278cyclictest1786790-21turbostat19:34:553
18511152560,4sleep1321rcuc/120:15:231
19479842330,2sleep01947991-21timerwakeupswit21:30:220
1758913992623,1cyclictest0-21swapper/323:45:003
1758913992219,1cyclictest0-21swapper/322:10:003
1758913991815,1cyclictest0-21swapper/323:15:233
1758913991815,1cyclictest0-21swapper/300:19:593
1758913991714,1cyclictest0-21swapper/320:25:003
18967002161,7sleep21896703-21cron20:55:002
1758913991613,1cyclictest0-21swapper/323:25:213
1758913991613,1cyclictest0-21swapper/323:20:243
1758913991613,1cyclictest0-21swapper/319:55:173
1758913991613,1cyclictest0-21swapper/300:36:403
175891399161,11cyclictest1935925-21taskset21:25:133
175891399161,11cyclictest1903162-21turbostat21:00:003
175891399159,3cyclictest0-21swapper/300:05:003
1758913991513,1cyclictest0-21swapper/323:50:003
1758913991512,1cyclictest0-21swapper/323:30:203
1758913991512,1cyclictest0-21swapper/322:00:173
1758913991512,1cyclictest0-21swapper/321:47:413
1758913991512,1cyclictest0-21swapper/321:45:013
1758913991512,1cyclictest0-21swapper/321:45:003
175891399151,11cyclictest1870920-21turbostat20:35:013
175891399150,10cyclictest20462862sleep322:50:123
175891399144,8cyclictest2046051-21turbostat22:50:013
175891399141,9cyclictest19487632sleep321:35:123
1758913991412,1cyclictest0-21swapper/323:53:463
1758913991412,1cyclictest0-21swapper/319:51:043
1758913991412,1cyclictest0-21swapper/319:45:073
1758913991412,1cyclictest0-21swapper/319:24:403
1758913991411,1cyclictest0-21swapper/323:35:213
1758913991411,1cyclictest0-21swapper/322:57:053
1758913991411,1cyclictest0-21swapper/322:30:233
1758913991411,1cyclictest0-21swapper/322:20:193
1758913991411,1cyclictest0-21swapper/322:00:013
1758913991411,1cyclictest0-21swapper/320:26:293
1758913991411,1cyclictest0-21swapper/320:10:183
175891399141,10cyclictest1768143-21taskset19:15:153
21706632134,7sleep12172537-21cut00:25:191
175891399137,4cyclictest1764911-21taskset19:10:303
175891399134,6cyclictest21768412sleep300:30:173
175891399134,6cyclictest21486282sleep300:05:213
175891399131,9cyclictest2175076-21chrt00:26:123
175891399131,9cyclictest2130432-21chrt23:55:023
175891399131,9cyclictest2078079-21chrt23:12:433
175891399131,9cyclictest2059167-21taskset23:00:013
175891399131,9cyclictest1877002-21taskset20:35:253
175891399131,9cyclictest1837314-21chrt20:05:193
175891399131,8cyclictest1994257-21taskset22:10:133
1758913991312,0cyclictest0-21swapper/321:31:273
1758913991312,0cyclictest0-21swapper/321:20:383
1758913991312,0cyclictest0-21swapper/321:13:053
1758913991312,0cyclictest0-21swapper/321:00:433
1758913991311,1cyclictest0-21swapper/321:51:403
175891399131,10cyclictest2162699-21chrt00:20:033
1758913991311,0cyclictest0-21swapper/319:37:433
1758913991310,1cyclictest0-21swapper/322:27:353
1758913991310,1cyclictest0-21swapper/320:40:243
1758913991310,1cyclictest0-21swapper/319:25:123
175891399130,9cyclictest1889128-21taskset20:45:183
175891399130,10cyclictest2149752-21taskset00:10:013
175891399130,10cyclictest1805886-21taskset19:41:593
21692022121,8sleep02169212-21cpu00:25:140
18063772121,2sleep10-21swapper/119:45:111
175891399125,4cyclictest1845292-21taskset20:15:013
175891399124,6cyclictest1928664-21taskset21:16:523
175891399123,7cyclictest20360582sleep322:40:193
175891399121,9cyclictest1915697-21taskset21:05:423
175891399121,8cyclictest2031447-21taskset22:35:173
175891399120,9cyclictest1831827-21taskset20:00:313
301798110,6rtkit-daemon1819096-21taskset19:53:310
21371492110,8sleep02137346-21iwlist00:00:170
21368972111,5sleep12136930-21apt-get00:00:141
21122632111,7sleep10-21swapper/123:40:191
20979212111,6sleep12097954-21apt-get23:30:131
18517942112,5sleep01851827-21apt-get20:20:120
17815412114,4sleep21782481-21cut19:25:172
301798100,5rtkit-daemon2040206-21hddtemp_smartct22:45:192
301798100,5rtkit-daemon2019081-21ntp_states22:25:222
301798100,5rtkit-daemon1934668-21cut21:20:192
301798100,3rtkit-daemon3205-21sendmail-mta20:16:272
301798100,1rtkit-daemon0-21swapper/123:14:321
301798100,1rtkit-daemon0-21swapper/019:05:160
19552332102,4sleep11955255-21apt-get21:40:111
19552332102,4sleep11955255-21apt-get21:40:111
18869652100,7sleep21889127-21cut20:45:182
18838632101,6sleep01883996-21latency_hist20:45:000
17677222100,7sleep01767874-21/usr/sbin/munin19:15:130
30179890,5rtkit-daemon638-21ntpd21:43:122
30179890,5rtkit-daemon638-21ntpd21:43:122
30179890,5rtkit-daemon448-21wpa_supplicant00:36:191
30179890,5rtkit-daemon447-21systemd-logind23:26:331
30179890,5rtkit-daemon3205-21sendmail-mta21:00:262
30179890,5rtkit-daemon317-21systemd-udevd20:32:332
30179890,4rtkit-daemon2133355-21grep23:55:191
30179890,4rtkit-daemon2117583-21apt-get23:45:111
30179890,4rtkit-daemon2008789-21irqstats22:20:192
30179890,4rtkit-daemon1760528-21irqstats19:10:160
30179890,1rtkit-daemon0-21swapper/222:08:542
30179890,1rtkit-daemon0-21swapper/221:18:282
2162714291,4sleep12162747-21apt-get00:20:141
2008828290,3sleep02010751-21kthreadcore22:20:200
2000584291,4sleep02000617-21apt-get22:15:110
1903765290,6sleep11903895-21ps21:00:161
1896961291,4sleep01896994-21apt-get20:55:140
1825672291,5sleep01825695-21munin-run20:00:010
1787652291,6sleep20-21swapper/219:30:182
1753717291,4sleep30-21swapper/319:05:183
30179880,5rtkit-daemon317-21systemd-udevd00:17:331
30179880,5rtkit-daemon1765413-21munin-node21:52:292
30179880,4rtkit-daemon638-21ntpd19:36:340
30179880,4rtkit-daemon636-21snmpd23:36:361
30179880,4rtkit-daemon636-21snmpd22:52:562
30179880,4rtkit-daemon636-21snmpd22:35:552
30179880,4rtkit-daemon636-21snmpd20:23:142
30179880,4rtkit-daemon636-21snmpd19:24:290
30179880,4rtkit-daemon492-21gmain23:18:281
30179880,4rtkit-daemon492-21gmain20:35:202
30179880,4rtkit-daemon492-21gmain00:06:481
30179880,4rtkit-daemon448-21wpa_supplicant19:29:430
30179880,4rtkit-daemon3205-21sendmail-mta23:23:011
30179880,4rtkit-daemon3205-21sendmail-mta21:30:312
30179880,4rtkit-daemon3205-21sendmail-mta21:26:172
30179880,4rtkit-daemon1952632-21grep21:35:172
30179880,4rtkit-daemon1765413-21munin-node22:55:412
30179880,4rtkit-daemon1765413-21munin-node22:44:142
30179880,4rtkit-daemon1765413-21munin-node20:05:542
30179880,4rtkit-daemon1765413-21munin-node20:00:280
30179880,4rtkit-daemon1765413-21munin-node19:31:050
30179880,4rtkit-daemon0-21swapper/221:07:082
30179880,1rtkit-daemon0-21swapper/223:04:082
30179880,1rtkit-daemon0-21swapper/222:33:212
30179880,1rtkit-daemon0-21swapper/222:12:002
30179880,1rtkit-daemon0-21swapper/100:31:061
30179880,1rtkit-daemon0-21swapper/100:10:511
30179880,1rtkit-daemon0-21swapper/019:49:180
2183920281,4sleep22185955-21kthreadcore00:35:212
2124012280,6sleep12124018-21cat23:50:131
2116220281,4sleep22116345-21python323:40:242
2085902281,4sleep02086751-21cut23:20:190
2078553280,6sleep22078679-21grep23:15:162
2033367281,4sleep12033400-21apt-get22:40:131
2000620281,4sleep12000772-21df_inode22:15:121
1982426280,1sleep21983881-21kthreadcore22:00:192
1933421281,4sleep01934599-21memory21:20:180
1884120280,4sleep01884153-21apt-get20:45:120
1864939280,6sleep11864742-21/usr/sbin/munin20:30:151
1850859281,4sleep01851113-21head20:15:230
1840725280,1sleep11842763-21grep20:10:191
1825955281,3sleep11825986-21apt-get20:00:141
30179870,4rtkit-daemon636-21snmpd20:05:030
30179870,4rtkit-daemon3200908-21packagekitd22:16:092
30179870,3rtkit-daemon638-21ntpd19:42:300
30179870,3rtkit-daemon636-21snmpd23:06:162
30179870,3rtkit-daemon636-21snmpd20:41:342
30179870,3rtkit-daemon3205-21sendmail-mta21:47:312
30179870,3rtkit-daemon1980320-21users21:55:242
30179870,3rtkit-daemon1844742-21chrt20:11:032
30179870,3rtkit-daemon1765413-21munin-node20:55:212
30179870,3rtkit-daemon1765413-21munin-node20:27:462
2039808270,3sleep02039841-21apt-get22:45:160
1974545270,4sleep11974684-21tr21:55:131
1936082271,4sleep01936203-21gltestperf21:25:150
1909830271,3sleep01909863-21apt-get21:05:120
1812535271,3sleep20-21swapper/219:50:012
1780304270,5sleep21780312-21latency_hist19:25:012
1752646270,2sleep20-21swapper/219:05:132
30179860,3rtkit-daemon3200908-21packagekitd21:10:192
2182057260,2sleep00-21swapper/000:35:150
2149366260,1sleep00-21swapper/000:08:100
2149361260,3sleep22149362-21sshd00:08:092
2124206260,4sleep02124348-21gltestperf23:50:160
2110398260,1sleep00-21swapper/023:38:220
2103771260,1sleep00-21swapper/023:31:120
2059372260,3sleep02059455-21tune2fs23:00:150
2058901260,3sleep12059053-21cat23:00:001
2032824260,3sleep12032890-21apt-config22:40:011
2032630260,1sleep00-21swapper/022:37:310
2025575260,3sleep02025658-21/usr/sbin/munin22:30:220
2013769260,3sleep02013802-21apt-get22:25:130
2000257260,1sleep00-21swapper/022:14:070
1994103260,2sleep11994136-21apt-get22:10:121
1993345260,1sleep00-21swapper/022:06:570
1989821261,2sleep10-21swapper/122:05:191
1980439260,1sleep00-21swapper/021:56:480
1970162261,2sleep10-21swapper/121:50:201
1967625261,3sleep0636-21snmpd21:47:150
1960302260,1sleep00-21swapper/021:40:180
1960302260,1sleep00-21swapper/021:40:180
1950217260,3sleep01951314-21cut21:35:170
1943669260,3sleep11944827-21grep21:30:171
1924157260,4sleep01925325-21kthreadcore21:15:170
1922914260,4sleep11922980-21/usr/sbin/munin21:15:141
1890548260,2sleep11890581-21apt-get20:50:141
1877640260,3sleep11877673-21apt-get20:40:121
1871351261,2sleep01871542-21head20:35:150
1833888261,2sleep11835186-21kthreadcore20:05:181
1813056260,2sleep20-21swapper/219:50:132
1798790260,1sleep20-21swapper/219:35:222
1786538261,3sleep1636-21snmpd19:27:081
2178008250,3sleep02180100-21kthreadcore00:30:180
2175640250,2sleep20-21swapper/200:30:112
2168066250,2sleep22168112-21ps00:20:232
2162301250,3sleep00-21swapper/000:18:200
2155866250,1sleep00-21swapper/000:14:000
2150780251,2sleep22151924-21kthreadcore00:10:172
2117032250,2sleep22117048-21apt-get23:45:012
2099344251,2sleep20-21swapper/223:30:182
2091436250,2sleep02091468-21apt-get23:25:100
2090455250,3sleep22090567-21grep23:20:252
2077980250,1sleep20-21swapper/223:11:262
2071772250,1sleep00-21swapper/023:09:410
2053736250,3sleep02054837-21kthreadcore22:55:180
2041264250,3sleep12042807-21kthreadcore22:45:221
2020185250,1sleep10-21swapper/122:30:111
2014673250,3sleep12015567-21cut22:25:181
2007530251,2sleep10-21swapper/122:20:171
1935313250,1sleep10-21swapper/121:20:551
1909530250,1sleep00-21swapper/021:04:280
1896510250,1sleep00-21swapper/020:52:320
1871543250,2sleep11871157-21/usr/sbin/munin20:35:161
1866581250,3sleep01868618-21cut20:30:180
1839833250,2sleep00-21swapper/020:10:170
1806025250,1sleep10-21swapper/119:43:471
1800615251,2sleep21801539-21kthreadcore19:40:152
1799354250,1sleep10-21swapper/119:35:451
1767875250,2sleep11767971-21fschecks_time19:15:141
2162092240,1sleep20-21swapper/200:15:422
2143078240,3sleep20-21swapper/200:04:482
2136523240,3sleep20-21swapper/223:58:592
2124019240,1sleep20-21swapper/223:50:132
2116062240,1sleep00-21swapper/023:40:220
2091626241,1sleep20-21swapper/223:25:132
2084365240,3sleep00-21swapper/023:15:300
2078175240,3sleep00-21swapper/023:13:570
2052474240,3sleep00-21swapper/022:54:500
2052322240,3sleep10-21swapper/122:52:551
1986966240,3sleep10-21swapper/122:02:421
1969090240,2sleep01969592-21kthreadcore21:50:190
1948952240,2sleep10-21swapper/121:35:141
1917838240,2sleep1321rcuc/121:10:171
1910884240,2sleep10-21swapper/121:05:171
1885575240,2sleep11886963-21cat20:45:171
1864130240,1sleep10-21swapper/120:26:101
1864114240,3sleep00-21swapper/020:25:570
1853248240,2sleep10-21swapper/120:20:171
1827410240,2sleep21829027-21cut20:00:192
1822823241,1sleep10-21swapper/119:55:211
1787420240,2sleep10-21swapper/119:30:171
1775605240,2sleep11777034-21kthreadcore19:20:191
1772735240,2sleep21772937-21strings19:15:182
1761540240,2sleep20-21swapper/219:10:182
1760373240,1sleep10-21swapper/119:10:171
2174608230,1sleep20-21swapper/200:25:242
2167685230,1sleep00-21swapper/000:20:210
2136279230,1sleep00-21swapper/023:55:530
2123408230,1sleep00-21swapper/023:45:350
2104550230,1sleep20-21swapper/223:35:152
2071678230,1sleep10-21swapper/123:08:301
2059170230,2sleep10-21swapper/123:00:041
2038161230,1sleep00-21swapper/022:40:200
1981275230,1sleep00-21swapper/022:00:170
1967026230,1sleep10-21swapper/121:45:231
1935888230,1sleep10-21swapper/121:25:121
1916324230,1sleep00-21swapper/021:10:120
1898418230,1sleep10-21swapper/120:55:191
1819255230,1sleep20-21swapper/219:55:012
1813054230,1sleep10-21swapper/119:50:121
1752633230,1sleep10-21swapper/119:05:041
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional