You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-16 - 19:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  Congatec
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot8s.osadl.org (updated Tue Sep 16, 2025 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
175458599750745,4cyclictest1735482-21kworker/3:0+events07:19:233
175457999747742,4cyclictest1760819-21kworker/1:3+events09:09:531
175458599746740,5cyclictest2099569-21turbostat11:49:553
175457999746742,3cyclictest1760819-21kworker/1:3+events09:33:521
175458599745740,4cyclictest1775449-21kworker/3:2+events09:41:503
175458599745739,5cyclictest1944014-21kworker/3:0+events10:40:223
175458599745739,5cyclictest1775449-21kworker/3:2+events08:09:403
175458599745739,5cyclictest1775449-21kworker/3:2+events07:43:283
175457999745740,4cyclictest1760819-21kworker/1:3+events08:58:111
175458599744739,4cyclictest2087007-21kworker/3:2+events11:59:323
175458599744739,4cyclictest2055773-21kworker/3:1+events11:13:473
175458599744739,4cyclictest1944014-21kworker/3:0+events11:05:443
175458599744739,4cyclictest1944014-21kworker/3:0+events09:59:113
175458599744739,4cyclictest1775449-21kworker/3:2+events07:52:563
175457999744740,3cyclictest1760819-21kworker/1:3+events09:24:061
175458599743739,3cyclictest1944014-21kworker/3:0+events10:36:063
175458599743739,3cyclictest1775524-21turbostat07:29:553
175458599743738,4cyclictest2087007-21kworker/3:2+events12:38:173
175458599743738,4cyclictest2087007-21kworker/3:2+events12:06:063
175458599743738,4cyclictest2055773-21kworker/3:1+events11:20:433
175458599743738,4cyclictest1944014-21kworker/3:0+events10:34:313
175458599743738,4cyclictest1944014-21kworker/3:0+events10:08:463
175458599743738,4cyclictest1735482-21kworker/3:0+events07:32:133
175458599743737,5cyclictest2055773-21kworker/3:1+events11:19:063
175458599743737,5cyclictest1944014-21kworker/3:0+events10:04:133
175458599743737,5cyclictest1775449-21kworker/3:2+events07:38:033
175458599743737,5cyclictest1735482-21kworker/3:0+events07:13:403
175458599742738,3cyclictest1775449-21kworker/3:2+events08:00:203
175458599742737,4cyclictest2087007-21kworker/3:2+events11:54:443
175458599742737,4cyclictest2055773-21kworker/3:1+events11:27:193
175458599742737,4cyclictest1944014-21kworker/3:0+events10:21:133
175458599742737,4cyclictest1944014-21kworker/3:0+events10:16:033
175458599742737,4cyclictest1944014-21kworker/3:0+events09:51:593
175458599742737,4cyclictest1775449-21kworker/3:2+events07:59:003
175458599742736,5cyclictest1944014-21kworker/3:0+events10:45:433
175458599742735,6cyclictest1944014-21kworker/3:0+events10:51:053
175458599742735,6cyclictest1944014-21kworker/3:0+events10:51:053
175458199742738,3cyclictest2056061-21kworker/2:1+events11:42:212
175458199742737,4cyclictest2056061-21kworker/2:1+events11:36:142
175458599741736,4cyclictest2087007-21kworker/3:2+events12:30:453
175458599741736,4cyclictest2087007-21kworker/3:2+events12:03:463
175458599741736,4cyclictest1944014-21kworker/3:0+events11:00:453
175458599741736,4cyclictest1944014-21kworker/3:0+events09:46:503
175458599741736,4cyclictest1775449-21kworker/3:2+events08:11:163
175458599741735,5cyclictest1735482-21kworker/3:0+events07:24:043
175458599740735,4cyclictest1944014-21kworker/3:0+events10:27:433
175458599739734,4cyclictest1944014-21kworker/3:0+events10:56:493
175458599739734,4cyclictest1775449-21kworker/3:2+events08:51:033
175458599738733,4cyclictest2055773-21kworker/3:1+events11:33:593
175458599738733,4cyclictest1944014-21kworker/3:0+events10:12:193
175458199738732,5cyclictest2080857-21turbostat11:34:552
175458599736732,3cyclictest1775449-21kworker/3:2+events08:15:203
175458599736731,4cyclictest2087007-21kworker/3:2+events12:27:223
175458199735731,3cyclictest2056061-21kworker/2:1+events11:47:522
175458199733728,4cyclictest2112145-21kworker/2:0+events12:24:402
175458599715710,4cyclictest1800476-21turbostat07:49:553
175457999715711,3cyclictest1760819-21kworker/1:3+events09:26:361
175457999715710,4cyclictest1760819-21kworker/1:3+events12:15:201
175457699712702,9cyclictest1869075-21kworker/0:2+events08:40:210
175457699704697,6cyclictest3839331-21kworker/0:0+events08:25:210
175457999702699,2cyclictest1760819-21kworker/1:3+events09:39:421
175457999702698,3cyclictest1794264-21turbostat07:44:551
175457999700696,3cyclictest1881841-21turbostat08:54:561
175457999700692,3cyclictest1760819-21kworker/1:3+events09:19:401
175457999692687,4cyclictest1760819-21kworker/1:3+events12:11:151
175457699692687,4cyclictest1825497-21kworker/0:1+events08:30:200
175457999690686,3cyclictest1760819-21kworker/1:3+events09:12:301
175457999689685,3cyclictest1760819-21kworker/1:3+events07:48:241
175457999686683,2cyclictest1760819-21kworker/1:3+events09:01:351
175457699685678,6cyclictest1825497-21kworker/0:1+events08:35:210
175458599680675,4cyclictest1875634-21turbostat08:49:553
175457699678675,2cyclictest1825497-21kworker/0:1+events08:21:440
175457699670666,3cyclictest1851952-21kworker/0:3+events08:45:220
175458599667662,4cyclictest2143369-21turbostat12:24:553
175458599658655,2cyclictest1937781-21turbostat09:39:553
175457999629625,3cyclictest2124701-21turbostat12:09:551
175457699624620,3cyclictest1838274-21turbostat08:19:550
175458199621617,3cyclictest2137146-21turbostat12:19:542
2957983820,378rtkit-daemon1735482-21kworker/3:0+pm07:07:443
19432602360,1chrt1943258-21python309:35:230
1754576992218,2cyclictest499-21polkitd11:40:000
1754579992118,2cyclictest2068426-21cron11:20:001
1754585992017,2cyclictest0-21swapper/309:20:013
1754581992016,2cyclictest2125200-21fschecks_time12:05:172
1754579992017,2cyclictest2082581-21kthreadcore11:30:221
1754579992017,2cyclictest2075884-21grep11:25:171
1754581991916,2cyclictest0-21swapper/209:10:002
1754581991913,5cyclictest0-21swapper/211:20:002
1754576991916,2cyclictest1824658-21needreboot08:00:200
1754576991916,2cyclictest0-21swapper/007:20:000
1754576991912,5cyclictest3763191-21systemd-journal10:50:000
1754585991815,2cyclictest0-21swapper/312:14:593
1754585991813,3cyclictest1888627-21gltestperf08:55:153
175458199184,2cyclictest191rcu_preempt08:50:002
1754581991813,4cyclictest0-21swapper/208:20:002
1754579991812,4cyclictest2156264-21sort12:30:141
1754576991815,2cyclictest491-21avahi-daemon12:05:000
295798170,3rtkit-daemon1857214-21/usr/sbin/munin08:30:172
1754585991713,2cyclictest0-21swapper/311:45:003
1754581991714,2cyclictest1832660-21gltestperf08:10:172
1754581991713,3cyclictest2168218-21cron12:40:002
1754579991714,2cyclictest1989370-21kthreadcore10:15:171
1754579991713,3cyclictest2020502-21kthreadcore10:40:181
1754579991713,3cyclictest1994577-21irqcore10:20:161
1754579991712,3cyclictest1794252-21idleruntime-cro07:40:001
175457999171,13cyclictest2162274-21apt-get12:35:141
1754576991714,2cyclictest2168246-21idleruntime-cro12:40:000
1754576991714,2cyclictest2144008-21cut12:20:160
1754576991714,2cyclictest0-21swapper/011:55:010
1754576991710,5cyclictest493-21dbus-daemon09:00:000
1754585991613,2cyclictest0-21swapper/308:40:013
1754581991612,3cyclictest2043653-21sendmail11:00:002
1754581991612,3cyclictest1-21systemd07:38:282
1754579991613,2cyclictest0-21swapper/108:00:001
1754579991612,3cyclictest2100092-21fschecks_time11:45:141
1754579991611,3cyclictest0-21swapper/108:25:001
1754579991611,3cyclictest0-21swapper/107:55:001
1754579991610,4cyclictest1760247-21grep07:10:251
175457699169,5cyclictest3763191-21systemd-journal07:55:000
1754576991612,3cyclictest2131530-21gltestperf12:10:160
17485542164,7sleep20-21swapper/207:05:122
1754585991512,2cyclictest1919760-21gltestperf09:20:143
1754585991512,2cyclictest1906714-21sh09:10:003
1754585991512,2cyclictest1851370-21gltestperf08:25:143
1754585991512,2cyclictest0-21swapper/308:45:003
1754581991512,2cyclictest2075004-21cpuspeed_turbos11:25:142
1754581991512,2cyclictest1930830-21systemctl09:25:222
1754581991512,2cyclictest0-21swapper/208:05:002
1754581991511,3cyclictest1826201-21cpuspeed_turbos08:05:132
1754581991511,2cyclictest0-21swapper/209:25:012
1754581991511,2cyclictest0-21swapper/208:35:192
1754579991512,2cyclictest509-21rs:main1
1754579991511,3cyclictest1863800-21gltestperf08:35:141
175457699155,9cyclictest2873-21cron07:30:000
1754576991512,2cyclictest2013255-21irqcore10:35:160
1754576991512,2cyclictest1800724-21/usr/sbin/munin07:45:180
1754576991512,2cyclictest1775043-21unixbench-2d07:20:250
1754576991511,3cyclictest2156413-21gltestperf12:30:150
1754576991511,2cyclictest0-21swapper/007:45:010
1754585991411,2cyclictest2137880-21grep12:15:173
175458199149,4cyclictest1984526-21tr10:10:192
175458199148,4cyclictest523-21gdbus10:55:012
175458199148,4cyclictest523-21gdbus10:55:002
175458199142,2cyclictest295798rtkit-daemon12:15:002
1754581991412,1cyclictest0-21swapper/210:20:002
1754581991411,2cyclictest1887609-21unixbench_multi08:50:252
1754581991411,2cyclictest1812920-21munin-plugin-st07:55:002
1754581991411,2cyclictest1800893-21cpuspeed_turbos07:45:162
1754581991411,2cyclictest0-21swapper/211:55:222
1754581991411,1cyclictest0-21swapper/207:25:172
1754581991410,3cyclictest2050393-21gltestperf11:05:152
1754581991410,3cyclictest1813518-21cpuspeed_turbos07:55:142
1754581991410,2cyclictest0-21swapper/208:20:172
175457999148,4cyclictest493-21dbus-daemon10:00:001
175457999148,4cyclictest2008016-21ps10:30:161
175457999147,5cyclictest493-21dbus-daemon08:30:011
1754579991411,2cyclictest2099564-21cron11:45:011
1754579991411,2cyclictest0-21swapper/111:00:161
1754579991410,3cyclictest1832808-21sed08:10:181
1754579991410,3cyclictest0-21swapper/110:25:181
1754579991410,2cyclictest0-21swapper/109:45:001
1754579991410,2cyclictest0-21swapper/108:10:001
175457699148,4cyclictest1794256-21ls07:40:000
175457699141,3cyclictest2043635-21munin-run11:00:000
1754576991412,1cyclictest1900347-21kworker/0:1+kacpid11:21:040
1754576991412,1cyclictest0-21swapper/011:30:000
1754576991411,2cyclictest502-21systemd-logind11:59:590
1754576991411,2cyclictest1900347-21kworker/0:1+kacpid12:20:000
1754576991411,2cyclictest1900347-21kworker/0:1+kacpid12:05:160
1754576991411,2cyclictest0-21swapper/007:15:010
1754576991410,3cyclictest2093757-21fschecks_count11:40:140
1754576991410,2cyclictest1907284-21gltestperf09:10:150
1754576991410,2cyclictest0-21swapper/009:45:190
1754585991311,1cyclictest1850735-21turbostat.cron08:25:003
1754585991310,2cyclictest2093280-21cron11:40:013
175458199139,2cyclictest312-21jbd2/sda2-810:40:322
175458199139,2cyclictest2062254-21idleruntime-cro11:15:012
175458199138,4cyclictest1943956-21cron09:40:002
175458199137,4cyclictest3763191-21systemd-journal10:30:002
175458199137,4cyclictest1893447-21ntp_states08:55:212
175458199133,2cyclictest295798rtkit-daemon11:00:132
175458199131,3cyclictest2956-21rtkit-daemon07:17:072
1754581991311,1cyclictest0-21swapper/212:02:242
1754581991311,1cyclictest0-21swapper/211:55:022
1754581991311,1cyclictest0-21swapper/207:15:012
1754581991310,2cyclictest2150801-21tr12:25:142
1754581991310,2cyclictest2006766-21fschecks_count10:30:132
1754581991310,2cyclictest1975507-21/usr/sbin/munin10:05:242
1754581991310,2cyclictest1957006-21cpuspeed_turbos09:50:122
1754581991310,2cyclictest1894667-21cpuspeed_turbos09:00:142
175458199130,11cyclictest0-21swapper/209:58:262
175457999139,3cyclictest2118482-21sh11:59:591
175457999139,3cyclictest1975606-21cpuspeed_turbos10:05:181
175457999139,3cyclictest0-21swapper/108:15:171
175457999138,3cyclictest2068876-21tr11:20:141
175457999137,4cyclictest3763191-21systemd-journal12:04:541
1754579991310,2cyclictest1956979-21cpuspeed_turbos09:50:121
1754579991310,2cyclictest1955985-21unixbench_singl09:45:261
1754579991310,2cyclictest0-21swapper/107:15:021
175457999130,1cyclictest0-21swapper/110:45:241
175457999130,1cyclictest0-21swapper/108:30:141
175457699139,3cyclictest1875335-21kworker/0:0+kacpi_notify09:00:170
175457699139,3cyclictest1-21systemd09:55:010
175457699139,2cyclictest1993384-21tail10:15:240
175457699139,2cyclictest0-21swapper/011:10:000
175457699131,3cyclictest2068441-21munin-run11:20:000
1754576991311,1cyclictest2149321-21kworker/0:2+kacpid12:25:110
1754576991311,1cyclictest1875335-21kworker/0:0+kacpid08:50:180
1754576991311,1cyclictest1754570-21cyclictest08:10:160
1754576991310,2cyclictest2105773-21cron11:50:000
1754576991310,2cyclictest1875335-21kworker/0:0+kacpid09:28:340
1754576991310,2cyclictest0-21swapper/010:35:000
1754576991310,2cyclictest0-21swapper/009:55:160
1754576991310,2cyclictest0-21swapper/009:15:200
175457699130,5cyclictest2020101-21sed10:40:180
175457699130,4cyclictest1924873-21sed09:20:230
175458599126,4cyclictest523-21gdbus09:15:003
175458199129,2cyclictest1969602-21gltestperf10:00:152
175458199127,4cyclictest493-21dbus-daemon09:20:012
175458199127,4cyclictest3763191-21systemd-journal08:30:002
175458199127,4cyclictest1955884-21grep09:45:252
175458199126,4cyclictest3763191-21systemd-journal09:15:002
175458199124,2cyclictest401ktimers/207:35:022
175458199121,2cyclictest295798rtkit-daemon10:23:142
175458199120,3cyclictest0-21swapper/208:44:532
175457999129,2cyclictest0-21swapper/111:00:001
175457999128,3cyclictest2150167-21gltestperf12:25:121
175457999128,3cyclictest2031832-21sed10:50:161
175457999128,3cyclictest2031832-21sed10:50:151
175457999128,3cyclictest0-21swapper/107:30:001
175457999128,2cyclictest0-21swapper/112:20:171
175457999127,4cyclictest1824588-21sort08:00:181
175457999126,5cyclictest493-21dbus-daemon10:35:231
175457999126,1cyclictest1775517-21cron07:25:001
175457999125,5cyclictest3763191-21systemd-journal07:35:021
175457699129,2cyclictest1975818-21gltestperf10:05:190
175457699128,3cyclictest2044199-21gltestperf11:00:170
175457699128,3cyclictest1901058-21gltestperf09:05:160
175457699128,3cyclictest1782085-21cpuspeed_turbos07:30:140
175457699128,2cyclictest1944578-21grep09:40:180
175457699126,5cyclictest1931873-21apt-get09:30:130
175457699121,9cyclictest2000341-21apt-get10:25:150
175457699121,10cyclictest0-21swapper/008:05:130
175458599118,2cyclictest0-21swapper/309:25:173
175458599116,4cyclictest1860113-21cut08:30:193
175458599110,11cyclictest0-21swapper/309:31:213
175458199119,1cyclictest0-21swapper/211:23:392
175458199119,1cyclictest0-21swapper/207:20:172
175458199118,2cyclictest0-21swapper/210:45:212
175458199118,2cyclictest0-21swapper/210:35:262
175458199117,3cyclictest1931925-21cpuspeed_turbos09:30:142
175458199112,2cyclictest2956-21rtkit-daemon07:42:202
175458199110,10cyclictest0-21swapper/209:41:362
175457999118,2cyclictest1969395-21cpuspeed_turbos10:00:141
175457999118,2cyclictest0-21swapper/111:35:141
175457999118,2cyclictest0-21swapper/108:45:171
175457999117,3cyclictest2055613-21unixbench_singl11:05:241
175457999115,4cyclictest1982701-21sed10:10:171
175457699117,2cyclictest0-21swapper/010:15:000
175457699115,5cyclictest1-21systemd10:55:010
175457699115,5cyclictest1-21systemd10:55:000
175457699114,4cyclictest2061479-21/usr/sbin/munin11:10:220
175457699111,8cyclictest3763191-21systemd-journal07:58:280
17543502112,2sleep10-21swapper/107:09:081
175458599100,9cyclictest1896854-21cut09:00:183
175458199107,2cyclictest2156347-21tail12:30:162
175457999107,2cyclictest1869976-21gltestperf08:40:121
175457999107,2cyclictest0-21swapper/111:10:181
175457699108,1cyclictest1900347-21kworker/0:1+kacpid10:20:130
175457699107,2cyclictest0-21swapper/010:00:200
175457699100,9cyclictest2081259-21cut11:30:180
17540952106,2sleep00-21swapper/007:05:510
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional