You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-11 - 17:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #f, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails
  
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackfslot7s.osadl.org (updated Sun Jan 11, 2026 00:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
18905292176145,28sleep10-21swapper/119:09:041
18905202149104,27sleep20-21swapper/219:08:572
18904182147117,25sleep00-21swapper/019:07:400
18905012145100,26sleep30-21swapper/319:08:423
189078799290,19cyclictest1912343-21cat20:15:021
189078799241,19cyclictest1970678-21ntpq23:15:171
189078799241,11cyclictest1922499-21irqstats20:45:141
189078599246,9cyclictest1971495-21awk23:19:590
189078599241,5cyclictest1904028-21munin-run19:50:000
1890785992415,7cyclictest1962012-21awk22:50:010
189078799239,12cyclictest1984399-21apt-get00:00:001
189078799238,3cyclictest1934852-21apt-get21:25:011
1890787992312,7cyclictest1927154-21packagekitd21:00:011
189078799231,12cyclictest1946981-21ntpq22:00:181
189078799231,11cyclictest1906580-21python319:55:211
189078799230,15cyclictest0-21swapper/119:35:151
1890785992318,3cyclictest0-21swapper/020:40:000
1890795992217,3cyclictest1920348-21idleruntime-cro20:39:593
189078799228,11cyclictest1913168-21python320:15:221
189078799221,11cyclictest1955085-21python322:25:201
189078799221,10cyclictest1950130-21ntpq22:10:181
189078799221,10cyclictest1937524-21ntpq21:30:181
189078799221,10cyclictest1930112-21munin-run21:09:591
189078799220,19cyclictest1986553-21apt-get00:05:131
189078799220,16cyclictest1941927-21diskstats21:45:141
189078799220,12cyclictest1946292-21cat22:00:001
1890785992218,2cyclictest1908997-21kworker/u16:0-events_unbound20:20:010
1890795992118,2cyclictest0-21swapper/323:15:163
1890790992117,2cyclictest1926688-21idleruntime-cro20:59:592
1890790992117,2cyclictest1926688-21idleruntime-cro20:59:592
1890790992117,2cyclictest0-21swapper/219:30:002
1890790992115,4cyclictest1941453-21idleruntime-cro21:44:592
1890787992117,2cyclictest659-21in:imuxsock00:17:241
1890787992112,8cyclictest1935073-21latency_hist21:25:011
189078799211,18cyclictest1892286-21perf19:15:011
189078799211,17cyclictest1952511-21dump-pmu-power22:20:001
189078799211,16cyclictest1920364-21perf20:40:001
189078799211,11cyclictest1975758-21ntpq23:30:191
189078799210,17cyclictest0-21swapper/100:00:021
1890785992113,6cyclictest1939875-21systemctl21:40:000
189078599211,12cyclictest1900861-21munin-run19:40:000
189079599201,3cyclictest1926956-21apt-config21:00:003
189079599201,3cyclictest1926956-21apt-config21:00:003
1890795992012,6cyclictest1941447-21sh21:44:593
189079099209,8cyclictest1971488-21systemctl23:20:002
1890790992017,2cyclictest661-21rs:main2
1890790992016,3cyclictest421ktimers/220:05:182
1890790992011,6cyclictest645-21dbus-daemon19:40:012
189079099201,16cyclictest1962018-21idleruntime-cro22:50:002
189078799201,9cyclictest1973884-21ntpq23:25:171
189078799201,9cyclictest1926077-21python320:55:191
189078799201,9cyclictest1926077-21python320:55:181
1890787992014,1cyclictest1952764-21/usr/sbin/munin22:20:131
189078799201,17cyclictest1914562-21ntpq20:20:181
189078799201,16cyclictest1979675-21idleruntime-cro23:44:591
189078799201,16cyclictest1931689-21awk21:14:591
189078799200,8cyclictest1940336-21cstates21:40:121
189078799200,17cyclictest1961197-21ntpq22:45:171
189078599208,5cyclictest1926683-21perf21:00:000
189078599208,5cyclictest1926683-21perf21:00:000
1890785992015,3cyclictest0-21swapper/020:10:010
189079599191,15cyclictest13236-21gnome-shell20:46:593
189079599190,18cyclictest13236-21gnome-shell19:15:583
1890790991916,2cyclictest1992673-21systemd-run00:25:012
1890790991915,3cyclictest421ktimers/200:25:132
189078799199,8cyclictest1959709-21/usr/sbin/munin22:40:191
189078799198,9cyclictest1896315-21df19:25:121
189078799191,15cyclictest1993553-21meminfo00:25:161
189078799191,15cyclictest1944621-21basename21:55:011
189078799191,15cyclictest1902453-21munin-run19:45:001
189078799191,13cyclictest1972253-21ntpq23:20:161
189078799191,13cyclictest1957734-21diskstats22:35:201
189078799191,10cyclictest1991837-21ntpq00:20:171
189078799191,10cyclictest1967738-21python323:05:221
189078799190,18cyclictest1980947-21seq23:46:271
189078799190,17cyclictest1977965-21taskset23:38:531
189078599199,2cyclictest191rcu_preempt22:00:190
189078599195,5cyclictest16-21ksoftirqd/022:30:010
1890785991913,5cyclictest1938287-21awk21:34:590
1890785991912,5cyclictest1901530-21memory19:40:150
1890785991912,2cyclictest191rcu_preempt21:50:010
1890785991912,2cyclictest191rcu_preempt20:35:000
1890785991910,7cyclictest753-21ntpd23:55:160
189078599190,17cyclictest1994264-21taskset00:27:310
189079599184,6cyclictest1992761-10apt-config00:25:013
1890795991815,2cyclictest645-21dbus-daemon00:00:013
1890795991813,3cyclictest0-21swapper/321:19:593
1890795991811,5cyclictest1946171-21cron22:00:013
189079599180,4cyclictest1955439-21runrttasks22:27:093
189079599180,16cyclictest1964236-21/usr/sbin/munin22:55:163
189079099187,5cyclictest1948601-10ld22:05:182
189079099187,5cyclictest1908019-10ld20:00:202
189079099181,9cyclictest1915358-21munin-run20:25:012
1890790991813,3cyclictest1892011-21seq19:11:502
1890790991813,3cyclictest13485-21evolution-alarm23:59:592
1890790991812,5cyclictest13248-21gdbus20:35:062
189079099180,16cyclictest1957921-21/usr/sbin/munin22:35:222
189078799186,10cyclictest1910334-21sh20:10:001
189078799181,9cyclictest1895263-21python319:20:181
1890787991811,4cyclictest1988509-21/usr/sbin/munin00:10:141
189078799181,14cyclictest1982813-21munin-run23:55:001
1890787991811,4cyclictest1932861-21/usr/sbin/munin21:15:201
189078799181,13cyclictest1904031-21perf19:50:001
189078799180,9cyclictest1956138-21df22:30:121
189078799180,16cyclictest1939872-21awk21:39:591
189078799180,16cyclictest1916777-21taskset20:28:121
189078799180,15cyclictest1894290-21awk19:20:011
189078799180,13cyclictest1907653-21df20:00:131
189078599186,4cyclictest1897437-21munin-run19:30:010
1890785991812,4cyclictest13065-10wireplumber21:25:010
1890785991810,2cyclictest191rcu_preempt23:30:000
189078599180,16cyclictest795-21runrttasks22:59:590
189078599180,16cyclictest795-21runrttasks22:40:140
94898170,1rtkit-daemon0-21swapper/021:50:400
189079599179,7cyclictest1947395-21/usr/sbin/munin22:00:213
1890795991711,4cyclictest355-21systemd-journal22:48:243
1890795991711,4cyclictest1990494-21/usr/sbin/munin00:15:193
1890795991711,4cyclictest1897091-21chrt19:25:473
189079599170,4cyclictest1944484-21taskset21:53:373
189079599170,15cyclictest1973846-21/usr/sbin/munin23:25:163
189079599170,15cyclictest1899295-21true19:35:003
189079099178,6cyclictest753-21ntpd00:35:182
189079099178,4cyclictest1905853-10awk19:55:132
189079099174,10cyclictest645-21dbus-daemon20:15:002
189079099171,6cyclictest1-21systemd21:15:002
1890790991714,2cyclictest659-21in:imuxsock21:25:002
1890790991710,5cyclictest13065-10wireplumber22:35:012
189079099170,5cyclictest1939657-21chrt21:37:342
189079099170,15cyclictest1959660-21ntpq22:40:182
189078799178,7cyclictest1965408-21seq23:00:041
1890787991712,4cyclictest1905502-21runrttasks19:53:581
1890787991711,4cyclictest1925031-21runrttasks20:54:201
189078799171,13cyclictest794-21snmpd20:30:181
189078799171,13cyclictest1995420-21proc_pri00:30:191
189078799171,13cyclictest1948434-21memory22:05:161
189078799171,13cyclictest1898193-21hddtemp_smartct19:30:151
189078799171,12cyclictest1968811-21hddtemp_smartct23:10:141
189078799170,9cyclictest1996703-21meminfo00:35:161
189078799170,15cyclictest0-21swapper/122:55:011
189078599173,5cyclictest16-21ksoftirqd/023:40:000
189078599172,13cyclictest794-21snmpd21:19:030
1890785991714,2cyclictest656-21systemd-logind20:26:490
1890785991710,6cyclictest650-21polkitd23:25:000
189078599170,4cyclictest795-21runrttasks00:23:010
189078599170,4cyclictest1892259-21cron19:15:000
189078599170,16cyclictest1995355-21/usr/sbin/munin00:30:190
189078599170,15cyclictest1954090-21cron22:25:010
189078599170,15cyclictest1924921-21taskset20:52:560
189078599170,15cyclictest1920608-21runrttasks20:40:090
189079599169,5cyclictest1973006-21chrt23:24:193
189079599168,5cyclictest753-21ntpd23:35:163
189079599166,5cyclictest1935248-21cat21:25:003
1890795991611,3cyclictest1936063-21ps21:25:203
189079599161,12cyclictest1960442-21idleruntime-cro22:45:013
189079599160,5cyclictest1906253-21memory19:55:163
189079599160,15cyclictest1952754-21taskset22:20:023
189079599160,14cyclictest1987653-21taskset00:07:313
189079099169,5cyclictest1946192-21run-parts22:00:012
1890790991613,1cyclictest0-21swapper/220:45:002
1890790991612,2cyclictest1904577-21irqstats19:50:142
1890790991611,3cyclictest795-21runrttasks21:29:232
189079099161,12cyclictest13236-21gnome-shell00:14:582
1890790991611,1cyclictest0-21swapper/222:00:152
1890790991610,4cyclictest795-21runrttasks21:50:142
1890790991610,4cyclictest1968043-21runrttasks23:06:332
1890790991610,1cyclictest1975154-21/usr/sbin/munin23:30:132
189079099160,5cyclictest1985365-21/usr/sbin/munin00:00:162
189079099160,4cyclictest1964885-21taskset22:56:442
189079099160,4cyclictest1955324-21seq22:25:402
189078799161,14cyclictest0-21swapper/120:40:151
189078799160,12cyclictest1962269-21apt-get22:50:121
189078599169,5cyclictest1895646-21taskset19:22:270
189078599168,5cyclictest1915386-21pluginstate20:25:000
189078599168,2cyclictest191rcu_preempt22:35:010
189078599166,2cyclictest191rcu_preempt21:45:000
189078599162,7cyclictest0-21swapper/020:10:170
1890785991613,2cyclictest1899271-21idleruntime-cro19:35:010
1890785991611,3cyclictest1979901-21/usr/sbin/munin23:45:110
1890785991610,4cyclictest1948263-21hddtemp_smartct22:05:140
189078599160,4cyclictest795-21runrttasks00:06:020
189079599159,4cyclictest1949310-21taskset22:09:373
189079599157,5cyclictest1916956-21http20:30:013
189079599156,4cyclictest1914046-10awk20:20:123
189079599151,7cyclictest1918784-21perf20:35:003
189079599151,5cyclictest1-21systemd22:20:013
1890795991512,2cyclictest1979495-21chrt23:43:023
1890795991512,2cyclictest1903432-21python319:45:213
1890795991512,2cyclictest1900605-21sshd19:37:023
189079599151,13cyclictest1994739-21apt-get00:30:123
1890795991511,2cyclictest1892049-21sshd19:12:183
1890795991511,2cyclictest1-21systemd21:50:003
1890795991511,2cyclictest0-21swapper/321:40:003
189079599150,6cyclictest661-21rs:main3
189079599150,4cyclictest1901594-21/usr/sbin/munin19:40:173
189079599150,13cyclictest1975179-21apt-get23:30:123
189079599150,13cyclictest1921933-21cron20:45:003
189079599150,12cyclictest13236-21gnome-shell22:36:593
189079099159,4cyclictest795-21runrttasks19:19:372
189079099159,4cyclictest1903994-21runrttasks19:49:482
189079099156,6cyclictest355-21systemd-journal19:35:002
189079099151,7cyclictest1973079-21munin-run23:25:002
1890790991511,3cyclictest1928204-21runrttasks21:00:592
1890790991511,2cyclictest1952510-21munin-run22:20:012
189079099151,10cyclictest1913769-21awk20:20:012
1890790991510,3cyclictest13065-10wireplumber00:35:002
189078599158,5cyclictest1989993-21uname00:15:150
189078599158,5cyclictest1935775-21memory21:25:160
189078599157,6cyclictest1945416-21ntpq21:55:170
189078599155,2cyclictest0-21swapper/022:15:170
189078599152,9cyclictest1927199-21/usr/sbin/munin21:00:150
1890785991512,2cyclictest1905334-21taskset19:51:500
189078599151,12cyclictest1996893-21ntpq00:35:190
189078599151,12cyclictest1894282-21basename19:20:000
189078599151,10cyclictest1982822-21awk23:55:000
189078599150,6cyclictest1976518-21munin-run23:35:000
189078599150,2cyclictest1949905-21irqstats22:10:140
189079599148,4cyclictest1962788-21ld22:50:163
189079599148,4cyclictest1909023-21awk20:05:123
1890795991412,1cyclictest1965728-21/usr/sbin/munin23:00:143
1890795991412,1cyclictest0-21swapper/321:07:523
1890795991410,2cyclictest1908384-21users20:00:233
189079599140,2cyclictest1980645-21python323:45:203
189079599140,1cyclictest1938270-21cron21:35:003
189079599140,1cyclictest1910584-21/usr/sbin/munin20:10:153
189079599140,13cyclictest1930334-21runrttasks21:10:023
189079599140,13cyclictest0-21swapper/321:00:143
189079599140,12cyclictest1968221-21taskset23:08:483
189079099147,4cyclictest355-21systemd-journal20:29:202
189079099141,9cyclictest1949847-21hddtemp_smartct22:10:142
189079099141,4cyclictest1933276-21basename21:19:592
189079099141,12cyclictest0-21swapper/223:41:312
1890790991410,3cyclictest1987209-21pmu-power00:05:202
1890790991410,3cyclictest1901114-21/usr/sbin/munin19:40:142
1890790991410,2cyclictest1973494-21cstates23:25:112
189079099140,13cyclictest0-21swapper/222:20:112
189079099140,13cyclictest0-21swapper/221:05:162
189078599147,5cyclictest1966751-21awk23:05:000
189078599147,5cyclictest1907959-21ld20:00:170
189078599146,6cyclictest1957725-21df22:35:180
189078599141,1cyclictest1968592-21apt-get23:10:120
1890785991411,2cyclictest1989123-21taskset00:11:100
1890785991410,3cyclictest1922185-21/usr/sbin/munin20:45:220
1890785991410,1cyclictest191rcu_preempt22:50:120
189078599140,4cyclictest0-21swapper/023:40:140
189078599140,2cyclictest1906936-21taskset19:57:120
189078599140,12cyclictest1986155-21taskset00:03:280
189079599136,5cyclictest1988638-21ntpq00:10:163
189079599131,9cyclictest1905596-21munin-run19:55:003
189079599131,8cyclictest1982819-21ls23:54:593
189079599131,8cyclictest1949841-21hddtemp_smartct22:10:143
189079599131,8cyclictest1913772-21perf20:20:003
1890795991311,1cyclictest1894520-21/usr/sbin/munin19:20:153
1890795991310,2cyclictest0-21swapper/323:10:163
189079599130,2cyclictest13236-21gnome-shell00:29:093
189079599130,12cyclictest1984973-21/usr/sbin/munin00:00:113
189079099138,4cyclictest1923380-21sshd20:48:222
189079099131,9cyclictest1969906-21munin-run23:15:002
189079099131,6cyclictest1942248-21ntpq21:45:182
189079099130,4cyclictest759-21mta-sts-daemon22:50:172
189079099130,13cyclictest0-21swapper/220:30:202
189079099130,12cyclictest1976729-21/usr/sbin/munin23:35:102
189079099130,11cyclictest1924853-21seq20:52:062
1890785991310,2cyclictest1967405-21/usr/sbin/munin23:05:170
189078599130,2cyclictest1930592-21hddtemp_smartct21:10:140
189079599121,8cyclictest1925100-21munin-run20:55:003
189079099128,3cyclictest1965416-21/usr/sbin/munin23:00:142
189079099123,6cyclictest1937744-21sed21:30:212
1890790991211,0cyclictest0-21swapper/200:15:522
189079099120,4cyclictest1895675-21chrt19:22:502
189079099120,12cyclictest0-21swapper/223:45:202
1890785991210,1cyclictest0-21swapper/021:05:120
189079599111,7cyclictest794-21snmpd22:30:173
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional