You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-08 - 21:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Sat Aug 08, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1820745994639,5cyclictest335557-21systemd-journal09:30:001
1820748994337,4cyclictest335557-21systemd-journal09:20:002
1820748994135,4cyclictest2049137-21latency_hist10:50:002
1820745994034,4cyclictest335557-21systemd-journal10:05:001
1820745993630,4cyclictest2122379-21latency_hist12:00:021
1820742993631,3cyclictest2091172-21/usr/sbin/munin11:30:160
1820748993529,4cyclictest1877521-21latency_hist08:05:012
1820745993429,3cyclictest1820659-21latency_hist07:10:011
1820745993427,5cyclictest2070459-21cp11:10:011
1820752993329,3cyclictest2122643-21apt-get12:00:123
1820752993328,4cyclictest335557-21systemd-journal09:05:003
1820748993326,5cyclictest1981144-21latency_hist09:45:012
1820742993328,4cyclictest2069826-21latency_hist11:10:000
1820752993128,2cyclictest2002271-21kworker/u8:2+flush-259:012:30:173
1820752993128,2cyclictest1882946-21apt-get08:10:133
1820745993127,3cyclictest2002271-21kworker/u8:2+flush-259:010:20:001
1820745993126,3cyclictest335557-21systemd-journal11:25:001
1820745993126,3cyclictest2007646-21/usr/sbin/munin10:10:201
1820742993125,4cyclictest2070517-21apt-get11:10:120
1820752993025,3cyclictest1955336-21latency_hist09:20:013
1820748993024,4cyclictest1902476-21ntpq08:25:222
1820742993024,4cyclictest1908474-21latency_hist08:35:010
1820752992926,2cyclictest1903317-21kworker/u8:1+flush-259:009:10:213
1820748992923,4cyclictest2127985-21diskstats12:05:152
1820742992915,12cyclictest1970859-21latency_hist09:35:010
1820742992824,3cyclictest1986561-21apt-get09:50:120
1820745992721,4cyclictest1939829-21latency_hist09:05:011
1820742992721,4cyclictest2153433-21latency_hist12:30:010
1820742992719,5cyclictest2138339-21diskstats12:15:160
1820742992714,12cyclictest335557-21systemd-journal12:40:010
1820752992622,3cyclictest335557-21systemd-journal07:15:013
1820752992622,2cyclictest0-21swapper/310:15:003
1820748992621,3cyclictest335557-21systemd-journal09:14:562
1820748992620,4cyclictest1996817-21cp10:00:012
1820742992622,2cyclictest2002271-21kworker/u8:2+flush-259:011:55:000
1820752992522,2cyclictest1903317-21kworker/u8:1+flush-259:008:40:173
1820752992522,2cyclictest1866994-21kworker/u8:3+events_unbound08:00:013
1820752992521,3cyclictest2075044-21kworker/u8:1+flush-259:012:30:013
1820748992522,2cyclictest2002271-21kworker/u8:2+events_unbound10:10:012
1820748992518,5cyclictest2121456-21ntpq11:55:222
1820748992517,5cyclictest1913649-21latency_hist08:40:012
1820745992520,4cyclictest335557-21systemd-journal10:45:001
1820745992518,6cyclictest1934204-21latency_hist09:00:011
182074299251,22cyclictest1-21systemd11:25:000
1820748992419,3cyclictest1934234-21gpgv09:00:002
1820745992419,4cyclictest335557-21systemd-journal12:30:001
182074299249,13cyclictest335557-21systemd-journal08:09:380
1820752992320,2cyclictest1903317-21kworker/u8:1+flush-259:009:40:253
1820748992320,2cyclictest2075044-21kworker/u8:1+events_unbound11:45:282
1820748992318,4cyclictest1940069-21kworker/u8:4+flush-259:009:50:002
1820748992318,3cyclictest1986536-21/usr/sbin/munin09:50:152
1820745992317,4cyclictest1986296-21latency_hist09:50:011
1820742992319,3cyclictest1898382-21apt-get08:25:120
1820742992310,12cyclictest0-21swapper/009:00:000
1820752992219,2cyclictest2002271-21kworker/u8:2+events_unbound11:15:183
1820752992219,2cyclictest1940069-21kworker/u8:4+flush-259:009:45:243
1820752992218,3cyclictest509-21rs:main3
1820752992218,3cyclictest509-21rs:main3
1820752992217,4cyclictest1975987-21latency_hist09:40:013
1820752992216,4cyclictest2096078-21latency_hist11:35:013
1820748992217,4cyclictest2116769-21latency_hist11:55:002
1820748992217,4cyclictest1888074-21/usr/sbin/munin08:15:222
1820748992215,5cyclictest1867190-21latency_hist07:55:012
1820745992211,9cyclictest1997011-21apt-key10:00:011
1820742992218,3cyclictest1872637-21apt-get08:00:120
182074299221,2cyclictest1986290-21awk09:50:000
182074299220,1cyclictest2004796-21kthreadcore10:05:180
1820752992118,2cyclictest2111867-21apt-get11:50:133
1820752992118,2cyclictest1940069-21kworker/u8:4+flush-259:010:15:203
1820745992117,3cyclictest335557-21systemd-journal11:35:001
1820745992117,3cyclictest335557-21systemd-journal08:19:481
1820742992117,3cyclictest2012544-21latency_hist10:15:010
1820742992116,4cyclictest1826127-21apt-get07:15:120
182074299210,3cyclictest2037428-21needreboot10:35:230
182074299210,2cyclictest1914372-21sed08:40:160
1820752992017,2cyclictest1903317-21kworker/u8:1+flush-259:008:55:193
1820752992017,2cyclictest1903317-21kworker/u8:1+events_unbound08:45:233
1820752992015,3cyclictest2148246-21latency_hist12:25:013
1820752992013,5cyclictest335557-21systemd-journal10:01:383
1820748992017,2cyclictest1830998-21kworker/u8:1+events_unbound07:30:182
1820748992016,3cyclictest2116578-21kworker/u8:0+flush-259:012:10:162
1820748992016,3cyclictest1997143-21base6410:00:012
1820748992016,3cyclictest1903317-21kworker/u8:1+flush-259:008:40:222
1820745992016,3cyclictest2158570-21cron12:35:001
1820745992014,4cyclictest2132735-21latency_hist12:10:011
1820745992010,9cyclictest1976321-21cpuspeed_turbos09:40:131
182074299208,10cyclictest0-21swapper/007:30:250
1820742992018,1cyclictest0-21swapper/008:20:000
1820742992016,3cyclictest2086042-21apt-get11:25:120
1820742992016,3cyclictest2086042-21apt-get11:25:120
1820742992015,4cyclictest2143330-21apt-get12:20:140
182074299201,4cyclictest1893646-21/usr/sbin/munin08:20:160
1820742992014,4cyclictest2158590-21latency_hist12:35:010
182074299200,1cyclictest2021928-21ntp_kernel_err10:20:210
1820752991916,2cyclictest2090952-21systemd-run11:30:003
1820752991916,2cyclictest2090952-21systemd-run11:30:003
1820752991916,2cyclictest1830998-21kworker/u8:1+flush-259:007:30:253
1820752991915,3cyclictest335557-21systemd-journal10:30:013
1820752991915,3cyclictest2962391-21kworker/u8:3+flush-259:007:25:163
1820752991915,3cyclictest1877750-21kworker/u8:0+flush-259:008:05:213
1820748991916,2cyclictest0-21swapper/211:45:002
1820748991915,3cyclictest2049154-21kworker/u8:3+flush-259:011:00:152
1820745991915,3cyclictest2017964-21apt-get10:20:121
1820745991915,3cyclictest2002271-21kworker/u8:2+flush-259:011:35:191
1820745991915,3cyclictest1862276-21apt-get07:50:111
1820745991912,5cyclictest1871509-21ntpq07:55:211
182074299197,11cyclictest1847002-21fschecks_time07:35:160
182074299194,13cyclictest1913163-21unixbench_multi08:35:250
182074299191,6cyclictest1825040-21ntpq07:10:220
1820742991916,2cyclictest2028001-21sh10:30:000
1820742991915,3cyclictest1830998-21kworker/u8:1+flush-259:007:45:180
182074299191,16cyclictest1831743-21irqcore07:20:180
182074299190,3cyclictest1935399-21irqcore09:00:170
182074299190,18cyclictest2059965-21fschecks_count11:00:150
1820752991816,1cyclictest503-21tpm2-abrmd10:00:013
1820752991815,2cyclictest507-21in:imuxsock07:25:003
1820752991815,2cyclictest2064673-21sh11:05:003
1820752991815,2cyclictest2002271-21kworker/u8:2+flush-259:012:35:263
1820752991815,2cyclictest2002271-21kworker/u8:2+events_unbound10:05:253
1820752991815,2cyclictest1940069-21kworker/u8:4-ext4-rsv-conversion10:35:013
1820752991814,3cyclictest1940069-21kworker/u8:4+events_unbound10:35:253
1820748991815,2cyclictest1903317-21kworker/u8:1+flush-259:009:30:242
1820748991815,2cyclictest1830998-21kworker/u8:1+flush-259:007:45:232
1820748991815,2cyclictest1830998-21kworker/u8:1+flush-259:007:40:192
1820748991814,3cyclictest509-21rs:main2
1820748991814,3cyclictest2070041-21gpgv11:10:012
1820745991816,1cyclictest2127559-21sh12:05:001
182074599181,5cyclictest1918770-21dump-pmu-power08:45:001
1820745991815,2cyclictest1959633-21ntp_kernel_pll_09:20:221
1820745991814,3cyclictest1861731-21kworker/u8:2+flush-259:007:50:011
182074299189,8cyclictest60350irq/132-enp0s29f1:rx-011:55:250
182074299184,12cyclictest1949398-21open_files09:10:230
182074299182,14cyclictest0-21swapper/008:15:000
182074299182,13cyclictest1924197-21apt-get08:50:120
182074299181,6cyclictest2079673-21ntpq11:15:230
182074299181,5cyclictest1840384-21ntpq07:25:200
182074299181,4cyclictest1996625-21sh10:00:000
182074299181,3cyclictest1940262-21diskstats09:05:150
182074299181,3cyclictest1940262-21diskstats09:05:150
182074299181,3cyclictest1856020-21ntpq07:40:230
1820742991813,4cyclictest2137002-21ntpq12:10:210
1820742991813,3cyclictest1955595-21apt-get09:20:120
1820742991812,4cyclictest2033182-21latency_hist10:35:010
182074299181,1cyclictest2016855-21ntpq10:15:260
182074299180,3cyclictest1997460-21tr10:00:120
1820752991715,2cyclictest2002271-21kworker/u8:2+flush-259:012:05:433
1820752991714,2cyclictest2018374-21irqcore10:20:173
1820752991713,3cyclictest1940069-21kworker/u8:4+flush-259:009:20:153
1820752991713,3cyclictest1903560-21apt-get08:30:123
1820752991713,3cyclictest1872389-21kworker/u8:2+flush-259:008:35:203
1820752991710,5cyclictest1929062-21latency_hist08:55:013
1820748991714,2cyclictest2022851-21sh10:25:002
1820748991714,2cyclictest2002271-21kworker/u8:2+events_unbound10:30:182
1820748991713,3cyclictest2148509-21apt-get12:25:142
1820748991713,3cyclictest1955570-21/usr/sbin/munin09:20:152
182074599177,9cyclictest1855859-21grep07:40:211
182074599174,11cyclictest335557-21systemd-journal11:16:381
182074599172,13cyclictest2054321-21cron10:55:001
182074599171,5cyclictest1913658-21munin-run08:39:591
1820745991714,2cyclictest1903317-21kworker/u8:1+flush-259:009:40:001
1820745991713,3cyclictest2075044-21kworker/u8:1+flush-259:011:50:111
182074599171,15cyclictest1908484-21munin-plugin-st08:35:001
182074599171,14cyclictest335557-21systemd-journal11:50:001
1820745991711,4cyclictest335557-21systemd-journal08:25:001
182074299179,7cyclictest60350irq/132-enp0s29f1:rx-010:50:190
1820742991713,3cyclictest335557-21systemd-journal09:36:380
182074299171,2cyclictest2059480-21idleruntime-cro11:00:000
182074299171,2cyclictest1923069-21ntpq08:45:210
182074299171,1cyclictest1954453-21ntpq09:15:220
182074299171,1cyclictest1871545-21ntpq07:55:220
182074299170,6cyclictest2105524-21ntpq11:40:200
182074299170,5cyclictest2129638-21grep12:05:200
182074299170,5cyclictest1976388-21cut09:40:140
182074299170,3cyclictest2038770-21tr10:40:130
182074299170,3cyclictest1867191-21turbostat.cron07:55:010
182074299170,2cyclictest2126718-21ntpq12:00:230
182074299170,15cyclictest2111604-21munin-plugin-st11:50:000
1820752991613,3cyclictest1903317-21kworker/u8:1+events_unbound09:25:013
1820752991613,3cyclictest1830998-21kworker/u8:1+events_unbound07:40:183
1820752991613,2cyclictest2002271-21kworker/u8:2+flush-259:010:40:253
1820752991613,2cyclictest1866994-21kworker/u8:3+events_unbound08:00:143
1820752991613,2cyclictest1-21systemd11:10:003
1820748991614,1cyclictest391rcuc/211:30:202
1820748991612,3cyclictest2075044-21kworker/u8:1+flush-259:012:00:192
182074599166,9cyclictest2971-21cron11:45:011
182074599161,5cyclictest1887855-21munin-run08:14:591
182074599161,5cyclictest1877881-21cpuspeed_turbos08:05:131
182074599161,4cyclictest1830974-21basename07:20:001
1820745991613,2cyclictest1765044-21kworker/u8:2+flush-259:007:35:211
1820745991611,4cyclictest1940069-21kworker/u8:4+flush-259:009:05:261
1820745991611,4cyclictest1940069-21kworker/u8:4+flush-259:009:05:261
1820745991611,3cyclictest509-21rs:main1
182074599161,13cyclictest2064940-21apt-get11:05:131
182074599160,2cyclictest1970655-21taskset09:32:421
182074599160,14cyclictest2147237-21needreboot12:20:211
182074299160,3cyclictest2096471-21grep11:35:140
182074299160,3cyclictest2044230-21apt10:45:130
182074299160,3cyclictest1960938-21cpuspeed_turbos09:25:110
182074299160,3cyclictest1820746-21latency_hist07:10:010
1820752991512,2cyclictest1898001-21kworker/u8:3+flush-259:008:25:153
1820752991511,2cyclictest0-21swapper/308:20:003
182075299150,4cyclictest20550251irqcore10:55:193
1820748991513,1cyclictest1940069-21kworker/u8:4+events_unbound09:05:222
1820748991513,1cyclictest1940069-21kworker/u8:4+events_unbound09:05:212
1820748991512,2cyclictest1830998-21kworker/u8:1+flush-259:008:20:162
1820748991510,4cyclictest1934874-21munin-node09:00:132
182074599153,10cyclictest335557-21systemd-journal10:05:011
182074599151,4cyclictest2027198-21grep10:25:241
182074599151,3cyclictest2090923-21cron11:30:011
182074599151,3cyclictest2090923-21cron11:30:001
182074599151,3cyclictest1872922-21fw_conntrack08:00:161
182074599151,2cyclictest335557-21systemd-journal09:55:001
182074599151,2cyclictest2064684-21sleep11:05:001
1820745991511,3cyclictest1872389-21kworker/u8:2+flush-259:008:50:001
182074599150,7cyclictest1949289-21ntpq09:10:221
182074599150,4cyclictest2037493-21sed10:35:241
182074599150,3cyclictest1838700-21ps07:25:181
1820752991412,2cyclictest2043575-21kworker/u8:0+events_unbound10:50:123
1820752991412,1cyclictest2098361-21kthreadcore11:35:183
1820752991410,3cyclictest1861731-21kworker/u8:2+flush-259:007:50:443
1820752991410,3cyclictest1830998-21kworker/u8:1+flush-259:008:20:243
1820748991411,2cyclictest2084815-21needreboot11:20:202
1820748991411,2cyclictest1991468-21kworker/u8:0+events_unbound10:10:152
1820748991411,2cyclictest1903783-21fschecks_time08:30:152
1820748991411,2cyclictest1830998-21kworker/u8:1+flush-259:007:55:212
1820748991410,3cyclictest2002271-21kworker/u8:2+flush-259:012:17:052
1820748991410,3cyclictest2002271-21kworker/u8:2+events_unbound12:35:242
1820748991410,3cyclictest2002271-21kworker/u8:2+events_unbound11:35:232
1820748991410,3cyclictest1830998-21kworker/u8:1+flush-259:008:05:152
182074599145,8cyclictest1950752-21gltestperf09:15:151
182074599143,10cyclictest1831867-21sed07:20:181
182074599141,4cyclictest2135777-21kthreadcore12:10:191
182074599141,4cyclictest2028491-21fschecks_count10:30:151
182074599141,2cyclictest2138269-21grep12:15:151
182074599141,1cyclictest1845656-21ntpq07:30:231
182074599141,1cyclictest0-21swapper/112:40:011
182074599140,4cyclictest2049136-21idleruntime-cro10:50:001
182074599140,4cyclictest1924439-21tune2fs08:50:141
182074599140,3cyclictest1899375-21kernelversion08:25:181
182075299139,3cyclictest1940069-21kworker/u8:4+flush-259:009:30:133
1820752991310,2cyclictest2002271-21kworker/u8:2+flush-259:011:45:253
182074899139,3cyclictest2962391-21kworker/u8:3+flush-259:007:25:252
1820748991310,2cyclictest2043575-21kworker/u8:0+flush-259:010:50:282
1820748991310,2cyclictest2023540-21irqcore10:25:182
1820748991310,2cyclictest1975975-21cron09:40:002
1820748991310,2cyclictest1877750-21kworker/u8:0+flush-259:008:10:142
182074599131,4cyclictest1821024-21cpuspeed_turbos07:10:131
182075299129,3cyclictest2962391-21kworker/u8:3+flush-259:007:15:163
182075299129,2cyclictest1851651-21kworker/u8:0+flush-259:007:40:013
182075299129,1cyclictest0-21swapper/311:55:133
182075299128,2cyclictest0-21swapper/311:10:173
182074899129,2cyclictest1903317-21kworker/u8:1+events_unbound08:45:192
182074899128,3cyclictest1940069-21kworker/u8:4+flush-259:010:41:542
182074899128,3cyclictest1831305-21cpuspeed_turbos07:20:142
1820748991210,1cyclictest0-21swapper/207:15:172
182075299115,4cyclictest2080866-21apt-get11:20:123
182074899118,3cyclictest1928106-21kworker/u8:3+flush-259:008:50:232
182074899118,2cyclictest1846809-21cpuspeed_turbos07:35:142
182074899117,3cyclictest2043575-21kworker/u8:0+flush-259:010:55:252
182075299108,1cyclictest0-21swapper/309:50:143
182074899106,3cyclictest1940069-21kworker/u8:4+flush-259:010:35:242
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional