You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-10 - 06:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Tue Mar 10, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1671855993631,4cyclictest1895357-21apt-get22:45:140
1671855993226,4cyclictest1770523-21kworker/u8:5+flush-259:022:10:170
1671860993126,4cyclictest252-21systemd-journal19:25:012
1671855993125,4cyclictest252-21systemd-journal20:00:010
1671857993025,3cyclictest1838307-21apt-get21:50:121
1671857993024,4cyclictest1853501-21latency_hist22:05:011
1671865992824,3cyclictest1770523-21kworker/u8:5+flush-259:021:45:163
1671865992822,4cyclictest1718532-21latency_hist19:55:013
1671855992820,5cyclictest1947058-21latency_hist23:35:000
1671860992722,4cyclictest1910092-21ntpq22:55:222
1671860992722,4cyclictest1894664-21apt-get22:45:002
1671857992722,3cyclictest252-21systemd-journal20:53:171
1671857992722,3cyclictest252-21systemd-journal20:53:171
1671857992722,3cyclictest2005171-21/usr/sbin/munin00:30:141
1671860992622,3cyclictest1946142-21ntpq23:30:202
1671860992620,4cyclictest252-21systemd-journal22:20:012
1671865992519,4cyclictest1817407-21diskstats21:30:133
1671855992520,3cyclictest1786048-21latency_hist21:00:010
1671855992520,3cyclictest1786048-21latency_hist21:00:010
1671860992418,4cyclictest1703140-21latency_hist19:40:012
1671860992418,4cyclictest1692845-21latency_hist19:30:012
1671857992421,2cyclictest1827412-21gpgv21:40:001
1671857992418,4cyclictest252-21systemd-journal20:20:001
1671855992419,3cyclictest1795981-21kworker/u8:0+flush-259:021:30:000
1671855992418,4cyclictest1957356-21latency_hist23:45:010
1671865992320,2cyclictest1770523-21kworker/u8:5+flush-259:021:10:003
1671860992319,3cyclictest1827322-21gpgv21:40:002
1671860992317,4cyclictest1750119-21apt-get20:25:122
1671855992317,4cyclictest1999752-21cp00:25:020
1671855992317,4cyclictest1858645-21latency_hist22:10:010
1671855992316,5cyclictest1770503-21latency_hist20:45:000
1671865992219,2cyclictest1674450-21kworker/u8:0+flush-259:020:40:223
1671865992218,3cyclictest1910953-21kworker/u8:0+events_unbound00:00:213
1671865992218,3cyclictest1848548-21kworker/u8:2+events_unbound22:20:163
1671860992219,2cyclictest1734396-21grep20:10:002
1671860992218,3cyclictest1900272-21kworker/u8:3+flush-259:023:05:162
1671860992217,4cyclictest1864033-21kworker/u8:0+flush-259:022:30:162
1671855992218,3cyclictest1749893-21kworker/u8:1+flush-259:020:25:180
1671855992217,4cyclictest509-21rs:main0
1671865992117,3cyclictest1925943-21kworker/u8:4+events_unbound23:45:203
1671865992117,3cyclictest1770523-21kworker/u8:5+flush-259:021:25:003
1671865992117,3cyclictest1770523-21kworker/u8:5+events_unbound21:41:483
1671865992117,3cyclictest1681846-21kworker/u8:3+events_unbound19:40:143
1671860992116,4cyclictest1910953-21kworker/u8:0+flush-259:000:00:162
1671860992116,3cyclictest1661402-21kworker/u8:2+flush-259:019:15:002
1671857992116,3cyclictest1910953-21kworker/u8:0+flush-259:023:55:241
167185599211,18cyclictest490-21dbus-daemon22:55:000
1671865992017,1cyclictest0-21swapper/321:40:003
1671865992015,3cyclictest1925943-21kworker/u8:4+flush-259:023:30:233
1671865992014,4cyclictest1999403-21apt-get00:25:023
1671865991916,2cyclictest1876648-21kthreadcore22:25:173
1671865991916,2cyclictest1674450-21kworker/u8:0+events_unbound19:20:233
1671865991915,3cyclictest1879240-21kworker/u8:3+events_unbound22:40:203
1671865991915,3cyclictest1848548-21kworker/u8:2+flush-259:022:00:243
1671865991915,2cyclictest1910926-21idleruntime-cro23:00:003
1671865991915,2cyclictest1744745-21cron20:20:003
1671865991914,3cyclictest1910953-21kworker/u8:0+flush-259:023:20:183
1671860991912,5cyclictest1962519-21latency_hist23:50:012
1671857991916,2cyclictest2971-21cron23:55:001
1671857991916,2cyclictest1879240-21kworker/u8:3+flush-259:022:40:231
1671857991915,3cyclictest507-21in:imuxsock00:25:001
1671857991915,3cyclictest1770523-21kworker/u8:5+flush-259:021:40:211
1671857991913,4cyclictest252-21systemd-journal22:25:161
1671857991913,4cyclictest1770739-21/usr/sbin/munin20:45:151
1671855991913,4cyclictest252-21systemd-journal21:33:170
1671865991815,2cyclictest1973852-21kworker/u8:3+events_unbound00:30:193
1671865991813,4cyclictest1713625-21kworker/u8:2+flush-259:020:05:183
1671860991815,2cyclictest1674450-21kworker/u8:0+flush-259:020:00:172
1671860991815,2cyclictest1674450-21kworker/u8:0+events_unbound19:30:152
1671860991815,2cyclictest0-21swapper/222:00:002
1671860991814,3cyclictest1795981-21kworker/u8:0+flush-259:021:20:152
1671860991812,4cyclictest1863798-21latency_hist22:15:012
167185799182,8cyclictest1682104-21munin-plugin-st19:20:001
1671857991815,2cyclictest1713625-21kworker/u8:2+flush-259:019:50:261
1671857991815,2cyclictest1687260-21true19:25:011
1671857991814,3cyclictest1681846-21kworker/u8:3+flush-259:019:40:231
1671865991714,2cyclictest1755245-21kworker/u8:4+flush-259:020:35:223
1671865991713,3cyclictest1925943-21kworker/u8:4+flush-259:023:55:003
1671865991713,2cyclictest1999609-21dpkg00:25:013
1671865991711,4cyclictest1801567-21latency_hist21:15:013
1671860991711,4cyclictest509-21rs:main2
1671857991713,3cyclictest1681846-21kworker/u8:3+flush-259:019:25:181
1671857991713,3cyclictest1661402-21kworker/u8:2+flush-259:019:10:291
1671857991711,4cyclictest252-21systemd-journal20:40:001
1671857991711,4cyclictest252-21systemd-journal20:33:171
1671855991714,2cyclictest60350irq/132-enp0s29f1:rx-023:25:160
1671855991714,2cyclictest1749893-21kworker/u8:1+flush-259:020:50:000
1671855991714,2cyclictest1713378-21cron19:50:010
1671855991713,2cyclictest60350irq/132-enp0s29f1:rx-022:35:250
1671865991613,2cyclictest1776374-21irqcore20:50:173
1671865991613,2cyclictest1776374-21irqcore20:50:163
1671865991613,2cyclictest1770523-21kworker/u8:5+flush-259:021:15:213
1671865991612,3cyclictest1900272-21kworker/u8:3+flush-259:023:05:203
1671865991612,3cyclictest1900272-21kworker/u8:3+events_unbound23:15:143
1671865991612,2cyclictest1858880-21/usr/sbin/munin22:10:183
1671860991613,2cyclictest1714083-21irqcore19:50:182
1671860991612,3cyclictest1925943-21kworker/u8:4+flush-259:023:23:182
1671860991610,4cyclictest252-21systemd-journal23:15:182
1671857991613,2cyclictest1796432-21idleruntime-cro21:10:001
1671857991612,3cyclictest1973795-21base6400:00:021
1671857991612,3cyclictest1833122-21/usr/sbin/munin21:45:161
1671857991610,4cyclictest1868972-21latency_hist22:20:011
1671855991614,1cyclictest60350irq/132-enp0s29f1:rx-023:03:280
1671855991614,1cyclictest60350irq/132-enp0s29f1:rx-022:20:110
1671855991614,1cyclictest60350irq/132-enp0s29f1:rx-022:00:240
1671855991614,1cyclictest60350irq/132-enp0s29f1:rx-019:20:130
1671855991614,1cyclictest60350irq/132-enp0s29f1:rx-000:10:190
1671855991613,2cyclictest60350irq/132-enp0s29f1:rx-023:50:190
1671855991613,2cyclictest60350irq/132-enp0s29f1:rx-023:10:200
1671855991613,2cyclictest60350irq/132-enp0s29f1:rx-021:55:170
1671855991613,2cyclictest60350irq/132-enp0s29f1:rx-019:10:170
1671855991612,2cyclictest60350irq/132-enp0s29f1:rx-000:25:160
1671855991612,2cyclictest1868977-21turbostat22:20:000
1671855991611,4cyclictest1800693-21ntpq21:10:220
1671855991611,3cyclictest1894687-21http22:45:000
167186599158,5cyclictest252-21systemd-journal19:56:303
1671865991512,2cyclictest1968750-21irqcore23:55:163
1671865991512,2cyclictest1728813-21idleruntime-cro20:05:003
1671865991511,3cyclictest1765338-21kworker/u8:3+flush-259:021:25:163
1671860991512,2cyclictest1989098-21kworker/u8:1+flush-259:000:30:012
1671860991512,2cyclictest1910953-21kworker/u8:0+flush-259:023:05:002
1671860991512,2cyclictest0-21swapper/221:05:002
1671860991512,2cyclictest0-21swapper/200:00:002
167185799159,4cyclictest1770523-21kworker/u8:5+flush-259:022:05:191
167185799159,4cyclictest1674450-21kworker/u8:0+flush-259:020:00:161
1671857991512,2cyclictest1713625-21kworker/u8:2+flush-259:020:05:161
1671857991511,3cyclictest1989098-21kworker/u8:1+flush-259:000:35:261
1671857991510,3cyclictest252-21systemd-journal23:36:171
1671857991510,3cyclictest252-21systemd-journal20:23:161
1671855991513,2cyclictest60350irq/132-enp0s29f1:rx-020:54:230
1671855991513,2cyclictest60350irq/132-enp0s29f1:rx-020:54:230
1671855991513,1cyclictest60350irq/132-enp0s29f1:rx-023:05:110
1671855991513,1cyclictest60350irq/132-enp0s29f1:rx-022:25:140
1671855991513,1cyclictest60350irq/132-enp0s29f1:rx-021:45:160
1671855991513,1cyclictest60350irq/132-enp0s29f1:rx-019:40:110
1671855991513,1cyclictest60350irq/132-enp0s29f1:rx-000:30:140
1671855991513,1cyclictest60350irq/132-enp0s29f1:rx-000:00:220
1671855991512,2cyclictest60350irq/132-enp0s29f1:rx-023:20:120
1671855991512,2cyclictest60350irq/132-enp0s29f1:rx-023:15:510
1671855991512,2cyclictest60350irq/132-enp0s29f1:rx-000:05:240
1671855991512,2cyclictest1994237-21cron00:20:000
1671855991512,2cyclictest1760205-21turbostat20:35:000
1671855991512,2cyclictest0-21swapper/021:45:000
167186599149,3cyclictest1952457-21apt-get23:40:113
167186599146,6cyclictest252-21systemd-journal23:30:003
1671865991411,2cyclictest2010804-21irqcore00:35:163
1671865991411,2cyclictest1749853-21sh20:25:003
1671865991410,3cyclictest1661402-21kworker/u8:2+flush-259:019:10:163
1671865991410,2cyclictest0-21swapper/319:20:003
167186099144,6cyclictest1989078-21munin-plugin-st00:15:012
1671860991412,1cyclictest1999421-21gpgv00:25:012
1671860991412,1cyclictest0-21swapper/219:45:012
1671860991410,3cyclictest1735067-21sed20:10:162
1671857991412,1cyclictest0-21swapper/123:50:001
1671857991412,1cyclictest0-21swapper/122:55:001
1671857991410,3cyclictest1765338-21kworker/u8:3+flush-259:021:10:341
1671857991410,2cyclictest1932226-21irqcore23:20:161
167185599148,4cyclictest1765312-21latency_hist20:40:000
1671855991412,1cyclictest60350irq/132-enp0s29f1:rx-021:52:170
1671855991412,1cyclictest60350irq/132-enp0s29f1:rx-021:35:260
1671855991412,1cyclictest60350irq/132-enp0s29f1:rx-021:20:580
1671855991412,1cyclictest60350irq/132-enp0s29f1:rx-021:00:130
1671855991412,1cyclictest60350irq/132-enp0s29f1:rx-020:03:390
1671855991412,1cyclictest60350irq/132-enp0s29f1:rx-019:15:250
1671855991412,1cyclictest60350irq/132-enp0s29f1:rx-000:35:120
1671855991411,2cyclictest60350irq/132-enp0s29f1:rx-023:35:190
1671855991411,2cyclictest60350irq/132-enp0s29f1:rx-020:20:110
167186599139,3cyclictest1942132-21kworker/u8:2+flush-259:023:35:143
167186599139,2cyclictest1713625-21kworker/u8:2+events_unbound20:10:173
1671865991311,1cyclictest0-21swapper/320:30:003
1671865991310,2cyclictest1894521-21kworker/u8:1+events_unbound22:45:163
167186099139,3cyclictest1755245-21kworker/u8:4+events_unbound20:35:192
1671860991310,2cyclictest1895777-21irqcore22:45:182
1671860991310,2cyclictest1801938-21cpuspeed_turbos21:15:142
1671860991310,2cyclictest0-21swapper/222:25:162
167186099130,11cyclictest1-21systemd20:00:012
167185799139,3cyclictest1910953-21kworker/u8:0+flush-259:000:10:271
167185799139,3cyclictest1894521-21kworker/u8:1+flush-259:023:10:251
167185799138,4cyclictest252-21systemd-journal22:00:001
1671857991310,2cyclictest1879655-21fschecks_count22:30:131
1671857991310,2cyclictest1869370-21cpuspeed_turbos22:20:131
1671857991310,2cyclictest1674450-21kworker/u8:0+flush-259:019:40:011
167185799130,11cyclictest252-21systemd-journal22:10:011
167185599139,3cyclictest1674450-21kworker/u8:0+flush-259:020:09:430
167185599138,4cyclictest252-21systemd-journal19:30:000
1671855991311,1cyclictest60350irq/132-enp0s29f1:rx-023:56:470
1671855991311,1cyclictest60350irq/132-enp0s29f1:rx-022:55:140
1671855991311,1cyclictest60350irq/132-enp0s29f1:rx-020:10:240
1671855991310,2cyclictest60350irq/132-enp0s29f1:rx-023:49:200
1671855991310,2cyclictest60350irq/132-enp0s29f1:rx-019:50:230
1671855991310,2cyclictest60350irq/132-enp0s29f1:rx-019:30:220
167186599129,2cyclictest1989098-21kworker/u8:1+flush-259:000:15:293
167186599129,2cyclictest1983949-21munin-run00:10:003
167186599129,2cyclictest1879240-21kworker/u8:3+flush-259:022:35:193
167186599129,2cyclictest1848548-21kworker/u8:2+flush-259:022:50:253
167186599129,2cyclictest1765338-21kworker/u8:3+events_unbound20:45:193
167186599129,2cyclictest1759390-21nvmesmart_nvme020:30:223
167186599127,4cyclictest1681846-21kworker/u8:3+events_unbound19:45:143
167186099129,2cyclictest1979111-21cpuspeed_turbos00:05:122
167186099129,2cyclictest1956949-21unixbench_singl23:40:262
167186099129,2cyclictest1671849-21cyclictest23:30:002
167186099127,3cyclictest1745225-21sort20:20:142
1671860991210,1cyclictest0-21swapper/223:40:002
167185799129,2cyclictest1906129-21cpuspeed_turbos22:55:131
167185799129,2cyclictest1900272-21kworker/u8:3+flush-259:023:05:301
167185799129,2cyclictest1713625-21kworker/u8:2+flush-259:020:15:001
167185799129,1cyclictest0-21swapper/122:45:181
167185799128,3cyclictest1770523-21kworker/u8:5+flush-259:021:15:331
167185599129,2cyclictest60350irq/132-enp0s29f1:rx-022:30:210
1671855991210,1cyclictest60350irq/132-enp0s29f1:rx-021:05:170
167186599117,3cyclictest1864033-21kworker/u8:0+events_unbound22:32:003
167186599117,3cyclictest1681846-21kworker/u8:3+flush-259:019:25:213
167186599111,3cyclictest252-21systemd-journal21:55:003
167186599110,9cyclictest1858771-21cat22:10:013
167186099119,1cyclictest1786064-21idleruntime-cro21:00:002
167186099119,1cyclictest1786064-21idleruntime-cro21:00:002
167186099119,1cyclictest0-21swapper/222:55:012
167186099119,1cyclictest0-21swapper/200:30:122
167186099118,2cyclictest2012646-21kthreadcore00:35:182
167186099118,2cyclictest1796426-21cron21:10:002
167186099118,2cyclictest0-21swapper/222:20:242
167186099118,2cyclictest0-21swapper/221:10:182
167186099116,3cyclictest252-21systemd-journal20:40:172
167186099115,4cyclictest1963066-21cp23:50:012
167185799119,1cyclictest0-21swapper/120:00:001
167185799118,2cyclictest1937219-21fschecks_time23:25:131
167185799118,2cyclictest1817656-21sed21:30:151
167185799111,8cyclictest1916065-21latency_hist23:05:011
167185799110,9cyclictest1-21systemd20:40:001
167185799107,2cyclictest1750184-21cpuspeed_turbos20:25:141
167185799106,3cyclictest1811830-21cron21:25:001
167185799105,4cyclictest1930927-21sed23:15:241
16718609996,2cyclictest1840606-21kthreadcore21:50:182
16718609996,1cyclictest0-21swapper/219:45:132
16718579997,1cyclictest0-21swapper/119:30:141
16718579996,2cyclictest1790439-21ntp_kernel_pll_21:00:191
16718579995,3cyclictest311ktimers/123:40:121
16718579986,1cyclictest0-21swapper/120:55:161
16718579986,1cyclictest0-21swapper/120:55:161
16718579985,2cyclictest1710808-21kthreadcore19:45:181
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional