You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-02 - 18:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Tue Jun 02, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1758258994339,3cyclictest1998653-21kworker/u8:0+flush-259:011:15:161
1758256994034,4cyclictest2055414-21latency_hist11:55:010
1758261993632,3cyclictest1877776-21kworker/u8:0+flush-259:010:10:012
1758266993327,4cyclictest335557-21systemd-journal11:20:263
1758256993326,5cyclictest335557-21systemd-journal09:00:000
1758258993226,4cyclictest1899395-21/usr/sbin/munin09:25:141
1758256993226,4cyclictest1825701-21latency_hist08:15:000
1758258993127,3cyclictest1877776-21kworker/u8:0+flush-259:009:50:011
1758258993127,3cyclictest1799827-21kworker/u8:0+flush-259:008:14:591
1758258993125,4cyclictest1892726-21ntpq09:15:221
1758258993125,4cyclictest1883123-21store09:10:011
1758258993125,4cyclictest1793927-21ntpq07:40:241
1758261993025,3cyclictest2034958-21apt-get11:35:132
1758258993025,3cyclictest2091964-21kworker/u8:1+flush-259:012:35:011
1758258993023,4cyclictest2060181-21ntpq11:55:221
1758256993025,4cyclictest335557-21systemd-journal11:30:000
1758266992923,4cyclictest335557-21systemd-journal12:15:003
1758261992925,3cyclictest1961437-21kworker/u8:0+flush-259:010:29:592
1758258992924,3cyclictest1779140-21apt-get07:30:111
1758258992922,5cyclictest2085963-21ntpq12:20:211
1758258992921,5cyclictest2045013-21latency_hist11:45:011
1758261992825,2cyclictest1904313-21kworker/u8:2+events_unbound09:35:182
1758261992824,3cyclictest2071410-21kworker/u8:0+flush-259:012:10:252
1758261992820,6cyclictest335557-21systemd-journal08:15:002
1758266992722,4cyclictest335557-21systemd-journal10:05:003
1758261992721,4cyclictest335557-21systemd-journal12:30:272
1758261992721,4cyclictest335557-21systemd-journal09:17:272
1758258992723,3cyclictest1867136-21kworker/u8:3+flush-259:008:55:001
1758258992722,3cyclictest1784629-21cat07:35:011
1758266992623,2cyclictest1950621-21kworker/u8:3+events_unbound10:45:153
1758261992621,3cyclictest1758221-21kworker/u8:3+flush-259:007:25:222
1758258992620,4cyclictest1758198-21latency_hist07:10:011
1758258992619,4cyclictest1909697-21/usr/sbin/munin09:35:141
1758266992522,2cyclictest2045247-21kworker/u8:2+events_unbound12:09:573
1758266992521,3cyclictest1904313-21kworker/u8:2+events_unbound09:59:573
1758266992521,2cyclictest2852396-21kworker/u8:2+events_unbound07:40:033
1758261992521,3cyclictest1998653-21kworker/u8:0+events_unbound11:15:232
1758258992522,2cyclictest1955850-21kworker/u8:1+flush-259:010:30:261
1758266992421,2cyclictest1789598-21cron07:40:003
1758266992420,3cyclictest1899291-21gpgv09:25:013
1758261992420,3cyclictest1904313-21kworker/u8:2+events_unbound09:40:282
1758261992420,3cyclictest1836712-21kworker/u8:1+events_unbound08:37:282
1758261992420,3cyclictest1836712-21kworker/u8:1+events_unbound08:37:272
1758256992420,3cyclictest1882913-21http09:10:010
1758256992418,4cyclictest1763381-21latency_hist07:15:020
1758266992320,2cyclictest1950621-21kworker/u8:3+events_unbound11:39:573
1758266992320,2cyclictest1846738-21kworker/u8:2+events_unbound09:21:283
1758261992316,5cyclictest2070459-21ntpq12:05:212
1758258992319,3cyclictest2091925-21apt-get12:29:201
1758258992319,3cyclictest1846738-21kworker/u8:2+flush-259:009:00:241
1758258992316,4cyclictest1914881-21/usr/sbin/munin09:40:141
1758266992219,2cyclictest2045247-21kworker/u8:2+events_unbound12:19:563
1758261992219,2cyclictest1950621-21kworker/u8:3+events_unbound10:17:282
1758261992218,3cyclictest2852396-21kworker/u8:2+flush-259:007:15:202
1758261992218,3cyclictest1955850-21kworker/u8:1+events_unbound10:30:182
1758261992218,3cyclictest1904313-21kworker/u8:2+events_unbound09:46:282
1758261992217,3cyclictest335557-21systemd-journal09:30:262
1758261992217,3cyclictest1877776-21kworker/u8:0+flush-259:009:55:212
1758261992216,4cyclictest335557-21systemd-journal10:55:002
1758261992215,5cyclictest1798965-21ntpq07:45:192
1758261992215,5cyclictest1784017-21latency_hist07:35:012
1758258992218,3cyclictest2045033-21kworker/u8:1+flush-259:011:45:261
1758258992217,3cyclictest1988136-21kworker/u8:4+flush-259:011:25:161
1758258992216,4cyclictest1835588-21ntpq08:20:191
1758266992116,4cyclictest1882890-21apt-get09:10:003
1758266992115,2cyclictest1852231-21apt-get08:40:113
1758266992113,6cyclictest1966989-21cat10:30:003
1758261992118,2cyclictest1988136-21kworker/u8:4+flush-259:011:05:222
1758261992118,2cyclictest1988136-21kworker/u8:4+flush-259:011:05:222
1758261992117,2cyclictest1950621-21kworker/u8:3+flush-259:010:50:002
1758261992116,3cyclictest1784676-21kworker/u8:1+events_unbound08:00:122
1758258992118,2cyclictest1950621-21kworker/u8:3+events_unbound11:00:181
1758258992117,3cyclictest2045033-21kworker/u8:1+flush-259:011:50:161
1758258992116,4cyclictest1758221-21kworker/u8:3+flush-259:007:45:151
1758266992018,2cyclictest2852396-21kworker/u8:2+events_unbound07:34:003
1758266992017,2cyclictest1998653-21kworker/u8:0+events_unbound11:27:353
1758266992017,2cyclictest1950621-21kworker/u8:3+events_unbound11:45:013
1758266992017,2cyclictest1950621-21kworker/u8:3+events_unbound11:03:403
1758266992017,2cyclictest1784676-21kworker/u8:1+events_unbound08:09:443
1758266992015,3cyclictest335557-21systemd-journal09:36:263
1758261992016,3cyclictest1867136-21kworker/u8:3+events_unbound09:05:142
1758261992016,3cyclictest1820513-21kworker/u8:4+events_unbound08:20:142
1758261992016,2cyclictest0-21swapper/209:05:012
1758258992016,3cyclictest1820512-21kworker/u8:3+flush-259:008:10:001
1758258992014,4cyclictest2076521-21latency_hist12:15:011
1758258992014,4cyclictest1972719-21apt-get10:35:131
1758266991917,2cyclictest2091964-21kworker/u8:1+events_unbound12:29:363
1758266991916,2cyclictest2045033-21kworker/u8:1+events_unbound11:53:353
1758266991916,2cyclictest1998653-21kworker/u8:0+events_unbound11:10:313
1758266991916,2cyclictest1820514-21kworker/u8:5+events_unbound08:15:493
1758266991916,2cyclictest1784676-21kworker/u8:1+events_unbound08:04:053
1758266991916,2cyclictest1784676-21kworker/u8:1+events_unbound07:53:383
1758266991916,2cyclictest1758221-21kworker/u8:3+events_unbound07:24:473
1758266991915,3cyclictest1940882-21apt-get10:05:123
1758266991914,4cyclictest1982761-21latency_hist10:45:013
1758261991917,1cyclictest1799827-21kworker/u8:0+events_unbound07:50:132
1758261991916,2cyclictest1988136-21kworker/u8:4+events_unbound11:30:182
1758261991916,2cyclictest1950621-21kworker/u8:3+flush-259:010:55:242
1758261991915,3cyclictest2071410-21kworker/u8:0+events_unbound12:15:182
1758261991915,3cyclictest1898437-21kworker/u8:1+flush-259:009:25:172
1758261991915,3cyclictest1805053-21kworker/u8:2+events_unbound08:05:232
1758258991914,3cyclictest2097456-21apt-get12:35:121
1758258991913,4cyclictest335557-21systemd-journal09:11:281
1758256991916,2cyclictest1778873-21sh07:30:000
1758256991915,3cyclictest1784529-21base6407:35:020
1758266991816,2cyclictest1820514-21kworker/u8:5+events_unbound08:11:123
1758266991816,1cyclictest1867136-21kworker/u8:3+events_unbound09:10:383
1758266991815,2cyclictest2852396-21kworker/u8:2+events_unbound07:45:563
1758266991815,2cyclictest2050432-21kworker/u8:5+events_unbound11:56:413
1758266991815,2cyclictest1998653-21kworker/u8:0+events_unbound11:34:023
1758266991815,2cyclictest1898437-21kworker/u8:1+events_unbound09:33:473
1758266991815,2cyclictest1857151-21kworker/u8:0+events_unbound08:46:433
1758261991814,3cyclictest2071410-21kworker/u8:0+events_unbound12:29:202
1758261991814,3cyclictest1998653-21kworker/u8:0+events_unbound11:11:272
1758261991814,3cyclictest1950621-21kworker/u8:3+events_unbound10:35:222
1758261991814,3cyclictest1846738-21kworker/u8:2+events_unbound08:40:302
1758261991814,3cyclictest1758221-21kworker/u8:3+flush-259:007:20:202
1758261991813,4cyclictest1846738-21kworker/u8:2+events_unbound09:20:132
1758261991811,5cyclictest2040278-21diskstats11:40:142
1758258991814,3cyclictest1877776-21kworker/u8:0+flush-259:010:10:171
1758258991813,3cyclictest335557-21systemd-journal09:50:061
1758258991811,5cyclictest1897891-21ntpq09:20:211
175825699182,6cyclictest1884413-21sed09:10:180
1758256991814,3cyclictest1956362-21apt-get10:20:120
175826699174,5cyclictest50-21ksoftirqd/307:10:003
1758266991715,2cyclictest1836712-21kworker/u8:1+events_unbound08:50:033
1758266991715,2cyclictest1758221-21kworker/u8:3+events_unbound07:13:423
1758266991714,2cyclictest2040527-21irqcore11:40:173
1758266991714,2cyclictest1955850-21kworker/u8:1+events_unbound10:34:033
1758266991714,2cyclictest1846738-21kworker/u8:2+events_unbound09:15:323
1758266991714,2cyclictest1820514-21kworker/u8:5+events_unbound08:26:453
1758261991713,3cyclictest2050432-21kworker/u8:5+flush-259:012:00:202
1758261991713,3cyclictest2045247-21kworker/u8:2+events_unbound11:45:322
1758261991713,3cyclictest2045033-21kworker/u8:1+events_unbound11:50:222
1758261991713,3cyclictest1904313-21kworker/u8:2+events_unbound10:04:552
1758261991713,3cyclictest1877776-21kworker/u8:0+flush-259:009:55:012
1758261991713,3cyclictest1867136-21kworker/u8:3+events_unbound08:55:232
1758261991711,4cyclictest1883743-21diskstats09:10:152
1758261991710,5cyclictest335557-21systemd-journal11:01:282
1758261991710,5cyclictest335557-21systemd-journal07:40:002
1758258991714,1cyclictest0-21swapper/108:35:011
1758258991712,3cyclictest335557-21systemd-journal10:25:261
1758258991712,3cyclictest1877776-21kworker/u8:0+flush-259:010:00:171
175825699174,2cyclictest1950008-21ntpq10:10:190
175825699173,6cyclictest2033767-21ntpq11:30:210
175825699173,2cyclictest1908578-21ntpq09:30:210
175825699172,9cyclictest2044146-21ntpq11:40:210
175825699172,4cyclictest1830732-21sshd08:18:270
1758256991710,5cyclictest1867425-21latency_hist08:55:010
1758266991614,2cyclictest2852396-21kworker/u8:2+events_unbound07:27:143
1758266991613,2cyclictest2091964-21kworker/u8:1+events_unbound12:37:023
1758266991613,2cyclictest2045247-21kworker/u8:2+events_unbound12:22:133
1758266991613,2cyclictest1961437-21kworker/u8:0+events_unbound10:37:153
1758266991613,2cyclictest1909476-21kworker/u8:5+events_unbound09:49:543
1758266991613,2cyclictest1877776-21kworker/u8:0+events_unbound10:12:033
1758266991613,2cyclictest1877776-21kworker/u8:0+events_unbound09:54:253
1758266991613,2cyclictest1846738-21kworker/u8:2+events_unbound08:56:163
1758266991613,2cyclictest1799827-21kworker/u8:0+flush-259:008:35:003
1758266991613,2cyclictest1799827-21kworker/u8:0+events_unbound08:20:583
1758266991612,3cyclictest2092299-21apt-get12:30:143
1758261991613,2cyclictest2086807-21cron12:25:002
1758261991613,2cyclictest1998653-21kworker/u8:0+events_unbound11:24:562
1758261991612,3cyclictest1950621-21kworker/u8:3+events_unbound10:23:292
1758261991612,3cyclictest1877776-21kworker/u8:0+flush-259:010:10:192
1758258991613,2cyclictest0-21swapper/108:00:191
1758258991611,3cyclictest1955850-21kworker/u8:1+flush-259:010:20:171
1758258991610,5cyclictest335557-21systemd-journal07:54:271
175825699163,7cyclictest2018396-21ntpq11:15:220
175825699163,6cyclictest2013230-21ntpq11:10:210
175825699163,3cyclictest2098058-21cut12:35:170
175825699162,7cyclictest2065422-21ntpq12:00:210
175825699162,5cyclictest1899886-21sed09:25:150
175825699161,6cyclictest1931425-21sed09:55:160
1758256991614,1cyclictest60350irq/132-enp0s29f1:rx-008:05:150
1758256991611,4cyclictest1966826-21latency_hist10:30:000
1758256991611,1cyclictest191rcu_preempt09:24:320
1758256991610,1cyclictest191rcu_preempt10:44:430
175825699160,15cyclictest2050220-21true11:50:000
175825699160,14cyclictest509-21rs:main0
175826699155,1cyclictest191rcu_preempt10:55:223
1758266991512,2cyclictest1998653-21kworker/u8:0+events_unbound11:15:553
1758266991512,2cyclictest1987945-21kworker/u8:2+events_unbound10:51:023
1758266991512,2cyclictest1846738-21kworker/u8:2+events_unbound09:01:343
1758266991512,1cyclictest191rcu_preempt08:38:443
1758266991512,1cyclictest191rcu_preempt08:38:433
1758261991512,2cyclictest509-21rs:main2
1758261991511,3cyclictest1950621-21kworker/u8:3+events_unbound12:35:292
1758261991511,3cyclictest1799827-21kworker/u8:0+events_unbound08:34:562
1758261991511,3cyclictest1758456-21/usr/sbin/munin07:10:202
1758258991511,3cyclictest1998653-21kworker/u8:0+flush-259:011:20:171
1758258991511,3cyclictest1988136-21kworker/u8:4+flush-259:011:05:261
1758258991511,3cyclictest1988136-21kworker/u8:4+flush-259:011:05:251
1758258991511,3cyclictest1977852-21/usr/sbin/munin10:40:201
175825699157,6cyclictest1973092-21if_err_enp0s29f10:35:160
175825699154,2cyclictest2096328-21ntpq12:30:250
175825699153,3cyclictest2055985-21apt-config11:55:000
175825699153,2cyclictest1831851-21fw_conntrack08:20:110
175825699153,1cyclictest2023558-21ntpq11:20:220
175825699152,8cyclictest1993948-21tune2fs10:55:150
175825699152,4cyclictest1943246-21kthreadcore10:05:180
175825699151,5cyclictest1987052-21collect210:45:230
175825699151,5cyclictest1808112-21cut07:55:180
175825699151,4cyclictest1940470-21modprobe10:03:270
1758256991514,1cyclictest60350irq/132-enp0s29f1:rx-012:20:150
1758256991514,1cyclictest60350irq/132-enp0s29f1:rx-007:34:150
1758256991514,1cyclictest60350irq/132-enp0s29f1:rx-007:20:460
1758256991513,1cyclictest60350irq/132-enp0s29f1:rx-012:15:190
1758256991513,1cyclictest60350irq/132-enp0s29f1:rx-012:05:200
1758256991513,1cyclictest60350irq/132-enp0s29f1:rx-011:05:370
1758256991513,1cyclictest60350irq/132-enp0s29f1:rx-011:05:360
1758256991513,1cyclictest60350irq/132-enp0s29f1:rx-011:00:160
1758256991513,1cyclictest60350irq/132-enp0s29f1:rx-009:35:160
1758256991513,1cyclictest60350irq/132-enp0s29f1:rx-008:45:160
1758256991513,1cyclictest60350irq/132-enp0s29f1:rx-007:45:240
1758256991513,1cyclictest0-21swapper/010:35:000
1758256991512,2cyclictest60350irq/132-enp0s29f1:rx-009:45:240
1758256991512,2cyclictest60350irq/132-enp0s29f1:rx-009:15:130
1758256991512,2cyclictest60350irq/132-enp0s29f1:rx-007:50:130
1758256991511,3cyclictest1909476-21kworker/u8:5+flush-259:009:45:000
1758256991511,3cyclictest1763643-21apt-get07:15:100
175825699150,6cyclictest1876240-21kthreadcore09:00:190
175826699143,2cyclictest0-21swapper/310:15:203
175826699142,7cyclictest50-21ksoftirqd/307:15:133
1758266991412,2cyclictest1799827-21kworker/u8:0+events_unbound07:59:423
1758266991411,2cyclictest1956427-21cpuspeed_turbos10:20:133
1758266991411,2cyclictest1914971-21cpuspeed_turbos09:40:133
175825899148,4cyclictest1810161-21latency_hist08:00:001
1758258991411,2cyclictest1867136-21kworker/u8:3+events_unbound08:55:211
1758258991411,2cyclictest1774214-21fschecks_time07:25:151
1758258991411,2cyclictest0-21swapper/108:40:001
1758258991411,2cyclictest0-21swapper/108:39:591
1758258991410,3cyclictest1737358-21kworker/u8:1+flush-259:007:15:261
175825699143,2cyclictest1790173-21/usr/sbin/munin07:40:160
175825699142,4cyclictest2091108-21modprobe12:25:190
175825699142,3cyclictest1990880-21tr10:50:180
175825699141,8cyclictest1842211-21hddtemp_smartct08:30:160
1758256991413,1cyclictest60350irq/132-enp0s29f1:rx-012:12:260
1758256991413,1cyclictest60350irq/132-enp0s29f1:rx-008:42:250
1758256991413,1cyclictest60350irq/132-enp0s29f1:rx-008:35:180
1758256991413,1cyclictest60350irq/132-enp0s29f1:rx-008:35:180
1758256991412,1cyclictest60350irq/132-enp0s29f1:rx-009:50:220
175826199136,5cyclictest1831081-21cp08:20:002
175825899138,3cyclictest2076785-21apt-get12:15:101
175825899131,10cyclictest1-21systemd10:50:001
175825699131,6cyclictest1810579-21df08:00:130
175826199129,2cyclictest1836712-21kworker/u8:1+events_unbound08:51:282
175826199128,3cyclictest1784676-21kworker/u8:1+flush-259:007:55:062
175826199128,2cyclictest1961437-21kworker/u8:0+flush-259:010:40:142
175825899129,2cyclictest2070403-21needreboot12:05:201
175825899129,2cyclictest0-21swapper/108:20:001
175825899128,3cyclictest1836712-21kworker/u8:1+flush-259:008:50:001
175825899125,5cyclictest335557-21systemd-journal10:10:011
175826199117,3cyclictest1836712-21kworker/u8:1+flush-259:008:47:532
175826199117,3cyclictest1820514-21kworker/u8:5+events_unbound08:25:202
175826199117,3cyclictest1758221-21kworker/u8:3+flush-259:007:40:332
175825899119,1cyclictest0-21swapper/110:55:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional