You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-17 - 01:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Sat May 16, 2026 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1781388994542,2cyclictest75-21kswapd011:13:511
1781388994441,2cyclictest2086885-21apt-get12:04:271
1781394994340,2cyclictest75-21kswapd011:13:523
1781394993833,3cyclictest2009023-21latency_hist10:50:003
1781394993329,3cyclictest2024707-21/usr/sbin/munin11:05:243
1781392993327,4cyclictest1946642-21apt-get09:50:022
1781392993230,1cyclictest1915682-21kworker/2:2+pm11:13:482
1781388993226,4cyclictest1905322-21latency_hist09:10:001
1781388993125,4cyclictest1781302-21latency_hist07:10:001
178138599311,28cyclictest1-21systemd09:45:010
1781392993023,5cyclictest252-21systemd-journal07:40:002
1781388993026,3cyclictest1848436-21kworker/u8:2+flush-259:009:00:011
1781385993024,4cyclictest1801937-21latency_hist07:30:010
1781392992912,5cyclictest1884779-21awk08:50:002
1781388992923,4cyclictest2118018-21latency_hist12:35:011
1781388992922,5cyclictest2029625-21latency_hist11:10:011
1781392992821,5cyclictest1978044-21latency_hist10:20:002
1781385992824,3cyclictest2055734-21apt-get11:35:120
1781388992622,3cyclictest1728295-21kworker/u8:4+flush-259:009:20:241
1781385992622,3cyclictest75-21kswapd011:13:520
1781394992522,2cyclictest2040004-21kworker/u8:3+flush-259:012:20:253
1781394992521,3cyclictest1905349-21kworker/u8:0+events_unbound10:20:173
1781394992520,4cyclictest252-21systemd-journal07:54:473
1781394992519,4cyclictest1895065-21latency_hist09:00:003
1781388992520,4cyclictest2065823-21latency_hist11:45:011
1781385992521,3cyclictest1728295-21kworker/u8:4+flush-259:009:40:010
1781394992420,3cyclictest1728295-21kworker/u8:4+flush-259:009:20:163
1781394992420,3cyclictest1728295-21kworker/u8:4+flush-259:009:00:243
1781394992420,3cyclictest1728295-21kworker/u8:4+flush-259:008:40:253
1781392992418,4cyclictest1848436-21kworker/u8:2+flush-259:008:50:172
1781388992418,4cyclictest1978309-21apt-get10:20:121
1781385992420,2cyclictest1728295-21kworker/u8:4+flush-259:009:20:000
1781385992417,5cyclictest1817360-21latency_hist07:45:010
1781385992318,3cyclictest2045161-21latency_hist11:25:000
1781385992317,4cyclictest1817789-40mandb07:45:020
1781394992217,3cyclictest1817355-21apt-get07:45:023
1781392992216,4cyclictest252-21systemd-journal12:27:572
1781392992216,4cyclictest252-21systemd-journal08:10:012
1781392992216,4cyclictest2081742-21latency_hist12:00:012
1781388992218,3cyclictest1807077-21kworker/u8:1+flush-259:008:05:001
1781388992217,3cyclictest1998960-21/usr/sbin/munin10:40:191
1781394992116,3cyclictest2014593-21diskstats10:55:153
1781394992115,4cyclictest1864202-21latency_hist08:30:003
1781392992115,4cyclictest2118031-21https12:35:012
178139299210,19cyclictest2076181-21munin-run11:55:012
1781388992116,3cyclictest1864466-21apt-get08:30:121
1781388992114,5cyclictest252-21systemd-journal10:20:001
1781385992119,2cyclictest2102448-21kworker/u8:1+events_unbound12:20:030
1781394992013,5cyclictest2013296-21ntpq10:50:223
1781392992015,4cyclictest2055699-21kworker/u8:4+flush-259:012:15:262
1781392992015,3cyclictest2014431-21apt-get10:55:132
1781388992015,4cyclictest1828357-21apt-get07:55:131
1781388992014,5cyclictest252-21systemd-journal10:35:001
1781388992013,5cyclictest1822947-21latency_hist07:50:011
1781385992015,4cyclictest1973143-21apt-get10:15:110
1781394991915,3cyclictest1738189-21kworker/u8:0+flush-259:007:15:223
1781394991914,3cyclictest1920899-21kworker/u8:3+flush-259:010:05:233
1781394991914,3cyclictest1848436-21kworker/u8:2+flush-259:009:05:233
1781394991913,4cyclictest252-21systemd-journal07:15:003
1781392991917,1cyclictest0-21swapper/211:44:592
1781392991912,5cyclictest2024470-21latency_hist11:05:012
1781388991915,3cyclictest2076308-21base6411:55:001
1781388991915,3cyclictest1905591-21apt-get09:10:131
1781388991914,4cyclictest1885025-21/usr/sbin/munin08:50:141
1781388991913,4cyclictest1988402-21latency_hist10:30:011
1781385991916,2cyclictest2014189-21kworker/u8:2+events_unbound11:09:550
1781385991914,3cyclictest252-21systemd-journal07:56:470
1781385991912,5cyclictest1786491-21latency_hist07:15:000
1781394991814,2cyclictest1848689-21latency_hist08:15:013
1781388991814,3cyclictest1946667-21gpgv09:50:001
1781388991812,4cyclictest252-21systemd-journal08:15:001
1781388991811,5cyclictest1873665-21ntpq08:35:231
1781385991816,2cyclictest2040004-21kworker/u8:3+events_unbound11:49:380
178138599180,17cyclictest1962583-21turbostat10:05:000
1781392991715,1cyclictest0-21swapper/207:45:002
1781392991714,2cyclictest1802203-21apt-get07:30:132
1781392991712,4cyclictest2050554-21apt-get11:30:112
1781388991713,3cyclictest2040004-21kworker/u8:3+events_unbound11:25:141
1781388991712,4cyclictest509-21rs:main1
178138899170,8cyclictest1859022-21awk08:25:001
178138599173,4cyclictest2011679-21grep10:50:180
178138599173,3cyclictest2052301-21cut11:30:180
178138599172,6cyclictest2097569-21cat12:15:010
178138599171,5cyclictest1967741-21sh10:10:010
1781385991710,5cyclictest1947192-21cp09:50:010
178139499169,5cyclictest1946641-21latency_hist09:50:013
1781394991613,2cyclictest1940516-21needreboot09:40:193
1781394991611,3cyclictest1738189-21kworker/u8:0+events_unbound07:35:193
1781394991610,4cyclictest252-21systemd-journal11:50:003
1781394991610,4cyclictest2113093-21/usr/sbin/munin12:30:213
1781388991614,1cyclictest1952235-21cron09:55:011
1781388991612,3cyclictest1728295-21kworker/u8:4+flush-259:007:25:231
178138599169,5cyclictest2003859-21latency_hist10:45:000
178138599163,8cyclictest1926689-21if_enp0s30f409:30:150
178138599163,3cyclictest1802660-21cut07:30:180
178138599163,2cyclictest1993824-21apt-get10:35:130
178138599162,5cyclictest1828080-21latency_hist07:55:000
178138599161,5cyclictest2075281-21ntpq11:50:220
1781385991613,2cyclictest1847733-21needreboot08:10:200
1781385991611,1cyclictest191rcu_preempt07:15:320
1781385991610,1cyclictest191rcu_preempt11:59:490
178139499159,4cyclictest2070969-21kworker/u8:1+events_unbound12:00:153
1781394991511,3cyclictest2077835-21kthreadcore11:55:163
1781394991511,2cyclictest1859005-21cron08:25:013
1781394991510,3cyclictest2003848-21cron10:45:013
178139299159,4cyclictest2045421-21apt-get11:25:112
1781392991512,2cyclictest252-21systemd-journal09:10:002
1781392991511,3cyclictest1920899-21kworker/u8:3+flush-259:010:05:252
1781392991510,4cyclictest2087248-21apt-get12:05:122
1781392991510,4cyclictest2040004-21kworker/u8:3+flush-259:012:15:002
178138899159,4cyclictest1791632-21latency_hist07:20:001
1781388991512,2cyclictest2066054-21kworker/u8:0+flush-259:011:45:191
178138599157,1cyclictest1967739-21kworker/u8:1-events_unbound10:10:150
178138599153,4cyclictest1980936-21cut10:20:180
178138599153,3cyclictest2115857-21cut12:30:170
178138599153,3cyclictest1854758-21cut08:20:150
178138599153,2cyclictest1888004-21cut08:50:180
178138599153,2cyclictest1864286-21cat08:30:000
178138599152,4cyclictest2039050-21ntpq11:15:210
178138599152,4cyclictest1992046-21cut10:30:200
1781385991513,2cyclictest60350irq/132-enp0s29f1:rx-012:09:330
1781385991513,1cyclictest61150irq/133-enp0s29f1:tx-012:35:240
1781385991513,1cyclictest60350irq/132-enp0s29f1:rx-012:25:240
1781385991513,1cyclictest60350irq/132-enp0s29f1:rx-011:44:260
1781385991513,1cyclictest60350irq/132-enp0s29f1:rx-008:45:240
1781385991513,1cyclictest60350irq/132-enp0s29f1:rx-008:35:250
1781385991513,1cyclictest60350irq/132-enp0s29f1:rx-007:35:150
1781385991512,2cyclictest60350irq/132-enp0s29f1:rx-009:45:120
1781385991512,2cyclictest60350irq/132-enp0s29f1:rx-009:00:180
1781385991512,2cyclictest60350irq/132-enp0s29f1:rx-007:23:480
1781385991512,2cyclictest2019557-21kworker/u8:1+events_unbound11:04:530
1781385991512,2cyclictest1865488-21kthreadcore08:30:180
1781385991511,3cyclictest60350irq/132-enp0s29f1:rx-009:10:140
1781385991511,3cyclictest1919925-21ntp_kernel_err09:20:210
1781385991510,1cyclictest191rcu_preempt12:18:590
178139499149,4cyclictest1848436-21kworker/u8:2+flush-259:008:45:183
1781394991411,2cyclictest1992577-21needreboot10:30:213
1781394991410,3cyclictest2076135-21apt-get11:55:003
1781394991410,3cyclictest2055699-21kworker/u8:4+flush-259:012:15:163
1781394991410,2cyclictest1911900-21kthreadcore09:15:183
1781394991410,2cyclictest1794091-21kthreadcore07:20:183
178139299149,4cyclictest1988660-21apt-get10:30:122
178139299149,3cyclictest509-21rs:main2
178139299148,4cyclictest252-21systemd-journal07:20:472
1781392991411,2cyclictest1848690-21sh08:15:002
178138899149,3cyclictest1848436-21kworker/u8:2+flush-259:008:25:221
1781388991411,2cyclictest1853844-21idleruntime-cro08:20:011
1781388991411,2cyclictest1792311-21irqcore07:20:161
1781388991411,2cyclictest0-21swapper/107:15:001
1781388991410,3cyclictest1817765-21gpgconf07:45:011
178138599146,1cyclictest191rcu_preempt08:00:150
178138599143,3cyclictest1893463-21tr08:55:200
178138599143,2cyclictest2014622-21fschecks_count10:55:140
178138599142,4cyclictest2046786-21kthreadcore11:25:180
178138599142,3cyclictest16-21ksoftirqd/010:25:110
1781385991413,1cyclictest60350irq/132-enp0s29f1:rx-009:25:130
1781385991412,2cyclictest60350irq/132-enp0s29f1:rx-008:06:370
1781385991412,1cyclictest60350irq/132-enp0s29f1:rx-008:40:260
1781385991411,2cyclictest1920899-21kworker/u8:3+events_unbound09:57:050
1781385991411,2cyclictest1848436-21kworker/u8:2+events_unbound08:16:230
1781385991410,3cyclictest2004127-21apt-get10:45:130
178139499136,5cyclictest252-21systemd-journal08:20:003
1781394991311,1cyclictest1869393-21latency_hist08:35:003
178139299139,3cyclictest1728295-21kworker/u8:4+flush-259:008:00:262
178139299138,4cyclictest2070969-21kworker/u8:1+flush-259:012:00:152
178139299137,4cyclictest1920841-21latency_hist09:25:002
178139299137,4cyclictest1848436-21kworker/u8:2+flush-259:008:25:172
1781392991310,2cyclictest2070131-21ntp_offset11:45:222
1781392991310,2cyclictest2056757-21kthreadcore11:35:182
1781392991310,2cyclictest1993891-21cpuspeed_turbos10:35:142
1781392991310,2cyclictest1956937-21unixbench_multi09:55:242
178138899139,3cyclictest1738189-21kworker/u8:0+flush-259:007:35:291
1781388991311,1cyclictest1958096-21irqcore10:00:171
1781388991310,2cyclictest1899715-21unixbench_multi09:00:261
178138599134,1cyclictest191rcu_preempt12:00:130
178139499129,2cyclictest2112397-21unixbench_multi12:25:263
178139499128,3cyclictest75-21kswapd011:25:143
178139499127,3cyclictest1978070-21kworker/u8:2+events_unbound10:40:003
178139499127,3cyclictest1833681-21diskstats08:00:143
178139499123,2cyclictest19331122kthreadcore09:35:183
178139499121,9cyclictest2065821-21awk11:44:593
178139499121,8cyclictest1801965-21munin-run07:30:013
1781394991210,1cyclictest0-21swapper/309:25:153
178139299129,2cyclictest1922130-21kthreadcore09:25:162
178139299129,2cyclictest1910489-21sh09:15:002
178139299129,2cyclictest1786852-21cpuspeed_turbos07:15:132
178139299128,3cyclictest1967739-21kworker/u8:1+flush-259:010:13:182
178139299128,3cyclictest1823215-21apt-get07:50:132
178139299128,3cyclictest1728295-21kworker/u8:4+flush-259:007:45:082
178139299127,3cyclictest1728295-21kworker/u8:4+flush-259:008:30:172
178139299123,3cyclictest2029849-21kworker/u8:0+flush-259:011:20:002
178139299120,5cyclictest20435252kthreadcore11:20:192
178138899129,2cyclictest2123568-21idleruntime-cro12:40:001
178138899128,3cyclictest507-21in:imuxsock09:45:011
178138899128,3cyclictest2108162-21fschecks_count12:25:151
178138899127,3cyclictest252-21systemd-journal11:23:471
178138899124,6cyclictest2076765-21cp11:55:011
1781388991210,1cyclictest0-21swapper/111:05:001
178138899120,11cyclictest587621-21sshd10:49:481
178139499117,3cyclictest1728295-21kworker/u8:4+events_unbound08:35:183
178139299118,2cyclictest1849025-21cpuspeed_turbos08:15:142
178139299117,3cyclictest1854513-21sshd08:20:162
178139299115,4cyclictest1978592-21perl10:20:142
178139299110,3cyclictest1833257-21idleruntime-cro08:00:002
178138899119,1cyclictest0-21swapper/108:50:001
178138899118,2cyclictest2098666-21kthreadcore12:15:171
178138899115,4cyclictest1828067-21cron07:55:001
178139499107,2cyclictest2119719-21kthreadcore12:35:173
178139499107,1cyclictest0-21swapper/311:15:243
178139499106,2cyclictest1953535-21kthreadcore09:55:153
178139499105,4cyclictest1822552-21users07:45:273
178139499105,3cyclictest252-21systemd-journal09:11:193
178139299107,2cyclictest1983899-21cut10:25:162
178139299107,2cyclictest0-21swapper/209:35:262
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional