You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-02 - 11:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Thu Jul 02, 2026 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1904540995544,8cyclictest335557-21systemd-journal22:20:012
1904538994742,4cyclictest2039313-21gpgv21:20:001
1904538994634,10cyclictest1971943-21base6420:15:001
1904538994035,3cyclictest1930455-21kworker/u8:2+flush-259:019:35:191
1904534994036,3cyclictest2128322-21kworker/u8:3+flush-259:023:25:000
1904538993933,4cyclictest509-21rs:main1
1904534993835,2cyclictest75-21kswapd020:15:030
1904544993731,4cyclictest1982522-21latency_hist20:25:013
1904544993730,5cyclictest2148742-21latency_hist23:05:013
1904534993733,3cyclictest2096999-21apt-get22:15:110
1904540993630,3cyclictest2164603-21kworker/u8:0+flush-259:023:25:162
1904534993630,4cyclictest2242398-21diskstats00:35:130
1904544993528,5cyclictest2247134-21latency_hist00:40:013
1904540993530,4cyclictest1987652-21kworker/u8:1+flush-259:021:05:212
1904540993530,4cyclictest1987652-21kworker/u8:1+flush-259:021:05:202
1904540993429,3cyclictest2133507-21kworker/u8:4+flush-259:023:15:002
1904534993320,11cyclictest2185589-21diskstats23:40:120
1904534993318,14cyclictest2133516-21/usr/sbin/munin22:50:210
1904544993226,4cyclictest2003439-21apt-get20:45:113
1904540993228,3cyclictest2133507-21kworker/u8:4+events_unbound22:50:112
1904534993227,4cyclictest2159024-21latency_hist23:15:010
1904544993125,4cyclictest2050186-21/usr/sbin/munin21:30:153
1904544993125,4cyclictest1904725-21kworker/u8:3+flush-259:019:15:263
1904540993124,5cyclictest2096735-21latency_hist22:15:002
1904538993125,4cyclictest2055085-21latency_hist21:35:001
1904534993113,17cyclictest2023783-21turbostat21:05:000
1904544993025,3cyclictest2206097-21/usr/sbin/munin00:00:143
1904538993025,4cyclictest2101903-21latency_hist22:20:011
1904544992922,4cyclictest1956521-21/usr/sbin/munin20:00:143
1904534992925,3cyclictest2045027-21apt-get21:25:110
1904544992824,3cyclictest2133507-21kworker/u8:4+flush-259:023:10:243
1904538992823,4cyclictest1904498-21kworker/u8:0+flush-259:019:20:161
1904534992822,4cyclictest1935581-21latency_hist19:40:010
1904540992723,3cyclictest2128322-21kworker/u8:3+flush-259:023:20:162
1904540992723,3cyclictest1961640-21kworker/u8:4+events_unbound20:20:012
1904540992722,4cyclictest335557-21systemd-journal20:47:122
1904540992722,4cyclictest2039196-21apt-get21:19:592
1904540992722,4cyclictest1904725-21kworker/u8:3+events_unbound19:15:232
1904544992619,5cyclictest2023770-21latency_hist21:05:013
1904540992623,2cyclictest1930455-21kworker/u8:2+flush-259:019:55:152
1904540992622,3cyclictest2164603-21kworker/u8:0+events_unbound00:05:252
1904540992622,3cyclictest2055100-21kworker/u8:1+flush-259:022:40:212
1904540992622,3cyclictest1846236-21kworker/u8:4+flush-259:019:45:242
1904538992622,3cyclictest2091805-21kworker/u8:2+flush-259:022:39:591
1904538992622,3cyclictest1961640-21kworker/u8:4+flush-259:020:24:591
1904544992520,4cyclictest1930455-21kworker/u8:2+flush-259:019:45:163
1904544992518,5cyclictest335557-21systemd-journal23:45:003
1904540992521,3cyclictest1961640-21kworker/u8:4+flush-259:020:20:232
1904538992521,3cyclictest2070586-21kworker/u8:3+flush-259:022:20:221
1904538992521,3cyclictest1925493-21/usr/sbin/munin19:30:131
1904538992520,4cyclictest335557-21systemd-journal20:58:111
1904534992521,3cyclictest2195077-21kworker/u8:1+flush-259:023:50:000
1904544992419,3cyclictest2128322-21kworker/u8:3+flush-259:023:15:153
1904544992417,5cyclictest2091669-21cat22:10:013
1904540992420,3cyclictest2023797-21kworker/u8:2+flush-259:021:40:252
1904540992420,3cyclictest1951287-21kworker/u8:3+events_unbound20:05:202
1904540992419,3cyclictest2133507-21kworker/u8:4+events_unbound00:22:232
1904538992419,4cyclictest335557-21systemd-journal21:05:001
1904538992419,3cyclictest2185159-21latency_hist23:40:011
1904544992319,3cyclictest75-21kswapd020:15:023
1904544992319,3cyclictest2091805-21kworker/u8:2+flush-259:023:00:003
1904544992318,4cyclictest2008351-21kworker/u8:0+flush-259:021:35:263
1904540992320,2cyclictest2164603-21kworker/u8:0+events_unbound00:15:222
1904540992320,2cyclictest2070586-21kworker/u8:3+events_unbound22:20:242
1904540992320,2cyclictest1930455-21kworker/u8:2+flush-259:019:50:222
1904540992319,3cyclictest2091805-21kworker/u8:2+events_unbound22:30:532
1904540992319,3cyclictest2070586-21kworker/u8:3+flush-259:022:25:192
1904540992319,3cyclictest2055100-21kworker/u8:1+events_unbound22:37:122
1904540992318,3cyclictest2008351-21kworker/u8:0+events_unbound21:55:162
1904540992316,4cyclictest2060263-21awk21:40:002
190453899238,4cyclictest335557-21systemd-journal00:00:001
1904544992217,4cyclictest2164603-21kworker/u8:0+flush-259:000:15:153
1904544992217,3cyclictest2185171-21sh23:39:593
1904540992219,2cyclictest2236644-21kworker/u8:1+flush-259:000:30:172
1904540992219,2cyclictest2164603-21kworker/u8:0+events_unbound23:40:262
1904540992217,3cyclictest2133507-21kworker/u8:4+flush-259:023:15:212
1904538992216,4cyclictest2190752-21diskstats23:45:131
1904538992210,2cyclictest2237466-21irqcore00:30:151
1904534992217,4cyclictest1987873-21/usr/sbin/munin20:30:230
1904534992216,5cyclictest1986809-21ntpq20:25:220
1904544992116,3cyclictest2029168-21kworker/u8:3+flush-259:021:10:243
1904544992114,5cyclictest335557-21systemd-journal22:35:003
1904544992113,6cyclictest2049097-21ntpq21:25:233
1904540992118,2cyclictest2164603-21kworker/u8:0+events_unbound00:13:352
1904540992118,2cyclictest2128322-21kworker/u8:3+flush-259:023:30:242
1904540992118,2cyclictest1993059-21kworker/u8:2+events_unbound20:37:292
1904540992117,3cyclictest2164603-21kworker/u8:0+events_unbound23:50:162
1904538992116,4cyclictest335557-21systemd-journal22:00:001
1904538992116,3cyclictest1904725-21kworker/u8:3+flush-259:019:15:161
1904534992117,2cyclictest1910016-21apt-get19:15:120
1904534992116,4cyclictest2107707-21/usr/sbin/munin22:25:150
1904544992017,2cyclictest1987652-21kworker/u8:1+events_unbound21:05:183
1904544992017,2cyclictest1987652-21kworker/u8:1+events_unbound21:05:173
1904544992014,4cyclictest2017815-21ntpq20:55:223
1904544992014,4cyclictest2003174-21latency_hist20:45:003
1904544992013,4cyclictest2153885-21latency_hist23:10:013
190454499200,18cyclictest2101952-21unin-run22:20:013
1904540992017,2cyclictest2023797-21kworker/u8:2+events_unbound21:25:132
1904540992016,3cyclictest335557-21systemd-journal20:29:592
1904538992017,2cyclictest2128322-21kworker/u8:3+flush-259:023:30:151
1904538992016,3cyclictest2154141-21apt-get23:10:111
1904538992016,3cyclictest2055353-21apt-get21:35:131
1904544991915,3cyclictest2112648-21kworker/u8:0+flush-259:022:50:003
1904544991913,4cyclictest2080846-21latency_hist22:00:013
1904544991913,4cyclictest2039198-21latency_hist21:20:013
1904540991916,2cyclictest1987652-21kworker/u8:1+flush-259:021:00:242
1904540991916,2cyclictest1930455-21kworker/u8:2+events_unbound20:00:202
1904540991916,2cyclictest1904498-21kworker/u8:0+flush-259:019:10:252
1904540991915,3cyclictest2164603-21kworker/u8:0+events_unbound00:00:102
1904540991915,3cyclictest1930455-21kworker/u8:2+events_unbound19:39:112
1904540991914,4cyclictest1987652-21kworker/u8:1+events_unbound20:42:122
1904540991911,5cyclictest2138868-21diskstats22:55:132
1904538991916,2cyclictest2222017-21irqcore00:15:151
1904538991916,2cyclictest2008337-21turbostat.cron20:50:001
1904534991916,2cyclictest2201332-21irqcore23:55:170
1904534991914,4cyclictest60350irq/132-enp0s29f1:rx-021:43:580
1904544991814,3cyclictest1961032-21kworker/u8:0+flush-259:020:29:593
1904544991812,4cyclictest2013477-21latency_hist20:55:013
1904544991810,5cyclictest1992830-21latency_hist20:35:003
1904540991815,2cyclictest1920122-21kworker/u8:1+flush-259:019:25:162
1904540991813,3cyclictest2008351-21kworker/u8:0+events_unbound21:45:162
1904538991816,1cyclictest2128322-21kworker/u8:3+flush-259:023:16:021
1904538991815,2cyclictest1925235-21cron19:30:001
1904538991814,3cyclictest2091805-21kworker/u8:2+flush-259:022:45:211
1904538991813,3cyclictest2070571-21latency_hist21:50:011
1904544991714,2cyclictest1932999-21kthreadcore19:35:183
1904544991712,4cyclictest2164829-21apt-get23:20:133
1904544991712,4cyclictest1946330-21diskstats19:50:143
1904540991714,2cyclictest2070586-21kworker/u8:3+events_unbound22:07:382
1904540991714,2cyclictest1987652-21kworker/u8:1+events_unbound20:34:112
1904540991714,1cyclictest0-21swapper/223:05:002
1904540991713,3cyclictest2133507-21kworker/u8:4+events_unbound00:28:162
1904540991713,3cyclictest2029168-21kworker/u8:3+events_unbound21:10:372
1904540991712,3cyclictest1993059-21kworker/u8:2+flush-259:020:50:192
1904540991710,2cyclictest2055092-21munin-run21:34:592
1904534991714,2cyclictest60350irq/132-enp0s29f1:rx-023:30:150
1904534991714,2cyclictest60350irq/132-enp0s29f1:rx-021:15:030
1904534991712,4cyclictest335557-21systemd-journal20:43:110
1904534991712,4cyclictest2220477-21ntpq00:10:230
1904534991711,5cyclictest1965692-21ntpq20:05:190
1904544991612,3cyclictest2133507-21kworker/u8:4+flush-259:000:20:173
1904544991612,3cyclictest2070586-21kworker/u8:3+flush-259:021:50:253
1904544991611,3cyclictest1961640-21kworker/u8:4+flush-259:020:10:013
190454099169,5cyclictest335557-21systemd-journal19:32:102
1904540991612,3cyclictest2133507-21kworker/u8:4+events_unbound23:45:202
1904540991612,3cyclictest2133507-21kworker/u8:4+events_unbound23:05:222
1904540991612,3cyclictest2091805-21kworker/u8:2+events_unbound22:45:172
1904538991610,4cyclictest335557-21systemd-journal21:55:011
1904538991610,4cyclictest2117769-21latency_hist22:35:001
190453899160,15cyclictest1-21systemd23:09:111
1904534991614,1cyclictest60350irq/132-enp0s29f1:rx-021:55:160
1904534991614,1cyclictest60350irq/132-enp0s29f1:rx-021:10:160
1904534991614,1cyclictest60350irq/132-enp0s29f1:rx-019:23:510
1904534991614,1cyclictest60350irq/132-enp0s29f1:rx-000:05:200
1904534991613,2cyclictest61150irq/133-enp0s29f1:tx-019:31:110
1904534991613,2cyclictest60350irq/132-enp0s29f1:rx-022:48:020
1904534991613,2cyclictest60350irq/132-enp0s29f1:rx-020:10:110
1904534991613,2cyclictest60350irq/132-enp0s29f1:rx-000:25:160
1904534991612,3cyclictest2107467-21latency_hist22:25:000
1904534991612,3cyclictest1993059-21kworker/u8:2+flush-259:020:50:150
1904534991611,4cyclictest335557-21systemd-journal21:50:010
190454499158,5cyclictest1970887-21ntpq20:10:213
1904544991512,2cyclictest2211717-21irqcore00:05:213
1904540991511,3cyclictest2070586-21kworker/u8:3+events_unbound21:50:192
1904538991513,1cyclictest0-21swapper/122:05:011
1904538991511,3cyclictest2086273-21apt-get22:05:111
1904534991513,2cyclictest60350irq/132-enp0s29f1:rx-020:49:390
1904534991513,1cyclictest60350irq/132-enp0s29f1:rx-023:50:130
1904534991513,1cyclictest60350irq/132-enp0s29f1:rx-023:28:440
1904534991513,1cyclictest60350irq/132-enp0s29f1:rx-023:04:590
1904534991513,1cyclictest60350irq/132-enp0s29f1:rx-022:55:250
1904534991513,1cyclictest60350irq/132-enp0s29f1:rx-021:31:090
1904534991513,1cyclictest60350irq/132-enp0s29f1:rx-021:21:320
1904534991513,1cyclictest60350irq/132-enp0s29f1:rx-020:00:030
1904534991513,1cyclictest60350irq/132-enp0s29f1:rx-019:50:140
1904534991513,1cyclictest60350irq/132-enp0s29f1:rx-019:47:420
1904534991512,2cyclictest60350irq/132-enp0s29f1:rx-000:20:210
1904534991512,2cyclictest2185170-21turbostat23:40:000
1904534991511,3cyclictest2024036-21apt-get21:05:110
1904534991511,3cyclictest2024036-21apt-get21:05:100
190454499147,4cyclictest2008351-21kworker/u8:0+flush-259:022:10:013
190454499146,6cyclictest2065418-21cron21:45:013
1904544991411,2cyclictest0-21swapper/319:45:013
1904544991410,3cyclictest2133507-21kworker/u8:4+events_unbound00:00:013
1904544991410,3cyclictest2055100-21kworker/u8:1+flush-259:022:35:253
1904540991410,3cyclictest335557-21systemd-journal19:42:112
1904540991410,3cyclictest2236644-21kworker/u8:1+flush-259:000:35:182
1904540991410,3cyclictest2164603-21kworker/u8:0+flush-259:023:55:262
1904540991410,3cyclictest2070586-21kworker/u8:3+events_unbound22:04:222
1904540991410,3cyclictest2013702-21kworker/u8:3+events_unbound20:58:462
190453899149,4cyclictest335557-21systemd-journal23:27:111
1904538991411,2cyclictest2040295-21irqcore21:20:141
1904538991410,3cyclictest75-21kswapd020:15:031
1904538991410,3cyclictest335557-21systemd-journal23:53:121
1904534991412,2cyclictest61150irq/133-enp0s29f1:tx-021:50:130
1904534991412,1cyclictest60350irq/132-enp0s29f1:rx-023:15:140
1904534991412,1cyclictest60350irq/132-enp0s29f1:rx-022:13:520
1904534991412,1cyclictest60350irq/132-enp0s29f1:rx-022:08:470
1904534991412,1cyclictest60350irq/132-enp0s29f1:rx-020:37:000
1904534991412,1cyclictest60350irq/132-enp0s29f1:rx-019:25:160
1904534991412,1cyclictest60350irq/132-enp0s29f1:rx-000:00:140
1904534991411,2cyclictest60350irq/132-enp0s29f1:rx-020:55:130
190454499139,3cyclictest1993059-21kworker/u8:2+events_unbound20:35:243
190454499136,5cyclictest335557-21systemd-journal21:24:593
190454099139,3cyclictest2008351-21kworker/u8:0+events_unbound21:20:232
190454099139,3cyclictest1904498-21kworker/u8:0+events_unbound19:20:252
190453899138,4cyclictest335557-21systemd-journal21:29:111
190453899137,4cyclictest335557-21systemd-journal22:51:121
1904538991311,1cyclictest1946342-21fschecks_count19:50:151
1904538991311,1cyclictest0-21swapper/123:05:001
1904538991310,2cyclictest2091805-21kworker/u8:2+events_unbound22:10:161
1904538991310,2cyclictest1966577-21idleruntime-cro20:10:011
1904534991312,1cyclictest60350irq/132-enp0s29f1:rx-022:34:020
1904534991312,1cyclictest60350irq/132-enp0s29f1:rx-022:00:220
1904534991311,1cyclictest60350irq/132-enp0s29f1:rx-019:10:130
1904534991311,1cyclictest60350irq/132-enp0s29f1:rx-000:15:140
1904534991310,2cyclictest60350irq/132-enp0s29f1:rx-021:35:170
1904534991310,2cyclictest2122091-21ntp_offset22:35:220
1904534991310,2cyclictest1955271-21needreboot19:55:190
190454499129,2cyclictest2108143-21irqcore22:25:163
190454499128,3cyclictest335557-21systemd-journal23:49:553
190454499128,2cyclictest0-21swapper/300:30:213
190454499127,4cyclictest1925488-21sshd19:30:103
190454099129,2cyclictest1961640-21kworker/u8:4+events_unbound20:10:132
190453899129,2cyclictest2164603-21kworker/u8:0+flush-259:000:10:001
190453899129,2cyclictest1909280-21unixbench_multi19:10:241
190453899128,3cyclictest2023797-21kworker/u8:2+flush-259:021:40:241
190453899128,3cyclictest2008351-21kworker/u8:0+flush-259:021:06:121
190453899128,3cyclictest2008351-21kworker/u8:0+flush-259:021:06:111
190453499129,2cyclictest60350irq/132-enp0s29f1:rx-020:20:100
190453499129,2cyclictest60350irq/132-enp0s29f1:rx-019:41:300
1904534991210,1cyclictest0-21swapper/000:30:130
190454499118,2cyclictest1951454-21cpuspeed_turbos19:55:133
190454499118,2cyclictest0-21swapper/322:40:133
190454499117,3cyclictest491ktimers/319:10:213
190454499117,2cyclictest2200646-21latency_hist23:55:013
190453899119,1cyclictest2165850-21sort23:20:181
190453899119,1cyclictest0-21swapper/100:25:251
190453899118,2cyclictest2012992-21unixbench_multi20:50:241
190453899118,2cyclictest0-21swapper/123:00:001
190453899117,3cyclictest1941188-21diskstats19:45:141
190453899116,4cyclictest1951287-21kworker/u8:3+flush-259:020:40:171
190453499118,2cyclictest2149073-21cpuspeed_turbos23:05:140
190454499105,4cyclictest1925342-21cat19:30:013
190454099107,2cyclictest2128322-21kworker/u8:3+events_unbound23:35:012
19045449996,2cyclictest2103614-21kthreadcore22:20:173
19045389996,2cyclictest1985822-21cut20:25:171
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional