You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-04 - 07:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Sat Apr 04, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1959912995451,2cyclictest2188603-21kworker/u8:2+events_unbound23:25:153
1959900995448,4cyclictest2199067-21diskstats23:00:130
1959900994945,3cyclictest2106066-21kworker/u8:4+flush-259:021:35:150
1959912994743,3cyclictest2085040-21kworker/u8:2+events_unbound21:30:253
1959912994340,2cyclictest2272570-21kworker/u8:3+events_unbound00:20:243
1959912994340,2cyclictest2188603-21kworker/u8:2+events_unbound22:55:233
1959912994340,2cyclictest2085040-21kworker/u8:2+events_unbound21:10:133
1959912994340,2cyclictest1959876-21kworker/u8:1+events_unbound19:15:133
1959912994339,3cyclictest2038672-21kworker/u8:0+events_unbound21:00:233
1959912994137,3cyclictest4088381-21kworker/u8:0+events_unbound19:25:183
1959912994036,3cyclictest2012884-21kworker/u8:2+events_unbound20:25:213
1959903994035,3cyclictest2116445-21apt-get21:40:121
1959903994034,4cyclictest2282273-21apt-get00:20:131
1959912993936,2cyclictest2038672-21kworker/u8:0+events_unbound21:15:163
1959912993936,2cyclictest2007538-21kworker/u8:3+events_unbound20:00:223
1959912993733,3cyclictest2059066-21kworker/u8:3+events_unbound20:55:143
1959906993732,3cyclictest252-21systemd-journal23:09:142
1959903993730,5cyclictest252-21systemd-journal23:40:001
1959912993634,2cyclictest2245764-21kworker/u8:0+events_unbound23:50:213
1959912993632,3cyclictest2094683-21kworker/u8:1+events_unbound22:05:283
1959900993630,4cyclictest2271962-21/usr/sbin/munin00:10:120
1959906993531,3cyclictest1965240-21kworker/u8:2+flush-259:019:35:162
1959900993530,3cyclictest2214365-21apt-get23:15:090
1959912993431,2cyclictest2064410-21kworker/u8:1+events_unbound20:50:143
1959912993429,4cyclictest2126813-21apt-get21:50:113
1959912993428,4cyclictest2178309-21/usr/sbin/munin22:40:143
1959912993428,4cyclictest2137299-21diskstats22:00:143
1959903993430,3cyclictest4088381-21kworker/u8:0+flush-259:019:25:151
1959903993429,4cyclictest2147435-21kworker/u8:0+flush-259:023:05:171
1959903993428,4cyclictest2007779-21apt-get19:55:111
1959900993430,3cyclictest2147435-21kworker/u8:0+flush-259:022:35:010
1959900993429,3cyclictest1991864-21/usr/sbin/munin19:40:200
1959912993330,2cyclictest2168023-21kworker/u8:3+events_unbound23:00:143
1959912993327,4cyclictest2219276-21latency_hist23:20:013
1959906993329,2cyclictest2012884-21kworker/u8:2+flush-259:020:35:182
1959906993326,5cyclictest2209366-21diskstats23:10:122
1959903993327,4cyclictest252-21systemd-journal20:44:151
1959912993229,2cyclictest1965240-21kworker/u8:2+events_unbound19:45:213
1959900993227,4cyclictest2240371-21latency_hist23:40:010
1959900993226,5cyclictest2012923-21apt-get20:00:110
1959906993128,2cyclictest2178307-21kworker/u8:1+events_unbound22:40:142
1959906993127,3cyclictest2168031-21/usr/sbin/munin22:30:142
1959903993126,3cyclictest1975733-21latency_hist19:25:001
1959903993125,4cyclictest2302685-21latency_hist00:40:011
1959912993027,2cyclictest2168023-21kworker/u8:3+events_unbound23:05:293
1959912993027,2cyclictest2085040-21kworker/u8:2+events_unbound21:25:293
1959912993023,5cyclictest2301803-21ntpq00:35:223
1959903993026,3cyclictest1986750-21apt-get19:35:111
1959912992926,2cyclictest2147435-21kworker/u8:0+events_unbound22:49:143
1959912992926,2cyclictest2147435-21kworker/u8:0+events_unbound22:21:333
1959912992925,3cyclictest2007538-21kworker/u8:3+events_unbound20:05:293
1959906992923,4cyclictest252-21systemd-journal00:23:502
1959903992925,3cyclictest2085040-21kworker/u8:2+flush-259:021:20:261
1959903992922,5cyclictest2002070-21base6419:49:591
1959900992926,2cyclictest2256284-21kworker/u8:1+events_unbound00:00:140
1959900992925,3cyclictest2245764-21kworker/u8:0+flush-259:023:50:250
1959912992825,2cyclictest2085040-21kworker/u8:2+events_unbound21:59:153
1959912992825,2cyclictest1959876-21kworker/u8:1+events_unbound19:10:213
1959900992823,3cyclictest2085047-21/usr/sbin/munin21:10:230
1959912992724,3cyclictest2038672-21kworker/u8:0+events_unbound20:34:163
1959903992720,5cyclictest252-21systemd-journal21:15:001
1959912992624,2cyclictest2094683-21kworker/u8:1+events_unbound22:15:123
1959912992623,2cyclictest2225084-21kworker/u8:1+events_unbound23:31:153
1959912992619,3cyclictest2147435-21kworker/u8:0+events_unbound22:50:263
1959906992623,2cyclictest2012884-21kworker/u8:2+flush-259:020:40:222
1959906992622,3cyclictest2162848-21/usr/sbin/munin22:25:142
1959906992622,3cyclictest2162848-21/usr/sbin/munin22:25:132
1959906992622,3cyclictest2007538-21kworker/u8:3+events_unbound20:00:192
1959906992621,3cyclictest2230645-21kworker/u8:3+events_unbound23:40:142
1959900992622,3cyclictest2287449-21apt-get00:25:110
1959912992522,2cyclictest2245764-21kworker/u8:0+events_unbound00:08:143
1959912992522,2cyclictest2094683-21kworker/u8:1+events_unbound21:39:153
1959912992519,4cyclictest252-21systemd-journal23:45:153
1959903992521,2cyclictest2193485-21latency_hist22:55:001
1959903992520,3cyclictest2152596-21/usr/sbin/munin22:15:131
1959903992519,4cyclictest1960276-21diskstats19:10:141
1959900992522,2cyclictest2085040-21kworker/u8:2+flush-259:022:35:150
1959900992521,3cyclictest2224597-21base6423:25:000
1959900992521,3cyclictest2007538-21kworker/u8:3+events_unbound20:15:150
1959912992422,2cyclictest2001769-21kworker/u8:0+events_unbound19:57:113
1959912992420,3cyclictest1965240-21kworker/u8:2+events_unbound19:33:133
1959906992421,2cyclictest2038672-21kworker/u8:0+flush-259:020:45:162
1959900992420,3cyclictest2012884-21kworker/u8:2+events_unbound20:25:160
1959900992419,3cyclictest2012884-21kworker/u8:2+flush-259:020:40:160
1959912992321,2cyclictest2240071-21kworker/u8:2+events_unbound23:39:143
1959912992320,2cyclictest2147435-21kworker/u8:0+events_unbound23:10:183
1959912992320,2cyclictest2012884-21kworker/u8:2+events_unbound20:35:193
1959912992320,2cyclictest1959876-21kworker/u8:1+events_unbound19:23:123
1959912992318,4cyclictest2023193-21/usr/sbin/munin20:10:133
1959912992318,4cyclictest2023193-21/usr/sbin/munin20:10:133
1959906992320,2cyclictest2094683-21kworker/u8:1+events_unbound22:05:172
1959906992320,2cyclictest1981345-21kworker/u8:3+events_unbound19:45:152
1959906992319,3cyclictest2168023-21kworker/u8:3+flush-259:023:00:212
1959906992319,3cyclictest2106066-21kworker/u8:4+flush-259:021:35:162
1959906992319,3cyclictest2012884-21kworker/u8:2+events_unbound20:30:142
1959903992319,3cyclictest2183493-21apt-get22:45:121
1959903992318,3cyclictest2147435-21kworker/u8:0+flush-259:023:30:241
1959900992319,3cyclictest2245653-21gpgv23:45:010
1959900992319,3cyclictest2079930-21apt-get21:05:110
1959900992316,5cyclictest1969284-21ntpq19:15:190
1959900992315,6cyclictest2001043-21ntpq19:45:200
1959900992312,9cyclictest2262052-21kworker/u8:2+flush-259:000:30:220
1959912992219,2cyclictest2245764-21kworker/u8:0+events_unbound00:01:463
1959912992219,2cyclictest2059066-21kworker/u8:3+events_unbound21:09:573
1959912992219,2cyclictest2007538-21kworker/u8:3+events_unbound20:16:083
1959912992218,3cyclictest1981345-21kworker/u8:3+events_unbound19:41:053
1959906992219,2cyclictest4088381-21kworker/u8:0+flush-259:019:20:212
1959906992219,2cyclictest2262052-21kworker/u8:2+flush-259:000:25:252
1959906992218,3cyclictest4088381-21kworker/u8:0+events_unbound19:15:192
1959906992218,3cyclictest252-21systemd-journal00:02:442
1959906992218,3cyclictest2147435-21kworker/u8:0+flush-259:022:45:202
1959906992216,5cyclictest2224432-21apt-get23:25:002
1959903992218,3cyclictest2245764-21kworker/u8:0+flush-259:023:55:161
1959903992217,3cyclictest2028377-21/usr/sbin/munin20:15:141
1959900992216,4cyclictest252-21systemd-journal20:40:000
1959900992216,4cyclictest1991624-21latency_hist19:40:000
1959912992118,2cyclictest2272570-21kworker/u8:3+events_unbound00:16:503
1959912992118,2cyclictest2085040-21kworker/u8:2+events_unbound22:14:163
1959906992119,1cyclictest1959876-21kworker/u8:1+flush-259:019:55:232
1959906992114,5cyclictest2094196-21ntpq21:15:222
1959903992116,4cyclictest2272570-21kworker/u8:3+events_unbound00:10:191
1959903992116,4cyclictest2225117-21apt-get23:25:121
1959903992114,5cyclictest2073596-21ntpq20:55:201
1959900992118,2cyclictest2245764-21kworker/u8:0+flush-259:000:05:160
1959900992118,2cyclictest2147435-21kworker/u8:0+flush-259:023:30:160
1959912992018,2cyclictest2287414-21kworker/u8:1+events_unbound00:25:293
1959912992018,2cyclictest2085040-21kworker/u8:2+events_unbound22:36:163
1959912992018,1cyclictest2012884-21kworker/u8:2+events_unbound20:41:143
1959912992017,3cyclictest2106066-21kworker/u8:4+events_unbound21:41:163
1959912992016,3cyclictest2121582-21/usr/sbin/munin21:45:153
1959906992017,2cyclictest2085040-21kworker/u8:2+flush-259:021:25:262
1959906992016,3cyclictest2188603-21kworker/u8:2+events_unbound22:50:152
1959903992016,3cyclictest2106066-21kworker/u8:4+flush-259:021:50:221
1959903992014,4cyclictest2126556-21latency_hist21:50:001
195990399200,19cyclictest1-21systemd20:13:151
195990399200,19cyclictest1-21systemd20:13:151
1959912991917,2cyclictest2262052-21kworker/u8:2+events_unbound00:14:523
1959912991916,2cyclictest1981345-21kworker/u8:3+events_unbound19:39:173
1959906991916,3cyclictest2287414-21kworker/u8:1+events_unbound00:35:162
1959906991916,2cyclictest2007538-21kworker/u8:3+flush-259:020:15:232
1959906991916,2cyclictest2001769-21kworker/u8:0+events_unbound19:50:192
1959906991916,2cyclictest1981345-21kworker/u8:3+events_unbound19:40:152
1959906991916,2cyclictest1959876-21kworker/u8:1+events_unbound19:10:252
1959906991915,3cyclictest2085040-21kworker/u8:2+events_unbound22:35:152
1959903991915,3cyclictest2272570-21kworker/u8:3+flush-259:000:15:201
1959900991916,2cyclictest2085040-21kworker/u8:2+flush-259:021:20:210
1959900991915,3cyclictest2022342-21kworker/u8:1+flush-259:020:06:070
1959900991913,4cyclictest252-21systemd-journal20:13:150
1959900991913,4cyclictest252-21systemd-journal20:13:150
1959912991815,2cyclictest2245764-21kworker/u8:0+events_unbound23:55:523
1959912991814,3cyclictest2262052-21kworker/u8:2+events_unbound00:33:203
1959912991814,3cyclictest2188603-21kworker/u8:2+events_unbound23:20:013
1959906991815,2cyclictest2085040-21kworker/u8:2+flush-259:022:00:202
1959906991814,3cyclictest4088381-21kworker/u8:0+flush-259:019:28:172
1959903991815,2cyclictest0-21swapper/121:20:001
1959903991814,3cyclictest2157223-21kworker/u8:3+flush-259:022:20:161
1959903991813,4cyclictest2111282-21apt-get21:35:101
1959900991815,2cyclictest2038672-21kworker/u8:0+events_unbound20:50:240
1959900991814,3cyclictest252-21systemd-journal20:30:150
1959900991814,3cyclictest2085040-21kworker/u8:2+events_unbound22:10:180
1959900991813,4cyclictest2007521-21latency_hist19:55:010
1959900991812,4cyclictest252-21systemd-journal21:57:150
1959912991713,3cyclictest2012884-21kworker/u8:2+events_unbound20:20:263
1959912991713,2cyclictest0-21swapper/323:45:003
195990699173,7cyclictest2131650-21cron21:55:002
1959906991714,2cyclictest2971-21cron20:15:002
1959906991714,2cyclictest2971-21cron20:15:002
1959906991714,2cyclictest2106066-21kworker/u8:4+flush-259:021:40:232
1959906991714,2cyclictest1965240-21kworker/u8:2+events_unbound19:30:142
1959906991713,3cyclictest2245764-21kworker/u8:0+flush-259:023:55:222
1959906991712,4cyclictest2272570-21kworker/u8:3+flush-259:000:10:162
195990399179,5cyclictest2042727-21ntpq20:25:211
1959903991713,2cyclictest0-21swapper/120:05:001
1959903991712,3cyclictest2266781-21/usr/sbin/munin00:05:141
1959900991714,2cyclictest61150irq/133-enp0s29f1:tx-023:05:250
1959900991714,2cyclictest2095061-21sh21:20:000
1959900991714,2cyclictest2094683-21kworker/u8:1+flush-259:022:15:230
1959900991713,3cyclictest2147435-21kworker/u8:0+flush-259:022:50:250
1959900991713,3cyclictest2106066-21kworker/u8:4+events_unbound21:50:140
1959900991712,4cyclictest2225278-21diskstats23:25:140
1959912991613,2cyclictest2168023-21kworker/u8:3+flush-259:022:34:353
1959906991613,2cyclictest0-21swapper/221:05:012
1959906991612,3cyclictest2147435-21kworker/u8:0+events_unbound22:20:212
1959906991611,3cyclictest2251386-21apt-get23:50:122
1959903991610,4cyclictest2012884-21kworker/u8:2+flush-259:020:30:261
1959900991614,1cyclictest60350irq/132-enp0s29f1:rx-021:25:190
1959900991612,3cyclictest2059066-21kworker/u8:3+events_unbound20:55:120
195991299159,1cyclictest191rcu_preempt22:25:143
195991299159,1cyclictest191rcu_preempt22:25:143
1959912991512,2cyclictest2038672-21kworker/u8:0+events_unbound20:46:223
1959912991512,2cyclictest1959876-21kworker/u8:1+events_unbound19:53:413
195990699159,4cyclictest252-21systemd-journal22:55:142
195990699159,4cyclictest2225084-21kworker/u8:1+flush-259:023:30:272
1959906991512,2cyclictest2281993-21cron00:19:592
1959906991512,2cyclictest2106066-21kworker/u8:4+events_unbound21:30:212
195990399159,5cyclictest2250238-21ntpq23:45:201
195990399158,1cyclictest0-21swapper/120:40:001
195990399157,5cyclictest252-21systemd-journal00:28:501
1959903991512,2cyclictest0-21swapper/123:55:001
1959903991511,3cyclictest2245743-21store23:45:021
1959900991514,1cyclictest60350irq/132-enp0s29f1:rx-021:30:200
1959900991513,1cyclictest60350irq/132-enp0s29f1:rx-023:10:140
1959900991513,1cyclictest60350irq/132-enp0s29f1:rx-022:00:230
1959900991513,1cyclictest60350irq/132-enp0s29f1:rx-019:55:230
1959900991512,2cyclictest60350irq/132-enp0s29f1:rx-023:55:110
1959900991511,3cyclictest2178307-21kworker/u8:1+events_unbound22:45:000
1959900991511,3cyclictest2094683-21kworker/u8:1+flush-259:021:40:180
1959900991511,3cyclictest2012884-21kworker/u8:2+events_unbound20:45:170
1959900991511,3cyclictest2012884-21kworker/u8:2+events_unbound20:20:180
1959912991411,2cyclictest2038672-21kworker/u8:0+events_unbound21:23:403
195990699147,5cyclictest2251110-21cron23:50:002
195990699145,6cyclictest252-21systemd-journal20:55:002
1959906991412,1cyclictest2007538-21kworker/u8:3+events_unbound20:05:152
1959906991411,2cyclictest2096167-21irqcore21:20:162
1959906991411,2cyclictest2064410-21kworker/u8:1+events_unbound20:55:222
1959906991410,3cyclictest2085040-21kworker/u8:2+flush-259:021:55:252
1959906991410,3cyclictest2038672-21kworker/u8:0+flush-259:021:05:262
1959906991410,3cyclictest2038672-21kworker/u8:0+events_unbound20:30:002
195990399149,4cyclictest2131689-21kworker/u8:5+flush-259:022:05:251
195990399149,4cyclictest1959876-21kworker/u8:1+flush-259:019:30:461
195990399149,3cyclictest2147437-21/usr/sbin/munin22:10:151
195990399148,4cyclictest2297489-21latency_hist00:35:001
1959903991411,2cyclictest2183221-21idleruntime-cro22:45:001
1959903991410,3cyclictest2106271-21fschecks_count21:30:151
1959900991413,1cyclictest60350irq/132-enp0s29f1:rx-019:10:220
1959900991412,1cyclictest60350irq/132-enp0s29f1:rx-022:25:150
1959900991412,1cyclictest60350irq/132-enp0s29f1:rx-022:25:150
1959900991412,1cyclictest60350irq/132-enp0s29f1:rx-019:34:210
1959900991412,1cyclictest60350irq/132-enp0s29f1:rx-019:25:130
1959900991412,1cyclictest60350irq/132-enp0s29f1:rx-000:35:260
1959900991411,2cyclictest60350irq/132-enp0s29f1:rx-021:00:130
1959900991410,3cyclictest2282005-21latency_hist00:20:000
1959900991410,3cyclictest2272570-21kworker/u8:3+flush-259:000:20:260
195990699139,3cyclictest2147435-21kworker/u8:0+events_unbound22:10:142
1959906991310,2cyclictest0-21swapper/223:30:002
195990399139,3cyclictest1965498-21fschecks_time19:15:141
1959900991311,1cyclictest60350irq/132-enp0s29f1:rx-022:21:270
1959900991310,2cyclictest4088381-21kworker/u8:0+events_unbound19:20:250
1959900991310,2cyclictest252-21systemd-journal23:50:000
195990699126,4cyclictest252-21systemd-journal20:20:152
195990699120,11cyclictest61-21kcompactd023:35:132
195990399129,2cyclictest2167369-21unixbench_singl22:25:241
195990399129,2cyclictest2167369-21unixbench_singl22:25:241
195990399129,2cyclictest2147435-21kworker/u8:0+flush-259:023:11:121
195990399129,2cyclictest2002706-21cpuspeed_turbos19:50:141
195990399129,1cyclictest0-21swapper/121:05:161
195990099129,2cyclictest60350irq/132-enp0s29f1:rx-022:45:180
195990699118,2cyclictest2214784-21irqcore23:15:152
195990699115,5cyclictest2262052-21kworker/u8:2+flush-259:000:30:172
195990699115,4cyclictest252-21systemd-journal21:50:002
195990399118,2cyclictest0-21swapper/120:45:151
195990099119,1cyclictest0-21swapper/022:55:190
195990699108,1cyclictest0-21swapper/221:10:152
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional