You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-03 - 05:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Sat Jan 03, 2026 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3726243993519,14cyclictest3861833-21/usr/sbin/munin21:20:230
3726243993226,4cyclictest3964936-21latency_hist23:00:000
3726245993127,3cyclictest3730932-21kworker/u8:3+events_unbound20:30:271
3726243992922,5cyclictest252-21systemd-journal00:00:010
3726243992922,4cyclictest3928793-21latency_hist22:25:010
3726243992815,12cyclictest3871930-21idleruntime-cro21:30:010
372624399267,5cyclictest3846104-21latency_hist21:05:010
3726243992511,12cyclictest4063731-21fschecks_time00:35:140
3726253992420,3cyclictest3730932-21kworker/u8:3+events_unbound20:30:292
3726243992417,5cyclictest4052915-21latency_hist00:25:010
3726243992411,11cyclictest3856702-21apt-get21:15:120
3726257992320,2cyclictest3793859-21kworker/u8:1+flush-259:020:15:193
3726257992318,4cyclictest252-21systemd-journal00:15:013
3726245992320,2cyclictest3730932-21kworker/u8:3+flush-259:019:35:011
372624399238,2cyclictest3895187-21kthreadcore21:50:170
372624399231,9cyclictest3788868-21awk20:10:000
3726243992317,4cyclictest4012122-21diskstats23:45:140
3726243992310,12cyclictest4011683-21cron23:45:010
3726257992217,3cyclictest3933985-21kworker/u8:1+flush-259:023:10:173
3726257992217,3cyclictest3933985-21kworker/u8:1+flush-259:023:10:163
3726257992216,4cyclictest3834554-21ntpq20:50:243
3726253992217,4cyclictest3804320-21latency_hist20:25:002
3726245992219,2cyclictest3788548-21kworker/u8:0+flush-259:020:25:011
372624399227,1cyclictest4042595-21turbostat00:15:010
372624399226,5cyclictest3918904-21diskstats22:15:130
3726257992118,2cyclictest4032325-21kworker/u8:1+flush-259:000:25:003
3726253992118,2cyclictest3892074-21kworker/u8:4+flush-259:021:55:172
3726245992118,2cyclictest3730932-21kworker/u8:3+flush-259:019:25:001
3726245992117,3cyclictest4032325-21kworker/u8:1+events_unbound00:10:201
3726245992114,5cyclictest252-21systemd-journal22:56:391
372624399210,7cyclictest1-21systemd00:27:390
3726257992015,3cyclictest3933985-21kworker/u8:1+flush-259:022:55:183
372624399208,11cyclictest3773615-21sed19:55:160
3726243992015,3cyclictest3954390-21kworker/u8:3+flush-259:022:50:170
3726257991916,2cyclictest4011373-21kworker/u8:2+events_unbound23:45:243
3726257991916,2cyclictest3864150-21kthreadcore21:20:193
3726253991916,2cyclictest3856456-21kworker/u8:3+flush-259:021:20:012
3726253991916,1cyclictest3819860-21idleruntime-cro20:40:012
3726253991915,3cyclictest3980702-21kworker/u8:2+flush-259:023:20:012
3726253991915,3cyclictest3933985-21kworker/u8:1+flush-259:022:55:002
3726245991916,2cyclictest3954390-21kworker/u8:3+flush-259:023:00:211
3726245991916,2cyclictest3933985-21kworker/u8:1+flush-259:022:49:591
3726245991916,2cyclictest3819732-21kworker/u8:1+flush-259:020:40:161
3726245991915,3cyclictest3867013-21apt-get21:25:121
3726245991915,2cyclictest0-21swapper/122:10:001
3726245991913,4cyclictest252-21systemd-journal21:03:401
3726257991815,2cyclictest3866294-21kworker/u8:2+flush-259:021:35:253
3726257991813,3cyclictest252-21systemd-journal21:15:003
372625399184,6cyclictest3991105-21cron23:24:592
3726253991815,2cyclictest4016720-21kworker/u8:0+events_unbound00:00:212
3726253991814,3cyclictest3768012-21kworker/u8:1+flush-259:019:55:142
3726245991815,2cyclictest4047779-21kworker/u8:2+flush-259:000:30:281
3726245991815,2cyclictest4011373-21kworker/u8:2+flush-259:023:45:211
3726245991815,2cyclictest3959812-21kworker/u8:0+flush-259:023:15:251
3726245991815,2cyclictest3730932-21kworker/u8:3+flush-259:020:45:131
3726245991814,3cyclictest4016720-21kworker/u8:0+events_unbound00:05:221
3726245991813,4cyclictest3985983-21kworker/u8:3+events_unbound23:55:171
372624399181,7cyclictest3970759-21apt-get23:05:130
3726243991814,3cyclictest3978195-21kthreadcore23:10:170
3726243991814,3cyclictest3978195-21kthreadcore23:10:160
3726243991812,4cyclictest3876260-21ntpq21:30:230
372624399180,7cyclictest3768465-21irqcore19:50:200
3726257991713,3cyclictest3881896-21kworker/u8:3+flush-259:022:30:223
3726253991714,2cyclictest3933985-21kworker/u8:1+events_unbound22:45:212
3726253991714,2cyclictest3819732-21kworker/u8:1+events_unbound20:40:192
3726253991713,3cyclictest3796569-21kthreadcore20:15:212
3726245991714,2cyclictest3830121-21kworker/u8:0+events_unbound21:40:211
3726245991713,3cyclictest3980702-21kworker/u8:2+events_unbound23:20:211
3726245991713,3cyclictest3881896-21kworker/u8:3+flush-259:022:21:491
3726245991713,3cyclictest3793859-21kworker/u8:1+flush-259:020:15:411
3726245991713,3cyclictest3730932-21kworker/u8:3+flush-259:020:05:161
3726245991712,3cyclictest2971-21cron22:15:011
372624399176,2cyclictest60350irq/132-enp0s29f1:rx-021:40:190
3726243991713,2cyclictest60350irq/132-enp0s29f1:rx-020:55:240
3726243991712,4cyclictest60350irq/132-enp0s29f1:rx-021:59:180
3726243991712,4cyclictest60350irq/132-enp0s29f1:rx-000:16:490
3726243991712,3cyclictest1-21systemd23:35:000
3726243991711,5cyclictest60350irq/132-enp0s29f1:rx-022:45:190
3726257991614,1cyclictest4027169-21idleruntime-cro00:00:013
3726257991613,2cyclictest3799823-21irqcore20:20:173
3726257991613,2cyclictest3779205-21irqcore20:00:173
3726257991613,2cyclictest3730932-21kworker/u8:3+flush-259:020:25:223
3726257991612,3cyclictest3887253-21kworker/u8:1+flush-259:021:55:213
3726257991612,2cyclictest0-21swapper/319:45:003
3726257991612,2cyclictest0-21swapper/319:35:203
3726253991613,2cyclictest3996217-21kworker/u8:1+flush-259:023:50:202
3726253991612,3cyclictest3959812-21kworker/u8:0+flush-259:023:25:162
3726253991612,3cyclictest3954390-21kworker/u8:3+events_unbound23:05:152
3726253991612,3cyclictest3730932-21kworker/u8:3+flush-259:019:40:202
372624599169,5cyclictest3850369-21ntpq21:05:231
3726245991612,3cyclictest3996217-21kworker/u8:1+flush-259:023:35:141
3726245991612,3cyclictest3866294-21kworker/u8:2+flush-259:021:30:221
3726245991612,2cyclictest3866744-21sh21:25:001
3726245991610,4cyclictest3892867-21/usr/sbin/munin21:50:141
3726245991610,4cyclictest252-21systemd-journal20:55:001
3726243991614,1cyclictest60350irq/132-enp0s29f1:rx-020:45:030
3726243991613,2cyclictest61150irq/133-enp0s29f1:tx-023:15:200
3726243991613,2cyclictest60350irq/132-enp0s29f1:rx-020:20:140
3726243991613,2cyclictest60350irq/132-enp0s29f1:rx-019:45:210
3726243991613,2cyclictest60350irq/132-enp0s29f1:rx-000:30:190
3726243991612,4cyclictest60350irq/132-enp0s29f1:rx-022:12:230
3726243991612,2cyclictest0-21swapper/000:00:160
3726243991611,4cyclictest60350irq/132-enp0s29f1:rx-019:40:150
372624399161,13cyclictest1-21systemd22:27:390
372624399160,1cyclictest490-21dbus-daemon20:30:280
3726257991511,3cyclictest4047779-21kworker/u8:2+flush-259:000:30:273
3726257991511,2cyclictest0-21swapper/322:39:593
3726253991512,2cyclictest3693987-21kworker/u8:0+flush-259:019:15:212
3726253991511,3cyclictest3830121-21kworker/u8:0+events_unbound21:40:142
3726253991510,3cyclictest3830121-21kworker/u8:0+flush-259:020:50:212
3726245991513,1cyclictest3730932-21kworker/u8:3+events_unbound20:25:241
3726245991512,2cyclictest3830121-21kworker/u8:0+flush-259:021:15:261
3726245991512,2cyclictest3721117-21kworker/u8:2+flush-259:019:10:261
3726245991511,3cyclictest3959812-21kworker/u8:0+events_unbound23:25:191
3726245991511,3cyclictest3933985-21kworker/u8:1+flush-259:022:35:161
3726245991511,3cyclictest3730932-21kworker/u8:3+flush-259:019:25:171
372624599150,9cyclictest3908146-21munin-plugin-st22:05:001
372624399159,4cyclictest3949455-21latency_hist22:45:000
372624399159,2cyclictest60350irq/132-enp0s29f1:rx-023:04:060
3726243991513,2cyclictest60350irq/132-enp0s29f1:rx-020:19:240
3726243991513,1cyclictest60350irq/132-enp0s29f1:rx-023:25:170
3726243991513,1cyclictest60350irq/132-enp0s29f1:rx-022:35:110
3726243991513,1cyclictest60350irq/132-enp0s29f1:rx-021:35:240
3726243991513,1cyclictest60350irq/132-enp0s29f1:rx-019:35:190
3726243991512,2cyclictest60350irq/132-enp0s29f1:rx-021:11:590
3726243991512,2cyclictest60350irq/132-enp0s29f1:rx-020:54:300
3726243991512,2cyclictest60350irq/132-enp0s29f1:rx-020:10:130
3726243991512,2cyclictest60350irq/132-enp0s29f1:rx-019:32:360
3726243991511,3cyclictest3779216-21sed20:00:170
3726243991511,2cyclictest61150irq/133-enp0s29f1:tx-000:05:200
3726257991412,1cyclictest0-21swapper/322:45:003
3726257991412,1cyclictest0-21swapper/319:59:593
3726257991411,2cyclictest3959812-21kworker/u8:0+flush-259:023:20:203
3726257991411,1cyclictest0-21swapper/323:05:163
3726257991410,3cyclictest3908404-21apt-get22:05:123
3726257991410,3cyclictest3830121-21kworker/u8:0+events_unbound21:20:003
372625399149,3cyclictest252-21systemd-journal20:46:382
372625399147,1cyclictest0-21swapper/223:40:002
3726253991411,2cyclictest3924007-21cpuspeed_turbos22:20:142
3726253991411,2cyclictest3887253-21kworker/u8:1+events_unbound22:00:182
3726253991411,2cyclictest3876610-21unixbench_multi21:30:272
3726253991411,2cyclictest3846088-21cron21:05:002
3726253991411,2cyclictest0-21swapper/222:15:012
3726253991410,3cyclictest3788548-21kworker/u8:0+events_unbound20:10:232
3726245991412,1cyclictest0-21swapper/100:30:011
3726245991411,2cyclictest3918287-21kworker/u8:0+events_unbound22:15:121
3726245991411,2cyclictest3892074-21kworker/u8:4+events_unbound21:55:241
3726245991411,2cyclictest3866294-21kworker/u8:2+flush-259:021:35:191
3726245991411,1cyclictest0-21swapper/100:00:161
3726245991410,3cyclictest3830121-21kworker/u8:0+events_unbound21:10:181
3726243991413,1cyclictest60350irq/132-enp0s29f1:rx-023:36:340
3726243991413,1cyclictest60350irq/132-enp0s29f1:rx-020:40:100
3726243991412,1cyclictest61150irq/133-enp0s29f1:tx-019:10:150
3726243991412,1cyclictest60350irq/132-enp0s29f1:rx-022:06:240
3726243991412,1cyclictest60350irq/132-enp0s29f1:rx-022:03:040
3726243991412,1cyclictest60350irq/132-enp0s29f1:rx-020:35:220
3726243991412,1cyclictest60350irq/132-enp0s29f1:rx-019:27:550
3726243991411,2cyclictest60350irq/132-enp0s29f1:rx-023:50:180
3726243991411,2cyclictest60350irq/132-enp0s29f1:rx-022:30:220
3726243991411,2cyclictest60350irq/132-enp0s29f1:rx-021:45:270
372625799139,3cyclictest3851293-21latency_hist21:10:013
372625799139,3cyclictest3768024-21/usr/sbin/munin19:50:263
372625799139,3cyclictest3768012-21kworker/u8:1+events_unbound20:05:153
372625799139,2cyclictest0-21swapper/321:30:213
372625799138,1cyclictest0-21swapper/321:05:003
3726257991310,2cyclictest3995223-21needreboot23:25:193
3726257991310,2cyclictest0-21swapper/321:55:003
372625399139,3cyclictest3846150-21kworker/u8:1+events_unbound21:25:172
372625399139,3cyclictest3768012-21kworker/u8:1+events_unbound19:50:242
3726253991310,3cyclictest3954390-21kworker/u8:3+events_unbound23:00:222
3726253991310,2cyclictest4032325-21kworker/u8:1+flush-259:000:10:172
3726253991310,2cyclictest3933985-21kworker/u8:1+flush-259:022:55:242
3726253991310,2cyclictest3752594-21irqcore19:35:172
3726253991310,2cyclictest3730932-21kworker/u8:3+events_unbound19:25:222
3726253991310,2cyclictest0-21swapper/220:55:192
3726253991310,2cyclictest0-21swapper/200:10:002
372624599139,3cyclictest4032325-21kworker/u8:1+flush-259:000:19:391
372624599139,3cyclictest4017122-21apt-get23:50:121
3726245991310,3cyclictest3693987-21kworker/u8:0+events_unbound19:15:151
3726245991310,2cyclictest3830121-21kworker/u8:0+events_unbound21:45:141
3726245991310,1cyclictest3939150-21munin-run22:34:591
372625799127,4cyclictest507-21in:imuxsock20:35:003
372625799126,4cyclictest3830121-21kworker/u8:0+flush-259:021:40:203
372625799126,4cyclictest252-21systemd-journal23:55:003
372625799126,4cyclictest252-21systemd-journal19:35:013
3726257991210,1cyclictest3927577-21kthreadcore22:20:203
372625399129,2cyclictest3866273-21unixbench_multi21:20:242
372625399129,2cyclictest3730932-21kworker/u8:3+events_unbound19:20:172
372625399129,2cyclictest0-21swapper/223:10:172
372625399129,2cyclictest0-21swapper/223:10:162
372625399128,3cyclictest3912922-21kworker/u8:2+flush-259:022:26:112
372625399128,3cyclictest3830121-21kworker/u8:0+flush-259:021:10:202
372625399123,1cyclictest39400292irqcore22:35:172
372625399122,7cyclictest3809511-21awk20:30:002
3726253991210,1cyclictest0-21swapper/200:35:012
372624599129,2cyclictest4000987-21unixbench_singl23:30:261
372624599129,2cyclictest3912922-21kworker/u8:2+events_unbound22:52:411
372624599129,2cyclictest3788548-21kworker/u8:0+flush-259:020:35:231
372624599128,3cyclictest3730932-21kworker/u8:3+flush-259:020:50:251
372624599127,4cyclictest3762783-21gpgv19:45:011
372624599121,3cyclictest4052903-21awk00:25:011
372624399129,2cyclictest0-21swapper/023:25:000
3726243991211,1cyclictest60350irq/132-enp0s29f1:rx-020:26:400
3726243991210,1cyclictest60350irq/132-enp0s29f1:rx-019:21:470
3726243991210,1cyclictest60350irq/132-enp0s29f1:rx-019:15:280
372625799119,1cyclictest4006069-21unixbench_multi23:35:253
372625799118,2cyclictest3912922-21kworker/u8:2+flush-259:022:10:203
372625799118,2cyclictest3830121-21kworker/u8:0+flush-259:021:45:163
372625799118,2cyclictest3815005-21cpuspeed_turbos20:35:163
372625799118,2cyclictest0-21swapper/322:25:183
3726257991111,0cyclictest2140888-21kworker/3:0+events_power_efficient20:14:483
372625399119,1cyclictest4047779-21kworker/u8:2+events_unbound00:40:002
372625399119,1cyclictest0-21swapper/221:40:002
372625399118,2cyclictest3881896-21kworker/u8:3+flush-259:022:40:202
372625399118,2cyclictest3846547-21fschecks_count21:05:172
372625399117,3cyclictest3907760-21kworker/u8:0+events_unbound22:05:222
372625399116,4cyclictest4042398-21kworker/u8:3+flush-259:000:15:262
372624599118,2cyclictest3985983-21kworker/u8:3+flush-259:023:40:161
372624599118,2cyclictest3752183-21apt-get19:35:121
372624599118,2cyclictest0-21swapper/119:55:241
372624599117,3cyclictest4047779-21kworker/u8:2+events_unbound00:35:241
372624599117,3cyclictest3954390-21kworker/u8:3+events_unbound23:12:411
372624599117,3cyclictest3954390-21kworker/u8:3+events_unbound23:12:401
372625799107,2cyclictest3730424-21ntp_kernel_pll_19:10:233
372625799107,1cyclictest0-21swapper/322:45:173
372625799106,3cyclictest3830121-21kworker/u8:0+flush-259:020:55:223
372625799104,4cyclictest3959774-21latency_hist22:55:003
372625799100,2cyclictest3746779-21latency_hist19:30:013
372625399100,8cyclictest4016995-21cat23:50:012
372624599100,8cyclictest3772940-21latency_hist19:55:011
37262579996,1cyclictest0-21swapper/320:40:163
37262579984,3cyclictest4001214-21sshd23:32:393
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional