You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-26 - 14:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Fri Jun 26, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3678355994640,4cyclictest3907540-21latency_hist22:50:023
3678350994542,2cyclictest3829395-21kworker/u8:2+flush-259:022:40:002
3678339993833,4cyclictest3709497-21kworker/u8:3+flush-259:019:40:210
3678339993629,4cyclictest3964716-21latency_hist23:45:010
3678339993629,4cyclictest3964716-21latency_hist23:45:000
3678355993428,4cyclictest3949656-21diskstats23:30:123
3678355993330,2cyclictest3640316-21kworker/u8:2+events_unbound19:10:173
3678346993327,4cyclictest3963892-21ntpq23:40:221
3678346993327,4cyclictest3963892-21ntpq23:40:211
3678346993327,4cyclictest3881724-21latency_hist22:25:011
3678339993327,4cyclictest3933308-21kworker/u8:1+flush-259:023:30:230
3678339993327,4cyclictest3793059-21cat21:00:010
3678355993228,3cyclictest3683454-21kworker/u8:3+events_unbound19:20:153
3678355993227,4cyclictest3995711-21kworker/u8:2+flush-259:000:30:003
3678350993228,3cyclictest3876386-21gpgv22:20:012
3678350993227,3cyclictest3756283-21base6420:25:012
3678339993228,2cyclictest3980392-21kworker/u8:0+flush-259:000:00:220
3678355993126,4cyclictest3793154-21cat21:00:013
3678355993124,5cyclictest335557-21systemd-journal20:30:013
3678355993123,6cyclictest3724802-21latency_hist19:55:013
3678350993128,2cyclictest3871017-21kworker/u8:1+flush-259:022:30:002
3678350993125,4cyclictest3829374-21latency_hist21:35:002
3678355993026,3cyclictest3944276-21kworker/u8:2+flush-259:023:50:263
3678355993026,2cyclictest3709285-21kworker/u8:1+flush-259:020:20:163
3678350993026,3cyclictest335557-21systemd-journal21:55:012
3678346993026,3cyclictest3819054-21kworker/u8:0+flush-259:021:30:011
3678346993026,3cyclictest3819054-21kworker/u8:0+flush-259:021:30:001
3678346993024,4cyclictest3912745-21latency_hist22:55:001
3678339993023,5cyclictest3938906-21latency_hist23:20:000
3678355992924,4cyclictest3845502-21apt-get21:50:113
3678355992924,3cyclictest3918517-21kworker/u8:3+events_unbound23:10:203
3678350992925,3cyclictest3849365-21kworker/u8:1+flush-259:022:00:002
3678339992925,3cyclictest3635064-21kworker/u8:0+flush-259:020:15:260
3678355992824,3cyclictest3944276-21kworker/u8:2+events_unbound23:55:123
3678350992824,3cyclictest3964980-21apt-get23:45:132
3678350992823,3cyclictest3683436-21latency_hist19:15:022
3678346992824,3cyclictest3635064-21kworker/u8:0+events_unbound19:45:131
3678355992723,3cyclictest3918517-21kworker/u8:3+events_unbound23:20:173
3678355992722,3cyclictest3918553-21sshd23:00:123
3678355992721,4cyclictest4005440-21kworker/u8:3+flush-259:000:34:043
3678346992723,3cyclictest335557-21systemd-journal22:12:561
3678355992622,3cyclictest3725029-21kworker/u8:4+events_unbound20:04:123
3678355992622,3cyclictest3700141-21kworker/u8:2+flush-259:019:55:163
3678355992622,3cyclictest3683454-21kworker/u8:3+flush-259:019:30:163
3678355992619,4cyclictest3792696-21latency_hist21:00:003
3678350992622,3cyclictest3798599-21apt-get21:05:122
3678350992619,5cyclictest3974992-21latency_hist23:55:012
3678346992622,3cyclictest4005440-21kworker/u8:3+events_unbound00:25:231
3678346992622,3cyclictest3975015-21kworker/u8:1+flush-259:023:55:261
3678339992622,3cyclictest3782030-21gpgv20:50:000
3678339992619,5cyclictest3850372-21latency_hist21:55:000
3678355992521,3cyclictest3683454-21kworker/u8:3+flush-259:019:25:263
3678355992519,4cyclictest3740566-21/usr/sbin/munin20:10:143
3678346992521,3cyclictest3995910-21/usr/sbin/munin00:15:211
3678346992521,3cyclictest3885907-21kworker/u8:0+flush-259:023:04:591
3678346992521,3cyclictest3704351-21apt-get19:35:121
3678346992520,4cyclictest3772083-21kworker/u8:1+events_unbound21:05:011
3678339992521,3cyclictest3640316-21kworker/u8:2+flush-259:019:15:000
3678339992519,4cyclictest3839860-21cp21:45:000
3678355992420,3cyclictest3886849-21kworker/u8:5+flush-259:022:35:253
3678355992420,3cyclictest3849365-21kworker/u8:1+events_unbound21:59:143
3678355992420,3cyclictest3635064-21kworker/u8:0+flush-259:020:05:163
3678355992419,3cyclictest3933308-21kworker/u8:1+flush-259:023:40:163
3678355992419,3cyclictest3933308-21kworker/u8:1+flush-259:023:40:163
3678355992418,4cyclictest3881962-21/usr/sbin/munin22:25:203
3678355992417,5cyclictest3750633-21latency_hist20:20:013
3678355992416,6cyclictest3803497-21latency_hist21:10:003
3678350992420,3cyclictest3829395-21kworker/u8:2+events_unbound22:10:142
3678350992418,4cyclictest3944045-21latency_hist23:25:002
3678350992418,4cyclictest335557-21systemd-journal20:10:012
3678346992421,2cyclictest3860696-21kworker/u8:0+flush-259:022:10:001
3678346992420,3cyclictest3886849-21kworker/u8:5+events_unbound22:45:151
3678346992420,3cyclictest3700141-21kworker/u8:2+flush-259:020:05:261
3678346992419,4cyclictest3781997-21latency_hist20:50:011
3678346992419,3cyclictest3985397-21latency_hist00:05:011
3678346992418,4cyclictest3834508-21latency_hist21:40:011
367834699240,23cyclictest1-21systemd21:55:011
3678339992420,3cyclictest3970094-21apt-get23:50:100
3678355992318,3cyclictest3885907-21kworker/u8:0+flush-259:023:35:183
3678355992318,3cyclictest3849365-21kworker/u8:1+flush-259:022:05:203
3678350992320,2cyclictest3700141-21kworker/u8:2+events_unbound19:55:182
3678350992319,3cyclictest3819054-21kworker/u8:0+flush-259:021:40:002
3678346992320,2cyclictest3829395-21kworker/u8:2+events_unbound21:40:141
3678346992320,2cyclictest3700141-21kworker/u8:2+events_unbound19:55:141
3678346992320,2cyclictest3635064-21kworker/u8:0+events_unbound19:30:221
3678346992319,3cyclictest3886849-21kworker/u8:5+flush-259:023:34:591
3678346992319,3cyclictest3709505-21/usr/sbin/munin19:40:221
3678346992319,3cyclictest3640316-21kworker/u8:2+events_unbound19:10:151
3678346992318,3cyclictest3808654-21latency_hist21:15:011
3678346992317,4cyclictest4005969-21latency_hist00:25:001
3678346992317,4cyclictest3927753-21ntpq23:05:221
3678339992319,3cyclictest3819054-21kworker/u8:0+flush-259:021:45:260
3678339992317,4cyclictest335557-21systemd-journal20:06:120
3678355992218,3cyclictest3808884-21kworker/u8:4+events_unbound21:20:123
3678355992218,3cyclictest3750372-21kworker/u8:3+events_unbound20:41:123
3678350992219,2cyclictest3886849-21kworker/u8:5+events_unbound22:40:202
3678350992219,2cyclictest3700141-21kworker/u8:2+events_unbound20:35:012
3678350992218,3cyclictest335557-21systemd-journal22:58:112
3678350992217,4cyclictest3635064-21kworker/u8:0+flush-259:019:22:432
3678350992217,3cyclictest3839670-21latency_hist21:45:002
3678346992219,2cyclictest3808884-21kworker/u8:4+events_unbound21:20:161
3678346992219,2cyclictest3709285-21kworker/u8:1+flush-259:020:10:211
3678346992219,2cyclictest3683454-21kworker/u8:3+flush-259:019:25:001
3678346992218,3cyclictest4005440-21kworker/u8:3+flush-259:000:35:221
3678346992218,3cyclictest3886849-21kworker/u8:5+flush-259:022:40:111
3678346992217,3cyclictest335557-21systemd-journal22:00:011
3678346992216,4cyclictest3812963-21ntpq21:15:221
3678339992217,4cyclictest3979299-21ntpq23:55:220
3678339992216,5cyclictest3755764-21apt-get20:25:000
3678355992117,3cyclictest3791872-21kworker/u8:3+flush-259:021:10:233
3678355992116,4cyclictest3829395-21kworker/u8:2+events_unbound22:57:123
3678355992116,4cyclictest3750372-21kworker/u8:3+events_unbound20:34:123
3678355992116,4cyclictest3683454-21kworker/u8:3+flush-259:019:15:243
3678355992116,3cyclictest4005440-21kworker/u8:3+events_unbound00:35:183
3678355992115,4cyclictest335557-21systemd-journal23:06:103
3678350992117,3cyclictest3944276-21kworker/u8:2+events_unbound23:40:162
3678350992117,3cyclictest3944276-21kworker/u8:2+events_unbound23:40:152
3678350992117,3cyclictest3791872-21kworker/u8:3+events_unbound21:10:212
3678350992116,4cyclictest3902805-21diskstats22:45:132
3678346992116,4cyclictest3886849-21kworker/u8:5+flush-259:023:50:201
3678346992115,5cyclictest3943209-21ntpq23:20:221
3678339992118,2cyclictest3944276-21kworker/u8:2+events_unbound23:45:220
3678339992117,3cyclictest3709285-21kworker/u8:1+flush-259:020:25:150
3678355992017,2cyclictest3944276-21kworker/u8:2+events_unbound23:25:203
3678355992016,3cyclictest3975015-21kworker/u8:1+flush-259:000:00:313
3678355992016,3cyclictest3975015-21kworker/u8:1+events_unbound00:15:243
3678355992015,4cyclictest4001066-21apt-get00:20:133
3678350992017,2cyclictest0-21swapper/220:25:012
3678350992015,4cyclictest335557-21systemd-journal23:04:112
3678346992017,2cyclictest4005440-21kworker/u8:3+flush-259:000:30:261
3678346992013,5cyclictest335557-21systemd-journal21:35:001
3678339992015,4cyclictest3933751-21latency_hist23:15:000
3678339992015,4cyclictest3761559-21kworker/u8:0+flush-259:020:50:200
3678339992014,4cyclictest335557-21systemd-journal00:19:110
3678355991916,2cyclictest3700141-21kworker/u8:2+events_unbound19:45:253
3678355991915,3cyclictest3881709-21idleruntime-cro22:25:013
3678355991915,3cyclictest3819054-21kworker/u8:0+events_unbound21:42:363
3678355991915,3cyclictest3808884-21kworker/u8:4+flush-259:022:10:213
3678355991914,4cyclictest3944276-21kworker/u8:2+events_unbound00:05:173
3678355991914,4cyclictest3871017-21kworker/u8:1+events_unbound22:17:583
3678350991915,3cyclictest3975015-21kworker/u8:1+flush-259:023:59:592
3678350991915,3cyclictest3912737-21apt-get22:54:592
3678350991915,3cyclictest3791872-21kworker/u8:3+events_unbound21:15:152
3678350991915,3cyclictest3750372-21kworker/u8:3+flush-259:020:44:592
3678350991915,3cyclictest3700141-21kworker/u8:2+events_unbound19:50:162
3678346991916,2cyclictest3873564-21kthreadcore22:15:201
3678346991916,2cyclictest3635064-21kworker/u8:0+events_unbound19:10:001
3678346991914,4cyclictest3745725-21/usr/sbin/munin20:15:141
3678346991913,4cyclictest3776834-21latency_hist20:45:001
3678339991914,4cyclictest3818149-21ntpq21:20:210
3678355991815,2cyclictest3808884-21kworker/u8:4+events_unbound21:15:273
3678355991815,2cyclictest0-21swapper/320:50:003
3678355991814,3cyclictest3944276-21kworker/u8:2+flush-259:023:49:593
3678355991813,4cyclictest3990279-21kworker/u8:3+events_unbound00:13:123
3678350991815,2cyclictest3772083-21kworker/u8:1+flush-259:020:55:202
3678350991815,2cyclictest3635064-21kworker/u8:0+flush-259:019:45:102
3678350991814,3cyclictest4006225-21apt-get00:25:122
3678346991814,3cyclictest3918517-21kworker/u8:3+flush-259:023:15:161
3678346991814,3cyclictest3885907-21kworker/u8:0+events_unbound22:30:121
3678339991814,3cyclictest3808884-21kworker/u8:4+events_unbound22:30:230
3678339991814,2cyclictest60350irq/132-enp0s29f1:rx-020:00:130
3678339991813,3cyclictest335557-21systemd-journal20:56:130
3678355991714,2cyclictest3761559-21kworker/u8:0+flush-259:020:50:193
3678355991713,3cyclictest3886849-21kworker/u8:5+events_unbound22:40:183
3678355991712,3cyclictest3673143-21kworker/u8:4+flush-259:019:35:243
3678355991711,4cyclictest3840224-21cp21:45:013
367835599170,15cyclictest2109693-21snmpd23:18:123
367835099177,8cyclictest3865872-21idleruntime-cro22:10:012
3678350991714,2cyclictest3819054-21kworker/u8:0+flush-259:021:50:002
3678350991713,3cyclictest3944276-21kworker/u8:2+flush-259:000:01:012
3678350991713,3cyclictest3791872-21kworker/u8:3+events_unbound21:00:142
3678350991713,3cyclictest3750372-21kworker/u8:3+flush-259:020:49:592
3678350991713,3cyclictest3635064-21kworker/u8:0+events_unbound19:30:122
3678350991712,3cyclictest3933308-21kworker/u8:1+flush-259:023:30:142
3678350991711,4cyclictest3770770-21ntpq20:35:212
3678346991714,2cyclictest3918517-21kworker/u8:3+flush-259:023:10:251
3678346991713,3cyclictest3886849-21kworker/u8:5+events_unbound22:35:221
3678346991713,3cyclictest3772083-21kworker/u8:1+events_unbound21:00:211
3678346991713,3cyclictest3761559-21kworker/u8:0+flush-259:020:50:151
3678339991714,2cyclictest60350irq/132-enp0s29f1:rx-022:42:560
3678339991714,2cyclictest60350irq/132-enp0s29f1:rx-022:29:530
3678339991714,2cyclictest3990503-21cron00:09:590
3678339991712,4cyclictest335557-21systemd-journal19:56:120
3678355991612,3cyclictest3829395-21kworker/u8:2+events_unbound22:50:253
3678355991612,3cyclictest3709285-21kworker/u8:1+events_unbound19:43:153
3678350991612,3cyclictest4005440-21kworker/u8:3+flush-259:000:31:122
3678350991610,4cyclictest3823323-21ntpq21:25:232
3678350991610,4cyclictest3823323-21ntpq21:25:222
3678346991613,2cyclictest3885907-21kworker/u8:0+events_unbound23:35:011
3678346991612,3cyclictest3683698-21apt-get19:15:111
3678346991611,4cyclictest3995667-21latency_hist00:15:011
3678339991614,1cyclictest60350irq/132-enp0s29f1:rx-021:05:170
3678339991613,2cyclictest60350irq/132-enp0s29f1:rx-022:50:230
3678339991613,2cyclictest60350irq/132-enp0s29f1:rx-022:38:540
3678339991613,2cyclictest60350irq/132-enp0s29f1:rx-020:35:120
3678339991613,2cyclictest60350irq/132-enp0s29f1:rx-020:10:150
3678339991613,2cyclictest60350irq/132-enp0s29f1:rx-019:47:360
3678339991613,2cyclictest60350irq/132-enp0s29f1:rx-000:39:340
3678339991613,2cyclictest60350irq/132-enp0s29f1:rx-000:34:170
3678339991612,3cyclictest3849365-21kworker/u8:1+flush-259:022:00:260
3678339991610,4cyclictest3918281-21latency_hist23:00:010
3678355991511,2cyclictest0-21swapper/322:35:003
3678350991512,2cyclictest4005440-21kworker/u8:3+flush-259:000:35:162
3678350991512,2cyclictest3944276-21kworker/u8:2+events_unbound23:25:242
3678350991511,3cyclictest3885907-21kworker/u8:0+events_unbound23:35:142
3678350991511,3cyclictest3709497-21kworker/u8:3+flush-259:019:40:262
367835099150,9cyclictest3881705-21cron22:25:012
3678346991512,2cyclictest3944276-21kworker/u8:2+events_unbound23:25:181
3678346991512,2cyclictest3755756-21idleruntime-cro20:25:001
3678346991511,3cyclictest3944276-21kworker/u8:2+flush-259:000:05:151
3678339991513,1cyclictest60350irq/132-enp0s29f1:rx-023:25:120
3678339991513,1cyclictest60350irq/132-enp0s29f1:rx-023:05:470
3678339991513,1cyclictest60350irq/132-enp0s29f1:rx-022:18:510
3678339991513,1cyclictest60350irq/132-enp0s29f1:rx-021:55:230
3678339991513,1cyclictest60350irq/132-enp0s29f1:rx-021:35:160
3678339991513,1cyclictest60350irq/132-enp0s29f1:rx-021:30:120
3678339991513,1cyclictest60350irq/132-enp0s29f1:rx-019:35:150
3678339991513,1cyclictest60350irq/132-enp0s29f1:rx-000:25:180
3678339991513,1cyclictest60350irq/132-enp0s29f1:rx-000:14:040
3678339991512,2cyclictest60350irq/132-enp0s29f1:rx-023:40:000
3678339991512,2cyclictest60350irq/132-enp0s29f1:rx-022:11:200
3678339991512,2cyclictest60350irq/132-enp0s29f1:rx-022:05:240
3678339991512,2cyclictest60350irq/132-enp0s29f1:rx-021:11:050
3678339991512,2cyclictest60350irq/132-enp0s29f1:rx-019:27:470
367835099148,5cyclictest3989615-21ntpq00:05:222
3678350991410,3cyclictest3683767-21cpuspeed_turbos19:15:132
367833999149,4cyclictest3688859-21/usr/sbin/munin19:20:130
3678339991412,1cyclictest60350irq/132-enp0s29f1:rx-023:00:180
3678339991411,2cyclictest60350irq/132-enp0s29f1:rx-022:45:170
3678339991411,2cyclictest60350irq/132-enp0s29f1:rx-022:20:130
3678339991411,2cyclictest60350irq/132-enp0s29f1:rx-021:26:290
3678339991411,2cyclictest60350irq/132-enp0s29f1:rx-021:26:280
3678339991411,2cyclictest60350irq/132-enp0s29f1:rx-019:50:230
3678339991411,2cyclictest60350irq/132-enp0s29f1:rx-019:15:190
3678339991411,2cyclictest60350irq/132-enp0s29f1:rx-000:20:240
367835599139,3cyclictest3808884-21kworker/u8:4+events_unbound21:33:123
3678355991310,2cyclictest3830051-21irqcore21:35:173
367835099139,3cyclictest3635064-21kworker/u8:0+flush-259:019:25:242
3678350991311,1cyclictest0-21swapper/220:54:592
3678350991310,2cyclictest3996955-21kthreadcore00:15:182
3678350991310,2cyclictest3814308-21fschecks_count21:20:142
367834699139,2cyclictest3808884-21kworker/u8:4+flush-259:022:25:261
367834699130,5cyclictest3768869-21grep20:35:181
367835099129,1cyclictest0-21swapper/220:15:212
367834699129,2cyclictest3700141-21kworker/u8:2+events_unbound20:00:121
367834699128,3cyclictest3819054-21kworker/u8:0+flush-259:021:45:191
367833999128,3cyclictest60350irq/132-enp0s29f1:rx-021:15:210
367835599117,3cyclictest3819467-21fschecks_count21:25:153
367835599117,3cyclictest3819467-21fschecks_count21:25:143
367835099117,3cyclictest3886849-21kworker/u8:5+events_unbound23:10:172
367835099117,2cyclictest4004995-21needreboot00:20:202
367834699119,1cyclictest0-21swapper/121:00:001
367834699119,1cyclictest0-21swapper/119:30:001
367834699118,2cyclictest3762020-21tr20:30:171
367834699116,4cyclictest3756453-21apt-get20:25:111
367835099107,2cyclictest0-21swapper/222:35:012
367835099106,3cyclictest3829395-21kworker/u8:2+events_unbound22:00:132
367834699106,3cyclictest3913276-21cat22:55:011
367834699106,2cyclictest3723950-21ntpq19:50:221
367834699105,4cyclictest3850610-21/usr/sbin/munin21:55:191
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional