You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-26 - 01:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Sat Jul 25, 2026 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3668596994337,4cyclictest3699750-21latency_hist07:40:011
3668605994035,3cyclictest3855741-21apt-get10:10:133
3668605994033,4cyclictest509-21rs:main3
3668599993935,3cyclictest3798403-21base6409:15:012
3668592993932,5cyclictest3840368-21diskstats09:55:150
3668605993832,4cyclictest335557-21systemd-journal10:45:443
3668599993834,2cyclictest3974682-21gpgv12:05:002
3668596993831,5cyclictest3964339-21latency_hist11:55:001
3668605993630,4cyclictest3679116-21latency_hist07:20:003
3668599993531,3cyclictest3871224-21/usr/sbin/munin10:25:162
3668596993529,4cyclictest3797796-21apt-get09:15:001
3668599993431,2cyclictest3751691-21apt-get08:30:142
3668596993428,4cyclictest3932076-21ntpq11:20:221
3668592993330,2cyclictest3860859-21kworker/u8:0+flush-259:011:05:000
3668592993330,2cyclictest3630358-21kworker/u8:1+flush-259:007:19:590
3668605993226,4cyclictest3735941-21latency_hist08:15:013
3668599993228,3cyclictest3917498-21latency_hist11:10:012
3668599993127,3cyclictest3917537-21kworker/u8:2+flush-259:011:45:162
3668592993129,2cyclictest3798459-21kworker/u8:3+events_unbound09:15:150
3668592993128,2cyclictest3614655-21kworker/u8:0+events_unbound07:23:300
3668592993126,3cyclictest3668668-21cat07:10:010
3668592993026,3cyclictest3777458-21apt-get08:55:120
3668599992926,2cyclictest3819383-21kworker/u8:3+flush-259:009:45:182
3668596992825,2cyclictest3860859-21kworker/u8:0+flush-259:010:20:251
3668596992825,2cyclictest3669345-21kworker/u8:2+flush-259:007:15:011
3668596992823,3cyclictest3819383-21kworker/u8:3+flush-259:009:55:191
3668592992825,3cyclictest3860859-21kworker/u8:0+events_unbound10:44:290
3668592992824,3cyclictest3824778-21apt-get09:40:110
3668592992822,4cyclictest335557-21systemd-journal07:35:010
3668605992722,4cyclictest3891896-21kworker/u8:4+flush-259:011:00:003
3668599992723,3cyclictest335557-21systemd-journal07:29:502
3668592992724,2cyclictest3746276-21kworker/u8:1+events_unbound09:25:150
3668592992722,4cyclictest3958311-21ntpq11:45:210
3668592992722,4cyclictest3704015-21ntpq07:40:210
366859299271,24cyclictest509-21rs:main0
3668605992620,4cyclictest3730795-21latency_hist08:10:013
3668605992619,5cyclictest4011214-21latency_hist12:40:013
3668599992623,2cyclictest3746276-21kworker/u8:1+events_unbound08:30:012
3668599992622,3cyclictest3917537-21kworker/u8:2+events_unbound11:41:282
3668599992622,3cyclictest3787518-21kworker/u8:0+events_unbound09:30:242
3668596992622,3cyclictest3746276-21kworker/u8:1+events_unbound09:00:301
3668596992622,2cyclictest3725890-21apt-get08:05:151
3668592992620,4cyclictest3885695-21ntpq10:35:250
3668605992521,3cyclictest3881568-21kworker/u8:3+flush-259:010:45:013
3668605992519,4cyclictest3942390-21ntpq11:30:233
3668599992522,2cyclictest3808586-21kworker/u8:2+events_unbound10:30:222
3668599992521,3cyclictest3725893-21kworker/u8:0+events_unbound08:19:282
3668596992522,3cyclictest3669345-21kworker/u8:2+events_unbound08:25:001
3668596992520,4cyclictest3922912-21apt-get11:15:131
3668592992519,4cyclictest3938106-21latency_hist11:30:000
3668605992419,3cyclictest335557-21systemd-journal08:35:003
3668605992418,4cyclictest335557-21systemd-journal08:29:273
3668599992421,2cyclictest3630358-21kworker/u8:1+flush-259:007:20:212
3668599992418,4cyclictest3673966-21latency_hist07:15:012
3668596992421,2cyclictest3630358-21kworker/u8:1+events_unbound07:30:151
3668596992420,3cyclictest3746276-21kworker/u8:1+flush-259:008:30:211
3668592992421,2cyclictest3974603-21kworker/u8:2+events_unbound12:20:000
3668605992320,2cyclictest3829691-21kworker/u8:1+flush-259:010:05:003
3668605992317,4cyclictest3834806-21latency_hist09:50:013
3668605992317,4cyclictest3797801-21latency_hist09:15:013
3668599992316,5cyclictest3719537-21ntpq07:55:212
3668596992319,3cyclictest3943313-21gpgv11:35:001
3668596992319,3cyclictest3829691-21kworker/u8:1+flush-259:010:15:261
3668592992319,3cyclictest3917537-21kworker/u8:2+events_unbound11:50:210
3668592992319,3cyclictest3819383-21kworker/u8:3+events_unbound10:10:250
3668592992319,3cyclictest3766923-21kworker/u8:2+events_unbound08:49:190
3668592992318,3cyclictest335557-21systemd-journal12:33:290
3668592992317,4cyclictest3819364-21latency_hist09:35:010
3668605992216,4cyclictest3980276-21latency_hist12:10:003
3668599992219,3cyclictest3990207-21kworker/u8:0+events_unbound12:30:292
3668599992219,2cyclictest3860859-21kworker/u8:0+events_unbound10:43:292
3668599992218,3cyclictest3932978-21kworker/u8:1+events_unbound11:30:292
3668596992220,2cyclictest3829691-21kworker/u8:1+events_unbound10:25:241
3668596992219,2cyclictest3630358-21kworker/u8:1+flush-259:007:20:261
3668596992218,3cyclictest3974603-21kworker/u8:2+flush-259:012:05:201
3668592992219,2cyclictest3819383-21kworker/u8:3+events_unbound09:45:190
3668592992219,2cyclictest3766923-21kworker/u8:2+flush-259:008:50:270
3668592992218,3cyclictest3881568-21kworker/u8:3+events_unbound10:49:290
3668592992218,3cyclictest3725893-21kworker/u8:0+events_unbound08:20:280
3668592992217,4cyclictest3669057-21apt-get07:10:110
3668592992217,4cyclictest335557-21systemd-journal09:40:000
3668592992216,4cyclictest3864897-21ntpq10:15:210
3668605992118,2cyclictest3787518-21kworker/u8:0+flush-259:009:20:193
3668599992118,2cyclictest3917537-21kworker/u8:2+events_unbound11:23:312
3668599992118,2cyclictest3891896-21kworker/u8:4+flush-259:010:50:232
3668599992117,3cyclictest3819383-21kworker/u8:3+flush-259:009:40:222
3668599992117,3cyclictest3819383-21kworker/u8:3+events_unbound10:10:192
3668599992117,3cyclictest3669345-21kworker/u8:2+events_unbound07:15:232
3668599992117,3cyclictest3630358-21kworker/u8:1+events_unbound07:37:282
3668596992120,1cyclictest3990207-21kworker/u8:0+flush-259:012:20:281
3668596992117,3cyclictest3798459-21kworker/u8:3+flush-259:009:15:141
3668596992116,4cyclictest3756818-21/usr/sbin/munin08:35:151
3668592992118,2cyclictest500-21systemd-logind12:40:000
3668592992118,2cyclictest3669345-21kworker/u8:2+events_unbound07:45:240
3668592992118,1cyclictest0-21swapper/009:04:590
3668592992117,3cyclictest3932978-21kworker/u8:1+events_unbound11:55:230
3668605992017,2cyclictest3829652-21cron09:45:003
3668605992016,3cyclictest3829691-21kworker/u8:1+flush-259:010:35:003
3668605992014,4cyclictest3746276-21kworker/u8:1+events_unbound09:00:203
3668599992016,3cyclictest3819383-21kworker/u8:3+events_unbound10:05:242
3668599992016,3cyclictest3819383-21kworker/u8:3+events_unbound09:50:172
3668599992016,3cyclictest3725893-21kworker/u8:0+events_unbound08:20:232
3668596992014,4cyclictest335557-21systemd-journal10:56:281
3668596992014,4cyclictest335557-21systemd-journal07:42:271
3668592992017,2cyclictest3819383-21kworker/u8:3+events_unbound10:06:290
3668592992016,3cyclictest3860859-21kworker/u8:0+flush-259:010:55:260
3668592992016,3cyclictest3808586-21kworker/u8:2+flush-259:010:30:000
3668592992014,5cyclictest3860859-21kworker/u8:0+flush-259:011:05:260
3668605991916,2cyclictest3669345-21kworker/u8:2+events_unbound08:00:013
3668605991914,4cyclictest3907183-21kworker/u8:1+flush-259:011:10:003
3668605991912,5cyclictest3812872-21ntpq09:25:213
3668599991916,2cyclictest3917537-21kworker/u8:2+flush-259:011:25:162
3668599991915,3cyclictest3813792-21gpgv09:30:002
3668599991915,3cyclictest3808586-21kworker/u8:2+events_unbound09:55:362
3668599991915,3cyclictest3630358-21kworker/u8:1+events_unbound07:44:302
3668596991915,3cyclictest3881568-21kworker/u8:3+events_unbound10:35:291
3668596991915,3cyclictest3860859-21kworker/u8:0+flush-259:011:05:211
3668596991915,3cyclictest3819383-21kworker/u8:3+flush-259:010:05:201
3668596991915,3cyclictest3808586-21kworker/u8:2+flush-259:010:00:261
3668596991915,3cyclictest3787518-21kworker/u8:0+events_unbound09:35:181
3668596991912,5cyclictest3974843-21cp12:05:001
3668592991916,2cyclictest3927554-21kworker/u8:3+events_unbound11:22:290
3668592991916,2cyclictest3819383-21kworker/u8:3+flush-259:009:50:250
3668605991816,1cyclictest0-21swapper/312:00:003
3668605991814,3cyclictest3829691-21kworker/u8:1+flush-259:010:10:003
3668605991812,4cyclictest3984525-21ntpq12:10:233
3668599991815,2cyclictest3746276-21kworker/u8:1+flush-259:008:55:212
3668599991814,3cyclictest3746276-21kworker/u8:1+flush-259:008:45:102
3668599991814,3cyclictest3725893-21kworker/u8:0+flush-259:008:10:052
3668599991814,3cyclictest3669345-21kworker/u8:2+events_unbound08:09:302
3668596991815,2cyclictest3974603-21kworker/u8:2+events_unbound12:25:191
3668596991814,3cyclictest3860859-21kworker/u8:0+events_unbound10:50:291
3668596991814,3cyclictest3808586-21kworker/u8:2+events_unbound10:30:181
3668592991816,2cyclictest3974603-21kworker/u8:2+events_unbound12:05:260
3668592991816,2cyclictest3964390-21kworker/u8:3+flush-259:012:00:290
3668592991815,2cyclictest3860859-21kworker/u8:0+flush-259:010:20:200
3668605991713,3cyclictest3787518-21kworker/u8:0+flush-259:009:35:173
3668605991713,3cyclictest3630358-21kworker/u8:1+flush-259:007:20:193
3668605991711,4cyclictest3819383-21kworker/u8:3+flush-259:009:50:193
3668599991714,2cyclictest3746276-21kworker/u8:1+flush-259:009:20:172
3668599991713,3cyclictest3974603-21kworker/u8:2+events_unbound12:15:582
3668599991713,3cyclictest3927554-21kworker/u8:3+events_unbound11:18:292
3668599991713,3cyclictest3917537-21kworker/u8:2+events_unbound11:51:292
3668599991713,3cyclictest3709373-21kworker/u8:3+events_unbound07:45:302
366859999170,8cyclictest3824526-21grep09:40:002
3668596991713,3cyclictest3725893-21kworker/u8:0+events_unbound08:25:191
3668596991712,3cyclictest335557-21systemd-journal09:30:151
3668592991714,3cyclictest3630358-21kworker/u8:1+events_unbound07:35:230
3668592991714,2cyclictest3990207-21kworker/u8:0+flush-259:012:20:200
3668592991714,2cyclictest3808586-21kworker/u8:2+flush-259:010:00:210
3668605991613,2cyclictest3709373-21kworker/u8:3+flush-259:007:50:213
3668605991612,3cyclictest3990207-21kworker/u8:0+flush-259:012:30:183
3668599991613,2cyclictest3974603-21kworker/u8:2+events_unbound12:25:252
3668599991613,2cyclictest3860859-21kworker/u8:0+events_unbound11:01:292
3668599991612,3cyclictest3808586-21kworker/u8:2+flush-259:010:00:212
3668599991612,3cyclictest3746276-21kworker/u8:1+events_unbound09:05:122
3668596991612,3cyclictest3688417-21kworker/u8:3+events_unbound07:25:241
3668592991614,2cyclictest3746276-21kworker/u8:1+events_unbound08:45:000
3668592991614,1cyclictest60350irq/132-enp0s29f1:rx-008:10:260
3668592991613,2cyclictest3669345-21kworker/u8:2+events_unbound08:00:130
3668592991613,2cyclictest335557-21systemd-journal10:35:000
3668605991512,2cyclictest0-21swapper/312:20:003
3668599991513,1cyclictest0-21swapper/208:45:002
3668599991513,1cyclictest0-21swapper/208:40:002
3668599991512,3cyclictest3860859-21kworker/u8:0+flush-259:010:20:212
3668599991511,3cyclictest3746276-21kworker/u8:1+flush-259:009:05:002
3668599991511,3cyclictest3746276-21kworker/u8:1+events_unbound09:13:302
3668599991511,3cyclictest3669345-21kworker/u8:2+events_unbound08:00:092
3668596991512,2cyclictest3881568-21kworker/u8:3+events_unbound11:00:251
3668596991512,2cyclictest3766923-21kworker/u8:2+flush-259:008:50:291
3668596991512,2cyclictest3725893-21kworker/u8:0+flush-259:008:15:261
3668596991510,3cyclictest4006311-21/usr/sbin/munin12:35:181
366859299159,4cyclictest3740289-21ntpq08:15:230
3668592991514,1cyclictest60350irq/132-enp0s29f1:rx-012:14:180
3668592991514,1cyclictest60350irq/132-enp0s29f1:rx-007:55:240
3668592991513,2cyclictest3917537-21kworker/u8:2+events_unbound11:43:290
3668592991513,1cyclictest60350irq/132-enp0s29f1:rx-008:25:170
3668592991513,1cyclictest60350irq/132-enp0s29f1:rx-008:05:280
3668592991512,3cyclictest3860859-21kworker/u8:0+events_unbound11:15:530
3668592991512,2cyclictest60350irq/132-enp0s29f1:rx-011:35:010
3668592991512,2cyclictest60350irq/132-enp0s29f1:rx-011:35:010
366860599148,4cyclictest3995763-21latency_hist12:25:003
366860599148,4cyclictest335557-21systemd-journal07:45:013
3668605991411,2cyclictest3829691-21kworker/u8:1+flush-259:010:15:213
3668605991411,2cyclictest3688458-21grep07:25:213
3668605991410,3cyclictest3808586-21kworker/u8:2+flush-259:009:55:243
366859999148,4cyclictest335557-21systemd-journal10:15:212
3668599991411,2cyclictest3974603-21kworker/u8:2+flush-259:012:10:272
3668599991411,2cyclictest3881568-21kworker/u8:3+events_unbound10:35:152
3668599991411,2cyclictest3766923-21kworker/u8:2+flush-259:008:50:262
3668599991411,2cyclictest3669345-21kworker/u8:2+flush-259:007:30:152
3668599991410,3cyclictest3860859-21kworker/u8:0+events_unbound11:14:292
3668596991410,3cyclictest3746276-21kworker/u8:1+events_unbound08:55:231
3668596991410,3cyclictest3669345-21kworker/u8:2+flush-259:007:15:281
3668592991413,1cyclictest60350irq/132-enp0s29f1:rx-011:32:580
3668592991412,2cyclictest3766923-21kworker/u8:2+events_unbound09:05:220
3668592991412,2cyclictest3746276-21kworker/u8:1+flush-259:009:10:250
366860599138,4cyclictest335557-21systemd-journal08:57:283
366860599138,3cyclictest335557-21systemd-journal07:10:293
366860599131,9cyclictest3699750-21latency_hist07:40:013
3668605991310,1cyclictest0-21swapper/310:40:003
366859999139,3cyclictest3860859-21kworker/u8:0+flush-259:010:55:512
366859999138,4cyclictest3975508-21diskstats12:05:162
366859699139,3cyclictest509-21rs:main1
366859699139,3cyclictest509-21rs:main1
366859699138,3cyclictest3891674-21latency_hist10:45:011
3668596991311,1cyclictest0-21swapper/111:15:001
366859299139,3cyclictest3725893-21kworker/u8:0+flush-259:008:35:160
3668592991312,1cyclictest60350irq/132-enp0s29f1:rx-007:50:240
3668592991311,2cyclictest60350irq/132-enp0s29f1:rx-009:20:180
3668592991311,2cyclictest3860859-21kworker/u8:0+events_unbound10:55:000
3668592991310,2cyclictest60350irq/132-enp0s29f1:rx-011:10:240
366860599129,2cyclictest3787518-21kworker/u8:0+flush-259:009:30:263
366860599129,2cyclictest3709125-21cut07:45:213
366860599128,3cyclictest3917537-21kworker/u8:2+flush-259:011:45:143
366860599128,2cyclictest0-21swapper/312:05:003
366860599127,3cyclictest3927798-21sh11:20:003
366860599126,4cyclictest3937811-21sshd11:26:283
366860599121,2cyclictest3772065-21awk08:50:003
3668605991210,1cyclictest3742304-21irqcore08:20:183
3668605991210,1cyclictest0-21swapper/307:30:133
366859999129,2cyclictest3669345-21kworker/u8:2+events_unbound07:50:242
366859699129,2cyclictest3985771-21cpuspeed_turbos12:15:131
366859699129,2cyclictest3932978-21kworker/u8:1+events_unbound11:55:151
366859699129,2cyclictest3808586-21kworker/u8:2+events_unbound09:40:171
366859699129,1cyclictest0-21swapper/111:45:121
3668596991210,1cyclictest0-21swapper/109:55:001
366859299128,3cyclictest507-21in:imuxsock08:30:200
3668592991210,1cyclictest60350irq/132-enp0s29f1:rx-012:25:200
366860599118,2cyclictest3949241-21cpuspeed_turbos11:40:153
366860599118,2cyclictest3946697-21kthreadcore11:35:203
366860599118,2cyclictest3946697-21kthreadcore11:35:193
366860599118,2cyclictest0-21swapper/309:15:263
366860599117,3cyclictest3860859-21kworker/u8:0+flush-259:011:15:003
366860599116,3cyclictest3776298-21ntpq08:50:233
366860599111,2cyclictest3870977-21awk10:25:003
366859999118,2cyclictest3892335-21irqcore10:45:182
366859699118,2cyclictest3766861-21cron08:45:001
366859699116,4cyclictest3974603-21kworker/u8:2+events_unbound12:10:181
366860599107,2cyclictest3959523-21cpuspeed_turbos11:50:153
366860599106,3cyclictest3761609-21sshd08:38:293
366859699106,3cyclictest3819383-21kworker/u8:3+flush-259:010:10:211
366859699106,2cyclictest3709221-21ntpq07:45:231
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional