You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-15 - 01:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Thu May 14, 2026 12:44:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3797297995147,3cyclictest3947916-21kworker/u8:1+flush-259:009:45:252
3797294994643,2cyclictest3984656-21kworker/u8:2+flush-259:010:15:271
3797297994237,3cyclictest4036358-21kworker/u8:3+flush-259:012:30:192
3797297994237,3cyclictest4036358-21kworker/u8:3+flush-259:012:30:192
3797297993935,3cyclictest3963735-21kworker/u8:3+events_unbound09:55:012
3797297993834,3cyclictest3958395-21kworker/u8:0+events_unbound10:15:282
3797290993832,4cyclictest509-21rs:main0
3797290993831,5cyclictest4099441-21diskstats12:00:140
3797297993733,3cyclictest4046995-21kworker/u8:2+flush-259:012:25:232
3797297993733,3cyclictest4005327-21kworker/u8:1+flush-259:010:45:002
3797297993632,3cyclictest3921212-21kworker/u8:2+flush-259:009:10:202
3797290993632,3cyclictest3797284-21kworker/u8:3+flush-259:007:25:160
3797297993532,2cyclictest3797284-21kworker/u8:3+events_unbound07:20:212
3797294993530,4cyclictest252-21systemd-journal09:50:001
3797294993529,4cyclictest4021368-21cat10:45:011
3797290993429,4cyclictest3797284-21kworker/u8:3+flush-259:008:35:170
3797290993429,3cyclictest3797284-21kworker/u8:3+flush-259:007:20:150
3797297993328,3cyclictest3890277-21kworker/u8:1+flush-259:008:50:212
3797290993326,5cyclictest4020771-21latency_hist10:45:010
3797297993229,2cyclictest3848783-21kworker/u8:2+events_unbound08:43:452
3797297993228,3cyclictest4109347-21kworker/u8:1+events_unbound12:18:452
3797297993128,2cyclictest628206-21kworker/u8:1+events_unbound07:15:002
3797297993127,3cyclictest3974226-21apt-get10:00:102
3797297993127,3cyclictest3963735-21kworker/u8:3+events_unbound10:09:462
3797297993127,3cyclictest3963735-21kworker/u8:3+events_unbound10:09:452
3797297993127,3cyclictest3890277-21kworker/u8:1+events_unbound09:08:452
3797297993125,4cyclictest3833750-21diskstats07:45:132
3797290993124,5cyclictest4071935-21ntpq11:30:200
3797299993026,3cyclictest3848783-21kworker/u8:2+flush-259:008:45:143
3797299993026,3cyclictest3797284-21kworker/u8:3+flush-259:008:05:253
3797297993026,3cyclictest3895919-21kworker/u8:3+events_unbound08:55:182
3797290993026,2cyclictest4109347-21kworker/u8:1+flush-259:012:25:000
3797290993024,4cyclictest3921656-21apt-get09:10:020
3797297992927,2cyclictest4036358-21kworker/u8:3+flush-259:011:35:232
3797297992925,3cyclictest3818121-21kworker/u8:0+events_unbound08:00:192
3797297992923,4cyclictest4099253-21/usr/sbin/munin12:00:152
3797290992925,3cyclictest4046995-21kworker/u8:2+flush-259:011:55:170
3797290992922,5cyclictest3916545-21latency_hist09:05:020
3797290992921,6cyclictest3858661-21ntpq08:05:220
3797290992921,5cyclictest4135197-21latency_hist12:35:010
3797290992921,5cyclictest4135197-21latency_hist12:35:010
3797297992825,2cyclictest3890277-21kworker/u8:1+events_unbound09:15:152
3797297992824,3cyclictest3818121-21kworker/u8:0+events_unbound07:38:462
3797297992824,3cyclictest3818121-21kworker/u8:0+events_unbound07:38:452
3797294992824,3cyclictest3995241-21apt-get10:20:121
3797294992823,3cyclictest252-21systemd-journal08:02:431
3797299992722,3cyclictest3812726-21latency_hist07:25:003
3797297992724,2cyclictest3797284-21kworker/u8:3+events_unbound07:53:442
3797299992622,3cyclictest4134835-21kworker/u8:1+flush-259:012:40:003
3797299992619,5cyclictest252-21systemd-journal10:50:003
3797297992622,3cyclictest4134835-21kworker/u8:1+flush-259:012:35:252
3797297992622,3cyclictest4109347-21kworker/u8:1+events_unbound12:24:442
3797297992622,3cyclictest3848783-21kworker/u8:2+flush-259:008:20:212
3797297992621,4cyclictest4036358-21kworker/u8:3+events_unbound11:50:452
3797297992620,4cyclictest3843892-21apt-get07:55:122
3797294992622,3cyclictest3932698-21apt-get09:20:111
3797290992623,2cyclictest4036358-21kworker/u8:3+flush-259:011:45:000
3797297992522,2cyclictest3848783-21kworker/u8:2+flush-259:008:45:182
3797297992522,2cyclictest3797284-21kworker/u8:3+events_unbound07:30:232
3797297992521,3cyclictest3958395-21kworker/u8:0+events_unbound10:26:432
3797297992521,3cyclictest3921212-21kworker/u8:2+events_unbound09:22:452
3797294992518,5cyclictest3901089-21latency_hist08:50:001
3797290992520,3cyclictest4125132-21apt-get12:25:100
3797290992516,6cyclictest252-21systemd-journal09:29:590
3797299992419,4cyclictest4109326-21latency_hist12:09:593
3797297992420,2cyclictest3895919-21kworker/u8:3+events_unbound09:00:222
3797297992419,4cyclictest3797284-21kworker/u8:3+events_unbound07:41:442
3797294992420,3cyclictest3797284-21kworker/u8:3+flush-259:008:10:151
3797290992420,3cyclictest3958395-21kworker/u8:0+flush-259:010:15:210
3797299992319,3cyclictest4005327-21kworker/u8:1+flush-259:011:40:013
3797299992317,4cyclictest4109561-21/usr/sbin/munin12:10:143
3797297992320,3cyclictest3984656-21kworker/u8:2+events_unbound10:20:252
3797297992319,3cyclictest4036358-21kworker/u8:3+flush-259:011:00:002
3797297992319,3cyclictest3921212-21kworker/u8:2+flush-259:009:25:142
3797290992316,5cyclictest4009567-21ntpq10:30:220
3797297992219,2cyclictest3963735-21kworker/u8:3+flush-259:009:55:232
3797297992218,3cyclictest4005327-21kworker/u8:1+flush-259:011:45:182
3797297992218,3cyclictest3984656-21kworker/u8:2+events_unbound10:32:442
3797297992216,4cyclictest4036358-21kworker/u8:3+events_unbound11:05:152
3797294992216,5cyclictest3868916-21ntpq08:15:191
3797290992217,4cyclictest4005327-21kworker/u8:1+flush-259:010:45:160
3797290992217,4cyclictest3958395-21kworker/u8:0+flush-259:010:35:160
3797290992216,4cyclictest4036860-21/usr/sbin/munin11:00:190
3797290992216,4cyclictest3875185-21apt-get08:25:120
3797299992117,3cyclictest3958395-21kworker/u8:0+flush-259:010:25:133
3797297992118,2cyclictest3921212-21kworker/u8:2+events_unbound09:40:212
3797297992117,3cyclictest4036358-21kworker/u8:3+events_unbound11:30:182
3797297992117,3cyclictest3958395-21kworker/u8:0+flush-259:010:35:212
3797297992116,3cyclictest4046995-21kworker/u8:2+flush-259:011:15:162
3797297992114,5cyclictest252-21systemd-journal11:11:432
3797294992119,2cyclictest4036358-21kworker/u8:3+flush-259:011:35:011
3797294992116,4cyclictest509-21rs:main1
379729999208,11cyclictest3964381-21irqcore09:50:153
379729999208,11cyclictest0-21swapper/310:05:013
3797299992014,4cyclictest252-21systemd-journal10:15:293
3797297992016,3cyclictest628206-21kworker/u8:1+events_unbound07:15:202
3797297992016,3cyclictest4104431-21apt-get12:05:112
3797297992014,2cyclictest4046995-21kworker/u8:2+events_unbound12:10:132
3797294992016,3cyclictest4067641-21latency_hist11:30:011
3797294992016,3cyclictest3812995-21apt-get07:25:111
3797294992015,4cyclictest252-21systemd-journal12:04:591
3797294992014,4cyclictest3911358-21latency_hist09:00:011
3797290992016,3cyclictest3797284-21kworker/u8:3+flush-259:007:55:240
3797290992015,3cyclictest4005327-21kworker/u8:1+flush-259:011:49:590
3797290992014,4cyclictest3854148-21cat08:05:000
3797290992013,5cyclictest4077103-21ntpq11:35:210
379729999197,10cyclictest0-21swapper/308:35:203
3797299991914,3cyclictest3921812-21base6409:10:003
3797299991913,4cyclictest3937814-21/usr/sbin/munin09:25:163
3797297991916,2cyclictest4036358-21kworker/u8:3+events_unbound11:00:112
3797297991916,2cyclictest4005327-21kworker/u8:1+events_unbound10:50:112
3797297991916,2cyclictest3848783-21kworker/u8:2+events_unbound08:25:222
3797297991915,3cyclictest4005327-21kworker/u8:1+flush-259:010:45:242
3797290991912,5cyclictest3931538-21ntpq09:15:210
3797299991816,1cyclictest4041819-21cron11:05:013
3797299991815,2cyclictest3895919-21kworker/u8:3+flush-259:009:00:173
3797299991814,3cyclictest3979688-21gpgv10:05:013
3797299991814,3cyclictest3979688-21gpgv10:05:003
3797297991815,2cyclictest3947916-21kworker/u8:1+flush-259:009:35:262
3797297991815,2cyclictest3797284-21kworker/u8:3+events_unbound08:05:202
3797294991816,1cyclictest0-21swapper/107:50:001
3797294991814,3cyclictest4020763-21apt-get10:45:001
379729499180,2cyclictest4093446-21munin-run11:55:001
3797290991815,2cyclictest509-21rs:main0
3797290991814,3cyclictest3848783-21kworker/u8:2+flush-259:008:20:160
3797290991814,2cyclictest0-21swapper/007:55:000
3797290991813,4cyclictest3848783-21kworker/u8:2+flush-259:008:15:150
3797290991811,5cyclictest3891159-21diskstats08:40:130
379729999177,2cyclictest4125267-21cpuspeed_turbos12:25:123
379729999176,2cyclictest3963249-21unixbench_multi09:45:243
379729999174,5cyclictest491ktimers/310:45:003
3797299991713,3cyclictest628206-21kworker/u8:1+flush-259:007:15:163
3797297991714,2cyclictest3797284-21kworker/u8:3+events_unbound08:10:152
3797297991711,4cyclictest4062721-21/usr/sbin/munin11:25:132
3797294991714,2cyclictest4134835-21kworker/u8:1+flush-259:012:35:171
3797294991713,3cyclictest4052413-21apt-get11:15:111
3797294991713,3cyclictest252-21systemd-journal07:40:001
3797294991713,3cyclictest252-21systemd-journal07:40:001
3797290991714,2cyclictest60350irq/132-enp0s29f1:rx-012:40:000
3797290991714,2cyclictest60350irq/132-enp0s29f1:rx-011:25:030
3797290991714,2cyclictest60350irq/132-enp0s29f1:rx-009:39:590
3797290991714,2cyclictest60350irq/132-enp0s29f1:rx-008:54:380
3797290991714,2cyclictest60350irq/132-enp0s29f1:rx-008:10:110
3797290991713,3cyclictest3984666-21sh10:10:000
3797290991713,3cyclictest3984666-21sh10:09:590
3797290991711,4cyclictest4118899-21ntpq12:15:210
379729999166,2cyclictest4130512-21fschecks_count12:30:123
379729999166,2cyclictest4130512-21fschecks_count12:30:123
3797299991613,2cyclictest4098934-21latency_hist12:00:003
3797297991613,2cyclictest4097654-21kworker/u8:0+events_unbound11:57:452
3797297991612,3cyclictest3984656-21kworker/u8:2+events_unbound10:10:142
3797297991612,3cyclictest3984656-21kworker/u8:2+events_unbound10:10:132
3797297991612,3cyclictest3797284-21kworker/u8:3+events_unbound07:25:272
3797294991614,1cyclictest4010438-21sh10:35:001
3797294991611,4cyclictest3890277-21kworker/u8:1+flush-259:009:10:481
3797290991614,1cyclictest60350irq/132-enp0s29f1:rx-009:50:210
3797290991614,1cyclictest0-21swapper/010:30:000
3797290991613,2cyclictest60350irq/132-enp0s29f1:rx-010:00:010
3797290991613,2cyclictest60350irq/132-enp0s29f1:rx-008:55:110
3797299991513,1cyclictest0-21swapper/311:00:003
3797299991512,2cyclictest3743884-21kworker/u8:0+flush-259:007:15:003
3797299991512,2cyclictest0-21swapper/308:30:013
3797294991512,2cyclictest3890277-21kworker/u8:1+events_unbound09:15:181
3797294991512,2cyclictest3797284-21kworker/u8:3+flush-259:007:40:171
3797294991511,3cyclictest4109347-21kworker/u8:1+flush-259:012:20:211
3797294991511,3cyclictest3953674-21/usr/sbin/munin09:40:191
379729099159,4cyclictest0-21swapper/008:45:170
3797290991514,1cyclictest60350irq/132-enp0s29f1:rx-011:22:210
3797290991513,1cyclictest60350irq/132-enp0s29f1:rx-012:10:160
3797290991513,1cyclictest60350irq/132-enp0s29f1:rx-011:10:220
3797290991513,1cyclictest60350irq/132-enp0s29f1:rx-010:57:290
3797290991513,1cyclictest60350irq/132-enp0s29f1:rx-010:20:090
3797290991512,2cyclictest0-21swapper/011:55:000
3797290991510,4cyclictest4036358-21kworker/u8:3+flush-259:011:10:010
3797299991412,1cyclictest3853904-21cron08:05:013
3797299991411,2cyclictest4030461-21needreboot10:50:193
3797299991411,2cyclictest3957610-21cut09:40:193
3797299991410,3cyclictest4036358-21kworker/u8:3+flush-259:011:50:483
3797299991410,3cyclictest3797284-21kworker/u8:3+flush-259:007:45:153
379729799149,4cyclictest3797284-21kworker/u8:3+flush-259:008:20:002
3797297991410,3cyclictest3927288-21kworker/u8:3+events_unbound09:30:172
379729499149,4cyclictest3963972-21apt-get09:50:091
379729499149,4cyclictest3797510-21apt-get07:10:121
379729499149,3cyclictest3890304-21apt-get08:40:011
3797294991410,3cyclictest4047386-21fschecks_count11:10:121
3797290991413,1cyclictest60350irq/132-enp0s29f1:rx-007:15:400
3797290991412,1cyclictest60350irq/132-enp0s29f1:rx-009:45:150
3797290991412,1cyclictest60350irq/132-enp0s29f1:rx-008:33:390
3797290991412,1cyclictest60350irq/132-enp0s29f1:rx-007:10:240
3797290991411,2cyclictest60350irq/132-enp0s29f1:rx-010:50:110
3797290991411,2cyclictest60350irq/132-enp0s29f1:rx-007:35:110
3797290991411,2cyclictest60350irq/132-enp0s29f1:rx-007:35:110
3797290991411,2cyclictest60350irq/132-enp0s29f1:rx-007:30:210
3797290991411,2cyclictest4104652-21fschecks_time12:05:130
3797290991411,2cyclictest3975992-21kthreadcore10:00:160
3797290991410,3cyclictest60350irq/132-enp0s29f1:rx-010:10:200
3797290991410,3cyclictest60350irq/132-enp0s29f1:rx-010:10:200
379729999139,3cyclictest4099132-21cat12:00:013
379729999139,3cyclictest3963735-21kworker/u8:3+flush-259:009:59:593
379729999138,4cyclictest4047189-21/usr/sbin/munin11:10:123
3797299991311,1cyclictest61450irq/139-enp0s29f1:tx-311:23:263
3797299991310,2cyclictest3927955-21sed09:15:153
3797297991310,2cyclictest4057614-21cpuspeed_turbos11:20:112
379729499138,4cyclictest4042083-21apt-get11:05:101
379729499138,3cyclictest4041808-21cron11:05:011
3797294991311,1cyclictest4088254-21sh11:49:591
3797294991311,1cyclictest0-21swapper/111:40:011
3797294991311,1cyclictest0-21swapper/109:40:011
3797294991310,2cyclictest3984656-21kworker/u8:2+flush-259:010:35:321
3797294991310,1cyclictest0-21swapper/108:10:001
3797290991312,1cyclictest60350irq/132-enp0s29f1:rx-009:24:030
3797290991311,1cyclictest60350irq/132-enp0s29f1:rx-007:47:230
379729499128,3cyclictest4078218-21apt-get11:40:111
379729499128,2cyclictest509-21rs:main1
379729099128,2cyclictest3828408-21apt-get07:40:130
3797290991210,1cyclictest60350irq/132-enp0s29f1:rx-009:05:200
379729999119,1cyclictest378876-21kworker/3:0+events_power_efficient10:10:113
379729999119,1cyclictest378876-21kworker/3:0+events_power_efficient10:10:113
379729999119,1cyclictest0-21swapper/311:05:173
379729999118,2cyclictest3891065-21cpuspeed_turbos08:40:143
379729999118,2cyclictest3838927-21fschecks_count07:50:133
379729999118,2cyclictest3815290-21kthreadcore07:25:173
379729999118,1cyclictest0-21swapper/312:20:243
379729999116,4cyclictest0-21swapper/310:30:133
379729999110,3cyclictest39511521kthreadcore09:35:173
379729799118,2cyclictest3797284-21kworker/u8:3+events_unbound08:38:432
379729499119,1cyclictest0-21swapper/107:30:161
379729499118,2cyclictest4093841-21cpuspeed_turbos11:55:131
379729499117,3cyclictest3890999-21apt-get08:40:111
379729499115,4cyclictest4130337-21apt-get12:30:101
379729499115,4cyclictest4130337-21apt-get12:30:101
3797294991110,1cyclictest0-21swapper/107:25:001
379729999106,3cyclictest3984656-21kworker/u8:2+flush-259:010:20:173
379729499108,1cyclictest0-21swapper/111:20:231
379729499107,2cyclictest3916872-21cpuspeed_turbos09:05:131
379729499105,3cyclictest3985209-21fw_conntrack10:10:151
379729499105,3cyclictest3985209-21fw_conntrack10:10:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional