You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-18 - 00:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Thu Sep 17, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3708388995046,3cyclictest3969489-21kworker/u8:3+flush-259:011:25:251
3708384994440,3cyclictest3807581-21apt-get08:45:110
3708388994339,2cyclictest3760440-21kworker/u8:0+events_unbound08:20:251
3708388994339,2cyclictest3760440-21kworker/u8:0+events_unbound08:20:251
3708394994239,2cyclictest3718455-21kworker/u8:2+events_unbound07:25:223
3708394994238,3cyclictest3811380-21kworker/u8:0+events_unbound10:05:233
3708394994234,6cyclictest3796548-21latency_hist08:35:003
3708394994138,2cyclictest3974850-21gpgv11:25:013
3708394994138,2cyclictest3811381-21kworker/u8:4+events_unbound08:58:513
3708394994137,3cyclictest3937770-21kworker/u8:1+events_unbound10:59:193
3708388994139,1cyclictest3911179-21lspci10:21:001
3708394994037,2cyclictest685178-21kworker/u8:3+events_unbound07:18:503
3708394994036,3cyclictest3949561-21kworker/u8:0+events_unbound11:02:503
3708388994035,3cyclictest3811380-21kworker/u8:0+events_unbound10:15:111
3708384994034,4cyclictest335557-21systemd-journal10:59:060
3708384994033,5cyclictest3953851-21cat11:05:010
3708394993937,2cyclictest3801694-21kworker/u8:2+events_unbound08:54:523
3708394993937,2cyclictest3760440-21kworker/u8:0+events_unbound08:20:093
3708394993937,2cyclictest3760440-21kworker/u8:0+events_unbound08:20:083
3708394993935,3cyclictest3811380-21kworker/u8:0+events_unbound09:48:513
3708388993934,4cyclictest3932983-21apt-get10:45:131
3708388993934,4cyclictest3932983-21apt-get10:45:131
3708388993931,5cyclictest3717746-21ntpq07:15:211
3708384993935,3cyclictest3811380-21kworker/u8:0+flush-259:010:15:250
3708394993836,2cyclictest3937770-21kworker/u8:1+events_unbound10:50:253
3708394993835,2cyclictest3749615-21kworker/u8:1+events_unbound08:41:523
3708394993835,2cyclictest3718455-21kworker/u8:2+events_unbound07:52:523
3708394993834,3cyclictest3995920-21kworker/u8:0+events_unbound12:36:513
3708394993734,2cyclictest3749615-21kworker/u8:1+events_unbound08:27:513
3708388993733,3cyclictest3911667-21kworker/u8:4+events_unbound10:25:111
3708394993633,2cyclictest3969489-21kworker/u8:3+events_unbound11:34:523
3708392993632,3cyclictest3995920-21kworker/u8:0+flush-259:012:20:242
3708388993633,2cyclictest3917301-21kworker/u8:2+flush-259:011:05:171
3708388993633,2cyclictest3654619-21kworker/u8:0+flush-259:007:10:241
3708394993531,3cyclictest3811380-21kworker/u8:0+events_unbound09:50:223
3708394993528,5cyclictest3852944-21ntpq09:25:213
3708388993531,3cyclictest3969489-21kworker/u8:3+events_unbound12:00:251
3708394993431,2cyclictest3995920-21kworker/u8:0+events_unbound12:25:523
3708394993431,2cyclictest3949561-21kworker/u8:0+events_unbound11:05:283
3708394993431,2cyclictest3917301-21kworker/u8:2+events_unbound11:19:503
3708394993430,3cyclictest3801694-21kworker/u8:2+events_unbound08:45:373
3708394993429,3cyclictest3771046-21apt-get08:10:123
3708392993430,3cyclictest3811380-21kworker/u8:0+flush-259:010:10:172
3708388993428,5cyclictest3980489-21apt-get11:30:111
3708388993428,4cyclictest3718455-21kworker/u8:2+flush-259:007:24:591
3708394993329,3cyclictest3811380-21kworker/u8:0+events_unbound10:21:193
3708384993328,4cyclictest3985595-21/usr/sbin/munin11:35:190
3708384993326,5cyclictest3842548-21ntpq09:15:200
3708388993228,3cyclictest3796239-21kworker/u8:3+flush-259:008:35:171
3708388993227,3cyclictest3922653-21/usr/sbin/munin10:35:271
3708384993227,4cyclictest3874670-21apt-get09:50:110
3708388993127,3cyclictest3911667-21kworker/u8:4+events_unbound10:50:451
3708384993128,2cyclictest3749615-21kworker/u8:1+flush-259:008:30:150
3708394993027,2cyclictest3879456-21kworker/u8:1+events_unbound10:00:273
3708392993026,3cyclictest4027046-21apt-get12:15:132
3708392993026,3cyclictest3796239-21kworker/u8:3+flush-259:008:40:002
3708388993026,3cyclictest3917301-21kworker/u8:2+flush-259:011:20:241
3708388993026,3cyclictest3754838-21apt-get07:55:021
3708388993025,3cyclictest3760440-21kworker/u8:0+flush-259:008:00:151
3708394992923,4cyclictest3838294-21latency_hist09:15:003
3708384992925,3cyclictest3703228-21kworker/u8:1+flush-259:007:15:010
3708394992825,2cyclictest3811380-21kworker/u8:0+events_unbound10:15:283
3708392992824,3cyclictest3718455-21kworker/u8:2+flush-259:007:35:002
3708392992822,4cyclictest3952589-21ntpq11:00:202
3708388992825,2cyclictest3995920-21kworker/u8:0+flush-259:011:50:131
3708388992825,2cyclictest3811381-21kworker/u8:4+events_unbound08:55:141
3708388992824,3cyclictest3969489-21kworker/u8:3+events_unbound11:40:121
3708388992823,4cyclictest3811380-21kworker/u8:0+events_unbound10:05:191
3708388992823,3cyclictest3995920-21kworker/u8:0+flush-259:012:10:111
3708384992822,4cyclictest3959332-21apt-get11:10:120
3708394992724,3cyclictest3917301-21kworker/u8:2+events_unbound10:35:383
3708394992724,2cyclictest4015509-21kworker/u8:2+events_unbound12:10:513
3708392992723,3cyclictest3811380-21kworker/u8:0+flush-259:009:25:242
3708392992721,5cyclictest335557-21systemd-journal08:00:002
3708388992723,3cyclictest3905248-21kworker/u8:1+flush-259:010:30:171
3708388992722,3cyclictest3943343-21apt-get10:55:111
3708384992725,1cyclictest3749672-21kworker/0:3+pm10:21:000
3708384992721,4cyclictest3833430-21apt-get09:10:120
3708394992623,2cyclictest3985394-21kworker/u8:1+events_unbound12:05:163
3708392992623,2cyclictest3760440-21kworker/u8:0+flush-259:008:20:182
3708392992623,2cyclictest3760440-21kworker/u8:0+flush-259:008:20:172
3708392992620,4cyclictest335557-21systemd-journal08:40:492
3708388992623,2cyclictest3811380-21kworker/u8:0+flush-259:009:35:241
3708388992622,3cyclictest3811380-21kworker/u8:0+events_unbound09:50:181
3708388992619,5cyclictest3708305-21latency_hist07:10:011
3708394992523,2cyclictest3718455-21kworker/u8:2+events_unbound07:45:513
3708394992522,3cyclictest3734172-21kworker/u8:0+events_unbound07:41:503
3708394992522,2cyclictest3949561-21kworker/u8:0+events_unbound11:10:423
3708394992522,2cyclictest3811380-21kworker/u8:0+events_unbound09:35:253
3708394992522,2cyclictest3749615-21kworker/u8:1+events_unbound07:55:533
3708394992521,3cyclictest4021603-21kworker/u8:4+events_unbound12:15:273
3708392992522,2cyclictest3985394-21kworker/u8:1+flush-259:011:55:002
3708392992522,2cyclictest3905248-21kworker/u8:1+flush-259:010:30:212
3708392992522,2cyclictest3811380-21kworker/u8:0+flush-259:010:05:172
3708388992521,3cyclictest3949561-21kworker/u8:0+flush-259:011:15:241
3708388992519,4cyclictest4042479-21/usr/sbin/munin12:30:131
3708384992521,3cyclictest3949561-21kworker/u8:0+flush-259:011:06:500
3708384992521,3cyclictest3729261-21apt-get07:30:100
3708384992519,4cyclictest3781553-21diskstats08:20:150
3708384992519,4cyclictest3781553-21diskstats08:20:140
3708384992515,8cyclictest3865140-21irqcore09:40:170
3708394992421,2cyclictest3760440-21kworker/u8:0+events_unbound08:39:513
3708392992416,6cyclictest3868388-21ntpq09:40:222
370839299241,22cyclictest1-21systemd10:25:002
3708388992421,2cyclictest3760440-21kworker/u8:0+flush-259:008:25:131
3708388992420,3cyclictest3995920-21kworker/u8:0+flush-259:012:35:251
3708388992420,3cyclictest3760440-21kworker/u8:0+flush-259:008:10:261
3708388992420,3cyclictest3749615-21kworker/u8:1+events_unbound08:30:201
3708388992420,3cyclictest3718455-21kworker/u8:2+events_unbound07:40:141
3708388992419,4cyclictest685178-21kworker/u8:3+flush-259:007:50:221
3708384992419,3cyclictest3843698-21apt-get09:20:110
3708394992320,2cyclictest4037136-21kworker/u8:2+events_unbound12:30:553
3708394992320,2cyclictest3811380-21kworker/u8:0+events_unbound09:22:593
3708392992320,2cyclictest3949561-21kworker/u8:0+flush-259:011:15:162
3708388992320,2cyclictest685178-21kworker/u8:3+flush-259:007:30:171
3708388992319,3cyclictest3749615-21kworker/u8:1+events_unbound08:45:001
3708394992220,2cyclictest3801694-21kworker/u8:2+events_unbound09:00:163
3708394992219,2cyclictest3995920-21kworker/u8:0+events_unbound12:00:083
3708394992219,2cyclictest3985394-21kworker/u8:1+events_unbound11:50:303
3708394992219,2cyclictest3760440-21kworker/u8:0+events_unbound08:01:533
3708394992219,2cyclictest3749615-21kworker/u8:1+events_unbound08:06:503
3708394992218,3cyclictest3917301-21kworker/u8:2+events_unbound11:28:043
3708392992217,3cyclictest3776234-21apt-get08:15:122
3708392992217,3cyclictest3754843-21latency_hist07:55:002
3708388992218,3cyclictest3827982-21kworker/u8:3+events_unbound09:09:501
3708388992217,4cyclictest3995920-21kworker/u8:0+events_unbound11:59:511
3708384992220,1cyclictest3969489-21kworker/u8:3+flush-259:012:00:210
3708384992217,3cyclictest509-21rs:main0
3708394992119,2cyclictest4031713-21kworker/u8:1+events_unbound12:24:553
3708394992118,2cyclictest3969489-21kworker/u8:3+events_unbound11:40:023
3708392992117,3cyclictest3879456-21kworker/u8:1+flush-259:009:55:232
3708392992117,3cyclictest3811380-21kworker/u8:0+flush-259:009:54:592
3708388992118,2cyclictest3969489-21kworker/u8:3+flush-259:011:39:551
3708388992118,2cyclictest3911667-21kworker/u8:4+events_unbound10:40:201
3708384992116,3cyclictest3911667-21kworker/u8:4+flush-259:010:55:000
3708394992018,2cyclictest3749615-21kworker/u8:1+events_unbound08:16:403
3708394992017,2cyclictest3827982-21kworker/u8:3+events_unbound09:17:513
3708394992017,2cyclictest3811380-21kworker/u8:0+events_unbound09:59:243
3708394992017,2cyclictest3801694-21kworker/u8:2+events_unbound09:08:523
3708394992017,2cyclictest3718455-21kworker/u8:2+events_unbound07:36:403
3708394992017,2cyclictest3718455-21kworker/u8:2+events_unbound07:34:563
3708394992017,2cyclictest3718455-21kworker/u8:2+events_unbound07:20:533
3708394992016,3cyclictest3969489-21kworker/u8:3+events_unbound11:35:563
3708388992017,2cyclictest3879456-21kworker/u8:1+events_unbound10:00:151
3708388992017,2cyclictest3858968-21kworker/u8:2+events_unbound09:40:171
3708388992015,4cyclictest3995920-21kworker/u8:0+events_unbound11:49:511
3708388992014,4cyclictest3776209-21/usr/sbin/munin08:15:201
3708384992014,4cyclictest3744909-21diskstats07:45:130
3708394991916,3cyclictest3927612-21kworker/u8:0+events_unbound10:45:263
3708394991916,3cyclictest3927612-21kworker/u8:0+events_unbound10:45:263
3708394991916,2cyclictest3917301-21kworker/u8:2+events_unbound10:42:593
3708394991916,2cyclictest3884778-21kworker/u8:2+events_unbound10:10:523
3708394991916,2cyclictest3811380-21kworker/u8:0+events_unbound09:43:293
3708392991916,2cyclictest3801694-21kworker/u8:2+flush-259:009:00:212
3708392991916,2cyclictest0-21swapper/207:20:012
3708392991913,4cyclictest3948322-21latency_hist11:00:012
3708388991916,2cyclictest3801694-21kworker/u8:2+events_unbound08:50:271
3708388991915,3cyclictest3953438-21idleruntime-cro11:05:001
3708388991915,3cyclictest3811380-21kworker/u8:0+events_unbound09:34:511
3708388991915,3cyclictest3801694-21kworker/u8:2+events_unbound08:45:231
3708388991915,3cyclictest3749615-21kworker/u8:1+events_unbound08:05:121
3708388991915,2cyclictest3843259-21kworker/u8:1+events_unbound09:20:111
3708388991914,3cyclictest3843259-21kworker/u8:1+flush-259:009:25:171
3708388991913,4cyclictest3899321-21ntpq10:10:211
3708384991913,4cyclictest3968518-21ntpq11:15:200
3708394991816,2cyclictest3811380-21kworker/u8:0+events_unbound09:31:533
3708394991815,2cyclictest3985394-21kworker/u8:1+events_unbound11:59:033
3708392991815,2cyclictest3911667-21kworker/u8:4+flush-259:010:35:212
3708392991814,3cyclictest3718455-21kworker/u8:2+flush-259:007:35:242
3708388991815,2cyclictest3718455-21kworker/u8:2+events_unbound07:25:211
3708388991813,3cyclictest3833405-21/usr/sbin/munin09:10:151
3708384991815,2cyclictest507-21in:imuxsock10:00:000
3708384991815,2cyclictest3718455-21kworker/u8:2+flush-259:007:35:220
3708384991814,3cyclictest3843259-21kworker/u8:1+flush-259:009:30:140
3708384991812,4cyclictest3791405-21latency_hist08:30:010
370838499180,17cyclictest3723816-21turbostat07:24:590
3708394991714,2cyclictest3995920-21kworker/u8:0+events_unbound11:48:503
3708394991711,4cyclictest335557-21systemd-journal10:32:513
3708392991714,2cyclictest3938560-21irqcore10:50:172
3708392991714,2cyclictest3728996-21sh07:30:002
3708392991714,2cyclictest3703228-21kworker/u8:1+flush-259:007:11:072
3708392991713,3cyclictest4047635-21/usr/sbin/munin12:35:232
3708388991714,2cyclictest3949561-21kworker/u8:0+events_unbound11:10:161
3708388991713,3cyclictest3995920-21kworker/u8:0+events_unbound12:20:211
3708388991713,3cyclictest3801694-21kworker/u8:2+flush-259:009:00:321
3708388991713,3cyclictest3718455-21kworker/u8:2+events_unbound07:35:231
3708388991711,4cyclictest335557-21systemd-journal12:27:491
3708384991714,2cyclictest60350irq/132-enp0s29f1:rx-010:25:240
3708384991714,2cyclictest60350irq/132-enp0s29f1:rx-009:10:010
3708384991713,3cyclictest3749615-21kworker/u8:1+flush-259:008:16:020
3708384991712,4cyclictest3995952-21apt-get11:45:110
3708384991711,4cyclictest4001297-21diskstats11:50:150
3708394991613,2cyclictest491ktimers/307:10:193
3708392991613,2cyclictest3985394-21kworker/u8:1+flush-259:011:55:142
3708392991613,2cyclictest3869903-21irqcore09:45:162
3708392991612,3cyclictest3811381-21kworker/u8:4+flush-259:008:51:072
3708388991613,2cyclictest3858968-21kworker/u8:2+events_unbound09:45:241
3708384991614,1cyclictest60350irq/132-enp0s29f1:rx-010:47:560
3708384991614,1cyclictest60350irq/132-enp0s29f1:rx-010:47:560
3708384991614,1cyclictest60350irq/132-enp0s29f1:rx-008:05:120
3708384991614,1cyclictest60350irq/132-enp0s29f1:rx-007:50:210
3708384991613,2cyclictest61150irq/133-enp0s29f1:tx-012:10:510
3708384991612,2cyclictest0-21swapper/012:10:000
370839299159,4cyclictest4021818-21/usr/sbin/munin12:10:202
370839299153,6cyclictest3959071-21munin-plugin-st11:10:002
3708392991513,1cyclictest500-21systemd-logind09:10:012
3708392991512,2cyclictest3775112-21ntp_kernel_pll_08:10:232
3708392991511,3cyclictest3811381-21kworker/u8:4+flush-259:008:55:242
3708384991513,1cyclictest60350irq/132-enp0s29f1:rx-012:25:140
3708384991513,1cyclictest60350irq/132-enp0s29f1:rx-011:40:140
3708384991513,1cyclictest60350irq/132-enp0s29f1:rx-010:40:230
3708384991513,1cyclictest60350irq/132-enp0s29f1:rx-010:35:200
3708384991513,1cyclictest60350irq/132-enp0s29f1:rx-010:30:150
3708384991513,1cyclictest60350irq/132-enp0s29f1:rx-010:05:250
3708384991513,1cyclictest60350irq/132-enp0s29f1:rx-009:35:140
3708384991513,1cyclictest60350irq/132-enp0s29f1:rx-008:50:160
3708384991513,1cyclictest60350irq/132-enp0s29f1:rx-008:42:160
3708384991513,1cyclictest60350irq/132-enp0s29f1:rx-007:25:470
3708384991511,3cyclictest335557-21systemd-journal08:38:490
3708392991412,1cyclictest3708380-21cyclictest09:10:182
3708392991411,2cyclictest4000455-21unixbench_singl11:45:242
3708392991410,3cyclictest4037136-21kworker/u8:2+flush-259:012:26:542
3708384991412,1cyclictest60350irq/132-enp0s29f1:rx-012:35:250
3708384991412,1cyclictest60350irq/132-enp0s29f1:rx-011:55:100
3708384991412,1cyclictest60350irq/132-enp0s29f1:rx-011:30:250
3708384991412,1cyclictest60350irq/132-enp0s29f1:rx-011:20:200
3708384991412,1cyclictest60350irq/132-enp0s29f1:rx-010:11:300
3708384991412,1cyclictest60350irq/132-enp0s29f1:rx-010:00:100
3708384991412,1cyclictest60350irq/132-enp0s29f1:rx-009:02:530
3708384991412,1cyclictest60350irq/132-enp0s29f1:rx-007:55:160
3708384991412,1cyclictest60350irq/132-enp0s29f1:rx-007:40:210
370839299139,3cyclictest4015509-21kworker/u8:2+flush-259:012:00:482
370839299139,3cyclictest3969489-21kworker/u8:3+flush-259:011:26:202
370839299139,3cyclictest3811380-21kworker/u8:0+flush-259:009:36:122
3708392991311,1cyclictest0-21swapper/211:40:192
3708392991310,2cyclictest685178-21kworker/u8:3+flush-259:007:45:002
370838899139,3cyclictest3811380-21kworker/u8:0+flush-259:009:55:381
3708388991310,2cyclictest4027181-21cpuspeed_turbos12:15:141
3708384991311,1cyclictest60350irq/132-enp0s29f1:rx-012:22:180
3708384991311,1cyclictest60350irq/132-enp0s29f1:rx-009:48:230
370839299129,2cyclictest3972026-21kthreadcore11:20:172
370839299129,2cyclictest3808607-21tr08:45:172
370839299129,2cyclictest3791392-21cron08:29:592
370839299128,3cyclictest685178-21kworker/u8:3+flush-259:007:50:012
370839299128,3cyclictest4015509-21kworker/u8:2+flush-259:012:05:252
370839299128,3cyclictest335557-21systemd-journal11:12:502
370838899128,3cyclictest3801694-21kworker/u8:2+events_unbound09:15:151
370838899127,3cyclictest4015509-21kworker/u8:2+events_unbound12:07:511
370838499129,2cyclictest60350irq/132-enp0s29f1:rx-007:15:230
3708384991210,1cyclictest60350irq/132-enp0s29f1:rx-012:30:230
3708384991210,1cyclictest60350irq/132-enp0s29f1:rx-008:10:150
370839299119,1cyclictest0-21swapper/209:20:192
370839299118,2cyclictest4042572-21cpuspeed_turbos12:30:122
370839299118,2cyclictest3927962-21cpuspeed_turbos10:40:122
370839299116,3cyclictest3723777-21sshd07:24:452
370839299116,3cyclictest335557-21systemd-journal10:28:502
370838899119,1cyclictest0-21swapper/107:45:161
370839299108,1cyclictest0-21swapper/208:05:142
370839299106,3cyclictest3811380-21kworker/u8:0+flush-259:009:31:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional