You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-01 - 00:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Sun May 31, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3720679994945,3cyclictest3705376-21kworker/u8:1+events_unbound08:10:131
3720679994744,2cyclictest3887166-21gpgv09:50:001
3720679994643,2cyclictest3980995-21kworker/u8:4+events_unbound11:40:141
3720676994539,4cyclictest3828853-21ntpq08:50:210
3720684994438,4cyclictest3803798-21kworker/u8:0+flush-259:008:40:162
3720685994340,2cyclictest3803798-21kworker/u8:0+flush-259:009:00:223
3720679994339,3cyclictest3980995-21kworker/u8:4+events_unbound11:20:191
3720684994136,3cyclictest4001540-21kworker/u8:2+flush-259:012:05:192
3720685994037,2cyclictest3845171-21kworker/u8:3+flush-259:009:20:003
3720679993936,2cyclictest3865826-21kworker/u8:2+events_unbound10:25:231
3720679993936,2cyclictest3845171-21kworker/u8:3+events_unbound09:25:221
3720679993936,2cyclictest3768057-21kworker/u8:0+flush-259:008:00:231
3720679993936,2cyclictest3705376-21kworker/u8:1+flush-259:007:10:241
3720679993936,2cyclictest3705376-21kworker/u8:1+events_unbound08:00:001
3720679993936,2cyclictest3705376-21kworker/u8:1+events_unbound07:40:241
3720679993936,2cyclictest3705376-21kworker/u8:1+events_unbound07:35:191
3720685993835,2cyclictest3959821-21kworker/u8:0+flush-259:011:10:153
3720679993835,2cyclictest3803798-21kworker/u8:0+flush-259:008:30:271
3720676993835,2cyclictest3845171-21kworker/u8:3+flush-259:009:10:260
3720679993734,2cyclictest3814279-21kworker/u8:2+events_unbound08:45:251
3720679993734,2cyclictest3725798-21kworker/u8:2+flush-259:007:25:201
3720676993732,3cyclictest3959821-21kworker/u8:0+flush-259:011:10:160
3720685993633,2cyclictest3865826-21kworker/u8:2+events_unbound10:25:173
3720679993633,2cyclictest3898020-21kworker/u8:1+events_unbound11:00:211
3720679993633,2cyclictest3715114-21kworker/u8:3+flush-259:007:15:151
3720679993631,4cyclictest335557-21systemd-journal12:10:161
3720685993531,3cyclictest4001540-21kworker/u8:2+flush-259:011:55:243
3720685993529,4cyclictest4063361-21latency_hist12:40:013
3720679993532,2cyclictest3887005-21kworker/u8:0+events_unbound09:55:201
3720679993532,2cyclictest3887005-21kworker/u8:0+events_unbound09:55:191
3720679993532,2cyclictest3865826-21kworker/u8:2+events_unbound10:05:211
3720679993532,2cyclictest3803798-21kworker/u8:0+events_unbound09:05:231
3720679993528,5cyclictest3797777-21ntpq08:20:201
3720676993531,3cyclictest3934515-21apt-get10:35:120
3720679993432,2cyclictest4001540-21kworker/u8:2+flush-259:012:00:211
3720679993431,2cyclictest4042790-21kworker/u8:3+flush-259:012:25:201
3720679993431,2cyclictest3898020-21kworker/u8:1+events_unbound11:05:221
3720679993431,2cyclictest3803798-21kworker/u8:0+events_unbound09:00:201
3720679993431,2cyclictest3741848-21kworker/u8:4+flush-259:007:45:251
3720679993431,2cyclictest3741848-21kworker/u8:4+flush-259:007:45:251
3720685993329,3cyclictest3898020-21kworker/u8:1+flush-259:011:00:223
3720685993329,3cyclictest3768057-21kworker/u8:0+flush-259:008:14:383
3720679993331,2cyclictest3865826-21kworker/u8:2+flush-259:010:30:141
3720679993330,2cyclictest3865826-21kworker/u8:2+flush-259:010:20:011
3720679993330,2cyclictest3865826-21kworker/u8:2+events_unbound10:20:241
3720676993328,4cyclictest3928244-21ntpq10:25:210
3720685993227,4cyclictest3715114-21kworker/u8:3+flush-259:007:20:163
3720685993227,4cyclictest3705376-21kworker/u8:1+events_unbound07:10:213
3720679993230,2cyclictest3934119-21kworker/u8:3+events_unbound10:40:221
3720679993229,2cyclictest3793771-21kworker/u8:2+events_unbound08:25:221
3720679993228,3cyclictest4012057-21/usr/sbin/munin11:50:131
3720676993229,2cyclictest3898020-21kworker/u8:1+flush-259:010:05:250
3720676993227,4cyclictest3923105-21ntpq10:20:210
3720679993126,2cyclictest4001540-21kworker/u8:2+events_unbound12:35:281
3720676993127,3cyclictest3980995-21kworker/u8:4+events_unbound11:40:240
3720685993027,2cyclictest4042790-21kworker/u8:3+flush-259:012:30:153
3720685993027,2cyclictest4017211-21kworker/u8:1+events_unbound12:05:233
3720685993027,2cyclictest3934119-21kworker/u8:3+flush-259:010:40:243
3720685993025,3cyclictest3768057-21kworker/u8:0+flush-259:008:00:013
3720679993028,2cyclictest4001540-21kworker/u8:2+flush-259:012:05:151
3720679993028,2cyclictest3845171-21kworker/u8:3+events_unbound09:30:191
3720685992926,2cyclictest4042790-21kworker/u8:3+events_unbound12:25:233
3720685992926,2cyclictest4001540-21kworker/u8:2+flush-259:012:20:243
3720685992923,4cyclictest3954836-21latency_hist10:55:013
3720679992927,2cyclictest3768057-21kworker/u8:0+events_unbound08:05:171
3720679992926,2cyclictest3959821-21kworker/u8:0+events_unbound12:20:191
3720679992926,2cyclictest3814279-21kworker/u8:2+events_unbound08:57:341
3720676992925,3cyclictest3725798-21kworker/u8:2+events_unbound07:25:240
3720685992825,2cyclictest4001540-21kworker/u8:2+flush-259:012:15:143
3720685992825,2cyclictest3980995-21kworker/u8:4+flush-259:011:45:153
3720685992823,4cyclictest3715114-21kworker/u8:3+flush-259:007:15:163
3720679992825,2cyclictest3865826-21kworker/u8:2+events_unbound10:35:231
3720679992825,2cyclictest3865826-21kworker/u8:2+events_unbound10:00:141
3720679992825,2cyclictest3715114-21kworker/u8:3+events_unbound07:20:131
3720676992823,4cyclictest335557-21systemd-journal10:00:350
3720685992724,2cyclictest3865826-21kworker/u8:2+flush-259:010:35:163
3720685992723,3cyclictest3793771-21kworker/u8:2+events_unbound08:25:283
3720685992723,3cyclictest3725798-21kworker/u8:2+events_unbound07:25:223
3720685992720,3cyclictest3855665-21kworker/u8:1+events_unbound09:20:183
3720679992725,2cyclictest3959821-21kworker/u8:0+flush-259:011:10:241
3720679992724,3cyclictest3814279-21kworker/u8:2+events_unbound08:54:381
3720679992724,2cyclictest3959821-21kworker/u8:0+events_unbound11:15:201
3720679992723,3cyclictest3803798-21kworker/u8:0+flush-259:008:40:351
3720676992724,2cyclictest3959821-21kworker/u8:0+events_unbound11:20:120
3720676992724,2cyclictest3803798-21kworker/u8:0+events_unbound09:00:230
3720676992723,3cyclictest3855665-21kworker/u8:1+flush-259:009:34:590
3720676992721,4cyclictest3891723-21ntpq09:50:220
3720676992721,4cyclictest3773436-21diskstats08:00:150
3720685992623,2cyclictest3871622-21kworker/u8:4+events_unbound09:40:203
3720685992623,2cyclictest3855665-21kworker/u8:1+events_unbound09:35:233
3720679992623,2cyclictest4001540-21kworker/u8:2+events_unbound11:55:211
3720679992623,2cyclictest3980995-21kworker/u8:4+events_unbound11:34:351
3720679992623,2cyclictest3741848-21kworker/u8:4+events_unbound07:50:171
3720676992623,2cyclictest4001540-21kworker/u8:2+flush-259:011:50:140
3720685992522,3cyclictest3934119-21kworker/u8:3+events_unbound10:45:273
3720685992522,2cyclictest3980995-21kworker/u8:4+events_unbound11:40:363
3720685992522,2cyclictest3876849-21kworker/u8:3+flush-259:009:45:153
3720685992521,3cyclictest3705376-21kworker/u8:1+events_unbound07:30:213
3720685992521,3cyclictest3705376-21kworker/u8:1+events_unbound07:30:203
3720684992518,4cyclictest3959941-21latency_hist11:00:002
3720684992516,6cyclictest335557-21systemd-journal09:09:592
3720679992522,2cyclictest3980995-21kworker/u8:4+events_unbound11:49:331
3720679992521,3cyclictest3898020-21kworker/u8:1+events_unbound10:50:011
3720676992521,3cyclictest3959821-21kworker/u8:0+flush-259:011:15:200
3720676992518,5cyclictest3782366-21ntpq08:05:220
3720685992421,2cyclictest3980995-21kworker/u8:4+flush-259:011:25:203
3720685992421,2cyclictest3803798-21kworker/u8:0+events_unbound08:30:213
3720685992420,3cyclictest3865826-21kworker/u8:2+flush-259:010:20:213
3720685992420,3cyclictest3803798-21kworker/u8:0+events_unbound09:05:193
3720684992421,2cyclictest3705376-21kworker/u8:1+flush-259:007:15:002
3720684992419,3cyclictest3768057-21kworker/u8:0+flush-259:008:20:002
3720679992420,3cyclictest3783306-21kworker/u8:3+flush-259:008:35:341
3720679992420,3cyclictest3741848-21kworker/u8:4+flush-259:007:30:161
3720679992420,3cyclictest3741848-21kworker/u8:4+flush-259:007:30:151
3720679992418,5cyclictest3793543-21latency_hist08:20:011
3720676992420,3cyclictest3793771-21kworker/u8:2+flush-259:008:25:160
3720676992420,3cyclictest3741848-21kworker/u8:4+events_unbound07:45:180
3720676992420,3cyclictest3741848-21kworker/u8:4+events_unbound07:45:170
3720676992420,3cyclictest3705376-21kworker/u8:1+flush-259:007:55:200
3720676992419,4cyclictest3813386-21ntpq08:35:210
3720676992418,4cyclictest335557-21systemd-journal11:00:010
3720685992320,2cyclictest3959821-21kworker/u8:0+flush-259:011:05:263
3720685992320,2cyclictest3865826-21kworker/u8:2+flush-259:010:30:243
3720676992316,5cyclictest3882191-21diskstats09:45:130
3720685992218,3cyclictest3980995-21kworker/u8:4+events_unbound11:30:133
3720685992218,3cyclictest3955101-21apt-get10:55:113
3720685992218,3cyclictest3783306-21kworker/u8:3+events_unbound08:42:343
3720685992218,3cyclictest3741848-21kworker/u8:4+events_unbound07:45:223
3720685992218,3cyclictest3741848-21kworker/u8:4+events_unbound07:45:213
3720685992216,4cyclictest4037632-21latency_hist12:15:013
3720684992218,2cyclictest0-21swapper/208:25:012
3720684992217,3cyclictest3975628-21gpgv11:15:002
3720679992219,2cyclictest4001540-21kworker/u8:2+flush-259:012:15:191
3720679992218,3cyclictest3949693-21kworker/u8:2+flush-259:010:55:191
3720676992219,2cyclictest3865826-21kworker/u8:2+flush-259:010:10:140
3720676992218,3cyclictest4042790-21kworker/u8:3+events_unbound12:30:190
3720676992218,3cyclictest3980995-21kworker/u8:4+events_unbound11:45:190
3720676992218,3cyclictest3705376-21kworker/u8:1+events_unbound07:10:130
3720676992217,3cyclictest3741848-21kworker/u8:4+events_unbound07:50:220
3720685992118,2cyclictest3865826-21kworker/u8:2+flush-259:010:00:153
3720685992118,2cyclictest3783306-21kworker/u8:3+events_unbound08:38:363
3720685992116,4cyclictest3908721-21/usr/sbin/munin10:10:183
3720685992116,3cyclictest335557-21systemd-journal09:55:003
3720684992115,4cyclictest3871817-21diskstats09:35:142
3720676992118,2cyclictest3959821-21kworker/u8:0+flush-259:011:10:000
3720676992118,2cyclictest3803798-21kworker/u8:0+events_unbound08:40:240
3720676992117,3cyclictest3855665-21kworker/u8:1+flush-259:009:35:160
3720676992117,3cyclictest3715114-21kworker/u8:3+events_unbound07:15:260
3720685992017,2cyclictest3871622-21kworker/u8:4+events_unbound10:15:283
3720685992017,2cyclictest3845171-21kworker/u8:3+flush-259:009:10:353
3720685992017,2cyclictest3803798-21kworker/u8:0+events_unbound08:55:163
3720685992017,1cyclictest0-21swapper/310:10:013
3720684992015,3cyclictest3829701-21idleruntime-cro08:55:002
3720679992017,2cyclictest3898020-21kworker/u8:1+events_unbound10:10:221
3720679992016,3cyclictest3855665-21kworker/u8:1+events_unbound09:20:261
3720679992016,2cyclictest0-21swapper/109:45:001
3720676992018,1cyclictest4017211-21kworker/u8:1+events_unbound12:10:210
3720676992015,4cyclictest4017211-21kworker/u8:1+events_unbound12:09:360
3720676992014,4cyclictest4041920-21ntpq12:15:210
3720684991915,3cyclictest4042790-21kworker/u8:3+flush-259:012:25:262
3720684991915,2cyclictest0-21swapper/211:55:002
3720684991913,4cyclictest335557-21systemd-journal10:48:352
3720679991913,4cyclictest3720583-21latency_hist07:10:001
3720676991916,2cyclictest3814279-21kworker/u8:2+flush-259:008:45:210
3720676991916,2cyclictest3741848-21kworker/u8:4+flush-259:007:30:340
3720676991916,2cyclictest3741848-21kworker/u8:4+flush-259:007:30:340
3720676991915,3cyclictest3887005-21kworker/u8:0+flush-259:009:55:210
3720676991915,3cyclictest3887005-21kworker/u8:0+flush-259:009:55:200
3720676991915,3cyclictest3803798-21kworker/u8:0+flush-259:009:05:210
3720685991815,2cyclictest3814279-21kworker/u8:2+events_unbound08:47:353
3720685991814,3cyclictest3803798-21kworker/u8:0+events_unbound08:50:253
3720684991814,3cyclictest3783306-21kworker/u8:3+flush-259:008:35:262
3720684991814,3cyclictest3715114-21kworker/u8:3+events_unbound07:15:172
3720684991812,4cyclictest3761801-21ntpq07:45:232
3720684991812,4cyclictest3761801-21ntpq07:45:232
3720679991814,2cyclictest0-21swapper/111:30:011
3720679991813,3cyclictest335557-21systemd-journal11:35:341
3720676991814,3cyclictest3803798-21kworker/u8:0+events_unbound08:30:180
3720676991813,4cyclictest3783306-21kworker/u8:3+flush-259:008:20:190
3720676991813,4cyclictest3705376-21kworker/u8:1+flush-259:007:44:350
3720684991710,5cyclictest3992435-21sed11:30:172
3720679991714,2cyclictest3814279-21kworker/u8:2+events_unbound09:15:341
3720679991713,3cyclictest3871622-21kworker/u8:4+events_unbound09:51:111
3720676991714,2cyclictest4063372-21cut12:40:010
3720676991714,2cyclictest4001540-21kworker/u8:2+flush-259:012:00:350
3720676991714,2cyclictest3980995-21kworker/u8:4+events_unbound11:25:230
3720676991714,2cyclictest3855665-21kworker/u8:1+flush-259:009:20:150
3720676991714,2cyclictest3845171-21kworker/u8:3+flush-259:009:25:210
3720676991712,3cyclictest3939628-21/usr/sbin/munin10:40:150
3720685991614,1cyclictest3705376-21kworker/u8:1+events_unbound07:35:173
3720685991613,2cyclictest3998955-21kthreadcore11:35:203
3720685991613,2cyclictest3934119-21kworker/u8:3+flush-259:011:15:343
3720685991612,3cyclictest3865826-21kworker/u8:2+events_unbound09:56:373
3720685991612,3cyclictest3865826-21kworker/u8:2+events_unbound09:56:363
3720685991612,3cyclictest3845171-21kworker/u8:3+events_unbound09:29:593
3720685991610,4cyclictest3870968-21latency_hist09:35:013
3720684991613,2cyclictest0-21swapper/207:35:002
3720684991613,2cyclictest0-21swapper/207:35:002
3720684991612,3cyclictest3916195-21tr10:15:192
3720684991611,4cyclictest3793771-21kworker/u8:2+flush-259:008:35:002
3720684991611,3cyclictest3871622-21kworker/u8:4+flush-259:010:11:382
3720684991611,3cyclictest3773698-21irqcore08:00:172
3720679991613,2cyclictest4042790-21kworker/u8:3+events_unbound12:33:211
3720679991613,2cyclictest3845171-21kworker/u8:3+events_unbound09:10:171
3720676991614,1cyclictest60350irq/132-enp0s29f1:rx-010:31:570
3720676991614,1cyclictest60350irq/132-enp0s29f1:rx-007:35:190
3720676991613,2cyclictest3934119-21kworker/u8:3+flush-259:010:45:350
3720676991612,3cyclictest3705376-21kworker/u8:1+flush-259:008:15:360
3720676991612,3cyclictest3705376-21kworker/u8:1+events_unbound08:11:340
3720676991611,4cyclictest4048160-21kworker/u8:1+flush-259:012:25:240
3720676991610,4cyclictest335557-21systemd-journal11:30:110
3720685991511,3cyclictest3741848-21kworker/u8:4+events_unbound07:52:073
3720685991511,3cyclictest3705376-21kworker/u8:1+events_unbound08:00:023
372068499159,4cyclictest4032705-21/usr/sbin/munin12:10:152
372068499159,4cyclictest3944532-21latency_hist10:45:002
3720684991512,2cyclictest3964186-21ntp_kernel_err11:00:212
3720684991512,2cyclictest3959821-21kworker/u8:0+events_unbound11:15:022
3720684991512,2cyclictest3902083-21ntp_kernel_pll_10:00:222
3720684991512,2cyclictest3741848-21kworker/u8:4+flush-259:007:52:052
3720684991510,4cyclictest335557-21systemd-journal07:25:342
3720684991510,3cyclictest3959821-21kworker/u8:0+events_unbound11:50:002
3720684991510,3cyclictest3886973-21apt-get09:50:012
3720684991510,3cyclictest3865774-21cron09:30:002
3720684991510,2cyclictest0-21swapper/211:10:012
372068499150,12cyclictest335557-21systemd-journal11:25:342
372067699159,4cyclictest335557-21systemd-journal07:25:000
3720676991514,1cyclictest60350irq/132-enp0s29f1:rx-011:55:230
3720676991513,2cyclictest60350irq/132-enp0s29f1:rx-010:53:090
3720676991513,1cyclictest60350irq/132-enp0s29f1:rx-012:20:240
3720676991513,1cyclictest60350irq/132-enp0s29f1:rx-011:37:500
3720676991512,2cyclictest60350irq/132-enp0s29f1:rx-009:40:170
3720676991511,3cyclictest3898020-21kworker/u8:1+events_unbound11:00:190
3720676991511,3cyclictest3803798-21kworker/u8:0+events_unbound08:55:360
3720685991411,2cyclictest3794215-21irqcore08:20:173
3720685991411,2cyclictest3705376-21kworker/u8:1+flush-259:007:40:203
372068499149,4cyclictest3814279-21kworker/u8:2+flush-259:008:55:262
3720684991412,1cyclictest4001540-21kworker/u8:2+events_unbound12:15:152
3720684991411,2cyclictest3819891-21fschecks_count08:45:152
3720684991411,2cyclictest3747877-21irqcore07:35:162
3720679991412,2cyclictest3934119-21kworker/u8:3+events_unbound10:45:181
3720679991410,3cyclictest3803798-21kworker/u8:0+events_unbound09:39:091
3720676991410,3cyclictest3865826-21kworker/u8:2+events_unbound10:17:340
372068599139,3cyclictest3768057-21kworker/u8:0+flush-259:008:05:413
3720685991310,2cyclictest4001540-21kworker/u8:2+events_unbound12:00:173
3720685991310,2cyclictest3980995-21kworker/u8:4+events_unbound11:21:113
372068499139,3cyclictest3865826-21kworker/u8:2+flush-259:010:05:182
372068499139,3cyclictest3705376-21kworker/u8:1+flush-259:007:57:052
372068499136,5cyclictest335557-21systemd-journal09:24:592
3720684991310,2cyclictest4022806-21sed12:00:142
3720684991310,2cyclictest3925242-21kthreadcore10:25:162
3720684991310,2cyclictest3845813-21irqcore09:10:172
3720676991312,1cyclictest60350irq/132-enp0s29f1:rx-009:17:040
372068599129,2cyclictest3705376-21kworker/u8:1+events_unbound08:15:283
372068499129,2cyclictest503-21tpm2-abrmd11:45:002
372068499129,2cyclictest0-21swapper/212:34:592
372068499129,2cyclictest0-21swapper/207:25:002
372068499128,3cyclictest3959821-21kworker/u8:0+flush-259:012:20:382
372068499128,3cyclictest3768057-21kworker/u8:0+flush-259:008:05:182
372068499116,3cyclictest335557-21systemd-journal12:00:002
372068499115,4cyclictest3939394-21basename10:40:012
372068499110,2cyclictest3954836-21latency_hist10:55:012
372068499108,1cyclictest0-21swapper/209:45:202
372068499107,1cyclictest0-21swapper/209:30:132
372068499106,2cyclictest4062530-21ntpq12:35:232
372068499101,8cyclictest3757497-21cron07:45:002
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional