You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-03 - 01:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Fri Jan 02, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2431993835,2cyclictest219697-21kworker/u8:3+events_unbound11:15:251
2431993833,4cyclictest167041-21ntpq09:45:231
2431993734,2cyclictest198858-21kworker/u8:0+events_unbound10:50:251
2431993633,2cyclictest4191056-21kworker/u8:2+events_unbound07:15:151
2431993633,2cyclictest183329-21kworker/u8:2+events_unbound10:25:211
2431993630,2cyclictest219697-21kworker/u8:3+events_unbound12:20:151
2439993531,3cyclictest85547-21kworker/u8:2+flush-259:009:00:223
2431993532,2cyclictest225065-21kworker/u8:4+events_unbound11:20:231
2431993532,2cyclictest225065-21kworker/u8:4+events_unbound11:10:241
2431993532,2cyclictest219697-21kworker/u8:3+events_unbound12:15:251
2435993430,3cyclictest17552-21kworker/u8:3+flush-259:007:55:152
2431993432,2cyclictest147312-21kworker/u8:3+events_unbound09:50:141
2431993431,2cyclictest48287-21kworker/u8:2+events_unbound08:00:141
2431993431,2cyclictest219697-21kworker/u8:3+events_unbound12:25:141
2431993431,2cyclictest198858-21kworker/u8:0+events_unbound10:40:241
2431993431,2cyclictest17552-21kworker/u8:3+events_unbound07:30:261
2431993330,2cyclictest4163856-21kworker/u8:0+flush-259:007:20:221
2431993330,2cyclictest100892-21kworker/u8:5+events_unbound09:10:271
2431993328,4cyclictest64405-21gpgv08:10:001
2429993328,3cyclictest137381-21diskstats09:20:130
242999330,32cyclictest1-21systemd08:25:010
2429993229,2cyclictest85547-21kworker/u8:2+flush-259:008:55:000
2435993128,2cyclictest17552-21kworker/u8:3+flush-259:009:10:002
2431993128,2cyclictest85547-21kworker/u8:2+events_unbound09:05:231
2431993128,2cyclictest286752-21kworker/u8:0+events_unbound12:35:231
2431993128,2cyclictest183329-21kworker/u8:2+events_unbound10:20:001
2431993128,2cyclictest147312-21kworker/u8:3+events_unbound09:35:171
2429993124,5cyclictest254859-21ntpq11:10:220
2435993025,3cyclictest132054-21/usr/sbin/munin09:15:242
2431993027,2cyclictest64943-21kworker/u8:0+events_unbound08:25:161
2431993027,2cyclictest219697-21kworker/u8:3+events_unbound11:45:231
2429993026,3cyclictest4191056-21kworker/u8:2+flush-259:007:15:160
2429993026,3cyclictest225065-21kworker/u8:4+flush-259:011:05:210
2439992926,2cyclictest17552-21kworker/u8:3+flush-259:008:15:263
2439992924,4cyclictest225074-21/usr/sbin/munin10:45:143
2431992926,2cyclictest23227-21kworker/u8:4+flush-259:007:40:161
2431992926,2cyclictest225065-21kworker/u8:4+events_unbound11:40:221
2431992926,2cyclictest198858-21kworker/u8:0+events_unbound10:35:161
2429992825,2cyclictest85547-21kworker/u8:2+flush-259:008:30:150
2429992823,3cyclictest168130-21/usr/sbin/munin09:50:150
2431992724,2cyclictest17552-21kworker/u8:3+events_unbound07:25:121
2431992724,2cyclictest131840-21kworker/u8:0+events_unbound09:25:221
2431992724,2cyclictest131840-21kworker/u8:0+events_unbound09:25:221
2429992723,3cyclictest286752-21kworker/u8:0+flush-259:012:35:170
2431992623,2cyclictest147312-21kworker/u8:3+events_unbound09:55:171
2431992623,2cyclictest147312-21kworker/u8:3+events_unbound09:30:231
2431992623,2cyclictest100892-21kworker/u8:5+events_unbound08:45:201
2429992622,3cyclictest286752-21kworker/u8:0+flush-259:012:10:150
2431992523,2cyclictest286752-21kworker/u8:0+events_unbound12:10:271
2429992522,2cyclictest23227-21kworker/u8:4+events_unbound07:35:230
2429992521,3cyclictest286752-21kworker/u8:0+events_unbound12:00:180
2429992521,3cyclictest17552-21kworker/u8:3+flush-259:007:25:220
2429992521,3cyclictest17552-21kworker/u8:3+events_unbound07:55:200
2429992521,3cyclictest147312-21kworker/u8:3+events_unbound10:05:130
2439992420,3cyclictest198858-21kworker/u8:0+events_unbound10:50:153
2439992418,4cyclictest188469-21latency_hist10:10:013
2435992420,3cyclictest286752-21kworker/u8:0+flush-259:012:00:162
2435992419,4cyclictest178190-21latency_hist10:00:012
2431992421,2cyclictest239958-21kworker/u8:1+events_unbound11:00:201
2431992421,2cyclictest219697-21kworker/u8:3+events_unbound11:35:261
2431992420,3cyclictest183329-21kworker/u8:2+events_unbound10:20:121
2429992421,2cyclictest219697-21kworker/u8:3+events_unbound11:25:220
2429992421,2cyclictest100892-21kworker/u8:5+flush-259:009:05:200
2439992319,3cyclictest147312-21kworker/u8:3+flush-259:009:50:263
2435992319,3cyclictest225065-21kworker/u8:4+flush-259:011:05:252
2431992321,2cyclictest147312-21kworker/u8:3+flush-259:010:05:271
2431992320,2cyclictest23227-21kworker/u8:4+events_unbound07:36:411
2429992320,2cyclictest48287-21kworker/u8:2+events_unbound08:00:160
2429992320,2cyclictest183329-21kworker/u8:2+events_unbound10:25:130
2429992319,3cyclictest64974-21apt-get08:10:100
2429992318,3cyclictest307948-21/usr/sbin/munin12:05:140
2429992318,3cyclictest297101-21gpgv11:55:000
2439992218,3cyclictest239958-21kworker/u8:1+flush-259:011:00:213
2431992218,3cyclictest219697-21kworker/u8:3+flush-259:011:34:561
2431992218,2cyclictest160146-21kthreadcore09:40:181
2429992218,3cyclictest147312-21kworker/u8:3+events_unbound09:55:230
2429992218,3cyclictest100892-21kworker/u8:5+events_unbound08:45:220
2439992117,3cyclictest131840-21kworker/u8:0+flush-259:009:45:163
2439992117,2cyclictest147312-21kworker/u8:3+events_unbound10:00:193
2439992115,4cyclictest252-21systemd-journal11:20:393
2435992117,3cyclictest131840-21kworker/u8:0+flush-259:009:40:242
2435992117,3cyclictest100892-21kworker/u8:5+flush-259:009:10:292
2435992114,5cyclictest252-21systemd-journal08:30:012
2431992118,2cyclictest225065-21kworker/u8:4+events_unbound10:45:141
2431992116,2cyclictest80201-21kworker/u8:1+events_unbound08:40:251
2429992118,2cyclictest23227-21kworker/u8:4+flush-259:007:40:210
2429992117,3cyclictest219697-21kworker/u8:3+events_unbound11:40:130
2429992116,4cyclictest240347-21latency_hist11:00:010
2439992017,2cyclictest142118-21sh09:25:003
2439992016,3cyclictest286752-21kworker/u8:0+flush-259:012:10:263
2439992015,3cyclictest173288-21/usr/sbin/munin09:55:133
2439992014,4cyclictest271464-21/usr/sbin/munin11:30:153
2439992014,4cyclictest111210-21latency_hist08:54:593
2431992017,2cyclictest286752-21kworker/u8:0+events_unbound12:34:401
2431992017,2cyclictest286752-21kworker/u8:0+events_unbound12:01:401
2431992015,4cyclictest219697-21kworker/u8:3+events_unbound11:55:011
2431992015,4cyclictest219697-21kworker/u8:3+events_unbound11:55:011
2429992016,3cyclictest80201-21kworker/u8:1+events_unbound08:40:150
2429992016,3cyclictest147312-21kworker/u8:3+events_unbound09:35:200
2429992015,3cyclictest225065-21kworker/u8:4+flush-259:011:15:200
2429992013,5cyclictest6691-21ntpq07:10:220
2439991915,3cyclictest509-21rs:main3
2439991914,3cyclictest235442-21apt-get10:55:113
2439991913,5cyclictest204266-21latency_hist10:25:013
2435991916,2cyclictest69863-21idleruntime-cro08:15:012
2435991916,2cyclictest23227-21kworker/u8:4+flush-259:007:35:212
2431991915,3cyclictest17552-21kworker/u8:3+flush-259:007:50:401
2429991916,2cyclictest85547-21kworker/u8:2+events_unbound08:35:210
2429991916,2cyclictest219697-21kworker/u8:3+events_unbound11:35:280
242999190,17cyclictest1-21systemd09:05:000
2439991815,2cyclictest64488-21gpgv08:10:013
243599182,10cyclictest338611-21munin-run12:35:002
2435991815,2cyclictest198858-21kworker/u8:0+events_unbound10:30:192
2435991814,3cyclictest4163856-21kworker/u8:0+flush-259:007:20:212
2435991814,3cyclictest239958-21kworker/u8:1+flush-259:011:00:272
2431991816,2cyclictest38712-21kworker/u8:1+events_unbound07:45:291
2431991815,2cyclictest85547-21kworker/u8:2+events_unbound09:15:141
2429991815,2cyclictest225065-21kworker/u8:4+flush-259:010:45:260
2429991815,2cyclictest167915-21kworker/u8:1+flush-259:010:15:150
2429991814,3cyclictest85547-21kworker/u8:2+events_unbound09:14:270
2429991814,2cyclictest0-21swapper/007:50:000
2429991811,5cyclictest193894-21base6410:15:010
2439991715,1cyclictest219697-21kworker/u8:3+events_unbound12:20:213
2439991714,2cyclictest85547-21kworker/u8:2+flush-259:008:35:013
2439991713,3cyclictest225065-21kworker/u8:4+flush-259:011:40:243
2439991713,3cyclictest219697-21kworker/u8:3+flush-259:012:10:003
2439991712,3cyclictest286752-21kworker/u8:0+flush-259:012:35:203
2439991710,5cyclictest252-21systemd-journal10:43:403
2435991713,3cyclictest199131-21cron10:20:002
2435991713,3cyclictest100892-21kworker/u8:5+flush-259:008:55:272
2435991713,2cyclictest0-21swapper/208:50:002
2435991710,5cyclictest167889-21latency_hist09:50:012
243199175,1cyclictest191rcu_preempt08:15:261
2431991714,2cyclictest17552-21kworker/u8:3+events_unbound08:55:221
2431991713,3cyclictest4163856-21kworker/u8:0+events_unbound07:14:411
2429991714,2cyclictest64943-21kworker/u8:0+flush-259:008:20:000
2429991714,2cyclictest323164-21cron12:20:000
2429991714,2cyclictest198858-21kworker/u8:0+flush-259:010:35:160
2429991713,3cyclictest4163856-21kworker/u8:0+events_unbound07:20:180
2429991711,4cyclictest333711-21/usr/sbin/munin12:30:140
2429991711,4cyclictest297537-21cp11:55:010
2429991711,4cyclictest297537-21cp11:55:000
2439991613,2cyclictest17552-21kworker/u8:3+flush-259:007:40:013
2439991611,3cyclictest90736-21/usr/sbin/munin08:35:133
2439991610,4cyclictest302598-21latency_hist12:00:003
2439991610,4cyclictest302598-21latency_hist12:00:003
2435991613,2cyclictest28157-21cron07:35:002
2435991612,3cyclictest85547-21kworker/u8:2+flush-259:008:35:222
2435991610,4cyclictest209416-21latency_hist10:30:012
2435991610,4cyclictest142125-21latency_hist09:25:012
2431991613,2cyclictest286752-21kworker/u8:0+events_unbound11:50:251
2431991613,2cyclictest239958-21kworker/u8:1+events_unbound10:57:421
2431991613,2cyclictest167915-21kworker/u8:1+flush-259:010:10:251
2431991612,3cyclictest271214-21sh11:30:001
2429991614,1cyclictest60350irq/132-enp0s29f1:rx-010:00:230
2429991613,2cyclictest60350irq/132-enp0s29f1:rx-012:20:180
2429991613,2cyclictest131840-21kworker/u8:0+flush-259:009:25:150
2429991613,2cyclictest131840-21kworker/u8:0+flush-259:009:25:140
2439991510,4cyclictest80201-21kworker/u8:1+flush-259:008:40:133
2435991512,2cyclictest257065-21kthreadcore11:15:192
2435991512,2cyclictest204264-21idleruntime-cro10:25:002
2435991511,3cyclictest219697-21kworker/u8:3+flush-259:012:05:252
2435991511,3cyclictest17552-21kworker/u8:3+events_unbound07:50:192
243199153,2cyclictest191rcu_preempt07:55:141
2431991513,2cyclictest85547-21kworker/u8:2+events_unbound08:35:251
2431991512,2cyclictest219697-21kworker/u8:3+flush-259:012:09:551
2431991512,2cyclictest17552-21kworker/u8:3+events_unbound09:00:051
2429991513,1cyclictest60350irq/132-enp0s29f1:rx-011:21:480
2429991512,2cyclictest60350irq/132-enp0s29f1:rx-011:30:150
2429991512,2cyclictest60350irq/132-enp0s29f1:rx-011:00:110
2429991512,2cyclictest60350irq/132-enp0s29f1:rx-008:05:170
2429991512,2cyclictest60350irq/132-enp0s29f1:rx-007:50:210
2429991510,3cyclictest224841-21latency_hist10:45:010
243999149,3cyclictest100892-21kworker/u8:5+flush-259:008:55:183
243599147,5cyclictest42737-21ntpq07:45:242
2435991412,1cyclictest0-21swapper/210:55:012
2435991411,2cyclictest74560-21unixbench_multi08:15:302
243199148,1cyclictest191rcu_preempt08:35:011
2431991412,2cyclictest100892-21kworker/u8:5+events_unbound08:54:241
2431991411,2cyclictest198858-21kworker/u8:0+events_unbound10:31:421
2431991411,2cyclictest17552-21kworker/u8:3+flush-259:008:20:211
242999149,4cyclictest60350irq/132-enp0s29f1:rx-009:15:140
2429991412,1cyclictest60350irq/132-enp0s29f1:rx-010:20:250
2429991411,2cyclictest60350irq/132-enp0s29f1:rx-010:30:180
2429991411,2cyclictest60350irq/132-enp0s29f1:rx-008:25:190
2429991411,2cyclictest131840-21kworker/u8:0+flush-259:009:45:200
243999139,3cyclictest48287-21kworker/u8:2+events_unbound08:10:203
243999139,3cyclictest38482-21apt07:44:593
243999138,4cyclictest64943-21kworker/u8:0+flush-259:008:25:253
243999138,4cyclictest245747-21/usr/sbin/munin11:05:163
243999135,5cyclictest297171-21cp11:55:003
2439991311,1cyclictest0-21swapper/309:45:003
2439991311,1cyclictest0-21swapper/309:15:203
2439991310,2cyclictest7862-21cpuspeed_turbos07:15:133
2439991310,2cyclictest48375-21unixbench_singl07:50:323
2439991310,2cyclictest0-21swapper/311:50:003
243599139,3cyclictest225065-21kworker/u8:4+flush-259:011:10:252
2435991310,2cyclictest58704-21unixbench_singl08:00:252
2435991310,2cyclictest33649-21cpuspeed_turbos07:40:122
2435991310,2cyclictest219697-21kworker/u8:3+flush-259:011:45:192
243599130,4cyclictest281551-21munin-run11:40:002
243599130,11cyclictest1-21systemd10:55:402
243199137,1cyclictest191rcu_preempt11:05:181
2431991310,2cyclictest137187-21kworker/u8:1+events_unbound09:20:481
2429991311,1cyclictest60350irq/132-enp0s29f1:rx-011:45:280
2429991311,1cyclictest60350irq/132-enp0s29f1:rx-010:50:150
2429991311,1cyclictest60350irq/132-enp0s29f1:rx-008:57:040
2429991310,2cyclictest60350irq/132-enp0s29f1:rx-012:29:230
2429991310,2cyclictest60350irq/132-enp0s29f1:rx-007:30:310
2429991310,2cyclictest147312-21kworker/u8:3+events_unbound09:40:150
243999129,2cyclictest22522-21unixbench_multi07:25:253
243999129,2cyclictest149822-21tr09:30:193
243999129,2cyclictest126224-21unixbench_multi09:05:233
243999128,3cyclictest17552-21kworker/u8:3+flush-259:007:55:233
243999127,4cyclictest131840-21kworker/u8:0+flush-259:009:25:553
243999127,4cyclictest131840-21kworker/u8:0+flush-259:009:25:553
243999120,4cyclictest50792cut07:10:193
243599128,3cyclictest48287-21kworker/u8:2+flush-259:008:05:172
243599128,3cyclictest2424-21cyclictest09:55:002
243599123,3cyclictest276392-21basename11:35:012
243999118,2cyclictest500-21systemd-logind10:20:003
243999118,2cyclictest0-21swapper/307:50:013
243999117,3cyclictest17737-21sshd07:24:033
243599118,2cyclictest318318-21cpuspeed_turbos12:15:142
243599118,2cyclictest106726-21grep08:50:172
243599117,3cyclictest225065-21kworker/u8:4+flush-259:010:45:302
243599115,4cyclictest188460-21basename10:10:002
243599113,7cyclictest286752-21kworker/u8:0+flush-259:011:55:002
243599113,2cyclictest12705-21munin-run07:20:002
243199113,2cyclictest32-21ksoftirqd/107:10:011
243999108,1cyclictest0-21swapper/310:30:153
243999107,2cyclictest127036-21cpuspeed_turbos09:10:173
243999105,3cyclictest256811-21sed11:15:193
243999101,8cyclictest0-21swapper/311:15:003
243599108,1cyclictest0-21swapper/212:30:012
243599106,3cyclictest80203-21pluginstate08:25:002
243599106,3cyclictest193051-21sed10:10:242
243599106,3cyclictest1-21systemd09:35:002
243599106,2cyclictest2611642sleep211:20:122
243599106,2cyclictest0-21swapper/207:25:172
243599104,4cyclictest317971-21cron12:15:012
243599103,3cyclictest183314-21munin-run10:05:002
24399994,4cyclictest54584-21smartctl08:00:153
24359996,2cyclictest266416-21cpuspeed_turbos11:25:132
24359996,1cyclictest0-21swapper/208:40:242
24359995,3cyclictest90590-21sed08:35:012
24359995,3cyclictest401ktimers/212:20:122
24359995,2cyclictest343738-21awk12:40:002
24359994,3cyclictest300610-21cut11:55:202
24359994,3cyclictest300610-21cut11:55:192
24359994,3cyclictest121055-21awk09:00:252
24359990,2cyclictest219684-21munin-plugin-st10:40:002
24359984,3cyclictest143040-21sed09:25:182
24359984,3cyclictest143040-21sed09:25:172
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional