You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-17 - 21:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Wed Dec 17, 2025 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3270536993024,4cyclictest3592050-21latency_hist12:20:011
3270537992923,4cyclictest3322134-21latency_hist08:00:002
3270532992926,2cyclictest3327361-21kworker/u8:0+events_unbound08:20:190
3270537992822,5cyclictest252-21systemd-journal07:55:012
3270532992825,2cyclictest3250162-21kworker/u8:4+flush-259:007:45:230
3270532992821,5cyclictest3503595-21ntpq10:50:210
3270539992723,3cyclictest3327361-21kworker/u8:0+flush-259:008:10:593
3270537992725,2cyclictest3482969-21kworker/u8:2+events_unbound10:47:562
3270537992722,4cyclictest3488511-21latency_hist10:40:002
3270536992724,2cyclictest3592084-21kworker/u8:2+flush-259:012:30:001
3270536992723,3cyclictest3494364-21kworker/u8:3+events_unbound11:10:131
3270532992724,2cyclictest3529698-21kworker/u8:2+events_unbound11:20:240
3270532992724,2cyclictest3462520-21kworker/u8:4+flush-259:010:25:200
3270539992620,4cyclictest252-21systemd-journal08:59:593
3270536992621,3cyclictest3447088-21kworker/u8:1+flush-259:010:30:011
3270536992621,3cyclictest3343214-21latency_hist08:20:021
3270532992622,3cyclictest3519943-21latency_hist11:10:000
3270532992522,2cyclictest3529208-21kworker/u8:1+events_unbound11:55:200
3270532992522,2cyclictest3250162-21kworker/u8:4+events_unbound09:20:150
3270539992421,2cyclictest3545715-21kworker/u8:2+flush-259:011:39:593
3270539992421,2cyclictest3327361-21kworker/u8:0+events_unbound08:05:213
3270537992421,2cyclictest3327361-21kworker/u8:0+events_unbound08:34:572
3270537992416,5cyclictest3306597-21latency_hist07:45:002
3270536992421,2cyclictest3545715-21kworker/u8:2+flush-259:011:40:211
3270532992421,2cyclictest3494364-21kworker/u8:3+events_unbound11:15:170
3270532992421,2cyclictest3420848-21kworker/u8:3+flush-259:010:20:220
3270539992319,3cyclictest3374168-21latency_hist08:50:013
3270537992316,5cyclictest3612609-21latency_hist12:40:012
3270536992320,2cyclictest3415063-21kworker/u8:0+flush-259:009:30:001
3270536992320,2cyclictest3415063-21kworker/u8:0+flush-259:009:29:591
3270532992320,2cyclictest3529208-21kworker/u8:1+flush-259:011:50:220
3270532992320,2cyclictest3384517-21kworker/u8:2+events_unbound09:15:170
3270532992319,3cyclictest3535399-21kworker/u8:0+flush-259:011:30:210
3270532992319,3cyclictest3369031-21kworker/u8:0+events_unbound09:10:020
3270532992316,2cyclictest3462520-21kworker/u8:4+events_unbound10:55:140
3270532992313,8cyclictest3369021-21idleruntime-cro08:45:000
3270536992219,2cyclictest3210982-21kworker/u8:0+flush-259:007:15:001
3270536992216,4cyclictest3529380-21ntpq11:15:221
3270532992219,2cyclictest3358911-21kworker/u8:3+flush-259:008:50:200
3270532992218,3cyclictest3400242-21kworker/u8:1+flush-259:009:45:240
3270532992218,3cyclictest3353760-21apt-get08:30:120
3270532992218,3cyclictest3210982-21kworker/u8:0+events_unbound07:10:320
3270539992118,2cyclictest3270488-21kworker/u8:2+flush-259:007:55:153
3270537992118,2cyclictest3358911-21kworker/u8:3+events_unbound09:09:552
3270537992112,6cyclictest252-21systemd-journal08:15:002
3270536992116,4cyclictest3420623-21latency_hist09:35:011
3270532992115,4cyclictest3446405-21latency_hist10:00:010
3270539992017,2cyclictest3545715-21kworker/u8:2+flush-259:012:10:003
3270539992017,2cyclictest3384517-21kworker/u8:2+flush-259:009:20:003
3270539992017,2cyclictest3270488-21kworker/u8:2+events_unbound07:35:223
3270539992017,2cyclictest3250162-21kworker/u8:4+events_unbound07:15:213
3270537992017,3cyclictest3250162-21kworker/u8:4+events_unbound07:27:532
3270537992017,2cyclictest3545715-21kworker/u8:2+events_unbound11:48:412
3270537992017,2cyclictest3545715-21kworker/u8:2+events_unbound11:48:412
3270537992017,2cyclictest3447088-21kworker/u8:1+events_unbound10:30:022
3270537992014,4cyclictest3504687-21/usr/sbin/munin10:55:232
3270532992017,2cyclictest3581745-21kworker/u8:3+events_unbound12:30:230
3270532992017,2cyclictest3447088-21kworker/u8:1+flush-259:010:15:010
3270532992017,2cyclictest3270488-21kworker/u8:2+flush-259:007:55:230
3270532992017,2cyclictest3270488-21kworker/u8:2+flush-259:007:54:270
3270532992017,2cyclictest3270488-21kworker/u8:2+flush-259:007:35:230
3270532992017,2cyclictest3250162-21kworker/u8:4+events_unbound07:30:130
3270539991916,2cyclictest3447088-21kworker/u8:1+flush-259:010:15:253
3270539991916,2cyclictest3301485-21kworker/u8:1+flush-259:007:45:013
3270537991916,2cyclictest3250162-21kworker/u8:4+events_unbound07:33:472
3270536991915,3cyclictest3358911-21kworker/u8:3+events_unbound08:40:221
327053299190,17cyclictest3494126-21unin-run10:45:010
3270539991815,2cyclictest3529208-21kworker/u8:1+flush-259:011:25:203
3270539991815,2cyclictest3430951-21kworker/u8:0+events_unbound09:45:223
3270539991815,2cyclictest3316866-40mandb07:54:283
327053799189,6cyclictest252-21systemd-journal08:10:012
3270537991815,2cyclictest3529208-21kworker/u8:1+events_unbound11:16:262
3270537991815,2cyclictest3499936-21irqcore10:50:152
3270537991814,2cyclictest2971-21cron08:30:002
3270536991816,1cyclictest3250162-21kworker/u8:4+events_unbound09:00:161
3270536991815,2cyclictest3494364-21kworker/u8:3+events_unbound11:05:211
3270536991815,2cyclictest3447088-21kworker/u8:1+flush-259:010:19:571
3270536991814,3cyclictest3250162-21kworker/u8:4+events_unbound07:20:241
3270532991815,2cyclictest3415063-21kworker/u8:0+flush-259:009:30:160
3270532991815,2cyclictest3358911-21kworker/u8:3+events_unbound08:35:230
3270532991815,2cyclictest3250162-21kworker/u8:4+events_unbound08:55:070
3270532991815,2cyclictest3250162-21kworker/u8:4+events_unbound08:15:150
327053299181,16cyclictest3327355-21unin-run08:05:010
3270539991714,2cyclictest3509612-21kworker/u8:0+flush-259:011:00:183
3270539991714,2cyclictest0-21swapper/310:40:013
327053799179,5cyclictest3514757-21latency_hist11:05:002
3270537991714,2cyclictest3545715-21kworker/u8:2+events_unbound12:03:342
3270537991714,2cyclictest3514984-21kworker/u8:1+events_unbound11:07:122
3270537991714,2cyclictest3441254-21kworker/u8:2+events_unbound10:05:102
3270537991710,5cyclictest3535413-21latency_hist11:25:002
3270536991714,2cyclictest3509612-21kworker/u8:0+flush-259:011:04:021
3270536991714,2cyclictest3327361-21kworker/u8:0+flush-259:008:30:171
3270536991711,4cyclictest3488700-21cp10:40:001
3270532991714,2cyclictest3462520-21kworker/u8:4+flush-259:010:35:130
3270532991714,2cyclictest3420640-21kworker/u8:2+events_unbound09:40:200
3270532991714,2cyclictest3400242-21kworker/u8:1+flush-259:009:25:260
3270532991714,2cyclictest3400242-21kworker/u8:1+flush-259:009:25:250
3270532991714,2cyclictest3250162-21kworker/u8:4+events_unbound07:20:130
3270532991713,3cyclictest3581745-21kworker/u8:3+events_unbound12:36:480
3270532991712,3cyclictest3555994-21kworker/u8:3+flush-259:012:00:160
3270539991614,1cyclictest0-21swapper/309:15:003
3270539991613,2cyclictest3581745-21kworker/u8:3+events_unbound12:13:463
3270539991613,2cyclictest3482969-21kworker/u8:2+flush-259:010:50:243
3270539991613,2cyclictest3420640-21kworker/u8:2+flush-259:009:40:213
3270539991612,3cyclictest3420848-21kworker/u8:3+flush-259:009:51:583
3270539991612,2cyclictest3555994-21kworker/u8:3+flush-259:012:00:183
327053799169,5cyclictest3489176-21munin-node10:40:122
327053799169,5cyclictest252-21systemd-journal11:40:002
327053799168,2cyclictest41-21ksoftirqd/211:30:002
327053799163,5cyclictest3462299-21munin-run10:15:002
3270537991615,1cyclictest3250162-21kworker/u8:4+events_unbound07:13:122
3270537991614,2cyclictest3250162-21kworker/u8:4+events_unbound08:36:532
3270537991613,2cyclictest3581745-21kworker/u8:3+events_unbound12:13:452
3270537991613,2cyclictest3420848-21kworker/u8:3+events_unbound09:41:422
3270537991613,2cyclictest3270488-21kworker/u8:2+events_unbound08:00:532
3270537991613,2cyclictest3250162-21kworker/u8:4+events_unbound09:13:342
3270536991614,2cyclictest3576304-21kworker/u8:0+flush-259:012:20:181
3270536991613,2cyclictest3529698-21kworker/u8:2+events_unbound11:20:221
3270536991613,2cyclictest3441254-21kworker/u8:2+flush-259:010:04:011
3270536991613,2cyclictest3420640-21kworker/u8:2+flush-259:009:40:201
3270536991613,2cyclictest3400242-21kworker/u8:1+flush-259:009:35:161
3270536991612,3cyclictest3504712-21apt-get10:55:131
3270536991612,3cyclictest3343237-21kworker/u8:1+events_unbound08:29:571
3270532991614,1cyclictest60350irq/132-enp0s29f1:rx-009:51:030
3270532991613,2cyclictest3509612-21kworker/u8:0+flush-259:011:00:190
3270532991613,2cyclictest3483187-21latency_hist10:35:010
3270532991613,2cyclictest3447088-21kworker/u8:1+events_unbound10:15:230
3270532991613,2cyclictest3358911-21kworker/u8:3+events_unbound09:07:570
3270532991613,2cyclictest3250162-21kworker/u8:4+events_unbound09:00:220
3270532991612,3cyclictest3250162-21kworker/u8:4+flush-259:008:10:110
3270532991612,3cyclictest3250162-21kworker/u8:4+events_unbound07:40:120
3270532991611,4cyclictest3587143-21apt-get12:15:100
3270539991513,1cyclictest3597835-21irqcore12:25:173
3270539991513,1cyclictest0-21swapper/312:35:003
3270539991512,2cyclictest3576304-21kworker/u8:0+flush-259:012:35:233
3270539991512,2cyclictest3535399-21kworker/u8:0+events_unbound11:30:233
3270539991512,2cyclictest3514984-21kworker/u8:1+flush-259:011:05:193
3270539991512,2cyclictest3462520-21kworker/u8:4+events_unbound11:00:003
3270539991512,2cyclictest3400242-21kworker/u8:1+flush-259:009:35:233
3270539991511,3cyclictest3415738-21/usr/sbin/munin09:30:133
327053799159,4cyclictest3270488-21kworker/u8:2+flush-259:007:35:182
327053799158,5cyclictest3383646-21ntpq08:55:232
327053799158,4cyclictest0-21swapper/208:22:352
327053799156,1cyclictest191rcu_preempt10:20:202
3270537991513,2cyclictest3358911-21kworker/u8:3+events_unbound08:51:082
3270537991513,2cyclictest3358911-21kworker/u8:3+events_unbound08:49:202
3270537991513,2cyclictest3270488-21kworker/u8:2+events_unbound07:21:372
3270537991512,2cyclictest3529208-21kworker/u8:1+events_unbound11:56:362
3270537991512,2cyclictest3529208-21kworker/u8:1+events_unbound11:30:042
3270537991512,2cyclictest3473281-21sed10:25:142
3270537991512,2cyclictest3420848-21kworker/u8:3+events_unbound09:51:402
3270537991512,2cyclictest3415063-21kworker/u8:0+events_unbound09:31:312
3270537991512,2cyclictest3358911-21kworker/u8:3+events_unbound08:44:512
3270537991512,2cyclictest3250162-21kworker/u8:4+events_unbound09:23:252
3270536991513,2cyclictest3327361-21kworker/u8:0+events_unbound08:07:561
3270536991513,1cyclictest3250162-21kworker/u8:4+events_unbound07:15:171
3270536991512,2cyclictest3576819-21apt-get12:05:121
3270536991512,2cyclictest3447088-21kworker/u8:1+flush-259:010:20:151
3270536991511,3cyclictest3545929-21/usr/sbin/munin11:35:241
3270532991513,1cyclictest60350irq/132-enp0s29f1:rx-011:45:240
3270532991513,1cyclictest60350irq/132-enp0s29f1:rx-011:45:240
3270532991513,1cyclictest60350irq/132-enp0s29f1:rx-010:00:150
3270532991513,1cyclictest60350irq/132-enp0s29f1:rx-007:25:230
3270532991512,2cyclictest3494364-21kworker/u8:3+events_unbound11:10:150
3270532991512,2cyclictest3462520-21kworker/u8:4+events_unbound10:45:560
3270532991512,2cyclictest3250162-21kworker/u8:4+flush-259:007:15:240
3270539991412,1cyclictest0-21swapper/311:15:003
3270539991411,2cyclictest3498795-21unixbench_multi10:45:243
3270539991411,2cyclictest3472859-21apt-get10:25:093
3270539991411,2cyclictest3358911-21kworker/u8:3+events_unbound08:54:573
3270539991411,2cyclictest3250162-21kworker/u8:4+flush-259:009:29:573
3270539991411,2cyclictest3250162-21kworker/u8:4+flush-259:009:29:573
3270539991411,2cyclictest3250162-21kworker/u8:4+events_unbound08:15:213
3270539991410,3cyclictest3462520-21kworker/u8:4+flush-259:010:30:313
327053799144,3cyclictest41-21ksoftirqd/212:25:002
327053799144,2cyclictest0-21swapper/212:25:152
327053799143,3cyclictest41-21ksoftirqd/212:05:172
3270537991412,2cyclictest3581745-21kworker/u8:3+events_unbound12:16:402
3270537991412,2cyclictest3400242-21kworker/u8:1+events_unbound09:16:472
3270537991411,2cyclictest391rcuc/209:45:172
3270537991411,2cyclictest3529208-21kworker/u8:1+events_unbound11:42:542
3270537991411,2cyclictest3441254-21kworker/u8:2+events_unbound09:57:502
3270537991411,2cyclictest3415501-21cron09:30:002
3270537991411,2cyclictest3415501-21cron09:30:002
3270537991411,2cyclictest3270488-21kworker/u8:2+events_unbound07:18:042
3270537991411,1cyclictest191rcu_preempt09:01:392
3270537991410,1cyclictest191rcu_preempt10:15:162
327053699149,4cyclictest3457578-21diskstats10:10:141
3270536991412,2cyclictest3581745-21kworker/u8:3+flush-259:012:15:001
3270536991412,2cyclictest3400242-21kworker/u8:1+events_unbound09:48:581
3270536991412,1cyclictest0-21swapper/110:10:011
3270536991411,2cyclictest3555994-21kworker/u8:3+flush-259:011:51:321
3270536991411,2cyclictest3369031-21kworker/u8:0+events_unbound09:10:181
3270536991411,2cyclictest3358911-21kworker/u8:3+flush-259:009:05:201
3270536991411,2cyclictest3270488-21kworker/u8:2+flush-259:008:03:001
3270536991411,1cyclictest3574495-21kthreadcore12:00:181
3270536991410,3cyclictest3400242-21kworker/u8:1+events_unbound09:53:571
3270536991410,3cyclictest252-21systemd-journal08:10:561
3270532991412,1cyclictest60350irq/132-enp0s29f1:rx-011:37:120
3270532991412,1cyclictest60350irq/132-enp0s29f1:rx-011:25:200
3270532991412,1cyclictest60350irq/132-enp0s29f1:rx-009:35:220
3270532991412,1cyclictest60350irq/132-enp0s29f1:rx-008:29:470
3270532991411,2cyclictest60350irq/132-enp0s29f1:rx-012:20:130
3270532991411,2cyclictest60350irq/132-enp0s29f1:rx-012:14:290
3270532991411,2cyclictest60350irq/132-enp0s29f1:rx-012:05:120
3270532991411,2cyclictest3529208-21kworker/u8:1+flush-259:011:40:180
327053999139,3cyclictest3343237-21kworker/u8:1+flush-259:008:30:133
3270539991311,1cyclictest3494364-21kworker/u8:3+events_unbound11:17:573
3270539991310,2cyclictest3576304-21kworker/u8:0+events_unbound12:20:143
3270539991310,2cyclictest3447088-21kworker/u8:1+events_unbound10:10:203
3270539991310,2cyclictest3420848-21kworker/u8:3+events_unbound10:03:203
3270539991310,2cyclictest3250162-21kworker/u8:4+flush-259:008:35:493
3270539991310,1cyclictest0-21swapper/307:25:003
327053799139,1cyclictest191rcu_preempt11:52:522
327053799135,1cyclictest191rcu_preempt11:10:202
327053799132,2cyclictest41-21ksoftirqd/207:45:122
327053799131,2cyclictest41-21ksoftirqd/210:00:172
3270537991311,2cyclictest3250162-21kworker/u8:4+events_unbound08:17:292
3270537991311,1cyclictest391rcuc/212:30:122
327053699139,3cyclictest3581745-21kworker/u8:3+events_unbound12:30:131
327053699139,3cyclictest3420848-21kworker/u8:3+events_unbound09:55:121
327053699138,3cyclictest3539694-21awk11:25:201
327053699138,3cyclictest3286235-21/usr/sbin/munin07:25:141
327053699138,3cyclictest3250162-21kworker/u8:4+flush-259:007:45:181
3270536991311,2cyclictest3270488-21kworker/u8:2+events_unbound07:50:131
3270536991310,2cyclictest3472828-21kworker/u8:0+events_unbound10:30:241
3270536991310,2cyclictest3462520-21kworker/u8:4+flush-259:010:40:161
3270536991310,2cyclictest3462520-21kworker/u8:4+events_unbound10:50:151
3270536991310,2cyclictest3348350-21idleruntime-cro08:25:001
327053299139,3cyclictest3592084-21kworker/u8:2+events_unbound12:25:020
327053299137,4cyclictest3332891-21latency_hist08:10:000
327053999129,2cyclictest3591061-21needreboot12:15:183
327053999129,2cyclictest3348841-21fschecks_time08:25:143
327053999129,2cyclictest3322821-21irqcore08:00:163
327053999129,2cyclictest3210982-21kworker/u8:0+flush-259:007:10:313
327053999122,1cyclictest32932232grep07:30:163
327053699129,2cyclictest3529208-21kworker/u8:1+events_unbound11:55:181
327053699129,2cyclictest3374503-21cpuspeed_turbos08:50:131
327053699129,2cyclictest3250162-21kworker/u8:4+flush-259:008:55:131
327053699122,1cyclictest33724352kthreadcore08:45:191
3270536991210,1cyclictest3576304-21kworker/u8:0-events_unbound12:40:001
3270536991210,1cyclictest301rcuc/110:45:171
327053999119,2cyclictest3250162-21kworker/u8:4+events_unbound07:25:573
327053999119,1cyclictest0-21swapper/308:44:593
327053999118,2cyclictest3529208-21kworker/u8:1+flush-259:011:45:003
327053999118,2cyclictest3347452-21ntp_kernel_pll_08:20:203
327053999118,2cyclictest3250162-21kworker/u8:4+flush-259:009:01:583
327053699118,2cyclictest3301793-21cpuspeed_turbos07:40:131
327053699117,3cyclictest3270488-21kworker/u8:2+flush-259:007:59:011
327053999107,2cyclictest3455714-21needreboot10:05:193
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional