You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-14 - 23:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Fri Nov 14, 2025 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3488354995349,3cyclictest75-21kswapd012:05:023
3488354995147,3cyclictest75-21kswapd011:30:133
3488347994239,2cyclictest75-21kswapd011:30:131
3488344994239,2cyclictest75-21kswapd012:05:020
3488347993936,2cyclictest75-21kswapd012:05:021
3488344993832,4cyclictest3685110-21latency_hist10:20:010
3488347993731,4cyclictest3492540-21ntpq07:10:201
3488344993633,2cyclictest3741799-21kworker/u8:1+events_unbound11:30:100
3488350993430,3cyclictest3768200-21kworker/u8:0+events_unbound12:10:232
3488354993323,9cyclictest3731659-21kworker/u8:0+flush-259:011:30:003
3488350993330,2cyclictest3669914-21kworker/u8:3+events_unbound10:20:172
3488350993229,2cyclictest1422024-21kworker/u8:0+flush-259:007:10:232
3488344993229,2cyclictest3503739-21kworker/u8:3+flush-259:008:50:190
3488350993128,2cyclictest3773652-21kworker/u8:2+events_unbound11:45:172
3488354993025,3cyclictest3591971-21apt-get08:50:063
3488354993023,4cyclictest3794352-21cp12:05:003
3488350993026,3cyclictest3669914-21kworker/u8:3+events_unbound11:05:142
3488354992926,2cyclictest3487995-21kworker/u8:2+flush-259:007:15:003
3488354992924,4cyclictest3669914-21kworker/u8:3+flush-259:011:05:173
3488350992926,2cyclictest3669914-21kworker/u8:3+events_unbound10:15:142
3488350992926,2cyclictest3503739-21kworker/u8:3+events_unbound09:15:242
3488350992923,2cyclictest3768200-21kworker/u8:0+events_unbound12:25:142
3488347992926,2cyclictest3503739-21kworker/u8:3+flush-259:007:35:011
3488350992825,2cyclictest3571088-21kworker/u8:0+events_unbound08:55:242
3488350992825,2cyclictest3514004-21kworker/u8:1+events_unbound07:50:222
3488344992824,3cyclictest1422024-21kworker/u8:0+flush-259:007:25:000
3488354992720,4cyclictest3747344-21latency_hist11:20:013
3488350992724,2cyclictest3649115-21kworker/u8:1+events_unbound10:25:262
3488350992724,2cyclictest3613234-21kworker/u8:0+events_unbound09:20:122
3488347992721,4cyclictest3700552-21latency_hist10:35:011
3488344992723,3cyclictest3669914-21kworker/u8:3+events_unbound10:15:000
3488354992620,4cyclictest3788251-21ntpq11:55:233
3488347992623,2cyclictest3773652-21kworker/u8:2+events_unbound11:45:231
3488347992623,2cyclictest3741799-21kworker/u8:1+flush-259:011:35:141
3488347992623,2cyclictest3523876-21kworker/u8:0+flush-259:007:50:001
3488347992623,2cyclictest3503739-21kworker/u8:3+flush-259:008:20:011
3488350992522,2cyclictest3768200-21kworker/u8:0+events_unbound12:16:592
3488350992522,2cyclictest3649115-21kworker/u8:1+events_unbound10:55:232
3488350992522,2cyclictest3623533-21kworker/u8:4+events_unbound09:30:232
3488350992521,3cyclictest75-21kswapd012:05:022
3488347992522,2cyclictest3758365-21kworker/u8:3+flush-259:011:55:201
3488347992519,5cyclictest3628465-21latency_hist09:25:001
3488344992522,2cyclictest3752601-21gpgv11:25:010
3488344992521,3cyclictest3731659-21kworker/u8:0+flush-259:011:10:270
3488344992519,4cyclictest3778786-21latency_hist11:50:000
348834499250,23cyclictest1-21systemd07:55:010
3488350992421,2cyclictest3649115-21kworker/u8:1+events_unbound11:00:282
3488350992421,2cyclictest3503739-21kworker/u8:3+events_unbound09:40:222
3488350992419,2cyclictest3747365-21kworker/u8:2+events_unbound11:30:282
3488350992418,4cyclictest2698871-21systemd-journal08:10:012
3488347992420,3cyclictest3613234-21kworker/u8:0+flush-259:009:10:221
3488354992317,4cyclictest3613234-21kworker/u8:0+flush-259:009:50:003
3488350992320,2cyclictest3649115-21kworker/u8:1+events_unbound10:50:212
3488350992319,3cyclictest3571088-21kworker/u8:0+events_unbound09:04:332
3488350992318,2cyclictest3514004-21kworker/u8:1+events_unbound08:15:192
3488347992320,2cyclictest3654481-21apt-get09:50:121
3488347992320,2cyclictest3503739-21kworker/u8:3+flush-259:009:35:141
3488347992319,3cyclictest3612577-21apt-get09:10:001
3488350992219,2cyclictest3519116-21idleruntime-cro07:39:592
3488347992219,2cyclictest3649115-21kworker/u8:1+events_unbound10:50:191
3488347992219,2cyclictest3514004-21kworker/u8:1+flush-259:008:50:061
3488347992218,3cyclictest2698871-21systemd-journal12:15:001
3488347992217,3cyclictest3000063-21rs:main1
3488347992217,3cyclictest2698871-21systemd-journal07:35:421
3488344992219,2cyclictest3649115-21kworker/u8:1+events_unbound10:25:140
3488344992219,2cyclictest3613234-21kworker/u8:0+events_unbound10:00:220
3488344992219,2cyclictest3523876-21kworker/u8:0+flush-259:007:45:010
3488344992219,2cyclictest3523876-21kworker/u8:0+flush-259:007:45:000
3488354992118,2cyclictest3612561-21idleruntime-cro09:10:013
3488350992118,2cyclictest3669914-21kworker/u8:3+flush-259:010:45:252
3488350992118,2cyclictest3649115-21kworker/u8:1+events_unbound10:30:142
3488350992116,3cyclictest3613234-21kworker/u8:0+events_unbound10:00:132
3488350992115,4cyclictest3794488-21gpgv12:05:002
3488344992118,2cyclictest3488339-21cyclictest07:49:590
3488354992016,3cyclictest3758365-21kworker/u8:3+flush-259:012:20:173
3488354992014,4cyclictest2698871-21systemd-journal09:14:593
3488354992013,5cyclictest3555596-21latency_hist08:15:013
3488354992013,5cyclictest2698871-21systemd-journal07:20:003
3488350992017,2cyclictest3773652-21kworker/u8:2+events_unbound11:50:242
3488350992017,2cyclictest3534799-21kworker/u8:2+events_unbound08:20:022
3488350992016,3cyclictest3669914-21kworker/u8:3+events_unbound10:05:142
3488350992015,3cyclictest3752489-21dump-pmu-power11:25:012
3488347992017,2cyclictest3503739-21kworker/u8:3+events_unbound09:45:001
3488354991915,2cyclictest3773652-21kworker/u8:2+flush-259:011:45:253
3488354991913,4cyclictest3513971-21awk07:35:003
3488350991916,2cyclictest3758365-21kworker/u8:3+flush-259:011:40:112
3488350991916,2cyclictest3613234-21kworker/u8:0+flush-259:009:54:482
3488350991916,2cyclictest3503739-21kworker/u8:3+flush-259:008:44:422
3488350991916,2cyclictest3503739-21kworker/u8:3+flush-259:008:30:262
3488350991915,3cyclictest503-21systemd-logind07:30:002
3488347991916,2cyclictest3758365-21kworker/u8:3+events_unbound11:40:231
3488347991916,2cyclictest3669914-21kworker/u8:3+events_unbound10:45:151
3488347991916,2cyclictest3659324-21kworker/u8:4+events_unbound09:55:251
3488347991916,2cyclictest3613234-21kworker/u8:0+events_unbound10:00:251
3488347991914,4cyclictest3830968-21cp12:40:001
3488344991916,2cyclictest3649115-21kworker/u8:1+events_unbound11:00:220
3488344991916,2cyclictest3613234-21kworker/u8:0+events_unbound09:25:190
3488344991916,2cyclictest2698871-21systemd-journal11:45:000
3488344991915,3cyclictest3571088-21kworker/u8:0+events_unbound08:35:220
3488344991915,3cyclictest2698871-21systemd-journal11:54:420
3488344991915,3cyclictest1422024-21kworker/u8:0+flush-259:007:25:180
3488344991915,2cyclictest1-21systemd07:35:420
3488344991913,4cyclictest2698871-21systemd-journal11:06:410
3488354991814,3cyclictest3763589-21apt-get11:35:123
3488354991814,2cyclictest0-21swapper/308:40:013
3488354991811,5cyclictest3752482-21latency_hist11:25:013
3488350991815,2cyclictest3741799-21kworker/u8:1+events_unbound11:36:422
3488350991815,2cyclictest3571088-21kworker/u8:0+events_unbound08:35:252
3488350991815,2cyclictest3487995-21kworker/u8:2+events_unbound07:30:122
3488350991814,3cyclictest3669914-21kworker/u8:3+events_unbound10:14:362
3488350991811,5cyclictest3643964-21latency_hist09:40:002
3488347991815,2cyclictest3534799-21kworker/u8:2+events_unbound08:10:231
3488344991815,2cyclictest3758365-21kworker/u8:3+flush-259:012:25:000
3488344991815,2cyclictest3747365-21kworker/u8:2+events_unbound11:25:260
3488344991815,2cyclictest3534799-21kworker/u8:2+events_unbound07:55:230
3488344991814,3cyclictest3571328-21apt-get08:30:120
3488354991714,2cyclictest3620715-21kthreadcore09:15:193
3488354991714,2cyclictest3556881-21kthreadcore08:15:173
3488354991712,3cyclictest3503964-21apt-get07:25:123
3488354991711,4cyclictest3669686-21latency_hist10:05:013
3488350991714,2cyclictest3773652-21kworker/u8:2+events_unbound11:59:562
3488350991714,2cyclictest3731659-21kworker/u8:0+events_unbound11:25:102
3488350991713,3cyclictest3514004-21kworker/u8:1+flush-259:008:50:442
3488350991711,4cyclictest3829901-21ntpq12:35:222
3488347991715,2cyclictest3810184-21kworker/u8:1+flush-259:012:30:221
3488347991715,2cyclictest3810184-21kworker/u8:1+flush-259:012:30:221
3488347991715,1cyclictest3669914-21kworker/u8:3+events_unbound10:35:181
3488347991714,2cyclictest3741799-21kworker/u8:1+flush-259:011:20:011
3488347991714,2cyclictest3669914-21kworker/u8:3+events_unbound11:10:051
3488347991714,2cyclictest3613234-21kworker/u8:0+flush-259:009:45:151
3488347991714,2cyclictest3503739-21kworker/u8:3+flush-259:008:25:011
3488347991714,2cyclictest3503739-21kworker/u8:3+events_unbound08:40:171
3488344991714,2cyclictest1422024-21kworker/u8:0+events_unbound07:15:170
3488354991614,1cyclictest0-21swapper/312:40:003
3488354991613,2cyclictest3722417-21kthreadcore10:55:223
3488354991613,2cyclictest3571088-21kworker/u8:0+flush-259:009:05:013
3488354991612,3cyclictest3810184-21kworker/u8:1+flush-259:012:30:163
3488354991612,3cyclictest3810184-21kworker/u8:1+flush-259:012:30:163
3488354991612,3cyclictest3514004-21kworker/u8:1+flush-259:008:05:253
3488350991613,2cyclictest3810184-21kworker/u8:1+events_unbound12:34:552
3488350991613,2cyclictest3810184-21kworker/u8:1+events_unbound12:34:552
3488350991613,2cyclictest3731659-21kworker/u8:0+events_unbound11:15:252
3488350991613,2cyclictest3534799-21kworker/u8:2+flush-259:008:10:252
3488350991613,2cyclictest3503739-21kworker/u8:3+events_unbound07:48:302
3488350991612,3cyclictest3653326-21ntp_kernel_pll_09:45:212
3488350991611,4cyclictest1422024-21kworker/u8:0+flush-259:007:20:122
3488347991614,2cyclictest3503739-21kworker/u8:3+events_unbound07:55:011
3488347991613,2cyclictest3758365-21kworker/u8:3+flush-259:012:00:221
3488347991613,2cyclictest3649115-21kworker/u8:1+events_unbound10:55:191
3488347991613,2cyclictest3581574-21kworker/u8:4+events_unbound08:45:141
3488347991613,2cyclictest3000061-21in:imuxsock08:05:001
3488347991611,4cyclictest3000063-21rs:main1
3488344991612,3cyclictest3613234-21kworker/u8:0+events_unbound09:45:220
3488344991611,4cyclictest3659785-21diskstats09:55:140
3488344991611,4cyclictest3659355-21latency_hist09:55:010
348835499158,5cyclictest3487995-21kworker/u8:2+flush-259:007:20:183
3488354991512,2cyclictest3649115-21kworker/u8:1+flush-259:010:35:283
3488354991511,3cyclictest3669914-21kworker/u8:3+flush-259:010:10:263
3488354991511,3cyclictest3533637-21ntp_kernel_err07:50:213
348835099159,1cyclictest191rcu_preempt08:00:002
3488350991511,3cyclictest3623533-21kworker/u8:4+events_unbound09:26:072
3488350991511,3cyclictest3514004-21kworker/u8:1+events_unbound08:00:102
3488350991511,3cyclictest3487995-21kworker/u8:2+events_unbound07:40:412
3488350991511,3cyclictest3487995-21kworker/u8:2+events_unbound07:40:402
3488347991513,2cyclictest3649115-21kworker/u8:1+events_unbound11:00:041
3488347991513,1cyclictest3503739-21kworker/u8:3+events_unbound09:15:141
3488347991512,2cyclictest3628957-21fschecks_time09:25:151
3488347991512,2cyclictest1422024-21kworker/u8:0+flush-259:007:15:161
3488344991514,1cyclictest61450irq/134-enp0s29f1:tx-012:35:230
3488344991513,1cyclictest61450irq/134-enp0s29f1:tx-010:22:430
3488344991513,1cyclictest60650irq/133-enp0s29f1:rx-012:33:320
3488344991513,1cyclictest60650irq/133-enp0s29f1:rx-012:33:320
3488344991513,1cyclictest60650irq/133-enp0s29f1:rx-008:10:240
3488344991513,1cyclictest3544845-21apt-get08:05:010
3488344991512,2cyclictest60650irq/133-enp0s29f1:rx-012:11:320
3488344991512,2cyclictest60650irq/133-enp0s29f1:rx-009:37:210
3488344991512,2cyclictest3768200-21kworker/u8:0+events_unbound12:25:220
3488344991512,2cyclictest3720119-21needreboot10:50:220
3488344991512,2cyclictest3534799-21kworker/u8:2+events_unbound08:05:160
3488344991511,3cyclictest3618492-21cpuspeed_turbos09:15:130
3488344991511,3cyclictest3581574-21kworker/u8:4+events_unbound09:05:130
3488354991411,2cyclictest0-21swapper/309:35:003
3488354991410,3cyclictest3684721-21kworker/u8:2+flush-259:010:20:003
3488354991410,2cyclictest0-21swapper/312:30:003
348835099149,4cyclictest3669914-21kworker/u8:3+events_unbound10:42:262
3488350991411,2cyclictest3810184-21kworker/u8:1+events_unbound12:24:542
3488350991411,2cyclictest3659324-21kworker/u8:4+events_unbound09:55:152
3488350991410,3cyclictest3503739-21kworker/u8:3+events_unbound08:47:332
348834799149,4cyclictest2698871-21systemd-journal10:20:001
3488347991412,2cyclictest3768200-21kworker/u8:0+flush-259:012:25:161
3488347991412,2cyclictest3731659-21kworker/u8:0+events_unbound11:20:061
3488347991412,2cyclictest3571088-21kworker/u8:0+events_unbound09:00:011
3488347991412,1cyclictest0-21swapper/107:55:011
3488347991411,2cyclictest3773652-21kworker/u8:2+flush-259:011:50:151
3488347991411,2cyclictest3669914-21kworker/u8:3+events_unbound11:05:221
3488347991411,2cyclictest3571088-21kworker/u8:0+flush-259:008:35:251
3488347991410,3cyclictest3669914-21kworker/u8:3+flush-259:010:20:231
348834499149,4cyclictest60650irq/133-enp0s29f1:rx-010:45:140
3488344991413,1cyclictest60650irq/133-enp0s29f1:rx-008:17:490
3488344991412,1cyclictest60650irq/133-enp0s29f1:rx-010:55:290
3488344991412,1cyclictest60650irq/133-enp0s29f1:rx-010:35:180
3488344991412,1cyclictest60650irq/133-enp0s29f1:rx-009:10:160
3488344991412,1cyclictest60650irq/133-enp0s29f1:rx-008:55:170
3488344991412,1cyclictest60650irq/133-enp0s29f1:rx-007:10:120
3488344991411,2cyclictest60650irq/133-enp0s29f1:rx-009:40:190
3488344991411,2cyclictest60650irq/133-enp0s29f1:rx-007:30:170
3488344991410,3cyclictest3758365-21kworker/u8:3+flush-259:012:03:480
3488344991410,3cyclictest3741799-21kworker/u8:1+events_unbound11:15:130
348835499137,4cyclictest2698871-21systemd-journal07:49:593
3488354991310,2cyclictest3783500-21unixbench_singl11:50:243
3488354991310,2cyclictest3629755-21kthreadcore09:25:173
3488354991310,2cyclictest3503739-21kworker/u8:3+flush-259:009:50:163
3488354991310,2cyclictest0-21swapper/310:34:593
3488354991310,2cyclictest0-21swapper/308:35:003
3488354991310,2cyclictest0-21swapper/308:30:013
3488350991310,2cyclictest3613234-21kworker/u8:0+events_unbound09:10:112
3488350991310,2cyclictest3503739-21kworker/u8:3+events_unbound08:26:102
3488347991311,2cyclictest3649115-21kworker/u8:1+events_unbound10:25:231
3488347991311,2cyclictest3487995-21kworker/u8:2+events_unbound07:21:431
3488347991311,2cyclictest1422024-21kworker/u8:0+events_unbound07:25:271
3488347991310,2cyclictest3768200-21kworker/u8:0+flush-259:012:23:471
3488347991310,2cyclictest3731659-21kworker/u8:0+events_unbound11:25:161
3488347991310,2cyclictest3636181-21kthreadcore09:30:191
3488344991312,1cyclictest61450irq/134-enp0s29f1:tx-010:30:250
3488344991312,1cyclictest60650irq/133-enp0s29f1:rx-010:07:410
3488344991312,1cyclictest60650irq/133-enp0s29f1:rx-008:42:540
3488344991311,1cyclictest60650irq/133-enp0s29f1:rx-011:55:260
3488344991311,1cyclictest60650irq/133-enp0s29f1:rx-009:20:160
3488344991311,1cyclictest60650irq/133-enp0s29f1:rx-009:02:530
3488344991311,1cyclictest60650irq/133-enp0s29f1:rx-008:20:240
3488344991310,2cyclictest3805494-21fschecks_count12:15:140
3488344991310,2cyclictest3503739-21kworker/u8:3+flush-259:008:25:250
348835499129,2cyclictest3690607-21cpuspeed_turbos10:25:153
348835499129,1cyclictest0-21swapper/310:05:203
348835499121,9cyclictest3539715-21awk08:00:013
3488354991210,1cyclictest3519804-21grep07:40:163
3488354991210,1cyclictest3519804-21grep07:40:153
3488354991210,1cyclictest0-21swapper/309:40:263
348835099121,5cyclictest3705190-21unixbench-2d10:35:282
348834799129,2cyclictest3710321-21unixbench_multi10:40:261
348834799129,2cyclictest3669914-21kworker/u8:3+events_unbound10:10:181
3488347991210,1cyclictest0-21swapper/107:40:141
3488347991210,1cyclictest0-21swapper/107:40:131
3488344991210,1cyclictest0-21swapper/011:35:160
348835499118,2cyclictest0-21swapper/308:25:013
348834799118,2cyclictest3581574-21kworker/u8:4+events_unbound09:01:441
348834499118,2cyclictest3669914-21kworker/u8:3+events_unbound10:40:230
3488344991110,1cyclictest60650irq/133-enp0s29f1:rx-008:45:050
348835499108,1cyclictest0-21swapper/309:55:193
348835499107,1cyclictest3588587-21kthreadcore08:45:183
348835499105,4cyclictest3669914-21kworker/u8:3+flush-259:010:20:113
348834799106,3cyclictest3534799-21kworker/u8:2+flush-259:008:05:571
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional