You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-22 - 20:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Sat Nov 22, 2025 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3916970994844,3cyclictest12628-21kworker/u8:3+events_unbound12:10:010
3916970994336,5cyclictest3953488-21latency_hist07:45:010
3916970994036,3cyclictest3906425-21kworker/u8:0+events_unbound07:15:250
3916976993933,4cyclictest58903-21latency_hist12:35:002
3916980993831,5cyclictest17751-21latency_hist11:55:003
3916970993834,3cyclictest4004979-21kworker/u8:2+events_unbound09:35:170
3916980993733,3cyclictest4072298-21kworker/u8:0+events_unbound09:55:183
3916970993632,3cyclictest4181045-21kworker/u8:1+events_unbound11:40:000
3916980993428,4cyclictest4107367-21ntpq10:10:223
3916970993431,2cyclictest4015703-21kworker/u8:1+events_unbound08:50:140
3916970993431,2cyclictest4004979-21kworker/u8:2+events_unbound08:45:210
3916970993330,2cyclictest4108475-21kworker/u8:2+events_unbound11:05:180
3916970993330,2cyclictest4015703-21kworker/u8:1+flush-259:009:15:280
3916970993330,2cyclictest3911791-21kworker/u8:2+flush-259:008:25:180
3916970993329,3cyclictest4181045-21kworker/u8:1+events_unbound11:50:160
3916980993228,3cyclictest4004979-21kworker/u8:2+flush-259:009:35:213
3916970993229,2cyclictest4092614-21kworker/u8:4+events_unbound11:10:190
3916970993229,2cyclictest3911791-21kworker/u8:2+events_unbound07:35:200
3916970993228,3cyclictest3979235-21latency_hist08:10:000
3916980993124,5cyclictest47692-21ntpq12:20:213
3916970993128,2cyclictest4138700-21kworker/u8:0+events_unbound10:55:150
3916970993128,2cyclictest4072298-21kworker/u8:0+flush-259:010:00:160
3916970993128,2cyclictest3906425-21kworker/u8:0+events_unbound07:25:230
3916980993025,3cyclictest4134342-21apt-get10:40:133
3916972993026,3cyclictest3948577-21/usr/sbin/munin07:40:241
3916972993022,6cyclictest2698871-21systemd-journal09:45:001
3916970993026,3cyclictest4103146-21kworker/u8:1+flush-259:010:45:270
3916970993026,3cyclictest4021055-21kworker/u8:0+events_unbound09:05:150
3916972992921,6cyclictest4025982-21latency_hist08:55:001
3916970992926,2cyclictest4092614-21kworker/u8:4+events_unbound10:05:170
3916970992926,2cyclictest4015703-21kworker/u8:1+events_unbound09:30:150
3916970992926,2cyclictest22884-21kworker/u8:2+events_unbound12:10:160
3916970992925,3cyclictest4175429-21kworker/u8:0+events_unbound11:30:130
3916980992824,3cyclictest4021055-21kworker/u8:0+flush-259:008:50:253
3916980992822,5cyclictest4129160-21apt-get10:35:133
3916972992824,3cyclictest4150155-21/usr/sbin/munin10:55:231
3916972992821,5cyclictest3932645-21/usr/sbin/munin07:25:151
3916970992825,2cyclictest4092614-21kworker/u8:4+events_unbound11:15:220
3916970992825,2cyclictest4092614-21kworker/u8:4+events_unbound10:40:140
3916970992825,2cyclictest4004979-21kworker/u8:2+flush-259:008:55:190
3916970992825,2cyclictest18010-21apt-get11:55:130
3916970992823,2cyclictest3906425-21kworker/u8:0+events_unbound07:20:410
3916970992822,4cyclictest7430-21latency_hist11:45:010
3916970992821,5cyclictest4123738-21latency_hist10:30:000
3916980992722,3cyclictest3974304-21kworker/u8:4+flush-259:008:10:183
3916976992721,4cyclictest2698871-21systemd-journal09:50:012
3916972992722,3cyclictest4010128-21gpgv08:40:011
3916970992724,2cyclictest2303-21kworker/u8:0+flush-259:012:15:210
3916970992721,4cyclictest3968933-21latency_hist08:00:000
3916970992720,5cyclictest4181240-21diskstats11:25:150
3916980992620,4cyclictest2698871-21systemd-journal08:35:003
3916980992620,4cyclictest2698871-21systemd-journal07:45:013
3916976992623,2cyclictest3000063-21rs:main2
3916976992622,3cyclictest4015703-21kworker/u8:1+flush-259:009:15:192
3916976992621,4cyclictest2698871-21systemd-journal08:00:002
3916970992623,2cyclictest3973070-21kworker/u8:3+events_unbound08:35:250
3916970992622,3cyclictest2303-21kworker/u8:0+flush-259:012:35:190
3916970992619,5cyclictest27112-21ntpq12:00:200
3916976992522,2cyclictest3906425-21kworker/u8:0+events_unbound07:25:172
3916970992522,2cyclictest4036487-21kworker/u8:3+events_unbound09:20:220
3916976992421,2cyclictest4103146-21kworker/u8:1+flush-259:010:25:212
3916976992418,4cyclictest4170963-21diskstats11:15:162
3916972992421,2cyclictest4092614-21kworker/u8:4+flush-259:011:15:001
3916972992420,3cyclictest3994944-21apt-get08:25:131
3916970992421,2cyclictest4108475-21kworker/u8:2+flush-259:010:50:160
3916970992421,2cyclictest4108475-21kworker/u8:2+flush-259:010:50:150
3916976992319,3cyclictest4004979-21kworker/u8:2+events_unbound08:45:152
3916972992319,3cyclictest3917148-21kworker/u8:1+flush-259:007:30:171
3916970992320,3cyclictest3999441-21kworker/u8:1+flush-259:008:30:200
3916970992320,2cyclictest4092614-21kworker/u8:4+events_unbound10:10:230
3916970992320,2cyclictest3911791-21kworker/u8:2+events_unbound08:00:180
3916980992218,3cyclictest4021055-21kworker/u8:0+flush-259:008:55:253
3916976992218,3cyclictest4015703-21kworker/u8:1+flush-259:008:50:152
3916972992218,3cyclictest59589-21apt-get12:35:141
3916972992217,3cyclictest4108475-21kworker/u8:2+flush-259:010:35:181
3916972992216,4cyclictest4108475-21kworker/u8:2+flush-259:011:20:171
3916972992215,5cyclictest3958947-21apt-get07:50:171
3916980992118,2cyclictest4138700-21kworker/u8:0+flush-259:011:00:003
3916976992118,2cyclictest4015703-21kworker/u8:1+events_unbound09:30:192
3916976992116,4cyclictest2698871-21systemd-journal11:30:002
3916972992118,2cyclictest4113419-21latency_hist10:19:591
3916972992118,2cyclictest4021055-21kworker/u8:0+flush-259:008:55:261
3916972992117,3cyclictest4108475-21kworker/u8:2+flush-259:011:05:261
3916972992115,4cyclictest58051-21ntpq12:30:241
3916970992118,2cyclictest4004979-21kworker/u8:2+events_unbound09:10:230
3916970992118,2cyclictest3973070-21kworker/u8:3+events_unbound08:40:210
3916970992118,2cyclictest12628-21kworker/u8:3+events_unbound12:30:250
3916970992117,3cyclictest4138700-21kworker/u8:0+events_unbound11:04:550
3916970992115,4cyclictest2698871-21systemd-journal10:33:420
3916980992017,2cyclictest4190891-21kworker/u8:3+flush-259:011:40:173
3916980992015,3cyclictest43437-21latency_hist12:20:023
3916976992017,2cyclictest4138700-21kworker/u8:0+flush-259:010:55:172
3916976992017,2cyclictest4072298-21kworker/u8:0+flush-259:010:00:002
3916976992016,3cyclictest3973070-21kworker/u8:3+events_unbound08:20:152
3916972992015,4cyclictest4077658-21/usr/sbin/munin09:45:191
3916972992014,4cyclictest22865-21latency_hist12:00:011
3916970992018,2cyclictest4108475-21kworker/u8:2+events_unbound10:15:260
3916970992017,2cyclictest3973070-21kworker/u8:3+events_unbound08:17:420
3916970992016,3cyclictest4108475-21kworker/u8:2+flush-259:010:35:440
3916980991916,2cyclictest4138700-21kworker/u8:0+flush-259:011:00:163
3916970991916,2cyclictest4036487-21kworker/u8:3+events_unbound09:41:160
3916970991916,2cyclictest4021055-21kworker/u8:0+events_unbound09:00:230
3916970991916,2cyclictest3911791-21kworker/u8:2+events_unbound08:22:040
3916970991916,2cyclictest2303-21kworker/u8:0+events_unbound11:45:290
3916980991815,2cyclictest4004979-21kworker/u8:2+flush-259:009:25:013
3916980991815,2cyclictest4004979-21kworker/u8:2+flush-259:008:45:253
3916980991815,2cyclictest3911791-21kworker/u8:2+flush-259:008:00:153
3916976991815,2cyclictest4108475-21kworker/u8:2+events_unbound10:35:192
3916976991814,3cyclictest4004979-21kworker/u8:2+events_unbound09:35:252
391697699180,17cyclictest1-21systemd11:22:422
3916972991810,6cyclictest4020822-21turbostat08:50:001
3916970991814,3cyclictest3906425-21kworker/u8:0+events_unbound07:12:490
3916970991813,2cyclictest2303-21kworker/u8:0+events_unbound12:20:260
3916980991714,2cyclictest4108475-21kworker/u8:2+flush-259:010:25:003
3916980991710,5cyclictest4010261-21cat08:40:003
3916976991715,1cyclictest0-21swapper/209:55:002
3916976991713,3cyclictest4092614-21kworker/u8:4+flush-259:010:10:152
3916976991713,3cyclictest2303-21kworker/u8:0+flush-259:011:45:212
3916976991713,2cyclictest4138700-21kworker/u8:0+flush-259:011:00:212
3916972991714,2cyclictest4139227-21gpgv10:45:001
3916972991713,3cyclictest0-21swapper/109:30:001
3916972991710,5cyclictest4160220-21latency_hist11:05:011
3916970991714,2cyclictest22884-21kworker/u8:2+events_unbound12:25:560
3916970991712,4cyclictest2698871-21systemd-journal09:30:000
3916970991710,5cyclictest3937986-21diskstats07:30:150
3916980991611,4cyclictest2303-21kworker/u8:0+flush-259:012:39:593
3916980991610,5cyclictest22884-21kworker/u8:2+flush-259:012:25:183
391697699166,8cyclictest48603-21munin-plugin-st12:25:002
3916976991612,3cyclictest4036487-21kworker/u8:3+events_unbound09:40:232
3916976991612,3cyclictest3926238-21needreboot07:15:202
3916976991612,3cyclictest3911791-21kworker/u8:2+flush-259:007:35:172
3916976991612,3cyclictest3906425-21kworker/u8:0+flush-259:007:20:282
3916976991612,3cyclictest2303-21kworker/u8:0+flush-259:012:35:212
391697299168,7cyclictest4041425-21awk09:10:001
3916972991613,2cyclictest4066249-21ntp_kernel_pll_09:30:221
3916972991613,2cyclictest12628-21kworker/u8:3+events_unbound12:05:191
3916972991612,2cyclictest3979241-21idleruntime-cro08:10:001
3916972991611,4cyclictest4181045-21kworker/u8:1+flush-259:011:50:001
3916972991611,4cyclictest4139884-21apt-get10:45:141
3916972991610,4cyclictest3000063-21rs:main1
3916972991610,4cyclictest2698871-21systemd-journal10:24:591
3916970991613,2cyclictest4015703-21kworker/u8:1+events_unbound09:52:310
3916980991512,2cyclictest4058165-21kthreadcore09:25:183
3916980991512,2cyclictest3974304-21kworker/u8:4+flush-259:008:05:203
3916980991512,2cyclictest3911791-21kworker/u8:2+flush-259:007:17:053
3916980991511,3cyclictest3984615-21/usr/sbin/munin08:15:163
391697699159,4cyclictest2698871-21systemd-journal08:09:412
391697699158,5cyclictest4010556-21gpgv08:40:012
391697699151,2cyclictest41373501kthreadcore10:40:202
3916976991512,2cyclictest4004979-21kworker/u8:2+events_unbound09:25:242
3916976991512,2cyclictest3984359-21cron08:15:002
3916976991512,2cyclictest0-21swapper/208:05:002
3916976991511,3cyclictest4036487-21kworker/u8:3+flush-259:009:20:152
3916976991511,3cyclictest2303-21kworker/u8:0+events_unbound12:15:192
3916976991511,3cyclictest2303-21kworker/u8:0+events_unbound12:00:162
3916976991510,4cyclictest3911791-21kworker/u8:2+flush-259:008:25:252
3916976991510,3cyclictest0-21swapper/211:15:002
391697299159,4cyclictest3969204-21apt-get08:00:121
3916972991512,2cyclictest4004951-21cron08:35:001
3916972991511,3cyclictest38586-21apt-get12:15:121
3916970991513,1cyclictest60650irq/133-enp0s29f1:rx-007:45:290
3916970991512,2cyclictest3974304-21kworker/u8:4+flush-259:008:11:510
3916970991511,3cyclictest4108475-21kworker/u8:2+events_unbound11:20:130
3916970991511,3cyclictest4036487-21kworker/u8:3+events_unbound09:45:020
391698099149,3cyclictest2698871-21systemd-journal10:17:423
391698099147,5cyclictest4015678-21latency_hist08:45:013
3916980991411,2cyclictest4092613-21kworker/u8:2+events_unbound10:05:253
391698099141,12cyclictest1-21systemd12:01:423
391697699149,4cyclictest3937573-21kworker/u8:4+flush-259:007:45:182
3916976991411,2cyclictest18420-21irqcore11:55:182
3916976991410,3cyclictest4004979-21kworker/u8:2+flush-259:009:10:012
391697299149,4cyclictest4181045-21kworker/u8:1+flush-259:011:35:251
391697299148,4cyclictest3000063-21rs:main1
3916972991412,1cyclictest4108475-21kworker/u8:2+flush-259:011:30:011
3916970991411,2cyclictest60650irq/133-enp0s29f1:rx-007:50:250
3916970991410,3cyclictest4108475-21kworker/u8:2+events_unbound10:20:290
391698099139,3cyclictest4191413-21cpuspeed_turbos11:35:133
391698099138,3cyclictest2698871-21systemd-journal09:32:413
3916980991310,2cyclictest4169648-21ntp_kernel_pll_11:10:213
391697699139,3cyclictest3917158-21/usr/sbin/munin07:10:222
391697699139,3cyclictest22884-21kworker/u8:2+flush-259:012:10:272
391697699138,4cyclictest12628-21kworker/u8:3+flush-259:012:05:202
391697699137,4cyclictest4092614-21kworker/u8:4+events_unbound11:05:162
391697699136,5cyclictest2698871-21systemd-journal07:35:012
3916976991310,2cyclictest3999441-21kworker/u8:1+events_unbound08:30:202
3916976991310,2cyclictest3937573-21kworker/u8:4+flush-259:007:40:192
3916976991310,2cyclictest3567-21kthreadcore11:40:182
391697299139,3cyclictest4108475-21kworker/u8:2+flush-259:010:50:181
391697299139,3cyclictest4108475-21kworker/u8:2+flush-259:010:50:171
391697299139,3cyclictest4015703-21kworker/u8:1+flush-259:009:18:111
391697299139,3cyclictest2303-21kworker/u8:0+flush-259:012:20:181
391697299138,4cyclictest3953763-21apt-get07:45:161
391697299137,4cyclictest4108251-21latency_hist10:15:011
391697299136,5cyclictest2698871-21systemd-journal09:55:001
3916972991311,1cyclictest837-21snmpd09:03:001
3916972991310,2cyclictest4173092-21sort11:15:201
3916972991310,2cyclictest4124066-21cpuspeed_turbos10:30:151
3916972991310,2cyclictest3990046-21fschecks_time08:20:161
391698099129,2cyclictest4180371-21unixbench_singl11:20:263
391698099129,2cyclictest3973070-21kworker/u8:3+events_unbound08:30:003
391698099129,2cyclictest0-21swapper/309:20:003
391698099128,3cyclictest4004979-21kworker/u8:2+flush-259:009:05:243
391698099128,3cyclictest3911791-21kworker/u8:2+flush-259:007:20:023
3916980991210,1cyclictest0-21swapper/310:50:183
3916980991210,1cyclictest0-21swapper/310:50:173
391697699127,4cyclictest4021055-21kworker/u8:0+flush-259:009:00:192
391697699127,4cyclictest4004979-21kworker/u8:2+flush-259:009:10:222
391697699127,3cyclictest1-21systemd08:40:232
391697699120,2cyclictest4113420-21awk10:19:592
391697299129,2cyclictest4181045-21kworker/u8:1+events_unbound11:50:191
391697299129,2cyclictest0-21swapper/110:30:001
391697299126,4cyclictest4010784-21apt-get08:40:131
391697299126,4cyclictest3989520-21latency_hist08:20:001
391698099119,1cyclictest20288-21kthreadcore11:55:193
391698099118,2cyclictest4181146-21cpuspeed_turbos11:25:143
391698099116,4cyclictest3911791-21kworker/u8:2+flush-259:007:50:013
391697699118,2cyclictest4191771-21cut11:35:172
391697699118,2cyclictest2303-21kworker/u8:0+flush-259:011:50:552
391697699115,4cyclictest4108475-21kworker/u8:2+flush-259:010:50:262
391697699115,4cyclictest4108475-21kworker/u8:2+flush-259:010:50:252
391697699115,4cyclictest3000063-21rs:main2
391697699113,5cyclictest4117674-21ntpq10:20:212
391697699110,9cyclictest4097984-21awk10:05:012
391697299118,2cyclictest6949-21unixbench_multi11:40:261
391697299117,2cyclictest0-21swapper/107:10:201
391697299112,7cyclictest3927252-21munin-run07:20:001
391698099106,3cyclictest491ktimers/308:20:143
391698099100,2cyclictest12585-21awk11:50:013
391697299105,3cyclictest2698871-21systemd-journal07:35:031
39169809997,1cyclictest0-21swapper/311:10:013
39169729996,2cyclictest0-21swapper/111:30:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional