You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-17 - 00:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Wed Jul 16, 2025 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
478384993933,4cyclictest556167-21latency_hist08:25:013
478380993835,2cyclictest716930-21kworker/u8:2+flush-259:011:05:202
478376993731,4cyclictest628344-21latency_hist09:35:011
478384993427,5cyclictest819996-21latency_hist12:40:013
478372993229,2cyclictest783661-21kworker/u8:3+events_unbound12:20:140
478372993128,2cyclictest711409-21kworker/u8:1+flush-259:011:30:000
478376993026,3cyclictest696098-21kworker/u8:2+flush-259:010:50:011
478380992924,3cyclictest664986-21/usr/sbin/munin10:10:152
478372992925,3cyclictest576999-21kworker/u8:1+flush-259:008:55:170
478372992924,3cyclictest814470-21latency_hist12:35:010
478380992825,2cyclictest504207-21kworker/u8:4+flush-259:007:55:162
478380992823,4cyclictest525275-21latency_hist07:55:002
478380992724,2cyclictest3000061-21in:imuxsock09:25:002
478380992723,3cyclictest654501-21kworker/u8:3+events_unbound10:25:132
478376992723,3cyclictest525544-21apt-get07:55:141
478372992724,2cyclictest514530-21kworker/u8:3+events_unbound08:10:010
478372992724,2cyclictest504207-21kworker/u8:4+events_unbound07:40:010
478384992622,3cyclictest711409-21kworker/u8:1+flush-259:011:50:173
478372992620,4cyclictest509581-21/usr/sbin/munin07:40:130
478376992522,2cyclictest551269-21apt-get08:20:131
478376992522,2cyclictest1644207-21kworker/u8:3+flush-259:007:15:181
478372992522,2cyclictest478002-21kworker/u8:2+events_unbound07:25:180
478380992421,2cyclictest535622-21kworker/u8:1+flush-259:008:15:012
478380992420,3cyclictest742440-21kworker/u8:4+events_unbound11:35:172
478376992421,2cyclictest716930-21kworker/u8:2+flush-259:012:15:001
478376992419,3cyclictest794157-21/usr/sbin/munin12:15:171
478380992320,2cyclictest551234-21kworker/u8:3+events_unbound08:50:292
478372992319,3cyclictest788778-21latency_hist12:10:010
47837299230,21cyclictest1-21systemd07:57:370
478380992219,2cyclictest664748-21sh10:09:592
478380992219,2cyclictest478002-21kworker/u8:2+flush-259:007:33:392
478372992218,3cyclictest711776-21/usr/sbin/munin10:55:240
478384992117,3cyclictest633839-21kworker/u8:3+flush-259:009:45:263
478384992115,4cyclictest803326-21ntpq12:20:213
478384992115,4cyclictest644406-21kworker/u8:0+flush-259:010:05:193
478384992114,5cyclictest3754036-21systemd-journal07:45:003
478376992118,2cyclictest654501-21kworker/u8:3+flush-259:010:35:001
478376992117,3cyclictest618281-21kworker/u8:0+flush-259:009:35:201
478372992115,4cyclictest3754036-21systemd-journal10:45:000
478384992016,3cyclictest576999-21kworker/u8:1+flush-259:008:50:013
478384992016,3cyclictest1631818-21kworker/u8:0+flush-259:007:20:003
478384992013,5cyclictest3754036-21systemd-journal10:02:383
478380992017,2cyclictest716930-21kworker/u8:2+events_unbound12:15:222
478380992017,2cyclictest716930-21kworker/u8:2+events_unbound11:50:142
478380992017,2cyclictest685371-21kworker/u8:1+flush-259:010:40:202
478380992015,3cyclictest561315-21cron08:30:002
478376992017,2cyclictest711409-21kworker/u8:1+events_unbound11:20:201
478376992017,2cyclictest675281-21kworker/u8:0+flush-259:010:50:191
478376992017,2cyclictest669522-21kworker/u8:2+flush-259:010:15:231
478376992017,2cyclictest607597-21kworker/u8:2+events_unbound09:40:221
478376992016,3cyclictest644406-21kworker/u8:0+events_unbound09:50:251
478376992015,4cyclictest592259-21latency_hist09:00:001
478376992015,4cyclictest561570-21/usr/sbin/munin08:30:181
478372992016,3cyclictest763044-21latency_hist11:45:000
478372992016,3cyclictest716930-21kworker/u8:2+events_unbound11:15:170
478372992016,3cyclictest654501-21kworker/u8:3+flush-259:010:15:160
478372992016,3cyclictest3754036-21systemd-journal07:19:390
478372992015,4cyclictest606809-21ntpq09:10:230
478384991915,3cyclictest701198-21cron10:45:013
478384991914,4cyclictest3754036-21systemd-journal07:50:013
478384991913,4cyclictest634140-21/usr/sbin/munin09:40:143
478384991912,6cyclictest3754036-21systemd-journal11:40:003
478384991911,5cyclictest716695-21latency_hist11:00:013
478380991917,2cyclictest711409-21kworker/u8:1+flush-259:012:05:222
478380991917,1cyclictest1631818-21kworker/u8:0+flush-259:008:20:262
478380991916,2cyclictest1631818-21kworker/u8:0+events_unbound07:15:002
478380991915,3cyclictest654501-21kworker/u8:3+events_unbound10:20:192
478380991915,3cyclictest633839-21kworker/u8:3+flush-259:009:39:592
478380991912,1cyclictest0-21swapper/207:14:592
478376991916,2cyclictest804182-21cron12:25:001
478376991916,2cyclictest566697-21kworker/u8:4+flush-259:008:45:191
478376991916,2cyclictest478002-21kworker/u8:2+flush-259:007:55:011
478372991915,3cyclictest654501-21kworker/u8:3+flush-259:010:25:160
478372991915,3cyclictest607597-21kworker/u8:2+flush-259:009:24:590
478384991811,5cyclictest3754036-21systemd-journal07:21:393
478384991811,5cyclictest3754036-21systemd-journal07:21:383
478380991816,2cyclictest566697-21kworker/u8:4+flush-259:008:40:242
478380991815,3cyclictest623460-21kworker/u8:1+events_unbound09:30:252
478380991815,2cyclictest711409-21kworker/u8:1+flush-259:011:55:262
478380991815,2cyclictest711409-21kworker/u8:1+flush-259:011:55:262
478380991815,2cyclictest576999-21kworker/u8:1+events_unbound08:55:382
478380991815,2cyclictest566697-21kworker/u8:4+flush-259:009:15:232
478380991815,2cyclictest1631818-21kworker/u8:0+flush-259:007:35:152
478380991815,1cyclictest0-21swapper/209:10:002
478380991814,3cyclictest716930-21kworker/u8:2+events_unbound12:01:392
478380991812,4cyclictest3000063-21rs:main2
478376991816,2cyclictest711409-21kworker/u8:1+flush-259:012:05:261
478376991816,1cyclictest612886-21kworker/u8:1+events_unbound09:20:261
478376991815,2cyclictest551054-21kworker/u8:2+events_unbound08:35:251
478376991815,2cyclictest478002-21kworker/u8:2+events_unbound07:25:241
478376991814,3cyclictest804435-21/usr/sbin/munin12:25:151
478376991814,3cyclictest793940-21kworker/u8:0+flush-259:012:35:161
478376991814,3cyclictest711409-21kworker/u8:1+events_unbound11:50:241
478376991813,3cyclictest488883-21/usr/sbin/munin07:20:141
478376991813,3cyclictest488883-21/usr/sbin/munin07:20:141
478372991815,2cyclictest809331-21idleruntime-cro12:30:000
478372991815,2cyclictest696098-21kworker/u8:2+events_unbound10:54:370
478372991815,2cyclictest639082-21kworker/u8:1+flush-259:010:00:200
478372991815,2cyclictest576999-21kworker/u8:1+events_unbound08:50:190
478372991815,2cyclictest478002-21kworker/u8:2+flush-259:007:50:000
478372991815,2cyclictest3000061-21in:imuxsock09:20:000
478372991814,3cyclictest514530-21kworker/u8:3+events_unbound08:00:180
478384991713,3cyclictest514530-21kworker/u8:3+flush-259:008:00:013
478384991713,2cyclictest809599-21apt-get12:30:123
478384991713,2cyclictest716930-21kworker/u8:2+flush-259:011:25:183
478384991712,3cyclictest670150-21apt-get10:15:113
478380991713,3cyclictest711409-21kworker/u8:1+flush-259:011:20:232
478380991713,3cyclictest654501-21kworker/u8:3+events_unbound10:30:132
478380991713,3cyclictest3000063-21rs:main2
478380991713,3cyclictest1644207-21kworker/u8:3+flush-259:007:20:262
478380991713,3cyclictest1644207-21kworker/u8:3+flush-259:007:20:252
478376991715,2cyclictest566697-21kworker/u8:4+events_unbound09:00:231
478376991714,2cyclictest742440-21kworker/u8:4+events_unbound11:30:221
478376991714,2cyclictest711409-21kworker/u8:1+events_unbound11:40:171
478376991713,3cyclictest514502-21apt07:45:011
478372991714,2cyclictest793940-21kworker/u8:0+flush-259:012:35:270
478372991714,2cyclictest716930-21kworker/u8:2+events_unbound11:05:180
478372991714,2cyclictest623460-21kworker/u8:1+flush-259:009:34:590
478372991714,2cyclictest60650irq/133-enp0s29f1:rx-008:45:210
478372991713,3cyclictest566697-21kworker/u8:4+events_unbound09:00:190
478384991612,3cyclictest520357-21apt-get07:50:123
478380991615,1cyclictest633839-21kworker/u8:3+events_unbound09:45:162
478380991613,2cyclictest810020-21irqcore12:30:172
478380991613,2cyclictest696098-21kworker/u8:2+events_unbound10:53:392
478380991613,2cyclictest675281-21kworker/u8:0+events_unbound11:00:382
478380991612,3cyclictest582479-21kworker/u8:0+flush-259:009:01:432
478380991612,3cyclictest551234-21kworker/u8:3+flush-259:008:30:222
478380991612,3cyclictest504207-21kworker/u8:4+flush-259:007:45:252
478376991613,2cyclictest716930-21kworker/u8:2+events_unbound11:45:291
478376991612,3cyclictest514530-21kworker/u8:3+events_unbound07:45:171
478372991614,1cyclictest60650irq/133-enp0s29f1:rx-009:55:180
478372991613,2cyclictest783661-21kworker/u8:3+events_unbound12:10:150
478372991613,2cyclictest690941-21sh10:35:010
478372991613,2cyclictest60650irq/133-enp0s29f1:rx-011:45:150
478372991611,4cyclictest3000063-21rs:main0
478384991513,1cyclictest0-21swapper/311:15:003
478384991512,2cyclictest701886-21irqcore10:45:163
478384991512,2cyclictest478002-21kworker/u8:2+flush-259:007:25:433
478384991511,3cyclictest695183-21ntp_kernel_pll_10:35:243
478384991510,3cyclictest592524-21apt-get09:00:113
478380991512,2cyclictest716930-21kworker/u8:2+events_unbound12:37:382
478380991512,2cyclictest711409-21kworker/u8:1+events_unbound11:40:182
478380991512,2cyclictest597656-21kworker/u8:1+flush-259:009:10:042
478380991512,2cyclictest535622-21kworker/u8:1+flush-259:008:05:302
478380991512,2cyclictest535622-21kworker/u8:1+events_unbound08:16:372
478380991511,3cyclictest639082-21kworker/u8:1+events_unbound10:03:392
478380991511,3cyclictest607597-21kworker/u8:2+flush-259:009:40:212
478376991512,2cyclictest773384-21kworker/u8:0+flush-259:011:55:151
478376991512,2cyclictest773384-21kworker/u8:0+flush-259:011:55:151
478376991512,2cyclictest728280-21kthreadcore11:10:191
478376991512,2cyclictest716930-21kworker/u8:2+events_unbound12:00:261
478376991510,4cyclictest757043-21ntpq11:35:221
478372991513,1cyclictest60650irq/133-enp0s29f1:rx-011:00:260
478372991513,1cyclictest60650irq/133-enp0s29f1:rx-009:05:120
478372991513,1cyclictest60650irq/133-enp0s29f1:rx-008:20:170
478372991512,2cyclictest60650irq/133-enp0s29f1:rx-009:50:050
478372991512,2cyclictest60650irq/133-enp0s29f1:rx-007:24:030
478372991512,2cyclictest60650irq/133-enp0s29f1:rx-007:24:030
478372991512,2cyclictest60650irq/133-enp0s29f1:rx-007:10:150
478372991512,2cyclictest566697-21kworker/u8:4+events_unbound08:40:170
478372991512,2cyclictest504207-21kworker/u8:4+flush-259:007:50:250
478372991512,2cyclictest478002-21kworker/u8:2+events_unbound07:30:130
478372991511,3cyclictest659901-21apt-get10:05:150
478372991511,3cyclictest1631818-21kworker/u8:0+events_unbound08:10:120
47838499149,3cyclictest571630-21latency_hist08:40:013
47838499149,3cyclictest0-21swapper/311:45:003
478384991411,2cyclictest0-21swapper/307:15:003
47838099149,3cyclictest804864-21irqcore12:25:172
478380991411,2cyclictest716930-21kworker/u8:2+events_unbound12:20:202
478380991411,2cyclictest711409-21kworker/u8:1+events_unbound11:25:382
478380991411,2cyclictest669522-21kworker/u8:2+events_unbound10:19:572
478380991411,2cyclictest644406-21kworker/u8:0+flush-259:009:55:232
478380991411,2cyclictest566697-21kworker/u8:4+flush-259:008:37:442
478380991411,2cyclictest566697-21kworker/u8:4+events_unbound08:45:262
478380991411,2cyclictest514530-21kworker/u8:3+events_unbound08:00:202
478380991411,1cyclictest0-21swapper/209:54:592
478376991412,2cyclictest654501-21kworker/u8:3+events_unbound10:20:171
478376991411,2cyclictest716705-21latency_hist11:00:001
478376991411,2cyclictest618281-21kworker/u8:0+flush-259:009:25:201
478376991411,2cyclictest566697-21kworker/u8:4+flush-259:009:15:211
478376991411,2cyclictest535750-21latency_hist08:05:001
478376991410,2cyclictest701232-21true10:45:011
478372991413,1cyclictest60650irq/133-enp0s29f1:rx-011:37:410
478372991412,2cyclictest742440-21kworker/u8:4+flush-259:011:30:170
478372991412,1cyclictest60650irq/133-enp0s29f1:rx-012:15:170
478372991412,1cyclictest60650irq/133-enp0s29f1:rx-012:00:210
478372991412,1cyclictest60650irq/133-enp0s29f1:rx-010:24:200
478372991412,1cyclictest0-21swapper/008:35:010
478372991410,3cyclictest551054-21kworker/u8:2+flush-259:008:25:230
47838499139,3cyclictest478002-21kworker/u8:2+flush-259:007:30:253
47838499138,3cyclictest1-21systemd09:55:003
47838499137,4cyclictest0-21swapper/310:30:003
478384991310,2cyclictest806165-21kthreadcore12:25:183
478384991310,2cyclictest737307-21munin-plugin-st11:19:593
478384991310,2cyclictest690461-21unixbench_multi10:30:243
478384991310,2cyclictest592284-21systemd-run08:59:593
47838499130,11cyclictest561338-21awk08:30:013
47838099139,3cyclictest711409-21kworker/u8:1+events_unbound11:30:382
47838099139,3cyclictest675281-21kworker/u8:0+flush-259:010:55:202
47838099139,3cyclictest504207-21kworker/u8:4+events_unbound07:40:212
478380991310,2cyclictest618726-21sed09:25:192
47837699139,3cyclictest742440-21kworker/u8:4+flush-259:011:25:261
47837699137,4cyclictest726996-21latency_hist11:10:001
478376991311,1cyclictest0-21swapper/112:35:001
478376991310,2cyclictest654501-21kworker/u8:3+events_unbound10:25:211
478376991310,2cyclictest576999-21kworker/u8:1+flush-259:008:50:261
478376991310,2cyclictest546192-21cpuspeed_turbos08:15:141
47837299139,3cyclictest654501-21kworker/u8:3+flush-259:010:14:590
478372991312,1cyclictest61450irq/134-enp0s29f1:tx-009:35:150
478372991312,1cyclictest60650irq/133-enp0s29f1:rx-010:48:400
478372991312,1cyclictest60650irq/133-enp0s29f1:rx-009:41:490
478372991311,1cyclictest60650irq/133-enp0s29f1:rx-011:20:190
478372991311,1cyclictest60650irq/133-enp0s29f1:rx-009:25:260
478372991310,2cyclictest716930-21kworker/u8:2+events_unbound12:00:000
478372991310,2cyclictest716930-21kworker/u8:2+events_unbound12:00:000
478372991310,2cyclictest0-21swapper/011:15:000
47838499129,2cyclictest598094-21irqcore09:05:183
47838499129,2cyclictest0-21swapper/310:25:003
47838499128,3cyclictest722128-21apt-get11:05:153
47838499127,3cyclictest783907-21apt-get12:05:143
47838099128,3cyclictest711409-21kworker/u8:1+flush-259:011:18:442
47838099126,5cyclictest696053-21latency_hist10:40:012
47837699129,2cyclictest711409-21kworker/u8:1+events_unbound11:15:151
478376991210,2cyclictest654501-21kworker/u8:3+events_unbound10:35:221
478376991210,2cyclictest535622-21kworker/u8:1+flush-259:008:05:251
478372991210,1cyclictest60650irq/133-enp0s29f1:rx-011:50:240
478372991210,1cyclictest567747-21kthreadcore08:35:190
47838499118,2cyclictest748017-21fschecks_count11:30:153
47838499118,2cyclictest1631818-21kworker/u8:0-events_unbound07:35:203
47838499116,3cyclictest1-21systemd10:55:013
47838499111,2cyclictest768181-21latency_hist11:50:003
47838099118,2cyclictest0-21swapper/212:10:262
47837699119,1cyclictest599968-21kthreadcore09:05:191
47837699118,2cyclictest1631818-21kworker/u8:0+events_unbound08:10:181
47837699117,3cyclictest639304-21apt-get09:45:141
47837299119,1cyclictest60650irq/133-enp0s29f1:rx-008:15:160
47838499108,1cyclictest0-21swapper/309:25:243
47838499107,2cyclictest789108-21cpuspeed_turbos12:10:143
47838499107,2cyclictest612851-21cron09:20:003
47838499105,4cyclictest773604-21apt-get11:55:123
47838499105,4cyclictest773604-21apt-get11:55:123
47837699106,3cyclictest566697-21kworker/u8:4+flush-259:008:44:421
4783849995,3cyclictest549549-21tr08:15:193
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional