You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-21 - 17:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Fri Nov 21, 2025 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
693961993722,9cyclictest771897-21/usr/sbin/munin08:25:151
693966993530,3cyclictest704477-21kworker/u8:0+events_unbound07:50:192
693966993226,5cyclictest761363-21latency_hist08:15:002
693966993225,5cyclictest2698871-21systemd-journal11:35:002
693961993228,3cyclictest838905-21kworker/u8:2+flush-259:009:40:241
693961993225,5cyclictest2698871-21systemd-journal12:20:001
693961993225,4cyclictest912164-21latency_hist10:40:001
693966992923,4cyclictest906579-21apt-get10:35:012
693961992922,4cyclictest714574-21latency_hist07:30:001
693966992825,2cyclictest927852-21kworker/u8:3+flush-259:010:55:202
693966992823,3cyclictest704477-21kworker/u8:0+events_unbound08:05:142
693972992723,3cyclictest907231-21kworker/u8:1+flush-259:011:20:013
693966992621,4cyclictest854559-21kworker/u8:1+flush-259:010:00:202
693966992621,3cyclictest870215-21kworker/u8:0+flush-259:010:25:002
693966992621,3cyclictest704477-21kworker/u8:0+flush-259:007:30:152
693958992622,3cyclictest704477-21kworker/u8:0+events_unbound08:05:000
693972992522,2cyclictest656182-21kworker/u8:2+flush-259:007:20:003
693972992520,3cyclictest963965-21/usr/sbin/munin11:30:193
693972992519,5cyclictest2698871-21systemd-journal11:10:203
693961992520,3cyclictest906797-21gpgv10:35:011
693972992421,2cyclictest999592-21kworker/u8:3+events_unbound12:25:013
693972992420,3cyclictest781487-21kworker/u8:0+flush-259:008:35:163
693966992420,3cyclictest943313-21kworker/u8:2+events_unbound11:15:192
693966992420,3cyclictest771653-21kworker/u8:1+flush-259:008:50:242
693961992420,3cyclictest704477-21kworker/u8:0+flush-259:008:10:151
693972992320,2cyclictest755814-21kworker/u8:2+events_unbound08:30:253
693972992319,3cyclictest704477-21kworker/u8:0+events_unbound07:25:223
693966992319,3cyclictest802518-21kworker/u8:3+flush-259:009:00:192
693966992319,3cyclictest771653-21kworker/u8:1+flush-259:008:37:292
693961992318,3cyclictest802518-21kworker/u8:3+flush-259:009:20:161
693961992315,5cyclictest709432-21latency_hist07:25:001
693961992314,8cyclictest714830-21apt-get07:30:101
693972992218,3cyclictest870468-21kworker/u8:4+flush-259:010:25:243
693972992218,3cyclictest704477-21kworker/u8:0+flush-259:008:25:003
693966992219,2cyclictest745956-21kworker/u8:3+events_unbound08:00:232
693966992218,3cyclictest844789-21kworker/u8:3+events_unbound09:45:222
693966992218,3cyclictest755814-21kworker/u8:2+events_unbound08:30:182
693961992216,4cyclictest761586-21kworker/u8:4+flush-259:008:20:181
693961992214,6cyclictest974027-21latency_hist11:40:021
693972992119,1cyclictest870468-21kworker/u8:4+events_unbound11:20:153
693972992117,3cyclictest927886-21apt-get10:55:113
693972992117,3cyclictest870215-21kworker/u8:0+flush-259:010:40:003
693972992116,4cyclictest745471-21latency_hist08:00:003
693966992117,3cyclictest838905-21kworker/u8:2+flush-259:009:40:252
693966992117,3cyclictest761586-21kworker/u8:4+events_unbound08:45:152
693966992116,3cyclictest802842-21/usr/sbin/munin08:55:162
693961992117,2cyclictest994892-21apt-get12:00:131
693961992113,6cyclictest2698871-21systemd-journal09:15:001
693958992118,2cyclictest854559-21kworker/u8:1+flush-259:010:00:230
693958992118,2cyclictest704477-21kworker/u8:0+events_unbound07:30:200
693958992117,3cyclictest740570-21apt-get07:55:110
693958992116,4cyclictest937944-21latency_hist11:05:000
693958992115,4cyclictest963730-21latency_hist11:30:010
693972992017,2cyclictest958027-21kworker/u8:3+flush-259:011:30:013
693972992017,2cyclictest885543-21kworker/u8:2+events_unbound10:40:203
693972992017,2cyclictest870215-21kworker/u8:0+flush-259:010:55:003
693972992017,2cyclictest694128-21kworker/u8:1+events_unbound07:20:173
693972992015,4cyclictest2698871-21systemd-journal10:20:003
693966992017,2cyclictest999592-21kworker/u8:3+flush-259:012:39:512
693966992017,2cyclictest875629-21kworker/u8:5+flush-259:010:05:262
693966992016,3cyclictest844789-21kworker/u8:3+events_unbound09:52:202
693966992016,3cyclictest823279-21kworker/u8:0+flush-259:009:25:232
693966992016,2cyclictest1005111-21kworker/u8:4+flush-259:012:15:202
693961992016,3cyclictest933020-21kworker/u8:0+flush-259:011:25:241
693961992014,5cyclictest704477-21kworker/u8:0+flush-259:008:05:131
69397299190,18cyclictest1-21systemd09:15:013
693966991916,2cyclictest974045-21kworker/u8:1+flush-259:012:20:202
693966991916,2cyclictest802518-21kworker/u8:3+events_unbound09:10:172
693966991916,2cyclictest761586-21kworker/u8:4+flush-259:008:25:192
693966991915,3cyclictest984558-21kworker/u8:2+events_unbound11:50:162
693966991915,3cyclictest958027-21kworker/u8:3+flush-259:011:45:152
693966991915,3cyclictest870468-21kworker/u8:4+events_unbound11:09:202
693966991915,3cyclictest870215-21kworker/u8:0+events_unbound10:35:182
693966991911,6cyclictest2698871-21systemd-journal10:15:002
693961991916,2cyclictest885934-21turbostat.cron10:15:001
693961991913,4cyclictest933020-21kworker/u8:0+flush-259:011:55:141
693961991911,7cyclictest849410-21idleruntime-cro09:39:591
69396199191,16cyclictest864839-21awk09:54:591
693958991916,2cyclictest0-21swapper/012:15:000
693958991915,3cyclictest802518-21kworker/u8:3+events_unbound09:10:190
693958991915,3cyclictest792542-21/usr/sbin/munin08:45:200
693958991915,3cyclictest750630-21apt-get08:05:020
693972991815,2cyclictest860382-21irqcore09:50:173
693972991813,4cyclictest891378-21/usr/sbin/munin10:20:153
693966991814,3cyclictest885543-21kworker/u8:2+events_unbound10:15:152
693966991814,3cyclictest870215-21kworker/u8:0+events_unbound10:54:192
693966991813,4cyclictest943346-21apt-get11:10:112
693961991812,4cyclictest2698871-21systemd-journal11:46:201
693961991810,6cyclictest869558-21ntpq09:55:201
693958991814,3cyclictest933020-21kworker/u8:0+flush-259:011:35:010
693972991714,2cyclictest870468-21kworker/u8:4+flush-259:011:05:253
693972991714,2cyclictest704477-21kworker/u8:0+flush-259:008:10:003
693972991713,3cyclictest933020-21kworker/u8:0+events_unbound11:55:003
693966991714,2cyclictest1005111-21kworker/u8:4+flush-259:012:25:202
693966991713,3cyclictest907231-21kworker/u8:1+flush-259:010:40:222
693966991713,3cyclictest838905-21kworker/u8:2+flush-259:009:55:232
693966991713,3cyclictest2831574-21kworker/u8:3+flush-259:007:45:002
693966991712,4cyclictest870468-21kworker/u8:4+flush-259:010:29:492
69396199174,12cyclictest1020431-21sed12:25:001
69396199172,13cyclictest0-21swapper/110:00:181
69396199171,4cyclictest828418-21sh09:20:001
69396199171,3cyclictest843797-21cron09:34:591
693961991713,3cyclictest1035832-21gpgv12:39:591
693961991712,4cyclictest2698871-21systemd-journal12:35:001
693961991711,4cyclictest725115-21apt-get07:40:151
69396199170,6cyclictest1004883-21dump-pmu-power12:10:001
693958991714,2cyclictest907231-21kworker/u8:1+flush-259:011:15:150
693958991714,2cyclictest704477-21kworker/u8:0+flush-259:007:25:000
693958991714,2cyclictest61450irq/134-enp0s29f1:tx-010:55:190
693972991613,2cyclictest802518-21kworker/u8:3+flush-259:008:55:183
693972991613,2cyclictest771653-21kworker/u8:1+flush-259:008:50:163
693972991612,3cyclictest933020-21kworker/u8:0+events_unbound11:00:243
693966991613,2cyclictest995911-21kthreadcore12:00:172
693966991613,2cyclictest974045-21kworker/u8:1+flush-259:011:40:202
693966991612,3cyclictest933020-21kworker/u8:0+flush-259:011:04:392
693966991612,3cyclictest761586-21kworker/u8:4+flush-259:008:20:212
693966991612,3cyclictest694128-21kworker/u8:1+events_unbound07:15:212
693966991612,2cyclictest0-21swapper/209:10:002
693966991611,4cyclictest843819-21apt-get09:35:022
693966991611,4cyclictest1004867-21cron12:10:002
693966991611,3cyclictest761586-21kworker/u8:4+events_unbound08:43:222
69396199169,5cyclictest2698871-21systemd-journal08:20:001
69396199166,9cyclictest0-21swapper/110:15:251
69396199162,12cyclictest0-21swapper/107:40:001
69396199161,4cyclictest958547-21idleruntime-cro11:25:011
69396199161,4cyclictest792990-21tr08:45:171
69396199161,4cyclictest697123-21sshd07:10:201
69396199161,3cyclictest806824-21grep08:55:201
69396199161,3cyclictest781934-21apt08:35:001
693961991613,2cyclictest781487-21kworker/u8:0+flush-259:008:40:241
69396199161,2cyclictest968026-21ntpq11:30:201
69396199161,2cyclictest931906-21ntpq10:55:211
69396199160,6cyclictest1010050-21turbostat12:14:591
69396199160,4cyclictest927618-21cron10:55:001
69396199160,4cyclictest750630-21apt-get08:05:001
69396199160,3cyclictest802615-21awk08:55:001
69396199160,3cyclictest704239-21basename07:20:001
693958991613,2cyclictest60650irq/133-enp0s29f1:rx-012:30:270
693958991613,2cyclictest60650irq/133-enp0s29f1:rx-011:10:220
693958991613,2cyclictest60650irq/133-enp0s29f1:rx-009:15:190
693958991612,3cyclictest974045-21kworker/u8:1+events_unbound12:25:150
693958991612,3cyclictest823279-21kworker/u8:0+events_unbound09:20:200
693958991612,3cyclictest755814-21kworker/u8:2+flush-259:008:25:250
693958991612,2cyclictest0-21swapper/012:39:590
693958991611,4cyclictest864834-21apt-get09:55:000
69395899161,13cyclictest2698871-21systemd-journal08:25:000
693972991513,1cyclictest0-21swapper/310:30:193
693972991512,2cyclictest933020-21kworker/u8:0+flush-259:011:35:203
693972991512,2cyclictest2831574-21kworker/u8:3+flush-259:007:40:263
693972991512,2cyclictest1025273-21kworker/u8:0+flush-259:012:29:593
693972991512,1cyclictest0-21swapper/308:05:003
693972991511,3cyclictest2698871-21systemd-journal12:17:003
693972991510,4cyclictest979161-21latency_hist11:45:013
693966991512,2cyclictest958027-21kworker/u8:3+events_unbound11:25:142
693966991512,2cyclictest823279-21kworker/u8:0+events_unbound09:34:172
693966991511,3cyclictest933020-21kworker/u8:0+events_unbound11:35:152
693966991511,3cyclictest823279-21kworker/u8:0+events_unbound09:21:202
693966991511,3cyclictest761586-21kworker/u8:4+flush-259:008:16:252
693966991511,3cyclictest694128-21kworker/u8:1+flush-259:007:10:262
693966991510,3cyclictest984558-21kworker/u8:2+flush-259:011:55:142
69396199152,2cyclictest837-21snmpd09:00:131
69396199151,4cyclictest834099-21hddtemp_smartct09:25:141
69396199151,3cyclictest740297-21idleruntime-cro07:55:001
69396199151,3cyclictest516-21gdbus11:05:001
69396199151,3cyclictest2698871-21systemd-journal08:00:001
69396199151,2cyclictest948253-21cron11:15:011
69396199151,2cyclictest891480-21grep10:20:131
69396199151,2cyclictest817215-21ntpq09:05:261
69396199151,2cyclictest782619-21tr08:35:161
69396199150,4cyclictest984675-21cpuspeed_turbos11:50:131
69396199150,4cyclictest952486-21ntpq11:15:191
69396199150,4cyclictest917665-21tr10:45:131
69396199150,4cyclictest732578-21ps07:45:161
69396199150,4cyclictest1020761-21grep12:25:121
69396199150,3cyclictest938621-21sort11:05:171
69396199150,3cyclictest912842-21sort10:40:201
69396199150,3cyclictest876891-21kthreadcore10:05:191
69396199150,3cyclictest854920-21cut09:45:131
69396199150,2cyclictest901425-21munin-run10:30:001
69396199150,2cyclictest6939672sleep107:10:011
693958991513,1cyclictest60650irq/133-enp0s29f1:rx-011:59:090
693958991513,1cyclictest60650irq/133-enp0s29f1:rx-011:20:180
693958991513,1cyclictest60650irq/133-enp0s29f1:rx-010:35:240
693958991513,1cyclictest60650irq/133-enp0s29f1:rx-009:51:220
693958991513,1cyclictest60650irq/133-enp0s29f1:rx-009:25:220
693958991513,1cyclictest60650irq/133-enp0s29f1:rx-009:00:150
693958991513,1cyclictest60650irq/133-enp0s29f1:rx-008:10:130
693958991513,1cyclictest60650irq/133-enp0s29f1:rx-007:15:190
693958991512,2cyclictest60650irq/133-enp0s29f1:rx-012:20:410
693958991512,2cyclictest60650irq/133-enp0s29f1:rx-012:15:240
693958991512,2cyclictest60650irq/133-enp0s29f1:rx-011:05:250
693958991512,2cyclictest60650irq/133-enp0s29f1:rx-010:40:200
693958991512,2cyclictest60650irq/133-enp0s29f1:rx-010:30:170
693958991511,3cyclictest838905-21kworker/u8:2+flush-259:009:35:240
693972991412,1cyclictest0-21swapper/310:15:003
693972991412,1cyclictest0-21swapper/308:45:223
693972991411,2cyclictest2831574-21kworker/u8:3+events_unbound07:45:193
69396699149,3cyclictest704477-21kworker/u8:0+events_unbound07:59:012
693966991411,2cyclictest694128-21kworker/u8:1+events_unbound07:20:252
693966991410,3cyclictest870468-21kworker/u8:4+events_unbound11:20:172
693966991410,3cyclictest2831574-21kworker/u8:3+events_unbound07:35:142
69396199141,3cyclictest974292-21apt-get11:40:121
693958991413,1cyclictest60650irq/133-enp0s29f1:rx-008:32:480
693958991412,2cyclictest60650irq/133-enp0s29f1:rx-008:35:120
693958991412,1cyclictest60650irq/133-enp0s29f1:rx-012:02:430
693958991412,1cyclictest60650irq/133-enp0s29f1:rx-011:40:190
693958991412,1cyclictest60650irq/133-enp0s29f1:rx-010:52:410
693958991412,1cyclictest60650irq/133-enp0s29f1:rx-010:23:550
693958991412,1cyclictest60650irq/133-enp0s29f1:rx-010:15:180
693958991412,1cyclictest60650irq/133-enp0s29f1:rx-010:10:160
693958991412,1cyclictest60650irq/133-enp0s29f1:rx-009:48:340
693958991412,1cyclictest60650irq/133-enp0s29f1:rx-007:50:200
693958991412,1cyclictest60650irq/133-enp0s29f1:rx-007:35:180
693958991412,1cyclictest60650irq/133-enp0s29f1:rx-007:10:210
693958991411,3cyclictest885543-21kworker/u8:2+flush-259:010:25:190
693958991411,2cyclictest60650irq/133-enp0s29f1:rx-008:18:080
693958991411,2cyclictest0-21swapper/009:05:290
69397299139,3cyclictest870709-21apt-get10:00:103
693972991311,1cyclictest0-21swapper/307:35:003
693972991310,2cyclictest999592-21kworker/u8:3+flush-259:012:30:203
69396699139,3cyclictest999592-21kworker/u8:3+events_unbound12:30:212
69396699139,3cyclictest2831574-21kworker/u8:3+flush-259:007:45:262
69396699138,4cyclictest802518-21kworker/u8:3+events_unbound09:17:002
693966991311,1cyclictest0-21swapper/210:50:012
693958991312,1cyclictest60650irq/133-enp0s29f1:rx-007:26:410
693958991311,1cyclictest60650irq/133-enp0s29f1:rx-011:45:210
693958991311,1cyclictest60650irq/133-enp0s29f1:rx-011:35:150
693958991310,2cyclictest854050-21unixbench_multi09:40:260
693958991310,2cyclictest60650irq/133-enp0s29f1:rx-007:45:200
69397299129,2cyclictest999592-21kworker/u8:3+flush-259:012:05:213
69397299129,2cyclictest1009590-21unixbench_multi12:10:253
69397299129,2cyclictest0-21swapper/307:55:013
69397299129,1cyclictest0-21swapper/309:25:153
69395899129,2cyclictest797761-21cpuspeed_turbos08:50:140
69395899129,2cyclictest60650irq/133-enp0s29f1:rx-009:30:140
693958991210,1cyclictest60650irq/133-enp0s29f1:rx-012:05:270
693958991210,1cyclictest60650irq/133-enp0s29f1:rx-008:43:200
693958991210,1cyclictest0-21swapper/011:50:190
69397299119,1cyclictest61750irq/140-enp0s29f1:tx-309:55:003
69396699117,3cyclictest2831574-21kworker/u8:3+flush-259:007:27:262
69397299107,2cyclictest849729-21cpuspeed_turbos09:40:153
69397299106,3cyclictest844127-21gpgv09:35:013
69397299105,4cyclictest720280-21fw_forwarded_lo07:35:153
6939729997,1cyclictest876004-21tr10:05:163
6939729996,2cyclictest989475-21turbostat11:59:553
6939729996,2cyclictest830961-21tr09:20:173
6939729995,3cyclictest981217-21cut11:45:183
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional