You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-15 - 21:26
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot6.osadl.org (updated Mon Sep 15, 2025 12:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
382956599141129,10cyclictest3916609-21apt-get11:39:171
382956599113107,4cyclictest3834553-21kworker/u8:0+events_unbound07:40:191
38295619911397,13cyclictest3919845-21latency_hist11:50:010
38295619911199,9cyclictest3869620-21latency_hist09:15:010
382956599110105,4cyclictest3881017-21kworker/u8:0+events_unbound09:50:221
38295659910998,7cyclictest3921693-21latency_hist11:55:001
382956599108103,4cyclictest3873112-21kworker/u8:2+events_unbound09:30:231
38295659910599,4cyclictest3858523-21kworker/u8:3+events_unbound09:00:181
38295659910499,4cyclictest3827695-21kworker/u8:2+events_unbound07:20:251
38295619910493,9cyclictest3916925-21apt-get11:40:120
38295659910297,4cyclictest3842781-21kworker/u8:0+events_unbound08:05:231
38295619910292,8cyclictest3889770-21apt-get10:15:120
38295659910196,4cyclictest3903886-21kworker/u8:0+events_unbound11:00:281
38295659910195,4cyclictest3842781-21kworker/u8:0+events_unbound08:35:201
38295659910189,9cyclictest3837736-21latency_hist07:35:021
3829561999991,5cyclictest3827695-21kworker/u8:2+flush-179:007:36:180
3829561999887,9cyclictest3913676-21apt-get11:30:130
3829561999785,10cyclictest3930452-21apt-get12:25:140
3829565999691,4cyclictest3869653-21kworker/u8:1+events_unbound09:40:271
3829565999685,8cyclictest3933375-21latency_hist12:35:011
3829565999585,7cyclictest601706-21systemd-journal07:55:001
3829561999581,10cyclictest3874903-21kworker/u8:0+flush-179:009:30:220
3829565999388,4cyclictest3873112-21kworker/u8:2+events_unbound10:05:281
3829565999388,4cyclictest3861686-21kworker/u8:4+events_unbound09:15:241
3829565999388,4cyclictest3861686-21kworker/u8:4+events_unbound08:55:241
3829565999387,4cyclictest3899070-21kworker/u8:1+events_unbound11:05:201
3829565999384,4cyclictest3921666-21kworker/u8:1+events_unbound12:05:201
3829561999267,6cyclictest601706-21systemd-journal09:26:180
3829565999186,4cyclictest3887720-21kworker/u8:3+events_unbound10:45:311
3829565999181,7cyclictest3842781-21kworker/u8:0+flush-179:008:00:241
3829565999081,3cyclictest3826174-21kworker/u8:0+events_unbound07:15:191
3829565999073,13cyclictest601706-21systemd-journal12:15:011
3829561999082,6cyclictest3921666-21kworker/u8:1+events_unbound12:20:220
3829561999082,5cyclictest3899070-21kworker/u8:1+events_unbound10:50:010
3829561998984,3cyclictest3910481-21kworker/u8:3+flush-179:011:55:250
3829561998980,6cyclictest3858523-21kworker/u8:3+events_unbound08:50:260
3829565998884,3cyclictest3895327-21kworker/u8:1+events_unbound10:30:261
3829565998783,3cyclictest3873112-21kworker/u8:2+events_unbound09:25:261
3829565998783,3cyclictest3869653-21kworker/u8:1+events_unbound09:45:141
3829565998782,4cyclictest3921666-21kworker/u8:1+events_unbound12:15:181
3829565998778,4cyclictest3933358-21kworker/u8:1+events_unbound12:35:241
3829565998776,4cyclictest3912884-21kworker/u8:2+events_unbound11:25:241
3829561998676,7cyclictest3873112-21kworker/u8:2+flush-179:010:05:230
3829565998579,4cyclictest3845775-21kworker/u8:1+events_unbound08:20:201
3829561998576,6cyclictest3861686-21kworker/u8:4+flush-179:008:55:180
3829565998480,3cyclictest3873112-21kworker/u8:2+events_unbound10:25:161
3829565998480,3cyclictest3873112-21kworker/u8:2+events_unbound10:25:151
3829561998478,4cyclictest3892734-21kworker/u8:0+events_unbound10:25:250
3829561998478,4cyclictest3892734-21kworker/u8:0+events_unbound10:25:250
3829561998478,4cyclictest3827695-21kworker/u8:2+flush-179:007:10:220
3829561998477,5cyclictest3892734-21kworker/u8:0+flush-179:010:40:220
3829565998379,3cyclictest3915668-21kworker/u8:0+events_unbound11:41:101
3829565998278,3cyclictest3881017-21kworker/u8:0+events_unbound10:00:231
3829565998275,6cyclictest3842781-21kworker/u8:0+flush-179:008:44:551
3829565998273,3cyclictest3923598-21kworker/u8:0+events_unbound12:26:151
3829561998275,4cyclictest3886318-21kworker/u8:1+flush-179:010:10:180
3829565998165,14cyclictest3840970-21kworker/u8:3+flush-179:008:00:001
3829561998175,4cyclictest3869653-21kworker/u8:1+flush-179:009:40:230
3829565997974,4cyclictest3899070-21kworker/u8:1+events_unbound11:15:221
3829565997973,5cyclictest3887720-21kworker/u8:3+events_unbound10:50:191
3829565997971,6cyclictest3827773-21kworker/u8:3+flush-179:007:30:011
3829561997971,6cyclictest3834553-21kworker/u8:0+events_unbound07:25:270
3829561997968,8cyclictest3916652-21latency_hist11:40:020
3829565997873,4cyclictest3858523-21kworker/u8:3+flush-179:008:50:291
3829561997872,4cyclictest3923598-21kworker/u8:0+flush-179:012:14:590
3829561997872,4cyclictest3845775-21kworker/u8:1+flush-179:008:10:220
3829561997865,9cyclictest3842781-21kworker/u8:0+flush-179:008:20:210
3829565997771,5cyclictest3921666-21kworker/u8:1+events_unbound12:20:111
3829565997766,9cyclictest3881017-21kworker/u8:0+flush-179:009:55:231
3829561997762,11cyclictest3871226-21apt-get09:20:000
3829565997672,3cyclictest3873112-21kworker/u8:2+events_unbound10:15:181
3829565997668,6cyclictest3851053-21journal-offline08:15:191
3829565997571,3cyclictest3869653-21kworker/u8:1+events_unbound09:35:191
3829565997570,4cyclictest3873112-21kworker/u8:2+events_unbound10:22:181
3829565997566,3cyclictest3892734-21kworker/u8:0+events_unbound10:40:201
3829561997570,3cyclictest3840944-21kworker/u8:2+flush-179:007:45:190
3829561997569,4cyclictest3864894-21kworker/u8:0+flush-179:009:20:260
3829561997564,9cyclictest3932023-21/usr/sbin/munin12:30:240
3829561997564,9cyclictest3901215-21diskstats10:50:130
3829565997470,3cyclictest3842781-21kworker/u8:0+events_unbound08:28:191
3829565997469,4cyclictest3839331-21kworker/u8:1+events_unbound07:46:201
3829561997364,6cyclictest3903886-21kworker/u8:0+flush-179:011:25:230
3829565997268,3cyclictest3899070-21kworker/u8:1+events_unbound11:20:181
3829565997268,3cyclictest3842781-21kworker/u8:0+events_unbound08:45:231
3829565997267,4cyclictest3826174-21kworker/u8:0+events_unbound07:14:191
3829565997166,4cyclictest3864894-21kworker/u8:0+events_unbound09:20:201
3829561997165,4cyclictest3887720-21kworker/u8:3+flush-179:011:00:190
3829565997066,3cyclictest3921667-21kworker/u8:2+events_unbound11:56:131
3829565996965,3cyclictest3845775-21kworker/u8:1+events_unbound08:11:191
3829565996964,4cyclictest3915668-21kworker/u8:0+events_unbound11:46:181
3829565996964,4cyclictest3887720-21kworker/u8:3+events_unbound10:56:021
3829561996961,5cyclictest3834553-21kworker/u8:0+flush-179:007:30:160
3829561996959,8cyclictest3907047-21latency_hist11:10:010
3829561996958,8cyclictest3834559-21latency_hist07:25:010
3829561996957,10cyclictest3843087-21diskstats07:50:170
3829565996864,3cyclictest3899070-21kworker/u8:1+flush-179:011:34:561
3829565996863,4cyclictest3907082-21kworker/u8:2+events_unbound11:10:311
3829561996853,8cyclictest3858448-21kworker/u8:1+events_unbound08:40:220
3829565996763,3cyclictest3842781-21kworker/u8:0+events_unbound08:33:201
3829561996750,13cyclictest601706-21systemd-journal08:30:010
3829565996661,4cyclictest3887720-21kworker/u8:3+events_unbound10:39:561
3829561996655,9cyclictest502-21rs:main0
3829561996557,5cyclictest3842781-21kworker/u8:0+events_unbound08:00:240
3829561996557,5cyclictest3840970-21kworker/u8:3+flush-179:008:05:160
3829565996443,19cyclictest3861686-21kworker/u8:4+flush-179:009:05:181
3829561996459,3cyclictest3857048-21sed08:35:000
3829565996351,9cyclictest3834553-21kworker/u8:0+flush-179:007:35:251
3829565996258,3cyclictest3921666-21kworker/u8:1+events_unbound12:03:181
3829561996256,4cyclictest3881080-21kworker/u8:3+flush-179:009:55:000
3829561996254,5cyclictest3887720-21kworker/u8:3+events_unbound10:25:010
3829561996054,4cyclictest3845775-21kworker/u8:1+flush-179:008:15:240
3829561996049,9cyclictest3842781-21kworker/u8:0+flush-179:008:35:230
3829561995956,2cyclictest0-21swapper/011:12:590
3829561995950,6cyclictest3869653-21kworker/u8:1+flush-179:009:45:180
3829561995949,7cyclictest3921666-21kworker/u8:1+flush-179:012:05:010
3829561995745,10cyclictest3860330-21apt-get08:45:090
3829565995545,8cyclictest3864894-21kworker/u8:0+flush-179:009:10:151
3829561995550,4cyclictest3834553-21kworker/u8:0+flush-179:007:40:200
3829561995549,5cyclictest3921666-21kworker/u8:1+flush-179:012:05:180
3829561995548,5cyclictest3923598-21kworker/u8:0+events_unbound12:15:020
3829565995450,3cyclictest3873112-21kworker/u8:2+events_unbound10:14:201
3829561995449,4cyclictest3933358-21kworker/u8:1+flush-179:012:35:230
3829561995447,4cyclictest3826174-21kworker/u8:0+flush-179:007:15:040
3829561995445,7cyclictest3895327-21kworker/u8:1+events_unbound10:35:180
3829561995445,6cyclictest3903886-21kworker/u8:0+events_unbound11:15:130
3829569995351,1cyclictest0-21swapper/310:48:593
3829561995146,3cyclictest3869653-21kworker/u8:1+events_unbound09:35:150
3829561995040,7cyclictest3861686-21kworker/u8:4+events_unbound09:05:230
3829561995039,8cyclictest3873112-21kworker/u8:2+flush-179:010:30:220
4505291470,14ptp4l61790irq/134-eno1-TxRx-007:10:013
382956699460,43cyclictest0-21swapper/208:42:592
3829561994642,3cyclictest3903886-21kworker/u8:0+flush-179:011:20:260
3829561994438,5cyclictest3881017-21kworker/u8:0+events_unbound09:55:170
382956699430,35cyclictest0-21swapper/212:34:582
382956699420,1cyclictest0-21swapper/211:19:562
3829561994236,4cyclictest3887720-21kworker/u8:3+events_unbound10:55:260
382956999410,5cyclictest3881042-21turbostat09:50:013
3829561994138,2cyclictest3842781-21kworker/u8:0+events_unbound07:55:170
382956699400,37cyclictest0-21swapper/207:24:552
3829569993937,1cyclictest0-21swapper/310:39:553
382956699390,36cyclictest0-21swapper/209:29:552
3829569993835,2cyclictest0-21swapper/309:31:003
3829566993835,2cyclictest0-21swapper/209:33:592
382956699380,36cyclictest0-21swapper/212:27:002
382956699380,36cyclictest0-21swapper/211:25:592
382956699380,35cyclictest0-21swapper/209:48:592
382956699380,35cyclictest0-21swapper/209:04:592
382956699380,1cyclictest0-21swapper/210:43:592
3829561993834,3cyclictest3919853-21apt-get11:50:030
3829569993734,2cyclictest0-21swapper/312:19:593
3829569993734,2cyclictest0-21swapper/308:35:593
382956699370,35cyclictest0-21swapper/210:35:592
382956699370,2cyclictest0-21swapper/210:52:592
3829569993634,1cyclictest0-21swapper/309:38:593
3829569993634,1cyclictest0-21swapper/308:59:593
3829569993634,1cyclictest0-21swapper/307:54:563
3829569993633,2cyclictest0-21swapper/311:49:593
382956999360,35cyclictest0-21swapper/311:24:583
382956999360,35cyclictest0-21swapper/308:30:593
382956999360,1cyclictest0-21swapper/310:44:553
382956999360,1cyclictest0-21swapper/310:18:593
3829566993633,2cyclictest0-21swapper/211:24:552
382956699360,1cyclictest0-21swapper/211:05:592
382956699360,1cyclictest0-21swapper/209:10:002
382956999350,1cyclictest0-21swapper/312:29:553
382956999350,1cyclictest0-21swapper/310:34:573
3829566993532,2cyclictest0-21swapper/212:36:592
3829566993532,2cyclictest0-21swapper/212:01:582
3829566993532,2cyclictest0-21swapper/210:29:552
3829566993532,2cyclictest0-21swapper/210:29:552
3829569993432,1cyclictest0-21swapper/312:02:593
3829569993331,1cyclictest0-21swapper/312:07:593
3829569993330,2cyclictest0-21swapper/312:14:583
3829569993327,5cyclictest3873089-21turbostat09:25:003
3829569993230,1cyclictest0-21swapper/312:33:583
3829569993230,1cyclictest0-21swapper/310:24:593
3829569993230,1cyclictest0-21swapper/307:58:583
3829566993230,1cyclictest0-21swapper/207:11:002
3829561993225,5cyclictest3881017-21kworker/u8:0+flush-179:010:00:330
3829569993129,1cyclictest0-21swapper/310:04:593
3829566993128,2cyclictest0-21swapper/207:40:132
382956699310,29cyclictest0-21swapper/212:20:002
382956699310,28cyclictest0-21swapper/209:11:592
3829569993028,1cyclictest0-21swapper/310:09:553
3829569993027,2cyclictest0-21swapper/311:09:583
3829569993027,2cyclictest0-21swapper/309:20:013
3829569993027,2cyclictest0-21swapper/307:43:583
382956999300,1cyclictest0-21swapper/311:50:593
3829566993028,1cyclictest0-21swapper/207:39:592
382956699300,2cyclictest0-21swapper/211:45:592
382956699300,2cyclictest0-21swapper/211:32:592
382956699300,2cyclictest0-21swapper/209:40:142
382956699300,28cyclictest0-21swapper/210:30:132
3829569992927,1cyclictest0-21swapper/311:04:593
3829569992927,1cyclictest0-21swapper/310:55:143
382956999290,28cyclictest0-21swapper/311:30:143
382956999290,1cyclictest0-21swapper/308:20:143
3829566992926,2cyclictest0-21swapper/211:10:142
3829566992926,2cyclictest0-21swapper/210:10:142
3829566992926,2cyclictest0-21swapper/208:39:582
3829566992926,2cyclictest0-21swapper/208:25:142
382956699290,28cyclictest0-21swapper/208:50:152
382956699290,27cyclictest0-21swapper/209:50:122
382956699290,27cyclictest0-21swapper/208:15:162
3829569992826,1cyclictest0-21swapper/311:42:003
3829569992826,1cyclictest0-21swapper/309:50:013
3829569992826,1cyclictest0-21swapper/307:30:593
3829569992826,1cyclictest0-21swapper/307:15:133
3829569992825,2cyclictest0-21swapper/311:55:133
3829569992825,2cyclictest0-21swapper/309:28:593
382956999280,27cyclictest0-21swapper/309:10:143
382956999280,26cyclictest0-21swapper/312:20:143
382956999280,26cyclictest0-21swapper/311:25:143
382956999280,26cyclictest0-21swapper/307:25:153
3829566992825,2cyclictest0-21swapper/210:45:162
3829566992825,2cyclictest0-21swapper/207:55:162
382956699280,2cyclictest0-21swapper/210:20:112
382956699280,2cyclictest0-21swapper/207:48:592
382956699280,26cyclictest0-21swapper/212:10:152
382956699280,25cyclictest0-21swapper/207:19:592
3829569992725,1cyclictest0-21swapper/308:18:593
3829569992725,1cyclictest0-21swapper/308:07:003
382956999270,26cyclictest0-21swapper/311:36:593
382956999270,26cyclictest0-21swapper/311:13:593
382956999270,26cyclictest0-21swapper/307:11:583
3829566992724,2cyclictest0-21swapper/208:35:002
3829566992724,2cyclictest0-21swapper/208:08:582
382956699270,25cyclictest0-21swapper/209:19:592
3829569992624,1cyclictest0-21swapper/312:38:003
3829569992623,2cyclictest0-21swapper/307:49:593
3829566992624,1cyclictest0-21swapper/210:15:022
3829566992623,2cyclictest0-21swapper/208:55:582
382956699260,2cyclictest0-21swapper/211:44:592
382956699260,2cyclictest0-21swapper/209:38:002
382956699260,2cyclictest0-21swapper/208:24:552
382956699260,24cyclictest0-21swapper/207:32:582
3829569992523,1cyclictest0-21swapper/310:13:593
382956999250,2cyclictest0-21swapper/309:03:003
382956999250,24cyclictest0-21swapper/309:44:593
3829566992523,1cyclictest0-21swapper/212:25:012
3829566992523,1cyclictest0-21swapper/208:15:012
3829566992522,2cyclictest0-21swapper/208:49:002
3829566992521,2cyclictest0-21swapper/207:28:592
382956699250,24cyclictest0-21swapper/212:05:592
4505291240,5ptp4l61790irq/134-eno1-TxRx-008:49:553
4505291240,5ptp4l61790irq/134-eno1-TxRx-007:40:003
4505291240,3ptp4l441ktimers/308:44:573
3829569992423,1cyclictest0-21swapper/310:51:593
3829566992422,1cyclictest0-21swapper/209:59:592
4505291230,3ptp4l61790irq/134-eno1-TxRx-007:25:013
4505291230,3ptp4l441ktimers/310:00:003
4505291230,3ptp4l441ktimers/308:04:593
3829569992321,1cyclictest0-21swapper/308:52:583
3829566992320,2cyclictest0-21swapper/211:59:552
3829566992320,2cyclictest0-21swapper/211:37:592
3829566992319,2cyclictest0-21swapper/210:56:592
4505291220,3ptp4l441ktimers/310:29:553
4505291220,3ptp4l441ktimers/310:29:553
4505291220,3ptp4l441ktimers/309:09:553
4505291220,3ptp4l441ktimers/308:29:573
3829569992219,2cyclictest0-21swapper/308:12:593
3829569992216,5cyclictest3910236-21turbostat11:20:003
3829561992218,3cyclictest0-21swapper/007:10:020
382956599184,6cyclictest29-21ksoftirqd/107:10:011
3829566991715,1cyclictest0-21swapper/211:50:032
3829566991714,2cyclictest0-21swapper/210:03:592
3829566991713,2cyclictest0-21swapper/208:02:002
3829566991613,2cyclictest0-21swapper/207:52:592
3829566991412,1cyclictest0-21swapper/209:25:002
382956699107,2cyclictest0-21swapper/211:01:592
382956699100,9cyclictest0-21swapper/210:10:002
38295669930,2cyclictest0-21swapper/207:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional