You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-05-25 - 06:58
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot6.osadl.org (updated Fri May 24, 2024 12:44:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
91167799121112,6cyclictest924329-21latency_hist07:50:010
91167799121102,17cyclictest1012004-21gpgv12:20:010
91168199119114,3cyclictest945316-21kworker/u8:0+flush-179:009:05:191
91167799115102,9cyclictest916350-21latency_hist07:25:010
91168199113108,4cyclictest958268-21kworker/u8:3+events_unbound09:55:331
9116819911097,11cyclictest922779-21kworker/u8:0+flush-179:007:50:221
9116779910699,4cyclictest921126-21kworker/u8:2+flush-179:008:20:150
9116779910694,9cyclictest1000216-21apt-get11:44:590
9116819910398,4cyclictest913189-21kworker/u8:0+events_unbound07:30:201
9116819910398,4cyclictest913189-21kworker/u8:0+events_unbound07:30:191
9116819910196,4cyclictest1005383-21kworker/u8:3+events_unbound12:00:191
9116819910191,7cyclictest985870-21latency_hist11:00:011
911681999995,3cyclictest942926-21kworker/u8:1+events_unbound09:15:281
911681999994,4cyclictest921126-21kworker/u8:2+events_unbound08:25:161
911681999987,10cyclictest2158010-21systemd-journal11:50:011
911681999984,8cyclictest979481-21kworker/u8:2+events_unbound11:00:311
911677999980,14cyclictest929128-21apt-get08:05:000
911681999793,3cyclictest955142-21kworker/u8:1+events_unbound09:25:191
911681999687,6cyclictest992258-21latency_hist11:20:001
911681999684,9cyclictest994094-21apt-get11:25:121
911677999386,5cyclictest958268-21kworker/u8:3+flush-179:009:45:200
911677999382,8cyclictest529-21rs:main0
911677999184,5cyclictest929102-21kworker/u8:0+flush-179:008:10:000
911681999079,8cyclictest961738-21latency_hist09:45:011
911681998983,4cyclictest999401-21kworker/u8:0+events_unbound11:55:191
911681998980,7cyclictest917936-21latency_hist07:30:011
911681998979,7cyclictest958285-21latency_hist09:35:011
911681998976,9cyclictest2158010-21systemd-journal12:20:011
911681998976,9cyclictest2158010-21systemd-journal12:20:011
911677998979,7cyclictest995413-21kworker/u8:0+flush-179:011:30:160
911677998975,11cyclictest970205-21apt-get10:10:120
911681998882,4cyclictest1011838-21kworker/u8:0+flush-179:012:25:271
911681998875,11cyclictest979481-21kworker/u8:2+flush-179:012:20:011
911677998882,4cyclictest962848-21kworker/u8:1+flush-179:009:50:290
911677998778,6cyclictest529-21rs:main0
911677998775,10cyclictest955366-21apt-get09:25:120
911681998673,10cyclictest1002308-21apt-get11:50:121
911681998580,3cyclictest921126-21kworker/u8:2+events_unbound08:20:171
911681998480,3cyclictest939589-21kworker/u8:2+events_unbound08:35:231
911681998479,4cyclictest973114-21kworker/u8:1+events_unbound11:20:231
911681998478,4cyclictest973117-21kworker/u8:3+events_unbound10:25:211
911681998476,6cyclictest940559-21kworker/u8:3+flush-179:008:50:211
911677998475,7cyclictest973117-21kworker/u8:3+flush-179:010:35:010
911681998378,4cyclictest981138-21kworker/u8:3+events_unbound11:05:311
911681998378,4cyclictest942926-21kworker/u8:1+flush-179:009:00:261
911681998377,5cyclictest973114-21kworker/u8:1+flush-179:010:20:261
911681998373,7cyclictest973143-21latency_hist10:20:021
911681998373,7cyclictest973143-21latency_hist10:20:011
911681998277,4cyclictest958268-21kworker/u8:3+events_unbound09:45:291
911681998277,4cyclictest1005383-21kworker/u8:3+events_unbound12:05:241
911681998268,13cyclictest935773-21kworker/u8:0+flush-179:008:44:561
911677998276,5cyclictest1013716-21kworker/u8:1+events_unbound12:30:180
911677998271,8cyclictest943697-21latency_hist08:50:010
911681998175,4cyclictest979481-21kworker/u8:2+events_unbound11:30:291
911681998167,11cyclictest932798-21apt-get08:15:141
911685998051,27cyclictest0-21swapper/307:57:003
911681998075,4cyclictest979481-21kworker/u8:2+events_unbound10:45:321
911681998075,4cyclictest904247-21kworker/u8:3+flush-179:007:20:311
911681998075,4cyclictest892929-21kworker/u8:2+flush-179:007:15:261
911677998074,5cyclictest921126-21kworker/u8:2+flush-179:007:39:570
911681997975,3cyclictest929102-21kworker/u8:0+flush-179:008:10:241
911681997968,8cyclictest990642-21latency_hist11:15:011
911677997971,6cyclictest931212-21apt-get08:10:110
911677997970,6cyclictest951902-21kworker/u8:2+events_unbound09:35:190
911681997874,3cyclictest951902-21kworker/u8:2+flush-179:009:20:341
911681997872,4cyclictest981138-21kworker/u8:3+events_unbound10:50:111
911681997863,13cyclictest973114-21kworker/u8:1+flush-179:011:40:001
911677997869,7cyclictest951913-21latency_hist09:15:010
911681997773,3cyclictest929102-21kworker/u8:0+flush-179:008:09:561
911681997772,4cyclictest940559-21kworker/u8:3+flush-179:008:45:221
911681997772,4cyclictest913189-21kworker/u8:0+events_unbound07:35:301
911677997771,4cyclictest955142-21kworker/u8:1+flush-179:009:30:230
911681997670,4cyclictest979481-21kworker/u8:2+events_unbound11:40:191
911681997670,4cyclictest973117-21kworker/u8:3+events_unbound10:30:191
911677997670,4cyclictest968145-21gpgv10:05:000
911681997570,4cyclictest951902-21kworker/u8:2+events_unbound09:35:251
911681997570,4cyclictest925950-21kworker/u8:1+flush-179:008:00:261
911681997569,4cyclictest973117-21kworker/u8:3+events_unbound10:35:141
911681997468,4cyclictest962848-21kworker/u8:1+events_unbound10:10:181
911677997366,5cyclictest958268-21kworker/u8:3+flush-179:009:44:550
911681997266,5cyclictest966478-21kworker/u8:0+events_unbound10:00:281
911681997253,17cyclictest904247-21kworker/u8:3+flush-179:008:00:001
911677997263,6cyclictest984274-21latency_hist10:55:000
911681997167,3cyclictest940559-21kworker/u8:3+events_unbound08:55:261
911681997166,4cyclictest962848-21kworker/u8:1+flush-179:009:50:251
911677997162,7cyclictest1015347-21gpgv12:30:000
911677997154,8cyclictest916385-21kworker/u8:1+flush-179:007:35:000
911677997154,8cyclictest916385-21kworker/u8:1+flush-179:007:35:000
911685997063,5cyclictest951925-21turbostat09:15:003
911681997066,3cyclictest973114-21kworker/u8:1+flush-179:010:40:161
911681997065,4cyclictest1005383-21kworker/u8:3+events_unbound12:10:171
911677997065,3cyclictest979481-21kworker/u8:2+flush-179:010:40:300
911677997062,5cyclictest979481-21kworker/u8:2+events_unbound12:35:140
911681996965,3cyclictest979481-21kworker/u8:2+flush-179:012:35:311
911681996959,7cyclictest529-21rs:main1
911677996962,5cyclictest981138-21kworker/u8:3+flush-179:011:10:250
911677996761,4cyclictest973114-21kworker/u8:1+flush-179:011:35:300
911677996760,5cyclictest979481-21kworker/u8:2+flush-179:011:15:230
911677996660,4cyclictest921126-21kworker/u8:2+flush-179:008:15:200
911677996659,5cyclictest940559-21kworker/u8:3+flush-179:008:54:550
911677996658,5cyclictest935773-21kworker/u8:0+flush-179:008:40:160
911677996656,7cyclictest2158010-21systemd-journal11:21:170
911681996459,4cyclictest904247-21kworker/u8:3+events_unbound07:41:191
911681996454,4cyclictest904247-21kworker/u8:3+events_unbound07:45:211
911677996458,4cyclictest979481-21kworker/u8:2+flush-179:010:45:330
911685996360,2cyclictest0-21swapper/308:14:593
911681996356,5cyclictest921126-21kworker/u8:2+flush-179:008:30:151
911681996253,7cyclictest904247-21kworker/u8:3+events_unbound07:15:001
911677996256,4cyclictest921126-21kworker/u8:2+flush-179:007:40:140
911677996250,9cyclictest1005383-21kworker/u8:3+events_unbound12:20:250
911677996250,9cyclictest1005383-21kworker/u8:3+events_unbound12:20:250
911677996156,2cyclictest922779-21kworker/u8:0+flush-179:007:50:290
911677996155,4cyclictest940559-21kworker/u8:3+events_unbound08:55:280
911677996148,11cyclictest950075-21apt-get09:10:000
911677996053,6cyclictest953492-21kworker/u8:0+events_unbound09:25:000
911677995953,4cyclictest973114-21kworker/u8:1+flush-179:011:05:010
911677995950,7cyclictest979481-21kworker/u8:2+events_unbound12:00:280
911677995949,8cyclictest937869-21kworker/u8:1+events_unbound08:35:000
911677995948,9cyclictest987707-21/usr/sbin/munin11:05:190
911677995852,4cyclictest973114-21kworker/u8:1+flush-179:010:20:160
911677995847,9cyclictest916607-21apt-get07:25:120
911677995750,5cyclictest1005383-21kworker/u8:3+events_unbound12:05:190
911677995650,5cyclictest921126-21kworker/u8:2+flush-179:007:55:310
911677995648,5cyclictest1005383-21kworker/u8:3+flush-179:012:10:340
911677995548,5cyclictest942926-21kworker/u8:1+flush-179:009:15:150
911677995447,5cyclictest979481-21kworker/u8:2+flush-179:010:55:320
911677995444,7cyclictest962848-21kworker/u8:1+events_unbound10:05:010
911677995348,3cyclictest979481-21kworker/u8:2+events_unbound11:45:320
911677995146,4cyclictest892929-21kworker/u8:2+flush-179:007:10:130
911677995144,5cyclictest892929-21kworker/u8:2+events_unbound07:15:120
911677995140,8cyclictest1005256-21apt-get12:00:000
911677995045,4cyclictest971527-21kworker/u8:0+flush-179:010:15:120
911677995045,4cyclictest971527-21kworker/u8:0+flush-179:010:15:120
911677995044,4cyclictest973114-21kworker/u8:1+events_unbound10:25:290
911677994943,5cyclictest921126-21kworker/u8:2+flush-179:008:25:270
911681994843,4cyclictest1005383-21kworker/u8:3+events_unbound12:30:211
911677994742,3cyclictest997003-21kworker/u8:3+flush-179:011:50:340
4652991470,13ptp4l441ktimers/307:10:003
911677994640,4cyclictest939589-21kworker/u8:2+events_unbound08:35:190
911677994540,3cyclictest992296-21kworker/u8:3+flush-179:011:25:300
91168399440,42cyclictest0-21swapper/209:32:002
911681994338,3cyclictest951950-21systemd-run09:15:001
91168399420,40cyclictest0-21swapper/211:35:112
91168599410,40cyclictest0-21swapper/310:42:003
911685994037,2cyclictest0-21swapper/309:34:593
91168599400,39cyclictest992241-21turbostat11:20:003
911683993937,1cyclictest0-21swapper/208:15:152
911685993836,1cyclictest0-21swapper/309:15:113
911685993835,2cyclictest0-21swapper/309:06:593
911683993835,2cyclictest0-21swapper/210:37:592
91167799380,37cyclictest0-21swapper/010:35:580
91168599370,36cyclictest0-21swapper/312:39:003
911683993734,2cyclictest0-21swapper/211:45:582
911683993734,2cyclictest0-21swapper/209:44:542
911683993734,2cyclictest0-21swapper/209:14:592
91168399370,35cyclictest0-21swapper/210:51:592
911685993634,1cyclictest0-21swapper/308:28:593
91168599360,35cyclictest0-21swapper/310:14:003
91168599360,35cyclictest0-21swapper/308:17:593
911683993633,2cyclictest0-21swapper/211:28:592
911683993633,2cyclictest0-21swapper/209:17:002
911685993533,1cyclictest0-21swapper/312:32:593
911685993533,1cyclictest0-21swapper/310:31:593
911685993532,2cyclictest0-21swapper/310:06:583
911685993532,2cyclictest0-21swapper/308:55:593
91168599350,34cyclictest0-21swapper/312:09:553
911683993532,2cyclictest0-21swapper/211:19:582
911683993532,2cyclictest0-21swapper/207:16:002
911683993531,2cyclictest0-21swapper/208:35:592
911685993432,1cyclictest0-21swapper/310:20:593
911685993431,2cyclictest0-21swapper/312:21:003
911685993431,2cyclictest0-21swapper/312:21:003
911685993431,2cyclictest0-21swapper/307:54:593
911685993431,2cyclictest0-21swapper/307:12:003
911683993431,2cyclictest0-21swapper/208:59:552
4652991340,3ptp4l61790irq/134-eno1-TxRx-010:00:113
911685993330,2cyclictest0-21swapper/311:31:593
911685993330,2cyclictest0-21swapper/310:35:583
911683993330,2cyclictest0-21swapper/210:09:592
91168399330,31cyclictest0-21swapper/211:40:132
911685993230,1cyclictest0-21swapper/309:58:003
911685993230,1cyclictest0-21swapper/308:43:593
911685993230,1cyclictest0-21swapper/307:24:553
911685993229,2cyclictest0-21swapper/310:52:583
911685993229,2cyclictest0-21swapper/307:42:003
911683993230,1cyclictest0-21swapper/208:25:002
911683993229,2cyclictest0-21swapper/211:04:592
911683993229,2cyclictest0-21swapper/209:49:552
911683993229,2cyclictest0-21swapper/209:29:592
911685993128,2cyclictest0-21swapper/309:43:593
911685993128,2cyclictest0-21swapper/307:45:593
91168599310,29cyclictest0-21swapper/308:35:133
911683993128,2cyclictest0-21swapper/211:09:562
911683993127,2cyclictest0-21swapper/209:24:552
91168399310,30cyclictest0-21swapper/212:00:022
91168399310,30cyclictest0-21swapper/209:55:122
91168399310,29cyclictest0-21swapper/209:00:142
4652991310,4ptp4l441ktimers/311:55:013
911685993028,1cyclictest0-21swapper/312:20:013
911685993028,1cyclictest0-21swapper/312:13:583
911685993027,2cyclictest0-21swapper/311:50:003
91168599300,2cyclictest0-21swapper/312:00:113
91168599300,2cyclictest0-21swapper/309:50:013
91168599300,29cyclictest0-21swapper/309:20:143
91168599300,29cyclictest0-21swapper/308:30:123
91168599300,28cyclictest0-21swapper/312:30:013
91168599300,28cyclictest0-21swapper/310:25:153
91168599300,28cyclictest0-21swapper/308:10:013
911683993027,2cyclictest0-21swapper/212:20:112
911683993027,2cyclictest0-21swapper/212:20:112
911683993027,2cyclictest0-21swapper/210:00:112
911683993027,2cyclictest0-21swapper/209:35:142
911683993027,2cyclictest0-21swapper/208:55:022
911683993027,2cyclictest0-21swapper/208:05:022
911683993027,2cyclictest0-21swapper/207:55:122
911683993027,2cyclictest0-21swapper/207:40:002
91168399300,2cyclictest0-21swapper/211:50:142
91168399300,2cyclictest0-21swapper/209:55:012
91168399300,28cyclictest0-21swapper/212:15:122
91168399300,28cyclictest0-21swapper/211:55:152
91168399300,28cyclictest0-21swapper/208:10:112
911685992927,1cyclictest0-21swapper/310:55:113
911685992927,1cyclictest0-21swapper/308:50:123
91168599290,28cyclictest0-21swapper/308:45:143
91168599290,28cyclictest0-21swapper/308:00:103
91168599290,1cyclictest0-21swapper/310:45:143
911683992926,2cyclictest0-21swapper/212:35:112
911683992926,2cyclictest0-21swapper/212:07:592
911683992926,2cyclictest0-21swapper/211:30:012
911683992926,2cyclictest0-21swapper/211:00:002
911683992926,2cyclictest0-21swapper/208:00:142
911683992926,2cyclictest0-21swapper/207:45:112
911683992925,2cyclictest0-21swapper/210:50:012
91168399292,1cyclictest0-21swapper/207:35:012
91168399290,2cyclictest0-21swapper/211:15:002
91168399290,28cyclictest0-21swapper/209:05:132
91168399290,27cyclictest0-21swapper/212:25:122
91168399290,27cyclictest0-21swapper/210:42:592
91168399290,27cyclictest0-21swapper/210:33:592
91168399290,27cyclictest0-21swapper/210:20:132
91168399290,27cyclictest0-21swapper/210:15:122
91168399290,27cyclictest0-21swapper/210:15:122
91168399290,27cyclictest0-21swapper/207:25:112
91168399290,27cyclictest0-21swapper/207:13:592
91168399290,26cyclictest0-21swapper/208:49:592
91168399290,1cyclictest0-21swapper/212:10:172
4652991290,4ptp4l61790irq/134-eno1-TxRx-009:04:563
911685992825,2cyclictest0-21swapper/311:36:003
911685992825,2cyclictest0-21swapper/311:25:003
911685992825,2cyclictest0-21swapper/311:25:003
911685992825,2cyclictest0-21swapper/310:15:023
911685992825,2cyclictest0-21swapper/310:15:023
91168599280,26cyclictest0-21swapper/311:10:163
91168599280,21cyclictest0-21swapper/309:26:593
911683992825,2cyclictest0-21swapper/208:40:142
911683992824,2cyclictest0-21swapper/212:31:592
91168399280,2cyclictest0-21swapper/207:40:122
91168399280,2cyclictest0-21swapper/207:20:142
91168399280,26cyclictest0-21swapper/208:33:592
91168399280,26cyclictest0-21swapper/208:25:102
911685992725,1cyclictest0-21swapper/307:25:133
911685992725,1cyclictest0-21swapper/307:15:113
91168599270,1cyclictest0-21swapper/311:01:593
911685992623,2cyclictest0-21swapper/311:05:113
91168599260,2cyclictest0-21swapper/308:20:593
91168599260,1cyclictest0-21swapper/309:54:583
911683992623,2cyclictest0-21swapper/210:11:592
91168399260,24cyclictest0-21swapper/207:30:142
4652991260,5ptp4l61790irq/134-eno1-TxRx-011:44:593
4652991260,5ptp4l61790irq/134-eno1-TxRx-009:40:003
911685992523,1cyclictest0-21swapper/312:00:013
911685992523,1cyclictest0-21swapper/307:35:133
911683992522,2cyclictest0-21swapper/210:27:592
91168399250,2cyclictest0-21swapper/207:50:582
176399250,15rtkit-daemon529-21rs:main1
911685992422,1cyclictest0-21swapper/307:34:593
911685992422,1cyclictest0-21swapper/307:34:593
911683992320,2cyclictest0-21swapper/211:24:552
911677991915,2cyclictest904247-21kworker/u8:3+events_unbound07:10:020
9116839942,1cyclictest0-21swapper/207:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional