You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-11-04 - 21:26
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot6.osadl.org (updated Mon Nov 04, 2024 12:44:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
193007699114102,9cyclictest1976197-21latency_hist09:30:010
19300769910495,7cyclictest1991758-21kworker/u8:1+events_unbound10:20:020
19300769910289,10cyclictest1979380-21latency_hist09:40:010
19300779910093,5cyclictest1928492-21kworker/u8:0+events_unbound07:15:011
1930077999681,6cyclictest1995586-21kworker/u8:3+events_unbound10:45:011
1930076999686,7cyclictest2034055-21latency_hist12:30:010
1930076999668,9cyclictest1985747-21apt-get10:00:030
1930076999580,12cyclictest3105085-21systemd-journal10:00:010
1930077999386,6cyclictest1992424-21kworker/u8:2+flush-179:010:25:021
1930077999386,6cyclictest1992424-21kworker/u8:2+flush-179:010:25:011
1930077999384,7cyclictest2005117-21kworker/u8:2+events_unbound11:05:001
1930077999284,6cyclictest1982570-21kworker/u8:3+events_unbound10:05:001
1930076999280,9cyclictest2024519-21latency_hist12:00:010
1930077999181,8cyclictest1982570-21kworker/u8:3+events_unbound10:10:011
1930076999082,6cyclictest2021358-21kworker/u8:2+flush-179:012:00:140
1930076998775,9cyclictest2015217-21apt-get11:30:110
1930076998680,4cyclictest2005117-21kworker/u8:2+events_unbound11:19:300
1930077998577,6cyclictest1943260-21latency_hist07:50:011
1930077998571,12cyclictest1972470-21kworker/u8:3+events_unbound09:20:011
1930076998572,10cyclictest1985822-21kworker/u8:0+flush-179:010:40:000
1930077998474,7cyclictest1967671-21latency_hist09:05:011
1930076998474,7cyclictest2013128-21apt-get11:25:000
1930076998372,8cyclictest1938175-21kworker/u8:0+flush-179:007:40:260
1930077998174,5cyclictest2021358-21kworker/u8:2+events_unbound12:20:241
1930077998168,10cyclictest2029532-21apt-get12:15:151
1930076998167,11cyclictest1950144-21apt-get08:10:120
1930077998072,5cyclictest1995586-21kworker/u8:3+events_unbound10:55:151
1930076997969,7cyclictest1956386-21gpgv08:30:000
1930077997873,4cyclictest2021358-21kworker/u8:2+flush-179:012:05:251
1930077997873,4cyclictest1941395-21kworker/u8:4+flush-179:007:50:261
1930077997870,6cyclictest1992424-21kworker/u8:2+flush-179:010:30:211
1930076997866,9cyclictest3105085-21systemd-journal09:45:000
1930077997770,5cyclictest1938175-21kworker/u8:0+events_unbound07:35:011
1930077997768,7cyclictest2021866-21diskstats11:50:141
1930077997668,6cyclictest2005078-21kworker/u8:0+flush-179:011:30:141
1930076997668,6cyclictest1964729-21apt-get08:55:140
1930076997664,9cyclictest2016802-21apt-get11:35:130
1930076997568,5cyclictest1946752-21kworker/u8:2+flush-179:008:30:150
1930077997468,4cyclictest2005117-21kworker/u8:2+flush-179:011:25:181
1930077997467,5cyclictest2005078-21kworker/u8:0+flush-179:011:15:241
1930077997365,6cyclictest2014937-21kworker/u8:3+flush-179:011:45:001
1930077997365,5cyclictest2005117-21kworker/u8:2+events_unbound11:20:141
1930076997363,8cyclictest2021562-21/usr/sbin/munin11:50:220
1930077997266,4cyclictest1916005-21kworker/u8:3+events_unbound07:20:281
1930077997265,5cyclictest1961538-21kworker/u8:1+flush-179:008:50:151
1930077997265,4cyclictest1961538-21kworker/u8:1+events_unbound08:55:171
1930077997263,6cyclictest1992424-21kworker/u8:2+events_unbound10:25:131
1930077997166,4cyclictest1941395-21kworker/u8:4+flush-179:008:10:251
1930076997164,5cyclictest1941395-21kworker/u8:4+events_unbound08:05:270
1930077997066,3cyclictest1970277-21kworker/u8:1+flush-179:009:29:311
1930077997065,4cyclictest1962598-21kworker/u8:3+flush-179:009:05:251
1930077997062,5cyclictest1985822-21kworker/u8:0+flush-179:010:15:251
1930077997055,5cyclictest1938240-21kworker/u8:2+events_unbound07:40:001
1930076997059,8cyclictest1975641-21kworker/u8:0+flush-179:009:45:200
1930077996964,4cyclictest2021358-21kworker/u8:2+flush-179:011:55:241
1930077996861,5cyclictest2001900-21kworker/u8:1+events_unbound10:50:131
1930077996861,5cyclictest1954653-21kworker/u8:0+events_unbound08:25:131
1930076996863,4cyclictest2021358-21kworker/u8:2+flush-179:012:20:000
1930076996859,7cyclictest2030919-21kworker/u8:0+flush-179:012:40:010
1930076996858,8cyclictest1989439-21apt-get10:10:110
1930077996762,4cyclictest2024563-21kworker/u8:3+flush-179:012:15:001
1930077996761,5cyclictest1961538-21kworker/u8:1+flush-179:008:45:281
1930077996761,5cyclictest1961538-21kworker/u8:1+flush-179:008:45:271
1930077996760,6cyclictest1982570-21kworker/u8:3+events_unbound09:50:281
1930077996760,5cyclictest2024486-21kworker/u8:0+flush-179:012:00:231
1930077996560,4cyclictest2014937-21kworker/u8:3+flush-179:011:35:261
1930076996459,4cyclictest2005117-21kworker/u8:2+events_unbound11:25:220
1930077996357,4cyclictest2034074-21kworker/u8:1+flush-179:012:30:241
1930077996355,6cyclictest1975641-21kworker/u8:0+flush-179:009:30:211
1930076996357,4cyclictest1928492-21kworker/u8:0+events_unbound07:15:130
1930077996250,9cyclictest1972707-21apt-get09:20:121
1930076996254,6cyclictest1952074-21tune2fs08:15:180
1930076996150,9cyclictest1939956-21gpgv07:40:000
1930077996055,4cyclictest1967681-21kworker/u8:0+events_unbound09:10:261
1930077996055,4cyclictest1967681-21kworker/u8:0+events_unbound09:10:251
1930077996049,8cyclictest2035880-21/usr/sbin/munin12:35:261
1930076996053,5cyclictest2005078-21kworker/u8:0+flush-179:011:00:160
1930076995952,4cyclictest1992424-21kworker/u8:2+events_unbound10:25:160
1930076995951,6cyclictest2021358-21kworker/u8:2+flush-179:012:10:190
1930077995847,9cyclictest1939829-21apt-get07:40:001
1930076995850,6cyclictest1954672-21latency_hist08:25:010
1930077995752,3cyclictest2030919-21kworker/u8:0+flush-179:012:25:211
1930077995752,3cyclictest2005078-21kworker/u8:0+events_unbound11:10:141
1930077995751,4cyclictest1946752-21kworker/u8:2+flush-179:008:05:231
1930076995749,5cyclictest1962598-21kworker/u8:3+flush-179:009:05:210
1930077995652,3cyclictest1975641-21kworker/u8:0+flush-179:009:40:261
1930077995651,3cyclictest1982570-21kworker/u8:3+flush-179:010:10:211
1930076995644,9cyclictest2018391-21/usr/sbin/munin11:40:250
1930077995548,4cyclictest1958087-21kworker/u8:0+flush-179:008:35:191
1930077995445,7cyclictest1948306-21latency_hist08:05:011
1930077995442,4cyclictest1928492-21kworker/u8:0+flush-179:007:15:251
1930076995449,4cyclictest1995586-21kworker/u8:3+flush-179:010:49:550
1930076995449,4cyclictest1916005-21kworker/u8:3+flush-179:007:30:320
1930076995447,5cyclictest1941395-21kworker/u8:4+events_unbound08:00:010
1930076995444,8cyclictest1982798-21/usr/sbin/munin09:50:240
1930076995347,5cyclictest2005117-21kworker/u8:2+flush-179:011:05:240
1930076995347,4cyclictest1982570-21kworker/u8:3+flush-179:010:05:160
1930076995243,6cyclictest1967681-21kworker/u8:0+flush-179:009:10:210
1930076995243,6cyclictest1967681-21kworker/u8:0+flush-179:009:10:210
1930077995146,4cyclictest2014937-21kworker/u8:3+flush-179:011:45:271
1930077995142,4cyclictest2005117-21kworker/u8:2+events_unbound11:05:271
4505291500,13ptp4l441ktimers/307:10:013
1930077995043,5cyclictest1975641-21kworker/u8:0+events_unbound09:45:141
1930076995045,4cyclictest2001900-21kworker/u8:1+events_unbound10:50:230
1930077994843,4cyclictest1943243-21kworker/u8:0+flush-179:007:55:241
1930076994842,5cyclictest1972470-21kworker/u8:3+events_unbound09:30:150
1930076994842,5cyclictest1967681-21kworker/u8:0+flush-179:009:15:250
1930076994840,6cyclictest2021358-21kworker/u8:2+flush-179:012:10:010
1930077994743,3cyclictest1931610-21kworker/u8:1+flush-179:007:25:191
1930077994737,8cyclictest3105085-21systemd-journal08:25:011
1930076994739,6cyclictest1995586-21kworker/u8:3+flush-179:010:55:140
1930077994639,6cyclictest1995586-21kworker/u8:3+flush-179:010:45:151
1930076994643,2cyclictest1992424-21kworker/u8:2+flush-179:010:30:250
1930076994641,4cyclictest2030919-21kworker/u8:0+flush-179:012:20:160
1930076994641,4cyclictest1931610-21kworker/u8:1+flush-179:007:45:140
1930076994635,9cyclictest1961302-21latency_hist08:45:020
1930077994537,5cyclictest1946681-21kworker/u8:1+events_unbound08:15:221
1930077994441,2cyclictest1958087-21kworker/u8:0+flush-179:008:34:591
1930077994437,5cyclictest1958087-21kworker/u8:0+flush-179:008:40:151
1930076994436,6cyclictest2034074-21kworker/u8:1+flush-179:012:30:290
1930076994433,9cyclictest1964474-21latency_hist08:55:010
1930082994341,1cyclictest0-21swapper/307:58:593
193008099430,41cyclictest0-21swapper/207:34:552
1930077994338,4cyclictest1985822-21kworker/u8:0+events_unbound10:35:151
1930076994337,5cyclictest1995586-21kworker/u8:3+flush-179:010:40:200
193008299420,41cyclictest0-21swapper/311:13:593
193007799421,38cyclictest1985749-21perf10:00:011
1930076994237,4cyclictest3105085-21systemd-journal07:15:010
1930076994237,4cyclictest1991758-21kworker/u8:1+events_unbound10:20:150
1930076994237,4cyclictest1991758-21kworker/u8:1+events_unbound10:20:150
1930076994234,6cyclictest2009273-21kworker/u8:1+flush-179:011:10:260
1930076994135,4cyclictest1946752-21kworker/u8:2+flush-179:008:00:160
1930082994036,2cyclictest0-21swapper/309:30:583
1930080994036,2cyclictest0-21swapper/209:42:592
1930076994036,3cyclictest2021355-21systemd-run11:50:000
1930076994030,8cyclictest3105085-21systemd-journal09:21:310
1930076993933,4cyclictest1943268-21turbostat07:54:550
193008299380,35cyclictest0-21swapper/309:45:583
1930080993835,2cyclictest0-21swapper/211:49:592
1930080993834,2cyclictest0-21swapper/211:13:592
193008099380,36cyclictest0-21swapper/210:22:002
193008099380,36cyclictest0-21swapper/210:21:592
193008099380,35cyclictest0-21swapper/211:54:552
193008099380,35cyclictest0-21swapper/207:54:552
1930076993834,3cyclictest1916005-21kworker/u8:3+flush-179:007:20:160
1930076993831,5cyclictest722-21Xorg08:48:590
1930076993831,5cyclictest722-21Xorg08:48:580
193008299370,36cyclictest0-21swapper/311:32:593
193008299370,36cyclictest0-21swapper/308:25:593
193008299370,36cyclictest0-21swapper/308:08:593
193008299370,36cyclictest0-21swapper/308:02:003
1930080993734,2cyclictest0-21swapper/210:56:582
1930080993734,2cyclictest0-21swapper/209:46:592
1930080993734,2cyclictest0-21swapper/209:06:592
1930080993734,2cyclictest0-21swapper/208:46:592
1930080993734,2cyclictest0-21swapper/208:46:592
1930080993733,2cyclictest0-21swapper/207:45:592
1930077993734,2cyclictest722-21Xorg09:38:001
1930082993634,1cyclictest0-21swapper/309:44:003
1930082993634,1cyclictest0-21swapper/307:36:593
1930082993634,1cyclictest0-21swapper/307:19:563
1930082993633,2cyclictest0-21swapper/311:29:593
1930082993630,5cyclictest1938205-21turbostat07:35:003
193008299360,35cyclictest0-21swapper/310:51:593
193008299360,35cyclictest0-21swapper/310:27:003
193008299360,35cyclictest0-21swapper/308:59:003
193008299360,1cyclictest0-21swapper/310:57:583
1930080993634,1cyclictest0-21swapper/208:45:002
1930080993633,2cyclictest0-21swapper/212:13:592
1930076993631,4cyclictest1961538-21kworker/u8:1+flush-179:009:00:230
1930082993532,2cyclictest0-21swapper/310:38:593
1930080993532,2cyclictest0-21swapper/208:01:002
1930080993431,2cyclictest0-21swapper/207:56:592
1930080993431,2cyclictest0-21swapper/207:10:592
1930076993426,6cyclictest1931610-21kworker/u8:1+events_unbound07:25:020
1930080993331,1cyclictest0-21swapper/208:54:582
1930080993329,2cyclictest0-21swapper/210:52:002
193007699331,30cyclictest0-21swapper/008:35:110
1930082993230,1cyclictest0-21swapper/310:01:593
1930082993229,2cyclictest0-21swapper/308:20:583
1930082993229,2cyclictest0-21swapper/308:18:593
1930080993229,2cyclictest0-21swapper/211:32:592
1930080993229,2cyclictest0-21swapper/210:26:592
1930080993229,2cyclictest0-21swapper/208:55:592
193008099320,2cyclictest0-21swapper/210:34:002
1930082993129,1cyclictest0-21swapper/308:49:563
1930082993129,1cyclictest0-21swapper/308:49:553
1930082993129,1cyclictest0-21swapper/308:45:003
193008299310,1cyclictest0-21swapper/311:40:583
1930080993128,2cyclictest0-21swapper/210:05:142
1930080993128,2cyclictest0-21swapper/209:29:392
1930080993128,2cyclictest0-21swapper/208:25:582
1930080993128,2cyclictest0-21swapper/208:08:592
1930080993127,2cyclictest0-21swapper/209:30:592
4505291300,6ptp4l441ktimers/310:34:553
193008299300,28cyclictest0-21swapper/310:05:133
1930080993027,2cyclictest0-21swapper/210:35:122
1930080993027,2cyclictest0-21swapper/207:38:582
193008099300,28cyclictest0-21swapper/212:35:152
4505291290,7ptp4l441ktimers/311:55:013
1930082992926,2cyclictest0-21swapper/309:55:133
1930082992925,2cyclictest0-21swapper/312:35:153
193008299290,28cyclictest0-21swapper/312:15:173
193008299290,28cyclictest0-21swapper/309:29:323
193008299290,28cyclictest0-21swapper/307:25:163
193008299290,27cyclictest0-21swapper/311:15:163
193008299290,27cyclictest0-21swapper/310:20:123
193008299290,27cyclictest0-21swapper/310:20:123
1930080992926,2cyclictest0-21swapper/212:15:172
1930080992926,2cyclictest0-21swapper/211:00:142
1930080992923,2cyclictest0-21swapper/211:15:152
193008099290,28cyclictest0-21swapper/211:25:022
193008099290,27cyclictest0-21swapper/211:40:132
193008099290,27cyclictest0-21swapper/209:55:122
193008099290,27cyclictest0-21swapper/209:50:132
193008099290,27cyclictest0-21swapper/207:25:162
4505291280,7ptp4l441ktimers/312:35:003
4505291280,6ptp4l61790irq/134-eno1-TxRx-009:10:003
4505291280,6ptp4l441ktimers/312:30:003
4505291280,6ptp4l441ktimers/310:45:003
4505291280,5ptp4l441ktimers/312:10:013
1930082992825,2cyclictest0-21swapper/311:35:153
193008299280,26cyclictest0-21swapper/312:20:163
193008299280,26cyclictest0-21swapper/309:50:123
193008299280,26cyclictest0-21swapper/308:50:123
193008299280,26cyclictest0-21swapper/307:10:143
193008299280,25cyclictest0-21swapper/307:45:153
193008299280,1cyclictest0-21swapper/311:05:593
193008299280,1cyclictest0-21swapper/311:00:153
1930080992825,2cyclictest0-21swapper/212:20:162
1930080992825,2cyclictest0-21swapper/211:35:152
1930080992825,2cyclictest0-21swapper/208:35:112
193008099280,26cyclictest0-21swapper/207:15:122
193008099280,1cyclictest0-21swapper/208:20:142
4505291270,7ptp4l61790irq/134-eno1-TxRx-010:15:003
4505291270,6ptp4l441ktimers/311:50:003
4505291270,6ptp4l441ktimers/309:05:013
4505291270,5ptp4l441ktimers/307:25:003
193008299270,2cyclictest0-21swapper/309:12:003
193008299270,2cyclictest0-21swapper/309:12:003
193008299270,26cyclictest0-21swapper/310:47:003
1930080992724,2cyclictest0-21swapper/211:58:592
1930080992723,2cyclictest0-21swapper/210:02:592
1930080992723,2cyclictest0-21swapper/209:03:592
4505291260,6ptp4l61790irq/134-eno1-TxRx-010:20:013
4505291260,6ptp4l441ktimers/312:09:553
4505291260,5ptp4l61790irq/134-eno1-TxRx-012:00:013
4505291260,5ptp4l61790irq/134-eno1-TxRx-009:40:013
4505291260,5ptp4l61790irq/134-eno1-TxRx-009:25:013
4505291260,5ptp4l61790irq/134-eno1-TxRx-008:14:553
4505291260,5ptp4l61790irq/134-eno1-TxRx-007:44:553
4505291260,5ptp4l441ktimers/312:04:563
4505291260,5ptp4l441ktimers/309:19:553
1930082992623,2cyclictest0-21swapper/307:54:593
193008099260,24cyclictest0-21swapper/211:20:142
4505291250,5ptp4l441ktimers/311:25:003
4505291250,5ptp4l441ktimers/308:39:553
1930080992522,2cyclictest0-21swapper/212:07:592
1930080992522,2cyclictest0-21swapper/211:06:002
1930080992321,1cyclictest0-21swapper/208:34:552
1930080992320,2cyclictest0-21swapper/210:46:592
1930080992320,2cyclictest0-21swapper/209:12:002
1930080992320,2cyclictest0-21swapper/209:11:592
1930080992319,2cyclictest0-21swapper/208:18:592
1930080992119,1cyclictest0-21swapper/212:04:592
1930080992118,2cyclictest0-21swapper/212:33:592
1930080992118,2cyclictest0-21swapper/212:25:122
1930080992118,2cyclictest0-21swapper/207:22:592
1930080992017,2cyclictest0-21swapper/208:10:592
1930080992017,2cyclictest0-21swapper/207:42:592
1930080991816,1cyclictest0-21swapper/210:15:192
1930080991815,2cyclictest0-21swapper/210:40:592
193007799176,9cyclictest1930179-21latency_hist07:10:021
193008099150,13cyclictest0-21swapper/209:39:572
1930076991511,2cyclictest0-21swapper/007:10:020
45063911411,1phc2sys0-21swapper/308:30:033
1930080991410,2cyclictest0-21swapper/209:21:582
19300809996,2cyclictest0-21swapper/209:15:252
19300809974,2cyclictest0-21swapper/210:14:272
19300809930,2cyclictest0-21swapper/207:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional