You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-20 - 19:57
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot6.osadl.org (updated Thu Nov 20, 2025 12:44:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3076599910288,11cyclictest328250-21latency_hist08:15:001
307659999584,8cyclictest339653-21latency_hist08:50:011
307659998452,31cyclictest0-21swapper/110:21:591
307657998451,31cyclictest723-21Xorg10:21:590
307659998373,7cyclictest401142-21/usr/sbin/munin12:00:231
307657998155,23cyclictest322123-21/usr/sbin/munin07:55:200
307659998070,7cyclictest343345-21/usr/sbin/munin09:00:231
307657997665,9cyclictest271-21systemd-journal10:10:000
307657997665,9cyclictest271-21systemd-journal10:10:000
307659997367,4cyclictest342802-21kworker/u8:1+events_unbound09:00:011
307657997161,8cyclictest342829-21apt-get09:00:000
307657997160,8cyclictest325318-21/usr/sbin/munin08:05:220
307665996859,5cyclictest323490-21turbostat08:00:013
307659996857,9cyclictest271-21systemd-journal10:15:001
307657996857,9cyclictest323751-21apt-get08:00:110
307659996759,6cyclictest397746-21kworker/u8:3+flush-179:012:10:001
307659996660,4cyclictest328746-21kworker/u8:0+flush-179:008:20:251
30765999660,63cyclictest0-21swapper/111:20:591
307657996560,4cyclictest394578-21kworker/u8:2+flush-179:012:05:250
307657996455,6cyclictest388165-21latency_hist11:20:020
307659996353,8cyclictest406159-21/usr/sbin/munin12:15:241
307657996356,5cyclictest323732-21kworker/u8:1+flush-179:008:20:000
307659996256,4cyclictest397746-21kworker/u8:3+events_unbound12:10:211
307657996252,7cyclictest317120-21latency_hist07:40:020
307657996250,9cyclictest375428-21latency_hist10:40:010
307659995851,5cyclictest355820-21latency_hist09:40:011
307659995847,8cyclictest321887-21latency_hist07:55:011
307657995849,7cyclictest271-21systemd-journal12:05:010
307657995848,8cyclictest366145-21apt-get10:10:140
307657995751,4cyclictest397746-21kworker/u8:3+flush-179:012:30:250
307657995750,5cyclictest364052-21apt-get10:05:010
307657995750,5cyclictest364052-21apt-get10:05:010
307657995546,7cyclictest312329-21latency_hist07:25:000
307657995448,5cyclictest370905-21kworker/u8:2+events_unbound10:30:010
307657995443,8cyclictest338070-21latency_hist08:45:010
307659995345,6cyclictest364045-21latency_hist10:05:011
307659995345,6cyclictest364045-21latency_hist10:05:011
307657995348,4cyclictest407486-21kworker/u8:0+flush-179:012:20:160
307657995347,5cyclictest301525-21kworker/u8:3+flush-179:007:20:010
307657995343,8cyclictest323732-21kworker/u8:1+flush-179:008:40:000
307659995244,6cyclictest365921-21kworker/u8:0+flush-179:010:20:001
307659995242,7cyclictest381804-21latency_hist11:00:001
307657995245,5cyclictest362455-21latency_hist10:00:010
307657995245,5cyclictest347898-21kworker/u8:3+events_unbound09:45:180
307657995244,6cyclictest377033-21latency_hist10:45:000
307657995242,8cyclictest271-21systemd-journal09:40:010
307659995148,2cyclictest372290-21kworker/u8:3+events_unbound11:25:281
307659995144,5cyclictest355872-21kworker/u8:2+events_unbound09:55:251
307659995141,8cyclictest271-21systemd-journal09:55:001
307659995042,7cyclictest344930-21kworker/u8:2+flush-179:009:20:211
307659995039,8cyclictest328717-21cat08:15:011
307657995043,5cyclictest342802-21kworker/u8:1+flush-179:009:05:140
307657995043,4cyclictest394578-21kworker/u8:2+flush-179:011:50:180
307657995041,7cyclictest380202-21latency_hist10:55:010
307663994946,2cyclictest0-21swapper/209:09:592
307659994944,4cyclictest372290-21kworker/u8:3+events_unbound11:30:271
307659994943,5cyclictest323732-21kworker/u8:1+flush-179:008:05:211
307659994943,4cyclictest370905-21kworker/u8:2+events_unbound10:40:241
307657994841,5cyclictest271-21systemd-journal09:20:010
4670791470,13ptp4l61690irq/136-eno1-TxRx-007:10:003
307665994745,1cyclictest0-21swapper/309:09:593
307659994740,5cyclictest372290-21kworker/u8:3+events_unbound10:30:251
307659994739,6cyclictest342802-21kworker/u8:1+flush-179:009:05:181
307659994738,7cyclictest336519-21kworker/u8:0+flush-179:008:50:211
307657994737,8cyclictest404594-21apt-get12:10:110
307657994641,4cyclictest382047-21kworker/u8:0+flush-179:011:20:170
307657994641,4cyclictest328746-21kworker/u8:0+events_unbound08:20:270
307659994541,2cyclictest723-21Xorg11:55:001
307659994540,4cyclictest355872-21kworker/u8:2+flush-179:010:05:211
307659994540,4cyclictest355872-21kworker/u8:2+flush-179:010:05:211
307659994539,5cyclictest330159-21kworker/u8:3+flush-179:008:30:151
307659994539,5cyclictest330159-21kworker/u8:3+flush-179:008:30:151
307659994534,8cyclictest397711-21latency_hist11:50:011
307659994534,8cyclictest397711-21latency_hist11:50:001
307657994539,5cyclictest386602-21kworker/u8:1+events_unbound11:40:000
307657994539,5cyclictest365921-21kworker/u8:0+events_unbound10:15:240
307657994538,5cyclictest342802-21kworker/u8:1+events_unbound09:00:030
307659994439,4cyclictest382047-21kworker/u8:0+flush-179:011:15:151
307659994438,5cyclictest292766-21kworker/u8:0+events_unbound07:20:181
30765799442,38cyclictest690-21snmpd10:55:140
30765999430,41cyclictest315513-21turbostat07:39:551
307657994337,4cyclictest336519-21kworker/u8:0+events_unbound09:10:230
307665994239,2cyclictest0-21swapper/309:41:593
307659994236,5cyclictest370905-21kworker/u8:2+flush-179:010:45:241
307659994233,6cyclictest351092-21kworker/u8:1+events_unbound09:30:151
30765999420,40cyclictest723-21Xorg11:58:591
30765999420,2cyclictest723-21Xorg10:26:591
307657994235,5cyclictest377068-21kworker/u8:1+flush-179:011:00:000
307659994134,5cyclictest323732-21kworker/u8:1+flush-179:008:40:181
307659994134,4cyclictest0-21swapper/107:31:001
30765999410,38cyclictest723-21Xorg09:28:591
307657994136,3cyclictest723-21Xorg11:12:000
307657994135,5cyclictest292766-21kworker/u8:0+events_unbound07:45:000
307657994134,6cyclictest382047-21kworker/u8:0+flush-179:011:05:150
30765799410,2cyclictest0-21swapper/007:32:000
307663994037,2cyclictest0-21swapper/208:35:592
307659994036,3cyclictest372290-21kworker/u8:3+i91511:11:591
307659994036,3cyclictest336514-21systemd-run08:40:011
307659994035,3cyclictest407973-21cstates12:20:131
307659994032,6cyclictest310769-21kworker/u8:1+flush-179:007:25:211
307657994035,4cyclictest410692-21kworker/u8:0+flush-179:012:35:180
307657994029,9cyclictest378630-21apt10:50:010
30766599390,36cyclictest0-21swapper/307:37:593
307663993935,2cyclictest0-21swapper/207:44:562
307659993932,4cyclictest0-21swapper/111:40:131
307659993931,6cyclictest348100-21/usr/sbin/munin09:15:161
30765999392,30cyclictest357936-21apt-get09:45:171
307657993933,4cyclictest330159-21kworker/u8:3+events_unbound08:30:190
307657993933,4cyclictest330159-21kworker/u8:3+events_unbound08:30:180
307657993928,9cyclictest328262-21apt-get08:15:010
307665993836,1cyclictest0-21swapper/308:35:593
307665993835,2cyclictest0-21swapper/310:34:593
307665993833,3cyclictest312335-21turbostat07:25:003
30766599380,35cyclictest0-21swapper/307:54:563
30766599380,2cyclictest0-21swapper/308:16:593
307663993835,2cyclictest0-21swapper/208:20:592
307663993834,2cyclictest0-21swapper/208:32:592
307663993834,2cyclictest0-21swapper/208:32:582
307663993834,2cyclictest0-21swapper/208:02:002
307663993834,2cyclictest0-21swapper/207:20:592
30766399380,36cyclictest0-21swapper/212:31:582
30766399380,36cyclictest0-21swapper/211:25:592
30766399380,35cyclictest0-21swapper/210:29:002
307659993834,3cyclictest333290-21/usr/sbin/munin08:30:011
307659993832,5cyclictest723-21Xorg09:10:591
307659993832,5cyclictest410692-21kworker/u8:0+events_unbound12:40:011
30765999382,31cyclictest383654-21apt-get11:05:121
307657993833,4cyclictest347898-21kworker/u8:3+flush-179:009:50:290
30765799380,37cyclictest0-21swapper/012:19:590
30765799380,37cyclictest0-21swapper/011:34:590
30765799380,2cyclictest723-21Xorg08:25:590
307665993734,2cyclictest0-21swapper/310:46:593
307665993734,2cyclictest0-21swapper/309:04:593
307665993732,4cyclictest355831-21turbostat09:40:003
307665993730,6cyclictest347872-21turbostat09:15:003
30766599370,36cyclictest0-21swapper/312:37:003
307663993734,2cyclictest0-21swapper/208:49:592
307663993734,2cyclictest0-21swapper/208:42:592
307663993733,2cyclictest0-21swapper/211:14:592
307663993733,2cyclictest0-21swapper/209:04:552
30766399370,35cyclictest0-21swapper/212:24:542
30766399370,35cyclictest0-21swapper/211:45:592
30766399370,35cyclictest0-21swapper/211:45:582
30766399370,35cyclictest0-21swapper/208:52:002
307659993732,4cyclictest408281-21kworker/u8:1+events_unbound12:25:111
307657993731,4cyclictest347898-21kworker/u8:3+events_unbound09:30:250
307657993730,5cyclictest394781-21/usr/sbin/munin11:40:170
30765799370,36cyclictest0-21swapper/011:58:590
30765799370,36cyclictest0-21swapper/009:28:590
307665993634,1cyclictest0-21swapper/308:20:593
307665993634,1cyclictest0-21swapper/307:44:553
307665993633,2cyclictest0-21swapper/308:32:593
307665993633,2cyclictest0-21swapper/308:32:583
30766599360,1cyclictest0-21swapper/312:04:553
307663993633,2cyclictest0-21swapper/207:30:592
30766399360,1cyclictest0-21swapper/212:36:592
30766399360,1cyclictest0-21swapper/207:54:552
30766399360,1cyclictest0-21swapper/207:37:582
307659993627,6cyclictest0-21swapper/112:30:141
30765799361,31cyclictest0-21swapper/007:25:120
4670791350,3ptp4l61690irq/136-eno1-TxRx-012:25:103
307665993532,2cyclictest0-21swapper/311:14:593
307665993532,2cyclictest0-21swapper/308:49:593
307663993533,1cyclictest0-21swapper/209:40:002
307659993529,4cyclictest386602-21kworker/u8:1+flush-179:011:40:001
307659993527,6cyclictest309390-21apt-get07:15:141
307659993527,6cyclictest301525-21kworker/u8:3+flush-179:007:10:151
307657993529,4cyclictest397761-21ls11:50:010
307657993529,4cyclictest397761-21ls11:50:010
30765799350,34cyclictest723-21Xorg07:13:000
307665993432,1cyclictest0-21swapper/308:42:593
307665993432,1cyclictest0-21swapper/308:01:593
307663993431,2cyclictest0-21swapper/208:14:542
307663993430,2cyclictest0-21swapper/207:46:592
307659993429,4cyclictest380239-21systemd-run10:55:011
307659993429,4cyclictest347898-21kworker/u8:3+flush-179:009:40:211
307659993429,3cyclictest320340-21systemd07:50:011
307659993423,7cyclictest271-21systemd-journal07:45:001
307657993431,2cyclictest0-21swapper/009:20:590
307657993430,3cyclictest320997-21hwlatdetect07:50:170
307657993430,3cyclictest0-21swapper/009:45:010
307665993331,1cyclictest0-21swapper/307:30:593
307659993326,5cyclictest325082-21latency_hist08:05:011
307659993324,7cyclictest377068-21kworker/u8:1+events_unbound11:00:221
307665993229,2cyclictest0-21swapper/307:46:593
307663993229,2cyclictest0-21swapper/209:23:592
307663993228,2cyclictest0-21swapper/211:07:582
30765999321,3cyclictest0-21swapper/110:35:131
307657993227,3cyclictest723-21Xorg12:28:580
307665993129,1cyclictest0-21swapper/308:14:553
30766399310,2cyclictest0-21swapper/209:40:162
307657993125,5cyclictest336519-21kworker/u8:0+flush-179:008:49:550
307665993027,2cyclictest0-21swapper/311:28:003
307665993027,2cyclictest0-21swapper/309:24:003
30766599300,23cyclictest0-21swapper/311:49:593
30766599300,23cyclictest0-21swapper/311:49:593
307663993028,1cyclictest0-21swapper/207:19:582
307663993027,2cyclictest0-21swapper/211:33:592
30766399300,28cyclictest0-21swapper/210:40:142
307657993024,1cyclictest3738721systemd-run10:35:010
307657993023,3cyclictest0-21swapper/011:25:170
30766599290,27cyclictest0-21swapper/311:55:113
30766599290,27cyclictest0-21swapper/310:40:133
30766599290,27cyclictest0-21swapper/310:25:133
30766599290,27cyclictest0-21swapper/307:15:163
307663992926,2cyclictest0-21swapper/210:10:152
307663992926,2cyclictest0-21swapper/208:15:032
30766399290,2cyclictest0-21swapper/211:55:112
30766399290,2cyclictest0-21swapper/207:57:592
30766399290,27cyclictest0-21swapper/212:05:132
30766399290,27cyclictest0-21swapper/209:45:032
307657992924,4cyclictest292766-21kworker/u8:0+flush-179:007:45:230
307657992922,5cyclictest307520-21latency_hist07:10:020
307665992826,1cyclictest0-21swapper/312:24:003
307665992826,1cyclictest0-21swapper/311:30:163
307665992826,1cyclictest0-21swapper/311:07:593
307665992825,2cyclictest0-21swapper/311:51:593
307665992825,2cyclictest0-21swapper/311:03:583
30766599280,2cyclictest0-21swapper/307:10:133
30766599280,27cyclictest0-21swapper/312:05:133
30766599280,27cyclictest0-21swapper/310:10:163
30766599280,26cyclictest0-21swapper/309:45:033
307663992825,2cyclictest0-21swapper/211:20:152
307663992825,2cyclictest0-21swapper/210:48:592
307663992825,2cyclictest0-21swapper/207:10:132
4670791270,6ptp4l61690irq/136-eno1-TxRx-011:20:013
4670791270,6ptp4l61690irq/136-eno1-TxRx-009:20:013
4670791270,6ptp4l441ktimers/311:25:003
4670791270,6ptp4l441ktimers/310:25:013
30766599270,26cyclictest0-21swapper/312:19:593
30766599270,25cyclictest0-21swapper/308:05:123
307663992724,2cyclictest0-21swapper/212:27:002
307663992724,2cyclictest0-21swapper/212:00:582
307663992724,2cyclictest0-21swapper/211:00:152
307663992723,2cyclictest0-21swapper/210:50:592
30766399270,25cyclictest0-21swapper/212:19:582
30766399270,18cyclictest0-21swapper/208:05:582
307659992722,4cyclictest318954-21kworker/u8:2+flush-179:007:59:551
4670791260,6ptp4l61690irq/136-eno1-TxRx-010:10:003
4670791260,6ptp4l61690irq/136-eno1-TxRx-010:09:593
4670791260,6ptp4l61690irq/136-eno1-TxRx-009:59:593
4670791260,6ptp4l441ktimers/311:00:003
4670791260,6ptp4l441ktimers/310:05:013
4670791260,6ptp4l441ktimers/310:05:013
4670791260,5ptp4l61690irq/136-eno1-TxRx-009:30:003
4670791260,5ptp4l61690irq/136-eno1-TxRx-008:59:563
4670791260,5ptp4l61690irq/136-eno1-TxRx-007:30:013
4670791260,5ptp4l441ktimers/311:40:003
4670791260,5ptp4l441ktimers/308:54:553
307665992623,2cyclictest0-21swapper/310:35:143
30766599260,24cyclictest0-21swapper/308:25:593
307663992623,2cyclictest0-21swapper/211:55:002
307663992623,2cyclictest0-21swapper/209:32:592
307663992622,2cyclictest0-21swapper/211:41:592
30766399260,2cyclictest0-21swapper/209:12:592
307657992623,2cyclictest0-21swapper/008:55:010
4670791250,6ptp4l61690irq/136-eno1-TxRx-012:35:003
4670791250,6ptp4l61690irq/136-eno1-TxRx-010:19:553
4670791250,5ptp4l61690irq/136-eno1-TxRx-009:55:003
4670791250,5ptp4l441ktimers/310:54:563
4670791250,5ptp4l441ktimers/309:34:553
4670791250,4ptp4l441ktimers/311:45:003
307663992522,2cyclictest0-21swapper/209:28:592
307663992521,2cyclictest0-21swapper/210:30:582
307663992521,2cyclictest0-21swapper/208:57:592
30766399250,2cyclictest0-21swapper/208:25:592
4670791240,4ptp4l61690irq/136-eno1-TxRx-012:14:553
307663992420,2cyclictest0-21swapper/209:16:582
307663992320,2cyclictest0-21swapper/210:02:592
307663992320,2cyclictest0-21swapper/210:02:592
307663992319,2cyclictest0-21swapper/210:23:592
307663992319,2cyclictest0-21swapper/210:05:032
307663992319,2cyclictest0-21swapper/210:05:032
307663992219,2cyclictest0-21swapper/212:13:002
307663992219,2cyclictest0-21swapper/210:15:592
30765999213,16cyclictest307691-21cat07:10:021
307663991916,2cyclictest0-21swapper/211:17:592
307663991916,2cyclictest0-21swapper/209:50:142
307663991915,2cyclictest0-21swapper/210:55:142
307663991614,1cyclictest0-21swapper/211:40:002
307663991411,2cyclictest0-21swapper/210:38:592
30766399129,2cyclictest0-21swapper/207:28:572
3076639974,2cyclictest0-21swapper/209:56:092
3076639930,2cyclictest0-21swapper/207:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional