You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-01 - 00:23
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot6.osadl.org (updated Fri Oct 31, 2025 12:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30239469910897,8cyclictest3040301-21latency_hist08:00:001
30239449910693,10cyclictest3024066-21/usr/sbin/munin07:10:230
30239449910393,7cyclictest3040308-21apt-get08:00:000
30239469910189,9cyclictest3105069-21latency_hist11:20:011
3023946999987,9cyclictest3113305-21latency_hist11:45:011
3023944999873,8cyclictest3113561-21apt-get11:45:140
3023944999686,7cyclictest3061806-21latency_hist09:05:010
3023946999585,7cyclictest3113663-21kworker/u8:2+flush-179:012:00:171
3023944999586,7cyclictest601706-21systemd-journal09:15:000
3023946999487,5cyclictest3051696-21kworker/u8:0+flush-179:008:45:171
3023946999283,6cyclictest3028890-21apt-get07:24:591
3023944999285,4cyclictest3048590-21kworker/u8:3+flush-179:008:35:240
3023944999281,9cyclictest3089398-21apt-get10:30:130
3023946999183,6cyclictest3071341-21kworker/u8:2+events_unbound09:35:001
3023946999079,9cyclictest3097357-21apt-get10:55:121
3023944999083,5cyclictest3045375-21kworker/u8:2+flush-179:008:25:170
3023944999082,5cyclictest3127674-21kworker/u8:3+flush-179:012:30:260
3023944999079,9cyclictest3083030-21apt-get10:10:140
3023944998983,4cyclictest3076384-21kworker/u8:1+flush-179:010:05:130
3023944998982,5cyclictest3056798-21kworker/u8:3+flush-179:009:20:170
3023944998781,4cyclictest3113663-21kworker/u8:2+flush-179:012:00:230
3023944998780,5cyclictest3077970-21kworker/u8:0+flush-179:011:05:260
3023946998575,7cyclictest3071804-21cat09:35:011
3023946998478,4cyclictest3090718-21kworker/u8:2+flush-179:010:45:001
3023946998471,10cyclictest3048540-21latency_hist08:25:011
3023946998464,18cyclictest3058362-21apt-get08:55:001
3023946998372,8cyclictest3114893-21latency_hist11:50:011
3023944998273,7cyclictest3045597-21apt-get08:15:120
3023944998270,10cyclictest3115148-21apt-get11:50:110
3023946998175,5cyclictest3085092-21kworker/u8:1+events_unbound10:25:201
3023946998073,5cyclictest3005867-21kworker/u8:1+flush-179:007:15:171
3023946998069,9cyclictest3044011-21apt-get08:10:121
3023946997972,4cyclictest3109836-21kworker/u8:3+flush-179:011:35:241
3023946997972,4cyclictest3109836-21kworker/u8:3+flush-179:011:35:241
3023946997971,7cyclictest3076384-21kworker/u8:1+events_unbound10:00:251
3023946997468,3cyclictest3056798-21kworker/u8:3+flush-179:009:45:211
3023944997468,4cyclictest3077970-21kworker/u8:0+flush-179:010:25:000
3023944997466,6cyclictest3093928-21latency_hist10:45:010
3023946997367,4cyclictest3095771-21apt-get10:50:121
3023946997362,9cyclictest3094183-21apt-get10:45:151
3023944997366,5cyclictest3045375-21kworker/u8:2+flush-179:008:30:160
3023946997264,6cyclictest3124720-21apt-get12:20:111
3023944997261,9cyclictest3058350-21latency_hist08:55:000
3023946997165,4cyclictest3045375-21kworker/u8:2+events_unbound08:15:151
3023946997162,7cyclictest3077970-21kworker/u8:0+events_unbound11:05:221
3023946996963,5cyclictest3048590-21kworker/u8:3+flush-179:008:34:551
3023944996963,4cyclictest3056798-21kworker/u8:3+flush-179:009:15:160
3023944996962,5cyclictest3045375-21kworker/u8:2+flush-179:008:49:590
3023946996858,8cyclictest3077970-21kworker/u8:0+flush-179:011:15:011
3023946996856,10cyclictest3032566-21apt-get07:35:121
3023944996861,6cyclictest3077970-21kworker/u8:0+events_unbound09:55:180
3023944996861,6cyclictest3077970-21kworker/u8:0+events_unbound09:55:180
3023944996861,5cyclictest3056798-21kworker/u8:3+flush-179:009:40:160
3023946996759,5cyclictest3108497-21kworker/u8:1+events_unbound11:55:181
3023950996662,2cyclictest0-21swapper/207:47:002
3023946996660,4cyclictest3032570-21kworker/u8:0+flush-179:008:05:181
3023944996658,6cyclictest3105342-21apt-get11:20:100
3023946996560,4cyclictest3077970-21kworker/u8:0+events_unbound10:20:181
3023944996558,5cyclictest3056798-21kworker/u8:3+events_unbound09:30:210
3023946996459,4cyclictest3127674-21kworker/u8:3+flush-179:012:34:561
3023946996455,6cyclictest3028890-21apt-get07:25:031
3023944996458,5cyclictest3051696-21kworker/u8:0+events_unbound08:40:170
3023944996457,5cyclictest3010747-21kworker/u8:3+events_unbound07:10:020
3023944996455,7cyclictest3088097-21diskstats10:25:160
3023944996357,4cyclictest3032570-21kworker/u8:0+flush-179:008:00:170
3023944996352,9cyclictest3042423-21apt-get08:05:130
3023946996256,4cyclictest3032570-21kworker/u8:0+flush-179:007:40:221
3023944996254,6cyclictest3076384-21kworker/u8:1+flush-179:009:50:200
3023946996152,7cyclictest3084355-21latency_hist10:15:011
3023946996150,9cyclictest3050132-21latency_hist08:30:001
3023944996155,4cyclictest3029388-21apt-get07:25:110
302394499611,54cyclictest1-21systemd12:36:340
3023946996054,4cyclictest3077970-21kworker/u8:0+flush-179:010:05:181
3023946996054,4cyclictest3045375-21kworker/u8:2+flush-179:008:55:221
3023944996055,4cyclictest3090718-21kworker/u8:2+flush-179:010:49:550
3023944996054,4cyclictest3103518-21kworker/u8:2+flush-179:011:30:220
3023944996046,11cyclictest3030979-21apt-get07:30:140
3023946995952,5cyclictest3066551-21kworker/u8:0+flush-179:009:25:001
3023944995956,2cyclictest722-21Xorg11:15:590
3023944995954,4cyclictest3026734-21kworker/u8:0+events_unbound07:15:250
3023944995951,6cyclictest601706-21systemd-journal08:59:560
3023946995853,4cyclictest3077970-21kworker/u8:0+flush-179:010:15:191
3023944995853,4cyclictest3056798-21kworker/u8:3+flush-179:009:37:350
3023944995853,3cyclictest3077970-21kworker/u8:0+flush-179:010:00:200
3023944995852,4cyclictest3077970-21kworker/u8:0+events_unbound10:35:270
3023944995852,4cyclictest3066551-21kworker/u8:0+flush-179:009:25:150
3023944995851,5cyclictest3095551-21kworker/u8:3+flush-179:010:55:230
302394499580,56cyclictest1-21systemd12:07:330
3023946995752,4cyclictest3060199-21kworker/u8:1+flush-179:009:05:231
3023946995751,5cyclictest3108497-21kworker/u8:1+flush-179:012:29:561
3023944995751,4cyclictest3108497-21kworker/u8:1+flush-179:011:55:160
3023944995750,5cyclictest3108497-21kworker/u8:1+flush-179:012:25:160
3023944995750,5cyclictest3075068-21apt-get09:45:120
3023946995650,4cyclictest3077970-21kworker/u8:0+events_unbound10:35:251
3023946995648,6cyclictest601706-21systemd-journal12:07:341
3023946995645,9cyclictest3040816-21munin-node08:00:121
3023944995650,4cyclictest3042202-21kworker/u8:3+flush-179:008:10:230
3023946995549,4cyclictest3051696-21kworker/u8:0+events_unbound09:00:141
3023946995547,6cyclictest601706-21systemd-journal12:36:331
3023946995547,6cyclictest3053360-21http08:40:011
3023944995550,4cyclictest3077970-21kworker/u8:0+flush-179:011:00:260
3023944995550,3cyclictest3122833-21kworker/u8:0+events_unbound12:15:250
302394499550,53cyclictest1-21systemd07:21:340
3023944995447,4cyclictest3043992-21kworker/u8:1+flush-179:008:20:180
3023944995347,5cyclictest3109836-21kworker/u8:3+flush-179:011:35:280
3023944995347,5cyclictest3109836-21kworker/u8:3+flush-179:011:35:280
3023946995247,4cyclictest3077970-21kworker/u8:0+events_unbound09:55:251
3023946995247,4cyclictest3077970-21kworker/u8:0+events_unbound09:55:241
3023944995146,4cyclictest3060199-21kworker/u8:1+flush-179:009:05:200
3023944995146,4cyclictest3032570-21kworker/u8:0+events_unbound07:40:240
3023944995144,5cyclictest502-21rs:main0
3023946995044,5cyclictest3051696-21kworker/u8:0+flush-179:008:40:211
3023946995043,6cyclictest3071341-21kworker/u8:2+events_unbound09:40:141
3023944995046,3cyclictest3103511-21systemd-run11:15:010
3023944995045,4cyclictest3035509-21kworker/u8:1+flush-179:007:50:230
3023946994944,4cyclictest3028915-21kworker/u8:3+flush-179:007:31:341
3023946994942,5cyclictest3103518-21kworker/u8:2+events_unbound11:30:121
3023946994941,6cyclictest3076656-21apt-get09:50:111
3023946994940,7cyclictest601706-21systemd-journal10:35:001
3023944994939,8cyclictest502-21rs:main0
4505291480,13ptp4l441ktimers/307:09:553
3023946994843,4cyclictest3113663-21kworker/u8:2+flush-179:012:10:201
3023946994842,5cyclictest3056798-21kworker/u8:3+flush-179:009:29:561
3023944994843,4cyclictest3028851-21kworker/u8:2+events_unbound07:35:200
3023955994744,2cyclictest0-21swapper/312:00:593
3023946994741,5cyclictest3035509-21kworker/u8:1+flush-179:007:45:141
3023946994738,7cyclictest3023801-21latency_hist07:10:011
3023946994737,7cyclictest3095551-21kworker/u8:3+flush-179:011:00:151
3023944994742,4cyclictest3108497-21kworker/u8:1+flush-179:012:20:200
3023944994742,4cyclictest3077970-21kworker/u8:0+events_unbound10:15:230
3023946994641,4cyclictest3103518-21kworker/u8:2+flush-179:011:25:351
3023946994641,4cyclictest3066551-21kworker/u8:0+flush-179:009:19:551
3023946994636,7cyclictest3064979-21latency_hist09:15:011
3023944994640,5cyclictest3077970-21kworker/u8:0+flush-179:010:50:200
3023944994540,4cyclictest3122889-21systemd-run12:15:000
3023946994427,16cyclictest3114917-21perf11:54:551
3023946994338,4cyclictest3025654-21apt-key07:15:021
3023955994240,1cyclictest0-21swapper/311:28:593
3023950994240,1cyclictest0-21swapper/211:29:592
3023950994138,2cyclictest0-21swapper/209:18:582
3023950994138,2cyclictest0-21swapper/209:14:552
3023946994136,4cyclictest3077970-21kworker/u8:0+flush-179:011:21:351
3023955994038,1cyclictest0-21swapper/308:28:003
302395599400,38cyclictest3053334-21turbostat08:40:003
302395099400,38cyclictest0-21swapper/210:29:592
3023955993934,4cyclictest3063405-21turbostat09:10:003
3023955993933,5cyclictest3111449-21turbostat11:40:003
3023955993933,5cyclictest3111449-21turbostat11:40:003
302395599390,38cyclictest3058358-21turbostat08:54:593
302395599390,37cyclictest3093935-21turbostat10:45:003
3023950993936,2cyclictest0-21swapper/211:12:592
3023950993936,2cyclictest0-21swapper/208:48:592
3023950993935,2cyclictest0-21swapper/210:24:562
3023944993934,4cyclictest3103518-21kworker/u8:2+flush-179:011:25:150
3023955993833,4cyclictest3084366-21turbostat10:15:003
3023955993833,4cyclictest3081190-21turbostat10:04:593
3023950993835,2cyclictest0-21swapper/208:50:582
302395099380,35cyclictest0-21swapper/208:35:582
3023955993735,1cyclictest0-21swapper/309:54:593
3023955993732,4cyclictest3089163-21turbostat10:30:003
3023955993731,5cyclictest3037138-21turbostat07:50:003
3023946993732,4cyclictest3122833-21kworker/u8:0+events_unbound12:15:151
3023955993634,1cyclictest0-21swapper/311:54:543
3023955993634,1cyclictest0-21swapper/307:55:593
3023955993633,2cyclictest0-21swapper/307:18:003
302395599360,35cyclictest0-21swapper/312:33:593
3023950993634,1cyclictest0-21swapper/210:09:592
3023950993633,2cyclictest0-21swapper/212:39:552
3023950993633,2cyclictest0-21swapper/210:30:592
3023950993633,2cyclictest0-21swapper/210:15:592
3023950993633,2cyclictest0-21swapper/209:39:002
3023950993633,2cyclictest0-21swapper/209:09:002
302395099360,1cyclictest0-21swapper/211:35:592
302395099360,1cyclictest0-21swapper/211:35:592
3023946993631,4cyclictest3035509-21kworker/u8:1+events_unbound07:50:191
3023955993533,1cyclictest0-21swapper/312:14:553
3023955993530,4cyclictest3113316-21turbostat11:45:013
3023955993530,4cyclictest3109862-21turbostat11:35:003
3023955993530,4cyclictest3025428-21turbostat07:15:003
302395599350,1cyclictest0-21swapper/307:32:003
3023950993532,2cyclictest0-21swapper/211:53:002
3023950993532,2cyclictest0-21swapper/208:07:582
3023950993431,2cyclictest0-21swapper/212:03:592
3023950993431,2cyclictest0-21swapper/208:55:592
3023950993431,2cyclictest0-21swapper/207:44:002
3023950993430,2cyclictest0-21swapper/210:54:592
4505291330,5ptp4l441ktimers/308:20:003
3023955993331,1cyclictest0-21swapper/312:07:003
3023955993331,1cyclictest0-21swapper/311:17:593
3023955993330,2cyclictest0-21swapper/307:27:583
3023950993331,1cyclictest0-21swapper/207:30:012
3023950993330,2cyclictest0-21swapper/211:45:592
3023955993230,1cyclictest0-21swapper/309:22:593
3023955993230,1cyclictest0-21swapper/307:44:593
302395599320,1cyclictest0-21swapper/307:38:593
3023950993229,2cyclictest0-21swapper/210:35:592
3023950993229,2cyclictest0-21swapper/207:23:592
3023950993229,1cyclictest0-21swapper/208:45:002
3023950993228,2cyclictest0-21swapper/209:58:582
3023950993228,2cyclictest0-21swapper/209:58:582
3023955993129,1cyclictest0-21swapper/309:59:553
3023955993129,1cyclictest0-21swapper/309:59:553
4505291300,7ptp4l441ktimers/311:15:013
3023955993027,2cyclictest3126064-21turbostat12:25:013
3023955993027,2cyclictest0-21swapper/312:15:593
3023950993027,2cyclictest0-21swapper/209:45:582
302395099300,2cyclictest0-21swapper/209:00:152
302395099300,28cyclictest0-21swapper/211:15:122
3023955992927,1cyclictest0-21swapper/311:49:563
3023955992927,1cyclictest0-21swapper/307:20:153
302395599290,28cyclictest0-21swapper/310:55:143
302395599290,27cyclictest0-21swapper/309:00:593
302395599290,21cyclictest0-21swapper/310:30:133
3023950992927,1cyclictest0-21swapper/211:25:002
3023950992927,1cyclictest0-21swapper/207:55:162
3023950992926,2cyclictest0-21swapper/210:55:142
3023950992926,2cyclictest0-21swapper/208:25:132
302395099290,27cyclictest0-21swapper/212:20:132
302395099290,27cyclictest0-21swapper/210:40:122
302395099290,27cyclictest0-21swapper/210:10:162
302395099290,27cyclictest0-21swapper/209:25:122
302395099290,27cyclictest0-21swapper/208:30:142
302395099290,27cyclictest0-21swapper/207:15:012
4505291280,6ptp4l61790irq/134-eno1-TxRx-008:05:003
3023955992826,1cyclictest0-21swapper/309:45:593
3023955992826,1cyclictest0-21swapper/308:55:033
3023955992825,2cyclictest3066584-21turbostat09:20:013
3023955992825,2cyclictest0-21swapper/310:15:123
302395599280,4cyclictest3100310-21turbostat11:05:003
302395599280,21cyclictest0-21swapper/312:25:153
3023950992825,2cyclictest0-21swapper/212:25:162
3023950992825,2cyclictest0-21swapper/211:56:582
3023950992825,2cyclictest0-21swapper/210:02:592
302395099280,2cyclictest0-21swapper/211:40:022
302395099280,2cyclictest0-21swapper/207:31:592
4505291270,6ptp4l61790irq/134-eno1-TxRx-012:00:013
4505291270,6ptp4l61790irq/134-eno1-TxRx-011:25:003
4505291270,6ptp4l61790irq/134-eno1-TxRx-009:30:003
4505291270,6ptp4l441ktimers/310:24:553
4505291270,6ptp4l441ktimers/309:14:593
4505291270,5ptp4l441ktimers/308:15:013
4505291270,4ptp4l441ktimers/308:10:013
3023955992725,1cyclictest0-21swapper/309:34:003
3023955992725,1cyclictest0-21swapper/308:40:023
3023955992725,1cyclictest0-21swapper/308:30:133
3023955992725,1cyclictest0-21swapper/307:54:563
302395599270,26cyclictest0-21swapper/310:37:593
3023950992724,2cyclictest0-21swapper/212:15:142
3023950992724,2cyclictest0-21swapper/208:24:592
302395099270,26cyclictest0-21swapper/208:00:012
302395099270,25cyclictest0-21swapper/207:14:032
302395099270,24cyclictest0-21swapper/212:08:002
4505291260,6ptp4l61790irq/134-eno1-TxRx-010:50:003
4505291260,6ptp4l441ktimers/310:10:003
4505291260,6ptp4l441ktimers/308:49:553
4505291260,4ptp4l441ktimers/312:40:013
4505291260,4ptp4l441ktimers/308:25:013
3023955992624,1cyclictest0-21swapper/311:08:593
3023955992623,2cyclictest0-21swapper/309:35:593
302395099260,24cyclictest0-21swapper/211:01:592
302395099260,24cyclictest0-21swapper/209:50:142
302395099260,24cyclictest0-21swapper/209:22:002
302395599250,24cyclictest0-21swapper/309:40:593
3023950992521,2cyclictest0-21swapper/209:31:582
302395099250,2cyclictest0-21swapper/209:44:542
4505291240,4ptp4l441ktimers/310:54:563
3023950992421,2cyclictest0-21swapper/208:15:142
3023950992219,2cyclictest0-21swapper/212:11:592
3023950991816,1cyclictest0-21swapper/210:49:592
3023950991815,2cyclictest0-21swapper/211:05:592
3023950991815,2cyclictest0-21swapper/207:36:592
3023950991613,2cyclictest351rcuc/208:14:552
3023950991613,2cyclictest0-21swapper/212:32:582
3023950991411,2cyclictest0-21swapper/207:51:582
30239509985,2cyclictest0-21swapper/211:30:482
30239509930,2cyclictest0-21swapper/207:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional