You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-02 - 17:22
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot6.osadl.org (updated Tue Dec 02, 2025 12:44:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4287629910490,10cyclictest271-21systemd-journal12:40:010
4287659910190,8cyclictest521502-21latency_hist11:55:011
428765999785,9cyclictest483886-21latency_hist10:00:001
428762999780,12cyclictest471968-21latency_hist09:25:010
428762998675,8cyclictest475400-21latency_hist09:35:020
428765998175,4cyclictest467222-21kworker/u8:2+flush-179:009:30:001
428765997871,5cyclictest454699-21kworker/u8:0+events_unbound09:05:011
428765997868,7cyclictest495011-21latency_hist10:35:001
42876599751,72cyclictest1-21systemd12:40:011
428762997458,12cyclictest482022-21apt-get09:55:000
428765997338,34cyclictest723-21Xorg12:09:591
428765997063,5cyclictest513088-21kworker/u8:2+flush-179:011:30:241
428765996660,4cyclictest428894-21kworker/u8:0+flush-179:007:20:251
428765996659,5cyclictest440184-21kworker/u8:2+events_unbound08:10:241
428762996658,5cyclictest473857-21kworker/u8:3+flush-179:010:20:260
428762996656,8cyclictest488898-21apt-get10:15:120
428765996557,5cyclictest519707-21kworker/u8:1+events_unbound12:00:011
428765996550,13cyclictest499199-21kworker/u8:2+flush-179:011:10:261
428762996557,5cyclictest433695-21latency_hist07:25:010
428762996556,6cyclictest462410-21latency_hist08:55:000
428765996359,3cyclictest513088-21kworker/u8:2+flush-179:011:39:561
428765996357,4cyclictest478603-21kworker/u8:1+flush-179:009:50:001
428765996356,4cyclictest482159-21kworker/u8:2+flush-179:010:35:191
428765996355,6cyclictest443317-21latency_hist07:55:001
428765996352,8cyclictest271-21systemd-journal10:55:001
428765996352,8cyclictest271-21systemd-journal10:54:591
428765996253,6cyclictest513088-21kworker/u8:2+events_unbound12:20:191
428765996057,2cyclictest0-21swapper/107:30:591
428765995950,6cyclictest482159-21kworker/u8:2+flush-179:010:10:191
428762995949,7cyclictest516746-21kworker/u8:0+flush-179:011:40:270
428762995949,7cyclictest509592-21latency_hist11:20:010
428765995852,4cyclictest516746-21kworker/u8:0+flush-179:011:45:241
428765995852,4cyclictest440184-21kworker/u8:2+flush-179:008:00:241
428765995847,8cyclictest451280-21latency_hist08:20:021
428762995834,22cyclictest513048-21ls11:30:000
428765995751,4cyclictest473857-21kworker/u8:3+flush-179:010:05:171
428765995748,7cyclictest452910-21kworker/u8:1+events_unbound08:40:261
428765995748,7cyclictest452910-21kworker/u8:1+events_unbound08:40:251
428765995743,11cyclictest428894-21kworker/u8:0+flush-179:007:20:011
428765995743,10cyclictest513034-21latency_hist11:30:011
428765995649,6cyclictest516746-21kworker/u8:0+flush-179:011:40:271
428765995648,6cyclictest467222-21kworker/u8:2+flush-179:009:20:001
428762995650,4cyclictest467167-21kworker/u8:1+flush-179:009:10:270
428765995548,5cyclictest459276-21kworker/u8:2+flush-179:008:55:011
428762995547,5cyclictest467167-21kworker/u8:1+flush-179:009:20:000
428762995544,8cyclictest441732-21latency_hist07:50:020
42876299551,49cyclictest0-21swapper/010:55:140
428765995449,3cyclictest454699-21kworker/u8:0+flush-179:008:55:191
428765995446,5cyclictest452910-21kworker/u8:1+flush-179:008:25:191
428765995346,5cyclictest505871-21kworker/u8:1+flush-179:011:15:211
42876299531,35cyclictest532902-21apt-get12:30:130
42876299531,35cyclictest532902-21apt-get12:30:130
42876299530,2cyclictest723-21Xorg12:14:590
428765995247,3cyclictest440184-21kworker/u8:2+flush-179:008:35:241
428765995246,5cyclictest440184-21kworker/u8:2+flush-179:008:20:201
428765995244,6cyclictest3421118-21kworker/u8:2+flush-179:007:10:201
428762995242,8cyclictest504822-21apt-get11:05:130
428765995146,4cyclictest473857-21kworker/u8:3+events_unbound10:00:241
428765995145,5cyclictest428894-21kworker/u8:0+events_unbound07:25:241
428765995143,6cyclictest490288-21kworker/u8:1+events_unbound10:25:011
428765995143,6cyclictest467222-21kworker/u8:2+flush-179:009:35:211
428765995046,3cyclictest490288-21kworker/u8:1+flush-179:010:25:261
428765995045,4cyclictest473857-21kworker/u8:3+flush-179:010:20:011
428765995042,5cyclictest507777-21kworker/u8:3+events_unbound11:20:201
428765995039,7cyclictest496063-21kworker/u8:1+flush-179:010:45:001
428762995044,5cyclictest467167-21kworker/u8:1+events_unbound09:30:020
428762995039,9cyclictest449682-21latency_hist08:15:010
428765994943,5cyclictest473857-21kworker/u8:3+events_unbound09:50:141
428765994943,4cyclictest501345-21kworker/u8:3+flush-179:011:00:201
428765994940,7cyclictest271-21systemd-journal12:03:191
428765994940,7cyclictest271-21systemd-journal12:03:181
428762994941,7cyclictest462665-21apt-get08:55:150
428762994939,7cyclictest487058-21latency_hist10:10:010
42876299490,45cyclictest519669-21apt-get11:50:030
428765994843,4cyclictest428894-21kworker/u8:0+flush-179:007:50:011
428765994839,6cyclictest467425-21/usr/sbin/munin09:10:181
428762994840,6cyclictest440184-21kworker/u8:2+events_unbound08:05:010
428762994815,20cyclictest0-21swapper/010:35:180
4670791470,14ptp4l61690irq/136-eno1-TxRx-007:10:003
428762994738,7cyclictest490288-21kworker/u8:1+events_unbound10:25:260
428762994730,15cyclictest480433-21cron09:50:010
428765994640,5cyclictest501345-21kworker/u8:3+flush-179:011:00:001
428765994633,10cyclictest271-21systemd-journal08:34:191
42876299461,43cyclictest501370-21cron10:54:590
42876299461,43cyclictest501370-21cron10:54:590
42876299461,16cyclictest459496-21apt-get08:45:150
428765994541,3cyclictest446547-21kworker/u8:0+flush-179:008:09:551
428765994540,4cyclictest526313-21kworker/u8:1+flush-179:012:15:211
428765994540,4cyclictest452910-21kworker/u8:1+flush-179:008:45:191
428762994532,10cyclictest431858-21apt-get07:20:000
42876299451,6cyclictest514895-21basename11:35:000
428762994437,6cyclictest519661-21grep11:50:000
428762994437,5cyclictest271-21systemd-journal07:41:190
42876299441,41cyclictest529470-21munin-run12:20:000
428765994336,4cyclictest462439-21kworker/u8:3+flush-179:009:05:211
428765994336,4cyclictest462439-21kworker/u8:3+flush-179:009:05:211
428762994335,6cyclictest1-21systemd11:40:010
428762994335,4cyclictest507748-21basename11:15:000
428762994333,8cyclictest521755-21apt-get11:55:130
42876299431,27cyclictest2523597-21snmpd10:43:190
428771994239,2cyclictest0-21swapper/307:44:553
42877199420,40cyclictest514889-21turbostat11:35:003
428765994236,4cyclictest528102-21kworker/u8:0+flush-179:012:30:161
428765994236,4cyclictest528102-21kworker/u8:0+flush-179:012:30:161
428765994232,7cyclictest437194-21kworker/u8:1+flush-179:007:35:261
428762994235,4cyclictest435293-21apt07:30:000
428762994233,7cyclictest466900-21cat09:05:250
428762994233,7cyclictest466900-21cat09:05:240
428771994137,3cyclictest0-21swapper/311:18:593
42877199410,38cyclictest501376-21turbostat10:54:593
42877199410,38cyclictest501376-21turbostat10:54:593
42877199410,36cyclictest0-21swapper/310:39:553
42876899410,38cyclictest0-21swapper/207:44:552
428762994138,2cyclictest0-21swapper/009:37:590
428762994132,7cyclictest501345-21kworker/u8:3+flush-179:011:00:240
428762994132,7cyclictest440184-21kworker/u8:2+events_unbound08:25:000
428762994131,7cyclictest271-21systemd-journal10:46:190
428762994122,17cyclictest532310-21switchtime12:25:240
428771994034,4cyclictest529477-21turbostat12:20:003
428768994027,11cyclictest0-21swapper/211:30:002
428765994034,5cyclictest528102-21kworker/u8:0+flush-179:012:25:231
428765994034,4cyclictest526313-21kworker/u8:1+flush-179:012:10:201
428765994034,4cyclictest440184-21kworker/u8:2+events_unbound07:55:201
428765994033,5cyclictest499199-21kworker/u8:2+events_unbound11:05:241
428762994021,18cyclictest2126-10pipewire-media-12:05:260
428762994020,18cyclictest484986-21phc2sys-jitter10:00:220
42876299401,2cyclictest454475-21perf08:30:000
428771993934,4cyclictest471973-21turbostat09:25:003
428771993934,4cyclictest449687-21turbostat08:15:013
42877199390,38cyclictest0-21swapper/309:19:543
428768993934,3cyclictest0-21swapper/209:20:002
42876899390,35cyclictest0-21swapper/208:30:002
42876899390,2cyclictest0-21swapper/209:23:592
428765993933,4cyclictest493458-21kworker/u8:0+flush-179:010:50:001
428765993922,16cyclictest0-21swapper/107:45:011
428762993930,6cyclictest271-21systemd-journal07:30:130
42876299390,3cyclictest723-21Xorg09:02:580
42877199380,35cyclictest0-21swapper/312:32:593
42877199380,35cyclictest0-21swapper/312:32:593
42877199380,35cyclictest0-21swapper/309:46:593
428768993832,5cyclictest0-21swapper/210:19:552
42876899380,1cyclictest0-21swapper/211:50:002
428765993833,3cyclictest472026-21systemd-run09:25:011
428762993834,2cyclictest0-21swapper/007:39:580
428762993833,3cyclictest457631-21cron08:40:000
428762993832,4cyclictest454699-21kworker/u8:0+flush-179:008:30:250
428762993829,7cyclictest478599-21apt-get09:45:000
428771993735,1cyclictest0-21swapper/311:05:593
428771993730,5cyclictest446520-21turbostat08:05:003
42877199370,35cyclictest0-21swapper/312:02:593
42877199370,35cyclictest0-21swapper/312:02:593
428768993734,1cyclictest0-21swapper/208:15:012
428768993732,2cyclictest0-21swapper/212:13:002
42876899370,1cyclictest0-21swapper/208:09:552
428765993732,4cyclictest467222-21kworker/u8:2+flush-179:009:40:251
428762993732,3cyclictest495016-21munin-run10:35:000
428762993730,3cyclictest488646-21munin-run10:15:000
428771993630,4cyclictest441742-21turbostat07:50:013
428768993633,2cyclictest0-21swapper/210:49:552
428768993633,2cyclictest0-21swapper/210:41:592
428768993633,2cyclictest0-21swapper/208:59:552
428768993631,2cyclictest0-21swapper/209:38:002
42876899360,29cyclictest0-21swapper/212:32:592
42876899360,29cyclictest0-21swapper/212:32:592
42876899360,1cyclictest0-21swapper/212:02:592
42876899360,1cyclictest0-21swapper/212:02:582
42876899360,1cyclictest0-21swapper/210:39:552
42876899360,1cyclictest0-21swapper/209:46:582
428765993631,4cyclictest467167-21kworker/u8:1+flush-179:009:30:361
428762993632,3cyclictest524730-21systemd-run12:05:000
428762993632,3cyclictest524730-21systemd-run12:05:000
428771993533,1cyclictest0-21swapper/310:49:553
428771993532,2cyclictest0-21swapper/310:19:553
428771993532,2cyclictest0-21swapper/308:54:593
428771993530,4cyclictest485481-21turbostat10:04:593
428768993532,2cyclictest0-21swapper/208:54:592
42876899350,33cyclictest0-21swapper/211:50:032
42876899350,33cyclictest0-21swapper/207:20:032
428762993515,17cyclictest0-21swapper/009:55:010
428771993432,1cyclictest0-21swapper/310:41:593
428771993432,1cyclictest0-21swapper/308:59:563
428771993430,2cyclictest0-21swapper/309:37:593
428771993428,5cyclictest516498-21turbostat11:40:013
428771993427,5cyclictest435290-21turbostat07:30:003
428768993431,2cyclictest0-21swapper/208:04:592
428768993430,2cyclictest0-21swapper/210:09:542
42876899340,32cyclictest0-21swapper/208:45:162
428762993431,2cyclictest0-21swapper/007:53:590
428762993428,5cyclictest449687-21turbostat08:19:550
428762993416,16cyclictest531185-21latency_hist12:25:010
42876299341,27cyclictest0-21swapper/007:55:170
42877199333,3cyclictest0-21swapper/311:50:033
428771993331,1cyclictest0-21swapper/312:13:003
428771993328,3cyclictest465617-21turbostat09:09:553
428771993328,3cyclictest465617-21turbostat09:09:553
428768993330,2cyclictest0-21swapper/207:50:002
428768993329,2cyclictest0-21swapper/211:10:592
428768993329,2cyclictest0-21swapper/209:09:562
428768993329,2cyclictest0-21swapper/209:09:552
42876299331,4cyclictest3402479-21munin-node11:20:130
42876299330,10cyclictest723-21Xorg07:11:590
42877199324,26cyclictest0-21swapper/311:46:593
42877199324,26cyclictest0-21swapper/307:20:023
428771993229,2cyclictest0-21swapper/310:09:553
428771993227,4cyclictest507745-21turbostat11:15:003
42877199321,29cyclictest0-21swapper/308:45:153
428768993230,1cyclictest0-21swapper/210:04:592
428768993228,2cyclictest0-21swapper/208:22:582
42876899320,31cyclictest0-21swapper/210:55:142
42876899320,30cyclictest0-21swapper/212:15:142
42876899320,29cyclictest0-21swapper/207:19:142
42876299321,26cyclictest0-21swapper/008:05:160
42877199311,27cyclictest0-21swapper/310:55:143
428768993129,1cyclictest0-21swapper/211:40:012
428771993026,3cyclictest513044-21turbostat11:30:003
428771993025,4cyclictest468785-21turbostat09:15:003
428771993025,4cyclictest457649-21turbostat08:40:003
428771993024,5cyclictest495019-21turbostat10:35:003
428771993023,6cyclictest0-21swapper/309:25:033
428768993027,2cyclictest0-21swapper/207:29:592
428768993026,2cyclictest0-21swapper/211:55:592
428768993026,2cyclictest0-21swapper/211:20:132
428768993024,2cyclictest0-21swapper/207:53:592
42876299301,8cyclictest458855-21/usr/sbin/munin08:40:250
42876299301,8cyclictest458855-21/usr/sbin/munin08:40:250
42877199294,18cyclictest0-21swapper/307:19:143
428771992926,2cyclictest0-21swapper/307:39:593
428771992925,1cyclictest0-21swapper/310:20:133
428771992923,4cyclictest488660-21turbostat10:15:003
42877199290,2cyclictest0-21swapper/308:40:163
42877199290,2cyclictest0-21swapper/308:40:163
42877199290,29cyclictest0-21swapper/308:20:133
42877199290,28cyclictest0-21swapper/311:56:593
42877199290,26cyclictest0-21swapper/311:20:133
42877199290,21cyclictest0-21swapper/309:55:023
428768992927,1cyclictest0-21swapper/207:39:592
428768992926,2cyclictest0-21swapper/210:20:132
428768992926,2cyclictest0-21swapper/209:30:132
428768992926,2cyclictest0-21swapper/209:25:022
428768992926,2cyclictest0-21swapper/209:14:592
428768992925,2cyclictest0-21swapper/212:35:162
42876899290,2cyclictest0-21swapper/211:30:132
42876899290,27cyclictest0-21swapper/211:05:152
42876899290,27cyclictest0-21swapper/208:40:152
42876899290,27cyclictest0-21swapper/208:40:142
42876899290,27cyclictest0-21swapper/208:15:122
42876899290,21cyclictest0-21swapper/211:15:032
428771992826,1cyclictest0-21swapper/307:10:133
428771992825,2cyclictest0-21swapper/312:35:163
428771992824,2cyclictest0-21swapper/307:53:593
42877199280,27cyclictest0-21swapper/308:15:113
42877199280,1cyclictest0-21swapper/309:30:143
428768992825,2cyclictest0-21swapper/208:40:002
428768992825,2cyclictest0-21swapper/207:10:592
428768992825,1cyclictest0-21swapper/209:55:032
42876299281,24cyclictest428758-21cyclictest07:10:010
428771992722,4cyclictest532660-21turbostat12:30:003
428771992722,4cyclictest504573-21turbostat11:05:003
428768992724,2cyclictest0-21swapper/207:32:002
42876899270,24cyclictest0-21swapper/208:33:592
4670791260,7ptp4l61690irq/136-eno1-TxRx-009:05:013
4670791260,6ptp4l441ktimers/309:45:003
428771992624,1cyclictest0-21swapper/308:25:593
428771992623,2cyclictest0-21swapper/307:32:003
42877199260,1cyclictest0-21swapper/311:44:563
428768992624,1cyclictest0-21swapper/210:35:012
428768992623,2cyclictest0-21swapper/212:30:012
428768992623,2cyclictest0-21swapper/210:52:002
428768992623,2cyclictest0-21swapper/210:51:592
428768992623,2cyclictest0-21swapper/210:27:592
428768992623,2cyclictest0-21swapper/210:14:002
428768992622,2cyclictest0-21swapper/211:43:592
42876899260,24cyclictest0-21swapper/209:01:592
4670791250,6ptp4l61690irq/136-eno1-TxRx-012:25:013
4670791250,6ptp4l61690irq/136-eno1-TxRx-009:54:553
4670791250,5ptp4l441ktimers/312:10:003
428771992520,4cyclictest0-21swapper/308:05:593
428771992519,4cyclictest493443-21turbostat10:30:013
428768992523,1cyclictest0-21swapper/211:05:002
4670791240,6ptp4l441ktimers/308:35:003
4670791240,4ptp4l441ktimers/308:00:003
428768992320,2cyclictest0-21swapper/212:07:592
428768992118,2cyclictest0-21swapper/209:50:142
428768991917,1cyclictest0-21swapper/209:45:002
428768991915,2cyclictest0-21swapper/212:21:592
428768991914,2cyclictest0-21swapper/207:55:592
42876599160,15cyclictest0-21swapper/107:10:011
4287689990,7cyclictest0-21swapper/207:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional