You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-20 - 04:01
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot6.osadl.org (updated Thu Nov 20, 2025 00:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31074929911394,14cyclictest3189841-40logrotate00:00:010
3107492999584,8cyclictest3109358-21latency_hist19:45:020
3107492999484,7cyclictest3131732-21latency_hist21:00:010
3107492999388,4cyclictest3148145-21kworker/u8:0+events_unbound21:55:230
3107492998775,5cyclictest3110121-21kworker/u8:0+events_unbound19:50:450
3107492998681,4cyclictest3138156-21kworker/u8:3+events_unbound21:30:230
310749299850,84cyclictest1-21systemd19:15:000
3107492998377,4cyclictest3127207-21kworker/u8:2+flush-179:021:25:220
3107495998274,5cyclictest3134927-21kworker/u8:0+events_unbound21:20:021
3107492998278,3cyclictest3144512-21kworker/u8:3+events_unbound21:40:230
3107492998269,5cyclictest3108364-21kworker/u8:2+events_unbound20:00:220
3107495997863,10cyclictest3151315-21kworker/u8:1+flush-179:022:10:001
3107492997873,4cyclictest3196318-21kworker/u8:1+events_unbound00:35:240
3107492997873,4cyclictest3127207-21kworker/u8:2+events_unbound21:45:230
3107492997672,3cyclictest3173713-21kworker/u8:2+events_unbound23:20:220
3107495997563,8cyclictest271-21systemd-journal19:55:011
3107495997465,6cyclictest3179238-21kworker/u8:3+flush-179:023:40:001
3107495997465,6cyclictest3179238-21kworker/u8:3+flush-179:023:40:001
3107492997469,4cyclictest3108364-21kworker/u8:2+events_unbound20:05:240
3107492997469,4cyclictest3108364-21kworker/u8:2+events_unbound20:05:230
3107492997160,8cyclictest3173677-21latency_hist23:10:010
3107492997065,4cyclictest3162523-21kworker/u8:1+events_unbound22:35:250
310749899690,66cyclictest0-21swapper/320:34:003
3107495996963,4cyclictest3108364-21kworker/u8:2+flush-179:019:55:251
3107495996956,10cyclictest3109365-21apt-get19:45:001
310749699680,65cyclictest0-21swapper/220:34:002
3107492996864,3cyclictest3196318-21kworker/u8:1+flush-179:000:30:000
3107492996862,4cyclictest3109345-21kworker/u8:3+events_unbound19:55:070
3107495996756,8cyclictest3133335-21latency_hist21:05:011
3107495996753,9cyclictest3127207-21kworker/u8:2+flush-179:021:35:221
3107492996763,3cyclictest3144512-21kworker/u8:3+events_unbound22:20:130
3107492996763,3cyclictest3108364-21kworker/u8:2+events_unbound20:10:240
3107492996761,4cyclictest3173713-21kworker/u8:2+events_unbound23:15:200
3107495996657,6cyclictest271-21systemd-journal23:35:011
3107492996561,3cyclictest3134927-21kworker/u8:0+events_unbound21:10:220
3107492996555,8cyclictest3131984-21/usr/sbin/munin21:00:230
3107492996460,3cyclictest3118714-21kworker/u8:4+events_unbound20:25:190
3107492996459,4cyclictest3173713-21kworker/u8:2+flush-179:023:35:200
3107492996459,4cyclictest3173713-21kworker/u8:2+flush-179:023:35:200
3107495996341,20cyclictest3146318-21apt-get21:45:091
3107495996252,7cyclictest3175262-21latency_hist23:15:001
3107492996258,3cyclictest3179238-21kworker/u8:3+events_unbound00:04:030
3107492996249,9cyclictest3149493-21latency_hist21:55:000
3107495996150,9cyclictest3125641-21apt-get20:40:131
3107492996156,4cyclictest3165774-21kworker/u8:0+events_unbound22:50:200
3107492996150,9cyclictest3196540-21apt-get00:20:110
3107492996150,9cyclictest3196540-21apt-get00:20:100
3107495996044,14cyclictest3123764-21kworker/u8:1+flush-179:020:50:251
3107492995955,3cyclictest3127207-21kworker/u8:2+events_unbound20:45:190
3107492995854,3cyclictest3162523-21kworker/u8:1+flush-179:022:41:210
3107495995738,17cyclictest3123764-21kworker/u8:1+flush-179:021:25:001
3107492995752,4cyclictest3144512-21kworker/u8:3+events_unbound22:15:170
3107492995747,8cyclictest3162523-21kworker/u8:1+flush-179:023:05:000
3107495995646,7cyclictest3134918-21latency_hist21:10:001
3107492995652,3cyclictest3172076-21kworker/u8:0+events_unbound23:14:560
3107492995652,3cyclictest191rcuc/020:34:000
3107492995645,9cyclictest3133589-21apt-get21:05:130
3107495995549,3cyclictest3193123-21kworker/u8:2+events_unbound00:15:251
3107495995547,5cyclictest3138156-21kworker/u8:3+events_unbound21:25:271
3107495995546,6cyclictest3157773-21latency_hist22:20:001
3107492995545,4cyclictest3151315-21kworker/u8:1+events_unbound22:10:130
3107492995450,3cyclictest3199695-21kworker/u8:2+events_unbound00:30:240
3107492995449,4cyclictest3127207-21kworker/u8:2+events_unbound20:54:200
3107492995349,3cyclictest3179238-21kworker/u8:3+flush-179:023:45:250
3107495995246,4cyclictest3109345-21kworker/u8:3+flush-179:020:05:171
3107495995246,4cyclictest3109345-21kworker/u8:3+flush-179:020:05:171
3107495995244,6cyclictest3108364-21kworker/u8:2+flush-179:020:14:591
3107492995247,4cyclictest3191492-21kworker/u8:0+flush-179:000:17:210
3107495995145,5cyclictest3172076-21kworker/u8:0+events_unbound23:15:241
3107495995140,8cyclictest271-21systemd-journal23:21:161
3107492995147,3cyclictest3172076-21kworker/u8:0+events_unbound23:25:290
3107492995146,4cyclictest3172076-21kworker/u8:0+events_unbound23:30:170
3107492995045,4cyclictest3138156-21kworker/u8:3+events_unbound21:20:220
3107495994944,4cyclictest3103935-21kworker/u8:3+flush-179:019:24:551
3107495994942,5cyclictest3108364-21kworker/u8:2+events_unbound20:05:011
3107495994941,6cyclictest3109345-21kworker/u8:3+flush-179:020:15:171
3107495994940,8cyclictest3122447-21apt-get20:30:151
3107492994940,3cyclictest3109345-21kworker/u8:3+events_unbound20:18:190
3107495994840,6cyclictest3191492-21kworker/u8:0+events_unbound00:20:231
3107495994840,6cyclictest3191492-21kworker/u8:0+events_unbound00:20:231
3107492994844,3cyclictest3159387-21kworker/u8:2+events_unbound22:25:240
3107492994844,3cyclictest3144512-21kworker/u8:3+events_unbound22:01:030
4670791470,11ptp4l61690irq/136-eno1-TxRx-019:05:203
3107495994741,4cyclictest3168932-21kworker/u8:3+events_unbound23:00:251
3107495994737,8cyclictest3109099-21latency_hist19:40:011
3107492994743,3cyclictest3191492-21kworker/u8:0+flush-179:000:05:000
3107492994743,3cyclictest3179238-21kworker/u8:3+flush-179:023:40:230
3107492994743,3cyclictest3168877-21kworker/u8:2+flush-179:022:59:550
3107492994743,3cyclictest3162524-21kworker/u8:4+flush-179:022:34:550
3107492994742,4cyclictest3162523-21kworker/u8:1+events_unbound22:45:150
3107495994641,4cyclictest271-21systemd-journal19:45:431
3107495994640,4cyclictest3191458-21systemd00:04:001
3107495994639,5cyclictest3179238-21kworker/u8:3+flush-179:023:50:001
3107495994637,6cyclictest3179238-21kworker/u8:3+flush-179:000:00:011
3107492994641,4cyclictest3108361-21kworker/u8:1+flush-179:019:29:540
3107492994639,5cyclictest3118714-21kworker/u8:4+flush-179:020:45:000
3107492994541,3cyclictest3107253-21kworker/u8:0+flush-179:019:34:560
3107498994439,2cyclictest3159369-21turbostat22:25:013
3107495994436,6cyclictest3151315-21kworker/u8:1+flush-179:022:05:001
3107495994433,9cyclictest3165984-21apt-get22:45:121
3107492994439,4cyclictest3109345-21kworker/u8:3+events_unbound20:21:180
3107496994339,2cyclictest0-21swapper/222:25:002
3107495994337,5cyclictest3123764-21kworker/u8:1+flush-179:021:35:001
3107495994336,5cyclictest3151315-21kworker/u8:1+flush-179:022:00:011
3107492994339,3cyclictest3193123-21kworker/u8:2+events_unbound00:13:540
3107492994333,7cyclictest3154531-21latency_hist22:10:010
310749299431,40cyclictest3107812-21turbostat19:19:560
3107495994239,2cyclictest723-21Xorg20:22:001
3107495994231,8cyclictest3149504-21apt-get21:54:591
3107492994238,3cyclictest3127207-21kworker/u8:2+flush-179:021:16:200
3107492994237,4cyclictest3118714-21kworker/u8:4+flush-179:020:35:270
3107498994136,3cyclictest0-21swapper/323:19:543
3107496994137,2cyclictest0-21swapper/223:19:552
3107495994130,6cyclictest3159617-21apt-get22:25:121
3107492994133,3cyclictest3179238-21kworker/u8:3+events_unbound23:50:020
3107498994034,5cyclictest3193104-21turbostat00:09:593
3107498994034,5cyclictest3152957-21turbostat22:05:013
310749899400,5cyclictest3109372-21turbostat19:45:013
310749899400,38cyclictest3120618-21turbostat20:25:003
310749699400,30cyclictest0-21swapper/220:25:002
3107495994035,3cyclictest723-21Xorg19:13:591
310749899390,4cyclictest3108070-21turbostat19:20:013
310749899390,37cyclictest0-21swapper/322:43:593
310749699390,36cyclictest0-21swapper/223:30:002
310749699390,36cyclictest0-21swapper/223:04:552
310749699390,2cyclictest0-21swapper/222:11:592
4670791380,4ptp4l431rcuc/322:10:003
3107498993834,2cyclictest0-21swapper/319:39:593
3107498993833,4cyclictest3159369-21turbostat22:29:553
3107498993832,5cyclictest3144482-21turbostat21:40:003
310749899380,1cyclictest0-21swapper/321:09:553
3107496993834,2cyclictest0-21swapper/222:29:562
310749699380,35cyclictest0-21swapper/222:43:592
310749699380,35cyclictest0-21swapper/220:50:002
310749699380,35cyclictest0-21swapper/219:15:012
310749699380,35cyclictest0-21swapper/200:25:002
310749699380,35cyclictest0-21swapper/200:25:002
3107495993835,2cyclictest0-21swapper/123:42:581
3107495993834,2cyclictest723-21Xorg19:16:591
3107495993831,5cyclictest3118714-21kworker/u8:4+flush-179:020:30:001
3107495993829,7cyclictest3144512-21kworker/u8:3+flush-179:022:40:001
3107495993829,7cyclictest3130396-21apt-get20:55:101
310749599380,37cyclictest723-21Xorg00:06:591
3107498993731,5cyclictest3194690-21turbostat00:15:003
3107498993729,6cyclictest3108339-21turbostat19:25:003
310749899370,1cyclictest0-21swapper/320:04:563
310749899370,1cyclictest0-21swapper/319:58:593
3107496993734,2cyclictest0-21swapper/219:39:582
3107496993734,1cyclictest0-21swapper/221:29:592
3107496993733,2cyclictest0-21swapper/200:14:592
310749699370,2cyclictest0-21swapper/221:09:542
310749699370,2cyclictest0-21swapper/220:04:562
310749699370,2cyclictest0-21swapper/219:58:582
3107495993731,5cyclictest3154536-21turbostat22:14:551
3107495993731,4cyclictest3191492-21kworker/u8:0+flush-179:000:25:171
3107492993733,3cyclictest3127207-21kworker/u8:2+events_unbound21:35:020
3107498993630,4cyclictest3186424-21turbostat23:50:003
3107498993630,4cyclictest3130147-21turbostat20:55:013
3107498993629,5cyclictest3167320-21turbostat22:50:003
310749899360,1cyclictest0-21swapper/322:11:593
310749899360,1cyclictest0-21swapper/321:00:593
3107496993634,1cyclictest0-21swapper/222:05:002
3107496993634,1cyclictest0-21swapper/219:29:592
3107496993631,2cyclictest0-21swapper/200:17:002
310749699360,1cyclictest0-21swapper/221:19:552
3107495993630,5cyclictest3196318-21kworker/u8:1+flush-179:000:35:231
3107495993630,4cyclictest3194760-21latency_hist00:15:011
3107492993631,3cyclictest3109106-21grep19:40:010
3107498993533,1cyclictest0-21swapper/321:30:003
3107498993533,1cyclictest0-21swapper/319:29:583
3107498993530,4cyclictest3147664-21turbostat21:50:003
310749899350,34cyclictest0-21swapper/323:04:553
3107496993533,1cyclictest0-21swapper/221:40:002
3107495993533,2cyclictest0-21swapper/122:43:591
3107495993525,7cyclictest3172076-21kworker/u8:0+flush-179:023:25:241
3107495993524,9cyclictest0-21swapper/100:34:591
3107498993428,4cyclictest3149496-21turbostat21:54:593
3107496993431,2cyclictest0-21swapper/223:50:002
3107496993430,2cyclictest0-21swapper/222:47:592
3107496993430,2cyclictest0-21swapper/221:15:002
3107496993429,2cyclictest0-21swapper/222:09:562
3107495993432,1cyclictest0-21swapper/123:09:551
3107495993430,3cyclictest3168905-21munin-run22:55:001
3107495993430,2cyclictest3156151-21kworker/u8:0+events_unbound22:30:211
3107495993428,5cyclictest3168877-21kworker/u8:2+events_unbound22:55:201
3107498993329,3cyclictest0-21swapper/321:14:593
3107498993328,4cyclictest3201055-21turbostat00:35:003
3107498993327,5cyclictest3114257-21turbostat20:09:563
3107498993327,5cyclictest3114257-21turbostat20:09:553
3107496993330,2cyclictest0-21swapper/221:50:002
3107496993330,2cyclictest0-21swapper/221:24:552
3107496993328,3cyclictest0-21swapper/223:22:592
3107498993230,1cyclictest0-21swapper/300:17:003
3107498993226,4cyclictest3178471-21turbostat23:25:003
3107498993225,5cyclictest3109874-21turbostat19:49:593
3107496993230,1cyclictest0-21swapper/220:55:012
3107496993230,1cyclictest0-21swapper/219:44:592
3107496993229,2cyclictest0-21swapper/221:54:592
3107496993228,2cyclictest0-21swapper/223:40:592
3107495993223,7cyclictest271-21systemd-journal19:35:011
310749599320,31cyclictest0-21swapper/123:50:161
3107492993228,3cyclictest3107584-21apt-key19:10:020
3107492993217,5cyclictest14-21ksoftirqd/019:49:590
3107498993128,2cyclictest0-21swapper/321:24:553
3107498993127,2cyclictest0-21swapper/323:40:593
3107496993129,1cyclictest0-21swapper/219:25:002
3107496993129,1cyclictest0-21swapper/200:35:002
3107496993127,2cyclictest0-21swapper/223:55:592
3107496993127,2cyclictest0-21swapper/222:54:552
310749699310,2cyclictest0-21swapper/223:35:122
310749699310,2cyclictest0-21swapper/223:35:112
3107495993114,15cyclictest0-21swapper/120:35:221
3107498993025,4cyclictest3183225-21turbostat23:40:003
3107498993025,4cyclictest3183225-21turbostat23:40:003
3107498993024,4cyclictest3119034-21turbostat20:19:593
3107496993027,2cyclictest0-21swapper/220:09:562
3107496993027,2cyclictest0-21swapper/220:09:552
310749699300,2cyclictest0-21swapper/223:30:142
310749699300,28cyclictest0-21swapper/220:25:142
3107495993027,2cyclictest0-21swapper/122:20:131
310749599300,29cyclictest0-21swapper/120:45:121
3107498992926,2cyclictest0-21swapper/323:55:593
3107498992924,3cyclictest3146073-21turbostat21:45:013
310749899290,27cyclictest0-21swapper/320:40:143
3107496992926,2cyclictest0-21swapper/222:16:592
3107496992926,2cyclictest0-21swapper/221:45:002
3107496992926,2cyclictest0-21swapper/220:55:122
3107496992925,2cyclictest0-21swapper/223:05:592
310749699290,27cyclictest0-21swapper/220:40:142
3107498992826,1cyclictest0-21swapper/322:54:553
3107498992826,1cyclictest0-21swapper/320:55:113
3107498992826,1cyclictest0-21swapper/320:25:153
3107498992825,2cyclictest0-21swapper/323:30:143
3107498992825,2cyclictest0-21swapper/322:16:593
310749899280,27cyclictest0-21swapper/323:25:163
310749899280,1cyclictest0-21swapper/323:05:593
3107496992826,1cyclictest0-21swapper/219:50:002
3107496992824,2cyclictest0-21swapper/222:57:592
3107496992824,2cyclictest0-21swapper/222:32:592
310749699280,24cyclictest0-21swapper/219:50:142
3107495992822,3cyclictest0-21swapper/119:28:591
3107492992824,3cyclictest3103935-21kworker/u8:3+events_unbound19:23:480
4670791270,6ptp4l441ktimers/320:15:003
3107498992724,2cyclictest0-21swapper/319:54:593
3107498992724,2cyclictest0-21swapper/300:24:593
3107498992724,2cyclictest0-21swapper/300:24:583
310749899270,25cyclictest0-21swapper/323:13:593
310749899270,1cyclictest0-21swapper/320:36:593
3107496992724,2cyclictest0-21swapper/220:20:002
310749699270,2cyclictest0-21swapper/220:37:002
3107495992722,4cyclictest3134927-21kworker/u8:0+flush-179:021:10:251
3107498992623,2cyclictest0-21swapper/322:57:593
3107498992623,2cyclictest0-21swapper/322:32:593
3107498992623,2cyclictest0-21swapper/321:56:593
3107496992623,2cyclictest0-21swapper/223:10:592
3107496992623,2cyclictest0-21swapper/221:03:592
3107496992622,2cyclictest0-21swapper/221:55:592
310749699260,24cyclictest0-21swapper/219:15:082
3107495992617,7cyclictest3144512-21kworker/u8:3+flush-179:021:45:001
4670791250,6ptp4l61690irq/136-eno1-TxRx-020:50:023
4670791250,6ptp4l441ktimers/322:40:013
4670791250,5ptp4l441ktimers/321:19:553
3107498992422,1cyclictest0-21swapper/323:51:593
3107496992422,1cyclictest0-21swapper/220:14:592
3107496992421,2cyclictest0-21swapper/200:05:152
3107496992421,2cyclictest0-21swapper/200:04:562
4670791230,3ptp4l441ktimers/321:34:553
4670791230,3ptp4l441ktimers/319:15:003
4670791230,3ptp4l441ktimers/300:30:003
3107498992320,2cyclictest0-21swapper/300:02:003
4670791220,3ptp4l61690irq/136-eno1-TxRx-019:34:543
4670791210,3ptp4l441ktimers/300:39:553
3107496992118,2cyclictest0-21swapper/200:37:592
3107496992116,2cyclictest0-21swapper/219:32:582
3107495991914,4cyclictest0-21swapper/119:10:021
3107496991714,2cyclictest0-21swapper/223:50:592
3107496991713,2cyclictest0-21swapper/200:27:582
310749699100,9cyclictest0-21swapper/222:37:502
31074969990,8cyclictest0-21swapper/221:31:212
31074969940,2cyclictest0-21swapper/219:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional