You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-11 - 14:15
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot6.osadl.org (updated Tue Nov 11, 2025 00:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2858430998881,4cyclictest2891982-21latency_hist20:55:001
2858430998683,2cyclictest0-21swapper/120:47:591
2858426998271,8cyclictest2866265-21latency_hist19:35:000
2858430997563,11cyclictest2845265-21kworker/u8:2+flush-179:019:20:001
2858426997563,9cyclictest2965208-21latency_hist00:40:010
2858436997366,5cyclictest2882461-21turbostat20:25:013
2858430997354,18cyclictest2892016-21kworker/u8:2+flush-179:021:00:001
2858430997062,5cyclictest2915282-21/usr/sbin/munin22:05:231
2858430996856,9cyclictest2922994-21latency_hist22:30:011
2858426996863,4cyclictest0-21swapper/021:43:590
2858426996857,8cyclictest2926441-21latency_hist22:40:010
2858426996857,8cyclictest2926441-21latency_hist22:40:000
2858430996761,4cyclictest723-21Xorg23:29:591
2858426996758,7cyclictest2960670-21/usr/sbin/munin00:25:240
2858430996656,7cyclictest271-21systemd-journal21:49:561
2858430996559,4cyclictest2934378-21kworker/u8:2+flush-179:000:03:591
2858430996455,7cyclictest271-21systemd-journal23:25:011
2858430996357,4cyclictest2875799-21kworker/u8:2+flush-179:020:20:261
2858426996357,5cyclictest723-21Xorg00:09:580
2858436996232,1cyclictest0-21swapper/319:16:593
2858430996250,11cyclictest2892016-21kworker/u8:2+flush-179:021:24:561
285843099621,57cyclictest2960185-21munin-run00:25:001
2858430996151,7cyclictest2934400-21latency_hist23:05:011
2858430996054,5cyclictest2898906-21kworker/u8:1+flush-179:021:20:011
2858426996049,8cyclictest2910018-21apt-get21:50:000
2858430995854,2cyclictest723-21Xorg20:09:581
2858430995846,10cyclictest2895184-21apt-get21:05:001
2858435995653,2cyclictest0-21swapper/200:04:592
2858426995645,9cyclictest2897821-21diskstats21:10:140
2858430995549,4cyclictest2897002-21kworker/u8:0+flush-179:021:10:001
2858430995545,7cyclictest2958572-21latency_hist00:20:001
2858430995449,4cyclictest2875843-21kworker/u8:3+flush-179:020:10:261
2858430995434,18cyclictest2885622-21idleruntime-cro20:35:011
2858426995347,4cyclictest475-21dbus-daemon00:04:000
2858426995344,7cyclictest271-21systemd-journal20:56:390
2858430995242,8cyclictest2955650-21apt-get00:10:121
2858430995223,26cyclictest2864136-21kworker/u8:0+flush-179:019:25:211
2858426995247,4cyclictest723-21Xorg20:31:590
2858426995244,5cyclictest2955385-21kworker/u8:1+flush-179:000:15:010
202199520,5rtkit-daemon1644344-21munin-node21:10:000
2858426995140,8cyclictest271-21systemd-journal22:55:010
2858430995045,3cyclictest2902101-21kworker/u8:3+flush-179:021:40:261
2858430994940,7cyclictest2950793-21kworker/u8:0+flush-179:000:30:281
2858430994940,7cyclictest2915050-21kworker/u8:1+flush-179:022:19:591
2858430994937,10cyclictest271-21systemd-journal22:13:381
2858430994937,10cyclictest271-21systemd-journal22:13:381
2858426994941,6cyclictest2902056-21latency_hist21:25:000
4670791480,13ptp4l61690irq/136-eno1-TxRx-019:09:563
2858430994834,10cyclictest2867839-21kworker/u8:2+flush-179:019:40:001
2858436994745,1cyclictest0-21swapper/323:15:003
2858430994739,6cyclictest2915050-21kworker/u8:1+events_unbound22:35:011
2858430994730,15cyclictest2913694-21kworker/u8:0+flush-179:022:04:591
2858426994742,3cyclictest0-21swapper/020:05:120
2858426994737,7cyclictest2911850-21latency_hist21:55:010
2858430994639,5cyclictest2934378-21kworker/u8:2+flush-179:023:15:261
2858430994639,5cyclictest2875843-21kworker/u8:3+flush-179:020:20:001
2858426994637,7cyclictest2859896-21latency_hist19:15:010
2858430994539,4cyclictest2892016-21kworker/u8:2+flush-179:021:30:001
2858430994528,2cyclictest2905484-21apt-get21:35:151
2858430994528,15cyclictest2954063-21apt-get00:05:151
2858430994523,4cyclictest2870800-21timerandwakeup19:45:231
285842699451,42cyclictest2884039-21awk20:30:000
2858430994432,9cyclictest2945550-21apt-get23:40:011
285843099441,4cyclictest271-21systemd-journal19:55:001
285842699441,41cyclictest2887217-21perf20:40:010
285843599430,41cyclictest0-21swapper/200:26:002
2858426994339,2cyclictest2916644-21ls22:10:010
2858426994331,9cyclictest2960179-21apt-get00:25:000
2858430994238,2cyclictest2874271-21systemd-run20:00:011
2858430994238,2cyclictest2874271-21systemd-run20:00:001
2858430994234,5cyclictest2884031-21latency_hist20:30:001
2858430994233,6cyclictest2952173-21latency_hist00:00:011
2858430994227,11cyclictest271-21systemd-journal22:25:001
285843099421,39cyclictest2950569-21perf23:55:001
285843099421,37cyclictest2874490-21apt-get20:00:141
285842699420,40cyclictest723-21Xorg23:47:000
285842699420,40cyclictest723-21Xorg23:46:590
2858430994130,7cyclictest2902101-21kworker/u8:3+flush-179:022:00:001
285843099410,39cyclictest723-21Xorg23:31:001
2858426994138,2cyclictest723-21Xorg20:47:590
2858426994137,3cyclictest2900479-21sendmail21:20:000
2858426994133,6cyclictest2934378-21kworker/u8:2+flush-179:023:15:250
285842699411,3cyclictest271-21systemd-journal22:50:000
2858430994036,3cyclictest2864136-21kworker/u8:0+flush-179:019:30:261
285843099400,3cyclictest2931213-21apt22:54:591
285843099400,38cyclictest723-21Xorg19:40:591
285843099400,38cyclictest2935984-21munin-run23:09:591
2858426994035,3cyclictest723-21Xorg21:26:590
2858426994030,7cyclictest2928033-21latency_hist22:45:010
285842699401,38cyclictest2945547-21idleruntime-cro23:40:000
2858436993937,1cyclictest0-21swapper/323:36:583
285843599390,37cyclictest0-21swapper/222:33:002
2858430993932,5cyclictest2921454-21kworker/u8:2+flush-179:022:35:261
2858430993932,5cyclictest2921454-21kworker/u8:2+flush-179:022:35:251
285843099390,38cyclictest0-21swapper/119:21:001
2858426993937,1cyclictest0-21swapper/020:54:550
2858426993933,4cyclictest2948979-21turbostat23:54:540
2858426993930,7cyclictest2952161-40mandb00:00:010
285842699391,36cyclictest2879507-21apt-get20:15:140
285842699390,2cyclictest723-21Xorg00:33:590
285843599380,36cyclictest0-21swapper/220:44:552
285843599380,36cyclictest0-21swapper/200:32:592
285843599380,35cyclictest0-21swapper/220:29:562
2858430993829,6cyclictest2934378-21kworker/u8:2+flush-179:023:10:161
2858426993833,4cyclictest723-21Xorg19:47:000
2858426993830,6cyclictest2931439-21/usr/sbin/munin22:55:170
2858426993827,9cyclictest2875830-21apt-get20:05:010
285842699380,36cyclictest723-21Xorg20:12:590
2858436993731,5cyclictest2908437-21turbostat21:45:003
285843699370,36cyclictest0-21swapper/323:54:583
285843699370,2cyclictest0-21swapper/320:16:593
2858435993734,2cyclictest0-21swapper/220:55:592
2858435993734,2cyclictest0-21swapper/200:07:592
285843599370,1cyclictest0-21swapper/200:38:582
2858430993734,2cyclictest0-21swapper/120:35:591
2858430993728,6cyclictest2875843-21kworker/u8:3+flush-179:020:40:221
2858426993734,2cyclictest723-21Xorg23:07:590
2858426993727,9cyclictest2915047-21munin-run22:05:000
2858436993634,1cyclictest0-21swapper/322:42:593
285843699360,35cyclictest0-21swapper/321:59:593
285843699360,35cyclictest0-21swapper/319:25:593
2858435993633,2cyclictest0-21swapper/223:39:552
2858435993633,2cyclictest0-21swapper/223:34:552
285843599360,1cyclictest0-21swapper/219:38:002
285843099363,29cyclictest2903899-21apt-get21:30:131
285843099363,11cyclictest2005-21lightdm-gtk-gre23:40:591
2858426993632,2cyclictest0-21swapper/020:40:150
2858426993628,6cyclictest2943935-21latency_hist23:35:000
2858426993626,8cyclictest2860150-21apt-get19:15:150
2858436993533,1cyclictest0-21swapper/322:36:593
2858436993533,1cyclictest0-21swapper/322:36:583
2858436993532,2cyclictest0-21swapper/320:29:593
285843699350,1cyclictest0-21swapper/319:46:003
2858435993532,2cyclictest0-21swapper/221:23:592
2858426993529,5cyclictest2875799-21kworker/u8:2+events_unbound20:25:010
2858426993526,7cyclictest2947386-21latency_hist23:45:020
2858426993511,9cyclictest475-21dbus-daemon23:05:000
2858436993431,2cyclictest0-21swapper/323:34:583
2858436993431,2cyclictest0-21swapper/319:59:593
2858436993431,2cyclictest0-21swapper/319:59:593
2858435993430,2cyclictest0-21swapper/200:18:592
2858426993430,3cyclictest0-21swapper/019:40:130
2858426993425,7cyclictest271-21systemd-journal23:15:000
4670791330,6ptp4l61690irq/136-eno1-TxRx-000:00:013
2858435993331,1cyclictest0-21swapper/220:49:592
2858435993330,2cyclictest0-21swapper/219:26:582
2858430993327,4cyclictest2126-10pipewire-media-23:00:011
285842699337,11cyclictest723-21Xorg21:55:590
2858426993328,3cyclictest0-21swapper/019:40:000
285842699332,3cyclictest2921664-21apt-get22:25:170
285842699332,3cyclictest2871299-21apt-get19:50:170
2858426993321,10cyclictest2923510-21apt-get22:30:140
285842699331,4cyclictest2895586-21cut21:05:010
4670791320,5ptp4l61690irq/136-eno1-TxRx-000:04:563
2858435993230,1cyclictest0-21swapper/223:20:012
2858435993229,2cyclictest0-21swapper/221:45:002
2858430993228,3cyclictest2915050-21kworker/u8:1+flush-179:022:40:231
2858426993230,1cyclictest723-21Xorg19:55:580
2858426993230,1cyclictest723-21Xorg19:55:580
2858426993227,3cyclictest2918470-21apt-get22:15:160
285842699321,29cyclictest0-21swapper/021:30:130
4670791310,5ptp4l61690irq/136-eno1-TxRx-022:50:003
2858436993129,1cyclictest0-21swapper/322:59:583
285843699310,30cyclictest0-21swapper/322:50:593
2858430993129,1cyclictest2858424-21cyclictest19:10:141
2858430993124,5cyclictest2961279-21latency00:25:201
2858436993028,1cyclictest0-21swapper/321:25:163
285843699300,28cyclictest0-21swapper/320:45:143
285843699300,28cyclictest0-21swapper/320:05:013
285843699300,27cyclictest0-21swapper/321:15:123
2858435993027,2cyclictest0-21swapper/222:10:122
2858435993027,2cyclictest0-21swapper/222:10:112
2858435993026,2cyclictest0-21swapper/220:20:152
285843599300,2cyclictest0-21swapper/223:40:022
285843599300,29cyclictest0-21swapper/222:55:592
285843599300,29cyclictest0-21swapper/221:05:032
2858426993026,2cyclictest723-21Xorg23:24:590
2858436992927,1cyclictest0-21swapper/322:30:023
2858436992926,2cyclictest0-21swapper/320:55:163
2858436992926,2cyclictest0-21swapper/319:50:173
2858436992924,4cyclictest2903658-21turbostat21:34:553
285843699290,28cyclictest0-21swapper/322:05:143
285843699290,28cyclictest0-21swapper/319:30:173
285843699290,27cyclictest0-21swapper/321:50:003
285843699290,1cyclictest0-21swapper/323:05:153
2858435992927,1cyclictest0-21swapper/222:05:002
2858435992926,2cyclictest0-21swapper/222:35:172
2858435992926,2cyclictest0-21swapper/222:35:172
2858435992926,2cyclictest0-21swapper/219:55:172
2858435992926,2cyclictest0-21swapper/219:55:162
2858435992926,2cyclictest0-21swapper/219:32:002
2858435992926,2cyclictest0-21swapper/219:10:152
285843599290,27cyclictest0-21swapper/221:50:152
285843599290,27cyclictest0-21swapper/221:10:022
285843599290,27cyclictest0-21swapper/220:10:132
2858430992924,3cyclictest2897509-21gdbus21:10:011
285842699297,9cyclictest2918059-21unixbench-2d22:10:240
285842699297,9cyclictest2918059-21unixbench-2d22:10:240
2858426992921,6cyclictest2958560-21cron00:20:000
4670791280,7ptp4l441ktimers/321:20:013
2858436992826,1cyclictest0-21swapper/323:20:143
2858436992825,2cyclictest0-21swapper/321:10:123
285843699280,2cyclictest0-21swapper/320:05:023
285843699280,27cyclictest0-21swapper/322:25:173
285843699280,27cyclictest0-21swapper/322:20:153
285843699280,26cyclictest0-21swapper/300:30:143
285843699280,21cyclictest0-21swapper/300:25:023
2858435992826,1cyclictest0-21swapper/200:00:002
2858435992825,2cyclictest0-21swapper/222:48:002
2858435992825,2cyclictest0-21swapper/221:18:592
2858435992825,2cyclictest0-21swapper/219:20:132
2858435992825,2cyclictest0-21swapper/200:25:022
285843599280,2cyclictest0-21swapper/221:35:142
285843599280,26cyclictest0-21swapper/222:07:592
285842699286,7cyclictest2858543-21cut19:10:020
285842699285,9cyclictest2941119-21cat23:25:170
2858426992825,2cyclictest723-21Xorg21:37:590
4670791270,6ptp4l61690irq/136-eno1-TxRx-019:25:003
4670791270,5ptp4l441ktimers/323:50:003
4670791270,5ptp4l441ktimers/323:49:593
285843699270,26cyclictest0-21swapper/320:34:593
285843699270,26cyclictest0-21swapper/300:20:153
2858435992724,2cyclictest0-21swapper/219:44:552
2858430992720,4cyclictest0-21swapper/121:55:011
2858430992719,6cyclictest271-21systemd-journal23:45:381
2858430992719,6cyclictest271-21systemd-journal23:45:371
4670791260,6ptp4l61690irq/136-eno1-TxRx-023:30:003
4670791260,6ptp4l441ktimers/323:05:003
4670791260,6ptp4l441ktimers/322:05:013
4670791260,6ptp4l441ktimers/321:04:553
4670791260,6ptp4l441ktimers/320:45:003
4670791260,6ptp4l441ktimers/320:15:003
4670791260,6ptp4l441ktimers/319:45:003
4670791260,6ptp4l441ktimers/319:14:563
4670791260,6ptp4l441ktimers/300:10:013
4670791260,5ptp4l441ktimers/300:40:013
4670791260,4ptp4l441ktimers/319:40:003
2858436992624,1cyclictest0-21swapper/321:45:173
2858436992623,2cyclictest0-21swapper/320:35:133
2858435992623,2cyclictest0-21swapper/220:06:002
285843599260,2cyclictest0-21swapper/223:02:592
285843599260,24cyclictest0-21swapper/219:15:142
285843599260,1cyclictest0-21swapper/221:35:002
2858430992621,3cyclictest0-21swapper/122:50:011
285842699265,7cyclictest2920083-21cpu22:20:150
285842699263,9cyclictest2863320-21latency_hist19:25:010
2858426992620,5cyclictest723-21Xorg19:27:590
4670791250,5ptp4l61690irq/136-eno1-TxRx-023:20:003
4670791250,5ptp4l61690irq/136-eno1-TxRx-000:14:553
2858436992523,1cyclictest0-21swapper/323:40:023
2858436992519,4cyclictest2895185-21turbostat21:09:553
285843699250,24cyclictest0-21swapper/320:54:593
285843699250,1cyclictest0-21swapper/322:10:123
285843699250,1cyclictest0-21swapper/322:10:123
2858435992523,1cyclictest0-21swapper/200:14:592
2858435992522,2cyclictest0-21swapper/223:53:582
2858435992522,2cyclictest0-21swapper/222:28:592
2858435992522,2cyclictest0-21swapper/221:30:002
285843599250,1cyclictest0-21swapper/222:25:002
4670791240,4ptp4l441ktimers/322:19:553
4670791240,4ptp4l441ktimers/300:20:013
2858435992421,2cyclictest0-21swapper/223:45:152
2858435992421,2cyclictest0-21swapper/223:45:152
2858435992421,2cyclictest0-21swapper/223:25:592
2858435992421,2cyclictest0-21swapper/222:17:592
285843099244,9cyclictest723-21Xorg00:35:581
4670791230,4ptp4l441ktimers/321:39:563
2858435992321,1cyclictest0-21swapper/223:25:002
2858435992321,1cyclictest0-21swapper/222:53:592
2858435992321,1cyclictest0-21swapper/220:36:592
2858435992320,2cyclictest0-21swapper/220:00:592
2858435992219,2cyclictest0-21swapper/220:50:142
2858435992219,2cyclictest0-21swapper/220:19:552
2858430992216,4cyclictest2858513-21latency_hist19:10:011
2858435991916,2cyclictest0-21swapper/222:43:582
2858435991916,2cyclictest0-21swapper/221:01:592
2858435991715,1cyclictest0-21swapper/220:33:592
2858435991513,1cyclictest0-21swapper/223:07:592
2858435991412,1cyclictest0-21swapper/219:46:592
2858435991411,2cyclictest0-21swapper/221:59:542
285843599107,2cyclictest0-21swapper/219:54:462
28584359980,2cyclictest0-21swapper/221:45:472
28584359970,1cyclictest0-21swapper/223:10:322
28584359930,2cyclictest0-21swapper/219:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional