You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-16 - 09:51
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot6.osadl.org (updated Thu Oct 16, 2025 00:44:58)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
278631699136116,14cyclictest2799368-21latency_hist19:50:010
278631999128112,14cyclictest2795918-21latency_hist19:40:011
27863199910897,8cyclictest2892374-21latency_hist00:40:011
27863199910796,8cyclictest2839949-21kworker/u8:1+flush-179:022:40:211
27863169910796,7cyclictest601706-21systemd-journal00:04:030
27863199910696,7cyclictest2851394-21latency_hist22:35:001
27863199910695,9cyclictest2792992-21apt-get19:30:131
27863169910694,9cyclictest2867854-21latency_hist23:25:010
27863169910292,8cyclictest2858020-21apt-get22:55:100
27863169910088,9cyclictest2809178-21latency_hist20:20:010
2786316999988,9cyclictest2871286-21apt-get23:35:090
2786319999888,7cyclictest2786134-21kworker/u8:2+flush-179:019:10:221
2786319999888,7cyclictest2786134-21kworker/u8:2+flush-179:019:10:221
2786319999786,9cyclictest2788206-21apt-get19:15:131
2786316999583,10cyclictest2861476-21apt-get23:05:120
2786319999284,6cyclictest2810793-21apt-get20:25:011
2786316999180,9cyclictest2816088-21apt-get20:40:140
2786316999180,9cyclictest2816088-21apt-get20:40:130
2786316998980,7cyclictest2850060-21apt-get22:30:120
2786319998882,4cyclictest2819038-21kworker/u8:1+flush-179:021:15:231
2786316998678,5cyclictest2805462-21kworker/u8:0+flush-179:020:55:260
2786319998574,9cyclictest601706-21systemd-journal00:00:011
2786316998577,6cyclictest2837052-21apt-get21:50:110
2786316998574,9cyclictest2843428-21apt-get22:10:160
2786316998572,11cyclictest2851649-21apt-get22:35:120
2786319998477,5cyclictest2877376-21kworker/u8:3+flush-179:023:55:001
2786319998476,5cyclictest2871063-21kworker/u8:0+flush-179:023:45:211
2786319998372,8cyclictest2802803-21latency_hist20:00:011
2786319998366,4cyclictest2796078-21kworker/u8:3+events_unbound19:45:231
2786319998173,5cyclictest2864439-21kworker/u8:3+flush-179:023:30:211
2786319998165,14cyclictest601706-21systemd-journal19:26:521
2786316998175,4cyclictest2786064-21kworker/u8:1+flush-179:019:25:240
2786319998069,8cyclictest2827295-21kworker/u8:0+flush-179:021:55:141
2786316997971,5cyclictest1-21systemd23:45:000
2786319997869,7cyclictest2805462-21kworker/u8:0+flush-179:020:10:231
2786319997867,9cyclictest2807842-21apt-get20:15:131
2786319997771,4cyclictest2830462-21kworker/u8:3+flush-179:021:30:191
2786319997771,4cyclictest2830462-21kworker/u8:3+flush-179:021:30:181
2786319997770,5cyclictest2851371-21kworker/u8:0+events_unbound22:50:211
2786319997762,12cyclictest2879310-21kworker/u8:0+flush-179:000:10:231
2786319997750,8cyclictest502-21rs:main1
2786316997766,8cyclictest2812888-21apt-get20:30:130
2786319997568,5cyclictest2805462-21kworker/u8:0+flush-179:020:35:131
2786319997554,3cyclictest601706-21systemd-journal21:36:221
2786316997567,5cyclictest1-21systemd21:57:220
2786319997468,4cyclictest2871063-21kworker/u8:0+flush-179:023:40:171
2786319997464,8cyclictest2890536-21apt-get00:35:011
2786319997367,4cyclictest2882588-21kworker/u8:3+flush-179:000:15:161
2786319997366,5cyclictest2827295-21kworker/u8:0+flush-179:021:40:171
2786319997363,8cyclictest502-21rs:main1
2786319997361,9cyclictest502-21rs:main1
2786319997361,9cyclictest502-21rs:main1
2786316997361,9cyclictest601706-21systemd-journal23:10:220
2786316997361,9cyclictest2805998-21latency_hist20:10:010
2786316997257,11cyclictest2810784-21latency_hist20:25:010
2786319997163,5cyclictest2815875-21kworker/u8:3+flush-179:020:55:151
2786319997151,18cyclictest2805462-21kworker/u8:0+flush-179:020:35:001
2786316997161,7cyclictest2819003-21latency_hist20:50:010
2786319997058,10cyclictest2859889-21apt-get23:00:131
2786316996961,6cyclictest2824086-21kworker/u8:2+flush-179:021:20:210
2786316996958,9cyclictest2814476-21apt-get20:35:090
2786316996957,8cyclictest2796078-21kworker/u8:3+flush-179:019:50:260
2786319996860,6cyclictest2819038-21kworker/u8:1+flush-179:021:25:251
2786319996859,6cyclictest2799433-21kworker/u8:2+events_unbound20:00:191
2786319996857,8cyclictest601706-21systemd-journal23:15:001
2786319996759,5cyclictest2843150-21kworker/u8:0+events_unbound22:10:251
2786319996757,7cyclictest2839949-21kworker/u8:1+flush-179:022:45:221
2786319996757,7cyclictest2839949-21kworker/u8:1+flush-179:022:45:211
2786316996759,6cyclictest601706-21systemd-journal21:17:140
2786320996663,2cyclictest0-21swapper/223:06:592
2786319996661,3cyclictest2805462-21kworker/u8:0+events_unbound20:45:231
2786319996654,8cyclictest2856157-21kworker/u8:2+events_unbound22:55:241
2786316996657,7cyclictest2877408-21apt-get23:55:000
278631699661,3cyclictest191rcuc/022:53:000
2786319996553,10cyclictest2864651-21apt-get23:15:111
2786316996553,10cyclictest2803079-21apt-get20:00:140
2786316996552,10cyclictest2786078-21latency_hist19:10:020
2786320996461,2cyclictest0-21swapper/219:55:582
2786319996450,11cyclictest2845012-21apt-get22:15:121
2786316996353,8cyclictest2787950-21latency_hist19:15:010
2786316996353,8cyclictest2787950-21latency_hist19:15:010
2786319996255,5cyclictest2864439-21kworker/u8:3+flush-179:023:25:231
2786319996255,4cyclictest2819038-21kworker/u8:1+flush-179:021:10:251
2786316996251,9cyclictest2846624-21apt-get22:20:140
2786316996251,9cyclictest2846624-21apt-get22:20:140
2786316996251,9cyclictest2829093-21apt-get21:25:120
2786322996157,2cyclictest0-21swapper/300:01:003
2786319996153,6cyclictest2839949-21kworker/u8:1+flush-179:022:25:201
2786319996151,7cyclictest2784564-21kworker/u8:0+flush-179:019:20:211
2786322996053,5cyclictest2847977-21turbostat22:25:003
2786322996053,5cyclictest2847977-21turbostat22:25:003
2786319996049,8cyclictest2835465-21apt-get21:45:111
2786316996048,10cyclictest2866011-21latency_hist23:20:010
2786320995956,2cyclictest0-21swapper/223:01:592
2786319995953,4cyclictest2805462-21kworker/u8:0+flush-179:020:50:251
2786319995947,9cyclictest2879544-21apt-get00:00:151
2786316995951,6cyclictest2885791-21kworker/u8:1+flush-179:000:25:220
2786316995949,8cyclictest2840250-21apt-get22:00:120
2786319995851,5cyclictest2833660-21kworker/u8:3+events_unbound22:00:231
2786316995851,5cyclictest2885791-21kworker/u8:1+events_unbound00:35:230
2786316995749,6cyclictest2822720-21apt-get21:05:090
2786316995749,6cyclictest2822720-21apt-get21:05:080
2786316995749,5cyclictest2885791-21kworker/u8:1+flush-179:000:30:170
2786316995748,6cyclictest2811299-21apt-get20:25:090
2786316995646,7cyclictest2784564-21kworker/u8:0+flush-179:019:35:220
2786320995552,2cyclictest0-21swapper/222:57:002
2786319995548,5cyclictest2879310-21kworker/u8:0+flush-179:000:05:211
2786319995546,6cyclictest2862808-21latency_hist23:10:021
2786319995545,7cyclictest2871063-21kworker/u8:0+flush-179:023:35:201
2786319995448,5cyclictest2839949-21kworker/u8:1+events_unbound22:20:241
2786319995448,5cyclictest2839949-21kworker/u8:1+events_unbound22:20:241
2786319995443,9cyclictest2866807-21diskstats23:20:181
2786316995448,4cyclictest2871038-21cut23:34:590
2786319995345,6cyclictest601706-21systemd-journal19:53:541
2786319995338,3cyclictest2805462-21kworker/u8:0+flush-179:020:05:241
2786316995344,6cyclictest2877376-21kworker/u8:3+events_unbound23:55:020
2786316995244,6cyclictest2849545-21latency_hist22:30:010
2786320995047,2cyclictest0-21swapper/220:46:002
2786319995041,7cyclictest601706-21systemd-journal22:08:211
2786316995042,6cyclictest2799433-21kworker/u8:2+flush-179:019:55:200
2786316994941,6cyclictest601706-21systemd-journal21:05:010
4505291480,13ptp4l61790irq/134-eno1-TxRx-019:10:013
2786316994836,9cyclictest2881220-21apt-get00:05:160
2786316994811,33cyclictest0-21swapper/019:30:150
2786322994745,1cyclictest0-21swapper/320:34:593
2786322994739,7cyclictest0-21swapper/323:19:593
2786319994739,6cyclictest2824086-21kworker/u8:2+flush-179:021:20:271
2786319994727,18cyclictest2885791-21kworker/u8:1+flush-179:000:25:001
2786316994637,6cyclictest2786064-21kworker/u8:1+events_unbound19:40:030
2786319994539,3cyclictest2822502-21systemd-run21:05:011
278631699451,42cyclictest2820599-21apt20:54:590
2786316994432,9cyclictest2843171-21latency_hist22:10:000
278632099430,41cyclictest0-21swapper/200:39:552
2786319994336,5cyclictest2885791-21kworker/u8:1+events_unbound00:25:171
278632299420,40cyclictest2822480-21turbostat21:05:013
278632299420,1cyclictest0-21swapper/322:53:003
2786319994237,3cyclictest2824091-21systemd-run21:10:011
2786319994237,3cyclictest2824091-21systemd-run21:10:001
2786316994232,7cyclictest601706-21systemd-journal00:18:420
2786316994213,11cyclictest2868128-21cpu23:25:160
2786322994134,5cyclictest2856192-21turbostat22:50:003
2786322994134,5cyclictest2856192-21turbostat22:50:003
278631699416,7cyclictest0-21swapper/021:40:140
2786322994038,1cyclictest0-21swapper/320:17:003
2786322994034,5cyclictest0-21swapper/322:17:593
2786316994010,15cyclictest0-21swapper/022:40:150
2786322993936,2cyclictest0-21swapper/319:32:593
278632299390,38cyclictest2879279-21turbostat00:00:003
2786319993931,6cyclictest2839949-21kworker/u8:1+flush-179:022:35:161
278631999390,2cyclictest0-21swapper/121:55:001
2786316993915,9cyclictest2806237-21/usr/sbin/munin20:10:250
2786316993914,11cyclictest2854926-21cpuspeed_turbos22:45:130
2786316993914,11cyclictest2854926-21cpuspeed_turbos22:45:130
2786322993826,10cyclictest2887356-21turbostat00:25:003
2786316993830,6cyclictest2887344-21latency_hist00:25:010
2786316993814,9cyclictest2830783-21grep21:30:170
2786316993814,9cyclictest2830783-21grep21:30:160
4505291370,3ptp4l441ktimers/320:25:023
2786322993734,2cyclictest0-21swapper/322:30:583
2786322993734,2cyclictest0-21swapper/320:24:553
2786322993734,2cyclictest0-21swapper/319:15:593
2786322993731,4cyclictest2824059-21turbostat21:10:013
2786322993731,4cyclictest2824059-21turbostat21:10:003
278632299370,36cyclictest0-21swapper/321:34:593
278632299370,36cyclictest0-21swapper/321:34:583
2786320993735,1cyclictest0-21swapper/222:29:582
278632099370,35cyclictest0-21swapper/222:19:592
278632099370,35cyclictest0-21swapper/220:38:592
2786316993712,11cyclictest2824506-21cstates21:10:150
278631699370,1cyclictest0-21swapper/022:15:590
2786322993634,1cyclictest0-21swapper/321:44:003
2786322993634,1cyclictest0-21swapper/319:56:593
2786322993633,2cyclictest0-21swapper/320:51:593
2786322993629,6cyclictest0-21swapper/321:59:563
278632299360,35cyclictest0-21swapper/322:01:593
2786320993633,2cyclictest0-21swapper/220:20:592
278632099360,2cyclictest0-21swapper/221:00:002
278632099360,1cyclictest0-21swapper/223:44:592
278632299350,2cyclictest0-21swapper/321:25:133
278632299350,1cyclictest0-21swapper/322:13:593
2786320993532,2cyclictest0-21swapper/222:52:592
2786320993532,2cyclictest0-21swapper/222:49:562
2786320993532,2cyclictest0-21swapper/222:49:552
2786320993532,2cyclictest0-21swapper/222:24:002
2786320993532,2cyclictest0-21swapper/222:23:592
2786320993532,2cyclictest0-21swapper/200:25:002
2786316993514,8cyclictest816013-21ntpd21:45:210
2786316993513,8cyclictest2883774-21grep00:10:250
2786320993431,2cyclictest0-21swapper/222:33:592
2786320993431,2cyclictest0-21swapper/221:24:592
2786316993423,9cyclictest2786064-21kworker/u8:1+flush-179:019:15:170
2786316993414,18cyclictest2790352-21/usr/sbin/munin19:20:180
2786322993330,2cyclictest0-21swapper/320:42:593
2786322993330,2cyclictest0-21swapper/320:42:583
278632299330,26cyclictest0-21swapper/322:35:133
2786320993330,2cyclictest0-21swapper/221:44:592
2786320993330,2cyclictest0-21swapper/200:13:592
2786316993313,8cyclictest2860186-21fschecks_count23:00:170
2786320993228,2cyclictest0-21swapper/220:09:552
2786316993226,4cyclictest2833617-21/usr/sbin/munin21:40:010
2786322993129,1cyclictest0-21swapper/300:25:593
2786322993128,2cyclictest0-21swapper/321:18:593
2786320993128,2cyclictest0-21swapper/223:20:022
2786320993128,2cyclictest0-21swapper/221:32:592
2786320993128,2cyclictest0-21swapper/221:32:592
2786320993128,2cyclictest0-21swapper/219:40:032
278632099310,2cyclictest0-21swapper/222:38:592
2786322993028,2cyclictest0-21swapper/300:12:593
2786322993028,1cyclictest0-21swapper/323:30:593
2786322993023,5cyclictest0-21swapper/319:40:163
278632299300,29cyclictest0-21swapper/323:00:153
278632299300,28cyclictest0-21swapper/320:05:143
2786320993027,2cyclictest0-21swapper/222:40:122
2786320993027,2cyclictest0-21swapper/221:12:592
2786320993027,2cyclictest0-21swapper/219:20:132
278632099300,28cyclictest0-21swapper/221:52:582
2786322992927,1cyclictest0-21swapper/321:50:133
2786322992927,1cyclictest0-21swapper/320:10:143
2786322992926,2cyclictest0-21swapper/323:20:013
278632299290,28cyclictest0-21swapper/320:45:123
278632299290,27cyclictest0-21swapper/300:35:123
2786320992927,1cyclictest0-21swapper/219:45:132
2786320992926,2cyclictest0-21swapper/221:45:142
2786320992926,2cyclictest0-21swapper/221:25:122
2786320992926,2cyclictest0-21swapper/221:08:592
2786320992926,2cyclictest0-21swapper/221:08:592
2786320992926,2cyclictest0-21swapper/219:30:152
2786320992926,2cyclictest0-21swapper/200:05:182
278632099290,27cyclictest0-21swapper/223:37:592
278632099290,27cyclictest0-21swapper/222:05:142
278632099290,27cyclictest0-21swapper/220:30:132
278632099290,27cyclictest0-21swapper/220:00:162
278632099290,27cyclictest0-21swapper/219:50:022
4505291280,7ptp4l61790irq/134-eno1-TxRx-019:15:013
4505291280,7ptp4l61790irq/134-eno1-TxRx-019:15:013
4505291280,6ptp4l441ktimers/321:00:013
2786322992825,2cyclictest0-21swapper/319:25:143
2786322992820,6cyclictest0-21swapper/323:35:593
278632299280,26cyclictest0-21swapper/323:10:153
278632299280,26cyclictest0-21swapper/322:55:123
278632299280,26cyclictest0-21swapper/322:25:123
278632299280,26cyclictest0-21swapper/319:50:023
2786320992825,2cyclictest0-21swapper/223:20:012
2786320992825,2cyclictest0-21swapper/222:10:162
2786320992825,2cyclictest0-21swapper/221:35:582
2786320992825,2cyclictest0-21swapper/219:15:142
2786320992825,2cyclictest0-21swapper/200:32:592
278632099280,2cyclictest0-21swapper/220:25:022
278632099280,26cyclictest0-21swapper/223:55:022
4505291270,7ptp4l441ktimers/319:50:003
4505291270,6ptp4l61790irq/134-eno1-TxRx-023:30:013
4505291270,6ptp4l441ktimers/323:55:003
4505291270,6ptp4l441ktimers/323:10:003
4505291270,6ptp4l441ktimers/321:25:003
4505291270,6ptp4l441ktimers/320:05:003
4505291270,6ptp4l441ktimers/300:20:003
4505291270,6ptp4l441ktimers/300:10:013
2786322992725,1cyclictest0-21swapper/321:49:583
278632299270,1cyclictest0-21swapper/319:35:133
278632099270,25cyclictest0-21swapper/220:52:592
278632099270,24cyclictest0-21swapper/223:29:582
2786316992724,2cyclictest722-21Xorg23:49:590
4505291260,6ptp4l441ktimers/323:45:003
4505291260,6ptp4l441ktimers/319:25:003
2786320992623,2cyclictest0-21swapper/219:10:232
2786320992623,2cyclictest0-21swapper/219:10:222
2786320992623,2cyclictest0-21swapper/200:16:592
2786320992623,2cyclictest0-21swapper/200:00:162
278632099260,2cyclictest0-21swapper/223:10:592
4505291250,6ptp4l61790irq/134-eno1-TxRx-021:40:003
4505291250,5ptp4l61790irq/134-eno1-TxRx-021:14:563
2786322992523,1cyclictest0-21swapper/322:09:543
278632299250,24cyclictest0-21swapper/323:46:593
2786320992523,1cyclictest0-21swapper/222:03:592
2786320992522,2cyclictest0-21swapper/200:27:592
4505291240,4ptp4l441ktimers/320:39:553
4505291240,4ptp4l441ktimers/300:35:013
2786320992422,1cyclictest0-21swapper/219:39:592
4505291230,4ptp4l441ktimers/322:44:563
2786320992321,1cyclictest0-21swapper/219:28:592
2786320992320,2cyclictest0-21swapper/220:14:552
2786319992318,3cyclictest2786078-21latency_hist19:10:021
2786320992219,2cyclictest0-21swapper/223:52:592
2786320992017,2cyclictest0-21swapper/221:16:592
2786320991815,2cyclictest0-21swapper/223:49:592
2786320991815,2cyclictest0-21swapper/220:40:162
2786320991815,2cyclictest0-21swapper/220:40:152
2786320991411,2cyclictest0-21swapper/220:16:002
2786320991210,1cyclictest0-21swapper/223:35:002
278632099119,1cyclictest0-21swapper/221:59:582
278632099107,2cyclictest0-21swapper/221:04:582
27863209930,2cyclictest0-21swapper/219:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional