You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-16 - 15:22
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot6.osadl.org (updated Mon Jun 16, 2025 00:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
274527699140129,8cyclictest2758281-21gpgv19:50:011
274527699137124,10cyclictest2798230-21apt-get21:55:001
274527699122102,6cyclictest2796610-21kworker/u8:0+events_unbound21:55:141
274527699119111,5cyclictest2772728-21kworker/u8:4+events_unbound20:35:161
274527699119110,7cyclictest2838322-40mandb00:00:021
274527499119101,15cyclictest2758157-21latency_hist19:50:020
274527699117109,6cyclictest2809665-21kworker/u8:3+events_unbound23:05:181
274527699116107,6cyclictest2822401-21kworker/u8:2+events_unbound23:15:181
274527699113105,5cyclictest2822401-21kworker/u8:2+events_unbound23:20:231
27452749911298,11cyclictest2776194-21apt-get20:45:180
274527699109101,5cyclictest2782342-21kworker/u8:2+events_unbound22:10:231
27452769910896,9cyclictest2844969-21latency_hist00:20:001
274527699108100,5cyclictest2806414-21kworker/u8:2+flush-179:022:20:211
27452749910693,10cyclictest2775940-21latency_hist20:45:000
27452769910393,8cyclictest2836687-21latency_hist23:55:011
27452749910389,11cyclictest2777781-21apt-get20:50:140
27452769910295,5cyclictest2753437-21kworker/u8:2+events_unbound20:10:211
2745276999883,12cyclictest2796610-21kworker/u8:0+events_unbound22:00:221
2745276999788,7cyclictest2846803-21kworker/u8:0+events_unbound00:30:191
2745276999690,4cyclictest2753437-21kworker/u8:2+events_unbound19:50:021
2745276999688,6cyclictest2772728-21kworker/u8:4+events_unbound20:50:211
2745276999684,9cyclictest2745417-21apt-get19:10:131
2745274999680,11cyclictest502-21rs:main0
2745276999586,7cyclictest2838266-21kworker/u8:3+flush-179:000:05:171
2745276999387,5cyclictest2772728-21kworker/u8:4+flush-179:020:40:261
2745274999382,8cyclictest2779350-21/usr/sbin/munin20:55:290
2745276999278,11cyclictest2805100-21apt-get22:15:091
2745274999276,12cyclictest3105085-21systemd-journal23:49:590
2745276999183,5cyclictest2814389-21kworker/u8:0+events_unbound22:50:301
2745276998982,5cyclictest2804825-21kworker/u8:1+flush-179:022:35:201
2745276998976,10cyclictest2804825-21kworker/u8:1+flush-179:022:35:011
2745274998981,5cyclictest1-21systemd00:04:000
2745276998778,7cyclictest2788677-21latency_hist21:25:011
2745276998680,4cyclictest2753437-21kworker/u8:2+events_unbound20:20:141
2745276998677,5cyclictest2841804-21kworker/u8:1+events_unbound00:10:211
2745276998672,12cyclictest2814667-21apt-get22:45:141
2745276998579,4cyclictest2830357-21kworker/u8:0+events_unbound23:40:171
2745276998575,7cyclictest2748374-21latency_hist19:20:001
2745274998576,7cyclictest2830357-21kworker/u8:0+flush-179:000:00:010
2745274998576,6cyclictest2830357-21kworker/u8:0+events_unbound23:50:260
2745274998272,8cyclictest2763199-21latency_hist20:05:010
2745274998272,7cyclictest2769569-21latency_hist20:25:010
2745276998175,4cyclictest2777504-21kworker/u8:1+flush-179:021:00:231
2745276998175,4cyclictest2777504-21kworker/u8:1+flush-179:021:00:231
2745276998174,5cyclictest2845015-21kworker/u8:2+events_unbound00:20:171
2745276998171,7cyclictest2754977-21latency_hist19:40:011
2745274998160,15cyclictest2796610-21kworker/u8:0+flush-179:022:00:140
2745276998074,5cyclictest2788688-21kworker/u8:0+flush-179:021:30:381
2745276998065,11cyclictest2804825-21kworker/u8:1+flush-179:022:40:211
2745276997973,4cyclictest2782342-21kworker/u8:2+flush-179:022:05:211
2745276997971,6cyclictest2753437-21kworker/u8:2+flush-179:020:30:161
2745276997966,4cyclictest2782342-21kworker/u8:2+events_unbound21:15:241
2745274997970,7cyclictest3105085-21systemd-journal19:43:360
2745274997970,5cyclictest2814389-21kworker/u8:0+flush-179:023:00:180
2745276997867,9cyclictest2838253-21kworker/u8:2+flush-179:000:04:001
2745274997852,23cyclictest2753126-21apt-get19:35:000
2745276997772,3cyclictest2753437-21kworker/u8:2+events_unbound20:00:211
2745276997770,5cyclictest2769617-21kworker/u8:1+flush-179:020:25:251
2745276997767,7cyclictest2808297-21apt-get22:25:121
2745276997670,4cyclictest2777504-21kworker/u8:1+events_unbound21:10:231
2745276997665,8cyclictest2845015-21kworker/u8:2+flush-179:000:30:011
2745274997667,7cyclictest2840455-21apt-get00:05:140
2745274997667,6cyclictest2821652-21kworker/u8:1+flush-179:023:25:230
2745274997666,7cyclictest2766365-21kworker/u8:3+flush-179:020:15:220
2745274997662,10cyclictest2814389-21kworker/u8:0+flush-179:023:10:240
2745276997569,4cyclictest2822401-21kworker/u8:2+events_unbound23:30:301
2745276997568,5cyclictest2726135-21kworker/u8:3+flush-179:019:30:221
2745276997564,8cyclictest3105085-21systemd-journal23:29:591
2745274997565,8cyclictest2798343-21gpgv21:55:000
2745274997466,6cyclictest2813062-21/usr/sbin/munin22:40:230
2745274997465,6cyclictest2804825-21kworker/u8:1+flush-179:022:35:160
2745274997461,10cyclictest2777504-21kworker/u8:1+flush-179:021:00:240
2745274997461,10cyclictest2777504-21kworker/u8:1+flush-179:021:00:240
2745274997264,6cyclictest2779349-21kworker/u8:0+events_unbound21:10:000
2745276997165,4cyclictest2777504-21kworker/u8:1+events_unbound21:40:151
2745276997162,3cyclictest2779349-21kworker/u8:0+events_unbound21:05:221
2745276997064,4cyclictest2791842-21kworker/u8:3+flush-179:021:35:151
2745274997048,18cyclictest3105085-21systemd-journal22:07:110
2745276996959,8cyclictest2777504-21kworker/u8:1+flush-179:021:25:201
2745274996859,6cyclictest2753437-21kworker/u8:2+events_unbound19:55:150
2745276996763,3cyclictest2814389-21kworker/u8:0+events_unbound22:55:291
2745274996761,4cyclictest2743681-21kworker/u8:1+flush-179:019:39:590
2745276996660,4cyclictest2821652-21kworker/u8:1+flush-179:023:35:251
2745276996650,6cyclictest2814389-21kworker/u8:0+events_unbound23:00:191
2745274996660,4cyclictest2814389-21kworker/u8:0+flush-179:022:50:300
2745274996658,5cyclictest2743614-21kworker/u8:0+events_unbound19:15:150
2745276996560,4cyclictest2850003-21kworker/u8:1+events_unbound00:35:301
2745274996558,6cyclictest2788688-21kworker/u8:0+flush-179:021:30:260
2745274996558,5cyclictest2850003-21kworker/u8:1+events_unbound00:35:200
2745276996459,4cyclictest2777504-21kworker/u8:1+events_unbound20:46:401
2745280996355,5cyclictest2764799-21turbostat20:10:003
2745279996360,2cyclictest0-21swapper/223:46:592
2745276996358,4cyclictest2814389-21kworker/u8:0+events_unbound23:10:371
2745276996358,4cyclictest2777504-21kworker/u8:1+flush-179:020:55:271
2745276996358,4cyclictest2761566-21kworker/u8:0+events_unbound20:09:371
2745274996355,6cyclictest2769617-21kworker/u8:1+flush-179:020:25:180
2745274996351,9cyclictest2777504-21kworker/u8:1+flush-179:021:40:010
2745274996348,8cyclictest2804825-21kworker/u8:1+events_unbound22:30:230
2745276996255,5cyclictest2753437-21kworker/u8:2+events_unbound20:18:381
2745276996253,7cyclictest2753437-21kworker/u8:2+events_unbound19:55:221
2745276996155,4cyclictest2782342-21kworker/u8:2+events_unbound21:45:151
2745274996153,6cyclictest2846803-21kworker/u8:0+events_unbound00:30:140
2745274996153,6cyclictest2822401-21kworker/u8:2+events_unbound23:30:250
2745274996150,9cyclictest3105085-21systemd-journal00:30:000
2745279995956,2cyclictest0-21swapper/221:40:592
274527699590,57cyclictest1-21systemd23:50:001
2745276995853,4cyclictest2743681-21kworker/u8:1+flush-179:019:25:271
2745276995846,4cyclictest2743681-21kworker/u8:1+events_unbound19:20:251
2745274995852,4cyclictest2777504-21kworker/u8:1+flush-179:021:40:170
2745274995749,6cyclictest2806414-21kworker/u8:2+flush-179:022:25:200
2745274995649,5cyclictest2845015-21kworker/u8:2+events_unbound00:20:190
2745274995647,6cyclictest2827183-21kworker/u8:3+events_unbound23:40:200
2745274995549,4cyclictest2753437-21kworker/u8:2+flush-179:020:32:040
2745274995446,5cyclictest2782342-21kworker/u8:2+events_unbound22:10:170
2745274995442,8cyclictest2841804-21kworker/u8:1+flush-179:000:10:260
274527499540,38cyclictest3105085-21systemd-journal21:50:000
2745274995346,5cyclictest2841804-21kworker/u8:1+flush-179:000:19:050
2745274995343,8cyclictest2782342-21kworker/u8:2+flush-179:021:20:000
2745274995343,8cyclictest2760002-21latency_hist19:55:010
2745276995242,4cyclictest2743681-21kworker/u8:1+events_unbound19:40:221
2745274995244,5cyclictest2821652-21kworker/u8:1+flush-179:023:35:240
2745274995242,8cyclictest2777504-21kworker/u8:1+events_unbound21:10:200
2745274995143,6cyclictest2785486-21kworker/u8:3+flush-179:021:20:150
2745274995141,8cyclictest2753437-21kworker/u8:2+events_unbound20:05:210
4505291490,14ptp4l61790irq/134-eno1-TxRx-019:10:003
274528099480,46cyclictest2830331-21turbostat23:34:593
2745274994841,6cyclictest2743681-21kworker/u8:1+flush-179:019:25:160
2745274994838,7cyclictest2773000-21apt-get20:35:140
2745279994642,2cyclictest0-21swapper/221:31:592
2745274994641,4cyclictest2822401-21kworker/u8:2+events_unbound23:15:230
2745274994638,5cyclictest2753437-21kworker/u8:2+flush-179:020:10:150
274527999450,42cyclictest0-21swapper/221:29:002
2745279994440,2cyclictest0-21swapper/200:06:002
2745274994438,5cyclictest2814389-21kworker/u8:0+flush-179:022:55:260
274527499433,3cyclictest2804853-21turbostat22:19:560
2745279994232,9cyclictest0-21swapper/221:50:002
274527999420,1cyclictest0-21swapper/219:34:592
2745274994237,4cyclictest2788688-21kworker/u8:0+flush-179:021:27:380
2745274994234,6cyclictest3105085-21systemd-journal22:46:370
274527499410,4cyclictest2749947-21munin-run19:25:000
274528099400,39cyclictest0-21swapper/321:08:593
2745279993936,2cyclictest0-21swapper/223:15:002
2745279993936,1cyclictest0-21swapper/221:55:002
274527999390,35cyclictest0-21swapper/220:50:002
274527499390,2cyclictest2822376-21basename23:10:000
2745280993835,2cyclictest0-21swapper/322:23:593
2745279993835,2cyclictest0-21swapper/219:25:002
2745279993834,2cyclictest0-21swapper/222:44:562
2745279993834,2cyclictest0-21swapper/219:40:592
274527999380,36cyclictest0-21swapper/219:57:592
274527999380,35cyclictest0-21swapper/222:10:592
274527999380,35cyclictest0-21swapper/219:20:012
2745280993735,1cyclictest0-21swapper/323:10:003
2745280993735,1cyclictest0-21swapper/320:55:593
2745280993734,2cyclictest0-21swapper/322:36:593
274528099370,35cyclictest0-21swapper/300:30:593
2745280993634,1cyclictest0-21swapper/323:10:593
2745280993634,1cyclictest0-21swapper/322:28:583
2745280993634,1cyclictest0-21swapper/321:17:593
2745280993634,1cyclictest0-21swapper/300:04:003
274528099360,35cyclictest0-21swapper/322:05:593
274528099360,1cyclictest0-21swapper/321:25:593
274528099360,1cyclictest0-21swapper/319:49:593
2745279993633,2cyclictest0-21swapper/223:08:002
2745279993633,2cyclictest0-21swapper/222:06:592
2745279993633,2cyclictest0-21swapper/221:12:592
2745279993632,2cyclictest0-21swapper/223:31:592
274527999360,1cyclictest0-21swapper/200:05:012
2745274993629,5cyclictest2726135-21kworker/u8:3+events_unbound19:14:560
2745280993532,2cyclictest0-21swapper/319:32:593
2745279993533,1cyclictest0-21swapper/222:35:002
2745279993532,2cyclictest0-21swapper/221:04:592
2745279993532,2cyclictest0-21swapper/221:04:592
2745279993531,1cyclictest0-21swapper/221:20:002
2745280993429,4cyclictest2788689-21turbostat21:25:013
2745279993431,2cyclictest0-21swapper/220:55:592
2745280993331,1cyclictest0-21swapper/323:53:593
2745280993331,1cyclictest0-21swapper/322:30:593
2745279993331,1cyclictest0-21swapper/222:55:002
2745279993331,1cyclictest0-21swapper/222:05:002
2745279993331,1cyclictest0-21swapper/220:30:012
2745280993230,1cyclictest0-21swapper/322:03:593
2745280993129,1cyclictest0-21swapper/323:24:563
2745280993129,1cyclictest0-21swapper/322:40:133
2745280993129,1cyclictest0-21swapper/319:35:023
2745279993129,1cyclictest0-21swapper/223:59:592
2745279993129,1cyclictest0-21swapper/223:30:002
274527999310,2cyclictest0-21swapper/200:10:172
2745276993123,5cyclictest2745135-21latency_hist19:10:031
2745274993127,2cyclictest2827164-21idleruntime-cro23:25:010
2745274993123,6cyclictest3105085-21systemd-journal22:20:010
2745280993028,1cyclictest0-21swapper/320:27:593
2745280993028,1cyclictest0-21swapper/300:09:553
2745280993027,2cyclictest0-21swapper/300:35:193
2745280993026,2cyclictest0-21swapper/323:42:593
274528099302,27cyclictest0-21swapper/319:50:013
274528099300,29cyclictest0-21swapper/319:40:143
274528099300,1cyclictest0-21swapper/319:15:593
2745279993028,1cyclictest0-21swapper/222:30:002
2745279993027,2cyclictest0-21swapper/220:15:142
274527999300,29cyclictest0-21swapper/200:30:122
274527999300,26cyclictest0-21swapper/219:45:142
4505291290,7ptp4l441ktimers/320:50:003
4505291290,6ptp4l441ktimers/321:40:003
2745280992927,1cyclictest0-21swapper/323:35:213
2745280992926,2cyclictest0-21swapper/321:55:013
2745280992926,2cyclictest0-21swapper/321:40:143
274528099290,28cyclictest0-21swapper/321:59:593
274528099290,28cyclictest0-21swapper/320:10:133
274528099290,27cyclictest0-21swapper/323:25:203
274528099290,27cyclictest0-21swapper/321:00:153
274528099290,27cyclictest0-21swapper/321:00:153
2745279992927,1cyclictest0-21swapper/219:40:012
2745279992927,1cyclictest0-21swapper/200:20:002
2745279992926,2cyclictest0-21swapper/220:01:582
274527999290,27cyclictest0-21swapper/222:20:152
274527999290,27cyclictest0-21swapper/220:12:592
274527999290,1cyclictest0-21swapper/222:15:102
4505291280,7ptp4l61790irq/134-eno1-TxRx-020:55:003
4505291280,7ptp4l441ktimers/323:00:013
4505291280,7ptp4l441ktimers/322:15:013
4505291280,6ptp4l441ktimers/320:45:003
4505291280,6ptp4l441ktimers/320:00:003
4505291280,6ptp4l441ktimers/319:15:003
2745279992826,1cyclictest0-21swapper/221:55:012
2745279992825,2cyclictest0-21swapper/223:15:162
2745279992825,2cyclictest0-21swapper/221:10:002
2745279992825,2cyclictest0-21swapper/220:20:132
2745279992825,2cyclictest0-21swapper/219:30:002
274527999280,27cyclictest0-21swapper/219:50:032
274527999280,26cyclictest0-21swapper/222:35:152
4505291270,6ptp4l61790irq/134-eno1-TxRx-020:05:013
4505291270,6ptp4l61790irq/134-eno1-TxRx-000:25:003
4505291270,6ptp4l441ktimers/323:05:013
4505291270,6ptp4l441ktimers/322:50:013
4505291270,6ptp4l441ktimers/321:15:003
4505291270,6ptp4l441ktimers/320:20:003
4505291270,5ptp4l441ktimers/320:30:013
4505291270,5ptp4l441ktimers/319:30:013
2745280992725,1cyclictest0-21swapper/322:15:103
2745280992725,1cyclictest0-21swapper/320:20:143
274528099270,26cyclictest0-21swapper/319:24:593
2745279992725,1cyclictest0-21swapper/221:40:002
2745279992724,2cyclictest0-21swapper/223:52:002
2745279992724,2cyclictest0-21swapper/223:01:592
2745279992724,2cyclictest0-21swapper/220:43:002
2745279992723,2cyclictest0-21swapper/200:22:002
274527999270,25cyclictest0-21swapper/222:57:592
4505291260,6ptp4l61790irq/134-eno1-TxRx-000:30:003
4505291260,6ptp4l441ktimers/321:50:013
4505291260,5ptp4l61790irq/134-eno1-TxRx-023:20:003
4505291260,5ptp4l61790irq/134-eno1-TxRx-022:55:013
2745280992624,1cyclictest0-21swapper/323:55:173
2745280992624,1cyclictest0-21swapper/321:32:593
2745280992623,2cyclictest0-21swapper/323:45:183
2745279992623,2cyclictest0-21swapper/200:39:592
4505291250,5ptp4l441ktimers/320:40:023
4505291250,4ptp4l61790irq/134-eno1-TxRx-000:20:013
2745279992522,2cyclictest0-21swapper/220:32:592
274527999250,2cyclictest0-21swapper/223:39:592
274527999250,1cyclictest0-21swapper/220:10:002
2745280992220,1cyclictest0-21swapper/300:11:593
2745279992219,2cyclictest0-21swapper/219:13:592
2745279992017,2cyclictest0-21swapper/220:50:592
2745279992017,2cyclictest0-21swapper/200:25:582
2745279991917,1cyclictest0-21swapper/223:25:002
2745279991916,2cyclictest0-21swapper/223:41:592
2745279991916,2cyclictest0-21swapper/220:40:002
274527499163,9cyclictest2745277-21latency_hist19:10:020
2745279991512,2cyclictest0-21swapper/222:47:582
2745279991511,2cyclictest0-21swapper/221:23:592
27452799930,2cyclictest0-21swapper/219:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional