You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-16 - 11:05
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot6.osadl.org (updated Wed Jul 16, 2025 00:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3583479910679,8cyclictest439862-21/usr/sbin/munin23:20:240
3583509910388,11cyclictest437757-21latency_hist23:15:011
3583479910290,9cyclictest384174-21cp20:30:020
3583479910093,5cyclictest418177-21kworker/u8:2+events_unbound22:35:010
358347999786,9cyclictest358496-21apt-get19:10:130
358347999585,7cyclictest367806-21latency_hist19:40:010
358347999386,5cyclictest374118-21kworker/u8:1+events_unbound20:00:010
358347999381,9cyclictest465388-21latency_hist00:40:010
358347999279,10cyclictest410448-21apt-get21:50:140
358350998980,6cyclictest396723-21kworker/u8:3+flush-179:021:25:011
358347998776,9cyclictest419746-21apt-get22:20:010
358350998476,6cyclictest395357-21apt-get21:05:141
358350998475,6cyclictest384216-21apt-get20:30:151
358350998472,9cyclictest445974-21apt-get23:39:591
358347998473,8cyclictest374418-21apt-get20:00:140
358347998471,11cyclictest390579-21apt-get20:50:110
358350998378,4cyclictest441189-21kworker/u8:0+flush-179:000:04:001
358347998276,5cyclictest418177-21kworker/u8:2+flush-179:022:20:030
358347998269,10cyclictest443053-21apt-get23:30:120
358347998173,6cyclictest406709-21gpgv21:40:010
358347998067,10cyclictest371226-21apt-get19:50:120
358350997968,9cyclictest422137-21diskstats22:25:201
358350997872,4cyclictest442833-21kworker/u8:3+flush-179:023:55:221
358350997864,11cyclictest3105085-21systemd-journal22:24:561
358347997870,7cyclictest423171-21kworker/u8:0+flush-179:022:35:010
358347997870,6cyclictest353743-21kworker/u8:3+flush-179:019:45:160
358347997867,8cyclictest387145-21latency_hist20:40:010
358347997867,6cyclictest396723-21kworker/u8:3+flush-179:021:40:020
358350997670,4cyclictest463259-21kworker/u8:0+flush-179:000:30:261
358347997670,4cyclictest452655-21kworker/u8:1+events_unbound00:15:260
358347997562,11cyclictest3105085-21systemd-journal20:15:010
358350997465,6cyclictest423171-21kworker/u8:0+flush-179:022:35:201
358350997463,8cyclictest455842-21latency_hist00:10:011
358350997462,10cyclictest376006-21apt-get20:05:161
358347997468,4cyclictest359862-21kworker/u8:2+flush-179:019:30:010
358350997355,16cyclictest433255-21apt-get23:00:131
358347997365,5cyclictest411757-21kworker/u8:1+flush-179:022:00:160
358350997261,8cyclictest375750-21latency_hist20:05:011
358347997267,4cyclictest383678-21kworker/u8:1+events_unbound20:50:010
358347997266,4cyclictest442833-21kworker/u8:3+flush-179:023:55:150
358347997262,8cyclictest437766-21apt-get23:14:590
358347997260,9cyclictest364862-21apt-get19:30:130
358350997166,3cyclictest398529-21apt-get21:15:121
358347997165,5cyclictest375730-21kworker/u8:0+flush-179:020:25:230
358347997165,4cyclictest396663-21kworker/u8:2+flush-179:021:15:210
358347997158,10cyclictest462460-21apt-get00:30:140
358350997060,7cyclictest353743-21kworker/u8:3+flush-179:019:10:261
358350997058,8cyclictest452655-21kworker/u8:1+events_unbound00:15:201
358347997063,5cyclictest396663-21kworker/u8:2+events_unbound21:10:280
358350996959,7cyclictest367765-21kworker/u8:0+flush-179:019:45:001
358347996962,5cyclictest418177-21kworker/u8:2+events_unbound23:05:140
358347996962,5cyclictest375798-21kworker/u8:2+flush-179:020:05:260
358350996856,8cyclictest396723-21kworker/u8:3+flush-179:021:25:201
358347996863,4cyclictest408399-21kworker/u8:2+flush-179:021:45:040
358347996861,5cyclictest396663-21kworker/u8:2+events_unbound21:25:300
358347996860,5cyclictest452655-21kworker/u8:1+events_unbound00:25:150
358355996740,1cyclictest0-21swapper/323:20:593
358350996762,3cyclictest460655-21kworker/u8:2+flush-179:000:25:221
358350996756,9cyclictest356761-21kworker/u8:1+flush-179:019:35:221
358347996762,4cyclictest367765-21kworker/u8:0+flush-179:019:40:170
358350996654,10cyclictest377590-21apt-get20:10:131
358347996655,8cyclictest454258-21latency_hist00:05:000
358347996653,11cyclictest387410-21apt-get20:40:150
358347996653,11cyclictest387410-21apt-get20:40:140
358350996455,6cyclictest424754-21kworker/u8:1+flush-179:023:25:221
358350996452,9cyclictest380785-21apt-get20:20:141
358355996355,6cyclictest459021-21turbostat00:20:003
358350996355,6cyclictest438270-21apt-get23:15:141
358350996353,8cyclictest428458-21/usr/sbin/munin22:45:251
358350996353,8cyclictest385801-21apt-get20:35:131
358350996259,2cyclictest396723-21kworker/u8:3+events_unbound21:40:261
358350996257,3cyclictest359862-21kworker/u8:2+events_unbound19:20:261
358350996252,7cyclictest441189-21kworker/u8:0+flush-179:023:50:181
358347996147,11cyclictest401594-21gpgv21:25:010
358355996053,5cyclictest411788-21turbostat21:55:003
358350996056,3cyclictest411757-21kworker/u8:1+flush-179:022:00:201
358350996048,9cyclictest424770-21latency_hist22:35:001
358347996055,4cyclictest375798-21kworker/u8:2+events_unbound20:25:010
358347995953,4cyclictest418177-21kworker/u8:2+flush-179:022:50:140
358347995950,7cyclictest424754-21kworker/u8:1+flush-179:022:55:210
358350995852,4cyclictest3105085-21systemd-journal22:11:371
358350995847,9cyclictest3105085-21systemd-journal21:57:361
358350995847,8cyclictest3105085-21systemd-journal19:20:001
358350995846,9cyclictest375730-21kworker/u8:0+flush-179:020:25:151
358350995840,15cyclictest396944-21apt-get21:10:151
358347995852,4cyclictest441189-21kworker/u8:0+events_unbound23:35:240
358350995751,5cyclictest383678-21kworker/u8:1+flush-179:020:55:261
358350995747,8cyclictest430062-21apt-get22:50:101
358347995751,5cyclictest441189-21kworker/u8:0+events_unbound23:40:030
358347995751,4cyclictest389971-21kworker/u8:0+flush-179:021:05:240
358347995651,4cyclictest411757-21kworker/u8:1+flush-179:021:57:050
358347995650,5cyclictest424754-21kworker/u8:1+flush-179:023:00:260
358347995645,8cyclictest379451-21kworker/u8:1+flush-179:020:15:210
358351995552,2cyclictest0-21swapper/221:07:582
358350995547,5cyclictest366193-21latency_hist19:35:011
358347995553,1cyclictest0-21swapper/021:02:000
358347995548,5cyclictest418177-21kworker/u8:2+flush-179:022:40:240
358350995447,4cyclictest424754-21kworker/u8:1+flush-179:022:55:201
358350995344,7cyclictest405134-21/usr/sbin/munin21:35:241
358347995347,5cyclictest415009-21kworker/u8:3+events_unbound22:10:250
358347995347,5cyclictest359862-21kworker/u8:2+events_unbound19:15:250
358350995243,7cyclictest424754-21kworker/u8:1+flush-179:023:05:221
358350995243,7cyclictest353743-21kworker/u8:3+events_unbound19:55:181
358347995247,2cyclictest441189-21kworker/u8:0+events_unbound23:45:140
358350995145,5cyclictest415009-21kworker/u8:3+flush-179:022:15:221
358347995146,4cyclictest455890-21kworker/u8:3+flush-179:000:15:000
358347995146,4cyclictest396723-21kworker/u8:3+events_unbound21:30:230
358347995144,5cyclictest418177-21kworker/u8:2+events_unbound22:25:170
358347995039,9cyclictest451006-21latency_hist23:55:020
358350994943,4cyclictest418177-21kworker/u8:2+events_unbound22:45:021
358350994939,8cyclictest359862-21kworker/u8:2+flush-179:019:25:241
358347994942,5cyclictest1-21systemd00:10:000
4505291480,13ptp4l61790irq/134-eno1-TxRx-019:09:553
358350994842,5cyclictest424754-21kworker/u8:1+flush-179:023:20:211
358350994839,7cyclictest424754-21kworker/u8:1+flush-179:023:40:211
358350994839,7cyclictest3105085-21systemd-journal00:25:001
358350994839,6cyclictest393752-21/usr/sbin/munin21:00:241
358350994837,8cyclictest3105085-21systemd-journal00:40:001
358350994739,6cyclictest380526-21latency_hist20:20:021
358350994738,7cyclictest3105085-21systemd-journal21:31:361
358350994540,3cyclictest406475-21kworker/u8:0+flush-179:021:55:021
358350994536,7cyclictest375798-21kworker/u8:2+flush-179:020:45:191
358350994438,5cyclictest353743-21kworker/u8:3+flush-179:019:45:231
35834799441,40cyclictest3105085-21systemd-journal23:19:590
358351994234,7cyclictest0-21swapper/223:13:592
358350994234,6cyclictest396723-21kworker/u8:3+events_unbound21:45:201
358347994236,5cyclictest418177-21kworker/u8:2+flush-179:022:49:560
35834799421,39cyclictest442799-21munin-run23:30:000
35835599410,1cyclictest0-21swapper/322:07:583
35835199410,39cyclictest0-21swapper/223:37:592
35834799411,3cyclictest460621-21cron00:25:000
358355994038,1cyclictest0-21swapper/322:57:593
358355994035,4cyclictest460622-21turbostat00:25:003
358355994034,5cyclictest439604-21turbostat23:19:593
35835599400,4cyclictest406493-21turbostat21:40:003
358351994034,5cyclictest0-21swapper/220:09:592
35835199400,1cyclictest0-21swapper/223:22:592
358355993933,5cyclictest442807-21turbostat23:30:003
35835199390,36cyclictest0-21swapper/223:19:582
358351993835,2cyclictest0-21swapper/222:36:592
35835199380,35cyclictest0-21swapper/221:04:562
358350993832,4cyclictest411757-21kworker/u8:1+flush-179:022:05:261
358355993734,2cyclictest0-21swapper/300:39:593
35835599370,36cyclictest0-21swapper/320:48:003
358351993734,2cyclictest0-21swapper/222:16:002
358351993734,2cyclictest0-21swapper/220:51:002
358351993733,2cyclictest0-21swapper/219:55:592
35835199370,35cyclictest0-21swapper/222:45:002
35835199370,35cyclictest0-21swapper/222:03:002
35835199370,35cyclictest0-21swapper/221:24:582
358355993634,1cyclictest0-21swapper/322:00:003
358355993631,4cyclictest404914-21turbostat21:35:003
35835599360,35cyclictest0-21swapper/322:39:593
35835599360,35cyclictest0-21swapper/321:27:593
35835599360,1cyclictest0-21swapper/322:25:593
358351993633,2cyclictest0-21swapper/222:49:552
358351993633,2cyclictest0-21swapper/220:03:592
358351993633,2cyclictest0-21swapper/219:26:592
35835199360,1cyclictest0-21swapper/220:22:002
358347993630,5cyclictest406475-21kworker/u8:0+flush-179:022:07:030
358355993533,1cyclictest0-21swapper/321:01:593
358355993533,1cyclictest0-21swapper/319:12:593
35835199350,1cyclictest0-21swapper/223:44:002
358350993529,4cyclictest442833-21kworker/u8:3+flush-179:023:50:011
358350993529,4cyclictest389971-21kworker/u8:0+events_unbound20:50:151
358347993528,5cyclictest389971-21kworker/u8:0+flush-179:020:55:240
358355993432,1cyclictest0-21swapper/300:34:553
358355993431,2cyclictest0-21swapper/319:47:593
358355993428,5cyclictest393516-21turbostat21:00:003
358350993426,6cyclictest383678-21kworker/u8:1+flush-179:020:45:011
358350993426,6cyclictest383678-21kworker/u8:1+flush-179:020:45:001
358350993424,8cyclictest372555-21latency_hist19:55:011
358355993330,2cyclictest0-21swapper/321:40:593
358351993330,2cyclictest0-21swapper/223:49:552
358351993330,2cyclictest0-21swapper/221:25:592
358351993330,2cyclictest0-21swapper/220:11:592
358355993230,1cyclictest0-21swapper/323:32:583
358355993230,1cyclictest0-21swapper/300:29:553
358351993229,2cyclictest0-21swapper/223:58:002
358355993129,1cyclictest0-21swapper/320:27:593
358355993128,2cyclictest0-21swapper/322:40:593
358355993128,2cyclictest0-21swapper/321:22:593
358351993129,1cyclictest0-21swapper/219:45:002
358351993128,2cyclictest0-21swapper/223:27:002
358351993128,2cyclictest0-21swapper/222:21:592
358351993128,2cyclictest0-21swapper/200:39:552
358351993127,2cyclictest0-21swapper/223:06:592
35835199310,29cyclictest0-21swapper/219:20:142
358350993127,3cyclictest59-21kcompactd023:35:001
358350993127,2cyclictest722-21Xorg00:13:591
358355993028,1cyclictest0-21swapper/323:44:543
358355993028,1cyclictest0-21swapper/323:04:593
358355993027,2cyclictest0-21swapper/322:32:003
358351993027,2cyclictest0-21swapper/223:52:592
358351993027,2cyclictest0-21swapper/220:36:592
358351993027,2cyclictest0-21swapper/220:15:142
358351993022,7cyclictest0-21swapper/222:13:592
35835199300,29cyclictest0-21swapper/221:50:162
35835199300,23cyclictest0-21swapper/221:45:032
358355992927,1cyclictest0-21swapper/323:39:563
358355992927,1cyclictest0-21swapper/320:20:163
358355992927,1cyclictest0-21swapper/319:54:593
358355992926,2cyclictest0-21swapper/323:55:143
358355992926,2cyclictest0-21swapper/320:35:143
358355992923,4cyclictest385561-21turbostat20:34:593
35835599290,28cyclictest0-21swapper/321:15:133
35835599290,27cyclictest0-21swapper/323:05:583
35835599290,27cyclictest0-21swapper/321:05:163
35835599290,27cyclictest0-21swapper/320:40:173
35835599290,27cyclictest0-21swapper/320:40:163
35835599290,1cyclictest0-21swapper/320:05:183
358351992927,1cyclictest0-21swapper/200:00:142
358351992926,2cyclictest0-21swapper/221:10:182
358351992926,2cyclictest0-21swapper/219:45:592
35835199290,27cyclictest0-21swapper/223:00:142
35835199290,27cyclictest0-21swapper/222:25:172
35835199290,27cyclictest0-21swapper/221:35:132
35835199290,27cyclictest0-21swapper/200:20:142
35835199290,27cyclictest0-21swapper/200:10:142
35835199290,21cyclictest0-21swapper/219:10:152
4505291280,7ptp4l441ktimers/323:50:003
4505291280,6ptp4l441ktimers/319:35:003
358355992826,1cyclictest0-21swapper/322:20:133
358355992825,2cyclictest0-21swapper/323:10:133
358355992825,2cyclictest0-21swapper/322:10:203
35835599280,27cyclictest0-21swapper/319:35:133
35835599280,26cyclictest0-21swapper/319:40:133
358351992825,2cyclictest0-21swapper/221:57:592
35835199280,2cyclictest0-21swapper/200:26:592
35835199280,2cyclictest0-21swapper/200:15:182
35835199280,27cyclictest0-21swapper/221:40:162
35835199280,26cyclictest0-21swapper/220:30:052
35835199280,26cyclictest0-21swapper/200:05:262
358347992821,5cyclictest359862-21kworker/u8:2+flush-179:019:20:130
4505291270,6ptp4l61790irq/134-eno1-TxRx-020:20:013
358355992720,5cyclictest375763-21turbostat20:05:003
35835599270,26cyclictest0-21swapper/322:51:583
358351992724,2cyclictest0-21swapper/223:34:552
358351992724,2cyclictest0-21swapper/222:54:552
358351992724,2cyclictest0-21swapper/221:19:002
35835199270,1cyclictest0-21swapper/219:30:142
4505291260,6ptp4l61790irq/134-eno1-TxRx-020:15:013
4505291260,6ptp4l441ktimers/300:15:003
4505291260,5ptp4l441ktimers/321:15:013
358355992623,2cyclictest0-21swapper/300:01:583
358351992623,2cyclictest0-21swapper/219:19:552
358351992623,1cyclictest0-21swapper/220:27:592
35835199260,24cyclictest0-21swapper/222:05:152
35835199260,24cyclictest0-21swapper/221:35:002
4505291250,5ptp4l441ktimers/321:49:553
4505291250,5ptp4l441ktimers/320:54:563
4505291250,5ptp4l441ktimers/319:24:563
4505291250,4ptp4l441ktimers/320:00:003
358355992523,1cyclictest0-21swapper/323:53:003
35835599250,24cyclictest0-21swapper/319:25:593
35835599250,24cyclictest0-21swapper/319:19:553
358351992522,2cyclictest0-21swapper/219:53:592
358351992522,2cyclictest0-21swapper/200:30:162
35835199250,1cyclictest0-21swapper/219:40:002
358350992521,3cyclictest0-21swapper/119:10:021
4505291240,5ptp4l61790irq/134-eno1-TxRx-000:09:553
4505291240,5ptp4l441ktimers/322:49:563
4505291240,5ptp4l441ktimers/322:04:563
358351992418,5cyclictest0-21swapper/222:55:162
358351992219,2cyclictest0-21swapper/222:30:592
358351992218,2cyclictest0-21swapper/220:40:592
358351992218,2cyclictest0-21swapper/220:40:592
358351992017,2cyclictest0-21swapper/220:55:592
201699200,11rtkit-daemon1999-21lightdm-gtk-gre19:08:250
35835599159,5cyclictest419747-21turbostat22:20:013
35835199145,2cyclictest0-21swapper/220:45:162
3583519930,2cyclictest0-21swapper/219:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional