You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-15 - 07:40
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot6.osadl.org (updated Mon Sep 15, 2025 00:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
245654299129105,20cyclictest2542755-21kworker/u8:2+flush-179:000:00:011
24565429910392,7cyclictest2492026-21latency_hist21:05:021
24565389910393,7cyclictest2550739-21latency_hist00:05:010
24565389910292,7cyclictest2478951-21kworker/u8:1+flush-179:020:25:220
2456538999790,5cyclictest2523075-21apt-get22:40:120
2456538999787,8cyclictest2509546-21latency_hist22:00:020
2456538999687,6cyclictest2498927-21kworker/u8:0+flush-179:021:25:180
2456542999382,9cyclictest2526508-21apt-get22:50:131
2456538999382,9cyclictest2557121-21latency_hist00:25:010
2456538999284,6cyclictest2459870-21kworker/u8:0+events_unbound19:55:230
2456538999281,9cyclictest601706-21systemd-journal21:00:170
2456538999178,10cyclictest2540983-21store23:35:010
2456538998985,3cyclictest2531083-21kworker/u8:3+flush-179:023:25:260
2456542998778,6cyclictest2498396-21latency_hist21:25:021
2456538998777,7cyclictest2464375-21kworker/u8:3+flush-179:019:35:240
2456538998681,3cyclictest2506405-21kworker/u8:1+flush-179:022:25:170
2456538998675,8cyclictest2519857-21kworker/u8:0+flush-179:022:35:220
2456542998573,9cyclictest2482190-21latency_hist20:30:011
2456538998574,8cyclictest2531083-21kworker/u8:3+flush-179:023:20:230
2456542998471,10cyclictest601706-21systemd-journal21:30:171
2456538998476,5cyclictest2475560-21kworker/u8:3+flush-179:021:15:230
2456542998370,9cyclictest601706-21systemd-journal23:25:011
2456538998375,6cyclictest2459870-21kworker/u8:0+flush-179:019:50:240
2456538998274,6cyclictest2503420-21kworker/u8:0+events_unbound22:10:010
2456538998272,7cyclictest2467594-21latency_hist19:45:010
2456542998165,14cyclictest2480603-21kworker/u8:0+flush-179:021:05:261
2456538998173,6cyclictest2506405-21kworker/u8:1+flush-179:022:10:250
2456538998171,7cyclictest2503420-21kworker/u8:0+flush-179:021:45:220
2456538998169,9cyclictest2540829-21kworker/u8:1+flush-179:023:45:260
2456542997971,6cyclictest601706-21systemd-journal00:38:451
2456542997862,14cyclictest2459870-21kworker/u8:0+flush-179:019:50:011
2456538997870,5cyclictest2496792-21kworker/u8:2+flush-179:021:50:230
2456538997864,7cyclictest2521188-21kworker/u8:2+events_unbound22:45:140
2456538997770,5cyclictest2484802-21kworker/u8:1+flush-179:020:35:260
2456538997770,5cyclictest2478951-21kworker/u8:1+flush-179:020:20:230
2456538997769,7cyclictest2524901-21kworker/u8:1+flush-179:023:10:230
2456538997767,8cyclictest2475811-21apt-get20:10:140
2456542997665,9cyclictest2475560-21kworker/u8:3+flush-179:020:45:181
2456538997669,6cyclictest2496792-21kworker/u8:2+flush-179:021:30:260
2456538997667,7cyclictest2475560-21kworker/u8:3+flush-179:021:20:210
2456538997664,10cyclictest2492280-21apt-get21:05:140
2456538997561,10cyclictest2438476-21kworker/u8:3+flush-179:019:10:220
2456538997550,9cyclictest2475560-21kworker/u8:3+flush-179:020:55:010
245653899741,70cyclictest816013-21ntpd22:55:220
2456542997349,21cyclictest601706-21systemd-journal21:14:591
2456538997363,7cyclictest2469182-21latency_hist19:50:010
2456538997362,8cyclictest2540829-21kworker/u8:1+flush-179:023:35:170
2456538997266,4cyclictest2506405-21kworker/u8:1+flush-179:022:30:180
2456538997263,7cyclictest2542755-21kworker/u8:2+events_unbound00:25:180
2456542997162,6cyclictest601706-21systemd-journal22:15:171
2456542996958,8cyclictest2542975-21apt-get23:40:111
2456542996858,7cyclictest502-21rs:main1
2456542996857,8cyclictest601706-21systemd-journal20:50:171
2456538996862,5cyclictest2524901-21kworker/u8:1+flush-179:023:15:270
2456538996862,5cyclictest2459870-21kworker/u8:0+flush-179:019:30:230
2456542996753,11cyclictest2540983-21store23:35:011
2456538996761,5cyclictest2506405-21kworker/u8:1+flush-179:022:00:230
2456538996759,6cyclictest2547482-21latency_hist23:55:010
2456542996655,8cyclictest2472368-21latency_hist20:00:011
2456538996661,3cyclictest2542755-21kworker/u8:2+flush-179:000:15:250
2456538996660,4cyclictest2515069-21kworker/u8:0+flush-179:022:15:230
2456538996657,6cyclictest2438476-21kworker/u8:3+flush-179:019:15:250
2456542996553,10cyclictest2542755-21kworker/u8:2+flush-179:000:20:241
2456538996561,3cyclictest2488842-21kworker/u8:1+events_unbound20:55:260
2456538996557,5cyclictest2479013-21latency_hist20:20:010
2456538996553,8cyclictest2459870-21kworker/u8:0+events_unbound20:05:200
2456538996458,4cyclictest2506405-21kworker/u8:1+flush-179:022:20:170
2456538996458,4cyclictest2480603-21kworker/u8:0+flush-179:020:30:180
2456542996352,8cyclictest601706-21systemd-journal21:50:001
2456538996354,6cyclictest2542755-21kworker/u8:2+flush-179:000:00:000
2456542996253,7cyclictest2549367-21apt-get00:00:131
2456538996256,4cyclictest2459889-21apt-get19:20:120
2456538996253,7cyclictest2524901-21kworker/u8:1+events_unbound22:50:220
2456542996151,8cyclictest2542755-21kworker/u8:2+events_unbound00:09:591
2456538996051,6cyclictest2549136-21kworker/u8:3+flush-179:000:05:240
2456542995946,10cyclictest2511657-21apt-get22:05:121
2456538995952,5cyclictest2530490-21kworker/u8:2+flush-179:023:05:240
2456542995853,4cyclictest2456462-21kworker/u8:1+flush-179:019:26:171
2456542995853,4cyclictest2456462-21kworker/u8:1+flush-179:019:26:171
2456542995851,5cyclictest2496792-21kworker/u8:2+flush-179:021:35:231
2456542995844,4cyclictest2477275-21store20:15:011
2456538995851,5cyclictest2475560-21kworker/u8:3+events_unbound21:35:240
2456542995751,5cyclictest2530490-21kworker/u8:2+flush-179:023:00:261
2456542995748,7cyclictest2469437-21apt-get19:50:131
2456542995747,8cyclictest2499986-21latency_hist21:30:011
2456542995746,9cyclictest2553920-21kworker/u8:1+flush-179:000:15:181
2456538995750,5cyclictest2475560-21kworker/u8:3+events_unbound20:45:200
2456538995748,6cyclictest2438476-21kworker/u8:3+events_unbound19:25:170
2456538995748,6cyclictest2438476-21kworker/u8:3+events_unbound19:25:160
2456538995649,5cyclictest2472424-21kworker/u8:3+flush-179:020:00:210
2456544995552,2cyclictest0-21swapper/219:21:592
2456542995548,5cyclictest2542723-21latency_hist23:40:011
2456542995543,10cyclictest2546150-21apt-get23:50:131
2456538995548,5cyclictest2530490-21kworker/u8:2+events_unbound23:00:250
2456538995548,5cyclictest2475560-21kworker/u8:3+flush-179:021:40:210
2456538995544,9cyclictest2547718-21kworker/u8:0+events_unbound00:10:290
2456542995442,9cyclictest601706-21systemd-journal21:17:171
2456542995349,3cyclictest2438476-21kworker/u8:3+flush-179:019:11:181
2456542995343,8cyclictest2480604-21latency_hist20:25:001
2456542995341,10cyclictest601706-21systemd-journal22:40:001
2456542995246,5cyclictest2496792-21kworker/u8:2+flush-179:021:50:271
2456542995145,5cyclictest2484802-21kworker/u8:1+flush-179:020:35:211
2456542995139,10cyclictest601706-21systemd-journal23:15:001
2456542995031,8cyclictest2472921-21diskstats20:00:151
4505291490,14ptp4l441ktimers/319:10:013
2456542994939,8cyclictest2506405-21kworker/u8:1+flush-179:022:25:001
2456542994938,9cyclictest2438476-21kworker/u8:3+flush-179:019:15:191
2456538994843,4cyclictest2475560-21kworker/u8:3+flush-179:021:10:270
2456542994741,4cyclictest2537929-21apt-get23:25:131
2456542994639,6cyclictest2524901-21kworker/u8:1+flush-179:022:55:221
2456542994639,6cyclictest2503420-21kworker/u8:0+flush-179:021:55:271
2456547994538,5cyclictest2542724-21turbostat23:40:003
2456538994537,6cyclictest2553963-21kworker/u8:3+events_unbound00:30:240
2456538994537,5cyclictest2558733-21kworker/u8:0+flush-179:000:35:230
2456542994434,8cyclictest2506405-21kworker/u8:1+flush-179:022:25:241
2456538994438,4cyclictest2540829-21kworker/u8:1+events_unbound23:40:240
2456542994235,5cyclictest601706-21systemd-journal19:33:161
2456542994234,6cyclictest2524901-21kworker/u8:1+flush-179:023:15:221
2456542994234,5cyclictest2485652-21latency_hist20:45:001
2456542994036,3cyclictest2465984-21cron19:40:001
2456542994035,4cyclictest722-21Xorg22:32:591
2456542994035,3cyclictest2558974-21munin-run00:30:011
2456542994033,5cyclictest2478645-21swap20:15:251
2456542994033,5cyclictest2456462-21kworker/u8:1+flush-179:019:20:261
2456542994031,7cyclictest2480603-21kworker/u8:0+events_unbound20:30:261
2456544993936,2cyclictest0-21swapper/219:47:592
2456542993934,4cyclictest2524441-21systemd-run22:45:001
2456547993831,5cyclictest2527851-21turbostat22:55:003
2456544993835,2cyclictest0-21swapper/220:14:562
245654499380,36cyclictest0-21swapper/221:41:582
245654499380,35cyclictest0-21swapper/200:09:592
2456542993829,7cyclictest2530490-21kworker/u8:2+flush-179:023:05:251
245653899380,37cyclictest2485402-21turbostat20:44:550
2456547993735,1cyclictest0-21swapper/319:51:593
2456547993733,2cyclictest0-21swapper/322:35:003
245654799371,33cyclictest67-21kworker/3:1+events19:25:143
245654799371,33cyclictest67-21kworker/3:1+events19:25:133
245654799370,36cyclictest0-21swapper/320:19:593
245654799370,35cyclictest0-21swapper/320:10:593
2456542993723,12cyclictest0-21swapper/121:45:011
2456547993634,1cyclictest0-21swapper/323:03:593
2456547993633,2cyclictest0-21swapper/322:26:583
245654799360,35cyclictest0-21swapper/320:58:003
245654799360,35cyclictest0-21swapper/300:39:553
245654799360,1cyclictest0-21swapper/319:18:593
245654799360,1cyclictest0-21swapper/300:14:553
2456544993633,2cyclictest0-21swapper/223:25:592
2456547993533,1cyclictest0-21swapper/319:12:593
2456544993532,2cyclictest0-21swapper/221:30:592
2456544993532,2cyclictest0-21swapper/220:52:592
2456544993532,2cyclictest0-21swapper/219:34:562
245654499350,1cyclictest0-21swapper/219:20:002
2456547993432,1cyclictest0-21swapper/323:28:593
2456547993432,1cyclictest0-21swapper/322:05:003
2456547993432,1cyclictest0-21swapper/320:25:003
2456542993429,4cyclictest476-21dbus-daemon20:10:001
2456547993330,2cyclictest0-21swapper/321:13:593
2456544993330,2cyclictest0-21swapper/222:16:592
2456542993329,2cyclictest722-21Xorg19:44:001
2456542993326,6cyclictest2506405-21kworker/u8:1+flush-179:022:10:051
2456547993230,2cyclictest0-21swapper/322:14:593
2456547993230,1cyclictest0-21swapper/322:35:593
2456547993229,2cyclictest0-21swapper/322:42:593
2456544993229,2cyclictest0-21swapper/223:39:552
2456544993229,2cyclictest0-21swapper/200:12:592
2456544993228,2cyclictest0-21swapper/219:10:592
245654499320,30cyclictest0-21swapper/219:56:002
245654299322,16cyclictest2490438-21sendmail21:00:001
245654299322,15cyclictest2560752-21cat00:35:011
2456547993129,1cyclictest0-21swapper/323:17:593
2456547993128,2cyclictest0-21swapper/319:55:593
2456547993128,2cyclictest0-21swapper/300:26:593
2456544993129,1cyclictest0-21swapper/221:54:592
2456544993128,2cyclictest0-21swapper/220:22:592
245654499310,29cyclictest0-21swapper/222:09:002
4505291300,10ptp4l441ktimers/320:34:563
2456547993028,1cyclictest0-21swapper/321:49:553
2456544993027,2cyclictest0-21swapper/222:25:142
2456544993027,2cyclictest0-21swapper/220:03:582
245654499300,29cyclictest0-21swapper/222:45:032
2456542993023,6cyclictest722-21Xorg23:47:591
245654299301,28cyclictest0-21swapper/122:00:031
245654299301,28cyclictest0-21swapper/100:10:161
2456547992926,2cyclictest0-21swapper/321:44:593
2456547992926,2cyclictest0-21swapper/300:20:003
245654799290,28cyclictest0-21swapper/322:20:143
245654799290,28cyclictest0-21swapper/321:50:143
245654799290,27cyclictest0-21swapper/322:45:133
245654799290,27cyclictest0-21swapper/300:00:143
2456544992926,2cyclictest0-21swapper/221:38:592
2456544992926,2cyclictest0-21swapper/220:45:082
2456544992926,2cyclictest0-21swapper/220:25:152
2456544992926,2cyclictest0-21swapper/200:20:152
245654499290,2cyclictest0-21swapper/222:10:012
245654499290,27cyclictest0-21swapper/223:50:142
245654499290,27cyclictest0-21swapper/222:00:022
245654499290,27cyclictest0-21swapper/219:40:132
245654499290,27cyclictest0-21swapper/200:25:022
4505291280,7ptp4l61790irq/134-eno1-TxRx-023:10:013
4505291280,7ptp4l441ktimers/320:10:013
2456547992826,1cyclictest0-21swapper/323:48:593
2456547992826,1cyclictest0-21swapper/321:55:153
2456547992826,1cyclictest0-21swapper/321:15:153
2456547992826,1cyclictest0-21swapper/320:35:133
2456547992826,1cyclictest0-21swapper/300:23:593
2456547992825,2cyclictest0-21swapper/319:45:143
245654799280,27cyclictest0-21swapper/321:00:153
245654799280,26cyclictest0-21swapper/323:20:153
245654799280,1cyclictest0-21swapper/323:40:133
2456544992825,2cyclictest0-21swapper/223:45:592
2456544992825,2cyclictest0-21swapper/223:12:002
2456544992825,2cyclictest0-21swapper/222:30:132
245654499280,2cyclictest0-21swapper/220:55:152
245654499280,26cyclictest0-21swapper/222:36:002
245654499280,26cyclictest0-21swapper/221:06:592
245654499280,26cyclictest0-21swapper/219:35:162
245654499280,1cyclictest0-21swapper/221:20:122
4505291270,7ptp4l441ktimers/321:10:003
2456547992725,1cyclictest0-21swapper/323:30:123
2456547992725,1cyclictest0-21swapper/321:25:153
2456547992724,2cyclictest0-21swapper/322:19:553
245654799270,26cyclictest0-21swapper/320:43:593
245654799270,1cyclictest0-21swapper/300:05:163
2456544992724,2cyclictest0-21swapper/223:55:142
2456544992724,2cyclictest0-21swapper/223:05:592
2456544992724,2cyclictest0-21swapper/200:00:592
245654499270,25cyclictest0-21swapper/223:17:592
4505291260,6ptp4l61790irq/134-eno1-TxRx-021:39:553
4505291260,6ptp4l441ktimers/322:59:553
4505291260,6ptp4l441ktimers/321:35:003
245654799260,1cyclictest0-21swapper/320:02:593
2456544992623,2cyclictest0-21swapper/223:40:592
2456544992623,2cyclictest0-21swapper/221:11:002
245654499260,2cyclictest0-21swapper/223:22:582
245654499260,2cyclictest0-21swapper/223:00:592
245654499260,25cyclictest0-21swapper/220:16:582
245654499260,24cyclictest0-21swapper/222:59:592
245654499260,1cyclictest0-21swapper/221:27:582
4505291250,6ptp4l61790irq/134-eno1-TxRx-021:25:013
4505291250,6ptp4l61790irq/134-eno1-TxRx-019:39:553
4505291250,5ptp4l61790irq/134-eno1-TxRx-020:54:563
4505291250,5ptp4l441ktimers/322:09:563
4505291250,5ptp4l441ktimers/300:35:003
2456547992523,1cyclictest0-21swapper/323:10:153
2456547992523,1cyclictest0-21swapper/320:26:593
2456544992522,2cyclictest0-21swapper/221:45:592
2456544992522,2cyclictest0-21swapper/220:33:582
2456544992522,2cyclictest0-21swapper/200:30:592
245654499250,2cyclictest0-21swapper/219:54:562
245654499250,1cyclictest0-21swapper/219:30:012
245654499250,1cyclictest0-21swapper/219:30:002
201699250,7rtkit-daemon486-21gpm19:08:201
4505291240,4ptp4l441ktimers/323:59:553
4505291240,4ptp4l441ktimers/323:55:013
2456547992422,1cyclictest0-21swapper/320:46:583
2456544992422,1cyclictest0-21swapper/200:36:592
2456544992421,2cyclictest0-21swapper/222:40:142
4505291230,5ptp4l61790irq/134-eno1-TxRx-019:35:003
4505291230,4ptp4l61790irq/134-eno1-TxRx-019:24:563
2456544992321,1cyclictest0-21swapper/220:44:002
2456544992320,2cyclictest0-21swapper/220:08:592
2456544992319,2cyclictest0-21swapper/221:57:582
4505291220,3ptp4l441ktimers/319:44:563
2456544992220,1cyclictest0-21swapper/220:40:002
2456544992219,2cyclictest0-21swapper/221:19:552
2456544992219,2cyclictest0-21swapper/200:15:592
2456544991916,2cyclictest0-21swapper/221:04:562
2456544991816,1cyclictest0-21swapper/222:21:002
245653899122,8cyclictest2456534-21cyclictest19:10:020
245654499107,2cyclictest0-21swapper/222:50:182
24565449985,2cyclictest0-21swapper/223:35:012
24565449920,1cyclictest0-21swapper/219:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional