You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-02 - 13:47
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot6.osadl.org (updated Tue Dec 02, 2025 00:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
32081699910797,7cyclictest3302214-40logrotate00:00:011
32081669910493,8cyclictest3264544-21latency_hist22:05:010
32081669910493,8cyclictest3264544-21latency_hist22:05:000
32081669910493,7cyclictest3243595-21latency_hist21:00:010
3208166999584,8cyclictest271-21systemd-journal23:45:000
3208169998978,8cyclictest3281219-21latency_hist22:55:011
3208169998574,8cyclictest3273010-21latency_hist22:30:001
3208169998373,7cyclictest3211264-21latency_hist19:20:011
3208169998273,7cyclictest3292363-21latency_hist23:30:021
320816699801,74cyclictest3302281-21munin-run00:00:010
3208166997767,7cyclictest3249959-21latency_hist21:20:010
3208166997564,8cyclictest3251557-21latency_hist21:25:000
3208169997464,7cyclictest502-21rs:main1
3208166997463,9cyclictest3307300-21apt-get00:15:120
3208169997267,4cyclictest3187022-21kworker/u8:2+flush-179:019:25:261
3208169997263,7cyclictest3218133-21/usr/sbin/munin19:40:241
3208166997262,8cyclictest3275115-21apt-get22:35:140
3208169997057,12cyclictest3225820-21kworker/u8:1+flush-179:020:04:591
3208166997061,6cyclictest3311822-21latency_hist00:30:020
3208166996959,7cyclictest3269569-21latency_hist22:20:010
202199680,6rtkit-daemon2020-21rtkit-daemon20:40:210
3208166996758,7cyclictest3271680-21apt-get22:25:130
3208166996756,9cyclictest3291319-21diskstats23:25:160
3208169996454,7cyclictest3300625-21latency_hist23:55:011
3208166996459,3cyclictest3261396-21kworker/u8:2+flush-179:022:20:260
3208169996050,7cyclictest271-21systemd-journal23:20:001
3208166995948,8cyclictest3216287-21latency_hist19:35:010
3208169995750,5cyclictest3313962-21kworker/u8:1+flush-179:000:40:001
3208169995750,4cyclictest3240414-21latency_hist20:50:011
3208169995745,10cyclictest3232208-21latency_hist20:25:011
3208166995646,6cyclictest3214698-21latency_hist19:30:000
3208166995644,10cyclictest3287840-21apt-get23:15:130
3208166995547,6cyclictest3262942-21latency_hist22:00:010
3208166995546,7cyclictest3281456-21/usr/sbin/munin22:55:240
3208166995545,7cyclictest271-21systemd-journal00:03:590
3208169995446,6cyclictest271-21systemd-journal23:06:141
3208166995450,2cyclictest723-21Xorg21:12:590
3208166995445,7cyclictest3266928-21diskstats22:10:140
4670791530,25ptp4l61690irq/136-eno1-TxRx-019:05:223
4670791530,25ptp4l61690irq/136-eno1-TxRx-019:05:223
3208169995344,6cyclictest3237239-21latency_hist20:40:011
3208169995331,14cyclictest271-21systemd-journal21:53:111
3208166995343,7cyclictest502-21rs:main0
3208166995343,7cyclictest502-21rs:main0
3208169995246,4cyclictest3295849-21kworker/u8:0+flush-179:000:15:221
3208166995241,8cyclictest3227425-21latency_hist20:10:010
3208169995045,4cyclictest3292085-21kworker/u8:1+flush-179:023:30:271
3208166995042,6cyclictest3279622-21kworker/u8:1+flush-179:023:15:000
3208169994941,6cyclictest271-21systemd-journal20:54:551
3208169994840,7cyclictest3261396-21kworker/u8:2+flush-179:022:40:011
320816999478,11cyclictest723-21Xorg22:22:581
3208169994739,6cyclictest271-21systemd-journal19:46:121
3208169994738,7cyclictest3264781-21/usr/sbin/munin22:05:231
3208169994732,14cyclictest0-21swapper/100:24:591
3208169994731,14cyclictest3243658-21sort21:00:001
320816999470,44cyclictest0-21swapper/119:20:141
3208166994738,7cyclictest271-21systemd-journal20:25:000
3208166994735,10cyclictest3221320-21apt-get19:50:120
3208166994735,10cyclictest3221320-21apt-get19:50:120
320817599460,44cyclictest3284426-21turbostat23:05:013
3208169994643,2cyclictest723-21Xorg21:25:591
3208169994641,4cyclictest3219505-21kworker/u8:3+flush-179:019:55:241
3208169994639,6cyclictest3226100-21apt-get20:05:171
320816999460,44cyclictest3237262-21turbostat20:44:561
320816999450,42cyclictest723-21Xorg21:17:581
320816999450,2cyclictest0-21swapper/119:37:591
3208169994441,2cyclictest723-21Xorg19:30:591
3208169994435,7cyclictest3303877-21latency_hist00:05:011
3208169994435,7cyclictest3287618-21kworker/u8:3+flush-179:000:05:211
3208169994339,3cyclictest3225820-21kworker/u8:1+flush-179:020:10:261
3208175994236,5cyclictest3240420-21turbostat20:49:593
3208171994238,2cyclictest0-21swapper/223:08:592
3208169994237,4cyclictest3279622-21kworker/u8:1+flush-179:023:00:251
3208166994239,2cyclictest723-21Xorg23:34:590
3208175994135,5cyclictest3293961-21turbostat23:35:003
320817199410,1cyclictest0-21swapper/220:50:002
3208169994137,3cyclictest3264582-21systemd-run22:05:001
3208169994137,3cyclictest3264582-21systemd-run22:05:001
3208169994131,8cyclictest3305715-21apt-get00:10:141
320816999411,38cyclictest3295829-21cut23:40:001
320816999411,38cyclictest3295829-21cut23:40:001
3208175994034,4cyclictest3299046-21turbostat23:50:003
3208175994033,5cyclictest3256584-21turbostat21:40:003
320817599400,39cyclictest3281231-21turbostat22:54:593
320817599400,38cyclictest3289186-21turbostat23:24:543
320817599400,38cyclictest3245210-21turbostat21:05:013
3208169994034,5cyclictest3295849-21kworker/u8:0+flush-179:023:45:241
3208169994030,6cyclictest3237279-21kworker/u8:1+flush-179:021:05:011
3208166994035,3cyclictest3284418-21munin-plugin-st23:05:000
320816699401,3cyclictest3224248-21idleruntime-cro20:00:000
320816699401,36cyclictest3299296-21apt-get23:50:130
202199400,4rtkit-daemon2020-21rtkit-daemon00:32:571
3208175993934,4cyclictest3253416-21turbostat21:30:013
3208175993934,4cyclictest3227430-21turbostat20:10:003
320817599390,4cyclictest3289186-21turbostat23:20:003
3208171993935,2cyclictest0-21swapper/221:04:552
320817199390,36cyclictest0-21swapper/222:05:002
320817199390,36cyclictest0-21swapper/222:04:592
3208169993935,3cyclictest1-21systemd21:25:011
3208169993932,5cyclictest3266134-21turbostat22:14:551
3208166993934,3cyclictest3313424-21awk00:35:000
3208166993911,14cyclictest3245195-21latency_hist21:05:000
320816699390,37cyclictest723-21Xorg21:51:000
3208175993834,3cyclictest3303890-21turbostat00:04:593
320817199380,36cyclictest0-21swapper/220:42:592
320817199380,35cyclictest0-21swapper/200:34:562
3208169993835,2cyclictest0-21swapper/120:19:591
3208169993828,8cyclictest3234311-21apt-get20:30:141
3208169993823,14cyclictest0-21swapper/121:31:581
3208166993835,2cyclictest3209676-21perf19:15:000
3208166993829,6cyclictest3212855-21latency_hist19:25:010
320816699381,9cyclictest3278308-21sshd22:45:140
320816699380,37cyclictest0-21swapper/022:45:000
320816699380,37cyclictest0-21swapper/000:21:590
3208175993735,1cyclictest0-21swapper/321:41:583
3208175993731,5cyclictest3264552-21turbostat22:05:003
3208175993731,5cyclictest3264552-21turbostat22:04:593
3208171993733,2cyclictest0-21swapper/221:53:592
320817199370,35cyclictest0-21swapper/221:20:002
320817199370,34cyclictest0-21swapper/219:20:002
320817199370,1cyclictest0-21swapper/219:59:592
3208169993731,4cyclictest3281455-21kworker/u8:0+flush-179:022:55:261
3208169993728,7cyclictest723-21Xorg19:50:591
3208169993728,7cyclictest723-21Xorg19:50:591
320816999372,30cyclictest0-21swapper/121:08:591
320816999371,3cyclictest0-21swapper/122:50:011
3208166993735,1cyclictest0-21swapper/023:49:550
3208166993734,2cyclictest723-21Xorg20:33:590
3208166993732,4cyclictest3211354-21latency_hist19:20:020
3208166993727,8cyclictest271-21systemd-journal21:09:560
3208166993723,11cyclictest3239086-21apt-get20:45:150
3208175993634,1cyclictest0-21swapper/320:32:593
3208175993634,1cyclictest0-21swapper/300:35:593
3208175993634,1cyclictest0-21swapper/300:24:553
3208175993633,2cyclictest0-21swapper/322:48:593
3208175993633,2cyclictest0-21swapper/321:31:593
3208175993633,2cyclictest0-21swapper/321:13:593
3208175993631,4cyclictest3234066-21turbostat20:29:593
320817599360,35cyclictest0-21swapper/323:12:593
320817599360,35cyclictest0-21swapper/319:29:593
320817599360,35cyclictest0-21swapper/319:17:593
320817599360,35cyclictest0-21swapper/300:28:003
320817599360,1cyclictest0-21swapper/323:09:553
3208171993633,2cyclictest0-21swapper/219:51:592
3208171993633,2cyclictest0-21swapper/219:51:592
320817199360,1cyclictest0-21swapper/221:31:592
320817199360,1cyclictest0-21swapper/220:30:002
320817199360,1cyclictest0-21swapper/220:23:002
320817199360,1cyclictest0-21swapper/219:30:002
320816999362,32cyclictest3310670-21cpuspeed_turbos00:25:141
320816999362,28cyclictest723-21Xorg21:50:001
320816999362,27cyclictest723-21Xorg23:22:581
320816999361,27cyclictest3286255-21apt-get23:10:121
3208175993533,1cyclictest0-21swapper/300:34:553
3208175993532,2cyclictest0-21swapper/319:59:593
3208175993529,4cyclictest3249966-21turbostat21:20:003
320817599350,1cyclictest0-21swapper/322:36:003
3208171993533,1cyclictest0-21swapper/223:15:002
3208171993532,2cyclictest0-21swapper/223:53:592
3208171993532,2cyclictest0-21swapper/223:48:592
3208171993532,2cyclictest0-21swapper/222:57:592
3208171993532,2cyclictest0-21swapper/222:47:002
3208171993532,2cyclictest0-21swapper/221:47:592
3208171993532,2cyclictest0-21swapper/219:44:552
3208171993532,2cyclictest0-21swapper/200:04:592
320816999353,26cyclictest723-21Xorg21:42:591
3208169993525,7cyclictest3209659-21latency_hist19:15:021
320816999352,26cyclictest723-21Xorg21:55:581
320816999352,26cyclictest723-21Xorg20:26:591
3208169993516,18cyclictest0-21swapper/122:40:171
3208166993530,4cyclictest3261396-21kworker/u8:2+flush-179:022:10:000
3208175993432,1cyclictest0-21swapper/320:23:003
3208171993432,1cyclictest0-21swapper/221:09:592
3208171993432,1cyclictest0-21swapper/219:40:002
3208171993431,2cyclictest0-21swapper/221:10:582
320816999344,22cyclictest723-21Xorg22:35:001
320816999342,27cyclictest0-21swapper/123:44:001
320816699343,16cyclictest3216523-21/usr/sbin/munin19:35:180
3208166993429,4cyclictest3249929-21kworker/u8:1+flush-179:021:40:260
3208166993428,4cyclictest3242203-21latency_hist20:55:010
3208166993427,4cyclictest1-21systemd23:06:140
3208166993426,7cyclictest3219482-21munin-plugin-st19:45:000
3208166993425,7cyclictest3229268-21apt-get20:15:130
3208166993425,7cyclictest271-21systemd-journal20:39:550
320816699340,32cyclictest723-21Xorg20:29:580
3208171993331,1cyclictest0-21swapper/220:40:012
3208171993331,1cyclictest0-21swapper/200:10:002
3208171993330,2cyclictest0-21swapper/222:36:582
3208171993330,2cyclictest0-21swapper/220:34:542
3208171993329,3cyclictest0-21swapper/220:06:592
3208171993329,2cyclictest0-21swapper/219:49:552
320816999332,26cyclictest0-21swapper/121:35:591
320816999331,29cyclictest3208163-21cyclictest19:10:011
320816999331,29cyclictest3208163-21cyclictest19:10:011
3208166993330,2cyclictest3256631-21kworker/u8:0+flush-179:021:47:460
3208166993328,4cyclictest723-21Xorg21:34:000
3208166993328,3cyclictest0-21swapper/022:54:590
3208166993327,5cyclictest3305318-21unixbench-2d00:05:260
3208166993323,8cyclictest3289447-21apt-get23:20:110
320816699332,2cyclictest3252073-21apt-get21:25:160
3208166993319,5cyclictest271-21systemd-journal00:15:000
3208175993230,1cyclictest0-21swapper/320:42:593
3208171993230,1cyclictest0-21swapper/223:25:012
3208171993230,1cyclictest0-21swapper/222:19:592
3208171993229,2cyclictest0-21swapper/223:39:562
3208171993229,2cyclictest0-21swapper/223:39:552
3208171993229,2cyclictest0-21swapper/220:18:592
3208171993229,2cyclictest0-21swapper/200:11:592
3208166993224,7cyclictest3273586-21fschecks_count22:30:160
320816699321,3cyclictest0-21swapper/019:45:140
320816699320,30cyclictest723-21Xorg20:02:590
3208175993128,2cyclictest0-21swapper/300:13:593
3208175993125,4cyclictest3217883-21turbostat19:40:003
3208171993129,1cyclictest0-21swapper/222:00:012
3208171993129,1cyclictest0-21swapper/221:40:002
3208171993128,2cyclictest0-21swapper/200:18:592
320816999311,27cyclictest0-21swapper/122:16:001
3208175993028,1cyclictest0-21swapper/322:08:593
3208175993025,4cyclictest3217883-21turbostat19:44:553
3208175993024,5cyclictest3305473-21turbostat00:10:003
320817599300,2cyclictest3292372-21turbostat23:30:003
320817599300,29cyclictest0-21swapper/323:35:043
320817599300,29cyclictest0-21swapper/323:35:033
320817599300,27cyclictest4672291phc2sys22:40:153
3208171993028,1cyclictest0-21swapper/222:45:002
3208171993027,2cyclictest0-21swapper/221:40:162
3208171993027,2cyclictest0-21swapper/221:25:152
3208171993027,2cyclictest0-21swapper/219:25:012
320817199300,1cyclictest0-21swapper/200:24:552
320816699303,12cyclictest3227968-21diskstats20:10:160
3208166993025,4cyclictest3314861-21unixbench-2d00:35:250
4670791290,7ptp4l441ktimers/319:55:003
4670791290,7ptp4l441ktimers/319:55:003
3208175992923,5cyclictest3237262-21turbostat20:40:013
320817599290,28cyclictest0-21swapper/319:14:593
320817599290,27cyclictest0-21swapper/320:55:173
3208171992926,2cyclictest0-21swapper/223:58:002
320817199290,2cyclictest0-21swapper/200:38:592
320817199290,27cyclictest0-21swapper/222:20:142
320817199290,27cyclictest0-21swapper/222:10:022
320817199290,27cyclictest0-21swapper/221:20:132
320816699295,11cyclictest3255979-21ntp_192.168.11421:35:200
4670791280,7ptp4l441ktimers/320:55:003
4670791280,7ptp4l441ktimers/300:20:013
3208175992825,2cyclictest0-21swapper/322:58:003
3208175992825,2cyclictest0-21swapper/321:09:593
320817599280,26cyclictest0-21swapper/322:10:123
3208171992825,2cyclictest0-21swapper/223:25:142
3208171992825,2cyclictest0-21swapper/219:30:152
320817199280,2cyclictest0-21swapper/222:05:132
3208175992725,1cyclictest0-21swapper/321:45:153
3208175992725,1cyclictest0-21swapper/319:30:153
3208175992725,1cyclictest0-21swapper/319:20:143
3208175992721,4cyclictest3262954-21turbostat22:00:003
320817599270,26cyclictest0-21swapper/322:20:153
320817599270,1cyclictest0-21swapper/321:20:123
3208171992723,2cyclictest0-21swapper/223:32:592
4670791260,6ptp4l441ktimers/320:20:013
4670791260,5ptp4l441ktimers/322:20:003
3208175992624,1cyclictest0-21swapper/323:50:593
3208175992624,1cyclictest0-21swapper/321:53:593
3208175992624,1cyclictest0-21swapper/320:11:593
320817199260,2cyclictest0-21swapper/219:10:192
4670791250,5ptp4l61690irq/136-eno1-TxRx-022:30:013
4670791250,5ptp4l441ktimers/319:50:013
4670791250,4ptp4l61690irq/136-eno1-TxRx-020:04:563
3208171992522,2cyclictest0-21swapper/222:30:152
3208171992522,2cyclictest0-21swapper/220:51:592
4670791240,5ptp4l441ktimers/323:59:553
4670791240,4ptp4l61690irq/136-eno1-TxRx-023:45:003
3208171992422,1cyclictest0-21swapper/223:44:592
3208171992320,2cyclictest0-21swapper/220:01:592
4670791220,2ptp4l441ktimers/322:34:543
3208171992220,1cyclictest0-21swapper/223:05:002
3208171992220,1cyclictest0-21swapper/200:29:552
3208171992219,2cyclictest0-21swapper/222:54:552
3208171991916,2cyclictest0-21swapper/223:15:142
3208171991815,2cyclictest0-21swapper/220:13:592
3208171991814,2cyclictest3243616-21turbostat21:00:002
320816699168,4cyclictest3208174-21sed19:10:010
320816699168,4cyclictest3208174-21sed19:10:010
3208171991512,2cyclictest0-21swapper/222:25:032
32081719930,2cyclictest0-21swapper/219:10:012
32081719930,2cyclictest0-21swapper/219:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional