You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-17 - 11:14
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot6.osadl.org (updated Mon Nov 17, 2025 00:44:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
29950729910088,9cyclictest3070895-21cat23:05:011
29950699910088,9cyclictest268544-21cups-browsed00:00:020
2995069999690,4cyclictest3030924-21kworker/u8:1+flush-179:021:00:260
2995069998764,20cyclictest271-21systemd-journal20:40:000
2995072998475,7cyclictest3093684-21latency_hist00:15:011
2995072998070,7cyclictest3017504-21latency_hist20:20:011
2995072997971,5cyclictest3030924-21kworker/u8:1+flush-179:021:05:261
2995072997560,13cyclictest3094988-21kworker/u8:2+events_unbound00:30:301
2995069997263,6cyclictest3059667-21/usr/sbin/munin22:30:280
2995069996933,27cyclictest3009628-21latency_hist19:55:020
2995072996459,3cyclictest723-21Xorg21:50:591
2995072996459,3cyclictest3059404-21kworker/u8:0+flush-179:022:30:181
2995072996255,5cyclictest3078247-21kworker/u8:1+flush-179:000:00:261
2995072996254,6cyclictest3028648-21latency_hist20:55:011
2995072996254,6cyclictest3023848-21kworker/u8:0+flush-179:021:30:001
2995069996250,10cyclictest271-21systemd-journal19:34:560
2995069996229,25cyclictest3056245-21turbostat22:24:550
2995072996055,3cyclictest3040577-21apt-get21:30:131
2995069995951,6cyclictest3037434-21diskstats21:20:190
2995069995854,2cyclictest0-21swapper/022:10:590
2995069995850,6cyclictest3084088-21apt-get23:45:150
2995072995737,18cyclictest3020951-21apt-get20:30:131
2995069995749,5cyclictest3016148-21kworker/u8:2+flush-179:020:25:270
2995069995749,5cyclictest3016148-21kworker/u8:2+flush-179:020:25:260
2995076995654,1cyclictest0-21swapper/319:48:593
2995072995651,4cyclictest3074020-21apt-get23:15:031
2995072995647,7cyclictest3000240-21/usr/sbin/munin19:25:241
2995069995650,3cyclictest3023848-21kworker/u8:0+flush-179:021:05:260
2995069995647,6cyclictest3001589-21latency_hist19:30:010
2995069995646,7cyclictest3059427-21latency_hist22:30:010
2995072995552,2cyclictest3060996-21kworker/u8:3+events_unbound22:45:011
2995072995552,2cyclictest3060996-21kworker/u8:3+events_unbound22:45:001
2995069995547,6cyclictest2999629-21kworker/u8:1+events_unbound19:40:000
2995069995543,9cyclictest3036618-21latency_hist21:20:010
2995072995450,3cyclictest3088790-21dpkg00:00:011
2995072995444,8cyclictest3067621-21/usr/sbin/munin22:55:271
2995073995350,2cyclictest0-21swapper/200:36:002
299507299531,31cyclictest0-21swapper/121:45:161
299507299530,34cyclictest3095551-21apt-get00:20:181
299507299530,34cyclictest3095551-21apt-get00:20:171
2995072995246,4cyclictest3053046-21kworker/u8:2+events_unbound22:20:011
2995072995242,8cyclictest3077465-21latency_hist23:25:001
2995069995246,4cyclictest3045138-21kworker/u8:0+flush-179:022:10:010
2995069995242,8cyclictest3006627-21apt-get19:45:130
2995069995233,17cyclictest3098472-21munin-run00:30:000
2995072995145,4cyclictest3072456-21kworker/u8:3+events_unbound23:15:011
2995069995133,17cyclictest3015942-21systemd-run20:15:010
2995069995127,22cyclictest3076081-21latency_hist23:20:010
2995076995047,2cyclictest0-21swapper/319:32:593
2995069995041,7cyclictest3062660-21kworker/u8:2+events_unbound23:40:010
2995076994947,1cyclictest0-21swapper/300:21:593
2995076994947,1cyclictest0-21swapper/300:21:593
2995072994943,4cyclictest3023848-21kworker/u8:0+events_unbound21:25:001
2995072994932,2cyclictest723-21Xorg22:46:581
4670791480,13ptp4l441ktimers/319:10:003
2995073994819,1cyclictest0-21swapper/219:22:592
299507299481,45cyclictest3051474-21munin-run22:05:001
2995069994843,3cyclictest979226-21gdbus00:04:000
2995069994836,9cyclictest2996556-21apt-get19:15:000
2995073994643,2cyclictest0-21swapper/222:11:592
2995072994640,5cyclictest3062660-21kworker/u8:2+events_unbound23:10:001
2995072994637,6cyclictest271-21systemd-journal20:10:011
2995072994636,8cyclictest3022537-21apt-get20:35:141
2995069994641,3cyclictest3017559-21systemd-run20:20:000
2995069994632,12cyclictest0-21swapper/000:20:170
2995069994632,12cyclictest0-21swapper/000:20:170
299506999460,29cyclictest723-21Xorg21:40:580
2995072994538,5cyclictest3036615-21apt-get21:20:001
2995072994537,6cyclictest271-21systemd-journal22:06:481
2995072994536,8cyclictest3005043-21apt-get19:40:151
2995069994526,16cyclictest0-21swapper/021:35:170
2995072994436,6cyclictest3060996-21kworker/u8:3+flush-179:022:40:011
2995069994441,2cyclictest0-21swapper/020:00:590
2995069994426,16cyclictest0-21swapper/019:40:170
2995069994419,22cyclictest3091175-21fschecks_time00:05:180
2995072994333,7cyclictest271-21systemd-journal19:56:471
299507299432,39cyclictest3048285-21turbostat21:59:551
2995069994338,4cyclictest3062660-21kworker/u8:2+flush-179:023:25:250
2995069994327,14cyclictest3045358-21apt-get21:45:150
299507299420,41cyclictest0-21swapper/100:09:561
2995069994237,3cyclictest3045138-21kworker/u8:0+flush-179:022:00:190
2995069994236,5cyclictest3034997-21kworker/u8:1+flush-179:021:15:000
2995069994236,4cyclictest3028109-21kworker/u8:2+flush-179:020:50:260
299507699410,39cyclictest3065809-21turbostat22:50:003
299507699410,39cyclictest3009550-21turbostat19:55:003
299507699410,39cyclictest3000017-21turbostat19:25:003
299507299410,39cyclictest723-21Xorg20:03:591
2995069994138,2cyclictest0-21swapper/021:50:590
2995069994137,3cyclictest723-21Xorg19:59:590
2995069994119,19cyclictest0-21swapper/019:15:010
299507699400,5cyclictest3028656-21turbostat20:55:003
299507699400,4cyclictest3012747-21turbostat20:04:593
299507699400,33cyclictest2996561-21turbostat19:14:593
2995073994037,2cyclictest0-21swapper/223:44:002
2995072994034,4cyclictest3099987-21kworker/u8:0+flush-179:000:40:001
299507299401,37cyclictest723-21Xorg22:13:591
2995069994034,4cyclictest3099987-21kworker/u8:0+events_unbound00:35:000
2995069994031,7cyclictest3023848-21kworker/u8:0+flush-179:021:25:240
2995069994029,9cyclictest3017778-21apt-get20:20:130
2995069994021,17cyclictest3083093-21latency23:40:180
299506999400,38cyclictest3095307-21idleruntime-cro00:20:000
299507699390,38cyclictest3062639-21turbostat22:40:003
2995072993936,2cyclictest723-21Xorg23:45:591
2995072993933,4cyclictest3011116-21kworker/u8:1+events_unbound20:15:011
299507299392,31cyclictest3024148-21apt-get20:40:161
299507299391,8cyclictest3082505-21sendmail-mta23:40:131
299507299391,33cyclictest2998672-21apt-get19:20:121
299507299390,38cyclictest0-21swapper/120:22:591
2995069993935,3cyclictest3041933-21perf21:35:000
2995069993932,5cyclictest3062590-21kworker/u8:1+flush-179:022:55:210
2995069993924,14cyclictest0-21swapper/022:44:000
2995069993924,14cyclictest0-21swapper/022:44:000
2995069993919,18cyclictest0-21swapper/020:45:240
299506999390,2cyclictest723-21Xorg20:31:590
299506999390,2cyclictest0-21swapper/023:06:580
202199390,4rtkit-daemon2005-21lightdm-gtk-gre19:13:461
2995069993835,2cyclictest0-21swapper/023:25:000
2995073993734,2cyclictest0-21swapper/223:50:582
2995073993734,2cyclictest0-21swapper/222:02:002
299507399370,35cyclictest0-21swapper/222:40:592
299507399370,35cyclictest0-21swapper/222:40:582
299507399370,35cyclictest0-21swapper/221:49:552
2995072993732,4cyclictest2999629-21kworker/u8:1+flush-179:019:45:161
299507299373,12cyclictest2005-21lightdm-gtk-gre22:26:591
299507299370,36cyclictest0-21swapper/123:54:561
2995069993717,18cyclictest3024499-21fschecks_time20:40:180
2995076993634,1cyclictest0-21swapper/322:22:593
2995076993633,2cyclictest0-21swapper/323:00:003
2995076993633,2cyclictest0-21swapper/319:40:593
299507699360,35cyclictest0-21swapper/320:49:553
299507699360,35cyclictest0-21swapper/320:30:593
2995073993634,1cyclictest0-21swapper/223:05:012
2995073993633,2cyclictest0-21swapper/222:23:582
2995073993633,2cyclictest0-21swapper/220:55:592
2995072993634,1cyclictest0-21swapper/119:54:551
299507299361,32cyclictest3042164-21apt-get21:35:171
2995069993631,4cyclictest3023848-21kworker/u8:0+flush-179:020:55:240
2995069993617,17cyclictest0-21swapper/022:15:230
299506999361,6cyclictest0-21swapper/019:22:580
299506999361,5cyclictest0-21swapper/000:13:000
2995073993532,2cyclictest0-21swapper/219:19:542
299507399350,1cyclictest0-21swapper/220:02:592
2995072993526,6cyclictest0-21swapper/100:15:151
2995069993531,3cyclictest2005-21lightdm-gtk-gre21:57:590
2995069993529,5cyclictest2994990-21kworker/u8:3+events_unbound19:10:010
2995069993515,18cyclictest0-21swapper/022:50:200
2995069993514,20cyclictest3062586-21sshd22:37:270
2995069993513,20cyclictest0-21swapper/020:10:000
2995076993432,1cyclictest0-21swapper/321:14:593
2995073993431,2cyclictest0-21swapper/220:40:592
2995072993427,5cyclictest3020701-21munin-plugin-st20:30:001
2995072993427,5cyclictest3020701-21munin-plugin-st20:30:001
2995069993417,15cyclictest0-21swapper/000:35:290
2995069993416,16cyclictest0-21swapper/023:50:120
2995076993331,1cyclictest0-21swapper/323:40:593
2995076993331,1cyclictest0-21swapper/319:17:003
2995073993331,1cyclictest0-21swapper/223:10:002
2995073993331,1cyclictest0-21swapper/220:14:592
2995073993331,1cyclictest0-21swapper/200:00:002
2995072993330,2cyclictest0-21swapper/121:42:591
2995072993328,4cyclictest3077478-21turbostat23:29:561
2995072993325,6cyclictest271-21systemd-journal20:47:091
299507299330,32cyclictest723-21Xorg21:01:591
299507299330,32cyclictest723-21Xorg19:18:591
2995069993330,2cyclictest0-21swapper/023:10:590
2995069993327,4cyclictest3065030-21latency22:45:180
299506999331,16cyclictest723-21Xorg23:03:590
2995076993230,1cyclictest0-21swapper/322:03:583
2995076993229,2cyclictest0-21swapper/321:53:593
299507699320,2cyclictest0-21swapper/320:43:593
2995073993229,2cyclictest0-21swapper/220:45:592
2995072993228,2cyclictest0-21swapper/100:25:131
299507299321,29cyclictest3080892-21apt-get23:35:141
299506999328,4cyclictest3080240-21basename23:30:260
2995076993128,2cyclictest0-21swapper/321:57:593
2995076993128,2cyclictest0-21swapper/300:26:003
2995073993129,1cyclictest0-21swapper/223:48:592
2995073993128,2cyclictest0-21swapper/223:39:592
2995073993128,2cyclictest0-21swapper/222:00:002
2995072993123,6cyclictest0-21swapper/122:54:591
2995076993028,1cyclictest0-21swapper/320:17:593
2995073993027,2cyclictest0-21swapper/222:50:152
2995073993027,2cyclictest0-21swapper/222:30:002
2995073993027,2cyclictest0-21swapper/221:51:592
2995073993027,2cyclictest0-21swapper/221:42:592
299507399300,28cyclictest0-21swapper/200:20:002
299507699291,2cyclictest0-21swapper/323:15:013
299507699290,28cyclictest0-21swapper/319:35:163
299507699290,27cyclictest0-21swapper/323:30:163
299507699290,27cyclictest0-21swapper/323:10:163
2995073992926,2cyclictest0-21swapper/221:20:172
2995073992926,2cyclictest0-21swapper/221:05:152
2995073992926,2cyclictest0-21swapper/220:15:142
2995073992926,2cyclictest0-21swapper/219:39:592
2995073992926,2cyclictest0-21swapper/219:13:592
299507399290,28cyclictest0-21swapper/220:25:152
299507399290,28cyclictest0-21swapper/220:25:152
299507399290,27cyclictest0-21swapper/223:25:152
299507399290,27cyclictest0-21swapper/222:55:152
299507399290,27cyclictest0-21swapper/221:00:152
299507399290,27cyclictest0-21swapper/200:30:192
299507399290,27cyclictest0-21swapper/200:10:162
299507399290,1cyclictest0-21swapper/200:00:152
2995072992926,2cyclictest3004784-21munin-plugin-st19:39:591
2995072992925,3cyclictest723-21Xorg23:31:591
2995076992826,1cyclictest0-21swapper/321:25:023
2995076992825,2cyclictest0-21swapper/323:05:023
2995076992825,2cyclictest0-21swapper/322:13:003
2995076992825,2cyclictest0-21swapper/300:30:583
299507699280,26cyclictest0-21swapper/323:05:013
299507699280,26cyclictest0-21swapper/321:40:143
299507699280,26cyclictest0-21swapper/321:20:033
2995073992825,2cyclictest0-21swapper/221:37:592
2995073992825,2cyclictest0-21swapper/220:21:592
2995073992825,2cyclictest0-21swapper/219:30:152
299507399280,2cyclictest0-21swapper/223:20:152
299507399280,27cyclictest0-21swapper/221:25:012
299507399280,26cyclictest0-21swapper/222:15:142
2995072992826,1cyclictest0-21swapper/122:21:591
2995072992824,3cyclictest0-21swapper/119:30:151
2995076992725,1cyclictest0-21swapper/319:25:133
2995076992721,4cyclictest3059439-21turbostat22:30:003
299507699270,26cyclictest0-21swapper/321:47:593
299507699270,26cyclictest0-21swapper/320:39:593
299507699270,26cyclictest0-21swapper/300:05:153
2995073992724,2cyclictest0-21swapper/222:48:592
2995073992724,2cyclictest0-21swapper/219:55:592
299507399270,2cyclictest0-21swapper/222:35:152
299507399270,25cyclictest0-21swapper/200:29:542
2995076992623,2cyclictest0-21swapper/319:55:593
299507699260,2cyclictest0-21swapper/322:08:593
299507699260,25cyclictest0-21swapper/300:00:003
2995073992623,2cyclictest0-21swapper/200:23:002
2995073992623,2cyclictest0-21swapper/200:22:592
299507399260,2cyclictest0-21swapper/221:13:592
299507399260,24cyclictest0-21swapper/220:31:592
2995072992619,5cyclictest3033080-21kworker/u8:2+events_unbound21:10:271
4670791250,3ptp4l441ktimers/323:40:013
2995076992522,2cyclictest0-21swapper/300:10:173
299507699250,24cyclictest0-21swapper/300:18:593
299507699250,1cyclictest0-21swapper/323:24:003
2995073992523,1cyclictest0-21swapper/219:27:592
2995073992523,1cyclictest0-21swapper/200:09:592
2995073992522,2cyclictest0-21swapper/223:15:152
299507399250,2cyclictest0-21swapper/221:18:592
2995072992523,1cyclictest0-21swapper/120:59:551
4670791240,4ptp4l441ktimers/321:19:563
4670791240,3ptp4l441ktimers/323:50:003
4670791240,3ptp4l441ktimers/322:19:593
4670791240,3ptp4l441ktimers/321:35:013
2995076992422,1cyclictest0-21swapper/322:42:003
2995076992422,1cyclictest0-21swapper/322:42:003
2995076992422,1cyclictest0-21swapper/320:56:593
2995073992422,1cyclictest0-21swapper/220:54:592
2995073992421,2cyclictest0-21swapper/222:34:552
4670791230,5ptp4l61690irq/136-eno1-TxRx-020:24:553
4670791230,3ptp4l61690irq/136-eno1-TxRx-021:09:553
4670791230,3ptp4l61690irq/136-eno1-TxRx-021:04:553
4670791230,3ptp4l61690irq/136-eno1-TxRx-000:40:003
4670791230,3ptp4l441ktimers/323:29:553
4670791230,3ptp4l441ktimers/322:54:553
4670791230,3ptp4l441ktimers/322:34:553
4670791230,3ptp4l441ktimers/320:10:003
2995076992321,1cyclictest0-21swapper/323:52:003
2995076992321,1cyclictest0-21swapper/320:30:003
2995076992321,1cyclictest0-21swapper/320:29:593
2995076992320,2cyclictest0-21swapper/320:10:593
2995076992320,2cyclictest0-21swapper/300:03:583
2995073992321,1cyclictest0-21swapper/223:31:582
2995076992219,2cyclictest0-21swapper/321:33:593
2995073992118,2cyclictest0-21swapper/220:38:592
2995073992118,2cyclictest0-21swapper/219:45:592
2995073992018,1cyclictest0-21swapper/219:53:592
2995073991917,1cyclictest0-21swapper/222:06:582
2995072991814,3cyclictest2995088-21latency_hist19:10:021
2995073991714,2cyclictest0-21swapper/223:10:592
2995073991613,2cyclictest0-21swapper/220:06:592
2995073991210,1cyclictest0-21swapper/221:34:592
29950739974,2cyclictest0-21swapper/219:41:252
29950739930,2cyclictest0-21swapper/219:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional