You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-27 - 06:15
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot6.osadl.org (updated Sat Dec 27, 2025 00:44:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
32673509911388,8cyclictest3281837-21latency_hist19:55:020
3267350998575,7cyclictest3308900-21latency_hist21:20:020
3267350998575,7cyclictest3308900-21latency_hist21:20:010
3267354998473,8cyclictest3273623-21latency_hist19:30:021
3267350998377,5cyclictest3350547-21kworker/u8:0+events_unbound23:50:250
3267350998174,5cyclictest3297791-21kworker/u8:3+events_unbound20:50:260
3267350998174,5cyclictest3297791-21kworker/u8:3+events_unbound20:50:260
3267350998073,5cyclictest3346449-21kworker/u8:1+events_unbound23:45:020
3267350998073,4cyclictest3320104-21kworker/u8:2+events_unbound22:00:010
3267350997969,8cyclictest3304374-21/usr/sbin/munin21:05:250
3267354997870,6cyclictest3350804-21/usr/sbin/munin23:30:231
3267354997861,14cyclictest3294490-21kworker/u8:2+flush-179:020:44:591
3267350997770,4cyclictest3320104-21kworker/u8:2+events_unbound22:10:130
3267350997570,4cyclictest3278368-21kworker/u8:0+flush-179:020:45:230
3267350997569,4cyclictest3350547-21kworker/u8:0+flush-179:023:55:170
3267350997568,5cyclictest3297791-21kworker/u8:3+flush-179:021:20:140
3267354997368,4cyclictest3247536-21kworker/u8:3+flush-179:019:30:231
3267350997367,5cyclictest3264976-21kworker/u8:0+flush-179:019:20:230
3267354997263,6cyclictest3307312-21latency_hist21:15:011
3267350997266,4cyclictest3333032-21kworker/u8:2+flush-179:022:35:180
3267350997265,5cyclictest3348981-21latency_hist23:25:020
3267350997265,5cyclictest3348981-21latency_hist23:25:010
3267350997166,4cyclictest3319618-21kworker/u8:0+events_unbound21:50:220
3267350997165,4cyclictest3264976-21kworker/u8:0+flush-179:019:10:240
3267350997158,10cyclictest3283987-21diskstats20:00:150
3267354997063,5cyclictest3283480-21kworker/u8:1+flush-179:020:05:221
3267354997062,6cyclictest3315279-21latency_hist21:40:011
3267350997060,8cyclictest3338064-21/usr/sbin/munin22:50:230
3267350996963,4cyclictest191rcuc/023:04:590
3267350996862,4cyclictest3368425-21kworker/u8:3+events_unbound00:35:140
3267350996862,4cyclictest3368425-21kworker/u8:3+events_unbound00:35:130
3267350996861,5cyclictest3346449-21kworker/u8:1+flush-179:023:15:240
3267354996760,5cyclictest3360438-21kworker/u8:1+events_unbound00:15:201
3267350996761,5cyclictest3368425-21kworker/u8:3+flush-179:000:25:010
3267350996660,4cyclictest3354810-21kworker/u8:2+flush-179:023:45:220
3267350996559,4cyclictest3247536-21kworker/u8:3+flush-179:019:20:000
3267350996456,5cyclictest3274692-21kworker/u8:1+events_unbound19:35:010
3267350996453,8cyclictest3275260-21kworker/u8:2+flush-179:019:40:200
3267354996358,3cyclictest3360438-21kworker/u8:1+flush-179:000:10:261
3267354996339,5cyclictest3267227-21latency_hist19:10:031
3267350996356,5cyclictest3328246-21kworker/u8:3+flush-179:022:25:250
3267354996254,6cyclictest3320104-21kworker/u8:2+events_unbound22:15:001
3267354996252,7cyclictest3305724-21latency_hist21:10:011
3267350996251,7cyclictest3290866-21kworker/u8:1+events_unbound20:25:140
3267354996154,5cyclictest3319618-21kworker/u8:0+flush-179:022:30:261
3267354996154,5cyclictest3319618-21kworker/u8:0+flush-179:022:30:261
3267354996056,2cyclictest3354810-21kworker/u8:2+events_unbound23:50:011
3267354996054,4cyclictest3275260-21kworker/u8:2+flush-179:019:40:001
3267350996054,4cyclictest3319618-21kworker/u8:0+events_unbound23:15:010
3267350996050,6cyclictest3338062-21kworker/u8:2+flush-179:023:00:010
3267354995948,9cyclictest3302807-21apt-get21:00:121
3267350995952,5cyclictest3307290-21kworker/u8:0+flush-179:021:30:220
3267350995852,4cyclictest3312329-21kworker/u8:1+flush-179:021:45:010
3267350995850,6cyclictest3360438-21kworker/u8:1+flush-179:000:20:000
3267350995850,6cyclictest3272580-21diskstats19:25:150
3267350995846,9cyclictest3275260-21kworker/u8:2+events_unbound20:00:000
3267354995751,4cyclictest3275260-21kworker/u8:2+flush-179:019:40:141
3267354995748,7cyclictest3278643-21/usr/sbin/munin19:45:261
3267354995748,7cyclictest266-21systemd-journal00:40:001
3267354995748,7cyclictest266-21systemd-journal00:40:001
3267354995747,8cyclictest3334909-21apt-get22:40:121
3267350995649,5cyclictest3360438-21kworker/u8:1+flush-179:000:05:230
3267350995646,7cyclictest3290866-21kworker/u8:1+flush-179:020:30:210
3267354995551,3cyclictest3278368-21kworker/u8:0+flush-179:020:55:261
3267354995537,5cyclictest3338062-21kworker/u8:2+flush-179:023:00:211
3267350995550,4cyclictest3319618-21kworker/u8:0+flush-179:023:05:220
3267350995548,5cyclictest3297791-21kworker/u8:3+flush-179:021:25:180
3267354995447,5cyclictest3368402-21latency_hist00:25:011
3267354995443,8cyclictest266-21systemd-journal23:15:161
3267350995448,4cyclictest3319618-21kworker/u8:0+events_unbound22:20:270
3267350995447,4cyclictest3294490-21kworker/u8:2+events_unbound20:40:010
3267350995445,5cyclictest3278368-21kworker/u8:0+flush-179:020:10:200
3267354995347,4cyclictest3278368-21kworker/u8:0+flush-179:020:35:001
3267350995348,4cyclictest3312329-21kworker/u8:1+flush-179:021:45:220
3267350995348,4cyclictest3283480-21kworker/u8:1+flush-179:020:05:200
3267354995141,8cyclictest3282092-21apt-get19:55:111
3267350995145,5cyclictest3286651-21kworker/u8:3+flush-179:020:20:010
3267350995145,5cyclictest3278368-21kworker/u8:0+flush-179:020:55:220
3267350995142,6cyclictest3368425-21kworker/u8:3+flush-179:000:25:210
3267354995045,4cyclictest3346449-21kworker/u8:1+flush-179:023:26:331
3267350995045,4cyclictest3278368-21kworker/u8:0+flush-179:020:40:200
3267350995045,3cyclictest3357431-21kworker/u8:3+events_unbound00:05:010
3267350994942,5cyclictest3297791-21kworker/u8:3+flush-179:021:35:240
3267350994941,5cyclictest3278368-21kworker/u8:0+events_unbound21:00:240
3267350994938,7cyclictest3347428-21kworker/u8:3+events_unbound23:35:180
3267350994842,4cyclictest3305703-21kworker/u8:1+flush-179:021:10:230
4577091470,13ptp4l61490irq/134-eno1-TxRx-019:09:563
3267354994742,4cyclictest3354810-21kworker/u8:2+events_unbound00:04:001
3267354994739,5cyclictest3320104-21kworker/u8:2+flush-179:022:15:261
3267354994637,6cyclictest3309700-21kworker/u8:1+flush-179:021:20:191
3267354994636,8cyclictest3332527-21kworker/u8:1+flush-179:022:45:251
3267350994643,1cyclictest151ktimers/023:35:000
3267350994636,7cyclictest3275260-21kworker/u8:2+events_unbound19:40:000
3267354994438,5cyclictest3264976-21kworker/u8:0+flush-179:019:25:001
3267354994437,5cyclictest3290866-21kworker/u8:1+events_unbound20:20:231
3267350994439,3cyclictest3333032-21kworker/u8:2+events_unbound22:35:000
3267350994439,3cyclictest3333032-21kworker/u8:2+events_unbound22:34:590
3267358994340,2cyclictest0-21swapper/320:15:593
3267358994334,6cyclictest3278421-21turbostat19:45:003
326735899430,3cyclictest3368412-21turbostat00:25:013
326735599430,41cyclictest0-21swapper/222:46:582
3267354994336,5cyclictest3369988-21latency_hist00:30:011
326735499432,40cyclictest3328279-21turbostat22:24:541
3267350994338,3cyclictest3368425-21kworker/u8:3+flush-179:000:35:000
3267350994338,3cyclictest3332527-21kworker/u8:1+events_unbound22:45:000
3267350994334,6cyclictest3350566-21latency_hist23:30:010
3267358994239,2cyclictest0-21swapper/323:29:003
326735899420,40cyclictest3315278-21turbostat21:40:003
3267354994236,4cyclictest3291639-21/usr/sbin/munin20:25:171
3267354994229,11cyclictest0-21swapper/122:00:001
326735499421,39cyclictest714-21Xorg21:40:591
326735099420,31cyclictest714-21Xorg22:47:580
3267354994133,5cyclictest3319618-21kworker/u8:0+flush-179:022:10:001
326735499411,38cyclictest714-21Xorg23:58:001
326735499411,31cyclictest714-21Xorg21:32:001
326735899400,5cyclictest3313688-21turbostat21:35:003
3267354994036,2cyclictest714-21Xorg20:14:591
3267354994035,4cyclictest3280147-21kworker/u8:3+flush-179:019:50:151
3267354994034,5cyclictest3333032-21kworker/u8:2+flush-179:022:35:221
3267354994034,5cyclictest3294490-21kworker/u8:2+flush-179:020:35:231
3267354994034,4cyclictest3297791-21kworker/u8:3+flush-179:021:15:221
3267354994034,4cyclictest3297791-21kworker/u8:3+flush-179:021:15:221
326735499401,37cyclictest714-21Xorg20:16:591
326735499400,38cyclictest3300944-21sh20:55:001
326735499400,38cyclictest3300944-21sh20:55:001
3267350994036,3cyclictest211irq_work/022:04:590
326735599390,30cyclictest0-21swapper/220:16:592
3267354993933,4cyclictest3346449-21kworker/u8:1+events_unbound23:40:231
3267358993832,4cyclictest3355365-21turbostat23:45:013
326735899380,37cyclictest3279994-21turbostat19:50:003
3267354993835,2cyclictest0-21swapper/100:34:561
3267354993831,5cyclictest3338062-21kworker/u8:2+flush-179:023:10:001
326735499380,37cyclictest0-21swapper/121:50:591
3267350993833,4cyclictest3319618-21kworker/u8:0+events_unbound22:15:210
3267350993832,5cyclictest3357431-21kworker/u8:3+events_unbound00:10:150
326735099380,37cyclictest0-21swapper/019:47:590
3267358993735,1cyclictest0-21swapper/322:33:593
3267358993735,1cyclictest0-21swapper/322:33:583
3267358993731,5cyclictest3286635-21turbostat20:10:003
3267355993734,2cyclictest0-21swapper/221:36:592
3267355993734,2cyclictest0-21swapper/221:02:002
3267355993734,2cyclictest0-21swapper/220:50:592
3267355993734,2cyclictest0-21swapper/220:50:592
3267355993733,2cyclictest0-21swapper/222:14:002
326735599370,35cyclictest0-21swapper/223:57:592
326735599370,35cyclictest0-21swapper/222:00:002
326735599370,34cyclictest0-21swapper/223:00:012
326735599370,1cyclictest0-21swapper/221:40:582
3267354993731,5cyclictest3329880-21turbostat22:29:551
3267354993731,5cyclictest3297791-21kworker/u8:3+flush-179:021:25:221
3267358993634,1cyclictest0-21swapper/320:35:593
3267358993634,1cyclictest0-21swapper/320:26:593
3267358993631,4cyclictest3310520-21turbostat21:25:003
326735899360,35cyclictest0-21swapper/323:13:003
3267355993634,1cyclictest0-21swapper/220:14:592
3267355993633,2cyclictest0-21swapper/221:23:592
3267355993633,2cyclictest0-21swapper/220:34:592
326735599360,2cyclictest0-21swapper/219:55:592
326735599360,1cyclictest0-21swapper/221:32:002
3267354993626,9cyclictest3360438-21kworker/u8:1+flush-179:000:10:001
326735099362,4cyclictest3323759-21apt-get22:05:150
3267358993533,1cyclictest0-21swapper/323:19:563
3267358993533,1cyclictest0-21swapper/320:48:593
3267358993533,1cyclictest0-21swapper/300:29:553
326735899350,1cyclictest0-21swapper/322:04:593
3267355993532,2cyclictest0-21swapper/222:22:582
3267354993527,6cyclictest3320104-21kworker/u8:2+flush-179:022:00:211
3267358993432,1cyclictest0-21swapper/322:09:563
3267358993432,1cyclictest0-21swapper/300:19:593
3267358993430,3cyclictest61390irq/133-eno123:54:553
3267355993431,2cyclictest0-21swapper/220:42:592
3267354993429,4cyclictest3340665-21switchtime22:55:231
3267354993429,3cyclictest3267510-21apt-get19:10:131
3267354993427,5cyclictest3275260-21kworker/u8:2+events_unbound20:00:201
3267350993427,4cyclictest3291402-21latency_hist20:25:010
3267358993331,1cyclictest0-21swapper/321:14:553
3267358993331,1cyclictest0-21swapper/319:59:553
3267358993330,2cyclictest0-21swapper/322:52:593
3267355993330,2cyclictest0-21swapper/220:23:592
3267355993330,2cyclictest0-21swapper/200:18:582
326735499330,3cyclictest714-21Xorg22:50:591
3267358993230,1cyclictest0-21swapper/322:14:553
3267358993227,4cyclictest3285041-21turbostat20:05:003
3267358993226,4cyclictest3276828-21turbostat19:40:003
3267355993229,2cyclictest0-21swapper/222:31:002
3267355993229,2cyclictest0-21swapper/222:30:592
326735599320,2cyclictest0-21swapper/219:50:032
3267354993229,2cyclictest0-21swapper/123:11:591
3267354993223,7cyclictest3278368-21kworker/u8:0+flush-179:020:50:011
326735499321,1cyclictest0-21swapper/121:45:121
3267358993129,1cyclictest0-21swapper/300:39:563
3267358993129,1cyclictest0-21swapper/300:39:553
326735899310,1cyclictest0-21swapper/300:00:593
3267355993128,2cyclictest0-21swapper/222:29:562
3267355993128,2cyclictest0-21swapper/220:27:592
3267355993127,2cyclictest0-21swapper/223:11:592
326735599310,27cyclictest0-21swapper/223:50:042
326735599310,27cyclictest0-21swapper/222:35:142
326735499311,28cyclictest0-21swapper/123:50:031
3267358993028,1cyclictest0-21swapper/321:44:563
3267358993027,2cyclictest0-21swapper/321:02:003
326735899300,28cyclictest0-21swapper/321:55:023
326735599300,2cyclictest0-21swapper/221:45:132
326735599300,28cyclictest0-21swapper/220:05:112
326735599300,28cyclictest0-21swapper/219:45:152
326735599300,27cyclictest0-21swapper/219:25:132
3267358992927,1cyclictest0-21swapper/320:34:593
3267358992927,1cyclictest0-21swapper/320:10:133
3267358992926,2cyclictest0-21swapper/320:55:133
3267358992925,2cyclictest0-21swapper/320:40:133
326735899290,28cyclictest0-21swapper/300:30:133
326735899290,28cyclictest0-21swapper/300:08:593
326735899290,1cyclictest0-21swapper/321:19:593
326735899290,1cyclictest0-21swapper/321:19:583
3267355992926,2cyclictest0-21swapper/220:03:592
326735599290,27cyclictest0-21swapper/222:40:142
326735599290,27cyclictest0-21swapper/219:10:132
326735599290,27cyclictest0-21swapper/200:35:112
326735599290,27cyclictest0-21swapper/200:35:112
326735599290,26cyclictest0-21swapper/219:15:122
326735599290,1cyclictest0-21swapper/223:25:132
3267354992926,2cyclictest0-21swapper/119:15:121
3267358992825,2cyclictest0-21swapper/319:20:133
326735899280,27cyclictest0-21swapper/300:14:593
326735899280,26cyclictest0-21swapper/323:20:113
326735899280,26cyclictest0-21swapper/323:20:113
326735899280,26cyclictest0-21swapper/323:00:143
326735899280,26cyclictest0-21swapper/322:15:143
326735899280,26cyclictest0-21swapper/319:50:133
3267355992826,1cyclictest0-21swapper/219:45:002
3267355992825,2cyclictest0-21swapper/223:30:142
3267355992824,2cyclictest0-21swapper/200:05:152
326735599280,26cyclictest0-21swapper/219:35:122
4577091270,7ptp4l61490irq/134-eno1-TxRx-023:10:003
4577091270,7ptp4l61490irq/134-eno1-TxRx-022:25:013
4577091270,7ptp4l61490irq/134-eno1-TxRx-021:55:003
4577091270,6ptp4l61490irq/134-eno1-TxRx-023:35:003
3267358992725,1cyclictest0-21swapper/323:35:103
3267358992725,1cyclictest0-21swapper/322:37:593
326735899270,26cyclictest0-21swapper/323:45:143
326735899270,1cyclictest0-21swapper/320:50:143
326735899270,1cyclictest0-21swapper/320:50:143
326735599270,26cyclictest0-21swapper/222:05:152
326735599270,24cyclictest0-21swapper/221:27:172
326735599270,24cyclictest0-21swapper/219:20:202
3267354992723,3cyclictest3347428-21kworker/u8:3+flush-179:023:35:211
4577091260,7ptp4l441ktimers/319:35:003
4577091260,6ptp4l441ktimers/322:50:003
4577091260,5ptp4l441ktimers/321:50:013
4577091260,5ptp4l441ktimers/321:09:553
3267358992624,1cyclictest0-21swapper/322:27:003
3267358992624,1cyclictest0-21swapper/321:29:553
3267358992624,1cyclictest0-21swapper/320:21:593
326735899260,24cyclictest0-21swapper/319:25:013
3267355992623,2cyclictest0-21swapper/221:15:592
3267355992623,2cyclictest0-21swapper/221:15:592
3267355992623,2cyclictest0-21swapper/221:06:222
3267355992623,2cyclictest0-21swapper/220:39:502
3267355992622,2cyclictest0-21swapper/219:30:062
326735599260,24cyclictest0-21swapper/223:15:142
4577091250,5ptp4l441ktimers/322:59:563
4577091250,5ptp4l441ktimers/319:20:013
326735899250,1cyclictest0-21swapper/323:56:593
326735899250,1cyclictest0-21swapper/319:10:133
3267355992523,1cyclictest0-21swapper/223:07:382
3267355992522,2cyclictest0-21swapper/220:57:502
3267355992522,2cyclictest0-21swapper/200:11:002
3267354992520,3cyclictest0-21swapper/123:20:221
3267354992520,3cyclictest0-21swapper/123:20:221
3267355992422,1cyclictest0-21swapper/223:50:012
3267355992421,2cyclictest0-21swapper/200:00:582
3267355992420,2cyclictest0-21swapper/222:01:592
4577091230,4ptp4l61490irq/134-eno1-TxRx-022:44:553
3267355992320,2cyclictest0-21swapper/223:37:112
3267355992320,2cyclictest0-21swapper/222:54:552
3267355992320,2cyclictest0-21swapper/200:33:592
3267355992318,2cyclictest0-21swapper/221:10:152
3267355992219,2cyclictest0-21swapper/221:52:502
3267355992219,2cyclictest0-21swapper/220:46:592
3267350992116,4cyclictest3267380-21latency_hist19:10:020
3267355991916,2cyclictest0-21swapper/223:40:122
3267355991916,2cyclictest0-21swapper/200:23:592
3267355991816,1cyclictest0-21swapper/222:19:582
3267355991613,2cyclictest0-21swapper/223:23:002
3267355991613,2cyclictest0-21swapper/223:22:592
3267355991311,1cyclictest0-21swapper/200:29:002
3267355991310,2cyclictest0-21swapper/223:04:592
32673559940,3cyclictest0-21swapper/219:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional