You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-09 - 05:58
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot6.osadl.org (updated Fri May 09, 2025 00:44:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
359405899142130,9cyclictest3687240-40mandb00:00:011
359405599133125,6cyclictest3613567-21kworker/u8:1+flush-179:020:30:220
359405599130120,8cyclictest3610112-21apt-get20:00:000
359405599130120,8cyclictest3610112-21apt-get20:00:000
359405599129115,12cyclictest3687197-40logrotate00:00:020
359405899127117,8cyclictest3662245-21kworker/u8:2+flush-179:023:00:221
359405599125118,5cyclictest3603786-21kworker/u8:0+events_unbound20:00:020
359405599124116,5cyclictest3632844-21kworker/u8:1+events_unbound21:15:200
359405599121112,7cyclictest3597439-21kworker/u8:2+flush-179:019:40:010
359405599120113,5cyclictest3646655-21kworker/u8:0+flush-179:022:15:230
359405599120113,5cyclictest3600599-21kworker/u8:3+flush-179:019:50:190
359405599119111,5cyclictest3660100-21kworker/u8:0+events_unbound23:40:250
359405599118101,5cyclictest3615354-21kworker/u8:3+events_unbound20:35:190
359405599117108,7cyclictest3637616-21kworker/u8:1+events_unbound21:55:190
359405599115105,7cyclictest3668114-21kworker/u8:3+flush-179:000:25:210
359405599114103,9cyclictest3595657-21gpgv19:15:020
359405599114103,9cyclictest3595657-21gpgv19:15:010
359405599113106,5cyclictest3615354-21kworker/u8:3+events_unbound20:40:230
359405599111104,5cyclictest3603786-21kworker/u8:0+flush-179:021:00:230
359405599110104,4cyclictest3660100-21kworker/u8:0+events_unbound23:05:230
359405599109103,4cyclictest3623356-21kworker/u8:2+flush-179:020:45:220
35940559910797,7cyclictest3635492-21kworker/u8:0+flush-179:021:20:230
35940589910693,10cyclictest3619904-21apt-get20:30:011
35940559910594,7cyclictest3639757-21kworker/u8:0+flush-179:021:30:210
35940589910493,8cyclictest3598981-21latency_hist19:25:011
35940559910491,10cyclictest3633117-21apt-get21:10:130
35940559910393,7cyclictest3626540-21kworker/u8:1+events_unbound20:50:180
35940559910392,8cyclictest3693631-21latency_hist00:20:010
35940559910295,5cyclictest3681796-21kworker/u8:1+events_unbound00:00:280
35940559910290,9cyclictest3611930-21kworker/u8:2+flush-179:020:15:200
35940559910195,4cyclictest3660100-21kworker/u8:0+flush-179:023:35:220
35940559910195,4cyclictest3660100-21kworker/u8:0+flush-179:023:35:210
35940559910192,7cyclictest2935139-21kworker/u8:0+events_unbound19:15:020
35940559910091,6cyclictest3629677-21kworker/u8:2+flush-179:021:05:210
3594058999779,12cyclictest3668114-21kworker/u8:3+flush-179:023:45:221
3594055999684,8cyclictest3650352-21latency_hist22:05:010
3594055999583,9cyclictest3666489-21latency_hist22:55:010
3594055999581,6cyclictest3640792-21kworker/u8:3+events_unbound21:35:210
3594058999482,10cyclictest3608757-21apt-get19:55:111
3594058999482,10cyclictest3608757-21apt-get19:55:111
3594058999480,11cyclictest3621990-21apt-get20:35:101
3594055999486,6cyclictest3651969-21kworker/u8:1+flush-179:022:35:250
3594058999388,3cyclictest3635492-21kworker/u8:0+flush-179:021:20:261
3594055999387,4cyclictest3603786-21kworker/u8:0+flush-179:020:10:230
3594055999386,5cyclictest3600596-21kworker/u8:0+flush-179:019:30:220
3594058999179,10cyclictest3610622-21apt-get20:00:111
3594058999172,14cyclictest3594202-21apt-get19:10:121
3594058999172,14cyclictest3594202-21apt-get19:10:111
3594058999078,9cyclictest3695217-21latency_hist00:25:021
3594058999076,10cyclictest3105085-21systemd-journal00:35:051
3594055999084,4cyclictest3668114-21kworker/u8:3+flush-179:023:45:220
3594055999079,4cyclictest3692016-21kworker/u8:0+events_unbound00:20:140
3594055998980,6cyclictest3660100-21kworker/u8:0+events_unbound22:35:000
3594058998875,10cyclictest3660391-21apt-get22:35:131
3594055998877,7cyclictest3651969-21kworker/u8:1+events_unbound22:20:190
3594055998876,9cyclictest3650606-21apt-get22:05:140
3594058998774,10cyclictest3692281-21apt-get00:15:141
3594055998773,10cyclictest3681796-21kworker/u8:1+flush-179:000:10:240
3594058998675,8cyclictest3620201-21apt-key20:30:021
3594055998680,4cyclictest3615354-21kworker/u8:3+flush-179:020:25:000
3594055998470,10cyclictest3658417-21gpgv22:30:010
3594058998374,6cyclictest3651969-21kworker/u8:1+flush-179:022:25:211
3594058998371,9cyclictest3624923-21latency_hist20:45:001
3594055998378,4cyclictest3647196-21kworker/u8:3+events_unbound22:10:190
3594055998374,6cyclictest3637616-21kworker/u8:1+events_unbound21:40:160
3594058998266,11cyclictest3681796-21kworker/u8:1+flush-179:000:05:211
3594058998168,9cyclictest3105085-21systemd-journal20:55:211
3594055998176,4cyclictest3668114-21kworker/u8:3+flush-179:000:35:270
3594055998172,7cyclictest3662245-21kworker/u8:2+flush-179:022:50:000
3594058998068,9cyclictest3676290-21apt-get23:25:151
3594055998075,4cyclictest3603786-21kworker/u8:0+events_unbound20:05:250
3594055998072,5cyclictest3668114-21kworker/u8:3+events_unbound23:30:180
3594055998071,7cyclictest3668114-21kworker/u8:3+flush-179:023:25:000
3594055997971,6cyclictest3662245-21kworker/u8:2+flush-179:022:55:210
3594055997765,8cyclictest3668114-21kworker/u8:3+events_unbound00:05:120
3594058997667,6cyclictest3632861-21latency_hist21:10:011
3594058997663,10cyclictest3684280-21apt-get23:50:101
3594058997662,11cyclictest3662245-21kworker/u8:2+flush-179:022:45:231
3594055997663,10cyclictest3105085-21systemd-journal19:50:000
3594058997561,11cyclictest3105085-21systemd-journal20:05:211
3594058997468,4cyclictest3647196-21kworker/u8:3+flush-179:022:05:231
3594058997465,6cyclictest3613567-21kworker/u8:1+events_unbound20:20:181
3594055997465,7cyclictest3613567-21kworker/u8:1+events_unbound20:25:200
3594058997364,7cyclictest3651969-21kworker/u8:1+events_unbound22:30:201
3594058997265,5cyclictest3662245-21kworker/u8:2+flush-179:022:40:211
3594058997263,6cyclictest3655158-21kworker/u8:2+flush-179:022:25:001
3594058997259,11cyclictest3652189-21apt-get22:10:121
3594055997267,4cyclictest3600599-21kworker/u8:3+events_unbound19:40:210
3594058997163,6cyclictest3695183-21kworker/u8:4+events_unbound00:30:251
3594055997165,5cyclictest3695183-21kworker/u8:4+flush-179:000:30:210
3594055997063,6cyclictest3660100-21kworker/u8:0+flush-179:023:19:550
3594058996958,9cyclictest3629677-21kworker/u8:2+flush-179:021:10:221
3594058996955,11cyclictest3637893-21apt-get21:25:181
3594058996863,3cyclictest2935139-21kworker/u8:0+events_unbound19:15:251
3594055996862,5cyclictest3637616-21kworker/u8:1+events_unbound21:50:220
3594055996862,4cyclictest3642460-21kworker/u8:2+flush-179:021:45:140
3594058996758,6cyclictest3637616-21kworker/u8:1+events_unbound22:00:011
3594058996758,5cyclictest3603786-21kworker/u8:0+flush-179:021:00:151
3594058996755,10cyclictest3666744-21apt-get22:55:121
3594055996760,5cyclictest2935139-21kworker/u8:0+events_unbound19:20:150
3594058996656,8cyclictest3673099-21apt-get23:15:101
3594055996660,4cyclictest3629677-21kworker/u8:2+events_unbound21:25:250
3594058996561,2cyclictest3600599-21kworker/u8:3+events_unbound19:40:191
3594058996550,12cyclictest3640792-21kworker/u8:3+flush-179:021:35:001
3594055996559,4cyclictest3593960-21kworker/u8:1+events_unbound19:25:170
3594058996455,6cyclictest3668114-21kworker/u8:3+flush-179:000:00:241
3594058996454,7cyclictest3634700-21apt-get21:15:111
3594055996459,4cyclictest3662245-21kworker/u8:2+events_unbound23:10:250
3594055996455,6cyclictest3681796-21kworker/u8:1+events_unbound23:50:110
3594058996251,8cyclictest3600596-21kworker/u8:0+flush-179:019:30:211
3594055996257,4cyclictest3615354-21kworker/u8:3+events_unbound20:55:180
3594058996157,2cyclictest3615354-21kworker/u8:3+events_unbound20:55:001
3594055996154,5cyclictest3662245-21kworker/u8:2+events_unbound23:00:110
3594058996053,5cyclictest3660100-21kworker/u8:0+flush-179:023:40:011
3594058996053,5cyclictest3660100-21kworker/u8:0+flush-179:023:40:001
3594058996050,8cyclictest3623356-21kworker/u8:2+flush-179:020:45:261
3594058995948,8cyclictest3668114-21kworker/u8:3+events_unbound23:30:221
3594055995951,5cyclictest3651969-21kworker/u8:1+events_unbound22:40:120
3594058995649,5cyclictest3600599-21kworker/u8:3+events_unbound19:50:131
3594058995443,8cyclictest3600562-21latency_hist19:30:011
3594055995348,4cyclictest3660100-21kworker/u8:0+events_unbound23:25:320
359405899511,48cyclictest1-21systemd22:20:011
3594058995043,6cyclictest3600599-21kworker/u8:3+events_unbound19:45:211
3594058995040,7cyclictest3640792-21kworker/u8:3+flush-179:021:40:221
4505291480,13ptp4l61790irq/134-eno1-TxRx-019:09:553
3594058994840,6cyclictest3637616-21kworker/u8:1+events_unbound21:45:231
3594058994639,5cyclictest3674705-21apt-get23:20:171
3594058994638,7cyclictest3642460-21kworker/u8:2+events_unbound21:50:191
3594058994436,6cyclictest3660100-21kworker/u8:0+flush-179:022:54:551
3594061994321,2cyclictest0-21swapper/223:24:582
359406199430,41cyclictest0-21swapper/223:35:002
3594058994232,8cyclictest3611930-21kworker/u8:2+events_unbound20:15:211
3594058994220,8cyclictest3593919-21latency_hist19:10:021
3594063994135,5cyclictest3679214-21turbostat23:35:003
3594058994133,6cyclictest3681796-21kworker/u8:1+events_unbound00:10:271
3594063994038,1cyclictest0-21swapper/322:20:003
359406199400,37cyclictest0-21swapper/220:12:592
3594058994034,5cyclictest722-21Xorg20:10:591
359406199390,36cyclictest0-21swapper/221:33:592
359406199390,36cyclictest0-21swapper/220:05:592
359406199390,35cyclictest0-21swapper/222:40:592
359406399380,36cyclictest0-21swapper/319:56:003
359406399380,36cyclictest0-21swapper/319:55:593
3594061993835,2cyclictest0-21swapper/223:50:582
3594058993834,2cyclictest722-21Xorg00:27:591
3594063993734,2cyclictest0-21swapper/319:15:583
359406399370,36cyclictest0-21swapper/319:23:593
3594061993734,2cyclictest0-21swapper/223:09:552
3594061993734,2cyclictest0-21swapper/222:00:582
3594061993734,2cyclictest0-21swapper/221:54:002
3594061993734,2cyclictest0-21swapper/200:13:592
359406199370,35cyclictest0-21swapper/222:54:552
3594058993729,6cyclictest3600599-21kworker/u8:3+flush-179:019:35:151
359405899371,6cyclictest688-21snmpd21:35:141
3594063993634,1cyclictest0-21swapper/321:50:593
359406399360,35cyclictest0-21swapper/320:51:593
359406399360,35cyclictest0-21swapper/320:48:593
359406399360,1cyclictest0-21swapper/323:07:593
3594061993634,1cyclictest0-21swapper/219:54:592
3594061993633,2cyclictest0-21swapper/221:47:592
3594061993633,2cyclictest0-21swapper/220:43:592
3594061993632,2cyclictest0-21swapper/221:59:582
359405899361,34cyclictest3649021-21apt-get22:00:151
3594061993533,1cyclictest0-21swapper/221:04:592
3594061993531,2cyclictest0-21swapper/222:36:592
3594058993520,3cyclictest3671292-21true23:10:011
4505291340,27ptp4l61790irq/134-eno1-TxRx-021:30:143
3594063993432,1cyclictest0-21swapper/323:39:593
3594063993432,1cyclictest0-21swapper/323:39:583
3594063993432,1cyclictest0-21swapper/322:45:593
3594063993432,1cyclictest0-21swapper/322:04:563
3594063993432,1cyclictest0-21swapper/321:13:593
3594063993431,2cyclictest0-21swapper/322:14:583
3594061993431,2cyclictest0-21swapper/222:56:592
3594061993430,2cyclictest0-21swapper/221:11:582
3594058993422,10cyclictest3662245-21kworker/u8:2+flush-179:023:10:201
3594063993326,6cyclictest3685617-21turbostat23:55:003
3594061993330,2cyclictest0-21swapper/223:39:552
3594061993330,2cyclictest0-21swapper/223:39:552
3594063993230,1cyclictest0-21swapper/300:19:583
3594061993229,2cyclictest0-21swapper/222:32:592
359406199320,29cyclictest0-21swapper/200:25:142
359406199320,28cyclictest0-21swapper/219:15:032
3594063993129,1cyclictest0-21swapper/323:41:593
3594063993129,1cyclictest0-21swapper/321:45:593
3594063993129,1cyclictest0-21swapper/300:22:593
359406399310,30cyclictest0-21swapper/320:27:003
359406399310,29cyclictest0-21swapper/319:25:143
3594061993128,2cyclictest0-21swapper/222:19:542
359406199310,1cyclictest0-21swapper/220:00:032
3594063993028,1cyclictest0-21swapper/319:12:003
3594063993028,1cyclictest0-21swapper/319:11:593
3594063993027,2cyclictest0-21swapper/321:09:003
3594063993027,2cyclictest0-21swapper/320:56:593
359406399300,29cyclictest0-21swapper/322:30:133
359406399300,29cyclictest0-21swapper/321:03:593
3594061993028,1cyclictest0-21swapper/223:45:002
3594061993027,2cyclictest0-21swapper/223:55:142
3594061993027,2cyclictest0-21swapper/223:10:142
3594061993027,2cyclictest0-21swapper/200:35:052
359406199300,27cyclictest0-21swapper/222:25:152
359406199300,27cyclictest0-21swapper/221:20:152
359406199300,27cyclictest0-21swapper/220:15:132
359406199300,27cyclictest0-21swapper/219:35:142
359406199300,27cyclictest0-21swapper/200:05:132
3594063992927,1cyclictest0-21swapper/323:25:593
3594063992926,2cyclictest3700260-21turbostat00:40:003
3594063992926,2cyclictest0-21swapper/320:40:123
3594063992926,2cyclictest0-21swapper/300:10:163
359406399290,2cyclictest0-21swapper/300:00:163
359406399290,28cyclictest0-21swapper/321:15:133
359406399290,27cyclictest0-21swapper/320:20:113
359406399290,1cyclictest0-21swapper/320:30:023
3594061992926,2cyclictest0-21swapper/222:20:122
3594061992926,2cyclictest0-21swapper/222:15:002
3594061992926,2cyclictest0-21swapper/221:25:182
359406199290,27cyclictest0-21swapper/223:25:162
359406199290,26cyclictest0-21swapper/219:45:142
4505291280,7ptp4l441ktimers/323:15:013
3594063992826,1cyclictest0-21swapper/322:55:143
3594063992826,1cyclictest0-21swapper/322:41:583
3594063992825,2cyclictest0-21swapper/321:25:193
359406399280,27cyclictest0-21swapper/322:35:153
359406399280,26cyclictest0-21swapper/323:55:163
359406399280,26cyclictest0-21swapper/322:25:143
359406399280,26cyclictest0-21swapper/319:45:143
359406399280,26cyclictest0-21swapper/300:25:143
359406399280,26cyclictest0-21swapper/300:05:133
359406399280,1cyclictest0-21swapper/322:53:593
3594061992825,2cyclictest0-21swapper/223:16:002
3594061992825,2cyclictest0-21swapper/220:39:542
359406199280,27cyclictest0-21swapper/221:15:132
359406199280,25cyclictest0-21swapper/223:03:002
4505291270,6ptp4l61790irq/134-eno1-TxRx-019:45:003
4505291270,6ptp4l441ktimers/321:45:003
3594063992725,1cyclictest0-21swapper/321:24:543
3594063992724,2cyclictest0-21swapper/321:57:583
359406399270,1cyclictest0-21swapper/319:36:593
3594061992724,2cyclictest0-21swapper/220:52:592
3594061992724,2cyclictest0-21swapper/200:20:582
359406199270,25cyclictest0-21swapper/220:27:592
359406199270,25cyclictest0-21swapper/220:23:592
3594063992624,1cyclictest431rcuc/322:24:563
3594063992624,1cyclictest0-21swapper/321:35:133
3594063992624,1cyclictest0-21swapper/320:08:593
3594061992623,2cyclictest0-21swapper/220:31:592
3594061992623,2cyclictest0-21swapper/219:40:592
3594058992623,2cyclictest0-21swapper/123:40:141
4505291250,4ptp4l61790irq/134-eno1-TxRx-023:50:003
4505291250,4ptp4l441ktimers/320:05:013
4505291250,3ptp4l441ktimers/323:25:003
3594063992523,1cyclictest0-21swapper/323:15:593
3594063992523,1cyclictest0-21swapper/323:04:543
3594063992522,2cyclictest3608512-21turbostat19:55:003
3594063992522,2cyclictest3602158-21turbostat19:35:003
359406399250,1cyclictest0-21swapper/320:14:553
3594061992522,2cyclictest0-21swapper/222:07:002
3594061992522,2cyclictest0-21swapper/219:55:132
3594061992522,2cyclictest0-21swapper/219:55:132
3594061992522,2cyclictest0-21swapper/200:16:002
4505291240,3ptp4l61790irq/134-eno1-TxRx-020:40:003
3594063992421,2cyclictest0-21swapper/320:19:593
359406399240,1cyclictest0-21swapper/300:30:143
3594055992419,4cyclictest0-21swapper/019:10:010
4505291230,3ptp4l441ktimers/322:09:563
3594061992320,2cyclictest0-21swapper/220:47:592
3594061992320,2cyclictest0-21swapper/200:03:002
3594061992219,2cyclictest0-21swapper/219:13:592
3594061992219,2cyclictest0-21swapper/219:13:592
3594061992017,2cyclictest0-21swapper/222:45:142
3594061992017,2cyclictest0-21swapper/219:34:552
3594061991916,2cyclictest0-21swapper/221:40:582
3594061991815,2cyclictest0-21swapper/200:33:592
3594061991714,2cyclictest0-21swapper/221:38:002
3594061991714,2cyclictest0-21swapper/220:56:002
3594061991613,2cyclictest0-21swapper/219:20:592
3594061991512,2cyclictest0-21swapper/223:47:592
3594061991512,2cyclictest0-21swapper/219:25:142
359406199100,1cyclictest0-21swapper/221:10:012
35940619940,3cyclictest0-21swapper/219:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional