You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-07-21 - 05:41
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot6.osadl.org (updated Sun Jul 21, 2024 00:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
37862099910696,7cyclictest3884305-21latency_hist00:15:001
37862139910034,63cyclictest0-21swapper/322:54:593
3786209997975,3cyclictest3866800-21kworker/u8:3+events_unbound00:04:021
3786209997357,12cyclictest3879494-40mandb00:00:011
3786208997358,11cyclictest3832973-21gpgv21:35:000
3786209997268,3cyclictest3866800-21kworker/u8:3+events_unbound00:05:281
3786208997265,5cyclictest3877912-21kworker/u8:2+flush-179:000:20:000
3786208997159,9cyclictest3797310-21latency_hist19:45:010
3786208996862,4cyclictest3832831-21kworker/u8:1+flush-179:021:45:000
3786209996762,3cyclictest3794100-21kworker/u8:0+events_unbound19:55:061
3786208996763,3cyclictest3848192-21kworker/u8:1+events_unbound22:44:560
3786209996555,7cyclictest3866767-21latency_hist23:20:001
3786208996559,4cyclictest3805545-21kworker/u8:3+events_unbound20:20:010
378620899650,58cyclictest0-21swapper/022:54:590
3786209996455,7cyclictest3847433-21latency_hist22:20:011
3786209996252,8cyclictest3876533-21/usr/sbin/munin23:50:221
3786208996255,4cyclictest3850580-21kworker/u8:2+flush-179:023:05:230
3786208996253,6cyclictest3866800-21kworker/u8:3+flush-179:023:30:190
3786209996152,7cyclictest3874711-21latency_hist23:45:011
3786209996055,4cyclictest3783073-21kworker/u8:0+events_unbound19:18:081
3786208996055,4cyclictest3815103-21kworker/u8:0+flush-179:020:55:150
3786208996050,8cyclictest3829676-21latency_hist21:25:010
3786209995950,7cyclictest3798892-21latency_hist19:50:001
3786209995854,3cyclictest3841104-21kworker/u8:3+events_unbound22:10:071
3786209995849,7cyclictest3841051-21latency_hist22:00:011
3786208995845,9cyclictest3829934-21apt-get21:25:130
3786208995746,9cyclictest3802597-21apt-get20:00:090
3786208995744,11cyclictest3848192-21kworker/u8:1+flush-179:022:30:000
3786208995650,4cyclictest3824657-21kworker/u8:0+flush-179:021:10:120
3786208995648,5cyclictest3882768-21kworker/u8:0+flush-179:000:25:220
378620899560,54cyclictest3834749-21munin-run21:40:010
3786209995551,3cyclictest3887502-21kworker/u8:1+events_unbound00:35:201
3786208995549,4cyclictest3834947-21kworker/u8:2+flush-179:022:00:140
3786208995548,5cyclictest3866800-21kworker/u8:3+flush-179:023:55:010
3786208995547,6cyclictest3841104-21kworker/u8:3+flush-179:022:20:150
3786209995450,3cyclictest3787740-21kworker/u8:1+events_unbound19:51:071
3786209995449,3cyclictest3848192-21kworker/u8:1+events_unbound22:40:031
3786209995449,3cyclictest3832831-21kworker/u8:1+events_unbound21:46:071
3786209995445,7cyclictest3807094-21latency_hist20:15:011
3786208995445,6cyclictest3781407-21kworker/u8:3+flush-179:019:10:140
3786209995349,3cyclictest3850580-21kworker/u8:2+events_unbound23:01:071
3786209995348,4cyclictest3848192-21kworker/u8:1+events_unbound22:35:021
3786208995348,4cyclictest3848192-21kworker/u8:1+events_unbound22:45:280
3786209995248,3cyclictest3866800-21kworker/u8:3+events_unbound23:21:071
3786209995247,4cyclictest3824657-21kworker/u8:0+events_unbound21:24:071
3786208995246,4cyclictest3832831-21kworker/u8:1+events_unbound21:45:190
3786209995146,4cyclictest3850580-21kworker/u8:2+events_unbound23:05:031
3786209995145,5cyclictest3794100-21kworker/u8:0+events_unbound19:43:061
3786208995143,6cyclictest3866800-21kworker/u8:3+flush-179:023:25:220
3786209995046,3cyclictest3841027-21kworker/u8:1+events_unbound22:05:071
3786209995046,3cyclictest3832831-21kworker/u8:1+events_unbound21:32:071
3786209995046,3cyclictest3811916-21kworker/u8:1+events_unbound20:55:031
3786209995046,3cyclictest3787740-21kworker/u8:1+events_unbound19:23:071
3786209995040,7cyclictest3890923-21latency_hist00:35:001
3786209994945,3cyclictest3815103-21kworker/u8:0+events_unbound20:51:071
3786209994944,4cyclictest3858037-21kworker/u8:0+events_unbound22:55:081
3786208994943,4cyclictest3866800-21kworker/u8:3+flush-179:023:40:140
3786208994941,5cyclictest3787740-21kworker/u8:1+flush-179:019:45:230
3786208994938,9cyclictest3867025-21apt-get23:20:130
3786209994844,3cyclictest3877912-21kworker/u8:2+events_unbound00:20:251
3786209994844,3cyclictest3846058-21kworker/u8:0+events_unbound22:23:071
3786209994843,4cyclictest3850580-21kworker/u8:2+events_unbound22:53:081
3786209994842,5cyclictest3875746-21kworker/u8:1+flush-179:023:45:221
3786208994842,4cyclictest151ktimers/021:55:110
3786208994840,6cyclictest3877912-21kworker/u8:2+flush-179:000:05:220
3786208994837,9cyclictest267-21systemd-journal00:35:010
4863991470,13ptp4l441ktimers/319:09:553
3786212994744,2cyclictest0-21swapper/223:04:592
3786209994744,2cyclictest0-21swapper/121:06:591
3786209994743,3cyclictest3810824-21kworker/u8:0+events_unbound20:30:071
3786209994743,3cyclictest3805545-21kworker/u8:3+events_unbound20:25:041
3786209994743,3cyclictest3805545-21kworker/u8:3+events_unbound20:25:031
3786209994743,3cyclictest3794106-21kworker/u8:3+events_unbound19:35:071
3786209994742,4cyclictest3787740-21kworker/u8:1+events_unbound19:14:081
378620999470,11cyclictest29-21ksoftirqd/121:10:021
3786208994740,6cyclictest3821432-21kworker/u8:2+flush-179:021:20:010
3786208994738,7cyclictest3791188-21/usr/sbin/munin19:25:210
3786209994642,3cyclictest3865141-21kworker/u8:1+events_unbound23:30:031
3786209994642,3cyclictest3824657-21kworker/u8:0+events_unbound21:29:071
378620999461,2cyclictest678-21Xorg20:43:591
3786208994641,4cyclictest3841027-21kworker/u8:1+flush-179:022:05:200
3786208994640,4cyclictest3803577-21kworker/u8:2+events_unbound20:20:180
3786208994637,7cyclictest3812127-21apt-get20:30:110
3786209994541,3cyclictest3848192-21kworker/u8:1+events_unbound22:46:081
3786209994535,8cyclictest3841308-21/usr/sbin/munin22:00:201
3786209994534,8cyclictest3850840-21/usr/sbin/munin22:30:221
3786208994539,4cyclictest3783073-21kworker/u8:0+flush-179:019:20:190
3786212994441,2cyclictest0-21swapper/221:06:592
378621299440,2cyclictest0-21swapper/220:25:592
378621299440,2cyclictest0-21swapper/220:25:592
378620999449,32cyclictest3834903-21sed21:40:011
3786209994440,3cyclictest3865141-21kworker/u8:1+events_unbound23:26:071
3786208994437,5cyclictest3803577-21kworker/u8:2+flush-179:020:05:190
3786209994339,3cyclictest3834947-21kworker/u8:2+events_unbound21:41:071
3786208994338,3cyclictest3811916-21kworker/u8:1+events_unbound21:00:240
3786208994336,5cyclictest3793079-21diskstats19:30:160
3786208994335,6cyclictest3877912-21kworker/u8:2+events_unbound00:00:010
3786209994238,3cyclictest3882768-21kworker/u8:0+events_unbound00:27:291
3786209994238,3cyclictest3803577-21kworker/u8:2+events_unbound20:00:591
3786209994238,3cyclictest271rcuc/120:20:001
3786209994237,4cyclictest3839444-21kworker/u8:0+events_unbound21:54:071
3786209994237,4cyclictest3839444-21kworker/u8:0+events_unbound21:54:071
3786209994232,8cyclictest3826723-21/usr/sbin/munin21:15:211
3786209994229,11cyclictest0-21swapper/123:40:001
378620999421,32cyclictest678-21Xorg20:36:591
3786208994236,4cyclictest3875746-21kworker/u8:1+events_unbound23:50:010
3786208994231,8cyclictest3863814-21/usr/sbin/munin23:10:230
378620899420,40cyclictest0-21swapper/022:55:120
3786208994134,4cyclictest3877912-21kworker/u8:2+flush-179:000:10:200
3786209994036,3cyclictest3815103-21kworker/u8:0+events_unbound20:45:021
3786208994036,3cyclictest3800514-21systemd-run19:55:000
3786208994034,5cyclictest3866800-21kworker/u8:3+flush-179:023:40:000
3786208994031,7cyclictest3815103-21kworker/u8:0+events_unbound20:40:180
3786213993936,2cyclictest0-21swapper/300:29:593
3786208993933,4cyclictest3846058-21kworker/u8:0+flush-179:022:20:010
378620899390,2cyclictest678-21Xorg22:14:590
3786212993834,2cyclictest0-21swapper/220:12:582
378621299380,36cyclictest0-21swapper/219:49:002
378621299380,35cyclictest0-21swapper/223:09:592
3786209993834,3cyclictest3794100-21kworker/u8:0+events_unbound19:31:141
3786209993834,3cyclictest3787740-21kworker/u8:1+events_unbound20:09:371
378620899381,33cyclictest3865415-21apt-get23:15:110
3786213993735,1cyclictest0-21swapper/323:30:593
3786213993734,2cyclictest0-21swapper/321:39:593
378621399370,36cyclictest0-21swapper/300:19:003
378621399370,36cyclictest0-21swapper/300:11:593
378621299370,1cyclictest0-21swapper/220:36:592
3786209993733,3cyclictest3787740-21kworker/u8:1+events_unbound19:25:321
378620999371,34cyclictest3884834-21sh00:15:121
3786208993733,3cyclictest3795763-21systemd-run19:40:010
3786208993731,5cyclictest3805545-21kworker/u8:3+flush-179:020:36:010
3786208993729,6cyclictest3858037-21kworker/u8:0+flush-179:023:00:120
378620899371,34cyclictest3801011-21df19:55:140
378620899370,35cyclictest3810829-21diskstats20:25:160
378620899370,35cyclictest3810829-21diskstats20:25:160
3786213993634,1cyclictest0-21swapper/321:16:593
378621399360,35cyclictest0-21swapper/323:53:593
378621399360,35cyclictest0-21swapper/321:20:593
378621399360,1cyclictest0-21swapper/319:58:593
378621399360,1cyclictest0-21swapper/319:42:003
3786212993634,1cyclictest0-21swapper/222:04:592
3786212993633,2cyclictest0-21swapper/221:11:582
3786209993632,3cyclictest3848192-21kworker/u8:1+events_unbound22:29:301
3786208993629,5cyclictest3850580-21kworker/u8:2+flush-179:022:35:150
378620899360,34cyclictest3823578-21cstates21:05:130
3786213993533,1cyclictest0-21swapper/320:19:003
3786213993532,2cyclictest0-21swapper/323:03:583
3786213993532,2cyclictest0-21swapper/321:26:593
3786213993532,2cyclictest0-21swapper/320:20:593
3786212993533,1cyclictest0-21swapper/223:20:002
3786212993531,2cyclictest0-21swapper/221:36:592
3786209993531,3cyclictest3803577-21kworker/u8:2+events_unbound20:24:121
3786208993531,3cyclictest3892529-21munin-run00:40:000
3786208993527,5cyclictest3879767-21apt-get00:00:120
3786208993523,9cyclictest3816903-21apt-get20:45:120
378620899350,34cyclictest3818538-21apt-get20:50:130
4863991340,3ptp4l441ktimers/323:05:143
3786213993432,1cyclictest0-21swapper/323:29:593
3786213993432,1cyclictest0-21swapper/321:55:593
3786213993431,2cyclictest0-21swapper/320:07:593
3786212993431,2cyclictest0-21swapper/219:42:582
3786212993431,2cyclictest0-21swapper/200:38:592
3786209993414,18cyclictest0-21swapper/121:00:151
3786208993426,5cyclictest0-21swapper/000:21:590
4863991330,3ptp4l441ktimers/322:10:133
3786213993331,1cyclictest0-21swapper/322:20:593
3786213993331,1cyclictest0-21swapper/322:09:593
3786213993331,1cyclictest0-21swapper/320:41:583
3786213993330,2cyclictest0-21swapper/323:15:593
3786212993330,2cyclictest0-21swapper/221:53:592
3786212993330,2cyclictest0-21swapper/221:53:592
3786212993330,2cyclictest0-21swapper/220:42:582
3786209993329,3cyclictest3850580-21kworker/u8:2+events_unbound23:11:301
378620899333,29cyclictest3851601-21ntp_192.168.11422:30:190
3786208993324,5cyclictest0-21swapper/019:20:000
3786213993229,2cyclictest0-21swapper/320:55:003
3786212993229,2cyclictest0-21swapper/220:53:592
378621299320,2cyclictest0-21swapper/222:10:132
378621299320,29cyclictest0-21swapper/220:30:132
378621299320,29cyclictest0-21swapper/220:00:012
378621299320,28cyclictest0-21swapper/219:15:112
378621299320,27cyclictest0-21swapper/222:25:122
3786212993128,2cyclictest0-21swapper/222:39:592
3786212993128,2cyclictest0-21swapper/220:01:592
3786212993128,2cyclictest0-21swapper/219:20:102
378621299310,29cyclictest0-21swapper/219:30:142
378621299310,28cyclictest0-21swapper/223:25:582
378621299310,27cyclictest0-21swapper/222:20:112
378621299310,27cyclictest0-21swapper/221:00:192
378621299310,27cyclictest0-21swapper/219:35:112
378621299310,27cyclictest0-21swapper/200:10:192
3786208993126,3cyclictest3838134-21apt-get21:50:110
3786208993126,3cyclictest3838134-21apt-get21:50:110
3786213993027,2cyclictest0-21swapper/322:40:133
378621399300,29cyclictest0-21swapper/323:40:103
378621399300,28cyclictest0-21swapper/323:55:133
378621399300,28cyclictest0-21swapper/320:45:133
378621399300,28cyclictest0-21swapper/320:30:123
378621399300,28cyclictest0-21swapper/319:45:213
3786212993027,2cyclictest0-21swapper/222:40:592
3786212993027,2cyclictest0-21swapper/200:07:592
3786212993026,2cyclictest0-21swapper/221:20:182
3786212993026,2cyclictest0-21swapper/220:45:222
3786212993026,2cyclictest0-21swapper/220:05:142
378621299300,28cyclictest0-21swapper/223:50:122
378621299300,28cyclictest0-21swapper/223:30:092
378621299300,28cyclictest0-21swapper/219:25:202
378621299300,28cyclictest0-21swapper/200:25:142
378621299300,27cyclictest0-21swapper/223:45:112
378621299300,27cyclictest0-21swapper/223:20:152
378621299300,27cyclictest0-21swapper/222:55:022
378621299300,27cyclictest0-21swapper/222:50:142
378621299300,27cyclictest0-21swapper/220:55:112
378621299300,27cyclictest0-21swapper/220:15:122
378621299300,27cyclictest0-21swapper/219:10:112
378621299300,27cyclictest0-21swapper/200:20:112
378621299300,26cyclictest0-21swapper/221:15:102
3786213992927,1cyclictest0-21swapper/322:55:133
3786213992927,1cyclictest0-21swapper/319:15:113
3786213992926,2cyclictest0-21swapper/322:40:013
3786213992926,2cyclictest0-21swapper/321:40:203
3786213992926,2cyclictest0-21swapper/319:10:133
3786213992926,2cyclictest0-21swapper/300:35:143
378621399290,28cyclictest0-21swapper/323:10:113
378621399290,28cyclictest0-21swapper/322:00:103
378621399290,28cyclictest0-21swapper/320:35:173
378621399290,27cyclictest0-21swapper/323:35:153
378621399290,27cyclictest0-21swapper/322:30:123
378621399290,27cyclictest0-21swapper/321:30:123
378621399290,27cyclictest0-21swapper/321:10:013
378621399290,27cyclictest0-21swapper/319:40:013
378621399290,27cyclictest0-21swapper/300:30:033
378621399290,1cyclictest0-21swapper/323:45:153
378621399290,1cyclictest0-21swapper/319:30:133
3786212992927,1cyclictest0-21swapper/223:59:592
3786212992927,1cyclictest0-21swapper/223:45:012
3786212992926,2cyclictest0-21swapper/223:35:122
3786212992926,2cyclictest0-21swapper/222:45:172
3786212992926,2cyclictest0-21swapper/220:20:112
3786212992926,2cyclictest0-21swapper/200:30:122
3786212992926,2cyclictest0-21swapper/200:15:212
3786212992925,2cyclictest0-21swapper/219:50:192
378621299290,2cyclictest0-21swapper/221:30:132
378621299290,2cyclictest0-21swapper/200:00:152
378621299290,28cyclictest0-21swapper/222:05:112
378621299290,27cyclictest0-21swapper/222:30:122
378621299290,27cyclictest0-21swapper/221:45:132
378621299290,27cyclictest0-21swapper/221:45:012
378621299290,27cyclictest0-21swapper/221:25:012
378621299290,26cyclictest0-21swapper/222:15:122
378621299290,21cyclictest0-21swapper/223:10:102
3786208992926,2cyclictest3805768-21apt-get20:10:110
3786213992826,1cyclictest0-21swapper/322:15:123
3786213992826,1cyclictest0-21swapper/320:25:133
3786213992826,1cyclictest0-21swapper/320:25:133
3786213992825,2cyclictest0-21swapper/319:50:133
378621399280,2cyclictest0-21swapper/321:05:133
378621399280,27cyclictest0-21swapper/320:55:133
378621399280,27cyclictest0-21swapper/319:20:143
378621399280,27cyclictest0-21swapper/300:05:143
378621399280,26cyclictest0-21swapper/322:25:133
378621399280,26cyclictest0-21swapper/321:50:123
378621399280,26cyclictest0-21swapper/321:50:123
378621399280,26cyclictest0-21swapper/321:45:123
378621399280,26cyclictest0-21swapper/321:00:133
378621399280,26cyclictest0-21swapper/320:10:113
378621399280,26cyclictest0-21swapper/320:00:043
378621399280,26cyclictest0-21swapper/319:25:113
378621399280,1cyclictest0-21swapper/323:20:163
3786212992825,2cyclictest0-21swapper/221:55:122
3786213992725,1cyclictest0-21swapper/322:45:163
378621399270,26cyclictest0-21swapper/300:20:123
378621399270,26cyclictest0-21swapper/300:00:123
378620999244,5cyclictest281ktimers/119:10:011
151299220,13rtkit-daemon650-21ntpd19:08:570
37862129940,2cyclictest0-21swapper/219:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional