You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-01-18 - 16:12
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot6.osadl.org (updated Sat Jan 18, 2025 12:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
102857899132123,7cyclictest1055777-21gpgv08:35:010
102858099129118,8cyclictest1103660-21latency_hist11:35:011
10285809910896,9cyclictest1088272-21latency_hist10:35:001
10285809910786,16cyclictest1041324-21latency_hist07:50:011
10285789910597,5cyclictest1075803-21kworker/u8:0+flush-179:010:05:010
10285809910491,10cyclictest1111904-21apt-get12:05:021
10285789910294,6cyclictest1102106-21kworker/u8:2+flush-179:011:40:010
1028580999985,10cyclictest1075820-21latency_hist09:50:011
1028580999886,9cyclictest1074212-21/usr/sbin/munin09:40:231
1028580999786,8cyclictest1108702-21latency_hist11:55:011
1028578999688,6cyclictest1054097-21kworker/u8:1+flush-179:008:40:260
1028580999478,12cyclictest502-21rs:main1
1028578999384,6cyclictest1065452-21kworker/u8:2+flush-179:009:05:020
1028578999281,9cyclictest1044774-21apt-get08:00:120
102857899921,88cyclictest1-21systemd09:35:000
1028580999180,8cyclictest1114243-21/usr/sbin/munin12:15:241
1028578999184,5cyclictest1040206-21journal-offline07:45:140
1028578999178,9cyclictest3105085-21systemd-journal09:10:000
1028578999176,8cyclictest1075803-21kworker/u8:0+events_unbound10:20:240
1028580999080,7cyclictest1096509-21latency_hist11:05:011
1028578999082,6cyclictest1043147-21kworker/u8:1+flush-179:008:20:000
1028580998970,16cyclictest1120648-21latency_hist12:40:011
1028578998980,6cyclictest1117467-21latency_hist12:30:010
1028578998880,5cyclictest1041312-21kworker/u8:3+flush-179:008:20:200
1028578998677,6cyclictest1075803-21kworker/u8:0+flush-179:010:20:010
1028578998574,8cyclictest1098355-21latency_hist11:15:010
1028578998475,7cyclictest1096543-21kworker/u8:3+flush-179:011:20:010
1028578998273,5cyclictest1086659-21kworker/u8:1+events_unbound10:50:260
1028578998172,7cyclictest1077633-21kworker/u8:2+flush-179:010:30:160
1028578997972,5cyclictest1110357-21kworker/u8:2+flush-179:012:15:230
1028578997870,5cyclictest1041312-21kworker/u8:3+flush-179:008:45:230
1028578997867,7cyclictest1098388-21kworker/u8:0+events_unbound11:30:190
1028580997765,10cyclictest1057772-21apt-get08:40:141
1028578997768,7cyclictest1111898-21latency_hist12:05:020
1028578997670,4cyclictest1077633-21kworker/u8:2+flush-179:010:35:120
1028578997666,7cyclictest1077633-21kworker/u8:2+flush-179:010:10:170
1028578997568,5cyclictest1040737-21kworker/u8:0+flush-179:007:55:220
1028578997562,8cyclictest1041312-21kworker/u8:3+flush-179:008:10:240
1028578997462,10cyclictest1038352-21apt-get07:40:130
1028580997364,7cyclictest1098611-21apt-get11:15:111
1028578997362,8cyclictest1031693-21latency_hist07:20:010
1028578997359,11cyclictest1079529-21apt-get10:00:090
1028578997266,4cyclictest1041312-21kworker/u8:3+events_unbound08:25:180
1028578997259,9cyclictest1075946-21gpgv09:50:000
1028580997161,8cyclictest1065734-21latency_hist09:10:011
1028578997165,4cyclictest1041312-21kworker/u8:3+events_unbound08:50:190
1028580997058,10cyclictest3105085-21systemd-journal07:55:011
1028580996961,6cyclictest1041312-21kworker/u8:3+flush-179:008:54:591
1028580996958,8cyclictest3105085-21systemd-journal09:15:001
1028580996951,15cyclictest1067575-21apt-get09:15:121
1028578996963,3cyclictest1011654-21kworker/u8:3+events_unbound07:10:170
1028578996957,5cyclictest1033270-21kworker/u8:0+events_unbound07:25:230
1028580996756,8cyclictest1028480-21latency_hist07:10:011
1028578996761,4cyclictest1105301-21kworker/u8:3+flush-179:011:55:250
1028578996760,5cyclictest1102106-21kworker/u8:2+flush-179:011:40:180
1028578996760,4cyclictest1026868-21kworker/u8:2+events_unbound07:30:250
1028578996759,5cyclictest1112455-21kworker/u8:0+flush-179:012:10:220
1028580996656,7cyclictest3105085-21systemd-journal07:35:001
1028580996655,7cyclictest1036702-21kworker/u8:3+flush-179:007:35:431
1028578996660,5cyclictest1011654-21kworker/u8:3+events_unbound07:20:260
1028578996660,4cyclictest1063511-21kworker/u8:0+flush-179:009:35:220
1028578996658,5cyclictest1063511-21kworker/u8:0+events_unbound09:40:250
102857899660,64cyclictest1079289-21munin-run10:00:000
1028578996559,4cyclictest1041312-21kworker/u8:3+flush-179:007:50:210
1028578996458,4cyclictest1036702-21kworker/u8:3+flush-179:007:35:210
1028578996457,5cyclictest1054097-21kworker/u8:1+flush-179:009:00:000
1028578996456,6cyclictest1086659-21kworker/u8:1+flush-179:010:40:250
1028578996351,9cyclictest3105085-21systemd-journal09:30:000
1028580996251,9cyclictest1097006-21/usr/sbin/munin11:10:181
1028580996251,8cyclictest3105085-21systemd-journal08:37:411
1028578996255,5cyclictest1057475-21kworker/u8:2+flush-179:008:40:000
1028578996254,6cyclictest1112455-21kworker/u8:0+events_unbound12:10:000
1028578996254,5cyclictest1088307-21kworker/u8:3+flush-179:010:55:160
1028580996150,9cyclictest1108956-21apt-get11:55:131
1028578996153,6cyclictest1096543-21kworker/u8:3+flush-179:011:15:120
1028580996048,9cyclictest3105085-21systemd-journal09:35:011
1028580996046,10cyclictest1011654-21kworker/u8:3+flush-179:007:20:231
1028580995951,6cyclictest1026868-21kworker/u8:2+flush-179:007:30:011
1028580995949,7cyclictest1077633-21kworker/u8:2+events_unbound10:25:021
1028580995947,9cyclictest3105085-21systemd-journal11:45:001
102857899591,56cyclictest1-21systemd09:15:240
1028580995850,6cyclictest1046866-21journal-offline08:05:181
1028580995849,7cyclictest1063511-21kworker/u8:0+flush-179:009:00:011
1028580995749,6cyclictest1088307-21kworker/u8:3+events_unbound10:55:001
1028580995746,8cyclictest1053524-21journal-offline08:25:231
1028578995751,5cyclictest1075803-21kworker/u8:0+events_unbound09:50:100
1028578995751,4cyclictest1065452-21kworker/u8:2+flush-179:009:25:010
1028578995750,4cyclictest1105301-21kworker/u8:3+flush-179:012:20:210
1028580995651,4cyclictest1043147-21kworker/u8:1+flush-179:008:10:251
1028580995445,7cyclictest3105085-21systemd-journal12:22:421
1028580995445,6cyclictest1070776-21latency_hist09:30:011
1028580995443,8cyclictest1073968-21latency_hist09:40:001
102858099540,3cyclictest722-21Xorg12:30:001
1028578995447,5cyclictest1043147-21kworker/u8:1+flush-179:008:05:230
1028580995349,3cyclictest1108692-21kworker/u8:1+flush-179:012:00:311
1028578995342,9cyclictest1065970-21/usr/sbin/munin09:10:200
1028580995243,7cyclictest1077922-21apt-get09:55:101
1028580995226,13cyclictest0-21swapper/110:25:021
102857899521,49cyclictest1118459-21ntp_192.168.11412:30:160
1028580995143,6cyclictest1117516-21kworker/u8:2+events_unbound12:35:001
1028580995143,6cyclictest1086659-21kworker/u8:1+flush-179:011:46:051
4505291490,14ptp4l441ktimers/307:10:013
1028580994943,4cyclictest1077633-21kworker/u8:2+flush-179:010:35:201
1028580994936,10cyclictest1069182-21apt-get09:20:101
1028578994937,10cyclictest1095175-21apt-get11:00:150
1028585994841,5cyclictest1084572-21turbostat10:20:003
1028584994846,1cyclictest0-21swapper/207:59:592
1028580994837,9cyclictest1096768-21latency_hist11:10:011
102857899481,46cyclictest1108710-21idleruntime-cro11:55:000
1028580994739,5cyclictest1055650-21latency_hist08:35:001
1028580994530,13cyclictest0-21swapper/107:13:591
102858099450,43cyclictest722-21Xorg11:24:591
1028580994438,5cyclictest1063511-21kworker/u8:0+flush-179:009:00:211
1028580994436,6cyclictest1088307-21kworker/u8:3+flush-179:010:55:231
102858099440,41cyclictest0-21swapper/110:06:001
102857899441,41cyclictest1091739-21awk10:50:000
1028580994334,6cyclictest1091467-21latency_hist10:45:011
1028580994334,6cyclictest1041312-21kworker/u8:3+events_unbound08:45:141
1028585994240,1cyclictest0-21swapper/308:05:593
1028584994239,2cyclictest0-21swapper/207:49:552
102858499420,7cyclictest0-21swapper/212:30:002
1028580994233,7cyclictest1105301-21kworker/u8:3+events_unbound12:15:011
1028585994138,2cyclictest0-21swapper/310:34:593
1028584994138,2cyclictest0-21swapper/210:05:592
1028580994125,14cyclictest1031412-21timerandwakeup07:15:221
1028578994130,8cyclictest3105085-21systemd-journal10:25:040
1028585994033,6cyclictest0-21swapper/308:58:583
102858499400,4cyclictest0-21swapper/209:29:562
1028580994030,8cyclictest1075803-21kworker/u8:0+flush-179:009:50:151
102857899400,37cyclictest0-21swapper/011:27:590
102858499390,36cyclictest0-21swapper/211:33:592
102858099390,37cyclictest722-21Xorg11:38:591
1028578993934,4cyclictest1120630-21kworker/u8:1+flush-179:012:40:010
1028585993832,4cyclictest1103666-21turbostat11:35:003
102858499380,35cyclictest0-21swapper/208:26:582
102858599370,30cyclictest0-21swapper/308:04:583
1028584993734,2cyclictest0-21swapper/207:31:592
102858499370,2cyclictest0-21swapper/211:51:592
1028585993633,2cyclictest0-21swapper/310:15:003
102858599360,35cyclictest0-21swapper/307:35:593
102858599360,1cyclictest0-21swapper/312:33:593
102858599360,1cyclictest0-21swapper/312:23:593
102858599360,1cyclictest0-21swapper/310:49:003
1028584993633,2cyclictest0-21swapper/210:27:592
1028580993629,6cyclictest1086659-21kworker/u8:1+flush-179:011:29:571
1028580993629,5cyclictest1041312-21kworker/u8:3+flush-179:008:20:161
4505291350,28ptp4l441ktimers/311:20:113
1028584993532,2cyclictest0-21swapper/208:24:002
1028584993532,2cyclictest0-21swapper/207:26:002
1028584993530,2cyclictest0-21swapper/211:35:592
1028580993527,6cyclictest1041312-21kworker/u8:3+flush-179:008:20:001
1028585993427,6cyclictest0-21swapper/308:49:563
1028584993431,2cyclictest0-21swapper/210:58:592
1028580993430,3cyclictest1039061-21needreboot07:40:201
1028585993326,6cyclictest0-21swapper/311:49:593
1028584993330,2cyclictest0-21swapper/212:08:592
1028584993330,2cyclictest0-21swapper/208:19:562
102858499330,30cyclictest0-21swapper/211:02:592
1028578993329,3cyclictest1107152-21systemd-run11:50:000
1028585993230,1cyclictest0-21swapper/310:04:553
1028585993230,1cyclictest0-21swapper/309:00:593
1028584993225,5cyclictest0-21swapper/207:13:592
102858499320,30cyclictest0-21swapper/209:46:592
1028580993226,4cyclictest1045562-21ntp_offset08:00:221
102858099320,30cyclictest722-21Xorg10:17:591
1028585993129,1cyclictest0-21swapper/310:21:593
1028585993129,1cyclictest0-21swapper/309:09:553
1028585993128,2cyclictest0-21swapper/307:55:003
102858599310,1cyclictest0-21swapper/309:35:143
1028584993127,2cyclictest0-21swapper/209:59:552
1028584993125,5cyclictest0-21swapper/207:20:162
1028584993123,7cyclictest0-21swapper/212:20:592
1028580993126,4cyclictest722-21Xorg07:57:581
102858099311,26cyclictest0-21swapper/110:49:591
1028585993028,1cyclictest0-21swapper/309:14:583
1028585993028,1cyclictest0-21swapper/307:28:593
1028585993025,4cyclictest0-21swapper/310:40:123
1028584993027,2cyclictest0-21swapper/212:10:592
1028584993026,2cyclictest0-21swapper/211:57:592
1028584993026,2cyclictest0-21swapper/211:11:592
1028584993022,7cyclictest0-21swapper/208:11:582
102858499300,28cyclictest0-21swapper/209:50:022
102858499300,27cyclictest0-21swapper/211:15:122
102858499300,27cyclictest0-21swapper/210:30:092
102858499300,27cyclictest0-21swapper/209:18:592
102858499300,27cyclictest0-21swapper/208:40:152
1028585992927,1cyclictest0-21swapper/312:38:593
1028585992925,3cyclictest0-21swapper/310:05:013
1028585992924,1cyclictest0-21swapper/307:20:163
102858599290,27cyclictest0-21swapper/310:25:043
102858599290,27cyclictest0-21swapper/307:45:133
1028584992926,2cyclictest0-21swapper/209:22:592
1028584992923,5cyclictest0-21swapper/209:35:132
102858499290,27cyclictest0-21swapper/212:00:162
102858499290,27cyclictest0-21swapper/210:05:002
102858499290,26cyclictest0-21swapper/207:44:592
4505291280,7ptp4l441ktimers/308:14:563
4505291280,6ptp4l441ktimers/311:40:013
4505291280,6ptp4l441ktimers/311:30:013
4505291280,6ptp4l441ktimers/308:00:013
4505291280,6ptp4l441ktimers/307:20:003
1028585992825,2cyclictest0-21swapper/312:10:163
1028585992825,2cyclictest0-21swapper/311:20:013
1028585992825,2cyclictest0-21swapper/310:35:133
1028585992825,2cyclictest0-21swapper/309:41:583
1028585992825,2cyclictest0-21swapper/308:35:023
1028585992824,2cyclictest0-21swapper/311:55:593
102858599280,27cyclictest0-21swapper/311:00:153
102858599280,27cyclictest0-21swapper/308:35:013
102858599280,21cyclictest0-21swapper/312:05:023
1028584992825,2cyclictest0-21swapper/211:42:592
1028584992825,2cyclictest0-21swapper/210:40:122
1028584992825,2cyclictest0-21swapper/210:24:592
1028584992825,2cyclictest0-21swapper/209:10:582
1028584992823,4cyclictest0-21swapper/212:30:092
102858499280,26cyclictest0-21swapper/207:35:112
102858099280,26cyclictest1082368-21netstat10:10:151
4505291270,6ptp4l61790irq/134-eno1-TxRx-009:55:013
4505291270,6ptp4l61790irq/134-eno1-TxRx-009:30:003
4505291270,6ptp4l61790irq/134-eno1-TxRx-009:25:013
4505291270,6ptp4l61790irq/134-eno1-TxRx-009:20:013
4505291270,6ptp4l441ktimers/309:35:003
1028585992725,1cyclictest0-21swapper/312:05:023
1028585992725,1cyclictest0-21swapper/311:14:553
1028585992724,2cyclictest0-21swapper/308:28:593
1028585992723,2cyclictest0-21swapper/308:50:593
102858599270,1cyclictest0-21swapper/311:07:593
1028584992724,2cyclictest0-21swapper/209:42:592
102858499270,26cyclictest0-21swapper/209:06:002
102858499270,25cyclictest0-21swapper/210:50:142
102858499270,25cyclictest0-21swapper/207:19:582
1028578992723,3cyclictest0-21swapper/011:10:000
4505291260,6ptp4l441ktimers/308:45:023
4505291260,5ptp4l441ktimers/312:29:563
4505291260,5ptp4l441ktimers/307:14:593
201699260,14rtkit-daemon1028295-21sed07:05:310
102858599260,25cyclictest1093320-21turbostat10:55:003
102858499260,24cyclictest0-21swapper/210:18:002
4505291250,5ptp4l61790irq/134-eno1-TxRx-011:00:003
4505291250,5ptp4l61790irq/134-eno1-TxRx-008:20:013
1028585992523,1cyclictest0-21swapper/309:58:593
102858599250,24cyclictest0-21swapper/307:42:593
1028584992522,2cyclictest0-21swapper/209:01:592
102858499250,2cyclictest0-21swapper/209:30:592
4505291240,4ptp4l61790irq/134-eno1-TxRx-011:54:553
4505291240,4ptp4l61790irq/134-eno1-TxRx-011:45:003
4505291240,4ptp4l61790irq/134-eno1-TxRx-009:50:003
4505291240,4ptp4l61790irq/134-eno1-TxRx-007:35:003
1028585992422,1cyclictest0-21swapper/312:19:593
1028584992320,2cyclictest0-21swapper/211:23:592
1028584992320,2cyclictest0-21swapper/210:48:592
1028584992320,2cyclictest0-21swapper/208:57:582
1028584992220,1cyclictest0-21swapper/208:05:132
1028584992219,2cyclictest0-21swapper/210:37:582
1028584992219,2cyclictest0-21swapper/208:45:592
1028584992219,2cyclictest0-21swapper/207:52:592
1028584992118,2cyclictest0-21swapper/208:04:592
1028584992017,2cyclictest0-21swapper/208:38:002
1028585991916,2cyclictest0-21swapper/308:20:123
1028584991815,2cyclictest0-21swapper/212:19:552
1028584991815,2cyclictest0-21swapper/211:06:592
1028584991714,2cyclictest0-21swapper/208:52:002
1028584991612,2cyclictest1120647-21turbostat12:40:002
1028584991512,2cyclictest0-21swapper/211:46:592
1028584991310,2cyclictest0-21swapper/210:10:082
102858499120,9cyclictest0-21swapper/208:35:012
102858499110,9cyclictest0-21swapper/211:30:012
10285849970,6cyclictest0-21swapper/207:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional