You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-02-25 - 12:18
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot6 (updated Sun Feb 25, 2024 00:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
116347999115105,7cyclictest1250898-21gpgv23:40:011
11634769910797,4cyclictest1158757-21kworker/u8:2+events_unbound19:15:360
11634769910498,4cyclictest1237763-21kworker/u8:3+events_unbound00:05:240
11634769910289,9cyclictest1221364-21latency_hist22:10:000
11634769910196,4cyclictest1179475-21kworker/u8:0+events_unbound20:00:360
11634769910092,3cyclictest1224771-21kworker/u8:1+events_unbound22:30:330
1163476999995,3cyclictest1264032-21kworker/u8:2+events_unbound00:25:250
1163476999990,4cyclictest1269069-21kworker/u8:3+events_unbound00:35:340
1163476999893,3cyclictest1226381-21kworker/u8:0+events_unbound22:25:240
1163476999792,4cyclictest1249157-21kworker/u8:2+events_unbound23:40:400
1163476999690,4cyclictest1158757-21kworker/u8:2+events_unbound19:55:330
1163476999690,4cyclictest1158757-21kworker/u8:2+events_unbound19:55:320
1163476999588,4cyclictest1237763-21kworker/u8:3+events_unbound23:55:280
1163476999388,4cyclictest1251268-21kworker/u8:0+events_unbound23:45:450
1163476999388,4cyclictest1226381-21kworker/u8:0+events_unbound22:40:360
1163476999287,4cyclictest1267483-21kworker/u8:0+events_unbound00:30:340
1163476999287,4cyclictest1267483-21kworker/u8:0+events_unbound00:30:340
1163476999286,4cyclictest1158757-21kworker/u8:2+events_unbound20:15:250
1163479999184,4cyclictest1226381-21kworker/u8:0+flush-179:023:05:311
1163479999167,15cyclictest1174470-21kworker/u8:1+flush-179:020:05:311
1163476999187,3cyclictest1203848-21kworker/u8:3+events_unbound21:50:300
1163476999186,4cyclictest1226381-21kworker/u8:0+events_unbound23:10:360
1163476999085,4cyclictest1237763-21kworker/u8:3+events_unbound00:20:250
1163476999085,4cyclictest1149935-21kworker/u8:0+events_unbound19:30:350
1163476998985,3cyclictest1221859-21kworker/u8:2+events_unbound22:55:350
1163476998883,4cyclictest1226381-21kworker/u8:0+events_unbound23:05:290
1163476998883,3cyclictest1211794-21kworker/u8:0+events_unbound21:40:350
1163476998876,10cyclictest1254454-21apt-get23:50:190
1163476998580,4cyclictest1224771-21kworker/u8:1+events_unbound22:35:310
1163476998578,5cyclictest1158757-21kworker/u8:2+flush-179:020:30:000
1163476998576,4cyclictest1156820-21kworker/u8:3+events_unbound19:20:300
1163476998574,8cyclictest1202287-21latency_hist21:10:010
1163476998479,4cyclictest1179475-21kworker/u8:0+events_unbound20:05:290
1163476998476,3cyclictest1256519-21kworker/u8:2+events_unbound00:00:300
1163476998378,4cyclictest1205433-21kworker/u8:2+events_unbound22:00:280
1163476998378,3cyclictest1166504-21kworker/u8:1+events_unbound19:25:210
1163476998373,4cyclictest1205433-21kworker/u8:2+events_unbound21:25:340
1163476998277,4cyclictest1256519-21kworker/u8:2+events_unbound00:10:280
1163476998276,5cyclictest1192222-21apt-get20:40:010
1163476998076,2cyclictest1149935-21kworker/u8:0+events_unbound19:40:250
1163476997970,7cyclictest1163321-21latency_hist19:10:020
1163479997843,20cyclictest0-21swapper/100:00:021
1163476997772,4cyclictest1226381-21kworker/u8:0+events_unbound22:45:030
1163476997767,4cyclictest1149935-21kworker/u8:0+events_unbound19:10:280
1163476997672,3cyclictest1221859-21kworker/u8:2+events_unbound23:00:370
1163476997672,3cyclictest1179475-21kworker/u8:0+events_unbound20:10:340
1163479997568,4cyclictest1237763-21kworker/u8:3+flush-179:023:30:011
1163476997470,3cyclictest1184254-21kworker/u8:1+events_unbound20:20:240
1163476997469,4cyclictest1216570-21kworker/u8:0+events_unbound22:10:210
1163479997368,3cyclictest1181090-21kworker/u8:3+flush-179:020:10:341
1163476997369,3cyclictest1211794-21kworker/u8:0+events_unbound21:45:370
1163476997368,4cyclictest1226381-21kworker/u8:0+events_unbound23:17:570
1163476997368,4cyclictest1185846-21kworker/u8:0+events_unbound20:59:570
1163476997362,4cyclictest1189036-21kworker/u8:3+events_unbound20:45:370
1163479997221,5cyclictest1221773-21cut22:10:011
1163479997165,4cyclictest1200958-21kworker/u8:0+flush-179:021:05:251
1163479997162,7cyclictest1189286-21/usr/sbin/munin20:30:251
1163476997166,4cyclictest1237763-21kworker/u8:3+events_unbound00:15:390
1163479997060,7cyclictest1249167-21latency_hist23:35:001
1163476997066,3cyclictest1205433-21kworker/u8:2+events_unbound21:55:360
1163476997059,4cyclictest1208884-21kworker/u8:1+events_unbound21:39:580
1163479996963,4cyclictest1264032-21kworker/u8:2+flush-179:000:25:241
1163479996859,7cyclictest1197766-21apt-get20:55:201
1163476996864,3cyclictest1184254-21kworker/u8:1+events_unbound20:35:320
1163476996863,4cyclictest1237763-21kworker/u8:3+events_unbound23:29:590
1163476996860,5cyclictest1200701-21latency_hist21:05:010
1163479996759,6cyclictest1234585-21kworker/u8:1+flush-179:022:55:301
1163479996655,9cyclictest1241214-21apt-get23:10:221
1163479996636,29cyclictest0-21swapper/121:22:591
1163476996661,4cyclictest1158757-21kworker/u8:2+events_unbound20:31:590
1163476996661,3cyclictest1249157-21kworker/u8:2+events_unbound23:35:290
1163476996657,3cyclictest1195909-21kworker/u8:1+events_unbound20:50:310
1163479996558,5cyclictest1269069-21kworker/u8:3+flush-179:000:35:311
1163479996557,6cyclictest1210231-21latency_hist21:35:011
1163479996458,4cyclictest1234585-21kworker/u8:1+flush-179:022:50:001
1163479996442,4cyclictest1174484-21latency_hist19:45:011
1163476996460,3cyclictest1195909-21kworker/u8:1+events_unbound21:11:590
1163476996459,4cyclictest1221859-21kworker/u8:2+events_unbound22:17:590
1163476996459,4cyclictest1208884-21kworker/u8:1+events_unbound21:32:590
1163476996459,4cyclictest1156820-21kworker/u8:3+events_unbound19:37:580
1163479996359,3cyclictest1226381-21kworker/u8:0+flush-179:022:40:311
1163479996353,8cyclictest1224771-21kworker/u8:1+flush-179:022:20:001
1163476996358,4cyclictest1226381-21kworker/u8:0+events_unbound22:54:580
1163476996257,4cyclictest1158757-21kworker/u8:2+events_unbound19:53:580
1163479996143,8cyclictest725-21Xorg22:35:591
1163476996056,3cyclictest1158757-21kworker/u8:2+events_unbound19:45:590
1163476995854,3cyclictest1200958-21kworker/u8:0+flush-179:021:20:310
1163479995749,6cyclictest2158010-21systemd-journal00:31:581
1163479995749,6cyclictest2158010-21systemd-journal00:31:571
1163479995648,5cyclictest1264312-21/usr/sbin/munin00:20:321
1163479995617,2cyclictest1244407-21apt-get23:20:211
1163476995652,3cyclictest1245723-21kworker/u8:1+events_unbound23:30:250
1163479995548,5cyclictest2158010-21systemd-journal00:08:561
1163479995546,7cyclictest1192222-21apt-get20:40:001
1163479995545,8cyclictest2158010-21systemd-journal22:30:011
1163479995447,5cyclictest1185846-21kworker/u8:0+flush-179:020:20:001
1163479995317,24cyclictest1195365-21ntp_kernel_pll_20:45:331
1163476995346,5cyclictest1226401-21latency_hist22:25:010
116347999527,43cyclictest0-21swapper/119:45:181
1163479995248,3cyclictest1240946-21kworker/u8:1+flush-179:023:15:291
1163479995246,4cyclictest1216570-21kworker/u8:0+flush-179:021:55:261
1163479995245,5cyclictest1229570-21latency_hist22:35:011
1163476995247,4cyclictest1242532-21kworker/u8:2+events_unbound23:23:570
1163479995146,4cyclictest0-21swapper/100:04:591
1163479995144,5cyclictest1224771-21kworker/u8:1+flush-179:022:20:311
1163479995143,7cyclictest1166664-21cat19:20:001
1163479995142,7cyclictest0-21swapper/121:50:231
1163479995135,10cyclictest1218158-21kworker/u8:3+flush-179:022:00:311
1163479995113,36cyclictest0-21swapper/122:10:201
1163479995043,4cyclictest1211818-21latency_hist21:40:011
1163479995041,7cyclictest2158010-21systemd-journal21:00:011
1163479994942,5cyclictest2158010-21systemd-journal20:42:581
1163479994842,4cyclictest1236387-21grep22:55:011
1163479994842,4cyclictest1176580-21apt-get19:50:211
1163479994842,4cyclictest1168380-21apt-get19:25:181
1163479994841,6cyclictest1166776-21latency_hist19:20:011
1163479994814,32cyclictest725-21Xorg20:00:001
1163479994814,32cyclictest725-21Xorg19:59:591
4652991470,13ptp4l61790irq/134-eno1-TxRx-019:10:003
1163485994741,4cyclictest1250785-21turbostat23:40:003
1163479994741,4cyclictest1149935-21kworker/u8:0+flush-179:019:30:251
116347999470,9cyclictest1164944-21basename19:15:001
1163479994640,4cyclictest1262724-21apt-get00:15:231
1163479994539,4cyclictest1195909-21kworker/u8:1+flush-179:021:10:231
1163479994538,6cyclictest1185846-21kworker/u8:0+flush-179:020:25:321
1163479994537,6cyclictest1237763-21kworker/u8:3+flush-179:023:50:271
1163479994534,8cyclictest1196180-21apt-get20:50:201
1163479994514,2cyclictest1212093-21apt-get21:40:241
1163479994514,16cyclictest1187494-21basename20:25:001
1163479994440,2cyclictest725-21Xorg20:04:591
1163479994431,10cyclictest2158010-21systemd-journal19:39:591
1163476994439,4cyclictest1203848-21kworker/u8:3+events_unbound21:16:570
1163485994340,2cyclictest0-21swapper/320:57:593
116348599430,42cyclictest0-21swapper/319:29:593
1163479994337,5cyclictest1200958-21kworker/u8:0+flush-179:021:15:271
1163479994332,9cyclictest2158010-21systemd-journal23:40:031
1163479994321,7cyclictest1253845-21mailstats23:45:441
116347999420,40cyclictest725-21Xorg00:13:001
1163481994137,2cyclictest0-21swapper/220:26:582
116348199410,2cyclictest0-21swapper/219:24:002
116348199410,1cyclictest0-21swapper/220:57:592
1163479994134,5cyclictest1205433-21kworker/u8:2+events_unbound21:25:321
1163479994118,8cyclictest1214411-21ntp_192.168.11421:45:301
1163485994038,1cyclictest0-21swapper/319:23:593
1163481994037,2cyclictest0-21swapper/200:26:002
116348199400,38cyclictest0-21swapper/219:32:592
1163479994019,5cyclictest725-21Xorg23:04:001
1163485993936,2cyclictest0-21swapper/321:03:593
116348599390,38cyclictest0-21swapper/323:09:593
1163485993836,1cyclictest0-21swapper/319:33:003
1163481993835,2cyclictest0-21swapper/220:45:232
116348199380,36cyclictest0-21swapper/222:17:592
116348199380,35cyclictest0-21swapper/222:59:592
116348199380,35cyclictest0-21swapper/200:02:582
1163479993816,5cyclictest0-21swapper/119:10:021
4652991370,3ptp4l441ktimers/323:10:253
1163485993734,2cyclictest0-21swapper/323:55:583
1163485993734,2cyclictest0-21swapper/322:59:593
1163485993734,2cyclictest0-21swapper/322:35:593
1163485993734,2cyclictest0-21swapper/300:17:593
1163485993734,2cyclictest0-21swapper/300:02:593
116348599370,36cyclictest0-21swapper/322:53:593
1163481993734,2cyclictest0-21swapper/223:58:002
1163481993734,2cyclictest0-21swapper/223:35:592
1163481993734,2cyclictest0-21swapper/223:09:552
1163481993734,2cyclictest0-21swapper/222:25:592
1163481993733,2cyclictest0-21swapper/221:03:592
116348199370,35cyclictest0-21swapper/219:18:002
116348199370,1cyclictest0-21swapper/200:17:582
1163485993634,1cyclictest0-21swapper/320:09:583
1163485993634,1cyclictest0-21swapper/300:25:593
1163485993634,1cyclictest0-21swapper/300:08:593
116348599360,35cyclictest0-21swapper/323:47:003
116348599360,35cyclictest0-21swapper/319:44:543
1163481993633,2cyclictest0-21swapper/223:18:592
1163481993633,2cyclictest0-21swapper/221:54:592
1163481993633,2cyclictest0-21swapper/221:25:592
1163481993633,2cyclictest0-21swapper/219:12:582
1163481993633,2cyclictest0-21swapper/200:23:592
1163481993632,2cyclictest0-21swapper/221:45:592
116348199360,1cyclictest0-21swapper/222:35:592
116348199360,1cyclictest0-21swapper/200:08:592
1163485993532,2cyclictest0-21swapper/323:45:003
1163485993532,2cyclictest0-21swapper/321:36:593
1163485993532,2cyclictest0-21swapper/320:43:593
1163485993532,2cyclictest0-21swapper/320:26:583
116348599351,28cyclictest67-21kworker/3:1+events22:45:023
116348599350,1cyclictest0-21swapper/321:58:593
1163481993533,1cyclictest0-21swapper/222:00:002
1163481993532,2cyclictest0-21swapper/221:30:592
1163481993532,2cyclictest0-21swapper/220:30:202
116348199350,1cyclictest0-21swapper/221:38:592
116348199350,1cyclictest0-21swapper/220:09:592
1163485993432,1cyclictest0-21swapper/319:18:003
1163485993431,2cyclictest0-21swapper/321:27:593
1163485993331,1cyclictest0-21swapper/323:53:583
1163485993331,1cyclictest0-21swapper/323:30:243
1163485993331,1cyclictest0-21swapper/322:17:593
1163485993331,1cyclictest0-21swapper/320:45:233
1163485993331,1cyclictest0-21swapper/319:11:593
1163485993330,2cyclictest0-21swapper/321:25:003
1163485993330,2cyclictest0-21swapper/321:11:593
1163481993330,2cyclictest0-21swapper/223:41:582
1163481993330,2cyclictest0-21swapper/222:15:012
1163481993330,2cyclictest0-21swapper/221:10:212
1163481993330,2cyclictest0-21swapper/220:45:012
1163481993330,2cyclictest0-21swapper/220:38:592
1163481993330,2cyclictest0-21swapper/220:11:582
1163481993329,2cyclictest0-21swapper/220:24:592
1163485993229,2cyclictest0-21swapper/300:35:013
1163485993229,2cyclictest0-21swapper/300:35:003
116348599320,30cyclictest0-21swapper/300:10:253
1163481993229,2cyclictest0-21swapper/223:52:592
1163481993229,2cyclictest0-21swapper/219:45:012
1163481993229,2cyclictest0-21swapper/219:36:002
1163485993129,1cyclictest0-21swapper/322:25:593
1163485993128,2cyclictest0-21swapper/321:54:593
1163481993128,2cyclictest0-21swapper/223:11:592
1163481993128,2cyclictest0-21swapper/220:50:212
116348199310,29cyclictest0-21swapper/223:00:232
1163485993027,2cyclictest0-21swapper/322:05:213
1163485993027,2cyclictest0-21swapper/321:45:593
116348599300,2cyclictest0-21swapper/323:00:233
116348599300,2cyclictest0-21swapper/322:00:203
116348599300,2cyclictest0-21swapper/321:45:013
116348599300,2cyclictest0-21swapper/320:35:013
116348599300,29cyclictest0-21swapper/323:30:013
116348599300,29cyclictest0-21swapper/320:35:203
116348599300,29cyclictest0-21swapper/319:50:223
1163481993027,2cyclictest0-21swapper/222:35:012
1163481993027,2cyclictest0-21swapper/222:05:222
1163481993027,2cyclictest0-21swapper/200:35:002
1163481993027,2cyclictest0-21swapper/200:35:002
116348199300,2cyclictest0-21swapper/219:50:232
116348199300,28cyclictest0-21swapper/223:30:002
116348199300,28cyclictest0-21swapper/221:45:012
116348199300,28cyclictest0-21swapper/220:15:222
116348199300,28cyclictest0-21swapper/219:55:202
116348199300,28cyclictest0-21swapper/219:55:202
116348199300,28cyclictest0-21swapper/219:40:222
116348199300,28cyclictest0-21swapper/200:10:032
1163485992927,1cyclictest0-21swapper/323:18:593
1163485992927,1cyclictest0-21swapper/322:15:013
1163485992927,1cyclictest0-21swapper/321:30:233
116348599290,28cyclictest0-21swapper/323:20:233
116348599290,28cyclictest0-21swapper/320:50:213
116348599290,28cyclictest0-21swapper/320:15:223
116348599290,28cyclictest0-21swapper/319:55:213
116348599290,28cyclictest0-21swapper/319:55:203
116348599290,28cyclictest0-21swapper/300:20:213
116348599290,27cyclictest0-21swapper/321:05:343
116348599290,21cyclictest0-21swapper/321:15:223
116348599290,1cyclictest0-21swapper/319:45:023
1163481992926,2cyclictest0-21swapper/223:30:012
1163481992926,2cyclictest0-21swapper/223:20:242
1163481992926,2cyclictest0-21swapper/222:40:212
1163481992926,2cyclictest0-21swapper/221:10:012
1163481992926,2cyclictest0-21swapper/220:00:222
1163481992925,2cyclictest0-21swapper/221:15:222
116348199290,2cyclictest0-21swapper/222:00:212
116348199290,27cyclictest0-21swapper/222:45:022
1163485992825,2cyclictest0-21swapper/320:00:233
116348599280,2cyclictest0-21swapper/322:30:233
116348599280,27cyclictest0-21swapper/322:40:213
116348599280,27cyclictest0-21swapper/320:20:253
116348599280,27cyclictest0-21swapper/320:10:583
116348599280,27cyclictest0-21swapper/300:35:233
1163481992825,2cyclictest0-21swapper/223:45:222
116348199280,2cyclictest0-21swapper/200:35:242
116348199280,27cyclictest0-21swapper/221:20:312
116348199280,26cyclictest0-21swapper/222:50:292
116348199280,26cyclictest0-21swapper/222:24:592
116348599270,26cyclictest0-21swapper/322:24:593
116348599270,1cyclictest0-21swapper/319:35:233
1163481992723,2cyclictest0-21swapper/219:25:272
11634819930,2cyclictest0-21swapper/219:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional