You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-09-22 - 06:21
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackdslot1.osadl.org (updated Fri Sep 22, 2023 00:46:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1176911730,9ptp4l0-21swapper/319:05:193
1176911650,42ptp4l216-21systemd-journal19:07:030
1176911310,8ptp4l0-21swapper/119:06:051
1176911110,9ptp4l0-21swapper/219:08:232
117691870,2ptp4l0-21swapper/221:18:572
117691810,10ptp4l0-21swapper/022:35:330
117691760,6ptp4l321ktimersoftd/220:36:312
117691750,5ptp4l216-21systemd-journal23:01:232
117691740,2ptp4l0-21swapper/321:55:013
117691740,22ptp4l20827-21ls21:10:412
117691740,22ptp4l20827-21ls21:10:402
117691720,2ptp4l0-21swapper/321:15:203
117691710,8ptp4l216-21systemd-journal19:46:302
117691700,2ptp4l0-21swapper/223:52:302
117691700,21ptp4l0-21swapper/321:25:183
117691690,5ptp4l11521-21taskset19:43:472
117691680,2ptp4l0-21swapper/300:10:223
117691680,2ptp4l0-21swapper/220:54:412
117691680,2ptp4l0-21swapper/200:25:322
117691680,2ptp4l0-21swapper/023:22:520
117691680,10ptp4l0-21swapper/220:43:032
117691670,5ptp4l1159-21snmpd19:17:562
117691670,2ptp4l0-21swapper/320:28:063
117691670,2ptp4l0-21swapper/300:29:593
117691660,5ptp4l33-21ksoftirqd/222:05:202
117691660,4ptp4l216-21systemd-journal21:51:163
117691660,3ptp4l216-21systemd-journal23:17:433
117691660,19ptp4l0-21swapper/300:20:053
117691650,5ptp4l1159-21snmpd22:56:023
117691650,4ptp4l216-21systemd-journal20:08:193
117691650,2ptp4l0-21swapper/322:41:453
117691650,2ptp4l0-21swapper/319:32:203
117691650,2ptp4l0-21swapper/222:52:592
117691650,2ptp4l0-21swapper/220:47:142
117691650,1ptp4l0-21swapper/200:38:032
117691640,5ptp4l1159-21snmpd23:21:152
27859996323,7cyclictest25-21ksoftirqd/121:00:271
117691630,9ptp4l0-21swapper/021:26:380
117691630,8ptp4l1159-21snmpd20:44:243
117691630,6ptp4l1159-21snmpd21:08:560
117691630,4ptp4l216-21systemd-journal22:00:202
117691630,4ptp4l1185-21lldpd21:35:042
117691630,4ptp4l1185-21lldpd21:10:003
117691630,2ptp4l0-21swapper/323:05:493
117691630,2ptp4l0-21swapper/323:05:493
117691630,2ptp4l0-21swapper/321:40:033
117691630,2ptp4l0-21swapper/320:04:403
117691630,2ptp4l0-21swapper/300:00:393
117691630,2ptp4l0-21swapper/300:00:393
117691620,9ptp4l0-21swapper/022:07:110
117691620,8ptp4l1185-21lldpd20:44:241
117691620,6ptp4l1185-21lldpd19:25:372
117691620,5ptp4l1159-21snmpd23:42:472
117691620,5ptp4l1159-21snmpd22:15:360
117691620,3ptp4l33-21ksoftirqd/221:30:382
117691620,3ptp4l121rcu_preempt23:29:152
117691620,2ptp4l0-21swapper/322:15:393
117691620,2ptp4l0-21swapper/320:32:313
117691620,2ptp4l0-21swapper/319:49:153
117691620,2ptp4l0-21swapper/300:17:313
117691620,2ptp4l0-21swapper/220:25:152
117691620,19ptp4l0-21swapper/322:13:073
117691620,19ptp4l0-21swapper/322:13:063
117691620,15ptp4l0-21swapper/220:13:452
117691610,9ptp4l0-21swapper/119:23:251
117691610,9ptp4l0-21swapper/023:32:210
117691610,9ptp4l0-21swapper/020:17:560
117691610,9ptp4l0-21swapper/020:11:350
117691610,9ptp4l0-21swapper/019:39:170
117691610,5ptp4l1159-21snmpd22:37:212
117691610,4ptp4l1185-21lldpd23:20:331
117691610,2ptp4l0-21swapper/323:26:393
117691610,2ptp4l0-21swapper/322:48:553
117691610,2ptp4l0-21swapper/223:46:252
117691610,2ptp4l0-21swapper/222:47:372
117691610,17ptp4l216-21systemd-journal23:05:132
117691610,17ptp4l216-21systemd-journal23:05:132
117691600,9ptp4l0-21swapper/021:22:260
117691600,8ptp4l0-21swapper/220:24:182
117691600,5ptp4l216-21systemd-journal20:29:270
117691600,4ptp4l19132-21df_abs00:35:233
117691600,4ptp4l1185-21lldpd22:44:471
117691600,4ptp4l1159-21snmpd20:01:450
117691600,3ptp4l1185-21lldpd00:08:342
117691600,3ptp4l1185-21lldpd00:08:342
117691600,3ptp4l1159-21snmpd23:33:472
117691600,2ptp4l0-21swapper/321:30:493
117691600,2ptp4l0-21swapper/321:04:203
117691600,2ptp4l0-21swapper/320:12:483
117691600,17ptp4l0-21swapper/200:30:522
117691600,13ptp4l0-21swapper/222:16:382
27858995927,8cyclictest9-21ksoftirqd/022:30:430
117691590,8ptp4l0-21swapper/019:25:380
117691590,8ptp4l0-21swapper/019:22:440
117691590,5ptp4l1185-21lldpd22:54:391
117691590,5ptp4l1159-21snmpd20:56:420
117691590,4ptp4l1185-21lldpd21:52:110
117691590,4ptp4l1159-21snmpd23:17:411
117691590,3ptp4l4815-21rs:main2
117691590,3ptp4l1159-21snmpd20:59:312
117691590,2ptp4l0-21swapper/321:37:383
117691590,2ptp4l0-21swapper/321:14:333
117691590,2ptp4l0-21swapper/321:14:323
117691590,2ptp4l0-21swapper/223:12:572
117691590,2ptp4l0-21swapper/222:34:412
117691590,2ptp4l0-21swapper/221:23:282
117691590,2ptp4l0-21swapper/219:51:062
117691590,2ptp4l0-21swapper/122:32:131
117691590,2ptp4l0-21swapper/022:27:430
117691590,2ptp4l0-21swapper/000:08:520
117691590,2ptp4l0-21swapper/000:08:520
117691590,15ptp4l0-21swapper/321:21:393
117691590,15ptp4l0-21swapper/220:06:292
117691590,13ptp4l0-21swapper/323:35:413
27858995825,5cyclictest9-21ksoftirqd/021:55:240
27858995825,5cyclictest9-21ksoftirqd/021:15:380
27858995825,5cyclictest9-21ksoftirqd/000:30:250
117691580,9ptp4l33-21ksoftirqd/221:43:392
117691580,5ptp4l1159-21snmpd00:26:290
117691580,2ptp4l0-21swapper/323:49:503
117691580,2ptp4l0-21swapper/223:59:592
117691580,2ptp4l0-21swapper/123:31:251
117691580,1ptp4l0-21swapper/320:37:063
117691580,12ptp4l4815-21rs:main3
117691580,12ptp4l4815-21rs:main3
117691570,5ptp4l1185-21lldpd23:19:120
117691570,4ptp4l14179-21sshd20:57:293
117691570,3ptp4l1159-21snmpd20:12:411
117691570,2ptp4l0-21swapper/322:39:583
117691570,2ptp4l0-21swapper/319:43:353
117691570,2ptp4l0-21swapper/221:56:422
117691570,2ptp4l0-21swapper/221:27:592
117691570,2ptp4l0-21swapper/020:30:360
117691570,2ptp4l0-21swapper/019:40:410
117691570,23ptp4l41-21ksoftirqd/322:04:573
117691570,22ptp4l1159-21snmpd22:44:522
117691570,13ptp4l0-21swapper/221:00:072
27858995626,11cyclictest9-21ksoftirqd/022:24:550
117691560,4ptp4l216-21systemd-journal20:50:430
117691560,4ptp4l1185-21lldpd23:39:110
117691560,3ptp4l4815-21rs:main0
117691560,2ptp4l0-21swapper/323:54:473
117691560,2ptp4l0-21swapper/323:31:573
117691560,2ptp4l0-21swapper/300:30:233
117691560,2ptp4l0-21swapper/221:09:482
117691560,2ptp4l0-21swapper/200:15:062
117691560,2ptp4l0-21swapper/023:53:510
117691560,2ptp4l0-21swapper/022:02:230
117691560,2ptp4l0-21swapper/019:56:230
117691560,12ptp4l0-21swapper/219:14:172
117691560,12ptp4l0-21swapper/219:14:162
27858995526,4cyclictest9-21ksoftirqd/019:20:010
27858995525,5cyclictest9-21ksoftirqd/020:48:100
27858995524,4cyclictest9-21ksoftirqd/022:50:280
27858995523,4cyclictest9-21ksoftirqd/021:40:210
27858995522,5cyclictest9-21ksoftirqd/000:38:170
117691550,7ptp4l33-21ksoftirqd/222:24:552
117691550,5ptp4l1185-21lldpd20:37:501
117691550,4ptp4l216-21systemd-journal22:50:463
117691550,4ptp4l1185-21lldpd00:18:490
117691550,2ptp4l0-21swapper/323:24:373
117691550,2ptp4l0-21swapper/320:49:583
117691550,2ptp4l0-21swapper/320:22:203
117691550,2ptp4l0-21swapper/222:57:492
117691550,2ptp4l0-21swapper/222:29:332
117691550,2ptp4l0-21swapper/220:30:392
117691550,2ptp4l0-21swapper/219:55:222
117691550,2ptp4l0-21swapper/119:32:431
117691550,2ptp4l0-21swapper/119:28:261
117691550,2ptp4l0-21swapper/023:58:210
117691550,2ptp4l0-21swapper/023:04:510
117691550,2ptp4l0-21swapper/021:00:350
117691550,2ptp4l0-21swapper/019:32:220
117691550,1ptp4l0-21swapper/323:13:513
117691550,1ptp4l0-21swapper/220:16:102
117691550,14ptp4l0-21swapper/123:48:311
27858995424,5cyclictest9-21ksoftirqd/020:20:220
27858995422,5cyclictest9-21ksoftirqd/020:10:000
27858995421,6cyclictest9-21ksoftirqd/000:20:240
117691540,3ptp4l33-21ksoftirqd/219:37:352
117691540,2ptp4l0-21swapper/322:25:353
117691540,2ptp4l0-21swapper/322:23:373
117691540,2ptp4l0-21swapper/321:48:283
117691540,2ptp4l0-21swapper/319:59:003
117691540,2ptp4l0-21swapper/319:39:563
117691540,2ptp4l0-21swapper/319:25:063
117691540,2ptp4l0-21swapper/223:15:502
117691540,2ptp4l0-21swapper/222:11:262
117691540,2ptp4l0-21swapper/222:11:252
117691540,2ptp4l0-21swapper/219:20:522
117691540,2ptp4l0-21swapper/200:00:282
117691540,2ptp4l0-21swapper/200:00:282
117691540,2ptp4l0-21swapper/121:21:201
117691540,2ptp4l0-21swapper/119:19:081
117691540,2ptp4l0-21swapper/022:49:090
117691540,2ptp4l0-21swapper/021:13:340
117691540,2ptp4l0-21swapper/021:13:330
117691540,2ptp4l0-21swapper/019:14:260
117691540,2ptp4l0-21swapper/019:14:260
117691540,13ptp4l1185-21lldpd00:22:511
27859995322,7cyclictest25-21ksoftirqd/122:17:221
27859995311,7cyclictest25-21ksoftirqd/122:10:311
27859995311,7cyclictest25-21ksoftirqd/122:10:311
27858995337,4cyclictest9-21ksoftirqd/021:45:260
27858995323,5cyclictest9-21ksoftirqd/020:35:220
27858995322,5cyclictest9-21ksoftirqd/022:55:350
117691530,2ptp4l0-21swapper/322:32:403
117691530,2ptp4l0-21swapper/319:22:023
117691530,2ptp4l0-21swapper/221:50:152
117691530,2ptp4l0-21swapper/221:47:132
117691530,2ptp4l0-21swapper/219:34:522
117691530,2ptp4l0-21swapper/200:14:042
117691530,2ptp4l0-21swapper/120:26:301
117691530,2ptp4l0-21swapper/023:46:350
117691530,2ptp4l0-21swapper/019:52:260
117691530,2ptp4l0-21swapper/019:46:030
117691530,2ptp4l0-21swapper/000:00:020
117691530,2ptp4l0-21swapper/000:00:020
117691530,1ptp4l0-21swapper/323:55:113
2785999522,9cyclictest19276-21/usr/sbin/munin21:10:331
2785999522,9cyclictest19276-21/usr/sbin/munin21:10:331
27859995219,7cyclictest25-21ksoftirqd/120:30:201
27859995213,7cyclictest25-21ksoftirqd/121:35:181
27858995219,5cyclictest9-21ksoftirqd/000:10:340
117691520,4ptp4l216-21systemd-journal19:14:153
117691520,4ptp4l216-21systemd-journal19:14:143
117691520,4ptp4l1159-21snmpd21:54:021
117691520,2ptp4l0-21swapper/023:41:530
2785999513,13cyclictest241ktimersoftd/100:05:481
2785999513,13cyclictest241ktimersoftd/100:05:481
2785999512,7cyclictest1098-21sed23:55:401
27859995113,4cyclictest25-21ksoftirqd/120:05:041
27859995110,7cyclictest25-21ksoftirqd/122:00:241
27858995119,5cyclictest9-21ksoftirqd/022:40:140
27858995119,5cyclictest9-21ksoftirqd/021:32:280
2785999505,7cyclictest25-21ksoftirqd/121:40:281
2785999504,8cyclictest25-21ksoftirqd/122:45:321
2785999502,11cyclictest241ktimersoftd/123:40:221
27859995010,7cyclictest25-21ksoftirqd/123:05:251
27859995010,7cyclictest25-21ksoftirqd/123:05:241
27858995023,8cyclictest9-21ksoftirqd/023:10:200
27858995020,5cyclictest9-21ksoftirqd/023:25:250
27858995019,5cyclictest9-21ksoftirqd/023:05:370
27858995019,5cyclictest9-21ksoftirqd/023:05:370
117691500,2ptp4l0-21swapper/320:17:033
117691500,1ptp4l0-21swapper/119:13:091
117691500,1ptp4l0-21swapper/119:13:091
117691500,16ptp4l1185-21lldpd22:25:151
2785999497,7cyclictest25-21ksoftirqd/100:26:091
2785999494,7cyclictest25-21ksoftirqd/119:55:211
2785999493,8cyclictest1185-21lldpd00:18:101
2785999491,8cyclictest11432-21irqrtprio23:10:291
27858994923,9cyclictest9-21ksoftirqd/022:14:540
27858994923,9cyclictest9-21ksoftirqd/022:14:530
117691490,13ptp4l1185-21lldpd21:35:170
117691490,13ptp4l1185-21lldpd20:59:341
2785999488,7cyclictest25-21ksoftirqd/100:38:491
2785999485,9cyclictest25-21ksoftirqd/119:47:421
2785999484,8cyclictest241ktimersoftd/120:45:381
2785999483,7cyclictest241ktimersoftd/122:35:211
2785999482,7cyclictest23079-21/usr/sbin/munin21:15:401
2785999481,8cyclictest241ktimersoftd/121:45:331
27859994818,10cyclictest25-21ksoftirqd/123:52:501
2785999481,6cyclictest23396-21irqstats23:35:341
2785999480,9cyclictest28149-21basename20:20:161
2785999480,9cyclictest10969-21switchtime19:40:381
117691480,4ptp4l25-21ksoftirqd/121:26:101
117691480,3ptp4l401ktimersoftd/319:19:463
117691480,14ptp4l25-21ksoftirqd/122:23:041
117691480,14ptp4l1159-21snmpd22:05:371
2785999474,8cyclictest241ktimersoftd/100:30:221
2785999472,8cyclictest241ktimersoftd/122:55:241
2785999471,8cyclictest7374-21df_abs00:10:271
2785999471,8cyclictest1159-21snmpd23:05:001
2785999470,8cyclictest25-21ksoftirqd/120:00:401
2785999470,7cyclictest1159-21snmpd23:25:221
117691470,8ptp4l0-21swapper/322:05:563
117691470,6ptp4l0-21swapper/319:50:033
2785999464,8cyclictest241ktimersoftd/120:50:321
2785999463,7cyclictest241ktimersoftd/120:15:191
2785999463,7cyclictest241ktimersoftd/119:50:441
117691460,12ptp4l1159-21snmpd21:06:461
2785999452,7cyclictest241ktimersoftd/121:55:251
2785999452,6cyclictest241ktimersoftd/119:35:371
117691450,10ptp4l0-21swapper/320:50:243
2785999443,6cyclictest241ktimersoftd/121:32:181
27859994337,3cyclictest231rcuc/100:03:231
27859994337,3cyclictest231rcuc/100:03:231
117691430,7ptp4l216-21systemd-journal23:35:062
117691430,2ptp4l0-21swapper/323:40:223
117691410,5ptp4l216-21systemd-journal23:01:533
117691410,3ptp4l1185-21lldpd20:02:082
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional