You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-06-05 - 15:54

x86 Intel Atom E3845 @1910 MHz, Linux 4.19.37-rt19 (Profile)

Latency plot of system in rack #d, slot #1
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Up99 -i200 -h400 --smi -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rackdslot1.osadl.org (updated Mon Jun 05, 2023 12:46:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1176911480,10ptp4l0-21swapper/307:06:113
1176911280,9ptp4l216-21systemd-journal07:09:372
1176911260,15ptp4l216-21systemd-journal07:07:521
351821130,4sleep12952599cyclictest09:40:301
351821130,4sleep12952599cyclictest09:40:301
1176911090,27ptp4l1769-21gmain07:05:350
117691900,8ptp4l0-21swapper/107:10:011
117691780,11ptp4l241ktimersoftd/108:59:521
117691750,30ptp4l9-21ksoftirqd/010:23:420
117691730,10ptp4l25-21ksoftirqd/112:21:121
91012700,3sleep10-21swapper/109:54:591
117691650,13ptp4l0-21swapper/207:24:232
117691630,14ptp4l0-21swapper/111:51:241
117691630,13ptp4l0-21swapper/112:15:471
29524996223,8cyclictest9-21ksoftirqd/007:25:440
117691620,3ptp4l9-21ksoftirqd/009:29:390
117691620,13ptp4l0-21swapper/111:08:391
117691620,13ptp4l0-21swapper/110:06:031
29524996126,7cyclictest9-21ksoftirqd/008:30:380
29524996123,7cyclictest9-21ksoftirqd/008:12:100
117691610,7ptp4l241ktimersoftd/109:10:381
117691610,12ptp4l0-21swapper/208:41:402
117691610,12ptp4l0-21swapper/112:32:261
117691610,12ptp4l0-21swapper/112:32:261
117691610,12ptp4l0-21swapper/110:39:171
117691610,11ptp4l0-21swapper/108:53:131
117691610,11ptp4l0-21swapper/108:53:131
29524996021,8cyclictest9-21ksoftirqd/008:51:450
29524996021,8cyclictest9-21ksoftirqd/008:51:440
117691600,12ptp4l0-21swapper/111:56:411
117691600,12ptp4l0-21swapper/110:51:251
29524995925,7cyclictest9-21ksoftirqd/010:29:290
29524995923,7cyclictest9-21ksoftirqd/011:40:340
29524995922,6cyclictest9-21ksoftirqd/012:00:330
117691590,11ptp4l0-21swapper/112:29:081
2952499588,5cyclictest9-21ksoftirqd/010:36:570
29524995824,7cyclictest9-21ksoftirqd/011:53:350
29524995824,7cyclictest9-21ksoftirqd/009:47:000
29524995823,8cyclictest9-21ksoftirqd/008:08:560
29524995817,7cyclictest9-21ksoftirqd/011:25:270
29524995815,12cyclictest9-21ksoftirqd/007:43:540
117691580,5ptp4l1159-21snmpd10:30:302
117691580,11ptp4l0-21swapper/109:25:291
117691580,10ptp4l0-21swapper/109:20:101
117691580,10ptp4l0-21swapper/107:58:061
117691580,10ptp4l0-21swapper/107:58:061
29524995724,8cyclictest9-21ksoftirqd/009:22:030
29524995724,7cyclictest9-21ksoftirqd/008:40:550
29524995724,6cyclictest9-21ksoftirqd/012:10:260
29524995723,7cyclictest9-21ksoftirqd/009:36:290
29524995722,7cyclictest9-21ksoftirqd/008:27:550
29524995718,7cyclictest9-21ksoftirqd/011:05:380
117691570,2ptp4l0-21swapper/111:32:101
117691570,10ptp4l0-21swapper/107:51:251
2952799567,16cyclictest1721-21gnome-shell11:20:593
29524995623,7cyclictest9-21ksoftirqd/012:18:040
29524995623,7cyclictest9-21ksoftirqd/009:17:000
29524995623,7cyclictest9-21ksoftirqd/007:47:570
29524995622,9cyclictest9-21ksoftirqd/010:33:320
29524995622,8cyclictest9-21ksoftirqd/012:23:050
29524995622,8cyclictest9-21ksoftirqd/011:20:030
29524995622,8cyclictest9-21ksoftirqd/010:53:320
29524995622,8cyclictest9-21ksoftirqd/010:14:310
29524995621,7cyclictest9-21ksoftirqd/011:49:050
29524995618,7cyclictest9-21ksoftirqd/012:25:050
117691560,4ptp4l216-21systemd-journal11:44:361
117691560,3ptp4l121rcu_preempt07:57:490
117691560,3ptp4l121rcu_preempt07:57:490
117691560,2ptp4l0-21swapper/210:24:072
117691560,21ptp4l25-21ksoftirqd/110:46:341
117691560,18ptp4l33-21ksoftirqd/207:40:472
29524995525,6cyclictest9-21ksoftirqd/012:05:190
29524995524,7cyclictest9-21ksoftirqd/008:47:560
29524995523,7cyclictest9-21ksoftirqd/009:51:440
29524995523,7cyclictest9-21ksoftirqd/008:18:250
29524995523,7cyclictest9-21ksoftirqd/008:02:250
29524995522,8cyclictest9-21ksoftirqd/007:39:110
29524995522,7cyclictest9-21ksoftirqd/009:34:590
29524995522,7cyclictest9-21ksoftirqd/008:56:300
29524995516,6cyclictest9-21ksoftirqd/007:20:450
148912550,2sleep30-21swapper/307:45:373
117691550,5ptp4l1185-21lldpd07:13:520
117691550,17ptp4l41-21ksoftirqd/310:55:353
2952699548,19cyclictest0-21swapper/208:51:462
2952699548,19cyclictest0-21swapper/208:51:452
29524995422,8cyclictest9-21ksoftirqd/012:33:050
29524995422,8cyclictest9-21ksoftirqd/012:33:040
29524995422,7cyclictest9-21ksoftirqd/011:01:570
29524995421,8cyclictest9-21ksoftirqd/011:55:580
29524995421,8cyclictest9-21ksoftirqd/011:30:150
29524995421,7cyclictest9-21ksoftirqd/007:53:580
29524995419,8cyclictest9-21ksoftirqd/010:18:030
29524995416,7cyclictest9-21ksoftirqd/012:37:340
29524995416,7cyclictest9-21ksoftirqd/012:37:340
117691540,5ptp4l1159-21snmpd10:41:450
117691540,5ptp4l1159-21snmpd10:41:440
117691540,3ptp4l41-21ksoftirqd/312:02:183
117691540,3ptp4l1185-21lldpd12:27:422
117691540,2ptp4l0-21swapper/111:00:191
29524995323,7cyclictest9-21ksoftirqd/010:06:590
29524995323,7cyclictest9-21ksoftirqd/008:36:550
117791530,13getstats0-21swapper/207:28:212
117791530,12getstats0-21swapper/208:13:182
117691530,19ptp4l1185-21lldpd08:49:582
2952799528,20cyclictest1159-21snmpd11:47:133
29524995225,6cyclictest9-21ksoftirqd/009:03:270
29524995223,8cyclictest9-21ksoftirqd/011:16:320
29524995223,10cyclictest9-21ksoftirqd/011:35:260
29524995223,10cyclictest9-21ksoftirqd/011:35:260
29524995222,7cyclictest9-21ksoftirqd/009:59:490
29524995221,8cyclictest9-21ksoftirqd/010:58:450
29524995221,7cyclictest9-21ksoftirqd/009:10:120
117691520,4ptp4l321ktimersoftd/212:30:042
117691520,4ptp4l321ktimersoftd/212:30:032
117691520,2ptp4l0-21swapper/111:22:391
117691520,2ptp4l0-21swapper/109:30:351
117691520,18ptp4l1159-21snmpd12:07:122
29524995124,7cyclictest9-21ksoftirqd/007:33:280
29524995124,6cyclictest9-21ksoftirqd/008:22:550
29524995120,7cyclictest9-21ksoftirqd/009:07:470
29524995119,7cyclictest9-21ksoftirqd/010:04:070
117691510,2ptp4l0-21swapper/111:26:081
117691510,2ptp4l0-21swapper/110:01:361
117691510,2ptp4l0-21swapper/109:45:441
117691510,2ptp4l0-21swapper/108:39:261
117691510,2ptp4l0-21swapper/108:04:031
117691510,18ptp4l1159-21snmpd08:34:382
117691510,15ptp4l1185-21lldpd11:38:451
117691510,15ptp4l1185-21lldpd11:38:451
117691510,13ptp4l9996-21apt-get12:15:202
117691510,10ptp4l0-21swapper/210:50:122
2952699505,13cyclictest1159-21snmpd09:01:102
29524995026,8cyclictest9-21ksoftirqd/010:45:240
29524995024,7cyclictest9-21ksoftirqd/011:12:030
29524995024,7cyclictest9-21ksoftirqd/009:41:590
29524995024,7cyclictest9-21ksoftirqd/009:41:590
117691500,6ptp4l0-21swapper/211:59:362
117691500,3ptp4l33-21ksoftirqd/210:28:332
117691500,2ptp4l0-21swapper/110:34:151
117691500,2ptp4l0-21swapper/107:46:441
117691500,13ptp4l23089-21apt-get11:31:172
2952699497,13cyclictest1159-21snmpd08:57:582
2952699495,12cyclictest1185-21lldpd09:41:482
2952699495,12cyclictest1185-21lldpd09:41:482
117691490,2ptp4l0-21swapper/112:02:301
117691490,2ptp4l0-21swapper/107:42:151
117691490,15ptp4l1185-21lldpd11:53:242
117691490,15ptp4l1159-21snmpd07:39:231
117691490,14ptp4l1185-21lldpd10:24:003
117691490,13ptp4l4813-21in:imuxsock12:35:322
117691490,13ptp4l4813-21in:imuxsock12:35:322
2952699486,13cyclictest1159-21snmpd07:15:382
2952699485,13cyclictest1185-21lldpd08:01:382
2952699483,15cyclictest1185-21lldpd10:36:232
29526994824,19cyclictest4815-21rs:main2
117691480,9ptp4l311rcuc/211:06:502
117691480,9ptp4l1159-21snmpd09:55:322
117691480,4ptp4l401ktimersoftd/311:11:353
117691480,3ptp4l121rcu_preempt11:25:322
117691480,2ptp4l0-21swapper/211:19:142
117691480,2ptp4l0-21swapper/110:22:121
117691480,2ptp4l0-21swapper/007:16:180
117691480,1ptp4l0-21swapper/112:13:501
117691480,14ptp4l1185-21lldpd09:15:421
117691480,14ptp4l1185-21lldpd09:05:262
117691480,14ptp4l1159-21snmpd10:11:212
117691480,13ptp4l1185-21lldpd10:58:361
117691480,13ptp4l1185-21lldpd09:58:061
117691480,13ptp4l1185-21lldpd08:09:112
2952799476,17cyclictest1185-21lldpd08:16:573
2952799475,17cyclictest1159-21snmpd12:24:303
2952699478,12cyclictest1159-21snmpd10:07:522
2952699477,13cyclictest1159-21snmpd07:36:332
2952699477,13cyclictest1159-21snmpd07:13:482
2952699476,13cyclictest1185-21lldpd10:42:022
2952699476,13cyclictest1185-21lldpd10:42:022
2952699476,13cyclictest1185-21lldpd10:19:032
2952699476,13cyclictest1159-21snmpd07:57:292
2952699476,13cyclictest1159-21snmpd07:57:292
2952699476,11cyclictest1159-21snmpd12:21:492
2952699475,12cyclictest1185-21lldpd08:17:552
117691470,9ptp4l311rcuc/211:49:422
117691470,6ptp4l17162-21needreboot09:00:311
117691470,2ptp4l0-21swapper/210:55:242
117691470,13ptp4l4813-21in:imuxsock09:32:562
117691470,12ptp4l1185-21lldpd11:37:502
117691470,12ptp4l1185-21lldpd11:37:492
117691470,12ptp4l1185-21lldpd11:14:201
117691470,12ptp4l1185-21lldpd09:08:451
117691470,12ptp4l1185-21lldpd07:53:222
2952799467,16cyclictest1159-21snmpd07:59:443
2952799467,16cyclictest1159-21snmpd07:59:443
2952799464,13cyclictest1185-21lldpd10:43:353
2952799464,13cyclictest1185-21lldpd10:43:353
2952799463,13cyclictest1185-21lldpd09:02:433
2952699468,12cyclictest1185-21lldpd10:01:012
2952699467,13cyclictest1185-21lldpd07:46:322
2952699467,12cyclictest1185-21lldpd08:37:552
2952699466,13cyclictest1185-21lldpd09:35:552
2952699466,12cyclictest1159-21snmpd11:43:272
2952699465,13cyclictest1185-21lldpd09:14:582
2952699465,13cyclictest1185-21lldpd08:21:132
2952699465,12cyclictest1159-21snmpd09:54:462
117791460,10getstats0-21swapper/111:15:231
117691460,9ptp4l241ktimersoftd/108:29:391
117691460,9ptp4l0-21swapper/311:56:203
117691460,7ptp4l1159-21snmpd11:49:141
117691460,7ptp4l0-21swapper/212:03:012
117691460,4ptp4l401ktimersoftd/308:52:363
117691460,4ptp4l401ktimersoftd/308:52:363
117691460,2ptp4l121rcu_preempt08:48:063
117691460,13ptp4l1159-21snmpd08:28:042
117691460,12ptp4l216-21systemd-journal10:11:471
117691460,12ptp4l1185-21lldpd12:11:232
117691460,10ptp4l1185-21lldpd10:48:022
2952799457,19cyclictest1185-21lldpd09:37:263
2952799456,18cyclictest1159-21snmpd12:33:423
2952799456,18cyclictest1159-21snmpd12:33:423
2952799456,18cyclictest1159-21snmpd11:39:033
2952799456,18cyclictest1159-21snmpd11:39:033
2952799456,11cyclictest216-21systemd-journal08:28:593
2952799453,13cyclictest1185-21lldpd07:33:253
2952699459,18cyclictest1185-21lldpd09:20:232
2952699457,14cyclictest1185-21lldpd11:01:032
2952699455,14cyclictest1185-21lldpd11:10:482
117691450,7ptp4l1185-21lldpd09:26:582
117691450,5ptp4l8375-21runrttasks07:32:352
117691450,2ptp4l0-21swapper/309:54:383
117691450,11ptp4l0-21swapper/209:49:062
2952799445,18cyclictest1185-21lldpd09:21:083
2952799445,18cyclictest1185-21lldpd08:22:413
2952799445,12cyclictest1185-21lldpd11:06:263
2952799445,11cyclictest4815-21rs:main3
2952799444,12cyclictest4813-21in:imuxsock09:28:243
2952799444,11cyclictest1185-21lldpd10:50:593
2952799443,13cyclictest216-21systemd-journal07:21:503
2952799442,13cyclictest1159-21snmpd09:07:133
2952799442,12cyclictest1159-21snmpd08:14:153
117791440,9getstats0-21swapper/110:17:351
117691440,7ptp4l0-21swapper/110:25:061
117691440,3ptp4l391rcuc/310:34:313
117691440,10ptp4l0-21swapper/211:23:202
117691440,10ptp4l0-21swapper/112:08:201
117691440,10ptp4l0-21swapper/108:21:331
2952799437,12cyclictest26986-21meminfo11:40:323
2952799434,13cyclictest1185-21lldpd10:17:253
2952799434,13cyclictest1185-21lldpd10:13:483
29527994318,18cyclictest11742-21apt-get10:00:183
2952799431,12cyclictest1185-21lldpd12:19:083
117791430,8getstats0-21swapper/107:17:221
117691430,9ptp4l0-21swapper/107:32:021
117691430,8ptp4l391rcuc/310:36:203
117691430,8ptp4l1159-21snmpd08:19:031
117691430,5ptp4l1159-21snmpd10:29:163
117691430,4ptp4l1185-21lldpd07:21:311
117691430,4ptp4l1159-21snmpd08:55:513
117691430,2ptp4l0-21swapper/307:44:333
117691430,10ptp4l1185-21lldpd09:39:051
2952799424,12cyclictest1159-21snmpd11:30:313
2952799424,12cyclictest1159-21snmpd09:34:203
2952799423,12cyclictest1159-21snmpd12:25:083
2952799423,12cyclictest1159-21snmpd11:00:113
2952799421,12cyclictest1185-21lldpd07:38:533
2952799421,12cyclictest0-21swapper/309:10:033
2952599426,7cyclictest120091phc2sys12:36:041
2952599426,7cyclictest120091phc2sys12:36:041
117691420,9ptp4l216-21systemd-journal08:31:543
117691420,8ptp4l1185-21lldpd11:51:533
117691420,8ptp4l1185-21lldpd10:44:271
117691420,8ptp4l1185-21lldpd10:44:271
117691420,8ptp4l1185-21lldpd09:45:563
117691420,6ptp4l20953-21sed08:00:233
117691420,2ptp4l0-21swapper/310:48:093
117691420,2ptp4l0-21swapper/108:32:191
2952799413,12cyclictest216-21systemd-journal07:11:263
2952799412,12cyclictest1185-21lldpd11:25:183
117691410,6ptp4l216-21systemd-journal07:26:213
117691410,5ptp4l216-21systemd-journal12:10:173
117691410,4ptp4l41-21ksoftirqd/309:55:043
117691410,4ptp4l14317-21apt-get11:15:203
117691410,2ptp4l0-21swapper/312:35:393
117691410,2ptp4l0-21swapper/312:35:393
117691410,2ptp4l0-21swapper/312:07:343
117691410,2ptp4l0-21swapper/307:50:163
117691410,2ptp4l0-21swapper/108:47:431
117691410,2ptp4l0-21swapper/108:12:311
117691400,6ptp4l4815-21rs:main3
117691400,6ptp4l1185-21lldpd08:42:023
117691400,5ptp4l4634-21diskstats07:25:261
117691400,4ptp4l5833-21python08:35:363
117691400,4ptp4l23563-21iostat08:05:283
117691400,2ptp4l0-21swapper/310:06:233
117691400,2ptp4l0-21swapper/108:08:131
117691390,6ptp4l1159-21snmpd09:41:053
117691390,6ptp4l1159-21snmpd09:41:053
117691390,5ptp4l216-21systemd-journal08:40:551
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional