You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-02-05 - 00:02

x86 Intel Atom E3845 @1910 MHz, Linux 4.19.37-rt19 (Profile)

Latency plot of system in rack #d, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Up99 -i200 -h400 --smi -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rackdslot2.osadl.org (updated Sat Feb 04, 2023 12:46:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2893421760,9sleep31611499cyclictest12:10:263
1339911360,4ptp4l401ktimersoftd/307:08:493
55498920,11rtkit-daemon0-21swapper/207:06:022
1569227454,9sleep10-21swapper/107:08:221
173842660,2sleep30-21swapper/308:20:183
156862662,15sleep00-21swapper/007:08:170
16112996549,10cyclictest25-21ksoftirqd/107:10:111
133991650,1ptp4l401ktimersoftd/312:21:503
179392630,9sleep31611499cyclictest11:50:113
16114996350,7cyclictest41-21ksoftirqd/308:05:143
16112996146,9cyclictest25-21ksoftirqd/108:30:131
16114996044,9cyclictest41-21ksoftirqd/307:30:123
70062590,2sleep10-21swapper/109:05:291
16114995844,8cyclictest41-21ksoftirqd/308:55:123
277992570,5sleep116110-21cyclictest11:00:241
16112995546,5cyclictest25-21ksoftirqd/108:20:141
129891540,9getstats1611499cyclictest11:00:003
16114995338,10cyclictest41-21ksoftirqd/309:55:133
16112995338,8cyclictest25-21ksoftirqd/109:20:141
55498520,2rtkit-daemon0-21swapper/209:22:282
203792520,2sleep30-21swapper/311:55:213
16114995237,8cyclictest41-21ksoftirqd/310:00:123
16112995242,5cyclictest25-21ksoftirqd/109:35:251
16112995137,8cyclictest25-21ksoftirqd/110:35:171
16114995036,8cyclictest41-21ksoftirqd/308:45:123
16114995036,7cyclictest41-21ksoftirqd/312:25:143
16114995034,10cyclictest41-21ksoftirqd/309:50:123
16112995036,9cyclictest25-21ksoftirqd/108:15:191
55498460,2rtkit-daemon0-21swapper/307:26:453
16114994633,7cyclictest41-21ksoftirqd/312:30:123
133991460,2ptp4l2193-21apt-get07:50:053
133991460,2ptp4l2193-21apt-get07:50:053
133991450,1ptp4l401ktimersoftd/309:15:393
16114994331,6cyclictest41-21ksoftirqd/312:00:133
16112994337,4cyclictest25-21ksoftirqd/110:05:001
133991420,1ptp4l401ktimersoftd/310:45:303
133991420,1ptp4l401ktimersoftd/308:12:363
16112994135,4cyclictest25-21ksoftirqd/107:19:591
16112994133,4cyclictest25-21ksoftirqd/111:55:001
16114994027,7cyclictest41-21ksoftirqd/309:45:243
16114994027,7cyclictest41-21ksoftirqd/309:40:203
133991400,1ptp4l401ktimersoftd/307:23:263
16114993932,4cyclictest41-21ksoftirqd/309:35:003
16114993932,4cyclictest41-21ksoftirqd/307:35:163
16114993925,8cyclictest41-21ksoftirqd/310:40:123
16112993933,4cyclictest25-21ksoftirqd/110:55:231
16112993932,4cyclictest25-21ksoftirqd/109:10:301
16112993932,4cyclictest25-21ksoftirqd/109:10:301
16114993833,3cyclictest41-21ksoftirqd/308:00:213
16114993830,5cyclictest41-21ksoftirqd/310:34:593
16114993830,5cyclictest41-21ksoftirqd/310:34:593
16114993830,4cyclictest41-21ksoftirqd/311:40:003
16114993825,7cyclictest41-21ksoftirqd/311:25:173
16112993832,4cyclictest25-21ksoftirqd/112:00:001
16112993832,3cyclictest25-21ksoftirqd/111:30:131
16111993832,4cyclictest9-21ksoftirqd/010:39:590
16111993821,14cyclictest9-21ksoftirqd/008:42:590
133991380,1ptp4l401ktimersoftd/311:00:263
133991380,1ptp4l401ktimersoftd/310:20:223
16114993732,3cyclictest41-21ksoftirqd/310:15:203
16114993731,4cyclictest41-21ksoftirqd/310:40:003
16114993731,4cyclictest41-21ksoftirqd/309:25:013
16114993731,3cyclictest41-21ksoftirqd/312:35:293
16114993730,4cyclictest41-21ksoftirqd/309:10:303
16114993730,4cyclictest41-21ksoftirqd/309:10:303
16114993724,8cyclictest41-21ksoftirqd/308:25:203
16112993731,3cyclictest25-21ksoftirqd/110:09:591
16112993730,4cyclictest25-21ksoftirqd/112:00:221
16112993730,4cyclictest25-21ksoftirqd/107:30:001
16111993720,14cyclictest9-21ksoftirqd/011:01:500
16111993713,4cyclictest5388-21sed07:55:160
16111993712,4cyclictest8200-21ptp4l-jitter12:35:280
1611199371,14cyclictest25649-21uniq12:05:240
16114993631,3cyclictest41-21ksoftirqd/308:44:593
16114993630,3cyclictest41-21ksoftirqd/308:35:153
16113993629,4cyclictest33-21ksoftirqd/207:54:222
16113993629,4cyclictest33-21ksoftirqd/207:54:222
16112993631,3cyclictest25-21ksoftirqd/111:05:291
16112993631,3cyclictest25-21ksoftirqd/110:20:161
16112993631,3cyclictest25-21ksoftirqd/108:40:321
16112993631,3cyclictest25-21ksoftirqd/107:55:261
16112993630,4cyclictest25-21ksoftirqd/112:30:351
16112993630,4cyclictest25-21ksoftirqd/111:45:211
16112993630,4cyclictest25-21ksoftirqd/111:45:201
16112993630,3cyclictest25-21ksoftirqd/110:50:011
16112993629,4cyclictest25-21ksoftirqd/112:35:581
16112993629,4cyclictest25-21ksoftirqd/112:06:221
16112993627,6cyclictest25-21ksoftirqd/107:40:591
16111993629,4cyclictest9-21ksoftirqd/011:35:060
16111993613,3cyclictest394-21sshd08:54:550
16111993612,4cyclictest3015-21sshd12:25:220
16111993610,4cyclictest1418-21timerandwakeup12:20:290
133991360,1ptp4l401ktimersoftd/312:08:153
133991360,1ptp4l401ktimersoftd/311:15:293
133991360,1ptp4l401ktimersoftd/311:10:253
133991360,14ptp4l401ktimersoftd/307:47:233
133991360,14ptp4l401ktimersoftd/307:47:233
133991360,12ptp4l0-21swapper/308:51:543
133991360,12ptp4l0-21swapper/307:13:303
16114993529,4cyclictest41-21ksoftirqd/309:00:143
16114993529,4cyclictest41-21ksoftirqd/307:19:593
16114993529,3cyclictest41-21ksoftirqd/311:05:283
16114993529,3cyclictest41-21ksoftirqd/307:40:173
16114993528,4cyclictest41-21ksoftirqd/312:15:193
16114993528,4cyclictest41-21ksoftirqd/311:45:013
16113993528,4cyclictest33-21ksoftirqd/207:37:112
16113993528,4cyclictest33-21ksoftirqd/207:20:192
16112993530,3cyclictest25-21ksoftirqd/112:25:321
16112993530,3cyclictest25-21ksoftirqd/108:35:151
16112993530,3cyclictest25-21ksoftirqd/107:55:001
16112993530,3cyclictest25-21ksoftirqd/107:55:001
16112993529,4cyclictest25-21ksoftirqd/109:40:261
16112993529,4cyclictest25-21ksoftirqd/109:00:201
16112993529,3cyclictest25-21ksoftirqd/112:15:151
16112993528,4cyclictest25-21ksoftirqd/111:25:261
16111993528,4cyclictest9-21ksoftirqd/008:15:050
16111993512,3cyclictest23429-21if_enp4s012:00:200
16111993510,4cyclictest30422-21cut11:05:280
133991350,29ptp4l35292irq/88-enp1s0-T09:37:143
16114993428,4cyclictest41-21ksoftirqd/308:34:523
16114993428,3cyclictest41-21ksoftirqd/310:14:023
16114993427,4cyclictest41-21ksoftirqd/311:22:293
16113993428,4cyclictest33-21ksoftirqd/212:15:002
16113993428,3cyclictest33-21ksoftirqd/211:20:022
16113993427,4cyclictest33-21ksoftirqd/211:04:462
16113993427,4cyclictest33-21ksoftirqd/209:46:292
16113993427,4cyclictest33-21ksoftirqd/209:13:552
16113993427,4cyclictest33-21ksoftirqd/209:13:552
16113993426,5cyclictest33-21ksoftirqd/211:28:472
16113993426,5cyclictest33-21ksoftirqd/208:11:272
16112993428,4cyclictest25-21ksoftirqd/108:50:001
16112993427,4cyclictest25-21ksoftirqd/111:35:121
16112993427,4cyclictest25-21ksoftirqd/110:25:141
16112993427,4cyclictest25-21ksoftirqd/109:59:591
16112993427,4cyclictest25-21ksoftirqd/109:45:281
16112993427,4cyclictest25-21ksoftirqd/109:19:421
16111993428,3cyclictest9-21ksoftirqd/010:14:390
16111993427,4cyclictest9-21ksoftirqd/010:58:460
16111993427,4cyclictest9-21ksoftirqd/008:33:320
16111993427,4cyclictest9-21ksoftirqd/007:48:100
16111993427,4cyclictest9-21ksoftirqd/007:48:090
16111993416,4cyclictest9-21ksoftirqd/011:28:440
16114993327,4cyclictest41-21ksoftirqd/310:05:293
16114993327,3cyclictest41-21ksoftirqd/310:50:153
16114993326,4cyclictest41-21ksoftirqd/309:10:003
16113993327,4cyclictest33-21ksoftirqd/208:22:222
16113993327,4cyclictest33-21ksoftirqd/207:13:112
16113993327,3cyclictest33-21ksoftirqd/208:43:292
16113993326,4cyclictest33-21ksoftirqd/212:04:542
16113993326,4cyclictest33-21ksoftirqd/211:34:002
16113993326,4cyclictest33-21ksoftirqd/211:12:592
16113993326,4cyclictest33-21ksoftirqd/210:54:052
16113993326,4cyclictest33-21ksoftirqd/209:58:022
16113993326,4cyclictest33-21ksoftirqd/209:15:382
16113993326,4cyclictest33-21ksoftirqd/208:51:462
16113993326,4cyclictest33-21ksoftirqd/208:06:262
16113993326,4cyclictest33-21ksoftirqd/207:49:222
16113993326,4cyclictest33-21ksoftirqd/207:49:222
16112993328,3cyclictest25-21ksoftirqd/110:40:301
16112993327,4cyclictest25-21ksoftirqd/112:12:231
16112993327,4cyclictest25-21ksoftirqd/110:30:141
16112993327,4cyclictest25-21ksoftirqd/110:30:141
16112993327,4cyclictest25-21ksoftirqd/107:37:441
16112993327,3cyclictest25-21ksoftirqd/110:20:001
16112993326,4cyclictest25-21ksoftirqd/109:33:241
16111993327,4cyclictest9-21ksoftirqd/009:16:160
16111993327,3cyclictest9-21ksoftirqd/011:34:430
16111993327,3cyclictest9-21ksoftirqd/009:51:090
16111993327,3cyclictest9-21ksoftirqd/009:13:250
16111993327,3cyclictest9-21ksoftirqd/009:13:240
16111993327,3cyclictest9-21ksoftirqd/008:55:100
16111993327,3cyclictest9-21ksoftirqd/008:01:060
16111993326,4cyclictest9-21ksoftirqd/010:48:500
16111993325,5cyclictest9-21ksoftirqd/012:13:440
16111993313,3cyclictest2630-21/usr/sbin/munin07:50:120
16111993313,3cyclictest2630-21/usr/sbin/munin07:50:110
16111993313,3cyclictest12570-21latency_hist10:30:000
133991330,1ptp4l401ktimersoftd/311:45:453
133991330,1ptp4l401ktimersoftd/311:45:443
16114993225,4cyclictest41-21ksoftirqd/307:59:433
16113993226,3cyclictest33-21ksoftirqd/212:10:002
16113993226,3cyclictest33-21ksoftirqd/209:40:412
16113993225,4cyclictest33-21ksoftirqd/211:44:042
16113993225,4cyclictest33-21ksoftirqd/209:50:512
16113993225,4cyclictest33-21ksoftirqd/208:36:322
16113993225,4cyclictest33-21ksoftirqd/208:31:422
16113993225,4cyclictest33-21ksoftirqd/208:26:422
16113993224,5cyclictest33-21ksoftirqd/209:38:422
16112993227,3cyclictest25-21ksoftirqd/108:55:291
16112993227,3cyclictest25-21ksoftirqd/108:07:031
16112993226,4cyclictest25-21ksoftirqd/108:27:231
16112993226,4cyclictest25-21ksoftirqd/107:48:531
16112993226,4cyclictest25-21ksoftirqd/107:48:521
16112993225,4cyclictest25-21ksoftirqd/111:24:311
16112993225,4cyclictest25-21ksoftirqd/108:54:581
16112993225,4cyclictest25-21ksoftirqd/108:12:251
16112993225,4cyclictest25-21ksoftirqd/107:30:311
16112993220,10cyclictest25-21ksoftirqd/109:50:281
16111993226,4cyclictest9-21ksoftirqd/012:33:210
16111993226,4cyclictest9-21ksoftirqd/011:48:560
16111993226,4cyclictest9-21ksoftirqd/011:48:560
16111993226,3cyclictest9-21ksoftirqd/009:37:260
16111993225,4cyclictest9-21ksoftirqd/011:19:090
16111993225,4cyclictest9-21ksoftirqd/009:26:320
16111993225,4cyclictest9-21ksoftirqd/008:25:090
16111993225,4cyclictest9-21ksoftirqd/007:34:580
16111993225,4cyclictest9-21ksoftirqd/007:28:080
16111993225,4cyclictest9-21ksoftirqd/007:10:580
16111993224,5cyclictest9-21ksoftirqd/010:08:440
16113993125,3cyclictest33-21ksoftirqd/211:39:262
16113993124,4cyclictest33-21ksoftirqd/212:15:112
16113993124,4cyclictest33-21ksoftirqd/211:07:522
16113993124,4cyclictest33-21ksoftirqd/210:49:002
16113993124,4cyclictest33-21ksoftirqd/210:17:062
16113993124,4cyclictest33-21ksoftirqd/209:29:342
16113993124,4cyclictest33-21ksoftirqd/207:41:192
16113993124,4cyclictest33-21ksoftirqd/207:31:052
16112993125,4cyclictest25-21ksoftirqd/111:40:171
16112993125,4cyclictest25-21ksoftirqd/110:54:271
16112993125,4cyclictest25-21ksoftirqd/110:14:351
16112993125,3cyclictest25-21ksoftirqd/111:20:001
16112993124,4cyclictest25-21ksoftirqd/109:25:471
16111993125,4cyclictest9-21ksoftirqd/012:16:100
16111993125,4cyclictest9-21ksoftirqd/010:32:040
16111993125,4cyclictest9-21ksoftirqd/010:32:040
16111993125,4cyclictest9-21ksoftirqd/008:39:310
16111993125,3cyclictest9-21ksoftirqd/011:53:230
16111993125,3cyclictest9-21ksoftirqd/011:23:570
16111993124,4cyclictest9-21ksoftirqd/011:43:080
16111993124,4cyclictest9-21ksoftirqd/010:44:090
16111993123,5cyclictest9-21ksoftirqd/008:11:160
16114993024,4cyclictest41-21ksoftirqd/311:30:143
16114993023,4cyclictest41-21ksoftirqd/309:25:503
16113993024,4cyclictest33-21ksoftirqd/210:34:392
16113993024,4cyclictest33-21ksoftirqd/210:34:392
16113993024,3cyclictest33-21ksoftirqd/211:46:332
16113993024,3cyclictest33-21ksoftirqd/211:46:322
16113993023,4cyclictest33-21ksoftirqd/212:25:112
16113993023,4cyclictest33-21ksoftirqd/211:18:222
16113993023,4cyclictest33-21ksoftirqd/210:01:462
16113993023,4cyclictest33-21ksoftirqd/209:30:432
16112993025,3cyclictest25-21ksoftirqd/111:10:241
16112993024,4cyclictest25-21ksoftirqd/108:03:101
16112993024,3cyclictest25-21ksoftirqd/112:20:261
16112993023,5cyclictest25-21ksoftirqd/107:20:401
16111993025,3cyclictest9-21ksoftirqd/010:00:110
16111993024,3cyclictest9-21ksoftirqd/010:22:460
16111993024,3cyclictest9-21ksoftirqd/008:05:080
16111993023,4cyclictest9-21ksoftirqd/009:55:590
16114992923,4cyclictest41-21ksoftirqd/310:29:223
16114992922,4cyclictest41-21ksoftirqd/308:15:453
16113992922,4cyclictest33-21ksoftirqd/212:22:152
16113992922,4cyclictest33-21ksoftirqd/210:20:072
16113992922,4cyclictest33-21ksoftirqd/209:03:172
16113992922,4cyclictest33-21ksoftirqd/208:18:042
16111992922,4cyclictest9-21ksoftirqd/009:00:060
16113992822,4cyclictest33-21ksoftirqd/207:18:412
16113992821,4cyclictest33-21ksoftirqd/212:39:372
16113992821,4cyclictest33-21ksoftirqd/210:11:502
16113992820,5cyclictest33-21ksoftirqd/208:03:162
16113992820,5cyclictest33-21ksoftirqd/207:58:092
16111992823,3cyclictest9-21ksoftirqd/008:45:220
16111992822,4cyclictest9-21ksoftirqd/011:13:550
16111992822,4cyclictest9-21ksoftirqd/009:41:250
16111992822,4cyclictest9-21ksoftirqd/009:32:090
16111992822,4cyclictest9-21ksoftirqd/009:22:390
16111992822,4cyclictest9-21ksoftirqd/007:43:300
16111992822,3cyclictest9-21ksoftirqd/010:15:070
16111992821,4cyclictest9-21ksoftirqd/007:20:590
16113992720,4cyclictest33-21ksoftirqd/211:50:482
16113992720,4cyclictest33-21ksoftirqd/210:40:092
16113992720,4cyclictest33-21ksoftirqd/210:27:332
16113992720,4cyclictest33-21ksoftirqd/208:49:202
16111992721,4cyclictest9-21ksoftirqd/008:21:560
16111992721,4cyclictest9-21ksoftirqd/007:15:510
16111992721,3cyclictest9-21ksoftirqd/009:06:250
16111992720,4cyclictest9-21ksoftirqd/009:47:110
16111992720,4cyclictest9-21ksoftirqd/007:37:360
16113992621,3cyclictest33-21ksoftirqd/207:26:242
16113992620,4cyclictest33-21ksoftirqd/210:08:272
16111992621,3cyclictest9-21ksoftirqd/011:58:530
16113992519,4cyclictest33-21ksoftirqd/212:33:542
16113992518,4cyclictest33-21ksoftirqd/209:08:072
16111992519,4cyclictest9-21ksoftirqd/010:51:550
27952241,19sleep22796-21sshd08:59:492
16113992416,5cyclictest33-21ksoftirqd/211:55:572
16113992416,5cyclictest33-21ksoftirqd/210:59:492
16113992316,4cyclictest33-21ksoftirqd/210:36:582
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional