You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-12-09 - 17:20

x86 Intel Atom E3845 @1910 MHz, Linux 4.19.37-rt19 (Profile)

Latency plot of system in rack #d, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Up99 -i200 -h400 --smi -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of all highest latencies:
System rackdslot2.osadl.org (updated Wed Dec 07, 2022 12:46:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2276821720,6sleep2233099cyclictest09:00:192
1339911110,4ptp4l401ktimersoftd/307:05:523
19002803,19sleep20-21swapper/207:08:172
18422803,20sleep10-21swapper/107:07:321
240142770,9sleep0232899cyclictest11:20:310
17962734,15sleep00-21swapper/007:06:540
122482660,8sleep3233199cyclictest08:35:353
93252640,3sleep00-21swapper/009:40:350
83632600,2sleep00-21swapper/011:55:320
55498600,15rtkit-daemon553-21rtkit-daemon08:34:202
232899590,7cyclictest8535-21/usr/sbin/munin08:30:150
133991590,1ptp4l391rcuc/307:41:383
232999577,11cyclictest25-21ksoftirqd/108:40:291
232999553,6cyclictest0-21swapper/109:30:301
232999543,6cyclictest0-21swapper/111:55:281
233199538,7cyclictest41-21ksoftirqd/311:08:143
232999536,7cyclictest25-21ksoftirqd/109:20:351
232999535,9cyclictest25-21ksoftirqd/107:20:011
232999526,10cyclictest25-21ksoftirqd/110:25:181
232999524,6cyclictest25-21ksoftirqd/112:20:201
232999522,6cyclictest0-21swapper/112:15:051
232999520,8cyclictest25-21ksoftirqd/111:10:361
2331995129,8cyclictest41-21ksoftirqd/307:49:003
232999516,7cyclictest25-21ksoftirqd/109:15:011
232999515,7cyclictest25-21ksoftirqd/111:07:291
232999515,7cyclictest25-21ksoftirqd/109:11:561
232999515,7cyclictest25-21ksoftirqd/109:11:561
232999515,7cyclictest25-21ksoftirqd/108:50:161
232999514,6cyclictest25-21ksoftirqd/110:50:281
232999506,8cyclictest25-21ksoftirqd/108:26:111
232999505,9cyclictest25-21ksoftirqd/107:20:211
232999504,9cyclictest25-21ksoftirqd/112:28:161
232999503,11cyclictest25-21ksoftirqd/112:30:031
232999502,6cyclictest25-21ksoftirqd/112:11:471
232999502,6cyclictest25-21ksoftirqd/112:00:231
232999496,9cyclictest25-21ksoftirqd/110:00:411
232999496,8cyclictest25-21ksoftirqd/112:38:391
232999496,8cyclictest25-21ksoftirqd/109:09:031
232999495,8cyclictest25-21ksoftirqd/111:45:361
232999495,8cyclictest25-21ksoftirqd/111:45:351
232999494,9cyclictest25-21ksoftirqd/109:01:031
232999494,7cyclictest0-21swapper/111:43:141
232999494,10cyclictest25-21ksoftirqd/110:45:011
232999493,8cyclictest25-21ksoftirqd/110:59:461
232999493,7cyclictest25-21ksoftirqd/111:25:181
2331994825,5cyclictest41-21ksoftirqd/310:14:173
233199480,3cyclictest55498rtkit-daemon10:00:313
232999487,8cyclictest25-21ksoftirqd/110:34:251
232999486,9cyclictest25-21ksoftirqd/111:51:571
232999486,9cyclictest25-21ksoftirqd/111:51:571
232999486,9cyclictest25-21ksoftirqd/111:30:461
232999486,9cyclictest25-21ksoftirqd/107:51:061
232999486,8cyclictest25-21ksoftirqd/112:08:561
232999484,9cyclictest25-21ksoftirqd/107:30:591
232999484,7cyclictest25-21ksoftirqd/111:15:311
232999484,6cyclictest25-21ksoftirqd/108:10:311
232999484,12cyclictest25-21ksoftirqd/110:20:271
232999483,9cyclictest25-21ksoftirqd/110:16:471
232999483,8cyclictest25-21ksoftirqd/109:25:281
232999476,9cyclictest25-21ksoftirqd/109:44:031
232999476,9cyclictest25-21ksoftirqd/109:00:001
232999476,9cyclictest25-21ksoftirqd/108:08:041
232999476,8cyclictest25-21ksoftirqd/110:45:021
232999476,8cyclictest25-21ksoftirqd/107:38:411
232999475,9cyclictest25-21ksoftirqd/108:45:181
232999475,8cyclictest25-21ksoftirqd/111:02:201
232999475,8cyclictest25-21ksoftirqd/109:56:481
232999474,9cyclictest25-21ksoftirqd/107:25:231
232999474,7cyclictest25-21ksoftirqd/109:50:231
2328994731,13cyclictest9-21ksoftirqd/012:08:130
2331994626,7cyclictest41-21ksoftirqd/311:50:343
2331994626,7cyclictest41-21ksoftirqd/311:50:333
232999466,9cyclictest25-21ksoftirqd/108:01:181
232999466,8cyclictest25-21ksoftirqd/108:36:581
232999465,6cyclictest25-21ksoftirqd/107:45:161
232999465,10cyclictest25-21ksoftirqd/107:40:211
232999463,10cyclictest25-21ksoftirqd/109:35:271
232999462,5cyclictest25-21ksoftirqd/108:15:241
2328994631,12cyclictest9-21ksoftirqd/011:11:000
55498450,3rtkit-daemon553-21rtkit-daemon09:55:292
233842450,8sleep0232899cyclictest12:30:000
232999456,8cyclictest25-21ksoftirqd/111:35:361
232999455,5cyclictest25-21ksoftirqd/111:20:461
232999453,11cyclictest25-21ksoftirqd/107:55:191
232999446,9cyclictest25-21ksoftirqd/110:06:191
232999444,9cyclictest25-21ksoftirqd/107:14:261
2328994426,15cyclictest9-21ksoftirqd/010:25:000
232899440,2cyclictest0-21swapper/011:02:450
232899440,2cyclictest0-21swapper/008:25:500
133991440,1ptp4l401ktimersoftd/308:15:293
233199435,6cyclictest0-21swapper/310:05:223
2328994326,14cyclictest9-21ksoftirqd/010:57:070
232899430,2cyclictest0-21swapper/008:46:140
232899430,2cyclictest0-21swapper/007:29:410
232999426,8cyclictest25-21ksoftirqd/110:36:111
232999424,9cyclictest25-21ksoftirqd/108:30:141
232999422,8cyclictest4603-21latency08:20:331
232899420,2cyclictest0-21swapper/011:08:460
232899420,2cyclictest0-21swapper/008:35:300
2330994134,4cyclictest15444-21anacron11:04:172
232999412,11cyclictest25-21ksoftirqd/110:10:281
2328994125,13cyclictest9-21ksoftirqd/008:08:500
232899410,5cyclictest14268-21cat09:50:260
232899410,37cyclictest4020-21sed07:10:310
2331994019,9cyclictest41-21ksoftirqd/308:05:223
232899402,31cyclictest3810-21apt-get09:30:170
2331993933,4cyclictest41-21ksoftirqd/311:10:313
2331993933,4cyclictest41-21ksoftirqd/309:00:163
232899394,31cyclictest0-21swapper/009:55:140
232899393,29cyclictest20448-21apt-get11:15:180
232899392,31cyclictest17612-21apt-get08:50:140
232899392,31cyclictest12183-21apt-get07:30:140
232899392,29cyclictest19351-21apt-get07:45:130
2328993918,18cyclictest29728-21tr10:25:170
232899390,4cyclictest20848-21sed10:05:220
232899390,3cyclictest19644-21vmstat10:00:420
233199386,8cyclictest41-21ksoftirqd/308:20:193
233199386,7cyclictest41-21ksoftirqd/311:23:593
233199385,8cyclictest41-21ksoftirqd/311:40:323
2331993833,3cyclictest41-21ksoftirqd/312:30:003
2331993832,4cyclictest41-21ksoftirqd/308:40:163
232999386,9cyclictest25-21ksoftirqd/109:45:221
232899383,29cyclictest1849-21apt-get10:35:170
232899382,29cyclictest11030-21apt-get09:45:120
232899380,4cyclictest24654-21cpuspeed09:05:260
232899380,2cyclictest0-21swapper/009:25:260
233199376,7cyclictest41-21ksoftirqd/310:35:383
233199374,8cyclictest41-21ksoftirqd/309:17:163
2331993732,4cyclictest41-21ksoftirqd/311:00:223
2331993731,4cyclictest41-21ksoftirqd/310:15:343
2331993731,4cyclictest41-21ksoftirqd/308:45:173
2331993731,4cyclictest41-21ksoftirqd/308:30:213
2331993730,4cyclictest41-21ksoftirqd/308:10:163
232899373,27cyclictest27411-21apt-get11:30:180
232899373,27cyclictest13865-21apt-get12:10:140
2328993730,4cyclictest9-21ksoftirqd/008:10:410
232899372,29cyclictest14497-21apt-get07:35:170
232899372,28cyclictest12983-21apt-get08:40:110
232899370,4cyclictest698-21which10:30:300
232899370,4cyclictest6651-21grep10:45:140
232899370,4cyclictest24712-21phc2sys-jitter12:30:300
232899370,4cyclictest20486-21timerwakeupswit12:20:380
2331993632,2cyclictest391rcuc/312:13:393
2331993631,3cyclictest41-21ksoftirqd/310:55:353
2331993631,3cyclictest391rcuc/311:27:273
2331993630,4cyclictest41-21ksoftirqd/311:35:413
2331993630,4cyclictest41-21ksoftirqd/309:55:273
2331993630,4cyclictest41-21ksoftirqd/309:35:403
2331993630,4cyclictest41-21ksoftirqd/307:25:233
2331993629,4cyclictest41-21ksoftirqd/310:50:283
2330993629,4cyclictest33-21ksoftirqd/210:57:482
2328993629,4cyclictest9-21ksoftirqd/009:35:090
232899362,28cyclictest31536-21apt-get09:20:150
232899362,28cyclictest25921-21apt-get12:35:160
2328993613,20cyclictest16995-21bc07:40:130
232899360,5cyclictest17987-21switchtime12:15:380
232899360,4cyclictest9896-21iostat12:00:260
232899360,3cyclictest7422-21df07:20:220
232899360,3cyclictest23144-21chrt10:10:210
233199355,8cyclictest41-21ksoftirqd/311:30:393
2331993531,2cyclictest391rcuc/309:41:043
2331993530,3cyclictest41-21ksoftirqd/309:10:183
2331993530,3cyclictest41-21ksoftirqd/309:10:183
2331993530,3cyclictest391rcuc/309:50:263
2331993529,4cyclictest41-21ksoftirqd/309:05:343
2331993529,4cyclictest41-21ksoftirqd/308:00:313
2331993529,3cyclictest41-21ksoftirqd/312:35:193
2331993528,4cyclictest41-21ksoftirqd/312:15:353
2331993528,4cyclictest41-21ksoftirqd/309:48:463
2331993528,4cyclictest41-21ksoftirqd/307:35:313
2330993528,4cyclictest33-21ksoftirqd/209:42:252
2330993528,4cyclictest33-21ksoftirqd/208:43:452
2330993528,4cyclictest33-21ksoftirqd/207:22:092
2328993529,4cyclictest9-21ksoftirqd/011:40:090
2328993528,4cyclictest9-21ksoftirqd/009:15:000
2328993528,4cyclictest9-21ksoftirqd/009:15:000
2328993528,4cyclictest9-21ksoftirqd/007:18:510
232899352,26cyclictest29200-21apt-get09:15:170
232899352,26cyclictest22253-21apt-get09:00:130
2328993519,13cyclictest9-21ksoftirqd/011:37:410
2328993513,19cyclictest5150-21seq11:50:230
2328993513,19cyclictest5150-21seq11:50:220
232899350,4cyclictest4477-21ls10:40:220
232899350,3cyclictest3224-21ptp4l-jitter11:45:330
232899350,3cyclictest3224-21ptp4l-jitter11:45:330
133991350,1ptp4l401ktimersoftd/310:42:243
55498340,2rtkit-daemon553-21rtkit-daemon09:26:362
2331993430,2cyclictest391rcuc/308:52:423
2331993429,3cyclictest41-21ksoftirqd/310:20:323
2331993428,4cyclictest41-21ksoftirqd/307:50:363
2331993428,4cyclictest41-21ksoftirqd/307:30:193
2331993428,3cyclictest41-21ksoftirqd/311:45:153
2331993428,3cyclictest41-21ksoftirqd/311:45:143
2331993428,3cyclictest41-21ksoftirqd/311:15:173
2331993427,4cyclictest41-21ksoftirqd/312:33:343
2331993427,4cyclictest41-21ksoftirqd/309:28:143
2330993429,3cyclictest33-21ksoftirqd/211:29:542
2330993428,4cyclictest33-21ksoftirqd/207:53:222
2330993428,3cyclictest33-21ksoftirqd/212:02:392
2330993427,4cyclictest33-21ksoftirqd/211:11:152
232899342,26cyclictest24924-21apt-get10:15:160
232899340,5cyclictest3966-21cstates08:20:260
232899340,2cyclictest0-21swapper/008:55:290
233199333,7cyclictest41-21ksoftirqd/311:59:113
2331993329,3cyclictest41-21ksoftirqd/308:55:283
2331993328,4cyclictest41-21ksoftirqd/312:00:243
2331993328,3cyclictest41-21ksoftirqd/312:20:283
2331993328,3cyclictest41-21ksoftirqd/307:15:183
2331993327,4cyclictest41-21ksoftirqd/307:20:203
2331993326,4cyclictest41-21ksoftirqd/307:10:163
2330993328,3cyclictest33-21ksoftirqd/212:13:182
2330993327,4cyclictest33-21ksoftirqd/208:00:342
2330993326,4cyclictest33-21ksoftirqd/212:31:232
2330993326,4cyclictest33-21ksoftirqd/211:32:062
2330993326,4cyclictest33-21ksoftirqd/210:52:132
2330993326,4cyclictest33-21ksoftirqd/210:24:402
2330993326,4cyclictest33-21ksoftirqd/210:05:082
2330993326,4cyclictest33-21ksoftirqd/210:04:062
2330993326,4cyclictest33-21ksoftirqd/209:48:232
2330993326,4cyclictest33-21ksoftirqd/208:47:172
232899336,24cyclictest28560-21latency_hist08:05:010
232899330,2cyclictest0-21swapper/010:51:290
2331993227,3cyclictest41-21ksoftirqd/312:05:333
2331993226,4cyclictest41-21ksoftirqd/310:50:013
2331993226,4cyclictest41-21ksoftirqd/308:25:183
2331993226,3cyclictest41-21ksoftirqd/309:24:453
2330993227,3cyclictest33-21ksoftirqd/209:50:332
2330993227,3cyclictest33-21ksoftirqd/208:57:582
2330993227,3cyclictest33-21ksoftirqd/207:15:512
2330993226,3cyclictest33-21ksoftirqd/210:16:562
2330993226,3cyclictest33-21ksoftirqd/208:21:502
2330993226,3cyclictest33-21ksoftirqd/207:45:532
2330993225,4cyclictest33-21ksoftirqd/212:22:202
2330993225,4cyclictest33-21ksoftirqd/211:58:592
2330993225,4cyclictest33-21ksoftirqd/209:35:382
2330993225,4cyclictest33-21ksoftirqd/209:24:142
2330993225,4cyclictest33-21ksoftirqd/208:26:202
2330993223,7cyclictest33-21ksoftirqd/211:45:232
2330993223,7cyclictest33-21ksoftirqd/211:45:222
233099321,3cyclictest121rcu_preempt08:05:202
232899327,22cyclictest24574-21if_enp4s007:55:190
232899323,24cyclictest21669-21apt-get07:50:180
2328993224,5cyclictest9-21ksoftirqd/008:18:220
232899320,4cyclictest25749-21iostat_ios11:25:220
233199315,7cyclictest41-21ksoftirqd/310:25:343
2330993126,3cyclictest33-21ksoftirqd/207:33:312
2330993125,4cyclictest33-21ksoftirqd/212:09:112
2330993125,4cyclictest33-21ksoftirqd/209:20:002
2330993125,4cyclictest33-21ksoftirqd/209:11:022
2330993125,4cyclictest33-21ksoftirqd/209:11:022
2330993125,4cyclictest33-21ksoftirqd/208:51:032
2330993125,4cyclictest33-21ksoftirqd/207:11:392
2330993125,3cyclictest33-21ksoftirqd/210:45:082
2330993125,3cyclictest33-21ksoftirqd/208:15:592
2330993124,4cyclictest33-21ksoftirqd/211:17:042
2330993124,4cyclictest33-21ksoftirqd/211:09:412
2330993124,4cyclictest33-21ksoftirqd/210:11:282
2331993024,4cyclictest41-21ksoftirqd/309:33:553
2330993025,3cyclictest33-21ksoftirqd/212:39:532
2330993024,3cyclictest33-21ksoftirqd/207:55:332
2330993023,4cyclictest33-21ksoftirqd/210:42:392
2330993023,4cyclictest33-21ksoftirqd/208:14:172
2330993023,4cyclictest33-21ksoftirqd/207:35:082
2330993022,5cyclictest33-21ksoftirqd/208:36:022
2331992924,3cyclictest41-21ksoftirqd/307:55:313
2330992924,3cyclictest33-21ksoftirqd/212:25:092
2330992924,3cyclictest33-21ksoftirqd/211:35:292
2330992923,4cyclictest33-21ksoftirqd/211:42:502
2330992923,3cyclictest33-21ksoftirqd/211:53:432
2330992923,3cyclictest33-21ksoftirqd/211:53:422
2330992922,4cyclictest33-21ksoftirqd/207:41:542
233099291,9cyclictest0-21swapper/211:21:032
233099290,3cyclictest121rcu_preempt10:35:342
2331992821,4cyclictest41-21ksoftirqd/310:30:263
2330992822,3cyclictest33-21ksoftirqd/210:34:432
2330992822,3cyclictest33-21ksoftirqd/209:33:242
2330992821,4cyclictest33-21ksoftirqd/209:05:082
2330992722,3cyclictest33-21ksoftirqd/210:25:542
2330992619,4cyclictest33-21ksoftirqd/212:16:352
2330992419,3cyclictest33-21ksoftirqd/207:25:142
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional