You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-08-13 - 02:01

x86 Intel Atom E3845 @1910 MHz, Linux 4.19.37-rt19 (Profile)

Latency plot of system in rack #d, slot #4
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Up99 -i200 -h400 --smi -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackdslot4.osadl.org (updated Fri Aug 12, 2022 12:46:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2811021650,8sleep32843599cyclictest11:40:223
1956021650,8sleep32843599cyclictest12:30:273
280152150127,10sleep20-21swapper/207:08:252
280632139118,10sleep10-21swapper/107:09:001
129891124101,10phc2sys0-21swapper/307:08:573
564981210,11rtkit-daemon0-21swapper/007:06:460
28435998534,10cyclictest41-21ksoftirqd/311:05:003
28435997437,9cyclictest41-21ksoftirqd/308:10:143
28435997265,4cyclictest41-21ksoftirqd/308:30:123
28435997235,8cyclictest41-21ksoftirqd/309:20:183
28435997164,4cyclictest41-21ksoftirqd/310:05:163
28435997033,11cyclictest41-21ksoftirqd/310:45:163
28435997031,8cyclictest41-21ksoftirqd/311:15:283
28435997028,10cyclictest41-21ksoftirqd/308:40:013
28435996963,4cyclictest41-21ksoftirqd/309:35:123
28435996963,4cyclictest41-21ksoftirqd/309:35:113
28435996934,4cyclictest41-21ksoftirqd/311:00:003
28435996851,4cyclictest41-21ksoftirqd/310:20:013
28435996831,10cyclictest41-21ksoftirqd/312:20:213
28435996828,5cyclictest41-21ksoftirqd/310:00:003
28435996760,4cyclictest41-21ksoftirqd/307:45:043
28435996731,8cyclictest41-21ksoftirqd/312:15:003
28435996731,4cyclictest41-21ksoftirqd/309:00:283
28435996660,4cyclictest41-21ksoftirqd/311:35:053
28435996641,10cyclictest41-21ksoftirqd/310:10:153
28435996634,10cyclictest41-21ksoftirqd/310:35:143
28435996633,5cyclictest41-21ksoftirqd/312:25:333
28435996633,4cyclictest41-21ksoftirqd/308:40:193
28435996633,4cyclictest41-21ksoftirqd/308:15:223
28435996633,4cyclictest41-21ksoftirqd/308:15:213
28435996631,9cyclictest41-21ksoftirqd/311:50:263
28435996628,5cyclictest41-21ksoftirqd/308:45:203
28435996627,11cyclictest41-21ksoftirqd/311:05:183
28435996627,11cyclictest41-21ksoftirqd/311:05:183
28435996626,9cyclictest41-21ksoftirqd/309:50:313
28435996551,4cyclictest41-21ksoftirqd/308:55:003
28435996532,4cyclictest41-21ksoftirqd/311:30:243
28435996529,8cyclictest41-21ksoftirqd/311:25:313
28435996529,8cyclictest41-21ksoftirqd/307:40:123
28435996456,5cyclictest41-21ksoftirqd/309:45:133
28435996451,5cyclictest41-21ksoftirqd/312:00:273
28435996431,4cyclictest41-21ksoftirqd/309:25:323
28435996326,11cyclictest41-21ksoftirqd/309:10:223
28435996255,4cyclictest41-21ksoftirqd/310:20:053
28435996244,11cyclictest41-21ksoftirqd/307:55:263
28435996233,7cyclictest41-21ksoftirqd/308:55:203
28435996232,4cyclictest41-21ksoftirqd/309:30:333
2843599623,10cyclictest41-21ksoftirqd/311:10:373
28435996229,4cyclictest41-21ksoftirqd/309:20:003
28435996228,5cyclictest41-21ksoftirqd/307:50:253
28435996227,5cyclictest41-21ksoftirqd/310:40:163
28435996147,4cyclictest41-21ksoftirqd/307:35:293
28435996127,10cyclictest41-21ksoftirqd/310:05:023
28435996126,5cyclictest41-21ksoftirqd/311:55:263
28435996026,10cyclictest41-21ksoftirqd/308:25:273
28435996024,8cyclictest41-21ksoftirqd/307:10:153
28435996024,11cyclictest41-21ksoftirqd/309:05:003
2843599601,4cyclictest28317-21cat10:35:013
2843599594,5cyclictest41-21ksoftirqd/310:50:243
28435995945,4cyclictest41-21ksoftirqd/307:15:263
28435995940,4cyclictest41-21ksoftirqd/312:10:013
28435995925,5cyclictest41-21ksoftirqd/311:45:263
28435995924,10cyclictest41-21ksoftirqd/308:20:143
2843599586,4cyclictest41-21ksoftirqd/307:30:283
28435995824,8cyclictest41-21ksoftirqd/309:40:203
2843599577,4cyclictest41-21ksoftirqd/310:25:203
28435995745,4cyclictest41-21ksoftirqd/308:00:253
28434995751,3cyclictest33-21ksoftirqd/211:20:012
28435995623,9cyclictest41-21ksoftirqd/308:05:123
124991560,7getstats2843599cyclictest07:26:343
2843599552,7cyclictest41-21ksoftirqd/312:15:293
2843599551,10cyclictest41-21ksoftirqd/307:20:153
2843599543,9cyclictest41-21ksoftirqd/311:20:193
28434995448,4cyclictest33-21ksoftirqd/210:00:012
87772520,2sleep00-21swapper/008:45:130
28435995145,3cyclictest41-21ksoftirqd/312:35:283
28435995145,3cyclictest41-21ksoftirqd/312:35:273
28434995145,4cyclictest33-21ksoftirqd/211:15:002
28434995145,4cyclictest33-21ksoftirqd/210:35:002
28434995145,4cyclictest33-21ksoftirqd/210:20:012
28434995145,3cyclictest33-21ksoftirqd/209:25:232
28434995043,4cyclictest33-21ksoftirqd/207:25:012
28434995042,4cyclictest33-21ksoftirqd/207:40:302
28434994943,4cyclictest33-21ksoftirqd/212:30:262
28434994843,3cyclictest33-21ksoftirqd/211:25:002
28434994842,4cyclictest33-21ksoftirqd/208:50:012
28432994831,14cyclictest9-21ksoftirqd/008:38:310
28434994742,3cyclictest33-21ksoftirqd/211:35:012
28434994740,4cyclictest33-21ksoftirqd/211:39:592
28434994740,4cyclictest33-21ksoftirqd/207:30:002
28432994740,4cyclictest31826-21cron09:35:000
28434994641,3cyclictest33-21ksoftirqd/210:15:012
28434994640,4cyclictest33-21ksoftirqd/211:25:212
28434994640,4cyclictest33-21ksoftirqd/210:35:212
28434994640,4cyclictest33-21ksoftirqd/208:14:592
28434994640,3cyclictest33-21ksoftirqd/209:15:002
28434994639,4cyclictest33-21ksoftirqd/211:40:152
28434994639,4cyclictest33-21ksoftirqd/209:00:162
28434994633,10cyclictest33-21ksoftirqd/208:21:512
126932460,9sleep12843399cyclictest12:15:251
28434994539,3cyclictest33-21ksoftirqd/210:55:002
28434994539,3cyclictest33-21ksoftirqd/209:34:592
28433994535,4cyclictest25-21ksoftirqd/108:43:461
28434994439,3cyclictest33-21ksoftirqd/212:30:002
28434994438,4cyclictest33-21ksoftirqd/209:00:012
28434994437,4cyclictest33-21ksoftirqd/210:55:192
28434994437,4cyclictest33-21ksoftirqd/210:40:172
28434994437,4cyclictest33-21ksoftirqd/210:20:242
28434994437,4cyclictest33-21ksoftirqd/209:55:002
28434994437,4cyclictest33-21ksoftirqd/209:10:002
28434994437,4cyclictest33-21ksoftirqd/207:30:222
2843499434,3cyclictest33-21ksoftirqd/210:30:002
28434994338,4cyclictest33-21ksoftirqd/208:50:202
28434994337,4cyclictest33-21ksoftirqd/212:05:002
28434994337,4cyclictest33-21ksoftirqd/211:54:592
28434994337,3cyclictest33-21ksoftirqd/211:05:012
28434994337,3cyclictest33-21ksoftirqd/210:10:002
28434994337,3cyclictest33-21ksoftirqd/209:50:012
28434994336,4cyclictest33-21ksoftirqd/212:35:232
28434994336,4cyclictest33-21ksoftirqd/212:35:222
28434994336,4cyclictest33-21ksoftirqd/212:10:142
28434994336,4cyclictest33-21ksoftirqd/212:10:002
28434994336,4cyclictest33-21ksoftirqd/208:40:132
28434994336,4cyclictest33-21ksoftirqd/208:30:002
28434994336,4cyclictest33-21ksoftirqd/207:10:202
28433994337,4cyclictest25-21ksoftirqd/108:05:131
28432994326,14cyclictest9-21ksoftirqd/011:13:050
28434994236,4cyclictest33-21ksoftirqd/212:15:302
28434994236,3cyclictest33-21ksoftirqd/208:10:002
28434994235,4cyclictest33-21ksoftirqd/211:05:222
28434994235,4cyclictest33-21ksoftirqd/211:05:212
28432994225,14cyclictest9-21ksoftirqd/010:48:090
28434994136,3cyclictest33-21ksoftirqd/207:20:012
28434994135,4cyclictest33-21ksoftirqd/208:35:342
28434994135,4cyclictest33-21ksoftirqd/207:50:002
28434994135,3cyclictest33-21ksoftirqd/209:15:252
28434994135,3cyclictest33-21ksoftirqd/208:00:212
28434994134,4cyclictest33-21ksoftirqd/211:50:012
28434994134,4cyclictest33-21ksoftirqd/207:50:282
28434994134,4cyclictest33-21ksoftirqd/207:39:592
28433994135,4cyclictest21336-21kworker/1:011:35:151
28433994134,4cyclictest25-21ksoftirqd/108:00:381
28433994134,4cyclictest25-21ksoftirqd/107:19:321
2843299410,5cyclictest20866-21apt-get12:35:130
2843299410,5cyclictest20866-21apt-get12:35:120
28434994035,3cyclictest33-21ksoftirqd/211:55:252
28434994035,3cyclictest33-21ksoftirqd/209:35:262
28434994035,3cyclictest33-21ksoftirqd/209:35:262
28434994034,3cyclictest33-21ksoftirqd/210:50:002
28434994033,4cyclictest33-21ksoftirqd/207:55:232
28434994028,10cyclictest33-21ksoftirqd/208:30:112
28433994033,4cyclictest25-21ksoftirqd/112:30:151
28433994033,4cyclictest25-21ksoftirqd/111:59:201
28433994033,4cyclictest25-21ksoftirqd/111:01:181
28433994032,4cyclictest25-21ksoftirqd/110:50:131
28432994034,3cyclictest9-21ksoftirqd/012:16:110
28432994034,3cyclictest9-21ksoftirqd/008:24:400
28432994033,4cyclictest9-21ksoftirqd/011:20:010
28432994032,5cyclictest9-21ksoftirqd/009:12:520
28434993933,4cyclictest33-21ksoftirqd/209:20:302
28434993932,4cyclictest33-21ksoftirqd/210:00:192
28433993932,4cyclictest25-21ksoftirqd/112:39:341
28433993932,4cyclictest25-21ksoftirqd/112:39:331
28433993932,4cyclictest25-21ksoftirqd/110:00:491
28433993932,4cyclictest25-21ksoftirqd/109:40:141
28433993932,4cyclictest25-21ksoftirqd/107:34:131
28433993932,4cyclictest25-21ksoftirqd/107:22:321
2843399391,36cyclictest21336-21kworker/1:010:45:211
28432993933,3cyclictest9-21ksoftirqd/012:01:590
28432993933,3cyclictest9-21ksoftirqd/011:50:090
28432993932,4cyclictest9-21ksoftirqd/007:11:360
2843299392,33cyclictest19936-21tr08:00:140
28432993922,14cyclictest9-21ksoftirqd/007:47:540
2843299390,4cyclictest28204-21cat08:15:290
2843299390,4cyclictest28204-21cat08:15:290
28434993832,3cyclictest33-21ksoftirqd/212:22:272
2843499382,4cyclictest33-21ksoftirqd/209:40:302
28433993832,3cyclictest25-21ksoftirqd/111:11:361
28433993832,3cyclictest25-21ksoftirqd/109:09:491
28433993831,4cyclictest25-21ksoftirqd/112:11:391
28433993831,4cyclictest25-21ksoftirqd/110:19:351
28433993831,4cyclictest25-21ksoftirqd/107:56:171
28433993831,4cyclictest25-21ksoftirqd/107:13:341
28432993832,4cyclictest9-21ksoftirqd/012:30:460
28432993832,4cyclictest9-21ksoftirqd/009:09:590
28432993831,5cyclictest9-21ksoftirqd/011:31:560
28432993831,4cyclictest9-21ksoftirqd/012:25:020
28432993831,4cyclictest9-21ksoftirqd/012:24:510
28432993831,4cyclictest9-21ksoftirqd/011:05:040
28432993831,4cyclictest9-21ksoftirqd/011:05:040
28432993831,4cyclictest9-21ksoftirqd/009:04:590
28432993830,5cyclictest9-21ksoftirqd/011:37:380
28432993810,25cyclictest10101-21cut09:55:230
28434993721,4cyclictest33-21ksoftirqd/208:15:162
28434993721,4cyclictest33-21ksoftirqd/208:15:152
28433993731,4cyclictest25-21ksoftirqd/108:12:511
28433993731,3cyclictest25-21ksoftirqd/112:01:241
28433993730,4cyclictest25-21ksoftirqd/111:28:121
28432993731,4cyclictest9-21ksoftirqd/009:53:410
28432993731,4cyclictest9-21ksoftirqd/008:53:200
28432993731,3cyclictest9-21ksoftirqd/010:17:410
28432993731,3cyclictest9-21ksoftirqd/010:10:220
28432993731,3cyclictest9-21ksoftirqd/009:29:480
28432993730,4cyclictest9-21ksoftirqd/012:11:540
28432993730,4cyclictest9-21ksoftirqd/011:04:150
28432993730,4cyclictest9-21ksoftirqd/008:33:290
2843299372,5cyclictest1308-21smartctl07:20:200
2843299370,4cyclictest32445-21grep07:15:290
28433993630,4cyclictest25-21ksoftirqd/112:20:541
28433993630,4cyclictest25-21ksoftirqd/108:55:261
28433993630,4cyclictest25-21ksoftirqd/108:46:511
28433993630,3cyclictest25-21ksoftirqd/108:29:251
28433993629,5cyclictest25-21ksoftirqd/111:24:181
28433993629,5cyclictest25-21ksoftirqd/107:50:511
28433993629,4cyclictest25-21ksoftirqd/112:05:541
28433993629,4cyclictest25-21ksoftirqd/111:19:251
28433993629,4cyclictest25-21ksoftirqd/110:56:051
28433993629,4cyclictest25-21ksoftirqd/110:06:511
28433993629,4cyclictest25-21ksoftirqd/108:24:101
28433993628,5cyclictest25-21ksoftirqd/110:14:331
2843399362,31cyclictest26410-21cron08:19:551
2843399362,31cyclictest26410-21cron08:19:541
28432993631,3cyclictest9-21ksoftirqd/009:39:550
28432993631,3cyclictest9-21ksoftirqd/009:39:550
28432993631,3cyclictest9-21ksoftirqd/007:34:280
28432993630,4cyclictest9-21ksoftirqd/008:05:310
28432993629,4cyclictest9-21ksoftirqd/011:41:310
28432993629,4cyclictest9-21ksoftirqd/008:29:220
28432993629,4cyclictest9-21ksoftirqd/008:10:400
28432993629,4cyclictest9-21ksoftirqd/007:56:080
28432993629,4cyclictest9-21ksoftirqd/007:41:580
28432993610,22cyclictest18007-21fschecks_time11:20:190
2843299360,4cyclictest561-21rs:main0
2843299360,4cyclictest24933-21proc_pri10:25:290
28433993530,3cyclictest25-21ksoftirqd/110:41:261
28433993529,4cyclictest25-21ksoftirqd/110:27:421
28433993529,4cyclictest25-21ksoftirqd/108:32:411
28433993529,3cyclictest25-21ksoftirqd/109:13:381
28433993528,4cyclictest25-21ksoftirqd/109:46:391
28433993528,4cyclictest25-21ksoftirqd/109:29:131
28433993528,4cyclictest25-21ksoftirqd/107:29:321
28433993527,5cyclictest25-21ksoftirqd/111:48:281
28432993529,4cyclictest9-21ksoftirqd/010:54:550
28432993529,4cyclictest9-21ksoftirqd/009:17:040
28432993529,3cyclictest9-21ksoftirqd/011:48:080
28432993528,5cyclictest9-21ksoftirqd/010:39:210
28432993528,4cyclictest9-21ksoftirqd/011:57:250
28432993528,4cyclictest9-21ksoftirqd/009:45:110
2843299350,5cyclictest15159-21phc2sys-jitter10:05:290
2843299350,3cyclictest95552sleep007:36:330
2843299350,3cyclictest0-21swapper/010:20:210
28433993428,4cyclictest25-21ksoftirqd/111:05:031
28433993428,4cyclictest25-21ksoftirqd/111:05:031
28433993427,4cyclictest25-21ksoftirqd/111:52:211
28433993427,4cyclictest25-21ksoftirqd/110:36:331
28433993427,4cyclictest25-21ksoftirqd/110:21:291
28433993427,4cyclictest25-21ksoftirqd/109:30:511
28433993427,4cyclictest25-21ksoftirqd/108:36:431
2843299348,22cyclictest26987-21sort10:30:240
28432993428,3cyclictest9-21ksoftirqd/007:50:010
28432993427,4cyclictest9-21ksoftirqd/012:09:210
28432993427,4cyclictest9-21ksoftirqd/010:04:050
28432993427,4cyclictest9-21ksoftirqd/007:27:540
2843299340,3cyclictest5892-21expr10:55:130
28433993328,3cyclictest25-21ksoftirqd/111:41:121
28433993327,4cyclictest25-21ksoftirqd/108:51:051
28433993327,3cyclictest25-21ksoftirqd/109:53:321
28433993326,4cyclictest25-21ksoftirqd/111:35:001
28433993326,4cyclictest25-21ksoftirqd/109:55:551
28433993326,4cyclictest25-21ksoftirqd/109:35:481
28433993326,4cyclictest25-21ksoftirqd/109:35:481
28433993326,4cyclictest25-21ksoftirqd/109:00:121
28433993326,4cyclictest25-21ksoftirqd/107:37:301
2843399332,29cyclictest21336-21kworker/1:010:30:201
28432993327,4cyclictest9-21ksoftirqd/009:42:510
28432993215,14cyclictest9-21ksoftirqd/010:40:120
28432993210,19cyclictest15316-21sh09:00:010
2843299320,4cyclictest25083-21/usr/sbin/munin09:20:370
28433993124,4cyclictest25-21ksoftirqd/109:22:271
2843299310,6cyclictest6920-21irqrtprio08:40:210
28433993023,4cyclictest25-21ksoftirqd/109:19:071
28433993023,4cyclictest25-21ksoftirqd/107:46:401
28433992922,4cyclictest25-21ksoftirqd/107:40:401
28433992417,4cyclictest25-21ksoftirqd/112:28:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional