You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-08-04 - 21:59

Intel(R) Xeon(R) CPU E31220L @ 2.20GHz, Linux 4.9.47-rt37 (Profile)

Latency plot of system in rack #5, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the alll highest latencies:
System rack5slot0.osadl.org (updated Wed Aug 04, 2021 12:44:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1638021180,4sleep1261499cyclictest11:58:051
1906621070,0sleep219065-21id10:06:222
838521060,3sleep1261499cyclictest10:14:311
30982960,1sleep10-21swapper/108:20:281
282772880,1sleep20-21swapper/210:17:242
241428832,12sleep20-21swapper/207:09:452
278162870,1sleep027810-21bash09:39:030
241328722,53sleep10-21swapper/107:09:441
9592860,2sleep30-21swapper/312:20:233
200362850,3sleep30-21swapper/309:47:243
148852850,2sleep30-21swapper/310:39:363
224482830,4sleep30-21swapper/312:28:403
215628231,38sleep30-21swapper/307:06:463
264512800,1sleep30-21swapper/309:10:163
201122780,0sleep00-21swapper/012:13:270
278662760,2sleep20-21swapper/209:48:342
183512750,2sleep20-21swapper/211:09:212
311822740,1sleep30-21swapper/309:39:353
94122730,2sleep10-21swapper/112:31:311
33272670,3sleep1261499cyclictest09:54:341
280212660,2sleep10-21swapper/112:04:501
235825632,21sleep00-21swapper/007:09:040
177122440,5sleep3262699cyclictest11:53:213
83262360,2sleep30-21swapper/312:36:173
224902360,1sleep022482-21sshd12:08:540
246562340,0sleep324657-21bash10:50:433
102992313,26sleep3138550irq/27-eth011:03:143
259572301,27sleep3138550irq/27-eth011:05:313
261899272,10cyclictest138550irq/27-eth011:58:322
88312250,22sleep3138550irq/27-eth010:09:483
291802240,22sleep3138550irq/27-eth009:58:233
291802240,22sleep3138550irq/27-eth009:58:233
284222240,2sleep30-21swapper/311:59:593
200932240,0sleep30-21swapper/311:29:133
157762240,2sleep115777-21sshd11:23:391
80612230,1sleep30-21swapper/309:40:473
323582230,21sleep1138550irq/27-eth011:11:211
261899234,6cyclictest138550irq/27-eth010:20:302
261499230,22cyclictest0-21swapper/112:38:231
117352230,1sleep011724-21sshd09:31:500
262699220,17cyclictest0-21swapper/311:32:313
261899226,13cyclictest0-21swapper/212:32:212
261899225,13cyclictest0-21swapper/212:22:492
261899224,12cyclictest0-21swapper/212:25:012
261899223,10cyclictest138550irq/27-eth010:58:122
261899222,8cyclictest138550irq/27-eth011:16:532
261499220,21cyclictest1877-21nfsd10:05:501
51172210,2sleep30-21swapper/310:14:003
262699210,17cyclictest29776-21sshd12:10:013
261899213,3cyclictest138550irq/27-eth010:13:182
168232210,2sleep20-21swapper/210:01:132
7862200,0sleep2785-21sshd11:11:302
261899204,11cyclictest0-21swapper/207:30:592
261899203,6cyclictest138550irq/27-eth011:38:092
261899203,4cyclictest138550irq/27-eth010:41:132
261899200,4cyclictest31628-21bash09:44:252
261399200,16cyclictest6673-21sshd11:22:110
262699190,3cyclictest0-21swapper/310:47:573
262699190,3cyclictest0-21swapper/309:20:493
262699190,16cyclictest0-21swapper/310:40:593
261899193,5cyclictest0-21swapper/208:33:052
261899193,13cyclictest0-21swapper/209:21:272
146302190,2sleep10-21swapper/110:00:511
262699185,3cyclictest0-21swapper/311:45:223
261899184,9cyclictest0-21swapper/207:57:212
261899184,13cyclictest0-21swapper/209:31:252
261899182,13cyclictest0-21swapper/212:01:082
261899182,11cyclictest12666-21sshd09:36:442
261899181,7cyclictest138550irq/27-eth009:54:252
261899181,16cyclictest0-21swapper/209:55:302
261899181,16cyclictest0-21swapper/209:55:302
192042180,16sleep328750irq/26-ahci[00010:15:573
262699176,4cyclictest17086-21sshd11:14:043
262699175,4cyclictest13292-21sshd10:20:013
262699174,3cyclictest0-21swapper/307:55:163
262699170,3cyclictest7202-21taskset09:16:513
261899174,8cyclictest0-21swapper/208:26:072
261899173,11cyclictest0-21swapper/208:58:162
261899172,4cyclictest0-21swapper/211:26:012
261899172,3cyclictest0-21swapper/208:15:462
261899172,12cyclictest0-21swapper/209:19:032
261899172,10cyclictest0-21swapper/211:23:192
261899171,15cyclictest0-21swapper/210:49:442
261899170,12cyclictest0-21swapper/207:13:472
261499174,12cyclictest138550irq/27-eth009:25:041
261499170,15cyclictest0-21swapper/111:31:531
261499170,15cyclictest0-21swapper/110:21:271
261499170,13cyclictest0-21swapper/107:20:481
261399178,8cyclictest0-21swapper/010:14:090
261399170,12cyclictest20202-21sshd11:09:390
136052170,15sleep2138550irq/27-eth010:29:472
262699162,9cyclictest11752-21sshd12:07:123
262699161,1cyclictest1360-21sshd10:32:413
262699161,0cyclictest0-21swapper/312:32:363
262699160,14cyclictest0-21swapper/312:02:393
262699160,14cyclictest0-21swapper/311:41:233
262699160,13cyclictest0-21swapper/312:15:463
262699160,13cyclictest0-21swapper/309:53:023
262699160,11cyclictest0-21swapper/311:22:313
261899165,8cyclictest0-21swapper/208:24:192
261899165,8cyclictest0-21swapper/207:40:312
261899165,6cyclictest0-21swapper/211:52:082
261899165,6cyclictest0-21swapper/210:32:392
261899164,7cyclictest8663-21sshd12:06:402
261899164,7cyclictest138550irq/27-eth011:01:292
261899163,8cyclictest0-21swapper/209:27:232
261899162,9cyclictest0-21swapper/212:19:572
261899162,9cyclictest0-21swapper/207:26:342
261899162,5cyclictest0-21swapper/208:07:142
261899162,13cyclictest0-21swapper/207:35:332
261899161,10cyclictest26579-21sshd11:49:502
261899161,10cyclictest0-21swapper/212:37:332
2614991613,2cyclictest0-21swapper/110:15:501
2614991612,3cyclictest22071-21sshd09:14:241
2614991610,5cyclictest0-21swapper/111:35:371
261499160,7cyclictest0-21swapper/112:13:301
261499160,14cyclictest5909-21sshd11:02:311
261499160,12cyclictest0-21swapper/109:34:251
261399168,5cyclictest0-21swapper/007:31:040
261399160,14cyclictest0-21swapper/007:55:150
245802160,14sleep1138550irq/27-eth009:57:401
245802160,14sleep1138550irq/27-eth009:57:401
262699157,6cyclictest2121-21runrttasks07:30:493
262699156,3cyclictest13297-21sshd10:58:483
262699152,12cyclictest138550irq/27-eth007:45:203
262699152,12cyclictest138550irq/27-eth007:45:203
262699150,6cyclictest0-21swapper/309:25:173
262699150,4cyclictest0-21swapper/310:25:213
262699150,3cyclictest138550irq/27-eth009:33:413
262699150,12cyclictest0-21swapper/308:20:013
262699150,12cyclictest0-21swapper/308:10:233
262699150,11cyclictest0-21swapper/311:15:283
261899155,6cyclictest0-21swapper/210:50:552
261899155,5cyclictest0-21swapper/210:36:102
261899154,6cyclictest138550irq/27-eth009:05:102
261899154,4cyclictest0-21swapper/208:02:272
261899154,10cyclictest0-21swapper/207:20:392
261899153,9cyclictest0-21swapper/207:19:212
261899153,6cyclictest138550irq/27-eth008:49:092
261899151,9cyclictest0-21swapper/207:46:092
261899151,9cyclictest0-21swapper/207:46:092
261899151,13cyclictest0-21swapper/209:12:502
261899150,14cyclictest0-21swapper/212:11:192
261899150,10cyclictest138550irq/27-eth011:40:012
261499158,6cyclictest0-21swapper/109:36:281
261499158,5cyclictest5187-21sshd11:26:531
261499158,3cyclictest0-21swapper/112:27:131
261499158,2cyclictest138550irq/27-eth009:15:081
261499157,7cyclictest1987-21sshd11:49:031
261499157,5cyclictest0-21swapper/112:06:501
261499152,12cyclictest5561-21sshd12:16:021
2614991511,3cyclictest21345-21diskstats10:45:231
261499150,6cyclictest29088-21sshd10:36:471
261499150,14cyclictest0-21swapper/111:08:141
261499150,13cyclictest0-21swapper/111:43:551
261499150,13cyclictest0-21swapper/111:15:221
261499150,13cyclictest0-21swapper/109:45:281
261499150,11cyclictest2260-21sshd10:57:041
261499150,10cyclictest24976-21basename08:00:001
261399157,5cyclictest0-21swapper/012:33:110
261399157,2cyclictest18217-21munin-run07:45:000
261399150,6cyclictest0-21swapper/007:20:000
261399150,2cyclictest0-21swapper/009:46:440
261399150,13cyclictest0-21swapper/012:04:570
261399150,12cyclictest0-21swapper/012:24:130
261399150,12cyclictest0-21swapper/011:57:380
261399150,12cyclictest0-21swapper/011:28:250
261399150,12cyclictest0-21swapper/011:02:060
261399150,11cyclictest0-21swapper/011:36:120
261399150,11cyclictest0-21swapper/011:10:090
261399150,11cyclictest0-21swapper/010:35:170
261399150,11cyclictest0-21swapper/009:56:020
261399150,11cyclictest0-21swapper/009:56:020
125472150,1sleep00-21swapper/010:34:250
43792140,12sleep04488-21bash12:35:380
262699145,3cyclictest27063-21sshd11:35:123
262699142,11cyclictest0-21swapper/308:49:403
262699141,12cyclictest1877-21nfsd07:16:323
262699140,5cyclictest0-21swapper/309:05:253
262699140,13cyclictest2121-21runrttasks07:24:123
262699140,13cyclictest0-21swapper/310:01:563
262699140,13cyclictest0-21swapper/309:01:033
262699140,13cyclictest0-21swapper/307:12:313
261899148,4cyclictest2011-21sshd11:31:162
261899141,10cyclictest138550irq/27-eth008:51:212
261899140,13cyclictest0-21swapper/209:00:422
261899140,13cyclictest0-21swapper/208:36:092
261499149,4cyclictest138550irq/27-eth009:23:461
261499147,6cyclictest1876-21nfsd12:20:581
261499146,6cyclictest0-21swapper/109:00:201
261499142,11cyclictest138550irq/27-eth010:50:471
261499140,13cyclictest9391-21sshd10:33:561
261499140,13cyclictest25468-21sshd10:41:021
261499140,11cyclictest138550irq/27-eth009:43:231
261499140,11cyclictest0-21swapper/111:50:231
261399147,5cyclictest0-21swapper/010:26:550
261399147,3cyclictest3950-21sshd09:21:040
261399142,9cyclictest138550irq/27-eth010:05:370
261399142,1cyclictest0-21swapper/007:20:200
261399141,6cyclictest0-21swapper/010:45:160
2613991412,1cyclictest28116-21head11:15:360
261399140,13cyclictest0-21swapper/012:18:250
261399140,13cyclictest0-21swapper/011:45:580
261399140,13cyclictest0-21swapper/010:51:410
261399140,13cyclictest0-21swapper/009:26:550
261399140,11cyclictest18331-21sshd10:01:270
261399140,11cyclictest0-21swapper/010:58:570
261399140,10cyclictest21367-21grep09:00:350
261399140,10cyclictest0-21swapper/010:23:130
262699138,1cyclictest0-21swapper/308:30:373
262699137,5cyclictest138550irq/27-eth008:56:173
2626991311,1cyclictest0-21swapper/307:40:243
262699130,5cyclictest0-21swapper/308:35:353
262699130,5cyclictest0-21swapper/308:27:413
262699130,12cyclictest0-21swapper/308:16:033
262699130,12cyclictest0-21swapper/307:36:433
262699130,11cyclictest9721-21df07:25:173
262699130,11cyclictest25951-21irqstats08:00:243
262699130,11cyclictest11149-21diskstats08:40:173
261899138,4cyclictest11553-21interrupts08:40:182
261899136,6cyclictest0-21swapper/207:50:032
261899132,7cyclictest0-21swapper/208:10:082
261499138,4cyclictest1877-21nfsd07:15:391
261499137,5cyclictest0-21swapper/108:35:301
261499132,10cyclictest138550irq/27-eth007:40:291
261499130,5cyclictest0-21swapper/109:06:481
261499130,4cyclictest1955-21snmpd08:43:111
261499130,2cyclictest138550irq/27-eth008:16:091
261499130,12cyclictest0-21swapper/108:59:381
261499130,12cyclictest0-21swapper/108:54:441
261499130,12cyclictest0-21swapper/108:46:101
261499130,12cyclictest0-21swapper/108:30:281
261499130,12cyclictest0-21swapper/108:12:281
261499130,12cyclictest0-21swapper/108:09:401
261499130,12cyclictest0-21swapper/107:30:341
261499130,12cyclictest0-21swapper/107:12:281
261499130,11cyclictest21448-21irqstats07:50:191
261499130,11cyclictest18056-21sshd10:25:291
261399137,1cyclictest9776-21sshd12:26:380
261399136,6cyclictest0-21swapper/008:30:180
261399135,5cyclictest0-21swapper/010:15:550
261399134,8cyclictest3-21ksoftirqd/011:51:280
261399133,9cyclictest0-21swapper/011:40:200
261399132,1cyclictest0-21swapper/008:16:220
2613991311,1cyclictest5376-21cat08:25:210
2613991311,1cyclictest0-21swapper/007:49:270
2613991311,1cyclictest0-21swapper/007:49:270
261399130,8cyclictest30552-21memory08:10:200
261399130,5cyclictest15634-21df_inode08:50:120
261399130,3cyclictest0-21swapper/007:12:260
261399130,1cyclictest0-21swapper/009:43:420
261399130,12cyclictest1987-21sshd09:54:390
261399130,12cyclictest1678-21bash10:42:250
261399130,12cyclictest0-21swapper/011:31:170
261399130,12cyclictest0-21swapper/009:15:030
261399130,12cyclictest0-21swapper/009:11:100
261399130,12cyclictest0-21swapper/008:05:150
262699127,4cyclictest0-21swapper/308:05:593
262699120,11cyclictest0-21swapper/307:51:053
262699120,10cyclictest15576-21df08:50:123
261499120,4cyclictest0-21swapper/107:25:021
261499120,3cyclictest1943-21snmpd08:25:171
261499120,11cyclictest25192-21munin-node08:00:181
261499120,11cyclictest0-21swapper/107:35:151
261499120,10cyclictest0-21swapper/107:45:141
261499120,10cyclictest0-21swapper/107:45:141
261399128,3cyclictest0-21swapper/007:25:320
261399127,4cyclictest1877-21nfsd08:56:000
261399120,2cyclictest0-21swapper/008:20:500
261399120,11cyclictest0-21swapper/009:05:000
261399120,11cyclictest0-21swapper/008:45:230
261399120,11cyclictest0-21swapper/008:40:580
261399120,11cyclictest0-21swapper/008:35:390
261399120,11cyclictest0-21swapper/007:50:210
261399120,11cyclictest0-21swapper/007:36:180
261399120,10cyclictest0-21swapper/008:01:170
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional