You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-07-26 - 15:16

Intel(R) Xeon(R) CPU E3-1220L V2 @ 2.30GHz, Linux 4.9.47-rt37 (Profile)

Latency plot of system in rack #a, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the alll highest latencies:
System rackaslot0.osadl.org (updated Mon Jul 26, 2021 12:44:35)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
244426023,19sleep00-21swapper/007:08:010
90912520,0sleep39087-21cp11:15:013
41932500,1sleep00-21swapper/012:21:190
274012500,0sleep10-21swapper/112:11:071
252725023,9sleep30-21swapper/307:09:003
122525023,8sleep20-21swapper/207:05:222
262552490,0sleep20-21swapper/211:04:012
217442480,0sleep30-21swapper/309:47:283
185622480,0sleep10-21swapper/111:52:001
135924825,9sleep10-21swapper/107:05:241
169972470,1sleep10-21swapper/110:13:491
231642440,1sleep10-21swapper/107:50:181
144512440,0sleep21714-21runrttasks10:09:022
141012430,0sleep327949-21diskmemload10:31:103
93342270,0sleep10-21swapper/111:32:481
2867992019,1cyclictest0-21swapper/209:18:022
322412190,0sleep20-21swapper/211:13:432
2868991917,1cyclictest0-21swapper/309:44:313
286599189,3cyclictest0-21swapper/009:44:350
272672180,1sleep127264-21bash11:21:551
286599179,3cyclictest0-21swapper/012:15:350
286599170,13cyclictest0-21swapper/009:53:440
2866991613,1cyclictest21292-21idleruntime-cro10:10:001
286599168,5cyclictest0-21swapper/012:32:460
286599168,4cyclictest0-21swapper/009:38:560
286599168,3cyclictest31010-21sshd10:46:490
286599162,1cyclictest0-21swapper/012:29:070
286599161,13cyclictest0-21swapper/009:57:430
2865991610,1cyclictest20050-21sshd10:00:360
286599160,3cyclictest0-21swapper/011:03:500
286599160,15cyclictest0-21swapper/010:06:450
248412160,0sleep30-21swapper/310:45:563
288732150,1sleep30-21swapper/311:30:593
286899158,1cyclictest29378-21bash12:38:263
286899150,12cyclictest0-21swapper/309:20:143
286699159,1cyclictest0-21swapper/112:04:201
286699151,1cyclictest114850irq/25-eth009:44:561
286599158,6cyclictest0-21swapper/011:20:380
286599158,6cyclictest0-21swapper/010:21:410
286599158,4cyclictest0-21swapper/011:16:420
286599153,11cyclictest27806-21rm11:08:410
286599152,1cyclictest0-21swapper/011:27:230
286599150,6cyclictest0-21swapper/010:38:210
286599150,3cyclictest23931-21sshd10:59:170
286599150,3cyclictest0-21swapper/010:12:490
286599150,14cyclictest22932-21sshd09:23:320
286599150,14cyclictest0-21swapper/011:33:160
286599150,14cyclictest0-21swapper/010:41:590
286599150,14cyclictest0-21swapper/009:31:000
286599150,10cyclictest0-21swapper/009:49:230
40882140,0sleep10-21swapper/111:05:211
286899140,13cyclictest0-21swapper/307:50:213
286899140,12cyclictest0-21swapper/309:53:153
286799140,13cyclictest25097-21sshd11:57:242
286799140,13cyclictest0-21swapper/211:45:012
286799140,13cyclictest0-21swapper/210:35:022
286699149,1cyclictest30883-21cp12:16:031
286699149,1cyclictest114850irq/25-eth012:09:191
286699149,1cyclictest0-21swapper/111:17:181
286599148,3cyclictest0-21swapper/012:35:320
286599148,2cyclictest0-21swapper/010:51:000
286599148,1cyclictest114850irq/25-eth009:15:210
286599147,3cyclictest0-21swapper/012:10:580
286599146,4cyclictest0-21swapper/011:12:130
286599143,10cyclictest0-21swapper/011:40:500
286599141,3cyclictest920-21sshd10:15:570
286599140,4cyclictest0-21swapper/011:57:040
286599140,2cyclictest0-21swapper/011:46:260
286599140,2cyclictest0-21swapper/011:38:510
286599140,13cyclictest6106-21id09:25:410
286599140,13cyclictest27797-21sshd11:53:190
286599140,13cyclictest0-21swapper/012:05:140
286599140,10cyclictest0-21swapper/010:31:140
146472140,0sleep314650-21id10:22:213
126402140,1sleep2286799cyclictest09:41:102
326632130,0sleep21714-21runrttasks10:15:512
286899139,1cyclictest0-21swapper/311:58:363
286899139,1cyclictest0-21swapper/310:10:073
286899138,4cyclictest6539-21sshd10:07:523
286899138,2cyclictest22548-21sshd12:01:273
286899135,3cyclictest0-21swapper/310:42:563
286899131,1cyclictest114850irq/25-eth009:36:203
2868991311,1cyclictest20295-21cp10:58:463
286899131,11cyclictest0-21swapper/311:21:213
2868991311,1cyclictest0-21swapper/310:27:363
286899131,11cyclictest0-21swapper/310:15:183
2868991310,2cyclictest0-21swapper/311:10:483
286899130,4cyclictest32118-21sshd11:04:513
286899130,4cyclictest2840-21sshd12:07:463
286899130,2cyclictest21263-21sshd11:38:573
286899130,2cyclictest0-21swapper/307:25:163
286899130,1cyclictest0-21swapper/309:57:063
286899130,12cyclictest8972-21bash12:26:293
286899130,12cyclictest3235-21sshd11:05:173
286899130,12cyclictest2507-21bash09:25:203
286899130,12cyclictest0-21swapper/310:51:243
286899130,12cyclictest0-21swapper/310:00:183
286899130,12cyclictest0-21swapper/309:33:163
286899130,10cyclictest29831-21sshd11:49:083
286799138,4cyclictest16965-21sshd10:27:122
286799138,4cyclictest0-21swapper/210:20:262
286799138,1cyclictest0-21swapper/209:21:342
286799130,1cyclictest6587-21sshd12:03:512
286799130,12cyclictest15408-21sshd12:14:012
286799130,12cyclictest0-21swapper/212:39:402
286799130,12cyclictest0-21swapper/212:08:302
286799130,12cyclictest0-21swapper/211:35:502
286799130,12cyclictest0-21swapper/211:15:062
286799130,12cyclictest0-21swapper/210:47:022
286799130,12cyclictest0-21swapper/210:10:332
286699139,3cyclictest0-21swapper/112:24:121
286699139,1cyclictest0-21swapper/112:38:031
286699139,1cyclictest0-21swapper/111:25:091
286699139,1cyclictest0-21swapper/109:23:251
286699138,2cyclictest0-21swapper/109:57:121
286699138,1cyclictest10250irq/24-ahci[00011:59:071
286699131,9cyclictest114850irq/25-eth010:36:161
2866991311,1cyclictest582-21bash11:35:591
2866991311,1cyclictest0-21swapper/111:03:221
286699131,11cyclictest0-21swapper/109:35:211
286699130,4cyclictest0-21swapper/109:34:331
286699130,4cyclictest0-21swapper/109:19:331
286699130,1cyclictest0-21swapper/110:45:131
286699130,1cyclictest0-21swapper/110:20:111
286699130,12cyclictest8662-21bash10:30:251
286699130,12cyclictest8583-21sshd12:26:261
286699130,12cyclictest4072-21bash10:16:241
286699130,12cyclictest20739-21sshd12:32:461
286699130,12cyclictest12640-21bash09:50:371
286699130,12cyclictest0-21swapper/111:41:441
286699130,12cyclictest0-21swapper/111:11:091
286699130,12cyclictest0-21swapper/109:46:211
286699130,12cyclictest0-21swapper/109:26:131
286699130,12cyclictest0-21swapper/108:30:191
286599138,2cyclictest0-21swapper/010:26:180
286599130,12cyclictest0-21swapper/012:02:080
286599130,12cyclictest0-21swapper/009:10:140
221612130,1sleep20-21swapper/209:56:322
286899129,2cyclictest0-21swapper/312:30:243
286899128,4cyclictest0-21swapper/311:51:503
286899128,2cyclictest0-21swapper/312:10:383
286899122,9cyclictest5348-21sshd09:11:173
286899121,2cyclictest0-21swapper/309:15:313
286899121,0cyclictest0-21swapper/307:40:003
2868991210,1cyclictest32097-21sshd12:20:393
2868991210,1cyclictest0-21swapper/311:40:173
286899120,12cyclictest0-21swapper/310:37:343
286899120,11cyclictest0-21swapper/311:25:123
286899120,10cyclictest0-21swapper/312:15:043
286799129,2cyclictest0-21swapper/210:40:432
286799128,3cyclictest635-21rm09:34:382
286799127,4cyclictest0-21swapper/210:03:342
2867991210,2cyclictest0-21swapper/210:55:002
2867991210,1cyclictest1714-21runrttasks10:50:382
286799120,9cyclictest0-21swapper/210:30:512
286799120,1cyclictest26363-21sshd11:26:132
286799120,1cyclictest0-21swapper/212:31:072
286799120,1cyclictest0-21swapper/211:06:002
286799120,11cyclictest3567-21sshd11:40:522
286799120,11cyclictest17955-21bash09:51:242
286799120,11cyclictest15707-21bash12:22:582
286799120,11cyclictest114850irq/25-eth009:45:132
286799120,11cyclictest0-21swapper/211:21:162
286799120,11cyclictest0-21swapper/209:37:272
286799120,11cyclictest0-21swapper/208:19:502
286699129,2cyclictest0-21swapper/110:55:111
286699128,1cyclictest114850irq/25-eth009:11:381
286699120,12cyclictest0-21swapper/110:40:071
286699120,11cyclictest32576-21sshd10:51:291
286699120,11cyclictest0-21swapper/110:01:001
286699120,0cyclictest29337-21sshd11:49:051
286599120,9cyclictest0-21swapper/007:50:210
226612120,0sleep20-21swapper/211:30:142
286899110,2cyclictest0-21swapper/307:15:023
286799119,1cyclictest0-21swapper/212:25:142
286799118,1cyclictest4103-21sshd11:50:072
286799110,2cyclictest0-21swapper/209:25:152
286799110,1cyclictest0-21swapper/208:10:592
286799110,1cyclictest0-21swapper/207:45:532
286799110,11cyclictest0-21swapper/208:51:102
286799110,11cyclictest0-21swapper/208:48:582
286799110,10cyclictest0-21swapper/212:15:092
286799110,10cyclictest0-21swapper/209:10:042
286699119,1cyclictest0-21swapper/107:25:201
2866991110,1cyclictest0-21swapper/107:24:551
286699110,11cyclictest0-21swapper/108:00:341
286699110,11cyclictest0-21swapper/107:17:561
286699110,10cyclictest0-21swapper/110:25:041
286599110,7cyclictest3622-21timerandwakeup08:15:330
286899100,1cyclictest0-21swapper/308:22:473
286899100,10cyclictest0-21swapper/308:30:553
286899100,10cyclictest0-21swapper/308:06:593
286899100,10cyclictest0-21swapper/307:55:503
286899100,10cyclictest0-21swapper/307:21:553
286799109,1cyclictest0-21swapper/209:00:002
286799109,1cyclictest0-21swapper/208:57:482
286799109,1cyclictest0-21swapper/208:03:122
286799109,1cyclictest0-21swapper/207:42:522
286799109,1cyclictest0-21swapper/207:25:482
286799109,1cyclictest0-21swapper/207:15:482
286799100,1cyclictest0-21swapper/208:34:002
286799100,10cyclictest0-21swapper/209:08:292
286799100,10cyclictest0-21swapper/207:55:042
286799100,10cyclictest0-21swapper/207:22:262
286699109,1cyclictest0-21swapper/109:01:471
286699109,1cyclictest0-21swapper/107:49:321
286699109,1cyclictest0-21swapper/107:39:051
286699100,9cyclictest0-21swapper/107:10:171
286699100,1cyclictest0-21swapper/107:56:311
286699100,10cyclictest0-21swapper/108:52:591
286699100,10cyclictest0-21swapper/108:46:221
286599100,9cyclictest9102-21tune2fs07:20:180
286599100,9cyclictest0-21swapper/007:15:170
286599100,7cyclictest6908-21sed08:25:210
286599100,7cyclictest0-21swapper/008:00:250
286599100,2cyclictest0-21swapper/008:10:180
286599100,1cyclictest0-21swapper/007:25:200
28689999,0cyclictest0-21swapper/307:40:373
28689998,1cyclictest0-21swapper/308:37:533
28689998,1cyclictest0-21swapper/308:18:293
28689998,1cyclictest0-21swapper/308:10:423
28689990,9cyclictest0-21swapper/307:45:163
28679998,1cyclictest0-21swapper/207:38:482
28679998,1cyclictest0-21swapper/207:11:582
28669999,0cyclictest0-21swapper/108:59:351
28669998,1cyclictest0-21swapper/108:19:241
28669998,1cyclictest0-21swapper/108:10:341
28659998,1cyclictest0-21swapper/008:20:240
28659995,2cyclictest0-21swapper/007:33:530
28659994,3cyclictest0-21swapper/009:07:520
28659991,6cyclictest0-21swapper/008:55:220
28659990,7cyclictest0-21swapper/008:52:280
28689987,1cyclictest0-21swapper/308:58:563
28689987,1cyclictest0-21swapper/308:52:593
28689983,5cyclictest0-21swapper/307:30:443
28689981,6cyclictest0-21swapper/309:05:193
28689981,6cyclictest0-21swapper/309:00:223
28689981,6cyclictest0-21swapper/308:45:223
28689981,6cyclictest0-21swapper/308:25:383
28689980,8cyclictest0-21swapper/308:40:063
28679987,1cyclictest0-21swapper/208:22:372
28679987,1cyclictest0-21swapper/208:09:072
28679987,1cyclictest0-21swapper/207:31:222
28679980,5cyclictest0-21swapper/208:25:332
28669981,2cyclictest0-21swapper/108:25:261
28669980,7cyclictest0-21swapper/107:40:401
28659984,3cyclictest0-21swapper/009:00:250
28659984,3cyclictest0-21swapper/007:40:300
28659981,6cyclictest0-21swapper/008:30:190
28659981,3cyclictest0-21swapper/007:10:150
28659980,7cyclictest16582-21cstates08:45:150
28659980,6cyclictest14852-21munin-node08:40:410
28659980,6cyclictest0-21swapper/007:45:480
28659980,5cyclictest0-21swapper/007:35:180
28659980,4cyclictest0-21swapper/008:35:170
28689971,5cyclictest0-21swapper/307:10:143
28689970,6cyclictest0-21swapper/308:00:273
28679975,2cyclictest0-21swapper/208:39:282
28679970,6cyclictest0-21swapper/208:40:422
28679970,6cyclictest0-21swapper/207:50:202
28669976,1cyclictest0-21swapper/109:06:121
28669975,1cyclictest15330-21processes08:40:511
28669970,6cyclictest0-21swapper/108:20:281
28659970,5cyclictest29708-21munin-node08:05:200
28659970,1cyclictest24940-21apache_accesses07:55:140
28669961,1cyclictest0-21swapper/107:30:271
28669950,2cyclictest0-21swapper/108:35:191
28669950,0cyclictest0-21swapper/108:05:171
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional