You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-11-28 - 02:35

x86 Intel Xeon E3-1220L V2 @2300 MHz, Linux 4.9.47-rt37 (Profile)

Latency plot of system in rack #a, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the alll highest latencies:
System rackaslot0.osadl.org (updated Sun Nov 27, 2022 12:44:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
579121030,0sleep35790-21bash11:35:183
579121030,0sleep35790-21bash11:35:183
40922910,0sleep30-21swapper/311:59:253
226602900,1sleep222650-21sshd11:31:162
18872850,0sleep11876-21sshd12:10:011
327052840,0sleep228-21rcuc/209:12:592
3642620,1sleep10-21swapper/112:25:591
256692620,6sleep230-21ksoftirqd/211:07:082
77602570,0sleep20-21swapper/210:46:332
184322570,1sleep20-21swapper/211:14:462
133092570,0sleep10-21swapper/109:22:161
113832570,0sleep10-21swapper/110:46:521
62262560,2sleep3389699cyclictest11:46:033
162692560,1sleep316268-21sshd10:36:213
110262560,0sleep111024-21bash11:43:501
67962540,1sleep06791-21sshd12:31:550
293392540,0sleep00-21swapper/009:48:140
40562530,0sleep30-21swapper/309:29:463
256132530,0sleep30-21swapper/312:03:553
149232530,0sleep314922-21bash11:00:453
359925226,8sleep00-21swapper/007:09:340
85042510,1sleep30-21swapper/310:33:003
337925129,18sleep30-21swapper/307:06:523
146525129,18sleep20-21swapper/207:05:112
64322490,1sleep36431-21sshd11:24:323
231952490,0sleep223191-21sshd10:15:052
124272490,0sleep20-21swapper/210:44:172
341424826,19sleep10-21swapper/107:07:181
254262480,1sleep00-21swapper/011:09:550
135652480,0sleep30-21swapper/311:52:043
67622470,0sleep26737-21sshd11:19:112
293532470,1sleep129354-21bash09:56:251
255512460,0sleep00-21swapper/009:42:260
173422460,0sleep21787-21runrttasks07:35:182
146552460,1sleep31632-21sshd10:58:053
123892460,0sleep10-21swapper/111:38:331
123892460,0sleep10-21swapper/111:38:331
119872460,0sleep10-21swapper/109:44:031
37052450,0sleep00-21swapper/010:49:000
103412450,0sleep10-21swapper/112:16:021
318702440,0sleep00-21swapper/011:53:380
106312390,0sleep310620-21sshd09:46:373
215352320,0sleep00-21swapper/010:06:410
3896992519,5cyclictest0-21swapper/309:55:443
389699250,1cyclictest8818-21bash11:27:233
321452250,1sleep232144-21bash09:42:592
3893992220,1cyclictest18364-21sshd11:36:170
3893992220,1cyclictest18364-21sshd11:36:170
3895992119,1cyclictest0-21swapper/211:27:072
3893992119,1cyclictest29205-21sshd12:23:040
389699200,19cyclictest19833-21sshd09:17:203
389699200,19cyclictest19833-21sshd09:17:203
3895992018,1cyclictest0-21swapper/212:24:302
389399208,9cyclictest0-21swapper/009:37:470
319682200,0sleep331965-21sshd10:21:143
3896991917,1cyclictest0-21swapper/310:27:223
3893991918,1cyclictest29013-21sshd11:26:210
276742190,1sleep327663-21sshd11:12:463
3895991816,1cyclictest0-21swapper/211:04:512
3894991817,1cyclictest30176-21sshd09:51:011
92642170,1sleep01787-21runrttasks09:54:470
286282170,1sleep228606-21sshd09:31:452
90132160,1sleep229020-21diskmemload10:05:352
52042160,0sleep00-21swapper/010:21:440
389599165,10cyclictest30-21ksoftirqd/212:31:312
389399168,6cyclictest28443-21sshd10:10:040
389399160,15cyclictest0-21swapper/009:23:190
389399160,13cyclictest0-21swapper/011:03:430
389399160,12cyclictest29407-21sshd12:12:170
389399160,12cyclictest0-21swapper/010:41:050
245932160,1sleep224583-21sshd09:28:462
236992160,0sleep10-21swapper/110:34:181
112012160,0sleep03-21ksoftirqd/011:46:290
389599155,9cyclictest30-21ksoftirqd/212:35:342
389599155,1cyclictest30-21ksoftirqd/209:21:202
3895991513,1cyclictest30-21ksoftirqd/211:46:342
3895991513,1cyclictest30-21ksoftirqd/210:34:372
3895991513,1cyclictest30-21ksoftirqd/209:17:122
3895991513,1cyclictest30-21ksoftirqd/209:17:122
389599151,1cyclictest120050irq/25-eth010:50:082
3895991510,4cyclictest30-21ksoftirqd/209:47:052
3894991513,1cyclictest8509-21sshd10:13:481
3894991513,1cyclictest23315-21sshd09:34:061
389399159,3cyclictest0-21swapper/009:55:360
389399153,1cyclictest0-21swapper/010:03:240
3893991511,3cyclictest17284-21sshd10:39:140
389399151,12cyclictest14912-21sshd09:30:340
389399150,6cyclictest0-21swapper/011:31:520
389399150,4cyclictest28331-21sshd11:42:330
389399150,3cyclictest0-21swapper/012:35:170
22392150,0sleep10-21swapper/112:39:421
389699144,1cyclictest21212-21sshd11:33:543
389699141,2cyclictest0-21swapper/310:00:143
3896991411,2cyclictest0-21swapper/310:11:083
3896991410,1cyclictest28028-21sshd09:23:313
389699140,2cyclictest0-21swapper/309:33:463
389699140,13cyclictest30902-21bash12:36:373
389599147,1cyclictest28-21rcuc/210:35:382
3895991413,1cyclictest30-21ksoftirqd/212:19:262
3895991413,1cyclictest30-21ksoftirqd/210:21:122
3895991412,1cyclictest30-21ksoftirqd/212:27:572
3895991412,1cyclictest30-21ksoftirqd/212:03:462
3895991412,1cyclictest30-21ksoftirqd/211:50:382
3895991412,1cyclictest30-21ksoftirqd/211:43:032
3895991412,1cyclictest30-21ksoftirqd/211:35:222
3895991412,1cyclictest30-21ksoftirqd/211:35:222
3895991412,1cyclictest30-21ksoftirqd/211:21:032
3895991412,1cyclictest30-21ksoftirqd/210:55:172
3895991412,1cyclictest30-21ksoftirqd/210:25:212
3895991412,1cyclictest30-21ksoftirqd/209:57:282
389599140,13cyclictest0-21swapper/210:14:572
389499149,2cyclictest0-21swapper/112:21:081
389499148,2cyclictest0-21swapper/112:32:481
389499141,2cyclictest24406-21sshd10:56:071
389499140,2cyclictest22993-21sshd09:39:321
389499140,13cyclictest0-21swapper/112:01:501
389499140,11cyclictest0-21swapper/109:10:051
389399148,5cyclictest6458-21sshd11:10:570
389399140,5cyclictest0-21swapper/010:53:110
389399140,1cyclictest0-21swapper/012:27:190
389399140,13cyclictest0-21swapper/011:20:380
389399140,13cyclictest0-21swapper/010:17:180
389399140,12cyclictest0-21swapper/011:17:580
389399140,11cyclictest12683-21sshd12:00:090
389399140,11cyclictest0-21swapper/012:15:140
389399140,11cyclictest0-21swapper/010:55:200
32132140,0sleep10-21swapper/111:26:551
296632140,0sleep30-21swapper/311:42:393
187752140,0sleep10-21swapper/109:47:191
389699139,1cyclictest0-21swapper/310:16:133
389699138,4cyclictest32627-21bash12:07:093
389699138,2cyclictest0-21swapper/311:06:453
3896991311,1cyclictest11247-21bash09:52:113
389699131,10cyclictest26748-21sshd10:07:093
389699130,4cyclictest0-21swapper/312:26:083
389699130,12cyclictest0-21swapper/310:41:093
389699130,11cyclictest5924-21sshd12:21:053
389699130,10cyclictest4144-21sshd09:35:133
389599133,9cyclictest30-21ksoftirqd/212:05:182
389599133,1cyclictest11697-21sshd10:03:092
3895991312,1cyclictest30-21ksoftirqd/211:55:402
3895991311,1cyclictest30-21ksoftirqd/209:35:452
389599130,12cyclictest9500-21bash12:10:322
389599130,12cyclictest0-21swapper/209:50:212
389499139,3cyclictest0-21swapper/111:00:291
389499139,3cyclictest0-21swapper/109:15:131
389499139,3cyclictest0-21swapper/109:15:131
389499139,3cyclictest0-21swapper/108:10:241
389499139,1cyclictest15675-21sshd10:17:081
389499139,1cyclictest0-21swapper/111:51:181
389499139,1cyclictest0-21swapper/110:20:221
389499139,1cyclictest0-21swapper/109:27:281
389499131,3cyclictest0-21swapper/111:22:421
389499131,1cyclictest120050irq/25-eth010:50:491
389499131,1cyclictest120050irq/25-eth010:38:371
389499131,11cyclictest0-21swapper/111:32:231
389499130,2cyclictest2503-21sshd11:10:351
389499130,12cyclictest17969-21sshd11:49:491
389499130,12cyclictest0-21swapper/112:07:591
389499130,12cyclictest0-21swapper/111:17:091
389499130,12cyclictest0-21swapper/111:05:131
389499130,12cyclictest0-21swapper/110:41:011
389499130,10cyclictest32294-21sshd10:04:561
389499130,10cyclictest0-21swapper/110:27:381
389499130,10cyclictest0-21swapper/110:06:051
389399139,3cyclictest0-21swapper/009:10:400
389399133,1cyclictest0-21swapper/009:17:110
389399133,1cyclictest0-21swapper/009:17:110
389399132,9cyclictest0-21swapper/010:33:370
3893991312,1cyclictest28825-21kworker/0:008:41:450
3893991311,1cyclictest15218-1kworker/0:1H09:26:330
3893991310,2cyclictest0-21swapper/011:55:180
389399130,12cyclictest13282-21bash12:05:310
389699129,2cyclictest0-21swapper/310:50:193
389699129,2cyclictest0-21swapper/309:40:403
389699129,2cyclictest0-21swapper/307:15:023
389699121,2cyclictest0-21swapper/312:10:243
389699121,2cyclictest0-21swapper/310:46:193
3896991210,1cyclictest27386-21munin-node07:55:223
3896991210,1cyclictest1632-21sshd12:30:193
389699120,1cyclictest0-21swapper/309:10:103
389699120,11cyclictest1663-21sshd12:15:213
389699120,11cyclictest0-21swapper/308:53:213
389699120,10cyclictest0-21swapper/311:15:023
389599120,1cyclictest0-21swapper/207:15:292
389499120,12cyclictest0-21swapper/111:58:181
389499120,11cyclictest0-21swapper/107:35:391
389399129,2cyclictest0-21swapper/007:15:020
3893991210,1cyclictest15512-21sshd10:25:210
3896991110,1cyclictest0-21swapper/308:46:163
3896991110,1cyclictest0-21swapper/308:25:483
389699110,1cyclictest0-21swapper/308:40:133
389699110,1cyclictest0-21swapper/308:21:443
389699110,1cyclictest0-21swapper/308:01:453
389699110,1cyclictest0-21swapper/307:33:263
389699110,11cyclictest0-21swapper/308:55:473
389699110,11cyclictest0-21swapper/308:07:063
389699110,11cyclictest0-21swapper/307:48:033
389699110,10cyclictest0-21swapper/308:37:323
389699110,10cyclictest0-21swapper/308:15:213
389599110,1cyclictest0-21swapper/208:50:372
389599110,11cyclictest0-21swapper/208:59:142
389599110,11cyclictest0-21swapper/208:00:412
389499110,10cyclictest0-21swapper/109:00:211
389399119,1cyclictest0-21swapper/007:55:150
389399113,4cyclictest0-21swapper/008:45:140
389399110,7cyclictest17648-21nfsd407:35:210
389399110,11cyclictest0-21swapper/009:01:080
389399110,11cyclictest0-21swapper/008:27:270
389399110,11cyclictest0-21swapper/008:21:320
389399110,11cyclictest0-21swapper/007:42:510
389699100,1cyclictest29001-21sshd09:10:003
389699100,1cyclictest0-21swapper/309:01:563
389699100,1cyclictest0-21swapper/308:34:523
389699100,1cyclictest0-21swapper/307:14:423
389699100,10cyclictest0-21swapper/308:10:493
389699100,10cyclictest0-21swapper/307:27:013
389599109,1cyclictest0-21swapper/208:33:472
389599109,1cyclictest0-21swapper/208:18:492
389599109,1cyclictest0-21swapper/207:33:092
389599108,2cyclictest13185-21kworker/2:207:48:212
389599100,1cyclictest0-21swapper/207:21:042
389599100,10cyclictest0-21swapper/208:41:412
389599100,10cyclictest0-21swapper/207:26:452
389499109,1cyclictest0-21swapper/108:45:001
389499109,1cyclictest0-21swapper/108:32:561
389499108,1cyclictest6615-21cpu07:15:021
389499100,1cyclictest26734-21perl07:55:151
389499100,10cyclictest0-21swapper/108:23:531
389499100,10cyclictest0-21swapper/107:11:571
389399109,1cyclictest0-21swapper/009:06:010
389399109,1cyclictest0-21swapper/008:50:420
389399109,1cyclictest0-21swapper/008:13:170
389399109,1cyclictest0-21swapper/007:50:170
389399109,1cyclictest0-21swapper/007:30:040
389399100,8cyclictest0-21swapper/008:55:160
389399100,7cyclictest0-21swapper/008:30:170
389399100,10cyclictest0-21swapper/008:05:020
389399100,10cyclictest0-21swapper/007:20:400
389399100,10cyclictest0-21swapper/007:10:110
38969990,8cyclictest0-21swapper/307:50:173
38959998,1cyclictest0-21swapper/209:08:252
38959998,1cyclictest0-21swapper/208:05:122
38959993,6cyclictest0-21swapper/208:28:362
38949990,9cyclictest0-21swapper/108:57:541
38939999,0cyclictest0-21swapper/008:36:320
38939998,1cyclictest0-21swapper/007:48:250
38939990,9cyclictest0-21swapper/008:15:120
38969987,1cyclictest0-21swapper/307:44:203
38959987,1cyclictest0-21swapper/208:36:272
38959983,5cyclictest0-21swapper/208:45:282
38959983,5cyclictest0-21swapper/207:50:102
38959983,5cyclictest0-21swapper/207:42:172
38959981,6cyclictest0-21swapper/209:00:202
38959981,6cyclictest0-21swapper/208:20:162
38959980,8cyclictest0-21swapper/207:10:292
38949986,1cyclictest0-21swapper/108:25:171
38949981,6cyclictest0-21swapper/109:05:191
38949981,6cyclictest0-21swapper/108:35:201
38949981,3cyclictest0-21swapper/108:05:171
38949980,5cyclictest0-21swapper/107:50:251
38949980,5cyclictest0-21swapper/107:30:281
38939980,6cyclictest0-21swapper/008:00:260
38939980,6cyclictest0-21swapper/007:26:140
38969975,1cyclictest11336-21sshd07:23:163
38969971,5cyclictest0-21swapper/307:35:223
38959976,1cyclictest30-21ksoftirqd/208:12:212
38959972,5cyclictest0-21swapper/207:55:422
38949972,3cyclictest0-21swapper/107:20:201
38949971,5cyclictest0-21swapper/107:45:251
38949971,5cyclictest0-21swapper/107:43:161
38949970,6cyclictest0-21swapper/108:50:261
38949970,6cyclictest0-21swapper/108:40:121
38949970,6cyclictest0-21swapper/108:15:141
38949970,6cyclictest0-21swapper/107:25:201
38949960,6cyclictest0-21swapper/108:00:221
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional