You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-06-04 - 23:04

x86 Intel Xeon E3-1220L V2 @2300 MHz, Linux 4.9.47-rt37 (Profile)

Latency plot of system in rack #a, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rackaslot0.osadl.org (updated Sun Jun 04, 2023 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
124602840,1sleep012448-21sshd11:47:010
172152630,1sleep21770-21runrttasks09:26:072
3149826226,18sleep20-21swapper/207:09:152
284042590,1sleep20-21swapper/212:20:282
174302580,1sleep024517-21diskmemload10:24:140
206502560,1sleep320617-21sshd09:23:153
21842550,1sleep32180-21cp09:41:393
21842550,1sleep32180-21cp09:41:383
143402550,0sleep314343-21bash12:32:483
103492550,1sleep210346-21sshd09:18:502
97442540,0sleep10-21swapper/109:56:301
48772540,0sleep041ktimersoftd/010:34:560
3147625428,9sleep00-21swapper/007:08:570
222142540,1sleep228-21rcuc/211:53:132
217052530,0sleep00-21swapper/011:29:030
185712530,1sleep20-21swapper/212:35:462
3140525125,9sleep10-21swapper/107:08:051
55852500,1sleep30-21swapper/310:31:583
3151325025,8sleep30-21swapper/307:09:253
296122500,0sleep20-21swapper/209:13:572
225702500,1sleep00-21swapper/011:18:220
142312500,1sleep314230-21sshd11:15:003
117812500,0sleep00-21swapper/010:06:550
44362490,1sleep34425-21sshd10:16:243
161422490,1sleep316119-21sshd11:47:213
34542460,0sleep20-21swapper/209:24:512
3272460,2sleep03180799cyclictest11:03:040
18652460,1sleep01770-21runrttasks07:15:010
178942460,1sleep11770-21runrttasks09:33:001
148212460,1sleep010-21rcuc/011:55:180
142412460,1sleep224517-21diskmemload11:17:402
96762440,0sleep30-21swapper/309:35:293
38282430,0sleep00-21swapper/011:40:560
204852430,0sleep220483-21bash09:57:402
223782420,0sleep20-21swapper/211:42:322
67172380,1sleep26714-21sshd09:31:452
62752320,0sleep20-21swapper/211:30:262
3181399221,20cyclictest32460-21id11:00:212
31813992119,1cyclictest0-21swapper/210:30:092
62042200,1sleep06196-21sshd12:02:360
3181399200,19cyclictest28-21rcuc/212:25:372
31807992010,6cyclictest23661-21sshd09:40:280
31807992010,6cyclictest23661-21sshd09:40:280
31811991918,1cyclictest7741-21sshd12:37:381
31811991917,1cyclictest13974-21sshd09:49:591
3180799199,5cyclictest23038-21tune2fs09:05:210
3180799194,12cyclictest15218-21sshd11:12:210
31807991913,2cyclictest0-21swapper/010:28:230
184382190,0sleep2291ktimersoftd/211:09:582
3181399179,5cyclictest0-21swapper/212:07:442
31813991715,1cyclictest0-21swapper/209:44:322
31813991715,1cyclictest0-21swapper/209:44:322
31813991714,1cyclictest24517-21diskmemload09:45:032
3180799172,14cyclictest27476-21id12:33:550
3180799172,10cyclictest0-21swapper/011:52:080
31807991716,1cyclictest14078-21sshd10:53:270
31807991715,1cyclictest13334-21sshd09:32:320
31807991711,3cyclictest0-21swapper/009:53:360
3180799170,14cyclictest26532-21sshd11:37:300
3180799170,13cyclictest0-21swapper/010:01:280
3180799170,13cyclictest0-21swapper/009:45:290
3180799170,12cyclictest0-21swapper/007:40:230
259992170,0sleep337-21rcuc/309:30:243
212602170,1sleep021259-21sshd11:07:280
323972160,1sleep20-21swapper/210:49:192
3181899160,15cyclictest0-21swapper/311:51:463
3180799166,6cyclictest0-21swapper/009:02:300
3180799164,5cyclictest0-21swapper/012:20:590
3180799163,9cyclictest0-21swapper/008:06:250
3180799161,14cyclictest31752-21bash10:15:490
253862160,0sleep325358-21sshd10:01:303
165512160,1sleep316539-21sshd11:58:073
165232160,0sleep00-21swapper/010:41:530
119682160,0sleep10-21swapper/111:49:441
3181899150,12cyclictest22462-21bash11:18:213
31813991514,1cyclictest30-21ksoftirqd/210:03:372
31813991513,1cyclictest0-21swapper/211:56:072
3181199151,13cyclictest0-21swapper/112:28:061
3180799155,5cyclictest0-21swapper/012:26:110
3180799153,8cyclictest0-21swapper/010:14:320
3180799153,8cyclictest0-21swapper/008:00:490
31807991512,2cyclictest0-21swapper/012:37:260
3180799150,7cyclictest0-21swapper/012:10:310
3180799150,12cyclictest17684-21sshd09:16:040
3180799150,11cyclictest0-21swapper/012:06:300
3180799150,11cyclictest0-21swapper/010:55:040
3180799150,11cyclictest0-21swapper/009:38:570
3180799150,10cyclictest0-21swapper/009:20:010
249062150,1sleep119-21rcuc/112:33:411
157392150,1sleep315740-21sshd11:33:543
3181899149,2cyclictest0-21swapper/311:09:513
3181899149,1cyclictest26294-21sshd09:58:183
3181899148,5cyclictest24745-21sshd10:08:233
31818991412,1cyclictest332-21sshd09:17:443
31818991412,1cyclictest20987-21cp12:17:153
3181899140,14cyclictest0-21swapper/312:00:113
3181399149,2cyclictest0-21swapper/210:37:222
3181399144,9cyclictest1621-21sshd11:22:552
31813991413,1cyclictest30-21ksoftirqd/210:28:142
3181399140,2cyclictest0-21swapper/209:51:252
31811991413,1cyclictest25546-21sshd11:56:061
31811991413,0cyclictest15773-21sshd10:07:231
3181199140,13cyclictest0-21swapper/111:25:441
3180799147,3cyclictest0-21swapper/010:35:390
3180799147,3cyclictest0-21swapper/009:25:370
3180799147,3cyclictest0-21swapper/008:20:200
3180799145,8cyclictest28564-21sshd09:55:080
3180799145,5cyclictest0-21swapper/007:52:130
3180799143,11cyclictest0-21swapper/007:33:170
3180799143,11cyclictest0-21swapper/007:21:050
3180799143,11cyclictest0-21swapper/007:21:050
3180799142,4cyclictest18699-21bash12:17:030
31807991412,1cyclictest14562-21sshd09:12:180
3180799140,7cyclictest0-21swapper/008:10:050
3180799140,13cyclictest0-21swapper/011:21:020
3180799140,12cyclictest0-21swapper/010:47:300
3180799140,10cyclictest0-21swapper/011:30:070
307392140,1sleep330737-21sshd11:00:133
144532140,0sleep114423-21sshd11:44:351
77892130,0sleep20-21swapper/210:20:162
3181899138,4cyclictest26200-21sshd09:51:103
3181899138,2cyclictest0-21swapper/312:12:483
3181899138,2cyclictest0-21swapper/309:46:233
3181899133,9cyclictest39-21ksoftirqd/312:05:233
3181899130,1cyclictest0-21swapper/309:13:263
3181899130,12cyclictest118950irq/25-eth010:20:453
3181899130,12cyclictest0-21swapper/310:25:523
3181399139,1cyclictest28-21rcuc/210:58:282
3181399138,4cyclictest0-21swapper/211:46:192
3181399137,5cyclictest0-21swapper/212:30:132
31813991312,1cyclictest30-21ksoftirqd/211:12:502
3181399131,1cyclictest118950irq/25-eth010:50:132
3181399130,2cyclictest10694-21sshd10:10:202
3181399130,12cyclictest28-21rcuc/211:26:162
3181399130,12cyclictest0-21swapper/212:19:562
3181399130,12cyclictest0-21swapper/211:38:092
3181399130,12cyclictest0-21swapper/210:19:102
3181399130,12cyclictest0-21swapper/210:07:312
3181399130,12cyclictest0-21swapper/207:45:222
3181199139,1cyclictest0-21swapper/110:46:071
3181199139,1cyclictest0-21swapper/110:16:141
3181199139,1cyclictest0-21swapper/109:11:241
3181199138,4cyclictest20057-21sshd12:09:081
3181199138,2cyclictest25738-21sshd09:54:531
3181199138,1cyclictest0-21swapper/110:04:451
3181199133,9cyclictest19114-21sshd12:00:571
3181199133,9cyclictest118950irq/25-eth009:29:211
3181199132,10cyclictest16493-21sshd09:22:511
31811991311,1cyclictest23410-21cp11:31:501
3181199131,11cyclictest0-21swapper/111:11:511
3181199131,11cyclictest0-21swapper/111:06:061
3181199131,10cyclictest16955-21sshd11:50:071
3181199130,2cyclictest29875-21bash10:54:481
3181199130,2cyclictest1630-21sshd10:34:371
3181199130,2cyclictest15075-21sshd10:38:471
3181199130,2cyclictest14428-21sshd11:15:011
3181199130,2cyclictest0-21swapper/110:10:201
3181199130,1cyclictest655-21sshd10:40:271
3181199130,1cyclictest0-21swapper/109:40:381
3181199130,1cyclictest0-21swapper/109:40:381
3181199130,12cyclictest15072-21sshd10:58:551
3181199130,12cyclictest0-21swapper/112:15:461
3181199130,12cyclictest0-21swapper/110:25:261
3181199130,12cyclictest0-21swapper/109:35:011
3181199130,10cyclictest3927-21sshd11:35:341
3181199130,10cyclictest31717-21cp12:20:451
3180799137,5cyclictest6076-21expr08:30:200
3180799137,3cyclictest0-21swapper/008:45:200
3180799136,7cyclictest0-21swapper/008:57:200
3180799136,3cyclictest0-21swapper/008:40:120
3180799131,7cyclictest0-21swapper/008:27:300
3180799130,9cyclictest0-21swapper/008:50:280
3180799130,3cyclictest0-21swapper/007:57:090
3181899128,3cyclictest12841-21sshd11:28:173
3181899128,2cyclictest0-21swapper/311:42:073
3181899127,4cyclictest2021-21sshd12:21:003
31818991210,1cyclictest11955-21sshd11:20:123
3181899120,9cyclictest0-21swapper/312:25:423
3181899120,3cyclictest7798-21sshd10:50:033
3181899120,1cyclictest27062-21sshd10:11:583
3181899120,1cyclictest0-21swapper/311:39:053
3181899120,1cyclictest0-21swapper/309:26:123
3181899120,12cyclictest0-21swapper/312:35:233
3181899120,11cyclictest1536-21nfsd10:55:233
3181899120,11cyclictest0-21swapper/310:46:053
3181899120,11cyclictest0-21swapper/310:35:133
31813991211,1cyclictest30-21ksoftirqd/212:11:502
3181399121,10cyclictest0-21swapper/209:35:092
3181399120,3cyclictest32704-21sshd12:02:072
3181399120,1cyclictest0-21swapper/210:42:172
3181199129,2cyclictest0-21swapper/112:12:521
3181199127,4cyclictest0-21swapper/111:00:091
3181199120,1cyclictest0-21swapper/109:15:301
3181199120,11cyclictest0-21swapper/111:20:201
3181199120,11cyclictest0-21swapper/110:21:031
3180799125,7cyclictest0-21swapper/007:28:050
3180799125,7cyclictest0-21swapper/007:10:570
3180799123,4cyclictest0-21swapper/008:15:410
3180799120,9cyclictest0-21swapper/008:39:140
3180799120,6cyclictest0-21swapper/007:45:150
3181899119,1cyclictest1536-21nfsd10:40:053
3181899110,1cyclictest0-21swapper/308:31:233
3181899110,11cyclictest0-21swapper/308:40:003
3181199119,1cyclictest0-21swapper/108:40:231
31811991110,1cyclictest0-21swapper/108:09:041
3181199110,11cyclictest0-21swapper/109:07:081
3181199110,11cyclictest0-21swapper/109:01:541
3181199110,11cyclictest0-21swapper/108:25:131
3180799110,11cyclictest0-21swapper/007:35:450
3181899108,1cyclictest0-21swapper/307:15:013
3181899100,9cyclictest11250-21perl08:40:233
3181899100,9cyclictest0-21swapper/309:00:223
3181899100,1cyclictest0-21swapper/307:50:183
3181899100,10cyclictest0-21swapper/307:37:023
3181899100,10cyclictest0-21swapper/307:22:593
3181899100,10cyclictest0-21swapper/307:22:593
3181399108,1cyclictest5641-21perl07:20:212
3181399108,1cyclictest5641-21perl07:20:202
3181399100,9cyclictest18352-21tune2fs08:55:212
3181199109,1cyclictest0-21swapper/108:53:191
3181199108,1cyclictest0-21swapper/108:55:201
3181199100,9cyclictest0-21swapper/108:20:221
3181199100,1cyclictest0-21swapper/108:10:231
318189998,1cyclictest0-21swapper/308:24:323
318189998,1cyclictest0-21swapper/307:45:193
318189992,5cyclictest0-21swapper/308:15:213
318189992,5cyclictest0-21swapper/307:40:273
318189990,9cyclictest0-21swapper/308:46:333
318189990,8cyclictest0-21swapper/308:25:333
318139990,9cyclictest0-21swapper/207:13:482
318119997,1cyclictest19894-21perl07:50:181
318189987,1cyclictest24671-21perl08:00:253
318189987,1cyclictest0-21swapper/309:05:543
318189980,0cyclictest0-21swapper/308:55:213
318139987,1cyclictest20745-21perl09:00:222
318139987,1cyclictest0-21swapper/207:25:442
318139982,4cyclictest0-21swapper/208:25:282
318139981,6cyclictest0-21swapper/208:40:202
318139981,6cyclictest0-21swapper/207:50:192
318139981,6cyclictest0-21swapper/207:40:222
318139981,6cyclictest0-21swapper/207:30:262
318139980,7cyclictest0-21swapper/207:36:172
318119987,1cyclictest0-21swapper/108:37:531
318119981,6cyclictest0-21swapper/108:30:191
318119981,6cyclictest0-21swapper/107:25:211
318189976,1cyclictest0-21swapper/308:51:443
318189973,2cyclictest0-21swapper/307:55:233
318189970,6cyclictest0-21swapper/308:10:223
318189970,6cyclictest0-21swapper/307:25:223
318139977,0cyclictest0-21swapper/208:05:192
318139973,3cyclictest30-21ksoftirqd/209:05:212
318139972,5cyclictest0-21swapper/208:20:242
318139971,5cyclictest0-21swapper/208:35:202
318139970,7cyclictest0-21swapper/208:50:182
318139970,7cyclictest0-21swapper/208:30:282
318139970,6cyclictest0-21swapper/208:45:252
318139970,6cyclictest0-21swapper/208:15:202
318139970,6cyclictest0-21swapper/208:10:202
318139970,6cyclictest0-21swapper/208:00:242
318139970,6cyclictest0-21swapper/207:15:332
318119977,0cyclictest0-21swapper/107:55:201
318119974,2cyclictest12898-21grep08:45:221
318119971,5cyclictest0-21swapper/107:45:201
318119970,6cyclictest0-21swapper/107:20:201
318119970,6cyclictest0-21swapper/107:20:201
318119970,6cyclictest0-21swapper/107:15:201
318189965,1cyclictest31852-21munin-node07:10:363
318189964,2cyclictest0-21swapper/307:34:483
318189960,5cyclictest0-21swapper/308:05:153
318139964,1cyclictest30-21ksoftirqd/207:55:182
318119961,5cyclictest0-21swapper/108:15:191
318119961,2cyclictest0-21swapper/108:00:271
318119960,5cyclictest13442-21sensors_temp07:35:291
318119960,5cyclictest0-21swapper/107:40:171
318119960,5cyclictest0-21swapper/107:30:211
318119960,5cyclictest0-21swapper/107:10:241
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional