You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-01-29 - 16:19

x86 Intel Xeon E3-1220L V2 @2300 MHz, Linux 4.9.47-rt37 (Profile)

Latency plot of system in rack #a, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rackaslot0.osadl.org (updated Sun Jan 29, 2023 12:44:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2539721110,0sleep325396-21sshd11:32:223
2647221000,1sleep226462-21sshd11:18:232
204942670,1sleep10-21swapper/111:37:331
158352650,1sleep315806-21sshd10:04:173
97852610,0sleep29751-21sshd12:10:232
279392610,1sleep327915-21sshd11:29:493
6742580,1sleep2675-21sshd10:56:272
129562570,0sleep00-21swapper/009:15:570
183002550,0sleep118295-21bash10:10:061
1535625524,19sleep10-21swapper/107:05:131
251542540,1sleep30-21swapper/309:59:283
244082540,2sleep337-21rcuc/311:18:113
55172530,0sleep30-21swapper/309:46:213
1731725326,9sleep20-21swapper/207:09:042
1727225332,18sleep00-21swapper/007:08:330
24522520,0sleep00-21swapper/010:14:220
181262520,0sleep30-21swapper/310:15:373
64572510,0sleep26452-21sshd10:23:102
200432510,0sleep30-21swapper/309:50:253
17862510,1sleep21771-21sshd10:17:052
1722525129,18sleep30-21swapper/307:08:013
147132510,0sleep20-21swapper/211:56:412
147132510,0sleep20-21swapper/211:56:412
14092510,1sleep110460-21diskmemload09:29:081
136162510,1sleep013612-21bash12:19:140
135042510,0sleep30-21swapper/309:10:193
72972500,1sleep37283-21sshd11:39:153
51632500,0sleep10-21swapper/110:51:131
302502500,0sleep00-21swapper/010:22:230
301712500,1sleep030170-21sshd11:24:220
175882500,0sleep10-21swapper/111:34:321
123752500,0sleep20-21swapper/209:18:482
321802490,1sleep032173-21sshd12:01:030
146312490,1sleep014605-21sshd10:40:470
81892480,0sleep10-21swapper/111:19:381
311552480,0sleep310460-21diskmemload09:28:543
279492480,0sleep027946-21sshd10:27:520
69242460,1sleep06920-21sshd11:02:380
223132460,1sleep30-21swapper/310:24:353
222642460,0sleep10-21swapper/112:17:071
231422450,0sleep21787-21runrttasks10:27:272
222662430,1sleep21765199cyclictest12:20:002
213262430,0sleep20-21swapper/209:39:202
147362430,2sleep21765199cyclictest10:43:412
74362410,0sleep00-21swapper/011:36:240
210992270,0sleep321100-21bash12:08:353
210992270,0sleep321100-21bash12:08:353
189182230,9sleep10-21swapper/112:39:271
276412220,1sleep11546-21nfsd09:11:371
17652992220,1cyclictest0-21swapper/312:24:183
74802210,2sleep11765099cyclictest10:40:161
49472200,0sleep30-21swapper/312:29:473
309702200,1sleep10-21swapper/107:35:221
309702200,1sleep10-21swapper/107:35:221
268202200,1sleep126819-21bash11:21:101
17651992018,1cyclictest14855-21sshd09:58:312
1765199200,19cyclictest0-21swapper/212:34:342
1764999204,12cyclictest4457-21kworker/0:109:30:310
213162190,0sleep210460-21diskmemload10:49:542
1765199195,6cyclictest0-21swapper/208:47:172
1765199193,6cyclictest0-21swapper/207:37:122
1765199193,6cyclictest0-21swapper/207:37:122
51462180,0sleep15149-21sshd12:24:081
46492180,1sleep00-21swapper/012:35:200
17652991817,1cyclictest0-21swapper/311:12:283
1765199185,6cyclictest0-21swapper/208:09:452
1765199185,5cyclictest0-21swapper/209:00:452
1765199183,6cyclictest0-21swapper/209:05:372
1765199182,6cyclictest0-21swapper/208:50:132
17651991815,2cyclictest1546-21nfsd11:35:542
17650991816,1cyclictest17437-21sshd11:42:541
203582170,1sleep010460-21diskmemload11:40:200
1765199174,8cyclictest0-21swapper/208:13:242
1765199174,6cyclictest0-21swapper/211:06:492
1765199171,6cyclictest0-21swapper/207:29:442
17651991715,2cyclictest26964-21sshd12:09:072
17651991715,2cyclictest26964-21sshd12:09:072
17651991715,2cyclictest2566-21bash11:02:152
17650991715,1cyclictest30618-21sshd11:10:181
1764999178,6cyclictest17794-21sshd10:18:310
1764999170,16cyclictest0-21swapper/009:55:080
113482170,5sleep03-21ksoftirqd/011:56:230
113482170,5sleep03-21ksoftirqd/011:56:230
314442160,0sleep00-21swapper/010:30:570
18672160,1sleep01862-21sshd10:56:340
1765199165,6cyclictest0-21swapper/208:44:052
1765199165,6cyclictest0-21swapper/207:43:202
1765199161,6cyclictest0-21swapper/207:59:412
17651991614,1cyclictest0-21swapper/211:27:182
1765199160,6cyclictest0-21swapper/211:20:132
1765199160,15cyclictest7682-21bash12:21:282
1765199160,15cyclictest0-21swapper/209:33:332
1765199160,11cyclictest0-21swapper/208:59:172
1765099160,15cyclictest0-21swapper/109:23:351
1764999162,3cyclictest0-21swapper/011:13:100
17649991614,1cyclictest16048-21sshd09:35:570
141992160,1sleep00-21swapper/010:46:240
115382160,0sleep10-21swapper/111:56:241
115382160,0sleep10-21swapper/111:56:231
111212160,0sleep30-21swapper/309:38:243
109862160,1sleep30-21swapper/311:47:563
102352160,0sleep310234-21bash10:37:373
40402150,0sleep30-21swapper/309:15:163
229302150,1sleep022929-21bash10:02:010
1765299155,9cyclictest23830-21sshd12:31:203
1765199155,6cyclictest0-21swapper/207:52:332
1765199155,6cyclictest0-21swapper/207:52:332
1765199152,6cyclictest0-21swapper/207:46:292
1765199152,6cyclictest0-21swapper/207:46:282
17651991513,1cyclictest0-21swapper/210:54:412
17651991513,1cyclictest0-21swapper/209:22:392
1765199150,6cyclictest0-21swapper/208:39:572
1764999159,5cyclictest0-21swapper/011:07:440
1764999158,3cyclictest30903-21sshd09:40:090
1764999158,3cyclictest30903-21sshd09:40:090
1764999152,2cyclictest25313-21sshd09:11:240
1764999152,2cyclictest0-21swapper/011:15:240
1764999150,12cyclictest0-21swapper/012:21:080
106782150,1sleep10-21swapper/109:58:091
89452140,0sleep10-21swapper/111:00:021
1765299149,4cyclictest0-21swapper/310:45:503
1765299149,2cyclictest0-21swapper/310:13:003
1765299149,2cyclictest0-21swapper/309:20:263
1765299141,1cyclictest127950irq/26-eth1-rx-09:34:363
1765299141,1cyclictest120050irq/25-eth010:29:053
1765299140,13cyclictest0-21swapper/311:51:483
1765299140,13cyclictest0-21swapper/310:33:513
1765199144,9cyclictest26854-21sshd09:45:232
1765199143,6cyclictest0-21swapper/208:33:412
1765199143,5cyclictest0-21swapper/207:18:332
1765199141,6cyclictest0-21swapper/208:00:492
1765199140,6cyclictest0-21swapper/211:13:212
1765199140,6cyclictest0-21swapper/207:32:442
1765199140,1cyclictest0-21swapper/211:42:092
1765199140,13cyclictest10041-21sshd09:29:542
1765199140,13cyclictest0-21swapper/209:11:062
17650991410,1cyclictest127950irq/26-eth1-rx-11:47:231
1765099140,13cyclictest0-21swapper/109:38:551
1764999149,4cyclictest0-21swapper/009:51:160
1764999149,2cyclictest0-21swapper/009:48:010
1764999147,4cyclictest0-21swapper/011:49:240
1764999142,1cyclictest0-21swapper/012:10:140
1764999140,3cyclictest5394-21sshd10:37:120
1764999140,13cyclictest1830-21bash12:26:360
1764999140,13cyclictest0-21swapper/012:30:510
1764999140,13cyclictest0-21swapper/011:52:080
1764999140,11cyclictest26445-21sshd10:05:110
1764999140,11cyclictest19258-21sshd09:27:510
1764999140,11cyclictest0-21swapper/011:30:040
1764999140,11cyclictest0-21swapper/011:25:030
126022140,0sleep10-21swapper/110:32:111
1765299139,1cyclictest0-21swapper/311:08:463
1765299139,1cyclictest0-21swapper/311:00:123
1765299137,5cyclictest0-21swapper/310:08:083
1765299131,9cyclictest1632-21sshd11:55:173
1765299131,9cyclictest1632-21sshd11:55:173
1765299130,4cyclictest14101-21sshd12:16:233
1765299130,12cyclictest6484-21sshd12:12:563
1765299130,12cyclictest5554-21sshd10:40:053
1765299130,12cyclictest29778-21sshd11:24:193
1765299130,12cyclictest26906-21id10:58:473
1765299130,12cyclictest11309-21sshd09:44:053
1765299130,12cyclictest11309-21sshd09:44:053
1765299130,10cyclictest32333-21sshd10:50:453
1765199134,9cyclictest0-21swapper/208:28:572
1765199133,1cyclictest15619-21sshd10:32:262
1765199131,5cyclictest0-21swapper/207:22:572
1765199131,11cyclictest0-21swapper/210:35:232
17651991310,1cyclictest120050irq/25-eth012:27:262
1765199130,6cyclictest0-21swapper/208:20:242
1765199130,4cyclictest20691-21sshd12:36:422
1765199130,4cyclictest15555-21sshd09:50:072
1765199130,1cyclictest0-21swapper/210:12:562
1765099139,3cyclictest0-21swapper/109:15:061
1765099139,1cyclictest12766-21sshd10:01:061
1765099139,1cyclictest0-21swapper/112:07:081
1765099139,1cyclictest0-21swapper/112:07:081
1765099139,1cyclictest0-21swapper/111:05:471
1765099138,4cyclictest20397-21sshd10:21:321
1765099138,4cyclictest0-21swapper/112:01:061
1765099137,2cyclictest0-21swapper/109:30:531
1765099133,9cyclictest21854-21cat10:55:301
1765099131,1cyclictest120050irq/25-eth009:54:281
17650991310,2cyclictest24234-21id10:07:471
1765099130,12cyclictest7245-21bash12:32:441
1765099130,12cyclictest32243-21sshd12:12:211
1765099130,12cyclictest0-21swapper/112:26:521
1765099130,12cyclictest0-21swapper/110:38:491
1765099130,12cyclictest0-21swapper/109:45:231
1765099130,10cyclictest7956-21sshd11:25:151
1764999131,1cyclictest10150irq/24-ahci[00009:20:330
1764999130,2cyclictest12207-21sshd12:05:020
1764999130,2cyclictest12207-21sshd12:05:020
1764999130,2cyclictest0-21swapper/010:51:130
1764999130,1cyclictest0-21swapper/007:15:010
118882130,0sleep10-21swapper/111:50:471
1765299129,2cyclictest0-21swapper/312:35:093
1765299129,2cyclictest0-21swapper/312:00:083
1765299129,2cyclictest0-21swapper/311:40:193
1765199129,2cyclictest0-21swapper/211:32:392
1765199129,2cyclictest0-21swapper/209:40:012
1765199129,2cyclictest0-21swapper/209:40:012
1765199122,9cyclictest21204-21sshd10:07:302
1765199121,1cyclictest120050irq/25-eth011:45:252
1765199121,10cyclictest0-21swapper/211:50:032
17651991210,2cyclictest0-21swapper/210:00:442
1765199120,12cyclictest0-21swapper/212:00:072
1765099121,10cyclictest0-21swapper/108:35:181
17650991210,1cyclictest2302-21interrupts10:45:221
1765099120,3cyclictest0-21swapper/109:40:241
1765099120,3cyclictest0-21swapper/109:40:241
1765099120,11cyclictest0-21swapper/110:25:201
1765099120,11cyclictest0-21swapper/110:15:171
1764999120,11cyclictest0-21swapper/008:49:080
1765299110,5cyclictest0-21swapper/307:51:583
1765299110,5cyclictest0-21swapper/307:51:583
1765299110,10cyclictest0-21swapper/308:47:083
1765199115,6cyclictest0-21swapper/207:12:532
1765199110,6cyclictest0-21swapper/208:15:322
1765099110,1cyclictest0-21swapper/108:21:091
1765099110,1cyclictest0-21swapper/107:49:051
1765099110,1cyclictest0-21swapper/107:49:051
1765099110,11cyclictest0-21swapper/109:05:181
1765099110,11cyclictest0-21swapper/108:08:361
1765099110,11cyclictest0-21swapper/107:59:541
1765099110,11cyclictest0-21swapper/107:50:431
1765099110,11cyclictest0-21swapper/107:50:421
1764999110,5cyclictest0-21swapper/008:26:120
1764999110,11cyclictest0-21swapper/009:04:420
1764999110,11cyclictest0-21swapper/008:34:230
1764999110,11cyclictest0-21swapper/008:17:530
1764999110,11cyclictest0-21swapper/007:22:060
1765299109,1cyclictest0-21swapper/308:55:223
1765299109,1cyclictest0-21swapper/308:20:253
1765299108,1cyclictest0-21swapper/308:40:193
1765299100,1cyclictest0-21swapper/308:25:243
1765099109,1cyclictest0-21swapper/108:49:441
1765099109,1cyclictest0-21swapper/108:33:201
1765099109,1cyclictest0-21swapper/108:17:401
1765099100,1cyclictest0-21swapper/108:04:461
1765099100,10cyclictest0-21swapper/109:01:491
1765099100,10cyclictest0-21swapper/108:54:021
1765099100,10cyclictest0-21swapper/108:11:241
1765099100,10cyclictest0-21swapper/107:21:551
1764999108,1cyclictest0-21swapper/007:45:160
1764999108,1cyclictest0-21swapper/007:45:160
1764999106,3cyclictest0-21swapper/008:59:290
1764999100,9cyclictest0-21swapper/008:00:190
1764999100,8cyclictest0-21swapper/008:10:280
1764999100,7cyclictest18142-21grep07:10:200
1764999100,7cyclictest0-21swapper/009:05:280
1764999100,7cyclictest0-21swapper/007:55:230
1764999100,10cyclictest0-21swapper/008:21:220
1764999100,10cyclictest0-21swapper/008:08:490
1764999100,10cyclictest0-21swapper/007:32:350
176529998,1cyclictest0-21swapper/308:10:463
176529998,1cyclictest0-21swapper/307:45:203
176529998,1cyclictest0-21swapper/307:45:203
176529998,1cyclictest0-21swapper/307:36:303
176529998,1cyclictest0-21swapper/307:36:293
176529998,1cyclictest0-21swapper/307:15:013
176529997,1cyclictest0-21swapper/308:15:193
176529990,9cyclictest0-21swapper/308:31:413
176529990,8cyclictest0-21swapper/308:00:573
176509998,1cyclictest0-21swapper/108:29:031
176509998,1cyclictest0-21swapper/107:13:461
176509990,9cyclictest0-21swapper/107:18:191
176509990,5cyclictest0-21swapper/107:44:421
176499998,1cyclictest0-21swapper/008:37:030
176499998,1cyclictest0-21swapper/007:43:160
176499997,1cyclictest26268-21perl07:25:220
176499990,7cyclictest0-21swapper/007:36:450
176499990,7cyclictest0-21swapper/007:36:450
176499990,6cyclictest0-21swapper/008:50:320
176529982,4cyclictest0-21swapper/309:00:233
176529981,6cyclictest0-21swapper/307:10:223
176509988,0cyclictest0-21swapper/108:58:191
176509988,0cyclictest0-21swapper/107:29:001
176509987,1cyclictest0-21swapper/107:32:221
176499981,6cyclictest0-21swapper/008:42:230
176499980,7cyclictest0-21swapper/007:54:360
176499980,7cyclictest0-21swapper/007:54:360
176529975,1cyclictest7475-21munin-node07:55:193
176529975,1cyclictest28058-21cpuspeed_turbos07:30:193
176529975,1cyclictest12482-21expr08:05:163
176529972,4cyclictest0-21swapper/308:35:183
176529971,5cyclictest0-21swapper/307:40:203
176509973,4cyclictest0-21swapper/108:41:421
176529966,0cyclictest0-21swapper/307:25:223
176529960,6cyclictest0-21swapper/308:50:233
176529960,5cyclictest0-21swapper/309:05:183
176529960,5cyclictest0-21swapper/307:20:133
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional