You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-07-03 - 07:14

x86 Intel Xeon E3-1220L V2 @2300 MHz, Linux 4.9.47-rt37 (Profile)

Latency plot of system in rack #a, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of all highest latencies:
System rackaslot0.osadl.org (updated Sun Jul 03, 2022 00:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5952990,0sleep2590-21sshd00:32:102
308062950,0sleep030786-21sshd22:43:400
282992910,1sleep028294-21sshd22:57:490
324302900,0sleep337-21rcuc/321:28:563
71152850,0sleep20-21swapper/223:27:332
286502850,0sleep228614-21sshd22:57:512
243242640,1sleep20-21swapper/221:36:472
1997926325,9sleep30-21swapper/319:07:363
275652590,1sleep227562-21bash00:25:332
1982625925,18sleep20-21swapper/219:05:472
190122590,0sleep20-21swapper/221:30:302
238742580,0sleep20-21swapper/222:43:012
117782580,0sleep20-21swapper/223:07:532
282562570,0sleep028216-21sshd21:16:580
311912560,0sleep00-21swapper/023:32:310
155662560,1sleep00-21swapper/000:12:250
264402550,0sleep326436-21sshd00:04:243
67242540,1sleep16719-21cp23:30:191
188102540,1sleep213138-21diskmemload22:13:492
170982530,0sleep30-21swapper/321:15:573
170772530,1sleep20-21swapper/200:18:362
108342530,0sleep310838-21id21:55:433
179462520,0sleep10-21swapper/122:05:061
100932520,1sleep10-21swapper/123:07:441
232642500,0sleep30-21swapper/322:54:323
171092500,0sleep117089-21sshd22:22:171
236292490,1sleep30-21swapper/322:11:173
137772490,1sleep30-21swapper/322:24:553
9412480,0sleep30-21swapper/300:23:053
324922480,0sleep30-21swapper/300:26:023
222862480,0sleep00-21swapper/021:28:000
2010724825,19sleep00-21swapper/019:09:150
5122470,0sleep10-21swapper/123:04:031
233142470,0sleep10-21swapper/121:16:311
1998224725,18sleep10-21swapper/119:07:391
41232460,0sleep03-21ksoftirqd/023:07:110
152702460,0sleep10-21swapper/100:39:421
148292460,0sleep20-21swapper/222:33:362
148292460,0sleep20-21swapper/222:33:362
126052460,2sleep22041699cyclictest00:03:012
88652440,1sleep38849-21sshd21:38:213
289202430,0sleep328918-21sshd21:40:103
132312400,0sleep30-21swapper/323:33:503
115312390,1sleep111529-21bash22:30:221
115312390,1sleep111529-21bash22:30:221
30402380,1sleep337-21rcuc/322:38:173
326662350,1sleep02040799cyclictest22:38:030
205182310,0sleep10-21swapper/122:28:221
181622200,1sleep018158-21bash00:27:470
170492200,1sleep10-21swapper/123:57:111
87372190,5sleep230-21ksoftirqd/223:13:232
277982190,3sleep337-21rcuc/300:16:323
276262190,0sleep10-21swapper/123:46:381
126422190,1sleep337-21rcuc/322:47:473
124582190,0sleep30-21swapper/322:19:033
20419991817,1cyclictest0-21swapper/322:31:273
20419991817,1cyclictest0-21swapper/322:31:273
20411991816,1cyclictest20989-21sshd23:28:471
214082170,2sleep32041999cyclictest23:51:503
20419991715,1cyclictest19132-21sshd00:30:503
20416991716,1cyclictest0-21swapper/222:17:432
20416991715,1cyclictest0-21swapper/200:20:392
2040799170,13cyclictest6271-21grep00:05:240
167632170,1sleep016762-21bash22:07:510
100202170,1sleep210018-21bash21:41:162
8092160,0sleep1802-21rm00:01:521
60512160,2sleep12041199cyclictest21:46:401
316262160,0sleep2291ktimersoftd/221:14:292
220852160,0sleep00-21swapper/022:54:250
2041199165,5cyclictest0-21swapper/123:19:581
2040799168,6cyclictest0-21swapper/000:34:560
2040799160,13cyclictest0-21swapper/023:10:180
139962160,1sleep30-21swapper/323:45:243
20419991514,0cyclictest118850irq/25-eth023:58:533
20419991510,2cyclictest4707-21cp23:10:103
2041699151,1cyclictest0-21swapper/223:50:242
2041199159,5cyclictest0-21swapper/122:13:041
2041199154,8cyclictest0-21swapper/119:49:321
2041199153,6cyclictest0-21swapper/122:41:101
2041199153,4cyclictest0-21swapper/119:52:321
2041199150,4cyclictest0-21swapper/121:37:081
2041199150,14cyclictest0-21swapper/121:27:531
2040799159,3cyclictest0-21swapper/023:48:110
2040799159,3cyclictest0-21swapper/022:15:360
2040799158,6cyclictest17289-21sshd00:21:340
2040799157,7cyclictest0-21swapper/022:11:360
2040799151,10cyclictest13579-21sshd22:21:590
2040799150,5cyclictest0-21swapper/023:57:460
2040799150,4cyclictest23963-21sshd23:26:060
2040799150,4cyclictest0-21swapper/021:43:130
2040799150,1cyclictest20794-21unixbench_singl22:25:250
2040799150,14cyclictest0-21swapper/021:54:360
12982150,0sleep31285-21sshd00:11:053
91622140,0sleep20-21swapper/200:05:422
88822140,0sleep00-21swapper/021:55:320
27182140,0sleep20-21swapper/223:55:522
2041999148,5cyclictest0-21swapper/323:41:223
2041999148,5cyclictest0-21swapper/323:04:493
20419991412,1cyclictest0-21swapper/300:36:093
2041699149,4cyclictest0-21swapper/221:59:162
2041699142,3cyclictest0-21swapper/222:54:032
2041699140,2cyclictest0-21swapper/223:49:422
2041699140,1cyclictest0-21swapper/200:35:542
2041699140,13cyclictest933-21bash22:26:352
2041699140,13cyclictest20213-21sshd22:48:292
2041699140,12cyclictest0-21swapper/221:54:072
2041199149,1cyclictest118850irq/25-eth000:15:171
2041199149,1cyclictest10487-21sshd00:33:081
2041199148,5cyclictest21765-21perf20:20:001
2041199148,2cyclictest1720-21bash00:11:061
2041199148,1cyclictest0-21swapper/121:24:131
2041199145,3cyclictest0-21swapper/120:05:001
2041199144,6cyclictest0-21swapper/122:04:001
20411991412,1cyclictest27839-21bash22:49:111
2041199141,1cyclictest0-21swapper/122:18:081
2041199141,12cyclictest0-21swapper/123:12:001
2041199141,12cyclictest0-21swapper/121:13:111
2041199140,6cyclictest0-21swapper/119:13:491
2041199140,11cyclictest0-21swapper/123:44:321
2041199140,10cyclictest1244-21sshd00:26:091
2040799149,4cyclictest0-21swapper/022:04:150
2040799149,4cyclictest0-21swapper/021:14:480
2040799148,4cyclictest0-21swapper/023:38:550
2040799148,4cyclictest0-21swapper/021:31:440
2040799148,3cyclictest23302-21sshd00:04:040
2040799148,3cyclictest1621-21sshd23:41:460
2040799140,14cyclictest0-21swapper/020:02:180
194782140,1sleep319438-21sshd22:25:193
2041999133,1cyclictest14655-21sshd23:39:463
20419991311,1cyclictest23841-21sshd23:08:593
20419991311,1cyclictest18656-21bash00:06:383
20419991311,1cyclictest13138-21diskmemload23:20:583
2041999130,2cyclictest28499-21bash22:05:573
2041999130,1cyclictest1535-21nfsd22:40:163
2041999130,12cyclictest0-21swapper/323:25:063
2041999130,12cyclictest0-21swapper/323:15:323
2041999130,12cyclictest0-21swapper/321:10:333
2041999130,10cyclictest0-21swapper/321:46:363
2041699138,4cyclictest0-21swapper/221:22:522
2041699131,9cyclictest118850irq/25-eth021:45:002
2041699131,11cyclictest0-21swapper/222:08:522
2041699131,11cyclictest0-21swapper/221:17:442
20416991310,2cyclictest22181-21sshd23:20:172
2041699130,1cyclictest0-21swapper/223:37:232
2041699130,1cyclictest0-21swapper/223:31:072
2041699130,12cyclictest12868-21sshd23:16:352
2041699130,12cyclictest0-21swapper/223:44:162
2041699130,12cyclictest0-21swapper/221:27:272
2041699130,10cyclictest0-21swapper/200:12:022
2041199138,4cyclictest0-21swapper/123:35:141
2041199138,2cyclictest17542-21sshd23:51:281
2041199138,1cyclictest0-21swapper/121:55:391
2041199138,1cyclictest0-21swapper/121:30:331
2041199135,3cyclictest0-21swapper/122:51:191
2041199131,11cyclictest0-21swapper/121:50:441
2041199131,11cyclictest0-21swapper/120:40:141
20411991310,2cyclictest0-21swapper/100:21:301
2041199130,6cyclictest0-21swapper/119:18:171
2041199130,4cyclictest0-21swapper/122:55:331
2041199130,12cyclictest9801-21sshd21:41:151
2041199130,12cyclictest0-21swapper/123:21:421
2040799139,3cyclictest0-21swapper/023:01:310
2040799138,4cyclictest28653-21sshd21:45:460
2040799138,4cyclictest24170-21sshd23:17:350
2040799138,2cyclictest0-21swapper/021:20:190
2040799130,3cyclictest0-21swapper/023:50:390
2040799130,2cyclictest0-21swapper/022:45:020
2040799130,1cyclictest21854-21cp00:19:030
2040799130,1cyclictest0-21swapper/019:57:540
2040799130,12cyclictest6149-21sshd00:35:410
2040799130,12cyclictest0-21swapper/023:22:240
2040799130,12cyclictest0-21swapper/021:36:130
2040799130,11cyclictest0-21swapper/022:30:590
2040799130,11cyclictest0-21swapper/022:30:590
2041999129,2cyclictest0-21swapper/322:00:013
2041999129,2cyclictest0-21swapper/321:50:113
2041999129,2cyclictest0-21swapper/320:20:123
2041999128,3cyclictest29541-21sshd22:55:053
20419991210,1cyclictest725-40kipmi021:30:183
2041999120,1cyclictest8561-21sshd21:20:583
2041999120,11cyclictest0-21swapper/320:35:063
2041699129,2cyclictest0-21swapper/222:24:512
2041699121,2cyclictest0-21swapper/220:30:152
2041699120,3cyclictest0-21swapper/223:01:042
2041699120,11cyclictest0-21swapper/222:35:562
2041699120,11cyclictest0-21swapper/222:00:002
2041699120,11cyclictest0-21swapper/220:45:262
2041199125,4cyclictest0-21swapper/120:14:361
2041199121,10cyclictest0-21swapper/122:35:031
2041199121,10cyclictest0-21swapper/100:05:191
2041199120,6cyclictest0-21swapper/119:21:211
2041199120,4cyclictest0-21swapper/120:55:251
2041199120,4cyclictest0-21swapper/120:55:251
2041999118,2cyclictest0-21swapper/319:15:013
2041999110,2cyclictest0-21swapper/320:40:143
2041999110,11cyclictest0-21swapper/320:50:203
2041999110,10cyclictest9355-21perl21:00:143
2041999110,10cyclictest0-21swapper/320:45:163
2041999110,10cyclictest0-21swapper/319:30:523
20416991110,1cyclictest0-21swapper/220:52:172
2041699110,1cyclictest0-21swapper/219:53:382
2041699110,11cyclictest0-21swapper/220:36:222
2041699110,11cyclictest0-21swapper/219:48:172
2041699110,11cyclictest0-21swapper/219:39:422
2041699110,11cyclictest0-21swapper/219:34:072
2041699110,10cyclictest0-21swapper/220:40:232
2041699110,10cyclictest0-21swapper/220:20:112
2041699110,10cyclictest0-21swapper/219:40:162
2041699110,10cyclictest0-21swapper/219:19:352
2041199118,2cyclictest0-21swapper/121:00:131
2041199113,4cyclictest0-21swapper/120:47:361
2041199113,4cyclictest0-21swapper/119:36:561
2041199111,5cyclictest0-21swapper/121:05:281
2041199110,8cyclictest0-21swapper/119:59:201
2040799118,2cyclictest0-21swapper/020:15:230
2040799114,4cyclictest0-21swapper/020:35:120
2040799114,4cyclictest0-21swapper/020:20:180
2040799110,7cyclictest0-21swapper/020:30:130
2040799110,1cyclictest0-21swapper/019:14:590
2040799110,11cyclictest0-21swapper/021:09:170
2040799110,10cyclictest0-21swapper/020:45:130
2040799110,10cyclictest0-21swapper/020:40:120
2041999107,2cyclictest0-21swapper/320:05:153
2041999100,1cyclictest19607-21apache_volume20:15:123
2041999100,1cyclictest0-21swapper/320:00:373
2041999100,1cyclictest0-21swapper/319:55:143
2041999100,10cyclictest0-21swapper/320:25:083
2041999100,10cyclictest0-21swapper/320:10:023
2041999100,10cyclictest0-21swapper/319:39:073
2041699109,1cyclictest0-21swapper/220:15:112
2041699109,1cyclictest0-21swapper/219:20:382
2041699108,1cyclictest0-21swapper/221:05:162
2041699100,1cyclictest9312-21perl21:00:142
2041699100,10cyclictest0-21swapper/220:02:412
2041699100,10cyclictest0-21swapper/219:27:092
2041199109,1cyclictest0-21swapper/119:25:141
2041199108,1cyclictest21986-21apache_volume20:20:111
2041199100,9cyclictest3830-21apache_volume20:50:131
2041199100,1cyclictest27471-21perl20:30:151
2041199100,10cyclictest0-21swapper/120:06:341
2041199100,10cyclictest0-21swapper/119:40:201
2040799109,1cyclictest0-21swapper/019:32:380
2040799108,2cyclictest0-21swapper/019:40:160
2040799103,4cyclictest0-21swapper/019:50:100
2040799100,9cyclictest0-21swapper/020:05:520
2040799100,9cyclictest0-21swapper/019:35:130
2040799100,6cyclictest0-21swapper/019:25:110
2040799100,1cyclictest3828-21apache_processe20:50:120
2040799100,10cyclictest0-21swapper/021:00:110
2040799100,10cyclictest0-21swapper/020:26:040
2040799100,10cyclictest0-21swapper/019:23:280
2040799100,10cyclictest0-21swapper/019:17:250
204199998,1cyclictest0-21swapper/321:06:493
204199998,1cyclictest0-21swapper/320:55:143
204199998,1cyclictest0-21swapper/320:55:143
204199998,1cyclictest0-21swapper/319:50:583
204199998,1cyclictest0-21swapper/319:25:133
204199998,1cyclictest0-21swapper/319:22:023
204199990,9cyclictest0-21swapper/320:30:123
204199990,9cyclictest0-21swapper/319:12:523
204199990,1cyclictest0-21swapper/319:45:093
204169998,1cyclictest0-21swapper/219:57:212
204119992,4cyclictest0-21swapper/119:30:131
204119990,9cyclictest0-21swapper/120:28:281
204119990,1cyclictest0-21swapper/120:35:131
204079998,1cyclictest0-21swapper/019:45:110
204079990,7cyclictest0-21swapper/020:10:120
204169988,0cyclictest0-21swapper/220:05:122
204169987,1cyclictest0-21swapper/220:56:352
204169987,1cyclictest0-21swapper/220:56:352
204169986,1cyclictest21200-21perl19:10:142
204079980,6cyclictest0-21swapper/020:56:280
204079980,6cyclictest0-21swapper/020:56:280
204199971,5cyclictest0-21swapper/319:40:153
204169976,1cyclictest0-21swapper/220:26:022
204169970,1cyclictest0-21swapper/220:10:082
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional