You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-04-01 - 19:53

x86 Intel Xeon E3-1220L V2 @2300 MHz, Linux 4.9.47-rt37 (Profile)

Latency plot of system in rack #a, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackaslot0.osadl.org (updated Sat Apr 01, 2023 12:44:25)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2470321000,1sleep337-21rcuc/309:36:353
283652890,0sleep00-21swapper/010:21:260
141562760,0sleep30-21swapper/312:13:303
281772630,1sleep128175-21sshd11:38:551
236822630,0sleep00-21swapper/010:54:190
317672570,0sleep031718-21sshd12:20:290
128142570,1sleep21579-21sshd11:32:042
185932560,0sleep30-21swapper/311:40:453
166002550,1sleep20-21swapper/210:59:142
52182540,0sleep228-21rcuc/212:04:322
319342540,3sleep010-21rcuc/011:00:290
217422540,0sleep1201ktimersoftd/109:47:541
169202530,0sleep31495-21nfsd09:13:273
97922520,1sleep29768-21sshd11:12:282
295352510,1sleep32292299cyclictest09:56:463
233342510,1sleep123337-21bash11:32:591
2248925125,8sleep00-21swapper/007:07:530
2233725124,9sleep20-21swapper/207:05:582
190022510,0sleep20-21swapper/210:01:212
91872500,1sleep337-21rcuc/310:03:193
79122500,1sleep37885-21sshd09:28:303
300952500,0sleep00-21swapper/011:14:150
229802500,0sleep10-21swapper/109:10:451
2257825025,8sleep30-21swapper/307:08:593
97542490,0sleep00-21swapper/012:29:470
2254424925,20sleep10-21swapper/107:08:341
210552490,0sleep00-21swapper/012:03:060
67142480,1sleep00-21swapper/011:28:490
244222480,0sleep30-21swapper/310:21:053
211782460,0sleep121177-21bash09:39:331
292812450,0sleep329282-21ipmitool07:20:253
259862450,0sleep10-21swapper/111:55:181
62752440,0sleep00-21swapper/010:11:170
297572440,0sleep20-21swapper/211:03:072
157032400,0sleep30-21swapper/309:22:533
22914992221,1cyclictest6122-21sshd09:25:091
22914992221,1cyclictest18211-21sshd10:28:551
2291799200,19cyclictest4328-21sshd10:16:382
2290999200,16cyclictest0-21swapper/009:45:240
22922991917,1cyclictest0-21swapper/310:28:543
193442190,5sleep121-21ksoftirqd/111:27:061
22922991816,1cyclictest0-21swapper/309:48:093
2290999189,8cyclictest0-21swapper/009:58:110
22909991817,1cyclictest20462-21sshd11:24:280
116502180,0sleep228-21rcuc/212:13:172
113672180,1sleep30-21swapper/311:04:203
42772170,1sleep14278-21sshd12:04:271
267532170,1sleep313913-21diskmemload11:55:213
22909991716,1cyclictest19844-21sshd11:43:420
2290999170,16cyclictest0-21swapper/011:55:200
157932170,0sleep20-21swapper/209:47:232
34052160,0sleep00-21swapper/012:18:090
308642160,0sleep230851-21sshd09:56:532
296062160,1sleep329604-21bash10:54:503
290572160,1sleep329044-21sshd11:33:283
2290999162,3cyclictest0-21swapper/010:35:060
22909991612,1cyclictest3408-21cp10:00:060
2290999160,5cyclictest0-21swapper/010:47:020
2290999160,10cyclictest0-21swapper/010:26:090
152672160,0sleep00-21swapper/012:05:230
3572150,0sleep00-21swapper/009:11:440
24592150,3sleep337-21rcuc/310:44:143
2292299158,6cyclictest3445-21cp09:34:313
22917991511,1cyclictest0-21swapper/211:05:382
22914991514,1cyclictest28952-21sshd10:46:251
2291499150,9cyclictest0-21swapper/109:34:311
2291499150,14cyclictest0-21swapper/110:20:431
2290999159,5cyclictest0-21swapper/009:19:480
2290999159,3cyclictest0-21swapper/012:32:510
2290999159,3cyclictest0-21swapper/011:32:330
2290999152,10cyclictest17550-21df_abs09:50:180
22909991510,3cyclictest30246-21sshd10:32:390
2290999150,3cyclictest0-21swapper/009:39:220
2290999150,12cyclictest25664-21sshd11:38:420
2290999150,12cyclictest0-21swapper/010:40:500
2290999150,12cyclictest0-21swapper/009:20:200
2290999150,10cyclictest10296-21sshd09:41:250
213282150,0sleep30-21swapper/312:38:583
68182140,0sleep10-21swapper/111:53:371
238512140,0sleep123836-21sshd11:16:251
2292299149,4cyclictest0-21swapper/310:06:513
2291799140,13cyclictest17351-21sshd11:21:222
2291799140,13cyclictest0-21swapper/209:19:022
2291799140,13cyclictest0-21swapper/207:50:182
2291799140,11cyclictest114750irq/25-eth009:50:572
2291499140,13cyclictest0-21swapper/112:30:581
2290999149,3cyclictest19005-21sshd09:26:200
2290999149,3cyclictest0-21swapper/009:33:230
2290999149,2cyclictest0-21swapper/011:15:020
2290999148,3cyclictest0-21swapper/010:57:110
2290999142,1cyclictest0-21swapper/012:36:320
22909991410,1cyclictest0-21swapper/011:50:250
2290999140,1cyclictest5436-21sshd10:16:440
2290999140,13cyclictest25373-21sshd11:08:180
2290999140,11cyclictest15372-21sshd11:46:020
2290999140,0cyclictest0-21swapper/012:12:250
227542140,1sleep30-21swapper/311:27:233
131522140,0sleep213121-21sshd11:54:112
262682130,0sleep20-21swapper/209:14:222
2292299138,4cyclictest0-21swapper/310:57:583
2292299138,4cyclictest0-21swapper/310:38:193
2292299133,9cyclictest3036-21sshd11:11:523
22922991311,1cyclictest27054-21bash10:49:063
2292299130,1cyclictest13913-21diskmemload11:54:423
2292299130,12cyclictest32151-21rm12:26:033
2292299130,12cyclictest29882-21sshd11:08:403
2292299130,12cyclictest29114-21sshd12:01:003
2292299130,12cyclictest0-21swapper/312:24:523
2292299130,12cyclictest0-21swapper/312:06:293
2292299130,12cyclictest0-21swapper/311:48:093
2292299130,12cyclictest0-21swapper/311:36:373
2292299130,12cyclictest0-21swapper/311:21:053
2292299130,12cyclictest0-21swapper/310:14:313
2292299130,12cyclictest0-21swapper/309:42:593
2291799138,2cyclictest0-21swapper/212:26:322
2291799138,2cyclictest0-21swapper/210:26:352
2291799131,1cyclictest24386-21sshd10:40:262
2291799130,3cyclictest0-21swapper/212:31:232
2291799130,12cyclictest32590-21sshd10:38:252
2291799130,12cyclictest18381-21sshd10:12:202
2291799130,12cyclictest10250irq/24-ahci[00011:15:022
2291799130,12cyclictest0-21swapper/212:39:192
2291799130,12cyclictest0-21swapper/211:57:572
2291799130,12cyclictest0-21swapper/211:46:492
2291799130,12cyclictest0-21swapper/211:35:302
2291799130,12cyclictest0-21swapper/209:40:352
2291799130,10cyclictest0-21swapper/211:44:382
2291799130,10cyclictest0-21swapper/210:47:072
2291499138,4cyclictest0-21swapper/111:11:541
2291499133,1cyclictest32474-21sshd10:44:001
2291499133,1cyclictest10253-21cp11:04:151
2291499132,10cyclictest13782-21sshd10:56:101
22914991312,1cyclictest18688-21sshd12:11:061
22914991312,1cyclictest0-21swapper/109:54:041
22914991311,2cyclictest0-21swapper/108:00:201
2291499130,4cyclictest11345-21sshd09:15:591
2291499130,12cyclictest22916-21bash10:51:251
2291499130,12cyclictest19852-21bash12:36:021
2291499130,12cyclictest0-21swapper/112:08:551
2291499130,12cyclictest0-21swapper/111:49:141
2291499130,12cyclictest0-21swapper/111:05:041
2291499130,12cyclictest0-21swapper/110:30:081
2291499130,12cyclictest0-21swapper/110:17:531
2291499130,10cyclictest4869-21sshd09:40:561
2291499130,10cyclictest21838-21sshd10:12:391
2291499130,10cyclictest0-21swapper/112:15:441
2291499130,10cyclictest0-21swapper/111:22:281
2290999139,3cyclictest0-21swapper/010:07:240
324962120,1sleep20-21swapper/209:34:092
2292299129,2cyclictest0-21swapper/310:18:063
2292299128,3cyclictest921-21bash12:15:103
2292299127,4cyclictest21376-21id11:16:103
2292299122,1cyclictest7595-21rm09:18:523
22922991211,1cyclictest26962-21sshd09:51:033
2291799129,2cyclictest0-21swapper/212:15:392
2291799129,2cyclictest0-21swapper/211:25:362
2291799129,2cyclictest0-21swapper/210:05:232
2291799128,1cyclictest13728-21sshd10:50:352
2291799128,1cyclictest0-21swapper/212:20:312
2291799128,1cyclictest0-21swapper/210:23:472
2291799121,1cyclictest0-21swapper/212:05:522
22917991210,2cyclictest0-21swapper/209:20:202
2291799120,12cyclictest0-21swapper/209:36:112
2291799120,11cyclictest0-21swapper/209:25:202
2291799120,11cyclictest0-21swapper/208:18:462
2291499128,1cyclictest0-21swapper/110:05:291
22914991211,1cyclictest4483-21sshd10:38:481
2291499121,10cyclictest0-21swapper/110:00:141
2291499120,2cyclictest0-21swapper/112:20:361
2291499120,1cyclictest15471-21sshd09:22:521
2291499120,12cyclictest0-21swapper/109:55:361
2292299110,1cyclictest0-21swapper/308:10:573
2292299110,11cyclictest0-21swapper/308:36:303
2292299110,10cyclictest31148-21sshd10:30:013
2292299110,10cyclictest0-21swapper/312:30:453
2291799111,9cyclictest114750irq/25-eth010:30:052
22917991110,1cyclictest0-21swapper/207:40:262
2291799110,1cyclictest0-21swapper/207:39:092
2291799110,11cyclictest0-21swapper/208:57:412
2291799110,11cyclictest0-21swapper/208:20:242
2291799110,10cyclictest0-21swapper/208:41:392
2291499119,1cyclictest0-21swapper/112:25:071
2291499110,10cyclictest0-21swapper/111:40:041
2290999119,1cyclictest0-21swapper/007:15:020
2290999116,4cyclictest0-21swapper/008:05:250
2290999110,7cyclictest0-21swapper/008:25:290
2290999110,10cyclictest0-21swapper/007:45:200
2290999110,10cyclictest0-21swapper/007:30:270
2292299109,1cyclictest0-21swapper/307:57:003
2292299100,1cyclictest0-21swapper/308:24:383
2292299100,10cyclictest0-21swapper/309:03:553
2292299100,10cyclictest0-21swapper/308:15:133
2292299100,10cyclictest0-21swapper/307:47:503
2291799108,1cyclictest0-21swapper/207:15:022
2291799100,1cyclictest0-21swapper/208:25:092
2291799100,1cyclictest0-21swapper/207:45:542
2291799100,10cyclictest0-21swapper/209:04:322
2291799100,10cyclictest0-21swapper/208:32:212
2291799100,10cyclictest0-21swapper/208:13:322
2291499109,1cyclictest0-21swapper/109:06:531
2291499109,1cyclictest0-21swapper/108:51:061
2291499109,1cyclictest0-21swapper/107:46:571
2291499100,1cyclictest0-21swapper/109:02:221
2291499100,1cyclictest0-21swapper/108:42:161
2291499100,1cyclictest0-21swapper/108:37:511
2291499100,10cyclictest0-21swapper/108:55:381
2291499100,10cyclictest0-21swapper/108:21:561
2291499100,10cyclictest0-21swapper/107:57:471
2291499100,10cyclictest0-21swapper/107:41:381
2291499100,10cyclictest0-21swapper/107:35:011
2290999109,1cyclictest0-21swapper/009:05:010
2290999100,8cyclictest0-21swapper/007:50:160
2290999100,7cyclictest29850-21switchtime07:20:290
2290999100,7cyclictest0-21swapper/008:00:200
2290999100,7cyclictest0-21swapper/007:40:180
2290999100,1cyclictest0-21swapper/009:02:480
2290999100,1cyclictest0-21swapper/008:15:460
229229998,1cyclictest0-21swapper/308:59:233
229229998,1cyclictest0-21swapper/308:47:533
229229998,1cyclictest0-21swapper/308:41:093
229229998,1cyclictest0-21swapper/307:50:023
229229998,1cyclictest0-21swapper/307:42:083
229229997,1cyclictest18152-21perl08:05:243
229179998,1cyclictest0-21swapper/209:06:512
229179998,1cyclictest0-21swapper/208:46:112
229179998,1cyclictest0-21swapper/208:02:152
229149998,1cyclictest0-21swapper/108:13:141
229149998,1cyclictest0-21swapper/107:10:371
229149990,1cyclictest0-21swapper/108:46:341
229149990,0cyclictest0-21swapper/107:54:031
229099998,1cyclictest0-21swapper/007:35:470
229099996,2cyclictest0-21swapper/008:45:170
229099990,9cyclictest0-21swapper/008:10:530
229099990,6cyclictest0-21swapper/008:36:100
229099990,5cyclictest5879-21ntp_states08:50:330
229099990,1cyclictest23227-21fschecks_count08:20:210
229229987,1cyclictest0-21swapper/309:06:143
229229987,1cyclictest0-21swapper/308:01:533
229229987,1cyclictest0-21swapper/307:39:003
229179987,1cyclictest0-21swapper/208:07:082
229149987,1cyclictest0-21swapper/108:26:351
229149987,1cyclictest0-21swapper/108:18:131
229149987,1cyclictest0-21swapper/107:30:201
229149981,6cyclictest0-21swapper/107:20:221
229149980,1cyclictest0-21swapper/108:06:341
229099984,1cyclictest0-21swapper/007:25:280
229099983,2cyclictest0-21swapper/008:55:300
229099980,6cyclictest0-21swapper/008:40:000
229099980,6cyclictest0-21swapper/007:10:210
229099980,1cyclictest27505-21apache_accesses08:30:180
229099980,1cyclictest13478-21if_eth107:55:230
229229977,0cyclictest0-21swapper/308:50:123
229229976,1cyclictest0-21swapper/307:17:573
229229970,7cyclictest0-21swapper/307:13:543
229229970,6cyclictest0-21swapper/308:25:213
229179976,1cyclictest0-21swapper/208:50:502
229179971,3cyclictest0-21swapper/207:55:232
229179970,6cyclictest0-21swapper/207:30:192
229179970,6cyclictest0-21swapper/207:20:282
229179970,6cyclictest0-21swapper/207:10:192
229229960,1cyclictest0-21swapper/308:30:023
229229960,1cyclictest0-21swapper/307:30:023
229229960,1cyclictest0-21swapper/307:25:023
229179962,2cyclictest0-21swapper/207:25:252
229179960,5cyclictest0-21swapper/208:35:202
229149960,5cyclictest0-21swapper/108:33:251
229149960,5cyclictest0-21swapper/107:15:011
229149960,1cyclictest0-21swapper/107:26:341
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional