You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-12-05 - 03:16

x86 Intel Xeon E3-1220L V2 @2300 MHz, Linux 4.9.47-rt37 (Profile)

Latency plot of system in rack #a, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackaslot0.osadl.org (updated Mon Dec 05, 2022 00:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2871321090,0sleep30-21swapper/322:10:293
1467221010,0sleep30-21swapper/322:17:503
242962930,0sleep31787-21runrttasks21:33:283
310432910,0sleep10-21swapper/121:45:201
3812670,0sleep30-21swapper/321:14:333
275702620,0sleep027572-21bash23:04:200
143782620,0sleep20-21swapper/223:17:152
43992600,0sleep00-21swapper/023:41:490
105842600,1sleep010561-21sshd00:11:450
302692580,0sleep30-21swapper/300:20:023
288052580,0sleep10-21swapper/123:41:061
243552580,0sleep10-21swapper/100:13:111
68712570,0sleep10-21swapper/123:59:011
49622560,0sleep1201ktimersoftd/122:19:531
1973525627,25sleep00-21swapper/019:06:130
120052540,1sleep312004-21sshd22:43:063
72302530,1sleep07225-21sshd21:37:370
72302530,1sleep07225-21sshd21:37:370
172302530,0sleep00-21swapper/022:52:060
169562530,1sleep20-21swapper/223:42:542
309602520,0sleep030962-21id00:23:180
304252520,1sleep10-21swapper/123:30:001
1992525230,18sleep30-21swapper/319:08:373
1974825227,21sleep10-21swapper/119:06:221
171202520,0sleep20-21swapper/223:48:362
171202520,0sleep20-21swapper/223:48:362
117322520,1sleep00-21swapper/023:25:260
295682510,0sleep20-21swapper/222:55:552
275632510,1sleep11787-21runrttasks20:30:161
215752510,5sleep10-21swapper/121:19:121
206922510,0sleep20-21swapper/223:00:482
140452510,1sleep01787-21runrttasks22:29:090
303052500,0sleep030292-21sshd00:01:030
278912500,1sleep227872-21sshd21:33:472
1972725028,18sleep20-21swapper/219:06:072
290722490,0sleep20-21swapper/200:26:142
282772490,0sleep30-21swapper/300:16:373
185802490,0sleep118553-21sshd22:06:511
171762490,0sleep10-21swapper/123:17:301
167962490,0sleep10-21swapper/122:35:031
84642480,1sleep28455-21sshd22:11:362
255522480,0sleep10-21swapper/121:36:221
255522480,0sleep10-21swapper/121:36:221
240642480,0sleep30-21swapper/323:23:443
27072470,0sleep30-21swapper/300:07:513
201132470,0sleep012985-21diskmemload22:24:000
160802470,1sleep216076-21sshd23:31:302
247612460,0sleep01787-21runrttasks21:30:360
47552440,0sleep30-21swapper/323:19:163
197672440,0sleep00-21swapper/023:48:500
197672440,0sleep00-21swapper/023:48:490
252022270,0sleep225199-21cp22:52:472
105222230,1sleep010488-21sshd21:23:510
258022210,0sleep00-21swapper/022:58:310
2029799211,1cyclictest27644-21sshd22:30:183
90802200,0sleep39068-21sshd22:25:483
261872200,1sleep226188-21sshd21:25:132
20283991917,1cyclictest28268-21sshd21:25:200
111902190,1sleep111189-21bash21:29:331
2029299180,3cyclictest0-21swapper/222:01:572
2028399188,7cyclictest0-21swapper/000:30:080
20283991813,4cyclictest0-21swapper/022:04:400
131432180,0sleep00-21swapper/023:05:480
35562170,0sleep1201ktimersoftd/123:47:231
35562170,0sleep1201ktimersoftd/123:47:231
78172160,0sleep20-21swapper/223:13:542
239942160,0sleep10-21swapper/123:04:001
2028899161,14cyclictest0-21swapper/123:33:231
2028399168,4cyclictest0-21swapper/021:56:050
2028399163,10cyclictest0-21swapper/022:49:370
2028399160,15cyclictest0-21swapper/000:16:590
59992150,1sleep25996-21bash22:48:182
253482150,1sleep225345-21sshd23:23:512
2029799155,9cyclictest39-21ksoftirqd/323:49:243
2029799155,9cyclictest39-21ksoftirqd/323:49:243
2029799155,1cyclictest14185-21sshd21:46:403
2029799152,10cyclictest37-21rcuc/321:53:283
20297991510,4cyclictest1632-21sshd00:13:133
2029799150,14cyclictest0-21swapper/321:36:433
2029799150,14cyclictest0-21swapper/321:36:423
2028899151,13cyclictest0-21swapper/122:24:581
2028899151,13cyclictest0-21swapper/100:30:521
20288991510,1cyclictest120050irq/25-eth000:24:481
2028899150,14cyclictest8603-21sshd22:45:361
2028399153,1cyclictest0-21swapper/023:11:490
2028399150,14cyclictest0-21swapper/023:58:140
2028399150,14cyclictest0-21swapper/023:50:510
85712140,0sleep10-21swapper/100:02:091
2029799144,1cyclictest17023-21bash23:37:173
20297991413,1cyclictest120050irq/25-eth022:05:203
20297991412,1cyclictest39-21ksoftirqd/323:29:303
2029799140,13cyclictest6829-21bash23:53:203
2029799140,13cyclictest0-21swapper/300:35:313
2029799140,0cyclictest120050irq/25-eth022:49:503
20292991412,1cyclictest23360-21bash22:21:242
20292991412,1cyclictest0-21swapper/200:36:242
20292991412,1cyclictest0-21swapper/200:18:552
2029299140,13cyclictest0-21swapper/200:05:172
2029299140,13cyclictest0-21swapper/200:03:592
2029299140,10cyclictest0-21swapper/200:34:392
2028899143,10cyclictest6360-21sshd21:23:291
2028899143,10cyclictest25477-21sshd23:23:531
2028899141,2cyclictest16613-21sshd00:09:141
2028899140,13cyclictest29526-21sshd22:58:501
20283991412,1cyclictest0-21swapper/023:32:150
2028399140,3cyclictest31069-21sshd21:17:100
2028399140,3cyclictest0-21swapper/023:37:440
2028399140,3cyclictest0-21swapper/000:38:260
2028399140,12cyclictest0-21swapper/021:12:070
2028399140,11cyclictest0-21swapper/023:20:560
2028399140,11cyclictest0-21swapper/022:38:590
81342130,0sleep30-21swapper/323:30:453
240842130,1sleep124055-21sshd00:38:301
2029799138,4cyclictest16011-21cp22:23:383
2029799138,4cyclictest0-21swapper/321:20:233
2029799138,4cyclictest0-21swapper/300:25:043
2029799138,2cyclictest0-21swapper/323:12:493
2029799138,2cyclictest0-21swapper/300:30:323
2029799133,9cyclictest39-21ksoftirqd/322:02:233
20297991311,1cyclictest39-21ksoftirqd/323:55:253
2029799130,4cyclictest31868-21sshd22:53:223
2029799130,12cyclictest18245-21sshd21:55:273
2029799130,12cyclictest0-21swapper/323:41:363
2029799130,12cyclictest0-21swapper/323:05:183
2029799130,12cyclictest0-21swapper/323:04:193
2029799130,12cyclictest0-21swapper/300:00:323
2029799130,10cyclictest30981-21sshd22:58:573
2029799130,10cyclictest26490-21sshd21:16:443
2029799130,10cyclictest14445-21kworker/3:122:36:003
2029299138,4cyclictest4694-21bash21:43:032
2029299138,4cyclictest14842-21sshd22:17:512
2029299138,2cyclictest21984-21sshd22:38:192
20292991311,1cyclictest26505-21bash21:36:282
20292991311,1cyclictest26505-21bash21:36:282
20292991311,1cyclictest1632-21sshd21:59:122
2029299131,11cyclictest0-21swapper/222:25:142
2029299130,2cyclictest22877-21cp22:32:432
2029299130,1cyclictest13406-21bash00:21:332
2029299130,12cyclictest25132-21sshd00:10:052
2029299130,12cyclictest17562-21sshd23:06:112
2029299130,12cyclictest1547-21nfsd21:22:292
2029299130,12cyclictest12013-21bash22:40:172
2029299130,12cyclictest0-21swapper/221:45:322
2029299130,12cyclictest0-21swapper/221:17:042
2028899139,1cyclictest0-21swapper/122:41:021
2028899139,1cyclictest0-21swapper/121:41:021
2028899138,5cyclictest0-21swapper/123:05:481
20288991312,1cyclictest0-21swapper/121:50:281
2028899131,1cyclictest120050irq/25-eth023:12:471
2028899131,11cyclictest0-21swapper/121:12:071
20288991311,1cyclictest0-21swapper/100:19:181
2028899130,12cyclictest23636-21sshd21:58:521
2028899130,12cyclictest1632-21sshd22:25:321
2028899130,12cyclictest0-21swapper/122:31:301
2028899130,12cyclictest0-21swapper/122:04:261
2028899130,12cyclictest0-21swapper/100:25:131
2028899130,10cyclictest0-21swapper/123:35:491
2028899130,10cyclictest0-21swapper/122:12:091
2028399139,3cyclictest2329-21sshd22:11:030
2028399139,3cyclictest0-21swapper/022:40:580
2028399139,1cyclictest0-21swapper/021:40:450
2028399138,4cyclictest32264-21sshd22:30:410
2028399138,4cyclictest0-21swapper/022:19:460
20283991312,1cyclictest15887-21sshd21:49:440
2028399130,2cyclictest0-21swapper/021:50:540
2028399130,1cyclictest0-21swapper/023:15:120
2028399130,12cyclictest0-21swapper/000:06:090
146642130,1sleep014665-21sshd22:06:300
2029799129,2cyclictest0-21swapper/321:40:033
2029799120,11cyclictest0-21swapper/321:25:223
2029299129,2cyclictest0-21swapper/223:25:182
2029299129,2cyclictest0-21swapper/221:51:072
2029299122,9cyclictest0-21swapper/221:10:352
2029299121,10cyclictest23012-21sshd23:57:342
2029299121,10cyclictest0-21swapper/223:35:092
20292991210,1cyclictest6091-21sshd22:05:432
2029299120,6cyclictest19476-21apache_processe20:15:122
2029299120,11cyclictest0-21swapper/223:51:192
2028899121,2cyclictest0-21swapper/123:50:191
20288991211,1cyclictest0-21swapper/121:30:081
2028899120,11cyclictest0-21swapper/122:52:081
2028399121,2cyclictest0-21swapper/019:15:010
20283991210,1cyclictest0-21swapper/000:25:020
2028399120,11cyclictest0-21swapper/020:30:200
20297991110,1cyclictest0-21swapper/319:26:503
2029799110,1cyclictest0-21swapper/320:55:083
2029799110,1cyclictest0-21swapper/319:56:073
2029799110,1cyclictest0-21swapper/319:47:373
2029799110,11cyclictest0-21swapper/320:40:573
2029799110,11cyclictest0-21swapper/319:22:463
2029799110,11cyclictest0-21swapper/319:13:073
2029799110,10cyclictest0-21swapper/320:06:553
2029299113,8cyclictest0-21swapper/219:18:592
20292991110,1cyclictest0-21swapper/220:32:292
20292991110,1cyclictest0-21swapper/219:12:562
2029299110,1cyclictest0-21swapper/220:25:182
2029299110,1cyclictest0-21swapper/220:05:532
2029299110,11cyclictest0-21swapper/221:03:232
2029299110,11cyclictest0-21swapper/220:21:482
2029299110,11cyclictest0-21swapper/220:01:422
2029299110,11cyclictest0-21swapper/219:48:292
2028399118,2cyclictest0-21swapper/019:45:190
2029799109,1cyclictest0-21swapper/320:37:213
2029799109,1cyclictest0-21swapper/320:19:353
2029799108,1cyclictest31286-21tune2fs19:30:153
2029799102,8cyclictest0-21swapper/319:44:133
20297991010,0cyclictest0-21swapper/319:54:293
2029799100,9cyclictest0-21swapper/321:05:193
2029799100,10cyclictest0-21swapper/321:00:563
2029799100,10cyclictest0-21swapper/320:52:063
2029799100,10cyclictest0-21swapper/319:35:113
2029299109,1cyclictest0-21swapper/219:39:532
2029299109,1cyclictest0-21swapper/219:35:002
2029299109,1cyclictest0-21swapper/219:29:462
2029299100,1cyclictest26520-21fschecks_time19:20:192
2029299100,1cyclictest0-21swapper/220:58:592
2029299100,1cyclictest0-21swapper/220:50:092
2029299100,1cyclictest0-21swapper/219:51:372
2029299100,10cyclictest0-21swapper/221:06:522
2028899109,1cyclictest0-21swapper/119:55:201
2028899109,1cyclictest0-21swapper/119:22:061
2028899100,9cyclictest0-21swapper/120:05:191
2028899100,1cyclictest29767-21perl20:35:191
2028899100,1cyclictest0-21swapper/119:35:191
2028899100,1cyclictest0-21swapper/119:15:021
2028399109,1cyclictest0-21swapper/020:50:150
2028399105,5cyclictest0-21swapper/020:15:260
2028399105,4cyclictest0-21swapper/020:40:230
2028399105,2cyclictest0-21swapper/020:25:170
2028399102,4cyclictest22485-21perf19:15:000
2028399100,5cyclictest0-21swapper/019:50:180
2028399100,1cyclictest0-21swapper/020:10:150
2028399100,10cyclictest0-21swapper/020:56:550
2028399100,10cyclictest0-21swapper/020:20:550
202979998,1cyclictest0-21swapper/320:28:113
202979998,1cyclictest0-21swapper/320:22:363
202979998,1cyclictest0-21swapper/319:17:323
202979991,7cyclictest0-21swapper/320:00:243
202979990,8cyclictest0-21swapper/320:10:193
202929998,1cyclictest0-21swapper/219:56:572
202929993,2cyclictest127950irq/26-eth1-rx-20:10:122
202889998,1cyclictest0-21swapper/121:08:451
202889998,1cyclictest0-21swapper/119:43:361
202889991,7cyclictest0-21swapper/120:55:221
202889990,9cyclictest0-21swapper/119:32:201
202889990,9cyclictest0-21swapper/119:25:171
202839998,1cyclictest0-21swapper/020:00:130
202839994,2cyclictest0-21swapper/020:05:230
202839993,3cyclictest4259-21gzip19:40:250
202839991,7cyclictest0-21swapper/019:25:140
202839990,2cyclictest0-21swapper/019:20:170
202929986,1cyclictest0-21swapper/219:40:202
202929981,6cyclictest0-21swapper/220:40:142
202929980,7cyclictest0-21swapper/220:37:522
202889981,6cyclictest0-21swapper/120:50:121
202889981,6cyclictest0-21swapper/120:40:131
202889981,6cyclictest0-21swapper/120:20:241
202889981,6cyclictest0-21swapper/119:50:161
202889980,7cyclictest0-21swapper/119:14:191
202889980,5cyclictest0-21swapper/120:11:281
202889980,1cyclictest0-21swapper/120:48:111
202839983,3cyclictest0-21swapper/019:38:250
202839982,1cyclictest29971-21kworker/0:220:45:130
202839981,6cyclictest0-21swapper/021:05:190
202839981,4cyclictest0-21swapper/019:30:180
202839980,6cyclictest0-21swapper/020:38:180
202839980,6cyclictest0-21swapper/019:56:590
202839980,5cyclictest0-21swapper/021:00:140
202979976,1cyclictest0-21swapper/320:45:013
202979975,1cyclictest27355-21fschecks_time20:30:143
202889975,1cyclictest5231-21taskset19:45:141
202889975,1cyclictest24719-21fschecks_count20:25:161
202889971,5cyclictest0-21swapper/121:00:151
202889971,5cyclictest0-21swapper/120:00:131
202889970,6cyclictest0-21swapper/120:15:211
202929964,1cyclictest1850-21fschecks_count20:45:152
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional