You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-05-21 - 04:48

x86 Intel Xeon E3-1220L V2 @2300 MHz, Linux 4.9.47-rt37 (Profile)

Latency plot of system in rack #a, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the alll highest latencies:
System rackaslot0.osadl.org (updated Fri May 20, 2022 00:44:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1756821090,0sleep017554-21sshd23:10:210
125342880,0sleep212535-21sshd22:51:142
275952630,1sleep00-21swapper/021:22:090
111682590,1sleep30-21swapper/323:34:483
234192580,1sleep123414-21sshd23:45:201
254952570,13sleep339-21ksoftirqd/321:25:093
192802560,1sleep31621-21sshd21:36:593
106732560,1sleep21776-21runrttasks00:23:432
207552540,0sleep00-21swapper/021:15:160
453725341,8sleep10-21swapper/119:09:151
157352530,0sleep10-21swapper/123:13:241
443925129,18sleep20-21swapper/219:07:592
355125121,14sleep30-21swapper/319:05:193
225872510,0sleep00-21swapper/000:04:380
77982500,0sleep30-21swapper/323:21:583
219982500,0sleep321999-21bash21:21:373
237902490,1sleep129951-21diskmemload22:36:411
228122490,0sleep322811-21sshd22:17:573
141032490,0sleep00-21swapper/022:01:270
129052490,0sleep00-21swapper/000:17:030
454124825,19sleep00-21swapper/019:09:170
37122480,1sleep03705-21sshd23:46:350
42492470,0sleep20-21swapper/222:03:422
42422470,0sleep20-21swapper/223:09:112
171492470,0sleep10-21swapper/100:38:081
73072450,0sleep00-21swapper/023:43:510
12422450,2sleep00-21swapper/021:57:070
54902420,0sleep10-21swapper/121:26:151
283702420,0sleep10-21swapper/121:22:151
144952420,0sleep10-21swapper/100:03:461
258182410,0sleep30-21swapper/323:48:473
199212380,0sleep319923-21id00:31:253
485699230,22cyclictest14941-21sshd22:51:271
4855992316,6cyclictest13354-21sshd23:16:130
279322230,0sleep10-21swapper/122:55:481
279322230,0sleep10-21swapper/122:55:481
204222230,0sleep120421-21bash00:21:091
485599228,1cyclictest0-21swapper/000:25:550
4855992119,1cyclictest23347-21sshd21:53:010
4858991917,1cyclictest0-21swapper/222:55:552
4858991917,1cyclictest0-21swapper/222:55:552
4856991917,1cyclictest4752-21sshd23:24:541
485599181,12cyclictest20421-21bash23:20:080
149052180,1sleep214903-21sshd00:00:202
485899170,16cyclictest598-21sshd00:19:182
485899169,1cyclictest0-21swapper/223:56:412
4856991615,1cyclictest21-21ksoftirqd/121:59:341
485599169,4cyclictest0-21swapper/023:57:130
485599168,6cyclictest20716-21sshd00:31:290
485599160,15cyclictest24613-21sshd22:08:470
485599160,13cyclictest19497-21id00:38:220
45192160,0sleep30-21swapper/321:10:323
284432160,1sleep10-21swapper/121:40:581
134442160,1sleep010-21rcuc/022:54:310
485899155,1cyclictest0-21swapper/220:48:092
4858991513,2cyclictest118850irq/25-eth022:46:222
485899150,15cyclictest0-21swapper/223:30:252
485899150,14cyclictest0-21swapper/221:31:342
485899150,12cyclictest0-21swapper/220:31:232
485699158,2cyclictest0-21swapper/123:04:281
485599159,3cyclictest0-21swapper/023:06:090
485599158,5cyclictest0-21swapper/022:47:330
485599156,6cyclictest18532-21sshd23:35:250
485599150,14cyclictest0-21swapper/023:31:530
285302150,0sleep10-21swapper/122:49:431
195262150,0sleep00-21swapper/021:40:120
178502150,1sleep217846-21bash00:31:112
153272150,0sleep1201ktimersoftd/123:32:021
485999140,13cyclictest0-21swapper/323:11:263
485999140,13cyclictest0-21swapper/300:07:563
485899149,4cyclictest0-21swapper/221:55:392
4858991412,2cyclictest126750irq/26-eth1-rx-00:28:492
4858991412,1cyclictest15561-21id21:24:122
4858991412,0cyclictest126750irq/26-eth1-rx-22:35:352
485899141,12cyclictest22360-21sshd00:07:582
485899140,9cyclictest0-21swapper/223:29:532
485699149,2cyclictest9876-21sshd00:13:261
485699148,2cyclictest4603-21sshd21:16:501
4856991411,2cyclictest12019-21sshd22:16:521
485699141,12cyclictest0-21swapper/121:12:001
485699140,11cyclictest10278-21sshd23:37:511
485599149,4cyclictest0-21swapper/022:11:010
485599149,3cyclictest0-21swapper/022:40:090
485599148,5cyclictest0-21swapper/000:06:190
485599142,1cyclictest10596-21sshd22:26:040
485599141,9cyclictest0-21swapper/020:45:160
4855991412,1cyclictest23901-21rm21:28:040
4855991412,1cyclictest0-21swapper/022:18:540
4855991412,1cyclictest0-21swapper/000:11:100
485599141,12cyclictest0-21swapper/022:36:490
4855991410,1cyclictest0-21swapper/022:32:170
485599140,13cyclictest25065-21sshd21:12:360
485599140,13cyclictest0-21swapper/022:55:120
485599140,13cyclictest0-21swapper/022:55:120
485599140,11cyclictest0-21swapper/000:21:130
485999139,1cyclictest0-21swapper/322:36:373
485999138,4cyclictest0-21swapper/321:33:073
485999138,4cyclictest0-21swapper/300:11:043
485999138,2cyclictest0-21swapper/323:26:223
485999133,9cyclictest5328-21sshd23:40:253
485999131,9cyclictest118850irq/25-eth021:10:013
485999131,2cyclictest118850irq/25-eth021:54:203
4859991312,1cyclictest39-21ksoftirqd/322:51:273
4859991311,1cyclictest0-21swapper/322:55:213
4859991311,1cyclictest0-21swapper/322:55:213
4859991310,2cyclictest0-21swapper/323:15:533
485999130,4cyclictest634-21sshd22:12:413
485999130,1cyclictest0-21swapper/321:56:153
485999130,12cyclictest13771-21sshd23:06:563
485999130,12cyclictest0-21swapper/323:56:133
485999130,12cyclictest0-21swapper/322:27:313
485999130,12cyclictest0-21swapper/322:03:323
485999130,12cyclictest0-21swapper/300:15:203
485899139,3cyclictest0-21swapper/222:44:212
485899138,4cyclictest28653-21sshd23:39:402
485899138,4cyclictest21431-21bash22:30:152
485899137,5cyclictest0-21swapper/221:42:372
485899132,10cyclictest25826-21bash23:51:522
485899131,1cyclictest118850irq/25-eth021:28:332
4858991311,2cyclictest118850irq/25-eth023:03:502
485899130,6cyclictest0-21swapper/219:30:252
485899130,6cyclictest0-21swapper/219:28:312
485899130,2cyclictest8863-21sshd21:45:212
485899130,1cyclictest29951-21diskmemload00:35:172
485899130,12cyclictest2474-21id22:15:552
485899130,12cyclictest10150irq/24-ahci[00022:29:122
485899130,12cyclictest0-21swapper/223:48:252
485899130,12cyclictest0-21swapper/223:23:402
485899130,12cyclictest0-21swapper/221:18:272
485899130,12cyclictest0-21swapper/200:11:442
485699138,4cyclictest0-21swapper/123:56:081
485699138,2cyclictest20830-21sshd22:11:281
485699138,2cyclictest0-21swapper/121:52:571
485699138,1cyclictest16174-21bash23:07:111
485699131,9cyclictest1945-21sshd23:40:121
4856991310,2cyclictest10477-21sshd22:41:391
485699130,9cyclictest0-21swapper/121:45:271
485699130,2cyclictest6085-21sshd22:28:501
485699130,2cyclictest0-21swapper/123:15:221
485699130,1cyclictest118850irq/25-eth022:22:511
485699130,1cyclictest10150irq/24-ahci[00023:51:031
485699130,12cyclictest8991-21sshd00:30:201
485699130,12cyclictest7784-21sshd00:26:411
485699130,12cyclictest30412-21sshd22:34:171
485699130,12cyclictest2863-21kworker/1:221:35:371
485699130,12cyclictest21664-21kworker/1:023:28:211
485699130,12cyclictest1157-21fw_forwarded_lo22:00:151
485699130,12cyclictest0-21swapper/100:15:581
485699130,10cyclictest15792-21sshd00:07:101
485599139,3cyclictest0-21swapper/021:46:300
485599131,11cyclictest0-21swapper/023:02:350
485599130,3cyclictest14463-21sshd23:28:550
485599130,12cyclictest8382-21sshd21:35:530
485599130,12cyclictest0-21swapper/022:22:040
485599130,11cyclictest0-21swapper/021:31:220
23462130,0sleep30-21swapper/321:41:353
10092130,1sleep30-21swapper/300:36:253
485999129,2cyclictest0-21swapper/322:40:123
485999128,1cyclictest6473-21sshd00:02:583
485999128,1cyclictest0-21swapper/322:46:343
485999121,10cyclictest0-21swapper/322:30:083
485999120,9cyclictest13935-21sshd23:50:413
485999120,9cyclictest0-21swapper/323:00:043
485999120,1cyclictest0-21swapper/300:25:403
485999120,11cyclictest0-21swapper/323:36:103
485999120,11cyclictest0-21swapper/321:45:163
485999120,11cyclictest0-21swapper/321:16:043
485999120,11cyclictest0-21swapper/300:22:063
485899129,2cyclictest0-21swapper/223:41:172
485899129,2cyclictest0-21swapper/223:15:572
485899129,2cyclictest0-21swapper/222:07:072
485899129,2cyclictest0-21swapper/221:12:042
485899128,1cyclictest3901-21sshd22:22:202
485899128,1cyclictest18457-21sshd22:14:272
485899128,1cyclictest0-21swapper/221:38:232
485899127,1cyclictest0-21swapper/223:11:262
485899120,11cyclictest0-21swapper/221:51:282
485899120,11cyclictest0-21swapper/219:24:112
485699129,2cyclictest0-21swapper/121:30:151
485699121,10cyclictest0-21swapper/122:05:151
485599129,2cyclictest0-21swapper/023:50:250
485999119,1cyclictest0-21swapper/322:05:073
485999111,9cyclictest0-21swapper/322:20:343
485899118,2cyclictest0-21swapper/219:15:002
485899113,8cyclictest0-21swapper/221:06:252
4856991110,1cyclictest0-21swapper/119:20:201
485699110,10cyclictest14320-21tune2fs20:35:151
4855991110,1cyclictest0-21swapper/020:06:530
485599110,11cyclictest0-21swapper/020:36:240
485899109,1cyclictest0-21swapper/220:52:262
485899109,1cyclictest0-21swapper/220:40:422
485899108,1cyclictest25321-21fschecks_time19:50:152
485899104,6cyclictest0-21swapper/219:40:532
485899102,3cyclictest0-21swapper/220:38:132
485899100,9cyclictest0-21swapper/220:20:142
485899100,9cyclictest0-21swapper/220:00:452
485899100,4cyclictest0-21swapper/219:10:062
485899100,3cyclictest0-21swapper/219:56:122
485699100,9cyclictest4814-21perl20:15:141
485599109,1cyclictest0-21swapper/019:50:440
485599109,1cyclictest0-21swapper/019:25:150
485599108,1cyclictest0-21swapper/019:15:010
485599100,9cyclictest9557-21fschecks_time20:25:140
485599100,9cyclictest0-21swapper/019:40:140
485599100,9cyclictest0-21swapper/019:10:200
485599100,8cyclictest0-21swapper/020:55:130
485599100,7cyclictest0-21swapper/021:05:130
485599100,1cyclictest11073-21perl19:20:160
485599100,1cyclictest0-21swapper/019:46:410
485599100,10cyclictest0-21swapper/020:30:070
485599100,10cyclictest0-21swapper/019:33:480
48599990,9cyclictest0-21swapper/319:20:153
48599990,1cyclictest0-21swapper/319:26:153
48599990,0cyclictest0-21swapper/320:20:293
48589994,5cyclictest0-21swapper/221:00:552
48589994,1cyclictest0-21swapper/220:25:442
48589992,7cyclictest0-21swapper/220:15:102
48589991,5cyclictest0-21swapper/220:11:242
48589990,8cyclictest0-21swapper/219:45:052
48589990,4cyclictest0-21swapper/220:09:082
48569998,1cyclictest20598-21conntrack19:40:141
48559998,1cyclictest0-21swapper/020:43:090
48559998,1cyclictest0-21swapper/020:19:190
48559993,5cyclictest0-21swapper/020:50:200
48559990,7cyclictest18294-21cat19:35:160
48599988,0cyclictest0-21swapper/320:55:213
48599988,0cyclictest0-21swapper/320:35:143
48599987,1cyclictest0-21swapper/319:33:493
48599983,5cyclictest0-21swapper/320:30:163
48599981,6cyclictest0-21swapper/320:25:133
48599981,6cyclictest0-21swapper/319:55:203
48599980,8cyclictest0-21swapper/320:17:493
48599980,7cyclictest0-21swapper/319:46:543
48589982,2cyclictest0-21swapper/220:55:162
48569987,1cyclictest0-21swapper/119:37:581
48569980,7cyclictest0-21swapper/119:58:431
48559987,1cyclictest0-21swapper/019:55:020
48559980,5cyclictest0-21swapper/020:20:180
48599975,1cyclictest18644-21cat20:45:143
48599973,2cyclictest0-21swapper/321:00:173
48599972,5cyclictest0-21swapper/319:40:163
48599970,6cyclictest0-21swapper/320:10:153
48599970,6cyclictest0-21swapper/320:00:133
48599970,6cyclictest0-21swapper/319:50:143
48599970,6cyclictest0-21swapper/319:35:113
48589970,6cyclictest0-21swapper/219:35:102
48569977,0cyclictest0-21swapper/120:45:141
48569976,1cyclictest0-21swapper/120:25:151
48569975,1cyclictest31134-21taskset20:01:041
48569975,1cyclictest11982sleep120:07:491
48569975,1cyclictest11861-21grep20:30:151
48569971,6cyclictest0-21swapper/121:05:171
48569971,5cyclictest0-21swapper/120:20:231
48569971,5cyclictest0-21swapper/119:50:141
48569971,5cyclictest0-21swapper/119:15:011
48569971,1cyclictest0-21swapper/119:25:231
48569970,6cyclictest0-21swapper/120:50:281
48569970,6cyclictest0-21swapper/119:45:141
48559970,6cyclictest0-21swapper/021:00:120
48559970,6cyclictest0-21swapper/020:10:140
48559970,6cyclictest0-21swapper/020:00:150
48599962,4cyclictest0-21swapper/320:50:183
48599960,5cyclictest0-21swapper/320:40:133
48599960,5cyclictest0-21swapper/320:05:133
48599960,5cyclictest0-21swapper/319:14:593
48599960,5cyclictest0-21swapper/319:10:103
48569964,1cyclictest5726-21if_err_eth119:10:161
48569962,4cyclictest0-21swapper/120:40:161
48569962,2cyclictest0-21swapper/121:00:161
48569960,6cyclictest0-21swapper/120:55:181
48569960,5cyclictest0-21swapper/119:30:111
48569950,2cyclictest1754-21atenall_rapower20:10:111
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional