You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-10-04 - 11:14

x86 Intel Xeon E3-1220L V2 @2300 MHz, Linux 4.19.8-rt6 (Profile)

Latency plot of system in rack #4, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack4slot0.osadl.org (updated Fri Aug 05, 2022 12:44:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1426421320,2sleep0568099cyclictest09:30:420
1077121270,0sleep010770-21sshd10:48:020
980421210,0sleep20-21swapper/211:42:552
651021160,0sleep26511-21sshd12:04:352
2941821110,2sleep2568299cyclictest11:52:242
180342930,1sleep21278-21runrttasks10:30:322
200462860,2sleep1568199cyclictest09:57:451
130462830,0sleep30-21swapper/310:15:223
42612790,0sleep30-21swapper/310:54:383
252902770,1sleep325289-21sshd11:55:343
5680997673,2cyclictest24124-21ipmitool12:10:170
299352760,0sleep10-21swapper/111:45:141
151332750,0sleep30-21swapper/310:26:363
146512750,0sleep00-21swapper/010:52:060
303372730,0sleep10-21swapper/109:51:271
230312720,1sleep1231rcuc/109:28:061
3452710,0sleep0111rcuc/012:22:160
89112700,0sleep10-21swapper/110:58:491
76242690,0sleep20-21swapper/209:56:172
176122690,0sleep10-21swapper/110:01:091
5681996866,1cyclictest24055-21ipmitool10:20:201
48532680,0sleep0111rcuc/011:20:190
233722680,0sleep00-21swapper/009:24:200
135162670,0sleep10-21swapper/112:35:121
131192670,0sleep20-21swapper/209:37:542
198542630,0sleep00-21swapper/011:03:410
5680996260,1cyclictest26139-21ipmitool12:25:160
52352620,1sleep225837-21nfsd11:20:222
506525625,10sleep10-21swapper/107:05:341
165762560,0sleep20-21swapper/209:49:552
549325425,9sleep20-21swapper/207:09:362
5682994745,1cyclictest14736-21ipmitool12:20:152
5683994643,2cyclictest24058-21ipmitool10:20:193
546924625,14sleep30-21swapper/307:09:163
530624327,10sleep00-21swapper/007:07:420
5681993531,2cyclictest14705-21ipmitool12:20:151
131552290,1sleep233-21ksoftirqd/209:53:132
291922280,1sleep329184-21sshd09:32:243
568399279,2cyclictest1205-21grep09:59:203
5681992624,1cyclictest3185-21ipmitool12:15:161
175852260,0sleep00-21swapper/012:16:500
25452250,0sleep20-21swapper/211:01:422
568299242,4cyclictest24331-21bash10:09:192
568099241,5cyclictest30643-21lspci10:02:390
279422240,1sleep1231rcuc/111:11:551
568399230,4cyclictest17832-21bash09:45:393
568299239,1cyclictest29569-21grep11:48:472
568299233,1cyclictest121rcu_preempt08:10:012
5682992312,2cyclictest593-21systemd-journal09:26:302
568099233,2cyclictest0-21swapper/008:14:240
568099233,2cyclictest0-21swapper/007:59:240
17692230,0sleep30-21swapper/311:45:363
130132230,1sleep013011-21sshd12:38:480
71912220,1sleep37187-21lspci11:38:593
568399220,4cyclictest0-21swapper/312:09:293
568299221,3cyclictest0-21swapper/208:46:192
568299220,4cyclictest20036-21bash10:01:252
568299220,4cyclictest0-21swapper/208:30:382
568199223,1cyclictest0-21swapper/109:00:571
568199223,1cyclictest0-21swapper/109:00:571
568099223,1cyclictest0-21swapper/008:17:000
568099222,2cyclictest0-21swapper/010:29:060
307992220,0sleep20-21swapper/212:11:002
301802220,0sleep30-21swapper/311:04:523
568399210,3cyclictest0-21swapper/309:26:263
568399210,3cyclictest0-21swapper/309:09:293
568399210,3cyclictest0-21swapper/308:31:593
568399210,3cyclictest0-21swapper/308:05:183
568399210,2cyclictest0-21swapper/309:23:143
568299213,2cyclictest0-21swapper/207:33:382
568299211,2cyclictest32191-21bash10:54:082
568299211,2cyclictest0-21swapper/209:34:262
568299211,2cyclictest0-21swapper/208:58:392
568299211,2cyclictest0-21swapper/208:24:242
568299210,4cyclictest0-21swapper/209:40:312
568299210,3cyclictest0-21swapper/212:09:242
568299210,3cyclictest0-21swapper/211:36:152
568299210,3cyclictest0-21swapper/208:41:512
568299210,3cyclictest0-21swapper/208:11:402
568199211,2cyclictest26052-21sshd11:08:021
568199211,2cyclictest0-21swapper/108:36:071
568199211,2cyclictest0-21swapper/108:00:161
568199210,3cyclictest0-21swapper/107:55:161
568099212,1cyclictest0-21swapper/011:12:020
568099212,1cyclictest0-21swapper/009:00:120
568099212,1cyclictest0-21swapper/009:00:120
568099212,1cyclictest0-21swapper/007:40:260
568099211,2cyclictest0-21swapper/011:09:370
568099211,2cyclictest0-21swapper/008:04:570
568099210,1cyclictest1592-21rm09:51:530
568099210,1cyclictest0-21swapper/011:36:030
229602210,0sleep10-21swapper/111:55:201
568399203,1cyclictest0-21swapper/308:15:163
568399201,1cyclictest0-21swapper/308:13:503
568399201,1cyclictest0-21swapper/307:35:303
568399200,4cyclictest31073-21sshd09:10:573
568399200,3cyclictest12324-21cp12:27:253
568399200,2cyclictest0-21swapper/309:39:433
568399200,1cyclictest0-21swapper/309:52:243
568299203,3cyclictest0-21swapper/210:48:212
568299200,2cyclictest0-21swapper/210:25:222
568199205,3cyclictest6256-21sshd11:35:151
568199200,2cyclictest0-21swapper/112:01:301
568199200,2cyclictest0-21swapper/108:29:031
568199200,2cyclictest0-21swapper/107:27:381
568099203,2cyclictest0-21swapper/012:09:120
568099200,2cyclictest0-21swapper/011:47:090
568099200,0cyclictest0-21swapper/007:15:010
144962200,1sleep214495-21bash10:41:062
568399190,1cyclictest0-21swapper/312:21:003
568399190,1cyclictest0-21swapper/311:08:163
568399190,1cyclictest0-21swapper/310:55:413
568399190,1cyclictest0-21swapper/310:30:253
568399190,1cyclictest0-21swapper/310:09:003
568399190,1cyclictest0-21swapper/310:02:483
568399190,1cyclictest0-21swapper/309:43:313
568399190,1cyclictest0-21swapper/308:57:153
568399190,1cyclictest0-21swapper/308:41:263
568399190,1cyclictest0-21swapper/307:49:233
568399190,1cyclictest0-21swapper/307:25:143
568399190,1cyclictest0-21swapper/307:23:113
568399190,1cyclictest0-21swapper/307:15:123
568299190,4cyclictest0-21swapper/211:25:382
568299190,1cyclictest104050irq/26-enp2s0-r10:39:332
568299190,1cyclictest0-21swapper/211:19:502
568299190,1cyclictest0-21swapper/211:12:132
568299190,1cyclictest0-21swapper/211:09:462
568299190,1cyclictest0-21swapper/209:14:192
568299190,1cyclictest0-21swapper/209:09:032
568299190,1cyclictest0-21swapper/208:36:442
568299190,1cyclictest0-21swapper/208:27:372
568299190,1cyclictest0-21swapper/207:47:262
568299190,1cyclictest0-21swapper/207:38:522
568299190,1cyclictest0-21swapper/207:27:412
568299190,1cyclictest0-21swapper/207:24:262
568199192,2cyclictest0-21swapper/108:08:331
5681991911,1cyclictest25-21ksoftirqd/111:25:301
568199190,4cyclictest22443-21sshd09:31:381
568199190,2cyclictest1432-21lspci11:23:411
568199190,1cyclictest27808-21bash09:14:081
568199190,1cyclictest0-21swapper/111:00:561
568199190,1cyclictest0-21swapper/110:05:121
568199190,1cyclictest0-21swapper/109:47:481
568199190,1cyclictest0-21swapper/108:57:381
568199190,1cyclictest0-21swapper/108:12:471
568199190,1cyclictest0-21swapper/107:41:041
568199190,1cyclictest0-21swapper/107:35:021
568199190,1cyclictest0-21swapper/107:31:491
568199190,1cyclictest0-21swapper/107:19:271
568199190,1cyclictest0-21swapper/107:10:251
568099194,0cyclictest722-40kipmi009:25:230
568099192,2cyclictest0-21swapper/009:18:450
568099190,1cyclictest0-21swapper/011:51:310
568099190,1cyclictest0-21swapper/010:15:490
568099190,1cyclictest0-21swapper/008:45:080
568099190,1cyclictest0-21swapper/008:37:370
568099190,1cyclictest0-21swapper/008:21:440
568099190,1cyclictest0-21swapper/007:34:280
293832190,0sleep20-21swapper/211:59:502
568399181,2cyclictest0-21swapper/307:51:253
568399180,3cyclictest0-21swapper/310:44:193
568399180,2cyclictest4017-21sshd11:53:133
568399180,2cyclictest0-21swapper/308:49:473
568399180,1cyclictest0-21swapper/312:13:163
568399180,1cyclictest0-21swapper/311:12:253
568399180,1cyclictest0-21swapper/307:10:363
568399180,0cyclictest0-21swapper/312:33:583
568399180,0cyclictest0-21swapper/312:33:583
568399180,0cyclictest0-21swapper/307:33:263
568299180,3cyclictest3807-21sshd10:17:592
568299180,3cyclictest15850-21sshd12:39:062
568299180,2cyclictest14376-21bash11:32:292
568299180,2cyclictest0-21swapper/210:21:032
568299180,1cyclictest0-21swapper/212:26:192
568299180,1cyclictest0-21swapper/210:55:262
568299180,0cyclictest0-21swapper/208:18:382
568199182,1cyclictest0-21swapper/110:39:211
5681991816,1cyclictest24091-21ipmitool12:10:161
5681991810,2cyclictest25-21ksoftirqd/110:14:041
568199180,3cyclictest0-21swapper/108:33:311
568199180,3cyclictest0-21swapper/108:18:301
568199180,2cyclictest15601-21sshd11:17:541
568199180,2cyclictest0-21swapper/109:21:121
568199180,1cyclictest0-21swapper/110:53:311
568199180,1cyclictest0-21swapper/108:24:531
568099181,2cyclictest0-21swapper/008:54:020
568099181,1cyclictest11068-21grepconf.sh12:34:530
568099181,1cyclictest11068-21grepconf.sh12:34:530
568099180,3cyclictest0-21swapper/010:23:270
568099180,2cyclictest25838-21nfsd10:44:210
568099180,1cyclictest0-21swapper/009:59:140
568099180,0cyclictest0-21swapper/008:33:130
568399170,2cyclictest0-21swapper/310:49:083
568399170,2cyclictest0-21swapper/309:16:193
568399170,2cyclictest0-21swapper/308:27:123
568299174,11cyclictest0-21swapper/212:33:582
568299174,11cyclictest0-21swapper/212:33:582
568299170,2cyclictest0-21swapper/209:00:192
568299170,2cyclictest0-21swapper/209:00:192
568299170,1cyclictest25840-21nfsd09:19:152
568199179,1cyclictest25-21ksoftirqd/110:34:281
568199176,4cyclictest4037-21sshd11:31:181
568199176,2cyclictest25-21ksoftirqd/112:07:391
568199174,1cyclictest25-21ksoftirqd/109:35:331
568199170,2cyclictest20979-21bash11:51:261
568199170,2cyclictest0-21swapper/110:42:481
568199170,1cyclictest0-21swapper/112:32:101
568199170,1cyclictest0-21swapper/112:32:101
568099170,2cyclictest25838-21nfsd09:48:220
568099170,2cyclictest14721-21bash10:30:140
568099170,0cyclictest0-21swapper/007:11:380
568399168,7cyclictest26744-21sshd12:17:553
568399160,3cyclictest0-21swapper/311:29:543
568399160,1cyclictest102550irq/24-eno112:38:133
568399160,1cyclictest0-21swapper/308:51:273
568399160,1cyclictest0-21swapper/308:38:533
568399160,1cyclictest0-21swapper/307:55:133
568299160,1cyclictest25840-21nfsd10:11:102
568299160,1cyclictest25839-21nfsd07:10:212
568299160,1cyclictest0-21swapper/209:21:352
568299160,1cyclictest0-21swapper/208:53:162
568299160,1cyclictest0-21swapper/208:02:222
568299160,1cyclictest0-21swapper/207:59:522
568299160,1cyclictest0-21swapper/207:40:462
568199161,2cyclictest0-21swapper/110:25:041
568199161,12cyclictest25-21ksoftirqd/112:25:401
568199160,1cyclictest0-21swapper/111:44:531
568199160,1cyclictest0-21swapper/110:46:471
568199160,1cyclictest0-21swapper/108:48:131
568099162,2cyclictest0-21swapper/011:34:470
568099161,1cyclictest18887-21bash11:43:560
568099160,1cyclictest0-21swapper/012:02:360
568099160,1cyclictest0-21swapper/011:56:360
568099160,1cyclictest0-21swapper/010:08:290
568099160,1cyclictest0-21swapper/007:47:000
218192160,0sleep00-21swapper/011:25:570
5683991513,1cyclictest593-21systemd-journal10:39:213
5683991511,3cyclictest3347-21grepconf.sh11:42:113
568399150,1cyclictest5507-21lspci11:24:093
568399150,1cyclictest0-21swapper/307:44:013
568299150,1cyclictest0-21swapper/207:15:012
568299150,0cyclictest0-21swapper/207:50:302
568199158,3cyclictest25-21ksoftirqd/110:15:551
568199155,4cyclictest25-21ksoftirqd/109:15:291
5681991513,1cyclictest231rcuc/108:44:591
568199150,1cyclictest0-21swapper/109:40:061
568199150,1cyclictest0-21swapper/107:52:001
5680991514,0cyclictest0-21swapper/009:37:200
568099151,2cyclictest0-21swapper/007:21:030
568099151,1cyclictest18164-21lspci09:13:050
568099150,1cyclictest0-21swapper/007:50:250
568099150,0cyclictest0-21swapper/009:08:520
568399144,9cyclictest8583-21bash10:11:093
568399140,3cyclictest0-21swapper/312:04:543
568399140,1cyclictest0-21swapper/311:15:433
5682991412,1cyclictest33-21ksoftirqd/212:20:002
568199140,1cyclictest0-21swapper/108:50:571
568099143,2cyclictest0-21swapper/010:35:180
5680991412,1cyclictest14973-21bash09:41:430
568099140,2cyclictest7869-21grepconf.sh10:58:410
568399130,12cyclictest0-21swapper/311:31:263
5680991310,2cyclictest15089-21polkitd07:24:590
568099130,2cyclictest0-21swapper/008:44:490
568099130,2cyclictest0-21swapper/008:08:070
568099130,1cyclictest0-21swapper/010:11:590
568399120,4cyclictest0-21swapper/308:21:113
5681991210,1cyclictest231rcuc/107:50:001
5680991210,1cyclictest27083-21grepconf.sh11:15:280
568099120,1cyclictest0-21swapper/008:58:010
568399118,2cyclictest27434-21ntp_states08:00:193
568399110,10cyclictest20461-21latency_hist09:05:003
568399110,10cyclictest20461-21latency_hist09:04:593
568199111,9cyclictest25-21ksoftirqd/107:20:231
568199110,10cyclictest21261-21memory09:05:171
568099110,2cyclictest16731-21expr07:35:110
568099110,10cyclictest0-21swapper/008:25:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional