You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-08-05 - 07:23

Intel(R) Xeon(R) CPU E3-1220L V2 @ 2.30GHz, Linux 4.19.8-rt6 (Profile)

Latency plot of system in rack #4, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the alll highest latencies:
System rack4slot0.osadl.org (updated Thu Aug 05, 2021 00:44:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
39752740,0sleep00-21swapper/023:49:220
73172700,0sleep10-21swapper/123:40:581
196102690,0sleep00-21swapper/022:02:540
224172670,0sleep00-21swapper/021:17:520
94662630,1sleep39471-21bash22:55:203
84112620,0sleep30-21swapper/321:32:253
84112620,0sleep30-21swapper/321:32:243
152122610,0sleep20-21swapper/222:01:372
939825725,9sleep20-21swapper/219:07:362
953425324,9sleep00-21swapper/019:08:470
919825119,10sleep10-21swapper/119:05:391
925224328,9sleep30-21swapper/319:06:143
9792992810,2cyclictest2265-21nfsd22:15:442
980099233,2cyclictest0-21swapper/321:49:153
980099233,2cyclictest0-21swapper/300:23:263
978099233,2cyclictest0-21swapper/021:32:070
978099233,2cyclictest0-21swapper/021:32:060
978099233,2cyclictest0-21swapper/019:43:030
980099220,4cyclictest0-21swapper/319:29:253
979299222,2cyclictest0-21swapper/200:35:432
979299220,4cyclictest0-21swapper/221:57:252
979299220,4cyclictest0-21swapper/221:08:292
978099222,2cyclictest0-21swapper/023:31:540
978099222,2cyclictest0-21swapper/023:26:200
978099222,2cyclictest0-21swapper/020:51:560
276332220,1sleep1241ktimersoftd/121:28:491
980099210,3cyclictest0-21swapper/323:46:143
980099210,3cyclictest0-21swapper/323:05:413
980099210,3cyclictest0-21swapper/321:51:163
980099210,3cyclictest0-21swapper/320:03:483
979299212,1cyclictest0-21swapper/221:23:232
979299211,4cyclictest19314-21sshd23:06:552
979299210,3cyclictest28318-21rs:main2
979299210,3cyclictest0-21swapper/220:55:142
979299210,3cyclictest0-21swapper/220:23:042
979299210,3cyclictest0-21swapper/219:51:062
979299210,3cyclictest0-21swapper/200:06:252
979299210,3cyclictest0-21swapper/200:06:252
979299210,3cyclictest0-21swapper/200:01:122
978099211,2cyclictest7596-21sshd00:08:570
978099211,2cyclictest7596-21sshd00:08:560
978099211,2cyclictest0-21swapper/019:26:140
978099211,2cyclictest0-21swapper/000:29:570
978099211,2cyclictest0-21swapper/000:20:490
978099210,3cyclictest0-21swapper/021:20:220
235462210,1sleep123548-21nfsd421:00:181
980099203,2cyclictest13171-21lspci22:37:563
980099203,2cyclictest0-21swapper/320:36:053
980099200,4cyclictest0-21swapper/322:31:553
980099200,2cyclictest0-21swapper/323:55:403
980099200,2cyclictest0-21swapper/322:47:513
980099200,2cyclictest0-21swapper/322:47:513
980099200,2cyclictest0-21swapper/300:05:593
980099200,2cyclictest0-21swapper/300:05:583
980099200,1cyclictest0-21swapper/322:00:193
979299209,1cyclictest666-21hddtemp_smartct20:05:132
979299202,2cyclictest0-21swapper/220:35:082
979299200,4cyclictest0-21swapper/220:48:312
979299200,2cyclictest0-21swapper/223:59:352
979299200,2cyclictest0-21swapper/223:01:492
979299200,2cyclictest0-21swapper/221:51:482
979299200,2cyclictest0-21swapper/219:32:352
978099203,2cyclictest0-21swapper/021:08:470
978099201,1cyclictest0-21swapper/023:37:460
978099201,1cyclictest0-21swapper/023:20:150
978099201,1cyclictest0-21swapper/022:57:130
978099201,1cyclictest0-21swapper/022:54:360
978099200,4cyclictest6800-21df20:20:130
978099200,2cyclictest0-21swapper/023:12:570
978099200,2cyclictest0-21swapper/022:16:040
980099193,1cyclictest0-21swapper/319:54:093
980099190,3cyclictest0-21swapper/300:29:513
980099190,2cyclictest0-21swapper/322:51:173
980099190,1cyclictest0-21swapper/323:14:583
980099190,1cyclictest0-21swapper/322:25:413
980099190,1cyclictest0-21swapper/322:12:073
980099190,1cyclictest0-21swapper/321:20:283
980099190,1cyclictest0-21swapper/321:15:533
980099190,1cyclictest0-21swapper/321:14:403
980099190,1cyclictest0-21swapper/320:31:483
980099190,1cyclictest0-21swapper/319:21:443
980099190,1cyclictest0-21swapper/319:11:013
980099190,1cyclictest0-21swapper/300:12:413
979299192,2cyclictest0-21swapper/223:47:342
979299192,2cyclictest0-21swapper/219:55:212
979299190,1cyclictest0-21swapper/223:36:382
979299190,1cyclictest0-21swapper/223:27:342
979299190,1cyclictest0-21swapper/223:10:472
979299190,1cyclictest0-21swapper/222:55:572
979299190,1cyclictest0-21swapper/222:49:052
979299190,1cyclictest0-21swapper/222:49:042
979299190,1cyclictest0-21swapper/222:37:372
979299190,1cyclictest0-21swapper/221:48:152
979299190,1cyclictest0-21swapper/221:36:282
979299190,1cyclictest0-21swapper/221:10:112
979299190,1cyclictest0-21swapper/221:03:382
979299190,1cyclictest0-21swapper/220:50:032
979299190,1cyclictest0-21swapper/220:43:282
979299190,1cyclictest0-21swapper/220:34:442
979299190,1cyclictest0-21swapper/220:13:492
979299190,1cyclictest0-21swapper/219:45:332
979299190,1cyclictest0-21swapper/219:42:152
979299190,1cyclictest0-21swapper/219:39:362
979299190,1cyclictest0-21swapper/219:21:402
979299190,1cyclictest0-21swapper/200:25:442
979299190,1cyclictest0-21swapper/200:22:082
978099193,1cyclictest0-21swapper/019:56:540
978099192,2cyclictest0-21swapper/023:51:190
978099192,2cyclictest0-21swapper/022:05:370
978099192,2cyclictest0-21swapper/020:41:460
978099192,2cyclictest0-21swapper/019:11:390
978099192,2cyclictest0-21swapper/000:15:540
978099190,4cyclictest32188-21sshd22:24:560
978099190,1cyclictest0-21swapper/022:40:240
978099190,1cyclictest0-21swapper/022:39:530
978099190,1cyclictest0-21swapper/020:45:550
978099190,1cyclictest0-21swapper/020:38:400
978099190,1cyclictest0-21swapper/020:06:180
978099190,1cyclictest0-21swapper/019:46:450
978099190,1cyclictest0-21swapper/019:36:310
978099190,1cyclictest0-21swapper/000:33:560
47352190,0sleep10-21swapper/122:53:531
980099180,3cyclictest0-21swapper/322:21:553
980099180,2cyclictest0-21swapper/320:47:363
980099180,2cyclictest0-21swapper/320:25:153
980099180,0cyclictest0-21swapper/323:26:003
980099180,0cyclictest0-21swapper/322:17:013
980099180,0cyclictest0-21swapper/321:35:283
980099180,0cyclictest0-21swapper/321:06:413
980099180,0cyclictest0-21swapper/320:59:383
980099180,0cyclictest0-21swapper/320:52:123
980099180,0cyclictest0-21swapper/320:43:183
980099180,0cyclictest0-21swapper/320:06:373
980099180,0cyclictest0-21swapper/319:57:393
980099180,0cyclictest0-21swapper/319:38:073
980099180,0cyclictest0-21swapper/319:33:373
979299182,2cyclictest0-21swapper/223:19:482
979299180,3cyclictest0-21swapper/223:40:072
979299180,3cyclictest0-21swapper/222:50:172
979299180,2cyclictest5975-21sshd23:21:462
979299180,1cyclictest0-21swapper/221:33:182
979299180,1cyclictest0-21swapper/221:33:182
979299180,1cyclictest0-21swapper/220:25:362
979299180,1cyclictest0-21swapper/219:13:582
978099182,2cyclictest0-21swapper/019:34:160
978099181,2cyclictest7404-21lspci21:41:150
978099181,2cyclictest0-21swapper/021:52:300
978099181,2cyclictest0-21swapper/000:10:260
978099180,3cyclictest22048-21lspci21:45:230
978099180,3cyclictest0-21swapper/021:12:430
978099180,2cyclictest0-21swapper/021:00:330
978099180,1cyclictest0-21swapper/023:16:520
978099180,1cyclictest0-21swapper/020:55:170
978099180,0cyclictest0-21swapper/023:07:390
978099180,0cyclictest0-21swapper/022:46:250
978099180,0cyclictest0-21swapper/022:46:240
978099180,0cyclictest0-21swapper/022:26:070
978099180,0cyclictest0-21swapper/021:56:350
978099180,0cyclictest0-21swapper/019:52:260
980099173,2cyclictest0-21swapper/319:46:313
980099173,2cyclictest0-21swapper/300:02:283
980099173,1cyclictest0-21swapper/319:15:183
980099170,1cyclictest102550irq/24-eno100:35:443
980099170,0cyclictest0-21swapper/323:21:153
979299170,2cyclictest0-21swapper/222:05:482
978099171,1cyclictest0-21swapper/021:26:220
978099170,2cyclictest0-21swapper/021:35:480
978099170,0cyclictest0-21swapper/023:00:150
980099160,1cyclictest17754-21lspci21:44:183
980099160,1cyclictest0-21swapper/323:43:023
980099160,1cyclictest0-21swapper/323:38:223
980099160,1cyclictest0-21swapper/300:19:493
979299162,2cyclictest17621-21sshd21:25:482
979299160,1cyclictest2266-21nfsd23:53:152
979299160,1cyclictest0-21swapper/222:31:192
979299160,1cyclictest0-21swapper/222:27:552
979299160,1cyclictest0-21swapper/222:24:122
979299160,1cyclictest0-21swapper/221:15:092
979299160,1cyclictest0-21swapper/220:17:432
979299160,1cyclictest0-21swapper/200:11:022
978599165,10cyclictest7237-21bash21:32:051
978599165,10cyclictest7237-21bash21:32:051
978099162,2cyclictest0-21swapper/022:30:510
978099162,2cyclictest0-21swapper/020:15:480
978099162,1cyclictest0-21swapper/019:17:200
978099161,2cyclictest0-21swapper/023:43:300
978099160,1cyclictest0-21swapper/023:55:010
978099160,1cyclictest0-21swapper/020:11:560
978099160,1cyclictest0-21swapper/000:02:110
980099150,3cyclictest0-21swapper/321:00:593
980099150,3cyclictest0-21swapper/320:21:233
980099150,3cyclictest0-21swapper/320:15:533
980099150,2cyclictest3307-21bash23:30:353
980099150,2cyclictest0-21swapper/323:17:513
979299156,8cyclictest12692-21sshd23:33:262
979299150,4cyclictest0-21swapper/221:40:142
979299150,3cyclictest0-21swapper/222:12:062
979299150,1cyclictest0-21swapper/222:44:242
979299150,1cyclictest0-21swapper/220:00:372
979299150,1cyclictest0-21swapper/219:18:152
979299150,1cyclictest0-21swapper/200:19:352
978599154,10cyclictest27429-21bash23:18:481
978599150,14cyclictest3065-21sshd00:07:331
978599150,14cyclictest3065-21sshd00:07:331
978099150,1cyclictest0-21swapper/022:10:190
978099150,1cyclictest0-21swapper/020:02:400
978099150,0cyclictest0-21swapper/000:37:160
980099140,0cyclictest0-21swapper/321:25:513
980099140,0cyclictest0-21swapper/319:43:173
978599144,9cyclictest20779-21bash00:34:411
978599143,10cyclictest21649-21bash22:12:421
978599142,10cyclictest32305-21bash21:11:211
9785991412,1cyclictest2087-21bash21:21:071
9785991411,2cyclictest5522-21nscd23:51:591
978599140,2cyclictest0-21swapper/121:15:161
978599140,13cyclictest31793-21sshd23:29:471
978099140,2cyclictest0-21swapper/020:27:070
980099132,2cyclictest27593-21sshd21:55:543
980099130,4cyclictest0-21swapper/323:01:223
980099130,1cyclictest0-21swapper/322:43:003
980099130,1cyclictest0-21swapper/322:06:563
978599133,9cyclictest8075-21nscd00:18:371
978599133,1cyclictest2796-21bash21:58:151
978599132,10cyclictest28317-21in:imjournal21:53:561
978599132,10cyclictest23561-21bash23:08:121
9785991311,1cyclictest27143-21cstates23:00:111
9785991311,1cyclictest24869-21sshd22:04:341
9785991311,1cyclictest21655-21lspci23:35:511
9785991310,2cyclictest7621-21sshd22:17:501
978099132,10cyclictest11590-21sensors20:30:320
980099120,1cyclictest0-21swapper/300:31:153
980099120,0cyclictest0-21swapper/323:50:343
979299120,10cyclictest0-21swapper/219:25:212
978599129,1cyclictest10912-21diskstats20:30:191
978599128,2cyclictest2411-21lspci21:40:041
978599128,2cyclictest21314-21sshd23:45:101
978599128,2cyclictest21263-21sshd21:35:571
978599122,1cyclictest20982-21grepconf.sh22:40:171
978599122,1cyclictest1510-21snmpd00:37:121
978599121,10cyclictest593-21systemd-journal21:45:171
9785991210,1cyclictest0-21swapper/123:55:041
9785991210,1cyclictest0-21swapper/100:12:511
978599120,3cyclictest13567-21sshd23:14:581
978599120,3cyclictest0-21swapper/122:25:011
978599120,2cyclictest29284-21sshd22:05:311
978599120,2cyclictest25533-21irqstats00:25:171
978599120,11cyclictest17442-21sshd00:02:251
978599120,11cyclictest0-21swapper/122:30:431
978599120,10cyclictest487-21sshd23:30:041
978599120,10cyclictest4767-21df20:15:181
978599120,10cyclictest31806-21grepconf.sh23:20:061
978599120,10cyclictest2987-21bash22:35:091
978599120,10cyclictest25083-21df21:05:131
978599120,10cyclictest17477-21ntp_states20:45:211
978599120,10cyclictest14803-21hddtemp_smartct19:20:151
978599120,10cyclictest10823-21sshd22:46:231
978599120,10cyclictest10823-21sshd22:46:221
978599120,10cyclictest10349-21cat22:55:261
978599120,10cyclictest0-21swapper/119:15:201
9780991210,1cyclictest1278-21runrttasks19:20:280
980099110,0cyclictest0-21swapper/320:11:233
978599119,1cyclictest0-21swapper/120:50:141
978599119,1cyclictest0-21swapper/120:00:221
978599119,1cyclictest0-21swapper/119:55:231
978599118,2cyclictest22589-21/usr/sbin/munin19:40:231
978599110,2cyclictest25306-21gdbus20:45:001
978599110,2cyclictest24890-21expr19:45:161
978599110,2cyclictest0-21swapper/120:22:321
978599110,1cyclictest13420-21ntp_states20:35:161
978599110,1cyclictest0-21swapper/120:55:141
978599110,10cyclictest928-21ntpd19:35:181
978599110,10cyclictest9111-21memory20:25:171
978599110,10cyclictest3252-21ntp_states20:10:161
978599110,10cyclictest27188-21memory19:50:191
978599110,10cyclictest17077-21memory19:25:171
978599110,10cyclictest10787-21ntp_states19:10:181
978599110,10cyclictest10008-21sshd00:20:021
978599110,10cyclictest0-21swapper/122:20:041
978599110,10cyclictest0-21swapper/120:05:111
978599110,10cyclictest0-21swapper/119:30:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional