You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-10-24 - 02:47

Intel(R) Core(TM) i5-6440EQ CPU @ 2.70GHz, Linux 4.9.47-rt37 (Profile)

Latency plot of system in rack #5, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the alll highest latencies:
System rack5slot2.osadl.org (updated Sun Oct 24, 2021 00:44:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
169222196181,11sleep10-21swapper/119:08:321
169482191157,29sleep00-21swapper/019:08:510
166892177160,12sleep30-21swapper/319:05:343
148172171154,12sleep20-21swapper/219:05:082
46832440,0sleep10-21swapper/122:38:101
263292420,0sleep10-21swapper/121:54:451
272832410,0sleep10-21swapper/123:48:001
152502400,0sleep30-21swapper/300:00:193
97982110,0sleep10-21swapper/122:40:331
172139993,1cyclictest40-21ksoftirqd/322:55:003
172139983,0cyclictest40-21ksoftirqd/321:25:143
172139982,2cyclictest40-21ksoftirqd/323:15:013
172139972,2cyclictest40-21ksoftirqd/321:00:233
172139972,1cyclictest40-21ksoftirqd/322:15:363
172139971,1cyclictest40-21ksoftirqd/323:55:313
172139971,1cyclictest40-21ksoftirqd/323:45:163
172139971,1cyclictest40-21ksoftirqd/323:40:453
172139971,1cyclictest40-21ksoftirqd/323:06:193
172139971,1cyclictest40-21ksoftirqd/322:12:573
172139971,1cyclictest40-21ksoftirqd/321:50:283
172139971,1cyclictest40-21ksoftirqd/321:48:253
172139971,1cyclictest40-21ksoftirqd/300:28:323
172139971,1cyclictest40-21ksoftirqd/300:17:463
172139971,1cyclictest10868-21latency_hist23:20:003
172139970,2cyclictest0-21swapper/322:56:163
172139970,1cyclictest40-21ksoftirqd/322:45:503
172139970,1cyclictest40-21ksoftirqd/320:30:153
172139970,1cyclictest40-21ksoftirqd/319:15:013
172139970,1cyclictest40-21ksoftirqd/300:30:043
172089970,3cyclictest21806-21perl22:10:272
171999971,3cyclictest26699-21perf21:55:000
172139962,2cyclictest40-21ksoftirqd/323:35:213
172139962,2cyclictest40-21ksoftirqd/322:25:173
172139961,1cyclictest40-21ksoftirqd/323:50:133
172139961,1cyclictest40-21ksoftirqd/323:25:133
172139961,1cyclictest40-21ksoftirqd/322:23:103
172139961,1cyclictest40-21ksoftirqd/322:05:223
172139961,1cyclictest40-21ksoftirqd/321:40:193
172139961,1cyclictest40-21ksoftirqd/321:36:223
172139961,1cyclictest40-21ksoftirqd/321:17:363
172139961,1cyclictest40-21ksoftirqd/321:10:243
172139961,1cyclictest40-21ksoftirqd/320:50:013
172139961,1cyclictest40-21ksoftirqd/300:20:373
172139961,1cyclictest40-21ksoftirqd/300:10:063
172139961,1cyclictest0-21swapper/321:30:113
172139961,1cyclictest0-21swapper/319:35:223
172139960,2cyclictest40-21ksoftirqd/319:20:203
172139960,1cyclictest40-21ksoftirqd/323:20:193
172139960,1cyclictest40-21ksoftirqd/322:40:293
172139960,1cyclictest40-21ksoftirqd/322:35:153
172139960,1cyclictest40-21ksoftirqd/322:34:343
172139960,1cyclictest40-21ksoftirqd/321:57:023
172139960,1cyclictest40-21ksoftirqd/319:30:193
172139960,1cyclictest40-21ksoftirqd/300:35:143
172139960,1cyclictest40-21ksoftirqd/300:05:203
172139960,1cyclictest0-21swapper/320:20:153
171999962,4cyclictest9467-21basename21:45:010
3308250,0sleep10-21swapper/122:55:391
172139954,0cyclictest40-21ksoftirqd/320:40:373
172139951,1cyclictest40-21ksoftirqd/320:35:173
172139951,1cyclictest40-21ksoftirqd/320:35:173
172139951,1cyclictest40-21ksoftirqd/320:26:073
172139951,1cyclictest40-21ksoftirqd/319:10:133
172139950,2cyclictest40-21ksoftirqd/322:00:193
172139950,2cyclictest0-21swapper/321:20:183
172139950,2cyclictest0-21swapper/320:05:013
172139950,1cyclictest40-21ksoftirqd/323:30:153
172139950,1cyclictest40-21ksoftirqd/323:00:343
172139950,1cyclictest40-21ksoftirqd/319:55:213
172139950,1cyclictest0-21swapper/319:25:193
172139950,0cyclictest81rcu_preempt19:50:003
172139950,0cyclictest0-21swapper/321:05:153
172139950,0cyclictest0-21swapper/320:55:213
172089950,5cyclictest0-21swapper/223:10:152
172089950,5cyclictest0-21swapper/220:55:182
172089950,5cyclictest0-21swapper/200:18:372
172089950,0cyclictest0-21swapper/221:45:592
171999950,5cyclictest0-21swapper/023:04:290
171999950,4cyclictest0-21swapper/020:45:130
171999950,1cyclictest6954-21ntp_states19:50:250
13161250,0sleep00-21swapper/023:40:060
32058240,0sleep10-21swapper/122:15:511
172139943,0cyclictest40-21ksoftirqd/320:50:173
172139941,1cyclictest40-21ksoftirqd/320:10:203
172139941,1cyclictest40-21ksoftirqd/319:50:213
172139940,4cyclictest0-21swapper/319:40:143
172139940,0cyclictest40-21ksoftirqd/320:00:223
172139940,0cyclictest0-21swapper/320:15:183
172089944,0cyclictest0-21swapper/222:00:062
172089943,1cyclictest482-21nscd23:30:012
172089943,1cyclictest26190-21grep19:25:212
172089942,1cyclictest713-21cut20:45:132
172089942,1cyclictest14915-21ssh00:00:022
172089940,4cyclictest0-21swapper/223:55:372
172089940,4cyclictest0-21swapper/223:52:232
172089940,4cyclictest0-21swapper/223:48:002
172089940,4cyclictest0-21swapper/223:40:152
172089940,4cyclictest0-21swapper/223:25:122
172089940,4cyclictest0-21swapper/223:20:192
172089940,4cyclictest0-21swapper/223:15:142
172089940,4cyclictest0-21swapper/223:05:212
172089940,4cyclictest0-21swapper/222:55:142
172089940,4cyclictest0-21swapper/222:47:252
172089940,4cyclictest0-21swapper/222:40:202
172089940,4cyclictest0-21swapper/222:30:152
172089940,4cyclictest0-21swapper/222:26:102
172089940,4cyclictest0-21swapper/221:50:232
172089940,4cyclictest0-21swapper/221:37:422
172089940,4cyclictest0-21swapper/221:25:162
172089940,4cyclictest0-21swapper/221:06:422
172089940,4cyclictest0-21swapper/220:40:002
172089940,4cyclictest0-21swapper/220:30:142
172089940,4cyclictest0-21swapper/220:26:582
172089940,4cyclictest0-21swapper/220:05:162
172089940,4cyclictest0-21swapper/219:43:202
172089940,4cyclictest0-21swapper/219:35:162
172089940,4cyclictest0-21swapper/219:10:122
172089940,4cyclictest0-21swapper/200:20:122
172089940,4cyclictest0-21swapper/200:13:512
172089940,4cyclictest0-21swapper/200:05:182
172089940,3cyclictest9674-21/usr/sbin/munin23:00:352
172089940,3cyclictest6753-21latency_hist21:25:012
172089940,3cyclictest5672-21ssh21:42:102
172089940,3cyclictest32538-21ssh22:16:142
172089940,3cyclictest2334-21rm22:36:222
172089940,3cyclictest2158-21perl00:30:162
172089940,3cyclictest20300-21munin-run22:10:002
172089940,3cyclictest19994-21munin-node19:15:012
172089940,3cyclictest11641-21ssh00:35:222
172089940,1cyclictest22255-21/usr/sbin/munin21:15:172
172089940,0cyclictest0-21swapper/222:53:482
172089940,0cyclictest0-21swapper/200:25:282
172059944,0cyclictest0-21swapper/100:12:181
172059941,2cyclictest0-21swapper/122:25:181
172059941,2cyclictest0-21swapper/121:25:111
172059941,2cyclictest0-21swapper/119:10:111
172059941,1cyclictest0-21swapper/123:50:141
172059940,4cyclictest0-21swapper/122:07:101
172059940,2cyclictest0-21swapper/121:20:101
172059940,2cyclictest0-21swapper/100:20:121
171999944,0cyclictest0-21swapper/020:30:220
171999944,0cyclictest0-21swapper/000:13:350
171999943,1cyclictest28159-21cut21:55:220
171999943,1cyclictest19576-21grep19:15:000
171999941,3cyclictest740-21snmpd23:16:430
171999940,4cyclictest0-21swapper/023:50:140
171999940,4cyclictest0-21swapper/023:45:140
171999940,4cyclictest0-21swapper/023:28:390
171999940,4cyclictest0-21swapper/023:05:130
171999940,4cyclictest0-21swapper/022:58:220
171999940,4cyclictest0-21swapper/022:35:110
171999940,4cyclictest0-21swapper/022:30:150
171999940,4cyclictest0-21swapper/021:35:130
171999940,4cyclictest0-21swapper/021:30:110
171999940,4cyclictest0-21swapper/021:26:470
171999940,4cyclictest0-21swapper/021:20:110
171999940,4cyclictest0-21swapper/021:18:580
171999940,4cyclictest0-21swapper/021:06:540
171999940,4cyclictest0-21swapper/020:13:380
171999940,3cyclictest9998-21users19:55:270
171999940,3cyclictest391-21latency_hist22:55:000
171999940,3cyclictest30612-21cat19:35:010
171999940,3cyclictest24253-21ssh22:49:590
171999940,3cyclictest22850-21wc22:10:390
171999940,3cyclictest21832-21ntp_states20:20:270
171999940,3cyclictest15332-21cron20:10:000
171999940,3cyclictest14843-21apt-config00:00:000
171999940,3cyclictest13413-21ssh21:10:170
171999940,3cyclictest0-21swapper/023:30:020
171999940,3cyclictest0-21swapper/020:40:180
171999940,3cyclictest0-21swapper/019:25:160
171999940,0cyclictest4726-21sed22:00:200
171999940,0cyclictest0-21swapper/023:55:510
172089932,1cyclictest740-21snmpd19:30:022
172089930,3cyclictest5581-21ssh22:20:092
172089930,3cyclictest21103-21grep20:20:182
172089930,3cyclictest18578-21df20:15:182
172089930,3cyclictest16495-21cpuspeed_turbos21:30:132
172089930,3cyclictest15584-21munin-node20:10:202
172089930,3cyclictest0-21swapper/223:35:052
172089930,3cyclictest0-21swapper/221:55:162
172089930,3cyclictest0-21swapper/221:00:142
172089930,3cyclictest0-21swapper/220:50:172
172089930,3cyclictest0-21swapper/220:35:082
172089930,3cyclictest0-21swapper/220:35:082
172089930,3cyclictest0-21swapper/220:00:162
172089930,3cyclictest0-21swapper/219:55:152
172089930,3cyclictest0-21swapper/219:50:142
172089930,3cyclictest0-21swapper/219:45:572
172089930,3cyclictest0-21swapper/219:20:172
172059933,0cyclictest0-21swapper/122:32:201
172059933,0cyclictest0-21swapper/121:19:431
172059931,1cyclictest173162chrt00:39:321
172059931,1cyclictest0-21swapper/123:25:131
172059931,1cyclictest0-21swapper/123:15:161
172059931,1cyclictest0-21swapper/123:10:131
172059931,1cyclictest0-21swapper/123:05:001
172059931,1cyclictest0-21swapper/122:50:131
172059931,1cyclictest0-21swapper/122:10:171
172059931,1cyclictest0-21swapper/121:45:131
172059931,1cyclictest0-21swapper/121:35:141
172059931,1cyclictest0-21swapper/121:30:121
172059931,1cyclictest0-21swapper/120:55:171
172059931,1cyclictest0-21swapper/120:25:141
172059931,1cyclictest0-21swapper/120:05:151
172059931,1cyclictest0-21swapper/119:40:131
172059931,1cyclictest0-21swapper/119:35:161
172059931,1cyclictest0-21swapper/119:25:171
172059931,1cyclictest0-21swapper/100:30:111
172059931,1cyclictest0-21swapper/100:05:181
172059931,1cyclictest0-21swapper/100:00:011
172059930,2cyclictest27633-21chrt00:25:381
172059930,2cyclictest242122sleep123:08:311
172059930,2cyclictest12791-21chrt20:04:071
172059930,2cyclictest0-21swapper/123:20:191
172059930,2cyclictest0-21swapper/122:00:151
172059930,2cyclictest0-21swapper/121:40:141
172059930,2cyclictest0-21swapper/121:05:131
172059930,2cyclictest0-21swapper/120:30:131
172059930,2cyclictest0-21swapper/120:10:211
172059930,2cyclictest0-21swapper/119:30:151
172059930,2cyclictest0-21swapper/119:20:181
171999933,0cyclictest19844-21kworker/0:122:40:080
171999933,0cyclictest10249-21kworker/0:200:20:080
171999933,0cyclictest0-21swapper/023:35:020
171999932,1cyclictest5543-21ssh22:20:060
171999930,3cyclictest29627-21grep20:35:250
171999930,3cyclictest29627-21grep20:35:250
171999930,3cyclictest25320-21ssh00:05:300
171999930,3cyclictest14902-21diskstats22:25:190
171999930,3cyclictest12834-21diskmemload23:10:000
171999930,3cyclictest12834-21diskmemload00:25:030
171999930,3cyclictest12834-21diskmemload00:15:080
171999930,3cyclictest12052-21df23:20:210
171999930,3cyclictest0-21swapper/022:15:140
171999930,3cyclictest0-21swapper/022:05:140
171999930,3cyclictest0-21swapper/021:45:140
171999930,3cyclictest0-21swapper/021:00:130
171999930,3cyclictest0-21swapper/020:55:170
171999930,3cyclictest0-21swapper/020:50:050
171999930,3cyclictest0-21swapper/020:25:150
171999930,3cyclictest0-21swapper/020:15:150
171999930,3cyclictest0-21swapper/020:00:170
171999930,3cyclictest0-21swapper/019:45:140
171999930,3cyclictest0-21swapper/019:40:130
171999930,3cyclictest0-21swapper/019:30:100
171999930,3cyclictest0-21swapper/019:20:160
171999930,3cyclictest0-21swapper/019:15:150
171999930,3cyclictest0-21swapper/000:35:120
171999930,3cyclictest0-21swapper/000:30:090
13105230,2sleep20-21swapper/221:10:032
172059921,1cyclictest0-21swapper/123:30:011
172059921,1cyclictest0-21swapper/122:45:151
172059921,1cyclictest0-21swapper/121:00:141
172059921,1cyclictest0-21swapper/120:15:171
172059921,1cyclictest0-21swapper/119:55:161
172059921,0cyclictest0-21swapper/123:55:141
172059920,1cyclictest0-21swapper/123:40:181
172059920,1cyclictest0-21swapper/123:35:121
172059920,1cyclictest0-21swapper/122:20:131
172059920,1cyclictest0-21swapper/121:55:081
172059920,1cyclictest0-21swapper/121:10:171
172059920,1cyclictest0-21swapper/120:50:171
172059920,1cyclictest0-21swapper/120:45:121
172059920,1cyclictest0-21swapper/120:40:181
172059920,1cyclictest0-21swapper/120:35:161
172059920,1cyclictest0-21swapper/120:35:151
172059920,1cyclictest0-21swapper/120:20:151
172059920,1cyclictest0-21swapper/119:50:151
172059920,1cyclictest0-21swapper/119:45:141
172059920,1cyclictest0-21swapper/119:15:151
172059920,1cyclictest0-21swapper/100:15:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional