You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-10-17 - 19:59

Intel(R) Core(TM) i5-6440EQ CPU @ 2.70GHz, Linux 4.9.47-rt37 (Profile)

Latency plot of system in rack #5, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot2.osadl.org (updated Sun Oct 17, 2021 12:44:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
93562203187,11sleep10-21swapper/107:05:551
94722178161,12sleep00-21swapper/007:07:260
94542177159,13sleep20-21swapper/207:07:122
96332176160,12sleep30-21swapper/307:09:293
212542470,0sleep30-21swapper/309:36:293
168932430,0sleep10-21swapper/110:10:591
11952430,0sleep10-21swapper/109:25:261
986299100,1cyclictest5977-21apt-get08:05:130
98629990,1cyclictest9937-21ls10:45:010
98629990,1cyclictest5073-21users09:45:340
98629990,1cyclictest4161-21df09:05:150
98629990,1cyclictest28067-21apt07:45:000
98629982,6cyclictest30565-21ssh10:37:440
98629982,6cyclictest30565-21ssh10:37:440
98629981,1cyclictest0-21swapper/007:25:130
98629980,7cyclictest7115-21cat10:05:210
98629980,1cyclictest716-21idleruntime-cro09:00:000
98629980,1cyclictest5315-21nscd11:19:590
98629980,1cyclictest25752-21awk08:44:590
98629980,1cyclictest23534-21awk09:20:010
98629980,1cyclictest23060-21apt07:35:010
98629980,1cyclictest228-21systemd-journal07:20:010
98629980,1cyclictest22302-21basename10:15:010
98629980,1cyclictest14896-21munin-run12:39:590
98629980,1cyclictest12567-21/usr/sbin/munin11:05:160
98629980,1cyclictest11700-21ssh11:41:020
98629980,1cyclictest0-21swapper/011:49:590
23655280,0sleep10-21swapper/112:25:261
98729970,1cyclictest4776-21ntp_states08:00:212
98629972,5cyclictest740-21snmpd08:13:150
98629971,1cyclictest30926-21grep12:30:150
98629971,1cyclictest10579-21aten2_r5power_v10:45:160
98629970,2cyclictest18131-21pmu-power10:30:200
98629970,1cyclictest9742-21ls10:25:220
98629970,1cyclictest6656-21grep09:10:170
98629970,1cyclictest5975-21ssh11:20:030
98629970,1cyclictest5128-21ssh12:15:160
98629970,1cyclictest3398-21turbostat.cron09:05:000
98629970,1cyclictest3384-21hddtemp_smartct11:55:230
98629970,1cyclictest32484-21apt-get11:35:120
98629970,1cyclictest32481-21python07:50:290
98629970,1cyclictest32087-21cat10:20:150
98629970,1cyclictest3104-21chrt07:58:260
98629970,1cyclictest29437-21irqstats08:50:180
98629970,1cyclictest29031-21apt-get10:00:110
98629970,1cyclictest28828-21df_inode07:45:190
98629970,1cyclictest28480-21ssh12:10:150
98629970,1cyclictest27836-21apt-get10:55:200
98629970,1cyclictest27526-21ssh09:59:160
98629970,1cyclictest27240-21awk10:55:000
98629970,1cyclictest26589-21apt-get09:40:170
98629970,1cyclictest26184-21wget11:50:150
98629970,1cyclictest25152-21ntpq11:30:220
98629970,1cyclictest23256-21dump-pmu-power08:40:000
98629970,1cyclictest23016-21munin-run11:30:000
98629970,1cyclictest22742-21ssh10:15:120
98629970,1cyclictest2249-21find09:25:360
98629970,1cyclictest22047-21expr11:10:150
98629970,1cyclictest22013-21http12:25:160
98629970,1cyclictest20780-21apt08:35:000
98629970,1cyclictest20344-21entropy12:05:190
98629970,1cyclictest19834-21vmstat09:35:240
98629970,1cyclictest15503-21users12:20:330
98629970,1cyclictest13706-21aten2_r5power_v08:20:160
98629970,1cyclictest12397-21ntpq08:15:280
98629970,1cyclictest11623-21ssh11:04:400
98629970,1cyclictest11555-21rm09:50:140
98629970,1cyclictest11063-21memory07:10:250
98629970,1cyclictest11062-21ssh12:00:140
98629970,1cyclictest10645-21ntp_states09:30:270
98629970,1cyclictest0-21swapper/007:20:180
98629970,0cyclictest5752-21perf08:05:000
98629970,0cyclictest24060-21apt-get09:20:150
98629960,1cyclictest232882chrt07:35:120
98629960,1cyclictest16067-21chrt08:25:080
98629960,0cyclictest25976-21apt-get08:45:150
27780260,0sleep20-21swapper/210:35:372
27780260,0sleep20-21swapper/210:35:372
27075260,0sleep10-21swapper/109:21:111
98769950,5cyclictest0-21swapper/311:32:403
98769950,5cyclictest0-21swapper/309:22:533
98769950,4cyclictest0-21swapper/312:35:153
98769950,4cyclictest0-21swapper/309:05:153
98769950,2cyclictest14393-21cat11:05:233
98769950,0cyclictest17576-21ssh12:03:493
98769950,0cyclictest0-21swapper/311:20:353
98729955,0cyclictest5820-21ssh11:38:102
98729955,0cyclictest0-21swapper/209:24:352
98729950,5cyclictest0-21swapper/210:30:482
98729950,5cyclictest0-21swapper/210:11:262
98729950,5cyclictest0-21swapper/209:46:072
98669950,5cyclictest0-21swapper/111:57:491
98769944,0cyclictest0-21swapper/311:56:413
98769944,0cyclictest0-21swapper/310:15:073
98769943,1cyclictest30522-21cron07:50:003
98769943,1cyclictest26754-21diskstats11:50:193
98769943,1cyclictest0-21swapper/310:30:583
98769942,1cyclictest740-21snmpd08:53:523
98769942,1cyclictest740-21snmpd07:37:553
98769942,1cyclictest2074-21awk09:25:343
98769941,3cyclictest54250irq/130-eth009:11:233
98769941,3cyclictest3906-21sh11:36:433
98769940,4cyclictest5373-21ntpq11:00:233
98769940,4cyclictest32668-21taskset12:30:283
98769940,4cyclictest32592-21hddtemp_smartct10:20:173
98769940,4cyclictest21511-21ssh10:14:273
98769940,4cyclictest19595-21df_inode10:50:133
98769940,4cyclictest0-21swapper/312:20:143
98769940,4cyclictest0-21swapper/312:15:113
98769940,4cyclictest0-21swapper/312:10:493
98769940,4cyclictest0-21swapper/311:25:143
98769940,4cyclictest0-21swapper/311:15:153
98769940,4cyclictest0-21swapper/310:55:193
98769940,4cyclictest0-21swapper/310:45:123
98769940,4cyclictest0-21swapper/310:25:133
98769940,4cyclictest0-21swapper/309:53:283
98769940,4cyclictest0-21swapper/309:15:143
98769940,4cyclictest0-21swapper/308:14:543
98769940,4cyclictest0-21swapper/308:00:403
98769940,4cyclictest0-21swapper/307:15:153
98769940,3cyclictest28352-21grep08:49:593
98769940,3cyclictest2656-21sh10:03:213
98769940,3cyclictest26306-21grep10:35:223
98769940,3cyclictest26306-21grep10:35:223
98769940,3cyclictest2556-21hddtemp_smartct10:40:153
98769940,3cyclictest23549-21sh11:10:383
98769940,3cyclictest23225-21memory12:25:233
98769940,3cyclictest18872-21df07:25:183
98769940,3cyclictest11950-21hddtemp_smartct08:15:183
98769940,3cyclictest0-21swapper/311:49:453
98769940,2cyclictest0-21swapper/309:43:363
98769940,1cyclictest9593-21perl11:40:173
98769940,1cyclictest5614-21grep10:05:123
98769940,1cyclictest25784-21/usr/sbin/munin07:40:253
98769940,1cyclictest21171-21ntp_states12:05:243
98769940,1cyclictest18544-21/usr/sbin/munin08:30:183
98769940,0cyclictest0-21swapper/309:55:033
98729944,0cyclictest0-21swapper/209:40:452
98729943,1cyclictest19840-21grep10:50:152
98729941,2cyclictest0-21swapper/210:55:202
98729941,2cyclictest0-21swapper/210:05:122
98729940,4cyclictest0-21swapper/212:35:542
98729940,4cyclictest0-21swapper/212:30:142
98729940,4cyclictest0-21swapper/212:15:122
98729940,4cyclictest0-21swapper/212:02:582
98729940,4cyclictest0-21swapper/211:55:172
98729940,4cyclictest0-21swapper/211:30:142
98729940,4cyclictest0-21swapper/211:10:112
98729940,4cyclictest0-21swapper/210:45:162
98729940,4cyclictest0-21swapper/210:40:592
98729940,4cyclictest0-21swapper/210:20:132
98729940,4cyclictest0-21swapper/210:15:142
98729940,4cyclictest0-21swapper/210:03:332
98729940,4cyclictest0-21swapper/209:50:162
98729940,4cyclictest0-21swapper/209:30:132
98729940,4cyclictest0-21swapper/209:16:452
98729940,4cyclictest0-21swapper/209:10:172
98729940,4cyclictest0-21swapper/207:35:142
98729940,3cyclictest29925-21sh11:15:152
98729940,3cyclictest26152-21cat09:40:012
98729940,3cyclictest25889-21sed08:45:002
98729940,3cyclictest20324-21/usr/sbin/munin09:55:232
98729940,3cyclictest18115-21latency_hist07:25:002
98729940,3cyclictest17069-21irqstats08:25:202
98729940,3cyclictest14067-21perl12:20:172
98729940,3cyclictest12579-21ssh11:05:142
98729940,3cyclictest0-21swapper/212:05:162
98729940,0cyclictest0-21swapper/211:25:212
98669944,0cyclictest0-21swapper/112:38:511
98669944,0cyclictest0-21swapper/112:13:151
98669944,0cyclictest0-21swapper/111:48:391
98669944,0cyclictest0-21swapper/111:24:301
98669944,0cyclictest0-21swapper/111:00:561
98669941,2cyclictest0-21swapper/111:35:001
98669941,2cyclictest0-21swapper/110:15:131
98669941,2cyclictest0-21swapper/108:00:111
98669941,1cyclictest0-21swapper/111:15:151
98669940,4cyclictest0-21swapper/111:31:381
98669940,4cyclictest0-21swapper/110:37:481
98669940,4cyclictest0-21swapper/110:37:481
98669940,4cyclictest0-21swapper/110:34:531
98669940,4cyclictest0-21swapper/110:07:411
98669940,0cyclictest0-21swapper/112:23:351
9141240,0sleep10-21swapper/109:10:581
25954240,0sleep10-21swapper/109:58:051
25890240,0sleep10-21swapper/111:50:031
98769932,1cyclictest740-21snmpd09:00:103
98769932,1cyclictest740-21snmpd08:40:093
98769931,1cyclictest740-21snmpd09:45:083
98769930,3cyclictest9005-21ssh09:30:103
98769930,3cyclictest30918-21/usr/sbin/munin08:55:133
98769930,3cyclictest21014-21apt-get08:35:133
98769930,3cyclictest15759-21dpkg07:20:023
98769930,3cyclictest10490-21df07:10:203
98769930,3cyclictest0-21swapper/308:25:043
98769930,3cyclictest0-21swapper/308:20:153
98769930,3cyclictest0-21swapper/308:05:143
98769930,3cyclictest0-21swapper/307:55:143
98769930,3cyclictest0-21swapper/307:50:183
98769930,3cyclictest0-21swapper/307:30:153
98729932,1cyclictest740-21snmpd12:25:062
98729932,1cyclictest740-21snmpd08:05:042
98729932,1cyclictest13083-21sh07:15:022
98729930,3cyclictest9295-21smartctl08:10:212
98729930,3cyclictest8594-21rm11:40:032
98729930,3cyclictest6919-21perl11:20:172
98729930,3cyclictest5887-21diskmemload10:25:072
98729930,3cyclictest3640-21/usr/sbin/munin09:05:232
98729930,3cyclictest30826-21apt-config07:50:002
98729930,3cyclictest25985-21ssh11:50:072
98729930,3cyclictest19821-21ntp_states08:30:232
98729930,3cyclictest19052-21perl07:25:182
98729930,3cyclictest10692-21apt-get08:15:012
98729930,3cyclictest0-21swapper/212:10:152
98729930,3cyclictest0-21swapper/211:45:142
98729930,3cyclictest0-21swapper/211:00:032
98729930,3cyclictest0-21swapper/209:25:062
98729930,3cyclictest0-21swapper/209:00:132
98729930,3cyclictest0-21swapper/208:55:132
98729930,3cyclictest0-21swapper/208:50:132
98729930,3cyclictest0-21swapper/208:35:132
98729930,3cyclictest0-21swapper/208:20:082
98729930,3cyclictest0-21swapper/207:55:142
98729930,3cyclictest0-21swapper/207:45:162
98729930,3cyclictest0-21swapper/207:40:042
98729930,3cyclictest0-21swapper/207:30:142
98729930,3cyclictest0-21swapper/207:10:182
98729930,2cyclictest26556-21fschecks_count08:45:182
98669933,0cyclictest0-21swapper/112:07:161
98669933,0cyclictest0-21swapper/109:00:171
98669933,0cyclictest0-21swapper/107:29:561
98669931,1cyclictest0-21swapper/112:30:131
98669931,1cyclictest0-21swapper/111:40:111
98669931,1cyclictest0-21swapper/111:10:111
98669931,1cyclictest0-21swapper/110:50:121
98669931,1cyclictest0-21swapper/110:00:121
98669931,1cyclictest0-21swapper/109:50:151
98669931,1cyclictest0-21swapper/109:30:131
98669931,1cyclictest0-21swapper/109:15:141
98669931,1cyclictest0-21swapper/109:05:151
98669931,1cyclictest0-21swapper/108:45:151
98669931,1cyclictest0-21swapper/108:40:141
98669931,1cyclictest0-21swapper/108:15:011
98669931,1cyclictest0-21swapper/107:55:131
98669931,1cyclictest0-21swapper/107:50:181
98669931,1cyclictest0-21swapper/107:40:171
98669931,1cyclictest0-21swapper/107:35:141
98669931,1cyclictest0-21swapper/107:30:141
98669931,1cyclictest0-21swapper/107:20:181
98669931,1cyclictest0-21swapper/107:15:161
98669931,1cyclictest0-21swapper/107:10:191
98669931,0cyclictest0-21swapper/112:15:121
98669930,2cyclictest25689-21chrt09:39:481
98669930,2cyclictest0-21swapper/112:00:161
98669930,2cyclictest0-21swapper/111:25:141
98669930,2cyclictest0-21swapper/111:05:131
98669930,1cyclictest0-21swapper/110:45:161
98669930,1cyclictest0-21swapper/110:40:101
98669930,1cyclictest0-21swapper/109:45:141
2548230,0sleep10-21swapper/110:21:201
98669921,0cyclictest0-21swapper/110:55:201
98669920,2cyclictest204412sleep108:30:531
98669920,1cyclictest304192sleep107:48:461
98669920,1cyclictest0-21swapper/110:25:461
98669920,1cyclictest0-21swapper/109:40:171
98669920,1cyclictest0-21swapper/108:55:121
98669920,1cyclictest0-21swapper/108:35:121
98669920,1cyclictest0-21swapper/108:20:151
98669920,1cyclictest0-21swapper/108:10:171
98669920,1cyclictest0-21swapper/108:05:141
98669920,0cyclictest0-21swapper/108:25:141
98669910,1cyclictest0-21swapper/108:50:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional