You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-07-26 - 16:16

Intel(R) Core(TM) i5-6440EQ CPU @ 2.70GHz, Linux 4.9.47-rt37 (Profile)

Latency plot of system in rack #5, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the alll highest latencies:
System rack5slot2.osadl.org (updated Fri Jun 11, 2021 12:44:39)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
101792189172,12sleep20-21swapper/207:09:542
99472183167,11sleep10-21swapper/107:06:521
101772181164,12sleep00-21swapper/007:09:530
101232180164,12sleep30-21swapper/307:09:103
69692100,0sleep30-21swapper/312:36:103
314072100,0sleep10-21swapper/110:20:131
221982100,0sleep10-21swapper/109:37:261
31097290,0sleep00-21swapper/012:31:230
103799972,5cyclictest12921-21kworker/3:011:41:123
103799971,6cyclictest12921-21kworker/3:011:47:563
103639971,3cyclictest30184-21munin-run11:35:010
103799962,4cyclictest12921-21kworker/3:012:14:463
103799961,5cyclictest12921-21kworker/3:011:50:033
103709961,3cyclictest0-21swapper/107:45:161
103639960,6cyclictest0-21swapper/010:42:300
103639960,5cyclictest28329-21munin-run08:50:010
103639960,3cyclictest6394-21munin-run11:40:000
11504250,0sleep3391ktimersoftd/310:26:593
103799955,0cyclictest0-21swapper/312:34:013
103799952,3cyclictest12921-21kworker/3:012:18:463
103799951,3cyclictest728-21snmpd11:15:363
103799950,5cyclictest0-21swapper/311:20:063
103799950,5cyclictest0-21swapper/311:06:353
103799950,5cyclictest0-21swapper/310:10:173
103799950,4cyclictest0-21swapper/311:10:193
103799950,4cyclictest0-21swapper/311:00:143
103799950,4cyclictest0-21swapper/309:45:133
103799950,4cyclictest0-21swapper/309:25:143
103799950,4cyclictest0-21swapper/309:10:143
103749955,0cyclictest5020-21kworker/2:211:55:362
103749955,0cyclictest5020-21kworker/2:209:18:522
103749952,2cyclictest11739-21ssh12:20:142
103749950,5cyclictest0-21swapper/211:30:522
103749950,5cyclictest0-21swapper/211:24:092
103749950,5cyclictest0-21swapper/210:30:012
103749950,5cyclictest0-21swapper/209:33:262
103749950,4cyclictest0-21swapper/212:25:122
103749950,4cyclictest0-21swapper/212:10:232
103749950,4cyclictest0-21swapper/211:05:172
103709951,2cyclictest0-21swapper/111:25:161
103709951,2cyclictest0-21swapper/108:15:101
103709950,5cyclictest0-21swapper/111:11:441
103709950,5cyclictest0-21swapper/111:07:561
103709950,5cyclictest0-21swapper/109:51:241
103639955,0cyclictest0-21swapper/011:06:570
103639951,4cyclictest19348-21munin-run09:55:010
103639950,5cyclictest23872-21/usr/sbin/munin09:20:230
103639950,5cyclictest0-21swapper/009:17:150
103639950,4cyclictest7098-21apt-get10:25:000
103639950,4cyclictest20264-21apt11:10:190
103639950,4cyclictest18239-21grep12:05:210
6444240,0sleep10-21swapper/110:05:201
103799944,0cyclictest0-21swapper/311:34:143
103799944,0cyclictest0-21swapper/308:45:173
103799942,2cyclictest12921-21kworker/3:012:26:013
103799942,2cyclictest12921-21kworker/3:012:02:233
103799942,2cyclictest12921-21kworker/3:011:26:593
103799942,2cyclictest12921-21kworker/3:010:50:283
103799942,2cyclictest12921-21kworker/3:010:20:083
103799942,2cyclictest12921-21kworker/3:010:17:033
103799942,2cyclictest12921-21kworker/3:010:01:143
103799942,2cyclictest12921-21kworker/3:009:32:583
103799942,2cyclictest12921-21kworker/3:009:22:213
103799942,1cyclictest0-21swapper/310:55:243
103799941,3cyclictest12921-21kworker/3:008:50:213
103799941,3cyclictest12921-21kworker/3:008:10:233
103799941,3cyclictest12921-21kworker/3:007:20:193
103799941,2cyclictest0-21swapper/308:40:133
103799940,4cyclictest967-21perl11:55:173
103799940,4cyclictest0-21swapper/312:20:103
103799940,4cyclictest0-21swapper/311:35:153
103799940,4cyclictest0-21swapper/310:40:113
103799940,3cyclictest728-21snmpd07:55:223
103799940,3cyclictest3688-21basename08:00:223
103799940,3cyclictest29258-21tune2fs07:45:213
103799940,3cyclictest23621-21ssh12:08:373
103799940,3cyclictest20717-21/usr/sbin/munin09:55:223
103799940,3cyclictest18471-21ssh09:16:203
103799940,3cyclictest18438-21ssh10:30:413
103799940,3cyclictest17333-21ssh09:35:083
103799940,3cyclictest10984-21ssh09:50:063
103799940,3cyclictest0-21swapper/310:45:103
103799940,3cyclictest0-21swapper/310:35:113
103799940,3cyclictest0-21swapper/310:05:123
103799940,3cyclictest0-21swapper/309:00:193
103799940,3cyclictest0-21swapper/308:30:133
103799940,0cyclictest0-21swapper/309:42:293
103749944,0cyclictest5020-21kworker/2:210:48:292
103749944,0cyclictest5020-21kworker/2:208:22:542
103749944,0cyclictest0-21swapper/210:52:262
103749944,0cyclictest0-21swapper/209:47:422
103749944,0cyclictest0-21swapper/208:25:022
103749942,2cyclictest0-21swapper/210:15:122
103749942,1cyclictest23689-21ssh11:11:332
103749942,1cyclictest15253-21python07:15:242
103749941,2cyclictest27861-21sed07:40:252
103749941,2cyclictest0-21swapper/211:50:142
103749940,4cyclictest9355-21ssh11:03:452
103749940,4cyclictest6882-21apt-get11:40:142
103749940,4cyclictest0-21swapper/209:55:152
103749940,4cyclictest0-21swapper/209:10:132
103749940,4cyclictest0-21swapper/208:37:322
103749940,4cyclictest0-21swapper/208:05:142
103749940,3cyclictest805-21apt-get10:40:112
103749940,3cyclictest7089-21ssh10:25:002
103749940,3cyclictest5710-21hddtemp_smartct10:05:152
103749940,3cyclictest4156-21diskstats08:00:252
103749940,3cyclictest3993-21/usr/sbin/munin12:35:262
103749940,3cyclictest3759-21apt-get09:05:142
103749940,3cyclictest32017-21irqstats08:55:192
103749940,3cyclictest29797-21needreboot10:00:162
103749940,3cyclictest24974-21sh12:09:412
103749940,3cyclictest24139-21diskstats08:40:142
103749940,3cyclictest18689-21memory09:35:202
103749940,3cyclictest1582-21ssh10:59:242
103749940,3cyclictest0-21swapper/211:25:162
103749940,3cyclictest0-21swapper/209:50:122
103749940,3cyclictest0-21swapper/209:25:152
103749940,3cyclictest0-21swapper/209:20:152
103749940,3cyclictest0-21swapper/207:35:152
103709944,0cyclictest0-21swapper/111:55:141
103709944,0cyclictest0-21swapper/111:32:411
103709944,0cyclictest0-21swapper/110:14:151
103709944,0cyclictest0-21swapper/109:30:021
103709941,2cyclictest0-21swapper/112:35:191
103709941,2cyclictest0-21swapper/112:25:111
103709941,2cyclictest0-21swapper/112:20:091
103709941,2cyclictest0-21swapper/111:40:141
103709941,2cyclictest0-21swapper/110:35:101
103709941,2cyclictest0-21swapper/110:30:111
103709941,2cyclictest0-21swapper/109:25:151
103709941,2cyclictest0-21swapper/108:30:141
103709941,2cyclictest0-21swapper/108:20:171
103709941,2cyclictest0-21swapper/107:50:131
103709941,2cyclictest0-21swapper/107:40:151
103709941,2cyclictest0-21swapper/107:15:131
103709940,4cyclictest0-21swapper/111:47:411
103709940,4cyclictest0-21swapper/107:30:251
103639944,0cyclictest3-21ksoftirqd/007:15:010
103639944,0cyclictest0-21swapper/012:18:580
103639944,0cyclictest0-21swapper/012:13:430
103639944,0cyclictest0-21swapper/012:03:590
103639944,0cyclictest0-21swapper/011:19:440
103639944,0cyclictest0-21swapper/010:01:100
103639944,0cyclictest0-21swapper/009:55:300
103639944,0cyclictest0-21swapper/009:47:390
103639944,0cyclictest0-21swapper/009:32:390
103639944,0cyclictest0-21swapper/009:11:430
103639943,1cyclictest2648-21pmu-power09:00:270
103639943,1cyclictest23403-21perf07:35:000
103639942,1cyclictest26015-21ssh09:40:070
103639940,4cyclictest0-21swapper/011:55:140
103639940,4cyclictest0-21swapper/008:55:150
103639940,3cyclictest6789-21df08:05:170
103639940,3cyclictest30418-21wc10:20:010
103639940,3cyclictest29073-21df08:50:140
103639940,3cyclictest24571-21sh10:35:100
103639940,3cyclictest19257-21/usr/sbin/munin12:25:180
103639940,3cyclictest19170-21df08:30:150
103639940,3cyclictest16700-21df08:25:160
103639940,3cyclictest15993-21df_inode11:45:140
103639940,3cyclictest10381-21sh12:20:010
103639940,2cyclictest0-21swapper/007:45:170
103639940,1cyclictest31058-21/usr/sbin/munin07:50:210
103639940,1cyclictest2422-21idleruntime-cro11:00:010
103639940,1cyclictest13500-21/usr/sbin/munin10:10:260
103639940,0cyclictest0-21swapper/012:37:260
103639940,0cyclictest0-21swapper/011:51:110
103639940,0cyclictest0-21swapper/010:47:530
4575230,0sleep10-21swapper/109:05:191
11280230,0sleep10-21swapper/111:23:401
103799932,1cyclictest728-21snmpd07:25:113
103799932,1cyclictest12921-21kworker/3:007:40:003
103799932,1cyclictest12921-21kworker/3:007:35:023
103799931,2cyclictest12921-21kworker/3:008:20:113
103799931,2cyclictest12921-21kworker/3:008:15:093
103799930,3cyclictest12921-21kworker/3:008:35:003
103799930,3cyclictest0-21swapper/309:05:143
103799930,3cyclictest0-21swapper/308:55:143
103799930,3cyclictest0-21swapper/308:25:143
103799930,3cyclictest0-21swapper/308:05:143
103799930,3cyclictest0-21swapper/307:50:143
103799930,3cyclictest0-21swapper/307:15:143
103799930,3cyclictest0-21swapper/307:10:163
103799930,1cyclictest0-21swapper/307:30:123
103749932,1cyclictest9543-21head08:10:172
103749932,1cyclictest21694-21diskstats07:30:162
103749932,1cyclictest16766-21cat07:20:182
103749931,1cyclictest0-21swapper/212:15:152
103749931,1cyclictest0-21swapper/211:35:162
103749931,1cyclictest0-21swapper/210:10:162
103749930,3cyclictest6005-21diskmemload10:20:012
103749930,3cyclictest11212-21apt-get08:15:092
103749930,3cyclictest0-21swapper/212:30:122
103749930,3cyclictest0-21swapper/212:00:132
103749930,3cyclictest0-21swapper/211:45:112
103749930,3cyclictest0-21swapper/211:15:122
103749930,3cyclictest0-21swapper/210:35:102
103749930,3cyclictest0-21swapper/209:40:122
103749930,3cyclictest0-21swapper/209:00:202
103749930,3cyclictest0-21swapper/208:50:112
103749930,3cyclictest0-21swapper/208:45:002
103749930,3cyclictest0-21swapper/208:30:112
103749930,3cyclictest0-21swapper/207:55:092
103749930,3cyclictest0-21swapper/207:50:142
103749930,3cyclictest0-21swapper/207:45:162
103749930,3cyclictest0-21swapper/207:25:222
103749930,3cyclictest0-21swapper/207:10:042
103709933,0cyclictest0-21swapper/109:57:501
103709933,0cyclictest0-21swapper/107:35:041
103709931,2cyclictest0-21swapper/110:50:111
103709931,2cyclictest0-21swapper/107:55:141
103709931,1cyclictest0-21swapper/112:30:121
103709931,1cyclictest0-21swapper/111:50:151
103709931,1cyclictest0-21swapper/111:35:151
103709931,1cyclictest0-21swapper/110:55:241
103709931,1cyclictest0-21swapper/110:40:101
103709931,1cyclictest0-21swapper/110:25:011
103709931,1cyclictest0-21swapper/109:45:111
103709931,1cyclictest0-21swapper/109:20:141
103709931,1cyclictest0-21swapper/108:35:181
103709931,1cyclictest0-21swapper/108:00:221
103709931,1cyclictest0-21swapper/107:20:141
103709930,2cyclictest29763-21chrt08:50:201
103709930,2cyclictest0-21swapper/112:10:231
103709930,2cyclictest0-21swapper/110:00:111
103709930,2cyclictest0-21swapper/108:05:151
103709930,2cyclictest0-21swapper/107:25:221
103639933,0cyclictest3-21ksoftirqd/010:20:140
103639933,0cyclictest0-21swapper/008:40:190
103639932,1cyclictest728-21snmpd07:10:050
103639931,1cyclictest24341-21fschecks_time07:35:170
103639931,1cyclictest18816-21fw_packets10:50:130
103639930,3cyclictest16505-21df_inode10:30:130
103639930,3cyclictest0-21swapper/011:25:050
103639930,3cyclictest0-21swapper/009:35:220
103639930,3cyclictest0-21swapper/009:25:030
103639930,3cyclictest0-21swapper/009:05:150
103639930,3cyclictest0-21swapper/008:35:040
103639930,3cyclictest0-21swapper/008:20:180
103639930,3cyclictest0-21swapper/008:15:100
103639930,3cyclictest0-21swapper/008:10:140
103639930,3cyclictest0-21swapper/008:00:100
103639930,3cyclictest0-21swapper/007:40:140
103639930,3cyclictest0-21swapper/007:20:150
103639930,1cyclictest5370-21df_inode10:05:140
103639930,1cyclictest4881-21/usr/sbin/munin11:20:140
103639930,1cyclictest2904-21ssh11:00:130
103639930,1cyclictest1094-21seq07:55:060
103639930,0cyclictest19410-21perl07:25:300
103639930,0cyclictest0-21swapper/011:40:180
103709921,1cyclictest0-21swapper/112:15:161
103709921,1cyclictest0-21swapper/111:15:511
103709921,1cyclictest0-21swapper/109:40:121
103709921,1cyclictest0-21swapper/109:10:131
103709921,1cyclictest0-21swapper/108:40:131
103709921,1cyclictest0-21swapper/108:10:141
103709921,1cyclictest0-21swapper/107:10:161
103709921,0cyclictest0-21swapper/109:00:021
103709920,1cyclictest2891-21chrt11:00:121
103709920,1cyclictest12469-21chrt10:46:081
103709920,1cyclictest0-21swapper/112:05:131
103709920,1cyclictest0-21swapper/112:00:121
103709920,1cyclictest0-21swapper/110:15:111
103709920,1cyclictest0-21swapper/109:15:161
103709920,1cyclictest0-21swapper/108:55:131
103709920,1cyclictest0-21swapper/108:45:121
103709920,1cyclictest0-21swapper/108:25:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional