You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-05-12 - 12:22

Intel(R) Core(TM) i5-6440EQ CPU @ 2.70GHz, Linux 4.9.47-rt37 (Profile)

Latency plot of system in rack #5, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot2.osadl.org (updated Tue May 11, 2021 12:44:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
293602235221,10sleep10-21swapper/107:06:531
295872198181,12sleep30-21swapper/307:09:473
293592198181,12sleep00-21swapper/007:06:520
295542186168,13sleep20-21swapper/207:09:222
150942490,0sleep00-21swapper/012:11:550
77222480,0sleep10-21swapper/109:18:041
232492480,0sleep10-21swapper/112:35:211
45772450,0sleep10-21swapper/110:50:161
141442150,0sleep20-21swapper/210:36:522
22342130,0sleep20-21swapper/210:49:292
127322110,0sleep10-21swapper/111:51:371
2977999102,4cyclictest18859-21munin-run09:25:000
110842100,0sleep10-21swapper/112:29:021
297949970,5cyclictest5309-21ssh11:10:133
297799970,3cyclictest9739-21munin-run10:35:000
297799962,4cyclictest30515-21apt-get11:25:010
297799960,3cyclictest15736-21munin-run07:45:000
24562260,0sleep00-21swapper/009:46:130
11584260,0sleep10-21swapper/109:20:161
297949955,0cyclictest0-21swapper/312:08:303
297949950,5cyclictest0-21swapper/311:50:103
297949950,4cyclictest8298-21ssh10:15:113
297949950,4cyclictest704-21apt-get07:15:103
297949950,4cyclictest0-21swapper/311:55:013
297899955,0cyclictest11934-21kworker/2:011:13:282
297899955,0cyclictest11934-21kworker/2:009:10:182
297899955,0cyclictest0-21swapper/212:27:382
297899953,1cyclictest31-21ksoftirqd/212:20:112
297899953,1cyclictest31-21ksoftirqd/209:05:112
297899953,1cyclictest21941-21turbostat11:20:002
297899950,5cyclictest0-21swapper/211:22:532
297899950,4cyclictest0-21swapper/211:55:142
297899950,4cyclictest0-21swapper/211:25:102
297859950,5cyclictest0-21swapper/110:09:311
297859950,5cyclictest0-21swapper/109:25:271
297799952,2cyclictest728-21snmpd09:02:300
297799952,2cyclictest728-21snmpd07:51:250
297799950,5cyclictest0-21swapper/009:55:070
297799950,4cyclictest6870-21munin-run11:30:000
297799950,4cyclictest0-21swapper/010:35:140
297799950,4cyclictest0-21swapper/010:10:130
297949944,0cyclictest0-21swapper/311:07:143
297949944,0cyclictest0-21swapper/309:30:203
297949941,3cyclictest728-21snmpd11:39:283
297949940,4cyclictest18435-21apt-get07:50:143
297949940,4cyclictest0-21swapper/312:15:103
297949940,4cyclictest0-21swapper/311:30:103
297949940,4cyclictest0-21swapper/311:22:153
297949940,4cyclictest0-21swapper/310:50:113
297949940,4cyclictest0-21swapper/310:45:173
297949940,4cyclictest0-21swapper/310:31:133
297949940,4cyclictest0-21swapper/310:25:113
297949940,4cyclictest0-21swapper/310:20:123
297949940,4cyclictest0-21swapper/310:10:133
297949940,4cyclictest0-21swapper/309:50:123
297949940,4cyclictest0-21swapper/309:45:163
297949940,4cyclictest0-21swapper/309:28:133
297949940,4cyclictest0-21swapper/309:20:123
297949940,4cyclictest0-21swapper/309:15:143
297949940,4cyclictest0-21swapper/308:30:113
297949940,4cyclictest0-21swapper/308:30:113
297949940,4cyclictest0-21swapper/307:55:153
297949940,4cyclictest0-21swapper/307:25:123
297949940,4cyclictest0-21swapper/307:10:303
297949940,3cyclictest6054-21dpkg09:55:013
297949940,3cyclictest3481-21/usr/sbin/munin08:25:143
297949940,3cyclictest26387-21expr08:05:133
297949940,3cyclictest26193-21ssh12:00:033
297949940,3cyclictest23641-21rm10:42:373
297949940,3cyclictest21358-21/usr/sbin/munin12:35:223
297949940,3cyclictest20531-21ssh11:00:113
297949940,3cyclictest16823-21chrt12:13:143
297949940,3cyclictest15962-21/usr/sbin/munin07:45:223
297949940,3cyclictest13656-21ssh11:15:013
297949940,3cyclictest12621-21ssh12:30:003
297949940,3cyclictest10028-21ssh10:35:043
297949940,0cyclictest0-21swapper/311:41:173
297899944,0cyclictest11934-21kworker/2:012:09:252
297899944,0cyclictest11934-21kworker/2:010:17:292
297899944,0cyclictest11934-21kworker/2:009:43:532
297899944,0cyclictest11934-21kworker/2:009:21:292
297899944,0cyclictest11934-21kworker/2:008:25:322
297899944,0cyclictest11934-21kworker/2:007:40:442
297899944,0cyclictest0-21swapper/210:02:432
297899944,0cyclictest0-21swapper/209:26:032
297899943,1cyclictest459-21nscd09:45:022
297899943,1cyclictest21595-21cut12:35:132
297899943,1cyclictest11934-21kworker/2:012:31:482
297899942,1cyclictest1244-21diskstats07:15:122
297899941,1cyclictest0-21swapper/208:50:102
297899940,4cyclictest7919-21ssh10:33:382
297899940,4cyclictest0-21swapper/211:50:122
297899940,4cyclictest0-21swapper/211:45:112
297899940,4cyclictest0-21swapper/211:45:112
297899940,4cyclictest0-21swapper/211:36:052
297899940,4cyclictest0-21swapper/211:32:302
297899940,4cyclictest0-21swapper/211:05:132
297899940,4cyclictest0-21swapper/210:40:132
297899940,4cyclictest0-21swapper/209:30:122
297899940,4cyclictest0-21swapper/208:35:102
297899940,4cyclictest0-21swapper/208:30:112
297899940,4cyclictest0-21swapper/208:30:112
297899940,3cyclictest5530-21ssh09:35:192
297899940,3cyclictest31346-21df_inode08:15:122
297899940,3cyclictest30735-21hddtemp_smartct07:10:162
297899940,3cyclictest23558-21apt10:05:132
297899940,3cyclictest21227-21perl11:00:142
297899940,3cyclictest19136-21perl08:55:212
297899940,3cyclictest11653-21hddtemp_smartct07:35:162
297899940,3cyclictest0-21swapper/210:25:112
297899940,0cyclictest0-21swapper/212:02:452
297859944,0cyclictest0-21swapper/112:01:521
297859944,0cyclictest0-21swapper/111:28:311
297859944,0cyclictest0-21swapper/111:21:061
297859944,0cyclictest0-21swapper/110:23:291
297859944,0cyclictest0-21swapper/109:34:531
297859944,0cyclictest0-21swapper/108:10:041
297859941,2cyclictest0-21swapper/112:05:111
297859941,2cyclictest0-21swapper/111:35:131
297859941,2cyclictest0-21swapper/109:40:131
297859941,2cyclictest0-21swapper/108:45:111
297859941,2cyclictest0-21swapper/108:20:121
297859941,2cyclictest0-21swapper/108:15:101
297859941,2cyclictest0-21swapper/107:40:121
297859941,2cyclictest0-21swapper/107:25:021
297859941,1cyclictest0-21swapper/107:50:141
297859940,4cyclictest0-21swapper/111:49:201
297859940,4cyclictest0-21swapper/111:49:201
297859940,4cyclictest0-21swapper/110:31:471
297859940,4cyclictest0-21swapper/109:56:171
297859940,0cyclictest0-21swapper/112:13:381
297859940,0cyclictest0-21swapper/110:55:471
297859940,0cyclictest0-21swapper/109:46:401
297799944,0cyclictest0-21swapper/012:26:180
297799944,0cyclictest0-21swapper/011:45:520
297799944,0cyclictest0-21swapper/011:45:510
297799944,0cyclictest0-21swapper/009:10:190
297799942,1cyclictest728-21snmpd09:05:070
297799942,1cyclictest728-21snmpd08:45:060
297799942,1cyclictest27957-21grep09:30:130
297799941,1cyclictest0-21swapper/008:15:100
297799940,4cyclictest22352-21ssh10:41:370
297799940,4cyclictest0-21swapper/011:00:110
297799940,4cyclictest0-21swapper/009:50:120
297799940,3cyclictest728-21snmpd10:04:020
297799940,3cyclictest7112-21sh11:30:020
297799940,3cyclictest32525-21munin-run11:45:000
297799940,3cyclictest3197-21cut07:20:000
297799940,3cyclictest24122-21df10:05:150
297799940,3cyclictest21431-21df_inode07:55:160
297799940,3cyclictest16417-21munin-run10:20:000
297799940,3cyclictest0-21swapper/011:05:010
297799940,3cyclictest0-21swapper/010:45:110
297799940,2cyclictest4798-21ssh12:05:320
297799940,1cyclictest8348-21/usr/sbin/munin07:30:250
297799940,1cyclictest2199-21/usr/sbin/munin09:15:170
297799940,1cyclictest19591-21/usr/sbin/munin12:15:130
297799940,1cyclictest16869-21/usr/sbin/munin10:20:200
297799940,1cyclictest12832-21/usr/sbin/munin12:30:130
297799940,0cyclictest0-21swapper/011:56:110
297799940,0cyclictest0-21swapper/011:38:260
297799940,0cyclictest0-21swapper/011:16:100
297799940,0cyclictest0-21swapper/010:51:350
19840240,0sleep30-21swapper/310:59:533
297949932,1cyclictest728-21snmpd07:40:093
297949932,1cyclictest24017-21tune2fs09:05:143
297949931,2cyclictest30451-21rm12:20:333
297949931,2cyclictest0-21swapper/309:35:023
297949930,3cyclictest30955-21ssh11:25:023
297949930,3cyclictest0-21swapper/312:25:043
297949930,3cyclictest0-21swapper/311:45:123
297949930,3cyclictest0-21swapper/311:45:113
297949930,3cyclictest0-21swapper/310:05:103
297949930,3cyclictest0-21swapper/309:40:133
297949930,3cyclictest0-21swapper/309:10:123
297949930,3cyclictest0-21swapper/309:00:133
297949930,3cyclictest0-21swapper/308:55:133
297949930,3cyclictest0-21swapper/308:50:103
297949930,3cyclictest0-21swapper/308:45:083
297949930,3cyclictest0-21swapper/308:40:113
297949930,3cyclictest0-21swapper/308:35:103
297949930,3cyclictest0-21swapper/308:20:113
297949930,3cyclictest0-21swapper/308:15:103
297949930,3cyclictest0-21swapper/308:10:123
297949930,3cyclictest0-21swapper/308:00:083
297949930,3cyclictest0-21swapper/307:35:123
297949930,3cyclictest0-21swapper/307:30:073
297949930,3cyclictest0-21swapper/307:20:113
297949930,0cyclictest0-21swapper/310:00:133
297899932,1cyclictest30766-21perl09:50:152
297899931,2cyclictest0-21swapper/209:15:042
297899931,1cyclictest0-21swapper/208:40:112
297899931,1cyclictest0-21swapper/207:50:142
297899930,3cyclictest0-21swapper/212:15:042
297899930,3cyclictest0-21swapper/212:10:112
297899930,3cyclictest0-21swapper/211:40:112
297899930,3cyclictest0-21swapper/210:55:192
297899930,3cyclictest0-21swapper/210:21:352
297899930,3cyclictest0-21swapper/209:55:012
297899930,3cyclictest0-21swapper/208:45:012
297899930,3cyclictest0-21swapper/208:20:122
297899930,3cyclictest0-21swapper/208:10:112
297899930,3cyclictest0-21swapper/208:05:122
297899930,3cyclictest0-21swapper/208:00:122
297899930,3cyclictest0-21swapper/207:55:152
297899930,3cyclictest0-21swapper/207:45:102
297899930,3cyclictest0-21swapper/207:30:142
297899930,3cyclictest0-21swapper/207:25:022
297899930,3cyclictest0-21swapper/207:20:112
297899930,2cyclictest4765-21ntp_states10:50:172
297899930,2cyclictest1985-21ssh10:10:332
297899930,1cyclictest0-21swapper/209:00:132
297859933,0cyclictest0-21swapper/110:10:151
297859931,1cyclictest0-21swapper/112:20:111
297859931,1cyclictest0-21swapper/111:55:141
297859931,1cyclictest0-21swapper/111:40:111
297859931,1cyclictest0-21swapper/111:00:111
297859931,1cyclictest0-21swapper/110:15:111
297859931,1cyclictest0-21swapper/109:35:121
297859931,1cyclictest0-21swapper/109:05:121
297859931,1cyclictest0-21swapper/108:55:131
297859931,1cyclictest0-21swapper/108:40:111
297859931,1cyclictest0-21swapper/108:05:121
297859931,1cyclictest0-21swapper/108:00:131
297859931,1cyclictest0-21swapper/107:45:141
297859931,1cyclictest0-21swapper/107:35:121
297859931,1cyclictest0-21swapper/107:10:141
297859930,2cyclictest5711-21chrt09:54:591
297859930,2cyclictest14804-21chrt10:37:231
297859930,2cyclictest0-21swapper/111:30:101
297859930,2cyclictest0-21swapper/111:10:131
297859930,2cyclictest0-21swapper/111:05:121
297859930,2cyclictest0-21swapper/109:10:111
297859930,2cyclictest0-21swapper/108:35:101
297859930,2cyclictest0-21swapper/107:55:151
297859930,1cyclictest0-21swapper/110:25:101
297799933,0cyclictest8084-21apt-get08:35:010
297799932,1cyclictest728-21snmpd08:40:070
297799932,1cyclictest6397-21df07:25:140
297799932,1cyclictest26393-21apt-get12:00:110
297799930,3cyclictest3223-21apt-get07:20:110
297799930,3cyclictest25867-21apt08:05:120
297799930,3cyclictest16463-21diskstats08:50:120
297799930,3cyclictest1427-21diskstats08:20:150
297799930,3cyclictest0-21swapper/011:50:080
297799930,3cyclictest0-21swapper/011:10:130
297799930,3cyclictest0-21swapper/009:40:130
297799930,3cyclictest0-21swapper/009:35:120
297799930,3cyclictest0-21swapper/008:55:010
297799930,3cyclictest0-21swapper/008:30:100
297799930,3cyclictest0-21swapper/008:30:100
297799930,3cyclictest0-21swapper/008:25:110
297799930,3cyclictest0-21swapper/008:10:120
297799930,3cyclictest0-21swapper/008:00:130
297799930,3cyclictest0-21swapper/007:45:140
297799930,3cyclictest0-21swapper/007:35:100
297799930,3cyclictest0-21swapper/007:10:120
297799930,2cyclictest19332-21sh09:25:130
297799930,2cyclictest0-21swapper/010:25:020
297799930,1cyclictest22003-21grep12:35:150
297799930,1cyclictest0-21swapper/012:20:060
297799930,1cyclictest0-21swapper/010:55:040
297859922,0cyclictest0-21swapper/112:15:051
297859921,1cyclictest172722chrt08:50:211
297859921,1cyclictest0-21swapper/110:40:131
297859921,0cyclictest0-21swapper/112:30:111
297859920,2cyclictest8333-21chrt07:30:021
297859920,1cyclictest694-21chrt07:15:061
297859920,1cyclictest0-21swapper/111:15:121
297859920,1cyclictest0-21swapper/110:45:141
297859920,1cyclictest0-21swapper/110:00:121
297859920,1cyclictest0-21swapper/109:00:141
297859920,1cyclictest0-21swapper/108:30:121
297859920,1cyclictest0-21swapper/108:30:111
297859920,1cyclictest0-21swapper/108:25:141
297859920,1cyclictest0-21swapper/107:20:111
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional