You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2022-01-23 - 19:59

Intel(R) Core(TM) i5-6440EQ CPU @ 2.70GHz, Linux 4.9.47-rt37 (Profile)

Latency plot of system in rack #5, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot2.osadl.org (updated Sun Jan 23, 2022 12:44:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
169302218201,12sleep00-21swapper/007:07:260
170382200184,11sleep10-21swapper/107:08:521
170502177160,12sleep30-21swapper/307:09:023
169272124109,11sleep20-21swapper/207:07:252
84902540,0sleep30-21swapper/311:00:183
294292480,0sleep10-21swapper/110:53:321
119572460,0sleep20-21swapper/212:00:142
323202450,0sleep10-21swapper/110:55:201
182892450,0sleep10-21swapper/108:15:011
74732120,0sleep10-21swapper/111:37:581
245042120,0sleep10-21swapper/111:48:041
160202100,0sleep30-21swapper/309:10:293
24361290,0sleep324365-21hwlatdetect10:50:203
173249960,6cyclictest31281-21diskstats07:35:163
173109960,6cyclictest0-21swapper/012:29:200
173109960,3cyclictest6025-21perf12:35:000
173109960,3cyclictest17219-21munin-run11:25:000
173249954,1cyclictest0-21swapper/311:21:053
173249950,5cyclictest9521-21ssh11:39:353
173249950,5cyclictest0-21swapper/310:35:423
173249950,5cyclictest0-21swapper/309:52:033
173249950,5cyclictest0-21swapper/309:46:053
173249950,5cyclictest0-21swapper/309:19:373
173249950,4cyclictest22006-21ssh10:30:173
173249950,4cyclictest0-21swapper/311:10:173
173209950,5cyclictest0-21swapper/212:36:412
173209950,5cyclictest0-21swapper/212:33:112
173209950,5cyclictest0-21swapper/212:10:032
173209950,5cyclictest0-21swapper/211:44:272
173209950,4cyclictest2920-21perl11:35:202
173209950,3cyclictest28999-21ssh12:29:142
173209950,3cyclictest16712-21cpuspeed_turbos09:30:202
173159951,2cyclictest0-21swapper/109:20:171
173159950,5cyclictest0-21swapper/111:32:451
173159950,0cyclictest0-21swapper/110:20:261
173109952,3cyclictest4700-21ssh10:57:550
173109952,3cyclictest21858-21ssh09:51:470
173109952,3cyclictest0-21swapper/012:22:320
173109952,3cyclictest0-21swapper/011:41:470
173109952,3cyclictest0-21swapper/011:17:110
173109952,3cyclictest0-21swapper/011:14:200
173109952,3cyclictest0-21swapper/011:03:280
173109952,3cyclictest0-21swapper/010:42:480
173109952,3cyclictest0-21swapper/010:32:200
173109952,3cyclictest0-21swapper/009:46:520
173109951,4cyclictest18157-21idleruntime-cro08:15:000
173109950,5cyclictest5760-21ssh11:55:360
173109950,5cyclictest22789-21ssh10:10:520
173109950,5cyclictest22369-21ssh11:08:320
173109950,5cyclictest0-21swapper/012:39:070
173109950,5cyclictest0-21swapper/012:15:270
173109950,5cyclictest0-21swapper/012:07:110
173109950,5cyclictest0-21swapper/012:00:080
173109950,5cyclictest0-21swapper/011:38:520
173109950,5cyclictest0-21swapper/011:30:200
173109950,5cyclictest0-21swapper/010:29:190
173109950,5cyclictest0-21swapper/010:21:110
173109950,5cyclictest0-21swapper/010:17:080
173109950,5cyclictest0-21swapper/010:02:400
173109950,5cyclictest0-21swapper/009:38:400
173109950,5cyclictest0-21swapper/009:32:430
173109950,5cyclictest0-21swapper/009:25:510
173109950,5cyclictest0-21swapper/009:24:360
173109950,5cyclictest0-21swapper/009:12:280
173109950,5cyclictest0-21swapper/007:30:240
173109950,4cyclictest0-21swapper/007:20:170
173109950,1cyclictest0-21swapper/011:47:270
173109950,1cyclictest0-21swapper/008:35:230
173109950,0cyclictest3274-21rm12:14:070
173109950,0cyclictest3004-21rm09:40:320
173109950,0cyclictest13884-21ssh10:05:320
173109950,0cyclictest0-21swapper/011:25:240
9780240,0sleep30-21swapper/312:36:233
26624240,0sleep10-21swapper/112:27:241
25259240,0sleep30-21swapper/307:21:123
173249944,0cyclictest0-21swapper/310:10:503
173249944,0cyclictest0-21swapper/309:20:553
173249943,1cyclictest3614-21munin-run08:50:003
173249943,1cyclictest17916-21grep09:30:263
173249942,1cyclictest22044-21ssh12:05:283
173249942,1cyclictest1586-21ssh09:40:223
173249941,3cyclictest8143-21sh12:16:193
173249941,2cyclictest0-21swapper/310:45:173
173249940,4cyclictest31708-21kworker/3:111:05:223
173249940,4cyclictest30303-21ssh11:51:023
173249940,4cyclictest26907-21memory07:25:233
173249940,4cyclictest0-21swapper/312:20:183
173249940,4cyclictest0-21swapper/312:10:153
173249940,4cyclictest0-21swapper/312:00:183
173249940,4cyclictest0-21swapper/311:15:193
173249940,4cyclictest0-21swapper/310:55:173
173249940,4cyclictest0-21swapper/308:25:183
173249940,4cyclictest0-21swapper/308:00:173
173249940,3cyclictest4786-21ssh10:20:033
173249940,3cyclictest4183-21hddtemp_smartct07:45:163
173249940,3cyclictest26072-21ssh11:30:123
173249940,3cyclictest19936-21df07:15:003
173249940,3cyclictest19290-21ssh11:45:193
173249940,3cyclictest1673-21ssh10:17:593
173249940,3cyclictest1132-21latency_hist08:44:593
173249940,3cyclictest10742-21sendmail-msp08:00:013
173249940,3cyclictest10625-21apt-get11:40:153
173249940,3cyclictest1062-21ssh09:58:593
173249940,3cyclictest0-21swapper/309:25:033
173249940,1cyclictest19660-21memory08:15:243
173209944,0cyclictest0-21swapper/211:18:242
173209944,0cyclictest0-21swapper/209:29:162
173209944,0cyclictest0-21swapper/208:34:592
173209942,2cyclictest3016-21kworker/2:109:50:042
173209942,1cyclictest12700-21ssh09:46:222
173209941,3cyclictest0-21swapper/210:20:182
173209940,4cyclictest4501-21ssh10:38:512
173209940,4cyclictest28283-21/usr/sbin/munin07:30:282
173209940,4cyclictest26500-21apt-get09:55:022
173209940,4cyclictest0-21swapper/210:18:452
173209940,4cyclictest0-21swapper/209:08:462
173209940,3cyclictest31491-21perl07:35:172
173209940,3cyclictest30529-21sh11:32:162
173209940,3cyclictest14726-21memory10:25:242
173209940,1cyclictest7909-21/usr/sbin/munin11:00:222
173209940,1cyclictest21966-21ntp_states12:05:272
173209940,1cyclictest20852-21/usr/sbin/munin07:15:212
173209940,1cyclictest17425-21/usr/sbin/munin07:10:262
173209940,1cyclictest16396-21/usr/sbin/munin11:05:282
173209940,1cyclictest15744-21tune2fs10:45:202
173209940,1cyclictest13483-21/usr/sbin/munin12:20:242
173209940,0cyclictest0-21swapper/210:00:322
173159944,0cyclictest0-21swapper/112:36:101
173159944,0cyclictest0-21swapper/112:05:111
173159944,0cyclictest0-21swapper/111:17:371
173159944,0cyclictest0-21swapper/111:05:581
173159944,0cyclictest0-21swapper/110:48:561
173159944,0cyclictest0-21swapper/110:27:511
173159944,0cyclictest0-21swapper/109:16:071
173159944,0cyclictest0-21swapper/108:45:241
173159941,2cyclictest0-21swapper/112:00:171
173159941,2cyclictest0-21swapper/111:55:171
173159941,2cyclictest0-21swapper/111:20:171
173159941,2cyclictest0-21swapper/110:40:171
173159941,2cyclictest0-21swapper/110:35:181
173159941,2cyclictest0-21swapper/110:30:171
173159941,2cyclictest0-21swapper/109:05:181
173159941,2cyclictest0-21swapper/108:30:171
173159941,1cyclictest0-21swapper/112:15:181
173159940,4cyclictest0-21swapper/109:26:451
173159940,4cyclictest0-21swapper/108:00:171
173159940,0cyclictest0-21swapper/109:42:451
173109944,0cyclictest0-21swapper/010:36:330
173109942,2cyclictest0-21swapper/009:55:430
173109942,2cyclictest0-21swapper/007:25:200
173109942,1cyclictest740-21snmpd09:03:040
173109941,3cyclictest25496-21ssh09:15:520
173109941,3cyclictest0-21swapper/008:15:280
173109941,2cyclictest0-21swapper/010:45:170
173109941,2cyclictest0-21swapper/008:45:210
173109940,4cyclictest0-21swapper/011:50:240
173109940,4cyclictest0-21swapper/010:50:430
173109940,4cyclictest0-21swapper/008:30:010
173109940,4cyclictest0-21swapper/008:05:190
173109940,4cyclictest0-21swapper/007:10:160
173109940,3cyclictest21061-21/usr/sbin/munin08:20:220
173109940,3cyclictest0-21swapper/008:55:160
173249931,1cyclictest740-21snmpd10:25:173
173249931,1cyclictest0-21swapper/309:00:173
173249931,1cyclictest0-21swapper/308:50:183
173249931,1cyclictest0-21swapper/308:05:183
173249930,3cyclictest7592-21ntpq08:55:253
173249930,3cyclictest3225-21ssh11:55:023
173249930,3cyclictest30351-21ssh12:30:093
173249930,3cyclictest29776-21awk07:30:283
173249930,3cyclictest2904-21ssh10:00:133
173249930,3cyclictest13514-21diskmemload11:25:123
173249930,3cyclictest11289-21apt09:05:173
173249930,3cyclictest0-21swapper/312:25:013
173249930,3cyclictest0-21swapper/310:40:073
173249930,3cyclictest0-21swapper/310:05:023
173249930,3cyclictest0-21swapper/309:35:023
173249930,3cyclictest0-21swapper/308:35:163
173249930,3cyclictest0-21swapper/308:30:013
173249930,3cyclictest0-21swapper/308:20:183
173249930,3cyclictest0-21swapper/308:10:183
173249930,3cyclictest0-21swapper/307:50:023
173249930,3cyclictest0-21swapper/307:40:113
173249930,3cyclictest0-21swapper/307:10:003
173209933,0cyclictest3016-21kworker/2:111:20:122
173209933,0cyclictest3016-21kworker/2:110:50:122
173209933,0cyclictest3016-21kworker/2:110:05:152
173209933,0cyclictest3016-21kworker/2:108:35:042
173209933,0cyclictest3016-21kworker/2:108:05:082
173209932,1cyclictest740-21snmpd10:40:122
173209932,1cyclictest31619-21ssh10:55:042
173209932,1cyclictest24549-21ssh11:10:042
173209932,1cyclictest24271-21awk07:20:212
173209932,1cyclictest16278-21ls08:10:212
173209931,2cyclictest0-21swapper/208:50:182
173209931,1cyclictest0-21swapper/209:00:172
173209931,1cyclictest0-21swapper/208:20:172
173209931,1cyclictest0-21swapper/207:50:132
173209930,3cyclictest4637-21irqstats11:55:212
173209930,3cyclictest25436-21diskstats09:35:202
173209930,3cyclictest19269-21perl08:15:222
173209930,3cyclictest18340-21df_inode11:25:192
173209930,3cyclictest11160-21grep08:00:192
173209930,3cyclictest0-21swapper/212:15:182
173209930,3cyclictest0-21swapper/211:45:092
173209930,3cyclictest0-21swapper/210:30:052
173209930,3cyclictest0-21swapper/209:20:182
173209930,3cyclictest0-21swapper/209:10:082
173209930,3cyclictest0-21swapper/208:55:172
173209930,3cyclictest0-21swapper/208:45:002
173209930,3cyclictest0-21swapper/208:40:022
173209930,3cyclictest0-21swapper/208:25:012
173209930,3cyclictest0-21swapper/207:55:012
173209930,3cyclictest0-21swapper/207:45:142
173209930,3cyclictest0-21swapper/207:40:022
173209930,3cyclictest0-21swapper/207:25:152
173209930,2cyclictest27347-21sh11:50:082
173209930,1cyclictest22783-21ssh09:15:112
173209930,1cyclictest20042-21ssh10:10:092
173209930,1cyclictest0-21swapper/209:40:102
173159933,0cyclictest0-21swapper/112:23:321
173159933,0cyclictest0-21swapper/112:10:081
173159933,0cyclictest0-21swapper/109:31:241
173159931,1cyclictest0-21swapper/111:10:161
173159931,1cyclictest0-21swapper/110:10:171
173159931,1cyclictest0-21swapper/110:05:021
173159931,1cyclictest0-21swapper/109:50:181
173159931,1cyclictest0-21swapper/108:50:171
173159931,1cyclictest0-21swapper/108:40:021
173159931,1cyclictest0-21swapper/107:50:131
173159931,1cyclictest0-21swapper/107:45:141
173159931,1cyclictest0-21swapper/107:40:121
173159931,1cyclictest0-21swapper/107:30:171
173159931,1cyclictest0-21swapper/107:10:171
173159930,3cyclictest0-21swapper/107:15:031
173159930,2cyclictest5628-21chrt10:00:551
173159930,2cyclictest2391-21chrt10:18:321
173159930,2cyclictest15429-21chrt11:42:361
173159930,2cyclictest0-21swapper/111:25:171
173159930,2cyclictest0-21swapper/108:05:171
173109931,1cyclictest0-21swapper/008:50:180
173109930,3cyclictest0-21swapper/009:05:020
173109930,3cyclictest0-21swapper/008:40:180
173109930,3cyclictest0-21swapper/008:25:180
173109930,3cyclictest0-21swapper/008:00:170
173109930,3cyclictest0-21swapper/007:55:240
173109930,3cyclictest0-21swapper/007:50:130
173109930,3cyclictest0-21swapper/007:45:130
173109930,3cyclictest0-21swapper/007:40:110
173109930,3cyclictest0-21swapper/007:35:000
173109930,3cyclictest0-21swapper/007:15:190
12335230,0sleep10-21swapper/109:46:061
173159921,1cyclictest233452chrt07:20:171
173159921,1cyclictest0-21swapper/112:30:141
173159921,1cyclictest0-21swapper/108:55:171
173159921,1cyclictest0-21swapper/108:20:181
173159921,1cyclictest0-21swapper/108:15:191
173159920,1cyclictest7925-21chrt11:00:161
173159920,1cyclictest308322sleep108:39:351
173159920,1cyclictest0-21swapper/111:50:021
173159920,1cyclictest0-21swapper/109:55:171
173159920,1cyclictest0-21swapper/109:35:191
173159920,1cyclictest0-21swapper/109:10:171
173159920,1cyclictest0-21swapper/109:00:181
173159920,1cyclictest0-21swapper/107:55:021
173159920,1cyclictest0-21swapper/107:35:131
173159910,1cyclictest0-21swapper/108:25:001
173159910,1cyclictest0-21swapper/107:25:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional