You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-10-04 - 11:43

x86 Intel Core i5-6440EQ @2700 MHz, Linux 4.9.47-rt37 (Profile)

Latency plot of system in rack #5, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot2.osadl.org (updated Tue Oct 04, 2022 00:44:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
308682226209,12sleep10-21swapper/119:09:441
308752193176,12sleep30-21swapper/319:09:513
305922189171,13sleep00-21swapper/019:06:140
306612126110,12sleep20-21swapper/219:07:092
82072500,0sleep3391ktimersoftd/321:37:233
79652480,0sleep20-21swapper/219:25:222
17432450,0sleep30-21swapper/321:33:503
296172420,0sleep00-21swapper/021:10:410
272292360,1sleep23108999cyclictest21:49:232
113792110,0sleep10-21swapper/121:39:571
51752100,0sleep10-21swapper/123:33:151
26722100,0sleep10-21swapper/123:11:411
2039290,0sleep10-21swapper/119:15:241
310809971,3cyclictest0-21swapper/121:15:161
310949961,4cyclictest0-21swapper/322:00:153
310809962,2cyclictest0-21swapper/119:50:141
310809961,3cyclictest0-21swapper/122:45:181
310809961,3cyclictest0-21swapper/121:30:181
310949955,0cyclictest1211-21kworker/3:223:41:073
310949955,0cyclictest1211-21kworker/3:200:37:063
310949950,5cyclictest0-21swapper/323:27:063
310949950,5cyclictest0-21swapper/323:05:433
310949950,5cyclictest0-21swapper/322:47:143
310949950,5cyclictest0-21swapper/322:18:183
310949950,4cyclictest0-21swapper/323:35:143
310949950,4cyclictest0-21swapper/321:50:143
310949950,4cyclictest0-21swapper/300:25:143
310899950,5cyclictest0-21swapper/223:23:272
310899950,5cyclictest0-21swapper/220:50:132
310899950,3cyclictest0-21swapper/222:35:192
310809952,2cyclictest0-21swapper/123:55:141
310809952,2cyclictest0-21swapper/120:25:141
310809952,2cyclictest0-21swapper/120:10:161
310809951,3cyclictest0-21swapper/121:45:141
310809951,3cyclictest0-21swapper/120:30:131
310809951,3cyclictest0-21swapper/100:20:161
310809951,2cyclictest0-21swapper/119:20:191
310809951,2cyclictest0-21swapper/100:35:131
310809950,5cyclictest0-21swapper/122:15:011
310809950,5cyclictest0-21swapper/100:33:411
310789950,5cyclictest0-21swapper/023:52:320
310789950,5cyclictest0-21swapper/023:09:550
310789950,5cyclictest0-21swapper/021:53:050
310789950,5cyclictest0-21swapper/000:16:330
310789950,0cyclictest0-21swapper/000:21:120
310949944,0cyclictest1211-21kworker/3:220:30:473
310949944,0cyclictest1211-21kworker/3:219:45:583
310949944,0cyclictest0-21swapper/322:52:023
310949944,0cyclictest0-21swapper/322:10:263
310949944,0cyclictest0-21swapper/321:19:213
310949944,0cyclictest0-21swapper/321:12:053
310949942,2cyclictest1211-21kworker/3:223:33:503
310949941,3cyclictest740-21snmpd23:56:493
310949941,3cyclictest740-21snmpd22:05:493
310949940,4cyclictest7022-21apt-get19:25:173
310949940,4cyclictest27344-21df_inode20:05:173
310949940,4cyclictest0-21swapper/322:40:023
310949940,4cyclictest0-21swapper/322:35:053
310949940,4cyclictest0-21swapper/322:20:143
310949940,4cyclictest0-21swapper/320:35:133
310949940,4cyclictest0-21swapper/320:25:163
310949940,4cyclictest0-21swapper/319:40:223
310949940,4cyclictest0-21swapper/319:30:183
310949940,4cyclictest0-21swapper/300:15:183
310949940,4cyclictest0-21swapper/300:10:183
310949940,3cyclictest8201-21python22:55:223
310949940,3cyclictest7357-21apt-get23:15:163
310949940,3cyclictest7035-21ssh21:55:583
310949940,3cyclictest498-21latency_hist00:10:013
310949940,3cyclictest4422-21date20:25:003
310949940,3cyclictest3774-21sh23:51:323
310949940,3cyclictest31965-21apt20:15:133
310949940,3cyclictest30370-21ssh23:48:293
310949940,3cyclictest24315-21cat20:00:003
310949940,3cyclictest19539-21ssh00:00:543
310949940,3cyclictest19292-21latency_hist20:55:003
310949940,3cyclictest14908-21ssh21:41:083
310949940,3cyclictest14908-21ssh21:41:083
310949940,3cyclictest0-21swapper/319:20:183
310949940,2cyclictest740-21snmpd23:10:173
310949940,1cyclictest13229-21ntp_states20:40:223
310899944,0cyclictest0-21swapper/223:10:122
310899944,0cyclictest0-21swapper/223:03:222
310899944,0cyclictest0-21swapper/200:24:592
310899943,1cyclictest3214-21grep00:30:212
310899943,1cyclictest26020-21grep00:05:162
310899943,1cyclictest15652-21pmu-power22:40:282
310899942,1cyclictest5496-21ssh21:55:202
310899940,4cyclictest0-21swapper/223:55:132
310899940,4cyclictest0-21swapper/223:53:002
310899940,4cyclictest0-21swapper/223:29:052
310899940,4cyclictest0-21swapper/222:50:112
310899940,4cyclictest0-21swapper/222:45:062
310899940,4cyclictest0-21swapper/222:20:142
310899940,4cyclictest0-21swapper/221:54:532
310899940,4cyclictest0-21swapper/221:11:052
310899940,4cyclictest0-21swapper/220:30:132
310899940,4cyclictest0-21swapper/220:25:142
310899940,3cyclictest5076-21ntp_states21:35:212
310899940,3cyclictest2709-21perl20:20:172
310899940,3cyclictest25820-21/usr/sbin/munin00:25:162
310899940,3cyclictest25519-21irqstats21:05:232
310899940,3cyclictest22382-21df_inode19:55:162
310899940,3cyclictest21819-21sed21:00:002
310899940,3cyclictest14346-21ssh23:00:002
310899940,3cyclictest13751-21/usr/sbin/munin22:00:202
310899940,3cyclictest0-21swapper/220:10:172
310899940,1cyclictest5535-21diskstats22:15:192
310899940,1cyclictest28707-21perl21:30:212
310899940,1cyclictest16148-21ssh23:40:162
310899940,1cyclictest14428-21/usr/sbin/munin20:45:232
310899940,1cyclictest11373-21/usr/sbin/munin21:20:272
310899940,1cyclictest10606-21sed00:15:252
310899940,0cyclictest0-21swapper/221:40:172
310899940,0cyclictest0-21swapper/221:40:172
310809944,0cyclictest0-21swapper/123:20:041
310809944,0cyclictest0-21swapper/122:40:031
310809941,2cyclictest0-21swapper/123:25:131
310809941,2cyclictest0-21swapper/123:00:131
310809941,2cyclictest0-21swapper/122:55:131
310809941,2cyclictest0-21swapper/122:50:121
310809941,2cyclictest0-21swapper/122:35:181
310809941,2cyclictest0-21swapper/122:05:171
310809941,2cyclictest0-21swapper/121:25:161
310809941,2cyclictest0-21swapper/121:10:191
310809941,2cyclictest0-21swapper/121:00:121
310809941,2cyclictest0-21swapper/120:55:151
310809941,2cyclictest0-21swapper/120:40:151
310809941,2cyclictest0-21swapper/120:35:131
310809941,2cyclictest0-21swapper/120:20:151
310809941,2cyclictest0-21swapper/119:40:221
310809941,2cyclictest0-21swapper/119:25:181
310809941,2cyclictest0-21swapper/119:10:131
310809941,2cyclictest0-21swapper/100:05:131
310809940,2cyclictest0-21swapper/123:05:131
310809940,2cyclictest0-21swapper/100:00:141
310809940,0cyclictest0-21swapper/122:00:021
310789944,0cyclictest0-21swapper/023:21:410
310789944,0cyclictest0-21swapper/022:48:400
310789944,0cyclictest0-21swapper/000:11:530
310789943,1cyclictest989-21latency_hist19:15:000
310789943,1cyclictest11507-21munin-run21:40:010
310789940,4cyclictest22055-21apt-get21:00:120
310789940,4cyclictest0-21swapper/023:55:130
310789940,4cyclictest0-21swapper/022:55:120
310789940,4cyclictest0-21swapper/022:10:190
310789940,4cyclictest0-21swapper/000:25:140
310789940,4cyclictest0-21swapper/000:05:140
310789940,3cyclictest7936-21ssh23:15:190
310789940,3cyclictest7608-21python22:35:280
310789940,3cyclictest4312-21ssh21:55:140
310789940,3cyclictest3408-21python00:30:230
310789940,3cyclictest32010-21/usr/sbin/munin23:30:170
310789940,3cyclictest2984-21ssh23:11:560
310789940,3cyclictest25037-21sed23:25:180
310789940,3cyclictest21231-21/usr/sbin/munin19:50:240
310789940,3cyclictest19983-21ssh23:02:550
310789940,3cyclictest1975-21munin-node19:15:010
310789940,3cyclictest19426-21ssh22:43:070
310789940,3cyclictest14153-21perl22:20:170
310789940,3cyclictest12748-21ssh21:20:240
310789940,3cyclictest0-21swapper/022:25:140
310789940,3cyclictest0-21swapper/020:20:160
310789940,1cyclictest5849-21latency_hist22:55:000
310789940,0cyclictest0-21swapper/023:49:110
310949932,1cyclictest740-21snmpd21:05:093
310949931,1cyclictest0-21swapper/320:45:163
310949930,3cyclictest30278-21irqstats20:10:213
310949930,3cyclictest26763-21diskmemload00:30:063
310949930,3cyclictest19897-21diskstats20:55:173
310949930,3cyclictest18650-21irqstats00:20:203
310949930,3cyclictest1418-21users19:15:003
310949930,3cyclictest1211-21kworker/3:221:25:143
310949930,3cyclictest0-21swapper/323:20:133
310949930,3cyclictest0-21swapper/323:00:133
310949930,3cyclictest0-21swapper/322:25:143
310949930,3cyclictest0-21swapper/321:45:143
310949930,3cyclictest0-21swapper/321:20:163
310949930,3cyclictest0-21swapper/321:00:023
310949930,3cyclictest0-21swapper/320:00:153
310949930,3cyclictest0-21swapper/319:50:143
310949930,3cyclictest0-21swapper/319:35:053
310949930,3cyclictest0-21swapper/319:10:133
310949930,2cyclictest29636-21sh22:30:053
310899932,1cyclictest8280-21wc23:35:152
310899932,1cyclictest21598-21/usr/sbin/munin22:25:172
310899932,1cyclictest19362-21ssh21:25:022
310899931,1cyclictest9464-21apt-get20:35:132
310899930,3cyclictest9490-21apt-get19:30:182
310899930,3cyclictest5070-21df19:20:222
310899930,3cyclictest29995-21perl22:10:222
310899930,3cyclictest2529-21df19:15:282
310899930,3cyclictest25084-21df23:45:172
310899930,3cyclictest24345-21/usr/sbin/munin20:00:282
310899930,3cyclictest19909-21df19:50:162
310899930,3cyclictest0-21swapper/223:30:042
310899930,3cyclictest0-21swapper/223:05:032
310899930,3cyclictest0-21swapper/222:30:122
310899930,3cyclictest0-21swapper/222:05:152
310899930,3cyclictest0-21swapper/221:15:042
310899930,3cyclictest0-21swapper/221:00:012
310899930,3cyclictest0-21swapper/220:40:162
310899930,3cyclictest0-21swapper/220:15:392
310899930,3cyclictest0-21swapper/220:05:152
310899930,3cyclictest0-21swapper/219:45:192
310899930,3cyclictest0-21swapper/219:40:222
310899930,3cyclictest0-21swapper/219:35:182
310899930,3cyclictest0-21swapper/219:10:132
310899930,3cyclictest0-21swapper/200:35:132
310899930,3cyclictest0-21swapper/200:10:212
310899930,3cyclictest0-21swapper/200:00:042
310899930,1cyclictest7140-21ssh23:15:052
310809933,0cyclictest0-21swapper/122:14:181
310809931,2cyclictest0-21swapper/121:05:151
310809931,1cyclictest0-21swapper/123:50:161
310809931,1cyclictest0-21swapper/122:30:141
310809931,1cyclictest0-21swapper/122:25:141
310809931,1cyclictest0-21swapper/121:40:161
310809931,1cyclictest0-21swapper/121:40:151
310809931,1cyclictest0-21swapper/121:20:161
310809931,1cyclictest0-21swapper/120:50:131
310809931,1cyclictest0-21swapper/119:45:181
310809930,2cyclictest7211-21taskset23:15:091
310809930,2cyclictest24405-21taskset23:45:081
310809930,2cyclictest0-21swapper/123:40:161
310809930,2cyclictest0-21swapper/123:35:141
310809930,2cyclictest0-21swapper/121:55:141
310809930,2cyclictest0-21swapper/121:50:131
310809930,2cyclictest0-21swapper/120:05:141
310789932,1cyclictest740-21snmpd20:10:030
310789932,1cyclictest740-21snmpd00:35:080
310789932,1cyclictest41ktimersoftd/023:35:150
310789932,1cyclictest12528-21ssh22:00:080
310789931,1cyclictest11957-21apt-get20:40:150
310789931,1cyclictest0-21swapper/020:30:130
310789930,3cyclictest28210-21grep21:30:190
310789930,3cyclictest26763-21diskmemload22:15:030
310789930,3cyclictest20690-21ssh22:05:060
310789930,3cyclictest0-21swapper/023:40:160
310789930,3cyclictest0-21swapper/022:30:130
310789930,3cyclictest0-21swapper/021:40:140
310789930,3cyclictest0-21swapper/021:40:140
310789930,3cyclictest0-21swapper/021:25:130
310789930,3cyclictest0-21swapper/021:15:150
310789930,3cyclictest0-21swapper/021:05:150
310789930,3cyclictest0-21swapper/020:55:160
310789930,3cyclictest0-21swapper/020:50:120
310789930,3cyclictest0-21swapper/020:45:150
310789930,3cyclictest0-21swapper/020:35:130
310789930,3cyclictest0-21swapper/020:25:150
310789930,3cyclictest0-21swapper/020:15:130
310789930,3cyclictest0-21swapper/020:05:140
310789930,3cyclictest0-21swapper/020:00:170
310789930,3cyclictest0-21swapper/019:45:180
310789930,3cyclictest0-21swapper/019:43:210
310789930,3cyclictest0-21swapper/019:35:160
310789930,3cyclictest0-21swapper/019:30:060
310789930,3cyclictest0-21swapper/019:25:040
310789930,3cyclictest0-21swapper/019:20:180
310789930,3cyclictest0-21swapper/000:00:140
310789930,2cyclictest0-21swapper/019:55:170
310789930,1cyclictest20058-21ssh21:45:070
310809921,1cyclictest0-21swapper/122:20:011
310809921,1cyclictest0-21swapper/120:15:011
310809921,1cyclictest0-21swapper/120:00:161
310809921,1cyclictest0-21swapper/119:55:141
310809921,1cyclictest0-21swapper/119:35:161
310809921,1cyclictest0-21swapper/100:15:181
310809920,2cyclictest0-21swapper/119:30:181
310809920,1cyclictest912-21chrt00:10:101
310809920,1cyclictest25697-21chrt00:25:071
310809920,1cyclictest0-21swapper/120:45:171
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional