You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-06-02 - 11:05

x86 Intel Core i5-6440EQ @2700 MHz, Linux 4.9.47-rt37 (Profile)

Latency plot of system in rack #5, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot2.osadl.org (updated Fri Jun 02, 2023 00:44:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
148652211195,11sleep30-21swapper/319:06:093
150682195180,11sleep00-21swapper/019:08:420
151462186171,10sleep10-21swapper/119:09:441
148122177159,13sleep20-21swapper/219:05:272
200312440,0sleep20-21swapper/222:46:182
82532410,0sleep10-21swapper/121:05:011
21440290,0sleep30-21swapper/323:42:003
153439991,4cyclictest13268-21awk21:30:010
153549980,7cyclictest0-21swapper/321:15:193
153439980,4cyclictest22069-21munin-run21:35:000
27972270,0sleep20-21swapper/223:09:402
153509960,6cyclictest8626-21/usr/sbin/munin20:00:372
153509960,5cyclictest0-21swapper/219:15:142
153489961,3cyclictest0-21swapper/123:50:221
153489961,3cyclictest0-21swapper/122:10:191
153489961,2cyclictest0-21swapper/119:45:171
153439966,0cyclictest3-21ksoftirqd/023:55:180
153439964,1cyclictest3-21ksoftirqd/022:05:020
153439963,1cyclictest3-21ksoftirqd/023:40:010
153439960,3cyclictest3440-21perf19:50:010
153439960,1cyclictest13945-21rm21:48:330
19618250,0sleep20-21swapper/223:40:402
153549955,0cyclictest0-21swapper/300:34:583
153549955,0cyclictest0-21swapper/300:22:273
153549955,0cyclictest0-21swapper/300:14:193
153549950,5cyclictest0-21swapper/322:47:013
153549950,4cyclictest0-21swapper/323:00:183
153549950,3cyclictest0-21swapper/300:09:053
153549950,0cyclictest0-21swapper/323:37:043
153549950,0cyclictest0-21swapper/321:52:363
153509955,0cyclictest0-21swapper/200:18:182
153509950,5cyclictest0-21swapper/222:35:512
153509950,5cyclictest0-21swapper/222:34:112
153509950,5cyclictest0-21swapper/200:27:112
153509950,4cyclictest1735-21apt-get22:00:182
153489952,2cyclictest0-21swapper/122:15:191
153489952,2cyclictest0-21swapper/119:10:201
153489951,3cyclictest0-21swapper/100:15:161
153489951,2cyclictest0-21swapper/121:50:171
153489951,2cyclictest0-21swapper/120:25:181
153489951,2cyclictest0-21swapper/119:50:161
153489950,5cyclictest0-21swapper/123:14:351
153489950,5cyclictest0-21swapper/122:56:501
153489950,5cyclictest0-21swapper/122:20:451
153489950,5cyclictest0-21swapper/121:11:041
153489950,5cyclictest0-21swapper/100:39:151
153489950,5cyclictest0-21swapper/100:14:491
153439955,0cyclictest3-21ksoftirqd/023:05:010
153439955,0cyclictest3-21ksoftirqd/019:20:150
153439955,0cyclictest3-21ksoftirqd/019:15:020
153439955,0cyclictest3-21ksoftirqd/000:00:140
153439954,0cyclictest3-21ksoftirqd/020:15:160
153439953,1cyclictest25715-21awk20:35:010
153439952,1cyclictest3-21ksoftirqd/020:10:010
153439952,1cyclictest3-21ksoftirqd/020:10:010
153439951,3cyclictest5106-21apt-get00:10:010
153439950,5cyclictest3925-21ssh23:32:090
153439950,5cyclictest0-21swapper/000:34:340
153439950,4cyclictest4184-21df_inode19:50:190
153439950,4cyclictest0-21swapper/023:40:210
153439950,4cyclictest0-21swapper/022:35:170
153439950,4cyclictest0-21swapper/022:10:190
153439950,4cyclictest0-21swapper/020:40:210
153439950,2cyclictest0-21swapper/021:14:280
2283240,3sleep310609-21diskmemload23:50:063
153549944,0cyclictest0-21swapper/323:45:043
153549944,0cyclictest0-21swapper/321:40:313
153549944,0cyclictest0-21swapper/321:20:443
153549943,1cyclictest32326-21tr00:25:173
153549943,1cyclictest2296-21ssh22:55:163
153549941,3cyclictest759-21snmpd22:00:273
153549941,2cyclictest0-21swapper/320:10:183
153549940,4cyclictest61-21kswapd023:05:173
153549940,4cyclictest10609-21diskmemload22:26:313
153549940,4cyclictest0-21swapper/323:55:183
153549940,4cyclictest0-21swapper/323:25:163
153549940,4cyclictest0-21swapper/323:23:233
153549940,4cyclictest0-21swapper/322:50:163
153549940,4cyclictest0-21swapper/322:40:193
153549940,4cyclictest0-21swapper/322:30:503
153549940,4cyclictest0-21swapper/322:22:063
153549940,4cyclictest0-21swapper/322:05:193
153549940,4cyclictest0-21swapper/321:45:033
153549940,4cyclictest0-21swapper/321:35:513
153549940,4cyclictest0-21swapper/321:30:063
153549940,4cyclictest0-21swapper/321:27:563
153549940,4cyclictest0-21swapper/321:10:223
153549940,4cyclictest0-21swapper/320:35:193
153549940,4cyclictest0-21swapper/319:11:573
153549940,4cyclictest0-21swapper/300:17:033
153549940,3cyclictest6123-21/usr/sbin/munin19:55:173
153549940,3cyclictest5477-21/usr/sbin/munin23:15:303
153549940,3cyclictest31088-21sh22:35:043
153549940,3cyclictest29444-21hddtemp_smartct19:35:213
153549940,3cyclictest25887-21df_inode21:55:193
153549940,3cyclictest2180-21irqstats19:45:233
153549940,3cyclictest21004-21/usr/sbin/munin20:25:193
153549940,3cyclictest19416-21/usr/sbin/munin22:10:273
153549940,3cyclictest18075-21sed19:15:013
153549940,3cyclictest17317-21wget00:35:193
153549940,3cyclictest1151-21cut19:45:013
153549940,3cyclictest10609-21diskmemload22:16:313
153549940,3cyclictest0-21swapper/320:20:003
153549940,1cyclictest19054-21df20:20:183
153549940,0cyclictest0-21swapper/321:02:583
153549940,0cyclictest0-21swapper/320:55:413
153509944,0cyclictest2661-21apt-get23:50:222
153509942,1cyclictest14141-21diskstats22:25:202
153509941,2cyclictest0-21swapper/220:35:192
153509941,2cyclictest0-21swapper/220:15:162
153509940,4cyclictest447-21perl23:30:262
153509940,4cyclictest26138-21/usr/sbin/munin19:30:222
153509940,4cyclictest19424-21apt-get22:10:182
153509940,4cyclictest12953-21python23:00:272
153509940,4cyclictest0-21swapper/223:35:062
153509940,4cyclictest0-21swapper/223:10:212
153509940,4cyclictest0-21swapper/222:50:162
153509940,4cyclictest0-21swapper/222:05:022
153509940,4cyclictest0-21swapper/221:50:182
153509940,4cyclictest0-21swapper/221:40:202
153509940,4cyclictest0-21swapper/221:35:182
153509940,4cyclictest0-21swapper/220:40:212
153509940,4cyclictest0-21swapper/219:40:242
153509940,4cyclictest0-21swapper/200:30:182
153509940,4cyclictest0-21swapper/200:22:252
153509940,4cyclictest0-21swapper/200:05:202
153509940,3cyclictest9915-21python21:05:412
153509940,3cyclictest7299-21sensors_temp00:10:322
153509940,3cyclictest7179-21ntp_states21:00:262
153509940,3cyclictest5836-21irqstats22:20:232
153509940,3cyclictest5611-21df_inode21:25:182
153509940,3cyclictest30903-21/usr/sbin/munin20:45:272
153509940,3cyclictest25805-21date23:45:012
153509940,3cyclictest24503-21hddtemp_smartct19:25:182
153509940,3cyclictest21004-21/usr/sbin/munin20:25:202
153509940,3cyclictest15432-21irqstats23:20:242
153509940,3cyclictest13797-21apt21:30:162
153509940,3cyclictest13587-21sh23:55:342
153509940,3cyclictest112632sleep222:41:262
153509940,3cyclictest0-21swapper/219:20:162
153509940,2cyclictest2721-21cstates22:55:192
153509940,1cyclictest6123-21/usr/sbin/munin19:55:182
153509940,1cyclictest26502-21memory21:55:222
153509940,1cyclictest17248-21/usr/sbin/munin00:35:222
153509940,1cyclictest13564-21/usr/sbin/munin20:10:252
153509940,1cyclictest11919-21diskstats21:10:272
153509940,1cyclictest0-21swapper/223:25:272
153509940,0cyclictest0-21swapper/221:45:132
153509940,0cyclictest0-21swapper/221:16:072
153489944,0cyclictest0-21swapper/123:32:551
153489944,0cyclictest0-21swapper/122:38:301
153489941,2cyclictest0-21swapper/123:45:181
153489941,2cyclictest0-21swapper/122:50:161
153489941,2cyclictest0-21swapper/122:40:181
153489941,2cyclictest0-21swapper/122:05:191
153489941,2cyclictest0-21swapper/121:45:191
153489941,2cyclictest0-21swapper/121:35:181
153489941,2cyclictest0-21swapper/120:35:191
153489941,2cyclictest0-21swapper/119:20:161
153489941,2cyclictest0-21swapper/119:15:141
153489941,2cyclictest0-21swapper/100:25:161
153489941,2cyclictest0-21swapper/100:05:201
153489940,4cyclictest0-21swapper/122:30:131
153489940,0cyclictest0-21swapper/121:58:211
153439944,0cyclictest3-21ksoftirqd/021:50:180
153439944,0cyclictest3-21ksoftirqd/021:40:200
153439944,0cyclictest3-21ksoftirqd/020:15:000
153439944,0cyclictest3-21ksoftirqd/019:35:180
153439944,0cyclictest0-21swapper/021:57:170
153439943,1cyclictest19135-21ssh00:35:320
153439942,2cyclictest3-21ksoftirqd/023:20:180
153439941,2cyclictest10609-21diskmemload21:20:080
153439941,2cyclictest0-21swapper/022:55:170
153439940,4cyclictest32552-21ssh23:48:410
153439940,4cyclictest28955-21ssh23:10:140
153439940,4cyclictest1295-21ssh22:54:440
153439940,4cyclictest0-21swapper/023:50:210
153439940,4cyclictest0-21swapper/023:19:220
153439940,4cyclictest0-21swapper/022:40:190
153439940,4cyclictest0-21swapper/022:30:180
153439940,4cyclictest0-21swapper/022:00:180
153439940,4cyclictest0-21swapper/021:15:180
153439940,4cyclictest0-21swapper/021:05:190
153439940,4cyclictest0-21swapper/020:54:510
153439940,4cyclictest0-21swapper/000:16:510
153439940,3cyclictest4574-21ssh22:20:100
153439940,3cyclictest34102sleep020:55:100
153439940,3cyclictest29066-21/usr/sbin/munin00:05:210
153439940,3cyclictest26105-21egrep20:35:210
153439940,3cyclictest23666-21apt-get19:25:150
153439940,3cyclictest22312-21ssh21:35:020
153439940,3cyclictest21310-21ssh23:05:220
153439940,3cyclictest2030-21ssh00:25:450
153439940,3cyclictest16727-21ssh22:45:120
153439940,3cyclictest0-21swapper/022:15:050
153439940,3cyclictest0-21swapper/020:45:220
153439940,3cyclictest0-21swapper/020:20:160
153439940,1cyclictest81rcu_preempt21:00:180
153439940,1cyclictest23916-21grep00:20:180
153439940,0cyclictest81rcu_preempt20:00:200
19296230,0sleep10-21swapper/121:32:591
153549930,3cyclictest759-21snmpd00:00:103
153549930,3cyclictest28785-21ssh23:10:063
153549930,3cyclictest26141-21apt-get19:30:193
153549930,3cyclictest24472-21irqstats20:30:263
153549930,3cyclictest0-21swapper/323:30:173
153549930,3cyclictest0-21swapper/321:05:263
153549930,3cyclictest0-21swapper/320:50:203
153549930,3cyclictest0-21swapper/320:45:173
153549930,3cyclictest0-21swapper/320:40:213
153549930,3cyclictest0-21swapper/320:05:193
153549930,3cyclictest0-21swapper/320:05:193
153549930,3cyclictest0-21swapper/320:00:203
153549930,3cyclictest0-21swapper/319:50:153
153549930,3cyclictest0-21swapper/319:25:143
153549930,3cyclictest0-21swapper/319:20:163
153509933,0cyclictest0-21swapper/221:20:042
153509930,3cyclictest3658-21apt-get19:50:162
153509930,3cyclictest3655-21grep20:55:222
153509930,3cyclictest2417-21netstat19:45:242
153509930,3cyclictest18595-21/usr/sbin/munin20:20:172
153509930,3cyclictest0-21swapper/223:15:042
153509930,3cyclictest0-21swapper/222:15:072
153509930,3cyclictest0-21swapper/220:50:082
153509930,3cyclictest0-21swapper/220:30:182
153509930,3cyclictest0-21swapper/220:05:022
153509930,3cyclictest0-21swapper/220:05:022
153509930,3cyclictest0-21swapper/219:35:042
153509930,3cyclictest0-21swapper/219:10:192
153509930,3cyclictest0-21swapper/200:00:132
153489933,0cyclictest0-21swapper/123:15:181
153489931,1cyclictest0-21swapper/123:55:181
153489931,1cyclictest0-21swapper/123:40:201
153489931,1cyclictest0-21swapper/123:25:171
153489931,1cyclictest0-21swapper/123:05:171
153489931,1cyclictest0-21swapper/122:00:181
153489931,1cyclictest0-21swapper/121:25:151
153489931,1cyclictest0-21swapper/121:20:171
153489931,1cyclictest0-21swapper/121:05:201
153489931,1cyclictest0-21swapper/120:55:201
153489931,1cyclictest0-21swapper/120:15:161
153489931,1cyclictest0-21swapper/120:10:181
153489931,1cyclictest0-21swapper/120:05:191
153489931,1cyclictest0-21swapper/120:05:191
153489931,1cyclictest0-21swapper/120:00:011
153489931,1cyclictest0-21swapper/119:55:151
153489931,1cyclictest0-21swapper/119:40:181
153489931,1cyclictest0-21swapper/119:25:151
153489931,1cyclictest0-21swapper/100:30:181
153489931,1cyclictest0-21swapper/100:00:141
153489930,2cyclictest0-21swapper/121:15:181
153489930,2cyclictest0-21swapper/100:20:151
153439933,0cyclictest3-21ksoftirqd/019:40:190
153439932,1cyclictest759-21snmpd19:30:150
153439931,1cyclictest0-21swapper/019:55:160
153439930,3cyclictest0-21swapper/022:25:170
153439930,3cyclictest0-21swapper/020:25:100
153439930,3cyclictest0-21swapper/019:10:190
153439930,2cyclictest22812-21ssh23:25:000
153489921,1cyclictest0-21swapper/122:45:151
153489921,1cyclictest0-21swapper/122:25:181
153489921,1cyclictest0-21swapper/121:40:201
153489921,1cyclictest0-21swapper/120:50:181
153489920,1cyclictest18524-21chrt20:20:081
153489920,1cyclictest0-21swapper/123:20:181
153489920,1cyclictest0-21swapper/123:00:181
153489920,1cyclictest0-21swapper/120:45:171
153489920,1cyclictest0-21swapper/120:40:211
153489920,1cyclictest0-21swapper/120:30:181
153489920,1cyclictest0-21swapper/119:35:181
153489920,1cyclictest0-21swapper/119:30:191
153489920,0cyclictest0-21swapper/123:35:181
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional