You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-17 - 16:06
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot2.osadl.org (updated Wed Sep 17, 2025 12:44:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
212772202186,11sleep10-21swapper/107:08:041
214062184167,12sleep20-21swapper/207:09:412
213752183166,12sleep30-21swapper/307:09:193
211372180162,13sleep00-21swapper/007:06:140
276082780,0sleep30-21swapper/311:40:133
197042500,0sleep00-21swapper/012:25:090
271462460,0sleep10-21swapper/110:19:041
175582450,0sleep20-21swapper/209:57:552
55102110,0sleep10-21swapper/110:56:111
215979980,4cyclictest27507-21munin-run09:15:010
216099970,6cyclictest0-21swapper/312:35:133
215979970,4cyclictest23872-21munin-run10:50:010
216099960,5cyclictest0-21swapper/309:05:153
216019961,3cyclictest0-21swapper/111:20:121
215979960,0cyclictest0-21swapper/008:05:010
6741250,0sleep30-21swapper/312:18:243
216099955,0cyclictest24148-21kworker/3:112:02:073
216099955,0cyclictest24148-21kworker/3:111:39:443
216099955,0cyclictest24148-21kworker/3:111:06:073
216099955,0cyclictest24148-21kworker/3:110:10:073
216099955,0cyclictest24148-21kworker/3:109:14:093
216099955,0cyclictest24148-21kworker/3:107:33:243
216099955,0cyclictest0-21swapper/310:31:353
216099950,5cyclictest0-21swapper/312:09:143
216099950,5cyclictest0-21swapper/311:49:523
216099950,5cyclictest0-21swapper/311:17:163
216099950,4cyclictest24160-21munin-node07:15:003
216069955,0cyclictest17230-21diskmemload12:04:062
216069950,5cyclictest0-21swapper/210:35:212
216069950,5cyclictest0-21swapper/210:02:202
216069950,5cyclictest0-21swapper/209:15:192
216069950,5cyclictest0-21swapper/208:05:002
216069950,0cyclictest0-21swapper/209:44:422
216019952,2cyclictest0-21swapper/109:45:131
216019952,2cyclictest0-21swapper/107:10:121
216019951,3cyclictest0-21swapper/110:45:121
216019951,3cyclictest0-21swapper/108:20:131
216019951,3cyclictest0-21swapper/107:20:121
216019951,2cyclictest0-21swapper/112:15:111
216019951,2cyclictest0-21swapper/112:05:131
216019951,2cyclictest0-21swapper/112:00:121
216019951,2cyclictest0-21swapper/111:40:121
216019951,2cyclictest0-21swapper/110:35:151
216019951,2cyclictest0-21swapper/109:55:121
216019951,2cyclictest0-21swapper/107:15:121
216019950,5cyclictest0-21swapper/111:06:081
216019950,5cyclictest0-21swapper/110:27:441
216019950,5cyclictest0-21swapper/109:34:021
216019950,5cyclictest0-21swapper/108:53:341
216019950,3cyclictest0-21swapper/109:05:151
215979952,2cyclictest0-21swapper/008:25:140
215979950,5cyclictest0-21swapper/012:07:350
215979950,1cyclictest12790-21ntp_states09:55:170
9436240,0sleep20-21swapper/210:42:152
216099944,0cyclictest24148-21kworker/3:111:50:553
216099944,0cyclictest0-21swapper/312:26:233
216099944,0cyclictest0-21swapper/310:55:533
216099944,0cyclictest0-21swapper/309:51:183
216099942,1cyclictest15104-21ssh09:40:203
216099941,3cyclictest282462chrt08:25:153
216099941,3cyclictest0-21swapper/307:55:133
216099940,4cyclictest23654-21ssh09:45:113
216099940,4cyclictest20581-21ssh09:27:053
216099940,4cyclictest0-21swapper/311:25:533
216099940,4cyclictest0-21swapper/311:21:413
216099940,4cyclictest0-21swapper/311:10:143
216099940,4cyclictest0-21swapper/311:00:123
216099940,4cyclictest0-21swapper/310:45:133
216099940,4cyclictest0-21swapper/310:40:123
216099940,4cyclictest0-21swapper/310:35:143
216099940,4cyclictest0-21swapper/310:16:053
216099940,4cyclictest0-21swapper/310:00:133
216099940,4cyclictest0-21swapper/308:55:123
216099940,4cyclictest0-21swapper/308:20:123
216099940,4cyclictest0-21swapper/307:15:123
216099940,3cyclictest9870-21/usr/sbin/munin07:50:133
216099940,3cyclictest9843-21/usr/sbin/munin12:20:143
216099940,3cyclictest9299-21tr11:30:213
216099940,3cyclictest7057-21ssh09:36:163
216099940,3cyclictest6820-21/usr/sbin/munin10:25:203
216099940,3cyclictest32516-21cut08:35:013
216099940,3cyclictest31142-21ntp_states12:30:193
216099940,3cyclictest28920-21ssh10:20:023
216099940,3cyclictest25807-21ssh10:50:183
216099940,3cyclictest25577-21ssh09:30:023
216099940,3cyclictest24942-21gpgv08:20:003
216099940,3cyclictest24894-21ssh11:55:063
216099940,3cyclictest21458-21/usr/sbin/munin08:10:203
216099940,3cyclictest17136-21ssh09:57:403
216099940,3cyclictest17066-21latency_hist08:05:013
216099940,2cyclictest0-21swapper/309:15:123
216099940,1cyclictest406-21cut08:35:143
216099940,1cyclictest32737-21memory10:05:173
216069944,0cyclictest0-21swapper/212:32:282
216069944,0cyclictest0-21swapper/212:10:052
216069944,0cyclictest0-21swapper/211:43:172
216069944,0cyclictest0-21swapper/210:22:502
216069944,0cyclictest0-21swapper/209:10:482
216069943,1cyclictest3783-21ls07:35:172
216069943,1cyclictest12555-21tr07:55:142
216069941,3cyclictest714-21snmpd08:00:422
216069940,4cyclictest7404-21apt-get07:45:132
216069940,4cyclictest32683-21df_inode11:10:142
216069940,4cyclictest14808-21rm10:12:372
216069940,4cyclictest0-21swapper/212:35:142
216069940,4cyclictest0-21swapper/212:19:572
216069940,4cyclictest0-21swapper/211:55:232
216069940,4cyclictest0-21swapper/211:08:492
216069940,4cyclictest0-21swapper/211:02:052
216069940,4cyclictest0-21swapper/210:56:542
216069940,4cyclictest0-21swapper/210:50:112
216069940,4cyclictest0-21swapper/210:45:262
216069940,4cyclictest0-21swapper/210:30:152
216069940,4cyclictest0-21swapper/210:05:552
216069940,4cyclictest0-21swapper/209:50:132
216069940,4cyclictest0-21swapper/209:45:252
216069940,4cyclictest0-21swapper/209:32:422
216069940,4cyclictest0-21swapper/209:27:162
216069940,4cyclictest0-21swapper/207:30:132
216069940,3cyclictest8696-21irqstats11:30:172
216069940,3cyclictest8285-21perl08:50:142
216069940,3cyclictest25675-21diskstats08:20:142
216069940,3cyclictest19673-21ssh11:35:232
216069940,3cyclictest15674-21sh11:17:422
216019944,0cyclictest0-21swapper/112:23:341
216019944,0cyclictest0-21swapper/111:30:361
216019944,0cyclictest0-21swapper/109:20:041
216019941,2cyclictest0-21swapper/111:10:131
216019941,2cyclictest0-21swapper/110:40:131
216019941,2cyclictest0-21swapper/110:30:141
216019941,2cyclictest0-21swapper/110:20:141
216019941,2cyclictest0-21swapper/110:10:131
216019941,2cyclictest0-21swapper/110:05:141
216019941,2cyclictest0-21swapper/109:10:021
216019941,2cyclictest0-21swapper/109:00:131
216019941,2cyclictest0-21swapper/108:05:111
216019941,2cyclictest0-21swapper/107:45:121
216019941,1cyclictest0-21swapper/112:25:121
216019941,1cyclictest0-21swapper/109:15:131
216019941,1cyclictest0-21swapper/108:30:091
216019941,1cyclictest0-21swapper/107:40:121
216019940,3cyclictest0-21swapper/111:50:151
215979944,0cyclictest0-21swapper/012:34:300
215979944,0cyclictest0-21swapper/012:02:210
215979944,0cyclictest0-21swapper/011:41:320
215979944,0cyclictest0-21swapper/010:38:400
215979944,0cyclictest0-21swapper/010:12:550
215979942,1cyclictest4028-21ssh09:35:120
215979941,2cyclictest0-21swapper/007:40:120
215979940,4cyclictest6480-21ssh10:56:460
215979940,4cyclictest30926-21awk11:25:180
215979940,4cyclictest0-21swapper/012:10:110
215979940,4cyclictest0-21swapper/011:53:390
215979940,4cyclictest0-21swapper/011:45:110
215979940,4cyclictest0-21swapper/011:15:130
215979940,4cyclictest0-21swapper/010:44:010
215979940,4cyclictest0-21swapper/010:23:480
215979940,4cyclictest0-21swapper/010:15:300
215979940,4cyclictest0-21swapper/010:00:130
215979940,4cyclictest0-21swapper/009:31:200
215979940,4cyclictest0-21swapper/009:20:160
215979940,4cyclictest0-21swapper/008:15:130
215979940,4cyclictest0-21swapper/007:35:200
215979940,4cyclictest0-21swapper/007:30:140
215979940,4cyclictest0-21swapper/007:20:110
215979940,3cyclictest852-21perl08:35:150
215979940,3cyclictest7930-21irqstats10:25:170
215979940,3cyclictest7660-21tr12:35:140
215979940,3cyclictest7467-21date08:50:000
215979940,3cyclictest3596-21sh11:11:310
215979940,3cyclictest3461-21hddtemp_smartct08:40:180
215979940,3cyclictest25580-21grep07:15:140
215979940,3cyclictest2485-21sh10:06:110
215979940,3cyclictest2458-21rm12:15:400
215979940,3cyclictest24287-21diskstats09:45:150
215979940,3cyclictest24062-21munin-node07:15:000
215979940,3cyclictest23191-21sh11:05:180
215979940,3cyclictest17595-21ssh11:35:150
215979940,3cyclictest17483-21ssh12:23:530
215979940,1cyclictest25143-21/usr/sbin/munin08:20:150
215979940,1cyclictest12652-21expr11:00:140
215979940,0cyclictest17228-21apt-get09:10:010
215979940,0cyclictest0-21swapper/011:56:220
215979940,0cyclictest0-21swapper/011:34:070
216099932,1cyclictest714-21snmpd07:25:033
216099930,3cyclictest5481-21ssh09:20:053
216099930,3cyclictest4934-21apt-get07:40:123
216099930,3cyclictest17064-21apt-get08:05:013
216099930,3cyclictest12522-21apt-get09:00:143
216099930,3cyclictest0-21swapper/312:10:063
216099930,3cyclictest0-21swapper/308:50:123
216099930,3cyclictest0-21swapper/308:45:113
216099930,3cyclictest0-21swapper/308:40:153
216099930,3cyclictest0-21swapper/307:45:093
216099930,3cyclictest0-21swapper/307:35:123
216099930,3cyclictest0-21swapper/307:20:113
216069931,1cyclictest714-21snmpd08:30:032
216069931,1cyclictest0-21swapper/207:10:132
216069930,3cyclictest9844-21apt12:20:122
216069930,3cyclictest6568-21rm10:25:042
216069930,3cyclictest5359-21expr11:45:132
216069930,3cyclictest3469-21ssh09:35:122
216069930,3cyclictest30839-21ssh11:25:172
216069930,3cyclictest27971-21sh07:20:132
216069930,3cyclictest27697-21wget08:25:142
216069930,3cyclictest24578-21cat07:15:012
216069930,3cyclictest24160-21munin-node08:45:112
216069930,3cyclictest19630-21sh11:20:022
216069930,3cyclictest17230-21diskmemload11:50:152
216069930,3cyclictest10093-21cut08:55:122
216069930,3cyclictest0-21swapper/212:25:112
216069930,3cyclictest0-21swapper/212:05:092
216069930,3cyclictest0-21swapper/210:15:122
216069930,3cyclictest0-21swapper/209:20:022
216069930,3cyclictest0-21swapper/209:05:152
216069930,3cyclictest0-21swapper/209:00:132
216069930,3cyclictest0-21swapper/208:40:162
216069930,3cyclictest0-21swapper/208:35:162
216069930,3cyclictest0-21swapper/208:15:142
216069930,3cyclictest0-21swapper/208:10:142
216069930,3cyclictest0-21swapper/207:50:132
216069930,3cyclictest0-21swapper/207:40:132
216069930,3cyclictest0-21swapper/207:25:142
216019933,0cyclictest0-21swapper/112:39:151
216019933,0cyclictest0-21swapper/108:43:411
216019933,0cyclictest0-21swapper/107:30:561
216019931,2cyclictest0-21swapper/111:00:131
216019931,2cyclictest0-21swapper/109:35:101
216019931,1cyclictest0-21swapper/111:45:111
216019931,1cyclictest0-21swapper/111:25:151
216019931,1cyclictest0-21swapper/111:15:011
216019931,1cyclictest0-21swapper/110:50:111
216019931,1cyclictest0-21swapper/110:00:131
216019931,1cyclictest0-21swapper/108:35:131
216019931,1cyclictest0-21swapper/108:25:131
216019931,1cyclictest0-21swapper/107:55:131
216019931,1cyclictest0-21swapper/107:50:141
216019930,2cyclictest795-21taskset12:31:151
216019930,2cyclictest15642-21taskset09:25:071
216019930,2cyclictest0-21swapper/111:55:121
216019930,2cyclictest0-21swapper/111:35:141
216019930,2cyclictest0-21swapper/109:40:151
216019930,2cyclictest0-21swapper/108:55:111
216019930,2cyclictest0-21swapper/108:15:131
216019930,2cyclictest0-21swapper/108:10:131
216019930,2cyclictest0-21swapper/107:35:111
216019930,1cyclictest0-21swapper/107:25:151
216019930,0cyclictest0-21swapper/108:03:401
215979933,0cyclictest0-21swapper/009:25:120
215979932,1cyclictest714-21snmpd08:12:540
215979932,1cyclictest24933-21df10:50:130
215979931,1cyclictest714-21snmpd07:45:060
215979931,1cyclictest0-21swapper/009:50:130
215979930,3cyclictest30439-21df07:25:150
215979930,3cyclictest30076-21/usr/sbin/munin08:30:110
215979930,3cyclictest27815-21ssh09:15:050
215979930,3cyclictest12547-21grep07:55:150
215979930,3cyclictest0-21swapper/011:20:110
215979930,3cyclictest0-21swapper/010:30:140
215979930,3cyclictest0-21swapper/009:40:140
215979930,3cyclictest0-21swapper/009:00:130
215979930,3cyclictest0-21swapper/008:55:120
215979930,3cyclictest0-21swapper/008:50:100
215979930,3cyclictest0-21swapper/008:00:130
215979930,3cyclictest0-21swapper/007:50:130
216019921,1cyclictest0-21swapper/108:45:121
216019920,1cyclictest0-21swapper/112:10:011
216019920,1cyclictest0-21swapper/109:50:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional