You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-20 - 20:02
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot2.osadl.org (updated Thu Nov 20, 2025 12:44:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
59342202188,10sleep10-21swapper/107:08:201
38432194177,12sleep30-21swapper/307:05:053
38452191174,12sleep00-21swapper/007:05:060
60412185169,12sleep20-21swapper/207:09:422
102502490,0sleep10-21swapper/107:15:161
158772430,1sleep015879-21missed_timers07:25:160
243942410,0sleep10-21swapper/110:40:291
237482380,0sleep30-21swapper/309:20:133
18702380,0sleep10-21swapper/108:00:191
207462130,0sleep30-21swapper/311:44:223
623599100,6cyclictest4718-21taskset08:06:560
623599100,0cyclictest9955-21awk08:20:000
174102100,0sleep30-21swapper/309:16:443
62359991,4cyclictest6983-21munin-run12:10:010
62359991,4cyclictest6983-21munin-run12:10:000
62359990,5cyclictest0-21swapper/012:30:100
62359990,1cyclictest711-21runrttasks09:08:050
62359990,1cyclictest32511-21ssh09:56:160
62359990,1cyclictest31108-21/usr/sbin/munin09:40:170
62359990,1cyclictest19611-21taskset07:34:380
62359990,0cyclictest31237-21rm12:38:260
13333290,0sleep10-21swapper/110:35:181
62359981,4cyclictest510-21nscd11:00:030
62359981,4cyclictest0-21swapper/011:22:210
62359980,1cyclictest9762-21rm09:29:350
62359980,1cyclictest8693-21systemctl07:15:000
62359980,1cyclictest367-21munin-run09:25:010
62359980,1cyclictest30498-21cron09:40:000
62359980,1cyclictest26861-21apt-get10:10:140
62359980,1cyclictest26296-21sh11:30:200
62359980,1cyclictest2305-21latency_hist08:05:000
62359980,1cyclictest20670-21ssh10:55:150
62359980,1cyclictest14989-21diskstats10:20:130
62359980,1cyclictest14865-21ssh09:15:220
62359980,0cyclictest24732-21ssh11:13:490
62359980,0cyclictest18919-21apt-get09:50:130
62359980,0cyclictest15825-21ssh09:48:240
62359980,0cyclictest14773-21sh10:35:460
22860280,0sleep10-21swapper/112:17:351
62359971,5cyclictest27701-21aten2_r5power_c08:55:130
62359971,5cyclictest20938-21tune2fs08:40:110
62359971,1cyclictest7162-21ssh11:05:100
62359970,1cyclictest9141-21latency_hist10:50:000
62359970,1cyclictest5927-21tr12:25:150
62359970,1cyclictest31308-21cut07:55:160
62359970,1cyclictest30966-21ssh11:49:240
62359970,1cyclictest2900-21ssh09:10:100
62359970,1cyclictest28859-21cat12:20:180
62359970,1cyclictest25745-21df_inode08:50:140
62359970,1cyclictest24860-21apt-get07:45:130
62359970,1cyclictest21695-21sed12:00:190
62359970,1cyclictest20922-21sh10:06:390
62359970,1cyclictest2018-21apt-get10:30:120
62359970,1cyclictest19843-21basename08:40:000
62359970,1cyclictest15949-21hddtemp_smartct08:30:130
62359970,1cyclictest14596-21ssh11:25:010
62359970,1cyclictest1450-21ssh11:50:190
62359970,1cyclictest14332-21awk07:25:000
62359970,1cyclictest13778-21ssh11:40:170
62359970,1cyclictest11848-21munin-run07:20:000
62359970,1cyclictest11448-21ntp_states11:55:160
62359970,1cyclictest11020-21ssh09:30:120
62359970,1cyclictest10743-21ssh10:18:030
62359970,1cyclictest10479-21perl10:50:150
62359970,0cyclictest7474-21apt-get12:10:110
62359970,0cyclictest4690-21ssh11:35:320
62359970,0cyclictest30177-21apt-get09:00:110
62359970,0cyclictest2467-21diskmemload11:18:580
62359970,0cyclictest24478-21diskmemload10:40:340
62359970,0cyclictest24389-21apt-get10:25:120
62359970,0cyclictest17348-21apt-get12:15:120
62359970,0cyclictest13610-21ssh10:03:200
62359970,0cyclictest10182-21apt-get08:20:130
62359970,0cyclictest0-21swapper/007:40:120
62359960,1cyclictest22862-21egrep08:45:110
62359960,1cyclictest12666-21cut08:25:100
62359960,1cyclictest0-21swapper/008:10:110
62359960,1cyclictest0-21swapper/007:50:110
62359960,0cyclictest19895-21apt-get07:35:110
7954250,0sleep20-21swapper/210:16:162
62459952,3cyclictest16018-21ntpq12:30:183
62459951,4cyclictest4798-21sed09:10:203
62459950,5cyclictest0-21swapper/311:10:093
62459950,5cyclictest0-21swapper/308:10:113
62459950,5cyclictest0-21swapper/308:05:013
62459950,4cyclictest6050-21rm11:52:543
62459950,4cyclictest1612-21perl09:25:143
62459950,4cyclictest12442-21sh10:19:063
62459950,3cyclictest0-21swapper/312:05:273
62459950,3cyclictest0-21swapper/312:05:263
62419952,3cyclictest10853-21ssh11:06:192
62419951,2cyclictest0-21swapper/208:10:122
62419950,5cyclictest0-21swapper/212:38:022
62419950,5cyclictest0-21swapper/212:07:362
62419950,5cyclictest0-21swapper/212:07:362
62419950,5cyclictest0-21swapper/211:27:362
62419950,5cyclictest0-21swapper/209:19:332
62419950,3cyclictest29522-21ssh11:32:012
62419950,0cyclictest0-21swapper/212:00:112
62399952,2cyclictest0-21swapper/111:35:111
62399951,2cyclictest0-21swapper/111:30:141
62399951,1cyclictest0-21swapper/110:20:121
31175250,0sleep10-21swapper/112:38:231
62459944,0cyclictest0-21swapper/310:45:073
62459943,1cyclictest6791-21/usr/sbin/munin10:00:193
62459943,1cyclictest17585-21/usr/sbin/munin08:35:163
62459942,2cyclictest0-21swapper/308:55:163
62459942,1cyclictest714-21snmpd11:48:333
62459942,1cyclictest714-21snmpd09:30:473
62459942,1cyclictest714-21snmpd07:25:103
62459942,1cyclictest30255-21grep07:55:103
62459942,1cyclictest25982-21ssh10:09:503
62459941,3cyclictest714-21snmpd09:40:563
62459941,1cyclictest0-21swapper/310:10:143
62459940,4cyclictest19051-21ssh10:38:273
62459940,4cyclictest0-21swapper/312:35:123
62459940,4cyclictest0-21swapper/312:20:103
62459940,4cyclictest0-21swapper/312:15:123
62459940,4cyclictest0-21swapper/312:10:013
62459940,4cyclictest0-21swapper/311:20:133
62459940,4cyclictest0-21swapper/311:07:293
62459940,4cyclictest0-21swapper/311:00:133
62459940,4cyclictest0-21swapper/310:41:573
62459940,4cyclictest0-21swapper/310:25:123
62459940,4cyclictest0-21swapper/310:20:123
62459940,4cyclictest0-21swapper/309:55:033
62459940,4cyclictest0-21swapper/309:35:103
62459940,4cyclictest0-21swapper/309:05:103
62459940,4cyclictest0-21swapper/309:00:113
62459940,4cyclictest0-21swapper/308:53:083
62459940,4cyclictest0-21swapper/308:30:113
62459940,4cyclictest0-21swapper/308:20:133
62459940,4cyclictest0-21swapper/307:49:383
62459940,4cyclictest0-21swapper/307:35:113
62459940,4cyclictest0-21swapper/307:10:123
62459940,3cyclictest8464-21mktemp09:45:003
62459940,3cyclictest6324-21ssh11:36:323
62459940,3cyclictest5766-21/usr/sbin/munin12:25:133
62459940,3cyclictest30736-21ssh11:16:103
62459940,3cyclictest29973-21ssh11:32:183
62459940,3cyclictest24530-21sh09:52:353
62459940,3cyclictest23287-21ssh12:01:083
62459940,3cyclictest21740-21users10:55:213
62459940,3cyclictest16992-21apt-get07:30:013
62459940,3cyclictest16038-21irqstats11:25:183
62459940,3cyclictest15808-21ssh10:52:553
62459940,3cyclictest10031-21sh11:55:103
62459940,3cyclictest0-21swapper/308:45:103
62459940,1cyclictest32481-21/usr/sbin/munin08:00:133
62459940,1cyclictest22384-21/usr/sbin/munin07:40:203
62459940,1cyclictest2011-21/usr/sbin/munin10:30:173
62459940,1cyclictest12641-21/usr/sbin/munin08:25:113
62419944,0cyclictest5852-21ssh11:52:472
62419944,0cyclictest0-21swapper/212:30:362
62419944,0cyclictest0-21swapper/212:15:452
62419944,0cyclictest0-21swapper/209:35:262
62419944,0cyclictest0-21swapper/208:05:382
62419943,1cyclictest8585-21rm09:45:012
62419943,1cyclictest13037-21grep09:30:212
62419942,1cyclictest714-21snmpd07:35:282
62419942,1cyclictest20601-21ssh09:50:282
62419940,4cyclictest15801-21/usr/sbin/munin10:20:172
62419940,4cyclictest0-21swapper/211:38:302
62419940,4cyclictest0-21swapper/211:15:112
62419940,4cyclictest0-21swapper/210:57:282
62419940,4cyclictest0-21swapper/210:52:322
62419940,4cyclictest0-21swapper/210:45:282
62419940,4cyclictest0-21swapper/210:40:132
62419940,4cyclictest0-21swapper/210:28:532
62419940,4cyclictest0-21swapper/210:00:122
62419940,4cyclictest0-21swapper/209:20:112
62419940,4cyclictest0-21swapper/209:10:122
62419940,4cyclictest0-21swapper/209:05:172
62419940,4cyclictest0-21swapper/208:35:362
62419940,4cyclictest0-21swapper/207:10:132
62419940,3cyclictest9285-21users07:15:012
62419940,3cyclictest9111-21ssh12:26:532
62419940,3cyclictest5920-21rm09:59:562
62419940,3cyclictest32248-21ssh12:22:152
62419940,3cyclictest29704-21/usr/sbin/munin11:00:172
62419940,3cyclictest22748-21apt-get08:45:102
62419940,3cyclictest21546-21ntp_states08:40:152
62419940,3cyclictest19849-21ssh11:43:472
62419940,3cyclictest19661-21munin-run07:35:002
62419940,3cyclictest14537-21date11:25:012
62419940,3cyclictest12053-21/usr/sbin/munin10:35:202
62419940,3cyclictest11581-21ssh12:11:402
62419940,3cyclictest10045-21ssh11:55:102
62419940,3cyclictest0-21swapper/208:50:122
62399942,1cyclictest0-21swapper/107:20:121
62399941,2cyclictest0-21swapper/112:30:111
62399941,2cyclictest0-21swapper/111:45:151
62399941,2cyclictest0-21swapper/111:25:141
62399941,2cyclictest0-21swapper/109:50:131
62399941,2cyclictest0-21swapper/109:30:121
62399941,2cyclictest0-21swapper/109:20:111
62399941,2cyclictest0-21swapper/109:05:101
62399941,2cyclictest0-21swapper/108:55:121
62399941,2cyclictest0-21swapper/108:45:111
62399941,2cyclictest0-21swapper/108:20:131
62399941,2cyclictest0-21swapper/107:50:121
62399941,1cyclictest0-21swapper/107:55:091
62399940,3cyclictest0-21swapper/111:10:101
9981230,0sleep10-21swapper/111:55:091
62459932,1cyclictest714-21snmpd07:50:103
62459930,3cyclictest8792-21munin-node07:15:013
62459930,3cyclictest0-21swapper/308:40:013
62459930,3cyclictest0-21swapper/308:15:143
62459930,3cyclictest0-21swapper/307:20:113
62419932,1cyclictest714-21snmpd08:55:042
62419932,1cyclictest1792-21ssh10:30:042
62419932,1cyclictest17111-21ssh11:10:092
62419931,1cyclictest920-21cut09:25:112
62419930,3cyclictest30877-21ssh09:40:052
62419930,3cyclictest26567-21ssh10:10:032
62419930,3cyclictest16539-21ssh10:05:022
62419930,3cyclictest15196-21grep08:30:102
62419930,3cyclictest0-21swapper/211:45:012
62419930,3cyclictest0-21swapper/209:00:072
62419930,3cyclictest0-21swapper/208:25:082
62419930,3cyclictest0-21swapper/208:20:132
62419930,3cyclictest0-21swapper/208:15:142
62419930,3cyclictest0-21swapper/208:00:112
62419930,3cyclictest0-21swapper/207:55:012
62419930,3cyclictest0-21swapper/207:50:112
62419930,3cyclictest0-21swapper/207:45:082
62419930,3cyclictest0-21swapper/207:40:132
62419930,3cyclictest0-21swapper/207:25:012
62419930,3cyclictest0-21swapper/207:20:132
62399931,1cyclictest0-21swapper/112:10:111
62399931,1cyclictest0-21swapper/112:00:111
62399931,1cyclictest0-21swapper/111:40:111
62399931,1cyclictest0-21swapper/111:15:111
62399931,1cyclictest0-21swapper/110:55:121
62399931,1cyclictest0-21swapper/110:30:121
62399931,1cyclictest0-21swapper/110:25:111
62399931,1cyclictest0-21swapper/110:00:111
62399931,1cyclictest0-21swapper/109:55:121
62399931,1cyclictest0-21swapper/109:35:101
62399931,1cyclictest0-21swapper/108:35:141
62399931,1cyclictest0-21swapper/108:10:111
62399931,1cyclictest0-21swapper/108:05:101
62399931,1cyclictest0-21swapper/107:45:131
62399931,1cyclictest0-21swapper/107:35:111
62399931,1cyclictest0-21swapper/107:30:011
62399931,1cyclictest0-21swapper/107:25:111
62399930,3cyclictest0-21swapper/110:47:381
62399930,2cyclictest14758-21chrt08:28:381
62399930,2cyclictest0-21swapper/112:25:091
62399930,2cyclictest0-21swapper/112:05:011
62399930,2cyclictest0-21swapper/112:05:011
62399930,2cyclictest0-21swapper/110:05:121
62399930,2cyclictest0-21swapper/109:45:011
62399930,2cyclictest0-21swapper/109:15:121
62399930,2cyclictest0-21swapper/108:50:121
62399930,2cyclictest0-21swapper/108:15:141
62399930,1cyclictest0-21swapper/110:10:141
6055230,0sleep10-21swapper/111:20:171
62399921,1cyclictest0-21swapper/109:25:111
62399921,0cyclictest0-21swapper/108:30:101
62399920,1cyclictest7317-21chrt09:11:461
62399920,1cyclictest7130-21chrt11:05:081
62399920,1cyclictest0-21swapper/112:20:101
62399920,1cyclictest0-21swapper/111:50:131
62399920,1cyclictest0-21swapper/111:00:141
62399920,1cyclictest0-21swapper/110:50:131
62399920,1cyclictest0-21swapper/110:15:121
62399920,1cyclictest0-21swapper/109:40:151
62399920,1cyclictest0-21swapper/109:00:111
62399920,1cyclictest0-21swapper/108:40:091
62399920,1cyclictest0-21swapper/107:40:121
62399920,1cyclictest0-21swapper/107:10:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional