You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-28 - 23:30
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot2.osadl.org (updated Sat Jun 28, 2025 12:44:39)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
165012196182,10sleep10-21swapper/107:06:271
167142192174,13sleep30-21swapper/307:09:153
167032191174,12sleep20-21swapper/207:09:052
167062186172,10sleep00-21swapper/007:09:070
78532500,0sleep30-21swapper/312:21:293
108952460,0sleep10-21swapper/112:23:261
316372450,0sleep20-21swapper/209:50:382
297172450,0sleep10-21swapper/109:18:061
254282450,0sleep10-21swapper/111:09:041
150612450,0sleep00-21swapper/011:35:490
56032440,0sleep10-21swapper/109:21:521
205392430,0sleep00-21swapper/009:13:410
166032420,0sleep10-21swapper/109:27:281
215992390,0sleep20-21swapper/208:20:232
169349980,4cyclictest25014-21perf08:30:000
169349980,4cyclictest20755-21munin-run09:30:020
169479960,5cyclictest0-21swapper/311:15:173
169409964,2cyclictest0-21swapper/211:51:282
169479955,0cyclictest0-21swapper/311:11:313
169479954,1cyclictest16470-21kworker/3:112:38:113
169479952,3cyclictest30997-21snmpd12:19:263
169479950,5cyclictest0-21swapper/312:10:053
169479950,0cyclictest0-21swapper/311:56:203
169409955,0cyclictest0-21swapper/209:20:352
169409950,4cyclictest4989-21latency_hist08:55:002
169409950,4cyclictest0-21swapper/212:00:152
169359951,3cyclictest0-21swapper/112:35:141
169359951,3cyclictest0-21swapper/111:00:171
169359951,3cyclictest0-21swapper/108:35:161
169359951,2cyclictest0-21swapper/110:05:141
169349955,0cyclictest0-21swapper/010:45:170
169349952,3cyclictest30997-21snmpd08:45:210
169349950,5cyclictest0-21swapper/012:19:520
169349950,5cyclictest0-21swapper/012:05:270
169349950,5cyclictest0-21swapper/011:51:300
169349950,0cyclictest0-21swapper/009:54:560
169479944,0cyclictest16470-21kworker/3:107:58:173
169479944,0cyclictest0-21swapper/311:00:113
169479944,0cyclictest0-21swapper/310:46:293
169479944,0cyclictest0-21swapper/309:56:093
169479943,1cyclictest24420-21perl12:30:153
169479943,1cyclictest21144-21unixbench-2d09:45:233
169479940,4cyclictest5434-21rm11:31:013
169479940,4cyclictest5099-21perl10:10:163
169479940,4cyclictest24321-21apt-get10:20:013
169479940,4cyclictest13688-21sh11:51:393
169479940,4cyclictest0-21swapper/311:40:123
169479940,4cyclictest0-21swapper/311:25:163
169479940,4cyclictest0-21swapper/310:56:243
169479940,4cyclictest0-21swapper/310:30:133
169479940,4cyclictest0-21swapper/309:35:163
169479940,4cyclictest0-21swapper/309:25:533
169479940,4cyclictest0-21swapper/307:50:173
169479940,3cyclictest9087-21latency_hist11:50:003
169479940,3cyclictest8445-21diskstats09:00:163
169479940,3cyclictest7935-21sh10:44:133
169479940,3cyclictest31685-21ntp_states07:35:203
169479940,3cyclictest29206-21rm12:00:083
169479940,3cyclictest28245-21ssh10:37:523
169479940,3cyclictest26187-21perl07:25:153
169479940,3cyclictest25747-21df08:30:153
169479940,3cyclictest21806-21sensors_temp08:20:233
169479940,3cyclictest20222-21perl07:15:013
169479940,3cyclictest19897-21sh10:50:133
169479940,3cyclictest18333-21sh11:21:143
169479940,3cyclictest16288-21ssh11:36:373
169479940,3cyclictest14525-21diskstats12:25:183
169479940,3cyclictest13836-21irqstats08:05:193
169479940,3cyclictest11088-21users09:40:263
169479940,1cyclictest29180-21/usr/sbin/munin09:50:183
169479940,1cyclictest28273-21ntp_states10:05:213
169479940,1cyclictest27933-21/usr/sbin/munin07:30:143
169479940,0cyclictest0-21swapper/312:07:153
169479940,0cyclictest0-21swapper/307:14:563
169409944,0cyclictest28364-21ssh12:32:052
169409944,0cyclictest0-21swapper/211:37:232
169409944,0cyclictest0-21swapper/211:15:412
169409944,0cyclictest0-21swapper/208:46:412
169409943,1cyclictest9665-21unixbench_multi09:00:232
169409943,1cyclictest31-21ksoftirqd/212:15:002
169409942,1cyclictest30997-21snmpd10:25:552
169409942,1cyclictest23328-21ssh09:15:152
169409941,3cyclictest2134-21ssh11:30:062
169409941,2cyclictest0-21swapper/208:00:132
169409941,2cyclictest0-21swapper/207:45:162
169409941,1cyclictest0-21swapper/209:30:172
169409940,4cyclictest7883-21rm09:55:172
169409940,4cyclictest7494-21chrt08:58:052
169409940,4cyclictest6862-21apt-get11:00:012
169409940,4cyclictest27536-21basename12:15:192
169409940,4cyclictest21918-21ssh11:55:362
169409940,4cyclictest13068-21ssh12:24:502
169409940,4cyclictest12626-21diskmemload10:10:142
169409940,4cyclictest12626-21diskmemload09:25:132
169409940,4cyclictest0-21swapper/211:41:002
169409940,4cyclictest0-21swapper/211:05:192
169409940,4cyclictest0-21swapper/210:55:132
169409940,4cyclictest0-21swapper/210:41:052
169409940,4cyclictest0-21swapper/210:37:232
169409940,4cyclictest0-21swapper/210:17:012
169409940,4cyclictest0-21swapper/209:41:062
169409940,4cyclictest0-21swapper/209:36:402
169409940,4cyclictest0-21swapper/208:30:132
169409940,4cyclictest0-21swapper/208:07:152
169409940,4cyclictest0-21swapper/207:35:312
169409940,3cyclictest30163-21sed08:40:002
169409940,3cyclictest27505-21ssh11:10:152
169409940,3cyclictest24036-21ntp_states08:25:232
169409940,3cyclictest23079-21ssh10:03:002
169409940,3cyclictest19820-21grep07:15:012
169409940,3cyclictest1974-21cat11:30:002
169409940,3cyclictest18787-21awk08:15:202
169409940,3cyclictest14526-21ssh12:25:192
169409940,3cyclictest12381-21awk10:30:132
169409940,3cyclictest12184-21ssh12:07:342
169409940,3cyclictest11653-21ntp_states09:05:212
169409940,3cyclictest10577-21perl10:45:172
169409940,3cyclictest0-21swapper/211:20:162
169409940,3cyclictest0-21swapper/210:05:012
169409940,3cyclictest0-21swapper/207:25:132
169409940,1cyclictest29325-21munin-run10:55:002
169409940,0cyclictest0-21swapper/209:46:042
169359944,0cyclictest0-21swapper/112:14:311
169359944,0cyclictest0-21swapper/111:42:131
169359943,1cyclictest0-21swapper/107:19:421
169359942,2cyclictest0-21swapper/109:55:131
169359942,1cyclictest0-21swapper/110:20:151
169359942,1cyclictest0-21swapper/107:50:171
169359941,2cyclictest0-21swapper/112:30:121
169359941,2cyclictest0-21swapper/111:25:151
169359941,2cyclictest0-21swapper/111:10:161
169359941,2cyclictest0-21swapper/110:50:011
169359941,2cyclictest0-21swapper/110:30:121
169359941,2cyclictest0-21swapper/110:10:131
169359941,2cyclictest0-21swapper/109:50:161
169359941,2cyclictest0-21swapper/109:45:151
169359941,2cyclictest0-21swapper/109:30:171
169359941,2cyclictest0-21swapper/109:10:141
169359941,2cyclictest0-21swapper/108:55:021
169359941,2cyclictest0-21swapper/107:55:131
169359941,2cyclictest0-21swapper/107:35:141
169359941,2cyclictest0-21swapper/107:25:131
169359940,4cyclictest0-21swapper/111:54:081
169359940,4cyclictest0-21swapper/111:48:101
169359940,2cyclictest0-21swapper/110:45:151
169359940,2cyclictest0-21swapper/110:25:161
169359940,0cyclictest0-21swapper/112:15:141
169359940,0cyclictest0-21swapper/112:00:501
169349944,0cyclictest0-21swapper/011:31:240
169349944,0cyclictest0-21swapper/011:25:260
169349944,0cyclictest0-21swapper/011:17:080
169349943,1cyclictest7674-21head09:55:170
169349942,2cyclictest30997-21snmpd11:00:070
169349942,2cyclictest30997-21snmpd07:14:570
169349940,4cyclictest15846-21diskstats08:10:140
169349940,4cyclictest0-21swapper/012:35:200
169349940,4cyclictest0-21swapper/012:33:140
169349940,4cyclictest0-21swapper/011:56:550
169349940,4cyclictest0-21swapper/011:40:120
169349940,4cyclictest0-21swapper/010:01:120
169349940,4cyclictest0-21swapper/009:40:330
169349940,4cyclictest0-21swapper/009:24:360
169349940,4cyclictest0-21swapper/007:20:150
169349940,3cyclictest8595-21cat09:40:010
169349940,3cyclictest8409-21df_inode07:55:140
169349940,3cyclictest4445-21ssh10:42:000
169349940,3cyclictest4366-21apt-get10:10:130
169349940,3cyclictest3714-21ssh12:20:010
169349940,3cyclictest32718-21ssh11:45:160
169349940,3cyclictest30819-21ssh10:55:180
169349940,3cyclictest27428-21diskstats10:05:160
169349940,3cyclictest27073-21users08:30:240
169349940,3cyclictest23485-21users09:30:280
169349940,3cyclictest20504-21sh11:05:560
169349940,3cyclictest20095-21hddtemp_smartct09:45:170
169349940,3cyclictest19983-21/usr/sbin/munin10:50:240
169349940,3cyclictest16519-21apt-get12:10:140
169349940,3cyclictest15215-21fschecks_time10:15:170
169349940,3cyclictest12626-21diskmemload09:15:120
169349940,3cyclictest11903-21python09:05:240
169349940,3cyclictest0-21swapper/010:35:150
169349940,1cyclictest6055-21grep07:50:200
169349940,1cyclictest22779-21awk07:20:000
169349940,1cyclictest15090-21munin-run08:10:010
169479932,1cyclictest30997-21snmpd08:00:033
169479932,1cyclictest1966-21ssh10:25:043
169479932,1cyclictest17336-21ssh11:05:043
169479931,2cyclictest0-21swapper/310:15:053
169479930,3cyclictest30615-21ls08:40:153
169479930,3cyclictest21021-21sh09:30:043
169479930,3cyclictest0-21swapper/310:00:183
169479930,3cyclictest0-21swapper/309:20:123
169479930,3cyclictest0-21swapper/309:05:163
169479930,3cyclictest0-21swapper/308:55:013
169479930,3cyclictest0-21swapper/308:50:133
169479930,3cyclictest0-21swapper/308:45:133
169479930,3cyclictest0-21swapper/308:35:083
169479930,3cyclictest0-21swapper/308:25:143
169479930,3cyclictest0-21swapper/308:15:073
169479930,3cyclictest0-21swapper/308:10:123
169479930,3cyclictest0-21swapper/307:45:073
169479930,3cyclictest0-21swapper/307:40:143
169479930,3cyclictest0-21swapper/307:20:073
169479930,2cyclictest0-21swapper/309:15:133
169479930,1cyclictest12973-21ssh09:10:073
169409932,1cyclictest7865-21taskset07:55:012
169409931,1cyclictest411-21cron12:35:002
169409930,3cyclictest24611-21ssh10:20:112
169409930,3cyclictest23562-21cut07:20:182
169409930,3cyclictest13053-21ssh09:10:102
169409930,3cyclictest0-21swapper/208:40:142
169409930,3cyclictest0-21swapper/208:10:122
169409930,3cyclictest0-21swapper/207:50:182
169409930,3cyclictest0-21swapper/207:40:132
169409930,3cyclictest0-21swapper/207:30:122
169409930,3cyclictest0-21swapper/207:10:002
169409930,2cyclictest31821-21rm11:45:012
169359933,0cyclictest211ktimersoftd/112:25:141
169359933,0cyclictest211ktimersoftd/110:55:471
169359933,0cyclictest211ktimersoftd/110:16:541
169359933,0cyclictest211ktimersoftd/109:35:291
169359933,0cyclictest211ktimersoftd/109:05:031
169359933,0cyclictest211ktimersoftd/108:50:091
169359933,0cyclictest211ktimersoftd/108:45:041
169359933,0cyclictest211ktimersoftd/108:40:041
169359933,0cyclictest211ktimersoftd/108:25:031
169359933,0cyclictest211ktimersoftd/108:15:301
169359933,0cyclictest211ktimersoftd/108:10:041
169359933,0cyclictest211ktimersoftd/108:05:521
169359933,0cyclictest211ktimersoftd/108:00:091
169359933,0cyclictest211ktimersoftd/107:45:061
169359933,0cyclictest211ktimersoftd/107:30:121
169359933,0cyclictest211ktimersoftd/107:20:051
169359931,1cyclictest0-21swapper/111:55:121
169359931,1cyclictest0-21swapper/110:40:161
169359931,1cyclictest0-21swapper/109:00:141
169359931,1cyclictest0-21swapper/108:30:131
169359931,1cyclictest0-21swapper/108:20:151
169359931,1cyclictest0-21swapper/107:40:131
169359931,1cyclictest0-21swapper/107:10:011
169359930,2cyclictest0-21swapper/111:35:141
169359930,2cyclictest0-21swapper/111:15:181
169359930,2cyclictest0-21swapper/110:00:171
169359930,1cyclictest0-21swapper/110:35:151
169349932,1cyclictest31322-21cat07:35:190
169349932,1cyclictest30997-21snmpd08:15:030
169349932,1cyclictest30997-21snmpd07:25:090
169349931,2cyclictest14965-21sh11:20:160
169349930,3cyclictest992-21df_inode07:40:150
169349930,3cyclictest4987-21apt-get08:55:000
169349930,3cyclictest3469-21df_inode07:45:160
169349930,3cyclictest31912-21cat08:40:260
169349930,3cyclictest29478-21awk08:35:260
169349930,3cyclictest20292-21apt-get08:20:150
169349930,3cyclictest11925-21ssh10:30:020
169349930,3cyclictest10373-21apt08:00:130
169349930,3cyclictest0-21swapper/012:25:040
169349930,3cyclictest0-21swapper/012:00:030
169349930,3cyclictest0-21swapper/011:10:150
169349930,3cyclictest0-21swapper/010:25:080
169349930,3cyclictest0-21swapper/009:00:150
169349930,3cyclictest0-21swapper/008:50:130
169349930,1cyclictest0-21swapper/007:30:130
169349930,0cyclictest24663-21apt-get10:20:140
169359920,1cyclictest0-21swapper/112:05:161
169359920,1cyclictest0-21swapper/111:30:081
169359920,1cyclictest0-21swapper/111:20:151
169359920,1cyclictest0-21swapper/109:40:161
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional