You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-19 - 02:55
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot2.osadl.org (updated Wed Nov 19, 2025 00:44:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
281652203187,12sleep10-21swapper/119:07:041
281052190174,11sleep30-21swapper/319:06:163
282262182165,12sleep00-21swapper/019:07:500
281512177160,12sleep20-21swapper/219:06:522
50522860,0sleep20-21swapper/200:31:362
51342830,0sleep10-21swapper/100:31:381
312342450,0sleep10-21swapper/121:29:401
194782440,0sleep10-21swapper/122:45:021
167742440,0sleep10-21swapper/122:10:221
3722330,1sleep12857499cyclictest23:40:191
285849992,1cyclictest16616-21cron23:00:013
285849990,4cyclictest6139-21sh23:43:253
285849960,5cyclictest0-21swapper/322:50:133
285749962,2cyclictest0-21swapper/123:25:131
285699962,4cyclictest31823-21kworker/0:122:56:480
285699962,4cyclictest31823-21kworker/0:100:33:260
285699961,5cyclictest31823-21kworker/0:100:20:150
285699961,4cyclictest31823-21kworker/0:123:05:030
285699960,5cyclictest0-21swapper/023:40:140
285849953,1cyclictest1993-21/usr/sbin/munin19:20:213
285849950,5cyclictest0-21swapper/323:37:403
285849950,5cyclictest0-21swapper/320:30:123
285849950,3cyclictest23638-21basename00:25:133
285799950,5cyclictest0-21swapper/223:10:212
285799950,5cyclictest0-21swapper/222:17:002
285799950,0cyclictest0-21swapper/200:39:352
285749951,3cyclictest0-21swapper/123:30:121
285749951,2cyclictest0-21swapper/122:50:141
285749951,2cyclictest0-21swapper/120:25:121
285749951,2cyclictest0-21swapper/100:15:141
285749950,5cyclictest0-21swapper/123:39:101
285749950,0cyclictest0-21swapper/120:35:241
285699955,0cyclictest0-21swapper/023:46:420
285699952,3cyclictest31823-21kworker/0:121:05:230
285699952,3cyclictest31823-21kworker/0:119:25:200
285699952,3cyclictest31823-21kworker/0:100:35:120
285699951,4cyclictest31823-21kworker/0:120:30:140
285699951,4cyclictest31811-21munin-run21:30:000
285699951,1cyclictest1142-21cat23:24:560
285699950,5cyclictest0-21swapper/023:03:040
285699950,4cyclictest0-21swapper/021:15:130
12783250,0chrt12784-21diskmemload21:52:273
9801240,0sleep00-21swapper/000:01:160
285849944,0cyclictest0-21swapper/323:52:303
285849944,0cyclictest0-21swapper/323:30:163
285849944,0cyclictest0-21swapper/320:20:133
285849944,0cyclictest0-21swapper/300:30:553
285849943,1cyclictest28679-21/usr/sbin/munin19:10:203
285849942,1cyclictest9890-21/usr/sbin/munin22:40:183
285849942,1cyclictest9588-21/usr/sbin/munin20:40:233
285849942,1cyclictest2529-21ssh23:57:553
285849942,1cyclictest15972-21ssh22:26:233
285849941,3cyclictest0-21swapper/322:46:183
285849940,4cyclictest30106-21ls21:45:163
285849940,4cyclictest14314-21latency_hist20:50:003
285849940,4cyclictest0-21swapper/323:25:123
285849940,4cyclictest0-21swapper/323:22:063
285849940,4cyclictest0-21swapper/323:13:573
285849940,4cyclictest0-21swapper/322:31:133
285849940,4cyclictest0-21swapper/322:23:103
285849940,4cyclictest0-21swapper/321:20:133
285849940,4cyclictest0-21swapper/320:00:153
285849940,4cyclictest0-21swapper/300:00:123
285849940,3cyclictest4661-21sh22:05:043
285849940,3cyclictest32488-21ssh22:02:163
285849940,3cyclictest32403-21ssh21:30:153
285849940,3cyclictest27462-21diskstats20:10:173
285849940,3cyclictest24720-21sh21:42:073
285849940,3cyclictest22750-21cut21:25:163
285849940,3cyclictest20130-21hddtemp_smartct19:55:163
285849940,3cyclictest17680-21sh23:16:063
285849940,3cyclictest16675-21cat22:10:233
285849940,3cyclictest14362-21apt-get19:45:143
285849940,3cyclictest13589-21/usr/sbin/munin00:20:133
285849940,3cyclictest11691-21ssh21:19:583
285849940,3cyclictest10629-21ssh00:35:003
285849940,3cyclictest10576-21ssh00:18:273
285849940,1cyclictest9164-21/usr/sbin/munin23:45:133
285849940,1cyclictest7094-21/usr/sbin/munin20:35:143
285849940,1cyclictest29410-21/usr/sbin/munin20:15:203
285849940,1cyclictest25627-21hddtemp_smartct21:10:163
285849940,1cyclictest19477-21/usr/sbin/munin21:00:133
285849940,1cyclictest17207-21/usr/sbin/munin23:00:193
285849940,1cyclictest1632-21sendmail_mailtr22:35:233
285849940,1cyclictest0-21swapper/300:05:123
285849940,0cyclictest0-21swapper/321:35:213
285849940,0cyclictest0-21swapper/321:09:593
285799944,0cyclictest0-21swapper/219:24:332
285799944,0cyclictest0-21swapper/200:12:222
285799943,1cyclictest7852-21gltestperf20:35:162
285799941,2cyclictest0-21swapper/220:25:132
285799941,2cyclictest0-21swapper/200:25:142
285799941,1cyclictest0-21swapper/220:45:132
285799941,1cyclictest0-21swapper/219:40:122
285799940,4cyclictest3756-21apt-get00:15:142
285799940,4cyclictest2839-21ssh23:58:072
285799940,4cyclictest0-21swapper/223:40:152
285799940,4cyclictest0-21swapper/223:20:132
285799940,4cyclictest0-21swapper/223:03:152
285799940,4cyclictest0-21swapper/222:55:132
285799940,4cyclictest0-21swapper/222:02:012
285799940,4cyclictest0-21swapper/221:54:022
285799940,4cyclictest0-21swapper/221:36:062
285799940,4cyclictest0-21swapper/221:20:142
285799940,4cyclictest0-21swapper/221:10:132
285799940,4cyclictest0-21swapper/220:50:132
285799940,4cyclictest0-21swapper/219:30:162
285799940,4cyclictest0-21swapper/200:05:132
285799940,3cyclictest5955-21ssh00:00:012
285799940,3cyclictest32392-21/usr/sbin/munin21:30:192
285799940,3cyclictest31858-21sort20:20:002
285799940,3cyclictest3133-21needreboot23:25:182
285799940,3cyclictest29912-21/usr/sbin/munin21:45:152
285799940,3cyclictest27215-21ssh22:48:472
285799940,3cyclictest26699-21cat20:10:012
285799940,3cyclictest25525-21ssh23:36:342
285799940,3cyclictest20370-21ssh22:29:122
285799940,3cyclictest14213-21ssh23:15:032
285799940,3cyclictest14052-21ssh23:30:322
285799940,3cyclictest13892-21/usr/sbin/munin00:20:132
285799940,3cyclictest0-21swapper/222:30:132
285799940,2cyclictest0-21swapper/222:11:352
285799940,1cyclictest31542-21perl19:15:012
285799940,1cyclictest299022ssh21:28:492
285799940,0cyclictest0-21swapper/223:53:362
285799940,0cyclictest0-21swapper/222:50:012
285799940,0cyclictest0-21swapper/222:36:472
285749944,0cyclictest0-21swapper/123:17:391
285749944,0cyclictest0-21swapper/123:08:451
285749944,0cyclictest0-21swapper/122:59:091
285749944,0cyclictest0-21swapper/122:24:311
285749941,2cyclictest0-21swapper/123:10:201
285749941,2cyclictest0-21swapper/122:35:141
285749941,2cyclictest0-21swapper/122:30:131
285749941,2cyclictest0-21swapper/121:45:141
285749941,2cyclictest0-21swapper/119:50:131
285749941,2cyclictest0-21swapper/119:35:131
285749941,2cyclictest0-21swapper/100:05:131
285749941,1cyclictest0-21swapper/121:40:131
285749941,1cyclictest0-21swapper/120:00:151
285749940,4cyclictest0-21swapper/121:39:511
285749940,4cyclictest0-21swapper/100:22:401
285699944,0cyclictest0-21swapper/023:31:080
285699944,0cyclictest0-21swapper/022:40:130
285699944,0cyclictest0-21swapper/021:45:220
285699944,0cyclictest0-21swapper/021:11:370
285699942,2cyclictest31823-21kworker/0:122:39:280
285699942,2cyclictest31823-21kworker/0:121:00:140
285699941,3cyclictest31823-21kworker/0:122:15:190
285699941,3cyclictest31823-21kworker/0:121:20:150
285699941,1cyclictest0-21swapper/000:15:150
285699940,4cyclictest31823-21kworker/0:123:52:590
285699940,4cyclictest31823-21kworker/0:121:42:240
285699940,4cyclictest31823-21kworker/0:121:30:230
285699940,4cyclictest31823-21kworker/0:100:27:180
285699940,4cyclictest0-21swapper/023:25:140
285699940,4cyclictest0-21swapper/023:15:140
285699940,4cyclictest0-21swapper/022:50:130
285699940,4cyclictest0-21swapper/022:30:150
285699940,4cyclictest0-21swapper/022:25:310
285699940,4cyclictest0-21swapper/022:20:140
285699940,4cyclictest0-21swapper/022:09:170
285699940,4cyclictest0-21swapper/019:30:150
285699940,3cyclictest30526-21users19:10:200
285699940,3cyclictest27887-21ssh00:10:220
285699940,3cyclictest26999-21sed22:00:010
285699940,3cyclictest1905-21wc19:20:000
285699940,3cyclictest13358-21sh23:14:390
285699940,0cyclictest0-21swapper/000:09:590
13900240,3sleep20-21swapper/223:47:042
285849933,0cyclictest0-21swapper/319:30:013
285849932,1cyclictest12052-21ls19:40:123
285849931,1cyclictest14525-21apt-get20:50:143
285849930,3cyclictest31410-21ntp_kernel_pll_19:15:013
285849930,3cyclictest27080-21/usr/sbin/munin23:05:123
285849930,3cyclictest26189-21sh00:10:103
285849930,3cyclictest25062-21expr22:15:143
285849930,3cyclictest17278-21ssh21:55:083
285849930,3cyclictest0-21swapper/320:55:123
285849930,3cyclictest0-21swapper/320:25:113
285849930,3cyclictest0-21swapper/320:05:043
285849930,3cyclictest0-21swapper/319:35:043
285849930,3cyclictest0-21swapper/319:25:143
285849930,1cyclictest0-21swapper/319:50:143
285799932,1cyclictest4710-21ssh22:05:052
285799932,1cyclictest23315-21pmu-power21:05:212
285799931,1cyclictest714-21snmpd21:00:002
285799931,1cyclictest0-21swapper/220:40:152
285799930,3cyclictest26927-21ssh23:05:072
285799930,3cyclictest26788-21apt-config20:10:002
285799930,3cyclictest2333-21ssh22:20:132
285799930,3cyclictest19879-21rm21:40:112
285799930,3cyclictest10838-21grep19:35:212
285799930,3cyclictest0-21swapper/222:40:022
285799930,3cyclictest0-21swapper/221:55:052
285799930,3cyclictest0-21swapper/221:15:132
285799930,3cyclictest0-21swapper/220:55:132
285799930,3cyclictest0-21swapper/220:30:132
285799930,3cyclictest0-21swapper/220:20:122
285799930,3cyclictest0-21swapper/220:00:162
285799930,3cyclictest0-21swapper/219:55:122
285799930,3cyclictest0-21swapper/219:50:132
285799930,3cyclictest0-21swapper/219:45:112
285799930,3cyclictest0-21swapper/219:25:132
285799930,3cyclictest0-21swapper/219:10:112
285749933,0cyclictest0-21swapper/121:11:111
285749933,0cyclictest0-21swapper/100:00:501
285749931,1cyclictest0-21swapper/123:45:131
285749931,1cyclictest0-21swapper/123:20:141
285749931,1cyclictest0-21swapper/123:00:141
285749931,1cyclictest0-21swapper/122:15:131
285749931,1cyclictest0-21swapper/122:05:141
285749931,1cyclictest0-21swapper/122:00:161
285749931,1cyclictest0-21swapper/121:55:141
285749931,1cyclictest0-21swapper/121:30:161
285749931,1cyclictest0-21swapper/120:50:131
285749931,1cyclictest0-21swapper/120:45:131
285749931,1cyclictest0-21swapper/120:15:151
285749931,1cyclictest0-21swapper/120:10:161
285749931,1cyclictest0-21swapper/120:05:121
285749931,1cyclictest0-21swapper/119:40:131
285749931,1cyclictest0-21swapper/119:15:151
285749931,1cyclictest0-21swapper/119:10:121
285749931,1cyclictest0-21swapper/100:25:131
285749930,2cyclictest16408-21chrt19:47:471
285749930,2cyclictest0-21swapper/121:15:131
285749930,2cyclictest0-21swapper/120:55:121
285749930,2cyclictest0-21swapper/119:20:141
285699931,2cyclictest31823-21kworker/0:123:55:110
285699930,3cyclictest9591-21apt-get20:40:160
285699930,3cyclictest7654-21cut20:35:150
285699930,3cyclictest32304-21cat20:20:140
285699930,3cyclictest29410-21/usr/sbin/munin20:15:190
285699930,3cyclictest24187-21diskmemload23:35:070
285699930,3cyclictest22314-21df_inode20:00:170
285699930,3cyclictest17568-21grep19:50:170
285699930,3cyclictest16992-21apt-get20:55:120
285699930,3cyclictest12420-21diskstats19:40:130
285699930,3cyclictest0-21swapper/022:45:050
285699930,3cyclictest0-21swapper/022:10:130
285699930,3cyclictest0-21swapper/021:50:030
285699930,3cyclictest0-21swapper/021:35:140
285699930,3cyclictest0-21swapper/020:50:130
285699930,3cyclictest0-21swapper/020:45:050
285699930,3cyclictest0-21swapper/020:25:000
285699930,3cyclictest0-21swapper/020:10:010
285699930,3cyclictest0-21swapper/020:05:120
285699930,3cyclictest0-21swapper/019:55:050
285699930,3cyclictest0-21swapper/019:45:140
285699930,3cyclictest0-21swapper/019:35:130
285699930,3cyclictest0-21swapper/019:20:150
285699930,2cyclictest27244-21ssh22:00:080
285749921,1cyclictest90332chrt19:33:291
285749921,1cyclictest0-21swapper/122:40:141
285749921,1cyclictest0-21swapper/121:00:131
285749921,1cyclictest0-21swapper/120:30:121
285749921,1cyclictest0-21swapper/119:25:131
285749920,2cyclictest14512sleep120:21:441
285749920,1cyclictest21931-21chrt21:05:021
285749920,1cyclictest0-21swapper/123:55:141
285749920,1cyclictest0-21swapper/123:50:111
285749920,1cyclictest0-21swapper/122:25:131
285749920,1cyclictest0-21swapper/121:50:091
285749920,1cyclictest0-21swapper/120:44:591
285749920,1cyclictest0-21swapper/119:55:121
285749920,1cyclictest0-21swapper/100:35:151
285749920,1cyclictest0-21swapper/100:10:151
285749910,1cyclictest0-21swapper/121:20:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional