You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-30 - 12:32
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot2.osadl.org (updated Sun Nov 30, 2025 00:44:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
309062195179,11sleep10-21swapper/119:08:401
307502190172,13sleep30-21swapper/319:06:403
307592180164,12sleep00-21swapper/019:06:460
306922171153,13sleep20-21swapper/219:05:562
125892450,0sleep00-21swapper/022:23:020
25142430,0sleep10-21swapper/123:39:421
161562120,0sleep00-21swapper/019:40:200
24632100,0sleep10-21swapper/123:55:191
27674290,0sleep30-21swapper/322:46:313
12451290,0sleep20-21swapper/223:11:212
18385280,0sleep10-21swapper/121:21:061
9257260,0sleep00-21swapper/000:31:330
311939962,4cyclictest23348-21kworker/0:119:55:210
311939961,5cyclictest23348-21kworker/0:123:25:260
311939960,6cyclictest23348-21kworker/0:120:30:140
311939960,3cyclictest3047-21munin-run23:40:000
311939960,3cyclictest20026-21perf00:04:590
16352260,0sleep30-21swapper/300:35:173
312069955,0cyclictest1323-21kworker/3:023:17:443
312069955,0cyclictest1323-21kworker/3:023:06:333
312069953,1cyclictest13813-21ssh22:40:123
312069950,5cyclictest0-21swapper/321:20:073
312069950,5cyclictest0-21swapper/320:15:053
312069950,5cyclictest0-21swapper/319:23:333
312069950,5cyclictest0-21swapper/300:03:393
312019955,0cyclictest0-21swapper/222:41:552
312019953,2cyclictest13355-21kworker/2:100:36:242
312019951,3cyclictest714-21snmpd22:53:492
312019950,5cyclictest0-21swapper/223:53:112
312019950,5cyclictest0-21swapper/223:03:172
312019950,5cyclictest0-21swapper/222:57:102
312019950,5cyclictest0-21swapper/221:13:262
311979951,2cyclictest0-21swapper/119:15:151
311979950,5cyclictest0-21swapper/122:31:001
311939955,0cyclictest23005-21ssh22:11:480
311939952,3cyclictest23348-21kworker/0:122:30:180
311939952,3cyclictest23348-21kworker/0:121:44:540
311939951,4cyclictest23348-21kworker/0:122:42:330
311939951,4cyclictest23348-21kworker/0:120:10:170
311939950,5cyclictest23348-21kworker/0:123:23:060
311939950,5cyclictest23348-21kworker/0:123:23:050
311939950,5cyclictest0-21swapper/023:50:130
311939950,5cyclictest0-21swapper/022:02:040
311939950,5cyclictest0-21swapper/021:21:020
311939950,5cyclictest0-21swapper/021:15:130
311939950,4cyclictest4773-21apt-get20:25:120
311939950,4cyclictest31300-21apt-get19:10:120
6172240,0sleep10-21swapper/123:40:431
312069944,0cyclictest0-21swapper/323:49:243
312069944,0cyclictest0-21swapper/322:35:043
312069944,0cyclictest0-21swapper/322:23:423
312069944,0cyclictest0-21swapper/321:34:573
312069944,0cyclictest0-21swapper/320:41:323
312069943,1cyclictest3375-21users22:50:223
312069943,1cyclictest2634-21cut20:20:153
312069942,2cyclictest24551-21memory23:50:183
312069942,1cyclictest8972-21irqstats00:15:163
312069942,1cyclictest22228-21/usr/sbin/munin21:55:193
312069942,1cyclictest0-21swapper/320:55:133
312069941,3cyclictest6626-21cat23:25:153
312069941,2cyclictest711-21runrttasks20:45:133
312069940,4cyclictest8823-21apt-get23:10:113
312069940,4cyclictest6696-21ntp_states00:30:183
312069940,4cyclictest30389-21ssh23:20:203
312069940,4cyclictest30389-21ssh23:20:203
312069940,4cyclictest28045-21ssh23:03:163
312069940,4cyclictest22116-21/usr/sbin/munin19:55:163
312069940,4cyclictest21297-21ssh22:10:443
312069940,4cyclictest10417-21wc22:05:223
312069940,4cyclictest0-21swapper/323:40:143
312069940,4cyclictest0-21swapper/323:30:173
312069940,4cyclictest0-21swapper/322:59:563
312069940,4cyclictest0-21swapper/322:30:123
312069940,4cyclictest0-21swapper/321:25:123
312069940,4cyclictest0-21swapper/320:30:293
312069940,4cyclictest0-21swapper/320:10:163
312069940,4cyclictest0-21swapper/320:00:013
312069940,4cyclictest0-21swapper/319:30:153
312069940,4cyclictest0-21swapper/319:12:303
312069940,4cyclictest0-21swapper/300:20:133
312069940,4cyclictest0-21swapper/300:05:083
312069940,3cyclictest6408-21ssh23:57:373
312069940,3cyclictest6044-21ntp_states20:25:183
312069940,3cyclictest5910-21ssh00:14:043
312069940,3cyclictest32072-21memory22:00:193
312069940,3cyclictest30859-21ssh23:37:063
312069940,3cyclictest28004-21perl20:05:163
312069940,3cyclictest27406-21apt-get21:10:153
312069940,3cyclictest24682-21wget21:05:153
312069940,3cyclictest22639-21latency_hist21:40:013
312069940,3cyclictest19648-21/usr/sbin/munin19:50:163
312069940,3cyclictest16219-21/usr/sbin/munin22:25:193
312069940,3cyclictest14538-21ssh21:51:273
312069940,3cyclictest12474-21sh21:18:433
312069940,1cyclictest9706-21/usr/sbin/munin20:35:133
312069940,1cyclictest29745-21sed00:25:233
312069940,1cyclictest28696-21/usr/sbin/munin22:15:183
312069940,1cyclictest23376-21ntp_states21:00:193
312019944,0cyclictest31-21ksoftirqd/221:20:142
312019944,0cyclictest31-21ksoftirqd/219:15:012
312019944,0cyclictest0-21swapper/221:29:012
312019944,0cyclictest0-21swapper/200:02:572
312019943,1cyclictest31-21ksoftirqd/221:45:142
312019943,1cyclictest31-21ksoftirqd/200:30:132
312019943,1cyclictest31-21ksoftirqd/200:10:182
312019943,1cyclictest1632-21tr23:55:152
312019941,2cyclictest0-21swapper/219:35:152
312019941,2cyclictest0-21swapper/200:15:112
312019941,1cyclictest0-21swapper/221:40:132
312019940,4cyclictest131592sleep222:06:552
312019940,4cyclictest0-21swapper/223:15:132
312019940,4cyclictest0-21swapper/222:45:152
312019940,4cyclictest0-21swapper/222:36:172
312019940,4cyclictest0-21swapper/222:23:002
312019940,4cyclictest0-21swapper/222:18:202
312019940,4cyclictest0-21swapper/222:10:122
312019940,4cyclictest0-21swapper/222:00:142
312019940,4cyclictest0-21swapper/221:35:122
312019940,4cyclictest0-21swapper/221:30:122
312019940,4cyclictest0-21swapper/221:05:152
312019940,4cyclictest0-21swapper/221:00:162
312019940,4cyclictest0-21swapper/220:05:142
312019940,4cyclictest0-21swapper/219:55:122
312019940,4cyclictest0-21swapper/219:15:142
312019940,3cyclictest5610-21smartctl20:25:162
312019940,3cyclictest30422-21wc22:00:002
312019940,3cyclictest29405-21perl23:20:142
312019940,3cyclictest29405-21perl23:20:142
312019940,3cyclictest28338-21diskstats00:25:162
312019940,3cyclictest27215-21sed23:35:172
312019940,3cyclictest26531-21ssh00:07:512
312019940,3cyclictest25902-21cat22:30:002
312019940,3cyclictest23364-21rm00:22:332
312019940,3cyclictest10243-21ssh23:26:382
312019940,3cyclictest0-21swapper/220:20:142
312019940,0cyclictest0-21swapper/223:43:262
311979944,0cyclictest0-21swapper/123:00:501
311979944,0cyclictest0-21swapper/121:47:131
311979944,0cyclictest0-21swapper/119:40:201
311979943,1cyclictest0-21swapper/123:06:571
311979941,2cyclictest0-21swapper/123:50:121
311979941,2cyclictest0-21swapper/122:55:031
311979941,2cyclictest0-21swapper/122:50:141
311979941,2cyclictest0-21swapper/122:25:121
311979941,2cyclictest0-21swapper/120:25:121
311979941,2cyclictest0-21swapper/120:15:121
311979941,2cyclictest0-21swapper/119:25:121
311979941,2cyclictest0-21swapper/100:10:131
311979941,1cyclictest0-21swapper/100:20:121
311979940,4cyclictest0-21swapper/100:07:381
311979940,0cyclictest0-21swapper/100:39:471
311939944,0cyclictest0-21swapper/023:08:070
311939944,0cyclictest0-21swapper/022:49:470
311939944,0cyclictest0-21swapper/021:35:090
311939943,1cyclictest29524-21munin-run20:10:000
311939941,3cyclictest23348-21kworker/0:122:35:020
311939941,3cyclictest23348-21kworker/0:120:35:200
311939941,3cyclictest23348-21kworker/0:100:36:360
311939941,3cyclictest23348-21kworker/0:100:16:090
311939941,2cyclictest31220-21sh22:15:400
311939941,2cyclictest0-21swapper/021:55:120
311939941,2cyclictest0-21swapper/021:30:120
311939941,2cyclictest0-21swapper/000:10:130
311939941,1cyclictest0-21swapper/023:55:110
311939940,4cyclictest23348-21kworker/0:123:32:290
311939940,4cyclictest12223-21apt-get19:35:140
311939940,4cyclictest0-21swapper/023:19:530
311939940,4cyclictest0-21swapper/022:25:110
311939940,4cyclictest0-21swapper/021:45:130
311939940,4cyclictest0-21swapper/020:50:120
311939940,4cyclictest0-21swapper/000:24:570
311939940,3cyclictest7878-21apt-get22:05:020
311939940,3cyclictest3410-21ssh22:50:230
311939940,3cyclictest3280-21ssh23:40:010
311939940,3cyclictest29844-21ssh21:10:380
311939940,3cyclictest26841-21ssh21:25:200
311939940,3cyclictest26806-21diskmemload21:50:530
311939940,3cyclictest25099-21df_inode21:05:170
311939940,3cyclictest21083-21ssh23:00:020
311939940,3cyclictest14295-21perl23:45:160
311939940,3cyclictest12934-21sh23:11:380
311939940,0cyclictest0-21swapper/000:06:190
312069930,3cyclictest323-21date21:45:003
312069930,3cyclictest17527-21expr20:50:133
312069930,3cyclictest1640-21if_err_eth019:15:013
312069930,3cyclictest0-21swapper/319:45:133
312069930,3cyclictest0-21swapper/319:40:133
312069930,3cyclictest0-21swapper/319:35:013
312069930,3cyclictest0-21swapper/319:25:083
312069930,1cyclictest22836-21ssh21:40:043
312019933,0cyclictest13355-21kworker/2:120:40:162
312019933,0cyclictest13355-21kworker/2:119:40:092
312019933,0cyclictest13355-21kworker/2:119:30:022
312019932,1cyclictest714-21snmpd19:50:092
312019932,1cyclictest714-21snmpd19:25:032
312019932,1cyclictest15912-21ssh23:30:062
312019931,1cyclictest0-21swapper/220:50:132
312019931,1cyclictest0-21swapper/220:45:132
312019931,1cyclictest0-21swapper/220:35:122
312019931,1cyclictest0-21swapper/220:30:142
312019930,3cyclictest4827-21ssh21:15:052
312019930,3cyclictest31054-21ssh23:05:032
312019930,3cyclictest29826-21cut20:10:142
312019930,3cyclictest0-21swapper/223:45:072
312019930,3cyclictest0-21swapper/220:55:132
312019930,3cyclictest0-21swapper/220:15:122
312019930,3cyclictest0-21swapper/220:00:022
312019930,3cyclictest0-21swapper/219:45:132
312019930,3cyclictest0-21swapper/219:20:122
312019930,2cyclictest26109-21ssh22:30:082
312019930,2cyclictest10408-21ssh21:50:022
311979933,0cyclictest0-21swapper/123:47:161
311979933,0cyclictest0-21swapper/122:08:101
311979933,0cyclictest0-21swapper/121:11:251
311979933,0cyclictest0-21swapper/100:29:181
311979931,2cyclictest0-21swapper/121:50:131
311979931,1cyclictest0-21swapper/123:15:131
311979931,1cyclictest0-21swapper/123:10:121
311979931,1cyclictest0-21swapper/122:45:001
311979931,1cyclictest0-21swapper/122:15:151
311979931,1cyclictest0-21swapper/121:35:121
311979931,1cyclictest0-21swapper/121:30:131
311979931,1cyclictest0-21swapper/120:35:131
311979931,1cyclictest0-21swapper/119:30:141
311979931,1cyclictest0-21swapper/100:30:121
311979931,1cyclictest0-21swapper/100:15:111
311979931,1cyclictest0-21swapper/100:00:141
311979930,2cyclictest0-21swapper/123:25:131
311979930,2cyclictest0-21swapper/123:20:131
311979930,2cyclictest0-21swapper/123:20:121
311979930,2cyclictest0-21swapper/121:55:141
311979930,2cyclictest0-21swapper/120:40:141
311979930,2cyclictest0-21swapper/120:20:131
311979930,2cyclictest0-21swapper/119:55:121
311979930,1cyclictest0-21swapper/121:15:131
311939931,1cyclictest0-21swapper/020:55:130
311939930,3cyclictest9745-21grep19:30:150
311939930,3cyclictest3230-21if_eth020:20:160
311939930,3cyclictest17704-21df_inode19:45:150
311939930,3cyclictest1392-21grep19:15:000
311939930,3cyclictest0-21swapper/022:55:030
311939930,3cyclictest0-21swapper/021:00:130
311939930,3cyclictest0-21swapper/020:45:070
311939930,3cyclictest0-21swapper/020:40:020
311939930,3cyclictest0-21swapper/020:15:130
311939930,3cyclictest0-21swapper/019:50:130
311939930,3cyclictest0-21swapper/019:25:130
311939930,3cyclictest0-21swapper/019:20:140
311939930,2cyclictest27678-21ssh00:25:100
311939930,2cyclictest0-21swapper/020:00:150
311979921,1cyclictest91532chrt20:31:061
311979921,1cyclictest0-21swapper/123:30:101
311979921,1cyclictest0-21swapper/120:10:151
311979921,1cyclictest0-21swapper/119:45:141
311979921,1cyclictest0-21swapper/119:10:131
311979920,1cyclictest216122chrt20:57:111
311979920,1cyclictest0-21swapper/122:45:151
311979920,1cyclictest0-21swapper/122:35:111
311979920,1cyclictest0-21swapper/122:20:131
311979920,1cyclictest0-21swapper/122:10:121
311979920,1cyclictest0-21swapper/122:00:151
311979920,1cyclictest0-21swapper/121:40:131
311979920,1cyclictest0-21swapper/121:25:131
311979920,1cyclictest0-21swapper/121:05:161
311979920,1cyclictest0-21swapper/121:00:141
311979920,1cyclictest0-21swapper/120:50:121
311979920,1cyclictest0-21swapper/120:45:131
311979920,1cyclictest0-21swapper/120:05:151
311979920,1cyclictest0-21swapper/119:50:121
311979920,1cyclictest0-21swapper/119:35:011
311979920,1cyclictest0-21swapper/119:20:131
311979920,0cyclictest0-21swapper/120:01:071
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional