You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-12 - 11:42
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot2.osadl.org (updated Sat Jul 12, 2025 00:44:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
281012206189,12sleep10-21swapper/119:05:071
301262183166,12sleep00-21swapper/019:07:270
302432178161,12sleep30-21swapper/319:08:583
302422158140,13sleep20-21swapper/219:08:572
172952650,0sleep017271-21ssh00:05:120
83702470,0sleep20-21swapper/221:16:232
83702470,0sleep20-21swapper/221:16:232
66852430,0sleep10-21swapper/122:53:551
44672430,0sleep30-21swapper/323:09:183
315312430,0sleep20-21swapper/200:11:582
36042410,0sleep30-21swapper/300:14:523
304799992,3cyclictest31431-21perf21:45:010
304799981,3cyclictest1611-21idleruntime-cro21:30:010
304799980,4cyclictest24093-21awk21:25:020
20871280,0sleep10-21swapper/123:34:021
304799973,4cyclictest21006-21idleruntime-cro19:55:000
304929960,6cyclictest0-21swapper/321:21:343
304869960,6cyclictest0-21swapper/221:35:562
304799962,4cyclictest16455-21perf20:50:000
304799961,5cyclictest0-21swapper/000:25:070
304799960,6cyclictest61-21kswapd022:30:120
304799960,1cyclictest15142-21ssh23:47:120
13124260,0sleep10-21swapper/122:24:441
3465250,0sleep20-21swapper/221:46:352
304929955,0cyclictest26544-21diskmemload22:27:293
304929955,0cyclictest0-21swapper/322:43:383
304929952,3cyclictest30997-21snmpd21:49:183
304929951,3cyclictest30997-21snmpd21:56:393
304929950,5cyclictest26544-21diskmemload23:35:343
304929950,5cyclictest0-21swapper/323:42:333
304929950,5cyclictest0-21swapper/323:17:323
304929950,5cyclictest0-21swapper/323:11:163
304929950,5cyclictest0-21swapper/300:20:493
304929950,4cyclictest31194-21apt-get22:50:013
304929950,0cyclictest0-21swapper/323:51:003
304929950,0cyclictest0-21swapper/322:22:493
304869951,3cyclictest30997-21snmpd21:59:302
304869951,1cyclictest0-21swapper/222:45:182
304869950,5cyclictest0-21swapper/223:25:132
304869950,5cyclictest0-21swapper/220:40:192
304869950,4cyclictest21514-21sh00:06:512
304869950,4cyclictest0-21swapper/220:00:132
304809954,1cyclictest0-21swapper/121:35:111
304809951,2cyclictest0-21swapper/100:10:121
304809950,5cyclictest0-21swapper/123:21:321
304809950,5cyclictest0-21swapper/121:53:221
304799952,3cyclictest4511-21grep23:25:160
304799952,3cyclictest30596-21python20:10:200
304799952,3cyclictest0-21swapper/023:58:480
304799952,3cyclictest0-21swapper/023:31:040
304799952,3cyclictest0-21swapper/023:19:200
304799952,3cyclictest0-21swapper/022:07:360
304799952,2cyclictest0-21swapper/019:30:120
304799951,4cyclictest0-21swapper/000:10:150
304799950,5cyclictest24453-21perl20:00:150
304799950,5cyclictest12647-21rm22:56:240
304799950,5cyclictest0-21swapper/023:40:100
304799950,5cyclictest0-21swapper/023:07:370
304799950,5cyclictest0-21swapper/023:00:110
304799950,5cyclictest0-21swapper/022:54:240
304799950,5cyclictest0-21swapper/022:42:330
304799950,5cyclictest0-21swapper/022:39:520
304799950,5cyclictest0-21swapper/022:11:040
304799950,5cyclictest0-21swapper/000:35:150
304799950,5cyclictest0-21swapper/000:33:270
304799950,5cyclictest0-21swapper/000:20:050
304799950,0cyclictest13474-21ssh00:19:510
304929944,0cyclictest0-21swapper/323:03:253
304929942,2cyclictest19850-21cat19:50:173
304929942,1cyclictest31651-21ssh23:22:453
304929942,1cyclictest30997-21snmpd21:27:183
304929941,3cyclictest30997-21snmpd23:48:243
304929940,4cyclictest26544-21diskmemload21:12:153
304929940,4cyclictest12072-21apt-get21:35:143
304929940,4cyclictest0-21swapper/323:55:063
304929940,4cyclictest0-21swapper/323:25:303
304929940,4cyclictest0-21swapper/322:55:573
304929940,4cyclictest0-21swapper/322:45:123
304929940,4cyclictest0-21swapper/322:30:263
304929940,4cyclictest0-21swapper/322:15:153
304929940,4cyclictest0-21swapper/322:14:263
304929940,4cyclictest0-21swapper/322:05:123
304929940,4cyclictest0-21swapper/321:53:033
304929940,4cyclictest0-21swapper/321:40:133
304929940,4cyclictest0-21swapper/321:31:013
304929940,4cyclictest0-21swapper/321:18:313
304929940,4cyclictest0-21swapper/321:18:313
304929940,4cyclictest0-21swapper/320:55:113
304929940,4cyclictest0-21swapper/320:45:133
304929940,4cyclictest0-21swapper/300:30:133
304929940,4cyclictest0-21swapper/300:15:253
304929940,3cyclictest9902-21perl20:35:153
304929940,3cyclictest6534-21wc20:30:003
304929940,3cyclictest18449-21ssh00:05:163
304929940,3cyclictest17293-21irqstats19:45:193
304929940,3cyclictest14921-21ssh00:03:563
304929940,3cyclictest13646-21df23:30:133
304929940,3cyclictest1265-21sh22:35:053
304869944,0cyclictest0-21swapper/223:17:212
304869944,0cyclictest0-21swapper/222:57:432
304869944,0cyclictest0-21swapper/222:52:112
304869944,0cyclictest0-21swapper/222:24:062
304869944,0cyclictest0-21swapper/221:24:092
304869944,0cyclictest0-21swapper/200:26:542
304869944,0cyclictest0-21swapper/200:00:512
304869943,1cyclictest13713-21grep23:30:142
304869942,1cyclictest30997-21snmpd22:42:272
304869942,1cyclictest30862-21ssh21:11:282
304869941,3cyclictest7984-21threads23:10:232
304869941,3cyclictest57250irq/125-eth023:50:152
304869941,3cyclictest19395-21cut19:50:152
304869941,3cyclictest15124-21awk20:45:162
304869940,4cyclictest20849-21ssh22:11:442
304869940,4cyclictest17934-21ntp_states20:50:182
304869940,4cyclictest0-21swapper/223:55:122
304869940,4cyclictest0-21swapper/222:30:212
304869940,4cyclictest0-21swapper/220:35:122
304869940,4cyclictest0-21swapper/220:25:112
304869940,4cyclictest0-21swapper/219:15:132
304869940,4cyclictest0-21swapper/219:10:132
304869940,4cyclictest0-21swapper/200:20:112
304869940,3cyclictest9162-21cat20:35:002
304869940,3cyclictest738-21sh23:40:122
304869940,3cyclictest6305-21cat19:25:002
304869940,3cyclictest30416-21/usr/sbin/munin22:00:182
304869940,3cyclictest29875-21memory23:05:162
304869940,3cyclictest28677-21cat22:15:242
304869940,3cyclictest24255-21cat21:05:022
304869940,3cyclictest21533-21ls19:55:162
304869940,3cyclictest19360-21df23:00:132
304869940,3cyclictest0-21swapper/220:20:002
304869940,3cyclictest0-21swapper/219:40:002
304869940,2cyclictest0-21swapper/219:40:202
304869940,1cyclictest6754-21/usr/sbin/munin22:05:152
304869940,1cyclictest6388-21/usr/sbin/munin19:25:172
304869940,1cyclictest2487-21basename00:30:162
304869940,1cyclictest24616-21/usr/sbin/munin21:25:192
304869940,1cyclictest10145-21tune2fs21:50:182
304869940,0cyclictest0-21swapper/200:15:242
304809944,0cyclictest0-21swapper/123:40:271
304809944,0cyclictest0-21swapper/123:17:371
304809944,0cyclictest0-21swapper/100:35:011
304809941,3cyclictest0-21swapper/122:35:131
304809941,2cyclictest0-21swapper/121:55:161
304809941,2cyclictest0-21swapper/121:40:141
304809941,2cyclictest0-21swapper/121:30:121
304809941,2cyclictest0-21swapper/120:25:121
304809941,2cyclictest0-21swapper/120:10:121
304809941,2cyclictest0-21swapper/120:05:131
304809941,2cyclictest0-21swapper/119:20:101
304809941,1cyclictest0-21swapper/120:15:111
304809941,1cyclictest0-21swapper/100:15:131
304809940,4cyclictest0-21swapper/121:16:421
304809940,4cyclictest0-21swapper/121:16:421
304809940,4cyclictest0-21swapper/119:55:041
304809940,4cyclictest0-21swapper/100:28:541
304809940,4cyclictest0-21swapper/100:04:191
304809940,2cyclictest0-21swapper/122:40:141
304809940,2cyclictest0-21swapper/119:25:141
304809940,0cyclictest0-21swapper/123:55:531
304809940,0cyclictest0-21swapper/123:09:451
304809940,0cyclictest0-21swapper/100:23:311
304799944,0cyclictest0-21swapper/000:00:090
304799942,2cyclictest0-21swapper/021:51:200
304799942,1cyclictest23494-21awk22:30:000
304799941,3cyclictest27706-21fschecks_count21:10:150
304799941,3cyclictest0-21swapper/022:20:400
304799941,2cyclictest0-21swapper/020:55:110
304799941,1cyclictest0-21swapper/020:20:140
304799940,4cyclictest17939-21ntp_states20:50:180
304799940,4cyclictest0-21swapper/023:50:130
304799940,4cyclictest0-21swapper/023:36:300
304799940,4cyclictest0-21swapper/023:10:320
304799940,4cyclictest0-21swapper/022:15:240
304799940,4cyclictest0-21swapper/022:00:140
304799940,4cyclictest0-21swapper/021:55:160
304799940,4cyclictest0-21swapper/021:35:140
304799940,4cyclictest0-21swapper/021:00:130
304799940,4cyclictest0-21swapper/019:55:150
304799940,4cyclictest0-21swapper/019:20:100
304799940,3cyclictest4420-21ssh21:15:040
304799940,3cyclictest4420-21ssh21:15:040
304799940,3cyclictest31635-21ssh21:45:020
304799940,3cyclictest2648-21ssh21:30:140
304799940,3cyclictest25552-21rm23:20:010
304799940,3cyclictest2448-21/usr/sbin/munin19:15:180
304799940,3cyclictest24220-21apt-config21:05:010
304799940,3cyclictest23242-21sort22:45:200
30357240,0sleep20-21swapper/223:38:482
23259240,0sleep10-21swapper/123:01:511
304929932,1cyclictest30997-21snmpd19:30:023
304929931,1cyclictest3916-21apt-get19:20:103
304929931,1cyclictest0-21swapper/320:50:123
304929930,3cyclictest32073-21df_inode20:15:133
304929930,3cyclictest24472-21perl20:00:153
304929930,3cyclictest23795-21ssh00:25:013
304929930,3cyclictest14362-21df_inode19:40:183
304929930,3cyclictest11223-21ssh00:35:053
304929930,3cyclictest0-21swapper/322:00:063
304929930,3cyclictest0-21swapper/321:05:083
304929930,3cyclictest0-21swapper/321:00:133
304929930,3cyclictest0-21swapper/320:40:133
304929930,3cyclictest0-21swapper/320:30:013
304929930,3cyclictest0-21swapper/320:20:153
304929930,3cyclictest0-21swapper/320:10:123
304929930,3cyclictest0-21swapper/320:05:013
304929930,3cyclictest0-21swapper/319:55:023
304929930,3cyclictest0-21swapper/319:35:123
304929930,3cyclictest0-21swapper/319:25:133
304929930,3cyclictest0-21swapper/319:15:053
304929930,3cyclictest0-21swapper/319:10:113
304869932,1cyclictest30997-21snmpd21:00:022
304869932,1cyclictest30997-21snmpd00:35:012
304869931,1cyclictest0-21swapper/219:30:122
304869930,3cyclictest26344-21find20:05:002
304869930,3cyclictest14051-21apt22:25:122
304869930,3cyclictest10100-21ssh23:45:072
304869930,3cyclictest0-21swapper/223:20:122
304869930,3cyclictest0-21swapper/221:40:152
304869930,3cyclictest0-21swapper/221:30:082
304869930,3cyclictest0-21swapper/220:55:042
304869930,3cyclictest0-21swapper/220:20:102
304869930,3cyclictest0-21swapper/220:10:122
304869930,3cyclictest0-21swapper/219:45:152
304869930,1cyclictest1674-21ssh22:35:142
304809933,0cyclictest0-21swapper/120:53:331
304809933,0cyclictest0-21swapper/119:47:561
304809933,0cyclictest0-21swapper/119:40:041
304809933,0cyclictest0-21swapper/119:30:461
304809931,2cyclictest0-21swapper/122:55:121
304809931,2cyclictest0-21swapper/122:05:131
304809931,2cyclictest0-21swapper/121:10:131
304809931,1cyclictest0-21swapper/123:45:111
304809931,1cyclictest0-21swapper/123:25:111
304809931,1cyclictest0-21swapper/122:30:121
304809931,1cyclictest0-21swapper/122:25:111
304809931,1cyclictest0-21swapper/122:15:151
304809931,1cyclictest0-21swapper/122:00:141
304809931,1cyclictest0-21swapper/121:25:141
304809931,1cyclictest0-21swapper/121:20:171
304809931,1cyclictest0-21swapper/121:00:131
304809931,1cyclictest0-21swapper/120:00:131
304809931,1cyclictest0-21swapper/119:50:131
304809931,1cyclictest0-21swapper/119:15:131
304809931,1cyclictest0-21swapper/119:10:111
304809930,2cyclictest0-21swapper/123:50:131
304809930,2cyclictest0-21swapper/123:10:151
304809930,2cyclictest0-21swapper/122:45:011
304809930,2cyclictest0-21swapper/121:05:021
304809930,2cyclictest0-21swapper/100:05:111
304799931,1cyclictest30997-21snmpd19:35:090
304799931,1cyclictest0-21swapper/020:25:110
304799930,3cyclictest16301-21/usr/sbin/munin19:45:200
304799930,3cyclictest0-21swapper/020:40:130
304799930,3cyclictest0-21swapper/020:35:120
304799930,3cyclictest0-21swapper/020:30:110
304799930,3cyclictest0-21swapper/020:15:110
304799930,3cyclictest0-21swapper/020:05:010
304799930,3cyclictest0-21swapper/019:40:150
304799930,3cyclictest0-21swapper/019:25:130
304799930,3cyclictest0-21swapper/019:10:110
304809921,1cyclictest0-21swapper/123:35:171
304809921,1cyclictest0-21swapper/100:30:131
304809921,0cyclictest0-21swapper/119:35:121
304809920,1cyclictest3487-21chrt20:20:291
304809920,1cyclictest0-21swapper/122:10:131
304809920,1cyclictest0-21swapper/121:45:121
304809920,1cyclictest0-21swapper/120:55:111
304809920,1cyclictest0-21swapper/120:45:121
304809920,1cyclictest0-21swapper/120:40:131
304809920,1cyclictest0-21swapper/120:35:121
304809920,1cyclictest0-21swapper/120:30:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional