You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-09-28 - 12:40
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot2.osadl.org (updated Thu Sep 28, 2023 00:44:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
261922204188,11sleep10-21swapper/119:08:241
260632189171,13sleep00-21swapper/019:06:420
260392177160,12sleep30-21swapper/319:06:263
262922174157,12sleep20-21swapper/219:09:412
288932520,0sleep10-21swapper/122:14:111
191212510,0sleep20-21swapper/221:48:222
290022470,0sleep10-21swapper/123:33:301
173512460,0sleep10-21swapper/121:27:151
14432450,0sleep30-21swapper/323:35:483
264759992,3cyclictest22179-21idleruntime-cro23:10:010
264759992,3cyclictest13850-21munin-run22:25:010
264829981,6cyclictest0-21swapper/322:25:163
264759980,4cyclictest29571-21cron21:55:020
264779962,2cyclictest0-21swapper/100:30:211
264759962,4cyclictest22887-21idleruntime-cro23:50:000
264759961,4cyclictest0-21swapper/023:15:190
264759960,6cyclictest0-21swapper/022:02:270
264759960,5cyclictest0-21swapper/022:30:200
264829950,5cyclictest0-21swapper/323:07:153
264829950,5cyclictest0-21swapper/321:30:563
264819950,5cyclictest0-21swapper/223:42:082
264819950,5cyclictest0-21swapper/222:10:112
264819950,5cyclictest0-21swapper/221:13:182
264819950,4cyclictest14705-21apt-get23:25:022
264819950,4cyclictest0-21swapper/221:00:172
264779951,2cyclictest0-21swapper/121:55:191
264779951,2cyclictest0-21swapper/121:40:171
264779951,2cyclictest0-21swapper/119:45:201
264779950,5cyclictest0-21swapper/121:35:141
264759954,1cyclictest29755-21ssh23:34:070
264759950,5cyclictest0-21swapper/022:35:200
264759950,5cyclictest0-21swapper/000:21:240
264759950,4cyclictest23412-21apt-get23:50:180
264759950,4cyclictest0-21swapper/000:25:170
3759240,0sleep00-21swapper/019:30:240
264829944,0cyclictest0-21swapper/322:41:043
264829944,0cyclictest0-21swapper/322:14:443
264829944,0cyclictest0-21swapper/322:02:133
264829944,0cyclictest0-21swapper/321:40:413
264829944,0cyclictest0-21swapper/321:27:103
264829943,1cyclictest27966-21head19:10:283
264829941,3cyclictest10738-21cpuspeed_turbos20:50:193
264829941,2cyclictest0-21swapper/320:45:183
264829940,4cyclictest32036-21ssh23:55:193
264829940,4cyclictest29565-21perl21:15:193
264829940,4cyclictest14185-21ntp_states19:50:233
264829940,4cyclictest0-21swapper/323:51:253
264829940,4cyclictest0-21swapper/323:20:163
264829940,4cyclictest0-21swapper/323:15:193
264829940,4cyclictest0-21swapper/323:10:563
264829940,4cyclictest0-21swapper/322:50:523
264829940,4cyclictest0-21swapper/322:47:053
264829940,4cyclictest0-21swapper/322:34:433
264829940,4cyclictest0-21swapper/322:23:523
264829940,4cyclictest0-21swapper/322:15:523
264829940,4cyclictest0-21swapper/321:55:403
264829940,4cyclictest0-21swapper/321:50:203
264829940,4cyclictest0-21swapper/321:35:193
264829940,4cyclictest0-21swapper/321:22:493
264829940,4cyclictest0-21swapper/321:10:193
264829940,4cyclictest0-21swapper/320:05:173
264829940,4cyclictest0-21swapper/319:35:183
264829940,4cyclictest0-21swapper/300:30:213
264829940,4cyclictest0-21swapper/300:15:183
264829940,4cyclictest0-21swapper/300:03:513
264829940,3cyclictest6883-21ssh23:40:023
264829940,3cyclictest6062-21sed23:00:003
264829940,3cyclictest30822-21df20:25:233
264829940,3cyclictest22983-21/usr/sbin/munin00:10:273
264829940,3cyclictest16021-21/usr/sbin/munin22:05:253
264829940,3cyclictest1171-21sh00:36:293
264829940,3cyclictest11283-21smartctl19:45:243
264829940,3cyclictest0-21swapper/320:40:203
264829940,1cyclictest14135-21df_inode21:45:223
264819944,0cyclictest0-21swapper/223:15:582
264819944,0cyclictest0-21swapper/222:45:502
264819944,0cyclictest0-21swapper/221:42:152
264819943,1cyclictest7090-21df00:20:182
264819943,1cyclictest29844-21/usr/sbin/munin19:15:182
264819942,1cyclictest0-21swapper/200:05:192
264819941,3cyclictest57250irq/125-eth023:56:082
264819941,2cyclictest509-21nscd22:54:402
264819941,1cyclictest0-21swapper/200:15:182
264819940,4cyclictest57250irq/125-eth021:35:222
264819940,4cyclictest30957-21perl21:55:222
264819940,4cyclictest30598-21ssh23:54:442
264819940,4cyclictest15690-21memory00:25:232
264819940,4cyclictest0-21swapper/223:13:412
264819940,4cyclictest0-21swapper/222:30:202
264819940,4cyclictest0-21swapper/221:50:022
264819940,4cyclictest0-21swapper/221:28:032
264819940,4cyclictest0-21swapper/200:30:222
264819940,3cyclictest9277-21ssh22:21:202
264819940,3cyclictest7612-21modprobe23:00:272
264819940,3cyclictest6473-21irqstats19:35:242
264819940,3cyclictest5579-21smartctl21:20:222
264819940,3cyclictest31045-21perl20:25:242
264819940,3cyclictest29808-21cat21:15:212
264819940,3cyclictest11191-21perl20:50:212
264819940,3cyclictest10975-21diskstats19:45:222
264819940,3cyclictest0-21swapper/223:30:202
264819940,2cyclictest14170-21ssh22:25:092
264819940,1cyclictest31557-21hddtemp_smartct22:55:202
264819940,1cyclictest26610-21/usr/sbin/munin19:10:222
264819940,0cyclictest0-21swapper/200:14:092
264819940,0cyclictest0-21swapper/200:00:022
264779944,0cyclictest0-21swapper/123:15:121
264779944,0cyclictest0-21swapper/122:24:511
264779943,1cyclictest0-21swapper/123:46:031
264779943,1cyclictest0-21swapper/120:12:521
264779941,2cyclictest0-21swapper/123:05:171
264779941,2cyclictest0-21swapper/121:30:251
264779941,2cyclictest0-21swapper/120:20:171
264779941,2cyclictest0-21swapper/119:55:191
264779941,2cyclictest0-21swapper/119:15:161
264779941,2cyclictest0-21swapper/119:10:181
264779941,2cyclictest0-21swapper/100:15:181
264779941,2cyclictest0-21swapper/100:05:191
264779941,1cyclictest0-21swapper/122:00:201
264779940,4cyclictest0-21swapper/120:34:501
264779940,4cyclictest0-21swapper/119:24:251
264779940,2cyclictest0-21swapper/120:50:181
264779940,2cyclictest0-21swapper/120:40:201
264779940,0cyclictest0-21swapper/122:56:341
264779940,0cyclictest0-21swapper/121:12:591
264779940,0cyclictest0-21swapper/100:24:271
264759944,0cyclictest0-21swapper/023:00:040
264759944,0cyclictest0-21swapper/021:45:170
264759943,1cyclictest7378-21cut23:20:200
264759942,1cyclictest756-21runrttasks00:36:500
264759941,2cyclictest61-21kswapd022:05:150
264759941,2cyclictest0-21swapper/023:40:160
264759941,2cyclictest0-21swapper/021:30:250
264759941,2cyclictest0-21swapper/020:55:180
264759941,2cyclictest0-21swapper/020:30:190
264759941,2cyclictest0-21swapper/019:35:200
264759941,2cyclictest0-21swapper/000:00:220
264759940,4cyclictest0-21swapper/021:40:170
264759940,4cyclictest0-21swapper/021:27:020
264759940,4cyclictest0-21swapper/021:10:190
264759940,4cyclictest0-21swapper/000:10:170
264759940,3cyclictest8518-21perl20:45:210
264759940,3cyclictest8368-21expr19:40:200
264759940,3cyclictest7179-21perl22:40:210
264759940,3cyclictest5254-21diskstats21:20:200
264759940,3cyclictest31442-21apt-get23:35:180
264759940,3cyclictest31041-21ssh21:35:290
264759940,3cyclictest30153-21ssh21:55:190
264759940,3cyclictest29282-21grep19:15:010
264759940,3cyclictest26321-21irqstats20:15:250
264759940,3cyclictest23734-21memory00:30:260
264759940,3cyclictest23323-21ls20:10:210
264759940,3cyclictest21941-21ssh22:10:010
264759940,3cyclictest20120-21munin-run21:10:010
264759940,3cyclictest15675-21ssh22:25:230
264759940,3cyclictest14779-21ssh00:05:030
264759940,1cyclictest30166-21perf22:55:000
264759940,0cyclictest0-21swapper/023:55:210
25794240,0sleep20-21swapper/220:15:222
22312240,0sleep20-21swapper/221:30:292
19546240,0sleep10-21swapper/122:08:151
19153240,0sleep30-21swapper/323:46:583
264829933,0cyclictest0-21swapper/319:21:253
264829931,1cyclictest0-21swapper/300:25:173
264829930,3cyclictest32759-21chrt20:30:083
264829930,3cyclictest20106-21diskmemload23:00:073
264829930,3cyclictest20106-21diskmemload00:05:033
264829930,3cyclictest18532-21fschecks_time20:00:213
264829930,3cyclictest14491-21apt-config23:25:013
264829930,3cyclictest0-21swapper/323:30:013
264829930,3cyclictest0-21swapper/322:35:153
264829930,3cyclictest0-21swapper/321:05:183
264829930,3cyclictest0-21swapper/321:00:183
264829930,3cyclictest0-21swapper/320:55:183
264829930,3cyclictest0-21swapper/320:35:173
264829930,3cyclictest0-21swapper/320:20:153
264829930,3cyclictest0-21swapper/320:15:203
264829930,3cyclictest0-21swapper/320:10:203
264829930,3cyclictest0-21swapper/319:55:183
264829930,3cyclictest0-21swapper/319:40:023
264829930,3cyclictest0-21swapper/319:30:203
264829930,3cyclictest0-21swapper/319:25:033
264829930,3cyclictest0-21swapper/319:15:023
264829930,3cyclictest0-21swapper/300:20:053
264819933,0cyclictest0-21swapper/222:35:042
264819932,1cyclictest759-21snmpd23:05:082
264819932,1cyclictest759-21snmpd20:30:102
264819931,2cyclictest30397-21ssh00:35:162
264819931,1cyclictest12923-21apt-get20:55:182
264819931,1cyclictest0-21swapper/220:20:172
264819931,1cyclictest0-21swapper/219:40:182
264819930,3cyclictest3907-21memory20:35:232
264819930,3cyclictest17879-21apt-get21:05:172
264819930,3cyclictest15179-21rm23:45:112
264819930,3cyclictest14070-21cut22:05:152
264819930,3cyclictest13202-21ls19:50:182
264819930,3cyclictest0-21swapper/223:35:052
264819930,3cyclictest0-21swapper/223:20:052
264819930,3cyclictest0-21swapper/222:40:182
264819930,3cyclictest0-21swapper/222:15:052
264819930,3cyclictest0-21swapper/222:00:062
264819930,3cyclictest0-21swapper/220:45:172
264819930,3cyclictest0-21swapper/220:40:232
264819930,3cyclictest0-21swapper/220:10:032
264819930,3cyclictest0-21swapper/220:05:192
264819930,3cyclictest0-21swapper/220:00:182
264819930,3cyclictest0-21swapper/219:55:182
264819930,3cyclictest0-21swapper/219:30:202
264819930,3cyclictest0-21swapper/219:25:112
264819930,3cyclictest0-21swapper/219:20:062
264779931,2cyclictest0-21swapper/123:10:211
264779931,1cyclictest0-21swapper/123:55:181
264779931,1cyclictest0-21swapper/123:50:181
264779931,1cyclictest0-21swapper/123:35:181
264779931,1cyclictest0-21swapper/123:25:021
264779931,1cyclictest0-21swapper/123:20:161
264779931,1cyclictest0-21swapper/123:00:211
264779931,1cyclictest0-21swapper/122:45:201
264779931,1cyclictest0-21swapper/122:25:161
264779931,1cyclictest0-21swapper/122:15:191
264779931,1cyclictest0-21swapper/121:50:201
264779931,1cyclictest0-21swapper/121:20:181
264779931,1cyclictest0-21swapper/121:15:161
264779931,1cyclictest0-21swapper/120:35:171
264779931,1cyclictest0-21swapper/119:40:181
264779931,1cyclictest0-21swapper/119:30:201
264779931,1cyclictest0-21swapper/100:35:181
264779930,3cyclictest0-21swapper/120:06:281
264779930,1cyclictest0-21swapper/122:40:181
264779930,1cyclictest0-21swapper/120:00:181
264759932,1cyclictest14774-21ssh23:25:060
264759932,1cyclictest11899-21wc19:45:290
264759931,1cyclictest759-21snmpd22:15:140
264759931,1cyclictest759-21snmpd20:25:010
264759931,1cyclictest0-21swapper/023:10:210
264759931,1cyclictest0-21swapper/021:00:170
264759931,1cyclictest0-21swapper/019:50:160
264759930,3cyclictest61-21kswapd022:55:160
264759930,3cyclictest11202sleep019:25:150
264759930,3cyclictest0-21swapper/022:45:060
264759930,3cyclictest0-21swapper/021:15:050
264759930,3cyclictest0-21swapper/020:50:000
264759930,3cyclictest0-21swapper/020:40:200
264759930,3cyclictest0-21swapper/020:35:160
264759930,3cyclictest0-21swapper/020:20:170
264759930,3cyclictest0-21swapper/020:05:180
264759930,3cyclictest0-21swapper/020:01:170
264759930,3cyclictest0-21swapper/019:56:460
264759930,3cyclictest0-21swapper/019:21:160
264759930,3cyclictest0-21swapper/019:10:180
264759930,2cyclictest30653-21date00:15:000
264779921,1cyclictest0-21swapper/122:30:201
264779921,1cyclictest0-21swapper/120:15:201
264779920,2cyclictest0-21swapper/120:55:181
264779920,1cyclictest7047-21chrt23:40:101
264779920,1cyclictest0-21swapper/122:50:001
264779920,1cyclictest0-21swapper/122:35:181
264779920,1cyclictest0-21swapper/121:45:011
264779920,1cyclictest0-21swapper/121:05:171
264779920,1cyclictest0-21swapper/121:00:241
264779920,1cyclictest0-21swapper/120:45:171
264779920,1cyclictest0-21swapper/120:25:211
264779920,1cyclictest0-21swapper/119:50:171
264779920,1cyclictest0-21swapper/119:35:181
264779920,1cyclictest0-21swapper/119:25:131
264779920,1cyclictest0-21swapper/100:25:171
264779920,1cyclictest0-21swapper/100:10:031
264779920,1cyclictest0-21swapper/100:00:211
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional