You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-05-29 - 07:01
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot2.osadl.org (updated Wed May 29, 2024 00:44:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
280492195179,11sleep00-21swapper/019:08:100
259632194178,12sleep30-21swapper/319:05:043
280922178162,11sleep10-21swapper/119:08:441
278632162144,13sleep20-21swapper/219:05:482
163662490,0sleep00-21swapper/021:43:520
225772470,0sleep20-21swapper/223:05:572
269502460,0sleep00-21swapper/000:27:460
108182420,0sleep10-21swapper/100:18:061
59502400,0sleep10-21swapper/123:35:291
283729950,5cyclictest0-21swapper/323:47:013
283729950,5cyclictest0-21swapper/323:31:373
283729950,4cyclictest6771-21apt-get19:30:133
283729950,4cyclictest2918-21apt-get22:35:143
283729950,4cyclictest23191-21python00:05:323
283729950,4cyclictest13023-21perl20:45:153
283729950,3cyclictest0-21swapper/300:38:053
283669955,0cyclictest0-21swapper/223:30:022
283669955,0cyclictest0-21swapper/222:21:212
283669955,0cyclictest0-21swapper/200:33:462
283669950,4cyclictest0-21swapper/222:50:122
283669950,0cyclictest0-21swapper/200:21:312
283649951,3cyclictest0-21swapper/122:05:101
283649951,3cyclictest0-21swapper/100:10:101
283649951,2cyclictest0-21swapper/122:25:131
283649951,2cyclictest0-21swapper/122:00:131
283649951,2cyclictest0-21swapper/119:10:121
283649951,2cyclictest0-21swapper/100:25:111
283649950,5cyclictest0-21swapper/123:13:331
283649950,5cyclictest0-21swapper/122:53:021
283649950,5cyclictest0-21swapper/122:42:501
283649950,5cyclictest0-21swapper/121:51:291
283649950,5cyclictest0-21swapper/119:41:151
283599955,0cyclictest0-21swapper/000:02:450
283599951,3cyclictest0-21swapper/023:00:150
283599950,5cyclictest0-21swapper/023:34:580
283599950,5cyclictest0-21swapper/023:06:010
283599950,5cyclictest0-21swapper/022:02:430
283599950,5cyclictest0-21swapper/021:27:210
283599950,5cyclictest0-21swapper/000:18:110
283599950,4cyclictest30037-21df_inode00:10:120
283599950,4cyclictest29742-21apt-get20:15:120
283599950,4cyclictest27651-21diskmemload22:50:130
23401250,0sleep20-21swapper/222:27:202
20514250,1sleep320517-21cstates23:25:153
29097240,0sleep20-21swapper/222:30:292
283729944,0cyclictest29419-21kworker/3:023:24:043
283729944,0cyclictest0-21swapper/321:41:553
283729944,0cyclictest0-21swapper/321:26:373
283729944,0cyclictest0-21swapper/321:05:143
283729943,1cyclictest27593-21cut22:10:173
283729942,2cyclictest3602-21snmpd22:45:483
283729942,1cyclictest13596-21/usr/sbin/munin00:20:243
283729940,4cyclictest30212-21apt-get00:30:123
283729940,4cyclictest29783-21ls00:10:113
283729940,4cyclictest28230-21df22:50:153
283729940,4cyclictest10332-21memory19:35:193
283729940,4cyclictest0-21swapper/323:50:533
283729940,4cyclictest0-21swapper/323:13:573
283729940,4cyclictest0-21swapper/322:25:123
283729940,4cyclictest0-21swapper/322:17:303
283729940,4cyclictest0-21swapper/322:05:343
283729940,4cyclictest0-21swapper/321:59:253
283729940,4cyclictest0-21swapper/321:50:113
283729940,4cyclictest0-21swapper/321:45:093
283729940,4cyclictest0-21swapper/321:35:103
283729940,4cyclictest0-21swapper/321:30:143
283729940,4cyclictest0-21swapper/321:10:043
283729940,4cyclictest0-21swapper/320:34:553
283729940,4cyclictest0-21swapper/320:15:123
283729940,4cyclictest0-21swapper/319:45:113
283729940,4cyclictest0-21swapper/319:25:143
283729940,4cyclictest0-21swapper/300:25:123
283729940,4cyclictest0-21swapper/300:02:053
283729940,3cyclictest9724-21ls20:39:593
283729940,3cyclictest3892-21apt-config23:35:013
283729940,3cyclictest24661-21sed20:05:013
283729940,3cyclictest24661-21sed20:05:003
283729940,3cyclictest2365-21grep20:25:113
283729940,3cyclictest12797-21if_eth423:00:213
283729940,3cyclictest12284-21sh23:40:013
283729940,3cyclictest10601-21ssh22:20:013
283729940,3cyclictest10246-21/usr/sbin/munin21:20:233
283729940,2cyclictest14438-21ssh22:41:243
283729940,1cyclictest727-21/usr/sbin/munin21:15:313
283729940,1cyclictest4612-21/usr/sbin/munin23:55:243
283729940,1cyclictest28476-21/usr/sbin/munin19:10:243
283729940,1cyclictest27170-21/usr/sbin/munin20:10:243
283729940,1cyclictest16873-21/usr/sbin/munin19:50:173
283669944,0cyclictest0-21swapper/221:37:242
283669944,0cyclictest0-21swapper/200:15:252
283669944,0cyclictest0-21swapper/200:10:032
283669944,0cyclictest0-21swapper/200:04:402
283669943,1cyclictest5690-21ssh21:56:282
283669943,1cyclictest27955-21gltestperf20:10:172
283669943,1cyclictest1335-21python20:20:252
283669943,1cyclictest10604-21head19:35:222
283669942,1cyclictest756-21runrttasks23:02:112
283669941,3cyclictest12278-21cat22:40:222
283669941,1cyclictest0-21swapper/221:15:122
283669940,4cyclictest6559-21diskstats00:35:162
283669940,4cyclictest18199-21apt-get21:45:092
283669940,4cyclictest14405-21apt-get19:45:112
283669940,4cyclictest0-21swapper/223:57:532
283669940,4cyclictest0-21swapper/223:35:122
283669940,4cyclictest0-21swapper/220:15:112
283669940,4cyclictest0-21swapper/200:25:112
283669940,3cyclictest2925-21ssh22:35:142
283669940,3cyclictest2660-21aten2_r5power_p22:15:112
283669940,3cyclictest19491-21ssh23:24:522
283669940,3cyclictest18436-21sed22:05:002
283669940,3cyclictest0-21swapper/219:25:132
283669940,0cyclictest0-21swapper/223:40:042
283649944,0cyclictest0-21swapper/123:57:461
283649944,0cyclictest0-21swapper/123:08:001
283649944,0cyclictest0-21swapper/121:45:221
283649944,0cyclictest0-21swapper/121:19:191
283649944,0cyclictest0-21swapper/120:43:311
283649944,0cyclictest0-21swapper/100:35:281
283649944,0cyclictest0-21swapper/100:07:491
283649942,2cyclictest0-21swapper/121:35:131
283649941,2cyclictest0-21swapper/123:45:111
283649941,2cyclictest0-21swapper/123:20:111
283649941,2cyclictest0-21swapper/122:35:131
283649941,2cyclictest0-21swapper/122:20:141
283649941,2cyclictest0-21swapper/121:55:141
283649941,2cyclictest0-21swapper/121:30:151
283649941,2cyclictest0-21swapper/120:20:121
283649941,2cyclictest0-21swapper/119:50:141
283649941,2cyclictest0-21swapper/119:20:161
283649941,2cyclictest0-21swapper/100:20:101
283649941,1cyclictest0-21swapper/120:55:101
283649940,4cyclictest0-21swapper/123:50:041
283649940,4cyclictest0-21swapper/122:14:011
283649940,4cyclictest0-21swapper/120:39:441
283649940,4cyclictest0-21swapper/120:29:061
283649940,0cyclictest0-21swapper/122:47:301
283599944,0cyclictest3-21ksoftirqd/022:15:100
283599944,0cyclictest0-21swapper/000:21:140
283599943,1cyclictest3-21ksoftirqd/021:15:120
283599941,1cyclictest81rcu_preempt22:30:140
283599940,4cyclictest4411-21systemd-udevd20:28:020
283599940,4cyclictest30819-21ssh00:30:170
283599940,4cyclictest0-21swapper/023:55:050
283599940,4cyclictest0-21swapper/023:25:060
283599940,4cyclictest0-21swapper/023:21:000
283599940,4cyclictest0-21swapper/023:15:090
283599940,4cyclictest0-21swapper/022:36:570
283599940,4cyclictest0-21swapper/022:20:120
283599940,4cyclictest0-21swapper/021:56:280
283599940,4cyclictest0-21swapper/021:52:500
283599940,4cyclictest0-21swapper/021:45:300
283599940,4cyclictest0-21swapper/000:35:390
283599940,3cyclictest9306-21aten2_r5power_p19:35:100
283599940,3cyclictest7765-21irqstats19:30:210
283599940,3cyclictest5811-21ntp_states23:35:280
283599940,3cyclictest5382-21diskstats20:30:150
283599940,3cyclictest506-21perl20:20:170
283599940,3cyclictest4363-21hddtemp_smartct22:55:180
283599940,3cyclictest31598-21rm23:11:390
283599940,3cyclictest28730-21ssh23:50:050
283599940,3cyclictest28450-21ntp_states22:10:260
283599940,3cyclictest28102-21if_eth020:10:210
283599940,3cyclictest24750-21ssh21:10:060
283599940,3cyclictest24418-21diskmemload22:25:040
283599940,3cyclictest24418-21diskmemload00:05:000
283599940,3cyclictest23366-21cat20:00:240
283599940,3cyclictest23366-21cat20:00:230
283599940,3cyclictest19099-21rawdevmodel19:53:360
283599940,3cyclictest18662-21/usr/sbin/munin22:05:120
283599940,3cyclictest17218-21apt20:55:090
283599940,3cyclictest16265-21vmstat19:45:310
283599940,3cyclictest14518-21latency_hist20:50:000
283599940,3cyclictest11937-21apt-get19:40:120
283599940,3cyclictest11122-21ssh22:40:100
283599940,3cyclictest10334-21df20:40:180
283599940,3cyclictest0-21swapper/021:39:010
283599940,3cyclictest0-21swapper/021:20:040
283599940,1cyclictest6540-21latency_hist19:30:010
283599940,1cyclictest509-21nscd23:45:320
17778240,0sleep10-21swapper/100:02:311
283729932,1cyclictest3602-21snmpd21:00:103
283729932,1cyclictest3602-21snmpd20:05:103
283729930,3cyclictest3568-21chrt23:15:013
283729930,3cyclictest32242-21apt-get20:20:123
283729930,3cyclictest31122-21processes19:15:003
283729930,3cyclictest27279-21grep22:30:143
283729930,3cyclictest15337-21fschecks_count20:50:153
283729930,3cyclictest0-21swapper/323:05:063
283729930,3cyclictest0-21swapper/322:55:103
283729930,3cyclictest0-21swapper/322:00:043
283729930,3cyclictest0-21swapper/320:55:113
283729930,3cyclictest0-21swapper/320:40:143
283729930,3cyclictest0-21swapper/319:55:103
283729930,3cyclictest0-21swapper/319:40:013
283729930,3cyclictest0-21swapper/319:20:153
283729930,3cyclictest0-21swapper/300:15:123
283669932,1cyclictest31013-21proc_pri19:15:012
283669932,1cyclictest31-21ksoftirqd/219:50:132
283669931,1cyclictest0-21swapper/220:25:092
283669931,1cyclictest0-21swapper/200:05:122
283669930,3cyclictest9815-21ssh21:40:042
283669930,3cyclictest8600-21ntp_states20:35:232
283669930,3cyclictest4583-21tune2fs23:15:182
283669930,3cyclictest27462-21df22:10:162
283669930,3cyclictest25440-21grep20:05:142
283669930,3cyclictest24418-21diskmemload23:50:102
283669930,3cyclictest20131-21perl22:45:172
283669930,3cyclictest19224-21cstates22:05:132
283669930,3cyclictest15587-21smartctl20:50:172
283669930,3cyclictest12627-21grep20:45:122
283669930,3cyclictest0-21swapper/223:45:102
283669930,3cyclictest0-21swapper/223:25:122
283669930,3cyclictest0-21swapper/222:55:092
283669930,3cyclictest0-21swapper/221:50:122
283669930,3cyclictest0-21swapper/221:30:142
283669930,3cyclictest0-21swapper/221:25:032
283669930,3cyclictest0-21swapper/221:20:142
283669930,3cyclictest0-21swapper/221:10:122
283669930,3cyclictest0-21swapper/221:01:372
283669930,3cyclictest0-21swapper/220:55:112
283669930,3cyclictest0-21swapper/220:40:142
283669930,3cyclictest0-21swapper/220:30:102
283669930,3cyclictest0-21swapper/220:00:122
283669930,3cyclictest0-21swapper/220:00:122
283669930,3cyclictest0-21swapper/219:55:092
283669930,3cyclictest0-21swapper/219:40:012
283669930,3cyclictest0-21swapper/219:30:132
283669930,3cyclictest0-21swapper/219:20:152
283669930,3cyclictest0-21swapper/219:10:032
283669930,2cyclictest61-21kswapd023:10:142
283669930,1cyclictest0-21swapper/221:05:102
283649933,0cyclictest0-21swapper/119:36:351
283649931,2cyclictest0-21swapper/120:30:101
283649931,1cyclictest0-21swapper/123:40:101
283649931,1cyclictest0-21swapper/123:30:111
283649931,1cyclictest0-21swapper/123:15:131
283649931,1cyclictest0-21swapper/123:00:111
283649931,1cyclictest0-21swapper/121:25:131
283649931,1cyclictest0-21swapper/121:10:101
283649931,1cyclictest0-21swapper/120:50:091
283649931,1cyclictest0-21swapper/120:00:121
283649931,1cyclictest0-21swapper/120:00:121
283649931,1cyclictest0-21swapper/119:55:091
283649931,1cyclictest0-21swapper/119:45:121
283649931,1cyclictest0-21swapper/100:30:111
283649930,2cyclictest19960-21taskset23:25:041
283649930,2cyclictest0-21swapper/122:55:101
283649930,2cyclictest0-21swapper/121:40:111
283649930,2cyclictest0-21swapper/120:45:111
283649930,2cyclictest0-21swapper/120:10:131
283649930,2cyclictest0-21swapper/119:30:121
283649930,2cyclictest0-21swapper/119:15:111
283649930,0cyclictest0-21swapper/121:00:591
283599932,1cyclictest31634-21hostname19:15:010
283599932,1cyclictest3-21ksoftirqd/019:20:160
283599930,3cyclictest25545-21apt-config21:30:010
283599930,3cyclictest24418-21diskmemload23:40:060
283599930,3cyclictest19540-21apt-get19:55:100
283599930,3cyclictest0-21swapper/022:45:050
283599930,3cyclictest0-21swapper/021:05:110
283599930,3cyclictest0-21swapper/021:00:110
283599930,3cyclictest0-21swapper/020:50:100
283599930,3cyclictest0-21swapper/020:35:110
283599930,3cyclictest0-21swapper/020:05:090
283599930,3cyclictest0-21swapper/019:10:020
283649921,1cyclictest0-21swapper/121:05:111
283649920,1cyclictest0-21swapper/122:30:111
283649920,1cyclictest0-21swapper/122:15:101
283649920,1cyclictest0-21swapper/121:20:131
283649920,1cyclictest0-21swapper/120:15:011
283649920,1cyclictest0-21swapper/120:05:011
283649920,1cyclictest0-21swapper/119:25:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional