You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-05-27 - 08:13
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot2.osadl.org (updated Mon May 27, 2024 00:44:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
120792201184,12sleep10-21swapper/119:07:111
119972193178,11sleep00-21swapper/019:06:090
119902190172,13sleep30-21swapper/319:06:043
119592181162,14sleep20-21swapper/219:05:402
139322500,0sleep00-21swapper/000:30:450
288412490,0sleep10-21swapper/121:41:071
310532460,0sleep10-21swapper/123:41:591
84642440,0sleep10-21swapper/121:49:161
222852440,0sleep10-21swapper/121:17:371
7642100,0sleep00-21swapper/021:24:500
124809971,3cyclictest20576-21basename19:25:010
124899960,6cyclictest0-21swapper/223:10:092
124809960,6cyclictest0-21swapper/021:43:420
124979955,0cyclictest5323-21kworker/3:022:41:583
124979955,0cyclictest5323-21kworker/3:021:46:003
124979955,0cyclictest5323-21kworker/3:021:12:263
124979955,0cyclictest5323-21kworker/3:000:33:553
124979955,0cyclictest5323-21kworker/3:000:22:463
124979955,0cyclictest5323-21kworker/3:000:11:333
124979955,0cyclictest27014-21kworker/3:220:05:143
124979955,0cyclictest0-21swapper/323:57:243
124979955,0cyclictest0-21swapper/323:39:113
124979955,0cyclictest0-21swapper/322:50:183
124979955,0cyclictest0-21swapper/322:24:593
124979951,4cyclictest0-21swapper/323:51:123
124979951,2cyclictest0-21swapper/322:30:113
124979950,5cyclictest0-21swapper/322:19:363
124979950,5cyclictest0-21swapper/322:11:543
124979950,5cyclictest0-21swapper/319:15:143
124979950,4cyclictest3375-21diskstats23:45:173
124979950,4cyclictest18580-21ssh22:35:103
124979950,0cyclictest8694-21diskmemload23:42:153
124899955,0cyclictest9580-21apt-get22:10:022
124899955,0cyclictest0-21swapper/223:06:122
124899954,1cyclictest19724-21if_eth020:25:142
124899952,3cyclictest3602-21snmpd23:40:242
124899951,3cyclictest3602-21snmpd21:30:472
124899950,5cyclictest18853-21ssh21:55:192
124899950,5cyclictest0-21swapper/222:57:442
124899950,5cyclictest0-21swapper/200:30:122
124899950,0cyclictest0-21swapper/200:39:542
124859952,2cyclictest0-21swapper/119:30:101
124859951,3cyclictest0-21swapper/120:50:071
124859951,2cyclictest0-21swapper/122:40:101
124859951,2cyclictest0-21swapper/120:20:111
124859950,5cyclictest0-21swapper/123:26:521
124859950,5cyclictest0-21swapper/122:07:161
124809953,1cyclictest28181-21munin-run19:40:000
124809950,5cyclictest0-21swapper/022:53:580
124809950,5cyclictest0-21swapper/022:46:160
124809950,5cyclictest0-21swapper/022:25:280
124809950,5cyclictest0-21swapper/021:52:080
124809950,5cyclictest0-21swapper/021:33:130
124809950,5cyclictest0-21swapper/021:26:140
124809950,5cyclictest0-21swapper/021:17:460
124809950,5cyclictest0-21swapper/000:06:250
124809950,4cyclictest15885-21apt-get20:20:000
124809950,4cyclictest0-21swapper/022:33:000
124809950,0cyclictest0-21swapper/022:17:330
124809950,0cyclictest0-21swapper/019:50:000
24990240,0sleep00-21swapper/023:38:410
124979944,0cyclictest5323-21kworker/3:023:26:463
124979944,0cyclictest5323-21kworker/3:023:04:223
124979944,0cyclictest5323-21kworker/3:020:38:493
124979944,0cyclictest27014-21kworker/3:219:54:023
124979944,0cyclictest0-21swapper/320:00:043
124979942,2cyclictest1090-21ssh22:04:453
124979941,3cyclictest0-21swapper/323:15:253
124979941,2cyclictest0-21swapper/321:05:143
124979941,1cyclictest0-21swapper/323:10:083
124979941,1cyclictest0-21swapper/321:20:133
124979940,4cyclictest8694-21diskmemload21:53:343
124979940,4cyclictest3045-21diskstats22:45:153
124979940,4cyclictest19429-21cut20:25:133
124979940,4cyclictest0-21swapper/323:30:093
124979940,4cyclictest0-21swapper/323:20:123
124979940,4cyclictest0-21swapper/322:25:313
124979940,4cyclictest0-21swapper/321:55:133
124979940,4cyclictest0-21swapper/321:40:133
124979940,4cyclictest0-21swapper/321:36:123
124979940,4cyclictest0-21swapper/321:25:133
124979940,4cyclictest0-21swapper/320:30:123
124979940,4cyclictest0-21swapper/300:05:133
124979940,4cyclictest0-21swapper/300:01:133
124979940,3cyclictest9582-21ssh21:30:103
124979940,3cyclictest8694-21diskmemload00:18:353
124979940,3cyclictest5260-21users19:55:313
124979940,3cyclictest3527-21latency_hist21:00:003
124979940,3cyclictest31773-21df_inode20:50:133
124979940,3cyclictest30776-21sed19:45:003
124979940,3cyclictest19623-21/usr/sbin/munin00:35:433
124979940,3cyclictest13326-21perl19:10:143
124979940,3cyclictest0-21swapper/319:20:123
124979940,0cyclictest0-21swapper/319:47:233
124899944,0cyclictest0-21swapper/221:16:282
124899944,0cyclictest0-21swapper/219:35:212
124899943,1cyclictest301ktimersoftd/200:00:142
124899943,1cyclictest26710-21/usr/sbin/munin22:40:272
124899940,4cyclictest27608-21perl23:00:152
124899940,4cyclictest0-21swapper/223:58:172
124899940,4cyclictest0-21swapper/223:26:182
124899940,4cyclictest0-21swapper/221:46:482
124899940,4cyclictest0-21swapper/221:39:182
124899940,4cyclictest0-21swapper/221:00:102
124899940,4cyclictest0-21swapper/220:20:102
124899940,4cyclictest0-21swapper/200:17:562
124899940,3cyclictest6332-21ssh22:07:222
124899940,3cyclictest6021-21apt-get21:05:002
124899940,3cyclictest31769-21df_inode20:50:122
124899940,3cyclictest30635-21users20:45:282
124899940,3cyclictest28055-21diskstats00:20:132
124899940,3cyclictest25875-21/usr/sbin/munin21:40:162
124899940,3cyclictest23595-21latency_hist20:35:012
124899940,3cyclictest15185-21perl19:15:012
124899940,3cyclictest11657-21df00:10:142
124899940,3cyclictest10952-21df22:30:122
124899940,3cyclictest10650-21sed23:50:002
124899940,3cyclictest10385-21diskstats21:50:152
124899940,3cyclictest0-21swapper/220:40:102
124899940,2cyclictest0-21swapper/221:26:002
124899940,1cyclictest26157-21/usr/sbin/munin22:00:142
124899940,1cyclictest25677-21/usr/sbin/munin21:20:312
124899940,1cyclictest23340-21tr19:30:112
124899940,1cyclictest2221-21ssh22:25:092
124899940,1cyclictest19024-21cut23:35:132
124899940,1cyclictest18561-21/usr/sbin/munin22:35:182
124899940,1cyclictest11034-21/usr/sbin/munin20:10:152
124899940,1cyclictest10867-21/usr/sbin/munin23:50:122
124899940,0cyclictest0-21swapper/222:49:192
124899940,0cyclictest0-21swapper/222:22:242
124899940,0cyclictest0-21swapper/200:06:172
124859944,0cyclictest0-21swapper/123:21:301
124859944,0cyclictest0-21swapper/122:30:331
124859941,2cyclictest0-21swapper/123:50:121
124859941,2cyclictest0-21swapper/123:05:131
124859941,2cyclictest0-21swapper/122:35:091
124859941,2cyclictest0-21swapper/122:20:101
124859941,2cyclictest0-21swapper/122:10:121
124859941,2cyclictest0-21swapper/121:55:141
124859941,2cyclictest0-21swapper/121:25:131
124859941,2cyclictest0-21swapper/121:10:131
124859941,2cyclictest0-21swapper/120:40:111
124859941,2cyclictest0-21swapper/120:30:121
124859941,2cyclictest0-21swapper/120:25:081
124859941,2cyclictest0-21swapper/119:10:101
124859941,1cyclictest0-21swapper/123:10:081
124859940,2cyclictest0-21swapper/119:15:111
124859940,0cyclictest0-21swapper/122:55:181
124859940,0cyclictest0-21swapper/100:07:201
124809944,0cyclictest0-21swapper/023:57:250
124809944,0cyclictest0-21swapper/023:16:130
124809944,0cyclictest0-21swapper/021:59:070
124809944,0cyclictest0-21swapper/020:50:090
124809941,3cyclictest3602-21snmpd00:19:540
124809941,3cyclictest0-21swapper/023:41:120
124809940,4cyclictest31870-21ssh22:23:050
124809940,4cyclictest0-21swapper/022:40:050
124809940,4cyclictest0-21swapper/022:10:130
124809940,4cyclictest0-21swapper/021:46:060
124809940,4cyclictest0-21swapper/000:35:150
124809940,4cyclictest0-21swapper/000:14:110
124809940,3cyclictest8914-21gpg-connect-age21:10:000
124809940,3cyclictest6359-21grep20:00:110
124809940,3cyclictest3371-21df_inode23:45:170
124809940,3cyclictest3268-21ssh00:25:120
124809940,3cyclictest3207-21ssh23:05:180
124809940,3cyclictest31178-21/usr/sbin/munin19:45:140
124809940,3cyclictest30247-21ssh00:21:000
124809940,3cyclictest28022-21tune2fs00:00:210
124809940,3cyclictest27714-21df_inode23:20:210
124809940,3cyclictest26977-21perl20:40:140
124809940,3cyclictest24097-21/usr/sbin/munin19:30:170
124809940,3cyclictest23082-21ssh21:38:080
124809940,3cyclictest2221-21cron23:25:010
124809940,3cyclictest15544-21python19:15:010
124809940,3cyclictest14293-21ssh23:31:340
124809940,3cyclictest110252sleep020:10:020
124809940,3cyclictest10730-21ssh23:50:040
124809940,3cyclictest0-21swapper/022:35:100
124809940,3cyclictest0-21swapper/022:05:040
124809940,3cyclictest0-21swapper/020:55:110
124809940,3cyclictest0-21swapper/020:20:020
124809940,0cyclictest0-21swapper/023:04:450
124809940,0cyclictest0-21swapper/022:59:220
124979931,1cyclictest0-21swapper/320:15:113
124979930,3cyclictest3172-21apt-get00:25:103
124979930,3cyclictest2623-21apt23:05:143
124979930,3cyclictest0-21swapper/322:55:113
124979930,3cyclictest0-21swapper/322:05:123
124979930,3cyclictest0-21swapper/321:15:063
124979930,3cyclictest0-21swapper/321:00:043
124979930,3cyclictest0-21swapper/320:45:113
124979930,3cyclictest0-21swapper/320:40:103
124979930,3cyclictest0-21swapper/320:20:023
124979930,3cyclictest0-21swapper/320:10:103
124979930,3cyclictest0-21swapper/319:30:113
124979930,3cyclictest0-21swapper/319:25:103
124979930,2cyclictest0-21swapper/319:35:013
124899932,1cyclictest30888-21apt-get19:45:122
124899930,3cyclictest8567-21apt20:05:102
124899930,3cyclictest61-21kswapd023:30:082
124899930,3cyclictest10382-21rm22:50:032
124899930,3cyclictest0-21swapper/223:20:122
124899930,3cyclictest0-21swapper/223:15:102
124899930,3cyclictest0-21swapper/222:15:132
124899930,3cyclictest0-21swapper/221:10:012
124899930,3cyclictest0-21swapper/220:55:032
124899930,3cyclictest0-21swapper/220:30:052
124899930,3cyclictest0-21swapper/220:15:112
124899930,3cyclictest0-21swapper/220:00:102
124899930,3cyclictest0-21swapper/219:50:092
124899930,3cyclictest0-21swapper/219:40:142
124899930,3cyclictest0-21swapper/219:10:102
124899930,3cyclictest0-21swapper/200:25:102
124899930,2cyclictest3997-21grep19:55:162
124899930,1cyclictest20783-21/usr/sbin/munin19:25:082
124899930,0cyclictest0-21swapper/219:20:152
124859933,0cyclictest0-21swapper/123:49:081
124859933,0cyclictest0-21swapper/123:03:001
124859933,0cyclictest0-21swapper/122:52:591
124859933,0cyclictest0-21swapper/122:28:141
124859933,0cyclictest0-21swapper/100:39:381
124859933,0cyclictest0-21swapper/100:32:551
124859933,0cyclictest0-21swapper/100:03:321
124859931,2cyclictest0-21swapper/119:35:011
124859931,1cyclictest0-21swapper/123:55:121
124859931,1cyclictest0-21swapper/123:30:081
124859931,1cyclictest0-21swapper/123:15:111
124859931,1cyclictest0-21swapper/122:00:131
124859931,1cyclictest0-21swapper/121:50:101
124859931,1cyclictest0-21swapper/121:30:111
124859931,1cyclictest0-21swapper/121:05:151
124859931,1cyclictest0-21swapper/121:00:101
124859931,1cyclictest0-21swapper/120:55:111
124859931,1cyclictest0-21swapper/120:45:121
124859931,1cyclictest0-21swapper/120:05:101
124859931,1cyclictest0-21swapper/119:55:101
124859931,1cyclictest0-21swapper/119:50:101
124859931,1cyclictest0-21swapper/119:40:131
124859931,1cyclictest0-21swapper/119:20:141
124859931,1cyclictest0-21swapper/100:25:091
124859931,1cyclictest0-21swapper/100:20:101
124859931,1cyclictest0-21swapper/100:15:021
124859930,3cyclictest0-21swapper/119:29:321
124859930,2cyclictest30883-21taskset19:45:101
124859930,2cyclictest0-21swapper/122:45:121
124859930,2cyclictest0-21swapper/122:15:121
124859930,2cyclictest0-21swapper/120:10:111
124859930,1cyclictest0-21swapper/123:35:111
124809930,3cyclictest0-21swapper/023:10:020
124809930,3cyclictest0-21swapper/022:00:130
124809930,3cyclictest0-21swapper/021:10:020
124809930,3cyclictest0-21swapper/021:00:090
124809930,3cyclictest0-21swapper/020:45:120
124809930,3cyclictest0-21swapper/020:35:100
124809930,3cyclictest0-21swapper/020:30:120
124809930,3cyclictest0-21swapper/020:25:090
124809930,3cyclictest0-21swapper/020:05:040
124809930,3cyclictest0-21swapper/019:55:110
124809930,3cyclictest0-21swapper/019:40:040
124809930,3cyclictest0-21swapper/019:25:090
124809930,3cyclictest0-21swapper/019:10:200
124859921,1cyclictest0-21swapper/120:15:121
124859921,1cyclictest0-21swapper/100:10:111
124859920,2cyclictest0-21swapper/121:35:131
124859920,1cyclictest25495-21chrt21:20:021
124859920,1cyclictest0-21swapper/120:35:111
124859920,1cyclictest0-21swapper/120:00:161
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional