You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-01 - 08:25
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot2.osadl.org (updated Tue Jul 01, 2025 00:44:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
102532204192,8sleep10-21swapper/119:05:031
124372179162,12sleep30-21swapper/319:09:273
124412178158,15sleep20-21swapper/219:09:292
123322175158,12sleep00-21swapper/019:08:030
45592470,0sleep00-21swapper/000:06:560
220582410,0sleep10-21swapper/123:42:071
15549290,0sleep20-21swapper/223:06:052
126599992,6cyclictest7366-21ssh22:13:483
126599991,1cyclictest0-21swapper/319:20:173
126599990,8cyclictest0-21swapper/319:50:213
126599990,1cyclictest9801-21ssh00:10:083
126599990,1cyclictest25903-21apt-get20:40:123
126499994,2cyclictest8086-21diskmemload21:48:500
126499993,2cyclictest8086-21diskmemload23:47:340
126499993,2cyclictest8086-21diskmemload23:43:010
126499993,2cyclictest25694-21chrt19:32:430
126499993,2cyclictest17246-21ntp_kernel_pll_20:20:200
126499993,2cyclictest10064-21sensors20:05:190
126499992,5cyclictest29565-21ssh00:02:200
126499992,2cyclictest0-21swapper/022:21:290
126499990,1cyclictest21736-21apt-get23:10:010
10849290,0sleep10-21swapper/123:36:191
22712280,0sleep00-21swapper/021:32:510
19488280,0sleep20-21swapper/223:57:132
126599982,1cyclictest6255-21cat20:00:113
126599982,1cyclictest28055-21ssh00:01:223
126599980,7cyclictest0-21swapper/321:50:203
126599980,2cyclictest23757-21ntp_states22:05:213
126599980,2cyclictest19558-21ssh21:30:523
126599980,2cyclictest14297-21rm23:21:493
126599980,1cyclictest9381-21cron00:10:013
126599980,1cyclictest8551-21/usr/sbin/munin20:05:193
126599980,1cyclictest8094-21munin-run21:10:013
126599980,1cyclictest7252-21sh21:57:203
126599980,1cyclictest5623-21munin-run21:05:013
126599980,1cyclictest30841-21apt-get20:50:113
126599980,1cyclictest29787-21cut00:20:143
126599980,1cyclictest29072-21users00:35:203
126599980,1cyclictest28346-21cron19:40:003
126599980,1cyclictest24348-21munin-run22:55:013
126599980,1cyclictest23421-21munin-run19:30:003
126599980,1cyclictest220-21systemd-journal20:24:593
126599980,1cyclictest19074-21/usr/sbin/munin22:35:223
126599980,1cyclictest1861-21tr23:00:013
126599980,1cyclictest18264-21munin-run19:20:003
126599980,1cyclictest1402-21ntp_states23:15:183
126599980,1cyclictest13983-21munin-run23:55:013
126599980,1cyclictest13271-21apt20:15:003
126599980,1cyclictest1-21systemd19:15:003
126499983,3cyclictest9851-21ssh22:30:370
126499983,3cyclictest8086-21diskmemload21:53:040
126499983,3cyclictest4553-21ssh21:24:110
126499983,3cyclictest19621-21ssh00:15:050
126499983,3cyclictest17109-21ssh22:35:020
126499983,3cyclictest15251-21ssh22:00:550
126499983,2cyclictest9634-21sh00:10:020
126499983,2cyclictest8340-21ssh21:10:020
126499983,2cyclictest5547-21rm22:29:120
126499983,2cyclictest30398-21ssh21:36:220
126499983,2cyclictest30264-21rm22:40:520
126499983,2cyclictest283202sleep019:39:450
126499983,2cyclictest26996-21ssh21:19:130
126499983,2cyclictest26735-21hddtemp_smartct20:40:150
126499983,2cyclictest26492-21ssh23:28:140
126499983,2cyclictest20391-21sshd19:20:440
126499983,2cyclictest19886-21ssh22:52:120
126499983,2cyclictest16547-21ssh22:18:180
126499983,2cyclictest15995-21ssh23:22:530
126499983,2cyclictest14973-21sshd19:14:440
126499983,2cyclictest12643-21cyclictest23:32:370
126499983,2cyclictest12108-21load20:10:180
126499982,2cyclictest8086-21diskmemload22:47:250
126499982,2cyclictest756-21runrttasks20:51:590
126599972,1cyclictest21808-21ssh23:41:573
126599971,1cyclictest934-21aten2_r5power_e20:55:123
126599971,1cyclictest7065-21cut21:25:163
126599971,1cyclictest29456-21diskstats21:20:173
126599971,1cyclictest28254-21awk21:35:233
126599971,1cyclictest19960-21cat21:15:173
126599971,0cyclictest0-21swapper/300:15:113
126599970,2cyclictest15080-21diskstats23:55:143
126599970,1cyclictest8837-21ntpq22:30:213
126599970,1cyclictest8645-21ssh21:10:143
126599970,1cyclictest7780-21diskstats00:25:153
126599970,1cyclictest7304-21ssh23:35:133
126599970,1cyclictest5729-21perl22:45:173
126599970,1cyclictest5011-21ssh21:40:163
126599970,1cyclictest4110-21df19:55:143
126599970,1cyclictest32070-21irqstats19:45:163
126599970,1cyclictest30468-21df22:25:183
126599970,1cyclictest30336-21df23:30:133
126599970,1cyclictest29671-21wc19:40:223
126599970,1cyclictest27681-21tr22:40:143
126599970,1cyclictest2632-21rm23:00:133
126599970,1cyclictest25649-21latency_hist20:40:003
126599970,1cyclictest24373-21perl19:30:163
126599970,1cyclictest22474-21ssh23:10:133
126599970,1cyclictest20942-21/usr/sbin/munin20:30:153
126599970,1cyclictest20685-21perl22:20:193
126599970,1cyclictest19987-21cut23:25:143
126599970,1cyclictest19174-21grep20:25:153
126599970,1cyclictest17149-21ssh00:30:133
126599970,1cyclictest14526-21aten2_r5power_c21:45:173
126599970,1cyclictest14342-21sh22:16:543
126599970,1cyclictest14311-21tail20:15:193
126599970,1cyclictest13005-21ssh23:05:173
126599970,1cyclictest12847-21diskstats22:00:143
126599970,1cyclictest0-21swapper/323:45:153
126499976,1cyclictest13510-21apt-get20:15:160
126499975,2cyclictest0-21swapper/021:00:130
126499973,2cyclictest24541-21ssh22:05:280
126499972,3cyclictest0-21swapper/019:45:430
126499972,2cyclictest756-21runrttasks21:09:130
126499972,2cyclictest5946-21ssh21:40:190
126499972,2cyclictest30391-21runrttasks20:47:240
126499972,2cyclictest23126-21runrttasks19:26:330
126499972,2cyclictest17896-21sshd19:15:440
126499972,2cyclictest13131-21ssh21:28:150
126499972,2cyclictest12643-21cyclictest20:38:120
126499972,2cyclictest0-21swapper/023:55:250
126499972,2cyclictest0-21swapper/023:50:360
126499972,2cyclictest0-21swapper/023:35:040
126499972,2cyclictest0-21swapper/023:15:330
126499972,2cyclictest0-21swapper/023:00:100
126499972,2cyclictest0-21swapper/022:55:050
126499972,2cyclictest0-21swapper/022:10:340
126499972,2cyclictest0-21swapper/021:55:020
126499972,2cyclictest0-21swapper/020:00:140
126499972,2cyclictest0-21swapper/000:30:330
126499972,2cyclictest0-21swapper/000:25:560
126499972,2cyclictest0-21swapper/000:20:030
126499971,3cyclictest0-21swapper/023:06:010
126499970,7cyclictest0-21swapper/000:35:110
126499970,2cyclictest20520-21sshd20:27:470
126499970,2cyclictest1151-21aten2_r5power_e19:50:130
5858260,0sleep10-21swapper/122:12:511
126599960,1cyclictest28431-21/usr/sbin/munin20:45:133
126549960,1cyclictest3761-21ssh00:39:532
126499964,2cyclictest1255-21ls20:55:130
126499961,3cyclictest0-21swapper/020:31:520
126499960,2cyclictest5493-21chrt19:55:530
126499960,2cyclictest307112sleep019:43:290
126549954,1cyclictest0-21swapper/223:10:322
126549950,5cyclictest0-21swapper/222:45:132
126549950,5cyclictest0-21swapper/221:53:422
126549950,5cyclictest0-21swapper/221:16:112
126549950,5cyclictest0-21swapper/200:24:452
126549950,4cyclictest19809-21ssh00:15:122
126549950,0cyclictest0-21swapper/223:42:162
126549950,0cyclictest0-21swapper/223:34:512
126509950,5cyclictest0-21swapper/123:13:261
126509950,5cyclictest0-21swapper/122:59:181
126509950,5cyclictest0-21swapper/121:53:161
23012240,0sleep10-21swapper/123:59:291
126549944,0cyclictest291rcuc/219:15:002
126549942,2cyclictest16779-21kworker/2:123:16:102
126549941,3cyclictest16779-21kworker/2:121:43:462
126549940,4cyclictest8086-21diskmemload23:35:122
126549940,4cyclictest30841-21apt-get20:50:122
126549940,4cyclictest0-21swapper/223:50:032
126549940,4cyclictest0-21swapper/223:45:142
126549940,4cyclictest0-21swapper/223:28:262
126549940,4cyclictest0-21swapper/223:00:452
126549940,4cyclictest0-21swapper/222:55:012
126549940,4cyclictest0-21swapper/222:52:582
126549940,4cyclictest0-21swapper/222:42:212
126549940,4cyclictest0-21swapper/222:31:092
126549940,4cyclictest0-21swapper/222:20:152
126549940,4cyclictest0-21swapper/222:10:172
126549940,4cyclictest0-21swapper/222:02:362
126549940,4cyclictest0-21swapper/221:55:162
126549940,4cyclictest0-21swapper/221:47:022
126549940,4cyclictest0-21swapper/221:35:162
126549940,4cyclictest0-21swapper/221:25:132
126549940,4cyclictest0-21swapper/221:23:012
126549940,4cyclictest0-21swapper/221:13:252
126549940,4cyclictest0-21swapper/219:40:162
126549940,4cyclictest0-21swapper/200:11:452
126549940,4cyclictest0-21swapper/200:06:072
126549940,4cyclictest0-21swapper/200:03:092
126549940,3cyclictest6749-21perl20:00:142
126549940,3cyclictest4119-21perl21:00:152
126549940,3cyclictest31113-21ssh22:09:372
126549940,3cyclictest29507-21ssh22:25:012
126549940,3cyclictest21894-21sh00:32:032
126549940,3cyclictest21176-21/usr/sbin/munin19:25:152
126549940,3cyclictest17078-21ssh22:35:022
126549940,3cyclictest15523-21diskstats19:15:012
126549940,3cyclictest15359-21ssh23:22:302
126549940,3cyclictest13464-21date20:15:012
126549940,3cyclictest12114-21ssh22:15:292
126549940,3cyclictest10099-21ssh00:25:532
126509944,0cyclictest0-21swapper/123:06:011
126509944,0cyclictest0-21swapper/122:35:531
126509944,0cyclictest0-21swapper/122:25:061
126509944,0cyclictest0-21swapper/121:59:571
126509944,0cyclictest0-21swapper/121:23:521
126509944,0cyclictest0-21swapper/120:45:181
126509944,0cyclictest0-21swapper/100:30:331
126509944,0cyclictest0-21swapper/100:13:301
126509941,2cyclictest0-21swapper/121:45:181
126509941,2cyclictest0-21swapper/119:15:111
126509941,2cyclictest0-21swapper/100:35:111
126509940,4cyclictest0-21swapper/121:40:191
126509940,2cyclictest0-21swapper/119:35:191
27491230,0sleep10-21swapper/123:28:541
126549932,1cyclictest30997-21snmpd20:40:042
126549931,2cyclictest11172chrt19:50:122
126549931,1cyclictest0-21swapper/220:35:122
126549931,1cyclictest0-21swapper/219:30:142
126549930,3cyclictest9249-21perl20:05:132
126549930,3cyclictest2001-21sed20:55:152
126549930,3cyclictest0-21swapper/221:30:022
126549930,3cyclictest0-21swapper/221:05:152
126549930,3cyclictest0-21swapper/220:45:052
126549930,3cyclictest0-21swapper/220:30:132
126549930,3cyclictest0-21swapper/220:25:152
126549930,3cyclictest0-21swapper/220:20:162
126549930,3cyclictest0-21swapper/220:15:162
126549930,3cyclictest0-21swapper/219:55:132
126549930,3cyclictest0-21swapper/219:45:122
126549930,3cyclictest0-21swapper/219:35:182
126549930,3cyclictest0-21swapper/219:20:012
126509933,0cyclictest0-21swapper/100:27:471
126509931,1cyclictest0-21swapper/123:50:141
126509931,1cyclictest0-21swapper/123:45:131
126509931,1cyclictest0-21swapper/123:15:111
126509931,1cyclictest0-21swapper/123:00:131
126509931,1cyclictest0-21swapper/122:50:131
126509931,1cyclictest0-21swapper/122:45:151
126509931,1cyclictest0-21swapper/122:40:151
126509931,1cyclictest0-21swapper/122:30:151
126509931,1cyclictest0-21swapper/122:20:161
126509931,1cyclictest0-21swapper/121:30:011
126509931,1cyclictest0-21swapper/121:00:141
126509931,1cyclictest0-21swapper/120:50:121
126509931,1cyclictest0-21swapper/120:15:151
126509931,1cyclictest0-21swapper/120:10:141
126509931,1cyclictest0-21swapper/120:05:111
126509931,1cyclictest0-21swapper/119:55:131
126509931,1cyclictest0-21swapper/119:45:111
126509931,1cyclictest0-21swapper/119:20:161
126509931,1cyclictest0-21swapper/100:20:151
126509931,1cyclictest0-21swapper/100:15:111
126509930,3cyclictest0-21swapper/120:25:301
126509930,2cyclictest4589-21chrt00:06:571
126509930,2cyclictest450-21chrt00:04:291
126509930,2cyclictest0-21swapper/123:30:131
126509930,2cyclictest0-21swapper/121:15:131
126509921,1cyclictest0-21swapper/120:35:131
126509921,1cyclictest0-21swapper/120:00:121
126509920,2cyclictest31042sleep119:51:551
126509920,1cyclictest28090-21chrt20:44:341
126509920,1cyclictest0-21swapper/123:20:141
126509920,1cyclictest0-21swapper/122:15:161
126509920,1cyclictest0-21swapper/122:05:051
126509920,1cyclictest0-21swapper/122:00:021
126509920,1cyclictest0-21swapper/121:35:541
126509920,1cyclictest0-21swapper/121:25:141
126509920,1cyclictest0-21swapper/121:10:141
126509920,1cyclictest0-21swapper/121:10:001
126509920,1cyclictest0-21swapper/120:30:151
126509920,1cyclictest0-21swapper/120:20:161
126509920,1cyclictest0-21swapper/119:40:171
126509920,1cyclictest0-21swapper/119:30:141
126509920,1cyclictest0-21swapper/119:25:021
126509920,1cyclictest0-21swapper/119:10:131
126509920,0cyclictest0-21swapper/120:55:551
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional