You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-26 - 00:35
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot2.osadl.org (updated Thu Apr 25, 2024 12:44:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
273932195179,11sleep10-21swapper/107:06:531
275122191173,13sleep30-21swapper/307:08:263
275362190173,12sleep20-21swapper/207:08:452
273922187171,11sleep00-21swapper/007:06:510
323002480,0sleep10-21swapper/111:30:261
132562420,0sleep013258-21grep10:40:130
278119992,3cyclictest13342-21awk08:50:010
278149970,6cyclictest0-21swapper/208:55:092
278189966,0cyclictest40-21ksoftirqd/311:15:003
278189960,5cyclictest0-21swapper/312:20:163
278149960,6cyclictest0-21swapper/210:45:132
278119962,4cyclictest15522-21perf12:20:000
278119960,6cyclictest0-21swapper/012:37:170
278119960,6cyclictest0-21swapper/011:09:380
278119960,6cyclictest0-21swapper/010:54:100
278119960,3cyclictest31785-21cron09:15:000
278119960,1cyclictest0-21swapper/011:19:060
278119960,1cyclictest0-21swapper/010:29:020
278119960,1cyclictest0-21swapper/010:09:260
278119960,1cyclictest0-21swapper/009:40:430
278189955,0cyclictest40-21ksoftirqd/310:40:103
278189955,0cyclictest40-21ksoftirqd/309:25:003
278189954,1cyclictest40-21ksoftirqd/311:00:153
278189954,1cyclictest0-21swapper/310:46:213
278189954,0cyclictest40-21ksoftirqd/308:45:093
278189951,1cyclictest18125-21cut09:45:123
278189950,5cyclictest0-21swapper/311:32:023
278189950,5cyclictest0-21swapper/309:38:593
278189950,4cyclictest15127-21apt-get12:00:103
278149955,0cyclictest0-21swapper/210:59:062
278149953,2cyclictest0-21swapper/208:40:162
278149950,5cyclictest18070-21ssh12:01:052
278149950,5cyclictest0-21swapper/212:25:402
278149950,5cyclictest0-21swapper/212:05:102
278129950,5cyclictest0-21swapper/109:35:581
278119953,1cyclictest29337-21ssh10:30:170
278119951,4cyclictest21021-21perf08:00:000
278119950,5cyclictest0-21swapper/012:28:180
278119950,5cyclictest0-21swapper/012:20:010
278119950,5cyclictest0-21swapper/012:08:170
278119950,5cyclictest0-21swapper/012:01:010
278119950,5cyclictest0-21swapper/011:47:090
278119950,5cyclictest0-21swapper/011:41:210
278119950,5cyclictest0-21swapper/011:33:100
278119950,5cyclictest0-21swapper/011:25:260
278119950,5cyclictest0-21swapper/010:36:070
278119950,5cyclictest0-21swapper/010:21:260
278119950,5cyclictest0-21swapper/010:19:180
278119950,5cyclictest0-21swapper/010:12:150
278119950,5cyclictest0-21swapper/009:55:460
278119950,5cyclictest0-21swapper/009:54:260
278119950,5cyclictest0-21swapper/009:45:540
278119950,5cyclictest0-21swapper/009:35:020
278119950,5cyclictest0-21swapper/009:25:260
278119950,5cyclictest0-21swapper/009:15:150
278119950,5cyclictest0-21swapper/009:08:020
278119950,5cyclictest0-21swapper/008:10:220
278119950,4cyclictest0-21swapper/012:10:110
278119950,4cyclictest0-21swapper/011:00:130
278119950,4cyclictest0-21swapper/007:35:110
278119950,1cyclictest0-21swapper/010:45:540
278119950,1cyclictest0-21swapper/009:20:020
278119950,0cyclictest0-21swapper/012:31:130
278119950,0cyclictest0-21swapper/011:50:050
278119950,0cyclictest0-21swapper/011:36:210
278119950,0cyclictest0-21swapper/011:21:220
278119950,0cyclictest0-21swapper/011:11:020
278119950,0cyclictest0-21swapper/010:55:140
278119950,0cyclictest0-21swapper/010:00:260
278119950,0cyclictest0-21swapper/009:30:380
278189944,0cyclictest40-21ksoftirqd/312:35:173
278189944,0cyclictest40-21ksoftirqd/312:30:123
278189944,0cyclictest40-21ksoftirqd/311:15:123
278189944,0cyclictest40-21ksoftirqd/310:00:103
278189944,0cyclictest40-21ksoftirqd/309:30:223
278189944,0cyclictest40-21ksoftirqd/309:00:003
278189944,0cyclictest40-21ksoftirqd/308:50:143
278189944,0cyclictest40-21ksoftirqd/308:30:003
278189944,0cyclictest40-21ksoftirqd/308:15:003
278189944,0cyclictest40-21ksoftirqd/308:00:163
278189944,0cyclictest40-21ksoftirqd/307:40:003
278189944,0cyclictest1117-21kworker/3:011:55:143
278189944,0cyclictest1117-21kworker/3:009:40:543
278189944,0cyclictest1117-21kworker/3:009:07:183
278189943,1cyclictest40-21ksoftirqd/310:50:123
278189943,1cyclictest40-21ksoftirqd/307:20:003
278189943,0cyclictest40-21ksoftirqd/307:35:013
278189942,2cyclictest40-21ksoftirqd/310:05:223
278189942,2cyclictest20561-21sed09:00:563
278189942,1cyclictest22194-21diskstats11:05:153
278189941,3cyclictest40-21ksoftirqd/307:15:113
278189941,3cyclictest381rcuc/311:20:153
278189941,2cyclictest0-21swapper/309:25:103
278189940,4cyclictest61-21kswapd011:40:133
278189940,4cyclictest4293-21ssh09:17:193
278189940,4cyclictest0-21swapper/312:25:103
278189940,4cyclictest0-21swapper/312:10:113
278189940,4cyclictest0-21swapper/312:05:093
278189940,4cyclictest0-21swapper/311:45:123
278189940,4cyclictest0-21swapper/311:25:113
278189940,4cyclictest0-21swapper/310:35:013
278189940,4cyclictest0-21swapper/310:10:133
278189940,4cyclictest0-21swapper/309:10:123
278189940,4cyclictest0-21swapper/308:20:123
278189940,4cyclictest0-21swapper/307:55:093
278189940,4cyclictest0-21swapper/307:10:103
278189940,3cyclictest9803-21tail08:40:153
278189940,3cyclictest7305-21touch12:15:013
278189940,3cyclictest7112-21perl11:35:183
278189940,3cyclictest26234-21apt-get09:50:113
278189940,3cyclictest18124-21users07:50:233
278189940,3cyclictest12170-21hddtemp_smartct07:40:143
278189940,3cyclictest12044-21diskstats10:20:133
278189940,3cyclictest10072-21ls10:00:003
278189940,3cyclictest0-21swapper/307:25:123
278189940,1cyclictest4906-21taskset10:55:093
278189940,1cyclictest31162-21/usr/sbin/munin11:50:263
278189940,1cyclictest3101-21/usr/sbin/munin10:15:183
278189940,1cyclictest13805-21/usr/sbin/munin07:45:143
278149944,0cyclictest0-21swapper/211:37:222
278149942,1cyclictest28999-21sh10:30:132
278149941,3cyclictest13939-21ssh11:20:082
278149941,2cyclictest0-21swapper/209:45:102
278149940,4cyclictest7916-21ssh09:39:152
278149940,4cyclictest7277-21gpgv12:15:012
278149940,4cyclictest23856-21apt-get09:10:112
278149940,4cyclictest22196-21apt-get11:25:112
278149940,4cyclictest16989-21ssh10:23:082
278149940,4cyclictest157872sleep207:46:552
278149940,4cyclictest0-21swapper/212:30:112
278149940,4cyclictest0-21swapper/211:55:102
278149940,4cyclictest0-21swapper/211:41:412
278149940,4cyclictest0-21swapper/211:10:122
278149940,4cyclictest0-21swapper/210:05:112
278149940,4cyclictest0-21swapper/209:55:462
278149940,4cyclictest0-21swapper/209:51:472
278149940,4cyclictest0-21swapper/209:05:122
278149940,4cyclictest0-21swapper/208:30:192
278149940,4cyclictest0-21swapper/207:28:592
278149940,3cyclictest6050-21sed11:35:012
278149940,3cyclictest30532-21wc07:15:012
278149940,3cyclictest30274-21munin-node07:15:002
278149940,3cyclictest27149-21/usr/sbin/munin09:30:242
278149940,3cyclictest2462-21ntp_states08:25:212
278149940,3cyclictest1818-21ssh09:15:222
278149940,3cyclictest16392-21chmod09:25:022
278149940,3cyclictest0-21swapper/208:10:112
278149940,0cyclictest0-21swapper/212:36:172
278149940,0cyclictest0-21swapper/210:52:332
278129944,0cyclictest0-21swapper/112:37:331
278129944,0cyclictest0-21swapper/110:35:551
278129944,0cyclictest0-21swapper/109:42:481
278129941,2cyclictest0-21swapper/112:25:101
278129941,2cyclictest0-21swapper/110:05:111
278129941,2cyclictest0-21swapper/108:55:091
278129940,4cyclictest0-21swapper/111:27:351
278129940,2cyclictest0-21swapper/109:30:101
278129940,0cyclictest0-21swapper/111:49:501
278129940,0cyclictest0-21swapper/111:04:431
278119942,1cyclictest3602-21snmpd08:25:090
278119942,1cyclictest11421-21aten2_r5power_v07:40:110
278119941,3cyclictest0-21swapper/007:22:460
278119941,3cyclictest0-21swapper/007:12:460
278119941,2cyclictest0-21swapper/007:30:090
278119941,2cyclictest0-21swapper/007:25:140
278119941,1cyclictest0-21swapper/008:05:090
278119940,4cyclictest0-21swapper/009:03:090
278119940,4cyclictest0-21swapper/008:53:010
278119940,4cyclictest0-21swapper/008:42:570
278119940,4cyclictest0-21swapper/007:52:500
278119940,4cyclictest0-21swapper/007:47:500
278119940,3cyclictest32660-21sensors_temp08:20:230
278119940,3cyclictest30563-21munin-node07:15:010
278119940,3cyclictest28651-21cut08:15:100
278119940,3cyclictest21240-21apt08:00:130
278119940,3cyclictest0-21swapper/008:55:010
278119940,0cyclictest0-21swapper/011:56:170
11537240,0sleep10-21swapper/111:57:281
278189931,2cyclictest57250irq/125-eth010:30:023
278189931,1cyclictest3602-21snmpd10:25:023
278189930,3cyclictest6125-21apt-get08:35:123
278189930,3cyclictest24247-21awk08:05:113
278189930,3cyclictest0-21swapper/308:30:093
278189930,3cyclictest0-21swapper/308:15:063
278149933,0cyclictest0-21swapper/208:50:102
278149932,1cyclictest32708-21awk12:10:212
278149932,1cyclictest21422-21ssh11:05:022
278149932,1cyclictest19742-21uname07:55:162
278149931,1cyclictest0-21swapper/207:35:112
278149930,3cyclictest27210-21apt-get10:10:122
278149930,3cyclictest22672-21ssh11:45:022
278149930,3cyclictest13313-21apt-get11:00:112
278149930,3cyclictest0-21swapper/212:20:052
278149930,3cyclictest0-21swapper/211:50:012
278149930,3cyclictest0-21swapper/211:15:132
278149930,3cyclictest0-21swapper/210:40:092
278149930,3cyclictest0-21swapper/210:35:012
278149930,3cyclictest0-21swapper/210:25:142
278149930,3cyclictest0-21swapper/210:15:022
278149930,3cyclictest0-21swapper/210:00:022
278149930,3cyclictest0-21swapper/209:40:112
278149930,3cyclictest0-21swapper/209:20:102
278149930,3cyclictest0-21swapper/209:00:092
278149930,3cyclictest0-21swapper/208:45:012
278149930,3cyclictest0-21swapper/208:35:012
278149930,3cyclictest0-21swapper/208:20:112
278149930,3cyclictest0-21swapper/208:15:102
278149930,3cyclictest0-21swapper/208:05:082
278149930,3cyclictest0-21swapper/208:00:112
278149930,3cyclictest0-21swapper/207:50:102
278149930,3cyclictest0-21swapper/207:40:102
278149930,3cyclictest0-21swapper/207:30:092
278149930,3cyclictest0-21swapper/207:20:112
278129933,0cyclictest0-21swapper/111:37:271
278129931,2cyclictest0-21swapper/112:30:171
278129931,2cyclictest0-21swapper/109:00:091
278129931,1cyclictest0-21swapper/112:15:021
278129931,1cyclictest0-21swapper/112:05:101
278129931,1cyclictest0-21swapper/112:00:101
278129931,1cyclictest0-21swapper/111:50:121
278129931,1cyclictest0-21swapper/111:15:121
278129931,1cyclictest0-21swapper/111:10:121
278129931,1cyclictest0-21swapper/110:55:101
278129931,1cyclictest0-21swapper/110:40:091
278129931,1cyclictest0-21swapper/110:10:121
278129931,1cyclictest0-21swapper/110:00:111
278129931,1cyclictest0-21swapper/109:50:111
278129931,1cyclictest0-21swapper/109:45:011
278129931,1cyclictest0-21swapper/109:15:101
278129931,1cyclictest0-21swapper/109:05:131
278129931,1cyclictest0-21swapper/108:50:131
278129931,1cyclictest0-21swapper/108:35:121
278129931,1cyclictest0-21swapper/108:30:091
278129931,1cyclictest0-21swapper/108:15:101
278129931,1cyclictest0-21swapper/108:05:081
278129931,1cyclictest0-21swapper/108:00:131
278129931,1cyclictest0-21swapper/107:55:081
278129931,1cyclictest0-21swapper/107:50:101
278129931,1cyclictest0-21swapper/107:30:091
278129931,1cyclictest0-21swapper/107:15:121
278129930,2cyclictest31212-21chrt09:14:341
278129930,2cyclictest0-21swapper/110:45:111
278129930,2cyclictest0-21swapper/107:35:111
278129930,1cyclictest0-21swapper/111:40:121
278119931,1cyclictest0-21swapper/008:30:090
278119930,3cyclictest0-21swapper/008:35:120
16437230,0sleep10-21swapper/111:20:411
1571230,0sleep10-21swapper/112:10:421
278129922,0cyclictest0-21swapper/107:45:061
278129921,1cyclictest0-21swapper/109:25:101
278129921,1cyclictest0-21swapper/108:40:091
278129921,1cyclictest0-21swapper/107:10:111
278129920,1cyclictest3076-21chrt10:15:081
278129920,1cyclictest28095-21chrt08:11:051
278129920,1cyclictest27932-21chrt10:30:011
278129920,1cyclictest1182-21chrt08:25:091
278129920,1cyclictest11410-21chrt10:20:081
278129920,1cyclictest11072-21chrt08:45:031
278129920,1cyclictest0-21swapper/112:20:171
278129920,1cyclictest0-21swapper/111:05:121
278129920,1cyclictest0-21swapper/110:50:091
278129920,1cyclictest0-21swapper/110:25:101
278129920,1cyclictest0-21swapper/109:55:111
278129920,1cyclictest0-21swapper/109:20:101
278129920,1cyclictest0-21swapper/108:20:111
278129920,1cyclictest0-21swapper/107:40:101
278129920,1cyclictest0-21swapper/107:25:131
278129920,1cyclictest0-21swapper/107:20:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional