You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-20 - 15:32
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rackbslot7.osadl.org (updated Thu Nov 20, 2025 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12347991123,35cyclictest2417-21timerwakeupswit09:35:030
123479910951,51cyclictest0-21swapper/011:44:320
123479910951,51cyclictest0-21swapper/011:44:320
12347991092,35cyclictest26548-21uniq07:49:500
123479910734,66cyclictest0-21swapper/011:09:440
12347991073,28cyclictest7488-21fschecks_time09:44:540
12347991072,33cyclictest31205-21/usr/sbin/munin08:04:520
12347991063,22cyclictest31828-21ssh11:24:520
12347991063,22cyclictest31828-21ssh11:24:520
12347991054,27cyclictest12230-21awk10:49:580
10874210510,68sleep210891-21/usr/sbin/munin07:04:402
10874210510,68sleep210891-21/usr/sbin/munin07:04:402
12347991043,26cyclictest5251-21/usr/sbin/munin12:30:020
12347991043,26cyclictest5251-21/usr/sbin/munin12:30:020
12347991042,38cyclictest16532-21kworker/u18:1+flush-179:011:29:470
12347991034,35cyclictest3081-21latency_hist10:34:320
12347991033,38cyclictest22881-21latency_hist07:39:320
123479910328,70cyclictest0-21swapper/009:56:100
123479910317,31cyclictest15383-21unixbench_singl10:55:070
123559910162,33cyclictest22421-21/usr/sbin/munin12:04:472
12347991014,36cyclictest456-21latency_hist10:29:320
12347991014,35cyclictest22256-21ssh12:04:430
12347991014,31cyclictest355-21runrttasks12:09:510
12347991013,41cyclictest32020-21timerwakeupswit12:20:040
12347991013,34cyclictest14078-21latency_hist07:14:320
123479910123,21cyclictest26546-21grep09:20:020
123479910038,56cyclictest0-21swapper/011:49:300
123479910038,56cyclictest0-21swapper/011:49:300
12347991003,31cyclictest12232-21latency_hist07:09:330
12347991003,31cyclictest12232-21latency_hist07:09:330
123479910029,9cyclictest0-21swapper/009:49:480
12355999956,37cyclictest31730-21proc_pri12:20:012
1234799994,34cyclictest19021-21fschecks_time10:04:510
12347999941,52cyclictest18985-21kworker/u17:1+xprtiod11:17:580
1234799993,38cyclictest1280-21latency_hist09:34:310
1234799993,36cyclictest355-21runrttasks12:02:160
12347999933,58cyclictest15326-21latency_hist08:54:320
1234799992,35cyclictest22254-21sh09:12:550
1234799983,37cyclictest4119-21fschecks_time08:19:530
1234799983,36cyclictest13892-21cstates11:50:150
1234799983,34cyclictest3241-21acpi10:34:400
1234799983,32cyclictest29459-21grep09:25:070
1234799983,26cyclictest23296-21/usr/sbin/munin07:40:210
1234799983,24cyclictest19851-21/usr/sbin/munin07:29:500
1234799983,24cyclictest19851-21/usr/sbin/munin07:29:500
1234799982,35cyclictest18175-21cstates07:24:460
1234799982,35cyclictest18175-21cstates07:24:460
1234799982,35cyclictest17799-21modprobe07:24:290
1234799974,32cyclictest1506-21/usr/sbin/munin12:24:470
12347999725,10cyclictest0-21swapper/009:14:500
12347999725,10cyclictest0-21swapper/009:14:500
1234799972,37cyclictest17124-21modprobe10:00:280
1234799972,35cyclictest29570-21timerandwakeup11:20:060
1234799972,34cyclictest1572-21grep08:10:450
12347999719,23cyclictest30959-21cat12:19:320
1234799967,34cyclictest9919-21awk10:45:060
1234799965,31cyclictest27985-21runrttasks07:54:430
1234799964,38cyclictest19997-21ssh09:09:280
1234799963,36cyclictest355-21runrttasks10:12:050
1234799963,35cyclictest13261-21modprobe08:49:290
1234799963,35cyclictest13261-21modprobe08:49:290
1234799963,34cyclictest17407-21latency08:59:540
1234799963,33cyclictest20205-21runrttasks11:04:450
1234799963,30cyclictest11860-21latency_hist08:44:320
12347999627,61cyclictest0-21swapper/010:15:070
1234799954,37cyclictest17669-21latency10:59:560
1234799954,34cyclictest8541-21latency_hist10:44:320
1234799954,34cyclictest8541-21latency_hist10:44:320
1234799953,38cyclictest30439-21timerwakeupswit08:00:040
1234799953,36cyclictest5572-21cstates08:24:430
1234799953,34cyclictest16476-21/usr/sbin/munin11:54:470
1234799953,34cyclictest16476-21/usr/sbin/munin11:54:470
1234799953,31cyclictest2473-21fschecks_time08:14:540
12347999527,8cyclictest0-21swapper/007:15:270
12355999447,41cyclictest10201-21latency_hist08:39:322
1235199943,57cyclictest31793-21apt-get11:24:441
1235199943,57cyclictest31793-21apt-get11:24:441
1234799944,32cyclictest4570-21fschecks_count09:39:530
1234799944,32cyclictest4570-21fschecks_count09:39:530
1234799943,35cyclictest27319-21ls10:19:460
1234799942,32cyclictest355-21runrttasks12:34:550
1234799934,34cyclictest6818-21latency_hist08:29:330
1234799934,34cyclictest6818-21latency_hist08:29:330
1234799934,31cyclictest26108-21latency_hist07:49:330
1234799933,30cyclictest7398-21latency_hist11:39:310
1234799924,36cyclictest355-21runrttasks08:37:560
12347999217,20cyclictest16196-21sort08:55:090
12355999049,34cyclictest0-21swapper/210:09:532
12355999048,34cyclictest0-21swapper/209:04:502
12355999047,35cyclictest0-21swapper/210:04:542
1235599903,60cyclictest13638-21ssh11:49:442
153162870,8sleep10-21swapper/110:55:061
12355998765,16cyclictest27861-21latency_hist07:54:322
12355998748,32cyclictest0-21swapper/211:14:542
12355998743,37cyclictest2124-21sh12:24:552
1235599873,42cyclictest23344-21cstates11:10:102
1235599873,29cyclictest25486-21fschecks_time12:09:512
12355998647,34cyclictest12339-21cyclictest08:15:002
12355998544,35cyclictest5524-21netstat11:34:552
1235599853,39cyclictest5455-21/usr/sbin/munin08:25:042
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional