You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-27 - 11:17
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rackbslot7.osadl.org (updated Sat Apr 27, 2024 00:44:20)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30430213524,52sleep125-21ksoftirqd/119:05:051
30578999754,36cyclictest8774-21munin-run23:45:043
3056699943,55cyclictest3277-21sh23:35:490
45302900,6sleep10-21swapper/123:36:521
30578998975,8cyclictest0-21swapper/323:05:553
3057799894,63cyclictest399-21runrttasks22:55:412
30578998844,37cyclictest17736-21unixbench_multi20:00:393
30577998829,18cyclictest0-21swapper/223:15:332
30578998631,46cyclictest0-21swapper/319:10:433
24542864,17sleep01986-21/usr/sbin/munin19:15:310
3057799858,45cyclictest0-21swapper/221:10:502
3057799858,45cyclictest0-21swapper/221:10:502
3057799845,47cyclictest0-21swapper/223:05:382
30566998430,20cyclictest141rcu_preempt23:19:280
30578998357,18cyclictest0-21swapper/322:50:353
30571998338,28cyclictest141rcu_preempt22:25:241
30566998347,8cyclictest141rcu_preempt20:05:070
3057899828,23cyclictest21804-21grep22:20:433
3057899823,24cyclictest3771-21wc23:36:223
3057899823,23cyclictest28005-21unixbench_singl22:30:443
30578998230,44cyclictest0-21swapper/319:50:433
30578998228,41cyclictest0-21swapper/323:34:103
30578998228,41cyclictest0-21swapper/323:34:103
30578998226,48cyclictest0-21swapper/321:30:153
30578998215,12cyclictest0-21swapper/323:30:063
30578998215,12cyclictest0-21swapper/323:30:063
3016528212,47sleep330175-21awk19:01:503
3057899813,25cyclictest26303-21cut23:20:273
3057899813,23cyclictest19069-21fschecks_count21:25:253
3057899813,23cyclictest15461-21fschecks_time19:55:243
30578998123,7cyclictest0-21swapper/300:35:043
30578998010,8cyclictest182052/usr/sbin/munin00:00:183
30578998010,8cyclictest182052/usr/sbin/munin00:00:183
30566998040,10cyclictest141rcu_preempt19:40:070
3057899794,19cyclictest28459-21latency_hist20:35:083
30578997934,12cyclictest0-21swapper/323:45:263
30578997934,12cyclictest0-21swapper/323:45:263
3057899793,33cyclictest14957-21latency_hist23:55:063
3057899793,22cyclictest7793-21unixbench_singl19:30:393
3057899793,19cyclictest6412-21grep21:55:433
3057899793,19cyclictest25216-21find21:35:153
30578997927,10cyclictest0-21swapper/323:10:243
3057899792,21cyclictest4675-21timerandwakeup22:45:423
3057899786,17cyclictest374-21sed20:45:423
3057899786,17cyclictest374-21sed20:45:423
3057899785,24cyclictest28969-21fschecks_time20:36:233
3057899784,21cyclictest248972unixbench_multi22:25:423
3057899783,20cyclictest4520-21sed19:20:413
3057899783,20cyclictest10966-21unixbench_multi21:11:403
3057899783,20cyclictest10966-21unixbench_multi21:11:403
3057899783,19cyclictest31234-21grep20:40:433
30578997830,39cyclictest0-21swapper/300:10:063
3057899782,20cyclictest399-21runrttasks22:06:573
3057899782,20cyclictest399-21runrttasks22:06:573
3057899774,29cyclictest18001-21sh21:23:453
3057899773,20cyclictest26142-21head20:25:413
30578997714,12cyclictest0-21swapper/321:01:243
30578997712,14cyclictest0-21swapper/320:10:053
30578997711,11cyclictest0-21swapper/322:35:263
30578997711,11cyclictest0-21swapper/322:35:263
30578997711,10cyclictest0-21swapper/322:56:383
3057799778,48cyclictest0-21swapper/221:31:222
3057799776,45cyclictest7073-21apt-get22:50:182
30577997726,17cyclictest21680-21ssh23:12:342
30577997716,31cyclictest2206-21/usr/sbin/munin21:50:362
30571997743,21cyclictest141rcu_preempt22:37:001
30571997743,21cyclictest141rcu_preempt22:37:001
30571997721,27cyclictest0-21swapper/123:40:191
3057899763,20cyclictest14753-21sshd22:10:213
3057899763,19cyclictest25123-21grep00:10:403
3057899763,19cyclictest1893-21timerwakeupswit20:50:413
30578997629,39cyclictest0-21swapper/319:46:133
30578997624,9cyclictest0-21swapper/319:40:233
30578997624,9cyclictest0-21swapper/319:40:233
3057899762,21cyclictest31213-21apt-config21:45:073
30577997661,9cyclictest6725-21apt-key22:50:072
3057899754,18cyclictest276-21systemd-journal21:20:043
3057899753,21cyclictest909-21fschecks_count22:40:253
3057899753,21cyclictest5598-21sed19:25:283
30578997514,12cyclictest0-21swapper/321:00:073
30571997531,29cyclictest141rcu_preempt22:14:331
30571997524,18cyclictest141rcu_preempt21:20:041
3057899743,29cyclictest681-21cron00:25:043
3057899743,21cyclictest11171-21modprobe22:05:043
3057899743,21cyclictest11171-21modprobe22:05:043
3057899743,19cyclictest22356-21latency20:15:293
30578997421,8cyclictest0-21swapper/322:15:263
30577997415,24cyclictest12380-21/usr/sbin/munin23:50:252
30566997452,14cyclictest14901-40ogrotate23:55:040
3057899733,20cyclictest21195-21timerwakeupswit20:11:323
3057899733,20cyclictest13654-21sshd23:00:213
3057899733,19cyclictest32111-21latency_hist19:10:073
30577997355,11cyclictest28150-21munin-plugin-st20:35:052
30577997323,18cyclictest31912-21/usr/sbin/munin20:45:202
30577997323,18cyclictest31912-21/usr/sbin/munin20:45:202
30577997319,21cyclictest14140-21/usr/sbin/munin19:50:392
30571997325,24cyclictest141rcu_preempt00:20:311
3057899724,19cyclictest28011-21ls00:15:453
30577997222,17cyclictest0-21swapper/221:07:372
30571997239,8cyclictest141rcu_preempt23:14:181
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional