You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-08-30 - 08:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Sat Aug 30, 2025 00:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
173417599572,50cyclictest1-21systemd19:50:011
173417099535,4cyclictest131rcu_preempt22:20:000
1734175995236,12cyclictest272-21dbus-daemon23:59:591
173417599522,4cyclictest131rcu_preempt00:30:021
173417099523,4cyclictest131rcu_preempt20:15:010
173418099515,3cyclictest131rcu_preempt23:50:012
1734175995133,13cyclictest1-21systemd20:05:001
1734180995011,7cyclictest131rcu_preempt22:10:012
173417599503,3cyclictest131rcu_preempt19:40:021
173417099502,4cyclictest131rcu_preempt21:00:000
1734185994923,11cyclictest272-21dbus-daemon20:45:013
173418099492,10cyclictest1750994-21modprobe00:34:592
173418099492,10cyclictest1750994-21modprobe00:34:592
173417599497,6cyclictest131rcu_preempt21:25:011
1734175994934,11cyclictest1-21systemd00:10:001
1734180994811,32cyclictest1738287-21unin-run20:30:012
173417099484,4cyclictest131rcu_preempt21:25:020
1734170994815,3cyclictest0-21swapper/000:20:030
1734180994730,13cyclictest272-21dbus-daemon19:55:022
1734180994730,13cyclictest272-21dbus-daemon19:55:012
1734175994735,9cyclictest1751280-21systemctl00:40:001
1734175994734,9cyclictest235-21systemd-journal20:50:021
1734175994734,9cyclictest235-21systemd-journal20:50:011
173417599471,3cyclictest131rcu_preempt20:35:021
173417599471,3cyclictest131rcu_preempt20:35:011
173417099473,4cyclictest131rcu_preempt20:10:000
173417099473,4cyclictest131rcu_preempt20:10:000
173417099472,41cyclictest1741626-21sshd21:32:280
1734185994637,5cyclictest1736977-21munin-plugin-st20:05:013
1734185994628,14cyclictest235-21systemd-journal20:45:023
1734185994628,14cyclictest235-21systemd-journal20:45:023
173418099461,6cyclictest131rcu_preempt00:40:012
173417599462,4cyclictest131rcu_preempt23:40:011
173417599461,4cyclictest131rcu_preempt22:40:001
173417099464,11cyclictest1-21systemd23:20:010
173417099463,3cyclictest131rcu_preempt21:55:030
1734185994530,11cyclictest37-21ksoftirqd/322:04:593
1734185994514,22cyclictest37-21ksoftirqd/321:20:023
1734180994532,8cyclictest1-21systemd00:05:012
1734180994529,13cyclictest1739799-21cron21:00:012
1734180994516,11cyclictest272-21dbus-daemon23:40:012
1734175994534,8cyclictest1-21systemd23:45:001
173417599453,3cyclictest131rcu_preempt23:15:021
173417599453,3cyclictest131rcu_preempt00:20:001
173417599453,11cyclictest23-21ksoftirqd/120:28:391
1734175994531,10cyclictest272-21dbus-daemon19:30:001
1734175994531,10cyclictest1742955-21readlink22:00:011
1734175994531,10cyclictest1742955-21readlink22:00:001
1734175994524,5cyclictest23-21ksoftirqd/123:45:401
173417599451,3cyclictest131rcu_preempt23:20:021
1734170994538,4cyclictest16950irq/224-800000023:45:000
1734170994528,12cyclictest272-21dbus-daemon19:55:020
1734170994528,12cyclictest272-21dbus-daemon19:55:010
173417099452,3cyclictest131rcu_preempt23:59:590
1734185994429,11cyclictest1737751-21run-parts20:20:003
173418599442,10cyclictest37-21ksoftirqd/322:06:403
173418099445,5cyclictest131rcu_preempt22:35:012
173417599446,3cyclictest131rcu_preempt21:30:001
173417599445,3cyclictest131rcu_preempt00:25:011
173417599444,3cyclictest131rcu_preempt21:05:031
173417599444,3cyclictest131rcu_preempt21:05:021
1734175994434,6cyclictest306-21in:imuxsock22:20:001
1734175994431,9cyclictest235-21systemd-journal23:35:011
1734175994410,5cyclictest131rcu_preempt20:55:011
173417099444,9cyclictest1744945-21latency_hist22:35:010
173417099444,36cyclictest0-21swapper/023:30:000
1734170994437,4cyclictest16850irq/134-800000023:40:010
173418599434,4cyclictest131rcu_preempt00:39:593
173418599434,3cyclictest131rcu_preempt00:05:013
1734185994327,13cyclictest272-21dbus-daemon23:55:023
1734185994326,12cyclictest235-21systemd-journal19:45:013
173418599431,3cyclictest131rcu_preempt22:40:023
1734180994331,6cyclictest1736685-21cron19:59:592
1734180994328,11cyclictest1747898-21pluginstate23:35:022
1734180994322,4cyclictest30-21ksoftirqd/200:10:022
173418099431,3cyclictest131rcu_preempt00:00:012
173417099436,3cyclictest131rcu_preempt19:20:010
173417099433,3cyclictest131rcu_preempt00:35:450
1734170994332,7cyclictest1740333-21cron21:09:590
173417099432,37cyclictest0-21swapper/023:25:010
173417099432,37cyclictest0-21swapper/023:25:010
1734170994321,5cyclictest12-21ksoftirqd/022:40:010
173417099431,3cyclictest131rcu_preempt21:20:020
173417099431,38cyclictest0-21swapper/020:35:010
173417099431,38cyclictest0-21swapper/020:35:000
1734185994228,11cyclictest1-21systemd22:45:003
173418599422,4cyclictest131rcu_preempt23:20:023
173418599421,3cyclictest131rcu_preempt23:25:013
173418599421,3cyclictest131rcu_preempt23:25:013
173418099424,3cyclictest131rcu_preempt20:45:002
1734180994231,9cyclictest30-21ksoftirqd/222:15:022
173418099422,3cyclictest131rcu_preempt22:00:032
173418099422,3cyclictest131rcu_preempt22:00:022
1734180994215,9cyclictest30-21ksoftirqd/223:25:022
1734180994215,9cyclictest30-21ksoftirqd/223:25:012
173418099421,4cyclictest131rcu_preempt21:05:012
173418099421,4cyclictest131rcu_preempt21:05:012
173417599427,5cyclictest131rcu_preempt20:10:021
173417599427,5cyclictest131rcu_preempt20:10:011
173417599424,3cyclictest131rcu_preempt22:15:011
173417599423,3cyclictest131rcu_preempt19:55:011
173417599423,3cyclictest131rcu_preempt19:55:011
173417599422,3cyclictest131rcu_preempt20:00:001
173417599421,3cyclictest131rcu_preempt23:05:011
173417099426,3cyclictest131rcu_preempt22:05:010
173417099424,4cyclictest131rcu_preempt20:48:400
173417099424,4cyclictest131rcu_preempt20:48:400
1734170994235,4cyclictest16950irq/224-800000000:10:010
1734170994225,14cyclictest1735641-21sshd19:37:400
1734170994223,15cyclictest1746561-21sshd23:07:400
1734170994221,5cyclictest12-21ksoftirqd/022:10:010
1734170994220,6cyclictest16850irq/134-800000019:30:400
173417099421,7cyclictest1-21systemd21:50:010
173417099421,12cyclictest1745536-21sshd22:49:400
1734170994210,5cyclictest131rcu_preempt23:15:010
173418599414,3cyclictest131rcu_preempt21:50:003
1734185994129,9cyclictest1740001-21sed21:00:023
1734185994128,9cyclictest272-21dbus-daemon00:25:023
1734185994128,10cyclictest1-21systemd22:30:003
1734185994124,13cyclictest1-21systemd23:50:003
173418599411,3cyclictest131rcu_preempt19:50:003
1734185994111,8cyclictest131rcu_preempt19:30:033
173418099414,4cyclictest131rcu_preempt20:40:012
173418099412,3cyclictest131rcu_preempt21:45:012
173417599413,4cyclictest131rcu_preempt21:50:031
1734175994125,12cyclictest235-21systemd-journal22:29:401
1734175994125,10cyclictest0-21swapper/123:10:001
173417599412,4cyclictest131rcu_preempt19:21:411
173417599412,4cyclictest131rcu_preempt19:21:401
1734175994121,5cyclictest23-21ksoftirqd/119:45:011
173417099414,4cyclictest131rcu_preempt20:30:010
173417099413,3cyclictest131rcu_preempt21:05:010
173417099413,3cyclictest131rcu_preempt21:05:000
1734170994129,10cyclictest12-21ksoftirqd/019:14:590
173417099412,4cyclictest131rcu_preempt22:00:010
173417099412,4cyclictest131rcu_preempt22:00:010
1734170994123,15cyclictest363-21snmpd22:51:530
173417099411,11cyclictest1738918-21latency_hist20:40:010
1734185994030,7cyclictest1745586-21cat22:50:013
1734185994030,6cyclictest1750206-21cron00:20:003
1734185994029,7cyclictest1740348-21sh21:09:593
173418599402,7cyclictest37-21ksoftirqd/323:00:013
1734185994027,9cyclictest272-21dbus-daemon20:55:013
1734185994027,10cyclictest235-21systemd-journal20:30:003
1734185994024,12cyclictest235-21systemd-journal00:10:023
1734185994023,13cyclictest235-21systemd-journal23:15:013
1734185994012,20cyclictest1-21systemd20:35:013
1734185994012,20cyclictest1-21systemd20:35:003
173418099404,3cyclictest131rcu_preempt21:30:002
173418099404,31cyclictest1738540-21unin-run20:35:012
173418099404,31cyclictest1738540-21unin-run20:35:012
1734180994027,9cyclictest272-21dbus-daemon00:20:022
1734180994026,11cyclictest1-21systemd23:30:012
1734180994024,12cyclictest1-21systemd21:10:012
173418099401,3cyclictest131rcu_preempt21:55:022
173417599403,3cyclictest131rcu_preempt00:35:011
173417599403,3cyclictest131rcu_preempt00:35:001
1734175994026,10cyclictest1734616-21cron19:20:001
173417599402,4cyclictest131rcu_preempt22:35:021
1734175994024,12cyclictest1-21systemd20:40:001
173417599402,3cyclictest131rcu_preempt19:10:031
173417599402,3cyclictest131rcu_preempt19:10:021
173417599401,4cyclictest131rcu_preempt22:05:001
173417099405,3cyclictest131rcu_preempt19:10:030
173417099405,3cyclictest131rcu_preempt19:10:020
1734170994031,6cyclictest1734929-21latency_hist19:25:010
1734170994031,6cyclictest1734929-21latency_hist19:25:010
1734170994028,8cyclictest235-21systemd-journal23:00:000
1734170994026,10cyclictest272-21dbus-daemon20:05:010
173417099402,5cyclictest131rcu_preempt21:40:000
173417099402,3cyclictest131rcu_preempt00:25:020
173417099401,8cyclictest12-21ksoftirqd/000:35:020
173417099401,8cyclictest12-21ksoftirqd/000:35:010
1734170994017,4cyclictest12-21ksoftirqd/021:30:010
173417099401,2cyclictest0-21swapper/019:55:400
173418599396,8cyclictest37-21ksoftirqd/300:12:403
173418599396,12cyclictest37-21ksoftirqd/319:25:013
173418599396,12cyclictest37-21ksoftirqd/319:25:013
173418599391,6cyclictest1748024-21wc23:35:013
173418099398,3cyclictest131rcu_preempt21:50:012
1734180993927,9cyclictest30-21ksoftirqd/219:15:012
1734180993925,10cyclictest1741913-21run-parts21:39:592
173418099391,4cyclictest131rcu_preempt23:45:022
173418099391,3cyclictest131rcu_preempt19:20:002
173417599394,3cyclictest131rcu_preempt20:25:011
173417599393,3cyclictest131rcu_preempt21:30:401
1734175993931,5cyclictest1734340-21sshd19:12:001
1734175993929,6cyclictest1747620-21cron23:29:591
1734175993924,11cyclictest1741976-21cat21:40:021
173417099396,6cyclictest131rcu_preempt00:00:050
173417099395,4cyclictest131rcu_preempt00:15:000
173417099393,4cyclictest131rcu_preempt19:50:010
1734170993928,9cyclictest12-21ksoftirqd/020:20:000
173418599385,3cyclictest131rcu_preempt23:05:023
173418599384,3cyclictest131rcu_preempt21:55:013
173418599383,4cyclictest131rcu_preempt19:59:593
173418599383,3cyclictest131rcu_preempt22:20:003
1734185993825,9cyclictest0-21swapper/323:30:013
173418599382,3cyclictest131rcu_preempt19:10:033
173418599382,3cyclictest131rcu_preempt19:10:023
1734185993823,10cyclictest306-21in:imuxsock23:10:013
173418599381,4cyclictest1748147-21systemctl23:40:013
173418099385,4cyclictest131rcu_preempt23:10:032
173418099385,3cyclictest131rcu_preempt21:15:002
1734180993827,8cyclictest1741649-21cron21:35:012
1734180993825,9cyclictest1-21systemd22:20:012
173418099382,4cyclictest131rcu_preempt22:40:002
173418099382,4cyclictest131rcu_preempt20:20:002
173418099382,3cyclictest131rcu_preempt20:10:032
173418099382,3cyclictest131rcu_preempt20:10:022
1734180993822,11cyclictest1744232-21sshd22:20:402
173418099381,3cyclictest131rcu_preempt23:20:002
173418099381,3cyclictest131rcu_preempt22:45:022
173417599387,4cyclictest131rcu_preempt23:25:021
173417599387,4cyclictest131rcu_preempt23:25:011
173417599386,4cyclictest131rcu_preempt19:35:001
173417599384,3cyclictest131rcu_preempt20:15:011
173417599383,4cyclictest131rcu_preempt21:00:001
1734175993823,11cyclictest23-21ksoftirqd/122:55:001
1734170993831,4cyclictest19050irq/34-mmc119:25:410
1734170993830,4cyclictest16950irq/224-800000000:30:020
1734170993829,6cyclictest12-21ksoftirqd/023:55:020
1734170993827,9cyclictest12-21ksoftirqd/022:25:010
1734170993824,9cyclictest1739027-21sshd20:41:410
173417099382,3cyclictest131rcu_preempt21:45:000
173417099382,32cyclictest1735920-21sshd19:42:400
173417099382,31cyclictest1743718-21modprobe22:15:010
1734170993820,10cyclictest306-21in:imuxsock20:55:010
173417099381,8cyclictest235-21systemd-journal23:01:530
173418599378,17cyclictest1-21systemd19:55:023
173418599378,17cyclictest1-21systemd19:55:013
173418599374,3cyclictest131rcu_preempt00:30:013
173418599373,3cyclictest131rcu_preempt00:00:003
1734185993726,8cyclictest1741294-21cat21:25:013
173418599372,14cyclictest0-21swapper/319:15:013
1734185993721,12cyclictest37-21ksoftirqd/321:58:403
1734185993721,12cyclictest37-21ksoftirqd/321:58:393
173418099373,4cyclictest131rcu_preempt22:03:412
173418099373,3cyclictest131rcu_preempt21:25:002
173418099373,3cyclictest131rcu_preempt19:10:022
173418099373,3cyclictest131rcu_preempt19:10:022
1734180993725,9cyclictest30-21ksoftirqd/223:55:012
1734180993725,8cyclictest1740859-21cron21:20:012
1734180993725,8cyclictest1735260-21cat19:30:022
173418099372,4cyclictest131rcu_preempt20:50:022
173418099372,4cyclictest131rcu_preempt00:30:012
173418099371,3cyclictest131rcu_preempt23:00:022
173418099371,3cyclictest131rcu_preempt19:35:012
173417599376,3cyclictest131rcu_preempt21:20:001
173417599375,3cyclictest131rcu_preempt21:07:401
1734175993727,7cyclictest23-21ksoftirqd/120:20:031
1734175993727,7cyclictest1742159-21sshd21:42:301
173417599372,3cyclictest131rcu_preempt22:50:011
173417599371,3cyclictest131rcu_preempt21:55:011
173417599371,3cyclictest131rcu_preempt20:45:001
173417099376,3cyclictest131rcu_preempt22:30:010
173417099374,3cyclictest131rcu_preempt23:50:010
173417099373,30cyclictest1740674-21ls21:15:020
173417099372,6cyclictest1745315-21munin-run22:45:000
1734170993717,13cyclictest272-21dbus-daemon23:35:000
1734170993714,6cyclictest16850irq/134-800000020:22:410
173418599362,5cyclictest1741768-21latency_hist21:35:023
1734185993625,7cyclictest37-21ksoftirqd/322:35:013
1734185993624,10cyclictest37-21ksoftirqd/321:05:013
1734185993624,10cyclictest37-21ksoftirqd/321:05:003
173418599361,3cyclictest131rcu_preempt00:35:023
173418599361,3cyclictest131rcu_preempt00:35:013
173418099365,4cyclictest131rcu_preempt23:15:012
173418099364,3cyclictest131rcu_preempt20:22:142
1734180993625,6cyclictest306-21in:imuxsock19:50:002
1734180993624,8cyclictest306-21in:imuxsock20:01:402
173418099362,3cyclictest131rcu_preempt22:30:022
1734180993619,13cyclictest30-21ksoftirqd/223:02:452
173418099361,4cyclictest131rcu_preempt22:50:002
173418099361,4cyclictest131rcu_preempt00:15:012
173418099361,3cyclictest131rcu_preempt19:40:022
173417599363,4cyclictest131rcu_preempt00:05:011
173417599363,3cyclictest131rcu_preempt22:05:031
173417599362,4cyclictest131rcu_preempt00:10:031
173417599362,3cyclictest131rcu_preempt22:55:021
173417599362,3cyclictest131rcu_preempt21:15:001
173417599361,3cyclictest131rcu_preempt23:55:001
173418599353,3cyclictest131rcu_preempt21:40:033
173418599352,4cyclictest131rcu_preempt23:45:003
1734185993524,9cyclictest37-21ksoftirqd/320:15:013
1734185993524,8cyclictest1734601-21sshd19:17:013
1734185993523,9cyclictest37-21ksoftirqd/321:41:403
1734185993520,11cyclictest272-21dbus-daemon20:25:023
1734185993520,10cyclictest37-21ksoftirqd/321:15:013
173418599351,4cyclictest131rcu_preempt21:30:013
173418599351,4cyclictest131rcu_preempt19:40:003
173418099355,12cyclictest0-21swapper/219:40:002
173418099354,4cyclictest131rcu_preempt20:15:022
1734180993526,7cyclictest30-21ksoftirqd/200:25:002
1734180993522,10cyclictest1-21systemd20:50:022
1734180993522,10cyclictest1-21systemd20:50:012
173417599352,5cyclictest131rcu_preempt22:20:411
173418599343,6cyclictest131rcu_preempt22:25:023
1734185993422,10cyclictest37-21ksoftirqd/322:55:023
1734185993419,10cyclictest272-21dbus-daemon19:35:003
173418099342,4cyclictest131rcu_preempt22:55:002
173417599343,3cyclictest131rcu_preempt22:44:391
173418599333,7cyclictest1738782-21modprobe20:39:593
173418599331,4cyclictest131rcu_preempt22:15:033
1734185993313,8cyclictest37-21ksoftirqd/320:10:003
1734185993313,8cyclictest37-21ksoftirqd/320:10:003
173418099333,3cyclictest131rcu_preempt19:25:022
173418099333,3cyclictest131rcu_preempt19:25:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional