You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-17 - 11:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot3s.osadl.org (updated Wed Sep 17, 2025 00:45:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3966469991411,3cyclictest131rcu_preempt00:40:031
3966469991411,3cyclictest131rcu_preempt00:40:021
3966465998045,4cyclictest16950irq/224-800000000:05:010
3966465996234,19cyclictest12-21ksoftirqd/022:48:220
3966469996025,12cyclictest3983223-21sed00:30:011
3966475995935,17cyclictest235-21systemd-journal19:30:013
3966475995840,14cyclictest1-21systemd19:20:013
3966475995838,14cyclictest235-21systemd-journal21:50:003
3966475995526,20cyclictest272-21dbus-daemon23:25:013
396647599544,3cyclictest131rcu_preempt20:50:003
3966475995436,12cyclictest1-21systemd23:10:003
3966475995428,18cyclictest235-21systemd-journal23:45:013
3966475995333,16cyclictest235-21systemd-journal22:05:003
3966475995332,16cyclictest1-21systemd20:15:003
3966465995336,12cyclictest235-21systemd-journal00:40:010
3966465995336,12cyclictest235-21systemd-journal00:40:010
3966475995236,12cyclictest235-21systemd-journal22:40:033
3966475995234,14cyclictest272-21dbus-daemon20:20:013
3966475995229,18cyclictest1-21systemd00:15:013
396646599522,4cyclictest131rcu_preempt23:50:020
396646599521,3cyclictest131rcu_preempt00:00:020
3966475995131,15cyclictest272-21dbus-daemon00:40:013
3966475995131,15cyclictest272-21dbus-daemon00:40:003
396647199514,4cyclictest131rcu_preempt00:00:002
3966475995044,3cyclictest371ksoftirqd/319:25:013
396647599504,31cyclictest3973166-21munin-plugin-st21:20:013
3966471995035,11cyclictest235-21systemd-journal00:20:012
396647199502,4cyclictest131rcu_preempt21:45:012
3966469995033,13cyclictest1-21systemd21:20:011
3966475994931,9cyclictest306-21in:imuxsock22:30:013
3966469994934,10cyclictest1-21systemd20:25:001
396646599499,35cyclictest3978659-21cron23:04:590
396646599493,4cyclictest131rcu_preempt21:59:590
396646599492,42cyclictest3980484-21systemctl23:40:000
396646599491,4cyclictest131rcu_preempt21:40:010
396646599491,4cyclictest131rcu_preempt21:40:000
3966475994832,9cyclictest37-21ksoftirqd/300:00:023
3966475994832,10cyclictest3974200-21munin-run21:40:013
3966475994832,10cyclictest3974200-21munin-run21:40:003
3966475994819,18cyclictest272-21dbus-daemon00:30:013
3966471994836,8cyclictest272-21dbus-daemon23:35:012
3966465994833,11cyclictest235-21systemd-journal19:33:260
3966465994828,4cyclictest0-21swapper/023:08:420
396646599482,4cyclictest131rcu_preempt00:20:000
3966475994727,13cyclictest3980212-21modprobe23:35:003
3966475994722,17cyclictest235-21systemd-journal20:40:003
3966471994720,6cyclictest30-21ksoftirqd/200:10:022
396646599474,3cyclictest131rcu_preempt22:25:010
3966465994740,4cyclictest16950irq/224-800000021:05:010
396646599473,3cyclictest131rcu_preempt21:15:010
396646599473,3cyclictest131rcu_preempt20:10:000
396646599473,3cyclictest131rcu_preempt20:10:000
396646599473,3cyclictest131rcu_preempt00:25:000
396646599472,4cyclictest131rcu_preempt21:50:020
3966475994627,11cyclictest3972115-21munin-plugin-st21:00:003
3966475994620,17cyclictest272-21dbus-daemon21:25:003
396647199467,4cyclictest131rcu_preempt22:55:002
396647199464,3cyclictest131rcu_preempt19:30:002
396647199464,3cyclictest131rcu_preempt19:16:252
396647199461,4cyclictest131rcu_preempt23:20:012
396646599463,4cyclictest131rcu_preempt22:40:000
3966465994634,9cyclictest1-21systemd23:20:010
396646599463,3cyclictest131rcu_preempt20:55:010
396646599462,4cyclictest131rcu_preempt20:20:010
396646599462,40cyclictest3976803-21munin-run22:30:000
396646599462,3cyclictest131rcu_preempt21:30:000
3966465994611,6cyclictest131rcu_preempt00:10:020
396647599457,25cyclictest3975217-21modprobe21:59:593
3966475994525,14cyclictest37-21ksoftirqd/323:20:003
3966475994523,16cyclictest272-21dbus-daemon22:35:013
3966475994520,7cyclictest37-21ksoftirqd/321:15:023
396647199454,4cyclictest131rcu_preempt20:58:272
396647199454,3cyclictest131rcu_preempt20:55:002
396647199453,4cyclictest131rcu_preempt20:15:002
396647199452,5cyclictest131rcu_preempt20:20:002
396647199452,4cyclictest131rcu_preempt23:06:492
396646999454,3cyclictest131rcu_preempt20:05:011
3966469994516,6cyclictest131rcu_preempt00:05:001
396646999451,4cyclictest131rcu_preempt19:10:011
396646599453,3cyclictest131rcu_preempt20:00:020
3966465994514,21cyclictest16850irq/134-800000019:40:000
396647599447,27cyclictest3976275-21cron22:20:013
396647199449,6cyclictest131rcu_preempt22:10:032
396647199444,3cyclictest131rcu_preempt23:22:282
396647199441,3cyclictest131rcu_preempt23:05:022
396646999445,4cyclictest131rcu_preempt20:55:001
3966469994432,9cyclictest3980972-21sshd23:46:271
3966469994427,13cyclictest3970130-21sed20:20:011
3966469994426,13cyclictest23-21ksoftirqd/119:15:011
396646999442,37cyclictest272-21dbus-daemon22:40:001
396646599444,3cyclictest131rcu_preempt21:20:030
3966465994430,10cyclictest235-21systemd-journal20:59:260
396646599441,4cyclictest131rcu_preempt20:45:020
396647599434,30cyclictest3979176-21modprobe23:15:013
396647599434,30cyclictest3979176-21modprobe23:15:003
3966475994324,14cyclictest37-21ksoftirqd/300:10:003
3966475994318,5cyclictest37-21ksoftirqd/323:50:023
3966475994317,21cyclictest37-21ksoftirqd/321:05:003
3966475994313,15cyclictest3974969-21modprobe21:55:013
396647199435,3cyclictest131rcu_preempt22:20:012
396647199434,4cyclictest131rcu_preempt22:00:012
396647199434,4cyclictest131rcu_preempt20:44:262
396647199434,4cyclictest131rcu_preempt20:25:022
396647199434,3cyclictest131rcu_preempt23:50:012
396647199433,3cyclictest131rcu_preempt00:35:032
396647199433,3cyclictest131rcu_preempt00:35:022
396647199433,3cyclictest131rcu_preempt00:30:012
396647199431,4cyclictest131rcu_preempt21:55:012
396647199431,3cyclictest131rcu_preempt22:30:272
3966469994329,10cyclictest1-21systemd00:20:001
396646599433,3cyclictest131rcu_preempt21:45:000
3966465994329,9cyclictest235-21systemd-journal20:05:010
396646599432,3cyclictest131rcu_preempt23:30:010
396646599431,3cyclictest131rcu_preempt22:20:030
396646599431,3cyclictest131rcu_preempt21:55:030
3966475994215,18cyclictest272-21dbus-daemon20:25:003
3966475994214,18cyclictest272-21dbus-daemon19:15:003
396647199425,3cyclictest131rcu_preempt19:40:012
396647199424,4cyclictest131rcu_preempt21:30:002
3966471994229,9cyclictest306-21in:imuxsock23:00:002
3966471994224,13cyclictest272-21dbus-daemon23:40:012
3966471994224,13cyclictest272-21dbus-daemon20:30:002
3966471994221,15cyclictest272-21dbus-daemon21:15:002
396646999423,3cyclictest131rcu_preempt20:00:001
396646599424,3cyclictest131rcu_preempt21:35:010
3966465994236,3cyclictest16950irq/224-800000020:10:260
3966465994235,4cyclictest16950irq/224-800000023:00:030
396646599423,3cyclictest131rcu_preempt20:50:000
3966465994227,11cyclictest3970291-21sshd20:23:260
3966465994227,11cyclictest272-21dbus-daemon19:55:010
396646599422,10cyclictest235-21systemd-journal00:32:260
396646599421,4cyclictest131rcu_preempt00:15:010
396646599421,3cyclictest131rcu_preempt19:10:020
396647599414,27cyclictest3978378-21cron23:00:013
396647599414,20cyclictest3967691-21modprobe19:35:003
396647599413,20cyclictest3970565-21cut20:30:003
3966475994129,9cyclictest37-21ksoftirqd/319:55:023
3966475994122,10cyclictest306-21in:imuxsock21:35:013
3966475994115,18cyclictest272-21dbus-daemon19:50:013
3966475994110,20cyclictest3971342-21modprobe20:45:003
396647199414,4cyclictest131rcu_preempt21:40:002
396647199414,4cyclictest131rcu_preempt21:40:002
396647199413,3cyclictest131rcu_preempt22:14:262
3966471994131,5cyclictest306-21in:imuxsock00:25:002
396646999413,3cyclictest131rcu_preempt23:20:021
3966469994127,11cyclictest235-21systemd-journal21:39:281
3966469994127,11cyclictest235-21systemd-journal21:39:271
3966469994127,11cyclictest1-21systemd00:35:001
396646599414,6cyclictest131rcu_preempt19:20:010
396646599413,7cyclictest3979715-21munin-plugin-st23:25:010
396646599413,3cyclictest131rcu_preempt22:55:000
3966465994126,12cyclictest3973644-21sed21:25:020
396647599406,20cyclictest3978675-21grep23:04:593
396647599403,3cyclictest131rcu_preempt22:40:023
396647599403,27cyclictest3971853-21cut20:55:013
3966475994018,14cyclictest0-21swapper/320:05:013
3966471994028,9cyclictest272-21dbus-daemon20:00:002
3966471994025,11cyclictest235-21systemd-journal21:05:002
396646999404,3cyclictest131rcu_preempt21:25:011
3966469994026,10cyclictest23-21ksoftirqd/120:15:021
3966469994022,15cyclictest272-21dbus-daemon19:20:011
3966469994022,13cyclictest1-21systemd00:10:021
396646999401,5cyclictest0-21swapper/119:50:011
396646599407,5cyclictest131rcu_preempt00:30:030
396646599401,3cyclictest131rcu_preempt23:55:000
396646599401,3cyclictest131rcu_preempt23:45:010
396646599401,3cyclictest131rcu_preempt23:35:020
396646599401,3cyclictest131rcu_preempt19:25:000
396646599401,35cyclictest0-21swapper/022:45:010
396647599396,21cyclictest3976547-21sh22:25:003
3966475993928,8cyclictest3979953-21sshd23:29:273
3966475993928,8cyclictest37-21ksoftirqd/300:02:023
3966475993927,9cyclictest3974439-21sshd21:41:263
3966475993927,9cyclictest37-21ksoftirqd/322:55:013
3966475993927,8cyclictest3973689-21/usr/sbin/munin21:30:013
3966475993925,11cyclictest37-21ksoftirqd/319:40:003
3966475993925,10cyclictest37-21ksoftirqd/320:00:003
3966475993923,10cyclictest3982548-21cron00:20:003
3966475993922,13cyclictest1-21systemd20:35:023
3966475993922,13cyclictest1-21systemd20:35:013
396647199396,3cyclictest131rcu_preempt19:46:092
396647199393,3cyclictest131rcu_preempt22:25:262
396647199393,3cyclictest131rcu_preempt19:25:012
3966471993928,8cyclictest3981983-21sed00:05:032
3966471993926,9cyclictest235-21systemd-journal20:10:012
3966471993926,9cyclictest235-21systemd-journal20:10:002
3966471993924,12cyclictest30-21ksoftirqd/220:04:272
396647199392,3cyclictest131rcu_preempt20:35:042
396647199392,3cyclictest131rcu_preempt20:35:042
3966471993923,11cyclictest1-21systemd19:55:012
396646999395,3cyclictest131rcu_preempt19:45:011
396646999393,7cyclictest23-21ksoftirqd/123:40:021
3966469993929,5cyclictest3976260-21cron22:20:001
3966469993919,13cyclictest235-21systemd-journal20:50:011
396646999391,3cyclictest131rcu_preempt00:00:031
396646599392,6cyclictest0-21swapper/020:40:020
396646599391,5cyclictest131rcu_preempt22:10:030
396646599391,3cyclictest131rcu_preempt20:35:050
396646599391,3cyclictest131rcu_preempt20:35:040
396647599385,19cyclictest3981259-21munin-plugin-st23:55:003
3966475993823,12cyclictest37-21ksoftirqd/319:45:033
396647199384,3cyclictest131rcu_preempt23:45:002
3966471993823,9cyclictest3974725-21cut21:50:002
3966471993822,12cyclictest3977293-21cron22:40:012
3966471993822,12cyclictest272-21dbus-daemon20:40:002
3966471993819,13cyclictest272-21dbus-daemon23:55:012
396646999384,4cyclictest131rcu_preempt23:30:011
396646999382,6cyclictest131rcu_preempt21:15:011
3966469993826,8cyclictest235-21systemd-journal21:53:271
3966469993825,10cyclictest3967180-21cron19:25:001
3966469993824,11cyclictest23-21ksoftirqd/120:37:101
396646999381,3cyclictest131rcu_preempt23:55:021
396646999381,3cyclictest131rcu_preempt21:09:591
396646599382,8cyclictest3970733-21cat20:30:020
396646599382,4cyclictest131rcu_preempt22:14:260
3966465993824,9cyclictest235-21systemd-journal19:15:000
396646599382,3cyclictest131rcu_preempt19:29:260
3966465993814,20cyclictest16850irq/134-800000023:13:280
3966465993814,20cyclictest16850irq/134-800000023:13:270
396646599381,3cyclictest131rcu_preempt22:05:020
396646599381,33cyclictest3972609-21modprobe21:07:170
396647599376,4cyclictest3969676-21cat20:10:033
396647599376,4cyclictest3969676-21cat20:10:023
396647599376,10cyclictest37-21ksoftirqd/322:12:373
3966475993726,8cyclictest37-21ksoftirqd/323:40:013
3966471993729,5cyclictest306-21in:imuxsock23:30:012
3966471993723,11cyclictest30-21ksoftirqd/223:15:022
3966471993723,11cyclictest30-21ksoftirqd/223:15:012
396647199371,3cyclictest131rcu_preempt19:35:022
396646999374,3cyclictest131rcu_preempt23:05:001
396646999373,3cyclictest131rcu_preempt21:50:021
3966469993728,6cyclictest3977026-21sshd22:32:271
3966469993727,6cyclictest23-21ksoftirqd/123:00:021
3966469993726,9cyclictest23-21ksoftirqd/123:45:021
3966469993726,8cyclictest23-21ksoftirqd/122:55:031
396646999372,4cyclictest131rcu_preempt19:55:021
3966469993724,10cyclictest23-21ksoftirqd/123:35:001
3966469993723,11cyclictest3973908-21sshd21:31:271
3966469993721,12cyclictest272-21dbus-daemon22:05:011
3966469993721,12cyclictest272-21dbus-daemon21:00:001
3966469993718,14cyclictest235-21systemd-journal00:25:001
396646999371,3cyclictest131rcu_preempt23:25:011
396646999371,3cyclictest131rcu_preempt20:35:031
396646999371,3cyclictest131rcu_preempt20:35:031
3966465993724,9cyclictest3968455-21sshd19:47:030
396646599371,4cyclictest131rcu_preempt22:35:020
396647599366,21cyclictest235-21systemd-journal21:09:593
3966475993625,9cyclictest37-21ksoftirqd/322:10:023
396647599361,5cyclictest131rcu_preempt00:24:593
3966471993627,7cyclictest30-21ksoftirqd/222:25:022
3966471993619,13cyclictest272-21dbus-daemon22:05:002
3966471993619,12cyclictest272-21dbus-daemon22:50:012
3966469993625,7cyclictest23-21ksoftirqd/121:05:031
3966469993624,8cyclictest3978892-21sshd23:06:281
3966465993623,10cyclictest3968256-21cat19:45:020
3966475993525,5cyclictest37-21ksoftirqd/322:50:003
3966475993521,11cyclictest37-21ksoftirqd/300:35:023
396647199353,3cyclictest131rcu_preempt20:50:002
396647199352,5cyclictest131rcu_preempt19:10:032
3966471993521,11cyclictest3972786-21sort21:10:022
396647199351,4cyclictest131rcu_preempt19:15:002
396646999354,3cyclictest131rcu_preempt20:10:001
396646999354,3cyclictest131rcu_preempt20:10:001
3966469993524,8cyclictest235-21systemd-journal00:14:591
3966469993522,9cyclictest3970756-21sed20:30:031
396646999351,3cyclictest131rcu_preempt20:45:011
396647199343,3cyclictest131rcu_preempt00:35:012
396647199342,4cyclictest131rcu_preempt21:35:002
3966471993424,7cyclictest3968217-21cron19:45:002
396647199342,3cyclictest131rcu_preempt22:45:012
396647199342,3cyclictest131rcu_preempt21:21:292
3966471993422,9cyclictest3982272-21sshd00:12:262
396646999345,7cyclictest23-21ksoftirqd/122:30:021
396646999343,3cyclictest131rcu_preempt22:00:011
3966469993425,5cyclictest412-21sshd22:07:271
3966469993425,5cyclictest3975998-21cron22:15:001
3966469993422,9cyclictest23-21ksoftirqd/119:27:021
3966469993421,10cyclictest23-21ksoftirqd/121:30:011
396646999341,4cyclictest131rcu_preempt21:45:021
396646999341,3cyclictest131rcu_preempt19:35:021
396646999341,14cyclictest235-21systemd-journal19:40:011
396647199331,3cyclictest131rcu_preempt21:20:012
396646999332,4cyclictest131rcu_preempt23:15:011
396646999332,4cyclictest131rcu_preempt23:15:011
396646999331,3cyclictest131rcu_preempt22:50:011
396646999331,3cyclictest131rcu_preempt22:45:011
3966475993224,5cyclictest3966551-21latency_hist19:10:023
396646999311,3cyclictest131rcu_preempt22:25:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional