You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-24 - 16:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Mon Nov 24, 2025 12:45:00)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4074512998064,9cyclictest4074513-21cat07:10:000
4074525997329,40cyclictest272-21dbus-daemon08:35:013
4074525997128,8cyclictest4080660-21sshd09:07:203
4074519997164,5cyclictest23-21ksoftirqd/109:45:031
4074519997164,5cyclictest23-21ksoftirqd/109:45:021
4074525996618,14cyclictest272-21dbus-daemon12:25:013
4074525996617,12cyclictest235-21systemd-journal10:25:013
407451999609,14cyclictest23-21ksoftirqd/108:45:021
407451999588,8cyclictest23-21ksoftirqd/111:50:011
4074519995815,10cyclictest23-21ksoftirqd/108:10:021
4074519995811,8cyclictest0-21swapper/109:20:011
4074525995622,10cyclictest272-21dbus-daemon09:15:013
4074525995621,10cyclictest37-21ksoftirqd/311:34:213
4074525995618,11cyclictest4086173-21sed10:55:023
4074525995618,11cyclictest4086173-21sed10:55:013
4074525995618,11cyclictest4084421-21cat10:20:023
4074525995617,11cyclictest272-21dbus-daemon08:05:013
407451999566,6cyclictest23-21ksoftirqd/107:25:011
4074519995613,11cyclictest23-21ksoftirqd/110:59:591
4074519995612,8cyclictest23-21ksoftirqd/112:00:011
4074525995538,12cyclictest1-21systemd10:50:013
4074525995523,8cyclictest37-21ksoftirqd/308:25:013
4074519995512,8cyclictest23-21ksoftirqd/108:55:021
407451999547,11cyclictest23-21ksoftirqd/112:05:011
4074525995335,14cyclictest1-21systemd12:00:013
407451999536,7cyclictest23-21ksoftirqd/112:35:011
4074519995311,9cyclictest23-21ksoftirqd/111:20:011
4074519995311,8cyclictest23-21ksoftirqd/111:30:021
407451999531,10cyclictest23-21ksoftirqd/111:10:031
407451999523,8cyclictest23-21ksoftirqd/111:10:011
4074512995210,30cyclictest19050irq/34-mmc107:30:020
407451999516,8cyclictest23-21ksoftirqd/112:30:011
407451999515,13cyclictest23-21ksoftirqd/109:40:011
4074525995029,15cyclictest235-21systemd-journal12:20:013
4074525995029,14cyclictest235-21systemd-journal08:20:013
407451999506,9cyclictest23-21ksoftirqd/110:10:001
4074519995038,9cyclictest23-21ksoftirqd/111:25:021
4074512995031,13cyclictest235-21systemd-journal11:30:010
4074512995013,33cyclictest12-21ksoftirqd/010:50:020
4074519994931,14cyclictest1-21systemd11:05:011
407451999492,7cyclictest4080165-21munin-run09:00:011
407451999491,9cyclictest235-21systemd-journal08:30:001
4074525994833,11cyclictest4091594-21systemctl12:40:003
4074525994829,15cyclictest272-21dbus-daemon08:45:003
4074521994831,13cyclictest272-21dbus-daemon10:50:012
407451999487,7cyclictest23-21ksoftirqd/107:55:021
407451999486,18cyclictest4075460-21kworker/u8:107:45:011
407451299482,40cyclictest4081556-21latency_hist09:25:020
407451999474,10cyclictest23-21ksoftirqd/110:00:021
407451999474,10cyclictest23-21ksoftirqd/109:05:021
4074519994733,10cyclictest235-21systemd-journal07:30:001
407451999473,13cyclictest23-21ksoftirqd/110:15:021
407451999471,7cyclictest0-21swapper/108:20:021
4074519994714,12cyclictest23-21ksoftirqd/110:35:001
4074519994712,7cyclictest0-21swapper/110:40:011
4074519994711,8cyclictest23-21ksoftirqd/111:55:011
4074521994629,14cyclictest272-21dbus-daemon07:35:002
407451999467,10cyclictest23-21ksoftirqd/109:55:031
407451999464,9cyclictest23-21ksoftirqd/111:40:011
407451999464,10cyclictest23-21ksoftirqd/112:14:591
4074519994637,6cyclictest23-21ksoftirqd/112:10:011
4074512994633,10cyclictest235-21systemd-journal08:47:210
407452199454,6cyclictest131rcu_preempt07:10:012
4074521994536,3cyclictest30-21ksoftirqd/211:05:002
407451999459,9cyclictest23-21ksoftirqd/109:30:021
407451999459,9cyclictest23-21ksoftirqd/109:30:011
407451999457,8cyclictest23-21ksoftirqd/107:35:021
407451999456,4cyclictest131rcu_preempt11:35:001
407451999455,9cyclictest23-21ksoftirqd/110:25:011
407451999455,9cyclictest23-21ksoftirqd/108:25:011
4074519994511,8cyclictest0-21swapper/108:01:211
407451999451,14cyclictest4074573-21latency_hist07:10:021
4074512994539,4cyclictest16950irq/224-800000008:00:000
4074512994535,7cyclictest19050irq/34-mmc107:45:010
4074512994530,11cyclictest235-21systemd-journal11:20:010
4074521994423,14cyclictest235-21systemd-journal07:24:592
407451999444,9cyclictest23-21ksoftirqd/108:00:021
407451999443,13cyclictest23-21ksoftirqd/107:20:031
4074519994429,12cyclictest272-21dbus-daemon09:25:011
4074519994428,11cyclictest272-21dbus-daemon10:30:021
407451999441,8cyclictest4078972-21sort08:35:011
407451299444,4cyclictest131rcu_preempt07:55:020
4074512994435,5cyclictest16850irq/134-800000011:39:590
407451299442,12cyclictest4083820-21true10:10:010
4074512994415,26cyclictest12-21ksoftirqd/010:39:590
4074525994332,6cyclictest306-21in:imuxsock09:50:003
4074525994331,9cyclictest4085091-21sshd10:34:213
4074521994333,6cyclictest306-21in:imuxsock08:40:022
4074521994329,11cyclictest235-21systemd-journal10:14:212
4074521994328,12cyclictest4076476-21sed07:45:032
407451999433,10cyclictest23-21ksoftirqd/110:50:001
407451999432,8cyclictest23-21ksoftirqd/108:40:001
4074512994330,9cyclictest12-21ksoftirqd/008:10:020
4074512994329,9cyclictest235-21systemd-journal10:30:000
4074512994328,11cyclictest1-21systemd10:35:010
407451299432,7cyclictest4082517-21modprobe09:45:010
407451299432,7cyclictest4082517-21modprobe09:45:010
4074512994326,10cyclictest306-21in:imuxsock11:20:020
4074512994321,5cyclictest12-21ksoftirqd/008:30:000
407451299431,4cyclictest12-21ksoftirqd/007:15:020
4074525994231,7cyclictest4074954-21cron07:20:003
407452199423,5cyclictest131rcu_preempt08:00:002
4074521994232,7cyclictest4090825-21sh12:25:002
4074521994232,6cyclictest271-21cron12:20:002
4074521994225,11cyclictest272-21dbus-daemon10:45:002
407452199422,3cyclictest131rcu_preempt11:40:022
4074521994222,15cyclictest272-21dbus-daemon10:05:002
4074521994221,14cyclictest272-21dbus-daemon12:15:012
4074521994219,19cyclictest30-21ksoftirqd/207:38:202
407451999423,11cyclictest23-21ksoftirqd/107:50:011
4074512994228,9cyclictest235-21systemd-journal08:40:000
4074512994228,10cyclictest272-21dbus-daemon12:40:010
4074512994227,12cyclictest12-21ksoftirqd/009:00:000
4074512994225,12cyclictest1-21systemd12:30:000
407451299421,37cyclictest363-21snmpd11:33:110
407452599415,6cyclictest131rcu_preempt11:10:013
407452599415,6cyclictest131rcu_preempt11:10:003
4074525994129,9cyclictest37-21ksoftirqd/308:55:003
407452599412,5cyclictest131rcu_preempt07:15:003
4074521994131,7cyclictest4082777-21munin-plugin-st09:50:002
4074521994130,8cyclictest4086163-21latency_hist10:55:022
4074521994130,8cyclictest4086163-21latency_hist10:55:012
4074521994130,8cyclictest272-21dbus-daemon09:30:012
4074521994130,8cyclictest272-21dbus-daemon09:30:002
4074521994128,10cyclictest235-21systemd-journal07:51:202
407452199412,5cyclictest131rcu_preempt09:34:592
407452199412,23cyclictest291rcuc/209:40:032
407451999416,7cyclictest23-21ksoftirqd/109:50:011
407451999416,7cyclictest23-21ksoftirqd/109:05:031
4074519994127,11cyclictest235-21systemd-journal12:22:191
407451999411,12cyclictest23-21ksoftirqd/108:50:021
4074512994120,15cyclictest235-21systemd-journal07:40:010
407451299411,3cyclictest131rcu_preempt10:55:010
407451299411,3cyclictest131rcu_preempt10:55:000
4074512994112,5cyclictest131rcu_preempt08:25:010
4074525994027,8cyclictest4086154-21kworker/3:011:00:003
4074525994026,10cyclictest0-21swapper/309:25:023
4074525994025,10cyclictest306-21in:imuxsock11:20:013
4074525994021,15cyclictest272-21dbus-daemon11:39:593
4074521994030,7cyclictest30-21ksoftirqd/210:19:592
4074521994022,13cyclictest4077559-21cron08:10:012
407452199401,6cyclictest131rcu_preempt10:10:012
407451999402,4cyclictest131rcu_preempt11:11:211
4074519994023,12cyclictest306-21in:imuxsock12:39:591
407451999401,7cyclictest0-21swapper/112:20:021
4074512994028,7cyclictest19050irq/34-mmc109:10:000
4074525993927,7cyclictest306-21in:imuxsock08:10:023
4074525993925,10cyclictest272-21dbus-daemon08:00:013
4074525993923,12cyclictest4080562-21sed09:05:033
4074525993920,13cyclictest272-21dbus-daemon07:50:003
4074521993926,9cyclictest272-21dbus-daemon12:00:012
4074521993925,10cyclictest4078074-21cron08:20:002
4074521993923,12cyclictest235-21systemd-journal10:00:022
4074521993923,11cyclictest272-21dbus-daemon07:15:002
407452199391,5cyclictest131rcu_preempt11:00:012
407451999396,7cyclictest23-21ksoftirqd/109:15:001
407451999396,6cyclictest23-21ksoftirqd/107:10:161
407451999393,8cyclictest23-21ksoftirqd/110:01:201
407451999393,7cyclictest23-21ksoftirqd/111:45:001
407451299393,7cyclictest4085655-21unin-run10:45:010
4074512993930,6cyclictest19050irq/34-mmc112:12:400
4074512993930,5cyclictest306-21in:imuxsock11:00:010
4074512993929,7cyclictest12-21ksoftirqd/010:25:020
4074512993928,8cyclictest235-21systemd-journal11:11:260
4074512993928,8cyclictest235-21systemd-journal09:39:590
4074512993928,6cyclictest4089494-21cron12:00:000
407451299392,33cyclictest363-21snmpd07:49:220
4074512993922,13cyclictest4086655-21sshd11:04:190
4074512993914,6cyclictest12-21ksoftirqd/009:25:030
4074512993914,6cyclictest12-21ksoftirqd/009:25:020
407451299391,34cyclictest0-21swapper/010:04:590
407451299391,10cyclictest4080015-21date08:55:010
4074525993825,9cyclictest235-21systemd-journal10:00:003
407452599381,3cyclictest131rcu_preempt08:15:013
407452199384,24cyclictest30-21ksoftirqd/208:35:012
4074521993830,5cyclictest4084846-21cron10:30:002
4074521993826,9cyclictest30-21ksoftirqd/212:30:022
4074521993826,9cyclictest235-21systemd-journal11:54:592
407452199382,5cyclictest131rcu_preempt09:20:012
4074521993821,12cyclictest235-21systemd-journal11:10:012
4074521993821,12cyclictest235-21systemd-journal11:10:012
407451999386,6cyclictest23-21ksoftirqd/110:20:021
4074519993827,8cyclictest235-21systemd-journal09:32:191
4074519993822,12cyclictest23-21ksoftirqd/110:55:011
4074519993822,12cyclictest23-21ksoftirqd/110:55:011
4074512993831,4cyclictest16950irq/224-800000009:05:020
4074512993830,5cyclictest0-21swapper/011:44:210
4074512993827,7cyclictest235-21systemd-journal08:17:010
4074512993823,12cyclictest12-21ksoftirqd/012:25:000
4074512993818,16cyclictest1-21systemd11:10:010
4074512993818,16cyclictest1-21systemd11:10:000
407451299381,32cyclictest0-21swapper/009:50:000
4074525993728,5cyclictest4089182-21latency_hist11:50:023
4074525993728,5cyclictest4079142-21sendmail08:40:003
4074525993727,7cyclictest37-21ksoftirqd/308:50:023
4074525993725,10cyclictest37-21ksoftirqd/307:55:013
4074525993716,10cyclictest4090152-21cut12:10:023
407452199374,29cyclictest291rcuc/209:05:032
4074521993726,8cyclictest4081458-21sshd09:20:192
407452199372,5cyclictest131rcu_preempt08:30:012
407451999379,14cyclictest4076001-21modprobe07:40:001
407451299372,5cyclictest4079412-21munin-run08:45:000
407451299372,22cyclictest0-21swapper/010:20:020
4074512993722,11cyclictest0-21swapper/009:35:020
407452599364,3cyclictest131rcu_preempt12:05:013
407452599364,3cyclictest131rcu_preempt11:55:023
4074525993626,7cyclictest37-21ksoftirqd/309:55:023
4074525993625,8cyclictest4085621-21sshd10:44:203
4074525993622,9cyclictest306-21in:imuxsock07:30:003
4074525993621,12cyclictest4088134-21sed11:30:023
4074525993621,11cyclictest306-21in:imuxsock11:25:003
407452599361,3cyclictest131rcu_preempt12:35:013
407452199363,3cyclictest131rcu_preempt11:20:012
4074521993627,7cyclictest30-21ksoftirqd/211:30:012
407451999369,4cyclictest131rcu_preempt08:11:211
4074519993626,7cyclictest23-21ksoftirqd/110:45:001
407451299362,4cyclictest235-21systemd-journal12:15:200
4074512993623,10cyclictest12-21ksoftirqd/008:05:010
4074512993619,13cyclictest1-21systemd12:05:000
407451299361,31cyclictest0-21swapper/012:06:400
407452599353,3cyclictest131rcu_preempt11:15:023
4074525993528,5cyclictest37-21ksoftirqd/309:30:013
4074525993528,5cyclictest37-21ksoftirqd/309:30:003
4074525993528,4cyclictest4083705-21latency_hist10:05:023
4074525993526,5cyclictest271-21cron07:40:013
4074525993525,8cyclictest37-21ksoftirqd/307:10:013
4074525993525,6cyclictest4075738-21cron07:35:003
407452599351,3cyclictest131rcu_preempt10:15:023
407452599351,3cyclictest131rcu_preempt09:20:003
4074521993527,5cyclictest4088746-21munin-plugin-st11:45:002
4074521993524,7cyclictest306-21in:imuxsock08:05:002
407452199351,5cyclictest131rcu_preempt08:55:012
4074521993513,16cyclictest131rcu_preempt09:15:002
407451299359,5cyclictest131rcu_preempt10:11:030
4074512993528,4cyclictest4081270-21latency_hist09:20:010
4074512993527,5cyclictest4081049-21latency_hist09:15:020
4074512993524,8cyclictest12-21ksoftirqd/012:31:210
4074512993523,9cyclictest4075215-21sshd07:20:310
4074512993522,10cyclictest12-21ksoftirqd/011:55:020
407451299351,5cyclictest4077556-21sshd08:09:210
407451299351,5cyclictest131rcu_preempt11:50:010
407451299351,30cyclictest0-21swapper/009:52:350
4074525993427,4cyclictest4085395-21sh10:40:003
4074525993426,6cyclictest37-21ksoftirqd/312:15:013
4074525993426,5cyclictest4088744-21latency_hist11:45:023
4074525993424,6cyclictest4082250-21sendmail09:40:003
4074521993426,5cyclictest4080159-21latency_hist09:00:012
4074521993426,5cyclictest4076532-21latency_hist07:50:002
4074521993422,9cyclictest4088185-21sshd11:31:192
4074521993422,8cyclictest30-21ksoftirqd/210:40:012
4074521993421,10cyclictest272-21dbus-daemon08:25:022
4074521993419,11cyclictest272-21dbus-daemon09:10:002
407452199341,3cyclictest131rcu_preempt08:50:032
4074512993423,8cyclictest235-21systemd-journal07:33:550
407451299341,30cyclictest0-21swapper/008:31:170
407451299341,29cyclictest0-21swapper/007:15:340
407451299341,12cyclictest4083513-21date10:00:020
4074525993325,5cyclictest271-21cron09:00:003
407452599332,4cyclictest131rcu_preempt10:10:023
4074525993323,7cyclictest37-21ksoftirqd/307:25:023
407452599331,3cyclictest131rcu_preempt07:45:003
4074521993322,7cyclictest30-21ksoftirqd/209:55:012
4074521993319,11cyclictest272-21dbus-daemon09:45:022
4074521993319,11cyclictest272-21dbus-daemon09:45:012
407452199331,8cyclictest131rcu_preempt07:30:002
407452199331,5cyclictest131rcu_preempt12:05:022
407452199331,3cyclictest131rcu_preempt10:25:022
407452599321,4cyclictest131rcu_preempt09:35:003
407452599321,4cyclictest131rcu_preempt08:25:203
407452199323,3cyclictest131rcu_preempt11:50:012
4074521993223,6cyclictest1-21systemd08:15:012
4074521993221,8cyclictest30-21ksoftirqd/211:15:012
4074521993219,9cyclictest4079548-21date08:45:012
4074521993217,12cyclictest4074970-21sh07:20:012
407452599314,4cyclictest131rcu_preempt09:43:213
407452599314,4cyclictest131rcu_preempt09:43:213
4074525993124,4cyclictest4084858-21munin-plugin-st10:30:003
4074525993120,9cyclictest37-21ksoftirqd/312:30:003
4074521993124,4cyclictest4087194-21kworker/u8:111:25:012
4074525993022,5cyclictest4086154-21kworker/3:011:05:003
407452199309,17cyclictest1-21systemd12:35:002
407452199304,3cyclictest131rcu_preempt12:40:012
407452199303,5cyclictest131rcu_preempt10:31:192
4074521992918,9cyclictest30-21ksoftirqd/212:09:592
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional