You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-08 - 13:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Sat Aug 08, 2026 00:43:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4163600995946,9cyclictest420-21avahi-daemon21:25:592
4163600995543,7cyclictest265-21systemd-journal23:38:422
4163599995447,4cyclictest0-21swapper/100:00:011
416360099522,38cyclictest76471-21ssh23:05:542
4163599995239,8cyclictest420-21avahi-daemon00:05:541
4163599995141,6cyclictest420-21avahi-daemon23:25:081
4163599995141,6cyclictest420-21avahi-daemon23:25:081
4163600994940,6cyclictest132745-21sendmail00:40:012
4163599994943,2cyclictest0-21swapper/122:40:011
4163600994637,5cyclictest117600-21ssh00:15:012
4163600994637,5cyclictest0-21swapper/222:40:512
4163598994332,6cyclictest91620-21ssh23:30:570
4163598994332,6cyclictest91620-21ssh23:30:570
4163600994233,6cyclictest421-21dbus-daemon23:45:522
4163600994228,9cyclictest694-21cron21:10:012
4163599994236,3cyclictest0-21swapper/123:00:561
4163599994231,7cyclictest108584-21cups-browsed00:02:241
4163598994232,7cyclictest87380-21munin-run23:25:000
4163598994232,7cyclictest87380-21munin-run23:25:000
4163600994133,5cyclictest72372-21latency_hist23:00:022
4163600994130,7cyclictest2454516-21cups-browsed21:58:082
4163599994135,3cyclictest0-21swapper/121:20:011
4163599994131,6cyclictest5621-21diskmemload22:11:041
4163599994131,6cyclictest5621-21diskmemload22:11:041
4163599994131,3cyclictest0-21swapper/100:15:521
4163599994130,8cyclictest96427-21sendmail23:40:001
4163598994133,5cyclictest34343-21mailstats21:56:010
4163600994033,3cyclictest0-21swapper/200:00:012
4163600994032,4cyclictest0-21swapper/223:50:502
4163598994018,18cyclictest12750irq/33-eth000:25:560
4163600993932,4cyclictest20049-21kworker/u8:3-xprtiod21:40:422
4163600993932,4cyclictest20049-21kworker/u8:3-xprtiod21:40:422
4163600993929,7cyclictest421-21dbus-daemon20:20:022
4163599993932,3cyclictest0-21swapper/121:40:531
4163599993932,3cyclictest0-21swapper/121:40:531
4163599993930,4cyclictest0-21swapper/122:50:531
4163598993933,4cyclictest42016-21idleruntime-cro22:10:000
4163598993933,4cyclictest42016-21idleruntime-cro22:10:000
4163598993932,4cyclictest1134-21systemd-run20:55:010
416360099389,7cyclictest5621-21diskmemload23:40:532
4163600993830,5cyclictest60175-21sendmail22:40:012
4163600993828,4cyclictest0-21swapper/200:20:522
4163599993832,3cyclictest0-21swapper/122:00:011
4163599993831,4cyclictest1003-21mta-sts-daemon21:13:031
4163599993830,5cyclictest70275-21needreboot22:55:491
4163599993830,5cyclictest126707-21systemd-run00:30:021
4163599993830,4cyclictest24083-21latency_hist21:40:031
4163599993830,4cyclictest24083-21latency_hist21:40:031
4163599993829,6cyclictest421-21dbus-daemon22:50:011
4163598993831,4cyclictest2609-21munin-plugin-st21:00:010
4163600993729,4cyclictest0-21swapper/223:30:462
4163600993729,4cyclictest0-21swapper/223:30:462
4163600993726,7cyclictest420-21avahi-daemon21:37:242
4163600993726,7cyclictest420-21avahi-daemon21:37:242
4163600993724,9cyclictest129855-21latency_hist00:35:032
4163599993729,5cyclictest4183075-21cron20:15:001
4163599993728,6cyclictest55218-21ntp_kernel_err22:30:511
4163598993729,4cyclictest0-21swapper/000:05:540
4163600993628,4cyclictest1003-21mta-sts-daemon22:30:412
4163600993626,6cyclictest567-21gdbus23:05:022
4163599993630,3cyclictest0-21swapper/120:40:011
4163599993629,4cyclictest1001-21mta-sts-daemon23:55:021
4163599993628,4cyclictest0-21swapper/123:25:001
4163599993628,4cyclictest0-21swapper/123:25:001
416359999362,27cyclictest61355-21ssh22:40:551
4163599993619,4cyclictest78327-21kworker/u8:12-rpciod23:34:051
4163599993619,4cyclictest78327-21kworker/u8:12-rpciod23:34:051
416359899364,26cyclictest115751-21diskmemload00:10:540
4163598993627,7cyclictest9272-21aten_r2power_po21:15:430
4163598993627,6cyclictest2454516-21cups-browsed22:14:070
4163598993627,6cyclictest2454516-21cups-browsed22:14:070
4163600993528,4cyclictest11871-21latency_hist21:20:022
4163600993526,6cyclictest439-21systemd-logind19:35:012
4163599993528,3cyclictest0-21swapper/121:46:001
4163599993528,3cyclictest0-21swapper/121:46:001
4163599993527,5cyclictest439-21systemd-logind19:15:011
4163599993524,8cyclictest54103-21latency_hist22:30:021
416359999351,27cyclictest127881-21ssh00:30:571
4163598993529,4cyclictest7286-21unixbench-2d21:11:030
4163598993526,6cyclictest93867-21sshd23:35:420
4163598993526,6cyclictest2454516-21cups-browsed21:33:320
4163598993525,7cyclictest55206-21ntp_kernel_err22:30:510
4163600993427,4cyclictest0-21swapper/220:21:002
4163600993425,6cyclictest4189014-21head20:31:042
4163600993425,6cyclictest4173970-21grep19:41:012
4163600993424,5cyclictest0-21swapper/223:27:202
4163600993424,5cyclictest0-21swapper/223:27:202
4163599993427,4cyclictest4193577-21cron20:50:011
4163599993426,5cyclictest4187888-21aten_r2power_en20:30:421
4163599993425,6cyclictest1000-21mta-sts-daemon22:09:191
4163599993425,6cyclictest1000-21mta-sts-daemon22:09:191
4163598993428,3cyclictest0-21swapper/021:10:020
4163598993427,4cyclictest0-21swapper/023:40:430
4163598993426,5cyclictest58431-21ssh22:35:580
4163598993425,6cyclictest121085-21/usr/sbin/munin00:20:500
4163600993325,5cyclictest45159-21ssh22:15:022
4163600993325,5cyclictest45159-21ssh22:15:022
4163600993325,5cyclictest113668-21sh00:07:042
4163600993321,8cyclictest66621-21/usr/sbin/munin22:50:452
4163599993326,4cyclictest132721-21munin-run00:40:011
4163599993326,4cyclictest0-21swapper/100:10:511
4163599993325,4cyclictest84327-21cron23:20:001
4163599993325,4cyclictest84327-21cron23:20:001
4163598993326,5cyclictest421-21dbus-daemon23:05:020
4163598993326,5cyclictest106003-21kworker/u8:423:55:420
4163598993326,4cyclictest72283-21systemd-run23:00:020
4163598993325,5cyclictest439-21systemd-logind23:20:010
4163598993325,5cyclictest439-21systemd-logind23:20:010
4163598993320,11cyclictest0-21swapper/021:55:000
4163598993320,11cyclictest0-21swapper/021:55:000
416360099323,24cyclictest79547-21ssh23:10:562
416360099323,23cyclictest118862-21ssh00:15:572
4163600993225,4cyclictest0-21swapper/219:50:022
4163600993224,3cyclictest0-21swapper/221:50:472
4163600993224,3cyclictest0-21swapper/221:50:472
4163600993223,6cyclictest420-21avahi-daemon19:25:572
4163600993223,4cyclictest0-21swapper/200:26:052
4163599993225,4cyclictest5472-21unixbench-2d21:06:071
4163599993225,4cyclictest4166522-21unixbench_singl19:16:051
4163599993224,5cyclictest76978-21sed23:06:031
4163598993227,2cyclictest0-21swapper/021:26:050
4163598993225,3cyclictest0-21swapper/022:40:450
4163598993220,4cyclictest4178560-21cron20:00:000
4163598993220,4cyclictest4178560-21cron20:00:000
4163598993210,15cyclictest12650irq/32-eth023:25:550
4163598993210,15cyclictest12650irq/32-eth023:25:550
416360099314,5cyclictest36-21ksoftirqd/221:21:082
416360099313,21cyclictest6735-21ssh21:10:512
4163600993123,5cyclictest109310-21fschecks_count00:00:472
4163600993122,6cyclictest485-21in:imuxsock20:00:002
4163600993122,6cyclictest485-21in:imuxsock20:00:002
4163600993122,5cyclictest2454516-21cups-browsed21:32:042
4163600993122,4cyclictest1003-21mta-sts-daemon20:10:512
4163600993122,4cyclictest0-21swapper/222:49:002
4163600993122,3cyclictest171rcu_preempt20:50:432
416360099311,23cyclictest52868-21awk22:26:042
4163599993125,3cyclictest0-21swapper/123:46:011
4163599993125,3cyclictest0-21swapper/100:25:011
4163599993124,4cyclictest18762-21latency21:30:481
4163599993124,4cyclictest14791-21munin-plugin-st21:25:011
4163599993123,5cyclictest4020-21tail21:01:131
4163599993123,3cyclictest0-21swapper/122:00:571
416359999311,23cyclictest16042-21ssh21:25:581
4163598993126,2cyclictest0-21swapper/022:50:450
4163598993125,4cyclictest63958-21tr22:45:470
4163598993125,4cyclictest51214-21latency_hist22:25:020
4163598993123,6cyclictest76337-21needreboot23:05:490
4163598993122,7cyclictest421-21dbus-daemon20:20:010
4163598993121,6cyclictest0-21swapper/021:45:560
4163598993121,6cyclictest0-21swapper/021:45:560
4163598993119,9cyclictest4164961-21unixbench_multi19:11:030
4163600993021,5cyclictest46779-21head22:16:052
4163600993021,2cyclictest171rcu_preempt20:41:012
4163600993020,3cyclictest171rcu_preempt20:35:442
4163600993018,9cyclictest4186965-21ntp_kernel_pll_20:25:532
4163600993015,3cyclictest171rcu_preempt20:45:452
4163600993014,9cyclictest351ktimers/223:20:552
4163600993014,9cyclictest351ktimers/223:20:552
4163599993026,3cyclictest4163597-21cyclictest19:25:571
4163599993023,4cyclictest4180092-21munin-plugin-st20:05:001
4163599993023,4cyclictest4180092-21munin-plugin-st20:05:001
4163599993022,5cyclictest1001-21mta-sts-daemon21:51:031
4163599993022,5cyclictest1001-21mta-sts-daemon21:51:031
4163599993011,4cyclictest171rcu_preempt22:21:041
416359899304,21cyclictest12996-21ssh21:21:040
416359899303,21cyclictest151ktimers/000:30:560
4163598993025,3cyclictest103605-21ntp_kernel_pll_23:50:510
4163598993024,4cyclictest100230-21fschecks_time23:45:490
4163598993023,3cyclictest0-21swapper/021:40:010
4163598993023,3cyclictest0-21swapper/021:40:010
4163600992924,3cyclictest341rcuc/219:35:562
4163600992923,3cyclictest0-21swapper/222:06:002
4163600992923,3cyclictest0-21swapper/222:06:002
4163600992922,3cyclictest0-21swapper/222:01:022
4163600992920,6cyclictest2454516-21cups-browsed19:10:442
4163600992913,11cyclictest421-21dbus-daemon22:20:552
4163599992923,3cyclictest0-21swapper/120:25:011
4163599992922,4cyclictest20048-21kworker/u8:1+xprtiod22:20:001
4163599992922,4cyclictest1924-21needreboot20:55:541
4163599992921,5cyclictest1003-21mta-sts-daemon19:20:441
4163598992921,5cyclictest4185116-21fschecks_count20:20:460
4163598992921,5cyclictest118043-21/usr/sbin/munin00:15:450
4163598992921,4cyclictest0-21swapper/021:40:530
4163598992921,4cyclictest0-21swapper/021:40:530
416359899291,22cyclictest79461-21ssh23:10:530
4163598992911,3cyclictest171rcu_preempt00:35:490
416360099286,5cyclictest36-21ksoftirqd/221:00:592
416360099284,4cyclictest36-21ksoftirqd/223:15:572
416360099284,4cyclictest36-21ksoftirqd/223:15:572
4163600992821,4cyclictest1003-21mta-sts-daemon19:10:012
4163600992821,4cyclictest1003-21mta-sts-daemon19:10:012
4163600992821,3cyclictest0-21swapper/221:46:042
4163600992821,3cyclictest0-21swapper/221:46:042
4163599992813,8cyclictest2454516-21cups-browsed23:42:011
416359999281,21cyclictest79441-21ssh23:10:521
416359899282,6cyclictest52188-21ntpq22:25:500
4163598992823,2cyclictest0-21swapper/000:03:520
4163598992821,4cyclictest4168089-21cron19:25:000
4163598992821,3cyclictest0-21swapper/022:15:520
4163598992817,8cyclictest4190578-21munin-plugin-st20:40:010
416360099275,5cyclictest36-21ksoftirqd/220:56:002
4163600992717,3cyclictest0-21swapper/219:15:572
4163600992716,4cyclictest341rcuc/220:05:552
4163600992711,3cyclictest171rcu_preempt19:51:052
4163599992722,2cyclictest0-21swapper/120:00:011
4163599992722,2cyclictest0-21swapper/120:00:011
4163599992719,5cyclictest4176101-21fschecks_count19:50:461
4163598992718,6cyclictest4168904-21needreboot19:25:500
4163598992716,4cyclictest37728-21unixbench_multi22:01:080
4163600992613,2cyclictest171rcu_preempt19:20:372
4163599992619,4cyclictest0-21swapper/120:45:011
4163599992618,5cyclictest4181044-21ntpq20:05:571
4163599992618,5cyclictest1104-21sh20:55:011
4163599992614,8cyclictest421-21dbus-daemon19:10:011
4163599992614,8cyclictest421-21dbus-daemon19:10:011
4163598992621,3cyclictest4170101-21fschecks_count19:30:490
4163598992618,6cyclictest0-21swapper/020:15:000
4163598992617,7cyclictest4172256-21switchtime19:36:030
4163598992617,6cyclictest4175584-21cut19:50:020
4163598992615,8cyclictest0-21swapper/019:45:020
4163600992518,3cyclictest1001-21mta-sts-daemon20:00:472
4163600992518,3cyclictest1001-21mta-sts-daemon20:00:472
4163599992517,5cyclictest2454516-21cups-browsed19:42:501
416359899253,14cyclictest2454509-21cupsd20:30:550
416359899251,20cyclictest4187111-21ps20:26:000
4163598992511,11cyclictest0-21swapper/019:51:040
4163599992419,2cyclictest0-21swapper/119:30:491
4163599992414,7cyclictest4187152-21grep20:26:001
4163598992416,5cyclictest4180201-21latency_hist20:05:020
4163598992416,5cyclictest4180201-21latency_hist20:05:020
4163598992416,5cyclictest3138-21fschecks_count21:00:490
4163599992318,3cyclictest0-21swapper/120:20:011
4163599992317,3cyclictest4171350-21aten_r2power_cu19:35:431
4163599992315,5cyclictest4174318-21/usr/sbin/munin19:45:481
4163598992314,4cyclictest12750irq/33-eth019:15:520
416359899222,15cyclictest4191476-21ntpq20:40:540
416359899215,11cyclictest428-21mta-sts-daemon20:50:010
4163598992116,3cyclictest4181430-21unixbench-2d20:06:040
416359899193,10cyclictest1-21systemd19:10:010
416359899193,10cyclictest1-21systemd19:10:010
416360199173,9cyclictest0-21swapper/322:20:553
416360199172,7cyclictest0-21swapper/321:55:513
416360199162,8cyclictest0-21swapper/300:11:003
416360199162,7cyclictest0-21swapper/323:25:583
416360199162,7cyclictest0-21swapper/323:25:583
416360199152,8cyclictest0-21swapper/323:45:583
416360199152,7cyclictest0-21swapper/323:15:503
416360199152,7cyclictest0-21swapper/323:15:503
416360199152,6cyclictest0-21swapper/320:00:533
416360199152,6cyclictest0-21swapper/320:00:533
416360199151,8cyclictest0-21swapper/322:15:523
416360199151,8cyclictest0-21swapper/321:20:563
416360199151,7cyclictest0-21swapper/320:45:543
416360199151,6cyclictest0-21swapper/319:35:563
416360199143,4cyclictest0-21swapper/319:40:503
416360199142,8cyclictest0-21swapper/323:50:443
416360199142,8cyclictest0-21swapper/321:40:533
416360199142,8cyclictest0-21swapper/321:40:533
416360199142,8cyclictest0-21swapper/321:26:043
416360199142,8cyclictest0-21swapper/300:35:513
416360199142,7cyclictest0-21swapper/323:55:523
416360199142,7cyclictest0-21swapper/300:31:013
416360199142,6cyclictest0-21swapper/300:00:523
416360199141,7cyclictest0-21swapper/322:55:543
416360199141,7cyclictest0-21swapper/321:10:493
416360199141,6cyclictest0-21swapper/320:20:523
416360199134,6cyclictest0-21swapper/322:35:523
416360199133,7cyclictest0-21swapper/322:30:423
416360199133,6cyclictest0-21swapper/323:35:583
416360199133,6cyclictest0-21swapper/321:50:423
416360199133,6cyclictest0-21swapper/321:50:423
416360199133,4cyclictest0-21swapper/320:05:533
416360199132,7cyclictest0-21swapper/323:30:573
416360199132,7cyclictest0-21swapper/323:30:573
416360199132,7cyclictest0-21swapper/323:10:483
416360199132,7cyclictest0-21swapper/321:45:453
416360199132,7cyclictest0-21swapper/321:45:453
416360199132,7cyclictest0-21swapper/321:36:013
416360199132,7cyclictest0-21swapper/321:36:013
416360199132,7cyclictest0-21swapper/321:30:423
416360199132,7cyclictest0-21swapper/300:05:443
416360199132,6cyclictest0-21swapper/323:40:523
416360199132,6cyclictest0-21swapper/319:50:543
416360199132,5cyclictest0-21swapper/320:30:523
416360199132,5cyclictest0-21swapper/320:15:523
416360199131,8cyclictest0-21swapper/322:45:533
416360199131,8cyclictest0-21swapper/319:20:013
416360199131,8cyclictest0-21swapper/300:15:523
416360199131,7cyclictest0-21swapper/322:50:543
416360199131,7cyclictest0-21swapper/322:05:533
416360199131,7cyclictest0-21swapper/322:05:533
416360199131,7cyclictest0-21swapper/321:15:413
416360199131,7cyclictest0-21swapper/319:55:553
416360199131,7cyclictest0-21swapper/319:55:553
416360199131,7cyclictest0-21swapper/300:25:513
416360199131,7cyclictest0-21swapper/300:20:423
416360199131,6cyclictest0-21swapper/321:05:533
416360199131,6cyclictest0-21swapper/320:50:523
416360199131,6cyclictest0-21swapper/320:25:563
416360199130,7cyclictest0-21swapper/319:30:573
416360199122,6cyclictest0-21swapper/323:00:423
416360199122,6cyclictest0-21swapper/321:01:013
416360199122,6cyclictest0-21swapper/321:00:013
416360199121,7cyclictest0-21swapper/323:20:453
416360199121,7cyclictest0-21swapper/323:20:453
416360199121,7cyclictest0-21swapper/322:25:423
416360199121,7cyclictest0-21swapper/322:10:533
416360199121,7cyclictest0-21swapper/322:10:533
416360199121,7cyclictest0-21swapper/322:00:553
416360199121,7cyclictest0-21swapper/320:35:523
416360199121,6cyclictest0-21swapper/320:10:423
416360199121,6cyclictest0-21swapper/319:45:563
416360199121,5cyclictest0-21swapper/323:05:413
416360199112,6cyclictest0-21swapper/322:40:413
416360199111,6cyclictest0-21swapper/320:45:013
416360199103,6cyclictest0-21swapper/319:10:013
416360199103,6cyclictest0-21swapper/319:10:013
416360199101,6cyclictest0-21swapper/319:20:413
41636019993,4cyclictest0-21swapper/319:10:523
41636019991,6cyclictest0-21swapper/319:25:503
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional