You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-03 - 22:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot8s.osadl.org (updated Wed Dec 03, 2025 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
33906239913140,1310cyclictest37952042sleep33
33906239912290,1226cyclictest38679952sleep33
33906239912290,1226cyclictest38679952sleep33
33906239912120,1208cyclictest37302932sleep33
35829552260,2sleep20-21swapper/22
37549432211,5sleep13754986-21ssh1
33904772201,3sleep30-21swapper/33
339062399181,13cyclictest3493525-21taskset3
339062399180,15cyclictest3730628-21taskset3
190898180,7rtkit-daemon3512872-21diskstats0
339062399171,13cyclictest0-21swapper/33
339062399164,8cyclictest36965142sleep33
3390623991613,1cyclictest0-21swapper/33
3390623991613,1cyclictest0-21swapper/33
339062399161,13cyclictest0-21swapper/33
339062399161,13cyclictest0-21swapper/33
339062399161,12cyclictest3649729-21turbostat3
339062399161,12cyclictest3649729-21turbostat3
339062399161,12cyclictest3625666-21turbostat3
339062399161,12cyclictest3499784-21sleep3
339062399161,12cyclictest3443586-21taskset3
339062399161,12cyclictest0-21swapper/33
339062399161,12cyclictest0-21swapper/33
339062399161,12cyclictest0-21swapper/33
339062399161,12cyclictest0-21swapper/33
339062399161,11cyclictest3518990-21taskset3
339062399161,11cyclictest0-21swapper/33
339062399160,12cyclictest0-21swapper/33
190898160,4rtkit-daemon3396993-21munin-node0
339062399152,10cyclictest3827589-21turbostat3
339062399152,10cyclictest3405274-21turbostat3
339062399151,2cyclictest0-21swapper/33
339062399151,2cyclictest0-21swapper/33
3390623991512,1cyclictest0-21swapper/33
339062399151,1cyclictest0-21swapper/33
339062399151,1cyclictest0-21swapper/33
339062399151,12cyclictest0-21swapper/33
339062399151,11cyclictest3803506-21taskset3
339062399151,11cyclictest3679590-21taskset3
339062399151,11cyclictest3512193-21taskset3
339062399151,11cyclictest3436849-21turbostat3
339062399151,11cyclictest0-21swapper/33
339062399151,11cyclictest0-21swapper/33
339062399151,11cyclictest0-21swapper/33
339062399151,11cyclictest0-21swapper/33
339062399151,11cyclictest0-21swapper/33
339062399151,11cyclictest0-21swapper/33
339062399151,11cyclictest0-21swapper/33
339062399151,11cyclictest0-21swapper/33
339062399151,11cyclictest0-21swapper/33
339062399151,11cyclictest0-21swapper/33
339062399151,10cyclictest0-21swapper/33
339062399150,1cyclictest0-21swapper/33
339062399150,11cyclictest33911332sleep33
190898150,3rtkit-daemon3396993-21munin-node0
34915792140,1sleep10-21swapper/11
339062399141,3cyclictest0-21swapper/33
3390623991411,1cyclictest0-21swapper/33
339062399141,11cyclictest0-21swapper/33
339062399141,11cyclictest0-21swapper/33
339062399141,11cyclictest0-21swapper/33
339062399141,11cyclictest0-21swapper/33
339062399141,10cyclictest3666107-21taskset3
339062399141,10cyclictest0-21swapper/33
339062399141,10cyclictest0-21swapper/33
339062399141,10cyclictest0-21swapper/33
339062399141,10cyclictest0-21swapper/33
339062399141,10cyclictest0-21swapper/33
339062399141,10cyclictest0-21swapper/33
339062399141,10cyclictest0-21swapper/33
339062399140,5cyclictest3819392-21taskset3
339062399140,11cyclictest0-21swapper/33
36101042131,8sleep23610279-21/usr/sbin/munin2
35394642130,2sleep00-21swapper/00
34812312131,3sleep13481264-21apt-get1
34194992131,9sleep20-21swapper/22
339062399131,9cyclictest3537996-21taskset3
339062399131,9cyclictest0-21swapper/33
339062399131,9cyclictest0-21swapper/33
339062399131,10cyclictest0-21swapper/33
339062399130,11cyclictest3405512-21taskset3
190898130,8rtkit-daemon3552684-21ssh0
35586702120,4sleep10-21swapper/11
34673622121,8sleep13467512-21awk1
190898120,5rtkit-daemon3673416-21ssh0
190898120,3rtkit-daemon3396993-21munin-node2
37783442110,3sleep00-21swapper/00
37564282110,3sleep23757885-21kthreadcore2
37526222110,1sleep00-21swapper/00
33996472110,4sleep03400635-21irqstats0
190898110,7rtkit-daemon3686373-21cut0
190898110,7rtkit-daemon3664169-21ssh0
190898110,4rtkit-daemon3786357-21ssh0
190898110,1rtkit-daemon0-21swapper/00
190898110,1rtkit-daemon0-21swapper/00
38356312101,2sleep20-21swapper/22
38278922103,4sleep13827927-21apt-get1
37303192101,5sleep23730336-21cron2
36963712100,3sleep00-21swapper/00
35249392100,7sleep23525003-21latency_hist2
190898100,7rtkit-daemon667-21snmpd0
190898100,5rtkit-daemon3706499-21apt-get0
190898100,5rtkit-daemon3570837-21/usr/sbin/munin0
190898100,3rtkit-daemon667-21snmpd2
190898100,3rtkit-daemon3656974-21ssh0
3873549290,6sleep23873631-21ntpq2
3859081291,6sleep13859086-21sh1
3810780290,3sleep13810788-21ssh1
3801948290,6sleep03801956-21ssh0
3769511291,5sleep03769519-21ssh0
3763376290,6sleep13763722-21cut1
3746194290,1sleep00-21swapper/00
3731655290,1sleep23732528-21cut2
3722636291,5sleep03722788-21df0
3696768290,3sleep20-21swapper/22
3690561290,7sleep13690369-21/usr/sbin/munin1
3681002291,5sleep03681010-21ssh0
3640952291,5sleep03544019-21diskmemload0
3598905290,1sleep13599137-21unixbench_multi1
3590476291,5sleep03590636-21sed0
3552705291,4sleep23552731-21apt-get2
3549550290,6sleep03549633-21ntp_states0
3531756291,4sleep13531789-21apt-get1
3498723290,7sleep13498876-21ntpq1
3405527291,4sleep23405560-21apt-get2
19089890,5rtkit-daemon3816958-21ntp_states0
19089890,5rtkit-daemon3424077-21cron0
19089890,4rtkit-daemon3608507-21ssh2
19089890,4rtkit-daemon3544019-21diskmemload2
19089890,4rtkit-daemon3522221-21grep0
19089890,4rtkit-daemon3442314-21sendmail_mailqu0
19089890,4rtkit-daemon3442314-21sendmail_mailqu0
19089890,1rtkit-daemon0-21swapper/00
19089890,1rtkit-daemon0-21swapper/00
19089890,1rtkit-daemon0-21swapper/00
3868021281,4sleep23868039-21dump-pmu-power2
3868021281,4sleep23868039-21dump-pmu-power2
3818058280,3sleep20-21swapper/22
3795228281,4sleep23795246-21munin-run2
3787990281,4sleep03788987-21irqstats0
3776612281,4sleep13776877-21grep1
3763724280,1sleep23764488-21kthreadcore2
3747318281,4sleep23748312-21cut2
3713239281,4sleep13713247-21ssh1
3656331281,4sleep13656339-21ssh1
3570862280,1sleep13572689-21ssh1
3538196281,5sleep13538311-21fw_packets1
3501581280,1sleep23503609-21cut2
3482526280,2sleep20-21swapper/22
3475091280,1sleep20-21swapper/22
3463080280,6sleep23464032-21cut2
3448678281,4sleep03448749-21ntp_states0
19089880,5rtkit-daemon473-21gmain0
19089880,4rtkit-daemon674-21lldpd0
19089880,4rtkit-daemon671-21ntpd0
19089880,4rtkit-daemon671-21ntpd0
19089880,4rtkit-daemon667-21snmpd2
19089880,4rtkit-daemon667-21snmpd0
19089880,4rtkit-daemon438-21wpa_supplicant0
19089880,4rtkit-daemon438-21wpa_supplicant0
19089880,4rtkit-daemon3567200-21ssh0
19089880,4rtkit-daemon3396993-21munin-node2
19089880,4rtkit-daemon3396993-21munin-node0
19089880,3rtkit-daemon3500204-21diskstats0
19089880,1rtkit-daemon0-21swapper/00
19089880,1rtkit-daemon0-21swapper/00
3850104270,5sleep13850110-21ssh1
3816857270,4sleep13816940-21ntp_states1
3803223270,5sleep23803471-21sh2
3780916270,4sleep13782500-21cut1
3730646270,3sleep13730685-21apt-get1
3689217270,1sleep10-21swapper/11
3680436270,1sleep10-21swapper/11
3675945271,3sleep20-21swapper/22
3665727270,5sleep13665729-21ssh1
3663289270,4sleep23663338-21ntp_states2
3657774271,4sleep23657800-21munin-run2
3633959271,3sleep13633996-21apt-get1
3632464271,4sleep03632470-21ssh0
3623986270,4sleep03623994-21ssh0
3601781271,3sleep03601820-21apt-get0
3587564270,4sleep23589604-21cut2
3585489270,4sleep13585617-21grep1
3568343270,4sleep13568351-21ssh1
3518004271,3sleep23518144-21timerwakeupswit2
3442398271,4sleep13442648-21grep1
3442398271,4sleep13442648-21grep1
3411801271,3sleep13411834-21apt-get1
3405514270,1sleep00-21swapper/00
3396605270,1sleep20-21swapper/22
3390823270,4sleep13390978-21diskstats1
19089870,4rtkit-daemon674-21lldpd0
19089870,4rtkit-daemon667-21snmpd2
19089870,4rtkit-daemon667-21snmpd1
19089870,4rtkit-daemon473-21gmain1
19089870,4rtkit-daemon421-21avahi-daemon2
19089870,4rtkit-daemon3396993-21munin-node1
19089870,4rtkit-daemon3396993-21munin-node0
19089870,4rtkit-daemon3396993-21munin-node0
19089870,4rtkit-daemon3396993-21munin-node0
19089870,4rtkit-daemon1928-21sendmail-mta1
19089870,4rtkit-daemon1928-21sendmail-mta1
19089870,3rtkit-daemon674-21lldpd0
19089870,3rtkit-daemon671-21ntpd2
19089870,3rtkit-daemon671-21ntpd0
19089870,3rtkit-daemon667-21snmpd2
19089870,3rtkit-daemon667-21snmpd2
19089870,3rtkit-daemon3836477-21ssh0
19089870,3rtkit-daemon3624806-21ssh2
19089870,3rtkit-daemon3396993-21munin-node2
19089870,3rtkit-daemon3377941-21kworker/0:0+kacpid0
19089870,3rtkit-daemon1928-21sendmail-mta2
19089870,3rtkit-daemon1928-21sendmail-mta0
19089870,3rtkit-daemon1223-21cron2
3852154261,2sleep00-21swapper/00
3826960260,3sleep13826970-21ssh1
3825763260,1sleep20-21swapper/22
3787474260,4sleep13787480-21ssh1
3730862260,4sleep03731065-21hddtemp_smartct0
3729832260,1sleep10-21swapper/11
3705720261,3sleep2667-21snmpd2
3672358260,4sleep13672366-21ssh1
3671964260,4sleep20-21swapper/22
3648518260,3sleep13648526-21ssh1
3648518260,3sleep13648526-21ssh1
3602798260,4sleep13603709-21cut1
3584663260,1sleep00-21swapper/00
3582793260,3sleep13582954-21sed1
3561086260,1sleep20-21swapper/22
3545844261,3sleep13505985-21kworker/u16:4+rpciod1
3531277260,3sleep23531236-21apt-get2
3525195261,2sleep20-21swapper/22
3500685260,4sleep10-21swapper/11
3494717260,3sleep23495719-21kthreadcore2
3492457261,2sleep20-21swapper/22
3468654261,2sleep10-21swapper/11
3430296260,1sleep20-21swapper/22
3390383260,2sleep10-21swapper/11
19089860,3rtkit-daemon671-21ntpd0
19089860,3rtkit-daemon671-21ntpd0
19089860,3rtkit-daemon3396993-21munin-node2
3884001251,2sleep10-21swapper/11
3882692250,3sleep20-21swapper/22
3875829251,2sleep10-21swapper/11
3851795250,2sleep00-21swapper/00
3843049250,3sleep10-21swapper/11
3802483250,3sleep10-21swapper/11
3795478251,1sleep20-21swapper/22
3704696250,3sleep13704704-21ssh1
3687589250,2sleep20-21swapper/22
3616753250,3sleep00-21swapper/00
3531145250,1sleep10-21swapper/11
3514839250,2sleep10-21swapper/11
3512404250,1sleep10-21swapper/11
3469999250,2sleep2421rcuc/22
3456159250,1sleep20-21swapper/22
3455815250,1sleep10-21swapper/11
3455815250,1sleep10-21swapper/11
3455796250,1sleep20-21swapper/22
3455796250,1sleep20-21swapper/22
3443083250,3sleep13443098-21apt-get1
3437084250,2sleep20-21swapper/22
3437084250,2sleep20-21swapper/22
3436577250,1sleep10-21swapper/11
3418068250,2sleep10-21swapper/11
3417606250,1sleep20-21swapper/22
3397485251,2sleep10-21swapper/11
19089850,1rtkit-daemon0-21swapper/11
3719894240,2sleep10-21swapper/11
3716002240,2sleep00-21swapper/00
3633027240,3sleep13607663-21kworker/u16:2+rpciod1
3598784240,2sleep03377941-21kworker/0:0+kacpid0
3551010240,2sleep23551018-21ssh2
3524668240,3sleep10-21swapper/11
3443573240,1sleep20-21swapper/22
3431263240,2sleep23431799-21irqrtprio2
3410171240,2sleep10-21swapper/11
3405042240,3sleep20-21swapper/22
3389633240,1sleep20-21swapper/22
3538000230,1sleep20-21swapper/22
3506346230,1sleep20-21swapper/22
3476360230,1sleep10-21swapper/11
3461458230,1sleep10-21swapper/11
3424333230,1sleep10-21swapper/11
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional