You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-22 - 21:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Sat Nov 22, 2025 12:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
382534999821,3cyclictest131rcu_preempt11:35:021
3825344997623,48cyclictest16950irq/224-800000011:55:010
3825360997530,41cyclictest3836685-21cat10:45:013
3825344997436,34cyclictest16950irq/224-800000012:05:000
3825360997322,14cyclictest3826346-21systemd-run07:30:023
3825344997326,44cyclictest16950irq/224-800000007:30:410
3825360997226,15cyclictest1-21systemd12:19:413
382534999703,14cyclictest3839665-21/usr/sbin/munin11:45:001
3825360996820,15cyclictest272-21dbus-daemon09:35:003
3825349996823,42cyclictest3836302-21sort10:40:021
3825349996823,42cyclictest3836302-21sort10:40:011
382534999682,14cyclictest3836489-21munin-run10:45:001
382534499662,15cyclictest3832054-21modprobe09:20:000
3825357996437,19cyclictest272-21dbus-daemon11:00:002
3825344996429,31cyclictest235-21systemd-journal09:00:000
3825360996345,15cyclictest3841698-21sshd12:20:413
382534999632,13cyclictest3835467-21munin-run10:25:011
382534999632,13cyclictest3835467-21munin-run10:25:011
3825360996245,12cyclictest3840375-21sshd11:56:133
3825360996243,15cyclictest3834387-21sshd10:03:413
3825349996244,13cyclictest235-21systemd-journal12:00:001
382534999622,10cyclictest3836721-21apt-get10:45:281
382534999621,15cyclictest3842525-21systemd-run12:40:011
3825344996222,9cyclictest12-21ksoftirqd/012:00:030
382534499622,18cyclictest3829221-21unin-run08:25:010
3825344996220,10cyclictest12-21ksoftirqd/010:35:020
3825360996146,11cyclictest3828883-21sshd08:15:353
3825360996145,12cyclictest3837551-21sshd11:03:423
3825360996143,14cyclictest3842205-21sshd12:31:163
382534999612,14cyclictest3832309-21sshd09:21:421
3825344996113,12cyclictest3831534-21sendmail09:10:010
3825360996044,13cyclictest3834891-21sshd10:13:413
3825360996044,11cyclictest3837804-21sshd11:08:413
382534999602,16cyclictest3827481-21sort07:50:011
382534999601,14cyclictest3826567-21modprobe07:32:411
3825344996026,31cyclictest235-21systemd-journal09:05:010
3825344996019,11cyclictest12-21ksoftirqd/009:35:000
3825360995944,11cyclictest3835135-21sshd10:16:413
3825360995944,11cyclictest3834096-21sshd09:55:583
3825360995942,13cyclictest3829685-21sshd08:32:403
3825360995939,16cyclictest3840920-21sshd12:08:423
382534999592,13cyclictest3834371-21sshd10:00:411
3825349995919,36cyclictest3838303-21cron11:17:011
3825349995919,35cyclictest272-21dbus-daemon08:00:021
382534999591,13cyclictest3835928-21sshd10:31:421
382534999591,10cyclictest3840143-21cron11:55:011
382534499593,18cyclictest3836509-21cat10:45:010
382534499592,17cyclictest281-21systemd-logind11:20:000
382534499591,4cyclictest3839106-21cron11:35:010
3825360995844,10cyclictest3835677-21sshd10:26:433
3825360995844,10cyclictest3828679-21munin-run08:15:013
3825360995842,12cyclictest3826851-21munin-run07:40:003
3825360995841,14cyclictest3827608-21sshd07:53:413
3825360995841,14cyclictest3827608-21sshd07:53:413
3825360995839,15cyclictest3834637-21sshd10:07:413
3825360995839,15cyclictest3834637-21sshd10:07:403
382534999582,14cyclictest3829888-21grep08:35:021
3825349995821,33cyclictest23-21ksoftirqd/109:45:001
382534999582,12cyclictest3831540-21modprobe09:10:001
3825349995820,34cyclictest3828133-21sshd08:01:411
3825349995818,35cyclictest23-21ksoftirqd/108:45:181
382534499582,19cyclictest3826875-21systemd-run07:40:000
3825344995817,13cyclictest3842502-21systemctl12:40:000
3825344995815,5cyclictest306-21in:imuxsock07:15:010
3825360995745,9cyclictest3837256-21sshd10:55:433
3825360995743,9cyclictest3827378-21munin-run07:50:003
3825360995743,10cyclictest3838088-21munin-run11:15:013
3825360995743,10cyclictest3837040-21munin-run10:55:003
3825360995743,10cyclictest3837040-21munin-run10:55:003
3825360995743,10cyclictest3833639-21munin-run09:50:013
3825360995743,10cyclictest3830462-21sshd08:45:473
3825360995742,11cyclictest3836758-21sshd10:46:043
3825360995742,11cyclictest3829184-21sshd08:22:403
3825360995733,21cyclictest3830923-21cat08:55:033
3825360995733,21cyclictest3830923-21cat08:55:023
3825360995727,25cyclictest3825392-21cat07:10:013
3825360995722,11cyclictest1-21systemd12:28:003
382534999574,11cyclictest3840084-21ls11:50:021
382534999573,13cyclictest3827124-21munin-run07:45:001
3825349995725,28cyclictest23-21ksoftirqd/110:10:411
382534999572,50cyclictest3826087-21munin-plugin-st07:25:001
3825349995714,37cyclictest0-21swapper/108:55:011
3825349995714,37cyclictest0-21swapper/108:55:011
382534999571,13cyclictest363-21snmpd08:12:141
3825344995735,20cyclictest12-21ksoftirqd/008:15:010
3825344995725,29cyclictest3836785-21latency_hist10:50:030
3825344995724,29cyclictest0-21swapper/008:10:020
3825360995643,9cyclictest48-21kcompactd009:14:573
3825360995643,9cyclictest3839341-21sshd11:36:113
3825360995643,10cyclictest3830210-21sshd08:40:423
3825360995643,10cyclictest3825508-21sshd07:10:213
3825360995642,9cyclictest48-21kcompactd010:24:173
3825360995642,9cyclictest48-21kcompactd010:24:163
3825360995642,10cyclictest3842455-21sshd12:36:163
3825360995642,10cyclictest3841160-21sshd12:11:143
3825360995641,12cyclictest3839870-21sshd11:46:113
3825360995641,11cyclictest3833345-21sshd09:41:413
3825360995640,13cyclictest48-21kcompactd010:30:233
3825357995628,18cyclictest272-21dbus-daemon09:40:012
382534999562,16cyclictest3839381-21munin-run11:40:011
3825344995621,3cyclictest0-21swapper/010:38:430
3825344995621,3cyclictest0-21swapper/010:38:420
3825344995619,14cyclictest272-21dbus-daemon12:20:000
382534499561,51cyclictest0-21swapper/011:11:590
382534499561,22cyclictest0-21swapper/008:48:140
3825360995547,4cyclictest3832067-21munin-run09:20:003
3825360995544,8cyclictest3827088-21sshd07:40:263
3825360995544,7cyclictest3838834-21sshd11:25:423
3825360995543,8cyclictest48-21kcompactd008:28:213
3825360995541,9cyclictest48-21kcompactd011:32:333
3825360995541,9cyclictest48-21kcompactd009:08:503
3825360995541,9cyclictest363-21snmpd08:06:313
3825360995541,11cyclictest3836184-21sshd10:37:433
3825360995541,11cyclictest3836184-21sshd10:37:423
3825360995540,11cyclictest363-21snmpd11:21:553
3825360995540,11cyclictest363-21snmpd11:21:543
3825360995537,13cyclictest3839615-21sshd11:40:433
3825360995535,14cyclictest1-21systemd12:05:003
382534999552,14cyclictest3840702-21/usr/sbin/munin12:05:011
3825349995519,32cyclictest23-21ksoftirqd/110:55:021
3825349995519,32cyclictest23-21ksoftirqd/110:55:011
382534999551,14cyclictest3832575-21cron09:29:591
3825344995520,5cyclictest3237915-1kworker/0:2H07:50:020
3825344995520,5cyclictest3237915-1kworker/0:2H07:50:020
3825360995446,4cyclictest3830006-21munin-run08:40:013
3825360995442,9cyclictest3838316-21sshd11:18:423
3825360995442,9cyclictest3826566-21sshd07:32:413
3825360995442,9cyclictest3825780-21sshd07:17:413
3825360995441,9cyclictest48-21kcompactd009:26:363
3825360995440,9cyclictest48-21kcompactd011:53:373
3825360995422,26cyclictest3832537-21latency_hist09:25:023
382534999542,5cyclictest363-21snmpd09:16:001
3825349995423,28cyclictest23-21ksoftirqd/108:38:061
382534999542,15cyclictest3826338-21munin-run07:30:001
3825349995421,30cyclictest3829457-21latency_hist08:30:011
382534999542,12cyclictest235-21systemd-journal11:06:431
3825349995421,29cyclictest272-21dbus-daemon07:40:021
3825349995419,29cyclictest23-21ksoftirqd/112:20:011
382534999541,15cyclictest363-21snmpd10:58:241
382534999541,14cyclictest3835719-21systemd-run10:30:001
382534999541,13cyclictest363-21snmpd08:23:561
382534499543,16cyclictest235-21systemd-journal12:10:000
3825344995430,21cyclictest16950irq/224-800000008:03:390
382534499542,17cyclictest3825791-21cron07:19:590
3825344995420,8cyclictest12-21ksoftirqd/011:00:020
3825344995416,15cyclictest272-21dbus-daemon07:59:590
382534499541,21cyclictest0-21swapper/009:15:010
3825344995410,8cyclictest3833435-21wc09:45:010
3825360995343,7cyclictest3833847-21sshd09:51:423
3825360995343,7cyclictest3833847-21sshd09:51:413
3825360995343,7cyclictest3830983-21sshd08:59:413
3825360995342,8cyclictest3833066-21sshd09:36:413
3825360995340,9cyclictest3826060-21sshd07:22:403
382534999532,12cyclictest3840946-21true12:10:011
3825344995326,23cyclictest12-21ksoftirqd/010:25:020
3825344995326,23cyclictest12-21ksoftirqd/010:25:010
3825344995315,8cyclictest12-21ksoftirqd/011:40:010
3825344995315,8cyclictest1-21systemd11:05:000
3825360995239,10cyclictest3831260-21sshd09:03:423
382534999523,12cyclictest3835182-21modprobe10:20:011
382534999522,13cyclictest306-21in:imuxsock07:15:021
3825349995213,4cyclictest3838612-21latency_hist11:25:031
382534999521,13cyclictest3831042-21munin-run09:00:011
3825344995211,10cyclictest3829999-21systemd-run08:40:010
3825357995142,7cyclictest30-21ksoftirqd/210:45:002
3825357995132,13cyclictest235-21systemd-journal08:35:012
382534999512,12cyclictest3833631-21systemd-run09:50:011
382534499519,12cyclictest3835160-21cron10:20:000
382534499512,18cyclictest3839648-21latency_hist11:45:000
3825344995116,7cyclictest3833090-21modprobe09:40:000
3825360995043,4cyclictest3828161-21munin-run08:05:013
3825360995037,10cyclictest363-21snmpd07:56:093
382535799506,25cyclictest3831524-21modprobe09:10:002
3825357995031,14cyclictest272-21dbus-daemon11:55:002
3825357995028,16cyclictest1-21systemd11:05:012
382534999504,15cyclictest3842167-21latency_hist12:30:021
3825349995033,13cyclictest1-21systemd07:55:001
3825349995033,13cyclictest1-21systemd07:55:001
3825349995024,23cyclictest3828929-21latency_hist08:20:021
382534999501,14cyclictest3837578-21true11:05:001
382534999501,14cyclictest363-21snmpd12:14:051
3825344995016,8cyclictest12-21ksoftirqd/008:55:010
3825344995016,8cyclictest12-21ksoftirqd/008:55:000
3825349994918,26cyclictest23-21ksoftirqd/110:09:561
3825349994918,26cyclictest23-21ksoftirqd/110:09:551
3825344994929,13cyclictest1-21systemd08:20:000
3825344994920,4cyclictest3825385-21latency_hist07:10:010
3825344994919,9cyclictest12-21ksoftirqd/009:50:010
382534499491,4cyclictest0-21swapper/012:32:590
3825357994828,14cyclictest235-21systemd-journal07:40:002
3825349994812,4cyclictest3830378-21wc08:45:011
382534999481,13cyclictest363-21snmpd09:04:021
382534499483,15cyclictest3826399-21wc07:30:020
3825344994819,10cyclictest12-21ksoftirqd/009:55:020
3825344994819,10cyclictest12-21ksoftirqd/009:55:010
3825357994710,20cyclictest3825340-21cyclictest11:15:012
382534999472,16cyclictest235-21systemd-journal10:00:001
3825349994713,7cyclictest3825513-21sshd07:10:411
3825344994730,13cyclictest235-21systemd-journal09:20:410
3825344994722,5cyclictest0-21swapper/011:05:270
3825344994717,7cyclictest12-21ksoftirqd/007:45:030
3825344994717,5cyclictest3826081-21latency_hist07:25:020
3825357994630,12cyclictest3825842-21systemd-run07:20:002
3825357994629,11cyclictest235-21systemd-journal12:25:012
382534999462,15cyclictest3842236-21munin-plugin-st12:35:001
382534999461,14cyclictest3833135-21unin-run09:40:021
382534499462,4cyclictest3834213-21wc10:00:010
3825344994620,4cyclictest0-21swapper/010:09:450
3825344994620,4cyclictest0-21swapper/010:09:440
3825344994615,9cyclictest12-21ksoftirqd/008:30:020
382534499461,41cyclictest0-21swapper/011:26:070
3825357994535,6cyclictest3829967-21sendmail08:40:002
382535799452,38cyclictest272-21dbus-daemon09:20:022
382534999452,14cyclictest235-21systemd-journal09:14:591
3825344994523,4cyclictest0-21swapper/011:45:310
3825357994432,9cyclictest30-21ksoftirqd/210:45:292
3825357994431,10cyclictest30-21ksoftirqd/207:25:002
3825357994429,11cyclictest272-21dbus-daemon08:50:012
3825357994428,11cyclictest3834663-21cut10:10:002
3825357994428,11cyclictest3834663-21cut10:10:002
3825357994427,13cyclictest1-21systemd09:55:022
3825357994427,13cyclictest1-21systemd09:55:012
3825357994424,15cyclictest272-21dbus-daemon07:15:012
3825357994424,14cyclictest1-21systemd07:50:012
3825349994419,22cyclictest23-21ksoftirqd/107:10:021
382534499442,17cyclictest3827400-21wc07:50:010
382534499441,39cyclictest0-21swapper/012:23:300
3825357994327,10cyclictest306-21in:imuxsock08:00:002
3825344994319,21cyclictest3830239-21latency_hist08:45:010
382535799424,5cyclictest131rcu_preempt11:40:022
382535799424,23cyclictest3838330-21modprobe11:20:002
3825357994221,14cyclictest1-21systemd09:25:012
382534999422,13cyclictest3833861-21cron09:55:001
382534999422,13cyclictest3833861-21cron09:55:001
3825344994223,4cyclictest0-21swapper/010:14:150
382534499421,37cyclictest0-21swapper/010:02:280
3825357994130,6cyclictest271-21cron09:59:592
3825357994126,10cyclictest235-21systemd-journal08:15:002
3825357994123,13cyclictest1-21systemd10:30:002
3825357994123,12cyclictest272-21dbus-daemon10:55:012
3825357994123,12cyclictest272-21dbus-daemon10:55:002
3825357994123,12cyclictest0-21swapper/212:10:012
3825357994118,16cyclictest272-21dbus-daemon09:30:002
3825357994118,15cyclictest235-21systemd-journal09:15:002
382534999411,17cyclictest3838076-21munin-run11:15:001
382534499411,4cyclictest0-21swapper/012:14:160
382535799407,4cyclictest408-21lldpd08:07:432
382535799404,28cyclictest235-21systemd-journal08:55:012
382535799404,28cyclictest235-21systemd-journal08:55:002
382535799403,19cyclictest3828642-21kworker/u8:208:20:012
3825357994022,13cyclictest272-21dbus-daemon10:40:012
3825357994022,13cyclictest272-21dbus-daemon10:40:012
3825357994021,14cyclictest235-21systemd-journal11:25:022
382534999404,5cyclictest131rcu_preempt11:25:011
382534999404,5cyclictest131rcu_preempt11:25:011
3825357993927,9cyclictest272-21dbus-daemon11:25:022
3825357993927,9cyclictest272-21dbus-daemon11:25:012
3825357993926,10cyclictest30-21ksoftirqd/212:31:542
3825357993925,11cyclictest30-21ksoftirqd/208:05:002
382535799391,3cyclictest131rcu_preempt09:01:412
382534999391,5cyclictest131rcu_preempt12:23:421
3825357993824,10cyclictest272-21dbus-daemon10:15:012
3825357993821,11cyclictest306-21in:imuxsock11:05:032
3825357993818,15cyclictest272-21dbus-daemon09:45:012
3825349993826,10cyclictest23-21ksoftirqd/109:35:011
3825344993829,6cyclictest12-21ksoftirqd/008:35:020
3825344993820,3cyclictest0-21swapper/010:50:110
3825344993820,3cyclictest0-21swapper/010:50:110
3825357993727,7cyclictest30-21ksoftirqd/209:34:422
3825357993725,9cyclictest30-21ksoftirqd/210:20:012
382535799372,4cyclictest131rcu_preempt12:40:002
3825357993724,9cyclictest30-21ksoftirqd/208:30:002
3825357993723,10cyclictest272-21dbus-daemon07:35:012
3825357993720,12cyclictest272-21dbus-daemon11:50:012
382534499374,6cyclictest131rcu_preempt10:30:020
382534499371,32cyclictest3838616-21munin-run11:25:010
382534499371,32cyclictest3838616-21munin-run11:25:000
3825357993625,8cyclictest30-21ksoftirqd/208:25:012
3825357993619,12cyclictest235-21systemd-journal09:50:002
3825357993616,15cyclictest272-21dbus-daemon08:45:002
382534999362,4cyclictest131rcu_preempt08:06:411
382535799354,22cyclictest3839113-21modprobe11:35:002
3825357993524,7cyclictest3839837-21sed11:45:012
3825357993522,9cyclictest306-21in:imuxsock07:40:252
3825357993519,12cyclictest272-21dbus-daemon09:00:002
3825357993515,14cyclictest272-21dbus-daemon12:00:012
3825344993523,8cyclictest12-21ksoftirqd/009:30:010
382535799344,3cyclictest131rcu_preempt12:20:022
3825357993424,8cyclictest3834445-21sed10:05:012
3825357993423,8cyclictest30-21ksoftirqd/212:30:002
3825344993422,10cyclictest12-21ksoftirqd/012:25:420
3825357993319,9cyclictest3827767-21grep07:55:022
3825357993319,9cyclictest3827767-21grep07:55:022
3825357993312,15cyclictest272-21dbus-daemon10:25:012
3825357993312,15cyclictest272-21dbus-daemon10:25:012
3825357993224,5cyclictest30-21ksoftirqd/212:05:012
3825357993117,10cyclictest306-21in:imuxsock10:34:422
3825357993021,7cyclictest30-21ksoftirqd/212:15:012
3825357993018,9cyclictest30-21ksoftirqd/207:30:012
3825357992513,9cyclictest3825429-21wc07:10:032
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional