You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-12 - 13:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Wed Nov 12, 2025 00:45:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2521552991313,11cyclictest131rcu_preempt19:25:010
252155299922,13cyclictest131rcu_preempt00:30:030
2521563997854,14cyclictest30-21ksoftirqd/221:25:022
2521557997025,13cyclictest272-21dbus-daemon19:30:011
2521563996826,8cyclictest2521736-21sshd19:14:382
2521557996813,15cyclictest272-21dbus-daemon19:20:001
2521557996425,4cyclictest2532905-21sshd22:45:381
2521557996420,9cyclictest2524875-21cut20:15:011
2521557996331,27cyclictest306-21in:imuxsock00:00:011
2521557996224,8cyclictest306-21in:imuxsock21:10:011
2521557996115,8cyclictest0-21swapper/100:40:001
2521567996041,15cyclictest272-21dbus-daemon00:39:593
2521557995912,10cyclictest235-21systemd-journal23:19:591
2521557995818,9cyclictest23-21ksoftirqd/122:04:181
2521563995645,8cyclictest30-21ksoftirqd/223:55:012
2521557995621,8cyclictest23-21ksoftirqd/100:20:391
2521567995535,15cyclictest408-21lldpd20:40:003
2521567995532,14cyclictest235-21systemd-journal23:05:003
2521567995530,18cyclictest272-21dbus-daemon19:25:003
2521567995514,8cyclictest37-21ksoftirqd/323:20:013
2521567995336,12cyclictest1-21systemd20:55:013
2521567995335,14cyclictest272-21dbus-daemon20:05:003
2521567995312,7cyclictest131rcu_preempt00:00:023
2521567995237,11cyclictest1-21systemd23:40:013
2521567995233,14cyclictest272-21dbus-daemon23:00:003
252155799523,21cyclictest2527213-21munin-run21:00:011
2521567995140,7cyclictest37-21ksoftirqd/319:20:023
2521567995135,12cyclictest2538383-21sshd00:33:403
252156799513,34cyclictest2524875-21cut20:15:003
2521557995132,14cyclictest1-21systemd20:39:591
2521557995131,13cyclictest235-21systemd-journal19:50:001
2521557995126,4cyclictest2524463-21latency_hist20:05:021
252155799511,31cyclictest221rcuc/100:35:021
252155299518,34cyclictest131rcu_preempt21:10:000
2521567995036,7cyclictest306-21in:imuxsock20:30:003
2521567995034,12cyclictest37-21ksoftirqd/321:08:373
2521567995028,15cyclictest272-21dbus-daemon21:05:003
2521552995033,12cyclictest235-21systemd-journal00:15:010
2521563994934,10cyclictest30-21ksoftirqd/221:00:002
252155799494,28cyclictest2531079-21kworker/u8:022:25:011
2521567994834,9cyclictest37-21ksoftirqd/321:29:593
2521567994831,13cyclictest1-21systemd23:10:003
2521567994815,7cyclictest131rcu_preempt21:55:033
2521567994813,6cyclictest131rcu_preempt19:15:003
2521567994811,7cyclictest131rcu_preempt22:55:003
2521567994811,7cyclictest131rcu_preempt20:25:013
2521563994834,10cyclictest235-21systemd-journal23:50:002
2521552994835,9cyclictest1-21systemd00:20:010
252156799471,14cyclictest37-21ksoftirqd/319:45:023
2521563994728,14cyclictest1-21systemd23:25:002
2521563994727,15cyclictest272-21dbus-daemon22:15:002
2521552994739,5cyclictest16950irq/224-800000020:00:010
2521552994731,12cyclictest1-21systemd20:20:000
2521552994730,13cyclictest272-21dbus-daemon23:50:000
2521552994718,4cyclictest2533942-21kworker/u8:200:05:380
252156799469,6cyclictest131rcu_preempt00:25:003
252156799466,7cyclictest131rcu_preempt19:40:003
2521567994638,5cyclictest37-21ksoftirqd/321:35:013
2521563994630,11cyclictest272-21dbus-daemon20:10:002
2521557994610,9cyclictest0-21swapper/120:50:021
2521552994628,14cyclictest2530587-21sshd22:03:140
252155299462,38cyclictest272-21dbus-daemon20:45:000
252156799455,8cyclictest37-21ksoftirqd/322:50:013
2521567994525,14cyclictest2524076-21cron20:00:013
2521563994529,12cyclictest30-21ksoftirqd/220:25:052
2521552994533,9cyclictest2531134-21cat22:15:010
2521552994532,10cyclictest235-21systemd-journal23:28:380
2521552994532,10cyclictest235-21systemd-journal23:28:370
2521567994431,9cyclictest235-21systemd-journal22:25:033
2521567994431,9cyclictest235-21systemd-journal22:25:023
2521567994428,12cyclictest37-21ksoftirqd/323:50:013
2521567994426,12cyclictest272-21dbus-daemon00:30:003
2521567994410,4cyclictest131rcu_preempt20:20:023
252156399443,5cyclictest131rcu_preempt23:40:012
2521563994430,10cyclictest272-21dbus-daemon19:30:012
252155799443,11cyclictest2521560-21latency_hist19:10:011
2521557994429,10cyclictest1-21systemd20:10:001
2521552994428,13cyclictest2528777-21sshd21:29:380
2521552994427,13cyclictest2535412-21latency_hist23:35:010
2521567994331,9cyclictest37-21ksoftirqd/323:30:013
2521567994331,9cyclictest37-21ksoftirqd/323:30:003
2521567994329,7cyclictest2529286-21cron21:40:013
2521567994329,7cyclictest2529286-21cron21:40:003
2521567994328,11cyclictest1-21systemd20:50:003
252156799431,4cyclictest131rcu_preempt19:35:013
2521567994312,6cyclictest131rcu_preempt00:09:593
2521567994310,6cyclictest131rcu_preempt23:25:013
2521563994328,9cyclictest30-21ksoftirqd/220:50:022
2521563994325,13cyclictest235-21systemd-journal00:03:382
252156399431,3cyclictest131rcu_preempt22:50:012
2521557994329,10cyclictest235-21systemd-journal22:41:381
2521557994329,10cyclictest235-21systemd-journal22:41:371
2521552994336,4cyclictest16950irq/224-800000020:25:000
2521552994336,4cyclictest16950irq/224-800000019:45:020
2521552994333,7cyclictest2526237-21latency_hist20:40:010
2521552994327,12cyclictest235-21systemd-journal23:17:370
2521552994325,12cyclictest235-21systemd-journal20:55:000
252156799425,5cyclictest131rcu_preempt22:40:003
2521567994233,6cyclictest37-21ksoftirqd/322:40:373
2521567994233,6cyclictest37-21ksoftirqd/322:40:373
2521567994232,7cyclictest37-21ksoftirqd/322:15:003
2521567994227,12cyclictest2523821-21cron19:55:003
252156799422,6cyclictest131rcu_preempt23:35:003
2521567994224,12cyclictest1-21systemd19:30:013
2521567994222,15cyclictest272-21dbus-daemon23:15:013
2521563994229,9cyclictest235-21systemd-journal21:05:012
2521563994228,9cyclictest235-21systemd-journal23:04:592
2521563994227,11cyclictest235-21systemd-journal00:13:372
252155799427,10cyclictest131rcu_preempt21:40:011
252155799427,10cyclictest131rcu_preempt21:40:011
2521557994233,5cyclictest1-21systemd21:15:001
2521552994230,8cyclictest2527230-21sendmail-msp21:00:010
2521552994225,13cyclictest1-21systemd22:30:020
2521552994225,13cyclictest1-21systemd22:30:010
2521567994129,9cyclictest2523577-21cron19:50:003
2521567994128,8cyclictest2528263-21sendmail21:20:003
2521567994110,5cyclictest131rcu_preempt00:15:013
252156399413,5cyclictest131rcu_preempt22:40:002
2521563994131,6cyclictest2538656-21cron00:40:002
2521563994129,8cyclictest30-21ksoftirqd/223:20:002
2521563994126,12cyclictest301ksoftirqd/221:14:592
2521563994124,12cyclictest2536575-21munin-run00:00:012
252155799418,5cyclictest131rcu_preempt23:30:021
252155799418,5cyclictest131rcu_preempt23:30:021
252155799414,31cyclictest306-21in:imuxsock20:55:021
2521557994133,4cyclictest2529815-21cron21:50:011
2521557994127,10cyclictest235-21systemd-journal23:46:381
2521557994122,13cyclictest306-21in:imuxsock20:25:001
2521552994130,8cyclictest235-21systemd-journal19:55:000
2521552994129,9cyclictest235-21systemd-journal21:17:000
2521552994123,13cyclictest235-21systemd-journal21:40:000
2521552994123,13cyclictest235-21systemd-journal21:40:000
252155299412,14cyclictest1-21systemd21:55:020
252156799404,6cyclictest131rcu_preempt21:00:033
2521567994030,7cyclictest37-21ksoftirqd/322:05:013
2521567994027,11cyclictest37-21ksoftirqd/321:50:003
252156799402,5cyclictest131rcu_preempt23:55:013
2521567994012,5cyclictest131rcu_preempt22:20:013
252156399404,5cyclictest131rcu_preempt21:10:002
2521563994030,7cyclictest2529265-21sshd21:36:392
2521563994030,7cyclictest2529265-21sshd21:36:382
2521563994030,6cyclictest306-21in:imuxsock21:45:012
2521563994026,11cyclictest30-21ksoftirqd/219:55:012
2521563994025,10cyclictest272-21dbus-daemon22:00:012
2521563994024,11cyclictest272-21dbus-daemon00:20:022
2521557994031,7cyclictest23-21ksoftirqd/100:05:001
2521557994030,5cyclictest2532413-21sendmail22:40:011
2521557994029,9cyclictest23-21ksoftirqd/100:10:011
2521557994028,8cyclictest306-21in:imuxsock22:06:371
2521557994026,10cyclictest272-21dbus-daemon21:30:011
2521557994010,14cyclictest308-21rs:main1
2521552994031,5cyclictest306-21in:imuxsock23:10:000
2521552994026,11cyclictest2533445-21cron23:00:000
2521552994024,12cyclictest235-21systemd-journal23:25:000
252155299402,34cyclictest2528741-21sed21:25:020
2521552994023,12cyclictest272-21dbus-daemon19:40:000
2521552994020,16cyclictest272-21dbus-daemon22:00:000
2521567993929,7cyclictest37-21ksoftirqd/322:35:013
2521567993928,8cyclictest37-21ksoftirqd/300:05:023
2521567993920,13cyclictest1-21systemd20:45:003
2521563993929,7cyclictest2534994-21sshd23:27:382
2521563993929,7cyclictest2534994-21sshd23:27:372
2521563993924,11cyclictest30-21ksoftirqd/223:10:012
2521563993922,13cyclictest30-21ksoftirqd/220:35:012
2521563993922,13cyclictest30-21ksoftirqd/220:35:012
2521563993920,13cyclictest235-21systemd-journal20:20:002
252156399391,3cyclictest131rcu_preempt20:15:082
252155799394,3cyclictest0-21swapper/121:25:011
252155799394,3cyclictest0-21swapper/121:25:011
2521557993931,5cyclictest306-21in:imuxsock20:00:001
2521557993927,8cyclictest306-21in:imuxsock21:50:021
2521552993932,4cyclictest16950irq/224-800000022:20:010
2521552993932,4cyclictest16950irq/224-800000000:00:020
2521552993928,8cyclictest2535512-21cron23:40:010
2521552993926,10cyclictest1-21systemd22:50:010
2521552993922,13cyclictest235-21systemd-journal21:35:010
2521552993920,14cyclictest2524335-21sshd20:01:380
252155299391,34cyclictest0-21swapper/019:16:390
2521567993830,4cyclictest2531652-21munin-plugin-st22:25:003
2521567993828,7cyclictest37-21ksoftirqd/322:00:003
2521567993825,8cyclictest2528527-21cron21:25:013
2521567993825,8cyclictest2528527-21cron21:25:003
2521567993825,10cyclictest37-21ksoftirqd/300:20:003
2521567993821,11cyclictest1-21systemd21:45:003
252156799381,11cyclictest37-21ksoftirqd/320:09:373
252156399386,7cyclictest131rcu_preempt22:35:002
252156399382,9cyclictest2530080-21munin-run21:55:002
2521563993828,7cyclictest2533459-21modprobe23:00:002
2521563993826,8cyclictest30-21ksoftirqd/221:20:022
2521563993825,8cyclictest30-21ksoftirqd/222:10:022
252155799383,8cyclictest23-21ksoftirqd/123:35:001
252155799383,4cyclictest131rcu_preempt20:35:031
252155799383,4cyclictest131rcu_preempt20:35:021
252155799383,3cyclictest131rcu_preempt23:45:031
252155799383,3cyclictest131rcu_preempt23:45:021
2521557993827,9cyclictest23-21ksoftirqd/122:20:021
252155799382,4cyclictest131rcu_preempt23:55:011
2521557993820,12cyclictest272-21dbus-daemon23:00:021
2521552993832,4cyclictest16950irq/224-800000019:35:000
2521552993831,4cyclictest16950irq/224-800000022:52:370
2521552993829,6cyclictest19050irq/34-mmc121:23:430
2521552993829,5cyclictest2523590-21sh19:50:010
2521552993826,8cyclictest12-21ksoftirqd/022:35:010
2521567993727,8cyclictest37-21ksoftirqd/322:10:023
2521567993724,8cyclictest37-21ksoftirqd/321:15:003
2521567993721,12cyclictest2535794-21cron23:45:013
2521567993721,12cyclictest2535794-21cron23:45:003
252156399373,3cyclictest131rcu_preempt23:15:022
2521563993728,5cyclictest2531378-21sendmail22:19:592
2521563993724,6cyclictest306-21in:imuxsock19:40:002
2521563993723,9cyclictest306-21in:imuxsock19:42:422
252156399371,3cyclictest131rcu_preempt00:30:022
2521563993712,5cyclictest131rcu_preempt21:25:012
2521563993712,5cyclictest131rcu_preempt21:25:002
2521557993728,6cyclictest2538116-21sshd00:26:371
2521557993724,10cyclictest23-21ksoftirqd/119:45:001
252155799372,3cyclictest131rcu_preempt23:10:011
2521557993723,11cyclictest2526607-21sed20:45:011
252155799372,12cyclictest235-21systemd-journal22:55:001
2521557993719,14cyclictest2535497-21sshd23:36:381
2521552993732,3cyclictest0-21swapper/019:15:000
2521552993727,6cyclictest306-21in:imuxsock00:05:010
2521552993725,8cyclictest1-21systemd23:55:000
2521552993723,10cyclictest2532671-21sshd22:44:380
2521552993723,10cyclictest2532671-21sshd22:44:380
2521552993721,12cyclictest272-21dbus-daemon21:15:000
252155299371,4cyclictest2525661-21latency_hist20:30:020
252156799368,5cyclictest131rcu_preempt20:35:023
252156799368,5cyclictest131rcu_preempt20:35:013
2521563993629,4cyclictest301ksoftirqd/222:05:002
2521563993625,5cyclictest30-21ksoftirqd/221:35:002
2521563993624,8cyclictest30-21ksoftirqd/200:10:012
2521563993624,8cyclictest2524100-21systemctl20:00:002
2521563993620,12cyclictest30-21ksoftirqd/220:40:002
252156399361,9cyclictest2526446-21modprobe20:44:592
252155799364,5cyclictest131rcu_preempt21:05:011
252155799363,3cyclictest131rcu_preempt19:25:011
2521557993625,8cyclictest2525647-21sshd20:29:381
2521557993622,10cyclictest1-21systemd23:05:001
2521552993627,6cyclictest2525046-21latency_hist20:15:080
2521552993626,7cyclictest2538653-21sshd00:39:380
2521552993626,7cyclictest12-21ksoftirqd/019:10:010
2521552993624,8cyclictest306-21in:imuxsock00:29:380
2521552993622,10cyclictest12-21ksoftirqd/021:05:000
252156399354,3cyclictest131rcu_preempt19:50:032
2521563993527,5cyclictest2535804-21latency_hist23:45:022
2521563993527,5cyclictest2535804-21latency_hist23:45:022
2521563993516,14cyclictest1-21systemd20:05:002
252155799353,10cyclictest23-21ksoftirqd/121:56:451
2521557993528,4cyclictest2528226-21sh21:17:011
2521557993526,6cyclictest23-21ksoftirqd/119:15:031
2521557993521,9cyclictest306-21in:imuxsock00:15:001
2521557993518,11cyclictest272-21dbus-daemon22:15:011
2521552993529,4cyclictest0-21swapper/000:22:180
2521552993525,7cyclictest2535772-21sshd23:40:380
2521552993525,7cyclictest2535772-21sshd23:40:370
2521552993524,8cyclictest235-21systemd-journal19:27:560
2521552993524,8cyclictest12-21ksoftirqd/021:45:010
252155299351,5cyclictest2532419-21munin-run22:40:000
252155299351,31cyclictest0-21swapper/023:05:020
252155299351,30cyclictest2525956-21latency_hist20:35:020
252155299351,30cyclictest2525956-21latency_hist20:35:010
252155299351,30cyclictest0-21swapper/023:15:030
252155299351,30cyclictest0-21swapper/022:20:490
252156399348,5cyclictest131rcu_preempt00:34:382
2521563993423,8cyclictest2531705-21sed22:25:022
2521563993423,7cyclictest30-21ksoftirqd/219:20:222
2521563993422,8cyclictest306-21in:imuxsock22:55:022
2521563993421,9cyclictest30-21ksoftirqd/222:30:012
2521563993421,9cyclictest30-21ksoftirqd/222:30:012
2521557993425,5cyclictest306-21in:imuxsock19:35:001
2521557993423,9cyclictest23-21ksoftirqd/123:15:021
2521557993422,9cyclictest23-21ksoftirqd/122:35:011
252155299341,30cyclictest0-21swapper/022:05:420
2521563993324,5cyclictest30-21ksoftirqd/220:55:002
2521563993323,8cyclictest30-21ksoftirqd/200:25:022
2521563993322,8cyclictest30-21ksoftirqd/219:35:002
2521563993320,10cyclictest2529840-21systemd-run21:50:002
2521563993319,7cyclictest306-21in:imuxsock20:30:012
252156399331,28cyclictest1-21systemd23:35:002
2521557993326,4cyclictest2529574-21latency_hist21:45:011
2521557993320,10cyclictest272-21dbus-daemon20:20:001
2521552993329,2cyclictest0-21swapper/021:46:380
2521552993327,4cyclictest0-21swapper/020:06:220
2521567993225,4cyclictest2521667-21latency_hist19:10:023
2521563993222,8cyclictest30-21ksoftirqd/219:17:382
252156399321,6cyclictest131rcu_preempt22:45:022
252156399321,6cyclictest131rcu_preempt22:45:012
2521557993224,5cyclictest2537638-21sendmail00:20:011
2521557993219,10cyclictest2523882-21cat19:55:021
2521552993222,8cyclictest12-21ksoftirqd/020:50:010
2521563993023,5cyclictest30-21ksoftirqd/219:10:022
252155799302,4cyclictest131rcu_preempt23:25:011
2521557992920,7cyclictest23-21ksoftirqd/122:30:021
2521557992920,7cyclictest23-21ksoftirqd/122:30:011
252155799283,4cyclictest131rcu_preempt21:35:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional