You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-20 - 06:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Sat Dec 20, 2025 00:45:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
9059819911697,16cyclictest30-21ksoftirqd/222:00:022
905981999992,5cyclictest30-21ksoftirqd/222:50:012
905971999238,49cyclictest12-21ksoftirqd/023:45:010
90597199822,6cyclictest131rcu_preempt19:45:010
90597199803,7cyclictest131rcu_preempt22:05:020
905971997928,4cyclictest16950irq/224-800000023:55:160
905971997424,13cyclictest275-21dbus-daemon22:00:010
905971997122,10cyclictest909548-21sendmail20:19:590
905971996929,4cyclictest16950irq/224-800000020:25:010
905985996629,33cyclictest37-21ksoftirqd/321:50:013
905971996421,15cyclictest906933-21sshd19:26:150
905971996231,3cyclictest0-21swapper/021:40:010
90597199621,12cyclictest920627-21modprobe23:51:160
90597199621,12cyclictest920627-21modprobe23:51:150
90598199617,11cyclictest30-21ksoftirqd/220:05:012
905971996126,6cyclictest12-21ksoftirqd/020:05:020
905981995918,10cyclictest30-21ksoftirqd/220:45:012
905981995912,12cyclictest30-21ksoftirqd/223:10:022
90598199581,15cyclictest30-21ksoftirqd/223:35:012
905971995825,4cyclictest16950irq/224-800000023:40:000
90598199576,7cyclictest30-21ksoftirqd/220:20:002
905981995714,11cyclictest30-21ksoftirqd/220:40:002
905981995710,11cyclictest30-21ksoftirqd/221:20:002
905985995635,15cyclictest275-21dbus-daemon19:40:003
90598199568,12cyclictest30-21ksoftirqd/222:20:032
90598199568,12cyclictest30-21ksoftirqd/222:20:022
90598199563,14cyclictest30-21ksoftirqd/222:15:012
90598199563,13cyclictest30-21ksoftirqd/221:15:022
90598199546,9cyclictest30-21ksoftirqd/221:35:022
90598199546,9cyclictest30-21ksoftirqd/219:35:022
90598199546,10cyclictest30-21ksoftirqd/219:30:002
90598199544,14cyclictest30-21ksoftirqd/223:40:002
90598199544,12cyclictest30-21ksoftirqd/219:25:022
90598199543,11cyclictest30-21ksoftirqd/223:30:022
905971995437,13cyclictest1-21systemd20:00:020
905971995425,18cyclictest905975-21latency_hist19:10:020
90598199536,8cyclictest30-21ksoftirqd/200:35:022
90598199534,9cyclictest30-21ksoftirqd/223:00:012
905981995314,11cyclictest30-21ksoftirqd/223:05:012
905971995324,8cyclictest12-21ksoftirqd/022:40:010
90598199528,12cyclictest30-21ksoftirqd/219:15:032
90598199527,11cyclictest30-21ksoftirqd/220:30:022
90598199525,9cyclictest30-21ksoftirqd/223:15:012
905971995245,5cyclictest12-21ksoftirqd/022:45:030
905971995245,5cyclictest12-21ksoftirqd/022:45:020
90597199521,4cyclictest131rcu_preempt21:30:000
90598199519,10cyclictest30-21ksoftirqd/222:35:022
90598199516,10cyclictest30-21ksoftirqd/200:00:022
90598199515,8cyclictest30-21ksoftirqd/219:50:002
90598199515,10cyclictest30-21ksoftirqd/222:45:032
90598199515,10cyclictest30-21ksoftirqd/200:10:012
90598199512,13cyclictest30-21ksoftirqd/223:45:022
905977995135,11cyclictest1-21systemd20:30:001
905971995128,19cyclictest912653-21systemctl21:20:010
90598199507,10cyclictest30-21ksoftirqd/222:20:012
90598199506,8cyclictest30-21ksoftirqd/200:30:022
90598199504,14cyclictest30-21ksoftirqd/222:55:022
90598199504,11cyclictest30-21ksoftirqd/220:10:022
90598199503,12cyclictest30-21ksoftirqd/220:15:032
90598199503,12cyclictest30-21ksoftirqd/220:15:022
90598199503,10cyclictest30-21ksoftirqd/222:45:012
90598199502,9cyclictest922266-21latency_hist00:25:022
90598199502,9cyclictest30-21ksoftirqd/220:35:002
90598599499,6cyclictest131rcu_preempt20:10:003
90598599494,3cyclictest131rcu_preempt22:10:023
90598599494,3cyclictest131rcu_preempt22:10:023
905985994931,13cyclictest921941-21cron00:20:013
90598199496,7cyclictest30-21ksoftirqd/221:25:022
90598199496,5cyclictest30-21ksoftirqd/223:55:022
90598199496,5cyclictest30-21ksoftirqd/223:55:012
90598199495,12cyclictest30-21ksoftirqd/222:40:002
90598199492,11cyclictest30-21ksoftirqd/221:10:022
90598199491,12cyclictest30-21ksoftirqd/219:20:002
90598199491,12cyclictest30-21ksoftirqd/219:20:002
90598199491,10cyclictest30-21ksoftirqd/223:25:012
90598199489,7cyclictest30-21ksoftirqd/220:00:012
90598199487,10cyclictest30-21ksoftirqd/219:40:032
90598199486,7cyclictest30-21ksoftirqd/223:50:022
90598199486,10cyclictest30-21ksoftirqd/222:00:012
90598199485,7cyclictest30-21ksoftirqd/221:55:002
90598199484,9cyclictest30-21ksoftirqd/222:10:032
90598199484,9cyclictest30-21ksoftirqd/222:10:022
90598199481,8cyclictest911868-21munin-run21:05:002
905981994815,11cyclictest30-21ksoftirqd/221:40:002
90598199481,11cyclictest30-21ksoftirqd/200:20:002
905981994810,9cyclictest30-21ksoftirqd/200:15:022
905977994837,9cyclictest23-21ksoftirqd/123:15:011
90597199483,7cyclictest912104-21cron21:10:010
90598599476,9cyclictest37-21ksoftirqd/300:14:153
90598599471,36cyclictest915509-21latency_hist22:15:003
90598199475,11cyclictest30-21ksoftirqd/220:50:012
90598199471,6cyclictest918876-21modprobe23:20:002
90597799474,4cyclictest131rcu_preempt00:00:011
905977994732,11cyclictest907467-21run-parts19:40:001
905971994739,4cyclictest16950irq/224-800000020:35:000
905971994727,16cyclictest909251-21sshd20:13:160
905971994727,16cyclictest909251-21sshd20:13:150
90598199466,6cyclictest30-21ksoftirqd/222:30:012
90598199464,11cyclictest30-21ksoftirqd/221:00:022
90598199463,10cyclictest30-21ksoftirqd/220:25:002
90597799462,3cyclictest131rcu_preempt21:20:011
90598599453,3cyclictest131rcu_preempt22:17:003
905985994529,12cyclictest238-21systemd-journal19:55:033
90598599452,3cyclictest131rcu_preempt21:45:033
90598199458,8cyclictest30-21ksoftirqd/220:55:022
90598199456,10cyclictest30-21ksoftirqd/200:40:012
90598199456,10cyclictest30-21ksoftirqd/200:40:012
90598199453,6cyclictest30-21ksoftirqd/221:50:022
90598199453,12cyclictest30-21ksoftirqd/219:10:012
905977994529,11cyclictest1-21systemd21:10:011
905977994527,13cyclictest1-21systemd21:35:001
90597799451,3cyclictest131rcu_preempt00:10:001
905985994435,5cyclictest909531-21latency_hist20:19:593
90598199442,13cyclictest30-21ksoftirqd/200:05:012
90598199442,13cyclictest30-21ksoftirqd/200:05:002
90598199441,7cyclictest908350-21date19:55:022
90597799444,3cyclictest131rcu_preempt23:55:031
90597799444,3cyclictest131rcu_preempt23:55:031
905977994430,10cyclictest23-21ksoftirqd/119:45:001
905977994430,10cyclictest1-21systemd21:15:011
905977994428,12cyclictest23-21ksoftirqd/119:55:001
905977994428,11cyclictest275-21dbus-daemon23:20:011
905971994434,7cyclictest907208-21munin-plugin-st19:35:010
905971994425,15cyclictest1-21systemd22:50:000
905985994327,12cyclictest238-21systemd-journal21:10:013
905985994323,15cyclictest238-21systemd-journal21:34:593
905985994316,10cyclictest37-21ksoftirqd/300:00:023
90598199433,8cyclictest30-21ksoftirqd/221:40:172
905981994334,7cyclictest30-21ksoftirqd/221:25:022
905981994334,7cyclictest30-21ksoftirqd/221:25:012
905977994332,6cyclictest906955-21munin-plugin-st19:30:001
905977994327,11cyclictest911598-21systemctl21:00:001
905977994322,15cyclictest275-21dbus-daemon00:30:011
90597799431,3cyclictest131rcu_preempt00:40:021
90597799431,3cyclictest131rcu_preempt00:40:021
905971994333,6cyclictest914224-21latency_hist21:50:010
905971994330,8cyclictest238-21systemd-journal23:45:020
90597199431,21cyclictest131rcu_preempt22:35:010
905985994232,7cyclictest914684-21sshd21:58:163
905977994227,10cyclictest1-21systemd22:35:001
905977994226,12cyclictest275-21dbus-daemon19:50:001
905971994229,10cyclictest914467-21systemd-run21:55:010
905971994225,13cyclictest922468-21sshd00:29:150
905971994225,12cyclictest275-21dbus-daemon23:05:010
905971994224,14cyclictest410-21lldpd23:33:160
90597199422,35cyclictest0-21swapper/021:35:000
905971994221,16cyclictest275-21dbus-daemon20:10:000
905985994128,8cyclictest37-21ksoftirqd/322:25:013
905985994128,8cyclictest37-21ksoftirqd/322:25:013
905985994126,11cyclictest37-21ksoftirqd/323:40:003
905985994126,11cyclictest275-21dbus-daemon20:05:013
905985994124,12cyclictest309-21in:imuxsock21:20:023
905985994124,12cyclictest309-21in:imuxsock21:20:023
905985994124,12cyclictest238-21systemd-journal20:30:003
90597199412,35cyclictest911597-21munin-run21:00:000
905971994122,15cyclictest915701-21sshd22:15:170
905985994031,6cyclictest37-21ksoftirqd/323:10:023
90598599402,4cyclictest131rcu_preempt22:30:003
905985994024,13cyclictest275-21dbus-daemon20:50:013
90598599402,3cyclictest131rcu_preempt00:40:013
90598599402,3cyclictest131rcu_preempt00:40:003
905985994021,14cyclictest275-21dbus-daemon21:20:013
90598599401,4cyclictest131rcu_preempt22:34:593
90598599401,3cyclictest131rcu_preempt21:55:013
90598199409,4cyclictest131rcu_preempt19:45:002
905977994028,7cyclictest908479-21cron20:00:011
905977994024,8cyclictest309-21in:imuxsock21:50:001
90597799402,3cyclictest131rcu_preempt23:45:001
905971994030,5cyclictest309-21in:imuxsock20:40:010
905971994029,6cyclictest309-21in:imuxsock19:15:000
90597199402,5cyclictest922081-21grep00:20:020
905971994024,13cyclictest913923-21sshd21:41:170
905971994023,10cyclictest309-21in:imuxsock23:20:020
905971994018,18cyclictest275-21dbus-daemon20:30:000
905971994012,25cyclictest12-21ksoftirqd/022:10:020
905971994012,25cyclictest12-21ksoftirqd/022:10:020
90598599393,4cyclictest131rcu_preempt23:55:013
90598599393,4cyclictest131rcu_preempt23:55:003
905985993931,6cyclictest37-21ksoftirqd/323:15:013
90598599392,4cyclictest131rcu_preempt21:14:593
905985993921,14cyclictest1-21systemd19:45:003
90598599391,3cyclictest131rcu_preempt20:55:003
90597799393,3cyclictest131rcu_preempt21:25:031
90597799393,3cyclictest131rcu_preempt21:25:021
905977993930,7cyclictest23-21ksoftirqd/100:35:011
905977993927,10cyclictest23-21ksoftirqd/120:15:011
905977993927,10cyclictest23-21ksoftirqd/120:15:001
905977993921,13cyclictest275-21dbus-daemon20:40:001
905977993919,15cyclictest275-21dbus-daemon22:50:011
905977993911,20cyclictest915741-21modprobe22:20:001
90597199394,31cyclictest0-21swapper/021:25:010
90597199394,31cyclictest0-21swapper/021:25:010
905971993930,7cyclictest12-21ksoftirqd/020:50:020
905971993927,10cyclictest12-21ksoftirqd/000:05:010
905971993927,10cyclictest12-21ksoftirqd/000:05:000
905971993926,10cyclictest915991-21sshd22:21:160
905971993926,10cyclictest915991-21sshd22:21:150
905971993925,11cyclictest923011-21systemctl00:40:010
905971993925,11cyclictest923011-21systemctl00:40:000
905971993924,10cyclictest275-21dbus-daemon23:00:000
90597199391,4cyclictest131rcu_preempt22:55:020
90597199391,3cyclictest131rcu_preempt23:25:010
90597199391,35cyclictest906693-21cron19:24:590
905985993831,4cyclictest905108-21kworker/u8:119:20:013
905985993831,4cyclictest905108-21kworker/u8:119:20:003
905985993828,7cyclictest916754-21sshd22:38:163
905985993824,8cyclictest309-21in:imuxsock20:15:013
905985993824,8cyclictest309-21in:imuxsock20:15:003
905985993824,11cyclictest275-21dbus-daemon20:45:003
905985993822,12cyclictest37-21ksoftirqd/300:25:013
905985993820,13cyclictest275-21dbus-daemon22:05:003
90597799383,3cyclictest131rcu_preempt22:00:011
905977993829,5cyclictest918074-21cron23:05:001
905977993828,6cyclictest309-21in:imuxsock00:17:001
905977993827,6cyclictest309-21in:imuxsock23:39:591
905977993826,7cyclictest309-21in:imuxsock20:35:011
905977993824,9cyclictest309-21in:imuxsock22:55:001
905977993823,11cyclictest275-21dbus-daemon23:25:001
905977993823,11cyclictest0-21swapper/120:45:001
905977993821,11cyclictest275-21dbus-daemon21:40:011
905977993818,16cyclictest275-21dbus-daemon23:00:011
90597799381,3cyclictest131rcu_preempt22:15:021
90597199386,9cyclictest12-21ksoftirqd/019:16:160
90597199386,9cyclictest12-21ksoftirqd/019:16:150
90597199383,9cyclictest12-21ksoftirqd/021:11:150
905971993827,8cyclictest12-21ksoftirqd/023:30:010
905971993823,11cyclictest238-21systemd-journal00:10:010
905985993726,8cyclictest920458-21sed23:50:023
905985993725,10cyclictest37-21ksoftirqd/320:59:593
90597799379,5cyclictest131rcu_preempt20:20:011
90597799374,3cyclictest131rcu_preempt20:50:021
90597799373,3cyclictest131rcu_preempt22:05:011
905977993724,9cyclictest23-21ksoftirqd/122:30:011
905977993720,12cyclictest238-21systemd-journal21:45:001
905971993728,5cyclictest907449-21cron19:40:000
90597199372,7cyclictest918337-21munin-run23:10:000
905971993727,7cyclictest16950irq/224-800000000:21:160
905971993713,3cyclictest131rcu_preempt22:15:010
90598599368,6cyclictest131rcu_preempt22:55:013
90598599365,6cyclictest131rcu_preempt22:40:173
90598599365,6cyclictest131rcu_preempt22:40:163
90598599362,4cyclictest131rcu_preempt19:15:013
905985993624,9cyclictest37-21ksoftirqd/320:25:013
905985993623,11cyclictest37-21ksoftirqd/321:40:013
905985993623,10cyclictest919717-21sed23:35:023
90598599362,11cyclictest131rcu_preempt20:40:003
905985993619,13cyclictest1-21systemd00:35:013
905977993625,8cyclictest0-21swapper/123:50:011
905977993621,10cyclictest309-21in:imuxsock22:40:011
905977993614,4cyclictest23-21ksoftirqd/119:22:161
90597799361,3cyclictest131rcu_preempt20:55:011
90597799361,3cyclictest131rcu_preempt20:10:011
905977993610,20cyclictest919368-21modprobe23:30:001
905971993630,3cyclictest16950irq/224-800000000:10:020
90597199362,5cyclictest908002-21munin-run19:50:010
905971993625,8cyclictest238-21systemd-journal00:34:220
90597199362,3cyclictest131rcu_preempt21:05:030
905971993621,10cyclictest309-21in:imuxsock20:55:010
90598599354,3cyclictest131rcu_preempt19:25:013
90598599353,4cyclictest907202-21modprobe19:35:013
90598599351,7cyclictest0-21swapper/323:29:593
90597799355,5cyclictest131rcu_preempt22:10:011
90597799355,5cyclictest131rcu_preempt22:10:001
905977993526,5cyclictest913158-21cron21:29:591
90597799351,3cyclictest131rcu_preempt19:15:011
90598599343,9cyclictest915798-21kworker/3:223:00:013
90598599343,4cyclictest131rcu_preempt20:35:013
90598599342,7cyclictest0-21swapper/319:55:013
905985993421,10cyclictest1-21systemd23:05:013
90598599341,9cyclictest907951-21modprobe19:45:033
90598599341,3cyclictest131rcu_preempt21:05:023
905985993410,8cyclictest37-21ksoftirqd/300:10:013
90597799345,20cyclictest921696-21munin-run00:15:001
90597799344,23cyclictest1-21systemd22:45:001
90597799344,23cyclictest1-21systemd22:45:001
90597799342,5cyclictest131rcu_preempt00:25:001
905977993425,5cyclictest906646-21latency_hist19:20:031
905977993425,5cyclictest906646-21latency_hist19:20:021
905977993422,8cyclictest911836-21sshd21:01:161
905977993417,12cyclictest0-21swapper/123:09:591
90597199342,7cyclictest918602-21munin-run23:15:000
90597199341,4cyclictest131rcu_preempt22:27:170
905985993324,6cyclictest906121-21latency_hist19:10:023
905985993324,5cyclictest274-21cron23:20:013
905985993323,7cyclictest919178-21sed23:25:023
90598599331,7cyclictest0-21swapper/319:29:593
90597799333,20cyclictest916023-21munin-run22:25:011
90597799333,20cyclictest916023-21munin-run22:25:001
905977993321,9cyclictest921167-21sshd00:02:171
905977993321,9cyclictest921167-21sshd00:02:171
905977993319,10cyclictest23-21ksoftirqd/121:55:031
90597199331,4cyclictest131rcu_preempt20:45:010
90598599323,3cyclictest131rcu_preempt23:45:013
90598599322,8cyclictest0-21swapper/300:30:013
905985993226,4cyclictest921192-21latency_hist00:05:023
905985993226,4cyclictest921192-21latency_hist00:05:013
90598599321,8cyclictest913231-21grep21:30:023
90597799324,4cyclictest131rcu_preempt20:25:001
90597799323,6cyclictest131rcu_preempt23:34:161
90597799314,3cyclictest131rcu_preempt19:35:001
90597799313,3cyclictest131rcu_preempt20:04:591
905971993123,5cyclictest908248-21sh19:55:000
90598599302,4cyclictest131rcu_preempt22:45:033
905977993021,5cyclictest23-21ksoftirqd/119:10:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional