You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-05 - 20:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot3s.osadl.org (updated Fri Dec 05, 2025 12:45:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1238225998379,2cyclictest0-21swapper/008:55:010
1238234997223,12cyclictest1-21systemd11:15:002
123823499693,4cyclictest131rcu_preempt07:20:022
123823499693,4cyclictest131rcu_preempt07:20:022
1238234996421,11cyclictest235-21systemd-journal10:35:022
1238234996325,11cyclictest30-21ksoftirqd/210:50:012
1238230996316,9cyclictest23-21ksoftirqd/111:10:031
1238230996316,9cyclictest23-21ksoftirqd/111:10:021
1238225996138,18cyclictest235-21systemd-journal10:15:000
1238230996016,8cyclictest23-21ksoftirqd/110:40:021
1238234995924,10cyclictest30-21ksoftirqd/212:05:012
123822599591,3cyclictest0-21swapper/009:45:020
1238230995749,6cyclictest23-21ksoftirqd/110:04:081
1238230995710,13cyclictest1238780-21cut07:20:021
1238230995710,13cyclictest1238780-21cut07:20:021
123822599561,3cyclictest0-21swapper/010:00:010
1238230995514,9cyclictest23-21ksoftirqd/112:25:011
123822599558,6cyclictest12-21ksoftirqd/009:35:020
1238225995512,21cyclictest19050irq/34-mmc110:30:020
123823699548,29cyclictest1255358-21modprobe12:40:003
1238230995413,10cyclictest23-21ksoftirqd/108:00:011
1238230995412,8cyclictest23-21ksoftirqd/111:50:021
1238225995413,36cyclictest12-21ksoftirqd/007:45:020
123823699533,3cyclictest131rcu_preempt09:30:023
1238230995313,9cyclictest23-21ksoftirqd/112:10:031
1238236995232,15cyclictest1-21systemd11:50:013
1238230995214,9cyclictest23-21ksoftirqd/110:35:011
1238236995134,5cyclictest131rcu_preempt08:15:013
1238230995136,10cyclictest1-21systemd07:50:011
1238230995113,7cyclictest23-21ksoftirqd/109:55:031
1238225995117,29cyclictest1245416-21sshd09:26:290
1238225995035,11cyclictest235-21systemd-journal11:25:010
1238225995035,11cyclictest235-21systemd-journal11:25:010
1238225995032,12cyclictest1242836-21systemctl08:40:010
123823699493,29cyclictest1238668-21modprobe07:20:003
123823699493,29cyclictest1238668-21modprobe07:20:003
1238236994930,16cyclictest1253285-21systemctl12:00:003
1238234994932,11cyclictest306-21in:imuxsock11:20:002
1238230994916,24cyclictest1238218-21cyclictest10:20:021
1238230994911,8cyclictest23-21ksoftirqd/111:30:031
1238225994932,13cyclictest1-21systemd11:00:000
1238225994911,10cyclictest0-21swapper/007:20:010
1238225994911,10cyclictest0-21swapper/007:20:010
1238236994833,10cyclictest235-21systemd-journal12:17:293
1238234994831,13cyclictest235-21systemd-journal11:04:532
1238234994817,4cyclictest131rcu_preempt09:20:002
1238234994817,4cyclictest131rcu_preempt09:20:002
1238230994828,17cyclictest1253215-21apt-get11:57:281
123822599489,9cyclictest12-21ksoftirqd/007:50:010
1238225994810,8cyclictest0-21swapper/008:00:010
1238236994736,7cyclictest37-21ksoftirqd/307:25:003
1238230994728,15cyclictest1254029-21sshd12:11:281
1238230994712,7cyclictest23-21ksoftirqd/108:45:001
123822599474,7cyclictest12-21ksoftirqd/012:05:010
123822599474,11cyclictest12-21ksoftirqd/010:50:020
123822599473,10cyclictest1254996-21cat12:30:020
1238225994724,19cyclictest16850irq/134-800000011:28:290
1238234994629,13cyclictest235-21systemd-journal09:06:292
1238230994622,20cyclictest1-21systemd09:30:001
123822599464,8cyclictest12-21ksoftirqd/012:25:010
1238225994611,3cyclictest19050irq/34-mmc111:20:010
1238236994530,11cyclictest37-21ksoftirqd/309:40:003
1238236994528,11cyclictest235-21systemd-journal09:10:003
1238234994534,8cyclictest1253790-21sshd12:09:292
123823099457,33cyclictest1254451-21cat12:20:021
1238225994531,10cyclictest1-21systemd11:45:000
1238236994428,11cyclictest272-21dbus-daemon07:15:013
1238236994426,11cyclictest235-21systemd-journal07:45:003
1238236994424,15cyclictest272-21dbus-daemon10:05:013
123823499442,37cyclictest235-21systemd-journal10:45:002
1238230994431,8cyclictest23-21ksoftirqd/109:20:031
1238230994431,8cyclictest23-21ksoftirqd/109:20:021
1238230994428,12cyclictest235-21systemd-journal10:12:291
1238230994413,27cyclictest1242093-21sort08:25:011
1238225994436,5cyclictest16950irq/224-800000012:20:000
123822599442,13cyclictest1241540-21munin-run08:15:010
123822599441,8cyclictest1244405-21munin-run09:10:000
1238225994410,4cyclictest131rcu_preempt11:59:280
1238236994321,15cyclictest272-21dbus-daemon10:10:013
1238234994329,11cyclictest30-21ksoftirqd/209:30:012
1238234994329,10cyclictest272-21dbus-daemon08:25:002
1238234994323,12cyclictest1245963-21systemctl09:40:002
1238230994332,7cyclictest1-21systemd08:50:011
1238230994332,7cyclictest1-21systemd08:50:011
1238230994330,9cyclictest23-21ksoftirqd/111:25:001
1238230994330,9cyclictest23-21ksoftirqd/111:25:001
1238230994324,15cyclictest1251197-21munin-run11:20:011
123822599437,6cyclictest12-21ksoftirqd/008:30:020
123822599436,9cyclictest1255189-21latency_hist12:35:030
1238225994327,12cyclictest235-21systemd-journal09:20:010
1238225994327,12cyclictest235-21systemd-journal09:20:010
1238225994325,12cyclictest235-21systemd-journal10:45:000
123822599431,5cyclictest0-21swapper/007:25:000
123822599431,37cyclictest1244623-21sshd09:11:290
1238225994312,9cyclictest12-21ksoftirqd/011:05:020
1238236994230,9cyclictest1240722-21sshd07:58:293
1238236994228,10cyclictest235-21systemd-journal08:05:003
1238236994225,12cyclictest272-21dbus-daemon12:25:013
1238236994222,13cyclictest1-21systemd11:25:013
1238236994222,13cyclictest1-21systemd11:25:013
1238236994217,17cyclictest272-21dbus-daemon08:45:003
1238234994224,13cyclictest272-21dbus-daemon12:00:002
1238234994223,14cyclictest235-21systemd-journal10:05:012
123823499422,12cyclictest235-21systemd-journal08:20:002
1238234994215,21cyclictest272-21dbus-daemon07:15:002
1238230994230,9cyclictest23-21ksoftirqd/109:15:021
1238230994228,11cyclictest1239970-21sshd07:44:211
1238230994226,13cyclictest1248546-21sshd10:28:291
123822599425,8cyclictest12-21ksoftirqd/008:10:020
1238225994229,10cyclictest1249087-21systemctl10:40:000
1238225994227,11cyclictest1-21systemd07:40:010
123822599421,37cyclictest1252177-21sshd11:40:000
1238236994129,10cyclictest1251127-21sshd11:17:293
1238236994126,9cyclictest37-21ksoftirqd/310:15:003
1238236994126,12cyclictest235-21systemd-journal10:52:293
1238236994126,12cyclictest235-21systemd-journal10:52:293
1238236994125,13cyclictest272-21dbus-daemon09:25:003
1238236994121,15cyclictest1-21systemd08:25:003
1238234994132,5cyclictest1248315-21munin-plugin-st10:25:002
1238234994128,10cyclictest30-21ksoftirqd/212:25:292
1238234994119,15cyclictest272-21dbus-daemon09:05:012
1238230994132,5cyclictest1246500-21munin-run09:50:011
1238230994130,5cyclictest1239697-21cron07:40:011
1238230994129,8cyclictest23-21ksoftirqd/110:55:011
1238230994129,8cyclictest23-21ksoftirqd/110:55:011
1238230994128,9cyclictest1246234-21munin-run09:45:001
1238230994128,10cyclictest1253534-21sshd12:03:291
1238230994127,9cyclictest272-21dbus-daemon09:40:011
1238230994127,10cyclictest363-21snmpd11:54:251
1238230994127,10cyclictest235-21systemd-journal07:30:011
1238230994127,10cyclictest23-21ksoftirqd/111:45:011
1238230994127,10cyclictest23-21ksoftirqd/110:50:011
1238230994127,10cyclictest1245714-21munin-run09:35:011
1238230994124,13cyclictest1252171-21sshd11:38:291
1238230994113,23cyclictest1-21systemd07:55:011
123822599419,4cyclictest1240494-21munin-run07:55:010
123822599415,8cyclictest1243131-21cat08:45:010
123822599414,7cyclictest12-21ksoftirqd/009:50:010
1238225994127,10cyclictest12-21ksoftirqd/008:20:000
123822599412,6cyclictest1249859-21munin-run10:55:010
123822599412,6cyclictest1249859-21munin-run10:55:010
1238225994125,11cyclictest272-21dbus-daemon07:30:010
1238225994124,12cyclictest272-21dbus-daemon07:15:010
123822599412,35cyclictest1244123-21sshd09:04:290
1238225994121,4cyclictest16850irq/134-800000009:39:410
123822599411,35cyclictest1248295-21sshd10:23:290
123822599411,2cyclictest0-21swapper/010:20:020
123823699405,26cyclictest235-21systemd-journal07:35:003
123823699404,3cyclictest131rcu_preempt09:55:023
123823699404,3cyclictest131rcu_preempt09:55:023
1238236994025,9cyclictest1246999-21systemctl10:00:003
1238236994022,13cyclictest235-21systemd-journal08:30:023
123823499405,3cyclictest131rcu_preempt09:00:002
1238234994032,5cyclictest30-21ksoftirqd/208:40:022
1238234994030,8cyclictest30-21ksoftirqd/211:55:002
1238234994024,12cyclictest1-21systemd08:05:012
1238234994021,14cyclictest272-21dbus-daemon07:30:012
1238234994021,13cyclictest272-21dbus-daemon08:45:002
123823099407,4cyclictest131rcu_preempt12:35:011
123823099407,4cyclictest131rcu_preempt08:06:291
1238230994031,5cyclictest1241528-21munin-run08:15:001
1238230994029,8cyclictest1254818-21sshd12:25:051
1238230994028,9cyclictest1255403-21munin-run12:40:011
1238230994027,9cyclictest1250907-21munin-run11:15:001
1238230994027,10cyclictest272-21dbus-daemon08:20:011
1238230994026,11cyclictest1243580-21sshd08:52:291
123822599407,5cyclictest0-21swapper/012:14:300
123822599404,6cyclictest12-21ksoftirqd/012:09:280
123822599403,7cyclictest12-21ksoftirqd/011:35:010
1238225994034,4cyclictest16950irq/224-800000012:35:290
1238225994022,14cyclictest1239433-21sshd07:33:290
123822599401,35cyclictest0-21swapper/008:25:010
1238236993927,8cyclictest341irq_work/311:10:003
1238236993927,8cyclictest341irq_work/311:10:003
1238236993924,11cyclictest37-21ksoftirqd/309:05:003
1238236993924,10cyclictest235-21systemd-journal11:05:003
1238236993923,12cyclictest37-21ksoftirqd/307:55:013
1238236993922,11cyclictest272-21dbus-daemon10:20:003
1238234993925,11cyclictest30-21ksoftirqd/211:10:032
1238234993925,11cyclictest30-21ksoftirqd/211:10:022
123823099398,26cyclictest1-21systemd10:25:011
123823099393,22cyclictest1230939-21kworker/u8:307:10:011
1238230993927,9cyclictest363-21snmpd07:23:301
1238230993926,9cyclictest23-21ksoftirqd/108:05:011
1238230993926,10cyclictest23-21ksoftirqd/107:11:291
1238230993921,13cyclictest235-21systemd-journal07:31:301
123822599396,6cyclictest12-21ksoftirqd/008:50:010
123822599396,6cyclictest12-21ksoftirqd/008:50:010
123822599393,8cyclictest12-21ksoftirqd/010:35:010
1238225993932,4cyclictest16950irq/224-800000011:55:010
1238225993926,10cyclictest1244076-21cut09:00:020
1238225993924,11cyclictest272-21dbus-daemon10:10:020
1238225993922,13cyclictest1-21systemd09:55:010
1238225993922,13cyclictest1-21systemd09:55:010
123823699382,7cyclictest131rcu_preempt09:20:003
123823699382,7cyclictest131rcu_preempt09:20:003
1238236993827,8cyclictest37-21ksoftirqd/310:45:023
1238236993825,10cyclictest37-21ksoftirqd/309:15:013
1238236993823,11cyclictest37-21ksoftirqd/312:35:013
1238234993828,8cyclictest30-21ksoftirqd/208:55:012
1238234993828,6cyclictest1242319-21munin-plugin-st08:30:002
1238234993825,10cyclictest30-21ksoftirqd/211:45:002
1238230993826,9cyclictest363-21snmpd08:28:441
1238230993826,8cyclictest48-21kcompactd010:43:011
1238230993825,10cyclictest235-21systemd-journal08:35:001
1238230993824,9cyclictest48-21kcompactd010:55:471
123822599384,8cyclictest12-21ksoftirqd/011:14:550
1238225993820,14cyclictest1242537-21sshd08:30:290
1238236993726,9cyclictest37-21ksoftirqd/308:37:293
123823699372,4cyclictest131rcu_preempt09:00:003
1238236993721,12cyclictest37-21ksoftirqd/311:32:303
1238236993719,13cyclictest272-21dbus-daemon12:10:023
123823499376,4cyclictest131rcu_preempt09:45:002
1238234993727,8cyclictest30-21ksoftirqd/212:11:282
1238234993724,8cyclictest30-21ksoftirqd/207:10:012
1238230993725,9cyclictest23-21ksoftirqd/109:25:001
1238230993724,9cyclictest363-21snmpd08:38:371
1238230993724,10cyclictest363-21snmpd11:34:201
1238230993720,14cyclictest363-21snmpd09:08:171
123822599372,7cyclictest12-21ksoftirqd/008:00:290
1238225993722,11cyclictest12-21ksoftirqd/011:07:290
1238225993722,11cyclictest12-21ksoftirqd/011:07:290
1238225993710,5cyclictest0-21swapper/010:02:300
123823699368,5cyclictest131rcu_preempt07:46:143
123823699365,4cyclictest131rcu_preempt09:50:003
1238236993626,6cyclictest235-21systemd-journal08:16:293
1238236993624,8cyclictest37-21ksoftirqd/309:45:013
1238236993624,8cyclictest1254059-21munin-run12:15:013
123823499364,6cyclictest131rcu_preempt09:50:012
123823499363,4cyclictest131rcu_preempt09:25:002
123823499363,3cyclictest131rcu_preempt07:35:012
123823499363,24cyclictest235-21systemd-journal11:35:002
1238234993629,5cyclictest1241492-21sshd08:10:292
1238234993628,6cyclictest30-21ksoftirqd/211:25:002
1238234993628,6cyclictest30-21ksoftirqd/211:25:002
1238234993626,8cyclictest30-21ksoftirqd/210:10:022
123823499362,4cyclictest131rcu_preempt11:30:002
123823499361,4cyclictest131rcu_preempt07:55:022
1238230993625,9cyclictest23-21ksoftirqd/111:05:011
1238230993624,9cyclictest1246701-21sshd09:50:291
1238230993624,9cyclictest1246701-21sshd09:50:291
1238225993625,7cyclictest1245183-21cron09:25:000
123823699353,3cyclictest131rcu_preempt12:30:003
1238236993527,5cyclictest1243609-21/usr/sbin/munin08:55:003
1238236993524,9cyclictest37-21ksoftirqd/311:15:013
1238236993523,8cyclictest1245683-21sshd09:34:413
1238236993522,9cyclictest1253568-21systemd-run12:05:013
1238236993522,10cyclictest37-21ksoftirqd/307:40:013
1238236993518,12cyclictest37-21ksoftirqd/308:05:013
123823499357,25cyclictest1252718-21sh11:50:002
1238234993521,9cyclictest272-21dbus-daemon12:20:012
1238234993520,12cyclictest1-21systemd10:15:002
1238234993516,14cyclictest306-21in:imuxsock09:30:022
1238230993522,9cyclictest363-21snmpd08:55:101
1238230993517,15cyclictest363-21snmpd10:07:241
123822599353,7cyclictest12-21ksoftirqd/011:49:290
123823699346,22cyclictest1251689-21grep11:30:003
1238236993425,7cyclictest37-21ksoftirqd/311:45:023
1238236993425,5cyclictest306-21in:imuxsock11:39:293
1238236993419,12cyclictest1239177-21sshd07:27:293
1238234993426,5cyclictest1255366-21latency_hist12:40:022
1238234993422,9cyclictest1239709-21cron07:40:012
1238234993421,10cyclictest1252228-21systemd-run11:40:022
1238234993420,11cyclictest1239954-21sshd07:40:292
1238230993422,9cyclictest23-21ksoftirqd/109:01:291
123823699335,3cyclictest131rcu_preempt11:00:003
1238236993325,5cyclictest1249704-21latency_hist10:50:023
1238236993325,4cyclictest271-21cron10:40:003
123823699332,22cyclictest1242422-21wc08:30:023
123823499335,4cyclictest131rcu_preempt10:30:012
123823499333,5cyclictest131rcu_preempt08:10:022
1238234993326,4cyclictest1248819-21latency_hist10:35:022
1238234993324,7cyclictest30-21ksoftirqd/212:24:292
1238234993320,9cyclictest306-21in:imuxsock07:50:002
1238234993320,10cyclictest1-21systemd10:55:012
1238234993320,10cyclictest1-21systemd10:55:012
1238234993318,11cyclictest1243362-21systemd-run08:50:012
1238234993318,11cyclictest1243362-21systemd-run08:50:012
123823699325,3cyclictest131rcu_preempt11:55:023
1238236993223,6cyclictest1243354-21latency_hist08:50:023
1238236993223,6cyclictest1243354-21latency_hist08:50:023
1238236993219,10cyclictest37-21ksoftirqd/310:25:033
123823499324,5cyclictest131rcu_preempt08:35:002
123823499323,4cyclictest131rcu_preempt07:25:002
123823499322,4cyclictest131rcu_preempt10:00:002
1238234993222,5cyclictest1250111-21sh11:00:002
1238234993221,8cyclictest0-21swapper/209:15:012
1238234993219,10cyclictest30-21ksoftirqd/209:55:012
1238234993219,10cyclictest30-21ksoftirqd/209:55:012
1238234993216,11cyclictest1-21systemd08:00:002
1238236993124,4cyclictest1238156-21latency_hist07:10:003
123823699311,27cyclictest37-21ksoftirqd/310:30:003
1238234993120,9cyclictest30-21ksoftirqd/210:17:002
1238225993118,9cyclictest12-21ksoftirqd/007:10:000
123823699303,20cyclictest235-21systemd-journal10:35:003
123823499301,4cyclictest131rcu_preempt12:35:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional