You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-10 - 19:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot3s.osadl.org (updated Wed Dec 10, 2025 12:45:01)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
185647199731,54cyclictest361rcuc/312:35:013
1856468996823,40cyclictest1871193-21cron11:55:002
185646899652,16cyclictest1871445-21cron12:00:002
1856461996456,6cyclictest12-21ksoftirqd/011:15:030
1856468996322,36cyclictest1859783-21cat08:15:012
185646899623,15cyclictest1863159-21grep09:20:012
185646899622,23cyclictest1872774-21modprobe12:25:002
185646899622,15cyclictest1870956-21munin-run11:50:002
1856468996125,30cyclictest306-21in:imuxsock12:27:182
1856468996125,30cyclictest306-21in:imuxsock12:27:182
1856465996141,18cyclictest1864492-21kworker/1:209:50:021
185646899603,18cyclictest1869393-21sendmail11:20:002
1856468995918,5cyclictest1872236-21latency_hist12:15:002
185646899583,17cyclictest1870723-21cat11:45:012
1856468995818,4cyclictest1873281-21latency_hist12:35:022
185646899581,16cyclictest1868351-21grep11:00:012
1856461995821,33cyclictest12-21ksoftirqd/008:55:120
1856468995728,23cyclictest30-21ksoftirqd/209:40:002
185646899572,16cyclictest1868895-21latency_hist11:10:012
1856468995721,4cyclictest1863726-21latency_hist09:30:022
1856468995718,7cyclictest1856750-21grep07:15:022
1856468995712,15cyclictest1871981-21modprobe12:10:002
185646899563,9cyclictest1863431-21modprobe09:25:002
1856468995621,30cyclictest1873535-21modprobe12:40:002
1856465995639,10cyclictest235-21systemd-journal07:59:591
185646899554,10cyclictest1862616-21cron09:10:002
185646899552,16cyclictest235-21systemd-journal08:09:592
185646899552,13cyclictest1870308-21cat11:35:022
1856465995533,15cyclictest235-21systemd-journal09:45:001
185646199551,14cyclictest1863526-21wc09:25:010
1856468995423,6cyclictest30-21ksoftirqd/211:25:012
1856468995423,6cyclictest30-21ksoftirqd/211:25:012
185646899541,17cyclictest1-21systemd10:04:592
1856465995432,17cyclictest272-21dbus-daemon09:05:001
185646199545,13cyclictest1191562-1kworker/0:0H07:40:020
1856468995322,8cyclictest30-21ksoftirqd/209:05:012
1856468995317,4cyclictest1858313-21latency_hist07:45:012
1856461995335,14cyclictest272-21dbus-daemon09:30:010
185646899523,12cyclictest1862106-21grep09:00:002
1856468995221,25cyclictest1866027-21latency_hist10:15:022
1856468995214,7cyclictest30-21ksoftirqd/207:25:052
1856468995213,8cyclictest1865896-21wc10:10:022
185646899521,13cyclictest235-21systemd-journal07:40:012
185646199526,9cyclictest12-21ksoftirqd/008:35:020
185646199526,10cyclictest12-21ksoftirqd/010:10:020
1856461995234,13cyclictest1-21systemd08:05:000
185646899515,9cyclictest0-21swapper/208:40:002
185646899513,15cyclictest308-21rs:main2
185646899513,15cyclictest1867599-21munin-run10:45:022
185646899512,17cyclictest1857204-21munin-run07:25:012
1856468995118,6cyclictest1861519-21wc08:45:022
1856468995115,10cyclictest30-21ksoftirqd/210:54:262
185646199517,10cyclictest12-21ksoftirqd/008:14:010
185646199514,13cyclictest12-21ksoftirqd/012:20:000
185646199513,8cyclictest12-21ksoftirqd/010:00:000
185646199512,15cyclictest12-21ksoftirqd/008:30:030
185646199511,14cyclictest12-21ksoftirqd/010:20:020
1856471995038,7cyclictest37-21ksoftirqd/307:35:023
185646899502,15cyclictest1867850-21systemd-run10:50:012
1856468995019,14cyclictest1857049-21sed07:20:022
1856468995019,14cyclictest1857049-21sed07:20:012
185646199502,12cyclictest12-21ksoftirqd/010:55:010
1856471994914,12cyclictest37-21ksoftirqd/311:00:023
185646899492,17cyclictest1857695-21munin-run07:35:012
1856468994913,10cyclictest30-21ksoftirqd/208:50:012
185646899491,20cyclictest1858620-21latency_hist07:50:022
1856461994945,2cyclictest0-21swapper/007:50:020
185646199493,10cyclictest12-21ksoftirqd/007:25:010
1856471994837,7cyclictest37-21ksoftirqd/308:15:023
185647199482,20cyclictest1856871-21kworker/u8:108:05:023
1856468994826,16cyclictest235-21systemd-journal08:35:002
1856468994816,9cyclictest30-21ksoftirqd/209:49:142
1856465994832,12cyclictest1-21systemd10:05:011
185646199486,8cyclictest12-21ksoftirqd/012:30:020
185646199486,8cyclictest12-21ksoftirqd/012:30:020
185646199484,9cyclictest1858944-21modprobe07:55:020
1856461994833,11cyclictest1-21systemd10:50:010
185646199482,8cyclictest12-21ksoftirqd/012:15:010
1856461994828,13cyclictest272-21dbus-daemon09:50:010
1856471994731,11cyclictest272-21dbus-daemon12:20:003
1856471994717,8cyclictest37-21ksoftirqd/312:40:013
1856468994732,12cyclictest235-21systemd-journal09:54:122
185646899472,6cyclictest1868761-21latency_hist11:05:022
1856468994718,5cyclictest1866936-21latency_hist10:30:022
185646899471,16cyclictest1866303-21systemd-run10:20:012
185646599472,5cyclictest131rcu_preempt12:20:001
1856465994717,23cyclictest23-21ksoftirqd/108:20:011
185646199471,7cyclictest0-21swapper/011:59:590
185646199471,11cyclictest12-21ksoftirqd/007:10:020
1856471994613,9cyclictest131rcu_preempt08:25:023
185646899462,14cyclictest1870450-21sendmail-msp11:40:012
185646899462,14cyclictest1870450-21sendmail-msp11:40:012
1856465994638,4cyclictest23-21ksoftirqd/107:55:001
1856465994635,7cyclictest1863661-21sshd09:29:131
1856465994631,8cyclictest23-21ksoftirqd/111:55:021
1856465994627,15cyclictest235-21systemd-journal11:00:001
1856465994612,25cyclictest221rcuc/112:35:011
185646199465,9cyclictest12-21ksoftirqd/007:30:000
185646199465,10cyclictest12-21ksoftirqd/011:45:020
185646199463,7cyclictest12-21ksoftirqd/007:35:010
1856461994627,14cyclictest235-21systemd-journal08:55:010
1856461994625,16cyclictest272-21dbus-daemon09:40:010
1856461994617,8cyclictest131rcu_preempt11:10:010
1856471994517,7cyclictest37-21ksoftirqd/311:10:013
1856468994530,11cyclictest272-21dbus-daemon08:25:012
1856468994516,9cyclictest1-21systemd12:20:002
1856465994526,14cyclictest272-21dbus-daemon09:35:011
185646199454,9cyclictest12-21ksoftirqd/012:39:180
1856461994536,5cyclictest1862624-21sh09:09:590
185646199452,8cyclictest12-21ksoftirqd/009:20:010
1856461994526,14cyclictest1-21systemd09:05:010
185646199451,12cyclictest1866696-21cut10:25:020
185646199451,12cyclictest12-21ksoftirqd/007:15:020
1856471994432,7cyclictest1862617-21cron09:09:593
1856471994430,8cyclictest37-21ksoftirqd/310:50:013
1856471994428,12cyclictest235-21systemd-journal10:20:003
1856471994413,8cyclictest131rcu_preempt09:55:023
1856471994410,6cyclictest131rcu_preempt10:55:013
185646199447,6cyclictest12-21ksoftirqd/009:45:020
185646199447,5cyclictest131rcu_preempt09:55:010
185646199446,9cyclictest12-21ksoftirqd/011:29:340
185646199443,8cyclictest12-21ksoftirqd/010:30:010
1856461994435,7cyclictest151rcuc/010:45:010
185646199442,35cyclictest235-21systemd-journal10:02:090
185646199442,12cyclictest12-21ksoftirqd/012:35:010
1856471994333,7cyclictest37-21ksoftirqd/309:30:033
185647199432,11cyclictest37-21ksoftirqd/309:47:123
185646899432,4cyclictest131rcu_preempt09:45:022
1856468994323,14cyclictest272-21dbus-daemon09:35:002
1856465994329,10cyclictest306-21in:imuxsock08:39:591
185646199436,7cyclictest12-21ksoftirqd/010:35:000
1856461994326,13cyclictest12-21ksoftirqd/007:20:020
1856461994326,13cyclictest12-21ksoftirqd/007:20:010
1856461994310,29cyclictest12-21ksoftirqd/010:56:120
185647199426,10cyclictest37-21ksoftirqd/311:15:023
185647199423,19cyclictest37-21ksoftirqd/312:05:213
1856471994230,9cyclictest37-21ksoftirqd/311:20:023
1856471994229,10cyclictest37-21ksoftirqd/310:00:003
1856471994212,6cyclictest131rcu_preempt11:50:013
1856468994230,9cyclictest30-21ksoftirqd/208:20:012
1856465994225,12cyclictest23-21ksoftirqd/110:00:011
1856465994222,15cyclictest235-21systemd-journal07:50:001
1856465994220,15cyclictest272-21dbus-daemon11:30:001
185646199424,10cyclictest1861516-21latency_hist08:45:010
185646199423,12cyclictest12-21ksoftirqd/012:25:010
185646199423,10cyclictest12-21ksoftirqd/011:50:000
1856461994230,9cyclictest12-21ksoftirqd/011:40:020
1856461994230,9cyclictest12-21ksoftirqd/011:40:010
1856461994230,8cyclictest1861065-21sendmail08:40:000
185646199422,14cyclictest1863066-21latency_hist09:15:020
185646199421,12cyclictest12-21ksoftirqd/008:25:010
185647199416,11cyclictest37-21ksoftirqd/311:38:133
185647199416,11cyclictest37-21ksoftirqd/311:38:123
185647199416,11cyclictest37-21ksoftirqd/310:38:113
185647199413,6cyclictest131rcu_preempt09:45:013
185647199413,6cyclictest131rcu_preempt07:25:023
1856471994133,6cyclictest37-21ksoftirqd/308:10:013
1856471994127,11cyclictest37-21ksoftirqd/311:55:013
1856471994127,10cyclictest1-21systemd08:00:013
1856471994121,14cyclictest272-21dbus-daemon10:05:003
1856465994129,8cyclictest1856749-21cat07:15:011
185646199419,5cyclictest131rcu_preempt08:10:000
1856461994133,6cyclictest12-21ksoftirqd/011:15:010
1856461994132,6cyclictest12-21ksoftirqd/010:39:590
185647199408,6cyclictest131rcu_preempt07:15:013
185647199406,7cyclictest131rcu_preempt12:00:003
1856471994032,6cyclictest37-21ksoftirqd/310:35:003
1856471994032,6cyclictest37-21ksoftirqd/310:30:033
1856471994029,8cyclictest37-21ksoftirqd/309:40:003
1856471994027,10cyclictest1857415-21sshd07:27:473
1856471994023,13cyclictest235-21systemd-journal08:44:593
1856471994022,13cyclictest272-21dbus-daemon09:05:013
185646899403,3cyclictest131rcu_preempt09:14:592
1856468994031,7cyclictest30-21ksoftirqd/210:00:032
1856465994028,6cyclictest306-21in:imuxsock08:05:011
1856465994027,10cyclictest23-21ksoftirqd/109:40:011
1856465994024,9cyclictest235-21systemd-journal08:27:021
1856465994024,11cyclictest235-21systemd-journal08:45:011
1856465994023,13cyclictest1-21systemd07:45:011
1856465994020,13cyclictest23-21ksoftirqd/107:20:011
1856465994020,13cyclictest23-21ksoftirqd/107:20:011
1856465994018,16cyclictest272-21dbus-daemon10:30:011
185646599401,5cyclictest131rcu_preempt08:35:011
185646599401,14cyclictest23-21ksoftirqd/109:46:131
185646599401,10cyclictest23-21ksoftirqd/110:17:121
1856461994034,4cyclictest16950irq/224-800000008:20:000
1856461994034,4cyclictest16950irq/224-800000007:55:010
185646199402,33cyclictest0-21swapper/009:35:000
185646199401,8cyclictest1871206-21munin-run11:55:000
185646199401,5cyclictest0-21swapper/011:25:010
185646199401,5cyclictest0-21swapper/011:25:010
1856471993930,6cyclictest37-21ksoftirqd/312:25:003
1856471993928,6cyclictest1860012-21cron08:20:013
1856471993927,7cyclictest37-21ksoftirqd/311:34:353
185647199392,6cyclictest131rcu_preempt12:30:013
185647199392,6cyclictest131rcu_preempt12:30:013
1856471993923,12cyclictest37-21ksoftirqd/309:15:013
185647199391,4cyclictest131rcu_preempt07:45:023
1856471993914,6cyclictest131rcu_preempt10:15:023
185646599392,5cyclictest131rcu_preempt12:15:011
185646599391,5cyclictest131rcu_preempt12:39:591
1856461993932,4cyclictest16950irq/224-800000012:10:010
185646199391,10cyclictest12-21ksoftirqd/011:35:020
185647199385,6cyclictest131rcu_preempt08:50:023
185647199385,4cyclictest131rcu_preempt11:27:133
1856471993828,7cyclictest37-21ksoftirqd/308:40:013
1856471993826,8cyclictest306-21in:imuxsock09:23:123
1856471993825,8cyclictest37-21ksoftirqd/307:20:013
1856471993825,8cyclictest37-21ksoftirqd/307:20:003
1856471993825,10cyclictest37-21ksoftirqd/311:45:013
1856471993819,14cyclictest272-21dbus-daemon08:55:013
1856471993816,4cyclictest1858020-21sed07:40:013
1856471993814,6cyclictest131rcu_preempt07:51:013
185647199381,21cyclictest0-21swapper/312:15:013
1856471993811,22cyclictest361rcuc/311:25:023
1856471993811,22cyclictest361rcuc/311:25:023
185647199381,10cyclictest37-21ksoftirqd/309:20:023
1856468993824,10cyclictest30-21ksoftirqd/211:29:342
1856465993828,8cyclictest23-21ksoftirqd/112:25:031
1856465993826,8cyclictest235-21systemd-journal09:25:001
1856465993824,10cyclictest235-21systemd-journal07:40:011
185646199386,5cyclictest12-21ksoftirqd/011:05:000
1856471993726,8cyclictest37-21ksoftirqd/307:50:013
1856471993725,7cyclictest1868617-21sh11:04:593
1856471993724,9cyclictest0-21swapper/312:05:023
1856471993724,10cyclictest1865728-21sshd10:06:123
185646899372,9cyclictest0-21swapper/207:55:002
1856468993718,14cyclictest272-21dbus-daemon08:55:012
1856465993728,6cyclictest271-21cron12:00:001
1856465993726,8cyclictest1857564-21sed07:30:021
1856465993720,12cyclictest272-21dbus-daemon08:55:511
185646199375,6cyclictest12-21ksoftirqd/012:05:010
1856461993727,6cyclictest1858213-21cron07:45:000
1856461993724,10cyclictest12-21ksoftirqd/010:15:000
1856461993718,15cyclictest272-21dbus-daemon08:50:010
185647199367,5cyclictest131rcu_preempt08:35:013
1856471993627,6cyclictest37-21ksoftirqd/309:00:013
1856471993624,9cyclictest37-21ksoftirqd/310:25:013
185647199362,11cyclictest37-21ksoftirqd/310:44:133
1856471993617,16cyclictest235-21systemd-journal08:30:003
185646899364,4cyclictest131rcu_preempt10:40:002
1856468993624,9cyclictest1869112-21sshd11:14:312
185646599362,5cyclictest131rcu_preempt11:05:011
1856465993624,9cyclictest23-21ksoftirqd/109:10:001
1856465993623,10cyclictest1869805-21cat11:25:021
1856465993623,10cyclictest1869805-21cat11:25:021
1856465993623,10cyclictest1866038-21sed10:15:021
1856465993619,11cyclictest235-21systemd-journal10:40:011
1856468993519,11cyclictest272-21dbus-daemon10:25:022
185646599356,7cyclictest131rcu_preempt10:50:001
185646599354,5cyclictest131rcu_preempt12:08:181
185646599353,3cyclictest131rcu_preempt07:25:001
1856465993525,6cyclictest23-21ksoftirqd/108:10:001
1856465993522,9cyclictest0-21swapper/110:45:021
1856465993520,11cyclictest272-21dbus-daemon10:10:001
185646599351,5cyclictest131rcu_preempt08:50:021
185646899345,5cyclictest131rcu_preempt10:35:022
1856468993423,9cyclictest30-21ksoftirqd/208:30:002
185646899341,29cyclictest272-21dbus-daemon12:05:012
185646599347,5cyclictest131rcu_preempt09:15:021
185646599344,4cyclictest131rcu_preempt10:55:011
1856465993426,5cyclictest1871732-21latency_hist12:05:011
1856465993422,8cyclictest23-21ksoftirqd/107:35:001
1856465993418,11cyclictest1-21systemd11:10:011
185646599336,23cyclictest1870725-21wc11:45:011
1856465993327,4cyclictest412-21sshd10:32:121
1856465993320,9cyclictest23-21ksoftirqd/111:35:011
1856465993320,10cyclictest1863324-21sed09:20:021
1856465993317,12cyclictest23-21ksoftirqd/111:20:001
185647199324,6cyclictest37-21ksoftirqd/307:10:013
185647199321,22cyclictest1863907-21sshd09:32:113
185646899323,4cyclictest131rcu_preempt07:10:012
185646899321,3cyclictest131rcu_preempt08:05:022
185646599323,4cyclictest131rcu_preempt11:11:111
1856465993225,4cyclictest1860302-21latency_hist08:25:021
1856465993224,5cyclictest1870954-21munin-plugin-st11:50:001
1856465993222,7cyclictest23-21ksoftirqd/111:40:021
1856465993222,7cyclictest23-21ksoftirqd/111:40:021
1856465993222,7cyclictest1866517-21sshd10:21:131
1856465993214,14cyclictest1-21systemd12:30:011
1856465993214,14cyclictest1-21systemd12:30:011
1856465993122,6cyclictest1859779-21latency_hist08:15:001
1856465993019,8cyclictest235-21systemd-journal08:50:031
185646599253,16cyclictest1856466-21latency_hist07:10:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional