You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-29 - 23:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Sat Nov 29, 2025 12:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
499100991038,12cyclictest131rcu_preempt07:15:013
499085991021,2cyclictest0-21swapper/008:10:020
49909599771,4cyclictest131rcu_preempt11:05:022
49909599771,4cyclictest131rcu_preempt11:05:022
49909599727,4cyclictest131rcu_preempt08:30:022
49909599727,4cyclictest131rcu_preempt08:30:022
499095997214,3cyclictest131rcu_preempt12:05:012
499095996531,29cyclictest272-21dbus-daemon12:00:012
499095996531,29cyclictest272-21dbus-daemon12:00:012
49909599652,11cyclictest516213-21sshd12:36:052
49909599622,15cyclictest510558-21munin-run10:50:012
49909599622,11cyclictest499320-21munin-run07:15:012
49909599621,10cyclictest502732-21munin-run08:20:022
49910099619,31cyclictest503456-21kworker/u8:108:55:023
49909599612,15cyclictest514960-21munin-run12:15:022
49909599612,15cyclictest514960-21munin-run12:15:012
49909599609,3cyclictest0-21swapper/209:42:142
49909599601,13cyclictest508961-21sshd10:19:212
49909599593,5cyclictest501622-21sshd07:56:322
499095995932,23cyclictest1-21systemd11:45:022
499095995932,23cyclictest1-21systemd11:45:012
49909599592,52cyclictest506895-21sh09:40:002
49909599591,14cyclictest515458-21sshd12:21:192
499095995910,44cyclictest272-21dbus-daemon09:55:022
499093995940,14cyclictest505866-21cat09:20:001
49909599582,7cyclictest514709-21/usr/sbin/munin12:10:012
49909599582,17cyclictest503485-21munin-run08:35:012
49909599582,17cyclictest235-21systemd-journal11:55:012
49909599582,15cyclictest500658-21munin-run07:40:012
499095995816,38cyclictest512037-21sed11:15:022
49909599581,14cyclictest508223-21munin-run10:05:012
49909599581,12cyclictest503973-21sshd08:41:162
49909599574,48cyclictest512088-21modprobe11:20:012
49909599573,6cyclictest511573-21modprobe11:09:592
49909599572,12cyclictest505352-21munin-run09:10:002
49909599572,11cyclictest501115-21modprobe07:45:292
49909599572,10cyclictest502178-21munin-run08:10:002
49909599571,9cyclictest506358-21sshd09:25:192
499095995717,35cyclictest272-21dbus-daemon12:30:022
499095995717,35cyclictest272-21dbus-daemon12:30:012
49909599571,12cyclictest363-21snmpd08:53:042
49909599562,7cyclictest363-21snmpd10:53:342
49909599562,6cyclictest1-21systemd07:30:002
499095995623,30cyclictest30-21ksoftirqd/211:40:012
49909599561,12cyclictest510017-21cat10:40:002
49909599561,11cyclictest363-21snmpd09:45:022
49909599559,42cyclictest0-21swapper/208:38:042
49909599554,7cyclictest511039-21cron11:00:012
49909599553,5cyclictest505841-21sshd09:18:182
49909599552,6cyclictest509258-21munin-run10:25:002
49909599552,12cyclictest504772-21munin-run09:00:012
49909599551,8cyclictest512851-21sshd11:32:202
49909599551,12cyclictest515979-21cron12:34:592
49909599544,6cyclictest499859-21modprobe07:25:002
49909599544,6cyclictest499859-21modprobe07:25:002
49909599543,6cyclictest500865-21sshd07:41:292
499095995432,19cyclictest30-21ksoftirqd/207:55:022
499095995431,20cyclictest500360-21latency_hist07:35:022
49909599542,13cyclictest515249-21systemd-run12:20:002
49909599542,11cyclictest510286-21sh10:45:002
49909599541,13cyclictest363-21snmpd08:21:482
49909599533,5cyclictest499539-21sshd07:15:152
49909599533,12cyclictest509762-21munin-run10:35:012
49909599532,14cyclictest513674-21munin-run11:50:002
49909599532,10cyclictest363-21snmpd09:13:492
49909599531,6cyclictest506657-21munin-run09:35:012
499100995232,14cyclictest501635-21cron08:00:003
499095995230,18cyclictest501920-21latency_hist08:05:012
49909599521,11cyclictest504235-21sshd08:48:162
499093995233,7cyclictest23-21ksoftirqd/109:10:021
499085995239,8cyclictest19050irq/34-mmc111:30:010
49909599512,6cyclictest505053-21systemd-run09:05:022
49909599512,11cyclictest506117-21sshd09:22:182
49909599511,12cyclictest363-21snmpd09:58:162
49909599505,3cyclictest0-21swapper/208:10:182
49909599505,3cyclictest0-21swapper/208:10:182
49909599503,8cyclictest508454-21modprobe10:10:012
49909599502,7cyclictest1-21systemd11:25:002
499095995017,29cyclictest509546-21sed10:30:012
499095995010,35cyclictest272-21dbus-daemon11:30:002
49908599501,4cyclictest131rcu_preempt07:45:010
49910099485,6cyclictest131rcu_preempt12:00:023
49910099485,6cyclictest131rcu_preempt12:00:023
499093994837,7cyclictest23-21ksoftirqd/110:54:591
499093994836,9cyclictest235-21systemd-journal10:40:201
499093994813,6cyclictest131rcu_preempt10:20:001
499085994834,8cyclictest508966-21cron10:20:000
499100994732,11cyclictest235-21systemd-journal07:35:023
499095994714,27cyclictest508918-21cat10:15:022
499093994713,11cyclictest23-21ksoftirqd/111:05:021
499093994713,11cyclictest23-21ksoftirqd/111:05:011
499093994712,7cyclictest131rcu_preempt12:40:011
49908599471,23cyclictest131rcu_preempt09:55:020
499100994629,11cyclictest272-21dbus-daemon09:30:003
499100994614,6cyclictest131rcu_preempt11:55:003
49908599466,7cyclictest12-21ksoftirqd/007:30:020
49908599464,8cyclictest515197-21modprobe12:20:000
499085994630,11cyclictest12-21ksoftirqd/011:40:010
499085994628,14cyclictest1-21systemd11:10:000
499085994627,12cyclictest306-21in:imuxsock10:30:010
49908599461,3cyclictest131rcu_preempt07:15:150
499100994530,11cyclictest272-21dbus-daemon08:20:003
499100994529,13cyclictest37-21ksoftirqd/310:45:003
499093994531,11cyclictest235-21systemd-journal08:48:161
49909399452,9cyclictest23-21ksoftirqd/108:15:021
49909399452,9cyclictest23-21ksoftirqd/108:15:021
499093994528,12cyclictest23-21ksoftirqd/112:00:011
499093994528,12cyclictest23-21ksoftirqd/112:00:001
499085994533,9cyclictest12-21ksoftirqd/008:55:020
499085994532,10cyclictest235-21systemd-journal11:15:570
499085994532,10cyclictest1-21systemd08:40:010
499085994531,10cyclictest235-21systemd-journal09:15:180
499085994520,22cyclictest12-21ksoftirqd/009:50:020
49910099446,10cyclictest37-21ksoftirqd/310:30:033
499100994426,14cyclictest272-21dbus-daemon11:05:013
499100994426,14cyclictest272-21dbus-daemon11:05:013
499093994432,8cyclictest512079-21cron11:20:001
499093994432,8cyclictest23-21ksoftirqd/112:20:021
499093994430,11cyclictest23-21ksoftirqd/110:10:001
499093994413,6cyclictest131rcu_preempt09:35:031
49908599444,35cyclictest513764-21latency_hist11:50:020
499085994429,11cyclictest1-21systemd11:15:010
499085994427,14cyclictest501124-21sshd07:47:290
49908599441,38cyclictest0-21swapper/011:05:030
49908599441,38cyclictest0-21swapper/011:05:020
499093994335,5cyclictest505590-21cron09:15:001
49909399433,21cyclictest501412-21kworker/u8:108:05:021
499093994329,11cyclictest235-21systemd-journal11:35:011
499093994313,4cyclictest40850lldpd10:29:541
499085994336,4cyclictest16950irq/224-800000008:20:000
499085994335,4cyclictest16950irq/224-800000010:40:010
49908599433,35cyclictest499318-21munin-plugin-st07:15:000
499085994333,5cyclictest306-21in:imuxsock07:35:000
49908599433,35cyclictest0-21swapper/009:05:010
499085994329,10cyclictest235-21systemd-journal09:40:020
499085994329,10cyclictest235-21systemd-journal07:40:000
49908599431,3cyclictest131rcu_preempt12:00:020
49908599431,3cyclictest131rcu_preempt12:00:010
499100994231,7cyclictest37-21ksoftirqd/311:35:013
49909399428,6cyclictest131rcu_preempt07:55:011
499093994229,9cyclictest306-21in:imuxsock07:15:001
499093994225,13cyclictest235-21systemd-journal07:35:001
499085994236,3cyclictest16950irq/224-800000009:09:190
499085994234,4cyclictest16950irq/224-800000008:45:000
499085994222,12cyclictest272-21dbus-daemon08:15:010
499085994222,12cyclictest272-21dbus-daemon08:15:010
499085994213,7cyclictest12-21ksoftirqd/012:10:020
49910099413,5cyclictest131rcu_preempt09:45:023
499100994130,8cyclictest512051-21sshd11:15:573
499100994128,9cyclictest515496-21systemd-run12:25:013
499100994122,13cyclictest272-21dbus-daemon12:40:003
49909599413,7cyclictest499130-21grep07:10:022
49909599413,7cyclictest499130-21grep07:10:022
49909399419,6cyclictest131rcu_preempt08:40:001
499093994130,8cyclictest235-21systemd-journal10:15:011
499093994130,8cyclictest23-21ksoftirqd/111:40:011
499093994127,11cyclictest507545-21sed09:50:021
499093994123,13cyclictest235-21systemd-journal07:40:011
499093994115,6cyclictest221rcuc/111:00:001
499085994127,9cyclictest306-21in:imuxsock09:55:030
499085994126,11cyclictest1-21systemd09:00:010
499085994124,12cyclictest12-21ksoftirqd/008:00:000
49910099408,5cyclictest131rcu_preempt09:00:023
499100994031,5cyclictest505346-21latency_hist09:10:003
499100994030,7cyclictest508188-21sshd10:02:193
499100994030,7cyclictest37-21ksoftirqd/311:40:023
499100994027,9cyclictest235-21systemd-journal09:43:173
499100994026,9cyclictest0-21swapper/312:20:013
499100994025,11cyclictest510000-21cron10:40:003
49910099402,4cyclictest131rcu_preempt07:35:013
499100994024,11cyclictest507935-21cron10:00:003
499100994021,14cyclictest235-21systemd-journal11:00:003
499100994021,13cyclictest1-21systemd10:15:003
499093994030,7cyclictest23-21ksoftirqd/110:40:021
499093994029,6cyclictest306-21in:imuxsock07:30:001
499093994024,12cyclictest235-21systemd-journal11:15:001
499093994017,19cyclictest23-21ksoftirqd/112:00:211
499093994011,6cyclictest131rcu_preempt09:40:011
499085994034,3cyclictest16950irq/224-800000010:08:200
499085994033,4cyclictest16950irq/224-800000008:50:000
499085994033,4cyclictest0-21swapper/011:55:020
499085994021,13cyclictest1-21systemd11:35:010
49908599401,35cyclictest515707-21sshd12:26:050
49908599401,35cyclictest515707-21sshd12:26:050
49908599401,35cyclictest0-21swapper/012:40:000
49908599401,35cyclictest0-21swapper/007:25:010
49908599401,35cyclictest0-21swapper/007:25:010
499100993929,7cyclictest37-21ksoftirqd/312:15:003
499100993929,7cyclictest37-21ksoftirqd/312:15:003
499100993928,7cyclictest37-21ksoftirqd/309:35:023
49910099391,27cyclictest502667-21kworker/u8:008:45:013
49909399398,12cyclictest23-21ksoftirqd/111:10:001
499093993931,5cyclictest23-21ksoftirqd/110:00:021
499093993930,6cyclictest23-21ksoftirqd/109:00:011
499093993929,7cyclictest23-21ksoftirqd/111:30:001
499093993924,12cyclictest23-21ksoftirqd/109:25:011
499093993912,19cyclictest221rcuc/112:35:011
499085993932,4cyclictest16950irq/224-800000012:11:030
499085993932,4cyclictest16950irq/224-800000012:11:030
499085993931,5cyclictest19050irq/34-mmc110:55:100
499085993928,8cyclictest12-21ksoftirqd/009:35:010
499085993927,9cyclictest408-21lldpd12:02:240
499085993925,10cyclictest12-21ksoftirqd/008:05:000
49910099384,6cyclictest131rcu_preempt10:20:003
499100993827,7cyclictest513383-21sshd11:42:203
499100993827,7cyclictest513383-21sshd11:42:203
499100993827,7cyclictest506944-21latency_hist09:40:023
499100993826,10cyclictest37-21ksoftirqd/307:45:013
499100993825,10cyclictest37-21ksoftirqd/308:30:013
499100993825,10cyclictest37-21ksoftirqd/308:30:013
499100993824,10cyclictest37-21ksoftirqd/310:25:003
49910099382,3cyclictest131rcu_preempt12:35:013
49910099381,30cyclictest412-21sshd07:10:023
49910099381,30cyclictest412-21sshd07:10:023
49909399383,8cyclictest23-21ksoftirqd/108:00:001
499093993830,6cyclictest23-21ksoftirqd/112:30:031
499093993830,6cyclictest23-21ksoftirqd/108:30:011
499093993830,6cyclictest23-21ksoftirqd/108:30:011
499093993824,10cyclictest1-21systemd08:55:011
49909399382,12cyclictest23-21ksoftirqd/112:15:011
49909399382,12cyclictest23-21ksoftirqd/112:15:001
499085993827,5cyclictest306-21in:imuxsock10:50:010
49908599382,6cyclictest510304-21munin-run10:45:010
499085993826,10cyclictest12-21ksoftirqd/008:35:010
499085993823,10cyclictest12-21ksoftirqd/012:35:030
49909399376,6cyclictest131rcu_preempt08:10:011
499093993730,4cyclictest501131-21sshd07:49:291
499093993728,7cyclictest23-21ksoftirqd/107:40:071
499093993728,6cyclictest23-21ksoftirqd/109:42:181
499093993726,7cyclictest23-21ksoftirqd/109:30:021
499093993725,10cyclictest23-21ksoftirqd/110:50:011
499093993717,15cyclictest272-21dbus-daemon10:25:011
499093993716,5cyclictest131rcu_preempt11:51:201
499093993710,5cyclictest131rcu_preempt10:35:021
499085993726,7cyclictest515457-21sshd12:21:040
499085993724,9cyclictest12-21ksoftirqd/007:51:310
499085993724,10cyclictest12-21ksoftirqd/011:45:010
499085993724,10cyclictest12-21ksoftirqd/011:45:010
49908599371,33cyclictest0-21swapper/009:25:000
49908599371,32cyclictest512369-21latency_hist11:25:010
49910099367,5cyclictest131rcu_preempt08:15:033
49910099367,5cyclictest131rcu_preempt08:15:023
499100993626,6cyclictest37-21ksoftirqd/308:25:003
499100993626,6cyclictest37-21ksoftirqd/308:10:013
499100993623,9cyclictest1-21systemd09:20:013
499100993621,12cyclictest37-21ksoftirqd/310:50:013
499100993618,13cyclictest235-21systemd-journal09:05:003
49909399366,7cyclictest131rcu_preempt08:25:011
49909399364,9cyclictest23-21ksoftirqd/111:50:001
499093993627,6cyclictest23-21ksoftirqd/110:05:011
499093993621,10cyclictest23-21ksoftirqd/111:43:211
499093993621,10cyclictest23-21ksoftirqd/111:43:201
499085993627,6cyclictest12-21ksoftirqd/009:45:010
49908599362,6cyclictest235-21systemd-journal09:28:220
499085993623,8cyclictest508210-21munin-run10:05:010
499085993619,13cyclictest272-21dbus-daemon08:30:020
499085993619,13cyclictest272-21dbus-daemon08:30:010
499085993619,11cyclictest1-21systemd10:35:010
49908599361,2cyclictest0-21swapper/009:15:010
499100993527,5cyclictest515738-21latency_hist12:30:023
499100993527,5cyclictest515738-21latency_hist12:30:023
499100993527,3cyclictest37-21ksoftirqd/311:25:013
499100993525,8cyclictest37-21ksoftirqd/311:27:203
499100993523,9cyclictest37-21ksoftirqd/311:14:213
499100993523,8cyclictest37-21ksoftirqd/307:25:013
499100993523,8cyclictest37-21ksoftirqd/307:25:013
499100993521,9cyclictest306-21in:imuxsock10:35:013
499100993521,10cyclictest235-21systemd-journal10:50:033
499100993520,12cyclictest37-21ksoftirqd/310:10:013
49910099351,6cyclictest131rcu_preempt11:50:023
499100993511,20cyclictest361rcuc/307:50:013
49909399356,8cyclictest23-21ksoftirqd/109:55:011
499093993525,8cyclictest23-21ksoftirqd/112:20:031
499093993525,7cyclictest23-21ksoftirqd/107:25:011
499093993525,7cyclictest23-21ksoftirqd/107:25:011
499093993525,6cyclictest23-21ksoftirqd/108:45:021
499093993524,8cyclictest23-21ksoftirqd/111:25:011
499093993524,8cyclictest23-21ksoftirqd/108:35:001
49908599353,8cyclictest235-21systemd-journal10:21:100
499085993522,9cyclictest12-21ksoftirqd/010:53:210
49910099347,18cyclictest131rcu_preempt07:25:043
499100993423,7cyclictest507873-21sed09:55:023
499100993420,9cyclictest1-21systemd08:35:013
499100993418,12cyclictest37-21ksoftirqd/308:50:003
49909399346,6cyclictest131rcu_preempt07:10:021
49909399346,6cyclictest131rcu_preempt07:10:021
499093993422,9cyclictest23-21ksoftirqd/112:25:051
499093993421,9cyclictest23-21ksoftirqd/109:05:001
49909399341,3cyclictest131rcu_preempt07:15:161
49908599341,29cyclictest0-21swapper/008:20:330
49910099333,19cyclictest37-21ksoftirqd/309:10:403
499100993322,6cyclictest306-21in:imuxsock07:20:003
499085993323,7cyclictest12-21ksoftirqd/007:10:020
499085993323,7cyclictest12-21ksoftirqd/007:10:020
49908599331,28cyclictest0-21swapper/010:10:580
499100993224,5cyclictest511577-21latency_hist11:10:023
499100993220,8cyclictest37-21ksoftirqd/308:40:003
49909399321,7cyclictest131rcu_preempt12:10:001
49909399316,19cyclictest502678-21cron08:20:001
499100993022,6cyclictest501934-21systemctl08:05:003
499100993022,5cyclictest271-21cron12:10:013
49910099301,5cyclictest131rcu_preempt12:04:593
499100992922,5cyclictest503234-21kworker/3:109:20:053
499100992918,8cyclictest501362-21sshd07:50:153
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional