You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-23 - 15:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Sun Nov 23, 2025 12:45:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
395043999812,48cyclictest363-21snmpd12:33:192
3950439997615,31cyclictest3961961-21date10:50:022
3950433997232,36cyclictest1-21systemd10:35:020
3950439997029,37cyclictest3951641-21sshd07:33:482
3950433996833,31cyclictest1-21systemd08:50:000
395043999662,60cyclictest3965678-21sshd12:03:142
3950443996557,6cyclictest37-21ksoftirqd/308:50:003
395043999652,59cyclictest363-21snmpd10:54:232
395043399652,6cyclictest1-21systemd11:42:130
395043999641,57cyclictest3956334-21sshd09:04:082
3950443996312,36cyclictest37-21ksoftirqd/307:50:023
395043999632,56cyclictest3965462-21/usr/sbin/munin12:00:012
395043999632,36cyclictest1-21systemd12:22:192
3950439996320,39cyclictest3956204-21latency_hist09:00:032
395043999631,57cyclictest3957170-21munin-run09:20:022
395043999623,51cyclictest3951148-21modprobe07:25:012
395043399621,5cyclictest3953485-21latency_hist08:10:020
395043399621,5cyclictest3953485-21latency_hist08:10:010
395043999612,54cyclictest3964636-21sshd11:41:132
395043999612,54cyclictest3958945-21munin-run09:55:012
395043999612,54cyclictest3958945-21munin-run09:55:002
395043999612,53cyclictest3956817-21latency_hist09:10:032
395043999612,53cyclictest3956817-21latency_hist09:10:022
395043999612,53cyclictest3955582-21munin-run08:50:002
395043999611,55cyclictest3965182-21munin-run11:55:022
3950433996126,31cyclictest3951062-21latency_hist07:20:030
3950433996117,12cyclictest3965729-21sed12:05:020
395043999602,54cyclictest3963392-21munin-run11:20:012
395043999602,53cyclictest3959729-21munin-run10:10:002
395043999602,53cyclictest3952214-21munin-run07:45:012
395043999602,53cyclictest363-21snmpd09:47:112
395043999602,53cyclictest363-21snmpd09:39:082
3950439996023,33cyclictest3950855-21sshd07:17:492
395043999601,6cyclictest3967008-21munin-run12:30:012
395043999601,55cyclictest1-21systemd11:10:002
395043999601,54cyclictest3961565-21munin-run10:45:012
395043999601,54cyclictest3960880-21cat10:30:022
395043999601,4cyclictest3960195-21sshd10:15:082
395043999601,20cyclictest3961007-21cron10:35:002
395043999591,54cyclictest3964918-21munin-run11:50:012
395043999591,54cyclictest3957891-21sshd09:34:532
395043999591,54cyclictest363-21snmpd10:14:152
395043999591,4cyclictest363-21snmpd11:04:242
395043999591,14cyclictest363-21snmpd08:05:502
395043999591,14cyclictest363-21snmpd08:05:502
395043999582,52cyclictest3963629-21munin-run11:25:002
395043999582,52cyclictest3956867-21munin-run09:14:592
395043999582,52cyclictest363-21snmpd07:38:442
395043999582,51cyclictest3964428-21munin-run11:40:012
395043999582,51cyclictest363-21snmpd08:36:542
395043999581,52cyclictest363-21snmpd11:26:112
395043999581,51cyclictest3951382-21sshd07:26:492
3950433995824,5cyclictest3961367-21latency_hist10:40:020
395043999572,50cyclictest3967582-21cat12:40:022
395043999571,52cyclictest3954521-21cron08:30:002
395043999571,51cyclictest363-21snmpd12:10:492
3950433995723,5cyclictest3954932-21latency_hist08:35:030
3950433995723,5cyclictest3954932-21latency_hist08:35:020
3950433995722,5cyclictest3956385-21latency_hist09:05:010
395043999562,49cyclictest363-21snmpd11:33:152
395043999562,49cyclictest363-21snmpd11:33:142
395043999562,49cyclictest363-21snmpd07:58:462
395043999562,49cyclictest0-21swapper/212:09:372
395043999561,3cyclictest363-21snmpd08:53:582
3950443995513,12cyclictest37-21ksoftirqd/311:40:013
395043999552,49cyclictest1-21systemd08:15:012
395043999551,49cyclictest363-21snmpd11:10:312
395043999551,49cyclictest363-21snmpd07:45:472
3950433995520,31cyclictest12-21ksoftirqd/012:15:130
3950433995520,13cyclictest12-21ksoftirqd/012:40:020
395044399541,8cyclictest3957138-21grep09:20:003
395043999543,5cyclictest131rcu_preempt09:25:022
395043999543,5cyclictest131rcu_preempt09:25:022
395043999543,46cyclictest363-21snmpd10:36:372
395043999542,48cyclictest363-21snmpd08:43:062
395043999542,47cyclictest3958418-21sshd09:43:092
395043999541,4cyclictest363-21snmpd09:29:052
395043999541,49cyclictest363-21snmpd12:19:162
395043999541,49cyclictest363-21snmpd08:23:552
395043999541,48cyclictest3950634-21munin-run07:15:012
395043999541,48cyclictest363-21snmpd08:31:042
395043999541,48cyclictest363-21snmpd08:31:032
395043999541,47cyclictest363-21snmpd07:54:472
3950433995423,4cyclictest3959199-21latency_hist10:00:020
3950433995420,8cyclictest12-21ksoftirqd/012:34:130
3950433995417,10cyclictest3957752-21date09:30:010
3950443995334,14cyclictest235-21systemd-journal09:29:593
3950443995313,14cyclictest37-21ksoftirqd/312:40:023
3950443995311,5cyclictest131rcu_preempt07:45:013
3950443995236,11cyclictest3956073-21cron09:00:003
395044399523,13cyclictest37-21ksoftirqd/308:00:023
3950443995213,9cyclictest37-21ksoftirqd/310:45:093
395043999522,44cyclictest3953416-21latency_hist08:05:032
395043999521,4cyclictest363-21snmpd10:55:102
395043999521,4cyclictest363-21snmpd10:23:212
395043999521,2cyclictest0-21swapper/208:15:182
395043399523,24cyclictest0-21swapper/008:30:010
3950433995224,4cyclictest3957402-21latency_hist09:25:020
3950433995224,4cyclictest3957402-21latency_hist09:25:010
3950433995219,6cyclictest16850irq/134-800000009:40:010
395043999511,44cyclictest3959449-21modprobe10:02:082
395044399507,12cyclictest37-21ksoftirqd/308:05:013
395044399501,7cyclictest0-21swapper/310:35:033
395044399501,6cyclictest0-21swapper/308:25:003
3950433995033,12cyclictest1-21systemd08:05:010
395044399498,9cyclictest37-21ksoftirqd/309:40:003
395044399498,6cyclictest131rcu_preempt12:20:133
395044399493,9cyclictest37-21ksoftirqd/310:30:013
3950433994931,14cyclictest1-21systemd09:50:010
395043399491,29cyclictest0-21swapper/007:58:310
395044399483,8cyclictest3955330-21unin-run08:45:023
395043999482,41cyclictest3959209-21systemctl10:00:002
3950437994833,12cyclictest3965441-21tr12:00:011
3950437994833,11cyclictest3953989-21cron08:20:011
3950437994832,12cyclictest1-21systemd11:10:011
395043399483,32cyclictest3950436-21latency_hist07:10:020
3950433994827,4cyclictest0-21swapper/010:14:180
395043399481,29cyclictest0-21swapper/011:28:330
395043399481,28cyclictest0-21swapper/012:22:580
395043399481,28cyclictest0-21swapper/010:29:200
395044399477,11cyclictest37-21ksoftirqd/307:30:013
395044399475,7cyclictest37-21ksoftirqd/309:35:013
395044399474,7cyclictest37-21ksoftirqd/311:35:003
395044399474,7cyclictest37-21ksoftirqd/311:34:593
3950443994712,5cyclictest131rcu_preempt12:15:013
3950437994734,9cyclictest3957133-21cat09:20:001
3950433994726,4cyclictest0-21swapper/010:19:150
3950433994724,4cyclictest0-21swapper/012:13:040
395043399471,27cyclictest0-21swapper/009:16:440
395044399467,13cyclictest37-21ksoftirqd/311:00:013
395044399465,7cyclictest37-21ksoftirqd/310:00:023
395044399465,11cyclictest37-21ksoftirqd/309:10:013
395044399465,11cyclictest37-21ksoftirqd/309:10:003
3950443994634,7cyclictest37-21ksoftirqd/309:05:003
3950443994628,13cyclictest272-21dbus-daemon09:50:013
3950443994611,6cyclictest131rcu_preempt07:25:013
3950433994638,5cyclictest16950irq/224-800000012:10:000
3950433994632,10cyclictest3963338-21cron11:19:590
3950433994630,12cyclictest1-21systemd07:55:000
3950433994630,11cyclictest1-21systemd12:00:020
395043399461,4cyclictest0-21swapper/011:38:180
395043399461,28cyclictest0-21swapper/007:41:270
395043399461,27cyclictest0-21swapper/007:11:220
395044399453,9cyclictest37-21ksoftirqd/312:20:003
3950443994534,8cyclictest37-21ksoftirqd/309:25:013
3950443994534,8cyclictest37-21ksoftirqd/309:25:003
395044399452,8cyclictest37-21ksoftirqd/312:04:593
395044399451,5cyclictest131rcu_preempt11:30:033
3950443994510,6cyclictest131rcu_preempt11:55:003
3950437994534,7cyclictest3963338-21cron11:19:591
3950437994527,13cyclictest235-21systemd-journal07:50:011
3950437994526,14cyclictest235-21systemd-journal10:55:001
3950433994532,9cyclictest1-21systemd08:55:010
3950433994527,13cyclictest1-21systemd11:35:010
3950433994527,13cyclictest1-21systemd11:35:000
3950443994424,15cyclictest3950890-21run-parts07:20:013
3950443994412,5cyclictest131rcu_preempt10:15:083
3950437994431,10cyclictest235-21systemd-journal12:10:021
3950437994428,11cyclictest306-21in:imuxsock07:10:021
395043399441,28cyclictest0-21swapper/011:23:280
3950443994332,8cyclictest37-21ksoftirqd/310:05:003
3950443994327,13cyclictest37-21ksoftirqd/312:00:013
3950443994317,5cyclictest131rcu_preempt12:10:013
3950443994314,5cyclictest131rcu_preempt11:25:013
395043999431,2cyclictest0-21swapper/207:10:012
3950433994331,8cyclictest235-21systemd-journal11:09:590
3950433994327,11cyclictest235-21systemd-journal10:50:020
395043399432,36cyclictest3958907-21sshd09:50:080
395043399432,36cyclictest3958907-21sshd09:50:080
395043399431,28cyclictest0-21swapper/007:25:490
395044399424,10cyclictest37-21ksoftirqd/311:20:003
395044399422,9cyclictest37-21ksoftirqd/308:20:003
3950443994223,13cyclictest37-21ksoftirqd/311:44:593
3950443994212,5cyclictest131rcu_preempt07:35:023
3950443994211,8cyclictest131rcu_preempt11:05:013
3950443994210,6cyclictest131rcu_preempt08:15:023
3950437994229,9cyclictest3953429-21date08:05:021
3950437994225,12cyclictest1-21systemd10:20:021
3950433994235,4cyclictest16950irq/224-800000008:20:010
395043399421,4cyclictest0-21swapper/010:54:040
395044399419,10cyclictest37-21ksoftirqd/310:15:013
3950443994132,7cyclictest37-21ksoftirqd/311:15:013
3950443994127,10cyclictest37-21ksoftirqd/312:35:003
3950443994111,6cyclictest131rcu_preempt09:55:003
3950443994111,6cyclictest131rcu_preempt09:55:003
395043799416,4cyclictest131rcu_preempt09:35:011
3950437994129,9cyclictest23-21ksoftirqd/111:45:021
3950433994129,9cyclictest0-21swapper/007:50:030
3950433994114,3cyclictest131rcu_preempt09:32:100
395043399411,26cyclictest0-21swapper/009:05:200
395043399411,26cyclictest0-21swapper/009:05:190
395044399407,8cyclictest37-21ksoftirqd/308:30:013
395044399407,5cyclictest131rcu_preempt07:15:013
3950443994015,22cyclictest37-21ksoftirqd/308:35:023
3950443994015,22cyclictest37-21ksoftirqd/308:35:013
3950437994028,8cyclictest3965163-21cron11:55:011
3950437994027,9cyclictest23-21ksoftirqd/111:35:011
3950437994027,9cyclictest23-21ksoftirqd/111:35:011
3950437994026,11cyclictest23-21ksoftirqd/110:45:031
3950437994023,12cyclictest272-21dbus-daemon09:30:001
3950433994027,10cyclictest3959456-21sshd10:04:080
395043399401,6cyclictest0-21swapper/009:45:000
395044399396,5cyclictest131rcu_preempt11:10:003
3950443993930,7cyclictest37-21ksoftirqd/308:10:013
3950443993930,7cyclictest37-21ksoftirqd/308:10:003
3950443993910,5cyclictest131rcu_preempt08:38:543
3950437993929,8cyclictest23-21ksoftirqd/107:45:021
3950437993925,11cyclictest23-21ksoftirqd/109:40:001
3950437993924,12cyclictest3958600-21sed09:45:021
3950437993913,19cyclictest3964396-21munin-run11:39:591
3950433993928,8cyclictest12-21ksoftirqd/010:10:020
3950433993926,11cyclictest12-21ksoftirqd/011:05:020
395043399392,3cyclictest0-21swapper/011:15:020
3950433993922,13cyclictest1-21systemd12:30:000
395043399391,34cyclictest0-21swapper/010:25:000
395043399391,10cyclictest3962316-21modprobe11:00:010
395044399384,7cyclictest37-21ksoftirqd/307:40:013
3950443993830,6cyclictest37-21ksoftirqd/308:54:463
3950443993826,9cyclictest37-21ksoftirqd/310:40:013
3950443993823,11cyclictest37-21ksoftirqd/312:29:593
3950437993828,6cyclictest3955040-21sendmail08:40:001
3950437993822,11cyclictest272-21dbus-daemon11:30:001
3950437993821,13cyclictest272-21dbus-daemon12:10:011
3950437993821,11cyclictest272-21dbus-daemon12:05:001
3950433993830,5cyclictest3954266-21cron08:25:000
3950433993825,10cyclictest235-21systemd-journal07:35:020
3950433993824,9cyclictest12-21ksoftirqd/008:15:000
395043399382,33cyclictest363-21snmpd11:54:340
3950443993727,6cyclictest3952693-21cron07:54:593
3950443993725,9cyclictest37-21ksoftirqd/310:55:013
395044399371,5cyclictest0-21swapper/310:25:013
395043799377,8cyclictest23-21ksoftirqd/108:10:001
395043799377,8cyclictest23-21ksoftirqd/108:09:591
3950437993726,8cyclictest3966469-21cat12:20:011
3950437993725,6cyclictest306-21in:imuxsock08:15:001
3950437993723,11cyclictest23-21ksoftirqd/109:15:011
3950437993723,10cyclictest1-21systemd12:30:011
3950437993722,9cyclictest306-21in:imuxsock11:25:011
3950433993726,7cyclictest235-21systemd-journal10:42:060
3950433993722,11cyclictest235-21systemd-journal07:25:000
395044399364,8cyclictest37-21ksoftirqd/310:10:023
395044399364,6cyclictest131rcu_preempt10:45:013
395043799368,22cyclictest3956492-21latency_hist09:05:021
395043799364,4cyclictest131rcu_preempt07:20:001
3950437993627,5cyclictest306-21in:imuxsock07:40:001
395043799362,3cyclictest131rcu_preempt10:30:011
3950437993620,12cyclictest272-21dbus-daemon09:55:021
3950437993620,12cyclictest272-21dbus-daemon09:55:011
3950433993627,5cyclictest19050irq/34-mmc107:38:080
3950433993623,10cyclictest3964965-21latency_hist11:50:020
395044399359,4cyclictest131rcu_preempt09:10:033
3950443993521,11cyclictest3965116-21sed11:50:033
3950443993520,10cyclictest272-21dbus-daemon09:45:003
395043799356,21cyclictest3954788-21munin-plugin-st08:35:001
395043799356,21cyclictest3954788-21munin-plugin-st08:35:001
3950437993526,6cyclictest3961272-21sh10:40:001
3950437993525,7cyclictest23-21ksoftirqd/109:25:011
3950437993525,7cyclictest23-21ksoftirqd/109:25:001
3950437993522,9cyclictest306-21in:imuxsock08:00:001
3950437993522,10cyclictest3951404-21modprobe07:30:011
3950437993521,10cyclictest23-21ksoftirqd/112:40:011
3950437993514,15cyclictest272-21dbus-daemon10:10:011
3950433993525,7cyclictest12-21ksoftirqd/009:00:010
395043399351,30cyclictest0-21swapper/008:43:040
395043799343,25cyclictest3961021-21munin-plugin-st10:35:011
3950437993427,4cyclictest3960682-21latency_hist10:25:011
3950437993425,6cyclictest3950428-21cyclictest07:10:011
3950437993424,8cyclictest23-21ksoftirqd/111:15:001
3950437993424,7cyclictest23-21ksoftirqd/108:24:591
3950437993422,9cyclictest23-21ksoftirqd/110:15:011
3950437993421,9cyclictest23-21ksoftirqd/107:35:001
3950437993419,10cyclictest272-21dbus-daemon08:50:011
3950437993413,16cyclictest272-21dbus-daemon12:35:001
395043399341,29cyclictest0-21swapper/009:10:490
395043399341,29cyclictest0-21swapper/008:36:230
395043799336,16cyclictest3956595-21cron09:10:001
395043799336,16cyclictest3956595-21cron09:10:001
395043799335,23cyclictest235-21systemd-journal09:50:001
395043799335,22cyclictest3959206-21grep09:59:591
395043799334,4cyclictest131rcu_preempt08:45:011
395043799334,23cyclictest3956081-21systemctl09:00:001
395043799333,24cyclictest3962317-21munin-plugin-st11:00:011
3950437993321,10cyclictest23-21ksoftirqd/111:50:001
3950437993319,12cyclictest23-21ksoftirqd/108:55:021
3950437993316,14cyclictest281-21systemd-logind10:05:011
3950437993225,4cyclictest3966841-21latency_hist12:25:021
3950443993118,9cyclictest3950513-21sort07:10:023
3950437993121,7cyclictest23-21ksoftirqd/111:00:021
3950437993121,6cyclictest3961790-21cron10:50:001
395043799311,5cyclictest131rcu_preempt07:25:011
3950437993018,9cyclictest3954547-21pluginstate08:30:021
3950437992915,10cyclictest23-21ksoftirqd/107:55:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional