You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-02 - 18:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Wed Jul 02, 2025 12:45:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
285386799741,5cyclictest0-21swapper/007:15:010
285386799691,63cyclictest2855984-21latency_hist07:50:020
285386799691,31cyclictest2857354-21wc08:15:020
285386799684,59cyclictest2859393-21latency_hist08:55:020
285386799662,59cyclictest2868077-21sshd11:42:140
285386799662,59cyclictest2868077-21sshd11:42:130
285386799651,60cyclictest2859468-21sshd08:55:130
2853867996430,28cyclictest306-21in:imuxsock10:50:010
285386799641,59cyclictest2854300-21sshd07:18:080
285386799641,57cyclictest2864191-21sshd10:29:320
2853867996331,29cyclictest272-21dbus-daemon09:30:000
285386799632,57cyclictest2870150-21sshd12:21:130
285386799632,55cyclictest2861358-21munin-run09:35:010
285386799631,58cyclictest2866251-21sshd11:08:130
285387699625,10cyclictest30-21ksoftirqd/209:20:002
285386799621,57cyclictest2867798-21sshd11:38:120
285386799621,57cyclictest2861571-21sshd09:38:130
285386799621,57cyclictest2860803-21sshd09:20:130
285386799621,56cyclictest2868331-21sshd11:48:130
285386799621,55cyclictest2855400-21munin-run07:40:010
285386799615,43cyclictest2869889-21cron12:20:000
285386799612,54cyclictest2862924-21munin-run10:05:010
285386799611,57cyclictest2856125-21sshd07:54:090
285386799611,55cyclictest2865712-21sshd10:56:130
285386799611,55cyclictest2860538-21sshd09:18:130
285386799611,55cyclictest2860301-21munin-run09:15:010
285386799602,53cyclictest2863162-21munin-run10:10:000
285386799602,53cyclictest2856637-21sshd08:00:100
285386799602,53cyclictest2855625-21sendmail-mta07:44:210
285386799601,55cyclictest2869625-21sshd12:14:480
285386799601,55cyclictest2869625-21sshd12:14:470
285386799601,55cyclictest2868823-21sshd11:55:140
285386799601,55cyclictest2868823-21sshd11:55:130
285386799601,55cyclictest2866492-21sshd11:10:130
285386799601,55cyclictest2865986-21sshd11:00:130
285386799601,55cyclictest2865495-21munin-run10:55:010
285386799601,55cyclictest2865495-21munin-run10:55:010
285386799601,54cyclictest2860046-21munin-run09:10:000
285386799601,54cyclictest2854568-21sshd07:20:090
285386799601,53cyclictest48-21kcompactd010:23:010
285386799601,53cyclictest48-21kcompactd010:23:010
285386799595,48cyclictest235-21systemd-journal12:05:000
285386799594,46cyclictest2863663-21cron10:20:000
285386799592,53cyclictest2857951-21sshd08:29:140
285386799592,52cyclictest2867084-21munin-run11:25:010
285386799591,54cyclictest2868621-21/usr/sbin/munin11:55:010
285386799591,54cyclictest2864421-21sshd10:30:130
285386799591,54cyclictest2858721-21sshd08:42:130
285386799591,53cyclictest2864871-21date10:40:020
285386799591,53cyclictest2864871-21date10:40:020
285386799584,50cyclictest2866918-21cat11:20:020
285386799583,50cyclictest2870420-21cron12:30:000
285386799582,51cyclictest2858981-21sshd08:49:130
285386799581,53cyclictest2864990-21munin-run10:45:000
285386799581,53cyclictest2862111-21sshd09:49:260
285386799581,52cyclictest363-21snmpd09:03:220
285387699576,5cyclictest131rcu_preempt11:35:022
2853872995734,18cyclictest2867071-21munin-run11:25:001
285386799572,50cyclictest2862380-21munin-run09:55:010
285386799572,50cyclictest2858441-21sshd08:37:130
2853872995639,13cyclictest2859240-21cron08:55:001
2853872995634,18cyclictest2860807-21sshd09:21:131
2853872995629,23cyclictest2855245-21latency_hist07:35:021
285386799563,47cyclictest2867309-21cron11:30:000
285386799563,47cyclictest2867309-21cron11:30:000
285386799562,50cyclictest2854862-21systemd07:30:010
285386799561,51cyclictest2857390-21sshd08:15:140
285386799561,51cyclictest2856903-21sshd08:09:090
285388199553,8cyclictest37-21ksoftirqd/309:30:013
285387699556,7cyclictest30-21ksoftirqd/210:00:022
285386799551,50cyclictest363-21snmpd11:32:220
285386799551,50cyclictest2857683-21sshd08:21:130
285386799551,49cyclictest48-21kcompactd009:59:100
285388199546,10cyclictest37-21ksoftirqd/308:55:023
2853881995416,9cyclictest37-21ksoftirqd/307:50:013
2853872995440,10cyclictest2869873-21sshd12:17:131
2853872995436,13cyclictest2865456-21sshd10:50:131
2853872995436,13cyclictest2865456-21sshd10:50:131
2853872995435,15cyclictest2869728-21cat12:15:021
2853872995435,15cyclictest2869728-21cat12:15:021
2853872995435,15cyclictest2868847-21cron12:00:001
2853872995435,15cyclictest2868847-21cron12:00:001
2853872995432,18cyclictest2857288-21latency_hist08:15:021
2853872995432,17cyclictest2862626-21modprobe10:00:011
2853872995432,17cyclictest2861631-21true09:40:011
2853872995429,20cyclictest2868389-21latency_hist11:50:011
2853867995427,22cyclictest2855094-21cron07:35:010
285386799541,49cyclictest48-21kcompactd009:40:180
285386799541,48cyclictest2869364-21sshd12:08:130
2853881995335,14cyclictest1-21systemd10:15:013
285387699537,9cyclictest30-21ksoftirqd/211:10:002
2853876995334,14cyclictest1-21systemd07:25:002
2853872995338,9cyclictest2863942-21cron10:25:011
2853872995338,9cyclictest2863942-21cron10:25:001
2853872995335,12cyclictest1-21systemd10:10:011
2853872995334,14cyclictest2863661-21cron10:20:001
2853872995333,16cyclictest2854131-21latency_hist07:15:011
2853872995333,14cyclictest2860549-21cron09:20:001
2853872995331,14cyclictest2861077-21sshd09:29:241
2853872995330,19cyclictest2862041-21cat09:45:021
285386799532,47cyclictest2870899-21sshd12:35:130
285388199524,6cyclictest37-21ksoftirqd/309:15:033
2853881995221,8cyclictest37-21ksoftirqd/308:05:113
285388199522,13cyclictest37-21ksoftirqd/312:05:013
2853876995218,9cyclictest30-21ksoftirqd/207:44:072
2853872995240,9cyclictest48-21kcompactd010:47:261
2853872995239,10cyclictest2862893-21sshd10:04:271
2853872995238,9cyclictest861385-21sendmail-mta12:24:201
2853872995238,10cyclictest2856130-21cron07:55:001
2853872995237,11cyclictest2857954-21cron08:30:001
2853872995235,11cyclictest2870420-21cron12:30:011
2853872995233,14cyclictest2855878-21cron07:50:001
2853872995233,14cyclictest2855368-21systemctl07:40:001
2853872995230,18cyclictest2870875-21latency_hist12:35:021
2853872995228,17cyclictest2856926-21cut08:10:001
285386799521,47cyclictest363-21snmpd10:11:200
285386799521,47cyclictest2858185-21sshd08:31:130
285388199514,11cyclictest37-21ksoftirqd/310:20:003
285387699517,7cyclictest30-21ksoftirqd/210:30:002
285387699515,6cyclictest30-21ksoftirqd/208:38:132
285387699515,10cyclictest30-21ksoftirqd/209:05:012
2853872995134,14cyclictest363-21snmpd11:18:561
2853872995134,11cyclictest2858757-21true08:45:011
2853872995133,15cyclictest2862171-21sed09:50:021
2853872995132,13cyclictest2869566-21latency_hist12:10:031
2853872995131,17cyclictest2866360-21latency_hist11:10:011
2853872995131,16cyclictest2854729-21cat07:25:021
2853872995131,15cyclictest2865168-21cat10:45:021
2853872995131,14cyclictest2867571-21munin-plugin-st11:35:001
2853872995126,20cyclictest2859531-21basename09:00:001
285386799511,45cyclictest363-21snmpd12:32:540
285388199509,9cyclictest37-21ksoftirqd/311:15:023
285388199507,8cyclictest37-21ksoftirqd/311:55:003
285387699505,8cyclictest30-21ksoftirqd/211:40:012
2853876995031,14cyclictest272-21dbus-daemon12:20:002
2853872995041,4cyclictest2868118-21munin-run11:45:011
2853872995041,4cyclictest2868118-21munin-run11:45:011
2853872995037,8cyclictest2867308-21cron11:30:001
2853872995037,8cyclictest2867308-21cron11:30:001
2853872995034,12cyclictest2864208-21munin-plugin-st10:30:001
2853872995031,15cyclictest2858289-21sed08:35:021
2853872995029,17cyclictest2855770-21latency_hist07:45:021
2853872995029,16cyclictest2868746-21latency_hist11:55:021
2853872995028,17cyclictest2856634-21modprobe08:00:021
2853872995023,23cyclictest2860199-21latency_hist09:10:021
285386799503,41cyclictest2856386-21modprobe08:00:000
285388199498,8cyclictest37-21ksoftirqd/310:30:023
285388199496,8cyclictest37-21ksoftirqd/311:45:023
285388199496,8cyclictest37-21ksoftirqd/311:45:013
285388199496,10cyclictest37-21ksoftirqd/307:30:003
285388199495,7cyclictest37-21ksoftirqd/309:10:023
285388199494,8cyclictest37-21ksoftirqd/311:05:003
285388199494,7cyclictest37-21ksoftirqd/309:05:023
2853881994935,10cyclictest235-21systemd-journal09:53:143
2853881994933,12cyclictest1-21systemd08:15:013
2853881994931,13cyclictest1-21systemd07:35:013
285387699497,9cyclictest30-21ksoftirqd/211:30:012
285387699497,9cyclictest30-21ksoftirqd/211:30:012
285387699496,7cyclictest30-21ksoftirqd/210:05:022
285387699494,14cyclictest30-21ksoftirqd/210:40:002
285387699494,14cyclictest30-21ksoftirqd/210:40:002
2853876994933,12cyclictest1-21systemd07:15:012
2853872994937,9cyclictest363-21snmpd11:39:311
2853872994936,10cyclictest48-21kcompactd009:14:131
2853872994936,10cyclictest2857446-21munin-run08:20:001
2853872994935,11cyclictest281-21systemd-logind08:50:011
2853872994934,11cyclictest363-21snmpd09:54:021
2853872994933,11cyclictest2858513-21munin-run08:40:011
2853872994931,13cyclictest363-21snmpd11:14:281
2853872994929,17cyclictest2869269-21cat12:05:011
2853872994927,18cyclictest2856511-21latency_hist08:00:021
2853872994925,21cyclictest2870924-21cron12:40:001
2853872994925,20cyclictest2863622-21latency_hist10:15:031
2853872994925,19cyclictest2855023-21sed07:30:021
2853872994922,21cyclictest2866178-21sed11:05:021
285388199486,6cyclictest37-21ksoftirqd/310:02:133
285388199483,13cyclictest37-21ksoftirqd/310:40:013
285388199483,13cyclictest37-21ksoftirqd/310:40:013
285387699486,6cyclictest30-21ksoftirqd/212:30:022
285387699486,5cyclictest30-21ksoftirqd/208:45:022
2853872994834,10cyclictest2857689-21sshd08:22:131
2853872994831,13cyclictest363-21snmpd09:32:341
2853872994830,14cyclictest363-21snmpd10:31:211
2853872994826,17cyclictest2865853-21latency_hist11:00:011
2853872994826,17cyclictest2859788-21grep09:05:011
2853872994826,15cyclictest2864915-21latency_hist10:40:021
2853872994826,15cyclictest2864915-21latency_hist10:40:021
285386799486,37cyclictest2853866-21latency_hist07:10:010
285388199475,10cyclictest37-21ksoftirqd/309:35:013
285387699478,8cyclictest30-21ksoftirqd/210:25:012
285387699478,8cyclictest30-21ksoftirqd/210:25:002
285387699476,7cyclictest30-21ksoftirqd/207:55:012
285387699476,6cyclictest30-21ksoftirqd/210:10:022
285387699476,4cyclictest131rcu_preempt12:23:132
2853872994734,10cyclictest2854307-21sshd07:19:091
285388199466,6cyclictest37-21ksoftirqd/308:05:013
285388199466,5cyclictest131rcu_preempt07:10:013
285387699467,10cyclictest30-21ksoftirqd/212:35:002
285387699466,9cyclictest30-21ksoftirqd/207:50:012
285387699466,10cyclictest30-21ksoftirqd/207:35:012
2853876994633,9cyclictest1-21systemd09:45:002
2853876994631,11cyclictest272-21dbus-daemon08:10:002
285387699462,10cyclictest30-21ksoftirqd/208:25:002
285388199459,6cyclictest131rcu_preempt10:50:013
285388199453,10cyclictest37-21ksoftirqd/310:35:013
2853881994530,11cyclictest235-21systemd-journal08:59:133
2853881994529,12cyclictest235-21systemd-journal07:17:003
285387699458,5cyclictest131rcu_preempt08:15:012
285387699456,7cyclictest30-21ksoftirqd/211:15:012
285387699455,8cyclictest30-21ksoftirqd/211:50:002
285388199449,6cyclictest131rcu_preempt12:14:133
285388199449,6cyclictest131rcu_preempt12:14:133
285388199446,8cyclictest37-21ksoftirqd/311:30:013
285388199446,8cyclictest37-21ksoftirqd/311:30:013
285388199444,7cyclictest37-21ksoftirqd/312:10:023
285388199443,7cyclictest37-21ksoftirqd/307:45:013
2853881994415,5cyclictest131rcu_preempt07:25:003
285387699449,5cyclictest131rcu_preempt08:20:012
285387699445,7cyclictest30-21ksoftirqd/210:15:022
285388199434,9cyclictest37-21ksoftirqd/307:15:003
285388199433,8cyclictest37-21ksoftirqd/311:20:023
2853881994334,7cyclictest37-21ksoftirqd/310:55:013
2853881994334,7cyclictest37-21ksoftirqd/310:55:013
2853881994333,6cyclictest306-21in:imuxsock09:25:013
2853881994330,10cyclictest235-21systemd-journal07:58:093
2853881994317,8cyclictest37-21ksoftirqd/309:43:563
285387699438,5cyclictest131rcu_preempt10:55:022
285387699438,5cyclictest131rcu_preempt10:55:022
285387699436,10cyclictest30-21ksoftirqd/207:17:012
285387699435,7cyclictest30-21ksoftirqd/208:05:022
2853876994333,8cyclictest291rcuc/207:30:022
285388199428,5cyclictest131rcu_preempt08:40:013
285388199425,8cyclictest37-21ksoftirqd/308:45:003
285388199425,7cyclictest37-21ksoftirqd/310:25:003
285388199425,7cyclictest37-21ksoftirqd/310:25:003
2853881994218,15cyclictest361rcuc/310:09:053
285387699429,5cyclictest131rcu_preempt11:05:002
285387699426,7cyclictest30-21ksoftirqd/207:40:012
285387699425,7cyclictest30-21ksoftirqd/212:00:022
285387699425,7cyclictest30-21ksoftirqd/212:00:022
285387699425,10cyclictest30-21ksoftirqd/211:20:012
285387699423,8cyclictest30-21ksoftirqd/211:45:012
285387699423,8cyclictest30-21ksoftirqd/211:45:012
2853872994225,14cyclictest2853873-21latency_hist07:10:011
285388199417,11cyclictest37-21ksoftirqd/308:17:003
285388199414,8cyclictest37-21ksoftirqd/308:50:013
285388199413,9cyclictest37-21ksoftirqd/310:45:003
285387699419,5cyclictest131rcu_preempt08:00:012
285387699418,5cyclictest131rcu_preempt11:25:012
2853876994125,12cyclictest1-21systemd12:10:002
285388199403,6cyclictest37-21ksoftirqd/308:35:023
2853881994029,8cyclictest2870165-21sshd12:24:483
2853881994027,10cyclictest235-21systemd-journal09:20:003
285388199401,10cyclictest37-21ksoftirqd/311:50:003
285387699408,4cyclictest131rcu_preempt08:48:412
285387699403,7cyclictest30-21ksoftirqd/208:30:012
2853876994029,7cyclictest2865730-21cron11:00:002
2853876994028,9cyclictest2863638-21cron10:17:002
2853876994025,10cyclictest30-21ksoftirqd/212:40:002
285388199396,7cyclictest37-21ksoftirqd/310:59:133
285388199395,10cyclictest37-21ksoftirqd/312:26:463
285388199393,8cyclictest37-21ksoftirqd/309:50:003
285388199393,7cyclictest37-21ksoftirqd/312:00:033
285388199393,7cyclictest37-21ksoftirqd/312:00:023
2853881993931,6cyclictest361rcuc/312:20:003
2853881993929,8cyclictest361rcuc/311:08:313
2853881993927,9cyclictest2862856-21date10:00:033
2853881993926,10cyclictest2867049-21sshd11:24:133
285387699392,6cyclictest30-21ksoftirqd/209:55:012
2853876993925,10cyclictest2861676-21cut09:40:022
2853876993923,11cyclictest2860038-21cron09:10:002
285388199389,6cyclictest131rcu_preempt09:38:133
285388199385,6cyclictest131rcu_preempt07:55:003
285388199384,6cyclictest37-21ksoftirqd/308:30:003
2853881993810,4cyclictest131rcu_preempt08:20:133
285387699383,7cyclictest30-21ksoftirqd/209:21:132
2853876993826,8cyclictest235-21systemd-journal10:45:002
285387699381,9cyclictest30-21ksoftirqd/210:50:002
2853881993711,8cyclictest37-21ksoftirqd/311:38:143
2853876993729,5cyclictest2864455-21latency_hist10:35:002
2853876993720,12cyclictest272-21dbus-daemon09:00:002
285388199369,6cyclictest37-21ksoftirqd/311:30:483
2853881993623,9cyclictest37-21ksoftirqd/312:30:133
2853881993623,10cyclictest2855345-21cron07:40:003
285387699362,8cyclictest30-21ksoftirqd/212:05:022
285387699362,7cyclictest30-21ksoftirqd/209:30:002
2853876993627,7cyclictest30-21ksoftirqd/208:55:002
285387699361,5cyclictest131rcu_preempt09:13:492
285387699357,7cyclictest30-21ksoftirqd/208:30:282
2853876993523,8cyclictest2862129-21grep09:50:002
285387699344,4cyclictest131rcu_preempt12:15:022
285387699344,4cyclictest131rcu_preempt12:15:022
285387699341,6cyclictest0-21swapper/209:31:312
2853881993324,5cyclictest2870958-21sendmail12:40:013
2853876993220,9cyclictest306-21in:imuxsock11:55:002
2853876993219,11cyclictest30-21ksoftirqd/207:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional