You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-19 - 16:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Wed Nov 19, 2025 12:45:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3454002997718,15cyclictest0-21swapper/109:50:001
3454006997318,11cyclictest30-21ksoftirqd/209:40:002
3454002997021,16cyclictest3458935-21sed08:45:011
3454002997015,16cyclictest3462797-21modprobe10:00:011
3454002996924,16cyclictest3459435-21date08:55:021
3454002996916,14cyclictest3467430-21cut11:25:021
3453997996763,2cyclictest0-21swapper/007:10:020
3454002996619,14cyclictest3464860-21/usr/sbin/munin10:40:011
3454002996618,17cyclictest3471169-21systemd-run12:40:011
3454011996421,5cyclictest306-21in:imuxsock07:35:003
3454002996417,15cyclictest3458461-21cat08:35:021
3454002996417,15cyclictest3456303-21latency_hist07:55:021
3454011996323,9cyclictest37-21ksoftirqd/309:55:003
3454011996322,10cyclictest37-21ksoftirqd/307:15:003
3454011996320,10cyclictest37-21ksoftirqd/311:55:013
3454011996316,15cyclictest272-21dbus-daemon11:05:003
3454011996226,5cyclictest3456198-21latency_hist07:50:033
3454002996124,8cyclictest3462026-21munin-run09:45:011
3454002996121,13cyclictest3464488-21ls10:30:021
3454002996121,13cyclictest3464488-21ls10:30:021
3454002996019,15cyclictest3455939-21sed07:45:011
3453997995941,14cyclictest235-21systemd-journal07:40:010
3454011995827,5cyclictest3465633-21munin-plugin-st10:55:003
3454011995826,10cyclictest37-21ksoftirqd/312:25:013
3454011995826,10cyclictest37-21ksoftirqd/312:25:013
345400299583,9cyclictest131rcu_preempt11:00:021
345400299583,9cyclictest131rcu_preempt11:00:021
3454002995838,14cyclictest3466179-21munin-run11:05:011
345399799582,9cyclictest131rcu_preempt10:10:020
3454011995724,8cyclictest272-21dbus-daemon11:06:353
345400699578,13cyclictest30-21ksoftirqd/209:10:032
3454011995618,12cyclictest272-21dbus-daemon09:05:013
3454011995616,21cyclictest37-21ksoftirqd/311:15:023
3454011995616,21cyclictest37-21ksoftirqd/311:15:023
3454006995638,16cyclictest30-21ksoftirqd/210:30:032
3454006995637,14cyclictest235-21systemd-journal10:45:002
3454006995637,14cyclictest235-21systemd-journal10:45:002
3454002995635,17cyclictest363-21snmpd07:58:041
3454011995521,7cyclictest3469504-21cat12:05:023
3454006995510,36cyclictest3461195-21kworker/u8:009:50:012
3454002995542,10cyclictest3468294-21munin-run11:45:001
345400699547,8cyclictest30-21ksoftirqd/210:15:012
3454006995420,9cyclictest30-21ksoftirqd/210:00:012
3454002995438,11cyclictest3464598-21munin-run10:35:001
3453997995433,15cyclictest272-21dbus-daemon12:35:000
345400699536,4cyclictest131rcu_preempt09:45:002
3454006995338,11cyclictest30-21ksoftirqd/209:00:012
345400699531,6cyclictest0-21swapper/212:25:022
345400699531,6cyclictest0-21swapper/212:25:022
3454002995346,4cyclictest3469831-21munin-run12:15:011
3454002995339,10cyclictest3461492-21munin-run09:35:021
3454002995338,11cyclictest3464077-21sshd10:24:351
3454002995338,11cyclictest3463314-21munin-run10:10:011
3454002995337,11cyclictest363-21snmpd11:54:261
3454002995337,11cyclictest3468058-21munin-run11:40:011
3454002995337,10cyclictest3465355-21sshd10:47:351
3454011995231,14cyclictest1-21systemd09:35:013
3454011995220,10cyclictest37-21ksoftirqd/309:20:013
3454011995220,10cyclictest37-21ksoftirqd/309:20:003
3454006995236,12cyclictest1-21systemd10:20:012
3454002995237,11cyclictest3465599-21sshd10:50:351
3454002995236,10cyclictest3454988-21munin-run07:30:001
3454011995133,13cyclictest3454440-21cron07:20:003
345401199512,43cyclictest1-21systemd12:10:013
345400699517,5cyclictest131rcu_preempt07:45:012
3454006995136,11cyclictest1-21systemd12:40:012
3454002995138,10cyclictest3470070-21sshd12:18:351
3454002995138,10cyclictest3469013-21sshd11:55:351
3454002995135,11cyclictest3468559-21munin-run11:50:011
3454002995131,15cyclictest363-21snmpd12:26:011
3454002995131,15cyclictest363-21snmpd12:26:011
3453997995111,9cyclictest3463322-21latency_hist10:10:010
3454011995031,14cyclictest235-21systemd-journal10:10:003
345400699502,7cyclictest30-21ksoftirqd/212:10:022
3454002995038,9cyclictest3461744-21munin-run09:40:001
3454002995037,9cyclictest3456035-21munin-run07:50:021
3454002995036,11cyclictest3466383-21sshd11:05:351
3454002995036,11cyclictest3463776-21cron10:17:001
3454002995036,10cyclictest3458648-21munin-run08:40:021
3454002995032,13cyclictest363-21snmpd12:04:241
3454002995030,16cyclictest363-21snmpd11:31:261
3454002995027,17cyclictest3466707-21cron11:15:001
3454002995027,17cyclictest3466707-21cron11:15:001
345400699499,6cyclictest131rcu_preempt11:00:012
345400699499,6cyclictest131rcu_preempt11:00:012
3454006994940,7cyclictest30-21ksoftirqd/209:50:022
3454002994937,9cyclictest3460420-21sshd09:14:351
3454002994936,10cyclictest3454713-21sshd07:24:051
3454002994935,10cyclictest3467513-21munin-run11:30:011
3454002994935,10cyclictest3467513-21munin-run11:30:011
3454002994932,14cyclictest363-21snmpd09:26:461
3454002994929,16cyclictest363-21snmpd08:11:251
3454002994928,15cyclictest3454182-21cron07:15:001
3454002994917,27cyclictest3461037-21grep09:25:021
345399799498,32cyclictest3454000-21grep07:10:000
3453997994942,4cyclictest16950irq/224-800000008:20:020
345401199485,6cyclictest37-21ksoftirqd/310:45:003
345401199485,6cyclictest37-21ksoftirqd/310:45:003
345400699487,10cyclictest30-21ksoftirqd/211:20:002
345400699486,8cyclictest30-21ksoftirqd/211:25:022
345400699486,12cyclictest30-21ksoftirqd/207:25:012
345400699485,38cyclictest3464864-21modprobe10:40:002
3454006994835,10cyclictest30-21ksoftirqd/208:55:022
3454002994840,4cyclictest3455239-21munin-run07:35:011
3454002994833,12cyclictest3465105-21sshd10:43:351
3454002994833,12cyclictest3465105-21sshd10:43:341
3454002994832,11cyclictest272-21dbus-daemon10:15:011
3454002994830,14cyclictest363-21snmpd10:00:441
3454002994827,16cyclictest3458086-21modprobe08:29:111
3454002994824,19cyclictest3457195-21latency_hist08:10:021
3454002994818,25cyclictest272-21dbus-daemon09:10:011
3453997994839,6cyclictest16950irq/224-800000010:50:000
3453997994831,13cyclictest235-21systemd-journal09:10:010
3453997994831,12cyclictest272-21dbus-daemon07:25:010
3453997994828,16cyclictest272-21dbus-daemon12:40:010
345401199473,6cyclictest1-21systemd10:20:003
3454011994729,14cyclictest272-21dbus-daemon08:30:013
345400699474,7cyclictest30-21ksoftirqd/207:39:592
3454006994710,11cyclictest30-21ksoftirqd/209:05:002
3454002994734,9cyclictest3454495-21munin-run07:20:001
3454002994733,11cyclictest363-21snmpd12:07:061
3454002994729,14cyclictest363-21snmpd08:04:311
3454002994728,13cyclictest363-21snmpd09:03:421
3454002994724,18cyclictest3457864-21systemd-run08:25:001
3454002994724,17cyclictest3466988-21run-parts11:20:001
345399799472,10cyclictest3462542-21latency_hist09:55:000
345400699465,8cyclictest30-21ksoftirqd/212:11:352
3454006994624,15cyclictest235-21systemd-journal12:05:002
3454002994633,9cyclictest3470889-21munin-run12:35:011
3454002994629,12cyclictest3459621-21sshd08:59:151
345400699459,5cyclictest131rcu_preempt09:20:012
345400699459,5cyclictest131rcu_preempt09:20:002
3454006994534,7cyclictest30-21ksoftirqd/211:30:012
3454006994534,7cyclictest30-21ksoftirqd/211:30:012
3454006994530,11cyclictest235-21systemd-journal12:19:422
3454006994511,10cyclictest30-21ksoftirqd/212:00:012
3454006994510,5cyclictest131rcu_preempt11:45:002
3454002994532,10cyclictest363-21snmpd12:24:331
3454002994532,10cyclictest363-21snmpd12:24:331
3454002994531,10cyclictest3455471-21sshd07:39:061
3453997994535,6cyclictest3467248-21cron11:25:000
3453997994528,12cyclictest3456514-21cron08:00:000
345401199444,5cyclictest131rcu_preempt11:50:003
345400699448,5cyclictest131rcu_preempt10:30:022
345400699448,5cyclictest131rcu_preempt10:30:022
345400699447,9cyclictest30-21ksoftirqd/210:24:592
3454006994429,12cyclictest1-21systemd11:35:002
3454006994418,22cyclictest291rcuc/210:51:352
3454002994436,5cyclictest363-21snmpd09:54:021
345399799448,26cyclictest19050irq/34-mmc109:20:020
345399799448,26cyclictest19050irq/34-mmc109:20:020
345399799442,8cyclictest235-21systemd-journal09:45:010
345400699438,10cyclictest30-21ksoftirqd/209:15:012
345400699437,6cyclictest131rcu_preempt07:35:012
345400699436,11cyclictest30-21ksoftirqd/208:10:012
345400699434,8cyclictest30-21ksoftirqd/208:25:002
345400699433,7cyclictest30-21ksoftirqd/211:50:012
3454006994336,5cyclictest30-21ksoftirqd/211:06:372
3454006994334,7cyclictest291rcuc/212:30:012
3454006994334,7cyclictest291rcuc/212:30:012
3454006994328,11cyclictest3463469-21sed10:10:022
3454006994328,10cyclictest30-21ksoftirqd/211:15:002
3454006994328,10cyclictest30-21ksoftirqd/211:14:592
3454006994326,12cyclictest272-21dbus-daemon08:05:002
345400699431,8cyclictest0-21swapper/207:45:352
3454006994315,24cyclictest291rcuc/207:14:032
3454006994310,4cyclictest131rcu_preempt08:45:002
3454002994330,10cyclictest363-21snmpd09:18:561
3454002994330,10cyclictest363-21snmpd09:18:561
3454002994329,10cyclictest3459116-21sshd08:49:131
3453997994336,4cyclictest16950irq/224-800000009:15:000
345399799432,7cyclictest3459971-21latency_hist09:05:020
3453997994326,13cyclictest3469546-21sshd12:05:350
345401199428,27cyclictest3467515-21cut11:30:003
345401199428,27cyclictest3467515-21cut11:29:593
345400699425,12cyclictest30-21ksoftirqd/209:30:012
3454006994231,7cyclictest306-21in:imuxsock07:20:002
345400699423,10cyclictest30-21ksoftirqd/211:03:352
3454002994228,9cyclictest363-21snmpd08:19:101
345399799426,31cyclictest3457294-21cron08:14:590
345399799424,31cyclictest3461746-21cut09:40:000
3453997994229,6cyclictest306-21in:imuxsock12:05:010
3453997994228,10cyclictest3470073-21sshd12:19:350
3453997994226,13cyclictest3464819-21sshd10:36:350
3453997994225,13cyclictest235-21systemd-journal07:20:010
345399799421,37cyclictest3462248-21sshd09:49:200
345399799421,37cyclictest272-21dbus-daemon11:50:010
3453997994213,26cyclictest16950irq/224-800000011:20:010
3454011994126,11cyclictest3468002-21cron11:40:003
345401199412,5cyclictest3462776-21munin-run10:00:013
345400699419,6cyclictest30-21ksoftirqd/211:53:352
345400699414,6cyclictest30-21ksoftirqd/208:15:002
345400699414,5cyclictest131rcu_preempt09:35:022
345400699413,11cyclictest30-21ksoftirqd/208:35:022
3454006994111,5cyclictest131rcu_preempt11:40:012
345400699411,10cyclictest30-21ksoftirqd/207:55:022
3453997994134,4cyclictest16950irq/224-800000012:30:000
3453997994134,4cyclictest16950irq/224-800000012:30:000
3453997994127,10cyclictest408-21lldpd10:30:080
345399799412,35cyclictest3459682-21unin-run09:00:020
3453997994122,15cyclictest272-21dbus-daemon10:05:010
3454011994031,7cyclictest37-21ksoftirqd/310:30:023
3454011994031,7cyclictest37-21ksoftirqd/310:30:023
3454011994030,7cyclictest3467021-21systemctl11:20:003
3454011994024,11cyclictest272-21dbus-daemon11:35:003
3454011994023,11cyclictest0-21swapper/310:25:003
3453997994024,12cyclictest3469021-21sshd11:57:350
3453997994024,12cyclictest235-21systemd-journal08:35:000
345401199398,6cyclictest131rcu_preempt09:08:353
3454011993929,7cyclictest37-21ksoftirqd/310:05:013
3454011993929,6cyclictest3461224-21sh09:30:003
3454011993924,9cyclictest3463540-21cron10:15:003
345400699395,7cyclictest30-21ksoftirqd/209:25:032
3454006993927,7cyclictest30-21ksoftirqd/207:28:362
345399799394,3cyclictest131rcu_preempt11:40:020
3453997993931,5cyclictest16850irq/134-800000011:43:320
3453997993923,11cyclictest235-21systemd-journal08:45:010
345401199384,4cyclictest131rcu_preempt08:15:013
345401199384,10cyclictest37-21ksoftirqd/308:34:233
3454011993827,6cyclictest306-21in:imuxsock08:50:033
3454011993823,10cyclictest37-21ksoftirqd/311:25:013
345400699385,5cyclictest131rcu_preempt08:29:112
3454002993819,16cyclictest3453991-21cyclictest07:10:011
3453997993832,4cyclictest3465213-21cut10:45:020
3453997993832,4cyclictest3465213-21cut10:45:020
3453997993830,6cyclictest19050irq/34-mmc111:15:010
3453997993830,6cyclictest19050irq/34-mmc111:15:010
3453997993825,10cyclictest3462735-21sshd09:55:360
3453997993825,10cyclictest272-21dbus-daemon10:20:010
3453997993824,11cyclictest3455918-21sed07:45:020
3453997993824,10cyclictest235-21systemd-journal10:30:010
3453997993824,10cyclictest235-21systemd-journal10:30:010
3453997993817,14cyclictest0-21swapper/011:05:000
345401199374,6cyclictest131rcu_preempt07:25:013
345401199373,3cyclictest131rcu_preempt08:00:003
3454011993725,9cyclictest3464955-21sed10:40:023
345401199372,4cyclictest131rcu_preempt11:45:003
3454011993720,13cyclictest3455468-21sshd07:38:363
3454011993720,13cyclictest1-21systemd09:40:013
345400699378,5cyclictest131rcu_preempt08:47:362
3454006993727,7cyclictest3456499-21sshd07:57:352
3454006993724,10cyclictest235-21systemd-journal10:05:012
3454006993723,10cyclictest30-21ksoftirqd/210:50:002
3453997993730,4cyclictest16950irq/224-800000011:06:350
3453997993724,10cyclictest12-21ksoftirqd/011:50:020
3453997993723,11cyclictest12-21ksoftirqd/010:52:350
3453997993723,10cyclictest1-21systemd08:55:000
3453997993720,15cyclictest0-21swapper/010:20:350
3453997993720,12cyclictest3465895-21systemctl11:00:000
3453997993720,12cyclictest3465895-21systemctl11:00:000
345401199368,23cyclictest3465897-21munin-run11:00:003
345401199368,23cyclictest3465897-21munin-run11:00:003
3454011993628,5cyclictest3462011-21munin-plugin-st09:45:003
3454011993626,7cyclictest3456834-21sed08:05:023
345401199362,5cyclictest131rcu_preempt09:00:013
3454011993622,9cyclictest3458887-21cut08:45:003
3454011993622,11cyclictest37-21ksoftirqd/308:10:023
3454011993620,11cyclictest272-21dbus-daemon09:25:003
345400699364,5cyclictest131rcu_preempt08:15:342
345399799363,26cyclictest3456024-21munin-plugin-st07:50:000
3453997993626,7cyclictest12-21ksoftirqd/008:00:360
3453997993622,9cyclictest306-21in:imuxsock08:30:010
3453997993620,12cyclictest235-21systemd-journal07:34:040
3453997993620,12cyclictest1-21systemd09:25:000
3453997993619,12cyclictest272-21dbus-daemon12:25:010
3453997993619,12cyclictest272-21dbus-daemon12:25:010
345399799361,31cyclictest0-21swapper/009:34:130
3454011993527,5cyclictest3469105-21kworker/3:212:20:023
345401199352,4cyclictest131rcu_preempt12:15:013
3454011993523,8cyclictest37-21ksoftirqd/308:25:013
345401199351,5cyclictest131rcu_preempt12:34:353
345401199351,4cyclictest131rcu_preempt09:50:003
3454006993526,5cyclictest306-21in:imuxsock08:40:012
3453997993523,8cyclictest235-21systemd-journal08:25:000
345399799352,29cyclictest235-21systemd-journal09:26:350
3453997993520,11cyclictest3457030-21sshd08:07:350
345399799351,5cyclictest3456242-21sshd07:51:360
345399799351,32cyclictest0-21swapper/011:29:220
345399799351,32cyclictest0-21swapper/011:29:220
345399799351,31cyclictest0-21swapper/007:28:170
345401199343,6cyclictest131rcu_preempt10:49:303
345401199342,5cyclictest131rcu_preempt07:10:003
3454011993424,7cyclictest37-21ksoftirqd/308:40:013
3454011993420,10cyclictest37-21ksoftirqd/312:00:023
3454011993418,12cyclictest3471128-21cron12:40:003
345401199341,5cyclictest131rcu_preempt07:55:023
3454006993424,6cyclictest306-21in:imuxsock12:35:002
345399799341,30cyclictest0-21swapper/011:32:070
345399799341,29cyclictest0-21swapper/008:47:270
345401199338,6cyclictest131rcu_preempt12:30:003
345401199338,6cyclictest131rcu_preempt12:30:003
3454011993320,9cyclictest37-21ksoftirqd/309:15:003
345401199331,4cyclictest131rcu_preempt07:45:033
345401199331,4cyclictest131rcu_preempt07:30:003
3453997993326,4cyclictest0-21swapper/012:10:030
345401199321,4cyclictest131rcu_preempt08:20:013
345399799321,7cyclictest235-21systemd-journal08:39:530
3454011993119,9cyclictest272-21dbus-daemon08:50:013
345401199311,3cyclictest131rcu_preempt10:35:013
3454006992820,6cyclictest291rcuc/207:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional