You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-29 - 06:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Wed Oct 29, 2025 00:40:00)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
513093995933,5cyclictest582678-21idleruntime-cro21:55:011
513094995644,7cyclictest2909538-21cups-browsed20:48:592
513093995634,3cyclictest0-21swapper/122:44:041
513093995631,6cyclictest2909538-21cups-browsed23:20:061
513094995137,10cyclictest421-21avahi-daemon21:24:022
513094995041,6cyclictest266-21systemd-journal00:35:012
513092995036,10cyclictest657298-21ssh23:34:030
513093994940,5cyclictest553032-21cron21:15:011
513093994940,5cyclictest553032-21cron21:15:011
513092994821,24cyclictest574253-21ssh21:44:000
513092994821,24cyclictest574253-21ssh21:44:000
513094994737,4cyclictest0-21swapper/200:20:022
513094994634,8cyclictest627986-21idleruntime-cro22:55:002
513093994638,5cyclictest266-21systemd-journal23:20:011
513092994637,6cyclictest12450irq/32-eth022:09:020
51309299463,34cyclictest683399-21ssh00:09:010
513093994540,2cyclictest0-21swapper/121:44:041
513093994540,2cyclictest0-21swapper/121:44:041
513093994535,6cyclictest601558-21munin-run22:20:011
51309299454,34cyclictest619513-21sshd22:44:050
513092994537,6cyclictest2909538-21cups-browsed21:51:120
513092994537,5cyclictest1-21systemd23:15:020
513092994537,5cyclictest1-21systemd22:35:020
51309399445,31cyclictest653541-21ssh23:29:051
513093994432,8cyclictest0-21swapper/121:35:011
51309299443,34cyclictest697713-21ssh00:29:000
513092994428,10cyclictest12450irq/32-eth021:08:590
513094994337,3cyclictest0-21swapper/222:04:042
513094994334,5cyclictest654477-21systemd-run23:30:022
513093994337,3cyclictest0-21swapper/123:09:001
513093994335,5cyclictest691805-21idleruntime-cro00:20:011
513093994334,6cyclictest519101-21systemd-run19:30:021
513092994335,5cyclictest266-21systemd-journal23:00:020
513092994334,6cyclictest646880-21munin-run23:20:010
513092994333,7cyclictest516129-21latency_hist19:20:020
513094994235,5cyclictest540051-21cron20:40:002
513094994235,5cyclictest540051-21cron20:40:002
513094994235,3cyclictest0-21swapper/221:59:062
513093994235,4cyclictest677226-21awk00:00:011
513093994234,6cyclictest0-21swapper/122:54:111
513092994232,7cyclictest537602-21fschecks_count20:33:510
513092994232,7cyclictest537602-21fschecks_count20:33:510
513092994230,9cyclictest673504-21cron23:55:010
513094994132,6cyclictest677358-21cups-browsed00:28:542
513093994132,6cyclictest605360-21munin-run22:25:011
513092994134,4cyclictest553044-21munin-plugin-st21:15:010
513092994134,4cyclictest553044-21munin-plugin-st21:15:010
513094994033,4cyclictest959-21mta-sts-daemon20:25:022
513094994031,5cyclictest635596-21munin-plugin-st23:05:002
513093994034,3cyclictest0-21swapper/119:40:011
51309399403,30cyclictest593098-21ssh22:09:051
513092994025,12cyclictest0-21swapper/022:13:490
51309299402,29cyclictest679801-21ssh00:04:000
513092994014,19cyclictest12450irq/32-eth021:19:050
51309499393,29cyclictest555729-21ssh21:19:002
513093993930,6cyclictest1-21systemd23:35:021
513093993927,8cyclictest623088-21missed_timers22:48:571
513093993927,8cyclictest623088-21missed_timers22:48:571
51309299397,28cyclictest621031-21ssh22:45:470
51309299397,28cyclictest621031-21ssh22:45:470
513092993930,6cyclictest2909510-21cupsd23:40:480
513092993930,6cyclictest2909510-21cupsd23:40:480
513092993924,11cyclictest12450irq/32-eth020:09:060
51309299392,31cyclictest657-21snmpd23:09:080
513094993829,6cyclictest1-21systemd23:20:012
51309499382,29cyclictest649774-21ssh23:24:082
51309499380,6cyclictest171rcu_preempt21:54:002
51309399384,25cyclictest589199-21ssh22:04:001
513093993831,4cyclictest528058-21munin-plugin-st20:00:011
513093993830,5cyclictest611772-21ntpq22:34:001
513094993729,6cyclictest591-21sshd21:29:062
513094993729,5cyclictest677241-21tr00:00:012
513094993726,7cyclictest0-21swapper/219:24:042
51309399375,22cyclictest679888-21ssh00:04:031
513093993729,5cyclictest693999-21kernelversion00:23:551
513093993729,5cyclictest433-21polkitd00:13:051
513093993728,6cyclictest630296-21fschecks_count22:58:521
513092993730,4cyclictest541566-21cron20:45:010
513092993730,4cyclictest541566-21cron20:45:010
51309299371,4cyclictest171rcu_preempt21:49:010
51309299371,4cyclictest171rcu_preempt21:49:010
513094993629,4cyclictest520563-21cron19:35:012
513094993628,5cyclictest616632-21idleruntime-cro22:40:012
513094993628,5cyclictest578974-21cron21:50:002
513094993628,5cyclictest578974-21cron21:50:002
513094993628,5cyclictest453-21systemd-logind20:10:012
513094993628,5cyclictest0-21swapper/222:20:012
513094993627,6cyclictest688251-21latency_hist00:15:022
513094993625,8cyclictest608399-21switchtime22:29:102
513094993625,7cyclictest662102-21cron23:40:012
513094993625,7cyclictest662102-21cron23:40:012
51309499361,7cyclictest0-21swapper/220:14:042
513093993629,3cyclictest0-21swapper/121:19:071
513093993627,6cyclictest664810-21ssh23:43:571
513093993627,6cyclictest664810-21ssh23:43:571
513093993627,6cyclictest421-21avahi-daemon23:12:221
513093993627,6cyclictest421-21avahi-daemon21:48:161
513093993627,6cyclictest421-21avahi-daemon21:48:161
513093993627,5cyclictest683003-21fschecks_time00:08:541
513093993623,7cyclictest615559-21sh22:39:001
513092993631,2cyclictest0-21swapper/020:40:010
513092993631,2cyclictest0-21swapper/020:40:010
513092993627,6cyclictest421-21avahi-daemon00:24:400
513092993620,12cyclictest0-21swapper/023:50:010
513094993528,3cyclictest0-21swapper/222:34:002
513094993526,5cyclictest959-21mta-sts-daemon23:49:062
513094993521,5cyclictest959-21mta-sts-daemon00:23:512
51309499351,5cyclictest0-21swapper/223:54:002
513093993527,5cyclictest571555-21munin-run21:40:011
513093993526,6cyclictest0-21swapper/120:09:051
513092993530,3cyclictest956-21mta-sts-daemon22:19:050
513092993520,12cyclictest528627-21fschecks_time20:03:540
513092993514,15cyclictest608020-21ssh22:28:590
513092993510,18cyclictest657-21snmpd22:39:010
51309499345,7cyclictest171rcu_preempt22:44:022
513094993428,3cyclictest0-21swapper/200:10:002
513094993427,3cyclictest0-21swapper/221:13:562
513094993427,3cyclictest0-21swapper/221:13:562
513094993426,5cyclictest518145-21gltestperf19:28:512
513094993425,6cyclictest523595-21systemd-run19:45:012
513094993424,7cyclictest631052-21switchtime22:59:092
513094993423,8cyclictest679593-21latency00:03:542
513094993422,9cyclictest704461-21latency00:38:552
513093993427,4cyclictest955-21mta-sts-daemon23:37:201
513093993427,4cyclictest955-21mta-sts-daemon23:37:201
513093993427,4cyclictest597213-21timerwakeupswit22:14:111
513093993427,4cyclictest0-21swapper/123:49:011
513093993426,5cyclictest677358-21cups-browsed00:31:171
51309399342,26cyclictest585473-21ssh21:59:031
51309499335,24cyclictest570601-21ssh21:39:032
51309499333,25cyclictest657-21snmpd20:19:042
513094993326,3cyclictest0-21swapper/221:10:012
513093993326,4cyclictest609075-21ssh22:30:011
513093993325,5cyclictest562696-21fschecks_count21:28:511
513093993325,5cyclictest532551-21cron20:15:001
513093993325,3cyclictest0-21swapper/123:03:511
513093993324,6cyclictest2909538-21cups-browsed23:52:451
513093993323,7cyclictest422-21dbus-daemon00:26:081
513092993327,3cyclictest0-21swapper/023:30:020
513092993325,6cyclictest566368-21fschecks_count21:33:510
513092993325,5cyclictest609102-21kworker/u8:11+xprtiod23:58:550
513092993325,5cyclictest588650-21aten_r2power_vo22:03:500
513092993324,6cyclictest686511-21aten_r2power_po00:13:480
513092993323,7cyclictest677358-21cups-browsed00:16:020
51309299332,22cyclictest657-21snmpd23:24:020
513092993319,11cyclictest523118-21proc_pri19:44:050
513092993316,14cyclictest525103-21latency_hist19:50:020
513094993225,3cyclictest0-21swapper/223:10:292
51309499322,25cyclictest665041-21ssh23:44:052
51309499322,25cyclictest665041-21ssh23:44:052
513094993221,7cyclictest623341-21ssh22:49:052
513094993221,7cyclictest623341-21ssh22:49:052
513094993217,12cyclictest351ktimers/220:29:042
513093993226,3cyclictest0-21swapper/120:05:011
513093993224,5cyclictest516083-21run-parts19:20:011
513093993222,7cyclictest453-21systemd-logind21:25:011
51309299322,26cyclictest570694-21ssh21:39:060
513094993124,4cyclictest956-21mta-sts-daemon19:10:022
513094993124,3cyclictest609102-21kworker/u8:11-xprtiod23:09:092
513094993124,3cyclictest0-21swapper/221:33:542
513094993123,5cyclictest657853-21unixbench_singl23:34:122
513094993122,4cyclictest0-21swapper/221:44:002
513094993122,4cyclictest0-21swapper/221:44:002
51309499312,22cyclictest596868-21ssh22:14:052
513093993126,2cyclictest0-21swapper/120:19:051
513093993125,3cyclictest0-21swapper/100:38:581
513093993123,5cyclictest541568-21latency_hist20:45:011
513093993123,5cyclictest541568-21latency_hist20:45:011
513093993123,5cyclictest526585-21systemd-run19:55:011
513092993125,4cyclictest585695-21mailstats21:59:090
513092993125,4cyclictest564163-21systemd-run21:30:010
513092993124,5cyclictest0-21swapper/023:04:130
513092993123,6cyclictest704663-21ntp_kernel_pll_00:39:000
513092993122,7cyclictest513047-21/usr/sbin/munin19:10:020
51309299311,22cyclictest660983-21ssh23:38:560
51309299311,22cyclictest660983-21ssh23:38:560
513094993022,5cyclictest592571-21fschecks_count22:08:522
513094993022,5cyclictest529028-21ntp_offset20:04:032
513094993019,8cyclictest698-21cron20:35:012
513094993019,8cyclictest698-21cron20:35:012
513094993019,8cyclictest0-21swapper/222:24:022
513093993022,5cyclictest522633-21fschecks_time19:43:521
513092993024,3cyclictest0-21swapper/000:35:010
513092993019,8cyclictest603662-21aten_r2power_fr22:23:500
513094992923,3cyclictest0-21swapper/221:00:002
513093992923,3cyclictest0-21swapper/121:10:021
513092992923,4cyclictest544514-21unixbench_singl20:54:120
51309299292,20cyclictest657-21snmpd20:24:040
51309299291,23cyclictest698-21cron19:34:010
51309399284,16cyclictest565-21gmain19:24:071
513093992821,4cyclictest524117-21fschecks_count19:48:521
513093992820,4cyclictest538559-21cron20:35:011
513093992820,4cyclictest538559-21cron20:35:011
513092992822,4cyclictest474-21in:imuxsock20:00:010
513092992822,3cyclictest0-21swapper/022:55:000
513092992821,5cyclictest559178-21kernelversion21:23:560
513092992821,4cyclictest532751-21latency_hist20:15:020
513092992816,9cyclictest504597-21kworker/u8:7-events_unbound19:11:560
513094992720,4cyclictest0-21swapper/219:20:012
513094992719,5cyclictest544551-21cron20:55:012
513094992718,6cyclictest491-21gdbus19:39:082
513093992719,5cyclictest535576-21cron20:25:011
513093992719,5cyclictest453-21systemd-logind20:30:011
513092992720,4cyclictest546084-21sendmail-msp21:00:010
513092992719,5cyclictest547597-21systemd21:05:020
513092992715,9cyclictest0-21swapper/020:18:530
513094992619,3cyclictest0-21swapper/221:05:012
513094992619,3cyclictest0-21swapper/220:00:012
513094992618,5cyclictest514463-21unixbench_multi19:14:092
513093992622,2cyclictest0-21swapper/120:54:071
513093992617,6cyclictest519625-21fschecks_time19:33:511
513092992620,4cyclictest474-21in:imuxsock19:55:010
513092992620,3cyclictest0-21swapper/019:39:040
513092992619,5cyclictest486-21gmain20:48:590
513093992517,5cyclictest546977-21ntp_kernel_pll_21:04:011
51309299251,3cyclictest0-21swapper/020:28:580
513092992513,7cyclictest0-21swapper/019:24:040
513093992417,4cyclictest539948-21unixbench_multi20:39:101
513093992417,4cyclictest539948-21unixbench_multi20:39:101
513093992416,5cyclictest545103-21fschecks_time20:58:551
513093992413,8cyclictest514528-21sed19:14:091
513094992315,5cyclictest486-21gmain19:53:552
513093992318,3cyclictest955-21mta-sts-daemon20:46:371
513093992317,3cyclictest0-21swapper/119:10:011
51309499222,15cyclictest437-21mta-sts-daemon20:44:032
51309499222,15cyclictest437-21mta-sts-daemon20:44:032
513094992215,4cyclictest525034-21unixbench_singl19:49:102
513092992114,5cyclictest508785-21kworker/u8:2+nfsiod19:27:120
51309599172,9cyclictest0-21swapper/323:28:593
51309599171,8cyclictest0-21swapper/322:28:583
51309599162,8cyclictest0-21swapper/322:44:023
51309599162,7cyclictest0-21swapper/323:24:053
51309599161,8cyclictest0-21swapper/300:09:053
51309599161,7cyclictest0-21swapper/300:04:013
51309599154,7cyclictest0-21swapper/321:49:053
51309599154,7cyclictest0-21swapper/321:49:053
51309599153,8cyclictest0-21swapper/322:59:033
51309599152,7cyclictest0-21swapper/322:49:023
51309599152,7cyclictest0-21swapper/322:49:023
51309599152,7cyclictest0-21swapper/322:39:033
51309599144,7cyclictest0-21swapper/322:33:563
51309599143,7cyclictest0-21swapper/300:14:003
51309599142,8cyclictest0-21swapper/300:38:593
51309599142,7cyclictest0-21swapper/323:59:063
51309599142,7cyclictest0-21swapper/323:54:063
51309599142,7cyclictest0-21swapper/322:18:583
51309599142,7cyclictest0-21swapper/322:13:593
51309599142,6cyclictest0-21swapper/323:09:003
51309599142,5cyclictest0-21swapper/319:39:013
51309599141,8cyclictest0-21swapper/321:19:003
51309599141,8cyclictest0-21swapper/321:13:583
51309599141,8cyclictest0-21swapper/321:13:583
51309599141,7cyclictest0-21swapper/322:53:553
51309599141,7cyclictest0-21swapper/322:04:043
51309599141,7cyclictest0-21swapper/321:58:583
51309599141,7cyclictest0-21swapper/321:44:053
51309599141,7cyclictest0-21swapper/321:44:053
51309599141,7cyclictest0-21swapper/321:33:563
51309599141,7cyclictest0-21swapper/319:24:003
51309599141,6cyclictest0-21swapper/323:46:053
51309599141,6cyclictest0-21swapper/323:44:033
51309599141,6cyclictest0-21swapper/323:44:033
51309599141,6cyclictest0-21swapper/320:54:033
51309599141,6cyclictest0-21swapper/320:38:593
51309599141,6cyclictest0-21swapper/320:38:593
51309599141,6cyclictest0-21swapper/320:24:033
51309599141,6cyclictest0-21swapper/319:59:013
51309599132,7cyclictest0-21swapper/321:53:583
51309599132,6cyclictest0-21swapper/323:37:053
51309599132,6cyclictest0-21swapper/323:37:053
51309599132,6cyclictest0-21swapper/323:04:003
51309599132,5cyclictest0-21swapper/321:24:023
51309599131,7cyclictest0-21swapper/323:14:033
51309599131,7cyclictest0-21swapper/321:39:013
51309599131,7cyclictest0-21swapper/300:29:003
51309599131,6cyclictest0-21swapper/320:19:053
51309599131,6cyclictest0-21swapper/319:49:023
51309599131,6cyclictest0-21swapper/319:28:573
51309599131,6cyclictest0-21swapper/300:23:583
51309599122,6cyclictest0-21swapper/321:10:013
51309599122,6cyclictest0-21swapper/300:18:513
51309599122,5cyclictest0-21swapper/320:45:013
51309599122,5cyclictest0-21swapper/320:45:013
51309599121,7cyclictest0-21swapper/322:24:003
51309599121,7cyclictest0-21swapper/322:09:003
51309599121,7cyclictest0-21swapper/321:28:483
51309599121,6cyclictest0-21swapper/320:48:593
51309599121,6cyclictest0-21swapper/320:28:583
51309599121,6cyclictest0-21swapper/320:09:053
51309599121,6cyclictest0-21swapper/319:54:003
51309599121,5cyclictest0-21swapper/320:33:573
51309599121,5cyclictest0-21swapper/320:33:573
51309599121,5cyclictest0-21swapper/320:14:013
51309599121,5cyclictest0-21swapper/319:44:013
51309599112,5cyclictest0-21swapper/319:33:593
51309599112,5cyclictest0-21swapper/300:33:523
51309599111,6cyclictest0-21swapper/323:31:063
51309599111,6cyclictest0-21swapper/323:18:503
51309599101,5cyclictest0-21swapper/319:13:493
5130959991,5cyclictest0-21swapper/320:03:523
5130959991,4cyclictest0-21swapper/319:10:023
5130959991,3cyclictest0-21swapper/319:18:063
5130959990,3cyclictest0-21swapper/321:00:053
5130959981,3cyclictest0-21swapper/320:59:043
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional