You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-03 - 10:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot3s.osadl.org (updated Wed Dec 03, 2025 00:45:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
931163997268,2cyclictest0-21swapper/000:20:020
931163997268,2cyclictest0-21swapper/000:20:020
931163996747,17cyclictest16950irq/224-800000000:00:000
931177996616,46cyclictest932635-21sshd19:37:293
93116399644,38cyclictest19050irq/34-mmc119:25:020
931172996226,6cyclictest935016-21latency_hist20:25:022
931177996120,37cyclictest944297-21cat23:20:033
93116399615,37cyclictest938906-21modprobe21:40:000
931163996152,7cyclictest16950irq/224-800000022:45:010
93117799591,54cyclictest940692-21sshd22:10:193
93117799583,49cyclictest935258-21cron20:30:003
93116399572,46cyclictest940478-21munin-run22:10:000
93117799563,47cyclictest934216-21cron20:10:003
931177995627,25cyclictest1-21systemd20:50:003
93117799562,49cyclictest235-21systemd-journal00:25:023
93117799562,49cyclictest235-21systemd-journal00:25:013
93117799562,48cyclictest946190-21cut00:00:003
93117799561,50cyclictest941968-21sed22:35:023
93117799552,49cyclictest947191-21cron00:17:013
93117799552,49cyclictest933180-21cron19:50:003
93117799552,48cyclictest946951-21sshd00:14:183
93117799552,48cyclictest939679-21cron21:55:013
93117799552,48cyclictest939679-21cron21:55:003
93117799552,48cyclictest937315-21sendmail-mta21:05:223
93117799552,48cyclictest935803-21sendmail-msp20:40:003
93117799551,50cyclictest363-21snmpd20:51:563
93117799551,4cyclictest48-21kcompactd023:02:383
93117799544,46cyclictest934450-21sshd20:12:193
93117799543,46cyclictest936792-21sshd20:58:193
93117799542,48cyclictest940964-21sshd22:17:583
93117799542,47cyclictest933662-21sendmail-mta19:55:223
93117799542,47cyclictest932120-21modprobe19:25:183
93117799541,49cyclictest938120-21sshd21:21:193
931177995414,35cyclictest0-21swapper/322:48:123
931177995414,35cyclictest0-21swapper/322:48:123
931167995437,13cyclictest1-21systemd21:05:001
931167995417,9cyclictest941907-21date22:35:021
931163995447,4cyclictest16950irq/224-800000000:20:010
931163995436,15cyclictest16950irq/224-800000023:50:010
93117799532,47cyclictest944612-21sshd23:28:113
93117799532,47cyclictest937854-21sshd21:18:203
93117799532,46cyclictest938962-21grep21:40:023
93117799532,46cyclictest933989-21munin-run20:05:023
93117799532,46cyclictest1-21systemd22:10:013
93117799531,48cyclictest946431-21sshd00:01:203
93117799531,47cyclictest943785-21cut23:10:023
93117799531,47cyclictest942783-21sshd22:53:053
93117799531,47cyclictest934869-21latency_hist20:20:023
93117799526,40cyclictest0-21swapper/319:55:003
93117799522,46cyclictest948248-21sshd00:37:203
93117799522,46cyclictest948248-21sshd00:37:203
93117799522,46cyclictest947722-21sshd00:25:183
93117799522,46cyclictest945420-21munin-run23:45:013
93117799522,46cyclictest941491-21sshd22:27:193
93117799522,44cyclictest948014-21cron00:35:003
93117799522,43cyclictest940236-21unin-run22:05:013
93117799522,43cyclictest939334-21latency_hist21:45:023
93117799521,48cyclictest931875-21sshd19:22:183
93117799521,47cyclictest937077-21sshd21:04:193
93117799521,47cyclictest932373-21sshd19:30:193
93117799521,46cyclictest48-21kcompactd023:51:503
93117799521,46cyclictest48-21kcompactd023:51:503
93117799521,45cyclictest937583-21modprobe21:11:193
931172995230,18cyclictest935776-21cron20:40:002
93116399524,3cyclictest131rcu_preempt20:10:190
931163995233,13cyclictest235-21systemd-journal20:10:000
93117799512,45cyclictest932909-21sshd19:41:183
93117799512,44cyclictest363-21snmpd00:07:333
931177995120,28cyclictest945124-21sshd23:39:193
93117799511,46cyclictest945655-21sshd23:49:193
93117799511,46cyclictest48-21kcompactd023:34:013
931177995114,33cyclictest0-21swapper/322:39:293
931177995113,34cyclictest0-21swapper/321:28:003
931163995132,12cyclictest235-21systemd-journal21:10:000
931163995129,15cyclictest235-21systemd-journal20:40:000
931163995114,7cyclictest946968-21/usr/sbin/munin00:15:010
93117799503,44cyclictest363-21snmpd20:31:343
93117799502,42cyclictest944403-21cat23:25:013
93117799501,45cyclictest941233-21sshd22:20:193
93117799501,45cyclictest931600-21sshd19:17:253
93117799501,45cyclictest931338-21sshd19:12:233
93117799501,44cyclictest939441-21munin-run21:50:003
93116399506,30cyclictest936807-21modprobe21:00:000
93116399502,29cyclictest151rcuc/021:20:020
93117799493,41cyclictest363-21snmpd20:42:263
93117799492,43cyclictest363-21snmpd21:30:063
93117799491,45cyclictest943806-21sshd23:10:193
93117799491,44cyclictest942458-21latency_hist22:45:023
93117799482,42cyclictest363-21snmpd22:58:093
931172994837,8cyclictest944061-21sshd23:16:192
931172994833,11cyclictest272-21dbus-daemon22:15:002
93117299481,44cyclictest0-21swapper/222:55:012
93116399489,9cyclictest12-21ksoftirqd/000:05:010
931163994833,11cyclictest942013-21cron22:40:000
93117799472,40cyclictest939907-21sshd21:55:193
931172994728,15cyclictest944390-21sh23:25:012
931172994726,16cyclictest235-21systemd-journal19:20:012
931163994738,7cyclictest12-21ksoftirqd/020:45:020
931163994729,10cyclictest306-21in:imuxsock20:30:010
93117799461,40cyclictest935118-21latency_hist20:25:013
931172994633,9cyclictest1-21systemd21:15:002
931172994628,13cyclictest1-21systemd22:50:022
931172994628,13cyclictest1-21systemd22:50:012
931163994620,17cyclictest235-21systemd-journal23:45:010
931172994523,17cyclictest939156-21sshd21:40:192
931167994528,11cyclictest235-21systemd-journal22:50:011
931167994528,11cyclictest235-21systemd-journal22:50:001
931167994515,6cyclictest23-21ksoftirqd/121:36:161
931163994534,9cyclictest12-21ksoftirqd/022:20:010
931163994528,14cyclictest272-21dbus-daemon19:55:000
93117799441,38cyclictest931247-21wc19:10:013
931172994431,9cyclictest235-21systemd-journal21:20:012
931172994429,12cyclictest272-21dbus-daemon00:25:022
931172994429,12cyclictest272-21dbus-daemon00:25:012
931167994427,13cyclictest272-21dbus-daemon21:55:021
931167994427,13cyclictest272-21dbus-daemon21:55:011
931167994423,15cyclictest235-21systemd-journal23:30:011
931163994438,4cyclictest16950irq/224-800000021:05:010
931163994437,4cyclictest16950irq/224-800000019:56:180
931163994437,4cyclictest16950irq/224-800000019:15:010
931163994426,13cyclictest272-21dbus-daemon00:40:020
931163994426,13cyclictest272-21dbus-daemon00:40:010
93116399442,38cyclictest933158-21sshd19:45:180
931172994326,13cyclictest1-21systemd22:35:012
931172994325,13cyclictest941189-21date22:20:022
931167994332,6cyclictest306-21in:imuxsock22:30:011
931167994330,9cyclictest235-21systemd-journal19:20:001
931167994329,11cyclictest272-21dbus-daemon00:00:031
931167994327,11cyclictest23-21ksoftirqd/121:30:001
93116399439,6cyclictest131rcu_preempt23:00:000
93116399435,33cyclictest0-21swapper/019:20:000
931163994329,10cyclictest235-21systemd-journal19:35:180
931163994327,12cyclictest12-21ksoftirqd/022:29:190
931163994327,10cyclictest306-21in:imuxsock22:20:020
931172994231,7cyclictest306-21in:imuxsock00:40:002
931172994231,7cyclictest306-21in:imuxsock00:40:002
931172994225,13cyclictest235-21systemd-journal00:00:192
931167994226,12cyclictest23-21ksoftirqd/121:20:001
93116399423,9cyclictest934746-21latency_hist20:20:000
931163994229,9cyclictest272-21dbus-daemon00:30:000
93116399422,36cyclictest0-21swapper/023:30:000
93116399422,23cyclictest151rcuc/021:35:020
931172994132,7cyclictest30-21ksoftirqd/220:30:022
931172994125,11cyclictest939548-21latency_hist21:50:032
931167994131,6cyclictest306-21in:imuxsock21:48:191
931167994113,22cyclictest943602-21systemctl23:10:001
93116399411,5cyclictest941768-21modprobe22:35:000
931172994025,12cyclictest938219-21cat21:25:032
93117299401,5cyclictest131rcu_preempt19:40:012
931172994012,5cyclictest131rcu_preempt20:48:192
931167994030,8cyclictest23-21ksoftirqd/121:00:001
931167994030,6cyclictest935762-21cron20:40:001
931167994026,9cyclictest306-21in:imuxsock21:15:011
931167994026,9cyclictest235-21systemd-journal23:35:001
931167994026,10cyclictest235-21systemd-journal00:31:201
931167994022,14cyclictest933156-21sshd19:45:181
931167994019,16cyclictest944090-21cron23:20:001
931167994019,15cyclictest235-21systemd-journal22:05:001
931167994016,19cyclictest940729-21grep22:15:001
931163994035,3cyclictest16950irq/224-800000021:45:010
931163994032,5cyclictest19050irq/34-mmc123:13:100
931163994025,11cyclictest12-21ksoftirqd/020:55:010
931163994024,13cyclictest12-21ksoftirqd/020:35:020
931163994024,12cyclictest272-21dbus-daemon23:40:000
931163994024,11cyclictest272-21dbus-daemon22:15:000
931163994020,16cyclictest946694-21sshd00:08:190
93116399401,35cyclictest944982-21date23:35:010
93117299392,7cyclictest0-21swapper/221:00:002
931172993927,8cyclictest30-21ksoftirqd/223:45:022
931172993926,10cyclictest1-21systemd23:10:002
931172993921,12cyclictest235-21systemd-journal23:00:012
931172993916,17cyclictest235-21systemd-journal20:55:012
93117299391,16cyclictest40850lldpd19:51:112
931167993927,9cyclictest947036-21sed00:15:031
931167993927,9cyclictest1-21systemd20:30:011
931167993925,10cyclictest23-21ksoftirqd/100:20:001
931167993924,11cyclictest272-21dbus-daemon23:25:001
931167993919,15cyclictest946199-40find00:00:011
931167993918,17cyclictest936220-21cat20:45:021
931167993917,17cyclictest235-21systemd-journal00:35:031
931167993917,17cyclictest235-21systemd-journal00:35:021
931167993915,19cyclictest943853-21systemd-run23:15:011
931167993915,19cyclictest940507-21sed22:10:011
931163993934,3cyclictest16950irq/224-800000021:56:190
93116399393,33cyclictest933962-21cron20:05:000
931163993928,9cyclictest12-21ksoftirqd/019:10:010
931163993925,11cyclictest942776-21sshd22:51:200
931172993826,8cyclictest937331-21sshd21:09:192
931172993823,11cyclictest272-21dbus-daemon20:00:012
931172993820,11cyclictest945130-21modprobe23:40:002
931167993825,9cyclictest1-21systemd19:40:001
931167993825,10cyclictest935143-21latency_hist20:25:031
931167993824,10cyclictest306-21in:imuxsock22:00:001
931167993823,12cyclictest941186-21sed22:20:021
931167993823,11cyclictest23-21ksoftirqd/121:10:011
931167993813,21cyclictest932087-21latency_hist19:25:021
931163993818,15cyclictest235-21systemd-journal21:15:000
93116399381,33cyclictest12-21ksoftirqd/023:55:010
93116399381,33cyclictest12-21ksoftirqd/023:55:000
93117299378,7cyclictest131rcu_preempt19:45:012
931172993727,6cyclictest941272-21cut22:25:012
931172993727,6cyclictest306-21in:imuxsock19:35:012
931172993726,8cyclictest944594-21date23:25:022
931172993726,8cyclictest938929-21tr21:40:002
93117299372,5cyclictest933186-21modprobe19:50:002
931172993725,9cyclictest948013-21cron00:35:002
931172993725,8cyclictest235-21systemd-journal19:29:182
93117299372,4cyclictest131rcu_preempt23:35:012
93117299372,3cyclictest131rcu_preempt21:05:012
93116799374,7cyclictest23-21ksoftirqd/123:52:311
93116799374,7cyclictest23-21ksoftirqd/123:52:301
931167993725,8cyclictest23-21ksoftirqd/120:05:011
931167993723,10cyclictest0-21swapper/120:00:001
931167993721,13cyclictest931326-21sshd19:10:181
931167993717,15cyclictest1-21systemd00:10:001
931167993717,14cyclictest235-21systemd-journal22:25:001
93116799371,32cyclictest1-21systemd20:35:011
931167993713,20cyclictest945416-21latency_hist23:45:021
931163993729,5cyclictest16950irq/224-800000021:20:010
931163993727,6cyclictest12-21ksoftirqd/023:25:020
931172993625,8cyclictest940580-21ls22:10:022
931172993624,9cyclictest947392-21wc00:20:022
931172993623,9cyclictest30-21ksoftirqd/221:58:202
931172993622,10cyclictest30-21ksoftirqd/220:20:022
931172993621,11cyclictest235-21systemd-journal19:25:002
931172993619,13cyclictest235-21systemd-journal21:30:002
931172993617,15cyclictest0-21swapper/221:55:012
931172993617,15cyclictest0-21swapper/221:55:012
931172993617,14cyclictest0-21swapper/223:15:002
931172993616,16cyclictest934077-21latency_hist20:05:022
931167993623,10cyclictest23-21ksoftirqd/120:20:031
931167993618,14cyclictest934229-21munin-run20:10:001
931167993613,19cyclictest945165-21grep23:40:011
931167993613,19cyclictest932560-21latency_hist19:35:021
931167993613,18cyclictest942049-21sendmail22:40:001
931163993631,3cyclictest0-21swapper/020:50:000
931163993618,14cyclictest940186-21sshd22:00:190
93117299356,6cyclictest131rcu_preempt20:15:012
931172993526,5cyclictest940389-21latency_hist22:05:022
931172993524,8cyclictest931537-21wc19:15:022
931172993524,7cyclictest947094-21latency_hist00:15:022
93116799354,5cyclictest23-21ksoftirqd/120:53:191
931167993522,10cyclictest934479-21munin-run20:15:001
931167993521,10cyclictest272-21dbus-daemon21:25:011
931167993519,13cyclictest947486-21sshd00:23:221
931167993519,13cyclictest947486-21sshd00:23:221
931167993512,18cyclictest933443-21modprobe19:55:001
931167993510,20cyclictest933094-21latency_hist19:45:031
93116399352,3cyclictest131rcu_preempt21:50:020
931163993522,10cyclictest12-21ksoftirqd/023:10:000
93116399351,30cyclictest0-21swapper/023:15:140
931167993427,4cyclictest931103-21latency_hist19:10:011
931167993421,10cyclictest939159-21sshd21:41:191
931167993412,18cyclictest947885-21latency_hist00:30:011
931163993420,11cyclictest932136-21sshd19:28:180
93116399341,7cyclictest1-21systemd21:50:200
93116399341,7cyclictest1-21systemd21:50:190
93116399341,31cyclictest0-21swapper/019:32:570
93116399341,30cyclictest0-21swapper/022:47:010
93116399341,30cyclictest0-21swapper/022:47:010
93116399341,30cyclictest0-21swapper/021:27:480
93117299333,8cyclictest362-21munin-node23:51:352
93117299333,8cyclictest362-21munin-node23:51:352
931172993322,5cyclictest306-21in:imuxsock22:41:192
931167993325,5cyclictest942807-21munin-plugin-st22:54:591
93116799332,10cyclictest1-21systemd23:50:011
931167993316,14cyclictest363-21snmpd21:32:031
93116399332,23cyclictest932963-21latency_hist19:45:010
931172993220,9cyclictest30-21ksoftirqd/200:28:092
931172993220,8cyclictest30-21ksoftirqd/200:00:002
931172993219,9cyclictest938759-21grep21:35:022
931172993218,11cyclictest272-21dbus-daemon22:40:012
93117299321,28cyclictest48-21kcompactd023:03:112
931167993222,8cyclictest23-21ksoftirqd/122:42:171
931167993221,8cyclictest23-21ksoftirqd/120:47:571
931167993220,9cyclictest281-21systemd-logind19:30:001
931167993219,11cyclictest23-21ksoftirqd/123:00:001
931167993214,14cyclictest363-21snmpd23:02:251
931163993219,9cyclictest948053-21latency_hist00:35:010
93116399321,4cyclictest0-21swapper/023:00:270
93117299312,21cyclictest934228-21modprobe20:10:002
931172993121,7cyclictest30-21ksoftirqd/220:35:022
931172993119,10cyclictest30-21ksoftirqd/220:45:012
93116399312,6cyclictest235-21systemd-journal20:20:490
931172993018,9cyclictest931297-21sed19:10:022
931172993016,9cyclictest306-21in:imuxsock00:07:202
931172992922,5cyclictest941664-21latency_hist22:30:012
931172992821,4cyclictest945640-21sshd23:46:202
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional