You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-23 - 06:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Thu Oct 23, 2025 00:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
41385998229,19cyclictest56059-21sed23:50:023
41385997921,15cyclictest44179-21munin-run20:05:003
41385997522,18cyclictest43274-21cat19:45:013
41385996820,16cyclictest44971-21sendmail-msp20:20:013
41385996717,18cyclictest48991-21date21:35:033
41370996763,2cyclictest0-21swapper/022:50:020
41370996763,2cyclictest0-21swapper/022:50:010
41385996628,9cyclictest56501-21date00:00:033
41385996520,15cyclictest363-21snmpd19:11:163
41385996445,15cyclictest43612-21sshd19:50:363
41385996445,15cyclictest43612-21sshd19:50:353
41385996114,15cyclictest57957-21systemd-run00:30:003
41385995942,14cyclictest56870-21sshd00:05:053
41385995924,29cyclictest55100-21unin-run23:35:013
41385995924,29cyclictest55100-21unin-run23:35:013
41385995850,4cyclictest50929-21munin-run22:14:593
41385995842,13cyclictest49869-21sshd21:52:343
41385995713,12cyclictest57602-21cat00:20:033
41385995713,12cyclictest57602-21cat00:20:033
41385995640,13cyclictest58180-21sshd00:33:063
41385995640,12cyclictest52736-21sshd22:48:353
41385995640,12cyclictest52736-21sshd22:48:343
41385995640,11cyclictest54043-21munin-run23:15:003
41385995636,16cyclictest53502-21sshd23:00:353
41385995636,16cyclictest53502-21sshd23:00:343
41385995625,26cyclictest49097-21cron21:40:003
4138099565,3cyclictest131rcu_preempt19:30:012
41385995547,4cyclictest57682-21sshd00:24:063
41385995542,9cyclictest54807-21sshd23:27:043
41385995542,9cyclictest53243-21sshd22:59:353
41385995542,9cyclictest52208-21sshd22:39:363
41385995542,10cyclictest43377-21sshd19:48:363
41385995542,10cyclictest363-21snmpd19:36:223
41385995542,10cyclictest363-21snmpd19:36:213
41385995541,11cyclictest54285-21sshd23:19:363
41385995541,11cyclictest54285-21sshd23:19:363
41385995539,11cyclictest46777-21munin-run20:55:023
41385995539,11cyclictest46777-21munin-run20:55:013
41385995535,15cyclictest363-21snmpd23:37:483
41370995539,10cyclictest1-21systemd00:00:010
41385995442,9cyclictest48570-21sshd21:25:343
41385995441,9cyclictest48-21kcompactd019:29:343
41385995441,10cyclictest45188-21sshd20:20:353
41385995441,10cyclictest363-21snmpd19:21:193
41385995440,9cyclictest48-21kcompactd022:07:093
41385995439,10cyclictest48-21kcompactd023:21:283
41385995436,14cyclictest50187-21munin-run22:00:013
41385995436,11cyclictest41787-21sendmail-mta19:15:063
41385995435,15cyclictest363-21snmpd20:57:513
41385995431,16cyclictest308-21rs:main3
41385995345,4cyclictest51944-21sshd22:31:593
41385995345,4cyclictest47306-21munin-run21:05:003
41385995342,8cyclictest46488-21sshd20:47:353
41385995342,7cyclictest52518-21munin-run22:45:003
41385995341,9cyclictest52976-21sshd22:51:353
41385995340,9cyclictest48-21kcompactd023:40:543
41385995340,9cyclictest48-21kcompactd023:40:533
41385995340,10cyclictest56404-40mandb00:00:043
41385995340,10cyclictest56404-40mandb00:00:033
41385995340,10cyclictest48-21kcompactd020:05:383
41385995340,10cyclictest363-21snmpd21:46:483
41385995340,10cyclictest363-21snmpd21:16:433
41385995339,10cyclictest48-21kcompactd020:44:273
41385995336,13cyclictest1-21systemd00:15:003
41385995246,3cyclictest48-21kcompactd020:33:233
41385995239,9cyclictest48-21kcompactd021:21:383
41385995237,11cyclictest363-21snmpd20:36:343
41385995229,16cyclictest45440-21sshd20:25:353
4138099521,5cyclictest131rcu_preempt20:00:002
41370995234,14cyclictest272-21dbus-daemon21:25:010
4138599514,36cyclictest51180-21kworker/u8:322:30:013
41385995143,4cyclictest44696-21munin-run20:15:003
41385995137,9cyclictest51177-21cron22:20:003
41385995136,11cyclictest363-21snmpd21:06:413
41380995132,14cyclictest1-21systemd20:15:012
41375995136,10cyclictest235-21systemd-journal00:39:591
4137099511,33cyclictest51180-21kworker/u8:322:40:030
41385995038,9cyclictest363-21snmpd00:36:273
41385995036,10cyclictest363-21snmpd22:23:153
41385995035,12cyclictest56088-21sshd23:50:353
41385995024,21cyclictest43896-21modprobe19:59:593
41380995033,13cyclictest235-21systemd-journal20:20:002
41385994920,24cyclictest49514-21latency_hist21:45:023
41380994930,13cyclictest56404-21mandb00:00:002
41375994933,12cyclictest235-21systemd-journal23:39:591
4137099494,3cyclictest131rcu_preempt19:59:590
4137099494,3cyclictest131rcu_preempt00:35:010
41385994841,4cyclictest363-21snmpd22:01:513
41385994733,10cyclictest42579-21modprobe19:30:343
41380994730,13cyclictest272-21dbus-daemon21:35:012
41385994616,26cyclictest41409-21sed19:10:023
4138099461,3cyclictest131rcu_preempt22:20:012
4138099461,3cyclictest131rcu_preempt21:05:012
41375994631,11cyclictest272-21dbus-daemon20:20:011
4137099466,7cyclictest131rcu_preempt19:55:030
4137099466,7cyclictest131rcu_preempt19:55:020
41385994532,10cyclictest48-21kcompactd021:11:063
41380994529,12cyclictest1-21systemd22:55:002
41375994531,12cyclictest23-21ksoftirqd/100:25:001
4137099456,4cyclictest131rcu_preempt20:55:020
4137099456,4cyclictest131rcu_preempt20:55:010
4137099456,3cyclictest131rcu_preempt21:10:020
41370994516,25cyclictest49083-21sshd21:36:520
41380994429,11cyclictest235-21systemd-journal21:36:532
4138099442,3cyclictest131rcu_preempt19:50:002
41375994426,13cyclictest1-21systemd21:05:011
4137099444,3cyclictest131rcu_preempt19:41:340
4137099444,3cyclictest131rcu_preempt00:36:150
41370994437,4cyclictest16950irq/224-800000000:20:010
41370994437,4cyclictest16950irq/224-800000000:20:000
41370994429,13cyclictest12-21ksoftirqd/000:29:590
41370994429,12cyclictest12-21ksoftirqd/020:25:000
41370994427,13cyclictest55062-21sshd23:32:340
41370994427,13cyclictest55062-21sshd23:32:340
41370994426,13cyclictest41820-21cron19:20:000
41370994425,14cyclictest272-21dbus-daemon20:15:000
41370994424,15cyclictest1-21systemd19:50:010
4137099442,38cyclictest54210-21sed23:15:010
4138099436,4cyclictest131rcu_preempt23:10:002
4138099434,5cyclictest131rcu_preempt00:40:002
4138099433,3cyclictest131rcu_preempt22:15:012
4138099433,3cyclictest131rcu_preempt00:15:022
41380994326,13cyclictest272-21dbus-daemon23:50:012
4138099432,4cyclictest131rcu_preempt22:59:592
4138099431,4cyclictest131rcu_preempt23:40:002
4138099431,4cyclictest131rcu_preempt22:35:002
41375994332,7cyclictest272-21dbus-daemon23:30:001
41375994322,16cyclictest272-21dbus-daemon00:05:011
41375994322,16cyclictest272-21dbus-daemon00:05:001
4137099435,4cyclictest131rcu_preempt20:20:000
4137099435,4cyclictest131rcu_preempt19:40:040
4137099435,4cyclictest131rcu_preempt19:40:030
4138099423,3cyclictest131rcu_preempt22:45:352
4138099423,3cyclictest131rcu_preempt22:45:342
41380994228,10cyclictest1-21systemd00:35:002
41380994227,11cyclictest1-21systemd21:25:022
4138099422,4cyclictest131rcu_preempt19:44:592
4138099421,3cyclictest131rcu_preempt20:30:012
4138099421,3cyclictest131rcu_preempt19:40:042
4138099421,3cyclictest131rcu_preempt19:40:032
4137599426,26cyclictest53256-21modprobe22:59:591
4137599426,22cyclictest23-21ksoftirqd/122:50:031
41375994227,8cyclictest272-21dbus-daemon22:50:011
41375994227,8cyclictest272-21dbus-daemon22:50:001
4137099425,4cyclictest131rcu_preempt22:15:000
4137099425,3cyclictest131rcu_preempt21:05:020
4137099424,4cyclictest131rcu_preempt21:30:000
4137099424,4cyclictest131rcu_preempt20:40:000
4137099423,4cyclictest131rcu_preempt22:35:000
41370994228,10cyclictest50392-21sshd22:01:340
41370994227,11cyclictest1-21systemd23:25:000
4137099422,4cyclictest131rcu_preempt19:30:000
41370994222,13cyclictest1-21systemd23:30:010
4138099414,3cyclictest131rcu_preempt23:55:002
41380994129,9cyclictest50127-21sshd21:58:352
4138099412,4cyclictest131rcu_preempt23:20:012
4138099412,4cyclictest131rcu_preempt23:20:002
4138099412,4cyclictest131rcu_preempt23:03:362
4138099412,4cyclictest131rcu_preempt23:03:352
4138099412,3cyclictest131rcu_preempt22:25:002
4138099412,3cyclictest131rcu_preempt19:10:012
4138099411,4cyclictest131rcu_preempt23:35:012
4138099411,4cyclictest131rcu_preempt23:35:002
4138099411,4cyclictest131rcu_preempt21:18:352
4138099411,3cyclictest131rcu_preempt21:55:002
4137599417,9cyclictest23-21ksoftirqd/100:10:001
41375994127,9cyclictest1-21systemd19:35:011
4137099414,4cyclictest131rcu_preempt21:40:350
4137099414,3cyclictest131rcu_preempt20:35:020
41370994131,6cyclictest306-21in:imuxsock00:15:000
4137099412,6cyclictest308-21rs:main0
41370994124,12cyclictest272-21dbus-daemon21:00:000
4138099403,4cyclictest131rcu_preempt23:15:002
4138099403,3cyclictest131rcu_preempt19:55:012
4138099403,3cyclictest131rcu_preempt19:55:012
41380994032,5cyclictest50676-21munin-plugin-st22:10:002
4138099402,3cyclictest131rcu_preempt21:46:492
4138099402,3cyclictest131rcu_preempt20:06:342
4138099402,3cyclictest0-21swapper/223:25:352
4138099401,4cyclictest131rcu_preempt23:22:342
4138099401,3cyclictest131rcu_preempt20:35:012
4137599408,20cyclictest49104-21modprobe21:40:011
41375994032,5cyclictest41631-21latency_hist19:15:011
41375994031,5cyclictest306-21in:imuxsock22:30:001
41375994030,6cyclictest53524-21cron23:05:011
41375994030,6cyclictest53524-21cron23:05:001
41375994029,7cyclictest235-21systemd-journal20:30:351
41375994028,10cyclictest23-21ksoftirqd/122:35:031
41375994026,10cyclictest272-21dbus-daemon20:55:011
41375994026,10cyclictest272-21dbus-daemon20:55:001
4137099404,4cyclictest131rcu_preempt20:05:000
41370994028,9cyclictest12-21ksoftirqd/023:55:010
41370994026,10cyclictest41553-21sshd19:13:350
4138099394,4cyclictest131rcu_preempt20:36:472
41380993927,9cyclictest57775-21sed00:25:022
41380993927,9cyclictest1-21systemd00:00:042
41380993927,9cyclictest1-21systemd00:00:032
4138099392,6cyclictest131rcu_preempt20:21:462
41380993926,9cyclictest56915-21systemd-run00:10:012
4138099392,4cyclictest131rcu_preempt20:51:342
4138099392,4cyclictest131rcu_preempt20:51:342
4138099392,3cyclictest131rcu_preempt21:14:342
4138099392,3cyclictest131rcu_preempt19:24:592
41380993922,12cyclictest272-21dbus-daemon20:45:002
4138099391,4cyclictest131rcu_preempt21:41:472
4138099391,3cyclictest131rcu_preempt00:25:072
41375993928,8cyclictest56611-21date00:00:031
41375993928,6cyclictest47027-21modprobe20:59:591
41375993927,8cyclictest235-21systemd-journal21:15:001
41375993926,8cyclictest23-21ksoftirqd/123:10:001
41375993925,10cyclictest49788-21sed21:50:021
4137099394,3cyclictest131rcu_preempt23:00:030
4137099394,3cyclictest131rcu_preempt00:25:000
41370993927,9cyclictest12-21ksoftirqd/020:50:030
41370993926,9cyclictest53008-21munin-plugin-st22:54:590
41370993923,12cyclictest306-21in:imuxsock23:05:010
41370993923,12cyclictest306-21in:imuxsock23:05:000
41370993921,13cyclictest1-21systemd21:35:010
41370993918,6cyclictest16850irq/134-800000019:20:350
4137099391,6cyclictest131rcu_preempt20:30:010
41380993827,7cyclictest30-21ksoftirqd/221:00:012
41380993818,14cyclictest272-21dbus-daemon22:05:012
4137599383,3cyclictest131rcu_preempt23:15:021
41375993827,6cyclictest306-21in:imuxsock19:50:001
41375993823,11cyclictest57436-21tr00:20:011
41375993823,11cyclictest57436-21tr00:20:001
41375993822,8cyclictest56124-21cut23:55:001
41375993822,13cyclictest272-21dbus-daemon20:05:011
41375993822,11cyclictest272-21dbus-daemon19:20:011
41375993819,15cyclictest52232-21run-parts22:40:001
4137099383,4cyclictest131rcu_preempt22:08:360
4137099383,30cyclictest235-21systemd-journal23:45:250
4137099382,6cyclictest49666-21ls21:50:010
41370993826,10cyclictest12-21ksoftirqd/020:07:360
41370993825,8cyclictest1-21systemd22:30:000
41370993824,11cyclictest235-21systemd-journal22:20:360
4138099372,3cyclictest131rcu_preempt23:45:012
4138099372,3cyclictest131rcu_preempt23:45:002
41380993723,11cyclictest30-21ksoftirqd/222:30:002
4138099371,3cyclictest131rcu_preempt21:30:002
41375993727,7cyclictest23-21ksoftirqd/121:30:021
41375993725,9cyclictest23-21ksoftirqd/122:20:011
41375993725,8cyclictest23-21ksoftirqd/120:40:011
41375993724,9cyclictest23-21ksoftirqd/123:45:011
41375993724,9cyclictest23-21ksoftirqd/123:45:011
41375993724,9cyclictest23-21ksoftirqd/120:25:001
41375993722,9cyclictest272-21dbus-daemon22:45:011
41375993720,13cyclictest23-21ksoftirqd/119:40:001
41375993720,13cyclictest23-21ksoftirqd/119:40:001
4137099371,4cyclictest131rcu_preempt19:35:010
4137099371,3cyclictest0-21swapper/000:05:010
4137099371,3cyclictest0-21swapper/000:05:010
4137099371,32cyclictest1-21systemd00:05:050
4138099362,4cyclictest131rcu_preempt20:47:362
41380993624,9cyclictest44208-21sed20:05:012
4138099362,3cyclictest131rcu_preempt00:20:002
4138099362,3cyclictest131rcu_preempt00:20:002
41380993622,9cyclictest42615-21cut19:35:002
41380993621,9cyclictest52212-21cron22:40:012
4137599368,18cyclictest50117-21kworker/u8:022:15:011
4137599366,15cyclictest50137-21modprobe21:59:591
41375993628,5cyclictest49389-21munin-plugin-st21:44:591
41375993626,7cyclictest23-21ksoftirqd/100:15:011
41375993624,9cyclictest48322-21sshd21:21:351
41375993622,10cyclictest23-21ksoftirqd/122:25:011
41375993621,10cyclictest235-21systemd-journal23:17:031
41375993621,10cyclictest235-21systemd-journal23:17:021
41375993618,14cyclictest1-21systemd20:10:011
41375993617,15cyclictest272-21dbus-daemon19:45:011
41375993612,15cyclictest23-21ksoftirqd/119:59:591
4137099362,30cyclictest1-21systemd21:51:340
41370993619,13cyclictest272-21dbus-daemon23:20:020
41370993619,13cyclictest272-21dbus-daemon23:20:020
41370993617,15cyclictest272-21dbus-daemon21:15:000
4137099361,4cyclictest131rcu_preempt22:20:000
41370993614,6cyclictest16850irq/134-800000023:42:050
41370993614,6cyclictest16850irq/134-800000023:42:040
41380993527,5cyclictest41798-21cron19:17:002
4137599355,6cyclictest131rcu_preempt20:30:021
4137599354,3cyclictest131rcu_preempt20:45:001
41375993528,4cyclictest41460-21latency_hist19:10:021
41375993520,8cyclictest0-21swapper/121:20:001
4137099351,6cyclictest55313-21sshd23:37:050
4138099341,3cyclictest131rcu_preempt21:05:352
4137599348,17cyclictest51180-21kworker/u8:300:35:021
41375993424,7cyclictest48895-21sed21:35:011
41375993423,8cyclictest23-21ksoftirqd/120:50:021
41375993422,10cyclictest23-21ksoftirqd/121:55:021
4137099342,3cyclictest131rcu_preempt23:10:010
4137099341,30cyclictest48-21kcompactd022:40:250
4137099341,10cyclictest412-21sshd21:15:350
4138099333,3cyclictest131rcu_preempt22:45:022
41380993320,9cyclictest41558-21cron19:15:012
4137599332,4cyclictest131rcu_preempt21:10:001
41375993323,7cyclictest50392-21sshd22:01:351
41370993319,11cyclictest12-21ksoftirqd/020:45:000
4137099331,3cyclictest131rcu_preempt19:10:010
41375993225,4cyclictest42209-21latency_hist19:25:011
41375993223,5cyclictest42363-21munin-plugin-st19:30:001
41375993220,9cyclictest43631-21sshd19:54:351
41375993220,9cyclictest43631-21sshd19:54:341
41375993219,10cyclictest55841-21sshd23:46:341
41375993217,12cyclictest1-21systemd23:35:021
41375993217,12cyclictest1-21systemd23:35:011
41375993212,16cyclictest0-21swapper/120:14:591
4137599312,3cyclictest131rcu_preempt22:10:021
41375993123,5cyclictest54709-21latency_hist23:25:021
41375993020,7cyclictest23-21ksoftirqd/100:30:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional