You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-10 - 01:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Wed Jun 10, 2026 00:44:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
272902930,3sleep20-21swapper/223:08:032
272902930,3sleep20-21swapper/223:08:032
303302710,3chrt0-21swapper/121:48:111
303302710,3chrt0-21swapper/121:48:111
51362700,3sleep30-21swapper/320:43:003
51362700,3sleep30-21swapper/320:43:003
53232630,7sleep1281ktimers/120:43:021
53232630,7sleep1281ktimers/120:43:021
238642630,5sleep123869-21unixbench_multi00:23:121
238642630,5sleep123869-21unixbench_multi00:23:121
10922620,2chrt0-21swapper/121:58:031
162902570,3chrt0-21swapper/100:03:141
162902570,3chrt0-21swapper/100:03:141
162902570,3chrt0-21swapper/100:03:141
35342540,2chrt3537-21idleruntime20:38:041
35342540,2chrt3537-21idleruntime20:38:041
220782540,3sleep129-21ksoftirqd/122:53:071
220782540,3sleep129-21ksoftirqd/122:53:071
220782540,3sleep129-21ksoftirqd/122:53:071
193522480,3sleep10-21swapper/122:47:521
193522480,3sleep10-21swapper/122:47:521
4146994736,7cyclictest0-21swapper/000:02:550
4146994736,7cyclictest0-21swapper/000:02:550
4146994736,7cyclictest0-21swapper/000:02:550
326432470,3chrt0-21swapper/123:23:021
326432470,3chrt0-21swapper/123:23:021
326432470,3chrt0-21swapper/123:23:021
98712430,2sleep110073-21sed20:53:121
98712430,2sleep110073-21sed20:53:121
51332400,3sleep30-21swapper/322:08:063
51332400,3sleep30-21swapper/322:08:063
197022400,13sleep1281ktimers/119:53:091
197022400,13sleep1281ktimers/119:53:091
299762390,3sleep00-21swapper/000:42:540
299762390,3sleep00-21swapper/000:42:540
260202390,2sleep30-21swapper/321:38:033
260202390,2sleep30-21swapper/321:38:033
81752370,2sleep10-21swapper/119:23:061
81752370,2sleep10-21swapper/119:23:061
4146993727,6cyclictest0-21swapper/000:03:480
4146993727,6cyclictest0-21swapper/000:03:480
4146993727,6cyclictest0-21swapper/000:03:480
4147993630,5cyclictest14173-21threads19:38:131
4147993630,5cyclictest14173-21threads19:38:131
414999351,24cyclictest15077-21chrt00:02:552
414999351,24cyclictest15077-21chrt00:02:552
414999351,24cyclictest15077-21chrt00:02:552
81872330,1sleep2351rcuc/219:23:072
81872330,1sleep2351rcuc/219:23:072
4147993322,2cyclictest181rcu_preempt22:40:311
4146993326,5cyclictest22317-21switchtime22:53:100
4146993326,5cyclictest22317-21switchtime22:53:100
4146993326,5cyclictest22317-21switchtime22:53:100
4147993225,5cyclictest25667-21/usr/sbin/munin21:38:091
4147993225,5cyclictest25667-21/usr/sbin/munin21:38:091
4147993225,5cyclictest2263-21fschecks_time23:28:021
4147993225,5cyclictest2263-21fschecks_time23:28:021
4147993225,5cyclictest10325-21cut22:23:021
4146993227,3cyclictest1805-21latency_hist23:28:030
4146993225,5cyclictest31711-21threads23:18:120
4146993225,5cyclictest31711-21threads23:18:120
4146993225,5cyclictest31711-21threads23:18:120
4146993225,5cyclictest25338-21fschecks_time23:03:020
4146993225,5cyclictest25338-21fschecks_time23:03:020
4146993225,5cyclictest25338-21fschecks_time23:03:020
4146993224,6cyclictest27885-21latency_hist20:17:490
4146993224,6cyclictest27885-21latency_hist20:17:490
39672320,4sleep30-21swapper/323:33:013
39672320,4sleep30-21swapper/323:33:013
39672320,4sleep30-21swapper/323:33:013
64942310,3sleep30-21swapper/323:38:073
64942310,3sleep30-21swapper/323:38:073
64942310,3sleep30-21swapper/323:38:073
4147993124,5cyclictest14166-21gltestperf22:33:031
4147993124,5cyclictest14166-21gltestperf22:33:031
414799311,3cyclictest700-21192.168.115.40-manager23:45:021
414799311,3cyclictest700-21192.168.115.40-manager23:45:021
414799311,3cyclictest700-21192.168.115.40-manager23:45:021
4146993125,4cyclictest545-21in:imuxsock21:38:020
4146993125,4cyclictest545-21in:imuxsock21:38:020
4146993124,5cyclictest23037-21head00:23:020
4146993124,5cyclictest23037-21head00:23:020
4146993123,6cyclictest22123-21fschecks_count21:28:020
4146993123,6cyclictest22123-21fschecks_count21:28:020
4146993123,6cyclictest22123-21fschecks_count21:28:020
4146993122,7cyclictest2866-21fschecks_time22:03:010
4146993122,7cyclictest2866-21fschecks_time22:03:010
4146993122,7cyclictest2866-21fschecks_time22:03:010
414799308,4cyclictest281ktimers/120:13:271
414799308,4cyclictest281ktimers/120:13:271
4147993026,3cyclictest90-1pr/ttyS020:48:281
4147993026,3cyclictest90-1pr/ttyS020:48:281
4147993025,4cyclictest19857-21sendmail21:22:491
4147993025,4cyclictest19857-21sendmail21:22:491
4147993025,4cyclictest19857-21sendmail21:22:491
4146993025,4cyclictest8018-21cut23:43:030
4146993025,4cyclictest8018-21cut23:43:030
4146993023,5cyclictest8480-21processes23:43:100
4146993023,5cyclictest8480-21processes23:43:100
4146993023,5cyclictest8480-21processes23:43:100
4146993022,6cyclictest4144-21cyclictest22:13:130
4146993022,6cyclictest4144-21cyclictest22:13:130
4146993022,6cyclictest25251-21tr00:28:080
4146993022,6cyclictest25251-21tr00:28:080
4146993022,6cyclictest25251-21tr00:28:080
285832300,1sleep30-21swapper/300:38:013
285832300,1sleep30-21swapper/300:38:013
142432300,3sleep30-21swapper/322:33:033
142432300,3sleep30-21swapper/322:33:033
121192300,2sleep012121-21fschecks_count22:28:010
121192300,2sleep012121-21fschecks_count22:28:010
121192300,2sleep012121-21fschecks_count22:28:010
415899292,20cyclictest15872-21chrt00:03:083
415899292,20cyclictest15872-21chrt00:03:083
415899292,20cyclictest15872-21chrt00:03:083
4147992925,2cyclictest271rcuc/120:08:571
4147992925,2cyclictest271rcuc/120:08:571
4147992925,2cyclictest271rcuc/120:08:571
4147992924,3cyclictest90-1pr/ttyS023:33:331
4147992924,3cyclictest90-1pr/ttyS023:33:331
4147992924,3cyclictest7369-21kworker/u9:1-xprtiod00:02:511
4147992924,3cyclictest7369-21kworker/u9:1-xprtiod00:02:511
4147992924,3cyclictest7369-21kworker/u9:1-xprtiod00:02:511
4147992924,3cyclictest22629-21needreboot21:28:071
4147992924,3cyclictest22629-21needreboot21:28:071
4147992924,3cyclictest22629-21needreboot21:28:071
4147992921,6cyclictest281ktimers/100:42:051
4147992921,6cyclictest281ktimers/100:42:051
4146992923,5cyclictest18394-21fschecks_count21:18:050
4146992923,5cyclictest18394-21fschecks_count21:18:050
4146992923,5cyclictest18394-21fschecks_count21:18:050
4146992922,5cyclictest1346-21avahi-daemon23:48:100
4146992922,5cyclictest1346-21avahi-daemon23:48:100
4146992922,5cyclictest1346-21avahi-daemon23:48:100
4146992919,8cyclictest13585-21threads21:03:120
4146992919,8cyclictest13585-21threads21:03:120
196802290,1sleep019781-21grep00:13:110
196802290,1sleep019781-21grep00:13:110
196802290,1sleep019781-21grep00:13:110
4147992823,3cyclictest22289-21latency_hist20:02:491
4147992823,3cyclictest22289-21latency_hist20:02:491
4147992823,2cyclictest0-21swapper/100:08:161
4147992823,2cyclictest0-21swapper/100:08:161
4147992823,2cyclictest0-21swapper/100:08:161
4147992821,5cyclictest29316-21head20:18:101
4147992821,5cyclictest29316-21head20:18:101
4147992820,2cyclictest181rcu_preempt22:08:101
4147992820,2cyclictest181rcu_preempt22:08:101
4147992819,7cyclictest18761-21cpu19:52:591
4147992819,7cyclictest18761-21cpu19:52:591
4147992819,2cyclictest181rcu_preempt22:05:301
4147992819,2cyclictest181rcu_preempt22:05:301
4147992819,2cyclictest181rcu_preempt22:05:301
4146992822,4cyclictest545-21in:imuxsock23:42:490
4146992821,5cyclictest9916-21latency_hist22:22:490
4146992821,5cyclictest9916-21latency_hist22:22:490
4146992821,5cyclictest5971-21fschecks_time23:38:020
4146992821,5cyclictest5971-21fschecks_time23:38:020
4146992821,5cyclictest11431-21cut19:33:020
4146992821,5cyclictest11431-21cut19:33:020
4146992821,4cyclictest0-21swapper/000:18:120
4146992821,4cyclictest0-21swapper/000:18:120
4146992820,6cyclictest17416-21latency_hist22:42:490
4146992819,7cyclictest17319-21fschecks_time00:08:060
4146992819,7cyclictest17319-21fschecks_time00:08:060
4146992819,7cyclictest17319-21fschecks_time00:08:060
4146992819,7cyclictest12100-21needreboot23:53:080
4146992819,7cyclictest12100-21needreboot23:53:080
4146992819,7cyclictest12100-21needreboot23:53:080
4146992818,7cyclictest3532-21idleruntime20:38:040
4146992818,7cyclictest3532-21idleruntime20:38:040
4146992818,7cyclictest0-21swapper/019:27:490
4146992818,7cyclictest0-21swapper/019:27:490
119842280,2sleep10-21swapper/119:33:071
119842280,2sleep10-21swapper/119:33:071
414799278,4cyclictest0-21swapper/121:33:041
414799278,4cyclictest0-21swapper/121:33:041
4147992723,3cyclictest90-1pr/ttyS020:35:581
4147992723,3cyclictest90-1pr/ttyS020:35:581
4147992722,3cyclictest20216-21kworker/u9:1-rpciod00:33:091
4147992722,3cyclictest20216-21kworker/u9:1-rpciod00:33:091
4147992718,7cyclictest15329-21gltestperf19:43:031
4147992718,7cyclictest15329-21gltestperf19:43:031
4147992718,6cyclictest16827-21irqstats21:13:041
4147992718,6cyclictest16827-21irqstats21:13:041
4147992718,6cyclictest16827-21irqstats21:13:041
4146992723,2cyclictest0-21swapper/020:03:120
4146992723,2cyclictest0-21swapper/020:03:120
4146992722,3cyclictest0-21swapper/023:13:050
4146992722,3cyclictest0-21swapper/023:13:050
4146992722,3cyclictest0-21swapper/023:13:050
4146992722,3cyclictest0-21swapper/022:02:490
4146992718,7cyclictest28585-21sort21:43:120
4146992718,7cyclictest28585-21sort21:43:120
4147992622,3cyclictest12374-21idleruntime22:28:031
4147992621,4cyclictest28292-21ntp_offset21:43:081
4147992621,4cyclictest28292-21ntp_offset21:43:081
4147992621,3cyclictest0-21swapper/122:23:021
4147992619,5cyclictest281ktimers/122:16:301
4147992619,5cyclictest281ktimers/122:16:301
4147992619,5cyclictest25087-21latency00:28:051
4147992619,5cyclictest25087-21latency00:28:051
4147992619,5cyclictest25087-21latency00:28:051
4147992619,4cyclictest0-21swapper/122:58:061
4147992619,4cyclictest0-21swapper/122:58:061
4147992619,4cyclictest0-21swapper/122:58:061
4147992618,6cyclictest20958-21fschecks_count00:18:031
4147992618,6cyclictest20958-21fschecks_count00:18:031
4146992622,3cyclictest15785-21ntp_offset19:43:080
4146992622,3cyclictest15785-21ntp_offset19:43:080
4146992621,3cyclictest0-21swapper/023:08:030
4146992621,3cyclictest0-21swapper/023:08:030
4146992620,4cyclictest0-21swapper/022:52:490
4146992620,4cyclictest0-21swapper/022:52:490
4146992620,4cyclictest0-21swapper/022:52:490
4146992620,3cyclictest0-21swapper/020:53:120
4146992620,3cyclictest0-21swapper/020:53:120
4146992614,10cyclictest14795-21cron19:42:490
4146992614,10cyclictest14795-21cron19:42:490
4147992520,3cyclictest0-21swapper/122:28:101
4147992519,4cyclictest4144-21cyclictest23:13:481
4147992519,4cyclictest4144-21cyclictest23:13:481
4147992519,4cyclictest4144-21cyclictest23:13:481
4147992519,3cyclictest0-21swapper/122:28:021
4147992519,3cyclictest0-21swapper/122:28:021
4147992517,6cyclictest2547-21cut23:28:041
4147992517,6cyclictest2547-21cut23:28:041
4147992517,6cyclictest2547-21cut23:28:041
4147992516,7cyclictest7633-21find23:43:001
4147992516,7cyclictest7633-21find23:43:001
4147992516,7cyclictest7633-21find23:43:001
4147992516,7cyclictest23218-21netstat20:03:071
4147992516,7cyclictest23218-21netstat20:03:071
4147992516,7cyclictest20586-21basename00:17:501
4147992516,7cyclictest20586-21basename00:17:501
4147992516,7cyclictest20586-21basename00:17:501
4147992516,4cyclictest271rcuc/119:30:261
4147992516,4cyclictest271rcuc/119:30:261
4147992513,9cyclictest14385-21latency_hist21:07:581
4147992513,9cyclictest14385-21latency_hist21:07:581
4146992521,2cyclictest0-21swapper/000:33:110
4146992521,2cyclictest0-21swapper/000:33:110
4146992519,3cyclictest0-21swapper/019:48:110
4146992519,3cyclictest0-21swapper/019:48:110
4146992518,5cyclictest32480-21missed_timers20:28:050
4146992518,5cyclictest32480-21missed_timers20:28:050
4146992518,5cyclictest25969-21sshd20:12:160
4146992518,5cyclictest25969-21sshd20:12:160
4146992518,5cyclictest25969-21sshd20:12:160
4146992518,5cyclictest0-21swapper/021:52:490
4146992518,5cyclictest0-21swapper/021:52:490
4147992420,2cyclictest0-21swapper/122:48:031
4147992420,2cyclictest0-21swapper/122:48:031
4147992420,2cyclictest0-21swapper/122:48:031
4147992418,4cyclictest30334-21head20:23:021
4147992418,4cyclictest30334-21head20:23:021
4147992418,4cyclictest30334-21head20:23:021
4147992418,4cyclictest0-21swapper/123:53:111
4147992418,4cyclictest0-21swapper/123:53:111
4147992418,4cyclictest0-21swapper/123:53:111
4147992418,3cyclictest0-21swapper/123:13:021
4147992418,3cyclictest0-21swapper/123:13:021
4147992418,3cyclictest0-21swapper/123:13:021
4147992417,4cyclictest0-21swapper/121:23:101
4147992417,4cyclictest0-21swapper/121:23:101
4147992417,4cyclictest0-21swapper/121:23:101
4147992416,6cyclictest32372-21cut20:28:041
4147992416,6cyclictest32372-21cut20:28:041
4146992417,4cyclictest0-21swapper/020:02:490
4146992417,4cyclictest0-21swapper/020:02:490
4146992417,4cyclictest0-21swapper/019:28:090
4146992417,4cyclictest0-21swapper/019:28:090
4146992416,6cyclictest3030-21latency_hist20:37:530
4146992416,6cyclictest3030-21latency_hist20:37:530
63292230,3sleep20-21swapper/222:13:002
63292230,3sleep20-21swapper/222:13:002
414799234,4cyclictest31359-21kworker/1:0-events22:33:101
414799234,4cyclictest31359-21kworker/1:0-events22:33:101
414799234,4cyclictest31359-21kworker/1:0-events22:33:101
4147992318,3cyclictest90-1pr/ttyS021:00:591
4147992318,3cyclictest90-1pr/ttyS021:00:591
4147992317,4cyclictest0-21swapper/123:48:041
4147992317,4cyclictest0-21swapper/123:48:041
4147992317,4cyclictest0-21swapper/123:48:041
4147992315,5cyclictest16102-21latency_hist21:13:031
4147992315,5cyclictest16102-21latency_hist21:13:031
4146992319,3cyclictest749-21runrttasks21:33:070
4146992319,3cyclictest749-21runrttasks21:33:070
4146992319,3cyclictest5235-21ntp_offset22:08:080
4146992319,3cyclictest5235-21ntp_offset22:08:080
4146992317,3cyclictest0-21swapper/023:33:020
4146992317,3cyclictest0-21swapper/023:33:020
4146992317,3cyclictest0-21swapper/022:33:100
4146992317,3cyclictest0-21swapper/022:33:100
4146992317,3cyclictest0-21swapper/022:33:100
4146992316,5cyclictest2131-21latency_hist23:28:020
414799227,4cyclictest0-21swapper/123:03:071
414799227,4cyclictest0-21swapper/123:03:071
414699222,10cyclictest16460-21fschecks_count21:13:010
414699222,10cyclictest16460-21fschecks_count21:13:010
414699222,10cyclictest16460-21fschecks_count21:13:010
4146992216,4cyclictest5140-21cpuspeed20:43:000
4146992216,4cyclictest5140-21cpuspeed20:43:000
4146992216,4cyclictest0-21swapper/020:23:020
4146992216,4cyclictest0-21swapper/020:23:020
4146992216,4cyclictest0-21swapper/020:23:020
4146992212,8cyclictest14678-21systemd-journal21:24:590
4146992212,8cyclictest14678-21systemd-journal21:24:590
4146992212,8cyclictest14678-21systemd-journal21:24:590
310902220,3chrt0-21swapper/221:52:502
310902220,3chrt0-21swapper/221:52:502
89692210,3sleep30-21swapper/319:25:263
89692210,3sleep30-21swapper/319:25:263
4158992116,3cyclictest241212sleep322:58:113
4158992116,3cyclictest241212sleep322:58:113
4158992116,3cyclictest241212sleep322:58:113
414999212,9cyclictest0-21swapper/200:03:022
414999212,9cyclictest0-21swapper/200:03:022
414999212,9cyclictest0-21swapper/200:03:022
4147992117,3cyclictest14819-21head21:08:041
4147992114,5cyclictest0-21swapper/121:57:501
4147992114,5cyclictest0-21swapper/121:57:501
414699215,6cyclictest7666-21tr20:48:080
414699215,6cyclictest7666-21tr20:48:080
414699212,3cyclictest0-21swapper/022:28:310
414699212,3cyclictest0-21swapper/022:28:310
414699212,3cyclictest0-21swapper/022:28:310
4146992117,2cyclictest0-21swapper/021:53:010
4146992117,2cyclictest0-21swapper/021:53:010
4146992116,3cyclictest0-21swapper/021:13:130
4146992116,3cyclictest0-21swapper/021:13:130
4146992116,3cyclictest0-21swapper/021:13:130
4146992116,3cyclictest0-21swapper/020:58:030
4146992116,3cyclictest0-21swapper/020:58:030
4146992115,4cyclictest2905-21mailstats23:28:100
4146992115,4cyclictest0-21swapper/022:43:030
4146992115,4cyclictest0-21swapper/022:43:030
4146992114,5cyclictest27685-21tr23:08:080
4146992114,5cyclictest27685-21tr23:08:080
4146992114,5cyclictest27685-21tr23:08:080
4146992112,6cyclictest19470-21netstat19:53:060
4146992112,6cyclictest19470-21netstat19:53:060
255622210,3sleep30-21swapper/320:08:113
255622210,3sleep30-21swapper/320:08:113
255622210,3sleep30-21swapper/320:08:113
415899191,13cyclictest12068-21chrt22:28:003
415899191,13cyclictest12068-21chrt22:28:003
415899191,13cyclictest12068-21chrt22:28:003
414699191,9cyclictest29804-21latency_hist20:22:500
414699191,9cyclictest29804-21latency_hist20:22:500
237922190,4sleep20-21swapper/221:33:002
237922190,4sleep20-21swapper/221:33:002
237922190,4sleep20-21swapper/221:33:002
414999183,9cyclictest0-21swapper/223:39:162
414999183,9cyclictest0-21swapper/223:39:162
414999183,9cyclictest0-21swapper/223:39:162
414999181,13cyclictest19610-21chrt22:48:012
414999181,13cyclictest19610-21chrt22:48:012
414999181,13cyclictest19610-21chrt22:48:012
414999181,12cyclictest22055-21chrt00:18:182
414999181,12cyclictest22055-21chrt00:18:182
414999181,12cyclictest19294-21chrt00:13:042
414999181,12cyclictest19294-21chrt00:13:042
414999181,12cyclictest19294-21chrt00:13:042
414999181,12cyclictest12640-21chrt23:53:142
414999181,12cyclictest12640-21chrt23:53:142
414999181,12cyclictest12640-21chrt23:53:142
414999181,12cyclictest1076-21taskset23:23:112
414999181,12cyclictest1076-21taskset23:23:112
414999171,11cyclictest7206-21chrt20:48:022
414999171,11cyclictest7206-21chrt20:48:022
414999171,11cyclictest4385-21taskset23:33:042
414999171,11cyclictest4385-21taskset23:33:042
414999171,11cyclictest28946-21taskset23:13:022
414999171,11cyclictest28946-21taskset23:13:022
414999171,11cyclictest28946-21taskset23:13:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional