You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-13 - 17:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Mon Apr 13, 2026 12:44:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2308221290,2sleep023085-21sed10:02:110
2308221290,2sleep023085-21sed10:02:110
131172930,2sleep00-21swapper/009:37:030
131172930,2sleep00-21swapper/009:37:030
17422770,3sleep20-21swapper/210:32:122
17422770,3sleep20-21swapper/210:32:122
64052650,3chrt0-21swapper/210:42:212
64052650,3chrt0-21swapper/210:42:212
64052650,3chrt0-21swapper/210:42:212
64052650,3chrt0-21swapper/210:42:212
228892650,3chrt0-21swapper/308:36:563
153392630,3sleep10-21swapper/108:16:041
153392630,3sleep10-21swapper/108:16:041
178152580,3chrt0-21swapper/209:47:182
178152580,3chrt0-21swapper/209:47:182
35262570,3chrt0-21swapper/210:37:112
35262570,3chrt0-21swapper/210:37:112
35262570,3chrt0-21swapper/210:37:112
263432550,4sleep20-21swapper/211:37:122
263432550,4sleep20-21swapper/211:37:122
128892550,4sleep2351rcuc/212:27:172
128892550,4sleep2351rcuc/212:27:172
48162540,3chrt0-21swapper/312:07:083
48162540,3chrt0-21swapper/312:07:083
223732500,3sleep30-21swapper/310:01:583
223732500,3sleep30-21swapper/310:01:583
100602490,3sleep20-21swapper/212:22:032
100602490,3sleep20-21swapper/212:22:032
100602490,3sleep20-21swapper/212:22:032
184262470,3chrt0-21swapper/008:22:180
184262470,3chrt0-21swapper/008:22:180
264302460,1sleep10-21swapper/111:37:161
264302460,1sleep10-21swapper/111:37:161
164452460,4sleep20-21swapper/212:37:142
164452460,4sleep20-21swapper/212:37:142
164452460,4sleep20-21swapper/212:37:142
105462450,3chrt0-21swapper/310:56:573
105462450,3chrt0-21swapper/310:56:573
82372430,3sleep30-21swapper/307:57:083
82372430,3sleep30-21swapper/307:57:083
82372430,3sleep30-21swapper/307:57:083
66982410,3sleep20-21swapper/212:12:102
66982410,3sleep20-21swapper/212:12:102
283432410,1sleep128462-21switchtime07:22:181
283432410,1sleep128462-21switchtime07:22:181
147062400,2sleep014788-21/usr/sbin/munin08:12:180
147062400,2sleep014788-21/usr/sbin/munin08:12:180
23830993831,5cyclictest5449-21threads07:47:180
23830993831,5cyclictest5449-21threads07:47:180
296632370,3sleep30-21swapper/308:52:193
296632370,3sleep30-21swapper/308:52:193
23831993728,7cyclictest8868-21netstat07:57:151
23831993728,7cyclictest8868-21netstat07:57:151
23831993728,7cyclictest8868-21netstat07:57:151
23830993730,6cyclictest9176-21threads07:57:190
23830993730,6cyclictest9176-21threads07:57:190
23830993730,6cyclictest9176-21threads07:57:190
23830993629,5cyclictest23824-21cyclictest11:07:180
23830993629,5cyclictest23824-21cyclictest11:07:180
200382360,3sleep00-21swapper/008:27:120
200382360,3sleep00-21swapper/008:27:120
23831993428,3cyclictest23824-21cyclictest11:57:171
23831993428,3cyclictest23824-21cyclictest11:57:171
23831993426,6cyclictest9471-21cpu09:27:091
196022340,3sleep20-21swapper/209:52:162
196022340,3sleep20-21swapper/209:52:162
297402330,2sleep029819-21irqstats07:27:110
297402330,2sleep029819-21irqstats07:27:110
28372330,3sleep20-21swapper/212:02:092
28372330,3sleep20-21swapper/212:02:092
28372330,3sleep20-21swapper/212:02:092
23831993326,5cyclictest550-21cron11:42:091
23831993326,5cyclictest550-21cron11:42:091
23830993326,5cyclictest7382-21sendmail09:21:570
23830993326,5cyclictest7382-21sendmail09:21:570
27202320,2sleep30-21swapper/307:42:093
27202320,2sleep30-21swapper/307:42:093
23830993126,4cyclictest23824-21cyclictest09:12:090
23830993126,4cyclictest23824-21cyclictest09:12:090
23830993126,3cyclictest28504-21latency_hist08:51:560
23830993126,3cyclictest28504-21latency_hist08:51:560
23830993124,5cyclictest6693-21sshd10:45:540
23830993124,5cyclictest6693-21sshd10:45:540
23830993124,5cyclictest6693-21sshd10:45:540
23830993124,5cyclictest6693-21sshd10:45:540
23830993122,7cyclictest8612-21threads09:22:190
23830993122,7cyclictest8612-21threads09:22:190
23830993122,7cyclictest8612-21threads09:22:190
23831993026,3cyclictest4579-21ntp_offset09:12:151
23831993026,3cyclictest4579-21ntp_offset09:12:151
23831993024,4cyclictest22320-21systemctl10:01:561
23831993024,4cyclictest22320-21systemctl10:01:561
23831993023,5cyclictest11960-21cpu08:07:081
23831993023,5cyclictest11960-21cpu08:07:081
23831993022,6cyclictest27138-21gltestperf08:47:081
23831993021,7cyclictest16710-21timerwakeupswit08:17:191
23831993021,7cyclictest16710-21timerwakeupswit08:17:191
23830993025,3cyclictest27209-21gltestperf08:47:080
23830993024,5cyclictest3967-21sshd07:43:480
23830993024,5cyclictest3967-21sshd07:43:480
23830993024,5cyclictest21416-21cut08:32:110
23830993021,6cyclictest171ktimers/007:22:180
23830993021,6cyclictest171ktimers/007:22:180
44722290,2sleep04475-21grep10:37:210
44722290,2sleep04475-21grep10:37:210
44722290,2sleep04475-21grep10:37:210
312892290,3sleep20-21swapper/207:32:072
312892290,3sleep20-21swapper/207:32:072
23831992921,6cyclictest4611-21fschecks_count07:47:091
23831992921,6cyclictest4611-21fschecks_count07:47:091
23831992921,6cyclictest445-21threads10:27:181
23831992920,7cyclictest648-21latency_hist11:57:031
23831992919,7cyclictest24671-21fschecks_time10:07:101
23831992919,7cyclictest24671-21fschecks_time10:07:101
23830992925,3cyclictest9805-21cut09:27:110
23830992925,3cyclictest32403-21timerwakeupswit11:52:200
23830992924,4cyclictest142-21systemd-journal08:07:540
23830992924,4cyclictest142-21systemd-journal08:07:540
23830992922,5cyclictest4672-21latency_hist12:07:070
23830992922,5cyclictest4672-21latency_hist12:07:070
23830992922,5cyclictest4672-21latency_hist12:07:070
23830992922,5cyclictest23325-21fschecks_time08:37:090
23830992922,5cyclictest23325-21fschecks_time08:37:090
23830992922,4cyclictest544-21lldpd08:44:340
23830992922,4cyclictest544-21lldpd08:44:340
23830992920,7cyclictest17616-21cat12:42:070
242502280,3sleep20-21swapper/208:37:192
242502280,3sleep20-21swapper/208:37:192
23831992824,3cyclictest3310-21ntp_offset07:42:141
23831992824,3cyclictest3310-21ntp_offset07:42:141
23831992824,3cyclictest23824-21cyclictest12:17:141
23831992824,3cyclictest23824-21cyclictest12:17:141
23831992824,3cyclictest23824-21cyclictest12:17:141
23831992823,3cyclictest13351-21ntp_offset11:02:151
23831992823,3cyclictest13351-21ntp_offset11:02:151
23831992821,5cyclictest6199-21kernelversion09:17:121
23831992821,5cyclictest6199-21kernelversion09:17:121
23831992821,5cyclictest3083-21timerwakeupswit09:07:191
23831992821,5cyclictest3083-21timerwakeupswit09:07:191
23831992820,6cyclictest23824-21cyclictest08:24:431
23831992820,6cyclictest23824-21cyclictest08:24:431
23831992820,6cyclictest12716-21netstat12:27:141
23831992820,6cyclictest12716-21netstat12:27:141
23831992819,7cyclictest550-21cron09:56:561
23831992819,7cyclictest550-21cron09:56:561
23830992823,3cyclictest90-1pr/ttyS010:18:520
23830992823,3cyclictest90-1pr/ttyS010:18:520
23830992823,3cyclictest90-1pr/ttyS010:18:520
23830992823,3cyclictest90-1pr/ttyS007:39:520
23830992823,3cyclictest90-1pr/ttyS007:39:520
23830992822,4cyclictest0-21swapper/011:41:570
23830992822,4cyclictest0-21swapper/011:41:570
23830992821,5cyclictest13642-21sshd12:31:530
23830992821,5cyclictest13642-21sshd12:31:530
23830992821,5cyclictest0-21swapper/012:17:180
23830992821,5cyclictest0-21swapper/012:17:180
23830992821,5cyclictest0-21swapper/012:17:180
23831992721,5cyclictest22801-21processes11:27:171
23831992721,5cyclictest22801-21processes11:27:171
23831992720,5cyclictest7939-21latency_hist07:56:561
23831992720,5cyclictest32207-21threads07:32:181
23831992720,5cyclictest32207-21threads07:32:181
23831992720,5cyclictest14587-21ntp_kernel_err12:32:151
23831992720,5cyclictest14587-21ntp_kernel_err12:32:151
23830992723,3cyclictest90-1pr/ttyS011:31:540
23830992723,3cyclictest90-1pr/ttyS011:31:540
23830992722,3cyclictest90-1pr/ttyS010:36:530
23830992722,3cyclictest90-1pr/ttyS010:36:530
23830992722,3cyclictest90-1pr/ttyS007:57:030
23830992721,5cyclictest25464-21latency_hist11:36:570
23830992721,5cyclictest18687-21kernelversion11:17:110
23830992721,5cyclictest18687-21kernelversion11:17:110
23830992721,5cyclictest13513-21gltestperf09:37:090
23830992721,5cyclictest13513-21gltestperf09:37:090
23830992721,4cyclictest749-21runrttasks12:27:080
23830992720,5cyclictest503-21kernelversion09:02:110
23830992720,5cyclictest503-21kernelversion09:02:110
23830992720,5cyclictest23824-21cyclictest10:08:150
23830992720,5cyclictest23824-21cyclictest10:08:150
23830992720,5cyclictest17011-21needreboot11:12:150
23830992720,5cyclictest17011-21needreboot11:12:150
169282270,2sleep016929-21sort09:47:080
169282270,2sleep016929-21sort09:47:080
23835992619,4cyclictest0-21swapper/307:50:333
23835992619,4cyclictest0-21swapper/307:50:333
23835992616,6cyclictest7985-21chrt12:16:213
23835992616,6cyclictest7985-21chrt12:16:213
23835992615,8cyclictest29457-21chrt10:17:503
23835992615,8cyclictest29457-21chrt10:17:503
23835992615,8cyclictest29457-21chrt10:17:503
23835992615,6cyclictest27085-21chrt11:38:543
23835992615,6cyclictest27085-21chrt11:38:543
23835992614,7cyclictest5312-21chrt09:14:473
23835992614,7cyclictest5312-21chrt09:14:473
23831992622,3cyclictest30768-21ntp_offset10:22:151
23831992622,3cyclictest30768-21ntp_offset10:22:151
23831992622,3cyclictest30768-21ntp_offset10:22:151
23831992622,2cyclictest0-21swapper/111:01:561
23831992622,2cyclictest0-21swapper/111:01:561
23831992620,3cyclictest0-21swapper/110:12:091
23831992620,3cyclictest0-21swapper/110:12:091
23831992620,3cyclictest0-21swapper/108:27:091
23831992620,3cyclictest0-21swapper/108:27:091
23831992619,6cyclictest5856-21sshd12:07:531
23831992619,6cyclictest5856-21sshd12:07:531
23830992622,2cyclictest0-21swapper/010:12:100
23830992622,2cyclictest0-21swapper/010:12:100
23830992621,3cyclictest0-21swapper/008:02:100
23830992621,3cyclictest0-21swapper/008:02:100
23830992617,7cyclictest7672-21needreboot10:47:160
23830992614,7cyclictest90-1pr/ttyS009:52:350
23830992614,7cyclictest90-1pr/ttyS009:52:350
284652250,3chrt0-21swapper/207:22:182
284652250,3chrt0-21swapper/207:22:182
23835992516,6cyclictest32134-21chrt09:00:443
23835992516,6cyclictest32134-21chrt09:00:443
23835992515,7cyclictest26588-21chrt08:46:163
23835992515,7cyclictest26588-21chrt08:46:163
23835992515,7cyclictest26588-21chrt08:46:163
23835992514,7cyclictest29147-21chrt11:46:143
23835992514,7cyclictest29147-21chrt11:46:143
23831992520,4cyclictest572-21cups-browsed10:17:191
23831992520,4cyclictest572-21cups-browsed10:17:191
23831992520,4cyclictest572-21cups-browsed10:17:191
23831992520,4cyclictest23824-21cyclictest08:52:211
23831992520,4cyclictest23824-21cyclictest08:52:211
23831992519,4cyclictest19844-21latency_hist11:22:061
23831992519,4cyclictest19844-21latency_hist11:22:061
23831992519,4cyclictest15804-21ntp_kernel_pll_09:42:151
23831992519,4cyclictest15804-21ntp_kernel_pll_09:42:151
23831992519,4cyclictest15804-21ntp_kernel_pll_09:42:151
23831992519,3cyclictest0-21swapper/109:22:181
23831992519,3cyclictest0-21swapper/109:22:181
23831992519,3cyclictest0-21swapper/108:46:561
23831992519,3cyclictest0-21swapper/108:46:561
23831992518,5cyclictest0-21swapper/107:27:171
23831992518,5cyclictest0-21swapper/107:27:171
23830992521,3cyclictest90-1pr/ttyS010:27:500
23830992521,2cyclictest0-21swapper/007:32:190
23830992521,2cyclictest0-21swapper/007:32:190
23830992520,3cyclictest15784-21ntp_kernel_err09:42:140
23830992520,3cyclictest0-21swapper/011:22:160
23830992520,3cyclictest0-21swapper/011:22:160
23830992519,5cyclictest23824-21cyclictest08:52:120
23830992519,5cyclictest23824-21cyclictest08:52:120
23830992519,4cyclictest0-21swapper/011:02:120
23830992519,4cyclictest0-21swapper/011:02:120
23830992519,3cyclictest0-21swapper/011:47:120
23830992519,3cyclictest0-21swapper/011:47:120
23830992519,3cyclictest0-21swapper/011:47:120
23830992517,6cyclictest11008-21cut10:57:110
23830992517,6cyclictest11008-21cut10:57:110
23830992516,7cyclictest5519-21ls12:07:160
23830992516,7cyclictest5519-21ls12:07:160
23830992510,13cyclictest16194-21ls08:17:120
23830992510,13cyclictest16194-21ls08:17:120
23835992414,7cyclictest30632-21chrt07:27:383
23835992414,7cyclictest30632-21chrt07:27:383
23835992414,7cyclictest23428-21chrt11:29:543
23835992414,7cyclictest23428-21chrt11:29:543
23835992414,7cyclictest1585-21chrt09:04:583
23835992414,7cyclictest1585-21chrt09:04:583
23835992414,6cyclictest16494-21chrt09:44:093
23835992414,6cyclictest16494-21chrt09:44:093
23835992414,6cyclictest16494-21chrt09:44:093
23835992413,7cyclictest25414-21chrt11:36:253
23835992413,7cyclictest21483-21chrt11:23:513
23835992413,7cyclictest21483-21chrt11:23:513
23835992413,7cyclictest13475-21chrt08:11:033
23835992413,7cyclictest13475-21chrt08:11:033
23835992413,6cyclictest5833-21chrt12:07:363
23835992413,6cyclictest5833-21chrt12:07:363
23835992410,10cyclictest24060-21chrt10:05:163
23835992410,10cyclictest24060-21chrt10:05:163
23831992419,3cyclictest17277-21grep09:47:101
23831992419,3cyclictest17277-21grep09:47:101
23831992419,3cyclictest0-21swapper/108:47:131
23831992419,3cyclictest0-21swapper/108:47:131
23831992417,6cyclictest1-21systemd08:35:531
23831992416,6cyclictest3508-21fschecks_time10:37:111
23831992416,6cyclictest3508-21fschecks_time10:37:111
23831992416,6cyclictest3508-21fschecks_time10:37:111
2383099249,4cyclictest181rcu_preempt12:37:080
2383099249,4cyclictest181rcu_preempt12:37:080
23830992419,3cyclictest90-1pr/ttyS010:24:010
23830992419,3cyclictest90-1pr/ttyS010:24:010
23830992419,3cyclictest90-1pr/ttyS010:24:010
23830992419,3cyclictest0-21swapper/009:47:170
23830992419,3cyclictest0-21swapper/009:47:170
23830992419,3cyclictest0-21swapper/009:07:190
23830992419,3cyclictest0-21swapper/009:07:190
23830992418,3cyclictest0-21swapper/011:57:190
23830992418,3cyclictest0-21swapper/011:57:190
23830992414,8cyclictest11551-21sshd12:23:540
23830992414,8cyclictest11551-21sshd12:23:540
23830992412,7cyclictest56-21khungtaskd11:42:090
23830992412,7cyclictest56-21khungtaskd11:42:090
23830992411,4cyclictest181rcu_preempt12:32:130
23830992411,4cyclictest181rcu_preempt12:32:130
23835992314,6cyclictest7934-21chrt07:56:563
23835992314,6cyclictest7017-21chrt09:17:443
23835992314,6cyclictest7017-21chrt09:17:443
23835992314,6cyclictest32499-21chrt07:32:403
23835992314,6cyclictest32499-21chrt07:32:403
23835992314,6cyclictest31629-21chrt10:26:263
23835992314,6cyclictest31629-21chrt10:26:263
23835992314,6cyclictest31629-21chrt10:26:263
23835992314,6cyclictest20812-21chrt08:29:123
23835992314,6cyclictest20812-21chrt08:29:123
23835992314,6cyclictest20102-21chrt09:52:373
23835992314,6cyclictest20102-21chrt09:52:373
23835992314,6cyclictest19710-21chrt11:20:173
23835992314,6cyclictest19710-21chrt11:20:173
23835992314,6cyclictest11565-21chrt12:24:053
23835992314,6cyclictest11565-21chrt12:24:053
23835992314,6cyclictest11565-21chrt12:24:053
23835992313,7cyclictest18224-21chrt09:47:273
23835992313,7cyclictest18224-21chrt09:47:273
23835992313,7cyclictest17070-21chrt12:38:003
23835992313,7cyclictest17070-21chrt12:38:003
23835992313,7cyclictest17070-21chrt12:38:003
23835992313,7cyclictest13616-21chrt12:31:323
23835992313,7cyclictest13616-21chrt12:31:323
23835992313,6cyclictest9675-21chrt12:18:583
23835992313,6cyclictest9675-21chrt12:18:583
23835992313,6cyclictest9675-21chrt12:18:583
23835992313,6cyclictest6612-21chrt10:44:513
23835992313,6cyclictest6612-21chrt10:44:513
23835992313,6cyclictest6612-21chrt10:44:513
23835992313,6cyclictest6612-21chrt10:44:513
23835992313,6cyclictest3503-21chrt09:10:373
23835992313,6cyclictest3503-21chrt09:10:373
23835992313,6cyclictest25939-21chrt10:10:303
23835992313,6cyclictest25939-21chrt10:10:303
23835992313,6cyclictest2174-21chrt11:58:443
23835992313,6cyclictest2174-21chrt11:58:443
23835992313,6cyclictest11543-21chrt08:05:083
23835992313,6cyclictest11543-21chrt08:05:083
23835992313,6cyclictest1060-21chrt10:30:443
23835992310,9cyclictest12197-21chrt10:59:513
23835992310,9cyclictest12197-21chrt10:59:513
23835992310,10cyclictest17777-21chrt11:14:263
23835992310,10cyclictest17777-21chrt11:14:263
23831992318,3cyclictest0-21swapper/108:07:181
23831992318,3cyclictest0-21swapper/108:07:181
23831992317,5cyclictest23046-21kernelversion10:02:111
23831992317,5cyclictest23046-21kernelversion10:02:111
23831992317,4cyclictest0-21swapper/111:07:201
23831992317,4cyclictest0-21swapper/111:07:201
23831992317,4cyclictest0-21swapper/109:06:561
23831992317,4cyclictest0-21swapper/109:06:561
23831992317,3cyclictest0-21swapper/111:12:211
23831992317,3cyclictest0-21swapper/111:12:211
23831992317,3cyclictest0-21swapper/108:57:191
23831992317,3cyclictest0-21swapper/108:57:191
23831992315,6cyclictest3348-21needreboot12:02:141
23831992315,6cyclictest3348-21needreboot12:02:141
23831992315,6cyclictest3348-21needreboot12:02:141
23831992313,8cyclictest142-21systemd-journal09:36:561
23831992313,8cyclictest142-21systemd-journal09:36:561
23831992310,4cyclictest181rcu_preempt08:37:101
23831992310,4cyclictest181rcu_preempt08:37:101
2383099234,16cyclictest171ktimers/010:00:290
2383099234,16cyclictest171ktimers/010:00:290
23830992318,3cyclictest0-21swapper/012:12:090
23830992318,3cyclictest0-21swapper/012:12:090
23830992315,6cyclictest9850-21ls10:52:170
23830992315,6cyclictest9850-21ls10:52:170
2383599229,9cyclictest30774-21taskset11:48:113
2383599229,9cyclictest30774-21taskset11:48:113
2383599229,9cyclictest30774-21taskset11:48:113
23835992217,2cyclictest0-21swapper/308:50:203
23835992217,2cyclictest0-21swapper/308:50:203
23835992213,7cyclictest8445-21chrt10:49:303
23835992213,6cyclictest3964-21chrt12:02:303
23835992213,6cyclictest24595-21chrt08:39:533
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional