You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-17 - 23:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Wed Jun 17, 2026 12:44:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
119021150,2sleep01192-21latency_hist08:42:560
119021150,2sleep01192-21latency_hist08:42:560
138462930,3sleep30-21swapper/307:48:113
138462930,3sleep30-21swapper/307:48:113
138462930,3sleep30-21swapper/307:48:113
181592730,2sleep118162-21unixbench_multi07:58:191
181592730,2sleep118162-21unixbench_multi07:58:191
118582700,2sleep111860-21cat09:08:171
118582700,2sleep111860-21cat09:08:171
293962660,3chrt0-21swapper/309:58:043
293962660,3chrt0-21swapper/309:58:043
293962660,3chrt0-21swapper/309:58:043
186352630,2sleep018634-21wc08:02:560
186352630,2sleep018634-21wc08:02:560
146182580,4sleep20-21swapper/210:43:102
146182580,4sleep20-21swapper/210:43:102
258292560,3sleep20-21swapper/211:13:132
258292560,3sleep20-21swapper/211:13:132
258292560,3sleep20-21swapper/211:13:132
60992550,2sleep30-21swapper/307:28:093
60992550,2sleep30-21swapper/307:28:093
120372520,2chrt0-21swapper/207:43:122
120372520,2chrt0-21swapper/207:43:122
147172510,4sleep114719-21fschecks_time09:18:061
147172510,4sleep114719-21fschecks_time09:18:061
44912500,3sleep20-21swapper/210:18:052
44912500,3sleep20-21swapper/210:18:052
44912500,3sleep20-21swapper/210:18:052
150132500,3sleep20-21swapper/207:53:012
150132500,3sleep20-21swapper/207:53:012
150132500,3sleep20-21swapper/207:53:012
134592500,2sleep013462-21fschecks_time07:48:080
134592500,2sleep013462-21fschecks_time07:48:080
134592500,2sleep013462-21fschecks_time07:48:080
75072490,3sleep17510-21latency_hist11:52:561
75072490,3sleep17510-21latency_hist11:52:561
233362470,2sleep30-21swapper/312:33:103
233362470,2sleep30-21swapper/312:33:103
106292450,3sleep30-21swapper/309:07:593
106292450,3sleep30-21swapper/309:07:593
288732440,3sleep30-21swapper/308:28:163
288732440,3sleep30-21swapper/308:28:163
288732440,3sleep30-21swapper/308:28:163
68322430,4sleep07002-21grep11:48:170
68322430,4sleep07002-21grep11:48:170
32640994031,6cyclictest19935-21grep09:32:560
32640994031,6cyclictest19935-21grep09:32:560
32640994031,6cyclictest19935-21grep09:32:560
206392390,3sleep10-21swapper/110:58:161
206392390,3sleep10-21swapper/110:58:161
322842380,4sleep032635-21cyclictest11:31:120
322842380,4sleep032635-21cyclictest11:31:120
322842380,4sleep032635-21cyclictest11:31:120
107872380,3chrt0-21swapper/209:08:062
231192370,2sleep122660-21/usr/sbin/munin12:33:081
231192370,2sleep122660-21/usr/sbin/munin12:33:081
41772360,9sleep14181-21gltestperf07:23:091
41772360,9sleep14181-21gltestperf07:23:091
25332360,4sleep30-21swapper/311:38:103
25332360,4sleep30-21swapper/311:38:103
25332360,4sleep30-21swapper/311:38:103
32643993527,6cyclictest351rcuc/211:36:002
32643993527,6cyclictest351rcuc/211:36:002
220612350,2sleep20-21swapper/212:28:182
220612350,2sleep20-21swapper/212:28:182
220612350,2sleep20-21swapper/212:28:182
32641993427,4cyclictest12420-21gltestperf10:38:111
32641993427,4cyclictest12420-21gltestperf10:38:111
3264099344,6cyclictest171ktimers/007:39:270
3264099344,6cyclictest171ktimers/007:39:270
32640993426,6cyclictest20573-21threads10:58:160
32640993426,6cyclictest20573-21threads10:58:160
32640993426,6cyclictest12412-21cron09:12:560
32640993426,6cyclictest12412-21cron09:12:560
196732340,3sleep00-21swapper/012:23:140
196732340,3sleep00-21swapper/012:23:140
32641993326,5cyclictest27760-21fschecks_count09:53:081
32640993327,5cyclictest9410-21kernelversion09:03:120
32640993327,5cyclictest9410-21kernelversion09:03:120
32640993326,5cyclictest30554-21kernelversion08:33:120
32640993326,5cyclictest30554-21kernelversion08:33:120
32640993326,5cyclictest30554-21kernelversion08:33:120
32640993324,7cyclictest25115-21needreboot08:18:130
32640993318,2cyclictest181rcu_preempt11:23:110
32640993318,2cyclictest181rcu_preempt11:23:110
72002320,9sleep129-21ksoftirqd/110:23:151
72002320,9sleep129-21ksoftirqd/110:23:151
32641993223,7cyclictest31748-21latency_hist08:37:581
32641993223,7cyclictest31748-21latency_hist08:37:581
32641993223,7cyclictest31748-21latency_hist08:37:581
32640993227,4cyclictest32635-21cyclictest10:03:140
32640993227,4cyclictest32635-21cyclictest10:03:140
32640993227,4cyclictest32635-21cyclictest10:03:140
32640993226,5cyclictest14040-21latency_hist10:43:060
32640993226,5cyclictest14040-21latency_hist10:43:060
32641993124,5cyclictest7310-21gltestperf08:58:081
3264099318,6cyclictest171ktimers/008:05:400
3264099318,6cyclictest171ktimers/008:05:400
32640993125,5cyclictest24382-21kernelversion09:43:090
32640993125,5cyclictest24382-21kernelversion09:43:090
32640993124,5cyclictest19081-21cpuspeed12:23:070
32640993124,5cyclictest10164-21/usr/sbin/munin10:33:070
32640993115,5cyclictest181rcu_preempt08:53:160
32640993115,5cyclictest181rcu_preempt08:53:160
32640993115,5cyclictest181rcu_preempt08:53:160
269932310,3sleep30-21swapper/308:23:133
269932310,3sleep30-21swapper/308:23:133
269932310,3sleep30-21swapper/308:23:133
32641993023,5cyclictest6229-21kernelversion07:28:111
32641993023,5cyclictest6229-21kernelversion07:28:111
32640993025,4cyclictest6071-21fschecks_time11:48:080
32640993025,4cyclictest6071-21fschecks_time11:48:080
32640993025,3cyclictest32635-21cyclictest11:38:040
32640993025,3cyclictest32635-21cyclictest11:38:040
32640993023,5cyclictest26331-21latency_hist12:42:590
32640993023,5cyclictest26331-21latency_hist12:42:590
32640993018,10cyclictest0-21swapper/009:02:560
3264099301,6cyclictest700-21192.168.115.40-manager11:13:390
3264099301,6cyclictest700-21192.168.115.40-manager11:13:390
3264099301,6cyclictest700-21192.168.115.40-manager11:13:390
32641992922,5cyclictest9989-21kernelversion07:38:121
32641992922,5cyclictest9989-21kernelversion07:38:121
32641992921,6cyclictest6702-21fschecks_time10:23:091
32641992921,6cyclictest1587-21fschecks_time08:43:091
32641992921,6cyclictest1587-21fschecks_time08:43:091
3264099299,3cyclictest17779-21needreboot12:18:120
32640992924,4cyclictest32635-21cyclictest10:19:210
32640992924,4cyclictest32635-21cyclictest10:19:210
32640992924,4cyclictest32635-21cyclictest10:19:210
32640992923,5cyclictest21364-21kernelversion12:28:100
32640992923,5cyclictest21364-21kernelversion12:28:100
32640992923,5cyclictest21364-21kernelversion12:28:100
32640992923,5cyclictest10704-21kernelversion10:33:110
32640992923,5cyclictest10704-21kernelversion10:33:110
32640992923,3cyclictest0-21swapper/011:22:560
32640992923,3cyclictest0-21swapper/011:22:560
32640992914,9cyclictest90-1pr/ttyS007:31:340
32640992914,9cyclictest90-1pr/ttyS007:31:340
32640992913,4cyclictest90-1pr/ttyS011:03:100
32640992913,4cyclictest90-1pr/ttyS011:03:100
32641992822,5cyclictest23589-21sshd09:42:121
32641992822,5cyclictest23589-21sshd09:42:121
32641992822,5cyclictest23589-21sshd09:42:121
32641992822,5cyclictest14483-21unixbench_singl07:48:191
32641992822,5cyclictest14483-21unixbench_singl07:48:191
32641992822,5cyclictest14483-21unixbench_singl07:48:191
32641992821,5cyclictest32635-21cyclictest07:43:081
32641992821,5cyclictest32635-21cyclictest07:43:081
32641992821,5cyclictest24976-21timerwakeupswit09:43:161
32641992821,5cyclictest24976-21timerwakeupswit09:43:161
32641992821,5cyclictest21067-21head08:08:111
32641992821,5cyclictest21067-21head08:08:111
32641992821,5cyclictest13284-21latency_hist12:08:051
32641992821,5cyclictest13284-21latency_hist12:08:051
32641992821,5cyclictest13284-21latency_hist12:08:051
32641992820,6cyclictest9480-21sort09:03:121
32641992820,6cyclictest9480-21sort09:03:121
32641992820,6cyclictest23600-21threads08:13:191
32641992820,6cyclictest23600-21threads08:13:191
32641992820,6cyclictest18993-21netstat09:28:141
32641992820,6cyclictest18993-21netstat09:28:141
32641992820,6cyclictest18993-21netstat09:28:141
32641992819,7cyclictest19493-21kernelversion12:23:121
32641992819,7cyclictest19493-21kernelversion12:23:121
32640992822,4cyclictest1346-21avahi-daemon09:33:190
32640992822,4cyclictest1346-21avahi-daemon09:33:190
32640992822,4cyclictest1346-21avahi-daemon09:33:190
32640992821,5cyclictest29697-21sshd08:31:120
32640992821,5cyclictest29697-21sshd08:31:120
32640992821,5cyclictest29697-21sshd08:31:120
32640992821,5cyclictest22352-21sed09:38:100
32640992820,6cyclictest12298-21cut07:43:160
32640992820,6cyclictest12298-21cut07:43:160
32640992820,3cyclictest181rcu_preempt08:11:510
32640992820,3cyclictest181rcu_preempt08:11:510
32640992818,3cyclictest181rcu_preempt09:27:580
32640992818,3cyclictest181rcu_preempt09:27:580
32640992818,3cyclictest181rcu_preempt09:27:580
32640992816,4cyclictest181rcu_preempt08:47:140
32640992816,4cyclictest181rcu_preempt08:47:140
32640992814,3cyclictest181rcu_preempt09:58:190
32640992814,3cyclictest181rcu_preempt09:58:190
32640992812,4cyclictest181rcu_preempt09:54:140
32640992812,4cyclictest181rcu_preempt09:54:140
32640992812,4cyclictest181rcu_preempt09:54:140
32640992811,3cyclictest181rcu_preempt11:53:110
32640992811,3cyclictest181rcu_preempt11:53:110
32641992723,3cyclictest879-21tail11:33:131
32641992723,3cyclictest879-21tail11:33:131
32641992723,2cyclictest0-21swapper/110:48:091
32641992723,2cyclictest0-21swapper/110:48:091
32641992721,5cyclictest6684-21sshd08:56:441
32641992721,5cyclictest6684-21sshd08:56:441
32641992721,5cyclictest6684-21sshd08:56:441
32641992721,4cyclictest5546-21kworker/u9:4-xprtiod09:53:061
32641992721,4cyclictest5546-21kworker/u9:4-xprtiod09:53:061
32640992723,3cyclictest15870-21missed_timers12:13:110
32640992723,3cyclictest15870-21missed_timers12:13:110
32640992723,3cyclictest15870-21missed_timers12:13:110
32640992722,4cyclictest14678-21systemd-journal12:03:120
32640992722,4cyclictest14678-21systemd-journal12:03:120
32640992722,4cyclictest14678-21systemd-journal12:03:120
32640992721,5cyclictest11259-21sendmail12:02:560
32640992721,5cyclictest11259-21sendmail12:02:560
32640992721,5cyclictest11259-21sendmail12:02:560
32640992721,4cyclictest7916-21gltestperf07:33:100
32640992721,4cyclictest7916-21gltestperf07:33:100
32640992720,2cyclictest181rcu_preempt10:28:080
32640992720,2cyclictest181rcu_preempt10:28:080
32640992719,6cyclictest4810-21sort07:23:170
32640992719,6cyclictest4810-21sort07:23:170
32640992717,3cyclictest181rcu_preempt09:43:140
32640992717,3cyclictest181rcu_preempt09:43:140
32640992715,3cyclictest181rcu_preempt08:15:010
32640992715,3cyclictest181rcu_preempt08:15:010
32640992715,2cyclictest181rcu_preempt08:48:100
32640992715,2cyclictest181rcu_preempt08:48:100
32640992713,3cyclictest181rcu_preempt09:16:590
32640992713,3cyclictest181rcu_preempt09:16:590
32640992711,3cyclictest181rcu_preempt11:38:170
32640992711,3cyclictest181rcu_preempt11:38:170
32640992711,3cyclictest181rcu_preempt11:38:170
242832270,3sleep20-21swapper/209:43:082
242832270,3sleep20-21swapper/209:43:082
242832270,3sleep20-21swapper/209:43:082
32641992620,4cyclictest31826-21if_wlan010:03:101
32641992620,4cyclictest31826-21if_wlan010:03:101
32641992620,4cyclictest31826-21if_wlan010:03:101
32640992622,3cyclictest18888-21grep10:53:190
32640992622,3cyclictest18888-21grep10:53:190
32640992620,2cyclictest181rcu_preempt09:48:150
32640992620,2cyclictest181rcu_preempt09:48:150
32640992620,2cyclictest181rcu_preempt09:48:150
32640992619,4cyclictest0-21swapper/010:13:170
32640992619,4cyclictest0-21swapper/010:13:170
32640992619,4cyclictest0-21swapper/010:13:170
32640992614,2cyclictest181rcu_preempt12:08:180
32640992614,2cyclictest181rcu_preempt12:08:180
32640992614,2cyclictest181rcu_preempt12:08:180
32640992614,2cyclictest181rcu_preempt10:09:470
32640992614,2cyclictest181rcu_preempt10:09:470
32640992613,2cyclictest181rcu_preempt09:23:090
32640992612,3cyclictest181rcu_preempt12:33:160
32640992612,3cyclictest181rcu_preempt12:33:160
154432260,2sleep115574-21timerandwakeup09:18:151
154432260,2sleep115574-21timerandwakeup09:18:151
32641992521,2cyclictest0-21swapper/110:33:091
32641992521,2cyclictest0-21swapper/110:33:091
32641992520,3cyclictest0-21swapper/110:52:551
32641992520,3cyclictest0-21swapper/110:52:551
32641992520,3cyclictest0-21swapper/110:52:551
32641992519,3cyclictest0-21swapper/112:28:091
3264099256,7cyclictest16-21ksoftirqd/009:21:300
32640992519,4cyclictest32635-21cyclictest10:28:170
32640992518,2cyclictest181rcu_preempt10:44:570
32640992518,2cyclictest181rcu_preempt10:44:570
32640992517,6cyclictest23936-21ls11:08:120
32640992517,6cyclictest23936-21ls11:08:120
32640992517,6cyclictest23936-21ls11:08:120
32640992513,3cyclictest181rcu_preempt08:23:120
32640992513,3cyclictest181rcu_preempt08:23:120
32640992513,3cyclictest181rcu_preempt08:23:120
32641992419,4cyclictest4471-21kernelversion11:43:101
32641992419,4cyclictest4471-21kernelversion11:43:101
32641992419,4cyclictest21067-21systemctl09:33:201
32641992419,4cyclictest21067-21systemctl09:33:201
32641992419,4cyclictest21067-21systemctl09:33:201
32641992419,3cyclictest0-21swapper/112:22:591
32641992419,3cyclictest0-21swapper/112:22:591
32641992419,3cyclictest0-21swapper/112:08:151
32641992419,3cyclictest0-21swapper/112:08:151
32641992419,3cyclictest0-21swapper/112:08:151
32641992419,3cyclictest0-21swapper/111:28:181
32641992419,3cyclictest0-21swapper/111:28:181
32641992419,3cyclictest0-21swapper/111:28:181
32641992419,2cyclictest0-21swapper/110:08:181
32641992419,2cyclictest0-21swapper/110:08:181
3264199241,8cyclictest25680-21sed08:18:191
32641992418,3cyclictest0-21swapper/110:58:071
32641992418,3cyclictest0-21swapper/110:58:071
32641992416,6cyclictest10292-21needreboot11:58:141
32641992416,6cyclictest10292-21needreboot11:58:141
32641992416,6cyclictest10292-21needreboot11:58:141
32640992418,4cyclictest0-21swapper/010:48:170
32640992418,4cyclictest0-21swapper/010:48:170
32640992418,4cyclictest0-21swapper/010:48:170
32640992417,5cyclictest15976-21rt-features07:53:160
32640992417,5cyclictest15976-21rt-features07:53:160
32640992417,5cyclictest15976-21rt-features07:53:160
32641992319,3cyclictest32635-21cyclictest11:53:201
32641992319,3cyclictest32635-21cyclictest11:53:201
32641992319,3cyclictest32635-21cyclictest10:13:121
32641992319,3cyclictest32635-21cyclictest10:13:121
32641992319,3cyclictest32635-21cyclictest10:13:121
32641992318,3cyclictest15254-21sort07:53:071
32641992318,3cyclictest15254-21sort07:53:071
32641992318,3cyclictest15254-21sort07:53:071
32641992318,3cyclictest0-21swapper/111:38:111
32641992318,3cyclictest0-21swapper/111:38:111
32641992318,3cyclictest0-21swapper/111:38:111
32641992317,4cyclictest0-21swapper/111:28:071
32641992317,3cyclictest0-21swapper/111:08:091
32641992317,3cyclictest0-21swapper/111:08:091
32641992317,3cyclictest0-21swapper/111:08:091
32641992218,2cyclictest0-21swapper/111:23:121
32641992217,3cyclictest0-21swapper/108:48:221
32641992217,3cyclictest0-21swapper/108:48:221
32641992217,3cyclictest0-21swapper/108:28:201
32641992217,3cyclictest0-21swapper/108:28:201
32641992217,3cyclictest0-21swapper/108:28:201
32641992217,3cyclictest0-21swapper/108:03:081
32641992217,3cyclictest0-21swapper/108:03:081
32641992217,2cyclictest0-21swapper/109:23:161
32641992217,2cyclictest0-21swapper/109:23:161
32641992217,2cyclictest0-21swapper/109:23:161
32641992216,4cyclictest0-21swapper/109:53:121
32641992216,4cyclictest0-21swapper/109:53:121
32641992216,4cyclictest0-21swapper/109:53:121
32641992214,6cyclictest27201-21grep08:23:171
32641992214,6cyclictest27201-21grep08:23:171
32641992214,6cyclictest27201-21grep08:23:171
3264199221,3cyclictest281242timerwakeupswit11:18:171
3264199221,3cyclictest281242timerwakeupswit11:18:171
32643992112,6cyclictest351rcuc/207:32:482
32643992112,6cyclictest351rcuc/207:32:482
3264199219,9cyclictest7848-21fschecks_time07:33:091
3264199219,9cyclictest7848-21fschecks_time07:33:091
32641992116,3cyclictest0-21swapper/111:03:101
32641992116,3cyclictest0-21swapper/111:03:101
32641992115,5cyclictest30130-21needreboot09:58:121
32641992115,5cyclictest30130-21needreboot09:58:121
32641992115,3cyclictest0-21swapper/112:13:151
32641992115,3cyclictest0-21swapper/112:13:151
32641992115,3cyclictest0-21swapper/112:13:151
32641992113,6cyclictest32484-21ls08:38:101
32641992113,6cyclictest32484-21ls08:38:101
32643992013,5cyclictest351rcuc/208:11:512
32643992013,5cyclictest351rcuc/208:11:512
32641992014,4cyclictest0-21swapper/110:18:081
32641992014,4cyclictest0-21swapper/110:18:081
32643991913,4cyclictest351rcuc/212:35:152
32643991913,4cyclictest351rcuc/212:35:152
32643991913,4cyclictest351rcuc/211:29:022
32643991913,4cyclictest351rcuc/211:29:022
32643991913,4cyclictest351rcuc/211:29:022
32641991913,4cyclictest0-21swapper/112:33:101
32641991913,4cyclictest0-21swapper/112:33:101
32641991913,4cyclictest0-21swapper/111:13:201
32641991913,4cyclictest0-21swapper/111:13:201
32641991913,4cyclictest0-21swapper/111:13:201
32641991913,3cyclictest0-21swapper/112:43:041
32641991913,3cyclictest0-21swapper/112:43:041
32641991912,4cyclictest0-21swapper/110:28:171
32641991912,4cyclictest0-21swapper/110:28:171
284462190,4sleep20-21swapper/208:28:102
284462190,4sleep20-21swapper/208:28:102
284462190,4sleep20-21swapper/208:28:102
3264599181,11cyclictest20969-21chrt12:28:073
3264599181,11cyclictest20969-21chrt12:28:073
32643991813,4cyclictest351rcuc/210:57:382
32643991813,4cyclictest351rcuc/210:57:382
32643991813,3cyclictest351rcuc/209:08:592
32643991813,3cyclictest351rcuc/209:08:592
32643991813,3cyclictest351rcuc/208:34:262
32643991813,3cyclictest351rcuc/208:34:262
32643991813,3cyclictest351rcuc/208:34:262
32643991812,4cyclictest351rcuc/209:04:202
32643991810,6cyclictest351rcuc/208:03:392
32643991810,6cyclictest351rcuc/208:03:392
3264599172,9cyclictest0-21swapper/310:54:123
3264599172,9cyclictest0-21swapper/310:54:123
3264599172,11cyclictest0-21swapper/312:18:183
3264599172,11cyclictest0-21swapper/312:18:183
3264599171,12cyclictest8426-21chrt09:00:113
3264599171,12cyclictest29116-21taskset11:23:073
3264599171,12cyclictest29116-21taskset11:23:073
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional