You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-14 - 11:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Sun Jun 14, 2026 00:44:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1769921140,1sleep00-21swapper/000:28:100
1769921140,1sleep00-21swapper/000:28:100
1742621020,2chrt17428-21sed20:08:091
1742621020,2chrt17428-21sed20:08:091
99432840,3chrt0-21swapper/319:48:103
99432840,3chrt0-21swapper/319:48:103
198442730,2sleep00-21swapper/023:08:060
198442730,2sleep00-21swapper/023:08:060
198442730,2sleep00-21swapper/023:08:060
265352720,2chrt26534-21sendmail_mailtr21:58:161
265352720,2chrt26534-21sendmail_mailtr21:58:161
224092650,3chrt0-21swapper/300:42:533
224092650,3chrt0-21swapper/300:42:533
46742630,2sleep00-21swapper/021:02:530
46742630,2sleep00-21swapper/021:02:530
154512630,2chrt0-21swapper/300:23:073
154512630,2chrt0-21swapper/300:23:073
108472630,2sleep30-21swapper/322:43:143
108472630,2sleep30-21swapper/322:43:143
108472630,2sleep30-21swapper/322:43:143
228442620,2sleep022944-21latency20:23:090
228442620,2sleep022944-21latency20:23:090
228442620,2sleep022944-21latency20:23:090
216722600,2chrt0-21swapper/021:47:540
216722600,2chrt0-21swapper/021:47:540
216722600,2chrt0-21swapper/021:47:540
318462550,2sleep031849-21grep23:38:170
318462550,2sleep031849-21grep23:38:170
318462550,2sleep031849-21grep23:38:170
263082550,3chrt0-21swapper/020:33:050
263082550,3chrt0-21swapper/020:33:050
310512540,3sleep20-21swapper/220:43:162
310512540,3sleep20-21swapper/220:43:162
183832540,4sleep00-21swapper/021:38:050
172042530,3sleep30-21swapper/321:33:143
172042530,3sleep30-21swapper/321:33:143
75492520,5sleep30-21swapper/300:03:023
75492520,5sleep30-21swapper/300:03:023
75492520,5sleep30-21swapper/300:03:023
5692510,3sleep30-21swapper/322:18:003
5692510,3sleep30-21swapper/322:18:003
31412480,1sleep3431rcuc/320:58:043
31412480,1sleep3431rcuc/320:58:043
275662470,2chrt0-21swapper/322:03:053
275662470,2chrt0-21swapper/322:03:053
262312470,7sleep026234-21sort21:58:110
262312470,7sleep026234-21sort21:58:110
23222430,1sleep02356-21latency_hist22:22:530
23222430,1sleep02356-21latency_hist22:22:530
23222430,1sleep02356-21latency_hist22:22:530
163912430,3sleep20-21swapper/221:33:052
163912430,3sleep20-21swapper/221:33:052
292162410,4sleep028759-21/usr/sbin/munin23:33:070
292162410,4sleep028759-21/usr/sbin/munin23:33:070
292162410,4sleep028759-21/usr/sbin/munin23:33:070
187772410,2sleep00-21swapper/021:38:090
187772410,2sleep00-21swapper/021:38:090
187772410,2sleep00-21swapper/021:38:090
187772410,2sleep00-21swapper/021:38:090
2885499381,28cyclictest7539-21chrt00:03:002
2885499381,28cyclictest7539-21chrt00:03:002
2885499381,28cyclictest7539-21chrt00:03:002
2885299381,29cyclictest7738-21entropy00:03:061
2885299381,29cyclictest7738-21entropy00:03:061
2885299371,29cyclictest8222-21memory00:03:101
2885299371,29cyclictest8222-21memory00:03:101
2885299371,29cyclictest8222-21memory00:03:101
50252360,2sleep20-21swapper/222:28:092
50252360,2sleep20-21swapper/222:28:092
28851993631,3cyclictest12257-21cron21:22:530
28851993631,3cyclictest12257-21cron21:22:530
28851993629,5cyclictest5293-21cron19:37:530
28851993629,5cyclictest5293-21cron19:37:530
86132350,2sleep20-21swapper/200:03:162
86132350,2sleep20-21swapper/200:03:162
86132350,2sleep20-21swapper/200:03:162
313482350,7sleep031325-21cat22:13:030
313482350,7sleep031325-21cat22:13:030
28851993529,4cyclictest0-21swapper/000:03:380
28851993529,4cyclictest0-21swapper/000:03:380
28851993529,4cyclictest0-21swapper/000:03:380
158442350,3sleep20-21swapper/220:03:122
158442350,3sleep20-21swapper/220:03:122
28851993429,3cyclictest0-21swapper/000:02:590
28851993429,3cyclictest0-21swapper/000:02:590
28851993429,3cyclictest0-21swapper/000:02:590
28851993428,4cyclictest572-21cups-browsed22:58:130
28851993428,4cyclictest572-21cups-browsed22:58:130
28851993428,4cyclictest1346-21avahi-daemon22:33:150
28851993428,4cyclictest1346-21avahi-daemon22:33:150
28852993328,3cyclictest27252-21gltestperf23:28:061
28852993328,3cyclictest27252-21gltestperf23:28:061
28852993328,3cyclictest27252-21gltestperf23:28:061
28852993326,5cyclictest17349-21latency_hist23:02:531
28852993326,5cyclictest17349-21latency_hist23:02:531
28852993324,7cyclictest30438-21cron23:37:531
28852993324,7cyclictest30438-21cron23:37:531
28852993324,7cyclictest30438-21cron23:37:531
28851993326,5cyclictest2487-21kernelversion23:48:090
28851993326,5cyclictest2487-21kernelversion23:48:090
28852993225,5cyclictest14199-21gltestperf22:53:081
28852993225,5cyclictest14199-21gltestperf22:53:081
28852993225,5cyclictest14199-21gltestperf22:53:081
28852993223,5cyclictest281ktimers/100:12:391
28852993223,5cyclictest281ktimers/100:12:391
28851993227,3cyclictest14691-21latency_hist20:02:530
28851993227,3cyclictest14691-21latency_hist20:02:530
28851993225,5cyclictest9659-21fschecks_count00:08:070
28851993225,5cyclictest9659-21fschecks_count00:08:070
28851993225,5cyclictest23007-21latency_hist23:17:550
28851993225,5cyclictest23007-21latency_hist23:17:550
129642320,2sleep10-21swapper/121:23:071
28852993127,3cyclictest14914-21cron00:22:521
28852993127,3cyclictest14914-21cron00:22:521
28852993124,5cyclictest5692-21ntp_offset21:03:121
28852993124,5cyclictest5692-21ntp_offset21:03:121
28852993110,19cyclictest281ktimers/120:58:341
28852993110,19cyclictest281ktimers/120:58:341
28851993126,3cyclictest25902-21latency_hist20:32:530
28851993126,3cyclictest25902-21latency_hist20:32:530
28851993126,3cyclictest24834-21idleruntime-cro23:22:530
28851993124,6cyclictest15659-21kernelversion00:23:090
28851993124,6cyclictest15659-21kernelversion00:23:090
114902310,2sleep111515-21/usr/sbin/munin00:13:091
114902310,2sleep111515-21/usr/sbin/munin00:13:091
2885299306,4cyclictest271rcuc/120:44:491
2885299306,4cyclictest271rcuc/120:44:491
28852993026,3cyclictest22386-21cron00:42:531
28852993026,3cyclictest22386-21cron00:42:531
28852993021,7cyclictest6220-21latency_hist22:32:591
28852993021,7cyclictest6220-21latency_hist22:32:591
28852993021,7cyclictest10811-21fschecks_time21:18:071
2885199309,4cyclictest171ktimers/020:11:330
2885199309,4cyclictest171ktimers/020:11:330
2885199307,7cyclictest201rcuc/023:23:080
2885199307,7cyclictest201rcuc/023:23:080
2885199307,7cyclictest201rcuc/023:23:080
28851993024,5cyclictest4863-21threads23:53:150
28851993024,5cyclictest4863-21threads23:53:150
28851993024,5cyclictest4863-21threads23:53:150
28851993024,4cyclictest11412-21fschecks_time19:53:070
28851993024,4cyclictest11412-21fschecks_time19:53:070
28851993023,5cyclictest486-21latency_hist22:17:560
28851993023,5cyclictest486-21latency_hist22:17:560
28851993022,6cyclictest2177-21cut23:48:060
28851993011,6cyclictest201rcuc/020:54:040
28851993011,6cyclictest201rcuc/020:54:040
28851993011,6cyclictest201rcuc/020:54:040
28851993010,4cyclictest201rcuc/021:06:350
28851993010,4cyclictest201rcuc/021:06:350
147882300,4sleep20-21swapper/221:28:062
66132290,3sleep20-21swapper/222:33:072
66132290,3sleep20-21swapper/222:33:072
28852992921,6cyclictest625-21ntp_offset19:23:111
28852992921,6cyclictest625-21ntp_offset19:23:111
28852992921,6cyclictest625-21ntp_offset19:23:111
28852992920,7cyclictest8855-21fschecks_count21:13:061
28852992920,7cyclictest8855-21fschecks_count21:13:061
28852992920,7cyclictest8855-21fschecks_count21:13:061
28852992914,3cyclictest90-1pr/ttyS023:13:121
28852992914,3cyclictest90-1pr/ttyS023:13:121
28851992923,3cyclictest0-21swapper/021:08:090
28851992923,3cyclictest0-21swapper/021:08:090
28851992922,5cyclictest25284-21sshd21:56:230
28851992922,5cyclictest25284-21sshd21:56:230
28851992922,5cyclictest25284-21sshd21:56:230
28851992922,5cyclictest11610-21sshd22:46:230
28851992922,5cyclictest11610-21sshd22:46:230
28851992922,5cyclictest11610-21sshd22:46:230
28851992915,4cyclictest90-1pr/ttyS023:46:340
28851992915,4cyclictest90-1pr/ttyS023:46:340
28852992823,3cyclictest7180-21gltestperf21:08:081
28852992823,3cyclictest7180-21gltestperf21:08:081
28852992823,3cyclictest24104-21gltestperf21:53:071
28852992823,3cyclictest24104-21gltestperf21:53:071
28852992823,3cyclictest24104-21gltestperf21:53:071
28852992823,3cyclictest0-21swapper/120:22:521
28852992823,3cyclictest0-21swapper/120:22:521
28852992821,5cyclictest16978-21tr21:33:111
28852992821,5cyclictest16978-21tr21:33:111
2885199286,5cyclictest201rcuc/022:54:080
2885199286,5cyclictest201rcuc/022:54:080
2885199286,5cyclictest201rcuc/022:54:080
28851992823,3cyclictest28850-21cyclictest21:33:150
28851992822,4cyclictest28850-21cyclictest19:47:310
28851992822,4cyclictest28850-21cyclictest19:47:310
28851992822,4cyclictest201rcuc/021:26:500
28851992822,4cyclictest201rcuc/021:26:500
28851992822,4cyclictest201rcuc/021:26:500
28851992822,4cyclictest13655-21latency_hist22:52:550
28851992822,4cyclictest13655-21latency_hist22:52:550
28851992822,4cyclictest13655-21latency_hist22:52:550
28851992821,5cyclictest898-21sshd20:52:240
28851992821,5cyclictest898-21sshd20:52:240
28851992821,5cyclictest20175-21sshd00:33:240
28851992821,5cyclictest20175-21sshd00:33:240
28851992821,4cyclictest572-21cups-browsed22:38:130
28851992821,4cyclictest572-21cups-browsed22:38:130
28851992821,4cyclictest201rcuc/020:43:540
28851992821,4cyclictest201rcuc/020:43:540
28851992820,6cyclictest13431-21fschecks_time00:18:050
28851992810,15cyclictest171ktimers/019:41:320
17972280,3sleep30-21swapper/323:47:533
17972280,3sleep30-21swapper/323:47:533
17972280,3sleep30-21swapper/323:47:533
28852992721,4cyclictest3240-21unixbench_singl23:48:161
28852992721,4cyclictest3240-21unixbench_singl23:48:161
2885199274,3cyclictest201rcuc/000:40:100
2885199274,3cyclictest201rcuc/000:40:100
28851992720,5cyclictest586-21netstat19:23:100
28851992720,5cyclictest586-21netstat19:23:100
28851992720,5cyclictest586-21netstat19:23:100
28851992711,14cyclictest171ktimers/020:42:530
28851992711,14cyclictest171ktimers/020:42:530
28852992622,3cyclictest60-21kcompactd023:11:081
28852992622,3cyclictest60-21kcompactd023:11:081
28852992622,3cyclictest60-21kcompactd023:11:081
28852992621,3cyclictest2935-21gltestperf22:23:071
28852992621,3cyclictest2935-21gltestperf22:23:071
28852992620,5cyclictest24995-21ntp_offset20:28:121
28852992620,5cyclictest24995-21ntp_offset20:28:121
28852992620,5cyclictest19138-21kernelversion20:13:071
28852992620,5cyclictest19138-21kernelversion20:13:071
28852992620,5cyclictest19138-21kernelversion20:13:071
28852992620,4cyclictest15650-21ntp_kernel_pll_20:03:091
28852992620,4cyclictest15650-21ntp_kernel_pll_20:03:091
28852992620,4cyclictest0-21swapper/123:58:071
28852992615,3cyclictest181rcu_preempt22:38:041
28852992615,3cyclictest181rcu_preempt22:38:041
2885199268,16cyclictest201rcuc/020:17:350
2885199268,16cyclictest201rcuc/020:17:350
2885199268,16cyclictest201rcuc/020:17:350
28851992620,4cyclictest617-21lldpd22:32:570
28851992620,4cyclictest617-21lldpd22:32:570
28851992620,3cyclictest171ktimers/020:22:520
28851992620,3cyclictest171ktimers/020:22:520
28851992619,5cyclictest3292-21ntp_kernel_pll_22:23:110
28851992619,5cyclictest3292-21ntp_kernel_pll_22:23:110
28851992616,4cyclictest181rcu_preempt19:48:070
28851992616,4cyclictest181rcu_preempt19:48:070
28851992614,4cyclictest181rcu_preempt20:04:300
28851992614,4cyclictest181rcu_preempt20:04:300
28851992611,5cyclictest90-1pr/ttyS022:06:240
28851992611,5cyclictest90-1pr/ttyS022:06:240
28851992611,5cyclictest90-1pr/ttyS022:06:240
2885299256,4cyclictest9698-21if_err_eth019:48:071
2885299256,4cyclictest9698-21if_err_eth019:48:071
28852992521,2cyclictest0-21swapper/120:37:521
28852992521,2cyclictest0-21swapper/120:37:521
28852992520,3cyclictest32308-21latency_hist23:42:531
28852992520,3cyclictest32308-21latency_hist23:42:531
28852992520,3cyclictest32308-21latency_hist23:42:531
28852992520,3cyclictest22811-21switchtime21:48:141
28852992520,3cyclictest22811-21switchtime21:48:141
28852992520,3cyclictest0-21swapper/123:27:521
28852992520,3cyclictest0-21swapper/123:27:521
28852992520,3cyclictest0-21swapper/123:27:521
28852992520,3cyclictest0-21swapper/122:18:091
28852992520,3cyclictest0-21swapper/122:18:091
28852992520,3cyclictest0-21swapper/122:18:091
28852992519,4cyclictest90-1pr/ttyS023:20:311
28852992519,3cyclictest0-21swapper/119:33:161
28852992519,3cyclictest0-21swapper/119:33:161
28852992518,5cyclictest30236-21rt-features22:08:141
28852992518,5cyclictest30236-21rt-features22:08:141
28852992514,9cyclictest14678-21systemd-journal22:45:221
28852992514,9cyclictest14678-21systemd-journal22:45:221
28852992514,9cyclictest14678-21systemd-journal22:45:221
28851992520,4cyclictest60-21kcompactd021:49:360
28851992520,4cyclictest60-21kcompactd021:49:360
28851992520,3cyclictest0-21swapper/023:33:040
28851992520,3cyclictest0-21swapper/023:33:040
28851992520,3cyclictest0-21swapper/023:33:040
28851992519,4cyclictest19669-21fschecks_time23:08:050
28851992512,7cyclictest90-1pr/ttyS000:20:280
28851992512,7cyclictest90-1pr/ttyS000:20:280
28851992512,3cyclictest181rcu_preempt19:28:100
28851992512,3cyclictest181rcu_preempt19:28:100
28851992512,3cyclictest181rcu_preempt19:28:100
98362240,3sleep20-21swapper/219:48:092
98362240,3sleep20-21swapper/219:48:092
2885299248,3cyclictest181rcu_preempt00:33:051
28852992420,2cyclictest0-21swapper/123:53:141
28852992420,2cyclictest0-21swapper/123:53:141
28852992420,2cyclictest0-21swapper/123:53:141
28852992418,4cyclictest90-1pr/ttyS022:40:021
28852992418,4cyclictest90-1pr/ttyS022:40:021
28852992418,4cyclictest2427-21needreboot19:28:081
28852992418,4cyclictest2427-21needreboot19:28:081
28852992418,4cyclictest2427-21needreboot19:28:081
28852992417,5cyclictest90-1pr/ttyS022:14:171
28852992417,5cyclictest90-1pr/ttyS022:14:171
28852992417,5cyclictest547-21rs:main1
28852992417,5cyclictest547-21rs:main1
28852992417,4cyclictest0-21swapper/119:43:071
28852992417,4cyclictest0-21swapper/119:43:071
28851992419,3cyclictest90-1pr/ttyS023:05:070
28851992419,3cyclictest0-21swapper/021:13:090
28851992419,3cyclictest0-21swapper/021:13:090
28851992419,3cyclictest0-21swapper/021:13:090
28851992419,3cyclictest0-21swapper/000:13:200
28851992418,4cyclictest16032-21idleruntime-cro21:32:530
28851992418,4cyclictest16032-21idleruntime-cro21:32:530
2885499231,12cyclictest28795-21taskset22:03:202
2885499231,12cyclictest28795-21taskset22:03:202
2885499231,12cyclictest28795-21taskset22:03:202
28852992318,4cyclictest29133-21unixbench-2d20:38:151
28852992318,4cyclictest29133-21unixbench-2d20:38:151
28852992318,3cyclictest0-21swapper/120:52:531
28852992318,3cyclictest0-21swapper/120:52:531
28852992317,4cyclictest23482-21timerwakeupswit20:23:161
28852992317,4cyclictest23482-21timerwakeupswit20:23:161
28852992317,4cyclictest23482-21timerwakeupswit20:23:161
28852992316,4cyclictest0-21swapper/119:38:071
2885299231,12cyclictest19333-21latency_hist23:07:581
2885299231,12cyclictest19333-21latency_hist23:07:581
2885299231,12cyclictest1165-21latency_hist20:53:041
2885299231,12cyclictest1165-21latency_hist20:53:041
2885299231,12cyclictest1165-21latency_hist20:53:041
28852992310,4cyclictest181rcu_preempt21:28:141
28852992310,4cyclictest181rcu_preempt21:28:141
227662230,3sleep30-21swapper/321:48:133
227662230,3sleep30-21swapper/321:48:133
2885299225,4cyclictest135012sleep122:51:431
2885299225,4cyclictest135012sleep122:51:431
2885299225,4cyclictest135012sleep122:51:431
2885299222,10cyclictest20192-21grep21:43:051
28852992218,3cyclictest845-21ntp_kernel_pll_23:43:111
28852992218,3cyclictest845-21ntp_kernel_pll_23:43:111
28852992218,3cyclictest845-21ntp_kernel_pll_23:43:111
28852992217,3cyclictest0-21swapper/122:03:181
28852992217,3cyclictest0-21swapper/122:03:181
28852992217,3cyclictest0-21swapper/122:03:181
2885299221,6cyclictest143202sleep119:58:211
2885299221,6cyclictest143202sleep119:58:211
28852992216,4cyclictest0-21swapper/100:33:161
28852992216,4cyclictest0-21swapper/100:33:161
28852992215,5cyclictest13199-21runrttasks21:23:101
28852992215,5cyclictest13199-21runrttasks21:23:101
28852992215,5cyclictest13199-21runrttasks21:23:101
280962220,2sleep30-21swapper/323:28:153
280962220,2sleep30-21swapper/323:28:153
280962220,2sleep30-21swapper/323:28:153
28854992115,3cyclictest194512sleep220:13:142
28854992115,3cyclictest194512sleep220:13:142
28854992115,3cyclictest194512sleep220:13:142
2885299216,4cyclictest194572chrt21:39:111
2885299216,4cyclictest194572chrt21:39:111
2885299216,4cyclictest194572chrt21:39:111
2885299212,9cyclictest17269-21/usr/sbin/munin00:28:061
28852992116,3cyclictest0-21swapper/121:43:061
28852992116,3cyclictest0-21swapper/121:43:061
28852992116,3cyclictest0-21swapper/121:43:061
28852992115,3cyclictest0-21swapper/119:53:081
28852992115,3cyclictest0-21swapper/119:53:081
231792210,3chrt0-21swapper/220:23:142
231792210,3chrt0-21swapper/220:23:142
231792210,3chrt0-21swapper/220:23:142
58452200,2chrt0-21swapper/323:58:063
2885599206,8cyclictest0-21swapper/300:03:083
2885599206,8cyclictest0-21swapper/300:03:083
2885599206,8cyclictest0-21swapper/300:03:083
2885599193,9cyclictest0-21swapper/321:54:233
2885599193,9cyclictest0-21swapper/321:54:233
2885599193,9cyclictest0-21swapper/321:54:233
2885599193,9cyclictest0-21swapper/300:19:233
2885599193,9cyclictest0-21swapper/300:19:233
94912180,3sleep20-21swapper/222:38:172
94912180,3sleep20-21swapper/222:38:172
2885599183,9cyclictest0-21swapper/319:53:233
2885599183,9cyclictest0-21swapper/319:53:233
28855991812,3cyclictest258922sleep320:32:533
28855991812,3cyclictest258922sleep320:32:533
2885599181,12cyclictest8864-21taskset22:38:103
2885599181,12cyclictest8864-21taskset22:38:103
2885599181,12cyclictest14369-21taskset22:53:103
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional