You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-21 - 23:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Sun Jun 21, 2026 12:44:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
385621190,3chrt0-21swapper/208:37:592
385621190,3chrt0-21swapper/208:37:592
385621190,3chrt0-21swapper/208:37:592
1584821080,3sleep20-21swapper/209:08:152
1584821080,3sleep20-21swapper/209:08:152
129832900,8sleep1281ktimers/111:53:161
129832900,8sleep1281ktimers/111:53:161
313132840,3sleep10-21swapper/111:16:501
313132840,3sleep10-21swapper/111:16:501
262152830,3sleep20-21swapper/212:28:182
262152830,3sleep20-21swapper/212:28:182
262152830,3sleep20-21swapper/212:28:182
147462730,3chrt0-21swapper/109:03:231
147462730,3chrt0-21swapper/109:03:231
72092710,4sleep30-21swapper/311:38:143
72092710,4sleep30-21swapper/311:38:143
27312680,2chrt2735-21latency08:33:150
27312680,2chrt2735-21latency08:33:150
27312680,2chrt2735-21latency08:33:150
300822630,2sleep10-21swapper/109:47:211
300822630,2sleep10-21swapper/109:47:211
300822630,2sleep10-21swapper/109:47:211
152552620,2chrt0-21swapper/311:58:203
152552620,2chrt0-21swapper/311:58:203
177522610,3sleep30-21swapper/307:48:083
177522610,3sleep30-21swapper/307:48:083
11722600,3chrt0-21swapper/311:23:113
227262590,1sleep122743-21latency_hist09:28:041
227262590,1sleep122743-21latency_hist09:28:041
227262590,1sleep122743-21latency_hist09:28:041
227262590,1sleep122743-21latency_hist09:28:041
85652580,3sleep20-21swapper/211:43:082
85652580,3sleep20-21swapper/211:43:082
234802560,3chrt0-21swapper/308:03:113
234802560,3chrt0-21swapper/308:03:113
251142500,4sleep124677-21/usr/sbin/munin09:33:141
251142500,4sleep124677-21/usr/sbin/munin09:33:141
251142500,4sleep124677-21/usr/sbin/munin09:33:141
251142500,4sleep124677-21/usr/sbin/munin09:33:141
180142470,4sleep20-21swapper/209:13:172
180142470,4sleep20-21swapper/209:13:172
291582460,2sleep20-21swapper/209:43:162
291582460,2sleep20-21swapper/209:43:162
291582460,2sleep20-21swapper/209:43:162
138422460,1sleep2351rcuc/210:28:182
138422460,1sleep2351rcuc/210:28:182
138422460,1sleep2351rcuc/210:28:182
190852440,3sleep119086-21vmstat07:48:221
190852440,3sleep119086-21vmstat07:48:221
190852440,3sleep119086-21vmstat07:48:221
47712430,1sleep04890-21df_inode11:33:100
47712430,1sleep04890-21df_inode11:33:100
321842410,2sleep00-21swapper/009:53:120
321842410,2sleep00-21swapper/009:53:120
321842410,2sleep00-21swapper/009:53:120
4849993921,16cyclictest0-21swapper/108:03:211
4849993921,16cyclictest0-21swapper/108:03:211
4849993914,2cyclictest0-21swapper/108:54:211
4849993914,2cyclictest0-21swapper/108:54:211
230292390,3sleep023031-21ntp_kernel_pll_10:53:170
230292390,3sleep023031-21ntp_kernel_pll_10:53:170
230292390,3sleep023031-21ntp_kernel_pll_10:53:170
4843993825,11cyclictest14182-21/usr/sbin/munin11:58:120
484999374,12cyclictest1648-21ntpd10:13:191
484999374,12cyclictest1648-21ntpd10:13:191
484999371,12cyclictest9381-21ntp_states11:43:161
484999371,12cyclictest9381-21ntp_states11:43:161
4849993627,7cyclictest25768-21threads09:33:221
4849993627,7cyclictest25768-21threads09:33:221
4849993620,13cyclictest0-21swapper/110:53:151
4849993620,13cyclictest0-21swapper/110:53:151
4849993620,13cyclictest0-21swapper/110:53:151
4849993617,17cyclictest0-21swapper/111:41:481
4849993617,17cyclictest0-21swapper/111:41:481
4849993613,3cyclictest0-21swapper/111:08:231
4849993613,3cyclictest0-21swapper/111:08:231
4849993613,3cyclictest0-21swapper/111:08:231
4849993613,3cyclictest0-21swapper/107:38:171
4849993613,3cyclictest0-21swapper/107:38:171
4849993612,3cyclictest0-21swapper/111:27:551
4849993612,3cyclictest0-21swapper/111:27:551
4849993612,3cyclictest0-21swapper/111:27:551
484999361,13cyclictest29213-21taskset12:38:101
484999361,13cyclictest29213-21taskset12:38:101
61712350,2sleep05971-21/usr/sbin/munin08:43:120
61712350,2sleep05971-21/usr/sbin/munin08:43:120
4849993513,2cyclictest0-21swapper/107:43:121
4849993513,2cyclictest0-21swapper/107:43:121
4849993510,4cyclictest0-21swapper/109:58:111
4849993510,4cyclictest0-21swapper/109:58:111
4849993414,18cyclictest0-21swapper/110:23:111
4849993414,18cyclictest0-21swapper/110:23:111
4849993414,18cyclictest0-21swapper/110:23:111
484999341,12cyclictest16987-21rt-features12:03:191
484999341,12cyclictest16987-21rt-features12:03:191
484999341,12cyclictest16987-21rt-features12:03:191
4843993428,4cyclictest4840-21cyclictest08:23:000
4843993428,4cyclictest4840-21cyclictest08:23:000
4843993427,5cyclictest19084-21vmstat07:48:230
4843993427,5cyclictest19084-21vmstat07:48:230
4843993427,5cyclictest19084-21vmstat07:48:230
4843993427,5cyclictest12136-21sh11:53:000
4843993427,5cyclictest12136-21sh11:53:000
4843993427,5cyclictest12136-21sh11:53:000
178122340,3sleep117815-21sort10:38:221
178122340,3sleep117815-21sort10:38:221
4849993312,2cyclictest0-21swapper/108:23:231
4849993312,2cyclictest0-21swapper/108:23:231
4843993326,5cyclictest28342-21threads12:33:220
4843993326,5cyclictest28342-21threads12:33:220
66552320,1sleep16656-21latency_hist11:38:081
66552320,1sleep16656-21latency_hist11:38:081
66552320,1sleep16656-21latency_hist11:38:081
4843993217,4cyclictest181rcu_preempt11:03:230
4843993217,4cyclictest181rcu_preempt11:03:230
4843993217,4cyclictest181rcu_preempt11:03:230
4843993215,4cyclictest181rcu_preempt11:24:520
4843993215,4cyclictest181rcu_preempt11:24:520
4843993215,4cyclictest181rcu_preempt11:24:520
4843993211,3cyclictest181rcu_preempt12:23:430
4843993211,3cyclictest181rcu_preempt12:23:430
4843993211,3cyclictest181rcu_preempt12:23:430
4856993116,11cyclictest18901-21chrt10:43:142
4856993116,11cyclictest18901-21chrt10:43:142
4856993116,11cyclictest18901-21chrt10:43:142
4849993125,5cyclictest9096-21unixbench_singl08:48:221
4849993125,5cyclictest9096-21unixbench_singl08:48:221
4849993125,5cyclictest9096-21unixbench_singl08:48:221
4849993124,5cyclictest19632-21kernelversion09:18:131
4849993124,5cyclictest19632-21kernelversion09:18:131
4849993124,5cyclictest19632-21kernelversion09:18:131
4843993123,6cyclictest415-21fschecks_count08:28:120
4843993123,6cyclictest415-21fschecks_count08:28:120
4843993119,4cyclictest181rcu_preempt09:33:140
4843993119,4cyclictest181rcu_preempt09:33:140
4843993119,4cyclictest181rcu_preempt09:33:140
4843993119,4cyclictest181rcu_preempt09:33:140
4843993113,4cyclictest181rcu_preempt10:13:190
4843993113,4cyclictest181rcu_preempt10:13:190
271622310,2sleep20-21swapper/209:38:142
271622310,2sleep20-21swapper/209:38:142
4849993025,3cyclictest27352-21ntp_offset09:38:171
4849993025,3cyclictest27352-21ntp_offset09:38:171
4849993023,5cyclictest15569-21cut09:08:131
4849993023,5cyclictest15569-21cut09:08:131
4849993022,6cyclictest12666-21kernelversion07:33:151
4849993022,6cyclictest12666-21kernelversion07:33:151
4849993012,2cyclictest0-21swapper/109:15:381
4849993012,2cyclictest0-21swapper/109:15:381
484399306,20cyclictest171ktimers/007:59:040
4843993025,3cyclictest60-21kcompactd010:24:300
4843993025,3cyclictest60-21kcompactd010:24:300
4843993025,3cyclictest60-21kcompactd010:24:300
4843993023,5cyclictest15647-21cut09:08:130
4843993023,5cyclictest15647-21cut09:08:130
4843993017,4cyclictest181rcu_preempt11:38:120
128572300,2sleep012981-21missed_timers11:53:160
4849992923,5cyclictest3002-21processes08:33:201
4849992923,5cyclictest3002-21processes08:33:201
4849992923,5cyclictest3002-21processes08:33:201
4849992920,7cyclictest3771-21processes11:28:201
4849992920,7cyclictest3771-21processes11:28:201
4843992920,7cyclictest995-21needreboot08:28:170
4843992920,7cyclictest995-21needreboot08:28:170
4843992917,3cyclictest181rcu_preempt08:53:140
4843992917,3cyclictest181rcu_preempt08:53:140
4843992914,3cyclictest181rcu_preempt10:41:060
4843992914,3cyclictest181rcu_preempt10:41:060
4849992823,4cyclictest4840-21cyclictest11:58:141
4849992823,4cyclictest4840-21cyclictest11:58:141
4849992823,3cyclictest28152-21sshd08:13:211
4849992823,3cyclictest28152-21sshd08:13:211
484399287,6cyclictest16-21ksoftirqd/009:25:310
484399287,6cyclictest16-21ksoftirqd/009:25:310
484399287,6cyclictest16-21ksoftirqd/009:25:310
484399287,6cyclictest16-21ksoftirqd/009:25:310
484399283,5cyclictest201rcuc/009:16:000
484399283,5cyclictest201rcuc/009:16:000
4843992823,3cyclictest90-1pr/ttyS007:42:120
4843992823,3cyclictest90-1pr/ttyS007:42:120
4843992822,5cyclictest10547-21sshd10:19:210
4843992822,5cyclictest10547-21sshd10:19:210
4843992822,4cyclictest572-21cups-browsed08:13:200
4843992822,4cyclictest572-21cups-browsed08:13:200
4843992822,3cyclictest90-1pr/ttyS010:29:200
4843992822,3cyclictest90-1pr/ttyS010:29:200
4843992822,3cyclictest90-1pr/ttyS010:29:200
4843992820,6cyclictest11962-21gltestperf08:58:120
4843992820,6cyclictest11962-21gltestperf08:58:120
4843992820,2cyclictest181rcu_preempt11:58:340
4843992820,2cyclictest181rcu_preempt11:58:340
4843992818,4cyclictest181rcu_preempt08:08:220
4843992818,4cyclictest181rcu_preempt08:08:220
35922280,3chrt0-21swapper/310:03:113
35922280,3chrt0-21swapper/310:03:113
4862992721,3cyclictest304292sleep309:48:123
4862992721,3cyclictest304292sleep309:48:123
4862992721,3cyclictest304292sleep309:48:123
4849992721,5cyclictest808-21kernelversion08:28:141
4849992721,5cyclictest808-21kernelversion08:28:141
4849992720,5cyclictest32452-21fschecks_time09:53:141
4849992720,5cyclictest20786-21gltestperf10:48:141
4849992720,5cyclictest20786-21gltestperf10:48:141
4843992722,3cyclictest90-1pr/ttyS010:48:320
4843992722,3cyclictest90-1pr/ttyS010:48:320
4843992721,4cyclictest8849-21fschecks_time11:43:100
4843992721,4cyclictest8849-21fschecks_time11:43:100
4843992720,5cyclictest4840-21cyclictest09:56:140
4843992720,5cyclictest4840-21cyclictest09:56:140
4843992718,7cyclictest25768-21threads09:33:220
4843992718,7cyclictest25768-21threads09:33:220
4843992711,4cyclictest90-1pr/ttyS012:13:130
4843992711,4cyclictest90-1pr/ttyS012:13:130
4849992621,4cyclictest4840-21cyclictest10:08:171
4849992621,4cyclictest4840-21cyclictest10:08:171
4849992621,4cyclictest4840-21cyclictest10:08:171
4849992620,5cyclictest4733-21needreboot08:38:181
4849992620,5cyclictest4733-21needreboot08:38:181
4849992620,5cyclictest25668-21sshd11:01:211
4849992620,5cyclictest25668-21sshd11:01:211
4849992620,5cyclictest25668-21sshd11:01:211
4849992617,7cyclictest32353-21fschecks_count09:53:141
484399266,17cyclictest201rcuc/010:47:340
484399266,17cyclictest201rcuc/010:47:340
484399266,17cyclictest201rcuc/010:47:340
484399264,7cyclictest16-21ksoftirqd/008:06:320
484399264,7cyclictest16-21ksoftirqd/008:06:320
484399264,4cyclictest201rcuc/009:07:260
484399264,4cyclictest201rcuc/009:07:260
4843992622,2cyclictest90-1pr/ttyS012:22:240
4843992622,2cyclictest90-1pr/ttyS012:22:240
4843992622,2cyclictest0-21swapper/008:42:590
4843992622,2cyclictest0-21swapper/008:42:590
4843992621,4cyclictest21114-21sh09:23:100
4843992621,4cyclictest21114-21sh09:23:100
4843992621,4cyclictest21114-21sh09:23:100
4843992621,3cyclictest90-1pr/ttyS012:17:230
4843992621,3cyclictest90-1pr/ttyS012:17:230
4843992621,3cyclictest0-21swapper/010:58:220
4843992621,3cyclictest0-21swapper/010:58:220
4843992621,3cyclictest0-21swapper/010:58:220
4843992620,5cyclictest2614-21switchtime09:58:190
4843992620,5cyclictest2614-21switchtime09:58:190
4843992620,5cyclictest2614-21switchtime09:58:190
4843992620,3cyclictest0-21swapper/010:08:250
4843992620,3cyclictest0-21swapper/010:08:250
4843992620,3cyclictest0-21swapper/010:08:250
4843992619,4cyclictest0-21swapper/010:03:140
4843992619,4cyclictest0-21swapper/010:03:140
4843992619,4cyclictest0-21swapper/010:03:140
4843992618,3cyclictest181rcu_preempt11:43:050
4843992618,3cyclictest181rcu_preempt11:43:050
4843992616,8cyclictest20685-21threads07:53:230
4843992616,8cyclictest20685-21threads07:53:230
4843992616,5cyclictest4840-21cyclictest09:43:100
4843992616,5cyclictest4840-21cyclictest09:43:100
4843992616,2cyclictest181rcu_preempt12:38:130
4843992616,2cyclictest181rcu_preempt12:38:130
213472260,2chrt0-21swapper/307:58:013
213472260,2chrt0-21swapper/307:58:013
4849992521,2cyclictest0-21swapper/112:43:111
4849992521,2cyclictest0-21swapper/112:43:111
4849992520,4cyclictest31244-21systemctl09:48:211
4849992518,5cyclictest30835-21latency_hist08:23:071
4849992518,5cyclictest30835-21latency_hist08:23:071
4849992518,5cyclictest12447-21sh11:53:121
4849992518,5cyclictest12447-21sh11:53:121
4849992518,5cyclictest12447-21sh11:53:121
4849992518,4cyclictest0-21swapper/108:48:001
4849992518,4cyclictest0-21swapper/108:48:001
4843992520,3cyclictest0-21swapper/011:18:220
4843992520,3cyclictest0-21swapper/011:18:220
4843992519,4cyclictest14678-21systemd-journal07:29:210
4843992519,4cyclictest14678-21systemd-journal07:29:210
4843992519,4cyclictest14678-21systemd-journal07:29:210
4843992515,3cyclictest181rcu_preempt11:10:470
4843992515,3cyclictest181rcu_preempt11:10:470
4843992515,3cyclictest181rcu_preempt11:10:470
4843992511,8cyclictest90-1pr/ttyS011:16:580
4843992511,8cyclictest90-1pr/ttyS011:16:580
4862992416,4cyclictest4182chrt08:28:123
4862992416,4cyclictest4182chrt08:28:123
4849992420,2cyclictest0-21swapper/111:18:221
4849992420,2cyclictest0-21swapper/111:18:221
4849992420,2cyclictest0-21swapper/107:28:221
4849992420,2cyclictest0-21swapper/107:28:221
4849992420,2cyclictest0-21swapper/107:28:221
4849992416,6cyclictest26584-21sort12:28:221
4849992416,6cyclictest26584-21sort12:28:221
4849992416,6cyclictest26584-21sort12:28:221
4849992416,6cyclictest20786-21grep12:13:211
4849992416,6cyclictest20786-21grep12:13:211
4849992415,7cyclictest25441-21cut08:08:131
4849992415,7cyclictest25441-21cut08:08:131
4849992415,7cyclictest12233-21ls08:58:151
4849992415,7cyclictest12233-21ls08:58:151
4843992420,3cyclictest4840-21cyclictest10:34:080
4843992420,3cyclictest4840-21cyclictest10:34:080
4843992419,3cyclictest90-1pr/ttyS012:29:050
4843992419,3cyclictest90-1pr/ttyS012:29:050
4843992419,3cyclictest90-1pr/ttyS012:29:050
4843992419,3cyclictest0-21swapper/009:43:120
4843992419,3cyclictest0-21swapper/009:43:120
4843992419,3cyclictest0-21swapper/009:43:120
4843992418,4cyclictest0-21swapper/007:46:110
4843992418,4cyclictest0-21swapper/007:46:110
4843992418,3cyclictest0-21swapper/012:03:220
4843992418,3cyclictest0-21swapper/012:03:220
4843992418,3cyclictest0-21swapper/012:03:220
259362240,2sleep20-21swapper/208:08:172
259362240,2sleep20-21swapper/208:08:172
83182230,2sleep30-21swapper/308:48:143
83182230,2sleep30-21swapper/308:48:143
83182230,2sleep30-21swapper/308:48:143
4849992316,5cyclictest21796-21fschecks_time07:58:111
4849992315,6cyclictest20164-21kernelversion07:53:161
4849992315,6cyclictest20164-21kernelversion07:53:161
4849992314,7cyclictest24470-21grep12:23:211
4849992314,7cyclictest24470-21grep12:23:211
4849992314,7cyclictest24470-21grep12:23:211
484399233,4cyclictest201rcuc/011:36:520
484399233,4cyclictest201rcuc/011:36:520
4843992315,3cyclictest181rcu_preempt08:50:520
4843992315,3cyclictest181rcu_preempt08:50:520
4843992315,3cyclictest181rcu_preempt08:50:520
122132230,3sleep30-21swapper/310:23:213
122132230,3sleep30-21swapper/310:23:213
122132230,3sleep30-21swapper/310:23:213
4849992217,3cyclictest0-21swapper/111:08:121
4849992217,3cyclictest0-21swapper/111:08:121
4849992216,3cyclictest0-21swapper/112:08:211
4849992216,3cyclictest0-21swapper/112:08:211
4849992215,5cyclictest4840-21cyclictest10:30:051
4849992215,5cyclictest4840-21cyclictest10:30:051
4849992215,5cyclictest4840-21cyclictest10:30:051
4849992214,6cyclictest9755-21ls10:18:161
4849992214,6cyclictest9755-21ls10:18:161
4843992216,5cyclictest12407-21cut07:33:130
4843992216,5cyclictest12407-21cut07:33:130
4849992117,2cyclictest0-21swapper/111:03:131
4849992116,4cyclictest20423-21latency_hist10:48:111
4849992116,4cyclictest20423-21latency_hist10:48:111
4849992116,4cyclictest20423-21latency_hist10:48:111
4849992116,3cyclictest0-21swapper/110:33:221
4849992116,3cyclictest0-21swapper/110:33:221
4849992115,4cyclictest0-21swapper/110:03:191
4849992115,4cyclictest0-21swapper/110:03:191
4849992115,4cyclictest0-21swapper/110:03:191
4856992014,3cyclictest105382sleep208:53:192
4856992014,3cyclictest105382sleep208:53:192
484999206,12cyclictest729-21dbus-daemon12:22:591
484999206,12cyclictest729-21dbus-daemon12:22:591
486299191,13cyclictest12975-21chrt07:33:203
486299191,13cyclictest12975-21chrt07:33:203
486299181,12cyclictest2638-21chrt08:33:143
486299181,12cyclictest2638-21chrt08:33:143
486299181,12cyclictest2638-21chrt08:33:143
484999182,13cyclictest2404-21ntp_states09:58:171
484999182,13cyclictest2404-21ntp_states09:58:171
484999182,13cyclictest2404-21ntp_states09:58:171
85032170,2sleep20-21swapper/208:48:172
85032170,2sleep20-21swapper/208:48:172
85032170,2sleep20-21swapper/208:48:172
4862991712,3cyclictest181572sleep310:40:473
4862991712,3cyclictest181572sleep310:40:473
486299171,12cyclictest27888-21chrt11:08:113
486299171,12cyclictest27888-21chrt11:08:113
486299171,12cyclictest27888-21chrt11:08:113
486299171,11cyclictest27383-21chrt08:13:123
486299171,11cyclictest27383-21chrt08:13:123
486299171,11cyclictest24869-21taskset10:58:183
486299171,11cyclictest24869-21taskset10:58:183
486299171,11cyclictest24869-21taskset10:58:183
486299171,11cyclictest13501-21chrt11:53:223
486299171,11cyclictest13501-21chrt11:53:223
485699173,10cyclictest0-21swapper/211:44:212
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional