You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-09 - 12:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Thu Jul 09, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1191921150,2sleep0201rcuc/019:53:310
1191921150,2sleep0201rcuc/019:53:310
1191921150,2sleep0201rcuc/019:53:310
31572990,3sleep00-21swapper/022:23:290
31572990,3sleep00-21swapper/022:23:290
31572990,3sleep00-21swapper/022:23:290
177512970,3sleep00-21swapper/000:28:320
177512970,3sleep00-21swapper/000:28:320
177512970,3sleep00-21swapper/000:28:320
36452920,2sleep20-21swapper/220:58:292
36452920,2sleep20-21swapper/220:58:292
36452920,2sleep20-21swapper/220:58:292
249792820,4sleep024980-21cat23:23:150
249792820,4sleep024980-21cat23:23:150
14662640,3sleep20-21swapper/219:25:062
14662640,3sleep20-21swapper/219:25:062
14662640,3sleep20-21swapper/219:25:062
285932620,1sleep00-21swapper/020:38:290
285932620,1sleep00-21swapper/020:38:290
29152590,2sleep02920-21proc_pri23:48:360
29152590,2sleep02920-21proc_pri23:48:360
148102590,1sleep00-21swapper/000:20:150
148102590,1sleep00-21swapper/000:20:150
29194995640,11cyclictest7686-21/usr/sbin/munin00:03:401
29194995640,11cyclictest7686-21/usr/sbin/munin00:03:401
84532520,2sleep30-21swapper/300:03:343
84532520,2sleep30-21swapper/300:03:343
257062510,2sleep025693-21acpi21:58:250
257062510,2sleep025693-21acpi21:58:250
257062510,2sleep025693-21acpi21:58:250
257062510,2sleep025693-21acpi21:58:250
92212500,3sleep20-21swapper/221:13:282
92212500,3sleep20-21swapper/221:13:282
60402490,3sleep30-21swapper/319:38:263
60402490,3sleep30-21swapper/319:38:263
240762480,4chrt0-21swapper/323:18:363
240762480,4chrt0-21swapper/323:18:363
248482470,3sleep30-21swapper/321:53:383
248482470,3sleep30-21swapper/321:53:383
248482470,3sleep30-21swapper/321:53:383
248482470,3sleep30-21swapper/321:53:383
190592440,2sleep30-21swapper/320:13:283
190592440,2sleep30-21swapper/320:13:283
2919299422,31cyclictest7638-21sed00:03:210
2919299422,31cyclictest7638-21sed00:03:210
2919299422,31cyclictest7638-21sed00:03:210
2919299414,33cyclictest8627-21grep00:03:370
2919299414,33cyclictest8627-21grep00:03:370
214262400,3sleep20-21swapper/220:18:332
214262400,3sleep20-21swapper/220:18:332
118382400,2chrt11462-21/usr/sbin/munin00:13:280
212142390,3sleep30-21swapper/300:38:293
212142390,3sleep30-21swapper/300:38:293
212142390,3sleep30-21swapper/300:38:293
19312390,3sleep20-21swapper/222:18:372
19312390,3sleep20-21swapper/222:18:372
19312390,3sleep20-21swapper/222:18:372
18622390,3sleep20-21swapper/220:53:302
18622390,3sleep20-21swapper/220:53:302
75892380,3sleep30-21swapper/300:03:193
75892380,3sleep30-21swapper/300:03:193
75892380,3sleep30-21swapper/300:03:193
318762380,3sleep20-21swapper/219:18:392
318762380,3sleep20-21swapper/219:18:392
301972370,3sleep20-21swapper/222:08:322
301972370,3sleep20-21swapper/222:08:322
301972370,3sleep20-21swapper/222:08:322
301972370,3sleep20-21swapper/222:08:322
232422370,3chrt0-21swapper/220:23:322
171822370,3sleep2351rcuc/220:08:282
171822370,3sleep2351rcuc/220:08:282
29194993628,6cyclictest21950-21timerandwakeup00:38:371
29194993628,6cyclictest21950-21timerandwakeup00:38:371
29194993628,6cyclictest21950-21timerandwakeup00:38:371
40772350,2sleep04179-21switchtime20:58:360
40772350,2sleep04179-21switchtime20:58:360
40772350,2sleep04179-21switchtime20:58:360
284672350,1sleep128569-21/usr/sbin/munin22:03:371
284672350,1sleep128569-21/usr/sbin/munin22:03:371
29194993425,7cyclictest30940-21fschecks_count23:38:271
29192993427,5cyclictest23063-21threads21:48:380
29192993427,5cyclictest23063-21threads21:48:380
29192993427,5cyclictest23063-21threads21:48:380
259582330,3sleep20-21swapper/221:58:272
259582330,3sleep20-21swapper/221:58:272
259582330,3sleep20-21swapper/221:58:272
259582330,3sleep20-21swapper/221:58:272
240432330,3sleep20-21swapper/221:53:282
240432330,3sleep20-21swapper/221:53:282
29194993225,5cyclictest550-21cron21:43:291
29194993225,5cyclictest550-21cron21:43:291
29192993225,5cyclictest0-21swapper/021:38:260
29192993225,5cyclictest0-21swapper/021:38:260
29192993224,6cyclictest601-21snmpd23:08:260
29192993224,6cyclictest601-21snmpd23:08:260
29192993224,6cyclictest601-21snmpd23:08:260
29194993127,2cyclictest0-21swapper/100:03:311
29194993125,4cyclictest2487-21sendmail22:23:161
29194993125,4cyclictest2487-21sendmail22:23:161
29194993125,4cyclictest2487-21sendmail22:23:161
29194993125,4cyclictest13495-21gltestperf19:58:281
29194993125,4cyclictest13495-21gltestperf19:58:281
29192993123,6cyclictest5794-21latency_hist23:58:240
29192993123,6cyclictest5794-21latency_hist23:58:240
29194993025,4cyclictest24655-21ntp_offset21:53:351
29194993025,4cyclictest24655-21ntp_offset21:53:351
29194993025,4cyclictest24655-21ntp_offset21:53:351
29194993025,4cyclictest24655-21ntp_offset21:53:351
29194993021,7cyclictest11257-21threads22:43:381
29194993021,7cyclictest11257-21threads22:43:381
29194993021,7cyclictest11257-21threads22:43:381
29192993024,5cyclictest6704-21fschecks_time22:33:290
29192993024,5cyclictest6704-21fschecks_time22:33:290
29192993024,5cyclictest22606-21sendmail00:43:160
29192993024,5cyclictest22606-21sendmail00:43:160
29192993024,5cyclictest22606-21sendmail00:43:160
29192993023,6cyclictest15154-21unixbench-2d22:53:390
29192993023,6cyclictest15154-21unixbench-2d22:53:390
29192993023,5cyclictest21335-21needreboot20:18:320
29192993023,5cyclictest21335-21needreboot20:18:320
29192993023,5cyclictest17553-21idleruntime-cro23:03:170
29192993023,5cyclictest17553-21idleruntime-cro23:03:170
29192993023,5cyclictest17553-21idleruntime-cro23:03:170
29194992922,5cyclictest584-21latency_hist22:18:261
29194992922,5cyclictest584-21latency_hist22:18:261
29194992922,5cyclictest584-21latency_hist22:18:261
29194992922,5cyclictest0-21swapper/100:03:171
29194992922,5cyclictest0-21swapper/100:03:171
29194992921,6cyclictest12691-21grep21:23:261
29194992921,6cyclictest12691-21grep21:23:261
29192992924,3cyclictest9518-21unixbench-2d22:38:370
29192992924,3cyclictest9518-21unixbench-2d22:38:370
29192992924,3cyclictest9518-21unixbench-2d22:38:370
29192992923,5cyclictest2792-21processes19:28:350
29192992923,5cyclictest2792-21processes19:28:350
29192992923,4cyclictest29188-21cyclictest21:08:270
29192992923,4cyclictest29188-21cyclictest21:08:270
29192992923,4cyclictest29188-21cyclictest21:08:270
29192992923,4cyclictest29188-21cyclictest21:08:270
29192992923,4cyclictest12768-21needreboot22:48:310
29192992923,4cyclictest12768-21needreboot22:48:310
29192992922,5cyclictest31521-21runrttasks22:13:280
29192992922,5cyclictest31521-21runrttasks22:13:280
29192992922,5cyclictest31521-21runrttasks22:13:280
29192992922,5cyclictest31521-21runrttasks22:13:280
29192992922,5cyclictest31459-21needreboot23:38:320
29192992922,5cyclictest14678-21systemd-journal00:36:560
29192992922,5cyclictest14678-21systemd-journal00:36:560
29192992921,6cyclictest4360-21latency_hist22:28:260
29192992921,6cyclictest4360-21latency_hist22:28:260
29192992921,6cyclictest4360-21latency_hist22:28:260
29192992921,6cyclictest1839-21sshd23:47:300
29192992921,6cyclictest1839-21sshd23:47:300
29192992921,6cyclictest1839-21sshd23:47:300
239932290,2sleep10-21swapper/121:53:281
239932290,2sleep10-21swapper/121:53:281
29194992823,4cyclictest5826-21fschecks_count19:38:251
29194992823,4cyclictest5826-21fschecks_count19:38:251
29194992822,5cyclictest27442-21sendmail22:03:151
29194992822,5cyclictest27442-21sendmail22:03:151
29194992822,4cyclictest6159-21gltestperf23:58:281
29192992822,4cyclictest0-21swapper/021:43:310
29192992822,4cyclictest0-21swapper/021:43:310
29192992821,5cyclictest20164-21sshd20:14:300
29192992821,5cyclictest20164-21sshd20:14:300
29192992815,11cyclictest0-21swapper/019:43:280
29192992815,11cyclictest0-21swapper/019:43:280
28722280,3sleep30-21swapper/323:48:353
28722280,3sleep30-21swapper/323:48:353
94612270,2sleep09588-21missed_timers21:13:310
94612270,2sleep09588-21missed_timers21:13:310
2921099276,18cyclictest0-21swapper/320:33:463
2921099276,18cyclictest0-21swapper/320:33:463
2921099271,6cyclictest5968-21chrt22:29:393
2921099271,6cyclictest5968-21chrt22:29:393
2921099271,6cyclictest5968-21chrt22:29:393
2921099271,6cyclictest27134-21chrt21:59:403
2921099271,6cyclictest27134-21chrt21:59:403
29194992722,4cyclictest29188-21cyclictest23:33:381
29194992722,4cyclictest29188-21cyclictest23:33:381
29194992721,5cyclictest23193-21latency_hist23:18:211
29194992721,5cyclictest23193-21latency_hist23:18:211
29194992721,5cyclictest11231-21sshd00:12:301
29194992721,5cyclictest11231-21sshd00:12:301
29194992721,5cyclictest11231-21sshd00:12:301
29194992720,5cyclictest22400-21head21:48:291
29194992720,5cyclictest19309-21sshd20:13:301
29194992720,5cyclictest19309-21sshd20:13:301
29192992721,4cyclictest30630-21latency_hist23:38:170
29192992721,4cyclictest30630-21latency_hist23:38:170
29192992720,6cyclictest10497-21fschecks_time22:43:290
29192992720,6cyclictest10497-21fschecks_time22:43:290
29192992720,6cyclictest10497-21fschecks_time22:43:290
29192992720,5cyclictest15603-21kernelversion20:03:320
29192992720,5cyclictest15603-21kernelversion20:03:320
29192992713,7cyclictest90-1pr/ttyS019:43:060
29192992713,7cyclictest90-1pr/ttyS019:43:060
2921099268,15cyclictest90832chrt19:46:143
2921099268,15cyclictest90832chrt19:46:143
2921099268,14cyclictest10032chrt20:51:513
2921099268,14cyclictest10032chrt20:51:513
2921099261,8cyclictest18202-21chrt21:38:223
2921099261,8cyclictest18202-21chrt21:38:223
2921099261,8cyclictest14039-21taskset21:24:113
2921099261,8cyclictest14039-21taskset21:24:113
2921099261,8cyclictest14039-21taskset21:24:113
2921099261,6cyclictest5309-21chrt19:36:133
2921099261,6cyclictest25870-21chrt20:30:403
2921099261,6cyclictest25870-21chrt20:30:403
2921099261,6cyclictest25870-21chrt20:30:403
2921099261,6cyclictest16569-21chrt20:06:083
2921099261,6cyclictest16569-21chrt20:06:083
2921099261,5cyclictest31460-21chrt19:18:363
2921099261,5cyclictest31460-21chrt19:18:363
2921099261,4cyclictest78692chrt22:35:103
2921099261,4cyclictest78692chrt22:35:103
2921099261,4cyclictest78692chrt22:35:103
29194992622,3cyclictest22345-21latency_hist20:23:151
29194992622,3cyclictest22345-21latency_hist20:23:151
29194992621,3cyclictest24971-21wc23:23:151
29194992621,3cyclictest24971-21wc23:23:151
29194992618,6cyclictest16133-21cut22:58:291
29194992618,6cyclictest16133-21cut22:58:291
29194992618,6cyclictest16133-21cut22:58:291
29192992622,2cyclictest0-21swapper/022:03:290
29192992622,2cyclictest0-21swapper/022:03:290
29192992622,2cyclictest0-21swapper/019:18:370
29192992622,2cyclictest0-21swapper/019:18:370
29192992621,3cyclictest0-21swapper/021:28:280
29192992620,4cyclictest60-21kcompactd021:07:020
29192992620,4cyclictest60-21kcompactd021:07:020
29192992620,4cyclictest60-21kcompactd021:07:020
29192992619,6cyclictest32665-21cut19:23:270
29192992619,6cyclictest32665-21cut19:23:270
29192992619,6cyclictest32665-21cut19:23:270
29192992618,6cyclictest27026-21ntp_kernel_pll_20:33:350
29192992618,6cyclictest27026-21ntp_kernel_pll_20:33:350
29192992617,7cyclictest26328-21ls23:23:380
29192992617,7cyclictest26328-21ls23:23:380
29192992616,8cyclictest12445-21cron21:23:160
29192992616,8cyclictest12445-21cron21:23:160
2921099252,4cyclictest0-21swapper/319:48:363
2921099252,4cyclictest0-21swapper/319:48:363
2921099251,9cyclictest30830-21chrt22:08:573
2921099251,9cyclictest30830-21chrt22:08:573
2921099251,9cyclictest30830-21chrt22:08:573
2921099251,9cyclictest30830-21chrt22:08:573
2921099251,8cyclictest6611-21chrt21:06:223
2921099251,8cyclictest6611-21chrt21:06:223
2921099251,8cyclictest6611-21chrt21:06:223
2921099251,7cyclictest14724-21chrt00:19:103
2921099251,7cyclictest14724-21chrt00:19:103
2921099251,6cyclictest2187-21chrt22:19:383
2921099251,6cyclictest2187-21chrt22:19:383
2921099251,6cyclictest2187-21chrt22:19:383
2921099251,4cyclictest116022sleep322:44:393
2921099251,4cyclictest116022sleep322:44:393
2921099251,4cyclictest116022sleep322:44:393
2921099251,10cyclictest13700-21chrt22:52:273
2921099251,10cyclictest13700-21chrt22:52:273
2921099250,6cyclictest26708-21chrt23:26:283
2921099250,6cyclictest26708-21chrt23:26:283
2921099250,6cyclictest23389-21chrt21:49:273
2921099250,6cyclictest23389-21chrt21:49:273
2921099250,6cyclictest23389-21chrt21:49:273
2921099250,6cyclictest10369-21chrt21:15:343
2921099250,6cyclictest10369-21chrt21:15:343
29194992520,4cyclictest28897-21ntp_offset20:38:341
29194992520,4cyclictest28897-21ntp_offset20:38:341
29194992520,3cyclictest0-21swapper/123:53:321
29194992520,3cyclictest0-21swapper/122:43:151
29194992520,3cyclictest0-21swapper/122:43:151
29194992520,3cyclictest0-21swapper/122:43:151
29194992520,3cyclictest0-21swapper/100:37:251
29194992520,3cyclictest0-21swapper/100:37:251
29194992519,4cyclictest24752-21head20:28:281
29194992519,4cyclictest24752-21head20:28:281
29194992519,4cyclictest24752-21head20:28:281
29192992520,4cyclictest4104-21gltestperf19:33:290
29192992520,3cyclictest0-21swapper/023:33:280
29192992520,3cyclictest0-21swapper/022:28:320
29192992520,3cyclictest0-21swapper/000:23:310
29192992520,3cyclictest0-21swapper/000:23:310
29192992520,3cyclictest0-21swapper/000:23:310
29192992519,5cyclictest13900-21kernelversion00:18:290
29192992519,4cyclictest0-21swapper/023:08:340
29192992519,4cyclictest0-21swapper/023:08:340
29192992519,4cyclictest0-21swapper/023:08:340
29192992519,4cyclictest0-21swapper/019:48:350
29192992519,4cyclictest0-21swapper/019:48:350
2921099241,8cyclictest4190-21chrt22:26:063
2921099241,8cyclictest4190-21chrt22:26:063
2921099241,8cyclictest4190-21chrt22:26:063
2921099241,8cyclictest28511-21chrt23:30:383
2921099241,8cyclictest28511-21chrt23:30:383
2921099241,7cyclictest374-21chrt22:15:483
2921099241,7cyclictest374-21chrt22:15:483
2921099241,7cyclictest374-21chrt22:15:483
2921099241,7cyclictest1690-21chrt19:27:573
2921099241,7cyclictest1690-21chrt19:27:573
2921099241,7cyclictest1690-21chrt19:27:573
2921099241,7cyclictest16563-21chrt00:23:403
2921099241,7cyclictest16563-21chrt00:23:403
2921099241,7cyclictest16563-21chrt00:23:403
2921099241,6cyclictest9853-21chrt22:41:253
2921099241,6cyclictest9853-21chrt22:41:253
2921099241,6cyclictest9853-21chrt22:41:253
2921099241,6cyclictest13451-21chrt00:18:263
2921099241,6cyclictest13451-21chrt00:18:263
2921099241,6cyclictest13451-21chrt00:18:263
2921099241,5cyclictest2671-21chrt20:54:123
2921099241,5cyclictest2671-21chrt20:54:123
2921099241,5cyclictest20527-21chrt00:36:353
2921099241,5cyclictest20527-21chrt00:36:353
2921099241,5cyclictest19333-21chrt23:07:463
2921099241,5cyclictest19333-21chrt23:07:463
2921099241,5cyclictest19333-21chrt23:07:463
2921099241,5cyclictest14536-21chrt19:59:123
2921099241,5cyclictest14536-21chrt19:59:123
2921099241,4cyclictest197682sleep321:41:033
2921099241,4cyclictest197682sleep321:41:033
2921099240,9cyclictest4554-21chrt20:59:263
2921099240,9cyclictest4554-21chrt20:59:263
2921099240,9cyclictest4554-21chrt20:59:263
2921099240,9cyclictest21459-21chrt20:18:343
2921099240,9cyclictest21459-21chrt20:18:343
2921099240,9cyclictest15326-21chrt21:28:333
2921099240,9cyclictest15326-21chrt21:28:333
2921099240,5cyclictest29065-21chrt22:05:153
2921099240,5cyclictest29065-21chrt22:05:153
2921099240,5cyclictest12214-21chrt21:20:173
2921099240,5cyclictest12214-21chrt21:20:173
2921099240,4cyclictest3427-21chrt19:31:193
2921099240,4cyclictest3427-21chrt19:31:193
2921099240,4cyclictest171412chrt22:58:463
2921099240,4cyclictest171412chrt22:58:463
2921099240,4cyclictest171412chrt22:58:463
29194992420,2cyclictest0-21swapper/121:13:301
29194992420,2cyclictest0-21swapper/121:13:301
29194992419,4cyclictest11276-21/usr/sbin/munin19:53:261
29194992419,4cyclictest11276-21/usr/sbin/munin19:53:261
29194992419,3cyclictest0-21swapper/119:28:271
29194992419,3cyclictest0-21swapper/119:28:271
29194992419,2cyclictest0-21swapper/119:33:381
29194992418,4cyclictest4714-21fschecks_count22:28:281
29194992418,4cyclictest4714-21fschecks_count22:28:281
29194992418,4cyclictest4714-21fschecks_count22:28:281
29194992418,4cyclictest30931-21systemctl20:43:351
29194992418,4cyclictest30931-21systemctl20:43:351
29194992418,4cyclictest30931-21systemctl20:43:351
29194992418,3cyclictest0-21swapper/121:33:301
29194992418,3cyclictest0-21swapper/121:33:301
29194992418,3cyclictest0-21swapper/121:33:161
29194992418,3cyclictest0-21swapper/121:33:161
29194992417,5cyclictest18964-21head23:03:381
29194992417,5cyclictest18964-21head23:03:381
29194992417,5cyclictest18964-21head23:03:381
29194992417,5cyclictest13138-21latency_hist19:58:241
29194992417,5cyclictest13138-21latency_hist19:58:241
29194992417,5cyclictest13138-21latency_hist19:58:241
29194992416,6cyclictest20411-21fschecks_time21:43:281
29194992414,8cyclictest8106-21cron22:38:151
29194992414,8cyclictest8106-21cron22:38:151
29194992414,8cyclictest8106-21cron22:38:151
29192992420,2cyclictest0-21swapper/023:28:300
29192992419,3cyclictest0-21swapper/021:28:270
29192992418,4cyclictest0-21swapper/022:18:280
29192992418,4cyclictest0-21swapper/022:18:280
29192992418,4cyclictest0-21swapper/022:18:280
29192992417,5cyclictest14444-21unixbench_singl19:58:380
29192992417,5cyclictest14444-21unixbench_singl19:58:380
29192992417,5cyclictest0-21swapper/020:28:360
29192992417,5cyclictest0-21swapper/020:28:360
29192992417,5cyclictest0-21swapper/020:28:360
29192992416,6cyclictest30287-21cut20:43:260
29192992416,6cyclictest30287-21cut20:43:260
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional