You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-08 - 13:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Fri May 08, 2026 00:44:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
293782820,2sleep1271rcuc/122:27:391
293782820,2sleep1271rcuc/122:27:391
230352640,2chrt0-21swapper/023:37:410
230352640,2chrt0-21swapper/023:37:410
230352640,2chrt0-21swapper/023:37:410
144152610,3sleep20-21swapper/223:17:212
144152610,3sleep20-21swapper/223:17:212
179322600,2chrt17935-21mailstats21:57:390
179322600,2chrt17935-21mailstats21:57:390
125412590,4chrt0-21swapper/000:37:350
125412590,4chrt0-21swapper/000:37:350
20343995141,8cyclictest31445-21/usr/sbin/munin00:02:320
20343995141,8cyclictest31445-21/usr/sbin/munin00:02:320
128062500,3sleep30-21swapper/300:37:383
128062500,3sleep30-21swapper/300:37:383
20343994537,6cyclictest0-21swapper/000:02:250
20343994537,6cyclictest0-21swapper/000:02:250
20344994031,7cyclictest26732-21latency_hist20:57:191
20343993628,6cyclictest21969-21unixbench_multi22:07:420
20343993628,6cyclictest21969-21unixbench_multi22:07:420
20344993529,5cyclictest32702-21netstat22:37:371
20344993529,5cyclictest32702-21netstat22:37:371
20344993529,4cyclictest24458-21kernelversion23:42:341
20343993529,4cyclictest26113-21gltestperf23:47:330
20343993529,4cyclictest26113-21gltestperf23:47:330
20343993525,8cyclictest4399-21kernelversion21:22:340
20343993525,8cyclictest4399-21kernelversion21:22:340
20344993429,3cyclictest0-21swapper/100:02:201
20344993429,3cyclictest0-21swapper/100:02:201
20344993426,6cyclictest31063-21latency_hist19:42:191
20344993426,6cyclictest31063-21latency_hist19:42:191
20344993325,6cyclictest13154-21threads00:37:431
20344993325,6cyclictest13154-21threads00:37:431
2034499332,25cyclictest32109-21strings00:02:371
2034499332,25cyclictest32109-21strings00:02:371
20343993328,3cyclictest23882-21head22:12:440
20343993328,3cyclictest23882-21head22:12:440
20343993327,4cyclictest1-21systemd21:17:390
20343993327,4cyclictest1-21systemd21:17:390
305602320,3chrt0-21swapper/319:37:413
305602320,3chrt0-21swapper/319:37:413
20344993225,5cyclictest31399-21netstat21:07:371
20344993225,5cyclictest31399-21netstat21:07:371
20344993225,5cyclictest18039-21if_eth020:32:331
20344993225,5cyclictest18039-21if_eth020:32:331
123732320,3sleep20-21swapper/221:42:432
123732320,3sleep20-21swapper/221:42:432
2034599312,14cyclictest0-21swapper/200:02:262
2034599312,14cyclictest0-21swapper/200:02:262
20344993125,4cyclictest9894-21grep20:12:191
20344993125,4cyclictest9894-21grep20:12:191
20344993123,6cyclictest9550-21/usr/sbin/munin21:37:371
20344993123,6cyclictest9550-21/usr/sbin/munin21:37:371
20344993123,6cyclictest12968-21fschecks_time23:12:321
20344993123,6cyclictest12968-21fschecks_time23:12:321
20344993122,7cyclictest30808-21needreboot22:32:381
20344993122,7cyclictest30808-21needreboot22:32:381
20343993125,4cyclictest11745-21latency_hist20:17:250
20343993125,4cyclictest11745-21latency_hist20:17:250
20343993124,5cyclictest20275-21latency_hist23:32:310
20343993124,5cyclictest20275-21latency_hist23:32:310
20343993124,5cyclictest20275-21latency_hist23:32:310
2034699301,24cyclictest31334-21chrt00:02:243
2034699301,24cyclictest31334-21chrt00:02:243
20344993026,3cyclictest12711-21ntp_offset20:17:381
20344993026,3cyclictest12711-21ntp_offset20:17:381
20344993023,4cyclictest0-21swapper/122:12:411
20344993023,4cyclictest0-21swapper/122:12:411
20344993022,6cyclictest21120-21latency_hist20:42:191
20344993022,6cyclictest21120-21latency_hist20:42:191
20344993022,5cyclictest2355-21sed21:17:321
20344993022,5cyclictest2355-21sed21:17:321
20344993021,6cyclictest14183-21gltestperf20:22:321
20344993021,6cyclictest14183-21gltestperf20:22:321
20343993023,5cyclictest550-21cron21:47:190
20343993023,5cyclictest550-21cron21:47:190
20343993023,5cyclictest13208-21kernelversion23:12:350
20343993023,5cyclictest13208-21kernelversion23:12:350
20343993022,6cyclictest25848-21fschecks_count19:27:320
20343993022,6cyclictest25848-21fschecks_count19:27:320
20343993022,6cyclictest2135-21needreboot22:42:360
20343993022,6cyclictest2135-21needreboot22:42:360
20343993022,6cyclictest2135-21needreboot22:42:360
20343993022,6cyclictest2135-21needreboot22:42:360
20343993021,7cyclictest8315-21latency_hist20:07:300
20344992922,5cyclictest749-21runrttasks23:17:331
20344992922,5cyclictest749-21runrttasks23:17:331
20344992922,5cyclictest28068-21latency_hist22:27:191
20344992922,5cyclictest28068-21latency_hist22:27:191
20344992922,5cyclictest28068-21latency_hist22:27:191
20344992921,6cyclictest20340-21cyclictest23:23:191
20344992921,6cyclictest20340-21cyclictest23:23:191
20344992921,6cyclictest20340-21cyclictest23:23:191
20344992920,6cyclictest601-21snmpd22:02:341
20344992920,6cyclictest601-21snmpd22:02:341
20343992922,5cyclictest8684-21if_wlan020:07:320
20343992922,5cyclictest8684-21if_wlan020:07:320
20343992922,5cyclictest20340-21cyclictest22:40:410
20343992922,5cyclictest20340-21cyclictest22:40:410
20343992914,12cyclictest0-21swapper/020:42:300
20343992914,12cyclictest0-21swapper/020:42:300
20344992822,4cyclictest20340-21cyclictest23:27:391
20344992822,4cyclictest20340-21cyclictest23:27:391
20344992822,4cyclictest20340-21cyclictest23:27:391
20344992820,6cyclictest10499-21fschecks_time00:32:311
20344992819,7cyclictest27713-21fschecks_count19:32:311
20344992819,7cyclictest27713-21fschecks_count19:32:311
20343992822,3cyclictest0-21swapper/022:02:400
20343992822,3cyclictest0-21swapper/022:02:400
209142270,4sleep30-21swapper/322:07:313
20344992721,5cyclictest4895-21gltestperf00:17:321
20344992721,5cyclictest4895-21gltestperf00:17:321
20344992721,4cyclictest24171-21head19:22:331
20344992721,4cyclictest24171-21head19:22:331
20344992720,5cyclictest9351-21latency_hist21:37:191
20344992720,5cyclictest1232-21kernelversion19:47:351
20344992720,5cyclictest1232-21kernelversion19:47:351
20344992719,6cyclictest23370-21fschecks_count20:47:311
20344992719,6cyclictest23370-21fschecks_count20:47:311
20344992717,7cyclictest6542-21fschecks_count20:02:301
20344992717,7cyclictest6542-21fschecks_count20:02:301
20343992723,3cyclictest20340-21cyclictest21:40:520
20343992723,3cyclictest20340-21cyclictest21:40:520
20343992722,3cyclictest0-21swapper/023:32:440
20343992722,3cyclictest0-21swapper/023:32:440
20343992722,3cyclictest0-21swapper/023:32:440
9522260,3sleep20-21swapper/200:07:302
9522260,3sleep20-21swapper/200:07:302
20344992621,2cyclictest0-21swapper/121:52:391
20344992621,2cyclictest0-21swapper/121:52:391
20344992620,5cyclictest4773-21gltestperf19:57:331
20344992620,5cyclictest4773-21gltestperf19:57:331
20344992620,4cyclictest12175-21ntp_offset21:42:391
20344992620,4cyclictest12175-21ntp_offset21:42:391
20344992618,6cyclictest26090-21fschecks_time23:47:321
20344992618,6cyclictest26090-21fschecks_time23:47:321
20344992618,6cyclictest25446-21cron19:27:311
20344992618,6cyclictest25446-21cron19:27:311
20343992621,3cyclictest0-21swapper/021:27:400
20343992621,3cyclictest0-21swapper/021:27:400
20343992620,5cyclictest15433-21fschecks_time21:52:320
20343992620,5cyclictest15433-21fschecks_time21:52:320
20343992620,4cyclictest5103-21kernelversion00:17:340
20343992620,4cyclictest5103-21kernelversion00:17:340
20343992620,4cyclictest0-21swapper/021:37:180
20343992620,3cyclictest0-21swapper/020:02:300
20343992619,5cyclictest6887-21gltestperf00:22:350
20343992619,5cyclictest6887-21gltestperf00:22:350
20343992619,5cyclictest25090-21/usr/sbin/munin20:52:370
20343992618,6cyclictest9192-21fschecks_time23:02:310
20343992618,6cyclictest9192-21fschecks_time23:02:310
20343992618,6cyclictest9192-21fschecks_time23:02:310
20343992617,6cyclictest1728-21processes00:07:390
20343992617,6cyclictest1728-21processes00:07:390
20343992617,6cyclictest1728-21processes00:07:390
20344992519,5cyclictest10599-21kernelversion20:12:331
20344992519,5cyclictest10599-21kernelversion20:12:331
20344992519,4cyclictest3321-21ntp_offset19:52:361
20344992519,4cyclictest3321-21ntp_offset19:52:361
20344992517,6cyclictest9919-21grep23:02:401
20344992517,6cyclictest9919-21grep23:02:401
20344992517,6cyclictest9919-21grep23:02:401
20343992521,3cyclictest30757-21cpu21:07:300
20343992521,2cyclictest0-21swapper/023:17:420
20343992521,2cyclictest0-21swapper/023:17:420
20343992521,2cyclictest0-21swapper/020:17:380
20343992521,2cyclictest0-21swapper/020:17:380
20343992520,4cyclictest20340-21cyclictest19:37:310
20343992520,4cyclictest20340-21cyclictest19:37:310
20343992520,3cyclictest0-21swapper/022:27:370
20343992520,3cyclictest0-21swapper/022:27:370
20343992519,5cyclictest545-21in:imuxsock20:59:090
20343992519,5cyclictest545-21in:imuxsock20:59:090
20343992519,4cyclictest3483-21ntp_offset00:12:380
20343992519,4cyclictest3483-21ntp_offset00:12:380
20343992519,4cyclictest0-21swapper/022:50:320
20343992519,4cyclictest0-21swapper/022:50:320
20343992519,3cyclictest0-21swapper/022:17:320
20343992519,3cyclictest0-21swapper/022:17:320
20343992519,3cyclictest0-21swapper/022:17:320
20343992519,3cyclictest0-21swapper/021:52:270
20343992519,3cyclictest0-21swapper/021:52:270
20343992519,3cyclictest0-21swapper/021:52:270
20343992519,3cyclictest0-21swapper/021:07:420
20343992519,3cyclictest0-21swapper/021:07:420
20343992519,3cyclictest0-21swapper/020:22:320
20343992519,3cyclictest0-21swapper/020:22:320
20343992518,5cyclictest9010-21apache222:25:250
20343992518,5cyclictest9010-21apache222:25:250
20343992518,5cyclictest9010-21apache222:25:250
20346992419,3cyclictest431rcuc/322:36:553
20346992419,3cyclictest431rcuc/322:36:553
20344992419,3cyclictest20340-21cyclictest22:07:321
20344992419,3cyclictest20340-21cyclictest22:07:321
20344992418,5cyclictest22298-21fschecks_time23:37:321
20344992418,5cyclictest22298-21fschecks_time23:37:321
20344992418,5cyclictest22298-21fschecks_time23:37:321
20344992418,4cyclictest0-21swapper/122:57:421
20344992418,4cyclictest0-21swapper/122:57:421
20344992417,5cyclictest3649-21gltestperf22:47:321
20344992417,5cyclictest3649-21gltestperf22:47:321
20344992417,5cyclictest0-21swapper/120:57:401
20344992417,5cyclictest0-21swapper/120:57:401
20343992420,2cyclictest0-21swapper/021:12:380
20343992420,2cyclictest0-21swapper/021:12:380
20343992420,2cyclictest0-21swapper/019:42:310
20343992420,2cyclictest0-21swapper/019:42:310
20343992420,2cyclictest0-21swapper/019:42:310
20343992419,3cyclictest0-21swapper/019:57:340
20343992419,3cyclictest0-21swapper/019:57:340
20343992418,3cyclictest0-21swapper/021:02:320
20343992418,3cyclictest0-21swapper/021:02:320
20343992417,5cyclictest16425-21needreboot20:27:360
20343992417,5cyclictest16425-21needreboot20:27:360
20343992416,6cyclictest17220-21ntp_kernel_pll_23:22:370
20343992416,6cyclictest17220-21ntp_kernel_pll_23:22:370
20343992416,6cyclictest17220-21ntp_kernel_pll_23:22:370
20343992416,6cyclictest11903-21sort23:07:410
20343992416,6cyclictest11903-21sort23:07:410
20346992318,4cyclictest431rcuc/321:37:573
20346992318,4cyclictest431rcuc/321:37:573
20346992317,4cyclictest240892sleep320:47:393
20346992317,4cyclictest240892sleep320:47:393
20344992318,3cyclictest0-21swapper/100:12:321
20344992318,3cyclictest0-21swapper/100:12:321
20344992316,5cyclictest4462-21latency21:22:341
20344992316,5cyclictest4462-21latency21:22:341
20344992315,6cyclictest2320-21grep22:42:401
20344992315,6cyclictest2320-21grep22:42:401
20344992315,6cyclictest2320-21grep22:42:401
20344992315,6cyclictest2320-21grep22:42:401
20343992319,2cyclictest0-21swapper/000:32:410
20343992319,2cyclictest0-21swapper/000:32:410
20343992318,3cyclictest0-21swapper/023:52:370
20343992318,3cyclictest0-21swapper/023:52:370
20343992318,3cyclictest0-21swapper/023:52:370
20343992318,3cyclictest0-21swapper/000:27:430
20343992318,3cyclictest0-21swapper/000:27:430
20343992317,4cyclictest0-21swapper/022:52:430
20343992317,4cyclictest0-21swapper/022:52:430
20343992317,4cyclictest0-21swapper/019:52:410
20343992317,4cyclictest0-21swapper/019:52:410
20343992316,4cyclictest0-21swapper/022:32:420
20343992316,4cyclictest0-21swapper/022:32:420
20346992217,4cyclictest431rcuc/319:54:463
20346992217,4cyclictest431rcuc/319:54:463
20346992217,3cyclictest431rcuc/323:18:353
20346992217,3cyclictest431rcuc/323:18:353
20344992218,2cyclictest0-21swapper/120:42:331
20344992218,2cyclictest0-21swapper/120:42:331
20344992217,4cyclictest20340-21cyclictest00:22:331
20344992217,4cyclictest20340-21cyclictest00:22:331
20344992217,3cyclictest0-21swapper/122:52:401
20344992217,3cyclictest0-21swapper/122:52:401
20344992217,3cyclictest0-21swapper/100:07:401
20344992217,3cyclictest0-21swapper/100:07:401
20344992217,3cyclictest0-21swapper/100:07:401
20344992217,2cyclictest0-21swapper/121:17:191
20344992217,2cyclictest0-21swapper/121:17:191
20344992216,4cyclictest20340-21cyclictest00:27:421
20344992216,3cyclictest0-21swapper/122:17:351
20344992216,3cyclictest0-21swapper/122:17:351
20344992216,3cyclictest0-21swapper/122:17:351
20344992216,3cyclictest0-21swapper/121:57:341
20344992216,3cyclictest0-21swapper/121:57:341
20344992215,5cyclictest28416-21ntp_kernel_pll_23:52:361
20344992215,5cyclictest28416-21ntp_kernel_pll_23:52:361
20344992215,5cyclictest28416-21ntp_kernel_pll_23:52:361
20344992211,8cyclictest13398-21df21:47:311
20344992211,8cyclictest13398-21df21:47:311
20344992211,8cyclictest13398-21df21:47:311
20343992217,3cyclictest0-21swapper/020:32:310
20343992217,3cyclictest0-21swapper/020:32:310
20343992216,3cyclictest0-21swapper/022:57:340
20343992216,3cyclictest0-21swapper/022:57:340
20343992215,5cyclictest23720-21kernelversion20:47:340
20343992215,5cyclictest23720-21kernelversion20:47:340
20343992215,5cyclictest0-21swapper/019:47:330
20343992215,5cyclictest0-21swapper/019:47:330
2034399221,11cyclictest21135-21awk20:42:190
2034399221,11cyclictest21135-21awk20:42:190
63672210,3sleep20-21swapper/221:27:342
63672210,3sleep20-21swapper/221:27:342
20346992117,3cyclictest431rcuc/320:14:083
20346992117,3cyclictest431rcuc/320:14:083
20346992116,3cyclictest431rcuc/323:48:583
20346992116,3cyclictest431rcuc/323:48:583
20344992117,2cyclictest0-21swapper/121:27:311
20344992117,2cyclictest0-21swapper/121:27:311
20344992116,3cyclictest0-21swapper/123:42:371
20344992116,3cyclictest0-21swapper/123:42:371
20344992116,3cyclictest0-21swapper/123:42:371
20344992116,3cyclictest0-21swapper/123:42:331
20344992116,3cyclictest0-21swapper/123:42:331
20344992115,4cyclictest29163-21grep21:02:321
20344992115,4cyclictest29163-21grep21:02:321
20344992115,4cyclictest29163-21grep21:02:321
20344992114,5cyclictest16022-21fschecks_time20:27:311
20344992114,5cyclictest16022-21fschecks_time20:27:311
20343992112,7cyclictest27538-21/usr/sbin/munin19:32:370
20343992112,7cyclictest27538-21/usr/sbin/munin19:32:370
20346992016,3cyclictest431rcuc/323:15:013
20346992016,3cyclictest431rcuc/323:15:013
20346992016,3cyclictest431rcuc/322:20:553
20346992016,3cyclictest431rcuc/322:20:553
20346992016,3cyclictest431rcuc/322:20:553
20346992016,3cyclictest431rcuc/322:08:453
20346992016,3cyclictest431rcuc/322:08:453
20346992016,3cyclictest431rcuc/321:48:353
20346992016,3cyclictest431rcuc/321:48:353
20346992016,3cyclictest431rcuc/321:48:353
20346992016,3cyclictest431rcuc/321:06:503
20346992016,3cyclictest431rcuc/321:06:503
20346992016,3cyclictest431rcuc/321:06:503
20346992016,3cyclictest431rcuc/320:00:413
20346992016,3cyclictest431rcuc/320:00:413
20346992015,4cyclictest431rcuc/322:38:523
20346992015,4cyclictest431rcuc/322:38:523
20346992015,3cyclictest431rcuc/322:15:443
20346992015,3cyclictest431rcuc/322:15:443
20346992015,3cyclictest431rcuc/321:46:383
20346992015,3cyclictest431rcuc/321:46:383
20346992014,5cyclictest431rcuc/320:18:463
20346992014,5cyclictest431rcuc/320:18:463
20346992014,4cyclictest431rcuc/300:21:063
20346992014,4cyclictest431rcuc/300:21:063
20346992013,5cyclictest431rcuc/322:50:493
20346992013,5cyclictest431rcuc/322:50:493
20344992014,4cyclictest0-21swapper/119:42:401
20344992014,4cyclictest0-21swapper/119:42:401
20344992014,4cyclictest0-21swapper/119:42:401
20344992014,3cyclictest0-21swapper/123:07:411
20344992014,3cyclictest0-21swapper/123:07:411
20343992014,4cyclictest0-21swapper/019:22:410
20343992014,4cyclictest0-21swapper/019:22:410
20343992013,5cyclictest24679-21ntp_kernel_err23:42:370
20343992013,5cyclictest24679-21ntp_kernel_err23:42:370
20343992013,5cyclictest24679-21ntp_kernel_err23:42:370
167822200,2sleep30-21swapper/323:22:323
167822200,2sleep30-21swapper/323:22:323
167822200,2sleep30-21swapper/323:22:323
46162190,3sleep30-21swapper/321:22:373
46162190,3sleep30-21swapper/321:22:373
20346991915,3cyclictest431rcuc/323:43:563
20346991915,3cyclictest431rcuc/323:43:563
20346991915,3cyclictest431rcuc/323:43:563
20346991915,3cyclictest431rcuc/323:28:483
20346991915,3cyclictest431rcuc/323:28:483
20346991915,3cyclictest431rcuc/323:28:483
20346991915,3cyclictest431rcuc/321:35:363
20346991915,3cyclictest431rcuc/321:18:163
20346991915,3cyclictest431rcuc/321:18:163
20346991915,3cyclictest431rcuc/320:33:213
20346991915,3cyclictest431rcuc/320:33:213
20346991915,3cyclictest431rcuc/320:28:073
20346991915,3cyclictest431rcuc/320:28:073
20346991915,3cyclictest431rcuc/320:24:213
20346991915,3cyclictest431rcuc/320:24:213
20346991915,3cyclictest431rcuc/319:33:153
20346991915,3cyclictest431rcuc/319:33:153
20346991914,4cyclictest431rcuc/320:05:563
20346991914,4cyclictest431rcuc/320:05:563
20346991914,4cyclictest431rcuc/319:30:303
20346991914,4cyclictest431rcuc/319:30:303
20346991914,3cyclictest431rcuc/300:10:133
20346991914,3cyclictest431rcuc/300:10:133
20346991914,3cyclictest431rcuc/300:10:133
20346991912,4cyclictest73302sleep300:22:423
20346991912,4cyclictest73302sleep300:22:423
201332190,3chrt0-21swapper/320:37:343
201332190,3chrt0-21swapper/320:37:343
2034699184,11cyclictest0-21swapper/300:02:323
2034699184,11cyclictest0-21swapper/300:02:323
20346991814,3cyclictest431rcuc/322:45:393
20346991814,3cyclictest431rcuc/322:45:393
20346991814,3cyclictest431rcuc/322:45:393
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional