You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-12 - 19:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Fri Jun 12, 2026 12:44:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2633721080,2sleep30-21swapper/308:03:153
2633721080,2sleep30-21swapper/308:03:153
2633721080,2sleep30-21swapper/308:03:153
320822900,2sleep00-21swapper/012:38:120
320822900,2sleep00-21swapper/012:38:120
320822900,2sleep00-21swapper/012:38:120
155572630,3chrt0-21swapper/009:03:050
155572630,3chrt0-21swapper/009:03:050
273942600,2chrt27389-21sed08:08:060
273942600,2chrt27389-21sed08:08:060
247262590,2chrt0-21swapper/209:28:042
247262590,2chrt0-21swapper/209:28:042
296872510,2sleep00-21swapper/012:33:060
296872510,2sleep00-21swapper/012:33:060
296872510,2sleep00-21swapper/012:33:060
309052480,2sleep00-21swapper/008:18:030
309052480,2sleep00-21swapper/008:18:030
252372450,4sleep20-21swapper/208:03:022
252372450,4sleep20-21swapper/208:03:022
252372450,4sleep20-21swapper/208:03:022
312342430,4sleep30-21swapper/308:18:063
312342430,4sleep30-21swapper/308:18:063
309172390,3chrt0-21swapper/309:43:083
309172390,3chrt0-21swapper/309:43:083
309172390,3chrt0-21swapper/309:43:083
6828993728,7cyclictest30703-21threads11:08:100
6828993728,7cyclictest30703-21threads11:08:100
6828993728,7cyclictest30703-21threads11:08:100
6828993728,7cyclictest30703-21threads11:08:100
110242370,2sleep011133-21fschecks_time10:18:040
6828993630,5cyclictest17488-21sendmail07:42:520
29402360,1sleep13104-21fschecks_time11:23:051
29402360,1sleep13104-21fschecks_time11:23:051
29402360,1sleep13104-21fschecks_time11:23:051
136292360,8sleep1271rcuc/108:58:041
6831993530,3cyclictest25155-21idleruntime-cro12:22:521
6831993530,3cyclictest25155-21idleruntime-cro12:22:521
6828993427,5cyclictest8966-21if_wlan011:38:050
6828993426,6cyclictest11436-21unixbench_singl07:23:150
6828993426,6cyclictest11436-21unixbench_singl07:23:150
6831993325,6cyclictest374-21ntp_offset09:48:101
6831993325,6cyclictest374-21ntp_offset09:48:101
6831993325,6cyclictest374-21ntp_offset09:48:101
6828993328,4cyclictest26303-21awk12:23:130
6828993328,4cyclictest26303-21awk12:23:130
6831993226,4cyclictest1346-21avahi-daemon12:13:111
6831993226,4cyclictest1346-21avahi-daemon12:13:111
6831993225,5cyclictest31994-21if_wlan011:13:061
6831993225,5cyclictest31994-21if_wlan011:13:061
6831993225,5cyclictest31994-21if_wlan011:13:061
6831993225,5cyclictest31994-21if_wlan011:13:061
6831993225,5cyclictest28130-21latency_hist09:37:521
6831993225,5cyclictest28130-21latency_hist09:37:521
205542320,1sleep120693-21/usr/sbin/munin10:43:051
205542320,1sleep120693-21/usr/sbin/munin10:43:051
205542320,1sleep120693-21/usr/sbin/munin10:43:051
6828993125,4cyclictest3046-21fschecks_time11:23:050
6828993125,4cyclictest3046-21fschecks_time11:23:050
6828993125,4cyclictest3046-21fschecks_time11:23:050
6828993124,5cyclictest6826-21cyclictest10:43:420
6828993124,5cyclictest6826-21cyclictest10:43:420
6828993124,5cyclictest6826-21cyclictest10:43:420
6828993123,5cyclictest18688-21fschecks_time10:38:040
6828993123,5cyclictest18688-21fschecks_time10:38:040
6831993024,4cyclictest8818-21cron10:12:521
6831993024,4cyclictest8818-21cron10:12:521
6828993026,3cyclictest1-21systemd12:03:130
6828993026,3cyclictest1-21systemd12:03:130
6828993026,3cyclictest1-21systemd12:03:130
6828993024,4cyclictest6826-21cyclictest10:23:120
6828993024,4cyclictest6826-21cyclictest10:23:120
6828993024,4cyclictest11288-21gltestperf10:18:050
6828993024,4cyclictest11288-21gltestperf10:18:050
6828993024,4cyclictest11288-21gltestperf10:18:050
6828993023,6cyclictest21510-21latency_hist12:12:570
6828993023,6cyclictest21510-21latency_hist12:12:570
6828993023,5cyclictest5608-21cron08:37:510
6828993023,5cyclictest5608-21cron08:37:510
6828993023,5cyclictest5608-21cron08:37:510
6828993023,5cyclictest15387-21sshd07:35:160
6828993023,5cyclictest15387-21sshd07:35:160
682899302,12cyclictest16-21ksoftirqd/009:19:400
682899302,12cyclictest16-21ksoftirqd/009:19:400
132142300,2sleep113235-21cat08:57:571
132142300,2sleep113235-21cat08:57:571
6831992923,4cyclictest5656-21gltestperf10:03:041
6831992923,4cyclictest5656-21gltestperf10:03:041
6831992923,4cyclictest5656-21gltestperf10:03:041
6831992923,4cyclictest21785-21gltestperf07:53:061
6831992923,4cyclictest21785-21gltestperf07:53:061
6831992923,4cyclictest15856-21missed_timers09:03:091
6831992923,4cyclictest15856-21missed_timers09:03:091
6831992920,7cyclictest27970-21fschecks_count11:03:061
6831992920,7cyclictest27970-21fschecks_count11:03:061
6831992920,7cyclictest27970-21fschecks_count11:03:061
6831992912,11cyclictest32436-21mailstats11:13:131
6831992912,11cyclictest32436-21mailstats11:13:131
6831992912,11cyclictest32436-21mailstats11:13:131
6828992922,5cyclictest6826-21cyclictest09:25:090
6828992922,5cyclictest6826-21cyclictest09:25:090
6828992922,5cyclictest6826-21cyclictest09:25:090
6828992922,5cyclictest3611-21fschecks_time09:58:040
6828992922,5cyclictest3611-21fschecks_time09:58:040
266882290,1sleep026705-21ntp_states10:58:120
266882290,1sleep026705-21ntp_states10:58:120
266882290,1sleep026705-21ntp_states10:58:120
6831992823,4cyclictest6826-21cyclictest08:08:161
6831992823,4cyclictest6826-21cyclictest08:08:161
6831992820,6cyclictest11604-21needreboot10:18:091
6831992820,6cyclictest11604-21needreboot10:18:091
6831992820,6cyclictest11604-21needreboot10:18:091
6828992823,4cyclictest536-21gltestperf08:23:050
6828992823,4cyclictest536-21gltestperf08:23:050
6828992823,3cyclictest0-21swapper/009:38:130
6828992823,3cyclictest0-21swapper/009:38:130
6828992821,5cyclictest24318-21cut10:53:040
6828992821,5cyclictest24318-21cut10:53:040
6828992821,5cyclictest24318-21cut10:53:040
6828992821,5cyclictest24318-21cut10:53:040
6828992821,5cyclictest12982-21netstat11:48:090
6828992821,5cyclictest12982-21netstat11:48:090
6831992722,4cyclictest6826-21cyclictest08:07:531
6831992722,4cyclictest6826-21cyclictest08:07:531
6831992722,4cyclictest6826-21cyclictest08:07:531
6831992722,3cyclictest12802-21ntp_offset07:28:131
6831992722,3cyclictest12802-21ntp_offset07:28:131
6831992722,3cyclictest12802-21ntp_offset07:28:131
6831992721,4cyclictest6826-21cyclictest08:15:071
6831992719,6cyclictest1702-21cut09:53:061
6831992719,6cyclictest1702-21cut09:53:061
6831992719,6cyclictest16704-21needreboot11:58:081
6831992719,6cyclictest16704-21needreboot11:58:081
6831992719,6cyclictest16704-21needreboot11:58:081
6828992722,3cyclictest0-21swapper/007:48:140
6828992722,3cyclictest0-21swapper/007:48:140
6828992721,5cyclictest24717-21sshd08:00:160
6828992721,5cyclictest24717-21sshd08:00:160
6828992721,3cyclictest0-21swapper/012:18:060
6828992721,3cyclictest0-21swapper/012:18:060
6828992720,5cyclictest32593-21kernelversion09:48:070
6828992720,5cyclictest32593-21kernelversion09:48:070
6828992720,5cyclictest32593-21kernelversion09:48:070
6828992719,6cyclictest32094-21latency_hist09:48:030
41482270,4sleep20-21swapper/208:33:042
41482270,4sleep20-21swapper/208:33:042
41482270,4sleep20-21swapper/208:33:042
683199263,11cyclictest155902cron07:37:531
683199263,11cyclictest155902cron07:37:531
6831992620,5cyclictest25738-21gltestperf12:23:051
6831992620,5cyclictest25738-21gltestperf12:23:051
6831992619,5cyclictest749-21runrttasks07:52:521
6831992619,5cyclictest749-21runrttasks07:52:521
6828992622,2cyclictest0-21swapper/009:47:510
6828992622,2cyclictest0-21swapper/009:47:510
6828992622,2cyclictest0-21swapper/008:42:510
6828992622,2cyclictest0-21swapper/008:42:510
6828992621,3cyclictest0-21swapper/009:53:160
6828992621,3cyclictest0-21swapper/009:53:160
6828992620,5cyclictest11849-21sshd11:45:150
6828992620,5cyclictest11849-21sshd11:45:150
6828992620,4cyclictest0-21swapper/011:28:030
6828992620,4cyclictest0-21swapper/011:28:030
6828992620,4cyclictest0-21swapper/008:43:050
6828992619,5cyclictest19287-21sshd07:47:170
6828992619,5cyclictest19287-21sshd07:47:170
6828992617,6cyclictest0-21swapper/008:13:170
6831992521,3cyclictest6826-21cyclictest09:33:031
6831992521,3cyclictest6826-21cyclictest09:33:031
6831992521,3cyclictest6826-21cyclictest09:33:031
6831992520,3cyclictest547-21rs:main1
6831992520,3cyclictest547-21rs:main1
6831992520,3cyclictest547-21rs:main1
6831992519,4cyclictest0-21swapper/111:08:021
6831992519,4cyclictest0-21swapper/111:08:021
6831992519,4cyclictest0-21swapper/111:08:021
6831992519,4cyclictest0-21swapper/111:08:021
6831992518,4cyclictest24205-21kworker/u9:0-xprtiod10:28:041
6831992514,3cyclictest0-21swapper/110:13:061
6831992514,3cyclictest0-21swapper/110:13:061
6831992514,3cyclictest0-21swapper/110:13:061
6828992521,2cyclictest0-21swapper/010:33:030
6828992521,2cyclictest0-21swapper/010:33:030
6828992519,4cyclictest14678-21systemd-journal12:32:550
6828992519,4cyclictest14678-21systemd-journal12:32:550
6828992519,4cyclictest14678-21systemd-journal12:32:550
6828992519,4cyclictest0-21swapper/010:58:060
6828992519,4cyclictest0-21swapper/010:58:060
6828992519,4cyclictest0-21swapper/010:58:060
6828992519,3cyclictest90-1pr/ttyS012:00:210
6828992519,3cyclictest90-1pr/ttyS012:00:210
6828992519,3cyclictest90-1pr/ttyS012:00:210
6828992518,5cyclictest28418-21latency_hist09:38:020
6828992518,5cyclictest28418-21latency_hist09:38:020
6828992518,5cyclictest17182-21cpuspeed09:08:030
6828992518,5cyclictest17182-21cpuspeed09:08:030
6828992513,11cyclictest74-21hwrng07:53:160
6828992513,11cyclictest74-21hwrng07:53:160
6831992419,4cyclictest6826-21cyclictest11:53:081
6831992419,4cyclictest6826-21cyclictest11:53:081
6831992419,3cyclictest6826-21cyclictest11:28:011
6831992419,3cyclictest6826-21cyclictest11:28:011
6831992419,3cyclictest0-21swapper/108:48:121
6831992419,3cyclictest0-21swapper/108:48:121
6831992419,3cyclictest0-21swapper/108:48:121
6831992418,4cyclictest8983-21sed08:43:161
6831992418,3cyclictest0-21swapper/109:23:021
6831992418,3cyclictest0-21swapper/109:23:021
6831992417,4cyclictest0-21swapper/110:38:041
6831992417,4cyclictest0-21swapper/110:38:041
6831992416,6cyclictest30853-21df08:18:021
6831992416,6cyclictest30853-21df08:18:021
6831992416,6cyclictest13240-21sort11:48:131
6831992416,6cyclictest13240-21sort11:48:131
6828992420,2cyclictest0-21swapper/008:32:520
6828992420,2cyclictest0-21swapper/008:32:520
6828992419,3cyclictest6826-21cyclictest12:13:110
6828992419,3cyclictest6826-21cyclictest12:13:110
6828992419,3cyclictest0-21swapper/009:28:150
6828992419,3cyclictest0-21swapper/009:28:150
6828992419,3cyclictest0-21swapper/009:28:150
6828992416,6cyclictest20653-21latency_hist09:18:030
6828992416,6cyclictest20653-21latency_hist09:18:030
6828992416,6cyclictest20653-21latency_hist09:18:030
6828992415,7cyclictest0-21swapper/011:03:160
6828992415,7cyclictest0-21swapper/011:03:160
6828992415,7cyclictest0-21swapper/011:03:160
6828992415,7cyclictest0-21swapper/011:03:160
6828992415,3cyclictest181rcu_preempt11:13:190
6828992415,3cyclictest181rcu_preempt11:13:190
6828992415,3cyclictest181rcu_preempt11:13:190
6828992410,11cyclictest0-21swapper/008:53:040
6828992410,11cyclictest0-21swapper/008:53:040
288562240,3chrt0-21swapper/211:03:152
288562240,3chrt0-21swapper/211:03:152
288562240,3chrt0-21swapper/211:03:152
288562240,3chrt0-21swapper/211:03:152
6831992318,3cyclictest0-21swapper/109:58:111
6831992318,3cyclictest0-21swapper/109:58:111
6831992317,5cyclictest24432-21unixbench-2d07:58:121
6831992317,5cyclictest24432-21unixbench-2d07:58:121
6831992317,5cyclictest18951-21/usr/sbin/munin09:13:061
6831992316,5cyclictest31332-21gltestperf12:38:031
6831992316,5cyclictest31332-21gltestperf12:38:031
6831992316,5cyclictest18934-21unixbench_singl07:43:171
6831992316,5cyclictest18934-21unixbench_singl07:43:171
6831992316,5cyclictest14678-21systemd-journal09:39:171
6831992316,5cyclictest14678-21systemd-journal09:39:171
6831992316,5cyclictest0-21swapper/112:03:141
6831992316,5cyclictest0-21swapper/112:03:141
6831992316,5cyclictest0-21swapper/112:03:141
6831992313,8cyclictest14678-21systemd-journal10:29:031
6831992313,8cyclictest14678-21systemd-journal10:29:031
6831992312,9cyclictest17323-21ntp_states10:33:091
6831992312,9cyclictest17323-21ntp_states10:33:091
6828992318,4cyclictest10117-21timerandwakeup10:13:130
6828992318,4cyclictest10117-21timerandwakeup10:13:130
6828992317,4cyclictest0-21swapper/010:03:130
6828992317,4cyclictest0-21swapper/010:03:130
6828992317,4cyclictest0-21swapper/010:03:130
6828992316,4cyclictest0-21swapper/010:08:140
6828992316,4cyclictest0-21swapper/010:08:140
6828992315,6cyclictest7179-21sort11:33:060
6828992315,6cyclictest7179-21sort11:33:060
6828992315,6cyclictest14130-21/usr/sbin/munin11:53:110
6828992315,6cyclictest14130-21/usr/sbin/munin11:53:110
266292230,3sleep20-21swapper/212:23:152
266292230,3sleep20-21swapper/212:23:152
6831992217,3cyclictest0-21swapper/108:32:521
6831992217,3cyclictest0-21swapper/108:32:521
6831992211,9cyclictest14678-21systemd-journal12:29:211
6831992211,9cyclictest14678-21systemd-journal12:29:211
6831992211,9cyclictest14678-21systemd-journal12:29:211
683199220,12cyclictest10598-21cat07:23:051
683199220,12cyclictest10598-21cat07:23:051
6828992218,2cyclictest0-21swapper/010:33:110
6828992218,2cyclictest0-21swapper/010:33:110
6828992218,2cyclictest0-21swapper/008:58:140
6828992217,3cyclictest0-21swapper/011:28:070
6828992217,3cyclictest0-21swapper/011:28:070
6828992217,3cyclictest0-21swapper/011:28:070
6828992217,3cyclictest0-21swapper/008:48:150
6828992217,3cyclictest0-21swapper/008:48:150
6828992217,3cyclictest0-21swapper/008:48:150
6828992217,3cyclictest0-21swapper/008:03:040
6828992217,3cyclictest0-21swapper/008:03:040
6828992217,3cyclictest0-21swapper/008:03:040
6828992211,9cyclictest12297-21tune2fs07:28:070
6828992211,9cyclictest12297-21tune2fs07:28:070
6828992211,9cyclictest12297-21tune2fs07:28:070
6831992117,2cyclictest0-21swapper/109:13:021
6831992116,3cyclictest0-21swapper/112:37:511
6831992116,3cyclictest0-21swapper/110:52:511
6831992116,3cyclictest0-21swapper/110:52:511
6831992116,3cyclictest0-21swapper/110:52:511
6831992116,3cyclictest0-21swapper/110:52:511
6831992115,5cyclictest30911-21needreboot09:43:081
6831992115,5cyclictest30911-21needreboot09:43:081
6831992115,5cyclictest30911-21needreboot09:43:081
6831992114,4cyclictest0-21swapper/110:23:121
6831992114,4cyclictest0-21swapper/109:23:161
6831992114,4cyclictest0-21swapper/109:23:161
6831992114,4cyclictest0-21swapper/109:23:161
6831992111,8cyclictest14678-21systemd-journal07:41:441
80582200,3sleep20-21swapper/208:43:052
6838992014,3cyclictest289182chrt12:32:522
6838992014,3cyclictest289182chrt12:32:522
6838992014,3cyclictest289182chrt12:32:522
683899201,14cyclictest11323-21chrt10:18:052
683899201,14cyclictest11323-21chrt10:18:052
683899201,14cyclictest11323-21chrt10:18:052
683199205,4cyclictest181rcu_preempt11:33:141
683199205,4cyclictest181rcu_preempt11:33:141
6831992016,2cyclictest0-21swapper/111:38:071
6831992016,2cyclictest0-21swapper/109:13:111
6831992016,2cyclictest0-21swapper/109:13:111
6831992016,2cyclictest0-21swapper/109:13:111
6831992015,2cyclictest0-21swapper/111:28:091
6831992015,2cyclictest0-21swapper/111:28:091
6831992015,2cyclictest0-21swapper/111:28:091
6831992014,4cyclictest7200-21sshd08:39:161
6831992014,4cyclictest7200-21sshd08:39:161
6831992013,5cyclictest10723-21head11:43:051
6831992013,5cyclictest10723-21head11:43:051
171352200,3chrt0-21swapper/309:08:033
171352200,3chrt0-21swapper/309:08:033
683999193,9cyclictest0-21swapper/307:23:173
683999193,9cyclictest0-21swapper/307:23:173
683899193,9cyclictest0-21swapper/207:23:172
683899193,9cyclictest0-21swapper/207:23:172
6838991913,4cyclictest23626-21taskset07:58:032
6838991912,3cyclictest317782sleep208:18:132
6838991912,3cyclictest317782sleep208:18:132
683199199,8cyclictest799-21memory08:23:081
683199199,8cyclictest799-21memory08:23:081
6831991915,2cyclictest0-21swapper/108:37:511
6831991915,2cyclictest0-21swapper/108:37:511
6831991915,2cyclictest0-21swapper/108:37:511
683199191,4cyclictest0-21swapper/112:38:121
683199191,4cyclictest0-21swapper/112:38:121
683199191,4cyclictest0-21swapper/112:38:121
6831991914,2cyclictest0-21swapper/112:12:551
6831991914,2cyclictest0-21swapper/112:12:551
683999181,9cyclictest0-21swapper/309:23:163
683999181,9cyclictest0-21swapper/309:23:163
683999181,9cyclictest0-21swapper/309:23:163
683999181,13cyclictest14617-21chrt08:58:143
683999181,13cyclictest13880-21chrt10:23:123
683999181,13cyclictest13880-21chrt10:23:123
683999181,12cyclictest8204-21chrt10:08:133
683999181,12cyclictest8204-21chrt10:08:133
683999181,12cyclictest23125-21chrt10:48:133
683999181,12cyclictest23125-21chrt10:48:133
683999181,12cyclictest23125-21chrt10:48:133
683999181,12cyclictest23125-21chrt10:48:133
683999181,12cyclictest16097-21chrt09:03:123
683999181,12cyclictest16097-21chrt09:03:123
683899181,13cyclictest349-21chrt09:48:102
683899181,13cyclictest349-21chrt09:48:102
683899181,13cyclictest349-21chrt09:48:102
683899181,12cyclictest8833-21chrt11:38:052
683899181,12cyclictest27633-21taskset11:02:512
683899181,12cyclictest27633-21taskset11:02:512
683899181,12cyclictest27633-21taskset11:02:512
683899181,12cyclictest0-21swapper/210:33:002
683899181,12cyclictest0-21swapper/210:33:002
683999173,9cyclictest0-21swapper/312:13:153
683999173,9cyclictest0-21swapper/312:13:153
683999171,13cyclictest32748-21chrt12:42:583
683999171,13cyclictest32748-21chrt12:42:583
683999171,13cyclictest32748-21chrt12:42:583
683999171,12cyclictest9975-21chrt10:13:113
683999171,12cyclictest9975-21chrt10:13:113
683999171,12cyclictest9975-21chrt10:13:113
683999171,12cyclictest12134-21taskset11:47:543
683999171,12cyclictest12134-21taskset11:47:543
683999171,11cyclictest29598-21chrt12:33:053
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional