You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-26 - 07:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Fri Jun 26, 2026 00:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2704821020,2sleep127047-21latency_hist21:08:041
2704821020,2sleep127047-21latency_hist21:08:041
318432920,3sleep30-21swapper/321:18:233
318432920,3sleep30-21swapper/321:18:233
318432920,3sleep30-21swapper/321:18:233
36132790,3sleep30-21swapper/321:28:283
36132790,3sleep30-21swapper/321:28:283
36132790,3sleep30-21swapper/321:28:283
193272790,2sleep119329-21sed23:38:181
193272790,2sleep119329-21sed23:38:181
193272790,2sleep119329-21sed23:38:181
262102770,3sleep30-21swapper/323:58:143
262102770,3sleep30-21swapper/323:58:143
262102770,3sleep30-21swapper/323:58:143
164762770,3sleep30-21swapper/322:03:253
164762770,3sleep30-21swapper/322:03:253
164762770,3sleep30-21swapper/322:03:253
197832730,3sleep20-21swapper/220:48:152
267162680,4chrt25962-21/usr/sbin/munin19:38:231
267162680,4chrt25962-21/usr/sbin/munin19:38:231
267162680,4chrt25962-21/usr/sbin/munin19:38:231
262022670,3sleep30-21swapper/321:03:233
262022670,3sleep30-21swapper/321:03:233
52392610,13sleep1271rcuc/121:33:241
52392610,13sleep1271rcuc/121:33:241
52392610,13sleep1271rcuc/121:33:241
289632580,3sleep10-21swapper/121:13:061
289632580,3sleep10-21swapper/121:13:061
289632580,3sleep10-21swapper/121:13:061
39722550,2sleep13974-21fschecks_time22:58:191
39722550,2sleep13974-21fschecks_time22:58:191
39722550,2sleep13974-21fschecks_time22:58:191
256202510,5sleep124294-21/usr/sbin/munin23:53:261
256202510,5sleep124294-21/usr/sbin/munin23:53:261
256202510,5sleep124294-21/usr/sbin/munin23:53:261
303162490,3sleep20-21swapper/222:43:042
303162490,3sleep20-21swapper/222:43:042
303162490,3sleep20-21swapper/222:43:042
104922480,3chrt0-21swapper/300:43:053
104922480,3chrt0-21swapper/300:43:053
67142470,3chrt0-21swapper/320:13:143
67142470,3chrt0-21swapper/320:13:143
218912470,2sleep30-21swapper/323:43:273
218912470,2sleep30-21swapper/323:43:273
218912470,2sleep30-21swapper/323:43:273
206522450,3chrt0-21swapper/320:48:243
206522450,3chrt0-21swapper/320:48:243
206522450,3chrt0-21swapper/320:48:243
210472440,4sleep117111-21cyclictest19:23:191
210472440,4sleep117111-21cyclictest19:23:191
97092430,2sleep20-21swapper/200:38:262
97092430,2sleep20-21swapper/200:38:262
207232430,2sleep20-21swapper/219:23:162
207232430,2sleep20-21swapper/219:23:162
17114994328,12cyclictest0-21swapper/000:03:120
17114994328,12cyclictest0-21swapper/000:03:120
17114994328,12cyclictest0-21swapper/000:03:120
129592410,2sleep10-21swapper/120:28:211
129592410,2sleep10-21swapper/120:28:211
82842400,4sleep30-21swapper/321:43:153
82842400,4sleep30-21swapper/321:43:153
32072400,3sleep20-21swapper/220:03:162
32072400,3sleep20-21swapper/220:03:162
1711599402,31cyclictest28706-21memory00:03:251
1711599402,31cyclictest28706-21memory00:03:251
1711599402,31cyclictest28706-21memory00:03:251
17115993934,3cyclictest12878-21cron23:23:041
17115993934,3cyclictest12878-21cron23:23:041
17115993927,9cyclictest550-21cron22:43:171
17115993927,9cyclictest550-21cron22:43:171
17115993833,3cyclictest17111-21cyclictest19:43:421
17115993833,3cyclictest17111-21cyclictest19:43:421
207182370,4sleep30-21swapper/323:43:153
207182370,4sleep30-21swapper/323:43:153
195612360,3chrt0-21swapper/122:13:161
195612360,3chrt0-21swapper/122:13:161
1711599365,7cyclictest281ktimers/120:33:451
1711599365,7cyclictest281ktimers/120:33:451
1711599365,7cyclictest281ktimers/120:33:451
17115993629,5cyclictest21444-21idleruntime-cro20:53:031
17115993629,5cyclictest21444-21idleruntime-cro20:53:031
17115993629,5cyclictest21444-21idleruntime-cro20:53:031
314512350,3sleep30-21swapper/319:53:083
314512350,3sleep30-21swapper/319:53:083
17115993529,4cyclictest2349-21kernelversion22:53:201
17115993529,4cyclictest2349-21kernelversion22:53:201
17115993529,4cyclictest2349-21kernelversion22:53:201
1711599349,22cyclictest271rcuc/120:38:461
1711599349,22cyclictest271rcuc/120:38:461
1711599347,5cyclictest281ktimers/120:14:461
1711599347,5cyclictest281ktimers/120:14:461
17115993430,3cyclictest90-1pr/ttyS023:23:331
17115993430,3cyclictest90-1pr/ttyS023:23:331
17114993427,5cyclictest27656-21processes22:33:240
17114993427,5cyclictest27656-21processes22:33:240
17114993427,5cyclictest27656-21processes22:33:240
17114993425,7cyclictest29354-21head00:03:340
17114993425,7cyclictest29354-21head00:03:340
17114993425,7cyclictest29354-21head00:03:340
192822330,3sleep30-21swapper/322:13:143
192822330,3sleep30-21swapper/322:13:143
192822330,3sleep30-21swapper/322:13:143
1711599336,6cyclictest281ktimers/121:41:001
1711599336,6cyclictest281ktimers/121:41:001
17115993325,6cyclictest3301-21fschecks_count00:23:181
17114993325,6cyclictest25060-21netstat23:53:200
17114993325,6cyclictest25060-21netstat23:53:200
17114993325,6cyclictest25060-21netstat23:53:200
23132320,3sleep1281ktimers/100:18:271
199072320,3sleep30-21swapper/320:48:163
17115993226,5cyclictest1116-21latency_hist00:18:091
17115993226,5cyclictest1116-21latency_hist00:18:091
17115993226,5cyclictest1116-21latency_hist00:18:091
17115993124,5cyclictest29954-21cut19:48:151
17115993124,5cyclictest29954-21cut19:48:151
17115993124,5cyclictest17377-21kernelversion23:33:171
17115993124,5cyclictest17377-21kernelversion23:33:171
17115993124,5cyclictest17377-21kernelversion23:33:171
17114993124,5cyclictest10209-21fschecks_time21:48:170
17114993124,5cyclictest10209-21fschecks_time21:48:170
17114993123,6cyclictest11391-21fschecks_count23:18:160
17114993123,6cyclictest11391-21fschecks_count23:18:160
113002310,4sleep3431rcuc/320:23:233
113002310,4sleep3431rcuc/320:23:233
1711699301,22cyclictest28015-21chrt00:03:122
1711699301,22cyclictest28015-21chrt00:03:122
1711699301,22cyclictest28015-21chrt00:03:122
1711599308,10cyclictest29-21ksoftirqd/120:54:341
1711599308,10cyclictest29-21ksoftirqd/120:54:341
1711599308,10cyclictest29-21ksoftirqd/120:54:341
1711599306,2cyclictest271rcuc/121:28:011
1711599306,2cyclictest271rcuc/121:28:011
1711599304,6cyclictest281ktimers/120:22:001
1711599304,6cyclictest281ktimers/120:22:001
1711599304,2cyclictest271rcuc/123:04:371
1711599304,2cyclictest271rcuc/123:04:371
17115993025,4cyclictest17111-21cyclictest23:08:281
17115993023,5cyclictest17176-21sshd22:07:161
17115993023,5cyclictest17176-21sshd22:07:161
17115993023,5cyclictest17176-21sshd22:07:161
17115993016,3cyclictest90-1pr/ttyS022:13:271
17115993016,3cyclictest90-1pr/ttyS022:13:271
17114993024,3cyclictest0-21swapper/022:43:040
17114993024,3cyclictest0-21swapper/022:43:040
17114993024,3cyclictest0-21swapper/022:43:040
17114993023,5cyclictest15827-21latency_hist20:38:030
17114993023,5cyclictest15827-21latency_hist20:38:030
17114993023,5cyclictest15827-21latency_hist20:38:030
17114993022,6cyclictest32613-21fschecks_time22:48:160
1711599298,4cyclictest281ktimers/123:18:091
1711599298,4cyclictest281ktimers/123:18:091
17115992925,2cyclictest0-21swapper/121:28:261
17115992925,2cyclictest0-21swapper/121:28:261
17115992925,2cyclictest0-21swapper/121:28:261
17115992921,6cyclictest0-21swapper/123:48:041
17115992921,6cyclictest0-21swapper/123:48:041
17115992921,6cyclictest0-21swapper/123:48:041
17115992920,7cyclictest29341-21needreboot22:38:221
17115992920,7cyclictest29341-21needreboot22:38:221
17115992920,7cyclictest29341-21needreboot22:38:221
17115992917,3cyclictest181rcu_preempt21:51:201
17115992917,3cyclictest181rcu_preempt21:51:201
17114992924,3cyclictest601-21snmpd22:28:150
17114992924,3cyclictest601-21snmpd22:28:150
17114992924,3cyclictest0-21swapper/019:38:270
17114992924,3cyclictest0-21swapper/019:38:270
17114992924,3cyclictest0-21swapper/019:38:270
17114992923,4cyclictest4061-21gltestperf22:58:190
17114992923,4cyclictest4061-21gltestperf22:58:190
17114992923,4cyclictest4061-21gltestperf22:58:190
17114992923,3cyclictest0-21swapper/020:13:030
17114992923,3cyclictest0-21swapper/020:13:030
17114992922,6cyclictest4816-21latency_hist00:28:030
17114992922,6cyclictest4816-21latency_hist00:28:030
17114992921,6cyclictest22514-21fschecks_time19:28:160
17114992921,6cyclictest22514-21fschecks_time19:28:160
17114992920,7cyclictest18637-21netstat20:43:190
17114992920,7cyclictest18637-21netstat20:43:190
17115992824,3cyclictest90-1pr/ttyS000:09:161
17115992824,3cyclictest90-1pr/ttyS000:09:161
17115992824,3cyclictest90-1pr/ttyS000:09:161
17115992818,3cyclictest90-1pr/ttyS023:53:141
17115992818,3cyclictest90-1pr/ttyS023:53:141
17115992811,4cyclictest29-21ksoftirqd/100:03:011
17115992811,4cyclictest29-21ksoftirqd/100:03:011
17115992811,4cyclictest29-21ksoftirqd/100:03:011
17114992822,4cyclictest545-21in:imuxsock19:53:030
17114992822,4cyclictest545-21in:imuxsock19:53:030
17114992821,5cyclictest9965-21sshd20:20:160
17114992821,5cyclictest9965-21sshd20:20:160
17114992821,3cyclictest0-21swapper/020:58:040
17114992821,3cyclictest0-21swapper/020:58:040
17114992821,3cyclictest0-21swapper/020:58:040
17114992820,6cyclictest32176-21grep19:53:190
17114992820,6cyclictest32176-21grep19:53:190
1711599274,20cyclictest281ktimers/122:52:471
1711599274,20cyclictest281ktimers/122:52:471
17115992723,3cyclictest90-1pr/ttyS022:10:161
17115992723,2cyclictest0-21swapper/121:58:161
17115992723,2cyclictest0-21swapper/121:58:161
17115992723,2cyclictest0-21swapper/121:58:161
17115992722,4cyclictest0-21swapper/121:47:311
17115992722,4cyclictest0-21swapper/121:47:311
17115992721,4cyclictest281ktimers/119:31:121
17115992721,4cyclictest281ktimers/119:31:121
17115992720,6cyclictest5586-21proc_pri20:08:241
17115992720,6cyclictest5586-21proc_pri20:08:241
17115992720,2cyclictest181rcu_preempt21:14:421
17115992720,2cyclictest181rcu_preempt21:14:421
17115992720,2cyclictest181rcu_preempt21:14:421
17115992719,6cyclictest4818-21sort00:28:031
17115992719,6cyclictest4818-21sort00:28:031
17115992719,6cyclictest27795-21ls22:33:261
17115992719,6cyclictest27795-21ls22:33:261
17115992719,6cyclictest27795-21ls22:33:261
17115992719,2cyclictest181rcu_preempt20:59:081
17115992719,2cyclictest181rcu_preempt20:59:081
17115992716,4cyclictest181rcu_preempt19:34:381
17115992716,4cyclictest181rcu_preempt19:34:381
17114992722,4cyclictest17111-21cyclictest22:43:220
1711599264,5cyclictest281ktimers/120:06:321
1711599264,5cyclictest281ktimers/120:06:321
17115992622,2cyclictest0-21swapper/119:54:341
17115992622,2cyclictest0-21swapper/119:54:341
17115992621,4cyclictest17111-21cyclictest23:31:351
17115992621,4cyclictest17111-21cyclictest23:31:351
17115992621,4cyclictest1346-21avahi-daemon00:38:251
17115992621,4cyclictest1346-21avahi-daemon00:38:251
17115992620,5cyclictest23586-21kernelversion22:23:201
17115992620,5cyclictest23586-21kernelversion22:23:201
17115992620,4cyclictest0-21swapper/120:43:181
17115992620,4cyclictest0-21swapper/120:43:181
17115992615,3cyclictest181rcu_preempt22:01:181
17115992615,3cyclictest181rcu_preempt22:01:181
17114992621,4cyclictest14678-21systemd-journal19:37:150
17114992621,4cyclictest14678-21systemd-journal19:37:150
17114992620,4cyclictest32737-21tail22:48:160
17114992620,4cyclictest32737-21tail22:48:160
17114992620,4cyclictest0-21swapper/020:48:170
17114992620,4cyclictest0-21swapper/020:48:170
17114992620,4cyclictest0-21swapper/020:48:170
17114992619,6cyclictest16602-21latency20:38:190
17114992619,6cyclictest16602-21latency20:38:190
17114992619,5cyclictest3347-21sshd22:56:160
17114992619,5cyclictest3347-21sshd22:56:160
17114992619,5cyclictest3347-21sshd22:56:160
17114992619,5cyclictest25188-21sendmail21:03:030
17114992619,5cyclictest25188-21sendmail21:03:030
17114992619,5cyclictest23004-21latency_hist22:23:120
17114992619,5cyclictest23004-21latency_hist22:23:120
17114992619,5cyclictest23004-21latency_hist22:23:120
17114992619,5cyclictest17788-21gltestperf22:08:190
17114992619,5cyclictest17788-21gltestperf22:08:190
17114992619,5cyclictest17788-21gltestperf22:08:190
17114992619,5cyclictest15971-21switchtime23:28:260
17114992619,5cyclictest15971-21switchtime23:28:260
17114992619,5cyclictest1067-21kernelversion21:23:190
17114992619,5cyclictest1067-21kernelversion21:23:190
17114992618,6cyclictest7104-21fschecks_count00:33:180
17114992616,7cyclictest13380-21fschecks_time23:23:160
17114992616,7cyclictest13380-21fschecks_time23:23:160
1711899252,16cyclictest29431-21taskset00:04:013
1711899252,16cyclictest29431-21taskset00:04:013
1711899252,16cyclictest29431-21taskset00:04:013
17115992521,3cyclictest90-1pr/ttyS022:21:091
17115992521,3cyclictest90-1pr/ttyS022:21:091
17115992521,3cyclictest90-1pr/ttyS022:21:091
17115992518,2cyclictest181rcu_preempt00:30:031
17115992518,2cyclictest181rcu_preempt00:30:031
17114992520,3cyclictest0-21swapper/021:13:160
17114992520,3cyclictest0-21swapper/021:13:160
17114992520,3cyclictest0-21swapper/021:13:160
17114992519,4cyclictest20592-21fschecks_time19:23:160
17114992519,4cyclictest20592-21fschecks_time19:23:160
17114992519,4cyclictest14678-21systemd-journal19:48:030
17114992519,4cyclictest14678-21systemd-journal19:48:030
17114992519,3cyclictest0-21swapper/021:48:160
17114992513,10cyclictest0-21swapper/021:18:430
17114992513,10cyclictest0-21swapper/021:18:430
17114992513,10cyclictest0-21swapper/021:18:430
1711899244,11cyclictest0-21swapper/300:03:103
1711899244,11cyclictest0-21swapper/300:03:103
1711899244,11cyclictest0-21swapper/300:03:103
17115992419,3cyclictest0-21swapper/100:33:261
17115992419,3cyclictest0-21swapper/100:33:261
17115992419,3cyclictest0-21swapper/100:33:261
17115992418,4cyclictest0-21swapper/120:03:031
17115992418,4cyclictest0-21swapper/120:03:031
17115992417,2cyclictest181rcu_preempt21:18:251
17115992417,2cyclictest181rcu_preempt21:18:251
17115992417,2cyclictest181rcu_preempt21:18:251
17114992420,3cyclictest32119-21gltestperf00:13:180
17114992420,3cyclictest32119-21gltestperf00:13:180
17114992420,3cyclictest32119-21gltestperf00:13:180
17114992420,2cyclictest0-21swapper/020:23:250
17114992420,2cyclictest0-21swapper/020:23:250
17114992419,3cyclictest0-21swapper/021:48:050
17114992418,4cyclictest1992-21ntp_kernel_pll_00:18:230
17114992418,4cyclictest1992-21ntp_kernel_pll_00:18:230
17114992418,3cyclictest0-21swapper/023:48:180
17114992418,3cyclictest0-21swapper/023:48:180
17114992418,3cyclictest0-21swapper/023:13:190
17114992418,3cyclictest0-21swapper/023:13:190
17114992418,3cyclictest0-21swapper/021:43:040
17114992418,3cyclictest0-21swapper/021:43:040
17114992416,6cyclictest12565-21ntp_offset21:53:230
17114992416,6cyclictest12565-21ntp_offset21:53:230
17114992416,6cyclictest12565-21ntp_offset21:53:230
17114992414,7cyclictest0-21swapper/021:33:140
17114992414,7cyclictest0-21swapper/021:33:140
17114992414,7cyclictest0-21swapper/021:33:140
1711599233,4cyclictest29-21ksoftirqd/120:23:261
1711599233,4cyclictest29-21ksoftirqd/120:23:261
17115992316,5cyclictest25995-21timerandwakeup22:28:261
17115992316,5cyclictest25995-21timerandwakeup22:28:261
17114992319,3cyclictest29954-21sh21:13:230
17114992319,3cyclictest29954-21sh21:13:230
17114992319,3cyclictest29954-21sh21:13:230
17114992319,3cyclictest17111-21cyclictest23:08:260
17114992318,3cyclictest0-21swapper/022:28:200
17114992318,3cyclictest0-21swapper/022:28:200
17114992318,3cyclictest0-21swapper/022:03:170
17114992318,3cyclictest0-21swapper/020:13:170
17114992318,3cyclictest0-21swapper/020:13:170
17114992317,3cyclictest0-21swapper/023:38:160
17114992317,3cyclictest0-21swapper/023:38:160
17114992317,3cyclictest0-21swapper/023:38:160
17114992316,5cyclictest2223-21grep19:58:260
17114992316,5cyclictest2223-21grep19:58:260
1711499229,10cyclictest4759-21date20:08:140
17114992218,2cyclictest0-21swapper/023:03:230
17114992218,2cyclictest0-21swapper/023:03:230
17114992218,2cyclictest0-21swapper/021:33:220
17114992218,2cyclictest0-21swapper/021:33:220
17114992218,2cyclictest0-21swapper/021:33:220
17114992217,3cyclictest0-21swapper/022:03:260
17114992217,3cyclictest0-21swapper/022:03:260
17114992217,3cyclictest0-21swapper/022:03:260
17114992214,5cyclictest0-21swapper/022:18:170
17114992210,9cyclictest30078-21df00:08:170
17114992210,9cyclictest30078-21df00:08:170
17114992210,9cyclictest30078-21df00:08:170
17118992115,4cyclictest25232sleep322:53:223
17118992115,4cyclictest25232sleep322:53:223
17118992115,4cyclictest25232sleep322:53:223
1711499219,11cyclictest90-1pr/ttyS000:31:330
17114992114,5cyclictest8137-21unixbench_singl00:33:280
17114992114,5cyclictest8137-21unixbench_singl00:33:280
17114992114,5cyclictest8137-21unixbench_singl00:33:280
1711499211,11cyclictest25511-21df21:03:150
1711499211,11cyclictest25511-21df21:03:150
17114992014,4cyclictest60-21kcompactd020:03:230
17114992014,4cyclictest0-21swapper/022:13:250
17114992013,5cyclictest0-21swapper/023:33:240
17114992013,5cyclictest0-21swapper/023:33:240
17114992013,5cyclictest0-21swapper/023:33:240
17114992013,4cyclictest0-21swapper/023:43:180
17114992013,4cyclictest0-21swapper/023:43:180
17114992013,4cyclictest0-21swapper/023:43:180
17114992012,6cyclictest15495-21latency_hist22:03:100
1711499201,16cyclictest9547-21ntp_states00:38:240
1711499201,16cyclictest9547-21ntp_states00:38:240
1711499201,16cyclictest13035-21ntp_states20:28:220
1711499201,16cyclictest13035-21ntp_states20:28:220
173102180,3sleep30-21swapper/320:38:273
173102180,3sleep30-21swapper/320:38:273
17118991811,4cyclictest312772chrt22:43:213
17118991811,4cyclictest312772chrt22:43:213
1711699183,10cyclictest0-21swapper/200:03:212
1711699183,10cyclictest0-21swapper/200:03:212
1711699183,10cyclictest0-21swapper/200:03:212
1711899179,6cyclictest26924-21chrt22:33:163
1711899179,6cyclictest26924-21chrt22:33:163
1711899179,6cyclictest26924-21chrt22:33:163
1711899176,8cyclictest26309-21chrt22:29:343
1711899176,8cyclictest26309-21chrt22:29:343
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional