You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-23 - 14:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Thu Apr 23, 2026 12:44:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3017621060,3sleep00-21swapper/009:22:190
3017621060,3sleep00-21swapper/009:22:190
2209421010,8sleep0171ktimers/007:32:290
2209421010,8sleep0171ktimers/007:32:290
11552870,3sleep00-21swapper/009:32:160
11552870,3sleep00-21swapper/009:32:160
11552870,3sleep00-21swapper/009:32:160
224912830,1sleep022494-21fschecks_count09:02:170
224912830,1sleep022494-21fschecks_count09:02:170
200332620,2sleep020036-21grep10:22:170
200332620,2sleep020036-21grep10:22:170
200332620,2sleep020036-21grep10:22:170
256082580,2chrt25609-21which07:42:260
256082580,2chrt25609-21which07:42:260
256082580,2chrt25609-21which07:42:260
198372580,2sleep019840-21head08:52:290
198372580,2sleep019840-21head08:52:290
224112550,3chrt0-21swapper/311:52:293
224112550,3chrt0-21swapper/311:52:293
51062530,2chrt5107-21fschecks_count09:42:181
51062530,2chrt5107-21fschecks_count09:42:181
4952500,3sleep30-21swapper/308:02:233
4952500,3sleep30-21swapper/308:02:233
201542500,1sleep00-21swapper/007:27:260
201542500,1sleep00-21swapper/007:27:260
118692460,3chrt0-21swapper/311:27:163
118692460,3chrt0-21swapper/311:27:163
105432440,4sleep30-21swapper/309:57:163
105432440,4sleep30-21swapper/309:57:163
69832420,3sleep20-21swapper/208:19:542
69832420,3sleep20-21swapper/208:19:542
73712390,2sleep20-21swapper/211:12:282
73712390,2sleep20-21swapper/211:12:282
13883993830,6cyclictest25141-21fschecks_time12:02:190
13883993830,6cyclictest25141-21fschecks_time12:02:190
13883993830,6cyclictest25141-21fschecks_time12:02:190
13883993628,6cyclictest6522-21threads08:17:290
13883993628,6cyclictest6522-21threads08:17:290
60252350,3sleep30-21swapper/308:17:223
60252350,3sleep30-21swapper/308:17:223
299972350,3sleep30-21swapper/309:22:183
299972350,3sleep30-21swapper/309:22:183
23762350,3sleep30-21swapper/308:07:213
23762350,3sleep30-21swapper/308:07:213
283692340,1sleep028393-21latency_hist07:52:080
283692340,1sleep028393-21latency_hist07:52:080
1388699341,3cyclictest0-21swapper/309:55:053
1388699341,3cyclictest0-21swapper/309:55:053
13884993430,3cyclictest3431-21latency_hist08:12:051
13884993430,3cyclictest3431-21latency_hist08:12:051
13884993428,4cyclictest25246-21gltestperf12:02:191
13884993428,4cyclictest25246-21gltestperf12:02:191
13884993428,4cyclictest25246-21gltestperf12:02:191
13883993427,5cyclictest17664-21cut11:42:180
13883993427,5cyclictest17664-21cut11:42:180
13883993427,5cyclictest13881-21cyclictest09:47:270
13883993427,5cyclictest13881-21cyclictest09:47:270
13883993426,6cyclictest21116-21cpu11:52:160
13883993426,6cyclictest21116-21cpu11:52:160
1388699331,3cyclictest0-21swapper/312:38:493
1388699331,3cyclictest0-21swapper/312:38:493
1388699331,3cyclictest0-21swapper/311:47:533
1388699331,3cyclictest0-21swapper/311:47:533
1388699331,3cyclictest0-21swapper/311:10:053
1388699331,3cyclictest0-21swapper/311:10:053
1388699331,3cyclictest0-21swapper/310:56:183
1388699331,3cyclictest0-21swapper/309:51:473
1388699331,3cyclictest0-21swapper/309:51:473
1388699331,3cyclictest0-21swapper/309:37:003
1388699331,3cyclictest0-21swapper/309:37:003
1388699331,3cyclictest0-21swapper/309:14:503
1388699331,3cyclictest0-21swapper/309:14:503
13883993328,3cyclictest90-1pr/ttyS009:15:440
13883993328,3cyclictest90-1pr/ttyS009:15:440
16462320,3sleep013881-21cyclictest09:32:200
16462320,3sleep013881-21cyclictest09:32:200
1388699321,3cyclictest0-21swapper/310:34:313
1388699321,3cyclictest0-21swapper/310:34:313
1388699321,3cyclictest0-21swapper/308:15:243
13884993225,5cyclictest24247-21missed_timers10:32:211
13884993225,5cyclictest24247-21missed_timers10:32:211
13884993225,5cyclictest0-21swapper/111:22:061
13884993225,5cyclictest0-21swapper/111:22:061
13884993223,6cyclictest28648-21netstat09:17:241
13884993223,6cyclictest28648-21netstat09:17:241
13884993223,6cyclictest28648-21netstat09:17:241
13884993222,8cyclictest14053-21cron10:07:051
13884993222,8cyclictest14053-21cron10:07:051
13883993226,4cyclictest22120-21systemctl09:02:060
13883993226,4cyclictest22120-21systemctl09:02:060
222812310,2sleep122351-21missed_timers10:27:221
222812310,2sleep122351-21missed_timers10:27:221
1388699311,3cyclictest0-21swapper/312:12:353
1388699311,3cyclictest0-21swapper/312:12:353
1388699311,3cyclictest0-21swapper/311:33:033
1388699311,3cyclictest0-21swapper/311:33:033
1388699311,3cyclictest0-21swapper/310:59:053
1388699311,3cyclictest0-21swapper/310:59:053
1388699311,3cyclictest0-21swapper/310:44:053
1388699311,3cyclictest0-21swapper/310:44:053
1388699311,3cyclictest0-21swapper/309:41:023
1388699311,3cyclictest0-21swapper/309:41:023
1388699311,3cyclictest0-21swapper/309:28:443
1388699311,3cyclictest0-21swapper/309:28:443
1388699311,3cyclictest0-21swapper/309:28:443
1388699311,3cyclictest0-21swapper/308:45:163
1388699311,3cyclictest0-21swapper/308:45:163
1388699311,3cyclictest0-21swapper/308:45:163
1388699311,2cyclictest0-21swapper/311:32:033
1388699311,2cyclictest0-21swapper/311:32:033
1388699311,2cyclictest0-21swapper/311:32:033
1388699311,2cyclictest0-21swapper/310:14:013
1388699311,2cyclictest0-21swapper/308:48:583
1388699311,2cyclictest0-21swapper/308:48:583
13884993122,7cyclictest30188-21switchtime10:47:291
13884993122,7cyclictest30188-21switchtime10:47:291
13884993122,7cyclictest30188-21switchtime10:47:291
13883993125,4cyclictest1648-21ntpd07:22:240
13883993124,5cyclictest25689-21fschecks_time10:37:170
13883993124,5cyclictest25689-21fschecks_time10:37:170
13883993122,7cyclictest4363-21missed_timers12:32:220
1388699301,3cyclictest0-21swapper/312:18:533
1388699301,3cyclictest0-21swapper/312:18:533
1388699301,3cyclictest0-21swapper/312:11:053
1388699301,3cyclictest0-21swapper/312:11:053
1388699301,3cyclictest0-21swapper/311:59:543
1388699301,3cyclictest0-21swapper/311:59:543
1388699301,3cyclictest0-21swapper/311:42:053
1388699301,3cyclictest0-21swapper/311:42:053
1388699301,3cyclictest0-21swapper/311:42:053
1388699301,3cyclictest0-21swapper/311:18:343
1388699301,3cyclictest0-21swapper/311:18:343
1388699301,3cyclictest0-21swapper/310:39:323
1388699301,3cyclictest0-21swapper/310:39:323
1388699301,3cyclictest0-21swapper/310:20:063
1388699301,3cyclictest0-21swapper/310:20:063
1388699301,3cyclictest0-21swapper/310:20:063
1388699301,3cyclictest0-21swapper/310:12:013
1388699301,3cyclictest0-21swapper/310:12:013
1388699301,3cyclictest0-21swapper/310:02:313
1388699301,3cyclictest0-21swapper/310:02:313
1388699301,3cyclictest0-21swapper/309:08:473
1388699301,3cyclictest0-21swapper/309:08:473
1388699301,3cyclictest0-21swapper/309:02:043
1388699301,3cyclictest0-21swapper/309:02:043
1388699301,3cyclictest0-21swapper/308:41:293
1388699301,3cyclictest0-21swapper/308:41:293
1388699301,3cyclictest0-21swapper/308:33:163
1388699301,3cyclictest0-21swapper/308:33:163
1388699301,3cyclictest0-21swapper/307:59:053
1388699301,3cyclictest0-21swapper/307:59:053
1388699301,3cyclictest0-21swapper/307:57:123
1388699301,3cyclictest0-21swapper/307:43:563
1388699301,3cyclictest0-21swapper/307:43:563
1388699301,3cyclictest0-21swapper/307:43:563
1388699301,3cyclictest0-21swapper/307:40:053
1388699301,3cyclictest0-21swapper/307:40:053
1388699301,3cyclictest0-21swapper/307:30:003
1388699301,3cyclictest0-21swapper/307:30:003
1388699301,2cyclictest0-21swapper/307:33:573
1388699301,2cyclictest0-21swapper/307:33:573
1388699301,13cyclictest23611-21chrt09:02:283
1388699301,13cyclictest23611-21chrt09:02:283
13884993026,2cyclictest0-21swapper/112:42:051
13884993026,2cyclictest0-21swapper/112:42:051
13884993024,5cyclictest2672-21processes12:27:271
13884993024,5cyclictest2672-21processes12:27:271
13884993024,5cyclictest2672-21processes12:27:271
13884993022,6cyclictest550-21cron09:57:141
13884993022,6cyclictest550-21cron09:57:141
13884993022,6cyclictest28263-21cut10:42:261
13884993022,6cyclictest28263-21cut10:42:261
13884993022,6cyclictest14685-21threads11:32:261
13884993022,6cyclictest14685-21threads11:32:261
13883993026,3cyclictest90-1pr/ttyS011:10:050
13883993026,3cyclictest90-1pr/ttyS011:10:050
13883993025,3cyclictest90-1pr/ttyS011:19:250
13883993025,3cyclictest90-1pr/ttyS011:19:250
98422290,2sleep20-21swapper/209:52:282
98422290,2sleep20-21swapper/209:52:282
1388699291,3cyclictest0-21swapper/312:28:493
1388699291,3cyclictest0-21swapper/312:28:493
1388699291,3cyclictest0-21swapper/312:28:493
1388699291,3cyclictest0-21swapper/311:43:213
1388699291,3cyclictest0-21swapper/311:43:213
1388699291,3cyclictest0-21swapper/311:12:513
1388699291,3cyclictest0-21swapper/311:12:513
1388699291,3cyclictest0-21swapper/310:31:013
1388699291,3cyclictest0-21swapper/310:31:013
1388699291,3cyclictest0-21swapper/309:18:523
1388699291,3cyclictest0-21swapper/309:18:523
1388699291,3cyclictest0-21swapper/309:18:523
1388699291,3cyclictest0-21swapper/308:26:023
1388699291,3cyclictest0-21swapper/308:26:023
1388699291,3cyclictest0-21swapper/307:48:573
1388699291,3cyclictest0-21swapper/307:48:573
1388699291,2cyclictest0-21swapper/309:44:343
1388699291,2cyclictest0-21swapper/309:44:343
13884992922,5cyclictest6318-21processes08:17:271
13884992922,5cyclictest6318-21processes08:17:271
13883992922,6cyclictest3381-21sshd08:11:350
13883992922,6cyclictest3381-21sshd08:11:350
13883992922,6cyclictest29037-21kernelversion07:52:200
13883992920,7cyclictest3308-21fschecks_time09:37:180
13883992920,7cyclictest3308-21fschecks_time09:37:180
13883992920,7cyclictest10648-21netstat11:22:240
13883992920,7cyclictest10648-21netstat11:22:240
13883992912,9cyclictest90-1pr/ttyS009:10:000
13883992912,9cyclictest90-1pr/ttyS009:10:000
1388699281,3cyclictest0-21swapper/312:33:553
1388699281,3cyclictest0-21swapper/311:02:353
1388699281,3cyclictest0-21swapper/311:02:353
1388699281,3cyclictest0-21swapper/308:56:063
1388699281,3cyclictest0-21swapper/308:56:063
1388699281,3cyclictest0-21swapper/308:29:133
13884992822,5cyclictest21198-21ntp_offset08:57:241
13884992822,5cyclictest21198-21ntp_offset08:57:241
13884992821,5cyclictest6719-21latency11:12:201
13884992821,5cyclictest6719-21latency11:12:201
13884992812,3cyclictest0-21swapper/108:32:251
13884992812,3cyclictest0-21swapper/108:32:251
13883992824,2cyclictest201rcuc/010:06:010
13883992824,2cyclictest201rcuc/010:06:010
13883992822,3cyclictest15919-21latency_hist10:12:150
13883992822,3cyclictest15919-21latency_hist10:12:150
13883992822,3cyclictest0-21swapper/010:57:240
13883992822,3cyclictest0-21swapper/010:57:240
13883992821,5cyclictest23433-21kernelversion07:37:200
13883992821,5cyclictest23433-21kernelversion07:37:200
13883992819,7cyclictest28392-21latency_hist12:12:070
13883992819,7cyclictest28392-21latency_hist12:12:070
13883992817,3cyclictest0-21swapper/008:42:260
13883992817,3cyclictest0-21swapper/008:42:260
13883992817,3cyclictest0-21swapper/008:42:260
315562270,3sleep20-21swapper/212:17:282
315562270,3sleep20-21swapper/212:17:282
1388699271,3cyclictest0-21swapper/310:25:053
1388699271,3cyclictest0-21swapper/310:25:053
1388699271,3cyclictest0-21swapper/310:25:053
1388699271,2cyclictest0-21swapper/312:05:493
1388699271,2cyclictest0-21swapper/312:05:493
1388699271,2cyclictest0-21swapper/312:05:493
1388699271,2cyclictest0-21swapper/307:25:133
13884992722,3cyclictest0-21swapper/111:42:061
13884992722,3cyclictest0-21swapper/111:42:061
13884992722,3cyclictest0-21swapper/111:42:061
13884992721,5cyclictest8137-21ntp_offset08:22:241
13884992721,5cyclictest8137-21ntp_offset08:22:241
13884992721,5cyclictest16880-21ntp_offset10:12:231
13884992720,5cyclictest22443-21sshd07:33:351
13884992720,5cyclictest22443-21sshd07:33:351
13884992717,8cyclictest656-21processes08:02:261
13884992717,8cyclictest656-21processes08:02:261
13883992722,4cyclictest572-21cups-browsed09:52:260
13883992722,4cyclictest572-21cups-browsed09:52:260
13883992721,4cyclictest30181-21switchtime10:47:290
13883992721,4cyclictest30181-21switchtime10:47:290
13883992721,4cyclictest30181-21switchtime10:47:290
13883992717,7cyclictest14020-21ls08:37:260
13883992717,7cyclictest14020-21ls08:37:260
1388699265,17cyclictest0-21swapper/310:50:223
1388699265,17cyclictest0-21swapper/310:50:223
1388699265,17cyclictest0-21swapper/310:50:223
13884992622,2cyclictest0-21swapper/108:42:181
13884992622,2cyclictest0-21swapper/108:42:181
13884992622,2cyclictest0-21swapper/108:42:181
13884992621,4cyclictest3828-21fschecks_count08:12:181
13884992621,4cyclictest18145-21ntp_kernel_pll_11:42:241
13884992621,4cyclictest18145-21ntp_kernel_pll_11:42:241
13884992621,3cyclictest30257-21latency_hist12:17:051
13884992621,3cyclictest30257-21latency_hist12:17:051
13884992620,4cyclictest24585-21latency_hist07:42:051
13884992620,4cyclictest24585-21latency_hist07:42:051
13884992617,7cyclictest21393-21fschecks_time11:52:181
13883992620,5cyclictest12805-21latency_hist08:37:070
13883992620,5cyclictest12805-21latency_hist08:37:070
13883992619,5cyclictest10526-21/usr/sbin/munin09:57:230
13883992619,5cyclictest10526-21/usr/sbin/munin09:57:230
13883992618,6cyclictest201rcuc/011:33:330
13883992618,6cyclictest201rcuc/011:33:330
13883992618,6cyclictest16527-21ls11:37:280
13883992618,6cyclictest16527-21ls11:37:280
13883992618,6cyclictest16527-21ls11:37:280
1388399261,7cyclictest28802xargs12:27:280
1388399261,7cyclictest28802xargs12:27:280
1388399261,7cyclictest28802xargs12:27:280
1388399261,10cyclictest75132latency_hist12:42:160
13883992610,8cyclictest90-1pr/ttyS012:14:210
13883992610,8cyclictest90-1pr/ttyS012:14:210
1388699251,2cyclictest0-21swapper/312:23:503
1388699251,2cyclictest0-21swapper/312:23:503
13884992520,2cyclictest0-21swapper/107:47:261
13884992520,2cyclictest0-21swapper/107:47:261
13884992519,4cyclictest0-21swapper/109:12:061
13884992519,4cyclictest0-21swapper/109:12:061
13884992519,3cyclictest0-21swapper/108:47:281
13884992519,3cyclictest0-21swapper/108:47:281
13884992516,7cyclictest13227-21cut08:37:171
13884992516,7cyclictest13227-21cut08:37:171
1388399257,4cyclictest90-1pr/ttyS008:02:150
13883992520,3cyclictest90-1pr/ttyS009:17:210
13883992520,3cyclictest90-1pr/ttyS009:17:210
13883992520,3cyclictest90-1pr/ttyS009:17:210
13883992519,4cyclictest1346-21avahi-daemon08:47:270
13883992519,4cyclictest1346-21avahi-daemon08:47:270
13883992517,6cyclictest24707-21ls10:32:270
13883992517,6cyclictest24707-21ls10:32:270
13883992512,2cyclictest181rcu_preempt08:27:020
13883992512,2cyclictest181rcu_preempt08:27:020
13884992420,2cyclictest0-21swapper/112:02:061
13884992420,2cyclictest0-21swapper/112:02:061
13884992420,2cyclictest0-21swapper/108:52:281
13884992420,2cyclictest0-21swapper/108:52:281
13884992420,2cyclictest0-21swapper/107:22:181
13884992419,3cyclictest27734-21latency_hist09:17:051
13884992419,3cyclictest27734-21latency_hist09:17:051
13884992419,3cyclictest13881-21cyclictest07:45:001
13884992419,3cyclictest13881-21cyclictest07:45:001
13884992419,3cyclictest13881-21cyclictest07:45:001
13884992419,3cyclictest0-21swapper/112:22:171
13884992419,3cyclictest0-21swapper/112:22:171
13884992418,4cyclictest16092-21latency_hist10:12:161
13884992418,4cyclictest16092-21latency_hist10:12:161
13884992416,6cyclictest12481-21needreboot11:27:221
13884992416,6cyclictest12481-21needreboot11:27:221
13884992416,6cyclictest12481-21needreboot11:27:221
13883992420,2cyclictest0-21swapper/011:57:250
13883992420,2cyclictest0-21swapper/011:57:250
13883992420,2cyclictest0-21swapper/008:02:060
13883992419,4cyclictest735-21systemd-logind11:02:260
13883992419,4cyclictest735-21systemd-logind11:02:260
13883992418,3cyclictest0-21swapper/009:42:270
13883992418,3cyclictest0-21swapper/009:42:270
13883992416,6cyclictest3908-21fschecks_time08:12:180
13883992416,2cyclictest181rcu_preempt10:57:090
13883992413,3cyclictest181rcu_preempt12:17:270
13883992413,3cyclictest181rcu_preempt12:17:270
13884992319,2cyclictest0-21swapper/110:52:251
13884992319,2cyclictest0-21swapper/109:37:051
13884992319,2cyclictest0-21swapper/109:37:051
13884992318,4cyclictest30651-21fschecks_count12:17:191
13884992318,4cyclictest30651-21fschecks_count12:17:191
13884992318,3cyclictest0-21swapper/111:07:211
13884992318,3cyclictest0-21swapper/111:07:211
13884992318,2cyclictest0-21swapper/107:27:261
13884992318,2cyclictest0-21swapper/107:27:261
13884992317,3cyclictest0-21swapper/107:57:191
13884992317,3cyclictest0-21swapper/107:57:191
13884992315,6cyclictest22893-21ls11:57:091
13884992315,6cyclictest22893-21ls11:57:091
13884992314,7cyclictest550-21cron09:52:051
13884992314,7cyclictest550-21cron09:52:051
13884992313,8cyclictest30012-21fschecks_count09:22:181
13884992313,8cyclictest30012-21fschecks_count09:22:181
13883992319,2cyclictest0-21swapper/011:32:170
13883992319,2cyclictest0-21swapper/011:32:170
13883992319,2cyclictest0-21swapper/011:32:170
13883992318,3cyclictest0-21swapper/011:52:260
13883992318,3cyclictest0-21swapper/011:52:260
13883992316,5cyclictest0-21swapper/011:12:180
13883992316,5cyclictest0-21swapper/011:12:180
13883992314,7cyclictest9539-21fschecks_time08:27:180
13883992314,7cyclictest27994-21ls10:42:210
13883992314,7cyclictest27994-21ls10:42:210
13884992218,3cyclictest28694-21latency_hist12:12:161
13884992218,3cyclictest28694-21latency_hist12:12:161
13884992218,2cyclictest0-21swapper/110:57:191
13884992218,2cyclictest0-21swapper/110:57:191
13884992218,2cyclictest0-21swapper/110:22:181
13884992218,2cyclictest0-21swapper/110:22:181
13884992218,2cyclictest0-21swapper/110:22:181
13884992217,3cyclictest0-21swapper/111:02:261
13884992217,3cyclictest0-21swapper/111:02:261
13884992217,3cyclictest0-21swapper/109:27:181
13884992217,3cyclictest0-21swapper/109:27:181
13884992217,3cyclictest0-21swapper/109:27:181
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional