You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-24 - 04:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot5s.osadl.org (updated Mon Aug 24, 2026 00:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3991150995042,5cyclictest4024306-21auditctl20:45:012
3991152994839,5cyclictest4067368-21sh22:50:013
3991150994728,16cyclictest0-21swapper/219:50:002
3991150994722,21cyclictest4024766-21tune2fs20:45:172
399115099471,32cyclictest645-21runrttasks23:28:142
3991150994627,14cyclictest0-21swapper/220:00:002
399115099461,27cyclictest4000541-21/usr/sbin/munin19:35:142
3991150994518,23cyclictest0-21swapper/219:30:022
3991145994537,5cyclictest4069061-21cron22:55:001
3991152994436,4cyclictest4082252-21unixbench_multi23:30:273
3991152994434,6cyclictest4024296-21cron20:45:003
399115099441,39cyclictest4083495-21ntp_kernel_pll_23:35:212
399115099441,26cyclictest4031352-21/usr/sbin/munin21:05:242
399115099431,39cyclictest4098999-21/usr/sbin/munin00:20:172
399115099430,38cyclictest4089985-21apt-get23:55:142
3991145994336,4cyclictest4036456-21munin-plugin-st21:20:021
3991152994228,8cyclictest4068645-21grep22:50:233
399115099423,34cyclictest4056099-21meminfo22:15:192
399115099421,38cyclictest4071300-21/usr/sbin/munin23:00:172
399115099421,38cyclictest4028631-21/usr/sbin/munin20:55:242
399115099421,36cyclictest4100556-21/usr/sbin/munin00:25:172
399115099421,36cyclictest4069511-21/usr/sbin/munin22:55:192
3991141994233,6cyclictest4060069-21unixbench_multi22:25:240
3991152994129,8cyclictest4100205-21auditctl00:25:003
399115099412,36cyclictest4003533-21cron19:44:012
399115099411,37cyclictest4052874-21/usr/sbin/munin22:05:242
399115099411,37cyclictest4018628-21cron20:26:002
399115099411,35cyclictest3995388-21/usr/sbin/munin19:20:142
399115099411,34cyclictest4077850-21apt-get23:20:132
399115099411,34cyclictest4072722-21apt-get23:05:122
399115099410,35cyclictest4097015-21apt-get00:15:132
399115099410,35cyclictest0-21swapper/220:50:142
3991141994135,4cyclictest4060169-21kworker/u8:3+events_unbound22:32:190
3991141994111,5cyclictest131rcu_preempt23:20:190
3991150994031,6cyclictest645-21runrttasks21:25:192
3991150994023,14cyclictest4093183-21cut00:05:022
399115099401,36cyclictest4089717-21sshd23:54:432
399115099401,36cyclictest4063097-21/usr/sbin/munin22:35:222
399115099401,36cyclictest4011018-21/usr/sbin/munin20:05:182
399115099401,36cyclictest3992027-21/usr/sbin/munin19:10:232
399115099401,34cyclictest4040215-21df_inode21:30:152
399115099401,34cyclictest4019957-21/usr/sbin/munin20:30:232
399115099401,33cyclictest3998737-21apt-get19:30:142
3991145994033,4cyclictest4024296-21cron20:45:001
3991141994031,4cyclictest0-21swapper/019:15:020
3991152993930,4cyclictest0-21swapper/320:25:163
399115099392,34cyclictest4086065-21sshd23:41:432
399115099392,34cyclictest4086065-21sshd23:41:422
399115099391,35cyclictest4081307-21/usr/sbin/munin23:30:152
399115099391,35cyclictest4064288-21/usr/sbin/munin22:40:132
399115099391,35cyclictest4047469-21irqstats21:50:212
399115099391,33cyclictest4043573-21/usr/sbin/munin21:40:132
399115099391,33cyclictest4034935-21latency_hist21:15:012
399115099391,32cyclictest4065899-21apt-get22:45:122
3991145993932,4cyclictest4044998-21munin-plugin-st21:45:011
3991141993927,6cyclictest4020505-21kworker/u8:1+events_unbound20:36:540
3991141993913,14cyclictest11-21ksoftirqd/021:45:190
3991152993828,7cyclictest4057131-21sendmail22:20:023
399115099383,32cyclictest4058580-21cron22:22:002
399115099382,31cyclictest4076123-21apt-get23:15:132
399115099382,31cyclictest4076123-21apt-get23:15:122
399115099382,30cyclictest4086586-21apt-get23:45:122
399115099381,34cyclictest4104818-21/usr/sbin/munin00:35:262
399115099381,34cyclictest4104818-21/usr/sbin/munin00:35:262
399115099381,34cyclictest4094219-21/usr/sbin/munin00:05:232
399115099381,34cyclictest4043008-21cron21:37:002
399115099381,34cyclictest4014851-21/usr/sbin/munin20:15:262
399115099381,34cyclictest4013213-21/usr/sbin/munin20:10:232
399115099381,34cyclictest3994361-21/usr/sbin/munin19:15:232
399115099381,32cyclictest4053951-21apt-get22:10:132
3991150993811,23cyclictest4060772-21/usr/sbin/munin22:30:122
399115099380,33cyclictest4015801-21apt-get20:20:142
399115099380,33cyclictest4015801-21apt-get20:20:132
3991145993831,4cyclictest4001908-21latency_hist19:40:011
3991145993830,5cyclictest4008251-21grep19:55:281
3991145993811,13cyclictest24-21ksoftirqd/122:15:161
3991141993831,4cyclictest3995075-21cron19:20:010
3991141993826,9cyclictest121ktimers/022:00:120
3991141993819,11cyclictest131rcu_preempt22:20:230
3991152993724,8cyclictest4037694-21grep21:20:263
399115099372,32cyclictest4035497-21/usr/sbin/munin21:15:222
399115099371,33cyclictest4095658-21/usr/sbin/munin00:10:202
399115099371,33cyclictest4075264-21/usr/sbin/munin23:10:232
399115099371,33cyclictest4075264-21/usr/sbin/munin23:10:232
399115099371,33cyclictest4067732-21/usr/sbin/munin22:50:132
399115099371,33cyclictest4050183-21seq21:58:432
399115099371,33cyclictest4050183-21seq21:58:422
399115099371,33cyclictest4046531-21sshd21:45:432
399115099371,33cyclictest4029995-21/usr/sbin/munin21:00:172
3991145993729,4cyclictest0-21swapper/120:20:001
3991145993728,4cyclictest0-21swapper/100:30:001
3991145993726,7cyclictest0-21swapper/123:15:241
3991145993726,7cyclictest0-21swapper/123:15:241
3991145993726,7cyclictest0-21swapper/120:35:011
3991141993726,7cyclictest0-21swapper/023:15:010
3991141993726,7cyclictest0-21swapper/023:15:000
3991141993726,6cyclictest0-21swapper/021:25:260
3991141993718,3cyclictest131rcu_preempt20:52:000
3991141993713,7cyclictest0-21swapper/022:20:020
399115099362,31cyclictest4009422-21meminfo20:00:182
399115099361,32cyclictest4051152-21/usr/sbin/munin22:00:222
399115099361,32cyclictest4036935-21/usr/sbin/munin21:20:172
399115099361,32cyclictest4021788-21/usr/sbin/munin20:35:242
399115099361,32cyclictest4005669-21df_inode19:50:152
399115099361,30cyclictest4102251-21df_inode00:30:152
3991145993629,4cyclictest4070784-21idleruntime-cro23:00:001
3991145993626,6cyclictest0-21swapper/123:00:271
3991145993625,7cyclictest4084423-21apt-get23:40:001
399114599362,10cyclictest4038622-21fschecks_time21:25:171
3991141993630,4cyclictest4086132-21kworker/u8:1+events_unbound23:50:420
3991141993628,5cyclictest4102328-21fschecks_time00:30:160
3991141993627,6cyclictest0-21swapper/023:05:000
3991141993626,6cyclictest0-21swapper/000:15:260
3991152993527,4cyclictest0-21swapper/321:00:003
3991152993524,7cyclictest4003158-21grep19:40:273
3991145993527,4cyclictest0-21swapper/123:20:271
3991145993526,5cyclictest0-21swapper/119:15:011
399114199357,15cyclictest121ktimers/023:30:000
3991141993525,7cyclictest53-21kcompactd023:15:140
3991141993525,7cyclictest53-21kcompactd023:15:140
3991141993524,7cyclictest4099786-21grep00:20:260
3991141993523,3cyclictest131rcu_preempt00:05:010
3991141993522,5cyclictest131rcu_preempt23:40:040
3991141993522,5cyclictest131rcu_preempt23:40:040
3991141993516,16cyclictest121ktimers/020:00:160
3991152993425,5cyclictest0-21swapper/323:25:143
3991152993423,7cyclictest0-21swapper/300:15:003
3991145993424,6cyclictest0-21swapper/100:15:011
3991141993426,4cyclictest0-21swapper/019:36:530
3991141993425,6cyclictest4034316-21grep21:10:270
3991141993424,6cyclictest0-21swapper/021:40:000
3991141993424,6cyclictest0-21swapper/020:25:160
3991141993414,10cyclictest131rcu_preempt19:20:140
3991141993413,13cyclictest131rcu_preempt23:45:200
3991141993412,14cyclictest131rcu_preempt00:30:000
399114199340,9cyclictest11-21ksoftirqd/021:00:010
3991152993325,6cyclictest391ktimers/320:30:133
3991145993325,4cyclictest0-21swapper/119:25:271
3991141993324,6cyclictest0-21swapper/022:45:000
3991141993314,11cyclictest131rcu_preempt23:55:220
3991152993223,5cyclictest0-21swapper/321:00:163
3991152993222,5cyclictest0-21swapper/319:50:003
399114199326,13cyclictest11-21ksoftirqd/022:15:000
3991141993223,6cyclictest0-21swapper/021:30:250
3991141993221,4cyclictest131rcu_preempt00:10:150
3991141993216,5cyclictest131rcu_preempt23:05:250
3991141993215,5cyclictest131rcu_preempt19:50:160
3991141993215,4cyclictest131rcu_preempt21:20:170
3991141993212,11cyclictest40700322mailstats22:55:250
3991141993210,4cyclictest0-21swapper/020:45:160
3991141993210,13cyclictest11-21ksoftirqd/022:55:010
3991152993123,4cyclictest0-21swapper/321:25:273
3991152993122,4cyclictest0-21swapper/320:50:173
3991152993121,5cyclictest0-21swapper/321:10:003
3991152993120,6cyclictest0-21swapper/321:50:303
3991152993119,7cyclictest0-21swapper/319:10:263
399114599318,6cyclictest0-21swapper/123:55:271
399114599317,6cyclictest24-21ksoftirqd/121:15:011
3991145993121,6cyclictest0-21swapper/120:50:281
3991145993121,6cyclictest0-21swapper/120:10:151
3991145993121,6cyclictest0-21swapper/119:45:261
399114199315,7cyclictest11-21ksoftirqd/019:25:250
399114199315,6cyclictest11-21ksoftirqd/023:35:190
399114199313,15cyclictest11-21ksoftirqd/019:45:190
3991141993123,5cyclictest151rcuc/000:39:040
3991141993123,5cyclictest151rcuc/000:39:030
3991141993122,6cyclictest0-21swapper/021:05:150
3991141993121,7cyclictest151rcuc/022:07:010
3991141993120,5cyclictest131rcu_preempt19:40:160
3991141993120,4cyclictest131rcu_preempt00:05:010
3991141993119,4cyclictest131rcu_preempt20:10:170
3991141993115,9cyclictest131rcu_preempt21:00:130
3991141993112,11cyclictest131rcu_preempt22:35:160
3991141993111,12cyclictest131rcu_preempt20:25:000
3991141993111,12cyclictest131rcu_preempt20:25:000
3991141993111,11cyclictest131rcu_preempt22:45:150
3991152993023,4cyclictest0-21swapper/321:35:163
3991152993023,3cyclictest40975472chrt00:15:213
3991152993022,4cyclictest0-21swapper/321:55:253
3991152993022,4cyclictest0-21swapper/321:55:253
3991152993019,5cyclictest0-21swapper/320:00:003
3991145993021,4cyclictest0-21swapper/100:05:011
3991145993020,6cyclictest0-21swapper/123:05:161
3991145993020,6cyclictest0-21swapper/120:55:261
3991145993020,6cyclictest0-21swapper/119:20:251
3991145993020,6cyclictest0-21swapper/100:20:261
3991145993020,3cyclictest131rcu_preempt21:32:331
3991145993016,4cyclictest131rcu_preempt21:47:261
399114199305,5cyclictest11-21ksoftirqd/020:00:010
3991141993014,9cyclictest131rcu_preempt19:30:250
3991141993013,5cyclictest131rcu_preempt21:55:000
3991141993013,10cyclictest131rcu_preempt21:58:280
3991141993013,10cyclictest131rcu_preempt21:58:280
399114199301,16cyclictest11-21ksoftirqd/020:15:240
3991141993010,9cyclictest131rcu_preempt20:40:210
3991152992921,6cyclictest4006886-21kworker/u8:2+events_unbound20:01:543
3991152992921,5cyclictest4103605-21sh00:35:003
399114599298,7cyclictest0-21swapper/122:35:241
399114599297,5cyclictest0-21swapper/120:05:141
399114599295,7cyclictest0-21swapper/120:25:181
399114599295,6cyclictest0-21swapper/100:15:261
3991145992922,4cyclictest0-21swapper/100:35:261
3991145992922,4cyclictest0-21swapper/100:35:261
3991145992920,4cyclictest0-21swapper/120:20:171
3991145992920,4cyclictest0-21swapper/120:20:161
3991145992919,6cyclictest0-21swapper/122:30:001
3991145992919,6cyclictest0-21swapper/100:35:001
3991145992918,7cyclictest0-21swapper/122:00:151
399114199298,3cyclictest0-21swapper/020:30:190
3991141992912,10cyclictest131rcu_preempt23:30:240
3991141992911,11cyclictest131rcu_preempt20:05:230
399114199290,4cyclictest11-21ksoftirqd/021:40:170
3991152992819,5cyclictest0-21swapper/323:55:263
3991152992817,8cyclictest4016842-21unixbench_singl20:20:263
3991152992817,8cyclictest4016842-21unixbench_singl20:20:263
399115299281,22cyclictest4094615-21tail00:05:263
399114599288,4cyclictest231ktimers/123:54:011
399114599285,11cyclictest0-21swapper/119:33:101
3991145992820,4cyclictest0-21swapper/123:30:271
3991145992819,5cyclictest4081015-21sh23:30:001
3991141992813,4cyclictest131rcu_preempt21:17:020
3991152992720,3cyclictest0-21swapper/322:35:253
3991152992719,5cyclictest0-21swapper/300:40:003
3991152992719,5cyclictest0-21swapper/300:40:003
3991152992718,5cyclictest0-21swapper/320:40:003
3991145992718,6cyclictest4066033-21fschecks_count22:45:171
3991145992717,6cyclictest0-21swapper/119:50:251
3991145992716,7cyclictest0-21swapper/120:35:261
3991145992715,9cyclictest4042248-21/usr/sbin/munin21:35:201
3991145992713,7cyclictest131rcu_preempt21:58:561
3991145992713,7cyclictest131rcu_preempt21:58:561
3991145992711,3cyclictest131rcu_preempt22:30:491
3991145992710,8cyclictest131rcu_preempt00:07:281
3991152992610,11cyclictest204-21dbus-daemon23:07:233
399114599267,6cyclictest0-21swapper/120:45:151
399114599267,10cyclictest131rcu_preempt22:11:181
3991145992619,5cyclictest231ktimers/121:05:121
3991145992617,6cyclictest0-21swapper/123:40:241
3991145992617,6cyclictest0-21swapper/123:40:241
3991145992617,5cyclictest0-21swapper/123:45:161
3991145992616,7cyclictest4064768-21ntp_states22:40:211
3991152992517,5cyclictest391ktimers/321:15:013
3991152992517,5cyclictest0-21swapper/320:15:283
3991152992517,4cyclictest0-21swapper/322:40:163
3991152992516,6cyclictest4053108-21sort22:05:263
399115299251,5cyclictest4087588-21unixbench_multi23:45:243
399114599259,9cyclictest131rcu_preempt22:05:511
399114599252,15cyclictest0-21swapper/119:43:401
3991145992517,6cyclictest231ktimers/122:21:211
3991145992515,6cyclictest0-21swapper/120:05:011
3991145992511,8cyclictest131rcu_preempt21:20:151
3991145992511,7cyclictest131rcu_preempt21:00:281
399114599251,15cyclictest0-21swapper/119:18:451
3991152992417,4cyclictest0-21swapper/322:25:153
3991152992416,6cyclictest3994780-21kworker/u8:1+events_unbound19:26:403
3991152992416,5cyclictest0-21swapper/321:40:243
3991152992416,4cyclictest0-21swapper/300:05:013
3991152992416,3cyclictest0-21swapper/322:10:253
3991152992413,7cyclictest0-21swapper/322:00:253
3991145992410,7cyclictest131rcu_preempt23:10:071
3991145992410,7cyclictest131rcu_preempt23:10:071
3991152992316,3cyclictest0-21swapper/300:25:273
399115299231,4cyclictest4041150-21sed21:30:263
3991152992311,8cyclictest204-21dbus-daemon23:12:273
3991152992311,8cyclictest204-21dbus-daemon23:12:263
3991152992311,8cyclictest204-21dbus-daemon22:32:173
3991145992314,6cyclictest0-21swapper/121:50:281
3991152992215,4cyclictest4085866-21unixbench_multi23:40:253
3991152992215,4cyclictest4085866-21unixbench_multi23:40:253
3991152992214,4cyclictest0-21swapper/320:05:163
3991152992213,6cyclictest4035113-21grep21:15:163
3991152992213,4cyclictest4084455-21sendmail-msp23:40:013
3991152992211,8cyclictest4025351-21sed20:45:243
3991152992211,8cyclictest3994178-21ntp_states19:15:213
399115299215,12cyclictest4046890-21cron21:50:003
399115299212,14cyclictest3999723-21tail19:30:273
3991152992115,3cyclictest0-21swapper/322:20:213
3991152992114,3cyclictest0-21swapper/319:55:003
399115299211,17cyclictest3995337-21taskset19:20:133
3991152991910,5cyclictest0-21swapper/322:55:173
3991152991812,4cyclictest4075679-21kworker/u8:0+events_unbound23:15:123
3991152991812,4cyclictest4075679-21kworker/u8:0+events_unbound23:15:123
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional