You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-23 - 13:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Wed Oct 22, 2025 12:59:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
159639911791157,18cyclictest17824-21munin-run07:30:001
156782561517,29sleep00-21swapper/007:22:420
158842465419,31sleep10-21swapper/107:24:491
378821060,2sleep00-21swapper/009:55:200
111172850,3sleep10-21swapper/112:16:441
41812810,5sleep04178-21ssh11:35:440
99412800,8sleep01596299cyclictest10:01:190
300762790,4sleep1211ktimersoftd/110:22:331
228282650,22sleep03-21ksoftirqd/011:21:520
147122620,8sleep00-21swapper/011:47:010
57222600,7sleep11596399cyclictest11:05:011
1596399601,21cyclictest10466-21ssh11:09:571
15962996054,4cyclictest17918-21munin-node07:30:000
15963995810,15cyclictest3904-21ssh11:35:311
1596399579,15cyclictest14777-21ssh09:33:251
1596399579,15cyclictest14777-21ssh09:33:251
1596399571,20cyclictest23833-21users10:15:311
1596399571,13cyclictest3020-21runrttasks12:25:361
1596399569,15cyclictest2291-21ssh10:27:181
1596399561,24cyclictest25334-21ssh09:10:281
1596399561,15cyclictest1620-21aten_r4power_cu09:20:131
1596399561,15cyclictest13741-21sh10:39:481
1596399557,16cyclictest10893-21ssh09:29:461
1596399552,22cyclictest641-21ssh09:52:001
1596399552,17cyclictest23763-21ssh10:50:071
1596399552,16cyclictest25193-21ssh09:44:311
1596399551,23cyclictest30220-21ssh10:55:411
1596399551,23cyclictest21091-21ssh10:46:261
1596399551,22cyclictest5519-21ssh12:10:361
1596399551,22cyclictest28946-21ssh11:28:451
1596399551,17cyclictest526-21ssh09:18:551
1596399551,17cyclictest31690-21ssh11:30:441
1596399542,21cyclictest11941-21ssh12:51:161
1596399541,23cyclictest4508-21ssh10:30:111
1596399541,22cyclictest25036-21ssh11:58:021
1596399541,22cyclictest18302-21ssh11:17:181
1596399541,22cyclictest18302-21ssh11:17:171
1596399541,22cyclictest17014-21ssh12:23:161
1596399541,21cyclictest21135-21ssh10:13:351
1596399541,17cyclictest3227-21ssh12:42:091
1596399541,17cyclictest24899-21ssh12:30:401
1596399538,14cyclictest12717-21taskset11:11:121
1596399532,14cyclictest24058-21diskmemload10:05:351
1596399531,21cyclictest5073-21ssh09:56:161
1596399531,21cyclictest27519-21ssh09:45:521
1596399531,20cyclictest21550-21ssh11:54:581
1596399531,17cyclictest30315-21ssh12:03:401
171682520,8sleep11596399cyclictest10:42:441
1596399522,17cyclictest1926-21chrt12:07:331
1596399522,16cyclictest13750-21ntp_states08:40:271
1596399521,18cyclictest18940-21grep08:55:191
1596399512,22cyclictest0-21swapper/111:47:381
1596399511,17cyclictest17076-21ntpq09:35:251
1596399502,15cyclictest26220-21/usr/sbin/munin07:50:131
1596399501,24cyclictest17943-21users08:50:291
15963995012,15cyclictest3020-21runrttasks07:48:251
1596399501,16cyclictest7286-21taskset12:46:101
1596399501,16cyclictest29151-21ssh12:35:281
1596399501,16cyclictest24747-21sh11:24:381
1596399501,14cyclictest1105-21chrt08:07:451
1596399491,16cyclictest6237-21open_files08:20:281
1596399484,15cyclictest9439-21rm10:00:341
1596399481,25cyclictest30791-21iostat08:00:221
1596399481,16cyclictest12374-21chrt08:36:171
1596399481,14cyclictest11323-21chrt11:44:061
1596399471,15cyclictest22628-21sh09:05:151
1596399471,15cyclictest15302-21smartctl08:45:241
1596399461,32cyclictest0-21swapper/107:55:201
1596399452,10cyclictest20038-21taskset07:31:411
1596399451,19cyclictest9977-21ntp_states08:30:261
1596399451,15cyclictest7269-21/usr/sbin/munin08:25:141
1596399451,15cyclictest2040-21idleruntime08:10:191
1596399451,14cyclictest5086-21runrttasks08:18:501
1596399448,14cyclictest3020-21runrttasks07:37:231
1596299444,4cyclictest81rcu_preempt11:27:320
1596399431,18cyclictest22267-21latency_hist09:05:011
1596399431,15cyclictest23815-21runrttasks07:41:151
99912420,9sleep01596299cyclictest12:50:100
10912420,4sleep00-21swapper/009:19:450
320182410,3sleep00-21swapper/010:25:090
15962993917,18cyclictest21645-21sed09:00:290
1596299391,14cyclictest3020-21runrttasks11:10:260
23532381,33sleep02404-21ps08:10:290
1596299379,14cyclictest7613-21cpuspeed_turbos11:40:150
1596299363,4cyclictest81rcu_preempt12:09:200
1596299362,11cyclictest81rcu_preempt08:06:260
15962993518,7cyclictest30276-21ssh10:22:500
1596299351,7cyclictest81rcu_preempt11:32:110
1596299346,5cyclictest81rcu_preempt10:41:140
1596299344,5cyclictest81rcu_preempt09:45:420
1596299344,4cyclictest81rcu_preempt12:43:130
1596299344,4cyclictest81rcu_preempt12:04:340
1596299344,4cyclictest81rcu_preempt10:19:070
1596299344,4cyclictest81rcu_preempt10:11:180
1596299344,4cyclictest81rcu_preempt09:40:590
1596299343,4cyclictest81rcu_preempt12:25:470
1596299341,21cyclictest5825-21ssh09:24:250
1596299341,18cyclictest11829-21ntpdc08:35:250
69792332,8sleep07032-21hddtemp_smartct09:25:150
1596299338,4cyclictest81rcu_preempt10:53:120
1596299334,4cyclictest81rcu_preempt12:35:040
1596299334,4cyclictest81rcu_preempt11:17:240
1596299334,4cyclictest81rcu_preempt11:17:230
1596299332,14cyclictest20648-21ssh10:45:470
1596299331,8cyclictest12314-21ssh10:37:440
1596299331,15cyclictest0-21swapper/011:59:470
15962993310,4cyclictest81rcu_preempt09:37:200
312502321,27sleep031406-21/usr/sbin/munin08:00:320
1596299327,4cyclictest81rcu_preempt10:56:530
1596299324,4cyclictest81rcu_preempt12:30:230
1596299324,4cyclictest81rcu_preempt11:02:310
1596299323,4cyclictest81rcu_preempt12:18:230
1596299323,4cyclictest81rcu_preempt11:09:370
1596299322,15cyclictest12772-21/usr/sbin/munin10:05:490
1596299322,11cyclictest4521-21/usr/sbin/munin10:30:110
1596299321,4cyclictest81rcu_preempt12:13:110
1596299321,4cyclictest81rcu_preempt11:51:450
1596299321,4cyclictest81rcu_preempt09:50:110
1596299321,4cyclictest81rcu_preempt09:30:550
1596299321,4cyclictest81rcu_preempt09:30:550
1596299321,4cyclictest81rcu_preempt08:55:070
1596299321,14cyclictest14580-21grep12:20:240
1596299313,14cyclictest0-21swapper/008:50:220
1596299312,24cyclictest4086-21iostat08:15:260
1596299312,16cyclictest0-21swapper/012:45:530
15962993115,12cyclictest7468-21cpuspeed_turbos08:25:130
1596299311,4cyclictest81rcu_preempt08:46:260
1596299311,4cyclictest81rcu_preempt08:33:030
1596299302,4cyclictest81rcu_preempt09:10:440
299362291,5sleep00-21swapper/007:59:160
1596299292,13cyclictest18605-21ntp_states07:30:060
1596299287,5cyclictest27991-21updatedb.mlocat07:53:570
1596299282,13cyclictest25594-21grep07:45:310
1596299281,15cyclictest5680-21df_inode08:20:160
221852271,5sleep00-21swapper/007:39:030
1596299271,13cyclictest23650-21unixbench_multi07:40:300
1596299270,4cyclictest81rcu_preempt09:07:350
1596299263,4cyclictest81rcu_preempt08:40:170
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional