You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-01-13 - 14:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack9slot4s (updated Mon Jan 13, 2025 00:59:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
314629912021175,21cyclictest0-21swapper/119:55:271
314619911931152,37cyclictest0-21swapper/019:25:120
314629911791160,14cyclictest0-21swapper/119:25:111
310962609564,30sleep10-21swapper/119:21:441
305292559510,32sleep00-21swapper/019:20:260
252621030,3sleep00-21swapper/022:40:130
31461999660,33cyclictest0-21swapper/019:55:270
150552860,4sleep13146299cyclictest20:05:121
31462996613,23cyclictest1941-21/usr/sbin/munin21:00:251
31462996612,24cyclictest17408-21/usr/sbin/munin23:30:221
31462996313,16cyclictest17072-21cstates20:10:151
31462996214,15cyclictest26928-21taskset23:56:261
31462996213,16cyclictest11651-21taskset00:52:051
31462996213,15cyclictest20526-21df21:55:131
31462996212,16cyclictest20804-21taskset23:38:001
226232610,3sleep022622-21egrep20:25:120
31462996013,17cyclictest3192-21awk22:40:261
31462996013,15cyclictest28813-21taskset00:01:211
31462996013,14cyclictest30268-21fschecks_count00:10:121
31462996012,15cyclictest7429-21runrttasks00:39:201
31462996011,19cyclictest11490-21runrttasks21:25:281
31462996011,16cyclictest10009-21runrttasks23:01:331
31462996011,14cyclictest32178-21users22:30:251
31462996011,14cyclictest32178-21users22:30:251
31462995912,17cyclictest14387-21taskset20:00:561
31462995912,16cyclictest14522-21chrt23:17:541
31462995912,15cyclictest18958-21grep21:50:151
31462995912,14cyclictest8994-21chrt21:15:471
31462995912,14cyclictest2945-21chrt00:23:281
31462995912,14cyclictest25117-21chrt23:52:061
31462995911,16cyclictest5308-21runrttasks00:31:551
31462995812,16cyclictest8221-21cat00:40:191
31462995812,16cyclictest8221-21cat00:40:191
31462995812,14cyclictest29767-21chrt00:09:461
31462995713,18cyclictest9300-21/usr/sbin/munin19:50:201
31462995711,15cyclictest7392-21chrt21:13:391
31462995711,14cyclictest3649-21/usr/sbin/munin19:35:181
31462995710,15cyclictest23354-21taskset23:48:141
31462995613,28cyclictest14911-21/usr/sbin/munin23:20:201
31462995612,14cyclictest26218-21/usr/sbin/munin20:35:261
31462995611,15cyclictest30198-21latency_hist22:25:021
31462995610,15cyclictest19820-21expr20:15:281
3146299558,16cyclictest27828-21chrt22:15:551
3146299557,17cyclictest6386-21chrt22:53:051
31462995512,18cyclictest9590-21/usr/sbin/munin21:20:191
31462995512,18cyclictest31839-21/usr/sbin/munin00:15:181
31462995511,12cyclictest30304-21chrt22:25:221
3146299548,11cyclictest22292-21sed23:40:271
31462995412,13cyclictest22964-21ntpq22:00:241
3146299539,14cyclictest0-21swapper/122:57:211
3146299538,15cyclictest7223-21chrt19:44:391
31462995313,17cyclictest31422-21awk20:50:191
31462995311,15cyclictest29131-21smartctl20:40:271
3146299526,24cyclictest0-21swapper/123:12:241
31462995213,11cyclictest880-21open_files20:55:261
31462995212,15cyclictest15449-21runrttasks21:37:361
31462995210,14cyclictest8013-21iostat19:45:171
31462995113,15cyclictest24634-21aten_r4power_cu22:10:071
31462995112,15cyclictest30944-21snmpd23:05:221
3146299508,13cyclictest30944-21snmpd21:42:061
31462995017,12cyclictest3020-21runrttasks22:05:081
31462995015,12cyclictest23304-21ntp_states20:25:301
31462995011,16cyclictest2236-21latency_hist22:40:001
31462994914,15cyclictest12640-21if_err_eth021:30:131
31462994911,15cyclictest10061-21latency_hist00:50:001
3146299489,16cyclictest4184-21cstates21:05:121
31462994814,10cyclictest3660-21sshd00:29:141
31462994811,17cyclictest21994-21chrt20:21:481
3146299479,17cyclictest18130-21sshd21:49:141
31462994713,12cyclictest30163-21cron20:47:001
31462994711,14cyclictest30944-21snmpd22:47:191
31462994710,15cyclictest3020-21runrttasks20:32:441
31462994613,14cyclictest3121-21sshd19:31:131
3146299459,14cyclictest30944-21snmpd23:27:281
31461994313,5cyclictest81rcu_preempt00:23:570
3146199381,25cyclictest27957-21sendmail20:40:000
3146199371,15cyclictest0-21swapper/021:46:480
3146199351,11cyclictest81rcu_preempt21:24:200
79202341,4sleep07975-21/usr/sbin/munin19:45:170
3146199339,4cyclictest22303-21timerwakeupswit23:40:270
3146199336,14cyclictest23839-21runrttasks22:04:560
3146199336,13cyclictest0-21swapper/000:55:000
3146199332,26cyclictest32669-21ntp_states00:15:240
3146199331,4cyclictest81rcu_preempt23:55:300
3146199327,13cyclictest2220-21dbus-daemon23:48:090
3146199321,4cyclictest81rcu_preempt22:18:210
3146199321,4cyclictest81rcu_preempt20:16:580
3146199321,12cyclictest0-21swapper/023:31:420
236802321,4sleep023785-21wget23:50:070
320362311,5sleep00-21swapper/020:51:540
3146199315,4cyclictest81rcu_preempt21:15:210
3146199312,5cyclictest81rcu_preempt21:50:240
3146199312,17cyclictest0-21swapper/020:40:110
3146199311,4cyclictest81rcu_preempt22:32:250
3146199311,4cyclictest81rcu_preempt22:32:240
3146199311,4cyclictest81rcu_preempt20:22:570
3146199311,4cyclictest81rcu_preempt00:32:130
3146199311,15cyclictest17628-21ntp_kernel_pll_20:10:270
3146199311,12cyclictest24245-21runrttasks22:07:380
3146199310,4cyclictest81rcu_preempt19:43:510
297162311,5sleep00-21swapper/000:09:140
98752301,5sleep00-21swapper/000:48:090
3146199306,11cyclictest19707-21cut23:35:150
3146199303,3cyclictest81rcu_preempt21:30:260
3146199301,4cyclictest81rcu_preempt23:28:540
31461993012,5cyclictest6204-21gdbus23:22:540
3146199301,25cyclictest1967-21diskstats19:30:180
3146199301,25cyclictest14053-21/usr/sbin/munin21:35:120
3146199301,12cyclictest0-21swapper/023:15:260
3146199300,4cyclictest81rcu_preempt00:41:280
3146199300,4cyclictest81rcu_preempt00:41:270
218742301,5sleep00-21swapper/021:59:130
126162301,5sleep00-21swapper/023:12:380
81022291,5sleep00-21swapper/022:56:250
52072291,5sleep00-21swapper/019:38:250
3146199292,14cyclictest3126-21/usr/sbin/munin21:00:260
3146199292,14cyclictest16152-21df_inode21:40:130
3146199291,24cyclictest0-21swapper/020:49:310
3146199291,13cyclictest6242-21fschecks_time00:35:170
31461992910,14cyclictest4762-21ntp_states21:05:260
3146199290,4cyclictest81rcu_preempt22:49:310
165322291,5sleep00-21swapper/020:08:450
15142291,5sleep00-21swapper/020:57:280
91842281,5sleep00-21swapper/019:50:020
61662281,5sleep00-21swapper/022:50:520
3146199288,6cyclictest6208-21fschecks_count21:10:180
3146199281,15cyclictest625-21df_inode22:35:140
3146199281,12cyclictest28138-21/usr/sbin/munin00:00:210
261772281,5sleep00-21swapper/022:13:130
257332281,5sleep00-21swapper/020:31:420
3146199273,11cyclictest9907-21vmstat23:00:290
3146199272,12cyclictest31375-21cron00:12:000
3146199271,16cyclictest0-21swapper/020:00:460
118002271,5sleep00-21swapper/021:28:420
3146199262,4cyclictest81rcu_preempt22:20:560
3146199251,15cyclictest0-21swapper/000:26:090
3146199241,14cyclictest0-21swapper/022:25:540
3146199241,13cyclictest10608-21sshd23:06:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional