You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-17 - 13:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Wed Sep 17, 2025 00:59:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
91409912401209,21cyclictest0-21swapper/019:25:060
91419912061197,6cyclictest0-21swapper/119:25:061
91419911871143,32cyclictest13567-21grep19:35:011
76582644598,31sleep00-21swapper/019:20:150
89542561517,30sleep10-21swapper/119:23:441
688621090,21sleep03-21ksoftirqd/023:47:540
276802910,3sleep041ktimersoftd/000:43:050
177012840,8sleep0914099cyclictest23:25:240
93452820,4sleep041ktimersoftd/022:10:100
9140997963,13cyclictest0-21swapper/019:35:010
9140996210,12cyclictest3051-21grep20:30:270
914099612,15cyclictest1367-21ssh22:00:200
914099611,15cyclictest16272-21ssh23:57:330
9140995816,7cyclictest12371-21diskstats23:20:170
914099569,6cyclictest30836-21ntp_states23:05:240
9140995611,6cyclictest32365-21munin-plugin-st20:25:000
914099552,13cyclictest14087-21wget22:15:130
9140995412,6cyclictest6332-21hddtemp_smartct22:40:180
914099541,11cyclictest23778-21aten_r4power_vo22:25:150
9140995411,13cyclictest6833-21ssh23:14:530
9140995410,6cyclictest23544-21users21:15:320
9140995410,6cyclictest23544-21users21:15:320
914099534,17cyclictest5530-21diskstats00:20:180
914099533,17cyclictest16781-21sed19:40:290
914099532,12cyclictest16871-21users22:50:300
914099532,11cyclictest22022-21diskstats23:30:180
9140995314,5cyclictest8502-21ssh23:15:290
9140995311,5cyclictest28852-21ntp_states00:10:260
914099523,15cyclictest8309-21iostat_ios20:45:210
914099523,15cyclictest27175-21iostat_ios23:35:220
914099522,14cyclictest29344-21iostat_ios20:15:230
914099522,12cyclictest0-21swapper/020:00:280
914099522,12cyclictest0-21swapper/020:00:270
9140995213,4cyclictest0-21swapper/020:05:070
9140995211,6cyclictest25132-21aten_r4power_en23:00:110
914099521,13cyclictest13744-21fschecks_count21:00:180
914099521,12cyclictest17921-21iostat_ios21:10:210
914099517,5cyclictest20836-21python00:35:310
914099512,5cyclictest26921-21/usr/sbin/munin21:20:330
914099512,5cyclictest26921-21/usr/sbin/munin21:20:330
9140995118,5cyclictest18699-21/usr/sbin/munin21:45:170
9140995117,6cyclictest3-21ksoftirqd/000:01:250
9140995112,4cyclictest12055-21ssh23:53:250
9140995112,4cyclictest0-21swapper/000:15:020
914099511,15cyclictest8402-21ssh22:09:160
914099511,11cyclictest15192-21/usr/sbin/munin00:30:200
9140995110,7cyclictest1809-21ntp_states22:35:270
914099509,5cyclictest28817-21sh22:30:160
914099507,8cyclictest698-21sed21:25:340
914099502,15cyclictest19190-21/usr/sbin/munin22:20:160
914099497,4cyclictest12421-21find20:55:290
914099492,14cyclictest29132-21ssh00:45:030
914099492,14cyclictest2053-21/usr/sbin/munin00:50:220
914099492,14cyclictest10975-21diskstats22:45:190
9140994912,7cyclictest2220-21dbus-daemon21:55:170
914099491,15cyclictest12808-21ssh21:38:530
914099488,3cyclictest0-21swapper/020:50:270
914099485,8cyclictest20961-21iostat_ios22:55:220
914099485,5cyclictest23885-21memory00:05:260
914099484,8cyclictest20020-21memory19:50:210
914099482,6cyclictest14207-21df_inode21:40:160
914099482,14cyclictest6439-21irqstats20:40:220
914099482,14cyclictest4622-21/usr/sbin/munin20:35:200
914099482,14cyclictest14937-21/usr/sbin/munin19:35:330
914099481,15cyclictest31908-21hddtemp_smartct23:40:210
914099476,14cyclictest0-21swapper/021:06:040
914099473,9cyclictest14543-21munin-run00:30:000
914099472,16cyclictest0-21swapper/021:51:040
914099471,15cyclictest357-21/usr/sbin/munin20:25:170
9140994710,5cyclictest0-21swapper/021:30:300
914099461,12cyclictest22794-21taskset19:58:390
9140994514,5cyclictest3-21ksoftirqd/020:13:380
914099436,5cyclictest17736-21df19:45:130
150742420,4sleep1211ktimersoftd/121:40:301
128792411,36sleep112889-21ssh00:27:361
914199372,23cyclictest0-21swapper/123:17:021
914199362,19cyclictest31023-21sh23:40:001
233672361,31sleep123379-21ssh23:30:531
914199355,18cyclictest0-21swapper/122:42:051
91522341,28sleep19172-21hddtemp_smartct23:50:191
914199347,6cyclictest0-21swapper/100:24:571
914199346,7cyclictest0-21swapper/123:09:411
914199346,24cyclictest0-21swapper/121:21:421
914199346,24cyclictest0-21swapper/121:21:421
914199344,4cyclictest81rcu_preempt22:35:481
9141993413,18cyclictest0-21swapper/100:05:141
183202341,29sleep118430-21ssh00:00:131
129512341,28sleep112965-21ssh22:46:511
914199339,20cyclictest0-21swapper/122:28:501
914199336,4cyclictest81rcu_preempt22:22:591
63942331,5sleep16429-21iostat20:40:201
914199329,19cyclictest0-21swapper/121:17:581
914199329,19cyclictest0-21swapper/121:17:581
9141993223,5cyclictest19572-21ssh22:54:251
914199322,16cyclictest24057-21hddtemp_smartct21:50:191
914199322,16cyclictest12395-21ssh22:13:031
914199321,10cyclictest17286-21fschecks_count23:25:181
152352320,5sleep115229-21ssh00:30:211
914199318,9cyclictest420-21sh00:48:581
914199317,13cyclictest81rcu_preempt00:52:161
914199315,14cyclictest0-21swapper/121:49:571
914199313,15cyclictest0-21swapper/121:35:041
914199312,25cyclictest0-21swapper/122:03:511
914199311,4cyclictest81rcu_preempt23:55:191
9141993113,14cyclictest24491-21sshd20:01:371
9141993113,14cyclictest24491-21sshd20:01:361
9141993112,15cyclictest22-21ksoftirqd/121:14:061
8222311,26sleep1880-21ntp_offset20:25:261
158162311,4sleep10-21swapper/122:16:031
914199307,5cyclictest22-21ksoftirqd/122:08:021
914199307,19cyclictest0-21swapper/123:23:081
914199305,4cyclictest81rcu_preempt20:59:251
914199305,14cyclictest0-21swapper/123:12:131
914199305,14cyclictest0-21swapper/100:15:381
914199302,15cyclictest0-21swapper/121:30:031
9141993016,8cyclictest28998-21iostat_ios21:55:201
9141993016,11cyclictest22-21ksoftirqd/100:35:211
9141993015,8cyclictest13670-21diskstats21:00:181
914199301,4cyclictest81rcu_preempt20:20:401
287632301,4sleep128829-21ntpq00:10:251
914199295,4cyclictest81rcu_preempt19:55:021
914199294,4cyclictest81rcu_preempt19:53:111
914199292,6cyclictest0-21swapper/122:56:571
9141992920,5cyclictest15401-21idleruntime-cro19:40:001
9141992914,11cyclictest22-21ksoftirqd/121:06:251
914199291,15cyclictest5504-21ssh23:45:531
188082291,5sleep10-21swapper/119:46:201
914199289,16cyclictest0-21swapper/123:02:031
914199285,8cyclictest0-21swapper/120:10:181
914199283,18cyclictest9893-21diskstats20:50:161
914199281,15cyclictest29475-21ssh22:30:271
243492281,5sleep10-21swapper/100:40:031
914199278,15cyclictest0-21swapper/123:43:151
914199273,10cyclictest0-21swapper/120:45:261
914199271,4cyclictest81rcu_preempt20:07:171
9141992711,5cyclictest22-21ksoftirqd/119:40:231
33392271,5sleep10-21swapper/120:30:331
14452271,5sleep10-21swapper/121:26:331
914199262,4cyclictest81rcu_preempt20:16:181
914199262,10cyclictest4765-21ntp_states20:35:241
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional