You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-01 - 04:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack5slot3s.osadl.org (updated Tue Jul 01, 2025 00:44:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
104582239162,24sleep00-21swapper/019:06:100
107072236163,23sleep30-21swapper/319:09:233
104592233161,58sleep10-21swapper/119:06:111
106462228163,51sleep20-21swapper/219:08:382
1946121620,4sleep3401ktimersoftd/320:40:153
2130521460,4sleep01085399cyclictest20:45:000
10863991147,105cyclictest0-21swapper/221:20:222
10863997370,2cyclictest0-21swapper/220:44:072
190402660,5sleep21086399cyclictest21:50:272
242232630,3sleep19-21rcu_preempt00:35:121
35552610,3sleep03559-21fschecks_time22:30:190
115732570,4sleep01085399cyclictest20:20:290
1085399529,31cyclictest0-21swapper/021:40:010
17862500,1sleep10-21swapper/120:00:141
1085399488,12cyclictest27039-21perf23:25:010
300562460,1sleep10-21swapper/121:05:011
1085399455,13cyclictest23317-21idleruntime-cro20:50:010
1085399442,6cyclictest21868-21munin-run22:00:010
1085399442,6cyclictest21868-21munin-run22:00:010
1085399423,9cyclictest836-21runrttasks21:48:340
1086899413,27cyclictest7872-21ls23:55:003
1086899408,23cyclictest166332cut19:20:203
1086899403,18cyclictest86750irq/128-enp1s0-23:30:243
1086899401,28cyclictest0-21swapper/321:40:153
1086899401,27cyclictest23591-21cpu20:50:123
1086899381,26cyclictest15511-21/usr/sbin/munin21:45:213
1086899381,21cyclictest19714-21/usr/sbin/munin00:25:223
1086899370,30cyclictest0-21swapper/320:55:223
1086899363,24cyclictest4237-21/usr/sbin/munin22:30:273
1086899362,31cyclictest16213-21/usr/sbin/munin23:00:263
1086899362,31cyclictest16213-21/usr/sbin/munin23:00:263
1086899361,6cyclictest0-21swapper/322:00:003
1086899361,6cyclictest0-21swapper/322:00:003
10868993615,18cyclictest29154-21turbostat19:50:013
1086899361,4cyclictest19233-21/usr/sbin/munin23:05:213
1086899361,26cyclictest24371-21sshd19:35:443
1086899353,22cyclictest8049-21hddtemp_smartct22:40:193
1086899351,24cyclictest21835-21wc23:10:253
1086899351,18cyclictest0-21swapper/322:05:173
10853993510,12cyclictest29223-21munin-run23:30:010
1086899341,7cyclictest14634-21snmpd21:11:013
1086899341,5cyclictest0-21swapper/323:29:273
1086899341,4cyclictest0-21swapper/300:11:353
1086899341,3cyclictest5170-21/usr/sbin/munin23:45:273
1086899341,29cyclictest11455-40dnf22:48:473
1086399341,25cyclictest0-21swapper/223:05:462
1086399341,25cyclictest0-21swapper/220:50:162
1086399341,15cyclictest0-21swapper/222:35:362
1086399341,14cyclictest0-21swapper/220:27:242
1086899332,28cyclictest1905-21grep20:00:153
1086899331,3cyclictest25692-21users00:35:263
1086899331,19cyclictest13279-21cat22:50:263
1086399337,10cyclictest12004-21memory21:35:232
10863993322,3cyclictest0-21swapper/220:49:122
1086399331,13cyclictest0-21swapper/222:55:282
1086399331,13cyclictest0-21swapper/220:55:442
1085399331,25cyclictest27963-21kworker/0:121:01:060
1086899324,19cyclictest7429-21cat21:25:193
1086899321,7cyclictest0-21swapper/321:31:083
1086899321,3cyclictest29831-21/usr/sbin/munin22:15:273
1086899321,23cyclictest0-21swapper/322:20:233
1086899321,14cyclictest0-21swapper/320:32:363
1086899321,13cyclictest0-21swapper/320:38:283
1086899320,2cyclictest0-21swapper/300:07:563
1086899320,2cyclictest0-21swapper/300:07:563
1086399329,10cyclictest780-21rngd19:13:362
1086399321,24cyclictest0-21swapper/223:41:282
1086399321,24cyclictest0-21swapper/221:27:362
1086399320,23cyclictest0-21swapper/222:25:302
1085399323,17cyclictest14972-21kworker/0:300:16:200
1085399320,12cyclictest19634-21kworker/0:122:20:240
1086899316,13cyclictest0-21swapper/322:35:323
1086899315,22cyclictest2368-21tr21:15:143
1086899312,15cyclictest11079-21cpu19:10:133
1086899311,4cyclictest0-21swapper/322:26:363
10868993113,15cyclictest12160-21ntp_states21:35:263
1086899311,2cyclictest25806-21if_enp2s023:20:213
1086899311,21cyclictest0-21swapper/300:24:323
1086899311,15cyclictest0-21swapper/300:03:173
1086899311,13cyclictest0-21swapper/319:55:543
1086899311,12cyclictest0-21swapper/319:32:523
1086899310,4cyclictest0-21swapper/322:55:183
1086899310,3cyclictest86750irq/128-enp1s0-21:20:163
1086399314,17cyclictest30-21rcuc/222:19:082
1086399311,11cyclictest0-21swapper/200:19:442
1086399311,10cyclictest0-21swapper/219:59:362
1085399312,22cyclictest19634-21kworker/0:122:35:100
1086899309,18cyclictest22062-21cpuspeed_turbos00:30:143
1086899307,19cyclictest11347-21ntpq20:20:253
1086899304,23cyclictest9575-21/usr/sbin/munin20:15:303
1086899303,24cyclictest27666-21proc_pri22:10:283
1086899301,5cyclictest0-21swapper/323:41:593
1086899301,4cyclictest24321-21unixbench-2d23:15:263
1086899301,24cyclictest29803-21uniq19:50:163
1086899301,24cyclictest29803-21uniq19:50:163
1086899301,15cyclictest26351-21cat19:40:303
1086899301,15cyclictest0-21swapper/323:37:483
1086899301,11cyclictest0-21swapper/321:51:143
1086399301,9cyclictest32196-21kworker/2:123:20:562
1086399301,14cyclictest7340-21unixbench-2d23:50:282
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional