You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-19 - 02:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Tue Nov 18, 2025 12:59:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
39799912311201,26cyclictest10233-21updatedb.mlocat07:37:371
39799911051090,12cyclictest0-21swapper/107:25:121
35572769723,30sleep10-21swapper/107:21:141
21072769722,31sleep00-21swapper/007:20:110
3978995511,543cyclictest5877-21kworker/0:207:37:370
397899239233,3cyclictest0-21swapper/007:25:120
258542960,9sleep0397899cyclictest10:30:280
247792480,4sleep1211ktimersoftd/111:36:241
275482470,24sleep10-21swapper/109:25:211
3979994129,9cyclictest185082sleep110:56:211
397899409,9cyclictest103522sh10:48:280
397999398,9cyclictest81rcu_preempt11:13:591
3978993915,19cyclictest20920-21ntpq10:25:270
251412391,34sleep125151-21ssh09:22:551
3978993821,6cyclictest3-21ksoftirqd/009:45:090
3979993723,4cyclictest22-21ksoftirqd/109:35:151
397999372,12cyclictest81rcu_preempt10:08:591
397999371,14cyclictest107532rm11:22:011
66482361,4sleep06660-21ssh11:18:060
397999364,4cyclictest81rcu_preempt10:13:441
3979993630,4cyclictest137350irq/90-eth011:57:131
397999361,4cyclictest81rcu_preempt12:23:411
397899369,6cyclictest0-21swapper/009:44:240
397899360,12cyclictest19751-21ssh09:50:430
3979993529,4cyclictest137350irq/90-eth012:34:421
397999352,19cyclictest0-21swapper/112:54:321
397999351,12cyclictest0-21swapper/110:22:481
397899358,5cyclictest81rcu_preempt11:03:340
397899357,5cyclictest81rcu_preempt11:40:280
397899356,4cyclictest81rcu_preempt10:40:290
397899356,16cyclictest0-21swapper/011:11:080
397899354,5cyclictest81rcu_preempt11:54:180
397899354,4cyclictest81rcu_preempt11:55:140
397899353,7cyclictest81rcu_preempt08:47:130
397899351,21cyclictest28368-21ssh09:25:350
205642351,30sleep020578-21ssh10:59:220
192492350,7sleep00-21swapper/009:16:220
161442351,30sleep016158-21ssh11:27:530
397999344,7cyclictest0-21swapper/112:29:511
397999343,17cyclictest0-21swapper/110:40:191
397999342,11cyclictest0-21swapper/112:42:521
397899347,5cyclictest81rcu_preempt12:27:020
397899346,4cyclictest81rcu_preempt10:11:580
397899341,4cyclictest81rcu_preempt12:07:130
397899341,15cyclictest15588-21cron12:35:000
397899341,14cyclictest1997-21grep10:05:320
3978993411,11cyclictest5749-21/usr/sbin/munin08:50:380
269382341,28sleep127033-21users08:20:301
1383223418,4sleep03-21ksoftirqd/009:10:320
397999339,11cyclictest0-21swapper/111:16:521
397999338,21cyclictest0-21swapper/110:32:181
3979993318,4cyclictest22-21ksoftirqd/112:00:241
397899335,5cyclictest81rcu_preempt10:54:160
397899331,20cyclictest25391-21taskset08:18:010
313182331,4sleep131329-21ssh10:35:571
284462331,5sleep00-21swapper/012:47:550
14512331,3sleep01633-21/usr/sbin/munin12:20:150
81702321,4sleep18180-21ssh11:53:161
397999328,5cyclictest40950irq/93-i91510:49:161
397999328,13cyclictest40950irq/93-i91511:30:181
397999326,9cyclictest3020-21runrttasks10:27:131
397999326,5cyclictest81rcu_preempt08:55:321
397999326,13cyclictest0-21swapper/109:11:011
397999323,9cyclictest17025-21ssh09:48:081
3979993227,3cyclictest137350irq/90-eth009:19:511
397999322,5cyclictest137350irq/90-eth012:08:071
3979993216,4cyclictest22-21ksoftirqd/109:50:431
3979993213,15cyclictest11789-21users10:15:341
397999321,18cyclictest0-21swapper/111:02:271
397899329,11cyclictest0-21swapper/008:05:180
397899329,11cyclictest0-21swapper/008:05:170
397899326,4cyclictest81rcu_preempt10:24:100
397899326,4cyclictest81rcu_preempt09:37:230
3978993222,6cyclictest23254-21python09:20:300
397899321,13cyclictest81rcu_preempt11:46:310
195142321,27sleep019528-21ssh12:38:410
123432321,5sleep00-21swapper/007:44:180
9742311,5sleep10-21swapper/112:19:571
3979993126,3cyclictest137350irq/90-eth012:10:481
3979993115,4cyclictest22-21ksoftirqd/107:30:341
397999311,4cyclictest81rcu_preempt08:03:371
397999311,26cyclictest1083-21sendmail_mailtr11:45:301
3979993112,15cyclictest22-21ksoftirqd/109:44:011
397899317,4cyclictest81rcu_preempt09:33:350
397899315,4cyclictest81rcu_preempt10:00:190
397899313,6cyclictest81rcu_preempt12:11:310
397899312,4cyclictest81rcu_preempt11:33:480
397899312,15cyclictest23282-21/usr/sbin/munin11:35:200
397899312,15cyclictest11040-21irqstats10:15:230
397899311,4cyclictest81rcu_preempt08:20:080
3978993113,14cyclictest2233-21cron08:45:010
397899311,14cyclictest15318-21memory12:00:250
309122311,4sleep00-21swapper/008:31:400
195652311,4sleep119576-21ssh12:38:471
76942301,5sleep07827-21/usr/sbin/munin07:30:280
397999306,10cyclictest0-21swapper/107:50:581
397999305,4cyclictest81rcu_preempt10:00:091
3979993022,5cyclictest32517-21rm09:30:281
397999302,10cyclictest28123-21hddtemp_smartct11:40:201
397999301,7cyclictest0-21swapper/111:08:431
397999301,4cyclictest81rcu_preempt08:40:151
3979993014,4cyclictest22-21ksoftirqd/108:50:391
397999301,15cyclictest12868-21df_inode07:45:141
397899307,5cyclictest81rcu_preempt11:20:280
397899303,14cyclictest0-21swapper/012:18:200
397899301,4cyclictest81rcu_preempt12:54:130
3978993011,6cyclictest26845-21iostat_ios11:05:240
397899301,15cyclictest15388-21/usr/sbin/munin07:50:250
397899301,13cyclictest24839-21users09:55:310
210402301,25sleep021059-21df_inode12:40:160
79102291,5sleep00-21swapper/008:55:480
397999299,10cyclictest22-21ksoftirqd/109:56:141
397999291,7cyclictest22-21ksoftirqd/112:45:231
3979992917,9cyclictest22-21ksoftirqd/111:25:331
397999291,4cyclictest81rcu_preempt08:34:561
397999291,4cyclictest81rcu_preempt08:06:291
397999291,4cyclictest81rcu_preempt08:06:281
3979992913,12cyclictest22-21ksoftirqd/109:05:211
3979992913,12cyclictest0-21swapper/110:51:531
397899293,4cyclictest81rcu_preempt10:38:190
397899292,15cyclictest0-21swapper/008:35:460
397899292,14cyclictest17075-21/usr/sbin/munin07:55:260
3978992910,7cyclictest23008-21python08:10:290
289492291,5sleep00-21swapper/008:25:460
397999280,4cyclictest81rcu_preempt09:03:111
397899281,15cyclictest19103-21pluginstate08:00:280
397899281,14cyclictest10891-21wget09:05:200
397899281,13cyclictest9581-21awk09:00:300
3978992810,6cyclictest14190-21seq07:49:000
121662281,5sleep10-21swapper/107:42:271
397999276,5cyclictest40950irq/93-i91508:25:221
397999272,4cyclictest81rcu_preempt07:56:481
397999272,11cyclictest3672-21ntp_states08:45:261
3979992714,10cyclictest22-21ksoftirqd/108:35:131
3979992619,4cyclictest22-21ksoftirqd/108:15:281
3979992518,4cyclictest22-21ksoftirqd/108:10:191
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional