You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-05 - 07:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack9slot4s (updated Wed Nov 05, 2025 00:59:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
204099911221093,26cyclictest0-21swapper/019:25:060
203242517471,31sleep10-21swapper/119:24:471
201372517470,31sleep00-21swapper/019:22:480
2040999213192,15cyclictest24908-21cut19:35:010
2041099156148,5cyclictest0-21swapper/119:25:061
60842830,3sleep10-21swapper/100:42:041
203102710,5sleep0111rcuc/021:35:300
64462520,8sleep02040999cyclictest22:28:530
80732440,4sleep08074-21fschecks_time20:15:200
231622410,8sleep02040999cyclictest23:20:010
70182380,9sleep02040999cyclictest21:21:530
240812381,32sleep124091-21ssh22:13:361
20409993816,7cyclictest13177-21munin-run20:30:000
20409993812,12cyclictest3793-21ssh22:59:590
2040999381,16cyclictest0-21swapper/022:45:290
131972380,14sleep041ktimersoftd/023:09:370
245362371,31sleep124546-21ssh23:55:001
2041099373,15cyclictest0-21swapper/121:41:011
2040999376,13cyclictest0-21swapper/023:51:470
35912361,31sleep13615-21ssh00:05:321
20410993625,7cyclictest18809-21ssh23:48:411
2040999365,4cyclictest81rcu_preempt22:13:020
2040999363,20cyclictest0-21swapper/020:37:320
2040999361,5cyclictest81rcu_preempt21:50:170
2040999359,13cyclictest16637-21vmstat23:45:300
20409993511,13cyclictest40950irq/93-i91521:20:000
165252354,27sleep10-21swapper/122:39:331
314962341,4sleep131510-21ssh21:47:251
274522341,5sleep127463-21ssh23:57:131
238082341,4sleep123819-21sh22:46:061
2041099346,7cyclictest0-21swapper/100:11:501
20410993416,8cyclictest22-21ksoftirqd/121:38:061
2040999346,4cyclictest81rcu_preempt23:20:270
2040999344,4cyclictest81rcu_preempt21:59:170
2040999341,4cyclictest81rcu_preempt22:17:170
20409993410,5cyclictest81rcu_preempt00:39:460
155092341,30sleep015520-21ssh23:10:590
87162331,4sleep18816-21ls22:30:341
2041099339,20cyclictest0-21swapper/122:55:541
2041099335,4cyclictest81rcu_preempt20:28:061
2041099331,4cyclictest81rcu_preempt21:24:461
20410993313,9cyclictest22-21ksoftirqd/122:44:461
2040999332,18cyclictest13265-21ntp_states22:35:260
2041099329,19cyclictest0-21swapper/100:20:091
2041099325,6cyclictest40950irq/93-i91521:20:001
2041099325,19cyclictest26928-21iostat_ios00:30:221
2041099322,26cyclictest0-21swapper/122:54:471
2041099321,5cyclictest80752grep20:15:201
20410993210,7cyclictest0-21swapper/120:37:321
2040999327,13cyclictest81rcu_preempt22:50:230
2040999324,4cyclictest81rcu_preempt23:35:230
2040999321,20cyclictest42-21kswapd000:00:340
2040999321,20cyclictest0-21swapper/021:33:580
2040999321,11cyclictest0-21swapper/021:45:160
20409993210,5cyclictest25196-21sh00:28:530
111372320,5sleep0111rcuc/000:14:290
39132311,5sleep00-21swapper/020:05:100
303642316,21sleep130403-21latency00:00:231
272802311,26sleep027291-21ssh23:56:580
2041099316,21cyclictest0-21swapper/122:00:271
2041099313,10cyclictest12090-21df00:15:161
20410993122,6cyclictest2343-21irqbalance22:08:471
2041099312,24cyclictest25665-21df_inode22:15:141
20410993115,4cyclictest22-21ksoftirqd/100:36:251
2041099311,17cyclictest22295-21ssh23:18:491
20410993110,11cyclictest40950irq/93-i91521:32:541
2040999313,15cyclictest14528-21unixbench_multi20:30:320
2040999312,4cyclictest81rcu_preempt21:25:570
2040999312,16cyclictest25263-21/usr/sbin/munin21:05:080
2040999312,13cyclictest29409-21memory19:45:220
2040999311,9cyclictest21659-21ssh22:44:580
20409993116,12cyclictest0-21swapper/021:40:360
2040999311,4cyclictest81rcu_preempt21:10:570
20409993114,5cyclictest3-21ksoftirqd/023:29:490
20409993113,6cyclictest12098-21df_inode00:15:160
2040999311,16cyclictest0-21swapper/023:03:560
105972311,5sleep10-21swapper/123:05:511
41302305,21sleep04150-21smartctl00:40:210
33762301,5sleep10-21swapper/120:01:071
284502301,5sleep00-21swapper/019:43:540
233032301,5sleep00-21swapper/020:55:210
2041099306,7cyclictest3020-21runrttasks00:28:151
2041099306,5cyclictest40950irq/93-i91522:21:181
2041099302,4cyclictest0-21swapper/123:01:361
2041099301,4cyclictest81rcu_preempt21:58:591
2040999303,4cyclictest81rcu_preempt23:44:110
2040999302,14cyclictest4376-21ssh00:06:390
2040999301,5cyclictest0-21swapper/022:31:200
20409993012,5cyclictest2065-21ssh22:24:410
2040999301,13cyclictest11548-21ssh22:00:200
174102301,5sleep00-21swapper/022:05:540
154972301,5sleep10-21swapper/123:10:561
71342291,5sleep10-21swapper/120:11:081
300402291,5sleep10-21swapper/119:45:561
249442291,5sleep00-21swapper/019:35:070
246822291,8sleep00-21swapper/021:00:390
2041099295,4cyclictest81rcu_preempt20:33:571
2041099293,22cyclictest0-21swapper/123:21:281
2041099291,5cyclictest81rcu_preempt19:55:301
20410992915,4cyclictest40950irq/93-i91521:10:411
2041099291,4cyclictest81rcu_preempt20:05:571
20410992911,15cyclictest22-21ksoftirqd/121:51:441
20410992911,14cyclictest0-21swapper/123:27:381
2040999295,7cyclictest3020-21runrttasks00:20:150
20409992922,4cyclictest3-21ksoftirqd/020:21:020
2040999292,14cyclictest8346-21/usr/sbin/munin00:45:210
2040999291,15cyclictest2887-21taskset20:00:250
20409992911,5cyclictest26820-21http_loadtime00:30:180
2040999290,4cyclictest81rcu_preempt20:53:160
73182281,5sleep00-21swapper/020:13:010
265572281,5sleep10-21swapper/121:05:391
224902281,5sleep10-21swapper/120:54:061
206582281,5sleep00-21swapper/020:49:380
2041099289,15cyclictest0-21swapper/121:27:001
2041099288,16cyclictest0-21swapper/123:33:391
2041099282,4cyclictest81rcu_preempt19:52:001
2041099281,18cyclictest0-21swapper/120:23:441
2041099281,11cyclictest0-21swapper/119:35:211
20410992810,5cyclictest0-21swapper/121:04:141
2040999282,6cyclictest12969-21/usr/sbin/munin00:50:240
2040999281,15cyclictest903-21expr23:30:170
20409992810,5cyclictest30884-21df19:50:130
187982281,5sleep00-21swapper/020:44:530
166662281,5sleep10-21swapper/100:54:051
105912281,4sleep10-21swapper/100:46:461
70332271,5sleep10-21swapper/123:35:351
243762271,5sleep10-21swapper/119:30:461
241002271,5sleep10-21swapper/120:56:211
2041099277,7cyclictest0-21swapper/122:29:531
20410992715,9cyclictest22-21ksoftirqd/123:40:271
8902261,21sleep00-21swapper/019:55:310
2041099262,10cyclictest3020-21runrttasks20:40:561
2041099252,9cyclictest19807-21ntp_states20:45:251
2041099252,10cyclictest0-21swapper/119:40:161
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional