You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-23 - 15:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Sun Nov 23, 2025 12:59:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
40319912061180,22cyclictest14778-21perl07:50:171
40309911021062,37cyclictest0-21swapper/007:25:120
40319910811068,10cyclictest4224-21grep07:25:101
37502478433,30sleep10-21swapper/107:22:491
35792462370,76sleep00-21swapper/007:21:000
403099171143,6cyclictest14593-21copy07:50:170
8322870,8sleep1403199cyclictest10:05:111
22092860,8sleep0403099cyclictest12:20:180
129342810,8sleep1403199cyclictest11:24:481
139632720,7sleep00-21swapper/011:25:180
186202680,2sleep00-21swapper/010:23:090
154532680,4sleep0111rcuc/009:12:480
71152630,3sleep07114-21ssh11:51:150
321882630,4sleep00-21swapper/010:03:560
278432580,9sleep1137350irq/90-eth010:32:321
201962540,8sleep0403099cyclictest12:39:110
7382511,38sleep1768-21hostname11:45:181
403099481,15cyclictest2394-21ssh10:40:130
281772480,3sleep1211ktimersoftd/111:40:161
281772480,3sleep1211ktimersoftd/111:40:161
188252450,9sleep0403099cyclictest09:15:380
4030994423,6cyclictest3-21ksoftirqd/011:20:470
403099422,15cyclictest4779-21/usr/sbin/munin08:50:220
4031994016,7cyclictest22-21ksoftirqd/111:32:321
196682390,3sleep019672-21cut09:50:270
403199381,8cyclictest3020-21runrttasks10:20:191
126692371,32sleep112681-21ssh11:57:201
67732360,2sleep06776-21cut12:25:110
403099366,5cyclictest81rcu_preempt09:36:000
283192361,5sleep128333-21ssh12:14:041
403099352,17cyclictest12277-21df10:50:150
4030993519,6cyclictest3-21ksoftirqd/012:08:120
4030993518,6cyclictest3-21ksoftirqd/011:19:270
403099351,20cyclictest18491-21grep08:00:120
169392351,29sleep116949-21ssh09:47:491
94232341,29sleep09439-21sh09:40:170
403199349,4cyclictest81rcu_preempt09:50:301
403199346,16cyclictest0-21swapper/112:20:221
403099344,4cyclictest81rcu_preempt09:47:410
403099342,28cyclictest0-21swapper/011:45:260
4030993412,6cyclictest28155-21df11:40:140
4030993412,6cyclictest28155-21df11:40:140
311472341,29sleep131158-21sh09:29:321
403199335,4cyclictest81rcu_preempt12:49:171
403199335,4cyclictest81rcu_preempt09:44:421
403099338,4cyclictest81rcu_preempt10:46:070
403099336,5cyclictest81rcu_preempt10:31:540
403099336,5cyclictest81rcu_preempt09:33:390
403099336,4cyclictest81rcu_preempt11:32:070
403099334,4cyclictest81rcu_preempt12:45:240
403099332,15cyclictest17121-21diskstats10:55:160
4030993318,4cyclictest40950irq/93-i91511:59:470
107282331,4sleep110821-21df07:40:151
97682321,4sleep19898-21users07:35:301
403199329,11cyclictest0-21swapper/109:59:351
403199326,4cyclictest81rcu_preempt10:01:111
403199326,22cyclictest0-21swapper/110:26:311
403199325,23cyclictest0-21swapper/110:57:071
403199324,4cyclictest81rcu_preempt08:22:251
403199322,10cyclictest81rcu_preempt10:42:201
403199321,4cyclictest81rcu_preempt08:01:321
403099329,5cyclictest25316-21seq09:55:570
403099323,4cyclictest81rcu_preempt09:22:350
403099322,4cyclictest81rcu_preempt12:32:250
4030993217,5cyclictest3-21ksoftirqd/007:59:060
403099321,18cyclictest31103-21iostat_ios12:50:190
309522320,5sleep0111rcuc/009:29:150
78742311,4sleep17950-21users08:55:491
403199315,4cyclictest81rcu_preempt08:52:131
403199312,7cyclictest23104-21ntp_states09:20:281
403199312,25cyclictest0-21swapper/110:35:231
403199312,16cyclictest26360-21ssh11:38:171
403199311,4cyclictest81rcu_preempt11:15:291
4031993114,14cyclictest22-21ksoftirqd/110:50:301
4031993110,9cyclictest30944-21snmpd08:31:111
403099311,5cyclictest31362-21ssh12:16:270
4030993111,7cyclictest310-21ssh10:37:420
403099311,16cyclictest27725-21/usr/sbin/munin11:05:520
4030993111,6cyclictest26660-21ntp_states08:20:260
4030993111,6cyclictest24719-21munin-run10:30:000
403099311,13cyclictest16013-21ssh12:00:300
403099310,4cyclictest81rcu_preempt10:15:360
255052310,9sleep1403199cyclictest11:04:041
13122312,6sleep10-21swapper/112:53:281
403199305,4cyclictest81rcu_preempt08:44:511
403199305,22cyclictest0-21swapper/110:17:131
4031993022,5cyclictest8302-21rm10:12:121
4031993021,5cyclictest12159-21diskmemload11:13:041
403199301,17cyclictest982-21seq12:19:301
403099305,4cyclictest81rcu_preempt10:14:220
403099302,5cyclictest81rcu_preempt12:41:170
403099302,5cyclictest81rcu_preempt10:06:300
4030993012,6cyclictest32491-21cat11:10:290
403099301,16cyclictest32669-21/usr/sbin/munin08:35:290
179872301,4sleep10-21swapper/107:58:491
167342301,5sleep10-21swapper/112:01:131
109952301,5sleep10-21swapper/112:29:521
104762301,5sleep10-21swapper/110:48:161
44452291,5sleep00-21swapper/008:48:170
403199299,16cyclictest0-21swapper/109:33:361
403199292,15cyclictest0-21swapper/112:36:091
403199291,7cyclictest29220-21ssh11:07:271
403199291,23cyclictest14609-21sed11:25:301
403099292,14cyclictest10994-21grep07:40:170
403099291,15cyclictest28170-21perl08:25:160
403099291,15cyclictest24636-21ssh11:02:470
403099291,13cyclictest25886-21ssh12:10:320
255482291,5sleep00-21swapper/008:19:060
238112291,5sleep10-21swapper/112:09:321
235352291,5sleep00-21swapper/008:12:430
83512281,5sleep00-21swapper/008:59:500
403199286,11cyclictest40950irq/93-i91511:51:431
403199284,3cyclictest81rcu_preempt08:17:521
403199283,4cyclictest81rcu_preempt09:18:061
4031992818,6cyclictest13861-21head09:10:321
4031992816,9cyclictest22-21ksoftirqd/112:32:591
4031992812,4cyclictest22-21ksoftirqd/108:11:031
403099289,6cyclictest9131-21hddtemp_smartct07:35:170
403099289,6cyclictest0-21swapper/009:07:440
4030992813,5cyclictest3-21ksoftirqd/008:43:500
403099281,14cyclictest7704-21/usr/sbin/munin07:30:480
292522281,5sleep10-21swapper/108:28:211
140222281,5sleep00-21swapper/007:48:590
117402281,5sleep10-21swapper/109:05:521
403199278,5cyclictest1969-21lldpad07:47:061
403199278,15cyclictest0-21swapper/112:40:331
403199275,18cyclictest4630-21munin-run08:49:591
403199271,7cyclictest4924-21ssh09:35:211
403099271,14cyclictest20356-21cpuspeed_turbos08:05:120
403199264,18cyclictest32659-21sed08:35:291
403199261,7cyclictest20362-21grep08:05:131
4031992614,9cyclictest22-21ksoftirqd/107:30:111
403099262,11cyclictest9381-21ntp_states09:00:240
403099261,11cyclictest30616-21switchtime08:30:280
4030992610,4cyclictest0-21swapper/011:35:240
403199241,4cyclictest81rcu_preempt09:04:311
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional