You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-06 - 15:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Sat Dec 06, 2025 12:59:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
232129912081178,26cyclictest23573-21cut07:25:141
232129911801130,32cyclictest4661-21smartctl09:15:191
229302579521,42sleep10-21swapper/107:22:431
227532576530,32sleep00-21swapper/007:20:510
2321199238221,12cyclictest0-21swapper/007:25:130
2321199202178,20cyclictest4665-21ssh09:15:180
194212820,9sleep02321199cyclictest11:10:300
183072820,3sleep041ktimersoftd/010:02:530
2321299819,34cyclictest27548-21munin-plugin-st09:00:011
2321299819,31cyclictest9482-21ssh11:00:291
2321299751,6cyclictest30693-21taskset11:55:381
2321299751,41cyclictest4781-21ssh12:02:561
2321299742,6cyclictest2220-21dbus-daemon10:34:421
2321299741,5cyclictest17963-21/usr/sbin/munin12:50:201
2321299721,5cyclictest2485-21ssh09:13:121
23212997110,33cyclictest31319-21diskmemload09:30:031
2321299701,5cyclictest11556-21taskset08:15:261
2321299691,6cyclictest30497-21chrt11:22:241
2321299691,5cyclictest31479-21needreboot12:30:261
2321299681,5cyclictest21570-21chrt08:41:431
23212996814,32cyclictest22-21ksoftirqd/107:55:511
2321299672,44cyclictest3020-21runrttasks12:06:391
2321299671,6cyclictest9134-21chrt11:34:211
2321299671,6cyclictest15349-21seq12:14:181
2321299671,44cyclictest2220-21dbus-daemon09:27:401
2321299652,38cyclictest26640-21ssh10:45:161
2321299651,5cyclictest6190-21cat09:50:201
23212996515,32cyclictest14839-21uniq08:25:161
2321299647,36cyclictest26603-21memory12:25:251
2321299642,5cyclictest32603-21chrt11:25:161
2321299642,40cyclictest25214-21users08:50:381
2321299641,38cyclictest31319-21diskmemload11:09:321
23212996411,33cyclictest27599-21wc11:20:011
2321299636,33cyclictest31319-21diskmemload10:44:031
2321299633,39cyclictest31319-21diskmemload09:58:361
2321299633,32cyclictest0-21swapper/110:15:361
2321299632,4cyclictest0-21swapper/109:44:041
2321299632,36cyclictest31319-21diskmemload09:37:181
2321299632,35cyclictest12411-21munin-run12:45:011
2321299631,6cyclictest18837-21if_eth008:35:181
2321299631,10cyclictest13538-21needreboot12:45:211
2321299624,30cyclictest0-21swapper/112:36:371
2321299621,49cyclictest29146-21sh10:14:371
2321299621,43cyclictest5717-21ssh10:23:321
2321299621,43cyclictest5717-21ssh10:23:321
2321299621,39cyclictest11167-21ssh10:29:261
23212996210,34cyclictest18579-21ssh10:36:111
2321299617,32cyclictest25579-21ssh12:25:001
2321299616,35cyclictest31319-21diskmemload10:00:021
2321299613,35cyclictest31819-21iostat10:50:211
2321299612,39cyclictest3020-21runrttasks11:39:091
2321299612,32cyclictest0-21swapper/110:08:431
23212996115,27cyclictest22-21ksoftirqd/107:32:281
2321299611,41cyclictest15023-21df_inode11:40:161
23212996111,37cyclictest0-21swapper/112:15:441
2321299603,30cyclictest0-21swapper/109:47:001
2321299601,4cyclictest11477-21sh09:22:041
2321299594,34cyclictest12939-21diskstats08:20:161
2321299592,6cyclictest22503-21ssh11:47:441
2321299591,40cyclictest7625-21ssh10:59:141
2321299578,33cyclictest40950irq/93-i91508:30:221
2321299571,38cyclictest795-21seq07:49:301
2321299562,5cyclictest4034-21kworker/1:211:12:141
2321299562,34cyclictest0-21swapper/111:53:561
2321299552,34cyclictest23244-21grep08:45:331
2321299551,6cyclictest28512-21chrt07:35:231
2321299551,36cyclictest5108-21/usr/sbin/munin08:00:301
2321299551,32cyclictest7477-21hddtemp_smartct08:05:171
2321299542,31cyclictest29702-21cpu09:05:141
2321299541,6cyclictest10290-21watchsnmpd08:11:591
2321299541,33cyclictest31213-21taskset07:42:451
2321299531,32cyclictest27997-21df09:00:161
2321299531,31cyclictest1459-21fschecks_time07:50:171
2321199481,16cyclictest88982chrt09:53:150
82552440,7sleep02321199cyclictest10:25:320
253692430,4sleep0111rcuc/011:50:270
130862420,4sleep03-21ksoftirqd/012:45:150
2321199411,11cyclictest0-21swapper/009:21:260
2321199389,5cyclictest81rcu_preempt12:35:400
2321199381,15cyclictest0-21swapper/012:31:290
2321199379,4cyclictest81rcu_preempt10:51:460
23211993728,6cyclictest9620-21latency_hist11:35:010
23211993715,5cyclictest2087-21ssh09:12:390
2321199362,21cyclictest26106-21df_inode08:55:160
2321199352,21cyclictest28591-21/usr/sbin/munin09:00:280
112542351,30sleep011265-21ssh11:02:290
235572341,28sleep023571-21iostat_ios11:15:200
2321199345,5cyclictest81rcu_preempt10:41:490
23211993426,5cyclictest3-21ksoftirqd/007:53:550
23211993422,6cyclictest3-21ksoftirqd/010:17:050
2321199341,4cyclictest81rcu_preempt12:40:300
23211993411,4cyclictest0-21swapper/011:58:030
23211993410,4cyclictest81rcu_preempt11:46:500
276232331,5sleep00-21swapper/009:39:340
2321199332,17cyclictest1592-21ntp_states09:45:260
23211993319,4cyclictest0-21swapper/012:29:360
23211993315,6cyclictest21905-21idleruntime-cro08:45:010
2321199331,4cyclictest81rcu_preempt11:36:270
2321199331,12cyclictest12680-21/usr/sbin/munin10:30:250
2321199329,12cyclictest29747-21aten_r4power_en07:40:110
2321199328,4cyclictest81rcu_preempt11:25:310
2321199324,5cyclictest81rcu_preempt11:20:530
2321199324,4cyclictest81rcu_preempt12:10:290
2321199324,4cyclictest81rcu_preempt11:44:070
23211993224,6cyclictest0-21swapper/009:44:000
2321199322,26cyclictest18771-21ssh12:50:340
2321199322,17cyclictest7285-21memory12:05:250
2321199322,17cyclictest27165-21/usr/sbin/munin07:30:300
2321199321,25cyclictest25905-21ntp_states10:10:320
2321199321,19cyclictest3689-21ssh10:20:340
2321199321,19cyclictest3689-21ssh10:20:330
2321199321,19cyclictest3020-21runrttasks07:48:260
2321199321,19cyclictest19789-21mailstats09:30:340
2321199318,4cyclictest81rcu_preempt12:03:400
2321199316,12cyclictest3020-21runrttasks10:46:330
2321199315,5cyclictest81rcu_preempt10:09:350
2321199314,4cyclictest81rcu_preempt09:56:140
23211993121,6cyclictest16233-21cron08:30:010
2321199311,4cyclictest81rcu_preempt10:39:220
2321199311,4cyclictest81rcu_preempt08:38:030
23211993113,5cyclictest16457-21/usr/sbin/munin12:15:150
23211993113,5cyclictest13968-21irqstats11:05:220
48652301,5sleep00-21swapper/007:59:260
2321199303,4cyclictest81rcu_preempt10:58:140
2321199302,14cyclictest21484-21hddtemp_smartct12:20:160
2321199302,13cyclictest24442-21hddtemp_smartct08:50:260
2321199301,4cyclictest81rcu_preempt08:01:160
2321199301,17cyclictest0-21swapper/008:05:270
2321199293,4cyclictest81rcu_preempt09:26:290
2321199291,15cyclictest29087-21users07:35:320
23211992911,5cyclictest11235-21hddtemp_smartct08:15:190
311672281,5sleep00-21swapper/009:08:370
2321199281,12cyclictest0-21swapper/008:31:470
105282281,5sleep00-21swapper/008:14:310
143152271,5sleep00-21swapper/008:24:380
2321199257,10cyclictest0-21swapper/008:48:490
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional