You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-21 - 02:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack9slot4s (updated Thu Nov 20, 2025 12:59:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
279889912661236,23cyclictest28153-21wget07:25:111
279889912451214,26cyclictest32431-21apt-get07:35:011
275742465421,31sleep10-21swapper/107:21:201
275032465369,29sleep00-21swapper/007:20:360
2798799256232,21cyclictest0-21swapper/007:25:100
2798799224198,20cyclictest32549-21latency_hist07:35:020
2338521040,6sleep0111rcuc/010:36:310
274702850,5sleep02798799cyclictest09:34:280
256642750,9sleep02798799cyclictest11:12:540
147052490,3sleep00-21swapper/012:42:030
27987994326,6cyclictest30205-21iostat10:10:220
27987994225,6cyclictest29352-21sort10:10:010
27987994225,6cyclictest11226-21memory09:50:400
27987994225,5cyclictest21146-21sed08:30:020
27987994224,13cyclictest24424-21ssh10:05:000
243742420,4sleep124352-21ssh11:45:201
97482410,17sleep041ktimersoftd/010:22:410
27987994125,6cyclictest30852-21ssh10:45:090
27987994125,6cyclictest11945-21diskstats10:25:150
27987994124,6cyclictest3980-21sort10:50:190
27987994123,7cyclictest0-21swapper/008:42:330
27987994118,8cyclictest3-21ksoftirqd/012:03:470
27987994024,6cyclictest30754-21irqstats12:25:180
231912401,35sleep123202-21ssh10:36:141
27988993927,8cyclictest4054-21ssh10:50:221
27988993921,13cyclictest2206-21df09:05:111
27987993924,5cyclictest11914-21sendmail12:40:000
27987993923,6cyclictest2805-21grep10:15:230
27987993923,6cyclictest27244-21fschecks_time08:45:160
27987993923,6cyclictest18736-21ntpq11:05:230
27987993923,6cyclictest14828-21ssh11:35:500
27987993815,7cyclictest3-21ksoftirqd/009:00:190
112812381,33sleep111295-21ssh10:57:561
2798799377,17cyclictest0-21swapper/009:17:090
27987993727,5cyclictest4703-21iostat09:10:200
27987993712,13cyclictest0-21swapper/012:13:180
2798899367,5cyclictest27904-21ls09:35:011
2798899362,7cyclictest3992-21users12:30:301
27988993623,8cyclictest1304-21ssh11:20:381
27987993623,4cyclictest4996-21expr11:25:170
27987993622,4cyclictest16756-21ssh10:30:160
27987993622,4cyclictest16756-21ssh10:30:150
27987993619,7cyclictest0-21swapper/009:23:040
2798799361,4cyclictest81rcu_preempt12:50:000
27987993614,10cyclictest21250-21basename08:30:110
2798899357,13cyclictest40950irq/93-i91509:45:071
2798899356,13cyclictest40950irq/93-i91512:45:231
2798899353,15cyclictest0-21swapper/111:14:531
2798899352,20cyclictest0-21swapper/111:01:081
27987993522,9cyclictest9853-21ssh11:30:180
27987993522,9cyclictest12716-21munin-run11:00:010
27987993521,9cyclictest27697-21df_inode11:15:140
27987993521,10cyclictest3502-21ssh11:23:400
27987993520,5cyclictest12310-21if_err_eth008:05:170
27987993513,7cyclictest3-21ksoftirqd/008:11:590
184742351,4sleep13497-1kworker/1:1H11:05:201
27988993411,5cyclictest40950irq/93-i91512:36:471
27988993410,5cyclictest40950irq/93-i91511:43:321
27988993410,5cyclictest40950irq/93-i91509:29:411
2798799349,5cyclictest21823-21df12:50:130
2798799349,15cyclictest0-21swapper/011:40:520
27987993422,3cyclictest0-21swapper/011:55:340
27987993421,5cyclictest0-21swapper/011:00:220
27987993421,5cyclictest0-21swapper/008:55:490
27987993417,6cyclictest21596-21ssh09:27:540
27987993415,7cyclictest0-21swapper/007:36:480
27987993415,6cyclictest5774-21df_inode09:45:170
2798799341,19cyclictest3020-21runrttasks11:50:540
2798799341,15cyclictest17995-21if_eth008:20:200
2798799341,14cyclictest20916-21fschecks_time12:15:160
273562340,2sleep027357-21rm11:48:230
2798899338,10cyclictest40950irq/93-i91510:47:391
2798899333,12cyclictest40950irq/93-i91511:30:071
27988993311,5cyclictest40950irq/93-i91512:15:161
27988993310,6cyclictest40950irq/93-i91507:50:141
2798799338,15cyclictest0-21swapper/007:41:380
2798799332,4cyclictest0-21swapper/012:30:300
27987993321,8cyclictest1472-21ntp_states09:40:270
27987993321,3cyclictest0-21swapper/007:45:240
27987993313,6cyclictest3-21ksoftirqd/012:06:410
27987993312,11cyclictest0-21swapper/008:50:260
2798799331,14cyclictest16324-21sort09:55:300
2798899329,5cyclictest40950irq/93-i91510:10:001
2798899329,5cyclictest40950irq/93-i91508:50:161
2798899329,19cyclictest0-21swapper/111:55:061
2798899328,6cyclictest40950irq/93-i91512:26:121
2798899328,6cyclictest40950irq/93-i91510:20:241
2798899324,4cyclictest40950irq/93-i91509:40:371
2798899323,11cyclictest0-21swapper/112:40:541
2798899322,5cyclictest0-21swapper/109:38:401
2798899322,13cyclictest0-21swapper/112:53:041
2798899321,6cyclictest15222-21ssh10:28:391
2798799322,5cyclictest27210-21runrttasks12:21:120
27987993220,4cyclictest0-21swapper/008:15:200
27987993211,7cyclictest3-21ksoftirqd/009:08:400
179502321,5sleep10-21swapper/110:30:371
179502321,5sleep10-21swapper/110:30:371
107752321,4sleep110786-21ssh12:05:061
95882311,26sleep19598-21ssh12:03:361
291762311,5sleep10-21swapper/111:16:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional