You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-17 - 23:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack7slot4s.osadl.org (updated Mon Nov 17, 2025 12:45:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
47499574,36cyclictest25178-21ssh11:40:140
47499564,39cyclictest13393-21ssh12:30:160
495995331,15cyclictest0-21swapper/210:40:132
48499521,43cyclictest24971-21ssh09:55:161
49599514,16cyclictest131rcu_preempt09:20:172
49599514,16cyclictest131rcu_preempt09:20:162
484994918,20cyclictest21-21ksoftirqd/108:05:161
48499472,7cyclictest0-21swapper/107:50:151
48499472,31cyclictest9122-21ssh11:50:141
50499454,21cyclictest29566irq/58-eth011:00:163
504994518,19cyclictest29566irq/58-eth010:00:163
48499434,28cyclictest1593-21ssh09:40:141
47499421,33cyclictest1814-21sh11:25:140
474994114,7cyclictest131rcu_preempt08:50:130
48499405,28cyclictest25843-21diskmemload12:00:151
49599393,11cyclictest131rcu_preempt10:10:182
50499376,12cyclictest29666irq/59-eth010:40:143
504993715,10cyclictest29566irq/58-eth007:25:143
49599371,17cyclictest18978-21ssh12:15:162
50499363,19cyclictest29566irq/58-eth010:05:153
50499363,19cyclictest29566irq/58-eth010:05:153
48499367,20cyclictest13902-21ssh10:30:141
48499366,6cyclictest6944-21kthreadcore10:25:211
484993627,5cyclictest0-21swapper/111:55:161
484993627,5cyclictest0-21swapper/111:55:151
48499361,9cyclictest27868-21grep08:45:191
48499361,11cyclictest27399-21latency_hist08:45:011
48499361,10cyclictest2170-21fschecks_time07:45:181
48499361,10cyclictest17161-21kthreadcore07:25:221
48499352,8cyclictest0-21swapper/110:40:131
48499351,10cyclictest12925-21tr11:10:221
47499353,25cyclictest17166-21ssh09:50:150
49599346,8cyclictest0-21swapper/211:25:162
495993421,3cyclictest0-21swapper/209:35:142
48499342,11cyclictest7105-21kthreadcore10:45:241
484993412,19cyclictest6008-21sed10:25:011
474993427,3cyclictest131rcu_preempt11:45:150
49599331,21cyclictest26354-21ssh11:20:132
48499339,3cyclictest512-21sh11:02:221
48499337,4cyclictest4864-21kthreadcore08:20:221
48499332,22cyclictest749-21ssh11:45:151
48499331,26cyclictest26159-21ssh09:35:141
48499331,10cyclictest1885-21latency_hist07:45:011
48499331,10cyclictest12263-21kthreadcore09:45:221
47499331,25cyclictest27488-21ssh09:15:150
49599324,9cyclictest261rcuc/210:50:162
48499328,3cyclictest19156-21diskmemload10:51:281
48499327,4cyclictest19098-21fschecks_count12:15:181
48499321,9cyclictest9565-21latency_hist09:00:021
48499321,9cyclictest4844-21irqcore12:25:201
48499321,9cyclictest4844-21irqcore12:25:191
48499321,9cyclictest3210-21latency_hist08:20:021
48499321,9cyclictest2206-21cut12:05:221
48499321,9cyclictest16087-21irqrtprio10:10:221
48499321,9cyclictest11188-21netstat10:05:271
48499321,9cyclictest11188-21netstat10:05:261
48499321,8cyclictest20389-21kthreadcore11:35:241
48499321,11cyclictest8357-21fschecks_count08:25:181
48499321,10cyclictest9849-21ssh11:30:181
48499321,10cyclictest15829-21kthreadcore09:05:221
48499320,10cyclictest13195-21ssh09:25:241
48499320,10cyclictest13195-21ssh09:25:231
50499312,13cyclictest29566irq/58-eth011:05:133
48499318,4cyclictest11791-21kthreadcore07:20:221
48499317,5cyclictest19156-21diskmemload09:15:471
48499313,10cyclictest26730-21ssh11:40:201
48499311,9cyclictest20365-21kthreadcore09:30:221
48499311,10cyclictest25049-21tr12:35:271
48499311,10cyclictest14553-21cut12:30:221
48499311,10cyclictest13300-21hwlatdetect08:30:181
50499304,15cyclictest29566irq/58-eth012:25:143
50499304,15cyclictest29566irq/58-eth012:25:133
49599304,16cyclictest26346irq/40-dwc2_hso09:40:152
495993022,4cyclictest2746ksoftirqd/207:45:152
48499302,20cyclictest32437-21ssh10:00:171
48499301,9cyclictest643-21/usr/sbin/munin07:10:131
48499301,9cyclictest18697-21kthreadcore08:35:211
48499301,11cyclictest3912-21grep08:50:291
47499304,21cyclictest12-21ksoftirqd/012:20:150
47499304,21cyclictest12-21ksoftirqd/012:20:150
474993013,11cyclictest12-21ksoftirqd/011:15:150
49599293,19cyclictest26346irq/40-dwc2_hso08:25:142
49599293,15cyclictest26346irq/40-dwc2_hso09:55:182
49599293,15cyclictest26346irq/40-dwc2_hso07:55:162
49599291,20cyclictest18427-21ssh09:30:142
49599291,18cyclictest27-21ksoftirqd/207:10:132
48499291,9cyclictest29549-21latency_hist07:40:011
48499291,9cyclictest26492-21tune2fs11:20:151
48499291,11cyclictest28438-21kthreadcore08:10:211
48499291,11cyclictest21650-21ssh10:35:151
48499291,10cyclictest13179-21tr12:10:241
484992910,5cyclictest26082-21ssh09:13:211
48499290,10cyclictest13688-21cut07:55:241
47499296,20cyclictest12-21ksoftirqd/011:05:150
474992915,9cyclictest12-21ksoftirqd/010:30:140
47499291,22cyclictest6217-21ssh10:25:150
50499283,16cyclictest29566irq/58-eth009:35:153
50499282,17cyclictest0-21swapper/311:50:153
49599284,15cyclictest26346irq/40-dwc2_hso08:35:142
49599283,17cyclictest457-21cyclictest07:15:142
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional