You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-13 - 17:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack7slot4s.osadl.org (updated Sat Sep 13, 2025 12:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2356799645,45cyclictest9045-21ssh11:40:151
23578995511,5cyclictest131rcu_preempt09:55:132
23578995346,4cyclictest13840-21kworker/u8:211:50:142
2356799532,5cyclictest131rcu_preempt08:25:141
2355799526,30cyclictest131rcu_preempt07:45:150
2357899511,27cyclictest30140-21ssh09:45:162
2357899468,29cyclictest0-21swapper/211:20:132
23578994615,24cyclictest21846irq/40-dwc2_hso07:25:142
2356799464,29cyclictest16357-21ssh11:45:151
2356799451,33cyclictest27861-21ssh10:05:151
23557994513,22cyclictest12-21ksoftirqd/007:15:140
23593994312,21cyclictest25366irq/58-eth011:15:163
2356799434,6cyclictest0-21swapper/110:20:151
2355799419,23cyclictest12-21ksoftirqd/011:55:140
2355799411,6cyclictest131rcu_preempt08:00:130
23557994112,21cyclictest12-21ksoftirqd/009:10:170
23557994112,21cyclictest12-21ksoftirqd/009:10:160
2359399403,22cyclictest25366irq/58-eth012:10:143
2357899402,21cyclictest22569-21sh09:40:182
23567994011,23cyclictest21-21ksoftirqd/109:45:171
2355799402,6cyclictest27689-21apt-get12:10:150
2355799402,28cyclictest20344-21ssh10:00:140
2357899394,14cyclictest5404-21ssh09:50:162
23578993919,15cyclictest4090-21apt-get12:15:162
2356799391,33cyclictest15632-21ssh10:40:121
2355799386,22cyclictest12-21ksoftirqd/011:25:140
23578993720,10cyclictest29786-21apt-get07:50:132
23578993616,14cyclictest10001-21apt-get12:00:132
23557993611,22cyclictest12-21ksoftirqd/010:25:130
2359399354,20cyclictest25366irq/58-eth011:50:133
2355799351,31cyclictest975-21sshd07:20:140
2359399343,18cyclictest25366irq/58-eth010:50:153
2356799346,16cyclictest0-21swapper/110:35:151
2356799343,28cyclictest10012-21ssh12:00:141
2356799342,24cyclictest6850-21ssh12:35:151
2359399336,16cyclictest25366irq/58-eth010:25:133
23593993324,6cyclictest25366irq/58-eth012:20:163
2359399332,22cyclictest25366irq/58-eth010:45:143
2359399332,16cyclictest25366irq/58-eth008:35:123
2357899331,18cyclictest21846irq/40-dwc2_hso10:35:162
2356799331,29cyclictest30455-21ssh12:30:141
2356799331,12cyclictest1296-21ps07:20:191
23578993216,12cyclictest14845-21ssh09:35:152
2356799329,7cyclictest26057-21grep09:40:261
2356799321,12cyclictest1329-21/usr/sbin/munin11:55:181
2359399314,14cyclictest25366irq/58-eth012:25:133
2357899318,18cyclictest26509-21rm11:30:142
2357899318,17cyclictest0-21swapper/211:00:182
23578993117,9cyclictest15638-21ssh10:40:132
2357899311,16cyclictest942-21snmpd07:10:152
2356799312,8cyclictest16586-21ntp_states12:20:281
2356799311,11cyclictest434-21munin-run10:30:011
2355799311,18cyclictest12-21ksoftirqd/007:10:140
2359399302,16cyclictest25366irq/58-eth010:20:143
2357899309,16cyclictest0-21swapper/209:25:122
2357899303,17cyclictest21846irq/40-dwc2_hso08:25:152
2357899303,15cyclictest21846irq/40-dwc2_hso08:55:162
2357899303,14cyclictest0-21swapper/212:26:352
23578993019,7cyclictest6818-21apt-get12:35:142
2357899301,15cyclictest14258-21grep12:20:212
2356799303,20cyclictest1362-21ssh11:35:151
2355799301,3cyclictest838-21ssh10:30:160
2357899292,20cyclictest0-21swapper/210:00:152
2357899292,18cyclictest20924-21ssh11:05:152
2357899292,18cyclictest20924-21ssh11:05:152
23578992920,5cyclictest1354-21apt-get11:35:142
2357899291,15cyclictest16483-21diskstats11:45:172
23578992911,13cyclictest0-21swapper/211:10:112
2359399283,15cyclictest25366irq/58-eth011:40:163
2357899284,16cyclictest6722-21ntp_192.168.11408:30:242
2357899282,16cyclictest21846irq/40-dwc2_hso07:40:142
2357899282,13cyclictest0-21swapper/210:10:222
2357899282,10cyclictest21761-21sed10:20:302
2357899281,18cyclictest168-21systemd-network08:50:142
2357899281,17cyclictest29637-21ntp_states08:20:262
2357899281,16cyclictest9129-21diskmemload11:25:152
2356799283,10cyclictest27702-21ssh12:10:151
2356799281,20cyclictest7046-21ssh09:30:151
2355799283,14cyclictest12-21ksoftirqd/008:20:130
2355799282,15cyclictest24552-21ssh09:20:140
23557992819,3cyclictest0-21swapper/009:05:130
23593992719,6cyclictest321rcuc/308:25:153
2357899272,20cyclictest0-21swapper/209:30:152
2357899272,19cyclictest0-21swapper/209:05:142
2357899272,11cyclictest4387-21ntp_kernel_err07:20:272
2357899271,14cyclictest16695-21fschecks_time08:10:162
23578992710,12cyclictest0-21swapper/208:05:152
2357899270,12cyclictest499-21kthreadcore09:00:222
2356799271,11cyclictest31361-21grep11:10:201
2356799271,11cyclictest12493-21idleruntime-cro09:55:001
2355799272,18cyclictest12-21ksoftirqd/007:55:140
2357899269,14cyclictest24182-21ls09:20:012
2357899269,14cyclictest24182-21ls09:20:002
2357899269,13cyclictest0-21swapper/210:05:142
2357899265,13cyclictest0-21swapper/212:10:032
2357899263,8cyclictest0-21swapper/210:50:572
23578992615,7cyclictest0-21swapper/208:45:122
2357899261,12cyclictest23665-21ls07:10:012
2359399253,11cyclictest25366irq/58-eth009:40:153
23593992510,11cyclictest25366irq/58-eth009:10:163
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional