You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-24 - 21:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack7slot4s.osadl.org (updated Fri Oct 24, 2025 12:45:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1205999581,43cyclictest26423-21ssh09:50:140
12079995019,23cyclictest0-21swapper/208:25:132
12079994928,13cyclictest30008-21diskmemload12:00:162
1205999473,36cyclictest28302-21ssh12:35:140
12059994524,10cyclictest131rcu_preempt10:05:120
12059994524,10cyclictest131rcu_preempt10:05:120
1208799412,24cyclictest25366irq/58-eth011:10:143
12087994018,15cyclictest25366irq/58-eth012:20:153
1205999402,28cyclictest24060-21ssh11:55:180
1206799393,10cyclictest17771-21ssh11:10:141
1206799391,12cyclictest3037-21grep11:40:281
1208799383,24cyclictest25366irq/58-eth010:45:123
1207999382,28cyclictest24001-21cat12:30:302
1207999382,28cyclictest24001-21cat12:30:302
1206799381,14cyclictest23644-21ssh11:35:021
1205999383,31cyclictest15096-21sh10:25:120
12059993810,22cyclictest12-21ksoftirqd/012:25:140
1206799378,5cyclictest30342-21fschecks_count10:35:201
1206799371,10cyclictest19660-21kthreadcore10:05:211
1206799371,10cyclictest19660-21kthreadcore10:05:211
1205999371,33cyclictest16943-21ssh12:10:120
1208799363,22cyclictest25366irq/58-eth007:45:133
1207999368,17cyclictest27-21ksoftirqd/212:35:142
1206799361,11cyclictest7419-21kthreadcore09:15:221
1206799361,11cyclictest4720-21ntp_states11:20:281
1206799361,11cyclictest20581-21kthreadcore10:50:221
1206799361,11cyclictest13845-21awk11:05:261
12059993612,17cyclictest12-21ksoftirqd/011:05:150
1208799351,22cyclictest25366irq/58-eth009:10:133
1208799351,22cyclictest25366irq/58-eth009:10:133
12079993514,11cyclictest3804-21kthreadcore12:20:232
1206799357,6cyclictest16609-21cut10:25:201
1206799351,9cyclictest9620-21kthreadcore10:20:201
1206799351,9cyclictest31484-21wc11:40:021
12067993517,12cyclictest22948-21cut12:30:231
12067993517,12cyclictest22948-21cut12:30:231
1206799351,11cyclictest31241-21missed_timers08:00:251
1206799351,11cyclictest29332-21cut09:50:241
1206799351,10cyclictest9824-21kthreadcore12:05:191
1206799351,10cyclictest23923-21kthreadcore10:30:211
1206799351,10cyclictest14410-21idleruntime-cro08:20:011
12059993518,13cyclictest12-21ksoftirqd/009:05:140
1205999351,29cyclictest13188-21ssh09:20:140
1206799341,9cyclictest8723-21apt-get11:25:131
1206799341,16cyclictest29392-21ssh11:15:301
1206799341,10cyclictest19246-21latency_hist12:30:011
1205999345,26cyclictest7495-21ssh11:45:130
12059993422,6cyclictest528-21usb-storage07:40:100
1208799331,19cyclictest25366irq/58-eth010:20:113
1207999332,20cyclictest27-21ksoftirqd/207:15:102
1206799336,18cyclictest18778-21ssh09:45:131
1206799331,26cyclictest0-21swapper/107:30:121
1206799331,26cyclictest0-21swapper/107:30:121
1206799331,25cyclictest30400-21ssh09:10:141
1206799331,25cyclictest30400-21ssh09:10:141
1206799331,10cyclictest28992-21latency_hist08:35:011
1206799331,10cyclictest2483-21kthreadcore09:55:191
1206799331,10cyclictest12005-21fschecks_time10:45:161
1206799327,5cyclictest28072-21timerwakeupswit11:55:321
1206799321,11cyclictest21775-21kthreadcore08:25:221
1206799321,11cyclictest1370-21cut12:00:231
1206799321,11cyclictest11609-21irqrtprio08:50:191
1206799321,11cyclictest10622-21tr10:00:231
1206799321,11cyclictest10429-21cron12:25:011
1206799321,10cyclictest7252-21sendmail10:20:011
12067993210,6cyclictest29440-1kworker/u9:012:10:141
1205999328,19cyclictest12-21ksoftirqd/008:30:130
1205999321,25cyclictest28273-21ssh09:30:110
1205999321,25cyclictest28273-21ssh09:30:110
12059993212,3cyclictest12-21ksoftirqd/011:10:140
1207999313,19cyclictest21846irq/40-dwc2_hso10:50:162
1207999313,17cyclictest21846irq/40-dwc2_hso09:40:142
1206799312,9cyclictest22107-21tail07:50:291
1206799311,12cyclictest7542-21cat07:35:301
1206799311,11cyclictest4153-21kthreadcore11:00:181
1206799311,10cyclictest8516-21kthreadcore11:45:201
1206799311,10cyclictest5553-21tr08:10:181
1206799311,10cyclictest32008-21vmstat10:55:341
1206799311,10cyclictest25548-21ssh10:10:211
1206799311,10cyclictest13320-21fschecks_count09:20:161
1206799311,10cyclictest12636-21kthreadcore09:40:201
1206799311,10cyclictest10954-21latency_hist09:40:011
1206799311,10cyclictest10954-21latency_hist09:40:011
1205999318,9cyclictest131rcu_preempt07:20:120
12059993118,8cyclictest12-21ksoftirqd/012:20:160
1208799303,18cyclictest25366irq/58-eth008:45:143
12087993015,11cyclictest25366irq/58-eth010:40:423
1207999308,12cyclictest0-21swapper/211:40:142
1207999304,19cyclictest21846irq/40-dwc2_hso09:30:132
1207999304,19cyclictest21846irq/40-dwc2_hso09:30:132
1207999302,21cyclictest0-21swapper/211:05:152
1207999302,17cyclictest17780-21ssh11:10:152
1206799305,6cyclictest0-21swapper/109:25:141
1206799301,9cyclictest9367-21sed07:40:171
1206799301,9cyclictest4766-21fschecks_time10:40:171
1206799301,9cyclictest29055-21tr12:35:201
1206799301,11cyclictest29827-21grep12:15:291
1206799301,11cyclictest29744-21kthreadcore08:35:211
1206799301,11cyclictest29744-21kthreadcore08:35:211
1206799301,11cyclictest25704-21sed07:20:341
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional