You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-01 - 04:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack7slot4s.osadl.org (updated Tue Jul 01, 2025 00:45:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1573799801,69cyclictest0-21swapper/219:30:152
15745996040,15cyclictest32466irq/58-eth021:45:183
1572899521,42cyclictest2084-21ssh23:30:131
1573799502,43cyclictest27027-21apt-get19:55:152
1573799501,42cyclictest23346irq/40-dwc2_hso23:05:172
1573799490,43cyclictest25546-21apt-get19:20:192
1573799490,43cyclictest25546-21apt-get19:20:182
1573799461,41cyclictest30356-21apt-get19:25:152
1573799460,41cyclictest0-21swapper/200:35:132
1573799451,41cyclictest15881-21apt-get19:10:132
1573799450,38cyclictest0-21swapper/220:30:172
1573799442,37cyclictest23346irq/40-dwc2_hso22:20:172
1573799441,40cyclictest0-21swapper/223:40:142
1573799430,34cyclictest0-21swapper/222:55:152
1574599422,27cyclictest32466irq/58-eth022:10:203
1573799421,36cyclictest0-21swapper/222:25:182
1573799420,39cyclictest0-21swapper/200:00:142
1573799411,34cyclictest31830-21apt-get20:00:132
1573799410,36cyclictest0-21swapper/223:45:152
1573799401,35cyclictest26723-21if_err_eth123:25:182
1573799401,34cyclictest0-21swapper/222:15:152
1573799395,31cyclictest6454-21cat23:30:322
1573799391,35cyclictest19517-21kthreadcore20:20:242
1573799391,34cyclictest15090-21ssh22:35:412
1573799391,34cyclictest0-21swapper/222:05:162
1574599384,21cyclictest32466irq/58-eth021:20:163
1574599383,18cyclictest32466irq/58-eth000:10:103
1574599381,19cyclictest32466irq/58-eth000:30:153
1573799381,33cyclictest26623-21/usr/sbin/munin21:45:342
1573799381,33cyclictest0-21swapper/219:35:182
1573799381,32cyclictest0-21swapper/200:30:142
15728993824,10cyclictest0-21swapper/100:00:121
1572099388,25cyclictest12-21ksoftirqd/022:10:180
1573799371,33cyclictest15525-21df_inode20:50:162
1573799370,33cyclictest10582-21apt-get21:35:192
1573799370,33cyclictest0-21swapper/221:30:152
1573799370,33cyclictest0-21swapper/220:55:122
1573799362,31cyclictest29903-21apt-get00:10:102
1573799361,32cyclictest9278-21ntp_states20:40:272
1573799361,32cyclictest20725-21missed_timers19:45:232
1573799361,31cyclictest17486-21ntp_states20:15:302
1573799361,30cyclictest0-21swapper/221:20:162
1573799361,27cyclictest26401-21apt-get22:45:162
1573799361,27cyclictest26401-21apt-get22:45:152
15737993611,13cyclictest18419-21sh22:00:132
1573799360,33cyclictest4252-21memory20:35:232
1573799360,29cyclictest0-21swapper/221:40:132
1572099363,22cyclictest12-21ksoftirqd/019:55:130
1573799351,31cyclictest9876-21grep23:55:012
1573799351,31cyclictest2462-21sed21:50:182
1573799351,27cyclictest15169-21users20:45:312
1573799350,32cyclictest2316-21fschecks_time22:30:182
1573799350,30cyclictest0-21swapper/223:15:122
1573799350,28cyclictest7914-21/usr/sbin/munin20:05:262
1573799350,28cyclictest12581-21ntp_192.168.11420:10:272
1572099352,28cyclictest18368-21ssh23:20:170
1572099351,31cyclictest18234-21sh22:40:170
1574599343,19cyclictest32466irq/58-eth022:25:163
1573799342,26cyclictest0-21swapper/221:10:152
1573799342,24cyclictest5540-21grep23:10:252
1573799341,30cyclictest5896-21if_eth100:15:182
1573799341,29cyclictest22947-21kthreadcore19:50:202
1573799341,28cyclictest13868-21apt-get00:20:152
1573799341,25cyclictest25831-21mailstats00:25:292
1573799341,25cyclictest15073-21users23:55:322
15737993411,21cyclictest24498-21ssh23:02:392
1573799340,31cyclictest0-21swapper/222:40:142
1573799340,25cyclictest10296-21grep23:35:152
1572099342,27cyclictest18800-21ssh21:20:160
1573799331,29cyclictest9963-21cron22:55:012
1573799331,29cyclictest22455-21unixbench_singl23:20:342
1573799331,29cyclictest10047-21ssh21:55:052
1573799330,30cyclictest0-21swapper/221:00:152
1574599323,19cyclictest32466irq/58-eth021:00:163
15745993225,4cyclictest32466irq/58-eth023:25:153
1573799321,28cyclictest16018-21ntp_states19:40:322
1573799320,29cyclictest2089-21diskmemload00:09:452
1573799320,29cyclictest14123-21ssh21:15:262
1573799320,22cyclictest0-21swapper/222:13:312
1572899322,21cyclictest662-21ntpd23:10:151
1572099321,25cyclictest10228-21ssh23:55:130
1573799311,27cyclictest24388-21/usr/sbin/munin19:15:272
1573799311,26cyclictest27108-21if_eth121:25:182
1573799310,28cyclictest27766-21usage_Anybus20:25:312
15728993118,6cyclictest0-21swapper/121:25:161
1572099313,19cyclictest10609-21ssh21:35:180
1572099311,23cyclictest26395-21ssh23:05:150
1574599303,19cyclictest0-21swapper/300:20:173
1573799301,26cyclictest863-21missed_timers21:05:252
1572899303,20cyclictest10141-21rm23:15:151
1572099305,19cyclictest12-21ksoftirqd/022:45:170
1572099305,19cyclictest12-21ksoftirqd/022:45:160
1572099301,23cyclictest26375-21ssh22:25:170
1574599298,13cyclictest32466irq/58-eth021:35:183
1574599293,15cyclictest32466irq/58-eth021:15:143
1572899293,19cyclictest18374-21ssh23:40:141
1572899292,19cyclictest26422-21ssh23:25:161
1572099294,17cyclictest10229-21ssh22:35:160
1572099292,22cyclictest5575-21ssh00:15:130
15720992917,10cyclictest12-21ksoftirqd/019:25:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional