You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-16 - 00:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Tue Jul 15, 2025 12:59:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26429911181096,19cyclictest4624-21munin-node07:30:001
22762787733,31sleep10-21swapper/107:22:011
23752548501,32sleep00-21swapper/007:22:580
264199205181,5cyclictest4604-21munin-run07:30:000
224742700,7sleep0264199cyclictest11:35:250
2642996113,16cyclictest23146-21ntp_states09:55:281
264299595,15cyclictest26777-21chrt11:40:151
264299574,14cyclictest17737-21sh09:16:031
264299573,22cyclictest42-21kswapd011:45:401
264299573,12cyclictest28912-21runrttasks10:01:111
264299566,16cyclictest25170-21aten_r4power_po11:05:131
264299564,15cyclictest26578-21ssh12:46:371
199982560,4sleep1201rcuc/112:40:151
264299557,16cyclictest30558-21iostat11:10:211
264299557,16cyclictest30558-21iostat11:10:211
264299556,15cyclictest28583-21iostat_ios12:15:231
264299555,14cyclictest5901-21which07:30:181
264299554,15cyclictest18614-21timerandwakeup09:50:291
264299554,15cyclictest18538-21munin-run10:25:011
264299554,14cyclictest15829-21sh11:28:571
264299551,14cyclictest4557-21ssh12:24:291
264299547,13cyclictest18753-21cut12:05:171
264299546,15cyclictest9437-21ssh12:29:351
264299546,14cyclictest19916-21find10:25:281
264299545,16cyclictest32150-21df10:05:161
264299545,15cyclictest8982-21mv07:38:331
264299544,15cyclictest4730-21ssh11:16:511
264299544,15cyclictest30944-21snmpd09:43:521
264299543,17cyclictest11049-21taskset09:10:111
264299536,13cyclictest17511-21grep11:30:211
264299536,12cyclictest26846-21perl08:25:201
264299535,16cyclictest14893-21runrttasks07:55:001
264299534,15cyclictest2775-21ssh10:41:021
264299533,17cyclictest28908-21taskset12:50:051
264299533,14cyclictest4385-21runrttasks09:35:361
264299526,15cyclictest25886-21df09:25:161
264299525,15cyclictest19118-21taskset12:39:391
264299525,14cyclictest14107-21chrt12:00:191
264299524,15cyclictest24544-21aten_r4power_po08:20:131
264299524,15cyclictest21423-21iostat_ios09:20:191
264299524,14cyclictest13494-21ntp_states09:45:261
264299524,14cyclictest10795-21diskmemload10:55:061
264299523,30cyclictest23118-21users11:35:321
264299523,18cyclictest4726-21runrttasks08:50:501
264299523,16cyclictest32662-21taskset10:39:311
264299523,15cyclictest25469-21diskmemload10:31:031
264299516,14cyclictest24281-21sensors_temp12:10:321
264299515,19cyclictest8525-21rm10:14:321
264299515,16cyclictest8759-21ssh11:20:381
264299514,16cyclictest7423-21ssh11:53:411
264299514,15cyclictest13968-21rm10:53:171
264299514,14cyclictest11261-21rm11:57:151
264299505,21cyclictest9494-21df10:15:181
264299505,16cyclictest8415-21ssh10:47:131
264299505,15cyclictest20832-21hddtemp_smartct11:00:191
264299504,19cyclictest11868-21sh12:31:041
264299504,15cyclictest17557-21grep08:00:221
264299503,16cyclictest670-21chrt09:32:231
264299503,16cyclictest23173-21iostat08:15:191
264299494,15cyclictest1992-21cat08:45:201
2642994910,8cyclictest9720-21df_inode07:40:141
81072480,8sleep0264199cyclictest07:35:230
264299475,14cyclictest6691-21runrttasks08:56:431
264299465,14cyclictest8034-21awk09:00:291
264299464,15cyclictest28234-21/usr/sbin/munin08:30:261
264299464,14cyclictest16643-21runrttasks07:58:411
264299456,8cyclictest11420-21wget07:45:151
264299451,10cyclictest20395-21chrt08:08:281
117232450,4sleep0111rcuc/009:44:150
264299437,32cyclictest9236-21tr09:05:131
284362420,5sleep0111rcuc/012:15:190
264299428,16cyclictest3020-21runrttasks08:41:431
2641994111,15cyclictest2220-21dbus-daemon12:04:010
264299394,14cyclictest30944-21snmpd08:38:201
2642993924,9cyclictest21310-21iostat_ios08:10:201
264199386,5cyclictest81rcu_preempt10:05:180
2641993822,5cyclictest21066-21memory11:00:240
264199382,22cyclictest6093-21memory07:30:220
264199379,5cyclictest12450-21runrttasks11:59:000
264199372,22cyclictest5860-21/usr/sbin/munin12:25:200
264199365,4cyclictest81rcu_preempt12:45:150
264199357,14cyclictest81rcu_preempt09:24:590
264199353,5cyclictest81rcu_preempt10:21:330
2641993520,6cyclictest0-21swapper/010:27:460
2641993510,13cyclictest0-21swapper/009:35:390
100892350,3sleep041ktimersoftd/009:05:300
38302341,4sleep03841-21ssh10:42:340
264199345,4cyclictest81rcu_preempt09:34:530
264199344,4cyclictest81rcu_preempt11:41:530
2641993421,4cyclictest0-21swapper/012:23:010
264199341,20cyclictest17036-21ssh10:55:440
2641993410,6cyclictest23411-21expr12:10:170
128622340,4sleep0264199cyclictest10:51:400
264199336,4cyclictest81rcu_preempt09:55:450
264199336,4cyclictest81rcu_preempt09:10:310
264199334,4cyclictest81rcu_preempt12:35:520
264199333,8cyclictest14572-21munin-run12:35:010
264199333,4cyclictest81rcu_preempt12:40:300
264199333,4cyclictest81rcu_preempt11:11:290
264199333,4cyclictest81rcu_preempt11:11:280
264199333,4cyclictest81rcu_preempt10:02:340
264199332,4cyclictest81rcu_preempt11:08:520
2641993323,6cyclictest13441-21ssh09:45:260
264199332,19cyclictest10795-21diskmemload11:31:550
264199331,4cyclictest81rcu_preempt10:15:050
2641993311,11cyclictest16722-21sh09:15:230
2641993310,12cyclictest18721-21fschecks_time12:05:160
265762321,4sleep026645-21df08:25:180
264199324,4cyclictest81rcu_preempt12:54:020
264199324,4cyclictest81rcu_preempt10:36:190
264199323,4cyclictest81rcu_preempt11:51:590
264199323,4cyclictest81rcu_preempt11:29:330
264199322,4cyclictest81rcu_preempt11:23:040
264199322,4cyclictest81rcu_preempt10:45:090
264199321,4cyclictest81rcu_preempt07:51:320
110962321,27sleep011097-21sshd07:44:330
264199317,5cyclictest6348-21switchtime08:55:300
264199314,8cyclictest18009-21iostat_ios09:50:200
264199312,4cyclictest81rcu_preempt11:18:210
264199312,13cyclictest0-21swapper/010:32:380
264199311,8cyclictest31680-21ssh11:45:130
2641993112,6cyclictest25878-21/usr/sbin/munin09:25:140
2641993111,15cyclictest8209-21python09:00:310
2641993110,7cyclictest23827-21sed08:15:310
2641993011,4cyclictest6806-21ssh10:12:010
264199290,4cyclictest81rcu_preempt08:01:430
282202281,5sleep00-21swapper/008:30:030
264199283,13cyclictest0-21swapper/008:51:010
2641992810,5cyclictest15895-21ntp_states07:55:280
2641992810,5cyclictest11461-21cpu07:45:150
9892271,5sleep00-21swapper/008:42:160
257822271,5sleep00-21swapper/008:20:440
220292271,5sleep00-21swapper/008:10:580
205382271,5sleep00-21swapper/008:09:560
2641992610,4cyclictest1731-21/usr/sbin/munin08:45:170
314082251,4sleep00-21swapper/008:35:440
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional