You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-17 - 12:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack9slot4s (updated Mon Nov 17, 2025 00:59:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
131789911971182,12cyclictest0-21swapper/119:35:181
131779911231092,21cyclictest0-21swapper/019:25:070
131789911081070,32cyclictest13322-21basename19:25:111
127402564517,31sleep00-21swapper/019:21:060
129042466419,32sleep10-21swapper/119:22:481
13177999083,4cyclictest0-21swapper/019:35:170
320882830,7sleep01317799cyclictest23:00:390
1317899736,18cyclictest1476-21ssh00:10:211
1317899736,18cyclictest1476-21ssh00:10:201
13178996610,19cyclictest14655-21hddtemp_smartct23:50:181
1317899619,16cyclictest26112-21sh22:21:081
1317899618,15cyclictest8947-21rm22:37:091
1317899607,16cyclictest30211-21sh00:05:481
1317899607,15cyclictest26227-21ssh23:29:041
1317899606,21cyclictest369-21ssh23:35:241
13178996010,16cyclictest17838-21diskstats23:20:171
1317899599,15cyclictest21015-21diskmemload22:48:181
1317899598,20cyclictest21863-21ntp_states22:50:271
1317899597,21cyclictest12331-21taskset21:34:171
1317899597,17cyclictest24345-21taskset21:12:501
1317899596,17cyclictest28234-21ssh22:57:051
1317899595,14cyclictest27598-21/usr/sbin/munin21:50:271
1317899595,14cyclictest10338-21sh00:53:241
1317899589,14cyclictest137350irq/90-eth023:10:201
1317899588,14cyclictest6088-21latency_hist22:35:011
1317899585,22cyclictest23764-21kworker/1:000:04:251
1317899582,23cyclictest8599-21ssh22:03:451
1317899578,17cyclictest30944-21snmpd20:06:061
1317899578,14cyclictest14009-21ssh23:15:321
1317899578,14cyclictest11048-21df_inode00:20:181
1317899577,15cyclictest17068-21chrt20:58:241
1317899576,20cyclictest21015-21diskmemload23:43:531
1317899576,16cyclictest6931-21latency_hist00:50:011
1317899575,29cyclictest0-21swapper/121:41:351
1317899569,15cyclictest20716-21diskstats00:30:191
1317899568,16cyclictest9964-21if_eth023:45:221
1317899568,15cyclictest4649-21python21:25:301
1317899568,15cyclictest3991-21chrt20:24:281
1317899568,15cyclictest15053-21taskset21:36:131
1317899568,14cyclictest16620-21/usr/sbin/munin00:25:291
1317899567,16cyclictest14791-21taskset22:43:351
1317899567,15cyclictest23708-21ntpq21:45:261
1317899567,15cyclictest13027-21ssh22:08:111
1317899563,23cyclictest31935-21ssh00:41:111
1317899562,22cyclictest32166-21ssh23:00:451
1317899558,14cyclictest22662-21ssh23:58:511
1317899557,14cyclictest30817-21cat21:20:011
1317899556,16cyclictest27999-21grep23:30:261
1317899556,14cyclictest31730-21ntp_states20:10:271
1317899552,30cyclictest0-21swapper/121:57:041
1317899552,22cyclictest9093-21ssh00:18:061
1317899548,16cyclictest31017-21/usr/sbin/munin21:20:251
1317899547,15cyclictest22890-21runrttasks19:46:481
1317899547,14cyclictest19413-21aten_r4power_en22:15:111
1317899546,15cyclictest9393-21chrt20:37:011
1317899546,14cyclictest4202-21tr23:05:331
1317899546,14cyclictest21015-21diskmemload22:27:401
1317899546,14cyclictest21015-21diskmemload00:37:421
1317899542,21cyclictest19041-21ssh22:14:551
1317899537,15cyclictest15271-21chrt20:54:161
1317899507,14cyclictest20459-21awk19:40:301
1317899506,14cyclictest27978-21ntpq20:00:291
1317899497,15cyclictest13374-21runrttasks20:49:021
1317899493,16cyclictest15838-21chrt19:30:131
1317899488,17cyclictest21123-21cat21:10:001
1317899488,15cyclictest7504-21runrttasks20:31:551
1317899478,17cyclictest19144-21basename21:05:011
1317899472,14cyclictest25408-21ls19:55:131
1317899467,14cyclictest4212-21/usr/sbin/munin20:25:201
1317899445,16cyclictest11338-21runrttasks20:42:281
1317899425,13cyclictest308-21/usr/sbin/munin20:15:151
1317899413,16cyclictest24204-21ntpq19:50:271
1317799411,12cyclictest26200-21fschecks_time22:55:180
80462400,6sleep01317799cyclictest22:02:560
303172401,3sleep030327-21ssh23:33:010
1317799393,11cyclictest81rcu_preempt21:45:090
106572390,3sleep0111rcuc/023:12:310
1317799377,5cyclictest81rcu_preempt00:25:230
214912361,31sleep021574-21python00:30:330
1317799364,4cyclictest81rcu_preempt22:18:440
50402350,6sleep00-21swapper/000:47:180
144652351,29sleep021015-21diskmemload00:23:540
1317799354,8cyclictest81rcu_preempt00:15:140
1317799354,4cyclictest81rcu_preempt21:32:230
1317799353,5cyclictest81rcu_preempt00:09:320
1317799351,14cyclictest81rcu_preempt23:37:040
1317799351,12cyclictest0-21swapper/023:52:390
1317799351,10cyclictest81rcu_preempt23:57:240
1317799348,4cyclictest81rcu_preempt00:40:490
1317799346,5cyclictest81rcu_preempt23:45:210
1317799345,4cyclictest81rcu_preempt21:40:480
1317799344,5cyclictest81rcu_preempt23:25:310
1317799344,4cyclictest81rcu_preempt22:26:340
1317799344,4cyclictest81rcu_preempt21:14:270
1317799344,4cyclictest81rcu_preempt00:04:440
1317799343,4cyclictest81rcu_preempt22:30:410
1317799341,20cyclictest7557-21pidof23:10:010
1317799334,5cyclictest81rcu_preempt00:10:330
1317799334,5cyclictest81rcu_preempt00:10:330
1317799334,4cyclictest81rcu_preempt22:48:250
1317799334,4cyclictest81rcu_preempt22:10:280
1317799333,5cyclictest81rcu_preempt22:36:260
1317799333,4cyclictest81rcu_preempt22:41:500
13177993322,7cyclictest18302-21rm23:20:230
13177993322,6cyclictest3-21ksoftirqd/000:51:240
1317799331,4cyclictest81rcu_preempt23:41:410
1317799324,4cyclictest81rcu_preempt21:57:590
1317799324,4cyclictest81rcu_preempt21:00:150
1317799324,21cyclictest9234-21users20:35:320
1317799323,4cyclictest81rcu_preempt21:35:350
1317799323,4cyclictest81rcu_preempt21:28:240
1317799322,4cyclictest81rcu_preempt22:21:210
1317799322,4cyclictest81rcu_preempt21:20:100
1317799322,4cyclictest81rcu_preempt00:39:060
1317799322,10cyclictest81rcu_preempt22:06:260
1317799322,10cyclictest0-21swapper/021:50:190
1317799321,4cyclictest81rcu_preempt20:18:370
1317799313,4cyclictest81rcu_preempt21:17:560
1317799312,4cyclictest81rcu_preempt23:15:310
1317799312,4cyclictest81rcu_preempt22:50:070
1317799312,14cyclictest29712-21memory20:05:250
1317799311,4cyclictest81rcu_preempt19:53:530
1317799311,15cyclictest27149-21/usr/sbin/munin20:00:240
1317799305,4cyclictest81rcu_preempt20:29:570
1317799302,4cyclictest81rcu_preempt19:30:300
1317799301,4cyclictest81rcu_preempt19:43:240
13177992922,4cyclictest3-21ksoftirqd/020:24:090
171142281,5sleep00-21swapper/020:58:550
1317799279,8cyclictest0-21swapper/020:50:130
13177992717,6cyclictest6443-21grep20:30:160
13177992717,5cyclictest32150-21grep20:10:310
1317799271,13cyclictest6204-21gdbus20:45:020
1317799270,4cyclictest81rcu_preempt21:05:140
101842271,5sleep00-21swapper/020:40:190
1317799262,14cyclictest0-21swapper/019:55:560
1317799251,16cyclictest0-21swapper/019:46:370
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional