You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-02 - 04:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Sat May 02, 2026 00:59:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
165539911621137,21cyclictest22810-21latency_hist19:40:001
165539911451135,7cyclictest0-21swapper/119:25:081
156132570518,36sleep00-21swapper/019:20:220
162002562518,29sleep10-21swapper/119:21:541
1655299552506,36cyclictest22952-21latency_hist19:40:000
165529911085,22cyclictest0-21swapper/019:25:080
41212660,3sleep00-21swapper/023:01:400
175742650,4sleep0111rcuc/021:35:240
16552996210,5cyclictest81rcu_preempt21:22:130
1655299611,13cyclictest7633-21ntp_states23:05:240
212152600,2sleep021216-21ssh22:12:510
1655299601,14cyclictest17304-21ntp_states23:15:250
242902580,8sleep11655399cyclictest23:22:561
1655299554,9cyclictest15378-21diskstats20:45:150
1655299521,17cyclictest8282-21missed_timers20:25:210
287932430,4sleep1211ktimersoftd/122:20:241
263302401,35sleep126338-21ssh22:18:161
309912395,30sleep122587-1kworker/1:0H21:15:361
1655299381,23cyclictest6316-21grep22:30:260
111572381,33sleep111167-21ssh00:15:511
244442371,32sleep124455-21sh23:56:101
1655399369,5cyclictest81rcu_preempt21:29:131
1655399369,24cyclictest0-21swapper/121:44:181
16553993613,4cyclictest40250irq/92-i91500:22:381
1655299365,19cyclictest0-21swapper/022:25:470
1655299361,22cyclictest5648-21ssh21:56:240
174732351,30sleep117552-21iostat20:50:161
1655399357,24cyclictest0-21swapper/122:07:051
16553993524,7cyclictest4167-21ssh22:28:571
1655399347,23cyclictest0-21swapper/122:54:421
1655299342,4cyclictest25467-21ssh22:17:000
1655399337,6cyclictest0-21swapper/121:33:171
1655399337,18cyclictest24341-21diskmemload21:47:141
1655399334,11cyclictest40250irq/92-i91523:51:161
1655399331,8cyclictest3286-21runrttasks00:27:361
16553993318,5cyclictest22-21ksoftirqd/122:31:421
16553993312,11cyclictest40250irq/92-i91522:55:541
16553993310,20cyclictest27217-21fw_packets19:50:161
1655299339,4cyclictest12026-21df_inode21:30:140
1655299332,15cyclictest30039-21iostat22:55:200
1655299331,21cyclictest0-21swapper/000:30:080
1655299331,11cyclictest0-21swapper/023:25:110
271832321,27sleep124341-21diskmemload23:25:291
1655399329,19cyclictest0-21swapper/100:03:371
1655399328,9cyclictest0-21swapper/119:30:101
16553993217,8cyclictest8155-21iostat_ios20:25:201
16553993210,18cyclictest0-21swapper/123:39:421
16553993210,18cyclictest0-21swapper/100:36:241
1655299324,4cyclictest81rcu_preempt21:45:430
1655299322,14cyclictest12471-21ntp_states23:10:240
1655299322,13cyclictest13613-21irqstats23:45:210
1655299321,19cyclictest23211-21iostat_ios21:05:190
1655299321,18cyclictest28706-21ssh22:54:280
16552993211,5cyclictest0-21swapper/000:00:240
1655299321,13cyclictest31703-21runrttasks21:16:360
109612321,4sleep124341-21diskmemload23:09:361
60972311,5sleep10-21swapper/121:57:021
3992311,5sleep10-21swapper/121:50:591
272112311,5sleep00-21swapper/021:12:090
257412311,5sleep10-21swapper/100:30:531
254402311,25sleep125447-21iostat_ios19:45:191
1655399314,4cyclictest81rcu_preempt20:13:501
1655399313,4cyclictest81rcu_preempt00:54:051
1655399312,15cyclictest17640-21sendmail_mailqu21:35:261
1655299319,3cyclictest0-21swapper/022:00:200
1655299318,5cyclictest8791-21iostat_ios23:40:200
1655299318,5cyclictest0-21swapper/023:22:240
1655299316,12cyclictest13298-21aten_r4power_cu22:05:100
1655299315,14cyclictest0-21swapper/023:37:060
1655299312,4cyclictest81rcu_preempt00:15:590
1655299312,24cyclictest20938-21munin-run19:35:000
1655299312,18cyclictest0-21swapper/023:55:110
16552993121,6cyclictest5582-21ssh00:44:400
1655299311,5cyclictest81rcu_preempt20:16:360
1655299311,4cyclictest81rcu_preempt00:11:030
16552993114,6cyclictest3171-21/usr/sbin/munin20:10:290
16552993112,6cyclictest28782-21ntp_states22:20:240
16552993112,6cyclictest10546-21hddtemp_smartct22:35:170
216742301,5sleep10-21swapper/122:46:161
169512301,4sleep116962-21ssh23:49:161
1655399309,5cyclictest40250irq/92-i91523:13:161
1655399308,18cyclictest0-21swapper/122:11:521
1655399305,3cyclictest81rcu_preempt21:08:231
16553993015,5cyclictest22-21ksoftirqd/123:01:371
16553993015,12cyclictest22-21ksoftirqd/123:44:051
16553993011,11cyclictest22-21ksoftirqd/100:11:051
1655299304,4cyclictest81rcu_preempt22:47:280
1655299304,4cyclictest81rcu_preempt00:20:070
1655299302,17cyclictest0-21swapper/020:00:370
1655299302,15cyclictest434-21iostat00:05:250
1655299302,15cyclictest407-21hddtemp_smartct20:05:170
16552993013,5cyclictest25146-21diskstats19:45:130
16552993012,5cyclictest29024-21fschecks_time19:55:150
16552993011,6cyclictest19844-21iostat00:25:180
1655299301,13cyclictest17913-21ssh22:42:480
89092291,5sleep00-21swapper/021:26:210
48552291,5sleep10-21swapper/100:43:261
37832291,5sleep10-21swapper/120:15:091
244062291,5sleep10-21swapper/119:42:021
203312291,5sleep10-21swapper/120:57:421
1655399293,22cyclictest0-21swapper/100:06:481
1655399292,23cyclictest15179-21diskstats22:40:141
16553992912,13cyclictest0-21swapper/100:46:351
16553992911,6cyclictest40250irq/92-i91520:25:001
16553992911,6cyclictest301512chrt19:58:181
1655299292,4cyclictest81rcu_preempt21:50:350
1655299292,15cyclictest0-21swapper/020:51:420
1655299292,13cyclictest21943-21hddtemp_smartct21:40:160
1655299291,4cyclictest81rcu_preempt20:56:010
1655299291,4cyclictest81rcu_preempt20:30:510
16552992912,5cyclictest11344-21/usr/sbin/munin00:50:240
16552992911,7cyclictest0-21swapper/019:40:130
121202291,5sleep10-21swapper/122:36:271
67012281,24sleep06816-21sed20:20:280
276572281,5sleep10-21swapper/121:12:481
1655399285,20cyclictest0-21swapper/123:16:291
1655399282,15cyclictest0-21swapper/123:31:331
16553992810,14cyclictest0-21swapper/120:45:231
1655299283,3cyclictest81rcu_preempt23:52:450
1655299282,14cyclictest0-21swapper/021:05:000
1655299281,24cyclictest13523-21fschecks_count20:40:140
1655299281,13cyclictest32108-21ls23:30:280
16552992810,5cyclictest29648-21/usr/sbin/munin00:35:200
321022271,5sleep10-21swapper/120:03:561
1655399278,16cyclictest0-21swapper/122:02:351
1655399274,3cyclictest81rcu_preempt21:02:591
1655399272,9cyclictest2192-21df21:20:121
1655299279,5cyclictest7534-21runrttasks00:45:330
1655299272,12cyclictest27020-21df_inode19:50:140
16552992711,12cyclictest12059-21grep20:35:200
16202271,5sleep10-21swapper/120:09:551
145512271,5sleep10-21swapper/120:41:111
1655399261,4cyclictest81rcu_preempt20:34:111
127842261,5sleep10-21swapper/120:37:321
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional