You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-21 - 04:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack9slot4s (updated Tue Oct 21, 2025 00:59:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
312389911571132,21cyclictest2811-21users19:30:321
307782595548,32sleep00-21swapper/019:20:540
305002549499,33sleep10-21swapper/119:20:331
3123799126117,6cyclictest0-21swapper/019:30:330
722621190,8sleep03123799cyclictest23:25:270
129282840,4sleep041ktimersoftd/000:05:160
115032730,3sleep03-21ksoftirqd/019:55:210
3123899469,20cyclictest0-21swapper/100:50:521
3123899469,20cyclictest0-21swapper/100:50:521
31237994424,5cyclictest9368-21df00:35:160
31237994326,6cyclictest3-21ksoftirqd/023:15:340
217132430,5sleep03123799cyclictest21:25:140
31237994218,8cyclictest3-21ksoftirqd/023:23:450
93132401,4sleep09324-21ssh21:46:320
93132401,4sleep09324-21ssh21:46:310
3123899402,13cyclictest0-21swapper/123:33:051
31237994017,7cyclictest3-21ksoftirqd/021:16:490
31237994017,7cyclictest3-21ksoftirqd/000:28:470
31237994016,8cyclictest3-21ksoftirqd/022:40:510
31237994016,7cyclictest3-21ksoftirqd/000:42:310
31237993924,6cyclictest3-21ksoftirqd/023:50:440
31237993923,5cyclictest3-21ksoftirqd/023:39:110
31237993923,5cyclictest3-21ksoftirqd/020:29:520
31237993916,7cyclictest3-21ksoftirqd/022:06:030
3123799391,5cyclictest18031-21ssh00:10:210
31238993817,4cyclictest22-21ksoftirqd/121:55:141
3123799382,11cyclictest81rcu_preempt00:23:140
31237993820,6cyclictest4133-21switchtime23:55:300
31237993816,7cyclictest3-21ksoftirqd/022:53:090
31237993815,8cyclictest3-21ksoftirqd/023:31:390
31237993815,8cyclictest3-21ksoftirqd/022:20:090
31237993815,8cyclictest3-21ksoftirqd/022:17:410
31237993814,8cyclictest3-21ksoftirqd/021:22:470
31237993814,8cyclictest3-21ksoftirqd/000:51:020
31237993814,8cyclictest3-21ksoftirqd/000:51:010
31237993814,8cyclictest3-21ksoftirqd/000:01:310
3123899375,14cyclictest40950irq/93-i91500:20:031
31237993715,7cyclictest3-21ksoftirqd/022:13:450
31237993714,7cyclictest3-21ksoftirqd/021:12:330
31238993624,8cyclictest21496-21aten_r4power_cu21:25:121
3123899361,16cyclictest4149-21cat23:55:301
31237993623,8cyclictest21166-21df_inode23:40:150
31237993620,6cyclictest29638-21cron20:45:000
31237993620,5cyclictest21943-21ssh22:00:150
31237993614,7cyclictest3-21ksoftirqd/020:34:200
212002361,31sleep121373-21cut23:40:181
3123899359,22cyclictest0-21swapper/123:22:451
3123899351,7cyclictest10576-21sshd19:52:401
31237993520,5cyclictest3-21ksoftirqd/019:29:480
31237993520,11cyclictest17526-21irqstats21:55:250
31237993515,9cyclictest9823-21grep19:50:210
31237993513,7cyclictest3-21ksoftirqd/020:10:020
31237993513,6cyclictest3-21ksoftirqd/021:00:490
209832351,31sleep120994-21ssh22:32:301
188752351,4sleep118886-21ssh21:21:361
3123899346,24cyclictest0-21swapper/100:07:021
31238993420,11cyclictest20527-21ssh23:05:281
31238993418,12cyclictest7223-21ls23:25:271
31238993410,20cyclictest0-21swapper/100:46:181
31237993413,7cyclictest3-21ksoftirqd/000:19:430
31237993412,8cyclictest3-21ksoftirqd/022:58:050
31237993412,7cyclictest3-21ksoftirqd/020:07:280
31237993412,7cyclictest3-21ksoftirqd/020:03:320
31237993412,6cyclictest3-21ksoftirqd/019:38:330
171692341,4sleep117179-21ssh00:43:131
112852341,29sleep111297-21ssh21:14:381
92022331,6sleep19225-21ssh00:00:321
62492331,3sleep16263-21ssh22:17:051
31237993320,5cyclictest0-21swapper/019:40:180
31237993320,4cyclictest23699-21iostat22:35:220
31237993316,5cyclictest0-21swapper/000:45:180
31237993313,6cyclictest3-21ksoftirqd/021:53:290
31237993313,6cyclictest3-21ksoftirqd/021:37:180
31237993312,16cyclictest18000-21cat22:30:010
31237993311,7cyclictest3-21ksoftirqd/000:32:160
31237993310,8cyclictest3-21ksoftirqd/021:07:560
8052321,5sleep10-21swapper/121:38:211
3123899324,24cyclictest0-21swapper/122:59:261
31238993224,4cyclictest22-21ksoftirqd/121:42:331
31238993215,14cyclictest22-21ksoftirqd/122:06:541
3123899321,4cyclictest81rcu_preempt21:16:491
3123799329,12cyclictest3-21ksoftirqd/020:51:050
3123799329,12cyclictest20070-21perl23:05:190
31237993218,10cyclictest1185-21basename20:55:130
31237993217,4cyclictest27585-21timerandwakeup21:30:300
31237993217,4cyclictest1106-21ssh22:45:260
31237993214,6cyclictest15080-21ssh23:00:170
31237993211,6cyclictest3-21ksoftirqd/020:39:170
31237993211,6cyclictest3-21ksoftirqd/020:21:250
31237993210,8cyclictest3-21ksoftirqd/020:16:060
3123899314,25cyclictest0-21swapper/121:32:251
3123899312,26cyclictest0-21swapper/122:40:311
31238993113,14cyclictest5092-21/usr/sbin/munin22:50:261
31238993112,16cyclictest22-21ksoftirqd/100:26:181
31237993117,4cyclictest31663-21latency_hist20:50:000
31237993111,6cyclictest3-21ksoftirqd/021:43:180
283422311,4sleep128418-21hddtemp_smartct20:40:181
180332311,4sleep118048-21iostat_ios00:10:231
3123899307,19cyclictest0-21swapper/122:00:511
3123899304,22cyclictest0-21swapper/123:48:091
31238993017,9cyclictest22-21ksoftirqd/121:49:191
31238993017,9cyclictest22-21ksoftirqd/121:49:191
31238993017,9cyclictest22-21ksoftirqd/100:39:061
3123899301,4cyclictest81rcu_preempt22:13:451
31238993011,15cyclictest25094-21ssh00:17:421
31238993011,15cyclictest0-21swapper/120:15:321
3123799309,9cyclictest0-21swapper/023:10:090
31237993015,6cyclictest7515-21/usr/sbin/munin19:45:190
31237993013,6cyclictest26675-21ssh23:45:260
31237993013,6cyclictest18880-21iostat22:30:220
143642301,5sleep10-21swapper/120:02:471
106892301,25sleep110701-21ssh22:21:321
88442291,5sleep10-21swapper/119:49:091
3123899299,16cyclictest0-21swapper/121:54:041
3123899294,21cyclictest0-21swapper/123:11:331
3123899293,4cyclictest81rcu_preempt21:09:291
3123899293,4cyclictest81rcu_preempt20:34:201
31238992916,9cyclictest22-21ksoftirqd/123:15:341
31238992916,10cyclictest22-21ksoftirqd/122:39:191
31238992910,14cyclictest15002-21cat23:00:161
183152291,5sleep10-21swapper/120:14:501
68852281,5sleep10-21swapper/119:43:191
316852281,5sleep10-21swapper/120:50:061
313932281,5sleep10-21swapper/120:48:391
3123899289,16cyclictest0-21swapper/122:47:311
3123899288,16cyclictest0-21swapper/123:39:441
3123899283,9cyclictest14376-21ntp_states22:25:301
31238992819,5cyclictest8785-21basename00:35:001
31238992818,6cyclictest22-21ksoftirqd/123:50:301
3123899281,4cyclictest81rcu_preempt20:28:221
220922281,5sleep10-21swapper/120:24:511
124702281,5sleep10-21swapper/119:57:501
327552271,5sleep10-21swapper/119:26:341
3123899273,4cyclictest81rcu_preempt19:37:131
31238992718,6cyclictest22-21ksoftirqd/120:58:261
31238992717,6cyclictest22-21ksoftirqd/120:06:021
3123899270,4cyclictest81rcu_preempt21:00:491
275982271,5sleep10-21swapper/120:38:321
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional