You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-12 - 17:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack9slot4s (updated Tue May 12, 2026 12:59:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
293499911201070,44cyclictest29790-21diskstats07:25:160
289452562518,29sleep10-21swapper/107:21:201
278752561516,29sleep00-21swapper/007:20:170
2382621120,10sleep02934999cyclictest11:25:120
293509910397,3cyclictest0-21swapper/107:25:161
305672890,7sleep12935099cyclictest11:40:301
177512870,3sleep00-21swapper/011:09:000
114212870,2sleep011420-21awk10:50:260
15962790,5sleep0111rcuc/007:35:130
265502780,4sleep10-21swapper/108:40:141
71042730,23sleep122-21ksoftirqd/110:40:161
120502610,13sleep0111rcuc/008:00:260
29349993822,6cyclictest3-21ksoftirqd/012:53:540
29349993821,6cyclictest3-21ksoftirqd/008:53:480
67872371,32sleep12225-21dbus-daemon09:13:411
29349993621,5cyclictest3-21ksoftirqd/007:41:140
29350993526,5cyclictest20393-21/usr/sbin/munin11:15:181
29349993520,6cyclictest3-21ksoftirqd/011:36:100
29349993519,5cyclictest3-21ksoftirqd/010:33:370
29349993519,5cyclictest3-21ksoftirqd/010:29:500
29349993518,5cyclictest3-21ksoftirqd/012:25:260
29349993510,13cyclictest3715-21timerandwakeup11:55:300
287502351,4sleep12695-21snmpd11:36:051
244522351,4sleep124594-21users12:50:301
29350993424,6cyclictest30397-21grep08:50:171
2934999341,21cyclictest7920-21taskset12:08:140
2934999341,21cyclictest7920-21taskset12:08:140
246562341,5sleep124712-21fschecks_count08:35:141
2935099335,3cyclictest81rcu_preempt11:49:461
29349993318,5cyclictest3-21ksoftirqd/012:42:120
29349993318,5cyclictest3-21ksoftirqd/010:43:010
29349993318,5cyclictest3-21ksoftirqd/010:01:340
29349993318,5cyclictest3-21ksoftirqd/008:24:150
29349993317,5cyclictest3-21ksoftirqd/011:49:300
29349993317,5cyclictest3-21ksoftirqd/009:30:320
29349993316,6cyclictest3-21ksoftirqd/007:30:300
56532321,4sleep05674-21ntpq10:35:250
2935099326,4cyclictest81rcu_preempt07:44:391
29350993216,9cyclictest1581-21iostat_ios10:25:191
29349993216,5cyclictest3-21ksoftirqd/012:39:250
29349993216,5cyclictest3-21ksoftirqd/007:45:240
29349993216,5cyclictest3-21ksoftirqd/007:45:240
29349993215,6cyclictest3-21ksoftirqd/010:59:290
29349993213,4cyclictest40250irq/92-i91509:14:030
256932321,4sleep12654-21sshd10:04:351
2935099316,4cyclictest81rcu_preempt11:13:191
2935099311,4cyclictest81rcu_preempt12:17:241
2934999312,3cyclictest0-21swapper/008:27:500
29349993118,4cyclictest3-21ksoftirqd/009:19:450
29349993115,6cyclictest3-21ksoftirqd/010:12:110
29349993115,5cyclictest3-21ksoftirqd/011:14:020
29349993112,6cyclictest18666-21df_inode09:45:140
5002301,5sleep00-21swapper/010:21:480
48492301,5sleep00-21swapper/009:08:020
35642301,25sleep13569-21irqstats10:30:201
2935099307,8cyclictest21164-21if_err_eth008:25:181
2935099306,4cyclictest81rcu_preempt12:34:571
2935099306,4cyclictest81rcu_preempt09:55:341
2935099305,4cyclictest81rcu_preempt07:45:471
2935099305,4cyclictest81rcu_preempt07:45:461
29350993016,8cyclictest14499-21iostat_ios12:25:191
2935099301,5cyclictest81rcu_preempt12:35:281
2935099301,16cyclictest850-21latency_hist10:25:011
2935099301,16cyclictest22053-21iostat_ios12:45:191
29349993016,4cyclictest3-21ksoftirqd/008:12:430
29349993015,5cyclictest3-21ksoftirqd/012:34:470
29349993015,5cyclictest3-21ksoftirqd/008:37:000
29349993014,5cyclictest3-21ksoftirqd/012:46:580
29349993014,5cyclictest3-21ksoftirqd/011:52:040
29349993012,5cyclictest1629-21df_inode09:00:150
204892301,10sleep020540-21df09:50:130
108792301,5sleep10-21swapper/107:58:291
63442291,5sleep10-21swapper/110:37:511
60942291,5sleep00-21swapper/012:03:470
3692291,24sleep1396-21ntp_states07:30:261
30472291,5sleep10-21swapper/109:04:001
296682291,5sleep10-21swapper/108:48:071
2935099299,8cyclictest2225-21dbus-daemon09:29:501
2935099294,9cyclictest13387-21diskstats08:05:151
2935099293,4cyclictest81rcu_preempt07:52:411
2935099293,19cyclictest15262-21diskstats08:10:141
2935099293,10cyclictest4504-21/usr/sbin/munin12:00:321
2935099292,6cyclictest0-21swapper/109:34:181
2935099291,4cyclictest81rcu_preempt11:03:291
29350992910,6cyclictest11332-21cpu08:00:131
2934999292,13cyclictest25835-21diskstats11:30:150
29349992914,5cyclictest3-21ksoftirqd/007:58:110
29349992913,5cyclictest3-21ksoftirqd/008:59:090
29349992912,5cyclictest7857-21irqstats07:50:220
29349992912,5cyclictest12321-21diskstats12:20:150
2934999291,24cyclictest31338-21chrt10:19:390
29349992911,4cyclictest40250irq/92-i91508:30:410
29349992911,14cyclictest181382chrt09:44:130
276092291,5sleep00-21swapper/008:41:170
274832291,5sleep00-21swapper/010:08:420
162762291,5sleep00-21swapper/009:39:360
139392291,5sleep10-21swapper/110:58:351
123462291,5sleep00-21swapper/009:27:430
101142291,5sleep00-21swapper/010:47:500
2935099286,9cyclictest12321-21diskstats12:20:151
2935099284,8cyclictest17102-21iostat_ios09:40:201
2935099284,4cyclictest81rcu_preempt08:21:041
29350992817,6cyclictest23657-21grep08:30:271
2935099281,23cyclictest20857-21irqstats09:50:211
2935099280,4cyclictest81rcu_preempt08:57:301
2934999289,5cyclictest13922-21ntp_states08:05:240
2934999288,5cyclictest19915-21wget11:15:110
29349992813,4cyclictest3-21ksoftirqd/011:44:040
29349992810,6cyclictest10410-21df12:15:130
234922281,5sleep00-21swapper/009:56:230
21562281,4sleep10-21swapper/111:53:061
185012281,5sleep10-21swapper/108:19:301
106042281,5sleep10-21swapper/109:24:081
85732271,5sleep10-21swapper/109:17:431
79122271,5sleep10-21swapper/112:08:101
79122271,5sleep10-21swapper/112:08:091
312322271,5sleep10-21swapper/110:18:331
2935099274,4cyclictest81rcu_preempt07:36:161
2935099271,13cyclictest26002-21cut10:05:131
2935099270,4cyclictest81rcu_preempt11:05:471
2934999275,3cyclictest0-21swapper/012:13:260
29349992712,5cyclictest3-21ksoftirqd/009:23:540
2934999271,12cyclictest28324-21taskset08:45:120
231832271,5sleep10-21swapper/111:21:431
231832271,5sleep10-21swapper/111:21:421
150982271,5sleep00-21swapper/011:00:270
2935099264,4cyclictest81rcu_preempt12:41:001
2935099264,4cyclictest81rcu_preempt11:29:201
29350992617,6cyclictest22-21ksoftirqd/110:12:321
2935099261,4cyclictest81rcu_preempt12:13:081
29350992610,5cyclictest40250irq/92-i91509:05:151
2934999261,13cyclictest21890-21hostname11:20:130
2934999261,13cyclictest21890-21hostname11:20:130
2934999261,13cyclictest17767-21processes08:15:280
197852261,5sleep10-21swapper/109:46:511
146492261,21sleep114722-21grep09:35:121
29350992516,6cyclictest22-21ksoftirqd/111:34:391
29350992511,11cyclictest9127-21hwlatdetect10:45:171
29350992510,4cyclictest40250irq/92-i91511:55:301
2935099242,9cyclictest11499-21sensors_temp10:50:271
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional