You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-28 - 06:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Fri Aug 28, 2026 00:59:01)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
127889912421215,22cyclictest14661-21munin-setwarn19:29:591
125522557510,32sleep00-21swapper/019:23:040
125532509465,29sleep10-21swapper/119:23:051
1278799195171,18cyclictest14657-21munin-run19:30:000
177382850,2sleep017736-21cut21:00:220
184212720,5sleep0111rcuc/023:55:130
274172570,8sleep01278799cyclictest21:25:280
304372425,6sleep03-21ksoftirqd/021:35:150
183532410,23sleep122-21ksoftirqd/119:35:281
71292360,9sleep11278899cyclictest00:50:181
1278799367,13cyclictest11565-21gdbus00:53:330
224212351,30sleep02401-21NetworkManager21:14:450
1278799351,23cyclictest29075-21seq00:22:540
1278799351,15cyclictest0-21swapper/019:40:050
218582341,4sleep122000-21users21:10:301
1278799349,5cyclictest9433-21seq22:05:170
1278799348,11cyclictest11565-21gdbus20:59:320
1278799347,11cyclictest11977-21latency20:45:210
1278799341,20cyclictest30861-21ls23:00:290
1278799341,20cyclictest22952-21fschecks_count21:15:170
240902331,4sleep124192-21uniq00:10:151
214852331,28sleep121580-21hddtemp_smartct19:45:161
1278899331,4cyclictest81rcu_preempt20:05:271
1278799336,10cyclictest28349-21grep22:55:160
1278799333,15cyclictest2225-21dbus-daemon22:24:490
12787993324,6cyclictest22296-21grep19:45:280
1278799332,15cyclictest2225-21dbus-daemon20:29:570
261772321,4sleep126257-21iostat00:15:181
1278899325,10cyclictest3286-21runrttasks23:43:511
310732311,4sleep131074-21sshd00:29:101
1278899314,4cyclictest81rcu_preempt20:02:021
12788993116,8cyclictest25111-21iostat_ios21:20:211
1278799319,8cyclictest6150-21ntp_states21:55:250
1278799319,11cyclictest8787-21head20:35:310
1278799314,20cyclictest24228-21users19:50:280
12787993115,13cyclictest3-21ksoftirqd/000:05:010
1278799311,4cyclictest81rcu_preempt00:41:220
12787993113,5cyclictest9014-21/usr/sbin/munin23:30:130
295992301,5sleep10-21swapper/121:31:071
276972301,5sleep10-21swapper/122:54:381
1278899307,10cyclictest0-21swapper/121:50:241
1278899306,10cyclictest0-21swapper/100:35:271
1278899305,19cyclictest19816-21iostat_ios19:40:201
1278899302,4cyclictest81rcu_preempt00:06:301
1278899302,12cyclictest3286-21runrttasks22:21:241
1278899301,5cyclictest81rcu_preempt00:04:501
1278899301,4cyclictest81rcu_preempt23:34:551
1278799305,18cyclictest32273-21iostat_ios23:05:220
1278799305,18cyclictest32273-21iostat_ios23:05:210
1278799302,14cyclictest30275-21ntp_states00:25:250
12787993012,5cyclictest31343-21/usr/sbin/munin20:10:250
1278799301,15cyclictest11038-21snmpd22:28:130
12787993011,14cyclictest138312chrt22:15:260
73042291,5sleep00-21swapper/020:34:090
254992291,5sleep10-21swapper/122:46:141
241722291,5sleep10-21swapper/121:18:241
187742291,5sleep00-21swapper/019:37:190
171522291,5sleep00-21swapper/019:34:590
1278899299,7cyclictest0-21swapper/122:35:221
1278899296,12cyclictest40250irq/92-i91521:00:141
1278899295,4cyclictest81rcu_preempt20:27:461
1278899295,4cyclictest81rcu_preempt00:23:321
1278899294,4cyclictest81rcu_preempt22:06:531
1278899294,4cyclictest81rcu_preempt21:56:221
1278899293,19cyclictest18440-21diskstats23:55:161
1278899291,4cyclictest81rcu_preempt23:15:381
1278899291,14cyclictest0-21swapper/120:43:561
1278899291,12cyclictest14552/usr/sbin/munin21:45:131
1278899291,10cyclictest0-21swapper/122:57:151
1278899290,4cyclictest81rcu_preempt23:35:531
1278799293,13cyclictest0-21swapper/022:40:130
1278799292,15cyclictest5062-21fschecks_time00:45:180
1278799292,15cyclictest1045-21/usr/sbin/munin00:35:160
1278799292,14cyclictest20609-21if_eth000:00:200
1278799292,14cyclictest14879-21/usr/sbin/munin23:45:170
1278799292,13cyclictest3682-21iostat_ios23:15:210
1278799291,4cyclictest81rcu_preempt23:27:060
1278799291,4cyclictest81rcu_preempt00:14:230
1278799291,24cyclictest26427-21uniq22:50:150
1278799291,20cyclictest0-21swapper/021:50:040
12787992911,6cyclictest1668-21cut21:45:120
1278799291,15cyclictest32300-21/usr/sbin/munin00:30:260
1278799291,15cyclictest32300-21/usr/sbin/munin00:30:260
12787992910,6cyclictest24443-21df_inode22:45:150
65922281,5sleep10-21swapper/123:24:011
60472281,4sleep10-21swapper/100:46:391
33392281,5sleep00-21swapper/020:21:590
310912281,5sleep10-21swapper/123:00:411
263322281,5sleep10-21swapper/119:57:431
25282281,5sleep10-21swapper/120:20:221
198502281,5sleep00-21swapper/022:31:130
144922281,5sleep00-21swapper/020:50:360
144922281,5sleep00-21swapper/020:50:360
13232281,5sleep10-21swapper/120:16:591
1278899285,4cyclictest81rcu_preempt21:26:431
1278899285,4cyclictest81rcu_preempt20:48:061
1278899284,18cyclictest1387-21diskstats23:10:151
1278899281,4cyclictest81rcu_preempt21:41:171
1278899281,18cyclictest0-21swapper/119:30:281
12787992811,5cyclictest81rcu_preempt20:20:000
12787992811,5cyclictest17305-21grep23:50:290
1278799281,14cyclictest29542-21vmstat21:30:310
1278799281,14cyclictest26703-21latency_hist20:00:010
1278799281,13cyclictest5732-21grep23:20:220
1278799281,13cyclictest13680-21grep23:40:260
12787992810,6cyclictest20381-21munin-node22:35:130
12787992810,5cyclictest19150-21df_inode21:05:150
126002281,5sleep10-21swapper/122:14:121
11062281,4sleep00-21swapper/021:43:090
109292281,5sleep00-21swapper/020:42:390
197852271,4sleep10-21swapper/122:30:351
188472271,5sleep10-21swapper/121:05:071
128422271,5sleep10-21swapper/122:15:081
1278899275,4cyclictest81rcu_preempt20:32:131
1278899274,4cyclictest81rcu_preempt22:41:341
1278899273,4cyclictest81rcu_preempt22:28:431
1278899272,6cyclictest3286-21runrttasks00:40:391
1278899272,10cyclictest15575-21hddtemp_smartct20:55:171
12788992710,6cyclictest315762chrt21:37:081
1278799274,8cyclictest7510-21diskstats22:00:150
12787992711,5cyclictest12037-21runrttasks23:37:020
1278799271,14cyclictest27431-21/usr/sbin/munin20:00:230
1278799271,13cyclictest28901-21/usr/sbin/munin20:05:120
1278799271,13cyclictest26462-21ntp_kernel_err00:15:230
1278799271,13cyclictest25560-21switchtime21:20:280
10872271,22sleep01131-21basename23:10:100
158502261,4sleep10-21swapper/123:47:231
149002261,5sleep10-21swapper/120:54:511
149002261,5sleep10-21swapper/120:54:511
1278899269,5cyclictest40250irq/92-i91523:25:121
12788992617,5cyclictest22-21ksoftirqd/122:02:401
4402251,4sleep10-21swapper/100:33:421
4402251,4sleep10-21swapper/100:33:421
1278899259,5cyclictest40250irq/92-i91520:10:141
1278899252,7cyclictest8140-21iostat20:35:191
12788992517,5cyclictest22-21ksoftirqd/123:53:181
1278899251,12cyclictest335-21expr23:05:291
1278899251,12cyclictest335-21expr23:05:291
1278899251,10cyclictest23020-21wget19:50:101
1278799251,16cyclictest0-21swapper/022:12:480
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional