You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-30 - 21:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack9slot4s (updated Sun Nov 30, 2025 12:59:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
219209912871219,53cyclictest22170-21wget07:25:130
219219911681146,18cyclictest21794-21smartctl08:45:291
219219911331117,10cyclictest22115-21wget07:25:131
215992557511,30sleep10-21swapper/107:22:141
217152524479,31sleep00-21swapper/007:23:290
219209911794,19cyclictest0-21swapper/008:45:280
73482820,4sleep00-21swapper/010:25:470
300502820,5sleep030031-21ssh10:50:150
250712820,8sleep02192099cyclictest11:18:280
68312790,3sleep1211ktimersoftd/112:40:131
275382780,4sleep027537-21ssh09:40:170
88722680,4sleep041ktimersoftd/012:08:480
297982620,5sleep12770-21sendmail-mta10:15:301
21921994317,4cyclictest22-21ksoftirqd/112:08:151
21921994313,5cyclictest22-21ksoftirqd/111:13:031
21920994223,6cyclictest22521-21grep09:35:160
2192099391,16cyclictest0-21swapper/009:12:380
2192199381,8cyclictest13475-21cstates08:25:131
2192099381,11cyclictest81rcu_preempt10:31:080
8372371,4sleep0848-21ssh11:26:240
73562371,4sleep07370-21sh09:52:530
21921993714,19cyclictest22-21ksoftirqd/109:20:581
21920993712,6cyclictest0-21swapper/010:09:370
21921993627,6cyclictest30944-21snmpd12:02:211
2192099367,4cyclictest81rcu_preempt11:55:300
2192099364,16cyclictest10775-21memory12:10:250
2192099363,11cyclictest81rcu_preempt10:00:370
21920993611,13cyclictest25983-21ntp_states10:45:270
151932361,6sleep115340-21/usr/sbin/munin08:30:131
2192199358,23cyclictest0-21swapper/112:52:121
2192199358,23cyclictest0-21swapper/112:33:451
2192199356,17cyclictest0-21swapper/111:49:591
21921993524,7cyclictest1156-21ssh11:26:501
21921993518,14cyclictest22-21ksoftirqd/111:05:211
2192199351,4cyclictest81rcu_preempt10:34:171
2192199351,12cyclictest0-21swapper/110:57:111
2192199351,12cyclictest0-21swapper/110:57:101
2192099359,13cyclictest24794-21sh12:25:160
127452351,30sleep112796-21users12:45:311
85482340,9sleep12192199cyclictest11:00:331
274792341,29sleep127540-21sh07:35:291
268822341,28sleep126924-21hddtemp_smartct09:00:181
251992341,4sleep125206-21iostat_ios07:30:211
2192199347,23cyclictest0-21swapper/110:41:001
21921993428,4cyclictest137350irq/90-eth010:48:111
21921993428,4cyclictest137350irq/90-eth009:33:421
2192199342,4cyclictest81rcu_preempt09:46:551
2192199342,4cyclictest81rcu_preempt09:46:541
2192099344,4cyclictest81rcu_preempt12:17:470
2192099343,18cyclictest0-21swapper/008:40:230
21920993414,4cyclictest66982sleep010:59:480
21920993414,4cyclictest66982sleep010:59:470
2192099341,20cyclictest8886-21diskstats11:35:170
2192099341,20cyclictest19954-21/usr/sbin/munin12:20:160
21920993410,8cyclictest0-21swapper/009:09:570
21920993410,8cyclictest0-21swapper/009:09:570
62432331,28sleep06257-21ssh12:39:570
57212331,28sleep15828-21cpu08:05:131
2192199336,4cyclictest81rcu_preempt10:08:531
2192199334,20cyclictest0-21swapper/108:58:071
21921993316,14cyclictest22-21ksoftirqd/110:14:401
21921993314,15cyclictest22-21ksoftirqd/112:36:321
2192099336,5cyclictest81rcu_preempt09:59:410
2192099336,4cyclictest81rcu_preempt10:39:190
2192099334,4cyclictest81rcu_preempt11:45:190
2192099333,5cyclictest81rcu_preempt12:45:230
21920993322,6cyclictest5472-21ssh09:17:150
40662320,8sleep02192099cyclictest10:23:010
256612321,5sleep00-21swapper/011:52:130
2192199325,23cyclictest0-21swapper/110:01:361
21921993227,3cyclictest137350irq/90-eth011:39:011
21921993227,3cyclictest137350irq/90-eth009:14:471
2192199322,5cyclictest0-21swapper/109:41:511
21921993217,12cyclictest22-21ksoftirqd/110:52:041
21921993216,9cyclictest22-21ksoftirqd/111:55:201
2192199321,4cyclictest81rcu_preempt12:29:031
2192099323,4cyclictest81rcu_preempt11:33:390
21920993222,6cyclictest15118-21ssh09:27:130
21920993218,8cyclictest12514-21ssh11:05:200
2192099321,4cyclictest81rcu_preempt12:34:300
2192099321,4cyclictest81rcu_preempt09:32:360
21920993213,6cyclictest32757-21ntp_states09:45:260
21920993213,6cyclictest32757-21ntp_states09:45:260
21920993210,5cyclictest3-21ksoftirqd/008:17:440
109982320,3sleep1201rcuc/112:10:291
9882311,3sleep01146-21unixbench_multi07:50:420
311872311,26sleep00-21swapper/010:17:290
289082311,26sleep028917-21diskstats07:40:170
269892310,3sleep041ktimersoftd/011:20:190
2192199317,5cyclictest40950irq/93-i91511:22:221
21921993125,4cyclictest137350irq/90-eth010:20:411
2192199312,23cyclictest2314-21hddtemp_smartct07:55:181
2192199312,17cyclictest0-21swapper/111:31:421
21921993119,4cyclictest22-21ksoftirqd/110:27:321
21921993119,4cyclictest2220-21dbus-daemon11:17:241
2192199311,4cyclictest81rcu_preempt11:51:411
2192199311,13cyclictest81rcu_preempt09:35:281
2192099319,7cyclictest20505-21diskstats10:40:160
2192099317,4cyclictest81rcu_preempt11:00:520
21920993116,11cyclictest0-21swapper/008:31:470
2192099311,4cyclictest81rcu_preempt12:41:100
2192099311,14cyclictest25404-21ssh10:11:030
2192099311,13cyclictest24030-21taskset08:51:290
182742311,5sleep10-21swapper/110:37:421
173232311,4sleep017454-21users12:50:370
25362301,5sleep00-21swapper/012:01:430
2192199304,4cyclictest81rcu_preempt08:54:591
2192199304,10cyclictest7871-21diskstats08:10:161
21921993022,5cyclictest22-21ksoftirqd/109:20:001
21921993015,12cyclictest22-21ksoftirqd/109:53:151
21921993011,14cyclictest3020-21runrttasks09:28:191
2192099309,5cyclictest26915-21cut09:00:160
2192099303,14cyclictest0-21swapper/007:55:350
2192099302,14cyclictest19478-21ssh11:12:200
2192099301,4cyclictest81rcu_preempt07:35:520
21920993013,5cyclictest4101-21df_inode08:00:140
21920993013,14cyclictest2233-21cron09:00:010
2192199295,21cyclictest0-21swapper/112:22:021
2192099292,14cyclictest15929-21ssh11:42:050
2192099291,4cyclictest81rcu_preempt08:08:080
2192099291,4cyclictest81rcu_preempt07:45:310
127992291,5sleep00-21swapper/008:22:080
113852291,5sleep00-21swapper/009:23:490
107392291,5sleep10-21swapper/108:15:361
2192199284,17cyclictest31100-21iostat_ios07:45:221
21921992820,5cyclictest16818-21ssh12:16:211
2192199281,4cyclictest81rcu_preempt07:52:261
21921992811,14cyclictest9463-21entropy09:55:181
21920992811,6cyclictest25835-21head07:30:290
141482281,5sleep10-21swapper/111:40:281
130242281,5sleep10-21swapper/108:24:301
203202271,5sleep10-21swapper/108:42:011
296482261,5sleep10-21swapper/109:06:081
296482261,5sleep10-21swapper/109:06:081
21921992618,5cyclictest22-21ksoftirqd/108:35:281
2192199261,7cyclictest4649-21ntp_states08:00:251
2192199261,4cyclictest81rcu_preempt07:41:201
2192099269,5cyclictest14575-21taskset08:26:020
2192099261,14cyclictest0-21swapper/008:38:560
2192099261,11cyclictest8466-21/usr/sbin/munin08:10:270
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional