You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-09 - 16:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack9slot4s (updated Sun Nov 09, 2025 12:59:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
39159912601229,25cyclictest8356-21apt-get07:35:011
39159912501241,6cyclictest0-21swapper/107:25:071
38332642597,30sleep10-21swapper/107:24:481
35192467372,79sleep00-21swapper/007:21:310
391499268214,49cyclictest8393-21latency_hist07:35:010
391499259227,17cyclictest0-21swapper/007:25:070
18932940,3sleep00-21swapper/012:55:000
146692830,5sleep1391599cyclictest07:50:181
12912650,4sleep041ktimersoftd/010:05:230
251862501,31sleep025199-21sh09:56:040
106122421,37sleep110622-21ssh10:48:561
3915994125,12cyclictest27617-21ssh11:05:351
3914994123,6cyclictest0-21swapper/012:08:180
3914994024,6cyclictest3-21ksoftirqd/012:17:250
3914994016,8cyclictest3-21ksoftirqd/012:36:240
3914994016,8cyclictest3-21ksoftirqd/010:23:130
3914993917,5cyclictest0-21swapper/009:39:450
3914993916,8cyclictest3-21ksoftirqd/008:27:060
22552391,8sleep00-21swapper/009:32:530
391599387,7cyclictest0-21swapper/109:18:251
391599387,13cyclictest40950irq/93-i91509:48:321
3914993822,5cyclictest4213-21runrttasks08:47:040
3914993818,5cyclictest12038-21awk09:10:010
3914993815,8cyclictest3-21ksoftirqd/010:11:210
3914993815,8cyclictest3-21ksoftirqd/010:11:200
3914993815,7cyclictest3-21ksoftirqd/008:15:270
3914993814,5cyclictest14299-21sh12:00:040
268512380,4sleep041ktimersoftd/011:39:340
128332380,4sleep1211ktimersoftd/110:50:271
121812380,4sleep041ktimersoftd/010:50:160
3914993721,12cyclictest9417-21ssh09:40:170
3914993716,7cyclictest3-21ksoftirqd/007:47:080
3914993715,8cyclictest3-21ksoftirqd/008:09:050
3914993715,7cyclictest3-21ksoftirqd/012:31:330
3914993713,8cyclictest3-21ksoftirqd/010:19:090
391599364,14cyclictest0-21swapper/112:06:421
391599361,7cyclictest6514-21ssh11:17:571
3914993619,7cyclictest0-21swapper/011:10:250
3914993615,7cyclictest3-21ksoftirqd/009:00:040
3914993615,7cyclictest3-21ksoftirqd/009:00:040
3914993613,7cyclictest23454-21users09:20:320
391499361,21cyclictest22764-21users11:00:330
3914993612,12cyclictest21638-21ssh09:19:560
3914993611,8cyclictest3-21ksoftirqd/010:32:570
3914993610,12cyclictest12304-21sh09:10:100
307722361,31sleep16204-21gdbus08:30:501
201482361,31sleep120160-21rm12:39:461
3914993518,7cyclictest0-21swapper/011:09:210
3914993517,6cyclictest28131-21ssh11:40:210
3914993515,6cyclictest4351-21ntp_states11:15:260
3914993514,8cyclictest15006-21ntp_states07:50:260
3914993514,17cyclictest30154-21memory08:30:240
3914993513,7cyclictest3-21ksoftirqd/008:42:010
3914993512,8cyclictest3-21ksoftirqd/007:55:010
3914993510,13cyclictest3-21ksoftirqd/011:51:110
272082351,3sleep12376-21NetworkManager12:13:031
272082351,3sleep12376-21NetworkManager12:13:021
129712351,29sleep112985-21ssh09:44:191
85712342,4sleep18582-21cron11:54:001
3915993412,5cyclictest0-21swapper/110:24:131
3914993418,5cyclictest20860-21ntp_states10:25:260
3914993412,7cyclictest3-21ksoftirqd/008:12:200
3914993412,12cyclictest16256-21ssh11:28:050
320802341,29sleep132090-21ssh12:51:341
289412341,29sleep128955-21ssh12:48:511
273692340,5sleep0111rcuc/009:25:190
79022331,5sleep00-21swapper/012:25:570
391599333,9cyclictest17046-21hddtemp_smartct10:55:221
3914993318,5cyclictest0-21swapper/010:55:120
3914993317,6cyclictest26296-21ntp_kernel_err12:45:240
3914993317,5cyclictest2505-21ssh10:40:190
3914993312,7cyclictest3-21ksoftirqd/008:35:490
3914993312,7cyclictest3-21ksoftirqd/007:44:490
3914993312,6cyclictest8705-21ssh11:20:170
391499331,20cyclictest26788-21ssh12:12:260
391499331,20cyclictest26788-21ssh12:12:250
3914993311,7cyclictest3-21ksoftirqd/011:56:120
3914993311,10cyclictest18193-21diskstats11:30:190
3914993311,10cyclictest18193-21diskstats11:30:190
3914993310,8cyclictest4132-21ssh12:22:290
3914993310,12cyclictest3-21ksoftirqd/010:36:520
322232330,5sleep1211ktimersoftd/108:35:271
273432331,28sleep127367-21ssh09:25:191
7142321,27sleep1728-21ssh11:11:431
391599326,22cyclictest0-21swapper/109:58:191
3915993222,7cyclictest25377-21awk10:30:201
391499329,8cyclictest7369-21ntp_states08:55:240
391499327,13cyclictest3-21ksoftirqd/010:01:000
391499322,16cyclictest19928-21/usr/sbin/munin09:50:300
3914993212,7cyclictest7404-21hddtemp_smartct10:45:200
3914993212,6cyclictest3-21ksoftirqd/008:52:530
3914993212,3cyclictest0-21swapper/011:46:160
3914993210,7cyclictest3-21ksoftirqd/008:23:320
248782321,5sleep10-21swapper/109:22:371
125652321,4sleep112576-21ssh10:16:451
52212311,4sleep15256-21iostat08:50:201
4992311,26sleep1584-21memory11:45:231
391599319,17cyclictest0-21swapper/110:01:561
391599316,21cyclictest0-21swapper/110:35:521
391499312,19cyclictest0-21swapper/007:36:080
3914993112,5cyclictest6204-21gdbus09:45:280
115062311,26sleep111587-21users09:05:331
62562301,5sleep10-21swapper/109:36:321
391599302,24cyclictest6961-21memory12:25:241
3915993021,6cyclictest22729-21ntp_kernel_err08:10:241
3915993016,11cyclictest1437-21tr10:05:261
391599301,4cyclictest81rcu_preempt12:31:331
3915993013,14cyclictest0-21swapper/112:16:341
220682301,5sleep00-21swapper/012:40:310
214892301,5sleep10-21swapper/108:07:081
133732301,25sleep113391-21diskstats11:25:181
121112301,5sleep10-21swapper/107:44:491
41242291,5sleep10-21swapper/108:46:101
391599299,16cyclictest0-21swapper/111:58:531
391599299,16cyclictest0-21swapper/110:25:151
3915992919,6cyclictest24557-21sed08:15:241
3915992917,9cyclictest22-21ksoftirqd/112:42:571
3915992916,10cyclictest22-21ksoftirqd/111:42:091
391599291,4cyclictest81rcu_preempt08:27:071
391599291,4cyclictest81rcu_preempt07:47:071
98872281,5sleep10-21swapper/107:36:281
80502281,5sleep10-21swapper/108:57:551
391599288,16cyclictest0-21swapper/110:42:261
391599283,21cyclictest0-21swapper/111:21:021
3915992816,9cyclictest22-21ksoftirqd/110:10:221
3915992816,9cyclictest22-21ksoftirqd/110:10:221
3915992816,5cyclictest22-21ksoftirqd/112:22:411
391599281,4cyclictest81rcu_preempt09:00:051
391599281,4cyclictest81rcu_preempt09:00:041
391499289,11cyclictest0-21swapper/008:00:180
271222281,5sleep10-21swapper/108:21:561
194452280,5sleep119443-21ssh09:50:231
391599277,16cyclictest0-21swapper/111:36:011
391599276,17cyclictest0-21swapper/109:12:521
391599274,8cyclictest16573-21diskstats07:55:151
391599272,4cyclictest81rcu_preempt08:41:081
391599272,3cyclictest81rcu_preempt12:00:421
391599272,12cyclictest0-21swapper/111:31:041
391599272,12cyclictest0-21swapper/111:31:031
391599272,11cyclictest0-21swapper/111:00:191
3915992716,8cyclictest22-21ksoftirqd/109:33:301
196192271,5sleep10-21swapper/108:02:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional