You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-03-17 - 00:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Sun Mar 16, 2025 12:59:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
206159911121103,6cyclictest0-21swapper/107:25:101
206159911121103,6cyclictest0-21swapper/107:25:091
206149910831059,20cyclictest0-21swapper/007:25:080
206149910831059,20cyclictest0-21swapper/007:25:080
206159910721054,15cyclictest28585-21apt-get07:42:421
205182563516,31sleep00-21swapper/007:24:350
201882469418,30sleep10-21swapper/107:21:071
20614992394,5cyclictest0-21swapper/007:42:430
39221100,4sleep1391-21unixbench_multi11:00:311
50532760,4sleep181rcu_preempt08:05:211
65842630,2sleep06585-21mailstats11:20:280
20614994118,4cyclictest111rcuc/012:30:300
20614993716,7cyclictest8420-21/usr/sbin/munin08:15:150
13232351,30sleep11364-21iostat12:50:151
49482341,29sleep05012-21fschecks_count09:35:150
20614993411,9cyclictest1867-21cpuspeed_turbos11:10:090
87022331,4sleep18784-21memory11:30:221
256252331,28sleep025671-21df_inode10:40:140
20614993316,6cyclictest10872-21/usr/sbin/munin08:20:490
20614993316,5cyclictest338-21munin-run07:55:000
20614993316,5cyclictest26695-21cron09:03:590
20614993315,6cyclictest1174-21python09:20:290
20614993314,7cyclictest4501-21munin-run09:35:000
20615993220,9cyclictest19591-21/usr/sbin/munin08:45:141
2061499328,12cyclictest30944-21snmpd08:45:100
20614993215,6cyclictest23130-21ntp_states07:30:060
20614993215,6cyclictest1606-21ntp_states12:50:200
20614993215,6cyclictest12733-21/usr/sbin/munin11:40:260
20614993215,6cyclictest12733-21/usr/sbin/munin11:40:260
20614993215,5cyclictest8924-21ntp_states11:30:250
20614993214,7cyclictest23299-21netstat12:15:220
20614993214,7cyclictest0-21swapper/012:06:150
321132311,26sleep032155-21hddtemp_smartct11:00:180
221422311,4sleep122146-21ntp_states08:50:281
221402311,5sleep00-21swapper/012:12:420
2061499312,24cyclictest3750-21/usr/sbin/munin11:15:110
20614993115,5cyclictest21911-21/usr/sbin/munin08:50:230
20614993115,5cyclictest20834-21basename10:25:000
20614993114,6cyclictest25262-21ntp_states12:20:250
20614993114,6cyclictest25262-21ntp_states12:20:240
20614993114,6cyclictest11226-21diskstats09:55:150
20614993114,5cyclictest22601-21grep10:30:220
20614993111,11cyclictest0-21swapper/012:04:480
20614993110,6cyclictest0-21swapper/008:04:080
162522311,4sleep116257-21irqstats08:35:191
150872311,5sleep00-21swapper/010:09:130
303562301,5sleep00-21swapper/009:13:010
2061599304,19cyclictest22199-21iostat_ios10:30:161
20614993015,4cyclictest30252-21hddtemp_smartct10:55:170
20614993015,4cyclictest24074-21/usr/sbin/munin10:35:230
20614993014,6cyclictest30944-21snmpd07:55:440
20614993014,5cyclictest18152-21iostat_ios08:40:210
20614993014,12cyclictest3-21ksoftirqd/008:57:030
20614993013,6cyclictest16053-21/usr/sbin/munin10:10:190
2061499301,24cyclictest7153-21iostat_ios09:40:220
20614993011,6cyclictest29894-21load07:45:310
165362301,4sleep016545-21diskstats11:55:160
109012300,8sleep02061499cyclictest09:55:000
86242291,5sleep00-21swapper/009:47:140
77502291,5sleep00-21swapper/011:28:050
327162291,5sleep10-21swapper/112:46:161
285622291,5sleep10-21swapper/112:34:111
275662291,5sleep10-21swapper/110:48:241
269572291,5sleep00-21swapper/009:05:100
2061599294,19cyclictest25682-21iostat_ios09:00:211
2061499296,9cyclictest16989-21runrttasks08:36:030
2061499291,8cyclictest0-21swapper/011:50:070
20614992914,5cyclictest31280-21if_eth012:40:170
20614992913,5cyclictest6481-21/usr/sbin/munin08:10:140
20614992913,5cyclictest31195-21fschecks_time09:15:180
20614992913,5cyclictest12344-21grep08:25:170
20614992912,6cyclictest4662-21df08:05:130
20614992912,5cyclictest10807-21ntp_states11:35:250
20614992911,5cyclictest0-21swapper/012:27:130
2061499291,14cyclictest3020-21runrttasks10:18:110
19892291,5sleep10-21swapper/107:57:401
144182291,5sleep00-21swapper/010:03:490
81672281,5sleep10-21swapper/109:44:011
75622281,5sleep10-21swapper/111:26:071
72602281,5sleep10-21swapper/111:24:331
267252281,5sleep10-21swapper/112:29:331
225222281,5sleep10-21swapper/112:15:071
211262281,5sleep00-21swapper/010:26:410
2061599283,23cyclictest0-21swapper/109:23:261
20615992821,5cyclictest0-21swapper/109:51:331
20615992820,5cyclictest25045-21uname12:20:211
20615992820,5cyclictest25045-21uname12:20:211
20615992819,5cyclictest16271-21find10:10:221
20615992816,6cyclictest21245-21iostat_ios12:10:171
20615992814,8cyclictest27255-21diskstats09:05:161
20615992810,14cyclictest23342-21cpu08:55:151
20614992810,5cyclictest0-21swapper/010:50:270
206062281,5sleep10-21swapper/110:22:521
202302281,5sleep10-21swapper/112:07:231
159702281,5sleep10-21swapper/111:53:531
150912281,5sleep10-21swapper/110:09:161
137702281,5sleep00-21swapper/008:30:030
114662281,5sleep10-21swapper/111:37:461
97202271,5sleep10-21swapper/108:18:431
52602271,5sleep10-21swapper/111:18:271
327052271,5sleep10-21swapper/107:54:121
323842271,5sleep10-21swapper/112:44:421
312202271,5sleep10-21swapper/110:57:231
249602271,5sleep10-21swapper/110:37:321
249312271,5sleep10-21swapper/107:34:161
210802271,5sleep10-21swapper/110:26:121
2061599273,17cyclictest11870-21/usr/sbin/munin11:40:331
2061599273,17cyclictest11870-21/usr/sbin/munin11:40:321
20615992719,4cyclictest29114-21grep10:50:331
20615992718,6cyclictest22-21ksoftirqd/109:15:291
20615992718,5cyclictest22-21ksoftirqd/108:10:121
20615992718,5cyclictest12107-21cpuspeed_turbos08:25:141
20615992717,6cyclictest2505-21grep08:00:121
20614992712,4cyclictest29244-21tune2fs12:35:160
20614992710,5cyclictest14216-21idleruntime-cro11:50:000
20422271,5sleep10-21swapper/109:25:091
191882271,5sleep10-21swapper/108:44:151
188972271,5sleep10-21swapper/110:19:501
177252271,5sleep10-21swapper/111:57:351
141432271,4sleep10-21swapper/110:00:571
140372271,5sleep10-21swapper/111:48:161
123192271,5sleep10-21swapper/109:56:391
10792271,5sleep10-21swapper/111:05:151
63072261,5sleep10-21swapper/109:39:251
41292261,5sleep10-21swapper/109:31:181
326372261,5sleep00-21swapper/012:45:270
305352261,5sleep10-21swapper/109:14:551
20614992612,6cyclictest0-21swapper/011:06:450
2061499261,21cyclictest0-21swapper/010:45:150
20614992611,6cyclictest0-21swapper/009:26:130
20614992611,4cyclictest268482sleep007:39:140
302402251,5sleep10-21swapper/112:37:051
20615992517,5cyclictest22-21ksoftirqd/110:43:201
20615992517,5cyclictest22-21ksoftirqd/108:21:441
20615992517,5cyclictest22-21ksoftirqd/107:49:431
20615992416,5cyclictest22-21ksoftirqd/109:48:131
20615992415,6cyclictest25649-21fschecks_time07:35:171
20615992414,6cyclictest18170-21grep12:00:121
20615992414,6cyclictest13998-21cpuspeed_turbos08:30:161
2061599231,18cyclictest1704-21/usr/sbin/munin11:10:101
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional