You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-13 - 03:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Thu Nov 13, 2025 00:59:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
158419911541145,6cyclictest0-21swapper/119:25:081
158409911531119,23cyclictest0-21swapper/019:25:060
158419911101093,14cyclictest0-21swapper/119:35:181
156042764719,30sleep10-21swapper/119:23:081
155162597551,32sleep00-21swapper/019:22:110
2417521180,3sleep00-21swapper/022:16:400
158409910484,15cyclictest0-21swapper/019:35:170
288992820,7sleep01584099cyclictest23:29:290
244942820,7sleep11584199cyclictest21:10:181
181932700,5sleep041ktimersoftd/020:55:120
292022420,5sleep01584099cyclictest21:48:580
15841994217,10cyclictest22-21ksoftirqd/100:22:301
1584199417,22cyclictest0-21swapper/121:49:111
15841994115,6cyclictest22-21ksoftirqd/100:07:191
15841994018,6cyclictest22-21ksoftirqd/121:55:131
15841994015,9cyclictest22-21ksoftirqd/122:10:271
15841994014,15cyclictest22-21ksoftirqd/123:11:391
15841994013,11cyclictest22-21ksoftirqd/123:55:241
15840994015,8cyclictest1403-21diskstats20:10:160
15840994015,8cyclictest1403-21diskstats20:10:160
1584099401,10cyclictest0-21swapper/023:24:100
15841993917,7cyclictest22-21ksoftirqd/123:20:011
15841993917,7cyclictest22-21ksoftirqd/122:38:341
15841993917,6cyclictest22-21ksoftirqd/122:29:281
15841993916,6cyclictest22-21ksoftirqd/123:34:221
15841993914,6cyclictest22-21ksoftirqd/122:21:041
15841993914,6cyclictest22-21ksoftirqd/121:41:121
15841993913,10cyclictest22-21ksoftirqd/100:35:231
1584099397,13cyclictest29341-21switchtime22:55:290
160742381,33sleep016172-21ssh23:15:290
15841993817,6cyclictest22-21ksoftirqd/123:51:141
15841993817,6cyclictest22-21ksoftirqd/122:46:311
15841993816,7cyclictest22-21ksoftirqd/122:32:261
15841993816,6cyclictest22-21ksoftirqd/100:10:141
15841993815,14cyclictest22-21ksoftirqd/122:54:061
15841993814,14cyclictest22-21ksoftirqd/123:46:141
15841993813,6cyclictest22-21ksoftirqd/122:57:571
1584099389,13cyclictest21268-21taskset22:14:270
319852371,4sleep032007-21fschecks_count20:05:170
1584199375,16cyclictest0-21swapper/123:37:361
15841993716,6cyclictest22-21ksoftirqd/122:40:391
15841993716,6cyclictest22-21ksoftirqd/122:03:121
15841993716,6cyclictest22-21ksoftirqd/121:25:271
15841993716,6cyclictest22-21ksoftirqd/100:49:281
15841993716,6cyclictest22-21ksoftirqd/100:00:131
15841993715,6cyclictest22-21ksoftirqd/122:15:111
15841993713,14cyclictest22-21ksoftirqd/123:05:301
15841993713,14cyclictest22-21ksoftirqd/121:37:321
15841993713,14cyclictest22-21ksoftirqd/100:25:391
15841993712,9cyclictest22-21ksoftirqd/122:05:191
15841993712,9cyclictest2219-21ssh21:54:551
15841993712,6cyclictest22-21ksoftirqd/100:50:541
15841993712,14cyclictest22-21ksoftirqd/121:24:511
1584099372,13cyclictest21840-21ssh23:55:200
15840993717,6cyclictest0-21swapper/023:10:320
1584099371,6cyclictest32259-21/usr/sbin/munin00:40:300
15840993710,6cyclictest3662-21open_inodes21:55:270
15841993615,6cyclictest22-21ksoftirqd/123:00:031
15841993612,6cyclictest22-21ksoftirqd/123:22:241
1584099362,15cyclictest0-21swapper/021:51:530
1584099361,17cyclictest0-21swapper/021:05:280
1584099361,14cyclictest20429-21fschecks_count21:00:150
1584099361,13cyclictest22983-21forks00:30:170
15840993610,5cyclictest8484-21df_inode00:15:130
15840993610,4cyclictest5035-21diskstats00:45:180
304042351,30sleep130409-21irqstats20:00:211
15841993516,6cyclictest22-21ksoftirqd/100:15:541
15841993514,6cyclictest22-21ksoftirqd/123:42:271
15841993510,14cyclictest22-21ksoftirqd/123:29:511
1584099353,4cyclictest26109-21basename00:00:110
1584099351,18cyclictest0-21swapper/023:33:070
15841993421,9cyclictest30944-21snmpd20:46:271
15841993414,7cyclictest22-21ksoftirqd/100:30:461
15841993412,6cyclictest0-21swapper/121:32:531
15840993419,11cyclictest23680-21diskmemload22:45:200
1584099341,24cyclictest0-21swapper/021:30:320
1584099341,15cyclictest18155-21diskstats00:25:170
1584099341,15cyclictest0-21swapper/022:03:090
1584099341,12cyclictest32697-21sh21:19:050
15840993410,7cyclictest3937-21/usr/sbin/munin20:15:280
71462331,4sleep07254-21fschecks_time23:40:130
197042330,5sleep041ktimersoftd/023:53:090
1584099338,6cyclictest25453-21runrttasks19:45:440
1584099338,5cyclictest0-21swapper/019:30:290
1584099333,7cyclictest0-21swapper/023:37:470
1584099332,15cyclictest0-21swapper/022:25:440
1584099331,6cyclictest27575-21diskmemload22:54:360
1584099331,4cyclictest14777-21kworker/0:322:30:020
1584099331,16cyclictest0-21swapper/022:24:180
1584099331,14cyclictest0-21swapper/000:53:350
1584099331,12cyclictest12856-21hddtemp_smartct22:05:190
15840993310,4cyclictest81rcu_preempt20:20:210
302762321,27sleep030305-21hddtemp_smartct20:00:180
168722321,4sleep116890-21iostat20:50:201
1584199321,6cyclictest1754-21ssh00:41:131
1584099328,5cyclictest11142-21/usr/sbin/munin20:35:210
1584099326,12cyclictest0-21swapper/021:26:150
1584099324,11cyclictest0-21swapper/021:37:000
1584099322,7cyclictest0-21swapper/020:49:190
1584099321,6cyclictest3973-21ssh21:22:170
1584099321,14cyclictest9353-21iostat20:30:210
1584099321,10cyclictest0-21swapper/000:05:330
12672320,5sleep00-21swapper/023:00:230
180082311,7sleep018010-21cron20:54:590
1584199311,16cyclictest0-21swapper/121:03:181
1584099319,6cyclictest9001-21wget22:35:150
1584099318,6cyclictest24037-21ssh21:43:270
1584099317,6cyclictest26720-21/usr/sbin/munin19:50:220
1584099317,11cyclictest0-21swapper/019:41:030
1584099315,22cyclictest12557-21aten_r4power_cu20:40:120
1584099314,13cyclictest0-21swapper/022:41:510
1584099313,11cyclictest0-21swapper/021:10:390
1584099312,5cyclictest2376-21NetworkManager00:21:520
1584099312,11cyclictest0-21swapper/019:57:220
1584099311,13cyclictest31193-21rm00:38:530
1584099311,13cyclictest13558-21ssh23:46:120
44292301,5sleep10-21swapper/120:16:021
1584199305,5cyclictest29475-21kernelversion21:15:201
1584199301,7cyclictest22808-21idleruntime19:40:201
1584099304,12cyclictest0-21swapper/020:26:560
1584099301,13cyclictest6080-21cat23:05:200
1584199295,4cyclictest25971-21latency_hist19:50:011
1584099294,11cyclictest0-21swapper/000:12:430
6942281,5sleep10-21swapper/120:07:281
24812281,5sleep10-21swapper/120:11:091
24812281,5sleep10-21swapper/120:11:091
1584199288,5cyclictest40950irq/93-i91520:25:181
1584199285,5cyclictest40950irq/93-i91520:20:181
1584199283,7cyclictest28203-21diskstats19:55:121
139782281,5sleep10-21swapper/120:42:111
15841992717,6cyclictest23151-21unixbench_multi21:05:291
121842271,4sleep10-21swapper/120:38:171
1584199267,5cyclictest40950irq/93-i91520:55:141
15841992617,6cyclictest22-21ksoftirqd/119:34:441
15841992515,6cyclictest26352-21cut19:50:171
15841992413,7cyclictest9583-21ntpq20:30:251
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional