You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-20 - 03:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Fri Mar 20, 2026 00:59:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
56439912501220,27cyclictest0-21swapper/019:26:200
56449911131092,18cyclictest2645-21snmpd19:26:151
51882580533,29sleep00-21swapper/019:20:480
53132563517,31sleep10-21swapper/119:22:061
146592940,4sleep114657-21ssh21:10:241
148002900,4sleep1211ktimersoftd/119:45:241
93712630,4sleep041ktimersoftd/000:25:270
5643994417,7cyclictest3-21ksoftirqd/022:54:390
5643994318,15cyclictest3-21ksoftirqd/021:55:190
5643994310,29cyclictest0-21swapper/000:11:350
327562430,4sleep1564499cyclictest23:43:131
188372431,25sleep018848-21ssh21:48:320
188372431,25sleep018848-21ssh21:48:310
5643994218,8cyclictest3-21ksoftirqd/000:05:530
5643994217,6cyclictest3-21ksoftirqd/022:00:460
5643994216,7cyclictest3-21ksoftirqd/022:41:330
564499418,12cyclictest40550irq/92-i91500:10:111
5643994119,8cyclictest3-21ksoftirqd/023:27:110
5643994118,7cyclictest3-21ksoftirqd/019:45:010
5643994115,7cyclictest3-21ksoftirqd/022:23:310
5643994018,7cyclictest3-21ksoftirqd/021:28:040
5643994018,7cyclictest3-21ksoftirqd/020:01:420
5643994017,7cyclictest3-21ksoftirqd/023:36:300
5643994015,6cyclictest3-21ksoftirqd/023:03:160
5643994014,7cyclictest3-21ksoftirqd/023:11:260
564499394,5cyclictest0-21swapper/123:22:351
564499391,34cyclictest0-21swapper/121:21:161
5643993914,6cyclictest3-21ksoftirqd/022:36:550
5643993914,15cyclictest3-21ksoftirqd/023:06:090
5643993912,7cyclictest3-21ksoftirqd/023:58:530
5643993821,5cyclictest26707-21ssh22:30:110
5643993816,8cyclictest3-21ksoftirqd/021:50:410
36622381,3sleep13672-21ssh22:40:041
256132380,23sleep03-21ksoftirqd/022:29:140
564499377,13cyclictest0-21swapper/123:17:521
564499373,31cyclictest0-21swapper/121:04:351
564499373,31cyclictest0-21swapper/120:53:561
564499372,14cyclictest30951-21ssh22:00:271
5643993722,5cyclictest15662-21cron19:49:590
5643993722,5cyclictest11012-21ssh23:20:220
5643993716,7cyclictest3-21ksoftirqd/000:35:070
5643993715,7cyclictest3-21ksoftirqd/019:56:470
5643993715,6cyclictest3-21ksoftirqd/000:00:130
5643993714,12cyclictest16114-21ssh00:32:460
5643993713,14cyclictest3-21ksoftirqd/021:41:280
5643993712,14cyclictest3-21ksoftirqd/022:16:490
5643993712,14cyclictest3-21ksoftirqd/000:43:320
237242371,32sleep113459-21diskmemload00:07:001
91352361,31sleep113459-21diskmemload23:51:511
82542361,5sleep18264-21/usr/sbin/munin00:25:111
564499366,27cyclictest15466-21fschecks_time23:25:161
564499362,30cyclictest0-21swapper/100:40:361
5643993622,5cyclictest0-21swapper/000:20:130
5643993621,5cyclictest0-21swapper/020:25:290
5643993614,7cyclictest3-21ksoftirqd/023:54:160
5643993614,7cyclictest3-21ksoftirqd/023:17:340
5643993614,7cyclictest3-21ksoftirqd/022:11:320
5643993613,7cyclictest3-21ksoftirqd/023:32:370
5643993610,7cyclictest3-21ksoftirqd/023:43:200
5643993610,7cyclictest3-21ksoftirqd/000:16:530
215962361,31sleep121608-21rm22:58:171
16712361,31sleep11681-21ssh00:51:591
564499355,27cyclictest4084-21ssh22:06:001
564499354,28cyclictest0-21swapper/122:19:161
564499351,9cyclictest13459-21diskmemload23:55:411
564499351,30cyclictest22173-21ssh21:18:361
5643993521,4cyclictest19217-21interrupts21:15:180
5643993520,5cyclictest9096-21fschecks_time22:45:140
5643993515,7cyclictest3-21ksoftirqd/020:14:250
5643993515,6cyclictest3-21ksoftirqd/022:06:530
5643993513,7cyclictest3-21ksoftirqd/020:34:500
5643993513,6cyclictest3-21ksoftirqd/021:37:340
5643993510,9cyclictest9850-21/usr/sbin/munin21:00:210
564499346,5cyclictest40550irq/92-i91522:23:021
5644993427,5cyclictest0-21swapper/120:11:161
564499342,29cyclictest0-21swapper/123:13:151
5644993421,8cyclictest2645-21snmpd21:50:231
564499342,13cyclictest0-21swapper/123:37:451
5644993417,5cyclictest40550irq/92-i91521:35:001
564499341,15cyclictest1304-21ssh00:17:321
5643993420,5cyclictest4890-21missed_timers20:45:230
5643993417,6cyclictest0-21swapper/021:05:370
5643993414,6cyclictest3-21ksoftirqd/020:07:490
5643993414,6cyclictest3-21ksoftirqd/000:51:220
5643993413,7cyclictest3-21ksoftirqd/021:10:450
5643993412,6cyclictest3-21ksoftirqd/000:46:560
34922341,4sleep13587-21users20:40:311
110012341,29sleep111013-21rm22:46:541
109882340,5sleep0111rcuc/019:35:220
564499334,26cyclictest22-21ksoftirqd/121:40:521
564499332,28cyclictest0-21swapper/122:27:161
564499331,6cyclictest30881-21/usr/sbin/munin20:30:191
564499331,5cyclictest0-21swapper/100:48:351
564499331,15cyclictest24959-21/usr/sbin/munin21:55:161
5643993319,5cyclictest22760-21sh22:59:560
5643993312,7cyclictest3-21ksoftirqd/021:30:070
5643993312,6cyclictest3-21ksoftirqd/021:20:490
5643993310,10cyclictest5208-21ssh23:48:080
237522331,28sleep123769-21iostat_ios23:00:191
5644993225,4cyclictest0-21swapper/100:22:361
564499322,13cyclictest20314-21grep20:00:231
5643993218,5cyclictest4130-21latency_hist20:45:000
5643993216,5cyclictest7465-21taskset20:51:380
564499313,7cyclictest912-21ssh22:36:271
5644993123,5cyclictest5054-21ssh23:47:551
564499311,28cyclictest0-21swapper/121:28:361
564499311,27cyclictest0-21swapper/122:11:551
5643993118,5cyclictest0-21swapper/019:30:060
5643993115,5cyclictest8240-21df20:55:130
5643993114,6cyclictest1567-21users20:35:310
5643993114,6cyclictest1567-21users20:35:300
258192311,26sleep025826-21iostat_ios20:15:210
5644993019,7cyclictest31117-21ssh23:08:031
564499301,27cyclictest0-21swapper/119:34:151
564499301,27cyclictest0-21swapper/100:32:351
564399308,8cyclictest3-21ksoftirqd/019:50:080
5643993021,5cyclictest28132-21mailstats20:20:270
298612301,5sleep10-21swapper/122:33:241
92472291,5sleep10-21swapper/120:55:391
564499292,8cyclictest25524-21diskstats20:15:161
564499291,9cyclictest19849-21ssh00:36:111
5644992917,9cyclictest22-21ksoftirqd/121:37:441
564499289,15cyclictest2206-21dbus-daemon20:20:111
5644992816,9cyclictest22-21ksoftirqd/123:33:531
5644992816,9cyclictest22-21ksoftirqd/122:51:021
5644992816,9cyclictest22-21ksoftirqd/100:00:571
564499280,4cyclictest81rcu_preempt19:45:021
564499274,18cyclictest23197-21munin-run20:10:001
5644992712,11cyclictest16020-21tr19:50:111
5644992710,12cyclictest0-21swapper/121:47:371
5644992710,12cyclictest0-21swapper/121:47:361
302882271,5sleep10-21swapper/120:25:361
19362271,5sleep10-21swapper/120:39:111
19362271,5sleep10-21swapper/120:39:101
116342271,5sleep10-21swapper/119:37:211
564499251,6cyclictest81rcu_preempt19:56:471
5644992417,4cyclictest22-21ksoftirqd/120:45:141
564499234,10cyclictest0-21swapper/121:05:051
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional