You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-27 - 07:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Thu Nov 27, 2025 00:59:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
48809912361201,24cyclictest0-21swapper/019:25:100
48819911831160,19cyclictest25088-21irqstats20:15:221
48819911741143,27cyclictest3020-21runrttasks19:25:111
46802702656,30sleep00-21swapper/019:23:340
47302448401,32sleep10-21swapper/119:24:061
488099123117,4cyclictest41ktimersoftd/020:15:220
3252921090,20sleep03-21ksoftirqd/000:54:130
151122890,4sleep00-21swapper/021:47:540
200592850,8sleep0488099cyclictest23:00:130
278792770,4sleep10-21swapper/100:15:161
68052760,4sleep00-21swapper/021:00:120
178272640,9sleep0488099cyclictest21:50:260
81012570,22sleep03-21ksoftirqd/022:14:370
488099521,17cyclictest1943-21ssh22:40:430
264432440,4sleep0111rcuc/022:33:120
316772420,9sleep0488099cyclictest22:05:170
196932420,3sleep0111rcuc/000:06:010
152062420,24sleep115205-21ssh00:35:211
238772410,4sleep1211ktimersoftd/100:45:011
488199401,16cyclictest20083-21taskset21:20:021
4880994022,6cyclictest3-21ksoftirqd/022:24:350
70362390,9sleep1488199cyclictest19:30:021
131882391,34sleep113201-21sh22:52:531
4880993828,6cyclictest3122-21diskstats20:50:140
4880993820,6cyclictest3-21ksoftirqd/022:48:120
4881993712,5cyclictest102202diskmemload00:54:131
4881993710,6cyclictest121292ssh23:59:051
488099373,14cyclictest23235-21hddtemp_smartct00:10:170
304342371,32sleep130445-21sh22:04:071
4880993614,11cyclictest3-21ksoftirqd/023:57:330
236702361,4sleep117090-1kworker/1:2H00:10:261
488199354,5cyclictest81rcu_preempt23:51:161
488099357,4cyclictest81rcu_preempt21:17:500
312452351,29sleep131255-21ssh22:38:111
29702351,30sleep12987-21/usr/sbin/munin21:35:201
240042351,29sleep024018-21ssh23:37:270
488199342,28cyclictest0-21swapper/123:45:491
488099348,12cyclictest1863-21sh00:21:120
488099344,5cyclictest81rcu_preempt23:50:260
488099344,5cyclictest81rcu_preempt00:48:130
488099344,4cyclictest81rcu_preempt19:30:260
4880993423,6cyclictest12767-21mailstats23:25:310
488099341,4cyclictest81rcu_preempt23:45:000
4880993410,4cyclictest81rcu_preempt22:03:480
244072342,27sleep10-21swapper/121:57:241
201232341,29sleep120137-21ssh23:33:501
199062340,5sleep119905-21sendmail21:20:001
89882331,4sleep19164-21perl22:15:151
488199339,10cyclictest21916-21sed20:05:291
488199338,21cyclictest0-21swapper/121:29:251
488199337,13cyclictest0-21swapper/123:13:381
488199336,23cyclictest0-21swapper/121:32:141
488099339,6cyclictest0-21swapper/000:02:020
488099338,5cyclictest0-21swapper/021:44:140
4880993313,5cyclictest19432-21aten_r4power_po00:40:120
4880993312,16cyclictest11601-21df19:40:140
488099331,20cyclictest9978-21taskset21:07:410
4880993311,17cyclictest3481-21/usr/sbin/munin21:35:270
187612331,9sleep00-21swapper/022:58:590
488199324,4cyclictest81rcu_preempt21:01:051
4881993210,7cyclictest2220-21dbus-daemon20:55:401
488099325,4cyclictest81rcu_preempt22:17:220
488099323,4cyclictest81rcu_preempt23:30:230
488099321,4cyclictest81rcu_preempt23:48:310
488099321,4cyclictest81rcu_preempt23:05:060
488099321,4cyclictest81rcu_preempt21:20:390
488099321,14cyclictest0-21swapper/023:17:250
200482321,5sleep10-21swapper/122:25:561
107982321,27sleep010816-21diskstats21:10:140
488199316,4cyclictest81rcu_preempt22:58:211
488199315,10cyclictest22-21ksoftirqd/123:26:001
488199313,4cyclictest81rcu_preempt20:36:431
4881993121,6cyclictest17421-21sh00:04:461
4881993118,4cyclictest22-21ksoftirqd/123:00:131
488099313,5cyclictest81rcu_preempt21:55:150
488099311,5cyclictest81rcu_preempt22:52:330
488099311,4cyclictest81rcu_preempt23:20:310
488099311,4cyclictest81rcu_preempt20:37:080
488099311,4cyclictest81rcu_preempt00:26:340
4880993114,6cyclictest3-21ksoftirqd/021:33:420
129142311,5sleep00-21swapper/000:33:060
86382301,5sleep10-21swapper/122:48:151
488199304,4cyclictest81rcu_preempt20:31:361
4881993021,6cyclictest16592-21ssh22:22:551
488199302,16cyclictest0-21swapper/123:44:171
488199301,8cyclictest20647-21ssh00:07:251
488199301,8cyclictest10203-21ssh21:42:481
4881993014,4cyclictest9033-21seq00:29:261
4881993012,8cyclictest22-21ksoftirqd/123:06:401
488099304,12cyclictest0-21swapper/000:35:330
488099302,15cyclictest28682-21ssh00:15:280
488099302,15cyclictest19349-21/usr/sbin/munin22:25:260
488099302,13cyclictest25973-21ntp_states21:25:240
488099301,4cyclictest81rcu_preempt20:55:350
4880993014,5cyclictest3-21ksoftirqd/020:24:340
4880993012,5cyclictest29740-21diskstats23:10:150
4880993011,5cyclictest21259-21/usr/sbin/munin20:05:180
26452301,5sleep10-21swapper/100:22:181
160252301,5sleep10-21swapper/121:49:161
99242291,23sleep19934-21ssh23:23:011
488199296,4cyclictest81rcu_preempt20:11:491
4881992923,4cyclictest40950irq/93-i91522:40:191
4881992920,6cyclictest24782-21if_eth000:45:201
4881992919,6cyclictest23156-21/usr/sbin/munin22:30:171
488199291,5cyclictest81rcu_preempt20:40:011
488199291,4cyclictest81rcu_preempt19:40:131
4881992913,12cyclictest22-21ksoftirqd/123:16:591
488099298,17cyclictest15439-21grep19:50:150
488099293,5cyclictest81rcu_preempt22:35:270
4880992914,5cyclictest3-21ksoftirqd/020:12:220
283792291,5sleep10-21swapper/120:26:271
238562291,5sleep10-21swapper/123:37:131
183402291,5sleep00-21swapper/019:56:370
148212291,5sleep00-21swapper/019:49:040
109492291,5sleep00-21swapper/019:38:050
100552291,5sleep10-21swapper/121:08:281
55212281,5sleep10-21swapper/122:10:531
5442281,4sleep00-21swapper/020:43:470
488199289,15cyclictest0-21swapper/121:51:011
488199289,15cyclictest0-21swapper/100:33:551
488199284,7cyclictest15674-21iostat_ios19:50:191
488199283,4cyclictest81rcu_preempt19:45:431
4881992811,12cyclictest2220-21dbus-daemon22:07:161
4880992812,5cyclictest3-21ksoftirqd/020:02:580
488099281,15cyclictest30041-21/usr/sbin/munin20:30:240
4880992810,7cyclictest1854-21python20:45:260
282562281,5sleep00-21swapper/020:25:100
279762281,5sleep10-21swapper/120:23:491
129812281,4sleep10-21swapper/121:12:001
488199274,8cyclictest1127-21diskstats20:45:151
110302271,5sleep10-21swapper/119:38:581
488199261,3cyclictest81rcu_preempt20:51:361
202612261,5sleep10-21swapper/120:01:531
488199250,4cyclictest81rcu_preempt19:57:291
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional