You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-27 - 18:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack9slot4s (updated Thu Nov 27, 2025 12:59:03)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
211539911521107,24cyclictest21344-21cut07:25:130
211549911421125,10cyclictest29236-21udevd07:41:521
211539911091075,30cyclictest29231-21updatedb.mlocat07:41:520
209032777729,32sleep10-21swapper/107:23:031
206842583539,29sleep00-21swapper/007:20:460
2115499165160,3cyclictest0-21swapper/107:25:131
2930421360,7sleep02115399cyclictest09:42:580
2972821270,8sleep02115399cyclictest10:50:200
151872970,9sleep02115399cyclictest08:30:250
78272940,9sleep12115499cyclictest09:20:261
78272940,9sleep12115499cyclictest09:20:261
327402800,3sleep1211ktimersoftd/112:34:341
315962770,7sleep00-21swapper/011:25:300
13162760,5sleep0111rcuc/009:14:330
32892600,8sleep02115399cyclictest08:00:110
24332540,9sleep02115399cyclictest12:02:430
51932470,8sleep02115399cyclictest12:05:220
2115499444,4cyclictest81rcu_preempt11:33:171
156632430,8sleep12115499cyclictest08:31:011
2115399415,10cyclictest37342ssh10:56:390
2115399411,12cyclictest0-21swapper/011:38:570
21154994015,10cyclictest22-21ksoftirqd/111:21:351
284832381,4sleep128493-21ssh10:49:381
2115499386,5cyclictest81rcu_preempt09:40:251
2115399373,17cyclictest16863-21diskstats09:30:170
113232370,3sleep00-21swapper/012:11:300
2115499368,5cyclictest40950irq/93-i91507:50:291
2115399362,21cyclictest3600-21irqstats11:30:220
2115399361,23cyclictest27370-21cron07:39:000
21153993610,13cyclictest16489-21hddtemp_smartct11:10:170
304962351,30sleep130507-21ssh10:17:381
2115499359,5cyclictest81rcu_preempt10:31:191
2115399359,5cyclictest23178-21/usr/sbin/munin11:50:250
2115399355,26cyclictest24167-21tune2fs08:55:170
21153993513,6cyclictest15224-21iostat_ios10:35:200
21153993513,6cyclictest10855-21cstates12:45:150
2115399351,21cyclictest10126-21ssh09:23:080
2115399351,21cyclictest10126-21ssh09:23:070
2115499345,15cyclictest0-21swapper/111:25:301
21154993423,7cyclictest27818-21diskstats09:05:121
2115399349,10cyclictest40950irq/93-i91508:05:110
2115399346,4cyclictest81rcu_preempt10:42:190
2115399341,20cyclictest7010-21grep08:10:110
21153993411,8cyclictest7266-21ntp_states11:00:260
21153993410,13cyclictest16737-21wc08:35:160
253362331,27sleep025347-21ssh09:39:130
245812331,28sleep024745-21perl10:45:180
243332331,27sleep124346-21irqstats12:25:191
2115499334,25cyclictest0-21swapper/112:45:221
2115499331,11cyclictest81rcu_preempt12:41:371
2115399336,5cyclictest81rcu_preempt09:53:540
2115399332,18cyclictest0-21swapper/008:25:140
2115399331,19cyclictest26170-21perl11:20:180
2115399331,13cyclictest14677-21iostat12:15:200
189332331,27sleep118942-21ntp_states08:40:241
147852330,5sleep1201rcuc/110:35:141
11092331,4sleep11158-21df_inode12:35:121
2115499325,23cyclictest0-21swapper/112:16:371
2115499325,11cyclictest0-21swapper/109:46:571
2115499324,24cyclictest0-21swapper/109:34:341
2115499321,4cyclictest81rcu_preempt09:56:221
2115499321,4cyclictest81rcu_preempt09:56:221
2115499321,18cyclictest30783-21switchtime07:45:281
2115499321,18cyclictest20723-21ssh11:15:071
2115399321,4cyclictest81rcu_preempt10:21:480
2115399321,17cyclictest10875-21diskstats08:20:130
2115399321,12cyclictest0-21swapper/010:31:030
111772321,27sleep111187-21ssh11:38:261
87672311,26sleep18906-21cstates08:15:121
233542311,5sleep00-21swapper/012:24:570
2115499319,12cyclictest40950irq/93-i91512:04:131
2115499313,4cyclictest40950irq/93-i91511:46:401
2115499312,25cyclictest0-21swapper/111:53:581
2115499312,17cyclictest0-21swapper/111:07:041
2115499311,9cyclictest11132-21ssh12:11:131
21154993112,5cyclictest3020-21runrttasks10:23:101
21154993112,11cyclictest22-21ksoftirqd/111:00:241
2115399318,4cyclictest81rcu_preempt10:12:080
2115399312,4cyclictest81rcu_preempt12:30:330
2115399311,8cyclictest7866-21ssh10:27:410
2115399311,21cyclictest0-21swapper/010:06:080
2115399311,20cyclictest0-21swapper/010:18:070
2115399311,12cyclictest28047-21/usr/sbin/munin11:55:270
134612311,5sleep10-21swapper/110:00:171
99692301,5sleep00-21swapper/008:15:360
251902301,5sleep00-21swapper/007:31:160
213942301,5sleep10-21swapper/112:22:061
2115499308,5cyclictest81rcu_preempt11:43:361
21154993020,6cyclictest29685-21iostat10:50:181
21154993019,7cyclictest7845-21ssh09:54:451
2115499301,4cyclictest81rcu_preempt12:50:481
2115499301,15cyclictest24258-21ssh09:37:381
2115499300,4cyclictest81rcu_preempt09:16:291
2115399307,4cyclictest81rcu_preempt12:25:440
2115399306,5cyclictest81rcu_preempt11:09:480
2115399306,4cyclictest81rcu_preempt09:29:580
2115399304,5cyclictest81rcu_preempt10:03:310
2115399303,4cyclictest81rcu_preempt12:37:150
2115399302,13cyclictest31681-21irqstats09:45:200
2115399301,4cyclictest81rcu_preempt12:53:090
2115399300,11cyclictest0-21swapper/009:15:350
70472291,5sleep17057-21ssh10:26:321
4412291,5sleep00-21swapper/007:50:360
313562291,5sleep00-21swapper/007:49:040
298092291,24sleep129823-21sh11:57:261
2115499299,16cyclictest0-21swapper/110:12:431
2115499298,8cyclictest7003-21cpuspeed_turbos08:10:111
2115499296,12cyclictest40950irq/93-i91509:00:001
2115499295,4cyclictest81rcu_preempt08:06:251
2115499292,7cyclictest0-21swapper/109:25:051
21154992920,5cyclictest21950-21aten_r4power_po08:50:131
2115499291,7cyclictest8749-21ssh12:09:471
2115499291,7cyclictest4479-21ssh10:57:441
2115399299,6cyclictest1987-21gzip07:55:290
2115399296,4cyclictest81rcu_preempt11:40:220
2115399296,4cyclictest81rcu_preempt11:18:450
2115399293,13cyclictest19401-21ssh11:46:180
2115399292,13cyclictest23052-21grep08:50:300
21153992910,5cyclictest12242-21rm09:59:090
21153992910,5cyclictest12242-21rm09:59:090
192112291,5sleep10-21swapper/111:13:091
44022281,5sleep10-21swapper/108:01:161
273262281,5sleep10-21swapper/109:04:421
216412281,5sleep10-21swapper/108:49:231
216412281,5sleep10-21swapper/108:49:221
2115499283,4cyclictest81rcu_preempt07:32:301
2115499282,22cyclictest0-21swapper/110:40:091
2115499281,4cyclictest81rcu_preempt07:55:561
2115499281,14cyclictest18275-21diskstats10:05:161
2115399282,13cyclictest6780-21ntp_states12:40:260
2115399282,12cyclictest20387-21fschecks_time08:45:190
2115399282,12cyclictest20387-21fschecks_time08:45:190
2115399282,12cyclictest19304-21sed08:40:290
2115399281,13cyclictest25871-21wc09:00:100
288612271,5sleep00-21swapper/009:06:050
2115499274,17cyclictest26355-21iostat_ios07:35:201
2115499272,4cyclictest81rcu_preempt08:27:361
2115499271,22cyclictest32364-21ssh09:12:311
2115499269,13cyclictest11071-21fw_packets08:20:161
2115499264,10cyclictest0-21swapper/108:36:421
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional