You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-01 - 17:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack9slot4s (updated Tue Jul 01, 2025 12:59:01)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
299409911971171,23cyclictest31826-21invoke-rc.d07:30:001
299409911921169,19cyclictest17031-21bash12:30:201
298292563517,31sleep10-21swapper/107:24:271
287812535452,32sleep00-21swapper/007:20:240
2993999141118,16cyclictest31804-21munin-run07:30:000
2993999123116,4cyclictest0-21swapper/012:30:200
22012760,7sleep10-21swapper/111:50:271
102852430,10sleep02993999cyclictest10:45:300
2994099386,27cyclictest0-21swapper/109:40:181
2993999368,15cyclictest3081-21python09:00:290
2993999367,14cyclictest20967-21load12:40:220
2993999365,9cyclictest1904-21turbostat.cron07:35:010
2993999363,11cyclictest81rcu_preempt11:21:460
29939993626,6cyclictest31705-21pidof10:17:000
2993999361,21cyclictest21400-21sort09:50:190
2993999351,22cyclictest20105-21cron12:40:000
29939993510,12cyclictest13187-21fschecks_time12:20:170
2993999343,8cyclictest1152-21munin-run11:50:000
2993999341,18cyclictest1822-21cpuspeed_turbos10:25:150
78662330,7sleep12994099cyclictest12:05:241
78662330,7sleep12994099cyclictest12:05:231
2993999338,12cyclictest20555-21unixbench_multi08:20:320
2993999332,16cyclictest14324-21users10:55:330
29940993216,12cyclictest22-21ksoftirqd/112:10:271
2993999324,25cyclictest1969-21lldpad08:11:050
2993999321,5cyclictest81rcu_preempt08:00:550
2993999321,23cyclictest0-21swapper/009:49:560
268312321,27sleep126835-21irqstats11:30:221
230302321,27sleep023037-21df_inode09:55:150
2994099315,4cyclictest81rcu_preempt11:44:331
2994099315,4cyclictest81rcu_preempt10:07:201
2994099315,4cyclictest81rcu_preempt09:51:251
2994099311,4cyclictest81rcu_preempt10:53:221
2994099311,10cyclictest2106-21aten_r4power_cu07:35:121
2993999316,12cyclictest25815-21unixbench_multi10:00:320
2993999311,17cyclictest32744-21/usr/sbin/munin10:20:230
29939993111,7cyclictest0-21swapper/009:23:140
2993999311,16cyclictest24880-21ntp_kernel_pll_12:50:240
243542311,4sleep124361-21diskstats12:50:151
105312311,4sleep110611-21sensors_temp09:20:301
86602301,25sleep08679-21ntp_states07:50:270
323122301,5sleep00-21swapper/008:54:420
2994099305,4cyclictest81rcu_preempt10:22:141
2994099305,4cyclictest81rcu_preempt10:10:031
2994099304,4cyclictest81rcu_preempt11:20:071
2994099303,4cyclictest81rcu_preempt12:44:061
2994099303,4cyclictest81rcu_preempt11:37:051
2994099303,4cyclictest81rcu_preempt07:51:441
2994099301,5cyclictest0-21swapper/112:27:391
2994099301,4cyclictest81rcu_preempt12:39:421
29940993012,14cyclictest26879-21/usr/sbin/munin08:40:291
2993999302,14cyclictest9237-21diskstats12:10:160
2993999302,13cyclictest15180-21iostat12:25:190
2993999301,19cyclictest0-21swapper/012:47:430
2993999301,15cyclictest18706-21/usr/sbin/munin11:10:140
2993999301,12cyclictest25170-21sort08:35:130
29939993010,6cyclictest5222-21/usr/sbin/munin12:00:130
226362301,4sleep122686-21hddtemp_smartct12:45:191
11872301,5sleep11302-21sed07:30:291
91132291,5sleep00-21swapper/010:44:460
88482291,24sleep18852-21pidof10:41:591
75742291,5sleep00-21swapper/007:48:560
57322291,5sleep00-21swapper/007:44:160
50452291,5sleep00-21swapper/010:31:410
2994099299,16cyclictest0-21swapper/109:05:341
2994099297,12cyclictest40950irq/93-i91511:15:141
2994099295,3cyclictest81rcu_preempt08:22:091
2994099293,7cyclictest0-21swapper/110:25:151
2994099292,4cyclictest81rcu_preempt08:13:221
2994099292,23cyclictest0-21swapper/110:55:301
2994099291,4cyclictest81rcu_preempt11:08:441
2993999294,18cyclictest29443-21memory08:45:250
2993999294,18cyclictest17755-21diskstats08:15:170
2993999292,15cyclictest0-21swapper/011:15:150
2993999292,14cyclictest29124-21python11:35:300
2993999292,14cyclictest11681-21hddtemp_smartct10:50:200
2993999292,14cyclictest0-21swapper/010:37:370
29939992915,10cyclictest0-21swapper/007:38:240
29939992913,13cyclictest0-21swapper/009:18:570
2993999291,24cyclictest24637-21df11:25:150
2993999291,13cyclictest14174-21ntp_states09:30:240
29939992910,5cyclictest3020-21runrttasks12:05:040
29939992910,5cyclictest3020-21runrttasks12:05:030
29939992910,15cyclictest17332chrt08:59:410
279102291,5sleep00-21swapper/010:06:430
243572291,5sleep10-21swapper/109:59:001
183892291,5sleep00-21swapper/011:08:270
165362291,5sleep00-21swapper/009:35:530
153092291,5sleep00-21swapper/008:08:580
133432291,5sleep10-21swapper/109:30:091
301962281,5sleep10-21swapper/108:47:121
2994099283,4cyclictest81rcu_preempt11:59:391
2994099283,4cyclictest81rcu_preempt08:38:181
2994099282,4cyclictest81rcu_preempt08:05:581
2994099282,13cyclictest81rcu_preempt11:10:231
2994099281,4cyclictest81rcu_preempt10:46:331
2994099281,4cyclictest81rcu_preempt10:19:341
2994099281,4cyclictest81rcu_preempt08:31:111
2994099281,4cyclictest81rcu_preempt08:03:591
29940992810,6cyclictest31587-21expr08:50:271
2993999282,13cyclictest27705-21ntp_states08:40:250
2993999281,14cyclictest31271-21/usr/sbin/munin11:40:310
259612281,5sleep10-21swapper/111:28:571
204312281,5sleep10-21swapper/109:47:071
141252281,5sleep10-21swapper/112:21:541
129172281,5sleep10-21swapper/109:27:151
70352271,5sleep10-21swapper/110:37:521
65982271,5sleep10-21swapper/112:01:501
54592271,5sleep10-21swapper/107:41:251
50492271,5sleep10-21swapper/110:31:431
2994099275,4cyclictest81rcu_preempt12:19:531
2994099275,4cyclictest81rcu_preempt07:47:191
2994099271,8cyclictest0-21swapper/109:00:291
29940992718,5cyclictest22-21ksoftirqd/108:18:481
2993999276,5cyclictest0-21swapper/009:10:200
2993999274,17cyclictest15654-21memory11:00:290
2993999273,5cyclictest0-21swapper/010:10:010
2993999272,13cyclictest0-21swapper/011:54:400
2993999271,13cyclictest26721-21hddtemp_smartct11:30:180
2993999271,13cyclictest17603-21tune2fs09:40:160
2993999271,13cyclictest13065-21chrt09:28:480
2993999271,13cyclictest11751-21gzip12:15:270
246362271,5sleep10-21swapper/110:00:071
229002271,5sleep10-21swapper/108:29:351
13612271,5sleep10-21swapper/108:55:521
2994099268,14cyclictest10026-21entropy07:55:191
2994099264,10cyclictest0-21swapper/111:47:091
2994099262,19cyclictest8010-21diskstats09:15:271
2993999262,14cyclictest0-21swapper/008:31:490
2993999261,21cyclictest4608-21taskset11:55:530
2993999261,13cyclictest5353-21chrt09:06:560
29940992516,6cyclictest22-21ksoftirqd/109:36:331
29940992516,5cyclictest6352-21ls09:10:201
29940992515,6cyclictest15633-21sort11:00:291
29939992515,6cyclictest3020-21runrttasks07:57:240
2993999251,16cyclictest0-21swapper/008:26:580
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional