You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-02-23 - 08:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack9slot4s.osadl.org (updated Fri Feb 23, 2024 00:59:00)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
37209912431213,27cyclictest2696-21kworker/0:519:30:000
34732669624,31sleep00-21swapper/019:22:590
36452562518,29sleep10-21swapper/119:24:481
2310721230,8sleep1372199cyclictest23:05:281
87112810,7sleep00-21swapper/021:00:310
3721998172,6cyclictest0-21swapper/119:30:001
109242760,2sleep00-21swapper/021:05:410
372099571,20cyclictest7067-21taskset22:21:400
372099503,6cyclictest18340-21python21:25:350
322992440,6sleep132005-21/usr/sbin/munin23:30:221
98842431,38sleep19798-21/usr/sbin/munin21:05:241
299072400,5sleep129908-21aten_r4power_vo00:50:161
372099369,15cyclictest20442-21cut00:25:190
142352353,3sleep014372-21timerwakeupswit22:40:390
3721993428,3cyclictest40750irq/92-i91521:39:541
372199341,6cyclictest31425-21iostat20:35:301
372199341,15cyclictest140542sh21:15:271
372099342,20cyclictest16375-21iostat19:55:250
372099342,20cyclictest12209-21ps22:35:380
372099341,20cyclictest18275-21iostat20:00:270
134432341,29sleep113461-21iostat_ios00:05:381
372199338,5cyclictest40750irq/92-i91520:20:321
372099331,11cyclictest81rcu_preempt19:30:430
294452331,4sleep129521-21users23:20:421
187962331,29sleep05535-21gdbus00:20:260
187962331,29sleep05535-21gdbus00:20:250
372199325,5cyclictest81rcu_preempt21:00:311
372199325,4cyclictest81rcu_preempt23:52:111
372199321,5cyclictest81rcu_preempt22:23:341
372099327,13cyclictest15170-21aten_r4power_po22:45:190
372099326,14cyclictest3227-21taskset23:39:580
217952321,4sleep121920-21/usr/sbin/munin20:10:241
80262311,26sleep08031-21ntp_states23:50:330
3721993115,4cyclictest40750irq/92-i91522:36:011
372199311,4cyclictest81rcu_preempt20:30:311
372199310,4cyclictest81rcu_preempt20:17:581
372099318,12cyclictest30574-21taskset20:34:090
372099312,17cyclictest0-21swapper/023:45:210
372099312,17cyclictest0-21swapper/019:40:340
372099312,16cyclictest0-21swapper/023:20:370
3720993115,13cyclictest3-21ksoftirqd/022:57:030
372099311,4cyclictest81rcu_preempt20:50:140
3720993111,16cyclictest19661-21wget20:05:180
167752311,4sleep116879-21df00:15:221
372199306,4cyclictest81rcu_preempt22:02:151
372199306,3cyclictest81rcu_preempt00:47:301
372199305,4cyclictest81rcu_preempt23:26:371
372199305,4cyclictest81rcu_preempt23:17:491
372199305,4cyclictest81rcu_preempt22:45:131
372199305,4cyclictest81rcu_preempt21:32:301
372199303,4cyclictest81rcu_preempt19:53:391
372199301,4cyclictest81rcu_preempt00:00:421
3721993014,13cyclictest22-21ksoftirqd/123:48:571
3721993012,6cyclictest40750irq/92-i91522:50:411
3721993010,4cyclictest81rcu_preempt21:15:001
372099302,14cyclictest10040-21iostat_ios22:30:270
3720993013,4cyclictest29828-21/usr/sbin/munin00:50:130
3720993012,5cyclictest21564-21ntp_states23:00:330
372099301,15cyclictest31632-21tr20:35:340
372099301,13cyclictest26913-21ls20:25:000
160432301,5sleep10-21swapper/100:11:041
95712291,5sleep10-21swapper/119:35:551
80552291,5sleep10-21swapper/122:25:251
372199295,4cyclictest81rcu_preempt23:57:461
372199295,4cyclictest81rcu_preempt21:43:571
372199295,3cyclictest81rcu_preempt19:55:441
372199294,4cyclictest81rcu_preempt19:43:471
372199293,4cyclictest81rcu_preempt22:12:081
372199293,4cyclictest81rcu_preempt00:24:301
372199293,4cyclictest81rcu_preempt00:24:291
3721992920,5cyclictest21968-21head23:00:371
372099299,16cyclictest23529-21cstates20:15:200
372099292,22cyclictest22746-21hddtemp_smartct00:30:310
372099291,15cyclictest26750-21aten_r4power_cu21:50:180
318502291,5sleep00-21swapper/022:01:270
214192291,5sleep00-21swapper/020:10:100
187162291,5sleep10-21swapper/121:26:481
95982281,5sleep00-21swapper/023:55:310
74802281,5sleep10-21swapper/120:57:281
37842281,5sleep10-21swapper/122:15:161
372199285,4cyclictest81rcu_preempt23:10:161
372199284,4cyclictest81rcu_preempt20:45:171
372199283,4cyclictest81rcu_preempt20:52:191
372199282,4cyclictest81rcu_preempt23:37:431
372199281,4cyclictest81rcu_preempt00:43:061
372099289,5cyclictest14414-21runrttasks00:08:390
372099286,5cyclictest6624-21runrttasks20:55:280
372099284,8cyclictest32666-21diskstats22:05:240
372099282,13cyclictest4107-21latency23:40:300
372099282,13cyclictest24775-21df_inode23:10:250
372099282,13cyclictest11268-21df_inode00:00:240
3720992819,5cyclictest5535-21gdbus00:40:550
3720992812,12cyclictest26548-21cron21:50:000
3720992811,5cyclictest27894-21ntp_kernel_pll_20:25:320
372099281,13cyclictest26713-21fschecks_count23:15:260
372099281,13cyclictest19655-21grep21:30:240
3720992810,6cyclictest14405-21which19:50:250
316022281,5sleep00-21swapper/023:27:400
212342281,5sleep10-21swapper/120:09:491
19582281,5sleep10-21swapper/120:44:501
148602281,5sleep10-21swapper/122:44:301
130292281,5sleep00-21swapper/021:11:240
50422271,5sleep10-21swapper/123:43:571
3721992718,5cyclictest26095-21unixbench_singl21:45:451
372199271,4cyclictest81rcu_preempt22:56:091
372199271,4cyclictest81rcu_preempt19:31:491
372099272,13cyclictest24961-21ntp_states00:35:420
372099272,12cyclictest8555-21df_inode19:35:220
372099272,12cyclictest2214-21dbus-daemon22:15:010
372099271,4cyclictest81rcu_preempt20:48:520
372099271,18cyclictest0-21swapper/023:07:000
372099271,16cyclictest0-21swapper/022:25:040
372099271,14cyclictest23490-21/usr/sbin/munin21:40:270
372099271,14cyclictest17095-21hddtemp_smartct00:15:270
372099271,14cyclictest16313-21/usr/sbin/munin21:20:330
3720992710,5cyclictest12889-21/usr/sbin/munin19:45:340
3720992710,5cyclictest0-21swapper/021:20:010
300992271,4sleep10-21swapper/121:57:571
296712271,23sleep029811-21unixbench_singl21:55:390
284182271,5sleep10-21swapper/120:26:161
281242271,5sleep10-21swapper/121:51:561
255632271,5sleep10-21swapper/100:37:221
219052271,5sleep10-21swapper/100:28:171
193392271,5sleep10-21swapper/120:04:361
169152271,5sleep10-21swapper/121:22:341
134062271,5sleep10-21swapper/119:46:381
108382271,5sleep10-21swapper/122:31:531
372199262,9cyclictest22746-21hddtemp_smartct00:30:311
372199261,11cyclictest32589-21tr22:05:211
372099261,21cyclictest820-21seq20:40:280
372099261,14cyclictest32662-21/usr/sbin/munin23:30:280
372099261,13cyclictest14909-21/usr/sbin/munin00:10:260
372099259,5cyclictest29139-21unixbench_multi00:45:440
372099259,5cyclictest21960-21/usr/sbin/munin21:35:360
372099251,9cyclictest2214-21dbus-daemon22:10:300
372099244,10cyclictest0-21swapper/022:50:200
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional