You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-25 - 11:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack9slot4s (updated Thu Apr 25, 2024 00:59:00)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
271259912471227,17cyclictest8764-21kworker/1:319:25:141
271249912301200,19cyclictest0-21swapper/019:25:140
271259911591139,17cyclictest0-21swapper/119:40:131
260462566517,32sleep10-21swapper/119:20:201
268112485438,32sleep00-21swapper/019:22:140
1480621420,2sleep00-21swapper/020:20:350
586221100,7sleep05855-21fschecks_count22:50:170
271249910782,22cyclictest0-21swapper/019:40:130
174572600,8sleep02712499cyclictest21:55:160
256302520,3sleep10-21swapper/120:50:201
27125994616,9cyclictest22-21ksoftirqd/120:30:361
2712599417,6cyclictest22-21ksoftirqd/122:40:331
27125994124,6cyclictest22-21ksoftirqd/122:15:271
27125994110,4cyclictest81rcu_preempt23:50:471
2712599406,26cyclictest40950irq/93-i91520:15:351
2712599395,4cyclictest81rcu_preempt19:55:331
2712599392,6cyclictest22-21ksoftirqd/123:15:311
2712599392,6cyclictest22-21ksoftirqd/120:35:241
2712599391,7cyclictest22-21ksoftirqd/123:00:151
2712599391,7cyclictest22-21ksoftirqd/121:40:351
2712599391,7cyclictest22-21ksoftirqd/121:20:321
2712599391,7cyclictest22-21ksoftirqd/121:00:351
2712599391,7cyclictest22-21ksoftirqd/100:40:341
2712599391,5cyclictest22-21ksoftirqd/123:25:231
27125993914,4cyclictest22-21ksoftirqd/119:50:191
2712599389,4cyclictest81rcu_preempt22:30:111
2712599389,4cyclictest81rcu_preempt00:50:201
2712599384,4cyclictest81rcu_preempt20:00:141
2712599381,7cyclictest22-21ksoftirqd/123:10:291
2712599381,7cyclictest22-21ksoftirqd/122:55:141
2712599381,7cyclictest22-21ksoftirqd/122:20:261
2712599381,7cyclictest22-21ksoftirqd/121:25:151
27125993812,14cyclictest22-21ksoftirqd/120:58:021
55582372,31sleep15652-21hddtemp_smartct00:15:261
4282371,32sleep0454-21ls00:00:350
2712599373,6cyclictest22-21ksoftirqd/119:45:171
2712599371,6cyclictest22-21ksoftirqd/121:10:351
2712599371,6cyclictest11598-21ntpq00:30:221
2712599371,6cyclictest11598-21ntpq00:30:211
7752360,8sleep02712499cyclictest21:10:260
2712599366,4cyclictest81rcu_preempt21:50:091
2712599361,7cyclictest22-21ksoftirqd/100:35:191
2712599361,6cyclictest22-21ksoftirqd/120:09:471
2712499361,10cyclictest81rcu_preempt23:42:370
2712499360,11cyclictest81rcu_preempt21:45:070
2712599353,5cyclictest22-21ksoftirqd/121:17:381
2712599351,7cyclictest22-21ksoftirqd/120:11:151
2712599351,6cyclictest22-21ksoftirqd/121:30:531
2712599351,6cyclictest22-21ksoftirqd/120:45:121
27125993514,4cyclictest81rcu_preempt23:40:241
27125993511,4cyclictest81rcu_preempt19:39:251
44282341,28sleep14550-21users00:10:401
2712599345,4cyclictest81rcu_preempt22:10:131
2712599342,5cyclictest22-21ksoftirqd/121:47:441
27125993419,4cyclictest22-21ksoftirqd/122:45:191
2712599341,6cyclictest22-21ksoftirqd/123:59:481
2712599341,6cyclictest22-21ksoftirqd/120:20:091
2712599341,5cyclictest22-21ksoftirqd/123:21:491
27125993412,6cyclictest22-21ksoftirqd/123:05:211
2712499341,21cyclictest25108-21hostname20:50:100
2712599334,4cyclictest81rcu_preempt19:33:561
27125993316,4cyclictest22-21ksoftirqd/123:37:211
2712599331,5cyclictest22-21ksoftirqd/100:22:041
2712599331,4cyclictest81rcu_preempt23:45:311
27125993312,4cyclictest81rcu_preempt22:00:231
2712499338,5cyclictest32376-21sort22:35:100
2712499335,13cyclictest6204-21gdbus00:41:420
128292331,9sleep00-21swapper/023:09:300
2712599329,4cyclictest81rcu_preempt22:06:461
27125993219,4cyclictest22-21ksoftirqd/121:55:251
27125993219,4cyclictest22-21ksoftirqd/121:05:241
27125993217,4cyclictest22-21ksoftirqd/121:35:181
2712599321,5cyclictest81rcu_preempt23:30:221
27125993215,6cyclictest22-21ksoftirqd/100:07:591
2712499327,11cyclictest0-21swapper/020:15:360
2712499322,4cyclictest81rcu_preempt23:20:140
2712499321,19cyclictest25614-21grep22:15:290
27124993210,18cyclictest40532chrt19:53:440
296092310,8sleep00-21swapper/019:30:270
2712599318,19cyclictest17200-21missed_timers00:45:271
2712599313,4cyclictest81rcu_preempt22:35:141
27125993116,12cyclictest22-21ksoftirqd/120:40:091
27125993113,5cyclictest22-21ksoftirqd/122:50:251
2712499317,11cyclictest0-21swapper/022:34:390
2712499312,17cyclictest6204-21gdbus20:05:440
27124993118,4cyclictest3-21ksoftirqd/022:40:330
27124993118,4cyclictest3-21ksoftirqd/021:40:310
2712499311,4cyclictest81rcu_preempt23:48:470
2712499311,4cyclictest81rcu_preempt21:50:060
27124993112,15cyclictest6186-21python00:15:420
2712499311,12cyclictest0-21swapper/021:28:500
27124993110,4cyclictest3-21ksoftirqd/021:30:290
2712499302,14cyclictest0-21swapper/022:10:160
2712499301,4cyclictest81rcu_preempt23:55:570
2712499301,4cyclictest81rcu_preempt20:12:500
2712499301,26cyclictest132-21jbd2/sda2-823:30:400
27124993012,5cyclictest13588-21/usr/sbin/munin23:10:180
27124993011,6cyclictest30065-21unixbench_singl21:00:340
2712499301,14cyclictest20510-21ls23:30:010
2712499301,14cyclictest11117-21sort00:30:140
2712499301,14cyclictest11117-21sort00:30:140
2712499300,4cyclictest81rcu_preempt23:16:020
2712599293,4cyclictest81rcu_preempt00:03:001
2712599291,4cyclictest81rcu_preempt22:25:281
2712599291,4cyclictest81rcu_preempt20:27:291
2712599291,11cyclictest22-21ksoftirqd/100:25:191
2712499298,6cyclictest0-21swapper/021:21:530
2712499293,14cyclictest0-21swapper/020:35:180
2712499292,4cyclictest81rcu_preempt21:15:130
2712499292,15cyclictest0-21swapper/000:06:560
2712499292,14cyclictest9952-21python00:25:330
27124992915,11cyclictest3-21ksoftirqd/021:06:190
2712499291,4cyclictest81rcu_preempt00:10:160
27124992911,5cyclictest23102-21/usr/sbin/munin23:35:230
27124992910,7cyclictest1126-21snmpd22:20:190
2712499290,4cyclictest81rcu_preempt19:35:020
85002281,5sleep00-21swapper/000:22:190
51382281,5sleep00-21swapper/022:47:480
2712499282,14cyclictest0-21swapper/022:55:260
2712499282,13cyclictest6739-21hddtemp_smartct20:00:210
2712499282,13cyclictest18127-21irqstats20:30:270
2712499281,4cyclictest81rcu_preempt22:03:240
2712499281,4cyclictest81rcu_preempt19:49:320
27124992814,11cyclictest13356-21sort00:35:260
27124992811,5cyclictest18362-21/usr/sbin/munin00:50:080
2712499281,14cyclictest28893-21grep23:50:270
2712499281,13cyclictest16778-21df_inode00:45:160
2712499281,13cyclictest10460-21tr21:35:310
27124992810,5cyclictest29286-21ntpq22:25:300
222232281,5sleep00-21swapper/022:05:440
2712499275,10cyclictest81rcu_preempt20:40:300
2712499272,13cyclictest27518-21irqstats20:55:220
2712499271,13cyclictest3020-21runrttasks23:00:160
2712499271,13cyclictest23655-21/usr/sbin/munin20:45:200
172842271,5sleep00-21swapper/020:29:210
2712499262,4cyclictest81rcu_preempt19:55:190
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional