You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-03-20 - 16:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems

x86 Intel Atom E3825 @1333 MHz, Linux 4.19.37-rt20 (Profile)

Latency plot of shadow in rack #b, slot #6
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot6s.osadl.org (updated Mon Mar 20, 2023 12:43:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
32725911030,3ptp4l241ktimersoftd/114:28:061
112322990,9sleep01071699cyclictest19:15:090
17462870,3sleep00-21swapper/015:20:210
186302840,3sleep00-21swapper/019:24:580
3272591820,1ptp4l241ktimersoftd/118:10:151
10721997715,16cyclictest18287-21chrt18:45:181
10721997617,16cyclictest8918-21ls17:19:581
10721997516,17cyclictest8426-21ls16:40:161
10721997420,17cyclictest16700-21sh19:21:401
10721997414,23cyclictest517-21snmp_o5powerctl19:40:131
10721997412,17cyclictest18620-21cut14:44:581
10721997315,22cyclictest27473-21users17:40:191
10721997315,15cyclictest709-21mailstats18:25:111
10721997216,15cyclictest16045-21snmp_o5powerctl18:05:191
10721997120,16cyclictest11190-21sh18:38:071
10721997117,16cyclictest23435-21diskstats18:14:591
10721997117,16cyclictest23435-21diskstats18:14:591
10721997020,15cyclictest16334-21ssh18:44:251
10721997018,17cyclictest30800-21expr16:29:541
10721997015,16cyclictest1305-21cat19:04:401
10721997014,16cyclictest22657-21cut16:09:581
10721996917,16cyclictest30291-21diskmemload16:50:011
10721996915,15cyclictest30291-21diskmemload19:50:141
10721996915,15cyclictest19620-21sh19:25:131
10721996915,15cyclictest10862-21latency_hist15:44:371
10721996914,16cyclictest18644-21sed17:30:161
10721996817,14cyclictest19565-21latency_hist16:04:371
10721996816,17cyclictest28859-21cat17:05:011
10721996816,15cyclictest24649-21gltestperf14:59:551
10721996815,15cyclictest20115-21grep16:05:001
10721996815,15cyclictest13809-21cstates14:34:541
10721996815,15cyclictest10203-21cstates19:14:561
10721996814,15cyclictest555-21grep15:20:021
10721996814,15cyclictest22303-21cut17:35:021
10721996718,16cyclictest1827-21runrttasks17:49:431
10721996718,15cyclictest16811-21sh17:29:251
10721996717,15cyclictest24789-21sh18:54:131
10721996715,15cyclictest27001-21cat19:34:411
10721996714,17cyclictest20737-21memory16:55:061
10721996714,16cyclictest28937-21mailstats18:20:171
10721996711,17cyclictest7009-21grep17:55:051
10721996710,17cyclictest27295-21cpuspeed_turbos19:34:571
10721996616,15cyclictest30291-21diskmemload19:08:021
10721996614,16cyclictest10892-21acpi14:29:471
10721996614,14cyclictest26985-21cut15:04:591
10721996516,15cyclictest28215-21cat16:24:381
10721996514,18cyclictest11480-21diskstats16:44:561
10721996513,15cyclictest30362-21latency_hist15:14:361
10721996418,15cyclictest12679-21taskset18:01:271
10721996417,16cyclictest6422-21taskset17:15:571
10721996415,16cyclictest30291-21diskmemload19:47:531
10721996414,15cyclictest24054-21acpi16:59:541
10721996315,17cyclictest30598-21smartctl17:44:591
10721996314,15cyclictest31679-21mailstats15:15:071
10721996311,12cyclictest8741-21ssh19:12:521
10721996216,16cyclictest2031-21ssh17:10:411
10721996212,13cyclictest21247-21timedrift14:50:121
10721996212,12cyclictest28661-21ssh18:58:261
10721996211,18cyclictest30493-21/usr/sbin/munin19:00:041
10721996117,14cyclictest25827-21taskset16:17:241
10721996116,15cyclictest13945-21taskset19:55:411
10721996116,14cyclictest5615-21taskset18:30:431
10721995814,17cyclictest3891-21mailstats16:35:111
10721995813,17cyclictest5661-21smartctl15:30:051
10721995717,16cyclictest15008-21seq15:52:591
10716995724,5cyclictest9-21ksoftirqd/014:29:360
10721995611,15cyclictest30370-21grep16:29:381
10721995517,16cyclictest4126-21seq15:28:021
1072199548,16cyclictest7120-21modprobe15:34:581
10721995417,15cyclictest16934-21seq15:55:591
10721995415,16cyclictest16427-21cut14:39:591
10721995310,16cyclictest22278-21cat14:54:521
144432510,2sleep014444-21ssh18:40:500
10721994911,12cyclictest11972-21phc2sys-jitter15:45:041
1071699482,9cyclictest24916-21irqstats17:00:070
1071699451,6cyclictest9-21ksoftirqd/016:29:580
1071699451,10cyclictest12703-21iwlist19:55:090
1071699442,8cyclictest6635-21apt-get15:34:490
32622430,3sleep00-21swapper/019:44:410
1071699426,4cyclictest121rcu_preempt18:58:370
1071699425,5cyclictest9-21ksoftirqd/014:49:360
1071699425,4cyclictest121rcu_preempt16:55:190
1071699423,5cyclictest9-21ksoftirqd/018:44:560
1071699416,9cyclictest121rcu_preempt16:40:130
1071699416,4cyclictest121rcu_preempt17:39:520
1071699416,4cyclictest121rcu_preempt15:27:420
1071699415,6cyclictest9-21ksoftirqd/018:31:210
1071699415,4cyclictest121rcu_preempt19:36:130
1071699415,4cyclictest121rcu_preempt19:32:500
1071699415,4cyclictest121rcu_preempt18:23:320
1071699415,4cyclictest121rcu_preempt18:06:110
1071699415,4cyclictest121rcu_preempt17:24:490
1071699415,4cyclictest121rcu_preempt16:51:200
1071699414,4cyclictest121rcu_preempt19:51:170
1071699414,4cyclictest121rcu_preempt16:38:150
1071699413,5cyclictest9-21ksoftirqd/016:48:460
1071699405,5cyclictest9-21ksoftirqd/017:08:060
1071699405,4cyclictest26893-21if_err_enp1s015:04:580
1071699405,4cyclictest121rcu_preempt18:15:590
1071699405,4cyclictest121rcu_preempt18:15:590
1071699405,4cyclictest121rcu_preempt17:36:220
1071699405,4cyclictest121rcu_preempt17:23:040
1071699405,4cyclictest121rcu_preempt17:14:480
1071699404,4cyclictest121rcu_preempt18:35:030
1071699404,4cyclictest121rcu_preempt18:03:350
1071699404,4cyclictest121rcu_preempt17:13:430
1071699403,4cyclictest121rcu_preempt19:12:150
1071699403,4cyclictest121rcu_preempt17:51:220
37852391,32sleep030291-21diskmemload19:45:040
1071699398,10cyclictest9-21ksoftirqd/017:49:400
1071699394,4cyclictest121rcu_preempt18:59:420
1071699394,4cyclictest121rcu_preempt18:59:420
1071699393,4cyclictest121rcu_preempt18:10:050
10716993915,19cyclictest9-21ksoftirqd/017:58:080
1071699391,4cyclictest121rcu_preempt15:15:060
1071699387,8cyclictest121rcu_preempt18:26:420
1071699386,10cyclictest9-21ksoftirqd/017:31:210
1071699384,4cyclictest121rcu_preempt19:22:310
10716993831,4cyclictest9-21ksoftirqd/016:25:050
1071699382,4cyclictest121rcu_preempt14:37:250
1071699376,9cyclictest121rcu_preempt18:52:150
10716993720,4cyclictest9-21ksoftirqd/015:00:120
10716993716,10cyclictest9-21ksoftirqd/014:49:570
1071699371,4cyclictest121rcu_preempt16:00:120
10716993629,4cyclictest9-21ksoftirqd/015:10:070
1071699361,4cyclictest9-21ksoftirqd/019:08:520
1071699361,4cyclictest121rcu_preempt15:54:570
1071699361,4cyclictest121rcu_preempt15:29:570
1071699351,4cyclictest121rcu_preempt16:09:510
1071699351,3cyclictest121rcu_preempt16:14:480
1071699344,23cyclictest22130-21apt-get14:54:510
10716993421,11cyclictest9-21ksoftirqd/016:19:580
1071699341,9cyclictest121rcu_preempt15:44:380
1071699340,31cyclictest42450irq/96-enp4s0-r15:50:130
1071699314,4cyclictest121rcu_preempt14:40:110
10716993122,5cyclictest9-21ksoftirqd/015:44:550
10716993121,6cyclictest20116-21fschecks_time16:05:000
1071699310,8cyclictest121rcu_preempt14:29:540
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional