You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-07-06 - 17:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems

x86 Intel Atom E3825 @1333 MHz, Linux 4.19.37-rt20 (Profile)

Latency plot of shadow in rack #b, slot #6
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot6s.osadl.org (updated Wed Jul 06, 2022 12:43:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1917225070,7sleep10-21swapper/112:58:351
3272591940,1ptp4l241ktimersoftd/113:58:161
25512940,4sleep10-21swapper/116:51:461
4682920,6sleep00-21swapper/016:48:430
231102920,9sleep12037799cyclictest18:32:521
2037699764,48cyclictest16436-21apt-get14:08:250
2037699752,38cyclictest26049-21python14:28:420
20376997511,35cyclictest845-21runrttasks18:07:520
2037699744,38cyclictest25120-21diskstats15:23:330
2037699741,42cyclictest2035-21ssh17:28:440
2037699731,45cyclictest19084-21ssh15:15:520
20376997314,36cyclictest5585-21sh18:11:570
2037699723,35cyclictest22804-21apt-get14:23:290
2037699721,44cyclictest21463-21ssh18:29:450
2037699721,39cyclictest7483-21ssh17:35:080
2037699721,39cyclictest6858-21ssh16:56:530
2037699721,38cyclictest12707-21seq18:19:180
20376997211,37cyclictest25672-21sh17:57:390
3272591710,1ptp4l241ktimersoftd/114:23:441
2037699714,37cyclictest1488-21apt-get13:33:260
2037699712,37cyclictest27373-21snmp_o5powerctl16:03:470
2037699711,42cyclictest30481-21ssh16:46:150
2037699711,39cyclictest9578-21ssh18:16:240
2037699711,38cyclictest28347-21ssh17:59:360
20376997113,35cyclictest20157-21sh15:55:320
20377997019,9cyclictest25-21ksoftirqd/118:18:231
2037699703,37cyclictest10701-21apt-get16:23:290
2037699703,33cyclictest21461-21ssh16:35:090
2037699702,40cyclictest25748-21irqstats17:18:390
2037699701,46cyclictest651-21latency_hist17:28:190
2037699701,36cyclictest3364-21ssh16:13:440
20376997011,32cyclictest20374-21cyclictest15:20:370
2037699692,36cyclictest2425-21python14:48:430
2037699692,36cyclictest2091-21ssh15:33:430
2037699692,34cyclictest7904-21apt-get15:03:300
2037699691,41cyclictest12110-21ssh15:46:120
20376996912,36cyclictest25592-21ssh16:02:550
20376996911,28cyclictest14895-21ssh17:43:460
20376996910,35cyclictest16923-21sh18:24:150
2037699682,36cyclictest17442-21snmpd17:08:560
2037699682,35cyclictest5481-21apt-get13:43:190
2037699681,45cyclictest7559-21diskmemload16:22:320
2037699681,41cyclictest17050-21ssh15:52:420
2037699681,37cyclictest23988-21ssh16:38:380
2037699681,36cyclictest8375-21smartctl16:58:370
2037699681,36cyclictest5030-21seq14:53:530
2037699681,36cyclictest483-21smartctl13:28:410
2037699681,35cyclictest15814-21ssh15:12:470
2037699672,35cyclictest197-21systemd-journal13:43:000
2037699671,42cyclictest11994-21ssh17:03:220
2037699671,41cyclictest28983-21seq14:36:560
2037699671,41cyclictest16737-21ssh16:29:130
2037699671,40cyclictest20021-21ssh17:49:540
2037699671,36cyclictest5031-21/usr/sbin/munin15:38:430
2037699671,35cyclictest30353-21ssh16:08:300
2037699662,34cyclictest21400-21ssh17:13:410
2037699661,34cyclictest7134-21taskset14:58:530
2037699652,39cyclictest14251-21chrt14:03:260
2037699651,40cyclictest11186-21runrttasks17:39:070
2037699643,37cyclictest11249-21python13:53:420
2037699641,40cyclictest31503-21sshd13:27:460
2037699641,40cyclictest13767-21taskset13:59:460
2037699641,39cyclictest31430-21ssh15:30:120
2037699641,39cyclictest1022-21runrttasks14:48:030
2037699641,38cyclictest31341-21runrttasks14:43:020
2037699632,38cyclictest18984-21hddtemp_smartct14:13:330
2037699621,38cyclictest24461-21systemctl13:08:550
200662624,9sleep00-21swapper/013:00:290
2037699611,36cyclictest21177-21ps13:03:360
2037699601,36cyclictest26600-21smartctl13:13:410
2037699601,36cyclictest197-21systemd-journal13:49:450
2037699592,33cyclictest28706-21python13:18:410
2037699581,35cyclictest20615-21/usr/sbin/munin14:18:320
2037699581,35cyclictest20615-21/usr/sbin/munin14:18:320
3272591510,1ptp4l241ktimersoftd/116:27:271
3272591510,1ptp4l241ktimersoftd/116:22:091
3272591500,2ptp4l241ktimersoftd/115:33:271
3272591500,1ptp4l241ktimersoftd/117:45:591
3272591500,1ptp4l241ktimersoftd/117:19:061
3272591500,1ptp4l241ktimersoftd/116:28:381
3272591500,1ptp4l241ktimersoftd/116:17:101
3272591500,1ptp4l241ktimersoftd/115:47:101
3272591500,1ptp4l241ktimersoftd/115:04:051
2037799503,13cyclictest29782-21ssh16:07:581
20377995019,9cyclictest25-21ksoftirqd/117:28:021
3272591490,1ptp4l241ktimersoftd/118:14:001
3272591490,1ptp4l241ktimersoftd/118:08:351
3272591490,1ptp4l241ktimersoftd/118:06:071
3272591490,1ptp4l241ktimersoftd/117:53:221
3272591490,1ptp4l241ktimersoftd/117:48:281
3272591490,1ptp4l241ktimersoftd/117:43:161
3272591490,1ptp4l241ktimersoftd/117:34:491
3272591490,1ptp4l241ktimersoftd/117:30:591
3272591490,1ptp4l241ktimersoftd/117:13:021
3272591490,1ptp4l241ktimersoftd/116:53:571
3272591490,1ptp4l241ktimersoftd/116:48:131
3272591490,1ptp4l241ktimersoftd/116:11:341
3272591490,1ptp4l241ktimersoftd/115:59:401
3272591490,1ptp4l241ktimersoftd/115:58:021
3272591490,1ptp4l241ktimersoftd/115:49:101
3272591490,1ptp4l241ktimersoftd/115:42:151
3272591490,1ptp4l241ktimersoftd/115:30:511
3272591490,1ptp4l241ktimersoftd/115:24:011
3272591490,1ptp4l241ktimersoftd/115:18:271
3272591490,1ptp4l241ktimersoftd/115:17:361
3272591490,1ptp4l241ktimersoftd/115:12:171
3272591490,1ptp4l241ktimersoftd/114:41:111
3272591490,1ptp4l241ktimersoftd/113:23:371
3272591490,1ptp4l241ktimersoftd/113:18:321
2037799494,17cyclictest9225-21snmp_o5powerctl16:58:531
3272591480,1ptp4l241ktimersoftd/117:18:061
3272591480,1ptp4l241ktimersoftd/116:37:361
3272591480,1ptp4l241ktimersoftd/113:13:361
3272591470,1ptp4l241ktimersoftd/116:42:481
3272591470,1ptp4l241ktimersoftd/115:00:101
3272591460,1ptp4l241ktimersoftd/113:33:431
3272591450,1ptp4l241ktimersoftd/113:03:281
3272591440,1ptp4l241ktimersoftd/113:47:041
2037799444,14cyclictest14565-21ssh17:05:271
2037799443,13cyclictest7559-21diskmemload18:26:101
2037799442,16cyclictest17442-21snmpd13:41:401
3272591430,1ptp4l241ktimersoftd/114:21:371
3272591430,1ptp4l241ktimersoftd/114:21:371
2037799428,11cyclictest12405-21df13:58:301
2037799412,10cyclictest28348-21ssh17:59:361
2037799411,13cyclictest3168-21apt-get14:53:191
3272591380,1ptp4l241ktimersoftd/114:47:381
3272591380,1ptp4l241ktimersoftd/113:08:471
3272591370,1ptp4l241ktimersoftd/114:17:141
2037799376,8cyclictest241ktimersoftd/114:48:461
2037799376,8cyclictest241ktimersoftd/113:28:501
2037799375,11cyclictest15469-21/usr/sbin/munin14:03:431
2037799374,11cyclictest9126-21snmp_o5powerctl13:48:441
2037799365,9cyclictest241ktimersoftd/114:33:391
2037799365,9cyclictest241ktimersoftd/114:28:501
2037799355,9cyclictest241ktimersoftd/114:08:411
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional