You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-03-24 - 07:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems

x86 Intel Atom E3825 @1333 MHz, Linux 4.19.37-rt20 (Profile)

Latency plot of shadow in rack #b, slot #6
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot6s.osadl.org (updated Fri Mar 24, 2023 00:43:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
32725911140,1ptp4l241ktimersoftd/102:48:241
32725911090,2ptp4l241ktimersoftd/107:28:501
32725911070,2ptp4l241ktimersoftd/107:06:241
32725911070,2ptp4l241ktimersoftd/106:25:271
32725911070,2ptp4l241ktimersoftd/105:21:461
3039021060,3sleep10-21swapper/105:58:041
32725911050,3ptp4l241ktimersoftd/102:30:041
32725911050,2ptp4l241ktimersoftd/107:41:541
32725911050,1ptp4l241ktimersoftd/104:35:501
32725911040,2ptp4l241ktimersoftd/107:38:471
32725911040,2ptp4l241ktimersoftd/105:32:031
32725911040,1ptp4l241ktimersoftd/106:36:141
32725911040,1ptp4l241ktimersoftd/106:36:141
32725911030,2ptp4l241ktimersoftd/104:06:091
32725911020,1ptp4l241ktimersoftd/105:47:531
2336221010,5sleep023049-21apt-get04:30:510
3272591990,2ptp4l241ktimersoftd/106:30:481
3272591990,1ptp4l241ktimersoftd/105:03:271
3272591980,1ptp4l241ktimersoftd/104:22:321
3272591940,1ptp4l241ktimersoftd/106:42:061
3272591920,2ptp4l241ktimersoftd/105:27:061
3270691900,8getstats395299cyclictest05:00:321
3272591880,1ptp4l241ktimersoftd/102:56:121
94072860,5sleep19405-21ssh07:31:351
3272591850,2ptp4l241ktimersoftd/105:37:231
3272591850,1ptp4l241ktimersoftd/106:05:021
3272591850,1ptp4l241ktimersoftd/106:05:021
190792850,3sleep00-21swapper/005:05:420
3272591810,2ptp4l241ktimersoftd/107:19:551
3272591810,2ptp4l241ktimersoftd/107:13:231
3272591810,2ptp4l13290-21kworker/1:106:15:421
3272591810,1ptp4l241ktimersoftd/107:21:211
3272591790,2ptp4l241ktimersoftd/105:06:181
3272591790,1ptp4l241ktimersoftd/104:51:071
3272591760,1ptp4l241ktimersoftd/104:04:491
3272591740,3ptp4l25437-21kworker/1:204:49:591
3272591730,3ptp4l6913-21kworker/1:207:51:201
3272591690,15ptp4l241ktimersoftd/106:19:481
3272591650,16ptp4l241ktimersoftd/107:02:551
438291590,8getstats395299cyclictest03:30:291
3272591570,2ptp4l25437-21kworker/1:204:11:311
3272591570,2ptp4l25437-21kworker/1:204:11:311
3272591560,3ptp4l25437-21kworker/1:202:54:371
3272591560,2ptp4l13290-21kworker/1:106:46:341
3272591550,1ptp4l241ktimersoftd/106:32:521
3272591540,1ptp4l241ktimersoftd/103:51:141
34742534,36sleep00-21swapper/002:26:370
3272591510,1ptp4l241ktimersoftd/107:46:311
3272591510,1ptp4l241ktimersoftd/106:08:081
3272591500,1ptp4l241ktimersoftd/106:52:551
3272591500,1ptp4l241ktimersoftd/105:44:081
3272591500,1ptp4l241ktimersoftd/105:18:121
3272591490,1ptp4l241ktimersoftd/107:57:281
3272591490,1ptp4l241ktimersoftd/105:53:051
3272591490,1ptp4l241ktimersoftd/105:14:471
3272591490,1ptp4l241ktimersoftd/104:42:071
3272591490,1ptp4l241ktimersoftd/104:00:031
3272591480,1ptp4l241ktimersoftd/107:00:071
3272591480,1ptp4l241ktimersoftd/104:31:451
3272591480,1ptp4l241ktimersoftd/103:11:391
3272591480,1ptp4l241ktimersoftd/103:00:521
3272591470,1ptp4l241ktimersoftd/104:30:431
3272591470,1ptp4l241ktimersoftd/102:41:201
3272591460,1ptp4l241ktimersoftd/103:47:141
3946994521,7cyclictest9-21ksoftirqd/004:12:010
3946994521,7cyclictest9-21ksoftirqd/004:12:010
3272591440,1ptp4l241ktimersoftd/102:36:071
3272591430,1ptp4l241ktimersoftd/104:20:451
3272591430,1ptp4l241ktimersoftd/103:23:511
3272591430,16ptp4l241ktimersoftd/103:44:151
3946994221,6cyclictest9-21ksoftirqd/006:57:090
3946994220,6cyclictest9-21ksoftirqd/004:47:260
3272591420,1ptp4l241ktimersoftd/103:36:041
3272591420,1ptp4l241ktimersoftd/103:16:541
3272591410,1ptp4l241ktimersoftd/102:35:011
3946994016,8cyclictest3881-21apt-get02:30:480
3272591380,1ptp4l241ktimersoftd/103:33:431
3272591370,1ptp4l241ktimersoftd/103:07:031
3946993614,6cyclictest17268-21ssh06:21:340
3946993613,6cyclictest9-21ksoftirqd/006:33:410
3946993527,5cyclictest9-21ksoftirqd/007:49:170
3946993522,9cyclictest5497-21sh06:06:540
3946993522,7cyclictest16485-21gltestperf07:01:140
3946993511,7cyclictest9-21ksoftirqd/006:45:170
394699349,9cyclictest6885-21snmp_o5powerctl02:35:540
3946993422,7cyclictest7735-21grep04:51:080
3946993411,7cyclictest9-21ksoftirqd/007:21:030
3946993411,7cyclictest9-21ksoftirqd/006:19:060
394699338,9cyclictest3142-21ssh06:05:260
394699338,9cyclictest3142-21ssh06:05:260
394699338,6cyclictest9-21ksoftirqd/007:38:410
394699334,22cyclictest29487-21snmp_o5powerctl06:36:290
394699334,22cyclictest29487-21snmp_o5powerctl06:36:290
3946993322,6cyclictest9-21ksoftirqd/003:00:140
3946993321,7cyclictest9514-21grep03:56:190
3946993313,15cyclictest21150-21phc2sys06:26:230
3946993310,5cyclictest121rcu_preempt07:10:500
3946993310,17cyclictest2166-21ssh04:44:170
27142330,6sleep09-21ksoftirqd/003:41:140
394699329,8cyclictest31471-21rm05:20:380
394699327,8cyclictest9-21ksoftirqd/004:03:490
3946993219,7cyclictest30658-21gltestperf03:31:060
3946993211,5cyclictest0-21swapper/007:11:550
3946993210,5cyclictest0-21swapper/005:41:490
138612321,4sleep013863-21snmp_o5powerctl04:06:200
394699319,6cyclictest0-21swapper/007:30:540
394699315,4cyclictest121rcu_preempt05:14:030
3946993121,6cyclictest9-21ksoftirqd/007:45:260
3946993121,6cyclictest9-21ksoftirqd/003:20:130
3946993120,6cyclictest9-21ksoftirqd/003:46:510
3946993120,6cyclictest5285-21grep05:26:320
3946993119,7cyclictest8332-21tail06:11:100
3946993118,8cyclictest19630-21grep04:21:110
394699305,18cyclictest11345-21apt-get04:56:030
394699302,23cyclictest3993-21ls06:46:100
394699302,22cyclictest30385-21sh05:58:030
3946993020,6cyclictest9-21ksoftirqd/003:38:120
3946993020,6cyclictest9-21ksoftirqd/002:50:180
3946993020,5cyclictest9-21ksoftirqd/002:53:270
3946993020,5cyclictest20715-21grep05:06:300
3946993019,6cyclictest9-21ksoftirqd/003:21:500
3946993019,6cyclictest9-21ksoftirqd/003:07:530
3946993019,6cyclictest9-21ksoftirqd/003:00:560
3946993010,15cyclictest7204-21needreboot03:51:120
394699297,13cyclictest7724-21apt-get05:31:060
3946992920,5cyclictest9-21ksoftirqd/002:43:540
3946992919,6cyclictest9-21ksoftirqd/004:18:450
3946992919,6cyclictest31062-21sh04:40:230
3946992919,5cyclictest25047-21processes05:51:190
3946992917,7cyclictest6822-21file07:29:050
3946992917,7cyclictest21566-21gltestperf04:26:070
3946992916,8cyclictest30652-21sh07:17:490
394699291,23cyclictest27056-21sh07:55:370
394699291,23cyclictest197-21systemd-journal07:57:240
3946992819,5cyclictest9-21ksoftirqd/006:51:230
3946992817,6cyclictest9-21ksoftirqd/003:28:150
3946992817,6cyclictest23050-21diskmemload05:23:010
3946992817,6cyclictest12516-21cut05:36:130
394699281,23cyclictest21740-21ssh05:47:100
394699272,20cyclictest21767-21/usr/sbin/munin03:11:050
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional