You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-09-19 - 20:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems

Intel(R) Atom(TM) CPU E3825 @ 1.33GHz, Linux 4.19.37-rt20 (Profile)

Latency plot of shadow in rack #b, slot #6
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the alll highest latencies:
System rackbslot6s.osadl.org (updated Sun Sep 19, 2021 12:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3157823330,25sleep031576-21fschecks_time11:20:510
858911050,3ptp4l241ktimersoftd/111:22:431
858911010,1ptp4l241ktimersoftd/115:52:591
85891950,1ptp4l241ktimersoftd/113:59:341
85891940,1ptp4l241ktimersoftd/115:11:211
85891900,11ptp4l241ktimersoftd/116:35:411
85891880,1ptp4l241ktimersoftd/114:11:551
107072860,3sleep00-21swapper/016:16:220
85891840,2ptp4l241ktimersoftd/112:41:111
275262840,2sleep00-21swapper/012:26:020
266022840,4sleep00-21swapper/015:58:200
265242790,5sleep126499-21ssh14:45:351
928997726,16cyclictest13346-21sh15:43:450
928997726,15cyclictest9806-21ssh16:52:320
928997725,17cyclictest21379-21ssh15:51:550
85891770,1ptp4l241ktimersoftd/111:45:591
928997625,16cyclictest29202-21ssh13:34:570
928997625,16cyclictest19578-21ssh16:26:210
928997625,15cyclictest29811-21ssh16:38:540
303062760,3sleep10-21swapper/112:31:041
928997525,15cyclictest15601-21ssh15:09:580
928997525,15cyclictest13382-21ssh14:30:350
928997524,15cyclictest7121-21ssh14:22:270
928997524,15cyclictest19443-21timedrift14:36:190
928997522,17cyclictest20481-21ssh14:01:000
85891750,1ptp4l241ktimersoftd/114:47:521
928997427,15cyclictest25039-21ssh14:43:050
928997425,15cyclictest30473-21seq14:11:530
928997424,16cyclictest23380-21ssh15:18:030
928997424,16cyclictest17085-21ssh16:24:490
928997422,17cyclictest30489-21rm14:50:000
928997422,12cyclictest20709-21diskmemload14:06:010
928997420,16cyclictest18858-21rm14:00:040
928997419,16cyclictest7749-21fschecks_count15:00:530
85891740,1ptp4l241ktimersoftd/113:34:181
928997325,17cyclictest7236-21ls16:50:360
928997324,16cyclictest350-21runrttasks16:05:530
928997324,15cyclictest31680-21runrttasks16:04:340
928997323,16cyclictest3670-21rm14:19:110
928997323,15cyclictest31008-21/usr/sbin/munin16:41:070
928997322,17cyclictest6176-21if_enp4s012:50:560
928997322,17cyclictest1455-21cat11:25:500
928997321,17cyclictest5138-21rm15:34:530
928997321,17cyclictest25861-21cut12:20:590
928997321,17cyclictest13652-21rm13:53:400
928997321,16cyclictest22211-21snmpd14:56:020
928997319,20cyclictest10708-21ssh14:26:240
928997225,14cyclictest16539-21fschecks_count15:10:540
928997224,14cyclictest8018-21ssh16:14:250
928997222,16cyclictest3130-21/usr/sbin/munin13:41:030
928997222,16cyclictest15256-21cat11:56:020
928997221,17cyclictest11850-21taskset11:47:570
928997221,16cyclictest22241-21bc16:30:500
85891720,3ptp4l14437-21kworker/1:215:16:311
928997122,17cyclictest2864-21chrt12:42:220
928997122,16cyclictest31890-21ssh15:27:300
928997121,17cyclictest7611-21chrt11:38:140
928997121,17cyclictest21513-21sed12:10:570
928997121,17cyclictest197-21systemd-journal12:37:240
928997121,17cyclictest18784-21bc13:20:460
928997121,17cyclictest15522-21cut15:45:590
928997121,17cyclictest11978-21taskset13:05:010
928997121,16cyclictest6782-21memory15:36:000
928997121,16cyclictest32297-21ssh13:37:330
928997121,16cyclictest24332-21cat12:16:290
928997121,16cyclictest18685-21latency_hist13:20:330
928997121,16cyclictest16534-21/usr/sbin/munin12:00:590
928997029,11cyclictest7653-21snmp_o5powerctl13:46:200
928997022,17cyclictest22211-21snmpd12:58:220
928997022,15cyclictest26154-21ssh15:21:040
928997021,17cyclictest19703-21/usr/sbin/munin12:06:040
928997021,17cyclictest12447-21df13:05:500
928997021,16cyclictest31265-21taskset12:35:120
928997021,16cyclictest1036-21sh14:52:260
928997021,15cyclictest8879-21cat11:41:090
928997020,16cyclictest3076-21munin-node11:30:380
928996921,17cyclictest16365-21chrt13:14:580
928996919,17cyclictest24640-21ssh13:29:380
928996820,16cyclictest3729-21/usr/sbin/munin12:45:590
85891680,3ptp4l31488-21kworker/1:215:39:051
928996721,16cyclictest12836-21if_enp4s011:51:030
85891670,3ptp4l7558-21kworker/1:212:58:121
85891670,1ptp4l241ktimersoftd/114:27:441
85891630,1ptp4l241ktimersoftd/114:32:101
85891610,1ptp4l241ktimersoftd/116:52:011
85891610,1ptp4l241ktimersoftd/115:27:091
85891610,1ptp4l241ktimersoftd/115:06:471
85891600,1ptp4l241ktimersoftd/114:17:301
85891590,1ptp4l241ktimersoftd/116:43:271
85891570,7ptp4l241ktimersoftd/113:47:481
87791560,3getstats231rcuc/111:40:561
85891560,2ptp4l4603-21kworker/1:016:21:101
85891550,3ptp4l9362-21kworker/1:015:05:171
85891550,3ptp4l28305-21kworker/1:013:42:341
85891540,3ptp4l23133-21kworker/1:214:07:071
85891540,1ptp4l241ktimersoftd/116:32:201
85891520,3ptp4l13937-21kworker/1:211:56:041
85891520,1ptp4l241ktimersoftd/113:51:161
85891510,1ptp4l241ktimersoftd/112:26:111
85891500,2ptp4l241ktimersoftd/116:28:511
85891500,1ptp4l241ktimersoftd/114:53:311
85891490,2ptp4l241ktimersoftd/116:15:011
85891490,2ptp4l241ktimersoftd/114:35:531
85891490,1ptp4l241ktimersoftd/116:10:181
85891490,1ptp4l241ktimersoftd/115:32:201
85891490,1ptp4l241ktimersoftd/115:21:241
85891490,1ptp4l241ktimersoftd/114:02:321
85891480,1ptp4l241ktimersoftd/116:48:401
85891480,1ptp4l241ktimersoftd/116:01:491
85891480,1ptp4l241ktimersoftd/115:56:071
85891480,1ptp4l241ktimersoftd/115:47:281
85891480,1ptp4l241ktimersoftd/115:43:331
85891480,1ptp4l241ktimersoftd/114:56:281
85891480,1ptp4l241ktimersoftd/114:21:481
85891480,1ptp4l241ktimersoftd/113:35:351
85891480,1ptp4l241ktimersoftd/113:30:121
85891480,1ptp4l241ktimersoftd/113:20:481
85891470,1ptp4l241ktimersoftd/116:19:251
85891470,1ptp4l241ktimersoftd/112:15:231
85891460,1ptp4l241ktimersoftd/113:07:341
85891460,1ptp4l241ktimersoftd/112:50:511
85891460,1ptp4l241ktimersoftd/111:35:111
85891460,1ptp4l241ktimersoftd/111:26:021
85891440,1ptp4l241ktimersoftd/113:15:001
85891440,1ptp4l241ktimersoftd/112:47:431
85891440,1ptp4l241ktimersoftd/111:51:211
85891430,1ptp4l241ktimersoftd/113:03:311
85891430,1ptp4l241ktimersoftd/112:38:471
85891430,1ptp4l241ktimersoftd/112:21:021
85891430,1ptp4l241ktimersoftd/112:16:511
85891420,1ptp4l241ktimersoftd/113:16:071
85891420,1ptp4l241ktimersoftd/112:02:291
85891420,1ptp4l241ktimersoftd/111:40:201
85891410,1ptp4l241ktimersoftd/112:08:191
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional