You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-09-30 - 11:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack4slot7s.osadl.org (updated Sat Sep 30, 2023 00:43:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
259002482401,58sleep00-21swapper/019:05:080
270102463397,52sleep10-21swapper/119:08:181
258482690,1chrt25847-21wc20:29:370
200462590,1sleep10-21swapper/120:14:361
205842550,1sleep10-21swapper/120:15:021
235092480,1sleep00-21swapper/023:10:030
28942231,19sleep12914-21awk19:29:371
2730799221,16cyclictest26159-21apt-get20:29:570
27308992113,6cyclictest34750irq/90-enp1s0-r23:45:501
27308992112,7cyclictest34850irq/91-enp1s0-r21:26:341
2730899211,18cyclictest2132-21awk20:50:071
2730799214,14cyclictest12578-21/usr/sbin/munin19:54:550
27307992113,6cyclictest1-21systemd23:00:060
27307992112,6cyclictest4425-21ntpq23:45:080
2730799211,15cyclictest24610-21apt-get23:14:560
2730899202,16cyclictest3067-21/usr/sbin/munin19:30:051
2730899202,14cyclictest15050-21apt-get22:49:551
27308992013,5cyclictest32052-21/usr/sbin/munin20:45:011
27308992013,5cyclictest31187-21ls22:05:101
27308992012,6cyclictest5471-21grep19:35:021
27308992012,6cyclictest23199-21apt-get00:34:541
27308992012,6cyclictest22829-21snmpd21:45:151
27308992012,6cyclictest18099-21seq21:30:201
27308992012,6cyclictest1560-21sed23:39:371
27308992011,7cyclictest22829-21snmpd22:47:351
27308992011,7cyclictest22829-21snmpd22:35:131
27308992011,7cyclictest22829-21snmpd22:13:311
27308992011,7cyclictest22829-21snmpd21:52:181
27308992011,7cyclictest22829-21snmpd21:00:251
27308992011,7cyclictest22829-21snmpd19:47:351
2730899200,18cyclictest13301-21ls21:19:581
2730799204,14cyclictest882-21/usr/sbin/munin22:10:140
2730799204,14cyclictest27447-21aten2_r4power_p19:09:590
2730799203,11cyclictest31741-21taskset19:19:590
2730799202,14cyclictest7424-21apt-get23:54:570
27307992012,6cyclictest1507-21seq22:14:580
27307992011,7cyclictest9330-21cat19:45:030
27307992011,7cyclictest22829-21snmpd20:58:490
27307992011,7cyclictest22829-21snmpd00:03:310
2730899197,3cyclictest26575-21aten2_r4power_e23:19:581
2730899195,3cyclictest34950irq/92-enp1s0-t20:09:141
2730899193,14cyclictest17786-21ntp_states22:55:071
27308991914,3cyclictest6524-21grep22:25:131
27308991912,5cyclictest4651-21seq20:56:291
27308991911,6cyclictest8199-21ntpq23:55:091
27308991911,6cyclictest34750irq/90-enp1s0-r00:05:541
27308991911,6cyclictest3304-21seq23:43:201
27308991911,6cyclictest29524-21seq20:37:021
27308991911,6cyclictest28721-21seq19:10:561
27308991911,6cyclictest27892-21seq21:58:161
27308991911,6cyclictest27565-21seq20:31:231
27308991911,6cyclictest2301-21cat22:15:081
27308991911,6cyclictest22829-21snmpd20:25:061
27308991911,6cyclictest22829-21snmpd19:41:171
27308991911,6cyclictest15943-21seq20:01:341
27308991911,6cyclictest10228-21ls21:10:101
2730899191,14cyclictest13354-21runrttasks00:09:561
2730899191,12cyclictest7039-21apt-get22:29:421
27308991910,7cyclictest22829-21snmpd21:38:291
27308991910,7cyclictest22795-21cut23:09:541
2730799196,11cyclictest12259-21ntp_states00:05:090
2730799195,9cyclictest17759-21ntp_states22:55:060
2730799193,12cyclictest22829-21snmpd21:19:560
2730799192,15cyclictest11102-21fschecks_count19:50:040
27307991913,4cyclictest5870-21sort23:50:010
27307991911,6cyclictest22829-21snmpd22:38:070
27307991911,6cyclictest22829-21snmpd22:24:280
27307991911,6cyclictest22829-21snmpd21:36:230
27307991911,6cyclictest22829-21snmpd21:06:550
27307991911,6cyclictest22829-21snmpd20:47:430
27307991911,6cyclictest22829-21snmpd19:17:190
27307991911,6cyclictest22829-21snmpd00:32:020
27307991911,6cyclictest22829-21snmpd00:22:430
27307991910,7cyclictest22829-21snmpd22:44:340
27307991910,7cyclictest22829-21snmpd19:59:410
27307991910,7cyclictest19893-21taskset20:13:110
27307991910,7cyclictest15456-21cat21:25:040
27307991910,7cyclictest12208-21kworker/u4:200:14:470
2730899189,7cyclictest22829-21snmpd21:14:411
2730899189,6cyclictest25159-21if_err_enp1s023:15:051
2730899184,9cyclictest19190-21apt-get00:24:561
2730899182,14cyclictest20812-21cut21:39:571
27308991811,5cyclictest9862-21expr23:59:581
27308991811,5cyclictest8537-21seq21:07:091
27308991811,5cyclictest4639-21awk22:20:141
27308991811,5cyclictest30837-21cat23:30:011
2730899181,15cyclictest29786-21/usr/sbin/munin19:15:111
27308991811,5cyclictest29001-21ntp_kernel_err22:00:081
27308991811,5cyclictest12197-21tr22:40:081
2730899181,14cyclictest30580-21taskset20:40:081
2730899181,13cyclictest21091-21apt-get00:29:571
27308991810,6cyclictest6189-21ls23:50:051
27308991810,6cyclictest29162-21ls23:25:051
27308991810,6cyclictest22829-21snmpd19:51:381
27308991810,6cyclictest22110-21seq20:19:511
27308991810,6cyclictest21797-21sensors_temp23:05:111
27308991810,6cyclictest19578-21ls23:00:051
27308991810,6cyclictest18684-21seq00:21:161
27308991810,6cyclictest17055-21seq00:19:251
27308991810,6cyclictest13269-21load19:55:041
2730799189,7cyclictest29028-21ntpq22:00:080
2730799189,7cyclictest22829-21snmpd19:37:590
2730799189,7cyclictest21578-21awk23:05:080
2730799189,6cyclictest20616-21seq21:39:550
2730799184,12cyclictest23557-21fschecks_count00:34:590
2730799184,12cyclictest19162-21/usr/sbin/munin00:25:040
2730799183,8cyclictest1345-21apt-get20:49:580
27307991813,3cyclictest31573-21grep23:30:110
27307991812,4cyclictest31119-21timerwakeupswit20:40:150
27307991812,4cyclictest28838-21ntpq20:35:080
27307991811,5cyclictest9552-21sort21:10:000
27307991811,5cyclictest22829-21snmpd20:21:060
2730799181,15cyclictest1878-21cat19:25:050
27307991810,6cyclictest22829-21snmpd23:44:270
27307991810,6cyclictest22829-21snmpd22:05:330
27307991810,6cyclictest22829-21snmpd19:31:410
27307991810,6cyclictest22829-21snmpd00:13:520
27308991710,5cyclictest22829-21snmpd19:21:341
2730799179,6cyclictest5645-21seq22:25:000
2730799177,7cyclictest15043-21cat22:49:530
2730799174,9cyclictest6914-21apt-get19:39:560
2730799174,9cyclictest16423-21apt-get20:04:550
2730799174,8cyclictest8773-21diskmemload21:14:560
2730799173,9cyclictest5165-21apt-get20:59:570
2730799173,8cyclictest26311-21apt-get21:54:560
2730799172,11cyclictest8773-21diskmemload23:34:580
2730799172,11cyclictest26536-21apt-get23:19:570
27307991712,3cyclictest25670-21grep21:50:110
27307991712,3cyclictest22829-21snmpd22:45:430
27307991711,4cyclictest20959-21grep20:15:090
27307991710,5cyclictest23030-21cat21:45:020
2730799164,7cyclictest8773-21diskmemload22:29:540
2730799162,11cyclictest28438-21apt-get23:24:570
27307991611,3cyclictest22829-21snmpd21:32:380
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional