You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-03-24 - 20:25

ARM Freescale i.MX6 Quad @996 MHz, Linux 5.16.2-rt19 (Profile)

Latency plot of system in rack #b, slot #7
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rackbslot7 (updated Fri Mar 24, 2023 12:44:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1614923430,13sleep316077-21latency_hist07:03:113
162239910543,21cyclictest28671-21ssh11:53:220
16223999846,13cyclictest7483-21apt-get11:18:230
127182920,8sleep112725-21latency09:38:401
127182920,8sleep112725-21latency09:38:401
16223999035,30cyclictest0-21swapper/008:12:350
16223999015,19cyclictest13622-21cstates11:28:290
16229998536,43cyclictest18584-21latency_hist10:43:091
1622399849,43cyclictest0-21swapper/007:23:190
16229998330,45cyclictest0-21swapper/108:48:441
16235998219,56cyclictest19362-21grep07:08:452
16223998241,28cyclictest23128-21diskmemload11:33:360
1623599813,48cyclictest6273-21apt-get09:28:332
16229998125,11cyclictest0-21swapper/107:33:301
16229998125,11cyclictest0-21swapper/107:33:301
16229998122,51cyclictest0-21swapper/108:18:411
16241998051,8cyclictest141rcu_preempt10:43:403
16241998051,8cyclictest141rcu_preempt10:43:403
16235998026,46cyclictest0-21swapper/208:18:112
1622999804,20cyclictest940-21grep07:53:311
1622999803,22cyclictest12401-21sed12:18:491
1622999803,20cyclictest16186-21tail08:38:491
1622999803,20cyclictest12222-21grep08:28:321
1622999803,20cyclictest12222-21grep08:28:321
16223998021,19cyclictest1-21systemd11:08:080
1623599793,18cyclictest22389-21latency_hist11:43:102
1622999794,21cyclictest3647-21sh08:03:081
159302795,52sleep00-21swapper/007:01:300
1622999784,22cyclictest16281-21grep09:43:521
1622999784,18cyclictest17732-21unixbench_multi11:34:021
1622999784,17cyclictest22388-21latency_hist11:43:101
1622999782,24cyclictest14503-21sort12:23:301
1622999782,24cyclictest14503-21sort12:23:301
16229997812,12cyclictest0-21swapper/108:33:301
16223997816,45cyclictest0-21swapper/009:28:190
16241997758,13cyclictest2481-21fschecks_count12:03:303
16241997735,11cyclictest141rcu_preempt11:49:133
1623599774,19cyclictest4761-21fschecks_time11:13:302
1623599774,19cyclictest4761-21fschecks_time11:13:302
1623599773,21cyclictest23126-21idleruntime-cro09:03:072
1623599773,19cyclictest15516-21fschecks_count09:43:382
16235997731,39cyclictest11260-21timerwakeupswit08:23:472
1623599772,19cyclictest9315-21tail12:13:442
1623599772,19cyclictest9315-21tail12:13:442
1622999773,20cyclictest31353-21munin-run11:58:071
16241997650,19cyclictest6910-21idleruntime-cro08:13:083
16241997645,18cyclictest141rcu_preempt10:11:363
16235997631,38cyclictest0-21swapper/209:38:482
16235997631,38cyclictest0-21swapper/209:38:482
16235997622,46cyclictest0-21swapper/211:58:452
1623599762,18cyclictest27046-21latency_hist10:03:102
16235997619,11cyclictest0-21swapper/209:09:142
16235997615,53cyclictest0-21swapper/207:48:312
1622999769,18cyclictest4392sleep110:13:071
1622999764,23cyclictest28464-21fschecks_count07:38:351
1622999764,17cyclictest16504-21latency_hist11:33:111
1622999763,20cyclictest23123-21latency_hist09:03:071
1622999763,20cyclictest21097-21ssh09:53:111
1622999763,20cyclictest21097-21ssh09:53:111
16229997628,42cyclictest0-21swapper/109:13:441
16223997624,20cyclictest18832-21/usr/sbin/munin10:43:260
16223997624,20cyclictest18832-21/usr/sbin/munin10:43:260
16223997621,26cyclictest141rcu_preempt12:03:070
16223997619,24cyclictest15669-21ssh10:38:100
16223997619,24cyclictest15669-21ssh10:38:100
16241997545,20cyclictest0-21swapper/309:58:303
1623599754,16cyclictest16330-21cat07:03:122
1623599753,20cyclictest8966-21fschecks_count08:18:292
1623599753,20cyclictest26847-21fschecks_count07:33:282
1623599753,20cyclictest26847-21fschecks_count07:33:282
16235997531,36cyclictest0-21swapper/209:13:262
16235997524,43cyclictest0-21swapper/211:28:462
16235997523,8cyclictest0-21swapper/210:43:432
16235997523,8cyclictest0-21swapper/210:43:432
16235997523,45cyclictest0-21swapper/208:33:082
16235997523,45cyclictest0-21swapper/208:33:082
1623599752,18cyclictest2212-21/usr/sbin/munin12:03:442
1622999754,19cyclictest21502-21fschecks_count09:53:341
1622999754,19cyclictest18919-21ntp_kernel_err07:08:371
1622999754,17cyclictest2501-21fschecks_time12:03:301
1622999753,20cyclictest27510-21fschecks_count10:03:351
1622999753,19cyclictest21080-21grep08:53:471
1622999753,19cyclictest21080-21grep08:53:471
1622999753,17cyclictest25869-21grep10:53:441
16229997531,36cyclictest0-21swapper/109:18:521
16229997526,44cyclictest0-21swapper/111:08:071
1622999752,21cyclictest5479-21grep11:13:431
1622999752,21cyclictest5479-21grep11:13:431
16223997529,25cyclictest379-21systemd-logind09:18:090
16223997524,31cyclictest20091-21/usr/sbin/munin07:13:220
16241997437,11cyclictest141rcu_preempt09:28:493
16241997416,23cyclictest41-21ksoftirqd/311:03:443
1623599744,17cyclictest397-21runrttasks09:23:422
1623599744,15cyclictest7890-21head10:23:452
1623599743,24cyclictest17570-21tune2fs12:28:302
1623599743,22cyclictest28467-21fschecks_count07:38:352
1623599743,20cyclictest29906-21modprobe10:08:072
1623599743,18cyclictest670-21latency09:18:422
1623599743,17cyclictest4814-21tail08:03:482
1623599743,15cyclictest31887-21sed11:03:452
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional