You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-07-06 - 08:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

x86 Intel Pentium N4200 @1100 MHz, Linux 5.9.1-rt18 (Profile)

Latency plot of shadow in rack #c, slot #4
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot4s.osadl.org (updated Wed Jul 06, 2022 00:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
322172318271,34sleep30-21swapper/319:06:013
324952271203,54sleep00-21swapper/019:08:280
307322262214,35sleep20-21swapper/219:05:032
322232245197,35sleep10-21swapper/119:06:021
226022780,1chrt22556-21apt-get21:10:132
281212510,2sleep30-21swapper/321:20:233
165982500,3sleep20-21swapper/222:10:002
46499452,19cyclictest16761-21apt-get22:10:132
46499431,29cyclictest31793-21lsb_release22:45:012
46499410,26cyclictest30648-21chrt21:26:392
87692400,1sleep20-21swapper/223:05:012
46499401,26cyclictest25236-21latency_hist22:30:012
46499399,15cyclictest115152chrt00:23:182
46499392,21cyclictest2672-21chrt00:02:132
46499391,23cyclictest29219-21/usr/sbin/munin23:50:162
46499391,19cyclictest17146-21apt-get19:45:152
46499391,19cyclictest17146-21apt-get19:45:142
46499390,24cyclictest11070-1kworker/2:0H22:25:012
46499390,16cyclictest8500-21apt-get20:40:102
282702390,1chrt28271-21/usr/sbin/munin20:10:152
46499385,18cyclictest92542chrt00:17:422
46499380,21cyclictest11784-21chrt19:34:492
46499375,18cyclictest220042chrt23:32:412
46499373,20cyclictest50142chrt19:17:462
46499371,21cyclictest25866-21chrt20:05:122
46499371,21cyclictest17409-21pidof23:20:302
46499370,22cyclictest769-21chrt21:34:512
46499370,22cyclictest2776-21chrt19:12:152
46499370,22cyclictest16231-21chrt19:42:172
46499370,21cyclictest28707-21chrt22:35:232
46499370,21cyclictest129032chrt20:47:112
46499369,13cyclictest7343-21latency_hist00:15:002
46499361,15cyclictest26932-21apt-get21:20:122
46499360,22cyclictest3736-21cut22:54:572
46499360,21cyclictest19421-21chrt21:01:562
46499360,21cyclictest1224-21chrt22:47:022
46499360,20cyclictest27094-21chrt22:32:212
46499360,20cyclictest27094-21chrt22:32:212
46499350,32cyclictest5478-21apt-config21:45:012
46499350,20cyclictest39362chrt20:28:132
46499320,17cyclictest10141-21grep23:05:272
46499320,13cyclictest6578-21apt-get20:35:132
46499310,27cyclictest24734-21apt-get21:15:132
46499310,15cyclictest15654-21apt-get20:55:122
46499310,13cyclictest8039-21apt-get21:50:132
278862300,2sleep10-21swapper/120:10:001
46999299,6cyclictest31-21ksoftirqd/322:10:133
46499292,21cyclictest32422-21apt-get20:20:132
46499292,21cyclictest23686-21apt-get20:00:122
46499290,15cyclictest0-21swapper/223:35:192
46499290,14cyclictest1887-21snmpd20:32:232
46499282,21cyclictest20382-21apt-get21:05:152
46499282,20cyclictest10211-21apt-get21:55:122
46499281,23cyclictest32030-21ntpq23:55:192
46499281,23cyclictest20906-21awk22:20:002
46499281,23cyclictest15439-21latency_hist20:55:002
46499281,23cyclictest14083-21latency_hist19:40:002
46499281,21cyclictest14502-21apt-get00:30:162
46499280,1cyclictest0-21swapper/222:55:172
46499280,1cyclictest0-21swapper/219:29:592
46499273,18cyclictest12408-21apt-get22:00:132
46499272,20cyclictest3245-21apt-get00:05:122
46499272,20cyclictest19322-21apt-get19:50:132
46499272,19cyclictest26906-21apt-esm-hook23:45:132
46499271,24cyclictest19501-21awk23:25:262
46499271,22cyclictest24720-21runrttasks23:40:112
46499271,21cyclictest15416-21awk23:20:012
46499271,20cyclictest0-21swapper/221:35:122
46499271,19cyclictest30208-21apt-get20:15:122
46999261,19cyclictest1887-21snmpd22:55:123
46499263,17cyclictest5561-21apt-esm-hook19:20:122
46499262,19cyclictest21496-21apt-get19:55:132
46499260,1cyclictest0-21swapper/221:45:132
46999259,5cyclictest111rcu_preempt23:35:193
46499253,17cyclictest11965-21apt-get00:25:122
464992514,9cyclictest16682-21/usr/sbin/munin00:35:202
46499250,15cyclictest0-21swapper/223:12:082
45899251,19cyclictest16761-21apt-get22:10:120
57742240,2sleep30-21swapper/319:20:133
46999234,8cyclictest31-21ksoftirqd/323:05:003
46999233,6cyclictest0-21swapper/319:25:133
469992312,7cyclictest27670-21runrttasks22:35:133
469992312,2cyclictest111rcu_preempt23:05:493
469992311,2cyclictest111rcu_preempt23:58:403
469992311,2cyclictest111rcu_preempt23:24:083
469992310,2cyclictest111rcu_preempt21:11:183
469992310,2cyclictest111rcu_preempt00:16:153
46299230,5cyclictest0-21swapper/121:23:191
189802230,8sleep119-21ksoftirqd/123:25:191
469992211,2cyclictest111rcu_preempt21:30:523
469992210,2cyclictest111rcu_preempt21:26:053
469992210,2cyclictest111rcu_preempt20:11:343
469992210,2cyclictest111rcu_preempt20:01:573
46299226,7cyclictest11266-21snmp_o5powerctl21:55:221
462992215,5cyclictest8552-21http20:40:001
46299220,5cyclictest0-21swapper/122:09:461
46299220,4cyclictest0-21swapper/121:17:271
46299220,4cyclictest0-21swapper/120:20:541
45899222,14cyclictest5832-40apt-check21:45:100
458992211,7cyclictest9-21ksoftirqd/022:30:170
458992211,7cyclictest9-21ksoftirqd/022:30:170
45899220,4cyclictest0-21swapper/021:55:220
16572220,2chrt0-21swapper/319:10:163
46999219,2cyclictest111rcu_preempt23:32:283
46999219,2cyclictest111rcu_preempt22:52:493
46999219,2cyclictest111rcu_preempt22:17:123
46999219,2cyclictest111rcu_preempt20:58:543
46999219,2cyclictest111rcu_preempt19:31:393
46999219,1cyclictest111rcu_preempt22:23:403
46999218,2cyclictest111rcu_preempt23:28:063
46999218,2cyclictest111rcu_preempt23:14:423
46999214,9cyclictest31-21ksoftirqd/300:09:473
46999212,14cyclictest8039-21apt-get21:50:133
469992110,2cyclictest111rcu_preempt20:45:383
469992110,2cyclictest111rcu_preempt00:12:243
469992110,1cyclictest111rcu_preempt20:50:093
462992114,5cyclictest7943-21users23:00:251
462992114,5cyclictest3608-21users20:25:251
46299210,5cyclictest0-21swapper/100:09:321
46299210,4cyclictest0-21swapper/100:15:311
45899219,7cyclictest17851-21apt-get21:00:120
45899212,13cyclictest9889-21apt-get19:30:130
45899212,13cyclictest6578-21apt-get20:35:130
45899212,13cyclictest15998-21apt-get23:20:130
458992113,6cyclictest1874-21sed19:10:180
458992112,4cyclictest1124-21runrttasks23:25:230
46999209,2cyclictest111rcu_preempt23:45:563
46999209,2cyclictest111rcu_preempt21:35:273
46999209,2cyclictest111rcu_preempt19:41:373
46999209,2cyclictest111rcu_preempt00:21:213
46999208,2cyclictest111rcu_preempt22:46:263
46999208,2cyclictest111rcu_preempt19:48:173
46999208,2cyclictest111rcu_preempt19:48:173
46999201,9cyclictest17744-21snmp_o5powerctl00:35:243
469992013,5cyclictest25454-21cut22:30:013
469992013,5cyclictest19852-21cat19:50:153
46999201,14cyclictest20394-21apt-get21:05:163
46299202,12cyclictest28036-21apt-get20:10:131
462992014,4cyclictest14437-21date22:05:001
462992013,5cyclictest8845-21ntpq21:50:201
462992013,5cyclictest8046-21tune2fs19:25:161
462992013,5cyclictest669-21ls19:10:001
462992013,5cyclictest6607-21snmp_o5powerctl19:20:251
462992013,5cyclictest4620-21forks22:55:181
462992013,5cyclictest21240-21pluginstate21:05:241
462992013,5cyclictest12244-21tune2fs00:25:141
462992012,6cyclictest18641-21lsb_release00:40:001
46299201,16cyclictest20667-21sed19:50:251
46299200,2cyclictest111rcu_preempt19:42:161
45899209,7cyclictest9-21ksoftirqd/000:15:130
45899202,14cyclictest20391-21apt-get21:05:140
45899202,14cyclictest18667-21latency_hist00:40:000
45899202,13cyclictest9100-40apt-check20:40:090
45899202,13cyclictest3380-21apt-esm-hook19:15:140
45899202,13cyclictest29851-21apt-get22:40:120
45899202,13cyclictest22573-21apt-esm-hook21:10:130
45899202,13cyclictest20351-21apt-get23:30:120
458992013,5cyclictest14147-21sed23:15:170
458992011,6cyclictest9-21ksoftirqd/019:40:060
45899200,18cyclictest13893-21cron00:30:000
147962200,2sleep30-21swapper/322:05:143
469991912,5cyclictest15002-21cat00:30:213
469991912,5cyclictest12968-21basename00:25:203
469991911,6cyclictest12581-21awk19:35:163
469991911,5cyclictest2255-21apt-get20:25:133
46999191,13cyclictest1887-21snmpd19:15:133
462991915,2cyclictest5973-21kernelversion00:10:161
462991915,2cyclictest23206-21missed_timers23:35:181
462991913,4cyclictest16346-21tune2fs23:20:181
462991912,5cyclictest27260-21fw_packets23:45:141
462991912,5cyclictest2630-21ntpq22:50:181
462991912,5cyclictest24815-21sed20:00:231
462991912,5cyclictest23347-21sshd19:59:271
462991912,5cyclictest22186-21snmp_o5powerctl22:20:241
462991912,5cyclictest1825-21netstat21:35:171
462991912,5cyclictest15176-21ls00:30:221
462991911,5cyclictest17636-21grep21:00:001
462991911,5cyclictest17496-21perl19:45:221
462991911,5cyclictest17496-21perl19:45:221
45899192,13cyclictest4895-21apt-get21:42:120
45899192,13cyclictest4376-21apt-esm-hook22:55:140
45899192,12cyclictest8158-21kworker/u8:123:05:050
45899192,12cyclictest4397-21apt-get20:30:120
45899192,12cyclictest26946-21apt-esm-hook21:20:120
45899192,12cyclictest21169-21apt-esm-hook22:20:120
458991913,3cyclictest16514-21idleruntime-cro22:10:000
458991912,5cyclictest2253-21sed00:00:250
458991912,5cyclictest20359-21grep19:50:200
458991912,5cyclictest14473-40apt-check00:30:090
458991912,5cyclictest11944-21sed20:45:170
458991911,5cyclictest5194-21idleruntime-cro00:10:000
45899191,13cyclictest8053-21apt-esm-hook21:50:130
45899191,13cyclictest31298-21apt-esm-hook21:30:130
45899191,13cyclictest27667-21apt-esm-hook22:35:120
45899191,13cyclictest24748-21apt-esm-hook21:15:130
45899191,13cyclictest23305-21apt-get22:25:120
45899191,13cyclictest14058-21latency_hist19:40:000
45899191,12cyclictest32022-21apt-get22:45:120
45899191,12cyclictest18930-21apt-get22:15:130
45899191,12cyclictest13516-21apt-esm-hook20:50:130
45899190,5cyclictest0-21swapper/023:00:260
41992190,1sleep10-21swapper/119:15:211
46999189,2cyclictest111rcu_preempt20:19:353
469991812,5cyclictest6363-21basename21:45:203
469991812,4cyclictest26022-21expr20:05:133
469991812,4cyclictest21657-21expr19:55:123
469991811,5cyclictest18977-21cat21:00:233
469991811,5cyclictest12571-21expr22:00:143
46999181,13cyclictest30134-21snmp_o5powerctl23:50:273
469991810,2cyclictest111rcu_preempt23:17:523
462991814,2cyclictest790-21timerandwakeup22:45:281
462991814,2cyclictest24106-21ntp_offset22:25:191
462991814,2cyclictest13537-21cpuspeed_turbos23:15:131
462991813,3cyclictest3938-21missed_timers21:40:171
462991811,5cyclictest9933-21expr00:20:131
462991811,5cyclictest26655-21basename23:45:001
462991811,5cyclictest19989-21snmp_o5powerctl22:15:261
462991811,5cyclictest12632-21unixbench_singl20:45:241
462991811,5cyclictest1015-21date00:00:001
462991811,4cyclictest301-21systemd-journal20:55:001
46299181,12cyclictest9093-21apt-get23:05:171
46299181,12cyclictest11283-21apt-esm-hook23:10:121
45899182,12cyclictest31277-21apt-esm-hook23:55:120
45899182,11cyclictest32456-21apt-esm-hook20:20:130
45899182,11cyclictest27428-21runrttasks20:06:540
458991814,2cyclictest23078-21kernelversion23:35:180
458991811,5cyclictest16969-21grep20:55:240
45899181,13cyclictest5544-21apt-get19:20:130
45899181,13cyclictest30245-21apt-esm-hook20:15:120
45899181,13cyclictest22908-21runrttasks19:55:310
45899181,12cyclictest9828-21apt-esm-hook00:20:120
45899181,12cyclictest7140-21runrttasks00:13:180
45899181,12cyclictest28036-21apt-get20:10:120
45899181,12cyclictest24736-21apt-esm-hook23:40:120
45899181,12cyclictest23703-21apt-esm-hook20:00:110
45899181,12cyclictest1837-21apt-esm-hook22:50:120
45899181,12cyclictest16556-21apt-get19:45:050
45899181,12cyclictest16556-21apt-get19:45:050
45899181,12cyclictest12425-21apt-esm-hook22:00:120
45899181,12cyclictest1062-21apt-get21:35:110
45899181,11cyclictest9482-21runrttasks19:28:400
469991713,2cyclictest11410-21timerwakeupswit21:55:243
469991711,4cyclictest32723-21cut20:20:153
469991711,4cyclictest25380-21expr23:40:173
469991710,5cyclictest7151-21cut20:35:173
469991710,5cyclictest30890-21grep22:40:213
46299179,6cyclictest23214-21cut21:10:221
46299179,5cyclictest19219-21grep21:00:241
46299178,7cyclictest4934-21cat20:30:151
46299172,12cyclictest30306-21head22:40:151
462991713,2cyclictest21607-21unixbench-2d23:30:241
462991713,2cyclictest17342-21latency22:10:171
462991713,2cyclictest10442-21kernelversion19:30:171
462991711,4cyclictest9172-21dpkg20:40:161
462991711,4cyclictest14147-21date19:40:011
46299171,11cyclictest29125-21apt-esm-hook21:25:131
46299171,11cyclictest29067-21apt-get23:50:131
462991710,5cyclictest1706-21memory00:00:181
46299170,13cyclictest5859-21apt-get21:45:111
45899171,13cyclictest29060-21runrttasks23:50:080
45899171,12cyclictest2220-21apt-get20:25:130
45899171,12cyclictest11269-21apt-get23:10:110
458991710,4cyclictest29491-21grep21:25:150
46999169,5cyclictest3490-21expr21:40:123
46999169,5cyclictest1248-21expr00:00:153
46999168,2cyclictest111rcu_preempt21:18:383
46999162,11cyclictest9100-40apt-check20:40:083
469991612,3cyclictest5108-21sed20:30:173
462991612,2cyclictest27744-21cpu22:35:131
462991611,3cyclictest30731-21if_err_enp2s020:15:151
462991611,3cyclictest25513-21sh22:30:181
462991611,3cyclictest25513-21sh22:30:181
45899162,10cyclictest27679-21grep23:45:180
46999151,11cyclictest26548-21snmp_o5powerctl22:30:303
46999151,11cyclictest26548-21snmp_o5powerctl22:30:303
46299151,10cyclictest31317-21apt-esm-hook21:30:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional