You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-03-23 - 06:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

x86 Intel Core i3-6100U @2300 MHz, Linux 5.15.79-rt54 (Profile)

Latency plot of shadow in rack #7, slot #7
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Thu Mar 23, 2023 00:43:58)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
22268210591,9sleep10-21swapper/119:09:441
22265210275,9sleep30-21swapper/319:09:423
2226928264,14sleep20-21swapper/219:09:452
2198728267,9sleep00-21swapper/019:06:050
322752590,0sleep30-21swapper/320:35:243
2238699230,1cyclictest0-21swapper/321:37:193
22382992322,1cyclictest0-21swapper/221:08:062
2238699220,0cyclictest0-21swapper/319:50:163
2238299221,21cyclictest0-21swapper/223:37:062
2237999202,17cyclictest18110-21munin-run22:20:001
22379992013,3cyclictest12587-21ssh22:38:461
2237999201,2cyclictest0-21swapper/122:40:171
2237999200,19cyclictest14908-21diskmemload21:35:161
2237799191,1cyclictest141rcu_preempt22:29:520
2237799190,1cyclictest0-21swapper/023:33:420
2238299180,18cyclictest0-21swapper/223:42:252
2238299180,18cyclictest0-21swapper/221:32:232
2238699170,16cyclictest0-21swapper/300:09:233
2238699163,11cyclictest0-21swapper/323:10:213
2238299161,5cyclictest224-21systemd-journal19:20:002
2238299160,15cyclictest29123-21kworker/u8:223:23:152
22377991614,1cyclictest0-21swapper/023:35:210
107262160,0chrt10639-21/usr/sbin/munin23:00:253
22382991510,2cyclictest8197-21tr21:50:152
22379991513,2cyclictest0-21swapper/122:29:371
2237799157,4cyclictest4551-21apache_processe21:25:170
2237799156,6cyclictest534-21snmpd22:06:200
2237799153,6cyclictest17258-21perf23:05:000
2237799152,7cyclictest0-21swapper/023:45:130
2237799151,7cyclictest12539-21awk21:05:000
22377991512,2cyclictest215832sleep021:59:350
2237799150,7cyclictest0-21swapper/021:30:380
2237799150,11cyclictest0-21swapper/021:35:160
2238699141,4cyclictest0-21swapper/322:00:243
22386991410,2cyclictest4615-21grep21:25:193
22386991410,2cyclictest430-21grep19:30:183
2238699140,13cyclictest7065-21chrt21:49:403
2238299149,3cyclictest18619-21apt00:15:002
2238299143,10cyclictest37850irq/131-snd_hda20:45:222
22382991410,3cyclictest21040-21ssh23:30:032
22382991410,2cyclictest31482-21cat00:00:002
2238299140,1cyclictest0-21swapper/223:06:432
2238299140,14cyclictest0-21swapper/221:55:442
2238299140,13cyclictest0-21swapper/222:32:452
22379991413,1cyclictest0-21swapper/122:20:271
2237799145,1cyclictest26435-21pmu-power23:10:310
2237799143,1cyclictest8419-21ssh22:12:370
22377991411,2cyclictest8336-21grep22:35:310
2238699139,3cyclictest23205-21sed23:30:343
2238699139,3cyclictest22880-21tr19:10:253
2238699133,5cyclictest0-21swapper/322:35:263
22386991311,2cyclictest0-21swapper/300:39:123
22386991310,2cyclictest30800-21sort19:25:263
22386991310,2cyclictest23096-21grep20:15:283
22386991310,2cyclictest19158-21grep00:15:173
22386991310,2cyclictest19158-21grep00:15:163
22386991310,2cyclictest13700-21tail21:30:393
2238699130,2cyclictest0-21swapper/319:15:143
2238699130,1cyclictest0-21swapper/321:19:213
2238299139,3cyclictest6253-21grep00:05:252
2238299139,3cyclictest13914-21cron20:00:002
2238299131,7cyclictest37850irq/131-snd_hda20:50:152
22382991311,1cyclictest0-21swapper/222:49:362
2238299130,1cyclictest0-21swapper/221:29:122
2237999139,3cyclictest5394-21sed21:25:231
2237999138,3cyclictest1919-21cat20:40:221
22379991311,2cyclictest25-21ksoftirqd/123:09:581
22379991310,2cyclictest8408-21grep23:20:271
22379991310,2cyclictest31476-21tr00:25:191
22379991310,2cyclictest23945-21grep20:20:231
22379991310,2cyclictest10180-21systemd-cgroups00:07:581
2237799139,3cyclictest30867-21ls21:20:240
2237799138,3cyclictest6879-21munin-plugin-st22:35:000
2237799134,9cyclictest17603-21apache223:26:120
2237799134,7cyclictest1247-21sed00:00:350
2237799132,6cyclictest0-21swapper/019:50:010
22377991310,2cyclictest26595-21grep19:15:270
22377991310,2cyclictest19396-21sort23:50:370
22377991310,2cyclictest17269-21sed21:10:280
2237799130,12cyclictest8641-21sh21:50:180
2237799130,12cyclictest7716-21sort23:20:220
162692130,1sleep30-21swapper/300:12:113
2238699129,2cyclictest8185-21tr21:50:163
2238699129,2cyclictest7092-21cut19:45:213
2238699129,2cyclictest5372-21tail22:55:323
2238699129,2cyclictest5372-21tail22:55:323
2238699129,2cyclictest4877-21chrt00:30:053
2238699129,2cyclictest32045-21sed00:25:213
2238699129,2cyclictest30175-21ssh23:35:353
2238699129,2cyclictest26922-21grep22:25:263
2238699129,2cyclictest26224-21wc00:20:263
2238699129,2cyclictest2547-21grep20:40:283
2238699129,2cyclictest15684-21tr21:10:153
2238699129,2cyclictest14469-21tr20:00:163
2238699129,2cyclictest13340-21grep22:15:313
2238699129,2cyclictest1313-21grep23:15:323
2238699128,3cyclictest9867-21sleep23:44:593
2238699128,3cyclictest30565-21ls22:50:333
2238699128,3cyclictest22925-21grep22:45:323
2238699128,1cyclictest0-21swapper/323:20:463
2238299129,2cyclictest9928-21df19:50:182
2238299129,2cyclictest9704-21grep20:55:292
2238299129,2cyclictest7412-21ssh21:50:002
2238299129,2cyclictest6729-21cat19:45:012
2238299129,2cyclictest3301-21ls19:35:242
2238299129,2cyclictest29340-21head20:30:252
2238299129,2cyclictest24663-21sleep00:20:002
2238299129,2cyclictest24663-21sleep00:20:002
2238299129,2cyclictest23511-21sh20:20:002
2238299129,2cyclictest14686-21tr22:40:182
2238299128,4cyclictest472-21avahi-daemon21:16:312
2238299128,3cyclictest4302-21grep22:55:232
2238299128,3cyclictest4302-21grep22:55:232
2238299128,3cyclictest30637-21head21:20:222
2238299128,3cyclictest17294-21wc00:40:012
2238299127,3cyclictest18339-21ssh22:20:002
22382991212,0cyclictest0-21swapper/222:02:462
2238299121,1cyclictest11511-21apt-get22:15:172
22382991211,1cyclictest32-21ksoftirqd/200:22:512
2238299120,2cyclictest0-21swapper/222:35:272
2238299120,12cyclictest0-21swapper/223:00:372
2238299120,12cyclictest0-21swapper/221:10:032
2238299120,11cyclictest27144-21/usr/sbin/munin22:25:292
2237999129,2cyclictest7089-21grep19:45:211
2237999129,2cyclictest6405-21head20:50:201
2237999129,2cyclictest4746-21ntp_kernel_err22:55:261
2237999129,2cyclictest4746-21ntp_kernel_err22:55:261
2237999129,2cyclictest4083-21cat21:25:011
2237999129,2cyclictest30227-21grep20:30:341
2237999129,2cyclictest29546-21head22:50:231
2237999129,2cyclictest28400-21ssh22:04:191
2237999129,2cyclictest25276-21df23:55:191
2237999129,2cyclictest23240-21sed23:30:351
2237999129,2cyclictest14676-21wc21:55:001
2237999129,2cyclictest1315-21grep23:15:321
2237999129,2cyclictest12188-21wc23:00:351
2237999129,2cyclictest10191-21head19:50:201
2237999128,3cyclictest790-21ls00:00:311
2237999128,2cyclictest32178-21wc20:35:221
2237999128,2cyclictest26439-21grep23:10:311
2237999128,2cyclictest26274-21grep20:25:161
2237999120,5cyclictest0-21swapper/120:45:211
2237999120,2cyclictest0-21swapper/123:50:471
2237799129,2cyclictest4002-21grep20:45:250
2237799129,2cyclictest30988-21head19:25:280
2237799129,2cyclictest28825-21grep22:50:200
2237799129,2cyclictest28284-21turbostat20:30:000
2237799129,2cyclictest2740-21tail21:45:380
2237799129,2cyclictest23655-21wc22:00:230
2237799129,2cyclictest22873-21grep19:10:250
2237799129,2cyclictest17885-21chrt22:41:550
2237799129,2cyclictest17863-21grep20:05:240
2237799129,2cyclictest12642-21grep00:35:260
2237799129,2cyclictest12589-21grep19:55:180
2237799128,3cyclictest6672-21ls20:50:220
2237799120,11cyclictest19724-21grep20:10:200
2238699119,1cyclictest29041-21tail19:20:283
2238699118,2cyclictest5715-21grep19:40:243
2238699118,2cyclictest4879-21grep20:45:353
2238699118,2cyclictest3196-21grep19:35:223
2238699118,2cyclictest27779-21ssh23:56:343
2238699118,2cyclictest25820-21rm21:40:053
2238699118,2cyclictest20652-21ssh22:44:343
2238699118,2cyclictest19726-21grep20:10:203
2238699118,2cyclictest18886-21ssh21:56:593
2238699118,2cyclictest18262-21grep20:05:293
2238699118,2cyclictest18229-21ssh23:50:263
2238699118,2cyclictest18054-21egrep23:05:213
2238699118,2cyclictest15920-21grep23:25:313
2238699118,2cyclictest10661-21grep21:00:243
2238699118,2cyclictest10623-21cut23:45:163
2238699118,2cyclictest10372-21ssh22:14:343
2238699117,3cyclictest5978-21ssh22:34:053
2238699110,10cyclictest14493-21sed21:05:313
2238299118,2cyclictest981-21expr23:15:282
2238299118,2cyclictest8407-21sh22:12:362
2238299118,2cyclictest3526-21sh22:09:472
2238299118,2cyclictest31639-21chrt19:27:152
2238299118,2cyclictest30777-21ssh22:50:362
2238299118,2cyclictest28097-21grep19:20:182
2238299118,2cyclictest26250-21wc23:10:282
2238299118,2cyclictest25932-21ssh21:40:122
2238299118,2cyclictest24869-21wc19:15:002
2238299118,2cyclictest24405-21grep20:20:252
2238299118,2cyclictest17372-21head20:05:202
2238299118,2cyclictest14750-21cut23:25:212
2238299118,2cyclictest11186-21head23:45:192
2238299118,2cyclictest11017-21sort21:00:262
2238299118,2cyclictest10602-21ssh00:34:462
2238299116,4cyclictest16307-21cron20:05:002
2238299115,2cyclictest224-21systemd-journal20:28:272
2238299110,10cyclictest1745-21sh00:26:412
2238299110,10cyclictest0-21swapper/223:50:232
2237999119,1cyclictest14962-21ssh21:31:471
2237999118,2cyclictest7631-21ssh22:11:521
2237999118,2cyclictest5530-21expr00:30:221
2237999118,2cyclictest477-21tr22:30:211
2237999118,2cyclictest4228-21grep23:40:271
2237999118,2cyclictest3900-21rm21:46:371
2237999118,2cyclictest30434-21cut19:25:231
2237999118,2cyclictest30098-21ssh21:42:321
2237999118,2cyclictest29450-21ssh22:05:071
2237999118,2cyclictest2744-21sh23:40:001
2237999118,2cyclictest24985-21apache_processe19:15:131
2237999118,2cyclictest23452-21ssh21:15:271
2237999118,2cyclictest22371-21ssh00:17:151
2237999118,2cyclictest22371-21ssh00:17:151
2237999118,2cyclictest19658-21ssh21:12:401
2237999118,2cyclictest19310-21tr20:10:181
2237999118,2cyclictest17316-21grep20:05:201
2237999118,2cyclictest17252-21cat00:40:011
2237999118,2cyclictest15321-21ssh21:55:171
2237999118,2cyclictest14103-21basename23:25:181
2237999118,2cyclictest13534-21sed19:55:271
2237999118,2cyclictest10834-21expr23:45:171
2237999117,3cyclictest16312-21sh20:05:001
2237999117,3cyclictest11199-21sed21:00:271
2237999110,10cyclictest17011-21rm00:13:051
2237799119,2cyclictest0-21swapper/022:20:500
2237799119,1cyclictest5220-21cut00:30:210
2237799118,2cyclictest9778-21taskset20:55:470
2237799118,2cyclictest8746-21ssh23:43:560
2237799118,2cyclictest4891-21grep22:55:280
2237799118,2cyclictest4891-21grep22:55:270
2237799118,2cyclictest31244-21ssh00:25:180
2237799118,2cyclictest30661-21grep20:35:000
2237799118,2cyclictest29202-21ssh23:57:580
2237799118,2cyclictest27675-21tr19:20:160
2237799118,2cyclictest25974-21ssh21:40:140
2237799118,2cyclictest23408-21ssh22:45:390
2237799118,2cyclictest22539-21ssh21:15:160
2237799118,2cyclictest21842-21ssh00:16:360
2237799118,2cyclictest21842-21ssh00:16:360
2237799118,2cyclictest19987-21ssh23:05:410
2237799118,2cyclictest16440-21ssh00:12:230
2237799118,2cyclictest14217-21taskset22:16:160
2237799118,2cyclictest12790-21apache_volume21:05:170
2237799118,2cyclictest10876-21sh00:08:440
2237799117,3cyclictest2343-21tr19:35:160
2237799115,2cyclictest224-21systemd-journal20:20:000
2237799113,4cyclictest0-21swapper/000:20:200
2238699108,1cyclictest30319-21chrt20:30:513
2238699108,1cyclictest279892sleep320:26:263
2238699108,1cyclictest23887-21cut20:20:233
2238699108,1cyclictest12227-21expr19:55:163
2238699107,2cyclictest6286-21fschecks_time20:50:193
2238699107,2cyclictest328-21cat00:00:263
2238699107,2cyclictest29613-21ssh21:20:083
2238699107,2cyclictest29340-21ssh22:05:003
2238699107,2cyclictest18720-21grep22:20:193
2238299108,1cyclictest2809-21taskset20:42:432
2238299108,1cyclictest20098-21cat20:10:232
2238299108,1cyclictest18564-21ssh21:35:012
2238299108,1cyclictest1115-21grep00:00:352
2238299107,2cyclictest31151-21tr20:35:152
2237999108,1cyclictest245842sleep119:13:281
2237999108,1cyclictest227972sleep120:15:261
2237999107,2cyclictest8415-21expr20:55:161
2237999107,2cyclictest28143-21head19:20:181
2237999107,2cyclictest2444-21cat19:35:171
2237999107,2cyclictest21520-21ssh22:45:111
2237999107,1cyclictest32337-21grep19:30:161
2237999106,3cyclictest27086-21sed00:20:341
2237799108,1cyclictest9002-21cat19:50:000
2237799108,1cyclictest32126-21ssh23:15:160
2237799108,1cyclictest2757-21taskset20:42:030
2237799108,1cyclictest160342sleep020:01:310
2237799107,2cyclictest630-21tr20:40:000
2237799107,2cyclictest5261-21grep19:40:200
2237799107,2cyclictest313-21sort19:30:180
2237799107,2cyclictest23768-21chrt20:20:140
223869996,2cyclictest8134-21aten2_r7power_v20:55:163
223829999,0cyclictest0-21swapper/219:30:162
223829996,2cyclictest7119-21grep19:45:212
223799997,1cyclictest4980-21sed19:40:181
223799997,1cyclictest13914-21taskset21:05:251
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional