You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-11-28 - 19:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack3slot6s.osadl.org (updated Tue Nov 28, 2023 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1839721370,2sleep11880299cyclictest09:45:401
1839721370,2sleep11880299cyclictest09:45:401
185852123101,16sleep20-21swapper/207:07:372
18651211695,16sleep30-21swapper/307:08:303
18745211187,19sleep00-21swapper/007:09:390
18429211087,18sleep10-21swapper/107:05:361
173022890,1sleep229-21rcuc/209:45:252
173022890,1sleep229-21rcuc/209:45:252
230702610,1sleep023073-21cpuspeed_turbos11:20:230
1880299271,8cyclictest0-21swapper/107:20:181
1880299230,21cyclictest0-21swapper/107:15:191
1880299180,16cyclictest0-21swapper/109:35:181
1880499161,3cyclictest8770-21copy09:25:002
1880799150,13cyclictest1586-21munin-node3
1880499151,3cyclictest3390-21cut11:50:312
1880499150,13cyclictest20848-21sort09:55:002
1880499150,13cyclictest20848-21sort09:55:002
1880499150,13cyclictest0-21swapper/209:05:182
1880299152,7cyclictest0-21swapper/110:30:011
1880299150,2cyclictest0-21swapper/110:15:281
1880299150,14cyclictest0-21swapper/108:15:001
1880199151,12cyclictest29026-21apt-get10:15:180
1880199150,13cyclictest0-21swapper/010:40:200
18807991411,2cyclictest0-21swapper/312:00:353
18807991411,2cyclictest0-21swapper/309:55:353
18807991410,2cyclictest13108-21apt-get08:15:183
1880799140,2cyclictest0-21swapper/311:25:253
1880799140,12cyclictest23146-21apt-get08:40:183
1880799140,12cyclictest0-21swapper/309:10:193
1880499140,12cyclictest0-21swapper/212:30:332
1880299142,7cyclictest0-21swapper/111:49:591
1880299142,7cyclictest0-21swapper/107:10:001
1880299142,6cyclictest0-21swapper/111:05:001
1880299140,2cyclictest0-21swapper/112:05:081
1880299140,2cyclictest0-21swapper/107:30:181
1880299140,12cyclictest0-21swapper/111:55:301
18801991411,2cyclictest0-21swapper/012:25:330
18801991411,2cyclictest0-21swapper/012:00:330
18801991411,2cyclictest0-21swapper/008:25:230
1880199140,13cyclictest0-21swapper/009:30:330
1880199140,12cyclictest0-21swapper/008:55:060
224142130,1sleep011-21rcuc/007:15:350
1880799138,3cyclictest1907-21munin-node3
1880799138,3cyclictest11573-21fschecks_time08:10:223
1880799133,6cyclictest0-21swapper/310:10:333
1880799132,6cyclictest0-21swapper/307:25:003
18807991310,2cyclictest0-21swapper/308:00:203
1880799130,8cyclictest22439-21basename12:40:003
1880799130,2cyclictest0-21swapper/311:50:003
18804991310,2cyclictest0-21swapper/207:20:202
1880499130,2cyclictest0-21swapper/207:50:062
1880499130,12cyclictest0-21swapper/210:45:242
1880299138,3cyclictest6868-21date09:20:001
1880299138,3cyclictest0-21swapper/110:35:391
1880299138,3cyclictest0-21swapper/109:55:221
1880299132,6cyclictest0-21swapper/110:30:011
1880299131,3cyclictest0-21swapper/108:15:231
18802991310,2cyclictest0-21swapper/109:30:191
1880299130,2cyclictest0-21swapper/110:40:231
1880299130,2cyclictest0-21swapper/107:25:311
1880299130,12cyclictest0-21swapper/112:10:011
1880299130,12cyclictest0-21swapper/108:05:341
18801991310,2cyclictest0-21swapper/011:35:340
18801991310,2cyclictest0-21swapper/009:20:200
18801991310,2cyclictest0-21swapper/008:15:190
1880199130,2cyclictest24974-21aten2.4_r3power11:25:250
1880199130,12cyclictest0-21swapper/011:40:220
1880199130,12cyclictest0-21swapper/011:30:350
1880199130,12cyclictest0-21swapper/008:50:320
1880199130,11cyclictest0-21swapper/012:05:010
98602120,2sleep10-21swapper/109:25:301
70792120,2sleep00-21swapper/008:00:190
219132120,2sleep20-21swapper/209:55:322
1880799129,2cyclictest0-21swapper/312:10:003
1880799129,2cyclictest0-21swapper/308:35:243
1880799129,2cyclictest0-21swapper/307:36:183
1880799120,2cyclictest0-21swapper/312:20:273
1880799120,11cyclictest0-21swapper/311:50:233
1880799120,11cyclictest0-21swapper/311:35:273
1880799120,11cyclictest0-21swapper/307:30:183
1880499129,2cyclictest18784-21apt-get07:10:042
1880499129,2cyclictest0-21swapper/210:20:212
1880499129,2cyclictest0-21swapper/210:00:282
1880499129,2cyclictest0-21swapper/209:00:292
1880499129,2cyclictest0-21swapper/208:50:322
1880499129,2cyclictest0-21swapper/207:40:362
1880499120,2cyclictest0-21swapper/211:20:342
1880499120,2cyclictest0-21swapper/207:45:262
1880499120,11cyclictest0-21swapper/212:05:202
1880499120,11cyclictest0-21swapper/211:55:262
1880499120,11cyclictest0-21swapper/211:45:272
1880499120,10cyclictest0-21swapper/209:25:302
1880299129,2cyclictest0-21swapper/112:30:231
1880299129,2cyclictest0-21swapper/112:20:391
1880299129,2cyclictest0-21swapper/112:15:021
1880299128,2cyclictest0-21swapper/112:25:201
1880299127,3cyclictest30835-21munin-node1
1880299127,3cyclictest0-21swapper/107:35:271
1880299122,7cyclictest0-21swapper/110:22:591
1880299121,9cyclictest0-21swapper/111:05:261
1880299121,9cyclictest0-21swapper/109:10:351
1880299121,3cyclictest0-21swapper/109:05:331
1880299121,3cyclictest0-21swapper/107:55:411
1880299120,2cyclictest0-21swapper/109:50:221
1880299120,2cyclictest0-21swapper/109:50:221
1880299120,2cyclictest0-21swapper/108:55:341
1880299120,2cyclictest0-21swapper/108:40:341
1880299120,11cyclictest0-21swapper/110:10:231
1880299120,11cyclictest0-21swapper/109:20:241
1880299120,11cyclictest0-21swapper/108:20:261
1880299120,10cyclictest1151-21grep07:45:191
1880299120,10cyclictest0-21swapper/110:55:291
1880199129,2cyclictest21024-21apt-get09:55:210
1880199129,2cyclictest0-21swapper/012:15:260
1880199129,2cyclictest0-21swapper/011:55:230
1880199129,2cyclictest0-21swapper/011:45:200
1880199129,2cyclictest0-21swapper/010:30:210
1880199129,2cyclictest0-21swapper/010:20:250
1880199129,2cyclictest0-21swapper/008:05:240
1880199120,2cyclictest0-21swapper/011:55:000
1880199120,2cyclictest0-21swapper/010:45:210
1880199120,2cyclictest0-21swapper/010:10:370
1880199120,2cyclictest0-21swapper/009:50:080
1880199120,2cyclictest0-21swapper/009:50:080
1880199120,2cyclictest0-21swapper/009:40:210
1880199120,2cyclictest0-21swapper/009:15:210
1880199120,2cyclictest0-21swapper/007:35:360
1880199120,11cyclictest0-21swapper/011:15:290
1880199120,11cyclictest0-21swapper/011:10:250
1880199120,11cyclictest0-21swapper/009:00:260
1880199120,11cyclictest0-21swapper/008:35:180
1880199120,11cyclictest0-21swapper/007:50:270
1880199120,11cyclictest0-21swapper/007:40:200
1880199120,11cyclictest0-21swapper/007:20:210
1880799118,2cyclictest0-21swapper/311:30:223
1880799118,2cyclictest0-21swapper/311:10:303
1880799118,2cyclictest0-21swapper/309:35:203
1880799118,2cyclictest0-21swapper/307:40:213
1880799112,6cyclictest0-21swapper/309:45:393
1880799112,6cyclictest0-21swapper/309:45:383
1880799112,5cyclictest0-21swapper/310:45:013
1880799111,6cyclictest0-21swapper/310:20:213
1880799110,2cyclictest0-21swapper/309:25:353
1880499118,2cyclictest18817-21apt-get11:10:212
1880499118,2cyclictest12855-21aten2.4-expect10:55:222
1880499118,2cyclictest0-21swapper/212:35:222
1880499118,2cyclictest0-21swapper/209:35:202
1880499118,2cyclictest0-21swapper/209:10:262
1880499118,2cyclictest0-21swapper/207:35:312
1880499110,9cyclictest15902-21sensors11:00:322
1880499110,9cyclictest13021-21munin-node2
1880499110,2cyclictest0-21swapper/211:15:382
1880499110,10cyclictest0-21swapper/212:10:352
1880299119,1cyclictest0-21swapper/108:00:251
1880299118,2cyclictest18800-21cyclictest07:40:351
1880299118,2cyclictest0-21swapper/108:35:191
1880299116,3cyclictest0-21swapper/108:50:221
1880299112,6cyclictest0-21swapper/110:55:011
1880299111,8cyclictest3570-21iostat_ios07:50:271
1880299111,8cyclictest25390-21perl11:25:271
1880299111,8cyclictest0-21swapper/110:05:271
1880299111,8cyclictest0-21swapper/108:25:301
1880299111,3cyclictest0-21swapper/112:00:291
1880299111,3cyclictest0-21swapper/110:00:211
1880299110,9cyclictest31187-21munin-node1
1880299110,9cyclictest0-21swapper/112:35:331
1880299110,9cyclictest0-21swapper/111:20:391
1880299110,9cyclictest0-21swapper/108:30:271
1880299110,3cyclictest8765-21munin-node1
1880299110,3cyclictest0-21swapper/111:10:291
1880199118,2cyclictest0-21swapper/012:32:330
1880199118,2cyclictest0-21swapper/011:00:190
1880199118,2cyclictest0-21swapper/010:55:320
1880199118,2cyclictest0-21swapper/008:40:210
1880199118,2cyclictest0-21swapper/008:10:330
1880199110,2cyclictest0-21swapper/012:10:080
82202100,1sleep20-21swapper/210:40:592
1880799107,2cyclictest0-21swapper/312:30:243
1880799107,2cyclictest0-21swapper/310:35:283
1880799107,2cyclictest0-21swapper/310:05:383
1880799107,2cyclictest0-21swapper/307:10:243
1880799105,3cyclictest29317-21tr10:15:213
1880799102,6cyclictest0-21swapper/312:15:343
1880799101,5cyclictest0-21swapper/308:00:003
1880799100,8cyclictest0-21swapper/310:55:223
1880799100,6cyclictest0-21swapper/308:10:003
1880499107,2cyclictest0-21swapper/211:25:372
1880499107,2cyclictest0-21swapper/210:05:302
1880499105,3cyclictest304-21jbd2/sda1-807:25:402
1880499104,4cyclictest27006-21ls08:50:002
1880499101,7cyclictest5473-21runrttasks10:37:292
1880499100,9cyclictest30268-21sensors08:55:342
1880499100,2cyclictest0-21swapper/212:00:292
1880299107,2cyclictest26825-21apt-get11:30:201
1880299107,2cyclictest0-21swapper/111:50:231
1880299107,2cyclictest0-21swapper/109:40:231
1880299107,2cyclictest0-21swapper/108:45:211
1880299100,3cyclictest0-21swapper/111:15:291
1880299100,2cyclictest0-21swapper/111:35:201
1880199107,2cyclictest0-21swapper/009:10:250
1880199107,2cyclictest0-21swapper/008:20:230
1880199107,2cyclictest0-21swapper/007:49:330
1880199106,2cyclictest1738-21switchtime10:25:350
1880199105,3cyclictest17124-21df_abs11:05:250
1880199100,9cyclictest0-21swapper/009:50:000
1880199100,9cyclictest0-21swapper/009:50:000
1880199100,2cyclictest0-21swapper/008:31:030
5846290,1sleep00-21swapper/007:55:350
188079994,3cyclictest5378-21Xorg12:25:593
188079994,3cyclictest5302-21ntp_kernel_pll_11:55:303
188079994,3cyclictest4375-21grep10:35:003
188079994,3cyclictest3551-21if_eth007:50:283
188079994,3cyclictest27559-21fschecks_count08:50:213
188079994,3cyclictest26432-21grep07:25:363
188079994,3cyclictest21935-21switchtime11:15:383
188079994,3cyclictest19756-21wc09:50:253
188079994,3cyclictest19756-21wc09:50:253
188079994,3cyclictest17697-21ntp_states11:05:303
188079994,3cyclictest15860-21latency08:20:293
188079994,3cyclictest15272-21cut09:40:223
188079994,3cyclictest11828-21timerandwakeup12:10:383
188079994,3cyclictest11506-21fschecks_time09:30:233
188079994,3cyclictest11106-21df_abs10:50:243
188079991,6cyclictest0-21swapper/309:15:323
188079991,6cyclictest0-21swapper/308:45:333
188079991,3cyclictest0-21swapper/310:45:223
188079990,8cyclictest0-21swapper/310:00:183
188079990,8cyclictest0-21swapper/307:15:003
188079990,2cyclictest0-21swapper/311:40:223
188079990,2cyclictest0-21swapper/311:00:223
188049995,3cyclictest12847-21apt-get08:15:062
188049994,3cyclictest7111-21aten2.4-expect08:00:202
188049994,3cyclictest4807-21nscd08:40:262
188049994,3cyclictest3258-21uname10:30:242
188049994,3cyclictest27293-21cpuspeed_turbos10:10:222
188049994,3cyclictest27168-21df_abs11:30:232
188049994,3cyclictest21106-21chrt08:35:022
188049994,3cyclictest19128-21munin-node2
188049994,3cyclictest180932sleep212:25:562
188049994,3cyclictest17585-21fschecks_count08:25:282
188049994,3cyclictest15285-21aten2.4-expect08:20:242
188049994,3cyclictest12671-21latency_hist10:55:012
188049994,3cyclictest0-21swapper/209:40:362
188049994,3cyclictest0-21swapper/207:15:222
188049993,4cyclictest0-21swapper/207:55:352
188049991,6cyclictest27154-21aten2.4_r3power07:30:192
188049991,3cyclictest0-21swapper/211:40:252
188049990,7cyclictest0-21swapper/208:10:282
188049990,2cyclictest0-21swapper/212:20:212
188049990,2cyclictest0-21swapper/210:15:312
188049990,2cyclictest0-21swapper/209:33:162
188019996,2cyclictest0-21swapper/012:35:300
188019996,2cyclictest0-21swapper/007:30:310
188019995,2cyclictest23099-21telnet10:00:180
188019990,8cyclictest0-21swapper/010:36:200
188019990,7cyclictest0-21swapper/008:45:250
188019990,2cyclictest0-21swapper/007:10:290
188079984,2cyclictest6946-21munin-node3
188079983,3cyclictest906-21aten2.4_r3power07:45:183
188079983,3cyclictest31629-21grep09:00:263
188079983,3cyclictest29235-21telnet08:55:213
188079983,3cyclictest22892-21aten2.4-expect11:20:223
188079983,3cyclictest19117-21chrt08:30:103
188079983,3cyclictest17148-21taskset08:25:243
188049984,2cyclictest0-21swapper/210:25:212
188049983,3cyclictest9400-21sed08:05:212
188049983,3cyclictest4942-21aten2.4_r3power09:15:202
188049983,3cyclictest28804-21chrt11:35:072
188049983,3cyclictest16965-21head11:05:242
188019984,3cyclictest726-21apt09:05:180
188019980,7cyclictest11780-21pzem10:50:360
188019980,7cyclictest0-21swapper/012:20:030
188019980,7cyclictest0-21swapper/007:27:090
188019980,2cyclictest0-21swapper/009:36:590
188019974,2cyclictest0-21swapper/009:25:150
188019973,3cyclictest25063-21cut10:05:220
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional