You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-12-02 - 19:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot6s.osadl.org (updated Sat Dec 02, 2023 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
13066210796,6sleep00-21swapper/007:08:590
12992210796,6sleep30-21swapper/307:08:023
12985210796,6sleep10-21swapper/107:07:561
12305210796,6sleep20-21swapper/207:05:292
309372700,0sleep20-21swapper/210:40:182
300252700,0sleep30-21swapper/310:37:193
1317999350,33cyclictest0-21swapper/009:05:200
81752330,0sleep00-21swapper/008:15:290
182992220,0sleep00-21swapper/011:30:200
1317999220,19cyclictest0-21swapper/011:35:210
1318199210,19cyclictest0-21swapper/209:05:202
1318199210,19cyclictest0-21swapper/207:30:202
19392200,1sleep10-21swapper/108:00:231
1318199200,2cyclictest0-21swapper/211:35:212
13179992015,3cyclictest5163-21snmpd08:05:190
1317999200,2cyclictest0-21swapper/007:50:180
1318099170,15cyclictest12385-21ps08:25:281
1317999170,16cyclictest0-21swapper/012:00:120
1318299161,13cyclictest32192-21fschecks_time07:55:243
1318099160,14cyclictest0-21swapper/112:30:281
13179991612,2cyclictest0-21swapper/011:45:220
13182991512,2cyclictest12249-21kworker/u8:011:21:053
1318299150,13cyclictest1949-21latency_hist10:50:013
1318199151,12cyclictest24073-21munin-run11:45:002
13181991510,3cyclictest5947-21munin-run11:00:012
13181991510,3cyclictest0-21swapper/209:50:242
1318199150,13cyclictest28067-21munin-plugin-st11:55:002
1318099151,3cyclictest14013-21hddtemp_smartct07:10:261
1318099150,3cyclictest32185-21wc12:05:351
1318099150,3cyclictest23652-21tr10:20:001
1318099150,3cyclictest0-21swapper/107:35:211
1318099150,13cyclictest31600-21sh09:15:191
1318099150,13cyclictest0-21swapper/111:45:231
1317999151,12cyclictest0-21swapper/009:15:000
13182991410,2cyclictest13844-21df_inode08:30:213
1318299140,12cyclictest9339-21telnet12:35:243
1318299140,12cyclictest5078-21apt-get09:30:223
1318299140,12cyclictest0-21swapper/311:00:283
1318199140,2cyclictest0-21swapper/212:35:322
1318199140,2cyclictest0-21swapper/207:10:322
1318199140,12cyclictest0-21swapper/208:45:282
1318099140,12cyclictest9211-21pzem_current11:05:331
1318099140,12cyclictest18469-21pzem_voltage07:20:341
1318099140,12cyclictest0-21swapper/112:25:211
1318099140,12cyclictest0-21swapper/109:20:241
1317999140,12cyclictest27974-21aten2.4-expect10:30:250
1317999140,12cyclictest0-21swapper/011:05:240
108792140,2sleep00-21swapper/011:10:300
1318299139,2cyclictest0-21swapper/307:15:363
13182991310,2cyclictest9250-21kworker/u8:009:47:053
13182991310,2cyclictest6340-21kworker/u8:211:39:053
13182991310,2cyclictest3368-21kworker/u8:208:22:053
13182991310,2cyclictest21994-21kworker/u8:210:24:053
13182991310,2cyclictest17518-21kworker/u8:108:04:053
13182991310,2cyclictest0-21swapper/310:50:333
13182991310,2cyclictest0-21swapper/307:50:193
1318299130,2cyclictest0-21swapper/311:50:273
1318299130,2cyclictest0-21swapper/307:10:103
1318299130,12cyclictest0-21swapper/310:55:233
1318299130,12cyclictest0-21swapper/307:45:393
1318299130,12cyclictest0-21swapper/307:35:253
13181991310,2cyclictest32487-21apt-get10:45:192
13181991310,2cyclictest0-21swapper/207:15:202
1318199130,2cyclictest0-21swapper/209:35:222
1318199130,12cyclictest0-21swapper/210:15:242
1318199130,11cyclictest0-21swapper/209:00:292
13180991310,2cyclictest3187-21cut12:20:141
13180991310,2cyclictest0-21swapper/111:35:241
13180991310,2cyclictest0-21swapper/111:15:331
13180991310,2cyclictest0-21swapper/110:55:191
13180991310,2cyclictest0-21swapper/110:45:191
13180991310,2cyclictest0-21swapper/110:30:331
13180991310,2cyclictest0-21swapper/110:25:331
13180991310,2cyclictest0-21swapper/109:00:341
1318099130,2cyclictest0-21swapper/108:40:231
1318099130,12cyclictest0-21swapper/111:20:241
1318099130,12cyclictest0-21swapper/110:40:251
1318099130,12cyclictest0-21swapper/109:45:201
1318099130,12cyclictest0-21swapper/108:35:271
1318099130,12cyclictest0-21swapper/107:40:041
1317999132,6cyclictest0-21swapper/010:45:010
1317999132,6cyclictest0-21swapper/009:50:000
13179991310,2cyclictest0-21swapper/012:25:290
13179991310,2cyclictest0-21swapper/009:25:000
1317999130,2cyclictest0-21swapper/012:05:260
1317999130,12cyclictest0-21swapper/011:20:360
1318299129,2cyclictest3368-21kworker/u8:208:44:053
1318299129,2cyclictest3368-21kworker/u8:208:36:053
1318299129,2cyclictest29462-21kworker/u8:009:16:053
1318299129,2cyclictest29462-21kworker/u8:009:14:053
1318299129,2cyclictest28968-21kworker/u8:012:11:053
1318299129,2cyclictest15592-21kworker/u8:010:03:053
1318299129,2cyclictest15592-21kworker/u8:010:03:053
1318299129,2cyclictest13151-21kworker/u8:207:20:053
1318299129,2cyclictest0-21swapper/308:25:373
1318299128,3cyclictest468-21kworker/u8:109:42:053
1318299128,3cyclictest18483-21munin-node3
1318299128,3cyclictest18483-21munin-node3
1318299120,2cyclictest0-21swapper/312:25:233
1318299120,11cyclictest0-21swapper/312:20:263
1318299120,11cyclictest0-21swapper/309:35:273
1318299120,11cyclictest0-21swapper/307:40:273
1318199129,2cyclictest0-21swapper/211:25:372
1318199129,2cyclictest0-21swapper/210:50:362
1318199123,6cyclictest0-21swapper/208:35:342
1318199122,7cyclictest0-21swapper/212:34:592
1318199122,7cyclictest0-21swapper/208:34:592
1318199122,6cyclictest0-21swapper/210:05:002
1318199122,6cyclictest0-21swapper/210:05:002
1318199120,2cyclictest0-21swapper/212:25:212
1318199120,2cyclictest0-21swapper/212:00:152
1318199120,2cyclictest0-21swapper/211:20:262
1318199120,2cyclictest0-21swapper/210:20:292
1318199120,2cyclictest0-21swapper/209:30:272
1318199120,2cyclictest0-21swapper/209:15:242
1318199120,2cyclictest0-21swapper/208:10:262
1318199120,2cyclictest0-21swapper/208:05:272
1318199120,2cyclictest0-21swapper/207:50:232
1318199120,11cyclictest0-21swapper/211:15:322
1318199120,11cyclictest0-21swapper/211:10:312
1318199120,11cyclictest0-21swapper/210:05:272
1318199120,11cyclictest0-21swapper/210:05:272
1318199120,11cyclictest0-21swapper/209:20:302
1318199120,11cyclictest0-21swapper/208:25:242
1318199120,11cyclictest0-21swapper/208:15:372
1318199120,11cyclictest0-21swapper/208:00:182
1318199120,11cyclictest0-21swapper/207:40:212
1318099129,2cyclictest6340-21kworker/u8:211:44:051
1318099129,2cyclictest3368-21kworker/u8:209:36:051
1318099129,2cyclictest3368-21kworker/u8:208:56:051
1318099129,2cyclictest0-21swapper/112:35:271
1318099129,2cyclictest0-21swapper/110:00:291
1318099129,2cyclictest0-21swapper/110:00:291
1318099129,2cyclictest0-21swapper/107:50:341
1318099129,2cyclictest0-21swapper/107:45:241
1318099120,2cyclictest0-21swapper/109:25:301
1318099120,11cyclictest0-21swapper/110:50:291
1318099120,11cyclictest0-21swapper/110:22:191
1318099120,11cyclictest0-21swapper/109:10:361
1318099120,11cyclictest0-21swapper/108:10:251
1318099120,10cyclictest0-21swapper/111:30:191
1317999129,2cyclictest0-21swapper/012:30:260
1317999127,3cyclictest0-21swapper/012:20:000
1317999123,6cyclictest0-21swapper/008:53:070
1317999122,7cyclictest0-21swapper/012:11:020
1317999122,7cyclictest0-21swapper/010:44:010
1317999122,7cyclictest0-21swapper/010:28:010
1317999122,6cyclictest0-21swapper/009:55:000
1317999122,6cyclictest0-21swapper/008:05:000
1317999122,6cyclictest0-21swapper/007:45:000
1317999121,6cyclictest0-21swapper/009:55:000
1317999121,6cyclictest0-21swapper/008:15:000
1317999120,2cyclictest0-21swapper/008:25:310
1317999120,11cyclictest0-21swapper/011:40:250
1317999120,11cyclictest0-21swapper/009:00:200
1317999120,11cyclictest0-21swapper/008:40:190
1317999120,11cyclictest0-21swapper/008:35:200
1317999120,11cyclictest0-21swapper/007:55:280
1317999120,11cyclictest0-21swapper/007:22:190
211932110,1sleep30-21swapper/310:10:343
1318299118,2cyclictest9975-21kworker/u8:111:15:053
1318299118,2cyclictest8888-21kworker/u8:108:18:053
1318299118,2cyclictest6340-21kworker/u8:211:41:053
1318299118,2cyclictest6340-21kworker/u8:211:26:053
1318299118,2cyclictest575-21kworker/u8:112:17:053
1318299118,2cyclictest3368-21kworker/u8:209:04:053
1318299118,2cyclictest27536-21kworker/u8:008:06:053
1318299118,2cyclictest25609-21kworker/u8:109:06:053
1318299118,2cyclictest21994-21kworker/u8:210:25:053
1318299118,2cyclictest17518-21kworker/u8:107:33:053
1318299118,2cyclictest13230-21kworker/u8:209:50:053
1318299118,2cyclictest0-21swapper/312:05:133
1318299118,2cyclictest0-21swapper/308:10:233
1318299110,10cyclictest0-21swapper/312:30:373
1318299110,10cyclictest0-21swapper/311:45:383
1318199118,2cyclictest0-21swapper/211:00:332
1318199118,2cyclictest0-21swapper/209:25:212
1318199118,2cyclictest0-21swapper/208:55:212
1318199118,2cyclictest0-21swapper/208:40:262
1318199110,9cyclictest0-21swapper/212:20:132
1318199110,9cyclictest0-21swapper/211:59:552
1318199110,6cyclictest0-21swapper/209:45:052
1318199110,1cyclictest0-21swapper/208:20:242
1318199110,10cyclictest0-21swapper/212:10:332
1318099118,2cyclictest6340-21kworker/u8:211:56:051
1318099118,2cyclictest575-21kworker/u8:112:16:051
1318099118,2cyclictest468-21kworker/u8:109:54:051
1318099118,2cyclictest13151-21kworker/u8:207:25:051
1318099118,2cyclictest0-21swapper/109:30:221
1318099112,8cyclictest0-21swapper/108:45:321
1318099110,9cyclictest0-21swapper/111:50:291
1318099110,9cyclictest0-21swapper/111:10:241
1318099110,9cyclictest0-21swapper/109:40:191
1318099110,9cyclictest0-21swapper/109:05:221
1318099110,9cyclictest0-21swapper/108:20:001
1318099110,9cyclictest0-21swapper/107:55:281
1318099110,9cyclictest0-21swapper/107:30:231
1318099110,3cyclictest21461-21munin-node1
1318099110,3cyclictest0-21swapper/110:05:201
1318099110,3cyclictest0-21swapper/110:05:201
1318099110,3cyclictest0-21swapper/108:30:241
1318099110,3cyclictest0-21swapper/108:05:341
1317999118,2cyclictest9276-21apt-get12:35:230
1317999118,2cyclictest0-21swapper/010:55:260
1317999118,2cyclictest0-21swapper/008:34:410
1317999116,3cyclictest6708-21cut11:00:280
1317999112,7cyclictest21330-1kworker/0:1H09:45:000
1317999112,7cyclictest0-21swapper/007:11:580
1317999112,5cyclictest0-21swapper/010:55:010
1317999111,8cyclictest0-21swapper/010:38:300
1317999111,6cyclictest0-21swapper/010:15:340
1317999111,6cyclictest0-21swapper/009:20:000
1317999110,10cyclictest0-21swapper/011:15:330
1318299107,2cyclictest6340-21kworker/u8:211:59:053
1318299107,2cyclictest468-21kworker/u8:109:59:053
1318299107,2cyclictest21500-21kworker/u8:008:55:063
1318299107,2cyclictest21500-21kworker/u8:008:51:053
1318299107,2cyclictest0-21swapper/309:20:243
1318199107,2cyclictest0-21swapper/211:30:352
1318199107,2cyclictest0-21swapper/210:25:272
1318199107,2cyclictest0-21swapper/209:10:352
1318199107,2cyclictest0-21swapper/207:35:282
1318199105,3cyclictest28430-21grep07:45:312
1318199105,3cyclictest23209-21sshd08:54:472
1318199105,3cyclictest20564-21switchtime07:25:332
1318199105,3cyclictest15640-21sendmail10:00:002
1318199105,3cyclictest1010-21cron08:00:002
1318199102,6cyclictest0-21swapper/207:22:502
1318199100,2cyclictest0-21swapper/212:05:172
1318099107,2cyclictest6340-21kworker/u8:212:00:051
1318099100,8cyclictest0-21swapper/111:00:301
1318099100,8cyclictest0-21swapper/108:20:181
1318099100,3cyclictest0-21swapper/107:15:261
1318099100,2cyclictest16241-21munin-node1
1318099100,2cyclictest0-21swapper/110:10:201
1317999107,2cyclictest0-21swapper/009:30:290
1317999107,2cyclictest0-21swapper/007:45:290
1317999105,3cyclictest20485-21pzem08:45:300
1317999102,6cyclictest0-21swapper/010:00:270
1317999102,6cyclictest0-21swapper/010:00:270
1317999102,6cyclictest0-21swapper/007:17:490
1317999102,5cyclictest0-21swapper/011:27:590
1317999102,5cyclictest0-21swapper/008:57:590
1317999102,5cyclictest0-21swapper/007:32:580
1317999100,7cyclictest0-21swapper/010:12:580
117502100,2sleep30-21swapper/311:12:093
3508290,1sleep00-21swapper/012:20:170
131829996,2cyclictest468-21kworker/u8:109:28:053
131829996,2cyclictest3673-21kworker/u8:011:08:053
131829996,2cyclictest31947-21kworker/u8:110:43:053
131829996,2cyclictest28968-21kworker/u8:012:03:053
131829996,2cyclictest18998-21memory11:30:263
131829994,3cyclictest0-21swapper/310:30:303
131829990,2cyclictest0-21swapper/307:25:543
131819994,3cyclictest8251-21aten2.4-expect11:05:242
131819994,3cyclictest30300-21cat10:40:002
131819994,3cyclictest28044-21cpu10:30:262
131819994,3cyclictest25135-21expr11:45:262
131819994,3cyclictest20187-21sed10:10:222
131819994,3cyclictest10122-21open_inodes09:40:232
131809996,2cyclictest29345-21kworker/u8:210:36:051
131809990,8cyclictest475-21munin-node1
131799994,3cyclictest3106-21aten2.4-expect09:25:220
131799994,3cyclictest26308-21basename11:50:230
131799994,3cyclictest25282-21munin-plugin-st07:40:000
131799994,3cyclictest19187-21grep10:05:360
131799994,3cyclictest19187-21grep10:05:350
131799994,3cyclictest10622-21switchtime08:20:340
131799992,6cyclictest0-21swapper/010:20:360
131799990,6cyclictest0-21swapper/009:38:180
131829981,3cyclictest0-21swapper/308:45:193
131829980,7cyclictest0-21swapper/310:15:053
131819985,2cyclictest0-21swapper/212:15:102
131809984,3cyclictest4958-21cron09:55:001
131799983,3cyclictest5473-21runrttasks11:55:520
131799983,3cyclictest19530-21aten2.4_r3power07:25:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional