You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-08-14 - 20:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

x86 Intel Core i3-6100U @2300 MHz, Linux 4.9.47-rt37 (Profile)

Latency plot of shadow in rack #7, slot #7
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the alll highest latencies:
System rack7slot7s.osadl.org (updated Sun Aug 14, 2022 12:44:39)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21579218265,111sleep30-21swapper/307:09:453
21527217065,81sleep10-21swapper/107:09:031
21345210462,12sleep20-21swapper/207:06:442
2145228362,17sleep00-21swapper/007:08:050
16782810,3sleep20-21swapper/210:40:422
186372660,1sleep10-21swapper/111:12:161
32422640,1sleep20-21swapper/210:02:332
27522630,2sleep10-21swapper/109:23:151
286822270,1sleep20-21swapper/210:59:042
2188399252,17cyclictest10312-21kworker/0:211:35:000
21885992117,3cyclictest7546-21apt-get12:25:152
273652200,1sleep10-21swapper/110:17:141
2188499200,19cyclictest0-21swapper/107:50:131
2188599181,2cyclictest0-21swapper/209:45:022
2188399180,17cyclictest0-21swapper/011:12:040
145182170,2sleep20-21swapper/209:30:242
21885991613,2cyclictest15170-21apt-get10:10:192
2188399161,5cyclictest10312-21kworker/0:211:45:140
2188399161,12cyclictest0-21swapper/011:40:160
310372150,13sleep157650irq/129-enp2s0-12:19:501
42492140,2sleep30-21swapper/310:42:433
2188699140,11cyclictest57450irq/127-enp2s0-11:05:193
21885991411,2cyclictest31390-21apt-get11:40:182
2188399140,13cyclictest8208-21ls11:25:250
2188399140,10cyclictest0-21swapper/011:08:230
2188699137,5cyclictest16850irq/16-i801_smb10:12:333
2188699133,1cyclictest234-21systemd-journal11:50:143
2188699130,6cyclictest0-21swapper/311:40:003
2188599137,5cyclictest7041-21grep12:25:012
2188399135,7cyclictest0-21swapper/009:15:260
2188399135,3cyclictest0-21swapper/008:55:180
2188399131,7cyclictest0-21swapper/009:45:210
2188399131,6cyclictest0-21swapper/007:55:000
2188399130,9cyclictest0-21swapper/010:58:360
2188399130,9cyclictest0-21swapper/010:46:320
2188399130,9cyclictest0-21swapper/010:35:040
2188399130,9cyclictest0-21swapper/010:04:410
2188399130,9cyclictest0-21swapper/009:35:180
2188399130,8cyclictest0-21swapper/012:26:270
2188399130,10cyclictest0-21swapper/010:07:170
296962120,1sleep30-21swapper/310:59:523
2188699129,2cyclictest31295-21awk10:40:003
2188699127,4cyclictest16850irq/16-i801_smb08:25:263
2188699126,1cyclictest0-21swapper/309:40:303
2188699124,3cyclictest57650irq/129-enp2s0-10:04:093
2188699120,8cyclictest0-21swapper/311:45:583
2188699120,6cyclictest0-21swapper/310:30:293
2188599128,3cyclictest15241-21apt-get11:30:222
2188599128,3cyclictest0-21swapper/208:10:222
2188499126,5cyclictest16850irq/16-i801_smb10:30:311
2188499124,7cyclictest15037-21perf10:29:591
2188499120,11cyclictest0-21swapper/107:27:341
2188399120,9cyclictest15873-21grep11:50:180
2188399120,9cyclictest0-21swapper/012:15:290
2188399120,9cyclictest0-21swapper/011:17:290
2188399120,9cyclictest0-21swapper/010:50:010
2188399120,9cyclictest0-21swapper/009:30:050
2188399120,9cyclictest0-21swapper/007:30:160
2188399120,8cyclictest0-21swapper/012:30:510
2188399120,7cyclictest0-21swapper/007:45:000
2188399120,11cyclictest0-21swapper/012:10:040
2188399120,11cyclictest0-21swapper/012:06:420
2188399120,11cyclictest0-21swapper/012:06:410
2188399120,11cyclictest0-21swapper/011:55:150
2188399120,11cyclictest0-21swapper/010:41:080
2188399120,11cyclictest0-21swapper/010:27:570
2188399120,11cyclictest0-21swapper/010:11:170
2188699116,3cyclictest0-21swapper/309:55:333
2188699111,6cyclictest0-21swapper/309:54:133
2188699110,8cyclictest0-21swapper/311:42:093
2188699110,6cyclictest0-21swapper/309:25:513
2188699110,4cyclictest0-21swapper/312:11:503
2188699110,4cyclictest0-21swapper/311:56:203
2188699110,4cyclictest0-21swapper/311:33:043
2188699110,4cyclictest0-21swapper/310:51:503
2188699110,4cyclictest0-21swapper/310:45:433
2188699110,4cyclictest0-21swapper/310:26:153
2188699110,4cyclictest0-21swapper/310:24:353
2188699110,4cyclictest0-21swapper/309:37:193
2188699110,4cyclictest0-21swapper/309:33:083
2188699110,4cyclictest0-21swapper/309:22:253
2188699110,3cyclictest57650irq/129-enp2s0-09:18:033
2188699110,3cyclictest0-21swapper/312:35:223
2188699110,3cyclictest0-21swapper/312:15:193
2188699110,2cyclictest0-21swapper/312:20:443
2188599118,2cyclictest9630-21ssh11:06:352
2188599118,2cyclictest30500-21munin-run10:00:012
2188599118,2cyclictest23848-21apt-get12:15:222
2188599118,1cyclictest57550irq/128-enp2s0-12:38:422
2188599117,3cyclictest15097-21munin-run12:10:012
2188599117,3cyclictest15097-21munin-run12:10:012
2188599117,3cyclictest10388-21sh09:04:592
2188599113,1cyclictest0-21swapper/210:05:152
2188599113,0cyclictest0-21swapper/211:15:242
2188599110,8cyclictest36-21ksoftirqd/212:00:152
2188599110,8cyclictest36-21ksoftirqd/211:24:262
2188599110,8cyclictest36-21ksoftirqd/209:20:222
2188499118,2cyclictest28-21rcuop/108:40:251
2188499118,2cyclictest2639-21ssh11:02:321
2188499117,3cyclictest16850irq/16-i801_smb09:50:301
2188499117,3cyclictest16850irq/16-i801_smb08:55:261
2188499111,9cyclictest31183-21awk08:40:001
2188499110,9cyclictest12691-21diskmemload12:25:221
2188399114,2cyclictest0-21swapper/011:20:220
2188399114,2cyclictest0-21swapper/008:50:120
2188399113,3cyclictest32032-21grep10:20:170
2188399112,5cyclictest30558-21expr09:40:210
2188399111,6cyclictest0-21swapper/008:17:130
2188399111,5cyclictest14040-21perl09:10:190
2188399111,5cyclictest0-21swapper/011:00:210
2188399111,5cyclictest0-21swapper/009:25:110
2188399110,8cyclictest0-21swapper/009:51:040
2188399110,8cyclictest0-21swapper/009:20:170
2188399110,8cyclictest0-21swapper/008:45:230
2188399110,8cyclictest0-21swapper/008:25:200
2188399110,7cyclictest0-21swapper/007:45:170
2188399110,6cyclictest23144-21munin-run10:35:000
2188399110,6cyclictest22242-21/usr/sbin/munin08:20:250
2188399110,6cyclictest17377-21latency_hist08:10:000
2188399110,6cyclictest0-21swapper/012:00:200
2188399110,6cyclictest0-21swapper/011:35:170
2188399110,5cyclictest31827-21apache_accesses12:20:140
2188399110,5cyclictest0-21swapper/009:57:040
2188399110,10cyclictest0-21swapper/012:37:120
94262100,2sleep20-21swapper/210:25:352
2188699106,3cyclictest16850irq/16-i801_smb09:00:263
2188699106,3cyclictest16850irq/16-i801_smb07:57:083
2188699106,3cyclictest16850irq/16-i801_smb07:30:263
2188699103,1cyclictest7493-21/usr/sbin/munin12:25:223
2188699103,0cyclictest0-21swapper/307:15:213
2188699100,7cyclictest0-21swapper/311:00:083
2188699100,5cyclictest0-21swapper/308:00:153
2188699100,4cyclictest23340-21apache_volume10:15:203
2188699100,4cyclictest0-21swapper/309:10:163
2188699100,3cyclictest0-21swapper/312:31:323
2188699100,3cyclictest0-21swapper/312:06:163
2188699100,3cyclictest0-21swapper/312:06:163
2188699100,3cyclictest0-21swapper/312:01:553
2188699100,3cyclictest0-21swapper/311:25:223
2188699100,3cyclictest0-21swapper/311:21:003
2188699100,3cyclictest0-21swapper/311:16:373
2188699100,3cyclictest0-21swapper/311:10:273
2188699100,2cyclictest0-21swapper/310:05:203
2188699100,2cyclictest0-21swapper/309:45:253
2188599108,1cyclictest0-21swapper/212:10:202
2188599107,2cyclictest24781-21df_abs07:15:162
2188599107,2cyclictest23348-21ssh09:15:342
2188599107,2cyclictest10058-21idleruntime-cro07:55:012
2188599106,3cyclictest3163-21apt07:40:012
2188599106,3cyclictest15651-21irqstats09:50:252
2188599105,3cyclictest20981-21grep08:15:222
2188599102,7cyclictest36-21ksoftirqd/211:35:142
2188599102,3cyclictest27825-21ssh09:38:412
2188599100,9cyclictest36-21ksoftirqd/209:25:482
2188599100,7cyclictest39-21rcuop/211:00:212
2188499106,3cyclictest16850irq/16-i801_smb12:30:281
2188499106,3cyclictest16850irq/16-i801_smb12:00:271
2188499105,4cyclictest16850irq/16-i801_smb09:10:261
2188499105,2cyclictest0-21swapper/108:10:221
2188499102,3cyclictest0-21swapper/109:45:181
2188499101,5cyclictest0-21swapper/109:15:181
2188499100,9cyclictest25-21ksoftirqd/110:35:231
2188399103,2cyclictest0-21swapper/007:10:190
2188399102,4cyclictest0-21swapper/010:15:210
2188399101,8cyclictest0-21swapper/007:35:170
2188399100,9cyclictest0-21swapper/009:00:240
2188399100,9cyclictest0-21swapper/007:55:200
2188399100,7cyclictest0-21swapper/007:15:200
25827290,1sleep20-21swapper/211:55:432
25296290,1sleep10-21swapper/111:16:011
218869996,2cyclictest16850irq/16-i801_smb07:50:223
218869995,3cyclictest16850irq/16-i801_smb08:20:263
218869995,3cyclictest16850irq/16-i801_smb08:07:113
218869994,4cyclictest12673-21idleruntime-cro09:10:003
218869992,0cyclictest0-21swapper/308:15:243
218869991,0cyclictest0-21swapper/308:40:193
218869990,2cyclictest16850irq/16-i801_smb07:10:263
218869990,2cyclictest0-21swapper/308:10:213
218859998,1cyclictest0-21swapper/210:45:362
218859996,2cyclictest234-21systemd-journal07:50:012
218859996,2cyclictest15674-21/usr/sbin/munin12:30:242
218859996,2cyclictest10-21rcuop/011:10:262
218859996,2cyclictest10-21rcuop/008:04:592
218859996,2cyclictest0-21swapper/208:48:322
218859995,3cyclictest8328-21ntp_states11:45:222
218859995,3cyclictest10-21rcuop/011:50:212
218859990,8cyclictest57650irq/129-enp2s0-07:25:222
218859990,8cyclictest36-21ksoftirqd/211:28:482
218859990,8cyclictest36-21ksoftirqd/210:37:062
218859990,8cyclictest36-21ksoftirqd/210:15:272
218859990,8cyclictest36-21ksoftirqd/209:40:172
218859990,8cyclictest36-21ksoftirqd/209:10:252
218849997,1cyclictest0-21swapper/112:22:491
218849997,1cyclictest0-21swapper/111:05:281
218849996,3cyclictest16850irq/16-i801_smb12:12:551
218849996,3cyclictest16850irq/16-i801_smb12:07:541
218849996,3cyclictest16850irq/16-i801_smb12:07:541
218849995,3cyclictest16850irq/16-i801_smb11:30:351
218849995,3cyclictest16850irq/16-i801_smb10:45:361
218849995,3cyclictest16850irq/16-i801_smb09:55:321
218849994,4cyclictest16850irq/16-i801_smb11:55:231
218849992,3cyclictest0-21swapper/111:28:051
218849991,2cyclictest0-21swapper/110:50:261
218849990,8cyclictest25-21ksoftirqd/110:05:311
218849990,8cyclictest0-21swapper/112:35:291
218849990,5cyclictest0-21swapper/110:59:181
218849990,5cyclictest0-21swapper/110:40:061
218849990,5cyclictest0-21swapper/110:13:171
218849990,5cyclictest0-21swapper/109:37:201
218849990,4cyclictest234-21systemd-journal10:00:011
218839991,7cyclictest0-21swapper/008:10:130
218839990,8cyclictest0-21swapper/008:35:360
218839990,5cyclictest0-21swapper/008:30:210
17777290,1sleep20-21swapper/210:51:522
218869985,2cyclictest16850irq/16-i801_smb07:45:253
218869984,3cyclictest16850irq/16-i801_smb07:25:273
218869981,6cyclictest3497-21perf08:49:593
218869981,1cyclictest26536-21/usr/sbin/munin07:20:253
218869980,7cyclictest0-21swapper/308:55:223
218869980,6cyclictest3436-21apache_processe07:40:153
218869980,4cyclictest31211-21idleruntime-cro08:39:593
218869980,2cyclictest27574-21hddtemp_smartct08:30:243
218859985,2cyclictest15436-21grep08:05:202
218859984,3cyclictest3442-21apt-get07:40:162
218859984,3cyclictest29157-21apache_processe08:35:232
218859984,3cyclictest22242-21/usr/sbin/munin08:20:142
218859984,3cyclictest10788-21cpu09:05:182
218859983,4cyclictest26753-21cpuspeed_turbos07:20:182
218859981,5cyclictest31450-21ssh10:20:142
218859980,7cyclictest1333-21runrttasks08:25:012
218859980,7cyclictest0-21swapper/210:30:192
218859980,7cyclictest0-21swapper/208:55:162
218849987,1cyclictest0-21swapper/107:22:021
218849986,1cyclictest0-21swapper/108:00:251
218849985,2cyclictest0-21swapper/108:15:241
218849984,4cyclictest16850irq/16-i801_smb08:50:231
218849984,0cyclictest0-21swapper/107:15:251
218849983,4cyclictest57450irq/127-enp2s0-11:50:171
218849981,6cyclictest0-21swapper/108:25:221
218849981,2cyclictest4366-21ssh11:43:141
218849980,7cyclictest23569-21head07:10:291
218849980,7cyclictest0-21swapper/111:45:191
218849980,7cyclictest0-21swapper/111:35:181
218849980,7cyclictest0-21swapper/109:40:211
218849980,7cyclictest0-21swapper/109:30:521
218849980,7cyclictest0-21swapper/108:20:271
218849980,6cyclictest0-21swapper/108:45:211
218849980,6cyclictest0-21swapper/107:40:141
218849980,5cyclictest0-21swapper/109:25:051
218849980,4cyclictest57350irq/126-enp2s0-08:30:201
218849980,4cyclictest0-21swapper/111:20:201
218849980,4cyclictest0-21swapper/110:24:381
218849980,3cyclictest0-21swapper/107:55:181
218839980,7cyclictest31462-21apache_volume08:40:140
218839980,7cyclictest28855-21apt-get07:25:150
218839980,7cyclictest26560-21apache_volume07:20:170
218839980,7cyclictest12840-21apache_accesses08:00:140
218839980,7cyclictest0-21swapper/009:05:150
218869970,6cyclictest0-21swapper/307:35:143
218869970,4cyclictest0-21swapper/308:50:133
218859977,0cyclictest0-21swapper/208:50:222
218859971,5cyclictest27260-21expr08:30:232
218859970,6cyclictest36-21ksoftirqd/208:43:302
218859970,6cyclictest0-21swapper/207:55:022
218859970,6cyclictest0-21swapper/207:10:152
218849974,1cyclictest57550irq/128-enp2s0-07:30:171
218849973,3cyclictest10615-21/usr/sbin/munin09:05:191
218849972,4cyclictest57350irq/126-enp2s0-07:45:221
218849971,5cyclictest0-21swapper/108:05:211
218849970,6cyclictest0-21swapper/109:00:181
218849970,6cyclictest0-21swapper/107:35:161
218859963,2cyclictest31150-21apt-get07:30:152
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional