You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-10-07 - 20:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

x86 Intel Core i3-6100U @2300 MHz, Linux 4.9.47-rt37 (Profile)

Latency plot of shadow in rack #7, slot #7
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Fri Oct 07, 2022 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5645218462,93sleep10-21swapper/107:07:461
5530218165,110sleep30-21swapper/307:06:153
5663210261,12sleep20-21swapper/207:07:592
579528463,17sleep00-21swapper/007:09:410
272822740,1sleep30-21swapper/309:44:283
27092650,0sleep00-21swapper/008:10:200
165542630,2sleep310-21rcuop/011:26:133
80052590,1sleep20-21swapper/211:03:442
204712590,3sleep3631899cyclictest09:22:093
631699360,17cyclictest0-21swapper/109:00:241
66002230,3sleep3631899cyclictest12:15:233
6315992315,5cyclictest3-21ksoftirqd/009:50:140
55452230,2sleep0631599cyclictest12:33:210
64602210,2sleep10-21swapper/109:15:091
631599203,11cyclictest22649-21awk12:25:000
6317991915,3cyclictest30752-21ssh10:04:082
631699184,2cyclictest0-21swapper/111:55:171
6315991812,4cyclictest3-21ksoftirqd/008:30:170
631599180,0cyclictest17623-21awk10:15:000
291472180,2sleep10-21swapper/109:26:541
208582180,1sleep30-21swapper/310:52:193
68392170,0sleep00-21swapper/010:26:140
631699170,5cyclictest0-21swapper/110:55:451
631699170,4cyclictest0-21swapper/109:41:201
631599177,5cyclictest3-21ksoftirqd/012:00:230
631599172,12cyclictest1539-21kworker/0:211:10:270
631699160,4cyclictest0-21swapper/112:30:241
631699160,4cyclictest0-21swapper/111:15:281
631699160,4cyclictest0-21swapper/110:43:221
631699160,4cyclictest0-21swapper/110:00:301
631699160,3cyclictest0-21swapper/111:20:121
631599163,12cyclictest13244-21munin-run07:25:000
6315991613,2cyclictest3-21ksoftirqd/011:45:150
631599161,12cyclictest9576-21kworker/0:007:30:320
631599160,4cyclictest13584-21apt-get12:20:010
631599160,4cyclictest0-21swapper/011:34:260
631599160,15cyclictest0-21swapper/009:47:520
631599160,15cyclictest0-21swapper/008:55:010
631599160,11cyclictest0-21swapper/010:40:150
169092160,0sleep00-21swapper/009:55:490
631699150,3cyclictest0-21swapper/112:38:541
631699150,3cyclictest0-21swapper/112:28:131
631599155,5cyclictest3-21ksoftirqd/010:30:220
631599154,6cyclictest20215-21basename07:40:010
631599154,10cyclictest3-21ksoftirqd/009:15:210
631599151,1cyclictest174-1kworker/0:1H10:49:590
631599150,14cyclictest1539-21kworker/0:210:55:520
631599150,14cyclictest0-21swapper/009:32:250
102842150,2sleep10-21swapper/110:28:381
87972140,12sleep257550irq/128-enp2s0-10:45:272
631599145,8cyclictest3-21ksoftirqd/009:27:080
631599141,12cyclictest1539-21kworker/0:212:10:270
631599140,4cyclictest0-21swapper/010:36:460
631599140,13cyclictest3705-21ssh11:19:350
631599140,10cyclictest1539-21kworker/0:208:49:590
631599140,10cyclictest0-21swapper/010:05:040
631899131,11cyclictest0-21swapper/310:25:143
631899130,9cyclictest47-21ksoftirqd/309:14:583
631899130,12cyclictest0-21swapper/310:55:373
631799138,1cyclictest21094-21ssh09:40:242
631599138,4cyclictest3-21ksoftirqd/007:55:150
631599135,1cyclictest3-21ksoftirqd/009:20:000
631599134,6cyclictest3-21ksoftirqd/009:10:230
631599133,9cyclictest0-21swapper/012:29:230
631599133,6cyclictest0-21swapper/008:40:150
631599132,4cyclictest556-21snmpd10:04:040
631599130,9cyclictest0-21swapper/011:24:130
631599130,9cyclictest0-21swapper/011:09:450
631599130,9cyclictest0-21swapper/009:39:210
631599130,8cyclictest0-21swapper/010:15:200
631599130,8cyclictest0-21swapper/008:25:180
631599130,11cyclictest9576-21kworker/0:008:01:390
631899127,2cyclictest57650irq/129-enp2s0-10:20:273
631899126,3cyclictest0-21swapper/311:45:153
631899123,1cyclictest0-21swapper/309:15:133
631899122,9cyclictest47-21ksoftirqd/311:16:213
631899122,9cyclictest47-21ksoftirqd/309:58:333
631899120,9cyclictest47-21ksoftirqd/311:24:173
631699126,5cyclictest16850irq/16-i801_smb12:15:241
631699126,2cyclictest0-21swapper/110:50:281
631699125,6cyclictest16850irq/16-i801_smb11:42:511
631699123,7cyclictest31701-21perf08:05:001
631699120,9cyclictest0-21swapper/111:05:151
631699120,9cyclictest0-21swapper/109:55:151
631599129,2cyclictest3-21ksoftirqd/007:10:010
631599128,3cyclictest41ktimersoftd/012:05:520
631599126,5cyclictest3-21ksoftirqd/012:36:060
631599125,6cyclictest3-21ksoftirqd/011:38:500
631599125,1cyclictest3-21ksoftirqd/010:53:230
631599124,7cyclictest3-21ksoftirqd/011:50:010
631599124,1cyclictest3-21ksoftirqd/011:55:160
631599122,6cyclictest19633-21aten2_r7power_e09:40:160
631599121,9cyclictest1539-21kworker/0:209:04:350
631599120,11cyclictest0-21swapper/011:40:310
631599120,11cyclictest0-21swapper/011:27:540
631599120,11cyclictest0-21swapper/007:40:240
631899119,1cyclictest0-21swapper/309:50:143
631899118,2cyclictest0-21swapper/311:05:153
631899117,3cyclictest50-21rcuop/309:09:593
631899117,3cyclictest18125-21apache_volume10:15:123
631899117,3cyclictest16850irq/16-i801_smb11:02:463
631899116,4cyclictest57350irq/126-enp2s0-10:35:303
631899116,4cyclictest16850irq/16-i801_smb10:00:263
631899116,4cyclictest16850irq/16-i801_smb09:45:303
631899116,3cyclictest15537-21munin-run07:30:003
631899110,4cyclictest0-21swapper/310:33:193
631899110,10cyclictest0-21swapper/311:30:233
631799117,4cyclictest16850irq/16-i801_smb09:20:242
631799112,3cyclictest29262-21diskmemload11:56:342
631799111,7cyclictest0-21swapper/208:00:202
631699116,4cyclictest16850irq/16-i801_smb10:10:241
631699115,4cyclictest11344-21cpuspeed_turbos08:30:181
631699114,6cyclictest16850irq/16-i801_smb10:05:241
631699113,0cyclictest0-21swapper/110:33:421
631699111,6cyclictest0-21swapper/109:33:351
631599118,2cyclictest3-21ksoftirqd/008:40:000
631599118,2cyclictest3-21ksoftirqd/008:15:150
631599116,4cyclictest3-21ksoftirqd/007:25:140
631599115,5cyclictest3-21ksoftirqd/010:20:170
631599114,0cyclictest0-21swapper/007:15:280
631599113,7cyclictest3-21ksoftirqd/007:45:130
631599110,5cyclictest32534-21diskstats08:05:190
631599110,3cyclictest0-21swapper/011:00:060
631599110,11cyclictest0-21swapper/009:05:140
631899108,1cyclictest29262-21diskmemload12:20:273
631899108,1cyclictest0-21swapper/312:35:263
631899108,1cyclictest0-21swapper/310:10:153
631899107,2cyclictest50-21rcuop/312:05:163
631899107,2cyclictest50-21rcuop/308:45:213
631899107,2cyclictest50-21rcuop/307:50:003
631899107,2cyclictest28-21rcuop/111:10:153
631899107,2cyclictest10-21rcuop/009:30:203
631899107,2cyclictest0-21swapper/308:25:013
631899107,2cyclictest0-21swapper/308:25:003
631899106,3cyclictest28-21rcuop/109:25:173
631899106,3cyclictest16850irq/16-i801_smb12:27:593
631899101,8cyclictest22373-21perf08:55:013
631899100,9cyclictest47-21ksoftirqd/312:12:153
631899100,9cyclictest47-21ksoftirqd/312:03:403
631899100,9cyclictest47-21ksoftirqd/311:55:253
631899100,9cyclictest47-21ksoftirqd/311:40:163
631899100,9cyclictest47-21ksoftirqd/310:42:313
631899100,9cyclictest47-21ksoftirqd/309:35:143
631899100,9cyclictest47-21ksoftirqd/308:35:003
631899100,4cyclictest10-21rcuop/011:52:453
631799107,3cyclictest16850irq/16-i801_smb07:35:212
631799107,2cyclictest9549-21cstates10:10:172
631799106,4cyclictest16850irq/16-i801_smb08:25:222
631799106,4cyclictest16850irq/16-i801_smb08:12:152
631799106,3cyclictest16850irq/16-i801_smb08:05:272
631799106,3cyclictest16850irq/16-i801_smb07:12:052
631799105,4cyclictest16850irq/16-i801_smb08:17:172
631799102,7cyclictest0-21swapper/211:21:502
631799102,2cyclictest0-21swapper/209:37:072
631799100,9cyclictest0-21swapper/210:45:002
631699105,4cyclictest16850irq/16-i801_smb10:20:261
631699103,6cyclictest22711-21wget08:55:151
631699101,2cyclictest0-21swapper/109:50:251
631699100,5cyclictest234-21systemd-journal12:02:381
631599106,3cyclictest3-21ksoftirqd/008:55:190
631599106,3cyclictest3-21ksoftirqd/008:20:160
631599106,3cyclictest3-21ksoftirqd/008:20:150
631599102,1cyclictest0-21swapper/007:50:160
255612100,3sleep20-21swapper/211:50:022
63189997,1cyclictest0-21swapper/310:45:213
63189997,1cyclictest0-21swapper/307:25:003
63189996,2cyclictest50-21rcuop/311:35:223
63189996,2cyclictest50-21rcuop/310:05:153
63189996,2cyclictest50-21rcuop/308:40:163
63189996,2cyclictest2920-21grep08:10:223
63189996,2cyclictest28-21rcuop/107:35:003
63189996,2cyclictest28-21rcuop/107:15:013
63189994,3cyclictest22493-21munin-run07:45:003
63189994,1cyclictest6334-21/usr/sbin/munin07:10:243
63189994,1cyclictest4133-21/usr/sbin/munin08:15:173
63189994,1cyclictest32061-21ssh12:30:183
63189990,8cyclictest0-21swapper/309:00:193
63179996,3cyclictest16850irq/16-i801_smb08:45:262
63179996,3cyclictest16850irq/16-i801_smb07:45:232
63179996,3cyclictest16850irq/16-i801_smb07:40:252
63179996,2cyclictest271982sleep210:37:542
63179996,2cyclictest0-21swapper/211:45:152
63179995,4cyclictest16850irq/16-i801_smb09:15:222
63179995,4cyclictest16850irq/16-i801_smb08:35:252
63179995,3cyclictest57650irq/129-enp2s0-09:25:152
63179995,3cyclictest13465-21/usr/sbin/munin07:25:172
63179995,1cyclictest8358-21/usr/sbin/munin12:35:242
63179995,1cyclictest10330-21/usr/sbin/munin11:05:162
63179994,4cyclictest57550irq/128-enp2s0-11:30:362
63179994,4cyclictest57350irq/126-enp2s0-12:00:352
63179994,4cyclictest57350irq/126-enp2s0-09:58:042
63179994,4cyclictest16850irq/16-i801_smb07:30:302
63179993,5cyclictest57350irq/126-enp2s0-09:30:042
63179993,5cyclictest57350irq/126-enp2s0-08:30:262
63179993,5cyclictest16177-21sh11:45:002
63179993,3cyclictest57550irq/128-enp2s0-12:09:212
63179993,3cyclictest57550irq/128-enp2s0-11:15:362
63179992,6cyclictest72-21kswapd012:27:222
63179992,5cyclictest72-21kswapd010:53:222
63179992,5cyclictest0-21swapper/207:15:102
63179992,4cyclictest0-21swapper/209:45:112
63179991,5cyclictest0-21swapper/209:50:122
63179990,8cyclictest36-21ksoftirqd/212:22:592
63179990,5cyclictest0-21swapper/211:35:262
63179990,4cyclictest57350irq/126-enp2s0-12:15:012
63169997,1cyclictest0-21swapper/107:35:211
63169994,4cyclictest57550irq/128-enp2s0-12:20:141
63169994,4cyclictest16850irq/16-i801_smb11:45:251
63169994,1cyclictest28732-21/usr/sbin/munin09:45:201
63169993,5cyclictest57650irq/129-enp2s0-10:19:291
63169993,5cyclictest15439-21sendmail-msp08:40:001
63169993,1cyclictest21954-21sed12:05:271
63169991,4cyclictest14564-21kernelversion11:25:191
63169990,8cyclictest8414-21perf07:15:011
63169990,8cyclictest0-21swapper/109:10:181
63169990,7cyclictest234-21systemd-journal07:55:001
63169990,7cyclictest0-21swapper/107:15:181
63169990,5cyclictest0-21swapper/111:36:041
63169990,5cyclictest0-21swapper/111:31:281
63169990,5cyclictest0-21swapper/109:39:231
63169990,4cyclictest29104-21sort12:10:161
63169990,3cyclictest25007-21ssh10:36:201
31471290,1sleep20-21swapper/210:58:392
15303290,1sleep30-21swapper/308:38:593
63189986,1cyclictest0-21swapper/308:00:293
63189986,1cyclictest0-21swapper/307:57:143
63189981,6cyclictest27086-21ls07:55:003
63189981,5cyclictest9045-21grep08:25:153
63189980,7cyclictest0-21swapper/308:55:143
63189980,6cyclictest1561-21idleruntime-cro08:10:003
63179987,1cyclictest28787-21ssh10:20:252
63179985,3cyclictest16850irq/16-i801_smb07:55:252
63179984,4cyclictest16850irq/16-i801_smb08:52:232
63179984,4cyclictest16850irq/16-i801_smb07:20:252
63179984,1cyclictest57350irq/126-enp2s0-07:50:222
63179983,4cyclictest57650irq/129-enp2s0-11:25:442
63179982,5cyclictest57350irq/126-enp2s0-08:40:222
63179982,2cyclictest0-21swapper/211:13:122
63179981,6cyclictest0-21swapper/212:10:292
63179981,4cyclictest0-21swapper/210:15:242
63179980,7cyclictest6769-21tr08:20:162
63179980,7cyclictest6769-21tr08:20:162
63179980,7cyclictest0-21swapper/212:30:112
63179980,7cyclictest0-21swapper/210:30:162
63179980,7cyclictest0-21swapper/210:25:122
63179980,7cyclictest0-21swapper/210:05:232
63179980,6cyclictest29801-21ssh09:10:122
63169986,1cyclictest0-21swapper/108:50:231
63169986,1cyclictest0-21swapper/108:40:261
63169985,3cyclictest16850irq/16-i801_smb11:10:241
63169985,2cyclictest57450irq/127-enp2s0-08:20:151
63169985,2cyclictest57450irq/127-enp2s0-08:20:151
63169984,3cyclictest57350irq/126-enp2s0-09:05:131
63169983,4cyclictest4133-21/usr/sbin/munin08:15:241
63169982,0cyclictest0-21swapper/111:52:091
63169981,6cyclictest0-21swapper/111:00:181
63169980,7cyclictest21633-21grep07:40:231
63169980,7cyclictest0-21swapper/108:45:151
63169980,7cyclictest0-21swapper/107:25:151
63169980,6cyclictest945-21timerandwakeup08:05:271
63169980,6cyclictest1333-21runrttasks07:30:201
63169980,5cyclictest0-21swapper/110:45:471
63169980,4cyclictest556-21snmpd09:21:381
63189970,6cyclictest0-21swapper/307:35:123
63179974,2cyclictest0-21swapper/208:55:142
63179973,3cyclictest24636-21cron09:00:002
63179970,5cyclictest27335-21cut09:05:142
63169974,2cyclictest27328-21apache_processe07:55:131
63169972,4cyclictest9058-21tr08:25:141
63169970,6cyclictest11306-21aten2_r7power_v07:20:151
63169970,6cyclictest0-21swapper/108:10:151
63169970,6cyclictest0-21swapper/107:45:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional