vcloud-edge_gateway 0.2.2 → 0.2.3

data/CHANGELOG.md

@@ -1,3 +1,14 @@
+## 0.2.3 (2014-04-22)
+
+Bugfixes:
+
+  - Requires vCloud Core v0.0.12 which fixes issue with progress bar falling over when progress is not returned
+
+Features:
+
+  - Now uses the config loader and validator in vcloud-core rather than its own duplicate.
+  - Require fog v1.21 to allow use of FOG_VCLOUD_TOKEN via ENV as an alternative to a .fog file
+
 ## 0.2.2 (2014-03-05)
 
 Bugfixes:

data/README.md

@@ -25,15 +25,63 @@ To configure an Edge Gateway:
 
     $ vcloud-configure-edge input.yaml
 
+### Credentials
 
-## Contributing
+vCloud Edge Gateway is based around [fog]. To use it you'll need to give it credentials that allow it to talk to a VMware
+environment. Fog offers two ways to do this.
 
-1. Fork it
-2. Create your feature branch (`git checkout -b my-new-feature`)
-3. Commit your changes (`git commit -am 'Add some feature'`)
-4. Push to the branch (`git push origin my-new-feature`)
-5. Create new Pull Request
+#### 1. Create a `.fog` file containing your credentials
+
+To use this method, you need a `.fog` file in your home directory.
+
+For example:
+
+    test:
+      vcloud_director_username: 'username@org_name'
+      vcloud_director_password: 'password'
+      vcloud_director_host: 'host.api.example.com'
+
+Unfortunately current usage of fog requires the password in this file. Multiple sets of credentials can be specified in the fog file, using the following format:
+
+    test:
+      vcloud_director_username: 'username@org_name'
+      vcloud_director_password: 'password'
+      vcloud_director_host: 'host.api.example.com'
+
+    test2:
+      vcloud_director_username: 'username@org_name'
+      vcloud_director_password: 'password'
+      vcloud_director_host: 'host.api.vendor.net'
+
+You can then pass the `FOG_CREDENTIAL` environment variable at the start of your command. The value of the `FOG_CREDENTIAL` environment variable is the name of the credential set in your fog file which you wish to use.  For instance:
+
+    $ FOG_CREDENTIAL=test2 vcloud-configure-edge input.yaml
+
+To understand more about `.fog` files, visit the 'Credentials' section here => http://fog.io/about/getting_started.html.
+
+#### 2. Log on externally and supply your session token
+
+You can choose to log on externally by interacting independently with the API and supplying your session token to the
+tool by setting the `FOG_VCLOUD_TOKEN` ENV variable. This option reduces the risk footprint by allowing the user to
+store their credentials in safe storage. The default token lifetime is '30 minutes idle' - any activity extends the life by another 30 mins.
+
+A basic example of this would be the following:
+
+    curl
+       -D-
+       -d ''
+       -H 'Accept: application/*+xml;version=5.1' -u '<user>@<org>'
+       https://host.com/api/sessions
+
+This will prompt for your password.
 
+From the headers returned, select the header below
+
+     x-vcloud-authorization: AAAABBBBBCCCCCCDDDDDDEEEEEEFFFFF=
+
+Use token as ENV var FOG_VCLOUD_TOKEN
+
+    $ FOG_VCLOUD_TOKEN=AAAABBBBBCCCCCCDDDDDDEEEEEEFFFFF= vcloud-configure-edge input.yaml
 
 ### Configure edge gateway services
 
@@ -93,6 +141,8 @@ Rule fields have the following behaviour
   * A CIDR range, eg `192.0.2.0/24`
   * A hyphened range, such as `192.0.2.50-192.0.2.60`
 
+
+
 #### nat_service
 
 The edge gateway NAT service offers simple stateful Source-NAT and
@@ -148,7 +198,7 @@ A DNAT rule has the following form, and translates packets going to the
 #### load_balancer_service
 
 The load balancer service comprises two sets of configurations: 'pools' and
-'virtual_servers'. These are coupled together to form a load balanced service:
+'virtual_servers'. These are coupled together to form load balanced services:
 
 * A virtual_server provides the front-end of a load balancer - the port and
   IP that clients connect to.
@@ -158,8 +208,13 @@ The load balancer service comprises two sets of configurations: 'pools' and
   it.
 * Multiple virtual_servers can specify the same pool (to run the same service
   on different FQDNs, for example)
+* virtual_servers define any 'session persistence' information, if sessions
+  are required to stick to the same pool member. (Session persistence is not currently supported by this tool.)
+* pools define 'member healthchecks', and so are aware of the health of their
+  member nodes.
 
-A typical load balancer configuration (for one service) would look something like:
+A typical load balancer configuration (for one service, mapping 192.0.2.0:80 to
+port 8080 on three servers) would look something like:
 
 ```
 load_balancer_service:
@@ -183,9 +238,146 @@ load_balancer_service:
     pool: 'example-pool-1' # must refer to a pool name detailed above
     service_profiles:
       http:  # protocol to balance, can be tcp/http/https.
-      port: '80'  # external port
+        port: '80'  # external port
+```
+
+The vCloud Director load balancer service is quite basic, but supports the following:
+
+* Layer 7 balancing of HTTP traffic
+* Balancing of HTTPS traffic (though no decryption is possible, so this is
+  purely level-4 based)
+* Layer 4 balancing of arbitrary TCP traffic.
+* URI-based healthchecks of backend nodes
+* Several balancing algorithms, such as 'round robin', and 'least connections'
+* Ability to persist sessions to the same backend member node, via a variety of
+  means (eg HTTP cookie value, SSL session ID, source IP hash).
+
+`vcloud-configure-edge` supports all of the above features.
+
+It is also worth noting that the vCloud Director load balancer *does not support*:
+
+* In vCD 5.1, TCP and HTTPS layer-4 balancing are based on TCP port forwarding.
+  There is no NAT in the mix, so the backend pools see the IP address/port of
+  the edge rather than the remote host.
+* There is no SSL offloading/decryption possible on the device, so traffic
+  inspection of HTTPS is not feasible.
+
+Rather unusually, each virtual server and pool combination can handle traffic
+balancing for HTTP, HTTPS, and a single TCP port simultaneously. For example:
+
+```
+load_balancer_service:
+  pools:
+  - name: 'example-multi-protocol-pool-1'
+    description: 'A pool balancing HTTP, HTTPS, and SMTP traffic'
+    service:
+      http: {}
+      https: {}
+      tcp:
+        port: 25
+    members:
+    - ip_address: 10.10.10.14
+    - ip_address: 10.10.10.15
+  virtual_servers:
+  - name: 'example-multi-protocol-virtual-server-1'
+    description: 'A virtual server connecting to example-pool-1'
+    ip_address: 192.0.2.11
+    network: '12345678-1234-1234-1234-123456789012'
+    pool: 'example-multi-protocol-pool-1'
+    service_profiles:
+      http: {}
+      https: {}
+      tcp:
+        port: 25
+```
+
+The above is particularly useful for services that require balancing of HTTP
+and HTTPS traffic together.
+
+#### load_balancer_service pool entries in detail
+
+Each pool entry consists of:
+
+* a pool name, and optional description
+* a 'service' section - which protocol(s) to balance, and how to balance them.
+* a 'members' list - which backend nodes to use.
+
+For example:
+
+```
+name: test-pool-1
+description: Balances HTTP and HTTPS
+service:
+  http: {}
+  https: {}
+members:
+- ip_address: 10.10.10.11
+- ip_address: 10.10.10.12
+  weight: 10
 ```
 
+Here we have:
+
+* HTTP and HTTPS traffic balanced across 10.10.10.11 and 10.10.10.12.
+* member 10.10.10.11 has a default `weight` of 1
+* member 10.10.10.12 has a `weight` of 10, so will receive 10x the traffic of
+  10.10.10.11
+* http and https services are using all defaults, which means:
+  * they use standard ports (80 for HTTP, 443 for HTTPS)
+  * they will use 'round robin' balancing
+  * HTTP service will 'GET /' from each node to check its health
+  * HTTPS service will check 'SSL hello' response to confirm its health.
+
+Service entries are the most complex, due to the available options on
+a per-service basis. The defaults we provide are suitable for most situations,
+but for more infomation see below.
+
+A more complete HTTP service entry looks like:
+
+```
+service:
+  http:
+    port: 8080
+    algorithm: 'ROUND_ROBIN'  # can also be 'LEAST_CONNECTED', 'IP_HASH', 'URI'
+    health_check:
+      port: 8081            # port to check health on, if not service port above.
+      uri: /healthcheck     # for HTTP, the URI to check for 200/30* response
+      protocol: HTTP        # the protocol to talk to health check service: HTTP, SSL, TCP
+      health_threshold:  2  # how many checks to success before reenabling member
+      unhealth_threshold: 3 # how many checks to fail before disabling member
+      interval: 5           # interval between checks
+      timeout: 15           # how long to wait before assuming healthcheck has failed
+
+```
+
+See [the vCloud Director Admin Guide](http://pubs.vmware.com/vcd-51/topic/com.vmware.vcloud.admin.doc_51/GUID-C12B3954-155F-48AF-9855-E0DE026752D0.html)
+for more details on configuring Pool entries.
+
+#### load_balancer_service virtual_server entries in detail
+
+Each virtual_server entry must consist of:
+
+* a virtual_server name, and optional description
+* a 'service_profiles' section: which protocol(s) to handle
+* a `network` reference - the UUID of the network which the ip_address sits on.
+* a backend `pool` to use, referenced by name
+
+For example:
+
+```
+name: test-virtual_server-1
+description: Public facing side of test-pool-1
+pool: test-pool-1
+ip_address: 192.0.2.55  # front-end IP address, usually external
+network: 12345678-1234-1234-1234-1234567890aa # UUID of network containing ip_address
+service_profiles:
+  http: { port: 8080 } # override default port 80
+  https: { }  # port defaults to 443
+```
+
+See [the vCloud Director Admin Guide](http://pubs.vmware.com/vcd-51/topic/com.vmware.vcloud.admin.doc_51/GUID-EC5EE5F9-1A2C-4609-9347-4C3143727704.html)
+for more details on configuring VirtualServer entries.
+
 ### Finding external network details from vcloud-walk
 
 You can find the network UUID and external address allocations using [vCloud
@@ -218,12 +410,28 @@ cat edges.out | jq '
       '
 ```
 
+### Full configuration examples
+
+You can find full configuration examples in the `examples` folder.
 
 
-### Debug output
+## Debugging
 
-Set environment variable `DEBUG=true` and/or `EXCON_DEBUG=true` to see Fog debug info.
+`export EXCON_DEBUG=true` - this will print out the API requests and responses.
+
+`export DEBUG=true` - this will show you the stack trace when there is an exception instead of just the message.
 
 ### References
 
 * [vCloud Director Edge Gateway documentation](http://pubs.vmware.com/vcd-51/topic/com.vmware.vcloud.admin.doc_51/GUID-ADE1DCAB-874F-45A9-9337-1E971DAC0F7D.html)
+
+## Contributing
+
+1. Fork it
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create new Pull Request
+
+
+[fog]: http://fog.io/

data/examples/firewall-rules.yaml

@@ -0,0 +1,50 @@
+# Example configuration file for defining Firewall rules on the vShield Edge Gateway
+#
+# Note that applying this configuration file will replace the current FW rules on the vShield Edge Gateway.
+#
+# Quick tips:
+# gateway: should contain the name of the edge gateway (human readable form, not the UUID)
+#
+---
+gateway: "My gateway name"
+firewall_service:
+  enabled: true
+  policy: 'drop'
+  log_default_action: true
+  firewall_rules:
+
+  - :id: '1'
+    :enabled: true
+    :match_on_translate: false
+    :description: 'allow_ssh'
+    :policy: 'allow'
+    :protocols: 'tcp'
+    :destination_port_range: '22'
+    :destination_ip: '00.01.02.03'
+    :source_port_range: 'Any'
+    :source_ip: 'Any'
+    :enable_logging: true
+
+  - :id: '2'
+    :enabled: true
+    :match_on_translate: false
+    :description: 'allow_access_to_good_sunny_days'
+    :policy: 'allow'
+    :protocols: 'tcp'
+    :destination_port_range: 'Any'
+    :destination_ip: 'external'
+    :source_port_range: 'Any'
+    :source_ip: '12.18.0.0/24'
+    :enable_logging: true
+
+  - :id: '6'
+    :enabled: true
+    :match_on_translate: false
+    :description: 'network_to_internet_ICMP'
+    :policy: 'allow'
+    :protocols: "icmp"
+    :destination_port_range: 'Any'
+    :destination_ip: 'external'
+    :source_port_range: 'Any'
+    :source_ip: '12.16.0.0/24'
+    :enable_logging: true

data/examples/loadbalancer-rules.yaml

@@ -0,0 +1,55 @@
+# Example configuration file for defining Load Balancing rules on the vShield Edge Gateway
+#
+# Note that applying this configuration file will replace the current Load Balancing rules on the vShield Edge Gateway.
+#
+# Quick tips:
+# gateway: should contain the name of the edge gateway (human readable form, not the UUID)
+# network: should contain the UUID for the internet facing network (not the internal ones).
+#          You can find the UUID for the vcloud network object using vcloud-walker.
+#
+---
+gateway: "My gateway name"
+load_balancer_service:
+  pools:
+    - name: 'pool-001'
+      description: 'pool for balancing http(s)"
+      service:
+        http:
+          port: 80
+          algorithm: 'IP_HASH'
+          health_check:
+            port: 80
+            uri: /
+            protocol: HTTP
+            health_threshold: 2
+            unhealth_threshold: 3
+            interval: 5
+            timeout: 15
+        https:
+          port: 443
+          algorithm: 'IP_HASH'
+          health_check:
+            port: 443
+            uri: /
+            protocol: SSL
+            health_threshold: 2
+            unhealth_threshold: 3
+            interval: 5
+            timeout: 15
+      members:
+        - ip_address: 192.168.1.1
+        - ip_address: 192.168.1.2
+        - ip_address: 192.168.1.3
+
+  virtual_servers:
+    - name: 'vs-website'
+      description: 'virtual server for my website"
+      ip_address: 20.20.20.20
+      network: '00000000-1111-2222-3333-444444444444'
+      pool: 'pool-001'
+      service_profiles:
+        http:
+          port: '80'
+        https:
+          port: '443'
+

data/examples/nat-rules.yaml

@@ -0,0 +1,58 @@
+# Example configuration file for defining NAT rules on the vShield Edge Gateway
+#
+# Note that applying this configuration file will replace the current SNAT/DNAT rules on the vShield Edge Gateway.
+#
+# Quick tips:
+# gateway: should contain the name of the edge gateway (human readable form, not the UUID)
+#
+# network_id: you can find the UUID for the vcloud network object using vcloud-walker
+#
+---
+gateway: "My gateway name"
+nat_service:
+  enabled: true
+  nat_rules:
+
+  - :id: '65537'
+    :enabled: true
+    :rule_type: 'DNAT'
+    :network_id: '00000000-1111-2222-3333-444444444444'
+    :original_ip: '00.01.02.03'
+    :original_port: '22'
+    :translated_ip: '10.20.30.40'
+    :translated_port: '22'
+    :protocol: 'tcp'
+
+  - :id: '65538'
+    :enabled: true
+    :rule_type: 'DNAT'
+    :network_id: '00000000-1111-2222-3333-444444444444'
+    :original_ip: '00.01.02.03'
+    :original_port: '80'
+    :translated_ip: '10.20.30.40'
+    :translated_port: '80'
+    :protocol: 'tcp'
+
+  - :id: '65539'
+    :enabled: true
+    :rule_type: 'DNAT'
+    :network_id: '00000000-1111-2222-3333-444444444444'
+    :original_ip: '00.01.02.03'
+    :original_port: '443'
+    :translated_ip: '10.20.30.40'
+    :translated_port: '443'
+    :protocol: 'tcp'
+
+  - :id: '65540'
+    :enabled: true
+    :rule_type: 'SNAT'
+    :network_id: '00000000-1111-2222-3333-444444444444'
+    :original_ip: '90.100.110.0/24'
+    :translated_ip: '00.01.02.03'
+
+  - :id: '65541'
+    :enabled: true
+    :rule_type: 'SNAT'
+    :network_id: '00000000-1111-2222-3333-444444444444'
+    :original_ip: '10.20.30.40/24'
+    :translated_ip: '00.01.02.03'