dev-lxc 3.2.0 → 3.3.0

data/example-clusters/README.md

@@ -1,35 +1,165 @@
-The files in this directory are working examples of cluster dev-lxc.yml files. You can upload them to separate directories in your dev-lxc-platform instance, rename each file to `dev-lxc.yml` and run `dl up` to build each cluster.
-
-### automate
-
-Cluster build time: 19 minutes
-
-The [automate_dev-lxc.yml](automate_dev-lxc.yml) file was created using the following command.
-
-```
-dl init --chef --compliance --supermarket --automate --runners --nodes --product-versions reporting:none
-```
-
-### chef-backend
-
-Cluster build time: 12 minutes
-
-The [chef-backend_dev-lxc.yml](chef-backend_dev-lxc.yml) file was created using the following command.
-
-The nodes' `chef_server_url` hostname was changed to `chef-ha.lxc` and the node's hostname was changed to `node-1-ha.lxc`.
-
-```
-dl init --chef-backend --nodes
-```
-
-### tier
-
-Cluster build time: 14 minutes
-
-The [tier_dev-lxc.yml](tier_dev-lxc.yml) file was created using the following command.
-
-The nodes' `chef_server_url` hostname was changed to `chef-tier.lxc` and the node's hostname was changed to `node-1-tier.lxc`.
-
-```
-dl init --chef-tier --analytics --nodes
-```
+The files in this directory are working examples of cluster dev-lxc.yml files. You can upload them to separate directories in your dev-lxc-platform instance, rename each file to `dev-lxc.yml` and run `dl up` to build each cluster.
+
+## automate
+
+Cluster build time: 19 minutes
+
+Download the [automate_dev-lxc.yml](automate_dev-lxc.yml) file to the dev-lxc-platform filesystem.
+
+The `automate` cluster needs the `delivery.license` file copied to an appropriate location on dev-lxc-platform's filesystem.
+
+The following is an example of `dl status` for this cluster.
+
+```
+chef.lxc            RUNNING         10.0.3.203
+  |_ snap0 2017:03:08 19:32:08 dev-lxc build: products installed
+  |_ snap1 2017:03:08 19:50:24 dev-lxc build: completed
+
+compliance.lxc      RUNNING         10.0.3.205
+  |_ snap0 2017:03:08 19:32:39 dev-lxc build: products installed
+  |_ snap1 2017:03:08 19:50:20 dev-lxc build: completed
+
+supermarket.lxc     RUNNING         10.0.3.206
+  |_ snap0 2017:03:08 19:33:15 dev-lxc build: products installed
+  |_ snap1 2017:03:08 19:50:18 dev-lxc build: completed
+
+automate.lxc        RUNNING         10.0.3.200
+  |_ snap0 2017:03:08 19:33:55 dev-lxc build: products installed
+  |_ snap1 2017:03:08 19:50:14 dev-lxc build: completed
+
+runner-1.lxc        RUNNING         10.0.3.87
+  |_ snap0 2017:03:08 19:50:02 dev-lxc build: completed
+
+node-1.lxc          RUNNING         10.0.3.90
+  |_ snap0 2017:03:08 19:34:16 dev-lxc build: products installed
+  |_ snap1 2017:03:08 19:50:01 dev-lxc build: completed
+```
+
+The following command was used to create the `automate_dev-lxc.yml` file.
+
+```
+dl init --chef --compliance --supermarket --automate --runners --nodes --product-versions reporting:none
+```
+
+## chef-backend
+
+Cluster build time: 12 minutes
+
+Download the [chef-backend_dev-lxc.yml](chef-backend_dev-lxc.yml) file to the dev-lxc-platform filesystem.
+
+The following is an example of `dl status` for this cluster.
+
+```
+Chef Server FQDN: chef-ha.lxc
+
+chef-backend1.lxc      RUNNING         10.0.3.208
+  |_ snap0 2017:03:13 20:19:03 dev-lxc build: products installed
+  |_ snap1 2017:03:13 20:27:27 dev-lxc build: backend cluster configured but frontend not bootstrapped
+  |_ snap2 2017:03:13 20:33:41 dev-lxc build: completed
+
+chef-backend2.lxc      RUNNING         10.0.3.209
+  |_ snap0 2017:03:13 20:19:39 dev-lxc build: products installed
+  |_ snap1 2017:03:13 20:27:24 dev-lxc build: backend cluster configured but frontend not bootstrapped
+  |_ snap2 2017:03:13 20:33:30 dev-lxc build: completed
+
+chef-backend3.lxc      RUNNING         10.0.3.210
+  |_ snap0 2017:03:13 20:20:16 dev-lxc build: products installed
+  |_ snap1 2017:03:13 20:27:22 dev-lxc build: backend cluster configured but frontend not bootstrapped
+  |_ snap2 2017:03:13 20:33:26 dev-lxc build: completed
+
+chef-frontend1.lxc     RUNNING         10.0.3.211
+  |_ snap0 2017:03:13 20:21:32 dev-lxc build: products installed
+  |_ snap1 2017:03:13 20:33:23 dev-lxc build: completed
+
+node-1-ha.lxc          RUNNING         10.0.3.87
+  |_ snap0 2017:03:13 20:22:52 dev-lxc build: products installed
+  |_ snap1 2017:03:13 20:33:16 dev-lxc build: completed
+```
+
+The following command and described modifications were used to create the `chef-backend_dev-lxc.yml` file.
+
+```
+dl init --chef-backend --nodes
+```
+
+The nodes' `chef_server_url` hostname was changed to `chef-ha.lxc` and the node's hostname was changed to `node-1-ha.lxc`.
+
+## tier
+
+Cluster build time: 14 minutes
+
+Download the [tier_dev-lxc.yml](tier_dev-lxc.yml) file to the dev-lxc-platform filesystem.
+
+The following is an example of `dl status` for this cluster.
+
+```
+Chef Server FQDN: chef-tier.lxc
+
+chef-be.lxc         RUNNING         10.0.3.201
+  |_ snap0 2017:03:07 21:48:04 dev-lxc build: products installed
+  |_ snap1 2017:03:07 22:00:45 dev-lxc build: completed
+
+chef-fe1.lxc        RUNNING         10.0.3.202
+  |_ snap0 2017:03:07 21:49:16 dev-lxc build: products installed
+  |_ snap1 2017:03:07 22:00:42 dev-lxc build: completed
+
+analytics.lxc       RUNNING         10.0.3.204
+  |_ snap0 2017:03:07 21:49:34 dev-lxc build: products installed
+  |_ snap1 2017:03:07 22:00:38 dev-lxc build: completed
+
+node-1-tier.lxc     RUNNING         10.0.3.87
+  |_ snap0 2017:03:07 21:49:48 dev-lxc build: products installed
+  |_ snap1 2017:03:07 22:00:26 dev-lxc build: completed
+```
+
+The following command and described modifications were used to create the `tier_dev-lxc.yml` file.
+
+```
+dl init --chef-tier --analytics --nodes
+```
+
+The nodes' `chef_server_url` hostname was changed to `chef-tier.lxc` and the node's hostname was changed to `node-1-tier.lxc`.
+
+## external
+
+Cluster build time: 5 minutes
+
+Download the [external_dev-lxc.yml](external_dev-lxc.yml) file to the dev-lxc-platform filesystem.
+
+The `external` cluster needs the [postgres-partial.rb](conf-files/chef-server/postgres-partial.rb), [elasticsearch-partial.rb](conf-files/chef-server/elasticsearch-partial.rb) and [ldap-partial.rb](conf-files/chef-server/ldap-partial.rb) files copied to an appropriate location on dev-lxc-platform's filesystem.
+
+Run the following commands as the dev-lxc-platform's root user to create new external postgres, elasticsearch and ldap servers for the `external` cluster.
+
+```
+docker rm my-postgres -f
+docker rm my-elasticsearch -f
+docker rm my-ldap -f
+
+docker run --name my-postgres -d -p 5432:5432 -e POSTGRES_PASSWORD=mysecretpassword postgres
+docker run --name my-elasticsearch -d -p 9200:9200 -e "http.host=0.0.0.0" -e "transport.host=127.0.0.1" elasticsearch:2.3
+docker run --name my-ldap -d -p 389:389 -e SLAPD_PASSWORD=mysecretpassword -e SLAPD_DOMAIN=ldap.example.org dinkel/openldap
+```
+
+The following is an example of `dl status` for this cluster.
+
+```
+chef-external.lxc       RUNNING         10.0.3.233
+  |_ snap0 2017:03:14 19:51:33 dev-lxc build: products installed
+  |_ snap1 2017:03:14 19:55:23 dev-lxc build: completed
+
+node-1-external.lxc     RUNNING         10.0.3.87
+  |_ snap0 2017:03:14 19:52:08 dev-lxc build: products installed
+  |_ snap1 2017:03:14 19:55:11 dev-lxc build: completed
+```
+
+The following command and described modifications were used to create the `external_dev-lxc.yml` file.
+
+```
+dl init --chef --nodes --product-versions push-jobs-server:none reporting:none
+```
+
+The Chef server's hostname was changed to `chef-external.lxc` and its IP address was changed to `10.0.3.233`
+
+Paths to the `postgres-partial.rb`, `elasticsearch-partial.rb` and `ldap-partial.rb` files were added to the Chef server's `chef-server.rb_partials` list.
+
+The nodes' `chef_server_url` hostname was changed to `chef-external.lxc` and the node's hostname was changed to `node-1-external.lxc`.

data/example-clusters/automate_dev-lxc.yml

@@ -1,98 +1,98 @@
-# enable_build_snapshots automatically makes container snapshots at key times during the build process
-# default value is `true`
-#enable_build_snapshots: true
-
-# base_container must be the name of an existing container
-base_container: b-ubuntu-1404
-
-# memory_per_server sets the maximum amount of user memory (including file cache) for each server.
-# dev-lxc will set the `memory.limit_in_bytes` cgroup for each server to apply this limit.
-# If no units are specified, the value is interpreted as bytes.
-# You can use suffixes to represent larger units - k or K for kilobytes, m or M for megabytes, and g or G for gigabytes.
-# The default behavior is that no limit is set.
-#memory_per_server: 4G
-
-# list any host directories you want mounted into the servers
-#mounts:
-#  - /root/clusters root/clusters
-
-# list any SSH public keys you want added to /home/dev-lxc/.ssh/authorized_keys
-#ssh-keys:
-#  - /root/clusters/id_rsa.pub
-
-# DHCP reserved (static) IPs must be selected from the IP range 10.0.3.150 - 254
-
-chef-server:
-  users:          # a user's password will be the same as its username
-    - mary-admin
-    - joe-user
-  orgs:
-    demo:
-      admins:
-        - mary-admin
-      non-admins:
-        - joe-user
-  servers:
-    chef.lxc:
-      ipaddress: 10.0.3.203
-      products:
-        chef-server:
-          channel: stable
-          version: latest
-        manage:
-          channel: stable
-          version: latest
-        push-jobs-server:
-          channel: stable
-          version: latest
-
-compliance:
-  admin_user: admin         # the password will be the same as the username
-  servers:
-    compliance.lxc:
-      ipaddress: 10.0.3.205
-      products:
-        compliance:
-          channel: stable
-          version: latest
-
-supermarket:
-  servers:
-    supermarket.lxc:
-      ipaddress: 10.0.3.206
-      products:
-        supermarket:
-          channel: stable
-          version: latest
-
-automate:
-  servers:
-    automate.lxc:
-      ipaddress: 10.0.3.200
-      products:
-        automate:
-          channel: stable
-          version: latest
-      license_path: ../delivery.license
-      chef_org: delivery
-      enterprise_name: demo-ent
-
-runners:
-  servers:
-    runner-1.lxc:
-      products:
-        chefdk:     # downloaded only
-          channel: stable
-          version: latest
-
-nodes:
-  chef_server_url: https://chef.lxc/organizations/demo
-  validation_client_name: demo-validator
-  # comment out or remove the validation_key path to use chef-server keys generated by dev-lxc
-  validation_key: # /path/for/ORG-validator.pem
-  servers:
-    node-1.lxc:
-      products:
-        chef:
-          channel: stable
-          version: latest
+# enable_build_snapshots automatically makes container snapshots at key times during the build process
+# default value is `true`
+#enable_build_snapshots: true
+
+# base_container must be the name of an existing container
+base_container: b-ubuntu-1404
+
+# memory_per_server sets the maximum amount of user memory (including file cache) for each server.
+# dev-lxc will set the `memory.limit_in_bytes` cgroup for each server to apply this limit.
+# If no units are specified, the value is interpreted as bytes.
+# You can use suffixes to represent larger units - k or K for kilobytes, m or M for megabytes, and g or G for gigabytes.
+# The default behavior is that no limit is set.
+#memory_per_server: 4G
+
+# list any host directories you want mounted into the servers
+#mounts:
+#  - /root/clusters root/clusters
+
+# list any SSH public keys you want added to /home/dev-lxc/.ssh/authorized_keys
+#ssh-keys:
+#  - /root/clusters/id_rsa.pub
+
+# DHCP reserved (static) IPs must be selected from the IP range 10.0.3.150 - 254
+
+chef-server:
+  users:          # a user's password will be the same as its username
+    - mary-admin
+    - joe-user
+  orgs:
+    demo:
+      admins:
+        - mary-admin
+      non-admins:
+        - joe-user
+  servers:
+    chef.lxc:
+      ipaddress: 10.0.3.203
+      products:
+        chef-server:
+          channel: stable
+          version: latest
+        manage:
+          channel: stable
+          version: latest
+        push-jobs-server:
+          channel: stable
+          version: latest
+
+compliance:
+  admin_user: admin         # the password will be the same as the username
+  servers:
+    compliance.lxc:
+      ipaddress: 10.0.3.205
+      products:
+        compliance:
+          channel: stable
+          version: latest
+
+supermarket:
+  servers:
+    supermarket.lxc:
+      ipaddress: 10.0.3.206
+      products:
+        supermarket:
+          channel: stable
+          version: latest
+
+automate:
+  servers:
+    automate.lxc:
+      ipaddress: 10.0.3.200
+      products:
+        automate:
+          channel: stable
+          version: latest
+      license_path: ../delivery.license
+      chef_org: delivery
+      enterprise_name: demo-ent
+
+runners:
+  servers:
+    runner-1.lxc:
+      products:
+        chefdk:     # downloaded only
+          channel: stable
+          version: latest
+
+nodes:
+  chef_server_url: https://chef.lxc/organizations/demo
+  validation_client_name: demo-validator
+  # comment out or remove the validation_key path to use chef-server keys generated by dev-lxc
+  validation_key: # /path/for/ORG-validator.pem
+  servers:
+    node-1.lxc:
+      products:
+        chef:
+          channel: stable
+          version: latest

data/example-clusters/chef-backend_dev-lxc.yml

@@ -1,81 +1,81 @@
-# enable_build_snapshots automatically makes container snapshots at key times during the build process
-# default value is `true`
-#enable_build_snapshots: true
-
-# base_container must be the name of an existing container
-base_container: b-ubuntu-1404
-
-# memory_per_server sets the maximum amount of user memory (including file cache) for each server.
-# dev-lxc will set the `memory.limit_in_bytes` cgroup for each server to apply this limit.
-# If no units are specified, the value is interpreted as bytes.
-# You can use suffixes to represent larger units - k or K for kilobytes, m or M for megabytes, and g or G for gigabytes.
-# The default behavior is that no limit is set.
-#memory_per_server: 4G
-
-# list any host directories you want mounted into the servers
-#mounts:
-#  - /root/clusters root/clusters
-
-# list any SSH public keys you want added to /home/dev-lxc/.ssh/authorized_keys
-#ssh-keys:
-#  - /root/clusters/id_rsa.pub
-
-# DHCP reserved (static) IPs must be selected from the IP range 10.0.3.150 - 254
-
-chef-backend:
-  api_fqdn: chef-ha.lxc
-  users:          # a user's password will be the same as its username
-    - mary-admin
-    - joe-user
-  orgs:
-    demo:
-      admins:
-        - mary-admin
-      non-admins:
-        - joe-user
-  servers:
-    chef-backend1.lxc:
-      ipaddress: 10.0.3.208
-      role: backend
-      leader: true
-      products:
-        chef-backend:
-          channel: stable
-          version: latest
-    chef-backend2.lxc:
-      ipaddress: 10.0.3.209
-      role: backend
-      products:
-        chef-backend:
-          channel: stable
-          version: latest
-    chef-backend3.lxc:
-      ipaddress: 10.0.3.210
-      role: backend
-      products:
-        chef-backend:
-          channel: stable
-          version: latest
-    chef-frontend1.lxc:
-      ipaddress: 10.0.3.211
-      role: frontend
-      bootstrap: true
-      products:
-        chef-server:
-          channel: stable
-          version: latest
-        manage:
-          channel: stable
-          version: latest
-
-nodes:
-  chef_server_url: https://chef-ha.lxc/organizations/demo
-  validation_client_name: demo-validator
-  # comment out or remove the validation_key path to use chef-server keys generated by dev-lxc
-  validation_key: # /path/for/ORG-validator.pem
-  servers:
-    node-1-ha.lxc:
-      products:
-        chef:
-          channel: stable
-          version: latest
+# enable_build_snapshots automatically makes container snapshots at key times during the build process
+# default value is `true`
+#enable_build_snapshots: true
+
+# base_container must be the name of an existing container
+base_container: b-ubuntu-1404
+
+# memory_per_server sets the maximum amount of user memory (including file cache) for each server.
+# dev-lxc will set the `memory.limit_in_bytes` cgroup for each server to apply this limit.
+# If no units are specified, the value is interpreted as bytes.
+# You can use suffixes to represent larger units - k or K for kilobytes, m or M for megabytes, and g or G for gigabytes.
+# The default behavior is that no limit is set.
+#memory_per_server: 4G
+
+# list any host directories you want mounted into the servers
+#mounts:
+#  - /root/clusters root/clusters
+
+# list any SSH public keys you want added to /home/dev-lxc/.ssh/authorized_keys
+#ssh-keys:
+#  - /root/clusters/id_rsa.pub
+
+# DHCP reserved (static) IPs must be selected from the IP range 10.0.3.150 - 254
+
+chef-backend:
+  api_fqdn: chef-ha.lxc
+  users:          # a user's password will be the same as its username
+    - mary-admin
+    - joe-user
+  orgs:
+    demo:
+      admins:
+        - mary-admin
+      non-admins:
+        - joe-user
+  servers:
+    chef-backend1.lxc:
+      ipaddress: 10.0.3.208
+      role: backend
+      leader: true
+      products:
+        chef-backend:
+          channel: stable
+          version: latest
+    chef-backend2.lxc:
+      ipaddress: 10.0.3.209
+      role: backend
+      products:
+        chef-backend:
+          channel: stable
+          version: latest
+    chef-backend3.lxc:
+      ipaddress: 10.0.3.210
+      role: backend
+      products:
+        chef-backend:
+          channel: stable
+          version: latest
+    chef-frontend1.lxc:
+      ipaddress: 10.0.3.211
+      role: frontend
+      bootstrap: true
+      products:
+        chef-server:
+          channel: stable
+          version: latest
+        manage:
+          channel: stable
+          version: latest
+
+nodes:
+  chef_server_url: https://chef-ha.lxc/organizations/demo
+  validation_client_name: demo-validator
+  # comment out or remove the validation_key path to use chef-server keys generated by dev-lxc
+  validation_key: # /path/for/ORG-validator.pem
+  servers:
+    node-1-ha.lxc:
+      products:
+        chef:
+          channel: stable
+          version: latest

data/example-clusters/conf-files/chef-server/elasticsearch-partial.rb

@@ -0,0 +1,38 @@
+
+# Chef Server doesn't work properly with elasticsearch 5.x but i'm keeping the following info here for reference anyway
+# reference: https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html
+# docker run --name my-elasticsearch -d -p 9200:9200 -e "http.host=0.0.0.0" -e "transport.host=127.0.0.1" -e "xpack.security.enabled=false docker.elastic.co/elasticsearch/elasticsearch:5.2.2
+
+
+# Chef Server works with elasticsearch 2.3 so use the following docker command to create an elasticsearch instance
+# reference: https://hub.docker.com/r/library/elasticsearch/
+# docker run --name my-elasticsearch -d -p 9200:9200 -e "http.host=0.0.0.0" -e "transport.host=127.0.0.1" elasticsearch:2.3
+
+
+# reference: https://github.com/chef/chef-server/blob/master/PRIOR_RELEASE_NOTES.md#elasticsearch-search-indexing
+# These settings ensure that we use remote elasticsearch
+# instead of local solr for search.  This also
+# set search_queue_mode to 'batch' to remove the indexing
+# dependency on rabbitmq, which is not supported in this HA configuration.
+opscode_solr4['external'] = true
+opscode_solr4['external_url'] = 'http://10.0.3.1:9200'
+opscode_erchef['search_provider'] = 'elasticsearch'
+opscode_erchef['search_queue_mode'] = 'batch'
+
+# RabbitMQ settings
+
+# Disable rabbit backend. Note that this makes
+# this incompatible with reporting and analytics unless you're bringing in
+# an external rabbitmq.
+rabbitmq['enable'] = false
+rabbitmq['management_enabled'] = false
+rabbitmq['queue_length_monitor_enabled'] = false
+
+# Opscode Expander
+#
+# opscode-expander isn't used when the search_queue_mode is batch.  It
+# also doesn't support the elasticsearch backend.
+opscode_expander['enable'] = false
+
+# Prevent startup failures due to missing rabbit host
+dark_launch['actions'] = false

data/example-clusters/conf-files/chef-server/ldap-partial.rb

@@ -0,0 +1,10 @@
+
+# reference: https://hub.docker.com/r/dinkel/openldap/
+# docker run --name my-ldap -d -p 389:389 -e SLAPD_PASSWORD=mysecretpassword -e SLAPD_DOMAIN=ldap.example.org dinkel/openldap
+
+# reference: https://docs.chef.io/server_ldap.html
+ldap['base_dn'] = 'DC=ldap,DC=example,DC=org'
+ldap['bind_dn'] = 'CN=admin,DC=ldap,DC=example,DC=org'
+ldap['bind_password'] = 'mysecretpassword'
+ldap['host'] = '10.0.3.1'
+ldap['login_attribute'] = 'cn'

data/example-clusters/conf-files/chef-server/postgres-partial.rb

@@ -0,0 +1,14 @@
+
+# reference: https://hub.docker.com/_/postgres/
+# docker run --name my-postgres -d -p 5432:5432 -e POSTGRES_PASSWORD=mysecretpassword postgres
+
+# reference:
+# https://docs.chef.io/server_components.html#external-postgresql
+# https://github.com/chef/chef-server/blob/master/PRIOR_RELEASE_NOTES.md#chef-server-5
+# Specify that postgresql is an external database, and provide the
+# VIP of this cluster.  This prevents the chef-server instance
+# from creating it's own local postgresql instance.
+postgresql['external'] = true
+postgresql['vip'] = '10.0.3.1'
+postgresql['db_superuser'] = 'postgres'
+postgresql['db_superuser_password'] = 'mysecretpassword'