dev-lxc 2.4.0 → 2.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: c4d56770910d2c693b886f65c3b233d8aeca752a
-  data.tar.gz: b0e22969f25e6e3def0807322f5041ca51482bb8
+  metadata.gz: 8824c3d24a5ba9e63bca4fdeeec59220ea35f656
+  data.tar.gz: b4f6c6304ceced41d0fc2913140d9540a55c6055
 SHA512:
-  metadata.gz: b216fc333e7b71c654ae83ee247169be68cdb75dd072bb72f2e0bcf5e1a8f4fa66d1ebfcb26647ac80af92720d352c7cd6d880fd5f9560ba94c913b229b02946
-  data.tar.gz: 2520dc769f5320ec1fecf416ad0b223d0553c0f1b44591288e5efac1e860e0258534ddd0097d1bbd71a4d0ae824f98bab8e9b21db7caf44654693b04d5be64da
+  metadata.gz: a9618dd49101a04df2d44a01ecbc67439af667bb93fa9314c5012f8af8c228dccece53eac7608590cc76599e229d7d9fab7568697f5b3523f33412c40b3c1093
+  data.tar.gz: 76897cac3f3b281060dc7ceb0fa667b71bfb74a127ea616bb1062f818c69a641062f0fd8aa95d8b646f2fe5499907286eb48f38b65c5e0134d7745ecd644fc85

data/CHANGELOG.md

@@ -1,5 +1,9 @@
 # dev-lxc Change Log
 
+## 2.5.0 (2017-02-08)
+
+* Add memory_per_server config option to limit memory per server
+
 ## 2.4.0 (2017-02-03)
 
 * Refactor DHCP address management

data/README.md

@@ -7,7 +7,7 @@ Cluster management includes the ability to manage snapshots of the containers wh
 
 ### Features
 
-1. LXC 2.0 Containers - Resource efficient servers with fast start/stop times and standard init
+1. LXC Containers - Resource efficient servers with fast start/stop times and standard init
 2. Btrfs - Efficient, persistent storage backend provides fast, lightweight container snapshots
 3. Dnsmasq - DHCP networking and DNS resolution
 4. Base Containers - Containers that are built to resemble a traditional server
@@ -26,33 +26,58 @@ The Btrfs backed snapshots provide a quick clean slate which is helpful especial
 experimenting and troubleshooting. Or it can be used to build a customized cluster
 for demo purposes and be able to bring it up quickly and reliably.
 
-If you aren't familiar with using containers please read this introduction.
+If you aren't familiar with using containers you might be interested in this introduction.
 
 [LXC 1.0 Introduction](https://www.stgraber.org/2013/12/20/lxc-1-0-blog-post-series/)
 
-### Requirements
+Additional dev-lxc related documentation can be found in the [docs folder](docs) in this repository.
 
-* Please follow the [Prerequisites Instructions](docs/prerequisites.md)
+## Build dev-lxc-platform instance
 
-When you are done with the prerequisites you should be able to log into the dev-lxc-platform VM and start using it.
+The dev-lxc tool is used in a system that has been configured by the dev-lxc-platform cookbook.
 
-You must login to the root user to use the dev-lxc command.
+The easiest way to build a dev-lxc-platform system is to download the dev-lxc-platform repository
+and use Test Kitchen to build a VirtualBox Vagrant instance or an AWS EC2 instance.
+
+Follow the instructions in the [dev-lxc-platform README](https://github.com/jeremiahsnapp/dev-lxc-platform) to build
+a dev-lxc-platform instance.
+
+## Login to the dev-lxc-platform instance
+
+Login to the dev-lxc-platform instance and switch to the root user to use the dev-lxc tool.
 
 ```
 cd dev-lxc-platform
-vagrant ssh
+kitchen login <ec2 or vagrant>
 sudo -i
 ```
 
-### Update dev-lxc gem
+When you are logged in as the root user you should automatically enter a [byobu session](http://byobu.co/).
+
+Byobu makes it easy to manage multiple terminal windows and panes. You can press `F1` to get help which includes a [list of keybindings](http://manpages.ubuntu.com/manpages/wily/en/man1/byobu.1.html#contenttoc8).
+
+The prefix key is set to `Ctrl-o`
+
+Some of the keys that will be most useful to you are:
+
+* `option-Up`, `option-Down` to switch between Byobu sessions
+* `option-Left`, `option-Right` to switch between windows in a session
+* `shift-Left`, `shift-Right`, `shift-Up`, `shift-Down` to switch between panes in a window
+
+## Update dev-lxc gem
 
-Run the following command if you ever need to upgrade the dev-lxc gem inside the dev-lxc-platform VM.
+Run the following command as the instance's root user if you ever need to upgrade the dev-lxc gem inside the dev-lxc-platform instance.
 
 ```
+cd dev-lxc-platform
+kitchen login <ec2 or vagrant>
+sudo -i
 chef gem update dev-lxc
 ```
 
-### Display Help
+## Demo: Build Chef Automate Cluster
+
+### Display dev-lxc help
 
 ```
 dev-lxc help
@@ -78,28 +103,7 @@ dev-lxc snapshot
 dl sn
 ```
 
-### Demo: Build Chef Automate Cluster
-
-Log into the dev-lxc-platform VM's root user.
-
-```
-cd dev-lxc-platform
-vagrant up     # if the VM is not already running
-vagrant ssh
-sudo -i
-```
-
-When you are logged in as the root user you should automatically enter a [byobu session](http://byobu.co/).
-
-Byobu makes it easy to manage multiple terminal windows and panes. You can press `fn-F1` to get help which includes a [list of keybindings](http://manpages.ubuntu.com/manpages/wily/en/man1/byobu.1.html#contenttoc8).
-
-Some of the keys that will be most useful to you are:
-
-* `option-Up`, `option-Down` to switch between Byobu sessions
-* `option-Left`, `option-Right` to switch between windows in a session
-* `shift-Left`, `shift-Right`, `shift-Up`, `shift-Down` to switch between panes in a window
-
-#### Create Base Container
+### Create Base Container
 
 The [base container](docs/base_containers.md) used for the cluster's containers must be created first. Let's use Ubuntu 14.04 for the base container.
 
@@ -107,7 +111,7 @@ The [base container](docs/base_containers.md) used for the cluster's containers
 dl create b-ubuntu-1404
 ```
 
-#### Create Config File
+### Create Config File
 
 Create the [dev-lxc.yml config file](docs/configuration.md) for the cluster.
 
@@ -138,7 +142,7 @@ Edit the dev-lxc.yml file:
 * (Optionally) If you built other clusters then you can modify the server names (including the nodes' `chef_server_url`) in this cluster to
   make them [unique from the other clusters](docs/manage_multiple_clusters.md).
 
-#### cluster-view
+### cluster-view
 
 Run the `cluster-view` command to create a Byobu session specifically for this cluster.
 
@@ -157,7 +161,7 @@ See the [usage docs](docs/usage.md) for more information about how to close/kill
 cluster-view /root/work/clusters/automate
 ```
 
-#### Specifying a Subset of Servers
+### Specifying a Subset of Servers
 
 Many dev-lxc subcommands can act on a subset of the cluster's servers by specifying a regular expression that matches the desired server names.
 
@@ -167,7 +171,7 @@ For example, the following command will show the status of the infrastructure no
 dl status node
 ```
 
-#### Build the Cluster
+### Build the Cluster
 
 dev-lxc knows to build the servers in an appropriate order.
 
@@ -182,13 +186,13 @@ dl up
 Note: You also have the option of running the `prepare-product-cache` subcommand which downloads required product packages to the cache.  
 This can be helpful when you don't want to start building the cluster yet but you want the package cache ready when you build the cluster later.
 
-#### Use the Servers
+### Use the Servers
 
 At this point all of the cluster's servers should be running.
 
 If you setup the workstation's networking correctly as described in the prerequisites you should be able to ping any server from your workstation using it's FQDN. You can also browse to any server that has a web interface.
 
-Since the cluster has a Chef Server you can use the `chef-repo` subcommand to create a chef-repo directory in the VM that contains a knife.rb and all of the keys for the users and org validator clients that are defined in dev-lxc.yml. This makes it very easy to use tools such as knife or berkshelf.
+Since the cluster has a Chef Server you can use the `chef-repo` subcommand to create a chef-repo directory in the host instance that contains a knife.rb and all of the keys for the users and org validator clients that are defined in dev-lxc.yml. This makes it very easy to use tools such as knife or berkshelf.
 
 ```
 dl chef
@@ -213,7 +217,7 @@ dl at chef
 
 Since the cluster has a Chef Server and an infrastructure node dev-lxc made sure it configured the node's chef-client for the Chef Server so it is easy to converge the node.
 
-#### Manage the Cluster
+### Manage the Cluster
 
 The right pane of the "cluster" window should show `dev-lxc status` output. This shows the status of each server including any existing snapshots.
 

data/docs/configuration.md

@@ -21,6 +21,13 @@ The contents of `dev-lxc.yml` should look like this.
 # base_container must be the name of an existing container
 base_container: b-ubuntu-1404
 
+# memory_per_server sets the maximum amount of user memory (including file cache) for each server.
+# dev-lxc will set the `memory.limit_in_bytes` cgroup for each server to apply this limit.
+# If no units are specified, the value is interpreted as bytes.
+# You can use suffixes to represent larger units — k or K for kilobytes, m or M for megabytes, and g or G for gigabytes.
+# The default behavior is that no limit is set.
+#memory_per_server: 4G
+
 # list any host directories you want mounted into the servers
 #mounts:
 #  - /root/work root/work
@@ -96,8 +103,7 @@ As you can see there are four server types represented by five servers.
 
 #### Global Settings
 
-The global settings used by each of the server types are `enable_build_snapshots`, the `base_container`, a list of `mounts` and
-a list of `ssh-keys`. These settings are described in the config comments.
+The global settings used by each of the server types are `enable_build_snapshots`, the `base_container`, `memory_per_server`, a list of `mounts` and a list of `ssh-keys`. These settings are described in the config comments.
 
 Be sure to set `base_container` in the `dev-lxc.yml` to an existing container's name.  
 This container will be cloned to create each container in the cluster.  
@@ -106,7 +112,7 @@ If you don't already have a container to use as a `base_container` then you can
 
 #### Server Specific Settings
 
-It is possible to define different values for `enable_build_snapshots`, `base_container`, `mounts` or `ssh-keys` for a particular server type or even for a particular server as you can see in the following snippet.
+It is possible to define different values for `enable_build_snapshots`, `base_container`, `memory_per_server`, `mounts` or `ssh-keys` for a particular server type or even for a particular server as you can see in the following snippet.
 
 ```
 nodes:
@@ -123,7 +129,7 @@ is not specified then a dynamic IP address is assigned when the server starts.
 #### mixlib-install Library Automatically Manages a Cache of Product Packages
 
 dev-lxc uses the [mixlib-install](https://github.com/chef/mixlib-install) library to download Chef products
-to a cache in `/var/dev-lxc` in the host VM. This cache is automatically mounted into each server when it starts.
+to a cache in `/var/dev-lxc` in the host instance. This cache is automatically mounted into each server when it starts.
 
 A list of Chef products to be installed can be defined for each server
 using [product names that mixlib-install understands](https://github.com/chef/mixlib-install/blob/master/PRODUCT_MATRIX.md).

data/lib/dev-lxc/cli.rb

@@ -67,6 +67,13 @@ module DevLXC::CLI
 # base_container must be the name of an existing container
 base_container: b-ubuntu-1404
 
+# memory_per_server sets the maximum amount of user memory (including file cache) for each server.
+# dev-lxc will set the `memory.limit_in_bytes` cgroup for each server to apply this limit.
+# If no units are specified, the value is interpreted as bytes.
+# You can use suffixes to represent larger units — k or K for kilobytes, m or M for megabytes, and g or G for gigabytes.
+# The default behavior is that no limit is set.
+#memory_per_server: 4G
+
 # list any host directories you want mounted into the servers
 #mounts:
 #  - /root/work root/work

data/lib/dev-lxc/cluster.rb

@@ -26,6 +26,10 @@ module DevLXC
               enable_build_snapshots = server_config["enable_build_snapshots"] if server_config.key?("enable_build_snapshots")
               enable_build_snapshots = true if enable_build_snapshots.nil?
 
+              memory_per_server = cluster_config[server_type]["memory_per_server"]
+              memory_per_server ||= cluster_config["memory_per_server"]
+              memory_per_server = server_config["memory_per_server"] if server_config["memory_per_server"]
+
               mounts = ["/var/dev-lxc var/dev-lxc"]
               if cluster_config[server_type]["mounts"]
                 mounts.concat(cluster_config[server_type]["mounts"])
@@ -49,6 +53,7 @@ module DevLXC
                 additional_fqdn: nil,
                 enable_build_snapshots: enable_build_snapshots,
                 first_run: false,
+                memory_per_server: memory_per_server,
                 mounts: mounts,
                 ssh_keys: ssh_keys,
                 base_container_name: base_container_name
@@ -266,9 +271,10 @@ module DevLXC
     def get_server(server_name)
       ipaddress = @server_configs[server_name][:ipaddress]
       additional_fqdn = @server_configs[server_name][:additional_fqdn]
+      memory_per_server = @server_configs[server_name][:memory_per_server]
       mounts = @server_configs[server_name][:mounts]
       ssh_keys = @server_configs[server_name][:ssh_keys]
-      Server.new(server_name, ipaddress, additional_fqdn, mounts, ssh_keys)
+      Server.new(server_name, ipaddress, additional_fqdn, memory_per_server, mounts, ssh_keys)
     end
 
     def get_sorted_servers(server_name_regex=nil)

data/lib/dev-lxc/server.rb

@@ -5,10 +5,11 @@ module DevLXC
   class Server
     attr_reader :container
 
-    def initialize(name, ipaddress, additional_fqdn, mounts, ssh_keys)
+    def initialize(name, ipaddress, additional_fqdn, memory_per_server, mounts, ssh_keys)
       @container = DevLXC::Container.new(name)
       @ipaddress = ipaddress
       @additional_fqdn = additional_fqdn
+      @memory_per_server = memory_per_server
       @mounts = mounts
       @ssh_keys = ssh_keys
     end
@@ -43,6 +44,7 @@ module DevLXC
       @container.sync_mounts(@mounts)
       @container.start
       @container.sync_ssh_keys(@ssh_keys)
+      @container.set_cgroup_item('memory.limit_in_bytes', @memory_per_server) if @memory_per_server
       puts
     end
 

data/lib/dev-lxc/version.rb

@@ -1,3 +1,3 @@
 module DevLXC
-  VERSION = "2.4.0"
+  VERSION = "2.5.0"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: dev-lxc
 version: !ruby/object:Gem::Version
-  version: 2.4.0
+  version: 2.5.0
 platform: ruby
 authors:
 - Jeremiah Snapp
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2017-02-03 00:00:00.000000000 Z
+date: 2017-02-08 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler
@@ -101,7 +101,6 @@ files:
 - docs/configuration.md
 - docs/dev-lxc_version_2.md
 - docs/manage_multiple_clusters.md
-- docs/prerequisites.md
 - docs/usage.md
 - lib/dev-lxc.rb
 - lib/dev-lxc/cli.rb
@@ -129,7 +128,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.6.8
+rubygems_version: 2.6.10
 signing_key: 
 specification_version: 4
 summary: A tool for building Chef server clusters using LXC containers

data/docs/prerequisites.md

@@ -1,81 +0,0 @@
-
-#### Download and Install Prerequisites
-
-Download and install [VirtualBox 5.0.14+](https://www.virtualbox.org/wiki/Downloads).
-
-Download and install [Vagrant 1.8.5+](https://www.vagrantup.com/downloads.html).
-
-Install the vagrant-persistent-storage plugin.
-
-```
-vagrant plugin install vagrant-persistent-storage
-```
-
-Download and install [ChefDK](http://downloads.chef.io/).
-
-Run `chef shell-init` to read its usage docs. Then run the appropriate command for your shell.
-
-#### Download dev-lxc-platform
-
-```
-git clone https://github.com/jeremiahsnapp/dev-lxc-platform.git
-```
-
-#### Configure .kitchen.yml
-
-The cpus and memory .kitchen.yml values are set high to give enough resources to comfortably run multiple containers.
-
-Configure .kitchen.yml settings such as cpus, memory, synced_folders as desired.
-
-#### Build the dev-lxc-platform VM
-
-This should take less than 15 minutes.
-
-```
-cd dev-lxc-platform
-kitchen converge
-```
-
-#### Enable Vagrant Control for VM
-
-Typically we want to be able to shutdown and startup the dev-lxc-platform VM rather than use the usual kitchen model of converge and destroy so we need to enable Vagrant control over the VM for easier management.
-
-Install [direnv](http://direnv.net/) to use the `.envrc` file included in the
-dev-lxc-platform repo to automatically set `VAGRANT_CWD` upon entering the top level directory
-of the dev-lxc-platform repo.
-
-Vagrant commands run from this directory such as `vagrant up`, `vagrant ssh` and `vagrant halt` will manage the dev-lxc-platform VM.
-
-```
-brew install direnv
-```
-
-Be sure to follow the [direnv install instructions](http://direnv.net/) to add the appropriate line to your user's shell rc file.
-
-Run the following to approve the `.envrc` file
-
-```
-direnv allow
-```
-
-#### Setup Networking
-
-Your workstation needs to know how to resolve the .lxc domain that dev-lxc containers use.
-
-For OS X you can run the following command.
-
-```
-sudo mkdir -p /etc/resolver
-echo nameserver 10.0.3.1 | sudo tee /etc/resolver/lxc
-```
-
-Adding a route entry to the workstation enables direct communication between
-the workstation and any dev-lxc container.
-
-For OS X run the following command.
-
-The route entry won't survive a workstation reboot. You will have to recreate it as needed.
-
-```
-sudo route -n add 10.0.3.0/24 33.33.34.13
-```