vagrant-cachier 0.5.1 → 0.6.0

data/docs/buckets/apt-lists.md

@@ -0,0 +1,18 @@
+# APT lists
+
+Used by Debian-like Linux distros, will get configured under guest's `/var/lib/apt/lists`.
+
+As explained on [Wikipedia](http://en.wikipedia.org/wiki/Advanced_Packaging_Tool#Files),
+`/var/lib/apt/lists` is the "storage area for state information for each package
+resource specified in sources.list". By enabling this bucket, `apt` will be able
+to install cached packages without hitting the remote repositories for the main
+package lists, [being particularly useful when developing offline](https://github.com/fgrehm/vagrant-cachier/pull/84#issue-27311414).
+
+To manually enable it:
+
+```ruby
+Vagrant.configure("2") do |config|
+  config.vm.box = 'some-debian-box'
+  config.cache.enable :apt_lists
+end
+```

data/docs/how-does-it-work.md

@@ -20,12 +20,11 @@ _as part of_ the boot process (synced folders are actually `lxc-start` parameter
 and as of now we are not able to get some information that this plugin requires
 about the guest machine / container before it is actually up and running.
 
-Under the hood, the plugin will monkey patch `Vagrant::Builtin::Provision` and
-will set things up for each configured cache bucket _before and after_ running each
-defined provisioner. Before halting the machine, it will also revert the changes
-required to set things up by hooking into calls to `Vagrant::Builtin::GracefulHalt`
-so that you can repackage the machine for others to use without requiring users to
-install the plugin as well.
+Under the hood, the plugin will hook into Vagrant in order to set things up for each
+configured cache bucket _before and after_ running each defined provisioner. Before
+halting the machine, it will also revert the changes required to set things up by
+hooking into calls to `Vagrant::Builtin::GracefulHalt` so that you can repackage
+the machine for others to use without requiring users to install the plugin as well.
 
 Please keep in mind that this plugin won't do magic, if you are compiling things
 during provisioning or manually downloading packages outside of a bucket you

data/docs/index.md

@@ -9,7 +9,7 @@ multiple package managers and Linux distros.
 
 ## Installation
 
-Make sure you have Vagrant 1.2+ and run:
+Make sure you have Vagrant 1.4+ and run:
 
 ```
 vagrant plugin install vagrant-cachier
@@ -23,21 +23,40 @@ from within your `Vagrantfile`:
 ```ruby
 Vagrant.configure("2") do |config|
   config.vm.box = 'your-box'
-  config.cache.auto_detect = true
-  # If you are using VirtualBox, you might want to enable NFS for shared folders
-  # config.cache.enable_nfs  = true
+  if Vagrant.has_plugin?("vagrant-cachier")
+    # Configure cached packages to be shared between instances of the same base box.
+    # More info on the "Usage" link above
+    config.cache.scope = :box
+
+    # If you are using VirtualBox, you might want to use that to enable NFS for
+    # shared folders. This is also very useful for vagrant-libvirt if you want
+    # bi-directional sync
+    config.cache.synced_folder_opts = {
+      type: :nfs,
+      # The nolock option can be useful for an NFSv3 client that wants to avoid the
+      # NLM sideband protocol. Without this option, apt-get might hang if it tries
+      # to lock files needed for /var/cache/* operations. All of this can be avoided
+      # by using NFSv4 everywhere. Please note that the tcp option is not the default.
+      mount_options: ['rw', 'vers=3', 'tcp', 'nolock']
+    }
+  end
 end
 ```
 
 For more information please check out the links on the menu above.
 
 
-## Compatible providers
+## Providers that are known to work
 
 * Vagrant's built in VirtualBox provider
 * [vagrant-lxc](https://github.com/fgrehm/vagrant-lxc)
 * [VMware providers](http://www.vagrantup.com/vmware) with NFS enabled (See
   [GH-24](https://github.com/fgrehm/vagrant-cachier/issues/24) for more info)
+* [vagrant-libvirt](https://github.com/pradels/vagrant-libvirt)
+* [vagrant-kvm](https://github.com/adrahon/vagrant-kvm)
+
+_Please note that as of v0.6.0 the plugin will automatically disable any
+previously defined configs for [cloud providers](lib/vagrant-cachier/plugin.rb#L19-22)_
 
 
 ## Contributing

data/docs/template.html

@@ -83,6 +83,7 @@
                 <a class="dropdown-toggle" data-toggle="dropdown" href="#">Available Buckets <span class="caret"></span></a>
                 <ul class="dropdown-menu">
                   <li><a tabindex="-1" href="buckets/apt">APT</a></li>
+                  <li><a tabindex="-1" href="buckets/apt-lists">apt-lists</a></li>
                   <li><a tabindex="-1" href="buckets/apt-cacher">apt-cacher</a></li>
                   <li><a tabindex="-1" href="buckets/chef">Chef</a></li>
                   <li><a tabindex="-1" href="buckets/composer">Composer</a></li>

data/docs/usage.md

@@ -1,51 +1,46 @@
 # Usage
 
-## Auto detect supported cache buckets
+## Being nice to others
 
-This is the easiest way to get started with plugin. By adding the code below to
-your `Vagrantfile` you can enable automatic detection of supported cache _buckets_:
+It is a good practice to wrap plugin specific configuration with `has_plugin?` checks
+so the user's Vagrantfiles do not break if `vagrant-cachier` is uninstalled or
+the Vagrantfile is shared with people that don't have the plugin installed:
 
 ```ruby
 Vagrant.configure("2") do |config|
   # ...
-  config.cache.auto_detect = true
+  if Vagrant.has_plugin?("vagrant-cachier")
+    # ... vagrant-cachier configs ...
+  end
 end
 ```
 
-This will make vagrant-cachier do its best to find out what is supported on the
-guest machine and will set buckets accordingly.
-
-## Enable buckets as needed
-
-If for whatever reason you need to have a fined grained control over what buckets
-are configured, you can do so by "cherry picking" them on your `Vagrantfile`:
-
-```ruby
-Vagrant.configure("2") do |config|
-  config.cache.enable :apt
-  config.cache.enable :gem
-end
-```
+## Cache scope
 
-_Please refer to the "Available Buckets" menu above to find out which buckets
-are supported._
+This is the only required configuration for the plugin to work and should be present
+on your project's specific `Vagrantfile` or on your `~/.vagrant.d/Vagrantfile` in
+order to enable it.
 
-## Cache scope
+### `:box` scope
 
-By default downloaded packages will get stored on a folder scoped to base boxes
-under your `$HOME/.vagrant.d/cache`. The idea is to leverage the cache by allowing
-downloaded packages to be reused across projects. So, if your `Vagrantfile` has
-something like:
+By setting `cache.scope` to `:box`, downloaded packages will get stored on a folder
+scoped to base boxes under your `~/.vagrant.d/cache`. The idea is to leverage the
+cache by allowing downloaded packages to be reused across projects. So, if your
+`Vagrantfile` has something like:
 
 ```ruby
 Vagrant.configure("2") do |config|
   config.vm.box = 'some-box'
-  config.cache.auto_detect = true
+  if Vagrant.has_plugin?("vagrant-cachier")
+    config.cache.scope = :box
+  end
 end
 ```
 
 The cached files will be stored under `$HOME/.vagrant.d/cache/some-box`.
 
+### `:machine` scope
+
 If you are on a [multi VM environment](http://docs.vagrantup.com/v2/multi-machine/index.html),
 there is a huge chance that you'll end up having issues by sharing the same bucket
 across different machines. For example, if you `apt-get install` from two machines
@@ -56,13 +51,71 @@ to be based on machines:
 ```ruby
 Vagrant.configure("2") do |config|
   config.vm.box = 'some-box'
-  config.cache.scope = :machine
+  if Vagrant.has_plugin?("vagrant-cachier")
+    config.cache.scope = :machine
+  end
 end
 ```
 
 This will tell vagrant-cachier to download packages to `.vagrant/machines/<machine-name>/cache`
 on your current project directory.
 
+## Cache buckets automatic detection
+
+This is the easiest way to get started with plugin and is enabled by default.
+Under the hood, `vagrant-cachier` does its best to find out what is supported on the
+guest machine and will set buckets accordingly.
+
+If you want that behavior to be disabled, you can set `cache.auto_detect` to `false`
+from your Vagrantfile:
+
+```ruby
+Vagrant.configure("2") do |config|
+  config.vm.box = 'some-box'
+  if Vagrant.has_plugin?("vagrant-cachier")
+    config.cache.scope       = :machine # or :box
+    config.cache.auto_detect = false
+  end
+end
+```
+
+## Enable buckets as needed
+
+If for whatever reason you need to have a fined grained control over what buckets
+are configured, you can do so by "cherry picking" them on your `Vagrantfile`:
+
+```ruby
+Vagrant.configure("2") do |config|
+  config.cache.auto_detect = false
+  config.cache.enable :apt
+  config.cache.enable :gem
+end
+```
+
+_Please refer to the "Available Buckets" menu above to find out which buckets
+are supported._
+
+## Custom cache buckets synced folders options
+
+For fine grained control over the cache bucket synced folder options you can use
+the `synced_folder_opts` config. That's useful if, for example, you are using
+VirtualBox and want to enable NFS for improved performance:
+
+```ruby
+Vagrant.configure("2") do |config|
+  config.cache.synced_folder_opts = {
+    type: :nfs,
+    # The nolock option can be useful for an NFSv3 client that wants to avoid the
+    # NLM sideband protocol. Without this option, apt-get might hang if it tries
+    # to lock files needed for /var/cache/* operations. All of this can be avoided
+    # by using NFSv4 everywhere. Please note that the tcp option is not the default.
+    mount_options: ['rw', 'vers=3', 'tcp', 'nolock']
+  }
+end
+```
+
+Please referer to http://docs.vagrantup.com/v2/synced-folders/basic_usage.html for
+more information about the supported parameters.
 
 ## Finding out disk space used by buckets
 
@@ -95,5 +148,5 @@ the code below if you are on a Linux machine:
 $ rm -rf $HOME/.vagrant.d/cache/<box-name>/<optional-bucket-name>
 
 # scope = :machine
-$ rm -rf .vagrant/cache/<box-name>/<optional-bucket-name>
+$ rm -rf .vagrant/machines/<machine-name>/cache/<optional-bucket-name>
 ```

data/lib/vagrant-cachier/action/clean.rb

@@ -56,7 +56,7 @@ module VagrantPlugins
 
         def remove_symlink(symlink)
           if @machine.communicate.test("test -L #{symlink}")
-            @logger.debug "Removing symlink for '#{symlink}'"
+            @logger.info "Removing symlink for '#{symlink}'"
             @machine.communicate.sudo("unlink #{symlink}")
           end
         end

data/lib/vagrant-cachier/action/configure_bucket_root.rb

@@ -0,0 +1,46 @@
+require 'timeout'
+
+module VagrantPlugins
+  module Cachier
+    class Action
+      class ConfigureBucketRoot
+        def initialize(app, env)
+          @app    = app
+          @logger = Log4r::Logger.new("vagrant::cachier::action::clean")
+        end
+
+        def call(env)
+          @env = env
+
+          if !env[:cache_buckets_folder_configured] && env[:machine].config.cache.enabled?
+            setup_buckets_folder
+            env[:cache_buckets_folder_configured] = true
+          end
+
+          @app.call env
+        end
+
+        def setup_buckets_folder
+          FileUtils.mkdir_p(cache_root.to_s) unless cache_root.exist?
+
+          synced_folder_opts = {id: "vagrant-cache"}
+          synced_folder_opts.merge!(@env[:machine].config.cache.synced_folder_opts || {})
+
+          @env[:machine].config.vm.synced_folder cache_root, '/tmp/vagrant-cache', synced_folder_opts
+          @env[:cache_dirs] = []
+        end
+
+        def cache_root
+          @cache_root ||= case @env[:machine].config.cache.scope.to_sym
+            when :box
+              @env[:home_path].join('cache', @env[:machine].box.name)
+            when :machine
+              @env[:machine].data_dir.parent.join('cache')
+            else
+              raise "Unknown cache scope: '#{@env[:machine].config.cache.scope}'"
+          end
+        end
+      end
+    end
+  end
+end

data/lib/vagrant-cachier/action/install_buckets.rb

@@ -0,0 +1,47 @@
+require_relative '../bucket'
+
+module VagrantPlugins
+  module Cachier
+    class Action
+      class InstallBuckets
+        def initialize(app, env, opts = {})
+          @app    = app
+          @logger = Log4r::Logger.new("vagrant::cachier::action::clean")
+          @opts   = opts
+        end
+
+        def call(env)
+          @app.call(env)
+
+          return unless env[:machine].config.cache.enabled?
+
+          chmod_bucket_root(env[:machine]) if @opts[:chmod]
+          configure_cache_buckets(env)
+        end
+
+        def chmod_bucket_root(machine)
+          @logger.info "'chmod'ing bucket root dir to 777..."
+          machine.communicate.sudo 'mkdir -p /tmp/vagrant-cache && chmod 777 /tmp/vagrant-cache'
+        end
+
+        def configure_cache_buckets(env)
+          if env[:machine].config.cache.auto_detect
+            Bucket.auto_detect(env)
+          end
+
+          return unless env[:machine].config.cache.buckets.any?
+
+          env[:ui].info 'Configuring cache buckets...'
+          cache_config = env[:machine].config.cache
+          cache_config.buckets.each do |bucket_name, configs|
+            @logger.info "Installing #{bucket_name} with configs #{configs.inspect}"
+            Bucket.install(bucket_name, env, configs)
+          end
+
+          data_file = env[:machine].data_dir.join('cache_dirs')
+          data_file.open('w') { |f| f.print env[:cache_dirs].uniq.join("\n") }
+        end
+      end
+    end
+  end
+end

data/lib/vagrant-cachier/bucket.rb

@@ -29,6 +29,60 @@ module VagrantPlugins
         @env     = env
         @configs = configs
       end
+
+      def machine
+        @env[:machine]
+      end
+
+      def guest
+        machine.guest
+      end
+
+      def comm
+        machine.communicate
+      end
+
+      # TODO: "merge" symlink and user_symlink methods
+      def symlink(guest_path, bucket_path = "/tmp/vagrant-cache/#{@name}", create_parent: true)
+        return if @env[:cache_dirs].include?(guest_path)
+
+        @env[:cache_dirs] << guest_path
+        comm.execute("mkdir -p #{bucket_path}")
+        unless symlink?(guest_path)
+          comm.sudo("mkdir -p `dirname #{guest_path}`") if create_parent
+          if empty_dir?(bucket_path) && !empty_dir?(guest_path)
+            # Warm up cache with guest machine data
+            comm.sudo("shopt -s dotglob && mv #{guest_path}/* #{bucket_path}")
+          end
+          comm.sudo("rm -rf #{guest_path}")
+          comm.sudo("ln -s #{bucket_path} #{guest_path}")
+        end
+      end
+
+      def user_symlink(guest_path)
+        return if @env[:cache_dirs].include?(guest_path)
+
+        @env[:cache_dirs] << guest_path
+        bucket_path = "/tmp/vagrant-cache/#{@name}"
+        comm.execute("mkdir -p #{bucket_path}")
+        unless symlink?(guest_path)
+          comm.execute("mkdir -p `dirname #{guest_path}`")
+          if empty_dir?(bucket_path) && !empty_dir?(guest_path)
+            # Warm up cache with guest machine data
+            comm.execute("shopt -s dotglob && mv #{guest_path}/* #{bucket_path}")
+          end
+          comm.execute("rm -rf #{guest_path}")
+          comm.execute("ln -s #{bucket_path} #{guest_path}")
+        end
+      end
+
+      def empty_dir?(path)
+        not comm.test("test \"$(ls -A #{path} 2>/dev/null)\"")
+      end
+
+      def symlink?(path)
+        comm.test("test -L #{path}")
+      end
     end
   end
 end
@@ -40,6 +94,7 @@ require_relative "bucket/pacman"
 require_relative "bucket/yum"
 require_relative "bucket/rvm"
 require_relative "bucket/apt_cacher"
+require_relative "bucket/apt_lists"
 require_relative "bucket/composer"
 require_relative "bucket/npm"
 require_relative "bucket/zypper"

data/lib/vagrant-cachier/bucket/apt.rb

@@ -7,22 +7,13 @@ module VagrantPlugins
         end
 
         def install
-          machine = @env[:machine]
-          guest   = machine.guest
-
           if guest.capability?(:apt_cache_dir)
             guest_path = guest.capability(:apt_cache_dir)
 
-            @env[:cache_dirs] << guest_path
+            return if @env[:cache_dirs].include?(guest_path)
 
-            machine.communicate.tap do |comm|
-              comm.execute("mkdir -p /tmp/vagrant-cache/#{@name}/partial")
-              unless comm.test("test -L #{guest_path}")
-                comm.sudo("rm -rf #{guest_path}")
-                comm.sudo("mkdir -p `dirname #{guest_path}`")
-                comm.sudo("ln -s /tmp/vagrant-cache/#{@name} #{guest_path}")
-              end
-            end
+            symlink(guest_path)
+            comm.execute("mkdir -p /tmp/vagrant-cache/#{@name}/partial")
           else
             @env[:ui].info I18n.t('vagrant_cachier.skipping_bucket', bucket: 'APT')
           end