defog 0.0.1 → 0.1.0

data/README.rdoc

@@ -15,6 +15,10 @@ it also works with the local file system as a "provider" so that you can,
 e.g. use the local file system for development and the cloud for
 production.
 
+Defog also provides a few simple remote-file management methods to minimize
+the need to dig down into the Fog layer; but full access to the underlying
+fog objects is available should it be needed.
+
 == Usage Summary
 
 Full Rdoc is available at http://rubydoc.info/gems/defog
@@ -35,7 +39,7 @@ object, which proxies files in a specific remote location, e.g.:
 
 === Proxy a file
 
-Proxy a remote file by creating a <code>Defog::File</code> object:
+Open a proxy to a remote file by creating a <code>Defog::File</code> object:
 
         file = defog.file("key/of/file", mode)
         # ... access file ...
@@ -46,7 +50,14 @@ Proxy a remote file by creating a <code>Defog::File</code> object:
         end
 
 <code>mode</code> can be "r", "r+", "w", "w+", "a", or "a+" with the usual
-semantics.  Closing the file object (explicitly or implicitly at the end of
+semantics. 
+
+When opened in a readable mode ("r", "r+", "w+", "a+"), first caches the
+cloud file in the local proxy.  When opened in a writeable mode ("r+", "w",
+"w+", "a", "a+"), arranges to upload the changes back to the cloud file
+at close time.
+
+Closing the file object (explicitly or implicitly at the end of
 the block) synchronizes the local proxy with the remote storage and (by
 default) deletes the local proxy file.
 
@@ -65,11 +76,24 @@ You can also access the proxy file via its path, allowing things such as
 
 (Note that the proxy file path has the same file extension as the cloud key string.)
 
-
 To suppress updating the remote storage, delete the local proxy file before
 closing (e.g. via <code>File.unlink(file.path)</code>) or pass
 <code>:synchronize => false</code> to the <code>#close</code> method.
 
+=== Proxy handle
+
+Calling Defog::Proxy#file without a mode returns a Defog::Handle object that supports remote file query and manipulation:
+
+    proxy.file("key").exist?  # => returns true if the remove file exists
+    proxy.file("key").delete  # => deletes the remote file
+
+In fact, <code>proxy.file("key", mode, options, &block)</code> is really just shorthand for
+
+    proxy.file("key").open(mode, options, &block)
+
+In addition, the handle allows you to look up the path where the local proxy file will be if/when you open the proxy (but without actually doing the proxying).
+
+    proxy.file("key").proxy_path  # returns pathname where proxy file will be when you open
 
 === Persistence
 
@@ -83,7 +107,7 @@ via
 
 or
 
-        file.close(:persist => true) # overrides setting at runtime
+        file.close(:persist => true)
 
 When opening a file whose local proxy has been persisted, Defog checks to see if
 the local proxy is out of date and if so replaces it.
@@ -100,9 +124,9 @@ Local proxy files are stored by default in
         #{tmproot}/defog/#{provider}/#{location}/#{key}
 
 where <code>tmproot</code> is <code>Rails.root+"tmp"</code> if Rails is
-defined, otherwise <code>Dir.tmpdir()</code>.  For AWS,
-  <code>location</code> is the bucket name, and for Local it's the
-  local_root directory path with slashes replaced with dashes
+defined, otherwise <code>Dir.tmpdir()</code>.  For :AWS,
+<code>location</code> is the bucket name, and for :local it's the
+<code>local_root</code> directory path with slashes replaced with underscores.
 
 See the documentation for configuring other locations.
 
@@ -113,10 +137,8 @@ Gemfile:
 
 == Compatibility
 
-Defog has (so far) been tested using MRI 1.9.3 and
-[fog](https://rubygems.org/gems/fog) storage providers :Local and :AWS
-
-== History
+Defog has (so far) been tested on MRI 1.9.3 using
+[fog](https://rubygems.org/gems/fog) storage providers :local and :AWS
 
 == Copyright
 

data/lib/defog.rb

@@ -1,5 +1,6 @@
 require 'defog/error'
 require 'defog/file'
 require 'defog/fog_wrapper'
+require 'defog/handle'
 require 'defog/proxy'
 require 'defog/version'

data/lib/defog/file.rb

@@ -32,18 +32,21 @@ module Defog
   # the proxy deleted.  See File#close for more details.
   class File < ::File
     def self.get(opts={}, &block) #:nodoc:
-      opts = opts.keyword_args(:proxy => :required, :key => :required, :mode => :required, :persist => :optional)
+      opts = opts.keyword_args(:handle => :required, :mode => :required, :persist => :optional)
+
+      handle = opts.handle
+      key = handle.key
+      proxy_path = handle.proxy_path
 
-      proxy_path = opts[:proxy_path] = Pathname.new("#{opts.proxy.proxy_root}/#{opts.key}").expand_path
       proxy_path.dirname.mkpath
 
       case opts.mode
       when "r" then
-        opts.proxy.fog_wrapper.get_file(opts.key, proxy_path)
+        handle.proxy.fog_wrapper.get_file(key, proxy_path)
       when "w", "w+" then
         opts[:upload] = true
       when "r+", "a", "a+" then
-        opts.proxy.fog_wrapper.get_file(opts.key, proxy_path)
+        handle.proxy.fog_wrapper.get_file(key, proxy_path)
         opts[:upload] = true
       else
         raise ArgumentError, "Invalid mode #{opts.mode.inspect}"
@@ -53,12 +56,12 @@ module Defog
     end
 
     def initialize(opts={}, &block) #:nodoc:
-      @defog = opts.keyword_args(:proxy => :required, :mode => :required, :key => :required, :proxy_path => :required, :upload => :optional, :persist => :optional)
-      super(@defog.proxy_path, @defog.mode, &block)
+      @defog = opts.keyword_args(:handle => :required, :mode => :required, :upload => :optional, :persist => :optional)
+      super(@defog.handle.proxy_path, @defog.mode, &block)
     end
 
-    # Closes the proxy file and, in the common case, synchronizes the cloud storage
-    # then deletes the proxy file.
+    # Closes the proxy file and synchronizes the cloud storage (if it was
+    # opened as writeable) then deletes the proxy file.
     #
     # Synchronization can be suppressed by passing the option
     #    :synchronize => false
@@ -74,9 +77,10 @@ module Defog
     def close(opts={})
       opts = opts.keyword_args(:persist => @defog.persist, :synchronize => true)
       super()
-      if @defog.proxy_path.exist?
-        @defog.proxy.fog_wrapper.put_file(@defog.key, @defog.proxy_path) if @defog.upload and opts.synchronize
-        @defog.proxy_path.unlink unless opts.persist
+      handle = @defog.handle
+      if handle.proxy_path.exist?
+        handle.proxy.fog_wrapper.put_file(handle.key, handle.proxy_path) if @defog.upload and opts.synchronize
+        handle.proxy_path.unlink unless opts.persist
       end
     end
   end

data/lib/defog/fog_wrapper.rb

@@ -6,6 +6,7 @@ module Defog #:nodoc: all
   class FogWrapper #:nodoc: all
 
     attr_reader :location
+    attr_reader :fog_connection
     attr_reader :fog_directory
 
     def self.connect(opts={})
@@ -40,9 +41,9 @@ module Defog #:nodoc: all
 
       def initialize(opts={})
         opts = opts.keyword_args(:local_root => :required)
-        directory = Pathname.new(opts.local_root).realpath
-        @location = directory.to_s.gsub(%r{/},'_')
-        @fog_connection = Fog::Storage.new(:provider => provider, :local_root => directory)
+        @local_root = Pathname.new(opts.local_root).realpath
+        @location = @local_root.to_s.gsub(%r{/},'_')
+        @fog_connection = Fog::Storage.new(:provider => provider, :local_root => @local_root)
         @fog_directory = @fog_connection.directories.get('.')
       end
 
@@ -50,6 +51,15 @@ module Defog #:nodoc: all
         Digest::MD5.hexdigest(fog_head(key).body)
       end
 
+      def url(key, expiry)
+        localpath = Pathname.new("#{@local_root}/#{key}").expand_path
+        if defined?(Rails)
+          relative = localpath.relative_path_from Rails.root + "public" rescue nil
+          return "/" + relative.to_s if relative and not relative.to_s.start_with? "../"
+        end
+        "file://#{localpath}"
+      end
+
     end
 
     class Aws < FogWrapper
@@ -67,6 +77,10 @@ module Defog #:nodoc: all
         fog_head(key).content_md5
       end
 
+      def url(key, expiry)
+        fog_head(key).url(expiry)
+      end
+
     end
   end
 end

data/lib/defog/handle.rb

@@ -0,0 +1,84 @@
+module Defog
+  # Create a Defog::Handle proxy instance via Defog::Proxy#file, such as
+  #
+  #    defog = Defog::Proxy.new(:provider => :AWS, :aws_access_key_id => access_key, ...)
+  #
+  #    handle = defog.file("key/to/my/file")
+  #
+  # or
+  #
+  #    defog.file("key/to/my/file") do |handle|
+  #       # ... access the proxy handle ...
+  #    end
+  #
+  # The #proxy_path attribute method returns a <code>Pathname</code>
+  # giving the local proxy file location.  Querying the attribute does 
+  # <i>not</i> upload, download, synchronize, or otherwise interact with
+  # the cloud or local proxy file in any way -- just returns a constructed
+  # a Pathname.
+  #  
+  class Handle
+
+    attr_reader :key
+    attr_reader :proxy #:nodoc:
+    attr_reader :proxy_path
+
+    def initialize(proxy, key) #:nodoc:
+      @proxy = proxy
+      @key = key
+      @proxy_path = Pathname.new("#{@proxy.proxy_root}/#{@key}").expand_path
+    end
+
+    # Returns true if the remote cloud file exists 
+    def exist?
+      !!@proxy.fog_wrapper.fog_head(@key)
+    end
+
+    # Deletes the remote cloud file
+    def delete
+      @proxy.fog_wrapper.fog_head(@key).destroy
+    end
+
+    # Returns a URL to access the remote cloud file.
+    #
+    # The option
+    #    :expiry => time
+    # Specifies the expiration of time-limited URLS when using :AWS.  The default is
+    # <code>Time.now + 10.minutes</code>.  The expiry is ignored when using :local
+    #
+    # For :local cloud files, if Rails is defined and the file is in the
+    # Rails app's public directory, returns a site path relative to
+    # the public directory.  Otherwise returns a <code>"file://"</code> URL 
+    def url(opts={})
+      opts = opts.keyword_args(:expiry => Time.now + 10*60)
+      @proxy.fog_wrapper.url(@key, opts.expiry)
+    end
+
+    # Returns the underlying Fog::Model, should you need it for something.
+    def fog_model
+      @proxy.fog_wrapper.fog_head(@key)
+    end
+
+    # Returns a Defog::File object, which is a specialization of ::File. 
+    #
+    # <code>mode</code> can be "r", "r+", "w", "w+", "a", or "a+" with the
+    # usual semantics.  When opened in a readable mode ("r", "r+", "w+",
+    # "a+"), first caches the cloud file in the local proxy.  When opened
+    # in a writeable mode ("r+", "w", "w+", "a", "a+"), arranges to upload
+    # the changes back to the cloud file at close time.
+    #
+    # Like ::File.open, if called with a block yields the file object to
+    # the block and ensures the file will be closed when leaving the block.
+    #
+    # Normally upon close the proxy file is synchronized as needed and then deleted.
+    # Pass
+    #    :persist => true
+    # to suppress deleting the file and so maintain the file after closing.   See File#close for more
+    # details.
+    def open(mode, opts={}, &block)
+      opts = opts.keyword_args(:persist)
+      File.get(opts.merge(:handle => self, :mode => mode), &block)
+    end
+
+  end
+end

data/lib/defog/proxy.rb

@@ -59,26 +59,35 @@ module Defog
       @fog_wrapper.location
     end
 
-    # Proxy a remote cloud file.  Returns a Defog::File object, which is a
-    # specialization of ::File. 
-    #
-    # <code>key</code> is the cloud storage key for the file.
-    #
-    # <code>mode</code> can be "r", "r+", "w", "w+", "a", or "a+" with the
-    # usual semantics.
+    # Returns the underlying Fog::Storage object for the cloud connection
+    def fog_connection
+      @fog_wrapper.fog_connection
+    end
+    
+    # Returns the Fog directory object for the root of the cloud files
+    def fog_directory
+      @fog_wrapper.fog_directory
+    end
+
+    # Proxy a remote cloud file.  Returns a Defog::Handle object that
+    # represents the file. 
     #
-    # Like ::File.open, if called with a block yields the file object to
-    # the block and ensures the file will be closed when leaving the block.
+    # If a <code>mode</code> is specified given opens a proxy file via
+    # Defog::Handle#open (passing it the mode and other options and
+    # optional block), returning instead the Defog::File object.
     #
-    # Normally the proxy file is synchronized and then deleted upon close.
-    # Pass
-    #    :persist => true
-    # to maintain the file after closing.   See File#close for more
-    # details.
+    # Thus 
+    #    proxy.file("key", mode, options, &block)
+    # is shorthand for
+    #    proxy.file("key").open(mode, options, &block)
     #
-    def file(key, mode, opts={}, &block)
-      opts = opts.keyword_args(:persist)
-      File.get(opts.merge(:proxy => self, :key => key, :mode => mode), &block)
+    def file(key, mode=nil, opts={}, &block)
+      handle = Handle.new(self, key)
+      case
+      when mode then handle.open(mode, opts, &block) if mode
+      when block then block.call(handle)
+      else handle
+      end
     end
 
   end

data/lib/defog/version.rb

@@ -1,3 +1,3 @@
 module Defog
-  VERSION = "0.0.1"
+  VERSION = "0.1.0"
 end

data/spec/file_spec.rb

@@ -229,29 +229,6 @@ describe "Defog::Proxy::File" do
 
   end
 
-  def key
-    example.metadata[:full_description].gsub(/\+/,'plus').gsub(/\W/,'-') + "/filename"
-  end
-
-  def create_remote(body)
-    @proxy.fog_wrapper.fog_directory.files.create(:key => key, :body => body)
-  end
-
-  def proxy_path
-    Pathname.new("#{@proxy.proxy_root}/#{key}").expand_path
-  end
-
-  def create_proxy(body)
-    path = proxy_path
-    path.dirname.mkpath
-    path.open("w") do |f|
-      f.write(body)
-    end
-  end
-
-  def remote_body
-    @proxy.fog_wrapper.fog_directory.files.get(key).body
-  end
 
 
 end

data/spec/handle_spec.rb

@@ -0,0 +1,92 @@
+require 'spec_helper'
+
+shared_examples "a handle" do |proxyargs|
+
+  before(:all) do
+    @proxy = Defog::Proxy.new(proxyargs)
+  end
+
+  before(:each) do
+    @handle = @proxy.file(key)
+  end
+
+  it "should report exist? true if remode cloud file exists" do
+    create_remote("i exist")
+    @handle.should be_exist
+  end
+
+  it "should report exist? false if remode cloud file does not exists" do
+    @handle.should_not be_exist
+  end
+
+  it "should delete a remote cloud file" do
+    create_remote("delete me")
+    remote_exist?.should be_true
+    @handle.delete
+    remote_exist?.should be_false
+  end
+
+  it "should return a URL to a file" do
+    create_remote("reach out to me")
+    @handle.url.should be_a String
+  end
+
+  it "should open a file" do
+    Defog::File.should_receive(:open).with(hash_including(:handle => @handle, :mode => "w"))
+    @handle.open("w")
+  end
+
+  it "should return a Fog model" do
+    create_remote("foggy")
+    @handle.fog_model.body.should == "foggy"
+  end
+
+
+end
+
+describe Defog::Handle do
+
+  context "Local" do
+    before(:all) do
+      Fog.unmock!
+    end
+
+    args = {:provider => :local, :local_root => LOCAL_CLOUD_PATH}
+
+    it_should_behave_like "a handle", args
+
+    it "should return a file:// URL" do
+      @proxy = Defog::Proxy.new(args)
+      @proxy.file(key).url.should == "file://" + (LOCAL_CLOUD_PATH + key).to_s
+    end
+
+    context "with a rails app" do
+
+      it "should return a path relative to public if in public" do
+        with_rails_defined do
+          @proxy = Defog::Proxy.new(:provider => :local, :local_root => (Rails.root + "public/defog").tap(&:mkpath))
+          @proxy.file(key).url.should == "/defog/#{key}"
+        end
+      end
+
+      it "should return a file:// path if not in public" do
+        with_rails_defined do
+          @proxy = Defog::Proxy.new(args)
+          @proxy.file(key).url.should == "file://" + (LOCAL_CLOUD_PATH + key).to_s
+        end
+      end
+    end
+
+  end
+
+  context "AWS" do
+    before(:all) do
+      Fog.mock!
+    end
+
+    args = {:provider => :AWS, :aws_access_key_id => "dummyid", :aws_secret_access_key => "dummysecret", :region => "eu-west-1", :bucket => "tester"}
+    it_should_behave_like "a handle", args
+
+  end
+
+end

data/spec/proxy_spec.rb

@@ -8,12 +8,9 @@ shared_examples "a proxy" do |args|
   end
 
   it "should default proxy root to Rails.root" do
-    begin
-      Kernel.const_set("Rails", Struct.new(:root).new("/dummy/rails/app/"))
+    with_rails_defined do
       proxy = Defog::Proxy.new(args)
       proxy.proxy_root.should == Rails.root + "defog" + proxy.provider.to_s + proxy.location
-    ensure
-      Kernel.send :remove_const, "Rails"
     end
   end
 
@@ -23,6 +20,42 @@ shared_examples "a proxy" do |args|
     proxy.proxy_root.should == path
   end
 
+  context do
+    before(:each) do
+      @proxy = Defog::Proxy.new(args)
+    end
+
+    it "file should return a handle" do
+      handle = @proxy.file(key)
+      handle.proxy.should == @proxy
+      handle.key.should == key
+    end
+
+    it "file should yield a handle" do
+      ret = @proxy.file(key) do |handle|
+        handle.proxy.should == @proxy
+        handle.key.should == key
+        123
+      end
+      ret.should == 123
+    end
+
+    it "should forward file open to handle" do
+      Defog::Handle.should_receive(:new).with(@proxy, key).and_return { mock('Handle').tap { |handle|
+        handle.should_receive(:open).with("r+", :persist => true)
+      } }
+      @proxy.file(key, "r+", :persist => true)
+    end
+
+    it "should return fog storage" do
+      @proxy.fog_connection.should == @proxy.fog_directory.connection
+    end
+
+    it "should return fog directory" do
+      create_remote("hello")
+      @proxy.fog_directory.files.get(key).body.should == "hello"
+    end
+  end
 
 end
 

data/spec/spec_helper.rb

@@ -12,14 +12,15 @@ require 'defog'
 Dir["#{File.dirname(__FILE__)}/support/**/*.rb"].each {|f| require f}
 
 RSpec.configure do |config|
-  
+  config.include Helpers
 end
 
 RSPEC_TMP_PATH = Pathname.new(__FILE__).dirname + "tmp"
 PROXY_BASE_PATH = RSPEC_TMP_PATH + "proxy"
 LOCAL_CLOUD_PATH = RSPEC_TMP_PATH + "cloud"
+RAILS_ROOT_PATH = RSPEC_TMP_PATH + "railsapp"
 
-[PROXY_BASE_PATH, LOCAL_CLOUD_PATH].each do |path|
+[PROXY_BASE_PATH, LOCAL_CLOUD_PATH, RAILS_ROOT_PATH].each do |path|
   path.rmtree if path.exist?
   path.mkpath
 end

data/spec/support/helpers.rb

@@ -0,0 +1,42 @@
+module Helpers
+
+  def key
+    # returns a key that will be different for each example, to avoid any
+    # cross-example interactions
+    example.metadata[:full_description].gsub(/\+/,'plus').gsub(/\W/,'-') + "/filename"
+  end
+
+  def create_remote(body)
+    @proxy.fog_directory.files.create(:key => key, :body => body)
+  end
+
+  def proxy_path
+    @proxy.file(key).proxy_path
+  end
+
+  def create_proxy(body)
+    path = proxy_path
+    path.dirname.mkpath
+    path.open("w") do |f|
+      f.write(body)
+    end
+  end
+
+  def remote_body
+    @proxy.fog_directory.files.get(key).body
+  end
+
+  def remote_exist?
+    !!@proxy.fog_directory.files.head(key)
+  end
+
+  def with_rails_defined
+    begin
+      Kernel.const_set("Rails", Struct.new(:root).new(RAILS_ROOT_PATH))
+      yield
+    ensure
+      Kernel.send :remove_const, "Rails"
+    end
+  end
+
+end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: defog
 version: !ruby/object:Gem::Version
-  version: 0.0.1
+  version: 0.1.0
   prerelease: 
 platform: ruby
 authors:
@@ -13,7 +13,7 @@ date: 2012-04-26 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: fog
-  requirement: &70247567066900 !ruby/object:Gem::Requirement
+  requirement: &70310792245820 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -21,10 +21,10 @@ dependencies:
         version: '0'
   type: :runtime
   prerelease: false
-  version_requirements: *70247567066900
+  version_requirements: *70310792245820
 - !ruby/object:Gem::Dependency
   name: hash_keyword_args
-  requirement: &70247567066480 !ruby/object:Gem::Requirement
+  requirement: &70310792245400 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -32,10 +32,10 @@ dependencies:
         version: '0'
   type: :runtime
   prerelease: false
-  version_requirements: *70247567066480
+  version_requirements: *70310792245400
 - !ruby/object:Gem::Dependency
   name: rake
-  requirement: &70247567066040 !ruby/object:Gem::Requirement
+  requirement: &70310792244960 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -43,10 +43,10 @@ dependencies:
         version: '0'
   type: :development
   prerelease: false
-  version_requirements: *70247567066040
+  version_requirements: *70310792244960
 - !ruby/object:Gem::Dependency
   name: rspec
-  requirement: &70247567065600 !ruby/object:Gem::Requirement
+  requirement: &70310792244520 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -54,10 +54,10 @@ dependencies:
         version: '0'
   type: :development
   prerelease: false
-  version_requirements: *70247567065600
+  version_requirements: *70310792244520
 - !ruby/object:Gem::Dependency
   name: simplecov
-  requirement: &70247567065000 !ruby/object:Gem::Requirement
+  requirement: &70310792243920 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -65,10 +65,10 @@ dependencies:
         version: '0'
   type: :development
   prerelease: false
-  version_requirements: *70247567065000
+  version_requirements: *70310792243920
 - !ruby/object:Gem::Dependency
   name: simplecov-gem-adapter
-  requirement: &70247567064460 !ruby/object:Gem::Requirement
+  requirement: &70310792243380 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -76,7 +76,7 @@ dependencies:
         version: '0'
   type: :development
   prerelease: false
-  version_requirements: *70247567064460
+  version_requirements: *70310792243380
 description: Wrapper to fog gem, proxying access to cloud files as local files.
 email:
 - ronen@barzel.org
@@ -94,13 +94,15 @@ files:
 - lib/defog/error.rb
 - lib/defog/file.rb
 - lib/defog/fog_wrapper.rb
+- lib/defog/handle.rb
 - lib/defog/proxy.rb
 - lib/defog/version.rb
 - spec/.gitignore
 - spec/file_spec.rb
+- spec/handle_spec.rb
 - spec/proxy_spec.rb
 - spec/spec_helper.rb
-- spec/support/connections.rb
+- spec/support/helpers.rb
 homepage: http://github.com/ronen/defog
 licenses: []
 post_install_message: 
@@ -128,7 +130,8 @@ summary: Wrapper to fog gem, proxying access to cloud files as local files.  Acc
   can be read-only (local cache), write-only (upload), or read-write (mirror)
 test_files:
 - spec/file_spec.rb
+- spec/handle_spec.rb
 - spec/proxy_spec.rb
 - spec/spec_helper.rb
-- spec/support/connections.rb
+- spec/support/helpers.rb
 has_rdoc: