virtfs 0.0.1

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: 6347d643cd70f383dfe6e5738525ca3ddfc0b40d
+  data.tar.gz: 18b7fb18ba46b1958fc27e26cf2e071c5b502f3e
+SHA512:
+  metadata.gz: 791b4b067addfe0070a50e8a78d8ad826e87452e9deed22dda8318c6ca414c7458aa5003ec30b71a74c942eefa01131f637f717fdd445051da140d0c05ecd269
+  data.tar.gz: 226904fefb083d84c7535e6e58e893f1c0f1e5b4061ef63f80cfdb0b9f90b974b1d6c8eeb18b2e37cf726ddbf08bea6afcf11d6e2498e23cfcb86eb11e6c8a99

data/.gitignore

@@ -0,0 +1,18 @@
+*.gem
+*.rbc
+.bundle
+.config
+.yardoc
+Gemfile.lock
+InstalledFiles
+_yardoc
+coverage
+doc/
+lib/bundler/man
+pkg
+rdoc
+spec/reports
+test/tmp
+test/version_tmp
+tmp
+*swp

data/.rspec

@@ -0,0 +1,2 @@
+--format documentation
+--color

data/.travis.yml

@@ -0,0 +1,8 @@
+language: ruby
+rvm:
+- "2.2"
+sudo: false
+cache: bundler
+env:
+- FS_INTERFACE=thin
+- FS_INTERFACE=thick

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in virtfs.gemspec
+gemspec

data/LICENSE.txt

@@ -0,0 +1,22 @@
+Copyright (c) 2015 Richard Oliveri
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,154 @@
+# VirtFS
+
+[![Build Status](https://travis-ci.org/ManageIQ/virtfs.svg)](https://travis-ci.org/ManageIQ/virtfs)
+[![Code Climate](https://codeclimate.com/github/ManageIQ/virtfs/badges/gpa.svg)](https://codeclimate.com/github/ManageIQ/virtfs)
+
+A virtual (pseudo) filesystem facility for Ruby.
+
+Provides an infrastructure where instances of various filesystem **plugins** can be _mounted_ within the filesystem namespace of a Ruby process, and accessed via Ruby `File` and `Dir` classes and objects.
+
+Typically, these **plugins** implement a filesystem access paradigm on top of various storage mechanisms and/or programmatic data generation.
+
+**For example:** a plugin can be implemented that stores and maintains filesystem-like information in a database - say Berkeley DB - we'll call that plugin `BdbFS`, which is also the name of the Ruby class implementing the plugin. Instantiating an instance of this class will return an instance of the `BdbFS` filesystem, which can then be mounted and accessed through the **VirtFS** facility.
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+    gem 'virtfs'
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install virtfs
+
+## Usage
+### Mounting root
+By default, **VirtFS** does not mount the native filesystem of the system on which the Ruby process is running. It needs to be done explicitly. This is accomplished through the use of a **plugin** that simply _wraps_ the native filsystem in a class that enables its access through **VirtFS**.
+
+Typically, one of the first things a **VirtFS** client does, is _mount_ the native filesystem:
+```ruby
+require "virtfs-nativefs-thick"
+
+# Instantiate an instance of the native filesystem.
+native_fs = VirtFS::NativeFS::Thick.new
+
+# Mount the native filesystem on root "/"
+VirtFS.mount(native_fs, "/")
+```
+### Access
+
+Once the native filesystem is mounted, **VirtFS** can be accessed explicitly through the `VirtFS` class, or it can be _activated_, enabling access through Ruby's `File` and `Dir` classes.
+
+#### Explicit access
+```ruby
+require "virtfs-nativefs-thick"
+
+# Instantiate an instance of the native filesystem
+native_fs = VirtFS::NativeFS::Thick.new
+
+# Mount the native filesystem on root "/"
+VirtFS.mount(native_fs, "/")
+
+VirtFS::VDir.chdir("/etc")
+VirtFS::VDir.getwd        # => "/etc"
+
+VirtFS::VDir.foreach(this_dir) do |f|
+  puts f
+end
+
+# Note: this does not change the state of Ruby's standard filesystem.
+Dir.getwd                 # => the original cwd of the ruby process.
+```
+#### Activation
+```ruby
+require "virtfs-nativefs-thick"
+
+# Instantiate an instance of the native filesystem.
+native_fs = VirtFS::NativeFS::Thick.new
+
+# Mount the native filesystem on root "/"
+VirtFS.mount(native_fs, "/")
+
+VirtFS.activate!
+VirtFS.activated?          # => true
+
+# Dir and File classes now go through VirtFS.
+Dir.chdir("/etc")
+Dir.getwd                  # => "/etc"
+
+Dir.foreach(this_dir) do |f|
+  puts f
+end
+
+VirtFS.deactivate!
+VirtFS.activated?          # => false
+```
+Activation can be confined to a block through the use of the `with` class method:
+```ruby
+VirtFS.activated?          # => false
+VirtFS.with do
+  VirtFS.activated?        # => true
+end
+VirtFS.activated?          # => false
+```
+
+### Mounting other filesystems
+
+Let's use the, yet to be implemented, `BdbFS` plugin as an example:
+```ruby
+require "virtfs-nativefs-thick"
+require "virtfs-bdbfs"
+
+# The Berkeley DB file containing the filesystem data.
+bdbfs_file = "/usr/me/my_bdbfs_file"
+
+# Where to mount the BdbFS instance under the native filesystem.
+bdbfs_mount_point = "/usr/me/bdbfs_root"
+
+#
+# Mount the native filesystem.
+#
+native_fs = VirtFS::NativeFS::Thick.new
+VirtFS.mount(native_fs, "/")
+
+#
+# Instantiate a BdbFS instance and mount it.
+#
+bdb_fs = VirtFS::BdbFS.new(bdbfs_file)
+VirtFS.mount(bdb_fs, bdbfs_mount_point)
+
+VirtFS.with do
+  Dir.chdir(bdbfs_mount_point)
+  Dir.getwd                  # => "/usr/me/bdbfs_root"
+
+  # List the files at the root of the BdbFS.
+  Dir.foreach(this_dir) do |f|
+    puts f
+  end
+end
+```
+
+## Plugins
+
+There are two types of filesystem plugins for **VirtFS**; they are distinguished by the type of interface they present to the **VirtFS** infrastructure - the interface that **VirtFS** uses to call into the plugin.
+
+* **Thin** interface plugins implement the minimum low-level methods required by **VirtFS**. Most of the common filesystem functionality is provided by the **VirtFS** infrastructure itself (like buffered IO and character encoding), making this type of plugin easier to implement.
+* **Thick** interface plugins implement most, if not all, of the filesystem functionality within the plugin, overriding the common implementations within **VirtFS**. While these plugins require more work to implement, they do provide a measurable performance improvement.
+
+In reality, most plugins will be of the **thin** type. More often than not, **thick** plugins will be used to interface **VirtFS** to an underlying technology that already implements the full filesystem interface. An obvious example of this is the plugin that interfaces with Ruby's native filesystem interface.
+
+Notice in the examples above, when mounting the native filesystem on root, we used the `VirtFS::NativeFS::Thick` plugin class. This plugin passes most requests directly to the underlying Ruby filesystem implementation, bypassing the common implementations in **VirtFS**.
+
+There is a **thin** version of this plugin: `VirtFS::NativeFS::Thin`. It only calls down into the underlying Ruby filesystem implementation for low-level operations, relying on the common functionality provided by **VirtFS** for everything else. While this plugin is less performant than its **thick** counterpart, it's extremely useful in testing the common filesystem functionality implemented within **VirtFS**. In fact the _spec_ tests for **VirtFS** are run using both **thick** and **thin** interfaces, to ensure the expected results are identical. 
+
+## Contributing 
+
+1. Fork it
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Add some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create new Pull Request

data/Rakefile

@@ -0,0 +1,5 @@
+require "bundler/gem_tasks"
+Dir.glob('tasks/**/*.rake').each(&method(:import))
+
+task :test => :spec
+task :default => :spec

data/lib/virtfs-nativefs-thick.rb

@@ -0,0 +1 @@
+require_relative "virtfs/nativefs/thick"

data/lib/virtfs-nativefs-thin.rb

@@ -0,0 +1 @@
+require_relative "virtfs/nativefs/thin"

data/lib/virtfs.rb

@@ -0,0 +1,38 @@
+require 'pathname'
+
+VfsRealDir       = ::Dir
+VfsRealFile      = ::File
+VfsRealIO        = ::IO
+VfsRealPathname  = ::Pathname
+
+require_relative 'virtfs/version.rb'
+require_relative 'virtfs/exception.rb'
+require_relative 'virtfs/context.rb'
+require_relative 'virtfs/context_manager.rb'
+require_relative 'virtfs/file_modes_and_options.rb'
+require_relative 'virtfs/context_switch_class_methods.rb'
+require_relative 'virtfs/find_class_methods.rb'
+require_relative 'virtfs/activation.rb'
+require_relative 'virtfs/delegate_module.rb'
+require_relative 'virtfs/stat.rb'
+require_relative 'virtfs/thin_dir_delegator.rb'
+require_relative 'virtfs/thin_file_delegator.rb'
+require_relative 'virtfs/kernel'
+require_relative 'virtfs/v_pathname.rb'
+
+# VirtFS - Ruby Virtual File System Interface
+module VirtFS
+  @activated = false
+
+  extend Activation
+  extend DelegateModule
+  extend ContextSwitchClassMethods
+  extend FindClassMethods
+end
+
+require_relative 'virtfs/dir_instance_delegate.rb'
+require_relative 'virtfs/file_instance_delegate.rb'
+require_relative 'virtfs/io_instance_delegate.rb'
+require_relative 'virtfs/v_io.rb'
+require_relative 'virtfs/v_file.rb'
+require_relative 'virtfs/v_dir.rb'

data/lib/virtfs/activation.rb

@@ -0,0 +1,97 @@
+module VirtFS
+  module Activation
+    def activate_mutex
+      @activate_mutex ||= Mutex.new
+    end
+
+    # @return [Boolean] indicating if VirtFS is active
+    #
+    # @see .activate!
+    # @see .deactivate!
+    def activated?
+      @activated
+    end
+
+    # Overrides Ruby's native Dir, File, and IO classes
+    # with corresponding VirtFS classes
+    def activate!(enable_require = false)
+      activate_mutex.synchronize do
+        raise "VirtFS.activate! already activated" if @activated
+        @activated = true
+
+        Object.class_eval do
+          remove_const(:Dir)
+          remove_const(:File)
+          remove_const(:IO)
+          remove_const(:Pathname)
+
+          const_set(:Dir,      VirtFS::VDir)
+          const_set(:File,     VirtFS::VFile)
+          const_set(:IO,       VirtFS::VIO)
+          const_set(:Pathname, VirtFS::VPathname)
+        end
+
+        if enable_require
+          VirtFS::Kernel.inject
+          VirtFS::Kernel.enable
+        end
+      end
+      true
+    end
+
+    # Restores Ruby's native Dir, File, and IO classes
+    # to their defaults
+    def deactivate!
+      activate_mutex.synchronize do
+        raise "VirtFS.deactivate! not activated" unless @activated
+        @activated = false
+
+        Object.class_eval do
+          remove_const(:Dir)
+          remove_const(:File)
+          remove_const(:IO)
+          remove_const(:Pathname)
+
+          const_set(:Dir,      VfsRealDir)
+          const_set(:File,     VfsRealFile)
+          const_set(:IO,       VfsRealIO)
+          const_set(:Pathname, VfsRealPathname)
+        end
+        VirtFS::Kernel.disable
+      end
+      true
+    end
+
+    # Invokes the given block in an activated context
+    #
+    # @see .activate!
+    def with(enable_require = false)
+      if activated?
+        yield
+      else
+        begin
+          activate!(enable_require)
+          yield
+        ensure
+          deactivate!
+        end
+      end
+    end
+
+    # Invokes the given block in a deactivated context
+    #
+    # @see .deactivate!
+    def without
+      if !activated?
+        yield
+      else
+        begin
+          deactivate!
+          yield
+        ensure
+          activate!
+        end
+      end
+    end
+  end
+end

data/lib/virtfs/block_io.rb

@@ -0,0 +1,140 @@
+module VirtFS
+  # Block IO Adapter, used internally by VirtFS to perform read and write operations
+  # via an underlying 'block-like' interface.
+  #
+  # The class requires a handle to an object defining #raw_read and #raw_write methods
+  # facilitating raw block access.
+  #
+  # This class behaves similarily to a tradional disk device, providing a 'seek'
+  # position at which read/write operations occur after which the pos is updated
+  #
+  class BlockIO
+    MIN_BLOCKS_TO_CACHE = 64
+
+    # Optional absolute block offset, always applied
+    attr_accessor :offset
+
+    # BlockIO initializer
+    #
+    # @param io_obj [RawIO] instance of class defining #raw_read and #raw_write
+    #
+    def initialize(io_obj)
+      @io_obj          = io_obj
+      @block_size      = @io_obj.block_size   # Size of block in bytes
+      @size            = @io_obj.size         # Size of file in bytes
+      @size_in_blocks  = @size / @block_size  # Size of file in blocks
+      @start_byte_addr = 0
+      @end_byte_addr   = @size - 1
+      @lba_end         = @size_in_blocks - 1
+      @seek_pos        = 0
+      @cache_range     = Range.new(-1, -1)
+      @offset          = 0
+    end
+
+    # Read len bytes from the block device and update seek_pos
+    #
+    # @param len [Integer] number of bytes to read
+    # @return [Array<Byte>,nil] array of up to len bytes read from block device,
+    #   or nil if no bytes can be read
+    def read(len)
+      return nil if @seek_pos >= @end_byte_addr
+      len = @end_byte_addr - @seek_pos if (@seek_pos + len) > @end_byte_addr
+
+      start_sector, start_offset = @seek_pos.divmod(@block_size)
+      end_sector = (@seek_pos + len - 1) / @block_size
+      num_sector = end_sector - start_sector + 1
+
+      rbuf = bread_cached(start_sector, num_sector)
+      @seek_pos += len
+
+      rbuf[start_offset, len]
+    end
+
+    # Write buffer of specified length to block device and update seek_pos
+    #
+    # @param buf [Array<Byte>] bytes to write to disk device
+    # @return [Integer,nil] number of bytes written to device, or nil if none
+    #   can be written
+    def write(buf, len)
+      return nil if @seek_pos >= @end_byte_addr
+      len = @end_byte_addr - @seek_pos if (@seek_pos + len) > @end_byte_addr
+
+      start_sector, start_offset = @seek_pos.divmod(@block_size)
+      end_sector = (@seek_pos + len - 1) / @block_size
+      num_sector = end_sector - start_sector + 1
+
+      rbuf = bread(start_sector, num_sector)
+      rbuf[start_offset, len] = buf[0, len]
+
+      bwrite(start_sector, num_sector, rbuf)
+      @seek_pos += len
+
+      len
+    end
+
+    # Advance seek pointer via the specified mechanism
+    #
+    # @param amt [Integer] absolute amount which to advance seek pointer
+    # @param whence [IO::SEEK_SET, IO::SEEK_CUR, IO::SEEK_END] mechanism
+    #   which to update seek pos
+    #
+    # @see ::IO::SEEK_SET
+    # @see ::IO::SEEK_CUR
+    # @see ::IO::SEEK_END
+    #
+    def seek(amt, whence = IO::SEEK_SET)
+      case whence
+      when IO::SEEK_CUR
+        @seek_pos += amt
+      when IO::SEEK_END
+        @seek_pos = @end_byte_addr + amt
+      when IO::SEEK_SET
+        @seek_pos = amt + @start_byte_addr
+      end
+      @seek_pos
+    end
+
+    # @return [Integer] size of block device
+    def size
+      @size
+    end
+
+    # Close block device, after this no additional read/writes can occur
+    def close
+      @io_obj.close
+    end
+
+    private
+
+    def bread(start_sector, num_sectors)
+      # $log.debug "RawBlockIO.bread: start_sector = #{start_sector}, num_sectors = #{num_sectors}, @lba_end = #{@lba_end}"
+      return nil if start_sector > @lba_end
+      num_sectors = @size_in_blocks - start_sector if (start_sector + num_sectors) > @size_in_blocks
+      @io_obj.raw_read(start_sector * @block_size + @offset, num_sectors * @block_size)
+    end
+
+    def bwrite(start_sector, num_sectors, buf)
+      return nil if start_sector > @lba_end
+      num_sectors = @size_in_blocks - start_sector if (start_sector + num_sectors) > @size_in_blocks
+      @io_obj.raw_write(buf, start_sector * @block_size + @offset, num_sectors * @block_size)
+    end
+
+    def bread_cached(start_sector, num_sectors)
+      # $log.debug "RawBlockIO.bread_cached: start_sector = #{start_sector}, num_sectors = #{num_sectors}"
+      if !@cache_range.include?(start_sector) || !@cache_range.include?(start_sector + num_sectors - 1)
+        sectors_to_read = [MIN_BLOCKS_TO_CACHE, num_sectors].max
+        @cache = bread(start_sector, sectors_to_read)
+        sectors_read   = @cache.length / @block_size
+        end_sector     = start_sector + sectors_read - 1
+        @cache_range   = Range.new(start_sector, end_sector)
+      end
+
+      sector_offset = start_sector  - @cache_range.first
+      buffer_offset = sector_offset * @block_size
+      length        = num_sectors   * @block_size
+      # $log.debug "RawBlockIO.bread_cached: buffer_offset = #{buffer_offset}, length = #{length}"
+
+      @cache[buffer_offset, length]
+    end
+  end
+end