modulation 0.27 → 0.28

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 04f2ab2cc42f117c9c8fb1b306b2239fbe9bca4a96f890e693bb92fb6c900318
-  data.tar.gz: 9ead222ae4fc275174b3e5d62a3b3b30e094761e5801de7b3a9fe21a346174be
+  metadata.gz: 630df5ec1eef61abafa86781e8f73c892dcf01c738bc02b695e767586208c728
+  data.tar.gz: 4f98dbd921afc4c3e0f33d82d19083e56cfacffee7192bf0ad621e638b5cf6a0
 SHA512:
-  metadata.gz: 2bd5727f5ca807fcc6238b0679234ba5df79102eaa6153a5acb8db8eb66054a1d9bdb2f8250292cad4784101c1c58b343766b19a3a07be013dfb67ce13b7c175
-  data.tar.gz: 63a1bc298932c0b1d4bbf804099df2940755b23358f4e4c72023c61054d03102c97f4138324570821685eaeb82bd0101146c9a4d469f245d8af093b906a623bd
+  metadata.gz: 726ac2711660e422fac0caeeb41828d4fafea79c3ef6da7abb24ea0160990248b2c1830b4c69f278b2f798a0038bd9c2b361dc8152bd33f726bda4f67eb71fca
+  data.tar.gz: 4ccfd538a57c5a8183409e42e04d9a5df14f22068d2900b269841690fbec878473c1b52ed84e8e2ec1b87736a4f915463d829670731f80e732fe1dee8068cc26

data/CHANGELOG.md

@@ -1,7 +1,13 @@
+0.28 2019-08-23
+---------------
+
+* Implement tagged sources
+* Improve application packer
+
 0.27 2019-08-21
 ---------------
 
-* Add initial packing functionality.
+* Add initial packing functionality
 
 0.26 2019-08-20
 ---------------

data/README.md

@@ -25,6 +25,28 @@ a functional style, minimizing boilerplate code.
 > Modulation, it is not intended as a comprehensive solution for using 
 > third-party libraries.
 
+## Features
+
+- Provides complete isolation of each module: constant definitions in one file
+  do not leak into another.
+- Enforces explicit exporting and importing of methods, classes, modules and 
+  constants.
+- Supports circular dependencies.
+- Supports [default exports](#default-exports) for modules exporting a single
+  class or value.
+- Modules can be [lazy loaded](#lazy-loading) to improve start up time and
+  memory consumption.
+- Modules can be [reloaded](#reloading-modules) at runtime without breaking your 
+  code in wierd ways.
+- Allows [mocking of dependencies](#mocking-dependencies) for testing purposes.
+- Can be used to [write gems](#writing-gems-using-modulation).
+- Module dependencies can be [introspected](#dependency-introspection).
+- Facilitates [unit-testing](#unit-testing-modules) of private methods and
+  constants.
+- Can load all source files in directory [at once](#importing-all-source-files-in-a-directory).
+- Packs entire applications [into a single
+  file](#packing-applications-with-modulation).
+
 ## Rationale
 
 You're probably asking yourself "what the ****?" , but when your Ruby app grows
@@ -65,33 +87,12 @@ are hidden unless explicitly exported, and the global namespace remains
 clutter-free. All dependencies between source files are explicit, visible, and 
 easy to understand.
 
-## Features
-
-- Provides complete isolation of each module: constant definitions in one file
-  do not leak into another.
-- Enforces explicit exporting and importing of methods, classes, modules and 
-  constants.
-- Supports circular dependencies.
-- Supports [default exports](#default-exports) for modules exporting a single
-  class or value.
-- Modules can be [lazy loaded](#lazy-loading) to improve start up time and
-  memory consumption.
-- Modules can be [reloaded](#reloading-modules) at runtime without breaking your 
-  code in wierd ways.
-- Allows [mocking of dependencies](#mocking-dependencies) for testing purposes.
-- Can be used to [write gems](#writing-gems-using-modulation).
-- Module dependencies can be [introspected](#dependency-introspection).
-- Facilitates [unit-testing](#unit-testing-modules) of private methods and
-  constants.
-- Can load all source files in directory [at once](#importing-all-source-files-in-a-directory).
-- Pack entire applications [into a single file](#packing-applications-with-modulation).
-
 ## Installing Modulation
 
-You can install the Modulation as a gem, or add it in your `Gemfile`:
+You can install the Modulation using `gem install`, or add it to your `Gemfile`:
 
-```bash
-$ gem install modulation
+```ruby
+gem 'modulation'
 ```
 
 ## Organizing your code with Modulation
@@ -198,8 +199,18 @@ User = import('./models')::User
 user = User.new(...)
 ```
 
-> **Note about paths**: module paths are always relative to the file
-> calling the `#import` method, just like `#require_relative`.
+### Using tags to designate common subdirectories
+
+Normally, module paths are always relative to the file calling the `#import`
+method, just like `#require_relative`. This can become a problem once you start
+moving your source files around. In addition, in applications where your source
+files are arranged in multiple directories, it can quickly become tedious to do
+stuff like `Post = import('../models/post')`.
+
+Modulation provides an alternative to relative paths in the form of tagged
+sources. A tagged source is simply a path associated with a label. For example,
+an application may tag `lib/models` simply as `@models`. Once tags are defined,
+they can be used when importing files, e.g. `import('@models/post')`.
 
 ### Importing all source files in a directory
 
@@ -484,6 +495,31 @@ settings = import('settings')
 settings = settings.__reload!
 ```
 
+### Retaining state between reloads
+
+Before a module is reloaded, all of its methods and constants are removed. In
+some cases, a module might need to retain state across reloads. You can do this
+by simply using instance variables:
+
+```ruby
+export :value, :inc
+
+@counter ||= 0
+
+def value
+  @counter
+end
+
+def incr
+  @counter += 1
+end
+```
+
+Care must be taken not to reassign values outside of methods, as this will
+overwrite any value retained in the instance variable. To assign initial values,
+use the `||=` operator as in the example above. See also the
+[reload example](examples/reload).
+
 ## Dependency introspection
 
 Modulation allows runtime introspection of dependencies between modules. You can

data/bin/mdl

@@ -10,6 +10,10 @@ def self.run
     mod = import(File.expand_path(fn))
     mod.send(method) if method
   end
+rescue StandardError => e
+  backtrace = e.backtrace.reject { |l| l =~ /^(#{Modulation::DIR})|(bin\/mdl)/ }
+  e.set_backtrace(backtrace)
+  raise e
 end
 
 def collect_deps(fn, paths)
@@ -31,16 +35,8 @@ def self.deps
 end
 
 def self.pack
-  paths = []
-  ARGV.each do |arg|
-    fn, method = (arg =~ /^([^\:]+)\:(.+)$/) ? [$1, $2.to_sym] : [arg, :main]
-    mod = import(File.expand_path(fn))
-    paths << File.expand_path(fn)
-    mod.__traverse_dependencies { |m| paths << m.__module_info[:location] }
-  end
-  
-  require 'modulation/packing'
-  STDOUT << Modulation::Packing.pack(paths, hide_filenames: true)
+  require 'modulation/packer'
+  STDOUT << Modulation::Packer.pack(ARGV, hide_filenames: true)
 end
 
 cmd = ARGV.shift

data/lib/modulation/builder.rb

@@ -100,6 +100,63 @@ module Modulation
         mod.__exported_symbols.clear
         mod.__reset_dependencies
       end
+
+      # Adds all or part of a module's methods to a target object
+      # If no symbols are given, all methods are added
+      # @param mod [Module] imported module
+      # @param target [Object] object to add methods to
+      # @param symbols [Array<Symbol>] list of methods to add
+      # @return [void]
+      def add_module_methods(mod, target, *symbols)
+        methods = mod.singleton_class.instance_methods(false)
+        unless symbols.empty?
+          symbols.select! { |s| s =~ /^[a-z]/ }
+          methods = filter_exported_symbols(methods, symbols)
+        end
+        methods.each do |sym|
+          target.send(:define_method, sym, &mod.method(sym))
+        end
+      end
+
+      # Adds all or part of a module's constants to a target object
+      # If no symbols are given, all constants are added
+      # @param mod [Module] imported module
+      # @param target [Object] object to add constants to
+      # @param symbols [Array<Symbol>] list of constants to add
+      # @return [void]
+      def add_module_constants(mod, target, *symbols)
+        exported = mod.__module_info[:exported_symbols]
+        unless symbols.empty?
+          symbols.select! { |s| s =~ /^[A-Z]/ }
+          exported = filter_exported_symbols(exported, symbols)
+        end
+        mod.singleton_class.constants(false).each do |sym|
+          next unless exported.include?(sym)
+
+          target.const_set(sym, mod.singleton_class.const_get(sym))
+        end
+      end
+
+      def filter_exported_symbols(exported, requested)
+        not_exported = requested - exported
+        unless not_exported.empty?
+          raise NameError, "symbol #{not_exported.first.inspect} not exported"
+        end
+
+        exported & requested
+      end
+
+      # Defines a const_missing method used for auto-importing on a given object
+      # @param receiver [Object] object to receive the const_missing method call
+      # @param auto_import_hash [Hash] a hash mapping constant names to a source
+      #   file and a caller location
+      # @return [void]
+      def define_auto_import_const_missing_method(receiver, auto_import_hash)
+        receiver.singleton_class.define_method(:const_missing) do |sym|
+          (path, caller_location) = auto_import_hash[sym]
+          path ? const_set(sym, import(path, caller_location)) : super(sym)
+        end
+      end
     end
   end
 end

data/lib/modulation/core.rb

@@ -32,8 +32,7 @@ module Modulation
     # @param caller_location [String] caller location
     # @return [Module] loaded module object
     def import(path, caller_location = caller(1..1).first)
-      abs_path = Paths.absolute_path(path, caller_location) ||
-                 Paths.lookup_gem_path(path)
+      abs_path = Paths.process(path, caller_location)
 
       case abs_path
       when String
@@ -71,63 +70,6 @@ module Modulation
       end
     end
 
-    # Adds all or part of a module's methods to a target object
-    # If no symbols are given, all methods are added
-    # @param mod [Module] imported module
-    # @param target [Object] object to add methods to
-    # @param symbols [Array<Symbol>] list of methods to add
-    # @return [void]
-    def add_module_methods(mod, target, *symbols)
-      methods = mod.singleton_class.instance_methods(false)
-      unless symbols.empty?
-        symbols.select! { |s| s =~ /^[a-z]/ }
-        methods = filter_exported_symbols(methods, symbols)
-      end
-      methods.each do |sym|
-        target.send(:define_method, sym, &mod.method(sym))
-      end
-    end
-
-    # Adds all or part of a module's constants to a target object
-    # If no symbols are given, all constants are added
-    # @param mod [Module] imported module
-    # @param target [Object] object to add constants to
-    # @param symbols [Array<Symbol>] list of constants to add
-    # @return [void]
-    def add_module_constants(mod, target, *symbols)
-      exported = mod.__module_info[:exported_symbols]
-      unless symbols.empty?
-        symbols.select! { |s| s =~ /^[A-Z]/ }
-        exported = filter_exported_symbols(exported, symbols)
-      end
-      mod.singleton_class.constants(false).each do |sym|
-        next unless exported.include?(sym)
-
-        target.const_set(sym, mod.singleton_class.const_get(sym))
-      end
-    end
-
-    def filter_exported_symbols(exported, requested)
-      not_exported = requested - exported
-      unless not_exported.empty?
-        raise NameError, "symbol #{not_exported.first.inspect} not exported"
-      end
-
-      exported & requested
-    end
-
-    # Defines a const_missing method used for auto-importing on a given object
-    # @param receiver [Object] object to receive the const_missing method call
-    # @param auto_import_hash [Hash] a hash mapping constant names to a source
-    #   file and a caller location
-    # @return [void]
-    def define_auto_import_const_missing_method(receiver, auto_import_hash)
-      receiver.singleton_class.define_method(:const_missing) do |sym|
-        (path, caller_location) = auto_import_hash[sym]
-        path ? const_set(sym, import(path, caller_location)) : super(sym)
-      end
-    end
-
     # Creates a new module from a source file
     # @param path [String] source file name
     # @return [Module] module
@@ -180,6 +122,10 @@ module Modulation
     ensure
       @loaded_modules[path] = old_module if block_given?
     end
+
+    def add_tags(tags)
+      Paths.add_tags(tags, caller(1..1).first)
+    end
   end
 end
 

data/lib/modulation/ext.rb

@@ -45,7 +45,7 @@ class Module
 
   def setup_auto_import_registry
     @__auto_import_registry = {}
-    Modulation.define_auto_import_const_missing_method(
+    Modulation::Builder.define_auto_import_const_missing_method(
       self,
       @__auto_import_registry
     )
@@ -56,8 +56,8 @@ class Module
   # @return [void]
   def extend_from(path)
     mod = import(path, caller(1..1).first)
-    Modulation.add_module_methods(mod, self.class)
-    Modulation.add_module_constants(mod, self)
+    Modulation::Builder.add_module_methods(mod, self.class)
+    Modulation::Builder.add_module_constants(mod, self)
   end
 
   # Includes exported methods from the given file name in the receiver
@@ -67,8 +67,8 @@ class Module
   # @return [void]
   def include_from(path, *symbols)
     mod = import(path, caller(1..1).first)
-    Modulation.add_module_methods(mod, self, *symbols)
-    Modulation.add_module_constants(mod, self, *symbols)
+    Modulation::Builder.add_module_methods(mod, self, *symbols)
+    Modulation::Builder.add_module_constants(mod, self, *symbols)
   end
 
   # Aliases the given method only if the alias does not exist, implementing in

data/lib/modulation/packer.rb

@@ -0,0 +1,107 @@
+# frozen_string_literal: true
+
+require_relative '../modulation'
+require_relative '../modulation/version'
+require 'zlib'
+
+module Modulation
+  # Implements packing functionality
+  module Packer
+    BOOTSTRAP_CODE = <<~SRC.encode('ASCII-8BIT').chomp
+      # encoding: ASCII-8BIT
+      require 'bundler/inline'
+
+      gemfile do
+        source 'https://rubygems.org'
+        gem 'modulation', '~> %<modulation_version>s'
+      end
+
+      require 'modulation/packer'
+      Modulation::Bootstrap.setup(DATA, %<dictionary>s)
+      import(%<entry_point>s).send(:main)
+      __END__
+      %<data>s
+    SRC
+
+    def self.pack(paths, _options = {})
+      paths = [paths] unless paths.is_a?(Array)
+      deps = collect_dependencies(paths)
+      entry_point_filename = File.expand_path(paths.first)
+      dictionary, data = generate_packed_data(deps)
+      generate_bootstrap(dictionary, data, entry_point_filename)
+    end
+
+    def self.collect_dependencies(paths)
+      paths.each_with_object([]) do |fn, deps|
+        mod = import(File.expand_path(fn))
+        deps << File.expand_path(fn)
+        mod.__traverse_dependencies { |m| deps << m.__module_info[:location] }
+      end
+    end
+
+    def self.generate_packed_data(deps)
+      files = deps.each_with_object({}) do |path, dict|
+        dict[path] = IO.read(path)
+      end
+      # files[INLINE_GEMFILE_PATH] = generate_gemfile
+      pack_files(files)
+    end
+
+    def self.pack_files(files)
+      data = (+'').encode('ASCII-8BIT')
+      last_offset = 0
+      dictionary = files.each_with_object({}) do |(path, content), dict|
+        zipped = Zlib::Deflate.deflate(content)
+        size = zipped.bytesize
+        
+        data << zipped
+        dict[path] = [last_offset, size]
+        last_offset += size
+      end
+      [dictionary, data]
+    end
+
+    # def self.generate_gemfile
+    #   format(INLINE_GEMFILE_CODE)
+    # end
+
+    def self.generate_bootstrap(dictionary, data, entry_point)
+      format(BOOTSTRAP_CODE, modulation_version: Modulation::VERSION,
+                             dictionary: dictionary.inspect,
+                             entry_point: entry_point.inspect,
+                             data: data)
+    end
+  end
+
+  # Packed app bootstrapping code
+  module Bootstrap
+    def self.setup(data, dictionary)
+      patch_builder
+      @data = data
+      @data_offset = data.pos
+      @dictionary = dictionary
+    end
+
+    def self.patch_builder
+      class << Modulation::Builder
+        alias_method :orig_make, :make
+        def make(info)
+          if Modulation::Bootstrap.find(info[:location])
+            info[:source] = Modulation::Bootstrap.read(info[:location])
+          end
+          orig_make(info)
+        end
+      end
+    end
+
+    def self.find(path)
+      @dictionary[path]
+    end
+
+    def self.read(path)
+      (offset, length) = @dictionary[path]
+      @data.seek(@data_offset + offset)
+      Zlib::Inflate.inflate(@data.read(length))
+    end
+  end
+end

data/lib/modulation/paths.rb

@@ -4,14 +4,37 @@ module Modulation
   # Implements methods for expanding relative or incomplete module file names
   module Paths
     class << self
-      TAGGED_REGEXP = /^@(.+)$/.freeze
-
-      def tagged_path(path)
-        path[TAGGED_REGEXP, 1]
+      def process(path, caller_location)
+        tagged_path(path) || absolute_path(path, caller_location) ||
+          lookup_gem_path(path)
       end
 
       # Regexp for extracting filename from caller reference
-      CALLER_FILE_REGEXP = /^([^\:]+)\:/.freeze
+      CALLER_FILE_REGEXP = /^([^\:]+)\:?/.freeze
+      TAGGED_REGEXP = /^@([^\/]+)(\/.+)?$/.freeze
+
+      def add_tags(tags, caller_location)
+        caller_file = caller_location[CALLER_FILE_REGEXP, 1]
+        caller_dir = caller_file ? File.dirname(caller_file) : nil
+
+        @tags ||= {}
+        tags.each do |k, path|
+          @tags[k.to_s] = caller_dir ? File.expand_path(path, caller_dir) : path
+        end
+      end
+
+      def tagged_path(path)
+        return nil unless @tags
+
+        _, tag, path = path.match(TAGGED_REGEXP).to_a
+        return nil unless tag
+
+        base_path = @tags[tag]
+        return nil unless base_path
+
+        path = path ? File.join(base_path, path) : base_path
+        check_path(path)
+      end
 
       # Resolves the absolute path to the provided reference. If the file is not
       # found, will try to resolve to a gem

data/lib/modulation/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Modulation
-  VERSION = '0.27'
+  VERSION = '0.28'
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: modulation
 version: !ruby/object:Gem::Version
-  version: '0.27'
+  version: '0.28'
 platform: ruby
 authors:
 - Sharon Rosner
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2019-08-21 00:00:00.000000000 Z
+date: 2019-08-23 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: minitest
@@ -61,7 +61,7 @@ files:
 - lib/modulation/ext.rb
 - lib/modulation/gem.rb
 - lib/modulation/module_mixin.rb
-- lib/modulation/packing.rb
+- lib/modulation/packer.rb
 - lib/modulation/paths.rb
 - lib/modulation/version.rb
 homepage: http://github.com/digital-fabric/modulation

data/lib/modulation/packing.rb

@@ -1,89 +0,0 @@
-# frozen_string_literal: true
-
-require_relative '../modulation'
-require 'zlib'
-require 'digest/md5'
-
-module Modulation
-  # Implements packing functionality
-  module Packing
-    BOOTSTRAP = <<~SRC.encode('ASCII-8BIT')
-      # encoding: ASCII-8BIT
-      require 'modulation'
-      require 'zlib'
-
-      b = Modulation::Builder
-      z = Zlib::Inflate
-      b::C = {%<module_dictionary>s}
-
-      class << b
-        alias_method :orig_make, :make
-
-        def make(info)
-          (pr = Modulation::Builder::C.delete(info[:location])) ? pr.call : orig_make(info)
-        end
-      end
-
-      import(%<entry_point>s).send(:main)
-    SRC
-
-    MAKE_CODE = <<~SRC
-      proc { b.orig_make(location: %<location>s, source: %<source>s) }
-    SRC
-
-    UNZIP_CODE = 'z.inflate(%<data>s)'
-
-    # MAKE_CODE = <<~SRC
-    #   Modulation::Builder.orig_make(location: %s, source: %s)
-    # SRC
-
-    # UNCAN_CODE = <<~SRC
-    #   proc { RubyVM::InstructionSequence.load_from_binary(
-    #     Zlib::Inflate.inflate(%s)).eval
-    #   }
-    # SRC
-
-    def self.pack(paths, _options = {})
-      paths = [paths] unless paths.is_a?(Array)
-
-      entry_point_filename = nil
-      dictionary = paths.each_with_object({}) do |path, dict|
-        warn "Processing #{path}"
-        source = IO.read(path)
-        entry_point_filename ||= path
-        dict[path] = pack_module(path, source)
-      end
-
-      generate_bootstrap(dictionary, entry_point_filename)
-    end
-
-    def self.pack_module(path, source)
-      # code = (MAKE_CODE % [fn.inspect, module_source.inspect])
-      # seq = RubyVM::InstructionSequence.compile(code, options)
-      # canned = Zlib::Deflate.deflate(seq.to_binary)
-      # dict[fn] = UNCAN_CODE % canned.inspect
-
-      zipped = Zlib::Deflate.deflate(source)
-      code = format(UNZIP_CODE, data: zipped.inspect)
-      format(MAKE_CODE, location: path.inspect, source: code)
-    end
-
-    def self.generate_bootstrap(module_dictionary, entry_point)
-      format(
-        BOOTSTRAP,
-        module_dictionary: format_module_dictionary(module_dictionary),
-        entry_point: entry_point.inspect
-      ).chomp.gsub(/^\s+/, '').gsub(/\n+/, "\n")
-    end
-
-    def self.format_module_dictionary(module_dictionary)
-      module_dictionary.map do |fn, code|
-        format(
-          '%<filename>s => %<code>s',
-          filename: fn.inspect,
-          code: code
-        ).chomp
-      end.join(',')
-    end
-  end
-end