modulation 0.33 → 0.34

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 66ef0e0890aa7cba8f462f22cfc1ebbe81ab4cdc48b3ad9733958dc002370d85
-  data.tar.gz: aeda0965bda28cd93eb2b318f8cacc24ea342e49084230cf0cdb522d05f1182d
+  metadata.gz: 1b35f333f45c8017d3fd44700fc5cd3d16dcdd575be012b3f86f2af17c2fcaef
+  data.tar.gz: de1b7cf9b6cd1d39080fd2ef971d99b23fa2b91f18a90db2bbcacf9debfdd7c7
 SHA512:
-  metadata.gz: 12ea537e1662feb88a7ae35cf46a6b23e8970f4255088b20c32962792d0cc4bff4cbd611a1eadc431feb81cb768191917dc6a7eb19e2d9bc75c50ecb50e8caf4
-  data.tar.gz: 1a6b08a109333d8a4ef4513e1607ceb84f177d4287f7114699f9d1d48d7c29959d37b16205dc36eaf627a8418cd886bb6eed6a71782bad4cd8fe7adbf1e50826
+  metadata.gz: 914789190308c5b9133d4bc3f2a5b6209fe9123ba1d984e7e77b1062f6392b9cb09a4db32b44b631db631080e655bf7dfb32b1955693eb39d0573a85487dba4d
+  data.tar.gz: 2cba7fed4ae3c0a36abb3f8a2f1505074191ba52cfa7510022060730accf39a932e00b6e3323e7b0643ac8e6932f6046be4e10fa7bff435004a6ceb75492bd68

data/CHANGELOG.md

@@ -1,3 +1,10 @@
+0.34 2019-10-14
+---------------
+
+* Obfuscate bootstrapping code in packages
+* Move packer, bootstrap, CLI, creator code into stock modules
+* Improve README
+
 0.33 2019-10-02
 ---------------
 

data/README.md

@@ -28,21 +28,21 @@ a functional style, minimizing boilerplate code.
 
 ## Features
 
-- Complete isolation of each module.
-- Explicit exporting and importing of methods and constants.
-- Support for circular dependencies.
-- Support for [default exports](#default-exports) for modules exporting a single
-  class or value.
-- [Lazy Loading](#lazy-loading) improves start up time and memory consumption.
-- [Hot module reloading](#reloading-modules)
-- [Mocking of dependencies](#mocking-dependencies) for testing purposes.
-- Can be used to [write gems](#writing-gems-using-modulation).
-- [Dependency introspection](#dependency-introspection).
-- Support for [creating modules programmatically](#programmatic-module-creation).
-- Easier [unit-testing](#unit-testing-modules) of private methods and
-  constants.
-- Pack entire applications [into a single
-  file](#packing-applications-with-modulation).
+- **[Complete isolation of each module](#organizing-your-code-with-modulation)**
+  prevents literring of the global namespace.
+- **Explicit [exporting](#exporting-declarations) and
+  [importing](#importing-declarations) of methods and constants** lets you
+  control the public interface for each module, as well as keep track of all
+  dependencies in your code.
+- **[Lazy Loading](#lazy-loading)** improves start up time and memory
+  consumption.
+- **[Hot module reloading](#reloading-modules)** streamlines your development
+  process.
+- **[Dependency mocking](#mocking-dependencies)** facilitates testing.
+- **[Dependency introspection](#dependency-introspection)** lets you introspect
+  your dependencies at runtime.
+- **[Application packing](#packing-applications-with-modulation)** lets you
+  bundle your code in a single, optionally obfuscated file (WIP).
 
 ## Rationale
 
@@ -96,7 +96,7 @@ gem 'modulation'
 ## Organizing your code with Modulation
 
 Modulation builds on the idea of a Ruby `Module` as a
-["collection of methods and constants"](https://ruby-doc.org/core-2.5.1/Module.html).
+["collection of methods and constants"](https://ruby-doc.org/core-2.6.5/Module.html).
 Using modulation, each Ruby source file becomes a module. Modules usually
 export method and constant declarations (usually an API for a specific, 
 well-defined functionality) to be shared with other modules. Modules can also
@@ -110,6 +110,8 @@ operations such as [hot reloading](#reloading-modules).
 Modulation provides an alternative APIs for loading modules. Instead of using
 `require` and `require_relative`, you use `import`, `import_map` and other APIs.
 
+## Basic Usage
+
 ### Exporting declarations
 
 Any class, module or constant be exported using `#export`:
@@ -211,6 +213,16 @@ User = import('./models')::User
 user = User.new(...)
 ```
 
+### A word about paths
+
+Paths given to `import` are always considered relative to the importing file,
+unless they are absolute (e.g. `/home/dave/repo/my_app`), specify a
+[tag](#using-tags-to-designate-common-subdirectories) or reference a
+[gem](#importing-gems-using-modulation). This is true for all Modulation APIs
+that accept path arguments.
+
+## Advanced Usage
+
 ### Using tags to designate common subdirectories
 
 Normally, module paths are always relative to the file calling the `#import`
@@ -224,12 +236,25 @@ sources. A tagged source is simply a path associated with a label. For example,
 an application may tag `lib/models` simply as `@models`. Once tags are defined,
 they can be used when importing files, e.g. `import('@models/post')`.
 
+To define tags, use `Modulation.add_tags`:
+
+```ruby
+Modulation.add_tags(
+  models: '../lib/models',
+  views:  '../lib/views'
+)
+
+...
+
+User = import '@models/user'
+```
+
 ### Importing all source files in a directory
 
 To load all source files in a directory you can use `#import_all`:
 
 ```ruby
-import_all('./ext') # will load ./ext/kernel.rb, ./ext/socket.rb etc 
+import_all('./ext')
 ```
 
 Groups of modules providing a uniform interface can also be loaded using
@@ -323,6 +348,12 @@ config = import('./config')
 db.connect(config[:host], config[:port])
 ```
 
+### Circular dependencies
+
+Circular dependencies, while not the best practice for organizing a code base,
+are sometimes useful. Modulation supports circular dependencies, with the
+exception of modules with default exports.
+
 ### Accessing a module's root namespace from nested modules within itself
 
 The special constant `MODULE` allows you to access the containing module from
@@ -577,7 +608,7 @@ overwrite any value retained in the instance variable. To assign initial values,
 use the `||=` operator as in the example above. See also the
 [reload example](examples/reload).
 
-## Dependency introspection
+### Dependency introspection
 
 Modulation allows runtime introspection of dependencies between modules. You can
 interrogate a module's dependencies (i.e. the modules it imports) by calling

data/bin/mdl

@@ -4,40 +4,5 @@
 require 'bundler/setup'
 require 'modulation'
 
-def self.run
-  ARGV.each do |arg|
-    fn, method = (arg =~ /^([^\:]+)\:(.+)$/) ? [$1, $2.to_sym] : [arg, :main]
-    mod = import(File.expand_path(fn))
-    mod.send(method) if method
-  end
-rescue StandardError => e
-  backtrace = e.backtrace.reject { |l| l =~ /^(#{Modulation::DIR})|(bin\/mdl)/ }
-  e.set_backtrace(backtrace)
-  raise e
-end
-
-def collect_deps(fn, paths)
-  if File.directory?(fn)
-    Dir["#{fn}/**/*.rb"].each { |fn| collect_deps(fn, paths) }
-  else
-    paths << File.expand_path(fn)
-    mod = import(File.expand_path(fn))
-    if mod.respond_to?(:__traverse_dependencies)
-      mod.__traverse_dependencies { |m| paths << m.__module_info[:location] }
-    end
-  end
-end
-
-def self.deps
-  paths = []
-  ARGV.each { |arg| collect_deps(arg, paths) }
-  puts *paths
-end
-
-def self.pack
-  require 'modulation/packer'
-  STDOUT << Modulation::Packer.pack(ARGV, hide_filenames: true)
-end
-
-cmd = ARGV.shift
-respond_to?(cmd.to_sym) ? send(cmd) : (ARGV.unshift(cmd); run)
+CLI = import '@modulation/cli'
+CLI.new ARGV

data/lib/modulation.rb

@@ -1,7 +1,7 @@
 # frozen_string_literal: true
 
 module Modulation
-  DIR = File.dirname(__FILE__)
+  DIR = __dir__
 end
 
 require_relative 'modulation/ext'

data/lib/modulation/core.rb

@@ -4,7 +4,6 @@
 module Modulation
   require_relative './paths'
   require_relative './builder'
-  require_relative './creator'
   require_relative './module_mixin'
 
   RE_CONST = /^[A-Z]/.freeze
@@ -156,13 +155,14 @@ module Modulation
     end
 
     def create(arg = nil, &block)
-      return Creator.from_block(block) if block
+      creator = import '@modulation/creator'
+      return creator.from_block(block) if block
 
       case arg
       when Hash
-        Creator.from_hash(arg)
+        creator.from_hash(arg)
       when String
-        Creator.from_string(arg)
+        creator.from_string(arg)
       else
         raise 'Invalid argument'
       end

data/lib/modulation/modules/bootstrap.rb

@@ -0,0 +1,49 @@
+# frozen_string_literal: true
+
+export :run, :transform_module_info
+
+require 'zlib'
+
+def run(data, dict_offset)
+  setup(data, dict_offset)
+  import(@dictionary[:entry_point]).send(:main)
+end
+
+def setup(data, dict_offset)
+  patch_builder
+  @data = data
+  @data_offset = data.pos
+  @dictionary = read_dictionary(dict_offset)
+end
+
+def patch_builder
+  class << Modulation::Builder
+    alias_method :orig_make, :make
+    def make(info)
+      info = MODULE.transform_module_info(info)
+      orig_make(info)
+    end
+  end
+end
+
+def transform_module_info(info)
+  if find(info[:location])
+    info[:source] = read_file(info[:location])
+  end
+  info
+end
+
+def find(path)
+  @dictionary[path]
+end
+
+def read_dictionary(offset)
+  @data.seek(@data_offset + offset)
+  eval Zlib::Inflate.inflate(@data.read)
+end
+
+def read_file(path)
+  (offset, length) = @dictionary[path]
+  @data.seek(@data_offset + offset)
+  Zlib::Inflate.inflate(@data.read(length))
+end

data/lib/modulation/modules/cli.rb

@@ -0,0 +1,78 @@
+# frozen_string_literal: true
+
+export_default :CLI
+
+class CLI
+  def initialize(argv)
+    @argv = argv
+
+    process
+  end
+
+  def process
+    cmd = ARGV.shift
+    if respond_to? cmd.to_sym
+      send cmd
+    else
+      ARGV.unshift cmd
+      run
+    end
+  end
+
+  def run
+    @argv.each { |arg| run_file arg }
+  rescue StandardError => e
+    cleanup_backtrace(e)
+    raise e
+  end
+  
+  def run_file(arg)
+    fn, method = filename_and_method_from_arg(arg)
+    mod = import(File.expand_path(fn))
+    mod.send(method) if method
+  end
+  
+  FILENAME_AND_METHOD_RE = /^([^\:]+)\:(.+)$/.freeze
+  
+  def filename_and_method_from_arg(arg)
+    if arg =~ FILENAME_AND_METHOD_RE
+      match = Regexp.last_match
+      [match[1], match[2].to_sym]
+    else
+      [arg, :main]
+    end
+  end
+  
+  BACKTRACE_RE = /^(#{Modulation::DIR})|(bin\/mdl)/.freeze
+  
+  def cleanup_backtrace(error)
+    backtrace = error.backtrace.reject { |l| l =~ BACKTRACE_RE }
+    error.set_backtrace(backtrace)
+  end
+  
+  def collect_deps(path, array)
+    if File.directory?(path)
+      Dir["#{path}/**/*.rb"].each { |fn| collect_deps(fn, array) }
+    else
+      array << File.expand_path(path)
+      mod = import(File.expand_path(path))
+      if mod.respond_to?(:__traverse_dependencies)
+        mod.__traverse_dependencies { |m| array << m.__module_info[:location] }
+      end
+    end
+  end
+  
+  def deps
+    paths = []
+    @argv.each { |arg| collect_deps(arg, paths) }
+    puts(*paths)
+  end
+  
+  def pack
+    STDOUT << import('@modulation/packer').pack(@argv, hide_filenames: true)
+  end
+  
+  def version
+    puts "Modulation version #{Modulation::VERSION}"
+  end
+end

data/lib/modulation/modules/creator.rb

@@ -0,0 +1,35 @@
+# frozen_string_literal: true
+
+export :from_block,
+       :from_hash,
+       :from_string
+
+RE_ATTR   = /^@(.+)$/.freeze
+
+def from_block(block)
+  Module.new.tap { |m| m.instance_eval(&block) }
+end
+
+# Creates a module from a prototype hash
+# @param hash [Hash] prototype hash
+# @return [Module] created object
+def from_hash(hash)
+  Module.new.tap do |m|
+    s = m.singleton_class
+    hash.each do |k, v|
+      if k =~ Modulation::RE_CONST
+        m.const_set(k, v)
+      elsif k =~ RE_ATTR
+        m.instance_variable_set(k, v)
+      elsif v.respond_to?(:to_proc)
+        s.send(:define_method, k) { |*args| instance_exec(*args, &v) }
+      else
+        s.send(:define_method, k) { v }
+      end
+    end
+  end
+end
+
+def from_string(str)
+  m = Modulation::Builder.make(source: str)
+end

data/lib/modulation/modules/packer.rb

@@ -0,0 +1,80 @@
+# frozen_string_literal: true
+
+export :pack
+
+require 'modulation/version'
+require 'zlib'
+
+BOOTSTRAP_CODE = <<~SRC
+  # encoding: ASCII-8BIT
+  require 'bundler/inline'
+
+  gemfile do
+    source 'https://rubygems.org'
+    gem 'modulation', '~> %<modulation_version>s'
+  end
+
+  import('@modulation/bootstrap').run(DATA, %<dict_offset>d)
+  __END__
+  %<data>s
+SRC
+
+def pack(paths, _options = {})
+  paths = [paths] unless paths.is_a?(Array)
+  entry_point_filename = File.expand_path(paths.first)
+
+  deps = collect_dependencies(paths)
+  package_info = generate_packed_data(deps, entry_point_filename)
+  generate_bootstrap(package_info, entry_point_filename)
+end
+
+def collect_dependencies(paths)
+  paths.each_with_object([]) do |fn, deps|
+    mod = import(File.expand_path(fn))
+    deps << File.expand_path(fn)
+    mod.__traverse_dependencies { |m| deps << m.__module_info[:location] }
+  end
+end
+
+def generate_packed_data(deps, entry_point_filename)
+  files = deps.each_with_object({}) do |path, dict|
+    dict[path] = IO.read(path)
+  end
+  pack_files(files, entry_point_filename)
+end
+
+def pack_files(files, entry_point_filename)
+  data = (+'').encode('ASCII-8BIT')
+  last_offset = 0
+  dictionary = {
+    entry_point: entry_point_filename
+  }
+  files.each_with_object(dictionary) do |(path, content), dict|
+    zipped = Zlib::Deflate.deflate(content)
+    size = zipped.bytesize
+
+    data << zipped
+    dict[path] = [last_offset, size]
+    last_offset += size
+  end
+  packed_dictionary = Zlib::Deflate.deflate(dictionary.inspect)
+  data << packed_dictionary
+
+  {
+    dict_offset:  last_offset,
+    data:         data
+  }
+end
+
+def generate_bootstrap(package_info, entry_point)
+  format(
+    bootstrap_template,
+    modulation_version: Modulation::VERSION,
+    dict_offset:        package_info[:dict_offset],
+    data:               package_info[:data]
+  )
+end
+
+def bootstrap_template
+  BOOTSTRAP_CODE.encode('ASCII-8BIT').gsub(/^\s+/, '').chomp
+end

data/lib/modulation/paths.rb

@@ -14,13 +14,18 @@ module Modulation
       CALLER_FILE_REGEXP = /^([^\:]+)\:?/.freeze
       TAGGED_REGEXP = /^@([^\/]+)(\/.+)?$/.freeze
 
-      def add_tags(tags, caller_location)
+      def tags
+        @tags ||= {
+          'modulation' =>  File.join(Modulation::DIR, 'modulation/modules')
+        }
+      end
+
+      def add_tags(new_tags, caller_location)
         caller_file = caller_location[CALLER_FILE_REGEXP, 1]
         caller_dir = caller_file ? File.dirname(caller_file) : nil
 
-        @tags ||= {}
-        tags.each do |k, path|
-          @tags[k.to_s] = caller_dir ? File.expand_path(path, caller_dir) : path
+        new_tags.each do |k, path|
+          tags[k.to_s] = caller_dir ? File.expand_path(path, caller_dir) : path
         end
       end
 
@@ -29,7 +34,7 @@ module Modulation
       def expand_tag(path)
         path.sub RE_TAG do
           tag = Regexp.last_match[1]
-          (@tags && @tags[tag]) || (raise "Invalid tag #{tag}")
+          tags[tag] || (raise "Invalid tag #{tag}")
         end
       end
 

data/lib/modulation/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Modulation
-  VERSION = '0.33'
+  VERSION = '0.34'
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: modulation
 version: !ruby/object:Gem::Version
-  version: '0.33'
+  version: '0.34'
 platform: ruby
 authors:
 - Sharon Rosner
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2019-10-02 00:00:00.000000000 Z
+date: 2019-10-14 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: minitest
@@ -56,14 +56,16 @@ files:
 - lib/modulation.rb
 - lib/modulation/builder.rb
 - lib/modulation/core.rb
-- lib/modulation/creator.rb
 - lib/modulation/export_default.rb
 - lib/modulation/export_from_receiver.rb
 - lib/modulation/exports.rb
 - lib/modulation/ext.rb
 - lib/modulation/gem.rb
 - lib/modulation/module_mixin.rb
-- lib/modulation/packer.rb
+- lib/modulation/modules/bootstrap.rb
+- lib/modulation/modules/cli.rb
+- lib/modulation/modules/creator.rb
+- lib/modulation/modules/packer.rb
 - lib/modulation/paths.rb
 - lib/modulation/version.rb
 homepage: http://github.com/digital-fabric/modulation
@@ -90,7 +92,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.0.3
+rubygems_version: 3.0.6
 signing_key: 
 specification_version: 4
 summary: 'Modulation: explicit dependency management for Ruby'

data/lib/modulation/creator.rb

@@ -1,39 +0,0 @@
-# frozen_string_literal: true
-
-module Modulation
-  # Implements programmtically created modules
-  module Creator
-    RE_CONST  = /^[A-Z]/.freeze
-    RE_ATTR   = /^@(.+)$/.freeze
-
-    class << self
-      # Creates a module from a prototype hash
-      # @param hash [Hash] prototype hash
-      # @return [Module] created object
-      def from_hash(hash)
-        Module.new.tap do |m|
-          s = m.singleton_class
-          hash.each do |k, v|
-            if k =~ RE_CONST
-              m.const_set(k, v)
-            elsif k =~ RE_ATTR
-              m.instance_variable_set(k, v)
-            elsif v.respond_to?(:to_proc)
-              s.send(:define_method, k) { |*args| instance_exec(*args, &v) }
-            else
-              s.send(:define_method, k) { v }
-            end
-          end
-        end
-      end
-
-      def from_string(str)
-        m = Builder.make(source: str)
-      end
-
-      def from_block(block)
-        Module.new.tap { |m| m.instance_eval(&block) }
-      end
-    end
-  end
-end

data/lib/modulation/packer.rb

@@ -1,105 +0,0 @@
-# frozen_string_literal: true
-
-require_relative '../modulation'
-require_relative '../modulation/version'
-require 'zlib'
-
-module Modulation
-  # Implements packing functionality
-  module Packer
-    BOOTSTRAP_CODE = <<~SRC
-      # encoding: ASCII-8BIT
-      require 'bundler/inline'
-
-      gemfile do
-        source 'https://rubygems.org'
-        gem 'modulation', '~> %<modulation_version>s', require: 'modulation/packer'
-      end
-
-      Modulation::Bootstrap.setup(DATA, %<dictionary>s)
-      import(%<entry_point>s).send(:main)
-      __END__
-      %<data>s
-    SRC
-
-    def self.pack(paths, _options = {})
-      paths = [paths] unless paths.is_a?(Array)
-      deps = collect_dependencies(paths)
-      entry_point_filename = File.expand_path(paths.first)
-      dictionary, data = generate_packed_data(deps)
-      generate_bootstrap(dictionary, data, entry_point_filename)
-    end
-
-    def self.collect_dependencies(paths)
-      paths.each_with_object([]) do |fn, deps|
-        mod = import(File.expand_path(fn))
-        deps << File.expand_path(fn)
-        mod.__traverse_dependencies { |m| deps << m.__module_info[:location] }
-      end
-    end
-
-    def self.generate_packed_data(deps)
-      files = deps.each_with_object({}) do |path, dict|
-        dict[path] = IO.read(path)
-      end
-      pack_files(files)
-    end
-
-    def self.pack_files(files)
-      data = (+'').encode('ASCII-8BIT')
-      last_offset = 0
-      dictionary = files.each_with_object({}) do |(path, content), dict|
-        zipped = Zlib::Deflate.deflate(content)
-        size = zipped.bytesize
-
-        data << zipped
-        dict[path] = [last_offset, size]
-        last_offset += size
-      end
-      [dictionary, data]
-    end
-
-    def self.generate_bootstrap(dictionary, data, entry_point)
-      format(bootstrap_template, modulation_version: Modulation::VERSION,
-                                 dictionary: dictionary.inspect,
-                                 entry_point: entry_point.inspect,
-                                 data: data)
-    end
-
-    def self.bootstrap_template
-      BOOTSTRAP_CODE.encode('ASCII-8BIT').gsub(/^\s+/, '').chomp
-    end
-  end
-
-  # Packed app bootstrapping code
-  module Bootstrap
-    def self.setup(data, dictionary)
-      patch_builder
-      @data = data
-      @data_offset = data.pos
-      @dictionary = dictionary
-    end
-
-    def self.patch_builder
-      class << Modulation::Builder
-        alias_method :orig_make, :make
-        def make(info)
-          if Modulation::Bootstrap.find(info[:location])
-            info[:source] = Modulation::Bootstrap.read(info[:location])
-          end
-          orig_make(info)
-        end
-      end
-    end
-
-    def self.find(path)
-      @dictionary[path]
-    end
-
-    def self.read(path)
-      (offset, length) = @dictionary[path]
-      @data.seek(@data_offset + offset)
-      Zlib::Inflate.inflate(@data.read(length))
-    end
-  end
-end