gem-dependencies 0.2.4 → 0.3.0.rc1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: b6ea647d3078c83342fe528635a5d61818350ff4
-  data.tar.gz: dd8c04aa51bf012d2359e20938e4fa049063e2c3
+  metadata.gz: ae07b23b2801b2e63733fa1a73f82299a6af6e21
+  data.tar.gz: 4a702611f1812fdd358d6125eac64db223dcb0a4
 SHA512:
-  metadata.gz: c335d4bed968497db19016d42e6457fb63161ac014badecf5613ad82344113abd077a0f4a11b4f199db414da688ad5d095d496e635d45de1e882722ecd129c51
-  data.tar.gz: 060fd20f9412a5111b071eff2c3626c4a036d8b9387fe57c3eea69635e143541203f43772610013fcdf959bd622a5058836a535348ff328cdc88259b5e301809
+  metadata.gz: f2baad6f32b9ccbe869adc28d73dd8bf28a959c290bc96017eb589884446c8840df51d0e78fc6cf08c3411a2db7111bb3bbd93977ce89d069bdfb560fb70cffe
+  data.tar.gz: 0d36fe6bd2ce0373d246b2808ed536171fc2ab2fe45d68736f4dda0ecbbe5dd77377a37dfea0defa25c52e4d9b487da9832a63e7e9fb231b1f1eaa365dcf3c65

data/README.md

@@ -1,46 +1,70 @@
 # gem-dependencies
 
-A RubyGems plugin to simplify installing binary gems on runtime systems. Runtime system are usually lightweight and might not even have a compiler. This is accomplished by using a development machine to determine these dependencies and to create tarballs of compiled extensions. Through the use of a dependency index file, these dependencies and extensions can be clearly specified and shared.
+A RubyGems plugin that simplifies working with gems that have binary extensions. It does this through the use of a succinct dependency index file. For development systems, this file specifies build dependencies and arguments necessary to compile and create a tarball with a gem's binary extensions. For runtime systems, which are lightweight and might not have access to a compiler, the dependency index file provides a list of runtime dependencies that must first be installed and also specifies the tarball which contains the pre-built binary extension files.
+
+Think of ```gem-dependencies``` as a helper that allows lightweight machines, such as docker containers, to be able to use native rubygems without having a compiler installed. At the moment that ```gem install``` would normally try to compile a binary extension, ```gem-dependencies``` will lookup the gem in it's dependency index and, if a matching version is found, install any required runtime packages and then download and extract any pre-compiled binary extensions. Everything works seamlessly and efficiently.
 
 ## Usage
 
 You don't need to do anything special, just use ```gem install``` as normal. The resulting behavior is controlled by the value of the ```GEM_DEPENDENCIES``` environment variable. If this environment variable is not set, then this gem has no effect and is simply ignored.
 
-## Development system
-
-Make sure you have a development system with the same architecture as your runtime system. On the development system, make a note of all of the package dependencies needed to install the desired gem. Finally, run:
-
-```shell
-$ GEM_DEPENDENCIES=1 gem install bcrypt
-```
-
-This will perform a normal install, but it will also create a tarball in the current directory with the compiled extensions for the given gem. The naming scheme is straightforward and based on the gem's name and version (e.g. ```bcrypt-3.1.10.tar.gz```).
-
 ## Dependency index
 
-The dependency index is a YAML file that contains information necessary to install gems on the runtime system. An example dependency file looks like:
+The dependency index is a YAML file that contains information necessary to compile gems on a development system or install them on a runtime system. An example dependency file looks like:
 
 ```yaml
 gems:
   "*":
-    command: sudo apk --update add ${packages}
+    command: apk --update add ${packages}
   bcrypt:
-    "~> 2.0, <= 3.3":
-      - bcrypt-2.1
-      - "*"
-  nokogiri:
-    "~> 1.6.2, < 1.8": "* libxml2-dev libxslt-dev"
-    "1.2.8": "s3://one:two@s3.amazon.com/three/* tree"
+  mysql2: "* mariadb-libs"
+  nokogiri: "* libxml2 libxslt +libxml2-dev +libxslt -- --use-system-libraries"
+  pokogiri:
+    - "*"
+    - libxml2
+    - libxslt
+    - +libxml2-dev
+    - +libxslt
+    - --
+    - --use-system-libraries
+  unf_ext:
+    "~> 0.0.7.1": "* libstdc++"
+    ">= 0.0.4, < 0.0.7": "*/old-gems/* libstdc++"
+    "< 0.0.4": "s3://foo:bar@s3.amazon.com/baz/* one two three"
+```
+
+The format of this file is quite flexible.  Keys in the above hash represent gems that have binary extensions. The hash values can be:
+
+* ```nil``` (like ```bcrypt``` above)
+* ```String``` (like ```mysql``` above)
+* ```Array``` (like ```pokogiri``` above)
+* ```Hash``` (like ```unf_ext``` above)
+
+
+
+After determining the runtime dependencies and creating a compiled extensions tarball, edit the dependency index to add a key with the name of the new gem and sub-keys to indicate the version requirements. The values of these sub-key are either an array of package dependencies and extension tarballs or a space-delimited string of the same. All items ending in ```.tar.gz``` are considered to be extension tarballs and everything else is considered to be a package dependency. Extension tarballs can also be given as a file system path or an http, https, git, or s3 url. As an additional help, all urls that begin with ```https://github.com/``` will be automatically suffixed with ```?raw=true```.
+
+There are several naming shortcuts that can be used for extension tarballs. In the common case, when there is one tarball with binary extensions for a gem and no package dependencies, you can simply leave the hash value empty (as shown in the case of ```bcrypt``` above). This will cause ```gem-dependencies``` to download and extract a file named ```bcrypt-3.1.10.tar.gz``` (or whatever gem name and version you are installing) from the base path of the ```GEM_DEPENDENCIES``` environment variable. If, as in the case of ```mysql2``` above, you do have some platform dependencies, then use the ```*``` to indicate the 'default' extension tarball and then list the package dependencies. If a ```*``` character is found anywhere else, it will be interpreted as the value ```{gemname}-{version}.tar.gz```. You can combine wildcards such as the case of ```nokogiri``` above, where ```*/old-gems/*``` will give ```https://github.com/shreeve/gemdeps-alpine-3.2-x86_64-2.2.0/blob/master/old-gems/nokogiri-1.4.4.tar.gz```. These shortcuts are much easier to type and they will dynamically use the proper gem versions.
+
+
+
+
+## Development system
+
+Make sure you have a development system with the same architecture as your runtime system. On the development system, make a note of all of the package dependencies needed to install the desired gem. Finally, run:
+
+```shell
+$ GEM_DEPENDENCIES=1 gem install bcrypt
 ```
 
-After determining the runtime dependencies and creating a compiled extensions tarball, edit the dependency index to add a key with the name of the new gem and sub-keys to indicate the version requirements. The values of these sub-key are either an array of package dependencies and extension tarballs or a space-delimited string of the same. All items ending in ```.tar.gz``` are considered to be extension tarballs and everything else is considered to be a package dependency. Extension tarballs can also be given as a file system path or an http, https, git, or s3 url.
+This will perform a normal install, but it will also create a tarball in the current directory with the compiled extensions for the given gem. The naming scheme is straightforward and based on the gem's name and version (e.g. ```bcrypt-3.1.10.tar.gz```).
 
 ## Runtime system
 
 If ```GEM_DEPENDENCIES``` is set to a file system path or an http, https, git, or s3 url, then this value will be used to fetch the dependency index. Suppose the following command is run from the runtime system:
 
 ```shell
-export GEM_DEPENDENCIES="https://github.com/shreeve/gem_dependencies/alpine-3.2-ruby-2.2-x86_64.yml"
+export GEM_DEPENDENCIES="https://github.com/shreeve/gemdeps-alpine-3.2-x86_64-2.2.0/blob/master/INDEX.yaml"
 gem install bcrypt
 ```
 
@@ -54,7 +78,10 @@ The dependency index will be downloaded and searched for the requested gem and v
 Note that a version requirement can also be specified in the ```gem install``` command. For example, the following are all valid:
 
 ```shell
-bcrypt, bcrypt:3.1.4, "bcrypt:~>3.1.8", "bcrypt > 3.1.4, ~> 3.2, < 3.8"
+gem install bcrypt
+gem install bcrypt:3.1.4
+gem install "bcrypt:~>3.1.8"
+gem install "bcrypt > 3.1.4, ~> 3.2, < 3.8"
 ```
 
 Using this approach, a runtime system can quickly and efficiently install dependencies and extensions without the need to compile them locally.
@@ -69,11 +96,11 @@ alias bundle='RUBYOPT="-rrubygems/gem_dependencies" bundle'
 
 ## Todos
 
-* Allow both compiler and runtime package dependencies in the dependency index file
-* Allow the use of flags (such as ```--use-system-libraries```) for packages
-* Make sure everything works seamlessly with ```bundler```
-* Document the various formats supported in the dependency index file
 * Document how to create a platform repository (e.g. on GitHub)
+* ~~Allow both compiler and runtime package dependencies in the dependency index file~~
+* ~~Allow the use of flags (such as ```--use-system-libraries```) for packages~~
+* ~~Document the various formats supported in the dependency index file~~
+* ~~Make sure everything works seamlessly with ```bundler```~~
 
 ## License
 

data/gem-dependencies.gemspec

@@ -2,7 +2,7 @@
 
 Gem::Specification.new do |s|
   s.name        = "gem-dependencies"
-  s.version     = "0.2.4"
+  s.version     = "0.3.0.rc1"
   s.summary     = "RubyGems plugin to simplify installing binary gems on runtime systems."
   s.description = "This gem makes it easy to install binary gems on systems without a compiler."
   s.homepage    = "https://github.com/shreeve/gem-dependencies"

data/lib/rubygems/gem_dependencies.rb

@@ -21,27 +21,35 @@ module Gem
     module Dependencies
       require 'rubygems/user_interaction'
       include Gem::UserInteraction
-      REGEXP_SCHEMA = %r|\A[a-z][a-z\d]{1,5}://|io
+
+      REGEXP_SCHEMA = %r|\A[a-z][a-z\d]{1,5}://|i
+      REGEXP_DEVPKG = %r|^\+|
+      REGEXP_DEVARG = %r|^(?=-)|
 
       def build_extensions
-        if (env = ENV["GEM_DEPENDENCIES"]) == "1"
-          super
-          path = "#{spec.full_name}.tar.gz"
-          File.binwrite(path, gzip(tar(spec.extension_dir)))
-          say "Extensions packaged as #{path}"
-        elsif deps = find_dependencies(env)
-          pkgs, exts = deps
+        if deps = find_dependencies(ENV["GEM_DEPENDENCIES"])
+          pkgs, exts, args = deps
           install_os_packages(*pkgs) if pkgs.any?
+          if args
+            @build_args.concat(args)
+            super
+            path = "#{spec.full_name}.tar.gz"
+            File.binwrite(path, gzip(tar(spec.extension_dir)))
+            say "Extensions packaged as #{path}"
+          end
           copy_gem_extensions(*exts) if exts.any?
         end
         true
       end
 
-      # return a gem's packages and extensions
+      # return a gem's packages, extensions, and arguments
       def find_dependencies(env)
+        args = [] if (env = env.dup).sub!(REGEXP_DEVPKG,'') # are we in compile mode?
+        return [[], [], args] if args && env.empty?
+
         require 'rubygems/remote_fetcher'
         @@deps = YAML.load(fetch(env))['gems'] unless defined?(@@deps)
-        @@deps.key?(spec.name) or return
+        @@deps.key?(spec.name) or return(args ? [[], [], args] : nil)
 
         # find dependencies
         case deps = @@deps[spec.name]
@@ -63,8 +71,19 @@ module Gem
         benv = File.dirname(env.split(/[?;#]/,2).first)
         name = "#{spec.full_name}.tar.gz"
 
-        # return packages and extensions
-        exts, pkgs = deps.partition {|item| item.include?("*") || item =~ REGEXP_SCHEMA || item.include?(".tar.gz")}
+        # grok dependencies
+        pkgs, exts = [], []
+        deps.each do |item|
+          if item.include?("*") || item =~ REGEXP_SCHEMA || item.include?(".tar.gz")
+            exts << item unless args
+          elsif item =~ REGEXP_DEVPKG
+            pkgs << $' if args
+          elsif item =~ REGEXP_DEVARG
+            args << $' if args
+          else
+            pkgs << item unless args
+          end
+        end
         exts.map! do |item|
           case item
             when "*"       then item = File.join(benv, name) # use complete default tarball name
@@ -73,7 +92,7 @@ module Gem
           end
           item.gsub("*", name) # swap inline wildcards with default tarball name
         end
-        [pkgs, exts]
+        [pkgs, exts, args]
       end
 
       def install_os_packages(*args)

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: gem-dependencies
 version: !ruby/object:Gem::Version
-  version: 0.2.4
+  version: 0.3.0.rc1
 platform: ruby
 authors:
 - Steve Shreeve
@@ -30,7 +30,7 @@ cert_chain:
   4PkbTeqnzZJmSZjUBwvnQFNhXY08hPUs7aASHu9deACBbOBY+Hg6tiSOdfw50uYS
   ksjokeQosImFuxgjMrlyoRcpbZmyfQ33yImqtLH6FMWF3hpoxJq9wPk=
   -----END CERTIFICATE-----
-date: 2015-06-16 00:00:00.000000000 Z
+date: 2015-06-17 00:00:00.000000000 Z
 dependencies: []
 description: This gem makes it easy to install binary gems on systems without a compiler.
 email: steve.shreeve@gmail.com
@@ -59,9 +59,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '0'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
-  - - ">="
+  - - ">"
     - !ruby/object:Gem::Version
-      version: '0'
+      version: 1.3.1
 requirements: []
 rubyforge_project: 
 rubygems_version: 2.4.7