rubygems-mini_mirror 1.0.0

Sign up to get free protection for your applications and to get access to all the features.
data/.gitignore ADDED
@@ -0,0 +1,5 @@
1
+ *.swp
2
+ *.gem
3
+ .bundle
4
+ Gemfile.lock
5
+ pkg/*
data/Gemfile ADDED
@@ -0,0 +1,4 @@
1
+ source "http://rubygems.org"
2
+
3
+ # Specify your gem's dependencies in mini_mirror.gemspec
4
+ gemspec
data/LICENSE ADDED
@@ -0,0 +1,22 @@
1
+ (The MIT License)
2
+
3
+ Copyright (c) 2011 hallelujah
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining
6
+ a copy of this software and associated documentation files (the
7
+ "Software"), to deal in the Software without restriction, including
8
+ without limitation the rights to use, copy, modify, merge, publish,
9
+ distribute, sublicense, and/or sell copies of the Software, and to
10
+ permit persons to whom the Software is furnished to do so, subject to
11
+ the following conditions:
12
+
13
+ The above copyright notice and this permission notice shall be
14
+ included in all copies or substantial portions of the Software.
15
+
16
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
17
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
18
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
19
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
20
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
21
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
22
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,129 @@
1
+ # Rubygems Mini Mirror
2
+
3
+ This gem aims at providing a way to mirror only some gems you want from rubygems.
4
+
5
+ Create a minigems file
6
+
7
+ ```ruby
8
+ # minigems
9
+ source :gemcutter
10
+ gem 'rails', ['~> 1.2.0','>= 3.0']
11
+ gem 'mislav-will_paginate', ['= 2.3.11'], :source => 'http://gems.github.com'
12
+ # You can also provide a resource file which provide a list of gems
13
+ # The filetype is guessed by the extension of the file and can be overriden.
14
+ # :type option can be :yaml, :ruby
15
+ resource :path => '/Users/hallelujah/repos/mini_gems.yml', :type => 'yaml'
16
+ ```
17
+
18
+ The princips
19
+ ------------
20
+
21
+ This nice DSL is mainly inspired by Bundler, so it should be familiar to you.
22
+
23
+ ```ruby
24
+ # Fetching rails with version 1.2.3 and 2.3.5 with runtime and development dependencies
25
+ # By default development dependencies are not fetched
26
+ # *WARNING* fetching dependencies may be a slow process !! Use it with caution.
27
+ gem 'rails', ['= 1.2.3', '2.3.5'], :development => true
28
+
29
+ # If you just want to fetch development dependencies for 1.2.3 version, you need separated definitions :
30
+ gem 'rails', ['= 1.2.3', '2.3.5'], :development => true
31
+ gem 'rails', ['2.3.5']
32
+
33
+ # To add another resource file :
34
+ resource :path => 'development_gems.yml'
35
+ ```
36
+
37
+ Building your own resource handler is also easy
38
+
39
+ ```ruby
40
+ # You can build your own resource handler :
41
+ # my_resource_handler.rb
42
+
43
+ require 'mysql'
44
+ require 'activerecord'
45
+ class MyResourceHandler
46
+
47
+ # db_gems table structure
48
+ # name: string
49
+ # requirements: text
50
+ # sources: text
51
+ # development: boolean
52
+ class DbGem < ActiveRecord::Base
53
+ serialize :sources, Array
54
+ serialize :requirements, Array
55
+ end
56
+
57
+ include Gem::MiniMirror::Resource
58
+ register :type => 'activerecord', :ext => ['.mysql']
59
+
60
+ def initialize(runner, options)
61
+ super
62
+ @config = YAML.load_file(@path).with_indifferent_access
63
+ sources(*options[:sources])
64
+ DbGem.establish_connection @config[:db]
65
+ end
66
+
67
+ def load!
68
+ super
69
+ DbGem.all.each do |g|
70
+ gem g.name, g.requirements, :source => g.sources, :development => g.development?
71
+ end
72
+ end
73
+ end
74
+ ```
75
+
76
+ ```yaml
77
+ # database.yml
78
+
79
+ sources: :gemcutter
80
+ db:
81
+ adapter: mysql
82
+ database: mini_mirror
83
+ host: localhost
84
+ username: root
85
+ password:
86
+ ```
87
+
88
+ ```ruby
89
+ # Usage :
90
+ require 'rubygems/mini_mirror'
91
+ require 'my_resource_handler'
92
+ Gem::MiniMirror::Runner.run :path => 'database.yml', :type => 'activerecord'
93
+ ```
94
+
95
+ By gem command
96
+ --------------
97
+
98
+ It provides a gem command
99
+
100
+ ```bash
101
+ gem install rubygems-mini_mirror
102
+ gem mini_mirror help
103
+ gem mini_mirror -c your_gems_resource.rb -p 5 -m /home/mirrors/rubygems
104
+ # Generate index
105
+ gem generate_index --directory=/home/mirrors/rubygems --modern
106
+ ```
107
+
108
+ # Thanks
109
+
110
+ Thank for the rubygems team for their work on rubygems-mirror which inspired a lot this gem.
111
+ Some parts of code of this gem are copied from [rubygems-mirror](https://github.com/rubygems/rubygems-mirror) gem.
112
+
113
+
114
+ # Note on Patches/Pull Requests
115
+
116
+ * Fork the project.
117
+
118
+ * Make your feature addition or bug fix.
119
+
120
+ * Add tests for it. This is important so I don’t break it in a future version unintentionally.
121
+
122
+ * Commit, do not mess with rakefile, version, or history. (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
123
+
124
+ * Send me a pull request. Bonus points for topic branches.
125
+
126
+ # Copyright
127
+
128
+ Copyright &copy; 2011 Ramihajamalala Hery. See LICENSE for details
129
+
data/Rakefile ADDED
@@ -0,0 +1,6 @@
1
+ require 'bundler/gem_tasks'
2
+ desc "Implement test"
3
+ task :test do
4
+ puts "Must be done anyway"
5
+ end
6
+ task :default => :test
data/TODO.md ADDED
@@ -0,0 +1,7 @@
1
+ # TODO list
2
+
3
+ * Implement downloading gems
4
+ * Implement caching
5
+ * Implement gem command plugin
6
+ * Make tests
7
+ * Add documentation
@@ -0,0 +1,16 @@
1
+ class Array
2
+ # Courtesy from activerecord extract_options
3
+ #
4
+ # Extracts options from a set of arguments. Removes and returns the last
5
+ # element in the array if it's a hash, otherwise returns a blank hash.
6
+ #
7
+ # def options(*args)
8
+ # args.extract_options!
9
+ # end
10
+ #
11
+ # options(1, 2) # => {}
12
+ # options(1, 2, :a => :b) # => {:a=>:b}
13
+ def extract_options!
14
+ last.is_a?(::Hash) ? pop : {}
15
+ end
16
+ end
@@ -0,0 +1,25 @@
1
+ require 'rubygems'
2
+ require "rubygems/mini_mirror/version"
3
+ require "rubygems/array_ext"
4
+ require 'yaml'
5
+
6
+ module Gem
7
+ module MiniMirror
8
+ POOL_SIZE = 10
9
+ autoload :Pool, 'rubygems/mini_mirror/pool'
10
+ autoload :Fetcher, 'rubygems/mini_mirror/fetcher'
11
+ autoload :Runner, 'rubygems/mini_mirror/runner'
12
+ autoload :Cli, 'rubygems/mini_mirror/cli'
13
+ autoload :Finder, 'rubygems/mini_mirror/finder'
14
+ autoload :Resource, 'rubygems/mini_mirror/resource'
15
+ autoload :Dependency, 'rubygems/mini_mirror/dependency'
16
+ autoload :ResourceHandler, 'rubygems/mini_mirror/resource_handler'
17
+ autoload :Resources, 'rubygems/mini_mirror/resources'
18
+ autoload :Runner, 'rubygems/mini_mirror/runner'
19
+
20
+
21
+ def self.resources_handler
22
+ ResourceHandler.instance
23
+ end
24
+ end
25
+ end
@@ -0,0 +1,56 @@
1
+ module Gem
2
+ module MiniMirror
3
+ module Cli
4
+
5
+ # Load default gem configs of the system
6
+ Gem::ConfigFile.new []
7
+
8
+ DEFAULT_SOURCES = {:gemcutter => ['http://gemcutter.org'], :github => ['http://gems.github.com'], :default => Gem.sources}
9
+
10
+ def initialize(runner, options)
11
+ @options = options
12
+ @runner ||= runner
13
+ @sources = nil
14
+ @dependencies ||= []
15
+ end
16
+
17
+ def gem(name, *reqs)
18
+ options = reqs.extract_options!
19
+ reqs.flatten!
20
+ if reqs.empty?
21
+ reqs = ['>=0']
22
+ end
23
+ reqs.each do |req|
24
+ @dependencies << Gem::MiniMirror::Dependency.new(name, req, options.delete(:source) || @sources, options)
25
+ end
26
+ end
27
+
28
+ def source(*sources)
29
+ @sources ||= []
30
+ @sources |= sources.map{|s| DEFAULT_SOURCES[s] || s }.flatten
31
+ end
32
+
33
+ def resource(options={})
34
+ errors = catch(:resource_load_error) do
35
+ @runner.load_resource!(options)
36
+ nil
37
+ end
38
+ if errors
39
+ handle_error(errors)
40
+ end
41
+ end
42
+
43
+ def load!
44
+ return if @loaded
45
+ @loaded = true
46
+ end
47
+
48
+ protected
49
+ def handle_error(errors)
50
+ # MiniMirror.ui.warn(errors.inspect)
51
+ warn errors.inspect
52
+ end
53
+
54
+ end
55
+ end
56
+ end
@@ -0,0 +1,62 @@
1
+ require 'rubygems/mini_mirror'
2
+ require 'rubygems/command'
3
+ require 'yaml'
4
+
5
+ class Gem::Commands::MiniMirrorCommand < Gem::Command
6
+ SUPPORTS_INFO_SIGNAL = Signal.list['INFO']
7
+
8
+ def initialize
9
+ super 'mirror', 'Mirror a gem repository'
10
+ add_option '-c', '--config-file=FILE', 'the main config file' do |file, options|
11
+ options[:path] = file
12
+ end
13
+ add_option '-t', '--file-type=FILETYPE', 'file type of config file if not guessed' do |t, options|
14
+ options[:type] = t
15
+ end
16
+ add_option '-d', '--cd=DIR', 'directory to cd into before running the command' do |dir,options|
17
+ options[:cd] = dir
18
+ end
19
+ add_option '-r', '--require=lib1,lib2', Array, 'A list of libs to require before running the command' do |libs, options|
20
+ options[:libs] ||= []
21
+ options[:libs].push(*libs)
22
+ end
23
+ add_option '-m', '--mirror-dir=MIRROR_DIR', 'The mirror root directory : whre to download gem files. Default to <Gem.user_home>/mirror' do |dir,options|
24
+ options[:gems_dir] = dir
25
+ end
26
+ add_option '-p', '--pool-size=POOLSIZE', Integer, 'The number of threads in the pool : fro fetching concurrency. Default 10' do |pool, options|
27
+ options[:pool_size] = pool
28
+ end
29
+ end
30
+
31
+ def description # :nodoc:
32
+ <<-EOF
33
+ The mini_mirror command mirrors remote gem
34
+ repositories to a local path. Not all the repository of rubygem is mirrored but only those you have chosen with their dependencies
35
+
36
+ See https://github.com/hallelujah/rubygems-mini_mirror
37
+
38
+ EOF
39
+ end
40
+
41
+ def execute
42
+ Dir.chdir(options.delete(:cd) || '.') do
43
+ if options[:requires]
44
+ options[:requires].each do |lib|
45
+ require lib
46
+ end
47
+ end
48
+ mirror = Gem::MiniMirror::Runner.new(options)
49
+ mirror.load_resource! options
50
+ say "Total gems: #{mirror.gems.size}"
51
+ num_to_fetch = mirror.gems_to_fetch.size
52
+ progress = ui.progress_reporter num_to_fetch, "Fetching #{num_to_fetch} gems"
53
+ trap(:INFO) { puts "Fetched: #{progress.count}/#{num_to_fetch}" } if SUPPORTS_INFO_SIGNAL
54
+ mirror.update_gems { progress.updated true }
55
+ num_to_delete = mirror.gems_to_delete.size
56
+ progress = ui.progress_reporter num_to_delete, "Deleting #{num_to_delete} gems"
57
+ trap(:INFO) { puts "Fetched: #{progress.count}/#{num_to_delete}" } if SUPPORTS_INFO_SIGNAL
58
+ mirror.delete_gems { progress.updated true }
59
+ end
60
+
61
+ end
62
+ end
@@ -0,0 +1,25 @@
1
+ module Gem
2
+ module MiniMirror
3
+ class Dependency
4
+
5
+ attr_reader :dependency, :sources
6
+
7
+ def initialize(name, requirements, *args)
8
+ options = args.extract_options!
9
+ srcs = args.empty? ? nil : args.flatten
10
+ @sources = srcs || Gem.sources
11
+ @dependency = Gem::Dependency.new(name,requirements)
12
+ @development = !! options[:development]
13
+ end
14
+
15
+ def development?
16
+ @development
17
+ end
18
+
19
+ def method_missing(*args)
20
+ @dependency.send(*args)
21
+ end
22
+
23
+ end
24
+ end
25
+ end
@@ -0,0 +1,58 @@
1
+ # This code is part of rubygems rubygems-mirror
2
+ # https://github.com/rubygems/rubygems-mirror
3
+ require 'net/http'
4
+ require 'net/http/persistent'
5
+
6
+ class Gem::MiniMirror::Fetcher
7
+ # TODO beef
8
+ class Error < StandardError; end
9
+
10
+ def initialize
11
+ @http = Net::HTTP::Persistent.new(self.class.name, :ENV)
12
+ end
13
+
14
+ # Fetch a source path under the base uri, and put it in the same or given
15
+ # destination path under the base path.
16
+ def fetch(uri, path)
17
+ modified_time = File.exists?(path) && File.stat(path).mtime.rfc822
18
+
19
+ req = Net::HTTP::Get.new URI.parse(uri).path
20
+ req.add_field 'If-Modified-Since', modified_time if modified_time
21
+
22
+ @http.request URI(uri), req do |resp|
23
+ return handle_response(resp, path)
24
+ end
25
+ end
26
+
27
+ # Handle an http response, follow redirects, etc. returns true if a file was
28
+ # downloaded, false if a 304. Raise Error on unknown responses.
29
+ def handle_response(resp, path)
30
+ case resp.code.to_i
31
+ when 304
32
+ when 302
33
+ fetch resp['location'], path
34
+ when 200
35
+ write_file(resp, path)
36
+ when 403, 404
37
+ warn "#{resp.code} on #{File.basename(path)}"
38
+ else
39
+ raise Error, "unexpected response #{resp.inspect}"
40
+ end
41
+ # TODO rescue http errors and reraise cleanly
42
+ end
43
+
44
+ # Efficiently writes an http response object to a particular path. If there
45
+ # is an error, it will remove the target file.
46
+ def write_file(resp, path)
47
+ FileUtils.mkdir_p File.dirname(path)
48
+ File.open(path, 'wb') do |output|
49
+ resp.read_body { |chunk| output << chunk }
50
+ end
51
+ true
52
+ ensure
53
+ # cleanup incomplete files, rescue perm errors etc, they're being
54
+ # raised already.
55
+ File.delete(path) rescue nil if $!
56
+ end
57
+
58
+ end
@@ -0,0 +1,56 @@
1
+ module Gem
2
+ module MiniMirror
3
+ module Finder
4
+
5
+ def initialize(options = {})
6
+ @spec_fetcher = Gem::SpecFetcher.fetcher
7
+ super()
8
+ end
9
+
10
+ def find_all_specs
11
+ @dependencies.each do |dep|
12
+ with_sources dep.sources do
13
+ found, errors = @spec_fetcher.fetch_with_errors dep, true, false
14
+ found.each do |spec,source_uri|
15
+ next if is_in_specs?(spec)
16
+ add_to_specs(spec,source_uri)
17
+ add_to_deps(*spec.runtime_dependencies)
18
+ add_to_deps(*spec.development_dependencies) if dep.development?
19
+ end
20
+ end
21
+ end
22
+ end
23
+
24
+ def add_to_specs(spec, source_uri)
25
+ @specs.push([spec, source_uri])
26
+ @specs_list[spec.platform.to_s][spec.name.to_s][spec.version.to_s] = true
27
+ end
28
+
29
+ def is_in_specs?(spec)
30
+ @specs_list.key?(spec.platform.to_s) && @specs_list[spec.platform.to_s].key?(spec.name.to_s) && @specs_list[spec.platform.to_s][spec.name.to_s].has_key?(spec.version.to_s)
31
+ end
32
+
33
+ def add_to_deps(*deps)
34
+ deps.each do |dep|
35
+ next if is_in_deps?(dep)
36
+ dep = Gem::MiniMirror::Dependency.new(dep.name, dep.requirement,dep.respond_to?(:sources) ? dep.sources : Gem.sources, {:development => dep.respond_to?(:development?) ? dep.development? : false})
37
+ @dependencies_list[dep.name.to_s][dep.requirement.to_s] = true
38
+ @dependencies.push(dep)
39
+ end
40
+ end
41
+
42
+ def is_in_deps?(dep)
43
+ @dependencies_list[dep.name.to_s][dep.requirement.to_s] == true
44
+ end
45
+
46
+
47
+ def with_sources(srcs,&block)
48
+ before_sources = Gem.sources
49
+ Gem::sources= srcs
50
+ block.call
51
+ Gem.sources= before_sources
52
+ end
53
+
54
+ end
55
+ end
56
+ end
@@ -0,0 +1,21 @@
1
+ require 'thread'
2
+ class Gem::MiniMirror::Pool
3
+ def initialize(size)
4
+ @size = size
5
+ @queue = Queue.new
6
+ end
7
+
8
+ def job(&blk)
9
+ @queue << blk
10
+ end
11
+
12
+ def run_til_done
13
+ threads = Array.new(@size) do
14
+ Thread.new { @queue.pop.call while true }
15
+ end
16
+ until @queue.empty? && @queue.num_waiting == @size
17
+ threads.each { |t| t.join(0.1) }
18
+ end
19
+ threads.each { |t| t.kill }
20
+ end
21
+ end
@@ -0,0 +1,27 @@
1
+ module Gem
2
+ module MiniMirror
3
+
4
+ module Resource
5
+
6
+ def self.included(base)
7
+ base.class_eval do
8
+ attr_accessor :sources
9
+ attr_reader :dependencies
10
+ include Gem::MiniMirror::Cli
11
+ extend ClassMethods
12
+ end
13
+ end
14
+
15
+ module ClassMethods
16
+ def register(options={})
17
+ Gem::MiniMirror.resources_handler.add(self, options[:type], options[:ext])
18
+ end
19
+ end
20
+
21
+ def tag
22
+ @options.to_s
23
+ end
24
+
25
+ end
26
+ end
27
+ end
@@ -0,0 +1,62 @@
1
+ require 'singleton'
2
+ module Gem
3
+ module MiniMirror
4
+ class ResourceHandler
5
+ class AlreadyRegistered < Exception
6
+ def initialize(type)
7
+ @type = type
8
+ end
9
+ def message
10
+ "already registered #@type"
11
+ end
12
+ end
13
+
14
+ class HandlerNotFound < Exception
15
+ def initialize(options = {})
16
+ @type = options[:type]
17
+ @extension = options[:ext]
18
+ end
19
+
20
+ def message
21
+ "could not find handler for file extension '#@extension'" +
22
+ @type.to_s.empty? ? "" : " or for file type #@type"
23
+ end
24
+ end
25
+
26
+ include Singleton
27
+
28
+ def initialize
29
+ @handlers = {}
30
+ @handlers_by_file_extension = {}
31
+ end
32
+
33
+ def add(klass, type, exts = [])
34
+ raise AlreadyRegistered, type if @handlers.key?(type.to_s)
35
+ @handlers[type.to_s] = klass
36
+ exts.each do |ext|
37
+ @handlers_by_file_extension[ext.to_s] = klass unless ext.to_s.empty?
38
+ end
39
+ end
40
+
41
+ def find_handler_by_extension(ext)
42
+ @handlers_by_file_extension[ext.to_s]
43
+ end
44
+
45
+ def find_handler_by_type(type)
46
+ @handlers[type.to_s]
47
+ end
48
+
49
+ def find(options = {})
50
+ unless options.key?(:ext)
51
+ if path = options[:path]
52
+ options[:ext] = File.extname(path)
53
+ end
54
+ end
55
+ handler = find_handler_by_type(options[:type]) || find_handler_by_extension(options[:ext])
56
+ raise HandlerNotFound, options unless handler
57
+ handler
58
+ end
59
+ end
60
+
61
+ end
62
+ end
@@ -0,0 +1,9 @@
1
+ module Gem
2
+ module MiniMirror
3
+ module Resources
4
+ require 'rubygems/mini_mirror/resources/base_file'
5
+ require 'rubygems/mini_mirror/resources/ruby_file'
6
+ require 'rubygems/mini_mirror/resources/yaml'
7
+ end
8
+ end
9
+ end
@@ -0,0 +1,20 @@
1
+ module Gem
2
+ module MiniMirror
3
+ module Resources
4
+ class BaseFile
5
+ include Gem::MiniMirror::Resource
6
+
7
+ def initialize(runner, options)
8
+ path = File.expand_path(options[:path])
9
+ throw :resource_load_error, :path => path unless File.exist?(path)
10
+ @path = path
11
+ super
12
+ end
13
+
14
+ def tag
15
+ 'path-' + @path
16
+ end
17
+ end
18
+ end
19
+ end
20
+ end
@@ -0,0 +1,16 @@
1
+ module Gem
2
+ module MiniMirror
3
+ module Resources
4
+ class RubyFile < BaseFile
5
+
6
+ register(:type => 'ruby', :ext => ['.rb'])
7
+
8
+ def load!
9
+ super
10
+ instance_eval(File.read(@path),__FILE__, __LINE__ + 1)
11
+ end
12
+
13
+ end
14
+ end
15
+ end
16
+ end
@@ -0,0 +1,21 @@
1
+ require 'yaml'
2
+ module Gem
3
+ module MiniMirror
4
+ module Resources
5
+ class Yaml < BaseFile
6
+ register(:type => 'yaml', :ext => ['.yml', '.yaml'])
7
+
8
+ def load!
9
+ super
10
+ config = YAML.load_file(@path)
11
+ config.each do |instructs|
12
+ instructs.each do |k, args|
13
+ send(k, *args)
14
+ end
15
+ end
16
+ end
17
+
18
+ end
19
+ end
20
+ end
21
+ end
@@ -0,0 +1,109 @@
1
+ require 'rubygems/mini_mirror/resources'
2
+ module Gem
3
+ module MiniMirror
4
+ class Runner
5
+
6
+ include Gem::MiniMirror::Finder
7
+ attr_reader :specs, :resource_handler
8
+ DEFAULT_GEMS_DIR = File.join(Gem.user_home, '.gem', 'mirror')
9
+
10
+ def initialize(options = {})
11
+ @resource_handler = ResourceHandler.instance
12
+ @dependencies = []
13
+ @specs_list = Hash.new{|h,k| h[k] = Hash.new{ |h1,k1| h1[k1] = Hash.new}}
14
+ @specs = []
15
+ @dependencies_list = Hash.new{|h,k| h[k] = Hash.new}
16
+ @to_fetch = {}
17
+ @resources = []
18
+ @resources_signatures = {}
19
+ @pool = Gem::MiniMirror::Pool.new(options[:pool_size] || Gem::MiniMirror::POOL_SIZE)
20
+ @gems_dir = options[:gems_dir] || DEFAULT_GEMS_DIR
21
+ @fetcher = Gem::MiniMirror::Fetcher.new
22
+ super
23
+ end
24
+
25
+ def load_resource!(options)
26
+ klass = @resource_handler.find(options)
27
+ resource = klass.new(self, options)
28
+ unless @resources_signatures[resource.tag]
29
+ resource.load!
30
+ @resources << resource
31
+ @resources_signatures[resource.tag] = true
32
+ end
33
+ end
34
+
35
+ def from(*args)
36
+ File.join(*args)
37
+ end
38
+
39
+ def to(*args)
40
+ File.join(@gems_dir,*args)
41
+ end
42
+
43
+ def existing_gems
44
+ Dir[to('gems','.*gem')].entries.map{|f| File.basename(f)}
45
+ end
46
+
47
+ def gems_with_sources
48
+ load_all!
49
+ @gems_with_sources ||= Hash[*specs.map{|s,src| [s.full_name + '.gem', src] }.flatten]
50
+ end
51
+
52
+ def gems
53
+ @gems ||= gems_with_sources.keys
54
+ end
55
+
56
+ def gems_to_fetch
57
+ gems - existing_gems
58
+ end
59
+
60
+ def gems_to_delete
61
+ existing_gems - gems
62
+ end
63
+
64
+ def update_gems
65
+ gems_to_fetch.each do |g|
66
+ @pool.job do
67
+ @fetcher.fetch(from(gems_with_sources[g],'gems',g),to('gems',g))
68
+ yield
69
+ end
70
+ end
71
+ @pool.run_til_done
72
+ end
73
+
74
+ def delete_gems
75
+ gems_to_delete.each do |g|
76
+ @pool.job do
77
+ File.delete(to('gems',g))
78
+ yield
79
+ end
80
+ end
81
+ @pool.run_til_done
82
+ end
83
+
84
+ def update
85
+ update_gems
86
+ delete_gems
87
+ end
88
+
89
+ def self.run(file, options = {})
90
+ runner = new(options)
91
+ runner.load_resource! :path => file, :type => 'ruby'
92
+ runner.update
93
+ end
94
+
95
+ protected
96
+ def load_all!
97
+ return if @loaded
98
+ @resources.each do |r|
99
+ add_to_deps(*r.dependencies)
100
+ end
101
+ @loaded = true
102
+ find_all_specs
103
+ end
104
+
105
+
106
+
107
+ end
108
+ end
109
+ end
@@ -0,0 +1,5 @@
1
+ module Gem
2
+ module MiniMirror
3
+ VERSION = "1.0.0"
4
+ end
5
+ end
@@ -0,0 +1,5 @@
1
+ require 'rubygems/command_manager'
2
+ require 'rubygems/mini_mirror/version'
3
+ require 'rubygems/mini_mirror/command'
4
+
5
+ Gem::CommandManager.instance.register_command :mini_mirror
@@ -0,0 +1,41 @@
1
+ # -*- encoding: utf-8 -*-
2
+ $:.push File.expand_path("../lib", __FILE__)
3
+ require "rubygems/mini_mirror/version"
4
+
5
+ Gem::Specification.new do |s|
6
+ s.name = "rubygems-mini_mirror"
7
+ s.version = Gem::MiniMirror::VERSION
8
+ s.authors = ["Ramihajamalala Hery"]
9
+ s.email = ["hery@rails-royce.org"]
10
+ s.homepage = ""
11
+ s.summary = %q{A rubygems mini mirror}
12
+ s.description = %q{
13
+ Mirror some version of gems with Gem::Version DSL
14
+ Create a mini_gem file and add this to it :
15
+ source :gemcutter
16
+ gem 'rails', ['~> 1.2.0', '>= 3.0']
17
+
18
+ or
19
+ source :gemcutter
20
+ resource :path => '/your_path/mini_gem.yml'
21
+
22
+ # /your_path/mini_gem.yml
23
+
24
+ gem:
25
+ - rails:
26
+ -
27
+ - '~> 1.2.0'
28
+ - '>= 3.0'
29
+
30
+ It will solve the dependencies for you so you don't have to write an exhaustive list of the gems you want to mirror
31
+ }
32
+
33
+ s.rubyforge_project = "rubygems-mini_mirror"
34
+ s.add_runtime_dependency 'net-http-persistent', '>=1.2.5'
35
+ s.add_development_dependency 'rake'
36
+
37
+ s.files = `git ls-files`.split("\n")
38
+ s.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
39
+ s.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
40
+ s.require_paths = ["lib"]
41
+ end
@@ -0,0 +1,140 @@
1
+ <?xml version="1.0" encoding="UTF-8"?>
2
+ <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
3
+ <plist version="1.0">
4
+ <dict>
5
+ <key>currentDocument</key>
6
+ <string>lib/rubygems/mini_mirror/cli.rb</string>
7
+ <key>documents</key>
8
+ <array>
9
+ <dict>
10
+ <key>expanded</key>
11
+ <true/>
12
+ <key>name</key>
13
+ <string>rubygems-mini_mirror</string>
14
+ <key>regexFolderFilter</key>
15
+ <string>!.*/(\.[^/]*|CVS|_darcs|_MTN|\{arch\}|blib|.*~\.nib|.*\.(framework|app|pbproj|pbxproj|xcode(proj)?|bundle))$</string>
16
+ <key>sourceDirectory</key>
17
+ <string></string>
18
+ </dict>
19
+ </array>
20
+ <key>fileHierarchyDrawerWidth</key>
21
+ <integer>262</integer>
22
+ <key>metaData</key>
23
+ <dict>
24
+ <key>lib/rubygems/mini_mirror.rb</key>
25
+ <dict>
26
+ <key>caret</key>
27
+ <dict>
28
+ <key>column</key>
29
+ <integer>30</integer>
30
+ <key>line</key>
31
+ <integer>11</integer>
32
+ </dict>
33
+ <key>firstVisibleColumn</key>
34
+ <integer>0</integer>
35
+ <key>firstVisibleLine</key>
36
+ <integer>0</integer>
37
+ </dict>
38
+ <key>lib/rubygems/mini_mirror/cli.rb</key>
39
+ <dict>
40
+ <key>caret</key>
41
+ <dict>
42
+ <key>column</key>
43
+ <integer>28</integer>
44
+ <key>line</key>
45
+ <integer>25</integer>
46
+ </dict>
47
+ <key>columnSelection</key>
48
+ <false/>
49
+ <key>firstVisibleColumn</key>
50
+ <integer>0</integer>
51
+ <key>firstVisibleLine</key>
52
+ <integer>0</integer>
53
+ <key>selectFrom</key>
54
+ <dict>
55
+ <key>column</key>
56
+ <integer>24</integer>
57
+ <key>line</key>
58
+ <integer>25</integer>
59
+ </dict>
60
+ <key>selectTo</key>
61
+ <dict>
62
+ <key>column</key>
63
+ <integer>32</integer>
64
+ <key>line</key>
65
+ <integer>25</integer>
66
+ </dict>
67
+ </dict>
68
+ <key>lib/rubygems/mini_mirror/resource.rb</key>
69
+ <dict>
70
+ <key>caret</key>
71
+ <dict>
72
+ <key>column</key>
73
+ <integer>78</integer>
74
+ <key>line</key>
75
+ <integer>16</integer>
76
+ </dict>
77
+ <key>firstVisibleColumn</key>
78
+ <integer>0</integer>
79
+ <key>firstVisibleLine</key>
80
+ <integer>0</integer>
81
+ </dict>
82
+ <key>lib/rubygems/mini_mirror/resources/ruby_file.rb</key>
83
+ <dict>
84
+ <key>caret</key>
85
+ <dict>
86
+ <key>column</key>
87
+ <integer>54</integer>
88
+ <key>line</key>
89
+ <integer>15</integer>
90
+ </dict>
91
+ <key>firstVisibleColumn</key>
92
+ <integer>0</integer>
93
+ <key>firstVisibleLine</key>
94
+ <integer>0</integer>
95
+ </dict>
96
+ <key>lib/rubygems/mini_mirror/runner.rb</key>
97
+ <dict>
98
+ <key>caret</key>
99
+ <dict>
100
+ <key>column</key>
101
+ <integer>16</integer>
102
+ <key>line</key>
103
+ <integer>16</integer>
104
+ </dict>
105
+ <key>columnSelection</key>
106
+ <false/>
107
+ <key>firstVisibleColumn</key>
108
+ <integer>0</integer>
109
+ <key>firstVisibleLine</key>
110
+ <integer>0</integer>
111
+ <key>selectFrom</key>
112
+ <dict>
113
+ <key>column</key>
114
+ <integer>10</integer>
115
+ <key>line</key>
116
+ <integer>16</integer>
117
+ </dict>
118
+ <key>selectTo</key>
119
+ <dict>
120
+ <key>column</key>
121
+ <integer>23</integer>
122
+ <key>line</key>
123
+ <integer>16</integer>
124
+ </dict>
125
+ </dict>
126
+ </dict>
127
+ <key>openDocuments</key>
128
+ <array>
129
+ <string>lib/rubygems/mini_mirror/resources/ruby_file.rb</string>
130
+ <string>lib/rubygems/mini_mirror/runner.rb</string>
131
+ <string>lib/rubygems/mini_mirror/resource.rb</string>
132
+ <string>lib/rubygems/mini_mirror.rb</string>
133
+ <string>lib/rubygems/mini_mirror/cli.rb</string>
134
+ </array>
135
+ <key>showFileHierarchyDrawer</key>
136
+ <true/>
137
+ <key>windowFrame</key>
138
+ <string>{{2192, -120}, {1648, 1200}}</string>
139
+ </dict>
140
+ </plist>
metadata ADDED
@@ -0,0 +1,121 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: rubygems-mini_mirror
3
+ version: !ruby/object:Gem::Version
4
+ hash: 23
5
+ prerelease: false
6
+ segments:
7
+ - 1
8
+ - 0
9
+ - 0
10
+ version: 1.0.0
11
+ platform: ruby
12
+ authors:
13
+ - Ramihajamalala Hery
14
+ autorequire:
15
+ bindir: bin
16
+ cert_chain: []
17
+
18
+ date: 2011-09-26 00:00:00 +02:00
19
+ default_executable:
20
+ dependencies:
21
+ - !ruby/object:Gem::Dependency
22
+ name: net-http-persistent
23
+ prerelease: false
24
+ requirement: &id001 !ruby/object:Gem::Requirement
25
+ none: false
26
+ requirements:
27
+ - - ">="
28
+ - !ruby/object:Gem::Version
29
+ hash: 21
30
+ segments:
31
+ - 1
32
+ - 2
33
+ - 5
34
+ version: 1.2.5
35
+ type: :runtime
36
+ version_requirements: *id001
37
+ - !ruby/object:Gem::Dependency
38
+ name: rake
39
+ prerelease: false
40
+ requirement: &id002 !ruby/object:Gem::Requirement
41
+ none: false
42
+ requirements:
43
+ - - ">="
44
+ - !ruby/object:Gem::Version
45
+ hash: 3
46
+ segments:
47
+ - 0
48
+ version: "0"
49
+ type: :development
50
+ version_requirements: *id002
51
+ description: "\n Mirror some version of gems with Gem::Version DSL\n Create a mini_gem file and add this to it :\n source :gemcutter\n gem 'rails', ['~> 1.2.0', '>= 3.0']\n\n or\n source :gemcutter\n resource :path => '/your_path/mini_gem.yml'\n\n # /your_path/mini_gem.yml\n\n gem:\n - rails:\n -\n - '~> 1.2.0'\n - '>= 3.0'\n\n It will solve the dependencies for you so you don't have to write an exhaustive list of the gems you want to mirror\n "
52
+ email:
53
+ - hery@rails-royce.org
54
+ executables: []
55
+
56
+ extensions: []
57
+
58
+ extra_rdoc_files: []
59
+
60
+ files:
61
+ - .gitignore
62
+ - Gemfile
63
+ - LICENSE
64
+ - README.md
65
+ - Rakefile
66
+ - TODO.md
67
+ - lib/rubygems/array_ext.rb
68
+ - lib/rubygems/mini_mirror.rb
69
+ - lib/rubygems/mini_mirror/cli.rb
70
+ - lib/rubygems/mini_mirror/command.rb
71
+ - lib/rubygems/mini_mirror/dependency.rb
72
+ - lib/rubygems/mini_mirror/fetcher.rb
73
+ - lib/rubygems/mini_mirror/finder.rb
74
+ - lib/rubygems/mini_mirror/pool.rb
75
+ - lib/rubygems/mini_mirror/resource.rb
76
+ - lib/rubygems/mini_mirror/resource_handler.rb
77
+ - lib/rubygems/mini_mirror/resources.rb
78
+ - lib/rubygems/mini_mirror/resources/base_file.rb
79
+ - lib/rubygems/mini_mirror/resources/ruby_file.rb
80
+ - lib/rubygems/mini_mirror/resources/yaml.rb
81
+ - lib/rubygems/mini_mirror/runner.rb
82
+ - lib/rubygems/mini_mirror/version.rb
83
+ - lib/rubygems_plugin.rb
84
+ - rubygems-mini_mirror.gemspec
85
+ - rubygems-mini_mirror.tmproj
86
+ has_rdoc: true
87
+ homepage: ""
88
+ licenses: []
89
+
90
+ post_install_message:
91
+ rdoc_options: []
92
+
93
+ require_paths:
94
+ - lib
95
+ required_ruby_version: !ruby/object:Gem::Requirement
96
+ none: false
97
+ requirements:
98
+ - - ">="
99
+ - !ruby/object:Gem::Version
100
+ hash: 3
101
+ segments:
102
+ - 0
103
+ version: "0"
104
+ required_rubygems_version: !ruby/object:Gem::Requirement
105
+ none: false
106
+ requirements:
107
+ - - ">="
108
+ - !ruby/object:Gem::Version
109
+ hash: 3
110
+ segments:
111
+ - 0
112
+ version: "0"
113
+ requirements: []
114
+
115
+ rubyforge_project: rubygems-mini_mirror
116
+ rubygems_version: 1.3.7
117
+ signing_key:
118
+ specification_version: 3
119
+ summary: A rubygems mini mirror
120
+ test_files: []
121
+