gemstash 2.0.0 → 2.2.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (62) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +57 -0
  3. data/exe/gemstash +3 -0
  4. data/lib/gemstash/api_key_authorization.rb +3 -0
  5. data/lib/gemstash/authorization.rb +7 -3
  6. data/lib/gemstash/cache.rb +42 -2
  7. data/lib/gemstash/cli/authorize.rb +5 -5
  8. data/lib/gemstash/cli/base.rb +4 -1
  9. data/lib/gemstash/cli/setup.rb +19 -4
  10. data/lib/gemstash/cli/start.rb +4 -1
  11. data/lib/gemstash/cli/status.rb +2 -0
  12. data/lib/gemstash/cli/stop.rb +2 -0
  13. data/lib/gemstash/cli.rb +2 -0
  14. data/lib/gemstash/config.ru +3 -3
  15. data/lib/gemstash/configuration.rb +8 -2
  16. data/lib/gemstash/db/authorization.rb +2 -0
  17. data/lib/gemstash/db/cached_rubygem.rb +3 -0
  18. data/lib/gemstash/db/dependency.rb +2 -0
  19. data/lib/gemstash/db/rubygem.rb +3 -0
  20. data/lib/gemstash/db/upstream.rb +3 -0
  21. data/lib/gemstash/db/version.rb +3 -0
  22. data/lib/gemstash/db.rb +3 -0
  23. data/lib/gemstash/dependencies.rb +6 -2
  24. data/lib/gemstash/env.rb +24 -4
  25. data/lib/gemstash/gem_fetcher.rb +4 -2
  26. data/lib/gemstash/gem_pusher.rb +7 -5
  27. data/lib/gemstash/gem_source/dependency_caching.rb +4 -4
  28. data/lib/gemstash/gem_source/private_source.rb +6 -3
  29. data/lib/gemstash/gem_source/rack_middleware.rb +3 -0
  30. data/lib/gemstash/gem_source/upstream_source.rb +9 -3
  31. data/lib/gemstash/gem_source.rb +4 -2
  32. data/lib/gemstash/gem_yanker.rb +4 -0
  33. data/lib/gemstash/health.rb +2 -0
  34. data/lib/gemstash/http_client.rb +12 -4
  35. data/lib/gemstash/logging.rb +3 -1
  36. data/lib/gemstash/man/gemstash-authorize.1.txt +0 -65
  37. data/lib/gemstash/man/gemstash-configuration.5.txt +0 -65
  38. data/lib/gemstash/man/gemstash-customize.7.txt +0 -65
  39. data/lib/gemstash/man/gemstash-debugging.7.txt +0 -65
  40. data/lib/gemstash/man/gemstash-deploy.7.txt +0 -65
  41. data/lib/gemstash/man/gemstash-mirror.7.txt +0 -65
  42. data/lib/gemstash/man/gemstash-multiple-sources.7.txt +0 -65
  43. data/lib/gemstash/man/gemstash-private-gems.7.txt +0 -65
  44. data/lib/gemstash/man/gemstash-readme.7.txt +0 -65
  45. data/lib/gemstash/man/gemstash-setup.1.txt +0 -65
  46. data/lib/gemstash/man/gemstash-start.1.txt +0 -65
  47. data/lib/gemstash/man/gemstash-status.1.txt +0 -65
  48. data/lib/gemstash/man/gemstash-stop.1.txt +0 -65
  49. data/lib/gemstash/man/gemstash-version.1.txt +0 -65
  50. data/lib/gemstash/migrations/01_gem_dependencies.rb +2 -0
  51. data/lib/gemstash/migrations/02_authorizations.rb +2 -0
  52. data/lib/gemstash/migrations/03_cached_gems.rb +2 -0
  53. data/lib/gemstash/migrations/04_health_tests.rb +2 -0
  54. data/lib/gemstash/puma.rb +2 -0
  55. data/lib/gemstash/rack_env_rewriter.rb +11 -2
  56. data/lib/gemstash/specs_builder.rb +5 -1
  57. data/lib/gemstash/storage.rb +17 -7
  58. data/lib/gemstash/upstream.rb +8 -5
  59. data/lib/gemstash/version.rb +4 -2
  60. data/lib/gemstash/web.rb +8 -4
  61. data/lib/gemstash.rb +3 -1
  62. metadata +40 -133
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 9bab8ddab9daaae6c24fd9c878f367ca1206d275b1d42baacaa990cc5048d62d
4
- data.tar.gz: '079b116b2cfb74f712a8300f46aa93d75867fbd4e8b74180595bda2ec3ee6c5b'
3
+ metadata.gz: 41416a39e4dabbc11e7f8f1ad812dc34ffce3fbfa47c6f997feec01c083bf483
4
+ data.tar.gz: c274c1eac63e99d2001f946b117c020dd1fe5c25bdcaf1c31dd28e0668603b2d
5
5
  SHA512:
6
- metadata.gz: b9d4634083a17c6cca10725523aec385964b42755ade1c43334c7436acf275c3b66c265ed256b2006076b10ae643cd17862bd35be5f622239d9eacd0474180cd
7
- data.tar.gz: 88fd7c8d2822fab3a50794453c1d7e1ea1e0014b6b80dbe3f15c05af9b4c078a54c16232c988cb3dbc0e3eb877e05177525daa75e8a9ee736e0e2e47ffc4986f
6
+ metadata.gz: 94941045dd2f8232b5c55a79e775df4e89243d62ce88f22bd737d9dda3bf718a9d30b184b790677beba415c98243b3190c362cab800333993e6d89677592d398
7
+ data.tar.gz: 3f771fae6b0a388dd730978ec0721c83514d3479fd30f9211def6d7bad2ce5f9a83239bbfde091b450f32c053d0f763799e61aa9a9f95b947802dc2c1533275a
data/CHANGELOG.md CHANGED
@@ -1,3 +1,60 @@
1
+ ## 2.1.0 (2020-02-26)
2
+
3
+ ### Bugfixes
4
+
5
+ - Shebang should be the first thing in the file ([#222](https://github.com/rubygems/gemstash/pull/222), [@bronzdoc](https://github.com/bronzdoc))
6
+
7
+ ### Changes
8
+
9
+ - Update rack-test requirement from ~> 0.6 to ~> 1.1 ([#199](https://github.com/rubygems/gemstash/pull/199), [@dependabot](https://github.com/dependabot))
10
+ - Update rake requirement from ~> 10.0 to ~> 12.3 ([#200](https://github.com/rubygems/gemstash/pull/200), [@dependabot](https://github.com/dependabot))
11
+ - CI matrix: Drop 2.0, 2.1, 2.2 ([#201](https://github.com/rubygems/gemstash/pull/201), [@olleolleolle](https://github.com/olleolleolle))
12
+ - Update rubocop version and update files to be inline with new cops ([#202](https://github.com/rubygems/gemstash/pull/202), [@bronzdoc](https://github.com/bronzdoc))
13
+ - Update rubocop requirement from = 0.60.0 to = 0.61.0 ([#208](https://github.com/rubygems/gemstash/pull/208), [@dependabot](https://github.com/dependabot))
14
+ - Update rubocop requirement from = 0.61.0 to = 0.61.1 ([#209](https://github.com/rubygems/gemstash/pull/209), [@dependabot](https://github.com/dependabot))
15
+ - Avoid Rubocop warnings ([#210](https://github.com/rubygems/gemstash/pull/210), [@olleolleolle](https://github.com/olleolleolle))
16
+ - RSpec: config.disable_monkey_patching! ([#211](https://github.com/rubygems/gemstash/pull/211), [@olleolleolle](https://github.com/olleolleolle))
17
+ - Travis: update JRuby to 9.2.5.0 ([#212](https://github.com/rubygems/gemstash/pull/212), [@olleolleolle](https://github.com/olleolleolle))
18
+ - Update rubocop requirement from = 0.61.1 to = 0.62.0 ([#213](https://github.com/rubygems/gemstash/pull/213), [@dependabot](https://github.com/dependabot))
19
+ - Update rubocop requirement from = 0.62.0 to = 0.63.0 ([#215](https://github.com/rubygems/gemstash/pull/215), [@dependabot](https://github.com/dependabot))
20
+ - gemspec: Allow Bundler 2 ([#217](https://github.com/rubygems/gemstash/pull/217), [@olleolleolle](https://github.com/olleolleolle))
21
+ - Update rubocop requirement from = 0.63.0 to = 0.63.1 ([#218](https://github.com/rubygems/gemstash/pull/218), [@dependabot](https://github.com/dependabot))
22
+ - Update rubocop requirement from = 0.63.1 to = 0.64.0 ([#220](https://github.com/rubygems/gemstash/pull/220), [@dependabot](https://github.com/dependabot))
23
+ - CI: matrix updates - update JRuby, add 2.6.1 ([#221](https://github.com/rubygems/gemstash/pull/221), [@olleolleolle](https://github.com/olleolleolle))
24
+ - Add a failing Aruba-based RSpec test for the CLI client output ([#223](https://github.com/rubygems/gemstash/pull/223), [@olleolleolle](https://github.com/olleolleolle))
25
+ - Update rubocop requirement from = 0.64.0 to = 0.65.0 ([#224](https://github.com/rubygems/gemstash/pull/224), [@dependabot](https://github.com/dependabot))
26
+ - CI: use 2.5.5, 2.6.2 ([#225](https://github.com/rubygems/gemstash/pull/225), [@olleolleolle](https://github.com/olleolleolle))
27
+ - Update rubocop requirement from = 0.65.0 to = 0.66.0 ([#226](https://github.com/rubygems/gemstash/pull/226), [@dependabot](https://github.com/dependabot), [@olleolleolle](https://github.com/olleolleolle), [@bronzdoc](https://github.com/bronzdoc))
28
+ - CI: Use 2.4.6 ([#228](https://github.com/rubygems/gemstash/pull/228), [@olleolleolle](https://github.com/olleolleolle))
29
+ - Update rubocop requirement from = 0.66.0 to = 0.67.1 ([#229](https://github.com/rubygems/gemstash/pull/229), [@dependabot](https://github.com/dependabot), [@olleolleolle](https://github.com/olleolleolle))
30
+ - CI: Drop 2.3.8 ([#230](https://github.com/rubygems/gemstash/pull/230), [@olleolleolle](https://github.com/olleolleolle))
31
+ - Update rubocop requirement from = 0.67.1 to = 0.67.2 ([#231](https://github.com/rubygems/gemstash/pull/231), [@dependabot](https://github.com/dependabot))
32
+ - CI: Use JRuby 9.2.7.0 ([#232](https://github.com/rubygems/gemstash/pull/232), [@olleolleolle](https://github.com/olleolleolle))
33
+ - Update rubocop requirement from = 0.67.2 to = 0.68.0 ([#233](https://github.com/rubygems/gemstash/pull/233), [@dependabot](https://github.com/dependabot), [@olleolleolle](https://github.com/olleolleolle))
34
+ - Update rubocop requirement from = 0.68.1 to = 0.69.0 ([#234](https://github.com/rubygems/gemstash/pull/234), [@dependabot](https://github.com/dependabot))
35
+ - Update rubocop requirement from = 0.69.0 to = 0.70.0 ([#235](https://github.com/rubygems/gemstash/pull/235), [@dependabot](https://github.com/dependabot))
36
+ - Set Travis badge to /rubygems user ([#236](https://github.com/rubygems/gemstash/pull/236), [@olleolleolle](https://github.com/olleolleolle))
37
+ - Fix broken links ([#240](https://github.com/rubygems/gemstash/pull/240), [@Aliciawyse](https://github.com/Aliciawyse))
38
+ - Fix various rubocop/codeclimate issues ([#241](https://github.com/rubygems/gemstash/pull/241), [@tchia04](https://github.com/tchia04))
39
+ - Fix some broken links in readme ([#243](https://github.com/rubygems/gemstash/pull/243), [@Aliciawyse](https://github.com/Aliciawyse))
40
+ - Improve webspec tests ([#244](https://github.com/rubygems/gemstash/pull/244), [@Aliciawyse](https://github.com/Aliciawyse))
41
+ - CI: Use jruby-9.2.8.0 ([#245](https://github.com/rubygems/gemstash/pull/245), [@olleolleolle](https://github.com/olleolleolle))
42
+ - CI: Use ruby 2.4.7, 2.5.6, 2.6.4 ([#246](https://github.com/rubygems/gemstash/pull/246), [@olleolleolle](https://github.com/olleolleolle))
43
+ - Add puma server pidfile configuration option ([#248](https://github.com/rubygems/gemstash/pull/248), [@krists](https://github.com/krists))
44
+ - CI: Use jruby-9.2.9.0 ([#249](https://github.com/rubygems/gemstash/pull/249), [@olleolleolle](https://github.com/olleolleolle))
45
+ - Officially drop support for older rubies ([#250](https://github.com/rubygems/gemstash/pull/250), [@deivid-rodriguez](https://github.com/deivid-rodriguez))
46
+ - Add config to ignore Gemfile source ([#253](https://github.com/rubygems/gemstash/pull/253), [@zpdcodes](https://github.com/zpdcodes))
47
+ - Update Puma gem ([#255](https://github.com/rubygems/gemstash/pull/255), [@bronzdoc](https://github.com/bronzdoc))
48
+ - Spec failures might be due to a bundler version ([#256](https://github.com/rubygems/gemstash/pull/256), [@bronzdoc](https://github.com/bronzdoc))
49
+ - Update spec homepage ([#257](https://github.com/rubygems/gemstash/pull/257), [@bronzdoc](https://github.com/bronzdoc))
50
+ - Update README.md links ([#258](https://github.com/rubygems/gemstash/pull/258), [@bronzdoc](https://github.com/bronzdoc))
51
+ - Add PR template ([#259](https://github.com/rubygems/gemstash/pull/259), [@bronzdoc](https://github.com/bronzdoc))
52
+ - Update docs ([#260](https://github.com/rubygems/gemstash/pull/260), [@bronzdoc](https://github.com/bronzdoc))
53
+
54
+ ### Features
55
+
56
+ - Add configuration options cache_expiration, cache_max_size ([#207](https://github.com/rubygems/gemstash/pull/207), [@optix2000](https://github.com/optix2000), [@olleolleolle](https://github.com/olleolleolle))
57
+
1
58
  ## 2.0.0 (2018-11-18)
2
59
 
3
60
  ### Changes
data/exe/gemstash CHANGED
@@ -1,3 +1,6 @@
1
1
  #!/usr/bin/env ruby
2
+
3
+ # frozen_string_literal: true
4
+
2
5
  require "gemstash/cli"
3
6
  Gemstash::CLI.start
@@ -1,3 +1,5 @@
1
+ # frozen_string_literal: true
2
+
1
3
  require "gemstash"
2
4
 
3
5
  module Gemstash
@@ -19,6 +21,7 @@ module Gemstash
19
21
  def self.parse_authorization(request_env)
20
22
  http_auth = Rack::Auth::Basic::Request.new(request_env)
21
23
  return http_auth.credentials.first if http_auth.provided? && http_auth.basic?
24
+
22
25
  request_env["HTTP_AUTHORIZATION"]
23
26
  end
24
27
 
@@ -1,3 +1,5 @@
1
+ # frozen_string_literal: true
2
+
1
3
  require "set"
2
4
 
3
5
  module Gemstash
@@ -17,9 +19,7 @@ module Gemstash
17
19
 
18
20
  unless permissions == "all"
19
21
  permissions.each do |permission|
20
- unless VALID_PERMISSIONS.include?(permission)
21
- raise "Invalid permission '#{permission}'"
22
- end
22
+ raise "Invalid permission '#{permission}'" unless VALID_PERMISSIONS.include?(permission)
23
23
  end
24
24
 
25
25
  permissions = permissions.join(",")
@@ -33,6 +33,7 @@ module Gemstash
33
33
  def self.remove(auth_key)
34
34
  record = Gemstash::DB::Authorization[auth_key: auth_key]
35
35
  return unless record
36
+
36
37
  record.destroy
37
38
  gemstash_env.cache.invalidate_authorization(auth_key)
38
39
  log.info "Authorization '#{auth_key}' with access to '#{record.permissions}' removed"
@@ -40,6 +41,7 @@ module Gemstash
40
41
 
41
42
  def self.check(auth_key, permission)
42
43
  raise NotAuthorizedError, "Authorization key required" if auth_key.to_s.strip.empty?
44
+
43
45
  auth = self[auth_key]
44
46
  raise NotAuthorizedError, "Authorization key is invalid" unless auth
45
47
  raise NotAuthorizedError, "Authorization key doesn't have #{permission} access" unless auth.can?(permission)
@@ -48,6 +50,7 @@ module Gemstash
48
50
  def self.[](auth_key)
49
51
  cached_auth = gemstash_env.cache.authorization(auth_key)
50
52
  return cached_auth if cached_auth
53
+
51
54
  record = Gemstash::DB::Authorization[auth_key: auth_key]
52
55
 
53
56
  if record
@@ -65,6 +68,7 @@ module Gemstash
65
68
 
66
69
  def can?(permission)
67
70
  raise "Invalid permission '#{permission}'" unless VALID_PERMISSIONS.include?(permission)
71
+
68
72
  all? || @permissions.include?(permission)
69
73
  end
70
74
 
@@ -1,3 +1,5 @@
1
+ # frozen_string_literal: true
2
+
1
3
  require "lru_redux"
2
4
  require "forwardable"
3
5
 
@@ -6,7 +8,7 @@ module Gemstash
6
8
  # for them. Under the hood is either a Memcached client via the dalli gem, or
7
9
  # an in memory client via the lru_redux gem.
8
10
  class Cache
9
- EXPIRY = 30 * 60
11
+ EXPIRY = Env.current.config[:cache_expiration]
10
12
  extend Forwardable
11
13
  def_delegators :@client, :flush
12
14
 
@@ -47,7 +49,7 @@ module Gemstash
47
49
 
48
50
  # Wrapper around the lru_redux gem to behave like a dalli Memcached client.
49
51
  class LruReduxClient
50
- MAX_SIZE = 500
52
+ MAX_SIZE = Env.current.config[:cache_max_size]
51
53
  EXPIRY = Gemstash::Cache::EXPIRY
52
54
  extend Forwardable
53
55
  def_delegators :@cache, :delete
@@ -68,6 +70,7 @@ module Gemstash
68
70
  # Atomic fetch... don't rely on nil meaning missing
69
71
  value = @cache.fetch(key) { found = false }
70
72
  next unless found
73
+
71
74
  yield(key, value)
72
75
  end
73
76
  end
@@ -76,4 +79,41 @@ module Gemstash
76
79
  @cache[key] = value
77
80
  end
78
81
  end
82
+
83
+ # Wrapper around the redis-rb gem to behave like a dalli Memcached client.
84
+ class RedisClient
85
+ extend Forwardable
86
+
87
+ def_delegator :@cache, :del, :delete
88
+
89
+ def initialize(redis_servers)
90
+ require "redis"
91
+ @cache = ::Redis.new(:url => redis_servers)
92
+ end
93
+
94
+ def alive!
95
+ @cache.ping == "PONG"
96
+ end
97
+
98
+ def get(key)
99
+ val = @cache.get(key)
100
+ YAML.load(val) unless val.nil?
101
+ end
102
+
103
+ def get_multi(keys)
104
+ @cache.mget(*keys).each do |k, v|
105
+ next if v.nil?
106
+
107
+ yield(k, YAML.load(v))
108
+ end
109
+ end
110
+
111
+ def set(key, value, expiry)
112
+ @cache.set(key, YAML.dump(value), :ex => expiry)
113
+ end
114
+
115
+ def flush
116
+ @cache.flushdb
117
+ end
118
+ end
79
119
  end
@@ -1,3 +1,5 @@
1
+ # frozen_string_literal: true
2
+
1
3
  require "gemstash"
2
4
  require "securerandom"
3
5
 
@@ -28,13 +30,11 @@ module Gemstash
28
30
  raise Gemstash::CLI::Error.new(@cli, "To remove individual permissions, you do not need --remove
29
31
  Instead just authorize with the new set of permissions")
30
32
  end
31
- Gemstash::Authorization.remove(auth_key(false))
33
+ Gemstash::Authorization.remove(auth_key(allow_generate: false))
32
34
  end
33
35
 
34
36
  def save_authorization
35
- if @args.include?("all")
36
- raise Gemstash::CLI::Error.new(@cli, "Don't specify permissions to authorize for all")
37
- end
37
+ raise Gemstash::CLI::Error.new(@cli, "Don't specify permissions to authorize for all") if @args.include?("all")
38
38
 
39
39
  @args.each do |arg|
40
40
  unless Gemstash::Authorization::VALID_PERMISSIONS.include?(arg)
@@ -46,7 +46,7 @@ Instead just authorize with the new set of permissions")
46
46
  Gemstash::Authorization.authorize(auth_key, permissions)
47
47
  end
48
48
 
49
- def auth_key(allow_generate = true)
49
+ def auth_key(allow_generate: true)
50
50
  if @cli.options[:key]
51
51
  @cli.options[:key]
52
52
  elsif allow_generate
@@ -1,3 +1,5 @@
1
+ # frozen_string_literal: true
2
+
1
3
  require "gemstash"
2
4
 
3
5
  module Gemstash
@@ -38,13 +40,14 @@ module Gemstash
38
40
  def check_gemstash_version
39
41
  version = Gem::Version.new(Gemstash::Storage.metadata[:gemstash_version])
40
42
  return if Gem::Requirement.new("<= #{Gemstash::VERSION}").satisfied_by?(Gem::Version.new(version))
43
+
41
44
  raise Gemstash::CLI::Error.new(@cli, "Gemstash version #{Gemstash::VERSION} does not support version " \
42
45
  "#{version}.\nIt appears you may have downgraded Gemstash, please " \
43
46
  "install version #{version} or later.")
44
47
  end
45
48
 
46
49
  def pidfile_args
47
- ["--pidfile", gemstash_env.base_file("puma.pid")]
50
+ ["--pidfile", gemstash_env.pidfile]
48
51
  end
49
52
  end
50
53
  end
@@ -1,3 +1,5 @@
1
+ # frozen_string_literal: true
2
+
1
3
  require "gemstash"
2
4
  require "fileutils"
3
5
  require "yaml"
@@ -44,11 +46,13 @@ module Gemstash
44
46
 
45
47
  def say_current_config(option, label)
46
48
  return if gemstash_env.config.default?(option)
49
+
47
50
  @cli.say "#{label}: #{gemstash_env.config[option]}"
48
51
  end
49
52
 
50
53
  def ask_with_default(prompt, options, default)
51
54
  raise "The options must all be lower case" if options.any? {|x| x.downcase != x }
55
+
52
56
  result = nil
53
57
  displayed_options = options.map {|x| x == default ? x.upcase : x }
54
58
  prompt = "#{prompt} [#{displayed_options.join(", ")}]"
@@ -72,8 +76,9 @@ module Gemstash
72
76
 
73
77
  def ask_cache
74
78
  say_current_config(:cache_type, "Current cache")
75
- @config[:cache_type] = ask_with_default("Cache with what?", %w[memory memcached], "memory")
79
+ @config[:cache_type] = ask_with_default("Cache with what?", %w[memory memcached redis], "memory")
76
80
  ask_memcached_details if @config[:cache_type] == "memcached"
81
+ ask_redis_details if @config[:cache_type] == "redis"
77
82
  end
78
83
 
79
84
  def ask_memcached_details
@@ -83,6 +88,13 @@ module Gemstash
83
88
  @config[:memcached_servers] = servers
84
89
  end
85
90
 
91
+ def ask_redis_details
92
+ say_current_config(:redis_servers, "Current Redis servers")
93
+ servers = @cli.ask "What is the comma-separated list of Redis servers? [localhost:6379]"
94
+ servers = "localhost:6379" if servers.empty?
95
+ @config[:redis_servers] = servers
96
+ end
97
+
86
98
  def ask_database
87
99
  say_current_config(:db_adapter, "Current database adapter")
88
100
  @config[:db_adapter] = ask_with_default("What database adapter?", %w[sqlite3 postgres mysql mysql2], "sqlite3")
@@ -135,8 +147,10 @@ module Gemstash
135
147
  # we don't want to store metadata just yet
136
148
  metadata_file = gemstash_env.base_file("metadata.yml")
137
149
  break unless File.exist?(metadata_file)
150
+
138
151
  version = Gem::Version.new(YAML.load_file(metadata_file)[:gemstash_version])
139
152
  break if Gem::Requirement.new("<= #{Gemstash::VERSION}").satisfied_by?(Gem::Version.new(version))
153
+
140
154
  raise Gemstash::CLI::Error.new(@cli, "The base path already exists with a newer version of Gemstash")
141
155
  else
142
156
  @cli.say "Creating the file storage path '#{dir}'"
@@ -160,6 +174,7 @@ module Gemstash
160
174
 
161
175
  def say_error(title, error)
162
176
  return unless @cli.options[:debug]
177
+
163
178
  @cli.say @cli.set_color("#{title}: #{error}", :red)
164
179
 
165
180
  error.backtrace.each do |line|
@@ -174,10 +189,10 @@ module Gemstash
174
189
  gemstash_env.reset
175
190
  end
176
191
 
177
- def try(thing)
192
+ def try(thing, &block)
178
193
  @cli.say "Checking that the #{thing} is available"
179
- with_new_config { yield }
180
- rescue => e
194
+ with_new_config(&block)
195
+ rescue StandardError => e
181
196
  say_error "Error checking #{thing}", e
182
197
  raise Gemstash::CLI::Error.new(@cli, "The #{thing} is not available")
183
198
  end
@@ -1,3 +1,5 @@
1
+ # frozen_string_literal: true
2
+
1
3
  require "gemstash"
2
4
  require "puma/cli"
3
5
 
@@ -18,6 +20,7 @@ module Gemstash
18
20
 
19
21
  def setup_logging
20
22
  return unless daemonize?
23
+
21
24
  Gemstash::Logging.setup_logger(gemstash_env.log_file)
22
25
  end
23
26
 
@@ -30,7 +33,7 @@ module Gemstash
30
33
  end
31
34
 
32
35
  def puma_config
33
- File.expand_path("../../puma.rb", __FILE__)
36
+ File.expand_path("../puma.rb", __dir__)
34
37
  end
35
38
 
36
39
  def args
@@ -1,3 +1,5 @@
1
+ # frozen_string_literal: true
2
+
1
3
  require "gemstash"
2
4
  require "puma/control_cli"
3
5
 
@@ -1,3 +1,5 @@
1
+ # frozen_string_literal: true
2
+
1
3
  require "gemstash"
2
4
  require "puma/control_cli"
3
5
 
data/lib/gemstash/cli.rb CHANGED
@@ -1,3 +1,5 @@
1
+ # frozen_string_literal: true
2
+
1
3
  require "gemstash"
2
4
  require "thor"
3
5
  require "thor/error"
@@ -1,12 +1,12 @@
1
+ # frozen_string_literal: true
2
+
1
3
  require "gemstash"
2
4
  require "puma/commonlogger"
3
5
 
4
6
  use Rack::Deflater
5
7
  use Gemstash::Logging::RackMiddleware
6
8
 
7
- if Gemstash::Env.daemonized?
8
- use Puma::CommonLogger, Gemstash::Logging::StreamLogger.for_stdout
9
- end
9
+ use Puma::CommonLogger, Gemstash::Logging::StreamLogger.for_stdout if Gemstash::Env.daemonized?
10
10
 
11
11
  use Gemstash::Env::RackMiddleware, Gemstash::Env.current
12
12
  use Gemstash::GemSource::RackMiddleware
@@ -1,8 +1,10 @@
1
+ # frozen_string_literal: true
2
+
1
3
  require "yaml"
2
4
  require "erb"
3
5
 
4
6
  module Gemstash
5
- #:nodoc:
7
+ # :nodoc:
6
8
  class Configuration
7
9
  DEFAULTS = {
8
10
  cache_type: "memory",
@@ -10,12 +12,15 @@ module Gemstash
10
12
  db_adapter: "sqlite3",
11
13
  bind: "tcp://0.0.0.0:9292",
12
14
  rubygems_url: "https://rubygems.org",
15
+ ignore_gemfile_source: false,
13
16
  protected_fetch: false,
14
17
  fetch_timeout: 20,
15
18
  # Actual default for db_connection_options is dynamic based on the adapter
16
19
  db_connection_options: {},
17
20
  puma_threads: 16,
18
- puma_workers: 1
21
+ puma_workers: 1,
22
+ cache_expiration: 30 * 60,
23
+ cache_max_size: 500
19
24
  }.freeze
20
25
 
21
26
  DEFAULT_FILE = File.expand_path("~/.gemstash/config.yml").freeze
@@ -35,6 +40,7 @@ module Gemstash
35
40
  end
36
41
 
37
42
  raise MissingFileError, file if file && !File.exist?(file)
43
+
38
44
  file ||= default_file
39
45
 
40
46
  if File.exist?(file)
@@ -1,3 +1,5 @@
1
+ # frozen_string_literal: true
2
+
1
3
  require "gemstash"
2
4
 
3
5
  module Gemstash
@@ -1,3 +1,5 @@
1
+ # frozen_string_literal: true
2
+
1
3
  require "gemstash"
2
4
 
3
5
  module Gemstash
@@ -9,6 +11,7 @@ module Gemstash
9
11
  upstream_id = Gemstash::DB::Upstream.find_or_insert(upstream)
10
12
  record = self[upstream_id: upstream_id, name: gem_name.name, resource_type: resource_type.to_s]
11
13
  return record.id if record
14
+
12
15
  new(upstream_id: upstream_id, name: gem_name.name, resource_type: resource_type.to_s).tap(&:save).id
13
16
  end
14
17
  end
@@ -1,3 +1,5 @@
1
+ # frozen_string_literal: true
2
+
1
3
  require "gemstash"
2
4
 
3
5
  module Gemstash
@@ -1,3 +1,5 @@
1
+ # frozen_string_literal: true
2
+
1
3
  require "gemstash"
2
4
 
3
5
  module Gemstash
@@ -7,6 +9,7 @@ module Gemstash
7
9
  def self.find_or_insert(spec)
8
10
  record = self[name: spec.name]
9
11
  return record.id if record
12
+
10
13
  new(name: spec.name).tap(&:save).id
11
14
  end
12
15
  end
@@ -1,3 +1,5 @@
1
+ # frozen_string_literal: true
2
+
1
3
  module Gemstash
2
4
  module DB
3
5
  # Sequel model for upstreams table.
@@ -5,6 +7,7 @@ module Gemstash
5
7
  def self.find_or_insert(upstream)
6
8
  record = self[uri: upstream.to_s]
7
9
  return record.id if record
10
+
8
11
  new(uri: upstream.to_s, host_id: upstream.host_id).tap(&:save).id
9
12
  end
10
13
  end
@@ -1,3 +1,5 @@
1
+ # frozen_string_literal: true
2
+
1
3
  require "gemstash"
2
4
 
3
5
  module Gemstash
@@ -52,6 +54,7 @@ module Gemstash
52
54
  def self.find_by_full_name(full_name)
53
55
  result = self[full_name: full_name]
54
56
  return result if result
57
+
55
58
  # Try again with the default platform, in case it is implied
56
59
  self[full_name: "#{full_name}-ruby"]
57
60
  end
data/lib/gemstash/db.rb CHANGED
@@ -1,9 +1,12 @@
1
+ # frozen_string_literal: true
2
+
1
3
  require "gemstash"
2
4
 
3
5
  module Gemstash
4
6
  # Module containing the DB models.
5
7
  module DB
6
8
  raise "Gemstash::DB cannot be loaded until the Gemstash::Env is available" unless Gemstash::Env.available?
9
+
7
10
  Sequel::Model.db = Gemstash::Env.current.db
8
11
  Sequel::Model.raise_on_save_failure = true
9
12
  Sequel::Model.plugin :timestamps, update_on_create: true
@@ -1,8 +1,10 @@
1
+ # frozen_string_literal: true
2
+
1
3
  require "cgi"
2
4
  require "set"
3
5
 
4
6
  module Gemstash
5
- #:nodoc:
7
+ # :nodoc:
6
8
  class Dependencies
7
9
  def self.for_private
8
10
  new(scope: "private", db_model: Gemstash::DB::Dependency)
@@ -22,7 +24,7 @@ module Gemstash
22
24
  Fetcher.new(gems, @scope, @http_client, @db_model).fetch
23
25
  end
24
26
 
25
- #:nodoc:
27
+ # :nodoc:
26
28
  class Fetcher
27
29
  include Gemstash::Env::Helper
28
30
  include Gemstash::Logging
@@ -59,6 +61,7 @@ module Gemstash
59
61
  def fetch_from_database
60
62
  return if done?
61
63
  return unless @db_model
64
+
62
65
  log.info "Querying dependencies: #{@gems.to_a.join(", ")}"
63
66
 
64
67
  @db_model.fetch(@gems) do |gem, value|
@@ -71,6 +74,7 @@ module Gemstash
71
74
  def fetch_from_web
72
75
  return if done?
73
76
  return unless @http_client
77
+
74
78
  log.info "Fetching dependencies: #{@gems.to_a.join(", ")}"
75
79
  gems_param = @gems.map {|gem| CGI.escape(gem) }.join(",")
76
80
  fetched = @http_client.get("api/v1/dependencies?gems=#{gems_param}")
data/lib/gemstash/env.rb CHANGED
@@ -1,9 +1,12 @@
1
+ # frozen_string_literal: true
2
+
1
3
  require "gemstash"
2
4
  require "active_support/core_ext/file/atomic"
3
5
  require "dalli"
4
6
  require "fileutils"
5
7
  require "sequel"
6
8
  require "uri"
9
+ require "pathname"
7
10
 
8
11
  module Gemstash
9
12
  # Storage for application-wide variables and configuration.
@@ -15,7 +18,7 @@ module Gemstash
15
18
 
16
19
  # Little module to provide easy access to the current Gemstash::Env.
17
20
  module Helper
18
- private
21
+ private # rubocop:disable Layout/AccessModifierIndentation
19
22
 
20
23
  def gemstash_env
21
24
  Gemstash::Env.current
@@ -48,6 +51,7 @@ module Gemstash
48
51
 
49
52
  def self.current
50
53
  raise EnvNotSetError unless Thread.current[:gemstash_env]
54
+
51
55
  Thread.current[:gemstash_env]
52
56
  end
53
57
 
@@ -57,6 +61,7 @@ module Gemstash
57
61
 
58
62
  def self.daemonized?
59
63
  raise "Daemonized hasn't been set yet!" if @daemonized.nil?
64
+
60
65
  @daemonized
61
66
  end
62
67
 
@@ -105,12 +110,25 @@ module Gemstash
105
110
  end
106
111
  end
107
112
 
113
+ def pidfile
114
+ if config[:pidfile]
115
+ pathname = Pathname.new(config[:pidfile])
116
+ if pathname.relative?
117
+ base_file(pathname.to_s)
118
+ else
119
+ pathname.to_s
120
+ end
121
+ else
122
+ base_file("puma.pid")
123
+ end
124
+ end
125
+
108
126
  def atomic_write(file, &block)
109
127
  File.atomic_write(file, File.dirname(file), &block)
110
128
  end
111
129
 
112
130
  def rackup
113
- File.expand_path("../config.ru", __FILE__)
131
+ File.expand_path("config.ru", __dir__)
114
132
  end
115
133
 
116
134
  def db
@@ -122,7 +140,7 @@ module Gemstash
122
140
  db = if RUBY_PLATFORM == "java"
123
141
  Sequel.connect("jdbc:sqlite:#{db_path}", config.database_connection_config)
124
142
  else
125
- Sequel.connect("sqlite://#{URI.escape(db_path)}", config.database_connection_config)
143
+ Sequel.connect("sqlite://#{CGI.escape(db_path)}", config.database_connection_config)
126
144
  end
127
145
  when "postgres", "mysql", "mysql2"
128
146
  db = Sequel.connect(config[:db_url], config.database_connection_config)
@@ -137,7 +155,7 @@ module Gemstash
137
155
 
138
156
  def self.migrate(db)
139
157
  Sequel.extension :migration
140
- migrations_dir = File.expand_path("../migrations", __FILE__)
158
+ migrations_dir = File.expand_path("migrations", __dir__)
141
159
  Sequel::Migrator.run(db, migrations_dir, :use_transactions => true)
142
160
  end
143
161
 
@@ -152,6 +170,8 @@ module Gemstash
152
170
  Gemstash::LruReduxClient.new
153
171
  when "memcached"
154
172
  Dalli::Client.new(config[:memcached_servers])
173
+ when "redis"
174
+ Gemstash::RedisClient.new(config[:redis_servers])
155
175
  else
156
176
  raise "Invalid cache client: '#{config[:cache_type]}'"
157
177
  end