gemstash 1.0.4 → 1.1.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (49) hide show
  1. checksums.yaml +5 -5
  2. data/CHANGELOG.md +33 -0
  3. data/lib/gemstash.rb +2 -0
  4. data/lib/gemstash/api_key_authorization.rb +29 -0
  5. data/lib/gemstash/authorization.rb +5 -1
  6. data/lib/gemstash/cli/authorize.rb +1 -1
  7. data/lib/gemstash/cli/setup.rb +50 -37
  8. data/lib/gemstash/cli/start.rb +1 -1
  9. data/lib/gemstash/config.ru +1 -0
  10. data/lib/gemstash/configuration.rb +35 -3
  11. data/lib/gemstash/db/version.rb +11 -0
  12. data/lib/gemstash/env.rb +16 -10
  13. data/lib/gemstash/gem_fetcher.rb +1 -1
  14. data/lib/gemstash/gem_pusher.rb +15 -17
  15. data/lib/gemstash/gem_source/private_source.rb +29 -44
  16. data/lib/gemstash/gem_source/upstream_source.rb +4 -4
  17. data/lib/gemstash/gem_unyanker.rb +10 -4
  18. data/lib/gemstash/gem_yanker.rb +10 -4
  19. data/lib/gemstash/health.rb +53 -0
  20. data/lib/gemstash/http_client.rb +5 -3
  21. data/lib/gemstash/logging.rb +1 -1
  22. data/lib/gemstash/man/gemstash-authorize.1 +6 -5
  23. data/lib/gemstash/man/gemstash-authorize.1.txt +14 -13
  24. data/lib/gemstash/man/gemstash-configuration.5 +90 -8
  25. data/lib/gemstash/man/gemstash-configuration.5.txt +82 -6
  26. data/lib/gemstash/man/gemstash-customize.7 +68 -12
  27. data/lib/gemstash/man/gemstash-customize.7.txt +64 -26
  28. data/lib/gemstash/man/gemstash-debugging.7 +1 -1
  29. data/lib/gemstash/man/gemstash-deploy.7 +33 -1
  30. data/lib/gemstash/man/gemstash-deploy.7.txt +25 -0
  31. data/lib/gemstash/man/gemstash-mirror.7 +1 -1
  32. data/lib/gemstash/man/gemstash-multiple-sources.7 +1 -1
  33. data/lib/gemstash/man/gemstash-private-gems.7 +52 -3
  34. data/lib/gemstash/man/gemstash-private-gems.7.txt +39 -4
  35. data/lib/gemstash/man/gemstash-readme.7 +34 -4
  36. data/lib/gemstash/man/gemstash-readme.7.txt +34 -12
  37. data/lib/gemstash/man/gemstash-setup.1 +3 -1
  38. data/lib/gemstash/man/gemstash-setup.1.txt +3 -1
  39. data/lib/gemstash/man/gemstash-start.1 +4 -3
  40. data/lib/gemstash/man/gemstash-start.1.txt +5 -4
  41. data/lib/gemstash/man/gemstash-status.1 +4 -3
  42. data/lib/gemstash/man/gemstash-status.1.txt +3 -2
  43. data/lib/gemstash/man/gemstash-stop.1 +4 -3
  44. data/lib/gemstash/man/gemstash-stop.1.txt +3 -2
  45. data/lib/gemstash/man/gemstash-version.1 +1 -1
  46. data/lib/gemstash/puma.rb +2 -2
  47. data/lib/gemstash/specs_builder.rb +14 -16
  48. data/lib/gemstash/version.rb +1 -1
  49. metadata +29 -7
@@ -25,6 +25,31 @@ DEPLOYING GEMSTASH
25
25
  $ bundle update
26
26
  $ bundle exec gemstash start
27
27
 
28
+ MONITORING
29
+ Health monitoring is built in to Gemstash using the serv-
30
+ er_health_check-rack (https://github.com/on-site/server_health_check-
31
+ rack) gem. If you request /health from your Gemstash instance, you
32
+ will get a JSON response along with an HTTP status code indicating suc-
33
+ cess or failure. The JSON response will look something like this for a
34
+ success case:
35
+
36
+ {
37
+ "status": {
38
+ "heartbeat": "OK",
39
+ "storage_read": "OK",
40
+ "storage_write": "OK",
41
+ "db_read": "OK",
42
+ "db_write": "OK"
43
+ }
44
+ }
45
+
46
+ This request will test storage and database access and report on the
47
+ result. Each key in the status can be requested alone to just report
48
+ on that status. For example, if you would like a health check that
49
+ doesn't interact with storage or the database, you can use
50
+ /health/heartbeat which will always respond with a success while your
51
+ Gemstash server is running.
52
+
28
53
  DOWNGRADING
29
54
  It is not recommended to go backwards in Gemstash versions. Migrations
30
55
  may have run that could leave the database in a bad state.
@@ -1,4 +1,4 @@
1
- .\" Automatically generated by Pandoc 1.16.0.2
1
+ .\" Automatically generated by Pandoc 1.19.2.1
2
2
  .\"
3
3
  .TH "gemstash\-mirror" "7" "October 25, 2015" "" ""
4
4
  .hy
@@ -1,4 +1,4 @@
1
- .\" Automatically generated by Pandoc 1.16.0.2
1
+ .\" Automatically generated by Pandoc 1.19.2.1
2
2
  .\"
3
3
  .TH "gemstash\-multiple\-sources" "7" "October 8, 2015" "" ""
4
4
  .hy
@@ -1,4 +1,4 @@
1
- .\" Automatically generated by Pandoc 1.16.0.2
1
+ .\" Automatically generated by Pandoc 1.19.2.1
2
2
  .\"
3
3
  .TH "gemstash\-private\-gems" "7" "October 8, 2015" "" ""
4
4
  .hy
@@ -29,8 +29,8 @@ Your\ new\ key\ is:\ e374e237fdf5fa5718d2a21bd63dc911
29
29
  \f[]
30
30
  .fi
31
31
  .PP
32
- This new key can \f[C]push\f[], \f[C]yank\f[], and \f[C]unyank\f[] gems
33
- from your Gemstash server.
32
+ This new key can \f[C]push\f[], \f[C]yank\f[], \f[C]unyank\f[], and
33
+ \f[C]fetch\f[] gems from your Gemstash server.
34
34
  Run \f[C]gemstash\ authorize\f[] with just the permissions you want to
35
35
  limit what the key will be allowed to do.
36
36
  You can similarly update a specific key by providing it via the
@@ -193,3 +193,52 @@ Gemstash will only let you unyank from \f[C]/private\f[].
193
193
  Unlike pushing, Rubygems doesn\[aq]t support \f[C]\-\-host\f[] for
194
194
  unyank and yank (yet), so you need to specify the host via the
195
195
  \f[C]RUBYGEMS_HOST\f[] environment variable.
196
+ .SS PROTECTED FETCHING
197
+ .PP
198
+ By default, private gems and specs can be accessed without
199
+ authentication.
200
+ .PP
201
+ Private gems often require protected fetching.
202
+ For backwards compatibility this is disabled by default, but can be
203
+ enabled via \f[C]$\ gemstash\ setup\f[] command.
204
+ .PP
205
+ When protected fetching is enabled API keys with the permissions
206
+ \f[C]all\f[] or \f[C]fetch\f[] can be used to download gems and specs.
207
+ .PP
208
+ On the Bundler side, there are a few ways to configure credentials for a
209
+ given gem source:
210
+ .PP
211
+ Add credentials globally:
212
+ .IP
213
+ .nf
214
+ \f[C]
215
+ $\ bundle\ config\ my\-gemstash.dev\ api_key
216
+ \f[]
217
+ .fi
218
+ .PP
219
+ Add credentials in Gemfile:
220
+ .IP
221
+ .nf
222
+ \f[C]
223
+ source\ "https://api_key\@my\-gemstash.dev"
224
+ \f[]
225
+ .fi
226
+ .PP
227
+ However, it\[aq]s not a good practice to commit credentials to source
228
+ control.
229
+ A recommended solution is to use Bundler\[aq]s configuration
230
+ keys (http://bundler.io/man/bundle-config.1.html#CONFIGURATION-KEYS),
231
+ e.g.:
232
+ .IP
233
+ .nf
234
+ \f[C]
235
+ $\ export\ BUNDLE_MYGEMSTASH__DEV=api_key
236
+ \f[]
237
+ .fi
238
+ .PP
239
+ Behind the scene, Bundler will pick up the ENV var according to the host
240
+ name (e.g.
241
+ mygemstash.dev) and add to \f[C]URI.userinfo\f[] for making requests.
242
+ .PP
243
+ The API key is treated as a HTTP Basic Auth username and any HTTP Basic
244
+ password supplied will be ignored.
@@ -22,10 +22,10 @@ PRIVATE GEMS
22
22
  $ gemstash authorize
23
23
  Your new key is: e374e237fdf5fa5718d2a21bd63dc911
24
24
 
25
- This new key can push, yank, and unyank gems from your Gemstash server.
26
- Run gemstash authorize with just the permissions you want to limit what
27
- the key will be allowed to do. You can similarly update a specific key
28
- by providing it via the --key option:
25
+ This new key can push, yank, unyank, and fetch gems from your Gemstash
26
+ server. Run gemstash authorize with just the permissions you want to
27
+ limit what the key will be allowed to do. You can similarly update a
28
+ specific key by providing it via the --key option:
29
29
 
30
30
  $ gemstash authorize push yank --key e374e237fdf5fa5718d2a21bd63dc911
31
31
 
@@ -128,6 +128,41 @@ PRIVATE GEMS
128
128
  support --host for unyank and yank (yet), so you need to specify the
129
129
  host via the RUBYGEMS_HOST environment variable.
130
130
 
131
+ PROTECTED FETCHING
132
+ By default, private gems and specs can be accessed without authentica-
133
+ tion.
134
+
135
+ Private gems often require protected fetching. For backwards compati-
136
+ bility this is disabled by default, but can be enabled via $ gem-
137
+ stash setup command.
138
+
139
+ When protected fetching is enabled API keys with the permissions all or
140
+ fetch can be used to download gems and specs.
141
+
142
+ On the Bundler side, there are a few ways to configure credentials for
143
+ a given gem source:
144
+
145
+ Add credentials globally:
146
+
147
+ $ bundle config my-gemstash.dev api_key
148
+
149
+ Add credentials in Gemfile:
150
+
151
+ source "https://api_key@my-gemstash.dev"
152
+
153
+ However, it's not a good practice to commit credentials to source con-
154
+ trol. A recommended solution is to use Bundler's configuration keys
155
+ (http://bundler.io/man/bundle-config.1.html#CONFIGURATION-KEYS), e.g.:
156
+
157
+ $ export BUNDLE_MYGEMSTASH__DEV=api_key
158
+
159
+ Behind the scene, Bundler will pick up the ENV var according to the
160
+ host name (e.g. mygemstash.dev) and add to URI.userinfo for making re-
161
+ quests.
162
+
163
+ The API key is treated as a HTTP Basic Auth username and any HTTP Basic
164
+ password supplied will be ignored.
165
+
131
166
 
132
167
 
133
168
  October 8, 2015 gemstash-private-gems(7)
@@ -1,4 +1,4 @@
1
- .\" Automatically generated by Pandoc 1.16.0.2
1
+ .\" Automatically generated by Pandoc 1.19.2.1
2
2
  .\"
3
3
  .TH "gemstash\-readme" "7" "November 30, 2015" "" ""
4
4
  .hy
@@ -22,6 +22,14 @@ Are you only using gems from https://rubygems.org, and don\[aq]t bundle
22
22
  the same gems frequently?
23
23
  Well, maybe you don\[aq]t need Gemstash...
24
24
  yet.
25
+ .PP
26
+ Gemstash is maintained by Ruby Together (https://rubytogether.org/), a
27
+ grassroots initiative committed to supporting the critical Ruby
28
+ infrastructure you rely on.
29
+ Contribute today as an individual (https://rubytogether.org/developers)
30
+ or even better, as a company (https://rubytogether.org/companies), and
31
+ ensure that Bundler, RubyGems, Gemstash, and other shared tooling is
32
+ around for years to come.
25
33
  .SS QUICKSTART GUIDE
26
34
  .SS SETUP
27
35
  .PP
@@ -86,9 +94,10 @@ $\ bundle\ install\ \-\-path\ .bundle
86
94
  Your Gemstash server has fetched the gems from https://rubygems.org and
87
95
  cached them for you! To prove this, you can disable your Internet
88
96
  connection and try again.
89
- The gem dependencies from https://rubygems.org are cached for 30
90
- minutes, so if you bundle again before that, you can successfully bundle
91
- without an Internet connection:
97
+ Gem files (*.gem) are cached indefinitely.
98
+ Gem dependencies metadata are cached for 30 minutes, so if you bundle
99
+ again before that, you can successfully bundle without an Internet
100
+ connection:
92
101
  .IP
93
102
  .nf
94
103
  \f[C]
@@ -97,6 +106,27 @@ $\ rm\ \-rf\ Gemfile.lock\ .bundle
97
106
  $\ bundle
98
107
  \f[]
99
108
  .fi
109
+ .SS FALLING BACK TO RUBYGEMS.ORG
110
+ .PP
111
+ If you want to make sure that your bundling from https://rubygems.org
112
+ still works as expected when the Gemstash server is not running, you can
113
+ easily configure Bundler to fallback to https://rubygems.org.
114
+ .IP
115
+ .nf
116
+ \f[C]
117
+ $\ bundle\ config\ mirror.https://rubygems.org.fallback_timeout\ true
118
+ \f[]
119
+ .fi
120
+ .PP
121
+ You can also configure this fallback as a number of seconds in case the
122
+ Gemstash server is simply unresponsive.
123
+ This example uses a 3 second timeout:
124
+ .IP
125
+ .nf
126
+ \f[C]
127
+ $\ bundle\ config\ mirror.https://rubygems.org.fallback_timeout\ 3
128
+ \f[]
129
+ .fi
100
130
  .SS STOPPING THE SERVER
101
131
  .PP
102
132
  Once you\[aq]ve finish using your Gemstash server, you can stop it just
@@ -20,27 +20,35 @@ GEMSTASH
20
20
  Are you only using gems from https://rubygems.org, and don't bundle the
21
21
  same gems frequently? Well, maybe you don't need Gemstash... yet.
22
22
 
23
+ Gemstash is maintained by Ruby Together (https://rubytogether.org/), a
24
+ grassroots initiative committed to supporting the critical Ruby infra-
25
+ structure you rely on. Contribute today as an individual
26
+ (https://rubytogether.org/developers) or even better, as a company
27
+ (https://rubytogether.org/companies), and ensure that Bundler,
28
+ RubyGems, Gemstash, and other shared tooling is around for years to
29
+ come.
30
+
23
31
  QUICKSTART GUIDE
24
32
  SETUP
25
33
  Gemstash is designed to be quick and painless to get set up. By the
26
34
  end of this Quickstart Guide, you will be able to bundle stashed gems
27
- from public sources against a Gemstash server running on your machine.
35
+ from public sources against a Gemstash server running on your machine.
28
36
 
29
37
  Install Gemstash to get started:
30
38
 
31
39
  $ gem install gemstash
32
40
 
33
- After it is installed, starting Gemstash requires no additional steps.
41
+ After it is installed, starting Gemstash requires no additional steps.
34
42
  Simply start the Gemstash server with the gemstash command:
35
43
 
36
44
  $ gemstash start
37
45
 
38
- You may have noticed that the command finished quickly. This is be-
39
- cause Gemstash will run the server in the background by default. The
46
+ You may have noticed that the command finished quickly. This is be-
47
+ cause Gemstash will run the server in the background by default. The
40
48
  server runs on port 9292.
41
49
 
42
50
  BUNDLING
43
- With the server running, you can bundle against it. Tell Bundler that
51
+ With the server running, you can bundle against it. Tell Bundler that
44
52
  you want to use Gemstash to find gems from RubyGems.org:
45
53
 
46
54
  $ bundle config mirror.https://rubygems.org http://localhost:9292
@@ -51,21 +59,35 @@ GEMSTASH
51
59
  source "https://rubygems.org"
52
60
  gem "rubywarrior"
53
61
 
54
- The gems you include should be gems you don't yet have installed, oth-
62
+ The gems you include should be gems you don't yet have installed, oth-
55
63
  erwise Gemstash will have nothing to stash. Now bundle:
56
64
 
57
65
  $ bundle install --path .bundle
58
66
 
59
67
  Your Gemstash server has fetched the gems from https://rubygems.org and
60
- cached them for you! To prove this, you can disable your Internet con-
61
- nection and try again. The gem dependencies from https://rubygems.org
62
- are cached for 30 minutes, so if you bundle again before that, you can
63
- successfully bundle without an Internet connection:
68
+ cached them for you! To prove this, you can disable your Internet con-
69
+ nection and try again. Gem files (*.gem) are cached indefinitely. Gem
70
+ dependencies metadata are cached for 30 minutes, so if you bundle again
71
+ before that, you can successfully bundle without an Internet connec-
72
+ tion:
64
73
 
65
74
  $ # Disable your Internet first!
66
75
  $ rm -rf Gemfile.lock .bundle
67
76
  $ bundle
68
77
 
78
+ FALLING BACK TO RUBYGEMS.ORG
79
+ If you want to make sure that your bundling from https://rubygems.org
80
+ still works as expected when the Gemstash server is not running, you
81
+ can easily configure Bundler to fallback to https://rubygems.org.
82
+
83
+ $ bundle config mirror.https://rubygems.org.fallback_timeout true
84
+
85
+ You can also configure this fallback as a number of seconds in case the
86
+ Gemstash server is simply unresponsive. This example uses a 3 second
87
+ timeout:
88
+
89
+ $ bundle config mirror.https://rubygems.org.fallback_timeout 3
90
+
69
91
  STOPPING THE SERVER
70
92
  Once you've finish using your Gemstash server, you can stop it just as
71
93
  easily as you started it:
@@ -87,8 +109,8 @@ GEMSTASH
87
109
  Gemstash uses SQLite (https://www.sqlite.org/) to store details about
88
110
  private gems. The database will be located in ~/.gemstash, however you
89
111
  won't see the database appear until you start using private gems. If
90
- you prefer, you can use a different database (gemstash help custom-
91
- ize.7).
112
+ you prefer, you can use a different database (gemstash help cus-
113
+ tomize.7).
92
114
 
93
115
  Gemstash temporarily caches things like gem dependencies in memory.
94
116
  Anything cached in memory will last for 30 minutes before being re-
@@ -1,4 +1,4 @@
1
- .\" Automatically generated by Pandoc 1.16.0.2
1
+ .\" Automatically generated by Pandoc 1.19.2.1
2
2
  .\"
3
3
  .TH "gemstash\-setup" "1" "October 9, 2015" "" ""
4
4
  .hy
@@ -39,3 +39,5 @@ Without this option, your configuration will be written to
39
39
  \f[C]~/.gemstash/config.yml\f[].
40
40
  If you write to a custom location, you will need to pass the
41
41
  \f[C]\-\-config\-file\f[] option to all Gemstash commands.
42
+ If you plan to use ERB in your config file (gemstash help customize.7),
43
+ you might want to use \f[C]~/.gemstash/config.yml.erb\f[].
@@ -31,7 +31,9 @@ OPTIONS
31
31
  o --config-file FILE: Specify the config file to write to. Without
32
32
  this option, your configuration will be written to ~/.gemstash/con-
33
33
  fig.yml. If you write to a custom location, you will need to pass
34
- the --config-file option to all Gemstash commands.
34
+ the --config-file option to all Gemstash commands. If you plan to
35
+ use ERB in your config file (gemstash help customize.7), you might
36
+ want to use ~/.gemstash/config.yml.erb.
35
37
 
36
38
 
37
39
 
@@ -1,4 +1,4 @@
1
- .\" Automatically generated by Pandoc 1.16.0.2
1
+ .\" Automatically generated by Pandoc 1.19.2.1
2
2
  .\"
3
3
  .TH "gemstash\-start" "1" "October 9, 2015" "" ""
4
4
  .hy
@@ -15,8 +15,9 @@ Starts the Gemstash server.
15
15
  .IP \[bu] 2
16
16
  \f[C]\-\-config\-file\ FILE\f[]: Specify the config file to use.
17
17
  If you aren\[aq]t using the default config file at
18
- \f[C]~/.gemstash/config.yml\f[], then you must specify the config file
19
- via this option.
18
+ \f[C]~/.gemstash/config.yml\f[] or
19
+ \f[C]~/.gemstash/config.yml.erb\f[] (gemstash help customize.7), then
20
+ you must specify the config file via this option.
20
21
  .IP \[bu] 2
21
22
  \f[C]\-\-no\-daemonize\f[]: The Gemstash server daemonizes itself by
22
23
  default.
@@ -13,11 +13,12 @@ DESCRIPTION
13
13
 
14
14
  OPTIONS
15
15
  o --config-file FILE: Specify the config file to use. If you aren't
16
- using the default config file at ~/.gemstash/config.yml, then you
17
- must specify the config file via this option.
16
+ using the default config file at ~/.gemstash/config.yml or ~/.gem-
17
+ stash/config.yml.erb (gemstash help customize.7), then you must spec-
18
+ ify the config file via this option.
18
19
 
19
- o --no-daemonize: The Gemstash server daemonizes itself by default.
20
- Provide this option to instead run the server until Ctrl-C is typed.
20
+ o --no-daemonize: The Gemstash server daemonizes itself by default.
21
+ Provide this option to instead run the server until Ctrl-C is typed.
21
22
  When not daemonized, the log will be output to standard out.
22
23
 
23
24
 
@@ -1,4 +1,4 @@
1
- .\" Automatically generated by Pandoc 1.16.0.2
1
+ .\" Automatically generated by Pandoc 1.19.2.1
2
2
  .\"
3
3
  .TH "gemstash\-status" "1" "October 28, 2015" "" ""
4
4
  .hy
@@ -15,5 +15,6 @@ Checks status of the Gemstash server.
15
15
  .IP \[bu] 2
16
16
  \f[C]\-\-config\-file\ FILE\f[]: Specify the config file to use.
17
17
  If you aren\[aq]t using the default config file at
18
- \f[C]~/.gemstash/config.yml\f[], then you must specify the config file
19
- via this option.
18
+ \f[C]~/.gemstash/config.yml\f[] or
19
+ \f[C]~/.gemstash/config.yml.erb\f[] (gemstash help customize.7), then
20
+ you must specify the config file via this option.
@@ -13,8 +13,9 @@ DESCRIPTION
13
13
 
14
14
  OPTIONS
15
15
  o --config-file FILE: Specify the config file to use. If you aren't
16
- using the default config file at ~/.gemstash/config.yml, then you
17
- must specify the config file via this option.
16
+ using the default config file at ~/.gemstash/config.yml or ~/.gem-
17
+ stash/config.yml.erb (gemstash help customize.7), then you must spec-
18
+ ify the config file via this option.
18
19
 
19
20
 
20
21
 
@@ -1,4 +1,4 @@
1
- .\" Automatically generated by Pandoc 1.16.0.2
1
+ .\" Automatically generated by Pandoc 1.19.2.1
2
2
  .\"
3
3
  .TH "gemstash\-stop" "1" "October 9, 2015" "" ""
4
4
  .hy
@@ -15,5 +15,6 @@ Stops the Gemstash server.
15
15
  .IP \[bu] 2
16
16
  \f[C]\-\-config\-file\ FILE\f[]: Specify the config file to use.
17
17
  If you aren\[aq]t using the default config file at
18
- \f[C]~/.gemstash/config.yml\f[], then you must specify the config file
19
- via this option.
18
+ \f[C]~/.gemstash/config.yml\f[] or
19
+ \f[C]~/.gemstash/config.yml.erb\f[] (gemstash help customize.7), then
20
+ you must specify the config file via this option.
@@ -13,8 +13,9 @@ DESCRIPTION
13
13
 
14
14
  OPTIONS
15
15
  o --config-file FILE: Specify the config file to use. If you aren't
16
- using the default config file at ~/.gemstash/config.yml, then you
17
- must specify the config file via this option.
16
+ using the default config file at ~/.gemstash/config.yml or ~/.gem-
17
+ stash/config.yml.erb (gemstash help customize.7), then you must spec-
18
+ ify the config file via this option.
18
19
 
19
20
 
20
21
 
@@ -1,4 +1,4 @@
1
- .\" Automatically generated by Pandoc 1.16.0.2
1
+ .\" Automatically generated by Pandoc 1.19.2.1
2
2
  .\"
3
3
  .TH "gemstash\-version" "1" "December 14, 2015" "" ""
4
4
  .hy