gemstash 2.1.0 → 2.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (48) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +57 -0
  3. data/lib/gemstash/cache.rb +37 -0
  4. data/lib/gemstash/cli/authorize.rb +2 -2
  5. data/lib/gemstash/cli/setup.rb +11 -3
  6. data/lib/gemstash/configuration.rb +1 -1
  7. data/lib/gemstash/dependencies.rb +2 -2
  8. data/lib/gemstash/env.rb +2 -0
  9. data/lib/gemstash/gem_fetcher.rb +2 -2
  10. data/lib/gemstash/gem_pusher.rb +2 -2
  11. data/lib/gemstash/gem_source/private_source.rb +3 -3
  12. data/lib/gemstash/gem_source.rb +2 -2
  13. data/lib/gemstash/http_client.rb +8 -3
  14. data/lib/gemstash/logging.rb +1 -1
  15. data/lib/gemstash/man/gemstash-authorize.1.txt +0 -45
  16. data/lib/gemstash/man/gemstash-configuration.5.txt +0 -209
  17. data/lib/gemstash/man/gemstash-customize.7.txt +0 -186
  18. data/lib/gemstash/man/gemstash-debugging.7.txt +0 -28
  19. data/lib/gemstash/man/gemstash-deploy.7.txt +0 -58
  20. data/lib/gemstash/man/gemstash-mirror.7.txt +0 -32
  21. data/lib/gemstash/man/gemstash-multiple-sources.7.txt +0 -70
  22. data/lib/gemstash/man/gemstash-private-gems.7.txt +0 -155
  23. data/lib/gemstash/man/gemstash-readme.7.txt +0 -181
  24. data/lib/gemstash/man/gemstash-setup.1.txt +0 -39
  25. data/lib/gemstash/man/gemstash-start.1.txt +0 -25
  26. data/lib/gemstash/man/gemstash-status.1.txt +0 -21
  27. data/lib/gemstash/man/gemstash-stop.1.txt +0 -21
  28. data/lib/gemstash/man/gemstash-version.1.txt +0 -20
  29. data/lib/gemstash/storage.rb +5 -4
  30. data/lib/gemstash/upstream.rb +6 -5
  31. data/lib/gemstash/version.rb +2 -2
  32. data/lib/gemstash/web.rb +6 -4
  33. data/lib/gemstash.rb +1 -1
  34. metadata +31 -171
  35. data/lib/gemstash/man/gemstash-authorize.1 +0 -51
  36. data/lib/gemstash/man/gemstash-configuration.5 +0 -229
  37. data/lib/gemstash/man/gemstash-customize.7 +0 -301
  38. data/lib/gemstash/man/gemstash-debugging.7 +0 -34
  39. data/lib/gemstash/man/gemstash-deploy.7 +0 -72
  40. data/lib/gemstash/man/gemstash-mirror.7 +0 -40
  41. data/lib/gemstash/man/gemstash-multiple-sources.7 +0 -89
  42. data/lib/gemstash/man/gemstash-private-gems.7 +0 -227
  43. data/lib/gemstash/man/gemstash-readme.7 +0 -233
  44. data/lib/gemstash/man/gemstash-setup.1 +0 -43
  45. data/lib/gemstash/man/gemstash-start.1 +0 -26
  46. data/lib/gemstash/man/gemstash-status.1 +0 -20
  47. data/lib/gemstash/man/gemstash-stop.1 +0 -20
  48. data/lib/gemstash/man/gemstash-version.1 +0 -22
@@ -1,301 +0,0 @@
1
- .\" Automatically generated by Pandoc 2.5
2
- .\"
3
- .TH "gemstash\-customize" "7" "October 28, 2015" "" ""
4
- .hy
5
- .SH CUSTOMIZING THE SERVER
6
- .PP
7
- Although Gemstash is designed for as minimal setup as possible, there
8
- may be times you will want to change some of the default configuration.
9
- By the end of this guide, you will be able to customize some of the
10
- Gemstash behavior, including where files are stored, what database
11
- Gemstash uses, and how Gemstash caches certain requests.
12
- .SS SETUP
13
- .PP
14
- Several customizable options are available via an interactive Gemstash
15
- command.
16
- Run \f[C]gemstash setup\f[R] and answer the questions it provides (a
17
- blank answer will use the default value):
18
- .RS
19
- .PP
20
- $ gemstash setup
21
- .PD 0
22
- .P
23
- .PD
24
- Where should files go?
25
- [\[ti]/.gemstash]
26
- .PD 0
27
- .P
28
- .PD
29
- Cache with what?
30
- [MEMORY, memcached] \f[B]memcached\f[R]
31
- .PD 0
32
- .P
33
- .PD
34
- What is the comma separated Memcached servers?
35
- [localhost:11211]
36
- .PD 0
37
- .P
38
- .PD
39
- What database adapter?
40
- [SQLITE3, postgres, mysql, mysql2] \f[B]postgres\f[R]
41
- .PD 0
42
- .P
43
- .PD
44
- Where is the database?
45
- [postgres:///gemstash]
46
- .PD 0
47
- .P
48
- .PD
49
- Checking that the cache is available
50
- .PD 0
51
- .P
52
- .PD
53
- Checking that the database is available
54
- .PD 0
55
- .P
56
- .PD
57
- The database is not available
58
- .PD 0
59
- .P
60
- .PD
61
- .RE
62
- .PP
63
- Once you\[cq]ve answered the questions, some checks will be made to
64
- ensure the configuration will work.
65
- For example, the database didn\[cq]t exist in the previous example, so
66
- the command failed and the configuration wasn\[cq]t saved.
67
- If the command passes, you may provide the \f[C]\-\-redo\f[R] option to
68
- force configuration to be redone:
69
- .RS
70
- .PP
71
- \f[C]$ gemstash setup \-\-redo\f[R]
72
- .PD 0
73
- .P
74
- .PD
75
- Where should files go?
76
- [\[ti]/.gemstash]
77
- .PD 0
78
- .P
79
- .PD
80
- Cache with what?
81
- [MEMORY, memcached] \f[B]memcached\f[R]
82
- .PD 0
83
- .P
84
- .PD
85
- What is the comma separated Memcached servers?
86
- [localhost:11211]
87
- .PD 0
88
- .P
89
- .PD
90
- What database adapter?
91
- [SQLITE3, postgres, mysql, mysql2]
92
- .PD 0
93
- .P
94
- .PD
95
- Checking that the cache is available
96
- .PD 0
97
- .P
98
- .PD
99
- Checking that the database is available
100
- .PD 0
101
- .P
102
- .PD
103
- You are all setup!
104
- .PD 0
105
- .P
106
- .PD
107
- .RE
108
- .PP
109
- Once all checks have passed, Gemstash will store your answers in the
110
- configuration file located at \f[C]\[ti]/.gemstash/config.yml\f[R].
111
- .SS FILES
112
- .PP
113
- Storage in Gemstash defaults to \f[C]\[ti]/.gemstash\f[R] unless
114
- otherwise specified.
115
- You can change this in your config file via the \f[C]:base_path\f[R]
116
- key:
117
- .IP
118
- .nf
119
- \f[C]
120
- # \[ti]/.gemstash/config.yml
121
- \-\-\-
122
- :base_path: \[dq]/var/gemstash\[dq]
123
- \f[R]
124
- .fi
125
- .PP
126
- When customizing the \f[C]base_path\f[R], the directory must exist,
127
- otherwise Gemstash will fail to run.
128
- Thus, if you want to use \f[C]/var/gemstash\f[R] like in the previous
129
- example, make sure to \f[C]mkdir /var/gemstash\f[R] and grant access to
130
- the directory for the user you run Gemstash with.
131
- .SS DATABASE
132
- .PP
133
- The \f[C]:db_adapter\f[R] configuration key specifies what database you
134
- will be using.
135
- The default \f[C]:db_adapter\f[R] is
136
- \f[C]sqlite3\f[R] (https://www.sqlite.org/), which will use a database
137
- file located within your \f[C]:base_path\f[R].
138
- The database file will always be named \f[C]gemstash.db\f[R].
139
- .PP
140
- You may also use \f[C]postgres\f[R] (http://www.postgresql.org/),
141
- \f[C]mysql\f[R] (http://www.mysql.com/), or
142
- \f[C]mysql2\f[R] (http://sequel.jeremyevans.net/rdoc/files/doc/opening_databases_rdoc.html#label-mysql2)
143
- for your \f[C]:db_adapter\f[R].
144
- When using any of these options, you need to specify the
145
- \f[C]:db_url\f[R] to point to an existing database.
146
- Here is an example configuration to use the \f[C]postgres\f[R] adapter:
147
- .IP
148
- .nf
149
- \f[C]
150
- # \[ti]/.gemstash/config.yml
151
- \-\-\-
152
- :db_adapter: postgres
153
- :db_url: postgres:///gemstash
154
- :db_connection_options: # Sequel.connect options
155
- :connect_timeout: 10
156
- :read_timeout: 5
157
- :timeout: 30
158
- \f[R]
159
- .fi
160
- .PP
161
- Regardless of the adapter you choose, the database will automatically
162
- migrate to your version of Gemstash whenever the database is needed.
163
- You only need to ensure the database exists and Gemstash will do the
164
- rest, except for \f[C]sqlite3\f[R] (for which Gemstash will also create
165
- the database for you).
166
- .SS CACHE
167
- .PP
168
- Certain things (like dependencies) are cached in memory.
169
- This avoids web calls to the gem source, and database calls for private
170
- gems.
171
- .IP
172
- .nf
173
- \f[C]
174
- # \[ti]/.gemstash/config.yml
175
- \-\-\-
176
- :cache_type: memory
177
- :cache_max_size: 2000
178
- \f[R]
179
- .fi
180
- .PP
181
- This configuration uses the default \f[C]memory\f[R] cache, and has
182
- increased the \f[C]cache_max_size\f[R] setting from its default of 500
183
- items.
184
- .PP
185
- The memory cache can optionally be swapped out with a
186
- Memcached (http://memcached.org/) server (or cluster of servers).
187
- .PP
188
- To use Memcached, use the \f[C]memcached\f[R] \f[C]:cache_type\f[R]
189
- configuration.
190
- .PP
191
- Provide the servers as a comma\-separated list to the
192
- \f[C]:memcached_servers\f[R] configuration key:
193
- .IP
194
- .nf
195
- \f[C]
196
- # \[ti]/.gemstash/config.yml
197
- \-\-\-
198
- :cache_type: memcached
199
- :memcached_servers: memcached1.local:11211,memcached2.local:11211
200
- :cache_expiration: 1800
201
- \f[R]
202
- .fi
203
- .PP
204
- All caching expires in \f[C]cache_expiration\f[R] number of seconds.
205
- Default is 1800 seconds (30 minutes).
206
- This option applies to all caching.
207
- .SS SERVER
208
- .PP
209
- Gemstash uses Puma (http://puma.io/) and Rack (http://rack.github.io/)
210
- as the server.
211
- Alternate server configurations are not currently supported, but you can
212
- take a look at the Puma
213
- configuration (https://github.com/rubygems/gemstash/blob/master/lib/gemstash/puma.rb)
214
- and the rackup
215
- file (https://github.com/rubygems/gemstash/blob/master/lib/gemstash/config.ru)
216
- for inspiration.
217
- .PP
218
- While the server is not customizable, the way Gemstash binds the port
219
- can be changed.
220
- To change the binding, update the \f[C]:bind\f[R] configuration key:
221
- .IP
222
- .nf
223
- \f[C]
224
- # \[ti]/.gemstash/config.yml
225
- \-\-\-
226
- :bind: tcp://0.0.0.0:4242
227
- \f[R]
228
- .fi
229
- .PP
230
- This maps directly to the Puma bind
231
- flag (https://github.com/puma/puma#binding-tcp--sockets), and will
232
- support anything valid for that flag.
233
- .PP
234
- The number of threads Puma uses is also customizable via the
235
- \f[C]:puma_threads\f[R] configuration key.
236
- The default is \f[C]16\f[R].
237
- .SS PROTECTED FETCH
238
- .PP
239
- Gemstash by default allows unauthenticated access for private gems.
240
- Authenticated access is available via the \f[C]:protected_fetch\f[R]
241
- configuration key.
242
- .IP
243
- .nf
244
- \f[C]
245
- # \[ti]/.gemstash/config.yml
246
- \-\-\-
247
- :protected_fetch: true
248
- \f[R]
249
- .fi
250
- .PP
251
- More details on protected_fetch are here (gemstash help private-gems.7).
252
- .SS FETCH TIMEOUT
253
- .PP
254
- The default fetch timeout is 20 seconds.
255
- Use the \f[C]:fetch_timeout\f[R] configuration key to change it.
256
- .IP
257
- .nf
258
- \f[C]
259
- \-\-\-
260
- :fetch_timeout: 20
261
- \f[R]
262
- .fi
263
- .SS CONFIG FILE LOCATION
264
- .PP
265
- By default, configuration for Gemstash will be at
266
- \f[C]\[ti]/.gemstash/config.yml\f[R].
267
- This can be changed by providing the \f[C]\-\-config\-file\f[R] option
268
- to the various Gemstash commands:
269
- .IP
270
- .nf
271
- \f[C]
272
- $ gemstash setup \-\-config\-file ./gemstash\-config.yml
273
- $ gemstash authorize \-\-config\-file ./gemstash\-config.yml
274
- $ gemstash start \-\-config\-file ./gemstash\-config.yml
275
- $ gemstash stop \-\-config\-file ./gemstash\-config.yml
276
- $ gemstash status \-\-config\-file ./gemstash\-config.yml
277
- \f[R]
278
- .fi
279
- .PP
280
- When providing \f[C]\-\-config\-file\f[R] to \f[C]gemstash setup\f[R],
281
- the provided file will be output to with the provided configuration.
282
- \f[B]This will overwrite\f[R] any existing configuration.
283
- If the file doesn\[cq]t exist when providing \f[C]\-\-config\-file\f[R]
284
- to \f[C]gemstash start\f[R], \f[C]gemstash stop\f[R],
285
- \f[C]gemstash status\f[R], and \f[C]gemstash authorize\f[R], the default
286
- configuration will be used.
287
- .SS ERB PARSED CONFIG
288
- .PP
289
- You may also create a \f[C]\[ti]/.gemstash/config.yml.erb\f[R] file.
290
- If present, this will be used instead of
291
- \f[C]\[ti]/.gemstash/config.yml\f[R].
292
- For example, with this you can use environment variables in the config:
293
- .IP
294
- .nf
295
- \f[C]
296
- # \[ti]/.gemstash/config.yml.erb
297
- \-\-\-
298
- :db_adapter: postgres
299
- :db_url: <%= ENV[\[dq]DATABASE_URL\[dq]] %>
300
- \f[R]
301
- .fi
@@ -1,34 +0,0 @@
1
- .\" Automatically generated by Pandoc 2.5
2
- .\"
3
- .TH "gemstash\-debugging" "7" "October 28, 2015" "" ""
4
- .hy
5
- .SH DEBUGGING GEMSTASH
6
- .PP
7
- If you are finding Gemstash isn\[cq]t behaving as you would expect, you
8
- might want to start by looking at the server log.
9
- You can find the log at \f[C]server.log\f[R] within your base directory.
10
- By default, this will be at \f[C]\[ti]/.gemstash/server.log\f[R].
11
- .PP
12
- You might find it easier to view the log directly in your terminal.
13
- If you run Gemstash in non\-daemonized form (gemstash help start.1), the
14
- log will be output directly to standard out:
15
- .IP
16
- .nf
17
- \f[C]
18
- $ gemstash start \-\-no\-daemonize
19
- \f[R]
20
- .fi
21
- .PP
22
- You can also check the status (gemstash help status.1) of the server:
23
- .IP
24
- .nf
25
- \f[C]
26
- $ gemstash status
27
- \f[R]
28
- .fi
29
- .PP
30
- The server status is checked by passing through to
31
- pumactl (https://github.com/puma/puma#pumactl).
32
- .PP
33
- If you find a bug, please don\[cq]t hesitate to open a bug
34
- report (https://github.com/rubygems/gemstash#contributing)!
@@ -1,72 +0,0 @@
1
- .\" Automatically generated by Pandoc 2.5
2
- .\"
3
- .TH "gemstash\-deploy" "7" "October 25, 2015" "" ""
4
- .hy
5
- .SH DEPLOYING GEMSTASH
6
- .PP
7
- Bundler is here for the rescue to keep Gemstash up to date! Create a
8
- \f[C]Gemfile\f[R] pointing to Gemstash:
9
- .IP
10
- .nf
11
- \f[C]
12
- # ./Gemfile
13
- source \[dq]https://rubygems.org\[dq]
14
- gem \[dq]gemstash\[dq]
15
- \f[R]
16
- .fi
17
- .PP
18
- Then \f[C]bundle\f[R] to create your \f[C]Gemfile.lock\f[R].
19
- When you are ready to upgrade, simply \f[C]bundle update\f[R].
20
- You may need to run \f[C]gemstash\f[R] via \f[C]bundle exec\f[R].
21
- Alternatively, you can \f[C]gem uninstall gemstash\f[R] and
22
- \f[C]gem install gemstash\f[R] when you want to upgrade.
23
- .PP
24
- Gemstash will automatically run any necessary migrations, so updating
25
- the gem is all that needs to be done.
26
- .PP
27
- It is probably wise to stop Gemstash before upgrading, then starting
28
- again once you are done:
29
- .IP
30
- .nf
31
- \f[C]
32
- $ bundle exec gemstash stop
33
- $ bundle update
34
- $ bundle exec gemstash start
35
- \f[R]
36
- .fi
37
- .SS MONITORING
38
- .PP
39
- Health monitoring is built in to Gemstash using the
40
- server_health_check\-rack (https://github.com/on-site/server_health_check-rack)
41
- gem.
42
- If you request \f[C]/health\f[R] from your Gemstash instance, you will
43
- get a JSON response along with an HTTP status code indicating success or
44
- failure.
45
- The JSON response will look something like this for a success case:
46
- .IP
47
- .nf
48
- \f[C]
49
- {
50
- \[dq]status\[dq]: {
51
- \[dq]heartbeat\[dq]: \[dq]OK\[dq],
52
- \[dq]storage_read\[dq]: \[dq]OK\[dq],
53
- \[dq]storage_write\[dq]: \[dq]OK\[dq],
54
- \[dq]db_read\[dq]: \[dq]OK\[dq],
55
- \[dq]db_write\[dq]: \[dq]OK\[dq]
56
- }
57
- }
58
- \f[R]
59
- .fi
60
- .PP
61
- This request will test storage and database access and report on the
62
- result.
63
- Each key in the status can be requested alone to just report on that
64
- status.
65
- For example, if you would like a health check that doesn\[cq]t interact
66
- with storage or the database, you can use \f[C]/health/heartbeat\f[R]
67
- which will always respond with a success while your Gemstash server is
68
- running.
69
- .SS DOWNGRADING
70
- .PP
71
- It is not recommended to go backwards in Gemstash versions.
72
- Migrations may have run that could leave the database in a bad state.
@@ -1,40 +0,0 @@
1
- .\" Automatically generated by Pandoc 2.5
2
- .\"
3
- .TH "gemstash\-mirror" "7" "October 25, 2015" "" ""
4
- .hy
5
- .SH USING GEMSTASH AS A MIRROR
6
- .PP
7
- If you don\[cq]t have control over your \f[C]Gemfile\f[R], or you
8
- don\[cq]t want to force everyone on your team to go through the Gemstash
9
- server, you can use Bundler mirroring to bundle against your Gemstash
10
- server.
11
- .PP
12
- For each source in your \f[C]Gemfile\f[R], add a mirror pointing to your
13
- Gemstash server:
14
- .IP
15
- .nf
16
- \f[C]
17
- $ bundle config mirror.http://rubygems.org http://localhost:9292
18
- $ bundle config mirror.https://my.gem\-source.local http://localhost:9292/upstream/$(ruby \-rcgi \-e \[aq]puts CGI.escape(\[dq]https://my.gem\-source.local\[dq])\[aq])
19
- \f[R]
20
- .fi
21
- .PP
22
- From now on, bundler will fetch gems from those sources via your
23
- Gemstash server.
24
- .SH SIMPLER GEMSTASH MIRRORS
25
- .PP
26
- \f[B]This feature requires Bundler to be at least version
27
- \f[CB]1.11.0\f[B].\f[R]
28
- .PP
29
- If you are using Bundler version \f[C]1.11.0\f[R] or greater, the
30
- mirroring becomes a bit easier:
31
- .IP
32
- .nf
33
- \f[C]
34
- $ bundle config mirror.http://rubygems.org http://localhost:9292
35
- $ bundle config mirror.https://my.gem\-source.local http://localhost:9292
36
- \f[R]
37
- .fi
38
- .PP
39
- Bundler will then send headers to Gemstash to indicate the correct
40
- upstream.
@@ -1,89 +0,0 @@
1
- .\" Automatically generated by Pandoc 2.5
2
- .\"
3
- .TH "gemstash\-multiple\-sources" "7" "October 8, 2015" "" ""
4
- .hy
5
- .SH MULTIPLE GEM SOURCES
6
- .PP
7
- Gemstash will stash from any amount of gem sources.
8
- By the end of this guide, you will be able to bundle using multiple gem
9
- sources, all stashed within your Gemstash server.
10
- .SS DEFAULT SOURCE
11
- .PP
12
- When you don\[cq]t provide an explicit source (as with the Quickstart
13
- Guide (../readme.md#quickstart-guide)), your gems will be fetched from
14
- https://rubygems.org.
15
- This default source is not set in stone.
16
- To change it, you need only edit the Gemstash configuration found at
17
- \f[C]\[ti]/.gemstash/config.yml\f[R]:
18
- .IP
19
- .nf
20
- \f[C]
21
- # \[ti]/.gemstash/config.yml
22
- \-\-\-
23
- :rubygems_url: https://my.gem\-source.local
24
- \f[R]
25
- .fi
26
- .PP
27
- Make sure to restart your Gemstash server after changing the config:
28
- .IP
29
- .nf
30
- \f[C]
31
- $ gemstash stop
32
- $ gemstash start
33
- \f[R]
34
- .fi
35
- .PP
36
- Once restarted, bundling against \f[C]http://localhost:9292\f[R] will
37
- fetch gems from \f[C]https://my.gem\-source.local\f[R].
38
- If you had bundled before making these changes, fear not; bundling with
39
- a different default gem source will store gems in a separate location,
40
- ensuring different sources won\[cq]t leak between each other.
41
- .SS BUNDLING WITH MULTIPLE SOURCES
42
- .PP
43
- Changing the default source won\[cq]t help you if you need to bundle
44
- against https://rubygems.org along with additional sources.
45
- If you need to bundle with multiple gem sources, Gemstash doesn\[cq]t
46
- need to be specially configured.
47
- Your Gemstash server will honor any gem source specified via a
48
- specialized URL.
49
- Consider the following \f[C]Gemfile\f[R]:
50
- .IP
51
- .nf
52
- \f[C]
53
- # ./Gemfile
54
- require \[dq]cgi\[dq]
55
- source \[dq]http://localhost:9292\[dq]
56
- gem \[dq]rubywarrior\[dq]
57
-
58
- source \[dq]http://localhost:9292/upstream/#{CGI.escape(\[dq]https://my.gem\-source.local\[dq])}\[dq] do
59
- gem \[dq]my\-gem\[dq]
60
- end
61
- \f[R]
62
- .fi
63
- .PP
64
- Notice the \f[C]CGI.escape\f[R] call in the second source.
65
- This is important, as it properly URL escapes the source URL so Gemstash
66
- knows what gem source you want.
67
- The \f[C]/upstream\f[R] prefix tells Gemstash to use a gem source other
68
- than the default source.
69
- You can now bundle with the additional source.
70
- .SS REDIRECTING
71
- .PP
72
- Gemstash supports an alternate mode of specifying your gem sources.
73
- If you want Gemstash to redirect Bundler to your given gem sources, then
74
- you can specify your \f[C]Gemfile\f[R] like so:
75
- .IP
76
- .nf
77
- \f[C]
78
- # ./Gemfile
79
- require \[dq]cgi\[dq]
80
- source \[dq]http://localhost:9292/redirect/#{CGI.escape(\[dq]https://rubygems.org\[dq])}\[dq]
81
- gem \[dq]rubywarrior\[dq]
82
- \f[R]
83
- .fi
84
- .PP
85
- Notice the \f[C]/redirect\f[R] prefix.
86
- This prefix tells Gemstash to redirect API calls to the provided URL.
87
- Redirected calls like this will not be cached by Gemstash, and gem files
88
- will not be stashed, even if they were previously cached or stashed from
89
- the same gem source.