gemstash 1.1.0 → 2.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (54) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +34 -0
  3. data/lib/gemstash.rb +0 -1
  4. data/lib/gemstash/authorization.rb +1 -5
  5. data/lib/gemstash/cli.rb +4 -4
  6. data/lib/gemstash/cli/authorize.rb +4 -2
  7. data/lib/gemstash/cli/base.rb +5 -4
  8. data/lib/gemstash/cli/setup.rb +2 -2
  9. data/lib/gemstash/cli/status.rb +1 -1
  10. data/lib/gemstash/cli/stop.rb +1 -1
  11. data/lib/gemstash/configuration.rb +3 -2
  12. data/lib/gemstash/gem_pusher.rb +7 -0
  13. data/lib/gemstash/gem_source/private_source.rb +0 -4
  14. data/lib/gemstash/gem_source/upstream_source.rb +0 -4
  15. data/lib/gemstash/health.rb +3 -3
  16. data/lib/gemstash/logging.rb +2 -4
  17. data/lib/gemstash/man/gemstash-authorize.1.txt +53 -33
  18. data/lib/gemstash/man/gemstash-configuration.5.txt +10 -141
  19. data/lib/gemstash/man/gemstash-customize.7.txt +66 -175
  20. data/lib/gemstash/man/gemstash-debugging.7.txt +54 -17
  21. data/lib/gemstash/man/gemstash-deploy.7.txt +51 -44
  22. data/lib/gemstash/man/gemstash-mirror.7.txt +53 -20
  23. data/lib/gemstash/man/gemstash-multiple-sources.7.txt +48 -53
  24. data/lib/gemstash/man/gemstash-private-gems.7.txt +15 -117
  25. data/lib/gemstash/man/gemstash-readme.7.txt +10 -126
  26. data/lib/gemstash/man/gemstash-setup.1.txt +54 -28
  27. data/lib/gemstash/man/gemstash-start.1.txt +56 -16
  28. data/lib/gemstash/man/gemstash-status.1.txt +57 -13
  29. data/lib/gemstash/man/gemstash-stop.1.txt +57 -13
  30. data/lib/gemstash/man/gemstash-version.1.txt +57 -12
  31. data/lib/gemstash/migrations/01_gem_dependencies.rb +2 -2
  32. data/lib/gemstash/migrations/03_cached_gems.rb +1 -1
  33. data/lib/gemstash/migrations/04_health_tests.rb +8 -0
  34. data/lib/gemstash/puma.rb +1 -1
  35. data/lib/gemstash/storage.rb +1 -0
  36. data/lib/gemstash/upstream.rb +2 -2
  37. data/lib/gemstash/version.rb +1 -1
  38. data/lib/gemstash/web.rb +5 -4
  39. metadata +20 -28
  40. data/lib/gemstash/gem_unyanker.rb +0 -67
  41. data/lib/gemstash/man/gemstash-authorize.1 +0 -51
  42. data/lib/gemstash/man/gemstash-configuration.5 +0 -215
  43. data/lib/gemstash/man/gemstash-customize.7 +0 -280
  44. data/lib/gemstash/man/gemstash-debugging.7 +0 -34
  45. data/lib/gemstash/man/gemstash-deploy.7 +0 -72
  46. data/lib/gemstash/man/gemstash-mirror.7 +0 -40
  47. data/lib/gemstash/man/gemstash-multiple-sources.7 +0 -89
  48. data/lib/gemstash/man/gemstash-private-gems.7 +0 -244
  49. data/lib/gemstash/man/gemstash-readme.7 +0 -234
  50. data/lib/gemstash/man/gemstash-setup.1 +0 -43
  51. data/lib/gemstash/man/gemstash-start.1 +0 -26
  52. data/lib/gemstash/man/gemstash-status.1 +0 -20
  53. data/lib/gemstash/man/gemstash-stop.1 +0 -20
  54. data/lib/gemstash/man/gemstash-version.1 +0 -22
@@ -1,280 +0,0 @@
1
- .\" Automatically generated by Pandoc 1.19.2.1
2
- .\"
3
- .TH "gemstash\-customize" "7" "October 28, 2015" "" ""
4
- .hy
5
- .SH CUSTOMIZING THE SERVER
6
- .PP
7
- Although Gemstash is designed for as minimal setup as possible, there
8
- may be times you will want to change some of the default configuration.
9
- By the end of this guide, you will be able to customize some of the
10
- Gemstash behavior, including where files are stored, what database
11
- Gemstash uses, and how Gemstash caches certain requests.
12
- .SS SETUP
13
- .PP
14
- Several customizable options are available via an interactive Gemstash
15
- command.
16
- Run \f[C]gemstash\ setup\f[] and answer the questions it provides (a
17
- blank answer will use the default value):
18
- .RS
19
- $ gemstash setup
20
- .PD 0
21
- .P
22
- .PD
23
- Where should files go?
24
- [~/.gemstash]
25
- .PD 0
26
- .P
27
- .PD
28
- Cache with what?
29
- [MEMORY, memcached] \f[B]memcached\f[]
30
- .PD 0
31
- .P
32
- .PD
33
- What is the comma separated Memcached servers?
34
- [localhost:11211]
35
- .PD 0
36
- .P
37
- .PD
38
- What database adapter?
39
- [SQLITE3, postgres, mysql, mysql2] \f[B]postgres\f[]
40
- .PD 0
41
- .P
42
- .PD
43
- Where is the database?
44
- [postgres:///gemstash]
45
- .PD 0
46
- .P
47
- .PD
48
- Checking that the cache is available
49
- .PD 0
50
- .P
51
- .PD
52
- Checking that the database is available
53
- .PD 0
54
- .P
55
- .PD
56
- The database is not available
57
- .PD 0
58
- .P
59
- .PD
60
- .RE
61
- .PP
62
- Once you\[aq]ve answered the questions, some checks will be made to
63
- ensure the configuration will work.
64
- For example, the database didn\[aq]t exist in the previous example, so
65
- the command failed and the configuration wasn\[aq]t saved.
66
- If the command passes, you may provide the \f[C]\-\-redo\f[] option to
67
- force configuration to be redone:
68
- .RS
69
- $ gemstash setup \-\-redo
70
- .PD 0
71
- .P
72
- .PD
73
- Where should files go?
74
- [~/.gemstash]
75
- .PD 0
76
- .P
77
- .PD
78
- Cache with what?
79
- [MEMORY, memcached] \f[B]memcached\f[]
80
- .PD 0
81
- .P
82
- .PD
83
- What is the comma separated Memcached servers?
84
- [localhost:11211]
85
- .PD 0
86
- .P
87
- .PD
88
- What database adapter?
89
- [SQLITE3, postgres, mysql, mysql2]
90
- .PD 0
91
- .P
92
- .PD
93
- Checking that the cache is available
94
- .PD 0
95
- .P
96
- .PD
97
- Checking that the database is available
98
- .PD 0
99
- .P
100
- .PD
101
- You are all setup!
102
- .PD 0
103
- .P
104
- .PD
105
- .RE
106
- .PP
107
- Once all checks have passed, Gemstash will store your answers in the
108
- configuration file located at \f[C]~/.gemstash/config.yml\f[].
109
- .SS FILES
110
- .PP
111
- Storage in Gemstash defaults to \f[C]~/.gemstash\f[] unless otherwise
112
- specified.
113
- You can change this in your config file via the \f[C]:base_path\f[] key:
114
- .IP
115
- .nf
116
- \f[C]
117
- #\ ~/.gemstash/config.yml
118
- \-\-\-
119
- :base_path:\ "/var/gemstash"
120
- \f[]
121
- .fi
122
- .PP
123
- When customizing the \f[C]base_path\f[], the directory must exist,
124
- otherwise Gemstash will fail to run.
125
- Thus, if you want to use \f[C]/var/gemstash\f[] like in the previous
126
- example, make sure to \f[C]mkdir\ /var/gemstash\f[] and grant access to
127
- the directory for the user you run Gemstash with.
128
- .SS DATABASE
129
- .PP
130
- The \f[C]:db_adapter\f[] configuration key specifies what database you
131
- will be using.
132
- The default \f[C]:db_adapter\f[] is
133
- \f[C]sqlite3\f[] (https://www.sqlite.org/), which will use a database
134
- file located within your \f[C]:base_path\f[].
135
- The database file will always be named \f[C]gemstash.db\f[].
136
- .PP
137
- You may also use \f[C]postgres\f[] (http://www.postgresql.org/),
138
- \f[C]mysql\f[] (http://www.mysql.com/), or
139
- \f[C]mysql2\f[] (http://sequel.jeremyevans.net/rdoc/files/doc/opening_databases_rdoc.html#label-mysql2)
140
- for your \f[C]:db_adapter\f[].
141
- When using any of these options, you need to specify the
142
- \f[C]:db_url\f[] to point to an existing database.
143
- Here is an example configuration to use the \f[C]postgres\f[] adapter:
144
- .IP
145
- .nf
146
- \f[C]
147
- #\ ~/.gemstash/config.yml
148
- \-\-\-
149
- :db_adapter:\ postgres
150
- :db_url:\ postgres:///gemstash
151
- :db_connection_options:\ #\ Sequel.connect\ options
152
- \ \ :connect_timeout:\ 10
153
- \ \ :read_timeout:\ 5
154
- \ \ :timeout:\ 30
155
- \f[]
156
- .fi
157
- .PP
158
- Regardless of the adapter you choose, the database will automatically
159
- migrate to your version of Gemstash whenever the database is needed.
160
- You only need to ensure the database exists and Gemstash will do the
161
- rest, except for \f[C]sqlite3\f[] (for which Gemstash will also create
162
- the database for you).
163
- .SS CACHE
164
- .PP
165
- Certain things (like dependencies) are cached in memory.
166
- This avoids web calls to the gem source, and database calls for private
167
- gems.
168
- The memory cache can optionally be swapped out with a
169
- Memcached (http://memcached.org/) server (or cluster of servers).
170
- To use Memcached, you must update the \f[C]:cache_type\f[] configuration
171
- key to be \f[C]memcached\f[], and provide the servers via the
172
- \f[C]:memcached_servers\f[] configuration key:
173
- .IP
174
- .nf
175
- \f[C]
176
- #\ ~/.gemstash/config.yml
177
- \-\-\-
178
- :cache_type:\ memcached
179
- :memcached_servers:\ memcached1.local:11211,memcached2.local:11211
180
- \f[]
181
- .fi
182
- .PP
183
- Note that the \f[C]:memcached_servers\f[] requires a comma separated
184
- list of servers.
185
- All caching lasts for 30 minutes.
186
- .SS SERVER
187
- .PP
188
- Gemstash uses Puma (http://puma.io/) and Rack (http://rack.github.io/)
189
- as the server.
190
- Alternate server configurations are not currently supported, but you can
191
- take a look at the Puma
192
- configuration (https://github.com/bundler/gemstash/blob/master/lib/gemstash/puma.rb)
193
- and the rackup
194
- file (https://github.com/bundler/gemstash/blob/master/lib/gemstash/config.ru)
195
- for inspiration.
196
- .PP
197
- While the server is not customizable, the way Gemstash binds the port
198
- can be changed.
199
- To change the binding, update the \f[C]:bind\f[] configuration key:
200
- .IP
201
- .nf
202
- \f[C]
203
- #\ ~/.gemstash/config.yml
204
- \-\-\-
205
- :bind:\ tcp://0.0.0.0:4242
206
- \f[]
207
- .fi
208
- .PP
209
- This maps directly to the Puma bind
210
- flag (https://github.com/puma/puma#binding-tcp--sockets), and will
211
- support anything valid for that flag.
212
- .PP
213
- The number of threads Puma uses is also customizable via the
214
- \f[C]:puma_threads\f[] configuration key.
215
- The default is \f[C]16\f[].
216
- .SS PROTECTED FETCH
217
- .PP
218
- Gemstash by default allows unauthenticated access for private gems.
219
- Authenticated access is available via the \f[C]:protected_fetch\f[]
220
- configuration key.
221
- .IP
222
- .nf
223
- \f[C]
224
- #\ ~/.gemstash/config.yml
225
- \-\-\-
226
- :protected_fetch:\ true
227
- \f[]
228
- .fi
229
- .PP
230
- More details on protected_fetch are here (gemstash help private-gems.7).
231
- .SS FETCH TIMEOUT
232
- .PP
233
- The default fetch timeout is 20 seconds.
234
- Use the \f[C]:fetch_timeout\f[] configuration key to change it.
235
- .IP
236
- .nf
237
- \f[C]
238
- \-\-\-
239
- :fetch_timeout:\ 20
240
- \f[]
241
- .fi
242
- .SS CONFIG FILE LOCATION
243
- .PP
244
- By default, configuration for Gemstash will be at
245
- \f[C]~/.gemstash/config.yml\f[].
246
- This can be changed by providing the \f[C]\-\-config\-file\f[] option to
247
- the various Gemstash commands:
248
- .IP
249
- .nf
250
- \f[C]
251
- $\ gemstash\ setup\ \-\-config\-file\ ./gemstash\-config.yml
252
- $\ gemstash\ authorize\ \-\-config\-file\ ./gemstash\-config.yml
253
- $\ gemstash\ start\ \-\-config\-file\ ./gemstash\-config.yml
254
- $\ gemstash\ stop\ \-\-config\-file\ ./gemstash\-config.yml
255
- $\ gemstash\ status\ \-\-config\-file\ ./gemstash\-config.yml
256
- \f[]
257
- .fi
258
- .PP
259
- When providing \f[C]\-\-config\-file\f[] to \f[C]gemstash\ setup\f[],
260
- the provided file will be output to with the provided configuration.
261
- \f[B]This will overwrite\f[] any existing configuration.
262
- If the file doesn\[aq]t exist when providing \f[C]\-\-config\-file\f[]
263
- to \f[C]gemstash\ start\f[], \f[C]gemstash\ stop\f[],
264
- \f[C]gemstash\ status\f[], and \f[C]gemstash\ authorize\f[], the default
265
- configuration will be used.
266
- .SS ERB PARSED CONFIG
267
- .PP
268
- You may also create a \f[C]~/.gemstash/config.yml.erb\f[] file.
269
- If present, this will be used instead of
270
- \f[C]~/.gemstash/config.yml\f[].
271
- For example, with this you can use environment variables in the config:
272
- .IP
273
- .nf
274
- \f[C]
275
- #\ ~/.gemstash/config.yml.erb
276
- \-\-\-
277
- :db_adapter:\ postgres
278
- :db_url:\ <%=\ ENV["DATABASE_URL"]\ %>
279
- \f[]
280
- .fi
@@ -1,34 +0,0 @@
1
- .\" Automatically generated by Pandoc 1.19.2.1
2
- .\"
3
- .TH "gemstash\-debugging" "7" "October 28, 2015" "" ""
4
- .hy
5
- .SH DEBUGGING GEMSTASH
6
- .PP
7
- If you are finding Gemstash isn\[aq]t behaving as you would expect, you
8
- might want to start by looking at the server log.
9
- You can find the log at \f[C]server.log\f[] within your base directory.
10
- By default, this will be at \f[C]~/.gemstash/server.log\f[].
11
- .PP
12
- You might find it easier to view the log directly in your terminal.
13
- If you run Gemstash in non\-daemonized form (gemstash help start.1), the
14
- log will be output directly to standard out:
15
- .IP
16
- .nf
17
- \f[C]
18
- $\ gemstash\ start\ \-\-no\-daemonize
19
- \f[]
20
- .fi
21
- .PP
22
- You can also check the status (gemstash help status.1) of the server:
23
- .IP
24
- .nf
25
- \f[C]
26
- $\ gemstash\ status
27
- \f[]
28
- .fi
29
- .PP
30
- The server status is checked by passing through to
31
- pumactl (https://github.com/puma/puma#pumactl).
32
- .PP
33
- If you find a bug, please don\[aq]t hesitate to open a bug
34
- report (https://github.com/bundler/gemstash#contributing)!
@@ -1,72 +0,0 @@
1
- .\" Automatically generated by Pandoc 1.19.2.1
2
- .\"
3
- .TH "gemstash\-deploy" "7" "October 25, 2015" "" ""
4
- .hy
5
- .SH DEPLOYING GEMSTASH
6
- .PP
7
- Bundler is here for the rescue to keep Gemstash up to date! Create a
8
- \f[C]Gemfile\f[] pointing to Gemstash:
9
- .IP
10
- .nf
11
- \f[C]
12
- #\ ./Gemfile
13
- source\ "https://rubygems.org"
14
- gem\ "gemstash"
15
- \f[]
16
- .fi
17
- .PP
18
- Then \f[C]bundle\f[] to create your \f[C]Gemfile.lock\f[].
19
- When you are ready to upgrade, simply \f[C]bundle\ update\f[].
20
- You may need to run \f[C]gemstash\f[] via \f[C]bundle\ exec\f[].
21
- Alternatively, you can \f[C]gem\ uninstall\ gemstash\f[] and
22
- \f[C]gem\ install\ gemstash\f[] when you want to upgrade.
23
- .PP
24
- Gemstash will automatically run any necessary migrations, so updating
25
- the gem is all that needs to be done.
26
- .PP
27
- It is probably wise to stop Gemstash before upgrading, then starting
28
- again once you are done:
29
- .IP
30
- .nf
31
- \f[C]
32
- $\ bundle\ exec\ gemstash\ stop
33
- $\ bundle\ update
34
- $\ bundle\ exec\ gemstash\ start
35
- \f[]
36
- .fi
37
- .SS MONITORING
38
- .PP
39
- Health monitoring is built in to Gemstash using the
40
- server_health_check\-rack (https://github.com/on-site/server_health_check-rack)
41
- gem.
42
- If you request \f[C]/health\f[] from your Gemstash instance, you will
43
- get a JSON response along with an HTTP status code indicating success or
44
- failure.
45
- The JSON response will look something like this for a success case:
46
- .IP
47
- .nf
48
- \f[C]
49
- {
50
- \ \ "status":\ {
51
- \ \ \ \ "heartbeat":\ "OK",
52
- \ \ \ \ "storage_read":\ "OK",
53
- \ \ \ \ "storage_write":\ "OK",
54
- \ \ \ \ "db_read":\ "OK",
55
- \ \ \ \ "db_write":\ "OK"
56
- \ \ }
57
- }
58
- \f[]
59
- .fi
60
- .PP
61
- This request will test storage and database access and report on the
62
- result.
63
- Each key in the status can be requested alone to just report on that
64
- status.
65
- For example, if you would like a health check that doesn\[aq]t interact
66
- with storage or the database, you can use \f[C]/health/heartbeat\f[]
67
- which will always respond with a success while your Gemstash server is
68
- running.
69
- .SS DOWNGRADING
70
- .PP
71
- It is not recommended to go backwards in Gemstash versions.
72
- Migrations may have run that could leave the database in a bad state.
@@ -1,40 +0,0 @@
1
- .\" Automatically generated by Pandoc 1.19.2.1
2
- .\"
3
- .TH "gemstash\-mirror" "7" "October 25, 2015" "" ""
4
- .hy
5
- .SH USING GEMSTASH AS A MIRROR
6
- .PP
7
- If you don\[aq]t have control over your \f[C]Gemfile\f[], or you
8
- don\[aq]t want to force everyone on your team to go through the Gemstash
9
- server, you can use Bundler mirroring to bundle against your Gemstash
10
- server.
11
- .PP
12
- For each source in your \f[C]Gemfile\f[], add a mirror pointing to your
13
- Gemstash server:
14
- .IP
15
- .nf
16
- \f[C]
17
- $\ bundle\ config\ mirror.http://rubygems.org\ http://localhost:9292
18
- $\ bundle\ config\ mirror.https://my.gem\-source.local\ http://localhost:9292/upstream/$(ruby\ \-rcgi\ \-e\ \[aq]puts\ CGI.escape("https://my.gem\-source.local")\[aq])
19
- \f[]
20
- .fi
21
- .PP
22
- From now on, bundler will fetch gems from those sources via your
23
- Gemstash server.
24
- .SH SIMPLER GEMSTASH MIRRORS
25
- .PP
26
- \f[B]This feature requires Bundler to be at least version
27
- \f[C]1.11.0\f[].\f[]
28
- .PP
29
- If you are using Bundler version \f[C]1.11.0\f[] or greater, the
30
- mirroring becomes a bit easier:
31
- .IP
32
- .nf
33
- \f[C]
34
- $\ bundle\ config\ mirror.http://rubygems.org\ http://localhost:9292
35
- $\ bundle\ config\ mirror.https://my.gem\-source.local\ http://localhost:9292
36
- \f[]
37
- .fi
38
- .PP
39
- Bundler will then send headers to Gemstash to indicate the correct
40
- upstream.
@@ -1,89 +0,0 @@
1
- .\" Automatically generated by Pandoc 1.19.2.1
2
- .\"
3
- .TH "gemstash\-multiple\-sources" "7" "October 8, 2015" "" ""
4
- .hy
5
- .SH MULTIPLE GEM SOURCES
6
- .PP
7
- Gemstash will stash from any amount of gem sources.
8
- By the end of this guide, you will be able to bundle using multiple gem
9
- sources, all stashed within your Gemstash server.
10
- .SS DEFAULT SOURCE
11
- .PP
12
- When you don\[aq]t provide an explicit source (as with the Quickstart
13
- Guide (gemstash help readme.7)), your gems will be fetched from
14
- https://rubygems.org.
15
- This default source is not set in stone.
16
- To change it, you need only edit the Gemstash configuration found at
17
- \f[C]~/.gemstash/config.yml\f[]:
18
- .IP
19
- .nf
20
- \f[C]
21
- #\ ~/.gemstash/config.yml
22
- \-\-\-
23
- :rubygems_url:\ https://my.gem\-source.local
24
- \f[]
25
- .fi
26
- .PP
27
- Make sure to restart your Gemstash server after changing the config:
28
- .IP
29
- .nf
30
- \f[C]
31
- $\ gemstash\ stop
32
- $\ gemstash\ start
33
- \f[]
34
- .fi
35
- .PP
36
- Once restarted, bundling against \f[C]http://localhost:9292\f[] will
37
- fetch gems from \f[C]https://my.gem\-source.local\f[].
38
- If you had bundled before making these changes, fear not; bundling with
39
- a different default gem source will store gems in a separate location,
40
- ensuring different sources won\[aq]t leak between each other.
41
- .SS BUNDLING WITH MULTIPLE SOURCES
42
- .PP
43
- Changing the default source won\[aq]t help you if you need to bundle
44
- against https://rubygems.org along with additional sources.
45
- If you need to bundle with multiple gem sources, Gemstash doesn\[aq]t
46
- need to be specially configured.
47
- Your Gemstash server will honor any gem source specified via a
48
- specialized URL.
49
- Consider the following \f[C]Gemfile\f[]:
50
- .IP
51
- .nf
52
- \f[C]
53
- #\ ./Gemfile
54
- require\ "cgi"
55
- source\ "http://localhost:9292"
56
- gem\ "rubywarrior"
57
-
58
- source\ "http://localhost:9292/upstream/#{CGI.escape("https://my.gem\-source.local")}"\ do
59
- \ \ gem\ "my\-gem"
60
- end
61
- \f[]
62
- .fi
63
- .PP
64
- Notice the \f[C]CGI.escape\f[] call in the second source.
65
- This is important, as it properly URL escapes the source URL so Gemstash
66
- knows what gem source you want.
67
- The \f[C]/upstream\f[] prefix tells Gemstash to use a gem source other
68
- than the default source.
69
- You can now bundle with the additional source.
70
- .SS REDIRECTING
71
- .PP
72
- Gemstash supports an alternate mode of specifying your gem sources.
73
- If you want Gemstash to redirect Bundler to your given gem sources, then
74
- you can specify your \f[C]Gemfile\f[] like so:
75
- .IP
76
- .nf
77
- \f[C]
78
- #\ ./Gemfile
79
- require\ "cgi"
80
- source\ "http://localhost:9292/redirect/#{CGI.escape("https://rubygems.org")}"
81
- gem\ "rubywarrior"
82
- \f[]
83
- .fi
84
- .PP
85
- Notice the \f[C]/redirect\f[] prefix.
86
- This prefix tells Gemstash to redirect API calls to the provided URL.
87
- Redirected calls like this will not be cached by Gemstash, and gem files
88
- will not be stashed, even if they were previously cached or stashed from
89
- the same gem source.