gemstash 1.1.0 → 2.0.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (54) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +34 -0
  3. data/lib/gemstash.rb +0 -1
  4. data/lib/gemstash/authorization.rb +1 -5
  5. data/lib/gemstash/cli.rb +4 -4
  6. data/lib/gemstash/cli/authorize.rb +4 -2
  7. data/lib/gemstash/cli/base.rb +5 -4
  8. data/lib/gemstash/cli/setup.rb +2 -2
  9. data/lib/gemstash/cli/status.rb +1 -1
  10. data/lib/gemstash/cli/stop.rb +1 -1
  11. data/lib/gemstash/configuration.rb +3 -2
  12. data/lib/gemstash/gem_pusher.rb +7 -0
  13. data/lib/gemstash/gem_source/private_source.rb +0 -4
  14. data/lib/gemstash/gem_source/upstream_source.rb +0 -4
  15. data/lib/gemstash/health.rb +3 -3
  16. data/lib/gemstash/logging.rb +2 -4
  17. data/lib/gemstash/man/gemstash-authorize.1.txt +53 -33
  18. data/lib/gemstash/man/gemstash-configuration.5.txt +10 -141
  19. data/lib/gemstash/man/gemstash-customize.7.txt +66 -175
  20. data/lib/gemstash/man/gemstash-debugging.7.txt +54 -17
  21. data/lib/gemstash/man/gemstash-deploy.7.txt +51 -44
  22. data/lib/gemstash/man/gemstash-mirror.7.txt +53 -20
  23. data/lib/gemstash/man/gemstash-multiple-sources.7.txt +48 -53
  24. data/lib/gemstash/man/gemstash-private-gems.7.txt +15 -117
  25. data/lib/gemstash/man/gemstash-readme.7.txt +10 -126
  26. data/lib/gemstash/man/gemstash-setup.1.txt +54 -28
  27. data/lib/gemstash/man/gemstash-start.1.txt +56 -16
  28. data/lib/gemstash/man/gemstash-status.1.txt +57 -13
  29. data/lib/gemstash/man/gemstash-stop.1.txt +57 -13
  30. data/lib/gemstash/man/gemstash-version.1.txt +57 -12
  31. data/lib/gemstash/migrations/01_gem_dependencies.rb +2 -2
  32. data/lib/gemstash/migrations/03_cached_gems.rb +1 -1
  33. data/lib/gemstash/migrations/04_health_tests.rb +8 -0
  34. data/lib/gemstash/puma.rb +1 -1
  35. data/lib/gemstash/storage.rb +1 -0
  36. data/lib/gemstash/upstream.rb +2 -2
  37. data/lib/gemstash/version.rb +1 -1
  38. data/lib/gemstash/web.rb +5 -4
  39. metadata +20 -28
  40. data/lib/gemstash/gem_unyanker.rb +0 -67
  41. data/lib/gemstash/man/gemstash-authorize.1 +0 -51
  42. data/lib/gemstash/man/gemstash-configuration.5 +0 -215
  43. data/lib/gemstash/man/gemstash-customize.7 +0 -280
  44. data/lib/gemstash/man/gemstash-debugging.7 +0 -34
  45. data/lib/gemstash/man/gemstash-deploy.7 +0 -72
  46. data/lib/gemstash/man/gemstash-mirror.7 +0 -40
  47. data/lib/gemstash/man/gemstash-multiple-sources.7 +0 -89
  48. data/lib/gemstash/man/gemstash-private-gems.7 +0 -244
  49. data/lib/gemstash/man/gemstash-readme.7 +0 -234
  50. data/lib/gemstash/man/gemstash-setup.1 +0 -43
  51. data/lib/gemstash/man/gemstash-start.1 +0 -26
  52. data/lib/gemstash/man/gemstash-status.1 +0 -20
  53. data/lib/gemstash/man/gemstash-stop.1 +0 -20
  54. data/lib/gemstash/man/gemstash-version.1 +0 -22
@@ -1,280 +0,0 @@
1
- .\" Automatically generated by Pandoc 1.19.2.1
2
- .\"
3
- .TH "gemstash\-customize" "7" "October 28, 2015" "" ""
4
- .hy
5
- .SH CUSTOMIZING THE SERVER
6
- .PP
7
- Although Gemstash is designed for as minimal setup as possible, there
8
- may be times you will want to change some of the default configuration.
9
- By the end of this guide, you will be able to customize some of the
10
- Gemstash behavior, including where files are stored, what database
11
- Gemstash uses, and how Gemstash caches certain requests.
12
- .SS SETUP
13
- .PP
14
- Several customizable options are available via an interactive Gemstash
15
- command.
16
- Run \f[C]gemstash\ setup\f[] and answer the questions it provides (a
17
- blank answer will use the default value):
18
- .RS
19
- $ gemstash setup
20
- .PD 0
21
- .P
22
- .PD
23
- Where should files go?
24
- [~/.gemstash]
25
- .PD 0
26
- .P
27
- .PD
28
- Cache with what?
29
- [MEMORY, memcached] \f[B]memcached\f[]
30
- .PD 0
31
- .P
32
- .PD
33
- What is the comma separated Memcached servers?
34
- [localhost:11211]
35
- .PD 0
36
- .P
37
- .PD
38
- What database adapter?
39
- [SQLITE3, postgres, mysql, mysql2] \f[B]postgres\f[]
40
- .PD 0
41
- .P
42
- .PD
43
- Where is the database?
44
- [postgres:///gemstash]
45
- .PD 0
46
- .P
47
- .PD
48
- Checking that the cache is available
49
- .PD 0
50
- .P
51
- .PD
52
- Checking that the database is available
53
- .PD 0
54
- .P
55
- .PD
56
- The database is not available
57
- .PD 0
58
- .P
59
- .PD
60
- .RE
61
- .PP
62
- Once you\[aq]ve answered the questions, some checks will be made to
63
- ensure the configuration will work.
64
- For example, the database didn\[aq]t exist in the previous example, so
65
- the command failed and the configuration wasn\[aq]t saved.
66
- If the command passes, you may provide the \f[C]\-\-redo\f[] option to
67
- force configuration to be redone:
68
- .RS
69
- $ gemstash setup \-\-redo
70
- .PD 0
71
- .P
72
- .PD
73
- Where should files go?
74
- [~/.gemstash]
75
- .PD 0
76
- .P
77
- .PD
78
- Cache with what?
79
- [MEMORY, memcached] \f[B]memcached\f[]
80
- .PD 0
81
- .P
82
- .PD
83
- What is the comma separated Memcached servers?
84
- [localhost:11211]
85
- .PD 0
86
- .P
87
- .PD
88
- What database adapter?
89
- [SQLITE3, postgres, mysql, mysql2]
90
- .PD 0
91
- .P
92
- .PD
93
- Checking that the cache is available
94
- .PD 0
95
- .P
96
- .PD
97
- Checking that the database is available
98
- .PD 0
99
- .P
100
- .PD
101
- You are all setup!
102
- .PD 0
103
- .P
104
- .PD
105
- .RE
106
- .PP
107
- Once all checks have passed, Gemstash will store your answers in the
108
- configuration file located at \f[C]~/.gemstash/config.yml\f[].
109
- .SS FILES
110
- .PP
111
- Storage in Gemstash defaults to \f[C]~/.gemstash\f[] unless otherwise
112
- specified.
113
- You can change this in your config file via the \f[C]:base_path\f[] key:
114
- .IP
115
- .nf
116
- \f[C]
117
- #\ ~/.gemstash/config.yml
118
- \-\-\-
119
- :base_path:\ "/var/gemstash"
120
- \f[]
121
- .fi
122
- .PP
123
- When customizing the \f[C]base_path\f[], the directory must exist,
124
- otherwise Gemstash will fail to run.
125
- Thus, if you want to use \f[C]/var/gemstash\f[] like in the previous
126
- example, make sure to \f[C]mkdir\ /var/gemstash\f[] and grant access to
127
- the directory for the user you run Gemstash with.
128
- .SS DATABASE
129
- .PP
130
- The \f[C]:db_adapter\f[] configuration key specifies what database you
131
- will be using.
132
- The default \f[C]:db_adapter\f[] is
133
- \f[C]sqlite3\f[] (https://www.sqlite.org/), which will use a database
134
- file located within your \f[C]:base_path\f[].
135
- The database file will always be named \f[C]gemstash.db\f[].
136
- .PP
137
- You may also use \f[C]postgres\f[] (http://www.postgresql.org/),
138
- \f[C]mysql\f[] (http://www.mysql.com/), or
139
- \f[C]mysql2\f[] (http://sequel.jeremyevans.net/rdoc/files/doc/opening_databases_rdoc.html#label-mysql2)
140
- for your \f[C]:db_adapter\f[].
141
- When using any of these options, you need to specify the
142
- \f[C]:db_url\f[] to point to an existing database.
143
- Here is an example configuration to use the \f[C]postgres\f[] adapter:
144
- .IP
145
- .nf
146
- \f[C]
147
- #\ ~/.gemstash/config.yml
148
- \-\-\-
149
- :db_adapter:\ postgres
150
- :db_url:\ postgres:///gemstash
151
- :db_connection_options:\ #\ Sequel.connect\ options
152
- \ \ :connect_timeout:\ 10
153
- \ \ :read_timeout:\ 5
154
- \ \ :timeout:\ 30
155
- \f[]
156
- .fi
157
- .PP
158
- Regardless of the adapter you choose, the database will automatically
159
- migrate to your version of Gemstash whenever the database is needed.
160
- You only need to ensure the database exists and Gemstash will do the
161
- rest, except for \f[C]sqlite3\f[] (for which Gemstash will also create
162
- the database for you).
163
- .SS CACHE
164
- .PP
165
- Certain things (like dependencies) are cached in memory.
166
- This avoids web calls to the gem source, and database calls for private
167
- gems.
168
- The memory cache can optionally be swapped out with a
169
- Memcached (http://memcached.org/) server (or cluster of servers).
170
- To use Memcached, you must update the \f[C]:cache_type\f[] configuration
171
- key to be \f[C]memcached\f[], and provide the servers via the
172
- \f[C]:memcached_servers\f[] configuration key:
173
- .IP
174
- .nf
175
- \f[C]
176
- #\ ~/.gemstash/config.yml
177
- \-\-\-
178
- :cache_type:\ memcached
179
- :memcached_servers:\ memcached1.local:11211,memcached2.local:11211
180
- \f[]
181
- .fi
182
- .PP
183
- Note that the \f[C]:memcached_servers\f[] requires a comma separated
184
- list of servers.
185
- All caching lasts for 30 minutes.
186
- .SS SERVER
187
- .PP
188
- Gemstash uses Puma (http://puma.io/) and Rack (http://rack.github.io/)
189
- as the server.
190
- Alternate server configurations are not currently supported, but you can
191
- take a look at the Puma
192
- configuration (https://github.com/bundler/gemstash/blob/master/lib/gemstash/puma.rb)
193
- and the rackup
194
- file (https://github.com/bundler/gemstash/blob/master/lib/gemstash/config.ru)
195
- for inspiration.
196
- .PP
197
- While the server is not customizable, the way Gemstash binds the port
198
- can be changed.
199
- To change the binding, update the \f[C]:bind\f[] configuration key:
200
- .IP
201
- .nf
202
- \f[C]
203
- #\ ~/.gemstash/config.yml
204
- \-\-\-
205
- :bind:\ tcp://0.0.0.0:4242
206
- \f[]
207
- .fi
208
- .PP
209
- This maps directly to the Puma bind
210
- flag (https://github.com/puma/puma#binding-tcp--sockets), and will
211
- support anything valid for that flag.
212
- .PP
213
- The number of threads Puma uses is also customizable via the
214
- \f[C]:puma_threads\f[] configuration key.
215
- The default is \f[C]16\f[].
216
- .SS PROTECTED FETCH
217
- .PP
218
- Gemstash by default allows unauthenticated access for private gems.
219
- Authenticated access is available via the \f[C]:protected_fetch\f[]
220
- configuration key.
221
- .IP
222
- .nf
223
- \f[C]
224
- #\ ~/.gemstash/config.yml
225
- \-\-\-
226
- :protected_fetch:\ true
227
- \f[]
228
- .fi
229
- .PP
230
- More details on protected_fetch are here (gemstash help private-gems.7).
231
- .SS FETCH TIMEOUT
232
- .PP
233
- The default fetch timeout is 20 seconds.
234
- Use the \f[C]:fetch_timeout\f[] configuration key to change it.
235
- .IP
236
- .nf
237
- \f[C]
238
- \-\-\-
239
- :fetch_timeout:\ 20
240
- \f[]
241
- .fi
242
- .SS CONFIG FILE LOCATION
243
- .PP
244
- By default, configuration for Gemstash will be at
245
- \f[C]~/.gemstash/config.yml\f[].
246
- This can be changed by providing the \f[C]\-\-config\-file\f[] option to
247
- the various Gemstash commands:
248
- .IP
249
- .nf
250
- \f[C]
251
- $\ gemstash\ setup\ \-\-config\-file\ ./gemstash\-config.yml
252
- $\ gemstash\ authorize\ \-\-config\-file\ ./gemstash\-config.yml
253
- $\ gemstash\ start\ \-\-config\-file\ ./gemstash\-config.yml
254
- $\ gemstash\ stop\ \-\-config\-file\ ./gemstash\-config.yml
255
- $\ gemstash\ status\ \-\-config\-file\ ./gemstash\-config.yml
256
- \f[]
257
- .fi
258
- .PP
259
- When providing \f[C]\-\-config\-file\f[] to \f[C]gemstash\ setup\f[],
260
- the provided file will be output to with the provided configuration.
261
- \f[B]This will overwrite\f[] any existing configuration.
262
- If the file doesn\[aq]t exist when providing \f[C]\-\-config\-file\f[]
263
- to \f[C]gemstash\ start\f[], \f[C]gemstash\ stop\f[],
264
- \f[C]gemstash\ status\f[], and \f[C]gemstash\ authorize\f[], the default
265
- configuration will be used.
266
- .SS ERB PARSED CONFIG
267
- .PP
268
- You may also create a \f[C]~/.gemstash/config.yml.erb\f[] file.
269
- If present, this will be used instead of
270
- \f[C]~/.gemstash/config.yml\f[].
271
- For example, with this you can use environment variables in the config:
272
- .IP
273
- .nf
274
- \f[C]
275
- #\ ~/.gemstash/config.yml.erb
276
- \-\-\-
277
- :db_adapter:\ postgres
278
- :db_url:\ <%=\ ENV["DATABASE_URL"]\ %>
279
- \f[]
280
- .fi
@@ -1,34 +0,0 @@
1
- .\" Automatically generated by Pandoc 1.19.2.1
2
- .\"
3
- .TH "gemstash\-debugging" "7" "October 28, 2015" "" ""
4
- .hy
5
- .SH DEBUGGING GEMSTASH
6
- .PP
7
- If you are finding Gemstash isn\[aq]t behaving as you would expect, you
8
- might want to start by looking at the server log.
9
- You can find the log at \f[C]server.log\f[] within your base directory.
10
- By default, this will be at \f[C]~/.gemstash/server.log\f[].
11
- .PP
12
- You might find it easier to view the log directly in your terminal.
13
- If you run Gemstash in non\-daemonized form (gemstash help start.1), the
14
- log will be output directly to standard out:
15
- .IP
16
- .nf
17
- \f[C]
18
- $\ gemstash\ start\ \-\-no\-daemonize
19
- \f[]
20
- .fi
21
- .PP
22
- You can also check the status (gemstash help status.1) of the server:
23
- .IP
24
- .nf
25
- \f[C]
26
- $\ gemstash\ status
27
- \f[]
28
- .fi
29
- .PP
30
- The server status is checked by passing through to
31
- pumactl (https://github.com/puma/puma#pumactl).
32
- .PP
33
- If you find a bug, please don\[aq]t hesitate to open a bug
34
- report (https://github.com/bundler/gemstash#contributing)!
@@ -1,72 +0,0 @@
1
- .\" Automatically generated by Pandoc 1.19.2.1
2
- .\"
3
- .TH "gemstash\-deploy" "7" "October 25, 2015" "" ""
4
- .hy
5
- .SH DEPLOYING GEMSTASH
6
- .PP
7
- Bundler is here for the rescue to keep Gemstash up to date! Create a
8
- \f[C]Gemfile\f[] pointing to Gemstash:
9
- .IP
10
- .nf
11
- \f[C]
12
- #\ ./Gemfile
13
- source\ "https://rubygems.org"
14
- gem\ "gemstash"
15
- \f[]
16
- .fi
17
- .PP
18
- Then \f[C]bundle\f[] to create your \f[C]Gemfile.lock\f[].
19
- When you are ready to upgrade, simply \f[C]bundle\ update\f[].
20
- You may need to run \f[C]gemstash\f[] via \f[C]bundle\ exec\f[].
21
- Alternatively, you can \f[C]gem\ uninstall\ gemstash\f[] and
22
- \f[C]gem\ install\ gemstash\f[] when you want to upgrade.
23
- .PP
24
- Gemstash will automatically run any necessary migrations, so updating
25
- the gem is all that needs to be done.
26
- .PP
27
- It is probably wise to stop Gemstash before upgrading, then starting
28
- again once you are done:
29
- .IP
30
- .nf
31
- \f[C]
32
- $\ bundle\ exec\ gemstash\ stop
33
- $\ bundle\ update
34
- $\ bundle\ exec\ gemstash\ start
35
- \f[]
36
- .fi
37
- .SS MONITORING
38
- .PP
39
- Health monitoring is built in to Gemstash using the
40
- server_health_check\-rack (https://github.com/on-site/server_health_check-rack)
41
- gem.
42
- If you request \f[C]/health\f[] from your Gemstash instance, you will
43
- get a JSON response along with an HTTP status code indicating success or
44
- failure.
45
- The JSON response will look something like this for a success case:
46
- .IP
47
- .nf
48
- \f[C]
49
- {
50
- \ \ "status":\ {
51
- \ \ \ \ "heartbeat":\ "OK",
52
- \ \ \ \ "storage_read":\ "OK",
53
- \ \ \ \ "storage_write":\ "OK",
54
- \ \ \ \ "db_read":\ "OK",
55
- \ \ \ \ "db_write":\ "OK"
56
- \ \ }
57
- }
58
- \f[]
59
- .fi
60
- .PP
61
- This request will test storage and database access and report on the
62
- result.
63
- Each key in the status can be requested alone to just report on that
64
- status.
65
- For example, if you would like a health check that doesn\[aq]t interact
66
- with storage or the database, you can use \f[C]/health/heartbeat\f[]
67
- which will always respond with a success while your Gemstash server is
68
- running.
69
- .SS DOWNGRADING
70
- .PP
71
- It is not recommended to go backwards in Gemstash versions.
72
- Migrations may have run that could leave the database in a bad state.
@@ -1,40 +0,0 @@
1
- .\" Automatically generated by Pandoc 1.19.2.1
2
- .\"
3
- .TH "gemstash\-mirror" "7" "October 25, 2015" "" ""
4
- .hy
5
- .SH USING GEMSTASH AS A MIRROR
6
- .PP
7
- If you don\[aq]t have control over your \f[C]Gemfile\f[], or you
8
- don\[aq]t want to force everyone on your team to go through the Gemstash
9
- server, you can use Bundler mirroring to bundle against your Gemstash
10
- server.
11
- .PP
12
- For each source in your \f[C]Gemfile\f[], add a mirror pointing to your
13
- Gemstash server:
14
- .IP
15
- .nf
16
- \f[C]
17
- $\ bundle\ config\ mirror.http://rubygems.org\ http://localhost:9292
18
- $\ bundle\ config\ mirror.https://my.gem\-source.local\ http://localhost:9292/upstream/$(ruby\ \-rcgi\ \-e\ \[aq]puts\ CGI.escape("https://my.gem\-source.local")\[aq])
19
- \f[]
20
- .fi
21
- .PP
22
- From now on, bundler will fetch gems from those sources via your
23
- Gemstash server.
24
- .SH SIMPLER GEMSTASH MIRRORS
25
- .PP
26
- \f[B]This feature requires Bundler to be at least version
27
- \f[C]1.11.0\f[].\f[]
28
- .PP
29
- If you are using Bundler version \f[C]1.11.0\f[] or greater, the
30
- mirroring becomes a bit easier:
31
- .IP
32
- .nf
33
- \f[C]
34
- $\ bundle\ config\ mirror.http://rubygems.org\ http://localhost:9292
35
- $\ bundle\ config\ mirror.https://my.gem\-source.local\ http://localhost:9292
36
- \f[]
37
- .fi
38
- .PP
39
- Bundler will then send headers to Gemstash to indicate the correct
40
- upstream.
@@ -1,89 +0,0 @@
1
- .\" Automatically generated by Pandoc 1.19.2.1
2
- .\"
3
- .TH "gemstash\-multiple\-sources" "7" "October 8, 2015" "" ""
4
- .hy
5
- .SH MULTIPLE GEM SOURCES
6
- .PP
7
- Gemstash will stash from any amount of gem sources.
8
- By the end of this guide, you will be able to bundle using multiple gem
9
- sources, all stashed within your Gemstash server.
10
- .SS DEFAULT SOURCE
11
- .PP
12
- When you don\[aq]t provide an explicit source (as with the Quickstart
13
- Guide (gemstash help readme.7)), your gems will be fetched from
14
- https://rubygems.org.
15
- This default source is not set in stone.
16
- To change it, you need only edit the Gemstash configuration found at
17
- \f[C]~/.gemstash/config.yml\f[]:
18
- .IP
19
- .nf
20
- \f[C]
21
- #\ ~/.gemstash/config.yml
22
- \-\-\-
23
- :rubygems_url:\ https://my.gem\-source.local
24
- \f[]
25
- .fi
26
- .PP
27
- Make sure to restart your Gemstash server after changing the config:
28
- .IP
29
- .nf
30
- \f[C]
31
- $\ gemstash\ stop
32
- $\ gemstash\ start
33
- \f[]
34
- .fi
35
- .PP
36
- Once restarted, bundling against \f[C]http://localhost:9292\f[] will
37
- fetch gems from \f[C]https://my.gem\-source.local\f[].
38
- If you had bundled before making these changes, fear not; bundling with
39
- a different default gem source will store gems in a separate location,
40
- ensuring different sources won\[aq]t leak between each other.
41
- .SS BUNDLING WITH MULTIPLE SOURCES
42
- .PP
43
- Changing the default source won\[aq]t help you if you need to bundle
44
- against https://rubygems.org along with additional sources.
45
- If you need to bundle with multiple gem sources, Gemstash doesn\[aq]t
46
- need to be specially configured.
47
- Your Gemstash server will honor any gem source specified via a
48
- specialized URL.
49
- Consider the following \f[C]Gemfile\f[]:
50
- .IP
51
- .nf
52
- \f[C]
53
- #\ ./Gemfile
54
- require\ "cgi"
55
- source\ "http://localhost:9292"
56
- gem\ "rubywarrior"
57
-
58
- source\ "http://localhost:9292/upstream/#{CGI.escape("https://my.gem\-source.local")}"\ do
59
- \ \ gem\ "my\-gem"
60
- end
61
- \f[]
62
- .fi
63
- .PP
64
- Notice the \f[C]CGI.escape\f[] call in the second source.
65
- This is important, as it properly URL escapes the source URL so Gemstash
66
- knows what gem source you want.
67
- The \f[C]/upstream\f[] prefix tells Gemstash to use a gem source other
68
- than the default source.
69
- You can now bundle with the additional source.
70
- .SS REDIRECTING
71
- .PP
72
- Gemstash supports an alternate mode of specifying your gem sources.
73
- If you want Gemstash to redirect Bundler to your given gem sources, then
74
- you can specify your \f[C]Gemfile\f[] like so:
75
- .IP
76
- .nf
77
- \f[C]
78
- #\ ./Gemfile
79
- require\ "cgi"
80
- source\ "http://localhost:9292/redirect/#{CGI.escape("https://rubygems.org")}"
81
- gem\ "rubywarrior"
82
- \f[]
83
- .fi
84
- .PP
85
- Notice the \f[C]/redirect\f[] prefix.
86
- This prefix tells Gemstash to redirect API calls to the provided URL.
87
- Redirected calls like this will not be cached by Gemstash, and gem files
88
- will not be stashed, even if they were previously cached or stashed from
89
- the same gem source.