gemstash 2.8.1 → 2.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/lib/gemstash/authorization.rb +0 -2
- data/lib/gemstash/cache.rb +5 -4
- data/lib/gemstash/cli/base.rb +9 -2
- data/lib/gemstash/cli/setup.rb +1 -1
- data/lib/gemstash/cli/start.rb +4 -0
- data/lib/gemstash/configuration.rb +71 -1
- data/lib/gemstash/db/upstream.rb +30 -2
- data/lib/gemstash/dependencies.rb +28 -5
- data/lib/gemstash/env.rb +4 -2
- data/lib/gemstash/gem_fetcher.rb +1 -2
- data/lib/gemstash/gem_pusher.rb +24 -20
- data/lib/gemstash/gem_source/private_source.rb +22 -6
- data/lib/gemstash/gem_source/upstream_source.rb +129 -34
- data/lib/gemstash/http_client.rb +167 -8
- data/lib/gemstash/logging.rb +3 -1
- data/lib/gemstash/man/gemstash-authorize.1 +29 -48
- data/lib/gemstash/man/gemstash-authorize.1.txt +14 -14
- data/lib/gemstash/man/gemstash-configuration.5 +231 -154
- data/lib/gemstash/man/gemstash-configuration.5.txt +208 -93
- data/lib/gemstash/man/gemstash-customize.7 +150 -168
- data/lib/gemstash/man/gemstash-customize.7.txt +80 -86
- data/lib/gemstash/man/gemstash-debugging.7 +20 -33
- data/lib/gemstash/man/gemstash-debugging.7.txt +5 -7
- data/lib/gemstash/man/gemstash-deploy.7 +31 -53
- data/lib/gemstash/man/gemstash-deploy.7.txt +17 -18
- data/lib/gemstash/man/gemstash-mirror.7 +24 -37
- data/lib/gemstash/man/gemstash-mirror.7.txt +14 -9
- data/lib/gemstash/man/gemstash-multiple-sources.7 +105 -110
- data/lib/gemstash/man/gemstash-multiple-sources.7.txt +62 -41
- data/lib/gemstash/man/gemstash-private-gems.7 +87 -136
- data/lib/gemstash/man/gemstash-private-gems.7.txt +30 -30
- data/lib/gemstash/man/gemstash-readme.7 +105 -127
- data/lib/gemstash/man/gemstash-readme.7.txt +43 -49
- data/lib/gemstash/man/gemstash-setup.1 +22 -42
- data/lib/gemstash/man/gemstash-setup.1.txt +12 -12
- data/lib/gemstash/man/gemstash-start.1 +9 -27
- data/lib/gemstash/man/gemstash-start.1.txt +5 -5
- data/lib/gemstash/man/gemstash-status.1 +9 -27
- data/lib/gemstash/man/gemstash-status.1.txt +5 -5
- data/lib/gemstash/man/gemstash-stop.1 +9 -27
- data/lib/gemstash/man/gemstash-stop.1.txt +5 -5
- data/lib/gemstash/man/gemstash-version.1 +8 -28
- data/lib/gemstash/man/gemstash-version.1.txt +2 -2
- data/lib/gemstash/puma.rb +1 -0
- data/lib/gemstash/rack_env_rewriter.rb +3 -0
- data/lib/gemstash/upstream.rb +83 -6
- data/lib/gemstash/version.rb +1 -1
- data/lib/gemstash/web.rb +12 -0
- metadata +7 -7
|
@@ -1,221 +1,336 @@
|
|
|
1
|
-
4mgemstash-configuration24m(5)
|
|
1
|
+
4mgemstash-configuration24m(5) File Formats Manual 4mgemstash-configuration24m(5)
|
|
2
2
|
|
|
3
3
|
1mName0m
|
|
4
4
|
gemstash-configuration
|
|
5
5
|
|
|
6
6
|
1mSynopsis0m
|
|
7
|
-
# ~/.gemstash/config.
|
|
7
|
+
4m#24m 4m~/.gemstash/config.yml0m
|
|
8
8
|
---
|
|
9
|
-
:
|
|
10
|
-
:
|
|
11
|
-
:
|
|
12
|
-
:
|
|
13
|
-
:
|
|
14
|
-
:
|
|
15
|
-
:
|
|
16
|
-
:
|
|
17
|
-
:
|
|
18
|
-
:
|
|
19
|
-
:
|
|
20
|
-
|
|
21
|
-
:
|
|
22
|
-
|
|
23
|
-
:
|
|
24
|
-
:
|
|
9
|
+
:base_path1m: 22m"/var/gemstash"
|
|
10
|
+
:cache_type1m: 22mmemcached
|
|
11
|
+
:memcached_servers1m: 22mlocalhost:11211
|
|
12
|
+
:db_adapter1m: 22mpostgres
|
|
13
|
+
:db_url1m: 22mpostgres:///gemstash
|
|
14
|
+
:db_connection_options1m:0m
|
|
15
|
+
:test1m: 22mtrue
|
|
16
|
+
:pool_timeout1m: 22m2
|
|
17
|
+
:rubygems_url1m: 22mhttps://my.gem-source.local
|
|
18
|
+
:ignore_gemfile_source1m: 22mfalse
|
|
19
|
+
:allowed_upstreams1m:0m
|
|
20
|
+
1m- 22mhttps://my-other.gem-source.local
|
|
21
|
+
:allowed_redirects1m:0m
|
|
22
|
+
1m- 22mhttps://cdn.my-other.gem-source.local
|
|
23
|
+
:puma_threads1m: 22m32
|
|
24
|
+
:bind1m: 22mtcp://0.0.0.0:4242
|
|
25
|
+
:protected_fetch1m: 22mtrue
|
|
26
|
+
:fetch_timeout1m: 22m10
|
|
27
|
+
:max_upstream_duration1m: 22m300
|
|
28
|
+
:open_timeout1m: 22m2
|
|
29
|
+
:max_upstream_bytes1m: 22m67108864
|
|
30
|
+
:log_file1m: 22mgemstash.log
|
|
25
31
|
|
|
26
32
|
1mBase Path0m
|
|
27
|
-
|
|
33
|
+
:base_path
|
|
28
34
|
|
|
29
35
|
Specifies where to store local files like the server log, cached gem
|
|
30
36
|
files, and the database (when using SQLite). If the default is being
|
|
31
37
|
used, the directory will be created if it does not exist. Any other
|
|
32
38
|
directory needs to be created ahead of time and be writable to the Gem-
|
|
33
|
-
stash server process. Specifying the
|
|
39
|
+
stash server process. Specifying the :base_path via gemstash setup
|
|
34
40
|
will create the directory for you.
|
|
35
41
|
|
|
36
42
|
1mDefault value0m
|
|
37
|
-
|
|
43
|
+
~/.gemstash
|
|
38
44
|
|
|
39
45
|
1mValid values0m
|
|
40
46
|
Any valid path
|
|
41
47
|
|
|
42
48
|
1mCache Type0m
|
|
43
|
-
|
|
49
|
+
:cache_type
|
|
44
50
|
|
|
45
51
|
Specifies how to cache values other than gem files (such as gem depen-
|
|
46
|
-
dencies).
|
|
47
|
-
point to 1 or more Memcached servers. Use the
|
|
52
|
+
dencies). memory will use an in memory cache while memcached will
|
|
53
|
+
point to 1 or more Memcached servers. Use the :memcached_servers con-
|
|
48
54
|
figuration key for specifying where the Memcached server(s) are.
|
|
49
55
|
|
|
50
56
|
1mDefault value0m
|
|
51
|
-
|
|
57
|
+
memory
|
|
52
58
|
|
|
53
59
|
1mValid values0m
|
|
54
|
-
|
|
60
|
+
memory, memcached
|
|
55
61
|
|
|
56
62
|
1mMemcached Servers0m
|
|
57
|
-
|
|
63
|
+
:memcached_servers
|
|
58
64
|
|
|
59
|
-
Specifies the Memcached servers to connect to when using
|
|
60
|
-
the
|
|
65
|
+
Specifies the Memcached servers to connect to when using memcached for
|
|
66
|
+
the :cache_type. Only used when memcached is used for :cache_type.
|
|
61
67
|
|
|
62
68
|
1mDefault value0m
|
|
63
|
-
|
|
69
|
+
localhost:11211
|
|
64
70
|
|
|
65
71
|
1mValid values0m
|
|
66
72
|
A comma delimited list of Memcached servers
|
|
67
73
|
|
|
68
74
|
1mDB Adapter0m
|
|
69
|
-
|
|
75
|
+
:db_adapter
|
|
70
76
|
|
|
71
|
-
Specifies what database adapter to use. When
|
|
72
|
-
database will be located at
|
|
73
|
-
by
|
|
74
|
-
|
|
75
|
-
nect to must be specified in the
|
|
76
|
-
base must already be created when using anything other than
|
|
77
|
+
Specifies what database adapter to use. When sqlite3 is used, the
|
|
78
|
+
database will be located at gemstash.db within the directory specified
|
|
79
|
+
by :base_path. The database will automatically be created when using
|
|
80
|
+
sqlite3. When postgres, mysql, or mysql2 is used, the database to con-
|
|
81
|
+
nect to must be specified in the :db_url configuration key. The data-
|
|
82
|
+
base must already be created when using anything other than sqlite3.
|
|
77
83
|
|
|
78
84
|
1mDefault value0m
|
|
79
|
-
|
|
85
|
+
sqlite3
|
|
80
86
|
|
|
81
87
|
1mValid values0m
|
|
82
|
-
|
|
88
|
+
sqlite3, postgres, mysql, mysql2
|
|
83
89
|
|
|
84
90
|
1mDB URL0m
|
|
85
|
-
|
|
91
|
+
:db_url
|
|
86
92
|
|
|
87
|
-
Specifies the database to connect to when using
|
|
88
|
-
|
|
89
|
-
|
|
93
|
+
Specifies the database to connect to when using postgres, mysql, or
|
|
94
|
+
mysql2 for the :db_adapter. Only used when the :db_adapter is not
|
|
95
|
+
sqlite3.
|
|
90
96
|
|
|
91
97
|
1mDefault value0m
|
|
92
98
|
None
|
|
93
99
|
|
|
94
100
|
1mValid values0m
|
|
95
|
-
A
|
|
96
|
-
vans.net/)
|
|
101
|
+
A valid database URL for the 8;;http://sequel.jeremyevans.net/Sequel gem8;;
|
|
97
102
|
|
|
98
103
|
1mDB Connection Options0m
|
|
99
|
-
|
|
104
|
+
:db_connection_options
|
|
100
105
|
|
|
101
|
-
Specifies
|
|
102
|
-
tions
|
|
103
|
-
specify the
|
|
106
|
+
Specifies additional Sequel.connect options to use. Note that any op-
|
|
107
|
+
tions here are merged in with the default options, so you need not
|
|
108
|
+
specify the max_connections if you customize this option.
|
|
104
109
|
|
|
105
110
|
1mDefault value0m
|
|
106
|
-
|
|
107
|
-
|
|
111
|
+
{ max_connections: 1 } for sqlite3 adapter, { max_connections: con-
|
|
112
|
+
fig[:puma_threads] + 1 } for any other adapter.
|
|
108
113
|
|
|
109
114
|
1mValid values0m
|
|
110
|
-
A
|
|
111
|
-
quel.jeremyevans.net/rdoc/files/doc/opening_databases_rdoc.html#label-
|
|
112
|
-
General+connection+options) method.
|
|
115
|
+
A valid connection options Hash for the 8;;http://sequel.jeremyevans.net/rdoc/files/doc/opening_databases_rdoc.html#label-General+connection+optionsSequel.connect8;; method.
|
|
113
116
|
|
|
114
117
|
1mRubygems URL0m
|
|
115
|
-
|
|
118
|
+
:rubygems_url
|
|
116
119
|
|
|
117
120
|
Specifies the default gem source URL. When any API endpoint is called
|
|
118
|
-
without a
|
|
121
|
+
without a /private or /upstream/<url> prefix, this URL will be used to
|
|
119
122
|
fetch the result. This value can be safely changed even if there are
|
|
120
123
|
already gems stashed for the previous value.
|
|
121
124
|
|
|
122
125
|
1mDefault value0m
|
|
123
|
-
|
|
126
|
+
https://rubygems.org
|
|
124
127
|
|
|
125
128
|
1mValid values0m
|
|
126
129
|
A valid gem source URL
|
|
127
130
|
|
|
128
131
|
1mIgnore Gemfile source0m
|
|
129
|
-
|
|
132
|
+
:ignore_gemfile_source
|
|
130
133
|
|
|
131
|
-
Ignore the source specified in Gemfile and always use
|
|
134
|
+
Ignore the source specified in Gemfile and always use :rubygems_url as
|
|
132
135
|
gems upstream.
|
|
133
136
|
|
|
134
137
|
1mDefault value0m
|
|
135
|
-
|
|
138
|
+
false
|
|
136
139
|
|
|
137
140
|
1mValid values0m
|
|
138
|
-
Boolean:
|
|
141
|
+
Boolean: true or false
|
|
142
|
+
|
|
143
|
+
1mAllowed Upstreams0m
|
|
144
|
+
:allowed_upstreams
|
|
145
|
+
|
|
146
|
+
Specifies the gem sources that clients may ask Gemstash to fetch from,
|
|
147
|
+
whether they use an /upstream/<url> or /redirect/<url> prefix, or the
|
|
148
|
+
header Bundler sends when Gemstash is used as a mirror. The gem source
|
|
149
|
+
specified by :rubygems_url is always allowed. A request for any other
|
|
150
|
+
gem source is answered with a 403 status, and that gem source is not
|
|
151
|
+
contacted.
|
|
152
|
+
|
|
153
|
+
Bundler reports a 403 as Access token could not be authenticated, so
|
|
154
|
+
that is the error someone sees when they bundle with a gem source that
|
|
155
|
+
is not allowed. The actual reason is in the body of the response and
|
|
156
|
+
in the Gemstash log, as a Refusing upstream warning.
|
|
157
|
+
|
|
158
|
+
A gem source is allowed when its scheme, host, port, and path are the
|
|
159
|
+
same as those of an entry. Credentials and a trailing slash are ig-
|
|
160
|
+
nored, so an entry of https://my.gem-source.local allows
|
|
161
|
+
https://user:password@my.gem-source.local/, but does not allow
|
|
162
|
+
http://my.gem-source.local or https://my.gem-source.local/private.
|
|
163
|
+
|
|
164
|
+
Credentials in an entry are ignored only for that comparison. When a
|
|
165
|
+
client asks for the gem source without credentials of its own, Gemstash
|
|
166
|
+
fetches from it with the credentials of the entry, so anyone who can
|
|
167
|
+
reach Gemstash can read that gem source. Gemstash does not pass them
|
|
168
|
+
on to clients: the requests it answers with a redirect, such as /ver-
|
|
169
|
+
sions and /info/*, reach the gem source without credentials. This also
|
|
170
|
+
applies to credentials in :rubygems_url.
|
|
171
|
+
|
|
172
|
+
By default only :rubygems_url is allowed. Before upgrading, list any
|
|
173
|
+
other Gemfile and mirror sources you use, or Bundler requests for them
|
|
174
|
+
will fail with 403. Set "*" only if you deliberately accept arbitrary
|
|
175
|
+
client-selected sources. Redirects from an allowed source to an un-
|
|
176
|
+
listed source are refused. If an allowed source redirects gem down-
|
|
177
|
+
loads to a CDN, list the CDN in :allowed_redirects. Redirect targets
|
|
178
|
+
containing credentials are refused.
|
|
179
|
+
|
|
180
|
+
1mDefault value0m
|
|
181
|
+
[] (only :rubygems_url)
|
|
182
|
+
|
|
183
|
+
1mValid values0m
|
|
184
|
+
"*" to allow any gem source, or a list of gem source URLs. An empty
|
|
185
|
+
list allows only the gem source specified by :rubygems_url.
|
|
186
|
+
|
|
187
|
+
Each URL needs a host, and its scheme, if it has one, must be http://
|
|
188
|
+
or https://. gemstash start refuses a configuration with any other en-
|
|
189
|
+
try. The error gives the position of the entry in the list and not its
|
|
190
|
+
value, because the value may contain credentials.
|
|
191
|
+
|
|
192
|
+
1mAllowed Redirects0m
|
|
193
|
+
:allowed_redirects
|
|
194
|
+
|
|
195
|
+
Specifies further URLs that a fetch from an allowed gem source may be
|
|
196
|
+
redirected to, such as the CDN a gem server sends its downloads to.
|
|
197
|
+
Clients cannot ask Gemstash to use these as gem sources. :rubygems_url
|
|
198
|
+
and the entries of :allowed_upstreams are always valid redirect tar-
|
|
199
|
+
gets.
|
|
200
|
+
|
|
201
|
+
A redirect is followed when its scheme, host, and port are those of an
|
|
202
|
+
entry and its path is the path of the entry or lies below it. An http
|
|
203
|
+
entry also accepts the same host over https, and a redirect from https
|
|
204
|
+
to http is never followed. Paths are compared segment by segment, and
|
|
205
|
+
a redirect below an entry with a path is refused when a segment could
|
|
206
|
+
be resolved to another path, such as ..; or a doubly encoded ... Do
|
|
207
|
+
not rely on the path of an entry alone to keep gem sources on one host
|
|
208
|
+
apart: the access control of that host is what separates them.
|
|
209
|
+
|
|
210
|
+
1mDefault value0m
|
|
211
|
+
[]
|
|
212
|
+
|
|
213
|
+
1mValid values0m
|
|
214
|
+
A list of URLs, checked like the entries of :allowed_upstreams. "*" is
|
|
215
|
+
not supported.
|
|
139
216
|
|
|
140
217
|
1mPuma Threads0m
|
|
141
|
-
|
|
218
|
+
:puma_threads
|
|
142
219
|
|
|
143
220
|
Specifies the number of threads used for the Gemstash server.
|
|
144
221
|
|
|
145
222
|
1mDefault value0m
|
|
146
|
-
|
|
223
|
+
16
|
|
147
224
|
|
|
148
225
|
1mValid values0m
|
|
149
|
-
Integer value with a minimum of
|
|
226
|
+
Integer value with a minimum of 1
|
|
150
227
|
|
|
151
228
|
1mBind Address0m
|
|
152
|
-
|
|
229
|
+
:bind
|
|
153
230
|
|
|
154
|
-
Specifies
|
|
231
|
+
Specifies the binding used to start the Gemstash server. Keep in mind
|
|
155
232
|
the user starting Gemstash needs to have access to bind in this manner.
|
|
156
|
-
For
|
|
233
|
+
For example, if you use a port below 1024, you will need to run Gem-
|
|
157
234
|
stash as the root user.
|
|
158
235
|
|
|
159
236
|
1mDefault value0m
|
|
160
|
-
|
|
237
|
+
tcp://0.0.0.0:9292
|
|
161
238
|
|
|
162
239
|
1mValid values0m
|
|
163
|
-
Any
|
|
164
|
-
(https://github.com/puma/puma#binding-tcp--sockets)
|
|
240
|
+
Any valid binding that 8;;https://github.com/puma/puma#binding-tcp--socketsis supported by Puma8;;
|
|
165
241
|
|
|
166
242
|
1mProtected Fetch0m
|
|
167
|
-
|
|
243
|
+
:protected_fetch
|
|
168
244
|
|
|
169
245
|
Tells Gemstash to authenticate via an API key before allowing the
|
|
170
246
|
fetching of private gems and specs. The default behavior is to allow
|
|
171
247
|
unauthenticated download of private gems and specs.
|
|
172
248
|
|
|
173
249
|
1mDefault value0m
|
|
174
|
-
|
|
250
|
+
false
|
|
175
251
|
|
|
176
252
|
1mValid values0m
|
|
177
|
-
Boolean values
|
|
253
|
+
Boolean values true or false
|
|
178
254
|
|
|
179
255
|
1mFetch Timeout0m
|
|
180
|
-
|
|
256
|
+
:fetch_timeout
|
|
257
|
+
|
|
258
|
+
This is the number of seconds to wait for an upstream read, not the to-
|
|
259
|
+
tal download time. A connection that keeps delivering data can take
|
|
260
|
+
longer; :max_upstream_duration bounds the entire request. After
|
|
261
|
+
:fetch_timeout seconds, the upstream connection must also average at
|
|
262
|
+
least 1 KiB/s across all response bytes, including headers, redirects,
|
|
263
|
+
chunk metadata and trailers, to prevent a connection that drips one
|
|
264
|
+
byte at a time from holding a worker. Fetching gems over a very slow
|
|
265
|
+
connection may cause timeout errors. The default is 20 seconds.
|
|
181
266
|
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
267
|
+
1mDefault value0m
|
|
268
|
+
20
|
|
269
|
+
|
|
270
|
+
1mValid values0m
|
|
271
|
+
Positive finite number of seconds (including fractions); nil is not
|
|
272
|
+
supported
|
|
273
|
+
|
|
274
|
+
1mMax Upstream Duration0m
|
|
275
|
+
:max_upstream_duration
|
|
276
|
+
|
|
277
|
+
Maximum seconds for an entire upstream request, including redirects and
|
|
278
|
+
streaming the body. Slow but steady downloads may exceed :fetch_time-
|
|
279
|
+
out without exceeding this limit. Increase it if legitimate downloads
|
|
280
|
+
need longer than five minutes.
|
|
187
281
|
|
|
188
282
|
1mDefault value0m
|
|
189
|
-
|
|
283
|
+
300 seconds
|
|
190
284
|
|
|
191
285
|
1mValid values0m
|
|
192
|
-
|
|
286
|
+
Positive finite number of seconds (including fractions); nil is not
|
|
287
|
+
supported
|
|
193
288
|
|
|
194
289
|
1mOpen Timeout0m
|
|
195
|
-
|
|
290
|
+
:open_timeout
|
|
196
291
|
|
|
197
|
-
The
|
|
198
|
-
|
|
199
|
-
upstream
|
|
200
|
-
failures instead of timeout errors,
|
|
201
|
-
|
|
292
|
+
The timeout setting for opening the connection to an upstream gem
|
|
293
|
+
server. On high-latency networks, even establishing the connection to
|
|
294
|
+
an upstream gem server can take a while. If you experience connection
|
|
295
|
+
failures instead of timeout errors, you may want to increase this
|
|
296
|
+
value. The default is 2 seconds.
|
|
297
|
+
|
|
298
|
+
1mDefault value0m
|
|
299
|
+
2
|
|
300
|
+
|
|
301
|
+
1mValid values0m
|
|
302
|
+
Integer value with a minimum of 1
|
|
303
|
+
|
|
304
|
+
1mMax Upstream Bytes0m
|
|
305
|
+
:max_upstream_bytes
|
|
306
|
+
|
|
307
|
+
The size in bytes of the largest response to accept from an upstream
|
|
308
|
+
gem server. This applies to every gem, gemspec, index of specs, and
|
|
309
|
+
list of dependencies that is fetched. A larger response is abandoned
|
|
310
|
+
while it is being downloaded and is not stashed, and the request for it
|
|
311
|
+
fails. The size is that of the response after any compression used to
|
|
312
|
+
transfer it has been undone. Make sure the value is larger than the
|
|
313
|
+
largest gem you need. Response headers are limited to 64 KiB total,
|
|
314
|
+
and each chunk metadata line to 64 KiB. A response can use about twice
|
|
315
|
+
the byte cap while buffers grow (over 2 GiB with 16 default Puma
|
|
316
|
+
threads). Increase this limit if your upstream has larger gems.
|
|
202
317
|
|
|
203
318
|
1mDefault value0m
|
|
204
|
-
|
|
319
|
+
67108864 (64 MiB)
|
|
205
320
|
|
|
206
321
|
1mValid values0m
|
|
207
|
-
|
|
322
|
+
Positive integer, or nil to disable the body cap (not recommended)
|
|
208
323
|
|
|
209
324
|
1mLog File0m
|
|
210
|
-
|
|
325
|
+
:log_file
|
|
211
326
|
|
|
212
|
-
Indicates
|
|
327
|
+
Indicates the name of the file to use for logging. The file will be
|
|
213
328
|
placed in the base path.
|
|
214
329
|
|
|
215
330
|
1mDefault value0m
|
|
216
|
-
|
|
331
|
+
server.log
|
|
217
332
|
|
|
218
333
|
1mValid values0m
|
|
219
|
-
Any valid file name, or
|
|
334
|
+
Any valid file name, or :stdout to log to $stdout
|
|
220
335
|
|
|
221
336
|
October 13, 2015 4mgemstash-configuration24m(5)
|