gemstash 1.1.0 → 2.0.0

Sign up to get free protection for your applications and to get access to all the features.
Files changed (54) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +34 -0
  3. data/lib/gemstash.rb +0 -1
  4. data/lib/gemstash/authorization.rb +1 -5
  5. data/lib/gemstash/cli.rb +4 -4
  6. data/lib/gemstash/cli/authorize.rb +4 -2
  7. data/lib/gemstash/cli/base.rb +5 -4
  8. data/lib/gemstash/cli/setup.rb +2 -2
  9. data/lib/gemstash/cli/status.rb +1 -1
  10. data/lib/gemstash/cli/stop.rb +1 -1
  11. data/lib/gemstash/configuration.rb +3 -2
  12. data/lib/gemstash/gem_pusher.rb +7 -0
  13. data/lib/gemstash/gem_source/private_source.rb +0 -4
  14. data/lib/gemstash/gem_source/upstream_source.rb +0 -4
  15. data/lib/gemstash/health.rb +3 -3
  16. data/lib/gemstash/logging.rb +2 -4
  17. data/lib/gemstash/man/gemstash-authorize.1.txt +53 -33
  18. data/lib/gemstash/man/gemstash-configuration.5.txt +10 -141
  19. data/lib/gemstash/man/gemstash-customize.7.txt +66 -175
  20. data/lib/gemstash/man/gemstash-debugging.7.txt +54 -17
  21. data/lib/gemstash/man/gemstash-deploy.7.txt +51 -44
  22. data/lib/gemstash/man/gemstash-mirror.7.txt +53 -20
  23. data/lib/gemstash/man/gemstash-multiple-sources.7.txt +48 -53
  24. data/lib/gemstash/man/gemstash-private-gems.7.txt +15 -117
  25. data/lib/gemstash/man/gemstash-readme.7.txt +10 -126
  26. data/lib/gemstash/man/gemstash-setup.1.txt +54 -28
  27. data/lib/gemstash/man/gemstash-start.1.txt +56 -16
  28. data/lib/gemstash/man/gemstash-status.1.txt +57 -13
  29. data/lib/gemstash/man/gemstash-stop.1.txt +57 -13
  30. data/lib/gemstash/man/gemstash-version.1.txt +57 -12
  31. data/lib/gemstash/migrations/01_gem_dependencies.rb +2 -2
  32. data/lib/gemstash/migrations/03_cached_gems.rb +1 -1
  33. data/lib/gemstash/migrations/04_health_tests.rb +8 -0
  34. data/lib/gemstash/puma.rb +1 -1
  35. data/lib/gemstash/storage.rb +1 -0
  36. data/lib/gemstash/upstream.rb +2 -2
  37. data/lib/gemstash/version.rb +1 -1
  38. data/lib/gemstash/web.rb +5 -4
  39. metadata +20 -28
  40. data/lib/gemstash/gem_unyanker.rb +0 -67
  41. data/lib/gemstash/man/gemstash-authorize.1 +0 -51
  42. data/lib/gemstash/man/gemstash-configuration.5 +0 -215
  43. data/lib/gemstash/man/gemstash-customize.7 +0 -280
  44. data/lib/gemstash/man/gemstash-debugging.7 +0 -34
  45. data/lib/gemstash/man/gemstash-deploy.7 +0 -72
  46. data/lib/gemstash/man/gemstash-mirror.7 +0 -40
  47. data/lib/gemstash/man/gemstash-multiple-sources.7 +0 -89
  48. data/lib/gemstash/man/gemstash-private-gems.7 +0 -244
  49. data/lib/gemstash/man/gemstash-readme.7 +0 -234
  50. data/lib/gemstash/man/gemstash-setup.1 +0 -43
  51. data/lib/gemstash/man/gemstash-start.1 +0 -26
  52. data/lib/gemstash/man/gemstash-status.1 +0 -20
  53. data/lib/gemstash/man/gemstash-stop.1 +0 -20
  54. data/lib/gemstash/man/gemstash-version.1 +0 -22
@@ -1,71 +1,66 @@
1
- gemstash-multiple-sources(7) gemstash-multiple-sources(7)
2
1
 
3
2
 
4
3
 
5
- MULTIPLE GEM SOURCES
6
- Gemstash will stash from any amount of gem sources. By the end of this
7
- guide, you will be able to bundle using multiple gem sources, all
8
- stashed within your Gemstash server.
9
4
 
10
- DEFAULT SOURCE
11
- When you don't provide an explicit source (as with the Quickstart Guide
12
- (gemstash help readme.7)), your gems will be fetched from
13
- https://rubygems.org. This default source is not set in stone. To
14
- change it, you need only edit the Gemstash configuration found at
15
- ~/.gemstash/config.yml:
16
5
 
17
- # ~/.gemstash/config.yml
18
- ---
19
- :rubygems_url: https://my.gem-source.local
20
6
 
21
- Make sure to restart your Gemstash server after changing the config:
22
7
 
23
- $ gemstash stop
24
- $ gemstash start
25
8
 
26
- Once restarted, bundling against http://localhost:9292 will fetch gems
27
- from https://my.gem-source.local. If you had bundled before making
28
- these changes, fear not; bundling with a different default gem source
29
- will store gems in a separate location, ensuring different sources
30
- won't leak between each other.
31
9
 
32
- BUNDLING WITH MULTIPLE SOURCES
33
- Changing the default source won't help you if you need to bundle
34
- against https://rubygems.org along with additional sources. If you
35
- need to bundle with multiple gem sources, Gemstash doesn't need to be
36
- specially configured. Your Gemstash server will honor any gem source
37
- specified via a specialized URL. Consider the following Gemfile:
38
10
 
39
- # ./Gemfile
40
- require "cgi"
41
- source "http://localhost:9292"
42
- gem "rubywarrior"
43
11
 
44
- source "http://localhost:9292/upstream/#{CGI.escape("https://my.gem-source.local")}" do
45
- gem "my-gem"
46
- end
47
12
 
48
- Notice the CGI.escape call in the second source. This is important, as
49
- it properly URL escapes the source URL so Gemstash knows what gem
50
- source you want. The /upstream prefix tells Gemstash to use a gem
51
- source other than the default source. You can now bundle with the ad-
52
- ditional source.
53
13
 
54
- REDIRECTING
55
- Gemstash supports an alternate mode of specifying your gem sources. If
56
- you want Gemstash to redirect Bundler to your given gem sources, then
57
- you can specify your Gemfile like so:
58
14
 
59
- # ./Gemfile
60
- require "cgi"
61
- source "http://localhost:9292/redirect/#{CGI.escape("https://rubygems.org")}"
62
- gem "rubywarrior"
63
15
 
64
- Notice the /redirect prefix. This prefix tells Gemstash to redirect
65
- API calls to the provided URL. Redirected calls like this will not be
66
- cached by Gemstash, and gem files will not be stashed, even if they
67
- were previously cached or stashed from the same gem source.
68
16
 
69
17
 
70
18
 
71
- October 8, 2015 gemstash-multiple-sources(7)
19
+
20
+
21
+
22
+
23
+
24
+
25
+
26
+
27
+
28
+
29
+
30
+
31
+
32
+
33
+
34
+
35
+
36
+
37
+
38
+
39
+
40
+
41
+
42
+
43
+
44
+
45
+
46
+
47
+
48
+
49
+
50
+
51
+
52
+
53
+
54
+
55
+
56
+
57
+
58
+
59
+
60
+
61
+
62
+
63
+
64
+
65
+
66
+
@@ -1,168 +1,66 @@
1
- gemstash-private-gems(7) gemstash-private-gems(7)
2
1
 
3
2
 
4
3
 
5
- PRIVATE GEMS
6
- Stashing private gems in your Gemstash server requires a bit of addi-
7
- tional setup. If you haven't read through the Quickstart Guide (gem-
8
- stash help readme.7), you should do that first. By the end of this
9
- guide, you will be able to interact with your Gemstash server to store
10
- and retrieve your private gems.
11
4
 
12
- AUTHORIZING
13
- IMPORTANT NOTE: Do not use the actual key value in this document, oth-
14
- erwise your Gemstash server will be vulnerable to anyone who wants to
15
- try to use the key against your server. Instead of the key value here,
16
- use whatever key is generated from running the commands.
17
5
 
18
- In order to push a gem to your Gemstash server, you need to first cre-
19
- ate an API key. Utilize the gemstash authorize command to create the
20
- API key:
21
6
 
22
- $ gemstash authorize
23
- Your new key is: e374e237fdf5fa5718d2a21bd63dc911
24
7
 
25
- This new key can push, yank, unyank, and fetch gems from your Gemstash
26
- server. Run gemstash authorize with just the permissions you want to
27
- limit what the key will be allowed to do. You can similarly update a
28
- specific key by providing it via the --key option:
29
8
 
30
- $ gemstash authorize push yank --key e374e237fdf5fa5718d2a21bd63dc911
31
9
 
32
- When no permissions are provided (like the first example), the key will
33
- be authorized for all permissions. Leave the key authorized with ev-
34
- erything if you want to use it to try all private gem interactions:
35
10
 
36
- $ gemstash authorize --key e374e237fdf5fa5718d2a21bd63dc911
37
11
 
38
- With the key generated, you'll need to tell Rubygems about your new
39
- key. If you've pushed a gem to https://rubygems.org, then you will al-
40
- ready have a credentials file to add the key to. If not, run the fol-
41
- lowing commands before modifying the credentials file:
42
12
 
43
- $ mkdir -p ~/.gem
44
- $ touch ~/.gem/credentials
45
- $ chmod 0600 ~/.gem/credentials
46
13
 
47
- Add your new key to credentials such that it looks something like this
48
- (but make sure not to remove any existing keys):
49
14
 
50
- # ~/.gem/credentials
51
- ---
52
- :test_key: e374e237fdf5fa5718d2a21bd63dc911
53
15
 
54
- The name test_key can be anything you want, but you will need to remem-
55
- ber it and use it again later in this guide for the --key option.
56
16
 
57
- CREATING A TEST GEM
58
- You'll need a test gem before you can play with private gems on your
59
- Gemstash server. If you have a gem you can use, move along to the next
60
- section. You can start by instantiating a test gem via Bundler:
61
17
 
62
- $ bundle gem private-example
63
18
 
64
- You'll need to add a summary and description to the new gem's gemspec
65
- file in order to successfully build it. Once you've built the gem, you
66
- will be ready to push the new gem.
67
19
 
68
- $ cd private-example
69
- $ rake build
70
20
 
71
- You will now have a gem at private-example/pkg/private-exam-
72
- ple-0.1.0.gem.
73
21
 
74
- PUSHING
75
- If your Gemstash server isn't running, go ahead and start it:
76
22
 
77
- $ gemstash start
78
23
 
79
- Push your test gem using Rubygems:
80
24
 
81
- $ gem push --key test_key --host http://localhost:9292/private pkg/private-example-0.1.0.gem
82
25
 
83
- The /private portion of the --host option tells Gemstash you are inter-
84
- acting with the private gems. Gemstash will not let you push, yank, or
85
- unyank from anything except /private.
86
26
 
87
- BUNDLING
88
- Once your gem is pushed to your Gemstash server, you are ready to bun-
89
- dle it. Create a Gemfile and specify the gem. You will probably want
90
- to wrap the private gem in a source block, and let the rest of Gemstash
91
- handle all other gems:
92
27
 
93
- # ./Gemfile
94
- source "http://localhost:9292"
95
- gem "rubywarrior"
96
28
 
97
- source "http://localhost:9292/private" do
98
- gem "private-example"
99
- end
100
29
 
101
- Notice that the Gemstash server points to /private again when in-
102
- stalling your private gem. Go ahead and bundle to install your new
103
- private gem:
104
30
 
105
- $ bundle
106
31
 
107
- YANKING
108
- If you push a private gem by accident, you can yank the gem with
109
- Rubygems:
110
32
 
111
- $ RUBYGEMS_HOST=http://localhost:9292/private gem yank --key test_key private-example --version 0.1.0
112
33
 
113
- Like with pushing, the /private portion of the host option tells Gem-
114
- stash you are interacting with private gems. Gemstash will only let
115
- you yank from /private. Unlike pushing, Rubygems doesn't support
116
- --host for yank and unyank (yet), so you need to specify the host via
117
- the RUBYGEMS_HOST environment variable.
118
34
 
119
- UNYANKING
120
- If you yank a private gem by accident, you can unyank the gem with
121
- Rubygems:
122
35
 
123
- $ RUBYGEMS_HOST=http://localhost:9292/private gem yank --key test_key private-example --version 0.1.0 --undo
124
36
 
125
- Like with pushing and yanking, the /private portion of the host option
126
- tells Gemstash you are interacting with private gems. Gemstash will
127
- only let you unyank from /private. Unlike pushing, Rubygems doesn't
128
- support --host for unyank and yank (yet), so you need to specify the
129
- host via the RUBYGEMS_HOST environment variable.
130
37
 
131
- PROTECTED FETCHING
132
- By default, private gems and specs can be accessed without authentica-
133
- tion.
134
38
 
135
- Private gems often require protected fetching. For backwards compati-
136
- bility this is disabled by default, but can be enabled via $ gem-
137
- stash setup command.
138
39
 
139
- When protected fetching is enabled API keys with the permissions all or
140
- fetch can be used to download gems and specs.
141
40
 
142
- On the Bundler side, there are a few ways to configure credentials for
143
- a given gem source:
144
41
 
145
- Add credentials globally:
146
42
 
147
- $ bundle config my-gemstash.dev api_key
148
43
 
149
- Add credentials in Gemfile:
150
44
 
151
- source "https://api_key@my-gemstash.dev"
152
45
 
153
- However, it's not a good practice to commit credentials to source con-
154
- trol. A recommended solution is to use Bundler's configuration keys
155
- (http://bundler.io/man/bundle-config.1.html#CONFIGURATION-KEYS), e.g.:
156
46
 
157
- $ export BUNDLE_MYGEMSTASH__DEV=api_key
158
47
 
159
- Behind the scene, Bundler will pick up the ENV var according to the
160
- host name (e.g. mygemstash.dev) and add to URI.userinfo for making re-
161
- quests.
162
48
 
163
- The API key is treated as a HTTP Basic Auth username and any HTTP Basic
164
- password supplied will be ignored.
165
49
 
166
50
 
167
51
 
168
- October 8, 2015 gemstash-private-gems(7)
52
+
53
+
54
+
55
+
56
+
57
+
58
+
59
+
60
+
61
+
62
+
63
+
64
+
65
+
66
+
@@ -1,182 +1,66 @@
1
- gemstash-readme(7) gemstash-readme(7)
2
1
 
3
2
 
4
3
 
5
- GEMSTASH
6
- WHAT IS GEMSTASH?
7
- Gemstash is both a cache for remote servers such as
8
- https://rubygems.org, and a private gem source.
9
4
 
10
- If you are using bundler (http://bundler.io/) across many machines that
11
- have access to a server within your control, you might want to use Gem-
12
- stash.
13
5
 
14
- If you produce gems that you don't want everyone in the world to have
15
- access to, you might want to use Gemstash.
16
6
 
17
- If you frequently bundle the same set of gems across multiple projects,
18
- you might want to use Gemstash.
19
7
 
20
- Are you only using gems from https://rubygems.org, and don't bundle the
21
- same gems frequently? Well, maybe you don't need Gemstash... yet.
22
8
 
23
- Gemstash is maintained by Ruby Together (https://rubytogether.org/), a
24
- grassroots initiative committed to supporting the critical Ruby infra-
25
- structure you rely on. Contribute today as an individual
26
- (https://rubytogether.org/developers) or even better, as a company
27
- (https://rubytogether.org/companies), and ensure that Bundler,
28
- RubyGems, Gemstash, and other shared tooling is around for years to
29
- come.
30
9
 
31
- QUICKSTART GUIDE
32
- SETUP
33
- Gemstash is designed to be quick and painless to get set up. By the
34
- end of this Quickstart Guide, you will be able to bundle stashed gems
35
- from public sources against a Gemstash server running on your machine.
36
10
 
37
- Install Gemstash to get started:
38
11
 
39
- $ gem install gemstash
40
12
 
41
- After it is installed, starting Gemstash requires no additional steps.
42
- Simply start the Gemstash server with the gemstash command:
43
13
 
44
- $ gemstash start
45
14
 
46
- You may have noticed that the command finished quickly. This is be-
47
- cause Gemstash will run the server in the background by default. The
48
- server runs on port 9292.
49
15
 
50
- BUNDLING
51
- With the server running, you can bundle against it. Tell Bundler that
52
- you want to use Gemstash to find gems from RubyGems.org:
53
16
 
54
- $ bundle config mirror.https://rubygems.org http://localhost:9292
55
17
 
56
- Now you can create a Gemfile and install gems through Gemstash:
57
18
 
58
- # ./Gemfile
59
- source "https://rubygems.org"
60
- gem "rubywarrior"
61
19
 
62
- The gems you include should be gems you don't yet have installed, oth-
63
- erwise Gemstash will have nothing to stash. Now bundle:
64
20
 
65
- $ bundle install --path .bundle
66
21
 
67
- Your Gemstash server has fetched the gems from https://rubygems.org and
68
- cached them for you! To prove this, you can disable your Internet con-
69
- nection and try again. Gem files (*.gem) are cached indefinitely. Gem
70
- dependencies metadata are cached for 30 minutes, so if you bundle again
71
- before that, you can successfully bundle without an Internet connec-
72
- tion:
73
22
 
74
- $ # Disable your Internet first!
75
- $ rm -rf Gemfile.lock .bundle
76
- $ bundle
77
23
 
78
- FALLING BACK TO RUBYGEMS.ORG
79
- If you want to make sure that your bundling from https://rubygems.org
80
- still works as expected when the Gemstash server is not running, you
81
- can easily configure Bundler to fallback to https://rubygems.org.
82
24
 
83
- $ bundle config mirror.https://rubygems.org.fallback_timeout true
84
25
 
85
- You can also configure this fallback as a number of seconds in case the
86
- Gemstash server is simply unresponsive. This example uses a 3 second
87
- timeout:
88
26
 
89
- $ bundle config mirror.https://rubygems.org.fallback_timeout 3
90
27
 
91
- STOPPING THE SERVER
92
- Once you've finish using your Gemstash server, you can stop it just as
93
- easily as you started it:
94
28
 
95
- $ gemstash stop
96
29
 
97
- You'll also want to tell Bundler that it can go back to getting gems
98
- from RubyGems.org directly, instead of going through Gemstash:
99
30
 
100
- $ bundle config --delete mirror.https://rubygems.org
101
31
 
102
- UNDER THE HOOD
103
- You might wonder where the gems are stored. After running the commands
104
- above, you will find a new directory at ~/.gemstash. This directory
105
- holds all the cached and private gems. It also has a server log, the
106
- database, and configuration for Gemstash. If you prefer, you can point
107
- to a different directory (gemstash help customize.7).
108
32
 
109
- Gemstash uses SQLite (https://www.sqlite.org/) to store details about
110
- private gems. The database will be located in ~/.gemstash, however you
111
- won't see the database appear until you start using private gems. If
112
- you prefer, you can use a different database (gemstash help cus-
113
- tomize.7).
114
33
 
115
- Gemstash temporarily caches things like gem dependencies in memory.
116
- Anything cached in memory will last for 30 minutes before being re-
117
- trieved again. You can use memcached (gemstash help customize.7) in-
118
- stead of caching in memory. Gem files are always cached permanently,
119
- so bundling with a Gemfile.lock with all gems cached will never call
120
- out to https://rubygems.org.
121
34
 
122
- The server you ran is provided via Puma (http://puma.io/) and Rack
123
- (http://rack.github.io/), however they are not customizable at this
124
- point.
125
35
 
126
- DEEP DIVE
127
- Deep dive into more subjects:
128
36
 
129
- o Private gems (gemstash help private-gems.7)
130
37
 
131
- o Multiple gem sources (gemstash help multiple-sources.7)
132
38
 
133
- o Using Gemstash as a mirror (gemstash help mirror.7)
134
39
 
135
- o Customizing the server (database, storage, caching, and more) (gem-
136
- stash help customize.7)
137
40
 
138
- o Deploying Gemstash (gemstash help deploy.7)
139
41
 
140
- o Debugging Gemstash (gemstash help debugging.7)
141
42
 
142
- REFERENCE
143
- An anatomy of various configuration and commands:
144
43
 
145
- o Configuration (gemstash help configuration.5)
146
44
 
147
- o Authorize (gemstash help authorize.1)
148
45
 
149
- o Start (gemstash help start.1)
150
46
 
151
- o Stop (gemstash help stop.1)
152
47
 
153
- o Status (gemstash help status.1)
154
48
 
155
- o Setup (gemstash help setup.1)
156
49
 
157
- o Version (gemstash help version.1)
158
50
 
159
- To see what has changed in recent versions of Gemstash, see the
160
- CHANGELOG (https://github.com/bundler/gemstash/blob/master/CHANGEL-
161
- OG.md).
162
51
 
163
- DEVELOPMENT
164
- After checking out the repo, run bin/setup to install dependencies.
165
- Then, run rake to run RuboCop and the tests. While developing, you can
166
- run bin/gemstash to run Gemstash. You can also run bin/console for an
167
- interactive prompt that will allow you to experiment.
168
52
 
169
- CONTRIBUTING
170
- Bug reports and pull requests are welcome on GitHub at
171
- https://github.com/bundler/gemstash. This project is intended to be a
172
- safe, welcoming space for collaboration, and contributors are expected
173
- to adhere to the Contributor Covenant (https://github.com/bundler/gem-
174
- stash/blob/master/CODE_OF_CONDUCT.md) code of conduct.
175
53
 
176
- LICENSE
177
- The gem is available as open source under the terms of the MIT License
178
- (http://opensource.org/licenses/MIT).
179
54
 
180
55
 
181
56
 
182
- November 30, 2015 gemstash-readme(7)
57
+
58
+
59
+
60
+
61
+
62
+
63
+
64
+
65
+
66
+