gemstash 1.0.0.pre.1-java

Sign up to get free protection for your applications and to get access to all the features.
Files changed (59) hide show
  1. checksums.yaml +7 -0
  2. data/CODE_OF_CONDUCT.md +13 -0
  3. data/Gemfile +4 -0
  4. data/LICENSE.txt +21 -0
  5. data/README.md +139 -0
  6. data/Rakefile +35 -0
  7. data/bin/console +14 -0
  8. data/bin/gemstash +3 -0
  9. data/bin/setup +5 -0
  10. data/docs/config.md +136 -0
  11. data/docs/debug.md +24 -0
  12. data/docs/deploy.md +30 -0
  13. data/docs/mirror.md +30 -0
  14. data/docs/multiple_sources.md +68 -0
  15. data/docs/private_gems.md +140 -0
  16. data/docs/reference.md +308 -0
  17. data/exe/gemstash +3 -0
  18. data/gemstash.gemspec +47 -0
  19. data/gemstash.png +0 -0
  20. data/lib/gemstash.rb +26 -0
  21. data/lib/gemstash/authorization.rb +87 -0
  22. data/lib/gemstash/cache.rb +79 -0
  23. data/lib/gemstash/cli.rb +71 -0
  24. data/lib/gemstash/cli/authorize.rb +69 -0
  25. data/lib/gemstash/cli/base.rb +46 -0
  26. data/lib/gemstash/cli/setup.rb +173 -0
  27. data/lib/gemstash/cli/start.rb +52 -0
  28. data/lib/gemstash/cli/status.rb +21 -0
  29. data/lib/gemstash/cli/stop.rb +21 -0
  30. data/lib/gemstash/config.ru +13 -0
  31. data/lib/gemstash/configuration.rb +41 -0
  32. data/lib/gemstash/db.rb +15 -0
  33. data/lib/gemstash/db/authorization.rb +20 -0
  34. data/lib/gemstash/db/dependency.rb +50 -0
  35. data/lib/gemstash/db/rubygem.rb +14 -0
  36. data/lib/gemstash/db/version.rb +51 -0
  37. data/lib/gemstash/dependencies.rb +93 -0
  38. data/lib/gemstash/env.rb +150 -0
  39. data/lib/gemstash/gem_fetcher.rb +50 -0
  40. data/lib/gemstash/gem_pusher.rb +125 -0
  41. data/lib/gemstash/gem_source.rb +37 -0
  42. data/lib/gemstash/gem_source/dependency_caching.rb +40 -0
  43. data/lib/gemstash/gem_source/private_source.rb +139 -0
  44. data/lib/gemstash/gem_source/rack_middleware.rb +22 -0
  45. data/lib/gemstash/gem_source/upstream_source.rb +183 -0
  46. data/lib/gemstash/gem_unyanker.rb +61 -0
  47. data/lib/gemstash/gem_yanker.rb +61 -0
  48. data/lib/gemstash/http_client.rb +77 -0
  49. data/lib/gemstash/logging.rb +93 -0
  50. data/lib/gemstash/migrations/01_gem_dependencies.rb +41 -0
  51. data/lib/gemstash/migrations/02_authorizations.rb +12 -0
  52. data/lib/gemstash/puma.rb +6 -0
  53. data/lib/gemstash/rack_env_rewriter.rb +66 -0
  54. data/lib/gemstash/specs_builder.rb +93 -0
  55. data/lib/gemstash/storage.rb +207 -0
  56. data/lib/gemstash/upstream.rb +65 -0
  57. data/lib/gemstash/version.rb +4 -0
  58. data/lib/gemstash/web.rb +97 -0
  59. metadata +304 -0
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA1:
3
+ metadata.gz: b4f42a90cf7dd5629135b2d0afa39f09b7a35d78
4
+ data.tar.gz: 6967678927e52142303a1122f34e800f7dbbaa8c
5
+ SHA512:
6
+ metadata.gz: 175724eeb41c706afd9db4d26f03e62925202c0d6644e120a889854469ea00bc82b0b9106187de3fbd91f96fc92d98f9ea0a04edbdc3dd3aaefc59598fde9c64
7
+ data.tar.gz: 54e6502a0cfb9a35679cde8d2469176e71f0d67b822b89427eea19ac11bd89a3d3356e1a7a903471749f6c5172995b5bd560cc4a2fd17f662438c033c8afb2fb
@@ -0,0 +1,13 @@
1
+ # Contributor Code of Conduct
2
+
3
+ As contributors and maintainers of this project, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities.
4
+
5
+ We are committed to making participation in this project a harassment-free experience for everyone, regardless of level of experience, gender, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, or religion.
6
+
7
+ Examples of unacceptable behavior by participants include the use of sexual language or imagery, derogatory comments or personal attacks, trolling, public or private harassment, insults, or other unprofessional conduct.
8
+
9
+ Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct. Project maintainers who do not follow the Code of Conduct may be removed from the project team.
10
+
11
+ Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by opening an issue or contacting one or more of the project maintainers.
12
+
13
+ This Code of Conduct is adapted from the [Contributor Covenant](http://contributor-covenant.org), version 1.0.0, available at [http://contributor-covenant.org/version/1/0/0/](http://contributor-covenant.org/version/1/0/0/)
data/Gemfile ADDED
@@ -0,0 +1,4 @@
1
+ source "https://rubygems.org"
2
+
3
+ # Specify your gem's dependencies in gemstash.gemspec
4
+ gemspec
data/LICENSE.txt ADDED
@@ -0,0 +1,21 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2015 Andre Arko
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in
13
+ all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21
+ THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,139 @@
1
+ [![Build Status](https://travis-ci.org/bundler/gemstash.svg?branch=master)](https://travis-ci.org/bundler/gemstash)
2
+
3
+ <p align="center"><img src="gemstash.png" /></p>
4
+
5
+ ## What is Gemstash?
6
+
7
+ Gemstash is both a cache for remote servers such as https://www.rubygems.org,
8
+ and a private gem source.
9
+
10
+ If you are using [bundler](http://bundler.io/) across many machines that have
11
+ access to a server within your control, you might want to use Gemstash.
12
+
13
+ If you produce gems that you don't want everyone in the world to have access to,
14
+ you might want to use Gemstash.
15
+
16
+ If you frequently bundle the same set of gems across multiple projects, you
17
+ might want to use Gemstash.
18
+
19
+ Are you only using gems from https://www.rubygems.org, and don't bundle the same
20
+ gems frequently? Well, maybe you don't need Gemstash... yet.
21
+
22
+ ## Quickstart Guide
23
+
24
+ ### Setup
25
+
26
+ Gemstash is designed to be quick and painless to get set up. By the end of this
27
+ Quickstart Guide, you will be able to bundle stashed gems from public sources
28
+ against a Gemstash server running on your machine.
29
+
30
+ Install Gemstash to get started:
31
+ ```
32
+ $ gem install gemstash
33
+ ```
34
+
35
+ After it is installed, starting Gemstash requires no additional steps. Simply
36
+ start the Gemstash server with the `gemstash` command:
37
+ ```
38
+ $ gemstash start
39
+ ```
40
+
41
+ You may have noticed that the command finished quickly. This is because Gemstash
42
+ will run the server in the background by default. The server runs on port 9292.
43
+
44
+ ### Bundling
45
+
46
+ With the server running, you can bundle against it. Create a simple `Gemfile`
47
+ like the following:
48
+ ```ruby
49
+ # ./Gemfile
50
+ source "http://localhost:9292"
51
+ gem "rubywarrior"
52
+ ```
53
+
54
+ The first line is important, as it will tell Bundler to use your new Gemstash
55
+ server. The gems you include should be gems you don't yet have installed,
56
+ otherwise Gemstash will have nothing to stash. Now bundle:
57
+ ```
58
+ $ bundle install --path .bundle
59
+ ```
60
+
61
+ Your Gemstash server has fetched the gems from https://www.rubygems.org and
62
+ cached them for you! To prove this, you can disable your Internet connection and
63
+ try again. The gem dependencies from https://www.rubygems.org are cached for 30
64
+ minutes, so if you bundle again before that, you can successfully bundle without
65
+ an Internet connection:
66
+ ```
67
+ $ # Disable your Internet first!
68
+ $ rm -rf Gemfile.lock .bundle
69
+ $ bundle
70
+ ```
71
+
72
+ ### Stopping the Server
73
+
74
+ Once you've finish using your Gemstash server, you can stop it just as easily as
75
+ you started it:
76
+ ```
77
+ $ gemstash stop
78
+ ```
79
+
80
+ ### Under the Hood
81
+
82
+ You might wonder where the gems are stored. After running the commands above,
83
+ you will find a new directory at `~/.gemstash`. This directory holds all the
84
+ cached and private gems. It also has a server log, the database, and
85
+ configuration for Gemstash. If you prefer, you can [point to a different
86
+ directory](docs/config.md#files).
87
+
88
+ Gemstash uses [SQLite](https://www.sqlite.org/) to store details about private
89
+ gems. The database will be located in `~/.gemstash`, however you won't see the
90
+ database appear until you start using private gems. If you prefer, you can [use
91
+ a different database](docs/config.md#database).
92
+
93
+ Gemstash temporarily caches things like gem dependencies in memory. Anything
94
+ cached in memory will last for 30 minutes before being retrieved again. You can
95
+ [use memcached](docs/config.md#cache) instead of caching in memory. Gem files
96
+ are always cached permanently, so bundling with a `Gemfile.lock` with all gems
97
+ cached will never call out to https://www.rubygems.org.
98
+
99
+ The server you ran is provided via [Puma](http://puma.io/) and
100
+ [Rack](http://rack.github.io/), however they are not customizable at this point.
101
+
102
+ ## Deep Dive
103
+
104
+ For a deep dive into the following subjects, follow the links:
105
+ * [Private gems](docs/private_gems.md)
106
+ * [Multiple gem sources](docs/multiple_sources.md)
107
+ * [Using Gemstash as a mirror](docs/mirror.md)
108
+ * [Customizing the server (database, storage, caching, and more)](docs/config.md)
109
+ * [Deploying Gemstash](docs/deploy.md)
110
+ * [Debugging Gemstash](docs/debug.md)
111
+
112
+ ## Reference
113
+
114
+ For an anatomy of various configuration and commands, follow the links:
115
+ * [Configuration](docs/reference.md#configuration)
116
+ * [Authorize](docs/reference.md#authorize)
117
+ * [Start](docs/reference.md#start)
118
+ * [Stop](docs/reference.md#stop)
119
+ * [Status](docs/reference.md#status)
120
+ * [Setup](docs/reference.md#setup)
121
+
122
+ ## Development
123
+
124
+ After checking out the repo, run `bin/setup` to install dependencies. Then, run
125
+ `rake` to run RuboCop and the tests. While developing, you can run
126
+ `bin/gemstash` to run Gemstash. You can also run `bin/console` for an
127
+ interactive prompt that will allow you to experiment.
128
+
129
+ ## Contributing
130
+
131
+ Bug reports and pull requests are welcome on GitHub at
132
+ https://github.com/bundler/gemstash. This project is intended to be a safe,
133
+ welcoming space for collaboration, and contributors are expected to adhere to
134
+ the [Contributor Covenant](CODE_OF_CONDUCT.md) code of conduct.
135
+
136
+ ## License
137
+
138
+ The gem is available as open source under the terms of the
139
+ [MIT License](http://opensource.org/licenses/MIT).
data/Rakefile ADDED
@@ -0,0 +1,35 @@
1
+ require "bundler/gem_tasks"
2
+ require "rspec/core/rake_task"
3
+ require "rubocop/rake_task"
4
+
5
+ RuboCop::RakeTask.new
6
+
7
+ desc "Run specs"
8
+ RSpec::Core::RakeTask.new(:spec) do |t|
9
+ t.rspec_opts = %w(--color)
10
+ end
11
+
12
+ task :spec => :rubocop
13
+ task :default => :spec
14
+
15
+ desc "Generate Table of Contents for certain docs"
16
+ task :toc do
17
+ toc_dir = File.expand_path("../tmp/", __FILE__)
18
+ toc = File.join(toc_dir, "gh-md-toc")
19
+
20
+ unless File.exist?(toc)
21
+ require "open-uri"
22
+ toc_contents = open("https://raw.githubusercontent.com/ekalinin/github-markdown-toc/master/gh-md-toc", &:read)
23
+ Dir.mkdir(toc_dir) unless Dir.exist?(toc_dir)
24
+ File.write(toc, toc_contents)
25
+ File.chmod(0776, toc)
26
+ end
27
+
28
+ doc = File.expand_path("../docs/reference.md", __FILE__)
29
+ old_contents = File.read(doc)
30
+ old_contents.sub!(/\A.*?^---$/m, "---")
31
+ File.write(doc, old_contents)
32
+ toc_contents = `"#{toc}" "#{doc}"`
33
+ toc_contents.sub!(/Created by.*$/, "")
34
+ File.write(doc, "#{toc_contents}\n#{old_contents}")
35
+ end
data/bin/console ADDED
@@ -0,0 +1,14 @@
1
+ #!/usr/bin/env ruby
2
+
3
+ require "bundler/setup"
4
+ require "gemstash"
5
+
6
+ # You can add fixtures and/or initialization code here to make experimenting
7
+ # with your gem easier. You can also use a different console, if you like.
8
+
9
+ # (If you use this, don't forget to add pry to your Gemfile!)
10
+ # require "pry"
11
+ # Pry.start
12
+
13
+ require "irb"
14
+ IRB.start
data/bin/gemstash ADDED
@@ -0,0 +1,3 @@
1
+ #!/usr/bin/env ruby
2
+ require "bundler/setup"
3
+ exec "exe/gemstash", *ARGV
data/bin/setup ADDED
@@ -0,0 +1,5 @@
1
+ #!/bin/bash
2
+ set -euo pipefail
3
+ IFS=$'\n\t'
4
+
5
+ bundle install
data/docs/config.md ADDED
@@ -0,0 +1,136 @@
1
+ # Customizing the Server
2
+
3
+ Although Gemstash is designed for as minimal setup as possible, there may be
4
+ times you will want to change some of the default configuration. By the end of
5
+ this guide, you will be able to customize some of the Gemstash behavior,
6
+ including where files are stored, what database Gemstash uses, and how Gemstash
7
+ caches certain requests.
8
+
9
+ ## Setup
10
+
11
+ Several customizable options are available via an interactive Gemstash command.
12
+ Run `gemstash setup` and answer the questions it provides (a blank answer will
13
+ use the default value):
14
+ <pre>
15
+ $ gemstash setup
16
+ Where should files go? [~/.gemstash]
17
+ Cache with what? [MEMORY, memcached] <strong>memcached</strong>
18
+ What is the comma separated Memcached servers? [localhost:11211]
19
+ What database adapter? [SQLITE3, postgres] <strong>postgres</strong>
20
+ Where is the database? [postgres:///gemstash]
21
+ Checking that the cache is available
22
+ Checking that the database is available
23
+ The database is not available
24
+ </pre>
25
+
26
+ Once you've answered the questsions, some checks will be made to ensure the
27
+ configuration will work. For example, the database didn't exist in the previous
28
+ example, so the command failed and the configuration wasn't saved. If the
29
+ command passes, you may provide the `--redo` option to force configuration to be
30
+ redone:
31
+ <pre>
32
+ $ gemstash setup --redo
33
+ Where should files go? [~/.gemstash]
34
+ Cache with what? [MEMORY, memcached] <strong>memcached</strong>
35
+ What is the comma separated Memcached servers? [localhost:11211]
36
+ What database adapter? [SQLITE3, postgres]
37
+ Checking that the cache is available
38
+ Checking that the database is available
39
+ You are all setup!
40
+ </pre>
41
+
42
+ Once all checks have passed, Gemstash will store your answers in the
43
+ configuration file located at `~/.gemstash/config.yml`.
44
+
45
+ ## Files
46
+
47
+ Storage in Gemstash defaults to `~/.gemstash` unless otherwise specified. You
48
+ can change this in your config file via the `:base_path` key:
49
+ ```yaml
50
+ # ~/.gemstash/config.yml
51
+ ---
52
+ :base_path: "/var/gemstash"
53
+ ```
54
+
55
+ When customizing the `base_path`, the directory must exist, otherwise Gemstash
56
+ will fail to run. Thus, if you want to use `/var/gemstash` like in the previous
57
+ example, make sure to `mkdir /var/gemstash` and grant access to the directory
58
+ for the user you run Gemstash with.
59
+
60
+ ## Database
61
+
62
+ The `:db_adapter` configuration key specifies what database you will be using.
63
+ The default `:db_adapter` is [`sqlite3`](https://www.sqlite.org/), which will
64
+ use a database file located within your `:base_path`. The database file will
65
+ always be named `gemstash.db`.
66
+
67
+ You may also use [`postgres`](http://www.postgresql.org/) for your
68
+ `:db_adapter`. When using PostgreSQL, you need to specify the `:db_url` to point
69
+ to an existing database. Here is an example configuration to use the `postgres`
70
+ adapter:
71
+ ```yaml
72
+ # ~/.gemstash/config.yml
73
+ ---
74
+ :db_adapter: postgres
75
+ :db_url: postgres:///gemstash
76
+ ```
77
+
78
+ Regardless of the adapter you choose, the database will automatically migrate to
79
+ your version of Gemstash whenever the database is needed. Except for `sqlite3`,
80
+ you only need to ensure the database exists and Gemstash will do the rest.
81
+
82
+ ## Cache
83
+
84
+ Certain things (like dependencies) are cached in memory. This avoids web calls
85
+ to the gem source, and database calls for private gems. The memory cache can
86
+ optionally be swapped out with a [Memcached](http://memcached.org/) server (or
87
+ cluster of servers). To use Memcached, you must update the `:cache_type`
88
+ configuration key to be `memcached`, and provide the servers via the
89
+ `:memcached_servers` configuration key:
90
+ ```yaml
91
+ # ~/.gemstash/config.yml
92
+ ---
93
+ :cache_type: memcached
94
+ :memcached_servers: memcached1.local:11211,memcached2.local:11211
95
+ ```
96
+
97
+ Note that the `:memcached_servers` requires a comma separated list of servers.
98
+ All caching lasts for 30 minutes.
99
+
100
+ ## Server
101
+
102
+ Gemstash uses [Puma](http://puma.io/) and [Rack](http://rack.github.io/) as the
103
+ server. Alternate server configurations are not currently supported, but you can
104
+ take a look at the [Puma configuration](../lib/gemstash/puma.rb) and the [rackup
105
+ file](../lib/gemstash/config.ru) for inspiration.
106
+
107
+ While the server is not customizable, the way Gemstash binds the port can be
108
+ changed. To change the binding, update the `:bind` configuration key:
109
+ ```yaml
110
+ # ~/.gemstash/config.yml
111
+ ---
112
+ :bind: tcp://0.0.0.0:4242
113
+ ```
114
+
115
+ This maps directly to the [Puma bind
116
+ flag](https://github.com/puma/puma#binding-tcp--sockets), and will support
117
+ anything valid for that flag.
118
+
119
+ ## Config File Location
120
+
121
+ By default, configuration for Gemstash will be at `~/.gemstash/config.yml`. This
122
+ can be changed by providing the `--config-file` option to the various Gemstash
123
+ commands:
124
+ ```
125
+ $ gemstash setup --config-file ./gemstash-config.yml
126
+ $ gemstash authorize --config-file ./gemstash-config.yml
127
+ $ gemstash start --config-file ./gemstash-config.yml
128
+ $ gemstash stop --config-file ./gemstash-config.yml
129
+ $ gemstash status --config-file ./gemstash-config.yml
130
+ ```
131
+
132
+ When providing `--config-file` to `gemstash setup`, the provided file will be
133
+ output to with the provided configuration. **This will overwrite** any existing
134
+ configuration. If the file doesn't exist when providing `--config-file` to
135
+ `gemstash start`, `gemstash stop`, `gemstash status`, and `gemstash authorize`,
136
+ the default configuration will be used.
data/docs/debug.md ADDED
@@ -0,0 +1,24 @@
1
+ # Debugging Gemstash
2
+
3
+ If you are finding Gemstash isn't behaving as you would expect, you might want
4
+ to start by looking at the server log. You can find the log at `server.log`
5
+ within your base directory. By default, this will be at
6
+ `~/.gemstash/server.log`.
7
+
8
+ You might find it easier to view the log directly in your terminal. If you run
9
+ Gemstash [in non-daemonized form](reference.md#--no-daemonize), the log will be
10
+ output directly to standard out:
11
+ ```
12
+ $ gemstash start --no-daemonize
13
+ ```
14
+
15
+ You can also [check the status](reference.md#status) of the server:
16
+ ```
17
+ $ gemstash status
18
+ ```
19
+
20
+ The server status is checked by passing through to
21
+ [pumactl](https://github.com/puma/puma#pumactl).
22
+
23
+ If you find a bug, please don't hesitate to [open a bug
24
+ report](../README.md#contributing)!
data/docs/deploy.md ADDED
@@ -0,0 +1,30 @@
1
+ # Deploying Gemstash
2
+
3
+ Bundler is here for the rescue to keep Gemstash up to date! Create a `Gemfile`
4
+ pointing to Gemstash:
5
+ ```ruby
6
+ # ./Gemfile
7
+ source "https://www.rubygems.org"
8
+ gem "gemstash"
9
+ ```
10
+
11
+ Then `bundle` to create your `Gemfile.lock`. When you are ready to upgrade,
12
+ simply `bundle update`. You may need to run `gemstash` via `bundle exec`.
13
+ Alternatively, you can `gem uninstall gemstash` and `gem install gemstash` when
14
+ you want to upgrade.
15
+
16
+ Gemstash will automatically run any necessary migrations, so updating the gem is
17
+ all that needs to be done.
18
+
19
+ It is probably wise to stop Gemstash before upgrading, then starting again once
20
+ you are done:
21
+ ```
22
+ $ bundle exec gemstash stop
23
+ $ bundle update
24
+ $ bundle exec gemstash start
25
+ ```
26
+
27
+ ## Downgrading
28
+
29
+ It is not recommended to go backwards in Gemstash versions. Migrations may have
30
+ run that could leave the database in a bad state.