gemvault 0.1.5 → 0.2.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (46) hide show
  1. checksums.yaml +4 -4
  2. data/.containerignore +10 -0
  3. data/.rubocop.yml +9 -1
  4. data/CHANGELOG.md +61 -0
  5. data/CLAUDE.md +70 -11
  6. data/Dockerfile.test +65 -10
  7. data/README.md +3 -11
  8. data/Rakefile +26 -2
  9. data/_claude/archives/issue-13-root-cause.md +197 -0
  10. data/docs/Rakefile.vault_example +76 -0
  11. data/issues.rec +614 -15
  12. data/lib/bundler/plugin/vault_source.rb +48 -73
  13. data/lib/bundler/plugin/vaulted_gem.rb +66 -0
  14. data/lib/gemvault/archive_entry.rb +6 -0
  15. data/lib/gemvault/cli/command.rb +9 -5
  16. data/lib/gemvault/cli/commands/extract.rb +11 -11
  17. data/lib/gemvault/cli/commands/new.rb +3 -2
  18. data/lib/gemvault/cli/commands/remove.rb +15 -5
  19. data/lib/gemvault/cli/commands/upgrade.rb +37 -13
  20. data/lib/gemvault/dbvault.rb +12 -9
  21. data/lib/gemvault/deprecation.rb +26 -24
  22. data/lib/gemvault/gem_entry.rb +20 -24
  23. data/lib/gemvault/gem_extraction.rb +19 -23
  24. data/lib/gemvault/gem_reference/any_version.rb +6 -2
  25. data/lib/gemvault/gem_reference/parser.rb +44 -0
  26. data/lib/gemvault/gem_reference/specific_version.rb +9 -25
  27. data/lib/gemvault/gem_reference.rb +13 -45
  28. data/lib/gemvault/manifest.rb +44 -42
  29. data/lib/gemvault/tarball.rb +53 -0
  30. data/lib/gemvault/tarvault.rb +47 -56
  31. data/lib/gemvault/vault.rb +17 -13
  32. data/lib/gemvault/vault_path.rb +33 -0
  33. data/lib/gemvault/vault_upgrade.rb +31 -24
  34. data/lib/gemvault/version.rb +1 -1
  35. data/lib/gemvault.rb +1 -0
  36. data/lib/rubygems/resolver/vault_set.rb +5 -7
  37. data/lib/rubygems/source/vault.rb +28 -32
  38. data/lib/rubygems_plugin.rb +8 -6
  39. metadata +23 -17
  40. data/LICENSE +0 -21
  41. data/lib/gemvault/tar_archive.rb +0 -66
  42. /data/{ASSESSMENT.md → _claude/archives/ASSESSMENT.md} +0 -0
  43. /data/{MACROPLAN.md → _claude/archives/MACROPLAN.md} +0 -0
  44. /data/{docs/superpowers → _claude/archives}/plans/2026-04-16-container-integration-tests.md +0 -0
  45. /data/{docs → _claude/archives}/tarvault-findings.md +0 -0
  46. /data/{docs → _claude/archives}/tarvault.md +0 -0
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 100a593cb7221bf6cc5842ad00c67335809a351905251a6088c3698b41fa9b95
4
- data.tar.gz: 6645dea254fbdd76dbd2fc1de7f8471f27185d205d5a23e038665d060f253cfa
3
+ metadata.gz: 6a1d5d0c990e232ac54514f5837db4224d92b127175f370a3443c8d15d44099f
4
+ data.tar.gz: ab40029793c2dd05206b9812e27da8921eb7fc478164f71fa7485296d5455f15
5
5
  SHA512:
6
- metadata.gz: a973851f5b6b7b9eaa48fcf21a25660bc3770d36c992ff758e4b9ace061a64e5b2cdd6a0270ee335ca3d2a6cf1b80f842b69e6768a98fd85741ec91141786a60
7
- data.tar.gz: 16b6a27438852230e49569493d526efdcf6b07f499137cdcb73a18b29ef06bd89ddf2def281649109bbf1cfbacce1e7e3d4bfdddae44984855fa378e1b79c51f
6
+ metadata.gz: 916b9a114590c9eb5fa3a377d59e9639c33a28e948429b8bac18399473eba0dcacf6711a9808106e3f91348945d71ebd051c1f65fe05239f9804289f56fcf859
7
+ data.tar.gz: 9ea36ee67195ca7f7a8bafd7ebc8658e000b72bcbf72356e3a2266c2f8d2679e0d8f5212f0b977a36a36c95441152f2dabae8ada78c5831770c0cd3498c4253d
data/.containerignore ADDED
@@ -0,0 +1,10 @@
1
+ .bundle
2
+ .git
3
+ gemv
4
+ pkg
5
+ tmp
6
+ vendor
7
+ *.gem
8
+ *.gemv
9
+ references
10
+ references/
data/.rubocop.yml CHANGED
@@ -17,10 +17,11 @@ plugins:
17
17
 
18
18
  AllCops:
19
19
  NewCops: enable
20
- TargetRubyVersion: 4.0
20
+ TargetRubyVersion: 3.4.8
21
21
  Exclude:
22
22
  - bin/*
23
23
  - vendor/**/*
24
+ - references/**/*
24
25
  - lib/core_ext/**/*
25
26
  - rakelib/project.rb
26
27
  - rakelib/project_version.rb
@@ -137,6 +138,13 @@ Style/ArgumentsForwarding:
137
138
  Style/RedundantBegin:
138
139
  Enabled: false
139
140
 
141
+ # Data value objects that carry constants, nested value types, or any class
142
+ # body read best as `class X < Data.define(...)`. The assignment form the cop
143
+ # prefers leaks block-level constants to the enclosing scope and forces a
144
+ # `.freeze` (Style/MutableConstant) on the class. Permitted by David.
145
+ Style/DataInheritance:
146
+ Enabled: false
147
+
140
148
  # Compact class names are fine for small files and tests.
141
149
  Style/ClassAndModuleChildren:
142
150
  Enabled: false
data/CHANGELOG.md CHANGED
@@ -19,10 +19,71 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
19
19
  preserving every gem and timestamp, writing a `.bak` backup by default, with
20
20
  `--dry-run` and `--no-backup` flags. It is a no-op on an already-current vault.
21
21
 
22
+ - All CLI commands accept `vault://` and `file://` locators wherever they take
23
+ a vault path, e.g. `gemvault list vault:///path/to/myvault.gemv`; resolution
24
+ is shared with the RubyGems source via `Gemvault::VaultPath` (issue #9).
25
+
26
+ ### Fixed
27
+ - `bundle plugin install bundler-source-vault` no longer dies with
28
+ `LoadError: cannot load such file -- bundler/plugin/vault_source`. The shim's
29
+ `plugins.rb` now derives the gem root from its own installed location (local
30
+ plugin root, global plugin root, or plain GEM_HOME), registers any spec dirs
31
+ RubyGems is not searching, and resets RubyGems' spec stub cache when it
32
+ predates the just-installed gems (issue #10).
33
+ - `bundle exec` no longer fails Gemfile parsing with `Could not find 'bundler'
34
+ (>= 2.0)` under `path: vendor` on rubies that ship bundler as a regular gem:
35
+ gemvault no longer declares `bundler` as a runtime dependency (issue #12).
36
+ - Vault gems whose versions carry a non-numeric suffix (e.g. `0.2.1.patch1`)
37
+ install from a vault with `gem install --pre`, matching RubyGems' prerelease
38
+ semantics (issue #6).
39
+ - Repeated `bundle install` no longer fails with `cannot load such file --
40
+ bundler/plugin/vault_source` on machines that have gemvault installed
41
+ system-wide. Bundler skips installing a plugin dependency already present on
42
+ the ambient GEM_PATH, so the plugin root holds only the shim; once the app
43
+ bundle is populated Bundler restricts GEM_PATH to it and the ambient copy
44
+ falls out of scope. The shim now locates gemvault's `lib` across every root
45
+ that can hold it — `Gem.default_path` above all, which is what RubyGems knows
46
+ about its own gem roots and therefore covers rubies from rbenv, asdf, chruby,
47
+ Homebrew and distros, none of which export `GEM_HOME` or `GEM_PATH` — and
48
+ loads it via `$LOAD_PATH` rather than gem activation (issue #13).
49
+ - The vault source registers correctly when Bundler evaluates the plugin more
50
+ than once in a process; gemvault is resolved and required only on the first
51
+ evaluation, so `Gemvault::GemEntry` can no longer be defined twice from two
52
+ different gem roots (issue #13).
53
+ - `gemvault new` no longer raises `cannot load such file -- json` on a stock
54
+ distro ruby. json backs the tarball vault's manifest, so every current-format
55
+ vault needs it; it is a default gem upstream but a separate package on
56
+ distros, where `dnf install ruby` leaves it absent. It is now a declared
57
+ runtime dependency.
58
+ - Reading a vault through the Bundler source no longer raises `cannot load such
59
+ file -- json` either. Loading gemvault off `$LOAD_PATH` skips activation, and
60
+ therefore skips its dependencies, so the declared dependency alone did not
61
+ reach the plugin path. The shim now resolves gemvault's declared runtime
62
+ dependencies the same way it resolves gemvault and puts their require paths —
63
+ extension directories included — on `$LOAD_PATH`. A dependency it cannot find
64
+ is skipped rather than fatal, which is what activation could not do.
65
+ - `bundle install` no longer fails with `Could not find 'command_kit' (~> 0.6)`
66
+ when gemvault is installed into the plugin root without its dependencies.
67
+ Loading the vault source no longer activates the gemvault gem, which would
68
+ demand the full runtime dependency set; `vault_source.rb` reaches the rest of
69
+ gemvault through `require_relative` alone (issue #13).
70
+
22
71
  ### Changed
23
72
  - `sqlite3` is no longer a runtime dependency. Gemvault runs dependency-free on
24
73
  the tarball path (including JRuby); `sqlite3` is loaded lazily only to read a
25
74
  legacy SQLite vault, with a clear error if it is not installed.
75
+ - `bundler` is no longer a runtime dependency; the Bundler plugin always runs
76
+ inside an existing Bundler process, and the declared dependency broke gem
77
+ activation in restricted GEM_PATH contexts.
78
+ - CI is unblocked (issue #11): RuboCop's TargetRubyVersion now matches the
79
+ gemspec's `required_ruby_version` floor (3.4.8), the previously-missing
80
+ `spec:integration` rake task exists alongside a non-integration `spec:core`, the
81
+ unit job no longer needs podman, and the integration job installs podman
82
+ when the runner image lacks it.
83
+ - Reading a vault's `manifest.json` is now strict: an entry missing any required
84
+ field (`name`, `version`, `platform`, `created_at`, `sha256`, `encrypted`)
85
+ raises instead of silently loading `nil` fields, so a truncated or hand-edited
86
+ manifest fails fast rather than yielding a subtly broken vault.
26
87
 
27
88
  ### Deprecated
28
89
  - The SQLite vault format is deprecated and now **read-only**: existing SQLite
data/CLAUDE.md CHANGED
@@ -17,6 +17,7 @@ Do NOT modify `.rubocop.yml` or use inline `# rubocop:disable` tags without expl
17
17
  9. ALWAYS write specs first. The workflow is: identify the domain concept (rule 5), write specs describing its behavior, then implement. No implementation without a failing spec.
18
18
  10. Integration specs are the first line of defense for CLI-tool bugs. For any bug reported from using the CLI tool (not the gemvault lib / Ruby API), the FIRST spec you write is an integration spec that reproduces the user's exact invocation — real subprocess, real vault, real exit code. Stub-heavy unit specs are complementary, not sufficient: they prove internal logic produces the expected value assuming surrounding wiring works, but a user's bug report is evidence the wiring didn't work.
19
19
  11. If an integration spec is not catching a reported CLI-tool bug, one of two things is true, and the fix starts by diagnosing which: (a) existing integration specs are not specific enough — extend them to cover the exact scenario before touching production code; or (b) the scenario is not spec'd at all, which means the work is not a bug fix but a new feature — write integration specs for the contract first (per rule 1), then implement.
20
+ 12. NEVER write to /tmp. Use /workspace/tmp
20
21
 
21
22
  ## Additional rules
22
23
 
@@ -25,7 +26,7 @@ Do NOT modify `.rubocop.yml` or use inline `# rubocop:disable` tags without expl
25
26
 
26
27
  ## Project Overview
27
28
 
28
- Multi-gem portable archives backed by SQLite. A single `.gemv` file contains multiple `.gem` files.
29
+ Multi-gem portable archives. A single `.gemv` file is a tarball holding multiple `.gem` files plus a `manifest.json` index; legacy SQLite vaults are read-only.
29
30
 
30
31
  Two gems, one repo:
31
32
 
@@ -56,7 +57,7 @@ gem install --source file:///path/to/myvault.gemv foo
56
57
  ## Architecture
57
58
 
58
59
  - `gemvault.gemspec` — main gem spec (name: `gemvault`)
59
- - `lib/gemvault/vault.rb` — Core vault class (SQLite CRUD for gem blobs + specs)
60
+ - `lib/gemvault/vault.rb` — Vault facade choosing a backend by file format (Tarvault current, legacy Dbvault read-only)
60
61
  - `lib/gemvault/cli.rb` — CLI dispatcher (new/add/list/remove/extract)
61
62
  - `lib/bundler/plugin/vault_source.rb` — Bundler `Plugin::API::Source` implementation
62
63
  - `lib/rubygems_plugin.rb` — RubyGems plugin: monkey-patches for `--source myvault.gemv` support
@@ -69,13 +70,22 @@ gem install --source file:///path/to/myvault.gemv foo
69
70
 
70
71
  ## Key Design Decisions
71
72
 
72
- - SQLite storage — random access, ACID, single file, inspectable with `sqlite3` CLI
73
+ - Tar storage — portable, dependency-free, single file, inspectable with `tar`; legacy SQLite vaults readable via lazily-loaded sqlite3
73
74
  - Specs extracted from gem blobs at runtime (no separate spec storage)
74
75
  - Vault opened/closed per operation in the source plugin (no persistent connection)
75
76
  - `fetch_gemspec_files` checks installed state — Bundler computes `full_gem_path` as `dirname(loaded_from)`, so `loaded_from` must point inside the gem directory
76
77
  - `bundler-source-vault` name exists because Bundler auto-infers plugin name from `type: :vault` → `bundler-source-vault`
77
78
  - `file://` URIs stripped to plain paths in `Gem::Source::Vault#initialize`
78
79
  - Verbose logging via `Gem::UserInteraction#verbose` for `--verbose` support
80
+ - `shim/plugins.rb` loads gemvault by putting its `lib` on `$LOAD_PATH`, never by
81
+ activating the gem. Bundler skips installing a plugin dependency already
82
+ present on the ambient GEM_PATH, so the plugin root often lacks gemvault, and
83
+ by load time GEM_PATH is restricted to the plugin root and the app bundle. The
84
+ shim therefore searches every root that can hold gemvault, including the ones
85
+ Bundler masked (`Bundler.original_env` GEM_HOME/GEM_PATH). Activation is
86
+ avoided because it demands gemvault's full runtime dependency set (command_kit),
87
+ which Bundler skips for the same reason — `vault_source.rb` reaches the rest of
88
+ gemvault through `require_relative` alone and needs none of it.
79
89
 
80
90
  ## Testing
81
91
 
@@ -83,17 +93,66 @@ gem install --source file:///path/to/myvault.gemv foo
83
93
  bundle exec rake test
84
94
  ```
85
95
 
86
- - `test/vault_test.rb` 33 unit tests for Vault class
87
- - `test/vault_source_test.rb` 17 unit tests for Bundler source plugin
88
- - `test/integration_test.rb` — 12 end-to-end bundle install tests
89
- - `test/cli_test.rb` — 32 CLI tests
90
- - `test/rubygems_plugin_test.rb` — 28 tests (source, resolver, monkey-patches, gem install integration, file:// URI, verbose logging)
96
+ Minitest covers the library; RSpec covers the CLI and the containerized
97
+ integration suite. `rake` (the default task) runs `test`, `spec` and `rubocop`.
91
98
 
92
- Integration tests use a manually-written Bundler plugin index to avoid rubygems.org resolution during testing.
99
+ ```bash
100
+ bundle exec rake test # minitest only
101
+ bundle exec rake spec:core # rspec, no containers
102
+ bundle exec rake spec:integration # rspec, containers (builds the image first)
103
+ bundle exec rake spec:build # rebuild the container image
104
+ bundle exec rake spec:teardown # remove it
105
+ ```
106
+
107
+ - `test/vault_test.rb` — unit tests for Vault class
108
+ - `test/vault_source_test.rb` — unit tests for Bundler source plugin
109
+ - `test/cli_test.rb` — CLI tests
110
+ - `test/rubygems_plugin_test.rb` — source, resolver, monkey-patches, gem install integration, file:// URI, verbose logging
111
+ - `spec/integration/` — end-to-end specs, each run inside a podman container
112
+ - `spec/support/` — script fragments the integration specs assemble into those containers
113
+
114
+ Integration specs serve the tree's own gems from a local gem index (`GemIndex`)
115
+ to avoid rubygems.org resolution during testing.
116
+
117
+ ### Container fidelity — do not undo these
118
+
119
+ The container has to look like a machine a user actually has. The `ruby` base
120
+ image does not, and every way it differs has already hidden a real defect. These
121
+ four are load-bearing; reverting any of them silently makes the suite green
122
+ against something other than the code under test:
123
+
124
+ 1. **`bundler-source-vault` is NOT installed system-wide in `Dockerfile.test`.**
125
+ An ambient copy satisfies Bundler's plugin resolution without ever populating
126
+ the plugin root. Specs resolve the shim from the local gem index (`GemIndex`)
127
+ so they exercise the tree's shim. `gemvault` IS installed system-wide on
128
+ purpose — that is a real user's machine and the trigger for issue #13.
129
+ 2. **`BUNDLE_APP_CONFIG` is not set in the container.** The `ruby` image sets it
130
+ to `/usr/local/bundle`, which moves `Bundler::Plugin.root` out of the project
131
+ so no spec ever touches a project-local `.bundle/plugin`. Fedora sets it
132
+ nowhere; do not add it, and do not adopt a base image that does.
133
+ 3. **`GEM_HOME` and `GEM_PATH` are not exported in the container, and nothing
134
+ pins them.** Gems land in RubyGems' own default dirs. The `ruby` image
135
+ exports `GEM_HOME`; rubies from rbenv, asdf, chruby, Homebrew and distros
136
+ export nothing. Anything that reads gem roots back out of the environment
137
+ finds them on the image and finds nothing on a user's machine — that is what
138
+ made the first fix for issue #13 pass its specs while still failing for the
139
+ reporter. Neither `Dockerfile.test` nor `ContainerHelper#podman_run` should
140
+ grow an `ENV`/`-e` for either variable.
141
+ 4. **Bundler is pinned to the version users run, not the image's default gem.**
142
+ The plugin machinery under test is Bundler's own, so a stale default silently
143
+ tests different code.
144
+
145
+ Fidelity is a property of the image, not of a script fragment: integration specs
146
+ run only commands a user would actually type.
147
+
148
+ If a spec fails only after removing a system-installed gem or an exported
149
+ variable, the spec was passing for the wrong reason — fix the spec, not the
150
+ container.
93
151
 
94
152
  ## Dependencies
95
153
 
96
- - `bundler` (>= 2.0) runtime
154
+ - `bundler` NOT a dependency; the plugin always runs inside an existing Bundler process, and declaring it breaks gem activation under `bundle exec`'s restricted GEM_PATH
97
155
  - `command_kit` (~> 0.6) — runtime (CLI)
156
+ - `json` (~> 2.0) — runtime; backs the tarball vault's manifest. A default gem upstream, but distros package it separately, so it is declared. Because the shim loads gemvault off `$LOAD_PATH` without activating it, it has to put json's require paths on `$LOAD_PATH` itself — see `dependency_specs` and `entries` in `shim/gemvault_load_path.rb`.
98
157
  - `sqlite3` (~> 2.0) — NOT a runtime dependency; loaded lazily only to read a legacy SQLite (Dbvault) vault. Declared in the Gemfile for development/test.
99
- - `minitest`, `rake` — development
158
+ - `minitest`, `rspec`, `rake` — development
data/Dockerfile.test CHANGED
@@ -1,15 +1,70 @@
1
- FROM docker.io/library/ruby:4.0.1-slim
1
+ # A dev machine, not a packaging convenience.
2
+ #
3
+ # The official ruby image is not shaped like anything a user runs: it redirects
4
+ # gems to /usr/local/bundle, exports GEM_HOME and BUNDLE_APP_CONFIG to match,
5
+ # and freezes bundler at whatever shipped. Each of those differences hid a real
6
+ # defect, and patching them one at a time only moved the blind spot. A distro
7
+ # ruby has none of them: gems land in RubyGems' own default dirs, nothing is
8
+ # exported, and bundler is a regular gem rather than a default one -- which is
9
+ # also the shape that broke `bundle exec` in issue #12.
10
+ FROM docker.io/library/fedora:44
2
11
 
3
- COPY . /tmp/build/
12
+ # redhat-rpm-config carries the hardening specs Fedora's ruby-devel compiles
13
+ # extensions against; without it every native gem fails in extconf. Any Fedora
14
+ # machine that can build a gem has it.
15
+ RUN dnf -y install --setopt=install_weak_deps=False \
16
+ ruby ruby-devel rubygem-bundler \
17
+ gcc make redhat-rpm-config sqlite-devel tar \
18
+ && dnf clean all
4
19
 
5
- RUN cd /tmp/build \
6
- && gem build gemvault.gemspec \
7
- && gem install --no-document gemvault-*.gem \
8
- && cd shim \
9
- && gem build bundler-source-vault.gemspec \
10
- && gem install --no-document bundler-source-vault-*.gem \
11
- && rm -rf /tmp/build
20
+ # Keep this in step with the `BUNDLED WITH` line the project's own lockfile
21
+ # carries. The plugin machinery these specs exercise is Bundler's own, so a
22
+ # stale version silently tests different code.
23
+ RUN gem install --no-document bundler -v 4.0.17
12
24
 
13
25
  # sqlite3 is not a gemvault runtime dependency; install it here so the upgrade
14
- # integration test can build a legacy SQLite (Dbvault) fixture to migrate.
26
+ # integration test can read the committed legacy SQLite (Dbvault) fixture it
27
+ # migrates.
15
28
  RUN gem install --no-document sqlite3
29
+
30
+ # Gem files for integration specs that install the plugin from a local gem
31
+ # index at spec runtime, kept for offline use: command_kit and json are
32
+ # gemvault's runtime dependencies, and `bundle plugin install` resolves them
33
+ # against the Gemfile's sources with GEM_PATH already narrowed to the plugin
34
+ # root, so an ambient copy does not satisfy it -- every declared dependency has
35
+ # to be servable from the index. The bundler gem lets specs reinstall bundler
36
+ # in place.
37
+ RUN mkdir -p /opt/gems \
38
+ && cd /opt/gems \
39
+ && gem fetch command_kit -v 0.6.0 \
40
+ && gem fetch json -v "~> 2.0" \
41
+ && gem fetch bundler -v "$(ruby -rbundler -e 'print Bundler::VERSION')"
42
+
43
+ # The tree arrives as a bind mount at /src rather than a COPY: the build context
44
+ # lives on a filesystem whose extended-attribute listing podman's copier cannot
45
+ # read. `rake spec:build` supplies both the mount and SOURCE_DIGEST.
46
+ #
47
+ # A bind mount's contents are not part of podman's cache key, so without
48
+ # SOURCE_DIGEST in the command line this layer is reused forever and the image
49
+ # keeps whichever gemvault the tree held the first time it was built -- the
50
+ # suite would then run green against code that is not under test. The digest
51
+ # changes whenever the installed source does, which invalidates the layer. It is
52
+ # last so that editing lib/ does not re-fetch the layers above.
53
+ #
54
+ # gemvault is installed system-wide because integration specs need the CLI to
55
+ # build vault fixtures, and because that is what a real user's machine looks
56
+ # like: `gem install gemvault` puts it on the ambient GEM_PATH. That ambient
57
+ # copy is load-bearing for the plugin root specs -- Bundler skips installing a
58
+ # plugin dependency that is already installed, so the plugin root only gets a
59
+ # complete dependency set if gemvault puts it there itself.
60
+ #
61
+ # bundler-source-vault is deliberately NOT installed system-wide. An ambient
62
+ # copy satisfies Bundler's plugin resolution without ever populating the plugin
63
+ # root, which masks every plugin-root defect the integration specs exist to
64
+ # catch. Specs install it from the local gem index built by GemIndex instead.
65
+ ARG SOURCE_DIGEST=unset
66
+ RUN test "$SOURCE_DIGEST" != unset \
67
+ && cd /src \
68
+ && gem build gemvault.gemspec -o /tmp/gemvault.gem \
69
+ && gem install --no-document /tmp/gemvault.gem \
70
+ && rm -f /tmp/gemvault.gem
data/README.md CHANGED
@@ -1,10 +1,8 @@
1
1
  # Gemvault
2
2
 
3
- A gem server in a file. No HTTP. No infrastructure.
3
+ A file that behaves like a private gem server.
4
4
 
5
- A `.gemv` file is a self-contained archive that contains Ruby gems. Commit it to your repo, drop it on S3, email it, put it on a USB drive Bundler and RubyGems read from it directly. Private gems without running a server.
6
-
7
- New vaults are portable tarballs (no runtime dependencies, works on JRuby); the original SQLite format is still read transparently. See [Vault format versioning](#vault-format-versioning).
5
+ You can commit it to your repo, drop it on S3, email it, put it on a USB drive, or whatever you feel like doing. Bundler and RubyGems read from it directly.
8
6
 
9
7
  ## Installation
10
8
 
@@ -29,7 +27,6 @@ end
29
27
  bundle install
30
28
  ```
31
29
 
32
- Bundler auto-discovers the `bundler-source-vault` plugin, installs it, and resolves gems from the vault alongside rubygems.org. No extra configuration.
33
30
 
34
31
  ### RubyGems CLI
35
32
 
@@ -67,12 +64,7 @@ gemvault upgrade myvault.gemv --no-backup # skip the default myvault.gemv.bak c
67
64
 
68
65
  ## How It Works
69
66
 
70
- A `.gemv` file is a self-contained archive of gem metadata and raw `.gem` blobs. A current-format (tarball) vault has a `manifest.json` first entry followed by the `.gem` files, inspectable with standard tools:
71
-
72
- ```bash
73
- tar -tf myvault.gemv # tarball (v2) vault
74
- sqlite3 myvault.gemv "SELECT name, version, platform FROM gems" # SQLite (v1) vault
75
- ```
67
+ The gemv file is a tarball containing your .gem files and a json manifest. It has no dependencies other than the tar utilities that rubygems provides.
76
68
 
77
69
  When Bundler sees `type: :vault` in your Gemfile, it auto-installs the `bundler-source-vault` plugin from rubygems.org. The plugin implements the `Bundler::Plugin::API::Source` interface — it reads gemspecs from the vault, participates in dependency resolution, then extracts and installs gems from the vault's blob storage.
78
70
 
data/Rakefile CHANGED
@@ -10,20 +10,43 @@ require "rspec/core/rake_task"
10
10
 
11
11
  RSpec::Core::RakeTask.new(:spec)
12
12
 
13
+ RSpec::Core::RakeTask.new("spec:core") do |t|
14
+ t.rspec_opts = "--tag ~integration"
15
+ end
16
+
17
+ RSpec::Core::RakeTask.new("spec:integration") do |t|
18
+ t.rspec_opts = "--tag integration"
19
+ end
20
+
13
21
  require "rubocop/rake_task"
14
22
  RuboCop::RakeTask.new
15
23
 
16
24
  require "gempilot/version_task"
17
25
  Gempilot::VersionTask.new
18
26
 
19
- CACHED_IMAGE = "gemvault-test:latest".freeze
27
+ require "digest"
28
+ require_relative "spec/support/container_helper"
29
+
30
+ CACHED_IMAGE = ContainerHelper::CACHED_IMAGE
31
+
32
+ # Dockerfile.test reads the tree through a bind mount, whose contents podman
33
+ # does not fold into its layer cache key. Feeding this digest in as a build arg
34
+ # is what makes `rake spec:build` notice that the source changed.
35
+ def image_source_digest
36
+ fingerprints = FileList["lib/**/*", "exe/*", "gemvault.gemspec"]
37
+ .select { |path| File.file?(path) }.sort
38
+ .map { |path| "#{path} #{Digest::SHA256.hexdigest(File.binread(path))}" }
39
+ Digest::SHA256.hexdigest(fingerprints.join("\n"))
40
+ end
20
41
 
21
42
  def cached_image_exists?
22
43
  system("podman", "image", "exists", CACHED_IMAGE, out: File::NULL, err: File::NULL)
23
44
  end
24
45
 
25
46
  def build_cached_image
26
- sh "podman", "build", "--network=host", "-t", CACHED_IMAGE, "-f", "Dockerfile.test", "."
47
+ sh "podman", "build", "--network=host", "-v", "#{__dir__}:/src:ro,z",
48
+ "--build-arg", "SOURCE_DIGEST=#{image_source_digest}",
49
+ "-t", CACHED_IMAGE, "-f", "Dockerfile.test", "."
27
50
  end
28
51
 
29
52
  def destroy_cached_image
@@ -59,6 +82,7 @@ namespace :shim do
59
82
  end
60
83
 
61
84
  Rake::Task[:spec].enhance ["spec:setup"]
85
+ Rake::Task["spec:integration"].enhance ["spec:setup"]
62
86
  Rake::Task[:build].enhance ["shim:build"]
63
87
  Rake::Task[:release].enhance ["shim:release"]
64
88
  Rake::Task[:clobber].enhance ["spec:teardown"]
@@ -0,0 +1,197 @@
1
+ # Issue #13 — `cannot load such file -- bundler/plugin/vault_source`
2
+
3
+ Line references are to the vendored RubyGems/Bundler checkout in
4
+ `references/rubygems` (`bundler-v4.0.0-1207-gf912ac9b0b`).
5
+
6
+ ## Symptom
7
+
8
+ ```
9
+ ❯ bundle install
10
+ Fetching gem metadata from https://rubygems.org/.......
11
+ Resolving dependencies...
12
+ Installing bundler-source-vault 0.2.0
13
+
14
+ [!] There was an error parsing `Gemfile`: cannot load such file -- bundler/plugin/vault_source. Bundler cannot continue.
15
+
16
+ # from /Users/davidgillis/repos/reversal-store/Gemfile:19
17
+ > source "vendor/vendored.gemv", type: :vault do
18
+ ```
19
+
20
+ Reported as permanent. `gemvault doctor`, `rm -rf .bundle/plugin`, `rm -f
21
+ Gemfile.lock`, and the two combined all left it unchanged.
22
+
23
+ ## Reproduction
24
+
25
+ The reporter's `.bundle` tarball showed a plugin root holding the shim and
26
+ nothing else:
27
+
28
+ ```
29
+ .bundle/plugin/cache/bundler-source-vault-0.2.0.gem
30
+ .bundle/plugin/gems/bundler-source-vault-0.2.0/plugins.rb
31
+ .bundle/plugin/specifications/bundler-source-vault-0.2.0.gemspec
32
+ .bundle/plugin/index
33
+ ```
34
+
35
+ No `gemvault` anywhere. Reproduced exactly with: gemvault installed on the
36
+ ambient `GEM_PATH`, the shim not installed, a project-local `.bundle` with a
37
+ configured bundle path, and `bundle install` run **twice**.
38
+
39
+ ## Root cause
40
+
41
+ Three independent mechanisms compose. Each is fine alone.
42
+
43
+ ### 1. Bundler never puts gemvault in the plugin root
44
+
45
+ `Source::Rubygems#install` (`lib/bundler/source/rubygems.rb:206`) short-circuits:
46
+
47
+ ```ruby
48
+ if (spec.default_gem? && !cached_built_in_gem(...)) || (installed?(spec) && !options[:force])
49
+ print_using_message "Using #{version_message(spec, options[:previous_spec])}"
50
+ return nil
51
+ end
52
+ ```
53
+
54
+ `installed?` (`:484`) consults `installed_specs`. On any machine where
55
+ `gem install gemvault` has run — that is, anyone who uses the CLI — the ambient
56
+ copy satisfies it. Bundler prints "Using gemvault" and the plugin root receives
57
+ the shim alone. Nothing is downloaded, so the plugin root's `cache/` has no
58
+ gemvault either, which is exactly what the reporter's tarball showed.
59
+
60
+ ### 2. The ambient copy disappears precisely when it is needed
61
+
62
+ `Plugin::Installer#install_all_sources` calls
63
+ `Bundler.configure_gem_home_and_path(Plugin.root)`
64
+ (`lib/bundler/plugin/installer.rb:100`), and `configure_gem_path`
65
+ (`lib/bundler.rb:661`) does:
66
+
67
+ ```ruby
68
+ unless use_system_gems?
69
+ Bundler::SharedHelpers.set_env "GEM_PATH", ""
70
+ end
71
+ ```
72
+
73
+ With a configured bundle path, `use_system_gems?` is false, so `GEM_PATH`
74
+ becomes empty and `Gem.path` collapses to the bundle. The copy that made step 1
75
+ "safe" is now out of scope.
76
+
77
+ This is why the failure is **permanent and why it starts on the second run**.
78
+ The first `bundle install` succeeds because the ambient gemvault is still
79
+ visible. It populates the bundle path. From then on every `bundle install`
80
+ narrows `GEM_PATH`, and every recovery step the reporter tried rebuilds the same
81
+ incomplete plugin root.
82
+
83
+ ### 3. Locating gemvault is not sufficient — activation cannot survive here
84
+
85
+ The obvious repair — find gemvault and load it — has a trap. Loading it via
86
+ `$LOAD_PATH` skips activation, and therefore skips its dependencies. gemvault
87
+ reads a vault manifest through `json` (`lib/gemvault/manifest.rb:1`), so the
88
+ plugin dies later, at first vault read, instead of at load.
89
+
90
+ Activating instead does not work, for two separate reasons:
91
+
92
+ - **The path would be stripped one line before it is used.** `Runtime#setup`
93
+ (`lib/bundler/runtime.rb:12-38`) runs `clean_load_path` at line 16 and
94
+ `@definition.specs_for(groups)` at line 18 — and resolution is what asks the
95
+ vault source for specs. `clean_load_path`
96
+ (`lib/bundler/shared_helpers.rb:371-380`) rejects a `$LOAD_PATH` entry only
97
+ when `loaded_gem_paths.delete(p)` is truthy, and `loaded_gem_paths`
98
+ (`lib/bundler/rubygems_integration.rb:132-135`) is built from
99
+ `Gem.loaded_specs` — i.e. **activated gems**. Activating is what would put the
100
+ path on the chopping block. A directory pushed without activation is absent
101
+ from that list, so `delete` returns nil and the entry survives.
102
+
103
+ - **`Gem::Specification` cannot see outside the bundle.** `stub_rubygems`
104
+ (`lib/bundler/rubygems_integration.rb:336-340`) sets
105
+ `Gem::Specification.all = specs` and reapplies it through `Gem.post_reset`, so
106
+ under `bundle exec` `find_by_name` cannot find a gem installed outside the
107
+ bundle, and `Gem::Specification.reset` will not restore it.
108
+
109
+ ## Why the test suite could not catch any of this
110
+
111
+ The integration container was the official `ruby` image. It differs from a
112
+ machine anyone develops on in four ways, and each one hid one layer:
113
+
114
+ | Image behaviour | What it hid |
115
+ |---|---|
116
+ | `gemvault` **and** `bundler-source-vault` installed system-wide | `require "bundler/plugin/vault_source"` resolved from `/usr/local/bundle` no matter what the plugin root held |
117
+ | `BUNDLE_APP_CONFIG=/usr/local/bundle` | `Bundler::Plugin.root` was never project-local, so no spec touched a real `.bundle/plugin` |
118
+ | `GEM_HOME` exported | a first repair that read gem roots out of the environment appeared to work |
119
+ | `json` present as a default gem | the undeclared dependency was invisible |
120
+
121
+ The first repair for this issue reconstructed gem roots from
122
+ `Bundler.original_env`, which is a snapshot of environment variables.
123
+ `GEM_HOME`/`GEM_PATH` are only present there if something exported them —
124
+ rbenv, asdf, chruby, Homebrew and distro rubies export neither. The reporter is
125
+ on rbenv (`issues.rec`: `/Users/davidgillis/.rbenv/versions/4.0.1/`,
126
+ `rbenv 1.3.2`), so that repair passed its specs in Docker and still failed on
127
+ their machine.
128
+
129
+ The suite now runs on `fedora:44` with a distro ruby: gems in RubyGems' own
130
+ default dirs, nothing exported, bundler a regular gem. Moving to it immediately
131
+ surfaced two further real defects — the undeclared `json` dependency, and the
132
+ `json/ext/parser` extension path.
133
+
134
+ ## The fix
135
+
136
+ `shim/gemvault_load_path.rb` locates gemvault and its declared runtime
137
+ dependencies across every root that could hold them, and pushes their require
138
+ paths onto `$LOAD_PATH`.
139
+
140
+ - **`Gem.default_path` is the load-bearing root.** It is what RubyGems knows
141
+ about its own gem roots, independent of the narrowing in step 2, so it covers
142
+ rubies that export nothing. `Bundler.original_env` is still consulted, for the
143
+ rubies that do export a root outside those defaults, such as RVM.
144
+ - **`$LOAD_PATH`, not activation**, for the reasons in step 3.
145
+ - **Dependencies resolved by hand**, using `full_require_paths`
146
+ (`lib/rubygems/basic_specification.rb:172-183`), which appends `extension_dir`
147
+ when `have_extensions?` — without it `json/ext/parser` is unfindable, since
148
+ json's compiled half lives outside its gem directory.
149
+ - **A dependency that cannot be found is skipped rather than fatal.** That is
150
+ the one thing activation could not do: it would abort on `command_kit`, which
151
+ only the CLI needs.
152
+ - **`json` is now a declared runtime dependency**, so `gemvault new` works on a
153
+ stock distro ruby.
154
+
155
+ ## Does the suite actually protect this?
156
+
157
+ Green is not evidence on its own — the whole reason this bug shipped is that a
158
+ green suite was being satisfied by an ambient gem. So the two pieces of the fix
159
+ were checked by mutation:
160
+
161
+ | Mutation | Result |
162
+ |---|---|
163
+ | `Gem.default_path` dropped from the searched roots | 18 examples fail |
164
+ | the shim's pinned gemvault requirement ignored, so the newest wins | both examples of the upgrade-residue scenario fail |
165
+
166
+ Restoring each returns the suite to green.
167
+
168
+ ## Platforms verified
169
+
170
+ | Platform | Status |
171
+ |---|---|
172
+ | CRuby 4.0.6, Fedora 44, aarch64 | Full suite, 265 examples |
173
+ | JRuby 10.1.1.0 (the reporter runs 10.1.0.0) | Verified once by hand, **not covered by the suite**: vault create/add/list, `bundle install` twice against a vault source, `bundle exec require`. `json 2.21.1 (java)` installed as a platform gem and resolved, which is the case `full_require_paths` has to get right |
174
+ | macOS `arm64-darwin-24` | **Not verified.** The reporter's own platform. Nothing here can run it |
175
+
176
+ JRuby has no permanent coverage because JRuby caps its own heap from the cgroup
177
+ and the development sandbox is small enough that the cap lands at 148 MB, where
178
+ even a single `gem fetch` dies. The one successful hand-run above happened in a
179
+ lucky memory window and did not reproduce. A JRuby image, rake task and spec
180
+ were written and then reverted rather than shipped unverified: harness code
181
+ nobody has watched pass is the same defect as a spec that cannot fail. On a
182
+ machine with ordinary memory this is worth adding, and it is the highest-value
183
+ gap left, since issue #13 was reported against JRuby as well as CRuby.
184
+
185
+ ## Coverage
186
+
187
+ `spec/support/vault_sourced_gemfile_examples.rb` applies "a complete bundle" to
188
+ a vault-sourced Gemfile and then re-applies it after each user action: deleting
189
+ `Gemfile.lock`, deleting `.bundle`, deleting both, removing the vaulted gem,
190
+ adding a gem from rubygems, adding the project's own gemspec (the reporter's
191
+ Gemfile shape), adding another gem from the same vault, and adding one from a
192
+ second vault. `spec/integration/vaulted_project_spec.rb` runs that whole
193
+ group under four bundler configurations — stock, an install path chosen, that
194
+ path undone, and the project's gems cached — and
195
+ `spec/integration/no_ambient_gemvault_spec.rb` covers the machine that has no
196
+ gemvault of its own, where the plugin root has to carry the whole dependency
197
+ set.