namespaced-gem 0.1.0.pre

data/ISSUE.md

@@ -0,0 +1,198 @@
+# gem.coop namespace servers missing legacy Marshal API endpoints
+
+## Summary
+
+`beta.gem.coop` namespace servers implement only the **Compact Index** API
+(used by Bundler) and the `/gems/` download endpoint, but not the **legacy
+Marshal API** (`quick/Marshal.4.8/`). The
+[namespaced-gem](https://gitlab.com/galtzo-floss/namespaced-gem) plugin works
+around this by synthesizing `Gem::Specification` objects from Compact Index data
+(via `ApiSpecPatch`), so **both `bundle install` and `gem install` now work**.
+
+A secondary issue: the production `gem.coop` server returns **HTTP 200 with
+body `"404"`** for namespace endpoints, instead of a proper HTTP 404 status
+code.
+
+---
+
+## Context
+
+The `namespaced-gem` RubyGems plugin enables gemspec dependencies to be declared
+as full URIs (e.g. `spec.add_dependency "https://beta.gem.coop/@kaspth/oaken"`).
+It patches both Bundler and RubyGems' native resolver to parse these URIs,
+derive the namespace source URL, and resolve against it.
+
+Each namespace (e.g. `https://beta.gem.coop/@kaspth/`) is treated as its own
+**discrete gem server** — completely independent of the root server or any
+other namespace.
+
+---
+
+## What works: Compact Index (Bundler)
+
+The Compact Index endpoints are served correctly under namespace paths.
+Bundler uses **only** these two endpoints, so `bundle install` / `bundle lock`
+works today.
+
+| Endpoint | URL | Status |
+|---|---|---|
+| `versions` | `https://beta.gem.coop/@kaspth/versions` | ✅ 200 — returns gem listing |
+| `info/{gem}` | `https://beta.gem.coop/@kaspth/info/oaken` | ✅ 200 — returns per-version dependency data |
+
+---
+
+## What's missing: Legacy Marshal API (`gem install`)
+
+When `gem install` resolves a dependency, it uses RubyGems' native resolver
+(`Gem::DependencyInstaller` → `Gem::Resolver::InstallerSet`). The flow is:
+
+1. **Discovery** — `Gem::Source#dependency_resolver_set` probes
+   `{source}/versions`. Since that returns 200, it creates an `APISet` backed
+   by `{source}/info/`. This step **succeeds** — the gem and its versions are
+   found via the Compact Index.
+
+2. **Spec fetch** — `InstallerSet#add_always_install` calls
+   `APISpecification#spec`, which calls `Gem::Source#fetch_spec`. This method
+   constructs the URL:
+
+   ```
+   {source}/quick/Marshal.4.8/{gem}-{version}.gemspec.rz
+   ```
+
+   and expects a **deflated (`Zlib`) Marshal-serialized `Gem::Specification`**
+   in response. This endpoint **returns 404**.
+
+3. **Gem download** — After resolution, RubyGems downloads the `.gem` file
+   from:
+
+   ```
+   {source}/gems/{gem}-{version}.gem
+   ```
+
+   This endpoint also **returns 404**.
+
+4. **Crash** — The 404 HTML body is passed to `Zlib::Inflate.inflate`, which
+   raises `Zlib::DataError: incorrect header check`.
+
+### Endpoints that return 404 (all under the namespace path)
+
+| Endpoint | URL | Expected response |
+|---|---|---|
+| `quick/Marshal.4.8/{gem}-{ver}.gemspec.rz` | `https://beta.gem.coop/@kaspth/quick/Marshal.4.8/oaken-2.5.1.gemspec.rz` | Deflated Marshal-serialized `Gem::Specification` |
+| `gems/{gem}-{ver}.gem` | `https://beta.gem.coop/@kaspth/gems/oaken-2.5.1.gem` | The `.gem` file |
+| `specs.4.8.gz` | `https://beta.gem.coop/@kaspth/specs.4.8.gz` | Gzipped Marshal array of all `[name, version, platform]` tuples |
+| `latest_specs.4.8.gz` | `https://beta.gem.coop/@kaspth/latest_specs.4.8.gz` | Gzipped Marshal array of latest `[name, version, platform]` tuples |
+
+**All of these must be served under the namespace path** (e.g.
+`https://beta.gem.coop/@kaspth/quick/…`, not `https://beta.gem.coop/quick/…`),
+because each namespace is its own discrete gem server.
+
+The minimum required for `gem install` to work are:
+
+1. **`quick/Marshal.4.8/{gem}-{ver}.gemspec.rz`** — the gemspec, serialized
+   with `Marshal.dump` then compressed with `Zlib::Deflate.deflate`.
+2. **`gems/{gem}-{ver}.gem`** — the `.gem` package file for download.
+
+---
+
+## Reproduction
+
+### Direct install
+
+```bash
+# Requires Ruby >= 3.2, RubyGems >= 4.0.5
+gem install namespaced-gem   # install the plugin first
+
+gem install @kaspth/oaken    # shorthand (defaults to https://beta.gem.coop)
+# => ERROR: Zlib::DataError — incorrect header check
+
+gem install https://beta.gem.coop/@kaspth/oaken   # full URI
+# => ERROR: Zlib::DataError — incorrect header check
+```
+
+### Transitive dependency (Use Case 1)
+
+Even when the plugin is already installed, `gem install` of a gem whose gemspec
+contains URI dependencies also fails:
+
+```bash
+gem install namespaced-gem   # plugin loaded on next boot
+
+gem install my-gem           # my-gem.gemspec has:
+                             #   spec.add_dependency "https://beta.gem.coop/@kaspth/oaken", "~> 1.0"
+```
+
+The resolution phase succeeds — the `InstallerSetPatch#find_all` intercept
+correctly remaps the URI dep and queries the compact index. But the
+**installation phase** fails because RubyGems downloads `.gem` files from
+`{source}/gems/{name}-{version}.gem`, which returns 404.
+
+Additionally, there is a **chicken-and-egg problem** for first-time installs:
+if `namespaced-gem` is listed as a dependency of `my-gem` (rather than being
+pre-installed), the plugin is not loaded when `gem install my-gem` starts —
+because RubyGems loads `rubygems_plugin.rb` files only from _already installed_
+gems at boot. In this case, RubyGems encounters the URI dependency string
+without the patches in place and tries to look it up as a literal gem name on
+rubygems.org, failing immediately.
+
+### Full stack trace (direct install)
+
+```
+ERROR:  While executing gem ... (Zlib::DataError)
+    incorrect header check
+        .../rubygems/util.rb:47:in 'Zlib::Inflate.inflate'
+        .../rubygems/util.rb:47:in 'Gem::Util.inflate'
+        .../rubygems/source.rb:132:in 'Gem::Source#fetch_spec'
+        .../rubygems/resolver/api_specification.rb:93:in 'Gem::Resolver::APISpecification#spec'
+        .../rubygems/resolver/installer_set.rb:99:in 'Gem::Resolver::InstallerSet#add_always_install'
+        .../rubygems/dependency_installer.rb:243:in 'Gem::DependencyInstaller#resolve_dependencies'
+        .../rubygems/commands/install_command.rb:198:in 'Gem::Commands::InstallCommand#install_gem'
+        ...
+```
+
+---
+
+## Impact: which code paths work today
+
+| Scenario | API used | Works? |
+|---|---|---|
+| `bundle lock` / `bundle install` with URI deps in gemspec | Compact Index only | ✅ |
+| `bundler/inline` with a namespace source block | Compact Index only | ✅ |
+| `gem install @kaspth/oaken` (direct, plugin pre-installed) | Compact Index + gem download | ✅ |
+| `gem install my-gem` (transitive URI deps, plugin pre-installed) | Compact Index + gem download | ✅ |
+| `gem install my-gem` (transitive URI deps, plugin NOT pre-installed) | N/A — plugin not loaded | ❌ |
+
+All code paths now work when the `namespaced-gem` plugin is installed, thanks
+to `ApiSpecPatch` (which synthesizes specs from Compact Index data, bypassing
+the missing `quick/Marshal.4.8/` endpoint) and the namespace server's `/gems/`
+download endpoint. The only remaining failure case is the cold-start scenario
+where the plugin is not yet installed — RubyGems loads plugins only from
+already-installed gems at boot.
+
+---
+
+## Separate issue: `gem.coop` (production) returns HTTP 200 with body `"404"`
+
+The production server at `gem.coop` (not `beta.gem.coop`) returns **HTTP 200**
+with a plain-text body of `"404"` for namespace endpoints:
+
+```
+GET https://gem.coop/@kaspth/versions     → 200, body: "404"
+GET https://gem.coop/@kaspth/info/oaken   → 200, body: "404"
+```
+
+This is problematic because RubyGems interprets an HTTP 200 response as a
+successful Compact Index reply. It creates an `APISet` and attempts to parse
+the string `"404"` as version data, leading to confusing downstream failures.
+
+These should return a proper **HTTP 404** status code so that RubyGems raises
+`Gem::RemoteFetcher::FetchError` and falls back gracefully.
+
+---
+
+## Environment
+
+- Ruby 4.0.1
+- RubyGems 4.0.5+
+- `namespaced-gem` (development, HEAD)
+- Tested: 2026-03-07

data/README.md

@@ -0,0 +1,369 @@
+# 🔌 namespaced-gem
+
+A RubyGems plugin that enables gemspec dependencies to be declared as
+full URIs, pointing to **namespaced gem sources** such as
+[gem.coop namespaces](https://gem.coop/updates/5/).
+
+This implements the ideas discussed in
+[gem-coop/gem.coop#12](https://github.com/gem-coop/gem.coop/issues/12).
+
+---
+
+## The Problem
+
+gem.coop's public beta introduced _namespaces_ — isolated gem registries per
+user or organization:
+
+```
+https://beta.gem.coop/@myspace          # namespace index
+https://beta.gem.coop/@myspace/my-gem   # canonical gem URI
+```
+
+Today, gemspecs declare dependencies as plain names:
+
+```ruby
+spec.add_dependency "rack", "~> 3.0"
+```
+
+There is no standard way to express _which gem server_ (and _which namespace_)
+a dependency comes from inside the gemspec itself. The user must manually add
+a `source` block to their Gemfile — which defeats the purpose of publishing a
+self-describing gemspec.
+
+The question this prototype asks: **can a gemspec declare its own source for a
+dependency, using the dependency name string alone?**
+
+```ruby
+spec.add_dependency "https://beta.gem.coop/@myspace/my-gem", "~> 1.0"
+
+# or using a Package URL (purl-spec):
+spec.add_dependency "pkg:gem/@myspace/my-gem?repository_url=https://beta.gem.coop", "~> 1.0"
+```
+
+---
+
+## Key Finding: RubyGems 4.0.5+ Happens to Allow URI Names
+
+RubyGems 4.0.5 removed the old `Gem::Dependency::VALID_NAME_PATTERN`
+restriction entirely — any String is now accepted as a dependency name:
+
+```ruby
+dep = Gem::Dependency.new("https://beta.gem.coop/@ns/foo", "~> 1.0")
+dep.name  # => "https://beta.gem.coop/@ns/foo"
+```
+
+However, **RubyGems has no idea what to do with a URI-named dependency.** It
+will happily store the string, but `gem install` will try to look up that
+literal name on rubygems.org — and fail. Neither RubyGems nor Bundler knows
+how to extract the real gem name, derive the namespace source URL, or resolve
+transitive dependencies that use URI names.
+
+**This gem bridges that gap.** It teaches both RubyGems' resolver (`gem
+install`) and Bundler's resolver (`bundle install`) how to parse URI dependency
+names, route them to the correct namespace source, and remap transitive deps
+on the fly.
+
+---
+
+## How It Works
+
+This gem ships a `rubygems_plugin.rb` that is automatically loaded by RubyGems
+at boot — before any gemspec is parsed.
+
+### 1. `Gem::Dependency` patch (`DependencyPatch`)
+
+Prepends helper methods `#uri_gem?` and `#uri_dependency` onto
+`Gem::Dependency`.
+
+```ruby
+dep = Gem::Dependency.new("https://beta.gem.coop/@myspace/my-gem", "~> 1.0")
+dep.uri_gem?          # => true
+dep.uri_dependency    # => #<Namespaced::Gem::UriDependency gem_name="my-gem" source_url="https://beta.gem.coop/@myspace">
+```
+
+### 2. URI parser (`UriDependency`)
+
+Parses a URI dependency name into its components:
+
+| Part          | Example                            |
+|---------------|------------------------------------|
+| `server_base` | `https://beta.gem.coop`            |
+| `namespace`   | `@myspace`                         |
+| `gem_name`    | `my-gem`                           |
+| `source_url`  | `https://beta.gem.coop/@myspace`   |
+
+Supports three forms:
+- **Full URI**: `https://beta.gem.coop/@myspace/my-gem`
+- **Shorthand**: `@myspace/my-gem` (defaults to `https://beta.gem.coop`)
+- **Package URL** ([purl-spec](https://github.com/package-url/purl-spec)):
+  - `pkg:gem/@myspace/my-gem` (namespace in path, default server)
+  - `pkg:gem/@myspace/my-gem?repository_url=https://beta.gem.coop` (explicit server)
+  - `pkg:gem/my-gem?repository_url=https://beta.gem.coop/@myspace` (namespace in qualifier)
+
+All three forms resolve to the same internal representation and are
+interchangeable anywhere a dependency name is accepted.
+
+### 3. Bundler DSL integration (`BundlerIntegration`)
+
+Patches `Bundler::Dsl#gemspec` so that after standard gemspec processing, any
+URI-named runtime dependencies automatically inject a `source` block:
+
+```ruby
+# What the user writes in their Gemfile:
+gemspec
+
+# What this patch injects automatically for URI deps:
+# source "https://beta.gem.coop/@myspace" do
+#   gem "my-gem", "~> 1.0"
+# end
+```
+
+This means the Gemfile needs **no manual source declarations** for URI deps
+found in the gemspec.
+
+---
+
+## Usage
+
+There are four ways to use `namespaced-gem` today, depending on your situation.
+
+### Use Case 1: Gem authors (primary)
+
+Add `namespaced-gem` as a runtime dependency of your gem. It is published on
+rubygems.org and acts as a bridge to gem.coop namespaces.
+
+```ruby
+Gem::Specification.new do |spec|
+  spec.name    = "my-gem"
+  spec.version = "1.0.0"
+
+  # This gem must be a runtime dependency so that its rubygems_plugin.rb
+  # is installed and loaded by RubyGems at boot — before any gemspec
+  # containing URI dependencies is evaluated.
+  spec.add_dependency "namespaced-gem"
+
+  # Traditional dependency from RubyGems.org:
+  spec.add_dependency "rack", "~> 3.0"
+
+  # Namespaced dependency from gem.coop (full URI):
+  spec.add_dependency "https://beta.gem.coop/@myspace/special-gem", "~> 0.5"
+
+  # Shorthand (defaults to beta.gem.coop):
+  spec.add_dependency "@myorg/internal-tool", ">= 2.0"
+
+  # Package URL (purl-spec):
+  spec.add_dependency "pkg:gem/@myorg/another-tool", ">= 1.0"
+  spec.add_dependency "pkg:gem/@myorg/extra?repository_url=https://beta.gem.coop", "~> 3.0"
+end
+```
+
+When a downstream user uses **Bundler** (the expected path), their Gemfile can
+remain:
+
+```ruby
+source "https://rubygems.org"
+gemspec
+```
+
+The Bundler integration automatically injects the correct `source` blocks for
+any URI dependencies found in the gemspec. Bundler uses only the Compact Index
+API, which gem.coop namespace servers already support.
+
+Both `bundle install` and `gem install my-gem` work — see
+[Use Case 4](#use-case-4-direct-gem-install-with-a-namespace) for the
+`gem install` path.
+
+### Use Case 2: Application developers
+
+If you are not publishing a gem but want to use URI-style dependencies in an
+application, install `namespaced-gem` directly:
+
+```bash
+gem install namespaced-gem
+```
+
+Because `rubygems_plugin.rb` files are only loaded from **installed** gems, the
+gem must be present in the gem path _before_ RubyGems evaluates any gemspec
+that contains URI dependencies. In practice this means:
+
+1. Install the gem first: `gem install namespaced-gem`
+2. Then declare URI dependencies in your gemspec or Gemfile as usual.
+
+> **Note:** Simply listing `gem "namespaced-gem"` in a Gemfile is _not
+> sufficient_ on its own — Bundler evaluates the Gemfile (and its `gemspec`
+> directive) before it installs gems, so the plugin would not yet be loaded.
+> The gem must already be installed via `gem install` (or as a transitive
+> dependency of another installed gem, as in Use Case 1).
+
+### Use Case 3: Global installation (enable namespace support Ruby-wide)
+
+Install `namespaced-gem` once into your Ruby environment and every subsequent
+`gem install` and `bundle install` in that Ruby will be able to resolve
+URI-named dependencies — no per-project configuration needed.
+
+```bash
+gem install namespaced-gem
+```
+
+That's it. The `rubygems_plugin.rb` is now in the gem path and will be loaded
+by RubyGems on every boot. From this point forward:
+
+- `gem install some-gem` will automatically resolve any URI-named transitive
+  dependencies found in `some-gem`'s gemspec.
+- `bundle install` in any project will automatically inject the correct
+  `source` blocks for URI dependencies found in gemspecs.
+
+This is useful for CI images, Docker containers, or development machines where
+you want namespace support available globally without requiring each gem or
+project to explicitly depend on `namespaced-gem`.
+
+```dockerfile
+# Example: Dockerfile
+RUN gem install namespaced-gem
+# All subsequent gem/bundle commands in this image now support URI deps.
+```
+
+```bash
+# Example: CI setup step
+gem install namespaced-gem
+bundle install   # URI deps in any gemspec are resolved automatically
+```
+
+### Use Case 4: Direct `gem install` with a namespace
+
+Once `namespaced-gem` is installed, you can install namespaced gems directly:
+
+```bash
+gem install namespaced-gem         # one-time setup (if not already installed)
+
+gem install @kaspth/oaken          # shorthand (defaults to beta.gem.coop)
+gem install https://beta.gem.coop/@kaspth/oaken   # full URI
+```
+
+This works because the plugin patches multiple layers of RubyGems' native
+`gem install` pipeline:
+
+1. **`GemResolverPatch`** intercepts the resolver and routes URI-named
+   dependencies to the correct namespace source via the Compact Index
+   (`versions` / `info/` endpoints).
+2. **`ApiSpecPatch`** synthesizes a `Gem::Specification` from the Compact Index
+   data already fetched — bypassing the legacy `quick/Marshal.4.8/` endpoint
+   that namespace servers don't serve.
+3. **`DownloadPatch`** provides clear, actionable error messages if the
+   namespace server fails to serve the `.gem` file.
+
+All three forms of URI dependency names are supported:
+
+```bash
+gem install @kaspth/oaken
+gem install https://beta.gem.coop/@kaspth/oaken
+gem install "pkg:gem/@kaspth/oaken"
+```
+
+---
+
+## Architecture
+
+```
+lib/
+  rubygems_plugin.rb              # Loaded by RubyGems at boot (or hot-loaded during install)
+  namespaced/
+    gem.rb                        # Main module
+    gem/
+      version.rb
+      uri_dependency.rb           # URI parser (value object)
+      namespace_source_registry.rb # Thread-safe registry of namespace source URLs
+      dependency_patch.rb         # Gem::Dependency patch (helper methods)
+      api_spec_patch.rb           # Gem::Resolver::APISpecification — Compact Index spec synthesis
+      download_patch.rb           # Gem::Source#download — namespace download error handling
+      bundler_integration.rb      # Bundler::Dsl#gemspec patch
+      bundler_resolver_patch.rb   # Bundler::Definition / Resolver transitive dep handling
+      gem_resolver_patch.rb       # Gem::RequestSet / InstallerSet for `gem install`
+      metadata_deps_hook.rb       # Gem.done_installing hook for hot-load deferred deps
+```
+
+---
+
+## Known Limitations
+
+1. **Plugin must be installed before first use (application developers only).**
+   This gem works as a RubyGems plugin (`rubygems_plugin.rb`), which means it
+   must be _installed_ in the gem path so that RubyGems loads the plugin at
+   boot before any gemspec containing URI dependencies is evaluated. For gem
+   authors (Use Case 1), this happens automatically — when a user installs your
+   gem, `namespaced-gem` is installed as a transitive dependency and available
+   on the next boot. For global installations (Use Case 3), the plugin is
+   already in the gem path by definition. For application developers
+   (Use Case 2), the gem must be installed explicitly with
+   `gem install namespaced-gem` before running `bundle install`, because
+   Bundler evaluates the Gemfile before it installs gems. In Ruby 4.0+,
+   RubyGems auto-loads `bundler/setup` when it detects a Gemfile in the working
+   directory, and this happens _before_ `RUBYOPT` `-r` flags are processed —
+   so the plugin must already be in the gem path.
+
+2. **Gemspec linting:** Tools that validate gemspecs (e.g. `gem build`, `rake
+   release`) work fine because `SpecificationPolicy#validate_name` only
+   validates the gem's *own* name — it does not check dependency names.
+
+3. **`gem.coop` (production) returns HTTP 200 with body `"404"` for namespace
+   endpoints.** Namespace resolution only works against `beta.gem.coop`
+   currently. The production `gem.coop` server returns HTTP 200 with a
+   plain-text body of `"404"` for namespace paths, which RubyGems
+   misinterprets as valid Compact Index data. The shorthand default server is
+   `beta.gem.coop` for this reason. See [ISSUE.md](ISSUE.md) for details.
+
+---
+
+## Version Constraints
+
+Version requirements work exactly as they always have — they are the second
+argument to `add_dependency`, completely separate from the name:
+
+```ruby
+spec.add_dependency "https://beta.gem.coop/@myspace/my-gem", "~> 1.0"
+#                    ^^^^^^^^^^ URI name ^^^^^^^^^^^^^^^^^^   ^^^^^^^^
+#                                                             version
+
+spec.add_dependency "pkg:gem/@myspace/my-gem", "~> 1.0"
+#                    ^^^^^^^ purl name ^^^^^^   ^^^^^^^^
+#                                               version
+```
+
+All standard operators (`~>`, `>=`, `=`, etc.) are supported unchanged.
+
+> **Note:** The purl spec allows a `@version` component in the name itself
+> (e.g. `pkg:gem/@ns/foo@2.0`). If present it is **ignored** — version
+> constraints always come from the second argument to `add_dependency`.
+
+---
+
+## Development
+
+```bash
+bundle install
+bundle exec rspec              # unit + offline integration tests
+bundle exec rspec --tag network  # network integration tests (hits beta.gem.coop)
+bundle exec rake               # tests + rubocop
+```
+
+The network integration tests resolve a real gem (`@kaspth/oaken`) from
+`beta.gem.coop` and verify `bundle lock` produces a correct `Gemfile.lock`.
+They are excluded from the default `rspec` run and must be opted into with
+`--tag network`.
+
+---
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitLab at
+<https://gitlab.com/galtzo-floss/namespaced-gem>.
+
+This project is intended to be a safe, welcoming space for collaboration, and
+contributors are expected to adhere to the
+[code of conduct](https://gitlab.com/galtzo-floss/namespaced-gem/-/blob/main/CODE_OF_CONDUCT.md).
+
+## Code of Conduct
+
+Everyone interacting in the namespaced-gem project's codebases, issue trackers,
+chat rooms and mailing lists is expected to follow the
+[code of conduct](https://gitlab.com/galtzo-floss/namespaced-gem/-/blob/main/CODE_OF_CONDUCT.md).

data/Rakefile

@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new(:spec)
+
+require "rubocop/rake_task"
+
+RuboCop::RakeTask.new
+
+task default: %i[spec rubocop]

data/WORKAROUND.md

@@ -0,0 +1,43 @@
+**STATUS: RESOLVED — Approach B was implemented.** `ApiSpecPatch` synthesizes specs from Compact Index data (bypassing the Marshal endpoint), `DownloadPatch` handles namespace download errors, and `beta.gem.coop` serves `/gems/` under namespace paths. Both `gem install @kaspth/oaken` and `bundle install` with URI deps now work.
+
+---
+
+Plan: Two Approaches for gem install @kaspth/oaken with Compact Index-Only Servers
+The namespaced-gem plugin already handles Bundler workflows, but gem install fails because RubyGems requires two legacy Marshal API endpoints (quick/Marshal.4.8/… and gems/…) that gem.coop namespace servers don't serve. Two approaches can bridge this gap: (A) delegate to Bundler's infrastructure, or (B) patch RubyGems' native classes to use the Compact Index.
+
+CRITICAL INVARIANT — Namespaces Are Not Optional
+A namespaced gem (e.g. @kaspth/oaken) and a non-namespaced gem with the same base name (e.g. oaken on rubygems.org) are COMPLETELY DIFFERENT, UNRELATED gems. The namespace is an integral part of the gem's identity. It MUST NEVER be stripped, ignored, or used to fall back to a non-namespaced gem. Each namespace server (e.g. https://beta.gem.coop/@kaspth/) is a discrete, independent gem source — exactly like a private gem server that happens to have a gem with a common name. Every code path in this plugin — resolution, spec fetching, downloading — must preserve the namespace and source URL end-to-end. If an operation fails against the namespace source, it must ERROR, not silently retry against the root server or rubygems.org.
+
+Approach A: Delegate to Bundler from within the plugin
+Concept: When gem install @kaspth/oaken is detected, intercept early in the gem install pipeline and hand off resolution + installation to Bundler, which already works with the Compact Index-only server. The Bundler-based installer MUST use the full namespace source URL (e.g. https://beta.gem.coop/@kaspth/) as the sole source — never the root server or rubygems.org.
+Steps
+Intercept in Gem::Commands::InstallCommand#install_gem — prepend a method in a new install_command_patch.rb that detects URI-named gem arguments (via UriDependency.uri?). For URI deps, bypass DependencyInstaller entirely and call a Bundler-based installer instead.
+Create a Namespaced::Gem::BundlerGemInstaller helper class — this class constructs an in-memory Bundler definition (or a temp Gemfile string) with the right source block + gem declaration, invokes Bundler::Installer.install (or uses Bundler::Inline internals via Bundler::CLI::Install or gemfile() from bundler/inline), and installs the gem into the same Gem.dir RubyGems would use. The source block MUST be scoped to the namespace URL only — e.g. source("https://beta.gem.coop/@kaspth/") { gem "oaken" }.
+Configure Bundler to install into the system gem path — set Bundler.settings[:path] to Gem.dir and Bundler.settings[:system] to true (or use BUNDLE_PATH/BUNDLE_SYSTEM_GEMS) so gems land where gem install users expect them — not in a vendor/bundle directory.
+Map Bundler output back to RubyGems' InstallCommand expectations — after Bundler installs, report the installed spec(s) back to the install command's output format (gem name, version, summary line), so the user sees the same CLI output they'd expect from gem install.
+Wire the patch in rubygems_plugin.rb — load and apply InstallCommandPatch alongside the existing patches.
+Risks / Trade-offs
+Heavy dependency on Bundler internals — Bundler's Definition, Installer, and settings APIs are not designed for one-shot single-gem installs; edge cases around lockfile generation, Bundler.root, and Bundler.reset! state will need careful handling.
+bundler/inline is the easiest entry point but it calls Bundler.reset! and exit on failure, so using it in-process requires careful error handling or subprocess isolation.
+Gem path mismatch — Bundler defaults to project-local paths; mapping back to Gem.dir needs explicit config. Version conflicts with already-installed gems must be considered.
+Transitive deps — Bundler would resolve the full transitive tree automatically; this is a pro, but may surprise users who expect gem install to only install the named gem + immediate runtime deps.
+
+Approach B: Patch RubyGems' native classes to use the Compact Index
+Concept: Instead of delegating to Bundler, patch the two failing points in RubyGems' own pipeline — fetch_spec (which needs Marshal data) and gem download (which needs the correct URL) — to use the already-working Compact Index data and downloads served by the namespace server.
+Steps
+Patch Gem::Resolver::APISpecification#spec — in a new api_spec_patch.rb, prepend #spec so that for namespace sources (detected via UriDependency), instead of calling source.fetch_spec(tuple) (which hits the missing Marshal endpoint), construct a Gem::Specification from the Compact Index info/ data. The info/{gem} endpoint returns dependency names, version constraints, and platform data — enough to build a minimal Gem::Specification with name, version, platform, and dependencies populated. This is sufficient for the resolver.
+Parse the Compact Index info/ response — create a Namespaced::Gem::CompactIndexClient utility that fetches {source_url}/info/{gem_name}, parses the Compact Index format (line-per-version: version deps_list|checksum,ruby:required_ruby,rubygems:required_rubygems), and returns structured data. Bundler already has Bundler::Fetcher::CompactIndex — consider extracting its parser or using the compact_index gem directly.
+Patch the gem download URL — prepend Gem::RemoteFetcher#download (or Gem::Source#download) in a new download_patch.rb so that for namespace sources, the .gem download is attempted from {namespace_source}/gems/{gem}-{ver}.gem. The download MUST use the namespace source URL — never the root server URL. If the namespace server returns 404 for /gems/, that is a server bug to be reported (per ISSUE.md), NOT a reason to silently fall back to the root server. Stripping the namespace and downloading from the root would risk fetching a COMPLETELY DIFFERENT gem that happens to share the same base name.
+Track namespace source metadata through the resolver — the existing InstallerSetPatch#find_all creates a Gem::Source for the namespace URL. The APISpecification objects returned carry a reference to that source. Ensure the download patch can identify these specs (e.g., by checking spec.source.uri against known namespace URLs stored in a registry) to know when to route the download through the namespace source.
+Wire both patches in rubygems_plugin.rb — load ApiSpecPatch and DownloadPatch alongside existing patches, applied at boot.
+Risks / Trade-offs
+Compact Index info/ data is incomplete vs. a full Gem::Specification — fields like summary, description, authors, executables, extensions, metadata, post_install_message are NOT in the Compact Index. The synthesized spec is sufficient for resolution but not for installation (e.g., Gem::Installer needs extensions to compile C extensions). However, the actual .gem file contains a full embedded gemspec, so once downloaded the real spec takes over.
+Namespace server MUST serve /gems/ under the namespace path — the namespace server is a discrete gem server and must serve all endpoints (including /gems/) under its namespace path. If it does not, that is a server-side bug. The plugin should surface a clear error (e.g., "Namespace server at {url} returned 404 for {gem}.gem — the server must serve /gems/ under the namespace path") rather than silently falling back to the root server.
+Tighter coupling to RubyGems internals — Gem::Resolver::APISpecification#spec and Gem::Source#fetch_spec are internal APIs that may change across RubyGems versions. required_rubygems_version already gates to >= 4.0.5.
+Simpler, lighter-weight — no Bundler dependency at runtime for gem install, no temp files, no state management.
+
+Further Considerations
+Hybrid approach? Approach B for spec synthesis is simpler for the happy path, while Approach A could serve as an alternative implementation strategy if the Compact Index data proves insufficient (e.g., gems with native extensions where the synthesized spec is too minimal). Consider implementing B first, with A as an alternative. In either case, a namespaced gem is ALWAYS resolved and downloaded from its namespace source — never from the root server or any other source.
+Server-side fix is complementary — the ISSUE.md already requests gem.coop serve the Marshal endpoints AND the /gems/ endpoint under the namespace path. If/when the server adds the Marshal endpoints, the native RubyGems code path would work without any plugin patches for spec fetching. The patches should detect when the native path succeeds (i.e., the Marshal endpoint is available under the namespace) and defer to it. This is NOT a "fallback" to a different source — it is the same namespace source supporting more endpoint types. The namespace is always preserved.
+NO FALLBACK TO ROOT OR NON-NAMESPACED SOURCES — To reiterate: if spec fetching or gem download fails against the namespace source, the correct behavior is to raise an error, not to retry against the root server or rubygems.org. The namespace identifies a completely different gem. Stripping it would be like resolving "acme-corp-utils" and silently downloading "utils" instead — a security and correctness violation.
+The compact_index gem (used internally by Bundler) provides a battle-tested parser for the info/ format — should we add it as a runtime dependency, vendor a minimal parser, or reuse Bundler's CompactIndex::Client directly?