bundler-spinel 0.0.1.pre → 0.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: fedb059cdc7900cd98ccfff408a28e1fb4e4eaad68d9e7afe48c9d967cb97b71
-  data.tar.gz: a1d6fbfb31256d58cacab1cc120290455bd38de869295504a1c6dcdee96ea8da
+  metadata.gz: 99f01821538cfb0ce86a06e2ecb1251f89e1086530c59423715f509d18e0c717
+  data.tar.gz: ca1c456f3bf3c4d9e4424a0466daa23bb4495b64357adfed8a69145953d54a06
 SHA512:
-  metadata.gz: acd75d23500e6af2d3e55c116bc0f5d82e8a4c8b1912049a349967e9cc920134ffe4820862abeeb6a753445f0a668cf26a9248c6aed4b983ea0dff7c262c13ee
-  data.tar.gz: 1bd2fa39872c215534c919f6a8c05cf01cf6dc30e5cce162d470d8d0ec3d750a09e06bc3a404d05f1ac4195a0c86232ca6e93f9d761e5d99771603361fd58756
+  metadata.gz: fa08674c1d4206f15713004a9452d3aba7b43c2f6a1997c5c0d6d97ac7bbe00a4b2a986eb50673518acebcf807cab58ad785916aa1c3d3c71dab939d1e4c9163
+  data.tar.gz: 41b3407d8dd4d7f115a50c3294e0e0204e9c3c719ee52042876245ff0a63fa1cfd4b1e2ba8b97037d94ba27a2f25c25f893660999b306ce912663e050ed674d6

data/ARCHITECTURE.md

@@ -53,7 +53,13 @@ Spinel's failure modes aren't exit codes, so one signal isn't enough:
 | `rejected` | unsupported **call** (`cannot resolve call to 'X'` → `unresolved:X`), or `analyze failed` / non-zero exit | gate fails the lock |
 | `risky` | compiles clean, but static scan found constructs Spinel degrades silently (`define_method`, `eval`, `send`, `method_missing`, C-ext…) | gate allows; `--strict` fails |
 | `clean` | compiles clean, no risky constructs | gate allows |
-| `verified` | `clean` **and** the gem's own tests pass through a Spinel-compiled harness | curated whitelist + platform badge |
+| `loaded` | `clean` **and** a require-only differential run loads identically under CRuby and Spinel — but its logic was never exercised, so a silent miscompile there is still possible | gate allows; **not** trusted by the curated source |
+| `verified` | `loaded` **and** a behaviour smoke (drives the gem's API) runs identically under CRuby and a Spinel-compiled harness | curated whitelist + platform badge |
+
+The `loaded`/`verified` split is empirical: gems that load identically still
+silently miscompile in untested logic (`strings-ansi`'s `sanitize` → `"0"`,
+`semantic_puppet`'s `1.2.3 < 1.10.0` → `false`). Only a behaviour smoke catches
+it — see `harness/`.
 
 Two probe signals feed this:
 

data/CHANGELOG.md

@@ -4,6 +4,45 @@ All notable changes to `bundler-spinel` are documented here. The format
 follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the
 project aims to follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.1.0] — 2026-06-01
+
+First non-prerelease. Installable without `--pre`. Closes the onboarding gap
+(spinelgems#9): a newcomer can `gem install bundler-spinel` and go from nothing
+to a compiled Spinel app without an out-of-band `git clone matz/spinel && make`.
+
+### Added
+- **`spinel-compat install-engine [REV]`** — provisions the Spinel compiler
+  itself: fetch matz/spinel at a pinned revision (arg › `SPINEL_PIN` file ›
+  default) → `make deps && make all` → cache under `~/.cache/spinel/<rev>/` →
+  point a `current` symlink at it. Idempotent, offline-after-first-build;
+  `--force` rebuilds. `Engine` now resolves that cache after `SPINEL_DIR` and
+  before PATH, so a provisioned engine is found with zero further configuration.
+- **`spinel-compat init [DIR]`** — scaffolds a minimal Spinel + Tep project
+  (`Gemfile` with the engine marker + `gem "tep"`, a hello `app.rb`, a
+  `bin/build` that runs `install-engine` + `vendor` + `tep build`). Onboarding
+  becomes `bundle install && spinel-compat init && bin/build`.
+- `spinel-compat verify --full`: force-requires every `lib/` file (no `autoload`
+  masking, no `LoadError` rescue) so verification covers the gem's whole surface,
+  not just the entrypoint. Ledger probe `verify-full`.
+- **Composable signal badges** on the catalog: `👤 human` (version-pinned
+  attestations), `✪ tests` (the gem's own suite passes under Spinel,
+  `verify --tests`), and a surfaced `rubric` tag (*why* a non-verified gem isn't
+  there yet). See `docs/verification-tiers.md`.
+- `docs/verification-tiers.md`: the trust ladder, the full-surface bar, the
+  badge model, and the matz/spinel bug pipeline.
+
+### Changed
+- **`verified` now means full-surface.** The catalog grants ★ only to gems with
+  a `verify-full` match — an entrypoint-only/constant smoke no longer qualifies
+  (it overstated usability; see the qdrant-ruby spike, spinelgems#4).
+- `GemFetcher` honours `SPINEL_COMPAT_CACHE` to relocate the source cache off a
+  tight root fs.
+
+### Fixed
+- `vendor`: a split `@MOD_O@` / `@MOD_CFLAGS@` C-extension pair now associates by
+  module name, so the source compile gets the sibling's `pkg_config` include
+  path (spinelgems#8 — tep pg → `libpq-fe.h`).
+
 ## [0.0.1.pre] — 2026-05-26
 
 First pre-release. **Experimental** — the CLI surface, the verdict vocabulary,
@@ -18,4 +57,5 @@ and the ledger format may all change before `0.0.1`. Install with `--pre`.
 - Wholesale `survey` with a thread-safe ledger, a per-compile wall-clock
   timeout (`analyze-timeout` reject reason), and a ledger-based report.
 
+[0.1.0]: https://github.com/OriPekelman/spinelgems/releases/tag/v0.1.0
 [0.0.1.pre]: https://github.com/OriPekelman/spinelgems/releases/tag/v0.0.1.pre

data/README.md

@@ -1,24 +1,35 @@
 # bundler-spinel
 
-> ⚠️ **Pre-release / experimental** (`0.0.1.pre`). The CLI surface, the verdict
-> vocabulary, and the ledger format may all change before `0.0.1`. Install with
-> `--pre`. (`spinelgems.org` is not live yet.)
-
-**Use a standard `Gemfile` for your [Spinel](https://github.com/matz/spinel)
-project — for now.** Spinel-compiled projects have no shared way to declare or
-exchange dependencies, so each vendors by hand. Rather than design a package
-manager, borrow a format everyone already knows and revisit later. See
-[RFC.md](RFC.md) for the proposal.
-
-`bundler-spinel` is a small Bundler plugin that makes that practical in two ways:
-
-1. **Makes it work** — places resolved dependencies where Spinel can actually
-   find them. Spinel has no load path and inlines `require_relative`, so a dep
-   has to be *placed* and wired. `spinel-compat vendor` does that from a lockfile.
-2. **Gates** — Spinel silently emits a no-op for unsupported Ruby (exit 0), so
-   "it compiled" ≠ "it works". The plugin probes gems and flags incompatible ones
-   at `bundle lock` time, with reasons that name the missing feature — nicer than
-   a silent miscompile. Verdicts are forward-compatible (keyed on the Spinel rev).
+> ⚠️ **Pre-release / experimental** (`0.1.0`). The CLI surface, the verdict
+> vocabulary, and the ledger format may still change. Browse the live catalog at
+> **<https://spinelgems.org>**.
+
+**Use a standard `Gemfile` for your [Spinel](https://github.com/matz/spinel) project.**
+
+[Spinel](https://github.com/matz/spinel) is a new ahead-of-time Ruby compiler.
+Spinel projects have no shared way to declare or exchange dependencies, so each
+vendors by hand. Rather than invent a package manager, we propose a plain
+`Gemfile` — for two reasons:
+
+- **Share Spinel code.** A `Gemfile` (with `path:` / `git:` siblings) becomes the
+  preferred way to declare and exchange dependencies *between* Spinel projects.
+- **Reach the Ruby ecosystem.** The same `Gemfile` lets a Spinel project pull from
+  the enormous existing body of Ruby gems.
+
+The catch: **most gems won't compile under Spinel yet.** Its scope is deliberately
+limited and still growing. So we surveyed the ecosystem and publish a
+**[catalog](https://spinelgems.org)** of what works today, at each engine revision.
+
+## The bundler plugin
+
+`bundler-spinel` makes the convention practical in two ways:
+
+1. **Makes it work** — Spinel has no load path and inlines `require_relative`, so a
+   dependency has to be *placed* where Spinel will follow it. `spinel-compat vendor`
+   does that from a lockfile.
+2. **Gates** — Spinel silently no-ops unsupported Ruby (exit 0), so "it compiled"
+   ≠ "it works". The plugin probes gems and flags incompatible ones at `bundle lock`
+   time, with reasons that name the missing feature.
 
 ## The Gemfile convention
 
@@ -26,59 +37,54 @@ manager, borrow a format everyone already knows and revisit later. See
 source "https://rubygems.org"
 ruby "3.3.0", engine: "spinel", engine_version: "0.0.0"
 
-gem "tep", git: "https://…/tep.git"   # siblings via path:/git: (replaces rsync)
+gem "tep", git: "https://…/tep.git"   # siblings via path:/git:
 gem "some_pure_ruby_lib"
 ```
 
 `bundle lock` resolves normally (it ignores the engine marker); the marker guards
-`bundle install` (exit 18 under CRuby). Nothing here is novel — that's the point.
+`bundle install`. Nothing here is novel — that's the point.
 
 ## Quick start
 
 ```sh
-# 1. make it work — place deps where Spinel finds them
-bundle lock
-exe/spinel-compat vendor                 # -> vendor/spinel/<gem>/lib + vendor/spinel/deps.rb
-#    then `require_relative "vendor/spinel/deps"` from your Spinel entrypoint
-
-# 2. gate — flag what Spinel can't compile, early
-exe/spinel-compat check Gemfile.lock     # exit 1 if any gem is rejected
+gem install bundler-spinel
+spinel-compat install-engine       # fetch + build the Spinel compiler (cached)
+spinel-compat init my_app          # scaffold a Gemfile + app.rb + bin/build
+cd my_app && bundle install && ./bin/build
 ```
 
-As a Bundler plugin:
+Or add the gate to an existing project:
 
 ```sh
-bundle plugin install bundler-spinel --git https://github.com/OriPekelman/spinelgems.git   # or --path .
-bundle spinel-lock      # bundle lock, then report incompatible gems
-bundle spinel-check     # gate an existing Gemfile.lock
+bundle plugin install bundler-spinel
+bundle spinel-lock                 # bundle lock + report incompatible gems
+spinel-compat vendor               # place deps -> vendor/spinel/<gem>/lib + deps.rb
+spinel-compat check Gemfile.lock   # gate: exit 1 if any gem is rejected
 ```
 
-## The rest of the toolbelt
-
-```sh
-exe/spinel-compat engine                 # detected compiler + engine rev
-exe/spinel-compat probe rake [--dir P]   # probe one gem (or a local/sibling dir)
-exe/spinel-compat verify NAME --smoke F  # differential CRuby-vs-Spinel run -> verified
-exe/spinel-compat survey --list F        # wholesale review -> reason histogram
-exe/spinel-compat serve --store DIR      # curated source (only vetted gems)
-exe/spinel-compat ledger / reprobe       # inspect / re-probe under current rev
-```
-
-Verdicts: `✓ clean` · `★ verified` · `~ risky` · `✗ rejected`.
-
-## Environment
-
-- `SPINEL_DIR` — path to the Spinel checkout (default `~/spinel`; falls back to a `spinel` on `PATH`).
-- `SPINEL_COMPAT_LEDGER` — ledger path (default `ledger/compat.jsonl`).
-
-## Status
-
-Working: the Gemfile convention, `vendor` (placement), the lock-time gate +
-Bundler plugin, the probe + forward-compat ledger, the `verified` differential
-harness, the curated source (`serve`), and the wholesale `survey`.
-
-The probe is a **lower bound** — Spinel's lack of a load path means multi-file
-plain-`require` gems under-probe, and silent miscompiles are invisible to it.
-Trust `verified` (smoke runs identically under CRuby and Spinel), not `clean`,
-where it matters. Empirically most third-party gems reject today, so the weight
-is on your own vetted gems and `path:`/`git:` siblings — not a rubygems mirror.
+Then `require_relative "vendor/spinel/deps"` from your Spinel entrypoint.
+
+> **The compiler builds from source.** `spinel-compat install-engine` clones
+> [Spinel](https://github.com/matz/spinel) and runs `make` (a few minutes, once
+> per revision; needs `git` + `make` + a C compiler), caching the result under
+> `~/.cache/spinel/`. This will become near-instant once there are **prebuilt
+> binaries per platform** — but Spinel is pre-release and moving fast (no stable
+> tags yet, revisions land daily), so building from source is deliberate for now:
+> it's portable, needs no release pipeline, and always matches the *exact* engine
+> revision your compatibility verdicts are keyed on. Prebuilts come once the
+> engine stabilizes; we're not there yet.
+
+Verdicts: `★ verified` · `○ loaded` · `✓ clean` · `~ risky` · `✗ rejected`. Trust
+`verified` (a behaviour smoke matches CRuby); `clean`/`loaded` are cheap lower
+bounds. [What the verdicts mean →](https://spinelgems.org)
+
+## More
+
+- [RFC.md](RFC.md) — the proposal.
+- [docs/cli.md](docs/cli.md) — the full `spinel-compat` toolbelt, verdict ladder, env vars.
+- [docs/adoption.md](docs/adoption.md) — adopting the convention + extracting libraries.
+- [ARCHITECTURE.md](ARCHITECTURE.md) — how the gate, the ledger, and the verify rung work.
+- [harness/](harness/README.md) — the behaviour-`verified` testing ground (and bug pipeline).
+- [docs/verification-tiers.md](docs/verification-tiers.md) — why `verified` means *full surface*.
+- [docs/deploying-tep-on-upsun.md](docs/deploying-tep-on-upsun.md) — the catalog site is itself a Spinel-compiled [Tep](https://github.com/OriPekelman/tep) app.
+- [docs/related.md](docs/related.md) — the two unrelated "Spinel" projects, `rv`, and `rubocop_spinel`.

data/lib/bundler/spinel/checker.rb

@@ -21,15 +21,13 @@ module Bundler
       def check(lockfile = "Gemfile.lock", strict: false)
         @engine.ensure!
         parsed = Bundler::LockfileParser.new(File.read(lockfile))
+        lock_dir = File.dirname(File.expand_path(lockfile))
         rejected = []
         risky = []
         verdicts = []
 
         parsed.specs.each do |spec|
-          # path:/git: sources (e.g. a sibling like tep) probe in place; the
-          # GEM source fetches. We only have name+version here, so prototype
-          # handles the rubygems case; path/git probing is a documented TODO.
-          v = verdict_for(spec.name, spec.version.to_s)
+          v = verdict_for(spec, lock_dir)
           next unless v # skipped (unfetchable / TODO source)
 
           verdicts << v
@@ -43,11 +41,24 @@ module Bundler
 
       private
 
-      def verdict_for(name, version)
+      # path:/git: sources probe in place (the local checkout); GEM
+      # sources go through the cache-backed fetcher. Closes the TODO
+      # noted in OriPekelman/spinelgems#3.
+      def verdict_for(spec, lock_dir)
+        name = spec.name
+        version = spec.version.to_s
         cached = @ledger.lookup(name, version, @engine.rev)
         return cached if cached
 
-        dir = @fetcher.fetch(name, version)
+        dir =
+          if spec.source.respond_to?(:path) && spec.source.path
+            path = spec.source.path.to_s
+            path = File.expand_path(path, lock_dir) unless File.absolute_path?(path)
+            File.directory?(path) or raise Error, "path: source for #{name} not found: #{path}"
+            path
+          else
+            @fetcher.fetch(name, version)
+          end
         @probe.probe(name, version, dir)
       rescue Error => e
         warn "[spinel-compat] skipped #{name} #{version}: #{e.message}"

data/lib/bundler/spinel/cli.rb

@@ -6,7 +6,7 @@ module Bundler
     # Bundler plugin command. Keeps all logic in the library classes.
     class CLI
       VERDICT_GLYPH = {
-        "clean" => "✓", "verified" => "★", "risky" => "~", "rejected" => "✗"
+        "clean" => "✓", "loaded" => "○", "verified" => "★", "risky" => "~", "rejected" => "✗"
       }.freeze
 
       def initialize(out: $stdout, err: $stderr)
@@ -18,15 +18,24 @@ module Bundler
         cmd = argv.shift
         case cmd
         when "engine"  then cmd_engine
+        when "install-engine" then cmd_install_engine(argv)
+        when "init"    then cmd_init(argv)
         when "probe"   then cmd_probe(argv)
         when "verify"  then cmd_verify(argv)
         when "vendor"  then cmd_vendor(argv)
         when "check"   then cmd_check(argv)
         when "serve"   then cmd_serve(argv)
         when "build-index" then cmd_build_index(argv)
+        when "build-site"  then cmd_build_site(argv)
+        when "build-db"    then cmd_build_db(argv)
+        when "build-history" then cmd_build_history(argv)
+        when "server"      then cmd_server(argv)
         when "ledger"  then cmd_ledger(argv)
+        when "diff"    then cmd_diff(argv)
+        when "detect-ext" then cmd_detect_ext(argv)
         when "reprobe" then cmd_reprobe(argv)
         when "survey"  then cmd_survey(argv)
+        when "enrich"  then cmd_enrich(argv)
         when nil, "-h", "--help", "help" then usage; 0
         else
           @err.puts "unknown command: #{cmd}"; usage; 2
@@ -42,9 +51,30 @@ module Bundler
         @out.puts "spinel binary : #{e.bin} (#{e.available? ? 'found' : 'MISSING'})"
         @out.puts "engine rev    : #{e.rev}"
         @out.puts "ledger        : #{Ledger.new.path}"
+        unless e.available?
+          @out.puts ""
+          @out.puts "No engine found. Provision one with: spinel-compat install-engine"
+        end
         e.available? ? 0 : 1
       end
 
+      # Provision the Spinel compiler from source (spinelgems#9). Optional REV
+      # arg pins the revision; --force rebuilds even if cached.
+      def cmd_install_engine(argv)
+        force = !!argv.delete("--force")
+        rev = argv.shift # nil → SPINEL_PIN file / default
+        EngineInstaller.new(rev: rev, out: @out).install(force: force)
+        0
+      end
+
+      # Scaffold a minimal Spinel + Tep project (spinelgems#9 stretch).
+      def cmd_init(argv)
+        rev = (i = argv.index("--rev")) ? argv[i + 1] : nil
+        dir = argv.find { |a| !a.start_with?("-") && a != rev } || "."
+        Scaffold.init(dir, out: @out, rev: rev)
+        0
+      end
+
       def cmd_probe(argv)
         dir = (i = argv.index("--dir")) ? argv.delete_at(i + 1).tap { argv.delete_at(i) } : nil
         name = argv.shift or raise Error, "usage: spinel-compat probe NAME [VERSION] [--dir PATH]"
@@ -64,7 +94,9 @@ module Bundler
       def cmd_verify(argv)
         dir = (i = argv.index("--dir")) ? argv.delete_at(i + 1).tap { argv.delete_at(i) } : nil
         smoke = (j = argv.index("--smoke")) ? argv.delete_at(j + 1).tap { argv.delete_at(j) } : nil
-        name = argv.shift or raise Error, "usage: spinel-compat verify NAME [VERSION] [--dir PATH] [--smoke FILE]"
+        full = !!argv.delete("--full")
+        tests = !!argv.delete("--tests")
+        name = argv.shift or raise Error, "usage: spinel-compat verify NAME [VERSION] [--dir PATH] [--smoke FILE | --tests] [--full]"
         engine = Engine.new
         if dir
           version = argv.shift || "path"
@@ -73,18 +105,43 @@ module Bundler
           version = argv.shift || latest_version(name)
           gem_dir = GemFetcher.new.fetch(name, version)
         end
-        v = Verifier.new(engine, Ledger.new).verify(name, version, gem_dir, smoke: smoke && File.expand_path(smoke))
+        # --tests: translate the gem's own minitest/test-unit suite into a
+        # Spinel-compilable runner and use it as the smoke (spinelgems#6).
+        if tests
+          require_relative "test_runner"
+          runner = TestRunner.generate_suite(gem_dir)
+          raise Error, "no runnable test suite found in #{gem_dir}/test" unless runner
+          smoke = File.join(Dir.tmpdir, "__spinel_tests_#{name}.rb")
+          File.write(smoke, runner)
+        end
+        v = Verifier.new(engine, Ledger.new).verify(name, version, gem_dir, smoke: smoke && File.expand_path(smoke), full: full)
         print_verdict(v)
-        v.verified? ? 0 : 1
+        File.delete(smoke) if tests && smoke && File.exist?(smoke)
+        (v.verified? || v.loaded?) ? 0 : 1
       end
 
       def cmd_vendor(argv)
         into = (i = argv.index("--into")) ? argv.delete_at(i + 1).tap { argv.delete_at(i) } : "vendor/spinel"
+        # --ext @PLACEHOLDER@=/abs/x.o : reuse a prebuilt .o instead of recompiling (repeatable).
+        ext_overrides = {}
+        while (e = argv.index("--ext"))
+          pair = argv.delete_at(e + 1).to_s
+          argv.delete_at(e)
+          k, v = pair.split("=", 2)
+          ext_overrides[k] = File.expand_path(v) if k && v
+        end
+        # --no-ext NAME : opt out of an optional C extension (repeatable; also SPINEL_EXT_DISABLE).
+        ext_disable = []
+        while (d = argv.index("--no-ext"))
+          ext_disable << argv.delete_at(d + 1).to_s
+          argv.delete_at(d)
+        end
         lock = argv.shift || "Gemfile.lock"
         raise Error, "no #{lock}; run `bundle lock` first" unless File.exist?(lock)
 
-        res = Vendorer.new.vendor(lock, into: into)
-        @out.puts "vendored #{res[:count]} gem(s) -> #{res[:into]}"
+        res = Vendorer.new.vendor(lock, into: into, ext_overrides: ext_overrides, ext_disable: ext_disable)
+        ext = res[:extensions].to_i
+        @out.puts "vendored #{res[:count]} gem(s)#{ext.positive? ? " (+#{ext} C ext)" : ''} -> #{res[:into]}"
         @out.puts "  require_relative \"#{res[:into]}/deps\" from your Spinel entrypoint"
         0
       end
@@ -128,6 +185,141 @@ module Bundler
         0
       end
 
+      # Fetch rubygems.org metadata (description, downloads, last-update, …) for a
+      # gem list (or the ledger at this rev) into a meta.jsonl sidecar.
+      def cmd_enrich(argv)
+        require_relative "enricher"
+        out = (j = argv.index("--out")) ? File.expand_path(argv[j + 1]) : raise(Error, "enrich needs --out FILE")
+        jobs = (i = argv.index("--jobs")) ? argv.delete_at(i + 1).to_i.tap { argv.delete_at(i) } : 8
+        list = (k = argv.index("--list")) ? argv[k + 1] : nil
+        names = if list
+                  File.readlines(File.expand_path(list)).map(&:strip).reject { |l| l.empty? || l.start_with?("#") }
+                else
+                  rev = Engine.new.rev
+                  seen = {}
+                  Ledger.new.each { |v| seen[v.gem] = true if v.rev == rev }
+                  seen.keys
+                end
+        Enricher.new(out: out, jobs: jobs).run(names)
+        @out.puts "wrote #{out} (#{names.size} gems)"
+        0
+      end
+
+      # Per-gem verdict diff between two revs in the ledger — see what improved
+      # or regressed between two surveys. Revs match by prefix (the leading
+      # `git:<sha>` is enough); pass --names to list the gems per transition.
+      #
+      #   spinel-compat diff git:2183a92 git:a03bb49 [--names]
+      def cmd_diff(argv)
+        names_flag = !!argv.delete("--names")
+        rev_a = argv.shift or raise Error, "usage: spinel-compat diff REV_A REV_B [--names]"
+        rev_b = argv.shift or raise Error, "usage: spinel-compat diff REV_A REV_B [--names]"
+
+        a = {}
+        b = {}
+        Ledger.new.each do |v|
+          a[v.gem] = v.verdict if v.rev.start_with?(rev_a)
+          b[v.gem] = v.verdict if v.rev.start_with?(rev_b)
+        end
+        raise Error, "no entries match REV_A=#{rev_a}" if a.empty?
+        raise Error, "no entries match REV_B=#{rev_b}" if b.empty?
+
+        both = a.keys & b.keys
+        unchanged = 0
+        transitions = Hash.new { |h, k| h[k] = [] }
+        both.each do |g|
+          if a[g] == b[g] then unchanged += 1
+          else transitions["#{a[g]} -> #{b[g]}"] << g
+          end
+        end
+
+        @out.puts "rev A: #{rev_a} (#{a.size} gems)"
+        @out.puts "rev B: #{rev_b} (#{b.size} gems)"
+        @out.puts "common: #{both.size} · unchanged: #{unchanged} · changed: #{both.size - unchanged}"
+        @out.puts "only in A: #{(a.keys - b.keys).size} · only in B: #{(b.keys - a.keys).size}"
+        @out.puts
+        transitions.sort_by { |_, gs| -gs.size }.each do |t, gs|
+          @out.printf("%-26s %d\n", t, gs.size)
+          gs.sort.each { |g| @out.puts "    #{g}" } if names_flag
+        end
+        0
+      end
+
+      # Infer a draft spinel-ext.json from a gem's `ffi_cflags "@PLACEHOLDER@"`
+      # declarations + nearby `.c` files. Keeps the C-ext convention strictly
+      # consumer-side: gem authors don't have to ship the manifest.
+      def cmd_detect_ext(argv)
+        require_relative "ext_detector"
+        out_file = (j = argv.index("--out")) ? argv.delete_at(j + 1).tap { argv.delete_at(j) } : nil
+        dir = argv.shift or raise Error, "usage: spinel-compat detect-ext GEM_DIR [--out FILE]"
+
+        json = ExtDetector.new(File.expand_path(dir)).to_json
+        if out_file
+          File.write(File.expand_path(out_file), json + "\n")
+          @out.puts "wrote #{out_file}"
+        else
+          @out.puts json
+        end
+        0
+      end
+
+      # Build the spinelgems.org static deploy tree: presentation + ledger-driven
+      # catalog, plus the Compact Index (apex double-duty) when a --store is given.
+      def cmd_build_site(argv)
+        require_relative "site"
+        out = (j = argv.index("--out")) ? argv[j + 1] : raise(Error, "build-site needs --out DIR")
+        store = (i = argv.index("--store")) ? File.expand_path(argv[i + 1]) : nil
+        min = (k = argv.index("--min")) ? argv[k + 1].to_sym : :verified
+        # Default to the committed snapshot — survey-193k/ holds compat.jsonl
+        # (the ledger backing the deploy) and meta.jsonl (the PG-dump-derived
+        # full per-gem metadata, 193k entries). `survey-out/` is a working
+        # directory for per-run probes; not what the public catalog renders from.
+        ledger_path = (l = argv.index("--ledger")) ? argv[l + 1] : "survey-193k/compat.jsonl"
+        meta_path = (m = argv.index("--meta")) ? argv[m + 1] : "survey-193k/meta.jsonl"
+        site = Site.new(ledger: Ledger.new(path: File.expand_path(ledger_path)),
+                        meta_path: File.expand_path(meta_path))
+        dir = site.build(File.expand_path(out), store: store, min_verdict: min)
+        @out.puts "built spinelgems.org site -> #{dir}" \
+                  "#{store ? " (+ Compact Index from #{store})" : ' (presentation + catalog; pass --store DIR to add the Compact Index)'}"
+        0
+      end
+
+      # Materialize the catalog into a read-only SQLite DB for the dynamic (Tep)
+      # server. Same ledger/meta defaults as build-site; reuses Site#rows.
+      def cmd_build_db(argv)
+        require_relative "site"
+        out = (j = argv.index("--out")) ? argv[j + 1] : raise(Error, "build-db needs --out DB")
+        ledger_path = (l = argv.index("--ledger")) ? argv[l + 1] : "survey-193k/compat.jsonl"
+        meta_path = (m = argv.index("--meta")) ? argv[m + 1] : "survey-193k/meta.jsonl"
+        site = Site.new(ledger: Ledger.new(path: File.expand_path(ledger_path)),
+                        meta_path: File.expand_path(meta_path))
+        db = site.build_db(File.expand_path(out))
+        @out.puts "built catalog DB -> #{db}"
+        0
+      end
+
+      # Render the historical-record page (verdict-mix timeline + deltas across
+      # the per-rev corpus snapshots).
+      def cmd_build_history(argv)
+        require_relative "history"
+        out = (j = argv.index("--out")) ? argv[j + 1] : raise(Error, "build-history needs --out FILE")
+        f = History.new(Dir.pwd).build_html(File.expand_path(out))
+        @out.puts "built history -> #{f}"
+        0
+      end
+
+      # Serve the static site + (with --store) the Compact Index from one process
+      # — what the deploy host runs. Port defaults to $PORT (Upsun) then 9292.
+      def cmd_server(argv)
+        require_relative "server"
+        pub = (i = argv.index("--public")) ? argv[i + 1] : "public"
+        port = (j = argv.index("--port")) ? argv[j + 1].to_i : Integer(ENV.fetch("PORT", "9292"))
+        store = (k = argv.index("--store")) ? File.expand_path(argv[k + 1]) : nil
+        min = (m = argv.index("--min")) ? argv[m + 1].to_sym : :verified
+        Server.new(public_dir: File.expand_path(pub), store: store, min_verdict: min).run(port: port)
+        0
+      end
+
       def cmd_ledger(argv)
         rev = (i = argv.index("--rev")) ? argv[i + 1] : nil
         Ledger.new.each do |v|
@@ -166,19 +358,27 @@ module Bundler
         jobs = (i = argv.index("--jobs")) ? argv.delete_at(i + 1).to_i.tap { argv.delete_at(i) } : 4
         out_file = (j = argv.index("--out")) ? argv.delete_at(j + 1).tap { argv.delete_at(j) } : nil
         list = (k = argv.index("--list")) ? argv.delete_at(k + 1).tap { argv.delete_at(k) } : nil
+        # --refresh: ignore cache hits in the ledger and re-probe every gem.
+        # Default (no flag) reuses any existing verdict at the current engine rev
+        # — so pointing SPINEL_COMPAT_LEDGER at the canonical ledger gives a
+        # cross-run incremental survey (only new gems get the compile cost).
+        refresh = !!argv.delete("--refresh")
         names = if list
                   File.readlines(File.expand_path(list)).map(&:strip).reject { |l| l.empty? || l.start_with?("#") }
                 else
                   argv.dup
                 end
-        raise Error, "usage: spinel-compat survey GEM... | --list FILE [--jobs N] [--out report.md]" if names.empty?
+        raise Error, "usage: spinel-compat survey GEM... | --list FILE [--jobs N] [--out report.md] [--refresh]" if names.empty?
 
-        survey = Survey.new(jobs: jobs)
+        survey = Survey.new(jobs: jobs, refresh: refresh)
         survey.run(names)
         report = survey.report(names)
         if out_file
-          File.write(File.expand_path(out_file), report)
-          @out.puts "wrote #{out_file} (#{names.size} gems)"
+          out_path = File.expand_path(out_file)
+          File.write(out_path, report)
+          tsv_path = File.join(File.dirname(out_path), "candidates.tsv")
+          File.write(tsv_path, survey.candidates_tsv(names))
+          @out.puts "wrote #{out_file} + candidates.tsv (#{names.size} gems)"
         else
           @out.puts report
         end
@@ -207,6 +407,8 @@ module Bundler
           spinel-compat — Spinel gem-compatibility ledger
 
             spinel-compat engine                 show detected compiler + engine rev
+            spinel-compat install-engine [REV]    fetch+build the Spinel compiler -> ~/.cache/spinel
+            spinel-compat init [DIR]              scaffold a Spinel+Tep project (Gemfile, app.rb, bin/build)
             spinel-compat probe NAME [VERSION]    probe one gem, record a verdict
             spinel-compat verify NAME [--smoke F]  differential CRuby-vs-Spinel run -> verified
             spinel-compat vendor [LOCK] [--into D] place deps where Spinel finds them + deps.rb
@@ -214,7 +416,11 @@ module Bundler
             spinel-compat survey GEM... | --list F  wholesale review -> reason histogram
             spinel-compat serve --store DIR        curated source (only vetted gems)
             spinel-compat build-index --store DIR --out DIR   static curated index
+            spinel-compat build-site --out DIR [--store DIR]  static site (presentation + catalog [+ index])
+            spinel-compat server --public DIR [--store DIR]   serve site + Compact Index (one process; $PORT)
             spinel-compat ledger [--rev REV]      dump recorded verdicts
+            spinel-compat diff REV_A REV_B [--names]  per-gem verdict changes between two revs
+            spinel-compat detect-ext GEM_DIR [--out F]  draft spinel-ext.json from a gem's ffi_cflags markers
             spinel-compat reprobe                 re-probe known gems under current rev
 
           Verdicts: ✓ clean   ★ verified   ~ risky   ✗ rejected

data/lib/bundler/spinel/engine.rb

@@ -18,15 +18,25 @@ module Bundler
     class Engine
       attr_reader :dir, :bin
 
-      def initialize(dir: ENV.fetch("SPINEL_DIR", File.expand_path("~/spinel")))
-        @dir = dir
+      def initialize(dir: nil)
+        # Resolution order: explicit SPINEL_DIR, then an engine provisioned by
+        # `spinel-compat install-engine` (~/.cache/spinel/current), then ~/spinel.
+        @dir = dir || ENV["SPINEL_DIR"] || default_dir
         # Prefer a checkout at `dir` (gives a git rev); otherwise fall back to a
         # `spinel` on PATH (installed binary — rev becomes a binary hash). This
         # makes the default work for most setups without configuration.
-        local = File.join(dir, "spinel")
+        local = File.join(@dir, "spinel")
         @bin = File.executable?(local) ? local : (which("spinel") || local)
       end
 
+      # The provisioned-engine cache (install-engine) if it has a built binary,
+      # else the conventional ~/spinel. Keeps zero-config working after a
+      # `spinel-compat install-engine`.
+      def default_dir
+        cur = File.expand_path("~/.cache/spinel/current")
+        File.executable?(File.join(cur, "spinel")) ? cur : File.expand_path("~/spinel")
+      end
+
       def available?
         File.executable?(@bin)
       end
@@ -76,7 +86,10 @@ module Bundler
         # Not a git checkout: hash the binary so distinct builds get distinct keys.
         return "missing" unless available?
 
-        "bin:#{Digest::SHA256.file(@bin).hexdigest[0, 12]}"
+        # Fully-qualified: inside `module Bundler`, a bare `Digest` resolves
+        # to Bundler's own `Bundler::Digest` (no SHA256), so probe/check
+        # NameError'd. Reach the stdlib digest explicitly.
+        "bin:#{::Digest::SHA256.file(@bin).hexdigest[0, 12]}"
       end
 
       def which(cmd)