gemkeeper 0.7.2 → 0.8.0

data/specs/20260529-131354-sync-serve-cache-contract/critique-consolidated-v-1.md

@@ -0,0 +1,95 @@
+# Spec 20260529-131354: Consolidated Critique (v1)
+
+## Overview
+
+**Critiques received from:** Claude, Codex, Copilot (claude-sonnet-4.6)
+**Critiques missing:** Gemini (not installed; Copilot used as the third critic)
+
+## Executive Summary
+
+All three critics agree the spec targets the right bug with the right shape (server-authoritative skip + artifact re-upload, uploader-seam placement, both deployment modes preserved). But they converge on one finding that **undercuts the chosen mechanism**: `GET /info/<name>` is *not* a private-store-authoritative signal. `CompactIndexServer#serve_info` falls back to `serve_upstream_info` (rubygems.org) when the gem isn't in the private `GemIndex` (`compact_index_server.rb:74`). That breaks the presence check two ways:
+
+1. **Public name-collision false positive** — a public gem sharing the private gem's name/version makes `/info` return 200, so `sync` wrongly skips while the private store is still missing the artifact (Codex risk 1, Claude point 2).
+2. **Offline failure / slow recovery** — a missing private gem triggers an *upstream* probe with 5s/10s timeouts, returning **503** (not 404) when offline. The spec only treats 404 as "not present," and 503 is exactly the recovery scenario the spec exists to fix (Copilot MR-2, Codex risk 2).
+
+This means **Q1 (the cache-check mechanism) needs to be reconsidered**: "reuse `/info`, no new server surface" is not actually authoritative. The fix is either a private-store-only signal (small dedicated endpoint, or a flag/header on the existing one) or a checksum comparison. The dedicated-endpoint option I originally dismissed is now the better-justified path.
+
+Beyond that, the strongest convergent gaps are the `version: latest` contradiction, platform filenames, the `build_and_upload` decomposition, and HTTP-status handling.
+
+## Consolidated Requirements Feedback
+
+### A. `/info` is not private-authoritative (mechanism flaw) — HIGHEST PRIORITY
+**Issue:** The presence check must reflect only the private uploaded store, never the upstream-proxied merge.
+**Agreement:** All three. Codex and Copilot independently trace the `serve_upstream_info` fallback; Claude flagged the public-name-collision edge.
+**Divergence:** Mechanism. Codex: add a private presence contract *or* a checksum-based rule. Copilot: at minimum treat 503 as not-present. Claude: pin an AR that presence is read only from the private index.
+**Recommendation:** Reverse the Q1 decision toward a **private-store-only presence signal**. Cleanest: a tiny read-only endpoint that consults `GemIndex` only (e.g. `GET /gemkeeper/has/<name>/<version>` → 200/404, or `HEAD /gems/<file>` wired to the private store), returning unambiguous present/absent with no upstream probe. This also removes the private-name leak to rubygems.org and the offline-timeout problem in one move. If avoiding a new endpoint is still preferred, define a checksum rule and explicit 503-means-not-present handling — but the endpoint is simpler and more correct.
+
+### B. `version: latest` contradicts "never re-clone"
+**Issue:** Goals promise recovery never re-clones, but `latest` resolves its version only post-checkout, so it must clone first.
+**Agreement:** All three (Claude point 4, Codex risk 3, Copilot MR-1/PI-1).
+**Recommendation:** Pick one and write it down: (a) scope the no-reclone guarantee to pinned + `from_lockfile` versions, and state `latest` keeps today's always-fetch behavior; or (b) for `latest` with a local artifact, read the version from the artifact via `Gem::Package.new(path).spec.version` to avoid cloning. (a) is the smaller, safer change and matches today's `!gem_def.latest?` cache bypass; recommend (a) unless offline `latest` recovery is a stated requirement.
+
+### C. Platform filenames
+**Issue:** Presence is framed as `(name, version)`, but artifacts/served files can be `<name>-<version>-<platform>.gem` (`SpecMapper.filename`).
+**Agreement:** Claude point 1, Codex risk 4.
+**Recommendation:** Either declare private gems pure-Ruby (filename `<name>-<version>.gem`) as an explicit assumption, or make presence/artifact lookup operate on the exact filename including platform. Given these are internal source-built gems, the pure-Ruby assumption is likely fine — but it must be stated.
+
+### D. Artifact integrity before re-upload
+**Issue:** FR-1.3 re-uploads a local `.gem` without validating it; a partial/corrupt artifact (interrupted build) would fail server-side in `GemIndex#add` (`Gem::Package.new(...).spec`).
+**Agreement:** Codex (missing req), Copilot EH-3.
+**Recommendation:** Require a pre-upload integrity/identity check — parse the artifact's spec and confirm name+version match the requested gem before declaring success — or explicitly scope corrupt-artifact handling out. Recommend the lightweight check; it also closes Codex's "upload the wrong thing" concern.
+
+### E. `build_and_upload` decomposition unspecified
+**Issue:** FR-1.3 needs an upload-without-build path, but `GemSyncer#build_and_upload` does both unconditionally and the spec doesn't name the structural split.
+**Agreement:** Copilot MR-3, Codex "refactoring GemSyncer ordering."
+**Recommendation:** Add an AR naming the flow: defer repo/manifest resolution until after the server-missing check; separate "upload existing artifact" from "build then upload." This also matters because today `sync` resolves the repo *before* the cache check (`gem_syncer.rb:21`) — ordering must change.
+
+### F. HTTP status handling + counting/messaging
+**Issue:** AR-1.4 covers unreachable + malformed but not the full status matrix (400/500/503), and the CLI summary only knows `:synced`/`:skipped` while the spec introduces a third outcome.
+**Agreement:** All three.
+**Recommendation:** Add a status table: 200→inspect, 404 (and 503-from-upstream-miss, if `/info` is retained)→not present, 400→programming error (raise, not "absent"), connection failure→`ServerNotReachableError`. Define whether artifact re-upload counts as `:synced` (recommended; differentiate via output text only) or a new symbol that `run_sync`/`report_results` must learn.
+
+### G. Faraday connection reuse for GET
+**Issue:** `GemUploader#connection` carries `:multipart`/`:url_encoded` middleware meant for `POST /upload`; reusing it for a GET is wasteful and theoretically fragile.
+**Agreement:** Copilot CA-1.
+**Recommendation:** Acceptable to reuse (note it), or use a plain connection for read requests. Low priority.
+
+### H. `list_gems` vs `has_version?` ambiguity
+**Issue:** AR-1.1's "e.g. `has_version?` / replacing the `list_gems` stub" conflates two different contracts.
+**Agreement:** Copilot AM-1.
+**Recommendation:** State explicitly: add `has_version?(name, version)` (or the chosen private-presence call); leave or remove `list_gems` deliberately, not as a side effect.
+
+### I. Security is not strictly N/A
+**Issue:** The `/info` upstream fallback can leak private gem names to rubygems.org; client doesn't validate the name before building the URL.
+**Agreement:** Codex (FAIL), Copilot (defense-in-depth note).
+**Recommendation:** If the private-endpoint fix (A) is adopted, the leak disappears. Regardless, add a note: client treats 400 as a programming error; gem names come from config/manifest and should match `VALID_NAME`. Downgrade from "N/A" to "low, with these mitigations."
+
+## Additional Requirements Identified
+
+- **AR:** Presence is determined solely from the private index, never an upstream-proxied response (resolves A, I).
+- **AR:** Repo/manifest resolution is deferred until after the server-presence and local-artifact checks (resolves E).
+- **FR:** Before re-uploading an existing artifact, verify its embedded spec name+version match the requested gem (resolves D).
+- **FR/AR:** Define the presence-check HTTP status → outcome mapping (resolves F).
+- **Testing:** `test_gem_syncer.rb` currently tests only `resolve_repo` — `sync()` orchestration tests must be **created**. Add integration coverage for: empty server + local artifact (the original bug), divergent server `gems_path`, offline upstream, upload 409 conflict, and public-name collision (all critics).
+
+## Ambiguities Requiring Clarification
+
+1. **Mechanism for private-authoritative presence** (new endpoint vs. checksum vs. retained `/info` + 503 handling) — the central open decision.
+2. **`latest` scope** — accept always-fetch, or add artifact version-read path.
+3. **Platform** — pure-Ruby assumption or full filename matching.
+4. **Third-outcome counting** — `:synced` vs new symbol.
+
+## Summary of Required Changes
+
+1. **Reconsider Q1:** make the presence check private-store-authoritative (recommend a small read-only private endpoint; eliminates name-collision, offline-503, and name-leak issues at once).
+2. Resolve the `latest` / "never re-clone" contradiction (recommend: scope guarantee to pinned + `from_lockfile`).
+3. State the platform assumption (recommend: private gems pure-Ruby, filename `<name>-<version>.gem`).
+4. Require pre-upload artifact identity check (name+version match).
+5. Specify the `GemSyncer` flow change: defer repo/manifest resolution; split build vs. upload-existing.
+6. Add the HTTP status→outcome table and the third-outcome counting decision.
+7. Resolve `list_gems`/`has_version?`; note Faraday connection choice.
+8. Update testing: create `sync()` tests + the listed integration scenarios; downgrade security from N/A with explicit mitigations.
+
+## Verdict
+
+Right problem, right overall shape, well-bounded scope — but **NEEDS WORK before implementation**, primarily because the recommended `/info` mechanism isn't private-authoritative. That single decision (change A) cascades into the security and offline-503 items. With A resolved plus the `latest`, platform, decomposition, and status-handling tightenings, this is ready to implement.

data/specs/20260529-131354-sync-serve-cache-contract/critique-v-1-claude.md

@@ -0,0 +1,47 @@
+# Critique (v1) — Claude
+
+Spec: Server-authoritative sync cache check (`20260529-131354`)
+
+## Summary
+
+The spec correctly diagnoses the root cause and picks the right core fix (server-authoritative skip via the existing `/info` endpoint, with artifact re-upload to avoid rebuilds). Scope is well-bounded and the deployment-mode constraint (AR-1.3) is the key insight that keeps the design honest. Below are gaps that would cause an implementer to stop and ask, plus a few correctness traps in the actual `/info` and upload mechanics.
+
+## Blocking / should-fix
+
+### 1. `/info` presence parsing is under-specified against the real format (FR-1.2)
+The compact-index info document produced by `CompactIndex.info` is **not** a flat list of versions. Each line is roughly `VERSION DEP:REQ,...|checksum:...,ruby:...`, and the document begins with a `---` header line. FR-1.2 says "contains a version line for that exact version," but doesn't pin down:
+- That the match must be on the **first whitespace-delimited token** of a line, anchored, not a substring (`1.0.5` must not match `1.0.50` or a checksum that happens to contain `1.0.5`).
+- That the leading `---` and any blank lines are ignored.
+- **Platform variants:** a gem can have multiple lines for the same version with different platforms (e.g. `1.0.5` and `1.0.5-x86_64-darwin`). The spec treats presence as `(name, version)` only. If a platformed gem is involved, "version present" may be true while the *specific artifact filename* `<name>-<version>-<platform>.gem` is still missing from the server. Either declare platforms out of scope explicitly, or check the artifact filename, not just the version token.
+
+Recommend FR-1.2 specify anchored first-token matching and state the platform assumption (private gems are pure-Ruby → filename is `<name>-<version>.gem`); if that assumption holds it should be written down, because `SpecMapper.filename` already branches on platform.
+
+### 2. Presence-vs-served gap: `/info` is built from `GemIndex`, but is it the same store the binary is served from? (FR-1.1/FR-1.2)
+The skip decision trusts `/info` to mean "the server can serve `/gems/<file>`." In the current server, `serve_info` reads `@index[gemname]` (from `GemIndex`, i.e. `gems_path/gems`) and `serve_gem_file` reads `@index.gem_path || @cache.gem_binary`. These share `@index`, so they should agree — but the spec should state this invariant explicitly as the thing it depends on: **a version appearing in `/info` implies `/gems/<file>` is serveable from the private store.** If that ever stops holding (e.g. `/info` proxied upstream), the skip becomes wrong. Worth an AR pinning "presence is determined only from the private index, never an upstream-proxied `/info`." Note `serve_info` falls back to `@cache.info(gemname)` (upstream) when the gem isn't private — for a private gem name that the server doesn't have, `/info` would proxy to rubygems.org, 404, and return not-found, which is fine; but a name collision with a public gem could make `/info` return a *public* document and produce a false "present." This edge (private gem sharing a name with a public gem) should be acknowledged.
+
+### 3. "Existing local artifact" lookup location is ambiguous (FR-1.3, AR-1.2)
+FR-1.3 says re-upload when "the corresponding `.gem` already exists in the local `gems_path`." But `cached?` today checks `gems_path/<name>-<bare>.gem` (flat) while the server store is `gems_path/gems/`. The spec should state unambiguously that the artifact lookup is the **flat build-output location** (`gems_path/<name>-<version>.gem`, where `GemBuilder` writes), to avoid an implementer re-introducing the same flat-vs-nested confusion the spec is trying to fix. Tie it to `SpecMapper.filename`/the bare-semver key so the filename is derived one way.
+
+### 4. `version: latest` interaction needs the ordering spelled out (AR-1.2)
+For `latest`, the version isn't known until after clone+checkout (`current_version` post-checkout). So the "ask server first, skip without building" optimization **cannot apply to `latest`** — you must fetch the repo to learn the version before you can query `/info`. The spec acknowledges ordering in AR-1.2 but doesn't state the consequence: `latest` gems always incur a fetch (today they do too — `cached?` is bypassed for `latest` via `!gem_def.latest?` in `sync`). Make explicit that `latest` keeps today's behavior (always fetch, then the server check applies to the resolved version for the *upload/skip* decision), so the skip optimization is for pinned versions only.
+
+## Edge cases / smaller
+
+- **FR-1.4 conflict handling:** `UploadHandler` maps `Errno::EEXIST` → `409 "Gem already exists"`. `GemUploader#handle_response` currently treats `409` as `{ success: true, skipped: true }`. Good — the spec's "treat conflict as skip" already matches code, but FR-1.4 should cite the 409 path so the implementer doesn't change it.
+- **Counting/reporting (FR-1.5):** the `sync` command tallies `:synced`/`:skipped` from `GemSyncer#sync`'s return symbol. Adding a third outcome (artifact re-upload) — is it `:synced` or a new symbol? `report_results` only knows two. Decide whether re-upload counts as `:synced` (simplest, keeps the tally) with differentiated *output text*, or a new `:uploaded` symbol (touches `run_sync`/`report_results`). The spec implies differentiated messaging but not the symbol contract.
+- **Idempotency race (FR-1.4):** "upload that races with an already-present gem" — concurrency isn't really present (sync is sequential), so this is just the 409 path. Reword to avoid implying real concurrency handling is required.
+- **Malformed `/info` → not-present (AR-1.4):** treating malformed as not-present means a flaky/garbage response triggers an upload attempt, which then 409s or 201s harmlessly. That's a safe failure direction; worth stating that "not-present on parse failure" is deliberately biased toward re-uploading rather than skipping.
+- **`reachable?` already exists** on `GemUploader` but `sync` doesn't currently call it; the new presence call effectively becomes the reachability probe. Consider whether the first presence call should produce the not-reachable error early (per-gem vs once up front).
+
+## Testing
+
+- The spec leans on stubbing `/info`. `test_gem_uploader.rb` likely already stubs Faraday — confirm the presence method is testable the same way (it is, if it's on `GemUploader`).
+- Add an integration test that reproduces the original bug: build artifact present, fresh/empty server, `sync` → server serves the gem afterward. This is the regression guard and should be called out as required, not optional.
+
+## Checklist assessment
+
+Honest and well-evidenced. Security N/A is justified (loopback, `VALID_NAME`, read-only `/info`) — but the **private-name-collides-with-public-gem** false-positive (point 2) is a small correctness/security-adjacent edge the checklist's security note should acknowledge. Performance and rollout are appropriately sized.
+
+## Verdict
+
+Sound design, right scope. Resolve the `/info` parsing precision + platform assumption (1), the presence-implies-serveable invariant incl. public-name-collision (2), the artifact-location wording (3), and the `latest` ordering consequence (4) before implementing. The rest are wording tightenings.

data/specs/20260529-131354-sync-serve-cache-contract/critique-v-1-codex.md

@@ -0,0 +1,112 @@
+# Critique: Server-authoritative sync cache check
+
+## Overview
+
+The spec targets the right bug: `sync` currently treats a flat local build artifact as proof that the running compact-index server can serve the gem.
+Moving the skip decision to a server-side check and re-uploading existing artifacts is the right shape, but the proposed `/info/<name>` contract is not yet authoritative enough because that endpoint is merged with the RubyGems upstream cache.
+
+## Approach Summary
+
+- Put the presence check in `GemUploader`, keeping HTTP concerns out of `GemSyncer`.
+- Replace the local `cached?` skip with a server query, then choose skip, upload existing artifact, or build and upload.
+- Reuse existing version normalization for fixed tags, `from_lockfile`, and `latest`.
+- Keep the current `POST /upload` bridge and avoid assuming `sync` and server `gems_path` are the same.
+- The major under-justified choice is "no new server endpoint": current `/info/<name>` is not private-store-only, so using it as the authoritative signal has correctness, privacy, and performance consequences.
+
+## Risks
+
+1. `/info/<name>` can report an upstream public gem, not just a privately uploaded gem.
+   Likelihood: medium.
+   Severity: high.
+   `CompactIndexServer#serve_info` falls back to `serve_upstream_info` when `@index[gemname]` is absent (`lib/gemkeeper/compact_index_server.rb:74`), so a public gem with the same name/version could make `sync` skip even though the private server store is missing the intended artifact.
+   The spec does not address this.
+
+2. Offline recovery can stall or fail on upstream RubyGems lookups.
+   Likelihood: high for the stated offline use case.
+   Severity: high.
+   A missing private gem causes `/info/<name>` to probe RubyGems through `GemCache#info` (`lib/gemkeeper/compact_index_server/gem_cache.rb:20`), with 5s open and 10s read timeouts in `RubygemsClient` (`lib/gemkeeper/compact_index_server/rubygems_client.rb:15`).
+   The spec says 404 means not-present, but offline misses may return 503 after waiting, which conflicts with the goal of fast local recovery.
+
+3. `version: latest` conflicts with the no-git recovery goal.
+   Likelihood: high.
+   Severity: medium.
+   The spec says `latest` must resolve from the checked-out gemspec before presence checking (`spec.md:49`), which requires clone/pull via the existing `GemSyncer` flow (`lib/gemkeeper/gem_syncer.rb:30`).
+   That contradicts the goal that recovery never re-clones when an artifact exists unless the spec explicitly scopes that guarantee to fixed or `from_lockfile` versions.
+
+4. Existing artifact selection can upload the wrong thing or miss valid platform gems.
+   Likelihood: medium.
+   Severity: medium.
+   The spec repeats the current `gems_path/<name>-<version>.gem` shape, but the server stores platform filenames as `<name>-<version>-<platform>.gem` via `SpecMapper.filename` (`lib/gemkeeper/compact_index_server/spec_mapper.rb:13`).
+   It also does not require validating that a reused local artifact's embedded gemspec name/version matches the requested gem before declaring success.
+
+5. Security is not actually N/A.
+   Likelihood: medium.
+   Severity: medium.
+   The client will construct a path from a config/manifest gem name, and `Configuration::GemDefinition` validates version but not name.
+   The server validates names after routing (`lib/gemkeeper/compact_index_server.rb:49`), but the client still needs path-segment escaping and the `/info` fallback can leak private gem names to RubyGems.org.
+
+## Complexity Hotspots
+
+### Making `/info` Authoritative
+
+This is the hardest part.
+The endpoint is a Bundler-facing merged compact index endpoint, not a private-store API.
+If the spec keeps "no new endpoint," it needs a precise rule for distinguishing private presence from upstream presence, probably by comparing the compact-index checksum to the local artifact when one exists or by changing server behavior for sync-specific checks.
+
+### Refactoring `GemSyncer` Ordering
+
+Current `sync` resolves the repo before cache handling (`lib/gemkeeper/gem_syncer.rb:21`) and fetches the repo before `latest_version!`.
+To satisfy "upload existing artifact without git/build," implementation likely must defer repo resolution and `GitRepository` creation until after the server-missing/local-artifact path.
+The spec names the high-level flow but does not call out this ordering change.
+
+### HTTP Status And Parse Semantics
+
+The presence method needs exact status handling: 200 parse, 404 absent, 400 invalid config/name, 503 upstream unavailable, 5xx server failure, redirects, and Faraday connection failures.
+AR-1.4 currently says both "erroring server" maps to `ServerNotReachableError` and "malformed `/info`" means not-present, which leaves important cases open.
+
+### Counting And Messaging
+
+FR-1.4 says an upload conflict is "success/skip," while FR-1.1 says re-uploaded missing gems report as synced, not skipped.
+The existing CLI summary only knows `:synced` and `:skipped` (`lib/gemkeeper/cli/commands/sync.rb:40`), so the spec should say how conflict, re-upload, and freshly built upload affect the summary counts.
+
+## Missing Or Ambiguous Requirements
+
+- Define whether "server reports exact version" means private uploaded gem only, merged private-or-public compact index entry, or matching checksum for the exact artifact.
+- Specify how `GET /info/<name>` names are encoded and what happens when the server returns 400 for an invalid name.
+- Clarify whether 503 from upstream miss while the local server is otherwise healthy should mean not-present or hard failure.
+- State whether local artifact reuse requires reading the `.gem` spec and verifying expected name/version/platform before upload.
+- Specify platform gem behavior: exact filename lookup, multiple artifacts for one name/version, and whether any platform version is considered present.
+- Clarify the `latest` guarantee: either accept that it still needs git to discover the current version or define a local-artifact discovery rule for latest.
+- Require deferring repo and manifest resolution when fixed-version or lockfile-version artifact upload can succeed without source checkout.
+- Define how upload conflicts are counted in the sync summary.
+- Add acceptance coverage for a server whose `/info` would proxy upstream, not only a stubbed 404.
+
+## Completeness Checklist Audit
+
+| Item                         | Status | Notes |
+|------------------------------|--------|-------|
+| Scope & acceptance criteria  | WARN   | Main behavior is clear, but `latest`, platform artifacts, and public upstream collisions are not bounded. |
+| Testing strategy             | WARN   | Needs `test_gem_syncer.rb` flow tests, CLI summary updates, upstream 503/offline tests, public name collision tests, and platform/corrupt artifact cases. |
+| Existing patterns            | WARN   | Correctly uses `GemUploader`, but misses current `GemSyncer` repo-resolution ordering and `CompactIndexServer` upstream fallback behavior. |
+| Dependencies                 | PASS   | No new library dependency is needed. |
+| Architecture & interfaces    | WARN   | Uploader seam is right, but `/info` is not currently a private-store interface. |
+| Error handling & failures    | WARN   | Unreachable server is covered, but HTTP status mapping and upstream 503 are underspecified. |
+| Security review              | FAIL   | The N/A claim misses private-name leakage to RubyGems.org, client path encoding, and local artifact validation. |
+| Performance impact           | FAIL   | The spec assumes one cheap loopback GET, but misses per-gem upstream lookups and offline timeout behavior. |
+| Rollout & migration          | PASS   | No data migration is needed and existing stores can remain in place. |
+| Assumptions & risks          | WARN   | Identifies `/info` parsing risk, but misses that `/info` is merged/proxied rather than private authoritative. |
+
+## Verdict
+
+NEEDS WORK.
+
+The intended sync behavior is implementable, but the spec needs to resolve the `/info` authority problem, offline 503 behavior, artifact validation, and `latest` semantics before implementation can proceed without guessing.
+
+## Suggested Next Steps
+
+1. Decide whether the authoritative check must be private-store-only.
+   If yes, either add a private presence contract or specify a checksum-based rule that cannot be fooled by upstream public gems.
+2. Clarify fixed, `from_lockfile`, and `latest` flows separately, including exactly when repo/manifest resolution is required.
+3. Specify local artifact lookup and validation, including platform suffixes and corrupt/wrong gem files.
+4. Define presence-check HTTP status handling and output/counting semantics.
+5. Update the test plan to include syncer-level orchestration tests plus integration coverage for empty server, offline upstream, divergent server path, upload conflict, and public name collision.

data/specs/20260529-131354-sync-serve-cache-contract/critique-v-1-copilot.md

@@ -0,0 +1,169 @@
+# Critique: Spec 20260529-131354 — Server-authoritative sync cache check
+
+Reviewer: GitHub Copilot (claude-sonnet-4.6)
+Date: 2026-05-29
+
+## Summary
+
+The spec is well-scoped and the core idea is sound.
+The major gap is an unresolved contradiction between the Goals and AR-1.2 around `version: latest`, and several smaller gaps in error handling, format parsing, and implementation structure that need tightening before handoff to an implementer.
+
+---
+
+## Missing Requirements
+
+### MR-1: The `version: latest` + clone contradiction is unresolved
+
+The Goals promise "Recovery never re-clones or rebuilds a gem whose `.gem` artifact already exists locally."
+AR-1.2 then acknowledges that for `version: latest`, version resolution happens *post-checkout*, meaning `clone_or_pull` still runs before any server check.
+These two statements directly contradict each other for the case where a `latest` gem is already on the server.
+
+The spec needs to explicitly choose one of:
+(a) The no-reclone goal only applies to non-`latest` gems (add that scope qualifier to the Goals section).
+(b) For `version: latest` with a local artifact, read the version from the `.gem` file via `Gem::Package.new(path).spec.version` to avoid cloning, then check the server — add this as a new code path.
+
+Without a resolution, an implementer will produce inconsistent behavior and the Goals will be technically false for `latest` gems.
+
+### MR-2: 503 response from `/info/<name>` is not addressed
+
+When a private gem has never been uploaded to a running server, `GemIndex#[]` returns `nil` for that name.
+The server then calls `serve_upstream_info`, which returns 503 (`upstream_unavailable`) when offline.
+This is the exact recovery scenario the spec is designed to fix — the gem doesn't exist on the server — yet only 404 is explicitly listed as "not present."
+A 503 from the server during the presence check should also be treated as "not present" (or at minimum documented as an expected response that triggers the upload path), but it is not mentioned in FR-1.2, AR-1.4, or the Assumptions & Risks.
+
+### MR-3: No description of how `build_and_upload` is decomposed
+
+FR-1.3 requires a new code path: upload an existing artifact without building.
+`GemSyncer#build_and_upload` currently performs both steps unconditionally.
+The spec never mentions that this method must be split (e.g., `upload_artifact(gem_path)` vs. `build_gem(local_path, gems_path)`) or what the resulting `GemSyncer` flow looks like.
+An implementer must infer the structural change from the FR alone, which increases the chance of inventing different designs across teams or in agentic handoff.
+
+### MR-4: Test file for `sync()` behavior does not yet exist in `test_gem_syncer.rb`
+
+The Constraints point to `test_gem_syncer.rb` for `GemSyncer` test coverage, and FR-1.3's Verify describes stubs for `GitRepository` and `GemBuilder`.
+The existing `test_gem_syncer.rb` tests only `resolve_repo` — it has no tests for `sync()` at all.
+The spec should acknowledge that new test methods for `sync()` must be added (not just extended), and whether the integration-level Verify for FR-1.3 is better placed in the lifecycle integration test or in a new unit-level test for `GemSyncer#sync`.
+
+---
+
+## Ambiguous Language
+
+### AM-1: "replacing the `list_gems` `NotImplementedError` stub" conflates two different methods
+
+AR-1.1 says to add a presence method "e.g. `has_version?(name, version)` / replacing the `list_gems` `NotImplementedError` stub."
+`list_gems` returns a list; `has_version?` checks existence by name+version.
+They are not substitutes for each other.
+The parenthetical implies `list_gems` would be removed and `has_version?` added in its place, but `list_gems` has its own contract (error message references `gemkeeper list`).
+The spec should state explicitly whether `list_gems` is being removed, renamed, or left alone, and where `has_version?` fits in the public interface.
+
+### AM-2: "version line for that exact version" underspecifies the compact-index format
+
+FR-1.2 says the gem is present when "the info document contains a version line for that exact version."
+The compact-index `/info` format emitted by `CompactIndex.info(versions)` includes a `---` separator and lines structured as `<version> <platform> <checksum>|<deps>`.
+"Version line" is not defined.
+The Assumptions & Risks note does say "parsing only the leading version token per line," which is a workable rule, but it belongs in FR-1.2 as a normative requirement, not only as a risk mitigation note.
+The spec should state explicitly: skip lines starting with `---` or `created_at:`; split on whitespace; compare the first token against the bare semver.
+
+### AM-3: "present only if the server reports that exact `<name>` and `<version>`" — case sensitivity unspecified
+
+Gem names in the compact-index are case-sensitive by convention, but the spec does not state whether the comparison is case-sensitive.
+This matters if a name is stored differently in the manifest vs. what the server indexes.
+A one-line clarification ("comparison is case-sensitive, matching the gem name exactly as configured") would prevent a subtle bug.
+
+---
+
+## Codebase Assumptions That May Be Wrong
+
+### CA-1: The `GemUploader` Faraday connection uses `multipart` middleware, which is unnecessary for a plain GET
+
+The existing `connection` method in `GemUploader` builds a Faraday connection with `:multipart` and `:url_encoded` middleware.
+Both are only relevant for `POST /upload`.
+A `GET /info/<name>` on that same connection is harmless but wasteful (adds a `Content-Type: multipart/form-data` header that is ignored by the server).
+The spec should either (a) note this is acceptable and leave it as-is, or (b) instruct the implementer to use a separate plain connection for read-only requests.
+Ignoring this risks the presence check failing on edge-case Rack middleware that rejects multipart headers on GETs.
+
+### CA-2: `latest_version!` raises `BuildError`, but the new flow may need a softer return
+
+`latest_version!` in `GemSyncer` raises `BuildError` if `current_version` returns `nil`.
+With the new server-check path for `version: latest`, there is a question of whether the existing exception should propagate before or after the server presence check.
+The spec does not address what happens if `current_version` returns `nil` and the server already has the gem (build failure, but gem is present — should it be a skip or an error?).
+
+### CA-3: `GemIndex#add` raises `Errno::EEXIST` on conflict — not HTTP 409
+
+AR-1.4 says "an upload that races with an already-present gem (server returns the existing-gem conflict) is treated as success/skip."
+`GemUploader#handle_response` already handles 409 as `{ success: true, skipped: true }`, so this is correct at the HTTP layer.
+However, the spec says the *presence check* comes before the upload.
+After a successful presence check shows the gem is absent, the gem could be uploaded by another process before this process's upload completes — the race actually produces a 409 at upload time, not a "missing" from the presence check.
+The spec's idempotency claim in FR-1.4 is correct by accident (existing 409 handling covers it), but the causal chain described is subtly wrong and could confuse an implementer.
+
+---
+
+## Error Handling & Edge Cases
+
+### EH-1: HTTP status codes other than 200, 404 are unspecified for the presence check
+
+AR-1.4 addresses "unreachable or erroring server" and "malformed `/info` response" but not intermediate HTTP errors: 400 (invalid name), 500, 503.
+A 503 is the expected response when the gem is absent and the server is offline (see MR-2 above).
+A 400 would indicate a bug in name validation, which is a different class of problem.
+The spec should enumerate what status codes are treated as "not present" vs. "raise `ServerNotReachableError`" vs. "raise a different error."
+
+### EH-2: Connection failure during presence check timing vs. upload timing
+
+The spec says a connection failure during the presence check raises `ServerNotReachableError`.
+But the current `GemSyncer#sync` does no upfront reachability check; it discovers unreachability only when `upload` is called.
+With the new flow, the error surface moves earlier (before any git work), which is an improvement.
+However, the spec should note whether this earlier detection changes any user-visible behavior (e.g., error message wording, exit code).
+Currently `GemUploader#upload` and the proposed `has_version?` would raise the same exception class, but the message may differ.
+
+### EH-3: What if the local artifact is corrupt?
+
+FR-1.3 says when the server is missing the gem and the local `.gem` exists, upload it directly.
+The spec does not address what happens if the local artifact is a partial/corrupt file (e.g., interrupted build).
+The server's `GemIndex#add` reads the file via `Gem::Package.new(source_path).spec` and will raise `StandardError` on a corrupt gem, which `UploadHandler` likely does not gracefully convert to a 4xx.
+The spec should either require a pre-upload integrity check (`Gem::Package.new(path).spec` before upload) or explicitly acknowledge this as out of scope.
+
+---
+
+## Security Concerns
+
+The spec's security review says "N/A beyond existing posture."
+This is broadly correct given the loopback-only deployment, but there is one minor gap worth noting:
+
+`GemUploader` currently does not validate the gem name before constructing the `/info/<name>` URL.
+On the server side, `VALID_NAME` (line 17 of `compact_index_server.rb`) rejects names that don't match `[a-zA-Z0-9._-]+` and returns 400.
+The client should treat a 400 response as a programming error (not a "not present" outcome) rather than silently proceeding.
+This is a defense-in-depth note, not a new attack surface, but it is relevant to EH-1 above.
+
+---
+
+## Performance Implications
+
+### PI-1: `version: latest` still always clones/pulls before the skip can fire
+
+For `version: latest`, the server presence check can only fire after `clone_or_pull` and version resolution.
+If the intent is fast recovery (re-run sync, get back to work quickly), `version: latest` gems will still incur a network round-trip to the git remote before they are skipped.
+The spec acknowledges this implicitly in AR-1.2 but does not flag it as a known cost or consider mitigations (e.g., reading the version from the existing artifact).
+This is worth calling out explicitly so the implementer does not optimize away the clone for `latest` and inadvertently break version freshness.
+
+---
+
+## Spec Completeness Checklist — Gaps
+
+| Item | Status | Issue |
+| ---- | ------ | ----- |
+| Scope & acceptance criteria | Partial | Goals "never re-clones" is untrue for `version: latest` (MR-1) |
+| Testing strategy | Partial | `test_gem_syncer.rb` has no `sync()` tests; spec implies extension, not creation (MR-4) |
+| Error handling & failure modes | Partial | 503 and other non-200/404 statuses unspecified (MR-2, EH-1) |
+| Assumptions & risks | Partial | 503/offline risk for never-uploaded private gems not listed; corrupt artifact not listed |
+| Architecture & interfaces | Partial | `build_and_upload` decomposition not specified (MR-3); `list_gems` ambiguity (AM-1) |
+
+---
+
+## Recommended Changes Before Implementation
+
+1. Clarify the `version: latest` goal: either scope "never re-clones" to non-`latest` gems, or add the `Gem::Package` version-read path as an explicit alternative to cloning (MR-1).
+2. Add 503 to FR-1.2 as a "not present" signal, or explain why it should be an error (MR-2).
+3. Replace the risk-mitigation note about `/info` parsing with a normative parsing rule in FR-1.2 (AM-2).
+4. Specify the `build_and_upload` decomposition or name the resulting private methods (MR-3).
+5. Clarify whether `list_gems` is removed or retained, and add `has_version?` as a standalone addition (AM-1).
+6. Add EH-1's status-code table (200 = check version, 404/503 = not present, others = raise).

data/specs/20260529-131354-sync-serve-cache-contract/implementation-summary.md

@@ -0,0 +1,59 @@
+# Implementation Summary: 20260529-131354-sync-serve-cache-contract
+
+**Status:** Completed
+**Date:** 2026-05-29
+
+## Overview
+
+Made `gemkeeper sync`'s skip decision authoritative against the server's *private* store instead of a local build artifact, fixing the bug where a fresh or repointed server could never be repopulated (every gem skipped, 404s on `bundle install`). Added a private-store-only presence endpoint, server-authoritative `serves?` on the uploader, and reordered `GemSyncer` to defer repo work — skipping when the server already has a gem and re-uploading an existing artifact without rebuilding.
+
+## Execution
+
+Solo, sequential (the client depends on the server endpoint contract): server endpoint → client `serves?` → `GemSyncer` reorder → tests → quality-gate refactor.
+
+## Files Created
+- `lib/gemkeeper/repo_fetcher.rb` — `RepoFetcher`: resolves a gem's repo URL (manifest + override) and clones/pulls it, with git-auth error mapping. Extracted from `GemSyncer` to keep it under the rubycritic gate.
+- `test/gemkeeper/test_repo_fetcher.rb` — repo-resolution tests (migrated from `test_gem_syncer.rb`).
+
+## Files Modified
+- `lib/gemkeeper/compact_index_server.rb` — new `GET /gemkeeper/has/<name>/<version>` route (`serve_presence`) + `present` responder; reads the private index only.
+- `lib/gemkeeper/compact_index_server/gem_index.rb` — `serves?(name, version)` predicate over the in-memory index.
+- `lib/gemkeeper/gem_uploader.rb` — `serves?(name, version)` (hits the private endpoint; 200/404/else→`ServerError`; connection failure→`ServerNotReachableError`); extracted `not_reachable!` helper.
+- `lib/gemkeeper/gem_syncer.rb` — reordered: `sync_pinned` (server check → artifact reuse → build) / `sync_latest` (always fetch); `reusable_artifact?` identity check; delegates repo acquisition to `RepoFetcher`.
+- `lib/gemkeeper.rb` — require `repo_fetcher`.
+- `test/gemkeeper/test_compact_index_server.rb` — 4 presence-endpoint tests.
+- `test/gemkeeper/test_gem_uploader.rb` — `serves?` connection-failure test.
+- `test/gemkeeper/test_gem_syncer.rb` — `reusable_artifact?` (4) + `sync()` orchestration (2) tests; resolve tests moved out.
+- `test/integration/test_server_lifecycle_integration.rb` — original-bug regression + skip-when-served tests.
+- `test/integration/test_cli_integration.rb` — repurposed the obsolete local-cache-skip test to assert the new unreachable-server error.
+- `CHANGELOG.md` — `[Unreleased]` Fixed entry.
+
+## Test Results
+`bundle exec rake test` → 230 runs, 512 assertions, 0 failures, 0 errors.
+`bundle exec rubocop` → 76 files, no offenses.
+`bundle exec rubycritic lib --no-browser` → 90.13 (gate ≥ 90); `gem_syncer` B, `repo_fetcher` A, no C/D/F.
+
+## Spec Adherence
+| Requirement | Status | Implementation | Test |
+|-------------|--------|---------------|------|
+| FR-1.1 | Done | `gem_syncer.rb` `sync_pinned`/`sync_latest` via `@uploader.serves?` | `test_sync_skips_when_server_already_serves` (unit + integration) |
+| FR-1.2 | Done | `gem_uploader.rb` `serves?` hits `/gemkeeper/has` | `test_gem_uploader`, presence endpoint tests |
+| FR-1.3 | Done | `reusable_artifact?` + `reupload` | `test_sync_reuploads_existing_artifact_without_rebuild`, regression |
+| FR-1.4 | Done | `GemUploader#handle_response` 409→skip | `test_sync_skips_when_server_already_serves` (runs sync twice) |
+| FR-1.5 | Done | `skip`/`reupload`/`build_gem` symbols + output text | syncer unit tests, integration `1 skipped` |
+| FR-1.6 | Done | `sync_latest` always fetches | covered by `sync_latest` path |
+| FR-2.1 | Done | `compact_index_server.rb` `serve_presence` + `GemIndex#serves?` | `test_presence_endpoint_*` (incl. no-upstream-probe) |
+| FR-2.2 | Done | `serve_presence` `VALID_NAME` check → 400 | `test_presence_endpoint_rejects_invalid_name` |
+| AR-1.1 | Done | `serves?` on `GemUploader`; `list_gems` untouched | `test_gem_uploader` |
+| AR-1.2 | Done | reuse `resolve_version`; `<name>-<version>.gem` | syncer tests |
+| AR-1.3 | Done | presence strictly over HTTP | integration (real server) |
+| AR-1.4 | Done | status mapping in `serves?` | `test_serves_raises_server_not_reachable_on_connection_failure`, CLI unreachable test |
+| AR-1.5 | Done | `RepoFetcher` deferral; build/upload split | unit stubs assert no fetch/build on skip/reuse |
+| AR-2.1 | Done | endpoint reads `@index` only; gates green | rubycritic 90.13 |
+
+## Deviations from Spec
+- **`RepoFetcher` extraction (not in the spec's file list).** The `GemSyncer` reorder added methods and pushed it to a rubycritic C (120); reek flagged `resolve_repo` and the auth helpers as belonging elsewhere. Extracting `RepoFetcher` both satisfied the spec's quality-gate constraint and is the cleaner boundary (sync orchestrates; `RepoFetcher` acquires the repo). The 6 repo-resolution tests moved with it. No behavior change.
+- **`test_cli_integration#test_sync_skips_already_cached_gem` repurposed.** It asserted the old local-cache skip (no server) that the spec deliberately removes; rewritten as `test_sync_errors_when_server_unreachable` to assert the new server-authoritative guard (AR-1.4).
+
+## Living docs
+`specs/docs/` does not exist — skipped. Run `/spec-docs --full` to bootstrap if desired.