scip-query 0.19.5 → 0.19.8

package/docs/CONFIGURATION_WRITE_SAFETY.md

@@ -0,0 +1,130 @@
+# Configuration and setup write safety
+
+scip-query updates files that people and other agents may edit at the same
+time: `.scipquery.json`, provider hook JSON, `.git/info/exclude`,
+`AGENTS.md`, `CLAUDE.md`, structured suppression records, and an owned
+pre-commit hook. A conflict-aware writer is a file updater that transforms one
+identified revision and refuses to claim success if an independent revision
+wins before commit. Its defining behavior is preservation: it either applies
+its narrow change to the newest valid input or leaves the newest input
+untouched and reports why.
+
+## Revision and commit protocol
+
+Every participating writer:
+
+1. acquires a short, token-owned process lock beside the target;
+2. reads a stable snapshot and records its SHA-256 hash plus file identity;
+3. computes only the domain change it owns;
+4. rereads the target immediately before commit;
+5. retries the merge when the revision changed and retry is safe, or reports a
+   conflict when it is not;
+6. stages and flushes complete bytes, atomically publishes them, and flushes
+   the parent directory where the platform exposes that operation.
+
+The lock serializes scip-query processes. Editors do not need to participate:
+their byte or identity change is detected by the optimistic revision check.
+The lock record is token-owned and carries process identity, so a live owner
+cannot be displaced and a dead owner can be reclaimed by the shared process
+lock protocol.
+
+First creation uses a flushed staging inode plus an exclusive public hard
+link. This is stronger than `exists` followed by replacement: only one creator
+can publish, and a reader sees no public file until every byte is present.
+
+## JSON merge rules
+
+Project setup rereads the latest valid object and changes only the requested
+field. Unknown current and future fields survive. A three-way check compares
+the caller's observed value, the latest value, and the requested value:
+
+- an unrelated latest edit is preserved and the owned field is updated;
+- the requested value already present is idempotent;
+- a different latest value for the same field is an explicit stale-field
+  conflict.
+
+Hook setup rereads the latest valid provider object on every bounded retry,
+removes only scip-query-owned hook entries, and merges the current owned hook
+groups. Unknown top-level fields and non-scip hook entries survive installation
+and removal.
+
+Malformed latest JSON is never repaired by replacement because doing so could
+erase information the writer cannot classify. The command reports the parse
+failure and leaves the exact bytes in place.
+
+### Project-config format boundary
+
+The project configuration is a durable policy record whose `schemaVersion`
+determines the meaning of its other fields. Current writers publish version 2
+and include a `$schema` URI reference to
+`docs/schemas/project-config.schema.json` in the installed package.
+Unversioned files and explicit `schemaVersion: 1` files have the same readable
+legacy meaning. They are migrated in memory and written as version 2 only when
+`init` creates a file or a setup action enters an authorized mutation path.
+
+The schema migration is part of the no-op decision. If the requested field is
+already correct but the durable record is legacy or lacks its editor-schema
+hint, the writer still publishes a current record and reports `changed: true`.
+Unknown root and nested fields survive because migration begins from the
+latest complete object and removes only the owned legacy discriminator.
+
+A non-integer discriminator, non-object top level, invalid `$schema` hint, or
+unsupported older/future version fails before options are exposed to runtime
+consumers. The loader names the supported legacy/current versions. A setup
+writer applies the same decoder to its latest stable snapshot and leaves
+rejected bytes byte-for-byte unchanged, so upgrading the CLI—not a blind
+rewrite—is the recovery path for a future version.
+
+## Suppression policy rules
+
+A suppression identity is the stable finding ID, or the deterministic hash of
+a check-and-file target when no finding ID exists. It is a policy conflict
+domain: decisions for different identities occupy different files, while two
+decisions for the same identity must be reconciled.
+
+The first decision uses exclusive durable creation. An identical replay
+returns the existing revision without rewriting metadata. A different reason,
+expiry, check, or file requires `--replace <revision>`, where the revision is
+the full SHA-256 hash reported when the existing decision was rejected or
+created. Replacement succeeds only if those exact reviewed bytes still occupy
+the path. A stale token, a malformed record, an unsupported future schema, or
+an edit at the commit boundary leaves the latest bytes untouched.
+
+New records use suppression schema version 1 and include the
+`scip-query-suppression` discriminator, their stable identity, and the
+`scip-query` writer version. Unversioned legacy records and v1 records written
+before the discriminator was added remain readable. They remain byte-for-byte
+unchanged on an idempotent replay and are upgraded only by an explicit
+compare-and-replace policy change. Incompatible files are counted and reported
+by `diff-gate`; they never authorize a suppression. See
+[`COMMITTED_RECORD_COMPATIBILITY.md`](COMMITTED_RECORD_COMPATIBILITY.md).
+
+## Managed text rules
+
+Agent guidance changes only the text between the exact
+`scip-query:agent-setup` markers. Text before and after the block is preserved.
+Missing markers permit first installation, but incomplete, duplicated, or
+reordered markers are a conflict because the intended ownership boundary is
+ambiguous.
+
+Managed Markdown and owned pre-commit operations use a strict final revision
+check rather than silently recomputing across an intervening edit. The command
+reports the expected and latest revision hashes and does not write.
+`.git/info/exclude` can safely retry because its exact owned marker block is
+recomputed from the newest text.
+
+## Recovery
+
+When a command reports a conflict:
+
+1. open the named file and preserve the latest independent edit;
+2. repair malformed JSON or marker structure deliberately, if reported;
+3. for a suppression policy change, review the latest decision and rerun
+   `suppress` with the reported `--replace <revision>`;
+4. for setup/configuration, rerun the command so it reads the new revision.
+
+Do not delete a live `.scip-query-write.lock`. If its owner crashed, the next
+writer reclaims it only after the shared lock protocol proves the recorded
+process identity is no longer live. A crash before publication leaves the
+previous target intact; a failed first publication leaves no partial public
+file.

package/docs/DETECTOR_GUIDE.md

@@ -13,20 +13,20 @@ maps these to the agent behaviors that create the problems.)
 Same disease — structure built for a future that never came — detected at
 four different altitudes:
 
-| Command | Altitude | What it measures | The fix |
-|---|---|---|---|
-| `unused-params` | **parameter** | Trailing parameters no body ever uses. TS/JS only, trailing-run only — removals that are type-safe by construction. `_`-prefixed and externally-published signatures are exempt. | Delete the parameters and their call-site arguments. |
-| `passthrough-candidates` | **function (fan-out view)** | Functions with exactly **one callee** and a small body — they just forward arguments to the real implementation. | Inline it: call the target directly. |
-| `wrapper-candidates` | **function (fan-in view)** | Symbols with exactly **one caller** — indirection that provides no reuse. Strongest when the sole caller is itself widely used. | Fold the body into the caller. |
-| `stale-abstractions` | **type** | Classes, interfaces, and type aliases with 0–1 *real* cross-file consumers (barrel re-exports don't count as consumers). Single-implementation interfaces, misplaced types. | De-abstract: replace the interface with the concrete thing, or move the type to its one consumer. |
+| Command                  | Altitude                    | What it measures                                                                                                                                                                 | The fix                                                                                           |
+| ------------------------ | --------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- |
+| `unused-params`          | **parameter**               | Trailing parameters no body ever uses. TS/JS only, trailing-run only — removals that are type-safe by construction. `_`-prefixed and externally-published signatures are exempt. | Delete the parameters and their call-site arguments.                                              |
+| `passthrough-candidates` | **function (fan-out view)** | Functions with exactly **one callee** and a small body — they just forward arguments to the real implementation.                                                                 | Inline it: call the target directly.                                                              |
+| `wrapper-candidates`     | **function (fan-in view)**  | Symbols with exactly **one caller** — indirection that provides no reuse. Strongest when the sole caller is itself widely used.                                                  | Fold the body into the caller.                                                                    |
+| `stale-abstractions`     | **type**                    | Classes, interfaces, and type aliases with 0–1 _real_ cross-file consumers (barrel re-exports don't count as consumers). Single-implementation interfaces, misplaced types.      | De-abstract: replace the interface with the concrete thing, or move the type to its one consumer. |
 
 How to keep them straight:
 
 - `passthrough` looks **down** (what does this function call? one thing) —
   `wrapper` looks **up** (who calls this function? one caller). A function can
   be both: a one-line forwarder with a single caller is the purest bloat.
-- `unused-params` is *inside* a signature; the other three are *about whole
-  symbols*.
+- `unused-params` is _inside_ a signature; the other three are _about whole
+  symbols_.
 - `stale-abstractions` is the only one about **types**, not behavior. Note its
   confidence ranking: a single-consumer `class` is usually deliberate
   encapsulation (low), a single-consumer `interface` is worth questioning
@@ -35,17 +35,17 @@ How to keep them straight:
 ## Cluster 2 — "This already exists" (the similarity family)
 
 All of these find duplication, but at different granularities and with
-different evidence — and two of them add *direction*:
-
-| Command | Granularity | Evidence | Question it answers |
-|---|---|---|---|
-| `similar <symbol>` | function | callee-fingerprint cosine (TF-IDF), source-token fallback | "What else does roughly what this function does?" |
-| `similar-signatures` | function | normalized parameter + return types | "What has the same *shape*, regardless of body?" |
-| `similar-files` | file | Jaccard on import/dependency profiles | "Which files are copy-paste variants of each other?" |
-| `similar-chains` | pipeline | edit distance on infrastructure-filtered dependency chains | "Which end-to-end flows are parallel re-implementations?" |
-| `recent-duplicates` | callable/frontend unit + **git age** | callable, React, and Vue similarity + file-add history | "Which side is the established original, which is the fresh echo?" (ECHO = new copies old; TWIN = both new) |
-| `incomplete-migration` | function + **git diff** | callee *containment* vs new-in-diff helpers | "I just extracted a helper — which call sites still have the logic inline and were never migrated?" |
-| `convergence <a> <b>` | a known pair | shared/unique callees | "I already know these two overlap — give me the merge prescription." |
+different evidence — and two of them add _direction_:
+
+| Command                | Granularity                          | Evidence                                                   | Question it answers                                                                                         |
+| ---------------------- | ------------------------------------ | ---------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
+| `similar <symbol>`     | function                             | callee-fingerprint cosine (TF-IDF), source-token fallback  | "What else does roughly what this function does?"                                                           |
+| `similar-signatures`   | function                             | normalized parameter + return types                        | "What has the same _shape_, regardless of body?"                                                            |
+| `similar-files`        | file                                 | Jaccard on import/dependency profiles                      | "Which files are copy-paste variants of each other?"                                                        |
+| `similar-chains`       | pipeline                             | edit distance on infrastructure-filtered dependency chains | "Which end-to-end flows are parallel re-implementations?"                                                   |
+| `recent-duplicates`    | callable/frontend unit + **git age** | callable, React, and Vue similarity + file-add history     | "Which side is the established original, which is the fresh echo?" (ECHO = new copies old; TWIN = both new) |
+| `incomplete-migration` | function + **git diff**              | callee _containment_ vs new-in-diff helpers                | "I just extracted a helper — which call sites still have the logic inline and were never migrated?"         |
+| `convergence <a> <b>`  | a known pair                         | shared/unique callees                                      | "I already know these two overlap — give me the merge prescription."                                        |
 
 How to keep them straight:
 
@@ -56,13 +56,13 @@ How to keep them straight:
   which copy to delete. Run it after agent sessions. It covers generic callables,
   React component structure, React hook behavior, Vue template structure, and
   Vue composable-like behavior.
-- `incomplete-migration` is the **inverse of an echo**: the *new* code is the
-  canonical one (the helper you just extracted), and the *established* code is
+- `incomplete-migration` is the **inverse of an echo**: the _new_ code is the
+  canonical one (the helper you just extracted), and the _established_ code is
   what should disappear. It also scores by containment, not symmetric
-  similarity, because an un-migrated site holds the helper's logic *plus* its
+  similarity, because an un-migrated site holds the helper's logic _plus_ its
   own — cosine under-scores exactly those.
 - `extract-candidates` is the **before** picture: seams inside one big
-  function that *should* become a helper. `incomplete-migration` is the
+  function that _should_ become a helper. `incomplete-migration` is the
   **after** picture: you made the helper but didn't finish moving everyone
   onto it.
 
@@ -71,11 +71,11 @@ How to keep them straight:
 Three detectors share the word "drift" or the concept; they watch different
 gaps:
 
-| Command | Watches the gap between | Evidence |
-|---|---|---|
-| `drift` | a file and its **siblings/declared architecture** | reference graph: unused imports, project-owned forbidden boundary edges, "no sibling imports this" deviations |
-| `doc-drift` | **docs** and the code they describe | doc file-citations + doc↔code co-change history; flags broken references and staleness scores |
-| `co-change` | two **files** with an invisible contract | git history: pairs that change together with no dependency edge |
+| Command     | Watches the gap between                           | Evidence                                                                                                      |
+| ----------- | ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
+| `drift`     | a file and its **siblings/declared architecture** | reference graph: unused imports, project-owned forbidden boundary edges, "no sibling imports this" deviations |
+| `doc-drift` | **docs** and the code they describe               | doc file-citations + doc↔code co-change history; flags broken references and staleness scores                 |
+| `co-change` | two **files** with an invisible contract          | git history: pairs that change together with no dependency edge                                               |
 
 How to keep them straight: `drift` is structural and intra-code, `doc-drift`
 is prose-vs-code, `co-change` is code-vs-code where the connection exists only
@@ -103,11 +103,11 @@ Baseline identities use `detector:file:shortName`. File or symbol renames can le
 
 ## Cluster 4 — "Nothing uses this" (the deadness family)
 
-| Command | Scope | Question |
-|---|---|---|
-| `dead` | symbols | "What has zero consumers?" (evidence-ranked, entrypoint-aware) |
-| `isolated` | callables | "What is fully disconnected — no callers *and* no callees?" |
-| `cleanup-plan` | the cascade | "If I delete the dead stuff, what *becomes* dead next — and will my compiler vouch for the whole batch?" (`--verify`) |
+| Command        | Scope       | Question                                                                                                              |
+| -------------- | ----------- | --------------------------------------------------------------------------------------------------------------------- |
+| `dead`         | symbols     | "What has zero consumers?" (evidence-ranked, entrypoint-aware)                                                        |
+| `isolated`     | callables   | "What is fully disconnected — no callers _and_ no callees?"                                                           |
+| `cleanup-plan` | the cascade | "If I delete the dead stuff, what _becomes_ dead next — and will my compiler vouch for the whole batch?" (`--verify`) |
 
 `dead` finds candidates; `isolated` finds the most extreme subset;
 `cleanup-plan --verify` turns candidates into a compiler-proven deletion plan.
@@ -118,21 +118,21 @@ Don't hand-delete from `dead` output when `cleanup-plan` can prove it.
 ## After-the-change check matrix
 
 The reflex to build (and the one the `scip-query` router skill teaches
-agents): match the check to what the change *did*. `diff-gate` runs the
+agents): match the check to what the change _did_. `diff-gate` runs the
 broad sweep on every diff; these are the targeted follow-ups.
 
-| You just... | Run |
-|---|---|
-| Extracted a helper / created an abstraction | `scip-query incomplete-migration` — did every site migrate? |
-| Wrote a brand-new helper or module | `scip-query similar <it>` and `scip-query recent-duplicates` — did it already exist? |
-| Added parameters, options, or config flags | `scip-query unused-params` — does anything use them yet? |
-| Added a forwarding/wrapper layer | `scip-query wrapper-candidates` and `scip-query passthrough-candidates` — does it earn its indirection? |
-| Added an interface, base class, or type alias | `scip-query stale-abstractions` — does it have more than one real consumer? |
-| Changed a schema, contract, config, or generated file | `scip-query co-change <file>` — who historically moves with it? |
-| Changed code that docs describe | `scip-query doc-drift` — which docs now lie? |
-| Deleted code | `scip-query cleanup-plan --verify` — what else just became dead, and does the compiler agree? |
-| Anything at all, before saying "done" | `scip-query reindex && scip-query diff-gate` |
+| You just...                                           | Run                                                                                                     |
+| ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------- |
+| Extracted a helper / created an abstraction           | `scip-query incomplete-migration` — did every site migrate?                                             |
+| Wrote a brand-new helper or module                    | `scip-query similar <it>` and `scip-query recent-duplicates` — did it already exist?                    |
+| Added parameters, options, or config flags            | `scip-query unused-params` — does anything use them yet?                                                |
+| Added a forwarding/wrapper layer                      | `scip-query wrapper-candidates` and `scip-query passthrough-candidates` — does it earn its indirection? |
+| Added an interface, base class, or type alias         | `scip-query stale-abstractions` — does it have more than one real consumer?                             |
+| Changed a schema, contract, config, or generated file | `scip-query co-change <file>` — who historically moves with it?                                         |
+| Changed code that docs describe                       | `scip-query doc-drift` — which docs now lie?                                                            |
+| Deleted code                                          | `scip-query cleanup-plan --verify` — what else just became dead, and does the compiler agree?           |
+| Anything at all, before saying "done"                 | `scip-query reindex && scip-query diff-gate`                                                            |
 
 And before any non-trivial change: plan with `scip-query plan-context
-<target>` (or the `scip-concrete-plan` skill, which requires a scip-query citation
+<target>` (or the `scip-plan` skill, which requires a scip-query citation
 for every claim in the plan).

package/docs/DURABILITY.md

@@ -0,0 +1,103 @@
+# Filesystem Publication and Durability
+
+This document classifies scip-query's file-backed state by the guarantee a
+writer must provide. The classification is part of the storage contract: a
+caller must choose a writer whose failure semantics match the record's role.
+
+## Guarantees
+
+A **visibility-atomic replacement** is a filesystem publication operation that
+writes complete bytes to a private staging file and renames that file over the
+target. The rename is the decisive characteristic: concurrent readers can
+observe the old complete file or the new complete file, but never the
+writer's partial staging bytes. It does not promise that the new bytes or
+directory entry survive power loss.
+
+A **crash-durable replacement** is a visibility-atomic replacement that also
+flushes the complete staging file before rename and flushes the containing
+directory after rename. Those ordered flushes are what make acknowledged file
+contents and the name that reaches them recoverable after an operating-system
+or machine crash, subject to the host filesystem and device honoring their
+flush contract.
+
+An **authoritative record** is file-backed state whose loss can change which
+generation, owner, policy, or accepted decision the program treats as current.
+Its causal role in choosing later behavior makes silent rollback unsafe, so it
+uses crash-durable replacement.
+
+A **rebuildable record** is file-backed state whose loss can cost time,
+diagnostics, or a retry but cannot make unverified facts authoritative. Because
+another computation or bounded request can reconstruct it, complete
+old-or-new visibility is sufficient.
+
+A **directory flush** is a request to persist a directory entry—the mapping
+from a filename to its file—after the file itself has been flushed. Flushing
+only file contents is insufficient because a crash can otherwise lose the
+rename that made those contents current.
+
+## APIs
+
+| API | Staging identity | File flush | Rename | Parent-directory flush |
+| --- | --- | --- | --- | --- |
+| `writeJsonAtomic` / `replaceFileAtomic(..., { durability: "visibility" })` | Exclusive random token | No | Yes | No |
+| `writeJsonDurable` / `replaceFileAtomic(..., { durability: "durable" })` | Exclusive random token | Yes | Yes | Yes where supported |
+
+`writeJsonAtomic` retains its original `void` return contract for compatibility.
+`writeJsonDurable` and `replaceFileAtomic` return the achieved directory-sync
+status. Verified binary installation owns an equivalent platform-local
+flush/rename sequence because the enforced architecture forbids dependencies
+between the sibling `platform` and `storage` boundaries.
+
+On Windows, Node can reject attempts to open or flush a directory handle.
+Known Windows "directory handles unsupported" errors produce
+`directorySync: "unsupported"` after the staged file itself has been flushed
+and renamed. That result means complete visibility plus flushed file contents,
+not the full POSIX directory-entry durability guarantee. Other directory-sync
+errors remain failures.
+
+## Failure Outcomes
+
+| Failure point | Target visible after return/throw | Owned staging file |
+| --- | --- | --- |
+| Exclusive create | Previous target | No owned file was created |
+| Write or short/invalid progress | Previous target | Closed and removed |
+| File flush | Previous target | Closed and removed |
+| Rename | Previous target | Removed by the writer |
+| POSIX directory flush | New complete target; durability unconfirmed and the call throws | Already renamed; no staging path |
+| Unsupported Windows directory flush | New complete target; result reports the limitation | Already renamed; no staging path |
+
+The post-rename failure row is deliberately explicit. Once rename succeeds,
+rolling back would be another publication with its own crash window. The
+writer therefore leaves the verified new value visible and reports that it
+could not confirm the stronger durability guarantee.
+
+## Call-Site Classification
+
+| Record | Contract | Reason |
+| --- | --- | --- |
+| Reindex `meta.json` | Durable | Names the accepted index status, fingerprint, and generation metadata |
+| SQLite generation manifest and `state.json` | Durable | Flushes one complete immutable artifact set, then atomically selects it for new readers |
+| Shared-generation manifest | Durable file within staging | Authenticates immutable artifacts; the later generation-directory publication remains a separate generation-store operation |
+| Worktree lease and local cache pointer | Durable | Protect generations from collection and bind a worktree to repository cache identity; generation changes, liveness touches, and cleanup serialize through the repository-cache lock |
+| Watch service state | Durable | Publishes the process instance and index generation accepted as current |
+| Rust semantic session `server.json` | Durable | Publishes the live server process and mailbox identity |
+| Project `.scipquery.json` | Durable | Controls indexing, watch, architecture, and detector policy; versioned reads reject unsupported meaning before use, and authorized writes migrate legacy bytes without dropping unknown fields |
+| Codex/Claude hook JSON | Durable | Controls whether and when agent hooks execute |
+| Structured suppression file | Durable | Records an accepted finding and its reason |
+| Health baseline | Durable | Acts as a committed regression policy |
+| Verified binary cache promotion | Durable | Makes checksum-accepted executable or tool bytes current |
+| TypeScript/Rust request and response mailboxes | Visibility-atomic | A timeout or retry reconstructs the ephemeral message |
+| Watch activity | Visibility-atomic | A newer timestamp supersedes an older idle-lifetime observation; it carries no durable intent |
+| Watch refresh request, claim, and completion records | Durable immutable admission and acknowledgement | Accepted intent survives activity replacement and owner crashes; exclusive claims may be recovered only by the next lock owner |
+| TypeScript fragment/overlay manifests | Visibility-atomic | Content-addressed cache artifacts are validated and rebuildable |
+| Repository GC state | Visibility-atomic | Sweep history can be reconstructed conservatively |
+| Affected-set shadow latest record | Visibility-atomic | Calibration telemetry does not control publication |
+
+Process locks use exclusive descriptor creation rather than replacement. Their
+durable token ownership, malformed-creation grace, guarded recovery, and
+legacy compatibility are defined in
+[Process Lock Ownership and Recovery](LOCK_PROTOCOL.md). Generation directories,
+stable compatibility mirrors, and their crash ordering are defined in
+[Local Index Generations](INDEX_GENERATIONS.md). Durable watch demand,
+idempotency, claim recovery, and acknowledgement ordering are defined in
+[Watch Refresh Requests](WATCH_REFRESH_REQUESTS.md).

package/docs/INDEX_GENERATIONS.md

@@ -0,0 +1,121 @@
+# Local Index Generations
+
+`scip-query` treats `index.db`, `index.scip`, and `meta.json` as one local
+index generation. A local index generation is an accepted set of compiler
+artifacts whose database rows, SCIP occurrences, and metadata describe the
+same indexing result. Keeping those files associated is what lets a query
+attach one truthful identity to every row and downstream semantic request.
+
+A generation handle is the storage-owned reference retained by one
+`ScipDatabase`. It differs from a group of cache paths by resolving the
+published pointer once, opening the database beneath that immutable
+generation directory, and retaining the corresponding metadata bytes and SCIP
+path for the connection's entire lifetime.
+
+## Layout
+
+The worktree cache keeps this internal layout:
+
+```text
+<cache>/
+  index.db                         compatibility mirror
+  index.scip                       compatibility mirror
+  meta.json                        compatibility mirror
+  .scipquery-generations/
+    state.json                     atomic current-generation pointer
+    <generation-sha256>/
+      manifest.json
+      index.db
+      index.scip
+      meta.json
+```
+
+The files inside a named generation directory are immutable. The SHA-256
+identity incorporates the database bytes, SCIP bytes when present, and the
+generation-bearing metadata fields. `manifest.json` records the exact size and
+digest of each stored artifact. `state.json` names the accepted directory and
+is the only publication decision read by internal database consumers.
+
+The top-level files remain for the SCIP CLI, indexers, older scip-query
+versions, and external inspection. They are derived mirrors, not the internal
+read authority. A current scip-query process never combines rows from a
+generation directory with metadata or SCIP bytes freshly reread through those
+replaceable paths.
+
+## Publication and crash behavior
+
+Publication proceeds in this order:
+
+1. Retain the previously published stable artifacts as an immutable
+   generation when upgrading a legacy cache.
+2. Copy-on-write clone or copy every accepted candidate artifact into a
+   private staging directory.
+3. Flush the artifact files and manifest, rename the complete staging
+   directory into its content-derived identity, and flush the generation
+   directory entry.
+4. Durably replace `state.json` so new internal readers select the complete
+   new directory.
+5. Replace the three compatibility mirrors and durably record their file
+   identities for drift diagnostics.
+
+A crash before step 4 leaves the prior pointer authoritative. A crash after
+step 4 leaves the new immutable generation authoritative even if one or more
+compatibility mirrors are old. `scip-query status` and freshness inspection
+report that mirror drift and a later refresh repairs it; database-backed
+queries remain generation-consistent throughout.
+
+Metadata-only refreshes create a new immutable generation and switch the same
+pointer. Old database handles retain their old metadata bytes. Result cursors
+and TypeScript semantic mailbox requests carry the handle identity, so a
+continuation or numeric symbol identifier from an older generation is rejected
+when a service has moved to a newer one.
+
+Mailbox protocol version 3 additionally binds that generation identity into a
+content-derived logical-operation key. The deterministic request ID lets a
+retry join an already pending, inflight, or retained completed operation,
+while the service still recomputes the key and rejects a path, payload, or
+generation mismatch. See
+[`MAILBOX_LIFECYCLE.md`](MAILBOX_LIFECYCLE.md) for ownership, expiry, limits,
+and legacy overlap.
+
+## Legacy overlap and retention
+
+A cache without `state.json` remains readable through a bounded legacy
+open-and-file-identity recheck. A state record written by the earlier local
+generation implementation, which did not name an immutable artifact set, also
+uses this compatibility path. New publications upgrade either layout without
+deleting the stable files.
+
+Published local generation directories are retained conservatively. Automatic
+collection is intentionally disabled until a cross-process reader lease can
+prove that no surviving handle can later need the retained SCIP companion.
+This can temporarily consume more cache space, but it preserves the stronger
+rule that storage reclamation may never change or remove evidence owned by a
+live query.
+
+The repository-wide shared generation store is a separate cache layer. It
+warms a worktree by copying a complete generation into the worktree's private
+cache; the local publisher then creates the local immutable directory and
+pointer described here. Later worktree writes cannot mutate either the shared
+source or a retained local reader.
+
+## Shared worktree lease invariant
+
+A worktree lease is a repository-cache reachability record whose generation
+IDs keep an immutable shared generation from collection while that worktree
+uses it. Its essential ownership fields bind the repository, worktree,
+project path, and local cache path through a checksum; `lastSeenAt` is only a
+liveness observation.
+
+Generation attachment, lease liveness touches, and repository cleanup
+serialize through one repository-cache lock. A touch may inspect the local
+pointer before waiting only to identify that lock. Once it owns the lock, it
+rereads the pointer and lease, validates the ownership checksum, current Git
+tree, local metadata fingerprint, generation IDs, and source artifacts, then
+merges only a newer `lastSeenAt`. It never writes a lease assembled from the
+pre-lock observation.
+
+Consequently, a touch waiting behind a new generation attaches to or rejects
+the new lease; it cannot restore the old generation. A deleted lease stays
+deleted, a recreated lease with different ownership stays intact, and a touch
+whose clock is behind another completed touch cannot move liveness backward.

package/docs/LOCK_PROTOCOL.md

@@ -0,0 +1,133 @@
+# Process Lock Ownership and Recovery
+
+scip-query uses one process-lock protocol for watch ownership, reindex
+publication, repository-cache garbage collection, shared-generation builds,
+verified-binary fetches, and the durable Rust semantic server.
+
+A **process lock** is a filesystem ownership record created exclusively at one
+resource path. Exclusive creation makes competing processes observe one
+winner; the recorded random token and operating-system process-start identity
+distinguish that winner from later processes and later lock records that reuse
+the same PID or pathname.
+
+A **process instance** is one execution occupying an operating-system PID
+slot. The PID locates the slot, while the process-start identity distinguishes
+successive executions that occupy it. A PID by itself is therefore accepted
+for conservative legacy inspection but never treated as proof that a live
+process is the original owner.
+
+A **reclaim guard** is a second exclusive, token-owned process lock at
+`<lock>.reclaim`. It serializes recovery attempts so that only one process may
+remove an unchanged abandoned record. The guard does not make an ambiguous
+live owner safe to remove.
+
+## Current Record
+
+New writers emit protocol version 1:
+
+```json
+{
+  "protocol": "scip-query-process-lock",
+  "version": 1,
+  "kind": "reindex",
+  "pid": 41001,
+  "token": "random-owner-token",
+  "processIdentity": {
+    "version": 1,
+    "pid": 41001,
+    "platform": "linux",
+    "startToken": "9147752"
+  },
+  "startedAt": "2026-07-25T19:00:00.000Z",
+  "detail": {
+    "projectRoot": "/repo"
+  }
+}
+```
+
+`kind` identifies the resource protocol using the lock. `detail` carries only
+that protocol's diagnostic metadata; it is not ownership evidence. Ownership
+is the combination of PID, random token, and process identity when the host can
+obtain one.
+
+The creator writes the complete record to a token-unique private candidate,
+flushes and closes that candidate, then publishes the complete inode under the
+public lock name with an exclusive hard link. The link either creates the
+public name or reports that another owner already has it; it never replaces an
+owner. A crash before the link can leave a private candidate, but it cannot
+expose an empty or truncated public lock. Successful publication removes the
+private name and flushes the containing directory where supported.
+
+## Observation and Recovery
+
+| Observed state                                                     | Recovery decision                                                       |
+| ------------------------------------------------------------------ | ----------------------------------------------------------------------- |
+| Current record; process instance is live and matches               | Contended; never remove                                                 |
+| Current record; PID is dead                                        | Reclaim after guarded unchanged recheck                                 |
+| Current record; PID is live but its process-start identity differs | Reclaim the old record without signaling the new PID occupant           |
+| Current record; PID is live but identity cannot be read            | Contended; fail closed                                                  |
+| Supported legacy record; PID is live                               | Contended because the process instance cannot be verified               |
+| Supported legacy record; PID is dead                               | Reclaim after guarded unchanged recheck                                 |
+| Empty, truncated, or malformed public record                       | Contended; fail closed because civil-clock age cannot identify an owner |
+| Record changes before the guarded recheck                          | Do not remove; report contention                                        |
+| Empty or malformed reclaim guard                                   | Do not remove                                                           |
+| Valid reclaim guard whose process instance is dead                 | Recover the guard, then retry once                                      |
+
+The unchanged recheck compares the original bytes, device, inode, size, and
+modification time. Recovery is limited to one retry: persistent ambiguity
+remains contention instead of becoming an unbounded delete loop.
+
+Malformed public records from a legacy writer, external damage, or manual
+editing require operator review. A civil timestamp is not causal ownership
+evidence: moving the system clock forward cannot prove which process created
+the bytes or whether that process still owns the resource.
+
+## Release
+
+Release rereads the current record and removes it only when PID and token still
+match the owner's retained record. When the owner recorded a process identity,
+that identity must match as well. A missing path, malformed record, legacy
+record, token mismatch, or successor record makes release a no-op. Successful
+removal flushes the containing directory where supported so an acknowledged
+release does not resurrect the prior directory entry after a crash.
+
+This protocol does not expire live ownership and therefore does not require a
+monotonically increasing fencing token. A **fencing token** is an ordered
+ownership value checked by the protected resource before it accepts a write;
+it is necessary when an old live owner can outlast a lease and attempt
+publication after a newer owner. scip-query instead refuses to steal a
+verifiably live lock.
+
+## Legacy Compatibility
+
+Readers retain narrow decoders for the prior formats:
+
+- watch and reindex JSON ownership records;
+- generic repository-cache and shared-build `{ "pid": ... }` records;
+- the durable Rust semantic server's numeric PID record.
+
+New writers always emit the current common format. A live legacy owner remains
+contended because its process instance cannot be proven. A dead legacy owner
+can be reclaimed. Unsupported or malformed JSON is never guessed into a
+legacy owner.
+
+## Diagnostics and Manual Recovery
+
+Successful reindex recovery reports that it recovered an abandoned lock. A
+manual refresh that encounters an already-stale
+watcher-owned reindex record retains the existing “preempting watcher refresh”
+diagnostic while stating that the prior owner was already stale.
+
+For other lock users, normal recovery is silent and contention reports the
+resource-specific lock path. If a lock remains contended:
+
+1. inspect the JSON without editing it;
+2. verify whether its PID and process-start identity name the current process;
+3. for a malformed record, establish ownership outside scip-query and move the
+   record aside rather than asking wall-clock age to authorize deletion;
+4. retry the operation so the guarded recovery path can run for attributable
+   dead owners.
+
+Do not manually delete a valid lock whose process instance is live. If host
+permissions prevent process-identity verification, stop the owning process or
+move the record aside only after establishing ownership outside scip-query.