@kaisers-io/refs 0.11.0 → 0.13.0

package/CHANGELOG.md

@@ -5,7 +5,343 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [Unreleased]
+## [0.13.0] - 2026-09-12
+
+### Upgrading
+
+**`refs add <url> --description "…"` now refuses a source with workspace members.** It holds one
+description, about the repository, and none for a package — and a package's description is no
+longer read out of its manifest, so there is nothing for it to fall back on. Such a source exits
+`3`, listing every package the proposal will need a description for and printing the two-phase
+commands to run instead, with the source quoted into them. The shortcut still finalizes a
+repository with no detected packages, and one whose only detected package is the root at `.`.
+
+If a script calls the one-shot against a monorepo, it will start failing. The two-phase flow is the
+replacement and always was the documented agent path.
+
+**Package descriptions already in `config.toml` are left exactly as they are.** A description
+imported from a manifest before this release stays; nothing distinguishes it from one someone
+wrote, so nothing rewrites it. To review them: `refs show <ref> --packages --json`, and
+`refs edit <ref> --package <name> description "…"` to replace one.
+
+**A ref key containing an unpaired surrogate is now rejected.** No configuration written by refs
+can contain one — TOML has no escape for it — so this only affects a hand-built key.
+
+### Fixed
+
+- **`refs add` says when it could not fully inspect a repository's declared workspaces.** A
+  repository declaring `packages/**` — an ordinary spelling the pattern classifier cannot expand —
+  produced a proposal missing every member, with no indication why. That is not self-explanatory:
+  a thin `packages` record is also what an ordinary repository produces, and a detected root makes
+  even a memberless result look complete. The configuration was then built without those packages,
+  and the drift probe returns immediately for a ref that configures none, so nothing downstream
+  could recover it either. Both `--dry-run` and the `--description` one-shot now carry a warning
+  naming what stopped the scan — an unexpandable pattern, an unreadable manifest, a candidate that
+  could not be inspected — alongside any clone warning rather than instead of it.
+
+- **A drift probe that could not finish looking says so, instead of reporting `ok`.** The pass that
+  finds packages a checkout declares and the configuration does not have stands down whenever
+  workspace detection may have missed something — an unreadable manifest, a pattern the classifier
+  cannot expand. That conservatism is right: a second declaration of the same name could be behind
+  the obstacle, and naming one path from a partial view would prescribe something registration
+  might not do. What was wrong is that it then said nothing at all, so `refs doctor` answered
+  `config-drift: ok — every configured package path resolves` while an unregistered package sat in
+  the checkout. One malformed `package.json` anywhere in a monorepo was enough, permanently.
+
+  Both discovery passes now report the obstacle. `structure` gains `discovery_incomplete`, naming
+  what stopped them (`packages/c: manifest_unreadable`), and its `status` is `unknown` rather than
+  `ok`. The findings about configured entries are unaffected — that half never needed the scan to
+  begin.
+
+- **A ref whose key carries `@`, a space or any non-ASCII character can be locked, and therefore
+  synced.** `zRefKey` admits every character but `/`, `\`, `%` and `:`; the lock alphabet is far
+  narrower, and the derived lock name was passed through unchanged. Such a ref could be added and
+  read but never locked — so `refs sync`, `refs remove`, `refs resolve`'s package verification and
+  `refs doctor`'s drift check all failed for it, with a message naming the lock name rather than
+  the ref. A name the lock alphabet will not accept now falls back to the same bounded digest form
+  a too-long name already used. Keys the alphabet does admit keep their readable name unchanged.
+  Reachable through a self-hosted url or a hand-edited `config.toml`; no forge allows these
+  characters in a repository path.
+- **A ref key carrying an unpaired surrogate is rejected.** Such a character is a valid JavaScript
+  string and has no UTF-8 encoding, so everything that writes the key out — the checkout directory,
+  the digest a lock name can fall back to — silently substituted U+FFFD for it. Keys ending
+  U+D800, U+D801 and U+FFFD therefore shared one directory and one lock name, and two unrelated
+  refs would have serialized against each other. No real config could carry one: TOML has no
+  escape for an unpaired surrogate.
+- **A failed sync no longer persists an unbounded amount of remote output into `state.json`.**
+  `last_error` holds the failure's message, most often git's own output — which quotes what git was
+  working on, ref names among them, chosen by the tracked repository. No persistence-specific limit
+  applied: an upstream with 120 conflicting tags produced a 47,816-character error, all of it
+  written into a file refs reads back on every command. A newly recorded message now keeps 1500
+  characters from the start and 500 from the end, with an exact count of what was dropped in
+  between — so both the command that failed and git's own closing hint survive. The result of the
+  run that produced the failure still carries the message as it arrived.
+
+### Changed
+
+- **Dependencies refreshed.** `zod` 4.5.2 → 4.6.2 — the only production dependency, and bundled
+  into the published CLI. The development toolchain moved too: `vitest` 4 → 5, `tsdown` 0.22 →
+  0.23 (rolldown 1.2.8), `oxlint`, `oxfmt` and `vite`. No behaviour depends on any of it.
+- **A package's description is never read out of a manifest.** Workspace detection carried each
+  package's `description` from its `package.json`, and `refs add <source> --description "…"`
+  persisted it into `config.toml` — text written by whoever owns the upstream repository, crossing
+  into a file refs reads back as its own configuration and replays on every `refs list` and
+  `refs show`. Detection now carries a package's name and path and nothing else, which is what
+  `refs sync`'s `unregistered` finding has done since 0.12.0. Every description in a config entry
+  is written by someone who read the source.
+
+  The one-shot `refs add <source> --description "…"` therefore no longer registers workspace
+  members: it has one description, about the repository, and none for a package. It still
+  finalizes a source with no detected packages, or one whose only detected package is the
+  repository root at `.` — and there the text now wins over whatever the root manifest said about
+  itself. For anything else it exits `3`, naming every package and printing the two-phase commands
+  to run instead, with the source it was given quoted into them, and listing every package the
+  proposal will need a description for — the repository root included, which is not the set the
+  refusal is about but is the set `refs add --proposal` requires.
+
+  A refusal registers no ref. The checkout it cloned stays, and is now recorded as a pending add —
+  so for the next 24 hours `doctor` reports it as one instead of offering to delete a checkout the
+  printed recovery is about to reuse, and the clone mode actually used survives into the finalize.
+  That mode cannot be recovered from the checkout afterwards: git records
+  `remote.origin.promisor` and `partialclonefilter` from the requested filter, whether or not the
+  server honoured it.
+
+  Existing entries are untouched. A description imported from a manifest before this release stays
+  exactly as it is — nothing distinguishes it from one written by hand, so nothing rewrites it. To
+  review them: `refs show <ref> --packages --json`, and `refs edit <ref> --package <name>
+description "…"` to replace one.
+
+### Security
+
+- `last_error` is documented as what it is — a failure message, usually git's own output, quoting
+  ref names the tracked repository chose — and `skills/refs/COMMANDS.md` now tells agents to read
+  it as untrusted evidence rather than as something refs vouches for.
+- `SECURITY.md` claimed refs "never reads checkout content as configuration", which was not true of
+  manifest descriptions and is still not true of package names, paths, the default branch, or a
+  `tag_format` derived from real tags. The claim is now stated as what the code enforces: refs never
+  imports a description out of a manifest. The values that do still cross are named, along with the
+  fact that being structural does not make them safe to read — a package name is whatever the
+  manifest declares.
+
+## [0.12.0] - 2026-09-08
+
+### Upgrading
+
+Run `refs sync` after updating. Two things are worth expecting on that first run.
+
+**Packages may be reported as `missing` that are still on disk.** Negated workspace patterns are
+applied now, so a package a repository excludes (`!examples/vue/2*`) is no longer a workspace
+member — and an entry registered for one before this release no longer verifies. Nothing is removed
+automatically; the finding names the entry and leaves the decision alone.
+
+**Packages that arrived upstream since the last sync are reported.** Only those: a package the
+configuration never had and that did not arrive in the fetched range stays unmentioned, however
+long it has been there.
+
+### Added
+
+- **`refs sync` now reports a package that arrived upstream, and `refs edit --create` registers
+  it.** The drift probe checked the packages the configuration already had, so a package added
+  upstream after `refs add` stayed invisible — and there was no command to register one either:
+  `refs add` refuses an already-tracked ref, and every `refs edit` mode needs an entry to edit. The
+  only instruction anyone could give was "hand-edit `config.toml`".
+
+  `refs sync` answers "did upstream gain a package?" from the range it just fetched, not by
+  comparing a scan against the configuration. That distinction is the whole design: a scan cannot
+  tell a package that just arrived from one the ref's owner deliberately never tracked, because
+  there is no inventory of what was there before — the fetch range is that inventory. A ref whose
+  owner tracks 3 packages out of 140 hears about the other 137 exactly never. `refs doctor` lists
+  every unregistered member instead, because it was asked to.
+
+  The question it asks of that range is about package NAMES, not manifest paths. A package
+  renamed in place modifies its manifest rather than adding one, and a package merely moved to
+  another directory adds one without being new — so a path-based reading is wrong in both
+  directions. Only the manifests the range actually changed have to be read out of history: an
+  untouched manifest is byte-identical at both ends, so the name it carries now is the name it
+  carried before.
+
+  The repair is a command now rather than a config fragment, with the ref key filled in so it
+  runs as printed:
+
+  ```
+  refs edit 'github.com/acme/alpha' --package '@acme/new' --create --path 'packages/new' \
+    --description "<what it is>"
+  ```
+
+  It is a distinct mode, not an upsert — an ordinary field edit naming an unregistered package
+  still fails with `not_found`, so a typo in `--package` can never become a new entry. The
+  finding carries `name` and `path`, both verified against the checkout and both shell-quoted
+  (being verified makes a value true, not shell-safe: `zPackagePath` permits `$()` and a manifest
+  `name` is checked only for being non-empty), and deliberately no description: a manifest description is untrusted third-party content, and copying it moves it
+  into a file refs later reads as its own configuration. The skill instructs agents to propose the
+  registration and wait for the user to agree, rather than run it on their own initiative.
+
+### Fixed
+
+- **Workspace patterns are matched by `minimatch`, the matcher npm itself uses.** Hand-written
+  matching disagreed with the real resolvers in five distinct ways, each found only after the last
+  was fixed: extglob (`@(a|b)`) read as a literal directory name, trailing slashes treated
+  symmetrically where `minimatch` is asymmetric, repeated separators silently matching nothing, and
+  two more. Delegating removes that class of defect rather than the current instance of it.
+
+  Measured before choosing: `picomatch` disagrees with `minimatch` on ten of 154 comparisons over
+  the shapes this scanner supports, exactly on trailing slashes and repeated separators — so it is
+  not a drop-in. pnpm matches through `picomatch` but normalizes first, and was measured to agree
+  with `minimatch` on every one of those shapes, so one matcher covers both ecosystems.
+
+  Walking stays here: containment guards, the diagnostics that say why a scan came up short, and
+  the deliberate one-level depth policy are unchanged. `minimatch` answers only whether a path
+  matches a pattern. Nothing new is installed by `refs` users — the CLI publishes a bundle with no
+  dependencies — and that bundle grows by 24 KB.
+
+  One behaviour improves as a consequence: a negation in a shape this scanner cannot WALK
+  (`!packages/{a,b}`) is now applied, because applying an exclusion needs matching and never
+  walking. It used to be reported as unsupported and silently ignored.
+
+- **Negated workspace patterns are applied instead of ignored.** `!packages/fixtures` was dropped
+  as an unsupported shape (a v1 simplification), so `refs add` registered packages the repository
+  had explicitly excluded, and every finding about a repository declaring one was silenced —
+  TanStack Query declares two, and all hundred of its packages came back `unverifiable`. Negations
+  are now expanded exactly like inclusive patterns and subtracted from the result, which makes the
+  scan an accurate statement of membership rather than an approximation of one.
+
+  A wildcard inside the last segment (`examples/vue/2*`) is supported too, since that is the shape
+  real repositories exclude by, and a negation nobody can expand costs every finding about the
+  repository rather than just the paths it names. Glob syntax that is still unimplemented —
+  `{a,b}`, `?`, `[…]` — now reports `unsupported_pattern` rather than reading as a literal
+  directory name and silently matching nothing.
+
+- **A monorepo can now be resolved by the name in its own root manifest.** Workspace detection
+  expands the globs a repository declares, and a workspace root is not one of its own targets — so a
+  root that names itself was registered nowhere, and `refs resolve @acme/toolkit` came back empty
+  for a repository that was tracked all along. `refs add` now registers a named root at `path: "."`
+  alongside the workspace members.
+
+  Both pnpm and Yarn address a workspace root by that name (`pnpm --filter <root-name>`,
+  `yarn workspace <root-name>`); npm and Turborepo use a positional handle instead. Of eighteen
+  well-known monorepos surveyed, eight carry a name someone would plausibly use for the repository
+  and ten carry a throwaway like `root` or `monorepo-root` — which is what settles it: registering
+  the name costs nothing where it is a throwaway, since nobody resolves `"root"`, and answers the
+  question where it is not.
+
+  Two things this deliberately does not do. A repository that declares no workspaces is untouched:
+  `refs add`'s npm fallback owns that shape, and probing the root there would displace a locator it
+  did not choose — as it also would where a workspace declaration selects nothing, so the package
+  named in an `npm:<pkg>` source survives there too. And where a workspace member already claims the
+  root's name — `@remix-run/react-router` is a real example, in a repository that also publishes
+  `react-router` — the member wins and the root is simply not registered, which costs that
+  repository nothing it had before. That rule lives in detection itself rather than in `refs add`,
+  so relocation agrees with registration: a member that moves is still found uniquely, instead of
+  becoming ambiguous against a same-named root and leaving `resolve` with no path for a package
+  that is plainly there. And a root is never reported as a package's new location: its name is an
+  alias for the repository, so a member that upstream deletes is reported as gone rather than as
+  having moved to the repository root — which would have sent a caller to the wrong directory and
+  described a move that never happened.
+
+  Refs tracked before this change keep the package map they were given, and no command adds one
+  entry to an existing ref — `refs add` refuses a tracked ref, `refs edit --package` needs an entry
+  to edit. So `refs sync` and `refs doctor`'s `config-drift` check now report a root the
+  configuration does not register, with the entry to add — including the path registration would
+  actually use, which is the member's rather than the root's where a workspace member declares the
+  same name. That costs one manifest read per ref, and a workspace scan only where there is
+  something to report; it is asked only of refs that already register packages, since a plain
+  reference repository registers none on purpose and is left alone.
+
+  The root package takes the ref's own description when its manifest carries none, which is the
+  ordinary case for a private workspace root. That is not the per-package fallback `refs add`
+  otherwise refuses: the root is not a package beside the repository, it is that repository.
+
+- **A failed lookup no longer reads as an absent repository.** `refs resolve` exits `4` when a query
+  matches nothing, and the message ended "run refs list, or add it: `refs add <url>`". That second
+  half is a guess: a query can miss every route while the repository is tracked perfectly well under
+  another identifier — a monorepo root whose own package name was never registered, for instance.
+  An agent read the suggestion as confirmation and told someone a repository they had tracked was
+  not tracked, then stopped.
+
+  The message now states what was searched and points at evidence rather than prescribing a fix, and
+  `--json` carries a `reason` on `resolve`'s routing misses: `unmatched_query` (nothing matched, by
+  any route), `package_not_registered` (the ref is tracked and registers no such package), or
+  `ref_not_registered` (a canonical git url named an absent ref — the one case where adding it is
+  the right answer, since only a canonical url establishes which ref was meant). There is deliberately no reason meaning "this repository does not exist", because nothing
+  refs can observe establishes that; and `reason` is absent on every other `not_found`, where its
+  absence means no narrowing is available rather than being a fourth value.
+
+  The skill's instruction changed with it. It used to say exit `4` means the ref is not tracked; it
+  now says exit `4` means the query matched no route, and requires a second lookup before any
+  conclusion. `refs resolve --ref <ref>`'s own miss also stopped suggesting a bare `refs show`,
+  which reports a package count and no names — it now suggests `--packages`, which actually shows
+  the map the reader was sent to inspect.
+
+- **A stale-lock reclaim could delete a lock another process was using.** refs reclaims a lock left
+  behind by a crashed process. The check that decided a lock was abandoned and the removal that
+  acted on it were two separate steps, and in the gap between them the lock could legitimately
+  become somebody else's: the original holder releases, a waiting process takes the same path, and
+  the reclaim then deletes a lock that is actively in use. Both processes go on to `reset --hard`
+  the same checkout.
+
+  Three changes close it for processes running this version:
+
+  - **Only a process the operating system reports as gone is reclaimed from automatically.** A lock
+    whose lease has run out but whose process still answers is now reported rather than taken — a
+    live process can release at any instant, and that release is what opened the gap. Same for a
+    lock whose metadata never finished being written, or carries no usable identity.
+  - **The acquisition is re-identified after the death check.** Proving the recorded process gone is
+    not enough on its own: the metadata is read first and the process probed after, so the path can
+    change hands in between and the probe then answers about the departed owner. Re-reading the
+    identity immediately before the removal is what ties the two together.
+  - **The marker that stops two reclaims colliding no longer expires.** It used to be taken over
+    after two seconds, so it only excluded a reclaim fast enough to finish inside that window; a
+    suspended one lost its marker mid-work and a second reclaim started on the same lock. Age is
+    not evidence of abandonment.
+
+  The protocol's own markers also moved into `locks/.claims/` and `locks/.tombstones/`, which no
+  lock name can reach — lock names must start with a letter or digit. That removes a collision
+  where a repository literally named `foo.steal-claim` produced the marker path of the lock for
+  `foo`, and with it the name-shape guessing `refs doctor` needed to tell the two apart.
+
+  **What this costs.** Three situations no longer recover on their own and need one explicit
+  command, which `refs doctor` prints along with the condition for running it safely — stop every
+  refs process on that home first, suspended ones included: a crash after the operating system has
+  reused the process id, a crash before the lock finished writing its metadata, and a crash while a
+  reclaim was starting.
+
+  The messages changed to match. `refs doctor`'s `locks` check separates a lock refs will reclaim
+  by itself from one it will not, and the failure message when a lock cannot be acquired no longer
+  says "already reclaimable — retry" for a lock nothing will ever reclaim. It also stopped
+  describing the window as the thing that frees a lock: waiting does not, and only a recorded
+  process the operating system reports as gone does.
+
+  **What it does not fix.** A refs process running an _older_ version follows none of this and
+  reclaims on its own terms. And no lock protocol can help when refs is hard-killed while its `git`
+  child survives: the successor's lock is honest about the lock, not about the directory.
+
+- **Two unrelated refs could share one lock.** The per-ref lock name replaced `/` with `_`, and `_`
+  is legal inside a ref key — so `github.com/acme_tools/widget` and `github.com/acme/tools_widget`
+  both derived `ref.github.com_acme_tools_widget`. The two then serialized against each other:
+  `sync` fans out four refs at a time and the loser failed on a lock conflict after the timeout,
+  `resolve`'s verification could block on a sync of a ref it has nothing to do with, and `doctor`'s
+  `config-drift` check reported the wrong ref as busy.
+
+  The name is now injective. A key containing no `_` encodes exactly as before, so it keeps the
+  lock name refs has always written for it; a key containing one moves into an escaped form under
+  `ref._`, where `_` becomes `_u` and `/` becomes `_s`. The two forms cannot collide, because a ref
+  key always starts with `[a-z0-9]` and so a plain name never begins `ref._`.
+
+  A lock name is one directory entry, so a key long enough to overflow one now falls back to a
+  digest under `ref.__` rather than failing `mkdir` with `ENAMETOOLONG`. The budget reserves room
+  for the sibling entries the steal protocol derives from a lock name — a name that could be
+  created but not renamed to its tombstone would strand an abandoned lock that nothing could then
+  reclaim, leaving the ref blocked until someone deleted the directory by hand. Both were already
+  true before this change, for keys past roughly 200 characters; reaching it needs a self-hosted
+  url, since no forge allows a path that long.
+
+  **One caveat if you run refs concurrently across an upgrade.** The lock name changes for a ref
+  whose key contains `_`, and for one long enough to reach the digest form — roughly 200
+  characters, either way. For such a ref, a process from any earlier release (the old scheme dates
+  to 0.1.1) derives a different name from a new one, so for the length of that overlap the two
+  would not exclude each other. The window is a mid-upgrade concurrent run on the same refs home;
+  if that is a situation you can be in, let running operations finish before upgrading. Every other
+  ref keeps its name and is unaffected.
 
 ## [0.11.0] - 2026-08-31
 
@@ -689,7 +1025,8 @@ trusted-publishing pipeline end to end.
   installed git hooks.
 - Agent skill (`skills/refs/`) documenting the investigate/add/maintain workflows.
 
-[Unreleased]: https://github.com/kaisers-io/refs/compare/v0.11.0...HEAD
+[Unreleased]: https://github.com/kaisers-io/refs/compare/v0.12.0...HEAD
+[0.12.0]: https://github.com/kaisers-io/refs/compare/v0.11.0...v0.12.0
 [0.11.0]: https://github.com/kaisers-io/refs/compare/v0.10.0...v0.11.0
 [0.10.0]: https://github.com/kaisers-io/refs/compare/v0.9.0...v0.10.0
 [0.9.0]: https://github.com/kaisers-io/refs/compare/v0.8.3...v0.9.0