textus 0.4.0 → 0.8.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 98c2ce5525bbf9c05ebdb5eeaacde8e208253ee878d7d0722ff192435168f69f
-  data.tar.gz: c16cd5657396884c646331912d988838e0f8ed2002fc76d47cc54383dc434f19
+  metadata.gz: 91be0acd415a4b41d96e896015dfd19e58fc7867d007332456499ec5ceb8a6aa
+  data.tar.gz: 576d361ebba900b33b2f612b1a0a51d0373cb4a5e1fbd503e43c2bb248ca9306
 SHA512:
-  metadata.gz: 7478459f9672474c32f37b59a7134309cf12f6a7455a808c24b6ac717820ff8c8aad5a5187a6dbe5e0c73f98c04296ce95eff2cac8ea75144f559df8cea3fe80
-  data.tar.gz: '08a29e9090f6558a010009e3562a7ec8d6ddf73454bb09b10b5aeaf3def24d9d11905078d62807b8e1f0936b00889a66df8c55d68b7dfc411addd90e60ae1a93'
+  metadata.gz: c551c60809be1eefc0d964092adc62a8555cb6f0df3585e775fd2fda1549f1e7d811fb2bfeec992dd3ce872dccb16fd9300a66314d5e51e8f24a98b57f29da5b
+  data.tar.gz: c96ec13692d6366a916b125266afb70aabcca439594db4e8fd1b434f1db1a88e216938cbf64bc69e64bcfef241ca7314664398c656f1d7333a7480a7d02852f9

data/CHANGELOG.md

@@ -5,10 +5,155 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
 The **gem version** (`0.x.y`) is distinct from the **protocol version**
-(currently `textus/1`, embedded in every envelope as `protocol`). The protocol
+(currently `textus/2`, embedded in every envelope as `protocol`). The protocol
 is additive within a major; a new major would change the wire string.
 
-## [Unreleased]
+## 0.8.0 — Folder restructure & Zeitwerk autoload (2026-05-21)
+
+### Breaking — internal Ruby renames
+Internal Ruby constants renamed. No deprecation aliases; downstream code referencing internals must update directly.
+- `Textus::EventBus` → `Textus::Hooks::Dispatcher`
+- `Textus::HookRegistry` → `Textus::Hooks::Registry`
+- `Textus::BuiltinHooks` → `Textus::Hooks::Builtin`
+- `Textus::Extensions` (module) → `Textus::Hooks::Loader`
+- `Textus::StoreView` → `Textus::Store::View`
+- `Textus::AuditLog` → `Textus::Store::AuditLog`
+- `Textus::ManifestEntry` → `Textus::Manifest::Entry`
+- `Textus::KeyDistance` → `Textus::Key::Distance`
+- `Textus::Path` → `Textus::Key::Path`
+- `Textus::SchemaTools` → `Textus::Schema::Tools`
+- `Textus::CLI::<Verb>` → `Textus::CLI::Verb::<Verb>` (all 23 verbs)
+- `Textus::CLI::<Name>Group` → `Textus::CLI::Group::<Name>` (key, schema, hook)
+- `Textus::Doctor::Check::Extensions` → `Textus::Doctor::Check::Hooks`
+- `Hooks::Registry#initialize` keyword `bus:` renamed to `dispatcher:`.
+
+### Breaking — doctor CLI surface
+- `textus doctor --check=extensions` → `textus doctor --check=hooks`. The check name listed in `ALL_CHECKS` and the SPEC §10.2 enumeration changes from `"extensions"` to `"hooks"`, matching the hook subsystem rename in 0.6.
+- Doctor issue `code` for broken hook files: `extension.load_failed` → `hook.load_failed`.
+- Doctor::Check::Hooks now inspects `.textus/hooks/` (matches `Store#load_extensions`). Previously inspected `.textus/extensions/`, which was the pre-0.6 directory — the check was dead code on any store created with current `textus init`.
+
+### Added
+- `Textus::Entry::Base` — explicit strategy interface for entry formats. Concrete strategies inherit and override.
+- `Textus::Builder::Renderer` — explicit base for output renderers.
+- `Textus::Doctor::Check` — explicit base for doctor checks. Each builtin check (9 total) is now its own file under `lib/textus/doctor/check/`.
+
+### Changed
+- Per-format schema validation moved from `Store::Reader`/`Store::Writer` onto `Entry::Base#validate_against`. Reader/Writer no longer carry a `case mentry.format` switch.
+- `Textus::Doctor` reduced to an orchestrator; the 9 builtin checks live under `Doctor::Check::*`.
+- `lib/textus.rb` switched to Zeitwerk autoload. The manual `require_relative` tree (75 lines) is gone.
+- `lib/textus/builder/renderers/` directory renamed to `renderer/` (singular) to match `Builder::Renderer::*` namespace.
+
+### Migration
+External code referencing the old internal constants must rename. `Textus.hook`, `Textus.with_registry`, the entire CLI surface, and the `textus/2` wire format are unchanged. The published API (`Store`, `Manifest`, `Envelope`, `Etag`, `Role`, `Error` hierarchy, `Builder`, `Doctor`, `Refresh`, `Init`, `CLI.run`) is unchanged.
+
+## 0.7.0 — Reader/Writer split, EventBus, Builder pipeline (2026-05-21)
+
+### Added
+- `Textus::EventBus` is now the publish/subscribe core for lifecycle events. Embedded callers can `store.bus.subscribe(:put, :name) { ... }` outside the `.textus/hooks/` directory. Hook semantics, audit behavior, and the 2-second timeout are unchanged.
+
+### Changed
+- Internal: extracted `Textus::Path` and `Textus::Envelope` value modules; `Manifest`, `Store`, `Staleness`, and `Builder` now share the same path/envelope construction.
+- Internal: split `Textus::Store` into `Store::Reader` and `Store::Writer`. Public API unchanged. `Mover`, `Validator`, and `Staleness` now take explicit collaborators instead of the full store.
+- Internal: removed `Store::Events`; replaced by the bus.
+- Internal: restructured `Textus::Builder` as a step pipeline (`LoadSources → Project → Render → Write`) with one renderer per format (`markdown/text/json/yaml`). Adding a new output format is now a single-file change.
+
+## 0.6.1 — Deprecation cleanup
+
+### Breaking
+- Flat verb aliases promised "removed in 0.6" are now actually removed:
+  - `textus mv` → `textus key mv`
+  - `textus uid` → `textus key uid`
+  - `textus migrate-keys` → `textus key migrate`
+  - `textus schema-init` → `textus schema init`
+  - `textus schema-diff` → `textus schema diff`
+  - `textus schema-migrate` → `textus schema migrate`
+  - `textus schema KEY` (positional) → `textus schema show KEY`
+  - `textus action NAME` → `textus hook run NAME`
+- `Textus::CLI::Action` class renamed to `Textus::CLI::HookRun`; file `cli/action.rb` → `cli/hook_run.rb`.
+- `Textus::CLI::DeprecatedAliasMixin` module deleted (no remaining users).
+- `textus migrate v2` command removed along with `Textus::MigrateV2` module and `Textus::CLI::Migrate` class. The migration was a one-line manifest rewrite (`version: textus/1` → `version: textus/2`); on-disk entry shapes never changed. To upgrade a `textus/1` manifest, edit `.textus/manifest.yaml` directly. `Manifest.load` still detects the old version and prints the exact edit in its error message.
+
+## 0.6.0 — Hook unification
+
+### Breaking
+- Four DSL verbs (`Textus.action`, `Textus.reducer`, `Textus.hook`, `Textus.doctor_check`) collapsed into one: `Textus.hook(event, name, **opts) { ... }`.
+- `ExtensionRegistry` class renamed to `HookRegistry`.
+- `.textus/extensions/` directory renamed to `.textus/hooks/`. No back-compat read.
+- Manifest: `source.action:` → `source.fetch:` (also renames the registry event from `:action` to `:fetch` and the `ManifestEntry#action`/`#action_config` accessors to `#fetch`/`#fetch_config`).
+- Manifest: `projection.reducer:` → `projection.reduce:`.
+- CLI: `textus extension list` → `textus hook list`; output rows keyed by `event`+`mode` instead of `kind`.
+- CLI: `textus put --action=NAME` → `textus put --fetch=NAME`.
+- Pub-sub hooks gain an optional `keys:` glob filter for per-key scoping.
+- Hook signatures standardized: `store:` is now mandatory and first on every hook (`:reduce` previously had no `store:`). Event-specific kwargs follow.
+- `:accept` event renames `pending_key:` → `key:` to match every other lifecycle event.
+- All event names are now verbs in a uniform grammar (RPC verbs `:fetch :reduce :check`; pub-sub verbs `:put :delete :refresh :build :accept`).
+
+### New
+- `EVENTS` metadata table on `HookRegistry` is the single source of truth for event names, argument shapes, return shapes, and failure semantics (rpc vs pubsub).
+- Shape-check at registration: callable kwargs are verified against the EVENTS table at load time; mismatched signatures raise `UsageError` immediately instead of surfacing at fire time.
+
+## 0.5.0 — Wire protocol `textus/2`; CLI restructure; Store split (breaking)
+
+This release reshapes the public surface ahead of 1.0. The wire protocol bumps to `textus/2`; the CLI grows nested subcommand groups; `Store` is decomposed into a thin facade plus four focused helpers; the audit log finally matches its documented NDJSON shape; and a pile of pre-0.4 cruft gets cut.
+
+### Wire protocol — `textus/1` → `textus/2`
+
+- **Breaking:** every envelope now carries `"_meta"` instead of `"frontmatter"`. For json/yaml entries, envelope `content` no longer carries a duplicate `_meta` — the metadata lives only at the envelope's top level.
+- **Breaking:** `Manifest.load` refuses `textus/1` manifests with a pointer at the new migration command. On-disk file shapes are unchanged — only the manifest version string changes.
+- **New:** `textus migrate v2` flips `version: textus/1` to `version: textus/2` in `.textus/manifest.yaml`. One command, no file edits.
+- **Internal cleanup:** `Store#extract_uid`, `enforce_name_match!`, `serialize_for_put`, `validate_all`, and `build_envelope` no longer format-switch — metadata access is uniform. Role-authority validation now works for json/yaml entries (was markdown-only).
+- **API:** `Store#put` keyword renamed from `frontmatter:` to `meta:`. Action callbacks return `_meta:` (formerly `frontmatter:`).
+
+### CLI — nested subcommand groups
+
+- **New:** `textus key {mv, uid, migrate}`, `textus schema {show, init, diff, migrate}`, `textus extension {list, run}`. Discoverable, groupable, scales.
+- **Deprecated (removed in 0.6):** the flat verbs `mv`, `uid`, `migrate-keys`, `schema-init`, `schema-diff`, `schema-migrate`, `extensions`, `action` still work but emit a stderr deprecation warning. `textus schema KEY` (positional dotted-key form) keeps working via a back-compat fallback in `SchemaGroup`.
+- **New:** `textus list`, `textus get`, etc. default to JSON output. `--format=json` is still accepted; non-json values still raise.
+- **CLI refactor:** `lib/textus/cli.rb` shrank from 434 LOC to ~100 LOC. Every verb is now a small command-object file under `lib/textus/cli/`. Dispatch is a frozen `VERBS` hash.
+
+### Audit log — true NDJSON
+
+- **Breaking:** `.textus/audit.log` rows are now one JSON object per line (`{"ts":..., "role":..., "verb":..., "key":..., "etag_before":..., "etag_after":...}`). Missing etags are `null`, not the string `"NULL"`.
+- **Structural shape:** `from_key`, `to_key`, `uid` (mv rows) live at the top level; arbitrary contextual data goes into an `extras` sub-object that is omitted when empty.
+- **Back-compat:** legacy TSV rows still parse during 0.5 — `AuditLog#last_writer_for` and `Doctor#check_audit_log` accept both formats. Legacy support removed in 0.6.
+
+### Doctor
+
+- **New:** `textus doctor --check=schema_violations[,name,…]` runs only the named built-in checks. The 9 built-ins are `manifest_files`, `schemas`, `templates`, `extensions`, `illegal_keys`, `sentinels`, `audit_log`, `unowned_schema_fields`, `schema_violations`. Extension checks always run.
+- **Breaking:** the standalone `textus validate-all` verb is gone. Use `textus doctor --check=schema_violations` instead. The internal `Store#validate_all` Ruby method is unchanged.
+
+### Manifest / store cleanup
+
+- **Breaking:** `LEGACY_ZONES` fallback removed. Manifests must declare a `zones:` block explicitly (init scaffold does this).
+- **Breaking:** legacy syntax errors removed for `source.parse` / `source.from` / `source.fetcher` / top-level `hooks:`. Those names were rejected with helpful errors in 0.4; in 0.5 they get the generic "unknown key" error from YAML parsing.
+- **Internal:** `ManifestEntry` moved to its own file (`lib/textus/manifest_entry.rb`).
+
+### Store split
+
+- **Internal:** `lib/textus/store.rb` shrank from 617 LOC to ~312 LOC. Four focused helpers live under `lib/textus/store/`:
+  - `events.rb` (31 LOC) — `fire_event` hook plumbing
+  - `validator.rb` (53 LOC) — `validate_all` body
+  - `staleness.rb` (142 LOC) — `stale` body (was 5 rubocop disables)
+  - `mover.rb` (118 LOC) — `mv` body
+- No public-API change. `Store` facade delegates to each helper one-line.
+
+### Migration cheat-sheet
+
+```sh
+# 1. Upgrade the gem
+gem update textus           # ≥ 0.5.0
+
+# 2. Upgrade the store
+cd /path/to/your/store
+textus migrate v2           # flips manifest version
+
+# 3. Anything else?
+#    - Audit log: existing TSV rows still readable; new rows are NDJSON.
+#    - CLI scripts: replace `textus mv ...` with `textus key mv ...`
+#      (and 7 similar aliases). Old forms work through 0.5 with a stderr warning.
+#    - Ruby callers of Store#put: pass `meta:` instead of `frontmatter:`.
+#    - Anything reading envelope["frontmatter"]: read envelope["_meta"] instead.
+```
 
 ## 0.4.0 — Extension API redesign (breaking)
 

data/README.md

@@ -7,14 +7,14 @@
 
 A context store for codebases that humans and AI agents both have to read and write. Dotted keys, schema-validated entries, role-gated writes, byte-copy publish, an audit log of every change. Built so an agent landing in your repo can run one command (`textus intro`) and know what to read, what to write, and what's off-limits.
 
-Reference implementation in Ruby. Wire format `textus/1`. SPEC: [`SPEC.md`](SPEC.md). Implementation notes: [`docs/`](docs/).
+Reference implementation in Ruby. Wire format `textus/2`. SPEC: [`SPEC.md`](SPEC.md). Implementation notes: [`docs/`](docs/).
 
 ## Versioning
 
 Two versions, deliberately independent:
 
-- **Protocol wire string:** `textus/1`. Stable; breaking changes require `textus/2`.
-- **Gem version:** semver, currently `0.2.0`. Gem `0.x.y` and `1.x` both speak `textus/1`.
+- **Protocol wire string:** `textus/2`. Stable; breaking changes require `textus/3`.
+- **Gem version:** semver, currently `0.8.0`. The gem version is decoupled from the protocol string — internal refactors bump the gem; only wire-format changes bump the protocol.
 
 Envelope payloads carry the `protocol` field. The gem version is irrelevant to the wire format.
 
@@ -60,23 +60,25 @@ Manifest `path:` fields are relative to `.textus/zones/`. So `working.network.or
 Read and write:
 
 ```sh
-textus get working.network.org.jane --format=json
-textus list --zone=working --format=json
-echo '{"frontmatter":{"name":"bob","org":"acme"},"body":"hi\n"}' \
-  | textus put working.network.org.bob --as=human --stdin --format=json
-textus stale --zone=derived --format=json
+textus get working.network.org.jane
+textus list --zone=working
+echo '{"_meta":{"name":"bob","org":"acme"},"body":"hi\n"}' \
+  | textus put working.network.org.bob --as=human --stdin
+textus stale --zone=derived
 ```
 
+(All verbs return JSON envelopes by default; pass `--format=json` explicitly if you prefer.)
+
 For the full shape — Claude plugin with agents, skills, commands, pending walkthrough, intake action — see [`examples/claude-plugin/`](examples/claude-plugin/).
 
-## What 0.2 ships
+## What ships today
 
 - **Per-entry formats.** `format: markdown | json | yaml | text` on a manifest entry. `cat .textus/zones/derived/marketplace.json | jq .` works without going through textus — the in-store file *is* the consumer-shaped artifact. Structured outputs carry `_meta` at the top level (`generated_at`, `from`, `template`, `reducer`).
 - **Per-leaf publishing.** Nested entries declare `publish_each: "skills/{basename}/SKILL.md"`. Every leaf byte-copies to its consumer location on `textus build`. No more hand-mirrored `agents/` / `skills/` / `commands/` directories.
-- **Stable identity (`uid:`).** 16-char hex, auto-minted on first `put`, preserved across writes and moves. `textus mv old.key new.key` renames in place — uid survives, audit row records `from_key`, `to_key`, `uid`. Reorganising a tree no longer breaks references.
-- **Strict key grammar.** `/^[a-z0-9][a-z0-9-]*$/`, max 8 segments × 64 chars. `textus migrate-keys --dry-run|--write` rewrites existing stores with illegal segments deterministically.
+- **Stable identity (`uid:`).** 16-char hex, auto-minted on first `put`, preserved across writes and moves. `textus key mv old.key new.key` renames in place — uid survives, audit row records `from_key`, `to_key`, `uid`. Reorganising a tree no longer breaks references.
+- **Strict key grammar.** `/^[a-z0-9][a-z0-9-]*$/`, max 8 segments × 64 chars. `textus key migrate --dry-run|--write` rewrites existing stores with illegal segments deterministically.
 - **`textus intro`.** One-shot store orientation: zones with writers + purposes, entry families with schemas and publish targets, loaded extensions, write flows per role, the full CLI verb table. The boot signal for any agent — one tool call and it knows your store.
-- **`textus doctor`.** Health check across 8 categories: missing schemas/templates, broken extensions, illegal nested keys, sentinel drift, audit log readability. Returns `ok: true` only when nothing is wrong; warnings and info don't flip the bit.
+- **`textus doctor`.** Health check across 9 categories: missing schemas/templates, broken extensions, illegal nested keys, sentinel drift, audit log readability, unowned schema fields, schema violations, and missing manifest files. Returns `ok: true` only when nothing is wrong; warnings and info don't flip the bit.
 - **Actionable hints on every error.** `UnknownKey` carries ranked "did you mean" suggestions. `WriteForbidden` names the role that *would* be allowed. `BadFrontmatter` tells you exactly what to rename. Printed to stderr alongside the JSON envelope on stdout.
 
 Symlink-mode publish was removed; publish is `FileUtils.cp` + sentinel. Sentinels for published files live under `.textus/sentinels/<target_rel>.textus-managed.json` so consumer directories stay clean. Legacy sibling sentinels auto-migrate on next publish.
@@ -93,22 +95,22 @@ All verbs accept `--format=json` and return the envelope defined in SPEC §8. Wr
 | `list [--prefix=K] [--zone=Z]` | Enumerate keys |
 | `where K` | Resolve a key to its filesystem path |
 | `get K` | Full envelope (frontmatter, body, uid, etag, format) |
-| `schema K` | Schema bound to an entry |
+| `schema show K` | Schema bound to an entry |
 | `stale [--prefix=K] [--zone=Z]` | List stale derived/intake entries |
 | `deps K` / `rdeps K` | Forward / reverse projection dependencies |
 | `published` | List `publish_to:` targets and their backing keys |
-| `validate-all` | Validate every entry against its schema |
-| `extensions list [--kind=K]` | Registered actions, reducers, hooks, doctor_checks |
+| `doctor --check=schema_violations` | Validate every entry against its schema |
+| `hook list [--event=E]` | Registered hooks grouped by event (fetch, reduce, check, put, delete, refresh, build, accept) |
 
 **Write:**
 
 | Verb | Role |
 |---|---|
 | `put K --stdin --as=R [--action=NAME]` | per zone |
-| `action NAME [--key=val] [--as=R]` | per zone written (invoke a registered action) |
+| `hook run NAME [--key=val] [--as=R]` | per zone written (invoke a registered fetch hook) |
 | `delete K --if-etag=E --as=R` | per zone |
 | `refresh K --as=script` | per zone (typically `script`) |
-| `mv old new --as=R [--dry-run]` | per zone (same-zone moves; uid preserved) |
+| `key mv old new --as=R [--dry-run]` | per zone (same-zone moves; uid preserved) |
 | `build [--prefix=K] [--dry-run]` | `build` |
 | `accept K --as=human` | `human` only |
 
@@ -117,16 +119,16 @@ All verbs accept `--format=json` and return the envelope defined in SPEC §8. Wr
 | Verb | Purpose |
 |---|---|
 | `doctor` | 8 health checks; `ok: true` when clean |
-| `migrate-keys [--dry-run]` | Rename files whose basenames violate the strict key grammar |
+| `key migrate [--dry-run]` | Rename files whose basenames violate the strict key grammar |
 
 **Scaffolding (human-only):**
 
 | Verb | Purpose |
 |---|---|
 | `init` | Scaffold a fresh `.textus/` |
-| `schema-init NAME` | Stub a schema |
-| `schema-diff NAME` | Compare a schema against entries that claim it |
-| `schema-migrate NAME [--rename=OLD:NEW]` | Rewrite frontmatter keys across affected entries |
+| `schema init NAME` | Stub a schema |
+| `schema diff NAME` | Compare a schema against entries that claim it |
+| `schema migrate NAME [--rename=OLD:NEW]` | Rewrite frontmatter keys across affected entries |
 
 ## Zones and roles
 
@@ -146,16 +148,24 @@ Derived entries declare a `projection:` (`select`, `pluck`, `sort_by`, `limit`,
 
 `publish_to: [path]` byte-copies a single derived file to one target. `publish_each: "template/{basename}.md"` on a nested entry byte-copies every leaf to its templated target — substitutes `{leaf}`, `{basename}`, `{key}`, `{ext}`. Sentinels for every published file live under `.textus/sentinels/`. See SPEC §5.2, §5.3, §5.12.
 
-## Extensions
+## Extension points
+
+textus exposes one DSL verb:
+
+```ruby
+Textus.hook(event, name, **opts) { |args| ... }
+```
+
+Drop `.rb` files into `.textus/hooks/`. Events:
 
-Four DSL verbs, registered in `.textus/extensions/*.rb`. Each `Store` gets its own registry — no global state.
+- `:fetch` — bring bytes in from elsewhere (returns `{frontmatter:, body:}`)
+- `:reduce` — transform rows during projection (returns rows)
+- `:check` — custom doctor check (returns issues)
+- `:put`, `:delete`, `:refresh`, `:build`, `:accept` — react to lifecycle events
 
-- **`Textus.action(:name) do |config:, store:, args:|`** — runs in three invocation modes (intake refresh, `textus action` verb, `put --action`). Returns `{frontmatter:, body:}`, `{content:}`, or `{body:}` when its return is consumed (intake and put-fetch); writes via `store.put` for side-effectful work (verb mode). The store normalizes all three return shapes. Configured via `source.action` in the manifest for intake. Five built-ins ship: `json`, `csv`, `markdown-links`, `ical-events`, `rss`.
-- **`Textus.reducer(:name) do |rows:, config:|`** — shapes rows in a derived projection. Pure function. Configured via `projection.reducer`. May return an Array (templated builds) or a Hash (templateless json/yaml).
-- **`Textus.hook(:event, :name) do |kwargs|`** — fires on `:put`, `:delete`, `:refresh`, `:build`, or `:accept`. In-process; 2 s timeout per hook; failures land in the audit log as `event_error` rows.
-- **`Textus.doctor_check(:name) do |store:|`** — contributes whole-tree validators to `textus doctor`. Returns an array of issue hashes `{code, level, subject, message, fix}` that merge into the doctor report. Timeouts and exceptions surface as `doctor_check.*` issues; they do not abort the doctor run.
+See SPEC.md §5.10 for the full contract.
 
-Schemas (`.textus/schemas/<name>.yaml`) declare field shapes, per-field `maintained_by:` ownership, and an `evolution:` block (`added_in`, `deprecated_at`, `migrate_from`). Full contract in SPEC §5.8 and §5.11.
+Schemas (`.textus/schemas/<name>.yaml`) declare field shapes, per-field `maintained_by:` ownership, and an `evolution:` block (`added_in`, `deprecated_at`, `migrate_from`). Full contract in SPEC §5.8.
 
 ## Examples