textus 0.2.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: '092917d22f25afea72145540f4dc8ce32e5a4f6c65889564f54e258e52eab7df'
+  data.tar.gz: 163d891ea50b7c651740f499ffe7ba5191aba0c2c327afbf058a1bf1fc019942
+SHA512:
+  metadata.gz: 1fa336c3387f503c7ac846fea1f7d1de92a9633ea0104d4c0882d48951bf6abbbb1366e0a9028abac1a138f048fe2b90c018228c7b9a19725eb251c835f43b6c
+  data.tar.gz: 2f99647d031c7f0004d4b0264c51a616f8da03bcb72a68f4e12c0e483b5865ee3d826786f7d519bb4d7aac2a8ec9d4f9a1092220ce376a226e1436e9c1e5affc

data/CHANGELOG.md

@@ -0,0 +1,163 @@
+# Changelog
+
+All notable changes to this project are documented in this file.
+The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+The **gem version** (`0.x.y`) is distinct from the **protocol version**
+(currently `textus/1`, embedded in every envelope as `protocol`). The protocol
+is additive within a major; a new major would change the wire string.
+
+## [Unreleased]
+
+## [0.2.0] — 2026-05-20 — Storage rewrite, agent surface, extension DSL (BREAKING)
+
+This release reshapes textus from a markdown-only frontmatter store into a
+multi-format, agent-introspectable context layer.
+
+### Breaking changes
+- **Per-entry formats.** Markdown is no longer the only storage shape. Manifest
+  entries declare `format: markdown|json|yaml|text`; format is inferred from
+  the path extension when omitted. JSON/YAML entries store `_meta` at the top
+  level (`generated_at`, `from`, `template`, `reducer`).
+- **Strict key grammar.** Segments must match `/^[a-z0-9][a-z0-9-]*$/`. No
+  underscores, no uppercase, no dots-in-segments. Max 8 segments × 64 chars.
+  Enforced at manifest load, `put`, and `mv`. Existing stores with illegal
+  segments migrate via `textus migrate-keys --dry-run|--write`.
+- **Publisher rename.** `Textus::Symlink` is now `Textus::Publisher`. Symlink
+  mode is gone; publish is `FileUtils.cp` + sentinel.
+- **Sentinels relocated.** `.textus-managed.json` files move from beside the
+  published file to `<store_root>/sentinels/<target_rel>.textus-managed.json`.
+  Legacy sibling sentinels are auto-migrated on next publish.
+- **Init profiles removed.** `textus init --profile=…` and
+  `lib/textus/profiles/*.yaml` are gone. `textus init` writes a single default
+  manifest declaring all five SPEC zones and pre-creates `zones/<name>/`
+  subdirectories.
+- **Extension surface (carried over from earlier 0.2 work).** `.textus/parsers/`
+  and `.textus/calculators/` are gone — drop `.rb` files into
+  `.textus/extensions/`. `Textus::Parsers` and `Textus::Calculators` modules
+  removed. Manifest `source.parse`/`source.from` → `source.fetcher` +
+  `source.config`; projection `transform:` → `reducer:`; `hooks:` → `events:`
+  (no `on_` prefix; `on_stale` removed entirely). CLI: `--parse=NAME` removed;
+  `textus hooks list` removed (use `textus extensions list --kind=hook`).
+
+### Added
+- **`textus intro`** — single-call orientation envelope (`zones`, `entries`,
+  `extensions`, `write_flows`, `cli_verbs`) for agents landing in a textus-
+  managed project.
+- **`inject_intro: true`** flag on derived markdown/text entries — merges the
+  intro envelope into the template data so `CLAUDE.md` (or any boot doc) can
+  render an orientation preamble.
+- **`textus doctor`** — health check with 8 categories (missing schemas /
+  templates / extensions, illegal nested keys, sentinel orphan/drift, audit log
+  readability, unowned schema fields). `ok: true` only when zero error-level
+  issues; warnings/info don't flip the bit.
+- **`textus mv <old> <new>`** — same-zone, same-format rename. Preserves uid,
+  writes an `mv` audit row with `from_key`, `to_key`, `from_path`, `to_path`,
+  `uid`. `--dry-run` plans without writing.
+- **`uid:`** field — 16-char hex stable identity (`SecureRandom.hex(8)`),
+  auto-minted on first `Store#put`, preserved across writes and moves. Lives
+  in frontmatter for markdown, `_meta.uid` for json/yaml. Surfaced on the
+  envelope.
+- **`publish_each:`** template on nested entries — each leaf byte-copies to a
+  per-leaf target derived from `{leaf}`, `{basename}`, `{key}`, `{ext}`. Closes
+  the per-file publish loop for plugins that mirror `working.*` into
+  `agents/`, `skills/<name>/SKILL.md`, `commands/`.
+- **`textus migrate-keys`** — run-once helper renaming files whose basenames
+  violate the new strict key grammar. `--dry-run` reports proposed renames and
+  collisions; `--write` applies them bottom-up and writes `migrate-keys` audit
+  rows.
+- **Per-format strategies** under `lib/textus/entry/{markdown,json,yaml,text}.rb`
+  with a uniform parse/serialize contract. `Entry.for_format(name)` dispatcher.
+- **`textus.intro` + CLAUDE.md preamble** — the example's CLAUDE.md now opens
+  with auto-generated zone-and-write-flow orientation for the agent.
+- **Actionable error hints.** Every `Textus::Error` exposes `code`, `message`,
+  and `hint`. `UnknownKey` carries up to 5 ranked "did you mean" suggestions
+  (shared-prefix + bounded Levenshtein). The CLI prints
+  `code: msg\n  → hint` to stderr alongside the JSON envelope on stdout.
+- **Extension surface (carried over from earlier 0.2 work).**
+  `Textus.fetcher`, `Textus.reducer`, `Textus.hook` DSL verbs. Per-`Store`
+  `ExtensionRegistry` (no global state). `textus refresh KEY --as=script`.
+  `textus extensions list [--kind=fetcher|reducer|hook]`. In-process lifecycle
+  events (`:put`, `:delete`, `:refresh`, `:build`, `:accept`). `Textus::Refresh`
+  driver with 2 s timeout. `Textus::StoreView` read-only proxy. Audit log gains
+  a 7th JSON-extras column. `Init.run` scaffolds `.textus/extensions/` with a
+  README stub.
+
+### Fixed
+- `Projection#run` no longer stamps `generated_at` onto Hash reducer results —
+  `_meta.generated_at` is the single source of truth for structured outputs
+  (avoids duplicate timestamps in `marketplace.json`-style files).
+- `UnknownKey` raised from `Store#get`/`put`/`mv` (nested-tree file misses) now
+  carries suggestions, matching `Manifest#resolve`.
+
+### Example
+- `examples/claude-plugin/` rewritten as a real Claude Code plugin
+  (`voice-tools`) entirely managed by textus: `.claude-plugin/{plugin,
+  marketplace}.json` and `CLAUDE.md` derived; `agents/`, `skills/<name>/
+  SKILL.md`, `commands/` mirrored via `publish_each:`; pending-zone walkthrough
+  exercising the AI propose → human accept loop; intake fetcher, in-process
+  reducer / hook, deep-nested key demos.
+
+### SPEC
+- Rewritten for §1–§5 (layers, manifest, formats, publish), §10 (envelope adds
+  `format`, `content`, `uid`), audit verb table (`mv`, `migrate-keys`), CLI
+  verb table, Fixture G.
+
+## [0.1.0] — 2026-05-19
+
+First public release. Implements protocol `textus/1`.
+
+### Added — storage and model
+- Zone-based storage layout under `.textus/zones/`: `canon`, `working`, `intake`,
+  `pending`, `derived`. Each zone declares `writable_by` roles in the manifest.
+- Role resolution order: `--as` flag → `TEXTUS_ROLE` env → `.textus/role` file →
+  default `human`. Recognized roles: `human`, `ai`, `script`, `build`.
+- Append-only TSV audit log at `.textus/audit.log`, file-locked on every write.
+- Schemas with required/optional fields, type checking, and per-field
+  `maintained_by` ownership plus `evolution` metadata (`added_in`,
+  `migrate_from`).
+- Manifest backwards compatibility: a manifest without `zones:` synthesizes the
+  legacy `fixed` / `state` / `derived` zones.
+
+### Added — compute and publish
+- Vendored Mustache renderer (~120 LOC, depth-bounded at 8).
+- Projection engine: `select` / `pluck` / `sort_by` / `limit` / `transform`
+  (1000-row cap, single 2 s timeout on transforms).
+- `textus build` materializes derived entries from `projection:` + `template:`
+  declarations.
+- Atomic symlink publish via `publish_to:`, with copy-mode fallback and a
+  `.textus-managed.json` sentinel for filesystems without symlinks.
+
+### Added — extension points
+- **Parsers** — built-ins for `json`, `csv`, `markdown-links`, `ical-events`,
+  `rss`. Auto-load from `.textus/parsers/<name>.rb`, 2 s timeout.
+- **Calculators** — pure projection-row transforms registered via
+  `Textus::Calculators.register`. Auto-load from `.textus/calculators/<name>.rb`,
+  2 s timeout.
+- **Hooks** — manifest entries declare `hooks:` keyed by lifecycle event
+  (`on_put`, `on_delete`, `on_refresh`, `on_stale`, `on_accept`, `on_build`).
+  textus enumerates hooks via `textus hooks list`; external runners execute.
+
+### Added — CLI verbs
+- Read: `list`, `where`, `get`, `schema`, `stale`, `deps`, `rdeps`, `published`,
+  `validate-all`, `hooks list`.
+- Write: `put`, `delete`, `build`, `accept`.
+- Scaffolding: `init` (profiles: `personal`, `claude-plugin`), `schema-init`,
+  `schema-diff`, `schema-migrate`.
+- Flags: `--as=ROLE`, `--zone=Z`, `--prefix=KEY`, `--parse=NAME`, `--if-etag=E`,
+  `--stale`, `--strict`, `--format=json`.
+
+### Added — examples
+- `examples/claude-plugin/` — a working `.textus/` tree that publishes a
+  Claude Code `CLAUDE.md` and a `marketplace.json` via projection + Mustache.
+  Demonstrates intake, parsers, calculators, hooks, and schema field ownership.
+- `examples/mcp-server/` — 50-line MCP server wrapping `textus get` / `list` /
+  `put` as tools.
+
+### Added — quality tooling
+- RuboCop config with rubocop-rspec; codebase passes with zero offenses.
+- Lefthook hooks (`pre-commit` runs rubocop on staged Ruby; `pre-push` runs
+  rspec + rubocop). Install via `brew bundle install`.
+- `gemspec` packages only `lib/`, `exe/`, `README.md`, `SPEC.md`, and two
+  curated `docs/` files. Internal plan documents stay out of the gem.

data/README.md

@@ -0,0 +1,200 @@
+# textus
+
+Reference Ruby implementation of the **textus/1** protocol — a storage convention and JSON wire protocol for agent-readable project memory: addressable dotted keys, schema-validated entries (markdown, JSON, YAML, or text per entry), role-gated writes, declarative compute, and copy-based publish targets.
+
+See [`SPEC.md`](SPEC.md) for the protocol. Implementation notes live in [`docs/`](docs/).
+
+## Versioning
+
+Two versions, deliberately independent:
+
+- **Protocol wire string:** `textus/1`. Stable; breaking changes require `textus/2`.
+- **Gem version:** semver, currently `0.2.0`. Gem `0.x.y` and `1.x` both speak `textus/1`.
+
+Envelope payloads carry the `protocol` field; the gem version is irrelevant to the wire format.
+
+## Install
+
+```sh
+gem install textus     # when published
+```
+
+Or from this repo:
+
+```sh
+bundle install
+bundle exec exe/textus --help
+```
+
+## Quick start
+
+Bootstrap a fresh tree:
+
+```sh
+bundle exec exe/textus init
+```
+
+This scaffolds `.textus/` with a starter manifest, the five zone directories, baseline schemas, and an empty audit log. The resulting layout:
+
+```
+.textus/
+  manifest.yaml
+  audit.log
+  role
+  schemas/
+  templates/
+  extensions/
+  zones/
+    canon/       # human-only
+    working/     # human, ai, script
+    intake/      # script (declared external inputs)
+    pending/     # ai (proposals awaiting accept)
+    derived/     # build only (computed outputs)
+```
+
+A minimal `manifest.yaml`:
+
+```yaml
+version: textus/1
+
+zones:
+  - { name: canon,   writable_by: [human] }
+  - { name: working, writable_by: [human, ai, script] }
+  - { name: intake,  writable_by: [script] }
+  - { name: pending, writable_by: [ai] }
+  - { name: derived, writable_by: [build] }
+
+entries:
+  - key: canon.identity
+    path: canon/identity.md
+    zone: canon
+    schema: identity
+
+  - key: working.network.org
+    path: working/network/org
+    zone: working
+    schema: person
+    owner: textus:network
+    nested: true
+```
+
+Manifest `path:` fields are relative to `.textus/zones/` — implementations prepend `zones/` when resolving. So `working.network.org.jane` lives at `.textus/zones/working/network/org/jane.md`.
+
+Read and write:
+
+```sh
+textus get working.network.org.jane --format=json
+textus list --zone=working --format=json
+echo '{"frontmatter":{"name":"bob","relationship":"peer","org":"acme"},"body":"hi\n"}' \
+  | textus put working.network.org.bob --as=human --stdin --format=json
+textus stale --zone=derived --format=json
+```
+
+## CLI verbs
+
+All verbs accept `--format=json` and emit the envelope defined in SPEC §8. Write verbs require `--as=<role>` (subject to role-resolution order, §5.1).
+
+**Read verbs (no role required):**
+
+| Verb | Purpose |
+|---|---|
+| `list [--prefix=K] [--zone=Z] [--stale]` | Enumerate keys, optionally filtered |
+| `where K` | Resolve a key to its filesystem path |
+| `get K` | Return the full envelope |
+| `schema K` | Return the schema bound to an entry |
+| `stale [--prefix=K] [--zone=Z] [--strict]` | List stale derived/intake entries |
+| `deps K` / `rdeps K` | Forward/reverse projection dependencies |
+| `published` | List `publish_to:` targets and their backing keys |
+| `validate-all` | Validate every entry against its schema (incl. `maintained_by`) |
+| `extensions list [--kind=K]` | Enumerate registered fetchers, reducers, and declared hooks |
+
+**Write verbs (role-gated per zone):**
+
+| Verb | Role |
+|---|---|
+| `put K --stdin --as=R [--fetcher=NAME]` | per zone |
+| `delete K --if-etag=E --as=R` | per zone |
+| `refresh K --as=script` | per zone (typically `script`) |
+| `build [--prefix=K] [--dry-run]` | `build` |
+| `accept K --as=human` | `human` only |
+
+**Scaffolding (human-only):**
+
+| Verb | Purpose |
+|---|---|
+| `init` | Scaffold a fresh `.textus/` tree |
+| `schema-init NAME` | Write a stub schema |
+| `schema-diff NAME` | Compare on-disk schema against entries claiming it |
+| `schema-migrate NAME [--rename=OLD:NEW]` | Rewrite frontmatter keys across affected entries |
+
+## Zones and roles
+
+| Zone | `writable_by` | Purpose |
+|---|---|---|
+| `canon` | `[human]` | Identity, voice, immutable principles |
+| `working` | `[human, ai, script]` | Active project state — notes, decisions, network |
+| `intake` | `[script]` | Declared external inputs (calendar, feeds, scraped pages) |
+| `pending` | `[ai]` | AI proposals awaiting `textus accept` |
+| `derived` | `[build]` | Computed outputs from `textus build` |
+
+The effective role for any CLI call is resolved in order: `--as` flag, then `TEXTUS_ROLE` env, then `.textus/role`, then default `human`. Mismatches return `write_forbidden`. Every write records the resolved role in `.textus/audit.log`.
+
+## Compute layer
+
+Derived entries are not authored by hand. Each declares a `projection:` block (select prefixes, pluck fields, optional sort/limit/transform) and optionally a Mustache template under `.textus/templates/`. textus implements a deliberately restricted Mustache subset (variables, sections, inverted sections, comments — no partials, no lambdas, no HTML escaping). Results are bounded at 1000 rows; template recursion at depth 8.
+
+Derived entries may declare `format:` to be `markdown` (default), `json`, `yaml`, or `text`. The in-store file is the consumer-shaped artifact — `cat .textus/zones/derived/marketplace.json` returns valid JSON without going through textus. `publish_to:` then performs a byte-for-byte file copy of that artifact to each destination, alongside a `.textus-managed.json` sentinel. See SPEC §5.2, §5.3, and §5.12.
+
+## Extension points
+
+Three DSL verbs:
+
+- **`Textus.fetcher(:name) do |config:, store:|`** — pulls data into an intake entry. Returns one of `{ frontmatter:, body: }`, `{ content: }` (for `format: json|yaml` entries), or `{ body: }` (raw bytes); the store normalizes all three. Configured via `source.fetcher` in the manifest. Five built-ins ship out of the box: `json`, `csv`, `markdown-links`, `ical-events`, `rss`.
+- **`Textus.reducer(:name) do |rows:, config:|`** — shapes rows in a derived projection. Pure function. Configured via `projection.reducer`.
+- **`Textus.hook(:event, :name) do |kwargs|`** — reacts to a lifecycle event. Five events: `:put`, `:delete`, `:refresh`, `:build`, `:accept`.
+
+Extension files live in `.textus/extensions/*.rb` (one per registration, by convention). Each Store instance gets its own registry; no global state.
+
+See SPEC.md §5.11 for the full contract.
+
+Schema fields may also declare `maintained_by:` and a top-level `evolution:` block (`added_in`, `deprecated_at`, `migrate_from`). SPEC §5.8.
+
+## Examples
+
+- [`examples/claude-plugin/`](examples/claude-plugin/) — full tour: fetcher, reducer, lifecycle events, schema ownership, and a `derived.claude.root` entry published to `CLAUDE.md`.
+- [`examples/mcp-server/`](examples/mcp-server/) — 50-line MCP server wrapping `textus get/put` as tools.
+
+## Tests
+
+```sh
+bundle exec rspec
+```
+
+Runs the full suite, including conformance fixtures A–I from SPEC §12.
+
+## Code quality
+
+```sh
+bundle exec rubocop      # lint
+bundle exec rubocop -A   # lint + autocorrect
+```
+
+Git hooks via [Lefthook](https://github.com/evilmartians/lefthook):
+
+```sh
+brew bundle install      # installs lefthook (see Brewfile)
+lefthook install         # writes .git/hooks/{pre-commit,pre-push}
+```
+
+Git hooks (defined in `lefthook.yml`):
+- `pre-commit` — runs `rubocop` on staged Ruby files.
+- `pre-push` — runs the full `rspec` suite and `rubocop` over the tree.
+
+Bypass with `LEFTHOOK=0 git commit ...` when needed.
+
+CI runs `rspec` (Ruby 3.3 / 3.4) and `rubocop` via GitHub Actions
+([`.github/workflows/ci.yml`](.github/workflows/ci.yml)).
+
+## License
+
+MIT.