textus 0.4.0 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +147 -2
- data/README.md +38 -28
- data/SPEC.md +84 -147
- data/docs/architecture.md +82 -28
- data/lib/textus/builder/pipeline.rb +56 -0
- data/lib/textus/builder/renderer/json.rb +42 -0
- data/lib/textus/builder/renderer/markdown.rb +22 -0
- data/lib/textus/builder/renderer/text.rb +14 -0
- data/lib/textus/builder/renderer/yaml.rb +42 -0
- data/lib/textus/builder/renderer.rb +17 -0
- data/lib/textus/builder.rb +9 -114
- data/lib/textus/cli/group/hook.rb +11 -0
- data/lib/textus/cli/group/key.rb +12 -0
- data/lib/textus/cli/group/schema.rb +13 -0
- data/lib/textus/cli/group.rb +51 -0
- data/lib/textus/cli/verb/accept.rb +15 -0
- data/lib/textus/cli/verb/build.rb +13 -0
- data/lib/textus/cli/verb/delete.rb +16 -0
- data/lib/textus/cli/verb/deps.rb +12 -0
- data/lib/textus/cli/verb/doctor.rb +15 -0
- data/lib/textus/cli/verb/get.rb +12 -0
- data/lib/textus/cli/verb/hook_run.rb +48 -0
- data/lib/textus/cli/verb/hooks.rb +50 -0
- data/lib/textus/cli/verb/init.rb +14 -0
- data/lib/textus/cli/verb/intro.rb +11 -0
- data/lib/textus/cli/verb/list.rb +14 -0
- data/lib/textus/cli/verb/migrate_keys.rb +16 -0
- data/lib/textus/cli/verb/mv.rb +17 -0
- data/lib/textus/cli/verb/published.rb +11 -0
- data/lib/textus/cli/verb/put.rb +50 -0
- data/lib/textus/cli/verb/rdeps.rb +12 -0
- data/lib/textus/cli/verb/refresh.rb +15 -0
- data/lib/textus/cli/verb/schema.rb +12 -0
- data/lib/textus/cli/verb/schema_diff.rb +12 -0
- data/lib/textus/cli/verb/schema_init.rb +16 -0
- data/lib/textus/cli/verb/schema_migrate.rb +16 -0
- data/lib/textus/cli/verb/stale.rb +14 -0
- data/lib/textus/cli/verb/uid.rb +12 -0
- data/lib/textus/cli/verb/where.rb +12 -0
- data/lib/textus/cli/verb.rb +62 -0
- data/lib/textus/cli.rb +44 -385
- data/lib/textus/doctor/check/audit_log.rb +50 -0
- data/lib/textus/doctor/check/hooks.rb +29 -0
- data/lib/textus/doctor/check/illegal_keys.rb +49 -0
- data/lib/textus/doctor/check/manifest_files.rb +38 -0
- data/lib/textus/doctor/check/schema_violations.rb +22 -0
- data/lib/textus/doctor/check/schemas.rb +26 -0
- data/lib/textus/doctor/check/sentinels.rb +57 -0
- data/lib/textus/doctor/check/templates.rb +26 -0
- data/lib/textus/doctor/check/unowned_schema_fields.rb +34 -0
- data/lib/textus/doctor/check.rb +30 -0
- data/lib/textus/doctor.rb +29 -264
- data/lib/textus/entry/base.rb +30 -0
- data/lib/textus/entry/json.rb +11 -5
- data/lib/textus/entry/markdown.rb +5 -5
- data/lib/textus/entry/text.rb +4 -4
- data/lib/textus/entry/yaml.rb +11 -5
- data/lib/textus/entry.rb +2 -7
- data/lib/textus/envelope.rb +30 -0
- data/lib/textus/errors.rb +2 -2
- data/lib/textus/hooks/builtin.rb +70 -0
- data/lib/textus/hooks/dispatcher.rb +49 -0
- data/lib/textus/hooks/loader.rb +26 -0
- data/lib/textus/hooks/registry.rb +73 -0
- data/lib/textus/init.rb +14 -11
- data/lib/textus/intro.rb +16 -18
- data/lib/textus/key/distance.rb +55 -0
- data/lib/textus/key/grammar.rb +33 -0
- data/lib/textus/key/path.rb +17 -0
- data/lib/textus/manifest/entry.rb +199 -0
- data/lib/textus/manifest.rb +20 -254
- data/lib/textus/migrate_keys.rb +1 -1
- data/lib/textus/projection.rb +6 -5
- data/lib/textus/proposal.rb +4 -4
- data/lib/textus/refresh.rb +17 -17
- data/lib/textus/schema/tools.rb +89 -0
- data/lib/textus/store/audit_log.rb +71 -0
- data/lib/textus/store/mover.rb +121 -0
- data/lib/textus/store/reader.rb +67 -0
- data/lib/textus/store/staleness.rb +133 -0
- data/lib/textus/store/validator.rb +56 -0
- data/lib/textus/store/view.rb +29 -0
- data/lib/textus/store/writer.rb +132 -0
- data/lib/textus/store.rb +26 -527
- data/lib/textus/version.rb +2 -2
- data/lib/textus.rb +14 -29
- metadata +78 -8
- data/lib/textus/audit_log.rb +0 -32
- data/lib/textus/builtin_actions.rb +0 -68
- data/lib/textus/extension_registry.rb +0 -61
- data/lib/textus/extensions.rb +0 -33
- data/lib/textus/key_distance.rb +0 -53
- data/lib/textus/schema_tools.rb +0 -87
- data/lib/textus/store_view.rb +0 -27
data/SPEC.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
|
-
# textus/
|
|
1
|
+
# textus/2 — Specification
|
|
2
2
|
|
|
3
|
-
**Status:** Draft
|
|
4
|
-
**Protocol identifier:** `textus/
|
|
3
|
+
**Status:** Draft v2.0 (2026-05-19)
|
|
4
|
+
**Protocol identifier:** `textus/2`
|
|
5
5
|
**Reference implementation:** Ruby gem `textus`
|
|
6
6
|
|
|
7
7
|
> *textus* — Latin for "the fabric a text is woven from," same root as *context* (from *con-texere*, "to weave together"). This spec defines a storage shape and wire protocol for that fabric.
|
|
@@ -31,7 +31,7 @@ textus is organized as five composable layers. Each layer has a single responsib
|
|
|
31
31
|
## 2. Goals and non-goals
|
|
32
32
|
|
|
33
33
|
**Goals**
|
|
34
|
-
- Stable wire format (`textus/
|
|
34
|
+
- Stable wire format (`textus/2`) any language can speak.
|
|
35
35
|
- Deterministic read/write of structured Markdown via a CLI returning JSON.
|
|
36
36
|
- Schema-validated frontmatter using YAML schemas as data.
|
|
37
37
|
- Role-based write gates (humans, scripts, AI, build runners get different permissions per zone).
|
|
@@ -91,7 +91,7 @@ The manifest declares: (a) which zones exist and which roles may write to each,
|
|
|
91
91
|
|
|
92
92
|
```yaml
|
|
93
93
|
# .textus/manifest.yaml
|
|
94
|
-
version: textus/
|
|
94
|
+
version: textus/2
|
|
95
95
|
|
|
96
96
|
zones:
|
|
97
97
|
- name: canon
|
|
@@ -139,7 +139,7 @@ zones:
|
|
|
139
139
|
|
|
140
140
|
Old manifests written against textus/1 draft v0.1 therefore parse without modification, and any tooling expecting `fixed`/`state`/`derived` continues to work.
|
|
141
141
|
|
|
142
|
-
**Key grammar (enforced from v1.2):** dotted segments matching `/^[a-z0-9][a-z0-9-]*$/`. Segments are joined by `.`. A key has at most 8 segments; each segment is at most 64 characters. Segments MUST NOT contain dots, slashes, uppercase letters, or underscores. Example: `working.projects.acme.dashboard`. Enforcement points: manifest load (rejects illegal `key:` declarations and illegal nested file/directory names), `put` (rejects illegal keys before any write), `enumerate` (filters and warns on illegal filenames so existing trees still load with a clear migration message). Run-once migration: `textus migrate
|
|
142
|
+
**Key grammar (enforced from v1.2):** dotted segments matching `/^[a-z0-9][a-z0-9-]*$/`. Segments are joined by `.`. A key has at most 8 segments; each segment is at most 64 characters. Segments MUST NOT contain dots, slashes, uppercase letters, or underscores. Example: `working.projects.acme.dashboard`. Enforcement points: manifest load (rejects illegal `key:` declarations and illegal nested file/directory names), `put` (rejects illegal keys before any write), `enumerate` (filters and warns on illegal filenames so existing trees still load with a clear migration message). Run-once migration: `textus key migrate --dry-run` then `--write` (see §audit).
|
|
143
143
|
|
|
144
144
|
**Per-entry `format:` (enforced from v1.2):** an entry MAY declare `format:` to be one of `markdown` (default), `json`, `yaml`, or `text`. The `format` controls the on-disk shape and which path extension is required:
|
|
145
145
|
|
|
@@ -250,38 +250,36 @@ A sentinel is written for each published file at `<store_root>/sentinels/<target
|
|
|
250
250
|
|
|
251
251
|
**Per-leaf publishing.** A nested entry MAY declare `publish_each:` instead of `publish_to:` (see §4). When the build runs, every leaf reachable under the nested entry is byte-copied to the path produced by substituting `{leaf}` / `{basename}` / `{key}` / `{ext}` in the template, with a sentinel written under `<store_root>/sentinels/` at the mirrored target path. The build envelope grows a `published_leaves` array — one row per leaf, with `key`, `source`, and `target` — alongside the existing `built` array. Targets that would resolve outside the repo root are refused.
|
|
252
252
|
|
|
253
|
-
### 5.4 Intake (declared, refreshed via registered
|
|
253
|
+
### 5.4 Intake (declared, refreshed via registered fetch hook)
|
|
254
254
|
|
|
255
|
-
Intake entries declare an external source by naming
|
|
255
|
+
Intake entries declare an external source by naming a **fetch hook** — a registered, named function that pulls data into the entry. textus itself still makes no implicit network calls: a fetch hook only runs in intake mode when explicitly invoked by `textus refresh KEY --as=script`. The declaration is data only:
|
|
256
256
|
|
|
257
257
|
```yaml
|
|
258
258
|
- key: intake.calendar.events
|
|
259
259
|
zone: intake
|
|
260
260
|
source:
|
|
261
|
-
|
|
261
|
+
fetch: ical-events
|
|
262
262
|
config:
|
|
263
263
|
url: "https://calendar.google.com/.../basic.ics"
|
|
264
264
|
ttl: 6h
|
|
265
265
|
```
|
|
266
266
|
|
|
267
|
-
`
|
|
267
|
+
`fetch` names a registered `:fetch` hook (see §5.10 for the hook contract); `config` is an opaque hash handed to the hook; `ttl` is the staleness budget. Implementations MUST reject legacy `source.from`, `source.parse`, `source.fetcher`, and `source.action` with a clear usage error.
|
|
268
268
|
|
|
269
|
-
|
|
269
|
+
In intake mode the hook MUST return one of three shapes, all normalized by the store into its internal `{_meta, body, content}` representation (§5.12):
|
|
270
270
|
|
|
271
|
-
- `{
|
|
271
|
+
- `{ _meta:, body: }` — markdown-friendly; `_meta` becomes the entry's parsed metadata hash.
|
|
272
272
|
- `{ content: }` — for `format: json|yaml` entries; the parsed object becomes the entry's content.
|
|
273
273
|
- `{ body: }` — raw bytes for `text` or for any format that prefers verbatim writes; the store re-parses and validates per `format:`.
|
|
274
274
|
|
|
275
|
-
|
|
276
|
-
|
|
277
|
-
**Built-in actions.** `json`, `csv`, `markdown-links`, `ical-events`, `rss` are always available. They expect raw bytes in `config["bytes"]` and produce structured frontmatter/body. Built-ins do not perform I/O themselves — the caller (or an outer action) is responsible for supplying bytes.
|
|
275
|
+
**Built-in fetch hooks.** `json`, `csv`, `markdown-links`, `ical-events`, `rss` are always available. They expect raw bytes in `config["bytes"]` and produce structured `_meta`/body. Built-ins do not perform I/O themselves — the caller (or an outer hook) is responsible for supplying bytes.
|
|
278
276
|
|
|
279
277
|
**Refresh paths.** Two are supported:
|
|
280
278
|
|
|
281
|
-
1. **In-process** — `textus refresh KEY --as=script` resolves the entry's `source.
|
|
279
|
+
1. **In-process** — `textus refresh KEY --as=script` resolves the entry's `source.fetch`, invokes the registered `:fetch` hook with `(config:, store:, args: {})`, and writes the result under role `script`.
|
|
282
280
|
2. **External runner** — a cron job or agent harness reads `textus list --zone=intake --stale --format=json`, fetches the source out of band, and pipes bytes back through `textus put KEY --as=script --stdin`.
|
|
283
281
|
|
|
284
|
-
Both paths share the same role gate, audit-log entry, and `:refresh` event
|
|
282
|
+
Both paths share the same role gate, audit-log entry, and `:refresh` event. User-supplied hooks live in `.textus/hooks/*.rb` and auto-load at `Store#initialize` — see §5.10 for the full hook contract.
|
|
285
283
|
|
|
286
284
|
### 5.5 Pending / accept workflow
|
|
287
285
|
|
|
@@ -306,15 +304,19 @@ Proposed body content.
|
|
|
306
304
|
|
|
307
305
|
### 5.6 Audit log
|
|
308
306
|
|
|
309
|
-
Every successful write appends one
|
|
307
|
+
Every successful write appends one compact JSON object (NDJSON) to `.textus/audit.log`. The file is opened with `flock(LOCK_EX)` for the duration of each append so concurrent writers serialize cleanly.
|
|
310
308
|
|
|
311
|
-
Schema (
|
|
309
|
+
Schema (one JSON object per line, no interior whitespace):
|
|
312
310
|
|
|
311
|
+
```json
|
|
312
|
+
{"ts":"<iso8601-utc>","role":"<role>","verb":"<verb>","key":"<key>","etag_before":<etag-or-null>,"etag_after":<etag-or-null>}
|
|
313
313
|
```
|
|
314
|
-
<iso8601-utc>\t<role>\t<verb>\t<key>\t<etag-before-or-NULL>\t<etag-after-or-NULL>
|
|
315
|
-
```
|
|
316
314
|
|
|
317
|
-
|
|
315
|
+
`ts` is the wall-clock timestamp in UTC with second precision. `role` is the resolved role for the invocation. `verb` is the audit-log payload string identifying the operation (`put`, `delete`, `accept`, `compute`, `migrate-keys`, `mv`, ...). Note that `migrate-keys` here is the on-disk payload key — the CLI surface is `textus key migrate`; the payload string is retained for log stability. `key` is the affected entry key. `etag_before` and `etag_after` are the entry etags before and after the write, or JSON `null` when not applicable (e.g. create has no before-etag, delete has no after-etag). `key migrate --write` emits one line per renamed file (with payload `verb: "migrate-keys"`) using the new key as `key` and the file's pre- and post-rename etags.
|
|
316
|
+
|
|
317
|
+
For `mv`, the structural fields `from_key`, `to_key`, and `uid` appear at the top level of the JSON object. Remaining verb-specific data (e.g. `from_path`, `to_path`) is nested under an `extras` key. The `extras` key is omitted entirely when empty.
|
|
318
|
+
|
|
319
|
+
**Backward compatibility (v0.5):** files written by v0.4 and earlier contain TSV rows. Readers MUST accept mixed-format files: lines starting with `{` are parsed as JSON; other lines are treated as legacy TSV (`ts\trole\tverb\tkey\tetag_before\tetag_after[\tjson_extras]`). TSV write support is removed in v0.6.
|
|
318
320
|
|
|
319
321
|
### 5.7 Security bounds
|
|
320
322
|
|
|
@@ -351,118 +353,51 @@ evolution:
|
|
|
351
353
|
OLD_FIELD: NEW_FIELD
|
|
352
354
|
```
|
|
353
355
|
|
|
354
|
-
`textus schema
|
|
356
|
+
`textus schema migrate NAME` consults `evolution.migrate_from` when invoked without `--rename=OLD:NEW`, applying every declared rename across affected entries in one pass. An explicit `--rename` flag overrides the schema-declared map for that invocation.
|
|
355
357
|
|
|
356
358
|
**Backwards compat:** v1.0 schemas (no `fields:`, no `evolution:`) continue to parse and behave identically. `schema.maintained_by(field)` returns `nil` for every field; `schema.evolution` returns `{}`.
|
|
357
359
|
|
|
358
|
-
**Override rule:** the role `human` is permitted to write any `maintained_by` field, regardless of declared owner. This preserves human authority over AI/script-managed data — humans curating canon over AI-written embeddings is a feature, not a bug. All other role mismatches are reported by `
|
|
360
|
+
**Override rule:** the role `human` is permitted to write any `maintained_by` field, regardless of declared owner. This preserves human authority over AI/script-managed data — humans curating canon over AI-written embeddings is a feature, not a bug. All other role mismatches are reported by `doctor --check=schema_violations` with code `role_authority`, including fields `key`, `field`, `expected`, and `last_writer`.
|
|
359
361
|
|
|
360
362
|
### 5.9 Reducers (v1.2)
|
|
361
363
|
|
|
362
|
-
Reducers are
|
|
364
|
+
Reducers are RPC hooks on the `:reduce` event. See §5.10.
|
|
363
365
|
|
|
364
|
-
|
|
365
|
-
Textus.reducer(:rank_by_recency) do |rows:, config:|
|
|
366
|
-
rows.sort_by { |r| r["updated_at"].to_s }.reverse
|
|
367
|
-
end
|
|
368
|
-
```
|
|
369
|
-
|
|
370
|
-
**Declaration.** A projection opts into a reducer via `projection.reducer`, with optional `projection.reducer_config`:
|
|
366
|
+
### 5.10 Hooks
|
|
371
367
|
|
|
372
|
-
|
|
373
|
-
projection:
|
|
374
|
-
select: [working.projects]
|
|
375
|
-
pluck: [name, status, updated_at]
|
|
376
|
-
reducer: rank_by_recency
|
|
377
|
-
reducer_config: { tiebreak: name }
|
|
378
|
-
sort_by: updated_at
|
|
379
|
-
limit: 50
|
|
380
|
-
```
|
|
368
|
+
textus has a single extension verb: `Textus.hook(event, name, **opts) { ... }`. The EVENTS table below defines every extension point. Files in `.textus/hooks/*.rb` are `load`ed at `Store#initialize` in lexical order.
|
|
381
369
|
|
|
382
|
-
|
|
370
|
+
| Event | Mode | Args | Return | Failure |
|
|
371
|
+
|----------|---------|-----------------------------------|---------------|---------|
|
|
372
|
+
| :fetch | rpc | store:, config:, args: | {_meta:, body:} | aborts op |
|
|
373
|
+
| :reduce | rpc | store:, rows:, config: | rows array | aborts op |
|
|
374
|
+
| :check | rpc | store: | issues array | aborts doctor |
|
|
375
|
+
| :put | pubsub | store:, key:, envelope: | (discarded) | logged |
|
|
376
|
+
| :delete | pubsub | store:, key: | (discarded) | logged |
|
|
377
|
+
| :refresh | pubsub | store:, key:, envelope:, change: | (discarded) | logged |
|
|
378
|
+
| :build | pubsub | store:, key:, envelope:, sources: | (discarded) | logged |
|
|
379
|
+
| :accept | pubsub | store:, key:, target_key: | (discarded) | logged |
|
|
383
380
|
|
|
384
|
-
**
|
|
381
|
+
**Signature invariant** — every hook receives `store:` as its first keyword argument. Event-specific kwargs follow in stable left-to-right order. The primary entity is always `key:` (for `:accept`, `key:` is the pending key being accepted and `target_key:` is the destination).
|
|
385
382
|
|
|
386
|
-
**
|
|
383
|
+
**RPC mode** — exactly one handler per (event, name). The manifest references the handler by name (`source.fetch: NAME`, `projection.reduce: NAME`). Failure or timeout aborts the calling operation.
|
|
387
384
|
|
|
388
|
-
**
|
|
385
|
+
**Pub-sub mode** — zero or more handlers per event. All matching handlers fire. The `keys:` option restricts a handler to keys matching one of the given globs (`File.fnmatch?` rules). Absence of `keys:` fires on every event of that type. Handler failures and 2s timeouts are logged to `audit.log` as `event_error` rows; they NEVER abort the triggering operation.
|
|
389
386
|
|
|
390
|
-
|
|
387
|
+
The `store:` argument is always a read-only store proxy. Write attempts raise `UsageError`.
|
|
391
388
|
|
|
392
|
-
|
|
393
|
-
|
|
394
|
-
**Event set and kwargs:**
|
|
395
|
-
|
|
396
|
-
| Event | Fired by | Kwargs |
|
|
397
|
-
|-----------|-------------------------|--------------------------------------------------------------|
|
|
398
|
-
| `:put` | `Store#put` | `key:, envelope:, store:` |
|
|
399
|
-
| `:delete` | `Store#delete` | `key:, store:` |
|
|
400
|
-
| `:refresh`| `Refresh.call` | `key:, envelope:, store:, change:` (`:created` or `:updated`)|
|
|
401
|
-
| `:build` | `Builder#materialize` | `key:, envelope:, store:, sources:` |
|
|
402
|
-
| `:accept` | `Proposal.accept` | `pending_key:, target_key:, store:` |
|
|
403
|
-
|
|
404
|
-
`:refresh` with `change: :unchanged` does NOT fire — only `:created` and `:updated` are emitted. The `store:` kwarg is always a `Textus::StoreView` (§5.11).
|
|
405
|
-
|
|
406
|
-
**Timeout and isolation.** Each hook runs under `Timeout.timeout(2)`. Hook errors and timeouts are recorded as `event_error` rows in `.textus/audit.log` (column 7, JSON-encoded extras with `event`, `hook`, `error`) but do NOT abort the triggering operation. The store write that fired the event is already committed by the time hooks run.
|
|
407
|
-
|
|
408
|
-
**Manifest declarations.** A manifest entry MAY declare external-runner hooks under an `events:` block, keyed by event name:
|
|
409
|
-
|
|
410
|
-
```yaml
|
|
411
|
-
events:
|
|
412
|
-
refresh:
|
|
413
|
-
- { exec: scripts/reindex.sh, as: script }
|
|
414
|
-
build:
|
|
415
|
-
- { exec: scripts/rebuild-index.sh, as: build }
|
|
416
|
-
```
|
|
417
|
-
|
|
418
|
-
Textus does NOT invoke these — they surface only through `textus extensions list --kind=hook` for orchestrators (lefthook, cron, CI) to consume. Each entry has `exec` (opaque runner-resolvable string) and `as` (role to claim, defaults to `script`).
|
|
419
|
-
|
|
420
|
-
**Removed.** The v1.1 `on_stale` event is removed in 0.2. Staleness is a poll, surfaced by `textus stale`. The `on_`-prefix convention from v1.1 is gone; events are bare symbols.
|
|
421
|
-
|
|
422
|
-
### 5.11 Extension surface (v1.3)
|
|
423
|
-
|
|
424
|
-
Four DSL verbs cover all user-supplied code:
|
|
425
|
-
|
|
426
|
-
```
|
|
427
|
-
Textus.action(:name) do |config:, store:, args:| ... end # intake mode: returns content; verb mode: writes via store.put
|
|
428
|
-
Textus.reducer(:name) do |rows:, config:| ... end # returns rows
|
|
429
|
-
Textus.hook(:event, :name) do |**kwargs| ... end # side effects; return ignored
|
|
430
|
-
Textus.doctor_check(:name) do |store:| ... end # returns array of issue hashes
|
|
431
|
-
```
|
|
432
|
-
|
|
433
|
-
Files in `.textus/extensions/*.rb` are loaded at `Store#initialize`, in lexical order, with the registry installed as the current registry for that store. Registries are per-Store: two Store instances in the same process do not share state.
|
|
434
|
-
|
|
435
|
-
**Action invocation modes.**
|
|
436
|
-
|
|
437
|
-
| Mode | Invoked by | `config:` | `store:` | `args:` | Return |
|
|
438
|
-
|---------|---------------------------|------------------------|------------------------|----------------------|------------------------------------------|
|
|
439
|
-
| intake | `textus refresh KEY` | manifest `source.config` | writable view (role from `--as`) | `{}` | required; normalized into entry write |
|
|
440
|
-
| verb | `textus action NAME ...` | `{}` | writable view (role from `--as`) | parsed CLI kv hash | ignored |
|
|
441
|
-
| put-fetch | `textus put K --action=N --stdin` | `{ "bytes" => stdin }` | read-only view | `{}` | required; merged into the put payload |
|
|
442
|
-
|
|
443
|
-
**Failure modes:**
|
|
444
|
-
|
|
445
|
-
| Surface | Timeout | Exception | Bad return |
|
|
446
|
-
|-----------------|------------|---------------------------------------------|------------|
|
|
447
|
-
| action | aborts op | aborts op (wrapped as `UsageError`) | aborts op |
|
|
448
|
-
| reducer | aborts op | aborts op | aborts op |
|
|
449
|
-
| hook | logged | logged (audit `event_error` row) | n/a |
|
|
450
|
-
| doctor_check | reported as `doctor_check.timeout` issue | reported as `doctor_check.failed` issue | reported as `doctor_check.bad_return` |
|
|
451
|
-
|
|
452
|
-
Actions and reducers are pure transforms in modes where their return matters; their return values flow into the store. Hooks and doctor_checks shape side outputs (event chain, doctor report) — only doctor_check return values are merged.
|
|
453
|
-
|
|
454
|
-
The `store:` argument is always a `Textus::StoreView`. In intake and verb modes it is writable and bound to the calling role; in put-fetch mode and inside doctor_checks it is read-only. Write attempts on a read-only view raise `Textus::UsageError`.
|
|
389
|
+
Each handler runs under `Timeout.timeout(2)`.
|
|
455
390
|
|
|
456
391
|
### 5.12 Storage formats (v1.2)
|
|
457
392
|
|
|
458
|
-
An entry's `format:` selects a storage strategy. All strategies expose the same `parse(bytes) → {
|
|
393
|
+
An entry's `format:` selects a storage strategy. All strategies expose the same `parse(bytes) → {_meta, body, content}` and `serialize(meta:, body:, content:) → bytes` contract. The store, audit, etag, and projection layers operate on the parsed shape; only (de)serialization differs.
|
|
459
394
|
|
|
460
|
-
- **markdown** — YAML frontmatter between `---` fences, free-form body. Parse: Psych `safe_load` on the
|
|
461
|
-
- **json** — entire file is a JSON document. Parse: `JSON.parse`. Serialize: `JSON.pretty_generate(content)` + trailing newline. `
|
|
462
|
-
- **yaml** — entire file is a YAML mapping. Parse: `YAML.safe_load(bytes, permitted_classes: [Date, Time], aliases: false)`; anchors/aliases rejected. Serialize: `YAML.dump(content).sub(/\A---\n/, "")`. Same `_meta` / `
|
|
463
|
-
- **text** — raw UTF-8 bytes. Parse: body is the file verbatim, `
|
|
395
|
+
- **markdown** — YAML frontmatter between `---` fences, free-form body. Parse: Psych `safe_load` on the frontmatter block; body is the remainder. Serialize: emit `---\n<yaml>\n---\n<body>`. `content` is always `nil`. `_meta` holds the parsed frontmatter hash.
|
|
396
|
+
- **json** — entire file is a JSON document. Parse: `JSON.parse`. Serialize: `JSON.pretty_generate(content)` + trailing newline. `_meta` is populated from the top-level `_meta` hash (if present, else `{}`); `body` is the raw bytes; `content` is the parsed object with `_meta` stripped.
|
|
397
|
+
- **yaml** — entire file is a YAML mapping. Parse: `YAML.safe_load(bytes, permitted_classes: [Date, Time], aliases: false)`; anchors/aliases rejected. Serialize: `YAML.dump(content).sub(/\A---\n/, "")`. Same `_meta` / `body` / `content` rules as JSON.
|
|
398
|
+
- **text** — raw UTF-8 bytes. Parse: body is the file verbatim, `_meta` is `{}`, `content` is `nil`. Serialize: write `body` bytes (with trailing newline if missing).
|
|
464
399
|
|
|
465
|
-
**Envelope shape.** Every envelope carries `format:` (always present, defaults to `markdown` for back-compat). For `json|yaml`, the envelope additionally carries `content:` (parsed object). `body` is always the raw on-disk bytes. `
|
|
400
|
+
**Envelope shape.** Every envelope carries `format:` (always present, defaults to `markdown` for back-compat). For `json|yaml`, the envelope additionally carries `content:` (parsed object). `body` is always the raw on-disk bytes. `_meta` always exists in the envelope: for `markdown` it holds the parsed YAML frontmatter; for `json|yaml` it mirrors the top-level `_meta` block (`{}` if absent); for `text` it is `{}`.
|
|
466
401
|
|
|
467
402
|
**`_meta` convention.** Derived structured entries (json, yaml) embed a `_meta` hash as the first top-level key. Builder-injected keys appear in a fixed order for etag stability:
|
|
468
403
|
|
|
@@ -529,13 +464,13 @@ Every successful CLI response (`--format=json`) is a single JSON envelope:
|
|
|
529
464
|
|
|
530
465
|
```json
|
|
531
466
|
{
|
|
532
|
-
"protocol": "textus/
|
|
467
|
+
"protocol": "textus/2",
|
|
533
468
|
"key": "working.network.org.jane",
|
|
534
469
|
"zone": "working",
|
|
535
470
|
"owner": "textus:network",
|
|
536
471
|
"path": "/absolute/path/to/.textus/zones/working/network/org/jane.md",
|
|
537
472
|
"format": "markdown",
|
|
538
|
-
"
|
|
473
|
+
"_meta": { "name": "jane", "relationship": "peer", "org": "acme" },
|
|
539
474
|
"body": "Short body in Markdown.\n",
|
|
540
475
|
"etag": "sha256:8f3c…",
|
|
541
476
|
"schema_ref": "person",
|
|
@@ -544,13 +479,13 @@ Every successful CLI response (`--format=json`) is a single JSON envelope:
|
|
|
544
479
|
```
|
|
545
480
|
|
|
546
481
|
**Field rules:**
|
|
547
|
-
- `protocol` MUST be the exact string `textus/
|
|
482
|
+
- `protocol` MUST be the exact string `textus/2`.
|
|
548
483
|
- `key` MUST be the canonical resolved key.
|
|
549
484
|
- `zone` MUST be one of the zones declared in the manifest (`canon`, `working`, `intake`, `pending`, `derived` for the default v1.0 model; legacy v0.1 manifests synthesize `fixed`, `state`, `derived` per §4).
|
|
550
485
|
- `path` MUST be an absolute filesystem path.
|
|
551
486
|
- `format` MUST be one of `markdown`, `json`, `yaml`, `text` (§5.12). Absent envelopes are treated as `markdown` for back-compat.
|
|
552
487
|
- `body` is the raw on-disk bytes as a UTF-8 string for every format.
|
|
553
|
-
- `content` is present only when `format` is `json` or `yaml`; equals the parsed object. For `json|yaml`, `
|
|
488
|
+
- `content` is present only when `format` is `json` or `yaml`; equals the parsed object. For `json|yaml`, `_meta` mirrors the top-level `_meta` block (or `{}` if absent). For `markdown`, `_meta` holds the parsed YAML frontmatter. For `text`, `_meta` is `{}`.
|
|
554
489
|
- `etag` MUST be `sha256:<hex>` of the raw file bytes, computed identically for every format.
|
|
555
490
|
- `schema_ref` MAY be `null` for entries in subtrees with `schema: null`.
|
|
556
491
|
- `uid` is the stable Textus UID (§7) if the entry carries one, else `null`. Always present in the envelope.
|
|
@@ -559,7 +494,7 @@ Errors use a distinct envelope:
|
|
|
559
494
|
|
|
560
495
|
```json
|
|
561
496
|
{
|
|
562
|
-
"protocol": "textus/
|
|
497
|
+
"protocol": "textus/2",
|
|
563
498
|
"ok": false,
|
|
564
499
|
"code": "write_forbidden",
|
|
565
500
|
"message": "zone 'canon' is not writable by role 'ai' for key 'canon.identity'",
|
|
@@ -590,29 +525,29 @@ All verbs accept `--format=json` and emit a canonical envelope (success or error
|
|
|
590
525
|
| `list [--prefix=K] [--zone=Z] [--stale]` | read | any |
|
|
591
526
|
| `where K` | read | any |
|
|
592
527
|
| `get K` | read | any |
|
|
593
|
-
| `schema K` | read | any |
|
|
528
|
+
| `schema show K` | read | any |
|
|
594
529
|
| `stale [--prefix=K] [--strict]` | read | any |
|
|
595
530
|
| `deps K` / `rdeps K` | read | any |
|
|
596
531
|
| `published` | read | any |
|
|
597
|
-
| `
|
|
598
|
-
| `
|
|
532
|
+
| `hook list` | read | any |
|
|
533
|
+
| `doctor [--check=NAME[,NAME]] [--format=json]` | read | any |
|
|
534
|
+
| `intro [--format=json]` | read | any |
|
|
535
|
+
| `put K --stdin --as=R [--fetch=NAME]` | write | per zone |
|
|
599
536
|
| `delete K --if-etag=E --as=R` | write | per zone |
|
|
600
537
|
| `refresh K --as=script` | write | per zone (typically `script`) |
|
|
601
538
|
| `build [--prefix=K] [--dry-run]` | write | `build` (default) |
|
|
602
539
|
| `accept K --as=human` | write | `human` |
|
|
603
540
|
| `init` | write | `human` |
|
|
604
|
-
| `schema
|
|
605
|
-
| `migrate
|
|
606
|
-
| `mv OLD NEW [--as=R] [--dry-run]` | write | per zone (same-zone only) |
|
|
607
|
-
| `uid K` | read | any |
|
|
608
|
-
| `
|
|
609
|
-
| `doctor [--format=json]` | read | any |
|
|
610
|
-
| `intro [--format=json]` | read | any |
|
|
541
|
+
| `schema init NAME` / `schema diff NAME` / `schema migrate NAME [--rename=OLD:NEW]` | write | `human` |
|
|
542
|
+
| `key migrate [--dry-run\|--write]` | write (with `--write`) | `human` |
|
|
543
|
+
| `key mv OLD NEW [--as=R] [--dry-run]` | write | per zone (same-zone only) |
|
|
544
|
+
| `key uid K` | read | any |
|
|
545
|
+
| `hook run NAME` | write | any |
|
|
611
546
|
|
|
612
547
|
**`put` input** (read from stdin when `--stdin` is given):
|
|
613
548
|
|
|
614
549
|
```json
|
|
615
|
-
{ "
|
|
550
|
+
{ "_meta": { "name": "jane", "relationship": "peer", "org": "acme" },
|
|
616
551
|
"body": "Short body.\n",
|
|
617
552
|
"if_etag": "sha256:8f3c…" }
|
|
618
553
|
```
|
|
@@ -637,7 +572,7 @@ All verbs accept `--format=json` and emit a canonical envelope (success or error
|
|
|
637
572
|
|
|
638
573
|
`textus init` scaffolds a fresh `.textus/` tree (manifest, zones, schemas, audit log) under the current directory with a default manifest. Customize by editing `.textus/manifest.yaml` after init.
|
|
639
574
|
|
|
640
|
-
`textus schema
|
|
575
|
+
`textus schema show K` prints the schema for entry `K`. `textus schema init NAME` writes a stub schema. `textus schema diff NAME` compares the on-disk schema against entries that claim it and prints the deltas. `textus schema migrate NAME --rename=OLD:NEW` rewrites the `_meta` key `OLD` to `NEW` across every entry that uses the named schema, in a single transactional sweep that logs each touched file.
|
|
641
576
|
|
|
642
577
|
## 10. ETag semantics
|
|
643
578
|
|
|
@@ -649,16 +584,17 @@ Every `Textus::Error` exposes `code`, `message`, and an optional `hint:`. The hi
|
|
|
649
584
|
|
|
650
585
|
## 10.2 `textus doctor`
|
|
651
586
|
|
|
652
|
-
`textus doctor` returns a health-check envelope: `{ "protocol": "textus/
|
|
587
|
+
`textus doctor` returns a health-check envelope: `{ "protocol": "textus/2", "ok": bool, "issues": [...], "summary": {error, warning, info} }`. Each issue carries `code`, `level` (`error|warning|info`), `subject`, `message`, and optionally `fix`. `ok` is true iff no error-level issues are present; warnings and info do not flip the bit. Builtin checks: `manifest_files`, `schemas`, `templates`, `hooks`, `illegal_keys`, `sentinels`, `audit_log`, `unowned_schema_fields`, `schema_violations`. Additional registered `:check` hooks (§5.10) run after the builtin set. Exit code is 0 on `ok`, 1 otherwise.
|
|
653
588
|
|
|
654
589
|
## 11. Versioning
|
|
655
590
|
|
|
656
|
-
- The wire string `textus/
|
|
657
|
-
-
|
|
658
|
-
-
|
|
591
|
+
- The current wire string is `textus/2`. It was introduced in gem v0.5, which unified the `_meta` block across all storage formats (markdown, json, yaml, text) and replaced the legacy TSV audit-log write path with NDJSON.
|
|
592
|
+
- `textus/1` was the original protocol (gem ≤ v0.4). Manifests declaring `version: textus/1` are still accepted for backward compatibility (§4).
|
|
593
|
+
- Backward-compatible additions (new fields, new error codes, new schema types) MAY be made under `textus/2`.
|
|
594
|
+
- Breaking changes (renamed/removed envelope fields, zone semantics, key grammar) require a new wire string `textus/3`.
|
|
659
595
|
- Implementations MUST reject envelopes whose `protocol` they do not recognize.
|
|
660
596
|
|
|
661
|
-
The reference Ruby gem follows semver independently.
|
|
597
|
+
The reference Ruby gem follows semver independently. The current gem version is `0.8.0`, which speaks `textus/2`.
|
|
662
598
|
|
|
663
599
|
## 12. Conformance fixtures
|
|
664
600
|
|
|
@@ -686,7 +622,7 @@ Given a derived entry with a `template` clause referencing a `.mustache` file an
|
|
|
686
622
|
Given a manifest entry with `publish_to: <path>`, a successful `textus build` for that entry leaves a plain file at `<path>` whose contents are byte-identical to the in-store artifact at `.textus/zones/<...>`, accompanied by a sentinel at `.textus/sentinels/<path>.textus-managed.json` recording `source`, `target`, `sha256`, and `mode: "copy"`. Re-running `build` is idempotent.
|
|
687
623
|
|
|
688
624
|
**Fixture H — Audit log format:**
|
|
689
|
-
Every successful write verb (`put`, `delete`, `build`, `accept`, `schema
|
|
625
|
+
Every successful write verb (`put`, `delete`, `build`, `accept`, `schema migrate`) appends exactly one line per affected key to the audit log, in the canonical format defined in §audit (timestamp, actor role, verb, key, etag-before, etag-after). No write produces zero or multiple lines per key.
|
|
690
626
|
|
|
691
627
|
**Fixture I — Pending → accept:**
|
|
692
628
|
Given a pending entry `pending.canon.identity.patch` proposing a change to `canon.identity`, `textus accept canon.identity --as=human` copies the patch body into `canon.identity`, deletes the pending entry, and appends two audit lines (one for the canon write, one for the pending delete) in that order.
|
|
@@ -695,19 +631,19 @@ Given a pending entry `pending.canon.identity.patch` proposing a change to `cano
|
|
|
695
631
|
|
|
696
632
|
- **Why not MCP?** MCP is a transport; textus is a data model. The two compose: a 50-line MCP server can wrap `textus get/put` as tools. textus exists because the *shape* of agent-readable project memory deserves a standalone spec, separate from how it's served.
|
|
697
633
|
|
|
698
|
-
- **Why doesn't textus execute generator commands itself?** textus is a dataflow oracle, not a build runner. The moment a spec includes process execution, it inherits shell-injection surface, OS-portability concerns, and signal-handling semantics — and ends up duplicating whatever build system the consumer already runs (make, rake, just, lefthook, CI). Keeping execution external means a Python or TypeScript port of `textus/
|
|
634
|
+
- **Why doesn't textus execute generator commands itself?** textus is a dataflow oracle, not a build runner. The moment a spec includes process execution, it inherits shell-injection surface, OS-portability concerns, and signal-handling semantics — and ends up duplicating whatever build system the consumer already runs (make, rake, just, lefthook, CI). Keeping execution external means a Python or TypeScript port of `textus/2` only has to parse YAML and emit JSON; it doesn't have to spawn processes safely. Build runners stay the executor; textus stays a data tool.
|
|
699
635
|
|
|
700
636
|
- **Why not plain Markdown vaults (Obsidian / Foam)?** No schema enforcement, no write-gating, no addressable wire format. Fine for human notes; underspecified for agents that must act on the contents deterministically.
|
|
701
637
|
|
|
702
638
|
- **Why not Notion / Coda?** Closed, hosted, lossy export. textus is local-first, plain-files, diffable in git.
|
|
703
639
|
|
|
704
|
-
- **Why not JSON Schema for the schemas?** Considered. Bespoke YAML chosen for v1: simpler implementation, lighter dependency footprint, matches the reference impl's house language. JSON Schema MAY be added as an alternate schema-language adapter in a future minor revision without breaking `textus/
|
|
640
|
+
- **Why not JSON Schema for the schemas?** Considered. Bespoke YAML chosen for v1: simpler implementation, lighter dependency footprint, matches the reference impl's house language. JSON Schema MAY be added as an alternate schema-language adapter in a future minor revision without breaking `textus/2`.
|
|
705
641
|
|
|
706
642
|
- **Why not a database (SQLite, kv store)?** textus's whole point is that the storage is plain files agents and humans both read. A binary store loses git-diff, grep, and editor support.
|
|
707
643
|
|
|
708
644
|
- **Why not vector embeddings?** Different problem. textus is for facts agents act on deterministically; embeddings are for fuzzy retrieval. They compose — index a textus tree into a vector store if you need both.
|
|
709
645
|
|
|
710
|
-
## 14. Open questions (
|
|
646
|
+
## 14. Open questions (v2.x scope)
|
|
711
647
|
|
|
712
648
|
- **Locking on `put`:** the reference impl uses sha256 etags. Should the spec also define a file-lock fallback for systems where read-before-write is racy?
|
|
713
649
|
- **Schema imports:** can one schema reference another (`type: $ref: person`)? Defer to v1.1.
|
|
@@ -716,21 +652,22 @@ Given a pending entry `pending.canon.identity.patch` proposing a change to `cano
|
|
|
716
652
|
|
|
717
653
|
## 15. Implementation checklist
|
|
718
654
|
|
|
719
|
-
A
|
|
655
|
+
A `textus/2` implementation MUST:
|
|
720
656
|
|
|
721
|
-
- [ ] Parse `.textus/manifest.yaml` and
|
|
657
|
+
- [ ] Parse `.textus/manifest.yaml` and accept `version: textus/2` (and `textus/1` for backward compat per §11).
|
|
722
658
|
- [ ] Resolve keys via longest-prefix match against manifest entries.
|
|
723
|
-
- [ ] Read
|
|
659
|
+
- [ ] Read `_meta` + body from `.md` files; validate against the named schema.
|
|
660
|
+
- [ ] Read `_meta` from the top-level `_meta` hash in `.json` / `.yaml` files; validate against the named schema.
|
|
724
661
|
- [ ] Compute `sha256:<hex>` etags over raw file bytes.
|
|
725
662
|
- [ ] Refuse writes whose resolved role is not in the target zone's `writable_by` list with `write_forbidden`.
|
|
726
|
-
- [ ] Return envelopes matching the shape in §8 exactly.
|
|
663
|
+
- [ ] Return envelopes matching the shape in §8 exactly (with `_meta`, not `frontmatter`).
|
|
727
664
|
- [ ] Use the error codes in §8 and the exit-code table.
|
|
728
665
|
- [ ] Implement `textus stale` per §5.1 and §9, comparing each derived entry's `generator.sources` against its `generated.at` timestamp without invoking any commands.
|
|
729
666
|
- [ ] Pass the conformance fixtures A–I in §12.
|
|
730
667
|
|
|
731
|
-
A
|
|
668
|
+
A `textus/2` implementation MAY:
|
|
732
669
|
|
|
733
|
-
- Add additional CLI verbs (e.g. `move`, vendor-specific reporters) beyond the
|
|
670
|
+
- Add additional CLI verbs (e.g. `move`, vendor-specific reporters) beyond the current set in §9.
|
|
734
671
|
- Provide alternate output formats (`--format=yaml`, `--format=table`) for human use.
|
|
735
672
|
- Support additional schema field types beyond §6, marked as `vendor:<name>` extensions.
|
|
736
673
|
|
data/docs/architecture.md
CHANGED
|
@@ -2,56 +2,110 @@
|
|
|
2
2
|
|
|
3
3
|
How the reference Ruby implementation is organized. The wire protocol itself lives in [`../SPEC.md`](../SPEC.md); this document covers *how* the gem implements that spec.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
The codebase is a flat graph of small modules under one CLI dispatcher, not a strict pyramid. The clusters below describe what each module exists for and which other modules it talks to.
|
|
6
|
+
|
|
7
|
+
## At a glance
|
|
6
8
|
|
|
7
9
|
```
|
|
8
|
-
|
|
9
|
-
│
|
|
10
|
-
|
|
11
|
-
│
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
10
|
+
exe/textus → Textus::CLI ──┬──► Store (facade — delegates to Reader/Writer/Mover)
|
|
11
|
+
│ ├──► Store::Reader (get/list/where/uid/deps/published/stale/validate_all)
|
|
12
|
+
│ ├──► Store::Writer (put/delete/accept)
|
|
13
|
+
│ ├──► Store::Mover (mv)
|
|
14
|
+
│ └──► Hooks::Dispatcher (lifecycle publish/subscribe)
|
|
15
|
+
├──► Builder (build verb — Pipeline + per-format renderers)
|
|
16
|
+
├──► Refresh (refresh verb)
|
|
17
|
+
├──► Doctor (doctor verb)
|
|
18
|
+
├──► Init (init verb)
|
|
19
|
+
├──► Intro (intro verb)
|
|
20
|
+
├──► MigrateKeys (key migrate, key mv verbs)
|
|
21
|
+
├──► Schema::Tools (schema init/diff/migrate verbs)
|
|
22
|
+
├──► Store::View (read-only projection over Store::Reader)
|
|
23
|
+
└──► Role (role gate)
|
|
19
24
|
```
|
|
20
25
|
|
|
21
|
-
|
|
26
|
+
CLI is the single entry point. It parses argv and dispatches each verb to whichever module owns that capability — there is no single mediator below CLI.
|
|
27
|
+
|
|
28
|
+
## Module clusters
|
|
29
|
+
|
|
30
|
+
### 1. Request path — core read/write verbs
|
|
31
|
+
|
|
32
|
+
`Store` is a thin facade (~110 LOC) that holds `Manifest`, `Hooks::Registry`, `Hooks::Dispatcher`, and the lazy `Store::AuditLog`, then delegates verbs to a small set of focused collaborators:
|
|
22
33
|
|
|
23
|
-
|
|
34
|
+
- **`Store::Reader`** — owns `get`, `list`, `where`, `uid`, `deps`, `rdeps`, `published`, `schema_envelope`, `stale`, `validate_all`. The only module that reads working-store entry files.
|
|
35
|
+
- **`Store::Writer`** — owns `put`, `delete`, `accept`. Handles serialization, uid minting, etag check, role gate, audit append, and event publication. The only module that writes working-store entry files.
|
|
36
|
+
- **`Store::Mover`** — owns `mv` (same-zone rename) with uid preservation and one audit row.
|
|
37
|
+
- **`Store::Validator`** / **`Store::Staleness`** — back the `validate_all` / `stale` reads. Take explicit collaborators (`reader:`, `manifest:`, `audit_log:`, `schema_for:`) instead of the full store.
|
|
24
38
|
|
|
25
|
-
|
|
39
|
+
Shared value modules and primitives consumed by Reader/Writer/Mover:
|
|
26
40
|
|
|
27
|
-
|
|
41
|
+
- **`Textus::Key::Path`** — `Key::Path.resolve(manifest, mentry)` returns the absolute leaf path for a manifest entry. Single source for zone-path construction; used by `Manifest`, `Staleness`, `Builder`, and Writer.
|
|
42
|
+
- **`Textus::Envelope`** — `Envelope.build(...)` returns the canonical envelope hash (protocol, key, zone, owner, path, format, `_meta`, body, etag, schema_ref, uid, optional content). Single source for envelope shape across `get` and `put`.
|
|
43
|
+
- **`Manifest`** — parses `.textus/manifest.yaml`; resolves a dotted key to a path via longest-prefix match. `nested: true` entries treat unmatched suffix segments as `/`-joined subdirs, with `.md` appended. Resolution is path-only; existence is the verb's concern.
|
|
44
|
+
- **`Schema`** — loads YAML schema files; validates frontmatter shape and surfaces unknown-key warnings (the §6 forward-compat rule).
|
|
45
|
+
- **`Entry`** + format adapters (`entry/markdown.rb`, `entry/text.rb`, `entry/json.rb`, `entry/yaml.rb`) — splits raw bytes on `---\n`, feeds the YAML chunk to `YAML.safe_load` (no aliases, restricted classes). The frontmatter `name:` field is enforced against the file basename in Reader/Writer (on read and on write) — mismatch raises `bad_frontmatter`.
|
|
46
|
+
- **`Etag`** — `sha256:<hex>` over raw file bytes. `put` accepts optional `if_etag:`; mismatch raises `etag_mismatch`. No locking, no temp-file-and-rename — v1 leaves stronger guarantees to v1.x.
|
|
47
|
+
- **`Role`** — agent-vs-human gate. Writer checks `Manifest::Entry#zone_writers` before doing anything else; otherwise raises `write_forbidden`.
|
|
48
|
+
- **`Store::AuditLog`** — append-only NDJSON; every successful write emits one line.
|
|
49
|
+
- **`Proposal`** — `accept` verb flow for promoting a pending entry into its target zone.
|
|
50
|
+
- **`Dependencies`** — `deps`/`rdeps`/`published` verb backing; walks manifest declarations.
|
|
28
51
|
|
|
29
|
-
|
|
52
|
+
### 2. Build / publish pipeline
|
|
53
|
+
|
|
54
|
+
Separate from the request path. Owns derived-entry materialization and byte-copy publish. `Builder` orchestrates per-entry materialization through `Builder::Pipeline`, which runs an ordered step list and dispatches the rendering step to one of four format-specific renderers. Adding a new output format is a single-file change under `lib/textus/builder/renderer/`.
|
|
55
|
+
|
|
56
|
+
```
|
|
57
|
+
Builder ──► Pipeline ──► LoadSources ──► Project ──► Render (per-format) ──► Write ──► Publisher ──► (sentinel)
|
|
58
|
+
│
|
|
59
|
+
└──► Renderer::{Markdown, Text, Json, Yaml}
|
|
60
|
+
```
|
|
30
61
|
|
|
31
|
-
|
|
62
|
+
- **`Builder`** — iterates `zone: derived` entries, hands each to `Pipeline.run`, then handles `Publisher` copy-out and fires the `:build` event. Holds no format-specific logic.
|
|
63
|
+
- **`Builder::Pipeline`** — `Pipeline.run(store:, mentry:, template_loader:)` is the orchestrator: runs the projection, merges `intro` if `inject_intro: true`, dispatches to the matching renderer, writes the bytes to the derived path.
|
|
64
|
+
- **`Builder::InjectMeta`** — builds the `_meta` block (`generated_at`, `from`, `template`, `reduce`) and threads it onto JSON/YAML content as the first key per SPEC §6 ordering.
|
|
65
|
+
- **`Builder::Renderer::{Markdown,Text,Json,Yaml}`** — one class per format, inheriting `Builder::Renderer`. Receives a template-loader lambda and `(mentry:, data:)`; returns rendered bytes. Markdown/Text always require a template; JSON/YAML optionally accept one (otherwise default-shape the projection rows).
|
|
66
|
+
- **`Projection`** — collects rows from manifest-declared source keys, applies optional reducer, sorts and positions. Pure data shaping.
|
|
67
|
+
- **`Mustache`** — minimal mustache renderer for templates in `.textus/templates/`.
|
|
68
|
+
- **`Publisher`** — byte-copy from store path to external target path. Refuses to overwrite unmanaged targets; writes a sentinel in `.textus/sentinels/` to track managed targets.
|
|
32
69
|
|
|
33
|
-
|
|
70
|
+
### 3. Extension surface
|
|
34
71
|
|
|
35
|
-
|
|
72
|
+
Declared in the manifest, loaded on demand, dispatched by `Store` and `Refresh`.
|
|
36
73
|
|
|
37
|
-
|
|
74
|
+
- **`Hooks::Registry`** — loads one `.rb` per hook from `.textus/hooks/`, registers callables under their `(event, name)`. Single source of truth via the `EVENTS` table (rpc vs pubsub, arg shape, failure semantics). For pub-sub events it also forwards registrations to the `Hooks::Dispatcher`.
|
|
75
|
+
- **`Hooks::Dispatcher`** — first-class pub/sub for lifecycle events (`:put`, `:delete`, `:refresh`, `:build`, `:accept`). Owns the 2-second per-handler timeout and the audit-on-failure middleware (raising handlers do not abort the write; they produce an `event_error` audit row). Embedded callers can `store.dispatcher.subscribe(:put, :name) { ... }` outside `.textus/hooks/`.
|
|
76
|
+
- **`Hooks::Builtin`** — ships built-in `:fetch` hooks (e.g. json, csv, ical-events, rss) available without user-supplied hooks.
|
|
77
|
+
- **`Refresh`** — `refresh` verb: looks up the `:fetch` hook for a key, invokes it, normalizes the result by declared format, writes through `Store::Writer` with an etag check.
|
|
38
78
|
|
|
39
|
-
|
|
79
|
+
### 4. Operational tooling
|
|
40
80
|
|
|
41
|
-
|
|
81
|
+
First-class CLI verbs that don't fit the read/write/build axes. Read-mostly; side modules off CLI.
|
|
42
82
|
|
|
43
|
-
|
|
83
|
+
- **`Doctor`** — `doctor` verb: orchestrator that runs 9 builtin checks under `Doctor::Check::*`. Talks to Manifest/Schema/Entry/Hooks::Registry directly.
|
|
84
|
+
- **`Doctor::Check`** — explicit base class for doctor checks. Each of the 9 builtin checks is its own file under `lib/textus/doctor/check/`.
|
|
85
|
+
- **`MigrateKeys`** — `key migrate` and `key mv` verbs; computes renames against the manifest.
|
|
86
|
+
- **`Schema::Tools`** — `schema init`, `schema diff`, `schema migrate` verbs.
|
|
87
|
+
- **`Init`** — `init` verb: scaffolds `.textus/` with the five zone directories, baseline schemas, empty audit log, starter manifest.
|
|
88
|
+
- **`Intro`** — `intro` verb: emits the human/agent-facing onboarding payload.
|
|
89
|
+
- **`Store::View`** — read-only projection over `Store::Reader` for hook code that should not mutate.
|
|
90
|
+
- **`Key::Distance`** — Levenshtein-ish suggestion for `did-you-mean` on unknown keys.
|
|
44
91
|
|
|
45
|
-
|
|
92
|
+
### 5. Primitives
|
|
46
93
|
|
|
47
|
-
|
|
94
|
+
- **`Errors`** — `Textus::Error` subclasses, each carrying a stable `code`, a `details` hash, and an `exit_code`. `CLI` catches them at the top level and emits the §8 error envelope on stdout. In `--format=json` mode, errors are **never** written to stderr — agents read stdout.
|
|
95
|
+
- **`version`** — gem semver string (independent of the wire protocol `textus/2`).
|
|
48
96
|
|
|
49
|
-
##
|
|
97
|
+
## Invariants
|
|
50
98
|
|
|
51
|
-
|
|
99
|
+
- **CLI is the only entry point.** No public API surface guarantees outside the verbs CLI exposes.
|
|
100
|
+
- **Manifest is pure.** Reads at load, no mutation.
|
|
101
|
+
- **Store::Writer is the only module that writes to working-store entry files.** Reader reads them; Mover moves them within a zone. Init, MigrateKeys, Publisher, Builder, AuditLog write to **other** parts of `.textus/` (scaffolding, sentinels, audit log, derived targets) — they do not edit existing entry files behind the Store facade's back.
|
|
102
|
+
- **`name:` frontmatter matches file basename.** Enforced on read and write.
|
|
103
|
+
- **Zone semantics live in the manifest, not in directory names.** A project may rename `state/` to anything; the manifest declares which zone each entry belongs to.
|
|
104
|
+
- **`stale` does not execute anything.** It walks `zone: derived` entries with a `generator:` block, compares `generated.at` against source mtimes, and returns offenders **plus their declared `command`**. Build runners execute. This is the §5.1 "dataflow oracle, not executor" boundary.
|
|
52
105
|
|
|
53
106
|
## What this implementation deliberately leaves out
|
|
54
107
|
|
|
55
108
|
- **No process spawning.** Even `stale` does not execute. Build runners do that.
|
|
56
109
|
- **No transport.** No HTTP server, no socket, no MCP server in this gem. Those are downstream wrappers (see [`./conventions.md`](./conventions.md)).
|
|
57
110
|
- **No indexes.** Listing walks the filesystem each time. Premature optimisation for v1.
|
|
111
|
+
- **No locking.** Etag is advisory; concurrent writers can still race. Left to v1.x (§14 open question).
|