textus 0.10.4 → 0.12.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +128 -3
- data/README.md +45 -86
- data/SPEC.md +266 -138
- data/docs/conventions.md +47 -15
- data/lib/textus/application/reads/freshness.rb +2 -2
- data/lib/textus/application/reads/get.rb +1 -1
- data/lib/textus/application/reads/policy_explain.rb +2 -2
- data/lib/textus/application/refresh/orchestrator.rb +1 -1
- data/lib/textus/application/refresh/worker.rb +5 -5
- data/lib/textus/application/writes/accept.rb +19 -1
- data/lib/textus/application/writes/build.rb +5 -5
- data/lib/textus/application/writes/delete.rb +2 -3
- data/lib/textus/application/writes/publish.rb +1 -1
- data/lib/textus/application/writes/put.rb +3 -6
- data/lib/textus/builder/pipeline.rb +1 -1
- data/lib/textus/builder/renderer/json.rb +1 -1
- data/lib/textus/builder/renderer/yaml.rb +1 -1
- data/lib/textus/cli/group/key.rb +1 -1
- data/lib/textus/cli/group/refresh.rb +21 -0
- data/lib/textus/cli/group/rule.rb +11 -0
- data/lib/textus/cli/verb/build.rb +1 -1
- data/lib/textus/cli/verb/hook_run.rb +3 -2
- data/lib/textus/cli/verb/hooks.rb +1 -1
- data/lib/textus/cli/verb/{migrate_keys.rb → key_normalize.rb} +1 -1
- data/lib/textus/cli/verb/put.rb +1 -1
- data/lib/textus/cli/verb/{policy_explain.rb → rule_explain.rb} +1 -1
- data/lib/textus/cli/verb/{policy_list.rb → rule_list.rb} +3 -3
- data/lib/textus/cli/verb.rb +3 -2
- data/lib/textus/cli.rb +6 -6
- data/lib/textus/doctor/check/handler_allowlist.rb +1 -1
- data/lib/textus/doctor/check/illegal_keys.rb +39 -16
- data/lib/textus/doctor/check/intake_registration.rb +4 -4
- data/lib/textus/doctor/check/protocol_version.rb +47 -0
- data/lib/textus/doctor/check/{policy_ambiguity.rb → rule_ambiguity.rb} +6 -6
- data/lib/textus/doctor.rb +5 -4
- data/lib/textus/domain/permission.rb +4 -4
- data/lib/textus/domain/policy/predicates/human_accept.rb +31 -0
- data/lib/textus/domain/policy/predicates/schema_valid.rb +50 -0
- data/lib/textus/domain/policy/promotion.rb +45 -0
- data/lib/textus/errors.rb +24 -5
- data/lib/textus/hooks/builtin.rb +5 -5
- data/lib/textus/hooks/dispatcher.rb +1 -1
- data/lib/textus/hooks/dsl.rb +3 -10
- data/lib/textus/hooks/loader.rb +1 -2
- data/lib/textus/hooks/registry.rb +22 -21
- data/lib/textus/infra/refresh/detached.rb +1 -1
- data/lib/textus/init.rb +25 -34
- data/lib/textus/intro.rb +9 -9
- data/lib/textus/manifest/entry.rb +66 -6
- data/lib/textus/manifest/{policies.rb → rules.rb} +12 -10
- data/lib/textus/manifest/schema.rb +49 -0
- data/lib/textus/manifest.rb +79 -39
- data/lib/textus/migrate_keys.rb +1 -1
- data/lib/textus/projection.rb +4 -4
- data/lib/textus/refresh.rb +1 -1
- data/lib/textus/store/mover.rb +91 -50
- data/lib/textus/store/staleness/generator_check.rb +88 -0
- data/lib/textus/store/staleness/intake_check.rb +46 -0
- data/lib/textus/store/staleness.rb +9 -104
- data/lib/textus/store/writer.rb +14 -12
- data/lib/textus/store.rb +1 -1
- data/lib/textus/version.rb +2 -2
- data/lib/textus.rb +1 -0
- metadata +15 -7
- data/lib/textus/cli/group/policy.rb +0 -11
data/SPEC.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
|
-
# textus/
|
|
1
|
+
# textus/3 — Specification
|
|
2
2
|
|
|
3
|
-
**Status:** Draft
|
|
4
|
-
**Protocol identifier:** `textus/
|
|
3
|
+
**Status:** Draft v3.0
|
|
4
|
+
**Protocol identifier:** `textus/3`
|
|
5
5
|
**Reference implementation:** Ruby gem `textus`
|
|
6
6
|
|
|
7
7
|
> *textus* — Latin for "the fabric a text is woven from," same root as *context* (from *con-texere*, "to weave together"). This spec defines a storage shape and wire protocol for that fabric.
|
|
@@ -10,20 +10,31 @@
|
|
|
10
10
|
|
|
11
11
|
## 1. What textus is
|
|
12
12
|
|
|
13
|
-
A storage convention and JSON wire protocol that lets humans,
|
|
13
|
+
A storage convention and JSON wire protocol that lets humans, agents, and runners read and write structured project memory **deterministically**, with addressable dotted keys, schema validation, role-based write gates, declarative compute, and copy-based publish targets.
|
|
14
14
|
|
|
15
|
-
The storage lives in a `.textus/` directory at the project root. Each entry is a Markdown file with YAML frontmatter. A manifest binds dotted keys to subtrees and declares which roles may write to each zone. Schemas (also YAML) define what frontmatter shape each entry must have. Derived entries are computed from other entries via pure projections and a vendored Mustache template engine, then optionally published to repo-relative paths as byte-for-byte file copies. The CLI surface (`textus get/put/list/where/schema/build/...` `--
|
|
15
|
+
The storage lives in a `.textus/` directory at the project root. Each entry is a Markdown file with YAML frontmatter. A manifest binds dotted keys to subtrees and declares which roles may write to each zone. Schemas (also YAML) define what frontmatter shape each entry must have. Derived entries are computed from other entries via pure projections and a vendored Mustache template engine, then optionally published to repo-relative paths as byte-for-byte file copies. The CLI surface (`textus get/put/list/where/schema/build/...` `--output=json`) returns a versioned envelope any caller can parse without knowing Markdown.
|
|
16
16
|
|
|
17
17
|
You **shape your own memory structure** inside `.textus/`. The protocol manages how it's read, written, addressed, validated, gated, computed, and published. The contents are entirely yours.
|
|
18
18
|
|
|
19
|
-
### 1.1
|
|
19
|
+
### 1.1 Vocabulary axes
|
|
20
|
+
|
|
21
|
+
textus/3 names its concepts along six axes. Reviewers who internalize these can map any part of the spec to the right category:
|
|
22
|
+
|
|
23
|
+
- **Actor** — who is interacting: `human`, `agent`, `runner`, `builder`.
|
|
24
|
+
- **Place** — where data lives: zones such as `identity`, `working`, `intake`, `review`, `output`.
|
|
25
|
+
- **Thing** — what is stored: entries, fields, keys.
|
|
26
|
+
- **Operation** — how you act on things: RPC and CLI verbs (`get`, `put`, `refresh`, `build`, …).
|
|
27
|
+
- **Event** — what gets fired after an operation: hook event names, split into RPC events (`:resolve_intake`, `:transform_rows`, `:validate`) and pub-sub events (`:entry_put`, `:build_completed`, …).
|
|
28
|
+
- **Rule** — constraints declared in the top-level `rules:` array of the manifest.
|
|
29
|
+
|
|
30
|
+
### 1.2 The five layers
|
|
20
31
|
|
|
21
32
|
textus is organized as five composable layers. Each layer has a single responsibility; later layers build on earlier ones.
|
|
22
33
|
|
|
23
34
|
| Layer | Name | Responsibility |
|
|
24
35
|
|---|---|---|
|
|
25
36
|
| L1 | **Store** | Plain-file backend: `.textus/zones/<zone>/...` with YAML frontmatter + Markdown body, addressed by dotted keys, schema-validated, etag-versioned. |
|
|
26
|
-
| L2 | **Sources** | Declared external inputs (the `
|
|
37
|
+
| L2 | **Sources** | Declared external inputs (the `intake` zone in the default scaffold; any zone writable by `runner`): URLs, files, feeds with declared parsers and TTLs. textus *describes* sources; external runners fetch and pipe results through `textus put`. |
|
|
27
38
|
| L3 | **Compute** | Pure transforms from store entries to derived entries. Projections (select/pluck/sort/limit/format) plus a vendored Mustache template subset. No shell execution. |
|
|
28
39
|
| L4 | **Publish** | Byte-for-byte file copy from derived entries to repo-relative paths declared via `publish_to:`. The in-store artifact is the consumer-shaped output; the published file is an identical copy. A sentinel under `.textus/sentinels/<target-rel-path>.textus-managed.json` records the source, sha256, and `mode: "copy"`. |
|
|
29
40
|
| L5 | **Consumers** | Anything that reads the published files or calls the CLI — editors, LLM tools, MCP servers, CI jobs, dashboards. textus is agnostic about who consumes; the envelope is the contract. |
|
|
@@ -31,10 +42,10 @@ textus is organized as five composable layers. Each layer has a single responsib
|
|
|
31
42
|
## 2. Goals and non-goals
|
|
32
43
|
|
|
33
44
|
**Goals**
|
|
34
|
-
- Stable wire format (`textus/
|
|
45
|
+
- Stable wire format (`textus/3`) any language can speak.
|
|
35
46
|
- Deterministic read/write of structured Markdown via a CLI returning JSON.
|
|
36
47
|
- Schema-validated frontmatter using YAML schemas as data.
|
|
37
|
-
- Role-based write gates (humans,
|
|
48
|
+
- Role-based write gates (humans, agents, runners, builders get different permissions per zone).
|
|
38
49
|
- Optimistic concurrency via ETags.
|
|
39
50
|
- Pure declarative compute: derived entries computed from projections + Mustache, no shell-out.
|
|
40
51
|
- Publish derived entries to well-known paths as body-only plain files.
|
|
@@ -63,10 +74,10 @@ The root is `.textus/` at the project working directory. A typical tree:
|
|
|
63
74
|
parsers/ # internal: project-local parser extensions
|
|
64
75
|
zones/ # ALL user content lives here
|
|
65
76
|
identity/ # zone: identity (human-only)
|
|
66
|
-
working/ # zone: working (human,
|
|
67
|
-
|
|
68
|
-
review/ # zone: review (
|
|
69
|
-
output/ # zone: output (
|
|
77
|
+
working/ # zone: working (human, agent, runner)
|
|
78
|
+
intake/ # zone: intake (runner — declared external inputs)
|
|
79
|
+
review/ # zone: review (agent/human — proposals awaiting accept)
|
|
80
|
+
output/ # zone: output (builder only — computed outputs)
|
|
70
81
|
```
|
|
71
82
|
|
|
72
83
|
Textus internals (`manifest.yaml`, `audit.log`, `role`, `schemas/`, `templates/`, `parsers/`) live directly under `.textus/`. **All user content lives under `.textus/zones/`.** Manifest `path:` fields are relative to `.textus/zones/` — they do **not** include the `zones/` prefix. Implementations MUST prepend `zones/` to every `path:` when resolving a key to a filesystem location.
|
|
@@ -91,19 +102,19 @@ The manifest declares: (a) which zones exist and which roles may write to each,
|
|
|
91
102
|
|
|
92
103
|
```yaml
|
|
93
104
|
# .textus/manifest.yaml
|
|
94
|
-
version: textus/
|
|
105
|
+
version: textus/3
|
|
95
106
|
|
|
96
107
|
zones:
|
|
97
108
|
- name: identity
|
|
98
|
-
|
|
109
|
+
write_policy: [human]
|
|
99
110
|
- name: working
|
|
100
|
-
|
|
101
|
-
- name:
|
|
102
|
-
|
|
111
|
+
write_policy: [human, agent, runner]
|
|
112
|
+
- name: intake
|
|
113
|
+
write_policy: [runner]
|
|
103
114
|
- name: review
|
|
104
|
-
|
|
115
|
+
write_policy: [agent, human]
|
|
105
116
|
- name: output
|
|
106
|
-
|
|
117
|
+
write_policy: [builder]
|
|
107
118
|
|
|
108
119
|
entries:
|
|
109
120
|
- key: identity.self
|
|
@@ -124,8 +135,8 @@ entries:
|
|
|
124
135
|
schema: null
|
|
125
136
|
owner: textus:build
|
|
126
137
|
|
|
127
|
-
|
|
128
|
-
- match:
|
|
138
|
+
rules:
|
|
139
|
+
- match: intake.**
|
|
129
140
|
refresh: { ttl: 6h, on_stale: warn }
|
|
130
141
|
```
|
|
131
142
|
|
|
@@ -144,6 +155,18 @@ Zone names are conventional — the manifest is the source of truth for write pe
|
|
|
144
155
|
|
|
145
156
|
For `nested: true`, the recursive glob matches the format's extension (markdown→`**/*.md`, json→`**/*.json`, yaml→`**/*.{yaml,yml}`, text→`**/*.txt`). All files under one nested entry share one format and one schema.
|
|
146
157
|
|
|
158
|
+
**Per-entry `index_filename:`.** A nested entry MAY declare `index_filename:` to surface a single fixed basename (e.g. `SKILL.md`) per directory as the row, with the row's key segments derived from the directory path. Sibling files are not enumerated. The basename's extension MUST match the entry's `format:`. This lets entries project spec-mandated filenames whose casing would otherwise be rejected by the key-segment grammar. Example:
|
|
159
|
+
|
|
160
|
+
```yaml
|
|
161
|
+
- key: skills
|
|
162
|
+
path: skills
|
|
163
|
+
zone: skills
|
|
164
|
+
nested: true
|
|
165
|
+
index_filename: SKILL.md
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
A file at `.textus/zones/skills/ask/SKILL.md` enumerates as `skills.ask`; `.textus/zones/skills/ask/references/algorithm.md` is not enumerated. Resolving `skills.ask` returns the `SKILL.md` path. `index_filename:` requires `nested: true`; the value must be a bare basename (no slashes).
|
|
169
|
+
|
|
147
170
|
**Per-leaf publishing (`publish_each:`).** A nested manifest entry MAY declare `publish_each:` to byte-copy every leaf to a templated repo-relative path. `publish_each:` and `publish_to:` are mutually exclusive on the same entry, and `publish_each:` requires `nested: true`. The template substitutes these variables (using `{name}` syntax):
|
|
148
171
|
|
|
149
172
|
| Variable | Value |
|
|
@@ -172,17 +195,17 @@ A leaf at `working.skills.writing.voice-writer` (authored at `.textus/zones/work
|
|
|
172
195
|
|
|
173
196
|
## 5. Zones and role-based write gates
|
|
174
197
|
|
|
175
|
-
Each zone declares which **roles** may write to it via `
|
|
198
|
+
Each zone declares which **roles** may write to it via `write_policy:` in the manifest. An optional `read_policy:` (default `[all]`) gates reads. Writes are gated; reads are unrestricted by default.
|
|
176
199
|
|
|
177
|
-
| Zone | `
|
|
200
|
+
| Zone | `write_policy` | Use case |
|
|
178
201
|
|---|---|---|
|
|
179
202
|
| `identity` | `[human]` | Identity, voice, immutable principles — things only a human edits. |
|
|
180
|
-
| `working` | `[human,
|
|
181
|
-
| `
|
|
182
|
-
| `review` | `[
|
|
183
|
-
| `output` | `[
|
|
203
|
+
| `working` | `[human, agent, runner]` | Active project state: notes, decisions, network — what humans and agents update day-to-day. |
|
|
204
|
+
| `intake` | `[runner]` | Declared external inputs (calendar, feeds, scraped pages). Refreshed by external runner scripts; never by humans or agents directly. |
|
|
205
|
+
| `review` | `[agent, human]` | Agent-generated proposals awaiting human review via `textus accept`. Lets agents stage changes without touching `working`. |
|
|
206
|
+
| `output` | `[builder]` | Computed outputs (catalogs, indexes, published context). Written only by the build runner via `textus build`. |
|
|
184
207
|
|
|
185
|
-
A write is gated by the caller's **role**, supplied via `--as=<role>`. If the role is not in the target zone's `
|
|
208
|
+
A write is gated by the caller's **role**, supplied via `--as=<role>`. If the role is not in the target zone's `write_policy` list, the write returns `write_forbidden`.
|
|
186
209
|
|
|
187
210
|
### 5.1 Role resolution
|
|
188
211
|
|
|
@@ -193,23 +216,36 @@ The effective role for any CLI invocation is resolved in this order; the first m
|
|
|
193
216
|
3. `.textus/role` file (one line, role name) at the project root.
|
|
194
217
|
4. Default: `human`.
|
|
195
218
|
|
|
196
|
-
|
|
219
|
+
**Canonical actors (textus/3):**
|
|
220
|
+
|
|
221
|
+
| Actor | Meaning |
|
|
222
|
+
|---|---|
|
|
223
|
+
| `human` | Interactive user at a terminal. |
|
|
224
|
+
| `agent` | Long-running AI or LLM process. |
|
|
225
|
+
| `runner` | Scheduled or one-shot automation script. |
|
|
226
|
+
| `builder` | Build/derive output (catalogs, indexes). |
|
|
227
|
+
|
|
228
|
+
Unknown role values are rejected with `invalid_role`.
|
|
197
229
|
|
|
198
230
|
Every successful write records the resolved role and a wall-clock timestamp in `.textus/audit.log`, so reviewers can later distinguish a human edit from an agent edit even though both live in the same file.
|
|
199
231
|
|
|
200
232
|
### 5.2 Compute layer (derived entries)
|
|
201
233
|
|
|
202
|
-
Derived entries live in a zone whose `
|
|
234
|
+
Derived entries live in a zone whose `write_policy:` list includes `builder` — `output` in the default scaffold. They are not authored by hand; their body is produced by projecting over other entries. A derived entry declares a `compute:` block with a `kind:` discriminator.
|
|
235
|
+
|
|
236
|
+
#### 5.2.1 Projection compute (`kind: projection`)
|
|
203
237
|
|
|
204
238
|
```yaml
|
|
205
239
|
- key: output.catalogs.people
|
|
206
240
|
zone: output
|
|
207
|
-
|
|
241
|
+
compute:
|
|
242
|
+
kind: projection
|
|
208
243
|
select: working.network.org # prefix OR [list of prefixes]
|
|
209
244
|
pluck: [name, relationship, org]
|
|
210
245
|
sort_by: name # optional
|
|
211
246
|
limit: 1000 # default 1000, max 1000
|
|
212
247
|
format: yaml-list-in-md # one of: list, hash, yaml-list-in-md, json, markdown-table
|
|
248
|
+
transform: rank_by_recency # optional — names a :transform_rows hook
|
|
213
249
|
template: people.mustache # optional; if absent, format determines body
|
|
214
250
|
```
|
|
215
251
|
|
|
@@ -217,6 +253,8 @@ Derived entries live in a zone whose `writable_by:` list includes `build` — `o
|
|
|
217
253
|
|
|
218
254
|
`format` controls the body serialization when no template is supplied. Permitted values: `list`, `hash`, `yaml-list-in-md`, `json`, `markdown-table`.
|
|
219
255
|
|
|
256
|
+
`transform:` (optional) names a registered `:transform_rows` hook (see §5.10). The hook receives the projected rows array and may reorder, filter, or augment before serialization.
|
|
257
|
+
|
|
220
258
|
If `template` is given, it names a Mustache template under `.textus/templates/`. textus implements a deliberately restricted Mustache subset:
|
|
221
259
|
|
|
222
260
|
- `{{var}}` — variable interpolation.
|
|
@@ -226,6 +264,45 @@ If `template` is given, it names a Mustache template under `.textus/templates/`.
|
|
|
226
264
|
|
|
227
265
|
No partials. No lambdas. No HTML escaping (output is raw text, intended for Markdown). Template recursion depth is bounded at 8; exceeding the limit is an error.
|
|
228
266
|
|
|
267
|
+
#### 5.2.2 External compute (`kind: external`)
|
|
268
|
+
|
|
269
|
+
A derived entry that is produced by a build tool *outside* textus — `rake`, `just`, a shell script, anything — declares `compute: { kind: external, ... }`. textus does **not** execute the command (consistent with §2); the external runner is responsible for writing the file. textus records `sources:` so `textus freshness` can compare source mtimes against the derived file's `_meta.generated.at` and report staleness.
|
|
270
|
+
|
|
271
|
+
```yaml
|
|
272
|
+
- key: output.catalogs.skills
|
|
273
|
+
path: output/catalogs/skills.md
|
|
274
|
+
zone: output
|
|
275
|
+
owner: build:catalog-skills
|
|
276
|
+
compute:
|
|
277
|
+
kind: external
|
|
278
|
+
command: "rake catalog:skills" # informational; the runner invokes it
|
|
279
|
+
sources: # dotted keys OR repo-relative paths
|
|
280
|
+
- working.projects
|
|
281
|
+
- working.network
|
|
282
|
+
```
|
|
283
|
+
|
|
284
|
+
**`sources:`** is a list. Each element is either a dotted key prefix (matched against manifest entries) or a filesystem path (relative to the repo root, or absolute). For each key prefix, every matching entry's file mtime is checked. For each path, file or directory mtime is checked.
|
|
285
|
+
|
|
286
|
+
**`command:`** is recorded in the staleness row's `generator` field but never executed. It exists so `textus freshness` output can carry a hint about how to refresh.
|
|
287
|
+
|
|
288
|
+
**Freshness contract.** An entry with `compute: { kind: external }` is reported by `textus freshness` as `stale` when:
|
|
289
|
+
- The derived file does not exist, OR
|
|
290
|
+
- `_meta.generated.at` is missing or unparseable, OR
|
|
291
|
+
- Any `sources:` element has been modified after `_meta.generated.at`.
|
|
292
|
+
|
|
293
|
+
**Frontmatter contract.** The external runner is responsible for writing the `generated:` frontmatter block when it produces the file:
|
|
294
|
+
|
|
295
|
+
```yaml
|
|
296
|
+
generated:
|
|
297
|
+
by: "rake catalog:skills"
|
|
298
|
+
at: "2026-05-25T12:00:00Z"
|
|
299
|
+
from: [working.projects, working.network]
|
|
300
|
+
```
|
|
301
|
+
|
|
302
|
+
`generated.from` SHOULD match `compute.sources` — they're the same list, recorded twice so a diff proves what was actually consumed.
|
|
303
|
+
|
|
304
|
+
`kind: external` and `kind: projection` are alternatives — exactly one per entry. Templates are not required for `kind: external`: the runner produces the bytes directly.
|
|
305
|
+
|
|
229
306
|
### 5.3 Publish layer (`publish_to:`)
|
|
230
307
|
|
|
231
308
|
A derived entry MAY declare `publish_to:` in its frontmatter, listing one or more destination paths relative to the project root:
|
|
@@ -244,25 +321,25 @@ A sentinel is written for each published file at `<store_root>/sentinels/<target
|
|
|
244
321
|
|
|
245
322
|
### 5.4 Intake (declared, refreshed via registered intake handler)
|
|
246
323
|
|
|
247
|
-
Intake entries declare an external source by naming an **intake handler** — a registered, named function that pulls data into the entry. textus itself still makes no implicit network calls: an intake handler only runs when explicitly invoked by `textus refresh KEY --as=
|
|
324
|
+
Intake entries declare an external source by naming an **intake handler** — a registered, named function that pulls data into the entry. textus itself still makes no implicit network calls: an intake handler only runs when explicitly invoked by `textus refresh KEY --as=runner` (or by `textus refresh stale`). The declaration is data only:
|
|
248
325
|
|
|
249
326
|
```yaml
|
|
250
|
-
- key:
|
|
251
|
-
zone:
|
|
327
|
+
- key: intake.calendar.events
|
|
328
|
+
zone: intake
|
|
252
329
|
intake:
|
|
253
330
|
handler: ical-events
|
|
254
331
|
config:
|
|
255
332
|
url: "https://calendar.google.com/.../basic.ics"
|
|
256
333
|
|
|
257
|
-
|
|
258
|
-
- match:
|
|
334
|
+
rules:
|
|
335
|
+
- match: intake.calendar.**
|
|
259
336
|
refresh:
|
|
260
337
|
ttl: 6h
|
|
261
338
|
on_stale: warn # warn | sync | timed_sync (default: warn)
|
|
262
339
|
sync_budget_ms: 500 # only used when on_stale: timed_sync (default: 500)
|
|
263
340
|
```
|
|
264
341
|
|
|
265
|
-
`handler` names a registered `:
|
|
342
|
+
`handler` names a registered `:resolve_intake` hook (see §5.10 for the hook contract); `config` is an opaque hash handed to the handler. The freshness budget (`ttl`, `on_stale`, `sync_budget_ms`) lives in a top-level **`rules:`** block matched by key glob (§5.11).
|
|
266
343
|
|
|
267
344
|
#### `on_stale:` semantics
|
|
268
345
|
|
|
@@ -272,7 +349,7 @@ policies:
|
|
|
272
349
|
|---|---|
|
|
273
350
|
| `warn` (default) | Return the entry immediately with `stale: true`, `stale_reason:` populated, and `refreshing: false`. No blocking. |
|
|
274
351
|
| `sync` | Block the `get` call, run the intake handler in-process, write the refreshed result, then return the fresh envelope. The caller waits. |
|
|
275
|
-
| `timed_sync` | Like `sync`, but with a `sync_budget_ms` deadline (default 500 ms). If the handler finishes within the budget the fresh envelope is returned. If it does not finish in time, return the stale envelope (with `stale: true`, `refreshing: true`) and let the refresh complete in the background. Fires `:
|
|
352
|
+
| `timed_sync` | Like `sync`, but with a `sync_budget_ms` deadline (default 500 ms). If the handler finishes within the budget the fresh envelope is returned. If it does not finish in time, return the stale envelope (with `stale: true`, `refreshing: true`) and let the refresh complete in the background. Fires `:refresh_backgrounded` when the deadline is exceeded. |
|
|
276
353
|
|
|
277
354
|
> **Note:** `list`/`where` paths do **not** annotate freshness — only `get` does.
|
|
278
355
|
|
|
@@ -286,14 +363,14 @@ In intake mode the handler MUST return one of three shapes, all normalized by th
|
|
|
286
363
|
|
|
287
364
|
**Refresh paths.** Two are supported:
|
|
288
365
|
|
|
289
|
-
1. **In-process** — `textus refresh KEY --as=
|
|
290
|
-
2. **External runner** — a cron job or agent harness reads `textus list --zone=
|
|
366
|
+
1. **In-process** — `textus refresh KEY --as=runner` resolves the entry's `intake.handler`, invokes the registered `:resolve_intake` hook with `(config:, store:, args: {})`, and writes the result under role `runner`.
|
|
367
|
+
2. **External runner** — a cron job or agent harness reads `textus list --zone=intake --stale --output=json`, fetches the source out of band, and pipes bytes back through `textus put KEY --as=runner --stdin`. The CLI verb `textus refresh stale [--prefix=K] [--zone=Z]` drives this loop in one shot.
|
|
291
368
|
|
|
292
|
-
Both paths share the same role gate, audit-log entry, and `:
|
|
369
|
+
Both paths share the same role gate, audit-log entry, and `:entry_refreshed` event. User-supplied hooks live in `.textus/hooks/**/*.rb` and auto-load at `Store#initialize` — see §5.10 for the full hook contract.
|
|
293
370
|
|
|
294
371
|
### 5.5 Pending / accept workflow
|
|
295
372
|
|
|
296
|
-
Proposal entries are full patches authored into a zone whose `
|
|
373
|
+
Proposal entries are full patches authored into a zone whose `write_policy:` list includes `agent` — `review` in the default scaffold — typically by agents or runners. The entry's frontmatter describes the patch it proposes against another zone:
|
|
297
374
|
|
|
298
375
|
```yaml
|
|
299
376
|
---
|
|
@@ -310,7 +387,7 @@ Proposed body content.
|
|
|
310
387
|
|
|
311
388
|
`proposal.target_key` names the entry the patch would create or modify, and `proposal.action` is `put` or `delete`. The remaining frontmatter and body are the proposed new content.
|
|
312
389
|
|
|
313
|
-
`textus accept <proposal-key>` is **human-only**: the resolved role must be `human`. It copies the patch into the target zone, records provenance (originating proposal key, original role, original timestamp) in the audit log, and removes the proposal entry. Agents and
|
|
390
|
+
`textus accept <proposal-key>` is **human-only**: the resolved role must be `human`. It copies the patch into the target zone, records provenance (originating proposal key, original role, original timestamp) in the audit log, and removes the proposal entry. Agents and runners can propose but cannot accept.
|
|
314
391
|
|
|
315
392
|
### 5.6 Audit log
|
|
316
393
|
|
|
@@ -345,11 +422,11 @@ Schemas may declare per-field ownership and version history. The `fields:` and `
|
|
|
345
422
|
```yaml
|
|
346
423
|
fields:
|
|
347
424
|
full_name: { type: string, maintained_by: human }
|
|
348
|
-
embedding: { type: array, maintained_by:
|
|
349
|
-
updated_at: { type: time, maintained_by:
|
|
425
|
+
embedding: { type: array, maintained_by: agent }
|
|
426
|
+
updated_at: { type: time, maintained_by: runner }
|
|
350
427
|
```
|
|
351
428
|
|
|
352
|
-
`maintained_by` values are free-form strings. The recognized set is `human |
|
|
429
|
+
`maintained_by` values are free-form strings. The recognized set is `human | agent | runner | builder | derived`. Unrecognized values do not affect role-authority validation; they pass through unchanged.
|
|
353
430
|
|
|
354
431
|
**`evolution:` block** — top-level, declares the schema's history and migration intent:
|
|
355
432
|
|
|
@@ -365,61 +442,62 @@ evolution:
|
|
|
365
442
|
|
|
366
443
|
**Defaults:** when `fields:` and `evolution:` are absent, `schema.maintained_by(field)` returns `nil` for every field and `schema.evolution` returns `{}`.
|
|
367
444
|
|
|
368
|
-
**Override rule:** the role `human` is permitted to write any `maintained_by` field, regardless of declared owner. This preserves human authority over
|
|
445
|
+
**Override rule:** the role `human` is permitted to write any `maintained_by` field, regardless of declared owner. This preserves human authority over agent/runner-managed data — humans curating canon over agent-written embeddings is a feature, not a bug. All other role mismatches are reported by `doctor --check=schema_violations` with code `role_authority`, including fields `key`, `field`, `expected`, and `last_writer`.
|
|
369
446
|
|
|
370
|
-
### 5.9
|
|
447
|
+
### 5.9 Row transforms
|
|
371
448
|
|
|
372
|
-
|
|
449
|
+
Row transforms are RPC hooks on the `:transform_rows` event. See §5.10.
|
|
373
450
|
|
|
374
451
|
### 5.10 Hooks
|
|
375
452
|
|
|
376
|
-
textus has a single hook verb: `Textus.
|
|
377
|
-
|
|
378
|
-
The subdirectory layout under `hooks/` is organizational only; the registered event and name come from the DSL call, not the file path. Files are loaded in alphabetical order by full path.
|
|
453
|
+
textus has a single hook registration verb: `Textus.on(event, name, **opts) { ... }`. The EVENTS table below defines every extension point. Files in `.textus/hooks/**/*.rb` are `load`ed at `Store#initialize` in alphabetical order by full path.
|
|
379
454
|
|
|
380
|
-
|
|
455
|
+
The subdirectory layout under `hooks/` is organizational only; the registered event and name come from the DSL call, not the file path.
|
|
381
456
|
|
|
382
|
-
|
|
457
|
+
#### Registration DSL
|
|
383
458
|
|
|
384
459
|
```ruby
|
|
385
|
-
|
|
386
|
-
Textus.
|
|
387
|
-
Textus.
|
|
388
|
-
Textus.
|
|
389
|
-
Textus.
|
|
460
|
+
# Canonical form — works for every event:
|
|
461
|
+
Textus.on(:resolve_intake, :my_source) { |config:, args:, **| … }
|
|
462
|
+
Textus.on(:transform_rows, :rank_by_recency) { |rows:, **| … }
|
|
463
|
+
Textus.on(:validate, :storage_writable) { |store:| … }
|
|
464
|
+
Textus.on(:entry_put, :audit, keys: ["working.*"]) { |key:, envelope:, **| … }
|
|
465
|
+
Textus.on(:file_published, :git_add, keys: ["derived.*"]) { |target:, **| `git add #{target.shellescape}` }
|
|
390
466
|
```
|
|
391
467
|
|
|
392
|
-
|
|
393
|
-
|
|
394
|
-
|
|
395
|
-
|
|
396
|
-
|
|
|
397
|
-
|
|
398
|
-
|
|
|
399
|
-
|
|
|
400
|
-
|
|
|
401
|
-
|
|
|
402
|
-
|
|
|
403
|
-
|
|
|
404
|
-
|
|
|
405
|
-
|
|
|
406
|
-
|
|
|
407
|
-
|
|
|
408
|
-
|
|
|
409
|
-
|
|
|
410
|
-
|
|
|
468
|
+
`Textus.on` is the sole entry point; there is no separate `Textus.hook` primitive.
|
|
469
|
+
|
|
470
|
+
#### Event table
|
|
471
|
+
|
|
472
|
+
| Event | Mode | Args | Return | Failure |
|
|
473
|
+
|-------------------------|---------|-----------------------------------------------------------|-----------------------|---------------|
|
|
474
|
+
| `:resolve_intake` | rpc | store:, config:, args: | {_meta:, body:} | aborts op |
|
|
475
|
+
| `:transform_rows` | rpc | store:, rows:, config: | rows array | aborts op |
|
|
476
|
+
| `:validate` | rpc | store: | issues array | aborts doctor |
|
|
477
|
+
| `:entry_put` | pubsub | store:, key:, envelope: | (discarded) | logged |
|
|
478
|
+
| `:entry_deleted` | pubsub | store:, key: | (discarded) | logged |
|
|
479
|
+
| `:entry_refreshed` | pubsub | store:, key:, envelope:, change: | (discarded) | logged |
|
|
480
|
+
| `:build_completed` | pubsub | store:, key:, envelope:, sources: | (discarded) | logged |
|
|
481
|
+
| `:proposal_accepted` | pubsub | store:, key:, target_key: | (discarded) | logged |
|
|
482
|
+
| `:file_published` | pubsub | store:, key:, envelope:, source:, target: | (discarded) | logged |
|
|
483
|
+
| `:entry_renamed` | pubsub | store:, key:, from_key:, to_key:, envelope: | (discarded) | logged |
|
|
484
|
+
| `:proposal_rejected` | pubsub | store:, key:, target_key: | (discarded) | logged |
|
|
485
|
+
| `:store_loaded` | pubsub | store: | (discarded) | logged |
|
|
486
|
+
| `:refresh_started` | pubsub | store:, key:, mode: | (discarded) | logged |
|
|
487
|
+
| `:refresh_failed` | pubsub | store:, key:, error_class:, error_message: | (discarded) | logged |
|
|
488
|
+
| `:refresh_backgrounded` | pubsub | store:, key:, started_at:, budget_ms: | (discarded) | logged |
|
|
411
489
|
|
|
412
490
|
The three `:refresh_*` lifecycle events report the progress and failures of background (timed_sync) refreshes.
|
|
413
491
|
|
|
414
|
-
**`:
|
|
492
|
+
**`:refresh_started`** fires immediately before an intake handler is invoked. `mode:` is one of `"sync"` or `"timed_sync"`.
|
|
415
493
|
|
|
416
494
|
**`:refresh_failed`** fires when an intake handler raises. `error_class:` is the exception class name string; `error_message:` is `e.message`.
|
|
417
495
|
|
|
418
|
-
**`:
|
|
496
|
+
**`:refresh_backgrounded`** fires when a `timed_sync` refresh exceeds its budget and is handed off to a background thread. `started_at:` is an ISO-8601 UTC string; `budget_ms:` is the configured deadline as an integer.
|
|
419
497
|
|
|
420
|
-
**Signature invariant** — every hook receives `store:` as its first keyword argument. Event-specific kwargs follow in stable left-to-right order. The primary entity is always `key:` (for `:
|
|
498
|
+
**Signature invariant** — every hook receives `store:` as its first keyword argument. Event-specific kwargs follow in stable left-to-right order. The primary entity is always `key:` (for `:proposal_accepted`, `key:` is the pending key being accepted and `target_key:` is the destination). For `:entry_renamed`, `key:` is present and equals `to_key:` — it is the entry's post-move home, present so `keys:` glob filters route correctly; `from_key:` is the prior key. For `:proposal_rejected`, `key:` is the pending key being rejected. For `:store_loaded`, no key — the event observes store readiness, not an entry.
|
|
421
499
|
|
|
422
|
-
**RPC mode** — exactly one handler per (event, name). The manifest references the handler by name (`intake.handler: NAME`, `
|
|
500
|
+
**RPC mode** — exactly one handler per (event, name). The manifest references the handler by name (`intake.handler: NAME`, `compute.transform: NAME`). Failure or timeout aborts the calling operation.
|
|
423
501
|
|
|
424
502
|
**Pub-sub mode** — zero or more handlers per event. All matching handlers fire. The `keys:` option restricts a handler to keys matching one of the given globs (`File.fnmatch?` rules). Absence of `keys:` fires on every event of that type. Handler failures and 2s timeouts are logged to `audit.log` as `event_error` rows; they NEVER abort the triggering operation.
|
|
425
503
|
|
|
@@ -427,21 +505,22 @@ The `store:` argument is always a read-only store proxy. Write attempts raise `U
|
|
|
427
505
|
|
|
428
506
|
Each handler runs under `Timeout.timeout(2)`.
|
|
429
507
|
|
|
430
|
-
### 5.11
|
|
508
|
+
### 5.11 Rules
|
|
431
509
|
|
|
432
|
-
A manifest MAY declare a top-level `
|
|
510
|
+
A manifest MAY declare a top-level `rules:` block — a list of rule blocks matched against entry keys by glob. Each block carries one or more slots:
|
|
433
511
|
|
|
434
512
|
```yaml
|
|
435
|
-
|
|
436
|
-
- match:
|
|
513
|
+
rules:
|
|
514
|
+
- match: intake.**
|
|
437
515
|
refresh: { ttl: 6h, on_stale: warn }
|
|
438
516
|
|
|
439
|
-
- match:
|
|
517
|
+
- match: intake.calendar.**
|
|
440
518
|
refresh: { ttl: 30m, on_stale: timed_sync, sync_budget_ms: 800 }
|
|
441
|
-
|
|
519
|
+
intake_handler_allowlist: [ical-events]
|
|
442
520
|
|
|
443
521
|
- match: review.**
|
|
444
|
-
|
|
522
|
+
promotion:
|
|
523
|
+
requires: [schema_valid, human_accept]
|
|
445
524
|
```
|
|
446
525
|
|
|
447
526
|
**Slots (all optional within a block):**
|
|
@@ -449,15 +528,15 @@ policies:
|
|
|
449
528
|
| Slot | Type | Meaning |
|
|
450
529
|
|---|---|---|
|
|
451
530
|
| `refresh` | `{ ttl, on_stale, sync_budget_ms }` | Freshness budget for intake entries. `on_stale` is `warn` (default), `sync`, or `timed_sync`. |
|
|
452
|
-
| `
|
|
453
|
-
| `
|
|
531
|
+
| `intake_handler_allowlist` | list of strings | Constrains which `intake.handler:` names may be used by entries matched by this block. Enforced by `textus doctor`. |
|
|
532
|
+
| `promotion` | `{ requires: [...] }` | Predicates a `review` entry must satisfy before `textus accept` will promote it. Built-in predicates: `schema_valid` (entry passes schema validation) and `human_accept` (the accepting role must be `human`). Additional predicates may be registered via `:validate` hooks. Enforced — `textus accept` refuses if any predicate fails. |
|
|
454
533
|
| `retention` | (reserved) | Slot reserved for future retention policy (cap by age / count). Implementations parse it but otherwise ignore. |
|
|
455
534
|
|
|
456
535
|
**Match grammar.** `match:` is a single glob using `*` (single segment) and `**` (any depth). A literal segment ranks more specifically than `*`; `*` ranks more specifically than `**`.
|
|
457
536
|
|
|
458
|
-
**Resolution.** For each key textus computes a `
|
|
537
|
+
**Resolution.** For each key textus computes a `RuleSet { refresh, intake_handler_allowlist, promotion, retention }` by walking every block whose `match` matches the key, ranked by specificity. **Per slot, the most specific block wins.** Two blocks of equal specificity that match the same key and fill the same slot is a manifest error reported by `textus doctor` (`rule_ambiguity`).
|
|
459
538
|
|
|
460
|
-
**Read surface.** `textus
|
|
539
|
+
**Read surface.** `textus rule list` dumps every block. `textus rule explain KEY` shows the resolved `RuleSet` for one key plus which block won each slot.
|
|
461
540
|
|
|
462
541
|
### 5.12 Storage formats
|
|
463
542
|
|
|
@@ -473,7 +552,7 @@ An entry's `format:` selects a storage strategy. All strategies expose the same
|
|
|
473
552
|
**`_meta` convention.** Derived structured entries (json, yaml) embed a `_meta` hash as the first top-level key. Builder-injected keys appear in a fixed order for etag stability:
|
|
474
553
|
|
|
475
554
|
```
|
|
476
|
-
generated_at, from, template,
|
|
555
|
+
generated_at, from, template, transform
|
|
477
556
|
```
|
|
478
557
|
|
|
479
558
|
Keys with `nil` values are omitted. User-shaped content (or the reducer's hash) follows `_meta`. The etag (§10) is the sha256 of the on-disk bytes regardless of format; key ordering MUST therefore be deterministic, which Ruby's `Hash` and `JSON.generate` / `YAML.dump` honor via insertion order.
|
|
@@ -531,11 +610,11 @@ Entries in `zone: derived` SHOULD additionally carry the `generated:` block defi
|
|
|
531
610
|
|
|
532
611
|
## 8. Envelope (the wire format)
|
|
533
612
|
|
|
534
|
-
Every successful CLI response (`--
|
|
613
|
+
Every successful CLI response (`--output=json`) is a single JSON envelope:
|
|
535
614
|
|
|
536
615
|
```json
|
|
537
616
|
{
|
|
538
|
-
"protocol": "textus/
|
|
617
|
+
"protocol": "textus/3",
|
|
539
618
|
"key": "working.network.org.jane",
|
|
540
619
|
"zone": "working",
|
|
541
620
|
"owner": "textus:network",
|
|
@@ -553,9 +632,9 @@ Every successful CLI response (`--format=json`) is a single JSON envelope:
|
|
|
553
632
|
```
|
|
554
633
|
|
|
555
634
|
**Field rules:**
|
|
556
|
-
- `protocol` MUST be the exact string `textus/
|
|
635
|
+
- `protocol` MUST be the exact string `textus/3`.
|
|
557
636
|
- `key` MUST be the canonical resolved key.
|
|
558
|
-
- `zone` MUST be one of the zones declared in the manifest (`identity`, `working`, `
|
|
637
|
+
- `zone` MUST be one of the zones declared in the manifest (`identity`, `working`, `intake`, `review`, `output` in the default scaffold).
|
|
559
638
|
- `path` MUST be an absolute filesystem path.
|
|
560
639
|
- `format` MUST be one of `markdown`, `json`, `yaml`, `text` (§5.12). Absent envelopes are treated as `markdown` for back-compat.
|
|
561
640
|
- `body` is the raw on-disk bytes as a UTF-8 string for every format.
|
|
@@ -563,7 +642,7 @@ Every successful CLI response (`--format=json`) is a single JSON envelope:
|
|
|
563
642
|
- `etag` MUST be `sha256:<hex>` of the raw file bytes, computed identically for every format.
|
|
564
643
|
- `schema_ref` MAY be `null` for entries in subtrees with `schema: null`.
|
|
565
644
|
- `uid` is the stable Textus UID (§7) if the entry carries one, else `null`. Always present in the envelope.
|
|
566
|
-
- `stale` is `true` when the entry's TTL has elapsed and the data has not yet been refreshed; `false` otherwise. Only populated for entries matched by a `refresh:`
|
|
645
|
+
- `stale` is `true` when the entry's TTL has elapsed and the data has not yet been refreshed; `false` otherwise. Only populated for entries matched by a `refresh:` rule slot (typically `intake` zone); always `false` elsewhere.
|
|
567
646
|
- `stale_reason` is a short human-readable string describing why the entry is stale (e.g. `"ttl_exceeded"`, `"never_refreshed"`), or `null` when `stale` is `false`.
|
|
568
647
|
- `refreshing` is `true` when a `timed_sync` background refresh is in flight for this entry; `false` otherwise. Callers observing `stale: true, refreshing: true` SHOULD retry after a short delay.
|
|
569
648
|
|
|
@@ -573,11 +652,11 @@ Errors use a distinct envelope:
|
|
|
573
652
|
|
|
574
653
|
```json
|
|
575
654
|
{
|
|
576
|
-
"protocol": "textus/
|
|
655
|
+
"protocol": "textus/3",
|
|
577
656
|
"ok": false,
|
|
578
657
|
"code": "write_forbidden",
|
|
579
|
-
"message": "zone 'identity' is not writable by role '
|
|
580
|
-
"details": { "key": "identity.self", "zone": "identity", "role": "
|
|
658
|
+
"message": "zone 'identity' is not writable by role 'agent' for key 'identity.self'",
|
|
659
|
+
"details": { "key": "identity.self", "zone": "identity", "role": "agent" }
|
|
581
660
|
}
|
|
582
661
|
```
|
|
583
662
|
|
|
@@ -588,7 +667,7 @@ Errors use a distinct envelope:
|
|
|
588
667
|
| `unknown_key` | Key does not resolve | 1 |
|
|
589
668
|
| `bad_frontmatter` | YAML parse failed or `name:` mismatch | 1 |
|
|
590
669
|
| `schema_violation` | Required field missing or wrong type | 1 |
|
|
591
|
-
| `write_forbidden` | Resolved role is not in the zone's `
|
|
670
|
+
| `write_forbidden` | Resolved role is not in the zone's `write_policy` | 1 |
|
|
592
671
|
| `etag_mismatch` | Concurrent write detected | 1 |
|
|
593
672
|
| `io_error` | Filesystem failure | 64 |
|
|
594
673
|
| `usage` | CLI argument error | 2 |
|
|
@@ -597,7 +676,7 @@ Errors use a distinct envelope:
|
|
|
597
676
|
|
|
598
677
|
The reference binary is `textus`. Conforming implementations MAY use any binary name; the protocol is in the JSON.
|
|
599
678
|
|
|
600
|
-
All verbs accept `--
|
|
679
|
+
All verbs accept `--output=json` and emit a canonical envelope (success or error). Write verbs require `--as=<role>`; the role must satisfy the target zone's write gate (§5). The per-entry `format:` field in the manifest is unchanged — `--output` controls only the CLI envelope rendering.
|
|
601
680
|
|
|
602
681
|
| Verb | Reads / writes | Role required |
|
|
603
682
|
|---|---|---|
|
|
@@ -608,24 +687,24 @@ All verbs accept `--format=json` and emit a canonical envelope (success or error
|
|
|
608
687
|
| `freshness [--prefix=K] [--zone=Z]` | read | any |
|
|
609
688
|
| `audit [--key=K] [--zone=Z] [--role=R] [--verb=V] [--since=X] [--correlation-id=ID] [--limit=N]` | read | any |
|
|
610
689
|
| `blame KEY` | read | any |
|
|
611
|
-
| `
|
|
690
|
+
| `rule list` / `rule explain KEY` | read | any |
|
|
612
691
|
| `deps K` / `rdeps K` | read | any |
|
|
613
692
|
| `published` | read | any |
|
|
614
693
|
| `hook list` | read | any |
|
|
615
|
-
| `
|
|
616
|
-
| `
|
|
694
|
+
| `hook run NAME` | write | any |
|
|
695
|
+
| `doctor [--check=NAME[,NAME]] [--output=json]` | read | any |
|
|
696
|
+
| `intro [--output=json]` | read | any |
|
|
617
697
|
| `put K --stdin --as=R [--fetch=NAME]` | write | per zone |
|
|
618
698
|
| `delete K --if-etag=E --as=R` | write | per zone |
|
|
619
|
-
| `refresh
|
|
620
|
-
| `refresh
|
|
621
|
-
| `build [--prefix=K] [--dry-run]` | write | `
|
|
699
|
+
| `refresh KEY --as=runner` | write | per zone (typically `runner`) |
|
|
700
|
+
| `refresh stale [--prefix=K] [--zone=Z] [--as=runner]` | write | per zone (typically `runner`) |
|
|
701
|
+
| `build [--prefix=K] [--dry-run]` | write | `builder` (default) |
|
|
622
702
|
| `accept K --as=human` | write | `human` |
|
|
623
703
|
| `init` | write | `human` |
|
|
624
|
-
| `schema init
|
|
625
|
-
| `key migrate [--dry-run\|--write]` | write (with `--write`) | `human` |
|
|
704
|
+
| `schema {show,init,diff,migrate}` | read/write | `human` for writes |
|
|
626
705
|
| `key mv OLD NEW [--as=R] [--dry-run]` | write | per zone (same-zone only) |
|
|
706
|
+
| `key normalize [--dry-run\|--write]` | write (with `--write`) | `human` |
|
|
627
707
|
| `key uid K` | read | any |
|
|
628
|
-
| `hook run NAME` | write | any |
|
|
629
708
|
|
|
630
709
|
**`put` input** (read from stdin when `--stdin` is given):
|
|
631
710
|
|
|
@@ -643,8 +722,8 @@ All verbs accept `--format=json` and emit a canonical envelope (success or error
|
|
|
643
722
|
{
|
|
644
723
|
"verb": "freshness",
|
|
645
724
|
"rows": [
|
|
646
|
-
{ "key": "
|
|
647
|
-
"zone": "
|
|
725
|
+
{ "key": "intake.upstream.notes",
|
|
726
|
+
"zone": "intake",
|
|
648
727
|
"last_refreshed_at": "2026-05-21T13:21:17Z",
|
|
649
728
|
"age_seconds": 65000,
|
|
650
729
|
"ttl_seconds": 43200,
|
|
@@ -655,7 +734,7 @@ All verbs accept `--format=json` and emit a canonical envelope (success or error
|
|
|
655
734
|
}
|
|
656
735
|
```
|
|
657
736
|
|
|
658
|
-
Each row reports one entry's verdict (`fresh`, `stale`, `never_refreshed`, or `no_policy`) against its matched `refresh:`
|
|
737
|
+
Each row reports one entry's verdict (`fresh`, `stale`, `never_refreshed`, or `no_policy`) against its matched `refresh:` rule. `textus build` consumes its own staleness signal and executes derived entries' projections under the `builder` role; `--dry-run` prints the plan without executing.
|
|
659
738
|
|
|
660
739
|
`textus accept K --as=human` promotes a pending entry into its target zone: it copies the patch body into the target key, deletes the pending entry, and writes one audit line per side (§audit). Only the `human` role may invoke `accept`.
|
|
661
740
|
|
|
@@ -673,35 +752,35 @@ Every `Textus::Error` exposes `code`, `message`, and an optional `hint:`. The hi
|
|
|
673
752
|
|
|
674
753
|
## 10.2 `textus doctor`
|
|
675
754
|
|
|
676
|
-
`textus doctor` returns a health-check envelope: `{ "protocol": "textus/
|
|
755
|
+
`textus doctor` returns a health-check envelope: `{ "protocol": "textus/3", "ok": bool, "issues": [...], "summary": {error, warning, info} }`. Each issue carries `code`, `level` (`error|warning|info`), `subject`, `message`, and optionally `fix`. `ok` is true iff no error-level issues are present; warnings and info do not flip the bit. Builtin checks: `manifest_files`, `schemas`, `schema_parse_error`, `templates`, `hooks`, `illegal_keys`, `sentinels`, `audit_log`, `unowned_schema_fields`, `schema_violations`, `rule_ambiguity`, `intake_handler_allowlist`. Additional registered `:validate` hooks (§5.10) run after the builtin set. Exit code is 0 on `ok`, 1 otherwise.
|
|
677
756
|
|
|
678
757
|
## 11. Versioning
|
|
679
758
|
|
|
680
|
-
- The current wire string is `textus/
|
|
681
|
-
- Backward-compatible additions (new fields, new error codes, new schema types) MAY be made under `textus/
|
|
682
|
-
- Breaking changes (renamed/removed envelope fields, zone semantics, key grammar) require a new wire string `textus/
|
|
759
|
+
- The current wire string is `textus/3`.
|
|
760
|
+
- Backward-compatible additions (new fields, new error codes, new schema types) MAY be made under `textus/3`.
|
|
761
|
+
- Breaking changes (renamed/removed envelope fields, zone semantics, key grammar) require a new wire string `textus/4`.
|
|
683
762
|
- Implementations MUST reject envelopes whose `protocol` they do not recognize.
|
|
684
763
|
|
|
685
|
-
The reference Ruby gem follows semver independently and speaks `textus/
|
|
764
|
+
The reference Ruby gem follows semver independently and speaks `textus/3`.
|
|
686
765
|
|
|
687
766
|
## 12. Conformance fixtures
|
|
688
767
|
|
|
689
768
|
A conformant implementation MUST pass these fixtures (the reference test suite ships a YAML file listing inputs and expected envelopes):
|
|
690
769
|
|
|
691
770
|
**Fixture A — Resolve and read:**
|
|
692
|
-
Given a manifest with `working.network.org` → `working/network/org` (nested), schema `person`, and a file `.textus/zones/working/network/org/jane.md` with valid frontmatter, `textus get working.network.org.jane --
|
|
771
|
+
Given a manifest with `working.network.org` → `working/network/org` (nested), schema `person`, and a file `.textus/zones/working/network/org/jane.md` with valid frontmatter, `textus get working.network.org.jane --output=json` returns the canonical envelope with `etag` matching the file's sha256.
|
|
693
772
|
|
|
694
773
|
**Fixture B — Role gate on write:**
|
|
695
|
-
Given a manifest entry where `key: identity.self` lives in the `identity` zone (human-only), `textus put identity.self --stdin --as=
|
|
774
|
+
Given a manifest entry where `key: identity.self` lives in the `identity` zone (human-only), `textus put identity.self --stdin --as=agent` (with any valid input) returns the error envelope with `code: "write_forbidden"` and exit code 1.
|
|
696
775
|
|
|
697
776
|
**Fixture C — Schema violation:**
|
|
698
777
|
Given the `person` schema and a `put` whose frontmatter omits `relationship`, the result is the error envelope with `code: "schema_violation"`, `details.missing: ["relationship"]`, and exit code 1.
|
|
699
778
|
|
|
700
779
|
**Fixture D — Staleness detection:**
|
|
701
|
-
Given a manifest entry `
|
|
780
|
+
Given a manifest entry `intake.notes` matched by a `rules: [{ match: intake.notes, refresh: { ttl: 1h } }]` block and an envelope on disk whose `_meta.last_refreshed_at` is older than `now - ttl`, `textus freshness --output=json` includes a row for `intake.notes` with `status: "stale"`. Calling `textus freshness` does NOT trigger a refresh.
|
|
702
781
|
|
|
703
782
|
**Fixture E — Projection build:**
|
|
704
|
-
Given a manifest entry `derived.catalogs.skills` whose `projection` clause selects fields from `working.projects` entries, `textus build derived.catalogs.skills` materializes the derived entry on disk with frontmatter and body matching the projected shape, and updates `generated.at` to the build timestamp.
|
|
783
|
+
Given a manifest entry `derived.catalogs.skills` whose `compute: { kind: projection }` clause selects fields from `working.projects` entries, `textus build derived.catalogs.skills` materializes the derived entry on disk with frontmatter and body matching the projected shape, and updates `generated.at` to the build timestamp.
|
|
705
784
|
|
|
706
785
|
**Fixture F — Mustache render:**
|
|
707
786
|
Given a derived entry with a `template` clause referencing a `.mustache` file and inputs drawn from other keys, `textus build` produces a body whose contents match the expected rendered output byte-for-byte (after trailing-newline normalization).
|
|
@@ -719,13 +798,13 @@ Given a review entry `review.identity.self.patch` proposing a change to `identit
|
|
|
719
798
|
|
|
720
799
|
- **Why not MCP?** MCP is a transport; textus is a data model. The two compose: a 50-line MCP server can wrap `textus get/put` as tools. textus exists because the *shape* of agent-readable project memory deserves a standalone spec, separate from how it's served.
|
|
721
800
|
|
|
722
|
-
- **Why doesn't textus execute generator commands itself?** textus is a dataflow oracle, not a build runner. The moment a spec includes process execution, it inherits shell-injection surface, OS-portability concerns, and signal-handling semantics — and ends up duplicating whatever build system the consumer already runs (make, rake, just, lefthook, CI). Keeping execution external means a Python or TypeScript port of `textus/
|
|
801
|
+
- **Why doesn't textus execute generator commands itself?** textus is a dataflow oracle, not a build runner. The moment a spec includes process execution, it inherits shell-injection surface, OS-portability concerns, and signal-handling semantics — and ends up duplicating whatever build system the consumer already runs (make, rake, just, lefthook, CI). Keeping execution external means a Python or TypeScript port of `textus/3` only has to parse YAML and emit JSON; it doesn't have to spawn processes safely. Build runners stay the executor; textus stays a data tool.
|
|
723
802
|
|
|
724
803
|
- **Why not plain Markdown vaults (Obsidian / Foam)?** No schema enforcement, no write-gating, no addressable wire format. Fine for human notes; underspecified for agents that must act on the contents deterministically.
|
|
725
804
|
|
|
726
805
|
- **Why not Notion / Coda?** Closed, hosted, lossy export. textus is local-first, plain-files, diffable in git.
|
|
727
806
|
|
|
728
|
-
- **Why not JSON Schema for the schemas?** Considered. Bespoke YAML chosen for v1: simpler implementation, lighter dependency footprint, matches the reference impl's house language. JSON Schema MAY be added as an alternate schema-language adapter in a future minor revision without breaking `textus/
|
|
807
|
+
- **Why not JSON Schema for the schemas?** Considered. Bespoke YAML chosen for v1: simpler implementation, lighter dependency footprint, matches the reference impl's house language. JSON Schema MAY be added as an alternate schema-language adapter in a future minor revision without breaking `textus/3`.
|
|
729
808
|
|
|
730
809
|
- **Why not a database (SQLite, kv store)?** textus's whole point is that the storage is plain files agents and humans both read. A binary store loses git-diff, grep, and editor support.
|
|
731
810
|
|
|
@@ -742,7 +821,7 @@ Textus internals are organized into four layers. The dependency rule is one-way
|
|
|
742
821
|
|
|
743
822
|
The `lib/textus/store/`, `lib/textus/manifest/`, `lib/textus/hooks/` namespaces are infrastructure adapters that predate this split and remain at their existing paths for backward-compat with the plugin DSL.
|
|
744
823
|
|
|
745
|
-
Plugin authors interact only with the Hook DSL (`Textus.
|
|
824
|
+
Plugin authors interact only with the Hook DSL (`Textus.on(:resolve_intake, ...)`, `Textus.on(:entry_refreshed, ...)`, etc.) and the manifest YAML schema. The layering is internal and may evolve.
|
|
746
825
|
|
|
747
826
|
Both read and write paths flow through the application layer:
|
|
748
827
|
|
|
@@ -753,7 +832,7 @@ Both read and write paths flow through the application layer:
|
|
|
753
832
|
|
|
754
833
|
See `ARCHITECTURE.md` for an ASCII diagram and the full read-path walkthrough.
|
|
755
834
|
|
|
756
|
-
## 14. Open questions (
|
|
835
|
+
## 14. Open questions (v3.x scope)
|
|
757
836
|
|
|
758
837
|
- **Locking on `put`:** the reference impl uses sha256 etags. Should the spec also define a file-lock fallback for systems where read-before-write is racy?
|
|
759
838
|
- **Schema imports:** can one schema reference another (`type: $ref: person`)?
|
|
@@ -762,26 +841,75 @@ See `ARCHITECTURE.md` for an ASCII diagram and the full read-path walkthrough.
|
|
|
762
841
|
|
|
763
842
|
## 15. Implementation checklist
|
|
764
843
|
|
|
765
|
-
A `textus/
|
|
844
|
+
A `textus/3` implementation MUST:
|
|
766
845
|
|
|
767
|
-
- [ ] Parse `.textus/manifest.yaml` and accept `version: textus/
|
|
846
|
+
- [ ] Parse `.textus/manifest.yaml` and accept `version: textus/3`.
|
|
768
847
|
- [ ] Resolve keys via longest-prefix match against manifest entries.
|
|
769
848
|
- [ ] Read `_meta` + body from `.md` files; validate against the named schema.
|
|
770
849
|
- [ ] Read `_meta` from the top-level `_meta` hash in `.json` / `.yaml` files; validate against the named schema.
|
|
771
850
|
- [ ] Compute `sha256:<hex>` etags over raw file bytes.
|
|
772
|
-
- [ ] Refuse writes whose resolved role is not in the target zone's `
|
|
851
|
+
- [ ] Refuse writes whose resolved role is not in the target zone's `write_policy` list with `write_forbidden`.
|
|
773
852
|
- [ ] Return envelopes matching the shape in §8 exactly (with `_meta`, not `frontmatter`).
|
|
774
853
|
- [ ] Use the error codes in §8 and the exit-code table.
|
|
775
|
-
- [ ] Implement `textus freshness` per §5.1 and §9, walking each entry, matching it against the top-level `
|
|
854
|
+
- [ ] Implement `textus freshness` per §5.1 and §9, walking each entry, matching it against the top-level `rules:` block, and reporting `fresh|stale|never_refreshed|no_policy` without invoking any refresh.
|
|
776
855
|
- [ ] Pass the conformance fixtures A–I in §12.
|
|
777
856
|
|
|
778
|
-
A `textus/
|
|
857
|
+
A `textus/3` implementation MAY:
|
|
779
858
|
|
|
780
859
|
- Add additional CLI verbs (e.g. `move`, vendor-specific reporters) beyond the current set in §9.
|
|
781
|
-
- Provide alternate output formats (`--
|
|
860
|
+
- Provide alternate output formats (`--output=yaml`, `--output=table`) for human use.
|
|
782
861
|
- Support additional schema field types beyond §6, marked as `vendor:<name>` extensions.
|
|
783
862
|
|
|
863
|
+
## 16. Migrating from textus/2
|
|
864
|
+
|
|
865
|
+
textus 0.12.0 does not ship a built-in migrator. Upgrade path:
|
|
866
|
+
|
|
867
|
+
1. Install textus **0.11.x** first.
|
|
868
|
+
2. Run `textus migrate --to=textus/3` (available in 0.11.x only). This rewrites `manifest.yaml`, renames the `inbox/` zone directory to `intake/`, sweeps frontmatter `owner:` fields, writes an audit-log marker, and reports legacy hook-DSL call sites for manual review.
|
|
869
|
+
3. Upgrade to textus **0.12.0**.
|
|
870
|
+
4. If `.textus/audit.log` contains pre-0.11.0 rows with `role: ai|script|build`, run `textus audit-rewrite-legacy-roles` once (one-shot verb; removed in 0.13.0).
|
|
871
|
+
|
|
872
|
+
**textus doctor refuses textus/2 stores.** The doctor check `protocol_version` emits an `error`-level issue when `manifest.yaml` carries `version: textus/2`. Install 0.11.x and migrate before upgrading to 0.12.0.
|
|
873
|
+
|
|
874
|
+
**Vocabulary summary** (textus/2 → textus/3 rename table, for reference):
|
|
875
|
+
|
|
876
|
+
| Category | textus/2 | textus/3 |
|
|
877
|
+
|---|---|---|
|
|
878
|
+
| Actor | `ai` | `agent` |
|
|
879
|
+
| Actor | `script` | `runner` |
|
|
880
|
+
| Actor | `build` | `builder` |
|
|
881
|
+
| Zone | `inbox` | `intake` |
|
|
882
|
+
| Manifest | `writable_by:` | `write_policy:` |
|
|
883
|
+
| Manifest | `policies:` | `rules:` |
|
|
884
|
+
| Manifest | `handler_allowlist:` | `intake_handler_allowlist:` |
|
|
885
|
+
| Manifest | `promote_requires:` | `promotion: { requires: [...] }` |
|
|
886
|
+
| Manifest | `projection:` | `compute: { kind: projection, ... }` |
|
|
887
|
+
| Manifest | `generator:` | `compute: { kind: external, ... }` |
|
|
888
|
+
| Hook event | `:intake` | `:resolve_intake` |
|
|
889
|
+
| Hook event | `:reduce` | `:transform_rows` |
|
|
890
|
+
| Hook event | `:check` | `:validate` |
|
|
891
|
+
| Hook event | `:put` | `:entry_put` |
|
|
892
|
+
| Hook event | `:deleted` | `:entry_deleted` |
|
|
893
|
+
| Hook event | `:refreshed` | `:entry_refreshed` |
|
|
894
|
+
| Hook event | `:built` | `:build_completed` |
|
|
895
|
+
| Hook event | `:accepted` | `:proposal_accepted` |
|
|
896
|
+
| Hook event | `:reject` | `:proposal_rejected` |
|
|
897
|
+
| Hook event | `:published` | `:file_published` |
|
|
898
|
+
| Hook event | `:mv` | `:entry_renamed` |
|
|
899
|
+
| Hook event | `:loaded` | `:store_loaded` |
|
|
900
|
+
| Hook event | `:refresh_began` | `:refresh_started` |
|
|
901
|
+
| Hook event | `:refresh_detached` | `:refresh_backgrounded` |
|
|
902
|
+
| Hook event | `:refresh_failed` | `:refresh_failed` (unchanged) |
|
|
903
|
+
| Hook DSL | `Textus.hook(ev, name)` / sugar | `Textus.on(ev, name)` |
|
|
904
|
+
| Compute field | `projection.reduce:` | `compute.transform:` |
|
|
905
|
+
| `_meta` key | `reducer` | `transform` |
|
|
906
|
+
| CLI flag | `--format=json` (envelope) | `--output=json` |
|
|
907
|
+
| CLI verb | `refresh-stale` | `refresh stale` |
|
|
908
|
+
| CLI verb | `policy list/explain` | `rule list/explain` |
|
|
909
|
+
|
|
910
|
+
**Notes on hook migration.** The 0.11.x migrator scanner reports each file and call site that uses a legacy event name or DSL method. No automatic rewrite is performed. Update each hook to use `Textus.on(:new_event_name, ...)` before re-enabling the hook. See CHANGELOG §0.11.0 for the full event rename table.
|
|
911
|
+
|
|
784
912
|
---
|
|
785
913
|
|
|
786
|
-
**Spec word count target:** <
|
|
787
|
-
**Reviewed against community-testing checklist (idea file §"Community-testing"):** ✅
|
|
914
|
+
**Spec word count target:** <2700 words (allowance widened to fit vocabulary axes intro + migration section).
|
|
915
|
+
**Reviewed against community-testing checklist (idea file §"Community-testing"):** ✅ implementable in a day in TS/Python (four concepts: manifest, schema, envelope, staleness check); ✅ conformance fixtures A–I; ✅ "Why not X?" section present (incl. why no execution); ✅ name picked.
|