textus 0.10.5 → 0.14.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/ARCHITECTURE.md +60 -40
- data/CHANGELOG.md +318 -3
- data/README.md +34 -27
- data/SPEC.md +226 -145
- data/docs/conventions.md +8 -8
- data/lib/textus/application/context.rb +4 -0
- data/lib/textus/application/reads/blame.rb +1 -1
- data/lib/textus/application/reads/deps.rb +15 -0
- data/lib/textus/application/reads/freshness.rb +4 -4
- data/lib/textus/application/reads/get.rb +9 -12
- data/lib/textus/application/reads/list.rb +15 -0
- data/lib/textus/application/reads/policy_explain.rb +2 -2
- data/lib/textus/application/reads/published.rb +15 -0
- data/lib/textus/application/reads/rdeps.rb +15 -0
- data/lib/textus/application/reads/schema_envelope.rb +15 -0
- data/lib/textus/application/reads/stale.rb +15 -0
- data/lib/textus/application/reads/uid.rb +15 -0
- data/lib/textus/application/reads/validate_all.rb +15 -0
- data/lib/textus/application/reads/where.rb +15 -0
- data/lib/textus/application/refresh/all.rb +2 -2
- data/lib/textus/application/refresh/orchestrator.rb +1 -1
- data/lib/textus/application/refresh/worker.rb +8 -8
- data/lib/textus/application/writes/accept.rb +26 -8
- data/lib/textus/application/writes/build.rb +12 -49
- data/lib/textus/application/writes/delete.rb +1 -1
- data/lib/textus/application/writes/mv.rb +144 -0
- data/lib/textus/application/writes/publish.rb +42 -10
- data/lib/textus/application/writes/put.rb +1 -1
- data/lib/textus/application/writes/reject.rb +37 -0
- data/lib/textus/builder/pipeline.rb +1 -1
- data/lib/textus/builder/renderer/json.rb +1 -1
- data/lib/textus/builder/renderer/yaml.rb +1 -1
- data/lib/textus/cli/group/key.rb +1 -1
- data/lib/textus/cli/group/refresh.rb +21 -0
- data/lib/textus/cli/group/rule.rb +11 -0
- data/lib/textus/cli/verb/accept.rb +1 -2
- data/lib/textus/cli/verb/audit.rb +3 -3
- data/lib/textus/cli/verb/blame.rb +1 -2
- data/lib/textus/cli/verb/build.rb +6 -2
- data/lib/textus/cli/verb/delete.rb +1 -2
- data/lib/textus/cli/verb/deps.rb +1 -1
- data/lib/textus/cli/verb/freshness.rb +1 -2
- data/lib/textus/cli/verb/get.rb +2 -3
- data/lib/textus/cli/verb/hook_run.rb +3 -2
- data/lib/textus/cli/verb/hooks.rb +1 -1
- data/lib/textus/cli/verb/{migrate_keys.rb → key_normalize.rb} +1 -1
- data/lib/textus/cli/verb/list.rb +1 -1
- data/lib/textus/cli/verb/mv.rb +1 -1
- data/lib/textus/cli/verb/published.rb +1 -1
- data/lib/textus/cli/verb/put.rb +3 -3
- data/lib/textus/cli/verb/rdeps.rb +1 -1
- data/lib/textus/cli/verb/refresh.rb +1 -2
- data/lib/textus/cli/verb/reject.rb +1 -1
- data/lib/textus/cli/verb/{policy_explain.rb → rule_explain.rb} +2 -3
- data/lib/textus/cli/verb/{policy_list.rb → rule_list.rb} +3 -3
- data/lib/textus/cli/verb/schema.rb +1 -1
- data/lib/textus/cli/verb/uid.rb +1 -1
- data/lib/textus/cli/verb/where.rb +1 -1
- data/lib/textus/cli/verb.rb +9 -3
- data/lib/textus/cli.rb +6 -6
- data/lib/textus/doctor/check/handler_allowlist.rb +1 -1
- data/lib/textus/doctor/check/illegal_keys.rb +39 -16
- data/lib/textus/doctor/check/intake_registration.rb +4 -4
- data/lib/textus/doctor/check/protocol_version.rb +47 -0
- data/lib/textus/doctor/check/{policy_ambiguity.rb → rule_ambiguity.rb} +6 -6
- data/lib/textus/doctor/check/schema_violations.rb +1 -1
- data/lib/textus/doctor.rb +6 -5
- data/lib/textus/domain/freshness/evaluator.rb +1 -1
- data/lib/textus/domain/permission.rb +4 -4
- data/lib/textus/domain/policy/predicates/human_accept.rb +31 -0
- data/lib/textus/domain/policy/predicates/schema_valid.rb +50 -0
- data/lib/textus/domain/policy/promotion.rb +45 -0
- data/lib/textus/entry/base.rb +28 -0
- data/lib/textus/entry/json.rb +59 -0
- data/lib/textus/entry/markdown.rb +46 -0
- data/lib/textus/entry/text.rb +35 -0
- data/lib/textus/entry/yaml.rb +59 -0
- data/lib/textus/entry.rb +16 -0
- data/lib/textus/envelope.rb +44 -14
- data/lib/textus/errors.rb +24 -5
- data/lib/textus/hooks/builtin.rb +5 -5
- data/lib/textus/hooks/dispatcher.rb +1 -1
- data/lib/textus/hooks/dsl.rb +3 -10
- data/lib/textus/hooks/loader.rb +1 -2
- data/lib/textus/hooks/registry.rb +22 -21
- data/lib/textus/infra/refresh/detached.rb +1 -1
- data/lib/textus/init.rb +25 -34
- data/lib/textus/intro.rb +65 -9
- data/lib/textus/manifest/entry/parser.rb +84 -0
- data/lib/textus/manifest/entry/validators/events.rb +21 -0
- data/lib/textus/manifest/entry/validators/format_matrix.rb +26 -0
- data/lib/textus/manifest/entry/validators/index_filename.rb +45 -0
- data/lib/textus/manifest/entry/validators/inject_intro.rb +18 -0
- data/lib/textus/manifest/entry/validators/publish_each.rb +37 -0
- data/lib/textus/manifest/entry/validators.rb +20 -0
- data/lib/textus/manifest/entry.rb +38 -189
- data/lib/textus/manifest/{policies.rb → rules.rb} +12 -10
- data/lib/textus/manifest/schema.rb +49 -0
- data/lib/textus/manifest.rb +50 -24
- data/lib/textus/migrate_keys.rb +1 -1
- data/lib/textus/operations/reads.rb +39 -0
- data/lib/textus/operations/refresh.rb +27 -0
- data/lib/textus/operations/writes.rb +21 -0
- data/lib/textus/operations.rb +44 -0
- data/lib/textus/projection.rb +9 -8
- data/lib/textus/refresh.rb +4 -5
- data/lib/textus/schema/tools.rb +8 -7
- data/lib/textus/store/reader.rb +1 -1
- data/lib/textus/store/staleness/intake_check.rb +1 -1
- data/lib/textus/store/validator.rb +3 -3
- data/lib/textus/store/writer.rb +5 -74
- data/lib/textus/store.rb +1 -55
- data/lib/textus/version.rb +2 -2
- data/lib/textus.rb +1 -0
- metadata +35 -10
- data/lib/textus/cli/group/policy.rb +0 -11
- data/lib/textus/composition.rb +0 -72
- data/lib/textus/proposal.rb +0 -10
- data/lib/textus/store/mover.rb +0 -167
data/SPEC.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
|
-
# textus/
|
|
1
|
+
# textus/3 — Specification
|
|
2
2
|
|
|
3
|
-
**Status:** Draft
|
|
4
|
-
**Protocol identifier:** `textus/
|
|
3
|
+
**Status:** Draft v3.0
|
|
4
|
+
**Protocol identifier:** `textus/3`
|
|
5
5
|
**Reference implementation:** Ruby gem `textus`
|
|
6
6
|
|
|
7
7
|
> *textus* — Latin for "the fabric a text is woven from," same root as *context* (from *con-texere*, "to weave together"). This spec defines a storage shape and wire protocol for that fabric.
|
|
@@ -10,20 +10,31 @@
|
|
|
10
10
|
|
|
11
11
|
## 1. What textus is
|
|
12
12
|
|
|
13
|
-
A storage convention and JSON wire protocol that lets humans,
|
|
13
|
+
A storage convention and JSON wire protocol that lets humans, agents, and runners read and write structured project memory **deterministically**, with addressable dotted keys, schema validation, role-based write gates, declarative compute, and copy-based publish targets.
|
|
14
14
|
|
|
15
|
-
The storage lives in a `.textus/` directory at the project root. Each entry is a Markdown file with YAML frontmatter. A manifest binds dotted keys to subtrees and declares which roles may write to each zone. Schemas (also YAML) define what frontmatter shape each entry must have. Derived entries are computed from other entries via pure projections and a vendored Mustache template engine, then optionally published to repo-relative paths as byte-for-byte file copies. The CLI surface (`textus get/put/list/where/schema/build/...` `--
|
|
15
|
+
The storage lives in a `.textus/` directory at the project root. Each entry is a Markdown file with YAML frontmatter. A manifest binds dotted keys to subtrees and declares which roles may write to each zone. Schemas (also YAML) define what frontmatter shape each entry must have. Derived entries are computed from other entries via pure projections and a vendored Mustache template engine, then optionally published to repo-relative paths as byte-for-byte file copies. The CLI surface (`textus get/put/list/where/schema/build/...` `--output=json`) returns a versioned envelope any caller can parse without knowing Markdown.
|
|
16
16
|
|
|
17
17
|
You **shape your own memory structure** inside `.textus/`. The protocol manages how it's read, written, addressed, validated, gated, computed, and published. The contents are entirely yours.
|
|
18
18
|
|
|
19
|
-
### 1.1
|
|
19
|
+
### 1.1 Vocabulary axes
|
|
20
|
+
|
|
21
|
+
textus/3 names its concepts along six axes. Reviewers who internalize these can map any part of the spec to the right category:
|
|
22
|
+
|
|
23
|
+
- **Actor** — who is interacting: `human`, `agent`, `runner`, `builder`.
|
|
24
|
+
- **Place** — where data lives: zones such as `identity`, `working`, `intake`, `review`, `output`.
|
|
25
|
+
- **Thing** — what is stored: entries, fields, keys.
|
|
26
|
+
- **Operation** — how you act on things: RPC and CLI verbs (`get`, `put`, `refresh`, `build`, …).
|
|
27
|
+
- **Event** — what gets fired after an operation: hook event names, split into RPC events (`:resolve_intake`, `:transform_rows`, `:validate`) and pub-sub events (`:entry_put`, `:build_completed`, …).
|
|
28
|
+
- **Rule** — constraints declared in the top-level `rules:` array of the manifest.
|
|
29
|
+
|
|
30
|
+
### 1.2 The five layers
|
|
20
31
|
|
|
21
32
|
textus is organized as five composable layers. Each layer has a single responsibility; later layers build on earlier ones.
|
|
22
33
|
|
|
23
34
|
| Layer | Name | Responsibility |
|
|
24
35
|
|---|---|---|
|
|
25
36
|
| L1 | **Store** | Plain-file backend: `.textus/zones/<zone>/...` with YAML frontmatter + Markdown body, addressed by dotted keys, schema-validated, etag-versioned. |
|
|
26
|
-
| L2 | **Sources** | Declared external inputs (the `
|
|
37
|
+
| L2 | **Sources** | Declared external inputs (the `intake` zone in the default scaffold; any zone writable by `runner`): URLs, files, feeds with declared parsers and TTLs. textus *describes* sources; external runners fetch and pipe results through `textus put`. |
|
|
27
38
|
| L3 | **Compute** | Pure transforms from store entries to derived entries. Projections (select/pluck/sort/limit/format) plus a vendored Mustache template subset. No shell execution. |
|
|
28
39
|
| L4 | **Publish** | Byte-for-byte file copy from derived entries to repo-relative paths declared via `publish_to:`. The in-store artifact is the consumer-shaped output; the published file is an identical copy. A sentinel under `.textus/sentinels/<target-rel-path>.textus-managed.json` records the source, sha256, and `mode: "copy"`. |
|
|
29
40
|
| L5 | **Consumers** | Anything that reads the published files or calls the CLI — editors, LLM tools, MCP servers, CI jobs, dashboards. textus is agnostic about who consumes; the envelope is the contract. |
|
|
@@ -31,10 +42,10 @@ textus is organized as five composable layers. Each layer has a single responsib
|
|
|
31
42
|
## 2. Goals and non-goals
|
|
32
43
|
|
|
33
44
|
**Goals**
|
|
34
|
-
- Stable wire format (`textus/
|
|
45
|
+
- Stable wire format (`textus/3`) any language can speak.
|
|
35
46
|
- Deterministic read/write of structured Markdown via a CLI returning JSON.
|
|
36
47
|
- Schema-validated frontmatter using YAML schemas as data.
|
|
37
|
-
- Role-based write gates (humans,
|
|
48
|
+
- Role-based write gates (humans, agents, runners, builders get different permissions per zone).
|
|
38
49
|
- Optimistic concurrency via ETags.
|
|
39
50
|
- Pure declarative compute: derived entries computed from projections + Mustache, no shell-out.
|
|
40
51
|
- Publish derived entries to well-known paths as body-only plain files.
|
|
@@ -63,10 +74,10 @@ The root is `.textus/` at the project working directory. A typical tree:
|
|
|
63
74
|
parsers/ # internal: project-local parser extensions
|
|
64
75
|
zones/ # ALL user content lives here
|
|
65
76
|
identity/ # zone: identity (human-only)
|
|
66
|
-
working/ # zone: working (human,
|
|
67
|
-
|
|
68
|
-
review/ # zone: review (
|
|
69
|
-
output/ # zone: output (
|
|
77
|
+
working/ # zone: working (human, agent, runner)
|
|
78
|
+
intake/ # zone: intake (runner — declared external inputs)
|
|
79
|
+
review/ # zone: review (agent/human — proposals awaiting accept)
|
|
80
|
+
output/ # zone: output (builder only — computed outputs)
|
|
70
81
|
```
|
|
71
82
|
|
|
72
83
|
Textus internals (`manifest.yaml`, `audit.log`, `role`, `schemas/`, `templates/`, `parsers/`) live directly under `.textus/`. **All user content lives under `.textus/zones/`.** Manifest `path:` fields are relative to `.textus/zones/` — they do **not** include the `zones/` prefix. Implementations MUST prepend `zones/` to every `path:` when resolving a key to a filesystem location.
|
|
@@ -91,19 +102,19 @@ The manifest declares: (a) which zones exist and which roles may write to each,
|
|
|
91
102
|
|
|
92
103
|
```yaml
|
|
93
104
|
# .textus/manifest.yaml
|
|
94
|
-
version: textus/
|
|
105
|
+
version: textus/3
|
|
95
106
|
|
|
96
107
|
zones:
|
|
97
108
|
- name: identity
|
|
98
|
-
|
|
109
|
+
write_policy: [human]
|
|
99
110
|
- name: working
|
|
100
|
-
|
|
101
|
-
- name:
|
|
102
|
-
|
|
111
|
+
write_policy: [human, agent, runner]
|
|
112
|
+
- name: intake
|
|
113
|
+
write_policy: [runner]
|
|
103
114
|
- name: review
|
|
104
|
-
|
|
115
|
+
write_policy: [agent, human]
|
|
105
116
|
- name: output
|
|
106
|
-
|
|
117
|
+
write_policy: [builder]
|
|
107
118
|
|
|
108
119
|
entries:
|
|
109
120
|
- key: identity.self
|
|
@@ -124,8 +135,8 @@ entries:
|
|
|
124
135
|
schema: null
|
|
125
136
|
owner: textus:build
|
|
126
137
|
|
|
127
|
-
|
|
128
|
-
- match:
|
|
138
|
+
rules:
|
|
139
|
+
- match: intake.**
|
|
129
140
|
refresh: { ttl: 6h, on_stale: warn }
|
|
130
141
|
```
|
|
131
142
|
|
|
@@ -184,17 +195,17 @@ A leaf at `working.skills.writing.voice-writer` (authored at `.textus/zones/work
|
|
|
184
195
|
|
|
185
196
|
## 5. Zones and role-based write gates
|
|
186
197
|
|
|
187
|
-
Each zone declares which **roles** may write to it via `
|
|
198
|
+
Each zone declares which **roles** may write to it via `write_policy:` in the manifest. An optional `read_policy:` (default `[all]`) gates reads. Writes are gated; reads are unrestricted by default.
|
|
188
199
|
|
|
189
|
-
| Zone | `
|
|
200
|
+
| Zone | `write_policy` | Use case |
|
|
190
201
|
|---|---|---|
|
|
191
202
|
| `identity` | `[human]` | Identity, voice, immutable principles — things only a human edits. |
|
|
192
|
-
| `working` | `[human,
|
|
193
|
-
| `
|
|
194
|
-
| `review` | `[
|
|
195
|
-
| `output` | `[
|
|
203
|
+
| `working` | `[human, agent, runner]` | Active project state: notes, decisions, network — what humans and agents update day-to-day. |
|
|
204
|
+
| `intake` | `[runner]` | Declared external inputs (calendar, feeds, scraped pages). Refreshed by external runner scripts; never by humans or agents directly. |
|
|
205
|
+
| `review` | `[agent, human]` | Agent-generated proposals awaiting human review via `textus accept`. Lets agents stage changes without touching `working`. |
|
|
206
|
+
| `output` | `[builder]` | Computed outputs (catalogs, indexes, published context). Written only by the build runner via `textus build`. |
|
|
196
207
|
|
|
197
|
-
A write is gated by the caller's **role**, supplied via `--as=<role>`. If the role is not in the target zone's `
|
|
208
|
+
A write is gated by the caller's **role**, supplied via `--as=<role>`. If the role is not in the target zone's `write_policy` list, the write returns `write_forbidden`.
|
|
198
209
|
|
|
199
210
|
### 5.1 Role resolution
|
|
200
211
|
|
|
@@ -205,23 +216,36 @@ The effective role for any CLI invocation is resolved in this order; the first m
|
|
|
205
216
|
3. `.textus/role` file (one line, role name) at the project root.
|
|
206
217
|
4. Default: `human`.
|
|
207
218
|
|
|
208
|
-
|
|
219
|
+
**Canonical actors (textus/3):**
|
|
220
|
+
|
|
221
|
+
| Actor | Meaning |
|
|
222
|
+
|---|---|
|
|
223
|
+
| `human` | Interactive user at a terminal. |
|
|
224
|
+
| `agent` | Long-running AI or LLM process. |
|
|
225
|
+
| `runner` | Scheduled or one-shot automation script. |
|
|
226
|
+
| `builder` | Build/derive output (catalogs, indexes). |
|
|
227
|
+
|
|
228
|
+
Unknown role values are rejected with `invalid_role`.
|
|
209
229
|
|
|
210
230
|
Every successful write records the resolved role and a wall-clock timestamp in `.textus/audit.log`, so reviewers can later distinguish a human edit from an agent edit even though both live in the same file.
|
|
211
231
|
|
|
212
232
|
### 5.2 Compute layer (derived entries)
|
|
213
233
|
|
|
214
|
-
Derived entries live in a zone whose `
|
|
234
|
+
Derived entries live in a zone whose `write_policy:` list includes `builder` — `output` in the default scaffold. They are not authored by hand; their body is produced by projecting over other entries. A derived entry declares a `compute:` block with a `kind:` discriminator.
|
|
235
|
+
|
|
236
|
+
#### 5.2.1 Projection compute (`kind: projection`)
|
|
215
237
|
|
|
216
238
|
```yaml
|
|
217
239
|
- key: output.catalogs.people
|
|
218
240
|
zone: output
|
|
219
|
-
|
|
241
|
+
compute:
|
|
242
|
+
kind: projection
|
|
220
243
|
select: working.network.org # prefix OR [list of prefixes]
|
|
221
244
|
pluck: [name, relationship, org]
|
|
222
245
|
sort_by: name # optional
|
|
223
246
|
limit: 1000 # default 1000, max 1000
|
|
224
247
|
format: yaml-list-in-md # one of: list, hash, yaml-list-in-md, json, markdown-table
|
|
248
|
+
transform: rank_by_recency # optional — names a :transform_rows hook
|
|
225
249
|
template: people.mustache # optional; if absent, format determines body
|
|
226
250
|
```
|
|
227
251
|
|
|
@@ -229,6 +253,8 @@ Derived entries live in a zone whose `writable_by:` list includes `build` — `o
|
|
|
229
253
|
|
|
230
254
|
`format` controls the body serialization when no template is supplied. Permitted values: `list`, `hash`, `yaml-list-in-md`, `json`, `markdown-table`.
|
|
231
255
|
|
|
256
|
+
`transform:` (optional) names a registered `:transform_rows` hook (see §5.10). The hook receives the projected rows array and may reorder, filter, or augment before serialization.
|
|
257
|
+
|
|
232
258
|
If `template` is given, it names a Mustache template under `.textus/templates/`. textus implements a deliberately restricted Mustache subset:
|
|
233
259
|
|
|
234
260
|
- `{{var}}` — variable interpolation.
|
|
@@ -238,16 +264,17 @@ If `template` is given, it names a Mustache template under `.textus/templates/`.
|
|
|
238
264
|
|
|
239
265
|
No partials. No lambdas. No HTML escaping (output is raw text, intended for Markdown). Template recursion depth is bounded at 8; exceeding the limit is an error.
|
|
240
266
|
|
|
241
|
-
|
|
267
|
+
#### 5.2.2 External compute (`kind: external`)
|
|
242
268
|
|
|
243
|
-
A derived entry that is produced by a build tool *outside* textus — `rake`, `just`, a shell script, anything — declares `
|
|
269
|
+
A derived entry that is produced by a build tool *outside* textus — `rake`, `just`, a shell script, anything — declares `compute: { kind: external, ... }`. textus does **not** execute the command (consistent with §2); the external runner is responsible for writing the file. textus records `sources:` so `textus freshness` can compare source mtimes against the derived file's `_meta.generated.at` and report staleness.
|
|
244
270
|
|
|
245
271
|
```yaml
|
|
246
272
|
- key: output.catalogs.skills
|
|
247
273
|
path: output/catalogs/skills.md
|
|
248
274
|
zone: output
|
|
249
275
|
owner: build:catalog-skills
|
|
250
|
-
|
|
276
|
+
compute:
|
|
277
|
+
kind: external
|
|
251
278
|
command: "rake catalog:skills" # informational; the runner invokes it
|
|
252
279
|
sources: # dotted keys OR repo-relative paths
|
|
253
280
|
- working.projects
|
|
@@ -258,7 +285,7 @@ A derived entry that is produced by a build tool *outside* textus — `rake`, `j
|
|
|
258
285
|
|
|
259
286
|
**`command:`** is recorded in the staleness row's `generator` field but never executed. It exists so `textus freshness` output can carry a hint about how to refresh.
|
|
260
287
|
|
|
261
|
-
**Freshness contract.** An entry with `
|
|
288
|
+
**Freshness contract.** An entry with `compute: { kind: external }` is reported by `textus freshness` as `stale` when:
|
|
262
289
|
- The derived file does not exist, OR
|
|
263
290
|
- `_meta.generated.at` is missing or unparseable, OR
|
|
264
291
|
- Any `sources:` element has been modified after `_meta.generated.at`.
|
|
@@ -272,9 +299,9 @@ generated:
|
|
|
272
299
|
from: [working.projects, working.network]
|
|
273
300
|
```
|
|
274
301
|
|
|
275
|
-
`generated.from` SHOULD match `
|
|
302
|
+
`generated.from` SHOULD match `compute.sources` — they're the same list, recorded twice so a diff proves what was actually consumed.
|
|
276
303
|
|
|
277
|
-
`
|
|
304
|
+
`kind: external` and `kind: projection` are alternatives — exactly one per entry. Templates are not required for `kind: external`: the runner produces the bytes directly.
|
|
278
305
|
|
|
279
306
|
### 5.3 Publish layer (`publish_to:`)
|
|
280
307
|
|
|
@@ -294,25 +321,25 @@ A sentinel is written for each published file at `<store_root>/sentinels/<target
|
|
|
294
321
|
|
|
295
322
|
### 5.4 Intake (declared, refreshed via registered intake handler)
|
|
296
323
|
|
|
297
|
-
Intake entries declare an external source by naming an **intake handler** — a registered, named function that pulls data into the entry. textus itself still makes no implicit network calls: an intake handler only runs when explicitly invoked by `textus refresh KEY --as=
|
|
324
|
+
Intake entries declare an external source by naming an **intake handler** — a registered, named function that pulls data into the entry. textus itself still makes no implicit network calls: an intake handler only runs when explicitly invoked by `textus refresh KEY --as=runner` (or by `textus refresh stale`). The declaration is data only:
|
|
298
325
|
|
|
299
326
|
```yaml
|
|
300
|
-
- key:
|
|
301
|
-
zone:
|
|
327
|
+
- key: intake.calendar.events
|
|
328
|
+
zone: intake
|
|
302
329
|
intake:
|
|
303
330
|
handler: ical-events
|
|
304
331
|
config:
|
|
305
332
|
url: "https://calendar.google.com/.../basic.ics"
|
|
306
333
|
|
|
307
|
-
|
|
308
|
-
- match:
|
|
334
|
+
rules:
|
|
335
|
+
- match: intake.calendar.**
|
|
309
336
|
refresh:
|
|
310
337
|
ttl: 6h
|
|
311
338
|
on_stale: warn # warn | sync | timed_sync (default: warn)
|
|
312
339
|
sync_budget_ms: 500 # only used when on_stale: timed_sync (default: 500)
|
|
313
340
|
```
|
|
314
341
|
|
|
315
|
-
`handler` names a registered `:
|
|
342
|
+
`handler` names a registered `:resolve_intake` hook (see §5.10 for the hook contract); `config` is an opaque hash handed to the handler. The freshness budget (`ttl`, `on_stale`, `sync_budget_ms`) lives in a top-level **`rules:`** block matched by key glob (§5.11).
|
|
316
343
|
|
|
317
344
|
#### `on_stale:` semantics
|
|
318
345
|
|
|
@@ -322,7 +349,7 @@ policies:
|
|
|
322
349
|
|---|---|
|
|
323
350
|
| `warn` (default) | Return the entry immediately with `stale: true`, `stale_reason:` populated, and `refreshing: false`. No blocking. |
|
|
324
351
|
| `sync` | Block the `get` call, run the intake handler in-process, write the refreshed result, then return the fresh envelope. The caller waits. |
|
|
325
|
-
| `timed_sync` | Like `sync`, but with a `sync_budget_ms` deadline (default 500 ms). If the handler finishes within the budget the fresh envelope is returned. If it does not finish in time, return the stale envelope (with `stale: true`, `refreshing: true`) and let the refresh complete in the background. Fires `:
|
|
352
|
+
| `timed_sync` | Like `sync`, but with a `sync_budget_ms` deadline (default 500 ms). If the handler finishes within the budget the fresh envelope is returned. If it does not finish in time, return the stale envelope (with `stale: true`, `refreshing: true`) and let the refresh complete in the background. Fires `:refresh_backgrounded` when the deadline is exceeded. |
|
|
326
353
|
|
|
327
354
|
> **Note:** `list`/`where` paths do **not** annotate freshness — only `get` does.
|
|
328
355
|
|
|
@@ -336,14 +363,14 @@ In intake mode the handler MUST return one of three shapes, all normalized by th
|
|
|
336
363
|
|
|
337
364
|
**Refresh paths.** Two are supported:
|
|
338
365
|
|
|
339
|
-
1. **In-process** — `textus refresh KEY --as=
|
|
340
|
-
2. **External runner** — a cron job or agent harness reads `textus list --zone=
|
|
366
|
+
1. **In-process** — `textus refresh KEY --as=runner` resolves the entry's `intake.handler`, invokes the registered `:resolve_intake` hook with `(config:, store:, args: {})`, and writes the result under role `runner`.
|
|
367
|
+
2. **External runner** — a cron job or agent harness reads `textus list --zone=intake --stale --output=json`, fetches the source out of band, and pipes bytes back through `textus put KEY --as=runner --stdin`. The CLI verb `textus refresh stale [--prefix=K] [--zone=Z]` drives this loop in one shot.
|
|
341
368
|
|
|
342
|
-
Both paths share the same role gate, audit-log entry, and `:
|
|
369
|
+
Both paths share the same role gate, audit-log entry, and `:entry_refreshed` event. User-supplied hooks live in `.textus/hooks/**/*.rb` and auto-load at `Store#initialize` — see §5.10 for the full hook contract.
|
|
343
370
|
|
|
344
371
|
### 5.5 Pending / accept workflow
|
|
345
372
|
|
|
346
|
-
Proposal entries are full patches authored into a zone whose `
|
|
373
|
+
Proposal entries are full patches authored into a zone whose `write_policy:` list includes `agent` — `review` in the default scaffold — typically by agents or runners. The entry's frontmatter describes the patch it proposes against another zone:
|
|
347
374
|
|
|
348
375
|
```yaml
|
|
349
376
|
---
|
|
@@ -360,7 +387,7 @@ Proposed body content.
|
|
|
360
387
|
|
|
361
388
|
`proposal.target_key` names the entry the patch would create or modify, and `proposal.action` is `put` or `delete`. The remaining frontmatter and body are the proposed new content.
|
|
362
389
|
|
|
363
|
-
`textus accept <proposal-key>` is **human-only**: the resolved role must be `human`. It copies the patch into the target zone, records provenance (originating proposal key, original role, original timestamp) in the audit log, and removes the proposal entry. Agents and
|
|
390
|
+
`textus accept <proposal-key>` is **human-only**: the resolved role must be `human`. It copies the patch into the target zone, records provenance (originating proposal key, original role, original timestamp) in the audit log, and removes the proposal entry. Agents and runners can propose but cannot accept.
|
|
364
391
|
|
|
365
392
|
### 5.6 Audit log
|
|
366
393
|
|
|
@@ -395,11 +422,11 @@ Schemas may declare per-field ownership and version history. The `fields:` and `
|
|
|
395
422
|
```yaml
|
|
396
423
|
fields:
|
|
397
424
|
full_name: { type: string, maintained_by: human }
|
|
398
|
-
embedding: { type: array, maintained_by:
|
|
399
|
-
updated_at: { type: time, maintained_by:
|
|
425
|
+
embedding: { type: array, maintained_by: agent }
|
|
426
|
+
updated_at: { type: time, maintained_by: runner }
|
|
400
427
|
```
|
|
401
428
|
|
|
402
|
-
`maintained_by` values are free-form strings. The recognized set is `human |
|
|
429
|
+
`maintained_by` values are free-form strings. The recognized set is `human | agent | runner | builder | derived`. Unrecognized values do not affect role-authority validation; they pass through unchanged.
|
|
403
430
|
|
|
404
431
|
**`evolution:` block** — top-level, declares the schema's history and migration intent:
|
|
405
432
|
|
|
@@ -415,61 +442,62 @@ evolution:
|
|
|
415
442
|
|
|
416
443
|
**Defaults:** when `fields:` and `evolution:` are absent, `schema.maintained_by(field)` returns `nil` for every field and `schema.evolution` returns `{}`.
|
|
417
444
|
|
|
418
|
-
**Override rule:** the role `human` is permitted to write any `maintained_by` field, regardless of declared owner. This preserves human authority over
|
|
445
|
+
**Override rule:** the role `human` is permitted to write any `maintained_by` field, regardless of declared owner. This preserves human authority over agent/runner-managed data — humans curating canon over agent-written embeddings is a feature, not a bug. All other role mismatches are reported by `doctor --check=schema_violations` with code `role_authority`, including fields `key`, `field`, `expected`, and `last_writer`.
|
|
419
446
|
|
|
420
|
-
### 5.9
|
|
447
|
+
### 5.9 Row transforms
|
|
421
448
|
|
|
422
|
-
|
|
449
|
+
Row transforms are RPC hooks on the `:transform_rows` event. See §5.10.
|
|
423
450
|
|
|
424
451
|
### 5.10 Hooks
|
|
425
452
|
|
|
426
|
-
textus has a single hook verb: `Textus.
|
|
427
|
-
|
|
428
|
-
The subdirectory layout under `hooks/` is organizational only; the registered event and name come from the DSL call, not the file path. Files are loaded in alphabetical order by full path.
|
|
453
|
+
textus has a single hook registration verb: `Textus.on(event, name, **opts) { ... }`. The EVENTS table below defines every extension point. Files in `.textus/hooks/**/*.rb` are `load`ed at `Store#initialize` in alphabetical order by full path.
|
|
429
454
|
|
|
430
|
-
|
|
455
|
+
The subdirectory layout under `hooks/` is organizational only; the registered event and name come from the DSL call, not the file path.
|
|
431
456
|
|
|
432
|
-
|
|
457
|
+
#### Registration DSL
|
|
433
458
|
|
|
434
459
|
```ruby
|
|
435
|
-
|
|
436
|
-
Textus.
|
|
437
|
-
Textus.
|
|
438
|
-
Textus.
|
|
439
|
-
Textus.
|
|
460
|
+
# Canonical form — works for every event:
|
|
461
|
+
Textus.on(:resolve_intake, :my_source) { |config:, args:, **| … }
|
|
462
|
+
Textus.on(:transform_rows, :rank_by_recency) { |rows:, **| … }
|
|
463
|
+
Textus.on(:validate, :storage_writable) { |store:| … }
|
|
464
|
+
Textus.on(:entry_put, :audit, keys: ["working.*"]) { |key:, envelope:, **| … }
|
|
465
|
+
Textus.on(:file_published, :git_add, keys: ["derived.*"]) { |target:, **| `git add #{target.shellescape}` }
|
|
440
466
|
```
|
|
441
467
|
|
|
442
|
-
|
|
443
|
-
|
|
444
|
-
|
|
445
|
-
|
|
446
|
-
|
|
|
447
|
-
|
|
448
|
-
|
|
|
449
|
-
|
|
|
450
|
-
|
|
|
451
|
-
|
|
|
452
|
-
|
|
|
453
|
-
|
|
|
454
|
-
|
|
|
455
|
-
|
|
|
456
|
-
|
|
|
457
|
-
|
|
|
458
|
-
|
|
|
459
|
-
|
|
|
460
|
-
|
|
|
468
|
+
`Textus.on` is the sole entry point; there is no separate `Textus.hook` primitive.
|
|
469
|
+
|
|
470
|
+
#### Event table
|
|
471
|
+
|
|
472
|
+
| Event | Mode | Args | Return | Failure |
|
|
473
|
+
|-------------------------|---------|-----------------------------------------------------------|-----------------------|---------------|
|
|
474
|
+
| `:resolve_intake` | rpc | store:, config:, args: | {_meta:, body:} | aborts op |
|
|
475
|
+
| `:transform_rows` | rpc | store:, rows:, config: | rows array | aborts op |
|
|
476
|
+
| `:validate` | rpc | store: | issues array | aborts doctor |
|
|
477
|
+
| `:entry_put` | pubsub | store:, key:, envelope: | (discarded) | logged |
|
|
478
|
+
| `:entry_deleted` | pubsub | store:, key: | (discarded) | logged |
|
|
479
|
+
| `:entry_refreshed` | pubsub | store:, key:, envelope:, change: | (discarded) | logged |
|
|
480
|
+
| `:build_completed` | pubsub | store:, key:, envelope:, sources: | (discarded) | logged |
|
|
481
|
+
| `:proposal_accepted` | pubsub | store:, key:, target_key: | (discarded) | logged |
|
|
482
|
+
| `:file_published` | pubsub | store:, key:, envelope:, source:, target: | (discarded) | logged |
|
|
483
|
+
| `:entry_renamed` | pubsub | store:, key:, from_key:, to_key:, envelope: | (discarded) | logged |
|
|
484
|
+
| `:proposal_rejected` | pubsub | store:, key:, target_key: | (discarded) | logged |
|
|
485
|
+
| `:store_loaded` | pubsub | store: | (discarded) | logged |
|
|
486
|
+
| `:refresh_started` | pubsub | store:, key:, mode: | (discarded) | logged |
|
|
487
|
+
| `:refresh_failed` | pubsub | store:, key:, error_class:, error_message: | (discarded) | logged |
|
|
488
|
+
| `:refresh_backgrounded` | pubsub | store:, key:, started_at:, budget_ms: | (discarded) | logged |
|
|
461
489
|
|
|
462
490
|
The three `:refresh_*` lifecycle events report the progress and failures of background (timed_sync) refreshes.
|
|
463
491
|
|
|
464
|
-
**`:
|
|
492
|
+
**`:refresh_started`** fires immediately before an intake handler is invoked. `mode:` is one of `"sync"` or `"timed_sync"`.
|
|
465
493
|
|
|
466
494
|
**`:refresh_failed`** fires when an intake handler raises. `error_class:` is the exception class name string; `error_message:` is `e.message`.
|
|
467
495
|
|
|
468
|
-
**`:
|
|
496
|
+
**`:refresh_backgrounded`** fires when a `timed_sync` refresh exceeds its budget and is handed off to a background thread. `started_at:` is an ISO-8601 UTC string; `budget_ms:` is the configured deadline as an integer.
|
|
469
497
|
|
|
470
|
-
**Signature invariant** — every hook receives `store:` as its first keyword argument. Event-specific kwargs follow in stable left-to-right order. The primary entity is always `key:` (for `:
|
|
498
|
+
**Signature invariant** — every hook receives `store:` as its first keyword argument. Event-specific kwargs follow in stable left-to-right order. The primary entity is always `key:` (for `:proposal_accepted`, `key:` is the pending key being accepted and `target_key:` is the destination). For `:entry_renamed`, `key:` is present and equals `to_key:` — it is the entry's post-move home, present so `keys:` glob filters route correctly; `from_key:` is the prior key. For `:proposal_rejected`, `key:` is the pending key being rejected. For `:store_loaded`, no key — the event observes store readiness, not an entry.
|
|
471
499
|
|
|
472
|
-
**RPC mode** — exactly one handler per (event, name). The manifest references the handler by name (`intake.handler: NAME`, `
|
|
500
|
+
**RPC mode** — exactly one handler per (event, name). The manifest references the handler by name (`intake.handler: NAME`, `compute.transform: NAME`). Failure or timeout aborts the calling operation.
|
|
473
501
|
|
|
474
502
|
**Pub-sub mode** — zero or more handlers per event. All matching handlers fire. The `keys:` option restricts a handler to keys matching one of the given globs (`File.fnmatch?` rules). Absence of `keys:` fires on every event of that type. Handler failures and 2s timeouts are logged to `audit.log` as `event_error` rows; they NEVER abort the triggering operation.
|
|
475
503
|
|
|
@@ -477,21 +505,22 @@ The `store:` argument is always a read-only store proxy. Write attempts raise `U
|
|
|
477
505
|
|
|
478
506
|
Each handler runs under `Timeout.timeout(2)`.
|
|
479
507
|
|
|
480
|
-
### 5.11
|
|
508
|
+
### 5.11 Rules
|
|
481
509
|
|
|
482
|
-
A manifest MAY declare a top-level `
|
|
510
|
+
A manifest MAY declare a top-level `rules:` block — a list of rule blocks matched against entry keys by glob. Each block carries one or more slots:
|
|
483
511
|
|
|
484
512
|
```yaml
|
|
485
|
-
|
|
486
|
-
- match:
|
|
513
|
+
rules:
|
|
514
|
+
- match: intake.**
|
|
487
515
|
refresh: { ttl: 6h, on_stale: warn }
|
|
488
516
|
|
|
489
|
-
- match:
|
|
517
|
+
- match: intake.calendar.**
|
|
490
518
|
refresh: { ttl: 30m, on_stale: timed_sync, sync_budget_ms: 800 }
|
|
491
|
-
|
|
519
|
+
intake_handler_allowlist: [ical-events]
|
|
492
520
|
|
|
493
521
|
- match: review.**
|
|
494
|
-
|
|
522
|
+
promotion:
|
|
523
|
+
requires: [schema_valid, human_accept]
|
|
495
524
|
```
|
|
496
525
|
|
|
497
526
|
**Slots (all optional within a block):**
|
|
@@ -499,15 +528,15 @@ policies:
|
|
|
499
528
|
| Slot | Type | Meaning |
|
|
500
529
|
|---|---|---|
|
|
501
530
|
| `refresh` | `{ ttl, on_stale, sync_budget_ms }` | Freshness budget for intake entries. `on_stale` is `warn` (default), `sync`, or `timed_sync`. |
|
|
502
|
-
| `
|
|
503
|
-
| `
|
|
531
|
+
| `intake_handler_allowlist` | list of strings | Constrains which `intake.handler:` names may be used by entries matched by this block. Enforced by `textus doctor`. |
|
|
532
|
+
| `promotion` | `{ requires: [...] }` | Predicates a `review` entry must satisfy before `textus accept` will promote it. Built-in predicates: `schema_valid` (entry passes schema validation) and `human_accept` (the accepting role must be `human`). Additional predicates may be registered via `:validate` hooks. Enforced — `textus accept` refuses if any predicate fails. |
|
|
504
533
|
| `retention` | (reserved) | Slot reserved for future retention policy (cap by age / count). Implementations parse it but otherwise ignore. |
|
|
505
534
|
|
|
506
535
|
**Match grammar.** `match:` is a single glob using `*` (single segment) and `**` (any depth). A literal segment ranks more specifically than `*`; `*` ranks more specifically than `**`.
|
|
507
536
|
|
|
508
|
-
**Resolution.** For each key textus computes a `
|
|
537
|
+
**Resolution.** For each key textus computes a `RuleSet { refresh, intake_handler_allowlist, promotion, retention }` by walking every block whose `match` matches the key, ranked by specificity. **Per slot, the most specific block wins.** Two blocks of equal specificity that match the same key and fill the same slot is a manifest error reported by `textus doctor` (`rule_ambiguity`).
|
|
509
538
|
|
|
510
|
-
**Read surface.** `textus
|
|
539
|
+
**Read surface.** `textus rule list` dumps every block. `textus rule explain KEY` shows the resolved `RuleSet` for one key plus which block won each slot.
|
|
511
540
|
|
|
512
541
|
### 5.12 Storage formats
|
|
513
542
|
|
|
@@ -523,7 +552,7 @@ An entry's `format:` selects a storage strategy. All strategies expose the same
|
|
|
523
552
|
**`_meta` convention.** Derived structured entries (json, yaml) embed a `_meta` hash as the first top-level key. Builder-injected keys appear in a fixed order for etag stability:
|
|
524
553
|
|
|
525
554
|
```
|
|
526
|
-
generated_at, from, template,
|
|
555
|
+
generated_at, from, template, transform
|
|
527
556
|
```
|
|
528
557
|
|
|
529
558
|
Keys with `nil` values are omitted. User-shaped content (or the reducer's hash) follows `_meta`. The etag (§10) is the sha256 of the on-disk bytes regardless of format; key ordering MUST therefore be deterministic, which Ruby's `Hash` and `JSON.generate` / `YAML.dump` honor via insertion order.
|
|
@@ -581,11 +610,11 @@ Entries in `zone: derived` SHOULD additionally carry the `generated:` block defi
|
|
|
581
610
|
|
|
582
611
|
## 8. Envelope (the wire format)
|
|
583
612
|
|
|
584
|
-
Every successful CLI response (`--
|
|
613
|
+
Every successful CLI response (`--output=json`) is a single JSON envelope:
|
|
585
614
|
|
|
586
615
|
```json
|
|
587
616
|
{
|
|
588
|
-
"protocol": "textus/
|
|
617
|
+
"protocol": "textus/3",
|
|
589
618
|
"key": "working.network.org.jane",
|
|
590
619
|
"zone": "working",
|
|
591
620
|
"owner": "textus:network",
|
|
@@ -603,9 +632,9 @@ Every successful CLI response (`--format=json`) is a single JSON envelope:
|
|
|
603
632
|
```
|
|
604
633
|
|
|
605
634
|
**Field rules:**
|
|
606
|
-
- `protocol` MUST be the exact string `textus/
|
|
635
|
+
- `protocol` MUST be the exact string `textus/3`.
|
|
607
636
|
- `key` MUST be the canonical resolved key.
|
|
608
|
-
- `zone` MUST be one of the zones declared in the manifest (`identity`, `working`, `
|
|
637
|
+
- `zone` MUST be one of the zones declared in the manifest (`identity`, `working`, `intake`, `review`, `output` in the default scaffold).
|
|
609
638
|
- `path` MUST be an absolute filesystem path.
|
|
610
639
|
- `format` MUST be one of `markdown`, `json`, `yaml`, `text` (§5.12). Absent envelopes are treated as `markdown` for back-compat.
|
|
611
640
|
- `body` is the raw on-disk bytes as a UTF-8 string for every format.
|
|
@@ -613,7 +642,7 @@ Every successful CLI response (`--format=json`) is a single JSON envelope:
|
|
|
613
642
|
- `etag` MUST be `sha256:<hex>` of the raw file bytes, computed identically for every format.
|
|
614
643
|
- `schema_ref` MAY be `null` for entries in subtrees with `schema: null`.
|
|
615
644
|
- `uid` is the stable Textus UID (§7) if the entry carries one, else `null`. Always present in the envelope.
|
|
616
|
-
- `stale` is `true` when the entry's TTL has elapsed and the data has not yet been refreshed; `false` otherwise. Only populated for entries matched by a `refresh:`
|
|
645
|
+
- `stale` is `true` when the entry's TTL has elapsed and the data has not yet been refreshed; `false` otherwise. Only populated for entries matched by a `refresh:` rule slot (typically `intake` zone); always `false` elsewhere.
|
|
617
646
|
- `stale_reason` is a short human-readable string describing why the entry is stale (e.g. `"ttl_exceeded"`, `"never_refreshed"`), or `null` when `stale` is `false`.
|
|
618
647
|
- `refreshing` is `true` when a `timed_sync` background refresh is in flight for this entry; `false` otherwise. Callers observing `stale: true, refreshing: true` SHOULD retry after a short delay.
|
|
619
648
|
|
|
@@ -623,11 +652,11 @@ Errors use a distinct envelope:
|
|
|
623
652
|
|
|
624
653
|
```json
|
|
625
654
|
{
|
|
626
|
-
"protocol": "textus/
|
|
655
|
+
"protocol": "textus/3",
|
|
627
656
|
"ok": false,
|
|
628
657
|
"code": "write_forbidden",
|
|
629
|
-
"message": "zone 'identity' is not writable by role '
|
|
630
|
-
"details": { "key": "identity.self", "zone": "identity", "role": "
|
|
658
|
+
"message": "zone 'identity' is not writable by role 'agent' for key 'identity.self'",
|
|
659
|
+
"details": { "key": "identity.self", "zone": "identity", "role": "agent" }
|
|
631
660
|
}
|
|
632
661
|
```
|
|
633
662
|
|
|
@@ -638,7 +667,7 @@ Errors use a distinct envelope:
|
|
|
638
667
|
| `unknown_key` | Key does not resolve | 1 |
|
|
639
668
|
| `bad_frontmatter` | YAML parse failed or `name:` mismatch | 1 |
|
|
640
669
|
| `schema_violation` | Required field missing or wrong type | 1 |
|
|
641
|
-
| `write_forbidden` | Resolved role is not in the zone's `
|
|
670
|
+
| `write_forbidden` | Resolved role is not in the zone's `write_policy` | 1 |
|
|
642
671
|
| `etag_mismatch` | Concurrent write detected | 1 |
|
|
643
672
|
| `io_error` | Filesystem failure | 64 |
|
|
644
673
|
| `usage` | CLI argument error | 2 |
|
|
@@ -647,7 +676,7 @@ Errors use a distinct envelope:
|
|
|
647
676
|
|
|
648
677
|
The reference binary is `textus`. Conforming implementations MAY use any binary name; the protocol is in the JSON.
|
|
649
678
|
|
|
650
|
-
All verbs accept `--
|
|
679
|
+
All verbs accept `--output=json` and emit a canonical envelope (success or error). Write verbs require `--as=<role>`; the role must satisfy the target zone's write gate (§5). The per-entry `format:` field in the manifest is unchanged — `--output` controls only the CLI envelope rendering.
|
|
651
680
|
|
|
652
681
|
| Verb | Reads / writes | Role required |
|
|
653
682
|
|---|---|---|
|
|
@@ -658,24 +687,24 @@ All verbs accept `--format=json` and emit a canonical envelope (success or error
|
|
|
658
687
|
| `freshness [--prefix=K] [--zone=Z]` | read | any |
|
|
659
688
|
| `audit [--key=K] [--zone=Z] [--role=R] [--verb=V] [--since=X] [--correlation-id=ID] [--limit=N]` | read | any |
|
|
660
689
|
| `blame KEY` | read | any |
|
|
661
|
-
| `
|
|
690
|
+
| `rule list` / `rule explain KEY` | read | any |
|
|
662
691
|
| `deps K` / `rdeps K` | read | any |
|
|
663
692
|
| `published` | read | any |
|
|
664
693
|
| `hook list` | read | any |
|
|
665
|
-
| `
|
|
666
|
-
| `
|
|
694
|
+
| `hook run NAME` | write | any |
|
|
695
|
+
| `doctor [--check=NAME[,NAME]] [--output=json]` | read | any |
|
|
696
|
+
| `intro [--output=json]` | read | any |
|
|
667
697
|
| `put K --stdin --as=R [--fetch=NAME]` | write | per zone |
|
|
668
698
|
| `delete K --if-etag=E --as=R` | write | per zone |
|
|
669
|
-
| `refresh
|
|
670
|
-
| `refresh
|
|
671
|
-
| `build [--prefix=K] [--dry-run]` | write | `
|
|
699
|
+
| `refresh KEY --as=runner` | write | per zone (typically `runner`) |
|
|
700
|
+
| `refresh stale [--prefix=K] [--zone=Z] [--as=runner]` | write | per zone (typically `runner`) |
|
|
701
|
+
| `build [--prefix=K] [--dry-run]` | write | `builder` (default) |
|
|
672
702
|
| `accept K --as=human` | write | `human` |
|
|
673
703
|
| `init` | write | `human` |
|
|
674
|
-
| `schema init
|
|
675
|
-
| `key migrate [--dry-run\|--write]` | write (with `--write`) | `human` |
|
|
704
|
+
| `schema {show,init,diff,migrate}` | read/write | `human` for writes |
|
|
676
705
|
| `key mv OLD NEW [--as=R] [--dry-run]` | write | per zone (same-zone only) |
|
|
706
|
+
| `key normalize [--dry-run\|--write]` | write (with `--write`) | `human` |
|
|
677
707
|
| `key uid K` | read | any |
|
|
678
|
-
| `hook run NAME` | write | any |
|
|
679
708
|
|
|
680
709
|
**`put` input** (read from stdin when `--stdin` is given):
|
|
681
710
|
|
|
@@ -693,8 +722,8 @@ All verbs accept `--format=json` and emit a canonical envelope (success or error
|
|
|
693
722
|
{
|
|
694
723
|
"verb": "freshness",
|
|
695
724
|
"rows": [
|
|
696
|
-
{ "key": "
|
|
697
|
-
"zone": "
|
|
725
|
+
{ "key": "intake.upstream.notes",
|
|
726
|
+
"zone": "intake",
|
|
698
727
|
"last_refreshed_at": "2026-05-21T13:21:17Z",
|
|
699
728
|
"age_seconds": 65000,
|
|
700
729
|
"ttl_seconds": 43200,
|
|
@@ -705,7 +734,7 @@ All verbs accept `--format=json` and emit a canonical envelope (success or error
|
|
|
705
734
|
}
|
|
706
735
|
```
|
|
707
736
|
|
|
708
|
-
Each row reports one entry's verdict (`fresh`, `stale`, `never_refreshed`, or `no_policy`) against its matched `refresh:`
|
|
737
|
+
Each row reports one entry's verdict (`fresh`, `stale`, `never_refreshed`, or `no_policy`) against its matched `refresh:` rule. `textus build` consumes its own staleness signal and executes derived entries' projections under the `builder` role; `--dry-run` prints the plan without executing.
|
|
709
738
|
|
|
710
739
|
`textus accept K --as=human` promotes a pending entry into its target zone: it copies the patch body into the target key, deletes the pending entry, and writes one audit line per side (§audit). Only the `human` role may invoke `accept`.
|
|
711
740
|
|
|
@@ -723,35 +752,35 @@ Every `Textus::Error` exposes `code`, `message`, and an optional `hint:`. The hi
|
|
|
723
752
|
|
|
724
753
|
## 10.2 `textus doctor`
|
|
725
754
|
|
|
726
|
-
`textus doctor` returns a health-check envelope: `{ "protocol": "textus/
|
|
755
|
+
`textus doctor` returns a health-check envelope: `{ "protocol": "textus/3", "ok": bool, "issues": [...], "summary": {error, warning, info} }`. Each issue carries `code`, `level` (`error|warning|info`), `subject`, `message`, and optionally `fix`. `ok` is true iff no error-level issues are present; warnings and info do not flip the bit. Builtin checks: `manifest_files`, `schemas`, `schema_parse_error`, `templates`, `hooks`, `illegal_keys`, `sentinels`, `audit_log`, `unowned_schema_fields`, `schema_violations`, `rule_ambiguity`, `intake_handler_allowlist`. Additional registered `:validate` hooks (§5.10) run after the builtin set. Exit code is 0 on `ok`, 1 otherwise.
|
|
727
756
|
|
|
728
757
|
## 11. Versioning
|
|
729
758
|
|
|
730
|
-
- The current wire string is `textus/
|
|
731
|
-
- Backward-compatible additions (new fields, new error codes, new schema types) MAY be made under `textus/
|
|
732
|
-
- Breaking changes (renamed/removed envelope fields, zone semantics, key grammar) require a new wire string `textus/
|
|
759
|
+
- The current wire string is `textus/3`.
|
|
760
|
+
- Backward-compatible additions (new fields, new error codes, new schema types) MAY be made under `textus/3`.
|
|
761
|
+
- Breaking changes (renamed/removed envelope fields, zone semantics, key grammar) require a new wire string `textus/4`.
|
|
733
762
|
- Implementations MUST reject envelopes whose `protocol` they do not recognize.
|
|
734
763
|
|
|
735
|
-
The reference Ruby gem follows semver independently and speaks `textus/
|
|
764
|
+
The reference Ruby gem follows semver independently and speaks `textus/3`.
|
|
736
765
|
|
|
737
766
|
## 12. Conformance fixtures
|
|
738
767
|
|
|
739
768
|
A conformant implementation MUST pass these fixtures (the reference test suite ships a YAML file listing inputs and expected envelopes):
|
|
740
769
|
|
|
741
770
|
**Fixture A — Resolve and read:**
|
|
742
|
-
Given a manifest with `working.network.org` → `working/network/org` (nested), schema `person`, and a file `.textus/zones/working/network/org/jane.md` with valid frontmatter, `textus get working.network.org.jane --
|
|
771
|
+
Given a manifest with `working.network.org` → `working/network/org` (nested), schema `person`, and a file `.textus/zones/working/network/org/jane.md` with valid frontmatter, `textus get working.network.org.jane --output=json` returns the canonical envelope with `etag` matching the file's sha256.
|
|
743
772
|
|
|
744
773
|
**Fixture B — Role gate on write:**
|
|
745
|
-
Given a manifest entry where `key: identity.self` lives in the `identity` zone (human-only), `textus put identity.self --stdin --as=
|
|
774
|
+
Given a manifest entry where `key: identity.self` lives in the `identity` zone (human-only), `textus put identity.self --stdin --as=agent` (with any valid input) returns the error envelope with `code: "write_forbidden"` and exit code 1.
|
|
746
775
|
|
|
747
776
|
**Fixture C — Schema violation:**
|
|
748
777
|
Given the `person` schema and a `put` whose frontmatter omits `relationship`, the result is the error envelope with `code: "schema_violation"`, `details.missing: ["relationship"]`, and exit code 1.
|
|
749
778
|
|
|
750
779
|
**Fixture D — Staleness detection:**
|
|
751
|
-
Given a manifest entry `
|
|
780
|
+
Given a manifest entry `intake.notes` matched by a `rules: [{ match: intake.notes, refresh: { ttl: 1h } }]` block and an envelope on disk whose `_meta.last_refreshed_at` is older than `now - ttl`, `textus freshness --output=json` includes a row for `intake.notes` with `status: "stale"`. Calling `textus freshness` does NOT trigger a refresh.
|
|
752
781
|
|
|
753
782
|
**Fixture E — Projection build:**
|
|
754
|
-
Given a manifest entry `derived.catalogs.skills` whose `projection` clause selects fields from `working.projects` entries, `textus build derived.catalogs.skills` materializes the derived entry on disk with frontmatter and body matching the projected shape, and updates `generated.at` to the build timestamp.
|
|
783
|
+
Given a manifest entry `derived.catalogs.skills` whose `compute: { kind: projection }` clause selects fields from `working.projects` entries, `textus build derived.catalogs.skills` materializes the derived entry on disk with frontmatter and body matching the projected shape, and updates `generated.at` to the build timestamp.
|
|
755
784
|
|
|
756
785
|
**Fixture F — Mustache render:**
|
|
757
786
|
Given a derived entry with a `template` clause referencing a `.mustache` file and inputs drawn from other keys, `textus build` produces a body whose contents match the expected rendered output byte-for-byte (after trailing-newline normalization).
|
|
@@ -769,13 +798,13 @@ Given a review entry `review.identity.self.patch` proposing a change to `identit
|
|
|
769
798
|
|
|
770
799
|
- **Why not MCP?** MCP is a transport; textus is a data model. The two compose: a 50-line MCP server can wrap `textus get/put` as tools. textus exists because the *shape* of agent-readable project memory deserves a standalone spec, separate from how it's served.
|
|
771
800
|
|
|
772
|
-
- **Why doesn't textus execute generator commands itself?** textus is a dataflow oracle, not a build runner. The moment a spec includes process execution, it inherits shell-injection surface, OS-portability concerns, and signal-handling semantics — and ends up duplicating whatever build system the consumer already runs (make, rake, just, lefthook, CI). Keeping execution external means a Python or TypeScript port of `textus/
|
|
801
|
+
- **Why doesn't textus execute generator commands itself?** textus is a dataflow oracle, not a build runner. The moment a spec includes process execution, it inherits shell-injection surface, OS-portability concerns, and signal-handling semantics — and ends up duplicating whatever build system the consumer already runs (make, rake, just, lefthook, CI). Keeping execution external means a Python or TypeScript port of `textus/3` only has to parse YAML and emit JSON; it doesn't have to spawn processes safely. Build runners stay the executor; textus stays a data tool.
|
|
773
802
|
|
|
774
803
|
- **Why not plain Markdown vaults (Obsidian / Foam)?** No schema enforcement, no write-gating, no addressable wire format. Fine for human notes; underspecified for agents that must act on the contents deterministically.
|
|
775
804
|
|
|
776
805
|
- **Why not Notion / Coda?** Closed, hosted, lossy export. textus is local-first, plain-files, diffable in git.
|
|
777
806
|
|
|
778
|
-
- **Why not JSON Schema for the schemas?** Considered. Bespoke YAML chosen for v1: simpler implementation, lighter dependency footprint, matches the reference impl's house language. JSON Schema MAY be added as an alternate schema-language adapter in a future minor revision without breaking `textus/
|
|
807
|
+
- **Why not JSON Schema for the schemas?** Considered. Bespoke YAML chosen for v1: simpler implementation, lighter dependency footprint, matches the reference impl's house language. JSON Schema MAY be added as an alternate schema-language adapter in a future minor revision without breaking `textus/3`.
|
|
779
808
|
|
|
780
809
|
- **Why not a database (SQLite, kv store)?** textus's whole point is that the storage is plain files agents and humans both read. A binary store loses git-diff, grep, and editor support.
|
|
781
810
|
|
|
@@ -792,18 +821,21 @@ Textus internals are organized into four layers. The dependency rule is one-way
|
|
|
792
821
|
|
|
793
822
|
The `lib/textus/store/`, `lib/textus/manifest/`, `lib/textus/hooks/` namespaces are infrastructure adapters that predate this split and remain at their existing paths for backward-compat with the plugin DSL.
|
|
794
823
|
|
|
795
|
-
Plugin authors interact only with the Hook DSL (`Textus.
|
|
824
|
+
Plugin authors interact only with the Hook DSL (`Textus.on(:resolve_intake, ...)`, `Textus.on(:entry_refreshed, ...)`, etc.) and the manifest YAML schema. The layering is internal and may evolve.
|
|
796
825
|
|
|
797
826
|
Both read and write paths flow through the application layer:
|
|
798
827
|
|
|
799
828
|
- **Reads** flow through `Application::Reads::Get`, which takes a `Context` and dispatches refresh via `Application::Refresh::Orchestrator`.
|
|
800
829
|
- **Writes** flow through `Application::Writes::{Put,Delete,Build,Accept,Publish}`, each taking a `Context`. Permission checks happen at the use-case layer (via `Context#can_write?`); I/O happens at `Store::Writer#write_envelope_to_disk` (pure).
|
|
801
830
|
- `Application::Context` is the universal request object: it carries `store`, `role`, `correlation_id`, `clock`, and `dry_run`. Use cases never thread these as separate kwargs.
|
|
802
|
-
- `Textus::
|
|
831
|
+
- `Textus::Operations` is the factory CLI verbs (and future MCP server / HTTP shim)
|
|
832
|
+
use to construct Contexts and use cases. `Operations.for(store, role:)` returns
|
|
833
|
+
a memoized facade with `.reads`, `.writes`, and `.refresh` namespaces mirroring the
|
|
834
|
+
files under `lib/textus/application/{reads,writes,refresh}/`.
|
|
803
835
|
|
|
804
836
|
See `ARCHITECTURE.md` for an ASCII diagram and the full read-path walkthrough.
|
|
805
837
|
|
|
806
|
-
## 14. Open questions (
|
|
838
|
+
## 14. Open questions (v3.x scope)
|
|
807
839
|
|
|
808
840
|
- **Locking on `put`:** the reference impl uses sha256 etags. Should the spec also define a file-lock fallback for systems where read-before-write is racy?
|
|
809
841
|
- **Schema imports:** can one schema reference another (`type: $ref: person`)?
|
|
@@ -812,26 +844,75 @@ See `ARCHITECTURE.md` for an ASCII diagram and the full read-path walkthrough.
|
|
|
812
844
|
|
|
813
845
|
## 15. Implementation checklist
|
|
814
846
|
|
|
815
|
-
A `textus/
|
|
847
|
+
A `textus/3` implementation MUST:
|
|
816
848
|
|
|
817
|
-
- [ ] Parse `.textus/manifest.yaml` and accept `version: textus/
|
|
849
|
+
- [ ] Parse `.textus/manifest.yaml` and accept `version: textus/3`.
|
|
818
850
|
- [ ] Resolve keys via longest-prefix match against manifest entries.
|
|
819
851
|
- [ ] Read `_meta` + body from `.md` files; validate against the named schema.
|
|
820
852
|
- [ ] Read `_meta` from the top-level `_meta` hash in `.json` / `.yaml` files; validate against the named schema.
|
|
821
853
|
- [ ] Compute `sha256:<hex>` etags over raw file bytes.
|
|
822
|
-
- [ ] Refuse writes whose resolved role is not in the target zone's `
|
|
854
|
+
- [ ] Refuse writes whose resolved role is not in the target zone's `write_policy` list with `write_forbidden`.
|
|
823
855
|
- [ ] Return envelopes matching the shape in §8 exactly (with `_meta`, not `frontmatter`).
|
|
824
856
|
- [ ] Use the error codes in §8 and the exit-code table.
|
|
825
|
-
- [ ] Implement `textus freshness` per §5.1 and §9, walking each entry, matching it against the top-level `
|
|
857
|
+
- [ ] Implement `textus freshness` per §5.1 and §9, walking each entry, matching it against the top-level `rules:` block, and reporting `fresh|stale|never_refreshed|no_policy` without invoking any refresh.
|
|
826
858
|
- [ ] Pass the conformance fixtures A–I in §12.
|
|
827
859
|
|
|
828
|
-
A `textus/
|
|
860
|
+
A `textus/3` implementation MAY:
|
|
829
861
|
|
|
830
862
|
- Add additional CLI verbs (e.g. `move`, vendor-specific reporters) beyond the current set in §9.
|
|
831
|
-
- Provide alternate output formats (`--
|
|
863
|
+
- Provide alternate output formats (`--output=yaml`, `--output=table`) for human use.
|
|
832
864
|
- Support additional schema field types beyond §6, marked as `vendor:<name>` extensions.
|
|
833
865
|
|
|
866
|
+
## 16. Migrating from textus/2
|
|
867
|
+
|
|
868
|
+
textus 0.12.0 does not ship a built-in migrator. Upgrade path:
|
|
869
|
+
|
|
870
|
+
1. Install textus **0.11.x** first.
|
|
871
|
+
2. Run `textus migrate --to=textus/3` (available in 0.11.x only). This rewrites `manifest.yaml`, renames the `inbox/` zone directory to `intake/`, sweeps frontmatter `owner:` fields, writes an audit-log marker, and reports legacy hook-DSL call sites for manual review.
|
|
872
|
+
3. Upgrade to textus **0.12.0**.
|
|
873
|
+
4. If `.textus/audit.log` contains pre-0.11.0 rows with `role: ai|script|build`, run `textus audit-rewrite-legacy-roles` once (one-shot verb; removed in 0.13.0).
|
|
874
|
+
|
|
875
|
+
**textus doctor refuses textus/2 stores.** The doctor check `protocol_version` emits an `error`-level issue when `manifest.yaml` carries `version: textus/2`. Install 0.11.x and migrate before upgrading to 0.12.0.
|
|
876
|
+
|
|
877
|
+
**Vocabulary summary** (textus/2 → textus/3 rename table, for reference):
|
|
878
|
+
|
|
879
|
+
| Category | textus/2 | textus/3 |
|
|
880
|
+
|---|---|---|
|
|
881
|
+
| Actor | `ai` | `agent` |
|
|
882
|
+
| Actor | `script` | `runner` |
|
|
883
|
+
| Actor | `build` | `builder` |
|
|
884
|
+
| Zone | `inbox` | `intake` |
|
|
885
|
+
| Manifest | `writable_by:` | `write_policy:` |
|
|
886
|
+
| Manifest | `policies:` | `rules:` |
|
|
887
|
+
| Manifest | `handler_allowlist:` | `intake_handler_allowlist:` |
|
|
888
|
+
| Manifest | `promote_requires:` | `promotion: { requires: [...] }` |
|
|
889
|
+
| Manifest | `projection:` | `compute: { kind: projection, ... }` |
|
|
890
|
+
| Manifest | `generator:` | `compute: { kind: external, ... }` |
|
|
891
|
+
| Hook event | `:intake` | `:resolve_intake` |
|
|
892
|
+
| Hook event | `:reduce` | `:transform_rows` |
|
|
893
|
+
| Hook event | `:check` | `:validate` |
|
|
894
|
+
| Hook event | `:put` | `:entry_put` |
|
|
895
|
+
| Hook event | `:deleted` | `:entry_deleted` |
|
|
896
|
+
| Hook event | `:refreshed` | `:entry_refreshed` |
|
|
897
|
+
| Hook event | `:built` | `:build_completed` |
|
|
898
|
+
| Hook event | `:accepted` | `:proposal_accepted` |
|
|
899
|
+
| Hook event | `:reject` | `:proposal_rejected` |
|
|
900
|
+
| Hook event | `:published` | `:file_published` |
|
|
901
|
+
| Hook event | `:mv` | `:entry_renamed` |
|
|
902
|
+
| Hook event | `:loaded` | `:store_loaded` |
|
|
903
|
+
| Hook event | `:refresh_began` | `:refresh_started` |
|
|
904
|
+
| Hook event | `:refresh_detached` | `:refresh_backgrounded` |
|
|
905
|
+
| Hook event | `:refresh_failed` | `:refresh_failed` (unchanged) |
|
|
906
|
+
| Hook DSL | `Textus.hook(ev, name)` / sugar | `Textus.on(ev, name)` |
|
|
907
|
+
| Compute field | `projection.reduce:` | `compute.transform:` |
|
|
908
|
+
| `_meta` key | `reducer` | `transform` |
|
|
909
|
+
| CLI flag | `--format=json` (envelope) | `--output=json` |
|
|
910
|
+
| CLI verb | `refresh-stale` | `refresh stale` |
|
|
911
|
+
| CLI verb | `policy list/explain` | `rule list/explain` |
|
|
912
|
+
|
|
913
|
+
**Notes on hook migration.** The 0.11.x migrator scanner reports each file and call site that uses a legacy event name or DSL method. No automatic rewrite is performed. Update each hook to use `Textus.on(:new_event_name, ...)` before re-enabling the hook. See CHANGELOG §0.11.0 for the full event rename table.
|
|
914
|
+
|
|
834
915
|
---
|
|
835
916
|
|
|
836
|
-
**Spec word count target:** <
|
|
837
|
-
**Reviewed against community-testing checklist (idea file §"Community-testing"):** ✅
|
|
917
|
+
**Spec word count target:** <2700 words (allowance widened to fit vocabulary axes intro + migration section).
|
|
918
|
+
**Reviewed against community-testing checklist (idea file §"Community-testing"):** ✅ implementable in a day in TS/Python (four concepts: manifest, schema, envelope, staleness check); ✅ conformance fixtures A–I; ✅ "Why not X?" section present (incl. why no execution); ✅ name picked.
|