textus 0.54.0 → 0.54.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 0f98dbc5f461b8cffaa4cf8fb17c8b79cf51f4c5f78af62527e0abe27d84d828
4
- data.tar.gz: 0bfd928765a9797078f7534aecbf89321de754a4d56ee71986c00fb6a3a7b255
3
+ metadata.gz: 53f01828833f9c5e6cb0249fe513b92d6afbb210d90cfca10f51c560c8b2e2ca
4
+ data.tar.gz: 1956232435c941479290225e20168979beb270a6e2f5874df37c302e170a43e7
5
5
  SHA512:
6
- metadata.gz: 32fd23741a0f3e69368dfde1fb8733381190b17eb592c459079020a692c98c181a689ab5a83af6b3bf2d16c37ab2b52e40faf6a47ce876e4bafc0709e082ebed
7
- data.tar.gz: daadc9db41cf87e2a348ddf9b4472fc515a48b4f5e7ca77d9c00a6c95ed736d26c59b3ad1ec799a42e3eba940052236dc77e5ee3edaf4737150e8f030a5fbcaa
6
+ metadata.gz: c086755b90faab7ecebdeebd722e9458dbede54c5efa9be212d6d861c999655093166a1f61826aa9e09ebee8cd1cf57996d958bd410c3b682763d911ed44abe5
7
+ data.tar.gz: aad04f9470c3de178ad6c762ef3102528cbdc94922fedb79d121ffbadc23acc5dd3f52871f6b3c0c75664afdb968c12fa814b88fbedcbd0d914161e4e1f337ab
data/CHANGELOG.md CHANGED
@@ -9,6 +9,37 @@ The **gem version** (`0.x.y`) is distinct from the **protocol version**
9
9
  bump is a breaking change that requires a store migration; the gem version
10
10
  tracks both additive improvements and breaking protocol bumps independently.
11
11
 
12
+ ## 0.54.2 — 2026-06-18 — ingest dedup, .run → .state rename
13
+
14
+ ### Added
15
+
16
+ - **Content-hash dedup in `Action::Ingest`**: re-ingesting the same URL or file bytes silently creates a supersede chain — new entry with full content, old stripped to tombstone with a `superseded_by` pointer.
17
+ - **`Ports::RawIndex`**: lightweight YAML-based O(1) index for raw lane content-hash/URL lookups, stored at `.textus/.state/indexes/raw.yaml`.
18
+ - **`content_hash` field**: every new raw entry stores a `sha256:` content hash for fast dedup lookups.
19
+ - **Asset file move on re-ingest**: asset files are moved (not copied) to the new date path on supersede.
20
+
21
+ ### Changed
22
+
23
+ - **`.run` → `.state` rename**: the runtime artifact directory is renamed from `.textus/.run/` to `.textus/.state/`. The `state/` subdirectory becomes `cursors/`. Layout path methods follow automatically. Manual migration step: `mv .textus/.run .textus/.state`.
24
+
25
+ ## 0.54.1 — 2026-06-17 — MCP SDK, dry ecosystem, ERB templates
26
+
27
+ ### Added
28
+
29
+ - **`Textus::Types` module**: shared `dry-types` constrained types (`RoleName`, `Cursor`, `FormatName`) used across all dry-struct structs.
30
+
31
+ ### Changed
32
+
33
+ - **MCP surface → official `mcp` SDK** (PR #223): the hand-rolled JSON-RPC 2.0 stdio loop is replaced by the official `mcp` gem (`~> 0.20`). Protocol version auto-negotiates up to `2025-11-25` (was hardcoded `2024-11-05`). `Catalog.build_tools` generates `MCP::Tool` instances from verb contracts; `Server` manages session lifecycle and delegates dispatch. `routing.rb`, the `MCP::Session` alias, and `ToolSchemas` are deleted.
34
+ - **`Session`, `Envelope`, `Container` → `Dry::Struct`** (PR #223): `Data.define` replaced with `Dry::Struct` for typed, loudly-failing construction at object creation time. `Session` enforces `RoleName` and non-negative `Cursor`; `Envelope` enforces known `FormatName`; `Container` has typed port references.
35
+ - **Manifest validator → dry-schema + Semantics** (PR #223): the 318-line programmatic validator is split into `contract.rb` (dry-schema structural contract) and `semantics.rb` (cross-field rules, `walk`-based unknown-key detection, ADR migration hints). Public interface `Validator.validate!` unchanged.
36
+ - **Mustache → ERB** (PR #223): `mustache` gem removed; all 8 `.mustache` templates converted to `.erb` using `ERB#result_with_hash`. `Produce::Render` now uses `ERB.new(template, trim_mode: "-").result_with_hash(ctx)`.
37
+
38
+ ### Removed
39
+
40
+ - `mustache` runtime dependency (replaced by Ruby stdlib ERB).
41
+ - `MCP::Routing` module, `MCP::Session` alias, `MCP::ToolSchemas` module — superseded by the SDK.
42
+
12
43
  ## 0.54.0 — 2026-06-16 — Workflow system, raw ingest pipeline, MCP resources
13
44
 
14
45
  ### Added
@@ -388,7 +419,7 @@ No `textus/3` wire-format change. Manifest schema gains one optional, backward-c
388
419
  segment-wise globstar — `**` spans zero or more path segments; within a
389
420
  segment `*` is anchored and `{a,b}` alternates (stdlib `File.fnmatch`,
390
421
  no new dependency). Documented in
391
- [`docs/reference/zones.md`](docs/reference/zones.md#nested-entries).
422
+ [`docs/reference/zones.md`](docs/reference/lanes.md#nested-entries).
392
423
 
393
424
  ### Internal
394
425
 
data/README.md CHANGED
@@ -39,6 +39,9 @@ flowchart LR
39
39
  agent -->|keep| notebook["notebook<br/>(workspace)"]
40
40
  agent -->|propose| proposals["proposals<br/>(queue)"]
41
41
  automation -->|drain| artifacts["artifacts<br/>(machine)"]
42
+ human -->|ingest| raw["raw<br/>(intake)"]
43
+ agent -->|ingest| raw
44
+ automation -->|ingest| raw
42
45
 
43
46
  proposals ==>|human accept| knowledge
44
47
  knowledge -.->|projection source| artifacts
@@ -46,13 +49,15 @@ flowchart LR
46
49
  classDef actor fill:#238636,stroke:#2ea043,color:#fff;
47
50
  classDef gate fill:#9e6a03,stroke:#bb8009,color:#fff;
48
51
  classDef anchor fill:#1f6feb,stroke:#388bfd,color:#fff;
52
+ classDef intake fill:#6e40c9,stroke:#8b5cf6,color:#fff;
49
53
  class human,agent,automation actor;
50
54
  class proposals gate;
51
55
  class knowledge anchor;
56
+ class raw intake;
52
57
  ```
53
58
 
54
59
  *Each actor writes only into its own lane; low-trust input climbs to authoritative lanes only by passing a guarded transition (an agent's proposal needs a human `accept`).*
55
- *Colour legend: **green** = writers · **amber** = the review gate (`proposals`) · **blue** = the trust anchor (`knowledge`).*
60
+ *Colour legend: **green** = writers · **amber** = the review gate (`proposals`) · **blue** = the trust anchor (`knowledge`) · **purple** = write-once intake (`raw`).*
56
61
 
57
62
  The point of those lanes is to **build context you can trust**. Place each lane on two axes — how durable it is, and how much you can rely on it without review — and the value shows up as a climb: the high-trust corner (durable *and* authoritative = `knowledge`) is the one place nothing is *written* directly. It's *earned* by crossing the `accept` gate.
58
63
 
@@ -64,22 +69,21 @@ DURABLE │ notebook │ knowledge ★ the goal
64
69
  (kept) │ agent's working truth │ canon — a human authors │
65
70
  │ durable, but low-trust │ here · the context you ship │
66
71
  ├──────────────────────────┼───────────────────────────────┤
67
- TRANSIENT │ artifacts.feeds.* │ proposals (queue) │
68
- (staging) │ raw external input, │ a candidate, in review │
69
- unverified (machine) │ ▲ climbs via human accept │
72
+ TRANSIENT │ artifacts (outputs) │ proposals (queue) │
73
+ (staging) │ computed, machine-made │ a candidate, in review │
74
+ raw (inputs) │ ▲ climbs via human accept │
75
+ │ ingested, write-once │ │
70
76
  └──────────────────────────┴───────────────────────────────┘
71
77
  raw material ──── propose ────► a human accept lifts it to canon
72
78
  ```
73
79
 
74
- *(The `machine` lane's other half, `artifacts.derived.*`, isn't on this grid — it's a computed **output** projected from the lanes, not an input climbing toward trust.)*
75
-
76
80
  Without coordination, they overwrite each other and nothing remembers why. textus gives each actor a **lane** — enforced at the protocol level, not by convention — routes everything they can't write directly through a **proposals queue**, and writes every successful change to an **append-only audit log**.
77
81
 
78
82
  ```
79
- knowledge/ author only — who you are, what you decide, how you sound (knowledge.identity.* for identity facts)
80
- notebook/ keep only — agent's own durable lane (agents keep theirs; bytes climb to knowledge only via propose→accept)
81
- proposals/ propose (agent + human) — proposals waiting on a human accept
82
- artifacts/ converge only — machine-maintained: external inputs (artifacts.feeds.*) + computed outputs (artifacts.derived.*)
83
+ knowledge/ author only — who you are, what you decide, how you sound
84
+ notebook/ keep only — agent's own durable lane (bytes climb to knowledge only via propose→accept)
85
+ proposals/ propose (agent+human) — proposals waiting on a human accept
86
+ artifacts/ converge only — machine-maintained: computed outputs + external inputs
83
87
  ```
84
88
 
85
89
  An agent that tries to write directly into `knowledge/` gets `write_forbidden`. It writes to `proposals/` (to change authoritative content) or its own `notebook/` (for working memory). You accept the good proposals; textus promotes them, records the move, and audits both halves. Stable per-entry `uid:` means a reorganization doesn't break references. A monotonic audit cursor (`textus pulse --since=N`) means the next session — possibly a different agent, possibly a different model — picks up exactly where the last one left off.
@@ -109,12 +113,12 @@ Try the gate the other way (`textus put knowledge.notes.X --as=agent`) and you g
109
113
 
110
114
  ## Try it
111
115
 
112
- - **Worked end-to-end store** — the role gate (propose → accept), drain/publish (`CLAUDE.md` / `AGENTS.md` generated from knowledge entries), schemas, templates, and a hook: [`.textus/`](.textus/)
116
+ - **Worked end-to-end store** — the role gate (propose → accept), drain/publish (`CLAUDE.md` / `AGENTS.md` generated from knowledge entries), schemas, ERB templates, and workflows: [`.textus/`](.textus/)
113
117
  - **Wire textus into Claude Code via MCP** — 4 steps, ~5 minutes: [`docs/how-to/agents-mcp.md`](docs/how-to/agents-mcp.md)
114
118
 
115
119
  ## Protocol, not just a gem
116
120
 
117
- This Ruby gem is the reference implementation of **`textus/3`** — a wire format and storage convention any language can speak. The protocol owns the envelope shape, the role/lane gate, the audit log format, and the key grammar. The gem version (semver, see badge) and the protocol version (`textus/3`) move independently; envelopes carry the `protocol` field so consumers can pin to the contract, not the implementation.
121
+ This Ruby gem is the reference implementation of **`textus/4`** — a wire format and storage convention any language can speak. The protocol owns the envelope shape, the role/lane gate, the audit log format, and the key grammar. The gem version (semver, see badge) and the protocol version (`textus/4`) move independently; envelopes carry the `protocol` field so consumers can pin to the contract, not the implementation.
118
122
 
119
123
  - Specification: [`SPEC.md`](SPEC.md)
120
124
  - Architecture: [`docs/architecture/README.md`](docs/architecture/README.md)
@@ -137,7 +141,7 @@ bundle exec exe/textus --help
137
141
 
138
142
  ## What `textus init` gives you
139
143
 
140
- You get `.textus/` with all four lane directories under `data/`, baseline schemas, a starter manifest, and a gitignored `.run/` for disposable runtime state (the audit log, per-role cursors, produce locks). Roles declare capabilities; each lane declares a `kind:`, and write authority is derived from the role's capabilities crossed with the lane's kind:
144
+ You get `.textus/` with all five lane directories under `data/`, baseline schemas, a starter manifest, and a gitignored `.state/` for disposable runtime state (the audit log, per-role cursors, produce locks). Roles declare capabilities; each lane declares a `kind:`, and write authority is derived from the role's capabilities crossed with the lane's kind:
141
145
 
142
146
  ```yaml
143
147
  roles:
@@ -146,29 +150,30 @@ roles:
146
150
  - { name: automation, can: [converge] }
147
151
 
148
152
  lanes:
149
- - { name: knowledge, kind: canon } # author — canonical truth
150
- - { name: notebook, kind: workspace } # keep — agent's own durable lane
151
- - { name: proposals, kind: queue } # propose — proposals awaiting accept
152
- - { name: artifacts, kind: machine } # converge — external inputs (artifacts.feeds.*) + computed outputs (artifacts.derived.*)
153
+ - { name: knowledge, kind: canon } # author — canonical truth
154
+ - { name: notebook, kind: workspace } # keep — agent's own durable lane
155
+ - { name: proposals, kind: queue } # propose — proposals awaiting accept
156
+ - { name: artifacts, kind: machine } # converge — computed outputs + external inputs
153
157
  ```
154
158
 
155
159
  ```
156
160
  .textus/
157
161
  manifest.yaml # role capabilities + lane kinds + key-to-path mapping
158
162
  schemas/ # YAML field shapes per entry family
159
- templates/ # mustache templates for derived entries
160
- steps/ # step subclasses: fetch/, transform/, validate/, observe/
161
- .gitignore # generated — ignores .run/ and any tracked:false entries
163
+ templates/ # ERB templates for produced entries
164
+ workflows/ # Ruby workflow files (Textus.workflow DSL) for data acquisition
165
+ .gitignore # generated — ignores .state/ and any tracked:false entries
162
166
  data/ # one dir per lane; kinds + capabilities are in the manifest above
163
167
  knowledge/ # e.g. identity (knowledge.identity.*), voice, decisions, notes
164
168
  notebook/
165
169
  proposals/
166
- artifacts/ # machine lane: feeds/ (external inputs) + derived/ (computed outputs)
167
- .run/ # disposable runtime state — gitignored, safe to delete (ADR 0038)
168
- audit/audit.log # append-only NDJSON event ledger, every write (rotates at ~50 MB)
169
- state/cursor.<role> # per-role pulse cursor — where `pulse --since` resumes
170
+ artifacts/ # machine lane: computed outputs + external inputs
171
+ .state/ # disposable runtime state — gitignored, safe to delete (ADR 0038)
172
+ audit/audit.log # append-only NDJSON event ledger, every write (rotates at ~10 MB)
173
+ cursors/<role> # per-role pulse cursor — where `pulse --since` resumes
170
174
  locks/ # per-key produce locks + the produce mutex
171
175
  sentinels/ # publish bookkeeping (target sha) — regenerated on drain (ADR 0070)
176
+ indexes/raw.yaml # raw lane content-hash/URL index — regenerable cache
172
177
  ```
173
178
 
174
179
  Manifest `path:` fields are relative to `.textus/data/`. So `knowledge.notes.org.jane` lives at `.textus/data/knowledge/notes/org/jane.md`.
@@ -181,104 +186,106 @@ textus list --lane=knowledge
181
186
  printf '%s' '{"_meta":{"name":"bob","org":"acme"},"body":"hi\n"}' \
182
187
  | textus put knowledge.notes.bob --as=human --stdin
183
188
  textus drain --as=automation # re-pull stale inputs + recompute derived outputs
184
- textus rule list # show every rule block
185
- textus audit --limit=20 # query the audit log
189
+ textus rule list # show every rule block
190
+ textus audit --limit=20 # query the audit log
186
191
  ```
187
192
 
188
- (All verbs return JSON envelopes; `--output=json` is the default and the only format in v1.)
193
+ (All verbs return JSON envelopes; `--output=json` is the default and the only format.)
189
194
 
190
- For a worked store — knowledge entries, a staged proposal, schemas, a template, and a `drain` that publishes `CLAUDE.md` / `AGENTS.md` — see [`.textus/`](.textus/).
195
+ For a worked store — knowledge entries, a staged proposal, schemas, ERB templates, and a `drain` that publishes `CLAUDE.md` / `AGENTS.md` — see [`.textus/`](.textus/).
191
196
 
192
197
  ## What's shipped
193
198
 
194
199
  - **Per-entry formats & publish.** `format: markdown|json|yaml|text` per entry; a typed `publish:` block (`to:` for file fan-out, `tree:` for a whole-subtree mirror) byte-copies derived files to their consumer paths. ([SPEC §5.2–5.3](SPEC.md))
195
200
  - **Stable identity.** Auto-minted `uid:` survives writes and `textus key mv`; reorganising never breaks references.
196
201
  - **Capability × lane-kind gate.** Writes carry `--as=<role>`; a role may write a lane iff it holds the capability the lane's `kind:` requires (`canon`→`author`, `workspace`→`keep`, `machine`→`converge`, `queue`→`propose`). The wrong role gets `write_forbidden` naming the capability needed and the roles that hold it. ([SPEC §5](SPEC.md))
197
- - **Agent loop.** `textus boot` orients a fresh session; `textus pulse --since=N` is the per-turn heartbeat (changed entries, stale keys, pending proposals). ([docs/how-to/agents-mcp.md](docs/how-to/agents-mcp.md))
198
- - **`textus doctor`.** Health checks across schemas, step registrations, keys, sentinels, and the audit log.
202
+ - **Agent loop.** `textus boot` orients a fresh session; `textus pulse --since=N` is the per-turn heartbeat (changed entries, pending proposals, index etag for catalog drift detection). ([docs/how-to/agents-mcp.md](docs/how-to/agents-mcp.md))
203
+ - **MCP surface.** The official `mcp` Ruby SDK drives the stdio JSON-RPC server; protocol version auto-negotiated up to `2025-11-25`. Wire textus into Claude Code, Cursor, or any MCP host in one config block.
204
+ - **`textus doctor`.** Health checks across schemas, workflow registrations, keys, sentinels, and the audit log.
205
+ - **`raw` lane and `ingest` verb.** Write-once intake lane for external URL bookmarks, files, and binary assets. Three source kinds (`url`/`file`/`asset`); daily key derivation; notebook stub per ingest. See "Intake and ingest" section below.
199
206
 
200
207
  ## CLI and lanes
201
208
 
202
209
  Every command operates on one store, located in this order: `--root <path>` flag → **`TEXTUS_ROOT`** env → walk up from the working directory for a `.textus/` ([SPEC §3.1](SPEC.md)). Write verbs require `--as=<role>`, resolved as: `--as` flag → **`TEXTUS_ROLE`** env → `.textus/role` file → default `human` ([SPEC §5.1](SPEC.md)). Default roles: `human`, `agent`, `automation` (rename or add your own in the manifest's `roles:` block). All verbs accept `--output=json` and return the envelope defined in [SPEC §8](SPEC.md).
203
210
 
204
211
  - Full verb table — read, write, health, scaffolding — is in [SPEC §9](SPEC.md).
205
- - Lane semantics and the capability × lane-kind mapping live in [SPEC §5](SPEC.md), with the reference in [`docs/reference/zones.md`](docs/reference/zones.md).
212
+ - Lane semantics and the capability × lane-kind mapping live in [SPEC §5](SPEC.md), with the reference in [`docs/reference/lanes.md`](docs/reference/lanes.md).
206
213
 
207
- `textus boot` prints the same information for the current store: lanes, entry families with schemas, registered steps, write flows, and the verb catalog. Run it inside a store and you get the live picture; reach for the SPEC when you want the contract.
214
+ `textus boot` prints the same information for the current store: lanes, entry families with schemas, registered workflows, write flows, and the verb catalog. Run it inside a store and you get the live picture; reach for the SPEC when you want the contract.
208
215
 
209
216
  ## Produce and publish
210
217
 
211
- Produced entries (`kind: produced`) declare how they're acquired in one `source:` block (ADR 0093/0094); `drain` materialises them:
218
+ Produced entries (`kind: produced`) declare how they're acquired in one `source:` block; `drain` materialises them. Two built-in modes, plus workflows for custom data acquisition:
212
219
 
213
- - **`source: { from: derive, select: [...], pluck:, sort_by:, limit:, transform: name }`** — a *derived* entry: textus computes its data from other entries, then renders it through a template under `.textus/templates/` (markdown/text) or a templateless path that lets a transform hook shape the output directly (json/yaml). Projections cap at 1000 rows; the vendored Mustache subset caps at depth 8. No partials, no lambdas, no HTML escaping.
214
- - **`source: { from: fetch, handler: name, ttl: 1h, config: {...} }`** — *intake*: a Step::Fetch handler pulls external bytes on a `ttl` cadence; `drain` re-pulls when the entry goes stale.
215
- - **`source: { from: external, sources: [...] }`** — *externally managed*: an out-of-band command writes the file; textus tracks the declared `sources` for staleness.
220
+ - **`source: { from: external, command: "...", sources: [...] }`** — *externally managed*: an out-of-band command or workflow writes the file; textus tracks staleness via declared `sources`.
221
+ - **`source: { from: external, command: "true", sources: [] }` + a workflow** — *workflow-driven*: a `Textus.workflow` block (in `.textus/workflows/`) acquires and shapes the data on `drain`.
216
222
 
217
- Publishing is one typed `publish:` block (ADR 0052). `publish: { to: [path, ...] }` byte-copies a single produced file to one or more targets. `publish: { tree: "dir" }` on a nested entry mirrors its whole stored subtree to one target directory, preserving layout (path-driven — no keys or template variables). Sentinels for every published file live under `.textus/.run/sentinels/` (git-ignored runtime state, regenerated on drain — ADR 0070). See SPEC §5.2, §5.3, §5.12.
223
+ Publishing is one typed `publish:` block (ADR 0052/0094). Each target is either `{ to: path, template?: name }` for a single file (optionally rendered through an ERB template) or `{ tree: "dir" }` to mirror a whole stored subtree. Sentinels for every published file live under `.textus/.state/sentinels/` (git-ignored, regenerated on drain). See SPEC §5.2, §5.3, §5.12.
218
224
 
219
- ## Extension points
225
+ Templates live in `.textus/templates/` as ERB files (`.erb`). The template receives the entry's `content` hash as local variables via `ERB#result_with_hash`. If `inject_boot: true`, a `boot` variable is also available with the live orientation context.
220
226
 
221
- textus extends through **steps** — subclass the right base, place the file in `.textus/steps/<kind>/`, and `drain` discovers it. Three kinds:
227
+ ## Workflows
222
228
 
223
- **`Step::Fetch`**acquires bytes for an intake entry (`from: fetch`). One class per named handler; `drain` invokes it when the entry is stale.
229
+ textus extends through **workflows** a `Textus.workflow` block placed in `.textus/workflows/**/*.rb`. Each workflow matches a produced entry by key glob, then runs one or more named steps to acquire its data:
224
230
 
225
231
  ```ruby
226
- # .textus/steps/fetch/local_file.rb
227
- module Textus
228
- module Step
229
- class LocalFileFetch < Fetch
230
- def call(config:, args:, **)
231
- path = config["path"] or raise "local-file requires source.config.path"
232
- { "_meta" => { "last_fetched_at" => Time.now.utc.iso8601 },
233
- "body" => File.read(File.expand_path(path)) }
234
- end
235
- end
232
+ # .textus/workflows/docs/my_report.rb
233
+ Textus.workflow "my_report" do
234
+ match "artifacts.my-report"
235
+
236
+ step :build do |_, ctx|
237
+ # read from knowledge, fetch external data, compute anything
238
+ rows = ctx.container.manifest.resolver
239
+ .enumerate(prefix: "knowledge.notes")
240
+ .map { |r| { "key" => r[:key], "title" => r[:entry].schema } }
241
+ { "content" => { "entries" => rows } }
236
242
  end
237
243
  end
238
244
  ```
239
245
 
240
- **`Step::Transform`** reshapes projected rows for a `from: derive` entry.
246
+ `drain` discovers all workflow files, matches them against produced entries, and runs the steps. The result is written back to the entry's data path; `publish:` then copies it to its consumer paths.
241
247
 
242
- ```ruby
243
- # .textus/steps/transform/rank_by_recency.rb
244
- module Textus
245
- module Step
246
- class RankByRecencyTransform < Transform
247
- def call(rows:, config:, **)
248
- rows.sort_by { |r| r["updated_at"].to_s }.reverse
249
- end
250
- end
251
- end
252
- end
253
- ```
248
+ ## Intake and ingest
254
249
 
255
- **`Step::Observe`** reacts to lifecycle events (fire-and-forget, 0..N per event):
250
+ The `raw` lane is the inbound counterpart to `artifacts`: where `drain` materialises
251
+ **outbound** computed outputs, `ingest` receives **inbound** external source material.
256
252
 
257
- ```ruby
258
- # .textus/steps/observe/log_writes.rb
259
- module Textus
260
- module Step
261
- class LogWritesObserve < Observe
262
- on :entry_written
263
-
264
- def call(key:, envelope:, **)
265
- $textus_event_log << [key, envelope.etag]
266
- end
267
- end
268
- end
269
- end
270
- ```
253
+ **The ingest principle:** prefer a reference over a copy. Store body or asset only when the
254
+ content itself is the value — human-authored notes, brainstorm outputs, context you want to
255
+ annotate. For everything else, the URL is enough. If the source is private or
256
+ access-restricted, set `access: private` in `source:` so downstream workflows can handle it
257
+ appropriately.
258
+
259
+ **Three source kinds:**
260
+
261
+ | Kind | Stores | Use when |
262
+ |------|--------|----------|
263
+ | `url` | URL reference only (`body: null`) | Bookmarking a page, skill, or doc for later annotation |
264
+ | `file` | File body text | Valuable human-authored content (brainstorm notes, meeting summaries) |
265
+ | `asset` | Binary at `assets/raw/` | Screenshots, PDFs — only when the asset itself is the artefact |
266
+
267
+ **Write-once** — the same slug on the same day cannot be overwritten. Delete and re-ingest to replace.
271
268
 
272
- Observable events: `:entry_written`, `:entry_deleted`, `:entry_fetched`, `:entry_renamed`, `:entry_produced`, `:entry_published`, `:produce_failed`, `:proposal_accepted`, `:proposal_rejected`, `:store_loaded`, `:session_opened`, `:entry_fetch_started`, `:entry_fetch_failed`.
269
+ ```sh
270
+ # bookmark a skill reference — URL only, body stays null
271
+ textus ingest url agentskills-io-brainstorming \
272
+ --url=https://agentskills.io/skills/brainstorming \
273
+ --label="brainstorming skill" \
274
+ --as=agent
275
+
276
+ # see what landed in the raw lane
277
+ textus list --lane=raw
278
+
279
+ # a notebook stub was created alongside — annotate it
280
+ textus get notebook.notes.raw
281
+ ```
273
282
 
274
- Stale intake entries are re-pulled by `drain`, not by reads — `get` is a pure
275
- read that annotates the returned envelope with a freshness verdict (ADR 0089).
276
- `drain` re-pulls anything past its `source.ttl` and recomputes derived outputs:
283
+ Stale produced entries are re-materialised by `drain`, not by reads — `get` is a pure read (ADR 0089).
277
284
 
278
285
  ```sh
279
- textus drain --as=automation # re-pull every stale intake + recompute derived
280
- textus drain artifacts.feeds --as=automation # scope to one prefix
281
- textus get artifacts.feeds.calendar.events # a pure read; carries a freshness verdict
286
+ textus drain --as=automation # re-materialise every stale produced entry
287
+ textus drain artifacts.feeds.skills --as=automation # scope to one prefix
288
+ textus get artifacts.feeds.skills # a pure read; carries a freshness verdict
282
289
  ```
283
290
 
284
291
  Schemas (`.textus/schemas/<name>.yaml`) declare field shapes, per-field `maintained_by:` ownership, and an `evolution:` block (`added_in`, `deprecated_at`, `migrate_from`). Full contract in SPEC §5.8.
@@ -287,7 +294,7 @@ See [`docs/how-to/agents-mcp.md`](docs/how-to/agents-mcp.md) for the agent boot
287
294
 
288
295
  ## Examples
289
296
 
290
- [`.textus/`](.textus/) — textus as a project's own context store (a fictional Rails service, `ledger`). Human-authored `knowledge/` (project facts, runbooks), a staged ADR in `proposals/` showing the agent-propose / human-accept loop, schemas validating each family, a mustache template plus a `Step::Transform` step, and a `drain` that publishes the `artifacts.derived.orientation` projection to `CLAUDE.md` and `AGENTS.md`. Includes a copy-paste adoption recipe for your own repo.
297
+ [`.textus/`](.textus/) — textus as a project's own context store. Human-authored `knowledge/` (project facts, runbooks, ADRs), a staged proposal showing the agent-propose / human-accept loop, schemas validating each family, ERB templates and workflows, and a `drain` that publishes the orientation artifact to `CLAUDE.md` and `AGENTS.md`. Includes a copy-paste adoption recipe for your own repo.
291
298
 
292
299
  ## Tests
293
300