textus 0.3.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: c72c516279aeb0df734da1a72dd89fd033ebbed2e891444b71015e04fab73e75
4
- data.tar.gz: a1075b8990c6c1749eb03f5042c1e5bd590456307cf9b3e1a9de9c33dcc0a862
3
+ metadata.gz: 98c2ce5525bbf9c05ebdb5eeaacde8e208253ee878d7d0722ff192435168f69f
4
+ data.tar.gz: c16cd5657396884c646331912d988838e0f8ed2002fc76d47cc54383dc434f19
5
5
  SHA512:
6
- metadata.gz: 2218efa63e1ae6f246af3341bbb27e713a71a381a4aa84d15d46e687b8585c3bb5f992d75456c6112c960fec8a286c3cb7e6e672af04db0d2657811b45627ffb
7
- data.tar.gz: 98ec77bf76fe14c850470e9dee3aebe55bba44ae6d4aa6303951eab2d697f2eed4d8d2d44278615e7e80873704d41dcd46109f244414ca2d16c7a4c4a7c12e2a
6
+ metadata.gz: 7478459f9672474c32f37b59a7134309cf12f6a7455a808c24b6ac717820ff8c8aad5a5187a6dbe5e0c73f98c04296ce95eff2cac8ea75144f559df8cea3fe80
7
+ data.tar.gz: '08a29e9090f6558a010009e3562a7ec8d6ddf73454bb09b10b5aeaf3def24d9d11905078d62807b8e1f0936b00889a66df8c55d68b7dfc411addd90e60ae1a93'
data/CHANGELOG.md CHANGED
@@ -10,6 +10,21 @@ is additive within a major; a new major would change the wire string.
10
10
 
11
11
  ## [Unreleased]
12
12
 
13
+ ## 0.4.0 — Extension API redesign (breaking)
14
+
15
+ - **Breaking:** `Textus.fetcher` removed. Use `Textus.action` instead. The block signature changes from `|config:, store:|` to `|config:, store:, args:|`.
16
+ - **Breaking:** Manifest field `source.fetcher` renamed to `source.action`. Legacy field is rejected with a migration error.
17
+ - **Breaking:** CLI flag `textus put --fetcher=NAME` renamed to `textus put --action=NAME`.
18
+ - **Breaking:** `BuiltinFetchers` module renamed to `BuiltinActions`.
19
+ - **Breaking:** Synthesized frontmatter key `fetched_with` renamed to `actioned_with` on `put --action`.
20
+ - **New:** `Textus.action` works in three invocation modes — intake refresh, the new `textus action NAME` verb, and `put --action`. See SPEC §5.11.
21
+ - **New:** `Textus.doctor_check(:name) { |store:| ... }` primitive; contributed checks merge into the doctor report.
22
+ - **New:** `textus action NAME [--key=val ...] [--as=ROLE]` CLI verb for invoking actions in verb mode.
23
+ - **New:** `StoreView` gains a writable mode (`writable: true, as: ROLE`); intake and verb-mode actions receive a writable view bound to the calling role.
24
+ - **New:** `extensions list` enumerates actions and doctor_checks.
25
+
26
+ Migration: in every `.textus/extensions/*.rb`, rename `Textus.fetcher(:x)` to `Textus.action(:x)` and add `args:` to the block signature. In every manifest, rename `source.fetcher:` to `source.action:`. In CI/scripts using `textus put --fetcher=`, switch to `--action=`.
27
+
13
28
  ## [0.3.0] — 2026-05-20 — Configurable store root
14
29
 
15
30
  ### Added
data/README.md CHANGED
@@ -45,12 +45,12 @@ You get `.textus/` with all five zone directories, baseline schemas, an empty au
45
45
  audit.log # append-only NDJSON, every write
46
46
  schemas/ # YAML field shapes per entry family
47
47
  templates/ # mustache templates for derived entries
48
- extensions/ # one .rb per fetcher / reducer / hook
48
+ extensions/ # one .rb per action / reducer / hook / doctor_check
49
49
  sentinels/ # publish bookkeeping
50
50
  zones/
51
51
  canon/ # human-only — identity, voice, decisions
52
52
  working/ # human / ai / script — day-to-day catalog
53
- intake/ # script — declared external inputs (fetchers)
53
+ intake/ # script — declared external inputs (actions)
54
54
  pending/ # ai + human — proposals awaiting accept
55
55
  derived/ # build only — computed outputs
56
56
  ```
@@ -67,7 +67,7 @@ echo '{"frontmatter":{"name":"bob","org":"acme"},"body":"hi\n"}' \
67
67
  textus stale --zone=derived --format=json
68
68
  ```
69
69
 
70
- For the full shape — Claude plugin with agents, skills, commands, pending walkthrough, intake fetcher — see [`examples/claude-plugin/`](examples/claude-plugin/).
70
+ For the full shape — Claude plugin with agents, skills, commands, pending walkthrough, intake action — see [`examples/claude-plugin/`](examples/claude-plugin/).
71
71
 
72
72
  ## What 0.2 ships
73
73
 
@@ -98,13 +98,14 @@ All verbs accept `--format=json` and return the envelope defined in SPEC §8. Wr
98
98
  | `deps K` / `rdeps K` | Forward / reverse projection dependencies |
99
99
  | `published` | List `publish_to:` targets and their backing keys |
100
100
  | `validate-all` | Validate every entry against its schema |
101
- | `extensions list [--kind=K]` | Registered fetchers, reducers, declared hooks |
101
+ | `extensions list [--kind=K]` | Registered actions, reducers, hooks, doctor_checks |
102
102
 
103
103
  **Write:**
104
104
 
105
105
  | Verb | Role |
106
106
  |---|---|
107
- | `put K --stdin --as=R [--fetcher=NAME]` | per zone |
107
+ | `put K --stdin --as=R [--action=NAME]` | per zone |
108
+ | `action NAME [--key=val] [--as=R]` | per zone written (invoke a registered action) |
108
109
  | `delete K --if-etag=E --as=R` | per zone |
109
110
  | `refresh K --as=script` | per zone (typically `script`) |
110
111
  | `mv old new --as=R [--dry-run]` | per zone (same-zone moves; uid preserved) |
@@ -133,7 +134,7 @@ All verbs accept `--format=json` and return the envelope defined in SPEC §8. Wr
133
134
  |---|---|---|
134
135
  | `canon` | `[human]` | Identity, voice, decisions — slow-changing |
135
136
  | `working` | `[human, ai, script]` | Active project state |
136
- | `intake` | `[script]` | Declared external inputs (fetchers) |
137
+ | `intake` | `[script]` | Declared external inputs (actions) |
137
138
  | `pending` | `[ai, human]` | AI proposals; humans run `textus accept` to apply |
138
139
  | `derived` | `[build]` | Computed outputs from `textus build` |
139
140
 
@@ -147,17 +148,18 @@ Derived entries declare a `projection:` (`select`, `pluck`, `sort_by`, `limit`,
147
148
 
148
149
  ## Extensions
149
150
 
150
- Three DSL verbs, registered in `.textus/extensions/*.rb`. Each `Store` gets its own registry — no global state.
151
+ Four DSL verbs, registered in `.textus/extensions/*.rb`. Each `Store` gets its own registry — no global state.
151
152
 
152
- - **`Textus.fetcher(:name) do |config:, store:|`** — pulls data into an intake entry. Returns `{frontmatter:, body:}`, `{content:}` (for json/yaml entries), or `{body:}` (raw). The store normalizes all three shapes. Configured via `source.fetcher` in the manifest. Five built-ins ship: `json`, `csv`, `markdown-links`, `ical-events`, `rss`.
153
+ - **`Textus.action(:name) do |config:, store:, args:|`** — runs in three invocation modes (intake refresh, `textus action` verb, `put --action`). Returns `{frontmatter:, body:}`, `{content:}`, or `{body:}` when its return is consumed (intake and put-fetch); writes via `store.put` for side-effectful work (verb mode). The store normalizes all three return shapes. Configured via `source.action` in the manifest for intake. Five built-ins ship: `json`, `csv`, `markdown-links`, `ical-events`, `rss`.
153
154
  - **`Textus.reducer(:name) do |rows:, config:|`** — shapes rows in a derived projection. Pure function. Configured via `projection.reducer`. May return an Array (templated builds) or a Hash (templateless json/yaml).
154
155
  - **`Textus.hook(:event, :name) do |kwargs|`** — fires on `:put`, `:delete`, `:refresh`, `:build`, or `:accept`. In-process; 2 s timeout per hook; failures land in the audit log as `event_error` rows.
156
+ - **`Textus.doctor_check(:name) do |store:|`** — contributes whole-tree validators to `textus doctor`. Returns an array of issue hashes `{code, level, subject, message, fix}` that merge into the doctor report. Timeouts and exceptions surface as `doctor_check.*` issues; they do not abort the doctor run.
155
157
 
156
158
  Schemas (`.textus/schemas/<name>.yaml`) declare field shapes, per-field `maintained_by:` ownership, and an `evolution:` block (`added_in`, `deprecated_at`, `migrate_from`). Full contract in SPEC §5.8 and §5.11.
157
159
 
158
160
  ## Examples
159
161
 
160
- [`examples/claude-plugin/`](examples/claude-plugin/) — a Claude Code plugin (`voice-tools`) whose entire content surface — agents, skills, commands, `CLAUDE.md`, `plugin.json`, `marketplace.json` — is textus-managed. Demonstrates per-entry formats, `publish_each`, intake fetchers, in-process reducers and hooks, the AI-propose / human-accept loop, and the `inject_intro:` flag that puts an orientation preamble at the top of `CLAUDE.md`.
162
+ [`examples/claude-plugin/`](examples/claude-plugin/) — a Claude Code plugin (`voice-tools`) whose entire content surface — agents, skills, commands, `CLAUDE.md`, `plugin.json`, `marketplace.json` — is textus-managed. Demonstrates per-entry formats, `publish_each`, intake actions, in-process reducers and hooks, the AI-propose / human-accept loop, and the `inject_intro:` flag that puts an orientation preamble at the top of `CLAUDE.md`.
161
163
 
162
164
  ## Tests
163
165
 
data/SPEC.md CHANGED
@@ -46,7 +46,7 @@ textus is organized as five composable layers. Each layer has a single responsib
46
46
  - Not a sync protocol. Single-writer per file, ETag-checked.
47
47
  - Not a transport. Spawn the CLI or wrap it in MCP/HTTP downstream.
48
48
  - Not a UI. Filesystem + CLI. Viewers ship elsewhere.
49
- - Not a fetcher. textus declares sources; external runners fetch them.
49
+ - Not a fetcher. textus declares sources; external runners invoke actions to materialize them.
50
50
  - Not an executor. textus computes pure projections but never spawns shell commands.
51
51
 
52
52
  ## 3. Storage layout
@@ -250,38 +250,38 @@ A sentinel is written for each published file at `<store_root>/sentinels/<target
250
250
 
251
251
  **Per-leaf publishing.** A nested entry MAY declare `publish_each:` instead of `publish_to:` (see §4). When the build runs, every leaf reachable under the nested entry is byte-copied to the path produced by substituting `{leaf}` / `{basename}` / `{key}` / `{ext}` in the template, with a sentinel written under `<store_root>/sentinels/` at the mirrored target path. The build envelope grows a `published_leaves` array — one row per leaf, with `key`, `source`, and `target` — alongside the existing `built` array. Targets that would resolve outside the repo root are refused.
252
252
 
253
- ### 5.4 Intake (declared, refreshed via registered fetcher)
253
+ ### 5.4 Intake (declared, refreshed via registered action)
254
254
 
255
- Intake entries declare an external source by naming a **fetcher** — a registered, named function that pulls data into the entry. textus itself still makes no implicit network calls: a fetcher only runs when explicitly invoked by `textus refresh KEY --as=script`. The declaration is data only:
255
+ Intake entries declare an external source by naming an **action** — a registered, named function that pulls data into the entry. textus itself still makes no implicit network calls: an action only runs in intake mode when explicitly invoked by `textus refresh KEY --as=script`. The declaration is data only:
256
256
 
257
257
  ```yaml
258
258
  - key: intake.calendar.events
259
259
  zone: intake
260
260
  source:
261
- fetcher: ical-events
261
+ action: ical-events
262
262
  config:
263
263
  url: "https://calendar.google.com/.../basic.ics"
264
264
  ttl: 6h
265
265
  ```
266
266
 
267
- `fetcher` names a registered fetcher; `config` is an opaque hash handed to the fetcher; `ttl` is the staleness budget. Implementations MUST reject legacy `source.from` and `source.parse` with a clear usage error.
267
+ `action` names a registered action; `config` is an opaque hash handed to the action; `ttl` is the staleness budget. Implementations MUST reject legacy `source.from`, `source.parse`, and `source.fetcher` with a clear usage error.
268
268
 
269
- **Fetcher contract.** A fetcher is registered via `Textus.fetcher(:name) do |config:, store:| ... end` and MUST return one of three shapes, all normalized by the store into its internal `{frontmatter, body, content}` representation (§5.12):
269
+ **Action contract (intake mode).** An action is registered via `Textus.action(:name) do |config:, store:, args:| ... end`. In intake mode the action MUST return one of three shapes, all normalized by the store into its internal `{frontmatter, body, content}` representation (§5.12):
270
270
 
271
- - `{ frontmatter:, body: }` — markdown-friendly (current shape).
271
+ - `{ frontmatter:, body: }` — markdown-friendly.
272
272
  - `{ content: }` — for `format: json|yaml` entries; the parsed object becomes the entry's content.
273
273
  - `{ body: }` — raw bytes for `text` or for any format that prefers verbatim writes; the store re-parses and validates per `format:`.
274
274
 
275
- The `store:` argument is a read-only `Textus::StoreView` (§5.11). Every fetcher call is wrapped in `Timeout.timeout(2)`; exceptions and timeouts surface as `usage` errors that abort the refresh.
275
+ The `store:` argument is a writable `Textus::StoreView` (§5.11) bound to the calling role; the `args:` argument is `{}` in intake mode (it carries CLI flags in verb mode — §5.11). Every action call is wrapped in `Timeout.timeout(2)`; exceptions and timeouts surface as `usage` errors that abort the refresh.
276
276
 
277
- **Built-in fetchers.** `json`, `csv`, `markdown-links`, `ical-events`, `rss` are always available. They expect raw bytes in `config["bytes"]` and produce structured frontmatter/body. Built-ins do not perform I/O themselves — the caller (or an outer fetcher) is responsible for supplying bytes.
277
+ **Built-in actions.** `json`, `csv`, `markdown-links`, `ical-events`, `rss` are always available. They expect raw bytes in `config["bytes"]` and produce structured frontmatter/body. Built-ins do not perform I/O themselves — the caller (or an outer action) is responsible for supplying bytes.
278
278
 
279
279
  **Refresh paths.** Two are supported:
280
280
 
281
- 1. **In-process** — `textus refresh KEY --as=script` resolves the entry's `source.fetcher`, invokes it with `(config:, store:)`, and writes the result under role `script`.
281
+ 1. **In-process** — `textus refresh KEY --as=script` resolves the entry's `source.action`, invokes it with `(config:, store:, args: {})`, and writes the result under role `script`.
282
282
  2. **External runner** — a cron job or agent harness reads `textus list --zone=intake --stale --format=json`, fetches the source out of band, and pipes bytes back through `textus put KEY --as=script --stdin`.
283
283
 
284
- Both paths share the same role gate, audit-log entry, and `:refresh` event (§5.10). User-supplied fetchers live in `.textus/extensions/*.rb` and auto-load at `Store#initialize` (§5.11).
284
+ Both paths share the same role gate, audit-log entry, and `:refresh` event (§5.10). User-supplied actions live in `.textus/extensions/*.rb` and auto-load at `Store#initialize` (§5.11).
285
285
 
286
286
  ### 5.5 Pending / accept workflow
287
287
 
@@ -419,29 +419,39 @@ Textus does NOT invoke these — they surface only through `textus extensions li
419
419
 
420
420
  **Removed.** The v1.1 `on_stale` event is removed in 0.2. Staleness is a poll, surfaced by `textus stale`. The `on_`-prefix convention from v1.1 is gone; events are bare symbols.
421
421
 
422
- ### 5.11 Extension surface (v1.2)
422
+ ### 5.11 Extension surface (v1.3)
423
423
 
424
- Three DSL verbs cover all user-supplied code:
424
+ Four DSL verbs cover all user-supplied code:
425
425
 
426
426
  ```
427
- Textus.fetcher(:name) do |config:, store:| ... end # returns {frontmatter:, body:} | {content:} | {body:}
428
- Textus.reducer(:name) do |rows:, config:| ... end # returns rows
429
- Textus.hook(:event, :name) do |**kwargs| ... end # side effects; return ignored
427
+ Textus.action(:name) do |config:, store:, args:| ... end # intake mode: returns content; verb mode: writes via store.put
428
+ Textus.reducer(:name) do |rows:, config:| ... end # returns rows
429
+ Textus.hook(:event, :name) do |**kwargs| ... end # side effects; return ignored
430
+ Textus.doctor_check(:name) do |store:| ... end # returns array of issue hashes
430
431
  ```
431
432
 
432
433
  Files in `.textus/extensions/*.rb` are loaded at `Store#initialize`, in lexical order, with the registry installed as the current registry for that store. Registries are per-Store: two Store instances in the same process do not share state.
433
434
 
434
- Failure modes:
435
+ **Action invocation modes.**
435
436
 
436
- | Surface | Timeout | Exception | Bad return |
437
- |----------|------------|---------------------------------------------|------------|
438
- | fetcher | aborts op | aborts op (wrapped as `UsageError`) | aborts op |
439
- | reducer | aborts op | aborts op | aborts op |
440
- | hook | logged | logged (audit `event_error` row) | n/a |
437
+ | Mode | Invoked by | `config:` | `store:` | `args:` | Return |
438
+ |---------|---------------------------|------------------------|------------------------|----------------------|------------------------------------------|
439
+ | intake | `textus refresh KEY` | manifest `source.config` | writable view (role from `--as`) | `{}` | required; normalized into entry write |
440
+ | verb | `textus action NAME ...` | `{}` | writable view (role from `--as`) | parsed CLI kv hash | ignored |
441
+ | put-fetch | `textus put K --action=N --stdin` | `{ "bytes" => stdin }` | read-only view | `{}` | required; merged into the put payload |
441
442
 
442
- Fetchers and reducers are pure transforms; return values flow into the store. Hooks are side effects; return values are discarded.
443
+ **Failure modes:**
443
444
 
444
- The `store:` argument is always a `Textus::StoreView` — a read-only proxy exposing `get`, `list`, `where`, `schema_envelope`, `deps`, `rdeps`, `published`, `stale`, `validate_all`. Write attempts raise `Textus::UsageError`.
445
+ | Surface | Timeout | Exception | Bad return |
446
+ |-----------------|------------|---------------------------------------------|------------|
447
+ | action | aborts op | aborts op (wrapped as `UsageError`) | aborts op |
448
+ | reducer | aborts op | aborts op | aborts op |
449
+ | hook | logged | logged (audit `event_error` row) | n/a |
450
+ | doctor_check | reported as `doctor_check.timeout` issue | reported as `doctor_check.failed` issue | reported as `doctor_check.bad_return` |
451
+
452
+ Actions and reducers are pure transforms in modes where their return matters; their return values flow into the store. Hooks and doctor_checks shape side outputs (event chain, doctor report) — only doctor_check return values are merged.
453
+
454
+ The `store:` argument is always a `Textus::StoreView`. In intake and verb modes it is writable and bound to the calling role; in put-fetch mode and inside doctor_checks it is read-only. Write attempts on a read-only view raise `Textus::UsageError`.
445
455
 
446
456
  ### 5.12 Storage formats (v1.2)
447
457
 
@@ -585,7 +595,7 @@ All verbs accept `--format=json` and emit a canonical envelope (success or error
585
595
  | `deps K` / `rdeps K` | read | any |
586
596
  | `published` | read | any |
587
597
  | `validate-all` | read | any |
588
- | `put K --stdin --as=R [--fetcher=NAME]` | write | per zone |
598
+ | `put K --stdin --as=R [--action=NAME]` | write | per zone |
589
599
  | `delete K --if-etag=E --as=R` | write | per zone |
590
600
  | `refresh K --as=script` | write | per zone (typically `script`) |
591
601
  | `build [--prefix=K] [--dry-run]` | write | `build` (default) |
@@ -595,7 +605,7 @@ All verbs accept `--format=json` and emit a canonical envelope (success or error
595
605
  | `migrate-keys [--dry-run\|--write]` | write (with `--write`) | `human` |
596
606
  | `mv OLD NEW [--as=R] [--dry-run]` | write | per zone (same-zone only) |
597
607
  | `uid K` | read | any |
598
- | `extensions list [--kind=fetcher\|reducer\|hook]` | read | any |
608
+ | `extensions list [--kind=action\|reducer\|hook]` | read | any |
599
609
  | `doctor [--format=json]` | read | any |
600
610
  | `intro [--format=json]` | read | any |
601
611
 
@@ -4,31 +4,35 @@ require "yaml"
4
4
  require "rexml/document"
5
5
 
6
6
  module Textus
7
- module BuiltinFetchers
7
+ module BuiltinActions
8
8
  # rubocop:disable Metrics/AbcSize, Metrics/MethodLength
9
9
  def self.register_all
10
- Textus.fetcher(:json) do |config:, store:|
10
+ Textus.action(:json) do |config:, store:, args:|
11
11
  _ = store
12
+ _ = args
12
13
  data = JSON.parse(config["bytes"].to_s)
13
14
  { frontmatter: {}, body: YAML.dump(data) }
14
15
  end
15
16
 
16
- Textus.fetcher(:csv) do |config:, store:|
17
+ Textus.action(:csv) do |config:, store:, args:|
17
18
  _ = store
19
+ _ = args
18
20
  rows = CSV.parse(config["bytes"].to_s, headers: true).map(&:to_h)
19
21
  { frontmatter: {}, body: YAML.dump(rows) }
20
22
  end
21
23
 
22
- Textus.fetcher(:"markdown-links") do |config:, store:|
24
+ Textus.action(:"markdown-links") do |config:, store:, args:|
23
25
  _ = store
26
+ _ = args
24
27
  links = config["bytes"].to_s.scan(%r{\[([^\]]+)\]\((https?://[^)\s]+)\)}).map do |text, href|
25
28
  { "text" => text, "href" => href }
26
29
  end
27
30
  { frontmatter: {}, body: YAML.dump(links) }
28
31
  end
29
32
 
30
- Textus.fetcher(:"ical-events") do |config:, store:|
33
+ Textus.action(:"ical-events") do |config:, store:, args:|
31
34
  _ = store
35
+ _ = args
32
36
  events = []
33
37
  current = nil
34
38
  config["bytes"].to_s.each_line do |line|
@@ -45,8 +49,9 @@ module Textus
45
49
  { frontmatter: {}, body: YAML.dump(events) }
46
50
  end
47
51
 
48
- Textus.fetcher(:rss) do |config:, store:|
52
+ Textus.action(:rss) do |config:, store:, args:|
49
53
  _ = store
54
+ _ = args
50
55
  doc = REXML::Document.new(config["bytes"].to_s)
51
56
  items = doc.elements.to_a("//item").map do |item|
52
57
  {
data/lib/textus/cli.rb CHANGED
@@ -44,6 +44,7 @@ module Textus
44
44
  when "schema-init" then verb_schema_init(argv)
45
45
  when "schema-diff" then verb_schema_diff(argv)
46
46
  when "schema-migrate" then verb_schema_migrate(argv)
47
+ when "action" then verb_action(argv)
47
48
  when "refresh" then verb_refresh(argv)
48
49
  when "extensions" then verb_extensions(argv)
49
50
  when "migrate-keys" then verb_migrate_keys(argv)
@@ -133,11 +134,11 @@ module Textus
133
134
  key = argv.shift or raise UsageError.new("put requires a key")
134
135
  as_flag = nil
135
136
  use_stdin = false
136
- fetcher_name = nil
137
+ action_name = nil
137
138
  OptionParser.new do |o|
138
139
  o.on("--stdin") { use_stdin = true }
139
140
  o.on("--as=ROLE") { |v| as_flag = v }
140
- o.on("--fetcher=NAME") { |v| fetcher_name = v }
141
+ o.on("--action=NAME") { |v| action_name = v }
141
142
  o.on("--format=FMT") {}
142
143
  end.permute!(argv)
143
144
  raise UsageError.new("put requires --stdin in v1") unless use_stdin
@@ -146,16 +147,16 @@ module Textus
146
147
 
147
148
  raw = @stdin.read
148
149
  payload =
149
- if fetcher_name
150
- callable = store.registry.fetcher(fetcher_name)
150
+ if action_name
151
+ callable = store.registry.action(action_name)
151
152
  result =
152
153
  begin
153
- Timeout.timeout(Textus::Refresh::FETCHER_TIMEOUT_SECONDS) do
154
- callable.call(config: { "bytes" => raw }, store: Textus::StoreView.new(store))
154
+ Timeout.timeout(Textus::Refresh::ACTION_TIMEOUT_SECONDS) do
155
+ callable.call(config: { "bytes" => raw }, store: Textus::StoreView.new(store), args: {})
155
156
  end
156
157
  rescue Timeout::Error
157
158
  raise UsageError.new(
158
- "fetcher '#{fetcher_name}' exceeded #{Textus::Refresh::FETCHER_TIMEOUT_SECONDS}s timeout",
159
+ "action '#{action_name}' exceeded #{Textus::Refresh::ACTION_TIMEOUT_SECONDS}s timeout",
159
160
  )
160
161
  end
161
162
  basename = key.split(".").last
@@ -163,7 +164,7 @@ module Textus
163
164
  "frontmatter" => {
164
165
  "name" => basename,
165
166
  "last_refreshed_at" => Time.now.utc.iso8601,
166
- "fetched_with" => fetcher_name,
167
+ "actioned_with" => action_name,
167
168
  }.merge(result[:frontmatter] || result["frontmatter"] || {}),
168
169
  "body" => result[:body] || result["body"] || "",
169
170
  }
@@ -271,6 +272,43 @@ module Textus
271
272
  emit(store.accept(key, as: role))
272
273
  end
273
274
 
275
+ def verb_action(argv)
276
+ name = argv.shift
277
+ raise UsageError.new("action requires a name") if name.nil?
278
+
279
+ as_flag = nil
280
+ args = {}
281
+ argv.each do |tok|
282
+ case tok
283
+ when /\A--as=(.+)\z/ then as_flag = ::Regexp.last_match(1)
284
+ when /\A--format=/ then next
285
+ when /\A--([\w-]+)=(.*)\z/ then args[::Regexp.last_match(1)] = ::Regexp.last_match(2)
286
+ else
287
+ raise UsageError.new("unknown arg to 'action #{name}': #{tok}")
288
+ end
289
+ end
290
+
291
+ role = Role.resolve(flag: as_flag, env: ENV, root: store.root)
292
+ callable = store.registry.action(name)
293
+ view = StoreView.new(store, writable: true, as: role)
294
+
295
+ begin
296
+ Timeout.timeout(Textus::Refresh::ACTION_TIMEOUT_SECONDS) do
297
+ callable.call(config: {}, store: view, args: args)
298
+ end
299
+ rescue Timeout::Error
300
+ raise UsageError.new(
301
+ "action '#{name}' exceeded #{Textus::Refresh::ACTION_TIMEOUT_SECONDS}s timeout",
302
+ )
303
+ rescue Textus::Error
304
+ raise
305
+ rescue StandardError => e
306
+ raise UsageError.new("action '#{name}' raised: #{e.class}: #{e.message}")
307
+ end
308
+
309
+ emit({ "protocol" => Textus::PROTOCOL, "action" => name, "ok" => true })
310
+ end
311
+
274
312
  def verb_refresh(argv)
275
313
  key = argv.shift or raise UsageError.new("refresh requires a key")
276
314
  as_flag = nil
@@ -293,7 +331,8 @@ module Textus
293
331
  end.permute!(argv)
294
332
 
295
333
  rows = []
296
- rows += store.registry.fetcher_names.map { |n| { "kind" => "fetcher", "name" => n.to_s } }
334
+ rows += store.registry.action_names.map { |n| { "kind" => "action", "name" => n.to_s } }
335
+ rows += store.registry.doctor_check_names.map { |n| { "kind" => "doctor_check", "name" => n.to_s } }
297
336
  rows += store.registry.reducer_names.map { |n| { "kind" => "reducer", "name" => n.to_s } }
298
337
  store.registry.hook_events.each do |evt|
299
338
  store.registry.hooks(evt).each do |h|
@@ -388,9 +427,10 @@ module Textus
388
427
  textus list [--prefix=KEY] --format=json
389
428
  textus where KEY --format=json
390
429
  textus get KEY --format=json
391
- textus put KEY --stdin --format=json
430
+ textus put KEY --stdin [--action=NAME] --format=json
392
431
  textus schema KEY --format=json
393
432
  textus stale [--prefix=KEY] --format=json
433
+ textus action NAME [--key=val ...] [--as=ROLE] --format=json
394
434
  HELP
395
435
  end
396
436
  end
data/lib/textus/doctor.rb CHANGED
@@ -1,12 +1,14 @@
1
1
  require "digest"
2
2
  require "json"
3
+ require "timeout"
3
4
 
4
5
  module Textus
5
6
  # Health check for a Textus store. Returns a JSON-friendly Hash envelope
6
7
  # with an `issues` array and a summary. Each issue is a Hash with
7
8
  # `code`, `level`, `subject`, `message`, and optionally `fix`.
8
- module Doctor
9
+ module Doctor # rubocop:disable Metrics/ModuleLength -- 8 built-in checks + extension dispatch
9
10
  LEVELS = %w[error warning info].freeze
11
+ DOCTOR_CHECK_TIMEOUT_SECONDS = 2
10
12
 
11
13
  module_function
12
14
 
@@ -20,6 +22,7 @@ module Textus
20
22
  issues.concat(check_sentinels(store))
21
23
  issues.concat(check_audit_log(store))
22
24
  issues.concat(check_unowned_schema_fields(store))
25
+ issues.concat(run_registered_checks(store))
23
26
 
24
27
  summary = LEVELS.to_h { |l| [l, issues.count { |i| i["level"] == l }] }
25
28
  {
@@ -255,6 +258,43 @@ module Textus
255
258
  out
256
259
  end
257
260
 
261
+ def run_registered_checks(store)
262
+ out = []
263
+ view = StoreView.new(store)
264
+ store.registry.doctor_check_names.each do |name|
265
+ callable = store.registry.doctor_check(name)
266
+ begin
267
+ result = Timeout.timeout(DOCTOR_CHECK_TIMEOUT_SECONDS) { callable.call(store: view) }
268
+ if result.is_a?(Array)
269
+ out.concat(result.map { |h| h.transform_keys(&:to_s) })
270
+ else
271
+ out << fail_issue(name, code: "doctor_check.bad_return",
272
+ message: "doctor_check '#{name}' returned #{result.class} (expected Array)",
273
+ fix: "return an array of issue hashes from the doctor_check block")
274
+ end
275
+ rescue Timeout::Error
276
+ out << fail_issue(name, code: "doctor_check.timeout",
277
+ message: "doctor_check '#{name}' exceeded #{DOCTOR_CHECK_TIMEOUT_SECONDS}s",
278
+ fix: "shorten the check or split it into smaller checks")
279
+ rescue StandardError => e
280
+ out << fail_issue(name, code: "doctor_check.failed",
281
+ message: "#{e.class}: #{e.message}",
282
+ fix: "fix the doctor_check block in .textus/extensions/")
283
+ end
284
+ end
285
+ out
286
+ end
287
+
288
+ def fail_issue(name, code:, message:, fix:)
289
+ {
290
+ "code" => code,
291
+ "level" => "error",
292
+ "subject" => name.to_s,
293
+ "message" => message,
294
+ "fix" => fix,
295
+ }
296
+ end
297
+
258
298
  # --- Helpers ----------------------------------------------------------
259
299
 
260
300
  def leaf_path_for(store, entry)
@@ -3,16 +3,17 @@ module Textus
3
3
  EVENTS = %i[put delete refresh build accept].freeze
4
4
 
5
5
  def initialize
6
- @fetchers = {}
6
+ @actions = {}
7
7
  @reducers = {}
8
8
  @hooks = {}
9
+ @doctor_checks = {}
9
10
  end
10
11
 
11
- def register_fetcher(name, &blk)
12
+ def register_action(name, &blk)
12
13
  name = name.to_sym
13
- raise UsageError.new("fetcher '#{name}' already registered") if @fetchers.key?(name)
14
+ raise UsageError.new("action '#{name}' already registered") if @actions.key?(name)
14
15
 
15
- @fetchers[name] = blk
16
+ @actions[name] = blk
16
17
  end
17
18
 
18
19
  def register_reducer(name, &blk)
@@ -29,8 +30,15 @@ module Textus
29
30
  (@hooks[event] ||= []) << { name: name.to_sym, callable: blk }
30
31
  end
31
32
 
32
- def fetcher(name)
33
- @fetchers[name.to_sym] or raise UsageError.new("unknown fetcher: #{name}")
33
+ def register_doctor_check(name, &blk)
34
+ name = name.to_sym
35
+ raise UsageError.new("doctor_check '#{name}' already registered") if @doctor_checks.key?(name)
36
+
37
+ @doctor_checks[name] = blk
38
+ end
39
+
40
+ def action(name)
41
+ @actions[name.to_sym] or raise UsageError.new("unknown action: #{name}")
34
42
  end
35
43
 
36
44
  def reducer(name)
@@ -41,8 +49,13 @@ module Textus
41
49
  @hooks[event.to_sym] || []
42
50
  end
43
51
 
44
- def fetcher_names = @fetchers.keys
45
- def reducer_names = @reducers.keys
46
- def hook_events = @hooks.keys
52
+ def doctor_check(name)
53
+ @doctor_checks[name.to_sym] or raise UsageError.new("unknown doctor_check: #{name}")
54
+ end
55
+
56
+ def action_names = @actions.keys
57
+ def reducer_names = @reducers.keys
58
+ def hook_events = @hooks.keys
59
+ def doctor_check_names = @doctor_checks.keys
47
60
  end
48
61
  end
@@ -15,8 +15,8 @@ module Textus
15
15
  raise UsageError.new("no active registry; extension code must be loaded by a Store")
16
16
  end
17
17
 
18
- def self.fetcher(name, &)
19
- current_registry.register_fetcher(name, &)
18
+ def self.action(name, &)
19
+ current_registry.register_action(name, &)
20
20
  end
21
21
 
22
22
  def self.reducer(name, &)
@@ -26,4 +26,8 @@ module Textus
26
26
  def self.hook(event, name, &)
27
27
  current_registry.register_hook(event, name, &)
28
28
  end
29
+
30
+ def self.doctor_check(name, &)
31
+ current_registry.register_doctor_check(name, &)
32
+ end
29
33
  end
data/lib/textus/init.rb CHANGED
@@ -31,12 +31,13 @@ module Textus
31
31
  File.write(File.join(target_root, "extensions", "README.md"), <<~MD)
32
32
  # Extensions
33
33
 
34
- Drop one Ruby file per extension. Three verbs are available:
34
+ Drop one Ruby file per extension. Four verbs are available:
35
35
 
36
36
  ```ruby
37
- Textus.fetcher(:name) { |config:, store:| ... }
38
- Textus.reducer(:name) { |rows:, config:| ... }
39
- Textus.hook(:event, :name) { |key:, envelope:, store:, **kw| ... }
37
+ Textus.action(:name) { |config:, store:, args:| ... }
38
+ Textus.reducer(:name) { |rows:, config:| ... }
39
+ Textus.hook(:event, :name) { |key:, envelope:, **kw| ... }
40
+ Textus.doctor_check(:name) { |store:| ... }
40
41
  ```
41
42
 
42
43
  Events: :put, :delete, :refresh, :build, :accept.
data/lib/textus/intro.rb CHANGED
@@ -13,7 +13,7 @@ module Textus
13
13
  ZONE_PURPOSES = {
14
14
  "canon" => "slow-changing identity; human-only writes",
15
15
  "working" => "active project state; humans, AI, and scripts share this surface",
16
- "intake" => "declared external inputs; script-refreshed via fetchers",
16
+ "intake" => "declared external inputs; script-refreshed via actions",
17
17
  "pending" => "AI proposals awaiting human accept",
18
18
  "derived" => "build-computed outputs; never hand-edited",
19
19
  }.freeze
@@ -22,7 +22,7 @@ module Textus
22
22
  "human" => "edit files in canon/working zones, then 'textus put KEY --as=human'",
23
23
  "ai" => "propose changes by writing 'pending.*' entries with --as=ai and a 'proposal:' frontmatter block; " \
24
24
  "a human runs 'textus accept' to apply",
25
- "script" => "refresh intake entries with 'textus refresh KEY --as=script' (uses the entry's declared fetcher)",
25
+ "script" => "refresh intake entries with 'textus refresh KEY --as=script' (uses the entry's declared action)",
26
26
  "build" => "'textus build' computes derived entries from projections; derived files are never hand-edited",
27
27
  }.freeze
28
28
 
@@ -40,11 +40,11 @@ module Textus
40
40
  { "name" => "mv", "summary" => "rename a key in place; uid preserved, audit row written" },
41
41
  { "name" => "delete", "summary" => "delete an entry; --as=<role>" },
42
42
  { "name" => "build", "summary" => "materialize derived entries; publish_to and publish_each fan out copies" },
43
- { "name" => "refresh", "summary" => "run a fetcher for an intake entry" },
43
+ { "name" => "refresh", "summary" => "run an action for an intake entry" },
44
44
  { "name" => "stale", "summary" => "list derived/intake entries past their freshness check" },
45
45
  { "name" => "doctor", "summary" => "health-check the store (missing schemas, illegal keys, sentinel drift, etc.)" },
46
46
  { "name" => "migrate-keys", "summary" => "rename files whose basenames violate the strict key grammar" },
47
- { "name" => "extensions", "summary" => "list registered reducers/fetchers/hooks" },
47
+ { "name" => "extensions", "summary" => "list registered actions, reducers, doctor_checks, declared hooks" },
48
48
  ].freeze
49
49
 
50
50
  def self.run(store)
@@ -80,7 +80,7 @@ module Textus
80
80
  "owner" => e.owner,
81
81
  "format" => e.format,
82
82
  "derived" => derived,
83
- "intake" => !e.fetcher.nil?,
83
+ "intake" => !e.action.nil?,
84
84
  "publish_to" => Array(e.publish_to),
85
85
  "publish_each" => e.publish_each,
86
86
  }
@@ -90,13 +90,15 @@ module Textus
90
90
  def self.extensions_for(store)
91
91
  reg = store.registry
92
92
  reducers = reg.reducer_names.map(&:to_s).sort
93
- fetchers = reg.fetcher_names.map(&:to_s).sort
93
+ actions = reg.action_names.map(&:to_s).sort
94
+ doctor_checks = reg.doctor_check_names.map(&:to_s).sort
94
95
  hooks = reg.hook_events.flat_map do |evt|
95
96
  reg.hooks(evt).map { |h| { "event" => evt.to_s, "name" => h[:name].to_s } }
96
97
  end.sort_by { |h| [h["event"], h["name"]] }
97
98
  {
98
99
  "reducers" => reducers,
99
- "fetchers" => fetchers,
100
+ "actions" => actions,
101
+ "doctor_checks" => doctor_checks,
100
102
  "hooks" => hooks,
101
103
  }
102
104
  end
@@ -199,7 +199,7 @@ module Textus
199
199
  PUBLISH_EACH_VAR_RE = /\{([a-z]+)\}/
200
200
 
201
201
  attr_reader :key, :path, :zone, :schema, :owner, :nested, :generator, :raw, :format,
202
- :projection, :template, :publish_to, :publish_each, :fetcher, :fetcher_config, :ttl, :events,
202
+ :projection, :template, :publish_to, :publish_each, :action, :action_config, :ttl, :events,
203
203
  :inject_intro
204
204
 
205
205
  def initialize(manifest, raw)
@@ -361,8 +361,8 @@ module Textus
361
361
 
362
362
  def parse_source!(src)
363
363
  src ||= {}
364
- @fetcher = src["fetcher"]
365
- @fetcher_config = src["config"] || {}
364
+ @action = src["action"]
365
+ @action_config = src["config"] || {}
366
366
  @ttl = src["ttl"]
367
367
  end
368
368
 
@@ -371,7 +371,13 @@ module Textus
371
371
  if src.key?("parse") || src.key?("from")
372
372
  raise UsageError.new(
373
373
  "entry '#{@key}': source.parse/source.from removed in 0.2; " \
374
- "use source.fetcher (+ source.config). See SPEC §5.4.",
374
+ "use source.action (+ source.config). See SPEC §5.4.",
375
+ )
376
+ end
377
+ if src.key?("fetcher")
378
+ raise UsageError.new(
379
+ "entry '#{@key}': source.fetcher renamed to source.action in 0.4; " \
380
+ "rename the key. See SPEC §5.4.",
375
381
  )
376
382
  end
377
383
  if raw.key?("hooks")
@@ -2,27 +2,29 @@ require "timeout"
2
2
 
3
3
  module Textus
4
4
  module Refresh
5
- FETCHER_TIMEOUT_SECONDS = 2
5
+ ACTION_TIMEOUT_SECONDS = 2
6
6
 
7
7
  def self.call(store, key, as:)
8
8
  mentry, path, = store.manifest.resolve(key)
9
- raise UsageError.new("no fetcher declared for '#{key}'") unless mentry.fetcher
9
+ raise UsageError.new("no action declared for '#{key}'") unless mentry.action
10
10
 
11
11
  before_etag = File.exist?(path) ? Etag.for_file(path) : nil
12
- callable = store.registry.fetcher(mentry.fetcher)
13
- view = StoreView.new(store)
12
+ callable = store.registry.action(mentry.action)
13
+ view = StoreView.new(store, writable: true, as: as)
14
14
  result =
15
15
  begin
16
- Timeout.timeout(FETCHER_TIMEOUT_SECONDS) { callable.call(config: mentry.fetcher_config, store: view) }
16
+ Timeout.timeout(ACTION_TIMEOUT_SECONDS) do
17
+ callable.call(config: mentry.action_config, store: view, args: {})
18
+ end
17
19
  rescue Timeout::Error
18
- raise UsageError.new("fetcher '#{mentry.fetcher}' exceeded #{FETCHER_TIMEOUT_SECONDS}s timeout")
20
+ raise UsageError.new("action '#{mentry.action}' exceeded #{ACTION_TIMEOUT_SECONDS}s timeout")
19
21
  rescue Textus::Error
20
22
  raise
21
23
  rescue StandardError => e
22
- raise UsageError.new("fetcher '#{mentry.fetcher}' raised: #{e.class}: #{e.message}")
24
+ raise UsageError.new("action '#{mentry.action}' raised: #{e.class}: #{e.message}")
23
25
  end
24
26
 
25
- normalized = normalize_fetcher_result(result, format: mentry.format)
27
+ normalized = normalize_action_result(result, format: mentry.format)
26
28
  envelope = store.put(
27
29
  key,
28
30
  frontmatter: normalized[:frontmatter],
@@ -43,9 +45,9 @@ module Textus
43
45
  envelope
44
46
  end
45
47
 
46
- # Normalize the three accepted fetcher return shapes into the store's
47
- # internal {frontmatter, body, content} representation. See plan-1.2 §7.
48
- def self.normalize_fetcher_result(res, format:)
48
+ # Normalize the three accepted action return shapes into the store's
49
+ # internal {frontmatter, body, content} representation.
50
+ def self.normalize_action_result(res, format:)
49
51
  res = res.transform_keys(&:to_s) if res.is_a?(Hash)
50
52
  res ||= {}
51
53
  fm = res["frontmatter"]
@@ -62,10 +64,9 @@ module Textus
62
64
  meta = content.is_a?(Hash) && content["_meta"].is_a?(Hash) ? content["_meta"] : {}
63
65
  { frontmatter: meta, body: nil, content: content }
64
66
  elsif !body.nil?
65
- # Store#put will re-parse and validate the bytes.
66
67
  { frontmatter: {}, body: body.to_s, content: nil }
67
68
  else
68
- raise UsageError.new("fetcher for #{format} returned neither content nor body")
69
+ raise UsageError.new("action for #{format} returned neither content nor body")
69
70
  end
70
71
  else
71
72
  raise UsageError.new("unknown format #{format.inspect}")
data/lib/textus/store.rb CHANGED
@@ -51,7 +51,7 @@ module Textus
51
51
 
52
52
  def load_extensions
53
53
  Textus.with_registry(@registry) do
54
- BuiltinFetchers.register_all
54
+ BuiltinActions.register_all
55
55
  dir = File.join(@root, "extensions")
56
56
  return unless File.directory?(dir)
57
57
 
@@ -296,7 +296,7 @@ module Textus
296
296
  end
297
297
 
298
298
  @manifest.entries.each do |mentry|
299
- next unless mentry.fetcher
299
+ next unless mentry.action
300
300
  next if zone && mentry.zone != zone
301
301
  next if prefix && !(mentry.key == prefix || mentry.key.start_with?("#{prefix}."))
302
302
 
@@ -529,7 +529,7 @@ module Textus
529
529
  end
530
530
 
531
531
  def intake_stale_row(mentry, path, reason)
532
- { "key" => mentry.key, "path" => path, "fetcher" => mentry.fetcher, "reason" => reason }
532
+ { "key" => mentry.key, "path" => path, "action" => mentry.action, "reason" => reason }
533
533
  end
534
534
 
535
535
  def stale_row(mentry, path, reason)
@@ -3,8 +3,12 @@ module Textus
3
3
  READ_METHODS = %i[get list where schema_envelope deps rdeps published stale validate_all].freeze
4
4
  WRITE_METHODS = %i[put delete accept].freeze
5
5
 
6
- def initialize(store)
6
+ def initialize(store, writable: false, as: nil)
7
+ raise UsageError.new("writable StoreView requires an as: role") if writable && (as.nil? || as.to_s.empty?)
8
+
7
9
  @store = store
10
+ @writable = writable
11
+ @as = as
8
12
  end
9
13
 
10
14
  READ_METHODS.each do |m|
@@ -12,7 +16,12 @@ module Textus
12
16
  end
13
17
 
14
18
  WRITE_METHODS.each do |m|
15
- define_method(m) { |*_args, **_kw| raise UsageError.new("StoreView is read-only") }
19
+ define_method(m) do |*args, **kw|
20
+ raise UsageError.new("StoreView is read-only") unless @writable
21
+
22
+ kw[:as] = @as unless kw.key?(:as)
23
+ @store.public_send(m, *args, **kw)
24
+ end
16
25
  end
17
26
  end
18
27
  end
@@ -1,4 +1,4 @@
1
1
  module Textus
2
- VERSION = "0.3.0"
2
+ VERSION = "0.4.0"
3
3
  PROTOCOL = "textus/1"
4
4
  end
data/lib/textus.rb CHANGED
@@ -19,7 +19,7 @@ require_relative "textus/store_view"
19
19
  require_relative "textus/refresh"
20
20
  require_relative "textus/mustache"
21
21
  require_relative "textus/projection"
22
- require_relative "textus/builtin_fetchers"
22
+ require_relative "textus/builtin_actions"
23
23
  require_relative "textus/publisher"
24
24
  require_relative "textus/builder"
25
25
  require_relative "textus/proposal"
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: textus
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.3.0
4
+ version: 0.4.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Patrick
@@ -97,7 +97,7 @@ files:
97
97
  - lib/textus.rb
98
98
  - lib/textus/audit_log.rb
99
99
  - lib/textus/builder.rb
100
- - lib/textus/builtin_fetchers.rb
100
+ - lib/textus/builtin_actions.rb
101
101
  - lib/textus/cli.rb
102
102
  - lib/textus/dependencies.rb
103
103
  - lib/textus/doctor.rb