textus 0.10.1 → 0.10.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +53 -0
- data/README.md +3 -5
- data/SPEC.md +41 -62
- data/docs/conventions.md +1 -3
- data/lib/textus/cli.rb +1 -6
- data/lib/textus/doctor/check/audit_log.rb +20 -36
- data/lib/textus/doctor/check/manifest_files.rb +2 -15
- data/lib/textus/doctor/check/schema_parse_error.rb +28 -0
- data/lib/textus/doctor/check/sentinels.rb +42 -41
- data/lib/textus/doctor/check/unowned_schema_fields.rb +27 -21
- data/lib/textus/doctor.rb +1 -1
- data/lib/textus/infra/publisher.rb +5 -45
- data/lib/textus/manifest/entry.rb +4 -29
- data/lib/textus/manifest.rb +1 -20
- data/lib/textus/schema.rb +10 -0
- data/lib/textus/store/audit_log.rb +24 -11
- data/lib/textus/store/sentinel.rb +86 -0
- data/lib/textus/version.rb +1 -1
- metadata +3 -2
- data/lib/textus/doctor/check/legacy_intake_fields.rb +0 -57
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: c38f76b221f200d4af26262a94de566a9f43949282c0a060472ed85974657ac5
|
|
4
|
+
data.tar.gz: 86231d26523777d9b6751e33c09138efb921bc36ed446c32ae4537371515e7dd
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: bc9c2677e749237c9bc78a3b4005e4cddd1228f95044e60afc04a2ac3323dd266aa02bffff9173d7b70d5c0cb33d04a08375ecefbf0ad0b852b43a294443fcf8
|
|
7
|
+
data.tar.gz: df18177f6df9bbcc9bb8467b2f4736e0d71f2a78a0f7dfed74865ad0f7e8b84028c0c75690357367bde106a5c5d779b03fee4f6d5e73b8e245c7e14c104d9c30
|
data/CHANGELOG.md
CHANGED
|
@@ -8,6 +8,59 @@ The **gem version** (`0.x.y`) is distinct from the **protocol version**
|
|
|
8
8
|
(currently `textus/2`, embedded in every envelope as `protocol`). The protocol
|
|
9
9
|
is additive within a major; a new major would change the wire string.
|
|
10
10
|
|
|
11
|
+
## 0.10.3 — Documentation refresh and legacy-code removal (2026-05-23)
|
|
12
|
+
|
|
13
|
+
Patch release. Two pieces of work: (1) docs describe current state only — every reference to pre-0.9.2 zone names, pre-0.10.2 sentinel layout, pre-0.5 audit-log format, and other version-history annotations is stripped from user-facing docs; (2) the corresponding backward-compatibility code paths are deleted from `lib/`. The wire protocol stays `textus/2`. Callers conforming to the current SPEC are unaffected; callers carrying obsolete config now hit silent drops or parse failures instead of helpful migration messages.
|
|
14
|
+
|
|
15
|
+
### Removed
|
|
16
|
+
|
|
17
|
+
- **`Doctor::Check::LegacyIntakeFields`** — deleted. Manifest parsing already rejected these fields at load; the doctor check was redundant.
|
|
18
|
+
- **TSV audit-log reader** in `Store::AuditLog#parse_row` and `#check_line` — pre-0.5 audit logs are no longer transparently read. Non-JSON lines surface as `invalid_json` integrity violations.
|
|
19
|
+
- **Legacy sibling sentinel migration** in `Infra::Publisher` and `Store::Sentinel.legacy_path` — pre-0.10.2 stores with sibling `<target>.textus-managed.json` files are no longer recognized as managed. Fix: `rm <target>.textus-managed.json && textus build`.
|
|
20
|
+
- **Manifest rename-migration rejections** — entries containing `source:`, `intake.fetch`, `intake.{ttl,on_stale,sync_budget_ms}`, or `projection.reducer` no longer raise migration hints. These obsolete fields are silently ignored by the parser.
|
|
21
|
+
- **`textus/1` helpful-message branch** in `Manifest.load` — unsupported versions now produce a single generic error.
|
|
22
|
+
- **`textus stale` CLI stub** — removed; calling it now returns `unknown verb: stale` like any other typo.
|
|
23
|
+
- **`Manifest::Entry#derived?`** alias — both internal callers now invoke `in_generator_zone?` directly.
|
|
24
|
+
- **Stale CLI help text** — `textus migrate {zones,policies}` (reverted in 0.9.2) and "`--format=json` accepted for back-compat" wording removed from `textus --help`.
|
|
25
|
+
|
|
26
|
+
### Documentation
|
|
27
|
+
|
|
28
|
+
- **`docs/plans/`** is now signposted in `CONTRIBUTING.md` as design history, not current documentation.
|
|
29
|
+
- **README, SPEC, ARCHITECTURE, docs/zones, docs/events, docs/conventions, examples/claude-plugin/README** — stripped `(0.8.2+)`, `(0.9.0+)`, `(0.9.2)`, `(v1.0)`, `(v1.1)`, `(v1.2)`, `(v0.3)` annotations from headings, parentheticals, and inline notes. Removed "Renamed in 0.9.2" / "Pre-0.9.2 stores" / "New in 0.9.0" / "Backward compatibility (v0.5)" callouts. Example code that used pre-0.9.2 zone names (`canon`, `intake`, `pending`, `derived`) now uses current names (`identity`, `inbox`, `review`, `output`).
|
|
30
|
+
- **`docs/events.md`** — header count corrected to "15 events: 3 RPC and 12 pub-sub" (previously read "12 events", with refresh\_\* mentioned in subtext); stale Linear manifest example updated to use top-level `policies:` block.
|
|
31
|
+
- **SPEC.md §10.2** — removed `legacy_intake_fields` from the builtin doctor-check list.
|
|
32
|
+
- **SPEC.md §11** — dropped `textus/1` back-compat acceptance from the implementation checklist; the spec no longer mentions the legacy v0.1 zone-synthesis fallback.
|
|
33
|
+
- **CHANGELOG entries for past releases are unchanged** — historical record stays intact.
|
|
34
|
+
|
|
35
|
+
### Tests
|
|
36
|
+
|
|
37
|
+
- Removed `spec/doctor/check/legacy_intake_fields_spec.rb`.
|
|
38
|
+
- Removed 4 manifest-intake migration-rejection specs and 3 publisher/audit-log legacy-format specs.
|
|
39
|
+
|
|
40
|
+
## 0.10.2 — Doctor and store cleanup (2026-05-23)
|
|
41
|
+
|
|
42
|
+
Patch release. Internal cleanup: extracts `Store::Sentinel`, moves audit-log integrity into `Store::AuditLog`, surfaces previously-swallowed schema parse errors, and tidies two doctor checks. No CLI, wire-protocol, or behavioral changes for plugin authors. Sentinel JSON shape changes (repo-relative paths) are forward-compatible; legacy absolute paths are still read correctly.
|
|
43
|
+
|
|
44
|
+
### Added
|
|
45
|
+
|
|
46
|
+
- `Textus::Store::Sentinel` value object owning the sentinel JSON shape (`source`/`target`/`sha256`/`mode`) and the on-disk path layout. Repo-relative paths on write; legacy absolute paths still accepted on read.
|
|
47
|
+
- `Textus::Store::AuditLog#verify_integrity` returns line-by-line integrity violations as `{lineno, reason, detail}` hashes.
|
|
48
|
+
- `Textus::Schema#unowned_fields` returns field names whose spec lacks `maintained_by`.
|
|
49
|
+
- New doctor check `schema_parse_error` (error level) surfaces YAML parse failures on `schemas/*.yaml`. Previously these were silently rescued in `UnownedSchemaFields`, leaving operators with no signal.
|
|
50
|
+
|
|
51
|
+
### Changed
|
|
52
|
+
|
|
53
|
+
- `Infra::Publisher` delegates sentinel I/O to `Store::Sentinel`. The sentinel JSON now stores repo-relative `source`/`target` so example trees can be committed without leaking author paths.
|
|
54
|
+
- `Doctor::Check::Sentinels` delegates parse/orphan/drift detection to `Store::Sentinel`. Drops `rubocop:disable Metrics/BlockLength`.
|
|
55
|
+
- `Doctor::Check::AuditLog` delegates parsing to `Store::AuditLog#verify_integrity`. Drops `rubocop:disable Metrics/BlockLength`.
|
|
56
|
+
- `Doctor::Check::ManifestFiles` uses `Textus::Key::Path.resolve` instead of reimplementing leaf-path math.
|
|
57
|
+
- `Doctor::Check::UnownedSchemaFields` uses `Schema#unowned_fields` instead of reaching into `schema.fields` and the raw `maintained_by` Hash key.
|
|
58
|
+
- `examples/claude-plugin/.gitignore` no longer excludes `.textus/sentinels/`. The example's sentinels are now committed with repo-relative paths.
|
|
59
|
+
|
|
60
|
+
### Documentation
|
|
61
|
+
|
|
62
|
+
- `SPEC.md` builtin doctor-check list updated to include `schema_parse_error`, and brings the prose up to date with three checks shipped in 0.9.x/0.10.0 that were missing from the list (`policy_ambiguity`, `handler_allowlist`, `legacy_intake_fields`).
|
|
63
|
+
|
|
11
64
|
## 0.10.1 — Documentation refresh and spec hygiene (2026-05-22)
|
|
12
65
|
|
|
13
66
|
Lightweight maintenance release: documentation refresh plus spec-suite hygiene. No `lib/` changes; no CLI, wire-protocol, or behavioral changes.
|
data/README.md
CHANGED
|
@@ -14,7 +14,7 @@ Reference implementation in Ruby. Wire format `textus/2`. SPEC: [`SPEC.md`](SPEC
|
|
|
14
14
|
Two versions, deliberately independent:
|
|
15
15
|
|
|
16
16
|
- **Protocol wire string:** `textus/2`. Stable; breaking changes require `textus/3`.
|
|
17
|
-
- **Gem version:** semver, currently `0.
|
|
17
|
+
- **Gem version:** semver, currently `0.10.3`. The gem version is decoupled from the protocol string — internal refactors bump the gem; only wire-format changes bump the protocol.
|
|
18
18
|
|
|
19
19
|
Envelope payloads carry the `protocol` field. The gem version is irrelevant to the wire format.
|
|
20
20
|
|
|
@@ -57,8 +57,6 @@ You get `.textus/` with all five zone directories, baseline schemas, an empty au
|
|
|
57
57
|
|
|
58
58
|
Manifest `path:` fields are relative to `.textus/zones/`. So `working.network.org.jane` lives at `.textus/zones/working/network/org/jane.md`.
|
|
59
59
|
|
|
60
|
-
> **Renamed in 0.9.2.** Pre-0.9.2 defaults were `canon`, `intake`, `pending`, `derived`. `working` is unchanged. Upgrading a 0.9.1 store is a hand-edit (see CHANGELOG migration recipe).
|
|
61
|
-
|
|
62
60
|
Read and write:
|
|
63
61
|
|
|
64
62
|
```sh
|
|
@@ -164,7 +162,7 @@ textus exposes a hook DSL. Drop `.rb` files into `.textus/hooks/` (subdirectorie
|
|
|
164
162
|
- `:reduce` — transform rows during projection (returns rows)
|
|
165
163
|
- `:check` — custom doctor check (returns issues)
|
|
166
164
|
- `:put`, `:deleted`, `:refreshed`, `:built`, `:accepted`, `:published`, `:mv`, `:reject`, `:loaded` — react to lifecycle events
|
|
167
|
-
- `:refresh_began`, `:refresh_failed`, `:refresh_detached` — background-refresh lifecycle
|
|
165
|
+
- `:refresh_began`, `:refresh_failed`, `:refresh_detached` — background-refresh lifecycle
|
|
168
166
|
|
|
169
167
|
```ruby
|
|
170
168
|
# Inside .textus/hooks/local_file.rb
|
|
@@ -205,7 +203,7 @@ Schemas (`.textus/schemas/<name>.yaml`) declare field shapes, per-field `maintai
|
|
|
205
203
|
bundle exec rspec
|
|
206
204
|
```
|
|
207
205
|
|
|
208
|
-
|
|
206
|
+
~490 examples; includes conformance fixtures A–I from SPEC §12.
|
|
209
207
|
|
|
210
208
|
## Code quality
|
|
211
209
|
|
data/SPEC.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# textus/2 — Specification
|
|
2
2
|
|
|
3
|
-
**Status:** Draft v2.0
|
|
3
|
+
**Status:** Draft v2.0
|
|
4
4
|
**Protocol identifier:** `textus/2`
|
|
5
5
|
**Reference implementation:** Ruby gem `textus`
|
|
6
6
|
|
|
@@ -51,7 +51,7 @@ textus is organized as five composable layers. Each layer has a single responsib
|
|
|
51
51
|
|
|
52
52
|
## 3. Storage layout
|
|
53
53
|
|
|
54
|
-
The root is `.textus/` at the project working directory. A typical
|
|
54
|
+
The root is `.textus/` at the project working directory. A typical tree:
|
|
55
55
|
|
|
56
56
|
```
|
|
57
57
|
.textus/
|
|
@@ -75,7 +75,7 @@ Zone directories under `zones/` are conventional; their write semantics are decl
|
|
|
75
75
|
|
|
76
76
|
`.textus/audit.log` is an append-only NDJSON file written under a file lock by every successful `put`, `delete`, `accept`, and `build`. `.textus/role` (one line containing a role name) is optional and participates in the role-resolution order (§5).
|
|
77
77
|
|
|
78
|
-
### 3.1 Store location precedence
|
|
78
|
+
### 3.1 Store location precedence
|
|
79
79
|
|
|
80
80
|
Implementations MUST resolve the store root in this order; the first match wins:
|
|
81
81
|
|
|
@@ -129,25 +129,11 @@ policies:
|
|
|
129
129
|
refresh: { ttl: 6h, on_stale: warn }
|
|
130
130
|
```
|
|
131
131
|
|
|
132
|
-
|
|
132
|
+
Zone names are conventional — the manifest is the source of truth for write permissions; rename freely.
|
|
133
133
|
|
|
134
|
-
**
|
|
134
|
+
**Key grammar:** dotted segments matching `/^[a-z0-9][a-z0-9-]*$/`. Segments are joined by `.`. A key has at most 8 segments; each segment is at most 64 characters. Segments MUST NOT contain dots, slashes, uppercase letters, or underscores. Example: `working.projects.acme.dashboard`. Enforcement points: manifest load (rejects illegal `key:` declarations and illegal nested file/directory names), `put` (rejects illegal keys before any write), `enumerate` (filters and warns on illegal filenames).
|
|
135
135
|
|
|
136
|
-
|
|
137
|
-
zones:
|
|
138
|
-
- name: fixed
|
|
139
|
-
writable_by: [human]
|
|
140
|
-
- name: state
|
|
141
|
-
writable_by: [human, ai, script]
|
|
142
|
-
- name: derived
|
|
143
|
-
writable_by: [build]
|
|
144
|
-
```
|
|
145
|
-
|
|
146
|
-
Old manifests written against textus/1 draft v0.1 therefore parse without modification, and any tooling expecting `fixed`/`state`/`derived` continues to work.
|
|
147
|
-
|
|
148
|
-
**Key grammar (enforced from v1.2):** dotted segments matching `/^[a-z0-9][a-z0-9-]*$/`. Segments are joined by `.`. A key has at most 8 segments; each segment is at most 64 characters. Segments MUST NOT contain dots, slashes, uppercase letters, or underscores. Example: `working.projects.acme.dashboard`. Enforcement points: manifest load (rejects illegal `key:` declarations and illegal nested file/directory names), `put` (rejects illegal keys before any write), `enumerate` (filters and warns on illegal filenames so existing trees still load with a clear migration message). Run-once migration: `textus key migrate --dry-run` then `--write` (see §audit).
|
|
149
|
-
|
|
150
|
-
**Per-entry `format:` (enforced from v1.2):** an entry MAY declare `format:` to be one of `markdown` (default), `json`, `yaml`, or `text`. The `format` controls the on-disk shape and which path extension is required:
|
|
136
|
+
**Per-entry `format:`** an entry MAY declare `format:` to be one of `markdown` (default), `json`, `yaml`, or `text`. The `format` controls the on-disk shape and which path extension is required:
|
|
151
137
|
|
|
152
138
|
| `format` | Path extension | `template:` | `schema:` |
|
|
153
139
|
|------------|-----------------------------|------------------------|-----------|
|
|
@@ -158,7 +144,7 @@ Old manifests written against textus/1 draft v0.1 therefore parse without modifi
|
|
|
158
144
|
|
|
159
145
|
For `nested: true`, the recursive glob matches the format's extension (markdown→`**/*.md`, json→`**/*.json`, yaml→`**/*.{yaml,yml}`, text→`**/*.txt`). All files under one nested entry share one format and one schema.
|
|
160
146
|
|
|
161
|
-
**Per-leaf publishing (`publish_each
|
|
147
|
+
**Per-leaf publishing (`publish_each:`).** A nested manifest entry MAY declare `publish_each:` to byte-copy every leaf to a templated repo-relative path. `publish_each:` and `publish_to:` are mutually exclusive on the same entry, and `publish_each:` requires `nested: true`. The template substitutes these variables (using `{name}` syntax):
|
|
162
148
|
|
|
163
149
|
| Variable | Value |
|
|
164
150
|
|--------------|----------------------------------------------------------------------------------------|
|
|
@@ -180,7 +166,7 @@ Validation at manifest load: any unknown variable raises `UsageError`; the templ
|
|
|
180
166
|
|
|
181
167
|
A leaf at `working.skills.writing.voice-writer` (authored at `.textus/zones/working/skills/writing/voice-writer.md`) publishes to `skills/voice-writer/SKILL.md`.
|
|
182
168
|
|
|
183
|
-
**`inject_intro
|
|
169
|
+
**`inject_intro:`.** A derived entry with a `template:` MAY declare `inject_intro: true`. When the builder materializes the entry, it merges the `textus intro` envelope (§9) into the projection data under the key `intro`, so the template can render orientation content (zones, write flows, CLI catalog) alongside its projected rows. The flag is rejected at manifest load on (a) non-derived entries or (b) derived entries without a `template:` — agents reading the rendered file should be able to trust the preamble was produced by the same source of truth `textus intro` exposes.
|
|
184
170
|
|
|
185
171
|
**Lookup rule:** to resolve a key, find the entry with the longest `key:` prefix that matches. If that entry has `nested: true`, the remaining segments map to subdirectories under its `path`. Otherwise the key must equal an entry exactly. The resolved filesystem path is `<.textus root>/zones/<entry.path>[/<remaining>...].md` — implementations MUST prepend `zones/` to the manifest `path:` when constructing the filesystem location.
|
|
186
172
|
|
|
@@ -190,11 +176,11 @@ Each zone declares which **roles** may write to it via `writable_by:` in the man
|
|
|
190
176
|
|
|
191
177
|
| Zone | `writable_by` | Use case |
|
|
192
178
|
|---|---|---|
|
|
193
|
-
| `identity` | `[human]` | Identity, voice, immutable principles — things only a human edits.
|
|
179
|
+
| `identity` | `[human]` | Identity, voice, immutable principles — things only a human edits. |
|
|
194
180
|
| `working` | `[human, ai, script]` | Active project state: notes, decisions, network — what humans and agents update day-to-day. |
|
|
195
|
-
| `inbox` | `[script]` | Declared external inputs (calendar, feeds, scraped pages). Refreshed by external runner scripts; never by humans or AI directly.
|
|
196
|
-
| `review` | `[ai, human]` | AI-generated proposals awaiting human review via `textus accept`. Lets agents stage changes without touching `working`.
|
|
197
|
-
| `output` | `[build]` | Computed outputs (catalogs, indexes, published context). Written only by the build runner via `textus build`.
|
|
181
|
+
| `inbox` | `[script]` | Declared external inputs (calendar, feeds, scraped pages). Refreshed by external runner scripts; never by humans or AI directly. |
|
|
182
|
+
| `review` | `[ai, human]` | AI-generated proposals awaiting human review via `textus accept`. Lets agents stage changes without touching `working`. |
|
|
183
|
+
| `output` | `[build]` | Computed outputs (catalogs, indexes, published context). Written only by the build runner via `textus build`. |
|
|
198
184
|
|
|
199
185
|
A write is gated by the caller's **role**, supplied via `--as=<role>`. If the role is not in the target zone's `writable_by` list, the write returns `write_forbidden`.
|
|
200
186
|
|
|
@@ -207,7 +193,7 @@ The effective role for any CLI invocation is resolved in this order; the first m
|
|
|
207
193
|
3. `.textus/role` file (one line, role name) at the project root.
|
|
208
194
|
4. Default: `human`.
|
|
209
195
|
|
|
210
|
-
Recognized roles
|
|
196
|
+
Recognized roles: `human`, `ai`, `script`, `build`. Unknown roles are rejected with `invalid_role`. The roles list is intentionally open-ended: a future minor revision MAY introduce additional roles without breaking the wire string.
|
|
211
197
|
|
|
212
198
|
Every successful write records the resolved role and a wall-clock timestamp in `.textus/audit.log`, so reviewers can later distinguish a human edit from an agent edit even though both live in the same file.
|
|
213
199
|
|
|
@@ -276,11 +262,11 @@ policies:
|
|
|
276
262
|
sync_budget_ms: 500 # only used when on_stale: timed_sync (default: 500)
|
|
277
263
|
```
|
|
278
264
|
|
|
279
|
-
`handler` names a registered `:intake` hook (see §5.10 for the hook contract); `config` is an opaque hash handed to the handler. The freshness budget (`ttl`, `on_stale`, `sync_budget_ms`) lives in a top-level **`policies:`** block matched by key glob (§5.11).
|
|
265
|
+
`handler` names a registered `:intake` hook (see §5.10 for the hook contract); `config` is an opaque hash handed to the handler. The freshness budget (`ttl`, `on_stale`, `sync_budget_ms`) lives in a top-level **`policies:`** block matched by key glob (§5.11).
|
|
280
266
|
|
|
281
267
|
#### `on_stale:` semantics
|
|
282
268
|
|
|
283
|
-
`on_stale:` declares what happens when `textus get` (or any read path that annotates freshness) encounters a stale intake entry. The value lives on the matching policy block, not on the entry. Vocabulary
|
|
269
|
+
`on_stale:` declares what happens when `textus get` (or any read path that annotates freshness) encounters a stale intake entry. The value lives on the matching policy block, not on the entry. Vocabulary: `warn | sync | timed_sync`.
|
|
284
270
|
|
|
285
271
|
| Value | Behaviour |
|
|
286
272
|
|---|---|
|
|
@@ -288,7 +274,7 @@ policies:
|
|
|
288
274
|
| `sync` | Block the `get` call, run the intake handler in-process, write the refreshed result, then return the fresh envelope. The caller waits. |
|
|
289
275
|
| `timed_sync` | Like `sync`, but with a `sync_budget_ms` deadline (default 500 ms). If the handler finishes within the budget the fresh envelope is returned. If it does not finish in time, return the stale envelope (with `stale: true`, `refreshing: true`) and let the refresh complete in the background. Fires `:refresh_detached` when the deadline is exceeded. |
|
|
290
276
|
|
|
291
|
-
> **Note:** `list`/`where` paths do **not** annotate freshness
|
|
277
|
+
> **Note:** `list`/`where` paths do **not** annotate freshness — only `get` does.
|
|
292
278
|
|
|
293
279
|
In intake mode the handler MUST return one of three shapes, all normalized by the store into its internal `{_meta, body, content}` representation (§5.12):
|
|
294
280
|
|
|
@@ -340,8 +326,6 @@ Schema (one JSON object per line, no interior whitespace):
|
|
|
340
326
|
|
|
341
327
|
For `mv`, the structural fields `from_key`, `to_key`, and `uid` appear at the top level of the JSON object. Remaining verb-specific data (e.g. `from_path`, `to_path`) is nested under an `extras` key. The `extras` key is omitted entirely when empty.
|
|
342
328
|
|
|
343
|
-
**Backward compatibility (v0.5):** files written by v0.4 and earlier contain TSV rows. Readers MUST accept mixed-format files: lines starting with `{` are parsed as JSON; other lines are treated as legacy TSV (`ts\trole\tverb\tkey\tetag_before\tetag_after[\tjson_extras]`). TSV write support is removed in v0.6.
|
|
344
|
-
|
|
345
329
|
### 5.7 Security bounds
|
|
346
330
|
|
|
347
331
|
textus enforces fixed bounds to keep behavior predictable under hostile or buggy input:
|
|
@@ -352,9 +336,9 @@ textus enforces fixed bounds to keep behavior predictable under hostile or buggy
|
|
|
352
336
|
- **Entry size:** 1 MB.
|
|
353
337
|
- **Audit log:** unbounded; rotation is the user's problem.
|
|
354
338
|
|
|
355
|
-
### 5.8 Schema evolution
|
|
339
|
+
### 5.8 Schema evolution
|
|
356
340
|
|
|
357
|
-
Schemas may declare per-field ownership and version history.
|
|
341
|
+
Schemas may declare per-field ownership and version history. The `fields:` and `evolution:` blocks are both optional; a schema may omit them and still parse.
|
|
358
342
|
|
|
359
343
|
**`fields:` block** — keyed by field name. Each entry is an object with at least `type`, plus optional `maintained_by` and any vendor extensions:
|
|
360
344
|
|
|
@@ -379,11 +363,11 @@ evolution:
|
|
|
379
363
|
|
|
380
364
|
`textus schema migrate NAME` consults `evolution.migrate_from` when invoked without `--rename=OLD:NEW`, applying every declared rename across affected entries in one pass. An explicit `--rename` flag overrides the schema-declared map for that invocation.
|
|
381
365
|
|
|
382
|
-
**
|
|
366
|
+
**Defaults:** when `fields:` and `evolution:` are absent, `schema.maintained_by(field)` returns `nil` for every field and `schema.evolution` returns `{}`.
|
|
383
367
|
|
|
384
368
|
**Override rule:** the role `human` is permitted to write any `maintained_by` field, regardless of declared owner. This preserves human authority over AI/script-managed data — humans curating canon over AI-written embeddings is a feature, not a bug. All other role mismatches are reported by `doctor --check=schema_violations` with code `role_authority`, including fields `key`, `field`, `expected`, and `last_writer`.
|
|
385
369
|
|
|
386
|
-
### 5.9 Reducers
|
|
370
|
+
### 5.9 Reducers
|
|
387
371
|
|
|
388
372
|
Reducers are RPC hooks on the `:reduce` event. See §5.10.
|
|
389
373
|
|
|
@@ -393,7 +377,7 @@ textus has a single hook verb: `Textus.hook(event, name, **opts) { ... }`. The E
|
|
|
393
377
|
|
|
394
378
|
The subdirectory layout under `hooks/` is organizational only; the registered event and name come from the DSL call, not the file path. Files are loaded in alphabetical order by full path.
|
|
395
379
|
|
|
396
|
-
#### Sugar surface
|
|
380
|
+
#### Sugar surface
|
|
397
381
|
|
|
398
382
|
Per-event methods are provided for ergonomics. They delegate to the same registry as `Textus.hook`.
|
|
399
383
|
|
|
@@ -425,7 +409,7 @@ The primitive `Textus.hook(:event, :name, &blk)` remains supported and is the au
|
|
|
425
409
|
| :refresh_failed | pubsub | store:, key:, error_class:, error_message: | (discarded) | logged |
|
|
426
410
|
| :refresh_detached | pubsub | store:, key:, started_at:, budget_ms: | (discarded) | logged |
|
|
427
411
|
|
|
428
|
-
|
|
412
|
+
The three `:refresh_*` lifecycle events report the progress and failures of background (timed_sync) refreshes.
|
|
429
413
|
|
|
430
414
|
**`:refresh_began`** fires immediately before an intake handler is invoked. `mode:` is one of `"sync"` or `"timed_sync"`.
|
|
431
415
|
|
|
@@ -443,7 +427,7 @@ The `store:` argument is always a read-only store proxy. Write attempts raise `U
|
|
|
443
427
|
|
|
444
428
|
Each handler runs under `Timeout.timeout(2)`.
|
|
445
429
|
|
|
446
|
-
### 5.11 Policies
|
|
430
|
+
### 5.11 Policies
|
|
447
431
|
|
|
448
432
|
A manifest MAY declare a top-level `policies:` block — a list of rule blocks matched against entry keys by glob. Each block carries one or more slots:
|
|
449
433
|
|
|
@@ -464,7 +448,7 @@ policies:
|
|
|
464
448
|
|
|
465
449
|
| Slot | Type | Meaning |
|
|
466
450
|
|---|---|---|
|
|
467
|
-
| `refresh` | `{ ttl, on_stale, sync_budget_ms }` | Freshness budget for intake entries
|
|
451
|
+
| `refresh` | `{ ttl, on_stale, sync_budget_ms }` | Freshness budget for intake entries. `on_stale` is `warn` (default), `sync`, or `timed_sync`. |
|
|
468
452
|
| `handler_allowlist` | list of strings | Constrains which `intake.handler:` names may be used by entries matched by this block. Enforced by `textus doctor`. |
|
|
469
453
|
| `promote_requires` | list of strings | Predicates a `review` entry must satisfy before `textus accept` will promote it. Implementations MAY use a built-in or hook-resolved predicate. Reserved for future enforcement; recorded today. |
|
|
470
454
|
| `retention` | (reserved) | Slot reserved for future retention policy (cap by age / count). Implementations parse it but otherwise ignore. |
|
|
@@ -475,9 +459,7 @@ policies:
|
|
|
475
459
|
|
|
476
460
|
**Read surface.** `textus policy list` dumps every block. `textus policy explain KEY` shows the resolved `PolicySet` for one key plus which block won each slot.
|
|
477
461
|
|
|
478
|
-
|
|
479
|
-
|
|
480
|
-
### 5.12 Storage formats (v1.2)
|
|
462
|
+
### 5.12 Storage formats
|
|
481
463
|
|
|
482
464
|
An entry's `format:` selects a storage strategy. All strategies expose the same `parse(bytes) → {_meta, body, content}` and `serialize(meta:, body:, content:) → bytes` contract. The store, audit, etag, and projection layers operate on the parsed shape; only (de)serialization differs.
|
|
483
465
|
|
|
@@ -573,7 +555,7 @@ Every successful CLI response (`--format=json`) is a single JSON envelope:
|
|
|
573
555
|
**Field rules:**
|
|
574
556
|
- `protocol` MUST be the exact string `textus/2`.
|
|
575
557
|
- `key` MUST be the canonical resolved key.
|
|
576
|
-
- `zone` MUST be one of the zones declared in the manifest (`identity`, `working`, `inbox`, `review`, `output`
|
|
558
|
+
- `zone` MUST be one of the zones declared in the manifest (`identity`, `working`, `inbox`, `review`, `output` in the default scaffold).
|
|
577
559
|
- `path` MUST be an absolute filesystem path.
|
|
578
560
|
- `format` MUST be one of `markdown`, `json`, `yaml`, `text` (§5.12). Absent envelopes are treated as `markdown` for back-compat.
|
|
579
561
|
- `body` is the raw on-disk bytes as a UTF-8 string for every format.
|
|
@@ -581,11 +563,11 @@ Every successful CLI response (`--format=json`) is a single JSON envelope:
|
|
|
581
563
|
- `etag` MUST be `sha256:<hex>` of the raw file bytes, computed identically for every format.
|
|
582
564
|
- `schema_ref` MAY be `null` for entries in subtrees with `schema: null`.
|
|
583
565
|
- `uid` is the stable Textus UID (§7) if the entry carries one, else `null`. Always present in the envelope.
|
|
584
|
-
- `stale` is `true` when the entry's TTL has elapsed and the data has not yet been refreshed; `false` otherwise. Only populated for entries matched by a `refresh:` policy slot (typically `inbox` zone); always `false` elsewhere.
|
|
585
|
-
- `stale_reason` is a short human-readable string describing why the entry is stale (e.g. `"ttl_exceeded"`, `"never_refreshed"`), or `null` when `stale` is `false`.
|
|
586
|
-
- `refreshing` is `true` when a `timed_sync` background refresh is in flight for this entry; `false` otherwise. Callers observing `stale: true, refreshing: true` SHOULD retry after a short delay.
|
|
566
|
+
- `stale` is `true` when the entry's TTL has elapsed and the data has not yet been refreshed; `false` otherwise. Only populated for entries matched by a `refresh:` policy slot (typically `inbox` zone); always `false` elsewhere.
|
|
567
|
+
- `stale_reason` is a short human-readable string describing why the entry is stale (e.g. `"ttl_exceeded"`, `"never_refreshed"`), or `null` when `stale` is `false`.
|
|
568
|
+
- `refreshing` is `true` when a `timed_sync` background refresh is in flight for this entry; `false` otherwise. Callers observing `stale: true, refreshing: true` SHOULD retry after a short delay.
|
|
587
569
|
|
|
588
|
-
> **Note:** `list`/`where` envelopes do **not** include `stale`, `stale_reason`, or `refreshing`
|
|
570
|
+
> **Note:** `list`/`where` envelopes do **not** include `stale`, `stale_reason`, or `refreshing` — freshness annotation is only provided by `get`.
|
|
589
571
|
|
|
590
572
|
Errors use a distinct envelope:
|
|
591
573
|
|
|
@@ -594,8 +576,8 @@ Errors use a distinct envelope:
|
|
|
594
576
|
"protocol": "textus/2",
|
|
595
577
|
"ok": false,
|
|
596
578
|
"code": "write_forbidden",
|
|
597
|
-
"message": "zone '
|
|
598
|
-
"details": { "key": "
|
|
579
|
+
"message": "zone 'identity' is not writable by role 'ai' for key 'identity.self'",
|
|
580
|
+
"details": { "key": "identity.self", "zone": "identity", "role": "ai" }
|
|
599
581
|
}
|
|
600
582
|
```
|
|
601
583
|
|
|
@@ -645,8 +627,6 @@ All verbs accept `--format=json` and emit a canonical envelope (success or error
|
|
|
645
627
|
| `key uid K` | read | any |
|
|
646
628
|
| `hook run NAME` | write | any |
|
|
647
629
|
|
|
648
|
-
**0.9.2 breaking:** `textus stale` was removed; use `textus freshness` (same input, slightly richer output shape — see below).
|
|
649
|
-
|
|
650
630
|
**`put` input** (read from stdin when `--stdin` is given):
|
|
651
631
|
|
|
652
632
|
```json
|
|
@@ -675,7 +655,7 @@ All verbs accept `--format=json` and emit a canonical envelope (success or error
|
|
|
675
655
|
}
|
|
676
656
|
```
|
|
677
657
|
|
|
678
|
-
|
|
658
|
+
Each row reports one entry's verdict (`fresh`, `stale`, `never_refreshed`, or `no_policy`) against its matched `refresh:` policy. `textus build` consumes its own staleness signal and executes derived entries' projections under the `build` role; `--dry-run` prints the plan without executing.
|
|
679
659
|
|
|
680
660
|
`textus accept K --as=human` promotes a pending entry into its target zone: it copies the patch body into the target key, deletes the pending entry, and writes one audit line per side (§audit). Only the `human` role may invoke `accept`.
|
|
681
661
|
|
|
@@ -693,17 +673,16 @@ Every `Textus::Error` exposes `code`, `message`, and an optional `hint:`. The hi
|
|
|
693
673
|
|
|
694
674
|
## 10.2 `textus doctor`
|
|
695
675
|
|
|
696
|
-
`textus doctor` returns a health-check envelope: `{ "protocol": "textus/2", "ok": bool, "issues": [...], "summary": {error, warning, info} }`. Each issue carries `code`, `level` (`error|warning|info`), `subject`, `message`, and optionally `fix`. `ok` is true iff no error-level issues are present; warnings and info do not flip the bit. Builtin checks: `manifest_files`, `schemas`, `templates`, `hooks`, `illegal_keys`, `sentinels`, `audit_log`, `unowned_schema_fields`, `schema_violations`. Additional registered `:check` hooks (§5.10) run after the builtin set. Exit code is 0 on `ok`, 1 otherwise.
|
|
676
|
+
`textus doctor` returns a health-check envelope: `{ "protocol": "textus/2", "ok": bool, "issues": [...], "summary": {error, warning, info} }`. Each issue carries `code`, `level` (`error|warning|info`), `subject`, `message`, and optionally `fix`. `ok` is true iff no error-level issues are present; warnings and info do not flip the bit. Builtin checks: `manifest_files`, `schemas`, `schema_parse_error`, `templates`, `hooks`, `illegal_keys`, `sentinels`, `audit_log`, `unowned_schema_fields`, `schema_violations`, `policy_ambiguity`, `handler_allowlist`. Additional registered `:check` hooks (§5.10) run after the builtin set. Exit code is 0 on `ok`, 1 otherwise.
|
|
697
677
|
|
|
698
678
|
## 11. Versioning
|
|
699
679
|
|
|
700
|
-
- The current wire string is `textus/2`.
|
|
701
|
-
- `textus/1` was the original protocol (gem ≤ v0.4). Manifests declaring `version: textus/1` are still accepted for backward compatibility (§4).
|
|
680
|
+
- The current wire string is `textus/2`.
|
|
702
681
|
- Backward-compatible additions (new fields, new error codes, new schema types) MAY be made under `textus/2`.
|
|
703
682
|
- Breaking changes (renamed/removed envelope fields, zone semantics, key grammar) require a new wire string `textus/3`.
|
|
704
683
|
- Implementations MUST reject envelopes whose `protocol` they do not recognize.
|
|
705
684
|
|
|
706
|
-
The reference Ruby gem follows semver independently
|
|
685
|
+
The reference Ruby gem follows semver independently and speaks `textus/2`.
|
|
707
686
|
|
|
708
687
|
## 12. Conformance fixtures
|
|
709
688
|
|
|
@@ -734,7 +713,7 @@ Given a manifest entry with `publish_to: <path>`, a successful `textus build` fo
|
|
|
734
713
|
Every successful write verb (`put`, `delete`, `build`, `accept`, `schema migrate`) appends exactly one line per affected key to the audit log, in the canonical format defined in §audit (timestamp, actor role, verb, key, etag-before, etag-after). No write produces zero or multiple lines per key.
|
|
735
714
|
|
|
736
715
|
**Fixture I — Pending → accept:**
|
|
737
|
-
Given a
|
|
716
|
+
Given a review entry `review.identity.self.patch` proposing a change to `identity.self`, `textus accept identity.self --as=human` copies the patch body into `identity.self`, deletes the review entry, and appends two audit lines (one for the identity write, one for the review delete) in that order.
|
|
738
717
|
|
|
739
718
|
## 13. Why not X?
|
|
740
719
|
|
|
@@ -752,7 +731,7 @@ Given a pending entry `pending.canon.identity.patch` proposing a change to `cano
|
|
|
752
731
|
|
|
753
732
|
- **Why not vector embeddings?** Different problem. textus is for facts agents act on deterministically; embeddings are for fuzzy retrieval. They compose — index a textus tree into a vector store if you need both.
|
|
754
733
|
|
|
755
|
-
## 13.1 Layered architecture (internal
|
|
734
|
+
## 13.1 Layered architecture (internal)
|
|
756
735
|
|
|
757
736
|
Textus internals are organized into four layers. The dependency rule is one-way — each layer may only import from the layer beneath it.
|
|
758
737
|
|
|
@@ -765,7 +744,7 @@ The `lib/textus/store/`, `lib/textus/manifest/`, `lib/textus/hooks/` namespaces
|
|
|
765
744
|
|
|
766
745
|
Plugin authors interact only with the Hook DSL (`Textus.intake`, `Textus.refreshed`, etc.) and the manifest YAML schema. The layering is internal and may evolve.
|
|
767
746
|
|
|
768
|
-
|
|
747
|
+
Both read and write paths flow through the application layer:
|
|
769
748
|
|
|
770
749
|
- **Reads** flow through `Application::Reads::Get`, which takes a `Context` and dispatches refresh via `Application::Refresh::Orchestrator`.
|
|
771
750
|
- **Writes** flow through `Application::Writes::{Put,Delete,Build,Accept,Publish}`, each taking a `Context`. Permission checks happen at the use-case layer (via `Context#can_write?`); I/O happens at `Store::Writer#write_envelope_to_disk` (pure).
|
|
@@ -777,7 +756,7 @@ See `ARCHITECTURE.md` for an ASCII diagram and the full read-path walkthrough.
|
|
|
777
756
|
## 14. Open questions (v2.x scope)
|
|
778
757
|
|
|
779
758
|
- **Locking on `put`:** the reference impl uses sha256 etags. Should the spec also define a file-lock fallback for systems where read-before-write is racy?
|
|
780
|
-
- **Schema imports:** can one schema reference another (`type: $ref: person`)?
|
|
759
|
+
- **Schema imports:** can one schema reference another (`type: $ref: person`)?
|
|
781
760
|
- **Internationalization:** non-ASCII in keys? Spec currently restricts segments to `[a-z0-9_-]`. Revisit if community wants Unicode.
|
|
782
761
|
- **Generated content in `derived/`:** the spec says `schema: null` is allowed, but should there be a separate marker (`generated: true`) for clarity?
|
|
783
762
|
|
|
@@ -785,7 +764,7 @@ See `ARCHITECTURE.md` for an ASCII diagram and the full read-path walkthrough.
|
|
|
785
764
|
|
|
786
765
|
A `textus/2` implementation MUST:
|
|
787
766
|
|
|
788
|
-
- [ ] Parse `.textus/manifest.yaml` and accept `version: textus/2
|
|
767
|
+
- [ ] Parse `.textus/manifest.yaml` and accept `version: textus/2`.
|
|
789
768
|
- [ ] Resolve keys via longest-prefix match against manifest entries.
|
|
790
769
|
- [ ] Read `_meta` + body from `.md` files; validate against the named schema.
|
|
791
770
|
- [ ] Read `_meta` from the top-level `_meta` hash in `.json` / `.yaml` files; validate against the named schema.
|
data/docs/conventions.md
CHANGED
|
@@ -27,8 +27,6 @@ Recommended top-level layout — the spec allows alternatives, but this is what
|
|
|
27
27
|
|
|
28
28
|
Inside `working/`, group by **domain** (people, projects, decisions, runbooks), not by file type or date. Inside `output/`, group by **producer** (`output/catalogs/`, `output/indexes/`) so it's clear which build job owns what.
|
|
29
29
|
|
|
30
|
-
> **0.9.2 rename.** Default zone names moved off historical artifact terms (`canon/intake/pending/derived`) to one lifecycle axis (`identity/inbox/review/output`); `working` is unchanged. Upgrade existing stores by hand-editing the manifest and `mv`-ing the zone directories (see 0.9.2 CHANGELOG).
|
|
31
|
-
|
|
32
30
|
## Schema design
|
|
33
31
|
|
|
34
32
|
- **One schema per entry type, not per directory.** `person.yaml`, `project.yaml`, `decision.yaml` — applied across multiple subtrees if the shape matches.
|
|
@@ -79,7 +77,7 @@ textus freshness --format=json \
|
|
|
79
77
|
|
|
80
78
|
- **Bodies are Markdown.** Headings, lists, code fences — whatever a human or agent finds useful.
|
|
81
79
|
- **The schema does not validate the body.** If a field belongs in structured data, put it in frontmatter, not the body.
|
|
82
|
-
- **Keep entries short.** If a project entry hits 500 lines, it probably wants to be split into sub-entries (e.g. `
|
|
80
|
+
- **Keep entries short.** If a project entry hits 500 lines, it probably wants to be split into sub-entries (e.g. `working.projects.acme.dashboard` + `working.projects.acme.api`) rather than one mega-document.
|
|
83
81
|
|
|
84
82
|
## Concurrency
|
|
85
83
|
|
data/lib/textus/cli.rb
CHANGED
|
@@ -53,10 +53,6 @@ module Textus
|
|
|
53
53
|
0
|
|
54
54
|
when "--help", "-h" then print_help
|
|
55
55
|
0
|
|
56
|
-
when "stale"
|
|
57
|
-
raise UsageError.new(
|
|
58
|
-
"textus stale was removed in 0.9.2 — use `textus freshness` instead",
|
|
59
|
-
)
|
|
60
56
|
else
|
|
61
57
|
klass = VERBS[verb] or raise UsageError.new("unknown verb: #{verb}")
|
|
62
58
|
dispatch(klass, argv)
|
|
@@ -88,7 +84,7 @@ module Textus
|
|
|
88
84
|
@stdout.puts <<~HELP
|
|
89
85
|
textus #{VERSION} — reference implementation of #{PROTOCOL}
|
|
90
86
|
|
|
91
|
-
Usage (json output is the default
|
|
87
|
+
Usage (json output is the default):
|
|
92
88
|
textus list [--prefix=KEY] [--zone=Z]
|
|
93
89
|
textus where KEY
|
|
94
90
|
textus get KEY
|
|
@@ -104,7 +100,6 @@ module Textus
|
|
|
104
100
|
textus schema {show,init,diff,migrate}
|
|
105
101
|
textus hook {list,run}
|
|
106
102
|
textus policy {list,explain}
|
|
107
|
-
textus migrate {zones,policies}
|
|
108
103
|
HELP
|
|
109
104
|
end
|
|
110
105
|
end
|
|
@@ -1,48 +1,32 @@
|
|
|
1
|
-
require "json"
|
|
2
|
-
|
|
3
1
|
module Textus
|
|
4
2
|
module Doctor
|
|
5
3
|
class Check
|
|
6
4
|
class AuditLog < Check
|
|
7
5
|
def call
|
|
8
|
-
out = []
|
|
9
6
|
path = File.join(store.root, "audit.log")
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
7
|
+
Textus::Store::AuditLog.new(store.root).verify_integrity.map do |v|
|
|
8
|
+
{
|
|
9
|
+
"code" => "audit.parse_error",
|
|
10
|
+
"level" => "warning",
|
|
11
|
+
"subject" => "#{path}:#{v["lineno"]}",
|
|
12
|
+
"message" => violation_message(v),
|
|
13
|
+
"fix" => "inspect #{path} at line #{v["lineno"]} and remove the corrupted row",
|
|
14
|
+
}
|
|
15
|
+
end
|
|
16
|
+
end
|
|
15
17
|
|
|
16
|
-
|
|
17
|
-
begin
|
|
18
|
-
JSON.parse(stripped)
|
|
19
|
-
rescue JSON::ParserError => e
|
|
20
|
-
out << {
|
|
21
|
-
"code" => "audit.parse_error",
|
|
22
|
-
"level" => "warning",
|
|
23
|
-
"subject" => "#{path}:#{lineno}",
|
|
24
|
-
"message" => "audit log line #{lineno} is invalid JSON: #{e.message}",
|
|
25
|
-
"fix" => "inspect #{path} at line #{lineno} and remove the corrupted row",
|
|
26
|
-
}
|
|
27
|
-
end
|
|
28
|
-
else
|
|
29
|
-
# Legacy TSV (pre-0.5): read-only support retained for on-disk logs
|
|
30
|
-
# written by older textus versions. Never written by current code.
|
|
31
|
-
# Minimum 6 fields.
|
|
32
|
-
fields = stripped.split("\t")
|
|
33
|
-
next if fields.length >= 6
|
|
18
|
+
private
|
|
34
19
|
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
20
|
+
def violation_message(v)
|
|
21
|
+
case v["reason"]
|
|
22
|
+
when "invalid_json"
|
|
23
|
+
"audit log line #{v["lineno"]} is invalid JSON: #{v["detail"]}"
|
|
24
|
+
when "short_tsv"
|
|
25
|
+
"audit log line #{v["lineno"]} #{v["detail"]} " \
|
|
26
|
+
"(consider migrating to NDJSON)"
|
|
27
|
+
else
|
|
28
|
+
v["detail"]
|
|
44
29
|
end
|
|
45
|
-
out
|
|
46
30
|
end
|
|
47
31
|
end
|
|
48
32
|
end
|
|
@@ -3,11 +3,10 @@ module Textus
|
|
|
3
3
|
class Check
|
|
4
4
|
class ManifestFiles < Check
|
|
5
5
|
def call
|
|
6
|
-
|
|
7
|
-
store.manifest.entries.each do |entry|
|
|
6
|
+
store.manifest.entries.each_with_object([]) do |entry, out|
|
|
8
7
|
next if entry.nested
|
|
9
8
|
|
|
10
|
-
path =
|
|
9
|
+
path = Textus::Key::Path.resolve(store.manifest, entry)
|
|
11
10
|
next if File.exist?(path)
|
|
12
11
|
|
|
13
12
|
out << {
|
|
@@ -19,18 +18,6 @@ module Textus
|
|
|
19
18
|
"(or leave empty if not yet authored)",
|
|
20
19
|
}
|
|
21
20
|
end
|
|
22
|
-
out
|
|
23
|
-
end
|
|
24
|
-
|
|
25
|
-
private
|
|
26
|
-
|
|
27
|
-
def leaf_path_for(entry)
|
|
28
|
-
primary_ext = Entry.for_format(entry.format).extensions.first
|
|
29
|
-
if File.extname(entry.path) == ""
|
|
30
|
-
File.join(store.root, "zones", entry.path + primary_ext)
|
|
31
|
-
else
|
|
32
|
-
File.join(store.root, "zones", entry.path)
|
|
33
|
-
end
|
|
34
21
|
end
|
|
35
22
|
end
|
|
36
23
|
end
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
module Textus
|
|
2
|
+
module Doctor
|
|
3
|
+
class Check
|
|
4
|
+
# Surfaces YAML parse failures for files in <store>/schemas/. Without
|
|
5
|
+
# this check, malformed schemas are silently skipped by other doctor
|
|
6
|
+
# checks (UnownedSchemaFields rescues, Schemas only checks filenames),
|
|
7
|
+
# leaving the operator with no signal that a schema is broken.
|
|
8
|
+
class SchemaParseError < Check
|
|
9
|
+
def call
|
|
10
|
+
dir = File.join(store.root, "schemas")
|
|
11
|
+
return [] unless File.directory?(dir)
|
|
12
|
+
|
|
13
|
+
Dir.glob(File.join(dir, "*.yaml")).each_with_object([]) do |path, out|
|
|
14
|
+
Schema.load(path)
|
|
15
|
+
rescue StandardError => e
|
|
16
|
+
out << {
|
|
17
|
+
"code" => "schema.parse_error",
|
|
18
|
+
"level" => "error",
|
|
19
|
+
"subject" => path,
|
|
20
|
+
"message" => "schema failed to parse: #{e.class}: #{e.message}",
|
|
21
|
+
"fix" => "fix the YAML at #{path} (check indentation, quoted scalars, and aliases)",
|
|
22
|
+
}
|
|
23
|
+
end
|
|
24
|
+
end
|
|
25
|
+
end
|
|
26
|
+
end
|
|
27
|
+
end
|
|
28
|
+
end
|
|
@@ -1,55 +1,56 @@
|
|
|
1
|
-
require "digest"
|
|
2
|
-
require "json"
|
|
3
|
-
|
|
4
1
|
module Textus
|
|
5
2
|
module Doctor
|
|
6
3
|
class Check
|
|
7
4
|
class Sentinels < Check
|
|
8
5
|
def call
|
|
9
|
-
out = []
|
|
10
6
|
dir = File.join(store.root, "sentinels")
|
|
11
|
-
return
|
|
7
|
+
return [] unless File.directory?(dir)
|
|
12
8
|
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
"code" => "sentinel.parse_error",
|
|
19
|
-
"level" => "warning",
|
|
20
|
-
"subject" => sp,
|
|
21
|
-
"message" => "sentinel is not valid JSON: #{e.message}",
|
|
22
|
-
"fix" => "delete #{sp} and re-run 'textus build' to regenerate",
|
|
23
|
-
}
|
|
24
|
-
next
|
|
25
|
-
end
|
|
9
|
+
repo_root = File.dirname(store.root)
|
|
10
|
+
Dir.glob(File.join(dir, "**", "*#{Textus::Store::Sentinel::SUFFIX}")).flat_map do |sentinel_path|
|
|
11
|
+
inspect_sentinel(sentinel_path, repo_root)
|
|
12
|
+
end
|
|
13
|
+
end
|
|
26
14
|
|
|
27
|
-
|
|
28
|
-
recorded_sha = data["sha256"]
|
|
15
|
+
private
|
|
29
16
|
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
"message" => "sentinel target #{target.inspect} no longer exists",
|
|
36
|
-
"fix" => "delete #{sp} (the published file is gone) or restore the target",
|
|
37
|
-
}
|
|
38
|
-
next
|
|
39
|
-
end
|
|
17
|
+
def inspect_sentinel(sentinel_path, repo_root)
|
|
18
|
+
sentinel = Textus::Store::Sentinel.load(sentinel_path, repo_root)
|
|
19
|
+
return [parse_error_issue(sentinel_path)] if sentinel.nil?
|
|
20
|
+
return [orphan_issue(sentinel_path, sentinel)] if sentinel.orphan?
|
|
21
|
+
return [drift_issue(sentinel)] if sentinel.drift?
|
|
40
22
|
|
|
41
|
-
|
|
42
|
-
|
|
23
|
+
[]
|
|
24
|
+
end
|
|
43
25
|
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
}
|
|
51
|
-
|
|
52
|
-
|
|
26
|
+
def parse_error_issue(sentinel_path)
|
|
27
|
+
{
|
|
28
|
+
"code" => "sentinel.parse_error",
|
|
29
|
+
"level" => "warning",
|
|
30
|
+
"subject" => sentinel_path,
|
|
31
|
+
"message" => "sentinel is not valid JSON",
|
|
32
|
+
"fix" => "delete #{sentinel_path} and re-run 'textus build' to regenerate",
|
|
33
|
+
}
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
def orphan_issue(sentinel_path, sentinel)
|
|
37
|
+
{
|
|
38
|
+
"code" => "sentinel.orphan",
|
|
39
|
+
"level" => "warning",
|
|
40
|
+
"subject" => sentinel_path,
|
|
41
|
+
"message" => "sentinel target #{sentinel.target.inspect} no longer exists",
|
|
42
|
+
"fix" => "delete #{sentinel_path} (the published file is gone) or restore the target",
|
|
43
|
+
}
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
def drift_issue(sentinel)
|
|
47
|
+
{
|
|
48
|
+
"code" => "sentinel.drift",
|
|
49
|
+
"level" => "warning",
|
|
50
|
+
"subject" => sentinel.target,
|
|
51
|
+
"message" => "published file at #{sentinel.target} was modified out-of-band",
|
|
52
|
+
"fix" => "re-run 'textus build' to overwrite, or copy the manual edit back into the store source",
|
|
53
|
+
}
|
|
53
54
|
end
|
|
54
55
|
end
|
|
55
56
|
end
|
|
@@ -3,30 +3,36 @@ module Textus
|
|
|
3
3
|
class Check
|
|
4
4
|
class UnownedSchemaFields < Check
|
|
5
5
|
def call
|
|
6
|
-
out = []
|
|
7
6
|
dir = File.join(store.root, "schemas")
|
|
8
|
-
return
|
|
7
|
+
return [] unless File.directory?(dir)
|
|
9
8
|
|
|
10
|
-
Dir.glob(File.join(dir, "*.yaml")).
|
|
11
|
-
|
|
12
|
-
Schema.load(sp)
|
|
13
|
-
rescue StandardError
|
|
14
|
-
next
|
|
15
|
-
end
|
|
16
|
-
unowned = schema.fields.each_with_object([]) do |(name, spec), acc|
|
|
17
|
-
acc << name if spec.is_a?(Hash) && spec["maintained_by"].nil?
|
|
18
|
-
end
|
|
19
|
-
next if unowned.empty?
|
|
20
|
-
|
|
21
|
-
out << {
|
|
22
|
-
"code" => "schema.unowned_fields",
|
|
23
|
-
"level" => "info",
|
|
24
|
-
"subject" => schema.name || File.basename(sp, ".yaml"),
|
|
25
|
-
"message" => "schema has fields without maintained_by: #{unowned.join(", ")}",
|
|
26
|
-
"fix" => "add 'maintained_by: <role>' to each field in #{sp} (optional but recommended)",
|
|
27
|
-
}
|
|
9
|
+
Dir.glob(File.join(dir, "*.yaml")).flat_map do |path|
|
|
10
|
+
issues_for(path)
|
|
28
11
|
end
|
|
29
|
-
|
|
12
|
+
end
|
|
13
|
+
|
|
14
|
+
private
|
|
15
|
+
|
|
16
|
+
def issues_for(path)
|
|
17
|
+
schema = safe_load(path)
|
|
18
|
+
return [] if schema.nil?
|
|
19
|
+
|
|
20
|
+
unowned = schema.unowned_fields
|
|
21
|
+
return [] if unowned.empty?
|
|
22
|
+
|
|
23
|
+
[{
|
|
24
|
+
"code" => "schema.unowned_fields",
|
|
25
|
+
"level" => "info",
|
|
26
|
+
"subject" => schema.name || File.basename(path, ".yaml"),
|
|
27
|
+
"message" => "schema has fields without maintained_by: #{unowned.join(", ")}",
|
|
28
|
+
"fix" => "add 'maintained_by: <role>' to each field in #{path} (optional but recommended)",
|
|
29
|
+
}]
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def safe_load(path)
|
|
33
|
+
Schema.load(path)
|
|
34
|
+
rescue StandardError
|
|
35
|
+
nil
|
|
30
36
|
end
|
|
31
37
|
end
|
|
32
38
|
end
|
data/lib/textus/doctor.rb
CHANGED
|
@@ -11,6 +11,7 @@ module Textus
|
|
|
11
11
|
CHECKS = [
|
|
12
12
|
Check::ManifestFiles,
|
|
13
13
|
Check::Schemas,
|
|
14
|
+
Check::SchemaParseError,
|
|
14
15
|
Check::Templates,
|
|
15
16
|
Check::Hooks,
|
|
16
17
|
Check::IntakeRegistration,
|
|
@@ -21,7 +22,6 @@ module Textus
|
|
|
21
22
|
Check::SchemaViolations,
|
|
22
23
|
Check::PolicyAmbiguity,
|
|
23
24
|
Check::HandlerAllowlist,
|
|
24
|
-
Check::LegacyIntakeFields,
|
|
25
25
|
].freeze
|
|
26
26
|
|
|
27
27
|
ALL_CHECKS = CHECKS.map(&:name_key).freeze
|
|
@@ -1,25 +1,21 @@
|
|
|
1
|
-
require "json"
|
|
2
|
-
require "digest"
|
|
3
1
|
require "fileutils"
|
|
4
2
|
|
|
5
3
|
module Textus
|
|
6
4
|
module Infra
|
|
7
5
|
# Publishes built artifacts from the store to repo-relative consumer paths.
|
|
8
6
|
# Publish = copy + sentinel. The in-store file is already the consumer-shaped
|
|
9
|
-
# artifact; no parsing or stripping.
|
|
7
|
+
# artifact; no parsing or stripping.
|
|
8
|
+
#
|
|
9
|
+
# Sentinel I/O is delegated to Store::Sentinel. Sentinels live under
|
|
10
10
|
# `<store_root>/sentinels/` and mirror the target's repo-relative layout so
|
|
11
11
|
# consumer directories aren't polluted with `.textus-managed.json` siblings.
|
|
12
12
|
module Publisher
|
|
13
|
-
SENTINEL_SUFFIX = ".textus-managed.json".freeze
|
|
14
|
-
SENTINEL_DIR = "sentinels".freeze
|
|
15
|
-
|
|
16
13
|
def self.publish(source:, target:, store_root:)
|
|
17
14
|
FileUtils.mkdir_p(File.dirname(target))
|
|
18
15
|
refuse_if_unmanaged(target, store_root)
|
|
19
16
|
File.delete(target) if File.symlink?(target)
|
|
20
17
|
FileUtils.cp(source, target)
|
|
21
|
-
|
|
22
|
-
cleanup_legacy_sentinel(target)
|
|
18
|
+
Store::Sentinel.write!(target: target, source: source, store_root: store_root)
|
|
23
19
|
end
|
|
24
20
|
|
|
25
21
|
def self.refuse_if_unmanaged(target, store_root)
|
|
@@ -30,43 +26,7 @@ module Textus
|
|
|
30
26
|
end
|
|
31
27
|
|
|
32
28
|
def self.managed?(target, store_root)
|
|
33
|
-
File.exist?(sentinel_path(target, store_root))
|
|
34
|
-
end
|
|
35
|
-
|
|
36
|
-
def self.write_sentinel(target, store_root:, source:)
|
|
37
|
-
path = sentinel_path(target, store_root)
|
|
38
|
-
FileUtils.mkdir_p(File.dirname(path))
|
|
39
|
-
File.write(path, JSON.generate(
|
|
40
|
-
"source" => source,
|
|
41
|
-
"target" => target,
|
|
42
|
-
"sha256" => Digest::SHA256.hexdigest(File.binread(target)),
|
|
43
|
-
"mode" => "copy",
|
|
44
|
-
))
|
|
45
|
-
end
|
|
46
|
-
|
|
47
|
-
# Sentinel layout: <store_root>/sentinels/<target_rel_to_repo>.textus-managed.json
|
|
48
|
-
# The full target extension is preserved so a marketplace.json and
|
|
49
|
-
# marketplace.yaml don't collide.
|
|
50
|
-
def self.sentinel_path(target, store_root)
|
|
51
|
-
repo_root = File.dirname(store_root)
|
|
52
|
-
rel = relative_to(target, repo_root) || File.basename(target)
|
|
53
|
-
File.join(store_root, SENTINEL_DIR, rel + SENTINEL_SUFFIX)
|
|
54
|
-
end
|
|
55
|
-
|
|
56
|
-
def self.legacy_sentinel_path(target)
|
|
57
|
-
target + SENTINEL_SUFFIX
|
|
58
|
-
end
|
|
59
|
-
|
|
60
|
-
def self.cleanup_legacy_sentinel(target)
|
|
61
|
-
FileUtils.rm_f(legacy_sentinel_path(target))
|
|
62
|
-
end
|
|
63
|
-
|
|
64
|
-
def self.relative_to(path, base)
|
|
65
|
-
path = File.expand_path(path)
|
|
66
|
-
base = File.expand_path(base)
|
|
67
|
-
return nil unless path.start_with?(base + File::SEPARATOR)
|
|
68
|
-
|
|
69
|
-
path[(base.length + 1)..]
|
|
29
|
+
File.exist?(Store::Sentinel.sentinel_path(target, store_root))
|
|
70
30
|
end
|
|
71
31
|
end
|
|
72
32
|
end
|
|
@@ -28,10 +28,7 @@ module Textus
|
|
|
28
28
|
@format = resolve_format!(raw["format"])
|
|
29
29
|
|
|
30
30
|
validate_events!
|
|
31
|
-
raise UsageError.new("entry '#{@key}': 'source:' key renamed to 'intake:' in 0.9") if raw.key?("source")
|
|
32
|
-
|
|
33
31
|
parse_intake!(raw["intake"])
|
|
34
|
-
reject_legacy_projection_keys!
|
|
35
32
|
validate_format_matrix!
|
|
36
33
|
validate_publish_each!
|
|
37
34
|
validate_inject_intro!
|
|
@@ -57,9 +54,8 @@ module Textus
|
|
|
57
54
|
end
|
|
58
55
|
|
|
59
56
|
# Signal-based zone-kind predicates: derive the "kind" of a zone from its
|
|
60
|
-
# writable_by signals rather than its literal name
|
|
61
|
-
# working when users rename the default zones
|
|
62
|
-
# → identity/inbox/review/output, etc.).
|
|
57
|
+
# writable_by signals rather than its literal name, so detection keeps
|
|
58
|
+
# working when users rename the default zones.
|
|
63
59
|
def in_generator_zone?
|
|
64
60
|
zone_writers.include?("build")
|
|
65
61
|
end
|
|
@@ -68,12 +64,6 @@ module Textus
|
|
|
68
64
|
zone_writers.include?("ai")
|
|
69
65
|
end
|
|
70
66
|
|
|
71
|
-
# Legacy alias for in_generator_zone?. Retained because internal validation
|
|
72
|
-
# callers (and external tools) read more naturally as `derived?`.
|
|
73
|
-
def derived?
|
|
74
|
-
in_generator_zone?
|
|
75
|
-
end
|
|
76
|
-
|
|
77
67
|
private
|
|
78
68
|
|
|
79
69
|
def zone_writers
|
|
@@ -85,7 +75,7 @@ module Textus
|
|
|
85
75
|
def validate_inject_intro!
|
|
86
76
|
return unless @inject_intro
|
|
87
77
|
|
|
88
|
-
unless
|
|
78
|
+
unless in_generator_zone?
|
|
89
79
|
raise UsageError.new(
|
|
90
80
|
"entry '#{@key}': inject_intro: is only valid on derived entries",
|
|
91
81
|
)
|
|
@@ -181,7 +171,7 @@ module Textus
|
|
|
181
171
|
|
|
182
172
|
# Template-required-for-derived rules. Skipped for entries materialized by an
|
|
183
173
|
# external generator: command (those produce the bytes themselves).
|
|
184
|
-
if
|
|
174
|
+
if in_generator_zone? && @template.nil? && @generator.nil? &&
|
|
185
175
|
(@format == "markdown" || @format == "text") && !@nested
|
|
186
176
|
raise UsageError.new("entry '#{@key}': derived #{@format} entries require a template")
|
|
187
177
|
end
|
|
@@ -189,26 +179,11 @@ module Textus
|
|
|
189
179
|
# rubocop:enable Metrics/AbcSize, Metrics/CyclomaticComplexity, Metrics/PerceivedComplexity
|
|
190
180
|
|
|
191
181
|
def parse_intake!(src)
|
|
192
|
-
raise UsageError.new("entry '#{@key}': source.fetch renamed to intake.handler in 0.9") if src.is_a?(Hash) && src.key?("fetch")
|
|
193
|
-
|
|
194
|
-
if src.is_a?(Hash) && (src.key?("ttl") || src.key?("on_stale") || src.key?("sync_budget_ms"))
|
|
195
|
-
raise UsageError.new(
|
|
196
|
-
"entry '#{@key}': intake.ttl/intake.on_stale/intake.sync_budget_ms removed in 0.9.2 — " \
|
|
197
|
-
"move into a top-level policies: block (see CHANGELOG migration recipe).",
|
|
198
|
-
)
|
|
199
|
-
end
|
|
200
|
-
|
|
201
182
|
src ||= {}
|
|
202
183
|
@intake_handler = src["handler"]
|
|
203
184
|
@intake_config = src["config"] || {}
|
|
204
185
|
end
|
|
205
186
|
|
|
206
|
-
def reject_legacy_projection_keys!
|
|
207
|
-
return unless @projection.is_a?(Hash) && @projection.key?("reducer")
|
|
208
|
-
|
|
209
|
-
raise UsageError.new("entry '#{@key}': projection.reducer renamed to projection.reduce in 0.6")
|
|
210
|
-
end
|
|
211
|
-
|
|
212
187
|
def validate_events!
|
|
213
188
|
pubsub_events = Hooks::Registry::EVENTS.select { |_, s| s[:mode] == :pubsub }.keys
|
|
214
189
|
@events.each_key do |evt|
|
data/lib/textus/manifest.rb
CHANGED
|
@@ -34,12 +34,7 @@ module Textus
|
|
|
34
34
|
|
|
35
35
|
raw = YAML.safe_load_file(manifest_path, aliases: false)
|
|
36
36
|
unless raw["version"] == PROTOCOL
|
|
37
|
-
|
|
38
|
-
"manifest is textus/1; edit manifest.yaml: change 'version: textus/1' to 'version: #{PROTOCOL}'"
|
|
39
|
-
else
|
|
40
|
-
"unsupported manifest version #{raw["version"].inspect}"
|
|
41
|
-
end
|
|
42
|
-
raise BadFrontmatter.new(manifest_path, msg)
|
|
37
|
+
raise BadFrontmatter.new(manifest_path, "unsupported manifest version #{raw["version"].inspect}; expected #{PROTOCOL.inspect}")
|
|
43
38
|
end
|
|
44
39
|
|
|
45
40
|
new(root, raw)
|
|
@@ -50,7 +45,6 @@ module Textus
|
|
|
50
45
|
@raw = raw
|
|
51
46
|
raise BadFrontmatter.new(File.join(root, "manifest.yaml"), "manifest must declare zones:") if Array(raw["zones"]).empty?
|
|
52
47
|
|
|
53
|
-
reject_legacy_entry_intake_policy!(Array(raw["entries"]))
|
|
54
48
|
@entries = Array(raw["entries"]).map { |e| Manifest::Entry.new(self, e) }
|
|
55
49
|
validate_declared_keys!
|
|
56
50
|
end
|
|
@@ -155,19 +149,6 @@ module Textus
|
|
|
155
149
|
@entries.each { |e| validate_key!(e.key) }
|
|
156
150
|
end
|
|
157
151
|
|
|
158
|
-
def reject_legacy_entry_intake_policy!(raw_entries)
|
|
159
|
-
raw_entries.each do |re|
|
|
160
|
-
intake = re["intake"]
|
|
161
|
-
next unless intake.is_a?(Hash)
|
|
162
|
-
next unless intake.key?("ttl") || intake.key?("on_stale") || intake.key?("sync_budget_ms")
|
|
163
|
-
|
|
164
|
-
raise UsageError.new(
|
|
165
|
-
"entry '#{re["key"]}': intake.ttl/intake.on_stale/intake.sync_budget_ms removed in 0.9.2 — " \
|
|
166
|
-
"move into a top-level policies: block (see CHANGELOG migration recipe).",
|
|
167
|
-
)
|
|
168
|
-
end
|
|
169
|
-
end
|
|
170
|
-
|
|
171
152
|
def resolve_leaf_path(entry)
|
|
172
153
|
Textus::Key::Path.resolve(self, entry)
|
|
173
154
|
end
|
data/lib/textus/schema.rb
CHANGED
|
@@ -26,6 +26,16 @@ module Textus
|
|
|
26
26
|
meta["maintained_by"]
|
|
27
27
|
end
|
|
28
28
|
|
|
29
|
+
# Returns the list of field names whose spec is a Hash but lacks the
|
|
30
|
+
# 'maintained_by' key. Used by Doctor::Check::UnownedSchemaFields.
|
|
31
|
+
def unowned_fields
|
|
32
|
+
@fields.each_with_object([]) do |(name, spec), acc|
|
|
33
|
+
next unless spec.is_a?(Hash)
|
|
34
|
+
|
|
35
|
+
acc << name if spec["maintained_by"].nil?
|
|
36
|
+
end
|
|
37
|
+
end
|
|
38
|
+
|
|
29
39
|
def evolution
|
|
30
40
|
raw = @raw["evolution"] || {}
|
|
31
41
|
raw.each_with_object({}) do |(k, v), h|
|
|
@@ -47,25 +47,38 @@ module Textus
|
|
|
47
47
|
end
|
|
48
48
|
end
|
|
49
49
|
|
|
50
|
+
# Returns an array of integrity-violation descriptors for the on-disk log.
|
|
51
|
+
# Each entry is { "lineno" => Integer, "reason" => String, "detail" => String }.
|
|
52
|
+
# Empty array means the log is well-formed (or doesn't exist yet).
|
|
53
|
+
def verify_integrity
|
|
54
|
+
return [] unless File.exist?(@path)
|
|
55
|
+
|
|
56
|
+
out = []
|
|
57
|
+
File.foreach(@path).with_index(1) do |line, lineno|
|
|
58
|
+
violation = check_line(line.chomp, lineno)
|
|
59
|
+
out << violation if violation
|
|
60
|
+
end
|
|
61
|
+
out
|
|
62
|
+
end
|
|
63
|
+
|
|
50
64
|
private
|
|
51
65
|
|
|
52
66
|
def parse_row(line)
|
|
53
67
|
return nil if line.empty?
|
|
54
68
|
|
|
55
|
-
|
|
56
|
-
JSON.parse(line)
|
|
57
|
-
else
|
|
58
|
-
# Legacy TSV (pre-0.5): read-only support retained for on-disk logs
|
|
59
|
-
# written by older textus versions. Never written by current code.
|
|
60
|
-
# Format: ts, role, verb, key, etag_before, etag_after [, json_extras]
|
|
61
|
-
fields = line.split("\t")
|
|
62
|
-
return nil if fields.length < 4
|
|
63
|
-
|
|
64
|
-
{ "ts" => fields[0], "role" => fields[1], "verb" => fields[2], "key" => fields[3] }
|
|
65
|
-
end
|
|
69
|
+
JSON.parse(line)
|
|
66
70
|
rescue JSON::ParserError
|
|
67
71
|
nil
|
|
68
72
|
end
|
|
73
|
+
|
|
74
|
+
def check_line(stripped, lineno)
|
|
75
|
+
return nil if stripped.empty?
|
|
76
|
+
|
|
77
|
+
JSON.parse(stripped)
|
|
78
|
+
nil
|
|
79
|
+
rescue JSON::ParserError => e
|
|
80
|
+
{ "lineno" => lineno, "reason" => "invalid_json", "detail" => e.message }
|
|
81
|
+
end
|
|
69
82
|
end
|
|
70
83
|
end
|
|
71
84
|
end
|
|
@@ -0,0 +1,86 @@
|
|
|
1
|
+
require "json"
|
|
2
|
+
require "digest"
|
|
3
|
+
require "fileutils"
|
|
4
|
+
|
|
5
|
+
module Textus
|
|
6
|
+
class Store
|
|
7
|
+
# Value object for sentinel files written by Infra::Publisher and inspected
|
|
8
|
+
# by Doctor::Check::Sentinels. Owns the JSON shape ({source, target,
|
|
9
|
+
# sha256, mode}) and the on-disk path layout (<store_root>/sentinels/
|
|
10
|
+
# <target-rel-to-repo>.textus-managed.json). Target/source are repo-relative
|
|
11
|
+
# when the published file is under the repo root, absolute otherwise.
|
|
12
|
+
class Sentinel
|
|
13
|
+
SUFFIX = ".textus-managed.json".freeze
|
|
14
|
+
DIR = "sentinels".freeze
|
|
15
|
+
|
|
16
|
+
attr_reader :target, :source, :sha256, :mode
|
|
17
|
+
|
|
18
|
+
def self.write!(target:, source:, store_root:)
|
|
19
|
+
path = sentinel_path(target, store_root)
|
|
20
|
+
FileUtils.mkdir_p(File.dirname(path))
|
|
21
|
+
repo_root = File.dirname(store_root)
|
|
22
|
+
File.write(path, JSON.generate(
|
|
23
|
+
"source" => rel_or_abs(source, repo_root),
|
|
24
|
+
"target" => rel_or_abs(target, repo_root),
|
|
25
|
+
"sha256" => Digest::SHA256.hexdigest(File.binread(target)),
|
|
26
|
+
"mode" => "copy",
|
|
27
|
+
))
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
def self.load(path, repo_root)
|
|
31
|
+
raw = JSON.parse(File.read(path))
|
|
32
|
+
new(
|
|
33
|
+
target: absolutize(raw["target"], repo_root),
|
|
34
|
+
source: absolutize(raw["source"], repo_root),
|
|
35
|
+
sha256: raw["sha256"],
|
|
36
|
+
mode: raw["mode"],
|
|
37
|
+
)
|
|
38
|
+
rescue JSON::ParserError, Errno::ENOENT
|
|
39
|
+
nil
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
def self.sentinel_path(target, store_root)
|
|
43
|
+
repo_root = File.dirname(store_root)
|
|
44
|
+
rel = relative_to(target, repo_root) || File.basename(target)
|
|
45
|
+
File.join(store_root, DIR, rel + SUFFIX)
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
def self.rel_or_abs(path, repo_root)
|
|
49
|
+
relative_to(path, repo_root) || File.expand_path(path)
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
def self.relative_to(path, repo_root)
|
|
53
|
+
path = File.expand_path(path)
|
|
54
|
+
base = File.expand_path(repo_root)
|
|
55
|
+
return nil unless path.start_with?(base + File::SEPARATOR)
|
|
56
|
+
|
|
57
|
+
path[(base.length + 1)..]
|
|
58
|
+
end
|
|
59
|
+
|
|
60
|
+
def self.absolutize(path, repo_root)
|
|
61
|
+
return path if path.nil?
|
|
62
|
+
return path if File.absolute_path?(path)
|
|
63
|
+
|
|
64
|
+
File.expand_path(path, repo_root)
|
|
65
|
+
end
|
|
66
|
+
|
|
67
|
+
def initialize(target:, source:, sha256:, mode:)
|
|
68
|
+
@target = target
|
|
69
|
+
@source = source
|
|
70
|
+
@sha256 = sha256
|
|
71
|
+
@mode = mode
|
|
72
|
+
end
|
|
73
|
+
|
|
74
|
+
def orphan?
|
|
75
|
+
@target.nil? || !File.exist?(@target)
|
|
76
|
+
end
|
|
77
|
+
|
|
78
|
+
def drift?
|
|
79
|
+
return false if orphan?
|
|
80
|
+
return false if @sha256.nil?
|
|
81
|
+
|
|
82
|
+
Digest::SHA256.hexdigest(File.binread(@target)) != @sha256
|
|
83
|
+
end
|
|
84
|
+
end
|
|
85
|
+
end
|
|
86
|
+
end
|
data/lib/textus/version.rb
CHANGED
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: textus
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.10.
|
|
4
|
+
version: 0.10.3
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Patrick
|
|
@@ -175,9 +175,9 @@ files:
|
|
|
175
175
|
- lib/textus/doctor/check/hooks.rb
|
|
176
176
|
- lib/textus/doctor/check/illegal_keys.rb
|
|
177
177
|
- lib/textus/doctor/check/intake_registration.rb
|
|
178
|
-
- lib/textus/doctor/check/legacy_intake_fields.rb
|
|
179
178
|
- lib/textus/doctor/check/manifest_files.rb
|
|
180
179
|
- lib/textus/doctor/check/policy_ambiguity.rb
|
|
180
|
+
- lib/textus/doctor/check/schema_parse_error.rb
|
|
181
181
|
- lib/textus/doctor/check/schema_violations.rb
|
|
182
182
|
- lib/textus/doctor/check/schemas.rb
|
|
183
183
|
- lib/textus/doctor/check/sentinels.rb
|
|
@@ -233,6 +233,7 @@ files:
|
|
|
233
233
|
- lib/textus/store/audit_log.rb
|
|
234
234
|
- lib/textus/store/mover.rb
|
|
235
235
|
- lib/textus/store/reader.rb
|
|
236
|
+
- lib/textus/store/sentinel.rb
|
|
236
237
|
- lib/textus/store/staleness.rb
|
|
237
238
|
- lib/textus/store/validator.rb
|
|
238
239
|
- lib/textus/store/writer.rb
|
|
@@ -1,57 +0,0 @@
|
|
|
1
|
-
require "yaml"
|
|
2
|
-
|
|
3
|
-
module Textus
|
|
4
|
-
module Doctor
|
|
5
|
-
class Check
|
|
6
|
-
# Scans the raw manifest YAML for entry-level intake.ttl /
|
|
7
|
-
# intake.on_stale / intake.sync_budget_ms keys. Manifest parsing
|
|
8
|
-
# already raises on these in 0.9.2 — this check exists for the case
|
|
9
|
-
# where doctor is run against a problem manifest separately (e.g. CI
|
|
10
|
-
# lint or an exploration session that loads the YAML directly).
|
|
11
|
-
class LegacyIntakeFields < Check
|
|
12
|
-
LEGACY_KEYS = %w[ttl on_stale sync_budget_ms].freeze
|
|
13
|
-
|
|
14
|
-
def call
|
|
15
|
-
out = []
|
|
16
|
-
path = File.join(store.root, "manifest.yaml")
|
|
17
|
-
return out unless File.exist?(path)
|
|
18
|
-
|
|
19
|
-
raw = safe_load(path)
|
|
20
|
-
return out unless raw.is_a?(Hash)
|
|
21
|
-
|
|
22
|
-
Array(raw["entries"]).each do |entry|
|
|
23
|
-
next unless entry.is_a?(Hash)
|
|
24
|
-
|
|
25
|
-
intake = entry["intake"]
|
|
26
|
-
next unless intake.is_a?(Hash)
|
|
27
|
-
|
|
28
|
-
offending = LEGACY_KEYS.select { |k| intake.key?(k) }
|
|
29
|
-
next if offending.empty?
|
|
30
|
-
|
|
31
|
-
out << issue_for(entry["key"], offending)
|
|
32
|
-
end
|
|
33
|
-
out
|
|
34
|
-
end
|
|
35
|
-
|
|
36
|
-
private
|
|
37
|
-
|
|
38
|
-
def safe_load(path)
|
|
39
|
-
YAML.load_file(path)
|
|
40
|
-
rescue StandardError
|
|
41
|
-
nil
|
|
42
|
-
end
|
|
43
|
-
|
|
44
|
-
def issue_for(key, fields)
|
|
45
|
-
{
|
|
46
|
-
"code" => "manifest.legacy_intake_fields",
|
|
47
|
-
"level" => "error",
|
|
48
|
-
"subject" => key.to_s,
|
|
49
|
-
"message" => "entry '#{key}' carries legacy intake.#{fields.join(", intake.")} " \
|
|
50
|
-
"(removed in 0.9.2 — freshness lives in top-level policies:)",
|
|
51
|
-
"fix" => "hand-edit these into a top-level policies: block (see CHANGELOG migration recipe)",
|
|
52
|
-
}
|
|
53
|
-
end
|
|
54
|
-
end
|
|
55
|
-
end
|
|
56
|
-
end
|
|
57
|
-
end
|