textus 0.51.0 → 0.52.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +12 -0
- data/README.md +19 -19
- data/SPEC.md +41 -39
- data/docs/architecture/README.md +9 -9
- data/docs/reference/conventions.md +8 -8
- data/lib/textus/boot.rb +7 -5
- data/lib/textus/cli/runner.rb +2 -2
- data/lib/textus/cli/verb/put.rb +1 -1
- data/lib/textus/cli/verb/serve.rb +19 -0
- data/lib/textus/dispatcher.rb +3 -1
- data/lib/textus/doctor/check/generator_drift.rb +1 -1
- data/lib/textus/doctor/check/sentinels.rb +2 -2
- data/lib/textus/domain/freshness/evaluator.rb +2 -2
- data/lib/textus/domain/jobs/job.rb +58 -0
- data/lib/textus/domain/jobs/registry.rb +37 -0
- data/lib/textus/domain/policy/base_guards.rb +1 -1
- data/lib/textus/domain/policy/retention.rb +1 -1
- data/lib/textus/domain/policy/source.rb +4 -10
- data/lib/textus/errors.rb +2 -2
- data/lib/textus/hooks/catalog.rb +0 -1
- data/lib/textus/init/templates/machine_intake.rb +1 -1
- data/lib/textus/init.rb +4 -4
- data/lib/textus/jobs/handlers.rb +62 -0
- data/lib/textus/jobs/scheduler.rb +36 -0
- data/lib/textus/jobs/seeder.rb +57 -0
- data/lib/textus/layout.rb +8 -0
- data/lib/textus/maintenance/drain.rb +42 -0
- data/lib/textus/maintenance/retention/apply.rb +52 -0
- data/lib/textus/maintenance/serve.rb +30 -0
- data/lib/textus/maintenance/worker.rb +74 -0
- data/lib/textus/manifest/capabilities.rb +1 -1
- data/lib/textus/manifest/data.rb +16 -1
- data/lib/textus/manifest/schema/keys.rb +1 -1
- data/lib/textus/manifest/schema/validator.rb +3 -3
- data/lib/textus/manifest/schema/vocabulary.rb +2 -2
- data/lib/textus/mcp/server.rb +1 -1
- data/lib/textus/ports/build_lock.rb +1 -1
- data/lib/textus/ports/produce_on_write_subscriber.rb +28 -24
- data/lib/textus/ports/queue.rb +130 -0
- data/lib/textus/produce/acquire/handler.rb +1 -1
- data/lib/textus/produce/acquire/intake.rb +3 -3
- data/lib/textus/produce/engine.rb +10 -58
- data/lib/textus/produce/events.rb +1 -1
- data/lib/textus/read/freshness.rb +2 -2
- data/lib/textus/read/get.rb +3 -3
- data/lib/textus/read/jobs.rb +31 -0
- data/lib/textus/role.rb +1 -1
- data/lib/textus/version.rb +1 -1
- data/lib/textus/write/enqueue.rb +50 -0
- metadata +14 -2
- data/lib/textus/maintenance/reconcile.rb +0 -160
|
@@ -51,7 +51,7 @@ Tooling around `git blame` or audit logs may filter on owner; the gem itself onl
|
|
|
51
51
|
|
|
52
52
|
A derived entry declares a `source:` block with a `from:` discriminator (ADR 0093/0094). A `source:` acquires **data** — it never renders; rendering is a publish concern (below). Two `from:` values for derived entries:
|
|
53
53
|
|
|
54
|
-
**`source: { from: project }`** — textus computes the entry's data on `textus
|
|
54
|
+
**`source: { from: project }`** — textus computes the entry's data on `textus drain` from other store entries. Declarative; nothing shells out. Projection fields are flat under `source:`.
|
|
55
55
|
|
|
56
56
|
```yaml
|
|
57
57
|
- key: artifacts.derived.people
|
|
@@ -69,7 +69,7 @@ A derived entry declares a `source:` block with a `from:` discriminator (ADR 009
|
|
|
69
69
|
- { to: docs/people.md, template: people.mustache } # render the data through a template
|
|
70
70
|
```
|
|
71
71
|
|
|
72
|
-
**`source: { from: command }`** — an external build tool (rake, just, a shell script) produces the file. textus never executes the `command:`; it only tracks `sources:` so `doctor`'s `generator_drift` check can compare source mtimes against the file's `_meta.generated.at`. The role running the build must hold `
|
|
72
|
+
**`source: { from: command }`** — an external build tool (rake, just, a shell script) produces the file. textus never executes the `command:`; it only tracks `sources:` so `doctor`'s `generator_drift` check can compare source mtimes against the file's `_meta.generated.at`. The role running the build must hold `converge` (default: `automation`).
|
|
73
73
|
|
|
74
74
|
```yaml
|
|
75
75
|
- key: artifacts.derived.skills
|
|
@@ -88,11 +88,11 @@ Full contract for both shapes is in [`../../SPEC.md` §5.2.1 and §5.2.2](../../
|
|
|
88
88
|
|
|
89
89
|
## Intake and freshness
|
|
90
90
|
|
|
91
|
-
External inputs land via `:resolve_handler` hooks, not shell commands. Each intake entry declares `source: { from: handler, handler: <name>, ttl: <dur> }`; re-pull is system-pushed via `
|
|
91
|
+
External inputs land via `:resolve_handler` hooks, not shell commands. Each intake entry declares `source: { from: handler, handler: <name>, ttl: <dur> }`; re-pull is system-pushed via `drain` (scheduled sweep) and `hook run` (event push) — a `get` never refreshes (ADR 0089):
|
|
92
92
|
|
|
93
93
|
```sh
|
|
94
94
|
textus pulse --output=json # `stale` lists expired entries; `next_due_at` is the soonest deadline
|
|
95
|
-
textus
|
|
95
|
+
textus drain --as=automation # re-pulls every intake entry past its source.ttl
|
|
96
96
|
```
|
|
97
97
|
|
|
98
98
|
The re-pull cadence is the entry's own `source.ttl`. Age-based garbage collection is the orthogonal `retention:` rule slot in the top-level `rules:` block, matched by glob (`{ ttl, action: drop | archive }`); the two compose — re-pull hourly *and* archive at 90 days:
|
|
@@ -108,11 +108,11 @@ rules:
|
|
|
108
108
|
retention: { ttl: 90d, action: archive }
|
|
109
109
|
```
|
|
110
110
|
|
|
111
|
-
A typical scheduled integration runs `
|
|
111
|
+
A typical scheduled integration runs `drain` on a cron to re-pull every
|
|
112
112
|
expired feed:
|
|
113
113
|
|
|
114
114
|
```sh
|
|
115
|
-
textus
|
|
115
|
+
textus drain --as=automation # in cron / CI — re-pulls all stale feeds
|
|
116
116
|
```
|
|
117
117
|
|
|
118
118
|
See [`./zones.md` §6](zones.md) for the full intake contract and [`../how-to/writing-hooks.md`](../how-to/writing-hooks.md) for writing custom handlers.
|
|
@@ -123,9 +123,9 @@ There is one public read operation, and it is pure (ADR 0089):
|
|
|
123
123
|
|
|
124
124
|
| Operation | Behaviour | Use for |
|
|
125
125
|
|-----------|-----------|---------|
|
|
126
|
-
| `ops.get` | A pure on-disk read annotated with a freshness verdict — it NEVER ingests, regardless of the entry's `action`. A stale `refresh` entry reads back stale until the next `
|
|
126
|
+
| `ops.get` | A pure on-disk read annotated with a freshness verdict — it NEVER ingests, regardless of the entry's `action`. A stale `refresh` entry reads back stale until the next `drain`. | every caller — interactive reads, dashboards, scripts, and internal pipelines (materializer, projection, schema tooling, accept/reject/publish, uid, validator) |
|
|
127
127
|
|
|
128
|
-
Refreshing a stale entry is `
|
|
128
|
+
Refreshing a stale entry is `drain`'s job (or a `hook run` event), never a read's — so no caller can accidentally trigger network I/O by reading.
|
|
129
129
|
|
|
130
130
|
## Body content
|
|
131
131
|
|
data/lib/textus/boot.rb
CHANGED
|
@@ -26,9 +26,9 @@ module Textus
|
|
|
26
26
|
"propose changes by writing #{manifest.policy.queue_zone}.* entries with --as=#{name} " \
|
|
27
27
|
"and a 'proposal:' frontmatter block; the #{authority} role runs 'textus accept' to apply"
|
|
28
28
|
end,
|
|
29
|
-
|
|
29
|
+
converge: lambda do |_name, manifest|
|
|
30
30
|
machine = zone_label(manifest, :machine, "machine")
|
|
31
|
-
"'textus
|
|
31
|
+
"'textus drain' materializes derived #{machine} entries from their sources and " \
|
|
32
32
|
"refreshes stale intake #{machine} entries from their declared source; " \
|
|
33
33
|
"derived files are never hand-edited (reactive on canon writes, or a full pass on demand)"
|
|
34
34
|
end,
|
|
@@ -84,13 +84,15 @@ module Textus
|
|
|
84
84
|
{ "name" => "put" },
|
|
85
85
|
{ "name" => "propose" },
|
|
86
86
|
{ "name" => "accept" },
|
|
87
|
+
{ "name" => "enqueue" },
|
|
87
88
|
{ "name" => "key", "summary" => "key operations: 'key delete', 'key mv', 'key uid'" },
|
|
88
|
-
{ "name" => "
|
|
89
|
+
{ "name" => "drain" },
|
|
89
90
|
{ "name" => "audit" },
|
|
90
91
|
{ "name" => "blame" },
|
|
91
92
|
{ "name" => "rule", "summary" => "inspect effective rules: 'rule list', 'rule explain KEY'" },
|
|
92
93
|
{ "name" => "doctor" },
|
|
93
94
|
{ "name" => "hook", "summary" => "list and run registered hooks: 'hook list', 'hook run NAME'" },
|
|
95
|
+
{ "name" => "jobs" },
|
|
94
96
|
{ "name" => "pulse" },
|
|
95
97
|
{ "name" => "capabilities" },
|
|
96
98
|
].freeze
|
|
@@ -172,11 +174,11 @@ module Textus
|
|
|
172
174
|
"accept #{queue}.KEY — promotes the proposal into its target zone",
|
|
173
175
|
],
|
|
174
176
|
},
|
|
175
|
-
"
|
|
177
|
+
"drain" => {
|
|
176
178
|
"purpose" => "keep the machine-maintained lanes fresh — re-pull stale intake entries from their declared source",
|
|
177
179
|
"steps" => [
|
|
178
180
|
"pulse — its `stale` list names entries past their ttl",
|
|
179
|
-
"
|
|
181
|
+
"drain (zone: #{feeds}) — re-pull the stale entries",
|
|
180
182
|
],
|
|
181
183
|
},
|
|
182
184
|
}
|
data/lib/textus/cli/runner.rb
CHANGED
|
@@ -132,7 +132,7 @@ module Textus
|
|
|
132
132
|
# affordance; the agent surface deliberately returns nil)
|
|
133
133
|
# put — reads the entry JSON from --stdin (ADR 0089: just stores bytes,
|
|
134
134
|
# no --fetch transform)
|
|
135
|
-
# (build removed in ADR 0087: materialization is system-pushed via
|
|
135
|
+
# (build removed in ADR 0087: materialization is system-pushed via drain/serve)
|
|
136
136
|
BEHAVIORAL_HATCHES = %i[get put].freeze
|
|
137
137
|
|
|
138
138
|
# Contract verbs whose CLI is a plain `< Verb` command, not a projection at
|
|
@@ -144,7 +144,7 @@ module Textus
|
|
|
144
144
|
# the exit_code: res["ok"] ? 0 : 1 behavior — two things the generic
|
|
145
145
|
# projection cannot yet express; kept in ADR 0101 pending a future pass.)
|
|
146
146
|
# (fetch/fetch_all were removed in ADR 0079: Produce::Acquire::Intake is now internal,
|
|
147
|
-
# driven by the
|
|
147
|
+
# driven by the converge sweep (drain/serve) and hook run — ADR 0089 removed the
|
|
148
148
|
# read-through that once also drove it.)
|
|
149
149
|
NON_PROJECTED_CLI = %i[doctor].freeze
|
|
150
150
|
|
data/lib/textus/cli/verb/put.rb
CHANGED
|
@@ -14,7 +14,7 @@ module Textus
|
|
|
14
14
|
role = resolved_role(store)
|
|
15
15
|
|
|
16
16
|
# put only stores the stdin JSON (ADR 0089): no transform-on-write.
|
|
17
|
-
# Ingest (running a handler over bytes) is system-pushed via
|
|
17
|
+
# Ingest (running a handler over bytes) is system-pushed via drain/serve
|
|
18
18
|
# and hook run, never a put flag.
|
|
19
19
|
payload = JSON.parse(@stdin.read)
|
|
20
20
|
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
module Textus
|
|
2
|
+
class CLI
|
|
3
|
+
class Verb
|
|
4
|
+
# Launches the convergence daemon in the current process. Blocks forever;
|
|
5
|
+
# reclaims crashed leases and drains the queue each tick (Phase 3 adds
|
|
6
|
+
# scheduled TTL re-pull/sweep). CLI-only — agents enqueue work, they do not
|
|
7
|
+
# run daemons. Acts as the automation role (the build authority).
|
|
8
|
+
class Serve < Verb
|
|
9
|
+
command_name "serve"
|
|
10
|
+
|
|
11
|
+
def call(store)
|
|
12
|
+
call = Textus::Call.build(role: Textus::Role::AUTOMATION)
|
|
13
|
+
Textus::Maintenance::Serve.new(container: store.container, call: call).run
|
|
14
|
+
0
|
|
15
|
+
end
|
|
16
|
+
end
|
|
17
|
+
end
|
|
18
|
+
end
|
|
19
|
+
end
|
data/lib/textus/dispatcher.rb
CHANGED
|
@@ -11,6 +11,7 @@ module Textus
|
|
|
11
11
|
key_mv: Textus::Write::KeyMv,
|
|
12
12
|
accept: Textus::Write::Accept,
|
|
13
13
|
reject: Textus::Write::Reject,
|
|
14
|
+
enqueue: Textus::Write::Enqueue,
|
|
14
15
|
# Read
|
|
15
16
|
get: Textus::Read::Get,
|
|
16
17
|
list: Textus::Read::List,
|
|
@@ -30,12 +31,13 @@ module Textus
|
|
|
30
31
|
doctor: Textus::Read::Doctor,
|
|
31
32
|
boot: Textus::Read::Boot,
|
|
32
33
|
capabilities: Textus::Read::Capabilities,
|
|
34
|
+
jobs: Textus::Read::Jobs,
|
|
33
35
|
|
|
34
36
|
# Maintenance
|
|
35
37
|
zone_mv: Textus::Maintenance::ZoneMv,
|
|
36
38
|
key_mv_prefix: Textus::Maintenance::KeyMvPrefix,
|
|
37
39
|
key_delete_prefix: Textus::Maintenance::KeyDeletePrefix,
|
|
38
|
-
|
|
40
|
+
drain: Textus::Maintenance::Drain,
|
|
39
41
|
rule_lint: Textus::Maintenance::RuleLint,
|
|
40
42
|
}.freeze
|
|
41
43
|
|
|
@@ -31,7 +31,7 @@ module Textus
|
|
|
31
31
|
"level" => "warning",
|
|
32
32
|
"subject" => sentinel_path,
|
|
33
33
|
"message" => "sentinel is not valid JSON",
|
|
34
|
-
"fix" => "delete #{sentinel_path} and re-run 'textus
|
|
34
|
+
"fix" => "delete #{sentinel_path} and re-run 'textus drain' to regenerate",
|
|
35
35
|
}
|
|
36
36
|
end
|
|
37
37
|
|
|
@@ -51,7 +51,7 @@ module Textus
|
|
|
51
51
|
"level" => "warning",
|
|
52
52
|
"subject" => sentinel.target,
|
|
53
53
|
"message" => "published file at #{sentinel.target} was modified out-of-band",
|
|
54
|
-
"fix" => "re-run 'textus
|
|
54
|
+
"fix" => "re-run 'textus drain' to overwrite, or copy the manual edit back into the store source",
|
|
55
55
|
}
|
|
56
56
|
end
|
|
57
57
|
end
|
|
@@ -10,7 +10,7 @@ module Textus
|
|
|
10
10
|
# (skipped — a cadence-less handler is not auto-re-pulled).
|
|
11
11
|
# - derived/external -> DRIFT signal: a source changed since generated.at
|
|
12
12
|
# (surfaced by the doctor generator_drift check; derived entries annotate
|
|
13
|
-
# fresh at read time because
|
|
13
|
+
# fresh at read time because converge runs them reactively).
|
|
14
14
|
# Replaces Domain::IntakeStaleness and Domain::Staleness::GeneratorCheck and
|
|
15
15
|
# the inline copies in Read::Get / Read::Freshness.
|
|
16
16
|
class Evaluator
|
|
@@ -32,7 +32,7 @@ module Textus
|
|
|
32
32
|
Verdict.build(stale: stale, reason: stale ? "ttl exceeded" : nil, fetching: false)
|
|
33
33
|
end
|
|
34
34
|
|
|
35
|
-
# Keys of intake entries past their source.ttl — the
|
|
35
|
+
# Keys of intake entries past their source.ttl — the converge produce
|
|
36
36
|
# scope (replaces Domain::IntakeStaleness#call). A ttl-less intake entry
|
|
37
37
|
# is :no_policy and skipped; a never-recorded one (with a ttl) is stale.
|
|
38
38
|
def stale_intake_keys(prefix: nil, zone: nil)
|
|
@@ -0,0 +1,58 @@
|
|
|
1
|
+
require "digest"
|
|
2
|
+
require "json"
|
|
3
|
+
|
|
4
|
+
module Textus
|
|
5
|
+
module Domain
|
|
6
|
+
module Jobs
|
|
7
|
+
# A unit of deferred work. Pure data. The id is `<type>:<digest>` where the
|
|
8
|
+
# digest is over the args with sorted keys, so logically-identical enqueues
|
|
9
|
+
# collide on the same id — which is how the Queue dedups (the file already
|
|
10
|
+
# exists). At-least-once delivery means handlers must be idempotent.
|
|
11
|
+
class Job
|
|
12
|
+
DIGEST_BYTES = 16
|
|
13
|
+
|
|
14
|
+
attr_reader :type, :args, :enqueued_by, :max_attempts
|
|
15
|
+
attr_accessor :attempts, :last_error
|
|
16
|
+
|
|
17
|
+
def initialize(type:, args:, enqueued_by: nil, attempts: 0, max_attempts: 3, last_error: nil)
|
|
18
|
+
@type = type.to_s
|
|
19
|
+
@args = stringify(args)
|
|
20
|
+
@enqueued_by = enqueued_by
|
|
21
|
+
@attempts = attempts
|
|
22
|
+
@max_attempts = max_attempts
|
|
23
|
+
@last_error = last_error
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
def id
|
|
27
|
+
"#{@type}:#{digest}"
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
def to_h
|
|
31
|
+
{
|
|
32
|
+
"type" => @type, "args" => @args, "enqueued_by" => @enqueued_by,
|
|
33
|
+
"attempts" => @attempts, "max_attempts" => @max_attempts, "last_error" => @last_error
|
|
34
|
+
}
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
def self.from_h(hash)
|
|
38
|
+
new(
|
|
39
|
+
type: hash["type"], args: hash["args"] || {}, enqueued_by: hash["enqueued_by"],
|
|
40
|
+
attempts: hash["attempts"] || 0, max_attempts: hash["max_attempts"] || 3,
|
|
41
|
+
last_error: hash["last_error"]
|
|
42
|
+
)
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
private
|
|
46
|
+
|
|
47
|
+
def digest
|
|
48
|
+
canonical = JSON.dump(@args.sort.to_h)
|
|
49
|
+
Digest::SHA256.hexdigest(canonical)[0, DIGEST_BYTES]
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
def stringify(hash)
|
|
53
|
+
hash.transform_keys(&:to_s)
|
|
54
|
+
end
|
|
55
|
+
end
|
|
56
|
+
end
|
|
57
|
+
end
|
|
58
|
+
end
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
module Textus
|
|
2
|
+
module Domain
|
|
3
|
+
module Jobs
|
|
4
|
+
# Closed allow-list of runnable job types. The general `enqueue` surface
|
|
5
|
+
# (a later phase) can only push types registered here — that is the safety
|
|
6
|
+
# boundary that stops the "general runner" from running arbitrary code.
|
|
7
|
+
class Registry
|
|
8
|
+
Entry = Struct.new(:handler, :max_attempts, :required_role, keyword_init: true)
|
|
9
|
+
|
|
10
|
+
def initialize
|
|
11
|
+
@entries = {}
|
|
12
|
+
end
|
|
13
|
+
|
|
14
|
+
# required_role: a role the caller must hold to enqueue this type via the
|
|
15
|
+
# general `enqueue` surface (nil = any caller). The closed allow-list is
|
|
16
|
+
# the primary safety boundary; this is defence-in-depth for destructive
|
|
17
|
+
# types.
|
|
18
|
+
def register(type, handler:, max_attempts: 3, required_role: nil)
|
|
19
|
+
@entries[type.to_s] = Entry.new(handler: handler, max_attempts: max_attempts, required_role: required_role)
|
|
20
|
+
end
|
|
21
|
+
|
|
22
|
+
def registered?(type)
|
|
23
|
+
@entries.key?(type.to_s)
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
def lookup(type)
|
|
27
|
+
@entries.fetch(type.to_s) do
|
|
28
|
+
raise Textus::UsageError.new(
|
|
29
|
+
"unregistered job type '#{type}'",
|
|
30
|
+
hint: "register the type in Domain::Jobs::Registry before enqueueing it",
|
|
31
|
+
)
|
|
32
|
+
end
|
|
33
|
+
end
|
|
34
|
+
end
|
|
35
|
+
end
|
|
36
|
+
end
|
|
37
|
+
end
|
|
@@ -15,7 +15,7 @@ module Textus
|
|
|
15
15
|
key_mv: %w[zone_writable_by],
|
|
16
16
|
accept: %w[author_held target_is_canon],
|
|
17
17
|
reject: %w[author_held],
|
|
18
|
-
|
|
18
|
+
converge: %w[zone_writable_by],
|
|
19
19
|
}.freeze
|
|
20
20
|
|
|
21
21
|
def self.for(transition) = BASE.fetch(transition, [])
|
|
@@ -3,7 +3,7 @@ module Textus
|
|
|
3
3
|
module Policy
|
|
4
4
|
# Garbage collection (ADR 0093). A glob-matched rule slot: when an entry
|
|
5
5
|
# ages past `ttl`, retire it. Destructive only — runs on the full
|
|
6
|
-
# `
|
|
6
|
+
# `converge` pass, never on a write (ADR 0079's invariant). Orthogonal to
|
|
7
7
|
# production (`source:`): an intake entry can re-pull hourly AND archive
|
|
8
8
|
# after 90 days. `warn`/`refresh` are gone (refresh is implied by an
|
|
9
9
|
# intake source; warn never fired after ADR 0089's pure-read get).
|
|
@@ -7,11 +7,11 @@ module Textus
|
|
|
7
7
|
# from: project -> derived (internal projection; observable -> rdeps staleness)
|
|
8
8
|
# from: handler -> intake (external fetch; unobservable -> ttl staleness)
|
|
9
9
|
# from: command -> external (out-of-band runner; staleness only, textus never runs it)
|
|
10
|
-
#
|
|
11
|
-
#
|
|
10
|
+
# Materialization is async-only (job-queue model): a write enqueues a
|
|
11
|
+
# `materialize` job, converged by a worker. There is no per-entry write
|
|
12
|
+
# trigger knob.
|
|
12
13
|
class Source
|
|
13
|
-
FROMS
|
|
14
|
-
STRATEGIES = %w[async sync].freeze
|
|
14
|
+
FROMS = %w[project handler command].freeze
|
|
15
15
|
|
|
16
16
|
attr_reader :from, :handler, :config, :command, :sources
|
|
17
17
|
|
|
@@ -21,11 +21,6 @@ module Textus
|
|
|
21
21
|
raise Textus::BadManifest.new("source.from must be one of #{FROMS.join("|")}, got #{raw["from"].inspect}")
|
|
22
22
|
end
|
|
23
23
|
|
|
24
|
-
@on_write = (raw["on_write"] || "async").to_s
|
|
25
|
-
unless STRATEGIES.include?(@on_write)
|
|
26
|
-
raise Textus::BadManifest.new("source.on_write must be one of #{STRATEGIES.join("/")}, got #{@on_write.inspect}")
|
|
27
|
-
end
|
|
28
|
-
|
|
29
24
|
@ttl = raw["ttl"]
|
|
30
25
|
@projection = {}
|
|
31
26
|
|
|
@@ -39,7 +34,6 @@ module Textus
|
|
|
39
34
|
def kind = @from == "handler" ? :intake : :derived
|
|
40
35
|
def external? = @from == "command"
|
|
41
36
|
def projection? = @from == "project"
|
|
42
|
-
def sync? = @on_write == "sync"
|
|
43
37
|
def ttl_seconds = @ttl.nil? ? nil : Textus::Domain::Duration.seconds(@ttl)
|
|
44
38
|
|
|
45
39
|
# Flattened projection accessors (ADR 0094) — read directly off the source
|
data/lib/textus/errors.rb
CHANGED
|
@@ -166,9 +166,9 @@ module Textus
|
|
|
166
166
|
def initialize(m, format: nil)
|
|
167
167
|
hint =
|
|
168
168
|
if format
|
|
169
|
-
"the template rendered invalid #{format}; try rendering with mock data and parsing the output before re-running
|
|
169
|
+
"the template rendered invalid #{format}; try rendering with mock data and parsing the output before re-running drain"
|
|
170
170
|
else
|
|
171
|
-
"the template rendered invalid content; try rendering with mock data and parsing the output before re-running
|
|
171
|
+
"the template rendered invalid content; try rendering with mock data and parsing the output before re-running drain"
|
|
172
172
|
end
|
|
173
173
|
super("bad_render", m, hint: hint)
|
|
174
174
|
end
|
data/lib/textus/hooks/catalog.rb
CHANGED
|
@@ -17,7 +17,6 @@ module Textus
|
|
|
17
17
|
entry_renamed: %i[ctx key from_key to_key envelope],
|
|
18
18
|
entry_produced: %i[ctx key envelope sources],
|
|
19
19
|
produce_failed: %i[ctx keys error],
|
|
20
|
-
reconcile_failed: %i[ctx failed],
|
|
21
20
|
proposal_accepted: %i[ctx key target_key],
|
|
22
21
|
proposal_rejected: %i[ctx key target_key],
|
|
23
22
|
entry_published: %i[ctx key envelope source target],
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# .textus/hooks/machine_intake.rb
|
|
2
2
|
# Scaffolded by `textus init` — CUSTOMIZE FREELY, or delete the feeds.machines
|
|
3
3
|
# entry from manifest.yaml if you don't want it.
|
|
4
|
-
# Feeds a per-host SNAPSHOT into feeds.machines.<host> on `textus
|
|
4
|
+
# Feeds a per-host SNAPSHOT into feeds.machines.<host> on `textus drain`
|
|
5
5
|
# (never on the per-turn boot/pulse path). It is NESTED so it grows to a fleet: the
|
|
6
6
|
# `local` leaf scans THIS host; add ssh hosts with the cookbook recipe
|
|
7
7
|
# (docs/cookbook/environment-scan.md). tracked:false → gitignored. Keep this an
|
data/lib/textus/init.rb
CHANGED
|
@@ -10,7 +10,7 @@ module Textus
|
|
|
10
10
|
roles:
|
|
11
11
|
- { name: human, can: [author, propose] }
|
|
12
12
|
- { name: agent, can: [propose, keep] }
|
|
13
|
-
- { name: automation, can: [
|
|
13
|
+
- { name: automation, can: [converge] }
|
|
14
14
|
zones:
|
|
15
15
|
- { name: knowledge, kind: canon, desc: "the maintained source of truth (identity.* lives here)" }
|
|
16
16
|
- { name: notebook, kind: workspace, owner: agent, desc: "the agent's own durable working notes" }
|
|
@@ -21,7 +21,7 @@ module Textus
|
|
|
21
21
|
- { key: knowledge.notes, path: knowledge/notes, zone: knowledge, schema: null, owner: human:self, nested: true, kind: nested }
|
|
22
22
|
- { key: notebook.notes, path: notebook/notes, zone: notebook, schema: null, owner: agent:self, nested: true, kind: nested }
|
|
23
23
|
- { key: proposals.notes, path: proposals/notes, zone: proposals, schema: null, owner: agent:self, nested: true, kind: nested }
|
|
24
|
-
# A per-host snapshot, refreshed from its declared intake by `textus
|
|
24
|
+
# A per-host snapshot, refreshed from its declared intake by `textus drain` (scheduled, or on demand).
|
|
25
25
|
# Nested so it grows to a fleet — add artifacts.feeds.machines.<host> leaves over SSH
|
|
26
26
|
# (see docs/cookbook/environment-scan.md) without renaming. tracked:false →
|
|
27
27
|
# gitignored (machine info can be sensitive/noisy) but still protocol-readable
|
|
@@ -95,7 +95,7 @@ module Textus
|
|
|
95
95
|
|
|
96
96
|
Events: :resolve_handler, :transform_rows, :validate (rpc — return value used)
|
|
97
97
|
:entry_written, :entry_deleted, :entry_fetched, :entry_renamed,
|
|
98
|
-
:entry_produced, :produce_failed,
|
|
98
|
+
:entry_produced, :produce_failed,
|
|
99
99
|
:proposal_accepted, :proposal_rejected,
|
|
100
100
|
:entry_published, :store_loaded, :session_opened,
|
|
101
101
|
:entry_fetch_started, :entry_fetch_failed (pub-sub — return discarded)
|
|
@@ -105,7 +105,7 @@ module Textus
|
|
|
105
105
|
|
|
106
106
|
AGENT_ENTRIES = <<~YAML.gsub(/^/, " ")
|
|
107
107
|
# --with-agent profile: project facts + runbooks feed the orientation
|
|
108
|
-
# projection below, which `textus
|
|
108
|
+
# projection below, which `textus drain` renders to CLAUDE.md/AGENTS.md.
|
|
109
109
|
- { key: knowledge.project, path: knowledge/project.md, zone: knowledge, schema: project, owner: human:self, kind: leaf }
|
|
110
110
|
- { key: knowledge.runbooks, path: knowledge/runbooks, zone: knowledge, schema: runbook, owner: human:self, nested: true, kind: nested }
|
|
111
111
|
- key: artifacts.derived.orientation
|
|
@@ -0,0 +1,62 @@
|
|
|
1
|
+
module Textus
|
|
2
|
+
module Jobs
|
|
3
|
+
# Wires the closed allow-list of convergence job types to the existing
|
|
4
|
+
# convergence code. Authority is read from the job's frozen `enqueued_by`
|
|
5
|
+
# and turned into the Call the handler runs under: produce self-elevates
|
|
6
|
+
# inside Produce::Engine regardless; destructive sweep runs AS the caller.
|
|
7
|
+
module Handlers
|
|
8
|
+
module_function
|
|
9
|
+
|
|
10
|
+
def registry
|
|
11
|
+
reg = Textus::Domain::Jobs::Registry.new
|
|
12
|
+
# produce is pure (self-elevates) — any caller may request a rematerialize.
|
|
13
|
+
reg.register("materialize", handler: method(:produce))
|
|
14
|
+
reg.register("re-pull", handler: method(:produce))
|
|
15
|
+
# sweep is destructive — gate ad-hoc enqueue to the automation authority.
|
|
16
|
+
reg.register("sweep", handler: method(:sweep), required_role: Textus::Role::AUTOMATION)
|
|
17
|
+
reg
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
# produce: render derived / re-pull intake for a single key. Engine
|
|
21
|
+
# self-elevates to the build actor internally; the passed call carries
|
|
22
|
+
# only correlation/dry_run plus the stamped role for audit. Engine#call
|
|
23
|
+
# isolates per-key produce errors into its result hash rather than raising,
|
|
24
|
+
# so surface them as :produce_failed events (the converge result hash used
|
|
25
|
+
# to carry them; the worker drops the return, so re-publish here).
|
|
26
|
+
def produce(job:, container:)
|
|
27
|
+
call = call_for(job)
|
|
28
|
+
result = Textus::Produce::Engine.converge(container: container, call: call, keys: [job.args["key"]])
|
|
29
|
+
return unless result.is_a?(Hash)
|
|
30
|
+
|
|
31
|
+
Array(result[:failed]).each do |failure|
|
|
32
|
+
container.events.publish(
|
|
33
|
+
:produce_failed,
|
|
34
|
+
ctx: Textus::Hooks::Context.for(container: container, call: call),
|
|
35
|
+
keys: [failure["key"]], error: failure["error"]
|
|
36
|
+
)
|
|
37
|
+
end
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
# sweep: compute retention rows for the scope, then apply destructively AS
|
|
41
|
+
# the job's role (no self-elevation).
|
|
42
|
+
def sweep(job:, container:)
|
|
43
|
+
call = call_for(job)
|
|
44
|
+
scope = job.args["scope"]
|
|
45
|
+
rows = Textus::Domain::Retention::Sweep.new(
|
|
46
|
+
manifest: container.manifest,
|
|
47
|
+
file_stat: Textus::Ports::Storage::FileStat.new,
|
|
48
|
+
clock: Textus::Ports::Clock.new,
|
|
49
|
+
).call(prefix: scope_prefix(scope), zone: scope_zone(scope))
|
|
50
|
+
Textus::Maintenance::Retention::Apply.new(container: container, call: call).call(rows)
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
def call_for(job)
|
|
54
|
+
Textus::Call.build(role: job.enqueued_by || Textus::Role::AUTOMATION)
|
|
55
|
+
end
|
|
56
|
+
|
|
57
|
+
# A scope is `{ "prefix" => ..., "zone" => ... }` or nil (whole store).
|
|
58
|
+
def scope_prefix(scope) = scope.is_a?(Hash) ? scope["prefix"] : nil
|
|
59
|
+
def scope_zone(scope) = scope.is_a?(Hash) ? scope["zone"] : nil
|
|
60
|
+
end
|
|
61
|
+
end
|
|
62
|
+
end
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
module Textus
|
|
2
|
+
module Jobs
|
|
3
|
+
# Time-based seeding for the daemon: at each tick, enqueue a re-pull job for
|
|
4
|
+
# every intake key past its source.ttl and a sweep job to GC entries past
|
|
5
|
+
# retention.ttl. Dedup means a job already queued from a prior tick is a
|
|
6
|
+
# no-op. Both are stamped automation (the daemon's own authority); the sweep
|
|
7
|
+
# handler runs retention as that role.
|
|
8
|
+
class Scheduler
|
|
9
|
+
def initialize(container:, queue:)
|
|
10
|
+
@container = container
|
|
11
|
+
@queue = queue
|
|
12
|
+
end
|
|
13
|
+
|
|
14
|
+
def run_once
|
|
15
|
+
stale_intake.each do |key|
|
|
16
|
+
@queue.enqueue(job("re-pull", { "key" => key }))
|
|
17
|
+
end
|
|
18
|
+
@queue.enqueue(job("sweep", { "scope" => { "prefix" => nil, "zone" => nil } }))
|
|
19
|
+
end
|
|
20
|
+
|
|
21
|
+
private
|
|
22
|
+
|
|
23
|
+
def stale_intake
|
|
24
|
+
Textus::Domain::Freshness::Evaluator.new(
|
|
25
|
+
manifest: @container.manifest,
|
|
26
|
+
file_stat: Textus::Ports::Storage::FileStat.new,
|
|
27
|
+
clock: Textus::Ports::Clock.new,
|
|
28
|
+
).stale_intake_keys(prefix: nil, zone: nil)
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
def job(type, args)
|
|
32
|
+
Textus::Domain::Jobs::Job.new(type: type, args: args, enqueued_by: Textus::Role::AUTOMATION)
|
|
33
|
+
end
|
|
34
|
+
end
|
|
35
|
+
end
|
|
36
|
+
end
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
module Textus
|
|
2
|
+
module Jobs
|
|
3
|
+
# Enqueues the full convergence set for a scope: a produce job per derived /
|
|
4
|
+
# publish_tree / publish.to entry, a re-pull job per stale intake key, and a
|
|
5
|
+
# single sweep job for the scope. The scope logic mirrors
|
|
6
|
+
# the converge scope (Produce::Engine) so `drain` and `serve` converge identically.
|
|
7
|
+
# Produce jobs self-elevate (stamped automation); the sweep job carries the
|
|
8
|
+
# caller's role (destructive runs as caller).
|
|
9
|
+
class Seeder
|
|
10
|
+
def initialize(container:, queue:, call:)
|
|
11
|
+
@container = container
|
|
12
|
+
@queue = queue
|
|
13
|
+
@call = call
|
|
14
|
+
@manifest = container.manifest
|
|
15
|
+
end
|
|
16
|
+
|
|
17
|
+
def seed(prefix:, zone:)
|
|
18
|
+
file_stat = Textus::Ports::Storage::FileStat.new
|
|
19
|
+
|
|
20
|
+
producible_keys(prefix, zone).each do |key|
|
|
21
|
+
@queue.enqueue(job("materialize", { "key" => key }, Textus::Role::AUTOMATION))
|
|
22
|
+
end
|
|
23
|
+
stale_intake_keys(prefix, zone, file_stat).each do |key|
|
|
24
|
+
@queue.enqueue(job("re-pull", { "key" => key }, Textus::Role::AUTOMATION))
|
|
25
|
+
end
|
|
26
|
+
@queue.enqueue(job("sweep", { "scope" => { "prefix" => prefix, "zone" => zone } }, @call.role))
|
|
27
|
+
end
|
|
28
|
+
|
|
29
|
+
private
|
|
30
|
+
|
|
31
|
+
def job(type, args, role)
|
|
32
|
+
Textus::Domain::Jobs::Job.new(type: type, args: args, enqueued_by: role)
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
# Mirrors the converge scope (the publishable arm).
|
|
36
|
+
def producible_keys(prefix, zone)
|
|
37
|
+
@manifest.data.entries
|
|
38
|
+
.select { |e| e.derived? || !e.publish_tree.nil? || !e.publish_to.empty? }
|
|
39
|
+
.select { |e| in_scope?(e, prefix, zone) }
|
|
40
|
+
.map(&:key)
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
def stale_intake_keys(prefix, zone, file_stat)
|
|
44
|
+
Textus::Domain::Freshness::Evaluator.new(
|
|
45
|
+
manifest: @manifest, file_stat: file_stat, clock: Textus::Ports::Clock.new,
|
|
46
|
+
).stale_intake_keys(prefix: prefix, zone: zone)
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
def in_scope?(entry, prefix, zone)
|
|
50
|
+
return false if zone && entry.zone != zone
|
|
51
|
+
return false if prefix && !entry.key.start_with?(prefix)
|
|
52
|
+
|
|
53
|
+
true
|
|
54
|
+
end
|
|
55
|
+
end
|
|
56
|
+
end
|
|
57
|
+
end
|
data/lib/textus/layout.rb
CHANGED
|
@@ -25,6 +25,14 @@ module Textus
|
|
|
25
25
|
File.join(run(root), "build.lock")
|
|
26
26
|
end
|
|
27
27
|
|
|
28
|
+
def self.queue(root)
|
|
29
|
+
File.join(run(root), "queue")
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def self.queue_state(root, state)
|
|
33
|
+
File.join(queue(root), state.to_s)
|
|
34
|
+
end
|
|
35
|
+
|
|
28
36
|
def self.audit_dir(root)
|
|
29
37
|
File.join(run(root), "audit")
|
|
30
38
|
end
|