textus 0.51.0 → 0.52.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (52) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +12 -0
  3. data/README.md +19 -19
  4. data/SPEC.md +41 -39
  5. data/docs/architecture/README.md +9 -9
  6. data/docs/reference/conventions.md +8 -8
  7. data/lib/textus/boot.rb +7 -5
  8. data/lib/textus/cli/runner.rb +2 -2
  9. data/lib/textus/cli/verb/put.rb +1 -1
  10. data/lib/textus/cli/verb/serve.rb +19 -0
  11. data/lib/textus/dispatcher.rb +3 -1
  12. data/lib/textus/doctor/check/generator_drift.rb +1 -1
  13. data/lib/textus/doctor/check/sentinels.rb +2 -2
  14. data/lib/textus/domain/freshness/evaluator.rb +2 -2
  15. data/lib/textus/domain/jobs/job.rb +58 -0
  16. data/lib/textus/domain/jobs/registry.rb +37 -0
  17. data/lib/textus/domain/policy/base_guards.rb +1 -1
  18. data/lib/textus/domain/policy/retention.rb +1 -1
  19. data/lib/textus/domain/policy/source.rb +4 -10
  20. data/lib/textus/errors.rb +2 -2
  21. data/lib/textus/hooks/catalog.rb +0 -1
  22. data/lib/textus/init/templates/machine_intake.rb +1 -1
  23. data/lib/textus/init.rb +4 -4
  24. data/lib/textus/jobs/handlers.rb +62 -0
  25. data/lib/textus/jobs/scheduler.rb +36 -0
  26. data/lib/textus/jobs/seeder.rb +57 -0
  27. data/lib/textus/layout.rb +8 -0
  28. data/lib/textus/maintenance/drain.rb +42 -0
  29. data/lib/textus/maintenance/retention/apply.rb +52 -0
  30. data/lib/textus/maintenance/serve.rb +30 -0
  31. data/lib/textus/maintenance/worker.rb +74 -0
  32. data/lib/textus/manifest/capabilities.rb +1 -1
  33. data/lib/textus/manifest/data.rb +16 -1
  34. data/lib/textus/manifest/schema/keys.rb +1 -1
  35. data/lib/textus/manifest/schema/validator.rb +3 -3
  36. data/lib/textus/manifest/schema/vocabulary.rb +2 -2
  37. data/lib/textus/mcp/server.rb +1 -1
  38. data/lib/textus/ports/build_lock.rb +1 -1
  39. data/lib/textus/ports/produce_on_write_subscriber.rb +28 -24
  40. data/lib/textus/ports/queue.rb +130 -0
  41. data/lib/textus/produce/acquire/handler.rb +1 -1
  42. data/lib/textus/produce/acquire/intake.rb +3 -3
  43. data/lib/textus/produce/engine.rb +10 -58
  44. data/lib/textus/produce/events.rb +1 -1
  45. data/lib/textus/read/freshness.rb +2 -2
  46. data/lib/textus/read/get.rb +3 -3
  47. data/lib/textus/read/jobs.rb +31 -0
  48. data/lib/textus/role.rb +1 -1
  49. data/lib/textus/version.rb +1 -1
  50. data/lib/textus/write/enqueue.rb +50 -0
  51. metadata +14 -2
  52. data/lib/textus/maintenance/reconcile.rb +0 -160
@@ -51,7 +51,7 @@ Tooling around `git blame` or audit logs may filter on owner; the gem itself onl
51
51
 
52
52
  A derived entry declares a `source:` block with a `from:` discriminator (ADR 0093/0094). A `source:` acquires **data** — it never renders; rendering is a publish concern (below). Two `from:` values for derived entries:
53
53
 
54
- **`source: { from: project }`** — textus computes the entry's data on `textus reconcile` from other store entries. Declarative; nothing shells out. Projection fields are flat under `source:`.
54
+ **`source: { from: project }`** — textus computes the entry's data on `textus drain` from other store entries. Declarative; nothing shells out. Projection fields are flat under `source:`.
55
55
 
56
56
  ```yaml
57
57
  - key: artifacts.derived.people
@@ -69,7 +69,7 @@ A derived entry declares a `source:` block with a `from:` discriminator (ADR 009
69
69
  - { to: docs/people.md, template: people.mustache } # render the data through a template
70
70
  ```
71
71
 
72
- **`source: { from: command }`** — an external build tool (rake, just, a shell script) produces the file. textus never executes the `command:`; it only tracks `sources:` so `doctor`'s `generator_drift` check can compare source mtimes against the file's `_meta.generated.at`. The role running the build must hold `reconcile` (default: `automation`).
72
+ **`source: { from: command }`** — an external build tool (rake, just, a shell script) produces the file. textus never executes the `command:`; it only tracks `sources:` so `doctor`'s `generator_drift` check can compare source mtimes against the file's `_meta.generated.at`. The role running the build must hold `converge` (default: `automation`).
73
73
 
74
74
  ```yaml
75
75
  - key: artifacts.derived.skills
@@ -88,11 +88,11 @@ Full contract for both shapes is in [`../../SPEC.md` §5.2.1 and §5.2.2](../../
88
88
 
89
89
  ## Intake and freshness
90
90
 
91
- External inputs land via `:resolve_handler` hooks, not shell commands. Each intake entry declares `source: { from: handler, handler: <name>, ttl: <dur> }`; re-pull is system-pushed via `reconcile` (scheduled sweep) and `hook run` (event push) — a `get` never refreshes (ADR 0089):
91
+ External inputs land via `:resolve_handler` hooks, not shell commands. Each intake entry declares `source: { from: handler, handler: <name>, ttl: <dur> }`; re-pull is system-pushed via `drain` (scheduled sweep) and `hook run` (event push) — a `get` never refreshes (ADR 0089):
92
92
 
93
93
  ```sh
94
94
  textus pulse --output=json # `stale` lists expired entries; `next_due_at` is the soonest deadline
95
- textus reconcile --as=automation # re-pulls every intake entry past its source.ttl
95
+ textus drain --as=automation # re-pulls every intake entry past its source.ttl
96
96
  ```
97
97
 
98
98
  The re-pull cadence is the entry's own `source.ttl`. Age-based garbage collection is the orthogonal `retention:` rule slot in the top-level `rules:` block, matched by glob (`{ ttl, action: drop | archive }`); the two compose — re-pull hourly *and* archive at 90 days:
@@ -108,11 +108,11 @@ rules:
108
108
  retention: { ttl: 90d, action: archive }
109
109
  ```
110
110
 
111
- A typical scheduled integration runs `reconcile` on a cron to re-pull every
111
+ A typical scheduled integration runs `drain` on a cron to re-pull every
112
112
  expired feed:
113
113
 
114
114
  ```sh
115
- textus reconcile --as=automation # in cron / CI — re-pulls all stale feeds
115
+ textus drain --as=automation # in cron / CI — re-pulls all stale feeds
116
116
  ```
117
117
 
118
118
  See [`./zones.md` §6](zones.md) for the full intake contract and [`../how-to/writing-hooks.md`](../how-to/writing-hooks.md) for writing custom handlers.
@@ -123,9 +123,9 @@ There is one public read operation, and it is pure (ADR 0089):
123
123
 
124
124
  | Operation | Behaviour | Use for |
125
125
  |-----------|-----------|---------|
126
- | `ops.get` | A pure on-disk read annotated with a freshness verdict — it NEVER ingests, regardless of the entry's `action`. A stale `refresh` entry reads back stale until the next `reconcile`. | every caller — interactive reads, dashboards, scripts, and internal pipelines (materializer, projection, schema tooling, accept/reject/publish, uid, validator) |
126
+ | `ops.get` | A pure on-disk read annotated with a freshness verdict — it NEVER ingests, regardless of the entry's `action`. A stale `refresh` entry reads back stale until the next `drain`. | every caller — interactive reads, dashboards, scripts, and internal pipelines (materializer, projection, schema tooling, accept/reject/publish, uid, validator) |
127
127
 
128
- Refreshing a stale entry is `reconcile`'s job (or a `hook run` event), never a read's — so no caller can accidentally trigger network I/O by reading.
128
+ Refreshing a stale entry is `drain`'s job (or a `hook run` event), never a read's — so no caller can accidentally trigger network I/O by reading.
129
129
 
130
130
  ## Body content
131
131
 
data/lib/textus/boot.rb CHANGED
@@ -26,9 +26,9 @@ module Textus
26
26
  "propose changes by writing #{manifest.policy.queue_zone}.* entries with --as=#{name} " \
27
27
  "and a 'proposal:' frontmatter block; the #{authority} role runs 'textus accept' to apply"
28
28
  end,
29
- reconcile: lambda do |_name, manifest|
29
+ converge: lambda do |_name, manifest|
30
30
  machine = zone_label(manifest, :machine, "machine")
31
- "'textus reconcile' materializes derived #{machine} entries from their sources and " \
31
+ "'textus drain' materializes derived #{machine} entries from their sources and " \
32
32
  "refreshes stale intake #{machine} entries from their declared source; " \
33
33
  "derived files are never hand-edited (reactive on canon writes, or a full pass on demand)"
34
34
  end,
@@ -84,13 +84,15 @@ module Textus
84
84
  { "name" => "put" },
85
85
  { "name" => "propose" },
86
86
  { "name" => "accept" },
87
+ { "name" => "enqueue" },
87
88
  { "name" => "key", "summary" => "key operations: 'key delete', 'key mv', 'key uid'" },
88
- { "name" => "reconcile" },
89
+ { "name" => "drain" },
89
90
  { "name" => "audit" },
90
91
  { "name" => "blame" },
91
92
  { "name" => "rule", "summary" => "inspect effective rules: 'rule list', 'rule explain KEY'" },
92
93
  { "name" => "doctor" },
93
94
  { "name" => "hook", "summary" => "list and run registered hooks: 'hook list', 'hook run NAME'" },
95
+ { "name" => "jobs" },
94
96
  { "name" => "pulse" },
95
97
  { "name" => "capabilities" },
96
98
  ].freeze
@@ -172,11 +174,11 @@ module Textus
172
174
  "accept #{queue}.KEY — promotes the proposal into its target zone",
173
175
  ],
174
176
  },
175
- "reconcile" => {
177
+ "drain" => {
176
178
  "purpose" => "keep the machine-maintained lanes fresh — re-pull stale intake entries from their declared source",
177
179
  "steps" => [
178
180
  "pulse — its `stale` list names entries past their ttl",
179
- "reconcile (zone: #{feeds}) — re-pull the stale entries",
181
+ "drain (zone: #{feeds}) — re-pull the stale entries",
180
182
  ],
181
183
  },
182
184
  }
@@ -132,7 +132,7 @@ module Textus
132
132
  # affordance; the agent surface deliberately returns nil)
133
133
  # put — reads the entry JSON from --stdin (ADR 0089: just stores bytes,
134
134
  # no --fetch transform)
135
- # (build removed in ADR 0087: materialization is system-pushed via reconcile)
135
+ # (build removed in ADR 0087: materialization is system-pushed via drain/serve)
136
136
  BEHAVIORAL_HATCHES = %i[get put].freeze
137
137
 
138
138
  # Contract verbs whose CLI is a plain `< Verb` command, not a projection at
@@ -144,7 +144,7 @@ module Textus
144
144
  # the exit_code: res["ok"] ? 0 : 1 behavior — two things the generic
145
145
  # projection cannot yet express; kept in ADR 0101 pending a future pass.)
146
146
  # (fetch/fetch_all were removed in ADR 0079: Produce::Acquire::Intake is now internal,
147
- # driven by the reconcile sweep and hook run — ADR 0089 removed the
147
+ # driven by the converge sweep (drain/serve) and hook run — ADR 0089 removed the
148
148
  # read-through that once also drove it.)
149
149
  NON_PROJECTED_CLI = %i[doctor].freeze
150
150
 
@@ -14,7 +14,7 @@ module Textus
14
14
  role = resolved_role(store)
15
15
 
16
16
  # put only stores the stdin JSON (ADR 0089): no transform-on-write.
17
- # Ingest (running a handler over bytes) is system-pushed via reconcile
17
+ # Ingest (running a handler over bytes) is system-pushed via drain/serve
18
18
  # and hook run, never a put flag.
19
19
  payload = JSON.parse(@stdin.read)
20
20
 
@@ -0,0 +1,19 @@
1
+ module Textus
2
+ class CLI
3
+ class Verb
4
+ # Launches the convergence daemon in the current process. Blocks forever;
5
+ # reclaims crashed leases and drains the queue each tick (Phase 3 adds
6
+ # scheduled TTL re-pull/sweep). CLI-only — agents enqueue work, they do not
7
+ # run daemons. Acts as the automation role (the build authority).
8
+ class Serve < Verb
9
+ command_name "serve"
10
+
11
+ def call(store)
12
+ call = Textus::Call.build(role: Textus::Role::AUTOMATION)
13
+ Textus::Maintenance::Serve.new(container: store.container, call: call).run
14
+ 0
15
+ end
16
+ end
17
+ end
18
+ end
19
+ end
@@ -11,6 +11,7 @@ module Textus
11
11
  key_mv: Textus::Write::KeyMv,
12
12
  accept: Textus::Write::Accept,
13
13
  reject: Textus::Write::Reject,
14
+ enqueue: Textus::Write::Enqueue,
14
15
  # Read
15
16
  get: Textus::Read::Get,
16
17
  list: Textus::Read::List,
@@ -30,12 +31,13 @@ module Textus
30
31
  doctor: Textus::Read::Doctor,
31
32
  boot: Textus::Read::Boot,
32
33
  capabilities: Textus::Read::Capabilities,
34
+ jobs: Textus::Read::Jobs,
33
35
 
34
36
  # Maintenance
35
37
  zone_mv: Textus::Maintenance::ZoneMv,
36
38
  key_mv_prefix: Textus::Maintenance::KeyMvPrefix,
37
39
  key_delete_prefix: Textus::Maintenance::KeyDeletePrefix,
38
- reconcile: Textus::Maintenance::Reconcile,
40
+ drain: Textus::Maintenance::Drain,
39
41
  rule_lint: Textus::Maintenance::RuleLint,
40
42
  }.freeze
41
43
 
@@ -19,7 +19,7 @@ module Textus
19
19
  "level" => "warning",
20
20
  "subject" => row["key"],
21
21
  "message" => row["reason"],
22
- "fix" => "rematerialize the entry: `textus reconcile`",
22
+ "fix" => "rematerialize the entry: `textus drain`",
23
23
  }
24
24
  end
25
25
  end
@@ -31,7 +31,7 @@ module Textus
31
31
  "level" => "warning",
32
32
  "subject" => sentinel_path,
33
33
  "message" => "sentinel is not valid JSON",
34
- "fix" => "delete #{sentinel_path} and re-run 'textus reconcile' to regenerate",
34
+ "fix" => "delete #{sentinel_path} and re-run 'textus drain' to regenerate",
35
35
  }
36
36
  end
37
37
 
@@ -51,7 +51,7 @@ module Textus
51
51
  "level" => "warning",
52
52
  "subject" => sentinel.target,
53
53
  "message" => "published file at #{sentinel.target} was modified out-of-band",
54
- "fix" => "re-run 'textus reconcile' to overwrite, or copy the manual edit back into the store source",
54
+ "fix" => "re-run 'textus drain' to overwrite, or copy the manual edit back into the store source",
55
55
  }
56
56
  end
57
57
  end
@@ -10,7 +10,7 @@ module Textus
10
10
  # (skipped — a cadence-less handler is not auto-re-pulled).
11
11
  # - derived/external -> DRIFT signal: a source changed since generated.at
12
12
  # (surfaced by the doctor generator_drift check; derived entries annotate
13
- # fresh at read time because reconcile converges them reactively).
13
+ # fresh at read time because converge runs them reactively).
14
14
  # Replaces Domain::IntakeStaleness and Domain::Staleness::GeneratorCheck and
15
15
  # the inline copies in Read::Get / Read::Freshness.
16
16
  class Evaluator
@@ -32,7 +32,7 @@ module Textus
32
32
  Verdict.build(stale: stale, reason: stale ? "ttl exceeded" : nil, fetching: false)
33
33
  end
34
34
 
35
- # Keys of intake entries past their source.ttl — the reconcile produce
35
+ # Keys of intake entries past their source.ttl — the converge produce
36
36
  # scope (replaces Domain::IntakeStaleness#call). A ttl-less intake entry
37
37
  # is :no_policy and skipped; a never-recorded one (with a ttl) is stale.
38
38
  def stale_intake_keys(prefix: nil, zone: nil)
@@ -0,0 +1,58 @@
1
+ require "digest"
2
+ require "json"
3
+
4
+ module Textus
5
+ module Domain
6
+ module Jobs
7
+ # A unit of deferred work. Pure data. The id is `<type>:<digest>` where the
8
+ # digest is over the args with sorted keys, so logically-identical enqueues
9
+ # collide on the same id — which is how the Queue dedups (the file already
10
+ # exists). At-least-once delivery means handlers must be idempotent.
11
+ class Job
12
+ DIGEST_BYTES = 16
13
+
14
+ attr_reader :type, :args, :enqueued_by, :max_attempts
15
+ attr_accessor :attempts, :last_error
16
+
17
+ def initialize(type:, args:, enqueued_by: nil, attempts: 0, max_attempts: 3, last_error: nil)
18
+ @type = type.to_s
19
+ @args = stringify(args)
20
+ @enqueued_by = enqueued_by
21
+ @attempts = attempts
22
+ @max_attempts = max_attempts
23
+ @last_error = last_error
24
+ end
25
+
26
+ def id
27
+ "#{@type}:#{digest}"
28
+ end
29
+
30
+ def to_h
31
+ {
32
+ "type" => @type, "args" => @args, "enqueued_by" => @enqueued_by,
33
+ "attempts" => @attempts, "max_attempts" => @max_attempts, "last_error" => @last_error
34
+ }
35
+ end
36
+
37
+ def self.from_h(hash)
38
+ new(
39
+ type: hash["type"], args: hash["args"] || {}, enqueued_by: hash["enqueued_by"],
40
+ attempts: hash["attempts"] || 0, max_attempts: hash["max_attempts"] || 3,
41
+ last_error: hash["last_error"]
42
+ )
43
+ end
44
+
45
+ private
46
+
47
+ def digest
48
+ canonical = JSON.dump(@args.sort.to_h)
49
+ Digest::SHA256.hexdigest(canonical)[0, DIGEST_BYTES]
50
+ end
51
+
52
+ def stringify(hash)
53
+ hash.transform_keys(&:to_s)
54
+ end
55
+ end
56
+ end
57
+ end
58
+ end
@@ -0,0 +1,37 @@
1
+ module Textus
2
+ module Domain
3
+ module Jobs
4
+ # Closed allow-list of runnable job types. The general `enqueue` surface
5
+ # (a later phase) can only push types registered here — that is the safety
6
+ # boundary that stops the "general runner" from running arbitrary code.
7
+ class Registry
8
+ Entry = Struct.new(:handler, :max_attempts, :required_role, keyword_init: true)
9
+
10
+ def initialize
11
+ @entries = {}
12
+ end
13
+
14
+ # required_role: a role the caller must hold to enqueue this type via the
15
+ # general `enqueue` surface (nil = any caller). The closed allow-list is
16
+ # the primary safety boundary; this is defence-in-depth for destructive
17
+ # types.
18
+ def register(type, handler:, max_attempts: 3, required_role: nil)
19
+ @entries[type.to_s] = Entry.new(handler: handler, max_attempts: max_attempts, required_role: required_role)
20
+ end
21
+
22
+ def registered?(type)
23
+ @entries.key?(type.to_s)
24
+ end
25
+
26
+ def lookup(type)
27
+ @entries.fetch(type.to_s) do
28
+ raise Textus::UsageError.new(
29
+ "unregistered job type '#{type}'",
30
+ hint: "register the type in Domain::Jobs::Registry before enqueueing it",
31
+ )
32
+ end
33
+ end
34
+ end
35
+ end
36
+ end
37
+ end
@@ -15,7 +15,7 @@ module Textus
15
15
  key_mv: %w[zone_writable_by],
16
16
  accept: %w[author_held target_is_canon],
17
17
  reject: %w[author_held],
18
- reconcile: %w[zone_writable_by],
18
+ converge: %w[zone_writable_by],
19
19
  }.freeze
20
20
 
21
21
  def self.for(transition) = BASE.fetch(transition, [])
@@ -3,7 +3,7 @@ module Textus
3
3
  module Policy
4
4
  # Garbage collection (ADR 0093). A glob-matched rule slot: when an entry
5
5
  # ages past `ttl`, retire it. Destructive only — runs on the full
6
- # `reconcile` pass, never on a write (ADR 0079's invariant). Orthogonal to
6
+ # `converge` pass, never on a write (ADR 0079's invariant). Orthogonal to
7
7
  # production (`source:`): an intake entry can re-pull hourly AND archive
8
8
  # after 90 days. `warn`/`refresh` are gone (refresh is implied by an
9
9
  # intake source; warn never fired after ADR 0089's pure-read get).
@@ -7,11 +7,11 @@ module Textus
7
7
  # from: project -> derived (internal projection; observable -> rdeps staleness)
8
8
  # from: handler -> intake (external fetch; unobservable -> ttl staleness)
9
9
  # from: command -> external (out-of-band runner; staleness only, textus never runs it)
10
- # `on_write` (sync|async, default async) is the write-trigger strategy for
11
- # observable (project) sources; meaningless for intake/command.
10
+ # Materialization is async-only (job-queue model): a write enqueues a
11
+ # `materialize` job, converged by a worker. There is no per-entry write
12
+ # trigger knob.
12
13
  class Source
13
- FROMS = %w[project handler command].freeze
14
- STRATEGIES = %w[async sync].freeze
14
+ FROMS = %w[project handler command].freeze
15
15
 
16
16
  attr_reader :from, :handler, :config, :command, :sources
17
17
 
@@ -21,11 +21,6 @@ module Textus
21
21
  raise Textus::BadManifest.new("source.from must be one of #{FROMS.join("|")}, got #{raw["from"].inspect}")
22
22
  end
23
23
 
24
- @on_write = (raw["on_write"] || "async").to_s
25
- unless STRATEGIES.include?(@on_write)
26
- raise Textus::BadManifest.new("source.on_write must be one of #{STRATEGIES.join("/")}, got #{@on_write.inspect}")
27
- end
28
-
29
24
  @ttl = raw["ttl"]
30
25
  @projection = {}
31
26
 
@@ -39,7 +34,6 @@ module Textus
39
34
  def kind = @from == "handler" ? :intake : :derived
40
35
  def external? = @from == "command"
41
36
  def projection? = @from == "project"
42
- def sync? = @on_write == "sync"
43
37
  def ttl_seconds = @ttl.nil? ? nil : Textus::Domain::Duration.seconds(@ttl)
44
38
 
45
39
  # Flattened projection accessors (ADR 0094) — read directly off the source
data/lib/textus/errors.rb CHANGED
@@ -166,9 +166,9 @@ module Textus
166
166
  def initialize(m, format: nil)
167
167
  hint =
168
168
  if format
169
- "the template rendered invalid #{format}; try rendering with mock data and parsing the output before re-running reconcile"
169
+ "the template rendered invalid #{format}; try rendering with mock data and parsing the output before re-running drain"
170
170
  else
171
- "the template rendered invalid content; try rendering with mock data and parsing the output before re-running reconcile"
171
+ "the template rendered invalid content; try rendering with mock data and parsing the output before re-running drain"
172
172
  end
173
173
  super("bad_render", m, hint: hint)
174
174
  end
@@ -17,7 +17,6 @@ module Textus
17
17
  entry_renamed: %i[ctx key from_key to_key envelope],
18
18
  entry_produced: %i[ctx key envelope sources],
19
19
  produce_failed: %i[ctx keys error],
20
- reconcile_failed: %i[ctx failed],
21
20
  proposal_accepted: %i[ctx key target_key],
22
21
  proposal_rejected: %i[ctx key target_key],
23
22
  entry_published: %i[ctx key envelope source target],
@@ -1,7 +1,7 @@
1
1
  # .textus/hooks/machine_intake.rb
2
2
  # Scaffolded by `textus init` — CUSTOMIZE FREELY, or delete the feeds.machines
3
3
  # entry from manifest.yaml if you don't want it.
4
- # Feeds a per-host SNAPSHOT into feeds.machines.<host> on `textus reconcile`
4
+ # Feeds a per-host SNAPSHOT into feeds.machines.<host> on `textus drain`
5
5
  # (never on the per-turn boot/pulse path). It is NESTED so it grows to a fleet: the
6
6
  # `local` leaf scans THIS host; add ssh hosts with the cookbook recipe
7
7
  # (docs/cookbook/environment-scan.md). tracked:false → gitignored. Keep this an
data/lib/textus/init.rb CHANGED
@@ -10,7 +10,7 @@ module Textus
10
10
  roles:
11
11
  - { name: human, can: [author, propose] }
12
12
  - { name: agent, can: [propose, keep] }
13
- - { name: automation, can: [reconcile] }
13
+ - { name: automation, can: [converge] }
14
14
  zones:
15
15
  - { name: knowledge, kind: canon, desc: "the maintained source of truth (identity.* lives here)" }
16
16
  - { name: notebook, kind: workspace, owner: agent, desc: "the agent's own durable working notes" }
@@ -21,7 +21,7 @@ module Textus
21
21
  - { key: knowledge.notes, path: knowledge/notes, zone: knowledge, schema: null, owner: human:self, nested: true, kind: nested }
22
22
  - { key: notebook.notes, path: notebook/notes, zone: notebook, schema: null, owner: agent:self, nested: true, kind: nested }
23
23
  - { key: proposals.notes, path: proposals/notes, zone: proposals, schema: null, owner: agent:self, nested: true, kind: nested }
24
- # A per-host snapshot, refreshed from its declared intake by `textus reconcile` (scheduled, or on demand).
24
+ # A per-host snapshot, refreshed from its declared intake by `textus drain` (scheduled, or on demand).
25
25
  # Nested so it grows to a fleet — add artifacts.feeds.machines.<host> leaves over SSH
26
26
  # (see docs/cookbook/environment-scan.md) without renaming. tracked:false →
27
27
  # gitignored (machine info can be sensitive/noisy) but still protocol-readable
@@ -95,7 +95,7 @@ module Textus
95
95
 
96
96
  Events: :resolve_handler, :transform_rows, :validate (rpc — return value used)
97
97
  :entry_written, :entry_deleted, :entry_fetched, :entry_renamed,
98
- :entry_produced, :produce_failed, :reconcile_failed,
98
+ :entry_produced, :produce_failed,
99
99
  :proposal_accepted, :proposal_rejected,
100
100
  :entry_published, :store_loaded, :session_opened,
101
101
  :entry_fetch_started, :entry_fetch_failed (pub-sub — return discarded)
@@ -105,7 +105,7 @@ module Textus
105
105
 
106
106
  AGENT_ENTRIES = <<~YAML.gsub(/^/, " ")
107
107
  # --with-agent profile: project facts + runbooks feed the orientation
108
- # projection below, which `textus reconcile` renders to CLAUDE.md/AGENTS.md.
108
+ # projection below, which `textus drain` renders to CLAUDE.md/AGENTS.md.
109
109
  - { key: knowledge.project, path: knowledge/project.md, zone: knowledge, schema: project, owner: human:self, kind: leaf }
110
110
  - { key: knowledge.runbooks, path: knowledge/runbooks, zone: knowledge, schema: runbook, owner: human:self, nested: true, kind: nested }
111
111
  - key: artifacts.derived.orientation
@@ -0,0 +1,62 @@
1
+ module Textus
2
+ module Jobs
3
+ # Wires the closed allow-list of convergence job types to the existing
4
+ # convergence code. Authority is read from the job's frozen `enqueued_by`
5
+ # and turned into the Call the handler runs under: produce self-elevates
6
+ # inside Produce::Engine regardless; destructive sweep runs AS the caller.
7
+ module Handlers
8
+ module_function
9
+
10
+ def registry
11
+ reg = Textus::Domain::Jobs::Registry.new
12
+ # produce is pure (self-elevates) — any caller may request a rematerialize.
13
+ reg.register("materialize", handler: method(:produce))
14
+ reg.register("re-pull", handler: method(:produce))
15
+ # sweep is destructive — gate ad-hoc enqueue to the automation authority.
16
+ reg.register("sweep", handler: method(:sweep), required_role: Textus::Role::AUTOMATION)
17
+ reg
18
+ end
19
+
20
+ # produce: render derived / re-pull intake for a single key. Engine
21
+ # self-elevates to the build actor internally; the passed call carries
22
+ # only correlation/dry_run plus the stamped role for audit. Engine#call
23
+ # isolates per-key produce errors into its result hash rather than raising,
24
+ # so surface them as :produce_failed events (the converge result hash used
25
+ # to carry them; the worker drops the return, so re-publish here).
26
+ def produce(job:, container:)
27
+ call = call_for(job)
28
+ result = Textus::Produce::Engine.converge(container: container, call: call, keys: [job.args["key"]])
29
+ return unless result.is_a?(Hash)
30
+
31
+ Array(result[:failed]).each do |failure|
32
+ container.events.publish(
33
+ :produce_failed,
34
+ ctx: Textus::Hooks::Context.for(container: container, call: call),
35
+ keys: [failure["key"]], error: failure["error"]
36
+ )
37
+ end
38
+ end
39
+
40
+ # sweep: compute retention rows for the scope, then apply destructively AS
41
+ # the job's role (no self-elevation).
42
+ def sweep(job:, container:)
43
+ call = call_for(job)
44
+ scope = job.args["scope"]
45
+ rows = Textus::Domain::Retention::Sweep.new(
46
+ manifest: container.manifest,
47
+ file_stat: Textus::Ports::Storage::FileStat.new,
48
+ clock: Textus::Ports::Clock.new,
49
+ ).call(prefix: scope_prefix(scope), zone: scope_zone(scope))
50
+ Textus::Maintenance::Retention::Apply.new(container: container, call: call).call(rows)
51
+ end
52
+
53
+ def call_for(job)
54
+ Textus::Call.build(role: job.enqueued_by || Textus::Role::AUTOMATION)
55
+ end
56
+
57
+ # A scope is `{ "prefix" => ..., "zone" => ... }` or nil (whole store).
58
+ def scope_prefix(scope) = scope.is_a?(Hash) ? scope["prefix"] : nil
59
+ def scope_zone(scope) = scope.is_a?(Hash) ? scope["zone"] : nil
60
+ end
61
+ end
62
+ end
@@ -0,0 +1,36 @@
1
+ module Textus
2
+ module Jobs
3
+ # Time-based seeding for the daemon: at each tick, enqueue a re-pull job for
4
+ # every intake key past its source.ttl and a sweep job to GC entries past
5
+ # retention.ttl. Dedup means a job already queued from a prior tick is a
6
+ # no-op. Both are stamped automation (the daemon's own authority); the sweep
7
+ # handler runs retention as that role.
8
+ class Scheduler
9
+ def initialize(container:, queue:)
10
+ @container = container
11
+ @queue = queue
12
+ end
13
+
14
+ def run_once
15
+ stale_intake.each do |key|
16
+ @queue.enqueue(job("re-pull", { "key" => key }))
17
+ end
18
+ @queue.enqueue(job("sweep", { "scope" => { "prefix" => nil, "zone" => nil } }))
19
+ end
20
+
21
+ private
22
+
23
+ def stale_intake
24
+ Textus::Domain::Freshness::Evaluator.new(
25
+ manifest: @container.manifest,
26
+ file_stat: Textus::Ports::Storage::FileStat.new,
27
+ clock: Textus::Ports::Clock.new,
28
+ ).stale_intake_keys(prefix: nil, zone: nil)
29
+ end
30
+
31
+ def job(type, args)
32
+ Textus::Domain::Jobs::Job.new(type: type, args: args, enqueued_by: Textus::Role::AUTOMATION)
33
+ end
34
+ end
35
+ end
36
+ end
@@ -0,0 +1,57 @@
1
+ module Textus
2
+ module Jobs
3
+ # Enqueues the full convergence set for a scope: a produce job per derived /
4
+ # publish_tree / publish.to entry, a re-pull job per stale intake key, and a
5
+ # single sweep job for the scope. The scope logic mirrors
6
+ # the converge scope (Produce::Engine) so `drain` and `serve` converge identically.
7
+ # Produce jobs self-elevate (stamped automation); the sweep job carries the
8
+ # caller's role (destructive runs as caller).
9
+ class Seeder
10
+ def initialize(container:, queue:, call:)
11
+ @container = container
12
+ @queue = queue
13
+ @call = call
14
+ @manifest = container.manifest
15
+ end
16
+
17
+ def seed(prefix:, zone:)
18
+ file_stat = Textus::Ports::Storage::FileStat.new
19
+
20
+ producible_keys(prefix, zone).each do |key|
21
+ @queue.enqueue(job("materialize", { "key" => key }, Textus::Role::AUTOMATION))
22
+ end
23
+ stale_intake_keys(prefix, zone, file_stat).each do |key|
24
+ @queue.enqueue(job("re-pull", { "key" => key }, Textus::Role::AUTOMATION))
25
+ end
26
+ @queue.enqueue(job("sweep", { "scope" => { "prefix" => prefix, "zone" => zone } }, @call.role))
27
+ end
28
+
29
+ private
30
+
31
+ def job(type, args, role)
32
+ Textus::Domain::Jobs::Job.new(type: type, args: args, enqueued_by: role)
33
+ end
34
+
35
+ # Mirrors the converge scope (the publishable arm).
36
+ def producible_keys(prefix, zone)
37
+ @manifest.data.entries
38
+ .select { |e| e.derived? || !e.publish_tree.nil? || !e.publish_to.empty? }
39
+ .select { |e| in_scope?(e, prefix, zone) }
40
+ .map(&:key)
41
+ end
42
+
43
+ def stale_intake_keys(prefix, zone, file_stat)
44
+ Textus::Domain::Freshness::Evaluator.new(
45
+ manifest: @manifest, file_stat: file_stat, clock: Textus::Ports::Clock.new,
46
+ ).stale_intake_keys(prefix: prefix, zone: zone)
47
+ end
48
+
49
+ def in_scope?(entry, prefix, zone)
50
+ return false if zone && entry.zone != zone
51
+ return false if prefix && !entry.key.start_with?(prefix)
52
+
53
+ true
54
+ end
55
+ end
56
+ end
57
+ end
data/lib/textus/layout.rb CHANGED
@@ -25,6 +25,14 @@ module Textus
25
25
  File.join(run(root), "build.lock")
26
26
  end
27
27
 
28
+ def self.queue(root)
29
+ File.join(run(root), "queue")
30
+ end
31
+
32
+ def self.queue_state(root, state)
33
+ File.join(queue(root), state.to_s)
34
+ end
35
+
28
36
  def self.audit_dir(root)
29
37
  File.join(run(root), "audit")
30
38
  end