ytdb-slate 0.3.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -70,6 +70,7 @@ Optional config file: `slate.json` in the project's pi config dir (`.pi/slate.js
70
70
  | `orchestratorModeDefault` | boolean | `false` | Start fresh interactive sessions with orchestrator mode ON. |
71
71
  | `episodeModel` | string | newest available Anthropic Sonnet, else the worker's own model | Model (`provider/id`) used to compress a finished action into an episode. |
72
72
  | `workerTools` | string[] | `["read", "bash", "edit", "write", "grep", "find", "ls"]` | Tools available to worker threads (an empty list also falls back to the default). |
73
+ | `workerExtensions` | string[] | `[]` | Regex patterns (matched **unanchored**) selecting which of the host session's already-loaded extensions also load into every worker thread; each matched extension's tools are added **on top of** `workerTools`. Empty (default) means workers load no extensions. The orchestrator keeps its restricted tool set but its doctrine is told what was whitelisted. Invalid patterns are dropped with a warning at session start. |
73
74
  | `maxConcurrent` | number | `4` | Maximum number of worker actions running concurrently (must be ≥ 1 — unenforced: a value of 0 or less silently hangs all dispatches). Excess dispatches wait in a queue; actions on the same thread always run in dispatch order. Default rationale: shipped `docs/design-principles.md` §5 (repo-local note). |
74
75
  | `contextBudget` | number \| object | `256000` (Anthropic models: `400000`) | Absolute orchestrator context budget (tokens) at which Slate auto-pauses and prepares a fresh-session handoff — semantics, defaults, per-model overrides, and rationale in [`docs/context-budget.md`](docs/context-budget.md). |
75
76
  | `orchestratorPromptDocs` | string[] | `[]` | Project markdown files (paths relative to the project root) whose **contents** are appended to the orchestrator system prompt. |
@@ -98,6 +99,27 @@ Example `.pi/slate.json` (the `docs/agents/...` paths are placeholders — point
98
99
 
99
100
  > **Silent skip:** the project-file keys fail silently — no error is shown. For the content-injected keys (`orchestratorPromptDocs`, `workerPromptDocs`, `doctrineExtraPath`) a missing, unreadable, or empty file is skipped and nothing is injected. For `reviewPerspectivesPath` the pointer is omitted only when the file is missing — the file is not read at injection time, so an unreadable or empty file is still cited. Verify your paths after copying the example.
100
101
 
102
+ **Worker extensions (`workerExtensions`).** By default worker threads load no extensions. This key is a list of regex patterns that select extensions the **host session has already loaded** and load them into every worker too. Each pattern is matched **unanchored** (unlike `contextBudget.overrides`, which is anchored) against a load unit's recorded source spec (e.g. `npm:pi-web-search@1.3.1`), its load-unit path, or the entry path of any tool that unit contributes, so a bare package name matches:
103
+
104
+ ```json
105
+ {
106
+ "workerExtensions": ["pi-smart-fetch", "pi-web-search"]
107
+ }
108
+ ```
109
+
110
+ Every worker then gets the fetch and web-search tools **on top of** `workerTools` (and on top of the per-dispatch `tools` argument of the `thread` tool — those two govern the built-in tools only). The orchestrator itself does **not** gain these tools — orchestrator mode keeps its restricted set — but its doctrine gains a rule naming each whitelisted extension and its tools, so it knows what it can delegate. pi's discovery, project-trust gating, and dedup remain the only ingress: a worker can never load an extension the host is not running, an extension that registers no tools cannot be whitelisted, and a host started with extensions disabled offers nothing to whitelist.
111
+
112
+ **What to know before whitelisting** — it reaches past Slate's isolation, so it is an operator decision:
113
+
114
+ - **Delegation is unbounded.** Slate guarantees only that no worker obtains Slate's own `thread`/`threads`/`episode` tools. A whitelisted extension that ships its own sub-agent or delegation tool under any other name gives workers delegation Slate can neither detect, bound, nor account for.
115
+ - **Credential and filesystem reach.** Inside a worker the extension has the same filesystem and credential access it has in the host; Slate's read-only settings snapshot blocks pi-settings writes and nothing else.
116
+ - **Unsynced model-scoped tools.** Worker sessions never fire the session-start event, so an extension whose tool set is model-scoped stays unsynced and may offer a tool the worker's model cannot serve (e.g. pi-web-search's `url_context` on a non-Google worker model) — the call simply fails and the episode records it.
117
+ - **Abort.** A third-party extension may ignore the abort signal, so its network activity can outlive an abort or a context-budget pause.
118
+ - **Cost.** Provider-native tool billing can escape Slate's worker cost accounting.
119
+ - **Pathological patterns.** The patterns are regexes from your own trusted config — the same file that already steers models, prompts, and tool lists — matched with no time bound while the extension set is resolved once per session. A pattern with catastrophic backtracking can stall that resolution; treat it as a footgun to avoid, not a privilege boundary.
120
+
121
+ The load-time recursion guard behind this — and the risks it does and does not cover — is in [`docs/design-principles.md`](docs/design-principles.md).
122
+
101
123
  ## Trust
102
124
 
103
125
  Slate reads project configuration (`.pi/slate.json`) and injects project files (`orchestratorPromptDocs`, `workerPromptDocs`, `doctrineExtraPath`, `reviewPerspectivesPath`) **only in trusted projects**. In untrusted projects Slate runs with built-in defaults and injects nothing from the working tree.
@@ -125,8 +125,11 @@ or two problems for the others:
125
125
  not blind retry.
126
126
  - **P6 — Make desired behavior the natural behavior.** In orchestrator
127
127
  mode, tactical tools are removed, so delegation is the only way to act.
128
- - **P7 — Guard against over-decomposition.** Workers cannot spawn threads
129
- (depth-1); recursion is structurally impossible.
128
+ - **P7 — Guard against over-decomposition.** Workers never receive
129
+ Slate's dispatch tools (`thread`/`threads`/`episode`), so no worker can
130
+ spawn a Slate thread — Slate's own recursion stays depth-1. That is a
131
+ narrow load-time invariant, not a general delegation bound (see the
132
+ recursion-guard note in §5).
130
133
  - **P8 — Per-episode feedback beats blind N-step execution.** Bounded
131
134
  actions return before the next decision, so course correction is always
132
135
  possible.
@@ -146,7 +149,7 @@ or two problems for the others:
146
149
  | P4 context by reference | `tools.ts` `context` parameter injects prior episodes by id |
147
150
  | P5 adaptive decomposition | doctrine rule 6 in `mode.ts`; no plan structure imposed anywhere |
148
151
  | P6 natural behavior | `mode.ts` `ORCHESTRATOR_TOOLS` restriction (read-only + slate tools) |
149
- | P7 over-decomposition guard | `worker.ts` recursion guard — workers load no extensions, so the `thread` tool never exists for them |
152
+ | P7 over-decomposition guard | `worker.ts` recursion guard — load-scoped barriers keep Slate's `thread`/`threads`/`episode` tools out of every worker (recursion-guard note below) |
150
153
  | P8 per-episode feedback | `threads.ts` synchronous dispatch; episode returned to orchestrator |
151
154
  | P9 parallelism | `threads.ts` `maxConcurrent` queueing; doctrine rule 2 in `mode.ts` |
152
155
  | P10 context as RAM | `handoff.ts` context-budget auto-pause + fresh-session handoff |
@@ -164,6 +167,97 @@ while staying safe on common consumer API tiers; wider fan-outs only
164
167
  pay tail latency, and projects with higher-tier keys raise the cap in
165
168
  `slate.json`.
166
169
 
170
+ Repo-local note (not from the report): the P7 guard was originally
171
+ absolute — workers loaded no extensions, so Slate's `thread` tool simply
172
+ never existed for them. The optional `workerExtensions` key (`slate.json`,
173
+ trusted projects only — see the README config reference) relaxes that: it
174
+ whitelists extensions the host session has ALREADY loaded and loads them
175
+ into every worker. The guard therefore no longer rests on "workers load
176
+ nothing" but on a narrower, more precise invariant: no worker can ever
177
+ obtain SLATE's dispatch tools (`thread`/`threads`/`episode`).
178
+
179
+ Three load-scoped barriers buy that invariant, all applied per LOAD UNIT
180
+ — the owning package directory when the extension is package-originated
181
+ and the package's declared entries provably match what the host loaded
182
+ (BG20, below), otherwise the extension's own entry file:
183
+
184
+ 1. Only whitelisted units load, through pi's resource loader in allowlist
185
+ mode; a worker's extension set is exactly the allowlist and no
186
+ non-whitelisted module is ever imported (load errors surface as
187
+ warnings). Candidates are only what the host session already loaded,
188
+ enumerated from its tool registry — so an extension registering no
189
+ tools is invisible, and a host started with extensions disabled yields
190
+ an empty candidate set.
191
+ 2. Any unit that contains Slate's own package root is dropped whatever the
192
+ patterns say.
193
+ 3. Any unit whose tools collide with Slate's `thread`/`threads`/`episode`
194
+ or with pi's built-ins (`read`/`bash`/`edit`/`write`/`grep`/`find`/`ls`)
195
+ is dropped WHOLE — pi's registry lets an extension tool overwrite a
196
+ same-named built-in, so a partial load could shadow the very tools the
197
+ guard protects.
198
+
199
+ Exclusion has to happen at load time, at unit granularity, rather than by
200
+ loading everything and filtering the tool list afterwards: a
201
+ worker-loaded extension is not tools-only. Its event handlers, tools, and
202
+ flags all go live on load, so once a unit has loaded its side effects have
203
+ already run and dropping its tools afterwards would leave its handlers and
204
+ flags active. Dropping the whole unit before it loads is the only clean
205
+ cut — which is also why barrier 3 rejects a colliding unit entirely
206
+ instead of suppressing just the offending tool. (A package directory is
207
+ handed to pi whole — letting pi's own manifest resolution expand it, so
208
+ no-tool companion entries load alongside the tool-bearing ones — only when
209
+ the manifest's declared entries are ALL literal relative paths AND the
210
+ host loaded every one of them. A glob or override-form entry, or a host
211
+ running only a filtered subset, fails that equivalence check and drops the
212
+ unit to exactly the entry files the host loaded; that fallback loses the
213
+ package's no-tool companion entries, since an entry that registered no
214
+ tool cannot be shown to be running.)
215
+
216
+ That collision barrier is best-effort and load-time only: it runs before a
217
+ unit loads, against its tools as the HOST registry reports them, and once
218
+ more after the load, against the tools the units actually registered — a
219
+ post-load collision fails the dispatch CLOSED rather than opening the
220
+ worker. Neither pass can see a tool an extension registers only LATER, on
221
+ the worker's first turn (a `before_agent_start` handler, say). Slate's own
222
+ dispatch tools do not depend on that scan: `createAgentSession` is given
223
+ an `excludeTools` denylist of `thread`/`threads`/`episode` that pi applies
224
+ AFTER the tool allowlist and re-applies on every tool-registry refresh, so
225
+ they can never be active in a worker no matter when or by whom they are
226
+ registered — a deferred registration included. A deferred registration
227
+ that shadows a pi BUILT-IN (`read`, `bash`, …) has no such gap-closer —
228
+ the worker needs the real built-in — so it stays outside Slate's control,
229
+ exactly as available to any extension the host session runs. The guard
230
+ bounds Slate's dispatch tools, not every tool an extension might register
231
+ at runtime.
232
+
233
+ The orchestrator does NOT get the whitelisted tools; orchestrator mode
234
+ keeps its restricted set. Instead its doctrine gains a rule listing each
235
+ whitelisted extension with its tool names and descriptions, so it knows
236
+ what it can delegate. That doctrine rule and the worker allowlists come
237
+ from a single memoized resolution per session, so the two cannot drift.
238
+
239
+ The invariant bought is exactly that: no worker gets Slate's dispatch
240
+ tools. It is NOT a general recursion or delegation bound, and the accepted
241
+ risks that follow are the operator's to weigh:
242
+
243
+ - A whitelisted extension that registers its OWN sub-agent or delegation
244
+ tool under any other name gives workers unbounded delegation that Slate
245
+ can neither detect nor bound, outside Slate's episode and cost
246
+ accounting. Whitelisting such an extension is a deliberate operator
247
+ decision.
248
+ - Inside a worker a whitelisted extension has the same filesystem and
249
+ credential reach it has in the host; Slate's read-only settings snapshot
250
+ blocks pi-settings writes and nothing else.
251
+ - Worker sessions never fire the session-start event, so an extension
252
+ whose model-scoped tool manager depends on it stays unsynced and may
253
+ offer a tool the worker's model cannot serve — the call simply fails and
254
+ the episode records it.
255
+ - Third-party extensions may ignore the abort signal, so their network
256
+ activity can outlive an abort or a context-budget pause.
257
+ - Host and worker copies of the same extension share module-level state
258
+ through pi's process-global module cache.
259
+ - Provider-native tool billing may escape Slate's worker cost accounting.
260
+
167
261
  ## 6. Runtime knowledge: what the orchestrator knows, and when
168
262
 
169
263
  The orchestrator's knowledge of these principles is two-tier, following
@@ -27,18 +27,11 @@ import {
27
27
  type ExtensionContext,
28
28
  } from "@earendil-works/pi-coding-agent";
29
29
  import { withGlobalModelDefaultRestored } from "./model-default.ts";
30
+ import { sanitizeForNotify } from "./notify.ts";
30
31
  import type { SlateConfig } from "./state.ts";
31
32
 
32
33
  type RegistryModel = NonNullable<ReturnType<ExtensionContext["modelRegistry"]["find"]>>;
33
34
 
34
- // Error messages reach ctx.ui.notify, and pi-tui renders control/ANSI codes
35
- // verbatim — strip control characters and cap length before display (same
36
- // pattern as handoff.ts's sanitizeForNotify).
37
- function sanitizeForNotify(s: string, max = 120): string {
38
- const clean = s.replace(/[\u0000-\u001f\u007f\u009b]/g, "");
39
- return clean.length > max ? `${clean.slice(0, max)}…` : clean;
40
- }
41
-
42
35
  // Console-first reporting, for FAILURES ONLY: the console line is
43
36
  // unconditional, so a failover that could not happen still surfaces in
44
37
  // headless runs and at teardown — the has-UI branch is exactly the one that
@@ -42,6 +42,7 @@ import {
42
42
  type ExtensionContext,
43
43
  } from "@earendil-works/pi-coding-agent";
44
44
  import { withGlobalModelDefaultRestored } from "./model-default.ts";
45
+ import { sanitizeForNotify } from "./notify.ts";
45
46
  import {
46
47
  orchestratorCostUsd,
47
48
  type ContextBudgetObject,
@@ -94,15 +95,6 @@ function pendingFile(cwd: string): string {
94
95
  return join(cwd, CONFIG_DIR_NAME, "slate", "pending-handoff.json");
95
96
  }
96
97
 
97
- // Strings from the pending file (and error messages derived from them) reach
98
- // ctx.ui.notify, and pi-tui renders control/ANSI codes verbatim — a seeded
99
- // pending file could inject terminal escapes. Strip control characters and
100
- // cap the length before display.
101
- function sanitizeForNotify(s: string, max = 120): string {
102
- const clean = s.replace(/[\u0000-\u001f\u007f\u009b]/g, "");
103
- return clean.length > max ? `${clean.slice(0, max)}…` : clean;
104
- }
105
-
106
98
  // Console-first reporting for the model-adoption block below, and for FAILURES
107
99
  // ONLY: the console line is unconditional, so the failure surfaces in headless
108
100
  // runs and at teardown too — the has-UI branch is exactly the one that vanishes
@@ -20,7 +20,8 @@
20
20
  * Optional config at <config dir>/slate.json (config dir = CONFIG_DIR_NAME,
21
21
  * ".pi" by default), honored ONLY when the project is trusted — untrusted
22
22
  * projects run on built-in defaults with no project file injection:
23
- * { "episodeModel": "provider/id", "workerTools": [...], "maxConcurrent": 4,
23
+ * { "episodeModel": "provider/id", "workerTools": [...],
24
+ * "workerExtensions": ["regex", ...], "maxConcurrent": 4,
24
25
  * "contextBudget": 256000, "orchestratorModeDefault": true,
25
26
  * "orchestratorPromptDocs": ["docs/orchestrator-guidelines.md"],
26
27
  * "workerPromptDocs": ["docs/thread-guidelines.md"],
@@ -46,6 +47,12 @@ import { registerSlateMode } from "./mode.ts";
46
47
  import { SlateStore, type SlateConfig } from "./state.ts";
47
48
  import { ThreadManager } from "./threads.ts";
48
49
  import { registerSlateTools } from "./tools.ts";
50
+ import {
51
+ createWorkerExtensionResolver,
52
+ EMPTY_WORKER_EXTENSION_SET,
53
+ sanitizeWorkerExtensions,
54
+ type WorkerExtensionSet,
55
+ } from "./worker-extensions.ts";
49
56
 
50
57
  function loadConfig(cwd: string): SlateConfig {
51
58
  const file = join(cwd, CONFIG_DIR_NAME, "slate.json");
@@ -68,7 +75,16 @@ function loadConfig(cwd: string): SlateConfig {
68
75
 
69
76
  export default function (pi: ExtensionAPI) {
70
77
  const store = new SlateStore(pi);
71
- let manager = new ThreadManager(store, {});
78
+ // One worker-extension resolver per session (AD41), reassigned every
79
+ // session_start below. The DOCTRINE consumer (registerSlateMode) reads it
80
+ // through the live `() => resolveWorkerExtensionSet()` indirection because it
81
+ // always belongs to the CURRENT session. ThreadManager instead binds the
82
+ // resolver instance BY VALUE at construction (CN20): a manager orphaned by a
83
+ // session swap must keep resolving against ITS OWN session's frozen set, not a
84
+ // later session's — the live indirection would leak the newer set into the
85
+ // stale manager. Starts as the empty set (feature off) until session_start.
86
+ let resolveWorkerExtensionSet: () => WorkerExtensionSet = () => EMPTY_WORKER_EXTENSION_SET;
87
+ let manager = new ThreadManager(store, {}, resolveWorkerExtensionSet);
72
88
 
73
89
  registerSlateTools(pi, store, () => manager);
74
90
 
@@ -78,15 +94,24 @@ export default function (pi: ExtensionAPI) {
78
94
  // it is honored only for trusted projects; untrusted → built-in defaults.
79
95
  const config = ctx.isProjectTrusted() ? loadConfig(ctx.cwd) : {};
80
96
  const warn = (msg: string) => (ctx.hasUI ? ctx.ui.notify(msg, "warning") : console.warn(msg));
81
- // modelFailover and contextBudget are validated eagerly: a malformed
82
- // value would otherwise fail silently mid-dispatch / exactly when the
83
- // auto-pause was supposed to save the orchestrator's context.
97
+ // modelFailover, contextBudget and workerExtensions are validated eagerly:
98
+ // a malformed value would otherwise fail silently mid-dispatch / exactly
99
+ // when the auto-pause was supposed to save the orchestrator's context.
84
100
  config.modelFailover = sanitizeModelFailover(config.modelFailover, warn);
85
101
  config.contextBudget = sanitizeContextBudget(config.contextBudget, warn);
102
+ config.workerExtensions = sanitizeWorkerExtensions(config.workerExtensions, warn);
86
103
  if (config.contextBudget !== undefined && config.pauseThresholdPercent !== undefined) {
87
104
  warn("slate: contextBudget is set — the deprecated pauseThresholdPercent is ignored");
88
105
  }
89
- manager = new ThreadManager(store, config);
106
+ // Fresh resolver AFTER sanitization (reads the cleaned patterns) and per
107
+ // session_start (a restart re-resolves). Same warn as the sanitizers so a
108
+ // withheld colliding unit surfaces the same way. First use is later, in a
109
+ // worker open or a doctrine build — after session_start finishes, so tools
110
+ // registered during session_start are captured (AD41).
111
+ resolveWorkerExtensionSet = createWorkerExtensionResolver(pi, () => config.workerExtensions ?? [], warn);
112
+ // Bound BY VALUE (CN20): this manager keeps this session's resolver even if a
113
+ // later session_start replaces the module variable above.
114
+ manager = new ThreadManager(store, config, resolveWorkerExtensionSet);
90
115
  store.restore(ctx);
91
116
  });
92
117
 
@@ -104,5 +129,5 @@ export default function (pi: ExtensionAPI) {
104
129
  // order-critical relative to the handlers above (different trigger events).
105
130
  registerOrchestratorFailover(pi, () => manager.getConfig());
106
131
 
107
- registerSlateMode(pi, store, handoff, () => manager.getConfig());
132
+ registerSlateMode(pi, store, handoff, () => manager.getConfig(), () => resolveWorkerExtensionSet());
108
133
  }
package/extension/mode.ts CHANGED
@@ -27,6 +27,7 @@ import {
27
27
  } from "./paths.ts";
28
28
  import { loadPromptDocs } from "./prompt-docs.ts";
29
29
  import { orchestratorCostUsd, type SlateConfig, type SlateStore } from "./state.ts";
30
+ import type { WorkerExtensionSet, WorkerExtensionUnit } from "./worker-extensions.ts";
30
31
 
31
32
  const ORCHESTRATOR_TOOLS = ["read", "grep", "find", "ls", "thread", "threads", "episode"];
32
33
 
@@ -40,7 +41,68 @@ const ORCHESTRATOR_TOOLS = ["read", "grep", "find", "ls", "thread", "threads", "
40
41
  * the file exists (cwd-resolved) — missing files are skipped silently like
41
42
  * every other project doc.
42
43
  */
43
- function buildDoctrine(cwd: string, config: SlateConfig, trusted: boolean): string {
44
+ // A source spec that names WHICH extension (e.g. "npm:foo") is worth showing;
45
+ // the generic scope-ish sources are not, so rule 11 falls back to the unit path.
46
+ const UNINFORMATIVE_SOURCES = new Set(["", "auto", "local", "cli", "project", "user", "temporary", "builtin", "sdk"]);
47
+
48
+ function unitLabel(unit: WorkerExtensionUnit): string {
49
+ return UNINFORMATIVE_SOURCES.has(unit.source) ? unit.path : unit.source;
50
+ }
51
+
52
+ // Per-field caps for the doctrine interpolations (WB21): a tool name is an
53
+ // identifier, a unit label is a package spec or path, a description is a
54
+ // sentence. Caps keep a pathological 2000-char value from bloating every turn.
55
+ const DOCTRINE_NAME_MAX = 64;
56
+ const DOCTRINE_LABEL_MAX = 128;
57
+ const DOCTRINE_DESC_MAX = 140;
58
+
59
+ // ONE sanitizer for every extension-supplied string interpolated into the
60
+ // orchestrator doctrine — tool names, unit labels and descriptions (WB20/WB21/
61
+ // WB22). The doctrine is a numbered, indented plain-text block inside the
62
+ // system prompt, so a raw value must not be able to: (WB20) inject a newline
63
+ // that forges a new numbered directive; (WB21) run on for thousands of
64
+ // characters; or (WB22) carry backticks/markdown that break the block's
65
+ // structure or read as an instruction. Control characters and newlines collapse
66
+ // to spaces, backticks and markdown structural markers are dropped, remaining
67
+ // whitespace is collapsed, and the result is capped with an ellipsis.
68
+ function sanitizeForDoctrine(value: string, max: number): string {
69
+ const cleaned = value
70
+ .replace(/[\u0000-\u001f\u007f\u009b]/g, " ") // control chars + newlines → space (WB20)
71
+ .replace(/[`*_~#>|]/g, " ") // code fences / markdown structure → space (WB22)
72
+ .replace(/\s+/g, " ")
73
+ .trim();
74
+ return cleaned.length > max ? `${cleaned.slice(0, max - 1)}\u2026` : cleaned;
75
+ }
76
+
77
+ /**
78
+ * Rule 11 (worker-extension awareness): appended ONLY when workers load extra
79
+ * pi extensions. With no units it returns "" so the doctrine is byte-identical
80
+ * to the pre-feature output (feature-off is unchanged). Phrased to match the
81
+ * imperative register of rules 1–10 (WS20/WS21/WS22): it LEADS with the action
82
+ * (delegate to a thread), states the constraint in the doctrine's own "cannot"
83
+ * voice, and CLOSES with an instruction. The extension/tool listing in the
84
+ * middle is data and renders verbatim.
85
+ */
86
+ function buildWorkerExtensionsRule(extensions: WorkerExtensionSet): string {
87
+ if (extensions.units.length === 0) return "";
88
+ const lines = [
89
+ "11. Delegate any action that needs one of these worker-loaded pi extensions to a",
90
+ " thread; you cannot call their tools yourself:",
91
+ ];
92
+ for (const unit of extensions.units) {
93
+ lines.push(` - ${sanitizeForDoctrine(unitLabel(unit), DOCTRINE_LABEL_MAX)}`);
94
+ for (const tool of unit.tools) {
95
+ lines.push(` ${sanitizeForDoctrine(tool.name, DOCTRINE_NAME_MAX)}: ${sanitizeForDoctrine(tool.description, DOCTRINE_DESC_MAX)}`);
96
+ }
97
+ }
98
+ lines.push(
99
+ " Assume every worker already has them — you need not pass them in a thread's",
100
+ " `tools` allowlist.",
101
+ );
102
+ return `\n${lines.join("\n")}`;
103
+ }
104
+
105
+ function buildDoctrine(cwd: string, config: SlateConfig, trusted: boolean, extensions: WorkerExtensionSet): string {
44
106
  // Rule 8 tail: with draft-PR publishing enabled, the umbrella draft PR is
45
107
  // one of the gates; otherwise durable records live in the workflow log.
46
108
  const rule8Tail =
@@ -92,7 +154,7 @@ threads execute. Rules:
92
154
  Read that file only when you must reason about slate itself (explaining
93
155
  it, changing the extension, or an unusual routing/compaction decision) —
94
156
  never for routine dispatching. Skip the read if it is already in your
95
- context.`;
157
+ context.${buildWorkerExtensionsRule(extensions)}`;
96
158
  }
97
159
 
98
160
  /**
@@ -130,6 +192,7 @@ export function registerSlateMode(
130
192
  store: SlateStore,
131
193
  hooks: SlateHandoffHooks,
132
194
  getConfig: () => SlateConfig,
195
+ getExtensions: () => WorkerExtensionSet,
133
196
  ): void {
134
197
  let savedTools: string[] | undefined;
135
198
  let uiCtx: ExtensionContext | undefined;
@@ -229,7 +292,7 @@ export function registerSlateMode(
229
292
  // addendum goes LAST so the pause directive is the final word in the
230
293
  // prompt, undiluted by the role guidelines.
231
294
  const parts = [
232
- buildDoctrine(ctx.cwd, config, trusted),
295
+ buildDoctrine(ctx.cwd, config, trusted, getExtensions()),
233
296
  ...loadDoctrineExtra(ctx.cwd, config, trusted).map((d) => `\n\n${d}`),
234
297
  ...docs.map((d) => `\n\n${d}`),
235
298
  ];
@@ -163,8 +163,12 @@ function stripControlChars(s: string): string {
163
163
  return s.replace(/[\u0000-\u001f\u007f\u009b]/g, "");
164
164
  }
165
165
 
166
- // Per-FRAGMENT sanitizer for text spliced into a message — same pattern and
167
- // same 120-char default as failover.ts / handoff.ts's sanitizeForNotify.
166
+ // Per-FRAGMENT sanitizer for text spliced into a message: the same logic as the
167
+ // shared helper in notify.ts (control-char strip + 120-char cap), kept LOCAL on
168
+ // purpose. This module also needs the bare stripControlChars primitive above for
169
+ // truncateForReport's word-boundary report cap, which notify.ts does not
170
+ // provide; keeping the fragment cap here lets both caps share that one strip
171
+ // primitive rather than importing one function and re-deriving its dependency.
168
172
  function sanitizeForNotify(s: string, max = 120): string {
169
173
  const clean = stripControlChars(s);
170
174
  return clean.length > max ? `${clean.slice(0, max)}…` : clean;
@@ -0,0 +1,20 @@
1
+ /**
2
+ * Shared notify-string sanitizer (CQ20).
3
+ *
4
+ * Strings that reach ctx.ui.notify / the console can carry user- or
5
+ * extension-supplied content: config values, regex patterns, extension load
6
+ * paths, loader error messages, pending-handoff fields. pi-tui renders
7
+ * control/ANSI codes verbatim, so a raw value could inject terminal escapes —
8
+ * strip control characters and cap the length before display.
9
+ *
10
+ * Imported by failover.ts, handoff.ts, worker.ts and worker-extensions.ts, which
11
+ * previously each carried a verbatim copy. model-default.ts deliberately does
12
+ * NOT import this: it also needs the bare control-char strip as a standalone
13
+ * primitive (for its word-boundary report truncation, which this helper does
14
+ * not do), so it keeps that primitive and its own fragment cap together locally
15
+ * — see the note there.
16
+ */
17
+ export function sanitizeForNotify(s: string, max = 120): string {
18
+ const clean = s.replace(/[\u0000-\u001f\u007f\u009b]/g, "");
19
+ return clean.length > max ? `${clean.slice(0, max)}…` : clean;
20
+ }
@@ -56,6 +56,7 @@ export interface ContextBudgetObject {
56
56
  export interface SlateConfig {
57
57
  episodeModel?: string; // "provider/id" for the episode compressor (D5)
58
58
  workerTools?: string[];
59
+ workerExtensions?: string[]; // regex patterns selecting which of the HOST session's pi extensions worker threads may load (default [] = none); see worker-extensions.ts
59
60
  maxConcurrent?: number; // global cap on concurrently running worker actions (default 4; must be ≥ 1 — unenforced, ≤ 0 silently hangs all dispatches; rationale: docs/design-principles.md §5 repo-local note)
60
61
  pauseThresholdPercent?: number; // DEPRECATED: legacy percent-based auto-pause (default 40); applies only when set AND contextBudget is absent or entirely invalid (invalid sanitizes to absent — a partially invalid object stays budget mode)
61
62
  contextBudget?: number | ContextBudgetObject; // absolute orchestrator token budget; bare number = { tokens: N }; {} opts into built-in defaults (256k, 400k for anthropic/*) — see handoff.ts
@@ -16,6 +16,7 @@ import { compressEpisode } from "./episodes.ts";
16
16
  import { isAuthFailure, isFailoverCandidate, resolveMappedModel } from "./failover.ts";
17
17
  import type { EpisodeRecord, SlateConfig, SlateStore, ThreadRecord } from "./state.ts";
18
18
  import { openWorkerSession, type WorkerSession } from "./worker.ts";
19
+ import { EMPTY_WORKER_EXTENSION_SET, type WorkerExtensionSet } from "./worker-extensions.ts";
19
20
 
20
21
  export interface DispatchOptions {
21
22
  threadId?: string;
@@ -113,6 +114,12 @@ export class ThreadManager {
113
114
  constructor(
114
115
  private store: SlateStore,
115
116
  private config: SlateConfig,
117
+ // This session's frozen worker-extension resolver (AD41), bound BY VALUE at
118
+ // construction (CN20) so a manager orphaned by a session swap keeps its own
119
+ // session's set instead of resolving against a later one's. Read lazily per
120
+ // new worker. Defaults to the empty-set function so existing construction
121
+ // sites and test harnesses keep working with the feature off.
122
+ private resolveExtensions: () => WorkerExtensionSet = () => EMPTY_WORKER_EXTENSION_SET,
116
123
  ) {
117
124
  this.semaphore = new Semaphore(config.maxConcurrent ?? 4); // default rationale: docs/design-principles.md §5 repo-local note
118
125
  }
@@ -272,12 +279,17 @@ export class ThreadManager {
272
279
 
273
280
  session = this.live.get(thread.id);
274
281
  if (!session) {
282
+ // A NEW session gets the CURRENT frozen worker-extension set. A LIVE
283
+ // cached session (reused when present) keeps whatever set it was opened
284
+ // with — which is precisely why the resolution is frozen per session
285
+ // (AD41): every worker in this session then shares one extension set.
275
286
  session = await openWorkerSession({
276
287
  ctx,
277
288
  sessionFile: thread.sessionFile || undefined,
278
289
  model: thread.model,
279
290
  tools: opts.tools ?? this.config.workerTools,
280
291
  promptDocs: this.config.workerPromptDocs,
292
+ extensionPaths: this.resolveExtensions().paths,
281
293
  });
282
294
  this.live.set(thread.id, session);
283
295
  // A freshly opened session starts on its configured model — drop any
@@ -0,0 +1,392 @@
1
+ /**
2
+ * Worker extensions: resolving which of the HOST session's pi extensions a
3
+ * worker thread is allowed to load.
4
+ *
5
+ * Workers normally load NO extensions (worker.ts, depth-1 recursion guard D7).
6
+ * This module lets a trusted project opt SPECIFIC host extensions back in, by
7
+ * regex, via the `workerExtensions` config. It NEVER loads anything itself —
8
+ * it only resolves a SET of extension load units (file paths or package
9
+ * directories) and the tool names they contribute, which part 2 hands to a
10
+ * worker's DefaultResourceLoader/tools allowlist. The resolution is what makes
11
+ * the feature safe; the load semantics stay pi's own.
12
+ *
13
+ * The mechanism, and the findings it answers:
14
+ *
15
+ * - CANDIDATES come from pi.getAllTools(), keyed by each tool's sourceInfo.
16
+ * Only tools that are (a) NOT built-in/sdk and (b) carry a real, absolute,
17
+ * on-disk entry path survive (AD25) — otherwise a broad pattern such as
18
+ * ".*" would re-advertise pi's own built-ins (read/bash/…) as delegatable
19
+ * "extension" tools. The sourceInfo shape is treated defensively with a
20
+ * loose cast, the way state.ts tolerates malformed session entries.
21
+ *
22
+ * - LOAD UNITS (RG1): a package-origin tool whose baseDir holds a parseable
23
+ * package.json with a `pi.extensions` array becomes a DIRECTORY unit at that
24
+ * baseDir — pi's own package-manifest resolution then expands the entry set
25
+ * (globs, override forms and all), so this module implements no manifest
26
+ * logic. Everything else becomes a FILE unit at the tool's own entry path.
27
+ * We NEVER walk parent directories looking for a manifest: in this very repo
28
+ * an auto-discovered project extension would walk up to the repo's own
29
+ * package.json and pull slate itself back into a worker.
30
+ *
31
+ * - BARRIERS drop whole units before any pattern is consulted, so a broad
32
+ * pattern can never punch through them:
33
+ * · Self-exclusion (AD24): slate's own package root is computed ONCE from
34
+ * THIS module's own location, never by asking who registered the
35
+ * `thread` tool — registration is first-wins, so with two slate copies
36
+ * loaded that lookup can resolve to a copy other than the one running.
37
+ * · Name collision (AD44/RG2): a unit that registers a tool named like a
38
+ * slate tool or a pi built-in is withheld — pi's final tool registry
39
+ * lets an extension-registered tool OVERWRITE a same-named built-in, so
40
+ * a colliding unit could silently replace a worker's own `read`.
41
+ *
42
+ * - MATCHING is unanchored `regex.test` against a unit's source spec, its
43
+ * path, or any of its tools' entry paths — any pattern hit selects the unit.
44
+ *
45
+ * - FREEZE AT FIRST USE (AD41): createWorkerExtensionResolver memoizes the
46
+ * first resolution forever. The orchestrator doctrine is rebuilt every turn
47
+ * while a live worker keeps the tool set it was opened with, and a live
48
+ * session's tool allowlist cannot be widened afterwards — freezing makes the
49
+ * doctrine and every worker opened in that session provably the same set.
50
+ * First use happens AFTER the host's session_start, so extensions that
51
+ * register tools during session_start are still captured; a fresh resolver
52
+ * is created per session_start, so a restart re-resolves.
53
+ *
54
+ * Warnings (a withheld colliding unit) go through an injected `warn` callback,
55
+ * never console.log — the same reporting discipline as failover.ts.
56
+ */
57
+
58
+ import { existsSync, readFileSync } from "node:fs";
59
+ import { dirname, isAbsolute, join, resolve, sep } from "node:path";
60
+ import { fileURLToPath } from "node:url";
61
+ import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
62
+ import { sanitizeForNotify } from "./notify.ts";
63
+
64
+ export interface WorkerExtensionTool {
65
+ name: string;
66
+ description: string;
67
+ }
68
+
69
+ export interface WorkerExtensionUnit {
70
+ path: string; // absolute path of the load unit (a file, or a package directory when isDirectory)
71
+ source: string; // recorded source spec of the unit's first tool (sourceInfo.source)
72
+ isDirectory: boolean; // true = package-directory unit (RG1); false = single extension file
73
+ tools: WorkerExtensionTool[];
74
+ }
75
+
76
+ export interface WorkerExtensionSet {
77
+ units: WorkerExtensionUnit[]; // selected units, sorted by path
78
+ paths: string[]; // selected unit paths, unit order
79
+ toolNames: string[]; // de-duplicated tool names of the selected units, unit order
80
+ }
81
+
82
+ /**
83
+ * The empty result — the off-by-default state. Shared so every consumer's
84
+ * feature-off path (index.ts's initial resolver, ThreadManager's default,
85
+ * the fast paths below) allocates nothing meaningful. DEEP-FROZEN (CQ22) so a
86
+ * stray `.push`/reassignment from any one consumer cannot corrupt the object
87
+ * every other feature-off consumer reads.
88
+ */
89
+ export const EMPTY_WORKER_EXTENSION_SET: WorkerExtensionSet = Object.freeze({
90
+ units: Object.freeze([]),
91
+ paths: Object.freeze([]),
92
+ toolNames: Object.freeze([]),
93
+ }) as unknown as WorkerExtensionSet;
94
+
95
+ /** Slate's own model-facing tools — a unit registering any of these is withheld (AD44/RG2). */
96
+ export const SLATE_TOOL_NAMES = ["thread", "threads", "episode"];
97
+
98
+ /** pi's built-in tools — a unit registering any of these would overwrite a worker's own (AD44/RG2). */
99
+ export const PI_BUILTIN_TOOL_NAMES = ["read", "bash", "edit", "write", "grep", "find", "ls"];
100
+
101
+ /** Union of the two lists above, as a set, for the collision barrier. */
102
+ const COLLISION_NAMES = new Set([...SLATE_TOOL_NAMES, ...PI_BUILTIN_TOOL_NAMES]);
103
+
104
+ /**
105
+ * Slate's own package root, computed ONCE from this module's location (AD24):
106
+ * fileURLToPath(import.meta.url) → .../extension/worker-extensions.ts, its
107
+ * directory → .../extension, its parent → the package root. Never derived by
108
+ * looking up who registered the `thread` tool (first-wins registration can
109
+ * resolve to a different slate copy than the one executing).
110
+ */
111
+ const SLATE_PACKAGE_ROOT = dirname(dirname(fileURLToPath(import.meta.url)));
112
+
113
+ /**
114
+ * Separator-aware containment: true iff `child` is `parent` or lies under it.
115
+ * The trailing separator is what keeps "/a/bc" from counting as inside "/a/b".
116
+ */
117
+ function pathContains(parent: string, child: string): boolean {
118
+ if (parent === child) return true;
119
+ const prefix = parent.endsWith(sep) ? parent : parent + sep;
120
+ return child.startsWith(prefix);
121
+ }
122
+
123
+ /** Loose views of the getAllTools() shape — tolerated defensively (state.ts pattern). */
124
+ interface LooseSourceInfo {
125
+ path?: unknown;
126
+ source?: unknown;
127
+ origin?: unknown;
128
+ baseDir?: unknown;
129
+ }
130
+ interface LooseTool {
131
+ name?: unknown;
132
+ description?: unknown;
133
+ sourceInfo?: LooseSourceInfo;
134
+ }
135
+
136
+ /** A surviving candidate tool with its resolved on-disk entry path and unit membership. */
137
+ interface Candidate {
138
+ name: string;
139
+ description: string;
140
+ source: string;
141
+ path: string; // the tool's own sourceInfo.path (absolute, exists on disk)
142
+ baseDir?: string; // package directory when this tool qualifies for a directory unit (RG1)
143
+ }
144
+
145
+ /** Working unit accumulated during grouping; toolPaths is kept for matching only. */
146
+ interface WorkingUnit {
147
+ path: string;
148
+ source: string;
149
+ isDirectory: boolean;
150
+ tools: WorkerExtensionTool[];
151
+ toolPaths: string[];
152
+ }
153
+
154
+ /** The `pi.extensions` array from a package.json, or undefined when absent/invalid/not a manifest. */
155
+ type ManifestEntries = string[] | undefined;
156
+
157
+ /**
158
+ * Read and parse <baseDir>/package.json, returning its `pi.extensions` array.
159
+ * CACHED per directory for the whole resolution (CQ21 — the manifest was
160
+ * previously re-read and re-parsed once per candidate tool). undefined when the
161
+ * file is missing, unreadable, invalid JSON, or lacks a `pi.extensions` array.
162
+ */
163
+ function readPackageExtensions(baseDir: string, cache: Map<string, ManifestEntries>): ManifestEntries {
164
+ if (cache.has(baseDir)) return cache.get(baseDir);
165
+ let entries: ManifestEntries;
166
+ const manifest = join(baseDir, "package.json");
167
+ if (existsSync(manifest)) {
168
+ try {
169
+ const parsed = JSON.parse(readFileSync(manifest, "utf8")) as { pi?: { extensions?: unknown } } | null;
170
+ const ext = parsed?.pi?.extensions;
171
+ if (Array.isArray(ext)) entries = ext as string[];
172
+ } catch {
173
+ /* unreadable/invalid manifest → undefined (fall back to file units) */
174
+ }
175
+ }
176
+ cache.set(baseDir, entries);
177
+ return entries;
178
+ }
179
+
180
+ /**
181
+ * RG1: is `info` a package-origin tool whose baseDir carries a valid extension
182
+ * manifest? Returns that baseDir (a directory-unit CANDIDATE) or undefined (a
183
+ * file unit). Whether the directory form is actually USED is decided later by
184
+ * the BG20 equivalence check, once all candidates are known. Anything short of
185
+ * a package origin with a `pi.extensions` manifest — wrong origin, no baseDir,
186
+ * missing/unreadable/invalid manifest — is a file unit. Parents are never
187
+ * consulted.
188
+ */
189
+ function packageBaseDir(info: LooseSourceInfo, cache: Map<string, ManifestEntries>): string | undefined {
190
+ if (info.origin !== "package") return undefined;
191
+ const baseDir = info.baseDir;
192
+ if (typeof baseDir !== "string" || baseDir === "") return undefined;
193
+ return readPackageExtensions(baseDir, cache) !== undefined ? baseDir : undefined;
194
+ }
195
+
196
+ /**
197
+ * BG20: a manifest entry is safe to hand to pi via the DIRECTORY form only if it
198
+ * is a plain literal relative path — no glob metacharacters and none of the
199
+ * !/+/- override prefixes pi's manifest matcher understands. A glob can expand
200
+ * differently in the worker than it did in the host, and an override prefix
201
+ * changes which declared entries load; either way the directory form risks
202
+ * loading a SUPERSET of what the host runs, so such a unit falls back to
203
+ * explicit file units.
204
+ */
205
+ function isLiteralManifestEntry(entry: string): boolean {
206
+ if (entry === "" || /^[!+\-]/.test(entry)) return false;
207
+ return !/[*?[\]{}()!]/.test(entry);
208
+ }
209
+
210
+ /**
211
+ * Validate the raw `workerExtensions` config value into a list of regex source
212
+ * strings. undefined → [] (feature off, silently). A non-array warns and
213
+ * yields []. Per element: non-strings, empty strings, and patterns that fail
214
+ * `new RegExp` are dropped with a warning; the rest are kept in order.
215
+ */
216
+ export function sanitizeWorkerExtensions(value: unknown, warn: (msg: string) => void): string[] {
217
+ if (value === undefined) return [];
218
+ if (!Array.isArray(value)) {
219
+ warn("slate: workerExtensions must be an array of regex strings — ignoring");
220
+ return [];
221
+ }
222
+ const patterns: string[] = [];
223
+ for (const entry of value) {
224
+ if (typeof entry !== "string" || entry === "") {
225
+ warn(`slate: ignoring workerExtensions entry ${sanitizeForNotify(JSON.stringify(entry))} — expected a non-empty regex string`);
226
+ continue;
227
+ }
228
+ try {
229
+ new RegExp(entry);
230
+ } catch {
231
+ warn(`slate: ignoring workerExtensions pattern ${sanitizeForNotify(JSON.stringify(entry))} — not a valid regular expression`);
232
+ continue;
233
+ }
234
+ patterns.push(entry);
235
+ }
236
+ return patterns;
237
+ }
238
+
239
+ /**
240
+ * Resolve the set of worker-loadable extension units for `patterns`.
241
+ *
242
+ * An empty pattern list returns an empty set WITHOUT walking the registry —
243
+ * the fast path, since the feature is off by default. `warn` receives the
244
+ * collision-barrier notices (AD44/RG2); it defaults to a no-op so a bare
245
+ * `resolveWorkerExtensions(pi, patterns)` still type-checks and runs.
246
+ */
247
+ export function resolveWorkerExtensions(
248
+ pi: ExtensionAPI,
249
+ patterns: string[],
250
+ warn: (msg: string) => void = () => {},
251
+ ): WorkerExtensionSet {
252
+ if (patterns.length === 0) return EMPTY_WORKER_EXTENSION_SET;
253
+
254
+ // One manifest read per package directory for this whole resolution (CQ21).
255
+ const manifestCache = new Map<string, ManifestEntries>();
256
+
257
+ // 1. CANDIDATES (AD25): extension-owned tools with a real on-disk entry path.
258
+ const candidates: Candidate[] = [];
259
+ for (const raw of pi.getAllTools()) {
260
+ const tool = raw as unknown as LooseTool;
261
+ const info = tool.sourceInfo;
262
+ if (!info) continue;
263
+ const source = typeof info.source === "string" ? info.source : "";
264
+ if (source === "builtin" || source === "sdk") continue;
265
+ const path = typeof info.path === "string" ? info.path : "";
266
+ if (path === "" || !isAbsolute(path) || !existsSync(path)) continue;
267
+ const name = typeof tool.name === "string" ? tool.name : "";
268
+ if (name === "") continue;
269
+ const description = typeof tool.description === "string" ? tool.description : "";
270
+ candidates.push({ name, description, source, path, baseDir: packageBaseDir(info, manifestCache) });
271
+ }
272
+ if (candidates.length === 0) return EMPTY_WORKER_EXTENSION_SET;
273
+
274
+ // 2. DIRECTORY UNITS (RG1 + BG20): a package baseDir becomes a directory unit
275
+ // — handed to pi WHOLE so pi's own manifest resolution expands the entries —
276
+ // ONLY when that is provably equivalent to what the host loaded: every manifest
277
+ // entry is a literal relative path AND every one of them is an entry the host
278
+ // actually loaded (a tool we saw). Otherwise the host may be running a filtered
279
+ // subset (a settings package entry can carry its own extensions filter) or a
280
+ // glob could expand differently in the worker, so the unit falls back to file
281
+ // units below. TRADE-OFF: the fallback drops a package's no-tool companion
282
+ // entries — correct, because an entry that registered no tool is one we cannot
283
+ // prove the host is running.
284
+ const dirUnits: string[] = [];
285
+ const seenDirs = new Set<string>();
286
+ for (const c of candidates) {
287
+ if (!c.baseDir || seenDirs.has(c.baseDir)) continue;
288
+ seenDirs.add(c.baseDir);
289
+ const baseDir = c.baseDir;
290
+ const entries = readPackageExtensions(baseDir, manifestCache);
291
+ if (!entries || entries.length === 0) continue;
292
+ const loadedEntryPaths = new Set(candidates.filter((o) => pathContains(baseDir, o.path)).map((o) => resolve(o.path)));
293
+ const equivalent = entries.every(
294
+ (e) => typeof e === "string" && isLiteralManifestEntry(e) && loadedEntryPaths.has(resolve(baseDir, e)),
295
+ );
296
+ if (equivalent) dirUnits.push(baseDir);
297
+ }
298
+
299
+ // 3. GROUP candidates into units. A candidate whose entry path lies inside a
300
+ // validated directory unit joins it (most specific wins); everything else
301
+ // (including candidates whose package failed the BG20 check) becomes a file
302
+ // unit keyed by its own path.
303
+ const working = new Map<string, WorkingUnit>();
304
+ for (const c of candidates) {
305
+ let unitPath: string | undefined;
306
+ for (const d of dirUnits) {
307
+ if (pathContains(d, c.path) && (unitPath === undefined || d.length > unitPath.length)) unitPath = d;
308
+ }
309
+ const isDirectory = unitPath !== undefined;
310
+ const key = unitPath ?? c.path;
311
+ let unit = working.get(key);
312
+ if (!unit) {
313
+ // unit.source is the source spec of the unit's FIRST tool.
314
+ unit = { path: key, source: c.source, isDirectory, tools: [], toolPaths: [] };
315
+ working.set(key, unit);
316
+ }
317
+ unit.tools.push({ name: c.name, description: c.description });
318
+ unit.toolPaths.push(c.path);
319
+ }
320
+
321
+ // Deterministic order (needed for stable doctrine/allowlist output).
322
+ const units = [...working.values()].sort((a, b) => (a.path < b.path ? -1 : a.path > b.path ? 1 : 0));
323
+
324
+ // 4. BARRIERS — applied to EVERY unit before matching, regardless of patterns.
325
+ const surviving: WorkingUnit[] = [];
326
+ for (const unit of units) {
327
+ // (a) Self-exclusion (AD24): drop when the unit equals slate's root, sits
328
+ // inside it, or contains it.
329
+ if (pathContains(SLATE_PACKAGE_ROOT, unit.path) || pathContains(unit.path, SLATE_PACKAGE_ROOT)) continue;
330
+ // (b) Name collision (AD44/RG2): checked across the WHOLE unit, not just a
331
+ // matched tool — any collision withholds the unit and warns.
332
+ const collisions = unit.tools.map((t) => t.name).filter((n) => COLLISION_NAMES.has(n));
333
+ if (collisions.length > 0) {
334
+ warn(
335
+ `slate: worker extension ${sanitizeForNotify(unit.path)} withheld from workers — it registers ` +
336
+ `${collisions.map((n) => `"${n}"`).join(", ")}, which would overwrite a slate or pi built-in tool`,
337
+ );
338
+ continue;
339
+ }
340
+ surviving.push(unit);
341
+ }
342
+
343
+ // 5. MATCH: compile each pattern once; a unit is selected on ANY hit against
344
+ // its source spec, its path, or any of its tools' entry paths. Patterns are
345
+ // pre-validated by sanitizeWorkerExtensions; a stray invalid one is dropped.
346
+ const regexes: RegExp[] = [];
347
+ for (const p of patterns) {
348
+ try {
349
+ regexes.push(new RegExp(p));
350
+ } catch {
351
+ /* pre-validated upstream; tolerate a raw invalid pattern */
352
+ }
353
+ }
354
+ const selected = surviving.filter((unit) =>
355
+ regexes.some((re) => re.test(unit.source) || re.test(unit.path) || unit.toolPaths.some((tp) => re.test(tp))),
356
+ );
357
+
358
+ // 6. RESULT: de-duplicate tool names across selected units, in unit order.
359
+ const seen = new Set<string>();
360
+ const toolNames: string[] = [];
361
+ for (const unit of selected) {
362
+ for (const t of unit.tools) {
363
+ if (seen.has(t.name)) continue;
364
+ seen.add(t.name);
365
+ toolNames.push(t.name);
366
+ }
367
+ }
368
+ return {
369
+ units: selected.map((u) => ({ path: u.path, source: u.source, isDirectory: u.isDirectory, tools: u.tools })),
370
+ paths: selected.map((u) => u.path),
371
+ toolNames,
372
+ };
373
+ }
374
+
375
+ /**
376
+ * Wrap resolveWorkerExtensions so it runs at most ONCE per session (AD41): the
377
+ * first call resolves and caches; every later call returns that same frozen
378
+ * set. `getPatterns` is read lazily at first use — after session_start, so
379
+ * tools registered during session_start are captured. A new resolver is
380
+ * created per session_start, so a restart re-resolves from scratch.
381
+ */
382
+ export function createWorkerExtensionResolver(
383
+ pi: ExtensionAPI,
384
+ getPatterns: () => string[],
385
+ warn: (msg: string) => void = () => {},
386
+ ): () => WorkerExtensionSet {
387
+ let cached: WorkerExtensionSet | undefined;
388
+ return () => {
389
+ if (cached === undefined) cached = resolveWorkerExtensions(pi, getPatterns(), warn);
390
+ return cached;
391
+ };
392
+ }
@@ -1,11 +1,29 @@
1
1
  /**
2
2
  * Worker sessions: in-process pi SDK AgentSessions with the recursion guard.
3
3
  *
4
- * A worker loads NO extensions, skills, prompt templates, or themes
5
- * (DefaultResourceLoader no* options) — so a worker can never see slate tools
6
- * (ExecPlan D7, depth-1 guard) — and inherits the HOST session's project-trust
7
- * state via an explicit SettingsManager, so untrusted projects get neither
8
- * project-local settings nor the project SYSTEM.md override in workers.
4
+ * A worker loads NO skills, prompt templates, or themes (DefaultResourceLoader
5
+ * no* options), and by default NO extensions either — so a worker can never see
6
+ * slate tools (ExecPlan D7, depth-1 guard). It inherits the HOST session's
7
+ * project-trust state via an explicit SettingsManager, so untrusted projects get
8
+ * neither project-local settings nor the project SYSTEM.md override in workers.
9
+ *
10
+ * When the host resolved a `workerExtensions` whitelist (worker-extensions.ts),
11
+ * the loader runs in ALLOWLIST mode: noExtensions STAYS true — so auto-discovery
12
+ * (slate itself included) never runs — while additionalExtensionPaths adds back
13
+ * EXACTLY the resolved units and nothing else. The depth-1 guard is STRUCTURAL
14
+ * (SE20): createAgentSession is given an excludeTools denylist of slate's
15
+ * dispatch tools (thread/threads/episode), and the SDK re-applies that denylist
16
+ * on every tool-registry refresh, so no worker can call them no matter when or
17
+ * by whom they are registered — even a deferred before_agent_start registration.
18
+ * The resolver's load-scoped barriers plus a best-effort post-load scan (RG2
19
+ * below) additionally reject a unit that shadows a pi BUILT-IN at load time; a
20
+ * deferred built-in shadow cannot be prevented from slate's side, exactly as it
21
+ * cannot be for any extension in the host session.
22
+ * Load units are ABSOLUTE paths (package directories or entry files), NEVER npm/
23
+ * git specs (AD20): a spec resolves at temporary scope against a separate install
24
+ * root, which would not reuse the host's installed copy and would attempt a
25
+ * network install.
26
+ *
9
27
  * Worker conversations persist under
10
28
  * <config dir>/slate/threads/*.jsonl (CONFIG_DIR_NAME, ".pi" by default)
11
29
  * and are reopened via SessionManager.open.
@@ -22,7 +40,9 @@ import {
22
40
  SettingsManager,
23
41
  type ExtensionContext,
24
42
  } from "@earendil-works/pi-coding-agent";
43
+ import { sanitizeForNotify } from "./notify.ts";
25
44
  import { loadPromptDocs } from "./prompt-docs.ts";
45
+ import { PI_BUILTIN_TOOL_NAMES, SLATE_TOOL_NAMES } from "./worker-extensions.ts";
26
46
 
27
47
  export type WorkerSession = Awaited<ReturnType<typeof createAgentSession>>["session"];
28
48
 
@@ -58,6 +78,7 @@ export async function openWorkerSession(opts: {
58
78
  model?: string; // "provider/id"
59
79
  tools?: string[];
60
80
  promptDocs?: string[]; // role-guideline doc paths, cwd-relative (default none)
81
+ extensionPaths?: string[]; // absolute worker-extension load units (package dirs or entry files); default none
61
82
  }): Promise<WorkerSession> {
62
83
  const { ctx } = opts;
63
84
  const dir = threadsDir(ctx.cwd);
@@ -110,11 +131,24 @@ export async function openWorkerSession(opts: {
110
131
  // joins appendSystemPrompt entries with "\n\n". Trust gate: project files
111
132
  // are never injected into worker prompts for untrusted projects.
112
133
  const promptDocs = ctx.isProjectTrusted() ? loadPromptDocs(ctx.cwd, opts.promptDocs ?? []) : [];
134
+ // Absolute load units resolved by the host (worker-extensions.ts); empty =
135
+ // feature off, the historical no-extensions worker. Defense in depth (CQ23):
136
+ // re-gate on project trust here even though the config that produced these
137
+ // paths is itself only loaded for trusted projects (index.ts) — an untrusted
138
+ // project must never load extensions into a worker, whatever a future caller
139
+ // passes.
140
+ const extensionPaths = ctx.isProjectTrusted() ? (opts.extensionPaths ?? []) : [];
113
141
  const loader = new DefaultResourceLoader({
114
142
  cwd: ctx.cwd,
115
143
  agentDir,
116
144
  settingsManager,
117
- noExtensions: true, // recursion guard (D7)
145
+ // Recursion guard (D7): auto-discovery stays OFF. additionalExtensionPaths
146
+ // runs the loader in allowlist mode — noExtensions keeps every
147
+ // auto-discovered extension out (slate included) while these absolute units
148
+ // are the only ones added back (AD20 — see the module header on why paths,
149
+ // never specs). undefined when empty so the default worker is untouched.
150
+ noExtensions: true,
151
+ additionalExtensionPaths: extensionPaths.length > 0 ? extensionPaths : undefined,
118
152
  noSkills: true,
119
153
  noPromptTemplates: true,
120
154
  noThemes: true,
@@ -122,6 +156,56 @@ export async function openWorkerSession(opts: {
122
156
  });
123
157
  await loader.reload();
124
158
 
159
+ // Extension tool names actually registered by the whitelisted units — needed
160
+ // both for the collision re-check and for the tools allowlist below. Stays
161
+ // empty (and this whole block is skipped) unless a whitelist was resolved.
162
+ const extensionToolNames: string[] = [];
163
+ if (extensionPaths.length > 0) {
164
+ const warn = (msg: string) => (ctx.hasUI ? ctx.ui.notify(msg, "warning") : console.warn(msg));
165
+ const loaded = loader.getExtensions();
166
+ // A whitelisted extension that failed to load must not vanish silently —
167
+ // surface every loader error naming the path. Paths and messages are
168
+ // extension-supplied and flow to the UI, the console and the persisted
169
+ // episode, so sanitize them (SE22) like every other displayed string.
170
+ for (const err of loaded.errors ?? []) {
171
+ warn(`slate: worker extension failed to load — ${sanitizeForNotify(String(err.path))}: ${sanitizeForNotify(String(err.error))}`);
172
+ }
173
+ // Collect the tool names each loaded extension actually registered. Loose
174
+ // cast + Map guard: tolerate a malformed extensions/tools shape (state.ts
175
+ // pattern).
176
+ const collisions: string[] = [];
177
+ for (const ext of loaded.extensions ?? []) {
178
+ const e = ext as unknown as { path?: unknown; tools?: unknown };
179
+ if (!(e.tools instanceof Map)) continue;
180
+ for (const name of e.tools.keys()) {
181
+ if (typeof name !== "string" || name === "") continue;
182
+ extensionToolNames.push(name);
183
+ // LOAD-TIME collision scan (RG2), BEST-EFFORT: it sees only what the
184
+ // extensions registered DURING loader.reload(). A deferred registration
185
+ // (e.g. from a before_agent_start handler on the worker's first prompt)
186
+ // is invisible here — and is equally available to any extension the host
187
+ // session runs, so it is outside slate's control. slate's OWN invariant
188
+ // does NOT rely on this scan: thread/threads/episode are denied
189
+ // structurally by the excludeTools denylist on createAgentSession below
190
+ // (re-applied on every tool-registry refresh), so no worker can call them
191
+ // whenever they are registered. For a pi BUILT-IN there is no such
192
+ // gap-closer (the worker needs the real built-in), so this scan catches a
193
+ // LOAD-TIME shadow and fails the dispatch closed; a deferred shadow cannot
194
+ // be prevented.
195
+ if (SLATE_TOOL_NAMES.includes(name) || PI_BUILTIN_TOOL_NAMES.includes(name)) {
196
+ const path = typeof e.path === "string" ? e.path : "extension";
197
+ collisions.push(`${sanitizeForNotify(path)} → "${sanitizeForNotify(name)}"`);
198
+ }
199
+ }
200
+ }
201
+ if (collisions.length > 0) {
202
+ // Thrown BEFORE createAgentSession below, so no worker session is leaked.
203
+ throw new Error(
204
+ `slate: refusing to open worker — whitelisted extension(s) register tool(s) that would overwrite a slate or pi built-in tool: ${collisions.join(", ")}`,
205
+ );
206
+ }
207
+ }
208
+
125
209
  const model = opts.model ? resolveModel(ctx, opts.model) : ctx.model;
126
210
 
127
211
  const sessionManager = opts.sessionFile
@@ -148,7 +232,21 @@ export async function openWorkerSession(opts: {
148
232
  // the SDK re-clamp against the CURRENT model instead. "medium" mirrors
149
233
  // the SDK's DEFAULT_THINKING_LEVEL (not exported).
150
234
  thinkingLevel: settingsManager.getDefaultThinkingLevel() ?? "medium",
151
- tools: opts.tools && opts.tools.length > 0 ? opts.tools : DEFAULT_WORKER_TOOLS,
235
+ // workerTools / the per-dispatch `tools` argument govern the BUILT-IN tools
236
+ // only; pi's allowlist gates extension tools too, so the whitelisted
237
+ // extensions' registered tool names are unioned in (de-duplicated) — without
238
+ // this the extensions would load but stay inert (present in the registry,
239
+ // absent from the active set). A colliding built-in name was rejected above.
240
+ tools: [
241
+ ...new Set([...(opts.tools && opts.tools.length > 0 ? opts.tools : DEFAULT_WORKER_TOOLS), ...extensionToolNames]),
242
+ ],
243
+ // STRUCTURAL depth-1 guard (SE20): slate's dispatch tools are denied to EVERY
244
+ // worker session. excludeTools applies AFTER the allowlist and the SDK
245
+ // re-applies it on every tool-registry refresh, so a tool named
246
+ // thread/threads/episode is filtered out no matter when it is registered
247
+ // (including a deferred before_agent_start registration) — a worker can never
248
+ // re-enter the orchestrator's dispatch surface (D7).
249
+ excludeTools: SLATE_TOOL_NAMES,
152
250
  resourceLoader: loader,
153
251
  sessionManager,
154
252
  settingsManager,
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "ytdb-slate",
3
- "version": "0.3.0",
3
+ "version": "0.4.0",
4
4
  "description": "Slate: thread-weaving orchestration for the pi coding agent — bounded worker actions, compressed episodes, mandatory research/design/adversarial/track review workflow, configurable model failover, and orchestrator context-budget handoff.",
5
5
  "license": "Apache-2.0",
6
6
  "repository": {