@andorra-labs/code 0.0.0-stage → 0.1.5

package/tests/jev-efficiency-test-plan.md

@@ -0,0 +1,198 @@
+# JEV efficiency features: QA test plan
+
+## Scope and prerequisites
+
+Five independent, off-by-default switches under `config.json` → `efficiency` reduce coding-model
+turns and request size (see `docs/concepts/jev-efficiency.md`):
+
+| Switch | What it changes | Code |
+| --- | --- | --- |
+| `preparedContinuations` | `prepare_steps` accepts `when: success/failure/always` on investigate/verify steps; the first matching step runs after the batch that prepared it without a decision round | `src/decision/PreparedSteps.ts`, `src/hardened-runtime.ts` |
+| `postflightOnlyWithCandidates` | the post-tool decision round is not requested when no prepared candidate could change execution | `src/hardened-runtime.ts` |
+| `toolResultFiltering` (`toolResultFilterThresholdChars`, default 6000) | bulky shell/search/listing results are parsed once at insertion; the original is archived and expandable through `expand_tool_output` | `src/tool-result-filter.ts`, `src/hardened-runtime.ts` |
+| `contextSelection` | workspace facts gate automatic retrieval; retrieved repository files and memory entries are ranked and the omitted ones named; unchanged blocks already in an inherited conversation are not re-injected | `src/workspace-facts.ts`, `src/context-selection.ts`, `src/agent-execution.ts`, `src/graph-runner.ts` |
+| `catalogSelection` | deterministic task-appropriate tool catalog from permitted tools, restorable with `restore_tools`; relevance-ordered, budgeted skill catalog; catalog dedup across nodes | `src/tool-catalog.ts`, `src/agent-execution.ts` |
+| `adaptiveRouting` | opt-in model-tier routing among configured models, pins respected, bounded upward escalation | `src/model-tiers.ts`, `src/agent-execution.ts` |
+
+Every switch records `efficiency.*` events with fixed reason codes and a local `efficiency` tally on the
+run telemetry (`src/telemetry/capture.ts`). Nothing is uploaded, and no hosted Jev payload grew.
+
+Platforms: every Node platform the CLI supports; the evidence below was gathered on Windows 11 with
+Node 24.13.1. Prerequisites: `npm ci`, `npm run build`. No credentials are needed for any check
+except the optional live evaluation. Every check uses its own throwaway `ANDORRACODE_HOME` and
+deletes it.
+
+## How to use the feature
+
+1. Add an `efficiency` block to `config.json` (documented in `docs/reference/config.md`), for example
+   `{ "efficiency": { "toolResultFiltering": true, "catalogSelection": true } }`. Omitting the block, or a
+   switch, is the shipped behavior.
+2. Run a task as usual (`andorra`, or `andorra --prompt ... --json --yolo` headless). Inspect
+   `<ANDORRACODE_HOME>/runs/<run-id>.jsonl` for `efficiency.*` events, and the run's telemetry
+   capture for the `efficiency` tally.
+3. For adaptive routing, add `adaptiveRouting: { enabled: true, tiers: { economy: { provider, model }, ... } }`
+   naming configured providers. A `--model`/`--provider` flag, a desktop "use for every node" override,
+   or a node pin disables routing for that run or node.
+4. Optionally add `pricing: { "<provider>/<model>": { inputPerMillionUsd, outputPerMillionUsd, cacheReadPerMillionUsd? } }`
+   under `adaptiveRouting`. The runtime bills every measured request against the table in its local
+   telemetry (`models[].costMicroUsd`, present only when every request of that model was priced), and
+   the live evaluator reports the run total. A resumed routed visit continues on the pair that wrote
+   its checkpointed conversation (the visit is pinned; routing decides again on the next fresh
+   activation) as long as the configuration still resolves to the same baseline; a changed baseline
+   restarts the visit, and an unresolvable routed pair refuses the resume instead of repeating work.
+5. To roll back, remove the switch; no stored state depends on it.
+
+## Executable checks
+
+```powershell
+npm run build
+npm run test:jev
+npm run eval:jev
+node tests/jev-efficiency-cli-smoke.mjs
+node --test scripts/ci-local.test.mjs
+```
+
+Optional, with a dedicated evaluation home that is signed in with an evaluation account (`ANDORRACODE_HOME=<home> andorra login`; never the day-to-day home, credentials are scoped to the exact home and are never copied): `ANDORRA_LIVE_EVAL=1 ANDORRA_LIVE_EVAL_HOME=<home> npm run eval:jev:live -- --pairs 2`. The script runs one probe first and stops with exit 3 if that run requested no decision, because an anonymous or decision-disabled home would measure the prepared-continuation and postflight switches as disabled. Every child process — the probe, each paired run and its executable check — is terminated at the remaining `--budget-minutes` allowance, and a run the deadline killed is recorded as timed out in `summary.json` instead of being awaited forever. `--feature adaptiveRouting` measures the evaluation home's own `efficiency.adaptiveRouting` configuration as the candidate; the script refuses a home that does not enable it with tiers resolvable in the home's providers.
+
+## Test cases and expected results
+
+| ID | Preconditions and steps | Expected result |
+| --- | --- | --- |
+| QA-01 | `test/jev-efficiency-runtime.test.ts` "a prepared verify step with a success trigger…": the model prepares a `verify` step with `when: success` and writes a file. | The check runs between the two model requests without a decision round (`efficiency.postflight.skipped` reason `deterministic_continuation`); one `decision.turn_bypassed` with `selection: deterministic_continuation` after the action executed; `efficiency.prepared.declared` carries the preparation size. With the switch off the `when` key is refused by the shipped schema and no `efficiency.prepared.*` event exists. |
+| QA-02 | Same file, "a failure trigger selects the diagnostic step…": steps with `when: failure` and `when: success` after a failing shell call. | Only the `investigate` step runs; a success trigger never fires on a failure, and the batch still gets its semantic round. |
+| QA-03 | Same file, "a triggered action that the catalog no longer advertises, or that permissions changed under…": a hook blocks the step; a tool changes the permission mode before the step runs. | `efficiency.prepared.rejected` with reason `blocked` / `stale_permissions`; no `decision.turn_bypassed`; the stale action never starts. |
+| QA-04 | Same file, "steering invalidates prepared steps…": a steer arrives after preparation; four `always` steps; `retry`/`complete` steps with triggers. | `efficiency.prepared.invalidated` reason `stale_task`; exactly one deterministic continuation per model batch (the rest become candidates of the next round, within the existing limit of three automatic turns); `when` on a retry or completion is refused. |
+| QA-05 | Same file, "without candidates the postflight round is skipped under the switch…". | 0 `decision.requested` with the switch, 1 without; a batch with candidates is still requested and awaited, with `efficiency.postflight.awaited.waitMs`. |
+| QA-06 | Same file, "a bulky test report is filtered once at insertion…": a 300-pass, 2-fail node:test report over the threshold. | The model-visible observation is under half the original and keeps both failures and the counts; `node.tool.completed` keeps the real output; `expand_tool_output` is advertised only after the first archive and returns the exact range with `efficiency.tool_result.expanded`; another invocation's reference is refused (`expansion_rejected`); small outputs and the feature-off run are byte-identical to the baseline. |
+| QA-07 | Same file, "a filtered observation never satisfies a prepared completion". | A decisive `finish` choice on a filtered passing report returns to the model (two requests, model answer). |
+| QA-08 | Same file, "two concurrent runs keep their archives, prepared steps and tallies apart". | Distinct `tool-output://` references, one executed continuation each, no shared invocation id. |
+| QA-09 | Same file, "executeAgentNode: catalog selection omits unindicated schemas, restoration brings one back…": empty temp workspace, a rename task, git/web tools permitted. | `git_read` and `web_lookup` are absent from the first request's schemas with a prompt note; `restore_tools` re-advertises `git_read`; `telemetry.efficiency.catalog.selections` = 1; without the switch the full catalog is advertised and no tally exists. |
+| QA-10 | Same file, "executeAgentNode: adaptive routing re-tiers a mechanical task…": tiers economy/balanced/premium, a mechanical task, then a pinned run, then an uncertain task. | First request on the economy model; after repeated tool failure one escalation to the baseline model whose task names the file the first attempt wrote; `routing.escalations` = 1; a pinned run and an uncertain task stay on the baseline (`pinned`, `low_confidence`). |
+| QA-11 | Same file, "feature-disabled requests equal the baseline request byte for byte". | `{}` and an all-false block serialize identically to no block. |
+| QA-12 | `test/tool-result-filter.test.ts`: TAP with 512 pass / 2 fail, a failure in the middle of 600, a verbose pytest report whose underscore-delimited failure sections (assertions, tracebacks, captured output) must survive while its 100 PASSED lines go, compiler output, 1,500 search matches, a 900-entry listing, progress bars, prose, small, source reads, denied errors, archive scoping; an 80-failure report under the 24,000-character clip; a report whose headings alone exceed the bound. | Failures with assertion and stack survive, counts survive, the exit header is pinned, injected text never enters the trailer, unknown formats and small outputs are unchanged, source reads and non-exit errors are protected, ranges reconstruct the original exactly, foreign and traversal references are refused; the over-budget fit keeps every failure heading and whole surviving blocks, and a report whose headings alone exceed the bound declines filtering with the original kept whole. |
+| QA-13 | `test/context-selection.test.ts`: the refresh-token fixture; a tight budget; dedup ledger; workspace facts. | Implementation, contracts, tests and history kept, billing and styling omitted; mandatory and dependencies survive; a changed file is not a duplicate; an empty folder gates both retrievals, a missing repository gates none. |
+| QA-14 | `test/tool-catalog.test.ts`: empty-folder UI task, repository task naming git/docs/delegation, explicit and historical tools, byte-stable ordering, skill ordering. | Families omitted only on certainties; protected, explicit, historical and unknown tools kept; identical inputs give identical serialization; required skills lead the catalog and the rest stay loadable by name. |
+| QA-15 | `test/model-tiers.test.ts` and `test/efficiency-config.test.ts`. | Pins respected, unconfigured tiers keep the baseline, risk prevents downgrades, low confidence keeps the baseline, escalation bounded and upward only, cost unavailable without a price; configuration validation rejects unknown or mistyped switches; the tally folds and merges. |
+| QA-16 | `npm run eval:jev` (deterministic replay, six workloads, nine configurations each). | Exit 0; `tests/jev-efficiency/results/latest.md` shows every criterion of `tests/jev-efficiency/criteria.json` passing, including the byte-identical feature-disabled comparison for every workload. |
+| QA-17 | `node tests/jev-efficiency-cli-smoke.mjs` (built CLI, scripted provider fixture, throwaway home). | Exit 0 for both runs; the enabled run's log carries `efficiency.catalog.selected`; the disabled run's log carries no `efficiency.*` event; home and workspace deleted. |
+| QA-18 | Regression: `npm run test:jev` plus `dist/test/decision-active`, `decision-engine`, `hardened-runtime`, `context-budget`, `usage-analytics`, `runtime-skills`, `config`, `graph-runner`, `cli`, `app-server`. | The same pre-existing failures as clean `origin/main` in this environment (listed under Gaps) and nothing new. |
+
+## Edge cases and potential bugs
+
+- **A trigger on a batch with no execution calls.** Triggers are evaluated only for batches the
+  post-tool path judges (execution calls present); a metadata-only batch prepares nothing to continue.
+- **Two triggers match.** Declaration order decides; only the first matching step runs, the rest are
+  candidates for the next round. Expected: never two automatic actions from one batch.
+- **The prepared tool is restorable but not advertised** (omitted by catalog selection). The step is
+  refused at `prepare_steps` time (catalog check) or rejected at execution (`catalog_mismatch`);
+  neither is counted as saved.
+- **Filtering a report whose failures are past the 24,000-character clip.** The shipped runtime drops
+  them (observed in the evaluation: `failureEvidenceVisible` 0 for the baseline); filtering keeps
+  them. A report the parser does not recognize still gets the clip. Fitting never drops a failure
+  heading or tears a block: what shrinks is whole blocks, and a report whose failure headings alone
+  exceed the excerpt bound is left unfiltered for the clip and compaction to handle.
+- **A tool that returns an error other than a non-zero exit** (denied, blocked, invalid arguments) is
+  never filtered, however large.
+- **Expansion of a reference after compaction.** The archive is invocation-scoped memory (plus the
+  run's artifact store when present); a reference survives compaction of the conversation. Expansion
+  pages are bounded to 8,000 characters and counted.
+- **Catalog selection when the saving is small.** Omission is applied only when it saves more than
+  the note plus the `restore_tools` schema (about 420 characters); otherwise the full catalog stays and
+  the event reports `skipped: no_benefit`.
+- **Adaptive routing with a tier naming an unlisted model.** The tier is treated as unconfigured and
+  the baseline is kept (`efficiency.routing.decided` reason `unconfigured`).
+- **A tier whose target already is the baseline model.** No model changes, but the attempt ran on
+  the tier's target: it counts as the tier that served, so its configured higher tier stays
+  reachable (`efficiency.routing.decided` reports `applied: false, served: true`).
+- **Escalation after a cancel or a deadline.** Not evidence; no escalation. After `repeatedToolFailure`
+  or `maxIterations` at most `maxEscalations` upward steps, within the same node budget.
+- **Provider continuation state across tiers.** An escalated attempt is a fresh conversation; no
+  `providerContinuation` or `reasoningContent` crosses models (the engine already drops them on a
+  model change).
+- **Memory selection when every entry is relevant.** Nothing is omitted and the block is unchanged.
+- **Suspected, not reproduced:** a very large search result (over 40 files) under filtering lists the
+  omitted file count but not their names; the model must narrow the search to see them.
+
+## Gaps and limitations
+
+- **Live behavior is unvalidated.** The evaluation is deterministic replay with scripted models; it
+  measures what the runtime sent, not how a model behaves with less context. `npm run eval:jev:live`
+  exists for a paired evaluation but was not run (no live provider in the QA environment).
+- **Tokens and cost are not measured.** Request sizes are serialized characters; scripted adapters
+  report no usage, so the deterministic evaluation still carries no cost figure. With a real provider
+  and configured prices the local accounting bills every measured request
+  (`models[].costMicroUsd` per model when all its requests were priced; a partially priced run
+  reports no total rather than a partial sum).
+- **Jev semantic selection of excerpts and of context candidates is not implemented**: the hosted
+  service accepts no new question ids and is not contracted to receive excerpts, so those paths stay
+  deterministic.
+- **Advertising `when` costs about 170 serialized characters per request** when no step uses it; the
+  `all` criterion allows 0.5% for this (criteria revision 3, documented in the file).
+- **Pre-existing failures on clean `origin/main` in this environment (Windows, no CodeGraph index,
+  no Graphiti):** 13 cases across `decision-active`, `decision-engine`, `decision-post-tool`
+  ("rejected prepared completion…"), `decision-preflight-execution` (skill discovery cases) and
+  `hardened-runtime` ("a live switch to a larger window…"). They fail identically before and after
+  this change, as do two in `app-server`: "completes a handshake and never leaks a provider secret"
+  (the engine advertises `run.setModel`, which the test's expected capability list lacks) and "desktop
+  refuses removal while another engine runs" (`Timed out waiting for run.failed`); both were rerun alone
+  on clean `origin/main` 573d7720 and on the branch with the same result.
+
+## Validation evidence
+
+Performed on October 31, 2026, Windows 11, Node 24.13.1, worktree `feat/jev-efficiency-opt` from
+`origin/main` 981807348a7466f9cae6c76277a49d85eaccc04e:
+
+- `npx tsc --noEmit -p tsconfig.json`: clean.
+- `npm run build`: succeeded.
+- New suites (`npm run test:jev` set, 36 new cases in six files plus the three existing decision
+  suites): all new cases pass.
+- Regression set (QA-18): the same 13 pre-existing failures as the baseline run on clean
+  `origin/main`, no new failure. `app-server`: the two pre-existing failures named above and nothing
+  else. `cli`: 31 tests, 25 pass, the same 6 failures on the branch build and on clean `origin/main` 573d7720 run the same way (load-dependent headless cases: typed handoff, Read Only + 409 retry, headless misuse, file workspace, fresh registry, SIGTERM cleanup). Both files leave a handle open after their last test on main and on this
+  branch, so a plain `node --test` run of them never exits; run them with `--test-force-exit`.
+- `npm run eval:jev`: every acceptance criterion passed (`tests/jev-efficiency/results/latest.md`).
+  Per-feature results, serialized characters: prepared continuations saved one of three model
+  requests on the greenfield workload (55,618 → 37,414); tool-result filtering cut the long-log
+  workload from 81,230 to 35,712 and made both failures visible where the baseline clip dropped them;
+  context selection omitted four of eight memory entries; catalog selection omitted two schemas on
+  the greenfield workload and deduplicated the skill catalog across the two-node handoff; adaptive
+  routing sent the mechanical task to the economy model and the security task to premium.
+- `node tests/jev-efficiency-cli-smoke.mjs`: exit 0. Enabled run: CLI exit 0, run log carried `efficiency.catalog.selected`, two scripted-provider requests (the health probe and the completion). Disabled run: CLI exit 0, no `efficiency.*` event. Both throwaway homes and workspaces were deleted.
+- `node --test scripts/ci-local.test.mjs`: 15 pass.
+
+Pending: the live paired evaluation (QA, with a configured provider), and `npm run ci:local` in a
+checkout with the desktop dependencies installed.
+
+Review round (codex findings, fixed and verified the same day, same environment):
+
+- **Routed checkpoints now continue.** A checkpoint stamped with the pair adaptive routing ran on
+  (plus the baseline it started from, `routedFrom`) is re-entered on that pair when the resumed
+  run recomputes the same baseline; a changed baseline restarts the visit as before, and an
+  unresolvable routed pair refuses the resume with an actionable error instead of dropping the
+  conversation. Verified by `test/graph-resume-seed.test.ts` (three new cases: continue on the
+  routed pair, restart under a changed baseline, refuse on an unresolvable pair) and by
+  `test/app-server.test.ts` "a routed checkpoint continues its visit on the routed pair…", which
+  drives the real `run.resume` path against a scripted provider and asserts the request names the
+  routed model and carries the prior conversation.
+- **Eval scripts resolve their checkout with `fileURLToPath`.** Both `scripts/jev-efficiency-eval.mjs`
+  and `scripts/jev-efficiency-live.mjs` decode the module URL instead of slicing `URL.pathname`,
+  so a checkout path containing spaces resolves to the directory that exists. Verified by running
+  `node scripts/jev-efficiency-eval.mjs` through a directory junction whose name contains a space
+  (`…\GitHub\Andorra Code JEV`): every acceptance criterion passed, exit 0. The old
+  conversion resolved a percent-encoded ghost path there and failed before any workload ran.
+- **Configured routing prices now reach the accounting.** `recordModelUse` prices each request
+  from the user's `adaptiveRouting.pricing` table (the production path in
+  `generateWithUsage`, and the health-probe tuples in both the CLI and the app-server), folding
+  the cost exactly as the telemetry aggregation expects: a model tuple reports
+  `costMicroUsd` only when every request in it was priced, and the exported record carries the
+  sum under the same completeness rule. The live evaluator reports each run's cost from the
+  summary's model tuples (null when any model went unpriced). Verified by
+  `test/runtime.test.ts` "configured routing prices turn a request's usage into a recorded
+  cost" and `test/telemetry-emitter.test.ts` "configured prices export a model cost…".
+- Groups re-run after the fixes: `npm run ci:local` — graph-studio and rust-shell pass; the only
+  failures in graph-editor (3 desktop cases) and telemetry (2 emitter recovery cases, 4 CLI
+  headless cases) are the pre-existing ones this plan's Gaps section names, rerun on the branch
+  baseline (stash, rebuild, rerun) with identical results. The full `npm run eval:jev` was rerun
+  after the fixes and every acceptance criterion passed again (`tests/jev-efficiency/results/latest.md`
+  refreshed).
+

package/tests/mock-graphiti.mjs

@@ -0,0 +1,106 @@
+#!/usr/bin/env node
+/**
+ * Minimal in-memory Graphiti REST stub for QA of Andorra persistent memory.
+ * Implements the routes Andorra uses: /healthcheck, /messages, /search,
+ * /episodes/:group, /group/:group (DELETE). Facts are derived 1:1 from
+ * episodes; an episode whose content says it supersedes an earlier title
+ * marks that earlier fact historical (invalid_at).
+ *
+ * Usage: node tests/mock-graphiti.mjs [port]   (default 8000)
+ */
+import { createServer } from 'node:http';
+import { randomUUID } from 'node:crypto';
+
+const port = Number(process.argv[2] ?? 8000);
+/** groupId -> { episodes: Map<uuid, episode>, facts: Map<uuid, fact> } */
+const groups = new Map();
+const group = id => {
+  if (!groups.has(id)) groups.set(id, { episodes: new Map(), facts: new Map() });
+  return groups.get(id);
+};
+const now = () => new Date().toISOString();
+
+function addEpisode(groupId, message) {
+  const g = group(groupId);
+  const id = message.uuid ?? randomUUID();
+  const episode = {
+    uuid: id, name: message.name ?? '', group_id: groupId, content: message.content ?? '',
+    source: 'message', source_description: message.source_description ?? '',
+    created_at: now(), valid_at: message.timestamp ?? now(),
+  };
+  g.episodes.set(id, episode);
+  let meta = {};
+  try { meta = JSON.parse(episode.source_description); } catch { /* foreign episode */ }
+  // Temporal stub: a superseding episode invalidates the fact whose title it names.
+  const supersede = episode.content.match(/supersedes and replaces the earlier fact: "([^"]+)"/);
+  if (supersede) {
+    for (const fact of g.facts.values()) {
+      if (fact._title === supersede[1] && fact.invalid_at === null) {
+        fact.invalid_at = now(); fact.expired_at = now();
+      }
+    }
+  }
+  const factId = randomUUID();
+  g.facts.set(factId, {
+    uuid: factId, name: episode.name || 'ANDORRA_MEMORY', fact: episode.content,
+    valid_at: episode.valid_at, invalid_at: null, created_at: episode.created_at, expired_at: null,
+    source_node_uuid: id, target_node_uuid: id, episodes: [id], _title: meta.title ?? '',
+  });
+  return episode;
+}
+
+function search(groupIds, query, maxFacts) {
+  const terms = query.toLowerCase().split(/\s+/).filter(w => w.length > 2);
+  const facts = [];
+  for (const id of groupIds) {
+    for (const fact of group(id).facts.values()) {
+      const text = fact.fact.toLowerCase();
+      const hits = terms.filter(term => text.includes(term)).length;
+      if (hits) facts.push({ fact, hits });
+    }
+  }
+  facts.sort((a, b) => b.hits - a.hits || b.fact.created_at.localeCompare(a.fact.created_at));
+  return facts.slice(0, maxFacts).map(({ fact }) => {
+    const { _title, ...rest } = fact;
+    return rest;
+  });
+}
+
+const readBody = req => new Promise(resolve => {
+  let body = '';
+  req.on('data', chunk => { body += chunk; });
+  req.on('end', () => resolve(body));
+});
+
+createServer(async (req, res) => {
+  const url = new URL(req.url, `http://localhost:${port}`);
+  const send = (status, value) => {
+    res.writeHead(status, { 'content-type': 'application/json' });
+    res.end(JSON.stringify(value));
+  };
+  try {
+    if (url.pathname === '/healthcheck') return send(200, { status: 'healthy' });
+    if (url.pathname === '/messages' && req.method === 'POST') {
+      const body = JSON.parse(await readBody(req));
+      for (const message of body.messages ?? []) addEpisode(body.group_id, message);
+      return send(202, { message: 'queued', success: true });
+    }
+    if (url.pathname === '/search' && req.method === 'POST') {
+      const body = JSON.parse(await readBody(req));
+      return send(200, { facts: search(body.group_ids ?? [], body.query ?? '', body.max_facts ?? 10) });
+    }
+    const episodes = url.pathname.match(/^\/episodes\/(.+)$/);
+    if (episodes && req.method === 'GET') {
+      const lastN = Number(url.searchParams.get('last_n') ?? 200);
+      return send(200, [...group(decodeURIComponent(episodes[1])).episodes.values()].slice(-lastN));
+    }
+    const deleteGroup = url.pathname.match(/^\/group\/(.+)$/);
+    if (deleteGroup && req.method === 'DELETE') {
+      groups.delete(decodeURIComponent(deleteGroup[1]));
+      return send(200, { message: 'Group deleted', success: true });
+    }
+    send(404, { error: 'not found' });
+  } catch (error) {
+    send(500, { error: String(error) });
+  }
+}).listen(port, () => console.log(`mock-graphiti listening on http://localhost:${port}`));

package/tests/permissions-sandbox-probe.mjs

@@ -0,0 +1,74 @@
+// Real Windows verification, including honest fail-closed behavior on unsupported Codex backends.
+import assert from 'node:assert/strict';
+import { mkdir, mkdtemp, readFile, rm, writeFile } from 'node:fs/promises';
+import { homedir } from 'node:os';
+import path from 'node:path';
+import { createTools } from '../dist/src/tools.js';
+import { resolveTrustedCodexExecutable } from '../dist/src/shell.js';
+
+if (process.platform !== 'win32') throw new Error('This probe verifies the Windows Codex backend only.');
+const base = path.join(homedir(), 'Documents');
+const root = await mkdtemp(path.join(base, 'AndorraPermissionProbe-'));
+assert.ok(!path.relative(base, root).startsWith('..'));
+const workspace = path.join(root, 'workspace');
+const config = path.join(root, 'config');
+const codexHome = path.join(workspace, 'custom-codex-home');
+const previous = { home: process.env.ANDORRACODE_HOME, codexHome: process.env.CODEX_HOME, path: process.env.PATH };
+try {
+process.env.ANDORRACODE_HOME = config;
+await Promise.all([mkdir(workspace), mkdir(config)]);
+await Promise.all([mkdir(path.join(workspace, '.git')), mkdir(path.join(workspace, '.andorra')), mkdir(codexHome), mkdir(path.join(workspace, 'shims'))]);
+process.env.CODEX_HOME = codexHome;
+await writeFile(path.join(root, 'host-read-sentinel.txt'), 'owned-host-sentinel');
+await writeFile(path.join(workspace, 'shims', 'codex.exe'), 'invalid untrusted fixture launcher');
+process.env.PATH = path.join(workspace, 'shims') + path.delimiter + previous.path;
+const trusted = await resolveTrustedCodexExecutable(workspace);
+const relativeLauncher = path.relative(workspace, trusted);
+assert.ok(path.isAbsolute(relativeLauncher) || relativeLauncher === '..' || relativeLauncher.startsWith('..' + path.sep));
+const script = path.join(workspace, 'probe.cjs');
+await writeFile(script, `
+const fs = require('node:fs'); const path = require('node:path'); const https = require('node:https');
+const root = process.argv[2]; const workspace = process.argv[3]; const mode = process.argv[4];
+try { console.log('hostRead=' + (fs.readFileSync(path.join(root, 'host-read-sentinel.txt'), 'utf8') === 'owned-host-sentinel' ? 'READ_OK' : 'INVALID')); } catch (error) { console.log('hostRead=' + error.code); }
+for (const [label, target] of [['inside', path.join(workspace, mode + '-inside.txt')], ['outside', path.join(root, mode + '-outside.txt')], ['protected', path.join(workspace, '.andorra', mode + '-protected.txt')], ['config', path.join(root, 'config', mode + '-config.txt')], ['codexHome', path.join(workspace, 'custom-codex-home', mode + '-codex.txt')]]) {
+  try { fs.writeFileSync(target, 'probe'); console.log(label + '=WRITE_OK'); } catch (error) { console.log(label + '=' + error.code); }
+}
+const request = https.get('https://example.com/', response => { console.log('network=HTTP_' + response.statusCode); response.resume(); });
+request.on('error', error => console.log('network=' + error.code));
+request.setTimeout(10000, () => request.destroy(new Error('network timeout')));
+`);
+const shell = createTools().find(tool => tool.name === 'PowerShell');
+const context = { workspace, signal: new AbortController().signal, yolo: false, approve: async () => { throw new Error('Default probe must not request escalation'); } };
+const quote = value => `'${value.replace(/'/g, "''")}'`;
+const commandFor = label => `& ${quote(process.execPath)} ${quote(script)} ${quote(root)} ${quote(workspace)} ${label}`;
+let unavailable = 0;
+  for (const mode of ['ask-for-approval', 'approve-for-me', 'read-only']) {
+    try {
+      const output = await shell.execute({ command: commandFor(mode), timeout: 30000 }, { ...context, permissions: mode });
+      console.log(`\n${mode}\n${output}`);
+      assert.doesNotMatch(output, /hostRead=READ_OK/);
+      assert.match(output, /hostRead=(EPERM|EACCES)/);
+      assert.match(output, mode === 'read-only' ? /inside=(EPERM|EACCES)/ : /inside=WRITE_OK/);
+      for (const label of ['outside', 'protected', 'config', 'codexHome']) assert.match(output, new RegExp(label + '=(EPERM|EACCES)'));
+      assert.match(output, /network=EACCES/);
+    } catch (error) {
+      assert.match(String(error), /Restricted shell is unavailable:.*cannot enforce workspace-only read access/);
+      unavailable++;
+      await assert.rejects(readFile(path.join(workspace, mode + '-inside.txt')), /ENOENT/);
+      console.log(`\n${mode}: UNAVAILABLE; wrapper refused before the command ran (no host fallback).`);
+    }
+    for (const target of [path.join(root, mode + '-outside.txt'), path.join(workspace, '.andorra', mode + '-protected.txt'), path.join(config, mode + '-config.txt'), path.join(codexHome, mode + '-codex.txt')]) await assert.rejects(readFile(target), /ENOENT/);
+  }
+  for (const [label, options, extra] of [['full-access', { permissions: 'full-access' }, {}], ['approved-escalation', { permissions: 'read-only', approve: async () => true }, { sandbox_permissions: 'require_escalated', justification: 'Verify host execution only after approval using owned fixtures.' }]]) {
+    const events = [];
+    const output = await shell.execute({ command: commandFor(label), timeout: 30000, ...extra }, { ...context, ...options, onApprovalEvent: (type, data) => { events.push({ type, data }); } });
+    console.log(`\n${label}\n${output}`);
+    assert.match(output, /hostRead=READ_OK/); assert.match(output, /inside=WRITE_OK/); assert.match(output, /outside=WRITE_OK/); assert.match(output, /codexHome=WRITE_OK/); assert.match(output, /network=HTTP_200/);
+    assert.equal(events.length, 2); assert.equal(events[1].data.approved, true);
+  }
+  assert.equal(await readFile(path.join(root, 'host-read-sentinel.txt'), 'utf8'), 'owned-host-sentinel');
+  console.log(unavailable ? `\nFAIL-CLOSED VERIFIED: ${unavailable} restricted modes unavailable on this Codex backend. Full Access/approved escalation controls and audit events pass. Restricted command usability is NOT verified.` : '\nPASS: workspace-only host reads, protected writes, network isolation, trusted launcher, and positive controls verified.');
+} finally {
+  for (const [key, value] of [['ANDORRACODE_HOME', previous.home], ['CODEX_HOME', previous.codexHome], ['PATH', previous.path]]) { if (value === undefined) delete process.env[key]; else process.env[key] = value; }
+  await rm(root, { recursive: true, force: true });
+}