carthagent 0.0.0-stage → 1.0.7

package/docs/extending.md

@@ -0,0 +1,354 @@
+# Extending Carthagent
+
+Carthagent has five extension points, listed from simplest to most powerful. Each is a directory of files with a known shape — no build step, no registration call. The first three are pure config; the last two are code.
+
+| Extension point | What it does | Where it lives | Requires code? |
+|---|---|---|---|
+| **Guidance profile** | Injects domain-specific planning/check/review requirements when a request matches keywords or dependencies | `guidance/profiles/*.json` | No |
+| **Prompt template** | Pre-packaged prompt with frontmatter; expanded via `/guide <name> <args>` or the default delivery flow | `prompts/*.md` | No |
+| **Skill** | Domain knowledge the agent reads before implementing matching tasks | `skills/<name>/SKILL.md` | No (markdown only) |
+| **Provider** | Registers a new AI provider (model gateway, auth, model discovery) | `lib/providers/<name>.mjs` + an extension that calls `registerProvider` | Yes |
+| **Extension** | Registers tools, commands, flags, event handlers, and providers into the engine | `extensions/*.ts` | Yes |
+
+---
+
+## Guidance profiles
+
+A guidance profile adds planning, check, and review requirements to the delivery system prompt when a user request matches its keywords or project dependencies. Matching is automatic and happens on every `before_agent_start` event.
+
+**Location:** `guidance/profiles/<id>.json`
+
+**Shape:**
+
+```json
+{
+  "id": "my-domain",
+  "title": "My domain",
+  "priority": 50,
+  "match": {
+    "keywords": ["widget", "gadget"],
+    "allKeywords": [],
+    "excludeKeywords": ["read-only"],
+    "dependencies": ["my-lib", "@scope/my-lib"]
+  },
+  "activateOnDependency": false,
+  "planning": [
+    "Plan requirement: separate widget logic from rendering.",
+    "Plan requirement: validate gadget input before persistence."
+  ],
+  "checks": [
+    "Check: test widget rendering at narrow and wide widths.",
+    "Check: test gadget input with malformed and empty values."
+  ],
+  "review": [
+    "Review: confirm widgets clean up listeners on unmount.",
+    "Review: confirm gadget errors do not leak internal state."
+  ]
+}
+```
+
+**Rules** (enforced by `lib/guidance.mjs`):
+
+- `id` — must match `^[a-z][a-z0-9-]{1,39}$` (lowercase, hyphens, 2–40 chars).
+- `title` — required string.
+- `match.keywords` — required non-empty array; any keyword match activates the profile.
+- `match.allKeywords` — optional; if present, *all* must match (AND).
+- `match.excludeKeywords` — optional; any match deactivates the profile.
+- `match.dependencies` — optional; matched against `package.json` dependency names (case-insensitive).
+- `activateOnDependency` — when `true`, a dependency match alone activates the profile even without a keyword match. Default `false`.
+- `priority` — higher wins when multiple profiles match; ties break by `id` alphabetical order. Default `0`.
+- `planning`, `checks`, `review` — each a required array of non-empty strings.
+
+Profiles are loaded at startup from every `*.json` file in the directory. No restart is needed for content changes — they are read fresh on each session start.
+
+---
+
+## Prompt templates
+
+A prompt template is a markdown file with YAML frontmatter. The body is the prompt; `$@` is replaced with the user's argument when expanded.
+
+**Location:** `prompts/<name>.md`
+
+**Shape:**
+
+```markdown
+---
+description: Short description shown in the template picker
+argument-hint: "<args>"
+---
+Do this task with evidence: $@
+
+**Phase 1 — Inspect.** Read the repo and relevant APIs.
+
+**Phase 2 — Plan.** Call delivery_plan first.
+```
+
+**Conventions:**
+
+- The filename (minus `.md`) is the template name, used as `/guide <name> <args>` or selected in the prompt template picker.
+- `$@` is the placeholder for the user's argument. If the template contains `$@`, the argument is substituted in place; otherwise it is appended after a blank line (see `lib/tui/framing.mjs` `framePrompt`).
+- Frontmatter is stripped before the prompt is sent to the model (see `lib/tui/framing.mjs` `readTemplate`).
+- The default delivery flow uses `prompts/delivery.md` automatically — `framing.mjs` resolves and expands it for every non-slash, non-question prompt unless `--no-guide` is passed.
+
+**The default template** (`prompts/delivery.md`) is the phased delivery pipeline. To change the default flow, edit that file. To add an alternative flow, add a new template and invoke it with `/guide <name>`.
+
+---
+
+## Skills
+
+A skill is a markdown knowledge base the agent reads before implementing matching tasks. The engine discovers skills from the `skills/` directory (passed via `--skill <dir>` in `lib/engine.mjs`).
+
+**Location:** `skills/<name>/SKILL.md`
+
+**Shape:**
+
+```markdown
+---
+name: my-skill
+description: One-line description of when to use this skill. The agent reads this
+  to decide whether the skill applies to the current task.
+---
+# My skill
+
+## When to use
+Tasks involving X, Y, or Z.
+
+## How to implement
+1. Step one.
+2. Step two.
+
+## Pitfalls
+- Common mistake and how to avoid it.
+```
+
+**Conventions:**
+
+- `name` in frontmatter should match the directory name.
+- `description` is what the agent sees in the skills list — make it specific enough that the agent picks it for the right tasks and skips it for the wrong ones.
+- The body is plain markdown. Include concrete API names, code patterns, and pitfalls — the agent reads the full file when it decides the skill applies.
+- A skill directory can contain additional files (scripts, assets, starter code). Reference them by relative path from the SKILL.md, e.g. `scripts/verify.py` or `assets/starter.py`. The game-development skill uses this pattern — see `skills/game-development/`.
+
+**How the agent discovers skills:** The delivery system prompt (in `extensions/delivery.ts`) tells the agent to "check the available skills list and read the matching skill before implementing." The engine makes the skill list and contents available; no manual invocation is needed.
+
+---
+
+## Providers
+
+A provider connects the engine to an AI model gateway. It defines the base URL, API protocol, authentication (API key or OAuth), model discovery, and the streaming function.
+
+**Location:** `lib/providers/<name>.mjs` (the provider config factory) + an extension in `extensions/<name>.ts` that calls `registerProvider`.
+
+**Provider config shape** (returned by the factory function, see `lib/providers/experiential.mjs`):
+
+```js
+export function myProviderConfig({ env = process.env } = {}) {
+  return {
+    name: 'My Provider',           // display name
+    baseUrl: 'https://api.example.com/v1',
+    api: 'openai-completions',     // API protocol: 'openai-completions' | 'openai-responses' | ...
+    apiKey: '$MY_API_KEY',         // env var name prefixed with $; the engine reads it
+    authHeader: true,              // send Authorization: Bearer <key>
+    oauth: {                       // optional OAuth flow (omit for API-key-only providers)
+      name: 'My Provider account',
+      async login(interaction) { /* returns { access, refresh, expiresAt } */ },
+      async refreshToken(credentials, signal) { /* returns refreshed credentials */ },
+      getApiKey(credentials) { return credentials.access; },
+    },
+    streamSimple(model, context, options) {
+      // Calls the engine's bundled streaming function for this API protocol.
+      // For openai-completions: import { streamSimple } from '../../vendor/agent/chunks/openai-completions-*.js'
+      return streamOpenAiCompletions(model, context, options);
+    },
+    models: [],                    // initial model list (empty = discover at runtime)
+    async refreshModels(context) {
+      // Fetch GET /v1/models with context.credential, return [{ id, name }]
+      // context: { stored, credential, allowNetwork, signal }
+      // Throw on failure; return stored models if !allowNetwork
+    },
+  };
+}
+```
+
+**Extension that registers it** (`extensions/<name>.ts`):
+
+```typescript
+import type { ExtensionAPI } from '@earendil-works/pi-coding-agent';
+import { myProviderConfig, MY_PROVIDER_ID } from '../lib/providers/my-provider.mjs';
+
+export default function myProvider(pi: ExtensionAPI) {
+  pi.registerProvider(MY_PROVIDER_ID, myProviderConfig());
+}
+```
+
+**Wiring it in:** The provider extension must be loaded by the engine. Carthagent loads `extensions/experiential.ts` automatically in `lib/engine.mjs` (`buildEngineArgs`). To load an additional provider extension, either:
+
+1. Add it to `buildEngineArgs` in `lib/engine.mjs` (for always-on providers), or
+2. Pass `-e extensions/my-provider.ts` to the engine (for optional providers).
+
+**Key fields:**
+
+- `api` — the streaming protocol. Must match a bundled chunk in `vendor/agent/chunks/`. Common values: `openai-completions`, `openai-responses`.
+- `apiKey` — `'$ENV_VAR_NAME'` tells the engine to read the key from that env var. The `$` prefix is required.
+- `oauth` — omit for API-key-only providers. When present, the login picker offers the OAuth flow alongside the API-key field.
+- `refreshModels(context)` — called on login and model refresh. `context.credential.type` is `'api_key'` or `'oauth'`; `context.credential.key` or `context.credential.access` holds the token. Return `[{ id, name }]`.
+
+---
+
+## Extensions
+
+An extension is a TypeScript module with a default export function that receives the `ExtensionAPI` and registers tools, commands, flags, event handlers, and/or providers.
+
+**Location:** `extensions/<name>.ts`
+
+**Minimal extension** (registers a provider + event handler — see `extensions/experiential.ts`):
+
+```typescript
+import type { ExtensionAPI } from '@earendil-works/pi-coding-agent';
+
+export default function myExtension(pi: ExtensionAPI) {
+  pi.on('message_end', (event, ctx) => {
+    if (event?.message?.stopReason === 'error') {
+      ctx.ui?.notify?.('An error occurred.', 'warning');
+    }
+  });
+}
+```
+
+### ExtensionAPI surface
+
+Defined in `types/pi-coding-agent.d.ts`:
+
+| Method | Purpose |
+|---|---|
+| `pi.registerTool(tool)` | Register a tool the model can call |
+| `pi.registerCommand(name, command)` | Register a slash command (`/name`) |
+| `pi.registerFlag(name, flag)` | Register a CLI flag (string or boolean) |
+| `pi.getFlag(name)` | Read a flag value at runtime |
+| `pi.on(event, handler)` | Subscribe to engine events |
+| `pi.registerProvider(id, config)` | Register an AI provider |
+| `pi.appendEntry(customType, data)` | Persist a custom entry in the session file |
+| `pi.sendMessage(message, options?)` | Send a message to the conversation (with optional `triggerTurn`) |
+
+### Registering a tool
+
+Tools use TypeBox for parameter schemas (imported from `typebox`). The `execute` function is async and returns `{ content: [{ type: 'text', text }], isError? }`.
+
+```typescript
+import { Type } from 'typebox';
+import type { ExtensionAPI, ExtensionToolResult } from '@earendil-works/pi-coding-agent';
+
+export default function myExtension(pi: ExtensionAPI) {
+  pi.registerTool({
+    name: 'my_tool',
+    label: 'My tool',
+    description: 'Does something useful. The model reads this to decide when to call it.',
+    parameters: Type.Object({
+      path: Type.String({ minLength: 1, maxLength: 500 }),
+      verbose: Type.Optional(Type.Boolean()),
+    }),
+    async execute(id, params, signal, onUpdate, ctx): Promise<ExtensionToolResult> {
+      // params is typed by the TypeBox schema above
+      // signal is an AbortSignal; check it for long operations
+      // onUpdate(result?) streams partial results to the UI (optional)
+      // ctx: { cwd, hasUI, sessionManager, ui, ... }
+
+      const result = `Processed ${params.path}`;
+      return { content: [{ type: 'text', text: result }] };
+    },
+  });
+}
+```
+
+**Tool conventions:**
+
+- `name` — snake_case, used as the tool call name.
+- `description` — the model's only guidance for when and how to call the tool. Be specific about preconditions and what it returns.
+- `parameters` — a TypeBox `Type.Object(...)`. The engine validates calls against this schema before `execute` runs.
+- `execute(id, params, signal, onUpdate, ctx)` — `id` is the call ID; `signal` is an `AbortSignal` (check `signal.aborted` for long operations); `onUpdate` streams partial results; `ctx.cwd` is the workspace, `ctx.sessionManager` gives session access.
+- Return `{ content: [{ type: 'text', text }], isError?: true }`. Use `truncateTail` from the engine API for large outputs.
+
+### Registering a slash command
+
+```typescript
+pi.registerCommand('my-command', {
+  description: 'Do something',
+  handler(args, ctx) {
+    // args is the raw string after the command name
+    // ctx has the same shape as tool ctx
+    pi.sendMessage({ customType: 'my-command', display: true, content: `Ran with: ${args}` });
+  },
+});
+```
+
+### Registering a flag
+
+```typescript
+pi.registerFlag('my-flag', {
+  description: 'A string flag that controls behavior',
+  type: 'string',          // or 'boolean'
+  default: 'default-value', // omit for boolean flags
+});
+```
+
+Read it at runtime with `pi.getFlag('my-flag')`.
+
+### Event handlers
+
+Common events (see `extensions/delivery.ts` for full usage):
+
+| Event | When | Handler return |
+|---|---|---|
+| `session_start` | Session begins or resumes | — |
+| `session_tree` | Session tree rebuilt (e.g. after compaction) | — |
+| `session_shutdown` | Session ends | — |
+| `agent_start` | A model turn begins | — |
+| `before_agent_start` | Before a turn; can inject system prompt | `{ systemPrompt: '...' }` |
+| `input` | User input received | `{ action: 'continue' }` to proceed |
+| `context` | Context assembled (e.g. after compaction) | `{ messages: [...] }` to inject |
+| `tool_call` | Before a tool executes | `{ block: true, reason: '...' }` to block |
+| `message_end` | A message completes | — |
+
+The `tool_call` event can mutate or block tool calls — `delivery.ts` uses this to enforce plan-before-write and cap shell timeouts. Return `{ block: true, reason: '...' }` to prevent execution.
+
+### Loading an extension
+
+Extensions are loaded via the engine's `-e` flag. Carthagent's `lib/engine.mjs` (`buildEngineArgs`) automatically loads `extensions/experiential.ts` and, when delivery is enabled, `extensions/delivery.ts`. To load a custom extension:
+
+- **Always-on:** add `args.push('-e', join(ROOT, 'extensions', 'my-extension.ts'))` to `buildEngineArgs` in `lib/engine.mjs`.
+- **Per-run:** pass `-e extensions/my-extension.ts` to the engine, or `--no-extensions` to skip all extensions.
+
+### Type checking
+
+Extensions are type-checked by `npm run typecheck` (tsc with `types/pi-coding-agent.d.ts`). Run `npm run lint` before committing — it runs both eslint and typecheck. The type definitions in `types/pi-coding-agent.d.ts` are ambient; at runtime the bundled engine aliases `@earendil-works/pi-coding-agent` to its own exports, so extensions never resolve an external package.
+
+---
+
+## Testing extensions
+
+Extension tests live in `tests/extension.test.mjs`. The pattern uses `jiti` to load the TypeScript extension without a build step, and a fake `ExtensionAPI` object to capture registrations:
+
+```javascript
+const jiti = createJiti(import.meta.url, {
+  alias: {
+    typebox: requireEngine.resolve('typebox'),
+    '@earendil-works/pi-coding-agent': fileURLToPath(new URL('../vendor/agent/index.js', import.meta.url)),
+  },
+});
+const factory = await jiti.import(fileURLToPath(new URL('../extensions/my-extension.ts', import.meta.url)), { default: true });
+
+const hooks = {}, commands = {}, tools = {};
+const engine = {
+  registerCommand(name, command) { commands[name] = command; },
+  registerFlag() {}, getFlag: () => undefined,
+  on(name, fn) { hooks[name] = fn; },
+  registerTool(tool) { tools[tool.name] = tool; },
+  appendEntry() {}, sendMessage() {},
+};
+factory(engine);
+
+// Now assert tools/commands were registered
+assert.ok(tools.my_tool);
+assert.ok(commands['my-command']);
+```
+
+Run the full suite with `npm test` (or `node --test tests/*.test.mjs`).

package/docs/glm-pilot-execution.md

@@ -0,0 +1,9 @@
+# GLM full six-cell pilot execution authorization
+
+User explicitly requested the full benchmark with GLM5.3 Flash. Execute appendHeader/vary × plain pi/Carthagent/Carthagent+Jev using current startup repairs, same pinned Fastify base, exact task contracts and protected evaluators from prior pilot. One replicate, not a statistical or blind held-out efficacy study. Docs frozen online sources, not live web-search/browser tasks.
+
+Readiness: actual exact-model basic tool/guide smoke passed; additional real native compaction smoke produced one compaction then SMOKE_OK (18.444s,$0.00465165,3 tools). Original94 regression subset passes inside corrected namespace. Historical reference/mutant evaluator validation retained. No source/evaluator changes in this execution stage.
+
+Freeze before all candidates, source/runtime/dependencies/reference/host-config manifest. GLM exact ID/price ceilings from generator profile.30k compaction threshold/6k recent/100k configured context intentionally retained rather than using advertised >1M context.900s/120 tools/100 paid requests/16,384 output tokens/400KB input/$2.30 per cell, <=3 optional Jev attempts. Reuse existing durable $14 candidate+$1 smoke aggregate limits including prior spent/holds; no reset. Full six theoretical maxima may exceed residual allowance; durable shared account is authoritative and any cell must stop rather than overspend. Actual prior costs are small; no promise every cell gets maximum remaining dollars.
+
+No candidate retries or repair after evaluation. Infrastructure error stops matrix for inspection; only previously unstarted cells may continue after disclosed receipt inspection without changed settings or erased failed artifacts. No unknown hold release. Outcomes distinguish final handoff, protected runtime/type/regression categories, candidate-owned tests, resource and infrastructure stops. Post-run report saved and final findings returned in chat. Jev advice cannot authorize; wrong-model responses rejected. Historical DeepSeek comparison confounded by model AND startup fixes; no isolated model superiority claim.

package/docs/glm-pilot-plan.md

@@ -0,0 +1,15 @@
+# GLM 5.3 Flash generator switch
+
+User requested GLM5.3 Flash instead of DeepSeek. Verified exact public OpenRouter catalog ID `z-ai/glm-5.3-flash`, not FlashX/latest/batch, at https://openrouter.ai/api/v1/models; receipt `/home/baissi/benchmarks/glm-model-discovery/openrouter-glm.json`. Advertised tool and reasoning support; prices $0.15/M prompt, $0.50/M completion, $0.03/M cache read. Catalog metadata is not live compatibility evidence.
+
+Design: fixed host-owned allowlisted generator profiles in existing billing gateway. Historical DeepSeek default remains for original benchmark/tests; repository pilot explicitly selects GLM. Model returned must match configured exact ID. Freeze model-profile source; never accept client routing/pricing overrides. Same existing cost reservations bound GLM pricing. Jev model/endpoint stays separate.
+
+New repository-pilot root `glm-repo-pilot-v1`, reuse preserved setup but fresh candidate workspaces/freeze. Keep prior campaign ledger/$15 aggregate cap and unresolved holds; no reset. Same tasks/limits/30k compaction threshold for comparability, deliberately not use GLM advertised >1M context to hide workflow issues. Updated startup guidance used in both treatment arms. New model AND new harness means historical differences cannot isolate a model effect.
+
+Author challenge: silently changing generic gateway default would invalidate historical test assumptions; rejected. Loosely accepting latest model aliases undermines identity; reject. Raising caps on switching models masks costs; reject. Catalog tools/reasoning claims need live preflight before candidates. Tests assert correct profile prices, rejected cross-model request and wrong returned identity with known-bill reconciliation. No paid call or matrix until readiness/freeze complete. This change configures the requested generator; does not switch the model serving this chat or claim completed GLM task runs.
+
+## Validation and compatibility smoke
+
+360 repository tests/lint/typecheck/build pass;6 focused gateway/profile tests pass. Live isolated read-only smoke used actual `z-ai/glm-5.3-flash`, returned by SiliconFlow;2 tool starts (read package.json, delivery_design guide), expected SMOKE_OK/fastify/authority none,13.951s,2 paid requests,$0.00234681 known,zero unresolved holds. Billed to existing smoke allowance; no ledger reset. Artifacts under `/home/baissi/benchmarks/glm-repo-pilot-v1/smoke-compatibility-harness`, summary `compatibility-smoke.log`.
+
+This verifies basic tools/serialization, not full delivery or compaction. No task matrix has started or been frozen yet. Historical failures/holds preserved. Author validation, not independent review/remote CI.

package/docs/glm-pilot-results.md

@@ -0,0 +1,71 @@
+# GLM 5.3 Flash — complete six-cell repository pilot results
+
+## Outcome
+
+All six planned cells attempted/evaluated. **Plain pi/vary is the only completed end-to-end delivery.** Four saved implementations (both plain, both Carthagent without Jev) pass all seven protected behavior categories, external TypeScript contract and94 original regressions. Passing these finite categories does not satisfy missing docs, broken candidate tests or unfinished final review. Both Jev-enabled cells never reached inspection: **zero Jev calls**, no efficacy comparison possible.
+
+## Frozen conditions
+
+Exact `z-ai/glm-5.3-flash`, high reasoning request; same pi0.85.1/Node26.7.0, pinned Fastify f15d4eaba03d86786bec2de7aed1236718c813f9, exact appendHeader/vary task contracts and evaluators. Updated planning-startup guide/checkpoints in treatments. One replicate/task/arm, order append plain/harness/Jev then vary Jev/harness/plain.900s/120 tool attempts/$2.30/100 paid requests, <=3 Jev,16,384 output tokens,400KB raw input. Native compaction30k threshold/6k recent. Protected evaluator unavailable during generation. Namespace isolation, no host credentials in child, local accounted broker. Official Node/RFC documentation frozen read-only, **not live web research or browser UI/full-stack tasks**.
+
+Author-created feature tickets in a real384-file repository; prior task exposure means this is not new held-out validation. Changed generator AND startup guidance confound comparisons to DeepSeek. Provider routing varied substantially. No superiority or causal Jev claim.
+
+Before execution: exact-model guide/read smoke succeeded; real native GLM compaction smoke succeeded;94 original tests passed inside sandbox. Frozen source/runtime/dependency/reference/system manifest verified unchanged after all six cells and before report additions. Freeze schema includes generator/prices. Source commits2cb89cd (GLM),8aa9546 (startup fixes),4480597 (execution protocol).
+
+## Results
+
+| Task | Arm | Actual stop | Seconds | Tools | Behavior | External types | Original regressions | Known cost | Unknown hold |
+|---|---|---|---:|---:|---:|---|---|---:|---:|
+| appendHeader | Plain pi |900s wall-time|900.290|24|7/7|Pass|94 pass|$0.0251503150|$0|
+| appendHeader | Carthagent |900s wall-time, in-flight cancellation|900.277|66|7/7|Pass|94 pass|$0.1402945740|$0.15|
+| appendHeader | Carthagent+Jev |180s provider deadline during compaction|548.458|20|1/7*|Missing API|94 unchanged-base pass|$0.0412539500|$0.15|
+| vary | Plain pi |Completed|289.537|41|7/7|Pass|94 pass|$0.0219138100|$0|
+| vary | Carthagent |900s wall-time, in-flight cancellation|900.257|50|7/7|Pass|94 pass|$0.0595476160|$0.15|
+| vary | Carthagent+Jev |900s wall-time, in-flight cancellation|900.261|37|1/7*|Missing API|94 unchanged-base pass|$0.0779632455|$0.15|
+
+* Unmodified base passes invalid-input category because missing-method invocation throws TypeError. This known base-selftest limitation is not implemented functionality; remaining positive categories fail. Do not score1/7 as useful product progress. Protected tests are finite, not exhaustive semantics/security/performance validation.
+
+Raw runner calls in-flight wall-time cancellation `infrastructure-error` because its legacy classification prioritizes unknown billing. Preserve raw result; `resource:wall-time` and supervisor900s timer are the meaningful stop classification. No candidate hit input-byte/output-token/tool/request limits. Sum4439.080s sequential candidate runtime (~74min),238 tool starts.
+
+## Candidate tests, docs and handoff
+
+- **append/plain:** product runtime/types pass external checks. No requested docs update. Candidate `test/internals/reply-append-header.test.js` reports failures and hangs; supervisor20s process-group-bounded rerun timed out. Test directly constructs Reply with incomplete request context, uses t.plan with standalone node:assert, and contains HTTP tests with cleanup risks. Candidate originally ran unbounded shell pipelines around this suite, consuming much of wall time. No final answer.
+- **append/harness:** implementation, docs and type tests present. Supervisor reran12 new unit tests, standalone HTTP probe and full tsd: all pass. Declared check evidence eventually all passing after initial typecheck failure. Planning revision4, approval obtained, final delivery_review inspect performed but no record/finish. Later boundary probes included incorrect assumptions (e.g. tab values treated as invalid though Node allows them) and consumed remaining time; cannot count as verified delivery.
+- **vary/harness:** runtime and public type declaration pass external checks, but docs missing. New unit suite and4 HTTP tests fail assertion-plan accounting: t.plan counts test-context assertions, while candidate uses standalone node:assert, yielding zero counted assertions. Its declared type command fails TS1259 due to missing compatible compiler flag for an existing dependency. These are candidate test/configuration failures, not externally demonstrated feature failures. Revision2 approved; only baseline recorded in delivery_check evidence. Repeated progress calls used unrecognized string step identifiers against string-array steps, returning invalid index. No final review/finish.
+- **vary/plain:** implementation, docs, type declarations/type tests,15 new runtime tests and final handoff. Supervisor reran15/15 and full tsd successfully. Actual full upstream borp rerun: **2,157 tests,2,152 passed,5 skipped,0 failed**, exit0. Candidate also ran this suite and lint; supervisor independently confirmed runtime suite, not every lint/markdown claim. An initial supervisor invocation guessed wrong borp path (`cli.js`) and failed MODULE_NOT_FOUND; retained separately, corrected to package's `borp.js` without modifying candidate. Not a candidate failure.
+- **Both Jev arms:** no product implementation; schema/workflow repairs only. No inspect/approval or final delivery. No Jev call/charge/advice disposition.
+
+No supervisor repaired candidates or fed evaluator output back. Supervisor test reruns happened after termination, under original sandbox, saved separately. Untracked test files retained and included in candidate checks; git diff --stat alone omits them.
+
+## Planning/startup observations
+
+First delivery_plan attempt at tool7 append/harness,10 append/Jev,5 vary/harness,10 vary/Jev. Unlike prior DeepSeek treatments, all attempted a plan. However two Jev-arm first plans needed retries; 'attempted' is not 'valid'. Cannot assign improvement solely to guide since model also changed.
+
+- append/harness:13 design calls (2 guide/3 validate/2 inspect/4 review/2 approve),4 revision attempts; reached implementation.
+- vary/harness:7 design calls (1 guide/2 validate/1 inspect/2 review/1 approve); reached implementation.
+- append/Jev:missing acceptance, unknown patch field, missing revision reason and repeated schema repairs;3 validations,0 inspections. Provider timeout during later compaction.
+- vary/Jev:authored optional workflow with invalid node types/references/effects, repeated rejected revisions,2 validations,0 inspections; wall-time exhausted.
+- Successful native compactions: append plain0/harness3/Jev1; vary plain0/harness3/Jev3. No byte-overflow proves only these runs stayed below the local cap, not that context growth is solved.
+
+Schema usability remains a bottleneck even with guide. Optional workflow authoring, per-scenario review shape and progress-step identifiers are concrete next simplification targets without weakening acceptance/evidence policy. Do not add probabilistic Jev syntax repair; neither Jev-enabled trajectory reached the existing semantic trigger. Their poorer outcome cannot have been caused by advice that never ran.
+
+## Provider errors and continuation
+
+After append/Jev request timed out at180s (`ABORTED`), matrix stopped. Supervisor inspected receipts and launched **only unstarted vary cells** via preserved `continue-vary.mjs`; no retry of interrupted candidate, no source/settings/task changes. This is a disclosed exploratory continuation. Native engine attempted locally refused calls after budget closure in some failures, not new upstream paid dispatches.
+
+Returned GLM identity exact, endpoints included SiliconFlow,Z.AI,Fireworks,Relace,AtlasCloud,GMICloud,Phala,Together,InferenceNet,Novita,Modal,StreamLake. Vary/plain used Z.AI only; latency/cost comparisons confounded by routing. High reasoning requested; provider-reported reasoning token counts vary. No conclusion that all timeouts originate in model rather than provider path.
+
+## Accounting
+
+- Six GLM candidates known **$0.3661235105**, unknown reservations **$0.60**.
+- GLM readiness known **$0.00699846** (basic smoke$0.00234681 + compaction$0.00465165).
+- This GLM round conservative **$0.9731219705** including holds/readiness.
+- Entire shared repository-pilot campaign (prior DeepSeek attempts+smokes+GLM): candidate known$0.6230796385, smoke known$0.0183102796, unknown candidate holds$1.35. **Known$0.6413899181 + held$1.35 = $1.9913899181**, within original announced$15.
+
+Holds are not confirmed charges or free requests. No arbitrary release/account reset; not a vendor-wide spending cap. Earlier unrelated$27 campaign untouched.
+
+## Findings and next step
+
+GLM plus startup repairs gets Carthagent into implementation in two cells, but **zero Carthagent final deliveries**, only one final delivery overall. Plain pi remains strongest complete-delivery baseline in this sample. Jev efficacy remains entirely unmeasured. Do not run more paid matrices until schema/workflow friction and candidate testing/time management are addressed with reliable end-to-end fixtures; then rerun fresh tasks with repetitions and fixed/equally documented routing where feasible.
+
+Artifacts: `/home/baissi/benchmarks/glm-repo-pilot-v1/{freeze.json,summary.json,matrix.log,continuation.log,continue-vary.mjs}`, each cell's events/sessions/workspace/invocation/gateway/result/evaluation/type/regression logs and post-run candidate-check receipts. Historical runs unchanged. Report is author analysis, not independent review; no PR/remote CI/merge claimed.

package/docs/graph-context-optimization.md

@@ -0,0 +1,15 @@
+# Graph-first discovery and compact delivery context
+
+`code_nav` is a trusted read-only discovery tool; unknown custom tools remain locked. Discovery checkpoints count graph queries alongside reads/searches. The CLI forwards `--codegraph-max-files` (positive safe integer, default 8000) through the shared engine adapter.
+
+Graph responses include results, returned/total counts, offset/nextOffset and current index coverage. `path` filters definition/symbol/caller call-site files; `container` filters declared containers or enclosing callers. Call edges remain name-based candidates, not resolved runtime bindings. Use pagination/filtering for incomplete results and source reads for authority. Parse/skip counts describe the current refresh; totalParseErrors/parsed/parseMs are cumulative. Unsupported and intentionally excluded files are outside coverage.
+
+Permanent delivery guidance now contains concise invariants; discovery/design/implementation instructions are supplied by phase. The compact delivery prompt retains classification, approval, verification and truthful handoff obligations. Browser instructions are conditional on task/profile relevance. The software-delivery skill recommends graph lookup -> consumers -> targeted source reads, with grep for strings/config/SQL.
+
+Context recovery separates a full exact-obligation checkpoint from current progress/findings/evidence. The outbound checkpoint retains its position while its source-history anchor remains. When compaction removes that anchor, restoration clears the cache, or contract identity changes, full obligations are restored. Stored session/report history and deterministic approval/freshness gates remain authoritative. This is not permission inferred from model memory.
+
+## Pilot outcome
+
+Follow-up adds advisory execution diagnostics after two/four consecutive failures and concise testing guidance against custom-loader overengineering, exit-masking pipelines, unrelated dependency changes and fake types. The repeat uses a natural user request with no benchmark/budget/test-technique hints; time/tool ceilings are external only. `/home/baissi/ctg_test/repeat-results.html` preserves the changed protocol. Neither repeat arm completed: updated timed out after core code (31/33 frozen checks, missing tests/docs), baseline ended before code on a malformed provider stream. No relative efficiency claim is justified; source review also found an auth-exception preservation gap missed by the evaluator.
+
+A one-task old/new pilot lives at `/home/baissi/ctg_test/results.html`, with task, frozen source manifests, provider receipts, events and independent quality checks alongside it. Both runs hit the 900-second allowance before delivery_finish. New used more aggregate input and cost despite lower mean input per request; its custom test-loader debugging consumed additional turns. This does not demonstrate a completion-rate or total-cost improvement. Follow-up should focus on test infrastructure guidance and limiting unproductive probe/revision loops, then use repeated randomized trials. No paid rerun was silently substituted for the failed outcome.

package/docs/jev-repository-pilot-plan.md

@@ -0,0 +1,25 @@
+# Integrated Jev + repository pilot: plan and author challenge
+
+User authorized implementation and execution. New campaign: at most $15 conservative total ($14 candidates, $1 diagnostics), no automatic retries or release of unknown holds. Preserve V1–V4. This is an exploratory pilot, not the held-out repeated suite.
+
+## Architecture
+
+1. Advisor transport: extend built-in Jev adapter with explicit OpenRouter support; fixed public endpoint or explicit trusted loopback broker for credential isolation. Bounded response/time/input, no retries. Existing direct TypeSafe remains supported.
+2. Planning orchestration: pure/advisory service records identity-bound attempts before network dispatch, max three/run; repeat inspect of identical run/revision/digest/source reuses receipt. Store advice in report and expose in recovery/status. Never changes gates, capture IDs, findings or requirements. Missing/bad configuration means unavailable, not clean advice. Explicit extension flags enable and choose provider; env key only used after opt-in. Benchmark uses this production path, not separate advisor extension.
+3. Reviewer: compact instruction asks model to address >=0.5 topics in normal review, without making probability a defect or approval. No new compulsory review field that could fake comprehension.
+4. Pilot runner: pinned real Node repository feasible without heavy DB services. Two additive feature tickets across public API/implementation/types/docs/regression tests. Three arms, one run/task/arm, balanced order, same engine/model/settings/limits, native compaction on. Plain/harness/Jev differences explicit. Online official reference retrieval equal in all arms; this is documentation access, not open-ended web research.
+5. Evaluator: supervisor reference patch and wrong-implementation mutants; behavioral tests on unmodified base must fail, reference pass, mutants fail. Protected evaluation after frozen candidate. Relevant upstream baseline tests and candidate tests reported separately. Final delivery not inferred from external pass.
+6. Execution: establish filesystem isolation for candidates and supervisor secrets/evaluators where possible using unprivileged user/mount namespaces; never claim sandbox without testing. Check source fingerprints/large repo compatibility, real installed engine, Jev, >100KB transport and native compaction before candidates. Freeze source/base/task/evaluator hashes before measured attempts.
+
+## Challenges before coding (author review, not independent)
+
+- No budget silently inherited: fresh exclusive campaign, explicit $15 ceiling including unresolved reservations.
+- Jev availability does not prove plan quality; scores may miss defects and dispositions remain model-authored.
+- Reading key only after explicit enable; no arbitrary configurable remote URLs/exfiltration or persisted credentials.
+- Durable reservation before request avoids re-dispatch on resume; errors preserve attempt and no automatic retry.
+- Full recovery must contain relevant advice identity/mode, but never duplicate full plan/provider responses.
+- Tests must cover same-identity reuse, revisions/source identity, cap, disabled/missing keys, timeout, malformed values and no gate mutation.
+- Real repository selection must be based on local feasibility, not treatment outcomes. Small library feature pilots are less representative than full commerce apps; report scope honestly.
+- Native compaction enabled identically and billed; request byte ceiling remains separate, not 'fixed' by raising it.
+- One replicate cannot establish Jev benefit. All failures/partial results retained, no repair after evaluator feedback.
+- Environment failure stops pilot; no synthetic success or 'pi cannot handle it' assumption.

package/docs/jev-repository-pilot-protocol.md

@@ -0,0 +1,39 @@
+# Fastify repository pilot protocol
+
+Exploratory six-cell pilot, not a held-out repeated benchmark. Authorized by user to plan/implement/run. Total ceiling $15 ($14 candidates + $1 diagnostics), fresh campaign, unresolved reservations retained. Each cell $2.30 / 900s /120 admitted tools /100 paid requests including <=3 Jev, 16,384 output tokens, 400KB raw/402KB rebuilt input. No retry or post-evaluation repair.
+
+## Selection and scope
+
+Fastify v5.6.2, SHA `f15d4eaba03d86786bec2de7aed1236718c813f9`, MIT repository, 384 tracked files, 72,231 lines across tracked JS/TS (includes tests). Node26.7.0, pi0.85.1. Local dependency installation with scripts disabled, generated lock preserved and frozen. Full-stack commerce fixtures deferred due to machine capacity. Both tasks are additive library API maintenance: reply.appendHeader and reply.vary, requiring implementation, types, docs and regression tests. Exact acceptance in `tools/repo-pilot/tasks.mjs`.
+
+Source: https://github.com/fastify/fastify/tree/f15d4eaba03d86786bec2de7aed1236718c813f9 . Author-created task contracts, not upstream tickets or claims these are desired upstream additions.
+
+## Arms and order
+
+append: plain -> harness -> Jev; vary: Jev -> harness -> plain. One replicate each; order partially balanced, no statistical efficacy claim. Same DeepSeek v4.1 flash/high through existing accounted gateway. Provider endpoint may vary; record it. Both Carthagent arms use new compact context; Jev arm uses **production built-in delivery flags**, not benchmark advisor.ts. No old-harness ablation, no causal attribution of context improvements versus changed tasks.
+
+Same native compaction configuration in every candidate: enabled, 100k configured context, reserve70k/keepRecent6k (threshold30k). No custom model summarizer. Smoke uses lower threshold solely to demonstrate native compaction. All summarization paid requests charged to corresponding account.
+
+## Documentation track
+
+Official Node22.19 HTTP markdown and RFC9110 fetched over HTTPS, frozen with URL/SHA receipts, mounted read-only at /reference identically. Model uses normal read/grep before approval, avoiding a custom read-only tool bypass. This tests **frozen online-documentation use**, not live unrestricted web search. No web research superiority claim.
+
+## Isolation
+
+Unprivileged extracted Ubuntu bubblewrap0.6.1, no system installation. New user/PID/network/IPC/UTS namespaces, all capabilities dropped, private proc/dev/tmp, read-only engine/harness/dependencies/reference mounts, only candidate workspace/config/session/logs writable. Host home/auth, supervisor tests/reference patch, other runs and original repository are not mounted. Tested inaccessible paths explicitly. Network namespace has only loopback; local Unix-socket relay exposes accounted generator/Jev broker endpoints. No Internet/production service credentials passed. This is stronger isolation than historical same-user cwd separation, but not a formal adversarial sandbox security audit; shared kernel and broker are trusted. No cgroup RAM/CPU limit; one sequential run and supervisor deadline/process-group cleanup.
+
+## Evaluation and readiness
+
+Protected external evaluator mounted only after candidate termination. Seven runtime categories per task: order/normalization, raw-state compatibility, special header semantics, arrays/no-op, atomic invalid calls, real HTTP onSend/plugin integration, request independence. Supervisor reference patches: 7/7 each; unmodified base1/7 each; overwrite/case-sensitive mutants3/7 and5/7. Initial evaluator rejected permitted array output; fixed BEFORE freeze; original failed selftest retained.
+
+External categories are finite behavioral coverage, not complete feature acceptance. Types/docs/new tests and upstream regressions must be scored separately. Relevant original tests mounted read-only for evaluator regression execution, preventing candidate weakening. Original test files still writable while developing; diff reports modifications. Full upstream suite/type suite may be candidate-run; disclose what actually ran.
+
+Baseline original `node --test test/internals/reply.test.js test/internals/request.test.js` passed. Two erroneous tap-path invocations during setup retained, replaced with repository's actual Node test runner, no product test failures claimed for those commands.
+
+Live sandbox smoke: built-in Jev advice on captured plan succeeded, no authority, all probabilities below0.5, expected blocked infrastructure handoff. Native pi compaction smoke recorded a real compaction at16,738 tokens then continued successfully. These are preflight, not measured task results. Current smoke known cost approximately$0.01132. Fresh-curl transport already covers exact large request wire behavior; focused transport tests rerun before freeze.
+
+Freeze all lib/extensions/docs/skill/benchmark/pilot sources, reference docs, lock, bubblewrap and engine package identity; baseline tree SHA. Not every transitive installed dependency hashed in first draft; final readiness must bind mounted runtime/dependencies before dispatch. Candidate verifies freeze before launch. No source edits during matrix; post-run report additions intentionally occur after all candidates.
+
+## Results required
+
+Report each arm/task: external categories, protected regression outcome, types/docs/test changes, candidate test outcome, terminal final answer, harness verified/blocked status, Jev topics/dispositions, native compactions, costs/unknown holds, runtime/tool/resource stops. Include infrastructure failures and protocol deviations. No blanket 'Carthagent wins' based on one replicate or passing a subset.

package/docs/jev-repository-pilot-restart.md

@@ -0,0 +1,7 @@
+# Uniform pilot restart after hostname infrastructure defect
+
+Initial root `/home/baissi/benchmarks/jev-repo-pilot` retained. Plain append external7/7 and types passed, but original94-test regression subset had localhost DNS failures because isolated `/etc/hosts` was absent. The initial baseline check was on host, not in the namespace: preflight gap. Do not count those as model failures. Harness append already running when discovered; manually terminated to avoid more invalid-environment spend. Its canceled in-flight request hold remains. Other four cells unstarted. Original freeze renamed `freeze-infrastructure-attempt.json` to prevent further dispatch before code changes.
+
+Repair: mount minimal frozen `/etc/hosts` (127.0.0.1 and ::1 localhost) and `nsswitch.conf` (`hosts: files`), keeping network isolated. Original94 tests now pass INSIDE corrected sandbox (`/tmp/repo-sandbox-baseline.log`). No general DNS or egress enabled.
+
+Uniform restart in `/home/baissi/benchmarks/jev-repo-pilot-v2`, fresh workspaces for all six cells. Same tasks/evaluators/model/arms/compaction/limits. Shared old campaign account, **no reset or release of unknown holds**, aggregate $15 unchanged. Setup directory referenced from first root; candidate cannot see this supervisor path. Source/runtime/dependency/doc/host-config hashes refrozen. All historical attempts reported separately. Evaluator feedback never given to any new candidate. No claim of preregistered confirmatory experiment.