vigiles 2.0.0

package/research/formal-proofs-for-agents.md

@@ -0,0 +1,338 @@
+# Formal Proofs for AI Agents
+
+Scope: should vigiles reach past its current deterministic proofs (monotonicity,
+NCD, Merkle, fixed-point, property tests in `src/proofs.ts`) into real formal
+verification, and if so, how?
+
+Bottom line up front: yes, but narrowly. Dafny is the only system where
+"LLM writes code + spec + proof" is remotely credible today, and even then only
+for small functions. Ship a `dafny()` enforce target first. Treat full
+evolution-engine proof gating as a research preview, not a product.
+
+---
+
+## 1. Landscape
+
+**Lean 4.** Dependent-type proof assistant, successor to Lean 3. Primary use is
+formalized mathematics (mathlib has ~1.8M lines of formalized math) and
+verification research. Production software use is negligible. Most
+LLM-theorem-proving papers target Lean because miniF2F and putnamBench are Lean
+benchmarks and mathlib is huge, clean training data. LLM-friendliness: highest
+of any system by volume of tooling, but proofs are long, tactic-heavy, and
+brittle.
+
+**Coq / Rocq** (renamed 2024). The grand-daddy: CompCert (verified C compiler,
+AbsInt ships it to Airbus), seL4 microkernel proofs, Iris separation logic.
+Real production use, real Fortune-500 deployment via CompCert. LLM tooling
+(Proverbot9001, CoqGym, Copra) is older and slower-moving than Lean's; the
+pivot to the Rocq name also fractured the ecosystem briefly.
+
+**Dafny.** Imperative/OO language with pre/postconditions and a built-in SMT
+backend (Z3). Written by Rustan Leino at MSR, now at AWS. Used in production at
+AWS: the ESDK (Encryption SDK), the Authenticated Encryption library, parts of
+S3's ShardStore, and Dafny-to-Rust compilation for cryptographic components.
+Dafny is the pragmatic choice: proof burden is 3-20 lines per function, not
+hundreds. LLM-friendliness is very good; Microsoft's own "Laurel" and "Clover"
+work specifically targets Dafny proof synthesis with LLMs.
+
+**F\*.** Dependent types + SMT, developed at MSR and Inria. Powers Project
+Everest: HACL\*, EverCrypt, Vale — these ship inside Firefox's NSS, the Linux
+kernel crypto path, WireGuard, mbedTLS, and ZincCrypto. Real production use,
+but learning curve is brutal. LLM tooling is thin; almost no papers target F\*
+specifically.
+
+**TLA+.** Leslie Lamport's specification language for concurrent and
+distributed systems. AWS uses it heavily — DynamoDB, S3, EBS designs were
+verified in TLA+ and Amazon has published about it. It does not verify code;
+it verifies designs and finds protocol bugs via model checking (TLC) or proof
+(TLAPS). LLM-friendliness is moderate — there are TLA+ generation papers but
+it's a small community.
+
+**Liquid Haskell.** Refinement types bolted onto GHC via SMT. Academic
+darling, small industrial footprint (Galois, Awake Security historically).
+Lightweight annotations, but depends on Haskell. LLM tooling: near zero.
+
+**Verus.** Verification for Rust, developed at MSR/CMU. Pre/postconditions,
+SMT-backed, targeting systems code. Still young (first stable release 2023)
+but Microsoft has a concerted push, including a verified storage engine
+(StorageNode in Verus) and verified parts of Hyper-V. LLM-friendliness: a few
+2024-25 papers (AutoVerus, SAFE from UCSD) exist, results are preliminary.
+
+**Kani.** Model checker for Rust from AWS, based on CBMC. Bounded verification,
+not full proof — you prove properties hold for all inputs up to bound N. Very
+practical and low-friction, and AWS uses it on Firecracker, s2n-quic, and
+aws-nitro-enclaves. LLM-friendliness: no dedicated tooling but the assertion
+style is easy to generate.
+
+---
+
+## 2. LLM-Assisted Theorem Proving: 2024-2026 State of the Art
+
+The field has moved fast on olympiad-style math and almost not at all on
+software verification. Honest pass rates below.
+
+**LeanDojo (2023, updated 2024).** CMU infrastructure paper: extracted mathlib
+as (state, tactic) pairs, released ReProver retrieval-augmented model. Baseline
+of ~51% on miniF2F-test for small open models. Still the standard harness
+everyone builds on.
+
+**Lean Copilot (2024).** Song et al., runs local LLMs as Lean tactics
+(`suggest_tactics`, `search_proof`). It's a human-in-the-loop authoring tool,
+not an autonomous prover. Useful, not transformative.
+
+**DeepSeek-Prover V1.5 (2024), V2 (2025).** Expert iteration over generated
+proofs. V1.5 reached ~63% miniF2F-test, V2 pushed to ~88% with RL and
+subgoal decomposition. State of the art for open models on olympiad math.
+
+**Goedel-Prover (2025, Princeton).** Trained on ~1.6M auto-formalized
+statements. ~64% miniF2F pass@32, ~7% on putnamBench pass@512. putnamBench is
+brutally hard — even the best closed models are in the single digits to low
+teens.
+
+**AlphaProof (DeepMind, July 2024).** Silver-medal IMO 2024 performance (4/6
+problems), Lean-based. Not released. Demonstrated that RL + massive compute
+on Lean can reach elite human level on contest math. Zero transfer demonstrated
+to real software.
+
+**Baldur (Meta/Google 2023, cited through 2025).** Whole-proof generation for
+Isabelle. ~41% on PISA benchmark with repair loop. Interesting because it
+generates whole proofs rather than tactic-by-tactic — similar ergonomics to
+how you'd want an agent to work.
+
+**Copra (2024).** GPT-4-based in-context proof agent for Lean and Coq. Uses
+error messages and retrieval. ~30% on miniF2F with GPT-4. Honest and modest.
+
+**The benchmark gap.** miniF2F and putnamBench are olympiad math. For actual
+software verification the closest thing is DafnyBench (Poesia et al., 2024),
+~750 Dafny problems drawn from textbooks and real code. GPT-4 solves ~68%.
+There is no equivalent large-scale Lean-for-software or Verus benchmark. This
+matters: 88% on miniF2F tells you nothing about whether an LLM can verify your
+sort function.
+
+**Summary.** On olympiad math, top systems are at 60-88% and the curve is
+steep. On real software verification, we have one benchmark (DafnyBench), one
+system that clearly works (GPT-4 + Dafny), and lots of arXiv preprints with
+<50% pass rates. Anyone claiming "LLMs write verified software" in 2026 is
+selling a demo.
+
+---
+
+## 3. Dafny vs Lean 4 for Wiring to Claude
+
+This is the core decision. Both can be driven by an LLM agent. They are not
+equivalent.
+
+| Dimension                 | Dafny                                     | Lean 4                               |
+| ------------------------- | ----------------------------------------- | ------------------------------------ |
+| Target user               | Programmers                               | Mathematicians, PL researchers       |
+| Proof style               | Declarative pre/post + SMT auto-discharge | Interactive tactic proofs            |
+| Typical burden            | 3-20 lines per function                   | 50-500 lines per theorem             |
+| What you verify           | Imperative/OO code                        | Arbitrary propositions               |
+| Backend                   | Z3 (automatic)                            | Kernel (manual)                      |
+| Tooling maturity for code | Production (AWS)                          | Research                             |
+| LLM pass rate in-domain   | ~68% (DafnyBench, GPT-4)                  | ~30-88% (miniF2F, math only)         |
+| Production users          | AWS (ESDK, S3 ShardStore)                 | None at code level; mathlib for math |
+| Error messages            | SMT counterexamples, often cryptic        | Type errors, very precise            |
+| Install footprint         | ~50 MB, single binary                     | ~500 MB, elan toolchain              |
+
+**The honest comparison.** Dafny is for verifying that a function satisfies a
+spec. Lean 4 is for proving theorems. If you want to say "this sort function
+returns a permutation that is sorted," Dafny does it in 5 lines. Lean 4 does it
+in 80 lines plus lemmas. If you want to say "this elliptic curve satisfies the
+Hasse bound," only Lean 4 can do that, and an LLM will not succeed unassisted.
+
+**For vigiles' audience** — programmers who write `.spec.ts` to guide coding
+agents — Dafny wins on every axis except theoretical expressiveness. The
+audience overlap between vigiles users and people who can read a Lean proof is
+near zero. The overlap between vigiles users and people who would accept a
+Dafny precondition is much larger, because Dafny looks like code.
+
+**LLM tooling.** Dafny's LLM story (Laurel, Clover, DafnyBench, plus Copilot
+working decently on small Dafny files) is behind Lean's in raw research volume
+but ahead in software-verification credibility. Lean's ecosystem is optimized
+for formalizing existing math, not synthesizing new code-level specs.
+
+**Recommendation: Dafny first, Lean as an experimental target later.**
+
+---
+
+## 4. Integration Patterns: Wiring a Prover to Claude
+
+Four options, brief pros and cons. These are not exclusive.
+
+### 4.1 Subagent calls prover via CLI
+
+Claude dispatches a subagent with the prompt "run `dafny verify foo.dfy` and
+fix errors until green." Subagent iterates until pass or budget exhausted.
+
+Pros: zero infrastructure, uses existing Task tool, fits Claude Code today.
+Cons: no caching, no shared proof state, subagent re-reads the whole file every
+loop, burns tokens fast on complex proofs.
+
+### 4.2 MCP server for Dafny/Lean
+
+Stand up an MCP server exposing `verify`, `suggest_tactic`, `get_goal_state`,
+`check_proof`. Claude calls these as tools.
+
+Pros: structured, stateful, interactive proofs become tractable, sharable
+across agents. Cons: building a good MCP server for an interactive prover is
+real work — Lean-Dojo took a team, and Dafny's LSP is less interactive. This
+is the right long-term answer and the expensive one.
+
+### 4.3 PostToolUse hook running prover on changed files
+
+Edit a `.dfy` file, hook fires `dafny verify`, output goes back into Claude's
+context on failure.
+
+Pros: mechanical, always on, cannot be skipped, fits vigiles' existing
+"enforce this externally" philosophy. Cons: feedback is one-shot per edit, not
+a conversation; large files re-verify from scratch.
+
+### 4.4 `vigiles verify` CLI command on spec-annotated blocks
+
+New CLI verb that scans the spec for `verify()` rules, extracts the referenced
+proof obligations, runs the chosen prover, and produces an audit report. Same
+shape as `vigiles check`.
+
+Pros: matches vigiles' current audit-at-commit-time model; deterministic; easy
+to integrate with CI; does not require Claude at all. Cons: not interactive —
+if verification fails, the user/agent has to loop manually.
+
+**Recommendation: start with 4.3 + 4.4.** Both are mechanical, both fit
+vigiles' existing architecture, neither requires Lean-Dojo-grade engineering.
+4.2 (MCP server) is the right v2 if users actually engage. 4.1 (subagent) is
+already possible without any vigiles change.
+
+---
+
+## 5. What Vigiles Should Ship First
+
+Three proposals, ordered by ambition. Ship the low one, prototype the medium,
+do not build the high one yet.
+
+### 5.1 Low: `dafny()` enforce target
+
+```ts
+enforce("dafny:ESDK/encrypt.dfy#EncryptIsInverseOfDecrypt");
+```
+
+Semantics: at `vigiles check` time, parse the reference, locate the `.dfy`
+file, run `dafny verify` on it, confirm the named lemma/method exists and
+verifies. Fits the existing stale-reference pattern in `src/linters.ts` — if
+the lemma is renamed or deleted, check fails.
+
+Effort: 1-2 weeks. Requires Dafny installed; follows the same model as the
+existing pylint/clippy/rubocop checks. This is a natural extension of
+vigiles' linter cross-referencing moat into the proof world.
+
+Production-grade: yes, for projects that already use Dafny. Demo-grade for
+anyone else. That's fine — vigiles never forced anyone to adopt pylint either.
+
+### 5.2 Medium: `verify()` rule builder that emits a proof obligation file
+
+```ts
+verify({
+  name: "MonotonicityProof",
+  target: "src/proofs.ts#monotonicityLattice",
+  obligations: [
+    "forall s1 s2 :: merge(s1, s2) >= s1",
+    "forall s1 s2 :: merge(s1, s2) >= s2",
+    "forall s1 s2 :: merge(s1, s2) == merge(s2, s1)",
+  ],
+  backend: "dafny", // or "lean4"
+});
+```
+
+Semantics: compiling the spec emits a `.dfy` skeleton with the obligations as
+method postconditions. Running `vigiles verify` tries to discharge them — with
+Dafny, often automatically; with Lean 4, the user (or an agent) fills in the
+proof. Either way vigiles records pass/fail alongside its existing proof
+record.
+
+Effort: 4-8 weeks. The hard part is the TS-to-Dafny type bridge. Stay very
+narrow — support int, bool, string, sequences, sets, and let users opt in per
+function. Do not try to verify arbitrary TypeScript.
+
+Production-grade: demo-grade initially. Becomes production-grade only for the
+subset of code users are willing to hand-translate or re-write in Dafny. Pitch
+it honestly: "for the 5% of your code that must be correct."
+
+### 5.3 High: Proof-gated evolution engine
+
+Today `src/evolve.ts` lets an LLM propose mutations to the spec, runs property
+tests, and commits passing mutations to a Merkle history. The ambitious version
+would require every mutation to also ship a proof obligation discharge —
+"the new rule implies the old rule, verified by Dafny" — before the Merkle
+commit is accepted.
+
+Effort: 3-6 months minimum, mostly research. The fundamental problem is that
+rule semantics are natural-language prose filtered through `enforce`, `check`,
+and `guidance`. There is no formal meaning of "the new rule implies the old
+rule" until someone defines one. You would need a specification logic for
+instruction files. That is a PhD project, not a sprint.
+
+Production-grade: no. Demo-grade at best, for a long time. Do not promise it.
+It is worth keeping as a research direction because it would be a genuine
+first — "the first self-evolving agent spec system with formally verified
+mutations" — but the honest engineering answer is "we are not there yet."
+
+---
+
+## Closing Honesty Check
+
+- LLMs solve 60-88% of olympiad-math problems in Lean. This does not transfer
+  to arbitrary software verification.
+- The one benchmark we have for code-level proof synthesis (DafnyBench) sits at
+  ~68% for the best closed models. That means for every three verified
+  functions, one is wrong.
+- No production system today lets an LLM "write code + spec + proof" for
+  general-purpose software. AWS's Dafny deployments are human-written with
+  machine-checked proofs. That is a different workflow than what an agent
+  would do.
+- Vigiles' existing deterministic proofs (monotonicity lattice, NCD, Merkle,
+  fixed-point, property tests) are already unusually rigorous for this product
+  category. Adding a Dafny enforce target is a cheap, honest extension.
+  Anything beyond that is marketing ahead of capability.
+
+Recommendation: ship 5.1. Prototype 5.2 behind a flag. Write a blog post about
+5.3 but do not build it.
+
+---
+
+## Additional Ideas (post-session)
+
+### 5.4. Dafny-in-comments — inline formal specs
+
+Mirror the inline enforce pattern but for formal contracts:
+`<!-- vigiles:verify dafny "ensures result > 0" -->` next to a function.
+vigiles extracts the contract, wraps the referenced function in a `.dfy`
+file with the contract as a postcondition, runs `dafny verify`. The
+developer writes zero Dafny — they write a one-line natural-language
+contract in a markdown comment, and the tool does the wiring. If the
+proof fails, the error goes back to the agent as a diagnostic. Same
+adoption shape as inline enforce: one comment, zero new files, zero build
+step. The agent can even generate the contract from the function's
+docstring.
+
+### 5.5. Property-test → Dafny bridge
+
+fast-check properties in the spec (from `propertyTest()` in proofs.ts)
+are already "lightweight formal methods" — they test an invariant over
+random inputs. Bridge to Dafny: when a property covers a pure function,
+offer to generate a Dafny contract from the property's invariant. The
+property test IS the spec; Dafny just machine-checks it exhaustively
+instead of sampling. The bridge is mechanical: `property("positive",
+(n) => f(n) > 0)` → `ensures f(n) > 0`. The hard part is translating
+the function body to Dafny, which is where the LLM earns its keep.
+
+### 5.6. Proof receipts in the Merkle chain
+
+When a Dafny/Lean verification passes during evolution, include the
+proof receipt in the Merkle history node alongside the existing
+monotonicity/NCD/bloom receipts. The chain becomes a certificate of
+correctness: "this spec change was not just fitness-positive and
+monotonically valid, it was FORMALLY VERIFIED by Z3/Lean kernel."
+Consumers can distinguish "verified by deterministic proofs" from
+"verified by formal prover" at the receipt level. The `verify()`
+method already checks receipt hashes, so formal receipts get
+tamper-evidence for free.

package/research/fp-for-agent-harness.md

@@ -0,0 +1,150 @@
+# FP for the Agent Harness: Railway, Effects, and Skills
+
+## Framing
+
+The previous FP doc (`fp-for-deterministic-ai.md`) was about the code agents **produce** — pure zones, Result types, exhaustive pattern matches in TypeScript. This doc is about the different layer: applying FP structure to the **agent harness itself** — Claude Code's skills, hooks, tool-use loop, session state. The same techniques, aimed inward instead of outward.
+
+The starting point is the half-joke from earlier: "what about railway programming structure for skills lol". It is not a joke. The Claude Code harness already has all the ingredients for an effect system and nobody has wired them up: skills are functions, hooks are effect handlers, tool calls are suspendable effects, the session transcript is event-sourced state. Structuring them with Railway / algebraic-effect idioms would make agent behavior **composable, inspectable, and replayable** — all three things that agents today are bad at.
+
+Claude Code is not going to adopt Effect.ts as an implementation detail. But vigiles can ship the contracts, conventions, and verification that act **as if** skills and hooks were Railway pipelines — and get most of the benefit at the spec layer without changing the harness.
+
+## What the harness already looks like
+
+A Claude Code turn decomposes into something like:
+
+```
+Input (user message + transcript)
+  ↓
+SessionStart hooks (can inject context)
+  ↓
+Model turn → tool call
+  ↓
+PreToolUse hooks (can block / rewrite)
+  ↓
+Tool execution
+  ↓
+PostToolUse hooks (can transform result / error)
+  ↓
+Model continues with result
+  ↓
+Stop hooks / output
+```
+
+Each arrow is a place a function gets called with a typed input and returns a typed output (possibly augmented with a side channel of diagnostics). That is a Railway. The harness already **is** a pipeline; the opportunity is to make the pipeline shape explicit so users can reason about it, skills can be composed, and failure paths collect instead of drop.
+
+## Ten ideas for FP applied to the harness
+
+### 1. Railway-typed skills — NOT BUILT
+
+Model a skill as `Skill<Ctx> = (Ctx) => Effect<Ctx, Diagnostic>` — it takes the current session context, returns either the next context or a list of diagnostics. Two skills compose with `.andThen`:
+
+```ts
+const reviewAndFix = review.andThen(runTests).andThen(fixFailures);
+```
+
+In Scott Wlaschin's formulation this is the two-track railway: the success track carries Ctx forward, the failure track carries Diagnostic short-circuited to the end. vigiles's contribution: a **spec-level declaration** of skill shape (`skill("review", { input: ..., output: ... })`) that compiles to a SKILL.md with the pipeline documented, and an audit check that each declared skill step actually exists. Users write skills any way they like; vigiles enforces the shape at the edges.
+
+### 2. Validation applicative for rule audits — PARTIALLY SHIPPED (audit collects all diagnostics across stages, but no formal Validation type)
+
+Today `vigiles audit` collects problems across stages (stale refs, dead enforcements, duplicates, coverage gaps) but each stage is written imperatively and short-circuits on its own. Switch to the Validation pattern: every check returns `Validation<OK, Diagnostic[]>` and the whole audit is `checks.map(run).sequence()`, collecting **all** failures across **all** checks in one pass. Contrast with Either monad: Either short-circuits at the first error. For audits you want the opposite — run everything, collect everything. This is a one-library change (neverthrow's `Result.combineWithAllErrors`) and makes audit output dramatically more useful per invocation.
+
+### 3. Hooks as algebraic effect handlers — NOT BUILT
+
+Reframe Claude Code hooks as handlers for algebraic effects. Tool use in the agent becomes `perform EditFile(path, content)` — a suspension, not a side effect. The harness interprets it by running registered handlers in order (PreToolUse handlers first, then the real tool, then PostToolUse). Each handler is a pure function `Request -> Handled | Rewrite | Continue`. This is what Effect.ts `provide` does. vigiles cannot change the harness, but it **can** ship a spec-time model of hooks with the same semantics, and a linter that catches hook ordering bugs. The payoff is: you can reason about hook stacks the way you reason about middleware, not the way you reason about shell scripts.
+
+### 4. Skill combinators: retry, fallback, parallel, race — NOT BUILT
+
+Once skills are typed as `Skill<Ctx>`, combinators fall out:
+
+- `retry(n, skill)` — run up to n times, return first success
+- `fallback(a, b)` — if `a` fails, try `b` with the same input
+- `parallel([...skills])` — fan-out/fan-in where all branches must succeed
+- `race([...skills])` — first to return a Result wins, others canceled
+- `timeout(ms, skill)` — bound wall-clock
+- `tap(log, skill)` — pre/post instrumentation without touching the skill body
+
+These are exactly the combinators `Effect.retry`, `Effect.orElse`, `Effect.all`, `Effect.race`, `Effect.timeout`, `Effect.tap` ship today for arbitrary computations. Porting them to skills gives users composable retry/fallback without hand-rolling control flow in bash. vigiles ships them as spec builders that compile to prose explaining the combinator in the target SKILL.md.
+
+### 5. Event-sourced session state — PARTIALLY SHIPPED (Merkle history is event-sourced with append-only chain + verify(); no fork/replay exposed)
+
+Treat the session transcript as an **event log**: each user message, model response, tool call, hook fire is an event. Session state at any point is `events.reduce(step, initial)`. Two properties fall out for free:
+
+- **Replay**: re-run `reduce` from event N to reconstruct state at turn N. Debugging a weird agent decision becomes point-in-time inspection.
+- **Fork**: `events.slice(0, n).concat(newEvent)` forks the session at turn n with a different branch. Lets you A/B a prompt change without losing the prefix.
+
+This is what Redux/Elm do and what Effect's `Fiber` model does internally. The harness already persists the transcript on disk; vigiles's role is a **spec-level invariant** that every rule produces deterministic output from the same input prefix — so replays match. An audit check: "replay session X from transcript, compare final state to recorded — diverge = bug."
+
+### 6. Lenses for settings.json — NOT BUILT
+
+`settings.json` is the agent's config: hooks, tool permissions, env. Today editing it is read-JSON, mutate, write. That loses atomicity and composition. Optics-ts / monocle-ts lenses let you express edits as `over(settingsLens.hooks.preToolUse, prepend(newHook))(settings)` — pure function, fully composable, trivially reversible. vigiles can ship a lens-based settings editor that underlies `vigiles init --install-hooks`, guaranteeing that concurrent edits from different commands don't stomp each other. The generalization: Claude Code config is a product type, and product types are what lenses are for.
+
+### 7. Skill type signatures in the spec — NOT BUILT
+
+`skill("review-pr", { input: PrNumber, output: ReviewReport, may: [ReadRepo, PostComment] })`. The `may` clause is an **effect annotation**: this skill is allowed to read the repo and post comments, nothing else. Compiles to a permissions section in SKILL.md that Claude Code can parse at invocation time, and a PreToolUse hook that blocks anything outside the declared effect set. This is how Koka / Eff / Frank handle effect rows at the type level; porting it down to skill metadata means the harness can enforce effect bounds **per skill** instead of per-session globally.
+
+### 8. Kleisli composition and `>=>` — NOT BUILT
+
+If skills are `A -> Effect<B>`, the operator to compose them is **Kleisli composition** (`>=>` in Haskell, `>>` in PureScript): `(f >=> g) = \a -> f(a).andThen(g)`. This is just function composition in the Effect category. Why it matters for vigiles: if the spec lets you write `pipeline("ship-pr", review >=> fix >=> push)`, the compiler can statically check that the output type of `review` matches the input type of `fix`. Type-safe multi-skill workflows. Today these workflows are written as bash or as prose instructions that the agent interprets; Kleisli composition makes them **compile-time verifiable graphs**.
+
+### 9. Reader monad for shared context — NOT BUILT
+
+Every hook and skill gets the same `{cwd, env, transcript, user, project}` passed in. That is the Reader monad: `Reader<Env, A> = (Env) -> A`. The payoff: `Reader.ask` lets a deeply nested combinator access the environment without threading it through every intermediate function, and `local(f, reader)` lets a combinator run its body with a **locally modified** environment without mutating the real one. For hooks this means you can write a hook that scopes `cwd` to a subdirectory for its inner work and transparently restores it afterward. vigiles can declare Reader-shaped context in the spec and verify every hook/skill's signature conforms.
+
+### 10. Pure replay harness for hook testing — NOT BUILT
+
+Today, testing a hook means setting up a real Claude Code session, triggering the tool, and reading the output. That is slow and flaky. If hooks are pure functions of `(Request, Env) -> Response`, you can **unit test** them with `fast-check` — generate random requests, assert invariants like "PreToolUse never returns both Allow and Rewrite" or "PostToolUse output is either the original result or a Result value, never undefined." vigiles ships a `vigiles test-hooks` subcommand that loads `settings.json`, extracts each hook as a pure function, and runs property tests against it. Catches the silent-matcher-ignore and trailing-wildcard anti-patterns from the agent-integration doc with **100% coverage** because the property test tries every shape.
+
+## Why Railway, specifically, for skills
+
+Scott Wlaschin's railway pattern has one killer property: **errors and happy path have the same shape.** Both are `Result<T, E>`. Both flow through the same pipeline. The skill author writes code as if nothing can fail; the combinators handle the failure plumbing. Agents are spectacularly bad at writing correct error handling — they `try { ... } catch (e) { console.log(e); }` and move on. Railway removes the temptation entirely: there is no `try` to write because there is no `throw`. The skill returns `Result.err(Diagnostic.MissingFile(path))` and the caller either keeps going or bails at the boundary.
+
+Concretely for a skill like "review a PR":
+
+```ts
+const reviewPr: Skill<PrCtx> = (ctx) =>
+  fetchPr(ctx.prNumber)
+    .andThen(loadDiff)
+    .andThen(checkStyle)
+    .andThen(runAudit)
+    .andThen(postComment)
+    .mapErr((diagnostic) => ({
+      ...ctx,
+      diagnostics: [...ctx.diagnostics, diagnostic],
+    }));
+```
+
+Every step can fail. The failure type is the same for all of them. The success path reads top-to-bottom with no `if` branches. A model looking at this pipeline can extend it (`checkSecrets.andThen(postComment)`) by adding one line. A model looking at the try/catch equivalent has to find the right `catch` block, decide if the new step needs its own, and often forgets. Railway is not just nice for humans — it is **easier for agents to modify correctly** than nested try/catch, which is the whole point.
+
+## What vigiles actually ships
+
+None of the above asks Claude Code's harness to change. vigiles's leverage is the **spec layer**:
+
+- `skill(name, { input, output, may, steps })` builder in `src/spec.ts`
+- Compiles to a SKILL.md documenting the pipeline, effect set, and failure modes
+- Audit check that every step referenced actually exists as a file or subskill
+- Optional: a tiny runtime wrapper (`@vigiles/skill`) that provides the neverthrow-based `Skill<Ctx>` type and combinators, so users who want to adopt the pattern in their own tooling can import a shared definition
+- Hook validation subcommand (`vigiles audit --hooks`) that treats hooks as pure functions and property-tests them
+
+The runtime wrapper is optional. The spec layer is the thing. Everything else is documentation of the pattern with a compile-time check that the documentation matches reality.
+
+### 11. Adversarial differential replay — NOT BUILT
+
+Record the Merkle chain from a session (every mutation + proof receipt). Replay the same sequence through a different LLM (or different temperature) and compare: which mutations does each accept/reject? Divergence = the mutation is model-sensitive, which means the proof suite isn't strict enough (the deterministic gate should have caught it). This is differential testing applied to agent behavior, and the Merkle chain is already the exact replay log it needs.
+
+### 12. Audit stage as a composable functor — NOT BUILT
+
+Each audit stage today is an imperative function that mutates `silent` state and accumulates counters. Make each stage a pure `(AuditReport) → AuditReport` transformation. Stages compose via plain function composition. The `silent`/`loud` threading becomes a Reader effect; the counter accumulation is just field merges. Users could define custom audit pipelines in their spec: `auditStages: [verifyHashes, inlineRules, coverage]` — pick what runs, skip what doesn't apply.
+
+### 13. Hook contract testing via fast-check — NOT BUILT
+
+Treat each Claude Code hook (PreToolUse, PostToolUse, SessionStart) as a pure function of `(Request, Env) → Response` and property-test it with fast-check: generate random tool-use requests, assert invariants like "PreToolUse never returns both Allow and Block" or "PostToolUse output is always valid JSON." The existing hook scripts are bash, so the "function" is actually `echo $INPUT | bash hook.sh | read $OUTPUT` — property testing over that subprocess boundary catches the silent-matcher-ignore and trailing-wildcard anti-patterns from the agent-integration doc with machine coverage instead of manual audit.
+
+## Priority
+
+1. **`skill()` builder with typed input/output** — the smallest change, unblocks everything else. Write the builder, compile it to the existing SKILL.md format, audit for missing referenced files.
+2. **Validation applicative for audit** — one-library refactor, immediate UX win ("audit reports 7 issues" instead of "audit failed at issue 1").
+3. **Skill combinators as spec builders** — `retry()`, `fallback()`, `parallel()` that compile to prose. No runtime required, just better SKILL.md output.
+4. **Hook property testing** — depends on having a spec model of hooks first, then layering fast-check.
+5. **Effect annotations + lens-based settings editor** — the sharpest tools but the most design work. Ship last.
+
+The short version: the previous FP doc was about TS code patterns for LLM output. This one is about applying the same shapes **one level up** to the agent runtime. Railway for skills is not a joke — it is the only way to compose agent pipelines that both humans and models can edit safely.

package/research/fp-for-deterministic-ai.md

@@ -0,0 +1,131 @@
+# Functional Programming Techniques for Deterministic AI Code
+
+## Framing
+
+LLMs are nondeterministic by construction. Temperature 0 reduces variance but does not eliminate it. The practical question is not "how do we make the model deterministic" but "how do we **shape the code the model writes** so that nondeterminism is harmless." Functional programming has spent four decades answering this exact question for a different reason (human reasoning about concurrent/parallel/distributed code), and its tools transplant cleanly.
+
+The thesis: **pure functions with total signatures and exhaustive pattern matches are the substrate on which agents fail loudest and recover fastest.** A pure function that returns the wrong value fails a property test. A function that throws produces a stack trace in production. Agents are much better at fixing the first one.
+
+This doc surveys the FP techniques that have shown up in the 2025–2026 literature on AI code quality, notes the specific tooling available in the TS/JS ecosystem, and proposes 10 vigiles features that operationalize them.
+
+## Why FP is a good fit for LLM output
+
+Three properties make FP uniquely useful here:
+
+1. **Totality.** A pure total function is a contract: "for every input in type A, I produce exactly one value in type B." Agents that write code under that contract cannot silently skip edge cases — unhandled cases become type errors. `ts-pattern`'s exhaustiveness check is the clearest example: the compiler refuses to build until every branch is handled.
+
+2. **Referential transparency.** `f(x) === f(x)` for all x. This is what makes property-based testing work: you can generate 10,000 random inputs and assert an invariant without worrying about order or state. arxiv 2506.18315 ("Property-Generated Solver") showed a **23–37% pass@1 improvement** on code tasks when the harness generated properties first and let the model iterate against failing counter-examples. That only works on pure code.
+
+3. **Composition.** Small building blocks with typed connectors force the agent into a local reasoning window. It is much easier for a model to write `pipe(x, validate, normalize, save)` correctly than it is to write a 200-line function that weaves the same steps through mutable state. Railway-oriented programming (Scott Wlaschin) is the cleanest articulation: every step is `A -> Result<B, E>`, errors short-circuit, success flows down the happy path.
+
+## The counter-evidence
+
+arxiv 2601.02060 ("FPEval") benchmarked code generation on **purely functional languages** (Haskell, OCaml, Elm) and found LLMs perform **15–40% worse** than on Python/TS. The reason is training-distribution: there is orders of magnitude more imperative code in the training set. So pure FP languages are not the answer.
+
+The answer is **FP-style code in mainstream languages**, specifically TypeScript. The training set is huge, the type system is expressive enough, and the ecosystem has a cluster of libraries (Effect, neverthrow, ts-pattern, fast-check, Zod) that make FP idioms first-class without forcing a paradigm shift. vigiles can bundle those libraries into enforce-preset bundles and get the benefits without the performance hit.
+
+## Ecosystem survey (TypeScript, 2025)
+
+| Tool                                             | What it gives you                                                                               | Relevance                             |
+| ------------------------------------------------ | ----------------------------------------------------------------------------------------------- | ------------------------------------- |
+| `eslint-plugin-functional`                       | `no-throw-statements`, `immutable-data`, `no-let`, `no-loop-statements`, `prefer-readonly-type` | Direct `enforce()` targets            |
+| `neverthrow`                                     | `Result<T, E>` + `ResultAsync` + `.map`/`.andThen`/`.mapErr`                                    | Railway-style error handling          |
+| `Effect.ts`                                      | Full effect system: `Effect<R, E, A>`, managed concurrency, resources, retries                  | Heavier but complete                  |
+| `fp-ts` / `ts-results`                           | Classical Either/Option/Task                                                                    | Smaller footprint than Effect         |
+| `ts-pattern`                                     | Exhaustive pattern matching via `P.infer`, `.exhaustive()`                                      | Totality on tagged unions             |
+| `monocle-ts` / `optics-ts`                       | Lenses, prisms, traversals                                                                      | Immutable deep updates                |
+| `zod` / `valibot` / `@effect/schema`             | Parse-don't-validate at boundaries                                                              | Purity zones need these               |
+| `fast-check`                                     | Property-based testing, shrinking                                                               | Matches 2506.18315 technique          |
+| `@typescript-eslint/switch-exhaustiveness-check` | Compile-time exhaustive switch                                                                  | Cheap totality win                    |
+| CodeQL (Datalog) / Flux / Liquid Haskell         | Refinement / dependent types                                                                    | Out of scope for us but worth knowing |
+
+## How this interacts with vigiles
+
+vigiles is a spec-to-markdown compiler with a linter-cross-reference engine. It does not write code. But it decides **which rules the agent is told to follow**, and it verifies those rules against real linter config. That makes it the natural home for a curated set of FP presets: instead of forcing every team to discover `neverthrow` and configure `eslint-plugin-functional` from scratch, vigiles can ship a `pureZone()` builder that encodes the whole bundle.
+
+Crucially, this is not reimplementing a linter. Every rule still runs in ESLint / TypeScript / ts-pattern. vigiles just decides **where** they apply, tells the agent in prose, and verifies the wiring.
+
+## Ten vigiles ideas
+
+### 1. FP Determinism Preset — DROPPED (just an eslint-plugin-functional wrapper, no moat)
+
+A named bundle: `fpDeterminism()` returns an array of `enforce()` calls covering the minimum viable FP-for-agents rule set. Target:
+
+- `functional/no-throw-statements`
+- `functional/no-let`
+- `functional/immutable-data`
+- `functional/no-loop-statements` (allows `map`/`filter`/`reduce`)
+- `@typescript-eslint/switch-exhaustiveness-check`
+- `@typescript-eslint/no-floating-promises`
+
+Users write `...fpDeterminism()` in a spec section and inherit the entire preset. Upgrading the preset upgrades every downstream spec at once.
+
+### 2. Pure zones — NOT BUILT
+
+Builder: `pureZone("src/core/**", { allow: ["date-fns"] })`. Compiles to a section in the target markdown saying "inside `src/core/**`, no I/O, no throws, no globals, no Date.now / Math.random / fetch / fs." Backed by `eslint-plugin-functional` + a focused ESLint override for that glob. The agent sees "you are in a pure zone" in context; the linter enforces it for real. Two-layer defense: prose for reasoning, lint for deterministic rejection.
+
+### 3. Rule combinator API — NOT BUILT
+
+Expose `Rule<A, B>` as a public plugin type so users can compose their own rules: `pipe(spec.rules, strengthen("no-console"), restrictGlob("src/core/**"))`. Internally this is the same data the compiler already works with; externally it turns vigiles into a spec-programming library rather than a config file. Enables `fpDeterminism()` (#1) and every other preset to be built out of the same primitives users can use.
+
+### 4. PBT coverage check — NOT BUILT
+
+Audit-time assertion: for every file matching `src/**/*.ts` that exports a pure function (no `Promise`, no parameter of type `unknown`, no `void` return), require a colocated `*.property.test.ts`. Dovetails directly with arxiv 2506.18315 — properties catch the failure modes tests miss, and agents iterate on property failures faster than they iterate on hand-written tests. `fast-check` already handles the runtime. vigiles just makes coverage visible.
+
+### 5. Content-addressed compile cache — NOT BUILT
+
+Hash every spec by its AST + transitive imports; cache the compiled markdown by that hash. Re-running `vigiles compile` on an unchanged spec becomes a no-op. Bazel-style memoization. The reason this belongs in an FP doc: it only works because the compiler is itself a pure function from spec AST to markdown. The more we lean on FP internally, the cheaper incremental work gets.
+
+### 6. Refinement at boundaries via schema libraries — NOT BUILT
+
+New `enforce()` target category: `zod/strict-object`, `valibot/parse-not-safeParse`, `@effect/schema/decodeUnknown`. vigiles verifies the schema library is in the project and the rule is enabled. The spec then tells the agent "all external input must be parsed via Zod before it enters a pure zone." Parse-don't-validate is the single most effective technique for containing AI-generated input handling code — it converts runtime surprises into boundary errors.
+
+### 7. Exhaustiveness-checking preset — NOT BUILT
+
+One-liner: `enforce("@typescript-eslint/switch-exhaustiveness-check")` + `enforce("ts-pattern/exhaustive")`. Ships as a standalone mini-preset because it is by far the highest leverage single rule: it turns every `switch` on a discriminated union into a type-level guarantee. LLMs drop cases constantly; this catches every single one at compile time, not review time.
+
+### 8. Result/Either error handling preset — NOT BUILT
+
+Banned: `throw`, `try/catch` outside top-level handlers, `.catch(() => null)`. Required: return types shaped `Result<T, E>` via neverthrow (or Effect.ts if the project already uses it). vigiles verifies the project has neverthrow in `package.json` and that the relevant eslint-plugin-functional rules are on. Agent sees "errors are values; use `.mapErr` to transform them." This is the single largest departure from standard TS agent output, so the preset needs to be loud in the compiled markdown — but the payoff is that error paths become testable.
+
+### 9. Immutable-by-default preset — NOT BUILT
+
+Bundle: `no-let`, `immutable-data`, `no-loop-statements`, `prefer-readonly-type`, plus a `check()`-equivalent that greps for `.push(`, `.splice(`, `Object.assign`, and direct array index assignment. Not every project can adopt this wholesale, which is why it ships as a separate preset applied via `pureZone()`. Inside a pure zone, the agent is forced into `[...xs, y]` and `{...obj, k: v}` — which it handles fine.
+
+### 10. Lens recommendation detector — NOT BUILT
+
+Audit-time scan: detect the pattern `{...a, b: {...a.b, c: {...a.b.c, d: value}}}` (spread nesting ≥ 3) and suggest switching to `monocle-ts` or `optics-ts`. Pure diagnostic — vigiles does not rewrite the code. But it does catch the single ugliest FP anti-pattern LLMs produce when asked to "update this field immutably": nested spreads that nobody can read and nobody tests. Flagging them directs the agent toward the lens library the project presumably already has.
+
+### 11. Immutability as a security property — NOT BUILT
+
+The 15+ defensive-copy fixes from PR #16 proved that shared mutable references are not just a style concern — they're **bypass vectors** in proof-gated systems. Every live reference the engine returned (rules, history nodes, receipts, allowWeaken Set) was a surface where a caller could mutate state without running proofs. Ship this insight as a rule: `enforce("functional/immutable-data")` in evolve.ts and proofs.ts themselves, documented with the rationale "immutability is tamper prevention, not taste." Dogfood vigiles on vigiles.
+
+### 12. Budget-aware fitness with a cliff — NOT BUILT
+
+The current fitness formula (`coverage × (1 - redundancy) × (1 - budgetPressure)`) penalizes ANY token growth linearly, which makes `acceptNeutral` nearly useless: adding a rule always increases tokens, always decreases fitness, always gets rejected. A better formula: `coverage × (1 - redundancy) × min(1, maxTokens / tokens)` — score is flat at 1.0 until you exceed the token budget, then drops off a cliff. This lets useful additions pass while still enforcing the hard cap. Simple change, fixes a real design bug discovered during the `proposeAll` test rewrite.
+
+### 13. Deterministic seed for property tests in CI — NOT BUILT
+
+`propertyTest()` in proofs.ts accepts a `seed` option, but nothing in the CI pipeline uses it. Ship a convention: `vigiles audit --pbt-seed=$GITHUB_SHA` so property-based tests are deterministic per commit (reproducible failures) but vary across commits (explore the search space over time). Combines the benefits of deterministic CI (no flakes) with the coverage of randomized testing (different inputs each push).
+
+## Railway-oriented programming for skills
+
+Tangent worth noting: the same Result-pipe pattern applies to vigiles's own internal surface. If every audit step, every compile step, every linter-verification step is typed as `(Input) => Result<Output, Diagnostic[]>`, the whole compiler is one `pipe()` chain from spec AST to final output. Errors collect instead of throwing; the top-level command decides whether to short-circuit or aggregate. This is how Effect.ts pipelines are structured, and it would make the compiler's own behavior under partial failure **provably** correct instead of "seems to work in tests." A good reason to adopt neverthrow internally even before we ship it as a preset.
+
+## What we are NOT doing
+
+- Not adopting Haskell. FPEval says no. TypeScript FP presets are the sweet spot.
+- Not shipping our own `Result` type. neverthrow is a single small dependency and is already the community default.
+- Not rewriting the compiler in Effect.ts. Too much churn. neverthrow + pipe is enough.
+- Not building a refinement-type system. Liquid TypeScript does not exist yet and Flux is Rust-only; Zod at boundaries is 90% of the value for 5% of the cost.
+
+## Priority
+
+1. **#1 fpDeterminism preset** + **#7 exhaustiveness preset** — single day of work, zero new dependencies beyond what users opt into, immediate win
+2. **#2 pureZone + #6 schema boundaries** — the core pair; these define where the other presets apply
+3. **#4 PBT coverage check** — wires into audit, no new subsystem, matches the strongest academic signal
+4. **#3 rule combinator API** — enables plugins, unlocks user-contributed presets, 1–2 days of refactor
+5. **#8 Result preset + #9 immutable preset + #10 lens detector** — powerful but opinionated; ship after the first four land so adopters have the machinery to opt in surgically
+6. **#5 content-addressed cache** — pure performance work; ship when compile times become a complaint
+
+The TL;DR is: we do not need to change the compiler to get the benefit of FP for AI code. We need to ship curated rule bundles, verify them against real linters (which vigiles already does), and let `pureZone()` and `fpDeterminism()` carry the reasoning load for users.