nookku 0.4.0__tar.gz

nookku-0.4.0/.gitignore

@@ -0,0 +1,26 @@
+__pycache__/
+*.py[cod]
+.venv/
+.venv-*/
+dist/
+build/
+*.egg-info/
+.coverage
+coverage.xml
+htmlcov/
+.mypy_cache/
+.pytest_cache/
+.ruff_cache/
+.DS_Store
+.env
+CLAUDE.local.md
+.nookku/
+# The state folder of verbatim-relay 0.3.x and earlier.
+.verbatim-relay/
+.proof/
+bench/smoke/
+bench/logs/
+tap.jsonl
+# The proofs commit the tap record of each run as their data.
+!proofs/**/tap.jsonl
+.gstack/

nookku-0.4.0/CHANGELOG.md

@@ -0,0 +1,127 @@
+# Changelog
+
+## 0.4.0
+
+verbatim-relay is now Nookku. The package, the CLI, the module, the plugin, the marketplace, the state folder and the variables have the new name. The record format and the audit do not change.
+
+**Upgrade.** End each running test. Then install `nookku`, update the plugin and run `nookku init` again. The first `nookku` command in a project moves `.verbatim-relay/` to `.nookku/`. [The upgrade guide](docs/how-to/upgrade.md#040) gives each step.
+
+- **New name.** The package and the CLI are `nookku`, the module is `nookku`, and the plugin is `nookku@nookku`. The variables start with `NOOKKU_`, for example `NOOKKU_HOME`. The repo is `mohanraj00/nookku` ([#181](https://github.com/mohanraj00/nookku/issues/181), [#208](https://github.com/mohanraj00/nookku/issues/208), [#210](https://github.com/mohanraj00/nookku/issues/210)).
+- **State folder move.** Each command with a project root moves `.verbatim-relay/` to `.nookku/`. A `record` path in the old folder changes to the same file in `.nookku/`. If both folders exist, the command stops and names both. A hook event is then blocked, so the relay fails closed ([SPEC.md section 7](SPEC.md#7-tests), [#181](https://github.com/mohanraj00/nookku/issues/181)).
+- **Old seal copies.** `verify` reads the seal copy of an old test from `~/.verbatim-relay/seals/` if `~/.nookku/seals/` has no copy ([SPEC.md section 7.4](SPEC.md#74-seal), [#181](https://github.com/mohanraj00/nookku/issues/181)).
+- **One core, one plugin.** [ADR 0001](docs/adr/0001-one-core-one-plugin.md) records the decision for one core and one plugin for both harnesses. It links the data and the method of each spike ([#176](https://github.com/mohanraj00/nookku/issues/176), [#177](https://github.com/mohanraj00/nookku/issues/177), [#178](https://github.com/mohanraj00/nookku/issues/178), [#179](https://github.com/mohanraj00/nookku/issues/179), [#180](https://github.com/mohanraj00/nookku/issues/180), [#193](https://github.com/mohanraj00/nookku/issues/193), [#194](https://github.com/mohanraj00/nookku/issues/194), [#205](https://github.com/mohanraj00/nookku/issues/205)).
+- **Reply in the chat.** The hook kit shows each reply as the reason of the blocked prompt, byte for byte. Claude Code and Codex show the reason to the person and do not give it to the model. `codex exec` shows no reason, so `nookku view` stays ([#214](https://github.com/mohanraj00/nookku/issues/214)).
+- **Faster hooks.** A hook event imports only the modules that its rule needs. Over the floor of the same run, a warm event takes 14 to 23 ms on Linux and on macOS (the second run of the data, with less load), not 28 to 34 ms ([data](proofs/hook-time/hook-latency.json), [method](scripts/spike_hook_latency.py), [#214](https://github.com/mohanraj00/nookku/issues/214)).
+- **Hook with no root.** `nookku hook` finds the project from `CLAUDE_PROJECT_DIR` or the `cwd` of the event, if it gets no `--root` ([#214](https://github.com/mohanraj00/nookku/issues/214)).
+- **MCP server.** `nookku mcp` gives the model the read-only tools `transcript` and `status`, with the standard library only. The transcript comes in pages with a footer and a SHA-256, so a page that a harness cut shows it. The default page got to the model unchanged in Claude Code and in Codex for 4 kinds of text ([data](proofs/mcp/transcript-pages.json), [method](scripts/measure_transcript_pages.py), [#215](https://github.com/mohanraj00/nookku/issues/215)).
+- **One plugin for Claude Code and Codex.** The plugin folder is `plugins/nookku`, with a manifest for each harness, one command hook file, the MCP server `nookku mcp` and one skill. Each command hook runs `nookku hook`, so the plugin and the project hooks share one core. If `nookku` is not on `PATH`, a shell guard blocks the prompt in relay mode. The Claude Code function hooks only show the output of the core: `/nookku`, the status line and the pane. One exception: a command hook gets only the text of a prompt, so in relay mode the function hook drops a prompt with an attachment, with the text of the core. The plugin loads in Claude Code 2.1.295 and in Codex 0.162.0 ([data](proofs/plugin/load.json), [method](scripts/proof_plugin_load.py), [#216](https://github.com/mohanraj00/nookku/issues/216)).
+- **Codex hook gate.** Codex skips a hook that a person did not trust, and then a message goes to the model. `start` and `mode on` now read the hooks of Codex with `hooks/list`, and refuse the test if a nookku hook is missing, disabled, not trusted or modified. The trust hash of Codex does not cover the plugin script, so the gate also checks its SHA-256. `end` checks again, and the manifest keeps both results. With the gate, the Codex marketplace lists the plugin as available ([SPEC.md section 7.8](SPEC.md#78-codex-hook-gate), [data](proofs/codex-gate/gate.json), [method](scripts/proof_codex_gate.py), [#217](https://github.com/mohanraj00/nookku/issues/217)).
+- **Docs for one core.** The docs name one relay, `nookku hook`, in 2 forms: the plugin and the project hooks (the hook kit of 0.3). The plugin works in Claude Code and in Codex. [Choose the plugin or the project hooks](docs/how-to/choose-a-relay.md) compares them. SPEC.md section 5 and [limits.md](docs/limits.md) state that `claude -p` and the Codex desktop app can send a blocked prompt to the model API ([#199](https://github.com/mohanraj00/nookku/issues/199), [#203](https://github.com/mohanraj00/nookku/issues/203), [#218](https://github.com/mohanraj00/nookku/issues/218)).
+- **Codex plugin upgrade.** The upgrade guide updates the Codex plugin with `codex plugin marketplace upgrade nookku` and `codex plugin add nookku@nookku`, not with a new install ([upgrade.md](docs/how-to/upgrade.md#3-update-the-plugin), [data](proofs/spikes/codex-plugin.json), [method](scripts/spike_codex_plugin.py), [#218](https://github.com/mohanraj00/nookku/issues/218)).
+- **Proofs of one core.** `scripts/proofs_relay.py <setup>` runs P1 to P4 in 4 setups: the plugin and the project hooks, each in Claude Code and in Codex. It replaces the proof scripts of the TypeScript plugin. The report, telemetry and evaluation proofs take the same setup names. Each proof passed in each setup with nookku 0.3.1 ([results.md](docs/results.md#1-proofs), [#219](https://github.com/mohanraj00/nookku/issues/219)).
+- **Sample outputs.** The docs show the output of nookku 0.4.0, in place of the output of verbatim-relay 0.3.0. A model answer and a command that changes the install of the computer keep the output of 0.3.0, and each page says so ([#212](https://github.com/mohanraj00/nookku/issues/212)).
+- **No plugin options.** The plugin reads `.nookku/config.json`. Write it with `nookku init plugin`, which writes no project hooks. `nookku mode on` with no config keeps relay mode off ([#216](https://github.com/mohanraj00/nookku/issues/216)).
+- **No record limit.** The plugin no longer stops at a relay record of 3.5 MiB, because the core writes the record ([#216](https://github.com/mohanraj00/nookku/issues/216)).
+- **Policy file.** `.nookku/policy.json` holds the keys that control the guard. Only a person changes it, with `nookku policy set` in a shell. The relay denies each model write to the file, also when no test runs and in a project with no `.nookku/` folder. The manifest records the value of each key. Its first key is `guard_fields` ([SPEC.md section 7.9](SPEC.md#79-policy-file), [#175](https://github.com/mohanraj00/nookku/issues/175)).
+- **Guard checks paths.** The deny of the state folder checks the paths that a tool call writes, not its text. When no test runs, a tool call can change `config.json` and the files of the entry, and the relay denies only a write to a record or a state file. A file whose content names a record, or a here-document that is data, is not denied. Each reason names the path ([SPEC.md section 5](SPEC.md#5-relays), [#149](https://github.com/mohanraj00/nookku/issues/149)).
+- **Guard fields.** For an MCP tool or another tool that is not a shell or a file tool, the guard reads only the fields of the policy key `guard_fields`: by default `path`, `file_path`, `url`, `uri` and `command`. A document body that names the state folder or the tap is not denied ([SPEC.md section 5](SPEC.md#5-relays), [#150](https://github.com/mohanraj00/nookku/issues/150)).
+- **Read commands and deny causes.** The read check accepts `which`, `type` and `command -v`, and the programs of the new policy key `guard_reads`. The records deny names the program or the redirect that failed, for example `"rm" is not a read command.` A variable can be an argument of a read program, but not the program ([SPEC.md section 5](SPEC.md#5-relays), [#172](https://github.com/mohanraj00/nookku/issues/172)).
+- **Both forms on.** In Claude Code, `start` and `mode on` refuse a test if the nookku plugin and the project hooks are both on, as the Codex hook gate does. `init` warns if the other form is on ([SPEC.md section 7.8](SPEC.md#78-codex-hook-gate), [#151](https://github.com/mohanraj00/nookku/issues/151)).
+- **Remove the project hooks.** `nookku init --remove claude-code` or `nookku init --remove codex` removes the hooks of nookku from the hook file, and keeps each other hook and key. You do not need `jq` now. It refuses while a test runs or relay mode is on. Only a person runs it: the relay denies it to a model ([remove.md](docs/how-to/remove.md#2-remove-the-project-hooks), [#163](https://github.com/mohanraj00/nookku/issues/163)).
+- **Test folder for the app.** The entry gets `NOOKKU_TEST_DIR` and `NOOKKU_TEST_ID`. An app can keep its state in `app-state/` of the test folder. The seal covers it, and the evaluation reads it. The toy shop writes `app-state/answers.jsonl` ([SPEC.md section 6.1](SPEC.md#61-environment-of-the-entry), [#145](https://github.com/mohanraj00/nookku/issues/145)).
+- **Check message.** The new config key `check_message` sets the one message of `nookku check`, and `init` takes `--check-message`. An app that declines the default message can now get a message that it answers ([SPEC.md section 7.1](SPEC.md#71-configuration), [#147](https://github.com/mohanraj00/nookku/issues/147)).
+- **Repeated reply.** The new trace check `repeated_reply` gives one finding for each run of 2 or more consecutive turns with the same reply bytes, for example a fixed fallback text. It is a finding, not a break. The new config key `repeated_reply` sets the count, and 0 turns the check off. `init` takes `--repeated-reply`, and the manifest records the value ([SPEC.md section 8.6](SPEC.md#86-findings), [#155](https://github.com/mohanraj00/nookku/issues/155)).
+- **Compact transcript.** `nookku transcript --trace --compact` shows a repeated prompt one time. After that, it shows the line, the SHA-256 and the length. An OpenTelemetry attribute with the text of a session item of the same turn also shows in this form. The new config key `transcript_compact` sets the default, and `--no-compact` overrides it. The `transcript` tool of `nookku mcp` takes `compact` ([SPEC.md section 9.2](SPEC.md#92-evaluation-prompt), [#152](https://github.com/mohanraj00/nookku/issues/152)).
+- **Reply not from the model.** The new trace check `reply_not_from_model` gives a finding for each turn whose reply does not contain the text of a model output of that turn, for example a fixed decline text in place of the model text. It is a finding, not a break. The new config key `model_reply_field` names the client text in a structured model output, and `model_reply_match` is `exact`, `normalized` (the default) or `off`. `init` takes `--model-reply-field` and `--model-reply-match`, and the manifest records both values ([SPEC.md section 8.6](SPEC.md#86-findings), [#154](https://github.com/mohanraj00/nookku/issues/154)).
+- **Scripted tests.** `nookku run --script <file>` sends each message of a script file to the entry, byte for byte, and ends the test. The script is JSONL with `{"said": "..."}` rows or one JSON array of strings. The test folder keeps a copy, each relay row has `"harness": "script"`, and the new break class `script_mismatch` names a turn that is not its script message. The new config key `entries` names more entries, and `--entry` takes only such a name. If the policy key `scripts` is `model`, the model can run a script with `nookku run` or with the `run_script` tool of the plugin ([SPEC.md section 7.10](SPEC.md#710-scripted-tests), [#171](https://github.com/mohanraj00/nookku/issues/171)).
+- **Drive commands.** The guard denies a model command that runs `nookku start`, `end`, `hook`, `run` or `mode on|off`, or that imports the package. The new policy key `scripts` lets the model run `nookku run` ([SPEC.md section 5](SPEC.md#5-relays), [#173](https://github.com/mohanraj00/nookku/issues/173)).
+- **Near-miss control prompts.** In relay mode, the relay refuses a message near a control prompt, for example `nookku stop` or `nookk end`, and sends nothing. The refusal lists the control prompts. The new policy key `control_distance` sets the edit distance, default 2 ([SPEC.md section 5](SPEC.md#5-relays), [#146](https://github.com/mohanraj00/nookku/issues/146)).
+- **`init` in the state folder.** `init` refuses a root in `.nookku/`, for example a test folder, and writes nothing. It warns if a new root is not the top level of its git work tree. Its output starts with the absolute root ([docs/reference/cli.md](docs/reference/cli.md#nookku-init), [#148](https://github.com/mohanraj00/nookku/issues/148)).
+- **One line for each untested version.** The transcript with the trace shows the `version_untested` findings of one harness and one version as one line, with the count of sessions. `findings.json` still has one finding for each session ([SPEC.md section 9.2](SPEC.md#92-evaluation-prompt), [#153](https://github.com/mohanraj00/nookku/issues/153)).
+- **Tool events by id.** The check `otel_tool_not_in_session` looks only at the tool calls of the session of the event. It matches a Claude Code `tool_result` event to its tool call by the tool use id, if both have one. Else it compares the tool name and the input in the same turn, and a cut string of the event agrees with the start of the session string. A structured output that the model sends 2 times in one turn now gives no finding ([SPEC.md section 8.6](SPEC.md#86-findings), [#174](https://github.com/mohanraj00/nookku/issues/174)).
+- **Last verbatim-relay.** The package `verbatim-relay` 0.4.0 has no dependency and does not install `nookku`. Its command prints one line with the new name and the link to the upgrade guide, and exits with 1 ([tests/test_redirect.py](tests/test_redirect.py)). It does not run `nookku`. `import verbatim_relay` gives the same message as an `ImportError`. An old project hook runs `python -m verbatim_relay hook`. It prints the same line and exits with 2, so the hook blocks the event and no prompt goes to the model. A separate workflow publishes it from a `verbatim-relay-v*` tag ([#213](https://github.com/mohanraj00/nookku/issues/213)).
+- **Old kit hooks.** `nookku init` replaces the hooks that verbatim-relay wrote, so the harness runs one hook for each event ([#181](https://github.com/mohanraj00/nookku/issues/181)).
+
+## 0.3.1
+
+A docs release. The docs have new how-to guides for each task of a test, a glossary, 4 diagrams and facts that match 0.3.0. The code of the relays, the taps and the audit does not change. The examples get a Node entry and an HTTP entry, and the toy HTTP server no longer looks up the host name before it listens.
+
+- **Facts.** The docs show version 0.3.0, the real `init` output, the deny rule for reads of `.verbatim-relay`, the error texts of 0.3.0 and the missing behavior of `mode`, `end`, `check`, `transcript` and `verify` ([#126](https://github.com/mohanraj00/nookku/issues/126)).
+- **Glossary.** [docs/reference/glossary.md](docs/reference/glossary.md) defines 20 terms, and each term links to its section of SPEC.md ([#127](https://github.com/mohanraj00/nookku/issues/127)).
+- **Choose a relay.** [docs/how-to/choose-a-relay.md](docs/how-to/choose-a-relay.md) compares the plugin and the hook kit, with a decision flow. The Claude Code guide is now 2 guides, one for each relay ([#128](https://github.com/mohanraj00/nookku/issues/128)).
+- **Test your own app.** [docs/how-to/test-your-app.md](docs/how-to/test-your-app.md) takes your app from the install to the results, with the expected output at each step. The docs give `!verbatim-relay setup` first, so the harness runs the command ([#129](https://github.com/mohanraj00/nookku/issues/129), [#125](https://github.com/mohanraj00/nookku/issues/125)).
+- **Read the results.** [docs/how-to/read-the-results.md](docs/how-to/read-the-results.md) reads the seal, the audit, the findings, the report and the transcript in order, with a table of the 7 break classes and the 12 trace checks ([#130](https://github.com/mohanraj00/nookku/issues/130)).
+- **What check proves.** The docs say that a PASS of `check` proves the connection, not the reply, and give the steps to make sure that `check` runs your real app ([#131](https://github.com/mohanraj00/nookku/issues/131)).
+- **Other agents.** An entry in Node ([examples/toy-shop-node/entry.mjs](examples/toy-shop-node/entry.mjs)), an entry for an HTTP server ([examples/toy-shop/http_entry.py](examples/toy-shop/http_entry.py)), a table that compares the entry with the HTTP tap, and [a guide for a streaming agent](docs/how-to/test-a-streaming-agent.md) ([#132](https://github.com/mohanraj00/nookku/issues/132)).
+- **Stuck tests, upgrades and removal.** [Recover a stuck test](docs/how-to/recover-a-stuck-test.md), [upgrade](docs/how-to/upgrade.md) and [remove](docs/how-to/remove.md) verbatim-relay ([#133](https://github.com/mohanraj00/nookku/issues/133)).
+- **Diagrams.** [docs/architecture.md](docs/architecture.md) has a corrected data-flow diagram, and new diagrams of one turn, the test lifecycle and the trace, with a table of the timeouts. Tests check the diagrams against the records page and the constants ([#134](https://github.com/mohanraj00/nookku/issues/134)).
+- **Navigation.** The README gives a route to try the toy shop, a route to connect your own app and a route to choose a relay, and a Docs table in task order. The docs map groups the guides by task. Troubleshooting starts with a table of symptoms ([#135](https://github.com/mohanraj00/nookku/issues/135)).
+
+## 0.3.0
+
+Streamed replies, and the fixes of the 0.2 code review. An agent can now stream its reply, and the relays still show and record it byte for byte. Each relay path fails closed: if a hook or a guard fails, the prompt does not reach the model, and the tester sees the cause and the next step. Each record has one reader with one rule in Python and in the plugin. Both taps have one deadline, and the proxies keep no secret query value. The docs have a tutorial, how-to guides, reference pages and an architecture page. [The proofs](docs/results.md) pass in the plugin, the hook kit in Claude Code and the hook kit in Codex.
+
+**Upgrade.** End each running test before you upgrade. `current.json` now needs `pid_start` ([#44](https://github.com/mohanraj00/nookku/issues/44)), so 0.3.0 treats a test that 0.2.0 started as not running.
+
+- **Streamed replies.** If the agent answers with an SSE stream (`text/event-stream`), the tap sends each part to the relay when it comes, and records the complete reply when the stream ends. The relay shows the reply when the stream is complete. A stream that ends early, sends an error or has a malformed chunk gives an error, not a part of the reply. The option `openai_stream` (`--openai-stream` in the hook kit) makes both relays send `"stream": true`, for an agent that streams only on request. [SPEC.md section 4.1](SPEC.md#41-http-mode) defines the rules ([#1](https://github.com/mohanraj00/nookku/issues/1), [#66](https://github.com/mohanraj00/nookku/issues/66)).
+- **Record format v0.3.** The row of a failed stream (a 2xx status with `reply: null`) has version `0.3`. Each other row keeps `0.2`, so a 0.2.0 reader still reads it, and refuses a 0.3 row with a message that names the version. Readers accept `0.1`, `0.2` and `0.3` ([SPEC.md section 2](SPEC.md#2-record-format), [#66](https://github.com/mohanraj00/nookku/issues/66)).
+- **Stream errors.** If the tap cannot write the row of a stream, it sends no second status and writes no second row. With a `Content-Length`, it holds the last byte until the row is in the record, so the caller sees an incomplete response. The model API proxy and the tap read SSE with one strict parser ([#66](https://github.com/mohanraj00/nookku/issues/66)).
+- **Fail closed in the plugin.** Each gating hook of the plugin has a `.catch` handler. If a hook fails or runs past its time budget, the handler blocks the prompt or denies the tool call, and it makes no file or state call that can stall again ([#63](https://github.com/mohanraj00/nookku/issues/63)).
+- **Fail closed in the hook kit.** A PreToolUse deny with a broken config, or with an error in the guard, still denies the call in relay mode or during a test ([#68](https://github.com/mohanraj00/nookku/issues/68)).
+- **Stale test.** If the plugin cannot reach the tap of `current.json`, it asks `verbatim-relay status`. If no test runs, it shows "the test stopped, and no test runs" and the next step. The turn stays in the record with `ok: false`, because the prompt can have reached the tap ([#64](https://github.com/mohanraj00/nookku/issues/64)).
+- **Bridge identity.** `current.json` has `pid_start`, the start time of the bridge process. A test whose pid now belongs to another process is not running, and `end` sends no signal to that process ([#44](https://github.com/mohanraj00/nookku/issues/44)).
+- **One reader.** Each part reads a record with `record.read_rows`, and the plugin reads the relay record with `relayTurns` and the same rule. A bad line stops the reader with an error that names the file and the line. The transcript of an invalid record shows no turn. The shared table `RELAY_LINES` checks the 2 readers against the conformance cases. If `relay.jsonl` is invalid at the end of a test, the bridge infers no Codex session ([#71](https://github.com/mohanraj00/nookku/issues/71)).
+- **One contract reader.** `contract.parse_reply` and the plugin refuse a line with both `reply` and `error`, and `NaN` or `Infinity`. The shared table `CONTRACT_LINES` checks both readers against the contract cases ([#45](https://github.com/mohanraj00/nookku/issues/45)).
+- **One config rule.** `start`, `check`, `init` and the hook kit read `.verbatim-relay/config.json` with one reader. An unknown key stops each of them with the same error. Before, `start` accepted it, and then the hook kit blocked each prompt. A second `init` keeps each key of the file, for example `backends` and `"evaluate": false`, and changes only the keys of its flags. It prints the changed and the kept keys ([SPEC.md section 7.1](SPEC.md#71-configuration), [#89](https://github.com/mohanraj00/nookku/issues/89)).
+- **Lone surrogates.** A row with a lone surrogate is invalid (`record_invalid`), not a crash. The relays refuse a prompt with a lone surrogate and say why. The trace, the OTLP receiver and the proxies write a lone surrogate as its escape text, so a write never fails ([#43](https://github.com/mohanraj00/nookku/issues/43), [#65](https://github.com/mohanraj00/nookku/issues/65)).
+- **Stray lines in stdio mode.** The stdio tap reads lines until it gets the line for the request id. A log line on stdout is a stray line with its own `unparsed` row, not a 502. A timeout after stray lines names them and gives the fix. `serve()` sends the output that the entry printed before it to stderr ([#62](https://github.com/mohanraj00/nookku/issues/62), [#46](https://github.com/mohanraj00/nookku/issues/46)).
+- **Check gate.** `verbatim-relay check` reads the audit of its own test. If the audit exit is not 0, the check fails, with one FAIL line and its fix for each error and each break. If `audit.json` is missing or not valid, the check fails. A line that the entry writes on stdout after its last reply is a stray line too ([SPEC.md section 7.1](SPEC.md#71-configuration), [section 4.2](SPEC.md#42-stdio-mode), [#90](https://github.com/mohanraj00/nookku/issues/90)).
+- **Blocked-call detail.** Both relays cut the `detail` of a `blocked_call` row at 300 code points, so a cut never splits a surrogate pair and the relay record stays valid. The shared table `DETAIL_CASES` checks both relays ([SPEC.md section 2.2](SPEC.md#22-relay-record), [#88](https://github.com/mohanraj00/nookku/issues/88)).
+- **Record errors in the view.** In follow mode, `view` writes an error for an invalid relay record to stderr, and shows the next turns when the record is valid again. `view --no-follow` writes the error and exits with 2 ([docs/reference/cli.md](docs/reference/cli.md), [#92](https://github.com/mohanraj00/nookku/issues/92)).
+- **Start and status texts.** The start text of both relays names the prompt `verbatim-relay end`, which ends the test and starts the evaluation, and the end with no evaluation. The plugin status asks `verbatim-relay status --json`, so a stale `current.json` does not show as a running test. `--help` does not show the internal `bridge` command ([SPEC.md section 5](SPEC.md#5-relays), [#91](https://github.com/mohanraj00/nookku/issues/91)).
+- **One deadline.** Both taps wait [240 seconds](src/nookku/stdio.py) for the agent, as one deadline for the connection, the headers and the body, also for a stream. Each timeout of the relay path ends before the next one, and [one test](tests/test_timeouts.py) checks the order ([SPEC.md section 4.3](SPEC.md#43-timeouts), [#67](https://github.com/mohanraj00/nookku/issues/67)).
+- **Deny.** Both relays deny a model command that writes to `/dev/tcp` or `/dev/udp` paths of the tap or the agent, also with leading zeros in the port. The shared table `DENY_CASES` checks both deny patterns ([#70](https://github.com/mohanraj00/nookku/issues/70)).
+- **No secret query values.** The backend and model API proxies remove the values of secret query parameters from the record, and forward them with no change ([#69](https://github.com/mohanraj00/nookku/issues/69)).
+- **Transcript.** The plugin and the hook kit give the model one rendered transcript, with a legend for `ok` and for a relay error, and with no hashes ([SPEC.md section 5](SPEC.md#5-relays), [#10](https://github.com/mohanraj00/nookku/issues/10)).
+- **OpenAI Responses and Decisions APIs.** The model API proxy reads the result of `/responses` calls, also streamed, and of `/decisions` calls. The trace and the evaluation view show them ([SPEC.md section 7.7](SPEC.md#77-model-api-proxies), [#33](https://github.com/mohanraj00/nookku/issues/33), [#40](https://github.com/mohanraj00/nookku/issues/40)).
+- **Isolated Agent SDK session.** The toy apps start their Agent SDK session with `setting_sources=[]`, `strict_mcp_config=True` and no plugin folders. The new trace check `server_not_from_app` names an MCP server that the app's configuration cannot give ([#28](https://github.com/mohanraj00/nookku/issues/28)).
+- **Docs.** A one-screen README, a docs map, a tutorial ([docs/getting-started.md](docs/getting-started.md)), how-to guides, reference pages for the CLI, the config and the records, an architecture page, troubleshooting and a FAQ. A code of conduct (Contributor Covenant 2.1), issue and PR templates, and a code quality spec ([docs/code-quality.md](docs/code-quality.md)). Tests check each relative link, the README counts, and the CLI and config pages against the code ([#72](https://github.com/mohanraj00/nookku/issues/72), [#73](https://github.com/mohanraj00/nookku/issues/73)).
+- **Claims.** The README marks the prompt-only examples as single observations, and the benchmark text says that each turn was a new harness call ([#9](https://github.com/mohanraj00/nookku/issues/9), [#8](https://github.com/mohanraj00/nookku/issues/8)).
+- **CI.** GitHub Actions are pinned to commit SHAs ([#4](https://github.com/mohanraj00/nookku/issues/4)).
+- **Counts.** 53 audit cases in [conformance/cases/](conformance/cases/), 20 contract cases in [conformance/contract/](conformance/contract/), 10 trace cases in [conformance/trace/](conformance/trace/), 11 seal cases in [conformance/seal/](conformance/seal/) and 7 receiver cases in [conformance/otlp/](conformance/otlp/) ([cases](conformance/build.py)). The trace has [12 checks](src/nookku/trace.py).
+
+## 0.2.0
+
+The telemetry MVP ([#27](https://github.com/mohanraj00/nookku/issues/27)). For each turn of a test, the record shows each action of the app that can change the reply or the state: its model sessions, its OpenTelemetry spans and logs, its backend calls and its direct model calls. Each action has its turn and its evidence line. The seal shows a changed record, and the evaluation at the end of a test cites these items. [The full telemetry proof](docs/results.md#full-telemetry-with-one-app) passes in the plugin, the hook kit in Claude Code and the hook kit in Codex.
+
+- **Tests.** The harness starts the app. `verbatim-relay start` (or `/verbatim-relay start` in the plugin) starts an entry through the tap and switches relay mode on. `verbatim-relay end` stops it. The agent needs no HTTP server and no extra terminal. Each test has a folder in `.verbatim-relay/tests/` with both records, `app.log` and a manifest.
+- **Agent contract v1.** One JSON line in and one JSON line out on stdin and stdout: a message, the history and a session in; a reply or an error out. `verbatim_relay.agent.serve(reply)` speaks it for a Python function. The tap's `--cmd` mode starts the agent and speaks it.
+- **Model sessions.** The tap finds the app's own Claude Code sessions by their process, and its Codex sessions by their folder and time. At the end, it copies their session files into the test folder.
+- **Trace.** At the end of a test, the bridge reads the copied session files into `trace.jsonl`: each message, tool call and command of the app's model sessions, with its result, its turn and its line in the session file. 11 checks write `findings.json`, with the checks of the later entries below. `verbatim-relay trace` builds both again. [SPEC.md section 8](SPEC.md#8-trace) defines them, and 7 conformance cases in [conformance/trace/](conformance/trace/) test them ([cases](conformance/build.py), [test](tests/test_trace.py)).
+- **Evaluation.** The prompt `verbatim-relay end` ends the test and gives the harness model the evaluation prompt, in the plugin and in the hook kit. The model reads the transcript with the trace (`verbatim-relay transcript --trace`), the findings and the audit, checks the app, and writes `report.md` in the test folder. `"evaluate": false` in the configuration stops it. After a test, the relays deny model writes to a test folder, except `report.md`. [SPEC.md section 9](SPEC.md#9-evaluation) defines it. [Proofs P5 and P6](docs/results.md#p5-and-p6-the-evaluation-at-the-end-of-a-test) pass with the plugin and the hook kit in Claude Code, and with the hook kit in Codex: the report finds a planted business-rule bug.
+- **Deny of the entry and the records.** During a test, both relays deny a model command that runs the entry, for example `python entry.py`. After a test, a model command that names `.verbatim-relay` can only read, or write `report.md`. A list of read programs in [SPEC.md section 5](SPEC.md#5-relays) defines "only read". Each other command and each tool that is not a shell is denied ([#20](https://github.com/mohanraj00/nookku/issues/20)).
+- **Seal.** At the end of a test, the bridge writes the SHA-256 of each file of the test folder to `seal.json`, and a copy to `~/.verbatim-relay/seals/`. `verbatim-relay verify` shows each file that changed after the end, and exits with 2 if the seal is broken. `transcript --trace` shows the result in its first line. `verbatim-relay trace` rebuilds the trace only from sources that agree with the seal. Denies after a test go to `denied.jsonl`. [SPEC.md section 7.4](SPEC.md#74-seal) defines the seal, and 11 conformance cases in [conformance/seal/](conformance/seal/) test it ([#21](https://github.com/mohanraj00/nookku/issues/21)).
+- **OpenTelemetry.** During a test, the bridge runs an OTLP/HTTP receiver and gives the entry its address in the `OTEL_*` variables. It takes JSON and protobuf (with a decoder that uses only the standard library), also with gzip, and writes `otel.jsonl`. It drops metrics and the spans of the harnesses, keeps the prompt, tool and reply events of the harnesses, and removes personal attributes. The trace reads `otel.jsonl` as `span` and `log` items (trace rows `0.3`, with the new field `service`), with 2 new checks: `span_error`, and `otel_tool_not_in_session`, which compares the tool events of the harness with its session file. `"otel": false` stops it. [SPEC.md section 7.5](SPEC.md#75-otlp-receiver) defines it. 6 receiver cases in [conformance/otlp/](conformance/otlp/) and 1 trace case test it ([#23](https://github.com/mohanraj00/nookku/issues/23)).
+- **Backend calls.** The config key `backends` lists the HTTP services of the app. During a test, the bridge runs a recording proxy for each one, and gives the entry the proxy URL in the backend's environment variable. The proxy forwards each byte, also to an HTTPS backend, and writes `backend.jsonl` without the values of secret headers. The trace reads it as `http` items, with the new check `backend_error`. [SPEC.md section 7.6](SPEC.md#76-backend-proxies) defines it, the trace case `backend_toy_shop` tests it, and [proofs/backend/](proofs/backend/results.json) shows 3 calls with the same bytes at the entry, the service and the record ([#24](https://github.com/mohanraj00/nookku/issues/24)).
+- **Direct model calls.** During a test, the bridge runs a recording proxy for the Anthropic and the OpenAI API, and gives the entry its URL in `ANTHROPIC_BASE_URL` and `OPENAI_BASE_URL`. The proxy sends each part of a streamed (SSE) response to the app when it comes, and writes `model_api.jsonl` with the text, the tool calls and the stop reason of each call. It removes the API key from the record. It keeps no text of a call from a harness, for example the Agent SDK, because that call holds the harness's own instructions. The trace reads the other calls as `message` and `tool_call` items, with the new check `model_api_error`. `"model_api": false` stops it. [SPEC.md section 7.7](SPEC.md#77-model-api-proxies) defines it, the trace case `model_api_toy_shop` tests it, and [proofs/model-api/](proofs/model-api/results.json) shows 3 calls with the same bytes at the entry, the toy API and the record, and 2 streams that reached the entry before their end ([#25](https://github.com/mohanraj00/nookku/issues/25)).
+- **Example.** `examples/toy-shop-models/` is a toy shop with a Claude Agent SDK session, 2 tools and a planted bug.
+- **Full example.** `examples/toy-shop-full/` is a toy shop with 3 parts: a Claude Agent SDK session with 3 tools, a stock service over HTTP, and a direct call to a case-note model with the OpenAI Chat Completions API and a stream. Its planted bug shows only in the backend calls. `toy_model.py` is a toy note model that needs no API key.
+- **Full telemetry proof.** `scripts/proof_telemetry.py` runs a test of `examples/toy-shop-full/` in each relay. Each turn has items from the Agent SDK session file, the backend proxy and the model API proxy, and each report cites the 2 backend calls of the planted bug ([results](docs/results.md#full-telemetry-with-one-app), [#26](https://github.com/mohanraj00/nookku/issues/26)).
+- **Agent SDK pin.** The proof scripts take the Agent SDK version from one pin in `scripts/proof_common.py`, now 0.2.164. The trace readers now also cover Claude Code 2.1.292, which this SDK bundles ([proofs/trace/](proofs/trace/results.json), [#35](https://github.com/mohanraj00/nookku/issues/35)). The proof scripts split model output on `\n` only ([#31](https://github.com/mohanraj00/nookku/issues/31)).
+- **Model API URLs in the config.** `model_api` can also be an object from an API name to its upstream URL, for example when the tester's harness reads the same variable ([SPEC.md section 7.7](SPEC.md#77-model-api-proxies)).
+- **Setup.** `verbatim-relay setup` prints the guide that a harness model follows to write the entry for an app. `verbatim-relay check` runs a test with one message and checks the reply and the model sessions. The plugin has a `setup` skill.
+- **Hook kit.** The prompts `verbatim-relay start`, `verbatim-relay end` and `verbatim-relay status` control a test from inside Codex or Claude Code. During a test, both relays deny model changes to `.verbatim-relay/`.
+- **Record format v0.2.** The `exchange` row gets `started`. The tap record gets the `model_session` row. The audit reads v0.1 and v0.2 rows and counts the model sessions. The matching does not change.
+- **Fix.** The hook kit read the relay record with `str.splitlines()`, which also splits on U+2028. After a tester message with that character, each later message failed closed. The kit now splits on `\n` only.
+- **Examples.** The toy shop agent speaks the agent contract. The HTTP version is `examples/toy-shop/http_agent.py`.
+- **Docs.** A worked evaluation of the toy shop agent, with what the model found and what it got wrong: `docs/evaluation-example.md`.
+
+## 0.1.0
+
+The first release.
+
+- **Relays.** A Claude Code plugin (function hooks) shows each reply in the chat. A hook kit for Codex and Claude Code (classic hooks) shows each reply in `verbatim-relay view`. In relay mode, the model does not run.
+- **Transcript.** After the test, the model reads the exact conversation: the plugin's `transcript` tool, or `verbatim-relay transcript`.
+- **Tap.** `verbatim-relay tap` records what the agent received and sent. Adapters: a JSON body with configurable field paths, and OpenAI-compatible chat completions.
+- **Audit.** `verbatim-relay audit` compares the two records and reports 7 break classes. It exits 0, 1 or 2, and it fails closed.
+- **Spec and evidence.** Record format v0.1 in SPEC.md with 33 conformance cases. Proofs P1 to P5 on Claude Code 2.1.288 and Codex 0.160.0, and a pre-registered benchmark: 0 breaks in 1,000 turns.

nookku-0.4.0/LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

nookku-0.4.0/PKG-INFO

@@ -0,0 +1,152 @@
+Metadata-Version: 2.5
+Name: nookku
+Version: 0.4.0
+Summary: A test harness for developers of agent apps. Test your agent through Claude Code or Codex, byte for byte, and audit the record.
+Project-URL: Repository, https://github.com/mohanraj00/nookku
+Project-URL: Documentation, https://github.com/mohanraj00/nookku/blob/main/docs/index.md
+Project-URL: Specification, https://github.com/mohanraj00/nookku/blob/main/SPEC.md
+Project-URL: Changelog, https://github.com/mohanraj00/nookku/blob/main/CHANGELOG.md
+Project-URL: Issues, https://github.com/mohanraj00/nookku/issues
+Author: Mohan Raj
+License-Expression: Apache-2.0
+License-File: LICENSE
+Keywords: agents,audit,claude-code,codex,llm,testing
+Classifier: Development Status :: 2 - Pre-Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Testing
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+
+<picture>
+  <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/mohanraj00/nookku/main/docs/assets/logo/nookku-wordmark-dark.svg">
+  <img src="https://raw.githubusercontent.com/mohanraj00/nookku/main/docs/assets/logo/nookku-wordmark.svg" alt="Nookku" width="332">
+</picture>
+
+# Nookku
+
+(நோக்கு - to observe)
+
+A test harness for developers of agent apps.
+
+[![CI](https://github.com/mohanraj00/nookku/actions/workflows/ci.yml/badge.svg)](https://github.com/mohanraj00/nookku/actions/workflows/ci.yml)
+[![PyPI](https://img.shields.io/pypi/v/nookku)](https://pypi.org/project/nookku/)
+[![Python versions](https://img.shields.io/pypi/pyversions/nookku)](https://pypi.org/project/nookku/)
+[![License](https://img.shields.io/github/license/mohanraj00/nookku)](https://github.com/mohanraj00/nookku/blob/main/LICENSE)
+
+**Test your chat agent through Claude Code or Codex. The [harness](https://github.com/mohanraj00/nookku/blob/main/docs/reference/glossary.md#harness) model does not retype one message or one reply.**
+
+Nookku is a test harness for developers of agent apps. You talk to your agent in the coding harness where you already work. The [relay](https://github.com/mohanraj00/nookku/blob/main/docs/reference/glossary.md#relay) sends each message to your agent byte for byte, and shows each reply byte for byte. The model does not run while you talk. At the end, the model reads the exact record and evaluates your agent: business logic, tone, accuracy.
+
+It adapts to your app through a thin [entry](https://github.com/mohanraj00/nookku/blob/main/docs/reference/glossary.md#entry) in `.nookku/`, and your app's code does not change. The cost is plumbing: you write the entry once, and you change it when the start or the wiring of your app changes.
+
+## How it works
+
+```text
+tester ──> harness ──> relay ──────> tap ──stdin/stdout──> entry ──> your app
+                         │            │
+                         ▼            ▼
+                    relay.jsonl    tap.jsonl
+            (what you typed      (what the agent
+             and saw)             received and sent)
+                         │            │
+                         └─> audit <──┘   exit 0 clean, 1 break, 2 invalid record
+```
+
+- [Harness](https://github.com/mohanraj00/nookku/blob/main/docs/reference/glossary.md#harness): Claude Code or Codex, where you type your test messages.
+- [Relay](https://github.com/mohanraj00/nookku/blob/main/docs/reference/glossary.md#relay): the command `nookku hook`, which the plugin or the project hooks run in the harness. It carries each message and each reply, and the model writes neither.
+- [Tap](https://github.com/mohanraj00/nookku/blob/main/docs/reference/glossary.md#tap): a proxy in front of your agent. It forwards each byte with no change, and records what the agent received and sent.
+- [Entry](https://github.com/mohanraj00/nookku/blob/main/docs/reference/glossary.md#entry): a thin wrapper that starts your app and speaks one JSON line in and one JSON line out.
+- [Audit](https://github.com/mohanraj00/nookku/blob/main/docs/reference/glossary.md#audit): a program that compares the two records byte for byte, and names each break.
+
+[docs/architecture.md](https://github.com/mohanraj00/nookku/blob/main/docs/architecture.md#data-flow) shows this flow with each part, and has diagrams of one turn, the test lifecycle and the trace.
+
+Two processes write two records, and the audit compares them byte for byte. It reports 8 break classes ([SPEC.md section 3.3](https://github.com/mohanraj00/nookku/blob/main/SPEC.md#33-break-classes)). It fails closed: it never reports clean on a record that it cannot read. 62 audit cases, 20 contract cases, 18 trace cases, 11 seal cases and 7 receiver cases in [conformance/](https://github.com/mohanraj00/nookku/blob/main/conformance/) test the spec ([cases](https://github.com/mohanraj00/nookku/blob/main/conformance/build.py), [tests](https://github.com/mohanraj00/nookku/blob/main/tests/)).
+
+A test also records what your app did. It copies the session files of the app's own Agent SDK or Codex sessions. It records the backend calls, the direct model calls and the OpenTelemetry spans of the app. [docs/architecture.md](https://github.com/mohanraj00/nookku/blob/main/docs/architecture.md) shows each part.
+
+## Install
+
+```bash
+uv tool install nookku
+```
+
+It needs Python 3.10 or later, and it has no runtime dependencies ([pyproject.toml](https://github.com/mohanraj00/nookku/blob/main/pyproject.toml)). The docs are for version 0.4.0. If you use verbatim-relay 0.3.x or earlier, do the [0.4.0 steps of the upgrade guide](https://github.com/mohanraj00/nookku/blob/main/docs/how-to/upgrade.md#040), because the package has a new name ([CHANGELOG.md](https://github.com/mohanraj00/nookku/blob/main/CHANGELOG.md#040)). To upgrade from a later version of nookku, end each running test first. Then run `uv tool upgrade nookku`. [docs/how-to/upgrade.md](https://github.com/mohanraj00/nookku/blob/main/docs/how-to/upgrade.md) gives each step.
+
+## Quick start
+
+Select one route: try the toy shop, connect your app, or choose a relay.
+
+### Try the toy shop
+
+In a clone of this repo, with the toy shop agent and the project hooks in Claude Code:
+
+```bash
+nookku init claude-code --entry "python3 examples/toy-shop/agent.py"
+nookku check        # one message through the entry: PASS proves the connection, not the reply
+nookku view         # in a second terminal: each reply shows here
+```
+
+In Claude Code, type the prompt `nookku start`, then your test messages, then `nookku end`. The end of the test writes `audit.json` in the [test folder](https://github.com/mohanraj00/nookku/blob/main/docs/reference/glossary.md#test-folder) `.nookku/tests/<test-id>/`. Then the model evaluates the test and writes `report.md`. [docs/getting-started.md](https://github.com/mohanraj00/nookku/blob/main/docs/getting-started.md) shows each step with its real output.
+
+Optional: to see the audit again, run it by hand:
+
+```bash
+nookku audit --tap .nookku/tests/<test-id>/tap.jsonl --relay .nookku/tests/<test-id>/relay.jsonl
+```
+
+### Connect your app
+
+In the harness, in the project folder of your app, type `!nookku setup`. Then type the prompt "Follow the Nookku setup guide, and connect a test to this app." [docs/how-to/test-your-app.md](https://github.com/mohanraj00/nookku/blob/main/docs/how-to/test-your-app.md) gives each step from the install to the results, with the expected output. To write the entry yourself, read [docs/how-to/connect-your-agent.md](https://github.com/mohanraj00/nookku/blob/main/docs/how-to/connect-your-agent.md).
+
+### Choose a relay
+
+You install the [relay](https://github.com/mohanraj00/nookku/blob/main/SPEC.md#5-relays) as the plugin or as the project hooks. Both work in Claude Code and in Codex, with the same rules. Both show each reply as the reason of a blocked prompt, and in `nookku view`. [docs/how-to/choose-a-relay.md](https://github.com/mohanraj00/nookku/blob/main/docs/how-to/choose-a-relay.md) compares them. Then read the guide of your setup: [Claude Code plugin](https://github.com/mohanraj00/nookku/blob/main/docs/how-to/claude-code-plugin.md), [Claude Code project hooks](https://github.com/mohanraj00/nookku/blob/main/docs/how-to/claude-code-hook-kit.md) or [Codex](https://github.com/mohanraj00/nookku/blob/main/docs/how-to/codex.md).
+
+## Results
+
+| Proof | Plugin, Claude Code 2.1.296 | Plugin, Codex 0.162.1 | Project hooks, Claude Code 2.1.296 | Project hooks, Codex 0.162.1 |
+|---|---|---|---|---|
+| Messages reach the agent byte for byte | 10/10 | 10/10 | 10/10 | 10/10 |
+| Replies reach the tester byte for byte | 10/10 | 10/10 | 10/10 | 10/10 |
+| Same, with a system prompt that tells the model to rewrite both | 5/5 | 5/5 | 5/5 | 5/5 |
+| Model call to the agent denied, agent receives nothing | yes | yes | yes | yes |
+| Audit finds planted faults | 5/5 | 5/5 | 5/5 | 5/5 |
+| After the test, the model has no memory of the conversation, and reads it from the transcript | yes | yes | yes | yes |
+
+Data: [plugin in Claude Code](https://github.com/mohanraj00/nookku/blob/main/proofs/plugin-claude-code/results.json), [plugin in Codex](https://github.com/mohanraj00/nookku/blob/main/proofs/plugin-codex/results.json), [project hooks in Claude Code](https://github.com/mohanraj00/nookku/blob/main/proofs/hooks-claude-code/results.json), [project hooks in Codex](https://github.com/mohanraj00/nookku/blob/main/proofs/hooks-codex/results.json), [evaluation](https://github.com/mohanraj00/nookku/blob/main/docs/results.md#p5-the-model-judges-the-record-not-its-memory). Method: [docs/results.md](https://github.com/mohanraj00/nookku/blob/main/docs/results.md#1-proofs).
+
+Under pressure, the mechanism had **0 breaks in 1,000 turns**. The test had 40 scripted conversations in each harness, 5 or 20 turns long. They had refusals, HTTP 500 errors, questions back to the tester and ambiguous messages. Each turn was a new harness call. I registered the design before the first run. Method, data and the one deviation: [docs/results.md](https://github.com/mohanraj00/nookku/blob/main/docs/results.md#2-benchmark-under-pressure).
+
+## Docs
+
+The how-to rows are in the order of the tasks of a test: choose, connect, run, read the results, fix. The row after them has the tasks that maintain Nookku.
+
+| Kind | Pages |
+|---|---|
+| Tutorial | [Get started](https://github.com/mohanraj00/nookku/blob/main/docs/getting-started.md) |
+| How-to: choose | [Choose a relay](https://github.com/mohanraj00/nookku/blob/main/docs/how-to/choose-a-relay.md) |
+| How-to: connect your app | [Test your own app](https://github.com/mohanraj00/nookku/blob/main/docs/how-to/test-your-app.md), [Connect your agent](https://github.com/mohanraj00/nookku/blob/main/docs/how-to/connect-your-agent.md), [HTTP tap](https://github.com/mohanraj00/nookku/blob/main/docs/how-to/http-tap.md), [Test a streaming agent](https://github.com/mohanraj00/nookku/blob/main/docs/how-to/test-a-streaming-agent.md), [Isolate an Agent SDK session](https://github.com/mohanraj00/nookku/blob/main/docs/how-to/isolate-agent-sdk.md), [Add a backend](https://github.com/mohanraj00/nookku/blob/main/docs/how-to/add-a-backend.md), [Model calls and OpenTelemetry](https://github.com/mohanraj00/nookku/blob/main/docs/how-to/record-model-calls.md) |
+| How-to: run a test | [Claude Code plugin](https://github.com/mohanraj00/nookku/blob/main/docs/how-to/claude-code-plugin.md), [Claude Code project hooks](https://github.com/mohanraj00/nookku/blob/main/docs/how-to/claude-code-hook-kit.md), [Codex](https://github.com/mohanraj00/nookku/blob/main/docs/how-to/codex.md) |
+| How-to: read the results | [Read the results of a test](https://github.com/mohanraj00/nookku/blob/main/docs/how-to/read-the-results.md) |
+| How-to: fix | [Troubleshooting](https://github.com/mohanraj00/nookku/blob/main/docs/troubleshooting.md), [Recover a stuck test](https://github.com/mohanraj00/nookku/blob/main/docs/how-to/recover-a-stuck-test.md) |
+| How-to: maintain | [Upgrade](https://github.com/mohanraj00/nookku/blob/main/docs/how-to/upgrade.md), [Remove](https://github.com/mohanraj00/nookku/blob/main/docs/how-to/remove.md), [Run the proofs](https://github.com/mohanraj00/nookku/blob/main/docs/how-to/run-the-proofs.md) |
+| Reference | [CLI](https://github.com/mohanraj00/nookku/blob/main/docs/reference/cli.md), [Configuration](https://github.com/mohanraj00/nookku/blob/main/docs/reference/config.md), [Records](https://github.com/mohanraj00/nookku/blob/main/docs/reference/records.md), [Glossary](https://github.com/mohanraj00/nookku/blob/main/docs/reference/glossary.md), [SPEC.md](https://github.com/mohanraj00/nookku/blob/main/SPEC.md), [Results](https://github.com/mohanraj00/nookku/blob/main/docs/results.md) |
+| Explanation | [Architecture](https://github.com/mohanraj00/nookku/blob/main/docs/architecture.md), [Limits](https://github.com/mohanraj00/nookku/blob/main/docs/limits.md), [FAQ](https://github.com/mohanraj00/nookku/blob/main/docs/faq.md), [Worked evaluations](https://github.com/mohanraj00/nookku/blob/main/docs/evaluation-example.md) |
+
+The full map is [docs/index.md](https://github.com/mohanraj00/nookku/blob/main/docs/index.md). Read [docs/limits.md](https://github.com/mohanraj00/nookku/blob/main/docs/limits.md) before you trust a result: the deny is best effort, and the entry is not audited.
+
+## Contribute
+
+Read [CONTRIBUTING.md](https://github.com/mohanraj00/nookku/blob/main/CONTRIBUTING.md) and the [code of conduct](https://github.com/mohanraj00/nookku/blob/main/CODE_OF_CONDUCT.md). Report a security problem as [SECURITY.md](https://github.com/mohanraj00/nookku/blob/main/SECURITY.md) says.
+
+## License
+
+Apache-2.0. See [LICENSE](https://github.com/mohanraj00/nookku/blob/main/LICENSE).