imprint-mcp 0.2.0

package/CHANGELOG.md

@@ -0,0 +1,168 @@
+# Changelog
+
+All notable changes to Imprint. Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) (Added / Changed / Deprecated / Removed / Fixed / Security).
+
+## [0.2.0] — 2026-05-14
+
+v0.2 overhauls the teach-to-tool pipeline, adds full observability tracing, and introduces state-aware API replay. 12 new features, 7 bug fixes across 24 commits.
+
+### Added
+- **Multi-tool teach** — `imprint teach` captures multiple tools in a single session with shared request triage across tools for optimized token usage (#23, #29).
+- **Deterministic MCP tools** — teach produces functional, deterministic tool definitions that agents can call reliably (#28).
+- **Full-depth tracing** — LLM calls and agent loop iterations are traced end-to-end for debugging and observability (#31).
+- **Dotenv support** — `.env` file with tracing defaults, auto-loaded by the CLI via dotenv (#32).
+- **State-aware API replay** — replay engine tracks API state across requests for more reliable automation (#24).
+- **Credential manager** — catch login secrets during teach, store in OS keychain, replay via `${credential.X}` placeholders (#14).
+- **Interactive compile provider picker** — choose your LLM provider interactively when compiling (#17).
+- **Claude CLI + MCP compilation** — drive compile-agent via claude-cli and stdio MCP server (#11).
+- **Landing page** — Imprint website at `web/` (#22).
+- **Improved session redaction** — broader PII and credential coverage across content types (#20).
+- **Gitleaks pre-commit hook** — automated secret scanning before commits; fail-closed when gitleaks is not installed.
+- **Teach UX improvements** — keybinding hints in multi-tool select prompt (#26), auto-prompt for missing site and starting URL (#10).
+
+### Changed
+- **Examples layout** — tools now require nested directory structure per tool (#16).
+- **README** — added google-flights and google-hotels examples, removed southwest-seats.
+- **README** — polished for GitHub discoverability and conversion.
+
+### Fixed
+- Wire parserModule and preserve query params in compile-agent (#27).
+- Dispatch tool_use blocks in max_tokens responses (#30).
+- Fix credential/PII leaks in sessions with non-JSON content-types (#13).
+- Use correct field name `stopCount` in parser test (#18).
+- Store recordings outside examples directory (#19).
+- Replace hardcoded API key with env var.
+
+### Documentation
+- State-aware replay guide (#25).
+- CI test count badge.
+
+---
+
+## [0.1.0] — 2026-05-05 (post-sprint polish)
+
+A de-slop pass on the v0.1 codebase: deep audit + rearchitect to make the implementation sleek and the docs adoption-friendly. **No behavioral changes to existing verbs** — every demo still works (live-verified end-to-end against Southwest via stealth-fetch). One new verb (`imprint doctor`) added.
+
+### Added
+- **`imprint doctor` verb** — one-shot environment health check (Bun, Chromium binary, Playwright Chromium, Vertex env vars, push providers). Exits 0 on all-pass / 1 on any required failure (CI-friendly). Each failure includes a `→ next step:` hint. Surfaced in README quickstart + docs/troubleshooting.md as the first thing to try.
+- `docs/architecture.md` — data flow diagram, module map, backend ladder cost table, per-example file taxonomy.
+- `docs/glossary.md` — Session, Workflow, Playbook, Backend, Stealth-fetch, Sensor headers, Token TTL refresh, Sentinel, CDP, Credential store, NotifyWhen.
+- `docs/decisions.md` — 12 ADR-style entries (D1-D12) covering the load-bearing calls.
+- `docs/getting-started.md` — 5-minute walkthrough from clone to MCP tool in Claude Desktop.
+- `docs/troubleshooting.md` — predictable failure modes with the same `→ next step:` format the in-code error messages use.
+- `docs/notifications.md` — Pushover + ntfy setup, predicate language for `notifyWhen`.
+- `docs/security.md` — what Imprint stores, redaction guarantees, credential handling, vuln reporting flow.
+- `examples/discoverandgo/README.md` and `examples/echo/README.md` — tutorial-style READMEs (each: what / why interesting / run / what you should see / notes).
+- `CHANGELOG.md` — this file.
+- `CONTRIBUTING.md` — table-stakes contributor guide.
+- Per-verb help: `imprint <verb> --help` shows summary + usage + flags + a concrete example. The verb registry is single-source.
+- Actionable `→ next step:` hints in every user-reachable error message (Pushover/ntfy not set, missing playbook, missing param, no requests in workflow, etc.).
+- `resolveLadder` helper in `backend-ladder.ts` so cron + mcp-server share the auto-ladder expansion logic.
+- MCP tool descriptions now include the operator's recorded narration (`intent.userSaid`) — gives the LLM real context for picking the right tool.
+- Test coverage: `test/compile.test.ts` (+15 tests for shrinkSession + error paths), `test/cli-help.test.ts` (+26 drift-guard tests for VERB_HELP / dispatcher sync), `test/backend-ladder.test.ts` (+5 resolveLadder tests), `test/doctor.test.ts` (+7), additional `test/notify.test.ts` cases for multi-path pricePath. 132 → 193 tests.
+- `bun run check` package.json script — combined typecheck + lint + tests. CI now runs this directly.
+- `bun run imprint <verb>` and `bun run doctor` package.json shortcuts (alternatives to `bun link`).
+- `imprint emit` prints "next steps" after generating, removing the "I have a tool, now what?" friction point.
+- `imprint cron <unknown-site>` now lists the configured sites in the error message ("available sites: a, b, c").
+- "command not found" troubleshooting section explicitly covering the bun-link / PATH failure mode.
+- **Typo suggestions** for unknown verbs: `imprint recrod` → "did you mean `imprint record`?" via Levenshtein distance (≤ 3 edits AND ≤ half-length). Same UX pattern as git/bun. Tested.
+- **Capped cron success-log preview** at 500 chars (full payload still available via IMPRINT_DEBUG=1) so long-running daemons don't flood stderr. Southwest's ~100KB shopping response went from log-flooding to one-line.
+- **`--quiet` flag for `imprint cron`** (and `IMPRINT_QUIET=1` env var) — suppresses info logs on success so OS schedulers (cron, systemd, launchd) only mail/alert when something's actually broken. Failures still surface to stderr (separate code path).
+- **`isDebug()` / `isQuiet()` env helpers** — internal `=== '1'` semantics. Fixes a subtle bug where `if (process.env.IMPRINT_DEBUG)` was truthy for the string `"0"` (non-empty string coerces to true), so `IMPRINT_DEBUG=0` would actually enable debug mode.
+- **Per-verb missing-arg errors point to `--help`**: `imprint record` (no site) → "→ run `imprint record --help` for usage." Shared via the `requirePositional` helper so all 11 positional-taking verbs benefit.
+- **`--param k=v` malformed input** now shows a concrete example (`→ example: --param origin_airport_code=SJC`) instead of just "requires k=v form".
+- **Pipeline next-step hints**: `record` → `redact` → `generate` → `emit` → `cron`. Every successful verb now ends with the exact command for the next step, including the file path produced. A first-time user can chain the entire pipeline by following the printed commands.
+- **MCP no-tools error** suggests the bundled `echo` fixture so first-time users can verify the MCP wire-up before they've recorded anything.
+- **Cron expression error** now shows the format and links crontab.guru.
+- **Vertex project ID error** shows the exact `export ANTHROPIC_VERTEX_PROJECT_ID=…` command and points at `imprint doctor` for the rest of the env-var checklist.
+- **Workflow placeholder error** lists the params that *were* passed (or, if none, shows the exact `--param X=<value>` to add).
+- **`loadJsonFile()` helper** — `src/imprint/load-json.ts`. Single-source for the file-not-found / invalid-JSON / schema-mismatch error pipeline used by cron.ts, emit.ts, compile.ts, and the cli.ts redact case. -44 LOC across the four callers, plus user-friendly messages everywhere ZodError used to spill raw to stderr (cron.json `{badkey:1}` → "  - schedule: Required" + minimum example + docs link, instead of `[{code:'invalid_type',...}]`).
+- **Pipeline next-step hints** — `imprint redact`, `generate`, `compile-playbook`, `record`, `check`, `login`, `probe-backends`, `mcp-server` (no-tools path) all now print "next step: imprint <verb> ..." after a successful run or hint-worthy error, so the full pipeline can be chained without alt-tabbing to docs.
+- **isDebug() / isQuiet() env helpers** — `=== '1'` semantics (not truthy coercion), so `IMPRINT_DEBUG=0` actually disables. Replaces 8 raw `process.env.IMPRINT_DEBUG` checks across record.ts, chromium.ts, cron.ts, cli.ts.
+- **`tryParseParamKV()` helper** in cli.ts — return-null-on-error pattern eliminates the duplicated try/catch for the two `--param` callers.
+- **Test coverage gap-fills**: `test/check.test.ts` (+8 tests pinning warning heuristics), `test/load-json.test.ts` (+7 tests pinning the shared error shape), `test/sites.test.ts` (+5 tests pinning availableSitesHint branches).
+- **`availableSitesHint()` shared across cron / probe-backends / mcp-server** — extracted from cron.ts to `src/imprint/sites.ts`. Now `imprint probe-backends <typo>` and `imprint mcp-server --site <typo>` also list the actual sites under examples/ so users can spot a one-character typo.
+- **MCP server version single-sourced** — was hard-coded `'0.1.0'`; now reads from the same `VERSION` constant the CLI uses, eliminating a future drift hazard on package.json bumps.
+- **Vertex SDK error enrichment** — `llm.ts` wraps `messages.create` and translates the four common SDK errors (404 model not enabled in region, 401 not authenticated, 403 permission denied, 429 quota) into actionable messages with the exact `gcloud` / IAM / Model Garden link to fix. The raw SDK error is preserved as the JS `cause` chain (visible under `IMPRINT_DEBUG=1`). Documented in docs/troubleshooting.md.
+- **Credential store malformed-file error** — `loadCredentialStore` now throws with the file path + "→ delete and re-run \`imprint login <site>\`" hint instead of letting `JSON.parse` throw bare.
+- **Test coverage for the JSON extractor** — `test/llm.test.ts` (+9 tests for `extractJsonObject`'s fenced-block / nested-brace / escaped-quote branches).
+- **`assemble` next-step hint** — points users to `imprint check` after recovering a session.json from a partial .jsonl. Closes the last verb that was missing a next-step pointer.
+- **Credential placeholder error** matches the param error style — lists available credential keys and suggests re-running `imprint login` if the named key is missing. Helps when the credential store has a partial set (e.g. session_id captured but patron_id wasn't).
+- **`notifyWhen` decision logging** — every cron tick now logs `notifyWhen X: matched → pushing` or `notifyWhen X: no match (predicate ran, threshold not crossed)`. Silent no-match used to confuse users. Suppressed by `--quiet`.
+- **Vertex error troubleshooting docs** — `docs/troubleshooting.md` covers the four Vertex SDK error buckets (404 / 401 / 403 / 429) with the exact gcloud / IAM / Model Garden remediation for each.
+- **D13 + D14 added to `docs/decisions.md`** — three-tool dead-code defense (knip + tsc-strict + madge) and "user-friendly errors are worth the LOC" become ADR entries.
+- **`scripts/mcp-smoke-test.ts` fixed** — was using positional `discoverandgo` (wrong syntax for the `--site=<name>` flag) AND requiring a generated demo. Now defaults to the bundled `echo` fixture so it actually works on a clean checkout. Override via `IMPRINT_SMOKE_SITE=<other>`.
+- **Single source of truth for the backend enum** — `BackendsCacheSchema` previously duplicated `['fetch', 'stealth-fetch', 'playbook']` literally. Now derives from `ReplayBackendSchema.exclude(['auto'])`. Adding a new backend updates one place.
+- **Tighter type narrowing** — `runWithLadder` parameter changed from `ReplayBackend[]` to `ConcreteBackend[]` (= `Exclude<ReplayBackend, 'auto'>`); the unreachable `case 'auto'` defensive throw deleted.
+- **Zero dead code, enforced** — three-tool defense:
+  - `knip` (unused exports/files/types/deps)
+  - `tsc` (`noUnusedLocals` + `noUnusedParameters` enabled — unused locals/params/imports)
+  - `madge` (circular dependencies)
+  All three are part of `bun run check` and CI. Removed 19 unused value exports, 35 unused type exports, plus a duplicate alias. Internal-only Options/Result interfaces are no longer exported (TypeScript structural typing means callers don't need a named import). Adding new dead exports / circular deps / unused symbols now fails CI.
+
+### Changed
+- **Module reshape**: clearer file boundaries.
+  - `replay-backend.ts` → `backend-ladder.ts` (clearer name; "replay" was jargon)
+  - `workflow-runtime.ts` → `runtime.ts` (the prefix was redundant inside `imprint/`)
+  - `discover-tools.ts` → `tool-loader.ts`
+  - `playbook-types.ts` → folded into `types.ts`
+- **Compiler unification**: `generate.ts` (208 LOC) + `playbook-compiler.ts` (130 LOC) collapsed into `compile.ts` (320 LOC). Skeleton (read session → redact → slim → call LLM → parse → validate → write) shared; per-task differences (slim/prompt/parser/schema) parameterized via a `CompileTask<T>` config.
+- **README**: rewritten for adoption — leads with value prop + 60-second quickstart + demo table. Verb table replaced with pointer to `imprint --help`. ~285 → ~109 lines.
+- **CLI HELP**: top-level reorganized into CAPTURE / COMPILE / RUN groups with one-liner per verb; pointer to per-verb help.
+- **CLAUDE.md**: trimmed from 60 lines of pre-sprint design doc to a slim ~30-line agent-context file. All load-bearing content relocated to `docs/` (per the "don't drop documentation, move it" rule).
+- **Comment hygiene**: stripped design-doc preambles, defensive validation for impossible scenarios, `log("starting…")`/`log("done in Yms")` pairs, and over-documented helpers whose docstring just restated the signature. Net `-691` LOC across `src/imprint/` + `src/cli.ts`.
+- **Test pruning**: consolidated redundant schema tests, parametrized micro-variations, dropped low-signal tests.
+- **Single source of truth for VERSION**: `src/imprint/version.ts` reads the version once from `package.json`. cli.ts, record.ts, probe-backends.ts all import from there — no more drift on bumps.
+- **`pricePath` accepts an array of fallback paths** (backward-compatible with single-string). Fixes a real bug in the southwest cron where `notifyWhen` silently never fired on the stealth-fetch backend (raw API shape) but worked on the playbook backend (reshaped output). Now both shapes are accepted.
+- **package.json description** rewritten to match the README tagline (was a technical surface description).
+
+### Removed
+- `src/imprint/replay-backend.ts`, `src/imprint/workflow-runtime.ts`, `src/imprint/discover-tools.ts`, `src/imprint/playbook-types.ts` (renamed/folded; see Changed).
+- `src/imprint/generate.ts`, `src/imprint/playbook-compiler.ts` (merged into `compile.ts`).
+- `test/sanity.test.ts` — its 3 cases tested Zod's `safeParse` on inline schemas, not Imprint logic.
+- `scripts/minimal-mcp.ts`, `scripts/minimal-mcp-with-import.ts`, `scripts/min-node-mcp.ts` — superseded MCP scratchpads from v0.1 bring-up.
+- `scripts/zoo-sniper.py` — standalone Python script for a personal use case (Zoo Pass automation), unrelated to imprint's pipeline. Pre-pivot artifact.
+
+### Metrics
+
+| | Before | After |
+|---|---|---|
+| `src/imprint/` + `src/cli.ts` LOC | 5,828 | 5,714 (-2%, includes +137 LOC for new doctor verb + 5 new shared helpers: compile/doctor/version/load-json/sites) |
+| Source files | 26 | 25 (− 4 renames/folds + 5 new) |
+| Tests | 137 / 13 files | 238 / 20 files (more user-path coverage) |
+| README | sprint-changelog flavor (285 lines) | adoption-friendly (110 lines) |
+| Docs files | 3 | 10 (+ CHANGELOG, CONTRIBUTING, scripts/README) |
+| CLI verbs | 12 | 13 (+ doctor) |
+| ADR entries | — | 14 (D1-D14, including D13 dead-code defense + D14 errors-over-LOC) |
+
+---
+
+## [0.1.0] — 2026-04 / 2026-05 (sprint)
+
+Initial public release. Two-week sprint to ship the full pipeline + two working demos.
+
+### Added
+- `imprint record` — CDP-based browser session capture with stdin narration loop, JSONL streaming, sidecar Session JSON on close.
+- `imprint redact` — credential / PII scrub with `[REDACTED:N]` markers preserving shape for the LLM.
+- `imprint generate` — Vertex Anthropic compilation of session → `workflow.json`.
+- `imprint compile-playbook` — Vertex Anthropic compilation of session → `playbook.yaml` (DOM replay artifact, switched from markdown to YAML mid-sprint).
+- `imprint emit` — code generation of `examples/<site>/<toolName>/index.ts` from `workflow.json`.
+- `imprint cron` — polling daemon with multi-provider notifications (Pushover + ntfy) and `notifyWhen: price_below` predicate.
+- `imprint mcp-server` — MCP server (stdio + Streamable HTTP) on the official `@modelcontextprotocol/sdk`.
+- `imprint playbook` — direct Playwright execution of a YAML playbook.
+- `imprint probe-backends` — per-site backend probing with cached `backends.json`.
+- `imprint login` — credential extraction from a recorded session into a per-site credential store.
+- **Backend ladder** — `fetch → stealth-fetch → playbook` with `auto` mode that escalates only on FORBIDDEN.
+- **stealth-fetch** — Playwright-bootstrapped sensor token mint + native fetch augmented with those tokens. Defeats Akamai (verified vs. Southwest: 403 → 200 with real flight data).
+- Working demo: `examples/southwest` (live fare watcher).
+- Working demo: `examples/discoverandgo` (authed museum-pass booking).
+
+### Decisions made
+- Approach: Teach + Replay (not Teach + Expand).
+- Network-level capture (CDP) over vision-based.
+- Two artifacts per recording (workflow.json + playbook.yaml) so the ladder always has a fallback.
+- YAML for the playbook format (replaced 425 LOC of hand-rolled markdown parsing with `YAML.parse` + Zod).
+- Probe-at-record-time + cached `backends.json` to skip futile rungs.
+- MCP stdio default; HTTP opt-in.
+
+See [docs/decisions.md](docs/decisions.md) for the full list with rationale.

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Ashay Changwani
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,322 @@
+<h1 align="center">Imprint</h1>
+
+<p align="center">
+  <strong>Don't do anything twice. Teach your AI agent once, and it remembers forever.</strong>
+</p>
+
+<p align="center">
+  <a href="https://github.com/ashaychangwani/imprint/actions/workflows/test.yml"><img src="https://github.com/ashaychangwani/imprint/actions/workflows/test.yml/badge.svg" alt="Tests"></a>
+  <img src="https://img.shields.io/endpoint?url=https://gist.githubusercontent.com/ashaychangwani/cbd3134e06fb4fabf24aed94b251bdfd/raw/test-count.json" alt="Test count">
+  <a href="https://github.com/ashaychangwani/imprint/releases"><img src="https://img.shields.io/github/v/release/ashaychangwani/imprint?label=release" alt="Release"></a>
+  <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue" alt="MIT License"></a>
+  <a href="https://github.com/ashaychangwani/imprint/stargazers"><img src="https://img.shields.io/github/stars/ashaychangwani/imprint?style=social" alt="GitHub Stars"></a>
+</p>
+
+<br>
+
+```bash
+bun install -g imprint-mcp
+imprint teach southwest --url https://www.southwest.com
+```
+
+That's it. Imprint opens a browser, you drive the workflow, and it compiles a deterministic **MCP tool** your AI agent can call from then on. No tokens burned on exploration, no "the LLM clicked the wrong button" variance. The recording *is* the executable.
+
+<br>
+
+## See it in action
+
+After teaching, your agent has a tool called `search_namecheap_domains`. The compile-agent reverse-engineered the site's CRC32 URL signing scheme from a captured JavaScript bundle, chains five API endpoints, and merges availability + pricing + aftermarket data:
+
+```
+$ claude "search for getimprint on Namecheap, under $20/yr renewal"
+
+  getimprint.com     taken         registered 2008         GoDaddy.com, LLC
+  getimprint.dev     available     $12.98/yr (19% off)     renews $20.98/yr
+  getimprint.org     available     $7.48/yr (42% off)      renews $15.98/yr
+  getimprint.fyi     available     $6.98/yr                renews $9.68/yr
+  getimprint.xyz     available     $2.00/yr (90% off)      renews $19.48/yr
+```
+
+Real-time domain availability with per-request URL signing — the agent wrote the signing function itself by reading the site's JS bundle.
+
+<br>
+
+## How it works
+
+<table>
+<tr>
+<td width="33%">
+
+### 1. Teach
+
+```bash
+imprint teach mysite \
+  --url https://example.com
+```
+
+A browser opens. You drive the workflow and narrate what you're doing. Imprint records every network request and DOM interaction.
+
+Raw recordings are stored locally under `~/.imprint/<site>/sessions/`, and each generated tool lives under `~/.imprint/<site>/<toolName>/` by default, outside the repo. The generated `index.ts` imports from `imprint/runtime` via a `node_modules/imprint` symlink that Imprint maintains automatically — created on `emit`, self-healed at runtime if a worktree moves or vanishes (so the next `imprint mcp-server`/`cron`/`probe-backends` repairs a stale link without re-emitting). The tracked `examples/` tree remains as source fixtures and demos.
+
+</td>
+<td width="33%">
+
+### 2. Compile
+
+Imprint generates replay artifacts:
+
+- **`workflow.json`** — API-level replay (fast, with named state captures)
+- **`playbook.yaml`** — DOM-level fallback (universal)
+- **`request-transform.ts`** — URL signing when the API requires per-call tokens (optional)
+
+Both artifacts are written into the generated tool directory (`~/.imprint/<site>/<toolName>/`). `compile-playbook` uses that nested location by default so cron and MCP discovery can see the fallback without a custom `--out`.
+
+Credentials and PII are redacted automatically: credential values become `${credential.NAME}` placeholders, sensitive values become redaction markers that preserve equality within the artifact, and a supplemental free-form scan catches common emails, phone numbers, SSNs, payment cards, JWTs, API keys, private keys, database URLs, and webhook URLs before LLM compile.
+
+</td>
+<td width="34%">
+
+### 3. Use
+
+A typed MCP tool is generated and wired into your AI platform. Re-run `imprint install <site>` any time to add the same emitted MCP server to another platform, or remove it later with `imprint uninstall <site>`.
+
+</td>
+</tr>
+</table>
+
+> All three steps happen in a single `imprint teach` command.
+
+<br>
+
+## Why Imprint?
+
+Other browser-tool frameworks (browser-use, Computer Use) ask the LLM to **decide every click at runtime**.
+
+| | Imprint | browser-use / Computer Use |
+|---|---|---|
+| **How it works** | Record once, replay deterministically | LLM decides every click at runtime |
+| **Token cost** | Zero at runtime | Scales with workflow complexity |
+| **Reliability** | Deterministic — same input, same output | Variable — exploration can diverge |
+| **Bot detection** | Real Chromium + stealth-fetch | Detectable automation fingerprint |
+| **When it breaks** | Automatic fallback via backend ladder | No fallback |
+| **Time to result** | 200ms – 9s | 30s+ |
+
+<br>
+
+## Install
+
+### npm (requires [Bun](https://bun.sh) >= 1.3)
+
+```bash
+bun install -g imprint-mcp
+```
+
+Or run without installing: `bunx imprint-mcp teach southwest --url https://www.southwest.com`
+
+### Standalone binary (no Bun needed)
+
+```bash
+curl -fsSL https://raw.githubusercontent.com/ashaychangwani/imprint/main/scripts/install.sh | bash
+```
+
+The standalone binary supports `mcp-server`, `install`, `cron`, and `credential` commands.
+Browser commands (`teach`, `record`, `login`, `playbook`) require a full Bun + Playwright install.
+
+### From source
+
+```bash
+git clone https://github.com/ashaychangwani/imprint.git && cd imprint
+bun install && bun link
+```
+
+### Browser commands
+
+The `teach`, `record`, `login`, and `playbook` commands need Playwright's Chromium. Install it once:
+
+```bash
+bunx playwright install chromium
+```
+
+### LLM providers
+
+Imprint detects LLM providers from what's already on your system:
+
+| Priority | Provider | Triggered by |
+|---|---|---|
+| 1 | `claude-cli` | `claude` on PATH (Claude Code subscription) |
+| 2 | `codex-cli` | `codex` on PATH (Codex subscription) |
+| 3 | `anthropic-api` | `ANTHROPIC_API_KEY` env var |
+| 4 | `cursor-cli` | `cursor` on PATH (generic prompt/playbook compile only; not `teach`/`generate`) |
+
+```bash
+imprint doctor
+```
+
+Shows which providers are detected. Interactive `imprint teach` prompts you to choose when multiple compatible compile providers are available, and also lists undetected providers as setup-help entries. Pick one of those help entries to see exactly which CLI or environment variable to add so it will be detected next time.
+
+To force a specific provider and skip the picker, pass `--provider <name>` to `teach`, `generate`, or `compile-playbook`. `teach` and `generate` require a compile-agent provider (`claude-cli`, `codex-cli`, or `anthropic-api`); `compile-playbook` can also use `cursor-cli`.
+
+After selecting a provider, `teach` prompts for a **model** (e.g. `claude-opus-4-7` vs `claude-sonnet-4-6` for Anthropic, `gpt-5.4` vs `o3` for Codex). Override with `--model <name>`. Each tool compiles with a **5-minute timeout** by default; override with `--timeout <duration>` (e.g. `--timeout 10m`, `--timeout 1h`). To skip the replay-and-diff stage (the automated second pass that classifies ephemeral vs constant values), pass `--skip-replay` — faster, but may reduce workflow accuracy for sites with dynamic request parameters.
+
+<br>
+
+## Local compile tracing
+
+Slow or suspicious compiles can be inspected in a local [Phoenix](https://arize.com/docs/phoenix/self-hosting/deployment-options/terminal) trace UI.
+
+```bash
+# one-time install with uv
+uv tool install arize-phoenix
+phoenix serve
+
+# in another terminal
+IMPRINT_TRACE=1 \
+IMPRINT_TRACE_BATCH=false \
+IMPRINT_TRACE_LLM_IO=1 \
+IMPRINT_TRACE_TOOL_IO=1 \
+PHOENIX_COLLECTOR_ENDPOINT=http://localhost:6006 \
+imprint teach namecheap-domains --from-session ~/.imprint/namecheap-domains/sessions/<ts>.json --provider codex-cli
+```
+
+Traces show the full compile pipeline at every level of detail: each `agent.turn.N` span captures per-turn token counts; each `llm.message_with_tools` span records model, provider, input/output tokens, and stop reason; each `agent.tool.X` span times individual tool dispatches. Drill from `cli.teach` → `compile.generate` → `agent.turn.1` → tool calls to find exactly which turn or tool is spending tokens. Set `IMPRINT_TRACE_IO_MAX_CHARS` to raise or lower captured payload size. Set `IMPRINT_TRACE_INPUT_USD_PER_1M` and `IMPRINT_TRACE_OUTPUT_USD_PER_1M` to add estimated cost attributes.
+
+<br>
+
+## Platform support
+
+At the end of `imprint teach`, you pick your AI platform and Imprint handles the wiring:
+
+| Platform | Integration |
+|---|---|
+| **Claude Code** | Automatic — runs `claude mcp add` for you |
+| **Codex CLI** | Automatic — runs `codex mcp add` for you |
+| **Claude Desktop** | Paste-ready JSON config |
+| **OpenClaw** | MCP config + SKILL.md export |
+| **Hermes** | MCP config + SKILL.md + cron mapping |
+
+Each site registers as its own MCP server (`imprint-southwest`, `imprint-discoverandgo`, ...) so tools never collide. See [Integrations](docs/integrations.md) for HTTP transport, Docker, and systemd options.
+
+Audit or clean up those registrations with `imprint mcp`:
+
+```bash
+imprint mcp status                         # registrations + local teach state
+imprint mcp                                # interactive cleanup TUI
+imprint mcp disable imprint-mysite --yes   # reversible; stores a local snapshot
+imprint mcp delete imprint-mysite --yes    # removes external MCP registrations only
+```
+
+Raw recordings under `~/.imprint/<site>/sessions/` may contain sensitive browser state. Cleanup commands leave them alone unless you explicitly choose `--local site`.
+
+See [MCP Maintenance](docs/mcp-maintenance.md) for status classifications, supported client config files, reversible disable behavior, and local artifact cleanup rules.
+
+<br>
+
+## Sharing skills across machines
+
+Teach on your laptop, ship the skill to a remote agent (OpenClaw, Hermes, a server-side cron host, ...). Skill folders committed to git contain **zero plaintext credentials** — only placeholders like `${credential.NAME}` / `${state.NAME}` and a `credentials.manifest.json` listing the secrets or durable storage keys the receiver must provision.
+
+For credentials, use the **encrypted bundle** flow when you can't (or don't want to) re-type passwords on the receiving machine:
+
+```bash
+# On the laptop where you taught the skill:
+imprint credential export southwest --out southwest.imprintbundle
+# → prompts for a passphrase. The bundle is libsodium-encrypted with an
+#   argon2id-derived key. Safe to send via Slack, email, scp, S3, etc.
+
+# On the OpenClaw machine (or any other receiver):
+imprint credential import southwest southwest.imprintbundle
+# → prompts for the same passphrase. Decrypts; secrets land in the OS keychain.
+```
+
+Pass the passphrase **out-of-band** (Signal, phone, password manager share — *not* the same channel as the bundle file).
+
+After import, the same `imprint mcp-server <site>` config you'd use locally works on the receiver — it resolves credentials from that machine's credential backend and initializes a fresh cookie/state jar for every tool call. If anything's missing, `imprint mcp-server` and `imprint cron` log/fail with the exact `imprint credential set`, `imprint login`, or `imprint credential import` commands you need.
+
+See [Sharing Skills](docs/credential-sharing.md) for the full flow including interactive `imprint credential set` (when you can re-type), threat model, rotation, and OpenClaw / Hermes wiring details.
+
+<br>
+
+## The backend ladder
+
+When an API call gets blocked or needs browser-minted state, Imprint doesn't jump straight to DOM replay. It escalates through the cheapest mode that can satisfy the workflow:
+
+| | Speed | Handles |
+|---|---|---|
+| **fetch** | ~200ms | Plain APIs, persisted cookies, in-flight HTTP captures |
+| **fetch-bootstrap** | browser bootstrap + API replay | Pages that only need Chromium to mint cookies, CSRF, storage, or DOM-derived state |
+| **stealth-fetch** | ~12s first call, ~1s after | Akamai, Cloudflare, DataDome, bot-defense state |
+| **playbook** | ~9s | Anything — full DOM replay as fallback |
+
+`fetch-bootstrap` is not a default rung for every workflow. `auto` inserts it only when the workflow declares bootstrap metadata, a capture requires browser/stealth bootstrap, or `fetch` returns structured `STATE_MISSING` that a browser bootstrap can satisfy. Every recording still compiles to *both* `workflow.json` and `playbook.yaml`, so the ladder has a DOM fallback when API replay cannot work.
+
+State-aware workflows use named captures and `${state.NAME}` placeholders. For example, request A can set a CSRF cookie, request B can project it into a header, and the whole run stays on plain `fetch` without launching Chromium.
+
+<br>
+
+## Examples
+
+The checked-in `examples/` directory contains committed fixtures and demos. Generated tools from `imprint teach` go into `~/.imprint/<site>/<toolName>/` by default (configurable via `IMPRINT_HOME`). Runtime discovery (cron, MCP, probe-backends) reads `IMPRINT_HOME`, so to run the checked-in examples, point it at the repo's `examples/` directory:
+
+You can also install an example directly into an MCP client:
+
+```bash
+imprint install google-flights --source examples --platform claude-desktop
+```
+
+Run `imprint install` with no arguments for an interactive install/uninstall picker. It only shows detected AI platforms; uninstall lists installed `imprint-*` MCP servers directly. For GUI config-file clients such as Claude Desktop, install writes an absolute Bun + Imprint CLI path so the app does not depend on your shell PATH.
+
+| Example | What it demonstrates | Run it |
+|---|---|---|
+| [**southwest**](examples/southwest) | Live fare watcher, defeats Akamai bot detection, price-drop notifications | `IMPRINT_HOME=examples imprint cron southwest --once` |
+| [**google-flights**](examples/google-flights) | Real-time flight search across all carriers, parses Google's raw protobuf response | `IMPRINT_HOME=examples imprint mcp-server google-flights` |
+| [**google-hotels**](examples/google-hotels) | Hotel search with star rating, guest scores, nightly + total prices | `IMPRINT_HOME=examples imprint mcp-server google-hotels` |
+| [**discoverandgo**](examples/discoverandgo) | Authenticated booking via per-site credential store | `IMPRINT_HOME=examples imprint cron discoverandgo --once` |
+| [**namecheap-domains**](examples/namecheap-domains) | Domain search with CRC32 URL signing reverse-engineered from JS, 5-endpoint chain with availability + aftermarket pricing | `IMPRINT_HOME=examples imprint mcp-server namecheap-domains` |
+| [**echo**](examples/echo) | MCP smoke-test fixture (no network, no LLM) | `IMPRINT_HOME=examples imprint mcp-server echo` |
+
+<br>
+
+## CLI reference
+
+```
+imprint --help              # all commands
+imprint <command> --help    # per-command options
+```
+
+| | Commands |
+|---|---|
+| **Pipeline** | `teach` · `record` · `redact` · `generate` · `compile-playbook` · `emit` |
+| **Runtime** | `cron` · `mcp-server` · `playbook` · `probe-backends` |
+| **Credentials** | `credential set` · `credential list` · `credential export` · `credential import` · `credential migrate` |
+| **Utilities** | `mcp` · `login` · `assemble` · `check` · `doctor` |
+
+`teach`, `generate`, and `compile-playbook` accept `--provider <name>` to override the auto-detected LLM (see [Install](#install) for valid names and compile-agent support). `teach` and `generate` also take `--keep-test` to retain the agent-written `parser.test.ts` for debugging — it's deleted by default since it reads the gitignored redacted session via `$IMPRINT_SESSION_PATH` and isn't reproducible elsewhere. For multi-tool sites, use `imprint cron <site> --tool <toolName>` and `imprint probe-backends <site> --tool <toolName>` unless `--config` or `--out` points inside the target tool directory.
+
+<br>
+
+## Docs
+
+- [Getting Started](docs/getting-started.md) — full walkthrough
+- [Integrations](docs/integrations.md) — per-platform setup
+- [MCP Maintenance](docs/mcp-maintenance.md) — audit, disable, restore, and prune Imprint MCP state
+- [Sharing Skills](docs/credential-sharing.md) — laptop ↔ OpenClaw / Hermes / remote-agent provisioning
+- [Architecture](docs/architecture.md) — data flow and module map
+- [Security](docs/security.md) — redaction, credential handling, what gets stored
+- [Troubleshooting](docs/troubleshooting.md) — common failures and fixes
+- [Decisions](docs/decisions.md) · [Glossary](docs/glossary.md) · [Capture Protocol](docs/capture-protocol.md) · [Playbook Debugging](docs/playbook-debugging.md) · [Notifications](docs/notifications.md)
+
+<br>
+
+## Contributing
+
+Conventional Commits enforced in CI. Run `bun run check` before submitting.
+
+Good first contributions: replay backends, notification predicates, auth extractors, example sites, docs.
+
+See [CONTRIBUTING.md](CONTRIBUTING.md) for full guidelines.
+
+<br>
+
+## License
+
+[MIT](LICENSE)

package/examples/discoverandgo/README.md

@@ -0,0 +1,57 @@
+# Discover & Go museum-pass booking
+
+> Reserve a museum pass on the San Diego Public Library's Discover & Go service. Authenticated workflow; uses the per-site credential store populated by `imprint login`.
+
+## What this shows off
+
+- **The `imprint login` flow.** Library cards are persistent; you log in once during recording and `imprint login` extracts the patron ID + cookies into `~/.config/imprint/credentials/discoverandgo.json`. Subsequent runs use those credentials without re-recording.
+- **`${credential.X}` substitutions.** Workflow templates reference `${credential.patron_id}` directly; the runtime fills it from the credential store.
+- **Plain `fetch` backend works** — Discover & Go has no bot detection, so this demo runs in ~200ms per call (no Playwright bootstrap, no DOM walk).
+
+## Run it
+
+```bash
+# One-time: log in once via record, then mine credentials
+imprint record discoverandgo --persist-profile --url https://sandiego.discoverandgo.net
+# (drive a real reservation; narrate; /done when finished)
+
+imprint login discoverandgo --from-session ~/.imprint/discoverandgo/sessions/<ts>.json
+
+# Then any time you want to book:
+imprint cron discoverandgo --once
+```
+
+## What you should see
+
+```
+[imprint cron] config: examples/discoverandgo/book_discoverandgo_museum_pass/cron.json
+[imprint cron] tool: book_discoverandgo_museum_pass (3 param(s))
+[imprint cron] schedule: 0 0 * * *
+[imprint cron] replayBackend: fetch
+[imprint backend] trying fetch…
+[imprint backend] fetch: OK in ~250ms
+[imprint cron]   OK in ~250ms via fetch: {"reservationID":...,"status":"confirmed",...}
+```
+
+## Files
+
+| File | What |
+|---|---|
+| `workflow.json` | API workflow with `${credential.patron_id}` and `${param.offer_id}` substitutions |
+| `index.ts` | Generated tool function |
+| `cron.json` | Daily 0am tick (placeholder offer + date — edit to your real reservation) |
+
+## Tuning
+
+- **Edit `cron.json`** with the actual `offer_id` (numeric attraction ID), `offer_date` (YYYY-MM-DD), and your `notification_email`.
+- **Schedule** is daily; if you want to grab a pass the moment it opens (D&G releases new dates at midnight Pacific), tighten to `*/5 0 * * *` for 5-minute polling around midnight.
+
+## Notes
+
+- Discover & Go's auth model is patron-ID + session cookies. The session cookie expires; re-run `imprint login` if you start seeing AUTH_EXPIRED.
+- The workflow chains an `epass_server.php?method=Login` followed by the booking call. Generated `workflow.json` references both via the `${response[N].patronID}` extraction syntax.
+
+## Not in this demo
+
+- Multi-attraction booking in one tick (one `cron.json` per attraction).
+- Auto-renewal (passes are calendar-bound; renewal would mean picking the next available date).

package/examples/discoverandgo/book_discoverandgo_museum_pass/cron.json

@@ -0,0 +1,8 @@
+{
+  "schedule": "0 0 * * *",
+  "params": {
+    "offer_id": 1175,
+    "offer_date": "1999-01-01",
+    "notification_email": "you@example.com"
+  }
+}