ashlr-plugin 1.36.4

package/docs/multi-turn-architecture.md

@@ -0,0 +1,108 @@
+# Multi-Turn Recompression Architecture
+
+## Phase 1: What Claude Code Exposes to Plugins
+
+### Hooks Available
+
+Claude Code's plugin hook system (as of v1.25 investigation, April 2026) exposes four hook events:
+
+| Hook | When it fires | Plugin sees |
+|------|---------------|-------------|
+| `SessionStart` | Once at session open | stdin: `{}` (no payload). `CLAUDE_PROJECT_DIR`, `CLAUDE_SESSION_ID`, `CLAUDE_PLUGIN_ROOT` via env. |
+| `PreToolUse` | Before every tool call | stdin: `{ tool_name, tool_input }` |
+| `PostToolUse` | After every tool call | stdin: `{ tool_name, tool_input, tool_result }` |
+| `SessionEnd` | Once at session close | stdin: `{}` |
+
+### What's NOT Exposed
+
+**Critical finding: no `PreCompletion` / `PreModel` hook exists.**
+
+Claude Code does NOT expose:
+- A hook that fires before the LLM is invoked with the full conversation.
+- The conversation history (prior turns' messages, tool results, assistant text).
+- Any `prompts/list` MCP surface with history-aware context.
+- A hook that can mutate or summarize the in-flight context window before model invocation.
+- Total token count for the current context.
+
+The only writable surface is `additionalContext` in `PostToolUse` and `SessionStart` responses, which is appended (never replaces) the context window.
+
+### What IS Possible as Pure Plugin Code
+
+1. **Result size tracking**: `PostToolUse` provides `tool_result` content, so we can measure every tool result's byte size as it leaves the hook.
+2. **Per-turn freshness modeling**: Since we know when each result was emitted (by recording it at `PostToolUse` time), we can estimate staleness by counting subsequent tool calls.
+3. **Nudge injection**: `additionalContext` from `PostToolUse` is injected into the agent's next turn context — not the full history, but visible to the model in the next request.
+4. **Session-scoped JSONL log**: `~/.ashlr/session-history/<sessionId>.jsonl` can store a running record of every ashlr tool result emitted, enabling staleness queries at any later point in the session.
+
+### The Fundamental Gap
+
+A true "stale result rewrite" would require intercepting the conversation array that Claude Code assembles before sending to Anthropic. This array includes all prior tool results verbatim. Without a `PreModel` hook that exposes and allows mutation of this array, the plugin cannot truncate or summarize old results.
+
+**Verdict: Phase 3 (true rewrite) is NOT possible with current Claude Code.** The 20-40% savings estimate is contingent on Anthropic exposing a `PreModel` hook in a future release. Until then, the plugin can deliver 5-10% improvement through visibility + user-driven recompression (Phase 2).
+
+---
+
+## Phase 2: What We Build Now
+
+### Architecture Overview
+
+```
+PostToolUse (any ashlr__* or Read/Grep call)
+    └─> posttooluse-stale-result.ts
+            ├── writes to servers/_history-tracker.ts
+            │       └── ~/.ashlr/session-history/<sessionId>.jsonl
+            └── checks stale byte threshold
+                    └── if >50KB stale → additionalContext nudge (once/session)
+
+/ashlr-compact (slash command)
+    └─> reads session-history JSONL
+    └─> identifies results >5 turns old
+    └─> prints savings estimate to user
+    └─> injects additionalContext to re-run stale reads
+
+orient-nudge-hook.ts (extended)
+    └─> checks stale result total
+    └─> injects stale-accumulation nudge (once/session, threshold 50KB)
+```
+
+### Freshness Decay Model
+
+Turn delta is measured in subsequent tool calls (not wall-clock time) because "turns" in LLM context are tool-call boundaries.
+
+| Turns since result emitted | Freshness score |
+|---------------------------|-----------------|
+| 0–4 | 1.0 (fresh) |
+| 5–14 | 0.5 (stale) |
+| 15+ | 0.2 (very stale) |
+
+These starting coefficients are intentionally conservative. v1.26 will tune via telemetry from `multi_turn_stale_estimate` events.
+
+### Session ID Strategy
+
+Uses `CLAUDE_SESSION_ID` (available in hook env) when present, falling back to the same `derivedSessionId` pattern established in `session-start.ts`. This ensures history files match across PostToolUse calls within the same session.
+
+---
+
+## Phase 3: What's Needed from Claude Code
+
+For true conversation-history recompression, Anthropic would need to add a `PreModel` hook with:
+
+```typescript
+// Hypothetical future hook contract
+interface PreModelPayload {
+  messages: ConversationMessage[];  // full history
+  systemPrompt: string;
+}
+interface PreModelOutput {
+  messages?: ConversationMessage[]; // mutated history (plugin can compress)
+  additionalSystemContext?: string;
+}
+```
+
+With this, the plugin could:
+1. Walk the `messages` array and find `tool_result` entries from ashlr tools.
+2. For results older than N turns with no downstream references, replace the content with a stub: `"[ashlr: result summarized — N bytes removed]"`.
+3. Return the compressed message array.
+
+Estimated impact once available: 20-40% context reduction on sessions with >20 turns of file-heavy work.
+
+**Request to Anthropic**: add a `PreModel` hook that provides the full `messages` array and allows returning a mutated copy.

package/docs/operations/deployment.md

@@ -0,0 +1,108 @@
+# Deployment Guide
+
+## Environment Variables
+
+### Required for boot — server will not start without these
+
+| Variable | What it does | Default when unset | How to get |
+|---|---|---|---|
+| `DATABASE_URL` | *(reserved — SQLite path uses `ASHLR_DB_PATH`)* | — | n/a |
+| `ASHLR_DB_PATH` | Absolute path to the SQLite `.db` file | `server/ashlr.db` (sibling of `src/`) | Set to a persistent volume path in production (e.g. `/data/ashlr.db`) |
+| `ASHLR_MASTER_KEY` | AES-256-GCM master key for encrypting secrets at rest (GitHub OAuth tokens, genome DEKs). Must decode to exactly 32 bytes base64. | Throws in production. Dev: set `ASHLR_MASTER_KEY_DEV=1` for an ephemeral key. | `node -e "console.log(require('crypto').randomBytes(32).toString('base64'))"` |
+
+### Required for features — server starts but feature degrades without these
+
+| Variable | What it does | Default when unset |
+|---|---|---|
+| `SENDGRID_API_KEY` | Sends transactional email (magic-link auth, weekly digest, status alerts). | Email is silently skipped; log line emitted. |
+| `STRIPE_SECRET_KEY` | Stripe API client for checkout sessions, portal, subscription management. | `getStripeClient()` throws; billing routes 500. |
+| `STRIPE_WEBHOOK_SECRET` | Verifies Stripe webhook HMAC signatures (`/webhooks/stripe`). | Webhook handler returns 400 on every event. |
+| `GITHUB_CLIENT_ID` | GitHub OAuth app client ID for `/auth/github/start` and scope-upgrade flow. | GitHub OAuth endpoints return 500. |
+| `GITHUB_CLIENT_SECRET` | GitHub OAuth app client secret. | Same as above. |
+| `GITHUB_WEBHOOK_SECRET` | HMAC secret for validating GitHub push webhooks (`/webhooks/github`). | Webhook rejected with 400. |
+| `SENTRY_DSN` | Sentry error-reporting DSN. | No-op; errors are still logged via pino. |
+
+### Optional / overrides — have safe defaults
+
+| Variable | What it does | Default |
+|---|---|---|
+| `PORT` | HTTP listen port. | `3001` |
+| `NODE_ENV` | Controls pino pretty-print (`development`) vs JSON (`production`). | `development` |
+| `LOG_LEVEL` | pino log level (`trace`, `debug`, `info`, `warn`, `error`). | `debug` in dev, `info` in production. |
+| `BASE_URL` | Public API base URL, used in OAuth redirect URIs and Stripe callback URLs. | `https://api.ashlr.ai` |
+| `SITE_URL` | Public site URL for Stripe cancel/return redirects. | `https://plugin.ashlr.ai` |
+| `FRONTEND_URL` | Frontend app URL for GitHub OAuth callback redirects. | `https://plugin.ashlr.ai` |
+| `ASHLR_SITE_URL` | Used by the weekly-digest cron for unsubscribe link base. | `https://plugin.ashlr.ai` |
+| `STATUS_BASE_URL` | Base URL for status page links. | `https://status.ashlr.ai` |
+| `API_BASE_URL` | API base for health-check worker synthetic probes. | `https://api.ashlr.ai` |
+| `PLUGIN_BASE_URL` | Plugin CDN base for health-check worker probes. | `https://plugin.ashlr.ai` |
+| `DOCS_BASE_URL` | Docs base for health-check worker probes. | `https://docs.ashlr.ai` |
+| `ASHLR_DISABLE_TRIAL` | Set to `"1"` to disable free-trial eligibility on new Pro checkouts. | Trial enabled. |
+| `ASHLR_MASTER_KEY_DEV` | Set to `"1"` to use an ephemeral (non-persistent) master key. Dev/test only. | Off. |
+| `METRICS_USER` | HTTP Basic auth username for the `/metrics` endpoint. | Endpoint open to all IPs. |
+| `METRICS_PASS` | HTTP Basic auth password for the `/metrics` endpoint. | — |
+| `METRICS_ALLOWED_IPS` | Comma-separated IPs allowed to scrape `/metrics`. | All IPs allowed. |
+| `SENTRY_ORG` | Sentry org slug for source-map upload in CI. | — |
+| `SENTRY_PROJECT` | Sentry project slug for source-map upload in CI. | — |
+| `SENTRY_INTERNAL_TOKEN` | Sentry auth token for source-map upload CLI in CI. | — |
+| `TESTING` | Set to `"1"` by the test harness. Enables Stripe stub client and skips real email sends. | Off. |
+
+---
+
+## Deployment Checklist
+
+### Secrets
+
+- [ ] `ASHLR_MASTER_KEY` set to a fresh 32-byte base64 secret — **NOT the dev default**
+- [ ] `SENDGRID_API_KEY` set (required for magic-link login and weekly digest)
+- [ ] `STRIPE_SECRET_KEY` set (live key for production, test key for staging)
+- [ ] `STRIPE_WEBHOOK_SECRET` set and Stripe webhook configured to POST to `https://<host>/webhooks/stripe`
+- [ ] `GITHUB_CLIENT_ID` + `GITHUB_CLIENT_SECRET` set (GitHub OAuth App, callback URL: `https://<host>/auth/github/callback`)
+- [ ] `GITHUB_WEBHOOK_SECRET` set and GitHub webhook configured on any repos using genome auto-build
+
+### Infrastructure
+
+- [ ] `ASHLR_DB_PATH` points to a persistent volume (not ephemeral container storage)
+- [ ] Persistent volume backed up (SQLite WAL — snapshot the `.db` file and `-wal`/`-shm` when idle, or use Litestream)
+- [ ] `PORT` set (or load balancer configured for default `3001`)
+- [ ] `NODE_ENV=production` — enables JSON logging
+
+### Observability
+
+- [ ] `SENTRY_DSN` set — error tracking active (see `docs/operations/error-monitoring.md`)
+- [ ] `LOG_LEVEL` set (recommend `info` for production)
+- [ ] Metrics scraper configured — set `METRICS_USER`/`METRICS_PASS` or `METRICS_ALLOWED_IPS`, point Prometheus at `/metrics`
+- [ ] `/healthz` returns `200` from the deployed instance
+- [ ] `/readyz` returns `200` (confirms SQLite is reachable)
+
+### CI / Cron
+
+- [ ] `SENTRY_ORG`, `SENTRY_PROJECT`, `SENTRY_INTERNAL_TOKEN` added to GitHub Actions secrets (for source-map upload)
+- [ ] Weekly-digest cron scheduled (GitHub Actions or Render Cron: `0 14 * * 0` → `bun server/src/jobs/weekly-digest-cron.ts`)
+- [ ] Cron environment variables include `ASHLR_DB_PATH`, `ASHLR_MASTER_KEY`, `SENDGRID_API_KEY`, `ASHLR_SITE_URL`
+
+### DNS / TLS
+
+- [ ] TLS termination in place (reverse proxy or platform-managed)
+- [ ] CORS origin list reviewed (`BASE_URL`, `FRONTEND_URL`)
+
+---
+
+## Running Migrations
+
+Migrations run automatically on every server boot via `getDb()` in `server/src/db/connection.ts`. They are fully idempotent — safe to re-run on a running DB.
+
+For CI pipelines, pre-deploy verification, or dry-run checks before a production deploy, use the migration CLI:
+
+```sh
+# Check what migrations are pending without applying them (exits 1 if any pending)
+bun server/src/db/migrate.ts --check
+
+# Apply all pending migrations (safe to run multiple times)
+bun server/src/db/migrate.ts
+
+# Target a specific DB file
+bun server/src/db/migrate.ts --db /data/ashlr.db
+```
+
+The CLI reports each migration step and exits `0` when up-to-date, `1` if `--check` found pending migrations.

package/docs/operations/error-monitoring.md

@@ -0,0 +1,70 @@
+# Error Monitoring
+
+## Overview
+
+Error monitoring is handled by `server/src/lib/sentry.ts`. Sentry is optional — the integration is a no-op when `SENTRY_DSN` is unset (local dev, tests). All errors are always logged via pino structured logs regardless.
+
+## Setup
+
+### Environment variables
+
+| Variable | Purpose |
+|---|---|
+| `SENTRY_DSN` | Sentry DSN — obtainable from Sentry project settings → Client Keys. When absent, the module is never imported. |
+| `SENTRY_ORG` | Org slug for `sentry-cli` source-map upload in CI. |
+| `SENTRY_PROJECT` | Project slug for source-map upload in CI. |
+| `SENTRY_INTERNAL_TOKEN` | Auth token for `sentry-cli` in CI (GitHub Actions secret). |
+
+### Initialization
+
+`initSentry(release?)` is called once at the top of `server/src/index.ts` before any routes are mounted. It reads `SENTRY_DSN` and, if present, dynamically imports `@sentry/bun` to avoid any side-effects in DSN-less environments (tests, local dev).
+
+```ts
+initSentry(`ashlr-server@${pluginVersion}`);
+```
+
+The `release` tag is set to `ashlr-server@<version>` read from `plugin.json`. This enables source-map lookup in Sentry for each deployed version.
+
+### Error handler
+
+`sentryErrorHandler` is mounted as Hono's `onError` hook (last in the chain):
+
+```ts
+app.onError(sentryErrorHandler);
+```
+
+It captures the exception with request metadata (method, path, x-request-id, user_id) and always returns `{"error": "Internal server error"}` with status 500.
+
+### Manual capture
+
+Use `captureException(err, extras?)` from `server/src/lib/sentry.ts` for non-HTTP errors (cron jobs, background workers):
+
+```ts
+import { captureException } from "../lib/sentry.js";
+captureException(err, { job: "weekly-digest", userId });
+```
+
+## PII Redaction
+
+Events are scrubbed before send via `beforeSend`. The following keys are redacted in the full event payload:
+
+- `text`, `systemPrompt` — LLM content
+- `email`, `authorization`, `cookie`, `password` — credentials and identity
+
+The redactor does a deep JSON walk so nested occurrences are caught. Log extras passed to `captureException` are also scoped — never include raw email addresses, full URLs with tokens, or unhashed session IDs in the `extras` argument.
+
+**Policy**: never pass raw `email` strings in extras. Use `user_id` (opaque UUID) instead. Never include `authorization` header values or Stripe/GitHub secrets in error context.
+
+## Accessing dashboards
+
+- Project: `https://sentry.io/organizations/<SENTRY_ORG>/projects/<SENTRY_PROJECT>/`
+- Issues: filter by `release:ashlr-server@<version>` to scope to a specific deploy.
+- Recommended alert: notify on first occurrence of any new issue with `level:error`.
+
+## Trace sampling
+
+`tracesSampleRate` is set to `0.1` (10%). Increase for debugging high-traffic paths; decrease or set to `0` in cost-sensitive environments.
+
+## Fallback: pino logs
+
+When Sentry is not configured, all exceptions still surface in pino JSON logs. In production, pipe logs to your preferred log aggregator (Datadog, Logtail, CloudWatch). The structured fields on every error log include `requestId`, `method`, `path`, and `user_id`.

package/docs/operations/weekly-digest.md

@@ -0,0 +1,70 @@
+# Weekly Digest — Operations Guide
+
+## Schedule
+
+Runs every **Sunday at 14:00 UTC** via GitHub Actions (`.github/workflows/weekly-digest.yml`).
+
+Eligible recipients: Pro and Team tier users with `weekly_digest_opt_in = 1` who have not received a digest in the last 6 days (dedupe window prevents double-sends on retries).
+
+---
+
+## Manual trigger
+
+```bash
+# Production send (requires repo write access)
+gh workflow run weekly-digest.yml --ref main
+
+# Dry-run (renders emails, no sends, no DB writes)
+gh workflow run weekly-digest.yml --ref main -f dry_run=true
+```
+
+Or via GitHub UI: **Actions → Weekly Digest → Run workflow**.
+
+---
+
+## Local dry-run
+
+```bash
+cd server
+ASHLR_MASTER_KEY=<key> SENDGRID_API_KEY=<key> DATABASE_URL=<url> \
+  bun src/jobs/weekly-digest-cron.ts --dry-run
+```
+
+No emails are sent; rendered output is written to stderr.
+
+---
+
+## Debugging a failed run
+
+1. **GitHub Actions logs** — go to Actions → Weekly Digest → failed run. The job streams structured pino JSON; search for `"event":"cron_end"` to see final counts, or `"event":"cron_start"` to confirm the job started.
+
+2. **Key log lines to look for:**
+
+   | Field | Meaning |
+   |-------|---------|
+   | `event: "cron_start"` | Job began, includes `dryRun` flag |
+   | `event: "cron_end"` | Job finished; `sent`, `skipped`, `failed`, `durationMs` |
+   | `weekly-digest: failed for user` | Per-user error with `userId` and `err` |
+
+3. **Partial failure is safe** — users whose send failed are not marked as sent in the DB, so the next scheduled run retries them automatically.
+
+4. **Re-running manually** — trigger via `gh workflow run` above. The 6-day dedupe window means already-sent users are skipped; only failed/missed users receive the email.
+
+---
+
+## Failure visibility
+
+GitHub Actions emails the repo owner on any job failure by default. No additional configuration needed.
+
+To add Slack/PagerDuty alerts, add a notification step at the end of the `send` job in `.github/workflows/weekly-digest.yml`.
+
+---
+
+## User opt-out
+
+Users can unsubscribe via:
+
+- **Unsubscribe link** in every digest email → `GET /unsubscribe?token=<signed-token>` → sets `weekly_digest_opt_in = 0`
+- **Email preferences API** → `GET/POST /api/email-prefs` (see `docs/email-templates.md` for the full unsubscribe flow)
+
+Opted-out users are excluded at query time; no re-opt-in happens automatically.