@cerefox/memory 0.5.4 → 0.7.0

package/docs/guides/migration-v0.5.md

@@ -331,6 +331,175 @@ test path and works for power users who want a specific file location.
 
 ---
 
+## v0.6.0 moved the web server to TypeScript
+
+**TL;DR**: `cerefox web` from the npm package now boots an in-process
+Hono server (TypeScript on Bun) instead of pointing you at
+`uv run cerefox web`. Three ingestion endpoints temporarily return
+503; the web UI shows a friendly toast pointing at the working CLI
+fallback. Full ingestion support lands in v0.7.
+
+### What's new
+
+- **`cerefox web` works from npm.** No clone, no `uv`. `npm install
+  -g @cerefox/memory` followed by `cerefox web` boots the local web
+  UI + JSON API on `http://127.0.0.1:8000/`.
+- **React SPA bundled** into `@cerefox/memory`. The web UI is part
+  of the npm tarball; you get the same UI Python's `uv run cerefox
+  web` serves, no extra install.
+- **Configure-agent grew Phase 2 writers**: `cerefox configure-agent
+  --tool cursor` / `--tool codex` / `--tool gemini` join the existing
+  `--tool claude-code` / `--tool claude-desktop`. Codex's config is
+  TOML (`~/.codex/config.toml`); the rest are JSON.
+
+### The 503-ingestion-stubs window
+
+Three endpoints return 503 with `{error: "Ingestion lands in v0.7",
+see: <url>, note: …}`:
+
+- `POST /api/v1/ingest` (paste)
+- `POST /api/v1/ingest/file` (file upload)
+- `POST /api/v1/documents/{id}/upload` (replace)
+
+The web UI detects this and shows a yellow Mantine toast — no scary
+error banner. `/documents/{id}/edit` also returns 503 if you try to
+change content (it compares SHA-256 hashes against the stored
+`content_hash` — title / metadata / project changes work fine).
+
+### Working fallbacks during the v0.6 window
+
+Both fully functional, no behaviour change:
+
+```bash
+# Option A — npm-installed CLI hits the deployed Edge Function.
+cerefox ingest my-notes.md
+cerefox ingest-dir docs/
+
+# Option B — keep using the Python web for ingestion until v0.7.
+uv run cerefox web
+```
+
+v0.7 swaps the 503 stubs for in-process pipeline calls. The toast
+just stops firing — no frontend changes, no config changes, no
+re-install.
+
+### Should I upgrade from v0.5.4 to v0.6.0?
+
+| Workflow | Recommendation |
+|---|---|
+| MCP client only (Claude Code, Cursor, etc.) | Yes — Phase 2 writers + faster install matter. No risk. |
+| `cerefox` CLI for ingest / search | Yes — same CLI, no API changes. Ingest still works via the Edge Function. |
+| Web UI to ingest documents | Optional — wait a few days for v0.7 if ingestion-via-web is your main flow. Or upgrade and use the CLI for ingest until v0.7. |
+| `uv run cerefox web` | Keep using it through v0.7; the Python web is unchanged. The deprecation banner lands in v0.7 once the TS web is feature-complete. |
+
+### Python web kept through v0.7.x
+
+`src/cerefox/api/app.py` and `routes_api.py` ship unchanged in v0.6.
+The Python web-specific deprecation banner is **deferred to v0.7's
+Part 25L** — we won't nudge users away from a fully-working Python
+web while the TS web's 3 ingest endpoints are still 503. v0.8 makes
+the banner prominent; v0.9 deletes the Python web code.
+
+---
+
+## v0.7.0 completes the TS migration arc
+
+**TL;DR**: the 3 ingestion endpoints that returned 503 in v0.6 now
+work. `cerefox ingest` and `cerefox ingest-dir` run in-process (no
+Edge Function round-trip). `cerefox reindex` (a v0.5 deferred stub)
+is a real command. PDF/DOCX support dropped. Python web prints a
+deprecation banner. Python MCP server keeps working unchanged.
+
+### The 503 toast is gone
+
+If you saw "Ingestion lands in v0.7 — use `cerefox ingest <file>` from
+the CLI for now" anywhere in the web UI during v0.6, that's gone in
+v0.7. The 3 ingestion endpoints (`POST /api/v1/ingest`,
+`POST /api/v1/ingest/file`, `POST /api/v1/documents/{id}/upload`) now
+call the in-process TS pipeline. The frontend's
+`V07IngestionDeferredError` toast detector stays in `api/client.ts`
+as dead code — no churn there, just stops firing.
+
+### CLI gets faster (no EF round-trip)
+
+`cerefox ingest <file>` and `cerefox ingest-dir <dir>` pre-v0.7 made
+an HTTP call to the `cerefox-ingest` Edge Function (Deno on Supabase).
+In v0.7+ they run the in-process TS pipeline directly: chunk +
+embed + atomic RPC, all in the same Bun/Node process. Faster + no
+network egress to Supabase Functions (only to Postgres + OpenAI).
+
+`cerefox reindex` is no longer a v0.5 stub. It re-embeds chunks via
+the same in-process pipeline. Defaults to stale-only (chunks with a
+different embedder model recorded); `--all` reindexes everything.
+`--batch <n>` controls the batch size. `--document-id <uuid>` scopes
+to one doc. `--dry-run` previews.
+
+### PDF / DOCX support dropped
+
+The `src/cerefox/chunking/converters.py` module and its tests are
+deleted. The Python CLI's `.pdf` / `.docx` branches now print a clear
+"support dropped in v0.7.0" error pointing at pandoc / docling for
+client-side conversion. The TS surfaces never had PDF/DOCX support;
+no UX change there.
+
+If you were using the Python CLI to ingest PDFs/DOCXs: convert them
+to markdown client-side first (`pandoc input.pdf -o input.md` or
+similar), then ingest the .md.
+
+### Python web shows a deprecation banner
+
+`uv run cerefox web` now prints a yellow ⚠ deprecation banner at
+startup:
+
+```
+  ⚠ Cerefox Python web server is deprecated as of v0.7.0.
+    The canonical web UI is `cerefox web` from `@cerefox/memory`
+    (npm install -g @cerefox/memory). The Python web stays through
+    v0.7.x and v0.8 as a husk; consider switching now.
+    See docs/guides/migration-v0.5.md § v0.7 for the migration path.
+```
+
+The Python web stays through v0.7.x and v0.8 (likely as a husk that
+returns 503 on every route in v0.8). v0.9's call on the Python web is
+TBD per the iter-26 design pass. Switch to `cerefox web` from npm
+when you can — it's been functionally complete since v0.6 + has had
+ingestion since v0.7.
+
+### Python MCP server stays unchanged
+
+Per the "Python minimization, not removal" policy locked at iter-24,
+the Python MCP server stays fully functional through v0.9+. If you
+check out the repo and run `uv run cerefox mcp`, that keeps working
+indefinitely. `CerefoxClient` stays in the Python tree for the same
+reason — MCP uses it.
+
+### Scripts: 3 ported, 2 stay Python
+
+| Script | Status in v0.7.0 |
+|---|---|
+| `scripts/db_deploy.py` | Husk; use `bun scripts/db_deploy.ts` |
+| `scripts/db_migrate.py` | Husk; use `bun scripts/db_migrate.ts` |
+| `scripts/reindex_all.py` | Husk; use `bun scripts/reindex_all.ts` |
+| `scripts/backup_create.py` | Stays Python through v0.7.x (port deferred) |
+| `scripts/backup_restore.py` | Stays Python through v0.7.x (port deferred) |
+
+The Postgres client used by `db_deploy.ts` / `db_migrate.ts` is the
+`postgres` (Porsager) library — small, well-typed, no native deps.
+Cross-runtime (Bun + Node).
+
+### Should I upgrade from v0.6.0 to v0.7.0?
+
+| Workflow | Recommendation |
+|---|---|
+| Web UI for ingestion | **Yes — that's the whole point.** v0.6 sent you to the CLI for ingest; v0.7 has it in the browser. |
+| MCP client only (Claude Code, Cursor, etc.) | Yes — no functional change for you, but you'll get the v0.7 npm cleanup. |
+| `cerefox` CLI | Yes — faster ingest paths + working reindex. |
+| `uv run cerefox web` (Python) | Optional — banner appears; can ignore for now. v0.8 will make this prominent. |
+| `uv run cerefox mcp` (Python) | No-op — Python MCP unchanged. |
+| PDF/DOCX ingest via Python CLI | Convert to markdown client-side before upgrading. |
+
+---
+
 ## Known gotchas
 
 ### `npx` from inside an npm workspace

package/docs/guides/quickstart.md

@@ -1,224 +1,104 @@
-# Quickstart -- Zero to First Document in 15 Minutes
+# Quickstart -- Zero to First Document in 5 Minutes
 
-Get Cerefox running locally and ingest your first document.
+Get Cerefox running on your machine via the npm install path. **No source
+clone, no Python required.**
 
-> **Upgrading from a previous version?** See the [Upgrading Guide](upgrading.md) for migration steps instead.
+> **Upgrading from an earlier version?** See [`upgrading.md`](upgrading.md)
+> for migration steps instead.
 
-## Two install paths
+---
 
-| You want | Read this section | Why |
-|---|---|---|
-| **Use Cerefox as an MCP server / CLI** | "Path A — npm install" below (fastest) | One install, callable from any directory. No Python needed. Recommended for end users since v0.5. |
-| **Contribute to Cerefox, run the web UI, deploy schema** | "Path B — source checkout" below | Full source: schema deploy, web UI, ingestion pipeline. Required for contributors and for the web UI (until v0.6). |
+## Prerequisites
 
-You'll need a Supabase project (free tier works) and an OpenAI API key for either
-path — those go into `.env`. The npm install asks for them interactively via
-`cerefox init`; the source checkout has you write them into a `.env` file
-yourself.
+- **Node.js 20+** (`node --version`) or **Bun 1.0+** (`bun --version`)
+- A **Supabase account** -- [supabase.com](https://supabase.com) (free tier works) -- with the Cerefox schema deployed (see [Note on schema deploy](#note-on-schema-deploy) below)
+- An **OpenAI API key** -- [platform.openai.com/api-keys](https://platform.openai.com/api-keys)
 
 ---
 
-## Path A — npm install (fastest, 5 min total)
-
-For end users who just want the Cerefox CLI + MCP server on their machine.
+## 1. Install
 
-### A.1 Prerequisites
-- Node.js 20+ (`node --version`) or Bun 1.0+ (`bun --version`)
-- A Supabase account -- [supabase.com](https://supabase.com) (free tier works)
-- An OpenAI API key -- [platform.openai.com/api-keys](https://platform.openai.com/api-keys)
-- **Schema must be deployed** to your Supabase. Until v0.6 ports the schema
-  deploy to TypeScript, this still requires the source checkout (Path B) once,
-  or someone else who has the source checkout to deploy it for you.
-
-### A.2 Install
 ```bash
-# One-line install (detects Bun or installs it, falls back to npm):
 curl -fsSL https://github.com/fstamatelopoulos/cerefox/releases/latest/download/install.sh | sh
-
-# Or direct:
-bun add -g @cerefox/memory       # preferred
-# npm install -g @cerefox/memory # alternative
 ```
 
-### A.3 First-run setup
-```bash
-cerefox init        # 5-step interactive setup
-cerefox doctor      # verify the install
-```
+Detects Bun (or installs it) and falls back to npm. After install,
+`cerefox` is on your PATH.
 
-### A.4 Wire up your MCP client
+Direct alternatives:
 ```bash
-# Run the ones that apply:
-cerefox configure-agent --tool claude-code
-cerefox configure-agent --tool claude-desktop
+bun add -g @cerefox/memory       # preferred (faster cold start)
+npm install -g @cerefox/memory   # equivalent
 ```
 
-`--tool claude-code` shells out to Claude Code's own `claude mcp add --scope user`
-to register the server (Claude Code knows where to store the config).
-`--tool claude-desktop` writes the JSON config file directly.
-
-Then restart your MCP client. **Path A users skip ahead to "[Connect an AI agent](#8-connect-an-ai-agent-optional-5-min)" (step 8) for the verification prompt.** Steps 3–7
-below are Path B-only (setting up `.env` by hand, deploying the schema, the web UI).
-
----
-
-## Path B — source checkout (contributors, schema deploy, web UI)
-
-For anyone hacking on Cerefox itself, deploying the schema for the first time,
-or running the web UI.
-
-### B.1 Prerequisites
-
-- Python 3.11+ (`python3 --version`)
-- Node.js 18+ and npm (`node --version`)
-- `uv` package manager (`pip install uv`)
-- A Supabase account -- [supabase.com](https://supabase.com) (free tier works)
-- An OpenAI API key -- [platform.openai.com/api-keys](https://platform.openai.com/api-keys)
-
-### B.2 Install Cerefox (2 min)
+## 2. First-run setup
 
 ```bash
-git clone https://github.com/fstamatelopoulos/cerefox.git
-cd cerefox
-uv sync
+cerefox init        # 5-step interactive setup — prompts for the credentials above
+cerefox doctor      # green across the board if everything's wired correctly
 ```
 
-> No heavy ML model downloads needed -- embeddings are handled by the OpenAI API.
+`cerefox init` validates each entry against the live service before saving,
+writes `~/.cerefox/.env` (mode 0600), and optionally ingests the bundled
+self-docs into the `_cerefox-self-docs` project so agents can search for
+Cerefox usage guidance.
 
----
-
-## 3. Set up Supabase (5 min)
-
-1. Create a new Supabase project at [app.supabase.com](https://app.supabase.com).
-2. Go to **Project Settings → API → Project URL** and copy it. Also note your project ref (the slug in the URL, e.g. `abcd1234`).
-3. Go to **Project Settings → API Keys** and copy the **Secret key** (`sb_secret_…`). The legacy `service_role` JWT also works if you prefer; either goes into `CEREFOX_SUPABASE_KEY`. See [`setup-supabase.md` → Supabase API keys (2026)](setup-supabase.md#supabase-api-keys-2026) for the full key story (including why the anon key, if you ever need it, must currently stay as the legacy JWT — `sb_publishable_…` does not work for Edge Functions).
-4. Go to **Project Settings → Database → Connection pooling** and copy the **Session Pooler** URI (host ends `.pooler.supabase.com`, port `5432`). If you only see the Transaction Pooler in the dashboard, take that URI and change `:6543` → `:5432`. **Do not use port 6543** — Transaction Pooler does not support DDL. See [`setup-supabase.md` → Connection pooling (2026)](setup-supabase.md#connection-pooling-2026) for context.
-
-Create a `.env` file:
-
-```env
-CEREFOX_SUPABASE_URL=https://your-project-ref.supabase.co
-CEREFOX_SUPABASE_KEY=sb_secret_...your-supabase-secret-key...
-CEREFOX_DATABASE_URL=postgresql://postgres.your-project-ref:your-db-password@aws-N-region.pooler.supabase.com:5432/postgres?sslmode=require
-OPENAI_API_KEY=sk-...your-openai-key...
-```
-
-The username must include the `.<project-ref>` suffix (e.g. `postgres.abcd1234`) — without it, Supabase returns "Tenant or user not found".
-
----
-
-## 4. Deploy the schema (1 min)
+## 3. Wire up an AI agent
 
 ```bash
-uv run python scripts/db_deploy.py
+# Run the commands that apply to your setup:
+cerefox configure-agent --tool claude-code      # Claude Code (~/.claude.json)
+cerefox configure-agent --tool claude-desktop   # Claude Desktop config
 ```
 
-You should see all steps complete with a final `Done` message.
+Then restart your client:
+- **Claude Code**: start a fresh session — running sessions cache the MCP tool list.
+- **Claude Desktop**: Cmd+Q to fully quit, then relaunch.
 
-Verify:
-```bash
-uv run python scripts/db_status.py
-```
+Cursor, OpenAI Codex CLI, and Gemini CLI ship in a follow-up (v0.6+).
+For manual setup of those today, see [`connect-agents.md`](connect-agents.md).
 
-This should show all checks passed.
+## 4. Try it
 
----
+From your AI agent, ask:
 
-## 5. Ingest your first document (2 min)
+> "Use cerefox_search to look for 'cerefox conventions' and tell me what you find."
 
-Have a markdown file? Ingest it:
-
-```bash
-uv run cerefox ingest my-notes.md
-```
-
-Or paste directly from the terminal:
-
-```bash
-echo "# My First Note
-
-This is the beginning of my personal knowledge base." | uv run cerefox ingest --paste --title "First Note"
-```
-
----
-
-## 6. Build and start the web app (1 min)
-
-Build the React frontend:
-
-```bash
-cd frontend && npm install && npm run build && cd ..
-```
-
-Start the web app:
-
-```bash
-uv run cerefox web
-```
-
-Open [http://localhost:8000/app/](http://localhost:8000/app/) -- your dashboard is live.
-
-> The root URL (`http://localhost:8000/`) redirects to `/app/` automatically.
+You should see results from the bundled self-docs.
 
 ---
 
-## 7. Search your knowledge (30 sec)
+## Note on schema deploy
 
-From the CLI:
+If your Supabase project is **brand new**, the Cerefox schema needs to be
+deployed once before the CLI works. Until v0.6 ports the schema deploy to
+TypeScript, this is the one step that still requires the source-checkout
+path:
 
 ```bash
-uv run cerefox search "my first note"
-```
-
-Or use the web UI search page at [http://localhost:8000/app/search](http://localhost:8000/app/search).
-
----
-
-## 8. Connect an AI agent (optional, 5 min)
-
-Cerefox ships a built-in MCP server. Add it to Claude Desktop's config file
-(`~/Library/Application Support/Claude/claude_desktop_config.json`):
-
-```json
-{
-  "mcpServers": {
-    "cerefox": {
-      "command": "uv",
-      "args": ["--directory", "/path/to/cerefox", "run", "cerefox", "mcp"]
-    }
-  }
-}
+git clone https://github.com/fstamatelopoulos/cerefox.git && cd cerefox
+uv sync
+# Add CEREFOX_DATABASE_URL to your .env, then:
+uv run python scripts/db_deploy.py
 ```
 
-Replace `/path/to/cerefox` with the absolute path to this checkout. Restart Claude Desktop.
-
-> **Recommended: remote MCP** -- if you deployed the Edge Functions (see the main
-> README), use the remote MCP path instead -- no Python install needed on the client machine.
-> See `docs/guides/connect-agents.md` for Path A-Remote.
->
-> **ChatGPT** does not support MCP -- use a Custom GPT with
-> Edge Functions instead (see `docs/guides/connect-agents.md`, Path B).
-
-For full setup details (remote MCP, Cursor, cloud clients, GPT Actions), see `docs/guides/connect-agents.md`.
+Detailed walkthrough: [`setup-supabase.md`](setup-supabase.md).
+After v0.6.0, `cerefox init` will offer to do this for you.
 
 ---
 
-## You're done!
-
-**What's next:**
-- Ingest a directory of notes: `cerefox ingest-dir ./notes/ --recursive`
-- Re-embed existing content: `cerefox reindex`
-- Create a backup: `python scripts/backup_create.py`
-- Sync project docs into your knowledge base: `python scripts/sync_docs.py`
-  (this also ingests the agent reference guides -- `AGENT_GUIDE.md` and `AGENT_QUICK_REFERENCE.md` --
-  so your AI agents can search for "How AI Agents Use Cerefox" and learn how to use the tools)
-- See all commands: `cerefox --help`
-
-**More guides:**
-- `AGENT_GUIDE.md` -- comprehensive reference for AI agents using Cerefox tools
-- `AGENT_QUICK_REFERENCE.md` -- minimal quick reference card for AI agents
-- `docs/guides/setup-supabase.md` -- detailed Supabase setup
-- `docs/guides/configuration.md` -- all configuration options
-- `docs/guides/connect-agents.md` -- connecting AI agents via MCP and Edge Functions
-- `docs/guides/setup-local.md` -- local Docker setup (no Supabase account needed)
-- `docs/guides/upgrading.md` -- upgrading from a previous version
+## What's next
+
+- **Ingest your notes**: `cerefox ingest my-notes.md`, or
+  `cerefox ingest-dir ./notes/ --recursive`
+- **Search from the CLI**: `cerefox search "your query"`
+- **Discover all commands**: `cerefox --help`
+- **Run the web UI** (Python-only until v0.6): [`setup-local.md`](setup-local.md)
+- **Connect more AI clients** (Cursor, Codex, ChatGPT GPT Actions, etc.):
+  [`connect-agents.md`](connect-agents.md)
+- **Configuration reference**: [`configuration.md`](configuration.md)
+- **Backup + restore**: [`ops-scripts.md`](ops-scripts.md)
+
+For the agent-facing reference (what tools agents have, how to use them well),
+read `AGENT_QUICK_REFERENCE.md` in the repo root — or have your agent run
+`cerefox_get_help` from any MCP-connected client.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@cerefox/memory",
-  "version": "0.5.4",
+  "version": "0.7.0",
   "description": "Cerefox — user-owned shared memory for AI agents. The local TypeScript runtime: stdio MCP server in v0.4; CLI binary added in v0.5; in-process web server in v0.6; ingestion pipeline in v0.7.",
   "license": "Apache-2.0",
   "homepage": "https://github.com/fstamatelopoulos/cerefox",
@@ -39,12 +39,16 @@
     "CHANGELOG.md"
   ],
   "dependencies": {
+    "@hono/node-server": "^2.0.4",
     "@modelcontextprotocol/sdk": "^1.0.0",
     "@supabase/supabase-js": "^2.45.0",
     "cli-progress": "^3.12.0",
     "commander": "^12.1.0",
+    "hono": "^4.12.23",
     "picocolors": "^1.0.0",
+    "postgres": "^3.4.9",
     "prompts": "^2.4.2",
+    "smol-toml": "^1.6.1",
     "zod": "^3.23.0"
   },
   "devDependencies": {
@@ -58,7 +62,9 @@
     "build": "bun build src/bin/cerefox.ts --outdir dist/bin --target node --format esm",
     "clean": "rm -rf dist docs AGENT_GUIDE.md AGENT_QUICK_REFERENCE.md",
     "bundle-docs": "bun run ../../scripts/bundle_package_docs.ts",
-    "prepublishOnly": "bun run clean && bun run bundle-docs && bun run build",
+    "build-frontend": "cd ../../frontend && bun install && bun run build",
+    "bundle-frontend": "rm -rf dist/frontend && mkdir -p dist/frontend && cp -R ../../frontend/dist/. dist/frontend/",
+    "prepublishOnly": "bun run clean && bun run bundle-docs && bun run build-frontend && bun run bundle-frontend && bun run build",
     "smoke": "node dist/bin/cerefox.js --help && node dist/bin/cerefox.js mcp --help"
   },
   "publishConfig": {