@cerefox/memory 1.17.4 → 1.19.0

package/docs/guides/connect-agents.md

@@ -24,7 +24,7 @@ client; you can also run both in parallel.
 Three top-level paths plus a few special cases:
 
 - **Path A** — MCP server (local subprocess or remote Edge Function). Best for purpose-built agent clients like Claude Desktop, Cursor, and Claude Code's MCP integration.
-- **Path B** — direct Edge Function HTTP. Best for ChatGPT Custom GPTs and any HTTP caller (curl, scripts).
+- **Path B** — direct Edge Function HTTP. For any HTTP caller (curl, scripts), and for ChatGPT Custom GPTs until OpenAI retires them on December 11, 2026.
 - **Path C** — local shell CLI invoked by a coding agent's Bash tool. Best for Claude Code, Codex CLI, opencode, OpenClaw, Hermes, and similar local-agent CLIs **when the user prefers not to configure MCP** but still wants the agent to read and write Cerefox.
 
 | Client | Path | Search | Requirements / caveats |
@@ -33,7 +33,8 @@ Three top-level paths plus a few special cases:
 | Claude Code (remote) | Path A-Remote — `cerefox-mcp` Edge Function | Hybrid | URL + Cerefox token only; no local install. Advanced/fallback — prefer Path A-Local |
 | Cursor (remote) | Path A-Remote — `cerefox-mcp` Edge Function | Hybrid | URL + Cerefox token only; no local install. Advanced/fallback — prefer Path A-Local |
 | OpenAI Codex CLI (remote) | Path A-Remote — `cerefox-mcp` Edge Function | Hybrid | URL + Cerefox token env var; TOML config. Advanced/fallback — prefer Path A-Local |
-| ChatGPT (chatgpt.com or desktop) | Path B — Custom GPT → Edge Functions | Hybrid | ChatGPT Plus required |
+| ChatGPT (chatgpt.com or desktop) | Path B — Custom GPT → Edge Functions | Hybrid | ChatGPT Plus required. **Ends 2026-12-11** when Custom GPTs retire; see [ChatGPT Custom GPT](#chatgpt-custom-gpt-cloud--chatgptcom) |
+| OpenAI Codex (CLI, and Codex threads in the ChatGPT desktop app) (local) | Path A-Local — `cerefox configure-agent --tool codex` | Hybrid | Writes `~/.codex/config.toml`; all 15 tools; zero Edge Function invocations. **Recommended for Codex** |
 | Claude Desktop (local) | Path A-Local — `@cerefox/memory` via `npx` | Hybrid | Local alternative; Node.js; zero Edge Function invocations |
 | Claude Code (local) | Path A-Local — `@cerefox/memory` via `npx` | Hybrid | Local alternative; zero Edge Function invocations |
 | Cursor (local) | Path A-Local — `@cerefox/memory` via `npx` | Hybrid | Local alternative; zero Edge Function invocations |
@@ -50,7 +51,8 @@ Three top-level paths plus a few special cases:
 > over the standard tool surface — no Cloud Run needed. Setup:
 > [setup-supabase Step 7](setup-supabase.md#step-7--oauth-for-cloud-agents-claudeai--mobile-optional)
 > + [Cloud Claude](#cloud-claude-claudeai-web--mobile-oauth) below. (An OAuth connector for
-> ChatGPT becomes possible on the same server but is not yet documented.)
+> ChatGPT is expected to replace the Custom GPT path when Custom GPTs retire; it is being
+> validated in [#326](https://github.com/fstamatelopoulos/cerefox/issues/326).)
 
 > **Perplexity** supports stdio-only MCP on macOS Desktop (via Helper App). Remote MCP is
 > "coming soon." Perplexity's CTO has signaled a strategic shift away from MCP (March 2026),
@@ -265,15 +267,26 @@ For the manual JSON (for the curious, or to debug if the CLI can't write your co
 
 ---
 
-### ChatGPT Desktop
+### ChatGPT Desktop and Codex
 
-> **ChatGPT Desktop does not support local stdio MCP servers.**
-> OpenAI's MCP implementation for ChatGPT only supports remote servers via SSE or
-> streaming HTTP — not local subprocess (stdio) servers like `cerefox mcp`.
-> The "dev mode" MCP connector visible in the app also requires a public URL.
->
-> **Use Path B (Custom GPT + Edge Functions) for all ChatGPT access** — both the web
-> app and the desktop app. The Custom GPT approach is fully validated and works well.
+The ChatGPT desktop app hosts two different things, and they connect differently:
+
+- **Codex threads** (and the Codex CLI) run local stdio MCP servers. Set them up with
+  `cerefox configure-agent --tool codex`, which writes the `cerefox` entry into
+  `~/.codex/config.toml`; the app's **Plugins → MCPs** screen edits the same entry. Start
+  a new Codex session after changing it (it reads the config at session start), then ask
+  it to call `cerefox_get_help`: the reply begins with the server version. All 15 tools
+  are available.
+- **Regular ChatGPT chats** do not get local MCP servers, in the desktop app or anywhere
+  else. ChatGPT only connects to **remote** MCP servers (OAuth or no auth). Today ChatGPT
+  chats reach Cerefox through the [Custom GPT](#chatgpt-custom-gpt-cloud--chatgptcom)
+  path, which ends on December 11, 2026; see the notice there.
+
+> **Tip**: `configure-agent` writes the `npx` form, which runs whatever version npx has
+> cached. To keep Codex on the same version as your installed CLI (and start faster),
+> point the entry at the binary instead: `command = "<path to cerefox>"`, `args = ["mcp"]`
+> (`which cerefox` prints the path; use the full path, since GUI apps may not see your
+> shell's `PATH`).
 
 ---
 
@@ -593,6 +606,8 @@ You have access to a personal knowledge base via the searchKnowledgeBase action.
 When the user asks a question, always search the knowledge base first using a
 relevant query. Present results by document title, citing the source for every claim.
 Use ingestNote to save any new information the user asks you to remember.
+Pass author: "ChatGPT" on every action call, so your reads and writes are
+attributed in the audit and usage logs.
 When UPDATING an existing document, first call getDocument and note its
 content_hash, then pass it as expected_content_hash on ingestNote. If you get a
 409 conflict, the document changed underneath you: call getDocument again, merge
@@ -620,6 +635,14 @@ Expected: JSON response with `results` array containing documents.
 
 ### ChatGPT Custom GPT (cloud — chatgpt.com)
 
+> **Retiring on December 11, 2026.** OpenAI is retiring Custom GPTs in favor of plugins,
+> and **custom actions do not carry over**: they have to be rebuilt as a connector or a
+> custom MCP server. This section works until then, but do not build a new setup on it.
+> ChatGPT will need a new connection, an MCP connector to `cerefox-mcp` over OAuth (the
+> way [cloud Claude](#cloud-claude-claudeai-web--mobile-oauth) connects), which is being
+> validated in [#326](https://github.com/fstamatelopoulos/cerefox/issues/326). Background and plan: [`chatgpt-after-custom-gpts.md`](../research/chatgpt-after-custom-gpts.md).
+> For **Codex**, use the local MCP instead (see [ChatGPT Desktop and Codex](#chatgpt-desktop-and-codex)).
+
 A Custom GPT with Actions pointing at the Edge Functions gives ChatGPT full hybrid search from
 any browser — no local install, no MCP client, works free with ChatGPT Plus.
 
@@ -638,7 +661,7 @@ In the action editor, paste this schema (replace `<your-project-ref>`):
 openapi: 3.1.0
 info:
   title: Cerefox Knowledge Base
-  version: 4.3.0
+  version: 4.4.0
 servers:
   - url: https://<your-project-ref>.supabase.co/functions/v1
 paths:
@@ -770,9 +793,10 @@ paths:
                 metadata:
                   type: object
                   description: >
-                    Arbitrary JSON metadata. On an UPDATE, omitting this keeps
-                    the document's existing metadata (v2.1.0); pass {} to
-                    deliberately clear all tags.
+                    Arbitrary JSON metadata. On an UPDATE it is MERGED into the
+                    existing metadata (v4.4.0): keys you pass are set, a null
+                    value removes that key, keys you omit are kept. Omitting
+                    the field keeps the metadata as it is.
                 update_if_exists:
                   type: boolean
                   default: false
@@ -790,6 +814,13 @@ paths:
                     If the document changed since it was read, the update fails
                     with HTTP 409 — re-read the document, merge your changes,
                     retry with the new hash. Not needed when creating.
+                expected_metadata_revision:
+                  type: integer
+                  description: >
+                    Optional, with metadata on an update (v4.4.0): the
+                    metadata_revision returned by getDocument. If the metadata
+                    changed since, the update fails with HTTP 409
+                    (error "metadata_conflict").
                 last_write_wins:
                   type: boolean
                   default: false
@@ -839,7 +870,9 @@ paths:
             changes, and retry with the new hash — do not overwrite blindly),
             or the target document is soft-deleted (error
             "document_deleted") — restore it first, then retry, or create a
-            new document.
+            new document, or the metadata changed since it was read (error
+            "metadata_conflict") — re-read with getDocument and retry with
+            the new metadata_revision.
         '422':
           description: >
             Unresolved document links (error "unresolved_links") — the content
@@ -899,7 +932,8 @@ paths:
           description: >
             Document content and metadata:
             { document_id, doc_title, full_content, chunk_count, total_chars,
-              is_archived, version_id, content_hash }.
+              is_archived, version_id, content_hash, metadata,
+              metadata_revision }.
             content_hash is the document's CURRENT hash — pass it back as
             expected_content_hash when updating via ingestNote.
         '404':
@@ -1084,15 +1118,16 @@ paths:
 In the action's **Authentication** settings:
 - Type: **API Key**
 - Auth type: **Bearer**
-- API key: your **Cerefox access token** (`cfx_pat_…`). Generate it with
-  `cerefox token generate` (it prints the token once and sets it on Supabase).
+- API key: your **Cerefox access token** (`cfx_pat_…`). If you already have one
+  (`cerefox token list` shows it; the value is `CEREFOX_ACCESS_TOKEN` in your `.env`),
+  reuse it. Only on a first setup run `cerefox token generate`: it **replaces** the
+  server's token, which breaks every client still using the old one. To add a second
+  token alongside the first, use `cerefox token rotate`.
   The legacy Supabase anon key is no longer accepted (iter-28E).
 
-> **Important:** ChatGPT resets the stored API key when you update the action
-> schema. This release bumps the schema `info.version`, so on your next schema
-> paste ChatGPT will clear the key — re-enter your **Cerefox access token** in the
-> authentication settings. (That reset is expected here: it's what swaps you off
-> the old anon key.)
+> **Important:** ChatGPT clears the stored API key whenever you save a changed action
+> schema. After pasting a new schema version, re-enter your **Cerefox access token** in
+> the authentication settings.
 
 **Step 4 — Save and test**
 
@@ -1131,7 +1166,7 @@ curl -s -X POST \
   }'
 ```
 
-If the same content was already ingested (SHA-256 hash match), returns `"skipped": true`.
+If a live document already has the same content (SHA-256 hash match), a create returns that document with `"skipped": true` instead of a copy. Documents in the trash do not count, and updates never de-duplicate.
 
 **Edge Function parameters — `cerefox-search`:**
 

package/docs/guides/ops-scripts.md

@@ -177,7 +177,7 @@ Output: `backup-data/cerefox-20260308T143022Z-before-v2-migration.json`
 
 ## backup_restore.ts — Restore from a backup
 
-Restores documents and chunks from a previously created backup file. Idempotent — documents with the same content hash are skipped. (End users use `cerefox backup restore`.)
+Restores documents and chunks from a previously created backup file. Idempotent — a document is skipped when its id is already present, or when a live document outside the backup already has the same content. (End users use `cerefox backup restore`.)
 
 ```bash
 bun scripts/backup_restore.ts BACKUP_FILE [OPTIONS]

package/docs/guides/quickstart.md

@@ -83,7 +83,8 @@ in place, so re-run it after upgrading. Detailed walkthrough:
 ## 3b. Generate the Edge Function access token
 
 The Edge Functions authenticate callers with a **Cerefox access token**. You need it
-if you'll connect a **Custom GPT** (GPT Actions) or a **remote HTTP MCP** client, and
+if you'll connect a **Custom GPT** (GPT Actions, until OpenAI retires Custom GPTs on
+December 11, 2026) or a **remote HTTP MCP** client, and
 for a fully-green `cerefox doctor`. The **local MCP** (local agents), **cloud Claude**
 (OAuth), and the CLI/web reach Supabase over the Data API and don't use it — so this
 step is optional if that's your whole setup, but it's quick and harmless to run:
@@ -147,8 +148,11 @@ clone the repo, run `bun install`, and use the contributor scripts
 - **Search from the CLI**: `cerefox search "your query"`
 - **Discover all commands**: `cerefox --help`
 - **Run the web UI**: `cerefox web` (TypeScript — Hono backend + React SPA); see [`cli.md`](cli.md)
-- **Connect more AI clients** (Cursor, Codex, ChatGPT GPT Actions, etc.):
-  [`connect-agents.md`](connect-agents.md)
+- **Connect more AI clients** (Cursor, Codex, Gemini, cloud Claude, etc.):
+  [`connect-agents.md`](connect-agents.md). Codex works with the local MCP
+  (`cerefox configure-agent --tool codex`). ChatGPT's Custom GPT path ends on
+  December 11, 2026; see
+  [`chatgpt-after-custom-gpts.md`](../research/chatgpt-after-custom-gpts.md)
 - **Configuration reference**: [`configuration.md`](configuration.md)
 - **Backup + restore**: [`ops-scripts.md`](ops-scripts.md)
 

package/docs/guides/setup-local.md

@@ -4,9 +4,11 @@ Run Cerefox **fully on your own machine** — no hosted Supabase, no cloud datab
 Docker container bundles everything: Postgres + pgvector, the PostgREST Data API, and the
 Cerefox web server. You get the same web UI, CLI, and MCP server as the cloud setup.
 
-> **Embeddings still use the OpenAI API.** An `OPENAI_API_KEY` is required even for a
-> local setup (the database and web server are local; embedding generation is not). A
-> fully offline embedder is on the roadmap.
+> **Two embedding choices.** By default embeddings use the OpenAI API (an
+> `OPENAI_API_KEY` is needed). Or choose the **local embedder**
+> (`--local-embedder`, `CEREFOX_EMBEDDER=local`): a model that runs inside the
+> container, fully offline, no API key. See
+> [Choose your embedder](#choose-your-embedder-openai-vs-fully-local).
 
 ## Cloud vs. Local — pick one
 
@@ -49,9 +51,14 @@ install; switching later requires a re-index (see below).
 The local model (~130 MB) downloads once — at install/init when selected — into the
 data volume, so it survives `cerefox-local upgrade`.
 
-> **Memory**: give the Docker VM **≥ 4 GB** for comfortable local-embedder use
-> (Colima defaults to 2 GB: `colima start --memory 4`). Inference is
-> sub-batched to keep peak memory flat, so smaller VMs work — just slower.
+> **Memory**: with the local embedder, each Cerefox Local instance's web
+> process sits around **0.3–0.5 GB** idle and reaches about **1.5 GB** while
+> ingesting long documents dense with tables, ids or code (measured on
+> synthetic 20,000-character documents; Postgres and PostgREST add a little
+> more). Give the Docker VM room for **each** instance you run, plus everything
+> else in it: **4 GB is comfortable for one instance**, not two. Colima defaults
+> to 2 GB (`colima start --memory 4`, or `--memory 8` for two instances). If the
+> VM runs out, the kernel kills the web process; see Troubleshooting.
 
 > **Switching embedders on existing data is breaking**: the two models produce
 > incompatible vector spaces, so documents embedded with one are invisible to
@@ -193,6 +200,23 @@ Re-run the installer with the key, or set it and `cerefox-local upgrade`.
 
 **Port already in use** — re-install with a free port: `PORT=8017 sh -c "$(curl -fsSL …/install-local.sh)"`.
 
+**Requests fail with "socket connection closed", or the web process restarts on
+its own** — usually the Docker VM ran out of memory and the kernel killed the
+web process (most often during ingest with the local embedder). s6 restarts it
+within seconds, so the container keeps running. Since v1.19.0 the container log
+says so (`docker logs cerefox-local` shows "killed (SIGKILL), most likely by the
+kernel's out-of-memory killer"); on older versions the log shows only the
+restart. Confirm it in the VM's kernel log:
+
+```sh
+colima ssh -- sudo dmesg | grep -i "out of memory"   # Colima
+```
+
+(Docker Desktop: the same `dmesg` from a privileged container, e.g.
+`docker run --rm --privileged alpine dmesg | grep -i "out of memory"`.) The fix is
+more memory for the VM, or fewer instances in it; see **Memory** above.
+`CEREFOX_ONNX_BATCH` (default 1) trades memory for nothing useful above 1, so leave it.
+
 ---
 
 ## Contributor notes

package/docs/guides/setup-supabase.md

@@ -220,6 +220,11 @@ FTS-only `mcp.supabase.com` path). This needs a one-time OAuth setup — see
 [Step 7](#step-7--oauth-for-cloud-agents-claudeai--mobile-optional) below.
 
 **For cloud ChatGPT** — create a Custom GPT with GPT Actions pointing at the Edge Functions.
+This works until **December 11, 2026**, when OpenAI retires Custom GPTs (custom actions do
+not carry over to its plugins). ChatGPT's replacement connection, an MCP connector over
+OAuth like the cloud Claude one, is being validated in
+[#326](https://github.com/fstamatelopoulos/cerefox/issues/326). **For Codex**, use the local
+MCP: `cerefox configure-agent --tool codex`.
 
 See `docs/guides/connect-agents.md` for the complete guide including system prompts,
 architecture explanation, and ChatGPT GPT Actions setup.

package/docs/guides/upgrading.md

@@ -194,6 +194,20 @@ knowing about:
   document must be restored before its content can be updated.
 - **v1.13.0 — the review workflow is a store setting.** Upgraded stores keep
   it on; fresh installs start with it off. See the callout above.
+- **Schema 0.20.0 — content no longer has to be unique.** Updates that match
+  another document, or a document in the trash, are accepted; only creating a
+  document de-duplicates, against live documents. Needs `cerefox server deploy`.
+  Until the server is deployed, a 1.19.0 client writing content another
+  document (trashed ones included) already has gets an error telling it to
+  deploy. A client older than 1.19.0 against the new server is safe: when two
+  identical creates race, the loser gets an error, as it did before.
+- **Schema 0.21.0 — metadata merges on content updates and has its own token.**
+  A content update's `metadata` now merges (`null` removes a key) instead of
+  replacing, so `{}` no longer clears tags; `replace` needs
+  `expected_metadata_revision`. Scripts that relied on replace-on-update should
+  use `document set-metadata --replace`. Needs `cerefox server deploy`; then
+  restart MCP clients and `cerefox web`, since an older client's metadata
+  `replace` is refused for lack of the token.
 - **v1.13.1 — the flag hides, it does not rewrite.** Agent writes are
   recorded `pending_review` whether or not the workflow is shown (v1.13.0
   stored `approved` while off). Redeploy to pick it up.
@@ -223,7 +237,8 @@ MCP path needs no reconfiguration; for the local stdio server, restart the
 client. `cerefox configure-agent --tool <client>` (re)writes a client's config
 if you need it.
 
-**ChatGPT Custom GPT (GPT Actions):** after an upgrade, check the OpenAPI schema
+**ChatGPT Custom GPT (GPT Actions, until December 11, 2026, when OpenAI retires
+Custom GPTs):** after an upgrade, check the OpenAPI schema
 version in [`connect-agents.md`](connect-agents.md); if it changed, paste the new
 schema into the Custom GPT editor and **re-enter your Cerefox access token**
 (`cfx_pat_…`, from `cerefox token generate`) as the Bearer token. (The editor

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@cerefox/memory",
-  "version": "1.17.4",
+  "version": "1.19.0",
   "description": "Cerefox \u2014 user-owned shared memory for AI agents. CLI + stdio MCP server + web UI + ingestion for a knowledge base on your own Supabase project (or fully self-hosted with Cerefox Local).",
   "license": "Apache-2.0",
   "homepage": "https://github.com/fstamatelopoulos/cerefox",
@@ -39,12 +39,12 @@
     "CHANGELOG.md"
   ],
   "dependencies": {
-    "@hono/node-server": "^2.1.1",
-    "@modelcontextprotocol/sdk": "^1.30.1",
-    "@supabase/supabase-js": "^2.117.2",
+    "@hono/node-server": "^2.1.4",
+    "@modelcontextprotocol/sdk": "^1.32.1",
+    "@supabase/supabase-js": "^2.117.3",
     "cli-progress": "^3.12.0",
     "commander": "^15.0.0",
-    "hono": "^4.13.9",
+    "hono": "^4.13.13",
     "mammoth": "^1.13.0",
     "ora": "^9.4.0",
     "picocolors": "^1.0.0",
@@ -54,13 +54,13 @@
     "zod": "^4.6.5"
   },
   "optionalDependencies": {
-    "@huggingface/transformers": "^4.3.0",
+    "@huggingface/transformers": "^4.3.1",
     "onnxruntime-node": "^1.21.0"
   },
   "devDependencies": {
     "@types/bun": "^1.4.2",
     "@types/cli-progress": "^3.11.5",
-    "@types/node": "^24.19.0",
+    "@types/node": "^24.19.1",
     "@types/prompts": "^2.4.9",
     "typescript": "^6.0.3"
   },