@cerefox/memory 1.0.0-rc.3 → 1.0.0

package/README.md

@@ -56,7 +56,7 @@ You deploy the server side with this package's CLI — `cerefox server deploy` s
 | Prerequisite | Why | How |
 |---|---|---|
 | A **Supabase project** | Hosts Postgres + pgvector + Edge Functions. Free tier is enough for most personal use. | [supabase.com](https://supabase.com) → New project |
-| An **embedding API key** | OpenAI `text-embedding-3-small` (the only embedder wired today). Pennies/month for typical personal use (see [operational-cost.md](https://github.com/fstamatelopoulos/cerefox/blob/main/docs/guides/operational-cost.md)). | Get an [OpenAI API key](https://platform.openai.com/api-keys). |
+| An **embedding API key** | OpenAI `text-embedding-3-small` (the cloud backend's embedder; Cerefox Local can instead run a fully-offline local model). Pennies/month for typical personal use (see [operational-cost.md](https://github.com/fstamatelopoulos/cerefox/blob/main/docs/guides/operational-cost.md)). | Get an [OpenAI API key](https://platform.openai.com/api-keys). |
 | **Node ≥ 20** or **Bun ≥ 1.0** | Runtime for the `cerefox` bin (and the bundled `cerefox mcp` server). | [nodejs.org](https://nodejs.org) · [bun.sh](https://bun.sh). The one-line installer below bootstraps Bun if neither is present. |
 
 ### One-time server-side setup (~10 min — no clone needed)

package/dist/bin/cerefox.js

@@ -7184,7 +7184,7 @@ var exports_meta = {};
 __export(exports_meta, {
   PKG_VERSION: () => PKG_VERSION
 });
-var PKG_VERSION = "1.0.0-rc.3";
+var PKG_VERSION = "1.0.0";
 var init_meta = () => {};
 
 // ../../node_modules/.bun/tslib@2.8.1/node_modules/tslib/tslib.js
@@ -25008,10 +25008,11 @@ function getMaxResponseBytes() {
 }
 function getMinSearchScore() {
   const raw = globalThis.process?.env?.CEREFOX_MIN_SEARCH_SCORE;
+  const fallback = globalThis.process?.env?.CEREFOX_EMBEDDER === "local" ? DEFAULT_MIN_SEARCH_SCORE_LOCAL : DEFAULT_MIN_SEARCH_SCORE;
   if (raw === undefined || raw === "")
-    return DEFAULT_MIN_SEARCH_SCORE;
+    return fallback;
   const n = Number.parseFloat(raw);
-  return Number.isNaN(n) || n < 0 || n > 1 ? DEFAULT_MIN_SEARCH_SCORE : n;
+  return Number.isNaN(n) || n < 0 || n > 1 ? fallback : n;
 }
 function applyByteBudget(rows, maxBytes) {
   const accepted = [];
@@ -25040,7 +25041,7 @@ function logUsage(supabase, params) {
     p_extra: params.extra ?? {}
   })).catch(() => {});
 }
-var MAX_RESPONSE_BYTES = 200000, DEFAULT_MIN_SEARCH_SCORE = 0.5;
+var MAX_RESPONSE_BYTES = 200000, DEFAULT_MIN_SEARCH_SCORE = 0.5, DEFAULT_MIN_SEARCH_SCORE_LOCAL = 0.6;
 
 // ../../_shared/mcp-tools/_projects.ts
 async function ensureDocumentInProject(supabase, documentId, projectName) {
@@ -74659,7 +74660,7 @@ import { homedir as homedir6 } from "node:os";
 import { join as join9 } from "node:path";
 
 // ../../_shared/ef-meta/index.ts
-var EF_VERSION = "1.0.0-rc.3";
+var EF_VERSION = "1.0.0-rc.4";
 
 // src/cli/util/checks.ts
 init_config();
@@ -74717,6 +74718,14 @@ function checkConfig() {
     };
   }
   if (!existsSync10(envPath)) {
+    const settings = loadSettings();
+    if (settings.supabaseUrl && settings.supabaseKey) {
+      return {
+        name: "config",
+        status: "ok",
+        detail: "configured via environment variables (no .env file — normal for Cerefox Local)"
+      };
+    }
     return {
       name: "config",
       status: "error",
@@ -75036,9 +75045,9 @@ function checkMcpConfigs() {
   if (found.length === 0) {
     return {
       name: "mcp clients",
-      status: "warn",
-      detail: "No MCP client configs reference Cerefox.",
-      hint: "Run `cerefox configure-agent --tool claude-code` (or `--tool claude-desktop`) to wire up a client."
+      status: isLocalBackend() ? "skipped" : "warn",
+      detail: isLocalBackend() ? "checked from inside the container — host MCP configs are not visible here." : "No MCP client configs reference Cerefox.",
+      hint: isLocalBackend() ? "Configure agents on the host with `cerefox-local configure-agent`." : "Run `cerefox configure-agent --tool claude-code` (or `--tool claude-desktop`) to wire up a client."
     };
   }
   return {
@@ -75110,7 +75119,17 @@ async function checkPostgres() {
     await sql.end({ timeout: 1 }).catch(() => {});
   }
 }
+function isLocalBackend() {
+  return Boolean(process.env.CEREFOX_POSTGREST_UPSTREAM);
+}
 async function checkEdgeFunctionsCompat() {
+  if (isLocalBackend()) {
+    return {
+      name: "edge functions",
+      status: "skipped",
+      detail: "local backend — Edge Functions are not used (cloud-only surface)."
+    };
+  }
   const settings = loadSettings();
   if (!settings.supabaseUrl) {
     return {
@@ -77247,7 +77266,7 @@ async function action29(query, options) {
   }
 }
 function registerSearch(program2) {
-  program2.command("search").description("Search the knowledge base (hybrid FTS + semantic).").argument("<query>", "Natural-language search query.").option("-c, --match-count <n>", "Maximum number of documents to return.", "5").option("-p, --project-name <name>", "Filter results to a specific project.").option("-f, --metadata-filter <json>", "JSON containment filter; only docs whose metadata contains ALL pairs are returned.").option("--mode <mode>", "Search mode: docs (default), hybrid, fts.", "docs").option("--alpha <float>", "Semantic weight 0..1 (default: 0.7).", "0.7").option("--min-score <float>", "Minimum cosine similarity threshold (default: CEREFOX_MIN_SEARCH_SCORE or 0.5).").option("--max-bytes <n>", "Response size budget in bytes (default: CEREFOX_MAX_RESPONSE_BYTES or 200000).").option("-r, --requestor <name>", "Agent / user name (recorded in usage log).").option("--json", "Emit machine-readable JSON instead of the default text.").option("--only-metadata", "List matching docs (id, score, chunks, chars, partial/full) WITHOUT their content — like the web UI's collapsed result list. Grab a [id:…] then `cerefox document get <id>`.").action(action29);
+  program2.command("search").description("Search the knowledge base (hybrid FTS + semantic).").argument("<query>", "Natural-language search query.").option("-c, --match-count <n>", "Maximum number of documents to return.", "5").option("-p, --project-name <name>", "Filter results to a specific project.").option("-f, --metadata-filter <json>", "JSON containment filter; only docs whose metadata contains ALL pairs are returned.").option("--mode <mode>", "Search mode: docs (default), hybrid, fts.", "docs").option("--alpha <float>", "Semantic weight 0..1 (default: 0.7).", "0.7").option("--min-score <float>", "Minimum cosine similarity threshold (default: CEREFOX_MIN_SEARCH_SCORE; else 0.5, or 0.6 with the local embedder).").option("--max-bytes <n>", "Response size budget in bytes (default: CEREFOX_MAX_RESPONSE_BYTES or 200000).").option("-r, --requestor <name>", "Agent / user name (recorded in usage log).").option("--json", "Emit machine-readable JSON instead of the default text.").option("--only-metadata", "List matching docs (id, score, chunks, chars, partial/full) WITHOUT their content — like the web UI's collapsed result list. Grab a [id:…] then `cerefox document get <id>`.").action(action29);
 }
 
 // src/cli/commands/self-update.ts

package/dist/server-assets/_shared/ef-meta/index.ts

@@ -18,7 +18,7 @@
  * doesn't touch `supabase/functions/` leaves it alone).
  */
 
-export const EF_VERSION = "1.0.0-rc.3";
+export const EF_VERSION = "1.0.0-rc.4";
 
 /**
  * The 8 peer EFs the cerefox-mcp aggregator probes (excludes cerefox-mcp

package/dist/server-assets/_shared/mcp-tools/_utils.ts

@@ -35,6 +35,15 @@ export function getMaxResponseBytes(): number {
 /** Built-in default cosine-similarity floor for hybrid/semantic search. */
 export const DEFAULT_MIN_SEARCH_SCORE = 0.5;
 
+/**
+ * Nomic's cosine-score distribution sits higher than OpenAI's: unrelated text
+ * lands ~0.4–0.55 (vs ~0.1–0.3), so the 0.5 floor calibrated for
+ * text-embedding-3-small lets weak matches through on the local embedder
+ * (rc.3 dogfood: an unrelated doc passed at vec≈0.54). 0.6 restores the
+ * intended precision; relevant nomic matches score ~0.7+.
+ */
+export const DEFAULT_MIN_SEARCH_SCORE_LOCAL = 0.6;
+
 /**
  * Resolve the minimum cosine-similarity floor for hybrid/semantic search
  * (vector-only matches below this are dropped; FTS matches always pass).
@@ -49,9 +58,14 @@ export const DEFAULT_MIN_SEARCH_SCORE = 0.5;
 export function getMinSearchScore(): number {
   const raw = (globalThis as { process?: { env?: Record<string, string | undefined> } })
     .process?.env?.CEREFOX_MIN_SEARCH_SCORE;
-  if (raw === undefined || raw === "") return DEFAULT_MIN_SEARCH_SCORE;
+  const fallback =
+    (globalThis as { process?: { env?: Record<string, string | undefined> } })
+      .process?.env?.CEREFOX_EMBEDDER === "local"
+      ? DEFAULT_MIN_SEARCH_SCORE_LOCAL
+      : DEFAULT_MIN_SEARCH_SCORE;
+  if (raw === undefined || raw === "") return fallback;
   const n = Number.parseFloat(raw);
-  return Number.isNaN(n) || n < 0 || n > 1 ? DEFAULT_MIN_SEARCH_SCORE : n;
+  return Number.isNaN(n) || n < 0 || n > 1 ? fallback : n;
 }
 
 export function applyByteBudget(

package/docs/guides/configuration.md

@@ -118,7 +118,10 @@ This handles intermittent OpenAI API errors (500s) that would otherwise cause se
 | Variable | Default | Description |
 |----------|---------|-------------|
 | `CEREFOX_MAX_RESPONSE_BYTES` | `200000` | Maximum bytes in a single search response (local MCP path). See explanation below. |
-| `CEREFOX_MIN_SEARCH_SCORE` | `0.50` | Minimum cosine similarity for hybrid and semantic search results (0.0–1.0). In **hybrid search**, chunks that matched the FTS keyword operator (`@@`) always pass through regardless of their vector score — the threshold only filters vector-only results. In **semantic search**, all results are filtered. The pure **FTS search** mode is unaffected. Increase for stricter precision; decrease for wider recall. |
+| `CEREFOX_MIN_SEARCH_SCORE` | `0.50` (`0.60` with the local embedder) | Minimum cosine similarity for hybrid and semantic search results (0.0–1.0). The default is embedder-aware: nomic scores unrelated text higher than OpenAI, so `CEREFOX_EMBEDDER=local` raises the floor to 0.60. In **hybrid search**, chunks that matched the FTS keyword operator (`@@`) always pass through regardless of their vector score — the threshold only filters vector-only results. In **semantic search**, all results are filtered. The pure **FTS search** mode is unaffected. Increase for stricter precision; decrease for wider recall. |
+| `CEREFOX_EMBED_MAX_INPUT_CHARS` | `20000` | Safety cap on the characters sent to the embedding model per input. The full chunk content is always stored and reconstructed untouched; only the embedding uses the (rare) truncated prefix, so an oversized chunk can never fail an ingest. |
+| `CEREFOX_MODELS_DIR` | `~/.cerefox/models` (in-container: inside the data volume) | Where the local embedder caches downloaded model weights (Cerefox Local; `CEREFOX_EMBEDDER=local`). |
+| `CEREFOX_ONNX_BATCH` | `4` | Texts per local-embedder inference call. Peak memory scales with this; the small default keeps ingest/reindex safe on small Docker VMs. |
 
 ### Metadata filter
 
@@ -257,9 +260,9 @@ OPENAI_API_KEY=sk-...
 # All other settings use defaults
 ```
 
-> **Fireworks is not wired in the TS runtime yet** — `CEREFOX_EMBEDDER=fireworks` /
-> `CEREFOX_FIREWORKS_*` are documented but currently no-ops (OpenAI is the only embedder
-> implemented today). Tracked for a future release.
+> **Fireworks is not wired yet** — `CEREFOX_EMBEDDER=fireworks` / `CEREFOX_FIREWORKS_*`
+> are documented but currently no-ops. The wired embedders are `openai` (default) and
+> `local` (Cerefox Local only). Fireworks is tracked for a future release.
 
 ---
 

package/docs/guides/migration-1.0.md

@@ -4,9 +4,6 @@ Cerefox 1.0.0 is the first stable release. Two changes need attention when you
 upgrade an existing Supabase deployment; a third is automatic. **Nothing here
 affects the local/self-hosted (World B) backend.**
 
-Pre-releases are published on the `1.0.0-beta.N` line (breaking changes may still
-occur between betas) leading up to `1.0.0`.
-
 ## 1. Edge Function auth: anon key → Cerefox access token (action required)
 
 The legacy Supabase **anon JWT** is no longer accepted for calling Cerefox Edge
@@ -52,7 +49,7 @@ Claude uses OAuth).
 Rotating the token later: `cerefox token rotate` (accepts new + old for zero-downtime),
 then `cerefox token rotate --finalize` once every client is on the new token.
 
-## 2. New schema (0.7.0 → 0.8.0): document reconstruction fix (redeploy required, no data action)
+## 2. New schema (→ 0.8.1): document reconstruction fix (redeploy required, no data action)
 
 1.0.0 fixes a document-reconstruction bug that could corrupt documents containing large
 tables or blank-line-free paragraphs. It adds a `content_format` column on
@@ -90,7 +87,7 @@ unaffected — they are not Python and remain the source of truth for the schema
 ```bash
 cerefox self-update            # or: npm install -g @cerefox/memory@latest
 cerefox token generate         # change #1: mint + set the access token
-cerefox server deploy          # changes #1 + #2: token-gated EFs + schema 0.8.0
+cerefox server deploy          # changes #1 + #2: token-gated EFs + schema 0.8.1
 cerefox doctor                 # verify (edge-functions green; content-format ℹ)
 # then: update GPT Actions / remote MCP clients to the token; revoke the anon key
 ```

package/docs/guides/operational-cost.md

@@ -142,8 +142,11 @@ embedding costs" below.
 
 If you want to keep costs as low as possible:
 
-- **Cheaper embedding models**: a lower-cost OpenAI-compatible provider (e.g. Fireworks AI)
-  is on the roadmap — **not yet wired in the TS runtime** (OpenAI only today).
+- **Zero-cost embeddings (Cerefox Local)**: the self-hosted backend can run a local
+  embedding model in-container (`CEREFOX_EMBEDDER=local`) — no API key, no per-token
+  cost at all. See [setup-local.md](setup-local.md#choose-your-embedder-openai-vs-fully-local).
+- **Cheaper cloud models**: a lower-cost OpenAI-compatible provider (e.g. Fireworks AI)
+  is on the roadmap — not yet wired.
 - **Batch ingest, don't re-ingest**: Cerefox deduplicates by content hash — re-ingesting the
   same file twice costs nothing. Only new or changed content triggers embedding calls.
 - **`cerefox server reindex`**: Re-embeds all existing chunks if you switch embedders. Run this once

package/docs/guides/setup-local.md

@@ -60,6 +60,10 @@ data volume, so it survives `cerefox-local upgrade`.
 > `cerefox-local server reindex` to re-embed everything. `cerefox-local doctor`
 > flags any mismatch.
 
+> Scores are calibrated per embedder: with the local model the default semantic
+> threshold is **0.6** (vs 0.5 for OpenAI) because nomic scores unrelated text
+> higher. Override per call with `--min-score` or via `CEREFOX_MIN_SEARCH_SCORE`.
+
 ---
 
 ## Step 1 — Install

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@cerefox/memory",
-  "version": "1.0.0-rc.3",
-  "description": "Cerefox — user-owned shared memory for AI agents. The local TypeScript runtime: stdio MCP server in v0.4; CLI binary added in v0.5; in-process web server in v0.6; ingestion pipeline in v0.7.",
+  "version": "1.0.0",
+  "description": "Cerefox — user-owned shared memory for AI agents. CLI + stdio MCP server + web UI + ingestion for a knowledge base on your own Supabase project (or fully self-hosted with Cerefox Local).",
   "license": "Apache-2.0",
   "homepage": "https://github.com/fstamatelopoulos/cerefox",
   "repository": {