@cerefox/memory 1.14.4 → 1.15.0-beta.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENT_GUIDE.md +49 -1
- package/AGENT_QUICK_REFERENCE.md +2 -2
- package/README.md +1 -1
- package/dist/bin/cerefox.js +4409 -4506
- package/dist/server-assets/_shared/ef-meta/index.ts +3 -3
- package/dist/server-assets/_shared/mcp-tools/get-help-content.ts +3 -3
- package/docs/guides/quickstart.md +1 -1
- package/package.json +5 -5
package/AGENT_GUIDE.md
CHANGED
|
@@ -481,7 +481,7 @@ changing.
|
|
|
481
481
|
3. **Always set `author`** to your agent name for attribution, on reads and writes alike. Every tool takes the same `author` parameter, no exceptions. (`requestor` is still accepted on every tool as the pre-1.13.1 alias, so older configurations keep working.)
|
|
482
482
|
4. **Use the `document_id` from search results** for `cerefox_get_document`, `cerefox_list_versions`, and targeted `cerefox_ingest` updates.
|
|
483
483
|
5. **Add metadata**: at minimum `type` (e.g., "research", "decision-log") and `status` ("active", "draft").
|
|
484
|
-
6. **
|
|
484
|
+
6. **Name the document for what it is about, then write structured Markdown** with H1/H2/H3 headings. The title is weighted more heavily than the body and reaches every chunk's embedding, so it is the single biggest lever on whether this document is found later. See "How titles affect search" below. The chunker uses heading structure.
|
|
485
485
|
7. **Distill, don't dump.** Summaries > transcripts. Decisions > discussions. Insights > raw data.
|
|
486
486
|
8. **Prove freshness on updates.** Pass `expected_content_hash` (the hash you read) on every content update. On conflict: re-read → merge → retry. Never `last_write_wins` your way out of a conflict.
|
|
487
487
|
|
|
@@ -500,6 +500,54 @@ Call `cerefox_list_metadata_keys` for the current list -- conventions evolve.
|
|
|
500
500
|
|
|
501
501
|
---
|
|
502
502
|
|
|
503
|
+
## How titles affect search
|
|
504
|
+
|
|
505
|
+
A Cerefox document's title is not just a label. It is indexed, and it is
|
|
506
|
+
indexed **twice over**, which makes it the strongest single influence on
|
|
507
|
+
whether a document is found later.
|
|
508
|
+
|
|
509
|
+
**Keyword search.** When a document is written, each chunk's full-text vector
|
|
510
|
+
is built from three parts at two weights: the **document title at weight A**,
|
|
511
|
+
the **chunk's own heading at weight A**, and the **chunk body at weight B**.
|
|
512
|
+
Postgres ranks weight-A matches above weight-B ones, so a query term that
|
|
513
|
+
appears in the title lifts the whole document above one where the same term
|
|
514
|
+
appears only in the prose.
|
|
515
|
+
|
|
516
|
+
**Semantic search.** Every chunk is embedded as `# {document title}` followed
|
|
517
|
+
by its heading breadcrumb and its text. The stored content is untouched, but
|
|
518
|
+
the vector each chunk is retrieved by is coloured by the title. A document
|
|
519
|
+
called "Notes" contributes nothing to any of its chunks' vectors; a document
|
|
520
|
+
called "Postgres connection pooling limits on the free tier" contributes to
|
|
521
|
+
all of them.
|
|
522
|
+
|
|
523
|
+
### What to do about it
|
|
524
|
+
|
|
525
|
+
- **Title with the words a future searcher would use.** Distinctive nouns over
|
|
526
|
+
generic labels. "Cerefox Response Size Limits" is findable; "Notes",
|
|
527
|
+
"Update 3", "Meeting" and "Misc" are not, and they dilute every chunk they
|
|
528
|
+
are attached to.
|
|
529
|
+
- **Put the subject in the title, not only in the body.** If the document is
|
|
530
|
+
about one system, one decision or one incident, name it. Search cannot boost
|
|
531
|
+
a term that is not there.
|
|
532
|
+
- **Do not stuff.** The title is also what a human reads in a result list and
|
|
533
|
+
in the web UI. A keyword-crammed title is worse than a clear one: relevance
|
|
534
|
+
is not the only thing a title has to do.
|
|
535
|
+
- **Fixing a bad title is cheap.** Renaming a document re-computes the
|
|
536
|
+
full-text vectors and re-embeds its current chunks, so the improvement
|
|
537
|
+
applies to everything already stored. If you meet a document whose title
|
|
538
|
+
does not describe it, say so, or fix it.
|
|
539
|
+
- **Titles cannot be changed by a partial edit.** `rename_section` changes a
|
|
540
|
+
heading inside the content; the stored title is a separate field and needs
|
|
541
|
+
`cerefox_ingest` (with `document_id` and `expected_content_hash`).
|
|
542
|
+
|
|
543
|
+
### Why this matters more for you than for a human
|
|
544
|
+
|
|
545
|
+
A person browsing a list can open three plausible documents and skim. An agent
|
|
546
|
+
usually takes the top result, and is working against a byte budget that may
|
|
547
|
+
admit only one. A precise title is what puts the right document in that slot.
|
|
548
|
+
|
|
549
|
+
---
|
|
550
|
+
|
|
503
551
|
## Writing linkable content
|
|
504
552
|
|
|
505
553
|
Documents you ingest may contain markdown links to other Cerefox documents. The Cerefox web UI intercepts these links at click time and resolves them to the target document. The resolution happens entirely in the browser; the stored markdown is untouched. (User-facing overview of the whole linking system, including *why* long ids corrupt during regeneration: [`docs/guides/linking.md`](docs/guides/linking.md).)
|
package/AGENT_QUICK_REFERENCE.md
CHANGED
|
@@ -7,7 +7,7 @@ Cerefox is a persistent, shared knowledge base. You have **15 core MCP tools** (
|
|
|
7
7
|
| Tool | Purpose | Key params |
|
|
8
8
|
|------|---------|------------|
|
|
9
9
|
| `cerefox_search` | Find documents (hybrid FTS + semantic) | `query` (required), `project_name`, `metadata_filter`, `author` |
|
|
10
|
-
| `cerefox_ingest` | Save or update a document | `title`, `content` (required), `document_id` (update by ID), `expected_content_hash` (**required on content updates** — see rule
|
|
10
|
+
| `cerefox_ingest` | Save or update a document | `title`, `content` (required), `document_id` (update by ID), `expected_content_hash` (**required on content updates** — see rule 10), `last_write_wins`, `update_if_exists`, `project_name` (single, non-destructive add on update), `project_names` (list, destructive replace on update), `metadata` (omit on update to keep existing tags; `{}` clears), `author` |
|
|
11
11
|
| `cerefox_insert` | **Add** to a document without resending it. Cannot destroy content. | `document_id`, `text`, `position` (`end_of_document`/`end_of_section`/`after_heading`/`before_heading`), `expected_content_hash` (required), `anchor_heading` (unless `end_of_document`), `section_part`, `author` |
|
|
12
12
|
| `cerefox_edit` | **Change** parts of a document: 1..n operations applied atomically | `document_id`, `operations` (`insert`/`replace_section`/`delete_section`/`rename_section`), `expected_content_hash` (required), `author` |
|
|
13
13
|
| `cerefox_delete_document` | **Soft**-delete a document (to trash; excluded from search; permanent purge is human-only) | `document_id`, `expected_content_hash` (**required** — a delete must follow a read), `reason` (recorded in the audit log — give one), `author` |
|
|
@@ -78,7 +78,7 @@ recoverable answer, not a failure: retry with what it gave you.
|
|
|
78
78
|
4. **Use `document_id` from search results** `[id: uuid]` for get_document and list_versions.
|
|
79
79
|
5. **`max_bytes` has to fit a whole document.** Search returns COMPLETE documents, so a budget smaller than the top hit fits nothing. You get a header-only list saying so (never "No results found." — that means the store really has nothing). Raise `max_bytes`, or read one document with `cerefox_get_document` using `outline: true` or `section`.
|
|
80
80
|
6. **Add metadata** -- at minimum `type` ("decision-log", "research", "design-doc") and `status` ("active", "draft").
|
|
81
|
-
7. **Write structured Markdown
|
|
81
|
+
7. **Write for retrieval: the title does real work.** The document title is indexed at the **highest weight** *and* prepended to every chunk before it is embedded, so it steers both halves of hybrid search (keyword and semantic) for every chunk in the document. Name the document with the distinctive terms someone would actually search for -- "Cerefox Response Size Limits", not "Notes" or "Update 3". Renaming re-indexes and re-embeds, so correcting a vague title is cheap and takes effect immediately. Inside the content, write structured Markdown with H1/H2/H3 headings: the chunker follows them, and a chunk's own heading carries the same weight as the title.
|
|
82
82
|
8. **Deletes are soft (recoverable); permanent purge is web-UI-only.** `cerefox_delete_document` requires the document's `content_hash` as you read it (read before you delete) and takes a `reason` — give one; it is what the human reviewing the trash sees. `cerefox_restore_document` undoes a mistaken delete (also audited, also takes a `reason`). Always surface deletes AND restores to the user. Once a human purges from the web UI, the document is gone for good.
|
|
83
83
|
9. **Cross-doc links inside content**: **always use `[Text](document-uuid)`.** UUIDs are the only fully reliable link form — stable across title changes, never ambiguous, no encoding gotchas. Every `cerefox_search` result shows `[id: <uuid>]` after the title; grab it and use it. Title-based linking (`[Text](<Title With Spaces>)`) is fragile (breaks on colons, parens, ampersands, brackets — silently navigates to wrong page) — **don't write title-based links**; do an extra search to get the UUID instead. Repo-path forms (`[Text](docs/path.md)`) exist for repo-ingested files; don't construct manually. **The server validates `](uuid)` links on every write** (v1.7.0): a link to a nonexistent id rejects the write, naming the offender — that means you mangled the UUID; re-read the source and correct it, do not retry unchanged. Example ids go in backticks (code is not validated). `[[Wikilinks]]` may dangle. See `AGENT_GUIDE.md → Writing linkable content` for the full rule.
|
|
84
84
|
10. **Concurrency: content updates require `expected_content_hash`.** Pass the `content_hash` you last saw — every read shows one (`cerefox_get_document` incl. outline mode, `cerefox_search`, `cerefox_metadata_search`) and **every write returns the new one, including create** (v1.3.0, #189), so after writing you already hold the token for your next edit; no re-read needed. If it's stale you get a **conflict** — re-read the document, merge your changes into the latest content, retry with the new hash. **Never resolve a conflict by overwriting blindly** — the current content includes another writer's work. `last_write_wins: true` skips the check; use it ONLY when an external source of truth makes conflicts meaningless (file re-sync), never to silence a conflict.
|
package/README.md
CHANGED
|
@@ -57,7 +57,7 @@ You deploy the server side with this package's CLI — `cerefox server deploy` s
|
|
|
57
57
|
|---|---|---|
|
|
58
58
|
| A **Supabase project** | Hosts Postgres + pgvector + Edge Functions. Free tier is enough for most personal use. | [supabase.com](https://supabase.com) → New project |
|
|
59
59
|
| An **embedding API key** | OpenAI `text-embedding-3-small` (the cloud backend's embedder; Cerefox Local can instead run a fully-offline local model). Pennies/month for typical personal use (see [operational-cost.md](https://github.com/fstamatelopoulos/cerefox/blob/main/docs/guides/operational-cost.md)). | Get an [OpenAI API key](https://platform.openai.com/api-keys). |
|
|
60
|
-
| **Node ≥
|
|
60
|
+
| **Node ≥ 24** or **Bun ≥ 1.0** | Runtime for the `cerefox` bin (and the bundled `cerefox mcp` server). | [nodejs.org](https://nodejs.org) · [bun.sh](https://bun.sh). The one-line installer below bootstraps Bun if neither is present. |
|
|
61
61
|
|
|
62
62
|
### One-time server-side setup (~10 min — no clone needed)
|
|
63
63
|
|