datanotes 3.3.0 → 3.4.0

package/dist/docs/guides/read-and-write/import-notes.md

@@ -314,7 +314,7 @@ Answer:
   "op": "create_table",
   "planned_texts": ["datanotes/metastore.yaml"],
   "commits": [{ "table": "main.public.film", "version": 0, "operation": "CREATE TABLE" }],
-  "migration_id": "20261020T173838-create_table-create-table-main-public-film"
+  "migration_id": "20261024T063914-create_table-create-table-main-public-film"
 }
 ```
 
@@ -359,7 +359,7 @@ Answer (70 lines)
       "path": "Films/Charade.md",
       "set": {
         "table": "main.public.film",
-        "id": "01a15a64-ce9f-7560-9add-1459237106c3",
+        "id": "01a16ca2-8b15-7b17-9621-47ebd428dcb2",
         "director": "Stanley Donen",
         "title": "Charade"
       },
@@ -369,7 +369,7 @@ Answer (70 lines)
       "path": "Films/His Girl Friday.md",
       "set": {
         "table": "main.public.film",
-        "id": "01a15a64-ce9f-7172-abbb-020e612ede56",
+        "id": "01a16ca2-8b15-736e-b2e9-b44812311845",
         "director": "Howard Hawks",
         "title": "His Girl Friday"
       },
@@ -379,7 +379,7 @@ Answer (70 lines)
       "path": "Films/Metropolis.md",
       "set": {
         "table": "main.public.film",
-        "id": "01a15a64-ce9f-7b50-990c-aa26f69b3ab6",
+        "id": "01a16ca2-8b16-7de3-b3c7-f99fea697aef",
         "director": "Fritz Lang",
         "title": "Metropolis"
       },
@@ -389,7 +389,7 @@ Answer (70 lines)
       "path": "Films/Nosferatu.md",
       "set": {
         "table": "main.public.film",
-        "id": "01a15a64-ce9f-7541-8429-fc436df30866",
+        "id": "01a16ca2-8b16-7f5e-a304-e715a1815b66",
         "director": "F. W. Murnau",
         "title": "Nosferatu"
       },
@@ -399,7 +399,7 @@ Answer (70 lines)
       "path": "Films/The Gold Rush.md",
       "set": {
         "table": "main.public.film",
-        "id": "01a15a64-ce9f-7cf2-8129-a75fb2745cc9",
+        "id": "01a16ca2-8b16-747f-9e49-aefd3f27fd58",
         "director": "Charlie Chaplin",
         "title": "The Gold Rush"
       },
@@ -458,7 +458,7 @@ Answer (73 lines)
       "path": "Films/Charade.md",
       "set": {
         "table": "main.public.film",
-        "id": "01a15a64-cea3-7bf1-ada5-aeb2d9dbb4fd",
+        "id": "01a16ca2-8b18-7e55-9cbc-a983a43ae96c",
         "director": "Stanley Donen",
         "title": "Charade"
       },
@@ -468,7 +468,7 @@ Answer (73 lines)
       "path": "Films/His Girl Friday.md",
       "set": {
         "table": "main.public.film",
-        "id": "01a15a64-cea3-777e-b51d-cfb871067faf",
+        "id": "01a16ca2-8b18-739c-9058-e32e9f3a61d4",
         "director": "Howard Hawks",
         "title": "His Girl Friday"
       },
@@ -478,7 +478,7 @@ Answer (73 lines)
       "path": "Films/Metropolis.md",
       "set": {
         "table": "main.public.film",
-        "id": "01a15a64-cea3-776e-b81b-389706767aa1",
+        "id": "01a16ca2-8b18-71f0-92e4-4e2492953fa5",
         "director": "Fritz Lang",
         "title": "Metropolis"
       },
@@ -488,7 +488,7 @@ Answer (73 lines)
       "path": "Films/Nosferatu.md",
       "set": {
         "table": "main.public.film",
-        "id": "01a15a64-cea3-7299-a73b-bb2b9eec0177",
+        "id": "01a16ca2-8b18-78be-9435-87eca4fa962e",
         "director": "F. W. Murnau",
         "title": "Nosferatu"
       },
@@ -498,7 +498,7 @@ Answer (73 lines)
       "path": "Films/The Gold Rush.md",
       "set": {
         "table": "main.public.film",
-        "id": "01a15a64-cea4-7624-b9be-0e6c6aeb6dfc",
+        "id": "01a16ca2-8b18-76bd-b55f-ba6aadec67b1",
         "director": "Charlie Chaplin",
         "title": "The Gold Rush"
       },
@@ -519,7 +519,7 @@ Answer (73 lines)
     }
   ],
   "commits": [{ "table": "main.public.film", "version": 1, "operation": "ADOPT ROWS" }],
-  "migration_id": "20261020T173838-adopt_rows-adopt-5-notes-into-film"
+  "migration_id": "20261024T063914-adopt_rows-adopt-5-notes-into-film"
 }
 ```
 
@@ -536,7 +536,7 @@ Answer:
 ```markdown
 ---
 table: main.public.film
-id: 01a15a64-cea3-7bf1-ada5-aeb2d9dbb4fd
+id: 01a16ca2-8b18-7e55-9cbc-a983a43ae96c
 year: 1963
 minutes: 113
 status: watched
@@ -623,7 +623,7 @@ Answer:
   "files_created": 1,
   "planned_creates": ["datanotes/main/public/film/Battleship Potemkin.md"],
   "commits": [{ "table": "main.public.film", "version": 2, "operation": "WRITE" }],
-  "migration_id": "20261020T173838-insert_row-insert-into-film"
+  "migration_id": "20261024T063914-insert_row-insert-into-film"
 }
 ```
 
@@ -655,7 +655,7 @@ Answer:
   "op": "add_column",
   "planned_texts": ["datanotes/metastore.yaml"],
   "commits": [{ "table": "main.public.film", "version": 4, "operation": "ADD COLUMNS" }],
-  "migration_id": "20261020T173839-add_column-add-column-added_by-to-film"
+  "migration_id": "20261024T063915-add_column-add-column-added_by-to-film"
 }
 ```
 
@@ -683,7 +683,7 @@ Answer:
   "files_created": 1,
   "planned_creates": ["datanotes/main/public/film/Night of the Living Dead.md"],
   "commits": [{ "table": "main.public.film", "version": 5, "operation": "WRITE" }],
-  "migration_id": "20261020T173839-insert_row-insert-into-film"
+  "migration_id": "20261024T063915-insert_row-insert-into-film"
 }
 ```
 
@@ -692,7 +692,7 @@ Answer:
 ```markdown
 ---
 table: main.public.film
-id: 01a15a64-d4a9-7905-af13-90c8506569c9
+id: 01a16ca2-915b-736c-a394-c98699327151
 director: George A. Romero
 minutes: 96
 status: to watch
@@ -734,7 +734,7 @@ Answer:
 ```markdown
 ---
 table: main.public.film
-id: 01a15a64-d731-7eff-804e-4148716ec5c3
+id: 01a16ca2-943a-7248-9a53-5c7f7532b5af
 year: 1924
 minutes: 45
 status: to watch
@@ -762,7 +762,7 @@ Answer:
   "op": "alter_table",
   "planned_texts": ["datanotes/metastore.yaml"],
   "commits": [{ "table": "main.public.film", "version": 7, "operation": "ALTER TABLE" }],
-  "migration_id": "20261020T173842-alter_table-alter-table-main-public-film"
+  "migration_id": "20261024T063918-alter_table-alter-table-main-public-film"
 }
 ```
 
@@ -825,7 +825,7 @@ This is the table as you inferred, corrected and extended it on this page. Its d
 
 ```yaml
 # Metastore (Datanotes): catalogs, schemas, tables, columns, views.
-id: 01a15a64-ce85-731f-bfea-eb573328eb39
+id: 01a16ca2-8b02-76f7-8686-40530eaf4e87
 config:
   history: true
 

package/dist/docs/guides/read-and-write/joins-and-aggregates.md

@@ -100,9 +100,9 @@ Answer:
   "table": "main.public.watch",
   "total": 3,
   "rows": [
-    { "date(occurred_at)": "2026-10-15", "viewings": 1 },
-    { "date(occurred_at)": "2026-10-18", "viewings": 1 },
-    { "date(occurred_at)": "2026-10-19", "viewings": 1 }
+    { "date(occurred_at)": "2026-10-19", "viewings": 1 },
+    { "date(occurred_at)": "2026-10-22", "viewings": 1 },
+    { "date(occurred_at)": "2026-10-23", "viewings": 1 }
   ]
 }
 ```
@@ -138,10 +138,10 @@ Answer:
     {
       "path": "datanotes/main/public/film/Night of the Living Dead.md",
       "title": "Night of the Living Dead",
-      "watched_on": "2026-10-19",
+      "watched_on": "2026-10-23",
       "viewing": [
         {
-          "path": "datanotes/main/public/watch/2026-10-19T2200.md",
+          "path": "datanotes/main/public/watch/2026-10-23T2200.md",
           "where": "cinema",
           "with": "Oscar"
         }
@@ -150,10 +150,10 @@ Answer:
     {
       "path": "datanotes/main/public/film/The Gold Rush.md",
       "title": "The Gold Rush",
-      "watched_on": "2026-10-18",
+      "watched_on": "2026-10-22",
       "viewing": [
         {
-          "path": "datanotes/main/public/watch/2026-10-18T2030.md",
+          "path": "datanotes/main/public/watch/2026-10-22T2030.md",
           "where": "home",
           "with": "my sister"
         }

package/dist/docs/guides/read-and-write/read-and-filter.md

@@ -332,23 +332,23 @@ Answer:
   "total": 3,
   "rows": [
     {
-      "path": "datanotes/main/public/watch/2026-10-15T2115.md",
+      "path": "datanotes/main/public/watch/2026-10-19T2115.md",
       "film": "[[Charade]]",
-      "occurred_at": "2026-10-15T21:15",
+      "occurred_at": "2026-10-19T21:15",
       "where": "home",
       "place": "my living room"
     },
     {
-      "path": "datanotes/main/public/watch/2026-10-18T2030.md",
+      "path": "datanotes/main/public/watch/2026-10-22T2030.md",
       "film": "[[The Gold Rush]]",
-      "occurred_at": "2026-10-18T20:30",
+      "occurred_at": "2026-10-22T20:30",
       "where": "home",
       "place": "my living room"
     },
     {
-      "path": "datanotes/main/public/watch/2026-10-19T2200.md",
+      "path": "datanotes/main/public/watch/2026-10-23T2200.md",
       "film": "[[Night of the Living Dead]]",
-      "occurred_at": "2026-10-19T22:00",
+      "occurred_at": "2026-10-23T22:00",
       "where": "cinema",
       "place": "the film club downtown"
     }
@@ -459,12 +459,12 @@ Answer:
   "total": 7,
   "rows": [
     {
-      "path": "datanotes/main/public/watch/2026-10-15T2115.md",
-      "occurred_at": "2026-10-15T21:15"
+      "path": "datanotes/main/public/watch/2026-10-19T2115.md",
+      "occurred_at": "2026-10-19T21:15"
     },
     {
-      "path": "datanotes/main/public/watch/2026-10-11T2130.md",
-      "occurred_at": "2026-10-11T21:30"
+      "path": "datanotes/main/public/watch/2026-10-15T2130.md",
+      "occurred_at": "2026-10-15T21:30"
     }
   ]
 }

package/dist/docs/guides/read-and-write/write-rows.md

@@ -31,7 +31,7 @@ Answer:
   "files_created": 1,
   "planned_creates": ["datanotes/main/public/film/Metropolis.md"],
   "commits": [{ "table": "main.public.film", "version": 1, "operation": "WRITE" }],
-  "migration_id": "20261020T173828-insert_row-insert-into-film"
+  "migration_id": "20261024T063903-insert_row-insert-into-film"
 }
 ```
 
@@ -40,7 +40,7 @@ Answer:
 ```markdown
 ---
 table: main.public.film
-id: 01a15a64-a8b5-7df3-aef8-9d133af2ff28
+id: 01a16ca2-615c-78e9-af82-5d7ee065ac91
 title: Metropolis
 year: 1927
 minutes: 153
@@ -103,7 +103,7 @@ Answer:
     }
   ],
   "commits": [{ "table": "main.public.film", "version": 6, "operation": "UPDATE" }],
-  "migration_id": "20261020T173832-update_row-update-film"
+  "migration_id": "20261024T063907-update_row-update-film"
 }
 ```
 
@@ -174,7 +174,7 @@ Answer:
     }
   ],
   "commits": [{ "table": "main.public.film", "version": 8, "operation": "UPDATE" }],
-  "migration_id": "20261020T173833-update_row-update-film-2",
+  "migration_id": "20261024T063907-update_row-update-film-3",
   "ambiguous_mentions": [
     {
       "path": "datanotes/main/public/film/Nosferatu.md",
@@ -193,7 +193,7 @@ Answer:
 ```markdown
 ---
 table: main.public.film
-id: 01a15a64-b97e-7825-b690-1c0bdc44583e
+id: 01a16ca2-718e-7219-9638-3d26346a9a88
 title: Nosferatu
 year: 1922
 status: watched
@@ -246,7 +246,7 @@ Answer:
     }
   ],
   "commits": [{ "table": "main.public.film", "version": 9, "operation": "UPDATE" }],
-  "migration_id": "20261020T173833-update_row-update-film-3"
+  "migration_id": "20261024T063907-update_row-update-film-4"
 }
 ```
 
@@ -332,7 +332,7 @@ Answer:
     }
   ],
   "commits": [{ "table": "main.public.film", "version": 10, "operation": "UPDATE" }],
-  "migration_id": "20261020T173833-update_rows-update-film"
+  "migration_id": "20261024T063907-update_rows-update-film"
 }
 ```
 
@@ -370,7 +370,7 @@ Answer:
   ],
   "planned_creates": ["datanotes/main/public/film/Battleship Potemkin.md"],
   "commits": [{ "table": "main.public.film", "version": 11, "operation": "MERGE" }],
-  "migration_id": "20261020T173833-merge_rows-merge-2-rows-into-film"
+  "migration_id": "20261024T063907-merge_rows-merge-2-rows-into-film"
 }
 ```
 
@@ -431,7 +431,7 @@ Answer:
     }
   ],
   "commits": [{ "table": "main.public.film", "version": 12, "operation": "DELETE" }],
-  "migration_id": "20261020T173833-delete_rows-archive-1-rows-of-film"
+  "migration_id": "20261024T063907-delete_rows-archive-1-rows-of-film"
 }
 ```
 
@@ -568,6 +568,6 @@ Answer (51 lines)
     { "table": "main.public.film", "version": 22, "operation": "MOVE ROWS" },
     { "table": "main.public.watchlist", "version": 1, "operation": "MOVE ROWS" }
   ],
-  "migration_id": "20261020T173833-move_rows-move-4-rows-from-film-to-watch"
+  "migration_id": "20261024T063907-move_rows-move-4-rows-from-film-to-watch"
 }
 ```

package/dist/docs/guides/search/hybrid.md

@@ -0,0 +1,162 @@
+# Search notes by words and meaning
+
+> Search every note by its words, and by its meaning too when semantic search is on, in one call.
+
+The [`search`](https://datanotes.dev/reference/operations/queries/#search) operation always works. It ranks the notes by their words, and by their meaning too when [semantic search](https://datanotes.dev/guides/search/semantic/) is on. A note found only one way is still in the answer. The vault has three actors, two classic films, a viewing and two plain notes.
+
+## Choose the search to use
+
+The engine has three ways to search the notes:
+
+| Operation         | Finds                                                                                           | Use it for                                  |
+| ----------------- | ----------------------------------------------------------------------------------------------- | ------------------------------------------- |
+| `search`          | Notes with any word of the query, ranked; with semantic search on, notes that mean the same too | Most searches, by agents and programs       |
+| `search_notes`    | Notes with every word of the query, as text                                                     | Exact passages, such as a code or a name    |
+| `semantic_search` | Notes by meaning only                                                                           | Vague questions, when semantic search is on |
+
+## How the words rank the notes
+
+`search` scores each note with BM25, the ranking of most search engines:
+
+* **Rare words count more.** A word that few notes have says more than a word that every note has.
+* **The name and the properties count more.** A word in the note’s name, or in a value of its front matter, adds to its score. Property names, `id` and `table` don’t count.
+* **Every word counts.** A note with every word of the query ranks above a note with some of them.
+* **Words in a row count more.** Two words next to each other, as in the query, add a little.
+
+The engine ignores case and accents, so `Uovo` finds `uovo` and `perche` finds `perché`. It doesn’t reduce words to a stem, because notes are in many languages: `uova` doesn’t find `uovo`. Semantic search finds those other forms. A single letter is never searched, but a single digit is, so `6 mesi` works.
+
+## Write a query
+
+A query takes the syntax of `search_docs`:
+
+* `"6 mesi"`: the words must appear in a row.
+* `+uovo`: the word must appear.
+* `-latte`: no note with the word is returned.
+* Any other word only ranks the notes.
+
+`tables` keeps the rows of some tables, by full or short name. `folder` keeps the notes of one folder. `limit` sets how many notes come back: 10 by default, at most 100.
+
+## Fuse the words with the meaning
+
+When semantic search is on, `search` also ranks the notes by meaning. It then fuses the two rankings with Reciprocal Rank Fusion. A note scores more for each ranking it is in, and more near the top of each.
+
+Each hit says how the engine found it in `matched_by`: `words`, `meaning` or both. The answer’s `meaning` field says what ran:
+
+* `on`: both rankings, fused.
+* `off`: semantic search is off, so only the words ranked.
+* `failed`: the embedding provider failed, so only the words ranked. `meaning_error` says why.
+
+The required and excluded words filter both rankings. `min_score` drops the passages of the meaning ranking below that similarity, from 0 to 1.
+
+## What a token may search
+
+A named token with [`read` patterns](https://datanotes.dev/guides/security/tokens/#limit-what-a-token-can-read) searches only the notes it may read. A table it may not read, named exactly in `tables`, answers 403.
+
+## Search by words and meaning
+
+`search` ranks every note that has a word of the query. A rare word counts more than a common one. A word in the note’s name or properties counts more than one in its text. Here semantic search is off, so `meaning` is `off` and every hit has `matched_by: ["words"]`.
+
+**`search`**
+
+```json
+{ "query": "cary grant film" }
+```
+
+Answer (52 lines)
+
+```json
+{
+  "query": "cary grant film",
+  "results": [
+    {
+      "path": "Notes/Film night.md",
+      "score": 4.57,
+      "heading": "",
+      "snippet": "…ry other Friday. Next up: [[Charade]]. October’s theme is [[Cary Grant]].",
+      "matched_by": ["words"]
+    },
+    {
+      "path": "Notes/Rewatch ideas.md",
+      "score": 1.95,
+      "heading": "",
+      "snippet": "A Cary Grant weekend: every film with him, in release order.",
+      "matched_by": ["words"]
+    },
+    {
+      "path": "datanotes/main/public/actor/Cary Grant.md",
+      "table": "main.public.actor",
+      "score": 1.92,
+      "heading": "",
+      "snippet": "Cary Grant",
+      "matched_by": ["words"]
+    },
+    {
+      "path": "datanotes/main/public/film/Charade.md",
+      "table": "main.public.film",
+      "score": 0.9,
+      "heading": "",
+      "snippet": "[[Cary Grant]] again, 23 years after [[His Girl Friday]]. Watch them back to back.",
+      "matched_by": ["words"]
+    },
+    {
+      "path": "datanotes/main/public/film/His Girl Friday.md",
+      "table": "main.public.film",
+      "score": 0.6,
+      "heading": "",
+      "snippet": "\"Cary Grant\"",
+      "matched_by": ["words"]
+    },
+    {
+      "path": "datanotes/main/public/watch/2026-09-25T2100.md",
+      "table": "main.public.watch",
+      "score": 0.07,
+      "heading": "",
+      "snippet": "Part of [[Film night]]. Four of us on the sofa, and nobody guessed the ending.",
+      "matched_by": ["words"]
+    }
+  ],
+  "meaning": "off"
+}
+```
+
+Quotes keep words together, and a minus sign leaves notes out. This query wants the two words in a row, and no note that names Charade:
+
+**`search`**
+
+```json
+{ "query": "\"cary grant\" -charade" }
+```
+
+Answer:
+
+```json
+{
+  "query": "\"cary grant\" -charade",
+  "results": [
+    {
+      "path": "datanotes/main/public/actor/Cary Grant.md",
+      "table": "main.public.actor",
+      "score": 4.31,
+      "heading": "",
+      "snippet": "Cary Grant",
+      "matched_by": ["words"]
+    },
+    {
+      "path": "datanotes/main/public/film/His Girl Friday.md",
+      "table": "main.public.film",
+      "score": 1.35,
+      "heading": "",
+      "snippet": "\"Cary Grant\"",
+      "matched_by": ["words"]
+    },
+    {
+      "path": "Notes/Rewatch ideas.md",
+      "score": 1.18,
+      "heading": "",
+      "snippet": "A Cary Grant weekend: every film with him, in release order.",
+      "matched_by": ["words"]
+    }
+  ],
+  "meaning": "off"
+}
+```

package/dist/docs/guides/search/text.md

@@ -87,7 +87,7 @@ Answer (62 lines)
         {
           "start": 81,
           "end": 86,
-          "context": "-ad6c-b8dee1b7285e\nname: Cary Grant\n---\n\n"
+          "context": "-9707-5a03c150e424\nname: Cary Grant\n---\n\n"
         }
       ]
     },

package/dist/docs/guides/security/security-model.md

@@ -60,13 +60,15 @@ The owner token is `httpServerToken` in the settings file. It can do everything:
 
 The owner creates a named token with `POST /datanotes/tokens/create/`, and the engine stores only its hash, in `tokens.json` in its store, outside the vault. A named token has these permissions:
 
-* **Read**: everything
+* **Read**: only the tables in its `read` patterns; without `read`, every table
 * **Write rows**: only the tables in its `write` patterns
 * **Change the schema**: only with `ddl`
 * **Maintain**: with `maintain`, it may also undo, restore, vacuum, configure the database and rebuild the semantic index; use it for an agent’s own token
 
 A named token can’t create webhooks, manage tokens or read the engine’s status, and without `maintain` it can’t undo.
 
+With `read` patterns, a token reads only the tables they match. It never reads the notes outside tables. Listings, search, the graph and the change feed leave out the rest. A query that names another table gets 403. A token always reads what it writes, and a token with `ddl` or `maintain` reads every table. Listings hide the other tables’ names. A refusal repeats a name the caller gave, so a guessed name isn’t secret. [Limit what a token can read](https://datanotes.dev/guides/security/tokens/#limit-what-a-token-can-read) lists the rules and the known gaps.
+
 ### Extensions
 
 An extension adds tables and Model Context Protocol (MCP) tools from a git repository. Its code is trusted code that runs inside the MCP server’s process, not in a sandbox. Install only extensions you trust, and read the plan first.
@@ -77,7 +79,7 @@ An extension adds tables and Model Context Protocol (MCP) tools from a git repos
 * **An invoker check**: before a tool that writes runs, the server checks the calling agent’s token. That token must be able to write the tool’s tables. A read-only client cannot write through an extension.
 * **The secret stays on one machine**: the token file sits under the data home with mode 0600. The registry in the engine’s store holds no secret.
 
-Known limits: extension code is not isolated from the MCP server’s process. Its token reads every table until read patterns ship. Migrations are declarative and run no code. [Install extensions](https://datanotes.dev/administration/extensions/) has the details.
+Known limits: extension code is not isolated from the MCP server’s process. The installer gives its token no `read`, so it reads every table. The owner may limit it with `read` patterns that cover the tables it writes. Migrations are declarative and run no code. [Install extensions](https://datanotes.dev/administration/extensions/) has the details.
 
 ### Who made each write
 
@@ -105,7 +107,7 @@ An agent reads your notes, and a note can hold text written to manipulate it: a
 
 Limit what each agent can do:
 
-* **Give it a named token**: grant only the tables it writes, and leave out `ddl` and `maintain` unless it needs them
+* **Give it a named token**: grant only the tables it reads and writes, and leave out `ddl` and `maintain` unless it needs them
 * **Give it only the tools it needs**: start the MCP server with `datanotes mcp --profile`, `--tools` or `--except`
 * **Keep the webhook tools off**: they’re off by default, because a webhook sends change events to any URL