datanotes 2.0.1 → 3.0.0

package/dist/docs/api.md

@@ -0,0 +1,135 @@
+# Datanotes HTTP API
+
+> OpenAPI 3.1.0 description of the Datanotes 3.0.0 HTTP API: https://datanotes.dev/openapi.json
+
+Database of Markdown notes: tables of notes, rows, queries, history, a change feed and the vault's notes.
+
+**Operations**: every operation is `POST /datanotes/operations/<name>/`, its body the operation's arguments (the same names as the MCP tool of the same name). An unknown argument is refused (400) with the closest real one.
+
+**Authentication**: every route but `GET /`, this document and `/datanotes/docs` needs `Authorization: Bearer <token>`: the owner's token (`httpServerToken` in the settings: everything) or a named token (`/datanotes/tokens/create/`: reads everything; writes the tables in its `write` patterns, changes the schema with `ddl`; the engine's administration stays the owner's: 403 otherwise).
+
+**Actor**: send `X-Datanotes-Actor: {"harness": …, "model": …, "session": …}` (JSON) to record on whose behalf a write is made; the journal adds the credential (`auth`: `owner`, or the named token's name), which the client cannot claim.
+
+**Errors**: `{error}` with 400 (bad input), 401 (missing or wrong token), 403 (a named token beyond its permissions), 404 (unknown table, view, migration or note), 409 (ambiguous table name), 500. A refused write answers 422 (409 for undo) with the operation's result, `ok: false`: nothing was written.
+
+## Servers
+
+- `/`
+
+## Authentication
+
+- `bearer`: http bearer. The owner's token or a named token.
+
+## Operations
+
+### Schema (DDL)
+
+Operations that define the database: tables, columns, constraints, bases and views.
+
+- [Create a table: `name`, in `catalog` and `schema` (default: the database's defaults); the definition](https://datanotes.dev/api/operations/create_table/): `POST /datanotes/operations/create_table/`
+- [Drop a table. Refused while it has rows unless `mode: "archive"`, which removes the](https://datanotes.dev/api/operations/drop_table/): `POST /datanotes/operations/drop_table/`
+- [Add a typed column. Sparse by default (written only on rows that have a value — use for](https://datanotes.dev/api/operations/add_column/): `POST /datanotes/operations/add_column/`
+- [Rename a column in metastore.yaml and on every row. Refused if `to_column` is already a column](https://datanotes.dev/api/operations/rename_column/): `POST /datanotes/operations/rename_column/`
+- [Remove a column from the schema and every row. Previous values are journaled; revert with](https://datanotes.dev/api/operations/drop_column/): `POST /datanotes/operations/drop_column/`
+- [Change a column's attributes (type, min/max/step, values, required, default, label,](https://datanotes.dev/api/operations/alter_column/): `POST /datanotes/operations/alter_column/`
+- [Alter a table: move it (`catalog`, `schema`), rename it (`name`), change](https://datanotes.dev/api/operations/alter_table/): `POST /datanotes/operations/alter_table/`
+- [Create or replace a base: a named set of columns (same spec as `create_table`) that tables](https://datanotes.dev/api/operations/define_base/): `POST /datanotes/operations/define_base/`
+- [Set the bases a table inherits from (`bases`, in order; `[]` removes them). Inherited](https://datanotes.dev/api/operations/set_table_bases/): `POST /datanotes/operations/set_table_bases/`
+- [COMMENT ON CATALOG / SCHEMA: set the `description` of a catalog (`catalog` alone) or of one of its](https://datanotes.dev/api/operations/comment_on/): `POST /datanotes/operations/comment_on/`
+- [Rename one value of an enum column (e.g. `high` → `very high`) in metastore.yaml and every](https://datanotes.dev/api/operations/rename_value/): `POST /datanotes/operations/rename_value/`
+- [ALTER TABLE ADD CONSTRAINT `name` CHECK: `check` is a where filter every row must match,](https://datanotes.dev/api/operations/add_constraint/): `POST /datanotes/operations/add_constraint/`
+- [Remove the CHECK constraint `name` from `table`.](https://datanotes.dev/api/operations/drop_constraint/): `POST /datanotes/operations/drop_constraint/`
+- [Create (or `replace`) a view: a named select stored in metastore.yaml. Name it with the prefix](https://datanotes.dev/api/operations/create_view/): `POST /datanotes/operations/create_view/`
+- [Remove a view from `metastore.yaml` (undo with `undo_migration`).](https://datanotes.dev/api/operations/drop_view/): `POST /datanotes/operations/drop_view/`
+- [CREATE TABLE `target` CLONE `source`: same definition plus independent copies of its rows](https://datanotes.dev/api/operations/clone_table/): `POST /datanotes/operations/clone_table/`
+
+### Rows (DML)
+
+Operations that write rows: insert, update, merge, delete, move, adopt and batch.
+
+- [Insert a row (new note). Values are validated against the columns; the note path comes from](https://datanotes.dev/api/operations/insert_row/): `POST /datanotes/operations/insert_row/`
+- [Update every row matching `where` (required: `{}` is every row). `where`: keys AND-ed; values are equality or operators](https://datanotes.dev/api/operations/update_rows/): `POST /datanotes/operations/update_rows/`
+- [Update exactly ONE row, addressed by its `id` (the system UUID) or its `path`; refused when](https://datanotes.dev/api/operations/update_row/): `POST /datanotes/operations/update_row/`
+- [Delete rows matching `where`. `mode`: "archive" (default, move to the table's archive](https://datanotes.dev/api/operations/delete_rows/): `POST /datanotes/operations/delete_rows/`
+- [Move rows matching `where` (required: `{}` is every row) to another table (e.g. split a domain out of `log` into its own](https://datanotes.dev/api/operations/move_rows/): `POST /datanotes/operations/move_rows/`
+- [Turn existing notes (not rows of any table yet) into rows of `table`, keeping their body and](https://datanotes.dev/api/operations/adopt_rows/): `POST /datanotes/operations/adopt_rows/`
+- [Several writes that succeed or fail together. `operations`: `[{"op": ..., ...arguments}]` where](https://datanotes.dev/api/operations/batch/): `POST /datanotes/operations/batch/`
+- [Add the reciprocal links missing on a table's `inverse` link columns (e.g. Alice lists Carol](https://datanotes.dev/api/operations/align_relations/): `POST /datanotes/operations/align_relations/`
+- [MERGE (upsert) `rows` into `table` by the key columns `on` (e.g. `["occurred_at"]` or](https://datanotes.dev/api/operations/merge_rows/): `POST /datanotes/operations/merge_rows/`
+- [Bring generated and identity columns up to date (all rows, or only `paths`), e.g. after](https://datanotes.dev/api/operations/recompute_columns/): `POST /datanotes/operations/recompute_columns/`
+
+### Queries
+
+Operations that read rows and notes: select_rows, link targets, text and semantic search, the link graph.
+
+- [Read a note or another file of the vault: `{path, content, front_matter, tags, modified_at,](https://datanotes.dev/api/operations/read_note/): `POST /datanotes/operations/read_note/`
+- [The direct children of a vault folder: `{folder, folders, files}`, each a vault path, sorted;](https://datanotes.dev/api/operations/list_folder/): `POST /datanotes/operations/list_folder/`
+- [Plain-text search: every word of `query` must occur, case- and accent-insensitive, in the](https://datanotes.dev/api/operations/search_notes/): `POST /datanotes/operations/search_notes/`
+- [Query rows: `{table, total, rows: [{path, ...}]}`. `table` may be a view,](https://datanotes.dev/api/operations/select_rows/): `POST /datanotes/operations/select_rows/`
+- [The rows a link column may point to, for one row: what an agent or a picker offers before writing](https://datanotes.dev/api/operations/list_link_targets/): `POST /datanotes/operations/list_link_targets/`
+- [Find notes by meaning, not exact words: `{query, results: [{path, score, heading, snippet,](https://datanotes.dev/api/operations/semantic_search/): `POST /datanotes/operations/semantic_search/`
+- [Direct neighbours of notes: what they link to (`out`) and what links to them (`in`,](https://datanotes.dev/api/operations/graph_neighbors/): `POST /datanotes/operations/graph_neighbors/`
+- [Breadth-first walk over links from `from_nodes` (`{"paths": [...]}` and/or `{"table", "where"}`).](https://datanotes.dev/api/operations/graph_traverse/): `POST /datanotes/operations/graph_traverse/`
+- [Shortest chain of links between two sets of notes (each `{"paths": [...]}` and/or](https://datanotes.dev/api/operations/graph_path/): `POST /datanotes/operations/graph_path/`
+
+### History
+
+Operations on the journal, a table's versions, the change feed and webhooks: undo, time travel, restore and vacuum.
+
+- [Revert a journaled table operation by its `migration_id`. Refused (nothing written) when any](https://datanotes.dev/api/operations/undo_migration/): `POST /datanotes/operations/undo_migration/`
+- [The operation journal, newest first: `total` (the matching operations) and `migrations`, each with](https://datanotes.dev/api/operations/list_migrations/): `POST /datanotes/operations/list_migrations/`
+- [DESCRIBE HISTORY: the table's versions, newest first — `version`, `timestamp`, `operation`](https://datanotes.dev/api/operations/describe_history/): `POST /datanotes/operations/describe_history/`
+- [Change data feed between two versions (both included, end defaults to latest): one entry](https://datanotes.dev/api/operations/table_changes/): `POST /datanotes/operations/table_changes/`
+- [Change feed across all tables: numbered row changes (`seq`), oldest first, from engine writes](https://datanotes.dev/api/operations/list_changes/): `POST /datanotes/operations/list_changes/`
+- [Webhooks of the change feed: id, url, filter (tables, ops, columns, exclude_actor), `cursor` (last change delivered), `status` (ok, failing, disabled), `last_delivery`, `last_error`. Secrets are not shown.](https://datanotes.dev/api/operations/list_webhooks/): `POST /datanotes/operations/list_webhooks/`
+- [Push row changes to a URL (a service that would rather be called than poll `list_changes`): batches of up to 100 events POSTed as `{webhook, events, last}`, signed with `X-Datanotes-Signature: sha256=<HMAC of the body>`, retried with backoff until a 2xx. Events are those of `list_changes` (hand edits with their `violations`; the hand edits the service has seen are recorded before it stops). Starts from now unless `since`. Returns the webhook with its `secret` — shown only here: give it to the receiver. Only on the user's explicit request: a webhook sends the database's changes (and, with `since`, its past ones) to that address.](https://datanotes.dev/api/operations/create_webhook/): `POST /datanotes/operations/create_webhook/`
+- [Stop and remove a webhook (by `id` from list_webhooks).](https://datanotes.dev/api/operations/delete_webhook/): `POST /datanotes/operations/delete_webhook/`
+- [RESTORE TABLE to a `version` or `timestamp`: rows and definition return to that state —](https://datanotes.dev/api/operations/restore_table/): `POST /datanotes/operations/restore_table/`
+- [RESTORE DATABASE to a `timestamp` (each table as of its last version committed at or before it) or](https://datanotes.dev/api/operations/restore_database/): `POST /datanotes/operations/restore_database/`
+- [Delete transaction-log history older than `retain_from_version` (default: keep only the](https://datanotes.dev/api/operations/vacuum_table/): `POST /datanotes/operations/vacuum_table/`
+
+### Auxiliary
+
+Operations that describe and check the database, configure it and maintain its indexes.
+
+- [List every table: `name`, `catalog`, `schema`, `qualified` (`personal.metrics.mood` — use it](https://datanotes.dev/api/operations/list_tables/): `POST /datanotes/operations/list_tables/`
+- [Describe a table (`catalog.schema.table` or a shorter unambiguous name): `qualified`, `name`,](https://datanotes.dev/api/operations/describe_table/): `POST /datanotes/operations/describe_table/`
+- [Describe the database: its folder (`root`), `metastore` (`metastore.yaml`: read it freely,](https://datanotes.dev/api/operations/describe_database/): `POST /datanotes/operations/describe_database/`
+- [Configure the database (persisted in metastore.yaml `config:`): `root` (rename the database](https://datanotes.dev/api/operations/configure_database/): `POST /datanotes/operations/configure_database/`
+- [Validate every row against the table (required, types, ranges, enums, links, unknown](https://datanotes.dev/api/operations/check_table/): `POST /datanotes/operations/check_table/`
+- [Read-only schema inference from notes' front matter.](https://datanotes.dev/api/operations/infer_table/): `POST /datanotes/operations/infer_table/`
+- [The semantic search index: `enabled`, `provider`, `model`, `files`, `passages`, `pending`,](https://datanotes.dev/api/operations/describe_semantic_index/): `POST /datanotes/operations/describe_semantic_index/`
+- [Bring the semantic search index up to date: embed the notes changed since the last run (the](https://datanotes.dev/api/operations/sync_semantic_index/): `POST /datanotes/operations/sync_semantic_index/`
+
+### change feed
+
+The change feed as Server-Sent Events, and the tool calls MCP servers report.
+
+- [Record an agent's tool call in the change feed (`tool_call` events): what an MCP server reports after each call, its own tools included (`served_by`). Any token may call it.](https://datanotes.dev/api/operations/record_tool_call/): `POST /datanotes/tool-calls/record/`
+- [The change feed as Server-Sent Events (id = sequence number). Resumes from Last-Event-ID or `since`.](https://datanotes.dev/api/operations/changes_stream/): `GET /datanotes/changes/stream`
+
+### tokens
+
+Named tokens and their permissions; what MCP servers report about their agents' sessions.
+
+- [Tell the engine the model and harness behind an agent session (what an MCP server reports): later requests carrying that session in `X-Datanotes-Actor` are journaled with it. Any token may call it.](https://datanotes.dev/api/operations/identify_session/): `POST /datanotes/sessions/identify/`
+- [List the named tokens (owner only).](https://datanotes.dev/api/operations/tokens_list/): `POST /datanotes/tokens/`
+- [The calling token's own entry (a named token; 400 for the owner).](https://datanotes.dev/api/operations/tokens_me/): `POST /datanotes/tokens/me/`
+- [Create a named token (owner only). Returns the token once.](https://datanotes.dev/api/operations/tokens_create/): `POST /datanotes/tokens/create/`
+- [Change a named token. A token may change its own description and autolink default; permissions are the owner's.](https://datanotes.dev/api/operations/tokens_update/): `POST /datanotes/tokens/update/`
+- [Revoke a named token (owner only): it stops working at once.](https://datanotes.dev/api/operations/tokens_delete/): `POST /datanotes/tokens/delete/`
+- [Give a named token a new value (owner only): the old one stops working.](https://datanotes.dev/api/operations/tokens_rotate/): `POST /datanotes/tokens/rotate/`
+
+### service
+
+Status, stop, this document, the interactive console, the docs.
+
+- [The sections of the docs of this version closest in meaning to a query: what `search_docs` fuses with its ranking by words (hybrid search).](https://datanotes.dev/api/operations/semantic_search_docs/): `POST /datanotes/manual/search/`
+- [The engine's health at a glance (owner only; `datanotes status` prints it): engine, database, journal, git sync, change feed, webhooks, named tokens, hand edits breaking the rules in the last 24 h, logs.](https://datanotes.dev/api/operations/engine_status/): `POST /datanotes/status/`
+- [Stop the engine gracefully (owner only; `datanotes stop` calls it): it answers at once, then finishes the requests under way, makes the last git sync and store backup commit and push, releases its lock and exits.](https://datanotes.dev/api/operations/stop_engine/): `POST /datanotes/stop/`
+- [This document (as `GET /datanotes/openapi.json`, with a token).](https://datanotes.dev/api/operations/openapi_post/): `POST /datanotes/openapi/`
+- [Status and version (no token needed; `authenticated` tells whether the token sent is accepted, `auth` whose it is).](https://datanotes.dev/api/operations/status/): `GET /`
+- [This document (public: it describes the API, it holds no data).](https://datanotes.dev/api/operations/openapi/): `GET /datanotes/openapi.json`
+- [This document in an interactive console (Swagger UI, served by the engine; public). Authorize with a token to try the routes.](https://datanotes.dev/api/operations/docs/): `GET /datanotes/docs`
+- [The index of the docs of this version (every page of datanotes.dev as Markdown, shipped with the package): each page with its description and route.](https://datanotes.dev/api/operations/manual_index/): `GET /datanotes/manual/`
+- [A page of the docs of this version as Markdown, or one of its sections with `section`, the anchor of its heading. The same text as the page with `.md` on datanotes.dev, offline.](https://datanotes.dev/api/operations/manual_page/): `GET /datanotes/manual/{page}.md`

package/dist/docs/develop/build-an-app.md

@@ -0,0 +1,320 @@
+# Build an app on Datanotes
+
+> Write a small TypeScript app that reads and writes your film diary with its own token, and follows its changes.
+
+This tutorial builds film night, a small app for the diary of classic films. The app is TypeScript on the [HTTP API](https://datanotes.dev/develop/http-api/), with the [JavaScript client](https://datanotes.dev/develop/client/) and its [own token](https://datanotes.dev/guides/security/tokens/), and every answer on this page comes from a real run against the engine. By the end, film night does three things:
+
+* Reads the films you still want to watch
+* Logs a viewing as one undoable change
+* Prints a line each time a film becomes watched
+
+## Before you start
+
+You need:
+
+* **Node.js 24** or newer, which runs `.ts` files directly
+* **An engine** that runs on your notes, with `npx datanotes serve --database your_database_folder`
+* **The diary’s tables**: `film` and `watch` as in [Read and filter rows](https://datanotes.dev/guides/read-and-write/read-and-filter/), with the films of that page
+* **The `datanotes` package** in the app’s folder: `npm install datanotes`, with `"type": "module"` in its `package.json`
+
+The app reads a view of the films to watch, and the engine refuses a watched film without a rating. Create both once with the owner token:
+
+```ts
+await owner.call("create_view", {
+  name: "vw_to_watch",
+  query: {
+    table: "film",
+    columns: ["title", "year", "minutes", "director"],
+    where: { status: "to watch" },
+    order_by: "minutes",
+  },
+});
+await owner.call("add_constraint", {
+  table: "film",
+  name: "rated_when_watched",
+  check: {
+    or: [{ status: { ne: "watched" } }, { rating: { exists: true } }],
+  },
+});
+```
+
+`owner` is a `DatanotesClient` with the owner token, as in step 1. [Store a query as a view](https://datanotes.dev/guides/database-objects/views/#store-a-query-as-a-view) and [Enforce rules with CHECK constraints](https://datanotes.dev/guides/data-quality/check-constraints/#enforce-rules-with-check-constraints) explain both operations.
+
+## Build film night
+
+The app has four small files in `src/`: the client, the films, the change feed and the entry point.
+
+1. Create the app’s token with the owner token. The engine answers with the token, once, and keeps only its hash:
+
+   ```ts
+   import { DatanotesClient } from "datanotes/client";
+
+
+   const owner = new DatanotesClient({
+     url: "http://127.0.0.1:27150",
+     token: "your_owner_token",
+   });
+
+
+   const r = await owner.post<{ token: string }>("/tokens/create/", {
+     name: "film-night",
+     description: "Logs film nights and picks the next film",
+     write: ["main.public.watch", "main.public.film"],
+   });
+   console.log(r.token);
+   ```
+
+   The token can read every table, and write rows of `watch` and `film` only. It can’t change the schema, undo, or touch `actor`. The engine lists it like this:
+
+   ```json
+   {
+     "name": "film-night",
+     "description": "Logs film nights and picks the next film",
+     "write": ["main.public.watch", "main.public.film"],
+     "ddl": false
+   }
+   ```
+
+   Keep the token in an environment variable, `FILM_NIGHT_TOKEN`.
+
+2. Create the client in `src/db.ts`. Every other file imports it:
+
+   ```ts
+   import { DatanotesClient } from "datanotes/client";
+
+
+   export const db = new DatanotesClient({
+     url: process.env.DATANOTES_URL ?? "http://127.0.0.1:27150",
+     token: process.env.FILM_NIGHT_TOKEN ?? "",
+     actor: { harness: "film-night" },
+   });
+   ```
+
+   The engine records `film-night` as the `auth` of every write this token makes, whatever `actor` says.
+
+3. Read the films to watch in `src/films.ts`. You select a view like a table:
+
+   ```ts
+   import { db } from "./db.ts";
+
+
+   export interface Film {
+     path: string;
+     title: string;
+     minutes: number;
+   }
+
+
+   export async function filmsToWatch(): Promise<Film[]> {
+     const r = await db.call("select_rows", { table: "vw_to_watch" });
+     if (!r.ok) throw new Error(String(r.value));
+     return (r.value as { rows: Film[] }).rows;
+   }
+   ```
+
+   The view answers with the two films still to watch, shortest first. Each row includes its note’s `path`, which the next step uses:
+
+   ```json
+   [
+     {
+       "path": "main/public/film/Battleship Potemkin.md",
+       "title": "Battleship Potemkin",
+       "year": 1925,
+       "minutes": 75,
+       "director": "Sergei Eisenstein"
+     },
+     {
+       "path": "main/public/film/Metropolis.md",
+       "title": "Metropolis",
+       "year": 1927,
+       "minutes": 153,
+       "director": "Fritz Lang"
+     }
+   ]
+   ```
+
+4. Log a viewing, also in `src/films.ts`. A [`batch`](https://datanotes.dev/reference/operations/rows/#batch) inserts the `watch` row and updates the film together:
+
+   ```ts
+   export interface Viewing {
+     at: string;
+     where: "cinema" | "home" | "outdoor screening";
+     place: string;
+     rating?: number;
+   }
+
+
+   export function logViewing(film: Film, v: Viewing) {
+     const { at, where, place, rating } = v;
+     const watch = { film: `[[${film.title}]]`, occurred_at: at, where, place };
+     const seen = { status: "watched", rating, watched_on: at.slice(0, 10) };
+     return db.call("batch", {
+       summary: `Film night: ${film.title}`,
+       operations: [
+         { op: "insert_row", table: "watch", values: watch },
+         { op: "update_row", table: "film", path: film.path, set: seen },
+       ],
+     });
+   }
+   ```
+
+   The engine checks both operations before it writes either. When it refuses one, it writes nothing. The batch is one entry in the journal, so one undo reverts both.
+
+5. Follow the films you mark watched, in `src/feed.ts`. The app long-polls the [change feed](https://datanotes.dev/develop/change-feed-and-webhooks/#choose-how-to-follow-the-feed) with [`list_changes`](https://datanotes.dev/reference/operations/history/#list_changes), keeping only `film` changes to `status` that leave it `watched`:
+
+   ```ts
+   import { db } from "./db.ts";
+
+
+   interface Change {
+     row: { title: string; rating?: number };
+   }
+   type Page = { events: Change[]; last: number; head: number };
+
+
+   const watched = {
+     tables: ["film"],
+     columns: ["status"],
+     where: { status: "watched" },
+   };
+
+
+   async function changes(args: object): Promise<Page> {
+     const r = await db.call("list_changes", { ...watched, ...args });
+     if (!r.ok) throw new Error(String(r.value));
+     return r.value as Page;
+   }
+   ```
+
+   `followWatched` starts at `head`, the feed’s latest change, so it reports only what happens after the app starts. Each call waits up to 20s for a change, and `last` is where the next one starts:
+
+   ```ts
+   export async function followWatched(onWatched: (c: Change) => void) {
+     let since = (await changes({ limit: 1 })).head;
+     for (;;) {
+       const page = await changes({ since, wait_ms: 20_000 });
+       page.events.forEach(onWatched);
+       since = page.last;
+     }
+   }
+   ```
+
+   The feed suits an app better than a webhook here: a named token can read the feed, while only the owner creates webhooks. To pick up where it stopped after a restart, save `last` and start from it.
+
+6. Start the app in `src/main.ts`. It follows the feed and prints each film you mark watched:
+
+   ```ts
+   import { followWatched } from "./feed.ts";
+
+
+   await followWatched(({ row }) => {
+     console.log(`${row.title} is watched, rated ${row.rating ?? "-"}`);
+   });
+   ```
+
+   Run it and leave it running:
+
+   ```bash
+   node src/main.ts
+   ```
+
+## Let agents pick the film
+
+The engine serves no tools of other programs. For agents to pick tonight’s film with one call, film night serves its own Model Context Protocol (MCP) server, with a `pick_film_for_tonight` tool next to the engine’s read tools and instructions of its own. Agents connect to it as to any MCP server. [Build your own MCP server](https://datanotes.dev/develop/build-on-datanotes/#build-your-own-mcp-server) writes that server for this diary, with `datanotes/mcp`.
+
+## Log tonight’s film
+
+`src/tonight.ts` logs the evening with `logViewing`, and prints the batch’s journal entry:
+
+```ts
+import { filmsToWatch, logViewing } from "./films.ts";
+
+
+const films = await filmsToWatch();
+console.log(films.map((f) => `${f.title}, ${f.minutes} min`));
+
+
+const potemkin = films.find((f) => f.title === "Battleship Potemkin")!;
+const r = await logViewing(potemkin, {
+  at: "2026-10-09T21:00:00",
+  where: "cinema",
+  place: "the film club downtown",
+  rating: 5,
+});
+if (!r.ok) console.log(r.value);
+else console.log((r.value as { migration_id: string }).migration_id);
+```
+
+The script prints the films to watch and the batch’s id:
+
+```text
+[ 'Battleship Potemkin, 75 min', 'Metropolis, 153 min' ]
+20261009T215615-batch-film-night-battleship-potemkin
+```
+
+A moment later, the running app prints the change it read from the feed:
+
+```text
+Battleship Potemkin is watched, rated 5
+```
+
+The diary now has a `watch` note, `main/public/watch/2026-10-09T2100.md`, and Metropolis is the only film left to watch.
+
+To revert the whole evening, the owner calls `undo_migration` with the batch’s id. The app can’t: its token answers `token film-night may not call undo_migration (reserved to the owner)`.
+
+## Show the engine’s refusals
+
+When the engine refuses a write, `ok` is `false` and `value` says why, in words a person can read. The app prints it as it is. This script logs Metropolis twice, first without a rating, then with a rating of 6:
+
+```ts
+const metropolis = (await filmsToWatch())
+  .find((f) => f.title === "Metropolis")!;
+const evening = {
+  at: "2026-10-10T20:30:00",
+  where: "home" as const,
+  place: "my living room",
+};
+
+
+for (const rating of [undefined, 6]) {
+  const r = await logViewing(metropolis, { ...evening, rating });
+  if (!r.ok) console.log(r.value);
+}
+```
+
+The first breaks the `rated_when_watched` CHECK, the second the column’s `max: 5`. In both cases the engine undoes the `watch` row of operation 0, and nothing stays written:
+
+```text
+batch refused at operation 1 — the operations before it were undone, nothing is left written:
+operation 1 (update_row): update_row refused — nothing was written:
+invalid main/public/film/Metropolis.md: rated_when_watched violates CHECK rated_when_watched: {"or":[{"status":{"ne":"watched"}},{"rating":{"exists":true}}]}
+batch refused at operation 1 — the operations before it were undone, nothing is left written:
+operation 1 (update_row): update_row refused — nothing was written:
+invalid main/public/film/Metropolis.md: rating must be <= 5
+```
+
+A write outside the app’s permissions comes back the same way. The app may not write `actor`:
+
+```ts
+const r = await db.call("update_rows", {
+  table: "actor",
+  where: { name: "Brigitte Helm" },
+  set: { born: "1906-03-17" },
+});
+if (!r.ok) console.log(r.value);
+```
+
+The engine answers with 403, and the client turns it into the reason:
+
+```text
+token film-night may not write main.public.actor
+```
+
+## Learn more about tokens, the client and the feed
+
+These pages cover each part of film night in full:
+
+* [Give each program its own token](https://datanotes.dev/guides/security/tokens/): every permission and route of named tokens
+* [Build on Datanotes](https://datanotes.dev/develop/build-on-datanotes/): the ways to build on the engine, your own MCP server included
+* [Call the engine from JavaScript](https://datanotes.dev/develop/client/): the client’s methods and errors
+* [Follow every change with the change feed](https://datanotes.dev/guides/history/change-data-feed/#follow-every-change-with-the-change-feed): the feed’s filters, streaming and webhooks

package/dist/docs/develop/build-on-datanotes.md

@@ -0,0 +1,138 @@
+# Build on Datanotes
+
+> Datanotes is a foundation: reach the database through the default MCP server, through a program with its own token, or through an MCP server of your own that reuses the engine’s tools.
+
+Datanotes is a foundation for your own tools: the engine holds your notes as typed tables, checks every write and journals it. You build on it in three ways, and you can mix them:
+
+* **The default MCP server**: `datanotes mcp` gives an agent direct access to the database
+* **A program with its own token**: a script, a service or an app calls the HTTP API, with only the permissions you give it
+* **Your own MCP server**: your tools and your instructions, next to the engine’s tools you choose
+
+Each way reaches the engine through its [HTTP API](https://datanotes.dev/develop/http-api/), so the same checks, permissions and journal apply to all three.
+
+## Give an agent direct access
+
+`datanotes mcp` is the Model Context Protocol (MCP) server of Datanotes. It offers every [operation](https://datanotes.dev/reference/operations/) of the engine as a tool, with instructions that tell the agent how to find tables and write rows. Use it when the agent should work with the database itself: create tables, query them, write rows.
+
+[Connect your agents](https://datanotes.dev/start/connect-agents/) shows how to add it to Claude, Cursor, Codex and other MCP clients. Give each agent its [own token](https://datanotes.dev/guides/security/tokens/) to limit what it may change.
+
+## Write a program with its own token
+
+A program reads and writes through the HTTP API, from any language. In JavaScript and TypeScript, [`datanotes/client`](https://datanotes.dev/develop/client/) runs the same operations agents call. The program can also follow the change feed to react when a note changes.
+
+Give the program its [own token](https://datanotes.dev/guides/security/tokens/). The engine then refuses what the program may not do, and the journal records the token’s name with every write. [Build an app on Datanotes](https://datanotes.dev/develop/build-an-app/) writes such a program step by step.
+
+## Build your own MCP server
+
+An MCP server of your own decides what an agent sees: the engine’s tools it needs, tools of your own that know your data, and instructions in your words. Use it when agents should work through your rules, for example a film diary where the agent picks tonight’s film with one call instead of querying three tables.
+
+`datanotes/mcp` gives your server the engine’s side, with no dependencies:
+
+* **`engineTools(filter?)`**: the engine’s operations as MCP tool definitions, all of them or those you name in `only`, minus those in `except`
+* **`callEngineTool(client, name, args, options?)`**: runs one through a `DatanotesClient` and answers its MCP result; a refusal comes back as an error result the agent can read, with a hint on how to fix it. A tool name or arguments that don’t match the tool’s schema are refused before the call reaches the engine, with the closest name and the tool’s arguments
+* **`isEngineTool(name, filter?)`**: tells your server which calls go to the engine
+* **`docsTools(filter?)`**, **`callDocsTool(name, args, options?)`** and **`isDocsTool(name, filter?)`**: the same for `list_docs`, `read_docs` and `search_docs`, which read the docs of the installed version from the package. With a `client` in the options, `search_docs` also ranks by meaning when the engine has semantic search on: see [hybrid search](https://datanotes.dev/guides/agents/docs-offline/#search-by-words-and-by-meaning)
+* **`ENGINE_INSTRUCTIONS`**: the instructions of `datanotes mcp`, to start your own with; they point agents to `search_docs` and `read_docs`
+* **`toolContent(ok, value)`**: turns a result of your own tools into MCP content
+* **`offlineDocs(content)`**: drops the web address from the `Docs:` lines of an error result and keeps the `read_docs` call. Use it when your server serves the docs tools, so agents read the docs offline as with `datanotes mcp`
+
+Your server brings its own MCP software development kit (SDK), and runs with its own token. This film night server offers three read tools of the engine, the docs tools and a tool of its own. It starts with the client and the tool:
+
+```ts
+import { DatanotesClient } from "datanotes/client";
+import { toolContent } from "datanotes/mcp";
+
+
+const db = new DatanotesClient({
+  url: "http://127.0.0.1:27150",
+  token: process.env.FILM_NIGHT_TOKEN ?? "",
+  actor: { harness: "film-night" },
+});
+
+
+const pickFilm = {
+  name: "pick_film_for_tonight",
+  description: "Pick the longest film to watch that fits in max_minutes.",
+  inputSchema: {
+    type: "object" as const,
+    properties: { max_minutes: { type: "integer", minimum: 1 } },
+  },
+};
+```
+
+`pickFilm` is an MCP tool definition like the engine’s. The function behind it reads the films to watch with the client, and answers with `toolContent`:
+
+```ts
+async function pick(args: Record<string, unknown>) {
+  const r = await db.call("select_rows", {
+    table: "film",
+    where: { status: "to watch" },
+    order_by: "minutes",
+  });
+  if (!r.ok) return toolContent(false, r.value);
+  const max = Number(args.max_minutes ?? Infinity);
+  const rows = (r.value as { rows: { title: string; minutes: number }[] }).rows;
+  const film = rows.filter((f) => f.minutes <= max).at(-1);
+  return toolContent(true, { film: film?.title ?? null });
+}
+```
+
+The server lists the engine’s tools it keeps next to its own, and sends each call to the engine or to `pick`. Its instructions start with the engine’s and add its own rule:
+
+```ts
+import { Server } from "@modelcontextprotocol/sdk/server/index.js";
+import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
+import {
+  CallToolRequestSchema,
+  ListToolsRequestSchema,
+} from "@modelcontextprotocol/sdk/types.js";
+import {
+  callDocsTool, callEngineTool, docsTools, engineTools,
+  isDocsTool, isEngineTool, offlineDocs, ENGINE_INSTRUCTIONS,
+} from "datanotes/mcp";
+
+
+const reads = { only: ["list_tables", "describe_table", "select_rows"] };
+const instructions = `${ENGINE_INSTRUCTIONS}\n\nTo suggest a film for ` +
+  "tonight, call pick_film_for_tonight.";
+const server = new Server(
+  { name: "film-night", version: "1.0.0" },
+  { capabilities: { tools: {} }, instructions },
+);
+```
+
+The two handlers answer the agent’s `tools/list` and `tools/call` requests. A name outside the list answers an error, even when the engine has that operation. The server serves the docs tools, so `offlineDocs` points its errors to `read_docs` instead of a web address:
+
+```ts
+server.setRequestHandler(ListToolsRequestSchema, () => ({
+  tools: [...engineTools(reads), ...docsTools(), pickFilm],
+}));
+
+
+server.setRequestHandler(CallToolRequestSchema, async (req) => {
+  const { name, arguments: args = {} } = req.params;
+  if (isEngineTool(name, reads)) {
+    return offlineDocs(await callEngineTool(db, name, args, reads));
+  }
+  if (isDocsTool(name)) {
+    return offlineDocs(await callDocsTool(name, args, { client: db }));
+  }
+  if (name === pickFilm.name) return pick(args);
+  return toolContent(false, `unknown tool: ${name}`);
+});
+
+
+await server.connect(new StdioServerTransport());
+```
+
+An agent connected to this server sees seven tools: `list_tables`, `describe_table`, `select_rows`, `list_docs`, `read_docs`, `search_docs` and `pick_film_for_tonight`. With a [token](https://datanotes.dev/guides/security/tokens/) that only reads, the server can’t change your notes even if its code asks to. To log its tool calls on the engine’s change feed, the server reports them with [`tool-calls/record/`](https://datanotes.dev/guides/security/who-did-what/#report-what-an-agent-does).
+
+## Choose a way
+
+Pick by who acts and what they need:
+
+* **An agent that manages the database**: `datanotes mcp`, with the agent’s own token
+* **A program that runs on its own**: the HTTP API or `datanotes/client`, with the program’s own token
+* **Agents that work through your rules**: your own MCP server, with `datanotes/mcp`
+
+The engine itself serves no tools of other programs: a program that offers tools to agents serves them from its own MCP server. An agent can connect to that server and to `datanotes mcp` at once.