datanotes 2.0.0 → 3.0.0

package/dist/docs/administration/configuration.md

@@ -0,0 +1,59 @@
+# Configure the engine and the database
+
+> The engine’s settings file in its store, the database’s own settings in metastore.yaml, and the environment variables of the datanotes commands.
+
+Datanotes keeps its configuration in two places, for two different things:
+
+* **The engine’s settings**: `settings.json` in the engine’s store, read when the engine starts; the port, the owner token, logs, telemetry, semantic search and the store backup
+* **The database’s settings**: the `config` of `metastore.yaml`, in the database folder, changed while the engine runs with `configure_database`; the default catalog and schema, the time zone, the archive folder, history and automatic links
+
+## Change the engine’s settings
+
+The engine reads its settings file when it starts, and writes it at the first start with the port `27150` and a random owner token. Missing keys take their default. The file is in the engine’s store, outside your notes. Find it with `datanotes store path`:
+
+```bash
+npx -y datanotes store path --database your_database_folder
+```
+
+To change a setting, stop the engine, edit `settings.json` in that folder and start the engine again:
+
+```json
+{
+  "httpServerPort": 27150,
+  "httpServerToken": "your_owner_token",
+  "journalRetentionDays": 90,
+  "fileLogEnabled": true
+}
+```
+
+`datanotes store backup` writes the store backup’s settings for you: [Back up the history](https://datanotes.dev/guides/history/back-up/) shows how. The settings file itself is never backed up.
+
+`--settings` (or `DATANOTES_SETTINGS`) names another file. The [settings reference](https://datanotes.dev/reference/settings/) lists every key and its default. The file holds the owner token, so keep it private.
+
+The settings file doesn’t name the database folder or the store. The engine finds the database folder at start, and the store by the database’s id: [Run the engine as a service](https://datanotes.dev/administration/run-the-service/#where-the-engine-keeps-its-store) explains both, with the `--store` and `DATANOTES_DATA_HOME` options.
+
+## Change the database’s settings
+
+The database’s settings live in `metastore.yaml`, which changes only through operations. [`configure_database`](https://datanotes.dev/reference/operations/auxiliary/#configure_database) sets them, and returns the database as `describe_database` describes it:
+
+* **`default_catalog`** and **`default_schema`**: where short table names, such as `film`, resolve
+* **`root`**: the database folder in a vault; renaming it moves the folder and rewrites the links to its notes, and `""` moves the database to the vault root
+* **`default_partition`** and **`filename`**: the sub-folders and file names of new time-based tables
+* **`archive_folder`**: where deleted rows go
+* **`timezone`**: the IANA time zone that completes date-times written without an offset
+* **`history`**: the transaction log for every table; `true` in a database the engine creates, and off when `config` lacks it. See [The transaction log keeps every version](https://datanotes.dev/start/how-it-works/#the-transaction-log-keeps-every-version) for its disk cost
+* **`autolink`**: [automatic links](https://datanotes.dev/guides/read-and-write/automatic-links/) in the text writes put in notes, on by default
+
+To revert a change, call `configure_database` again with the previous values. A move of the database folder with `root` is recorded in the journal and in the history of every table, so undo and time travel find the rows in their new folder. [Configure the database settings](https://datanotes.dev/guides/database-objects/catalogs-and-schemas/#configure-the-database-settings) shows it on a real engine.
+
+`metastore.yaml` also holds the database’s `id`, at the top. The engine writes it and finds the database’s store by it, so leave it as it is.
+
+## Set options with environment variables
+
+Every option of the `datanotes` commands also has an environment variable, which applies when the command line doesn’t give the option. They suit containers and services:
+
+* **`datanotes serve`**: [Run the engine as a service](https://datanotes.dev/administration/run-the-service/#set-the-options-of-serve), [Sync the vault with git](https://datanotes.dev/administration/git-sync/) and [Back up the history](https://datanotes.dev/guides/history/back-up/#set-the-backup-for-one-run)
+* **`datanotes mcp`**: [Connect your agents](https://datanotes.dev/start/connect-agents/#set-options-with-flags-or-environment-variables)
+* **Telemetry**: the `OTEL_EXPORTER_OTLP_*` variables, in [Send logs to your observability backend](https://datanotes.dev/develop/telemetry/)
+
+The [command line reference](https://datanotes.dev/reference/cli/) shows every command and option.

package/dist/docs/administration/docker.md

@@ -0,0 +1,80 @@
+# Run in Docker
+
+> The engine in a container, on a server.
+
+You can run the headless engine (`src/node/service.ts`) in a container, on any machine that runs Docker, such as a virtual private server (VPS), a network-attached storage (NAS) device or a Linux server. This page shows how to:
+
+* Start the engine and find its token
+* Set up file watching and permissions
+* Open network access and connect agents
+
+## Start the engine
+
+Build the engine first, then start the container with Docker Compose:
+
+1. Build the service from the repository root:
+
+   ```bash
+   npm run build:service
+   ```
+
+2. Start the container on your vault folder:
+
+   ```bash
+   DATANOTES_VAULT=your_vault_folder docker compose \
+     -f deploy/docker/docker-compose.yml up -d --build
+   ```
+
+The container mounts your vault folder at `/vault`, and its entrypoint starts the engine with `--vault /vault`. The database is then a folder inside the vault, `datanotes/` for a new database. A folder that holds `metastore.yaml` at its root works too: the engine then finds the database at the vault root. [Use Datanotes with Obsidian](https://datanotes.dev/administration/use-with-obsidian/) explains both setups. The first start creates the settings file, `/config/settings.json`, in the `datanotes-config` volume, with port 27150 and a random owner token. The [settings reference](https://datanotes.dev/reference/settings/) lists every key you can set there, such as semantic search and logs.
+
+## Keep the engine’s store in a volume
+
+The engine keeps its store outside the vault: the journal, the history of your tables, the named tokens, the change feed and the logs. In the container, the store is `/data/<database id>/`, in the `datanotes-data` volume. The image sets `DATANOTES_DATA_HOME=/data`. Keep that volume: without it, a new container starts with an empty journal and no history.
+
+## Get the owner token
+
+The owner token is in the settings file inside the container. Print it with:
+
+```bash
+docker compose -f deploy/docker/docker-compose.yml \
+  exec datanotes cat /config/settings.json
+```
+
+The owner token gives full access. Keep it for administration, and [create a named token](https://datanotes.dev/guides/security/tokens/#create-a-token) for each agent, program or device.
+
+## Run programs next to the engine
+
+Programs such as a task planner or your own agent run as their own containers or processes. Each one gets its own token from `POST /datanotes/tokens/create/` and reaches the engine over HTTP. [Build on Datanotes](https://datanotes.dev/develop/build-on-datanotes/) explains the ways to build on the engine.
+
+## Choose how the engine sees file changes
+
+The container polls the vault for changes every 3s, because `DATANOTES_WATCH` defaults to `poll`. Bind mounts from Windows and macOS hosts deliver no file events, so polling is the only option there. On a Linux host, set `DATANOTES_WATCH=events` to react to changes as they happen.
+
+Polling a large vault over a bind mount is slow: one scan of 10,000 notes takes seconds. The engine then scans less often, so a change you make on the host shows later. On Windows and macOS, keep a large vault on a Linux volume instead, and set `DATANOTES_WATCH=events`. [Limits](https://datanotes.dev/administration/limits/#keep-a-docker-vault-on-a-linux-volume) shows the compose file and the times we measured.
+
+## Run one engine per vault
+
+Run one engine per database folder. The engine holds a lock file, `engine.lock`, in its store, so it refuses only a second engine with the same store. The container keeps its store in its own volume. An engine on the host has another store, so it starts on the same folder, and the two engines lose each other’s writes.
+
+Never run the engine on the host and the engine in Docker on the same folder. Stop the host engine with `datanotes stop` before you start the container. [Sync the vault between devices](https://datanotes.dev/guides/data-quality/sync-between-devices/) explains the same rule for a synced vault.
+
+## Give the container write access to the vault
+
+The container runs as the user `engine`, and the engine writes notes in the vault. On its first start it also writes the database’s id into `metastore.yaml`. Make the vault folder writable by that user in one of two ways:
+
+* Add a `user:` entry to the compose file with the owner of the folder
+* Change the folder’s owner with `chown`
+
+## Reach the engine from other machines
+
+The compose file publishes the port on `127.0.0.1` only. To reach the engine from outside, put a reverse proxy or a tunnel in front of it. The [security page](https://datanotes.dev/guides/security/security-model/) explains what to expose and which tokens to use.
+
+## Connect agents to the engine in Docker
+
+Agents reach the engine through the Model Context Protocol (MCP) server. Run it next to the engine:
+
+```bash
+npx -y datanotes mcp serve --url http://127.0.0.1:27150
+```
+
+[Connect agents](https://datanotes.dev/start/connect-agents/) explains how to point your agents at it.

package/dist/docs/administration/git-sync.md

@@ -0,0 +1,71 @@
+# Sync the vault with git
+
+> Have the engine commit, pull and push the vault’s git repository, as a backup and to keep copies on other machines in step.
+
+Git sync keeps the vault’s own git repository committed and pushed while the engine runs. Use it as a backup, and to keep a copy of the vault on another machine, such as a server, in step with this one. Git only carries the files: undo and history stay in the engine’s journal and transaction log.
+
+## Turn on git sync
+
+The vault must be a git repository whose branch has an upstream: push it once with `git push -u`. Start the engine with `--git-sync`:
+
+```bash
+npx -y datanotes serve --database your_database_folder --git-sync
+```
+
+Git runs as the user that runs the engine, with that user’s identity and SSH keys. These options turn it on and tune it:
+
+| Option                  | Variable                     | Meaning                                                                     |
+| ----------------------- | ---------------------------- | --------------------------------------------------------------------------- |
+| `--git-sync`            | `DATANOTES_GIT_SYNC=1`       | Turns git sync on                                                           |
+| `--git-interval-min 10` | `DATANOTES_GIT_INTERVAL_MIN` | Minutes between two syncs; at least 1, default 10                           |
+| `--git-quiet-s 60`      | `DATANOTES_GIT_QUIET_S`      | Seconds after the engine’s last write before a sync; at least 5, default 60 |
+| `--git-no-push`         | None                         | Commits and pulls, but never pushes                                         |
+
+## What a sync does
+
+A sync runs every `--git-interval-min` minutes, and `--git-quiet-s` seconds after the engine’s last write, so a burst of writes becomes one commit:
+
+1. It commits the vault’s changes, with a message that lists the operations applied since the last commit and who made them.
+2. It fetches, and when the remote has new commits, pulls them with `git pull --rebase --autostash`.
+3. It pushes when the vault is ahead of the remote.
+
+The commit leaves out `.obsidian/` and `.trash/`. The engine’s store, with the journal and the history of this machine, is outside the vault, so git never sees it. When the engine stops, it makes a last commit and push.
+
+The commit also leaves out these engine files, whatever `.gitignore` says:
+
+* **`.datanotes/`**: the store of a Datanotes 2 vault, with the owner token. The engine logs a warning when the vault still holds one. Add `.datanotes/` to `.gitignore`, or move the folder out of the vault. If an earlier commit holds it, run `git rm -r --cached .datanotes`, commit, and change the token.
+* **`.datanotes-*.tmp`** and **`.datanotes-rename-*`**: files the engine is still writing or renaming.
+
+Git runs from its folder on `PATH`, never from the vault folder. A `git.exe` inside the vault never runs.
+
+## Git never waits for a person
+
+The engine runs as a service, with nobody to answer a prompt. So git never asks:
+
+* **No password prompt**: a missing credential fails the command at once.
+* **No Git Credential Manager window**: the engine sets `GCM_INTERACTIVE=never`.
+* **SSH in batch mode**: an unknown host key or a key with a passphrase fails the command. Your own `GIT_SSH_COMMAND` or `core.sshCommand` wins.
+
+Each git command also has a time limit. A fetch, pull or push may take 10 minutes, any other command 60 seconds. Past it, the engine stops git and the processes it started. The sync then fails, `datanotes status` shows why, and the next sync tries again. Set `DATANOTES_GIT_NETWORK_TIMEOUT_S` to change the limit of a fetch, pull or push, in seconds.
+
+Check once, as the user that runs the engine, that `git push` works without a prompt. For SSH, add the host to `known_hosts` and load the key in an SSH agent, or use a key without a passphrase.
+
+## Back up the store too
+
+Undo, table history and time travel read the store, not the vault. To keep them with the vault, turn on the store backup: it pushes the store to a repository of its own. It commits every 60 seconds by default, between two operations, so a restore finds both repositories at nearly the same moment. [Back up the history](https://datanotes.dev/guides/history/back-up/) shows how.
+
+## Resolve a conflict
+
+When a pull conflicts with local changes, the engine aborts the rebase and stops pushing, so nothing is lost. `datanotes status` then reports git sync as `blocked`, with the reason, and exits with code 1. Resolve the conflict in the repository with git, as you would for any repository; the next sync commits and pushes again.
+
+A pull can also succeed and still leave files in conflict. This happens when the engine writes a note while the pull runs: git puts that change back after the pull, and it conflicts. The engine then stops the sync before pushing, and never commits the conflict markers. The status names the files. To resolve them:
+
+1. Edit each file named in the status, and keep the text you want.
+2. Run `git add` on each file.
+3. Run `git stash drop` when git says your changes are safe in the stash.
+
+The next sync commits and pushes again.
+
+## Keep every copy on the same version
+
+Each copy of the vault that runs an engine reads the notes that git brings in as hand edits: the engine checks them against the tables’ rules and reports what they break on the change feed. Run the same version of Datanotes on every copy, and upgrade them together. The [release notes](https://datanotes.dev/release-notes/) list what each version changes.

package/dist/docs/administration/limits.md

@@ -0,0 +1,69 @@
+# Limits
+
+> How the engine performs on large databases: start time, select and insert times, memory, and how to run it on 10,000 notes or more.
+
+The engine keeps every row in a Markdown note, so its speed depends on the number of notes and on the disk that holds them. This page gives the times we measured and shows how to run the engine on a large database.
+
+## Measured times
+
+We measured the engine on databases of 10,000 and 50,000 notes. On Windows 11, Microsoft Defender scanned the files. On Linux, the engine ran in a container with the notes on a Linux volume.
+
+| Measure                                 | 10,000 notes | 50,000 notes |
+| --------------------------------------- | ------------ | ------------ |
+| Start, Windows 11 with Defender         | 15 to 18 s   | 63 to 115 s  |
+| Start, Linux container                  | 4 to 5 s     | 22 s         |
+| `select_rows` with `where`              | 30 to 35 ms  | 178 ms       |
+| `insert_row`                            | 91 to 128 ms | 414 ms       |
+| Memory (resident set size) after writes | Not measured | Up to 2 GB   |
+
+These numbers set the size the engine handles well:
+
+* Up to about 10,000 notes, it starts within 20 seconds and answers within a few hundred milliseconds
+* From 10,000 to 20,000 notes, start it before your agents and keep it running
+* Above about 20,000 notes, run it as a service on Linux, and give it 2 GB of memory
+
+## Start the engine before the agent
+
+`datanotes mcp --database` starts an engine when none runs on the folder. The agent then waits for the whole start, up to 60 seconds. On a large database, the start can take longer than the agent waits.
+
+Above about 20,000 notes, run `datanotes serve` as a service that starts with the machine. [Run the engine as a service](https://datanotes.dev/administration/run-the-service/) shows how. `datanotes mcp --database` then finds the running engine and connects at once.
+
+To let `datanotes mcp` wait longer for the engine it starts, set `DATANOTES_ENGINE_START_TIMEOUT_S` in seconds:
+
+```bash
+DATANOTES_ENGINE_START_TIMEOUT_S=300 \
+  npx -y datanotes mcp --database your_database_folder
+```
+
+## Change many notes at once
+
+A git pull, a sync tool or a find and replace can change thousands of notes outside the engine. The engine then brings their generated and identity columns up to date:
+
+* It waits until the notes of a table stay quiet for 3 seconds.
+* It recomputes them in one operation per table, with one entry in the journal.
+* It skips the notes whose columns need nothing, such as notes that already hold a valid `id`.
+* It pauses between rows, so the engine still answers requests meanwhile.
+
+## Keep a Docker vault on a Linux volume
+
+On Windows and macOS, a Docker bind mount of a host folder delivers no file events. The engine must then poll the vault: it compares the whole folder with what it knows, every few seconds. Over a bind mount, one comparison of 10,000 notes takes seconds.
+
+The engine adapts to slow comparisons:
+
+* It waits at least 4 times as long as the last comparison before the next one, up to a minute.
+* It skips the comparison before each operation while comparisons are slow. A change you make outside then shows at the next comparison.
+
+For a large vault, keep the notes on a Linux volume instead, and watch them with file events:
+
+```yaml
+services:
+  datanotes:
+    volumes:
+      - vault:/vault
+    environment:
+      DATANOTES_WATCH: events
+volumes:
+  vault:
+```
+
+On a Linux host, a bind mount delivers file events: set `DATANOTES_WATCH=events` there too. [Run in Docker](https://datanotes.dev/administration/docker/) explains the container.

package/dist/docs/administration/run-the-service.md

@@ -0,0 +1,219 @@
+# Run the engine as a service
+
+> Start datanotes serve on a database folder or a vault, find its store, choose its address, its port and how it sees file changes, keep one engine per database, and stop it cleanly with datanotes stop.
+
+The engine is one process, `datanotes serve`, that runs on one folder: a database folder or a vault. It serves the [HTTP API](https://datanotes.dev/develop/http-api/) that programs, the MCP server and other clients call. It watches the folder for hand edits, and it keeps its records in its store, a folder outside it. This page shows how to start it, configure it and stop it.
+
+## Start the engine
+
+You need [Node.js](https://nodejs.org) 22 or newer. Start the engine on your database folder:
+
+```bash
+npx -y datanotes serve --database your_database_folder
+```
+
+The folder is the database: `metastore.yaml` sits at its root, and a new database creates it there. Every path is relative to the folder, such as `main/public/film/Metropolis.md` for a row.
+
+When an app such as Obsidian opens a whole vault, start the engine on the vault with `--vault your_vault_folder` instead. The database is then a subfolder of the vault, and paths start from the vault root. [Use Datanotes with Obsidian](https://datanotes.dev/administration/use-with-obsidian/) explains this mode. Give one of the two options: the engine refuses both together.
+
+The first start creates the engine’s store and its settings file, `settings.json`, with the port `27150` and a random owner token. The engine then serves the HTTP API on `http://127.0.0.1:27150`. To see where the store is, run:
+
+```bash
+npx -y datanotes store path --database your_database_folder
+```
+
+To run the engine in a container, follow [Run in Docker](https://datanotes.dev/administration/docker/).
+
+## Where the engine keeps its store
+
+The store holds everything the engine writes besides your notes:
+
+* the settings file `settings.json`, with the owner token
+* the named tokens, the webhooks and the lock `engine.lock`
+* the journal of operations, which [undo](https://datanotes.dev/guides/history/journal-and-undo/) reads
+* the transaction logs of [table history](https://datanotes.dev/guides/history/table-history/)
+* the change feed, the semantic index and the log files
+
+Each database has its own store, named after its id: `<data home>/<database id>/`. The id is the `id` line at the top of the database’s `metastore.yaml`. The data home is the operating system’s folder for per-user application data:
+
+| System  | Data home                                                         |
+| ------- | ----------------------------------------------------------------- |
+| Windows | `%LOCALAPPDATA%\Datanotes`                                        |
+| macOS   | `~/Library/Application Support/Datanotes`                         |
+| Linux   | `$XDG_DATA_HOME/datanotes`, by default `~/.local/share/datanotes` |
+
+Everything the engine keeps is in that one folder, so you can back it up or archive it as a whole. The [store backup](https://datanotes.dev/guides/history/back-up/) does it for you: it pushes the journal and the history to a private git repository. Your folder holds only your notes and `metastore.yaml`: sync tools and git see no engine files.
+
+You can place the store elsewhere:
+
+* `DATANOTES_DATA_HOME` moves the data home, for every database
+* `--store your_store_folder`, or `DATANOTES_STORE`, names the store of this database
+
+A store named with `--store` isn’t recorded in the data home. Give the same `--store` to `datanotes status` and set `DATANOTES_STORE` for `datanotes mcp --database`, so they find the engine.
+
+## Set the options of serve
+
+Each option has an environment variable, which applies when the command line doesn’t give the option:
+
+| Option                               | Variable                                                                       | Meaning                                                                                              |
+| ------------------------------------ | ------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------- |
+| `--database your_database_folder`    | `DATANOTES_DATABASE`                                                           | The database folder the engine runs on, with `metastore.yaml` at its root                            |
+| `--vault your_vault_folder`          | `DATANOTES_VAULT`                                                              | The vault the engine runs on, with the database in a subfolder; instead of `--database`              |
+| `--store your_store_folder`          | `DATANOTES_STORE`                                                              | The engine’s store; default `<data home>/<database id>/`                                             |
+| None                                 | `DATANOTES_DATA_HOME`                                                          | The data home, the folder of every store                                                             |
+| `--settings your_settings_file`      | `DATANOTES_SETTINGS`                                                           | The settings file; default `settings.json` in the store                                              |
+| `--host your_address`                | `DATANOTES_HOST`                                                               | The address the HTTP API listens on; default `127.0.0.1`                                             |
+| `--port 27150`                       | `DATANOTES_PORT`                                                               | The port of the HTTP API for this run, over the `httpServerPort` setting; `0` for any free port      |
+| `--watch events` or `--watch poll`   | `DATANOTES_WATCH`                                                              | How the engine sees file changes; default `events`                                                   |
+| `--poll-ms 3000`                     | `DATANOTES_POLL_MS`                                                            | The interval of `--watch poll`, in milliseconds; at least 200                                        |
+| `--embedding-key-file your_key_file` | `DATANOTES_EMBEDDING_KEY_FILE`, or the key itself in `DATANOTES_EMBEDDING_KEY` | The API key of the embedding provider for [semantic search](https://datanotes.dev/guides/search/semantic/)                |
+| `--git-sync`                         | `DATANOTES_GIT_SYNC=1`                                                         | Commit, pull and push the folder: [Sync the vault with git](https://datanotes.dev/administration/git-sync/)               |
+| `--store-backup`                     | `DATANOTES_STORE_BACKUP=1`                                                     | Back up the store to git: [Back up the history](https://datanotes.dev/guides/history/back-up/#set-the-backup-for-one-run) |
+| `--store-backup-remote your_url`     | `DATANOTES_STORE_BACKUP_REMOTE`                                                | The store backup’s repository                                                                        |
+| `--store-backup-interval-s 60`       | `DATANOTES_STORE_BACKUP_INTERVAL_S`                                            | Seconds between two looks for changes by the store backup, at least 10                               |
+| None                                 | `DATANOTES_GIT_NETWORK_TIMEOUT_S`                                              | Seconds a git fetch, pull or push of git sync and the store backup may take; default 600             |
+| `--log your_log_file`                | None                                                                           | Appends the console output to a file                                                                 |
+
+The port and the owner token live in the settings file, with every other setting: [Configure the engine and the database](https://datanotes.dev/administration/configuration/) explains them.
+
+## Choose the port
+
+The engine listens on the port of the `httpServerPort` setting, `27150` by default. To use another port for one run, pass `--port`:
+
+```bash
+npx -y datanotes serve --database your_database_folder --port 27180
+```
+
+With `--port 0`, the system picks a free port. The engine prints the port at start, writes it in its lock and reports it in `datanotes status`. Tools such as `datanotes mcp --database` read the port from the lock, so they find the engine on any port. `--port` doesn’t change the settings file.
+
+When the port is in use, the engine waits up to 45 seconds for it, because an engine that restarts may still be closing. It prints a line while it waits. When the engine can’t listen, it exits with code 1 and says why and what to do:
+
+* **The port is in use**: another program holds it, often another engine. Stop that program, or pass another `--port`.
+* **The system reserves the port**: choose another port from 1024 to 65535. On Windows, `netsh interface ipv4 show excludedportrange protocol=tcp` lists the reserved ports.
+* **The address isn’t this machine’s**: give `--host` an address of this machine, such as `127.0.0.1`.
+
+## One database per vault
+
+With `--database`, the folder is the database, and `metastore.yaml` sits at its root.
+
+With `--vault`, a vault holds exactly one database: the folder of its one `metastore.yaml`. A new database goes to `datanotes/` at the top of the vault. The folder can be anywhere in the vault, also in a subfolder or at the vault root.
+
+At each start with `--vault`, the engine looks for `metastore.yaml` in the whole vault. It skips folders whose names start with a dot, and `node_modules`. Then:
+
+* **One `metastore.yaml`**: its folder is the database folder, whatever its name, such as `Datanotes/` or `notes/db/`. The engine doesn’t rename it.
+* **A different folder from last time, with the same id**: you moved the database folder by hand. The engine records the move.
+* **No `metastore.yaml`**: a new database, in `datanotes/`.
+
+When the vault holds more than one `metastore.yaml`, at any depth, the engine refuses to start and lists them. Keep the database’s folder, then remove the extra `metastore.yaml` and start the engine again. To keep its tables, create them in the database with `create_table` and copy the rows with `insert_row`.
+
+To rename or move the database folder, call `configure_database` with `root`, or move the folder by hand. `root: ""` moves the database to the vault root, so you can start the engine with `--database` from then on. [Move the vault or the database folder](https://datanotes.dev/start/how-it-works/#move-the-vault-or-the-database-folder) explains both.
+
+## Run a copy of a vault
+
+A copy of a vault holds the same database id as the original. Two databases can’t share a store, so the engine refuses to start on the copy while the original vault still exists. Give the copy its own id, with the engine stopped:
+
+```bash
+npx -y datanotes store new-id --vault your_copy_folder
+```
+
+The copy then gets a new, empty store at its next start: new settings, a new owner token, an empty journal and no history. To run both engines at once, start the copy with `--port 0`, or set its `httpServerPort`. To keep the original’s history in the copy, copy the original’s store too and start the copy with `--store`.
+
+`new-id` changes `metastore.yaml`, and git or a sync tool carries that change to every copy of the vault. For a second clone of a synced vault on the same machine, start the clone with its own `--store` instead.
+
+Moving a vault isn’t a copy. When the vault recorded in the store no longer exists, or no longer holds the database, the engine follows the vault to its new place.
+
+## Choose how the engine sees file changes
+
+The engine reads hand edits, from an editor, git or a sync tool, as soon as it sees them:
+
+* **`events`**: the operating system reports each change, and the engine reacts within a fraction of a second; use it on a local disk
+* **`poll`**: the engine scans the vault every `--poll-ms` milliseconds; use it where file events don’t arrive, such as network shares and bind mounts from Windows and macOS hosts
+
+A scan of a large vault over a slow share takes seconds. The engine then scans less often, and doesn’t scan before each operation. [Limits](https://datanotes.dev/administration/limits/#keep-a-docker-vault-on-a-linux-volume) explains this, and how to keep a large vault on a Linux volume.
+
+In both modes, the engine reads a note you rename or move as a rename, and [repairs the links to it](https://datanotes.dev/start/how-it-works/#the-engine-repairs-links-after-a-hand-move).
+
+## Run one engine per database
+
+While it runs, the engine holds a lock file, `engine.lock`, in its store, with its process, host and port. A second engine with the same store refuses to start while a running engine holds it. Tools such as `datanotes status` and `datanotes mcp --database` find the engine through the lock.
+
+On the same machine, the engine checks the lock’s process:
+
+* a lock left by a process that is gone doesn’t block the next start, even with a fresh heartbeat
+* a live process holds the lock, even when it hangs: the next start refuses and changes nothing
+* a lock written before the machine started is left over
+
+If the refusal names a process that isn’t a Datanotes engine, delete the `engine.lock` it names and start again.
+
+Two engines started at once on the same vault take turns. The first writes the database’s id. The second waits up to 60 seconds, then refuses because the first holds the database.
+
+The lock lives in the store, not in the database folder. So it refuses only a second engine that uses the same store. An engine with a store of its own doesn’t see it and starts, and the two engines then lose each other’s writes. That happens in three cases:
+
+* an engine started with another `--store` or `DATANOTES_STORE`
+* the engine in Docker next to an engine on the host, on the same folder
+* engines on two machines that sync the same vault
+
+Run one engine per database folder. Never run the host engine and the Docker engine on the same folder. [Sync the vault between devices](https://datanotes.dev/guides/data-quality/sync-between-devices/) shows how to run one engine on a synced vault.
+
+## Keep the engine local
+
+The engine listens on `127.0.0.1` unless you start it with `--host`, and it answers no Cross-Origin Resource Sharing (CORS) preflight, so web pages can’t call it. To reach it from another device, put a reverse proxy or a tunnel in front of it, over HTTPS, as the [security model](https://datanotes.dev/guides/security/security-model/) explains, and give each device its own [named token](https://datanotes.dev/guides/security/tokens/).
+
+## Stop the engine
+
+Stop the engine with `datanotes stop`, from any terminal of the same machine:
+
+```bash
+npx -y datanotes stop --database your_database_folder
+```
+
+The engine then stops gracefully:
+
+1. It answers new requests with status 503, and finishes the requests under way, for up to 30 seconds.
+2. It records the hand edits it has seen, and stops writing.
+3. The store backup makes its last commit and push, for up to 15 seconds.
+4. Git sync makes its last commit and push, for up to 10 seconds.
+5. It releases the lock and exits.
+
+The engine holds its lock until git is done, so no other engine starts on the store meanwhile. A git step that runs out of time is stopped, and the next start makes it. The whole stop fits in the 60 seconds that `datanotes stop` waits.
+
+`datanotes stop` waits for the engine to exit, for up to 60 seconds; `--timeout-s` changes the wait. It finds the engine as [`datanotes status`](https://datanotes.dev/administration/status/) does, and calls it with the owner token. With `--url` and `--token`, it stops the engine at that address. `--json` prints the report as JSON.
+
+The exit code says what happened:
+
+| Code | Meaning                                                                                                                     |
+| ---- | --------------------------------------------------------------------------------------------------------------------------- |
+| `0`  | The engine stopped                                                                                                          |
+| `1`  | The engine didn’t stop: it didn’t answer, or still runs after the timeout                                                   |
+| `2`  | No engine runs on the folder                                                                                                |
+| `3`  | The engine runs but `datanotes stop` couldn’t ask it: it runs on another machine, is still starting, or refused the request |
+
+Ctrl+C and the `SIGINT`, `SIGTERM` and `SIGBREAK` signals stop the engine the same way. A program can call the route itself, `POST /datanotes/stop/`, with the owner token. The route takes POST only: a `GET` gets status 405 and stops nothing. A named token can’t stop the engine.
+
+A forced stop, such as `kill -9` or `Stop-Process -Force`, has two costs:
+
+* **Git**: it skips the last commit and push. The engine makes them at its next start.
+* **Writes**: a write it interrupts may stay half applied. The engine writes the notes first and records the write in the journal last, so `undo_migration` can’t revert that part. Run `check_table` after the next start to find the rows it left.
+
+So use `datanotes stop` first, with a longer `--timeout-s` when the engine is busy. Outside Windows, `kill <pid>` sends `SIGTERM`, which stops the engine gracefully too. Force the stop only as a last resort.
+
+## Stop the engine as a Windows service
+
+Many service wrappers on Windows stop a service by ending its process, so the engine never makes its last push. Give the service a stop script instead. The script calls `datanotes stop`, and ends the process only when the engine doesn’t exit in time. Save it as `stop-datanotes.ps1`:
+
+```powershell
+# Stop the engine gracefully; end its process after the timeout.
+param(
+  [string]$Database = "C:\Notes",
+  [int]$Timeout = 60
+)
+$out = npx -y datanotes stop --database $Database `
+  --timeout-s $Timeout --json
+$report = $out | Out-String | ConvertFrom-Json
+if ($report.exit_code -eq 1 -and $report.pid) {
+  Stop-Process -Id $report.pid -Force
+}
+exit 0
+```
+
+Set the script as the stop command of your service wrapper, and give the wrapper a stop timeout longer than `$Timeout`. Exit code 2 needs nothing more: no engine runs.

package/dist/docs/administration/status.md

@@ -0,0 +1,61 @@
+# Check the engine’s status
+
+> datanotes status and POST /datanotes/status/: the engine’s health at a glance, with exit codes for scripts and monitors.
+
+The engine reports its health in one place: its version and uptime, the database, git sync, the store backup, the change feed, webhooks, tokens and the hand edits that break the rules. You read it from a shell with `datanotes status`, or from a program with `POST /datanotes/status/`.
+
+## Check the status from a shell
+
+`datanotes status` prints the engine’s health report. It finds the engine through the lock file in the engine’s store, `engine.lock`, and calls the route with the owner token from the settings file. Give it the same folder and option as `datanotes serve`, `--database` or `--vault`. For a store named with `--store`, give `--store` here too:
+
+```bash
+npx datanotes status --database your_database_folder
+```
+
+The report looks like this. Add `--json` to get the whole object instead:
+
+```text
+datanotes status: needs attention
+engine     3.0.0, pid 8124 on nas, up 3d 4h, http://127.0.0.1:27150, watch events
+vault      /srv/vault (settings /home/me/.local/share/datanotes/01a1…/settings.json)
+store      /home/me/.local/share/datanotes/01a1…, database Datanotes (id 01a1…)
+database   datanotes/metastore.yaml: 14 tables (6 with history), 3 views
+journal    42 operations in the last 24 h; last update_rows at 2026-10-09T08:12:03.120Z by claude-code / claude-opus-5-5
+git sync   on, repo /srv/vault, last commit 2026-10-09T08:13:10.002Z, last push 2026-10-09T08:13:12.441Z, 0 pending operations
+backup     on, every 60 s, remote git@github.com:me/vault-store.git, last commit 3f9c2a1 at 2026-10-09T08:13:00.120Z, last push 2026-10-09T08:13:01.020Z, 0 pending operations, 0 commits to push
+changes    feed running, last seq 18342
+webhooks   2 (1 failing)
+  ! wh_3fa9c1d2e4b5 hooks.example.com: 4 failed attempts, retried with backoff; last error: HTTP 502
+tokens     phone (write main.public.film); assistant (write *, ddl, maintain)
+hand edits 3 hand edits in the last 24 h, none breaking the tables' rules
+logs       files /home/me/.local/share/datanotes/01a1…/logs
+problems:
+  - webhook wh_3fa9c1d2e4b5 (hooks.example.com) failing: HTTP 502
+```
+
+The exit code suits scripts and monitors:
+
+* **0**: the engine is healthy
+* **1**: something needs attention, such as git sync blocked, a failing store backup push, a failing webhook, the change feed not running, hand edits breaking the rules or a refused token
+* **2**: no engine runs on the folder; the command then shows what the files say: the lock, stale or held by which process, the settings file and the metastore
+* **3**: an engine runs, but its status couldn’t be read: no owner token, or the engine refused or failed the request
+
+To check an engine on another machine, pass `--url` and `--token`.
+
+## Read the status over HTTP
+
+`POST /datanotes/status/` returns the same report as JSON. Only the owner token can call it: a named token gets 403, even with `maintain`. The report never contains a token or a secret. It covers:
+
+* **Engine**: version, process ID, uptime, address, watch mode and folders
+* **Database**: the tables, how many keep history, the operations of the last 24 hours and the latest one
+* **Git sync**: the last commit and push, the operations waiting, and `blocked` when a conflict stops it
+* **Store backup**: in `store_backup`, the cadence, the remote without credentials, the last commit and push, what waits, and the last error; see [Back up the history](https://datanotes.dev/guides/history/back-up/#check-the-backup)
+* **Change feed**: whether it runs, or why not
+* **Webhooks**: each failing one, with its URL’s host and last error
+* **Tokens**: each named token with its permissions and when it was last used
+* **Hand edits**: the edits of the last 24 hours that leave rows breaking the tables’ rules
+* **Logs**: where the engine writes them
+
+`healthy` is `false` when `problems` lists something.
+
+The engine checks nothing again to answer: it judges hand edits by the checks the change feed ran when it recorded them. A row counts while its latest change is a hand edit that broke a rule. The next change of the row, by hand or through the engine, clears it. To check a whole table, use [`check_table`](https://datanotes.dev/reference/operations/auxiliary/#check_table).