datanotes 3.0.0 → 3.1.0

package/dist/docs/administration/limits.md

@@ -6,21 +6,86 @@ The engine keeps every row in a Markdown note, so its speed depends on the numbe
 
 ## Measured times
 
-We measured the engine on databases of 10,000 and 50,000 notes. On Windows 11, Microsoft Defender scanned the files. On Linux, the engine ran in a container with the notes on a Linux volume.
+We measured Datanotes 3.1.0 on synthetic databases of 10,000 and 50,000 notes, made with `bench/scale/`. One machine ran every test:
 
-| Measure                                 | 10,000 notes | 50,000 notes |
-| --------------------------------------- | ------------ | ------------ |
-| Start, Windows 11 with Defender         | 15 to 18 s   | 63 to 115 s  |
-| Start, Linux container                  | 4 to 5 s     | 22 s         |
-| `select_rows` with `where`              | 30 to 35 ms  | 178 ms       |
-| `insert_row`                            | 91 to 128 ms | 414 ms       |
-| Memory (resident set size) after writes | Not measured | Up to 2 GB   |
+* Windows 11 Pro on an Intel Core i7-13700HX, with 32 GB of memory and an NVMe SSD
+* Node.js 24.21
+* Microsoft Defender scanning every file, with no exclusion
+* Other programs running at the same time
 
-These numbers set the size the engine handles well:
+The times are medians. The first start reads every note and builds the [vault index](#start-faster-with-the-vault-index). A restart reads only the notes that changed.
 
-* Up to about 10,000 notes, it starts within 20 seconds and answers within a few hundred milliseconds
-* From 10,000 to 20,000 notes, start it before your agents and keep it running
-* Above about 20,000 notes, run it as a service on Linux, and give it 2 GB of memory
+| Measure                                               | 10,000 notes | 50,000 notes |
+| ----------------------------------------------------- | ------------ | ------------ |
+| First start                                           | 2.7 s        | 15 s         |
+| Restart                                               | 0.7 s        | 2.0 to 2.3 s |
+| First `select_rows` after a restart                   | 0.15 s       | 1.1 to 2 s   |
+| Background work after a restart                       | 0.6 s        | 4 to 6 s     |
+| Memory (resident set size) after a restart            | 130 MB       | 330 MB       |
+| Memory (resident set size) after a series of searches | 150 MB       | 460 MB       |
+| Vault index on disk                                   | 21 MB        | 104 MB       |
+
+We measured the memory once the engine was idle, after a garbage collection, with the read cache off.
+
+At 50,000 notes, the operations took these times:
+
+| Operation                                      | Median time |
+| ---------------------------------------------- | ----------- |
+| `insert_row`                                   | 37 ms       |
+| `update_row`                                   | 47 ms       |
+| `select_rows` on a table of 117 rows           | 2.8 ms      |
+| `select_rows` of 5 notes by path               | 0.3 ms      |
+| `describe_database`                            | 24 ms       |
+| `graph_neighbors` right after a write          | 2.5 ms      |
+| `search_notes`, two words found in 1,213 notes | 14 ms       |
+| `search_notes`, a person’s name                | 7 ms        |
+| `search_notes`, a word found in 15,905 notes   | 140 ms      |
+
+The first `select_rows` waits until the engine has compared the vault with its index. These numbers set the size the engine handles well:
+
+* Up to about 10,000 notes, it starts within 3 seconds and answers within a few hundred milliseconds
+* From 10,000 to 50,000 notes, start it before your agents and keep it running
+* Above about 20,000 notes, run it as a service, and give it 2 GB of memory
+
+## Start faster with the vault index
+
+The engine keeps a vault index in its store, in `index/vault.sqlite`. It records what the engine read from each note: its properties, tags, links and text. It also indexes the words of each note, so `search_notes` reads only the notes that can match. It keeps each text compressed. At the next start, the engine reads only the notes that changed since.
+
+* A note counts as unchanged while its size, its modification time and its change time stay the same.
+* The change time moves with every write, also one that restores the old modification time, as `cp -p` and `rsync -t` do.
+* A note written within 2 seconds of being recorded is read again. Some file systems keep times to the second only.
+* A note you moved while the engine was stopped is found by its file id, or by its content.
+
+The index is a cache, never the source of truth. Keep these points in mind:
+
+* You can delete it at any time. The next start builds it again.
+* The store backup leaves it out, and so does Time Machine on macOS.
+* It needs Node.js 22.13 or later. On an older Node.js, every start reads every note.
+* After each start, the engine checks the index in the background, one table at a time.
+* When SQLite or that check finds damage, the engine stops using the index until the next start rebuilds it.
+* A query with only one- or two-letter words reads every note.
+
+To turn it off, set `vaultIndexEnabled` to `false` in the [engine’s settings](https://datanotes.dev/administration/configuration/). The engine then deletes the index, and every start reads every note.
+
+## Run a large database on Windows
+
+Microsoft Defender scans every file the engine opens. On a large vault, these scans take most of the start. The engine marks its store as not content indexed, so Windows Search skips it. It never changes your antivirus settings.
+
+To speed up the engine, choose one of these options:
+
+* Exclude the store folder from real-time scanning. Run this in a PowerShell window opened as administrator:
+
+  ```powershell
+  Add-MpPreference -ExclusionPath "$env:LOCALAPPDATA\Datanotes"
+  ```
+
+* Exclude the vault folder too, but only when its notes come from sources you trust.
+
+* Put the vault and the data home on a Dev Drive. Its performance mode defers the scans.
+
+Never exclude the `node.exe` process: that exempts every Node.js program on the machine.
+
+Keep the store out of OneDrive, Dropbox and iCloud folders. Sync clients and scanners lock files, and the engine then retries its renames. On macOS, the engine writes a `.metadata_never_index` file in its store, so Spotlight skips it. It also keeps the vault index out of Time Machine, since the engine can rebuild it. Time Machine still backs up the rest of the store.
 
 ## Start the engine before the agent
 
@@ -67,3 +132,22 @@ volumes:
 ```
 
 On a Linux host, a bind mount delivers file events: set `DATANOTES_WATCH=events` there too. [Run in Docker](https://datanotes.dev/administration/docker/) explains the container.
+
+## Check the engine’s indexes
+
+The engine keeps indexes of the vault in memory, so a read does not go through every note. It learns each change from the vault’s file events: its own writes, your edits, renames and deletes. It never trusts them over the notes, which stay the source of truth.
+
+When an answer looks wrong after many changes outside the engine, check the indexes. Start the engine with `DATANOTES_INDEX_CHECK=1`:
+
+```bash
+DATANOTES_INDEX_CHECK=1 \
+  npx -y datanotes serve --database your_database_folder
+```
+
+After every operation, the engine then compares its indexes with a full read of the vault:
+
+* A difference shows as an error in the engine’s log, with the notes it concerns.
+* The engine then reads the vault again, so the next operations answer right.
+* Each comparison reads every note, so keep the check for debugging, never in daily use.
+
+Every 10 minutes, once no write ran for a minute, the engine also compares the transaction log’s view of the rows with the vault. A difference shows as a warning, and the engine logs the changes it missed.

package/dist/docs/administration/run-the-service.md

@@ -22,6 +22,24 @@ The first start creates the engine’s store and its settings file, `settings.js
 npx -y datanotes store path --database your_database_folder
 ```
 
+Add `--json` to get the answer as an object:
+
+| Field             | Meaning                                                                       |
+| ----------------- | ----------------------------------------------------------------------------- |
+| `vault`           | The folder you gave, as an absolute path                                      |
+| `store`           | The store folder                                                              |
+| `database_id`     | The database’s id                                                             |
+| `database_folder` | The database folder, relative to the folder you gave; empty with `--database` |
+| `database_path`   | The database folder on disk, with `metastore.yaml` at its root                |
+
+Every field but `vault` is `null` when the engine cannot tell where the store would be. That happens only when all three hold:
+
+* You pass no `--store`, and `DATANOTES_STORE` is not set.
+* This machine never ran the engine on the folder.
+* `metastore.yaml` has no id yet.
+
+With `--store`, only `database_id` can be `null`: neither the store nor `metastore.yaml` records the id.
+
 To run the engine in a container, follow [Run in Docker](https://datanotes.dev/administration/docker/).
 
 ## Where the engine keeps its store

package/dist/docs/api/operations/graph_neighbors.md

@@ -5,8 +5,10 @@
 Direct neighbours of notes: what they link to (`out`) and what links to them (`in`,
 backlinks), with counts per edge label. `direction`: out, in or both (default).
 
-`from_nodes` selects the notes: `{"paths": [...]}` and/or `{"table": ..., "where": {...}}`.
-Edges are labelled with their link column (`people`, `event`, …) or `link` for other links.
+`from_nodes` selects the notes: `{"paths": [...]}`, `{"table": ..., "where": {...}}` and/or
+`{"tables": [...]}`: the rows of several tables, by name or pattern (`["*"]` every table, `["personal.*"]` a
+catalog's). The whole vault's graph in one call: `{"tables": ["*"]}` with direction `out` (at most 5000
+start notes). Edges are labelled with their link column (`people`, `event`, …) or `link` for other links.
 
 ## Request body
 

package/dist/docs/api/operations/graph_path.md

@@ -1,9 +1,9 @@
-# Shortest chain of links between two sets of notes (each `{"paths": [...]}` and/or
+# Shortest chain of links between two sets of notes (each `{"paths": [...]}`, `{"table", "where"}`
 
 `POST /datanotes/operations/graph_path/` (operation `graph_path`)
 
-Shortest chain of links between two sets of notes (each `{"paths": [...]}` and/or
-`{"table", "where"}`): `found`, `length` and `path`: the steps (`from`, `label`, `direction`, `to`).
+Shortest chain of links between two sets of notes (each `{"paths": [...]}`, `{"table", "where"}`
+and/or `{"tables": [...]}`): `found`, `length` and `path`: the steps (`from`, `label`, `direction`, `to`).
 `direction` both by default; `max_depth` default 6 (max 10).
 
 ## Request body

package/dist/docs/api/operations/graph_traverse.md

@@ -1,8 +1,8 @@
-# Breadth-first walk over links from `from_nodes` (`{"paths": [...]}` and/or `{"table", "where"}`).
+# Breadth-first walk over links from `from_nodes` (`{"paths": [...]}`, `{"table", "where"}` and/or `{"tables": [...]}`).
 
 `POST /datanotes/operations/graph_traverse/` (operation `graph_traverse`)
 
-Breadth-first walk over links from `from_nodes` (`{"paths": [...]}` and/or `{"table", "where"}`).
+Breadth-first walk over links from `from_nodes` (`{"paths": [...]}`, `{"table", "where"}` and/or `{"tables": [...]}`).
 
 - `direction`: out (default, follow links), in (follow backlinks) or both.
 - `edges`: edge labels to follow (link columns such as `people`, `event`, or `link`).

package/dist/docs/api/operations/record_tool_call.md

@@ -23,6 +23,7 @@
 | `session` | `string` | no |  |  |
 | `result_chars` | `number` | no |  |  |
 | `result_tokens` | `number` | no |  |  |
+| `summary` | `object` | no |  | The result's summary, when `value` is sent already cut: the engine records it (the answer says `takes_summary`). |
 
 ## Responses
 

package/dist/docs/api/operations/tags/queries.md

@@ -11,5 +11,5 @@ Operations that read rows and notes: select_rows, link targets, text and semanti
 - [The rows a link column may point to, for one row: what an agent or a picker offers before writing](https://datanotes.dev/api/operations/list_link_targets/): `POST /datanotes/operations/list_link_targets/`
 - [Find notes by meaning, not exact words: `{query, results: [{path, score, heading, snippet,](https://datanotes.dev/api/operations/semantic_search/): `POST /datanotes/operations/semantic_search/`
 - [Direct neighbours of notes: what they link to (`out`) and what links to them (`in`,](https://datanotes.dev/api/operations/graph_neighbors/): `POST /datanotes/operations/graph_neighbors/`
-- [Breadth-first walk over links from `from_nodes` (`{"paths": [...]}` and/or `{"table", "where"}`).](https://datanotes.dev/api/operations/graph_traverse/): `POST /datanotes/operations/graph_traverse/`
-- [Shortest chain of links between two sets of notes (each `{"paths": [...]}` and/or](https://datanotes.dev/api/operations/graph_path/): `POST /datanotes/operations/graph_path/`
+- [Breadth-first walk over links from `from_nodes` (`{"paths": [...]}`, `{"table", "where"}` and/or `{"tables": [...]}`).](https://datanotes.dev/api/operations/graph_traverse/): `POST /datanotes/operations/graph_traverse/`
+- [Shortest chain of links between two sets of notes (each `{"paths": [...]}`, `{"table", "where"}`](https://datanotes.dev/api/operations/graph_path/): `POST /datanotes/operations/graph_path/`

package/dist/docs/api.md

@@ -1,6 +1,6 @@
 # Datanotes HTTP API
 
-> OpenAPI 3.1.0 description of the Datanotes 3.0.0 HTTP API: https://datanotes.dev/openapi.json
+> OpenAPI 3.1.0 description of the Datanotes 3.1.0 HTTP API: https://datanotes.dev/openapi.json
 
 Database of Markdown notes: tables of notes, rows, queries, history, a change feed and the vault's notes.
 
@@ -69,8 +69,8 @@ Operations that read rows and notes: select_rows, link targets, text and semanti
 - [The rows a link column may point to, for one row: what an agent or a picker offers before writing](https://datanotes.dev/api/operations/list_link_targets/): `POST /datanotes/operations/list_link_targets/`
 - [Find notes by meaning, not exact words: `{query, results: [{path, score, heading, snippet,](https://datanotes.dev/api/operations/semantic_search/): `POST /datanotes/operations/semantic_search/`
 - [Direct neighbours of notes: what they link to (`out`) and what links to them (`in`,](https://datanotes.dev/api/operations/graph_neighbors/): `POST /datanotes/operations/graph_neighbors/`
-- [Breadth-first walk over links from `from_nodes` (`{"paths": [...]}` and/or `{"table", "where"}`).](https://datanotes.dev/api/operations/graph_traverse/): `POST /datanotes/operations/graph_traverse/`
-- [Shortest chain of links between two sets of notes (each `{"paths": [...]}` and/or](https://datanotes.dev/api/operations/graph_path/): `POST /datanotes/operations/graph_path/`
+- [Breadth-first walk over links from `from_nodes` (`{"paths": [...]}`, `{"table", "where"}` and/or `{"tables": [...]}`).](https://datanotes.dev/api/operations/graph_traverse/): `POST /datanotes/operations/graph_traverse/`
+- [Shortest chain of links between two sets of notes (each `{"paths": [...]}`, `{"table", "where"}`](https://datanotes.dev/api/operations/graph_path/): `POST /datanotes/operations/graph_path/`
 
 ### History
 

package/dist/docs/develop/concurrency.md

@@ -24,7 +24,42 @@ Every client goes through that engine:
 
 ## Writes run one at a time
 
-The engine applies writes one at a time, under the lock of its transaction log. Reads don’t wait for that lock: they run in parallel, with each other and with writes.
+The engine runs writes one at a time, in the order they arrive, under the lock of its transaction log. Reads run in parallel with each other.
+
+A write that still runs after a minute lets the next write start, so one stuck write can’t hold up the rest. The engine logs a warning that names the operation.
+
+## Reads wait for writes
+
+The engine runs on one thread. Under a stream of reads, a write would share that thread with every read that arrives while it runs, and take several times longer. So writes go first:
+
+* **A write runs or waits**: a read that arrives waits until no write is left.
+* **A read already under way**: it finishes normally. The write doesn’t wait for it.
+* **A read waits at most `writePriorityMaxWaitMs`**: 100 ms by default. It also starts once 8 writes started after it came. A burst of writes never starves the reads.
+* **A long poll of the change feed**: `list_changes` with `wait_ms` never waits, and never holds a write.
+
+What each operation does is unchanged: a read sees every write that ended before it started. Set `writePriority` to `false` in the [settings](https://datanotes.dev/reference/settings/) to let reads and writes start in the order they arrive. The span `engine.schedule.wait` shows the wait in [traces](https://datanotes.dev/develop/telemetry/#the-spans-the-engine-sends).
+
+## Repeated reads come from memory
+
+A read you send again with the same arguments, with the same token, often has the same answer. The engine keeps the answers of recent reads in memory, and sends the kept answer while nothing it read has changed:
+
+* **A write**: a read of the tables it changed is computed again. Reads of other tables keep their answers.
+* **Undo, a hand edit on disk, a sync tool**: they count as writes. The engine sees them before it answers.
+* **A schema change**: every answer is computed again.
+* **A view**: it depends on the tables its query reads.
+
+Some reads depend on less than a table, or on more:
+
+* **`read_note`**: it depends on its note. Deleting or renaming a folder also counts.
+* **`list_folder`**: it depends on the notes and folders created, deleted or renamed in its folder. Deleting or renaming any folder also counts.
+* **`list_tables`**: it depends on the schema only. A write to a table keeps its answer.
+* **`describe_database`**: it depends on every table, and on the notes whose properties can’t be read.
+* **A filter through a link column**: a key such as `project.status` follows the link column `project`. The read also depends on the tables the link leads to.
+* **Reads that follow links or search the text**: `expand`, `within`, the graph operations and `search_notes` depend on the whole vault.
+
+Reads that compare with the current time (`{now}`) are never kept. A read with `{today}` is kept 15 minutes at most. Reads of history, of the change feed and of `system.information_schema` are never kept either.
+
+`readCacheMaxBytes` sets the memory for the answers: 33554432 bytes (32 MiB) by default. The least recently used answers go first. Set it to `0` to turn the cache off.
 
 The engine checks each write against the database as the writes before it left it. A write is checked first, then applied. If another write is applied while the engine checks yours, the engine checks yours again on the new state. It tries up to 5 times before it gives up.
 

package/dist/docs/develop/http-api.md

@@ -32,7 +32,7 @@ An operation’s body takes the same arguments as the agent’s tool of the same
 * **[`list_folder`](https://datanotes.dev/reference/operations/queries/#list_folder)**: the notes and folders in a `folder`
 * **[`search_notes`](https://datanotes.dev/reference/operations/queries/#search_notes)**: the notes that contain a `query`
 
-Every route and field keeps working in every 3.x version: see [Compatibility](https://datanotes.dev/reference/compatibility/).
+Routes and fields follow semantic versioning: see [Compatibility](https://datanotes.dev/reference/compatibility/).
 
 ## Record who makes each write
 
@@ -72,6 +72,16 @@ Several clients can write at the same time. [Concurrent writes](https://datanote
 
 The engine binds to `127.0.0.1` unless you start it with `--host`. It answers no Cross-Origin Resource Sharing (CORS) preflight, so web pages can’t call it. To reach it from another device, follow the [security model](https://datanotes.dev/guides/security/security-model/).
 
+## Get large answers compressed
+
+A client on another machine gets answers of 16 KB or more compressed with gzip. It asks for that with the `Accept-Encoding: gzip` header, as most HTTP clients do. A client on the engine’s machine gets them as they are: compressing would cost the engine more time than it saves.
+
+The `httpCompression` [setting](https://datanotes.dev/reference/settings/) changes this:
+
+* **`remote`**: clients on other machines only (the default).
+* **`always`**: this machine’s clients too.
+* **`off`**: no compression.
+
 ## Generate a client or explore the routes
 
 The [HTTP API reference](https://datanotes.dev/api/) documents every route, generated from the same OpenAPI description, [openapi.json](https://datanotes.dev/openapi.json). From JavaScript, use the [JavaScript client](https://datanotes.dev/develop/client/). Agents use the [Model Context Protocol (MCP) server](https://datanotes.dev/start/connect-agents/).

package/dist/docs/develop/telemetry.md

@@ -1,77 +1,276 @@
-# Send logs to your observability backend
+# Send logs and traces to your observability backend
 
-> Send the logs of the engine, the MCP server and the apps to any OpenTelemetry backend.
+> Send the logs and traces of the engine, the MCP server and the apps to any OpenTelemetry backend, and find slow operations.
 
-Every part of Datanotes sends its logs with OpenTelemetry, the vendor-neutral standard, so any backend with an OpenTelemetry Protocol (OTLP) endpoint can receive them. This page shows how to point the engine, the Model Context Protocol (MCP) server and the apps at your backend. It also shows how to follow one agent call across all three. Datanotes has no vendor-specific code.
+Every part of Datanotes sends logs and traces with OpenTelemetry, the vendor-neutral standard. Any backend with an OpenTelemetry Protocol (OTLP) endpoint can receive them. This page shows you how to:
+
+* Point the engine, the Model Context Protocol (MCP) server and the apps at your backend
+* Read the spans the engine sends, and what each one measures
+* Follow one agent call across the MCP server and the engine
+* Find the operations that are slow, and why
+
+Datanotes has no vendor-specific code.
 
 ## What Datanotes sends and where
 
-Datanotes sends logs only, over HTTP, to `your_endpoint/v1/logs`. It encodes them as `http/protobuf`, the OpenTelemetry default, or as `http/json`. Any OpenTelemetry Collector receives them, and so do these backends, among others:
+Datanotes sends two signals over HTTP:
+
+* **Logs** to `your_endpoint/v1/logs`
+* **Traces** to `your_endpoint/v1/traces`
+
+It encodes them as `http/protobuf`, the OpenTelemetry default, or as `http/json`. Any OpenTelemetry Collector receives them, and so do these backends, among others:
 
 * SigNoz
-* Grafana: Alloy, Loki or Grafana Cloud
+* Grafana: Alloy, Tempo, Loki or Grafana Cloud
 * Elastic, Datadog, Honeycomb, New Relic and Dynatrace
 
-Datanotes sends no traces or metrics. For a backend that accepts only gRPC, put an OpenTelemetry Collector in front of it. When the collector doesn’t answer, Datanotes counts the records as lost and keeps working.
+Datanotes sends no metrics: your backend computes rates and latency percentiles from the spans. For a backend that accepts only gRPC, put an OpenTelemetry Collector in front of it. When the collector doesn’t answer, Datanotes counts the records and spans as lost and keeps working.
 
 ## Configure the export with environment variables
 
 The engine, the MCP server and the apps all read the standard OpenTelemetry variables:
 
-| Variable                                              | Meaning                                                                                                |
-| ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------ |
-| `OTEL_EXPORTER_OTLP_ENDPOINT`                         | The collector’s base address, for example `http://127.0.0.1:4318`                                      |
-| `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT`                    | Or the full `…/v1/logs` address                                                                        |
-| `OTEL_EXPORTER_OTLP_HEADERS`                          | `key=value,key2=value2`, with URL-encoded values: the API key or `Authorization` your backend asks for |
-| `OTEL_EXPORTER_OTLP_PROTOCOL`                         | `http/protobuf` (default) or `http/json`                                                               |
-| `OTEL_SERVICE_NAME`                                   | The service name; defaults to `datanotes`, `datanotes-mcp` (for `datanotes mcp`) or the app’s name     |
-| `OTEL_RESOURCE_ATTRIBUTES`                            | Extra resource attributes, for example `deployment.environment=home`                                   |
-| `OTEL_SDK_DISABLED=true` or `OTEL_LOGS_EXPORTER=none` | Turns the export off                                                                                   |
+| Variable                                                 | Meaning                                                                                                       |
+| -------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
+| `OTEL_EXPORTER_OTLP_ENDPOINT`                            | The collector’s base address, for example `http://127.0.0.1:4318`                                             |
+| `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT`                       | Or the full `…/v1/logs` address                                                                               |
+| `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`                     | Or the full `…/v1/traces` address                                                                             |
+| `OTEL_EXPORTER_OTLP_HEADERS`                             | `key=value,key2=value2`, with URL-encoded values: the API key or `Authorization` your backend asks for        |
+| `OTEL_EXPORTER_OTLP_PROTOCOL`                            | `http/protobuf` (default) or `http/json`                                                                      |
+| `OTEL_SERVICE_NAME`                                      | The service name; defaults to `datanotes`, `datanotes-mcp` (for `datanotes mcp`) or the app’s name            |
+| `OTEL_RESOURCE_ATTRIBUTES`                               | Extra resource attributes, for example `team=home`                                                            |
+| `OTEL_TRACES_SAMPLER` and `OTEL_TRACES_SAMPLER_ARG`      | `traceidratio` with a share from 0 to 1, `always_on` or `always_off`, plain or with the `parentbased_` prefix |
+| `OTEL_SDK_DISABLED=true`                                 | Turns all the export off                                                                                      |
+| `OTEL_LOGS_EXPORTER=none` or `OTEL_TRACES_EXPORTER=none` | Turns off the logs or the traces only                                                                         |
 
-Datanotes also reads the `…_LOGS_…` variants of the headers and protocol variables. Without an endpoint, it sends nothing.
+Datanotes also reads the `…_LOGS_…` and `…_TRACES_…` variants of the headers and protocol variables. Without an endpoint, it sends nothing.
 
 ## Configure the engine in its settings file
 
-Instead of the variables, you can configure the engine in its settings file, `settings.json` in its store (`datanotes store path` shows where). A variable you set overrides the matching setting:
+Instead of the variables, you can configure the engine in its settings file, `settings.json` in its store. `datanotes store path` shows where. A variable you set overrides the matching setting:
 
 ```json
 {
   "telemetryEnabled": true,
   "telemetryEndpoint": "http://127.0.0.1:4318",
   "telemetryServiceName": "datanotes",
+  "telemetryEnvironment": "home",
   "telemetryProtocol": "http/protobuf",
-  "telemetryHeaders": {}
+  "telemetryHeaders": {},
+  "telemetryTraces": true,
+  "telemetryTraceSampleRatio": 1,
+  "telemetryMaxSpansPerTrace": 1000,
+  "telemetrySlowSpanMs": 0
 }
 ```
 
-`telemetryHeaders` holds what your backend asks for, such as an API key or `Authorization: Bearer …`. The engine keeps it in the settings file. To keep a key out of that file, set `OTEL_EXPORTER_OTLP_HEADERS` instead.
+Each setting does one thing:
+
+* **`telemetryEnabled`** turns the export of logs and traces on.
+* **`telemetryEnvironment`** names the environment, such as `dev` or `home`. The engine sends it as `deployment.environment.name` and as the older `deployment.environment`, which some backends group by.
+* **`telemetryHeaders`** holds what your backend asks for, such as an API key. To keep a key out of the file, set `OTEL_EXPORTER_OTLP_HEADERS` instead.
+* **`telemetryTraces`** sends the traces too. It is on by default: turning telemetry on gives you both signals.
+* **`telemetryTraceSampleRatio`** keeps that share of new traces, from 0 to 1. The default, 1, keeps every trace. A trace that a caller started follows the caller’s choice.
+* **`telemetryMaxSpansPerTrace`** caps the spans of one trace. The engine drops the rest and counts them in `datanotes.spans_dropped` on the trace’s first span.
+* **`telemetrySlowSpanMs`** logs a request, an operation or a background job as a WARN when it lasts at least that many milliseconds. 0 turns it off.
+
+The engine reads the settings again while it runs, so a change applies within a few seconds. With traces off, tracing costs the engine one check per instrumented call.
 
 ## What the engine logs
 
 The engine logs these events:
 
 * Every HTTP request and operation, with the route, duration, outcome and actor
-* The tool calls MCP servers report, with the tool, duration, result size in characters and tokens, client, model and session
+* The tool calls MCP servers report, with the tool, duration, result size and client
 * Refused requests
 * The semantic index and git sync
 * Errors
 
+A record made during a traced request names its trace. The OTLP record carries the trace and span IDs, so your backend links the log to the trace. The log files carry them as `trace_id` and `span_id`. While a request’s trace is sent, its INFO records become DEBUG records: the span already says what they say.
+
 Whatever the collector does, the engine also writes its records to files in `logs/`, in its store outside the vault. The `fileLogEnabled` setting turns these files on or off, and `logRetentionDays` sets how long the engine keeps them.
 
+## The spans the engine sends
+
+A trace is a tree of spans. Each span measures one piece of work and names its parent. A request to the engine gives this tree:
+
+```text
+POST /datanotes/operations/insert_row/      SERVER: the HTTP request
+└── engine.op insert_row                    the operation
+    ├── engine.schedule.wait                a read waits for writes (only then)
+    ├── engine.settle                       wait for the vault to settle
+    └── engine.insert_row                   the engine's work
+        ├── engine.autolink                 links found in the text
+        ├── engine.checks                   CHECKs and link rules
+        ├── engine.lock.acquire             wait for the write lock
+        └── engine.lock.hold                work under the lock
+            ├── engine.transaction_log      hand edits logged first
+            ├── engine.apply                files written
+            │   └── engine.journal.append   the undo entry
+            └── engine.transaction_log      the operation's commit
+```
+
+A read gives a similar tree under `engine.query.execute`:
+
+* **`engine.query.view`** runs the stored query of a view.
+* **`engine.query.plan`** resolves dates and `within` and compiles the filter.
+* **`engine.query.scan`** reads every row of the table. Its `datanotes.query.plan` is `full_scan`: Datanotes has no indexes, and a file cache spares the reads of unchanged notes.
+* **`engine.query.filter`**, **`engine.query.join`**, **`engine.query.group`**, **`engine.query.sort`** and **`engine.query.project`** follow, as the query needs them.
+
+Other spans cover the rest of the engine’s time:
+
+* **`engine.metastore.parse`** when `metastore.yaml` changed and the engine parses it again
+
+* **`engine.history.read`** for a read as of a version or a time
+
+* **`engine.graph.build`** when a graph operation rebuilds the link graph after a change. Calls that need it at the same time wait for that one build and count it in `datanotes.graph.build_waits`.
+
+* **`engine.change_feed.record`** for the change feed’s events of an operation
+
+* **`POST`** (CLIENT) for each webhook delivery, with `url.full`, `server.address` and `http.response.status_code`
+
+* **`job <name>`** for background work. The jobs are:
+
+  * `job hand_edits_log`
+  * `job adopt_note`
+  * `job schema_reload`
+  * `job semantic_index`
+  * `job store_maintenance`
+  * `job change_feed_prune`
+  * `job change_feed_hand_edits`
+  * `job transaction_log_warmup`
+  * `job auto_recompute`
+  * `job inverse_links`
+  * `job datanotes_start`, the engine’s start
+
+  Background work is always a trace of its own, even when a request saw the change behind it. After a start, `job transaction_log_warmup` reads each table’s transaction log once, one table at a time, so a write waits for one table at most.
+
+### Attributes of the operation span
+
+The `engine.op <operation>` span follows the conventions of database spans. Most backends list these spans as database calls:
+
+| Attribute                                            | Meaning                                                                                                                              |
+| ---------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
+| `db.system.name`, `db.system`                        | `datanotes`                                                                                                                          |
+| `db.operation.name`                                  | The operation, such as `select_rows`                                                                                                 |
+| `db.collection.name`                                 | The table or view the call names                                                                                                     |
+| `db.namespace`                                       | The catalog and schema, when the call gives a full name                                                                              |
+| `db.query.summary`                                   | The operation and its table, such as `select_rows task`                                                                              |
+| `db.response.returned_rows`                          | The rows the call returned                                                                                                           |
+| `datanotes.rows_scanned`                             | The rows the engine read to answer                                                                                                   |
+| `datanotes.notes_changed`, `datanotes.files_created` | What a write changed                                                                                                                 |
+| `datanotes.migration_id`                             | The write’s journal entry, for `undo_migration`                                                                                      |
+| `datanotes.dry_run`                                  | Whether the call was a dry run                                                                                                       |
+| `datanotes.long_poll`                                | `true` for a `list_changes` call that waits for changes                                                                              |
+| `datanotes.schedule.class`                           | `read`, `write` or `free`: how [write priority](https://datanotes.dev/develop/concurrency/#reads-wait-for-writes) treats the call                         |
+| `datanotes.schedule.wait_ms`                         | How long the call waited for writes before it started                                                                                |
+| `datanotes.schedule.released`                        | Why a waiting read started: `writes_done`, `max_wait` or `max_writes`                                                                |
+| `datanotes.read_cache`                               | For a read: `hit` (answered from memory), `miss` or `not_kept` ([read cache](https://datanotes.dev/develop/concurrency/#repeated-reads-come-from-memory)) |
+| `error.type`                                         | Why the call failed or was refused                                                                                                   |
+
+The SERVER span adds `http.request.method`, `http.route`, `http.response.status_code`, `client.address` and `user_agent.original`. It also names the caller: `datanotes.token.name` holds the token’s name, never the token. `datanotes.client.harness` and `datanotes.client.model` hold what the agent says of itself.
+
+### Calls to the vault
+
+Reading a table of thousands of notes means thousands of file reads. The engine doesn’t make a span of each one. It adds them up on the span that made them and on its operation span:
+
+* `datanotes.io.read_count`, `datanotes.io.read_chars` and `datanotes.io.read_ms`
+* The same for `write`, `create`, `list`, `exists`, `rename` and the other calls
+* `datanotes.cache.hits` and `datanotes.cache.misses` of the file cache
+* `datanotes.cache.text_reads` and `datanotes.cache.text_source`: note texts the file cache read again, or took from the [vault index](https://datanotes.dev/administration/limits/#start-faster-with-the-vault-index)
+* `datanotes.validate.rows` and `datanotes.validate.ms`, `datanotes.checks.rows` and `datanotes.checks.ms` for the checks run once per row
+
+`_chars` counts characters, not bytes: the length of the text as JavaScript counts it, in UTF-16 code units. In English Markdown, characters and bytes are about the same. Accents, other alphabets and emoji take more bytes than characters. For `list`, `_chars` counts the files and folders listed. The calls that move no text count 0.
+
+A single call that takes 20 ms or more gets a span of its own, `vault.<call>`, with its `file.path` and `datanotes.io.chars`.
+
+### Errors and refusals
+
+The engine tells failures and refusals apart:
+
+* **A failure** is a bug, a disk error, a write that failed partway or a 5xx answer. Its span has status ERROR and an `exception` event with the type, message and stack.
+* **A refusal** is a call the caller can fix: bad arguments, an unknown table, a broken CHECK. Its span keeps the status unset and sets `error.type`, such as `violation` or `TableNotFoundError`.
+
+An empty answer to a `list_changes` poll sends no trace and no log line: services poll every few seconds.
+
+## Follow one agent call across services
+
+One agent call is one trace across the MCP server and the engine:
+
+1. The MCP server makes a SERVER span, `tools/call <tool>`. It continues the client’s trace when the client sends `traceparent` in the call’s `_meta`.
+2. Each request it sends to the engine is a CLIENT span, child of the tool’s span. The request carries that span as its `traceparent` header.
+3. The engine’s SERVER span continues that trace, and its operation spans follow.
+
+The tool span carries `mcp.method.name`, `gen_ai.operation.name` (`execute_tool`), `gen_ai.tool.name` and `mcp.session.id`. Find the agent call by its tool or session, open the trace, and you see where the time went in both processes.
+
+The engine, the MCP server and the apps also log the same `session` and `tool` for each agent call. `session` is the first 16 characters of the MCP session ID.
+
+## Find the slow operations
+
+Your backend computes the latency of each span name from the traces. To find where the engine spends its time:
+
+1. List the `engine.op <operation>` spans by duration, or open your backend’s database calls view. Leave out the spans with `datanotes.long_poll`: they wait on purpose.
+2. Open a slow trace and compare its children. A long `engine.lock.acquire` means the call waited for another write. A long `engine.lock.hold` means the write itself was slow. A long `engine.schedule.wait` means a read waited for writes to end.
+3. In a read, compare `engine.query.scan` with the rest. A slow scan with many `datanotes.cache.misses` reads many notes from disk. A slow scan with few misses works on a large table: narrow it with `where`, or split the table.
+4. Look at the `datanotes.io.*_ms` attributes of the operation span. They show how much of its time went to the disk.
+5. Set `telemetrySlowSpanMs` to log the slow calls as WARN records, and alert on them.
+
 ## Configure the MCP server and the apps
 
 The MCP server and the apps read the same variables, and each has its own alternative:
 
-* **`datanotes mcp`**: the `--otlp-endpoint your_endpoint` option
+* **`datanotes mcp`**: the `--otlp-endpoint your_endpoint` option, for its logs and traces
 * **Apps**: an `otlp` block in the app’s configuration, `{"endpoint": "http://127.0.0.1:4318", "headers": {}, "serviceName": "…"}`
 
-## Follow one agent call across services
+## Continue a trace from your own program
+
+Your program continues its trace in the engine when it sends the W3C `traceparent` header, and `tracestate` when it has one. `datanotes/client` does it for you with two options of `DatanotesClient`:
+
+* **`trace`** is the trace to continue: a `traceparent` value, a context, or a function that returns the current one.
+* **`tracer`** makes a CLIENT span for each request, child of `trace`, and sends that span as the engine’s parent.
+
+```ts
+import {
+  DatanotesClient,
+  OtlpTraceExporter,
+  Tracer,
+  otlpTracesFromEnv,
+} from "datanotes/client";
+
+
+const otlp = otlpTracesFromEnv(process.env, { serviceName: "my-app" });
+const exporter = otlp ? new OtlpTraceExporter(otlp) : null;
+const tracer = new Tracer({ export: (s) => exporter?.export(s) });
+
+
+const run = tracer.startSpan("import films", { kind: "server" });
+const client = new DatanotesClient({
+  url: "http://127.0.0.1:27150",
+  token: process.env.DATANOTES_TOKEN ?? "",
+  tracer,
+  trace: run.context,
+});
+await client.call("insert_row", {
+  table: "film",
+  values: { title: "Arrival" },
+});
+run.end();
+await exporter?.close();
+```
+
+Without a tracer, pass the incoming header as it is. A call can also name its own trace:
+
+```ts
+await client.call("select_rows", { table: "film" }, undefined, {
+  trace: request.headers.traceparent,
+});
+```
 
-The engine, the MCP server and the apps log the same `session` and `tool` for each agent call. `session` is the first 16 characters of the MCP session ID. Filter by `session` in your backend to see the call in the MCP server, in the app whose tool it reached, and the writes it made in the engine.
+Any HTTP client works too: send `traceparent: 00-<trace id>-<span id>-01` with the request.
 
 ## Send logs from your own program
 
-`datanotes/client` exports the exporter that Datanotes uses, so your own programs can log the same way:
+`datanotes/client` exports the log exporter that Datanotes uses, so your own programs can log the same way:
 
 ```ts
 import { OtlpLogExporter, otlpFromEnv } from "datanotes/client";
@@ -83,4 +282,4 @@ log?.log("INFO", "imported films", { count: 8 });
 await log?.close();
 ```
 
-`otlpFromEnv` reads the variables above and returns `null` when you set no endpoint or turn the export off; the program then logs nothing. Call `close()` before the program exits to send the remaining records.
+`otlpFromEnv` reads the variables above and returns `null` when you set no endpoint or turn the export off. The program then logs nothing. Pass a span’s context as the fourth argument of `log` to link the record to that span. Call `close()` before the program exits to send the remaining records.