datanotes 3.4.1 → 3.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (52) hide show
  1. package/README.md +1 -1
  2. package/dist/api/extension-columns.d.cts +64 -0
  3. package/dist/api/extension-columns.d.ts +64 -0
  4. package/dist/api/extension-plan.d.cts +1 -1
  5. package/dist/api/extension-plan.d.ts +1 -1
  6. package/dist/api/extensions.cjs +1088 -5
  7. package/dist/api/extensions.js +1088 -5
  8. package/dist/api/index.cjs +1088 -5
  9. package/dist/api/index.js +1088 -5
  10. package/dist/api/mcp.cjs +1088 -5
  11. package/dist/api/mcp.js +1088 -5
  12. package/dist/datanotes.cjs +4701 -1665
  13. package/dist/docs/administration/extensions.md +2 -1
  14. package/dist/docs/administration/git-sync.md +17 -0
  15. package/dist/docs/administration/limits.md +29 -0
  16. package/dist/docs/administration/use-with-obsidian.md +12 -0
  17. package/dist/docs/api/operations/alter_table.md +6 -1
  18. package/dist/docs/api/operations/create_table.md +5 -0
  19. package/dist/docs/api/operations/describe_table.md +1 -1
  20. package/dist/docs/api/operations/pack_rows.md +45 -0
  21. package/dist/docs/api/operations/tags/auxiliary.md +2 -0
  22. package/dist/docs/api/operations/unpack_rows.md +40 -0
  23. package/dist/docs/api.md +3 -1
  24. package/dist/docs/develop/write-an-extension.md +8 -2
  25. package/dist/docs/guides/data-quality/check-constraints.md +10 -10
  26. package/dist/docs/guides/data-quality/hand-edits.md +8 -8
  27. package/dist/docs/guides/database-objects/bases.md +11 -11
  28. package/dist/docs/guides/database-objects/catalogs-and-schemas.md +10 -10
  29. package/dist/docs/guides/database-objects/columns.md +41 -41
  30. package/dist/docs/guides/database-objects/packs.md +563 -0
  31. package/dist/docs/guides/database-objects/relations.md +28 -28
  32. package/dist/docs/guides/database-objects/tables.md +28 -28
  33. package/dist/docs/guides/database-objects/views.md +44 -44
  34. package/dist/docs/guides/history/change-data-feed.md +34 -34
  35. package/dist/docs/guides/history/journal-and-undo.md +12 -12
  36. package/dist/docs/guides/history/restore.md +20 -20
  37. package/dist/docs/guides/history/table-history.md +31 -31
  38. package/dist/docs/guides/read-and-write/automatic-links.md +15 -15
  39. package/dist/docs/guides/read-and-write/import-notes.md +21 -21
  40. package/dist/docs/guides/read-and-write/joins-and-aggregates.md +7 -7
  41. package/dist/docs/guides/read-and-write/read-and-filter.md +10 -10
  42. package/dist/docs/guides/read-and-write/write-rows.md +10 -10
  43. package/dist/docs/guides/search/text.md +1 -1
  44. package/dist/docs/reference/cli.md +3 -1
  45. package/dist/docs/reference/operations/auxiliary.md +30 -1
  46. package/dist/docs/reference/operations/queries.md +1 -1
  47. package/dist/docs/reference/operations/rows.md +3 -3
  48. package/dist/docs/reference/operations/schema.md +8 -4
  49. package/dist/docs/reference/operations.md +55 -53
  50. package/dist/docs/reference/settings.md +1 -0
  51. package/dist/docs/release-notes.md +42 -0
  52. package/package.json +1 -1
@@ -89,7 +89,7 @@ Five options carry your choices. They apply to `install` and, except the last tw
89
89
  | `--catalog` | Sets the catalog of created tables and views. | `--catalog work` |
90
90
  | `--schema` | Sets the schema of created owned tables and views. | `--schema reading` |
91
91
 
92
- Each option takes a comma-separated list. `--map role=catalog.schema.table` binds a table. `--map role.column=column` maps a column. A column of the extension’s own table keeps the extension’s name: you cannot map it.
92
+ Each option takes a comma-separated list. `--map role=catalog.schema.table` binds a table. `--map role.column=column` maps a column of a table the extension depends on. The views, CHECKs and link filters of the extension then read your column, through links too. A column of the extension’s own table keeps the extension’s name: you cannot map it.
93
93
 
94
94
  The plan ends with a message for the AI agent that runs the install:
95
95
 
@@ -101,6 +101,7 @@ An agent never guesses. It shows the plan, asks the user, and runs the command a
101
101
  * `ambiguous_dependency`: several tables could be the role. Pass `--map` or `--create`.
102
102
  * `missing_column`: the matched table lacks a column. Map another column, or add it with `--add`.
103
103
  * `column_differs`: a column exists with another type or rule. Map another column or table.
104
+ * `column_map_unsupported`: the extension names a mapped column where the installer cannot rewrite it, such as the outer clauses of a `union` view. Rename your column to the extension’s name and leave the mapping out.
104
105
  * `table_exists`: a table the extension would create exists already. Use it with `--map`, or pick another catalog.
105
106
  * `tool_name_taken`: a tool name is already an operation or a tool of another extension.
106
107
  * `token_exists`: a token named `ext-<name>` exists already.
@@ -66,6 +66,23 @@ A pull can also succeed and still leave files in conflict. This happens when the
66
66
 
67
67
  The next sync commits and pushes again.
68
68
 
69
+ ## Merge packs row by row
70
+
71
+ A [pack of rows](https://datanotes.dev/guides/database-objects/packs/) is one note that holds many rows. Two copies that add rows to the same pack would get a conflict from a plain text merge. So git sync registers a merge driver that merges a pack row by row, by `id`. A conflict then stays in the one row changed on both sides.
72
+
73
+ At each start, git sync writes two things:
74
+
75
+ * **In the repository’s config**: `merge.datanotes-pack.driver`, the engine’s own `datanotes pack-merge` command.
76
+ * **In `.gitattributes`** at the vault’s root: the line `*.md merge=datanotes-pack`. Git sync commits it with the vault.
77
+
78
+ The driver merges any note that isn’t a pack as text, as git does. Every copy that pulls should register the driver too. Start its engine with `--git-sync`, or set it by hand:
79
+
80
+ ```bash
81
+ git config merge.datanotes-pack.name "Datanotes packs of rows"
82
+ git config merge.datanotes-pack.driver \
83
+ "npx -y datanotes pack-merge %O %A %B %P"
84
+ ```
85
+
69
86
  ## Keep every copy on the same version
70
87
 
71
88
  Each copy of the vault that runs an engine reads the notes that git brings in as hand edits: the engine checks them against the tables’ rules and reports what they break on the change feed. Run the same version of Datanotes on every copy, and upgrade them together. The [release notes](https://datanotes.dev/release-notes/) list what each version changes.
@@ -47,6 +47,35 @@ The first `select_rows` waits until the engine has compared the vault with its i
47
47
  * From 10,000 to 50,000 notes, start it before your agents and keep it running
48
48
  * Above about 20,000 notes, run it as a service, and give it 2 GB of memory
49
49
 
50
+ ## Keep fewer files with packs of rows
51
+
52
+ A table with many small rows, such as a log, can keep the rows of each closed month in one note. [Packs of rows](https://datanotes.dev/guides/database-objects/packs/) explains how. We measured 50,000 rows of one table, from 2014 to 2018, as notes of their own and then in 118 monthly packs. The machine:
53
+
54
+ * Linux on an Intel Core i7-13700HX, with 32 GB of memory
55
+ * the vault on a RAM disk (`tmpfs`)
56
+ * Node.js 26.12
57
+
58
+ | Measure | 50,000 notes | 118 packs |
59
+ | --------------------------------------------- | ------------ | --------- |
60
+ | Files in the vault | 50,001 | 119 |
61
+ | First start | 2.9 s | 4.1 s |
62
+ | Restart | 0.9 s | 1.3 s |
63
+ | First `select_rows` after a restart | 0.72 s | 0.24 s |
64
+ | `select_rows` of one month (419 rows) | 1 ms | 12 ms |
65
+ | Memory (resident set size) when idle | 670 MB | 671 MB |
66
+ | Vault index on disk | 48 MB | 46 MB |
67
+ | `git add` and `git commit` of the whole vault | 684 ms | 47 ms |
68
+ | `git status` | 27 ms | 2.7 ms |
69
+
70
+ `pack_rows` packed the 50,000 rows in 35 seconds. A restart reads every pack, but takes the values of a section it read before from the store’s `packs.json`. The first start parses every section, so it takes longer.
71
+
72
+ To measure on your machine, build the engine and run the benchmark on a folder of its own:
73
+
74
+ ```bash
75
+ npm run build:service
76
+ node bench/scale/packs.mjs --rows 50000 --out /tmp/packs-bench
77
+ ```
78
+
50
79
  ## Start faster with the vault index
51
80
 
52
81
  The engine keeps a vault index in its store, in `index/vault.sqlite`. It records what the engine read from each note: its properties, tags, links and text. It also indexes the words of each note, so `search_notes` reads only the notes that can match. It keeps each text compressed. At the next start, the engine reads only the notes that changed since.
@@ -41,6 +41,18 @@ A note at the vault’s root is `intro.md`. Operations take and return paths in
41
41
 
42
42
  With `--database`, paths start from the database folder instead: the same row is `main/public/film/Metropolis.md`. When you open that folder in Obsidian, its root is the vault root, so links resolve there too.
43
43
 
44
+ ## Follow links into packs of rows
45
+
46
+ A table with [packs of rows](https://datanotes.dev/guides/database-objects/packs/) keeps the rows of a closed period in one note, one `##` section per row. Obsidian shows each pack as one note. The engine still lists every row at its own path.
47
+
48
+ A link to a packed row names the pack and the row’s heading:
49
+
50
+ ```markdown
51
+ [[reading 2025-03#2025-03-02T2100]]
52
+ ```
53
+
54
+ Obsidian follows it to the row’s section, and the engine to the row. Inside a row’s text, use headings from `###` down: a `#` or `##` heading would end the row’s section, so such a row stays a note of its own.
55
+
44
56
  ## Move between a vault and a database folder
45
57
 
46
58
  [`configure_database`](https://datanotes.dev/reference/operations/auxiliary/#configure_database) with `root` moves the database within the folder the engine serves. The move rewrites every link in one operation, as for any [move of the database folder](https://datanotes.dev/start/how-it-works/#move-the-vault-or-the-database-folder).
@@ -13,7 +13,10 @@ labels merged into the table's, e.g. `{"domain": "health"}`; a `null` value remo
13
13
  `archive` (folder name inside the table folder) or `fields` (untyped fields, name → default; a
14
14
  `null` value removes one). Rows, archive and transaction log follow; links are updated; renamed
15
15
  rows keep their old name in `aliases`. A change of metadata only (tags, description, aliases,
16
- history, autolink_target, row_template, instructions, fields) rewrites the table entry alone: no row is moved or rewritten. Refused
16
+ history, autolink_target, row_template, instructions, fields, pack_after_days) rewrites the table entry alone: no row is moved or rewritten.
17
+ `pack` sets the period of the table's packs of rows (`"{occurred_at:YYYY-MM}"`; "" removes it), `pack_after_days` the days
18
+ after a period's end before its rows go into a pack ("default": 7). While rows are in packs, changing `pack`,
19
+ `partition`, `filename` or the table's name is refused: unpack_rows first. Refused
17
20
  when a row cannot fill a pattern. Renaming or moving the table rewrites what names it in other
18
21
  definitions: the `ref` and `vocabulary` of the columns of other tables and bases (lists of links
19
22
  included) and the tables of views (`table`, `union[].table`, `join.table`); `schema_references`
@@ -38,6 +41,8 @@ The schema changes only through operations like this one: never edit metastore.y
38
41
  | `schema` | `string \| null` | no | `null` | |
39
42
  | `name` | `string \| null` | no | `null` | |
40
43
  | `partition` | `string \| null` | no | `null` | |
44
+ | `pack` | `string \| null` | no | `null` | Packs of rows: a template of a date column, e.g. "{occurred_at:YYYY-MM}"; "" removes it (only while no row is in a pack). |
45
+ | `pack_after_days` | `integer \| "default" \| null` | no | `null` | Days after the end of a period before its rows go into a pack, or "default" (7). |
41
46
  | `filename` | `string \| null` | no | `null` | |
42
47
  | `description` | `string \| null` | no | `null` | The new description: what a row records (it changes, never goes: "" is refused). |
43
48
  | `aliases` | `string[] \| null` | no | `null` | |
@@ -24,6 +24,9 @@ digits, `_` (no dots: pass the catalog and the schema as their own arguments).
24
24
  description), `partition`, `exclude`, `archive`, `history` (keep its versions; default: the
25
25
  database's setting), `autolink_target` (true / false: may automatic links link its rows? default:
26
26
  yes, unless its rows are named by a date or a time, as with `time_column`).
27
+ `pack` (a template of a date column, such as `"{occurred_at:YYYY-MM}"`: once a period is over, one note
28
+ holds its rows, `##` section by section; select_rows still gives one row each) and `pack_after_days` (default 7):
29
+ see pack_rows.
27
30
  - `row_template`: markdown body of new rows. Shared columns come from bases: `set_table_bases`.
28
31
  - `instructions`: how to fill a row, for agents (Markdown); describe_table returns them.
29
32
  - `tags`: key/value labels of the table, strings (`{"domain": "health"}`); list_tables and
@@ -55,6 +58,8 @@ The schema changes only through operations like this one: never edit metastore.y
55
58
  | `time_column` | `string \| null` | no | `null` | |
56
59
  | `filename` | `string \| null` | no | `null` | File name pattern of the rows, e.g. "{name}" (instead of time_column). |
57
60
  | `partition` | `string \| null` | no | `null` | Sub-folder pattern of the rows, e.g. "{occurred_at:YYYY/MM}"; "" = flat. |
61
+ | `pack` | `string \| null` | no | `null` | Packs of rows: a template of a date column, the period of each pack, e.g. "{occurred_at:YYYY-MM}": once a month is over, one note holds its rows (pack_rows, or the engine on its own). Needs the engine service. |
62
+ | `pack_after_days` | `integer \| null` | no | `null` | Days after the end of a period before its rows go into a pack (default 7). |
58
63
  | `exclude` | `string[] \| null` | no | `null` | Folders whose notes are never rows of the table (e.g. templates that create them). |
59
64
  | `archive` | `string \| null` | no | `null` | Archive folder: a folder name inside the table folder (default: the database's archive_folder). |
60
65
  | `aliases` | `string[] \| null` | no | `null` | Other words for what the table records, for agents. |
@@ -4,7 +4,7 @@
4
4
 
5
5
  Describe a table (`catalog.schema.table` or a shorter unambiguous name): `qualified`, `name`,
6
6
  `catalog`, `schema`, `description`, `aliases`, `layout` (`folder`, `row_folder`: the folder pattern
7
- of new rows, `archive`), `options` (`partition`, `filename`, `archive`, `exclude`), typed `columns`
7
+ of new rows, `archive`), `options` (`partition`, `filename`, `archive`, `exclude`, `pack`, `pack_after_days`), typed `columns`
8
8
  with label/aliases/description and the dense flag, `untyped_fields`, `row_count`, and when set:
9
9
  `tags`, `bases` (the bases it inherits from, in order), `constraints` (its CHECKs, name → where), `constraint_hints` (their own hints, name → hint), `instructions` (how to fill a row: follow them before you write one), `row_template`, `history`,
10
10
  `autolink_target`, `fields` (their values).
@@ -0,0 +1,45 @@
1
+ # Put the rows of a table's closed periods into packs: one note per period (`<table> <key>.md` in the
2
+
3
+ `POST /datanotes/operations/pack_rows/` (operation `pack_rows`)
4
+
5
+ Put the rows of a table's closed periods into packs: one note per period (`<table> <key>.md` in the
6
+ table's folder), a `##` section per row, `key:: value` lines for its values. The table needs a `pack`
7
+ option (create_table / alter_table, e.g. `"{occurred_at:YYYY-MM}"`). A period is closed
8
+ `pack_after_days` days after it ends (default 7); `key` packs only that period.
9
+ Nothing changes for the engine and its readers: select_rows, links, the graph and the change feed
10
+ still see each row at its path, with its values, text and file_modified (no change event, no new
11
+ version). Links from other notes to a packed row become `[[pack#row]]`, which Obsidian follows too.
12
+ A row stays a note of its own (`excluded`, with the reason) when it changed in the last 24 hours, has a
13
+ `#` or `##` heading in its text, is not at the path its values give, or its note is not in the form
14
+ the engine writes. The engine also packs on its own once a day (setting `autoPack`). Needs the engine
15
+ service. Journaled: undo_migration takes the rows out again. Use `dry_run` to see `packs` first.
16
+
17
+ ## Parameters
18
+
19
+ | Name | In | Type | Required | Description |
20
+ | --- | --- | --- | --- | --- |
21
+ | `X-Datanotes-Actor` | header | `string` | no | JSON: on whose behalf the write is made, recorded in the journal (`harness`, `model`, `session`). |
22
+
23
+ ## Request body
24
+
25
+ `application/json`, required
26
+
27
+ | Property | Type | Required | Default | Description |
28
+ | --- | --- | --- | --- | --- |
29
+ | `table` | `string` | yes | | |
30
+ | `key` | `string \| null` | no | `null` | Only the period of this key (e.g. "2026-10"); every closed period when absent. |
31
+ | `dry_run` | `boolean` | no | `false` | |
32
+
33
+ ## Responses
34
+
35
+ | Status | Description | Body |
36
+ | --- | --- | --- |
37
+ | 200 | Done. | `PackRowsResult` (application/json) |
38
+ | 400 | Bad input: the body is not a JSON object, or an argument is unknown, missing or of the wrong type. | `Error` (application/json) |
39
+ | 401 | Missing or wrong bearer token. | `Error` (application/json) |
40
+ | 403 | A named token beyond its permissions (the engine's administration, tables outside its `write` patterns, schema changes without `ddl`). | `Error` (application/json) |
41
+ | 404 | Unknown table, view, migration, note or folder. | `Error` (application/json) |
42
+ | 409 | Ambiguous table name: it matches several tables (`paths` or `tables` lists them). | `AmbiguousName` (application/json) |
43
+ | 500 | Unexpected error. | `Error` (application/json) |
44
+
45
+ The schemas named here are in [openapi.json](https://datanotes.dev/openapi.json) (`components.schemas`).
@@ -12,3 +12,5 @@ Operations that describe and check the database, configure it and maintain its i
12
12
  - [Read-only schema inference from notes' front matter.](https://datanotes.dev/api/operations/infer_table/): `POST /datanotes/operations/infer_table/`
13
13
  - [The semantic search index: `enabled`, `provider`, `model`, `files`, `passages`, `pending`,](https://datanotes.dev/api/operations/describe_semantic_index/): `POST /datanotes/operations/describe_semantic_index/`
14
14
  - [Bring the semantic search index up to date: embed the notes changed since the last run (the](https://datanotes.dev/api/operations/sync_semantic_index/): `POST /datanotes/operations/sync_semantic_index/`
15
+ - [Put the rows of a table's closed periods into packs: one note per period (`<table> <key>.md` in the](https://datanotes.dev/api/operations/pack_rows/): `POST /datanotes/operations/pack_rows/`
16
+ - [Take rows out of their packs as notes of their own, at their paths, with the text and the modification](https://datanotes.dev/api/operations/unpack_rows/): `POST /datanotes/operations/unpack_rows/`
@@ -0,0 +1,40 @@
1
+ # Take rows out of their packs as notes of their own, at their paths, with the text and the modification
2
+
3
+ `POST /datanotes/operations/unpack_rows/` (operation `unpack_rows`)
4
+
5
+ Take rows out of their packs as notes of their own, at their paths, with the text and the modification
6
+ time they had (the opposite of pack_rows): every pack of the table, the pack of `key`, or only the
7
+ rows `where` matches (as select_rows filters). A pack left without rows goes. Links `[[pack#row]]`
8
+ from other notes become links to the notes. Needed before alter_table changes the table's `pack`,
9
+ `partition`, `filename` or name. Silent for the rows, journaled (undo_migration packs them again).
10
+
11
+ ## Parameters
12
+
13
+ | Name | In | Type | Required | Description |
14
+ | --- | --- | --- | --- | --- |
15
+ | `X-Datanotes-Actor` | header | `string` | no | JSON: on whose behalf the write is made, recorded in the journal (`harness`, `model`, `session`). |
16
+
17
+ ## Request body
18
+
19
+ `application/json`, required
20
+
21
+ | Property | Type | Required | Default | Description |
22
+ | --- | --- | --- | --- | --- |
23
+ | `table` | `string` | yes | | |
24
+ | `key` | `string \| null` | no | `null` | Only the pack of this key (e.g. "2026-10"). |
25
+ | `where` | `object \| null` | no | `null` | Only the rows that match, as select_rows' where. |
26
+ | `dry_run` | `boolean` | no | `false` | |
27
+
28
+ ## Responses
29
+
30
+ | Status | Description | Body |
31
+ | --- | --- | --- |
32
+ | 200 | Done. | `PackRowsResult` (application/json) |
33
+ | 400 | Bad input: the body is not a JSON object, or an argument is unknown, missing or of the wrong type. | `Error` (application/json) |
34
+ | 401 | Missing or wrong bearer token. | `Error` (application/json) |
35
+ | 403 | A named token beyond its permissions (the engine's administration, tables outside its `write` patterns, schema changes without `ddl`). | `Error` (application/json) |
36
+ | 404 | Unknown table, view, migration, note or folder. | `Error` (application/json) |
37
+ | 409 | Ambiguous table name: it matches several tables (`paths` or `tables` lists them). | `AmbiguousName` (application/json) |
38
+ | 500 | Unexpected error. | `Error` (application/json) |
39
+
40
+ The schemas named here are in [openapi.json](https://datanotes.dev/openapi.json) (`components.schemas`).
package/dist/docs/api.md CHANGED
@@ -1,6 +1,6 @@
1
1
  # Datanotes HTTP API
2
2
 
3
- > OpenAPI 3.1.0 description of the Datanotes 3.4.1 HTTP API: https://datanotes.dev/openapi.json
3
+ > OpenAPI 3.1.0 description of the Datanotes 3.5.0 HTTP API: https://datanotes.dev/openapi.json
4
4
 
5
5
  Database of Markdown notes: tables of notes, rows, queries, history, a change feed and the vault's notes.
6
6
 
@@ -101,6 +101,8 @@ Operations that describe and check the database, configure it and maintain its i
101
101
  - [Read-only schema inference from notes' front matter.](https://datanotes.dev/api/operations/infer_table/): `POST /datanotes/operations/infer_table/`
102
102
  - [The semantic search index: `enabled`, `provider`, `model`, `files`, `passages`, `pending`,](https://datanotes.dev/api/operations/describe_semantic_index/): `POST /datanotes/operations/describe_semantic_index/`
103
103
  - [Bring the semantic search index up to date: embed the notes changed since the last run (the](https://datanotes.dev/api/operations/sync_semantic_index/): `POST /datanotes/operations/sync_semantic_index/`
104
+ - [Put the rows of a table's closed periods into packs: one note per period (`<table> <key>.md` in the](https://datanotes.dev/api/operations/pack_rows/): `POST /datanotes/operations/pack_rows/`
105
+ - [Take rows out of their packs as notes of their own, at their paths, with the text and the modification](https://datanotes.dev/api/operations/unpack_rows/): `POST /datanotes/operations/unpack_rows/`
104
106
 
105
107
  ### change feed
106
108
 
@@ -197,11 +197,17 @@ The installer compares the manifest with a table it adopts or matches:
197
197
  * Instructions that differ are replaced, and the plan shows it.
198
198
  * Tags are added. Nothing is migrated again.
199
199
 
200
- Version 1 has three limits:
200
+ Users may bind the columns of a dependency to theirs with `--map role.column=column`. Write the manifest and the tools so that a mapping works:
201
+
202
+ * Read and write every dependency column through `ctx.column(role, column)`, never by its manifest name.
203
+ * Name dependency columns in view queries, CHECKs, `ref_where` and `ref_by` freely. The installer rewrites them to the user’s columns, through links too, such as `author.name`.
204
+ * Expect the rows of your views to carry the user’s names. A view with the column `author.name` returns `author.full_name` once `name` is mapped to `full_name`. Aggregate names, such as `n` in `{"n": "count"}`, stay as you wrote them.
205
+ * Keep dependency columns out of the outer clauses of a `union` view and out of migrations. The installer does not rewrite them there: it refuses the mapping with `column_map_unsupported`, or the migration with `migration_conflict`.
206
+
207
+ Version 1 has two limits:
201
208
 
202
209
  * A manifest cannot declare an `inverse` across two tables. Only `ref: "self"` takes `inverse`.
203
210
  * The columns of the extension’s own tables keep the manifest’s names. Users map only dependency columns.
204
- * A dotted key into a dependency, such as `project.kind`, is not mapped in CHECKs and view queries.
205
211
 
206
212
  ## Write the tools
207
213
 
@@ -29,7 +29,7 @@ Answer:
29
29
  "commits": [
30
30
  { "table": "diary.classics.film", "version": 8, "operation": "ADD CONSTRAINT" }
31
31
  ],
32
- "migration_id": "20261024T081203-add_constraint-add-constraint-rated_when_watc"
32
+ "migration_id": "20261026T080029-add_constraint-add-constraint-rated_when_watc"
33
33
  }
34
34
  ```
35
35
 
@@ -86,7 +86,7 @@ Answer:
86
86
  "files_created": 1,
87
87
  "planned_creates": ["datanotes/diary/classics/film/His Girl Friday.md"],
88
88
  "commits": [{ "table": "diary.classics.film", "version": 10, "operation": "WRITE" }],
89
- "migration_id": "20261024T081203-insert_row-insert-into-film-2"
89
+ "migration_id": "20261026T080029-insert_row-insert-into-film-2"
90
90
  }
91
91
  ```
92
92
 
@@ -108,7 +108,7 @@ Answer:
108
108
  "commits": [
109
109
  { "table": "diary.classics.film", "version": 11, "operation": "DROP CONSTRAINT" }
110
110
  ],
111
- "migration_id": "20261024T081203-drop_constraint-drop-constraint-at_most_three_"
111
+ "migration_id": "20261026T080029-drop_constraint-drop-constraint-at_most_three_"
112
112
  }
113
113
  ```
114
114
 
@@ -149,7 +149,7 @@ Answer:
149
149
  "commits": [
150
150
  { "table": "main.public.genre", "version": 4, "operation": "ADD CONSTRAINT" }
151
151
  ],
152
- "migration_id": "20261024T081149-add_constraint-add-constraint-parent_by_level"
152
+ "migration_id": "20261026T080015-add_constraint-add-constraint-parent_by_level"
153
153
  }
154
154
  ```
155
155
 
@@ -228,34 +228,34 @@ Answer:
228
228
  "ok": true,
229
229
  "op": "batch",
230
230
  "dry_run": false,
231
- "migration_id": "20261024T081149-batch-add-the-silent-era-and-slapsti",
231
+ "migration_id": "20261026T080015-batch-add-the-silent-era-and-slapsti",
232
232
  "steps": [
233
233
  {
234
234
  "index": 0,
235
235
  "op": "insert_row",
236
236
  "ok": true,
237
- "migration_id": "20261024T081149-insert_row-insert-into-genre-4",
237
+ "migration_id": "20261026T080015-insert_row-insert-into-genre-4",
238
238
  "result": {
239
239
  "ok": true,
240
240
  "op": "insert_row",
241
241
  "files_created": 1,
242
242
  "planned_creates": ["datanotes/main/public/genre/Silent cinema.md"],
243
243
  "commits": [{ "table": "main.public.genre", "version": 5, "operation": "WRITE" }],
244
- "migration_id": "20261024T081149-insert_row-insert-into-genre-4"
244
+ "migration_id": "20261026T080015-insert_row-insert-into-genre-4"
245
245
  }
246
246
  },
247
247
  {
248
248
  "index": 1,
249
249
  "op": "insert_row",
250
250
  "ok": true,
251
- "migration_id": "20261024T081149-insert_row-insert-into-genre-5",
251
+ "migration_id": "20261026T080015-insert_row-insert-into-genre-5",
252
252
  "result": {
253
253
  "ok": true,
254
254
  "op": "insert_row",
255
255
  "files_created": 1,
256
256
  "planned_creates": ["datanotes/main/public/genre/Slapstick comedy.md"],
257
257
  "commits": [{ "table": "main.public.genre", "version": 6, "operation": "WRITE" }],
258
- "migration_id": "20261024T081149-insert_row-insert-into-genre-5"
258
+ "migration_id": "20261026T080015-insert_row-insert-into-genre-5"
259
259
  }
260
260
  }
261
261
  ]
@@ -343,7 +343,7 @@ Answer:
343
343
  }
344
344
  ],
345
345
  "commits": [{ "table": "main.public.genre", "version": 7, "operation": "UPDATE" }],
346
- "migration_id": "20261024T081149-update_row-update-genre"
346
+ "migration_id": "20261026T080015-update_row-update-genre"
347
347
  }
348
348
  ```
349
349
 
@@ -35,7 +35,7 @@ You edit `datanotes/main/public/genre/Silent film.md` by hand, outside the engin
35
35
  ```markdown
36
36
  ---
37
37
  table: main.public.genre
38
- id: 01a16cf7-4fdb-728b-a68f-7e167b3b5a58
38
+ id: 01a17739-7021-7335-bd91-607920af44cc
39
39
  name: Silent film
40
40
  kind: cinema
41
41
  parent: "[[Cinema]]"
@@ -152,7 +152,7 @@ Answer:
152
152
  }
153
153
  ],
154
154
  "commits": [{ "table": "main.public.genre", "version": 9, "operation": "UPDATE" }],
155
- "migration_id": "20261024T081151-update_row-update-genre"
155
+ "migration_id": "20261026T080016-update_row-update-genre"
156
156
  }
157
157
  ```
158
158
 
@@ -167,7 +167,7 @@ You edit `datanotes/main/public/watch/2026-10-03T2130.md` by hand, outside the e
167
167
  ```markdown
168
168
  ---
169
169
  table: main.public.watch
170
- id: 01a16cf7-481b-7d97-bd70-f521486d2c67
170
+ id: 01a17739-6859-7d40-bcee-bed431f0b5fc
171
171
  film: His Girl Friday
172
172
  occurred_at: 2026-10-03T21:30
173
173
  where: cinema
@@ -226,7 +226,7 @@ Answer:
226
226
  }
227
227
  ],
228
228
  "commits": [{ "table": "main.public.watch", "version": 3, "operation": "RECOMPUTE" }],
229
- "migration_id": "20261024T081149-recompute_columns-recompute-id-day-label-of-watc"
229
+ "migration_id": "20261026T080014-recompute_columns-recompute-id-day-label-of-watc"
230
230
  }
231
231
  ```
232
232
 
@@ -292,7 +292,7 @@ Answer:
292
292
  ```markdown
293
293
  ---
294
294
  table: main.public.watch
295
- id: 01a16cf7-adee-720a-ad18-7e59d48b411b
295
+ id: 01a17739-cae3-7776-973c-b43948a4c694
296
296
  occurred_at: 2026-09-25T21:00
297
297
  film: "[[Charade]]"
298
298
  where: home
@@ -311,7 +311,7 @@ You edit `datanotes/main/public/film/Metropolis.md` by hand, outside the engine.
311
311
  ```markdown
312
312
  ---
313
313
  table: main.public.film
314
- id: 01a16cf7-6b1e-7e74-9fb2-5f094a71a39f
314
+ id: 01a17739-8ba4-7812-9a1d-e09d7a786e91
315
315
  year: 1927
316
316
  minutes: 153
317
317
  status: to watch
@@ -384,7 +384,7 @@ You edit `datanotes/main/public/film/Sherlock Jr..md` by hand, outside the engin
384
384
  ```markdown
385
385
  ---
386
386
  table: main.public.film
387
- id: 01a16cf7-5643-7af3-888a-fbbab87f32ae
387
+ id: 01a17739-7689-7c24-97b2-599276b0b1ba
388
388
  title: Sherlock Jr.: the projectionist
389
389
  year: 1924
390
390
  status: watched
@@ -471,7 +471,7 @@ You edit `datanotes/main/public/film/Sherlock Jr..md` by hand, outside the engin
471
471
  ```markdown
472
472
  ---
473
473
  table: main.public.film
474
- id: 01a16cf7-5643-7af3-888a-fbbab87f32ae
474
+ id: 01a17739-7689-7c24-97b2-599276b0b1ba
475
475
  title: "Sherlock Jr.: the projectionist"
476
476
  year: 1924
477
477
  status: watched
@@ -30,7 +30,7 @@ Answer:
30
30
  "ok": true,
31
31
  "op": "define_base",
32
32
  "planned_texts": ["datanotes/metastore.yaml"],
33
- "migration_id": "20261024T081203-define_base-create-base-entry"
33
+ "migration_id": "20261026T080029-define_base-create-base-entry"
34
34
  }
35
35
  ```
36
36
 
@@ -50,7 +50,7 @@ Answer:
50
50
  "commits": [
51
51
  { "table": "diary.classics.film", "version": 6, "operation": "SET TABLE BASES" }
52
52
  ],
53
- "migration_id": "20261024T081203-set_table_bases-diary-classics-film-extends-en"
53
+ "migration_id": "20261026T080029-set_table_bases-diary-classics-film-extends-en"
54
54
  }
55
55
  ```
56
56
 
@@ -118,14 +118,14 @@ Answer:
118
118
  "rows": [
119
119
  {
120
120
  "path": "datanotes/diary/classics/actors/Max Schreck.md",
121
- "id": "01a16cf7-863f-7e8b-a6ba-88298d804353",
121
+ "id": "01a17739-a5cf-787a-bbd9-f7d256aec609",
122
122
  "added_on": "2026-10-05",
123
123
  "note": "The face of Nosferatu",
124
124
  "_source": "diary.classics.actors"
125
125
  },
126
126
  {
127
127
  "path": "datanotes/diary/classics/film/Nosferatu.md",
128
- "id": "01a16cf7-85ca-7d5a-8952-a21fee1584a0",
128
+ "id": "01a17739-a55e-750a-9874-527e3063d15a",
129
129
  "added_on": "2026-10-05",
130
130
  "note": null,
131
131
  "_source": "diary.classics.film"
@@ -169,7 +169,7 @@ Answer:
169
169
  "commits": [
170
170
  { "table": "diary.classics.actors", "version": 3, "operation": "DEFINE BASE" }
171
171
  ],
172
- "migration_id": "20261024T081203-define_base-alter-base-entry"
172
+ "migration_id": "20261026T080029-define_base-alter-base-entry"
173
173
  }
174
174
  ```
175
175
 
@@ -246,7 +246,7 @@ Answer:
246
246
  "commits": [
247
247
  { "table": "diary.classics.actors", "version": 4, "operation": "DEFINE BASE" }
248
248
  ],
249
- "migration_id": "20261024T081203-define_base-alter-base-entry-2"
249
+ "migration_id": "20261026T080029-define_base-alter-base-entry-2"
250
250
  }
251
251
  ```
252
252
 
@@ -280,7 +280,7 @@ Answer:
280
280
  "ok": true,
281
281
  "op": "define_base",
282
282
  "planned_texts": ["datanotes/metastore.yaml"],
283
- "migration_id": "20261024T081203-define_base-create-base-credited"
283
+ "migration_id": "20261026T080029-define_base-create-base-credited"
284
284
  }
285
285
  ```
286
286
 
@@ -300,7 +300,7 @@ Answer:
300
300
  "commits": [
301
301
  { "table": "diary.classics.actors", "version": 5, "operation": "SET TABLE BASES" }
302
302
  ],
303
- "migration_id": "20261024T081203-set_table_bases-diary-classics-actors-extends--2"
303
+ "migration_id": "20261026T080029-set_table_bases-diary-classics-actors-extends--2"
304
304
  }
305
305
  ```
306
306
 
@@ -329,7 +329,7 @@ Answer:
329
329
  "ok": true,
330
330
  "op": "define_base",
331
331
  "planned_texts": ["datanotes/metastore.yaml"],
332
- "migration_id": "20261024T081203-define_base-create-base-tree"
332
+ "migration_id": "20261026T080029-define_base-create-base-tree"
333
333
  }
334
334
  ```
335
335
 
@@ -369,7 +369,7 @@ Answer:
369
369
  "commits": [
370
370
  { "table": "diary.classics.genre", "version": 1, "operation": "SET TABLE BASES" }
371
371
  ],
372
- "migration_id": "20261024T081203-set_table_bases-diary-classics-genre-extends-t"
372
+ "migration_id": "20261026T080029-set_table_bases-diary-classics-genre-extends-t"
373
373
  }
374
374
  ```
375
375
 
@@ -475,6 +475,6 @@ Answer:
475
475
  "operation": "SET TABLE BASES"
476
476
  }
477
477
  ],
478
- "migration_id": "20261024T081203-set_table_bases-diary-classics-genre_draft-ext"
478
+ "migration_id": "20261026T080029-set_table_bases-diary-classics-genre_draft-ext"
479
479
  }
480
480
  ```
@@ -63,7 +63,7 @@ Answer:
63
63
  "commits": [
64
64
  { "table": "diary.classics.film", "version": 0, "operation": "CREATE TABLE" }
65
65
  ],
66
- "migration_id": "20261024T081203-create_table-create-table-diary-classics-fi"
66
+ "migration_id": "20261026T080029-create_table-create-table-diary-classics-fi"
67
67
  }
68
68
  ```
69
69
 
@@ -91,7 +91,7 @@ Answer:
91
91
  "files_created": 1,
92
92
  "planned_creates": ["datanotes/diary/classics/film/Nosferatu.md"],
93
93
  "commits": [{ "table": "diary.classics.film", "version": 1, "operation": "WRITE" }],
94
- "migration_id": "20261024T081203-insert_row-insert-into-film"
94
+ "migration_id": "20261026T080029-insert_row-insert-into-film"
95
95
  }
96
96
  ```
97
97
 
@@ -127,7 +127,7 @@ Answer:
127
127
  "commits": [
128
128
  { "table": "diary.classics.film", "version": 3, "operation": "ALTER TABLE" }
129
129
  ],
130
- "migration_id": "20261024T081203-alter_table-alter-table-diary-classics-fil-2"
130
+ "migration_id": "20261026T080029-alter_table-alter-table-diary-classics-fil-2"
131
131
  }
132
132
  ```
133
133
 
@@ -144,7 +144,7 @@ Answer:
144
144
  "ok": true,
145
145
  "op": "comment_on",
146
146
  "planned_texts": ["datanotes/metastore.yaml"],
147
- "migration_id": "20261024T081203-comment_on-comment-on-catalog-diary"
147
+ "migration_id": "20261026T080029-comment_on-comment-on-catalog-diary"
148
148
  }
149
149
  ```
150
150
 
@@ -165,7 +165,7 @@ Answer:
165
165
  "ok": true,
166
166
  "op": "comment_on",
167
167
  "planned_texts": ["datanotes/metastore.yaml"],
168
- "migration_id": "20261024T081203-comment_on-comment-on-schema-diary-classi"
168
+ "migration_id": "20261026T080029-comment_on-comment-on-schema-diary-classi"
169
169
  }
170
170
  ```
171
171
 
@@ -173,7 +173,7 @@ Answer:
173
173
 
174
174
  ```yaml
175
175
  # Metastore (Datanotes): catalogs, schemas, tables, columns, views.
176
- id: 01a16cf7-85ba-7448-adb2-79a7636a1226
176
+ id: 01a17739-a54c-781c-897f-cbf429ce48f6
177
177
  config:
178
178
  history: true
179
179
 
@@ -270,7 +270,7 @@ Answer:
270
270
  "commits": [
271
271
  { "table": "diary.classics.film", "version": 4, "operation": "ALTER TABLE" }
272
272
  ],
273
- "migration_id": "20261024T081203-alter_table-alter-table-diary-classics-fil-3"
273
+ "migration_id": "20261026T080029-alter_table-alter-table-diary-classics-fil-3"
274
274
  }
275
275
  ```
276
276
 
@@ -287,7 +287,7 @@ Answer:
287
287
  "ok": true,
288
288
  "op": "comment_on",
289
289
  "planned_texts": ["datanotes/metastore.yaml"],
290
- "migration_id": "20261024T081203-comment_on-remove-the-comment-on-schema-d"
290
+ "migration_id": "20261026T080029-comment_on-remove-the-comment-on-schema-d"
291
291
  }
292
292
  ```
293
293
 
@@ -309,7 +309,7 @@ Answer:
309
309
  "commits": [
310
310
  { "table": "diary.classics.film", "version": 5, "operation": "ALTER TABLE" }
311
311
  ],
312
- "migration_id": "20261024T081203-alter_table-alter-table-diary-classics-fil-4"
312
+ "migration_id": "20261026T080029-alter_table-alter-table-diary-classics-fil-4"
313
313
  }
314
314
  ```
315
315
 
@@ -593,7 +593,7 @@ This is the database of this page as the engine wrote it. `metastore.yaml` holds
593
593
 
594
594
  ```yaml
595
595
  # Metastore (Datanotes): catalogs, schemas, tables, columns, views.
596
- id: 01a16cf7-85ba-7448-adb2-79a7636a1226
596
+ id: 01a17739-a54c-781c-897f-cbf429ce48f6
597
597
  config:
598
598
  history: true
599
599
  default_catalog: diary