datanotes 3.0.0 → 3.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (73) hide show
  1. package/README.md +1 -5
  2. package/dist/api/client.d.cts +37 -4
  3. package/dist/api/client.d.ts +37 -4
  4. package/dist/api/index.cjs +563 -60
  5. package/dist/api/index.d.cts +3 -2
  6. package/dist/api/index.d.ts +3 -2
  7. package/dist/api/index.js +563 -60
  8. package/dist/api/mcp.cjs +181 -11
  9. package/dist/api/mcp.js +181 -11
  10. package/dist/api/otlp.d.cts +63 -15
  11. package/dist/api/otlp.d.ts +63 -15
  12. package/dist/api/redact.d.cts +25 -0
  13. package/dist/api/redact.d.ts +25 -0
  14. package/dist/api/tokens.d.cts +5 -0
  15. package/dist/api/tokens.d.ts +5 -0
  16. package/dist/api/tracing.d.cts +107 -0
  17. package/dist/api/tracing.d.ts +107 -0
  18. package/dist/datanotes.cjs +9934 -2656
  19. package/dist/docs/administration/limits.md +96 -12
  20. package/dist/docs/administration/run-the-service.md +18 -0
  21. package/dist/docs/api/operations/add_column.md +1 -0
  22. package/dist/docs/api/operations/add_constraint.md +2 -0
  23. package/dist/docs/api/operations/alter_column.md +1 -1
  24. package/dist/docs/api/operations/alter_table.md +3 -2
  25. package/dist/docs/api/operations/check_table.md +1 -1
  26. package/dist/docs/api/operations/create_table.md +3 -1
  27. package/dist/docs/api/operations/describe_table.md +1 -1
  28. package/dist/docs/api/operations/graph_neighbors.md +4 -2
  29. package/dist/docs/api/operations/graph_path.md +3 -3
  30. package/dist/docs/api/operations/graph_traverse.md +2 -2
  31. package/dist/docs/api/operations/insert_row.md +2 -2
  32. package/dist/docs/api/operations/record_tool_call.md +1 -0
  33. package/dist/docs/api/operations/tags/queries.md +2 -2
  34. package/dist/docs/api/operations/tags/rows-dml.md +1 -1
  35. package/dist/docs/api.md +4 -4
  36. package/dist/docs/develop/concurrency.md +36 -1
  37. package/dist/docs/develop/http-api.md +11 -1
  38. package/dist/docs/develop/telemetry.md +224 -25
  39. package/dist/docs/guides/data-quality/check-constraints.md +14 -13
  40. package/dist/docs/guides/data-quality/hand-edits.md +10 -10
  41. package/dist/docs/guides/database-objects/bases.md +11 -11
  42. package/dist/docs/guides/database-objects/catalogs-and-schemas.md +16 -10
  43. package/dist/docs/guides/database-objects/columns.md +38 -38
  44. package/dist/docs/guides/database-objects/relations.md +173 -36
  45. package/dist/docs/guides/database-objects/tables.md +90 -109
  46. package/dist/docs/guides/database-objects/views.md +44 -44
  47. package/dist/docs/guides/history/back-up.md +12 -4
  48. package/dist/docs/guides/history/change-data-feed.md +34 -34
  49. package/dist/docs/guides/history/journal-and-undo.md +12 -12
  50. package/dist/docs/guides/history/restore.md +20 -20
  51. package/dist/docs/guides/history/table-history.md +31 -31
  52. package/dist/docs/guides/history/vacuum.md +1 -10
  53. package/dist/docs/guides/read-and-write/automatic-links.md +15 -15
  54. package/dist/docs/guides/read-and-write/import-notes.md +21 -21
  55. package/dist/docs/guides/read-and-write/joins-and-aggregates.md +7 -7
  56. package/dist/docs/guides/read-and-write/read-and-filter.md +10 -10
  57. package/dist/docs/guides/read-and-write/write-rows.md +10 -10
  58. package/dist/docs/guides/search/link-graph.md +95 -0
  59. package/dist/docs/guides/search/text.md +1 -1
  60. package/dist/docs/guides/security/who-did-what.md +2 -0
  61. package/dist/docs/reference/cli.md +10 -3
  62. package/dist/docs/reference/compatibility.md +33 -13
  63. package/dist/docs/reference/errors.md +1 -1
  64. package/dist/docs/reference/operations/auxiliary.md +2 -2
  65. package/dist/docs/reference/operations/queries.md +3 -3
  66. package/dist/docs/reference/operations/rows.md +1 -1
  67. package/dist/docs/reference/operations/schema.md +15 -11
  68. package/dist/docs/reference/operations.md +39 -39
  69. package/dist/docs/reference/settings.md +37 -25
  70. package/dist/docs/release-notes.md +145 -0
  71. package/dist/docs/start/how-it-works.md +35 -3
  72. package/dist/docs/start/quick-start.md +2 -0
  73. package/package.json +3 -2
@@ -236,7 +236,7 @@ Refused:
236
236
 
237
237
  ```text
238
238
  pass "time_column" or "filename" (e.g. "{name}") so rows get file names
239
- Hint: Check the arguments of create_table: name (string, required), description (string), columns (list of objects, required), catalog (string), schema (string), time_column (string), filename (string), partition (string), exclude (list of strings), archive (string), aliases (list of strings), history (boolean), autolink_target (boolean), row_template (string), fields (object), tags (object), dry_run (boolean).
239
+ Hint: Check the arguments of create_table: name (string, required), description (string), columns (list of objects, required), catalog (string), schema (string), time_column (string), filename (string), partition (string), exclude (list of strings), archive (string), aliases (list of strings), history (boolean), autolink_target (boolean), row_template (string), instructions (string), fields (object), tags (object), dry_run (boolean).
240
240
  Docs: https://datanotes.dev/reference/operations/schema.md#create_table (offline: read_docs("reference/operations/schema#create_table"))
241
241
  ```
242
242
 
@@ -314,7 +314,7 @@ Answer:
314
314
  "op": "create_table",
315
315
  "planned_texts": ["datanotes/metastore.yaml"],
316
316
  "commits": [{ "table": "main.public.film", "version": 0, "operation": "CREATE TABLE" }],
317
- "migration_id": "20261012T071957-create_table-create-table-main-public-film"
317
+ "migration_id": "20261019T040648-create_table-create-table-main-public-film"
318
318
  }
319
319
  ```
320
320
 
@@ -359,7 +359,7 @@ Answer (70 lines)
359
359
  "path": "Films/Charade.md",
360
360
  "set": {
361
361
  "table": "main.public.film",
362
- "id": "01a12efb-81a4-717d-a664-f60882bb8996",
362
+ "id": "01a15257-31b6-75a9-bb7e-06ebf5b385a4",
363
363
  "director": "Stanley Donen",
364
364
  "title": "Charade"
365
365
  },
@@ -369,7 +369,7 @@ Answer (70 lines)
369
369
  "path": "Films/His Girl Friday.md",
370
370
  "set": {
371
371
  "table": "main.public.film",
372
- "id": "01a12efb-81a4-7e0e-8864-4d93d474be34",
372
+ "id": "01a15257-31b6-7aac-9ba1-16fb7e23aa0a",
373
373
  "director": "Howard Hawks",
374
374
  "title": "His Girl Friday"
375
375
  },
@@ -379,7 +379,7 @@ Answer (70 lines)
379
379
  "path": "Films/Metropolis.md",
380
380
  "set": {
381
381
  "table": "main.public.film",
382
- "id": "01a12efb-81a4-747e-8aef-048494381ac2",
382
+ "id": "01a15257-31b6-7d6a-998e-fd8f06c27786",
383
383
  "director": "Fritz Lang",
384
384
  "title": "Metropolis"
385
385
  },
@@ -389,7 +389,7 @@ Answer (70 lines)
389
389
  "path": "Films/Nosferatu.md",
390
390
  "set": {
391
391
  "table": "main.public.film",
392
- "id": "01a12efb-81a4-71bf-9b8f-a28f778461db",
392
+ "id": "01a15257-31b6-7a33-a502-afc11c66b564",
393
393
  "director": "F. W. Murnau",
394
394
  "title": "Nosferatu"
395
395
  },
@@ -399,7 +399,7 @@ Answer (70 lines)
399
399
  "path": "Films/The Gold Rush.md",
400
400
  "set": {
401
401
  "table": "main.public.film",
402
- "id": "01a12efb-81a4-7628-817c-ce1281d27329",
402
+ "id": "01a15257-31b6-7124-9bc9-40a032066bd3",
403
403
  "director": "Charlie Chaplin",
404
404
  "title": "The Gold Rush"
405
405
  },
@@ -458,7 +458,7 @@ Answer (73 lines)
458
458
  "path": "Films/Charade.md",
459
459
  "set": {
460
460
  "table": "main.public.film",
461
- "id": "01a12efb-81ad-7350-b6fc-3607a2c36814",
461
+ "id": "01a15257-31bb-7a46-8ac7-181fa5bacea7",
462
462
  "director": "Stanley Donen",
463
463
  "title": "Charade"
464
464
  },
@@ -468,7 +468,7 @@ Answer (73 lines)
468
468
  "path": "Films/His Girl Friday.md",
469
469
  "set": {
470
470
  "table": "main.public.film",
471
- "id": "01a12efb-81ad-76cb-a7d9-1d8bcd578f6d",
471
+ "id": "01a15257-31bb-7754-a84c-ada8c44c4f96",
472
472
  "director": "Howard Hawks",
473
473
  "title": "His Girl Friday"
474
474
  },
@@ -478,7 +478,7 @@ Answer (73 lines)
478
478
  "path": "Films/Metropolis.md",
479
479
  "set": {
480
480
  "table": "main.public.film",
481
- "id": "01a12efb-81ad-7b8f-aa3d-16ff4a7685be",
481
+ "id": "01a15257-31bb-7113-b86a-5d6fdca79420",
482
482
  "director": "Fritz Lang",
483
483
  "title": "Metropolis"
484
484
  },
@@ -488,7 +488,7 @@ Answer (73 lines)
488
488
  "path": "Films/Nosferatu.md",
489
489
  "set": {
490
490
  "table": "main.public.film",
491
- "id": "01a12efb-81ad-7a1c-b7d7-c6ce93fd565e",
491
+ "id": "01a15257-31bc-7f8b-b4a7-376f074e01cb",
492
492
  "director": "F. W. Murnau",
493
493
  "title": "Nosferatu"
494
494
  },
@@ -498,7 +498,7 @@ Answer (73 lines)
498
498
  "path": "Films/The Gold Rush.md",
499
499
  "set": {
500
500
  "table": "main.public.film",
501
- "id": "01a12efb-81ad-7816-bbb2-0c7ac62d182a",
501
+ "id": "01a15257-31bc-7554-a326-0e8b427aa5c6",
502
502
  "director": "Charlie Chaplin",
503
503
  "title": "The Gold Rush"
504
504
  },
@@ -519,7 +519,7 @@ Answer (73 lines)
519
519
  }
520
520
  ],
521
521
  "commits": [{ "table": "main.public.film", "version": 1, "operation": "ADOPT ROWS" }],
522
- "migration_id": "20261012T071957-adopt_rows-adopt-5-notes-into-film"
522
+ "migration_id": "20261019T040648-adopt_rows-adopt-5-notes-into-film"
523
523
  }
524
524
  ```
525
525
 
@@ -536,7 +536,7 @@ Answer:
536
536
  ```markdown
537
537
  ---
538
538
  table: main.public.film
539
- id: 01a12efb-81ad-7350-b6fc-3607a2c36814
539
+ id: 01a15257-31bb-7a46-8ac7-181fa5bacea7
540
540
  year: 1963
541
541
  minutes: 113
542
542
  status: watched
@@ -623,7 +623,7 @@ Answer:
623
623
  "files_created": 1,
624
624
  "planned_creates": ["datanotes/main/public/film/Battleship Potemkin.md"],
625
625
  "commits": [{ "table": "main.public.film", "version": 2, "operation": "WRITE" }],
626
- "migration_id": "20261012T071957-insert_row-insert-into-film"
626
+ "migration_id": "20261019T040648-insert_row-insert-into-film"
627
627
  }
628
628
  ```
629
629
 
@@ -655,7 +655,7 @@ Answer:
655
655
  "op": "add_column",
656
656
  "planned_texts": ["datanotes/metastore.yaml"],
657
657
  "commits": [{ "table": "main.public.film", "version": 4, "operation": "ADD COLUMNS" }],
658
- "migration_id": "20261012T071958-add_column-add-column-added_by-to-film"
658
+ "migration_id": "20261019T040650-add_column-add-column-added_by-to-film"
659
659
  }
660
660
  ```
661
661
 
@@ -683,7 +683,7 @@ Answer:
683
683
  "files_created": 1,
684
684
  "planned_creates": ["datanotes/main/public/film/Night of the Living Dead.md"],
685
685
  "commits": [{ "table": "main.public.film", "version": 5, "operation": "WRITE" }],
686
- "migration_id": "20261012T071958-insert_row-insert-into-film"
686
+ "migration_id": "20261019T040650-insert_row-insert-into-film"
687
687
  }
688
688
  ```
689
689
 
@@ -692,7 +692,7 @@ Answer:
692
692
  ```markdown
693
693
  ---
694
694
  table: main.public.film
695
- id: 01a12efb-87b5-7e4d-95f6-04c05995e9df
695
+ id: 01a15257-37bf-770c-a384-b360bb939197
696
696
  director: George A. Romero
697
697
  minutes: 96
698
698
  status: to watch
@@ -734,7 +734,7 @@ Answer:
734
734
  ```markdown
735
735
  ---
736
736
  table: main.public.film
737
- id: 01a12efb-8a37-7b33-bb22-169f2afb9934
737
+ id: 01a15257-3a49-7617-a437-430fcfdc292a
738
738
  year: 1924
739
739
  minutes: 45
740
740
  status: to watch
@@ -762,7 +762,7 @@ Answer:
762
762
  "op": "alter_table",
763
763
  "planned_texts": ["datanotes/metastore.yaml"],
764
764
  "commits": [{ "table": "main.public.film", "version": 7, "operation": "ALTER TABLE" }],
765
- "migration_id": "20261012T072001-alter_table-alter-table-main-public-film"
765
+ "migration_id": "20261019T040652-alter_table-alter-table-main-public-film"
766
766
  }
767
767
  ```
768
768
 
@@ -825,7 +825,7 @@ This is the table as you inferred, corrected and extended it on this page. Its d
825
825
 
826
826
  ```yaml
827
827
  # Metastore (Datanotes): catalogs, schemas, tables, columns, views.
828
- id: 01a12efb-8191-7a5c-9aee-8ffe9c6f215a
828
+ id: 01a15257-319e-7663-863b-41ff43d87deb
829
829
  config:
830
830
  history: true
831
831
 
@@ -100,9 +100,9 @@ Answer:
100
100
  "table": "main.public.watch",
101
101
  "total": 3,
102
102
  "rows": [
103
- { "date(occurred_at)": "2026-10-07", "viewings": 1 },
104
- { "date(occurred_at)": "2026-10-10", "viewings": 1 },
105
- { "date(occurred_at)": "2026-10-11", "viewings": 1 }
103
+ { "date(occurred_at)": "2026-10-14", "viewings": 1 },
104
+ { "date(occurred_at)": "2026-10-17", "viewings": 1 },
105
+ { "date(occurred_at)": "2026-10-18", "viewings": 1 }
106
106
  ]
107
107
  }
108
108
  ```
@@ -138,10 +138,10 @@ Answer:
138
138
  {
139
139
  "path": "datanotes/main/public/film/Night of the Living Dead.md",
140
140
  "title": "Night of the Living Dead",
141
- "watched_on": "2026-10-11",
141
+ "watched_on": "2026-10-18",
142
142
  "viewing": [
143
143
  {
144
- "path": "datanotes/main/public/watch/2026-10-11T2200.md",
144
+ "path": "datanotes/main/public/watch/2026-10-18T2200.md",
145
145
  "where": "cinema",
146
146
  "with": "Oscar"
147
147
  }
@@ -150,10 +150,10 @@ Answer:
150
150
  {
151
151
  "path": "datanotes/main/public/film/The Gold Rush.md",
152
152
  "title": "The Gold Rush",
153
- "watched_on": "2026-10-10",
153
+ "watched_on": "2026-10-17",
154
154
  "viewing": [
155
155
  {
156
- "path": "datanotes/main/public/watch/2026-10-10T2030.md",
156
+ "path": "datanotes/main/public/watch/2026-10-17T2030.md",
157
157
  "where": "home",
158
158
  "with": "my sister"
159
159
  }
@@ -332,23 +332,23 @@ Answer:
332
332
  "total": 3,
333
333
  "rows": [
334
334
  {
335
- "path": "datanotes/main/public/watch/2026-10-07T2115.md",
335
+ "path": "datanotes/main/public/watch/2026-10-14T2115.md",
336
336
  "film": "[[Charade]]",
337
- "occurred_at": "2026-10-07T21:15",
337
+ "occurred_at": "2026-10-14T21:15",
338
338
  "where": "home",
339
339
  "place": "my living room"
340
340
  },
341
341
  {
342
- "path": "datanotes/main/public/watch/2026-10-10T2030.md",
342
+ "path": "datanotes/main/public/watch/2026-10-17T2030.md",
343
343
  "film": "[[The Gold Rush]]",
344
- "occurred_at": "2026-10-10T20:30",
344
+ "occurred_at": "2026-10-17T20:30",
345
345
  "where": "home",
346
346
  "place": "my living room"
347
347
  },
348
348
  {
349
- "path": "datanotes/main/public/watch/2026-10-11T2200.md",
349
+ "path": "datanotes/main/public/watch/2026-10-18T2200.md",
350
350
  "film": "[[Night of the Living Dead]]",
351
- "occurred_at": "2026-10-11T22:00",
351
+ "occurred_at": "2026-10-18T22:00",
352
352
  "where": "cinema",
353
353
  "place": "the film club downtown"
354
354
  }
@@ -459,12 +459,12 @@ Answer:
459
459
  "total": 7,
460
460
  "rows": [
461
461
  {
462
- "path": "datanotes/main/public/watch/2026-10-07T2115.md",
463
- "occurred_at": "2026-10-07T21:15"
462
+ "path": "datanotes/main/public/watch/2026-10-14T2115.md",
463
+ "occurred_at": "2026-10-14T21:15"
464
464
  },
465
465
  {
466
- "path": "datanotes/main/public/watch/2026-10-03T2130.md",
467
- "occurred_at": "2026-10-03T21:30"
466
+ "path": "datanotes/main/public/watch/2026-10-10T2130.md",
467
+ "occurred_at": "2026-10-10T21:30"
468
468
  }
469
469
  ]
470
470
  }
@@ -31,7 +31,7 @@ Answer:
31
31
  "files_created": 1,
32
32
  "planned_creates": ["datanotes/main/public/film/Metropolis.md"],
33
33
  "commits": [{ "table": "main.public.film", "version": 1, "operation": "WRITE" }],
34
- "migration_id": "20261012T071946-insert_row-insert-into-film"
34
+ "migration_id": "20261019T040638-insert_row-insert-into-film"
35
35
  }
36
36
  ```
37
37
 
@@ -40,7 +40,7 @@ Answer:
40
40
  ```markdown
41
41
  ---
42
42
  table: main.public.film
43
- id: 01a12efb-5925-7551-9fc3-ca301818d73d
43
+ id: 01a15257-0a57-7094-a53c-2ffddb1a5284
44
44
  title: Metropolis
45
45
  year: 1927
46
46
  minutes: 153
@@ -103,7 +103,7 @@ Answer:
103
103
  }
104
104
  ],
105
105
  "commits": [{ "table": "main.public.film", "version": 6, "operation": "UPDATE" }],
106
- "migration_id": "20261012T071951-update_row-update-film"
106
+ "migration_id": "20261019T040643-update_row-update-film"
107
107
  }
108
108
  ```
109
109
 
@@ -174,7 +174,7 @@ Answer:
174
174
  }
175
175
  ],
176
176
  "commits": [{ "table": "main.public.film", "version": 8, "operation": "UPDATE" }],
177
- "migration_id": "20261012T071951-update_row-update-film-3",
177
+ "migration_id": "20261019T040643-update_row-update-film-3",
178
178
  "ambiguous_mentions": [
179
179
  {
180
180
  "path": "datanotes/main/public/film/Nosferatu.md",
@@ -193,7 +193,7 @@ Answer:
193
193
  ```markdown
194
194
  ---
195
195
  table: main.public.film
196
- id: 01a12efb-6c2e-7852-8b4b-ed121d84aec5
196
+ id: 01a15257-1ca9-7d11-a4d0-bc403634a138
197
197
  title: Nosferatu
198
198
  year: 1922
199
199
  status: watched
@@ -246,7 +246,7 @@ Answer:
246
246
  }
247
247
  ],
248
248
  "commits": [{ "table": "main.public.film", "version": 9, "operation": "UPDATE" }],
249
- "migration_id": "20261012T071951-update_row-update-film-4"
249
+ "migration_id": "20261019T040643-update_row-update-film-4"
250
250
  }
251
251
  ```
252
252
 
@@ -332,7 +332,7 @@ Answer:
332
332
  }
333
333
  ],
334
334
  "commits": [{ "table": "main.public.film", "version": 10, "operation": "UPDATE" }],
335
- "migration_id": "20261012T071951-update_rows-update-film"
335
+ "migration_id": "20261019T040643-update_rows-update-film"
336
336
  }
337
337
  ```
338
338
 
@@ -370,7 +370,7 @@ Answer:
370
370
  ],
371
371
  "planned_creates": ["datanotes/main/public/film/Battleship Potemkin.md"],
372
372
  "commits": [{ "table": "main.public.film", "version": 11, "operation": "MERGE" }],
373
- "migration_id": "20261012T071951-merge_rows-merge-2-rows-into-film"
373
+ "migration_id": "20261019T040643-merge_rows-merge-2-rows-into-film"
374
374
  }
375
375
  ```
376
376
 
@@ -431,7 +431,7 @@ Answer:
431
431
  }
432
432
  ],
433
433
  "commits": [{ "table": "main.public.film", "version": 12, "operation": "DELETE" }],
434
- "migration_id": "20261012T071951-delete_rows-archive-1-rows-of-film"
434
+ "migration_id": "20261019T040643-delete_rows-archive-1-rows-of-film"
435
435
  }
436
436
  ```
437
437
 
@@ -568,6 +568,6 @@ Answer (51 lines)
568
568
  { "table": "main.public.film", "version": 22, "operation": "MOVE ROWS" },
569
569
  { "table": "main.public.watchlist", "version": 1, "operation": "MOVE ROWS" }
570
570
  ],
571
- "migration_id": "20261012T071951-move_rows-move-4-rows-from-film-to-watch"
571
+ "migration_id": "20261019T040643-move_rows-move-4-rows-from-film-to-watch"
572
572
  }
573
573
  ```
@@ -45,6 +45,101 @@ Answer:
45
45
  }
46
46
  ```
47
47
 
48
+ ## Read the whole graph in one call
49
+
50
+ `tables` starts from the rows of several tables at once. Name the tables, or use a pattern: `*` is every table, `main.*` every table of the `main` catalog. With `direction: "out"`, one call returns every link of every row: the whole graph, for an app that draws it. The start set holds at most 5,000 notes.
51
+
52
+ **`graph_neighbors`**
53
+
54
+ ```json
55
+ { "from_nodes": { "tables": ["*"] }, "direction": "out" }
56
+ ```
57
+
58
+ Answer (80 lines)
59
+
60
+ ```json
61
+ {
62
+ "nodes": [
63
+ {
64
+ "path": "datanotes/main/public/actor/Audrey Hepburn.md",
65
+ "table": "main.public.actor",
66
+ "counts": {},
67
+ "edges": []
68
+ },
69
+ {
70
+ "path": "datanotes/main/public/actor/Cary Grant.md",
71
+ "table": "main.public.actor",
72
+ "counts": {},
73
+ "edges": []
74
+ },
75
+ {
76
+ "path": "datanotes/main/public/actor/Rosalind Russell.md",
77
+ "table": "main.public.actor",
78
+ "counts": {},
79
+ "edges": []
80
+ },
81
+ {
82
+ "path": "datanotes/main/public/film/Charade.md",
83
+ "table": "main.public.film",
84
+ "counts": { "out:cast": 2, "out:link": 1 },
85
+ "edges": [
86
+ {
87
+ "path": "datanotes/main/public/actor/Cary Grant.md",
88
+ "table": "main.public.actor",
89
+ "label": "cast",
90
+ "direction": "out"
91
+ },
92
+ {
93
+ "path": "datanotes/main/public/actor/Audrey Hepburn.md",
94
+ "table": "main.public.actor",
95
+ "label": "cast",
96
+ "direction": "out"
97
+ },
98
+ {
99
+ "path": "datanotes/main/public/film/His Girl Friday.md",
100
+ "table": "main.public.film",
101
+ "label": "link",
102
+ "direction": "out"
103
+ }
104
+ ]
105
+ },
106
+ {
107
+ "path": "datanotes/main/public/film/His Girl Friday.md",
108
+ "table": "main.public.film",
109
+ "counts": { "out:cast": 2 },
110
+ "edges": [
111
+ {
112
+ "path": "datanotes/main/public/actor/Cary Grant.md",
113
+ "table": "main.public.actor",
114
+ "label": "cast",
115
+ "direction": "out"
116
+ },
117
+ {
118
+ "path": "datanotes/main/public/actor/Rosalind Russell.md",
119
+ "table": "main.public.actor",
120
+ "label": "cast",
121
+ "direction": "out"
122
+ }
123
+ ]
124
+ },
125
+ {
126
+ "path": "datanotes/main/public/watch/2026-09-25T2100.md",
127
+ "table": "main.public.watch",
128
+ "counts": { "out:film": 1, "out:link": 1 },
129
+ "edges": [
130
+ {
131
+ "path": "datanotes/main/public/film/Charade.md",
132
+ "table": "main.public.film",
133
+ "label": "film",
134
+ "direction": "out"
135
+ },
136
+ { "path": "Notes/Film night.md", "label": "link", "direction": "out" }
137
+ ]
138
+ }
139
+ ]
140
+ }
141
+ ```
142
+
48
143
  ## Walk the links from a note
49
144
 
50
145
  `graph_traverse` follows links breadth-first, in the `direction` you choose: `out`, `in` or `both`. Here it starts from Audrey Hepburn, goes up to two steps in both directions and keeps only films. It finds Charade, the film she’s in, and His Girl Friday, which the Charade note links to. `via` shows the route to each one.
@@ -87,7 +87,7 @@ Answer (62 lines)
87
87
  {
88
88
  "start": 81,
89
89
  "end": 86,
90
- "context": "-bd1b-d5bf80031dd6\nname: Cary Grant\n---\n\n"
90
+ "context": "-a8aa-d00b2c25ad08\nname: Cary Grant\n---\n\n"
91
91
  }
92
92
  ]
93
93
  },
@@ -31,6 +31,8 @@ A program that serves its own Model Context Protocol (MCP) server can report its
31
31
 
32
32
  Tool calls appear on the change feed as events with `kind: "tool_call"`, sent only to consumers that ask for that kind. The `call` field holds `tool`, `ok`, `served_by`, `duration_ms`, `summary` and `session`, the arguments cut at 300 characters, the value cut at 1,000, and the result’s size in `result_chars` and `result_tokens`.
33
33
 
34
+ A large result costs less to report when the program cuts it first. The engine answers `takes_summary: true` when it accepts a cut call. Then send the value cut at 1,000 characters, and the result’s `summary` next to it. The event is the same as for the whole call. `datanotes mcp` does this after its first report.
35
+
34
36
  ## Read who did what
35
37
 
36
38
  Three operations read the record:
@@ -3,7 +3,7 @@
3
3
  > The datanotes command and its options.
4
4
 
5
5
  ```text
6
- datanotes 3.0.0: Markdown notes as database tables, for you and your agents.
6
+ datanotes 3.2.0: Markdown notes as database tables, for you and your agents.
7
7
 
8
8
 
9
9
  Usage:
@@ -14,12 +14,14 @@ Usage:
14
14
  datanotes store new-id --database <folder> a new database id (a copied folder gets its own store)
15
15
  datanotes store backup --database <folder> show or set the store's git backup (--remote <url>, --off)
16
16
  datanotes store restore --from <url> --database <folder> restore the store from its git backup
17
+ datanotes store convert-log --to <files|segments> --database <folder> rewrite the transaction log in that format
17
18
  Every command takes --vault <folder> instead of --database for a vault holding the database.
18
19
  datanotes mcp --database <folder> MCP server over stdio, on the engine running on the folder
19
20
  (or one it starts for the session)
20
21
  datanotes mcp --url <url> --token-file <f> MCP server over stdio, on an engine at an address
21
22
  datanotes mcp serve [options] MCP server over Streamable HTTP at /mcp (each agent sends its token)
22
23
  datanotes --version | --help
24
+ dnotes is a short name for datanotes: every command works under both names.
23
25
 
24
26
 
25
27
  Options of serve:
@@ -49,6 +51,8 @@ Options of serve:
49
51
  between two operations, then a push (or DATANOTES_STORE_BACKUP_INTERVAL_S;
50
52
  default: the setting storeBackupIntervalSeconds, 60)
51
53
  --log <file> append console output to a file
54
+ DATANOTES_INDEX_CHECK=1: check the engine's indexes against the vault
55
+ after every operation, and log any difference as an error (slow: debugging)
52
56
 
53
57
 
54
58
  Options of status (the engine is found through the lock in its store, called with the owner's token):
@@ -79,7 +83,7 @@ commit and push, releases its lock and exits):
79
83
 
80
84
  Options of store:
81
85
  --database <folder> the database folder (or DATANOTES_DATABASE; --vault <folder> for a vault)
82
- --json store path, store backup: the answer as JSON
86
+ --json store path, store backup, store convert-log: the answer as JSON
83
87
  --remote <url> store backup: turn it on, pushing to this git repository (a first push now)
84
88
  --ssh-command <cmd> store backup, with --remote: the store repository's core.sshCommand
85
89
  (default: the vault repository's), e.g. C:/Windows/System32/OpenSSH/ssh.exe
@@ -90,7 +94,10 @@ Options of store:
90
94
  --port <port> store restore: the HTTP port in the new settings (default 27150)
91
95
  --force store restore: restore over a store that exists (kept aside, renamed)
92
96
  --no-backup store restore: leave the store backup off in the new settings
93
- backup and restore change the store only while no engine runs on the database.
97
+ --to <files|segments> store convert-log: segments (a few large files) or files (one per
98
+ version, what Datanotes 3.0 reads); sets transactionLogFormat
99
+ new-id, backup, restore and convert-log run only while no engine runs on the database, and
100
+ hold its store while they run: an engine started meanwhile refuses to start.
94
101
  The data home holds one store per database: %LOCALAPPDATA%\Datanotes (Windows),
95
102
  ~/Library/Application Support/Datanotes (macOS), $XDG_DATA_HOME/datanotes or
96
103
  ~/.local/share/datanotes (Linux); DATANOTES_DATA_HOME moves it.
@@ -1,25 +1,29 @@
1
1
  # Compatibility
2
2
 
3
- > What Datanotes 3 promises: every 3.x version adds to what you use and never takes it away.
3
+ > How Datanotes numbers its versions, what counts as its public interface, and how to write clients that keep working.
4
4
 
5
- Datanotes 3 keeps working with your programs, scripts and vaults in every 3.x version. The number after the 3 only says how recent a release is: 3.1, 3.2, and on.
5
+ Datanotes follows [semantic versioning](https://semver.org). The version number tells you what an update can change.
6
6
 
7
- ## The promise
7
+ ## Versions
8
8
 
9
- What works in Datanotes 3 keeps working the same way in every 3.x version:
9
+ * **Major** (4.0.0): can change or remove parts of the public interface. The release notes list those changes first, with what to do.
10
+ * **Minor** (3.1.0): adds to the public interface without breaking what works.
11
+ * **Patch** (3.0.1): fixes bugs and changes nothing you rely on.
12
+
13
+ The [release notes](https://datanotes.dev/release-notes/) describe every version.
14
+
15
+ ## The public interface
16
+
17
+ These parts of Datanotes follow the version rules:
10
18
 
11
19
  * Operations, their arguments and their answer fields
12
20
  * HTTP routes and headers
13
21
  * Violation codes
14
- * Commands, their options and their exit codes
22
+ * The commands the package installs (`datanotes` and `dnotes`), their options and their exit codes
15
23
  * Environment variables and settings
16
24
  * The words of `metastore.yaml`
17
25
 
18
- A new version can add to these. It never removes or renames them, and a name never changes meaning. When Datanotes needs a different behavior, it adds a new name. The old name keeps doing what it did.
19
-
20
- For example, a program that calls `insert_row` with `table` and `values`, then reads `planned_creates`, works with every 3.x version. A later version can add an optional argument, a new answer field or a new operation, and the program doesn’t notice.
21
-
22
- These cases fall outside the promise:
26
+ These cases fall outside them:
23
27
 
24
28
  * **Security fixes**: a fix can change a behavior. The release notes say so.
25
29
  * **Bugs**: Datanotes fixes behavior that contradicts the documentation.
@@ -27,6 +31,22 @@ These cases fall outside the promise:
27
31
  * **Messages**: the text of refusals and hints can improve in any version.
28
32
  * **Node.js versions**: support follows the Node.js long-term support releases.
29
33
 
34
+ ## Downgrade to 3.0
35
+
36
+ From 3.1, the engine writes the transaction log in segments by default. Datanotes 3.0 can’t read segments, so convert the log before you downgrade:
37
+
38
+ 1. Stop the engine.
39
+
40
+ 2. Convert every table’s log to one file per version:
41
+
42
+ ```bash
43
+ datanotes store convert-log --to files --vault ~/notes
44
+ ```
45
+
46
+ 3. Install Datanotes 3.0 and start it.
47
+
48
+ The command also sets `transactionLogFormat` to `files`, so a 3.1 engine started on the store again keeps writing files. [How the engine writes the log](https://datanotes.dev/start/how-it-works/#how-the-engine-writes-the-log) describes both formats.
49
+
30
50
  ## Write tolerant clients
31
51
 
32
52
  Answers can gain fields, and enums can gain values. Write your client to accept them:
@@ -56,8 +76,8 @@ const r = await db.call("insert_row", {
56
76
  if (r.ok) console.log((r.value as Created).planned_creates);
57
77
  ```
58
78
 
59
- ## How Datanotes keeps it
79
+ ## How the tests check it
60
80
 
61
- The repository keeps the whole public contract in `contract/datanotes-3.txt`. It has one line for every operation, argument, answer field, route, command option, environment variable, setting and violation code.
81
+ The repository keeps the public interface in `contract/datanotes-3.txt`. It has one line for every operation, argument, answer field, route, installed command, command option, environment variable, setting and violation code.
62
82
 
63
- The test suite fails when a line goes or changes, so a machine checks the promise. A second test checks every new name against the naming rules before it ships.
83
+ The test suite fails when a line goes or changes within a major version, so a breaking change cannot ship by accident. A second test checks every new name against the naming rules.
@@ -67,6 +67,6 @@ A write the engine refuses lists its violations, each with a `code`; `check_tabl
67
67
  | `unknown_column` | Write only the table’s columns: describe_table lists them. To keep a new front matter key, add_column first. | [Describe a table before writing to it](https://datanotes.dev/guides/database-objects/tables/#describe-a-table-before-writing-to-it) |
68
68
  | `check` | The row breaks a CHECK constraint of the table (named in the message, with its rule): change the values so the rule holds. describe_table lists the table’s constraints. | [Enforce rules with CHECK constraints](https://datanotes.dev/guides/data-quality/check-constraints/#enforce-rules-with-check-constraints) |
69
69
  | `generated` | A generated column is computed by the engine: do not write it; recompute_columns sets it again. | [Recompute generated columns after a hand edit](https://datanotes.dev/guides/data-quality/hand-edits/#recompute-generated-columns-after-a-hand-edit) |
70
- | `cycle` | A row cannot be its own ancestor: point the hierarchy column to a row that is not below this one. | [Build a tree of rows with a parent link](https://datanotes.dev/guides/database-objects/relations/#build-a-tree-of-rows-with-a-parent-link) |
70
+ | `cycle` | A row cannot be its own ancestor in a hierarchy column, nor depend on itself through an acyclic column: the message shows the path. Point the column to a row that is not on that path. | [Build a tree of rows with a parent link](https://datanotes.dev/guides/database-objects/relations/#build-a-tree-of-rows-with-a-parent-link) |
71
71
  | `inverse` | Both sides of a two-way link must agree: write one side, or run align_relations to repair them. | [Repair two-way links with `align_relations`](https://datanotes.dev/guides/database-objects/relations/#repair-two-way-links-with-align_relations) |
72
72
  | `unreadable` | The note’s front matter is not valid YAML (the message quotes the parser), so every read leaves the row out: open the note and fix the YAML, such as a quote or bracket left open or a line without `key: value`. read_note shows its text. | [Find rows whose front matter can’t be read](https://datanotes.dev/guides/data-quality/hand-edits/#find-rows-whose-front-matter-cant-be-read) |
@@ -20,7 +20,7 @@ Examples: [Tables](https://datanotes.dev/guides/database-objects/tables/), [View
20
20
 
21
21
  **Describe Table** · read-only
22
22
 
23
- Describe a table (`catalog.schema.table` or a shorter unambiguous name): `qualified`, `name`, `catalog`, `schema`, `description`, `aliases`, `layout` (`folder`, `row_folder`: the folder pattern of new rows, `archive`), `options` (`partition`, `filename`, `archive`, `exclude`), typed `columns` with label/aliases/description and the dense flag, `untyped_fields`, `row_count`, and when set: `tags`, `bases` (the bases it inherits from, in order), `constraints` (its CHECKs, name → where), `row_template`, `history`, `autolink_target`, `fields` (their values).
23
+ Describe a table (`catalog.schema.table` or a shorter unambiguous name): `qualified`, `name`, `catalog`, `schema`, `description`, `aliases`, `layout` (`folder`, `row_folder`: the folder pattern of new rows, `archive`), `options` (`partition`, `filename`, `archive`, `exclude`), typed `columns` with label/aliases/description and the dense flag, `untyped_fields`, `row_count`, and when set: `tags`, `bases` (the bases it inherits from, in order), `constraints` (its CHECKs, name → where), `constraint_hints` (their own hints, name → hint), `instructions` (how to fill a row: follow them before you write one), `row_template`, `history`, `autolink_target`, `fields` (their values).
24
24
 
25
25
  Before adding a column (e.g. a new metric), read `columns` and match your concept against `name`, `label`, `aliases` and `description`; reuse an existing column when it means the same.
26
26
 
@@ -66,7 +66,7 @@ Examples: [Catalogs, schemas and the database](https://datanotes.dev/guides/data
66
66
 
67
67
  **Check Table** · read-only
68
68
 
69
- Validate every row against the table (required, types, ranges, enums, links, unknown columns, CHECK constraints, those that follow links included, filtered link targets (ref_where / ref_by), stale generated/identity values, missing inverse links, front matter that cannot be read: code `unreadable`, with no column, for a note whose `table` line names the table but whose YAML is broken, which every other read leaves out). A CHECK violation has code `check` and the CHECK’s name as `constraint` (every other code names its `column`), as in the `violations` of list_changes. Use it to find rows edited by hand that break the schema. Without `table`, every table is checked: each row then names its `table`, `tables_checked` counts the tables checked, and a table that cannot be checked is listed in `errors` ({table, error}) while the others are. The rows with violations are paged: `limit` (default 500, at most 5000) from `offset`; `total` counts them all. (Hand edits are also checked as they happen: see `violations` in list_changes.)
69
+ Validate every row against the table (required, types, ranges, enums, links, unknown columns, CHECK constraints, those that follow links included, filtered link targets (ref_where / ref_by), stale generated/identity values, missing inverse links, cycles of hierarchy and acyclic columns, front matter that cannot be read: code `unreadable`, with no column, for a note whose `table` line names the table but whose YAML is broken, which every other read leaves out). A CHECK violation has code `check` and the CHECK’s name as `constraint` (every other code names its `column`), as in the `violations` of list_changes. Use it to find rows edited by hand that break the schema. Without `table`, every table is checked: each row then names its `table`, `tables_checked` counts the tables checked, and a table that cannot be checked is listed in `errors` ({table, error}) while the others are. The rows with violations are paged: `limit` (default 500, at most 5000) from `offset`; `total` counts them all. (Hand edits are also checked as they happen: see `violations` in list_changes.)
70
70
 
71
71
  Examples: [Relations and hierarchies](https://datanotes.dev/guides/database-objects/relations/), [Hand edits](https://datanotes.dev/guides/data-quality/hand-edits/), [Find rows that break the schema](https://datanotes.dev/guides/data-quality/check-table/).
72
72