datanotes 3.4.2 → 3.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (46) hide show
  1. package/README.md +1 -1
  2. package/dist/api/extensions.cjs +1088 -5
  3. package/dist/api/extensions.js +1088 -5
  4. package/dist/api/index.cjs +1088 -5
  5. package/dist/api/index.js +1088 -5
  6. package/dist/api/mcp.cjs +1088 -5
  7. package/dist/api/mcp.js +1088 -5
  8. package/dist/datanotes.cjs +4408 -1618
  9. package/dist/docs/administration/git-sync.md +17 -0
  10. package/dist/docs/administration/limits.md +29 -0
  11. package/dist/docs/administration/use-with-obsidian.md +12 -0
  12. package/dist/docs/api/operations/alter_table.md +6 -1
  13. package/dist/docs/api/operations/create_table.md +5 -0
  14. package/dist/docs/api/operations/describe_table.md +1 -1
  15. package/dist/docs/api/operations/pack_rows.md +45 -0
  16. package/dist/docs/api/operations/tags/auxiliary.md +2 -0
  17. package/dist/docs/api/operations/unpack_rows.md +40 -0
  18. package/dist/docs/api.md +3 -1
  19. package/dist/docs/guides/data-quality/check-constraints.md +10 -10
  20. package/dist/docs/guides/data-quality/hand-edits.md +8 -8
  21. package/dist/docs/guides/database-objects/bases.md +11 -11
  22. package/dist/docs/guides/database-objects/catalogs-and-schemas.md +10 -10
  23. package/dist/docs/guides/database-objects/columns.md +41 -41
  24. package/dist/docs/guides/database-objects/packs.md +563 -0
  25. package/dist/docs/guides/database-objects/relations.md +28 -28
  26. package/dist/docs/guides/database-objects/tables.md +28 -28
  27. package/dist/docs/guides/database-objects/views.md +44 -44
  28. package/dist/docs/guides/history/change-data-feed.md +34 -34
  29. package/dist/docs/guides/history/journal-and-undo.md +12 -12
  30. package/dist/docs/guides/history/restore.md +20 -20
  31. package/dist/docs/guides/history/table-history.md +31 -31
  32. package/dist/docs/guides/read-and-write/automatic-links.md +15 -15
  33. package/dist/docs/guides/read-and-write/import-notes.md +21 -21
  34. package/dist/docs/guides/read-and-write/joins-and-aggregates.md +7 -7
  35. package/dist/docs/guides/read-and-write/read-and-filter.md +10 -10
  36. package/dist/docs/guides/read-and-write/write-rows.md +10 -10
  37. package/dist/docs/guides/search/text.md +1 -1
  38. package/dist/docs/reference/cli.md +3 -1
  39. package/dist/docs/reference/operations/auxiliary.md +30 -1
  40. package/dist/docs/reference/operations/queries.md +1 -1
  41. package/dist/docs/reference/operations/rows.md +3 -3
  42. package/dist/docs/reference/operations/schema.md +8 -4
  43. package/dist/docs/reference/operations.md +55 -53
  44. package/dist/docs/reference/settings.md +1 -0
  45. package/dist/docs/release-notes.md +29 -0
  46. package/package.json +1 -1
@@ -236,7 +236,7 @@ Refused:
236
236
 
237
237
  ```text
238
238
  pass "time_column" or "filename" (e.g. "{name}") so rows get file names
239
- Hint: Check the arguments of create_table: name (string, required), description (string), columns (list of objects, required), catalog (string), schema (string), time_column (string), filename (string), partition (string), exclude (list of strings), archive (string), aliases (list of strings), history (boolean), autolink_target (boolean), row_template (string), instructions (string), fields (object), tags (object), dry_run (boolean).
239
+ Hint: Check the arguments of create_table: name (string, required), description (string), columns (list of objects, required), catalog (string), schema (string), time_column (string), filename (string), partition (string), pack (string), pack_after_days (integer), exclude (list of strings), archive (string), aliases (list of strings), history (boolean), autolink_target (boolean), row_template (string), instructions (string), fields (object), tags (object), dry_run (boolean).
240
240
  Docs: https://datanotes.dev/reference/operations/schema.md#create_table (offline: read_docs("reference/operations/schema#create_table"))
241
241
  ```
242
242
 
@@ -314,7 +314,7 @@ Answer:
314
314
  "op": "create_table",
315
315
  "planned_texts": ["datanotes/metastore.yaml"],
316
316
  "commits": [{ "table": "main.public.film", "version": 0, "operation": "CREATE TABLE" }],
317
- "migration_id": "20261024T203000-create_table-create-table-main-public-film"
317
+ "migration_id": "20261026T080022-create_table-create-table-main-public-film"
318
318
  }
319
319
  ```
320
320
 
@@ -359,7 +359,7 @@ Answer (70 lines)
359
359
  "path": "Films/Charade.md",
360
360
  "set": {
361
361
  "table": "main.public.film",
362
- "id": "01a16f9b-230f-733e-bbe4-83739c8553c0",
362
+ "id": "01a17739-8ba0-705c-9dab-b7ee827fd020",
363
363
  "director": "Stanley Donen",
364
364
  "title": "Charade"
365
365
  },
@@ -369,7 +369,7 @@ Answer (70 lines)
369
369
  "path": "Films/His Girl Friday.md",
370
370
  "set": {
371
371
  "table": "main.public.film",
372
- "id": "01a16f9b-2310-7285-bd86-19c52b42de66",
372
+ "id": "01a17739-8ba0-7c30-b138-2b4e68c53f25",
373
373
  "director": "Howard Hawks",
374
374
  "title": "His Girl Friday"
375
375
  },
@@ -379,7 +379,7 @@ Answer (70 lines)
379
379
  "path": "Films/Metropolis.md",
380
380
  "set": {
381
381
  "table": "main.public.film",
382
- "id": "01a16f9b-2310-7849-8987-2433f84c5cd4",
382
+ "id": "01a17739-8ba0-738c-b096-f979386e8feb",
383
383
  "director": "Fritz Lang",
384
384
  "title": "Metropolis"
385
385
  },
@@ -389,7 +389,7 @@ Answer (70 lines)
389
389
  "path": "Films/Nosferatu.md",
390
390
  "set": {
391
391
  "table": "main.public.film",
392
- "id": "01a16f9b-2310-7fed-ab01-51ef740bd322",
392
+ "id": "01a17739-8ba0-7dbe-9ab5-50c4ce32d03f",
393
393
  "director": "F. W. Murnau",
394
394
  "title": "Nosferatu"
395
395
  },
@@ -399,7 +399,7 @@ Answer (70 lines)
399
399
  "path": "Films/The Gold Rush.md",
400
400
  "set": {
401
401
  "table": "main.public.film",
402
- "id": "01a16f9b-2310-7d23-9b9a-57fa004619ec",
402
+ "id": "01a17739-8ba0-71b1-bd5d-efdb783b9feb",
403
403
  "director": "Charlie Chaplin",
404
404
  "title": "The Gold Rush"
405
405
  },
@@ -458,7 +458,7 @@ Answer (73 lines)
458
458
  "path": "Films/Charade.md",
459
459
  "set": {
460
460
  "table": "main.public.film",
461
- "id": "01a16f9b-2314-76cc-a0b0-3737ae7c5e61",
461
+ "id": "01a17739-8ba4-75cd-8d89-591ad0529c6e",
462
462
  "director": "Stanley Donen",
463
463
  "title": "Charade"
464
464
  },
@@ -468,7 +468,7 @@ Answer (73 lines)
468
468
  "path": "Films/His Girl Friday.md",
469
469
  "set": {
470
470
  "table": "main.public.film",
471
- "id": "01a16f9b-2314-7ec6-b8a4-9769d2eb74ed",
471
+ "id": "01a17739-8ba4-7677-ace4-2eeadf460ac6",
472
472
  "director": "Howard Hawks",
473
473
  "title": "His Girl Friday"
474
474
  },
@@ -478,7 +478,7 @@ Answer (73 lines)
478
478
  "path": "Films/Metropolis.md",
479
479
  "set": {
480
480
  "table": "main.public.film",
481
- "id": "01a16f9b-2314-790b-b772-e40f5d94e4b1",
481
+ "id": "01a17739-8ba4-7812-9a1d-e09d7a786e91",
482
482
  "director": "Fritz Lang",
483
483
  "title": "Metropolis"
484
484
  },
@@ -488,7 +488,7 @@ Answer (73 lines)
488
488
  "path": "Films/Nosferatu.md",
489
489
  "set": {
490
490
  "table": "main.public.film",
491
- "id": "01a16f9b-2314-75a4-9a76-5d74cf364168",
491
+ "id": "01a17739-8ba4-7dfd-8fba-2a8d475b8c71",
492
492
  "director": "F. W. Murnau",
493
493
  "title": "Nosferatu"
494
494
  },
@@ -498,7 +498,7 @@ Answer (73 lines)
498
498
  "path": "Films/The Gold Rush.md",
499
499
  "set": {
500
500
  "table": "main.public.film",
501
- "id": "01a16f9b-2314-70f1-89fe-b68f7e2d3b9f",
501
+ "id": "01a17739-8ba5-7a2b-8930-9fa5a3ce1c1d",
502
502
  "director": "Charlie Chaplin",
503
503
  "title": "The Gold Rush"
504
504
  },
@@ -519,7 +519,7 @@ Answer (73 lines)
519
519
  }
520
520
  ],
521
521
  "commits": [{ "table": "main.public.film", "version": 1, "operation": "ADOPT ROWS" }],
522
- "migration_id": "20261024T203000-adopt_rows-adopt-5-notes-into-film"
522
+ "migration_id": "20261026T080022-adopt_rows-adopt-5-notes-into-film"
523
523
  }
524
524
  ```
525
525
 
@@ -536,7 +536,7 @@ Answer:
536
536
  ```markdown
537
537
  ---
538
538
  table: main.public.film
539
- id: 01a16f9b-2314-76cc-a0b0-3737ae7c5e61
539
+ id: 01a17739-8ba4-75cd-8d89-591ad0529c6e
540
540
  year: 1963
541
541
  minutes: 113
542
542
  status: watched
@@ -623,7 +623,7 @@ Answer:
623
623
  "files_created": 1,
624
624
  "planned_creates": ["datanotes/main/public/film/Battleship Potemkin.md"],
625
625
  "commits": [{ "table": "main.public.film", "version": 2, "operation": "WRITE" }],
626
- "migration_id": "20261024T203000-insert_row-insert-into-film"
626
+ "migration_id": "20261026T080022-insert_row-insert-into-film"
627
627
  }
628
628
  ```
629
629
 
@@ -655,7 +655,7 @@ Answer:
655
655
  "op": "add_column",
656
656
  "planned_texts": ["datanotes/metastore.yaml"],
657
657
  "commits": [{ "table": "main.public.film", "version": 4, "operation": "ADD COLUMNS" }],
658
- "migration_id": "20261024T203002-add_column-add-column-added_by-to-film"
658
+ "migration_id": "20261026T080023-add_column-add-column-added_by-to-film"
659
659
  }
660
660
  ```
661
661
 
@@ -683,7 +683,7 @@ Answer:
683
683
  "files_created": 1,
684
684
  "planned_creates": ["datanotes/main/public/film/Night of the Living Dead.md"],
685
685
  "commits": [{ "table": "main.public.film", "version": 5, "operation": "WRITE" }],
686
- "migration_id": "20261024T203002-insert_row-insert-into-film"
686
+ "migration_id": "20261026T080023-insert_row-insert-into-film"
687
687
  }
688
688
  ```
689
689
 
@@ -692,7 +692,7 @@ Answer:
692
692
  ```markdown
693
693
  ---
694
694
  table: main.public.film
695
- id: 01a16f9b-2919-7508-87ca-f5b7fa31c65d
695
+ id: 01a17739-91a8-7d50-b7e0-af83bffa43f7
696
696
  director: George A. Romero
697
697
  minutes: 96
698
698
  status: to watch
@@ -734,7 +734,7 @@ Answer:
734
734
  ```markdown
735
735
  ---
736
736
  table: main.public.film
737
- id: 01a16f9b-2ba1-7ac8-8f00-71d7b7fe5143
737
+ id: 01a17739-943f-76a1-9d85-9c3352710227
738
738
  year: 1924
739
739
  minutes: 45
740
740
  status: to watch
@@ -762,7 +762,7 @@ Answer:
762
762
  "op": "alter_table",
763
763
  "planned_texts": ["datanotes/metastore.yaml"],
764
764
  "commits": [{ "table": "main.public.film", "version": 7, "operation": "ALTER TABLE" }],
765
- "migration_id": "20261024T203004-alter_table-alter-table-main-public-film"
765
+ "migration_id": "20261026T080026-alter_table-alter-table-main-public-film"
766
766
  }
767
767
  ```
768
768
 
@@ -825,7 +825,7 @@ This is the table as you inferred, corrected and extended it on this page. Its d
825
825
 
826
826
  ```yaml
827
827
  # Metastore (Datanotes): catalogs, schemas, tables, columns, views.
828
- id: 01a16f9b-22f5-7be8-a003-1d43db75c224
828
+ id: 01a17739-8b2f-792b-9de9-2dce203b6c01
829
829
  config:
830
830
  history: true
831
831
 
@@ -100,9 +100,9 @@ Answer:
100
100
  "table": "main.public.watch",
101
101
  "total": 3,
102
102
  "rows": [
103
- { "date(occurred_at)": "2026-10-19", "viewings": 1 },
104
- { "date(occurred_at)": "2026-10-22", "viewings": 1 },
105
- { "date(occurred_at)": "2026-10-23", "viewings": 1 }
103
+ { "date(occurred_at)": "2026-10-21", "viewings": 1 },
104
+ { "date(occurred_at)": "2026-10-24", "viewings": 1 },
105
+ { "date(occurred_at)": "2026-10-25", "viewings": 1 }
106
106
  ]
107
107
  }
108
108
  ```
@@ -138,10 +138,10 @@ Answer:
138
138
  {
139
139
  "path": "datanotes/main/public/film/Night of the Living Dead.md",
140
140
  "title": "Night of the Living Dead",
141
- "watched_on": "2026-10-23",
141
+ "watched_on": "2026-10-25",
142
142
  "viewing": [
143
143
  {
144
- "path": "datanotes/main/public/watch/2026-10-23T2200.md",
144
+ "path": "datanotes/main/public/watch/2026-10-25T2200.md",
145
145
  "where": "cinema",
146
146
  "with": "Oscar"
147
147
  }
@@ -150,10 +150,10 @@ Answer:
150
150
  {
151
151
  "path": "datanotes/main/public/film/The Gold Rush.md",
152
152
  "title": "The Gold Rush",
153
- "watched_on": "2026-10-22",
153
+ "watched_on": "2026-10-24",
154
154
  "viewing": [
155
155
  {
156
- "path": "datanotes/main/public/watch/2026-10-22T2030.md",
156
+ "path": "datanotes/main/public/watch/2026-10-24T2030.md",
157
157
  "where": "home",
158
158
  "with": "my sister"
159
159
  }
@@ -332,23 +332,23 @@ Answer:
332
332
  "total": 3,
333
333
  "rows": [
334
334
  {
335
- "path": "datanotes/main/public/watch/2026-10-19T2115.md",
335
+ "path": "datanotes/main/public/watch/2026-10-21T2115.md",
336
336
  "film": "[[Charade]]",
337
- "occurred_at": "2026-10-19T21:15",
337
+ "occurred_at": "2026-10-21T21:15",
338
338
  "where": "home",
339
339
  "place": "my living room"
340
340
  },
341
341
  {
342
- "path": "datanotes/main/public/watch/2026-10-22T2030.md",
342
+ "path": "datanotes/main/public/watch/2026-10-24T2030.md",
343
343
  "film": "[[The Gold Rush]]",
344
- "occurred_at": "2026-10-22T20:30",
344
+ "occurred_at": "2026-10-24T20:30",
345
345
  "where": "home",
346
346
  "place": "my living room"
347
347
  },
348
348
  {
349
- "path": "datanotes/main/public/watch/2026-10-23T2200.md",
349
+ "path": "datanotes/main/public/watch/2026-10-25T2200.md",
350
350
  "film": "[[Night of the Living Dead]]",
351
- "occurred_at": "2026-10-23T22:00",
351
+ "occurred_at": "2026-10-25T22:00",
352
352
  "where": "cinema",
353
353
  "place": "the film club downtown"
354
354
  }
@@ -459,12 +459,12 @@ Answer:
459
459
  "total": 7,
460
460
  "rows": [
461
461
  {
462
- "path": "datanotes/main/public/watch/2026-10-19T2115.md",
463
- "occurred_at": "2026-10-19T21:15"
462
+ "path": "datanotes/main/public/watch/2026-10-21T2115.md",
463
+ "occurred_at": "2026-10-21T21:15"
464
464
  },
465
465
  {
466
- "path": "datanotes/main/public/watch/2026-10-15T2130.md",
467
- "occurred_at": "2026-10-15T21:30"
466
+ "path": "datanotes/main/public/watch/2026-10-17T2130.md",
467
+ "occurred_at": "2026-10-17T21:30"
468
468
  }
469
469
  ]
470
470
  }
@@ -31,7 +31,7 @@ Answer:
31
31
  "files_created": 1,
32
32
  "planned_creates": ["datanotes/main/public/film/Metropolis.md"],
33
33
  "commits": [{ "table": "main.public.film", "version": 1, "operation": "WRITE" }],
34
- "migration_id": "20261024T202950-insert_row-insert-into-film"
34
+ "migration_id": "20261026T080012-insert_row-insert-into-film"
35
35
  }
36
36
  ```
37
37
 
@@ -40,7 +40,7 @@ Answer:
40
40
  ```markdown
41
41
  ---
42
42
  table: main.public.film
43
- id: 01a16f9a-fc9a-7240-be4c-cdf811716ef7
43
+ id: 01a17739-663d-79f3-9d66-608a7164e699
44
44
  title: Metropolis
45
45
  year: 1927
46
46
  minutes: 153
@@ -103,7 +103,7 @@ Answer:
103
103
  }
104
104
  ],
105
105
  "commits": [{ "table": "main.public.film", "version": 6, "operation": "UPDATE" }],
106
- "migration_id": "20261024T202955-update_row-update-film"
106
+ "migration_id": "20261026T080017-update_row-update-film"
107
107
  }
108
108
  ```
109
109
 
@@ -174,7 +174,7 @@ Answer:
174
174
  }
175
175
  ],
176
176
  "commits": [{ "table": "main.public.film", "version": 8, "operation": "UPDATE" }],
177
- "migration_id": "20261024T202955-update_row-update-film-3",
177
+ "migration_id": "20261026T080017-update_row-update-film-3",
178
178
  "ambiguous_mentions": [
179
179
  {
180
180
  "path": "datanotes/main/public/film/Nosferatu.md",
@@ -193,7 +193,7 @@ Answer:
193
193
  ```markdown
194
194
  ---
195
195
  table: main.public.film
196
- id: 01a16f9b-0dbb-7154-b84c-122a05f5cb15
196
+ id: 01a17739-767a-77ab-a08d-62945d014f77
197
197
  title: Nosferatu
198
198
  year: 1922
199
199
  status: watched
@@ -246,7 +246,7 @@ Answer:
246
246
  }
247
247
  ],
248
248
  "commits": [{ "table": "main.public.film", "version": 9, "operation": "UPDATE" }],
249
- "migration_id": "20261024T202955-update_row-update-film-4"
249
+ "migration_id": "20261026T080017-update_row-update-film-4"
250
250
  }
251
251
  ```
252
252
 
@@ -332,7 +332,7 @@ Answer:
332
332
  }
333
333
  ],
334
334
  "commits": [{ "table": "main.public.film", "version": 10, "operation": "UPDATE" }],
335
- "migration_id": "20261024T202955-update_rows-update-film"
335
+ "migration_id": "20261026T080017-update_rows-update-film"
336
336
  }
337
337
  ```
338
338
 
@@ -370,7 +370,7 @@ Answer:
370
370
  ],
371
371
  "planned_creates": ["datanotes/main/public/film/Battleship Potemkin.md"],
372
372
  "commits": [{ "table": "main.public.film", "version": 11, "operation": "MERGE" }],
373
- "migration_id": "20261024T202955-merge_rows-merge-2-rows-into-film"
373
+ "migration_id": "20261026T080017-merge_rows-merge-2-rows-into-film"
374
374
  }
375
375
  ```
376
376
 
@@ -431,7 +431,7 @@ Answer:
431
431
  }
432
432
  ],
433
433
  "commits": [{ "table": "main.public.film", "version": 12, "operation": "DELETE" }],
434
- "migration_id": "20261024T202955-delete_rows-archive-1-rows-of-film"
434
+ "migration_id": "20261026T080017-delete_rows-archive-1-rows-of-film"
435
435
  }
436
436
  ```
437
437
 
@@ -568,6 +568,6 @@ Answer (51 lines)
568
568
  { "table": "main.public.film", "version": 22, "operation": "MOVE ROWS" },
569
569
  { "table": "main.public.watchlist", "version": 1, "operation": "MOVE ROWS" }
570
570
  ],
571
- "migration_id": "20261024T202955-move_rows-move-4-rows-from-film-to-watch"
571
+ "migration_id": "20261026T080017-move_rows-move-4-rows-from-film-to-watch"
572
572
  }
573
573
  ```
@@ -87,7 +87,7 @@ Answer (62 lines)
87
87
  {
88
88
  "start": 81,
89
89
  "end": 86,
90
- "context": "-a4e8-0331bc07052a\nname: Cary Grant\n---\n\n"
90
+ "context": "-8ec2-3a91780e27f5\nname: Cary Grant\n---\n\n"
91
91
  }
92
92
  ]
93
93
  },
@@ -3,7 +3,7 @@
3
3
  > The datanotes command and its options.
4
4
 
5
5
  ```text
6
- datanotes 3.4.2: Markdown notes as database tables, for you and your agents.
6
+ datanotes 3.5.0: Markdown notes as database tables, for you and your agents.
7
7
 
8
8
 
9
9
  Usage:
@@ -20,6 +20,8 @@ Usage:
20
20
  datanotes extension update | rollback | remove <name> --database <folder> change an installed one
21
21
  datanotes extension list --database <folder> the installed extensions
22
22
  datanotes extension check <folder> check an extension's repository (for its authors)
23
+ datanotes pack-merge <base> <ours> <theirs> [<path>] git's merge driver of packs of rows, row by row
24
+ (git sync registers it: merge.datanotes-pack.driver)
23
25
  Every command takes --vault <folder> instead of --database for a vault holding the database.
24
26
  datanotes mcp --database <folder> MCP server over stdio, on the engine running on the folder
25
27
  (or one it starts for the session)
@@ -20,7 +20,7 @@ Examples: [Tables](https://datanotes.dev/guides/database-objects/tables/), [View
20
20
 
21
21
  **Describe Table** · read-only
22
22
 
23
- Describe a table (`catalog.schema.table` or a shorter unambiguous name): `qualified`, `name`, `catalog`, `schema`, `description`, `aliases`, `layout` (`folder`, `row_folder`: the folder pattern of new rows, `archive`), `options` (`partition`, `filename`, `archive`, `exclude`), typed `columns` with label/aliases/description and the dense flag, `untyped_fields`, `row_count`, and when set: `tags`, `bases` (the bases it inherits from, in order), `constraints` (its CHECKs, name → where), `constraint_hints` (their own hints, name → hint), `instructions` (how to fill a row: follow them before you write one), `row_template`, `history`, `autolink_target`, `fields` (their values).
23
+ Describe a table (`catalog.schema.table` or a shorter unambiguous name): `qualified`, `name`, `catalog`, `schema`, `description`, `aliases`, `layout` (`folder`, `row_folder`: the folder pattern of new rows, `archive`), `options` (`partition`, `filename`, `archive`, `exclude`, `pack`, `pack_after_days`), typed `columns` with label/aliases/description and the dense flag, `untyped_fields`, `row_count`, and when set: `tags`, `bases` (the bases it inherits from, in order), `constraints` (its CHECKs, name → where), `constraint_hints` (their own hints, name → hint), `instructions` (how to fill a row: follow them before you write one), `row_template`, `history`, `autolink_target`, `fields` (their values).
24
24
 
25
25
  Before adding a column (e.g. a new metric), read `columns` and match your concept against `name`, `label`, `aliases` and `description`; reuse an existing column when it means the same.
26
26
 
@@ -114,3 +114,32 @@ Examples: [Find notes by meaning](https://datanotes.dev/guides/search/semantic/)
114
114
  | Parameter | Type | Required | Default | Description |
115
115
  | --------- | ------- | -------- | ------- | ----------------------------------------------- |
116
116
  | `rebuild` | boolean | | `false` | Re-embed every note, not only the changed ones. |
117
+
118
+ ## `pack_rows`
119
+
120
+ **Pack Rows** · destructive write
121
+
122
+ Put the rows of a table’s closed periods into packs: one note per period (`<table> <key>.md` in the table’s folder), a `##` section per row, `key:: value` lines for its values. The table needs a `pack` option (create_table / alter_table, e.g. `"{occurred_at:YYYY-MM}"`). A period is closed `pack_after_days` days after it ends (default 7); `key` packs only that period. Nothing changes for the engine and its readers: select_rows, links, the graph and the change feed still see each row at its path, with its values, text and file_modified (no change event, no new version). Links from other notes to a packed row become `[[pack#row]]`, which Obsidian follows too. A row stays a note of its own (`excluded`, with the reason) when it changed in the last 24 hours, has a `#` or `##` heading in its text, is not at the path its values give, or its note is not in the form the engine writes. The engine also packs on its own once a day (setting `autoPack`). Needs the engine service. Journaled: undo_migration takes the rows out again. Use `dry_run` to see `packs` first.
123
+
124
+ Examples: [Packs of rows](https://datanotes.dev/guides/database-objects/packs/).
125
+
126
+ | Parameter | Type | Required | Default | Description |
127
+ | --------- | --------- | -------- | ------- | ----------- |
128
+ | `table` | string | yes | | |
129
+ | `key` | string \\ | null | | `null` |
130
+ | `dry_run` | boolean | | `false` | |
131
+
132
+ ## `unpack_rows`
133
+
134
+ **Unpack Rows** · destructive write
135
+
136
+ Take rows out of their packs as notes of their own, at their paths, with the text and the modification time they had (the opposite of pack_rows): every pack of the table, the pack of `key`, or only the rows `where` matches (as select_rows filters). A pack left without rows goes. Links `[[pack#row]]` from other notes become links to the notes. Needed before alter_table changes the table’s `pack`, `partition`, `filename` or name. Silent for the rows, journaled (undo_migration packs them again).
137
+
138
+ Examples: [Packs of rows](https://datanotes.dev/guides/database-objects/packs/).
139
+
140
+ | Parameter | Type | Required | Default | Description |
141
+ | --------- | --------- | -------- | ------- | ----------- |
142
+ | `table` | string | yes | | |
143
+ | `key` | string \\ | null | | `null` |
144
+ | `where` | object \\ | null | | `null` |
145
+ | `dry_run` | boolean | | `false` | |
@@ -19,7 +19,7 @@ Query rows: `{table, total, rows: [{path, ...}]}`. `table` may be a view, `syste
19
19
  * `this_note`: vault path of the note the result is for: `{this.*}` tokens in a view’s query and in `where` read it (`{this.date}`, `{this.link}`, `{this.name}`, `{this.path}`); without it (or a front matter key the note lacks) a token stays as written and matches no row, whatever the operator (`within` included).
20
20
  * Ordered enums compare by position. Time travel: `version` (of one table) or `timestamp`. With `timestamp` a join, a view, a union or `base:<name>` reads every table in it as of that time (`versions`: the version read of each); a table that did not exist then adds no rows to a union or base. `version` is refused where several tables are read; information_schema has no history. Views and bases keep today’s definition, and links followed by `where` (`project.kind`, `within`) or `expand` read the linked rows as they are now.
21
21
 
22
- Examples: [Catalogs, schemas and the database](https://datanotes.dev/guides/database-objects/catalogs-and-schemas/), [Tables](https://datanotes.dev/guides/database-objects/tables/), [Columns and types](https://datanotes.dev/guides/database-objects/columns/), [Views](https://datanotes.dev/guides/database-objects/views/), [Bases](https://datanotes.dev/guides/database-objects/bases/), [Relations and hierarchies](https://datanotes.dev/guides/database-objects/relations/), [Validation](https://datanotes.dev/guides/data-quality/validation/), [CHECK constraints](https://datanotes.dev/guides/data-quality/check-constraints/), [Hand edits](https://datanotes.dev/guides/data-quality/hand-edits/), [Read and filter rows](https://datanotes.dev/guides/read-and-write/read-and-filter/), [Joins and aggregates](https://datanotes.dev/guides/read-and-write/joins-and-aggregates/), [Write and move rows](https://datanotes.dev/guides/read-and-write/write-rows/), [Batch writes](https://datanotes.dev/guides/read-and-write/batch/), [Import existing notes](https://datanotes.dev/guides/read-and-write/import-notes/), [Journal and undo](https://datanotes.dev/guides/history/journal-and-undo/), [Table history and time travel](https://datanotes.dev/guides/history/table-history/), [Restore a table or the database](https://datanotes.dev/guides/history/restore/).
22
+ Examples: [Catalogs, schemas and the database](https://datanotes.dev/guides/database-objects/catalogs-and-schemas/), [Tables](https://datanotes.dev/guides/database-objects/tables/), [Columns and types](https://datanotes.dev/guides/database-objects/columns/), [Views](https://datanotes.dev/guides/database-objects/views/), [Bases](https://datanotes.dev/guides/database-objects/bases/), [Relations and hierarchies](https://datanotes.dev/guides/database-objects/relations/), [Packs of rows](https://datanotes.dev/guides/database-objects/packs/), [Validation](https://datanotes.dev/guides/data-quality/validation/), [CHECK constraints](https://datanotes.dev/guides/data-quality/check-constraints/), [Hand edits](https://datanotes.dev/guides/data-quality/hand-edits/), [Read and filter rows](https://datanotes.dev/guides/read-and-write/read-and-filter/), [Joins and aggregates](https://datanotes.dev/guides/read-and-write/joins-and-aggregates/), [Write and move rows](https://datanotes.dev/guides/read-and-write/write-rows/), [Batch writes](https://datanotes.dev/guides/read-and-write/batch/), [Import existing notes](https://datanotes.dev/guides/read-and-write/import-notes/), [Journal and undo](https://datanotes.dev/guides/history/journal-and-undo/), [Table history and time travel](https://datanotes.dev/guides/history/table-history/), [Restore a table or the database](https://datanotes.dev/guides/history/restore/).
23
23
 
24
24
  | Parameter | Type | Required | Default | Description |
25
25
  | -------------- | ------------ | ------------ | ------- | ----------- |
@@ -12,7 +12,7 @@ Every write is validated. Writes to rows and to the schema are journaled and und
12
12
 
13
13
  Insert a row (new note). When describe_table gives the table’s `instructions`, fill the row as they say. Values are validated against the columns; the note path comes from the table’s `path`/`filename` patterns (e.g. an event logged today about last night lands in last night’s day folder when you set `occurred_at` to last night). `body` is the note text. Generated / identity columns are computed (do not pass them); CHECK constraints are enforced. Returns `migration_id` (undo) and the created path in `planned_creates`. `body` links the notes it mentions by name, at their first mention (automatic links; off with `autolink: false` or the database’s `autolink` setting). `links_added` lists the links made; `ambiguous_mentions` lists the mentions left as text with the notes they may name: read those, then link the right one with update_row and `autolink: false`, or leave the text.
14
14
 
15
- Examples: [Catalogs, schemas and the database](https://datanotes.dev/guides/database-objects/catalogs-and-schemas/), [Tables](https://datanotes.dev/guides/database-objects/tables/), [Columns and types](https://datanotes.dev/guides/database-objects/columns/), [Bases](https://datanotes.dev/guides/database-objects/bases/), [Relations and hierarchies](https://datanotes.dev/guides/database-objects/relations/), [Validation](https://datanotes.dev/guides/data-quality/validation/), [CHECK constraints](https://datanotes.dev/guides/data-quality/check-constraints/), [Write and move rows](https://datanotes.dev/guides/read-and-write/write-rows/), [Automatic links](https://datanotes.dev/guides/read-and-write/automatic-links/), [Batch writes](https://datanotes.dev/guides/read-and-write/batch/), [Import existing notes](https://datanotes.dev/guides/read-and-write/import-notes/), [Journal and undo](https://datanotes.dev/guides/history/journal-and-undo/), [Table history and time travel](https://datanotes.dev/guides/history/table-history/), [Change data feed](https://datanotes.dev/guides/history/change-data-feed/), [Restore a table or the database](https://datanotes.dev/guides/history/restore/).
15
+ Examples: [Catalogs, schemas and the database](https://datanotes.dev/guides/database-objects/catalogs-and-schemas/), [Tables](https://datanotes.dev/guides/database-objects/tables/), [Columns and types](https://datanotes.dev/guides/database-objects/columns/), [Bases](https://datanotes.dev/guides/database-objects/bases/), [Relations and hierarchies](https://datanotes.dev/guides/database-objects/relations/), [Packs of rows](https://datanotes.dev/guides/database-objects/packs/), [Validation](https://datanotes.dev/guides/data-quality/validation/), [CHECK constraints](https://datanotes.dev/guides/data-quality/check-constraints/), [Write and move rows](https://datanotes.dev/guides/read-and-write/write-rows/), [Automatic links](https://datanotes.dev/guides/read-and-write/automatic-links/), [Batch writes](https://datanotes.dev/guides/read-and-write/batch/), [Import existing notes](https://datanotes.dev/guides/read-and-write/import-notes/), [Journal and undo](https://datanotes.dev/guides/history/journal-and-undo/), [Table history and time travel](https://datanotes.dev/guides/history/table-history/), [Change data feed](https://datanotes.dev/guides/history/change-data-feed/), [Restore a table or the database](https://datanotes.dev/guides/history/restore/).
16
16
 
17
17
  | Parameter | Type | Required | Default | Description |
18
18
  | ---------- | ---------- | -------- | ------- | ----------- |
@@ -34,7 +34,7 @@ Update exactly ONE row, addressed by its `id` (the system UUID) or its `path`; r
34
34
  * `expect`: compare-and-set — `{"status": "open"}` writes only if the row still has those values (a person may have changed it since you read it); otherwise refused, nothing written.
35
35
  * `autolink`: the text written links the notes it mentions by name, at their first mention (off with `autolink: false` or the database’s setting). `links_added` lists the links made; `ambiguous_mentions` the mentions left as text with the notes they may name: read those, then link the right one with update_row and `autolink: false`, or leave the text.
36
36
 
37
- Examples: [Tables](https://datanotes.dev/guides/database-objects/tables/), [Columns and types](https://datanotes.dev/guides/database-objects/columns/), [Bases](https://datanotes.dev/guides/database-objects/bases/), [Relations and hierarchies](https://datanotes.dev/guides/database-objects/relations/), [CHECK constraints](https://datanotes.dev/guides/data-quality/check-constraints/), [Hand edits](https://datanotes.dev/guides/data-quality/hand-edits/), [Write and move rows](https://datanotes.dev/guides/read-and-write/write-rows/), [Automatic links](https://datanotes.dev/guides/read-and-write/automatic-links/), [Batch writes](https://datanotes.dev/guides/read-and-write/batch/), [Journal and undo](https://datanotes.dev/guides/history/journal-and-undo/), [Table history and time travel](https://datanotes.dev/guides/history/table-history/), [Change data feed](https://datanotes.dev/guides/history/change-data-feed/), [Restore a table or the database](https://datanotes.dev/guides/history/restore/).
37
+ Examples: [Tables](https://datanotes.dev/guides/database-objects/tables/), [Columns and types](https://datanotes.dev/guides/database-objects/columns/), [Bases](https://datanotes.dev/guides/database-objects/bases/), [Relations and hierarchies](https://datanotes.dev/guides/database-objects/relations/), [Packs of rows](https://datanotes.dev/guides/database-objects/packs/), [CHECK constraints](https://datanotes.dev/guides/data-quality/check-constraints/), [Hand edits](https://datanotes.dev/guides/data-quality/hand-edits/), [Write and move rows](https://datanotes.dev/guides/read-and-write/write-rows/), [Automatic links](https://datanotes.dev/guides/read-and-write/automatic-links/), [Batch writes](https://datanotes.dev/guides/read-and-write/batch/), [Journal and undo](https://datanotes.dev/guides/history/journal-and-undo/), [Table history and time travel](https://datanotes.dev/guides/history/table-history/), [Change data feed](https://datanotes.dev/guides/history/change-data-feed/), [Restore a table or the database](https://datanotes.dev/guides/history/restore/).
38
38
 
39
39
  | Parameter | Type | Required | Default | Description |
40
40
  | --------------- | ------------ | -------- | ------- | ----------- |
@@ -147,7 +147,7 @@ Several writes that succeed or fail together. `operations`: `[{"op": ..., ...arg
147
147
 
148
148
  The schema changes only through operations like this one: never edit metastore.yaml by hand (no validation, no undo).
149
149
 
150
- Examples: [Views](https://datanotes.dev/guides/database-objects/views/), [Relations and hierarchies](https://datanotes.dev/guides/database-objects/relations/), [CHECK constraints](https://datanotes.dev/guides/data-quality/check-constraints/), [Batch writes](https://datanotes.dev/guides/read-and-write/batch/), [Restore a table or the database](https://datanotes.dev/guides/history/restore/).
150
+ Examples: [Views](https://datanotes.dev/guides/database-objects/views/), [Relations and hierarchies](https://datanotes.dev/guides/database-objects/relations/), [Packs of rows](https://datanotes.dev/guides/database-objects/packs/), [CHECK constraints](https://datanotes.dev/guides/data-quality/check-constraints/), [Batch writes](https://datanotes.dev/guides/read-and-write/batch/), [Restore a table or the database](https://datanotes.dev/guides/history/restore/).
151
151
 
152
152
  | Parameter | Type | Required | Default | Description |
153
153
  | ------------ | --------- | -------- | ------- | ----------- |
@@ -15,14 +15,14 @@ Create a table: `name`, in `catalog` and `schema` (default: the database’s def
15
15
  * `description` (required): what a row of the table records. Every column needs its own `description` too, what it holds: the table is refused otherwise, with every missing one listed. Descriptions are how agents and people understand the schema, so write them for a reader who never saw the notes. The engine writes those of `id` and of `identity` “uuid” / “created” columns.
16
16
  * `columns`: specs like `{"name": "kind", "type": "enum", "values": ["a","b"], "required": true, "description": "What kind of entry the row is"}`. Types: string, integer, number, boolean, date, datetime, enum, link, list (+ `items`), json. Attributes: required, default (`"{now}"`/`"{today}"`), min, max, step, values, `ordered` (enum in order: filters/avg use the position), ref (link target, full `catalog.schema.table` or “self”), unique, label, description, aliases, `generated` (e.g. `"date(occurred_at)"`, `"{name} {surname}"`), `identity` (“filename” | “created”), `vocabulary` (allowed values = names of another table’s rows), `hierarchy: true` (one parent link to its own table), `acyclic: true` (links to rows of its own table that never loop back, such as tasks blocked by tasks; with `inverse`, the rule holds on both sides), `dense` (written on every row), `inverse` on a link column: the column of the `ref` table that links back (`friends` ↔ `friends`, `children` ↔ `parents`, `partner` ↔ `partner`); writes keep both sides in step.
17
17
  * `time_column`: rows named by that date/datetime; otherwise `filename` (e.g. `"{name}"`).
18
- * The table’s options, as alter_table names them: `aliases` (how agents find it, with the description), `partition`, `exclude`, `archive`, `history` (keep its versions; default: the database’s setting), `autolink_target` (true / false: may automatic links link its rows? default: yes, unless its rows are named by a date or a time, as with `time_column`).
18
+ * The table’s options, as alter_table names them: `aliases` (how agents find it, with the description), `partition`, `exclude`, `archive`, `history` (keep its versions; default: the database’s setting), `autolink_target` (true / false: may automatic links link its rows? default: yes, unless its rows are named by a date or a time, as with `time_column`). `pack` (a template of a date column, such as `"{occurred_at:YYYY-MM}"`: once a period is over, one note holds its rows, `##` section by section; select_rows still gives one row each) and `pack_after_days` (default 7): see pack_rows.
19
19
  * `row_template`: markdown body of new rows. Shared columns come from bases: `set_table_bases`.
20
20
  * `instructions`: how to fill a row, for agents (Markdown); describe_table returns them.
21
21
  * `tags`: key/value labels of the table, strings (`{"domain": "health"}`); list_tables and describe_table return them, alter_table changes them. Enum columns: `values` and, for each, its meaning in `value_descriptions` (`{"stated": "the user said it", …}`): describe_table shows them and a refused value lists them. Generated link columns: `generated: "link({occurred_at:YYYY-MM-DD})"` with `ref` = the days table links each row to the row of `ref` named by the template (its folder read from the schema; the target may not exist yet). Generated aggregates of a json column holding a list of objects (integer or number columns; avg: number): `count(feeds)`, `count(feeds.min)` (the items with a min), `sum(feeds.min)`, `avg(…)`, `min(…)`, `max(…)`, with an optional filter on one field: `count(feeds, side = "left")`, `sum(feeds.min, min > 0)` (=, !=, <, <=, >, >=; “text”, a number, true or false). Recomputed on every write; no list: count and sum 0, avg/min/max empty; a field that is not a number: empty. Filtered link targets (link columns): `ref_where` (a `where` every target must match) and `ref_by` (a column of the row picks the target level: `{"column": "kind", "levels": ["area", "project", "workstream"]}` or `{"column": "kind", "map": {"area": null, "project": ["area"]}}`; null = top level, no link; `target_column` when the target names it differently: an enum target column must have every value the map allows, a string one takes any value). A link to no row of the table, or to the row itself, is refused too. Existing rows that break them refuse the change; `list_link_targets` lists the valid targets of a row.
22
22
 
23
23
  The schema changes only through operations like this one: never edit metastore.yaml by hand (no validation, no undo).
24
24
 
25
- Examples: [Catalogs, schemas and the database](https://datanotes.dev/guides/database-objects/catalogs-and-schemas/), [Tables](https://datanotes.dev/guides/database-objects/tables/), [Columns and types](https://datanotes.dev/guides/database-objects/columns/), [Bases](https://datanotes.dev/guides/database-objects/bases/), [Relations and hierarchies](https://datanotes.dev/guides/database-objects/relations/), [Write and move rows](https://datanotes.dev/guides/read-and-write/write-rows/), [Automatic links](https://datanotes.dev/guides/read-and-write/automatic-links/), [Import existing notes](https://datanotes.dev/guides/read-and-write/import-notes/), [Table history and time travel](https://datanotes.dev/guides/history/table-history/).
25
+ Examples: [Catalogs, schemas and the database](https://datanotes.dev/guides/database-objects/catalogs-and-schemas/), [Tables](https://datanotes.dev/guides/database-objects/tables/), [Columns and types](https://datanotes.dev/guides/database-objects/columns/), [Bases](https://datanotes.dev/guides/database-objects/bases/), [Relations and hierarchies](https://datanotes.dev/guides/database-objects/relations/), [Packs of rows](https://datanotes.dev/guides/database-objects/packs/), [Write and move rows](https://datanotes.dev/guides/read-and-write/write-rows/), [Automatic links](https://datanotes.dev/guides/read-and-write/automatic-links/), [Import existing notes](https://datanotes.dev/guides/read-and-write/import-notes/), [Table history and time travel](https://datanotes.dev/guides/history/table-history/).
26
26
 
27
27
  | Parameter | Type | Required | Default | Description |
28
28
  | ----------------- | ------------ | -------- | ------- | ----------- |
@@ -34,6 +34,8 @@ Examples: [Catalogs, schemas and the database](https://datanotes.dev/guides/data
34
34
  | `time_column` | string \\ | null | | `null` |
35
35
  | `filename` | string \\ | null | | `null` |
36
36
  | `partition` | string \\ | null | | `null` |
37
+ | `pack` | string \\ | null | | `null` |
38
+ | `pack_after_days` | integer \\ | null | | `null` |
37
39
  | `exclude` | string\[] \\ | null | | `null` |
38
40
  | `archive` | string \\ | null | | `null` |
39
41
  | `aliases` | string\[] \\ | null | | `null` |
@@ -65,11 +67,11 @@ Examples: [Tables](https://datanotes.dev/guides/database-objects/tables/).
65
67
 
66
68
  **Alter Table** · destructive write
67
69
 
68
- Alter a table: move it (`catalog`, `schema`), rename it (`name`), change `partition` / `filename` patterns, `description` (changes, never goes: “” is refused), `aliases`, `exclude` folders, `row_template` (“” removes it), `instructions` (how to fill a row, for agents; “” removes them), `history` (true | false | “default”), `autolink_target` (true | false | “default”: may automatic links link its rows? default: yes, unless its rows are named by a date or a time) or `tags` (key/value labels merged into the table’s, e.g. `{"domain": "health"}`; a `null` value removes that tag; `clear_tags: true` removes every tag first, so with `tags` the table keeps exactly those), `archive` (folder name inside the table folder) or `fields` (untyped fields, name → default; a `null` value removes one). Rows, archive and transaction log follow; links are updated; renamed rows keep their old name in `aliases`. A change of metadata only (tags, description, aliases, history, autolink_target, row_template, instructions, fields) rewrites the table entry alone: no row is moved or rewritten. Refused when a row cannot fill a pattern. Renaming or moving the table rewrites what names it in other definitions: the `ref` and `vocabulary` of the columns of other tables and bases (lists of links included) and the tables of views (`table`, `union[].table`, `join.table`); `schema_references` lists them. CHECKs and `ref_where` / `ref_by` that read it through a link follow the new `ref`. Use `dry_run` to see `planned_moves`; undo with undo_migration.
70
+ Alter a table: move it (`catalog`, `schema`), rename it (`name`), change `partition` / `filename` patterns, `description` (changes, never goes: “” is refused), `aliases`, `exclude` folders, `row_template` (“” removes it), `instructions` (how to fill a row, for agents; “” removes them), `history` (true | false | “default”), `autolink_target` (true | false | “default”: may automatic links link its rows? default: yes, unless its rows are named by a date or a time) or `tags` (key/value labels merged into the table’s, e.g. `{"domain": "health"}`; a `null` value removes that tag; `clear_tags: true` removes every tag first, so with `tags` the table keeps exactly those), `archive` (folder name inside the table folder) or `fields` (untyped fields, name → default; a `null` value removes one). Rows, archive and transaction log follow; links are updated; renamed rows keep their old name in `aliases`. A change of metadata only (tags, description, aliases, history, autolink_target, row_template, instructions, fields, pack_after_days) rewrites the table entry alone: no row is moved or rewritten. `pack` sets the period of the table’s packs of rows (`"{occurred_at:YYYY-MM}"`; “” removes it), `pack_after_days` the days after a period’s end before its rows go into a pack (“default”: 7). While rows are in packs, changing `pack`, `partition`, `filename` or the table’s name is refused: unpack_rows first. Refused when a row cannot fill a pattern. Renaming or moving the table rewrites what names it in other definitions: the `ref` and `vocabulary` of the columns of other tables and bases (lists of links included) and the tables of views (`table`, `union[].table`, `join.table`); `schema_references` lists them. CHECKs and `ref_where` / `ref_by` that read it through a link follow the new `ref`. Use `dry_run` to see `planned_moves`; undo with undo_migration.
69
71
 
70
72
  The schema changes only through operations like this one: never edit metastore.yaml by hand (no validation, no undo).
71
73
 
72
- Examples: [Catalogs, schemas and the database](https://datanotes.dev/guides/database-objects/catalogs-and-schemas/), [Tables](https://datanotes.dev/guides/database-objects/tables/), [Automatic links](https://datanotes.dev/guides/read-and-write/automatic-links/), [Import existing notes](https://datanotes.dev/guides/read-and-write/import-notes/), [Table history and time travel](https://datanotes.dev/guides/history/table-history/).
74
+ Examples: [Catalogs, schemas and the database](https://datanotes.dev/guides/database-objects/catalogs-and-schemas/), [Tables](https://datanotes.dev/guides/database-objects/tables/), [Packs of rows](https://datanotes.dev/guides/database-objects/packs/), [Automatic links](https://datanotes.dev/guides/read-and-write/automatic-links/), [Import existing notes](https://datanotes.dev/guides/read-and-write/import-notes/), [Table history and time travel](https://datanotes.dev/guides/history/table-history/).
73
75
 
74
76
  | Parameter | Type | Required | Default | Description |
75
77
  | ----------------- | ------------ | -------------- | ------- | -------------------------------------------------------------------- |
@@ -78,6 +80,8 @@ Examples: [Catalogs, schemas and the database](https://datanotes.dev/guides/data
78
80
  | `schema` | string \\ | null | | `null` |
79
81
  | `name` | string \\ | null | | `null` |
80
82
  | `partition` | string \\ | null | | `null` |
83
+ | `pack` | string \\ | null | | `null` |
84
+ | `pack_after_days` | integer \\ | `"default"` \\ | null | |
81
85
  | `filename` | string \\ | null | | `null` |
82
86
  | `description` | string \\ | null | | `null` |
83
87
  | `aliases` | string\[] \\ | null | | `null` |