letopis 1.3.0 → 1.4.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (56) hide show
  1. package/AGENT-CHEATSHEET.en.md +14 -6
  2. package/AGENT-CHEATSHEET.md +18 -10
  3. package/CHANGELOG.md +247 -0
  4. package/README.en.md +146 -58
  5. package/README.md +129 -51
  6. package/dist/acl.d.ts +11 -5
  7. package/dist/admin.d.ts +30 -14
  8. package/dist/admin.js +42 -16
  9. package/dist/auth.d.ts +61 -17
  10. package/dist/auth.js +16 -2
  11. package/dist/cache.d.ts +5 -1
  12. package/dist/chain.d.ts +91 -15
  13. package/dist/chain.js +47 -4
  14. package/dist/cli.js +6 -0
  15. package/dist/errors.d.ts +2 -2
  16. package/dist/errors.js +5 -5
  17. package/dist/import.js +7 -0
  18. package/dist/index.d.ts +74 -45
  19. package/dist/index.js +18 -4
  20. package/dist/load.d.ts +10 -3
  21. package/dist/load.js +19 -4
  22. package/dist/mcp.d.ts +20 -11
  23. package/dist/mcp.js +4 -4
  24. package/dist/model.d.ts +4 -1
  25. package/dist/ops.d.ts +58 -18
  26. package/dist/ops.js +58 -18
  27. package/dist/project.d.ts +44 -12
  28. package/dist/project.js +90 -8
  29. package/dist/registry.d.ts +4 -1
  30. package/dist/registry.js +4 -1
  31. package/dist/sql.d.ts +4 -1
  32. package/dist/sql.js +12 -4
  33. package/dist/sync.js +11 -5
  34. package/dist/tx.js +10 -1
  35. package/dist/types.d.ts +14 -7
  36. package/dist/types.js +13 -6
  37. package/dist/up.d.ts +65 -25
  38. package/dist/up.js +152 -40
  39. package/dist/validate.d.ts +9 -2
  40. package/dist/validate.js +19 -3
  41. package/dist/watch.d.ts +17 -5
  42. package/dist/watch.js +40 -8
  43. package/dist/write.d.ts +3 -2
  44. package/dist/write.js +29 -10
  45. package/llms.txt +1 -1
  46. package/package.json +1 -1
  47. package/sql/10-core.sql +6 -1
  48. package/sql/50-acl.sql +5 -1
  49. package/sql/60-write.sql +77 -22
  50. package/sql/80-auth.sql +39 -2
  51. package/sql/88-derived.sql +82 -1
  52. package/sql/90-time.sql +75 -38
  53. package/sql/95-seed.system.sql +6 -3
  54. package/sql/99-revision.sql +3 -2
  55. package/sql/README.md +2 -0
  56. package/sql/migrate/0007-secret-account-fk.sql +12 -0
package/README.en.md CHANGED
@@ -1,4 +1,4 @@
1
- # letopis 1.3
1
+ # letopis 1.4
2
2
 
3
3
  **letopis** is a Node.js library: a store of versioned entities on PostgreSQL 18. Every change is saved as a version
4
4
  in a journal. So every object has a history, a state as of any date and a change subscription. The database itself
@@ -130,7 +130,7 @@ What happened here:
130
130
  from the journal.
131
131
 
132
132
  Calling `up()` again is safe: it re-applies the same revision and does not touch the data, and it does not issue the
133
- service key again while the installation still has a valid API key. Next, section 2 explains the design, and sections
133
+ service key again while the installation still has at least one API key that is not revoked. Next, section 2 explains the design, and sections
134
134
  3–13 go through the tasks in order.
135
135
 
136
136
  ## 2. How letopis works
@@ -243,11 +243,16 @@ functions and the system seed. On the first installation, it issues a service ke
243
243
  | `owner`, `app` | the names of the owner and application roles (`letopis_owner`, `letopis_app` by default) |
244
244
 
245
245
  Result: `schema` (the full name), `serviceKey` (on the first installation, and on a re-run when the installation has no
246
- valid API key left; otherwise `null`), `created`, `upgraded`, `revision`, `migrations`, `analyzed`, `owner`, `app`,
246
+ API key left except revoked ones: an expired or disabled key counts too; otherwise `null`), `created`, `upgraded`, `revision`, `migrations`, `analyzed`, `owner`, `app`,
247
247
  `signals` (the signal mode of the installation), `warnings` (without `quiet` they are printed), and also `tenant` and
248
248
  `pglite` if they were requested. On PostgreSQL below 18.2, `up()` warns: such a server could truncate CLOG that unread
249
249
  signals still need — upgrade it.
250
250
 
251
+ Several instances of a service may call `up()` for the same schema at the same time: the installation runs under a lock
252
+ on the schema. The first `up()` installs or upgrades it, the others wait and re-apply the ready schema
253
+ (`created: false`, `upgraded: false`); one of them creates the first `tenant`, the others find it by name. `up()` calls
254
+ for different schemas of one database do not wait for each other.
255
+
251
256
  To hide class names from signals, use the `opaque` mode (or `tenant` if the names are not secret but you want the
252
257
  cache to be precise per tenant); after changing the mode, restart the application processes — they read the mode on
253
258
  connect:
@@ -291,7 +296,9 @@ A repeat with the same name and login duplicates nothing and does not change the
291
296
  session. The login goes into the id as is: `Anna@shop.ru` and `anna@shop.ru` would give two accounts, and the second one
292
297
  would have no password, so write logins in one case. The `created` field in the result tells whether this call created
293
298
  the tenant: for an existing tenant with a new user it is `false`. The roles are set by `user.roles`, a non-empty list
294
- of names (not masks), otherwise `invalid_data`. A repeat does not change the roles of an existing membership: they are
299
+ of names (not masks), otherwise `invalid_data`. The password kind `user.kind` (`PASSWORD` by default) follows the rule
300
+ for a credential kind, as in `auth.setPassword` ([API reference](#dbauth)), otherwise it is `invalid_data` too. A
301
+ repeat does not change the roles of an existing membership: they are
295
302
  changed by editing the membership ([Granting a permission](#granting-a-permission)). A repeat does not bring back
296
303
  what was deleted: if the tenant, the user account or the membership is deleted (the row is gone but its history is in
297
304
  the journal), `createTenant` answers `invalid_data` and writes nothing. The error text names what was deleted and
@@ -363,11 +370,21 @@ const adm = await connect({ dsn: ADMIN, schema: r.schema, token: r.tenant!.token
363
370
 
364
371
  ### Upgrading the package and schema
365
372
 
366
- The schema has an engine revision (currently 6). If a new package version brings a new revision, `connect()` only warns
373
+ The schema has an engine revision (currently 7). If a new package version brings a new revision, `connect()` only warns
367
374
  and names the command. It is `up()` that refuses: a lagging schema without `upgrade: true`, and a schema newer than the
368
- package. So a startup script that calls `up()` on every start will stop after a package update until it gets
375
+ package. To a schema newer than the package (a newer package has already upgraded it, and old application instances
376
+ are still running) `connect()` connects anyway and warns that the letopis package should be updated. So a startup script that calls `up()` on every start will stop after a package update until it gets
369
377
  `upgrade: true`. `up({ dsn, schema, upgrade: true })` applies the migrations from `sql/migrate/` and re-applies the
370
- functions; the data is kept.
378
+ functions; the data is kept. Revision 7 (1.4.1) adds the foreign key `secret.account`: migration `0007` first erases
379
+ the `secret` rows whose accounts are already gone (the credentials of accounts deleted earlier were kept with the
380
+ `deleted_at` mark), and then sets the key.
381
+
382
+ The revision does not change with every version: 1.1–1.4.0 share one (6), while their functions differ. So `up()` also writes
383
+ an engine stamp into the schema comment: the package version and a hash of its SQL files. If the stamp does not match
384
+ the package files, `connect()` warns. When the schema was applied by an older version (or by a version before 1.4,
385
+ which has no stamp), it suggests `up({ schema })`. When a newer version has already upgraded the schema, it suggests
386
+ updating the package: `up()` from the older version would bring the old functions back. A new package version without
387
+ SQL changes keeps the stamp, and there is no warning.
371
388
 
372
389
  The first `up()` after moving to 1.2 builds the journal index `log_class_tenant_at (class, tenant, at)` instead of
373
390
  `log_class_at`: an `asOf` and `withDeleted` slice by class no longer reads the journal of other tenants with the same
@@ -377,8 +394,9 @@ the owner role — then `up()` only drops the old one.
377
394
 
378
395
  The `upgrade` test checks upgrading 1.1.0 and 1.0.1 schemas. The old package installs the schema and the old engine
379
396
  writes the data, then the current version's `up()` runs. After it, the versions and hashes written by the old engine
380
- are in place, `verify({ data: true })` is clean, and the schema catalog matches a clean installation. A 1.1.0 schema is
381
- upgraded with a plain `up()`, since its revision is the same. A 1.0.1 schema is upgraded with `up({ upgrade: true })`.
397
+ are in place, `verify({ data: true })` is clean, and the schema catalog matches a clean installation. Both schemas are
398
+ upgraded with `up({ upgrade: true })`: 1.1.0 (revision 6) with migration `0007`, 1.0.1 (revision 5) with migrations
399
+ `0006` and `0007`.
382
400
 
383
401
  ### Development without PostgreSQL: PGlite
384
402
 
@@ -567,13 +585,21 @@ const Movement = link('Movement', { amount: z.number().int() }, { ends: { Wallet
567
585
  key; the `via` end is single and accepts the target class; the `sum` field is a number; aggregates do not form a
568
586
  cycle; the target and the source are not system or derived classes. A source class is described before the
569
587
  aggregate: an aggregate over a class that does not exist yet is refused. For a tenant class, the source is looked up
570
- among the tenant's classes. `syncModels` describes sources before targets itself. The `of` class is not deleted while
571
- an aggregate of a class of the same tenant names it: first the aggregate is removed from the target's description.
588
+ among the tenant's classes. `syncModels` describes sources before targets itself, and models in a circle (an
589
+ aggregate over the class's own descendant, a cycle of aggregates) it refuses before writing (`invalid_class` naming
590
+ the circle). The `of` class is not deleted while an aggregate of a class of the same tenant names it: first the
591
+ aggregate is removed from the target's description.
572
592
  - Rows get into an aggregate only with the knowledge of someone who reads them all: the field also counts rows that
573
593
  the author cannot see. Only someone who reads all rows of the source and its descendants without conditions can
574
594
  describe or change an aggregate. Only someone who reads all rows of a class and its descendants without conditions,
575
595
  both before the change and after it, can make the class a source (a new descendant of a source, a `parent` change).
576
596
  Otherwise `acl_denied`.
597
+ - A new source is also checked against whoever described the aggregate: they too must read all rows of the class
598
+ without conditions. The describer is the author of the version of the target's `Class` row from which the aggregate
599
+ description became the current one. If they cannot read those rows, the class does not become a source
600
+ (`acl_denied`) until someone who can read them describes the aggregate again. For an aggregate of a System class, the
601
+ describer is checked against System classes, as at description time. A tenant class under a System aggregate is
602
+ checked only for whoever creates it.
577
603
  - The option is inherited, and a descendant replaces it as a whole. Adding or changing an aggregate is a tightening:
578
604
  writes to the target and to the sources wait while the field is recalculated for all rows.
579
605
  - `db.rebuildAggregates(class)` recalculates the field from the current rows, for repair. A version is written only
@@ -608,9 +634,10 @@ ends type is `InferLinks<typeof Product>`. Your own `meta` shape is typed by ext
608
634
 
609
635
  ### Class names
610
636
 
611
- A chain step is named after a class, its alias or a role name. The Proxy parses the names of chain and root methods
612
- (`count`, `sort`, `set`, `sql`, `auth` and others) before classes. So a class with such a name cannot be reached as a
613
- step, and the database does not accept a link with such a name. The full list is in the
637
+ A chain step is named after a class, its alias or a role name. Where a chain or root method exists, the library parses
638
+ its name (`count`, `sort`, `set`, `sql`, `auth` and others) before classes: a class `count` is reachable as the first
639
+ step `db.count(…)`, but not after another step. The database does not accept a link with such a name. Do not give
640
+ classes these names; the full list is in the
614
641
  ["Reserved names"](#reserved-names) section.
615
642
 
616
643
  ## 5. Writing
@@ -664,8 +691,9 @@ fields are not lost. If the step has no filter of its own, the target is taken f
664
691
  `data.id` all entities of the step are edited: `db.Product().update(…)` edits all products of the tenant. If the step
665
692
  found nothing, the result is empty and there is no error, so check the length of the result. If `update` affected fewer
666
693
  rows than the step found, the whole plan is rolled back: a row deleted in the meantime gives `target_not_found`, and a
667
- row that is visible but may not be edited gives `acl_denied`. The `.tags(…)` modifier before `update` both selects the
668
- rows and replaces their tags entirely.
694
+ row that is visible but may not be edited gives `acl_denied`. The `.tags(…)` modifier before `update` selects the rows;
695
+ a string or a list also replaces their tags entirely, while an operator (`has`, `hasAny`, `hasAll`) only selects — the
696
+ tags stay as they are.
669
697
 
670
698
  ### Create or replace: `upsert`
671
699
 
@@ -800,8 +828,15 @@ await db.Order(order).line().create({ qty: 1, price: 900 }).Product.set(sencha).
800
828
  journal conflict, the library retries the whole plan, up to three times.
801
829
  - An error in any part rolls back the whole plan. `exists` and `conflict` are not retried: they are answers for the
802
830
  application.
803
- - Modifiers before a verb (`limit`, `sort`) narrow its targets.
831
+ - Modifiers before a verb (`limit`, `offset`, `sort`) narrow its targets.
832
+ - A modifier after a verb applies to what follows it: a class step, `versions()`, `diff()`, `blame()` and aggregates,
833
+ or the targets of the next verb. If a `rows`, `first`, `ids`, `count` or `paths` terminal follows right away, there is
834
+ nothing left to narrow — the verb has gone through all targets of the step — so
835
+ `db.Product().update(…).limit(1).rows()` gives `invalid_query` and writes nothing. In a batch, `run()` refuses the
836
+ same way.
804
837
  - A verb right after a verb writes to the same entities.
838
+ - The result rows of a verb come in the order of its targets: by `sort()` before the verb, otherwise by id; `first()`
839
+ after a verb is the first one in that order.
805
840
 
806
841
  ## 6. Reading
807
842
 
@@ -846,7 +881,7 @@ const roots = await db.Category({ Parent: null }).rows() // root categorie
846
881
  | `ne(v)`, `gt(v)`, `gte(v)`, `lt(v)`, `lte(v)` | ≠ (taking `null` into account), >, ≥, <, ≤ |
847
882
  | `between(a, b)`, `inList([…])` | a ≤ x ≤ b; a value from the list |
848
883
  | `like(s)`, `ilike(s)`, `starts(s)`, `ends(s)` | a pattern with `%` and `_`; case-insensitive; starts with; ends with |
849
- | `has(v)`, `hasAny([…])`, `hasAll([…])` | an array field contains the value; at least one; all |
884
+ | `has(v)`, `hasAny([…])`, `hasAll([…])` | an array field contains the value; at least one; all (`hasAll([])` matches every row, `hasAny([])` matches none) |
850
885
  | `exists(true \| false)`, `isNull()` | the field is present or not; the field is `null` or missing |
851
886
  | `not(v)` | negation of an operator; for a scalar, "not equal" |
852
887
  | `or(f1, f2, …)` | any of the step filters |
@@ -917,7 +952,9 @@ const byStatus = await db.Order().countBy('data.status') // { new: 1 }
917
952
  ```
918
953
 
919
954
  The database computes aggregates over the entities of the last step. `limit(n)` and `offset(n)` before `countBy()`
920
- apply to the groups: `db.Order().limit(3).countBy('data.status')` gives the three most frequent values.
955
+ apply to the groups: `db.Order().limit(3).countBy('data.status')` gives the three most frequent values. For a boolean
956
+ field, `min()` is `false` if there is at least one `false` (whether all values are `true`), and `max()` is `true` if
957
+ there is at least one `true`; `sum()` and `avg()` do not work on a boolean field — PostgreSQL returns an error.
921
958
 
922
959
  ### Paths
923
960
 
@@ -950,9 +987,10 @@ try {
950
987
  `delete()` without `{ confirm: true }` is only a preview: the whole cascade closure. Every preview row has `$action`
951
988
  (`delete`, `unset` or `restrict`) and `$depth`, the cascade level (0 is the target itself; `unset` and `restrict` rows
952
989
  do not have it). With the confirmation, all targets of the step are deleted in one statement, and the cascade applies
953
- the rules of the ends. The preview does not take permissions into account: a row with `$action: 'delete'` may fail to
954
- be deleted for a user without the permission (`acl_denied`), so a user interface should ask
955
- `acl.checkData(class, 'DELETE')`. The rules of the ends:
990
+ the rules of the ends. The result is the tombstones level by level, by id within a level (the journal has them in the
991
+ same order); targets of classes without a journal (`history: false`) come last. The preview does not take permissions
992
+ into account: a row with `$action: 'delete'` may fail to be deleted for a user without the permission (`acl_denied`),
993
+ so a user interface should ask `acl.checkData(class, 'DELETE')`. The rules of the ends:
956
994
 
957
995
  - `restrict`: deletion is forbidden while the target is referenced (`delete_restricted` with a list of the visible rows
958
996
  and the number of the others);
@@ -1416,8 +1454,10 @@ denial that the identifier is taken ([section 18](#18-limitations)).
1416
1454
  <!-- run: guide -->
1417
1455
  ```ts
1418
1456
  const shortToken = await svc.auth.sessionFor(BORIS, { ttl: 600 }) // a session for 10 minutes of idle time for another process
1457
+ const whose = await svc.auth.session(shortToken) // { account, tenant, member, expiresAt, loginAt, methods }
1419
1458
  const boris = await connect({ dsn: APP, schema: r.schema, token: shortToken })
1420
1459
  await boris.auth.revoke() // sign-out: the session is revoked
1460
+ const ended = await svc.auth.session(shortToken) // null
1421
1461
  await boris.close()
1422
1462
  ```
1423
1463
 
@@ -1430,6 +1470,16 @@ account is deleted. Deleting a credential revokes the other sessions of the acco
1430
1470
  "key + secret" pair revokes only the sessions opened by that key (and old sessions of the same sign-in method without
1431
1471
  a key mark); the current session stays unless you pass `keepCurrent: false`.
1432
1472
 
1473
+ A service that holds a user's token (for example, in a cookie) learns about the session with `svc.auth.session(token)`:
1474
+ no connection is opened with that token, and the session is not extended. The answer tells whose session it is
1475
+ (`account`), which tenant it works in (`tenant`: the one chosen by `auth.switch()` or the account's own tenant),
1476
+ whether the membership there is intact (`member`; always `true` in the own tenant), when it expires (`expiresAt`), and
1477
+ when and how the sign-in happened (`loginAt`, `methods`: `PASSWORD`, `APIKEY`, `SERVICE`, `TOTP` …). A wrong, revoked
1478
+ or expired token and a disabled account give `null`; the reason is not told apart. This is how a web service checks the
1479
+ session at the start of every request: reading under a revoked session gives no error, the answers are simply empty.
1480
+ The method is called by the service or the system administrator (the `auth.session` permission); a user gets
1481
+ `acl_denied`.
1482
+
1433
1483
  ### Working in another tenant
1434
1484
 
1435
1485
  A service acts on behalf of a user through `svc.as(account, { tenant, owner })`. The database checks the service's
@@ -1542,7 +1592,15 @@ Fields of the system permission classes:
1542
1592
 
1543
1593
  `db.auth.purgeAccount(account)` deletes an account and the data of its tenant. The system administrator or the owner of
1544
1594
  its tenant can call it; you cannot delete yourself or the last owner of a tenant. Ownership of rows in other tenants
1545
- passes to those tenants, sessions are revoked, credentials are deleted.
1595
+ passes to those tenants, sessions are revoked, credentials are deleted. An owner can also delete the tenant they are
1596
+ working in right now: their session returns to their personal tenant.
1597
+
1598
+ The system administrator can also delete the account row itself — `db.Account(id).delete({ confirm: true })` in the
1599
+ System tenant; the data of the account's tenant stays then. Either way, the account's rows in `secret` are erased at
1600
+ once: sessions, password and key hashes, TOTP secrets, external accounts (the foreign key `secret.account` with a
1601
+ cascade, revision 7). Before that, the sessions are revoked with a signal, and processes drop the cached answers of
1602
+ these sessions. The System account cannot be deleted (`acl_denied`): the service rows of the installation in `secret`
1603
+ belong to it.
1546
1604
 
1547
1605
  ## 11. History and point-in-time reads
1548
1606
 
@@ -1712,7 +1770,7 @@ const sub = db.watch({ from: 'start', classes: ['Product'] }) // the whole jou
1712
1770
  await db.Product(kettle).update({ price: 2100 }).rows()
1713
1771
  let cursor = ''
1714
1772
  for await (const ev of sub) { // ev: { cursor, row, id, class, op, rev, at }
1715
- cursor = ev.cursor // save the cursor: watch({ from: cursor }) continues after it
1773
+ cursor = sub.cursor! // the subscription cursor: watch({ from: cursor }) continues after this event
1716
1774
  if (ev.id === kettle.id && ev.row.data.price === 2100) break // break (or sub.close()) stops the subscription
1717
1775
  }
1718
1776
  ```
@@ -1731,10 +1789,14 @@ for await (const ev of sub) { // ev: { cursor, row, id, class, op, r
1731
1789
  within a transaction in the order of writing. This is not the commit order: compare versions of one object by `rev`.
1732
1790
  - A long write transaction in any database of the cluster delays the stream: `await sub.lag()` shows the lag in ms.
1733
1791
  - A cursor older than the `all` tier of the retention policy gives `cursor_expired`: read the current state again and
1734
- subscribe with `'now'`.
1792
+ subscribe with `'now'`. The time in the subscription cursor `sub.cursor` is the time of the last event, and once the
1793
+ subscription has caught up with the journal, the time of the poll, so a subscription that got no events for a day
1794
+ does not expire. The event cursor `ev.cursor` keeps the time of the event itself.
1735
1795
  - Without `LISTEN` (`connect({ listen: false })`), the subscription polls the journal.
1736
- - A subscription without losses is a saved cursor: store `ev.cursor` of the last processed event and resume from it
1737
- after a restart.
1796
+ - A subscription without losses is a saved cursor: after processing an event, store `sub.cursor` (the position after
1797
+ the last delivered event) and resume from it after a restart. `ev.cursor` also continues the stream, but after a
1798
+ quiet period longer than the `all` tier such a cursor is older than the tier, and resuming from it gives
1799
+ `cursor_expired`.
1738
1800
 
1739
1801
  ### Projectors
1740
1802
 
@@ -1821,6 +1883,9 @@ sales.close() // stop() after the current bat
1821
1883
  `out.inc(class, row, deltas, { tenant })` creates a row or completes it and adds the numbers without reading the row
1822
1884
  (missing fields count from 0). `out.delete(class, key or id, { tenant })` deletes a row and does nothing if there is
1823
1885
  none. `await out.get(class, key or id, { tenant })` reads the current row together with the writes of the batch.
1886
+ `delete` and `get` find a row of a class with a key by the key fields (`{ day: '2026-10-29' }`), and by id (a string
1887
+ or `{ $id }`) only a row of a class without a key; an id alone for a class with a key is `invalid_data`, and the
1888
+ batch is rolled back.
1824
1889
  `out` writes only to the `into` classes and only to the tenant of the changes passed to `handle`; otherwise you get
1825
1890
  `derived_class` or `tenant_denied`, and the batch is rolled back.
1826
1891
  - `out` adds up the increments of a row given by key fields only (or by `$id` alone) before sending them to the
@@ -1873,17 +1938,23 @@ sales.close() // stop() after the current bat
1873
1938
  at all — `derived_class`. Only an administrator may mark the classes of a tenant, so it is the administrator who
1874
1939
  chooses the participants: a tenant user cannot join a projector on their own. The option `tenants: [id, …]`
1875
1940
  is an explicit list of participants: the projector works only with these tenants (their `into` classes must be
1876
- marked too) and ignores the marks of the others. The sources are the `from` families of these tenants; a class
1877
- cannot be a source and a result at the same time. A projector does not see the changes of a class with
1878
- `history: false`: it has no journal. So a tenant with such a class among the sources drops out with the code
1879
- `invalid_class`, and the projector does not build its derived rows. Turn the journal of the class on and rebuild the
1880
- tenant with `rebuild({ tenant })`.
1941
+ marked too) and ignores the marks of the others. The sources are the `from` families of these tenants; an `into`
1942
+ class is never a source, even if it falls into `from`: a projector does not read its own results. A projector does
1943
+ not see the changes of a class with `history: false`: it has no journal. So a tenant with such a class among the
1944
+ sources drops out with the code `invalid_class`, and the projector does not build its derived rows. Turn the journal
1945
+ of the class on and rebuild the tenant with `rebuild({ tenant })`.
1881
1946
  - `batch` is how many changes go into a batch (500), `pollMs` is fallback polling (1000 ms): source signals wake the
1882
1947
  projector earlier.
1883
1948
  - `lockTimeout` is how long a projector transaction waits for another transaction's row lock (5000 ms; 0 — no limit).
1884
1949
  Otherwise a transaction of any tenant holding a total row or an end target would hold the batch of all tenants.
1885
1950
  Longer than that, the batch rolls back, the cursor stays, and `run()` does not reject but skips the pass
1886
- (`skipped`) and repeats it.
1951
+ (`skipped`) and repeats it. The report names the tenant whose row stopped the batch (`locked`).
1952
+ `rebuild({ tenant, chunked: true })` waits such a lock out.
1953
+ - `lockRetries` is how many passes in a row the writes of one tenant may run into `lockTimeout` before it drops out
1954
+ (0 by default — it never drops out). The dropped tenant gets the code `55P03`, and the batch goes on for the others;
1955
+ its changes are skipped until `rebuild({ tenant })`. The process keeps the count, and a successful write of the
1956
+ tenant resets it. Note: a long exclusive lock on the family of a System class (a tightening of its description) will
1957
+ hit every tenant in turn.
1887
1958
  - Every edit that `out` sends to the database is a separate edit of a row, and the speed of a projector is set by the
1888
1959
  number of edits, not by the number of changes. The folding of `inc` brings the edits of a row down to one per batch,
1889
1960
  so the handle from the example above (`out.inc` for every change) caught up with 6 thousand changes per second with
@@ -1965,8 +2036,10 @@ stays correct:
1965
2036
  - changes of permissions and classes, writes through `db.sql`, a revoked session and a listener reconnect clear the
1966
2037
  whole cache or the results of the session;
1967
2038
  - the default term is 60 seconds; the cache does not work inside `db.begin()`;
1968
- - without `LISTEN` and in the `none` mode the cache is off, and `connect({ cache: { ttlOnly: true } })` enables it by
1969
- term only.
2039
+ - without `LISTEN` and in the `none` mode the cache is off, and `connect({ cache: { ttlOnly: true } })` enables it: an
2040
+ answer lives until its term ends unless a write of this connection resets it;
2041
+ - `db.load()`, the projector and `db.rebuildAggregates()` reset the cache, even of this connection, only by a database
2042
+ signal.
1970
2043
 
1971
2044
  A process re-reads the class registry right away on a class change signal (one re-read at a time), and the registry
1972
2045
  and the signal key on `*`, on an unknown token and on a payload of another mode (and without the key in the `opaque`
@@ -1987,10 +2060,17 @@ Between processes, the cache goes stale for the signal sending interval (`notify
1987
2060
  const report = await adm.maintain() // { sessions, codes, thinned, batches, more, queue, skipped }
1988
2061
  ```
1989
2062
 
1990
- `maintain()` deletes expired sessions and one-time codes, thins the journal by the retention policy and checks the
1991
- PostgreSQL signal queue: if it is more than half full, you get a warning. The system administrator, a role with the
1992
- `letopis.maintain` permission or an admin connection can call it. An external scheduler runs it (`letopis maintain`),
1993
- or the library itself does: `connect({ maintain: '1h' })`. Of several processes, only one works: each batch is its own
2063
+ `maintain()` deletes sessions and one-time codes of all kinds (`OTP` and custom `OTP_<NAME>` from `issueOtp({ kind })`)
2064
+ from the `secret` table — expired ones, as well as
2065
+ revoked sessions and codes that were used, replaced by a new one or ran out of attempts, — thins the journal by the
2066
+ retention policy and checks the PostgreSQL signal queue: if it is more than half full, you get a warning. You do not
2067
+ need `maintain()` for the rows of a deleted account: the deletion itself erases them ([Deleting an account](#deleting-an-account)).
2068
+ The system administrator, a role with the `letopis.maintain` permission or an admin connection can call it. An external scheduler
2069
+ runs it (`letopis maintain`), or the library itself does: `connect({ maintain: '1h' })`. The interval is a number of
2070
+ milliseconds or a string (`'500ms'`, `'30s'`, `'15m'`, `'1h'`, `'1d'`) from 1 ms to 2³¹−1 ms (about 24 days):
2071
+ `connect()` rejects any other value, including zero, a negative number or `NaN`, with `invalid_data` without opening
2072
+ connections. `close()` waits for a scheduled run in progress: it finishes the current batch and does not start the
2073
+ next one. Of several processes, only one works: each batch is its own
1994
2074
  transaction, and the first thing it does is take the maintenance lock — a row of the `secret` table that neither the
1995
2075
  application role nor any other database role can take; the lock is released when the batch commits. If another
1996
2076
  process holds the lock, the call answers `skipped: true` before the first batch, and between batches it stops (that
@@ -2201,6 +2281,10 @@ letopis mcp --dsn <dsn> --schema <v2.name> --token <user token> [--write]
2201
2281
  (or LETOPIS_DSN, LETOPIS_SCHEMA, LETOPIS_TOKEN; the service key --api-key is not accepted)
2202
2282
  ```
2203
2283
 
2284
+ `letopis types` and `letopis describe` build their output from the session's classes, so without a valid session (the
2285
+ token expired, was revoked, is wrong or not given, or the API key is wrong) they exit with code 1 and the `no_session`
2286
+ error and leave the `--out` file untouched.
2287
+
2204
2288
  ### Monitoring
2205
2289
 
2206
2290
  - `connect({ onQuery, slowMs })`: for every chain query, the hook gets the mode, the classes, the duration, the row
@@ -2320,17 +2404,19 @@ Removed 0.21 names (`uuidv7`, `LETOPIS_NS` and the methods listed below) throw `
2320
2404
  | `link`, `lookup` | `{ account, kind, identifier, meta?, confirmed? }` → id; `{ kind, identifier, requireConfirmed?, ttl?, totp? }` → sign-in or `null` |
2321
2405
  | `credentials`, `revokeCredential`, `setFlags` | `(account)` → credentials without secrets; `(id, { keepCurrent? })`; `(id, { confirmed?, enabled? })` |
2322
2406
  | `sessionFor`, `refresh`, `revoke`, `revokeAll` | `(account, { ttl? })` → token; `({ ttl? })` → the new expiration time (the token stays the same); `()`; `(account?)` → number of sessions |
2407
+ | `session` | `(token)` → `{ account, tenant, member, expiresAt, loginAt, methods }` or `null` |
2323
2408
  | `tenants`, `switch` | `()` → `[{ tenant, roles }]`; `(tenant)` |
2324
2409
  | `purgeAccount` | `(account)` → `{ account, rows, versions, memberships, owned }` |
2325
2410
 
2326
- Password, code and TOTP checks, session issuing and credential confirmation are called by the service (`auth.*`
2327
- permissions); a regular user gets `acl_denied`. The credentials of System, services and system administrators can be
2328
- changed only by the account itself after a recent interactive sign-in, or by the system administrator. The exception is
2329
- the service's keys: the service issues and revokes its own API keys and "key + secret" pairs itself with a session of
2330
- its own key, but not its other credentials; nobody can revoke the last active key of a service (`acl_denied`). With
2331
- `totp`, `verifyPassword` and `lookup` also check the TOTP code (the `auth.totp` permission) and issue a session with the
2332
- `TOTP` method. The credential kind `kind` is `^[A-Z][A-Z0-9_]{1,31}$` (`PASSWORD`, `EMAIL`,
2333
- `TELEGRAM` …); `SESSION`, `APIKEY`, `KEYSECRET`, `OTP`, `TOTP` and `KEY` are taken.
2411
+ Password, code and TOTP checks, session issuing, session details by token and credential confirmation are called by the
2412
+ service (`auth.*` permissions); a regular user gets `acl_denied`. The credentials of System, services and system
2413
+ administrators can be changed only by the account itself after a recent interactive sign-in, or by the system
2414
+ administrator. The exception is the service's keys: the service issues and revokes its own API keys and "key + secret"
2415
+ pairs itself with a session of its own key, but not its other credentials; nobody can revoke the last active key of a
2416
+ service (`acl_denied`). With `totp`, `verifyPassword` and `lookup` also check the TOTP code (the `auth.totp` permission)
2417
+ and issue a session with the `TOTP` method. The credential kind `kind` is `^[A-Z][A-Z0-9_]{1,31}$` (`PASSWORD`, `EMAIL`,
2418
+ `TELEGRAM` …); `SESSION`, `APIKEY`, `KEYSECRET`, `OTP`, `TOTP`, `KEY`, `PROJECTOR` and `LOCK` are taken. The same rule
2419
+ applies to the password kind `user.kind` in `createTenant` and `up({ tenant })`.
2334
2420
 
2335
2421
  ### Projector
2336
2422
 
@@ -2340,14 +2426,14 @@ give `invalid_data`.
2340
2426
 
2341
2427
  | Member | What it does |
2342
2428
  |---|---|
2343
- | options `ProjectorOptions` | `from`, `into`: classes with descendants; `version`: an integer from 1; `handle(changes, out)`; `withBefore`; `onExpired: 'rebuild' \| 'error'` (`'rebuild'` rebuilds in batches); `batch`: changes in a batch (500, from 1 to 10,000); `pollMs`: fallback polling (1000 ms); `lockTimeout`: waiting for another transaction's row lock (5000 ms, 0 — no limit); `tenants`: ids of the participant tenants |
2429
+ | options `ProjectorOptions` | `from`, `into`: classes with descendants; `version`: an integer from 1; `handle(changes, out)`; `withBefore`; `onExpired: 'rebuild' \| 'error'` (`'rebuild'` rebuilds in batches); `batch`: changes in a batch (500, from 1 to 10,000); `pollMs`: fallback polling (1000 ms); `lockTimeout`: waiting for another transaction's row lock (5000 ms, 0 — no limit); `lockRetries`: passes in a row before a tenant with another transaction's lock drops out (0 — never); `tenants`: ids of the participant tenants |
2344
2430
  | `run({ once })` | catch up with the journal (`once`) or follow it until `stop()` → `ProjectorReport` |
2345
2431
  | `rebuild({ chunked, tenant })` | rebuild from the current state: atomically, in batches or for one tenant → `ProjectorReport` |
2346
2432
  | `status()` | → `ProjectorStatus`: `{ name, state, version, cursor, at, lag, failed, rebuildingTenants }`; `failed` is `{ [tenant id]: ProjectorFailure }`, where `ProjectorFailure` is `{ cursor, code, message }` |
2347
2433
  | `stop()`, `close()` | stop `run()` after the current batch; the same plus unsubscribing from signals |
2348
- | `ProjectorReport` | `{ batches, changes, created, updated, deleted, unchanged, skipped, rebuilds }` |
2434
+ | `ProjectorReport` | `{ batches, changes, created, updated, deleted, unchanged, skipped, rebuilds, locked }` |
2349
2435
  | `ProjectorChange` | `{ cursor, id, class, tenant, op, rev, at, author, row }` and, when present, `agent`, `reason`, `moved`, `before`, `rebuild` |
2350
- | `ProjectorOut` | `upsert(class, row, { tenant })`, `inc(class, row, deltas, { tenant })`, `delete(class, key or id, { tenant })`, `get(class, key or id, { tenant })` → `Row \| null`; a row is a `ProjectorRow` in the loader format |
2436
+ | `ProjectorOut` | `upsert(class, row, { tenant })`, `inc(class, row, deltas, { tenant })`, `delete(class, key or id, { tenant })`, `get(class, key or id, { tenant })` → `Row \| null`; a row is a `ProjectorRow` in the loader format; for a class with a key `delete` and `get` go by the key, an id only for a class without a key |
2351
2437
 
2352
2438
  ### Result row
2353
2439
 
@@ -2364,9 +2450,11 @@ give `invalid_data`.
2364
2450
 
2365
2451
  ### Reserved names
2366
2452
 
2367
- The Proxy parses the names below before classes, so a class with such a name cannot be reached as a step, and the
2368
- database does not accept a link with such a name (`reserved_names()`; the installer fills in the list from
2369
- `RESERVED_CLASS_NAMES`).
2453
+ A method name hides a class only where that method exists: after a step — chain methods, at the root `db` — its
2454
+ methods, in a batch — its own. So a hub named like a chain-only method (for example, `count`) is reachable as the first
2455
+ step `db.count(…)` but not after another step, and a hub named like a root method cannot be the first step. The
2456
+ database does not accept a link with any of these names (`reserved_names()`; the installer fills in the list from
2457
+ `RESERVED_CLASS_NAMES`). Do not give classes these names.
2370
2458
 
2371
2459
  - chain methods: `then`, `entity`, `paths`, `rows`, `first`, `ids`, `count`, `limit`, `offset`, `sort`, `asOf`,
2372
2460
  `withDeleted`, `deep`, `exact`, `sum`, `avg`, `min`, `max`, `countBy`, `after`, `versions`, `create`, `update`,
@@ -2423,7 +2511,7 @@ Library codes:
2423
2511
  | `tx_rolled_back` | PostgreSQL answered `ROLLBACK` to `COMMIT`: the error was caught inside `tr.transaction(fn)` |
2424
2512
  | `no_key` | `upsert`, `rekey` or `idOf` on a class without a key |
2425
2513
  | `no_path` | there is no hop between the classes of neighboring steps |
2426
- | `invalid_query` | the chain or the API is used incorrectly (a step modifier `deep`, `exact`, `alias`, `tags` or `owner` after a verb, a slot not after a verb, an `after()` cursor with a value of a different type than the `sort` field, without `v` or with an `id` that is not a uuid, and so on) |
2514
+ | `invalid_query` | the chain or the API is used incorrectly (a step modifier `deep`, `exact`, `alias`, `tags` or `owner` after a verb, a selection modifier between a verb and a `rows`, `first`, `ids`, `count` or `paths` terminal, a slot not after a verb, an `after()` cursor with a value of a different type than the `sort` field, without `v` or with an `id` that is not a uuid, and so on) |
2427
2515
  | `admin_required` | the loader, a projector and `rebuildAggregates` need an admin connection |
2428
2516
  | `rebuild_required` | the projection was built by another version of the projector (`version`): a rebuild with `rebuild()` is needed; `rebuild({ tenant })` while there is no projection or a batch rebuild is in progress |
2429
2517
  | `projector_busy` | a rebuild cannot start: the projector is running in another process right now |
@@ -2493,7 +2581,7 @@ runs) showed no write or read regressions against 1.1: two outliers did not repe
2493
2581
 
2494
2582
  ## 18. Limitations
2495
2583
 
2496
- This section lists what letopis 1.3 deliberately does not do or does with caveats, and where you need to watch the
2584
+ This section lists what letopis 1.4 deliberately does not do or does with caveats, and where you need to watch the
2497
2585
  behavior.
2498
2586
 
2499
2587
  **Platform**
@@ -2558,8 +2646,8 @@ behavior.
2558
2646
  - Hot reference targets (an organization that almost everything references) accumulate `for key share` locks and
2559
2647
  MultiXact, as with regular foreign keys; this needs monitoring. Operations on one wallet go one after another under a
2560
2648
  lock of its row ([section 8](#8-transactions-batches-and-money), "Recipe: wallet").
2561
- - A link cannot have a reserved name, and a hub with such a name cannot be reached as a step
2562
- (["Reserved names"](#reserved-names)).
2649
+ - A link cannot have a reserved name, and a hub with such a name is not reachable everywhere: for example, `count` —
2650
+ only as the first step (["Reserved names"](#reserved-names)).
2563
2651
 
2564
2652
  **Access**
2565
2653
 
@@ -2758,7 +2846,7 @@ This section is for those who develop letopis itself. Run: `cd lib && npm test`.
2758
2846
  # cache — result cache: hits, own and others' writes, the key with the user, permissions, revoked sessions, listener drop, ttl, class marking, without LISTEN, a race, db.sql, the limit of signal counters (stage 6; plan 1.1, checkpoint A)
2759
2847
  # maintain — maintenance: expired sessions and codes, permissions, one process under a lock, schedule (stage 6)
2760
2848
  # up — the full installer: fresh installation, re-apply, custom seeds, automatic database creation, UTF8, version 17, upgrade with migrations, PGlite, the first tenant and user (up({ tenant }), createTenant) (stages 6, 8)
2761
- # upgrade — schema upgrade from 1.1.0 (re-apply) and from 1.0.1 (upgrade: true, migration 0006) with the letopis-11 and letopis-1 packages: versions and hashes written by the old engine, verify(), the catalog against a clean installation, lock rows, the application, a subscription and a projector over old data, a repeated up() (work after 1.2.0, stage 1)
2849
+ # upgrade — schema upgrade from 1.1.0 (upgrade: true, migration 0007) and from 1.0.1 (upgrade: true, migrations 0006 and 0007) with the letopis-11 and letopis-1 packages: versions and hashes written by the old engine, verify(), the catalog against a clean installation, lock rows, secret rows without an account erased, the application, a subscription and a projector over old data, a repeated up() (work after 1.2.0, stage 1; revision 7)
2762
2850
  # scenario-salon — end-to-end salon scenario: 21 acts from 0.21 (salon) with a coverage matrix of the 1.0 API (stage 6)
2763
2851
  # telemetry — onQuery and slowMs, describe() and class metadata with inheritance, describeText (stage 6)
2764
2852
  # history — diff and blame over the log, the reason of a change for verbs, db.begin() and batches; a version hidden by an ACL condition, the changed column, a missing version, row_changes over 20 thousand targets (plan 1.1, stage 1 and checkpoint A)