letopis 1.3.0 → 1.4.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENT-CHEATSHEET.en.md +14 -6
- package/AGENT-CHEATSHEET.md +18 -10
- package/CHANGELOG.md +247 -0
- package/README.en.md +146 -58
- package/README.md +129 -51
- package/dist/acl.d.ts +11 -5
- package/dist/admin.d.ts +30 -14
- package/dist/admin.js +42 -16
- package/dist/auth.d.ts +61 -17
- package/dist/auth.js +16 -2
- package/dist/cache.d.ts +5 -1
- package/dist/chain.d.ts +91 -15
- package/dist/chain.js +47 -4
- package/dist/cli.js +6 -0
- package/dist/errors.d.ts +2 -2
- package/dist/errors.js +5 -5
- package/dist/import.js +7 -0
- package/dist/index.d.ts +74 -45
- package/dist/index.js +18 -4
- package/dist/load.d.ts +10 -3
- package/dist/load.js +19 -4
- package/dist/mcp.d.ts +20 -11
- package/dist/mcp.js +4 -4
- package/dist/model.d.ts +4 -1
- package/dist/ops.d.ts +58 -18
- package/dist/ops.js +58 -18
- package/dist/project.d.ts +44 -12
- package/dist/project.js +90 -8
- package/dist/registry.d.ts +4 -1
- package/dist/registry.js +4 -1
- package/dist/sql.d.ts +4 -1
- package/dist/sql.js +12 -4
- package/dist/sync.js +11 -5
- package/dist/tx.js +10 -1
- package/dist/types.d.ts +14 -7
- package/dist/types.js +13 -6
- package/dist/up.d.ts +65 -25
- package/dist/up.js +152 -40
- package/dist/validate.d.ts +9 -2
- package/dist/validate.js +19 -3
- package/dist/watch.d.ts +17 -5
- package/dist/watch.js +40 -8
- package/dist/write.d.ts +3 -2
- package/dist/write.js +29 -10
- package/llms.txt +1 -1
- package/package.json +1 -1
- package/sql/10-core.sql +6 -1
- package/sql/50-acl.sql +5 -1
- package/sql/60-write.sql +77 -22
- package/sql/80-auth.sql +39 -2
- package/sql/88-derived.sql +82 -1
- package/sql/90-time.sql +75 -38
- package/sql/95-seed.system.sql +6 -3
- package/sql/99-revision.sql +3 -2
- package/sql/README.md +2 -0
- package/sql/migrate/0007-secret-account-fk.sql +12 -0
package/README.en.md
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# letopis 1.
|
|
1
|
+
# letopis 1.4
|
|
2
2
|
|
|
3
3
|
**letopis** is a Node.js library: a store of versioned entities on PostgreSQL 18. Every change is saved as a version
|
|
4
4
|
in a journal. So every object has a history, a state as of any date and a change subscription. The database itself
|
|
@@ -130,7 +130,7 @@ What happened here:
|
|
|
130
130
|
from the journal.
|
|
131
131
|
|
|
132
132
|
Calling `up()` again is safe: it re-applies the same revision and does not touch the data, and it does not issue the
|
|
133
|
-
service key again while the installation still has
|
|
133
|
+
service key again while the installation still has at least one API key that is not revoked. Next, section 2 explains the design, and sections
|
|
134
134
|
3–13 go through the tasks in order.
|
|
135
135
|
|
|
136
136
|
## 2. How letopis works
|
|
@@ -243,11 +243,16 @@ functions and the system seed. On the first installation, it issues a service ke
|
|
|
243
243
|
| `owner`, `app` | the names of the owner and application roles (`letopis_owner`, `letopis_app` by default) |
|
|
244
244
|
|
|
245
245
|
Result: `schema` (the full name), `serviceKey` (on the first installation, and on a re-run when the installation has no
|
|
246
|
-
|
|
246
|
+
API key left except revoked ones: an expired or disabled key counts too; otherwise `null`), `created`, `upgraded`, `revision`, `migrations`, `analyzed`, `owner`, `app`,
|
|
247
247
|
`signals` (the signal mode of the installation), `warnings` (without `quiet` they are printed), and also `tenant` and
|
|
248
248
|
`pglite` if they were requested. On PostgreSQL below 18.2, `up()` warns: such a server could truncate CLOG that unread
|
|
249
249
|
signals still need — upgrade it.
|
|
250
250
|
|
|
251
|
+
Several instances of a service may call `up()` for the same schema at the same time: the installation runs under a lock
|
|
252
|
+
on the schema. The first `up()` installs or upgrades it, the others wait and re-apply the ready schema
|
|
253
|
+
(`created: false`, `upgraded: false`); one of them creates the first `tenant`, the others find it by name. `up()` calls
|
|
254
|
+
for different schemas of one database do not wait for each other.
|
|
255
|
+
|
|
251
256
|
To hide class names from signals, use the `opaque` mode (or `tenant` if the names are not secret but you want the
|
|
252
257
|
cache to be precise per tenant); after changing the mode, restart the application processes — they read the mode on
|
|
253
258
|
connect:
|
|
@@ -291,7 +296,9 @@ A repeat with the same name and login duplicates nothing and does not change the
|
|
|
291
296
|
session. The login goes into the id as is: `Anna@shop.ru` and `anna@shop.ru` would give two accounts, and the second one
|
|
292
297
|
would have no password, so write logins in one case. The `created` field in the result tells whether this call created
|
|
293
298
|
the tenant: for an existing tenant with a new user it is `false`. The roles are set by `user.roles`, a non-empty list
|
|
294
|
-
of names (not masks), otherwise `invalid_data`.
|
|
299
|
+
of names (not masks), otherwise `invalid_data`. The password kind `user.kind` (`PASSWORD` by default) follows the rule
|
|
300
|
+
for a credential kind, as in `auth.setPassword` ([API reference](#dbauth)), otherwise it is `invalid_data` too. A
|
|
301
|
+
repeat does not change the roles of an existing membership: they are
|
|
295
302
|
changed by editing the membership ([Granting a permission](#granting-a-permission)). A repeat does not bring back
|
|
296
303
|
what was deleted: if the tenant, the user account or the membership is deleted (the row is gone but its history is in
|
|
297
304
|
the journal), `createTenant` answers `invalid_data` and writes nothing. The error text names what was deleted and
|
|
@@ -363,11 +370,21 @@ const adm = await connect({ dsn: ADMIN, schema: r.schema, token: r.tenant!.token
|
|
|
363
370
|
|
|
364
371
|
### Upgrading the package and schema
|
|
365
372
|
|
|
366
|
-
The schema has an engine revision (currently
|
|
373
|
+
The schema has an engine revision (currently 7). If a new package version brings a new revision, `connect()` only warns
|
|
367
374
|
and names the command. It is `up()` that refuses: a lagging schema without `upgrade: true`, and a schema newer than the
|
|
368
|
-
package.
|
|
375
|
+
package. To a schema newer than the package (a newer package has already upgraded it, and old application instances
|
|
376
|
+
are still running) `connect()` connects anyway and warns that the letopis package should be updated. So a startup script that calls `up()` on every start will stop after a package update until it gets
|
|
369
377
|
`upgrade: true`. `up({ dsn, schema, upgrade: true })` applies the migrations from `sql/migrate/` and re-applies the
|
|
370
|
-
functions; the data is kept.
|
|
378
|
+
functions; the data is kept. Revision 7 (1.4.1) adds the foreign key `secret.account`: migration `0007` first erases
|
|
379
|
+
the `secret` rows whose accounts are already gone (the credentials of accounts deleted earlier were kept with the
|
|
380
|
+
`deleted_at` mark), and then sets the key.
|
|
381
|
+
|
|
382
|
+
The revision does not change with every version: 1.1–1.4.0 share one (6), while their functions differ. So `up()` also writes
|
|
383
|
+
an engine stamp into the schema comment: the package version and a hash of its SQL files. If the stamp does not match
|
|
384
|
+
the package files, `connect()` warns. When the schema was applied by an older version (or by a version before 1.4,
|
|
385
|
+
which has no stamp), it suggests `up({ schema })`. When a newer version has already upgraded the schema, it suggests
|
|
386
|
+
updating the package: `up()` from the older version would bring the old functions back. A new package version without
|
|
387
|
+
SQL changes keeps the stamp, and there is no warning.
|
|
371
388
|
|
|
372
389
|
The first `up()` after moving to 1.2 builds the journal index `log_class_tenant_at (class, tenant, at)` instead of
|
|
373
390
|
`log_class_at`: an `asOf` and `withDeleted` slice by class no longer reads the journal of other tenants with the same
|
|
@@ -377,8 +394,9 @@ the owner role — then `up()` only drops the old one.
|
|
|
377
394
|
|
|
378
395
|
The `upgrade` test checks upgrading 1.1.0 and 1.0.1 schemas. The old package installs the schema and the old engine
|
|
379
396
|
writes the data, then the current version's `up()` runs. After it, the versions and hashes written by the old engine
|
|
380
|
-
are in place, `verify({ data: true })` is clean, and the schema catalog matches a clean installation.
|
|
381
|
-
upgraded with
|
|
397
|
+
are in place, `verify({ data: true })` is clean, and the schema catalog matches a clean installation. Both schemas are
|
|
398
|
+
upgraded with `up({ upgrade: true })`: 1.1.0 (revision 6) with migration `0007`, 1.0.1 (revision 5) with migrations
|
|
399
|
+
`0006` and `0007`.
|
|
382
400
|
|
|
383
401
|
### Development without PostgreSQL: PGlite
|
|
384
402
|
|
|
@@ -567,13 +585,21 @@ const Movement = link('Movement', { amount: z.number().int() }, { ends: { Wallet
|
|
|
567
585
|
key; the `via` end is single and accepts the target class; the `sum` field is a number; aggregates do not form a
|
|
568
586
|
cycle; the target and the source are not system or derived classes. A source class is described before the
|
|
569
587
|
aggregate: an aggregate over a class that does not exist yet is refused. For a tenant class, the source is looked up
|
|
570
|
-
among the tenant's classes. `syncModels` describes sources before targets itself
|
|
571
|
-
|
|
588
|
+
among the tenant's classes. `syncModels` describes sources before targets itself, and models in a circle (an
|
|
589
|
+
aggregate over the class's own descendant, a cycle of aggregates) it refuses before writing (`invalid_class` naming
|
|
590
|
+
the circle). The `of` class is not deleted while an aggregate of a class of the same tenant names it: first the
|
|
591
|
+
aggregate is removed from the target's description.
|
|
572
592
|
- Rows get into an aggregate only with the knowledge of someone who reads them all: the field also counts rows that
|
|
573
593
|
the author cannot see. Only someone who reads all rows of the source and its descendants without conditions can
|
|
574
594
|
describe or change an aggregate. Only someone who reads all rows of a class and its descendants without conditions,
|
|
575
595
|
both before the change and after it, can make the class a source (a new descendant of a source, a `parent` change).
|
|
576
596
|
Otherwise `acl_denied`.
|
|
597
|
+
- A new source is also checked against whoever described the aggregate: they too must read all rows of the class
|
|
598
|
+
without conditions. The describer is the author of the version of the target's `Class` row from which the aggregate
|
|
599
|
+
description became the current one. If they cannot read those rows, the class does not become a source
|
|
600
|
+
(`acl_denied`) until someone who can read them describes the aggregate again. For an aggregate of a System class, the
|
|
601
|
+
describer is checked against System classes, as at description time. A tenant class under a System aggregate is
|
|
602
|
+
checked only for whoever creates it.
|
|
577
603
|
- The option is inherited, and a descendant replaces it as a whole. Adding or changing an aggregate is a tightening:
|
|
578
604
|
writes to the target and to the sources wait while the field is recalculated for all rows.
|
|
579
605
|
- `db.rebuildAggregates(class)` recalculates the field from the current rows, for repair. A version is written only
|
|
@@ -608,9 +634,10 @@ ends type is `InferLinks<typeof Product>`. Your own `meta` shape is typed by ext
|
|
|
608
634
|
|
|
609
635
|
### Class names
|
|
610
636
|
|
|
611
|
-
A chain step is named after a class, its alias or a role name.
|
|
612
|
-
(`count`, `sort`, `set`, `sql`, `auth` and others) before classes
|
|
613
|
-
step
|
|
637
|
+
A chain step is named after a class, its alias or a role name. Where a chain or root method exists, the library parses
|
|
638
|
+
its name (`count`, `sort`, `set`, `sql`, `auth` and others) before classes: a class `count` is reachable as the first
|
|
639
|
+
step `db.count(…)`, but not after another step. The database does not accept a link with such a name. Do not give
|
|
640
|
+
classes these names; the full list is in the
|
|
614
641
|
["Reserved names"](#reserved-names) section.
|
|
615
642
|
|
|
616
643
|
## 5. Writing
|
|
@@ -664,8 +691,9 @@ fields are not lost. If the step has no filter of its own, the target is taken f
|
|
|
664
691
|
`data.id` all entities of the step are edited: `db.Product().update(…)` edits all products of the tenant. If the step
|
|
665
692
|
found nothing, the result is empty and there is no error, so check the length of the result. If `update` affected fewer
|
|
666
693
|
rows than the step found, the whole plan is rolled back: a row deleted in the meantime gives `target_not_found`, and a
|
|
667
|
-
row that is visible but may not be edited gives `acl_denied`. The `.tags(…)` modifier before `update`
|
|
668
|
-
|
|
694
|
+
row that is visible but may not be edited gives `acl_denied`. The `.tags(…)` modifier before `update` selects the rows;
|
|
695
|
+
a string or a list also replaces their tags entirely, while an operator (`has`, `hasAny`, `hasAll`) only selects — the
|
|
696
|
+
tags stay as they are.
|
|
669
697
|
|
|
670
698
|
### Create or replace: `upsert`
|
|
671
699
|
|
|
@@ -800,8 +828,15 @@ await db.Order(order).line().create({ qty: 1, price: 900 }).Product.set(sencha).
|
|
|
800
828
|
journal conflict, the library retries the whole plan, up to three times.
|
|
801
829
|
- An error in any part rolls back the whole plan. `exists` and `conflict` are not retried: they are answers for the
|
|
802
830
|
application.
|
|
803
|
-
- Modifiers before a verb (`limit`, `sort`) narrow its targets.
|
|
831
|
+
- Modifiers before a verb (`limit`, `offset`, `sort`) narrow its targets.
|
|
832
|
+
- A modifier after a verb applies to what follows it: a class step, `versions()`, `diff()`, `blame()` and aggregates,
|
|
833
|
+
or the targets of the next verb. If a `rows`, `first`, `ids`, `count` or `paths` terminal follows right away, there is
|
|
834
|
+
nothing left to narrow — the verb has gone through all targets of the step — so
|
|
835
|
+
`db.Product().update(…).limit(1).rows()` gives `invalid_query` and writes nothing. In a batch, `run()` refuses the
|
|
836
|
+
same way.
|
|
804
837
|
- A verb right after a verb writes to the same entities.
|
|
838
|
+
- The result rows of a verb come in the order of its targets: by `sort()` before the verb, otherwise by id; `first()`
|
|
839
|
+
after a verb is the first one in that order.
|
|
805
840
|
|
|
806
841
|
## 6. Reading
|
|
807
842
|
|
|
@@ -846,7 +881,7 @@ const roots = await db.Category({ Parent: null }).rows() // root categorie
|
|
|
846
881
|
| `ne(v)`, `gt(v)`, `gte(v)`, `lt(v)`, `lte(v)` | ≠ (taking `null` into account), >, ≥, <, ≤ |
|
|
847
882
|
| `between(a, b)`, `inList([…])` | a ≤ x ≤ b; a value from the list |
|
|
848
883
|
| `like(s)`, `ilike(s)`, `starts(s)`, `ends(s)` | a pattern with `%` and `_`; case-insensitive; starts with; ends with |
|
|
849
|
-
| `has(v)`, `hasAny([…])`, `hasAll([…])` | an array field contains the value; at least one; all |
|
|
884
|
+
| `has(v)`, `hasAny([…])`, `hasAll([…])` | an array field contains the value; at least one; all (`hasAll([])` matches every row, `hasAny([])` matches none) |
|
|
850
885
|
| `exists(true \| false)`, `isNull()` | the field is present or not; the field is `null` or missing |
|
|
851
886
|
| `not(v)` | negation of an operator; for a scalar, "not equal" |
|
|
852
887
|
| `or(f1, f2, …)` | any of the step filters |
|
|
@@ -917,7 +952,9 @@ const byStatus = await db.Order().countBy('data.status') // { new: 1 }
|
|
|
917
952
|
```
|
|
918
953
|
|
|
919
954
|
The database computes aggregates over the entities of the last step. `limit(n)` and `offset(n)` before `countBy()`
|
|
920
|
-
apply to the groups: `db.Order().limit(3).countBy('data.status')` gives the three most frequent values.
|
|
955
|
+
apply to the groups: `db.Order().limit(3).countBy('data.status')` gives the three most frequent values. For a boolean
|
|
956
|
+
field, `min()` is `false` if there is at least one `false` (whether all values are `true`), and `max()` is `true` if
|
|
957
|
+
there is at least one `true`; `sum()` and `avg()` do not work on a boolean field — PostgreSQL returns an error.
|
|
921
958
|
|
|
922
959
|
### Paths
|
|
923
960
|
|
|
@@ -950,9 +987,10 @@ try {
|
|
|
950
987
|
`delete()` without `{ confirm: true }` is only a preview: the whole cascade closure. Every preview row has `$action`
|
|
951
988
|
(`delete`, `unset` or `restrict`) and `$depth`, the cascade level (0 is the target itself; `unset` and `restrict` rows
|
|
952
989
|
do not have it). With the confirmation, all targets of the step are deleted in one statement, and the cascade applies
|
|
953
|
-
the rules of the ends. The
|
|
954
|
-
|
|
955
|
-
|
|
990
|
+
the rules of the ends. The result is the tombstones level by level, by id within a level (the journal has them in the
|
|
991
|
+
same order); targets of classes without a journal (`history: false`) come last. The preview does not take permissions
|
|
992
|
+
into account: a row with `$action: 'delete'` may fail to be deleted for a user without the permission (`acl_denied`),
|
|
993
|
+
so a user interface should ask `acl.checkData(class, 'DELETE')`. The rules of the ends:
|
|
956
994
|
|
|
957
995
|
- `restrict`: deletion is forbidden while the target is referenced (`delete_restricted` with a list of the visible rows
|
|
958
996
|
and the number of the others);
|
|
@@ -1416,8 +1454,10 @@ denial that the identifier is taken ([section 18](#18-limitations)).
|
|
|
1416
1454
|
<!-- run: guide -->
|
|
1417
1455
|
```ts
|
|
1418
1456
|
const shortToken = await svc.auth.sessionFor(BORIS, { ttl: 600 }) // a session for 10 minutes of idle time for another process
|
|
1457
|
+
const whose = await svc.auth.session(shortToken) // { account, tenant, member, expiresAt, loginAt, methods }
|
|
1419
1458
|
const boris = await connect({ dsn: APP, schema: r.schema, token: shortToken })
|
|
1420
1459
|
await boris.auth.revoke() // sign-out: the session is revoked
|
|
1460
|
+
const ended = await svc.auth.session(shortToken) // null
|
|
1421
1461
|
await boris.close()
|
|
1422
1462
|
```
|
|
1423
1463
|
|
|
@@ -1430,6 +1470,16 @@ account is deleted. Deleting a credential revokes the other sessions of the acco
|
|
|
1430
1470
|
"key + secret" pair revokes only the sessions opened by that key (and old sessions of the same sign-in method without
|
|
1431
1471
|
a key mark); the current session stays unless you pass `keepCurrent: false`.
|
|
1432
1472
|
|
|
1473
|
+
A service that holds a user's token (for example, in a cookie) learns about the session with `svc.auth.session(token)`:
|
|
1474
|
+
no connection is opened with that token, and the session is not extended. The answer tells whose session it is
|
|
1475
|
+
(`account`), which tenant it works in (`tenant`: the one chosen by `auth.switch()` or the account's own tenant),
|
|
1476
|
+
whether the membership there is intact (`member`; always `true` in the own tenant), when it expires (`expiresAt`), and
|
|
1477
|
+
when and how the sign-in happened (`loginAt`, `methods`: `PASSWORD`, `APIKEY`, `SERVICE`, `TOTP` …). A wrong, revoked
|
|
1478
|
+
or expired token and a disabled account give `null`; the reason is not told apart. This is how a web service checks the
|
|
1479
|
+
session at the start of every request: reading under a revoked session gives no error, the answers are simply empty.
|
|
1480
|
+
The method is called by the service or the system administrator (the `auth.session` permission); a user gets
|
|
1481
|
+
`acl_denied`.
|
|
1482
|
+
|
|
1433
1483
|
### Working in another tenant
|
|
1434
1484
|
|
|
1435
1485
|
A service acts on behalf of a user through `svc.as(account, { tenant, owner })`. The database checks the service's
|
|
@@ -1542,7 +1592,15 @@ Fields of the system permission classes:
|
|
|
1542
1592
|
|
|
1543
1593
|
`db.auth.purgeAccount(account)` deletes an account and the data of its tenant. The system administrator or the owner of
|
|
1544
1594
|
its tenant can call it; you cannot delete yourself or the last owner of a tenant. Ownership of rows in other tenants
|
|
1545
|
-
passes to those tenants, sessions are revoked, credentials are deleted.
|
|
1595
|
+
passes to those tenants, sessions are revoked, credentials are deleted. An owner can also delete the tenant they are
|
|
1596
|
+
working in right now: their session returns to their personal tenant.
|
|
1597
|
+
|
|
1598
|
+
The system administrator can also delete the account row itself — `db.Account(id).delete({ confirm: true })` in the
|
|
1599
|
+
System tenant; the data of the account's tenant stays then. Either way, the account's rows in `secret` are erased at
|
|
1600
|
+
once: sessions, password and key hashes, TOTP secrets, external accounts (the foreign key `secret.account` with a
|
|
1601
|
+
cascade, revision 7). Before that, the sessions are revoked with a signal, and processes drop the cached answers of
|
|
1602
|
+
these sessions. The System account cannot be deleted (`acl_denied`): the service rows of the installation in `secret`
|
|
1603
|
+
belong to it.
|
|
1546
1604
|
|
|
1547
1605
|
## 11. History and point-in-time reads
|
|
1548
1606
|
|
|
@@ -1712,7 +1770,7 @@ const sub = db.watch({ from: 'start', classes: ['Product'] }) // the whole jou
|
|
|
1712
1770
|
await db.Product(kettle).update({ price: 2100 }).rows()
|
|
1713
1771
|
let cursor = ''
|
|
1714
1772
|
for await (const ev of sub) { // ev: { cursor, row, id, class, op, rev, at }
|
|
1715
|
-
cursor =
|
|
1773
|
+
cursor = sub.cursor! // the subscription cursor: watch({ from: cursor }) continues after this event
|
|
1716
1774
|
if (ev.id === kettle.id && ev.row.data.price === 2100) break // break (or sub.close()) stops the subscription
|
|
1717
1775
|
}
|
|
1718
1776
|
```
|
|
@@ -1731,10 +1789,14 @@ for await (const ev of sub) { // ev: { cursor, row, id, class, op, r
|
|
|
1731
1789
|
within a transaction in the order of writing. This is not the commit order: compare versions of one object by `rev`.
|
|
1732
1790
|
- A long write transaction in any database of the cluster delays the stream: `await sub.lag()` shows the lag in ms.
|
|
1733
1791
|
- A cursor older than the `all` tier of the retention policy gives `cursor_expired`: read the current state again and
|
|
1734
|
-
subscribe with `'now'`.
|
|
1792
|
+
subscribe with `'now'`. The time in the subscription cursor `sub.cursor` is the time of the last event, and once the
|
|
1793
|
+
subscription has caught up with the journal, the time of the poll, so a subscription that got no events for a day
|
|
1794
|
+
does not expire. The event cursor `ev.cursor` keeps the time of the event itself.
|
|
1735
1795
|
- Without `LISTEN` (`connect({ listen: false })`), the subscription polls the journal.
|
|
1736
|
-
- A subscription without losses is a saved cursor: store `
|
|
1737
|
-
after a restart.
|
|
1796
|
+
- A subscription without losses is a saved cursor: after processing an event, store `sub.cursor` (the position after
|
|
1797
|
+
the last delivered event) and resume from it after a restart. `ev.cursor` also continues the stream, but after a
|
|
1798
|
+
quiet period longer than the `all` tier such a cursor is older than the tier, and resuming from it gives
|
|
1799
|
+
`cursor_expired`.
|
|
1738
1800
|
|
|
1739
1801
|
### Projectors
|
|
1740
1802
|
|
|
@@ -1821,6 +1883,9 @@ sales.close() // stop() after the current bat
|
|
|
1821
1883
|
`out.inc(class, row, deltas, { tenant })` creates a row or completes it and adds the numbers without reading the row
|
|
1822
1884
|
(missing fields count from 0). `out.delete(class, key or id, { tenant })` deletes a row and does nothing if there is
|
|
1823
1885
|
none. `await out.get(class, key or id, { tenant })` reads the current row together with the writes of the batch.
|
|
1886
|
+
`delete` and `get` find a row of a class with a key by the key fields (`{ day: '2026-10-29' }`), and by id (a string
|
|
1887
|
+
or `{ $id }`) only a row of a class without a key; an id alone for a class with a key is `invalid_data`, and the
|
|
1888
|
+
batch is rolled back.
|
|
1824
1889
|
`out` writes only to the `into` classes and only to the tenant of the changes passed to `handle`; otherwise you get
|
|
1825
1890
|
`derived_class` or `tenant_denied`, and the batch is rolled back.
|
|
1826
1891
|
- `out` adds up the increments of a row given by key fields only (or by `$id` alone) before sending them to the
|
|
@@ -1873,17 +1938,23 @@ sales.close() // stop() after the current bat
|
|
|
1873
1938
|
at all — `derived_class`. Only an administrator may mark the classes of a tenant, so it is the administrator who
|
|
1874
1939
|
chooses the participants: a tenant user cannot join a projector on their own. The option `tenants: [id, …]`
|
|
1875
1940
|
is an explicit list of participants: the projector works only with these tenants (their `into` classes must be
|
|
1876
|
-
marked too) and ignores the marks of the others. The sources are the `from` families of these tenants;
|
|
1877
|
-
|
|
1878
|
-
`history: false`: it has no journal. So a tenant with such a class among the
|
|
1879
|
-
`invalid_class`, and the projector does not build its derived rows. Turn the journal
|
|
1880
|
-
tenant with `rebuild({ tenant })`.
|
|
1941
|
+
marked too) and ignores the marks of the others. The sources are the `from` families of these tenants; an `into`
|
|
1942
|
+
class is never a source, even if it falls into `from`: a projector does not read its own results. A projector does
|
|
1943
|
+
not see the changes of a class with `history: false`: it has no journal. So a tenant with such a class among the
|
|
1944
|
+
sources drops out with the code `invalid_class`, and the projector does not build its derived rows. Turn the journal
|
|
1945
|
+
of the class on and rebuild the tenant with `rebuild({ tenant })`.
|
|
1881
1946
|
- `batch` is how many changes go into a batch (500), `pollMs` is fallback polling (1000 ms): source signals wake the
|
|
1882
1947
|
projector earlier.
|
|
1883
1948
|
- `lockTimeout` is how long a projector transaction waits for another transaction's row lock (5000 ms; 0 — no limit).
|
|
1884
1949
|
Otherwise a transaction of any tenant holding a total row or an end target would hold the batch of all tenants.
|
|
1885
1950
|
Longer than that, the batch rolls back, the cursor stays, and `run()` does not reject but skips the pass
|
|
1886
|
-
(`skipped`) and repeats it.
|
|
1951
|
+
(`skipped`) and repeats it. The report names the tenant whose row stopped the batch (`locked`).
|
|
1952
|
+
`rebuild({ tenant, chunked: true })` waits such a lock out.
|
|
1953
|
+
- `lockRetries` is how many passes in a row the writes of one tenant may run into `lockTimeout` before it drops out
|
|
1954
|
+
(0 by default — it never drops out). The dropped tenant gets the code `55P03`, and the batch goes on for the others;
|
|
1955
|
+
its changes are skipped until `rebuild({ tenant })`. The process keeps the count, and a successful write of the
|
|
1956
|
+
tenant resets it. Note: a long exclusive lock on the family of a System class (a tightening of its description) will
|
|
1957
|
+
hit every tenant in turn.
|
|
1887
1958
|
- Every edit that `out` sends to the database is a separate edit of a row, and the speed of a projector is set by the
|
|
1888
1959
|
number of edits, not by the number of changes. The folding of `inc` brings the edits of a row down to one per batch,
|
|
1889
1960
|
so the handle from the example above (`out.inc` for every change) caught up with 6 thousand changes per second with
|
|
@@ -1965,8 +2036,10 @@ stays correct:
|
|
|
1965
2036
|
- changes of permissions and classes, writes through `db.sql`, a revoked session and a listener reconnect clear the
|
|
1966
2037
|
whole cache or the results of the session;
|
|
1967
2038
|
- the default term is 60 seconds; the cache does not work inside `db.begin()`;
|
|
1968
|
-
- without `LISTEN` and in the `none` mode the cache is off, and `connect({ cache: { ttlOnly: true } })` enables it
|
|
1969
|
-
term
|
|
2039
|
+
- without `LISTEN` and in the `none` mode the cache is off, and `connect({ cache: { ttlOnly: true } })` enables it: an
|
|
2040
|
+
answer lives until its term ends unless a write of this connection resets it;
|
|
2041
|
+
- `db.load()`, the projector and `db.rebuildAggregates()` reset the cache, even of this connection, only by a database
|
|
2042
|
+
signal.
|
|
1970
2043
|
|
|
1971
2044
|
A process re-reads the class registry right away on a class change signal (one re-read at a time), and the registry
|
|
1972
2045
|
and the signal key on `*`, on an unknown token and on a payload of another mode (and without the key in the `opaque`
|
|
@@ -1987,10 +2060,17 @@ Between processes, the cache goes stale for the signal sending interval (`notify
|
|
|
1987
2060
|
const report = await adm.maintain() // { sessions, codes, thinned, batches, more, queue, skipped }
|
|
1988
2061
|
```
|
|
1989
2062
|
|
|
1990
|
-
`maintain()` deletes
|
|
1991
|
-
|
|
1992
|
-
|
|
1993
|
-
|
|
2063
|
+
`maintain()` deletes sessions and one-time codes of all kinds (`OTP` and custom `OTP_<NAME>` from `issueOtp({ kind })`)
|
|
2064
|
+
from the `secret` table — expired ones, as well as
|
|
2065
|
+
revoked sessions and codes that were used, replaced by a new one or ran out of attempts, — thins the journal by the
|
|
2066
|
+
retention policy and checks the PostgreSQL signal queue: if it is more than half full, you get a warning. You do not
|
|
2067
|
+
need `maintain()` for the rows of a deleted account: the deletion itself erases them ([Deleting an account](#deleting-an-account)).
|
|
2068
|
+
The system administrator, a role with the `letopis.maintain` permission or an admin connection can call it. An external scheduler
|
|
2069
|
+
runs it (`letopis maintain`), or the library itself does: `connect({ maintain: '1h' })`. The interval is a number of
|
|
2070
|
+
milliseconds or a string (`'500ms'`, `'30s'`, `'15m'`, `'1h'`, `'1d'`) from 1 ms to 2³¹−1 ms (about 24 days):
|
|
2071
|
+
`connect()` rejects any other value, including zero, a negative number or `NaN`, with `invalid_data` without opening
|
|
2072
|
+
connections. `close()` waits for a scheduled run in progress: it finishes the current batch and does not start the
|
|
2073
|
+
next one. Of several processes, only one works: each batch is its own
|
|
1994
2074
|
transaction, and the first thing it does is take the maintenance lock — a row of the `secret` table that neither the
|
|
1995
2075
|
application role nor any other database role can take; the lock is released when the batch commits. If another
|
|
1996
2076
|
process holds the lock, the call answers `skipped: true` before the first batch, and between batches it stops (that
|
|
@@ -2201,6 +2281,10 @@ letopis mcp --dsn <dsn> --schema <v2.name> --token <user token> [--write]
|
|
|
2201
2281
|
(or LETOPIS_DSN, LETOPIS_SCHEMA, LETOPIS_TOKEN; the service key --api-key is not accepted)
|
|
2202
2282
|
```
|
|
2203
2283
|
|
|
2284
|
+
`letopis types` and `letopis describe` build their output from the session's classes, so without a valid session (the
|
|
2285
|
+
token expired, was revoked, is wrong or not given, or the API key is wrong) they exit with code 1 and the `no_session`
|
|
2286
|
+
error and leave the `--out` file untouched.
|
|
2287
|
+
|
|
2204
2288
|
### Monitoring
|
|
2205
2289
|
|
|
2206
2290
|
- `connect({ onQuery, slowMs })`: for every chain query, the hook gets the mode, the classes, the duration, the row
|
|
@@ -2320,17 +2404,19 @@ Removed 0.21 names (`uuidv7`, `LETOPIS_NS` and the methods listed below) throw `
|
|
|
2320
2404
|
| `link`, `lookup` | `{ account, kind, identifier, meta?, confirmed? }` → id; `{ kind, identifier, requireConfirmed?, ttl?, totp? }` → sign-in or `null` |
|
|
2321
2405
|
| `credentials`, `revokeCredential`, `setFlags` | `(account)` → credentials without secrets; `(id, { keepCurrent? })`; `(id, { confirmed?, enabled? })` |
|
|
2322
2406
|
| `sessionFor`, `refresh`, `revoke`, `revokeAll` | `(account, { ttl? })` → token; `({ ttl? })` → the new expiration time (the token stays the same); `()`; `(account?)` → number of sessions |
|
|
2407
|
+
| `session` | `(token)` → `{ account, tenant, member, expiresAt, loginAt, methods }` or `null` |
|
|
2323
2408
|
| `tenants`, `switch` | `()` → `[{ tenant, roles }]`; `(tenant)` |
|
|
2324
2409
|
| `purgeAccount` | `(account)` → `{ account, rows, versions, memberships, owned }` |
|
|
2325
2410
|
|
|
2326
|
-
Password, code and TOTP checks, session issuing and credential confirmation are called by the
|
|
2327
|
-
permissions); a regular user gets `acl_denied`. The credentials of System, services and system
|
|
2328
|
-
changed only by the account itself after a recent interactive sign-in, or by the system
|
|
2329
|
-
the service's keys: the service issues and revokes its own API keys and "key + secret"
|
|
2330
|
-
its own key, but not its other credentials; nobody can revoke the last active key of a
|
|
2331
|
-
`totp`, `verifyPassword` and `lookup` also check the TOTP code (the `auth.totp` permission)
|
|
2332
|
-
`TOTP` method. The credential kind `kind` is `^[A-Z][A-Z0-9_]{1,31}$` (`PASSWORD`, `EMAIL`,
|
|
2333
|
-
`TELEGRAM` …); `SESSION`, `APIKEY`, `KEYSECRET`, `OTP`, `TOTP` and `
|
|
2411
|
+
Password, code and TOTP checks, session issuing, session details by token and credential confirmation are called by the
|
|
2412
|
+
service (`auth.*` permissions); a regular user gets `acl_denied`. The credentials of System, services and system
|
|
2413
|
+
administrators can be changed only by the account itself after a recent interactive sign-in, or by the system
|
|
2414
|
+
administrator. The exception is the service's keys: the service issues and revokes its own API keys and "key + secret"
|
|
2415
|
+
pairs itself with a session of its own key, but not its other credentials; nobody can revoke the last active key of a
|
|
2416
|
+
service (`acl_denied`). With `totp`, `verifyPassword` and `lookup` also check the TOTP code (the `auth.totp` permission)
|
|
2417
|
+
and issue a session with the `TOTP` method. The credential kind `kind` is `^[A-Z][A-Z0-9_]{1,31}$` (`PASSWORD`, `EMAIL`,
|
|
2418
|
+
`TELEGRAM` …); `SESSION`, `APIKEY`, `KEYSECRET`, `OTP`, `TOTP`, `KEY`, `PROJECTOR` and `LOCK` are taken. The same rule
|
|
2419
|
+
applies to the password kind `user.kind` in `createTenant` and `up({ tenant })`.
|
|
2334
2420
|
|
|
2335
2421
|
### Projector
|
|
2336
2422
|
|
|
@@ -2340,14 +2426,14 @@ give `invalid_data`.
|
|
|
2340
2426
|
|
|
2341
2427
|
| Member | What it does |
|
|
2342
2428
|
|---|---|
|
|
2343
|
-
| options `ProjectorOptions` | `from`, `into`: classes with descendants; `version`: an integer from 1; `handle(changes, out)`; `withBefore`; `onExpired: 'rebuild' \| 'error'` (`'rebuild'` rebuilds in batches); `batch`: changes in a batch (500, from 1 to 10,000); `pollMs`: fallback polling (1000 ms); `lockTimeout`: waiting for another transaction's row lock (5000 ms, 0 — no limit); `tenants`: ids of the participant tenants |
|
|
2429
|
+
| options `ProjectorOptions` | `from`, `into`: classes with descendants; `version`: an integer from 1; `handle(changes, out)`; `withBefore`; `onExpired: 'rebuild' \| 'error'` (`'rebuild'` rebuilds in batches); `batch`: changes in a batch (500, from 1 to 10,000); `pollMs`: fallback polling (1000 ms); `lockTimeout`: waiting for another transaction's row lock (5000 ms, 0 — no limit); `lockRetries`: passes in a row before a tenant with another transaction's lock drops out (0 — never); `tenants`: ids of the participant tenants |
|
|
2344
2430
|
| `run({ once })` | catch up with the journal (`once`) or follow it until `stop()` → `ProjectorReport` |
|
|
2345
2431
|
| `rebuild({ chunked, tenant })` | rebuild from the current state: atomically, in batches or for one tenant → `ProjectorReport` |
|
|
2346
2432
|
| `status()` | → `ProjectorStatus`: `{ name, state, version, cursor, at, lag, failed, rebuildingTenants }`; `failed` is `{ [tenant id]: ProjectorFailure }`, where `ProjectorFailure` is `{ cursor, code, message }` |
|
|
2347
2433
|
| `stop()`, `close()` | stop `run()` after the current batch; the same plus unsubscribing from signals |
|
|
2348
|
-
| `ProjectorReport` | `{ batches, changes, created, updated, deleted, unchanged, skipped, rebuilds }` |
|
|
2434
|
+
| `ProjectorReport` | `{ batches, changes, created, updated, deleted, unchanged, skipped, rebuilds, locked }` |
|
|
2349
2435
|
| `ProjectorChange` | `{ cursor, id, class, tenant, op, rev, at, author, row }` and, when present, `agent`, `reason`, `moved`, `before`, `rebuild` |
|
|
2350
|
-
| `ProjectorOut` | `upsert(class, row, { tenant })`, `inc(class, row, deltas, { tenant })`, `delete(class, key or id, { tenant })`, `get(class, key or id, { tenant })` → `Row \| null`; a row is a `ProjectorRow` in the loader format |
|
|
2436
|
+
| `ProjectorOut` | `upsert(class, row, { tenant })`, `inc(class, row, deltas, { tenant })`, `delete(class, key or id, { tenant })`, `get(class, key or id, { tenant })` → `Row \| null`; a row is a `ProjectorRow` in the loader format; for a class with a key `delete` and `get` go by the key, an id only for a class without a key |
|
|
2351
2437
|
|
|
2352
2438
|
### Result row
|
|
2353
2439
|
|
|
@@ -2364,9 +2450,11 @@ give `invalid_data`.
|
|
|
2364
2450
|
|
|
2365
2451
|
### Reserved names
|
|
2366
2452
|
|
|
2367
|
-
|
|
2368
|
-
|
|
2369
|
-
`
|
|
2453
|
+
A method name hides a class only where that method exists: after a step — chain methods, at the root `db` — its
|
|
2454
|
+
methods, in a batch — its own. So a hub named like a chain-only method (for example, `count`) is reachable as the first
|
|
2455
|
+
step `db.count(…)` but not after another step, and a hub named like a root method cannot be the first step. The
|
|
2456
|
+
database does not accept a link with any of these names (`reserved_names()`; the installer fills in the list from
|
|
2457
|
+
`RESERVED_CLASS_NAMES`). Do not give classes these names.
|
|
2370
2458
|
|
|
2371
2459
|
- chain methods: `then`, `entity`, `paths`, `rows`, `first`, `ids`, `count`, `limit`, `offset`, `sort`, `asOf`,
|
|
2372
2460
|
`withDeleted`, `deep`, `exact`, `sum`, `avg`, `min`, `max`, `countBy`, `after`, `versions`, `create`, `update`,
|
|
@@ -2423,7 +2511,7 @@ Library codes:
|
|
|
2423
2511
|
| `tx_rolled_back` | PostgreSQL answered `ROLLBACK` to `COMMIT`: the error was caught inside `tr.transaction(fn)` |
|
|
2424
2512
|
| `no_key` | `upsert`, `rekey` or `idOf` on a class without a key |
|
|
2425
2513
|
| `no_path` | there is no hop between the classes of neighboring steps |
|
|
2426
|
-
| `invalid_query` | the chain or the API is used incorrectly (a step modifier `deep`, `exact`, `alias`, `tags` or `owner` after a verb, a slot not after a verb, an `after()` cursor with a value of a different type than the `sort` field, without `v` or with an `id` that is not a uuid, and so on) |
|
|
2514
|
+
| `invalid_query` | the chain or the API is used incorrectly (a step modifier `deep`, `exact`, `alias`, `tags` or `owner` after a verb, a selection modifier between a verb and a `rows`, `first`, `ids`, `count` or `paths` terminal, a slot not after a verb, an `after()` cursor with a value of a different type than the `sort` field, without `v` or with an `id` that is not a uuid, and so on) |
|
|
2427
2515
|
| `admin_required` | the loader, a projector and `rebuildAggregates` need an admin connection |
|
|
2428
2516
|
| `rebuild_required` | the projection was built by another version of the projector (`version`): a rebuild with `rebuild()` is needed; `rebuild({ tenant })` while there is no projection or a batch rebuild is in progress |
|
|
2429
2517
|
| `projector_busy` | a rebuild cannot start: the projector is running in another process right now |
|
|
@@ -2493,7 +2581,7 @@ runs) showed no write or read regressions against 1.1: two outliers did not repe
|
|
|
2493
2581
|
|
|
2494
2582
|
## 18. Limitations
|
|
2495
2583
|
|
|
2496
|
-
This section lists what letopis 1.
|
|
2584
|
+
This section lists what letopis 1.4 deliberately does not do or does with caveats, and where you need to watch the
|
|
2497
2585
|
behavior.
|
|
2498
2586
|
|
|
2499
2587
|
**Platform**
|
|
@@ -2558,8 +2646,8 @@ behavior.
|
|
|
2558
2646
|
- Hot reference targets (an organization that almost everything references) accumulate `for key share` locks and
|
|
2559
2647
|
MultiXact, as with regular foreign keys; this needs monitoring. Operations on one wallet go one after another under a
|
|
2560
2648
|
lock of its row ([section 8](#8-transactions-batches-and-money), "Recipe: wallet").
|
|
2561
|
-
- A link cannot have a reserved name, and a hub with such a name
|
|
2562
|
-
(["Reserved names"](#reserved-names)).
|
|
2649
|
+
- A link cannot have a reserved name, and a hub with such a name is not reachable everywhere: for example, `count` —
|
|
2650
|
+
only as the first step (["Reserved names"](#reserved-names)).
|
|
2563
2651
|
|
|
2564
2652
|
**Access**
|
|
2565
2653
|
|
|
@@ -2758,7 +2846,7 @@ This section is for those who develop letopis itself. Run: `cd lib && npm test`.
|
|
|
2758
2846
|
# cache — result cache: hits, own and others' writes, the key with the user, permissions, revoked sessions, listener drop, ttl, class marking, without LISTEN, a race, db.sql, the limit of signal counters (stage 6; plan 1.1, checkpoint A)
|
|
2759
2847
|
# maintain — maintenance: expired sessions and codes, permissions, one process under a lock, schedule (stage 6)
|
|
2760
2848
|
# up — the full installer: fresh installation, re-apply, custom seeds, automatic database creation, UTF8, version 17, upgrade with migrations, PGlite, the first tenant and user (up({ tenant }), createTenant) (stages 6, 8)
|
|
2761
|
-
# upgrade — schema upgrade from 1.1.0 (
|
|
2849
|
+
# upgrade — schema upgrade from 1.1.0 (upgrade: true, migration 0007) and from 1.0.1 (upgrade: true, migrations 0006 and 0007) with the letopis-11 and letopis-1 packages: versions and hashes written by the old engine, verify(), the catalog against a clean installation, lock rows, secret rows without an account erased, the application, a subscription and a projector over old data, a repeated up() (work after 1.2.0, stage 1; revision 7)
|
|
2762
2850
|
# scenario-salon — end-to-end salon scenario: 21 acts from 0.21 (salon) with a coverage matrix of the 1.0 API (stage 6)
|
|
2763
2851
|
# telemetry — onQuery and slowMs, describe() and class metadata with inheritance, describeText (stage 6)
|
|
2764
2852
|
# history — diff and blame over the log, the reason of a change for verbs, db.begin() and batches; a version hidden by an ACL condition, the changed column, a missing version, row_changes over 20 thousand targets (plan 1.1, stage 1 and checkpoint A)
|