letopis 0.20.3 → 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (89) hide show
  1. package/AGENT-CHEATSHEET.en.md +368 -0
  2. package/AGENT-CHEATSHEET.md +354 -0
  3. package/CHANGELOG.md +348 -0
  4. package/MIGRATION.md +190 -0
  5. package/README.en.md +1937 -0
  6. package/README.md +1493 -3466
  7. package/dist/acl.d.ts +26 -50
  8. package/dist/acl.js +22 -267
  9. package/dist/admin.d.ts +138 -0
  10. package/dist/admin.js +170 -0
  11. package/dist/auth.d.ts +120 -73
  12. package/dist/auth.js +121 -306
  13. package/dist/cache.d.ts +73 -0
  14. package/dist/cache.js +148 -0
  15. package/dist/chain.d.ts +124 -191
  16. package/dist/chain.js +369 -551
  17. package/dist/cli.d.ts +2 -0
  18. package/dist/cli.js +164 -0
  19. package/dist/demo/booking.d.ts +289 -0
  20. package/dist/demo/booking.js +159 -0
  21. package/dist/errors.d.ts +29 -0
  22. package/dist/errors.js +70 -0
  23. package/dist/import.d.ts +179 -0
  24. package/dist/import.js +792 -0
  25. package/dist/index.d.ts +172 -26
  26. package/dist/index.js +304 -178
  27. package/dist/jsonschema.d.ts +22 -0
  28. package/dist/jsonschema.js +167 -0
  29. package/dist/load.d.ts +76 -0
  30. package/dist/load.js +884 -0
  31. package/dist/model.d.ts +166 -0
  32. package/dist/model.js +224 -0
  33. package/dist/ops.d.ts +7 -6
  34. package/dist/ops.js +7 -51
  35. package/dist/pglite.d.ts +22 -0
  36. package/dist/pglite.js +45 -0
  37. package/dist/registry.d.ts +57 -0
  38. package/dist/registry.js +82 -0
  39. package/dist/sql.d.ts +59 -142
  40. package/dist/sql.js +568 -654
  41. package/dist/sync.d.ts +31 -0
  42. package/dist/sync.js +108 -0
  43. package/dist/tx.d.ts +129 -8
  44. package/dist/tx.js +300 -73
  45. package/dist/typed.d.ts +97 -0
  46. package/dist/typed.js +1 -0
  47. package/dist/types.d.ts +71 -250
  48. package/dist/types.js +27 -108
  49. package/dist/up.d.ts +140 -47
  50. package/dist/up.js +339 -267
  51. package/dist/uuid.d.ts +21 -6
  52. package/dist/uuid.js +48 -64
  53. package/dist/validate.d.ts +24 -0
  54. package/dist/validate.js +251 -0
  55. package/dist/watch.d.ts +62 -0
  56. package/dist/watch.js +168 -0
  57. package/dist/write.d.ts +117 -74
  58. package/dist/write.js +658 -720
  59. package/llms.txt +26 -0
  60. package/package.json +49 -19
  61. package/sql/10-core.sql +136 -0
  62. package/sql/15-errors.sql +60 -0
  63. package/sql/20-context.sql +153 -0
  64. package/sql/30-validate.sql +423 -0
  65. package/sql/40-class.sql +259 -0
  66. package/sql/50-acl.sql +539 -0
  67. package/sql/60-write.sql +1369 -0
  68. package/sql/70-read.sql +245 -0
  69. package/sql/80-auth.sql +827 -0
  70. package/sql/90-time.sql +957 -0
  71. package/sql/95-seed.system.sql +178 -0
  72. package/sql/99-revision.sql +3 -0
  73. package/sql/README.md +56 -0
  74. package/sql/seed.booking.sql +39 -112
  75. package/dist/schema.d.ts +0 -15
  76. package/dist/schema.js +0 -351
  77. package/dist/sessions.d.ts +0 -32
  78. package/dist/sessions.js +0 -114
  79. package/dist/tables.d.ts +0 -105
  80. package/dist/tables.js +0 -248
  81. package/docker/Dockerfile +0 -40
  82. package/docker/start.sh +0 -18
  83. package/scripts/check-docs.mjs +0 -375
  84. package/scripts/gen-api-contract.mjs +0 -226
  85. package/scripts/gen-types.mjs +0 -350
  86. package/scripts/release-notes.mjs +0 -76
  87. package/scripts/schema-sync.mjs +0 -185
  88. package/sql/ddl.sql +0 -600
  89. package/sql/seed.auth.sql +0 -73
package/README.en.md ADDED
@@ -0,0 +1,1937 @@
1
+ # letopis 1.0
2
+
3
+ **letopis** is a Node.js library: a store of versioned entities on PostgreSQL 18. Every change is saved as a version
4
+ in a journal. So every object has a history, a state as of any date and a change subscription. The database itself
5
+ checks data integrity, tenant isolation and permissions, with triggers and RLS policies. The rules work the same for
6
+ the library and for raw SQL. Data is read and written with chains:
7
+ `db.Shop(shop).Product({ price: lt(1000) }).rows()`.
8
+
9
+ [Русская версия](README.md) · [Cheatsheet for AI agents](AGENT-CHEATSHEET.en.md) · [Migration from 0.21 (in Russian)](MIGRATION.md) · [Changelog (in Russian)](CHANGELOG.md) · [llms.txt](llms.txt)
10
+
11
+ What letopis can do:
12
+
13
+ - **Zod models** are stored in the database as classes: hubs (objects) and links (relations with their own data) — [section 4](#4-data-model).
14
+ - **Writing with chains**: `create`, `update` with merge, `upsert`, deletion with preview and cascade, the `inc` increment
15
+ for money and counters — [sections 5](#5-writing), [7](#7-deleting-and-restoring) and [8](#8-transactions-batches-and-money).
16
+ - **Reading with chains**: hops along links, filters, trees, aggregates, pagination — [section 6](#6-reading).
17
+ - **History**: versions, point-in-time reads, restoring deleted data, retention policy, hash chain verification —
18
+ [section 11](#11-history-and-point-in-time-reads).
19
+ - **Tenants, sign-in and permissions** in the database itself: passwords, API keys, one-time codes, TOTP, sessions,
20
+ impersonation, access rules — [section 10](#10-users-sign-in-and-permissions).
21
+ - **Change subscription** by cursor without losses, and a **result cache** — [section 12](#12-change-subscription-and-cache).
22
+ - **Bulk loading** of millions of rows and **import from 0.21** — [sections 9](#9-bulk-loading) and [14](#14-migrating-from-021).
23
+
24
+ Requirements:
25
+
26
+ - PostgreSQL 18 or newer with the pgcrypto extension (it is part of the standard distribution), and a database in UTF8
27
+ encoding.
28
+ - Node.js 22 or newer; the package is an ES module.
29
+ - One runtime dependency: `postgres`. Zod 4 is an optional peer dependency: only the `letopis/model` subpath needs it,
30
+ and `import { connect } from 'letopis'` works without it. For development without PostgreSQL there are the optional
31
+ `@electric-sql/pglite` and `@electric-sql/pglite-socket`.
32
+
33
+ ## Contents
34
+
35
+ 1. [Quick start](#1-quick-start)
36
+ 2. [How letopis works](#2-how-letopis-works)
37
+ 3. [Installation and connection](#3-installation-and-connection)
38
+ 4. [Data model](#4-data-model)
39
+ 5. [Writing](#5-writing)
40
+ 6. [Reading](#6-reading)
41
+ 7. [Deleting and restoring](#7-deleting-and-restoring)
42
+ 8. [Transactions, batches and money](#8-transactions-batches-and-money)
43
+ 9. [Bulk loading](#9-bulk-loading)
44
+ 10. [Users, sign-in and permissions](#10-users-sign-in-and-permissions)
45
+ 11. [History and point-in-time reads](#11-history-and-point-in-time-reads)
46
+ 12. [Change subscription and cache](#12-change-subscription-and-cache)
47
+ 13. [Operations](#13-operations)
48
+ 14. [Migrating from 0.21](#14-migrating-from-021)
49
+ 15. [API reference](#15-api-reference)
50
+ 16. [Errors](#16-errors)
51
+ 17. [Performance](#17-performance)
52
+ 18. [Limitations](#18-limitations)
53
+ 19. [Tests](#19-tests)
54
+
55
+ Sections 3–13 are built on one example, a shop: installation, model, writing, reading and so on. The code blocks of
56
+ these sections go in order and together form one program. The `docs` test runs this program, like the quick start
57
+ example, against a real database, so the examples do not drift away from the code.
58
+
59
+ ## 1. Quick start
60
+
61
+ You need PostgreSQL 18 and an admin role: a superuser or a role with the `CREATEROLE` privilege. Install the package:
62
+
63
+ ```sh
64
+ npm install letopis zod postgres # postgres: for createTenant and your own SQL queries
65
+ ```
66
+
67
+ The application connects to the database with a separate role that is a member of `letopis_app`. The installation
68
+ creates `letopis_app`. So after the first installation, create the application role once:
69
+
70
+ ```sql
71
+ create role shop_app login password 'app-secret';
72
+ grant letopis_app to shop_app;
73
+ ```
74
+
75
+ The example installs the schema, creates a shop with its first user, describes two classes and works with them:
76
+
77
+ <!-- run: quickstart -->
78
+ ```ts
79
+ import { up, connect, syncModels, inc, gte } from 'letopis'
80
+ import { hub } from 'letopis/model'
81
+ import { z } from 'zod'
82
+
83
+ const ADMIN = 'postgres://postgres:secret@localhost:5432/shop' // admin: installs the schema
84
+ const APP = 'postgres://shop_app:app-secret@localhost:5432/shop' // application: a member of letopis_app
85
+
86
+ // 1. Install the v2.shop schema. tenant is the first tenant (the shop) and its user Anna.
87
+ const r = await up({ dsn: ADMIN, schema: 'shop',
88
+ tenant: { name: 'Chamomile', user: { name: 'Anna', login: 'anna@shop.ru', password: 'Chamomile-2026!' } } })
89
+ // r.serviceKey is the application's service key: it is issued on the first installation, so save it
90
+
91
+ // 2. The application service signs in with the key, checks Anna's password and works on her behalf in the shop.
92
+ const svc = await connect({ dsn: APP, schema: r.schema, apiKey: r.serviceKey! })
93
+ const login = await svc.auth.verifyPassword({ identifier: 'anna@shop.ru', password: 'Chamomile-2026!' })
94
+ const db = await svc.as(login!.account, { tenant: r.tenant!.tenant })
95
+
96
+ // 3. Model: a shop and a product. syncModels writes the class descriptions to the database.
97
+ const Shop = hub('Shop', { name: z.string() }, { key: ['name'] })
98
+ const Product = hub('Product', { sku: z.string(), title: z.string(), price: z.number().int().min(0) },
99
+ { ends: { Shop }, key: ['Shop', 'sku'] })
100
+ await syncModels(db, [Shop, Product], { apply: true })
101
+
102
+ // 4. Writing and reading with chains.
103
+ const [shop] = await db.Shop().create({ name: 'Chamomile' }).rows()
104
+ const [kettle] = await db.Shop(shop).Product().create({ sku: 'K-1', title: 'Tea kettle', price: 2500 }).rows()
105
+ await db.Product(kettle).update({ price: inc(-500) }).rows() // discount: the database subtracts inside the statement
106
+ const expensive = await db.Shop(shop).Product({ price: gte(1000) }).rows()
107
+ const history = await db.Product(kettle).versions() // two versions: creation and edit
108
+
109
+ await svc.close()
110
+ ```
111
+
112
+ What happened here:
113
+
114
+ - `up()` installed the `v2.shop` schema in the database: three tables, triggers, RLS policies, functions and the
115
+ system seed. The `tenant` option created the tenant "Chamomile" with the rule "authenticated users may do
116
+ everything", and the user Anna with the `owner` role in it.
117
+ - `connect({ apiKey })` exchanged the service key for a session. `verifyPassword` returned `{ account, token }` (or
118
+ `null` if the password is wrong). `svc.as(account, { tenant })` is the same connection pool, but acting as Anna in
119
+ the shop; the database checks that the service has the right to do this.
120
+ - `hub('Product', fields, options)` described an object class. `ends: { Shop }` is a required reference to the shop.
121
+ `key: ['Shop', 'sku']` is the key: the product id is computed from the shop and the SKU, and the database does not
122
+ accept a second product with the same key.
123
+ - `db.Shop(shop).Product().create(…)` created a product and took the reference to the shop from the chain path.
124
+ `inc(-500)` lowered the price inside the `UPDATE` statement, without reading it first. `versions()` read the history
125
+ from the journal.
126
+
127
+ Calling `up()` again is safe: it re-applies the same revision and does not touch the data, and it does not issue the
128
+ service key again while the installation still has a valid API key. Next, section 2 explains the design, and sections
129
+ 3–13 go through the tasks in order.
130
+
131
+ ## 2. How letopis works
132
+
133
+ ### Core concepts
134
+
135
+ | Concept | What it is |
136
+ |---|---|
137
+ | installation | a PostgreSQL schema `v2.<name>` with the letopis tables and functions; one database can hold several |
138
+ | class | a data type: name, fields (JSON Schema), ends and key. A class description is also a row in the database (class `Class`) |
139
+ | hub | an object class: a shop, a product, a customer |
140
+ | link | a relation class between objects, with its own data: an order line, a specialist's skill |
141
+ | end (role) | a reference from one row to another row. An end has a name, the role: `Shop`, `Parent`. Ends are in `row.links`, fields are in `row.data` |
142
+ | key | the fields and ends that identify an object within a class and a tenant. The id is computed from them, so uniqueness is the key |
143
+ | id | a version 5 UUID: a hash of the tenant, the class and the key values; for a class without a key, a hash of a random part |
144
+ | account | a user, a service or an organization: a row of class `Account` |
145
+ | tenant | the account whose data is being worked with. Data of different tenants is isolated |
146
+ | membership | a `member` link "user — tenant" with roles; it allows work in another tenant |
147
+ | session | a token by which the database identifies the account and the tenant. A service gets it with an API key, a user by signing in |
148
+ | version, journal | each change of a row is a version in the `log` journal, with author, time and hash |
149
+ | tombstone | a "deleted" version with the last snapshot of the object |
150
+ | chain | a query like `db.Shop(shop).Product().rows()`: steps, modifiers, write verbs and a terminal |
151
+ | terminal | a method that executes the chain: `rows`, `first`, `ids`, `count`, `paths`, aggregates |
152
+
153
+ ### What is stored in the database
154
+
155
+ There are three tables. `entity` is the current state: one row per live entity, including the rows of system classes
156
+ (`Class`, `Account`, `Resource`, `member`, `rule`). `log` is the journal: a full snapshot of every version with author,
157
+ time, kind of operation and hash. `secret` holds credentials and sessions: the application sees none of its rows; only
158
+ the sign-in functions work with it.
159
+
160
+ Classes are entities too. A class description is stored in a `Class` row; the database also keeps the expanded
161
+ description (with ancestors applied) and the compiled data check there. The version hash is a generated column: sha256 of the
162
+ previous version's hash and the row snapshot. So the next version exposes a tampered version, and for the last version
163
+ an anchor outside the database does this ([section 11](#11-history-and-point-in-time-reads)).
164
+
165
+ ### Who checks the data
166
+
167
+ There are two roles. `letopis_owner` owns the schema, tables and functions. `letopis_app` is the application role: the
168
+ application's LOGIN role is a member of this role only. Everything the application runs, both the library and raw SQL,
169
+ goes through triggers and RLS.
170
+
171
+ - **Triggers.** On insert, a trigger computes the id, fills in defaults and checks the data against the class JSON
172
+ Schema (the schema is compiled to jsonpath). It checks the ends and their targets, fills in the service columns and
173
+ writes a version to the journal. An update merges the patch inside the statement (`merge`, `inc`). After a delete,
174
+ the trigger applies the `restrict`, `cascade` and `unset` rules and writes tombstones.
175
+ - **RLS and permissions.** Policies isolate tenants and apply access rules. At the start of a query, the database
176
+ builds the actor's permission plan once (`acl_plan()`); each row then needs only the cheap `acl_ok` check.
177
+ - **Query context** is the transaction settings: `letopis.token` (the session), `letopis.tenant` and `letopis.account`
178
+ (the tenant and impersonation), `letopis.reason` (the reason for the change) and `letopis.acl` (the permission pass,
179
+ see [section 10](#10-users-sign-in-and-permissions)). The database finds the session by the sha256 of the token, so
180
+ without a real token the settings give nothing.
181
+ - **Bypassing the checks** is possible only for code that runs as the owner: owner functions (`security definer` with
182
+ a fixed `search_path`) and the loader on an admin connection. `connect()` refuses a role that bypasses RLS (a
183
+ superuser, `BYPASSRLS`, a member of `letopis_owner`) unless `allowBypassRls: true` is set.
184
+
185
+ ### How reads work
186
+
187
+ Under RLS, the indexes on `data` and on back references are not directly available to the policy. So the ids for chain
188
+ steps are found by the owner functions `find_ids()`, `find_hop()` and `find_deep()`. They apply the same tenant and
189
+ permission conditions as the policy. The rows themselves are then read by primary key under RLS. A point-in-time read
190
+ is built from the journal.
191
+
192
+ ### Signals
193
+
194
+ A write reports the changed classes through `NOTIFY`. Raw SQL sends the signal from a trigger, and PostgreSQL delivers
195
+ it on commit. Library transactions defer their signals: the process notifier sends them every 100 ms
196
+ (`notifyIntervalMs`). A signal carries only a class name or the start of the token hash of a revoked session: no ids,
197
+ no data. On signals, the class registry is re-read, the result cache is cleared and subscriptions wake up.
198
+
199
+ ## 3. Installation and connection
200
+
201
+ ### Installation: `up()`
202
+
203
+ `up()` waits for the database and creates it if it does not exist. It checks for PostgreSQL 18 and UTF8 encoding,
204
+ installs the roles `letopis_owner` and `letopis_app`, the pgcrypto extension and the `v2.<name>` schema with all
205
+ functions and the system seed. On the first installation, it issues a service key. Call it from an admin role.
206
+
207
+ | Option | What it does |
208
+ |---|---|
209
+ | `dsn` or `sql` | an admin connection string or a ready postgres.js pool |
210
+ | `schema` | the installation name: `'shop'` gives the schema `v2.shop`; you can also pass `'v2.shop'` directly |
211
+ | `tenant` | the first tenant and its user (see below) |
212
+ | `seeds` | seeds after the core: `'booking'` is the salon demo domain; or paths to your own `.sql` files |
213
+ | `history` | the history retention policy for classes without their own ([section 11](#11-history-and-point-in-time-reads)) |
214
+ | `serviceKey` | your own service key instead of a random one: `lts_` and 32–128 hexadecimal characters |
215
+ | `allowReset` | allow `db.reset()` ([section 13](#13-operations)); off by default |
216
+ | `fresh` | drop the schema and install it again — **the schema data is lost** |
217
+ | `upgrade` | upgrade a schema of an older revision: migrations and a re-apply |
218
+ | `pglite` | a database for development without PostgreSQL (see below) |
219
+ | `quiet`, `waitTimeoutMs` | no console messages; how long to wait for the database (60,000 ms by default) |
220
+ | `owner`, `app` | the names of the owner and application roles (`letopis_owner`, `letopis_app` by default) |
221
+
222
+ Result: `schema` (the full name), `serviceKey` (on the first installation, and on a re-run when the installation has no
223
+ valid API key left; otherwise `null`), `created`, `upgraded`, `revision`, `migrations`, `analyzed`, `owner`, `app`,
224
+ and also `tenant` and `pglite` if they were requested.
225
+
226
+ <!-- run: guide -->
227
+ ```ts
228
+ import { up, connect } from 'letopis'
229
+
230
+ const ADMIN = 'postgres://postgres:secret@localhost:5432/shop'
231
+ const APP = 'postgres://shop_app:app-secret@localhost:5432/shop'
232
+
233
+ const r = await up({ dsn: ADMIN, schema: 'shop',
234
+ tenant: { name: 'Chamomile', user: { name: 'Anna', login: 'anna@shop.ru', password: 'Chamomile-2026!' } } })
235
+ const SHOP = r.tenant!.tenant // the tenant: the Chamomile shop
236
+ const ANNA = r.tenant!.user! // its first user
237
+ ```
238
+
239
+ ### Application role
240
+
241
+ The application works as a LOGIN role that is a member of `letopis_app` only:
242
+ `create role shop_app login password '…'; grant letopis_app to shop_app;`. Then RLS and triggers apply to it, as to
243
+ any raw SQL from this role. `connect()` rejects a role that bypasses RLS with the error `bypass_rls`. The admin
244
+ connection (`allowBypassRls: true`) is needed only for the loader, verification and maintenance; keep it in a separate
245
+ process, not in the web application.
246
+
247
+ ### First tenant and user
248
+
249
+ Permissions in letopis are always on and deny everything by default. So a new tenant needs rules. The
250
+ `up({ tenant })` option and the `createTenant(sql, schema, opts)` function do this in one transaction:
251
+
252
+ - the tenant: an account in System with an id from the name (`tenant:<name>`);
253
+ - the rule "authenticated users may do everything" in this tenant (`grantAll`, on by default, weight 10);
254
+ - a user (id from the login) with a membership in the tenant with the roles `user.roles` (`['owner']` by default), a
255
+ password and a session in the tenant.
256
+
257
+ A repeat with the same name and login duplicates nothing and does not change the password; it only issues a new
258
+ session. The login goes into the id as is: `Anna@shop.ru` and `anna@shop.ru` would give two accounts, and the second one
259
+ would have no password, so write logins in one case. The `created` field in the result tells whether this call created
260
+ the tenant: for an existing tenant with a new user it is `false`. The roles are set by `user.roles`, a non-empty list
261
+ of names (not masks), otherwise `invalid_data`. A repeat does not change the roles of an existing membership: they are
262
+ changed by editing the membership ([Granting a permission](#granting-a-permission)). A repeat does not bring back
263
+ what was deleted: if the tenant, the user account or the membership is deleted (the row is gone but its history is in
264
+ the journal), `createTenant` answers `invalid_data` and writes nothing. The error text names what was deleted and
265
+ advises bringing it back with `restore()`: an account with `db.Account(id).restore()` on behalf of the system
266
+ administrator, a membership with `db.member(id).restore()` in that tenant; the other way out is a different name or
267
+ login. The session from `createTenant` has no sign-in method, so the user cannot change their own password with it:
268
+ for that the user signs in with the password. `createTenant` needs an admin postgres.js pool:
269
+
270
+ <!-- run: guide -->
271
+ ```ts
272
+ import postgres from 'postgres'
273
+ import { createTenant } from 'letopis'
274
+
275
+ const pg = postgres(ADMIN)
276
+ const lutik = await createTenant(pg, r.schema, {
277
+ name: 'Buttercup', user: { name: 'Boris', login: 'boris@lutik.ru', password: 'Buttercup-2026!' } })
278
+ await pg.end()
279
+ const BORIS = lutik.user! // lutik: { tenant, user, token, created: true }
280
+ ```
281
+
282
+ ### Connecting: `connect()`
283
+
284
+ A connection gets a session in one of two ways: with an API key (`apiKey`) or with a token (`token`). An application
285
+ service usually keeps one connection by key and acts on behalf of users through `as()`:
286
+
287
+ <!-- run: guide -->
288
+ ```ts
289
+ // service: the API key is exchanged for a service session
290
+ const svc = await connect({ dsn: APP, schema: r.schema, apiKey: r.serviceKey! })
291
+ // sign-in: the service checks the password and acts on behalf of the user in the tenant
292
+ const login = await svc.auth.verifyPassword({ identifier: 'anna@shop.ru', password: 'Chamomile-2026!' })
293
+ const db = await svc.as(login!.account, { tenant: SHOP })
294
+ // a separate connection by session token, for example in another process
295
+ const anna = await connect({ dsn: APP, schema: r.schema, token: r.tenant!.token! })
296
+ ```
297
+
298
+ | `connect()` option | What it does |
299
+ |---|---|
300
+ | `dsn` or `sql` | a connection string or a ready postgres.js pool |
301
+ | `schema` | the full installation name: `'v2.shop'` (it is handy to take `r.schema`) |
302
+ | `token`, `apiKey` | a session token or an API key (`service` is a synonym of `apiKey`) |
303
+ | `models` | `letopis/model` models, only for chain types ([section 4](#4-data-model)) |
304
+ | `max` | pool size (10 by default) |
305
+ | `listen` | listen to database signals (yes by default; `false` for PgBouncer in transaction mode) |
306
+ | `cache` | result cache: `{ ttl, max, ttlOnly }` or `false` ([section 12](#12-change-subscription-and-cache)) |
307
+ | `maintain` | run `maintain()` on a schedule: `'1h'`, `'15m'` or milliseconds |
308
+ | `onQuery`, `slowMs` | a hook on every chain query and the slow query threshold |
309
+ | `refreshSession` | extend the session once per minute of activity (yes by default) |
310
+ | `notifyIntervalMs` | the interval for sending deferred signals (100 ms) |
311
+ | `allowBypassRls`, `owner` | allow a role that bypasses RLS (admin connection); the name of the owner role |
312
+
313
+ A connection has `db.schema`, `db.token`, `db.tenant` (the session tenant) and `db.close()`. Connections obtained
314
+ through `as()` share the pool with the original one: close the original one.
315
+
316
+ ### Admin connection
317
+
318
+ The loader needs a connection of a role that can become the owner, with `allowBypassRls: true`. `verify()` and
319
+ `maintain()` work on it too, and also for a system administrator on a regular connection. The admin connection prints
320
+ the warning "RLS не действует" ("RLS does not apply"); this is expected. The session sets the tenant it works with. A
321
+ separate loading process is better off with a service key:
322
+ `connect({ dsn: ADMIN, schema, apiKey, allowBypassRls: true })`, and the tenant of the rows is set by the
323
+ `load(…, { tenant })` option. The example uses the session from `up()`:
324
+
325
+ <!-- run: guide -->
326
+ ```ts
327
+ const adm = await connect({ dsn: ADMIN, schema: r.schema, token: r.tenant!.token!, allowBypassRls: true })
328
+ ```
329
+
330
+ ### Upgrading the package and schema
331
+
332
+ The schema has an engine revision (currently 5). If a new package version brings a new revision, `connect()` only warns
333
+ and names the command. It is `up()` that refuses: a lagging schema without `upgrade: true`, and a schema newer than the
334
+ package. So a startup script that calls `up()` on every start will stop after a package update until it gets
335
+ `upgrade: true`. `up({ dsn, schema, upgrade: true })` applies the migrations from `sql/migrate/` and re-applies the
336
+ functions; the data is kept.
337
+
338
+ ### Development without PostgreSQL: PGlite
339
+
340
+ `up({ pglite: true })` starts PGlite (PostgreSQL 18 on WebAssembly) with pgcrypto and a socket on a free port. It needs
341
+ the `@electric-sql/pglite` and `@electric-sql/pglite-socket` packages. There is only one connection, so:
342
+
343
+ ```ts
344
+ const dev = await up({ schema: 'dev', pglite: true })
345
+ const local = await connect({ dsn: dev.pglite!.dsn, schema: dev.schema, apiKey: dev.serviceKey!,
346
+ listen: false, max: 1, allowBypassRls: true })
347
+ // … work …
348
+ await local.close()
349
+ await dev.pglite!.close()
350
+ ```
351
+
352
+ The connection to PGlite is a superuser, so it needs `allowBypassRls: true`, and RLS does not apply on it: tenant
353
+ isolation and permissions cannot be checked on PGlite. It is not fit for race tests and CI either: there is one
354
+ connection and no `LISTEN`.
355
+
356
+ ## 4. Data model
357
+
358
+ Classes are described with Zod models from the `letopis/model` subpath and written to the database. The database
359
+ stores a normalized JSON Schema and checks the data itself on every write.
360
+
361
+ ### Hubs and links
362
+
363
+ `hub()` describes an object, `link()` describes a link with its own data. Both have two forms:
364
+
365
+ - short: `hub(name, fields, options)`: fields as an object of Zod types, options as the third argument;
366
+ - full: `hub(name, { fields, ends, key, extends, … })`: it is needed, for example, for inheritance (`extends`), so
367
+ that TypeScript infers the data types of the descendant.
368
+
369
+ The shop model that the following examples are built on:
370
+
371
+ <!-- run: guide -->
372
+ ```ts
373
+ import { z } from 'zod'
374
+ import { hub, link, one, many } from 'letopis/model'
375
+ import { syncModels } from 'letopis'
376
+
377
+ const Shop = hub('Shop', { name: z.string(), city: z.string().optional() }, { key: ['name'] })
378
+ const Category = hub('Category', { name: z.string() }, {
379
+ key: ['name'],
380
+ ends: { Parent: one('Category').optional().onDelete('unset') }, // tree: a reference to the parent
381
+ })
382
+ const Product = hub('Product', {
383
+ sku: z.string(), title: z.string(), price: z.number().int().min(0), views: z.number().int().optional(),
384
+ }, {
385
+ ends: { Shop, Category: many(Category, { max: 5, onDelete: 'unset' }) },
386
+ key: ['Shop', 'sku'],
387
+ meta: { title: 'Product', icon: 'mdi:package-variant' },
388
+ })
389
+ const Customer = hub('Customer', { name: z.string(), email: z.string() }, {}) // no key: the database assigns the id
390
+ const Vip = hub('Vip', { extends: Customer, fields: { discount: z.number().int().min(0).max(50) } })
391
+ const Order = hub('Order', {
392
+ number: z.string(), status: z.enum(['new', 'paid', 'cancelled']).default('new'),
393
+ }, { ends: { Customer, Shop }, key: ['Shop', 'number'] })
394
+ const line = link('line', { qty: z.number().int().min(1), price: z.number().int().min(0) }, {
395
+ ends: { Order: one(Order).onDelete('cascade'), Product },
396
+ key: ['Order', 'Product'],
397
+ })
398
+ const Wallet = hub('Wallet', { number: z.string(), balance: z.number().int().min(0).default(0) },
399
+ { ends: { Customer }, key: ['number'] })
400
+ const Movement = link('Movement', {
401
+ opId: z.string(), kind: z.enum(['topup', 'debit', 'adjust']), amount: z.number().int(), target: z.number().int().optional(),
402
+ }, { ends: { Wallet }, key: ['Wallet', 'opId'] })
403
+
404
+ const models = [Shop, Category, Product, Customer, Vip, Order, line, Wallet, Movement]
405
+ const plan = await syncModels(db, models) // dry run: what will be created and changed
406
+ const applied = await syncModels(db, models, { apply: true }) // { applied: true, created: ['Shop', …], … }
407
+ ```
408
+
409
+ ### Fields
410
+
411
+ Fields are Zod types: strings, numbers (`int`, `min`, `max`), booleans, enums (`z.enum`), dates and times
412
+ (`z.iso.date()`, `z.iso.datetime()`), nested objects of any depth, arrays and `z.record`. `.optional()` makes a field
413
+ optional. `.default(…)` sets a default that the database fills in on insert. The database stores the `.meta()`
414
+ annotations of a field (`title`, `description`, your own keys), but they do not affect data checks.
415
+
416
+ The `refine`, `superRefine` and `transform` rules cannot be expressed in JSON Schema, and the database does not run
417
+ them. `syncModels` returns them in `lost`: check such rules in the application. A description with a JSON Schema
418
+ keyword that the database does not know is rejected, so no rule is lost silently. `pattern` regexes are a limited
419
+ subset of regular expressions shared by Zod, the validator and the database.
420
+
421
+ ### Ends
422
+
423
+ - `ends: { Shop }` is the same as `one(Shop)`: a single required end with the role `Shop`.
424
+ - `one(A, B)` is a union: the end accepts rows of several classes. `one('Category')` is a target by name: this is how
425
+ a class refers to itself.
426
+ - `.optional()` makes the end optional.
427
+ - `many(X, { min, max, onDelete })` is a multiple end: a list of targets in one end.
428
+ - `.onDelete(…)` sets what happens when a target is deleted: `restrict` (the default: deleting the target is
429
+ forbidden), `cascade` (delete this row too), `unset` (remove the reference; only for an optional single end and for a
430
+ multiple end without `min`).
431
+
432
+ The database checks the targets: the target exists, belongs to the same tenant or to System, and is of the declared
433
+ class or its descendant. A role name points to one class across the whole schema. So the roles `User` and `Tenant`
434
+ (pointing to `Account`), `Group` and `Object` (pointing to `Resource`) are taken by system classes: your own end with
435
+ such a name is possible only with the same target. In a result row, a single end is the target id (`row.links.Shop`),
436
+ and a multiple end is a list of ids.
437
+
438
+ ### Key and id
439
+
440
+ A key is made of required string fields and required single ends. Numbers, booleans, objects, optional fields and
441
+ multiple ends cannot be part of a key. The id of a class with a key is a version 5 UUID of the tenant, the class and
442
+ the key values. The values are taken as they are, without normalization: an email in a different case gives a
443
+ different id. Set the required format of a key field in the model, and the database will check it.
444
+
445
+ - You do not pass the id: the database computes it. A different id gives `id_mismatch`; an id for a class without a
446
+ key gives `id_from_db`.
447
+ - `db.idOf(class, key)` computes the id in advance, for example `db.idOf('Shop', { name: 'Chamomile' })`.
448
+ - A class without a key (`Customer` above) gets an id from a random part; you can learn it from the write result.
449
+ - A key field cannot be changed by a regular update (`immutable_key`): a new key means a new id, and that is `rekey`
450
+ ([section 5](#5-writing)).
451
+
452
+ If something other than the key must be unique, make it a separate class. For example, the email is the key of an
453
+ `Email` hub, and the link from a person to it has the key `['Email']`. It is better not to make personal data the key
454
+ of the person itself: an id from an email can be found by brute force, and it stays in references and in the journal.
455
+
456
+ ### Inheritance and aliases
457
+
458
+ `extends`: the descendant gets the fields, ends and key of the ancestor. Its own field with the same name replaces the
459
+ ancestor's field entirely. `abstract: true`: the class never has its own rows. A chain step by class also returns the
460
+ rows of descendants: `db.Customer()` also finds `Vip`, and `.exact()` keeps only the class itself. `alias` is a second
461
+ name of the class for a chain step, for example a name in another language.
462
+
463
+ ### Class settings
464
+
465
+ - `history: false`: no journal is kept for the class: no versions, no point-in-time reads, no restore and no
466
+ subscription events. A retention policy is `history: { all: '1 day', daily: '3 months', … }`
467
+ ([section 11](#11-history-and-point-in-time-reads)).
468
+ - `ownerDefault: 'actor'`: the owner of new rows is the user, not the tenant (the default is `'tenant'`). This is
469
+ useful for the rule "everyone sees their own records".
470
+ - `cache: { ttl }`: reads of this class alone are cached without `.cache()` ([section 12](#12-change-subscription-and-cache)).
471
+ - `meta`: free-form metadata for the application: a title, an icon, a color, an order in a menu. Up to 64 KB,
472
+ inherited key by key (`null` removes an inherited value), visible to anyone who has a session. For an end, use
473
+ `.meta()`.
474
+
475
+ ### Writing the model to the database
476
+
477
+ `syncModels(db, models)` without `apply` is a dry run; with `{ apply: true }` it applies the changes. The result is
478
+ `{ applied, created, updated, unchanged, skipped, lost }`; `skipped` lists System classes, which only the installation
479
+ changes. `syncModels` touches only the classes passed to it and deletes nothing. Safe changes (a new optional field or
480
+ end, metadata, an alias) are applied at once. A tightening checks all rows of the family; if some rows conflict with it, you get
481
+ `tightening_conflict` with a list. Classes are written to the session tenant, and other tenants do not see them.
482
+ Classes shared by everyone are in System: the installation seed or the import puts them there.
483
+
484
+ The same from the command line:
485
+
486
+ ```sh
487
+ npx letopis sync ./models.js --dsn "$APP" --schema v2.shop --token "$TOKEN" # dry run
488
+ npx letopis sync ./models.js --dsn "$APP" --schema v2.shop --token "$TOKEN" --apply # apply
489
+ npx letopis types --dsn "$APP" --schema v2.shop --token "$TOKEN" --out types.ts # class types from the database
490
+ ```
491
+
492
+ ### Chain types
493
+
494
+ `connect({ …, models })` gives types of steps, rows and filters from the models. The library still takes the class
495
+ descriptions from the database; only TypeScript needs the models. The row data type is `Infer<typeof Product>`, the
496
+ ends type is `InferLinks<typeof Product>`. Your own `meta` shape is typed by extending the `ClassMeta` interface.
497
+
498
+ ### Class names
499
+
500
+ A chain step is named after a class, its alias or a role name. The Proxy parses the names of chain and root methods
501
+ (`count`, `sort`, `set`, `sql`, `auth` and others) before classes. So a class with such a name cannot be reached as a
502
+ step, and the database does not accept a link with such a name. The full list is in the
503
+ ["Reserved names"](#reserved-names) section.
504
+
505
+ ## 5. Writing
506
+
507
+ The write verbs — `create`, `update`, `upsert`, `delete`, `anonymize`, `reclass`, `rekey`, `restore`, `purge` — are
508
+ parts of a plan. A verb does nothing by itself: a terminal (`rows`, `first`, `ids`, `count`) executes the plan, and the
509
+ whole plan is one transaction.
510
+
511
+ ### Create: `create`
512
+
513
+ <!-- run: guide -->
514
+ ```ts
515
+ import { LetopisError } from 'letopis'
516
+
517
+ const [shop] = await db.Shop().create({ name: 'Chamomile', city: 'Kazan' }).rows()
518
+ const [tea] = await db.Category().create({ name: 'Tea' }).rows()
519
+ const [green] = await db.Category().create({ name: 'Green tea' }).Parent.set(tea).rows()
520
+ const [kettle] = await db.Shop(shop).Product().create({ sku: 'K-1', title: 'Tea kettle', price: 2500 }).rows()
521
+ const [sencha] = await db.Shop(shop).Product()
522
+ .create({ sku: 'T-7', title: 'Sencha', price: 900 }).Category.add(green).rows()
523
+ const [olga] = await db.Customer().create({ name: 'Olga', email: 'olga@mail.ru' }).rows()
524
+
525
+ try {
526
+ await db.Shop().create({ name: 'Chamomile' }).rows()
527
+ } catch (e) {
528
+ if (!(e instanceof LetopisError) || e.code !== 'exists') throw e // e.detail.id === shop.id: this shop already exists
529
+ }
530
+ ```
531
+
532
+ - `create` only creates. A taken key gives the error `exists` with `detail.id`: the library computes the id itself. So
533
+ for a class with a key, a retry after a network failure shows at once that the first attempt went through. A class
534
+ without a key has no such protection: a retry creates a second object.
535
+ - The tags of a new row are set by `.tags([…])` before the verb, the owner by `.owner(id)`; an explicit owner must be
536
+ allowed by the permissions.
537
+ - A deleted object can be created again with the same key: its history continues.
538
+ - The result is the rows of the created objects with all columns ([section 15](#15-api-reference), "Result row").
539
+
540
+ ### Update: `update`
541
+
542
+ <!-- run: guide -->
543
+ ```ts
544
+ import { gte } from 'letopis'
545
+
546
+ await db.Product(kettle).update({ title: 'Glass tea kettle' }).rows()
547
+ await db.Shop(shop).Product({ price: gte(2000) }).update({ title: 'Glass tea kettle, 1 L' }).rows()
548
+ ```
549
+
550
+ `update` edits every target of the step and never creates. The patch is merged with the data inside the statement:
551
+ nested objects are merged recursively, while arrays, scalars and `null` are replaced. So concurrent edits of different
552
+ fields are not lost. If the step has no filter of its own, the target is taken from `data.id` of the patch, and without
553
+ `data.id` all entities of the step are edited: `db.Product().update(…)` edits all products of the tenant. If the step
554
+ found nothing, the result is empty and there is no error, so check the length of the result. If `update` affected fewer
555
+ rows than the step found, the whole plan is rolled back: a row deleted in the meantime gives `target_not_found`, and a
556
+ row that is visible but may not be edited gives `acl_denied`. The `.tags(…)` modifier before `update` both selects the
557
+ rows and replaces their tags entirely.
558
+
559
+ ### Create or replace: `upsert`
560
+
561
+ <!-- run: guide -->
562
+ ```ts
563
+ const [kept] = await db.Shop().upsert({ name: 'Chamomile', city: 'Kazan' }).rows() // kept.$upsert: 'unchanged'
564
+ const [moved] = await db.Shop().upsert({ name: 'Chamomile', city: 'Ufa' }).rows() // moved.$upsert: 'updated'
565
+ ```
566
+
567
+ `upsert` creates an object, and if an object with this key already exists, it replaces its data, ends and tags
568
+ entirely. This is one `INSERT … ON CONFLICT` statement, so the race "checked that it does not exist, then created it"
569
+ cannot happen. Rules:
570
+
571
+ - only for a class with a key, otherwise `no_key` without a call to the database;
572
+ - the object is passed whole: fields that are not in the passed object disappear;
573
+ - the result carries `$upsert`: `created`, `updated` or `unchanged`;
574
+ - `upsert` cannot increment and overwrites concurrent increments, so it is not suitable for money;
575
+ - to sync large reference tables, the loader with `existing: 'update'` is cheaper ([section 9](#9-bulk-loading)).
576
+
577
+ ### Ends: from the path and slots
578
+
579
+ The ends of a created row are taken from the chain path: `db.Shop(shop).Product().create(…)` references `shop`. If
580
+ several roles fit, the role whose slot is free is chosen, then the role named after the step class; otherwise you get
581
+ `invalid_query`. A slot after the verb sets an end explicitly: `.RoleName.set(value)` and `.RoleName.unset()`, and for
582
+ a multiple end also `.add()` and `.remove()`. The value is an id, a list of ids, a result row or a nested chain. Slots
583
+ change `links` atomically, without reading the list:
584
+
585
+ <!-- run: guide -->
586
+ ```ts
587
+ await db.Product(sencha).update({}).Category.add(tea).rows() // one more category
588
+ await db.Product(sencha).update({}).Category.remove(tea).rows()
589
+ await db.Category(green).update({}).Parent.unset().rows() // the category is now a root
590
+ await db.Category(green).update({}).Parent.set(tea).rows() // and inside "Tea" again
591
+ ```
592
+
593
+ ### Numbers and counters: `inc`
594
+
595
+ <!-- run: guide -->
596
+ ```ts
597
+ import { inc } from 'letopis'
598
+
599
+ await db.Product(kettle).update({ price: inc(-500) }).rows() // the price went down by 500
600
+ await db.Product(kettle).update({ views: inc(1, { start: 0 }) }).rows() // the field does not exist yet: start from 0
601
+ ```
602
+
603
+ `inc(n)` adds the amount inside the `UPDATE` statement, over the fresh locked row. So concurrent increments are not
604
+ lost, and a class rule like `min(0)` is checked against the result: going below zero is rejected with `invalid_data`,
605
+ without reads or locks in the application. Rules:
606
+
607
+ - only in `update`; a marker in `create`, `upsert`, the loader or inside an array gives `invalid_data` before sending;
608
+ - the field must already be a number; if it is missing, you get `invalid_data`, or `start` is used. `null` and a
609
+ non-number are an error even with `start`;
610
+ - the path goes only through existing objects;
611
+ - an `inc` marker cannot come from JSON: `{ "$inc": 5 }` that came from the network stays data;
612
+ - store money as an integer number of minor units (cents), no greater than 2^53 − 1.
613
+
614
+ ### Protection against concurrent edits: `{ rev }`
615
+
616
+ <!-- run: guide -->
617
+ ```ts
618
+ const cur = await db.Product(kettle).first()
619
+ await db.Product(kettle).update({ price: 1900 }, { rev: cur!.rev }).rows() // the version matched: written
620
+ try {
621
+ await db.Product(kettle).update({ price: 1800 }, { rev: cur!.rev }).rows() // the version is already different
622
+ } catch (e) {
623
+ if (!(e instanceof LetopisError) || e.code !== 'conflict') throw e // re-read and decide again
624
+ }
625
+ ```
626
+
627
+ The strict mode is needed when the new value is computed from what was read: if the row was changed in the meantime,
628
+ you get `conflict`, and nothing is written. The mode works when the step has one target (otherwise `rev_ambiguous`).
629
+ The library does not retry `conflict` by itself: the application decides what to do with the changed data. Another way
630
+ is a read with a `forUpdate()` lock ([section 8](#8-transactions-batches-and-money)).
631
+
632
+ ### Changing the key or class
633
+
634
+ <!-- run: guide -->
635
+ ```ts
636
+ const [mug] = await db.Shop(shop).Product().create({ sku: 'M-1', title: 'Mug', price: 400 }).rows()
637
+ const [mug2] = await db.Product(mug).rekey({ sku: 'M-100' }).rows() // new key, new id; references moved
638
+ const [vip] = await db.Customer(olga).reclass('Vip', { name: 'Olga', email: 'olga@mail.ru', discount: 10 }).rows()
639
+ ```
640
+
641
+ - `rekey(patch)` sets a new key value: a row with a new id, all references moved, the old row deleted. If a referencing
642
+ row includes this end in its own key, its id changes too, and the move goes on. The journal links the old and the new
643
+ id (`moved`).
644
+ - `db.rekeyClass(class, key)` sets a new key for all rows of the family, in one transaction.
645
+ - `reclass(class, data)` changes the class; the data is optional, and if given, it is given whole. If the new class has
646
+ no key, the id is kept (as for `vip` above); if it has one, the id is computed anew and the references are moved.
647
+ The roles of referencing rows must accept the new class, otherwise `reclass_denied`. A raw `UPDATE … SET class`
648
+ passes only if neither the old nor the new class has a key (otherwise `use_reclass`).
649
+ - Deleting, changing the class and changing the key work only in `read committed`: library transactions always open at
650
+ this level.
651
+
652
+ ### Anonymize: `anonymize`
653
+
654
+ <!-- run: guide -->
655
+ ```ts
656
+ await db.Customer(olga).anonymize(['email']).rows() // email → '[erased]', tag anonymized
657
+ ```
658
+
659
+ `anonymize(fields)` overwrites string fields with a new version and marks the row with the `anonymized` tag. The
660
+ earlier versions stay in the journal. For complete erasure, the next steps are history trimming
661
+ ([section 11](#11-history-and-point-in-time-reads)), then deletion and `purge` ([section 7](#7-deleting-and-restoring)).
662
+
663
+ ### How a plan runs
664
+
665
+ One plan can create several related objects. The chain after a verb continues from the verb's result: after `create`
666
+ and `upsert`, from the created row; after other verbs, from all rows.
667
+
668
+ <!-- run: guide -->
669
+ ```ts
670
+ const [l1] = await db.Customer(olga).Order()
671
+ .create({ number: 'A-1' }).Shop.set(shop) // order: Customer from the path, Shop via a slot
672
+ .line().create({ qty: 2, price: 2000 }).Product.set(kettle) // order line: Order from the path
673
+ .rows() // rows of the last step: the order line
674
+ const order = l1.links.Order as string
675
+ await db.Order(order).line().create({ qty: 1, price: 900 }).Product.set(sencha).rows()
676
+ ```
677
+
678
+ - The whole plan is one `read committed` transaction. On a deadlock (`40P01`), a serialization failure (`40001`) and a
679
+ journal conflict, the library retries the whole plan, up to three times.
680
+ - An error in any part rolls back the whole plan. `exists` and `conflict` are not retried: they are answers for the
681
+ application.
682
+ - Modifiers before a verb (`limit`, `sort`) narrow its targets.
683
+ - A verb right after a verb writes to the same entities.
684
+
685
+ ## 6. Reading
686
+
687
+ ### Steps and hops
688
+
689
+ A chain starts with a class: `db.Shop()` is all shops, `db.Shop(shop)` is one shop. The next step is a class, an alias
690
+ or an end role. A hop goes along ends: a role step goes along its own end; other steps go along the ends of all classes
691
+ of both families. From an object to the rows that reference it, the hop goes backward; to the target of a reference,
692
+ forward; to the same class, to the children.
693
+
694
+ <!-- run: guide -->
695
+ ```ts
696
+ const products = await db.Shop(shop).Product().rows() // products of the shop: backward along the Shop end
697
+ const shopOf = await db.Product(kettle).Shop().first() // the shop of the product: forward
698
+ const inOrder = await db.Order(order).line().Product().rows() // products of the order: hub → link → hub
699
+ const parent = await db.Category(green).Parent().first() // role step: the parent category
700
+ const allCustomers = await db.Customer().rows() // together with descendants (Vip)
701
+ const onlyCustomer = await db.Customer().exact().rows() // only the class itself
702
+ ```
703
+
704
+ ### Filters
705
+
706
+ A step filter is an id, a list of ids, a result row, an object of fields or `or(…)`. An object of fields can have any
707
+ depth. A plain value means equality through containment in `data`, without type casting: `{ price: 900 }` will not find
708
+ a record with the price `"900"`. A list as a field value means "the array contains all the elements". Operators cast
709
+ their value to the field type from the JSON Schema. An end role in a filter takes a target id or `null`.
710
+
711
+ <!-- run: guide -->
712
+ ```ts
713
+ import { lt, ilike, between, or, not } from 'letopis'
714
+
715
+ const cheap = await db.Product({ price: lt(1000) }).rows()
716
+ const teaCount = await db.Product({ title: ilike('%tea%'), price: between(500, 3000) }).count()
717
+ const twoIds = await db.Product(or({ sku: 'K-1' }, { sku: 'T-7' })).ids()
718
+ const active = await db.Order({ status: not('cancelled') }).rows()
719
+ const ofShop = await db.Product({ Shop: shop.id }).rows()
720
+ const roots = await db.Category({ Parent: null }).rows() // root categories
721
+ ```
722
+
723
+ | Operator | Condition |
724
+ |---|---|
725
+ | `ne(v)`, `gt(v)`, `gte(v)`, `lt(v)`, `lte(v)` | ≠ (taking `null` into account), >, ≥, <, ≤ |
726
+ | `between(a, b)`, `inList([…])` | a ≤ x ≤ b; a value from the list |
727
+ | `like(s)`, `ilike(s)`, `starts(s)`, `ends(s)` | a pattern with `%` and `_`; case-insensitive; starts with; ends with |
728
+ | `has(v)`, `hasAny([…])`, `hasAll([…])` | an array field contains the value; at least one; all |
729
+ | `exists(true \| false)`, `isNull()` | the field is present or not; the field is `null` or missing |
730
+ | `not(v)` | negation of an operator; for a scalar, "not equal" |
731
+ | `or(f1, f2, …)` | any of the step filters |
732
+
733
+ Tags and the row owner are filtered with the `.tags(…)` and `.owner(…)` modifiers.
734
+
735
+ ### Terminals and the result row
736
+
737
+ | Terminal | What it returns |
738
+ |---|---|
739
+ | `rows()` | the rows of the entities of the last step, without duplicates |
740
+ | `first()` | the first row or `null` |
741
+ | `ids()` | the ids of the entities of the last step |
742
+ | `count()` | the number of entities of the last step; `count({ paths: true })` is the number of paths |
743
+ | `paths()` | paths: `[{ Step: Row, … }]`, a node for each step |
744
+ | `versions()` | all versions of the entities of the last step ([section 11](#11-history-and-point-in-time-reads)) |
745
+ | `sum`, `avg`, `min`, `max`, `countBy` | aggregates over the field `'data.<path>'`; on an empty set, `null`, and `{}` for `countBy` |
746
+
747
+ Result row: `id`, `class`, `rev` (version number), `tenant`, `owner`, `links` (ends), `data` (fields), `tags`, `at`
748
+ (version time), `author`, `op` (kind of operation) and, if present, `agent`, `reason`, `moved`. Service fields start
749
+ with `$`: `$deleted`, `$depth`, `$upsert`, `$action`, `$versions`, `$purged`.
750
+
751
+ ### Sorting and pagination
752
+
753
+ <!-- run: guide -->
754
+ ```ts
755
+ import { cursorOf } from 'letopis'
756
+
757
+ const page1 = await db.Product().sort('data.price', 'desc').limit(2).rows()
758
+ const page2 = await db.Product().sort('data.price', 'desc')
759
+ .after(cursorOf(page1.at(-1)!, 'data.price')).limit(2).rows() // strictly after the last row of the page
760
+ ```
761
+
762
+ `sort(field, 'asc' | 'desc')` sorts by `'at'`, `'rev'`, `'id'` or `'data.<path>'`. `after(cursorOf(row, field))` is
763
+ pagination "after this row"; it requires `sort()`. Such pagination withstands inserts and deletes between pages, but a
764
+ row whose sort field has changed may drop out or repeat. A cursor is an object `{ v, id }` (the field value and the row
765
+ id): it survives JSON, so you can hand it to a client. The field in `cursorOf` must match the `sort` field. The cursor
766
+ value is checked against the type of the `sort` field before the query, and a wrong one is `invalid_query`: a numeric
767
+ field needs a finite number, `rev` an integer, a boolean field `true` or `false`, a text field a string, uuid fields
768
+ and `id` a uuid, time fields and `at` an ISO string with a time zone, a date field `YYYY-MM-DD`, a field without a
769
+ description a string, a number or `true`/`false`. `null` fits only data fields, and the cursor's `id` is always a uuid.
770
+ The check cannot tell apart a cursor of the same type taken from another field. `limit` and `offset` are available
771
+ too.
772
+
773
+ A data field may be `null` or missing. `sort` places such rows the way `order by` in PostgreSQL does: last when
774
+ ascending, first when descending, and `after()` treats `null` as a value after all others (ascending) or before them
775
+ (descending). So pages go through such rows without gaps or repeats, and a cursor with the value `null` gives the next
776
+ page.
777
+
778
+ ### Trees: `deep`
779
+
780
+ <!-- run: guide -->
781
+ ```ts
782
+ const subtree = await db.Category(tea).Category().deep().rows() // subcategories of any depth; $depth is the level
783
+ ```
784
+
785
+ `deep(max)` repeats the hop to the same class up to the depth `max` (32 levels by default). The filter, tags and owner
786
+ of the step select nodes at any level but do not cut the traversal. Only a node invisible by permissions cuts it: such a
787
+ node also hides its subtree.
788
+
789
+ ### Aggregates
790
+
791
+ <!-- run: guide -->
792
+ ```ts
793
+ const total = await db.Order(order).line().sum('data.price') // sum of the order lines
794
+ const avgPrice = await db.Shop(shop).Product().avg('data.price')
795
+ const byStatus = await db.Order().countBy('data.status') // { new: 1 }
796
+ ```
797
+
798
+ The database computes aggregates over the entities of the last step.
799
+
800
+ ### Paths
801
+
802
+ <!-- run: guide -->
803
+ ```ts
804
+ const orderPaths = await db.Order(order).line().Product().paths() // [{ Order: Row, line: Row, Product: Row }, …]
805
+ ```
806
+
807
+ `paths()` returns the nodes of every variant of the path; the key of a node is the step name or the name from
808
+ `.alias(name)`. Repeating a link in the chain returns to its node: this way you can go from one link in several
809
+ directions. `db.entity(row | chain)` inserts a ready node into the path.
810
+
811
+ ## 7. Deleting and restoring
812
+
813
+ ### Preview and deletion
814
+
815
+ <!-- run: guide -->
816
+ ```ts
817
+ const preview = await db.Order(order).delete().rows() // deletes nothing: the order and its lines, $action of each
818
+ await db.Order(order).delete({ confirm: true }).rows() // deleted: tombstones with $deleted: true
819
+
820
+ const blockers = (await db.Shop(shop).delete().rows()).filter((x) => x.$action === 'restrict')
821
+ try {
822
+ await db.Shop(shop).delete({ confirm: true }).rows() // products block the shop: the Shop end is restrict
823
+ } catch (e) {
824
+ if (!(e instanceof LetopisError) || e.code !== 'delete_restricted') throw e
825
+ }
826
+ ```
827
+
828
+ `delete()` without `{ confirm: true }` is only a preview: the whole cascade closure. Every preview row has `$action`
829
+ (`delete`, `unset` or `restrict`) and `$depth`, the cascade level (0 is the target itself; `unset` and `restrict` rows
830
+ do not have it). With the confirmation, all targets of the step are deleted in one statement, and the cascade applies
831
+ the rules of the ends. The preview does not take permissions into account: a row with `$action: 'delete'` may fail to
832
+ be deleted for a user without the permission (`acl_denied`), so a user interface should ask
833
+ `acl.checkData(class, 'DELETE')`. The rules of the ends:
834
+
835
+ - `restrict`: deletion is forbidden while the target is referenced (`delete_restricted` with a list of the visible rows
836
+ and the number of the others);
837
+ - `cascade`: the referencing rows are deleted together with the target (the order lines above);
838
+ - `unset`: the reference is removed.
839
+
840
+ The cascade requires delete permission on every deleted row, and removing a reference requires write permission. A raw
841
+ `DELETE … WHERE id IN (…)` of any set of related rows works the same way.
842
+
843
+ ### Restore: `restore`
844
+
845
+ <!-- run: guide -->
846
+ ```ts
847
+ await db.Order(order).restore().rows() // the order, from its last snapshot, with the same id
848
+ await db.Order(order).line().withDeleted().restore().rows() // and its lines deleted by the cascade
849
+ ```
850
+
851
+ `restore()` brings back the last snapshot of a deleted object as a new version and checks it again against the current
852
+ class description. Only the targets of the step are restored, not the whole cascade. A live object gives `not_deleted`.
853
+
854
+ ### Erase history: `purge`
855
+
856
+ <!-- run: guide -->
857
+ ```ts
858
+ const [temp] = await db.Customer().create({ name: 'Test', email: 'test@example.com' }).rows()
859
+ await db.Customer(temp).delete({ confirm: true }).rows()
860
+ const before = await db.Customer(temp).purge().rows() // preview: $versions is how many versions the journal has
861
+ await db.Customer(temp).purge({ confirm: true }).rows() // history erased; the journal has a purge record
862
+ ```
863
+
864
+ `purge` works only for deleted objects (a live one gives `not_deleted`) and erases their whole history; in place of
865
+ version 1, a `purge` record stays with the number of erased versions. The history of system classes is not erased. To
866
+ delete an account, use `db.auth.purgeAccount` ([section 10](#10-users-sign-in-and-permissions)).
867
+
868
+ ## 8. Transactions, batches and money
869
+
870
+ ### Every operation is a transaction
871
+
872
+ Every chain terminal is its own `read committed` transaction, whatever the default level in the database is. At this
873
+ level, increments and the cascade see fresh rows. The library retries deadlocks, serialization failures and journal
874
+ conflicts by itself. If the connection drops during `COMMIT`, it does not retry and throws `commit_unknown`: the
875
+ transaction may have committed. A repeat of the same operation with the same key finds out the outcome: `exists` means
876
+ "already written".
877
+
878
+ ### Explicit transaction: `db.begin()`
879
+
880
+ <!-- run: guide -->
881
+ ```ts
882
+ const tr = await db.begin()
883
+ try {
884
+ const [o2] = await tr.Customer(olga).Order().create({ number: 'A-2' }).Shop.set(shop).rows()
885
+ await tr.Order(o2).line().create({ qty: 1, price: 2000 }).Product.set(kettle).rows()
886
+ await tr.commit()
887
+ } catch (e) {
888
+ await tr.rollback()
889
+ throw e
890
+ }
891
+ ```
892
+
893
+ - Inside, you have the same chains, `db.sql`, `forUpdate()`, `lock()`, `commit()` and `rollback()`.
894
+ - An error in any query breaks the transaction: every next call and `commit()` throw the original error, and `commit()`
895
+ rolls back. `rollback()` goes through. If the error was caught inside `tr.transaction(fn)` and the library does not
896
+ know about it, PostgreSQL silently answers `ROLLBACK` to `COMMIT`, and the library turns this into the
897
+ `tx_rolled_back` error.
898
+ - There are no retries inside `db.begin()`: on `40P01`, repeat the whole block.
899
+ - `lock(...keys)` is an advisory lock until the end of the transaction. It protects only if everyone who edits this data
900
+ takes it.
901
+
902
+ ### Batch
903
+
904
+ <!-- run: guide -->
905
+ ```ts
906
+ const b = db.batch('storefront')
907
+ b.Shop(shop).Product().create({ sku: 'C-1', title: 'Cup', price: 300 })
908
+ b.Shop(shop).Product().create({ sku: 'C-2', title: 'Saucer', price: 200 })
909
+ const results = await b.run() // Row[][]: an array of rows for each plan, in one transaction
910
+ ```
911
+
912
+ A batch collects plans, and `run()` executes them in one transaction. Consecutive simple `create` and `upsert` calls of
913
+ a class with a key are merged into one multi-row statement. `b.size()` is the number of queued plans, `b.discard()`
914
+ clears the queue. A terminal on a queued plan gives `invalid_query`: only `run()` executes. The name is the key of a
915
+ queue of the facade: the root connection, each `db.as()` and each `db.begin()` have their own queues. A plan runs on
916
+ behalf of the facade that queued it (actor, tenant, owner, transaction), and `run()` never executes plans of other
917
+ facades. Collect plans through one and the same facade object.
918
+
919
+ ### Recipe: wallet
920
+
921
+ The balance is the `balance` field of the `Wallet` hub (integer cents, `min(0)`); operations are the `Movement` link
922
+ with the key `[Wallet, opId]` ([section 4](#4-data-model)). An operation is never edited or deleted; it is cancelled by
923
+ a reverse operation with a new `opId`. The key makes an operation idempotent: a repeat after a lost response gets
924
+ `exists`, and the recipe checks whether it is the same request.
925
+
926
+ <!-- run: guide -->
927
+ ```ts
928
+ import { inc, type Db } from 'letopis'
929
+
930
+ type Kind = 'topup' | 'debit' | 'adjust'
931
+
932
+ /** Operation: write a Movement and increment the balance — one plan, one transaction. */
933
+ async function operate(db: Db, wallet: string, opId: string, kind: 'topup' | 'debit', amount: number) {
934
+ try {
935
+ await db.Wallet(wallet).Movement().create({ opId, kind, amount })
936
+ .Wallet().update({ balance: inc(kind === 'topup' ? amount : -amount) }).rows()
937
+ return 'done'
938
+ } catch (e) {
939
+ if (e instanceof LetopisError && e.code === 'exists') return same(db, wallet, opId, kind, amount) // repeat
940
+ throw e // invalid_data: not enough funds (min(0))
941
+ }
942
+ }
943
+
944
+ /** Repeat fingerprint: is the request already written under this opId the same request? */
945
+ async function same(db: Db, wallet: string, opId: string, kind: Kind, value: number) {
946
+ const m = await db.Movement(db.idOf('Movement', { Wallet: wallet, opId })).first()
947
+ if (!m || m.data.kind !== kind || (kind === 'adjust' ? m.data.target : m.data.amount) !== value)
948
+ throw new Error(`operation key ${opId} is already used by another request`)
949
+ return 'repeat'
950
+ }
951
+
952
+ /** Adjustment "balance = target": an adjust operation for the difference, the total under { rev }. */
953
+ async function adjust(db: Db, wallet: string, opId: string, target: number) {
954
+ for (;;) {
955
+ if (await db.Movement(db.idOf('Movement', { Wallet: wallet, opId })).first()) return same(db, wallet, opId, 'adjust', target)
956
+ const cur = (await db.Wallet(wallet).first())!
957
+ try {
958
+ await db.Wallet(wallet).Movement().create({ opId, kind: 'adjust', amount: target - (cur.data.balance as number), target })
959
+ .Wallet().update({ balance: target }, { rev: cur.rev }).rows()
960
+ return 'done'
961
+ } catch (e) {
962
+ if (e instanceof LetopisError && e.code === 'conflict') continue // the wallet changed: re-read
963
+ if (e instanceof LetopisError && e.code === 'exists') return same(db, wallet, opId, 'adjust', target)
964
+ throw e
965
+ }
966
+ }
967
+ }
968
+
969
+ const [wallet] = await db.Customer(olga).Wallet().create({ number: 'W-1' }).rows()
970
+ await operate(db, wallet.id, 'op-1', 'topup', 10_000) // 'done': balance 10,000
971
+ await operate(db, wallet.id, 'op-1', 'topup', 10_000) // 'repeat': a repeat after a lost response, the balance is the same
972
+ await operate(db, wallet.id, 'op-2', 'debit', 3_000) // balance 7,000
973
+ await adjust(db, wallet.id, 'op-3', 5_000) // adjustment: adjust by −2,000, balance 5,000
974
+ ```
975
+
976
+ The same result can be written under a locking read, without retries on `conflict`:
977
+
978
+ <!-- run: guide -->
979
+ ```ts
980
+ const [wallet2] = await db.Customer(olga).Wallet().create({ number: 'W-2' }).rows()
981
+ const tx = await db.begin()
982
+ try {
983
+ const w = (await tx.Wallet(wallet2).forUpdate().first())! // the wallet row is locked until commit
984
+ await tx.Wallet(wallet2).Movement()
985
+ .create({ opId: 'op-1', kind: 'adjust', amount: 6_000 - (w.data.balance as number), target: 6_000 })
986
+ .Wallet().update({ balance: 6_000 }).rows()
987
+ await tx.commit()
988
+ } catch (e) {
989
+ await tx.rollback()
990
+ throw e
991
+ }
992
+ ```
993
+
994
+ - `inc` adds inside the statement, over the locked row: concurrent debits are not lost, and `min(0)` rejects a debit
995
+ beyond the balance.
996
+ - A plain `update({ balance: 700 })` writes the value; it does not add. A total computed from what was read, without
997
+ `{ rev }` or `forUpdate()`, loses concurrent debits. An advisory `lock()` does not help if not all writers take it.
998
+ `upsert` overwrites increments.
999
+ - `forUpdate()` works only inside `db.begin()` (otherwise `tx_required`) and only with `rows`, `first`, `ids`
1000
+ (otherwise `lock_unsupported`). Rows are locked in id order; a row that no longer matches the conditions of the step
1001
+ is not included in the result.
1002
+ - Choose one path for one wallet: under `{ rev }` and under `forUpdate()`, the wallet and the operation key are locked
1003
+ in a different order, and mixing the two in concurrent requests gives `40P01`. Inside a block with `forUpdate()`, do
1004
+ not wait for external calls: all writers wait for the wallet (`lock_timeout` and
1005
+ `idle_in_transaction_session_timeout` help here).
1006
+ - A transfer is two operations in one `db.begin()`, with the wallets edited in id order: then there are no deadlocks.
1007
+ - Insufficient funds are told apart from other data errors by `e.issues`: the field `balance`, the rule `minimum`.
1008
+
1009
+ ## 9. Bulk loading
1010
+
1011
+ ### When you need the loader
1012
+
1013
+ The loader is the path for import, seeding and syncing large reference tables. It checks rows in the library code
1014
+ with the same validator as the database, and writes them with the `COPY` command as the owner role, without triggers
1015
+ and RLS. In the benchmark, this is about 6,400 rows per second against 130 for row-by-row `create`
1016
+ ([section 17](#17-performance)). It works only on an admin connection (`allowBypassRls` and the right to become the
1017
+ owner), otherwise you get `admin_required`.
1018
+
1019
+ ### `db.load()`
1020
+
1021
+ <!-- run: guide -->
1022
+ ```ts
1023
+ const rep = await adm.load({
1024
+ Shop: [{ name: 'Cornflower', city: 'Perm' }],
1025
+ Product: [
1026
+ { sku: 'V-1', title: 'Mug', price: 450, Shop: { $key: ['Cornflower'] } },
1027
+ { sku: 'V-2', title: 'Spoon', price: 150, Shop: { $key: ['Cornflower'] }, $tags: ['new'] },
1028
+ ],
1029
+ Customer: [{ name: 'Petr', email: 'petr@mail.ru', $ext: 'crm:1001' }],
1030
+ })
1031
+ // rep: { rows: 4, loaded: 4, updated: 0, skipped: 0, duplicates: 0, byClass: { Shop: 1, Product: 2, Customer: 1 }, … }
1032
+ ```
1033
+
1034
+ - The input is an object "class → rows", an array, or a stream of rows with a `$class` field.
1035
+ - A row has its fields and end roles at the top level. Service keys: `$class` is the class of the row, `$ext` is an
1036
+ external key of a class without a key (loading the same file again gives the same ids), `$tags` are the tags,
1037
+ `$owner` is the owner.
1038
+ - A reference is a target id, a key `{ $key: [...] }` or an external key `{ $ext: '…' }`; for a polymorphic end, add
1039
+ the `$class` of the target. The order of rows and classes does not matter, and circular references are allowed.
1040
+ - Keys that start with `$` inside the data are ordinary data.
1041
+ - Rows go to the session tenant (or to `tenant`). A load is one transaction: an error in any row cancels everything.
1042
+ Subscriptions see the rows after the commit.
1043
+
1044
+ ### Repeats and existing objects
1045
+
1046
+ <!-- run: guide -->
1047
+ ```ts
1048
+ const again = await adm.load({ Shop: [{ name: 'Cornflower', city: 'Perm' }] }, { existing: 'skip' }) // skipped: 1
1049
+ const changed = await adm.load({ Shop: [{ name: 'Cornflower', city: 'Samara' }] }, { existing: 'update' }) // updated: 1
1050
+ ```
1051
+
1052
+ | Option | Values |
1053
+ |---|---|
1054
+ | `duplicates`: a repeated id within the load | `'error'` (default): cancel; `'first'` or `'last'`: keep the first or the last row |
1055
+ | `existing`: the object already exists in the database | `'error'` (default): cancel; `'skip'`: skip, do not resurrect deleted objects; `'update'`: a new version if the content differs (unchanged objects are counted in `skipped`), deleted objects are created again |
1056
+
1057
+ ### Portions, history and verification
1058
+
1059
+ - `commitEvery: N` commits in portions of about N rows. Portions go by class in dependency order; the classes of one
1060
+ cycle go in one portion. An error cancels only the current portion, and a repeat with `existing: 'skip'` writes the
1061
+ rest.
1062
+ - `withHistory: true`: the rows are versions with `$rev`, `$at` and `$op`; this is how the import carries history over.
1063
+ - `verify: true`: at the end, check the loaded rows again with the database check.
1064
+ - `batch` (rows per `COPY` batch, 10,000), `class` (the class of rows without `$class`), `tenant`, `agent` (the agent of
1065
+ the versions).
1066
+
1067
+ ### Loading on behalf of a user
1068
+
1069
+ <!-- run: guide -->
1070
+ ```ts
1071
+ await adm.load({ Category: [{ name: 'Tableware' }] }, { as: ANNA, tenant: SHOP })
1072
+ ```
1073
+
1074
+ With `as`, the user becomes the author of the versions, and the loader checks the same things as the database does on a
1075
+ regular write: the tenant is available to the user, write permission on each class, the permission conditions on rows,
1076
+ the visibility of reference targets, the owner per `ownerDefault`. Without `tenant`, the rows go to the user's own
1077
+ tenant. This is how a loading process can accept files sent by users.
1078
+
1079
+ ### Command line
1080
+
1081
+ ```sh
1082
+ npx letopis load shops.ndjson products.ndjson --dsn "$ADMIN" --schema v2.shop --token "$TOKEN" \
1083
+ [--class Product] [--duplicates first] [--existing skip] [--commit-every 100000] [--verify]
1084
+ # shops.ndjson: {"$class":"Shop","name":"South"}
1085
+ # products.ndjson: {"$class":"Product","sku":"Y-1","title":"Vase","price":800,"Shop":{"$key":["South"]}}
1086
+ ```
1087
+
1088
+ NDJSON files have one object per line; all files form one load.
1089
+
1090
+ ### Report and errors
1091
+
1092
+ Report: `rows`, `loaded`, `updated`, `skipped`, `duplicates`, `byClass`, `external` (how many targets outside the load
1093
+ were checked by a query at the end), `portions`, `ms`. An error is a `LetopisError` with the code of the first
1094
+ problem. Its `detail`: `rows` is the first 100 violations with the row number, code and details (`line`, `code`,
1095
+ `message`, `id`, `class`, `issues`), `total` is how many violations were found, `truncated` means the list is cut,
1096
+ `limit` is the list limit (100), `complete` means the checks that found violations went through all rows. Row codes:
1097
+ the codes of the database checks, and also `duplicate` (a repeated id), `exists` (the object already exists), `deleted`
1098
+ (an object with this id existed before) and `class_not_loadable` (the loader does not accept `Class` rows: classes are
1099
+ set by `syncModels`).
1100
+
1101
+ Violations are searched for in stages: parsing the rows (classes, ids, repeats), permissions on classes
1102
+ (`load({ as })`), data and ends in batches, and at the end of a portion the targets outside the load, owners and the
1103
+ tenant. The load is cancelled after the first stage that found violations, and `total` counts the violations of that
1104
+ stage. A target outside the load is one violation per target (the row is the first one that references it).
1105
+ "(и ещё N)" ("and N more") in the error text is the exact number of the remaining violations. If not all rows were
1106
+ checked, `complete` is `false`, and the text says "(и ещё не меньше N: причины)" ("at least N more: reasons") or, if
1107
+ one violation was found, "(возможно, не единственная: причины)" ("possibly not the only one: reasons"). The reasons:
1108
+ the following `commitEvery` portions were not checked; the ACL conditions of the following batches were not checked
1109
+ (`load({ as })`); the database check (`verify: true`) names at most 100 rows.
1110
+
1111
+ ## 10. Users, sign-in and permissions
1112
+
1113
+ ### Accounts, tenants and memberships
1114
+
1115
+ An account is a row of class `Account` in System: a name, categories (`User`, `Service` and your own) and an `enabled`
1116
+ flag. Every account is its own tenant. A `member` membership allows work in another tenant: it is a link from a user
1117
+ (`User`) to a tenant (`Tenant`) with roles in its data. Accounts and their categories are edited by the system
1118
+ administrator, a member of System with the `admin` role. There is no separate method to appoint one: a membership in
1119
+ System with the `admin` role is put in place by your own installation seed or by the loader on an admin connection (a
1120
+ `member` row with `tenant: <System id>`). New users of a tenant are easy to create with `createTenant`
1121
+ ([section 3](#3-installation-and-connection)).
1122
+
1123
+ ### Service and API keys
1124
+
1125
+ A service is an account of category `Service` with an API key. `up()` issues the installation key. The service checks
1126
+ passwords and codes, issues sessions and acts on behalf of users. It issues and revokes the keys of regular users
1127
+ itself (the `auth.credential` permission), as for Boris below. The service also issues and revokes its own API keys
1128
+ and "key + secret" pairs itself, with a session opened by its own key (`connect({ apiKey })`, `verifyApiKey`,
1129
+ `verifyKeySecret`). Its other credentials and the keys of other services, of System and of system administrators are
1130
+ issued and revoked only by a system administrator; a new service key is also registered by `up({ serviceKey })`.
1131
+
1132
+ There can be several keys. A session opened by a key remembers it, and revoking a key revokes only its sessions:
1133
+ connections by other keys keep working. So rotation goes without downtime: the service issues itself a new key
1134
+ (`issueApiKey({ account: <service id> })`), moves its processes to it and revokes the old one (`revokeCredential(id)`).
1135
+ The revocation keeps the current session; `keepCurrent: false` revokes it too. The last active key of a service
1136
+ cannot be revoked: `acl_denied` "последний действующий ключ сервиса не отзывается: сначала выпустите новый" ("the
1137
+ last active key of a service is not revoked: issue a new one first"). To disable a service, set `enabled: false` on
1138
+ its account.
1139
+
1140
+ <!-- run: guide -->
1141
+ ```ts
1142
+ const { key, id: keyId } = await svc.auth.issueApiKey({ account: BORIS, name: 'integration' }) // the key is returned only once
1143
+ const viaKey = await svc.auth.verifyApiKey(key) // { account, token } | null
1144
+ await svc.auth.revokeCredential(keyId)
1145
+ ```
1146
+
1147
+ A "key + secret" pair: `issueKeySecret({ account, name })` and `verifyKeySecret(key, secret)`.
1148
+
1149
+ ### Passwords
1150
+
1151
+ <!-- run: guide -->
1152
+ ```ts
1153
+ await svc.auth.setPassword({ account: BORIS, identifier: 'boris@lutik.ru', password: 'New-password-2026', confirmed: true })
1154
+ const failed = await svc.auth.verifyPassword({ identifier: 'boris@lutik.ru', password: 'Buttercup-2026!' }) // null: the password was changed
1155
+ ```
1156
+
1157
+ - A password is stored as a scrypt hash; the service checks it, and the response time for a nonexistent login is
1158
+ equalized.
1159
+ - The identifier is compared case-insensitively: `Anna@Shop.ru` and `anna@shop.ru` are one login.
1160
+ - A password change revokes the other sessions of the account (`keepCurrent` keeps the current one).
1161
+ - By default, sign-in works only by a confirmed credential, otherwise `null`; `requireConfirmed: false` also lets in by
1162
+ an unconfirmed one. The service confirms a credential: `confirmed: true` in `setPassword` or `setFlags`.
1163
+ - The second factor is the `totp` option: `verifyPassword({ identifier, password, totp })` checks the account's TOTP
1164
+ code after a correct password and issues a session with the methods `[kind, 'TOTP']`, `['PASSWORD', 'TOTP']` by
1165
+ default (see below). Without `totp`, sign-in does not check the code: the service decides whether to require the
1166
+ second factor.
1167
+
1168
+ ### One-time codes and TOTP
1169
+
1170
+ <!-- run: guide -->
1171
+ ```ts
1172
+ import { totpCode } from 'letopis'
1173
+
1174
+ const { code } = await svc.auth.issueOtp({ account: BORIS, identifier: 'boris@lutik.ru', ttl: 300 }) // the service sends the code by email
1175
+ const byCode = await svc.auth.verifyOtp({ identifier: 'boris@lutik.ru', code }) // { account, token } | null
1176
+ const { secret, uri } = await svc.auth.enrollTotp({ account: BORIS, issuer: 'Buttercup' }) // uri is for a QR code
1177
+ const totpOk = await svc.auth.verifyTotp({ account: BORIS, code: totpCode(secret) }) // the first check enables the factor
1178
+ ```
1179
+
1180
+ A code is single-use: a successful check and brute force (5 attempts by default) burn it. The attempt counters are
1181
+ rolled back together with the caller's transaction, so the service limits the rate of attempts. A repeat of the same
1182
+ TOTP step is rejected. `totpCode(secret)` is a helper for tests and services.
1183
+
1184
+ Sign-in with the second factor is `verifyPassword({ …, totp })` and `lookup({ …, totp })`. The code is checked only
1185
+ after a correct password or a found identity, by the same check as `verifyTotp`: the `auth.totp` permission is needed,
1186
+ the window is one step each way, and the failure counter is shared (five in a row close the check for 5 minutes). A
1187
+ correct code gives a session with the methods `[kind, 'TOTP']` and enables a not yet confirmed factor; a wrong or
1188
+ repeated code, like a code for an account without TOTP, gives `null`. With such a session, a user with TOTP changes
1189
+ their own credentials for 10 minutes.
1190
+
1191
+ ### External identities
1192
+
1193
+ `link({ account, kind, identifier, confirmed })` links an external identity (Telegram, Google, SSO): the service checks
1194
+ its token, and the database stores the binding. `lookup({ kind, identifier })` signs in with it (with the `totp`
1195
+ option, also by a TOTP code, like `verifyPassword`):
1196
+
1197
+ <!-- run: guide -->
1198
+ ```ts
1199
+ await svc.auth.link({ account: BORIS, kind: 'TELEGRAM', identifier: '100500', confirmed: true })
1200
+ const viaTelegram = await svc.auth.lookup({ kind: 'TELEGRAM', identifier: '100500' }) // { account, token } | null
1201
+ ```
1202
+
1203
+ A user can link their own credentials only after a recent sign-in (10 minutes; an API key, a "key + secret" pair and a
1204
+ session from a service do not count), and cannot confirm them. With TOTP enabled, the database requires a session that
1205
+ has passed TOTP: `verifyPassword` and `lookup` with the `totp` option issue one. If the identifier is taken, linking
1206
+ answers with the same `acl_denied` as without a recent sign-in. But a user with a recent sign-in can tell from such a
1207
+ denial that the identifier is taken ([section 18](#18-limitations)).
1208
+
1209
+ ### Sessions
1210
+
1211
+ <!-- run: guide -->
1212
+ ```ts
1213
+ const shortToken = await svc.auth.sessionFor(BORIS, { ttl: 600 }) // a session for 10 minutes of idle time for another process
1214
+ const boris = await connect({ dsn: APP, schema: r.schema, token: shortToken })
1215
+ await boris.auth.revoke() // sign-out: the session is revoked
1216
+ await boris.close()
1217
+ ```
1218
+
1219
+ A token is 256 random bits; the database stores only its sha256. The default lifetime is 7 days. `refresh()` extends
1220
+ the current session: the new lifetime is counted from the current moment, but it is not longer than the lifetime issued
1221
+ at sign-in; it returns the new expiration time. `connect()` extends the session itself once per minute of work
1222
+ (`refreshSession`), so a session in use does not expire: the lifetime is the idle time. `revoke()` signs out,
1223
+ `revokeAll()` signs out everywhere. Sessions are revoked when the account is disabled, the password is changed, a credential is deleted or the
1224
+ account is deleted. Deleting a credential revokes the other sessions of the account, while revoking an API key or a
1225
+ "key + secret" pair revokes only the sessions opened by that key (and old sessions of the same sign-in method without
1226
+ a key mark); the current session stays unless you pass `keepCurrent: false`.
1227
+
1228
+ ### Working in another tenant
1229
+
1230
+ A service acts on behalf of a user through `svc.as(account, { tenant, owner })`. The database checks the service's
1231
+ `auth.impersonate` permission (by System rules), that the user exists and is enabled, and that the tenant is available
1232
+ to the user, either their own or through membership. You cannot act on behalf of System, services, system
1233
+ administrators or disabled accounts. The user's visibility under the access rules is not checked. In the journal,
1234
+ `author` is the user and `agent` is the service. A user's connection by their own token switches between tenants like
1235
+ this:
1236
+
1237
+ <!-- run: guide -->
1238
+ ```ts
1239
+ const mine = await anna.auth.tenants() // [{ tenant, roles }]: her own tenant and the shop via membership
1240
+ await anna.auth.switch(SHOP) // the session tenant is now the shop
1241
+ ```
1242
+
1243
+ ### Permissions
1244
+
1245
+ Permissions are always on and deny by default. A rule connects a group with an object:
1246
+
1247
+ - **group**: a `Resource` row of category `ACCOUNT`: accounts by category (`{ categories: '{User}' }`) or members by
1248
+ membership role (`{ roles: '{staff}' }`);
1249
+ - **object**: a `Resource` row of category `API` (an endpoint by mask, `{ endpoint: 'shop.reports.*' }`), or of
1250
+ category `READ`, `WRITE`, `DELETE` (the rows of a class by mask, with descendants, `{ class: 'Product' }`, and a
1251
+ condition on the columns `id`, `owner`, `tenant`, `tags`, `data`, `links` with the values `$actor` and `$tenant`);
1252
+ - **rule**: a `rule` link group → object: `permission` (`allow` or `deny`) and `weight`.
1253
+
1254
+ Exactly one rule decides: the highest weight; at equal weight, deny beats allow. A tenant rule weight above 999 is
1255
+ cut down to 999, so System rules with a weight of 1000 or more are an administrative layer that a tenant cannot
1256
+ outweigh. Tenant rules apply to the tenant's rows, System rules apply everywhere. Permissions apply to raw SQL too: rows
1257
+ without permission are invisible, and an edit changes nothing or fails with `acl_denied`.
1258
+
1259
+ ### Granting a permission
1260
+
1261
+ Example: Oleg is a shop employee with the `staff` role, and employees may not delete products.
1262
+
1263
+ <!-- run: guide -->
1264
+ ```ts
1265
+ const pg2 = postgres(ADMIN)
1266
+ const OLEG = (await createTenant(pg2, r.schema, { // the same shop: only the user is created
1267
+ name: 'Chamomile', user: { name: 'Oleg', login: 'oleg@shop.ru', password: 'Oleg-2026!' } })).user!
1268
+ await pg2.end()
1269
+ await db.member(db.idOf('member', { User: OLEG, Tenant: SHOP })).update({ roles: ['staff'] }).rows()
1270
+
1271
+ const [staff] = await db.Resource().create({ alias: 'staff:ACCOUNT', category: 'ACCOUNT', pattern: { roles: '{staff}' } }).rows()
1272
+ const [delProduct] = await db.Resource().create({ alias: 'product:DELETE', category: 'DELETE', pattern: { class: 'Product' } }).rows()
1273
+ await db.rule().create({ permission: 'deny', weight: 50 }).Group.set(staff).Object.set(delProduct).rows() // stronger than "may do everything" (10)
1274
+ const [reports] = await db.Resource().create({ alias: 'reports:API', category: 'API', pattern: { endpoint: 'shop.reports.*' } }).rows()
1275
+ await db.rule().create({ permission: 'allow', weight: 50 }).Group.set(staff).Object.set(reports).rows()
1276
+
1277
+ const asOleg = await svc.as(OLEG, { tenant: SHOP })
1278
+ const canDelete = await asOleg.acl.checkData('Product', 'DELETE') // { allow: false }
1279
+ const canRead = await asOleg.acl.checkData('Product', 'READ') // { allow: true }
1280
+ const canReport = await asOleg.acl.check('shop.reports.daily') // { allow: true, rule: { weight: 50, … } }
1281
+ ```
1282
+
1283
+ - `acl.check(endpoint)` → `{ allow, rule, code, message }`: the `code` and `message` fields of the denying rule set the
1284
+ endpoint's own denial.
1285
+ - `acl.checkData(class, 'READ' | 'WRITE' | 'DELETE')` → `{ allow, filter }`: `filter` is the condition on rows.
1286
+ - The database recomputes permissions by itself. To remove a permission, delete the rule:
1287
+ `db.rule(id).delete({ confirm: true })`.
1288
+ - The library takes the actor's signed permission plan — the permission pass — from the database once and passes it
1289
+ in every transaction, so a permission check costs tens of microseconds. Raw SQL without the pass pays 0.3–1.3 ms per
1290
+ query.
1291
+ - The question "who had access at a given date" is answered by an `asOf` read over the classes `rule`, `member` and
1292
+ `Resource`.
1293
+
1294
+ Fields of the system permission classes:
1295
+
1296
+ - `member`: the ends `User` and `Tenant`, the field `roles` (a list of roles);
1297
+ - `Resource`: `alias` (the unique name of the resource and its key; this is not a class alias), `category` (`ACCOUNT`,
1298
+ `API`, `READ`, `WRITE`, `DELETE`) and `pattern`, the mask. A group's mask is `categories` or `roles`, and only a
1299
+ string: `'{staff}'`, `'{a,b}'`, negation `'!{a}'`. An object's mask is `endpoint` or `class` (also strings only) plus
1300
+ conditions on the columns. **The database rejects a non-string mask (for example, the array `['staff']`, a number or
1301
+ `null`)** with `invalid_data`, whether written by the library, raw SQL or the loader. Old rows with such a mask
1302
+ (written before this check or around it) grant no permissions, while a denial by them applies to everyone: by a
1303
+ group, to all accounts, by a `class` mask, to all classes (a non-string `endpoint` mask, as before, matches no
1304
+ endpoint). The write check comes with the description of the `Resource` class: it arrives with a fresh installation
1305
+ and after `reset({ all })`; in an existing schema, the corrected matching works after `up()` is run again;
1306
+ - `rule`: the ends `Group` and `Object` (`Resource` rows), the fields `permission` (`allow` or `deny`), `weight`, and,
1307
+ for the endpoint's own denial, `code` and `message`.
1308
+
1309
+ ### Deleting an account
1310
+
1311
+ `db.auth.purgeAccount(account)` deletes an account and the data of its tenant. The system administrator or the owner of
1312
+ its tenant can call it; you cannot delete yourself or the last owner of a tenant. Ownership of rows in other tenants
1313
+ passes to those tenants, sessions are revoked, credentials are deleted.
1314
+
1315
+ ## 11. History and point-in-time reads
1316
+
1317
+ ### Versions
1318
+
1319
+ <!-- run: guide -->
1320
+ ```ts
1321
+ const versions = await db.Product(kettle).versions() // all versions: op, rev, author, at, data
1322
+ const followed = await db.Product(mug2).versions({ follow: true }) // also across id changes (rekey, reclass)
1323
+ ```
1324
+
1325
+ Every version has an author, an agent, a time, a number, a kind of operation (`create`, `update`, `delete`, `reclass`,
1326
+ `rekey`, `restore`, `purge`, `trim`, `reset`) and a reason (`reason`), if the `letopis.reason` setting of the
1327
+ transaction set one; there is no separate API for the reason. A deletion version has `$deleted`.
1328
+
1329
+ ### Point-in-time read
1330
+
1331
+ <!-- run: guide -->
1332
+ ```ts
1333
+ const createdAt = versions[0].at
1334
+ const atCreation = await db.Product(kettle).asOf(createdAt).first() // the price at creation time: 2500
1335
+ const assortment = await db.Shop(shop).Product().asOf(createdAt).rows()
1336
+ ```
1337
+
1338
+ `asOf(moment)` is the state "as it was", from the journal; the moment is an ISO string or a `Date`. A point-in-time
1339
+ read works at any step of a chain, including with `deep()`. Only the current state is cheap: a point-in-time read is
1340
+ rebuilt from the journal. The version time `at` stores microseconds, while a `Date` holds only milliseconds: pass the
1341
+ `at` string to `asOf` as is, otherwise a read at the moment of a version may miss it.
1342
+
1343
+ ### Deleted rows
1344
+
1345
+ <!-- run: guide -->
1346
+ ```ts
1347
+ const withGone = await db.Customer().withDeleted().rows() // live and deleted; deleted rows have $deleted: true
1348
+ ```
1349
+
1350
+ ### Retention policy
1351
+
1352
+ Without a policy, the journal keeps every version. A policy sets how long to keep old versions and with what
1353
+ granularity:
1354
+
1355
+ ```ts
1356
+ history: { all: '1 day', daily: '1 week', weekly: '1 month', monthly: '1 year', yearly: 'forever', tz: 'Europe/Moscow' }
1357
+ ```
1358
+
1359
+ This means: all versions for the last day; for the last week, one per day; for the last month, one per week; for the
1360
+ last year, one per month; older, one per year, forever. Each period keeps its last version. A week starts on Monday;
1361
+ period boundaries are in the `tz` time zone (UTC by default).
1362
+
1363
+ - A policy is set on a class (descendants inherit it) or for all classes without their own policy: `up({ history })`.
1364
+ - If the last tier has a term instead of `'forever'`, the history older than the term is deleted: `{ all: '30 days' }`.
1365
+ - While the object's history is alive, the current version, version 1 and versions of special kinds (`create`,
1366
+ `delete`, `reclass`, `rekey`, `restore`, `purge`, `reset`, `trim`) are not deleted. If the tombstone of a deleted
1367
+ object is older than the last tier with a term, its history is erased entirely. The history of system classes is not
1368
+ thinned.
1369
+ - `maintain()` does the thinning in batches of 10,000 versions ([section 13](#13-operations)).
1370
+ - In the thinned part, a point-in-time read shows the last kept version before the moment, so its precision equals the
1371
+ period of the tier. A subscriber that lags behind by more than the `all` tier gets `cursor_expired`.
1372
+
1373
+ ### Trimming history
1374
+
1375
+ <!-- run: guide -->
1376
+ ```ts
1377
+ const trimmed = await db.trimHistory(kettle, new Date()) // erase the versions of the kettle before this moment
1378
+ ```
1379
+
1380
+ `db.trimHistory(target, moment)` erases the history before the moment right away. The target is a class with its
1381
+ descendants (by name), an id, a list of ids or rows. This is the middle step of the "right to be forgotten":
1382
+ `anonymize` → history trimming → deletion and `purge`. In place of version 1, a `trim` record stays with its time, so a
1383
+ point-in-time read knows when the object appeared.
1384
+
1385
+ ### Integrity check
1386
+
1387
+ <!-- run: guide -->
1388
+ ```ts
1389
+ const audit = await adm.verify({ data: true }) // { ok, issues, anchor }
1390
+ ```
1391
+
1392
+ `verify()` recomputes the hash chains of all versions and returns an anchor: a hash of all chain heads. Keep the anchor
1393
+ outside the database: a mismatch with it exposes tampering with the latest versions. `{ data: true }` also checks again
1394
+ the data, ends, tenants, owners and ids of all rows. The system administrator or an admin connection can call it; from
1395
+ the command line, use `letopis verify [--data]` (exit code 1 if something is found). Legitimate gaps (`trim` and
1396
+ `purge` records, thinning and `reset`) are not counted as errors.
1397
+
1398
+ ## 12. Change subscription and cache
1399
+
1400
+ ### Subscription
1401
+
1402
+ <!-- run: guide -->
1403
+ ```ts
1404
+ const sub = db.watch({ from: 'start', classes: ['Product'] }) // the whole journal of the class, then new events
1405
+ await db.Product(kettle).update({ price: 2100 }).rows()
1406
+ let cursor = ''
1407
+ for await (const ev of sub) { // ev: { cursor, row, id, class, op, rev, at }
1408
+ cursor = ev.cursor // save the cursor: watch({ from: cursor }) continues after it
1409
+ if (ev.id === kettle.id && ev.row.data.price === 2100) break // break (or sub.close()) stops the subscription
1410
+ }
1411
+ ```
1412
+
1413
+ - `from` is `'now'` (the default: only changes after subscribing), `'start'` (the whole journal) or a saved cursor.
1414
+ `db.watch()` returns at once, and the subscription takes the "now" moment with its first query to the database: a
1415
+ write made right after `db.watch()` may commit earlier and miss the stream. That is why the example above reads from
1416
+ `'start'`.
1417
+ - `classes`: only these classes with their descendants; `pollMs`: fallback polling (1000 ms).
1418
+ - Events come without losses or repeats, under the reader's permissions, in the order of transaction numbers, and
1419
+ within a transaction in the order of writing. This is not the commit order: compare versions of one object by `rev`.
1420
+ - A long write transaction in any database of the cluster delays the stream: `await sub.lag()` shows the lag in ms.
1421
+ - A cursor older than the `all` tier of the retention policy gives `cursor_expired`: read the current state again and
1422
+ subscribe with `'now'`.
1423
+ - Without `LISTEN` (`connect({ listen: false })`), the subscription polls the journal.
1424
+ - A subscription without losses is a saved cursor: store `ev.cursor` of the last processed event and resume from it
1425
+ after a restart.
1426
+
1427
+ ### Result cache
1428
+
1429
+ <!-- run: guide -->
1430
+ ```ts
1431
+ const cached = await db.Product().cache({ ttl: 30 }).rows() // the result is kept in process memory for 30 s
1432
+ const stats = db.cache.stats() // { size, hits, misses, stores, raced, … }
1433
+ ```
1434
+
1435
+ The cache is enabled only explicitly: `.cache()` in a chain or `cache: { ttl }` on a class in the model. This is how it
1436
+ stays correct:
1437
+
1438
+ - the result key includes the query, the parameters, the session, the actor and the tenant, so one user never gets the
1439
+ result of another user;
1440
+ - your own writes clear the results of the changed classes right after commit (including the cascade classes); other
1441
+ writers' changes clear them on a database signal;
1442
+ - changes of permissions and classes, writes through `db.sql`, a revoked session and a listener reconnect clear the
1443
+ whole cache or the results of the session;
1444
+ - the default term is 60 seconds; the cache does not work inside `db.begin()`;
1445
+ - without `LISTEN` the cache is off, and `connect({ cache: { ttlOnly: true } })` enables it by term only.
1446
+
1447
+ Between processes, the cache goes stale for the signal sending interval (`notifyIntervalMs`, 100 ms) plus delivery;
1448
+ `db.flushSignals()` sends the accumulated signals at once. `db.cache.clear()` clears the cache.
1449
+
1450
+ ## 13. Operations
1451
+
1452
+ ### Maintenance: `maintain()`
1453
+
1454
+ <!-- run: guide -->
1455
+ ```ts
1456
+ const report = await adm.maintain() // { sessions, codes, thinned, batches, more, queue, skipped }
1457
+ ```
1458
+
1459
+ `maintain()` deletes expired sessions and one-time codes, thins the journal by the retention policy and checks the
1460
+ PostgreSQL signal queue: if it is more than half full, you get a warning. The system administrator, a role with the
1461
+ `letopis.maintain` permission or an admin connection can call it. An external scheduler runs it (`letopis maintain`),
1462
+ or the library itself does: `connect({ maintain: '1h' })`. Of several processes, only one works: each batch is its own
1463
+ transaction, and the first thing it does is take a transaction-level advisory lock (`pg_try_advisory_xact_lock`),
1464
+ which is released when the batch commits. If another process holds the lock, the call answers `skipped: true` before
1465
+ the first batch, and between batches it stops (that process continues) and reports what it has done with
1466
+ `more: true`. There is no session-level lock, so `maintain()` works behind any pool, including PgBouncer in
1467
+ transaction mode.
1468
+
1469
+ ### Reset: `reset()`
1470
+
1471
+ ```ts
1472
+ await adm.reset({ level: 'tenant', confirm: 'v2.shop', tenant: SHOP }) // sessions | tenant | all
1473
+ ```
1474
+
1475
+ Reset is meant for tests and test environments. It works only if the installation was set up with
1476
+ `up({ allowReset: true })` (otherwise `reset_disabled`, even for System), the caller has the `letopis.reset`
1477
+ permission, and `confirm` matches the schema name (`confirm_mismatch`). Levels: `sessions` resets all sessions
1478
+ except the current one; `tenant` the tenant's data; `all` everything, with the system seed laid down again and a new
1479
+ service key. The first journal record after a reset is `reset`.
1480
+
1481
+ ### Model description for people and AI
1482
+
1483
+ <!-- run: guide -->
1484
+ ```ts
1485
+ import { describeText } from 'letopis'
1486
+
1487
+ const model = db.describe() // { schema, tenant, classes: [...], methods }
1488
+ const modelText = describeText(model) // the same as text: for the application's llms.txt or a prompt for an AI agent
1489
+ ```
1490
+
1491
+ `db.describe()` returns a description of the classes available to the session: fields, ends, keys, policies and
1492
+ metadata. From the command line: `letopis describe --format text --out llms.txt`.
1493
+
1494
+ ### Raw SQL
1495
+
1496
+ <!-- run: guide -->
1497
+ ```ts
1498
+ const [{ n }] = await db.sql`select count(*)::int as n from "v2.shop".entity where class = 'Product'`
1499
+ ```
1500
+
1501
+ `` db.sql`…` `` runs a query in one transaction under the session of the connection and under RLS: it sees the same
1502
+ rows as chains do. `db.transaction(fn)` gives a postgres.js connection for the whole transaction. On a deadlock or a
1503
+ serialization failure, the query and `fn` are retried up to three times, so `fn` must withstand a retry. You can also
1504
+ write with raw SQL: the same triggers check the data, and the database functions `merge` and `inc` are available to the
1505
+ application.
1506
+
1507
+ ### Command line
1508
+
1509
+ ```sh
1510
+ letopis sync <module> --dsn <dsn> --schema <v2.name> (--token <token> | --api-key <key>) [--apply]
1511
+ letopis types --dsn <dsn> --schema <v2.name> (--token <token> | --api-key <key>) [--out <file>]
1512
+ letopis load <files…> --dsn <dsn> --schema <v2.name> (--token <token> | --api-key <key>) [--class <Class>]
1513
+ [--duplicates error|first|last] [--existing error|skip|update] [--commit-every <N>] [--verify]
1514
+ letopis verify --dsn <dsn> --schema <v2.name> [--token <token> | --api-key <key>] [--data]
1515
+ letopis maintain --dsn <dsn> --schema <v2.name> [--token <token> | --api-key <key>] [--batches <N>]
1516
+ letopis describe --dsn <dsn> --schema <v2.name> (--token <token> | --api-key <key>) [--format json|text] [--out <file>]
1517
+ letopis import --from <0.21 dsn> --from-schema <v1.name> --dsn <dsn> --schema <v2.name> [--tenant <name>] [--keyless <Class>]…
1518
+ [--rename <Old>=<New>]… [--update] [--commit-every <N>] [--partition <name>]
1519
+ ```
1520
+
1521
+ ### Monitoring
1522
+
1523
+ - `connect({ onQuery, slowMs })`: for every chain query, the hook gets the mode, the classes, the duration, the row
1524
+ count and a slow query flag; without a hook, a slow query prints a warning.
1525
+ - `sub.lag()` is the lag of a subscription; the `maintain()` report shows how full the signal queue is.
1526
+ - Hot reference targets (an organization that almost everything references) accumulate `for key share` locks and
1527
+ MultiXact, as with regular foreign keys. This is worth monitoring.
1528
+
1529
+ ### Connection pools and PgBouncer
1530
+
1531
+ With PgBouncer in transaction mode, `LISTEN` is not available: use `connect({ listen: false })`. Then the registry is
1532
+ re-read with `db.refresh()`, the subscription polls the journal, and the result cache is off or works by term only.
1533
+ Prepared statements work with PgBouncer 1.21 and newer; with older versions, pass your own postgres.js pool with
1534
+ `prepare: false` in `connect({ sql })`. `maintain()` takes a transaction-level lock in each batch and works behind
1535
+ such a pool.
1536
+
1537
+ <!-- run: guide -->
1538
+ ```ts
1539
+ await adm.close()
1540
+ await anna.close()
1541
+ await svc.close() // and the connections obtained through svc.as()
1542
+ ```
1543
+
1544
+ ## 14. Migrating from 0.21
1545
+
1546
+ 1.0 is a new implementation, not a continuation of 0.21: a 0.21 schema is not upgraded in place. 1.0 is installed into
1547
+ a new schema, the `letopis import` command moves the data, and the code is ported with the replacement table in
1548
+ [MIGRATION.md](MIGRATION.md) (in Russian). The main differences in behavior: `create` does not overwrite (a taken key
1549
+ gives `exists`), `count()` counts entities, paths are `paths()`, permissions and tenant isolation are always on.
1550
+
1551
+ ```sh
1552
+ npx letopis import --from postgres://…/old --from-schema v1.salon --dsn postgres://…/new --schema v2.salon \
1553
+ --tenant salon [--keyless booking] [--rename link=link_] [--update] [--commit-every 100000]
1554
+ ```
1555
+
1556
+ - The 1.0 schema is installed in advance (`up()`); both connections are admin connections. The source is the 0.21
1557
+ tables (`Schema`, `Entity`, `Account`, `Credential`, `Resource`, `Rule`) in any database: PostgreSQL 17 with
1558
+ TimescaleDB or a dump of plain tables. The loader (history mode, `existing: 'skip'`, agent `letopis import`) and the
1559
+ owner functions do the writing.
1560
+ - Classes become `Class` rows in System. fastest-validator rules are translated to JSON Schema (`optional` lets `null`
1561
+ through, `$$strict` → `additionalProperties: false`, `record` → `propertyNames`). Ends become roles named after the
1562
+ target class; a union of all descendants of a class becomes a polymorphic end. The `Entity` and `link` roots are
1563
+ dropped; `description`, `appearance` and `order` move to `meta`. For names taken by 1.0 methods, use `--rename` (by
1564
+ default, the name with `_`).
1565
+ - Ids: for a class with a key (the v5 rule from 0.21), v5 of the key; for a class without a key, v5 of the old id.
1566
+ System data moves to the `--tenant` tenant, accounts to `Account` rows, credentials to `secret` as they are (scrypt,
1567
+ keys); one-time codes are not carried over. Resources and rules go to System.
1568
+ - The import stops before the first write if two 0.21 objects give one 1.0 id (the data violates the key; use
1569
+ `--keyless <Class>`), if a key changed between versions, or if it references a missing object. A dangling reference
1570
+ in a live row gives `target_not_found` with the 0.21 object.
1571
+ - If the loader rejects the data, the error starts with `import: арендатор …`, and its `detail` is the loader's
1572
+ ([Report and errors](#report-and-errors)), except that rows are named by 0.21 objects: `rows` are the violations
1573
+ with the `class`, `id` and `rev` of the 0.21 object, `code` and `message`; `total`, `truncated`, `limit` and
1574
+ `complete` are as in the loader's report; `notes` are the untranslated rules.
1575
+ - A repeat adds what is missing without duplicates; `--update` also adds new versions of changed objects. The report is
1576
+ JSON at the end of the output: objects and versions by class in 0.21 and in 1.0, `verify()`, untranslated rules,
1577
+ classes without a key, and uuids in `data` that match old ids (the import does not rewrite them). The exit code is 1
1578
+ if `verify()` found discrepancies.
1579
+
1580
+ ## 15. API reference
1581
+
1582
+ The full machine-readable list of exports, methods and error texts is `docs/api-contract.json` in the repository.
1583
+
1584
+ ### Package exports
1585
+
1586
+ | Entry point | What it exports |
1587
+ |---|---|
1588
+ | `letopis` | `connect`, `up`, `createTenant`; operators `ne`, `gt`, `gte`, `lt`, `lte`, `between`, `inList`, `like`, `ilike`, `starts`, `ends`, `has`, `hasAny`, `hasAll`, `exists`, `isNull`, `not`, `or`, `isOp`; `inc`, `isInc`, `cursorOf`; `idOf`, `uuidv5`, `v5Name`, `v5Id`; `LetopisError`, `ValidationError`, `fromDbError`; `syncModels`, `generateTypes`, `schemaToTs`; `compileValidator`, `applyDefaults`, `Registry`; `totpCode`, `hashPassword`, `verifyHash`; `describeText`; constants `DDL_REVISION`, `RESERVED_CLASS_NAMES`, `CHAIN_METHODS`, `DB_METHODS`; types `Db`, `DbTx`, `TypedDb`, `Chain`, `Row`, `Filter`, `Cursor`, `ConnectOptions`, `UpOptions`, `LoadOptions`, `LoadReport`, `WatchOptions`, `WatchEvent`, `AuthApi`, `AclApi` and others |
1589
+ | `letopis/model` | `hub`, `link`, `one`, `many`, `End`, types `Infer`, `InferLinks`, `Model`, `ClassDef`, `ClassMeta`, `HistoryPolicy`; `normalizeSchema`, `translatePattern`, type `JsonSchema` |
1590
+ | CLI `letopis` | `sync`, `types`, `load`, `verify`, `maintain`, `describe`, `import` ([section 13](#13-operations)) |
1591
+
1592
+ Removed 0.21 names (`uuidv7`, `LETOPIS_NS` and the methods listed below) throw `removed` with the replacement.
1593
+
1594
+ ### The `db` root
1595
+
1596
+ | Member | What it does |
1597
+ |---|---|
1598
+ | `db.ClassName(filter)` | the start of a chain |
1599
+ | `schema`, `token`, `tenant`, `registry` | the installation name, the token, the session tenant, the class registry (`resolve`, `find`, `has`, `all`, `family`) |
1600
+ | `auth`, `acl` | sign-in, credentials and sessions; permission decisions ([section 10](#10-users-sign-in-and-permissions)) |
1601
+ | `as(account, { tenant, owner })` | the same pool on behalf of an account (async) |
1602
+ | `begin()`, `commit(tr)`, `rollback(tr)`, `lock(…keys)` | an explicit transaction ([section 8](#8-transactions-batches-and-money)) |
1603
+ | `batch(name)` | a queue of plans: `run()`, `discard()`, `size()` |
1604
+ | `load(input, options)` | the loader ([section 9](#9-bulk-loading)) |
1605
+ | `watch(options)`, `cache` | subscription; result cache (`clear()`, `stats()`) |
1606
+ | `` sql`…` ``, `transaction(fn)` | raw SQL under the session and RLS |
1607
+ | `idOf(class, key)` | the id of an object of a class with a key, in the session tenant |
1608
+ | `entity(row \| chain)` | start a path from a ready node |
1609
+ | `describe()` | a machine-readable description of the model |
1610
+ | `verify(options)`, `maintain(options)`, `reset(options)` | verification, maintenance, reset ([sections 11](#11-history-and-point-in-time-reads) and [13](#13-operations)) |
1611
+ | `trimHistory(target, moment)`, `rekeyClass(class, key)` | history trimming; a new class key |
1612
+ | `refresh()`, `flushSignals()`, `close()` | re-read the classes; send the signals now; close |
1613
+
1614
+ ### Chain
1615
+
1616
+ | Group | Methods |
1617
+ |---|---|
1618
+ | steps | `.ClassName(filter)`, `.AliasName(filter)`, `.RoleName(filter)` |
1619
+ | modifiers | `limit(n)`, `offset(n)`, `sort(field, 'asc' \| 'desc')`, `after(cursor)`, `asOf(moment)`, `withDeleted()`, `deep(max)`, `exact()`, `alias(name)`, `tags(…)`, `owner(…)`, `entity(…)`, `forUpdate()`, `cache({ ttl })` |
1620
+ | verbs | `create(data)`, `update(patch, { rev })`, `upsert(data)`, `delete({ confirm })`, `anonymize(fields)`, `reclass(class, data)`, `rekey(patch)`, `restore()`, `purge({ confirm })` |
1621
+ | slots | after a verb: `.RoleName.set(value)`, `.RoleName.unset()`, `.RoleName.add(value)`, `.RoleName.remove(value)` |
1622
+ | terminals | `rows()`, `first()`, `ids()`, `count({ paths })`, `paths()`, `versions({ follow })`, `sum(field)`, `avg(field)`, `min(field)`, `max(field)`, `countBy(field)` |
1623
+
1624
+ ### `db.auth`
1625
+
1626
+ | Method | Parameters → result |
1627
+ |---|---|
1628
+ | `setPassword` | `{ account, identifier, password, kind?, confirmed?, keepCurrent? }` → credential id |
1629
+ | `verifyPassword` | `{ identifier, password, kind?, requireConfirmed?, ttl?, totp? }` → `{ account, token }` or `null` |
1630
+ | `issueApiKey`, `verifyApiKey` | `{ account, name?, ttl? }` → `{ key, id }`; `(key)` → sign-in or `null` |
1631
+ | `issueKeySecret`, `verifyKeySecret` | `{ account, name? }` → `{ key, secret, id }`; `(key, secret)` → sign-in or `null` |
1632
+ | `enrollTotp`, `verifyTotp`, `totpEnabled` | `{ account, issuer?, label? }` → `{ secret, uri }`; `{ account, code, window? }` → `boolean`; `(account)` → `boolean` |
1633
+ | `issueOtp`, `verifyOtp` | `{ account, identifier, kind?, ttl?, digits?, maxAttempts? }` → `{ code }`; `{ identifier, code, kind? }` → sign-in or `null` |
1634
+ | `link`, `lookup` | `{ account, kind, identifier, meta?, confirmed? }` → id; `{ kind, identifier, requireConfirmed?, ttl?, totp? }` → sign-in or `null` |
1635
+ | `credentials`, `revokeCredential`, `setFlags` | `(account)` → credentials without secrets; `(id, { keepCurrent? })`; `(id, { confirmed?, enabled? })` |
1636
+ | `sessionFor`, `refresh`, `revoke`, `revokeAll` | `(account, { ttl? })` → token; `({ ttl? })` → the new expiration time (the token stays the same); `()`; `(account?)` → number of sessions |
1637
+ | `tenants`, `switch` | `()` → `[{ tenant, roles }]`; `(tenant)` |
1638
+ | `purgeAccount` | `(account)` → `{ account, rows, versions, memberships, owned }` |
1639
+
1640
+ Password, code and TOTP checks, session issuing and credential confirmation are called by the service (`auth.*`
1641
+ permissions); a regular user gets `acl_denied`. The credentials of System, services and system administrators can be
1642
+ changed only by the account itself after a recent interactive sign-in, or by the system administrator. The exception is
1643
+ the service's keys: the service issues and revokes its own API keys and "key + secret" pairs itself with a session of
1644
+ its own key, but not its other credentials; nobody can revoke the last active key of a service (`acl_denied`). With
1645
+ `totp`, `verifyPassword` and `lookup` also check the TOTP code (the `auth.totp` permission) and issue a session with the
1646
+ `TOTP` method. The credential kind `kind` is `^[A-Z][A-Z0-9_]{1,31}$` (`PASSWORD`, `EMAIL`,
1647
+ `TELEGRAM` …); `SESSION`, `APIKEY`, `KEYSECRET`, `OTP`, `TOTP` and `KEY` are taken.
1648
+
1649
+ ### Result row
1650
+
1651
+ | Field | Meaning |
1652
+ |---|---|
1653
+ | `id`, `class`, `rev` | id, class, version number (for `update(patch, { rev })`) |
1654
+ | `tenant`, `owner` | the tenant and the owner of the row |
1655
+ | `links`, `data`, `tags` | ends, fields, tags |
1656
+ | `at`, `author`, `agent`, `op`, `reason` | version time (ISO, UTC, microseconds), author, agent (the service under impersonation), kind of operation, reason |
1657
+ | `moved` | the reference to the old or the new id after `reclass` and `rekey` |
1658
+ | `$deleted` | tombstone: a deleted entity or a deletion version |
1659
+ | `$depth` | the depth of a node in `deep()`; the cascade level in a deletion preview |
1660
+ | `$upsert`, `$action`, `$versions`, `$purged` | what `upsert` did; what the deletion will do; the number of versions in `purge`; the history is erased |
1661
+
1662
+ ### Reserved names
1663
+
1664
+ The Proxy parses the names below before classes, so a class with such a name cannot be reached as a step, and the
1665
+ database does not accept a link with such a name (`reserved_names()`; the installer fills in the list from
1666
+ `RESERVED_CLASS_NAMES`).
1667
+
1668
+ - chain methods: `then`, `entity`, `paths`, `rows`, `first`, `ids`, `count`, `limit`, `offset`, `sort`, `asOf`,
1669
+ `withDeleted`, `deep`, `exact`, `sum`, `avg`, `min`, `max`, `countBy`, `after`, `versions`, `create`, `update`,
1670
+ `upsert`, `set`, `unset`, `add`, `remove`, `delete`, `anonymize`, `reclass`, `restore`, `purge`, `alias`, `tags`,
1671
+ `owner`, `forUpdate`, `cache`, `rekey`; guards of the removed API: `run`, `execute`, `account`;
1672
+ - the `db` root: `as`, `begin`, `commit`, `rollback`, `lock`, `batch`, `watch`, `close`, `registry`, `sql`, `load`,
1673
+ `verify`, `maintain`, `reset`, `describe`, `auth`, `acl`, `idOf`, `refresh`, `transaction`, `flushSignals`,
1674
+ `schema`, `token`, `tenant`, `trimHistory`, `rekeyClass`, `cache`; guards: `reloadSchema`, `accounts`,
1675
+ `credentials`, `resources`, `rules`, `compact`;
1676
+ - batch: `discard`, `size`.
1677
+
1678
+ ## 16. Errors
1679
+
1680
+ The text of an error is `letopis: <code>: <message>`. The code is in `err.code`, the details are in `err.detail`.
1681
+ Errors are `LetopisError` objects; data errors are `ValidationError` with `err.issues: [{ path, keyword, message }]`.
1682
+
1683
+ Database codes. Raw SQL gets them too: SQLSTATE class `LT`, the details are JSON in `DETAIL`.
1684
+
1685
+ | Code | SQLSTATE | When |
1686
+ |---|---|---|
1687
+ | `no_session` | LT001 | there is no valid session |
1688
+ | `target_not_found` | LT002 | the target is not found; the same for a nonexistent, a foreign and an invisible target |
1689
+ | `invalid_data` | LT003 | the data does not match the class description; `err.issues` gives the path, keyword and message |
1690
+ | `undeclared_end` | LT004 | the end is not declared in the class |
1691
+ | `tenant_denied` | LT005 | a foreign tenant or no membership |
1692
+ | `acl_denied` | LT006 | denied by the access rules |
1693
+ | `id_mismatch`, `id_from_db` | LT007, LT008 | the id is not v5 of the key; the id of a class without a key is assigned by the database |
1694
+ | `immutable_key` | LT009 | a key field, the id, the tenant and the owner do not change (a new key is `rekey`) |
1695
+ | `invalid_class` | LT010 | the class description is rejected, the class is not found or is abstract |
1696
+ | `use_reclass`, `reclass_denied` | LT011, LT012 | changing the class with `update` when the old or the new class has a key: the id changes, so use `reclass()`; the roles of referencing rows do not accept the new class |
1697
+ | `tightening_conflict` | LT013 | rows prevent tightening the description |
1698
+ | `isolation_level` | LT014 | deleting, changing the class and changing the key work only in `read committed` |
1699
+ | `recreated_in_statement` | LT015 | an id was deleted and created again in one statement |
1700
+ | `cursor_expired` | LT016 | the subscription cursor is older than the `all` tier of the retention policy |
1701
+ | `delete_restricted` | LT017 | referencing rows prevent the deletion (`onDelete: 'restrict'`) |
1702
+ | `not_deleted` | LT018 | `restore` and `purge` work only for deleted objects |
1703
+ | `reset_disabled` | LT019 | reset is disabled in the installation (`allowReset`) |
1704
+ | `confirm_mismatch` | LT020 | the reset confirmation does not match the schema name |
1705
+
1706
+ Library codes:
1707
+
1708
+ | Code | When |
1709
+ |---|---|
1710
+ | `exists` | the object already exists: `create` (`detail.id` is its id), `rekey` (the new id is taken) or the loader |
1711
+ | `duplicate`, `deleted` | the loader: a repeated id in the load; an object with this id existed before. The loader throws the code of the first problem row, the first 100 are in `detail.rows`, the total is `detail.total` |
1712
+ | `conflict` | `update(patch, { rev })`: the row was changed in the meantime (`detail`: `id`, `rev`, `expected`) |
1713
+ | `rev_ambiguous` | `{ rev }` on a step that does not have exactly one target |
1714
+ | `tx_required` | `forUpdate()`, `lock()`, `commit()` or `rollback()` outside an active `db.begin()` |
1715
+ | `lock_unsupported` | `forUpdate()` with an aggregate, `asOf`, `versions`, `withDeleted`, `deep` or a verb |
1716
+ | `commit_unknown` | the connection dropped during `COMMIT`: the outcome is unknown |
1717
+ | `tx_rolled_back` | PostgreSQL answered `ROLLBACK` to `COMMIT`: the error was caught inside `tr.transaction(fn)` |
1718
+ | `no_key` | `upsert`, `rekey` or `idOf` on a class without a key |
1719
+ | `no_path` | there is no hop between the classes of neighboring steps |
1720
+ | `invalid_query` | the chain or the API is used incorrectly (a step modifier `deep`, `exact`, `alias`, `tags` or `owner` after a verb, a slot not after a verb, an `after()` cursor with a value of a different type than the `sort` field, without `v` or with an `id` that is not a uuid, and so on) |
1721
+ | `admin_required` | the loader needs an admin connection |
1722
+ | `class_not_loadable` | the loader does not accept `Class` rows |
1723
+ | `bypass_rls` | `connect()` with a role that bypasses RLS, without `allowBypassRls` |
1724
+ | `removed` | a removed 0.21 API; the text gives the replacement |
1725
+
1726
+ A PostgreSQL permission denial (`42501`) comes, for example, on an `upsert` over someone else's row that you may not
1727
+ edit.
1728
+
1729
+ ## 17. Performance
1730
+
1731
+ Benchmark of 1.0 against 0.21 on the same datasets (`cd lib && npx tsx bench/compare.bench.mjs`, 2026-10-09,
1732
+ PostgreSQL 18, and 0.21 on PostgreSQL 17 with TimescaleDB, one machine): salon has ~980 thousand rows (in 0.21 they
1733
+ belong to System without isolation, in 1.0 they are in a tenant under RLS and permissions); tenants has 500 thousand
1734
+ customers with 2 versions each, across ten tenants. The numbers depend on the machine; what matters is the ratio.
1735
+
1736
+ | Measurement | 1.0 | 0.21 | Unit | Note |
1737
+ |---|---|---|---|---|
1738
+ | loading the salon dataset | 6359 | 3609 | rows/s | 1.0: the loader in history mode with checks; 0.21: raw INSERTs into Entity |
1739
+ | loading the tenants dataset | 8566 | 35186 | rows/s | 0.21: INSERT … SELECT with triggers disabled |
1740
+ | row-by-row create | 128 | 156 | rows/s | |
1741
+ | read by key | 2.2 | 3.81 | ms (p50) | |
1742
+ | three-step chain (count) | 222.79 | 1233.83 | ms (p50) | organization → specialists → bookings |
1743
+ | point-in-time query (asOf) | 202.77 | 327.98 | ms (p50) | a specialist's bookings as of mid-year |
1744
+ | 20 concurrent writers | 982 | 522 | rows/s | |
1745
+ | class change (reclass) | 9.5 | — | ms (p50) | 0.21 has no such verb |
1746
+ | count() of bookings: a tenant under RLS and permissions | 272.25 | 890.73 | ms (p50) | 0.21: without isolation and permissions (enforceAccount: false); 1.0 without RLS (admin): 18.59 ms |
1747
+ | count() of customers of tenant 1 (50.0%) | 114.09 | 667.19 | ms (p50) | |
1748
+ | count() of customers of tenant 3 (10.0%) | 23.8 | 229.94 | ms (p50) | |
1749
+ | count() of customers of tenant 8 (1.0%) | 5.3 | 88.02 | ms (p50) | |
1750
+ | count() of customers of tenant 10 (0.3%) | 3.54 | 26.08 | ms (p50) | |
1751
+
1752
+ A full import of the salon dataset from 0.21 (`npm run test:import-salon`): 539,580 objects and 980,180 versions in
1753
+ 193 s (5,084 rows/s); the per-class counters matched, and `verify()` is clean. Checkpoint B tuning (revision 5): the
1754
+ journal index on reference targets, `log_ends`, made an `asOf` read across a hop 16 times faster (3.37 s → 0.20 s) and
1755
+ costs the loader 15–17% of its speed. Reading `letopis.changed` and `COMMIT` in one pipeline removed a round trip to the
1756
+ database from every write (row-by-row writes: 119 → 128–147 rows/s in different runs). Row-by-row `create` is still
1757
+ 12–18% slower than 0.21 (the measurements vary): most of the time goes to the check triggers in the database and to the
1758
+ commit with a disk write.
1759
+
1760
+ ## 18. Limitations
1761
+
1762
+ This section lists what letopis 1.0 deliberately does not do or does with caveats, and where you need to watch the
1763
+ behavior.
1764
+
1765
+ **Platform**
1766
+
1767
+ - PostgreSQL 18 or newer is required; there are no workarounds for 15–17. AWS RDS, Google Cloud SQL, Azure Flexible
1768
+ Server and Neon support 18, while Supabase, as of October 2026, offers only 15 and 17, so 1.0 will not work there.
1769
+ - The database must be in UTF8 encoding (`casefold()` and the `pg_unicode_fast` collation). If a new major PostgreSQL
1770
+ version updates the Unicode tables, the unique index of credential identifiers is rebuilt with the `reindex` command.
1771
+ - PGlite has one superuser connection: no concurrent writers and no `LISTEN`, and RLS does not apply
1772
+ (`allowBypassRls: true`); it does not replace `postgres:18` for permissions, race tests and CI.
1773
+ - A pool in transaction mode (PgBouncer): `LISTEN` is not available, so use `connect({ listen: false })`; the
1774
+ subscription polls the journal, and the result cache is off. Prepared statements work with PgBouncer 1.21; with older
1775
+ versions, pass a postgres.js pool with `prepare: false` in `connect({ sql })`.
1776
+ - The NUL character (`\u0000`) is impossible in data strings: PostgreSQL does not store it in `jsonb`; such data is
1777
+ rejected already when the JSON is parsed (in the official JSON-Schema-Test-Suite, the "nul characters" groups are
1778
+ skipped).
1779
+
1780
+ **Writing and integrity**
1781
+
1782
+ - Row-by-row writing costs more than a plain insert: triggers check every row (data, ends, id, journal, hash, RLS).
1783
+ Row-by-row `create` is 12–18% slower than 0.21 ([section 17](#17-performance)); for bulk writing there is the loader.
1784
+ - Deleting, changing the class and changing the key work only in `read committed` (`isolation_level`); library
1785
+ transactions always open at this level.
1786
+ - PostgreSQL does not run `COPY` into tables under RLS; bulk loading works only through the loader on an admin
1787
+ connection.
1788
+ - The database trusts the loader: the data, ends and ids of loaded rows are checked by the library code (the TypeScript
1789
+ validator), not by the triggers; the database checks only the targets outside the load, the tenants and the owners,
1790
+ with one query at the end. The safeguards: the loader is available only to an admin, differential tests of the
1791
+ validator against the database, `load({ verify: true })` and `db.verify({ data: true })`. Keep the admin connection
1792
+ in a separate loading process, not in the web application: an SQL injection there would get the owner's rights.
1793
+ - The schema owner and a superuser can disable the triggers and change anything. letopis does not prevent this; it
1794
+ detects it: `verify()` recomputes the hash chains, `verify({ data: true })` checks the rows again, and the anchor (a
1795
+ hash of all chain heads) should be kept outside the database.
1796
+ - The database checks a subset of JSON Schema: Zod `refine`, `superRefine` and `transform` work only in the library,
1797
+ and the database does not accept a description with an unknown keyword; `pattern` is a limited subset of regular
1798
+ expressions shared by Zod, the validator and the database. The output of `z.toJSONSchema()` changes between Zod
1799
+ versions: the database stores an already normalized schema, and a `syncModels` dry run shows the difference before it
1800
+ is applied.
1801
+ - The key is the only way to get uniqueness: the id is computed from the key values without normalization, so the same
1802
+ value written in a different way (an email in a different case, a time with and without milliseconds) gives different
1803
+ ids. The model sets the format of a key field, and the database checks it.
1804
+ - Frequent changes of a key value (email, phone) require `rekey`. Values that change often and personal values are
1805
+ better moved to a separate hub with a link: an id from an email can be found by brute force and stays in references and in the journal
1806
+ even after `anonymize`, while a separate hub is deleted together with the link through `purge`.
1807
+ - `create` with a taken key answers `exists` with the id, and `upsert` without permission on the existing row answers
1808
+ with a `42501` denial: this reveals that the key is taken, but only within your own tenant, because in other tenants
1809
+ the same keys give other ids.
1810
+ - Changing the class and the key rewrites all incoming references (and, recursively, the links whose key includes this
1811
+ end), so its cost is proportional to the number of references. Tightening a class description and `rekeyClass` pause
1812
+ writes to the family while they check or move rows.
1813
+ - `rekeyClass` moves rows in one transaction under an exclusive lock of the family, not in batches: between batches,
1814
+ rows with old ids would break the rule "the id is v5 of the key".
1815
+ - v5 ids land in the primary key index in scattered places; on very large tables with heavy inserts this is more
1816
+ noticeable than with sequential ids.
1817
+ - Hot reference targets (an organization that almost everything references) accumulate `for key share` locks and
1818
+ MultiXact, as with regular foreign keys; this needs monitoring. Operations on one wallet go one after another under a
1819
+ lock of its row ([section 8](#8-transactions-batches-and-money), "Recipe: wallet").
1820
+ - A link cannot have a reserved name, and a hub with such a name cannot be reached as a step
1821
+ (["Reserved names"](#reserved-names)).
1822
+
1823
+ **Access**
1824
+
1825
+ - The service key is the root of trust: its compromise allows acting on behalf of any enabled user of the
1826
+ installation except System, services and system administrators; the access rules do not narrow this scope. The key
1827
+ is reissued (`db.auth.issueApiKey`) and revoked (`db.auth.revokeCredential`) by the service itself with a session of
1828
+ its own key, or by the system administrator; with its key the service also issues itself new keys, so after a leak
1829
+ revoke all its keys except the new one (`credentials` lists them). Nobody can revoke the last active key of a
1830
+ service: to cut a service off, set `enabled: false` on its account. A database password alone, without a token,
1831
+ gives nothing.
1832
+ - The permission pass reveals its own plan to the client (the classes and rule conditions that apply to it).
1833
+ - Raw SQL without the permission pass pays for the permission plan on every query: 0.3–1.3 ms depending on the number
1834
+ of rules.
1835
+ - The end names `User`, `Tenant` (membership), `Group` and `Object` (rule) are taken by system classes across the
1836
+ whole schema.
1837
+ - OTP and TOTP attempt counters are rolled back together with the caller's transaction: the service limits the rate of
1838
+ attempts.
1839
+ - When users link a taken identifier themselves, the answer is `acl_denied`, so a user with a recent sign-in learns
1840
+ this way that the identifier is taken. Permission changes in all tenants are serialized through one counter row.
1841
+ - The number of invisible rows that block a deletion, and the existence of an account by id (as a reference target),
1842
+ are not hidden.
1843
+ - Class metadata (`meta`) is visible to anyone who has a session: anyone can read `Class` rows. Secrets do not belong
1844
+ there.
1845
+ - An application role can slow the installation down for everyone: `LOCK TABLE entity`, advisory locks with
1846
+ predictable keys, fake `NOTIFY` signals (they only clear the cache), frequent permission changes (one counter row per
1847
+ installation). The global journal number reveals the overall write activity.
1848
+
1849
+ **History, signals, cache**
1850
+
1851
+ - An `asOf` read is rebuilt from the journal; only the current state is cheap. In the thinned part of the journal, a
1852
+ point-in-time read shows the last kept version before the moment: its precision there equals the period of the tier,
1853
+ a day, a week, a month or a year. A history import into a class with a policy is thinned right away, and a subscriber
1854
+ may not see the old versions.
1855
+ - The thinned part of the journal proves neither that the history is complete nor that old versions are authentic: for
1856
+ a class with a policy, `verify()` cannot tell the tampering of a version older than the `all` tier from thinning.
1857
+ - The journal has no partitions: thinning deletes versions in batches, and the table file does not shrink by itself
1858
+ (new versions take the space after autovacuum); to return the space to the system, use `VACUUM FULL` or `pg_repack`
1859
+ in a maintenance window.
1860
+ - Signals reveal activity: PostgreSQL lets any role listen to a channel, and the class names in signals show when and
1861
+ to which classes writes happen (a signal has no ids, no tenant and no data). A fake signal only clears the cache one
1862
+ extra time.
1863
+ - The `NOTIFY` queue (8 GB) is shared by the whole cluster: a stuck listener fills it up, and then transactions with
1864
+ `NOTIFY` fail on commit. The library reads signals right away, and `maintain()` warns when the queue is more than
1865
+ half full.
1866
+ - The subscription horizon is shared by the whole cluster: a long write transaction delays the events of all
1867
+ subscribers (`sub.lag()`); long operations are better done in batches.
1868
+ - Between processes, the result cache goes stale for the signal sending interval (100 ms) plus delivery, and for the
1869
+ time of a listener reconnect; the `ttl` term is a safeguard.
1870
+ - A "from now" subscription cursor carries a snapshot of the subscription moment until it passes that boundary: such a
1871
+ cursor is longer than a regular one.
1872
+
1873
+ **Import from 0.21**
1874
+
1875
+ - 1.0 is a new major version, not a continuation of 0.21: `count()`, `paths()`, `create`, roles, multiple ends and
1876
+ `reclass` behave in a new way, and the data is moved by an import; see [MIGRATION.md](MIGRATION.md) (in Russian).
1877
+ - Import: the steps (classes, accounts, credentials, permissions, the data of each tenant) are separate transactions,
1878
+ and the source is read into memory entirely; a row owner that is not found among the accounts becomes the tenant;
1879
+ 0.21 access rules move to System and apply in all tenants; the `onDelete` of ends is `cascade`, the way 0.21 deleted.
1880
+
1881
+ ## 19. Tests
1882
+
1883
+ This section is for those who develop letopis itself. Run: `cd lib && npm test`. The harness starts the
1884
+ `letopis-pg18` container (`postgres:18`, port 15433) by itself if the database does not respond; each test file works in its own schema `v2.t_<name>`.
1885
+
1886
+ ```
1887
+ # connect — connecting under a session, refusing roles that bypass RLS, the registry, the deferred signal, error translation (stage 2)
1888
+ # context — sessions, service sign-in by key, the RLS policies of stage 1 (stage 1)
1889
+ # access-rest — the remaining rows of stage 5: loading on behalf of a user, balance No. 20, 22, 29, upsert and { rev } under ACL, inlining of acl_ok (stage 5)
1890
+ # acl — ACL permissions: the 0.21 suite ported, row conditions, weight and default deny, cascade, deny on a descendant, deep, an invisible target, raw SQL versus the library (stage 5)
1891
+ # acl-parity — 1000 random rule sets: the database permission plan and endpoint decisions versus decide from 0.21 (stage 5)
1892
+ # auth — credentials (password, key, key + secret, OTP, TOTP, identity), timing equalization, case, session revocation, recent sign-in (stage 5)
1893
+ # balance — top-up and debit: idempotency by opId, races, deadlock, connection drop (TCP proxy), adjustment under { rev } and forUpdate (stage 4)
1894
+ # chain-api — guards of the removed API, reserved names, build errors, the plan cache, onQuery (stage 3)
1895
+ # chain-read — read chains on the demo domain: hops, roles, multiple ends, 18 operators, keyset, aggregates, deep, asOf, withDeleted, pivot (stage 3)
1896
+ # chain-types — types of steps, rows and filters from models: connect({ models }) (stages 3, 4)
1897
+ # chain-write — writing with chains: create, update, upsert, delete with preview, anonymize, reclass, slots, inc, { rev }, fan-out, plan rollback (stage 4)
1898
+ # demo — the booking demo domain on Zod: seed from models, writing data, id by key, agreement between Zod and the database (stage 2)
1899
+ # docs — documentation examples are executed: the marked blocks of the README and the cheatsheet (Russian and English) run as scenarios against PostgreSQL 18 (stage 8)
1900
+ # engine-attacks — raw SQL attacks from the application role: the rows of §7.3.1 for stages 1 and 3 with specific outcomes (stages 1, 3)
1901
+ # engine-class — classes as entities: metaschema, inheritance, recomputing descendants, tightening, metadata (stage 1)
1902
+ # engine-write — the write pipeline in raw SQL: v5 ids, defaults, merge, inc, deletion, resurrection, journal, signals (stage 1)
1903
+ # engine-races — races on two connections in both orders, without timed pauses (stage 1)
1904
+ # explain — chain indexes from the application role: outside, the primary key and (class, tenant); inside find_*(), GIN (stage 3)
1905
+ # find-ids-parity — 30 chains through find_*() and through a direct read under RLS give the same answers (stage 3)
1906
+ # hash — the version hash chain, journal tampering and gaps, verify() (stage 1)
1907
+ # harness — the test harness: starting the database, a schema per file, two connection strings (stage 0)
1908
+ # jsonschema — compiling JSON Schema to jsonpath and checking data in the database (stage 1)
1909
+ # impersonation — impersonation: letopis.account only for a service with the permission, target restrictions, author and agent in the journal, sessionFor (stage 5)
1910
+ # install — the installer and roles: a repeated installation, two installations in one database, three tables (stage 1)
1911
+ # member — memberships: working in a tenant by roles, auth.switch, leaving with ownership transfer, system rows, tenant roles without System permissions (stage 5)
1912
+ # model — Zod models, the JSON Schema normalizer, pattern translation, agreement with the database metaschema (stage 2)
1913
+ # load — the loader: several classes, cycles, $key and $ext, repeats, existing, portions, history mode, the query for targets outside the load, CLI (stage 4)
1914
+ # roles — roles and privileges: installation without a superuser, owner functions, triggers by catalog (stage 1)
1915
+ # write-races — write races: 20 concurrent edits of different fields without losses and strict rev, concurrent add/remove on a multiple end, the loader versus class tightening and versus a concurrent edit (existing: update) in both orders (stage 4)
1916
+ # validator-parity — the TypeScript validator versus the database check: the official suite, 10,000 random documents, defaults (stage 2)
1917
+ # scenario-barbershop — end-to-end barbershop scenario: 9 scenes from 0.21 (integration) on 1.0 chains — catalog, free slots in a batch, bookings under lock, multi-slot and multi-specialist, versions, cancellation and booking again on the same id, closing a month with restrict (stage 4)
1918
+ # scenario-real-life — the "real life" of a salon: 16 scenes from 0.21 (real-life) — a booking service on top of chains, races of double booking, rescheduling and cancellation, walk-in, refusals, reports and daily invariants (stage 4)
1919
+ # system-classes — system classes instead of the 0.21 facades: the System seed, accounts, credentials, resources and rules, transactions (stage 5)
1920
+ # sync — letopis sync (dry run, apply, tightening, lost refine) and letopis types (stage 2)
1921
+ # tx — explicit db.begin() transactions: a broken transaction, lock(), 40P01; forUpdate(); the batch and its merging, a batch queue per facade; db.as(…, { owner }) (stage 4)
1922
+ # v5 — v5 identifiers on the SQL side versus the vectors of an independent implementation (stage 1)
1923
+ # zod-output — a snapshot of the z.toJSONSchema() output by field kind of the demo domain versus a reference (stage 0)
1924
+ # time — history: versions({ follow }) across reclass and rekey, withDeleted, restore, purge, history trimming, account deletion (stage 6)
1925
+ # rekey — a new key value of an object and a new key of a class: new ids, moving references, recursion through links with the end in the key (stage 6)
1926
+ # reset — reset of sessions, tenant, all: the allowReset flag, the letopis.reset permission, confirmation by the schema name, the reset record, a new service key (stage 6)
1927
+ # verify — legitimate gaps and tampering, verify({ data: true }): data, ends, tenants, id by key; permissions and CLI (stage 6)
1928
+ # retention — retention policy: tiers, a finite term, special versions, deleted rows, asOf, cursor_expired, time zone, catch-up run, inheritance and default — versus a reference in TypeScript (stage 6)
1929
+ # watch — cursor subscription: commit order, a long transaction, resuming, listener drop, two tenants (stage 6)
1930
+ # cache — result cache: hits, own and others' writes, the key with the user, permissions, revoked sessions, listener drop, ttl, class marking, without LISTEN, a race, db.sql (stage 6)
1931
+ # maintain — maintenance: expired sessions and codes, permissions, one process under a lock, schedule (stage 6)
1932
+ # up — the full installer: fresh installation, re-apply, custom seeds, automatic database creation, UTF8, version 17, upgrade with migrations, PGlite, the first tenant and user (up({ tenant }), createTenant) (stages 6, 8)
1933
+ # scenario-salon — end-to-end salon scenario: 21 acts from 0.21 (salon) with a coverage matrix of the 1.0 API (stage 6)
1934
+ # telemetry — onQuery and slowMs, describe() and class metadata with inheritance, describeText (stage 6)
1935
+ # import — import from 0.21 with a fixture of a real 0.21: key violation, --keyless, per-class counters, verify, self-reference, repeat, ids without a key, a dangling reference, $$strict and fields starting with $, update, credentials, CLI (stage 7)
1936
+ # security-final — the final security review (stage 8, §7.10 No. 8.4): a family lock by the tenant of the root, account_purge without permission and restore/purge of a foreign id give neither a leak nor a lock; the functions of stage 6, log_refs_at, journal-based find_* reads and watch_read stay within their own tenant; NOTIFY carries only names; the cache key distinguishes the token, the actor and the tenant (stage 8)
1937
+ ```