letopis 1.4.2 → 3.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (123) hide show
  1. package/AGENT-CHEATSHEET.md +516 -398
  2. package/CHANGELOG.md +739 -0
  3. package/LICENSE +75 -21
  4. package/LICENSE-COMMERCIAL.md +49 -0
  5. package/README.md +1312 -1190
  6. package/api-contract.json +381 -0
  7. package/dist/acl.d.ts +81 -1
  8. package/dist/acl.js +281 -13
  9. package/dist/admin.d.ts +11 -141
  10. package/dist/admin.js +28 -186
  11. package/dist/auth.d.ts +20 -5
  12. package/dist/auth.js +80 -21
  13. package/dist/cache.d.ts +8 -1
  14. package/dist/cache.js +30 -8
  15. package/dist/chain.d.ts +46 -48
  16. package/dist/chain.js +556 -223
  17. package/dist/cli.js +73 -96
  18. package/dist/demo/booking.d.ts +352 -36
  19. package/dist/describe.d.ts +47 -0
  20. package/dist/describe.js +195 -0
  21. package/dist/errors.d.ts +77 -13
  22. package/dist/errors.js +231 -30
  23. package/dist/index.d.ts +62 -74
  24. package/dist/index.js +312 -155
  25. package/dist/internal.d.ts +7 -0
  26. package/dist/jsonschema.js +18 -17
  27. package/dist/load.d.ts +29 -17
  28. package/dist/load.js +235 -190
  29. package/dist/log.d.ts +121 -0
  30. package/dist/log.js +633 -0
  31. package/dist/mcp.d.ts +5 -7
  32. package/dist/mcp.js +194 -225
  33. package/dist/messages/access.d.ts +90 -0
  34. package/dist/messages/access.js +113 -0
  35. package/dist/messages/api.d.ts +237 -0
  36. package/dist/messages/api.js +304 -0
  37. package/dist/messages/boot.d.ts +201 -0
  38. package/dist/messages/boot.js +256 -0
  39. package/dist/messages/codes.d.ts +172 -0
  40. package/dist/messages/codes.js +173 -0
  41. package/dist/messages/index.d.ts +98 -0
  42. package/dist/messages/index.js +121 -0
  43. package/dist/messages/log.d.ts +229 -0
  44. package/dist/messages/log.js +102 -0
  45. package/dist/messages/mcp.d.ts +306 -0
  46. package/dist/messages/mcp.js +387 -0
  47. package/dist/messages/model.d.ts +256 -0
  48. package/dist/messages/model.js +314 -0
  49. package/dist/messages/query.d.ts +141 -0
  50. package/dist/messages/query.js +171 -0
  51. package/dist/messages/sql-access.d.ts +70 -0
  52. package/dist/messages/sql-access.js +87 -0
  53. package/dist/messages/sql-core.d.ts +164 -0
  54. package/dist/messages/sql-core.js +198 -0
  55. package/dist/messages/sql-time.d.ts +86 -0
  56. package/dist/messages/sql-time.js +111 -0
  57. package/dist/messages/sql-write.d.ts +246 -0
  58. package/dist/messages/sql-write.js +270 -0
  59. package/dist/messages/time.d.ts +23 -0
  60. package/dist/messages/time.js +29 -0
  61. package/dist/messages/types.d.ts +27 -0
  62. package/dist/messages/types.js +5 -0
  63. package/dist/messages/write.d.ts +452 -0
  64. package/dist/messages/write.js +572 -0
  65. package/dist/model.d.ts +77 -60
  66. package/dist/model.js +43 -34
  67. package/dist/ops.d.ts +35 -0
  68. package/dist/ops.js +101 -0
  69. package/dist/pglite.d.ts +0 -9
  70. package/dist/pglite.js +3 -1
  71. package/dist/registry.d.ts +3 -19
  72. package/dist/registry.js +3 -16
  73. package/dist/signals.d.ts +1 -5
  74. package/dist/signals.js +1 -5
  75. package/dist/sql.d.ts +9 -1
  76. package/dist/sql.js +175 -107
  77. package/dist/sync.d.ts +10 -4
  78. package/dist/sync.js +90 -17
  79. package/dist/tx.d.ts +21 -2
  80. package/dist/tx.js +76 -13
  81. package/dist/typed.d.ts +252 -53
  82. package/dist/types.d.ts +93 -90
  83. package/dist/types.js +17 -14
  84. package/dist/up.d.ts +56 -35
  85. package/dist/up.js +229 -83
  86. package/dist/uuid.d.ts +8 -0
  87. package/dist/uuid.js +14 -6
  88. package/dist/validate.d.ts +13 -1
  89. package/dist/validate.js +82 -33
  90. package/dist/watch.d.ts +10 -16
  91. package/dist/watch.js +36 -77
  92. package/dist/write.d.ts +87 -17
  93. package/dist/write.js +441 -206
  94. package/llms.txt +48 -31
  95. package/package.json +13 -14
  96. package/sql/10-core.sql +7 -50
  97. package/sql/15-errors.sql +64 -10
  98. package/sql/20-context.sql +18 -9
  99. package/sql/30-validate.sql +79 -30
  100. package/sql/40-class.sql +41 -115
  101. package/sql/50-acl.sql +8 -13
  102. package/sql/60-write.sql +220 -676
  103. package/sql/70-read.sql +11 -13
  104. package/sql/80-auth.sql +109 -42
  105. package/sql/90-time.sql +108 -947
  106. package/sql/95-seed.system.sql +7 -14
  107. package/sql/99-revision.sql +6 -1
  108. package/sql/README.md +32 -24
  109. package/sql/migrate/0009-restore-rev.tenants-name.sql +10 -0
  110. package/sql/migrate/0010-impersonation-no-account.sql +5 -0
  111. package/sql/migrate/0011-key-lock.otp-enabled.sql +7 -0
  112. package/sql/migrate/README.md +18 -0
  113. package/AGENT-CHEATSHEET.en.md +0 -488
  114. package/MIGRATION.md +0 -190
  115. package/README.en.md +0 -2910
  116. package/dist/import.d.ts +0 -185
  117. package/dist/import.js +0 -820
  118. package/dist/project.d.ts +0 -359
  119. package/dist/project.js +0 -1258
  120. package/sql/75-history.sql +0 -436
  121. package/sql/88-derived.sql +0 -1236
  122. package/sql/migrate/0006-changed-paths.sql +0 -3
  123. package/sql/migrate/0007-secret-account-fk.sql +0 -12
package/README.en.md DELETED
@@ -1,2910 +0,0 @@
1
- # letopis 1.4
2
-
3
- **letopis** is a Node.js library: a store of versioned entities on PostgreSQL 18. Every change is saved as a version
4
- in a journal. So every object has a history, a state as of any date and a change subscription. The database itself
5
- checks data integrity, tenant isolation and permissions, with triggers and RLS policies. The rules work the same for
6
- the library and for raw SQL. Data is read and written with chains:
7
- `db.Shop(shop).Product({ price: lt(1000) }).rows()`.
8
-
9
- [Русская версия](README.md) · [Cheatsheet for AI agents](AGENT-CHEATSHEET.en.md) · [Migration from 0.21 (in Russian)](MIGRATION.md) · [Changelog (in Russian)](CHANGELOG.md) · [llms.txt](llms.txt)
10
-
11
- What letopis can do:
12
-
13
- - **Zod models** are stored in the database as classes: hubs (objects) and links (relations with their own data) — [section 4](#4-data-model).
14
- - **Writing with chains**: `create`, `update` with merge, `upsert`, deletion with preview and cascade, the `inc` increment
15
- for money and counters — [sections 5](#5-writing), [7](#7-deleting-and-restoring) and [8](#8-transactions-batches-and-money).
16
- - **Reading with chains**: hops along links, filters, trees, aggregates, pagination — [section 6](#6-reading).
17
- - **History**: versions, point-in-time reads, what changed and who changed it (`diff`, `blame`), restoring deleted
18
- data, retention policy, hash chain verification — [section 11](#11-history-and-point-in-time-reads).
19
- - **Tenants, sign-in and permissions** in the database itself: passwords, API keys, one-time codes, TOTP, sessions,
20
- impersonation, access rules — [section 10](#10-users-sign-in-and-permissions).
21
- - **Change subscription** by cursor without losses, and a **result cache** — [section 12](#12-change-subscription-and-cache).
22
- - **Derived data**: aggregates in the database, sums and counts that the database itself maintains, and projectors
23
- that build summaries from the journal — [sections 4](#aggregates-in-the-database) and [12](#projectors).
24
- - **Agents and MCP**: an MCP server gives an AI agent the model, reads, history and, if allowed, writes, with the
25
- rights of one user — [section 13](#agents-and-mcp).
26
- - **Bulk loading** of millions of rows and **import from 0.21** — [sections 9](#9-bulk-loading) and [14](#14-migrating-from-021).
27
-
28
- Requirements:
29
-
30
- - PostgreSQL 18 or newer with the pgcrypto extension (it is part of the standard distribution), and a database in UTF8
31
- encoding.
32
- - Node.js 22 or newer; the package is an ES module.
33
- - One runtime dependency: `postgres`. Zod 4 is an optional peer dependency: only the `letopis/model` subpath needs it,
34
- and `import { connect } from 'letopis'` works without it. For development without PostgreSQL there are the optional
35
- `@electric-sql/pglite` and `@electric-sql/pglite-socket`. The MCP server (`letopis/mcp` and `letopis mcp`) needs
36
- the optional `@modelcontextprotocol/server`, version 2.4 or newer in the 2.x line.
37
-
38
- ## Contents
39
-
40
- 1. [Quick start](#1-quick-start)
41
- 2. [How letopis works](#2-how-letopis-works)
42
- 3. [Installation and connection](#3-installation-and-connection)
43
- 4. [Data model](#4-data-model)
44
- 5. [Writing](#5-writing)
45
- 6. [Reading](#6-reading)
46
- 7. [Deleting and restoring](#7-deleting-and-restoring)
47
- 8. [Transactions, batches and money](#8-transactions-batches-and-money)
48
- 9. [Bulk loading](#9-bulk-loading)
49
- 10. [Users, sign-in and permissions](#10-users-sign-in-and-permissions)
50
- 11. [History and point-in-time reads](#11-history-and-point-in-time-reads)
51
- 12. [Change subscription and cache](#12-change-subscription-and-cache)
52
- 13. [Operations](#13-operations)
53
- 14. [Migrating from 0.21](#14-migrating-from-021)
54
- 15. [API reference](#15-api-reference)
55
- 16. [Errors](#16-errors)
56
- 17. [Performance](#17-performance)
57
- 18. [Limitations](#18-limitations)
58
- 19. [Tests](#19-tests)
59
-
60
- Sections 3–13 are built on one example, a shop: installation, model, writing, reading and so on. The code blocks of
61
- these sections go in order and together form one program. The `docs` test runs this program, like the quick start
62
- example, against a real database, so the examples do not drift away from the code.
63
-
64
- ## 1. Quick start
65
-
66
- You need PostgreSQL 18 and an admin role: a superuser or a role with the `CREATEROLE` privilege. Install the package:
67
-
68
- ```sh
69
- npm install letopis zod postgres # postgres: for createTenant and your own SQL queries
70
- ```
71
-
72
- The application connects to the database with a separate role that is a member of `letopis_app`. The installation
73
- creates `letopis_app`. So after the first installation, create the application role once:
74
-
75
- ```sql
76
- create role shop_app login password 'app-secret';
77
- grant letopis_app to shop_app;
78
- ```
79
-
80
- The example installs the schema, creates a shop with its first user, describes two classes and works with them:
81
-
82
- <!-- run: quickstart -->
83
- ```ts
84
- import { up, connect, syncModels, inc, gte } from 'letopis'
85
- import { hub } from 'letopis/model'
86
- import { z } from 'zod'
87
-
88
- const ADMIN = 'postgres://postgres:secret@localhost:5432/shop' // admin: installs the schema
89
- const APP = 'postgres://shop_app:app-secret@localhost:5432/shop' // application: a member of letopis_app
90
-
91
- // 1. Install the v2.shop schema. tenant is the first tenant (the shop) and its user Anna.
92
- const r = await up({ dsn: ADMIN, schema: 'shop',
93
- tenant: { name: 'Chamomile', user: { name: 'Anna', login: 'anna@shop.ru', password: 'Chamomile-2026!' } } })
94
- // r.serviceKey is the application's service key: it is issued on the first installation, so save it
95
-
96
- // 2. The application service signs in with the key, checks Anna's password and works on her behalf in the shop.
97
- const svc = await connect({ dsn: APP, schema: r.schema, apiKey: r.serviceKey! })
98
- const login = await svc.auth.verifyPassword({ identifier: 'anna@shop.ru', password: 'Chamomile-2026!' })
99
- const db = await svc.as(login!.account, { tenant: r.tenant!.tenant })
100
-
101
- // 3. Model: a shop and a product. syncModels writes the class descriptions to the database.
102
- const Shop = hub('Shop', { name: z.string() }, { key: ['name'] })
103
- const Product = hub('Product', { sku: z.string(), title: z.string(), price: z.number().int().min(0) },
104
- { ends: { Shop }, key: ['Shop', 'sku'] })
105
- await syncModels(db, [Shop, Product], { apply: true })
106
-
107
- // 4. Writing and reading with chains.
108
- const [shop] = await db.Shop().create({ name: 'Chamomile' }).rows()
109
- const [kettle] = await db.Shop(shop).Product().create({ sku: 'K-1', title: 'Tea kettle', price: 2500 }).rows()
110
- await db.Product(kettle).update({ price: inc(-500) }).rows() // discount: the database subtracts inside the statement
111
- const expensive = await db.Shop(shop).Product({ price: gte(1000) }).rows()
112
- const history = await db.Product(kettle).versions() // two versions: creation and edit
113
-
114
- await svc.close()
115
- ```
116
-
117
- What happened here:
118
-
119
- - `up()` installed the `v2.shop` schema in the database: three tables, triggers, RLS policies, functions and the
120
- system seed. The `tenant` option created the tenant "Chamomile" with the rule "authenticated users may do
121
- everything", and the user Anna with the `owner` role in it.
122
- - `connect({ apiKey })` exchanged the service key for a session. `verifyPassword` returned `{ account, token }` (or
123
- `null` if the password is wrong). `svc.as(account, { tenant })` is the same connection pool, but acting as Anna in
124
- the shop; the database checks that the service has the right to do this.
125
- - `hub('Product', fields, options)` described an object class. `ends: { Shop }` is a required reference to the shop.
126
- `key: ['Shop', 'sku']` is the key: the product id is computed from the shop and the SKU, and the database does not
127
- accept a second product with the same key.
128
- - `db.Shop(shop).Product().create(…)` created a product and took the reference to the shop from the chain path.
129
- `inc(-500)` lowered the price inside the `UPDATE` statement, without reading it first. `versions()` read the history
130
- from the journal.
131
-
132
- Calling `up()` again is safe: it re-applies the same revision and does not touch the data, and it does not issue the
133
- service key again while the installation still has at least one API key that is not revoked. Next, section 2 explains the design, and sections
134
- 3–13 go through the tasks in order.
135
-
136
- ## 2. How letopis works
137
-
138
- ### Core concepts
139
-
140
- | Concept | What it is |
141
- |---|---|
142
- | installation | a PostgreSQL schema `v2.<name>` with the letopis tables and functions; one database can hold several |
143
- | class | a data type: name, fields (JSON Schema), ends and key. A class description is also a row in the database (class `Class`) |
144
- | hub | an object class: a shop, a product, a customer |
145
- | link | a relation class between objects, with its own data: an order line, a specialist's skill |
146
- | end (role) | a reference from one row to another row. An end has a name, the role: `Shop`, `Parent`. Ends are in `row.links`, fields are in `row.data` |
147
- | key | the fields and ends that identify an object within a class and a tenant. The id is computed from them, so uniqueness is the key |
148
- | id | a version 5 UUID: a hash of the tenant, the class and the key values; for a class without a key, a hash of a random part |
149
- | account | a user, a service or an organization: a row of class `Account` |
150
- | tenant | the account whose data is being worked with. Data of different tenants is isolated |
151
- | membership | a `member` link "user — tenant" with roles; it allows work in another tenant |
152
- | session | a token by which the database identifies the account and the tenant. A service gets it with an API key, a user by signing in |
153
- | version, journal | each change of a row is a version in the `log` journal, with author, time and hash |
154
- | tombstone | a "deleted" version with the last snapshot of the object |
155
- | chain | a query like `db.Shop(shop).Product().rows()`: steps, modifiers, write verbs and a terminal |
156
- | terminal | a method that executes the chain: `rows`, `first`, `ids`, `count`, `paths`, aggregates |
157
-
158
- ### What is stored in the database
159
-
160
- There are three tables. `entity` is the current state: one row per live entity, including the rows of system classes
161
- (`Class`, `Account`, `Resource`, `member`, `rule`). `log` is the journal: a full snapshot of every version with author,
162
- time, kind of operation and hash. `secret` holds credentials, sessions and projector checkpoints: the application sees
163
- none of its rows; only the sign-in and projector functions work with it.
164
-
165
- Classes are entities too. A class description is stored in a `Class` row; the database also keeps the expanded
166
- description (with ancestors applied) and the compiled data check there. The version hash is a generated column: sha256 of the
167
- previous version's hash and the row snapshot. So the next version exposes a tampered version, and for the last version
168
- an anchor outside the database does this ([section 11](#11-history-and-point-in-time-reads)).
169
-
170
- ### Who checks the data
171
-
172
- There are two roles. `letopis_owner` owns the schema, tables and functions. `letopis_app` is the application role: the
173
- application's LOGIN role is a member of this role only. Everything the application runs, both the library and raw SQL,
174
- goes through triggers and RLS.
175
-
176
- - **Triggers.** On insert, a trigger computes the id, fills in defaults and checks the data against the class JSON
177
- Schema (the schema is compiled to jsonpath). It checks the ends and their targets, fills in the service columns and
178
- writes a version to the journal. An update merges the patch inside the statement (`merge`, `inc`). After a delete,
179
- the trigger applies the `restrict`, `cascade` and `unset` rules and writes tombstones.
180
- - **RLS and permissions.** Policies isolate tenants and apply access rules. At the start of a query, the database
181
- builds the actor's permission plan once (`acl_plan()`); each row then needs only the cheap `acl_ok` check.
182
- - **Query context** is the transaction settings: `letopis.token` (the session), `letopis.tenant` and `letopis.account`
183
- (the tenant and impersonation), `letopis.reason` (the reason for the change) and `letopis.acl` (the permission pass,
184
- see [section 10](#10-users-sign-in-and-permissions)). The database finds the session by the sha256 of the token, so
185
- without a real token the settings give nothing.
186
- - **Bypassing the checks** is possible only for code that runs as the owner: owner functions (`security definer` with
187
- a fixed `search_path`) and the loader on an admin connection. `connect()` refuses a role that bypasses RLS (a
188
- superuser, `BYPASSRLS`, a member of `letopis_owner`) unless `allowBypassRls: true` is set.
189
-
190
- ### How reads work
191
-
192
- Under RLS, the indexes on `data` and on back references are not directly available to the policy. So the ids for chain
193
- steps are found by the owner functions `find_ids()`, `find_hop()` and `find_deep()`. They apply the same tenant and
194
- permission conditions as the policy. The rows themselves are then read by primary key under RLS. A point-in-time read
195
- is built from the journal.
196
-
197
- ### Signals
198
-
199
- A write reports changes through `NOTIFY`. The channel is the full name of the installation schema, version included,
200
- for example `v2.shop`; because of the dot, SQL takes the name in double quotes: `LISTEN "v2.shop"`. A raw SQL write on
201
- its own connection sends the signal from a trigger, and PostgreSQL delivers it on commit. Library transactions,
202
- `db.sql` and `db.transaction()` included, defer their signals: the process notifier sends them every 100 ms
203
- (`notifyIntervalMs`) and retries the ones that failed to send. What a write signal carries (its payload) is set by the
204
- installation mode `up({ signals })`:
205
-
206
- | Mode | Write signal payload | What any role that can connect to the database sees |
207
- |---|---|---|
208
- | `class` (default) | the class name: `Booking` | class names and write times |
209
- | `tenant` | `Class@tenant token`: `Booking@3f9c…` | also tenant pseudonyms |
210
- | `opaque` | `class token@tenant token`: `9a1e…@3f9c…` | only write times and frequency per pseudonym |
211
- | `none` | no signals | nothing |
212
-
213
- A token is 24 hexadecimal characters of a keyed HMAC: only a process with the signal key can match a pseudonym to a
214
- name, and the key is given to services (the `letopis.signals` permission). Besides write signals, there are service
215
- ones: a revoked session sends `session:<16 hex>` (the first 16 hexadecimal characters of the sha256 of its token), a
216
- session reset sends `session:*`, and a reset of a tenant or of the whole installation sends `*` (a reset of the whole
217
- installation also sends `session:*`). In the `none` mode there are none of them either. Signals carry no ids and no
218
- data. On signals, the class registry is re-read, the result cache is cleared and subscriptions wake up; what the mode
219
- changes for the cache and subscriptions is in [section 12](#12-change-subscription-and-cache).
220
-
221
- ## 3. Installation and connection
222
-
223
- ### Installation: `up()`
224
-
225
- `up()` waits for the database and creates it if it does not exist. It checks for PostgreSQL 18 and UTF8 encoding,
226
- installs the roles `letopis_owner` and `letopis_app`, the pgcrypto extension and the `v2.<name>` schema with all
227
- functions and the system seed. On the first installation, it issues a service key. Call it from an admin role.
228
-
229
- | Option | What it does |
230
- |---|---|
231
- | `dsn` or `sql` | an admin connection string or a ready postgres.js pool |
232
- | `schema` | the installation name: `'shop'` gives the schema `v2.shop`; you can also pass `'v2.shop'` directly |
233
- | `tenant` | the first tenant and its user (see below) |
234
- | `seeds` | seeds after the core: `'booking'` is the salon demo domain; or paths to your own `.sql` files |
235
- | `history` | the history retention policy for classes without their own ([section 11](#11-history-and-point-in-time-reads)) |
236
- | `serviceKey` | your own service key instead of a random one: `lts_` and 32–128 hexadecimal characters |
237
- | `allowReset` | allow `db.reset()` ([section 13](#13-operations)); off by default |
238
- | `signals` | the signal mode: `'class'` (default), `'tenant'`, `'opaque'` or `'none'` (["Signals"](#signals)); the mode is stored in the schema — `up()` without the option keeps it |
239
- | `fresh` | drop the schema and install it again — **the schema data is lost** |
240
- | `upgrade` | upgrade a schema of an older revision: migrations and a re-apply |
241
- | `pglite` | a database for development without PostgreSQL (see below) |
242
- | `quiet`, `waitTimeoutMs` | no console messages; how long to wait for the database (60,000 ms by default) |
243
- | `owner`, `app` | the names of the owner and application roles (`letopis_owner`, `letopis_app` by default) |
244
-
245
- Result: `schema` (the full name), `serviceKey` (on the first installation, and on a re-run when the installation has no
246
- API key left except revoked ones: an expired or disabled key counts too; otherwise `null`), `created`, `upgraded`, `revision`, `migrations`, `analyzed`, `owner`, `app`,
247
- `signals` (the signal mode of the installation), `warnings` (without `quiet` they are printed), and also `tenant` and
248
- `pglite` if they were requested. On PostgreSQL below 18.2, `up()` warns: such a server could truncate CLOG that unread
249
- signals still need — upgrade it.
250
-
251
- Several instances of a service may call `up()` for the same schema at the same time: the installation runs under a lock
252
- on the schema. The first `up()` installs or upgrades it, the others wait and re-apply the ready schema
253
- (`created: false`, `upgraded: false`); one of them creates the first `tenant`, the others find it by name. `up()` calls
254
- for different schemas of one database do not wait for each other.
255
-
256
- To hide class names from signals, use the `opaque` mode (or `tenant` if the names are not secret but you want the
257
- cache to be precise per tenant); after changing the mode, restart the application processes — they read the mode on
258
- connect:
259
-
260
- ```ts
261
- await up({ dsn: ADMIN, schema: 'shop', signals: 'opaque' }) // later up({ dsn, schema }) keeps the mode
262
- ```
263
-
264
- <!-- run: guide -->
265
- ```ts
266
- import { up, connect } from 'letopis'
267
-
268
- const ADMIN = 'postgres://postgres:secret@localhost:5432/shop'
269
- const APP = 'postgres://shop_app:app-secret@localhost:5432/shop'
270
-
271
- const r = await up({ dsn: ADMIN, schema: 'shop',
272
- tenant: { name: 'Chamomile', user: { name: 'Anna', login: 'anna@shop.ru', password: 'Chamomile-2026!' } } })
273
- const SHOP = r.tenant!.tenant // the tenant: the Chamomile shop
274
- const ANNA = r.tenant!.user! // its first user
275
- ```
276
-
277
- ### Application role
278
-
279
- The application works as a LOGIN role that is a member of `letopis_app` only:
280
- `create role shop_app login password '…'; grant letopis_app to shop_app;`. Then RLS and triggers apply to it, as to
281
- any raw SQL from this role. `connect()` rejects a role that bypasses RLS with the error `bypass_rls`. The admin
282
- connection (`allowBypassRls: true`) is needed only for the loader, projectors, aggregate repair (`rebuildAggregates`),
283
- verification and maintenance; keep it in a separate process, not in the web application.
284
-
285
- ### First tenant and user
286
-
287
- Permissions in letopis are always on and deny everything by default. So a new tenant needs rules. The
288
- `up({ tenant })` option and the `createTenant(sql, schema, opts)` function do this in one transaction:
289
-
290
- - the tenant: an account in System with an id from the name (`tenant:<name>`);
291
- - the rule "authenticated users may do everything" in this tenant (`grantAll`, on by default, weight 10);
292
- - a user (id from the login) with a membership in the tenant with the roles `user.roles` (`['owner']` by default), a
293
- password and a session in the tenant.
294
-
295
- A repeat with the same name and login duplicates nothing and does not change the password; it only issues a new
296
- session. The login goes into the id as is: `Anna@shop.ru` and `anna@shop.ru` would give two accounts, and the second one
297
- would have no password, so write logins in one case. The `created` field in the result tells whether this call created
298
- the tenant: for an existing tenant with a new user it is `false`. The roles are set by `user.roles`, a non-empty list
299
- of names (not masks), otherwise `invalid_data`. The password kind `user.kind` (`PASSWORD` by default) follows the rule
300
- for a credential kind, as in `auth.setPassword` ([API reference](#dbauth)), otherwise it is `invalid_data` too. A
301
- repeat does not change the roles of an existing membership: they are
302
- changed by editing the membership ([Granting a permission](#granting-a-permission)). A repeat does not bring back
303
- what was deleted: if the tenant, the user account or the membership is deleted (the row is gone but its history is in
304
- the journal), `createTenant` answers `invalid_data` and writes nothing. The error text names what was deleted and
305
- advises bringing it back with `restore()`: an account with `db.Account(id).restore()` on behalf of the system
306
- administrator, a membership with `db.member(id).restore()` in that tenant; the other way out is a different name or
307
- login. The session from `createTenant` has no sign-in method, so the user cannot change their own password with it:
308
- for that the user signs in with the password. `createTenant` needs an admin postgres.js pool:
309
-
310
- <!-- run: guide -->
311
- ```ts
312
- import postgres from 'postgres'
313
- import { createTenant } from 'letopis'
314
-
315
- const pg = postgres(ADMIN)
316
- const lutik = await createTenant(pg, r.schema, {
317
- name: 'Buttercup', user: { name: 'Boris', login: 'boris@lutik.ru', password: 'Buttercup-2026!' } })
318
- await pg.end()
319
- const BORIS = lutik.user! // lutik: { tenant, user, token, created: true }
320
- ```
321
-
322
- ### Connecting: `connect()`
323
-
324
- A connection gets a session in one of two ways: with an API key (`apiKey`) or with a token (`token`). An application
325
- service usually keeps one connection by key and acts on behalf of users through `as()`:
326
-
327
- <!-- run: guide -->
328
- ```ts
329
- // service: the API key is exchanged for a service session
330
- const svc = await connect({ dsn: APP, schema: r.schema, apiKey: r.serviceKey! })
331
- // sign-in: the service checks the password and acts on behalf of the user in the tenant
332
- const login = await svc.auth.verifyPassword({ identifier: 'anna@shop.ru', password: 'Chamomile-2026!' })
333
- const db = await svc.as(login!.account, { tenant: SHOP })
334
- // a separate connection by session token, for example in another process
335
- const anna = await connect({ dsn: APP, schema: r.schema, token: r.tenant!.token! })
336
- ```
337
-
338
- | `connect()` option | What it does |
339
- |---|---|
340
- | `dsn` or `sql` | a connection string or a ready postgres.js pool |
341
- | `schema` | the full installation name: `'v2.shop'` (it is handy to take `r.schema`) |
342
- | `token`, `apiKey` | a session token or an API key (`service` is a synonym of `apiKey`) |
343
- | `models` | `letopis/model` models, only for chain types ([section 4](#4-data-model)) |
344
- | `max` | pool size (10 by default) |
345
- | `listen` | listen to database signals (yes by default; `false` for PgBouncer in transaction mode) |
346
- | `cache` | result cache: `{ ttl, max, ttlOnly }` or `false` ([section 12](#12-change-subscription-and-cache)) |
347
- | `maintain` | run `maintain()` on a schedule: `'1h'`, `'15m'` or milliseconds |
348
- | `onQuery`, `slowMs` | a hook on every chain query and the slow query threshold |
349
- | `refreshSession` | extend the session once per minute of activity (yes by default) |
350
- | `notifyIntervalMs` | the interval for sending deferred signals (100 ms) |
351
- | `allowBypassRls`, `owner` | allow a role that bypasses RLS (admin connection); the name of the owner role |
352
-
353
- A connection has `db.schema`, `db.token`, `db.tenant` (the session tenant) and `db.close()`. Connections obtained
354
- through `as()` share the pool with the original one: close the original one.
355
-
356
- ### Admin connection
357
-
358
- The loader, projectors and `rebuildAggregates` need a connection of a role that can become the owner, with
359
- `allowBypassRls: true`. `verify()` and
360
- `maintain()` work on it too, and also for a system administrator on a regular connection. The admin connection prints
361
- the warning "RLS не действует" ("RLS does not apply"); this is expected. The session sets the tenant it works with. A
362
- separate loading process is better off with a service key:
363
- `connect({ dsn: ADMIN, schema, apiKey, allowBypassRls: true })`, and the tenant of the rows is set by the
364
- `load(…, { tenant })` option. The example uses the session from `up()`:
365
-
366
- <!-- run: guide -->
367
- ```ts
368
- const adm = await connect({ dsn: ADMIN, schema: r.schema, token: r.tenant!.token!, allowBypassRls: true })
369
- ```
370
-
371
- ### Upgrading the package and schema
372
-
373
- The schema has an engine revision (currently 7). If a new package version brings a new revision, `connect()` only warns
374
- and names the command. It is `up()` that refuses: a lagging schema without `upgrade: true`, and a schema newer than the
375
- package. To a schema newer than the package (a newer package has already upgraded it, and old application instances
376
- are still running) `connect()` connects anyway and warns that the letopis package should be updated. So a startup script that calls `up()` on every start will stop after a package update until it gets
377
- `upgrade: true`. `up({ dsn, schema, upgrade: true })` applies the migrations from `sql/migrate/` and re-applies the
378
- functions; the data is kept. Revision 7 (1.4.1) adds the foreign key `secret.account`: migration `0007` first erases
379
- the `secret` rows whose accounts are already gone (the credentials of accounts deleted earlier were kept with the
380
- `deleted_at` mark), and then sets the key.
381
-
382
- The revision does not change with every version: 1.1–1.4.0 share one (6), while their functions differ. So `up()` also writes
383
- an engine stamp into the schema comment: the package version and a hash of its SQL files. If the stamp does not match
384
- the package files, `connect()` warns. When the schema was applied by an older version (or by a version before 1.4,
385
- which has no stamp), it suggests `up({ schema })`. When a newer version has already upgraded the schema, it suggests
386
- updating the package: `up()` from the older version would bring the old functions back. A new package version without
387
- SQL changes keeps the stamp, and there is no warning.
388
-
389
- The first `up()` after moving to 1.2 builds the journal index `log_class_tenant_at (class, tenant, at)` instead of
390
- `log_class_at`: an `asOf` and `withDeleted` slice by class no longer reads the journal of other tenants with the same
391
- class name. While the index is being built, writes to the journal wait. On a large installation, build it in advance
392
- without stopping writes: `create index concurrently log_class_tenant_at on "v2.<schema>".log (class, tenant, at)` as
393
- the owner role — then `up()` only drops the old one.
394
-
395
- The `upgrade` test checks upgrading 1.4.1, 1.1.0 and 1.0.1 schemas; every release checks the upgrade from the previous
396
- release. The old package installs the schema and the old engine
397
- writes the data, then the current version's `up()` runs. After it, the versions and hashes written by the old engine
398
- are in place, `verify({ data: true })` is clean, and the schema catalog matches a clean installation. A 1.4.1 schema of the
399
- same revision 7 is upgraded with a plain `up()`, without migrations; 1.1.0 (revision 6) and 1.0.1 (revision 5) — with
400
- `up({ upgrade: true })`: with migration `0007` and with migrations `0006` and `0007`.
401
-
402
- ### Versions and compatibility
403
-
404
- letopis versions follow semver. What each version step promises:
405
-
406
- - **Patch** (1.4.1 → 1.4.2) — fixes only. The engine revision stays the same: a plain `up({ schema })` re-applies the
407
- fixed functions, there are no migrations; until it runs, `connect()` warns about the stamp mismatch. Working code gets
408
- no new refusals, and no methods, options, exports or error codes disappear.
409
- - **Minor** (1.4 → 1.5) — new features: methods, options, error codes; behavior changes that do not break working code;
410
- a new engine revision. `up({ schema, upgrade: true })` applies the new revision. Nothing is removed or renamed.
411
- - **Major** (1.x → 2.0) — removed and renamed methods, options, exports and error codes, changed signatures and types,
412
- new refusals for everyone. `MIGRATION.md` describes what to change in your code.
413
-
414
- The text of an error message is not part of the API: a patch may make it more precise. Rely on `err.code`.
415
-
416
- Before every release the `check:semver` script checks the version step: it compares the API contract
417
- (`docs/api-contract.json`) and the engine revision with the previous release and requires at least the matching step.
418
- Every release checks the schema upgrade from the previous release (the `upgrade` test).
419
-
420
- ### Development without PostgreSQL: PGlite
421
-
422
- `up({ pglite: true })` starts PGlite (PostgreSQL 18 on WebAssembly) with pgcrypto and a socket on a free port. It needs
423
- the `@electric-sql/pglite` and `@electric-sql/pglite-socket` packages. There is only one connection, so:
424
-
425
- ```ts
426
- const dev = await up({ schema: 'dev', pglite: true })
427
- const local = await connect({ dsn: dev.pglite!.dsn, schema: dev.schema, apiKey: dev.serviceKey!,
428
- listen: false, max: 1, allowBypassRls: true })
429
- // … work …
430
- await local.close()
431
- await dev.pglite!.close()
432
- ```
433
-
434
- The connection to PGlite is a superuser, so it needs `allowBypassRls: true`, and RLS does not apply on it: tenant
435
- isolation and permissions cannot be checked on PGlite. It is not fit for race tests and CI either: there is one
436
- connection and no `LISTEN`.
437
-
438
- ## 4. Data model
439
-
440
- Classes are described with Zod models from the `letopis/model` subpath and written to the database. The database
441
- stores a normalized JSON Schema and checks the data itself on every write.
442
-
443
- ### Hubs and links
444
-
445
- `hub()` describes an object, `link()` describes a link with its own data. Both have two forms:
446
-
447
- - short: `hub(name, fields, options)`: fields as an object of Zod types, options as the third argument;
448
- - full: `hub(name, { fields, ends, key, extends, … })`: it is needed, for example, for inheritance (`extends`), so
449
- that TypeScript infers the data types of the descendant.
450
-
451
- The shop model that the following examples are built on:
452
-
453
- <!-- run: guide -->
454
- ```ts
455
- import { z } from 'zod'
456
- import { hub, link, one, many } from 'letopis/model'
457
- import { syncModels } from 'letopis'
458
-
459
- const Shop = hub('Shop', { name: z.string(), city: z.string().optional() }, { key: ['name'] })
460
- const Category = hub('Category', { name: z.string() }, {
461
- key: ['name'],
462
- ends: { Parent: one('Category').optional().onDelete('unset') }, // tree: a reference to the parent
463
- })
464
- const Product = hub('Product', {
465
- sku: z.string(), title: z.string(), price: z.number().int().min(0), views: z.number().int().optional(),
466
- }, {
467
- ends: { Shop, Category: many(Category, { max: 5, onDelete: 'unset' }) },
468
- key: ['Shop', 'sku'],
469
- meta: { title: 'Product', icon: 'mdi:package-variant' },
470
- })
471
- const Customer = hub('Customer', { name: z.string(), email: z.string() }, {}) // no key: the database assigns the id
472
- const Vip = hub('Vip', { extends: Customer, fields: { discount: z.number().int().min(0).max(50) } })
473
- const Order = hub('Order', {
474
- number: z.string(), status: z.enum(['new', 'paid', 'cancelled']).default('new'),
475
- }, { ends: { Customer, Shop }, key: ['Shop', 'number'] })
476
- const line = link('line', { qty: z.number().int().min(1), price: z.number().int().min(0) }, {
477
- ends: { Order: one(Order).onDelete('cascade'), Product },
478
- key: ['Order', 'Product'],
479
- })
480
- const Wallet = hub('Wallet', { number: z.string(), balance: z.number().int().min(0).default(0) },
481
- { ends: { Customer }, key: ['number'] })
482
- const Movement = link('Movement', {
483
- opId: z.string(), kind: z.enum(['topup', 'debit', 'adjust']), amount: z.number().int(), target: z.number().int().optional(),
484
- }, { ends: { Wallet }, key: ['Wallet', 'opId'] })
485
-
486
- const models = [Shop, Category, Product, Customer, Vip, Order, line, Wallet, Movement]
487
- const plan = await syncModels(db, models) // dry run: what will be created and changed
488
- const applied = await syncModels(db, models, { apply: true }) // { applied: true, created: ['Shop', …], … }
489
- ```
490
-
491
- ### Fields
492
-
493
- Fields are Zod types: strings, numbers (`int`, `min`, `max`), booleans, enums (`z.enum`), dates and times
494
- (`z.iso.date()`, `z.iso.datetime()`), nested objects of any depth, arrays and `z.record`. `.optional()` makes a field
495
- optional. `.default(…)` sets a default that the database fills in on insert. The database stores the `.meta()`
496
- annotations of a field (`title`, `description`, your own keys), but they do not affect data checks.
497
-
498
- The `refine`, `superRefine` and `transform` rules cannot be expressed in JSON Schema, and the database does not run
499
- them. `syncModels` returns them in `lost`: check such rules in the application. A description with a JSON Schema
500
- keyword that the database does not know is rejected, so no rule is lost silently. `pattern` regexes are a limited
501
- subset of regular expressions shared by Zod, the validator and the database.
502
-
503
- ### Ends
504
-
505
- - `ends: { Shop }` is the same as `one(Shop)`: a single required end with the role `Shop`.
506
- - `one(A, B)` is a union: the end accepts rows of several classes. `one('Category')` is a target by name: this is how
507
- a class refers to itself.
508
- - `.optional()` makes the end optional.
509
- - `many(X, { min, max, onDelete })` is a multiple end: a list of targets in one end.
510
- - `.onDelete(…)` sets what happens when a target is deleted: `restrict` (the default: deleting the target is
511
- forbidden), `cascade` (delete this row too), `unset` (remove the reference; only for an optional single end and for a
512
- multiple end without `min`).
513
-
514
- The database checks the targets: the target exists, belongs to the same tenant or to System, and is of the declared
515
- class or its descendant. A role name points to one class across the whole schema. So the roles `User` and `Tenant`
516
- (pointing to `Account`), `Group` and `Object` (pointing to `Resource`) are taken by system classes: your own end with
517
- such a name is possible only with the same target. In a result row, a single end is the target id (`row.links.Shop`),
518
- and a multiple end is a list of ids.
519
-
520
- ### Key and id
521
-
522
- A key is made of required string fields and required single ends. Numbers, booleans, objects, optional fields and
523
- multiple ends cannot be part of a key. The id of a class with a key is a version 5 UUID of the tenant, the class and
524
- the key values. The values are taken as they are, without normalization: an email in a different case gives a
525
- different id. Set the required format of a key field in the model, and the database will check it.
526
-
527
- - You do not pass the id: the database computes it. A different id gives `id_mismatch`; an id for a class without a
528
- key gives `id_from_db`.
529
- - `db.idOf(class, key)` computes the id in advance, for example `db.idOf('Shop', { name: 'Chamomile' })`.
530
- - A class without a key (`Customer` above) gets an id from a random part; you can learn it from the write result.
531
- - A key field cannot be changed by a regular update (`immutable_key`): a new key means a new id, and that is `rekey`
532
- ([section 5](#5-writing)).
533
-
534
- If something other than the key must be unique, make it a separate class. For example, the email is the key of an
535
- `Email` hub, and the link from a person to it has the key `['Email']`. It is better not to make personal data the key
536
- of the person itself: an id from an email can be found by brute force, and it stays in references and in the journal.
537
-
538
- ### Inheritance and aliases
539
-
540
- `extends`: the descendant gets the fields, ends and key of the ancestor. Its own field with the same name replaces the
541
- ancestor's field entirely. `abstract: true`: the class never has its own rows. A chain step by class also returns the
542
- rows of descendants: `db.Customer()` also finds `Vip`, and `.exact()` keeps only the class itself. `alias` is a second
543
- name of the class for a chain step, for example a name in another language.
544
-
545
- ### Class settings
546
-
547
- - `history: false`: no journal is kept for the class: no versions, no point-in-time reads, no restore and no
548
- subscription events. A retention policy is `history: { all: '1 day', daily: '3 months', … }`
549
- ([section 11](#11-history-and-point-in-time-reads)). Only someone with the `READ` and `DELETE` permissions without
550
- conditions on all rows of the class and its descendants may change `history` of an existing class: `maintain()` thins
551
- the journal by the policy, and `false` turns it off.
552
- - `ownerDefault: 'actor'`: the owner of new rows is the user, not the tenant (the default is `'tenant'`). This is
553
- useful for the rule "everyone sees their own records".
554
- - `cache: { ttl }`: reads of this class alone are cached without `.cache()` ([section 12](#12-change-subscription-and-cache)).
555
- - `meta`: free-form metadata for the application: a title, an icon, a color, an order in a menu. Up to 64 KB,
556
- inherited key by key (`null` removes an inherited value), visible to anyone who has a session. For an end, use
557
- `.meta()`.
558
- - `derivedBy: 'projector-name'`: a derived class: only this projector writes its rows ([Projectors](#projectors)).
559
- The name is lowercase Latin letters, digits and `_.:-` (the first character is a letter or a digit), up to 64
560
- characters. The option is inherited, and a descendant does not change it. The ends of a derived class are only
561
- `onDelete: 'cascade'` or `'unset'`: a derived row does not prevent deleting its target. Nothing refers to the rows
562
- of a derived class. The application, raw SQL and the loader do not write to such a class: writing, editing and
563
- deleting its rows gives `derived_class`. The derived class itself (own or inherited) is created, edited and deleted
564
- only by an administrator — an admin connection or an account with the System permission `letopis.projector`; for an
565
- application session it is `acl_denied` ([Projectors](#projectors)).
566
- - `aggregate: { field: { sum | count, of, via, where } }`: an aggregate in the database: the database itself keeps
567
- the sum or count of source rows ([Aggregates in the database](#aggregates-in-the-database)).
568
-
569
- ### Aggregates in the database
570
-
571
- An aggregate is a field that the database itself maintains: the sum of a field or the number of live rows of a
572
- source class whose end leads to the row. It is needed where the total must match the source even for raw SQL: a
573
- wallet balance, the number of order lines.
574
-
575
- ```ts
576
- const Wallet = hub('Wallet', { number: z.string(), balance: z.number().int().min(0), deposits: z.number().int().optional() }, {
577
- key: ['number'],
578
- aggregate: {
579
- balance: { sum: 'amount', of: 'Movement', via: 'Wallet' }, // sum of movements
580
- deposits: { count: true, of: 'Movement', via: 'Wallet', where: { properties: { amount: { minimum: 1 } } } }, // number of top-ups
581
- },
582
- })
583
- const Movement = link('Movement', { amount: z.number().int() }, { ends: { Wallet } })
584
- ```
585
-
586
- - `sum: 'field'` is the sum of a numeric field of the source, `count: true` is the number of rows. `of` is the source
587
- class (a name or a model) together with its descendants, `via` is its single end that leads to the target row.
588
- `where` is a JSON Schema over the source data: rows that do not pass it are not counted.
589
- - Only live rows of the same tenant as the target are counted. A reference to a System row is not counted: a shared
590
- directory does not count the data of all tenants. An aggregate of a tenant class, including one inherited from a
591
- System class, counts only classes of its own tenant: a System class named `of` is not counted, while the tenant's
592
- own descendant of a System class is. An aggregate of a System class counts System classes and the classes of the
593
- target row's tenant.
594
- - A trigger recalculates the field after every statement: writes by the library, raw SQL, a delete cascade,
595
- `restore`, `reclass`, `rekey`, the loader, account deletion. The operations of one statement add up to one increment
596
- of the target, and the class rule (`min(0)`) is checked on the total: an operation that would take the balance below
597
- zero is rejected with `invalid_data`. Every change of the target is its new version with the author and reason of
598
- the operation.
599
- - Only the database writes the field. A value in `create`, `update`, `upsert`, `rekey`, `reclass` or raw SQL is an
600
- `aggregate_field` error. A new row gets the total of already existing sources, usually 0. Replacing the data as a
601
- whole without the field keeps the previous value. The loader does not write a passed value. No permission to write
602
- the target is needed: the permission to write the source is enough.
603
- - The database checks the description (`invalid_class`): the field is in the schema, it is numeric and not part of the
604
- key; the `via` end is single and accepts the target class; the `sum` field is a number; aggregates do not form a
605
- cycle; the target and the source are not system or derived classes. A source class is described before the
606
- aggregate: an aggregate over a class that does not exist yet is refused. For a tenant class, the source is looked up
607
- among the tenant's classes. `syncModels` describes sources before targets itself, and models in a circle (an
608
- aggregate over the class's own descendant, a cycle of aggregates) it refuses before writing (`invalid_class` naming
609
- the circle). The `of` class is not deleted while an aggregate of a class of the same tenant names it: first the
610
- aggregate is removed from the target's description.
611
- - Rows get into an aggregate only with the knowledge of someone who reads them all: the field also counts rows that
612
- the author cannot see. Only someone who reads all rows of the source and its descendants without conditions can
613
- describe or change an aggregate. Only someone who reads all rows of a class and its descendants without conditions,
614
- both before the change and after it, can make the class a source (a new descendant of a source, a `parent` change).
615
- Otherwise `acl_denied`.
616
- - A new source is also checked against whoever described the aggregate: they too must read all rows of the class
617
- without conditions. The describer is the author of the version of the target's `Class` row from which the aggregate
618
- description became the current one. If they cannot read those rows, the class does not become a source
619
- (`acl_denied`) until someone who can read them describes the aggregate again. For an aggregate of a System class, the
620
- describer is checked against System classes, as at description time. A tenant class under a System aggregate is
621
- checked only for whoever creates it.
622
- - The option is inherited, and a descendant replaces it as a whole. Adding or changing an aggregate is a tightening:
623
- writes to the target and to the sources wait while the field is recalculated for all rows.
624
- - `db.rebuildAggregates(class)` recalculates the field from the current rows, for repair. A version is written only
625
- where the value has changed. `db.verify({ data: true })` shows discrepancies. It needs an admin connection.
626
- - A raw `INSERT … ON CONFLICT DO UPDATE SET data = excluded.data` for a target is rejected: the insert trigger has
627
- already set the field in `excluded.data`. Write `excluded.data - array['balance']`, and the field stays as it was;
628
- the library's `upsert` does this itself.
629
-
630
- ### Writing the model to the database
631
-
632
- `syncModels(db, models)` without `apply` is a dry run; with `{ apply: true }` it applies the changes. The result is
633
- `{ applied, created, updated, unchanged, skipped, lost }`; `skipped` lists System classes, which only the installation
634
- changes. `syncModels` touches only the classes passed to it and deletes nothing. Safe changes (a new optional field or
635
- end, metadata, an alias) are applied at once. Anything else, including any change to the `anyOf` and `oneOf` conditions, is a
636
- tightening: it checks all rows of the family; if some rows conflict with it, you get
637
- `tightening_conflict` with a list. Classes are written to the session tenant, and other tenants do not see them.
638
- Classes shared by everyone are in System: the installation seed or the import puts them there.
639
-
640
- The same from the command line:
641
-
642
- ```sh
643
- npx letopis sync ./models.js --dsn "$APP" --schema v2.shop --token "$TOKEN" # dry run
644
- npx letopis sync ./models.js --dsn "$APP" --schema v2.shop --token "$TOKEN" --apply # apply
645
- npx letopis types --dsn "$APP" --schema v2.shop --token "$TOKEN" --out types.ts # class types from the database
646
- ```
647
-
648
- ### Chain types
649
-
650
- `connect({ …, models })` gives types of steps, rows and filters from the models. The library still takes the class
651
- descriptions from the database; only TypeScript needs the models. The row data type is `Infer<typeof Product>`, the
652
- ends type is `InferLinks<typeof Product>`. Your own `meta` shape is typed by extending the `ClassMeta` interface.
653
-
654
- ### Class names
655
-
656
- A chain step is named after a class, its alias or a role name. Where a chain or root method exists, the library parses
657
- its name (`count`, `sort`, `set`, `sql`, `auth` and others) before classes: a class `count` is reachable as the first
658
- step `db.count(…)`, but not after another step. The database does not accept a link with such a name. Do not give
659
- classes these names; the full list is in the
660
- ["Reserved names"](#reserved-names) section.
661
-
662
- ## 5. Writing
663
-
664
- The write verbs — `create`, `update`, `upsert`, `delete`, `anonymize`, `reclass`, `rekey`, `restore`, `purge` — are
665
- parts of a plan. A verb does nothing by itself: a terminal (`rows`, `first`, `ids`, `count`) executes the plan, and the
666
- whole plan is one transaction.
667
-
668
- ### Create: `create`
669
-
670
- <!-- run: guide -->
671
- ```ts
672
- import { LetopisError } from 'letopis'
673
-
674
- const [shop] = await db.Shop().create({ name: 'Chamomile', city: 'Kazan' }).rows()
675
- const [tea] = await db.Category().create({ name: 'Tea' }).rows()
676
- const [green] = await db.Category().create({ name: 'Green tea' }).Parent.set(tea).rows()
677
- const [kettle] = await db.Shop(shop).Product().create({ sku: 'K-1', title: 'Tea kettle', price: 2500 }).rows()
678
- const [sencha] = await db.Shop(shop).Product()
679
- .create({ sku: 'T-7', title: 'Sencha', price: 900 }).Category.add(green).rows()
680
- const [olga] = await db.Customer().create({ name: 'Olga', email: 'olga@mail.ru' }).rows()
681
-
682
- try {
683
- await db.Shop().create({ name: 'Chamomile' }).rows()
684
- } catch (e) {
685
- if (!(e instanceof LetopisError) || e.code !== 'exists') throw e // e.detail.id === shop.id: this shop already exists
686
- }
687
- ```
688
-
689
- - `create` only creates. A taken key gives the error `exists` with `detail.id`: the library computes the id itself. So
690
- for a class with a key, a retry after a network failure shows at once that the first attempt went through. A class
691
- without a key has no such protection: a retry creates a second object.
692
- - The tags of a new row are set by `.tags([…])` before the verb, the owner by `.owner(id)`; an explicit owner must be
693
- allowed by the permissions.
694
- - A deleted object can be created again with the same key: its history continues.
695
- - The result is the rows of the created objects with all columns ([section 15](#15-api-reference), "Result row").
696
-
697
- ### Update: `update`
698
-
699
- <!-- run: guide -->
700
- ```ts
701
- import { gte } from 'letopis'
702
-
703
- await db.Product(kettle).update({ title: 'Glass tea kettle' }).rows()
704
- await db.Shop(shop).Product({ price: gte(2000) }).update({ title: 'Glass tea kettle, 1 L' }).rows()
705
- ```
706
-
707
- `update` edits every target of the step and never creates. The patch is merged with the data inside the statement:
708
- nested objects are merged recursively, while arrays, scalars and `null` are replaced. So concurrent edits of different
709
- fields are not lost. If the step has no filter of its own, the target is taken from `data.id` of the patch, and without
710
- `data.id` all entities of the step are edited: `db.Product().update(…)` edits all products of the tenant. If the step
711
- found nothing, the result is empty and there is no error, so check the length of the result. If `update` affected fewer
712
- rows than the step found, the whole plan is rolled back: a row deleted in the meantime gives `target_not_found`, and a
713
- row that is visible but may not be edited gives `acl_denied`. The `.tags(…)` modifier before `update` selects the rows;
714
- a string or a list also replaces their tags entirely, while an operator (`has`, `hasAny`, `hasAll`) only selects — the
715
- tags stay as they are.
716
-
717
- ### Create or replace: `upsert`
718
-
719
- <!-- run: guide -->
720
- ```ts
721
- const [kept] = await db.Shop().upsert({ name: 'Chamomile', city: 'Kazan' }).rows() // kept.$upsert: 'unchanged'
722
- const [moved] = await db.Shop().upsert({ name: 'Chamomile', city: 'Ufa' }).rows() // moved.$upsert: 'updated'
723
- ```
724
-
725
- `upsert` creates an object, and if an object with this key already exists, it replaces its data, ends and tags
726
- entirely. This is one `INSERT … ON CONFLICT` statement, so the race "checked that it does not exist, then created it"
727
- cannot happen. Rules:
728
-
729
- - only for a class with a key, otherwise `no_key` without a call to the database;
730
- - the object is passed whole: fields that are not in the passed object disappear;
731
- - the result carries `$upsert`: `created`, `updated` or `unchanged`;
732
- - `upsert` cannot increment and overwrites concurrent increments, so it is not suitable for money;
733
- - to sync large reference tables, the loader with `existing: 'update'` is cheaper ([section 9](#9-bulk-loading)).
734
-
735
- ### Ends: from the path and slots
736
-
737
- The ends of a created row are taken from the chain path: `db.Shop(shop).Product().create(…)` references `shop`. If
738
- several roles fit, the role whose slot is free is chosen, then the role named after the step class; otherwise you get
739
- `invalid_query`. A slot after the verb sets an end explicitly: `.RoleName.set(value)` and `.RoleName.unset()`, and for
740
- a multiple end also `.add()` and `.remove()`. The value is an id, a list of ids, a result row or a nested chain. Slots
741
- change `links` atomically, without reading the list:
742
-
743
- <!-- run: guide -->
744
- ```ts
745
- await db.Product(sencha).update({}).Category.add(tea).rows() // one more category
746
- await db.Product(sencha).update({}).Category.remove(tea).rows()
747
- await db.Category(green).update({}).Parent.unset().rows() // the category is now a root
748
- await db.Category(green).update({}).Parent.set(tea).rows() // and inside "Tea" again
749
- ```
750
-
751
- ### Numbers and counters: `inc`
752
-
753
- <!-- run: guide -->
754
- ```ts
755
- import { inc } from 'letopis'
756
-
757
- await db.Product(kettle).update({ price: inc(-500) }).rows() // the price went down by 500
758
- await db.Product(kettle).update({ views: inc(1, { start: 0 }) }).rows() // the field does not exist yet: start from 0
759
- ```
760
-
761
- `inc(n)` adds the amount inside the `UPDATE` statement, over the fresh locked row. So concurrent increments are not
762
- lost, and a class rule like `min(0)` is checked against the result: going below zero is rejected with `invalid_data`,
763
- without reads or locks in the application. Rules:
764
-
765
- - only in `update`; a marker in `create`, `upsert`, the loader or inside an array gives `invalid_data` before sending;
766
- - the field must already be a number; if it is missing, you get `invalid_data`, or `start` is used. `null` and a
767
- non-number are an error even with `start`;
768
- - the path goes only through existing objects;
769
- - an `inc` marker cannot come from JSON: `{ "$inc": 5 }` that came from the network stays data;
770
- - store money as an integer number of minor units (cents), no greater than 2^53 − 1.
771
-
772
- A counter that must match the source even for raw SQL is easier to keep as an aggregate
773
- ([Aggregates in the database](#aggregates-in-the-database)): the database recalculates it itself, and you cannot forget
774
- an `inc`. An aggregate and `inc` cost the same: operations on one object go one after another under the lock of its
775
- row, and each writes its version. On the benchmark machine, one object took 150–190 operations per second this way, no
776
- matter how many writers were waiting for it ([section 17](#17-performance)). For hotter counters, use a projector that
777
- folds a batch ([Projectors](#projectors)): the total lags behind the source, but writers do not wait for each other.
778
-
779
- ### Protection against concurrent edits: `{ rev }`
780
-
781
- <!-- run: guide -->
782
- ```ts
783
- const cur = await db.Product(kettle).first()
784
- await db.Product(kettle).update({ price: 1900 }, { rev: cur!.rev }).rows() // the version matched: written
785
- try {
786
- await db.Product(kettle).update({ price: 1800 }, { rev: cur!.rev }).rows() // the version is already different
787
- } catch (e) {
788
- if (!(e instanceof LetopisError) || e.code !== 'conflict') throw e // re-read and decide again
789
- }
790
- ```
791
-
792
- The strict mode is needed when the new value is computed from what was read: if the row was changed in the meantime,
793
- you get `conflict`, and nothing is written. The mode works when the step has one target (otherwise `rev_ambiguous`).
794
- The library does not retry `conflict` by itself: the application decides what to do with the changed data. Another way
795
- is a read with a `forUpdate()` lock ([section 8](#8-transactions-batches-and-money)).
796
-
797
- ### Changing the key or class
798
-
799
- <!-- run: guide -->
800
- ```ts
801
- const [mug] = await db.Shop(shop).Product().create({ sku: 'M-1', title: 'Mug', price: 400 }).rows()
802
- const [mug2] = await db.Product(mug).rekey({ sku: 'M-100' }).rows() // new key, new id; references moved
803
- const [vip] = await db.Customer(olga).reclass('Vip', { name: 'Olga', email: 'olga@mail.ru', discount: 10 }).rows()
804
- ```
805
-
806
- - `rekey(patch)` sets a new key value: a row with a new id, all references moved, the old row deleted. If a referencing
807
- row includes this end in its own key, its id changes too, and the move goes on. The journal links the old and the new
808
- id (`moved`).
809
- - `db.rekeyClass(class, key)` sets a new key for all rows of the family, in one transaction. For a derived class and
810
- for a class with a derived descendant — `derived_class`: only its projector writes the rows of a derived class.
811
- - `reclass(class, data)` changes the class; the data is optional, and if given, it is given whole. The row leaves its
812
- previous class, so it needs the `DELETE` permission on it, and `WRITE` on the new class and on every row whose
813
- references the move rewrites, at any depth. If the new class has no key, the id is kept (as for `vip` above); if
814
- it has one, the id is computed anew and the references are moved.
815
- The roles of referencing rows must accept the new class, otherwise `reclass_denied`. A raw `UPDATE … SET class`
816
- passes only if neither the old nor the new class has a key (otherwise `use_reclass`).
817
- - Deleting, changing the class and changing the key work only in `read committed`: library transactions always open at
818
- this level.
819
-
820
- ### Anonymize: `anonymize`
821
-
822
- <!-- run: guide -->
823
- ```ts
824
- await db.Customer(olga).anonymize(['email']).rows() // email → '[erased]', tag anonymized
825
- ```
826
-
827
- `anonymize(fields)` overwrites string fields with a new version and marks the row with the `anonymized` tag. The
828
- earlier versions stay in the journal. For complete erasure, the next steps are history trimming
829
- ([section 11](#11-history-and-point-in-time-reads)), then deletion and `purge` ([section 7](#7-deleting-and-restoring)).
830
-
831
- ### How a plan runs
832
-
833
- One plan can create several related objects. The chain after a verb continues from the verb's result: after `create`
834
- and `upsert`, from the created row; after other verbs, from all rows.
835
-
836
- <!-- run: guide -->
837
- ```ts
838
- const [l1] = await db.Customer(olga).Order()
839
- .create({ number: 'A-1' }).Shop.set(shop) // order: Customer from the path, Shop via a slot
840
- .line().create({ qty: 2, price: 2000 }).Product.set(kettle) // order line: Order from the path
841
- .rows() // rows of the last step: the order line
842
- const order = l1.links.Order as string
843
- await db.Order(order).line().create({ qty: 1, price: 900 }).Product.set(sencha).rows()
844
- ```
845
-
846
- - The whole plan is one `read committed` transaction. On a deadlock (`40P01`), a serialization failure (`40001`) and a
847
- journal conflict, the library retries the whole plan, up to three times.
848
- - An error in any part rolls back the whole plan. `exists` and `conflict` are not retried: they are answers for the
849
- application.
850
- - Modifiers before a verb (`limit`, `offset`, `sort`) narrow its targets.
851
- - A modifier after a verb applies to what follows it: a class step, `versions()`, `diff()`, `blame()` and aggregates,
852
- or the targets of the next verb. If a `rows`, `first`, `ids`, `count` or `paths` terminal follows right away, there is
853
- nothing left to narrow — the verb has gone through all targets of the step — so
854
- `db.Product().update(…).limit(1).rows()` gives `invalid_query` and writes nothing. In a batch, `run()` refuses the
855
- same way.
856
- - A verb right after a verb writes to the same entities.
857
- - The result rows of a verb come in the order of its targets: by `sort()` before the verb, otherwise by id; `first()`
858
- after a verb is the first one in that order.
859
-
860
- ## 6. Reading
861
-
862
- ### Steps and hops
863
-
864
- A chain starts with a class: `db.Shop()` is all shops, `db.Shop(shop)` is one shop. The next step is a class, an alias
865
- or an end role. A hop goes along ends: a role step goes along its own end; other steps go along the ends of all classes
866
- of both families. From an object to the rows that reference it, the hop goes backward; to the target of a reference,
867
- forward; to the same class, to the children.
868
-
869
- <!-- run: guide -->
870
- ```ts
871
- const products = await db.Shop(shop).Product().rows() // products of the shop: backward along the Shop end
872
- const shopOf = await db.Product(kettle).Shop().first() // the shop of the product: forward
873
- const inOrder = await db.Order(order).line().Product().rows() // products of the order: hub → link → hub
874
- const parent = await db.Category(green).Parent().first() // role step: the parent category
875
- const allCustomers = await db.Customer().rows() // together with descendants (Vip)
876
- const onlyCustomer = await db.Customer().exact().rows() // only the class itself
877
- ```
878
-
879
- ### Filters
880
-
881
- A step filter is an id, a list of ids, a result row, an object of fields or `or(…)`. An object of fields can have any
882
- depth. A plain value means equality through containment in `data`, without type casting: `{ price: 900 }` will not find
883
- a record with the price `"900"`. A list as a field value means "the array contains all the elements". Operators cast
884
- their value to the field type from the JSON Schema. An end role in a filter takes a target id or `null`.
885
-
886
- <!-- run: guide -->
887
- ```ts
888
- import { lt, ilike, between, or, not } from 'letopis'
889
-
890
- const cheap = await db.Product({ price: lt(1000) }).rows()
891
- const teaCount = await db.Product({ title: ilike('%tea%'), price: between(500, 3000) }).count()
892
- const twoIds = await db.Product(or({ sku: 'K-1' }, { sku: 'T-7' })).ids()
893
- const active = await db.Order({ status: not('cancelled') }).rows()
894
- const ofShop = await db.Product({ Shop: shop.id }).rows()
895
- const roots = await db.Category({ Parent: null }).rows() // root categories
896
- ```
897
-
898
- | Operator | Condition |
899
- |---|---|
900
- | `ne(v)`, `gt(v)`, `gte(v)`, `lt(v)`, `lte(v)` | ≠ (taking `null` into account), >, ≥, <, ≤ |
901
- | `between(a, b)`, `inList([…])` | a ≤ x ≤ b; a value from the list |
902
- | `like(s)`, `ilike(s)`, `starts(s)`, `ends(s)` | a pattern with `%` and `_`; case-insensitive; starts with; ends with |
903
- | `has(v)`, `hasAny([…])`, `hasAll([…])` | an array field contains the value; at least one; all (`hasAll([])` matches every row, `hasAny([])` matches none) |
904
- | `exists(true \| false)`, `isNull()` | the field is present or not; the field is `null` or missing |
905
- | `not(v)` | negation of an operator; for a scalar, "not equal" |
906
- | `or(f1, f2, …)` | any of the step filters |
907
-
908
- Tags and the row owner are filtered with the `.tags(…)` and `.owner(…)` modifiers.
909
-
910
- ### Terminals and the result row
911
-
912
- | Terminal | What it returns |
913
- |---|---|
914
- | `rows()` | the rows of the entities of the last step, without duplicates |
915
- | `first()` | the first row or `null` |
916
- | `ids()` | the ids of the entities of the last step |
917
- | `count()` | the number of entities of the last step; `count({ paths: true })` is the number of paths |
918
- | `paths()` | paths: `[{ Step: Row, … }]`, a node for each step |
919
- | `versions()` | all versions of the entities of the last step ([section 11](#11-history-and-point-in-time-reads)) |
920
- | `sum`, `avg`, `min`, `max`, `countBy` | aggregates over the field `'data.<path>'`; on an empty set, `null`, and `{}` for `countBy` |
921
-
922
- Result row: `id`, `class`, `rev` (version number), `tenant`, `owner`, `links` (ends), `data` (fields), `tags`, `at`
923
- (version time), `author`, `op` (kind of operation) and, if present, `agent`, `reason`, `moved`. Service fields start
924
- with `$`: `$deleted`, `$depth`, `$upsert`, `$action`, `$versions`, `$purged`.
925
-
926
- ### Sorting and pagination
927
-
928
- <!-- run: guide -->
929
- ```ts
930
- import { cursorOf } from 'letopis'
931
-
932
- const page1 = await db.Product().sort('data.price', 'desc').limit(2).rows()
933
- const page2 = await db.Product().sort('data.price', 'desc')
934
- .after(cursorOf(page1.at(-1)!, 'data.price')).limit(2).rows() // strictly after the last row of the page
935
- ```
936
-
937
- `sort(field, 'asc' | 'desc')` sorts by `'at'`, `'rev'`, `'id'` or `'data.<path>'`. `after(cursorOf(row, field))` is
938
- pagination "after this row"; it requires `sort()`. Such pagination withstands inserts and deletes between pages, but a
939
- row whose sort field has changed may drop out or repeat. A cursor is an object `{ v, id, rev }` (the field value, the
940
- row id and its version number): it survives JSON, so you can hand it to a client. The field in `cursorOf` must match the `sort` field. The cursor
941
- value is checked against the type of the `sort` field before the query, and a wrong one is `invalid_query`: a numeric
942
- field needs a finite number, `rev` an integer, a boolean field `true` or `false`, a text field a string, uuid fields
943
- and `id` a uuid, time fields and `at` an ISO string with a time zone, a date field `YYYY-MM-DD`, a field without a
944
- description a string, a number or `true`/`false`. `null` fits only data fields, and the cursor's `id` is always a uuid.
945
- The check cannot tell apart a cursor of the same type taken from another field. `limit` and `offset` are available
946
- too.
947
-
948
- `after()` also pages `versions()`: the cursor of a version carries its `rev`, so versions of one object with an equal
949
- sort field are not lost. In `paths()` the cursor comes from `cursorOf(path, field)`: the field and the id are those of
950
- the last node of the path, and `path` holds the ids of all nodes, so paths to the same node are not lost. A cursor
951
- `cursorOf(path.Node, field)` taken from a single node skips the other paths to that node. `count()`, aggregates,
952
- `diff()` and `blame()` ignore the cursor.
953
-
954
- A data field may be `null` or missing. `sort` places such rows the way `order by` in PostgreSQL does: last when
955
- ascending, first when descending, and `after()` treats `null` as a value after all others (ascending) or before them
956
- (descending). So pages go through such rows without gaps or repeats, and a cursor with the value `null` gives the next
957
- page.
958
-
959
- ### Trees: `deep`
960
-
961
- <!-- run: guide -->
962
- ```ts
963
- const subtree = await db.Category(tea).Category().deep().rows() // subcategories of any depth; $depth is the level
964
- ```
965
-
966
- `deep(max)` repeats the hop to the same class up to the depth `max` (32 levels by default). The filter, tags and owner
967
- of the step select nodes at any level but do not cut the traversal. Only a node invisible by permissions cuts it: such a
968
- node also hides its subtree.
969
-
970
- ### Aggregates
971
-
972
- <!-- run: guide -->
973
- ```ts
974
- const total = await db.Order(order).line().sum('data.price') // sum of the order lines
975
- const avgPrice = await db.Shop(shop).Product().avg('data.price')
976
- const byStatus = await db.Order().countBy('data.status') // { new: 1 }
977
- ```
978
-
979
- The database computes aggregates over the entities of the last step. `limit(n)` and `offset(n)` before `countBy()`
980
- apply to the groups: `db.Order().limit(3).countBy('data.status')` gives the three most frequent values. For a boolean
981
- field, `min()` is `false` if there is at least one `false` (whether all values are `true`), and `max()` is `true` if
982
- there is at least one `true`; `sum()` and `avg()` do not work on a boolean field — PostgreSQL returns an error.
983
-
984
- ### Paths
985
-
986
- <!-- run: guide -->
987
- ```ts
988
- const orderPaths = await db.Order(order).line().Product().paths() // [{ Order: Row, line: Row, Product: Row }, …]
989
- ```
990
-
991
- `paths()` returns the nodes of every variant of the path; the key of a node is the step name or the name from
992
- `.alias(name)`. The keys follow the order of the steps; if the chain ends by returning to an earlier node, that node
993
- comes last. Repeating a link in the chain returns to its node: this way you can go from one link in several
994
- directions. `db.entity(row | chain)` inserts a ready node into the path.
995
-
996
- ## 7. Deleting and restoring
997
-
998
- ### Preview and deletion
999
-
1000
- <!-- run: guide -->
1001
- ```ts
1002
- const preview = await db.Order(order).delete().rows() // deletes nothing: the order and its lines, $action of each
1003
- await db.Order(order).delete({ confirm: true }).rows() // deleted: tombstones with $deleted: true
1004
-
1005
- const blockers = (await db.Shop(shop).delete().rows()).filter((x) => x.$action === 'restrict')
1006
- try {
1007
- await db.Shop(shop).delete({ confirm: true }).rows() // products block the shop: the Shop end is restrict
1008
- } catch (e) {
1009
- if (!(e instanceof LetopisError) || e.code !== 'delete_restricted') throw e
1010
- }
1011
- ```
1012
-
1013
- `delete()` without `{ confirm: true }` is only a preview: the whole cascade closure. Every preview row has `$action`
1014
- (`delete`, `unset` or `restrict`) and `$depth`, the cascade level (0 is the target itself; `unset` and `restrict` rows
1015
- do not have it). With the confirmation, all targets of the step are deleted in one statement, and the cascade applies
1016
- the rules of the ends. The result is the tombstones level by level, by id within a level (the journal has them in the
1017
- same order); targets of classes without a journal (`history: false`) come last. The preview does not take permissions
1018
- into account: a row with `$action: 'delete'` may fail to be deleted for a user without the permission (`acl_denied`),
1019
- so a user interface should ask `acl.checkData(class, 'DELETE')`. The rules of the ends:
1020
-
1021
- - `restrict`: deletion is forbidden while the target is referenced (`delete_restricted` with a list of the visible rows
1022
- and the number of the others);
1023
- - `cascade`: the referencing rows are deleted together with the target (the order lines above);
1024
- - `unset`: the reference is removed.
1025
-
1026
- The cascade requires delete permission on every deleted row, and removing a reference requires write permission. A raw
1027
- `DELETE … WHERE id IN (…)` of any set of related rows works the same way.
1028
-
1029
- ### Restore: `restore`
1030
-
1031
- <!-- run: guide -->
1032
- ```ts
1033
- await db.Order(order).restore().rows() // the order, from its last snapshot, with the same id
1034
- await db.Order(order).line().withDeleted().restore().rows() // and its lines deleted by the cascade
1035
- ```
1036
-
1037
- `restore()` brings back the last snapshot of a deleted object as a new version and checks it again against the current
1038
- class description. Only the targets of the step are restored, not the whole cascade. A live object gives `not_deleted`.
1039
-
1040
- ### Erase history: `purge`
1041
-
1042
- <!-- run: guide -->
1043
- ```ts
1044
- const [temp] = await db.Customer().create({ name: 'Test', email: 'test@example.com' }).rows()
1045
- await db.Customer(temp).delete({ confirm: true }).rows()
1046
- const before = await db.Customer(temp).purge().rows() // preview: $versions is how many versions the journal has
1047
- await db.Customer(temp).purge({ confirm: true }).rows() // history erased; the journal has a purge record
1048
- ```
1049
-
1050
- `purge` works only for deleted objects (a live one gives `not_deleted`) and erases their whole history; in place of
1051
- version 1, a `purge` record stays with the number of erased versions. The history of system classes is not erased.
1052
- While a projector whose `from` sources include the class of the object ([“Projectors”](#projectors)) has not read
1053
- the deletion, `purge` refuses (`projector_pending`): the projector would never see an erased tombstone, and the
1054
- object's contribution would stay in the totals. So a deletion and a `purge` in one transaction do not go through in
1055
- such a class. Likewise, `db.trimHistory` refuses while the projector has not read the first remaining version: it
1056
- would no longer find the previous version for `before`. To delete an account, use `db.auth.purgeAccount` ([section 10](#10-users-sign-in-and-permissions)).
1057
-
1058
- ## 8. Transactions, batches and money
1059
-
1060
- ### Every operation is a transaction
1061
-
1062
- Every chain terminal is its own `read committed` transaction, whatever the default level in the database is. At this
1063
- level, increments and the cascade see fresh rows. The library retries deadlocks, serialization failures and journal
1064
- conflicts by itself. If the connection drops during `COMMIT`, it does not retry and throws `commit_unknown`: the
1065
- transaction may have committed. A repeat of the same operation with the same key finds out the outcome: `exists` means
1066
- "already written".
1067
-
1068
- ### Explicit transaction: `db.begin()`
1069
-
1070
- <!-- run: guide -->
1071
- ```ts
1072
- const tr = await db.begin()
1073
- try {
1074
- const [o2] = await tr.Customer(olga).Order().create({ number: 'A-2' }).Shop.set(shop).rows()
1075
- await tr.Order(o2).line().create({ qty: 1, price: 2000 }).Product.set(kettle).rows()
1076
- await tr.commit()
1077
- } catch (e) {
1078
- await tr.rollback()
1079
- throw e
1080
- }
1081
- ```
1082
-
1083
- - Inside, you have the same chains, `db.sql`, `forUpdate()`, `lock()`, `commit()` and `rollback()`.
1084
- - An error in any query breaks the transaction: every next call and `commit()` throw the original error, and `commit()`
1085
- rolls back. `rollback()` goes through. If the error was caught inside `tr.transaction(fn)` and the library does not
1086
- know about it, PostgreSQL silently answers `ROLLBACK` to `COMMIT`, and the library turns this into the
1087
- `tx_rolled_back` error.
1088
- - There are no retries inside `db.begin()`: on `40P01`, repeat the whole block.
1089
- - `lock(...keys)` is an advisory lock until the end of the transaction. It protects only if everyone who edits this data
1090
- takes it.
1091
-
1092
- ### Batch
1093
-
1094
- <!-- run: guide -->
1095
- ```ts
1096
- const b = db.batch('storefront')
1097
- b.Shop(shop).Product().create({ sku: 'C-1', title: 'Cup', price: 300 })
1098
- b.Shop(shop).Product().create({ sku: 'C-2', title: 'Saucer', price: 200 })
1099
- const results = await b.run() // Row[][]: an array of rows for each plan, in one transaction
1100
- ```
1101
-
1102
- A batch collects plans, and `run()` executes them in one transaction. Consecutive simple `create` and `upsert` calls of
1103
- a class with a key are merged into one multi-row statement. `b.size()` is the number of queued plans, `b.discard()`
1104
- clears the queue. A terminal on a queued plan gives `invalid_query`: only `run()` executes. The name is the key of a
1105
- queue of the facade: the root connection, each `db.as()` and each `db.begin()` have their own queues. A plan runs on
1106
- behalf of the facade that queued it (actor, tenant, owner, transaction), and `run()` never executes plans of other
1107
- facades. Collect plans through one and the same facade object.
1108
-
1109
- ### Recipe: wallet
1110
-
1111
- The balance is the `balance` field of the `Wallet` hub (integer cents, `min(0)`); operations are the `Movement` link
1112
- with the key `[Wallet, opId]` ([section 4](#4-data-model)). An operation is never edited or deleted; it is cancelled by
1113
- a reverse operation with a new `opId`. The key makes an operation idempotent: a repeat after a lost response gets
1114
- `exists`, and the recipe checks whether it is the same request.
1115
-
1116
- <!-- run: guide -->
1117
- ```ts
1118
- import { inc, type Db } from 'letopis'
1119
-
1120
- type Kind = 'topup' | 'debit' | 'adjust'
1121
-
1122
- /** Operation: write a Movement and increment the balance — one plan, one transaction. */
1123
- async function operate(db: Db, wallet: string, opId: string, kind: 'topup' | 'debit', amount: number) {
1124
- try {
1125
- await db.Wallet(wallet).Movement().create({ opId, kind, amount })
1126
- .Wallet().update({ balance: inc(kind === 'topup' ? amount : -amount) }).rows()
1127
- return 'done'
1128
- } catch (e) {
1129
- if (e instanceof LetopisError && e.code === 'exists') return same(db, wallet, opId, kind, amount) // repeat
1130
- throw e // invalid_data: not enough funds (min(0))
1131
- }
1132
- }
1133
-
1134
- /** Repeat fingerprint: is the request already written under this opId the same request? */
1135
- async function same(db: Db, wallet: string, opId: string, kind: Kind, value: number) {
1136
- const m = await db.Movement(db.idOf('Movement', { Wallet: wallet, opId })).first()
1137
- if (!m || m.data.kind !== kind || (kind === 'adjust' ? m.data.target : m.data.amount) !== value)
1138
- throw new Error(`operation key ${opId} is already used by another request`)
1139
- return 'repeat'
1140
- }
1141
-
1142
- /** Adjustment "balance = target": an adjust operation for the difference, the total under { rev }. */
1143
- async function adjust(db: Db, wallet: string, opId: string, target: number) {
1144
- for (;;) {
1145
- if (await db.Movement(db.idOf('Movement', { Wallet: wallet, opId })).first()) return same(db, wallet, opId, 'adjust', target)
1146
- const cur = (await db.Wallet(wallet).first())!
1147
- try {
1148
- await db.Wallet(wallet).Movement().create({ opId, kind: 'adjust', amount: target - (cur.data.balance as number), target })
1149
- .Wallet().update({ balance: target }, { rev: cur.rev }).rows()
1150
- return 'done'
1151
- } catch (e) {
1152
- if (e instanceof LetopisError && e.code === 'conflict') continue // the wallet changed: re-read
1153
- if (e instanceof LetopisError && e.code === 'exists') return same(db, wallet, opId, 'adjust', target)
1154
- throw e
1155
- }
1156
- }
1157
- }
1158
-
1159
- const [wallet] = await db.Customer(olga).Wallet().create({ number: 'W-1' }).rows()
1160
- await operate(db, wallet.id, 'op-1', 'topup', 10_000) // 'done': balance 10,000
1161
- await operate(db, wallet.id, 'op-1', 'topup', 10_000) // 'repeat': a repeat after a lost response, the balance is the same
1162
- await operate(db, wallet.id, 'op-2', 'debit', 3_000) // balance 7,000
1163
- await adjust(db, wallet.id, 'op-3', 5_000) // adjustment: adjust by −2,000, balance 5,000
1164
- ```
1165
-
1166
- The same result can be written under a locking read, without retries on `conflict`:
1167
-
1168
- <!-- run: guide -->
1169
- ```ts
1170
- const [wallet2] = await db.Customer(olga).Wallet().create({ number: 'W-2' }).rows()
1171
- const tx = await db.begin()
1172
- try {
1173
- const w = (await tx.Wallet(wallet2).forUpdate().first())! // the wallet row is locked until commit
1174
- await tx.Wallet(wallet2).Movement()
1175
- .create({ opId: 'op-1', kind: 'adjust', amount: 6_000 - (w.data.balance as number), target: 6_000 })
1176
- .Wallet().update({ balance: 6_000 }).rows()
1177
- await tx.commit()
1178
- } catch (e) {
1179
- await tx.rollback()
1180
- throw e
1181
- }
1182
- ```
1183
-
1184
- - `inc` adds inside the statement, over the locked row: concurrent debits are not lost, and `min(0)` rejects a debit
1185
- beyond the balance.
1186
- - A plain `update({ balance: 700 })` writes the value; it does not add. A total computed from what was read, without
1187
- `{ rev }` or `forUpdate()`, loses concurrent debits. An advisory `lock()` does not help if not all writers take it.
1188
- `upsert` overwrites increments.
1189
- - `forUpdate()` works only inside `db.begin()` (otherwise `tx_required`) and only with `rows`, `first`, `ids`
1190
- (otherwise `lock_unsupported`). Rows are locked in id order; a row that no longer matches the conditions of the step
1191
- is not included in the result.
1192
- - Choose one path for one wallet: under `{ rev }` and under `forUpdate()`, the wallet and the operation key are locked
1193
- in a different order, and mixing the two in concurrent requests gives `40P01`. Inside a block with `forUpdate()`, do
1194
- not wait for external calls: all writers wait for the wallet (`lock_timeout` and
1195
- `idle_in_transaction_session_timeout` help here).
1196
- - A transfer is two operations in one `db.begin()`, with the wallets edited in id order: then there are no deadlocks.
1197
- - Insufficient funds are told apart from other data errors by `e.issues`: the field `balance`, the rule `minimum`.
1198
-
1199
- ### Recipe: balance as an aggregate
1200
-
1201
- The same wallet, but the database itself keeps the balance: `balance` is an aggregate, the sum of `amount` over the
1202
- live operations ([Aggregates in the database](#aggregates-in-the-database)). An operation is one `Movement` record;
1203
- there is no `inc` increment in the code, so it cannot be forgotten. The balance stays correct even when raw SQL or the
1204
- loader writes the operations and a cascade deletes them.
1205
-
1206
- ```ts
1207
- const Wallet = hub('Wallet', { number: z.string(), balance: z.number().int().min(0) }, {
1208
- ends: { Customer }, key: ['number'],
1209
- aggregate: { balance: { sum: 'amount', of: 'Movement', via: 'Wallet' } }, // the balance is the sum of live operations
1210
- })
1211
- const Movement = link('Movement', { opId: z.string(), amount: z.number().int() },
1212
- { ends: { Wallet }, key: ['Wallet', 'opId'] }) // the operation key gives idempotency
1213
-
1214
- /** Operation: an amount above zero tops up, below zero debits; the database recalculates the balance in the same transaction. */
1215
- async function operate(db: Db, wallet: string, opId: string, amount: number) {
1216
- try {
1217
- await db.Wallet(wallet).Movement().create({ opId, amount }).rows()
1218
- return 'done'
1219
- } catch (e) {
1220
- if (e instanceof LetopisError && e.code === 'exists') return 'repeat' // a repeat after a lost response
1221
- throw e // invalid_data: the balance would go below zero (min(0))
1222
- }
1223
- }
1224
-
1225
- /** Adjustment "balance = target": an operation for the difference, with the wallet locked until commit. */
1226
- async function adjust(db: Db, wallet: string, opId: string, target: number) {
1227
- const tx = await db.begin()
1228
- try {
1229
- const w = (await tx.Wallet(wallet).forUpdate().first())!
1230
- await tx.Wallet(wallet).Movement().create({ opId, amount: target - (w.data.balance as number) }).rows()
1231
- await tx.commit()
1232
- return 'done'
1233
- } catch (e) {
1234
- await tx.rollback()
1235
- if (e instanceof LetopisError && e.code === 'exists') return 'repeat' // an adjustment with this opId is already written
1236
- throw e
1237
- }
1238
- }
1239
-
1240
- const [wallet] = await db.Customer(olga).Wallet().create({ number: 'W-1' }).rows() // balance is 0: the database sets it
1241
- await operate(db, wallet.id, 'op-1', 10_000) // balance 10,000
1242
- await operate(db, wallet.id, 'op-2', -3_000) // balance 7,000
1243
- await adjust(db, wallet.id, 'op-3', 5_000) // an operation of −2,000, balance 5,000
1244
- ```
1245
-
1246
- - The model here is its own, and the block is not part of the example program: there the balance grows through `inc`,
1247
- and an aggregate field does not change that way.
1248
- - `balance` is passed neither to `create` nor to `update` (`aggregate_field`), so an adjustment is an operation too,
1249
- not an `update` under `{ rev }`. Writing an operation needs no permission to write the wallet: the permission to
1250
- write `Movement` is enough.
1251
- - Insufficient funds are `invalid_data`, as with `inc`: `e.issues` holds the field `balance` and the rule `minimum`.
1252
- Check a repeat as `same()` in the recipe above does: `exists` only says that an operation with this `opId` already
1253
- exists.
1254
- - Editing and deleting an operation change the balance too: the database subtracts the previous value. It is still
1255
- better to cancel an operation with a reverse one, so that the history of the wallet stays complete.
1256
- - Every operation locks the wallet row and writes its version, so the operations of one wallet go one after another,
1257
- as with `inc`: about 150 per second on the benchmark machine ([section 17](#17-performance)). If one object gets
1258
- more, compute the total with a projector instead ([Recipe: daily revenue](#recipe-daily-revenue)).
1259
-
1260
- ## 9. Bulk loading
1261
-
1262
- ### When you need the loader
1263
-
1264
- The loader is the path for import, seeding and syncing large reference tables. It checks rows in the library code
1265
- with the same validator as the database, and writes them with the `COPY` command as the owner role, without triggers
1266
- and RLS. In the benchmark, this is about 6,400 rows per second against 130 for row-by-row `create`
1267
- ([section 17](#17-performance)). It works only on an admin connection (`allowBypassRls` and the right to become the
1268
- owner), otherwise you get `admin_required`.
1269
-
1270
- ### `db.load()`
1271
-
1272
- <!-- run: guide -->
1273
- ```ts
1274
- const rep = await adm.load({
1275
- Shop: [{ name: 'Cornflower', city: 'Perm' }],
1276
- Product: [
1277
- { sku: 'V-1', title: 'Mug', price: 450, Shop: { $key: ['Cornflower'] } },
1278
- { sku: 'V-2', title: 'Spoon', price: 150, Shop: { $key: ['Cornflower'] }, $tags: ['new'] },
1279
- ],
1280
- Customer: [{ name: 'Petr', email: 'petr@mail.ru', $ext: 'crm:1001' }],
1281
- })
1282
- // rep: { rows: 4, loaded: 4, updated: 0, skipped: 0, duplicates: 0, byClass: { Shop: 1, Product: 2, Customer: 1 }, … }
1283
- ```
1284
-
1285
- - The input is an object "class → rows", an array, or a stream of rows with a `$class` field.
1286
- - A row has its fields and end roles at the top level. Service keys: `$class` is the class of the row, `$ext` is an
1287
- external key of a class without a key (loading the same file again gives the same ids), `$tags` are the tags,
1288
- `$owner` is the owner.
1289
- - A reference is a target id, a key `{ $key: [...] }` or an external key `{ $ext: '…' }`; for a polymorphic end, add
1290
- the `$class` of the target. The order of rows and classes does not matter, and circular references are allowed.
1291
- - Keys that start with `$` inside the data are ordinary data.
1292
- - Rows go to the session tenant (or to `tenant`). A load is one transaction: an error in any row cancels everything.
1293
- Subscriptions see the rows after the commit.
1294
-
1295
- ### Repeats and existing objects
1296
-
1297
- <!-- run: guide -->
1298
- ```ts
1299
- const again = await adm.load({ Shop: [{ name: 'Cornflower', city: 'Perm' }] }, { existing: 'skip' }) // skipped: 1
1300
- const changed = await adm.load({ Shop: [{ name: 'Cornflower', city: 'Samara' }] }, { existing: 'update' }) // updated: 1
1301
- ```
1302
-
1303
- | Option | Values |
1304
- |---|---|
1305
- | `duplicates`: a repeated id within the load | `'error'` (default): cancel; `'first'` or `'last'`: keep the first or the last row |
1306
- | `existing`: the object already exists in the database | `'error'` (default): cancel; `'skip'`: skip, do not resurrect deleted objects; `'update'`: a new version if the content differs (unchanged objects are counted in `skipped`), deleted objects are created again; an object of another class with the same id (a class without a key after `reclass`) is `exists` |
1307
-
1308
- ### Portions, history and verification
1309
-
1310
- - `commitEvery: N` commits in portions of about N rows. Portions go by class in dependency order; the classes of one
1311
- cycle go in one portion. An error cancels only the current portion, and a repeat with `existing: 'skip'` writes the
1312
- rest.
1313
- - `withHistory: true`: the rows are versions with `$rev`, `$at` and `$op`; this is how the import carries history over.
1314
- - `verify: true`: at the end, check the loaded rows again with the database check.
1315
- - `batch` (rows per `COPY` batch, 10,000), `class` (the class of rows without `$class`), `tenant`, `agent` (the agent of
1316
- the versions).
1317
-
1318
- ### Loading on behalf of a user
1319
-
1320
- <!-- run: guide -->
1321
- ```ts
1322
- await adm.load({ Category: [{ name: 'Tableware' }] }, { as: ANNA, tenant: SHOP })
1323
- ```
1324
-
1325
- With `as`, the user becomes the author of the versions, and the loader checks the same things as the database does on a
1326
- regular write: the tenant is available to the user, write permission on each class, the permission conditions on rows,
1327
- the visibility of reference targets, the owner per `ownerDefault`. Without `tenant`, the rows go to the user's own
1328
- tenant. This is how a loading process can accept files sent by users.
1329
-
1330
- ### Initial load
1331
-
1332
- ```ts
1333
- const r = await adm.load(rows, { initial: true }) // empty schema: one transaction, indexes and triggers at the end
1334
- ```
1335
-
1336
- `db.load(rows, { initial: true })` (`letopis load --initial`) loads data into an empty schema faster than the regular
1337
- loader — about 2.2 times on a million objects: everything goes in one transaction, the tables are closed for reading and
1338
- writing during the load, indexes, constraints and triggers are dropped and rebuilt at the end, and rows are written with
1339
- `COPY … FREEZE`. Empty means that the schema has nothing besides the rows of system classes (class descriptions,
1340
- accounts, memberships, resources, rules) and `reset` records; otherwise you get `not_empty` before any change. The mode
1341
- is incompatible with `commitEvery`, `existing` (other than `'error'`) and `{ as }`; an error in any row rolls
1342
- everything back, indexes and triggers included. `letopis import --initial` runs the whole import this way: in one
1343
- transaction, without `--update` and `--commit-every`. WAL is written here too (it is skipped only with
1344
- `wal_level = minimal`), but the indexes are built once, so there is almost three times less of it; after `FREEZE`, the
1345
- first read and the first vacuum do not rewrite the table.
1346
-
1347
- ### Command line
1348
-
1349
- ```sh
1350
- npx letopis load shops.ndjson products.ndjson --dsn "$ADMIN" --schema v2.shop --token "$TOKEN" \
1351
- [--class Product] [--duplicates first] [--existing skip] [--commit-every 100000] [--verify]
1352
- # shops.ndjson: {"$class":"Shop","name":"South"}
1353
- # products.ndjson: {"$class":"Product","sku":"Y-1","title":"Vase","price":800,"Shop":{"$key":["South"]}}
1354
- ```
1355
-
1356
- NDJSON files have one object per line; all files form one load.
1357
-
1358
- ### Report and errors
1359
-
1360
- Report: `rows`, `loaded`, `updated`, `skipped`, `duplicates`, `byClass`, `external` (how many targets outside the load
1361
- were checked by a query at the end), `portions`, `ms`. An error is a `LetopisError` with the code of the first
1362
- problem. Its `detail`: `rows` is the first 100 violations with the row number, code and details (`line`, `code`,
1363
- `message`, `id`, `class`, `issues`), `total` is how many violations were found, `truncated` means the list is cut,
1364
- `limit` is the list limit (100), `complete` means the checks that found violations went through all rows. Row codes:
1365
- the codes of the database checks, and also `duplicate` (a repeated id), `exists` (the object already exists), `deleted`
1366
- (an object with this id existed before) and `class_not_loadable` (the loader does not accept `Class` rows: classes are
1367
- set by `syncModels`). The loader rejects rows of derived classes and references to them with the code
1368
- `derived_class`: only a projector writes such rows.
1369
-
1370
- Violations are searched for in stages: parsing the rows (classes, ids, repeats), permissions on classes
1371
- (`load({ as })`), data and ends in batches, and at the end of a portion the targets outside the load, owners and the
1372
- tenant. The load is cancelled after the first stage that found violations, and `total` counts the violations of that
1373
- stage. A target outside the load is one violation per target (the row is the first one that references it).
1374
- "(и ещё N)" ("and N more") in the error text is the exact number of the remaining violations. If not all rows were
1375
- checked, `complete` is `false`, and the text says "(и ещё не меньше N: причины)" ("at least N more: reasons") or, if
1376
- one violation was found, "(возможно, не единственная: причины)" ("possibly not the only one: reasons"). The reasons:
1377
- the following `commitEvery` portions were not checked; the ACL conditions of the following batches were not checked
1378
- (`load({ as })`); the database check (`verify: true`) names at most 100 rows.
1379
-
1380
- ## 10. Users, sign-in and permissions
1381
-
1382
- ### Accounts, tenants and memberships
1383
-
1384
- An account is a row of class `Account` in System: a name, categories (`User`, `Service` and your own) and an `enabled`
1385
- flag. Every account is its own tenant. A `member` membership allows work in another tenant: it is a link from a user
1386
- (`User`) to a tenant (`Tenant`) with roles in its data. Accounts and their categories are edited by the system
1387
- administrator, a member of System with the `admin` role. There is no separate method to appoint one: a membership in
1388
- System with the `admin` role is put in place by your own installation seed or by the loader on an admin connection (a
1389
- `member` row with `tenant: <System id>`). New users of a tenant are easy to create with `createTenant`
1390
- ([section 3](#3-installation-and-connection)).
1391
-
1392
- ### Service and API keys
1393
-
1394
- A service is an account of category `Service` with an API key. `up()` issues the installation key. The service checks
1395
- passwords and codes, issues sessions and acts on behalf of users. It issues and revokes the keys of regular users
1396
- itself (the `auth.credential` permission), as for Boris below. The service also issues and revokes its own API keys
1397
- and "key + secret" pairs itself, with a session opened by its own key (`connect({ apiKey })`, `verifyApiKey`,
1398
- `verifyKeySecret`). Its other credentials and the keys of other services, of System and of system administrators are
1399
- issued and revoked only by a system administrator; a new service key is also registered by `up({ serviceKey })`.
1400
-
1401
- There can be several keys. A session opened by a key remembers it, and revoking a key revokes only its sessions:
1402
- connections by other keys keep working. So rotation goes without downtime: the service issues itself a new key
1403
- (`issueApiKey({ account: <service id> })`), moves its processes to it and revokes the old one (`revokeCredential(id)`).
1404
- The revocation keeps the current session; `keepCurrent: false` revokes it too. The last active key of a service
1405
- cannot be revoked: `acl_denied` "последний действующий ключ сервиса не отзывается: сначала выпустите новый" ("the
1406
- last active key of a service is not revoked: issue a new one first"). To disable a service, set `enabled: false` on
1407
- its account.
1408
-
1409
- <!-- run: guide -->
1410
- ```ts
1411
- const { key, id: keyId } = await svc.auth.issueApiKey({ account: BORIS, name: 'integration' }) // the key is returned only once
1412
- const viaKey = await svc.auth.verifyApiKey(key) // { account, token } | null
1413
- await svc.auth.revokeCredential(keyId)
1414
- ```
1415
-
1416
- A "key + secret" pair: `issueKeySecret({ account, name })` and `verifyKeySecret(key, secret)`.
1417
-
1418
- ### Passwords
1419
-
1420
- <!-- run: guide -->
1421
- ```ts
1422
- await svc.auth.setPassword({ account: BORIS, identifier: 'boris@lutik.ru', password: 'New-password-2026', confirmed: true })
1423
- const failed = await svc.auth.verifyPassword({ identifier: 'boris@lutik.ru', password: 'Buttercup-2026!' }) // null: the password was changed
1424
- ```
1425
-
1426
- - A password is stored as a scrypt hash; the service checks it, and the response time for a nonexistent login is
1427
- equalized.
1428
- - The identifier is compared case-insensitively: `Anna@Shop.ru` and `anna@shop.ru` are one login.
1429
- - A password change revokes the other sessions of the account (`keepCurrent` keeps the current one).
1430
- - By default, sign-in works only by a confirmed credential, otherwise `null`; `requireConfirmed: false` also lets in by
1431
- an unconfirmed one. The service confirms a credential: `confirmed: true` in `setPassword` or `setFlags`.
1432
- - The second factor is the `totp` option: `verifyPassword({ identifier, password, totp })` checks the account's TOTP
1433
- code after a correct password and issues a session with the methods `[kind, 'TOTP']`, `['PASSWORD', 'TOTP']` by
1434
- default (see below). Without `totp`, sign-in does not check the code: the service decides whether to require the
1435
- second factor.
1436
-
1437
- ### One-time codes and TOTP
1438
-
1439
- <!-- run: guide -->
1440
- ```ts
1441
- import { totpCode } from 'letopis'
1442
-
1443
- const { code } = await svc.auth.issueOtp({ account: BORIS, identifier: 'boris@lutik.ru', ttl: 300 }) // the service sends the code by email
1444
- const byCode = await svc.auth.verifyOtp({ identifier: 'boris@lutik.ru', code }) // { account, token } | null
1445
- const { secret, uri } = await svc.auth.enrollTotp({ account: BORIS, issuer: 'Buttercup' }) // uri is for a QR code
1446
- const totpOk = await svc.auth.verifyTotp({ account: BORIS, code: totpCode(secret) }) // the first check enables the factor
1447
- ```
1448
-
1449
- A code is single-use: a successful check and brute force (5 attempts by default) burn it. The attempt counters are
1450
- rolled back together with the caller's transaction, so the service limits the rate of attempts. A repeat of the same
1451
- TOTP step is rejected. `totpCode(secret)` is a helper for tests and services.
1452
-
1453
- Sign-in with the second factor is `verifyPassword({ …, totp })` and `lookup({ …, totp })`. The code is checked only
1454
- after a correct password or a found identity, by the same check as `verifyTotp`: the `auth.totp` permission is needed,
1455
- the window is one step each way, and the failure counter is shared (five in a row close the check for 5 minutes). A
1456
- correct code gives a session with the methods `[kind, 'TOTP']` and enables a not yet confirmed factor; a wrong or
1457
- repeated code, like a code for an account without TOTP, gives `null`. With such a session, a user with TOTP changes
1458
- their own credentials for 10 minutes.
1459
-
1460
- ### External identities
1461
-
1462
- `link({ account, kind, identifier, confirmed })` links an external identity (Telegram, Google, SSO): the service checks
1463
- its token, and the database stores the binding. `lookup({ kind, identifier })` signs in with it (with the `totp`
1464
- option, also by a TOTP code, like `verifyPassword`):
1465
-
1466
- <!-- run: guide -->
1467
- ```ts
1468
- await svc.auth.link({ account: BORIS, kind: 'TELEGRAM', identifier: '100500', confirmed: true })
1469
- const viaTelegram = await svc.auth.lookup({ kind: 'TELEGRAM', identifier: '100500' }) // { account, token } | null
1470
- ```
1471
-
1472
- A user can link their own credentials only after a recent sign-in (10 minutes; an API key, a "key + secret" pair and a
1473
- session from a service do not count), and cannot confirm them. With TOTP enabled, the database requires a session that
1474
- has passed TOTP: `verifyPassword` and `lookup` with the `totp` option issue one. If the identifier is taken, linking
1475
- answers with the same `acl_denied` as without a recent sign-in. But a user with a recent sign-in can tell from such a
1476
- denial that the identifier is taken ([section 18](#18-limitations)).
1477
-
1478
- ### Sessions
1479
-
1480
- <!-- run: guide -->
1481
- ```ts
1482
- const shortToken = await svc.auth.sessionFor(BORIS, { ttl: 600 }) // a session for 10 minutes of idle time for another process
1483
- const whose = await svc.auth.session(shortToken) // { account, tenant, member, expiresAt, loginAt, methods }
1484
- const boris = await connect({ dsn: APP, schema: r.schema, token: shortToken })
1485
- await boris.auth.revoke() // sign-out: the session is revoked
1486
- const ended = await svc.auth.session(shortToken) // null
1487
- await boris.close()
1488
- ```
1489
-
1490
- A token is 256 random bits; the database stores only its sha256. The default lifetime is 7 days. `refresh()` extends
1491
- the current session: the new lifetime is counted from the current moment, but it is not longer than the lifetime issued
1492
- at sign-in; it returns the new expiration time. `connect()` extends the session itself once per minute of work
1493
- (`refreshSession`), so a session in use does not expire: the lifetime is the idle time. `revoke()` signs out,
1494
- `revokeAll()` signs out everywhere. Sessions are revoked when the account is disabled, the password is changed, a credential is deleted or the
1495
- account is deleted. Deleting a credential revokes the other sessions of the account, while revoking an API key or a
1496
- "key + secret" pair revokes only the sessions opened by that key (and old sessions of the same sign-in method without
1497
- a key mark); the current session stays unless you pass `keepCurrent: false`.
1498
-
1499
- A service that holds a user's token (for example, in a cookie) learns about the session with `svc.auth.session(token)`:
1500
- no connection is opened with that token, and the session is not extended. The answer tells whose session it is
1501
- (`account`), which tenant it works in (`tenant`: the one chosen by `auth.switch()` or the account's own tenant),
1502
- whether the membership there is intact (`member`; always `true` in the own tenant), when it expires (`expiresAt`), and
1503
- when and how the sign-in happened (`loginAt`, `methods`: `PASSWORD`, `APIKEY`, `SERVICE`, `TOTP` …). A wrong, revoked
1504
- or expired token and a disabled account give `null`; the reason is not told apart. This is how a web service checks the
1505
- session at the start of every request: reading under a revoked session gives no error, the answers are simply empty.
1506
- The method is called by the service or the system administrator (the `auth.session` permission); a user gets
1507
- `acl_denied`.
1508
-
1509
- ### Working in another tenant
1510
-
1511
- A service acts on behalf of a user through `svc.as(account, { tenant, owner })`. The database checks the service's
1512
- `auth.impersonate` permission (by System rules), that the user exists and is enabled, and that the tenant is available
1513
- to the user, either their own or through membership. You cannot act on behalf of System, services, system
1514
- administrators or disabled accounts. The user's visibility under the access rules is not checked. In the journal,
1515
- `author` is the user and `agent` is the service. The class registry of `db.as()` is one per tenant, and a connection
1516
- keeps at most 1000 of them: a long unused one is evicted, and a facade whose registry was evicted keeps working — the
1517
- registry is re-read. A user's connection by their own token switches between tenants like this:
1518
-
1519
- <!-- run: guide -->
1520
- ```ts
1521
- const mine = await anna.auth.tenants() // [{ tenant, roles }]: her own tenant and the shop via membership
1522
- await anna.auth.switch(SHOP) // the session tenant is now the shop
1523
- ```
1524
-
1525
- ### Permissions
1526
-
1527
- Permissions are always on and deny by default. A rule connects a group with an object:
1528
-
1529
- - **group**: a `Resource` row of category `ACCOUNT`: accounts by category (`{ categories: '{User}' }`) or members by
1530
- membership role (`{ roles: '{staff}' }`);
1531
- - **object**: a `Resource` row of category `API` (an endpoint by mask, `{ endpoint: 'shop.reports.*' }`), or of
1532
- category `READ`, `WRITE`, `DELETE` (the rows of a class by mask, with descendants, `{ class: 'Product' }`, and a
1533
- condition on the columns `id`, `owner`, `tenant`, `tags`, `data`, `links` with the values `$actor` and `$tenant`);
1534
- - **rule**: a `rule` link group → object: `permission` (`allow` or `deny`) and `weight`.
1535
-
1536
- Exactly one rule decides: the highest weight; at equal weight, deny beats allow. A tenant rule weight above 999 is
1537
- cut down to 999, so System rules with a weight of 1000 or more are an administrative layer that a tenant cannot
1538
- outweigh. Tenant rules apply to the tenant's rows, System rules apply everywhere. Permissions apply to raw SQL too: rows
1539
- without permission are invisible, and an edit changes nothing or fails with `acl_denied`.
1540
-
1541
- A class mask in a `READ`, `WRITE` or `DELETE` object matches the descendants of the class too, by the names of the
1542
- ancestors and their aliases. So an edit of a class that changes `parent`, `alias` or `derivedBy` needs the `READ`,
1543
- `WRITE` and `DELETE` permissions without conditions on all rows of the class and of each of its descendants,
1544
- otherwise `acl_denied`: the `WRITE` permission on `Class` alone is not enough to put the rows of a class under other
1545
- rules. An alias does not match the name of another class (including `Class` and the system ones) or another class's
1546
- alias in the tenant and System, and the name of a new class does not match another class's alias: otherwise a rule
1547
- written for one class would act on the rows of another too. A violation gives `invalid_class`.
1548
-
1549
- By the same names, the rules also find the old versions of rows in the journal. So a class is deleted only when it has
1550
- no rows, no descendants and no history of rows: `purge()` erases the history of deleted rows first. For a System class,
1551
- the rows, descendants and history of a tenant that has its own class with the same name do not count: they belong to
1552
- that tenant's class. A class deleted with history before is created again only with the same `parent`, `alias`,
1553
- `derivedBy` and `history` — otherwise `invalid_class`: the old versions of its rows would fall under other rules or a
1554
- different thinning. The rules also find old versions by the aliases of ancestors. So while deleted descendants of a
1555
- class keep row history, the class does not change its alias (`invalid_class`), and once deleted it is created again
1556
- only with the same description. If the journal holds rows of a class with this name but not its tombstone (for
1557
- example, rows of a System class deleted in 1.1), a class with this name is not created until `purge()` erases the
1558
- history.
1559
-
1560
- A derived class (`derivedBy`) is created, edited and deleted only by an administrator: an admin connection or an
1561
- account with the System permission `letopis.projector` (the system administrator has it through the `letopis.*`
1562
- mask). The projector writes totals over all source rows of the tenant into such a class, without read permissions,
1563
- so a user who created it on their own would read totals over rows they may not read. A class that has derived
1564
- descendants is also edited only by an administrator: editing an ancestor changes the description of the descendant
1565
- (fields, key, `history`). The database itself sets the `aggregatedBy` mark (the class became an aggregate source) on a
1566
- derived class too: it does not change its fields or key.
1567
-
1568
- ### Granting a permission
1569
-
1570
- Example: Oleg is a shop employee with the `staff` role, and employees may not delete products.
1571
-
1572
- <!-- run: guide -->
1573
- ```ts
1574
- const pg2 = postgres(ADMIN)
1575
- const OLEG = (await createTenant(pg2, r.schema, { // the same shop: only the user is created
1576
- name: 'Chamomile', user: { name: 'Oleg', login: 'oleg@shop.ru', password: 'Oleg-2026!' } })).user!
1577
- await pg2.end()
1578
- await db.member(db.idOf('member', { User: OLEG, Tenant: SHOP })).update({ roles: ['staff'] }).rows()
1579
-
1580
- const [staff] = await db.Resource().create({ alias: 'staff:ACCOUNT', category: 'ACCOUNT', pattern: { roles: '{staff}' } }).rows()
1581
- const [delProduct] = await db.Resource().create({ alias: 'product:DELETE', category: 'DELETE', pattern: { class: 'Product' } }).rows()
1582
- await db.rule().create({ permission: 'deny', weight: 50 }).Group.set(staff).Object.set(delProduct).rows() // stronger than "may do everything" (10)
1583
- const [reports] = await db.Resource().create({ alias: 'reports:API', category: 'API', pattern: { endpoint: 'shop.reports.*' } }).rows()
1584
- await db.rule().create({ permission: 'allow', weight: 50 }).Group.set(staff).Object.set(reports).rows()
1585
-
1586
- const asOleg = await svc.as(OLEG, { tenant: SHOP })
1587
- const canDelete = await asOleg.acl.checkData('Product', 'DELETE') // { allow: false }
1588
- const canRead = await asOleg.acl.checkData('Product', 'READ') // { allow: true }
1589
- const canReport = await asOleg.acl.check('shop.reports.daily') // { allow: true, rule: { weight: 50, … } }
1590
- ```
1591
-
1592
- - `acl.check(endpoint)` → `{ allow, rule, code, message }`: the `code` and `message` fields of the denying rule set the
1593
- endpoint's own denial.
1594
- - `acl.checkData(class, 'READ' | 'WRITE' | 'DELETE')` → `{ allow, filter }`: `filter` is the condition on rows.
1595
- - The database recomputes permissions by itself. To remove a permission, delete the rule:
1596
- `db.rule(id).delete({ confirm: true })`.
1597
- - The library takes the actor's signed permission plan — the permission pass — from the database once and passes it
1598
- in every transaction, so a permission check costs tens of microseconds. Raw SQL without the pass pays 0.3–1.3 ms per
1599
- query.
1600
- - The question "who had access at a given date" is answered by an `asOf` read over the classes `rule`, `member` and
1601
- `Resource`.
1602
-
1603
- Fields of the system permission classes:
1604
-
1605
- - `member`: the ends `User` and `Tenant`, the field `roles` (a list of roles);
1606
- - `Resource`: `alias` (the unique name of the resource and its key; this is not a class alias), `category` (`ACCOUNT`,
1607
- `API`, `READ`, `WRITE`, `DELETE`) and `pattern`, the mask. A group's mask is `categories` or `roles`, and only a
1608
- string: `'{staff}'`, `'{a,b}'`, negation `'!{a}'`. An object's mask is `endpoint` or `class` (also strings only) plus
1609
- conditions on the columns. **The database rejects a non-string mask (for example, the array `['staff']`, a number or
1610
- `null`)** with `invalid_data`, whether written by the library, raw SQL or the loader. Old rows with such a mask
1611
- (written before this check or around it) grant no permissions, while a denial by them applies to everyone: by a
1612
- group, to all accounts, by a `class` mask, to all classes (a non-string `endpoint` mask, as before, matches no
1613
- endpoint). The write check comes with the description of the `Resource` class: it arrives with a fresh installation
1614
- and after `reset({ all })`; in an existing schema, the corrected matching works after `up()` is run again;
1615
- - `rule`: the ends `Group` and `Object` (`Resource` rows), the fields `permission` (`allow` or `deny`), `weight`, and,
1616
- for the endpoint's own denial, `code` and `message`.
1617
-
1618
- ### Deleting an account
1619
-
1620
- `db.auth.purgeAccount(account)` deletes an account and the data of its tenant. The system administrator or the owner of
1621
- its tenant can call it; you cannot delete yourself or the last owner of a tenant. Ownership of rows in other tenants
1622
- passes to those tenants, sessions are revoked, credentials are deleted. An owner can also delete the tenant they are
1623
- working in right now: their session returns to their personal tenant.
1624
-
1625
- The system administrator can also delete the account row itself — `db.Account(id).delete({ confirm: true })` in the
1626
- System tenant; the data of the account's tenant stays then. Either way, the account's rows in `secret` are erased at
1627
- once: sessions, password and key hashes, TOTP secrets, external accounts (the foreign key `secret.account` with a
1628
- cascade, revision 7). Before that, the sessions are revoked with a signal, and processes drop the cached answers of
1629
- these sessions. The System account cannot be deleted (`acl_denied`): the service rows of the installation in `secret`
1630
- belong to it.
1631
-
1632
- ## 11. History and point-in-time reads
1633
-
1634
- ### Versions
1635
-
1636
- <!-- run: guide -->
1637
- ```ts
1638
- const versions = await db.Product(kettle).versions() // all versions: op, rev, author, at, data
1639
- const followed = await db.Product(mug2).versions({ follow: true }) // also across id changes (rekey, reclass)
1640
- ```
1641
-
1642
- Every version has an author, an agent, a time, a number, a kind of operation (`create`, `update`, `delete`, `reclass`,
1643
- `rekey`, `restore`, `purge`, `trim`, `reset`) and a reason (`reason`). The reason comes from the `reason` option of a verb
1644
- (`update(patch, { reason })`, `create(data, { reason })`, `delete({ confirm, reason })` and others); for every
1645
- operation of a block, from `db.begin({ reason })` and `db.batch(name, { reason })`; in raw SQL, from the
1646
- `letopis.reason` setting of the transaction. A deletion version has `$deleted`.
1647
-
1648
- Without `sort()`, versions go by id and version number (with `follow`, by time). `sort()` sets the order of versions
1649
- too, and `limit()` limits them in the query itself: `db.Product(kettle).sort('at', 'desc').limit(5).versions()` gives
1650
- the five latest versions, newest first.
1651
-
1652
- `follow: true` gives the history of the object across all its ids: before a move, the versions of the former id;
1653
- after it, of the new one. An object created later on the former id (the same key after `rekey`) is a different one,
1654
- and its versions are not in this history; its own history with `follow` starts at its creation. `diff({ follow: true
1655
- })` and `blame()` work the same way. Without `follow`, `versions()` gives every version of the id, whoever it
1656
- belonged to.
1657
-
1658
- ### Point-in-time read
1659
-
1660
- <!-- run: guide -->
1661
- ```ts
1662
- const createdAt = versions[0].at
1663
- const atCreation = await db.Product(kettle).asOf(createdAt).first() // the price at creation time: 2500
1664
- const assortment = await db.Shop(shop).Product().asOf(createdAt).rows()
1665
- ```
1666
-
1667
- `asOf(moment)` is the state "as it was", from the journal; the moment is an ISO string or a `Date`. A point-in-time
1668
- read works at any step of a chain, including with `deep()`. Only the current state is cheap: a point-in-time read is
1669
- rebuilt from the journal. The version time `at` stores microseconds, while a `Date` holds only milliseconds: pass the
1670
- `at` string to `asOf` as is, otherwise a read at the moment of a version may miss it.
1671
-
1672
- ### Deleted rows
1673
-
1674
- <!-- run: guide -->
1675
- ```ts
1676
- const withGone = await db.Customer().withDeleted().rows() // live and deleted; deleted rows have $deleted: true
1677
- ```
1678
-
1679
- ### Retention policy
1680
-
1681
- Without a policy, the journal keeps every version. A policy sets how long to keep old versions and with what
1682
- granularity:
1683
-
1684
- ```ts
1685
- history: { all: '1 day', daily: '1 week', weekly: '1 month', monthly: '1 year', yearly: 'forever', tz: 'Europe/Moscow' }
1686
- ```
1687
-
1688
- This means: all versions for the last day; for the last week, one per day; for the last month, one per week; for the
1689
- last year, one per month; older, one per year, forever. Each period keeps its last version. A week starts on Monday;
1690
- period boundaries are in the `tz` time zone (UTC by default).
1691
-
1692
- - A policy is set on a class (descendants inherit it) or for all classes without their own policy: `up({ history })`.
1693
- - If the last tier has a term instead of `'forever'`, the history older than the term is deleted: `{ all: '30 days' }`.
1694
- - While the object's history is alive, the current version, version 1 and versions of special kinds (`create`,
1695
- `delete`, `reclass`, `rekey`, `restore`, `purge`, `reset`, `trim`) are not deleted. If the tombstone of a deleted
1696
- object is older than the last tier with a term, its history is erased entirely. The history of system classes is not
1697
- thinned.
1698
- - `maintain()` does the thinning in batches of 10,000 versions ([section 13](#13-operations)).
1699
- - In the thinned part, a point-in-time read shows the last kept version before the moment, so its precision equals the
1700
- period of the tier. A subscriber that lags behind by more than the `all` tier gets `cursor_expired`.
1701
-
1702
- ### What changed and who: `diff` and `blame`
1703
-
1704
- ```ts
1705
- const [d] = await db.Order(id).diff() // previous version → latest
1706
- await db.Order(id).diff({ from: 3, to: 7 }) // by version numbers
1707
- await db.Order(id).diff({ from: '2026-10-01' }) // state at a moment → latest
1708
- await db.Order(id).diff({ follow: true }) // across id changes (reclass, rekey)
1709
- const [b] = await db.Order(id).blame() // every field of the latest version
1710
- await db.Order(id).blame({ at: '2026-10-01', limit: 200 }) // the version at a moment; scan 200 versions
1711
- ```
1712
-
1713
- Both terminals return an array, one item per object of the step. This is what the answers look like for a kettle
1714
- whose price was lowered with a reason:
1715
-
1716
- ```ts
1717
- import type { BlameResult, DiffResult } from 'letopis'
1718
-
1719
- // db.Product(kettle).blame(): version 2 changed the price, the other leaves date from the creation (version 1)
1720
- const blamed: BlameResult = {
1721
- id: '6f1c…', class: 'Product', rev: 2, complete: true,
1722
- fields: [ // in path order
1723
- { path: ['class'], rev: 1, at: '2026-10-27T09:15:02.118204+00:00', author: '<Anna id>', agent: '<service id>',
1724
- op: 'create', exact: true },
1725
- { path: ['data', 'price'], rev: 2, at: '2026-10-27T09:20:41.502617+00:00', author: '<Anna id>', agent: '<service id>',
1726
- op: 'update', reason: 'promotion discount', exact: true },
1727
- // ['data', 'sku'], ['data', 'title'], ['links', 'Shop'], ['owner'] — also version 1, like ['class']
1728
- ],
1729
- }
1730
- // db.Product(kettle).diff(): previous version → latest
1731
- const diffed: DiffResult = {
1732
- id: '6f1c…', class: 'Product',
1733
- from: { rev: 1, at: '2026-10-27T09:15:02.118204+00:00' },
1734
- to: { rev: 2, at: '2026-10-27T09:20:41.502617+00:00' },
1735
- changes: [{ path: ['data', 'price'], kind: 'changed', before: 2500, after: 2000 }],
1736
- }
1737
- ```
1738
-
1739
- `agent` here is the service that writes on behalf of Anna (`db = await svc.as(…)`); a write without impersonation has
1740
- none.
1741
-
1742
- - `diff()` returns an array of `DiffResult`: `{ id, class, from, to, changes }`. `from` and `to` are the compared
1743
- versions `{ rev, at }`. They have `id` if the version belongs to another id (`follow: true` across `reclass` or
1744
- `rekey`), and `to` has `deleted: true` if the object was deleted at that moment; then `changes` is empty. `from` is
1745
- `null` if the object did not exist yet: every path is `added`. If there is no version at all, `class` and `to` are
1746
- `null` too.
1747
- - A change, `DiffChange`, is `{ path, kind, before, after }`. `path` is an array of segments, not a dotted string;
1748
- the first segment is the section of the row: `data`, `links`, `tags`, `owner`, `class`. `kind` is `added` (there
1749
- was no value), `removed` (the value is gone) or `changed`. A missing value is omitted: `added` has no `before`,
1750
- `removed` has no `after`. Data objects are compared by keys, arrays as a whole. A multi-valued end and tags are
1751
- sets: one change per id or tag, the path is `['links', role]` or `['tags']`, and the id or tag itself is in
1752
- `before` or `after`.
1753
- - `blame()` returns an array of `BlameResult`: `{ id, class, rev, complete, fields }`, plus `deleted: true` if that
1754
- version is a tombstone. An object with no versions at that moment gives no item. `fields` is an array of
1755
- `BlameField`, one item per leaf of the version, ordered by path: `{ path, rev, at, author, op, exact }` and, when
1756
- present, `agent`, `reason`, `id`. A leaf is a scalar or an array in the data, an empty object, the role of a
1757
- single-valued end, every id of a multi-valued end (`['links', role, id]`), every tag (`['tags', tag]`), `owner` and
1758
- `class`. `rev`, `at`, `author`, `agent`, `op` and `reason` come from the version that changed the leaf last. Empty
1759
- `agent` and `reason` are omitted; `id` is present only if it is a version of the former id (the history went
1760
- through `reclass` or `rekey`). `blame()` does not look at versions of another object on the same id — one that
1761
- lived there before a move or was created after it.
1762
- - `exact: false` means there are gaps between the current version and the found one (the history is thinned,
1763
- trimmed or partly invisible): a missing version may have changed the leaf too. `complete: true` means the whole
1764
- history was scanned without gaps; by default at most the 1000 latest versions are scanned.
1765
- - The database computes both with the caller's rights: raw SQL gets the same, `select "v2.shop".blame('<id>')`,
1766
- `select "v2.shop".diff_versions('<id>')`.
1767
- - The journal column `changed` (the paths changed by a version, which make `blame` fast) is not granted to the
1768
- application: the paths describe the difference from the previous version, and the reader may not see that one.
1769
- So `select * from log` does not work for the application — list the columns; `blame` gets the paths through a
1770
- database function only when both versions are visible.
1771
-
1772
- ### Trimming history
1773
-
1774
- <!-- run: guide -->
1775
- ```ts
1776
- const trimmed = await db.trimHistory(kettle, new Date()) // erase the versions of the kettle before this moment
1777
- ```
1778
-
1779
- `db.trimHistory(target, moment)` erases the history before the moment right away. The target is a class with its
1780
- descendants (by name), an id, a list of ids or rows. This is the middle step of the "right to be forgotten":
1781
- `anonymize` → history trimming → deletion and `purge`. In place of version 1, a `trim` record stays with its time, so a
1782
- point-in-time read knows when the object appeared.
1783
-
1784
- ### Integrity check
1785
-
1786
- <!-- run: guide -->
1787
- ```ts
1788
- const audit = await adm.verify({ data: true }) // { ok, issues, anchor }
1789
- ```
1790
-
1791
- `verify()` recomputes the hash chains of all versions and returns an anchor: a hash of all chain heads. Keep the anchor
1792
- outside the database: a mismatch with it exposes tampering with the latest versions. `{ data: true }` also checks again
1793
- the data, ends, tenants, owners and ids of all rows. The system administrator or an admin connection can call it; from
1794
- the command line, use `letopis verify [--data]` (exit code 1 if something is found). Legitimate gaps (`trim` and
1795
- `purge` records, thinning and `reset`) are not counted as errors.
1796
-
1797
- ## 12. Change subscription and cache
1798
-
1799
- ### Subscription
1800
-
1801
- <!-- run: guide -->
1802
- ```ts
1803
- const sub = db.watch({ from: 'start', classes: ['Product'] }) // the whole journal of the class, then new events
1804
- await db.Product(kettle).update({ price: 2100 }).rows()
1805
- let cursor = ''
1806
- for await (const ev of sub) { // ev: { cursor, row, id, class, op, rev, at }
1807
- cursor = sub.cursor! // the subscription cursor: watch({ from: cursor }) continues after this event
1808
- if (ev.id === kettle.id && ev.row.data.price === 2100) break // break (or sub.close()) stops the subscription
1809
- }
1810
- ```
1811
-
1812
- - `from` is `'now'` (the default: only changes after subscribing), `'start'` (the whole journal) or a saved cursor.
1813
- `db.watch()` returns at once, and the subscription takes the "now" moment with its first query to the database: a
1814
- write made right after `db.watch()` may commit earlier and miss the stream. `await sub.ready()` waits for that
1815
- moment: a write after it comes to the stream. The example above reads from `'start'`.
1816
- - Calls of `sub.next()` go in turn: parallel calls get different events. A call abandoned by a timeout
1817
- (`Promise.race`) still gets its event — keep its promise and wait for it again.
1818
- - `classes`: only these classes with their descendants; `pollMs`: fallback polling (1000 ms). A row that left these
1819
- classes through a class change without an id change arrives as a deletion event in the previous class (`op:
1820
- 'delete'`, `row` is the previous version with `$deleted`), even if the subscriber cannot see the new class. The
1821
- event comes if the subscriber could see the previous version.
1822
- - Events come without losses or repeats, under the reader's permissions, in the order of transaction numbers, and
1823
- within a transaction in the order of writing. This is not the commit order: compare versions of one object by `rev`.
1824
- - A long write transaction in any database of the cluster delays the stream: `await sub.lag()` shows the lag in ms.
1825
- - A cursor older than the `all` tier of the retention policy gives `cursor_expired`: read the current state again and
1826
- subscribe with `'now'`. The time in the subscription cursor `sub.cursor` is the time of the last event, and once the
1827
- subscription has caught up with the journal, the time of the poll, so a subscription that got no events for a day
1828
- does not expire. The event cursor `ev.cursor` keeps the time of the event itself.
1829
- - Without `LISTEN` (`connect({ listen: false })`), the subscription polls the journal.
1830
- - A subscription without losses is a saved cursor: after processing an event, store `sub.cursor` (the position after
1831
- the last delivered event) and resume from it after a restart. `ev.cursor` also continues the stream, but after a
1832
- quiet period longer than the `all` tier such a cursor is older than the tier, and resuming from it gives
1833
- `cursor_expired`.
1834
-
1835
- ### Projectors
1836
-
1837
- A projector builds rows of derived classes from the journal: summaries, denormalized cards and anything else that may
1838
- be computed with a lag behind the writes. The application writes the source classes (`from`), and only the projector
1839
- writes the rows of the `into` classes. Each `into` class is marked with the option `derivedBy` holding the projector
1840
- name ([Class settings](#class-settings)). Only an administrator creates derived classes: sync them with an admin
1841
- connection holding a tenant session — `adm` from section 3 (`connect({ dsn: ADMIN, schema, token, allowBypassRls:
1842
- true })`) or `as()` of a service on such a connection; in the CLI, `letopis sync` with the `--dsn` of the admin role.
1843
- For an application session it is `acl_denied`. In the example below, the projector counts product sales from the
1844
- order lines `line`.
1845
-
1846
- ```ts
1847
- const ProductSales = hub('ProductSales', { qty: z.number().int(), revenue: z.number().int() }, {
1848
- ends: { Product: one(Product).onDelete('cascade') }, key: ['Product'], derivedBy: 'product-sales', history: false,
1849
- }) // the derived class is in the model; syncModels — on an admin connection
1850
-
1851
- const sales = adm.projector('product-sales', { // adm is an admin connection
1852
- from: ['line'], // source classes with descendants; they must keep a journal
1853
- into: ['ProductSales'], // the derived classes of this projector
1854
- version: 1, // the handle logic changed: a new version and a rebuild
1855
- withBefore: true, // c.before is the previous version of the object or null
1856
- handle(changes, out) {
1857
- for (const c of changes) {
1858
- if (c.op === 'purge' || c.op === 'trim' || c.op === 'reset') continue // service changes, not versions
1859
- const was = c.before // the previous version of the order line: subtract it
1860
- const now = c.row.$deleted ? null : c.row // the new version: add it; a deletion has none
1861
- if (was) out.inc('ProductSales', { Product: was.links.Product },
1862
- { qty: -Number(was.data.qty), revenue: -Number(was.data.qty) * Number(was.data.price) }, { tenant: c.tenant })
1863
- if (now) out.inc('ProductSales', { Product: now.links.Product },
1864
- { qty: Number(now.data.qty), revenue: Number(now.data.qty) * Number(now.data.price) }, { tenant: c.tenant })
1865
- }
1866
- },
1867
- })
1868
- const report = await sales.run({ once: true }) // catch up with the journal: { batches, changes, created, updated, … }
1869
- const state = await sales.status() // { name, state, version, cursor, at, lag, failed, rebuildingTenants }
1870
- await sales.rebuild() // rebuild from the current state; { chunked: true } works in batches
1871
- await sales.rebuild({ tenant: tenantId }) // rebuild one tenant, for example a dropped one (failed)
1872
- await sales.rebuild({ tenant: tenantId, chunked: true }) // the same in batches: meanwhile the journal tail goes on for all
1873
- sales.run().catch((err) => console.error('the projector stopped', err)) // follow the journal until stop()
1874
- sales.close() // stop() after the current batch and unsubscribe from signals
1875
- ```
1876
-
1877
- - A projector needs an admin connection: `connect({ allowBypassRls: true })` with a role that may become the owner of
1878
- the schema. Otherwise you get `admin_required`, and inside `db.begin()` you get `invalid_query`. The projector reads
1879
- the journal of all tenants.
1880
- - The first `run()` rebuilds the projection from the current state: the journal is thinned by the retention policy
1881
- and cut by `purge`, so the projection cannot be assembled from the start of the journal.
1882
- - Each batch is one transaction: the journal changes, `handle`, the written results and the cursor move are committed
1883
- together. A break in the middle of a batch rolls the whole batch back, and the retry starts from the same cursor, so
1884
- every change is applied exactly once. A projector has one writer: a second process of the same projector skips the
1885
- batch (the `skipped` counter of the report), and `rebuild()` started while another process writes a batch gives
1886
- `projector_busy`.
1887
- - A batch is split by tenant: `handle` gets the changes of one tenant, and its results are written under a savepoint
1888
- of its own. If the database rejects writing the results of a tenant (the data do not pass the description of its
1889
- derived class, the target of an end is not found and so on) or `out` rejects a row or a delta built from its data
1890
- (a delta that is not a number), only its changes are rolled back: the tenant drops out, and the others are written
1891
- in the same batch. A dropped tenant gets into `status().failed` — `{ [tenant id]: { cursor, code, message } }`, and
1892
- `message` names the tenant and the row id — and its changes are skipped until it is rebuilt. The derived rows of a
1893
- dropped tenant no longer change: a tenant dropped in `run()` keeps them as they were before the failed batch, and a
1894
- tenant dropped in the middle of a rebuild is left without them — the partly built rows are deleted. An exception
1895
- from `handle`, a breach of the `out` limits and a failure of the environment (a deadlock, a cancelled query, a
1896
- broken connection) roll the whole batch back, and `run()` is rejected. So catch `run()` without `once`: after an
1897
- error it stops.
1898
- - A change (`ProjectorChange`) is `cursor`, `id`, `class`, `tenant`, `op`, `rev`, `at`, `author`, `agent`, `reason`,
1899
- `moved` and `row`, the version from the journal (for a deletion, a tombstone with `$deleted`). During a rebuild, the
1900
- changes are rows of the current state: `rebuild: true`, `op` is `create`, `cursor` is empty.
1901
- - A class change that keeps the id (classes without a key: a raw `UPDATE … SET class` and `reclass()`) is written to
1902
- the journal as a version of the new class. If it takes the row out of the `from` family, the projector gets a
1903
- deletion: `op` is `delete`, `row` is the last version of the row in the family with `$deleted`, and `rev`, `at` and
1904
- the author are those of the class change. A row that enters the family this way comes as a `reclass` change with
1905
- `before` = `null`.
1906
- - With `withBefore`, a change has `before`, the previous version of the same id. `before` is `null` if there is no
1907
- previous version (a creation and the first version of a new id after `reclass` and `rekey`), if the previous
1908
- version is a deletion (this happens after `restore`) or lies outside the `from` family, and for every change of a
1909
- rebuild. With `reclass` and `rekey`, the former data are subtracted through the tombstone of the former id: it comes
1910
- as a change with `$deleted`, its `before` is the last version of the former id, and `moved` of both changes points
1911
- to the other id. So the fold of the example gives the same sum after `rekey`. The service changes `purge`, `trim`
1912
- and `reset` have no `before` field at all. So subtract `before` only after checking it for `null`, as in the example
1913
- above.
1914
- - The service journal records `purge`, `trim` and `reset` also come as changes, and their `rev` is 1. So check `op`
1915
- first, and only then compare `rev`.
1916
- - `out.upsert(class, row, { tenant })` creates a row or replaces it entirely by the class key.
1917
- `out.inc(class, row, deltas, { tenant })` creates a row or completes it and adds the numbers without reading the row
1918
- (missing fields count from 0). `out.delete(class, key or id, { tenant })` deletes a row and does nothing if there is
1919
- none. `await out.get(class, key or id, { tenant })` reads the current row together with the writes of the batch.
1920
- `delete` and `get` find a row of a class with a key by the key fields (`{ day: '2026-10-29' }`), and by id (a string
1921
- or `{ $id }`) only a row of a class without a key; an id alone for a class with a key is `invalid_data`, and the
1922
- batch is rolled back.
1923
- `out` writes only to the `into` classes and only to the tenant of the changes passed to `handle`; otherwise you get
1924
- `derived_class` or `tenant_denied`, and the batch is rolled back.
1925
- - `out` adds up the increments of a row given by key fields only (or by `$id` alone) before sending them to the
1926
- database, as long as there is no `upsert`, `delete` or `get` between them: one edit of a row per batch, and the class
1927
- rule is checked on the total of its increments. `inc` does not fold a row with other fields (data, `$tags`,
1928
- `$owner`): its fields edit the row, and the order of such edits is kept.
1929
- - A row for `out` is in the loader format: fields and end roles at the top level, the service keys `$tags`, `$owner`
1930
- and `$id` (`$id` is needed only by a class without a key). The value of a role is the id of the target. An edit that
1931
- changes nothing gives no version. Versions of derived rows are written by System with the agent
1932
- `letopis projector <name>`; the journal is kept according to the `history` of the class, and the change signal is
1933
- always sent.
1934
- - `rebuild()` rebuilds the projection atomically, in one transaction: until it commits, readers see the previous rows,
1935
- but the subscription horizon stands still. `rebuild({ chunked: true })` works in batches: the derived rows are
1936
- deleted at once, the data stays incomplete until the rebuild ends, and `status().state` is `'rebuilding'`. The next
1937
- start continues an interrupted batch rebuild. `run()` itself, on the first start of a new projection and with
1938
- `onExpired: 'rebuild'`, rebuilds in batches: the subscription horizon of the cluster stands still no longer than a
1939
- batch. Meanwhile a second process of the same projector waits or continues the same rebuild instead of starting it
1940
- anew. A rebuild deletes the rows of the participants only, and only of the classes of their `into` families, and
1941
- starts all participants anew: the marks of dropped tenants are removed, and a tenant whose data fail again drops
1942
- out again.
1943
- - `rebuild({ tenant: id })` rebuilds one tenant in one transaction: it deletes its derived rows, builds them from its
1944
- current source rows and removes its mark in `status().failed`; the projector does not apply again the changes the
1945
- rebuild has already counted. Fix the data or the class description of the tenant, then rebuild it. If the database
1946
- rejects the write again, the rebuild is rolled back with this error, and the tenant stays dropped. No projection yet
1947
- or a batch rebuild in progress gives `rebuild_required`; a tenant that is not a participant gives `tenant_denied`.
1948
- With `chunked: true` — in batches, each in a transaction of its own with its own snapshot, while the journal tail goes
1949
- on for all: the tail skips the tenant's changes above the last batch (a batch will read them) and, in a passed batch,
1950
- those whose transaction is visible in its snapshot. The tenant's derived rows are deleted at once and are incomplete
1951
- until the end of the rebuild (`status().rebuildingTenants`); the rebuild waits out a lock held by `run()` of another
1952
- process. An interrupted rebuild (the process stopped) is continued by `run()` of any process, with or without
1953
- `onExpired`.
1954
- - `status().state` describes the projection in the database, not a process: `'new'` means there is no projection yet,
1955
- and the first `run()` or `rebuild()` builds it; `'running'` means the projection is built and kept up with the
1956
- journal (it does not mean that some process is executing `run()` right now); `'rebuilding'` means a batch rebuild
1957
- is in progress or was interrupted.
1958
- - `version` is the version of the `handle` logic. If the projection was built by another version, `run()` throws
1959
- `rebuild_required`, and a rebuild with `rebuild()` is needed.
1960
- - If the cursor is older than the `all` tier of the retention policy of the sources and there are changes after it,
1961
- some changes may have been thinned out. Each tenant has a tier of its own: it is the description of its classes. If
1962
- the cursor is older than the tier of every participant, `run()` throws `cursor_expired`, and with
1963
- `onExpired: 'rebuild'` the projector rebuilds entirely, in batches. If it is older than the tier of only some of
1964
- them, they drop out (`status().failed`, code `cursor_expired`) and the others go on, and with
1965
- `onExpired: 'rebuild'` the dropped tenants rebuild themselves, in batches: the rebuild state lies in the checkpoint,
1966
- and `run()` of any process continues it; a lock held by another process is only a retry on the next pass, and a data
1967
- error drops the tenant with its code, without retries. The tier of a rebuilt tenant counts from the start of its
1968
- rebuild until the cursor catches up with it: a lagging projector does not drop it again. A projector without new
1969
- changes does not expire. `status().lag` is the age of the oldest unprocessed change in ms.
1970
- - A projector works with the tenants in which every `into` class is marked with its `derivedBy`; it skips the changes
1971
- of other tenants, so a class with the same name and no mark in another tenant does not get in its way. No such tenant
1972
- at all — `derived_class`. Only an administrator may mark the classes of a tenant, so it is the administrator who
1973
- chooses the participants: a tenant user cannot join a projector on their own. The option `tenants: [id, …]`
1974
- is an explicit list of participants: the projector works only with these tenants (their `into` classes must be
1975
- marked too) and ignores the marks of the others. The sources are the `from` families of these tenants; an `into`
1976
- class is never a source, even if it falls into `from`: a projector does not read its own results. A projector does
1977
- not see the changes of a class with `history: false`: it has no journal. So a tenant with such a class among the
1978
- sources drops out with the code `invalid_class`, and the projector does not build its derived rows. Turn the journal
1979
- of the class on and rebuild the tenant with `rebuild({ tenant })`.
1980
- - A tenant that became a participant after the first projection (the administrator marked its classes later) gets an
1981
- initial batch rebuild: the rows created before it joined get into the projection too. A projector re-reads its
1982
- participants on a class signal, and on a connection without LISTEN (`listen: false`) at the start of `run()` and at
1983
- least every 5 seconds.
1984
- - `batch` is how many changes go into a batch (500), `pollMs` is fallback polling (1000 ms): source signals wake the
1985
- projector earlier.
1986
- - `lockTimeout` is how long a projector transaction waits for another transaction's row lock (5000 ms; 0 — no limit).
1987
- Otherwise a transaction of any tenant holding a total row or an end target would hold the batch of all tenants.
1988
- Longer than that, the batch rolls back, the cursor stays, and `run()` does not reject but skips the pass
1989
- (`skipped`) and repeats it. The report names the tenant whose row stopped the batch (`locked`).
1990
- `rebuild({ tenant, chunked: true })` waits such a lock out.
1991
- - `lockRetries` is how many passes in a row the writes of one tenant may run into `lockTimeout` before it drops out
1992
- (0 by default — it never drops out). The dropped tenant gets the code `55P03`, and the batch goes on for the others;
1993
- its changes are skipped until `rebuild({ tenant })`. The process keeps the count, and a successful write of the
1994
- tenant resets it. Note: a long exclusive lock on the family of a System class (a tightening of its description) will
1995
- hit every tenant in turn.
1996
- - Every edit that `out` sends to the database is a separate edit of a row, and the speed of a projector is set by the
1997
- number of edits, not by the number of changes. The folding of `inc` brings the edits of a row down to one per batch,
1998
- so the handle from the example above (`out.inc` for every change) caught up with 6 thousand changes per second with
1999
- `batch` 500 and 33 thousand with `batch` 2000 on the benchmark machine ([section 17](#17-performance)): the larger
2000
- the batch, the more increments of one row fold into one edit.
2001
-
2002
- ### Recipe: daily revenue
2003
-
2004
- An aggregate in the database and a projector solve similar tasks in different ways:
2005
-
2006
- | | Aggregate in the database | Projector |
2007
- |---|---|---|
2008
- | when it is computed | in the same transaction as the write of the source | after the commit, with a lag (`status().lag`) |
2009
- | what it computes | the sum of a field or the number of source rows for one target row | anything: summaries by day and by group, denormalized cards |
2010
- | matches the source | always, raw SQL included | when the projector has caught up with the journal |
2011
- | cost of a write | every operation locks the target row and writes its version, so the writers of one target queue up | writers do not wait for each other: the projector applies changes in batches after the commit |
2012
- | rule over the total | `min(0)` rejects an operation that would take the total beyond the rule | the total does not block writes: a data error drops the tenant out of the projection |
2013
- | what to run | nothing: the database keeps the field | a process on an admin connection |
2014
-
2015
- A wallet balance that must not go below zero is an aggregate
2016
- ([Recipe: balance as an aggregate](#recipe-balance-as-an-aggregate)). The revenue of a shop by day is a total over many
2017
- sales that may lag a little, which is a job for a projector:
2018
-
2019
- ```ts
2020
- const Sale = hub('Sale', { day: z.iso.date(), amount: z.number().int().min(0) }, { ends: { Shop } }) // a sale: day and amount
2021
- const DayRevenue = hub('DayRevenue', { day: z.iso.date(), revenue: z.number().int(), sales: z.number().int() }, {
2022
- ends: { Shop: one(Shop).onDelete('cascade') }, key: ['Shop', 'day'], derivedBy: 'day-revenue', history: false,
2023
- })
2024
- await syncModels(db, [Sale], { apply: true })
2025
- await syncModels(adm, [DayRevenue], { apply: true }) // the derived class is created by an administrator: adm holds a tenant session
2026
-
2027
- const revenue = adm.projector('day-revenue', {
2028
- from: ['Sale'], into: ['DayRevenue'], version: 1, withBefore: true,
2029
- handle(changes, out) {
2030
- for (const c of changes) {
2031
- if (c.op === 'purge' || c.op === 'trim' || c.op === 'reset') continue // service changes
2032
- const was = c.before // the previous version of the sale: subtract it from its day
2033
- const now = c.row.$deleted ? null : c.row // the new one: add it to its day; a deletion has none
2034
- if (was) out.inc('DayRevenue', { Shop: was.links.Shop, day: was.data.day },
2035
- { revenue: -Number(was.data.amount), sales: -1 }, { tenant: c.tenant })
2036
- if (now) out.inc('DayRevenue', { Shop: now.links.Shop, day: now.data.day },
2037
- { revenue: Number(now.data.amount), sales: 1 }, { tenant: c.tenant })
2038
- }
2039
- },
2040
- })
2041
- revenue.run().catch((err) => console.error('the revenue projector stopped', err)) // follows the journal until stop()
2042
- const week = await db.Shop(shop).DayRevenue().sort('data.day', 'desc').limit(7).rows() // the last seven days
2043
- ```
2044
-
2045
- - The day is a field of the sale, not the version time `at`: with the version time, an edit, `restore` or `rekey`
2046
- would move the amount of the sale to its own day.
2047
- - `out.inc` adds without reading the row, so the total does not depend on the order of changes: the journal gives
2048
- transactions in the order of their first write, not of their commit. Editing a sale subtracts the previous amount
2049
- from the previous day and adds the new one to the new day; a deletion only subtracts. A day with no sales left stays
2050
- as a row of zeros. `out` folds the increments of one day in a batch into one edit.
2051
- - `DayRevenue` has `history: false`: the derived rows do not double the journal, and a rebuild assembles them without
2052
- history anyway. `Sale` needs a journal: a projector does not see the changes of a class with `history: false`.
2053
- - The application reads `DayRevenue` rows like any class, and only the projector writes them: `create` or `update` of
2054
- such a row gives `derived_class`. The total lags behind the sales until the next batch: write signals wake the
2055
- projector, and without them it polls every `pollMs`.
2056
-
2057
- ### Result cache
2058
-
2059
- <!-- run: guide -->
2060
- ```ts
2061
- const cached = await db.Product().cache({ ttl: 30 }).rows() // the result is kept in process memory for 30 s
2062
- const stats = db.cache.stats() // { size, hits, misses, stores, raced, … }
2063
- ```
2064
-
2065
- The cache is enabled only explicitly: `.cache()` in a chain or `cache: { ttl }` on a class in the model. This is how it
2066
- stays correct:
2067
-
2068
- - the result key includes the query, the parameters, the session, the actor and the tenant, so one user never gets the
2069
- result of another user;
2070
- - your own writes clear the results of the changed classes right after commit (including the cascade classes); other
2071
- writers' changes clear them on a database signal. In the `tenant` and `opaque` modes, a process with the signal key
2072
- clears a class only in the tenant of the write (System rows — in all tenants); a process without the key clears it
2073
- in all tenants, and in the `opaque` mode any signal clears its whole cache;
2074
- - changes of permissions and classes, writes through `db.sql`, a revoked session and a listener reconnect clear the
2075
- whole cache or the results of the session;
2076
- - the default term is 60 seconds; the cache does not work inside `db.begin()`;
2077
- - without `LISTEN` and in the `none` mode the cache is off, and `connect({ cache: { ttlOnly: true } })` enables it: an
2078
- answer lives until its term ends unless a write of this connection resets it;
2079
- - `db.load()`, the projector and `db.rebuildAggregates()` reset the cache, even of this connection, only by a database
2080
- signal.
2081
-
2082
- A process re-reads the class registry right away on a class change signal (one re-read at a time), and the registry
2083
- and the signal key on `*`, on an unknown token and on a payload of another mode (and without the key in the `opaque`
2084
- mode — on any signal) not more often than once in 5 seconds: a stream of signals, fake ones included, does not turn
2085
- into a stream of queries. In the `none` mode, the registry checks the class
2086
- fingerprint every 5 seconds, and subscriptions poll the log. A subscription wakes up only for its own classes and its
2087
- own tenant, and without the key — for any signal; it always reads the events from the log, so nothing is lost.
2088
-
2089
- Between processes, the cache goes stale for the signal sending interval (`notifyIntervalMs`, 100 ms) plus delivery;
2090
- `db.flushSignals()` sends the accumulated signals at once. `db.cache.clear()` clears the cache.
2091
-
2092
- ## 13. Operations
2093
-
2094
- ### Maintenance: `maintain()`
2095
-
2096
- <!-- run: guide -->
2097
- ```ts
2098
- const report = await adm.maintain() // { sessions, codes, thinned, batches, more, queue, skipped }
2099
- ```
2100
-
2101
- `maintain()` deletes sessions and one-time codes of all kinds (`OTP` and custom `OTP_<NAME>` from `issueOtp({ kind })`)
2102
- from the `secret` table — expired ones, as well as
2103
- revoked sessions and codes that were used, replaced by a new one or ran out of attempts, — and the lock rows of deleted
2104
- classes (two per class; a row held by a transaction goes on the next call), thins the journal by the
2105
- retention policy and checks the PostgreSQL signal queue: if it is more than half full, you get a warning. You do not
2106
- need `maintain()` for the rows of a deleted account: the deletion itself erases them ([Deleting an account](#deleting-an-account)).
2107
- The system administrator, a role with the `letopis.maintain` permission or an admin connection can call it. An external scheduler
2108
- runs it (`letopis maintain`), or the library itself does: `connect({ maintain: '1h' })`. The interval is a number of
2109
- milliseconds or a string (`'500ms'`, `'30s'`, `'15m'`, `'1h'`, `'1d'`) from 1 ms to 2³¹−1 ms (about 24 days):
2110
- `connect()` rejects any other value, including zero, a negative number or `NaN`, with `invalid_data` without opening
2111
- connections. `close()` waits for a scheduled run in progress: it finishes the current batch and does not start the
2112
- next one. Of several processes, only one works: each batch is its own
2113
- transaction, and the first thing it does is take the maintenance lock — a row of the `secret` table that neither the
2114
- application role nor any other database role can take; the lock is released when the batch commits. If another
2115
- process holds the lock, the call answers `skipped: true` before the first batch, and between batches it stops (that
2116
- process continues) and reports what it has done with `more: true`. There is no session-level lock, so `maintain()`
2117
- works behind any pool, including PgBouncer in transaction mode.
2118
-
2119
- ### Reset: `reset()`
2120
-
2121
- ```ts
2122
- await adm.reset({ level: 'tenant', confirm: 'v2.shop', tenant: SHOP }) // sessions | tenant | all
2123
- ```
2124
-
2125
- Reset is meant for tests and test environments. It works only if the installation was set up with
2126
- `up({ allowReset: true })` (otherwise `reset_disabled`, even for System), the caller has the `letopis.reset`
2127
- permission, and `confirm` matches the schema name (`confirm_mismatch`). Levels: `sessions` resets all sessions
2128
- except the current one; `tenant` the tenant's data; `all` everything, with the system seed laid down again and a new
2129
- service key. The first journal record after a reset is `reset`.
2130
-
2131
- ### Model description for people and AI
2132
-
2133
- <!-- run: guide -->
2134
- ```ts
2135
- import { describeText } from 'letopis'
2136
-
2137
- const model = db.describe() // { schema, tenant, classes: [...], methods }
2138
- const modelText = describeText(model) // the same as text: for the application's llms.txt or a prompt for an AI agent
2139
- ```
2140
-
2141
- `db.describe()` returns a description of the classes available to the session: fields, ends, keys, policies and
2142
- metadata, plus the multi-field conditions `anyOf` and `oneOf` with inheritance applied, the derived class mark
2143
- `derivedBy` and the aggregates `aggregate`, when a class has them. From the command line:
2144
- `letopis describe --format text --out llms.txt`.
2145
-
2146
- ### Agents and MCP
2147
-
2148
- An agent — Claude, Cursor or your own — works with the data through MCP tools: it learns the model, reads with
2149
- chains, looks at the history and, if allowed, writes. The MCP server acts on behalf of one user and only with that
2150
- user's rights: every call is a chain under the user's session, and the database checks rights and tenants. The agent
2151
- has no raw SQL.
2152
-
2153
- | Tool | What it does |
2154
- |---|---|
2155
- | `letopis_model` | the model: classes, fields (JSON Schema), the `anyOf` and `oneOf` conditions, ends, keys, aliases, metadata; with `class`, one class in detail |
2156
- | `letopis_query` | reading with a JSON chain: the start is `class` and `filter`, `id` or `ids`; `steps` are hops by roles and classes; `asOf`, `withDeleted`, `sort`, `limit`, `cursor`; the terminal is `rows`, `first`, `ids`, `count`, `sum`, `avg`, `min`, `max` or `countBy` |
2157
- | `letopis_history` | `versions`, `diff` or `blame` of one object |
2158
- | `letopis_write` | `create`, `update` (only with `rev`), `upsert`, `delete`, only with `write: true` (`letopis mcp --write`) |
2159
-
2160
- A `letopis_query` request is the same chain as `db.Product({ price: lt(1000) }).Shop().sort('data.name').rows()`, written as JSON:
2161
-
2162
- ```json
2163
- { "class": "Product", "filter": { "price": { "$lt": 1000 } }, "steps": [{ "step": "Shop" }], "sort": { "field": "data.name" } }
2164
- ```
2165
-
2166
- A filter value means equality, a nested object addresses the fields of nested objects, `{ "Role": id }` is an end.
2167
- Operators: `$ne`, `$gt`, `$gte`, `$lt`, `$lte`, `$between`, `$in`, `$like`, `$ilike`, `$starts`, `$ends`, `$has`,
2168
- `$hasAny`, `$hasAll`, `$exists`, `$isNull`, `$not`; OR is `{ "$or": [filter, …] }`. An unknown class, role, parameter
2169
- or operator gives an `isError` answer with a hint of similar names. There are at most 16 `steps` and at most 64
2170
- conditions in the `$or` of one filter; otherwise `invalid_query`. These limits do not bound the time of a query: set
2171
- `statement_timeout` for the database role the MCP server connects as (`alter role … set statement_timeout = '5s'`).
2172
-
2173
- The answer is `structuredContent` and its text copy. `structuredContent` is `{ notice, data }`: `notice` is a constant
2174
- text of the package saying that `data` holds database data, not instructions, and `data` is the answer itself. The
2175
- shape of `data` of `letopis_query` depends on the terminal:
2176
-
2177
- | Terminal | `structuredContent.data` |
2178
- |---|---|
2179
- | `rows` (default) | `{ rows, count, more, cursor? }` |
2180
- | `ids` | `{ ids, count, more, cursor? }` |
2181
- | `first` | `{ row }`, a row or `null` |
2182
- | `count` | `{ count }` |
2183
- | `countBy` | `{ counts, truncated? }` |
2184
- | `sum`, `avg`, `min`, `max` | `{ value }` |
2185
-
2186
- For `rows` and `ids`, `count` is the number of rows on this page, not the total. A page holds at most `limit`,
2187
- `maxRows` rows (100) and `maxBytes` bytes of row JSON (50,000); a row that is itself larger than `maxBytes` comes
2188
- alone on its page. `more: true` means there are more rows, and then the same request with `cursor` returns the next
2189
- page. `counts` is `{ value: number }`, the most frequent values; if the groups did not fit into the limits,
2190
- `truncated: true` comes. `countBy` reads at most `maxRows` + 1 groups from the database. The
2191
- field of an aggregate and of `countBy` is `field: 'data.<path>'`. Sorting is `sort: { field, dir }`: `field` is `at`,
2192
- `rev`, `id` or `data.<path>`, `dir` is `'asc'` (default) or `'desc'`; without `sort`, rows go by `id`. `sort`, `limit`
2193
- and `cursor` work only with `rows`, `first` and `ids`.
2194
-
2195
- `letopis_model` without `class` answers (in `data`) `{ schema, tenant, classes }`, the same classes as in `db.describe()`. If such
2196
- a model is larger than `maxBytes`, a brief one comes: `brief: true`, and the classes have only the names of their
2197
- fields and ends; `letopis_model` with `class` returns each class in detail. `letopis_history` takes `class`, `id` and
2198
- `kind`. `versions` answers `{ versions, more }`, the latest versions within the answer limits, from the oldest to the
2199
- newest; `diff` with `from` and `to` (a version number or a moment) answers `{ diff }`; `blame` with `at` and `limit`
2200
- answers `{ blame }`. If the changes or the leaves did not fit, the answer is cut, and it has `truncated: true` and
2201
- `total`. `follow` works with `versions` and `diff`.
2202
-
2203
- A tool refusal is `isError: true` with a `structuredContent` of the form `{ notice, error }`, where `error` is `{ code, message }`, plus `similar` (similar names),
2204
- `detail` (details) and `sqlstate` (the PostgreSQL code of an error not from letopis) when present. A parameter that
2205
- does not match the tool input schema (a wrong type, a value outside the list, a missing required one) is rejected by
2206
- the SDK before letopis: then it is `isError: true` with text only, without `structuredContent`.
2207
-
2208
- In the text of the answer, everything that came from the database (rows, names and descriptions of classes, error
2209
- texts, hints) sits inside `<untrusted-data-…>` with a random suffix and a note that this is data, not instructions.
2210
- Tool descriptions and the server's `instructions` are fixed text of the package: tenants write the `meta` of classes,
2211
- so it only appears in results.
2212
-
2213
- Writing. Without `write`, the `letopis_write` tool is not in the tool list, and a call to it is an SDK protocol error
2214
- (`Tool letopis_write not found`), not a tool answer with `isError`. The ends of a write are set by
2215
- `links: { Role: id | [id] | null }`, and `create`, `update` and `upsert` answer `{ rows }`. `update` takes only the
2216
- `rev` of the version that was read: someone else's version gives `conflict`, and no update is lost. `delete` without
2217
- `confirm` returns a preview of the cascade and the operation code: `{ preview, operation }`. With `confirm: true`, a
2218
- human confirms the deletion in the client through elicitation (in protocol revision 2025 as a request to the client,
2219
- in 2026-07-28 as an `input_required` answer), the confirmation is bound to the hash of the operation, and the answer
2220
- is `{ deleted, operation }`. In revision 2026-07-28, the `requestState` signature with which the client repeats the
2221
- call is also bound to the user of the server and is valid for 10 minutes: a signature of another user's server and an
2222
- expired one give the refusal `confirmation_required`. The confirmation window shows the canonical class name and the ids of the targets (the
2223
- first 10, then «и ещё K», that is "and K more"); a tenant writes the alias and the `meta` of a class, so they do not
2224
- get into the window. An operation code sent by the agent is not a confirmation; a client without elicitation gets the
2225
- refusal `confirmation_required`. Every write gets the reason `mcp: letopis_write`, and `blame()` shows it. By default,
2226
- `letopis_write` does not create, edit or delete the permission classes, `rule`, `Resource`, `member`, `Account`,
2227
- `Class` and their descendants (rules, memberships, class descriptions): the refusal is `acl_denied`. An agent under an
2228
- injection in the data would grant a stranger a membership or a rule without a human confirmation. Only the option
2229
- `allowAclWrite: true` allows it.
2230
-
2231
- **stdio.** `letopis mcp` is a stdio server for MCP clients. It needs the SDK package: `npm i @modelcontextprotocol/server`
2232
- (an optional peer dependency). The client passes the connection string, the schema and the token as flags or as the
2233
- variables `LETOPIS_DSN`, `LETOPIS_SCHEMA`, `LETOPIS_TOKEN`; `--write` turns writing on. Only a user session token is
2234
- accepted, and the application issues it: `svc.auth.sessionFor(user, { ttl })`, while `auth.switch(tenant)` sets the
2235
- working tenant of the session. The command rejects the service key (`--api-key`): it grants all `auth.*` rights and
2236
- would lie on the agent's machine. Without a token and with `--api-key`, the command exits with code 2 and writes the
2237
- reason to stderr. The command also rejects the token of a session in the System tenant (this is how a system
2238
- administrator works): it has no `allowSystemAdmin` and `allowAclWrite` options. Client configuration (Claude Desktop,
2239
- Claude Code, Cursor):
2240
-
2241
- ```json
2242
- {
2243
- "mcpServers": {
2244
- "letopis": {
2245
- "command": "npx",
2246
- "args": ["letopis", "mcp"],
2247
- "env": {
2248
- "LETOPIS_DSN": "postgres://shop_app:app-secret@localhost:5432/shop",
2249
- "LETOPIS_SCHEMA": "v2.shop",
2250
- "LETOPIS_TOKEN": "<user session token>"
2251
- }
2252
- }
2253
- }
2254
- }
2255
- ```
2256
-
2257
- **HTTP.** There is no built-in HTTP server. `createMcpServer(db, { write, maxRows, maxBytes })` from `letopis/mcp`
2258
- returns an MCP server, and the application plugs it into its own `createMcpHandler` handler from the SDK with its own
2259
- OAuth 2.1 authorization: the application checks the request token and finds the letopis user by it. The connection
2260
- is a user session (`connect({ token })`) or the facade `svc.as(user)`: the service key stays in the application
2261
- process. A connection that bypasses RLS is refused with `bypass_rls`; the service root without `as()` and a connection
2262
- without a session, with `user_session_required`; the facade of an explicit transaction `db.begin()`, with
2263
- `invalid_query`. The server also refuses a session in the System tenant with `user_session_required` unless the option
2264
- `allowSystemAdmin: true` is passed: this is how a system administrator works, and their `admin` role grants rights to
2265
- the accounts, memberships and rules of the installation.
2266
-
2267
- <!-- run: mcp -->
2268
- ```ts
2269
- import { up, connect } from 'letopis'
2270
- import { createMcpServer } from 'letopis/mcp'
2271
- import { createMcpHandler } from '@modelcontextprotocol/server'
2272
-
2273
- const ADMIN = 'postgres://postgres:secret@localhost:5432/shop'
2274
- const APP = 'postgres://shop_app:app-secret@localhost:5432/shop'
2275
- const r = await up({ dsn: ADMIN, schema: 'shop', quiet: true,
2276
- tenant: { name: 'Chamomile', user: { name: 'Anna', login: 'anna@shop.ru', password: 'Chamomile-2026!' } } })
2277
- const svc = await connect({ dsn: APP, schema: r.schema, apiKey: r.serviceKey! })
2278
-
2279
- // authInfo is the result of the application's check of the request token; it puts the letopis account id into extra
2280
- const mcp = createMcpHandler(async ({ authInfo }) =>
2281
- createMcpServer(await svc.as(String(authInfo?.extra?.account), { tenant: r.tenant!.tenant })))
2282
- // in the application's HTTP handler, after checking the token: return mcp.fetch(request, { authInfo })
2283
- ```
2284
-
2285
- The handler calls the factory on every HTTP request, so `createMcpServer` checks the connection every time (one query
2286
- to the database).
2287
-
2288
- The handler can be checked without a network: the SDK client from the `@modelcontextprotocol/client` package reaches
2289
- it through the application's `fetch`: `new StreamableHTTPClientTransport(new URL('http://localhost/mcp'), { fetch:
2290
- (url, init) => mcp.fetch(new Request(url, init), { authInfo }) })`. This is how the `docs` test checks the example
2291
- above (the `mcp` scenario in `lib/test/docs.test.ts` of the repository): without `write` the client sees three tools,
2292
- and `letopis_model` answers on behalf of Anna in the shop.
2293
-
2294
- ### Raw SQL
2295
-
2296
- <!-- run: guide -->
2297
- ```ts
2298
- const [{ n }] = await db.sql`select count(*)::int as n from "v2.shop".entity where class = 'Product'`
2299
- ```
2300
-
2301
- `` db.sql`…` `` runs a query in one transaction under the session of the connection and under RLS: it sees the same
2302
- rows as chains do. `db.transaction(fn)` gives a postgres.js connection for the whole transaction. On a deadlock or a
2303
- serialization failure, the query and `fn` are retried up to three times, so `fn` must withstand a retry. You can also
2304
- write with raw SQL: the same triggers check the data, and the database functions `merge` and `inc` are available to the
2305
- application.
2306
-
2307
- ### Command line
2308
-
2309
- ```sh
2310
- letopis sync <module> --dsn <dsn> --schema <v2.name> (--token <token> | --api-key <key>) [--apply]
2311
- letopis types --dsn <dsn> --schema <v2.name> (--token <token> | --api-key <key>) [--out <file>]
2312
- letopis load <files…> --dsn <dsn> --schema <v2.name> (--token <token> | --api-key <key>) [--class <Class>]
2313
- [--duplicates error|first|last] [--existing error|skip|update] [--commit-every <N>] [--verify] [--initial]
2314
- letopis verify --dsn <dsn> --schema <v2.name> [--token <token> | --api-key <key>] [--data]
2315
- letopis maintain --dsn <dsn> --schema <v2.name> [--token <token> | --api-key <key>] [--batches <N>]
2316
- letopis describe --dsn <dsn> --schema <v2.name> (--token <token> | --api-key <key>) [--format json|text] [--out <file>]
2317
- letopis import --from <0.21 dsn> --from-schema <v1.name> --dsn <dsn> --schema <v2.name> [--tenant <name>] [--keyless <Class>]…
2318
- [--rename <Old>=<New>]… [--update] [--commit-every <N>] [--partition <name>] [--initial]
2319
- letopis mcp --dsn <dsn> --schema <v2.name> --token <user token> [--write]
2320
- (or LETOPIS_DSN, LETOPIS_SCHEMA, LETOPIS_TOKEN; the service key --api-key is not accepted)
2321
- ```
2322
-
2323
- `letopis types` and `letopis describe` build their output from the session's classes, so without a valid session (the
2324
- token expired, was revoked, is wrong or not given, or the API key is wrong) they exit with code 1 and the `no_session`
2325
- error and leave the `--out` file untouched.
2326
-
2327
- ### Monitoring
2328
-
2329
- - `connect({ onQuery, slowMs })`: for every chain query, the hook gets the mode, the classes, the duration, the row
2330
- count and a slow query flag; without a hook, a slow query prints a warning.
2331
- - `sub.lag()` is the lag of a subscription; the `maintain()` report shows how full the signal queue is.
2332
- - Hot reference targets (an organization that almost everything references) accumulate `for key share` locks and
2333
- MultiXact, as with regular foreign keys. This is worth monitoring.
2334
-
2335
- ### Connection pools and PgBouncer
2336
-
2337
- With PgBouncer in transaction mode, `LISTEN` is not available: use `connect({ listen: false })`. Then the registry is
2338
- re-read with `db.refresh()`, the subscription polls the journal, and the result cache is off or works by term only.
2339
- Prepared statements work with PgBouncer 1.21 and newer; with older versions, pass your own postgres.js pool with
2340
- `prepare: false` in `connect({ sql })`. `maintain()` takes a transaction-level lock in each batch and works behind
2341
- such a pool.
2342
-
2343
- <!-- run: guide -->
2344
- ```ts
2345
- await adm.close()
2346
- await anna.close()
2347
- await svc.close() // and the connections obtained through svc.as()
2348
- ```
2349
-
2350
- ## 14. Migrating from 0.21
2351
-
2352
- 1.0 is a new implementation, not a continuation of 0.21: a 0.21 schema is not upgraded in place. 1.0 is installed into
2353
- a new schema, the `letopis import` command moves the data, and the code is ported with the replacement table in
2354
- [MIGRATION.md](MIGRATION.md) (in Russian). The main differences in behavior: `create` does not overwrite (a taken key
2355
- gives `exists`), `count()` counts entities, paths are `paths()`, permissions and tenant isolation are always on.
2356
-
2357
- ```sh
2358
- npx letopis import --from postgres://…/old --from-schema v1.salon --dsn postgres://…/new --schema v2.salon \
2359
- --tenant salon [--keyless booking] [--rename link=link_] [--update] [--commit-every 100000]
2360
- ```
2361
-
2362
- - The 1.0 schema is installed in advance (`up()`); both connections are admin connections. The source is the 0.21
2363
- tables (`Schema`, `Entity`, `Account`, `Credential`, `Resource`, `Rule`) in any database: PostgreSQL 17 with
2364
- TimescaleDB or a dump of plain tables. The loader (history mode, `existing: 'skip'`, agent `letopis import`) and the
2365
- owner functions do the writing.
2366
- - Classes become `Class` rows in System. fastest-validator rules are translated to JSON Schema (`optional` lets `null`
2367
- through, `$$strict` → `additionalProperties: false`, `record` → `propertyNames`). Ends become roles named after the
2368
- target class; a union of all descendants of a class becomes a polymorphic end. The `Entity` and `link` roots are
2369
- dropped; `description`, `appearance` and `order` move to `meta`. For names taken by 1.0 methods, use `--rename` (by
2370
- default, the name with `_`).
2371
- - Ids: for a class with a key (the v5 rule from 0.21), v5 of the key; for a class without a key, v5 of the old id.
2372
- System data moves to the `--tenant` tenant, accounts to `Account` rows, credentials to `secret` as they are (scrypt,
2373
- keys); one-time codes are not carried over. Resources and rules go to System.
2374
- - The import stops before the first write if two 0.21 objects give one 1.0 id (the data violates the key; use
2375
- `--keyless <Class>`), if a key changed between versions, or if it references a missing object. A dangling reference
2376
- in a live row gives `target_not_found` with the 0.21 object.
2377
- - If the loader rejects the data, the error starts with `import: арендатор …`, and its `detail` is the loader's
2378
- ([Report and errors](#report-and-errors)), except that rows are named by 0.21 objects: `rows` are the violations
2379
- with the `class`, `id` and `rev` of the 0.21 object, `code` and `message`; `total`, `truncated`, `limit` and
2380
- `complete` are as in the loader's report; `notes` are the untranslated rules.
2381
- - A repeat adds what is missing without duplicates; `--update` also adds new versions of changed objects. The report is
2382
- JSON at the end of the output: objects and versions by class in 0.21 and in 1.0, `verify()`, untranslated rules,
2383
- classes without a key, and uuids in `data` that match old ids (the import does not rewrite them). The exit code is 1
2384
- if `verify()` found discrepancies.
2385
-
2386
- ## 15. API reference
2387
-
2388
- The full machine-readable list of exports, methods and error texts is `docs/api-contract.json` in the repository.
2389
-
2390
- ### Package exports
2391
-
2392
- | Entry point | What it exports |
2393
- |---|---|
2394
- | `letopis` | `connect`, `up`, `createTenant`; operators `ne`, `gt`, `gte`, `lt`, `lte`, `between`, `inList`, `like`, `ilike`, `starts`, `ends`, `has`, `hasAny`, `hasAll`, `exists`, `isNull`, `not`, `or`, `isOp`; `inc`, `isInc`, `cursorOf`; `idOf`, `uuidv5`, `v5Name`, `v5Id`; `LetopisError`, `ValidationError`, `fromDbError`; `syncModels`, `generateTypes`, `schemaToTs`; `compileValidator`, `applyDefaults`, `Registry`; `totpCode`, `hashPassword`, `verifyHash`; `describeText`; constants `DDL_REVISION`, `RESERVED_CLASS_NAMES`, `CHAIN_METHODS`, `DB_METHODS`; types `Db`, `DbTx`, `TypedDb`, `Chain`, `Row`, `Filter`, `Cursor`, `ConnectOptions`, `UpOptions`, `LoadOptions`, `LoadReport`, `WatchOptions`, `WatchEvent`, `AuthApi`, `AclApi`, `DiffResult`, `DiffChange`, `BlameResult`, `BlameField`, `Projector`, `ProjectorOptions`, `ProjectorChange`, `ProjectorOut`, `ProjectorRow`, `ProjectorReport`, `ProjectorStatus`, `ProjectorFailure` ([Projector](#projector)), `AggregateInfo` and others |
2395
- | `letopis/model` | `hub`, `link`, `one`, `many`, `End`, types `Infer`, `InferLinks`, `Model`, `ClassDef`, `ClassMeta`, `HistoryPolicy`, `AggregateDef`; `normalizeSchema`, `translatePattern`, type `JsonSchema` |
2396
- | `letopis/mcp` | `createMcpServer(db, options)`: an MCP SDK server with the letopis tools; the type `McpOptions`: `write` adds the `letopis_write` tool (absent by default), `maxRows` is the rows in an answer (100, from 1 to 10,000), `maxBytes` is the bytes in an answer (50,000, from 1,000 to 50,000,000); `allowAclWrite` lets `letopis_write` write the permission classes too (by default `acl_denied`); `allowSystemAdmin` accepts a session in the System tenant (by default `user_session_required`); a value out of range gives `invalid_data`. Needs the `@modelcontextprotocol/server` package ([section 13](#agents-and-mcp)) |
2397
- | CLI `letopis` | `sync`, `types`, `load`, `verify`, `maintain`, `describe`, `import`, `mcp` ([section 13](#13-operations)); `mcp` takes `--dsn`, `--schema`, `--token` (or `LETOPIS_DSN`, `LETOPIS_SCHEMA`, `LETOPIS_TOKEN`) and `--write` |
2398
-
2399
- Removed 0.21 names (`uuidv7`, `LETOPIS_NS` and the methods listed below) throw `removed` with the replacement.
2400
-
2401
- ### The `db` root
2402
-
2403
- | Member | What it does |
2404
- |---|---|
2405
- | `db.ClassName(filter)` | the start of a chain |
2406
- | `schema`, `token`, `tenant`, `registry` | the installation name, the token, the session tenant, the class registry (`resolve`, `find`, `has`, `all`, `family`) |
2407
- | `auth`, `acl` | sign-in, credentials and sessions; permission decisions ([section 10](#10-users-sign-in-and-permissions)) |
2408
- | `as(account, { tenant, owner })` | the same pool on behalf of an account (async) |
2409
- | `begin()`, `commit(tr)`, `rollback(tr)`, `lock(…keys)` | an explicit transaction ([section 8](#8-transactions-batches-and-money)) |
2410
- | `batch(name)` | a queue of plans: `run()`, `discard()`, `size()` |
2411
- | `load(input, options)` | the loader ([section 9](#9-bulk-loading)) |
2412
- | `projector(name, options)` | a projector from the journal: `run()`, `rebuild({ chunked, tenant })`, `status()`, `stop()`, `close()` ([Projectors](#projectors)) |
2413
- | `rebuildAggregates(class)` | repair of the aggregates of a class and its descendants in all tenants: the field anew from live sources, a version only for corrected rows; → their number ([Aggregates in the database](#aggregates-in-the-database)) |
2414
- | `watch(options)`, `cache` | subscription; result cache (`clear()`, `stats()`) |
2415
- | `` sql`…` ``, `transaction(fn)` | raw SQL under the session and RLS |
2416
- | `idOf(class, key)` | the id of an object of a class with a key, in the session tenant |
2417
- | `entity(row \| chain)` | start a path from a ready node |
2418
- | `describe()` | a machine-readable description of the model |
2419
- | `verify(options)`, `maintain(options)`, `reset(options)` | verification, maintenance, reset ([sections 11](#11-history-and-point-in-time-reads) and [13](#13-operations)) |
2420
- | `trimHistory(target, moment)`, `rekeyClass(class, key)` | history trimming; a new class key |
2421
- | `refresh()`, `flushSignals()`, `close()` | re-read the classes; send the signals now; close |
2422
-
2423
- ### Chain
2424
-
2425
- | Group | Methods |
2426
- |---|---|
2427
- | steps | `.ClassName(filter)`, `.AliasName(filter)`, `.RoleName(filter)` |
2428
- | modifiers | `limit(n)`, `offset(n)`, `sort(field, 'asc' \| 'desc')`, `after(cursor)`, `asOf(moment)`, `withDeleted()`, `deep(max)`, `exact()`, `alias(name)`, `tags(…)`, `owner(…)`, `entity(…)`, `forUpdate()`, `cache({ ttl })` |
2429
- | verbs | `create(data)`, `update(patch, { rev })`, `upsert(data)`, `delete({ confirm })`, `anonymize(fields)`, `reclass(class, data)`, `rekey(patch)`, `restore()`, `purge({ confirm })` |
2430
- | slots | after a verb: `.RoleName.set(value)`, `.RoleName.unset()`, `.RoleName.add(value)`, `.RoleName.remove(value)` |
2431
- | terminals | `rows()`, `first()`, `ids()`, `count({ paths })`, `paths()`, `versions({ follow })`, `diff({ from, to, follow })`, `blame({ at, limit })`, `sum(field)`, `avg(field)`, `min(field)`, `max(field)`, `countBy(field)` |
2432
-
2433
- ### `db.auth`
2434
-
2435
- | Method | Parameters → result |
2436
- |---|---|
2437
- | `setPassword` | `{ account, identifier, password, kind?, confirmed?, keepCurrent? }` → credential id |
2438
- | `verifyPassword` | `{ identifier, password, kind?, requireConfirmed?, ttl?, totp? }` → `{ account, token }` or `null` |
2439
- | `issueApiKey`, `verifyApiKey` | `{ account, name?, ttl? }` → `{ key, id }`; `(key)` → sign-in or `null` |
2440
- | `issueKeySecret`, `verifyKeySecret` | `{ account, name? }` → `{ key, secret, id }`; `(key, secret)` → sign-in or `null` |
2441
- | `enrollTotp`, `verifyTotp`, `totpEnabled` | `{ account, issuer?, label? }` → `{ secret, uri }`; `{ account, code, window? }` → `boolean`; `(account)` → `boolean` |
2442
- | `issueOtp`, `verifyOtp` | `{ account, identifier, kind?, ttl?, digits?, maxAttempts? }` → `{ code }`; `{ identifier, code, kind? }` → sign-in or `null` |
2443
- | `link`, `lookup` | `{ account, kind, identifier, meta?, confirmed? }` → id; `{ kind, identifier, requireConfirmed?, ttl?, totp? }` → sign-in or `null` |
2444
- | `credentials`, `revokeCredential`, `setFlags` | `(account)` → credentials without secrets; `(id, { keepCurrent? })`; `(id, { confirmed?, enabled? })` |
2445
- | `sessionFor`, `refresh`, `revoke`, `revokeAll` | `(account, { ttl? })` → token; `({ ttl? })` → the new expiration time (the token stays the same); `()`; `(account?)` → number of sessions |
2446
- | `session` | `(token)` → `{ account, tenant, member, expiresAt, loginAt, methods }` or `null` |
2447
- | `tenants`, `switch` | `()` → `[{ tenant, roles }]`; `(tenant)` |
2448
- | `purgeAccount` | `(account)` → `{ account, rows, versions, memberships, owned }` |
2449
-
2450
- Password, code and TOTP checks, session issuing, session details by token and credential confirmation are called by the
2451
- service (`auth.*` permissions); a regular user gets `acl_denied`. The credentials of System, services and system
2452
- administrators can be changed only by the account itself after a recent interactive sign-in, or by the system
2453
- administrator. The exception is the service's keys: the service issues and revokes its own API keys and "key + secret"
2454
- pairs itself with a session of its own key, but not its other credentials; nobody can revoke the last active key of a
2455
- service (`acl_denied`). With `totp`, `verifyPassword` and `lookup` also check the TOTP code (the `auth.totp` permission)
2456
- and issue a session with the `TOTP` method. The credential kind `kind` is `^[A-Z][A-Z0-9_]{1,31}$` (`PASSWORD`, `EMAIL`,
2457
- `TELEGRAM` …); `SESSION`, `APIKEY`, `KEYSECRET`, `OTP`, `TOTP`, `KEY`, `PROJECTOR` and `LOCK` are taken. The same rule
2458
- applies to the password kind `user.kind` in `createTenant` and `up({ tenant })`.
2459
-
2460
- ### Projector
2461
-
2462
- `db.projector(name, options)` works only on an admin connection ([Projectors](#projectors)). The name is lowercase
2463
- Latin letters, digits and `_.:-`, up to 64 characters; it is also the `derivedBy` of the derived classes. Wrong options
2464
- give `invalid_data`.
2465
-
2466
- | Member | What it does |
2467
- |---|---|
2468
- | options `ProjectorOptions` | `from`, `into`: classes with descendants; `version`: an integer from 1; `handle(changes, out)`; `withBefore`; `onExpired: 'rebuild' \| 'error'` (`'rebuild'` rebuilds in batches); `batch`: changes in a batch (500, from 1 to 10,000); `pollMs`: fallback polling (1000 ms); `lockTimeout`: waiting for another transaction's row lock (5000 ms, 0 — no limit); `lockRetries`: passes in a row before a tenant with another transaction's lock drops out (0 — never); `tenants`: ids of the participant tenants |
2469
- | `run({ once })` | catch up with the journal (`once`) or follow it until `stop()` → `ProjectorReport` |
2470
- | `rebuild({ chunked, tenant })` | rebuild from the current state: atomically, in batches or for one tenant → `ProjectorReport` |
2471
- | `status()` | → `ProjectorStatus`: `{ name, state, version, cursor, at, lag, failed, rebuildingTenants }`; `failed` is `{ [tenant id]: ProjectorFailure }`, where `ProjectorFailure` is `{ cursor, code, message }` |
2472
- | `stop()`, `close()` | stop `run()` after the current batch; the same plus unsubscribing from signals |
2473
- | `ProjectorReport` | `{ batches, changes, created, updated, deleted, unchanged, skipped, rebuilds, locked }` |
2474
- | `ProjectorChange` | `{ cursor, id, class, tenant, op, rev, at, author, row }` and, when present, `agent`, `reason`, `moved`, `before`, `rebuild` |
2475
- | `ProjectorOut` | `upsert(class, row, { tenant })`, `inc(class, row, deltas, { tenant })`, `delete(class, key or id, { tenant })`, `get(class, key or id, { tenant })` → `Row \| null`; a row is a `ProjectorRow` in the loader format; for a class with a key `delete` and `get` go by the key, an id only for a class without a key |
2476
-
2477
- ### Result row
2478
-
2479
- | Field | Meaning |
2480
- |---|---|
2481
- | `id`, `class`, `rev` | id, class, version number (for `update(patch, { rev })`) |
2482
- | `tenant`, `owner` | the tenant and the owner of the row |
2483
- | `links`, `data`, `tags` | ends, fields, tags |
2484
- | `at`, `author`, `agent`, `op`, `reason` | version time (ISO, UTC, microseconds), author, agent (the service under impersonation), kind of operation, reason |
2485
- | `moved` | the reference to the old or the new id after `reclass` and `rekey` |
2486
- | `$deleted` | tombstone: a deleted entity or a deletion version |
2487
- | `$depth` | the depth of a node in `deep()`; the cascade level in a deletion preview |
2488
- | `$upsert`, `$action`, `$versions`, `$purged` | what `upsert` did; what the deletion will do; the number of versions in `purge`; the history is erased |
2489
-
2490
- ### Reserved names
2491
-
2492
- A method name hides a class only where that method exists: after a step — chain methods, at the root `db` — its
2493
- methods, in a batch — its own. So a hub named like a chain-only method (for example, `count`) is reachable as the first
2494
- step `db.count(…)` but not after another step, and a hub named like a root method cannot be the first step. The
2495
- database does not accept a link with any of these names (`reserved_names()`; the installer fills in the list from
2496
- `RESERVED_CLASS_NAMES`). Do not give classes these names.
2497
-
2498
- - chain methods: `then`, `entity`, `paths`, `rows`, `first`, `ids`, `count`, `limit`, `offset`, `sort`, `asOf`,
2499
- `withDeleted`, `deep`, `exact`, `sum`, `avg`, `min`, `max`, `countBy`, `after`, `versions`, `create`, `update`,
2500
- `upsert`, `set`, `unset`, `add`, `remove`, `delete`, `anonymize`, `reclass`, `restore`, `purge`, `alias`, `tags`,
2501
- `owner`, `forUpdate`, `cache`, `rekey`, `diff`, `blame`; guards of the removed API: `run`, `execute`, `account`;
2502
- - the `db` root: `as`, `begin`, `commit`, `rollback`, `lock`, `batch`, `watch`, `close`, `registry`, `sql`, `load`,
2503
- `verify`, `maintain`, `reset`, `describe`, `auth`, `acl`, `idOf`, `refresh`, `transaction`, `flushSignals`,
2504
- `schema`, `token`, `tenant`, `trimHistory`, `rekeyClass`, `cache`, `projector`, `rebuildAggregates`; guards: `reloadSchema`,
2505
- `accounts`, `credentials`, `resources`, `rules`, `compact`;
2506
- - batch: `discard`, `size`.
2507
-
2508
- ## 16. Errors
2509
-
2510
- The text of an error is `letopis: <code>: <message>`. The code is in `err.code`, the details are in `err.detail`.
2511
- Errors are `LetopisError` objects; data errors are `ValidationError` with `err.issues: [{ path, keyword, message }]`.
2512
-
2513
- Database codes. Raw SQL gets them too: SQLSTATE class `LT`, the details are JSON in `DETAIL`.
2514
-
2515
- | Code | SQLSTATE | When |
2516
- |---|---|---|
2517
- | `no_session` | LT001 | there is no valid session |
2518
- | `target_not_found` | LT002 | the target is not found; the same for a nonexistent, a foreign and an invisible target |
2519
- | `invalid_data` | LT003 | the data does not match the class description; `err.issues` gives the path, keyword and message |
2520
- | `undeclared_end` | LT004 | the end is not declared in the class |
2521
- | `tenant_denied` | LT005 | a foreign tenant or no membership |
2522
- | `acl_denied` | LT006 | denied by the access rules |
2523
- | `id_mismatch`, `id_from_db` | LT007, LT008 | the id is not v5 of the key; the id of a class without a key is assigned by the database |
2524
- | `immutable_key` | LT009 | a key field, the id, the tenant and the owner do not change (a new key is `rekey`) |
2525
- | `invalid_class` | LT010 | the class description is rejected, the class is not found or is abstract; a class with rows, descendants or a history of rows is not deleted |
2526
- | `use_reclass`, `reclass_denied` | LT011, LT012 | changing the class with `update` when the old or the new class has a key: the id changes, so use `reclass()`; the roles of referencing rows do not accept the new class |
2527
- | `tightening_conflict` | LT013 | rows prevent tightening the description |
2528
- | `isolation_level` | LT014 | deleting, changing the class and changing the key work only in `read committed` |
2529
- | `recreated_in_statement` | LT015 | an id was deleted and created again in one statement |
2530
- | `cursor_expired` | LT016 | the cursor of a subscription or a projector is older than the `all` tier of the retention policy |
2531
- | `delete_restricted` | LT017 | referencing rows prevent the deletion (`onDelete: 'restrict'`) |
2532
- | `not_deleted` | LT018 | `restore` and `purge` work only for deleted objects |
2533
- | `reset_disabled` | LT019 | reset is disabled in the installation (`allowReset`) |
2534
- | `confirm_mismatch` | LT020 | the reset confirmation does not match the schema name |
2535
- | `derived_class` | LT021 | a row of a derived class (`derivedBy`) is written, edited or deleted by someone other than its projector; an end leads to such a row or to a derived class; a projector writes to a class outside its `into` or has found no tenant in which every `into` class is marked with its `derivedBy` |
2536
- | `aggregate_field` | LT022 | the application passed or changed an aggregate field (`aggregate`): the database calculates it |
2537
- | `projector_pending` | LT023 | `purge` of a deleted object whose deletion a projector of its class has not read yet, or `trimHistory` whose first remaining version it has not read yet: run the projector first |
2538
-
2539
- Library codes:
2540
-
2541
- | Code | When |
2542
- |---|---|
2543
- | `exists` | the object already exists: `create` (`detail.id` is its id), `rekey` (the new id is taken) or the loader |
2544
- | `duplicate`, `deleted` | the loader: a repeated id in the load; an object with this id existed before. The loader throws the code of the first problem row, the first 100 are in `detail.rows`, the total is `detail.total` |
2545
- | `conflict` | `update(patch, { rev })`: the row was changed in the meantime (`detail`: `id`, `rev`, `expected`) |
2546
- | `rev_ambiguous` | `{ rev }` on a step that does not have exactly one target |
2547
- | `tx_required` | `forUpdate()`, `lock()`, `commit()` or `rollback()` outside an active `db.begin()` |
2548
- | `lock_unsupported` | `forUpdate()` with an aggregate, `asOf`, `versions`, `withDeleted`, `deep` or a verb |
2549
- | `commit_unknown` | the connection dropped during `COMMIT`: the outcome is unknown |
2550
- | `tx_rolled_back` | PostgreSQL answered `ROLLBACK` to `COMMIT`: the error was caught inside `tr.transaction(fn)` |
2551
- | `no_key` | `upsert`, `rekey` or `idOf` on a class without a key |
2552
- | `no_path` | there is no hop between the classes of neighboring steps |
2553
- | `invalid_query` | the chain or the API is used incorrectly (a step modifier `deep`, `exact`, `alias`, `tags` or `owner` after a verb, a selection modifier between a verb and a `rows`, `first`, `ids`, `count` or `paths` terminal, a slot not after a verb, an `after()` cursor with a value of a different type than the `sort` field, without `v` or with an `id` that is not a uuid, and so on) |
2554
- | `admin_required` | the loader, a projector and `rebuildAggregates` need an admin connection |
2555
- | `rebuild_required` | the projection was built by another version of the projector (`version`): a rebuild with `rebuild()` is needed; `rebuild({ tenant })` while there is no projection or a batch rebuild is in progress |
2556
- | `projector_busy` | a rebuild cannot start: the projector is running in another process right now |
2557
- | `not_empty` | the initial load (`initial`) goes only into an empty schema |
2558
- | `class_not_loadable` | the loader does not accept `Class` rows |
2559
- | `bypass_rls` | `connect()` with a role that bypasses RLS, without `allowBypassRls`; `createMcpServer` with a connection that bypasses RLS |
2560
- | `user_session_required` | `createMcpServer` and `letopis mcp`: a user session is needed — the connection has no session, it is the service root without `as()`, a session in the System tenant without `allowSystemAdmin`, or the command line got the service key or no token |
2561
- | `confirmation_required` | `letopis_write` (MCP): a deletion without a human confirmation — the client has no elicitation, the human declined, the answer belongs to another operation, or the data changed after the preview |
2562
- | `removed` | a removed 0.21 API; the text gives the replacement |
2563
-
2564
- A PostgreSQL permission denial (`42501`) comes, for example, on an `upsert` over someone else's row that you may not
2565
- edit.
2566
-
2567
- ## 17. Performance
2568
-
2569
- Benchmark of 1.0 against 0.21 on the same datasets (`cd lib && npx tsx bench/compare.bench.mjs`, 2026-10-09,
2570
- PostgreSQL 18, and 0.21 on PostgreSQL 17 with TimescaleDB, one machine): salon has ~980 thousand rows (in 0.21 they
2571
- belong to System without isolation, in 1.0 they are in a tenant under RLS and permissions); tenants has 500 thousand
2572
- customers with 2 versions each, across ten tenants. The numbers depend on the machine; what matters is the ratio.
2573
-
2574
- | Measurement | 1.0 | 0.21 | Unit | Note |
2575
- |---|---|---|---|---|
2576
- | loading the salon dataset | 6359 | 3609 | rows/s | 1.0: the loader in history mode with checks; 0.21: raw INSERTs into Entity |
2577
- | loading the tenants dataset | 8566 | 35186 | rows/s | 0.21: INSERT … SELECT with triggers disabled |
2578
- | row-by-row create | 128 | 156 | rows/s | |
2579
- | read by key | 2.2 | 3.81 | ms (p50) | |
2580
- | three-step chain (count) | 222.79 | 1233.83 | ms (p50) | organization → specialists → bookings |
2581
- | point-in-time query (asOf) | 202.77 | 327.98 | ms (p50) | a specialist's bookings as of mid-year |
2582
- | 20 concurrent writers | 982 | 522 | rows/s | |
2583
- | class change (reclass) | 9.5 | — | ms (p50) | 0.21 has no such verb |
2584
- | count() of bookings: a tenant under RLS and permissions | 272.25 | 890.73 | ms (p50) | 0.21: without isolation and permissions (enforceAccount: false); 1.0 without RLS (admin): 18.59 ms |
2585
- | count() of customers of tenant 1 (50.0%) | 114.09 | 667.19 | ms (p50) | |
2586
- | count() of customers of tenant 3 (10.0%) | 23.8 | 229.94 | ms (p50) | |
2587
- | count() of customers of tenant 8 (1.0%) | 5.3 | 88.02 | ms (p50) | |
2588
- | count() of customers of tenant 10 (0.3%) | 3.54 | 26.08 | ms (p50) | |
2589
-
2590
- A full import of the salon dataset from 0.21 (`npm run test:import-salon`): 539,580 objects and 980,180 versions in
2591
- 193 s (5,084 rows/s); the per-class counters matched, and `verify()` is clean. Checkpoint B tuning (revision 5): the
2592
- journal index on reference targets, `log_ends`, made an `asOf` read across a hop 16 times faster (3.37 s → 0.20 s) and
2593
- costs the loader 15–17% of its speed. Reading `letopis.changed` and `COMMIT` in one pipeline removed a round trip to the
2594
- database from every write (row-by-row writes: 119 → 128–147 rows/s in different runs). Row-by-row `create` is still
2595
- 12–18% slower than 0.21 (the measurements vary): most of the time goes to the check triggers in the database and to the
2596
- commit with a disk write.
2597
-
2598
- Initial load (`cd lib && npx tsx bench/initial-load.mjs`, 2026-10-11, median of three rounds): the regular loader
2599
- writes 1.07 million objects of the salon dataset without history in 169 s (6,351 rows/s, 4.0 GB of WAL), the initial
2600
- load in 77 s (13,902 rows/s, 1.4 GB of WAL). Most of the difference comes from the GIN indexes: maintaining them during
2601
- the load costs 36 s, building them at the end about 5 s. Preparing rows in the loader itself (checks, ends, ids) runs
2602
- alongside the `COPY` of the previous batch and now sets the pace: about 60 s of the 77.
2603
-
2604
- Derived data in 1.2 (`cd lib && npx tsx bench/derived.bench.mjs`, 2026-10-27, PostgreSQL 18.6, one machine; for
2605
- aggregates, the median of three rounds; the projector rows are a repeat of series 2 and 3 after the folding of `inc` in
2606
- `out`, `BENCH_SERIES=2,3`):
2607
-
2608
- | Measurement | 1.2 | For comparison |
2609
- |---|---|---|
2610
- | an operation with an aggregate, one writer | 94 operations/s, p50 10.7 ms | without an aggregate 137 and 6.9 ms; the `inc` recipe 69 |
2611
- | operations on one wallet with an aggregate, 20 writers | 152 operations/s, p95 0.47 s | without an aggregate 1,987; the `inc` recipe 190 |
2612
- | operations on 20 wallets with an aggregate, 20 writers | 1,379 operations/s | without an aggregate 2,023 |
2613
- | projector catch-up, `out.inc` for every change | 6,076 and 33,395 changes/s (`batch` 500 and 2000) | before the folding of `inc` in `out`: 992 and 1,203 |
2614
- | rebuild of 75 thousand rows | atomic 13.3–13.5 s, in batches 13.0–13.2 s | before the folding of `inc` in `out`: 101 s and 58–63 s |
2615
- | delay of a subscription event during a rebuild | atomic up to 13.4 s, in batches up to 0.21 s | without a rebuild up to 0.12 s |
2616
-
2617
- The aggregate triggers fire on every write statement, but for classes without aggregates the measurement does not see
2618
- their cost: the p50 difference is from −0.28 to +0.21 ms. The benchmark against 1.0.1 (`bench/vs-101.bench.mjs`, four
2619
- runs) showed no write or read regressions against 1.1: two outliers did not repeat in separate checks.
2620
-
2621
- ## 18. Limitations
2622
-
2623
- This section lists what letopis 1.4 deliberately does not do or does with caveats, and where you need to watch the
2624
- behavior.
2625
-
2626
- **Platform**
2627
-
2628
- - PostgreSQL 18 or newer is required; there are no workarounds for 15–17. AWS RDS, Google Cloud SQL, Azure Flexible
2629
- Server and Neon support 18, while Supabase, as of October 2026, offers only 15 and 17, so letopis will not work there.
2630
- - The database must be in UTF8 encoding (`casefold()` and the `pg_unicode_fast` collation). If a new major PostgreSQL
2631
- version updates the Unicode tables, the unique index of credential identifiers is rebuilt with the `reindex` command.
2632
- - PGlite has one superuser connection: no concurrent writers and no `LISTEN`, and RLS does not apply
2633
- (`allowBypassRls: true`); it does not replace `postgres:18` for permissions, race tests and CI.
2634
- - A pool in transaction mode (PgBouncer): `LISTEN` is not available, so use `connect({ listen: false })`; the
2635
- subscription polls the journal, and the result cache is off. Prepared statements work with PgBouncer 1.21; with older
2636
- versions, pass a postgres.js pool with `prepare: false` in `connect({ sql })`.
2637
- - The NUL character (`\u0000`) is impossible in data strings: PostgreSQL does not store it in `jsonb`; such data is
2638
- rejected already when the JSON is parsed (in the official JSON-Schema-Test-Suite, the "nul characters" groups are
2639
- skipped).
2640
- - On PostgreSQL before 18.6, a parallel GIN index build could write a wrong row count into the table statistics
2641
- (`pg_class.reltuples`), after which autovacuum stopped processing the table. That is why the initial load always ends
2642
- with `ANALYZE`, but it is better to upgrade the server to 18.6 or newer.
2643
-
2644
- **Writing and integrity**
2645
-
2646
- - Row-by-row writing costs more than a plain insert: triggers check every row (data, ends, id, journal, hash, RLS).
2647
- Row-by-row `create` is 12–18% slower than 0.21 ([section 17](#17-performance)); for bulk writing there is the loader.
2648
- - Deleting, changing the class and changing the key work only in `read committed` (`isolation_level`); library
2649
- transactions always open at this level.
2650
- - PostgreSQL does not run `COPY` into tables under RLS; bulk loading works only through the loader on an admin
2651
- connection.
2652
- - The initial load closes the schema's tables for the whole load, for writing and for reading: it is meant for an
2653
- empty schema (seeding, migration, import), not for a running system. Its gain is in the database (indexes and
2654
- constraints are built once); it does not speed up the row preparation in the loader, only runs it alongside `COPY`,
2655
- so on a million objects it is about 2.2 times faster than the regular load.
2656
- - The database trusts the loader: the data, ends and ids of loaded rows are checked by the library code (the TypeScript
2657
- validator), not by the triggers; the database checks only the targets outside the load, the tenants and the owners,
2658
- with one query at the end. The safeguards: the loader is available only to an admin, differential tests of the
2659
- validator against the database, `load({ verify: true })` and `db.verify({ data: true })`. Keep the admin connection
2660
- in a separate loading process, not in the web application: an SQL injection there would get the owner's rights.
2661
- - The schema owner and a superuser can disable the triggers and change anything. letopis does not prevent this; it
2662
- detects it: `verify()` recomputes the hash chains, `verify({ data: true })` checks the rows again, and the anchor (a
2663
- hash of all chain heads) should be kept outside the database.
2664
- - The database checks a subset of JSON Schema: Zod `refine`, `superRefine` and `transform` work only in the library,
2665
- and the database does not accept a description with an unknown keyword; `pattern` is a limited subset of regular
2666
- expressions shared by Zod, the validator and the database. The output of `z.toJSONSchema()` changes between Zod
2667
- versions: the database stores an already normalized schema, and a `syncModels` dry run shows the difference before it
2668
- is applied.
2669
- - The key is the only way to get uniqueness: the id is computed from the key values without normalization, so the same
2670
- value written in a different way (an email in a different case, a time with and without milliseconds) gives different
2671
- ids. The model sets the format of a key field, and the database checks it.
2672
- - Frequent changes of a key value (email, phone) require `rekey`. Values that change often and personal values are
2673
- better moved to a separate hub with a link: an id from an email can be found by brute force and stays in references and in the journal
2674
- even after `anonymize`, while a separate hub is deleted together with the link through `purge`.
2675
- - `create` with a taken key answers `exists` with the id, and `upsert` without permission on the existing row answers
2676
- with a `42501` denial: this reveals that the key is taken, but only within your own tenant, because in other tenants
2677
- the same keys give other ids.
2678
- - Changing the class and the key rewrites all incoming references (and, recursively, the links whose key includes this
2679
- end), so its cost is proportional to the number of references. Tightening a class description and `rekeyClass` pause
2680
- writes to the family while they check or move rows.
2681
- - `rekeyClass` moves rows in one transaction under an exclusive lock of the family, not in batches: between batches,
2682
- rows with old ids would break the rule "the id is v5 of the key".
2683
- - v5 ids land in the primary key index in scattered places; on very large tables with heavy inserts this is more
2684
- noticeable than with sequential ids.
2685
- - Hot reference targets (an organization that almost everything references) accumulate `for key share` locks and
2686
- MultiXact, as with regular foreign keys; this needs monitoring. Operations on one wallet go one after another under a
2687
- lock of its row ([section 8](#8-transactions-batches-and-money), "Recipe: wallet").
2688
- - A link cannot have a reserved name, and a hub with such a name is not reachable everywhere: for example, `count` —
2689
- only as the first step (["Reserved names"](#reserved-names)).
2690
-
2691
- **Access**
2692
-
2693
- - The service key is the root of trust: its compromise allows acting on behalf of any enabled user of the
2694
- installation except System, services and system administrators; the access rules do not narrow this scope. The key
2695
- is reissued (`db.auth.issueApiKey`) and revoked (`db.auth.revokeCredential`) by the service itself with a session of
2696
- its own key, or by the system administrator; with its key the service also issues itself new keys, so after a leak
2697
- revoke all its keys except the new one (`credentials` lists them). Nobody can revoke the last active key of a
2698
- service: to cut a service off, set `enabled: false` on its account. A database password alone, without a token,
2699
- gives nothing.
2700
- - The permission pass reveals its own plan to the client (the classes and rule conditions that apply to it).
2701
- - Raw SQL without the permission pass pays for the permission plan on every query: 0.3–1.3 ms depending on the number
2702
- of rules.
2703
- - The end names `User`, `Tenant` (membership), `Group` and `Object` (rule) are taken by system classes across the
2704
- whole schema.
2705
- - OTP and TOTP attempt counters are rolled back together with the caller's transaction: the service limits the rate of
2706
- attempts.
2707
- - When users link a taken identifier themselves, the answer is `acl_denied`, so a user with a recent sign-in learns
2708
- this way that the identifier is taken. Permission changes in all tenants are serialized through one counter row.
2709
- - Refusal texts name the class, role and id of a referencing row only if the actor can see it: a cascade delete,
2710
- unsetting an end, rewriting references on a class or key change, and a `parent` change conflict
2711
- (`tightening_conflict`). The number of invisible rows that block a deletion or a
2712
- class change (`reclass_denied`), and the existence of an account by id (as a reference target), are not hidden.
2713
- - Tightening a class description and `rekeyClass` check all rows of the family, so they need the `READ` permission
2714
- without conditions on all rows of the class and its descendants: otherwise a trial tightening (`maximum`, `enum`,
2715
- `pattern`) would answer with the number of invisible rows that do not match it. A system administrator who tightens
2716
- a System class gets the number of violating rows of other tenants (`others`): this is the installation level.
2717
- - Class metadata (`meta`) is visible to anyone who has a session: anyone can read `Class` rows. Secrets do not belong
2718
- there.
2719
- - An application role can slow the installation down for everyone: `LOCK TABLE entity`, advisory locks with the keys
2720
- of the application's `lock()`, fake `NOTIFY` signals (they only clear the cache), frequent permission changes (one
2721
- counter row per installation). The global journal number reveals the overall write activity. Internal locks — class
2722
- families, projectors and maintenance — are rows of the `secret` table, not advisory locks: neither the application
2723
- role nor any other database role can take them. A family writer passes the tightening gate in a subtransaction — one
2724
- per transaction for the first write to the family.
2725
-
2726
- **History, signals, cache**
2727
-
2728
- - An `asOf` read is rebuilt from the journal; only the current state is cheap. In the thinned part of the journal, a
2729
- point-in-time read shows the last kept version before the moment: its precision there equals the period of the tier,
2730
- a day, a week, a month or a year. A history import into a class with a policy is thinned right away, and a subscriber
2731
- may not see the old versions.
2732
- - The thinned part of the journal proves neither that the history is complete nor that old versions are authentic: for
2733
- a class with a policy, `verify()` cannot tell the tampering of a version older than the `all` tier from thinning.
2734
- - The journal has no partitions: thinning deletes versions in batches, and the table file does not shrink by itself
2735
- (new versions take the space after autovacuum); to return the space to the system, use `VACUUM FULL` or `pg_repack`
2736
- in a maintenance window.
2737
- - Signals reveal activity: PostgreSQL lets any role listen to a channel. In the `class` mode one sees when and to which
2738
- classes writes happen; `tenant` adds tenant pseudonyms; `opaque` hides what was written but not when: the write
2739
- time and frequency per pseudonym stay visible, and a deterministic token is the basis of frequency attacks. Write
2740
- times are visible without signals too: `pg_stat_all_tables` and `pg_locks` are open to PUBLIC. The `none` mode
2741
- removes signals, but then a revoked session leaves the cache of other processes only by term, and a new class
2742
- appears in their registry within 5 seconds. Real protection from foreign roles is a separate database. A fake
2743
- signal only clears the cache one extra time: on `*` and unknown tokens the registry is re-read not more often than
2744
- once in 5 seconds, and a fake class change signal re-reads it like a real one, but with at most one re-read at a
2745
- time.
2746
- - The signal mode is changed by `up({ signals })`, and the processes have to be restarted after that: a process reads
2747
- the mode and the signal key on connect (it treats a payload of another mode as foreign and clears the whole cache).
2748
- - PostgreSQL below 18.2 could truncate CLOG that unread signals still need: `up()` warns about it.
2749
- - The `NOTIFY` queue (8 GB) is shared by the whole cluster: a stuck listener fills it up, and then transactions with
2750
- `NOTIFY` fail on commit. The library reads signals right away, and `maintain()` warns when the queue is more than
2751
- half full.
2752
- - PostgreSQL commits transactions with `NOTIFY` one at a time: a commit holds the cluster-wide lock of the signal queue
2753
- together with the disk write. A raw SQL write on its own connection sends the signal from a trigger in the same
2754
- transaction, so concurrent raw SQL writers commit one after another: on the benchmark machine, 20 such writers did
2755
- about 520 inserts per second. Library transactions, `db.sql` and `db.transaction()` included, commit the data
2756
- without `NOTIFY` and send the signals later, separately, so their writers do not wait in this queue: 20 library
2757
- writers did about 2,000 operations per second.
2758
- - The subscription horizon is shared by the whole cluster: a long write transaction delays the events of all
2759
- subscribers (`sub.lag()`); long operations are better done in batches.
2760
- - Between processes, the result cache goes stale for the signal sending interval (100 ms) plus delivery, and for the
2761
- time of a listener reconnect: postgres.js does not report a broken listener connection, so the cache works until the
2762
- reconnect and is cleared completely after it. The `ttl` term is a safeguard.
2763
- - A "from now" subscription cursor carries a snapshot of the subscription moment until it passes that boundary: such a
2764
- cursor is longer than a regular one.
2765
-
2766
- **Projectors**
2767
-
2768
- - A projector is an admin process: it works only on an admin connection, reads the journal of all tenants bypassing
2769
- RLS and writes its results as the owner role. Keep it in a separate process, like the loader. A participant of a
2770
- projector is a tenant in which all `into` classes are marked with its `derivedBy`; only an administrator may mark a
2771
- class (the `letopis.projector` permission).
2772
- - The totals of a derived class are seen by everyone whom the tenant rules give `READ` on it — even by those who do not
2773
- read the source rows: decide who reads it with rules on the derived class itself. With `history: true`, its journal
2774
- shows the change of every batch, that is, the contribution of single source rows too.
2775
- - Only their projector writes the rows of derived classes: the application, raw SQL and the loader get
2776
- `derived_class`, and ends may not refer to such rows.
2777
- - An atomic rebuild is one long write transaction: while it runs, the subscription horizon stands still, and events
2778
- are delayed for the subscribers and projectors of all processes. On the benchmark machine, a rebuild of 75 thousand
2779
- rows took 13 s, and the subscriber got no event until it ended. That is why `run()` itself rebuilds in batches, and
2780
- only an explicit `rebuild()` runs an atomic rebuild.
2781
- - A batch rebuild holds the horizon only for the time of a batch (in the same measurements, events were delayed by up
2782
- to 0.2 s), but the derived data is incomplete until it ends: the rows are deleted at once and reappear as the
2783
- rebuild goes on.
2784
- - A projector sees only the changes from the journal, so a class with `history: false` cannot be a source: a tenant
2785
- with such a source drops out of the projector (`invalid_class`) until the journal of the class is turned on and the
2786
- tenant is rebuilt.
2787
- - Thinning the journal older than the `all` tier leads to a rebuild: a projector that lags behind by more than that
2788
- tier gets `cursor_expired` or, with `onExpired: 'rebuild'`, rebuilds itself. The tier is per tenant: if it has
2789
- expired only for some participants, only they drop out and rebuild.
2790
- - A projector works only on a schema with the engine revision the library expects; on a schema of another revision it
2791
- answers `invalid_data`.
2792
- - `purge()` waits for projectors: while a projector whose sources include the class of the object, and which has a
2793
- derived class in the object's tenant, has not read the deletion, `purge` answers `projector_pending`. A projector
2794
- that is no longer run holds `purge` of its sources while the tenant has its derived classes: run it or delete them.
2795
- `trimHistory` waits the same way: while the projector has not read the first remaining version, trimming answers
2796
- `projector_pending`.
2797
-
2798
- **Aggregates in the database**
2799
-
2800
- - An aggregate queues the writers of one object: every operation locks the target row and writes its version. On the
2801
- benchmark machine, 20 writers of one wallet with an aggregate did 152 operations per second (p95 is 0.47 s), without
2802
- an aggregate 1,987. For hot counters, use a projector that folds a batch.
2803
- - `reclass` and `rekey` accumulate contributions until the end of the move, while a delete cascade and the loader apply
2804
- them per statement: the class rule is checked on the total of each cascade statement and of each loader batch. So a
2805
- load in which debits come in a batch before top-ups gets `invalid_data`, although the total of the whole load is
2806
- valid. Every batch that touches a target writes its version.
2807
- - A tenant aggregate does not count rows of a System class, and the field of a System row counts only System sources.
2808
-
2809
- **Agents and MCP**
2810
-
2811
- - The `<untrusted-data-…>` wrapper is a mitigation, not a protection: the model may follow the text inside it. The
2812
- protection is the rights in the database: the agent reads and writes only what its user is allowed to. In
2813
- `structuredContent`, the database data lies in `data` and `error` next to the `notice`, without marks: a client that
2814
- gives the model that instead of the text passes the data with the note but without a random mark.
2815
- - By default the server only reads; writing is `write: true` (`--write`); deletion is only with a human confirmation.
2816
- The server writes the permission classes only with `allowAclWrite` and accepts a session in the System tenant only
2817
- with `allowSystemAdmin`; `create`, `update` and `upsert` of the other classes need no human confirmation.
2818
- - The HTTP handler serves revision 2025 statelessly: the per-request instance does not know the client's
2819
- capabilities, so deletion through it is refused (`confirmation_required`). Delete over stdio or in revision
2820
- 2026-07-28.
2821
- - A confirmation in revision 2026-07-28 is signed with a process key: a retried call that lands in another process of
2822
- the application (several instances behind a load balancer) is refused with `confirmation_required`.
2823
- - `versions` returns the latest versions within `maxRows` and `maxBytes` (at most `maxRows` + 1 versions are read
2824
- from the database) and `more`; earlier ones, through `diff` and `blame` with `from`, `to`, `at`. A mode where the
2825
- agent writes code against the API (code mode) is not part of the package.
2826
-
2827
- **Import from 0.21**
2828
-
2829
- - 1.0 is a new major version, not a continuation of 0.21: `count()`, `paths()`, `create`, roles, multiple ends and
2830
- `reclass` behave in a new way, and the data is moved by an import; see [MIGRATION.md](MIGRATION.md) (in Russian).
2831
- - Import: the steps (classes, accounts, credentials, permissions, the data of each tenant) are separate transactions,
2832
- and the source is read into memory entirely; a row owner that is not found among the accounts becomes the tenant;
2833
- 0.21 access rules move to System and apply in all tenants; the `onDelete` of ends is `cascade`, the way 0.21 deleted.
2834
-
2835
- ## 19. Tests
2836
-
2837
- This section is for those who develop letopis itself. Run: `cd lib && npm test`. The harness starts the
2838
- `letopis-pg18` container (`postgres:18`, port 15433) by itself if the database does not respond; each test file works in its own schema `v2.t_<name>`.
2839
-
2840
- CI runs no tests: before a push and a release, everything that checks behavior runs as one command,
2841
- `cd lib && npm run prerelease` — typecheck, the documentation check, all tests, the site examples against snapshots, the
2842
- sandbox, the service scenarios, a quick soak run (15 minutes of mixed load) and the package check with a live run, in
2843
- order; the first red step stops the run. The same load for a day: `cd lib && npm run soak` (duration — `SOAK_HOURS`,
2844
- 24 hours by default). A test for a fix is named after its test-plan item ("plan 2.0, 7.3 No. 1") and fails without the fix.
2845
-
2846
- ```
2847
- # connect — connecting under a session, refusing roles that bypass RLS, the registry, the deferred signal, error translation (stage 2)
2848
- # context — sessions, service sign-in by key, the RLS policies of stage 1 (stage 1)
2849
- # access-rest — the remaining rows of stage 5: loading on behalf of a user, balance No. 20, 22, 29, upsert and { rev } under ACL, inlining of acl_ok (stage 5)
2850
- # aggregate — aggregates in the database: 20 writers of one wallet, raw SQL, a cascade, reclass, restore, the loader, account deletion, restore and rekey of the target, a description change under writes and race No. 18, aggregate_field, repair and verify, a System row (plan 1.1, stage 4b)
2851
- # acl — ACL permissions: the 0.21 suite ported, row conditions, weight and default deny, cascade, deny on a descendant, deep, an invisible target, raw SQL versus the library; a change of parent and alias of a class only with permissions on all its rows, an alias is not the name of another class, a name is not another class's alias (stage 5; security review No. 2)
2852
- # acl-parity — 1000 random rule sets: the database permission plan and endpoint decisions versus decide from 0.21 (stage 5)
2853
- # auth — credentials (password, key, key + secret, OTP, TOTP, identity), timing equalization, case, session revocation, recent sign-in (stage 5)
2854
- # balance — top-up and debit: idempotency by opId, races, deadlock, connection drop (TCP proxy), adjustment under { rev } and forUpdate (stage 4)
2855
- # chain-api — guards of the removed API, reserved names, build errors, the plan cache, onQuery (stage 3)
2856
- # chain-read — read chains on the demo domain: hops, roles, multiple ends, 18 operators, keyset, aggregates, deep, asOf, withDeleted, pivot (stage 3)
2857
- # chain-types — types of steps, rows and filters from models: connect({ models }) (stages 3, 4)
2858
- # chain-write — writing with chains: create, update, upsert, delete with preview, anonymize, reclass, slots, inc, { rev }, fan-out, plan rollback (stage 4)
2859
- # demo — the booking demo domain on Zod: seed from models, writing data, id by key, agreement between Zod and the database (stage 2)
2860
- # docs — documentation examples are executed: the marked blocks of the README and the cheatsheet (Russian and English) run as scenarios against PostgreSQL 18 (stage 8)
2861
- # engine-attacks — raw SQL attacks from the application role: the rows of §7.3.1 for stages 1 and 3 with specific outcomes; the signal key without permission, flooding with fake signals, Class included; writing to a derived class and a reference to it, a projector checkpoint through the credential functions; an agent through MCP without --write and without a human confirmation; a change of parent, alias and derivedBy of a class with the WRITE permission on Class alone, deleting and re-creating a class with a history of rows, a trial tightening as an oracle, reclass without DELETE and rewriting references deeper than the first level, a retention policy with the WRITE permission on Class alone; an agent through MCP grants a membership or a rule, a system administrator session; a tenant user creates a derived class on their own, `rekeyClass` of an ancestor of a derived class; moving a row in a way that would rewrite references of derived class rows; the application role holds advisory locks with the former keys of internal locks, while lock rows are out of its reach; a forged note of the family locks already taken (stages 1, 3; plan 1.1, stages 2, 4a, 5, checkpoint A, security review No. 2, checkpoint B and the work after it, security review No. 3)
2862
- # engine-class — classes as entities: metaschema, inheritance, recomputing descendants, tightening, metadata; a class with a history of rows is deleted only after purge() (stage 1; the work after checkpoint B)
2863
- # engine-write — the write pipeline in raw SQL: v5 ids, defaults, merge, inc, deletion, resurrection, journal, signals (stage 1)
2864
- # engine-races — races on two connections in both orders, without timed pauses; tightening during continuous writes to the family does not starve; locking families whose roots differ only in case, and a planted row of kind LOCK (stage 1; security review No. 3)
2865
- # explain — chain indexes from the application role: outside, the primary key and (class, tenant); inside find_*(), GIN; an asOf slice by class does not read another tenant's journal (stage 3; the work after checkpoint B)
2866
- # find-ids-parity — 30 chains through find_*() and through a direct read under RLS give the same answers; own memberships in other tenants and the Class rows of System — now, asOf and withDeleted (stage 3; the work after checkpoint B)
2867
- # hash — the version hash chain, journal tampering and gaps, verify() (stage 1)
2868
- # harness — the test harness: starting the database, a schema per file, two connection strings (stage 0)
2869
- # jsonschema — compiling JSON Schema to jsonpath and checking data in the database (stage 1)
2870
- # impersonation — impersonation: letopis.account only for a service with the permission, target restrictions, author and agent in the journal, sessionFor; the db.as() registry is one per tenant, does not empty, is re-read on a signal of its tenant (stage 5; the work after checkpoint B); the facade registry is not shared with the root, an empty registry is not cached (security review No. 3 leftovers)
2871
- # install — the installer and roles: a repeated installation, two installations in one database, three tables (stage 1)
2872
- # member — memberships: working in a tenant by roles, auth.switch, leaving with ownership transfer, system rows, tenant roles without System permissions (stage 5)
2873
- # model — Zod models, the JSON Schema normalizer, pattern translation, agreement with the database metaschema (stage 2)
2874
- # load — the loader: several classes, cycles, $key and $ext, repeats, existing, portions, history mode, the query for targets outside the load, CLI (stage 4)
2875
- # roles — roles and privileges: installation without a superuser, owner functions, triggers by catalog, the log without the changed column (stage 1; plan 1.1, checkpoint A)
2876
- # write-races — write races: 20 concurrent edits of different fields without losses and strict rev, concurrent add/remove on a multiple end, the loader versus class tightening and versus a concurrent edit (existing: update) in both orders (stage 4)
2877
- # validator-parity — the TypeScript validator versus the database check: the official suite, 10,000 random documents, defaults (stage 2)
2878
- # scenario-barbershop — end-to-end barbershop scenario: 9 scenes from 0.21 (integration) on 1.0 chains — catalog, free slots in a batch, bookings under lock, multi-slot and multi-specialist, versions, cancellation and booking again on the same id, closing a month with restrict (stage 4)
2879
- # scenario-real-life — the "real life" of a salon: 16 scenes from 0.21 (real-life) — a booking service on top of chains, races of double booking, rescheduling and cancellation, walk-in, refusals, reports and daily invariants (stage 4)
2880
- # system-classes — system classes instead of the 0.21 facades: the System seed, accounts, credentials, resources and rules, transactions (stage 5)
2881
- # sync — letopis sync (dry run, apply, tightening, lost refine) and letopis types (stage 2)
2882
- # tx — explicit db.begin() transactions: a broken transaction, lock(), 40P01; forUpdate(); the batch and its merging, a batch queue per facade; db.as(…, { owner }) (stage 4)
2883
- # v5 — v5 identifiers on the SQL side versus the vectors of an independent implementation (stage 1)
2884
- # zod-output — a snapshot of the z.toJSONSchema() output by field kind of the demo domain versus a reference (stage 0)
2885
- # time — history: versions({ follow }) across reclass and rekey, withDeleted, restore, purge, history trimming, account deletion (stage 6)
2886
- # rekey — a new key value of an object and a new key of a class: new ids, moving references, recursion through links with the end in the key (stage 6)
2887
- # reset — reset of sessions, tenant, all: the allowReset flag, the letopis.reset permission, confirmation by the schema name, the reset record, a new service key (stage 6)
2888
- # verify — legitimate gaps and tampering, verify({ data: true }): data, ends, tenants, id by key; permissions and CLI (stage 6)
2889
- # retention — retention policy: tiers, a finite term, special versions, deleted rows, asOf, cursor_expired, time zone, catch-up run, inheritance and default — versus a reference in TypeScript (stage 6)
2890
- # watch — cursor subscription: commit order, a long transaction, resuming, listener drop, two tenants (stage 6)
2891
- # cache — result cache: hits, own and others' writes, the key with the user, permissions, revoked sessions, listener drop, ttl, class marking, without LISTEN, a race, db.sql, the limit of signal counters (stage 6; plan 1.1, checkpoint A)
2892
- # maintain — maintenance: expired sessions and codes, permissions, one process under a lock, schedule (stage 6), lock rows of deleted classes
2893
- # up — the full installer: fresh installation, re-apply, custom seeds, automatic database creation, UTF8, version 17, upgrade with migrations, PGlite, the first tenant and user (up({ tenant }), createTenant) (stages 6, 8)
2894
- # upgrade — schema upgrade from 1.4.1 (a plain up() without migrations, plan 2.0, 7.8 No. 2), from 1.1.0 (upgrade: true, migration 0007) and from 1.0.1 (upgrade: true, migrations 0006 and 0007) with the letopis-141, letopis-11 and letopis-1 packages: versions and hashes written by the old engine, verify(), the catalog against a clean installation, lock rows, secret rows without an account erased, the application, a subscription and a projector over old data, a repeated up() (work after 1.2.0, stage 1; revision 7)
2895
- # scenario-salon — end-to-end salon scenario: 21 acts from 0.21 (salon) with a coverage matrix of the 1.0 API (stage 6)
2896
- # telemetry — onQuery and slowMs, describe() and class metadata with inheritance, describeText (stage 6)
2897
- # history — diff and blame over the log, the reason of a change for verbs, db.begin() and batches; a version hidden by an ACL condition, the changed column, a missing version, row_changes over 20 thousand targets (plan 1.1, stage 1 and checkpoint A)
2898
- # signals — signal modes: the payload seen by a foreign role, cache precision per tenant, System, permission classes, a process without the key, the loader, subscriptions, the key, none, coalescing, the server version, a class change, a resend, the mode after up() and reset({ all }) (plan 1.1, stage 2)
2899
- # initial-load — the initial load: equality with the regular loader, structure and trigger modes, system rows, a non-empty schema, an insert during the start, rollback, MULTI_INSERT in WAL, commitEvery, import --initial, aggregates (agg_fill) (plan 1.1, stages 3 and 4b)
2900
- # projector — projectors from the journal: catching up, exactly once on a break, one writer, a long transaction, a transaction on the boundary of the rebuild snapshot, no writes to derived classes and no references to them, the bounds of out, atomic and batch rebuilds during source writes, purge of a source, lag and cursor_expired, the credential functions; the version, the name, an admin connection; a race of two processes; a data error of one tenant drops out only this tenant (failed, rebuild({ tenant }), the tenants option), the checkpoint is only a System row, a rebuild touches only the rows of participants and into families, before after rekey, the all tier and the journal of sources per tenant; folding of out.inc by key in a batch, batch rebuilds on the automatic paths of run() and a second process during them; rebuilding one tenant in batches while the tail goes on; onExpired in batches in another process, a data error without retries, the tier from the start of the rebuild; an empty last pass of a batch rebuild, a tenant that drops out in the middle of its rebuild keeps no partial rows, a class change that keeps the id out of the source family and into it, purge of an unread tombstone (plan 1.1, stage 4a; security review No. 2; checkpoint B and the work after it; security review No. 3)
2901
- # mcp — MCP tools: the model and meta only in results, tenant rights, the JSON chain and hints of similar names, row and byte limits with a cursor, the untrusted-data wrapper, writing only with write, update only with rev, deletion with a human confirmation (elicitation in revisions 2025 and 2026-07-28), the reason in blame, credentials, tool schemas, letopis without the SDK; letopis mcp over stdio, the HTTP handler; the confirmation window without the alias and meta, countBy and versions limited in the query itself, limits of steps and $or conditions; the permission classes only with allowAclWrite, a session in the System tenant only with allowSystemAdmin; structuredContent with a note, a requestState signature of the server user with an expiry (plan 1.1, stage 5; security review No. 2; checkpoint B and the work after it)
2902
- # import — import from 0.21 with a fixture of a real 0.21: key violation, --keyless, per-class counters, verify, self-reference, repeat, ids without a key, a dangling reference, $$strict and fields starting with $, update, credentials, CLI (stage 7)
2903
- # security-final — the final security review (stage 8, §7.10 No. 8.4): a family lock by the tenant of the root, account_purge without permission and restore/purge of a foreign id give neither a leak nor a lock; the functions of stage 6, log_refs_at, journal-based find_* reads and watch_read stay within their own tenant; NOTIFY carries only names or mode tokens; the cache key distinguishes the token, the actor and the tenant (stage 8)
2904
- # semver — version rules: check:semver requires at least the matching version step — a major for a removed method or error code, a minor for a new method and a new engine revision, a patch for texts only (plan 2.0, 7.8 No. 1)
2905
- # prerelease — the pre-release check: steps run strictly in order, the first red one stops the run and the following ones do not start (plan 2.0, 7.8 No. 3)
2906
- # write-contract — README promises about writes (plan 2.0, 7.2): a repeated key in a batch gives exists and a rollback, the row order of reclass, restore and rekey responses, a chain with a verb as a slot value, the loader with existing: update does not edit a row of another class, a cached answer is not spoiled by editing an earlier answer
2907
- # watch-contract — README promises about subscriptions and connections (plan 2.0, 7.4): an idle subscription polls the journal once per pollMs, close() of the connection closes live subscriptions, a reclass without a key resets the cache of the old class, a new projector participant gets an initial rebuild, a projector without LISTEN re-reads its participants, at most a limit of db.as() registries
2908
- # model-contract — README promises about models, types, the CLI and errors (plan 2.0, 7.5): letopis types gives valid TypeScript for Cyrillic names, a class without fields and anyOf, the full form of hub() with z.object, clear errors instead of raw ones, CLI exit codes
2909
- # read-contract — README promises about reads and history (plan 2.0, 7.1): an asOf slice by class and by ends after a class change without an id change, versions, diff and blame with follow after rekey, the after() cursor in versions() and paths(), a field name with @S@
2910
- ```