letopis 1.1.0 → 1.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.en.md CHANGED
@@ -1,4 +1,4 @@
1
- # letopis 1.1
1
+ # letopis 1.3
2
2
 
3
3
  **letopis** is a Node.js library: a store of versioned entities on PostgreSQL 18. Every change is saved as a version
4
4
  in a journal. So every object has a history, a state as of any date and a change subscription. The database itself
@@ -14,11 +14,15 @@ What letopis can do:
14
14
  - **Writing with chains**: `create`, `update` with merge, `upsert`, deletion with preview and cascade, the `inc` increment
15
15
  for money and counters — [sections 5](#5-writing), [7](#7-deleting-and-restoring) and [8](#8-transactions-batches-and-money).
16
16
  - **Reading with chains**: hops along links, filters, trees, aggregates, pagination — [section 6](#6-reading).
17
- - **History**: versions, point-in-time reads, restoring deleted data, retention policy, hash chain verification —
18
- [section 11](#11-history-and-point-in-time-reads).
17
+ - **History**: versions, point-in-time reads, what changed and who changed it (`diff`, `blame`), restoring deleted
18
+ data, retention policy, hash chain verification — [section 11](#11-history-and-point-in-time-reads).
19
19
  - **Tenants, sign-in and permissions** in the database itself: passwords, API keys, one-time codes, TOTP, sessions,
20
20
  impersonation, access rules — [section 10](#10-users-sign-in-and-permissions).
21
21
  - **Change subscription** by cursor without losses, and a **result cache** — [section 12](#12-change-subscription-and-cache).
22
+ - **Derived data**: aggregates in the database, sums and counts that the database itself maintains, and projectors
23
+ that build summaries from the journal — [sections 4](#aggregates-in-the-database) and [12](#projectors).
24
+ - **Agents and MCP**: an MCP server gives an AI agent the model, reads, history and, if allowed, writes, with the
25
+ rights of one user — [section 13](#agents-and-mcp).
22
26
  - **Bulk loading** of millions of rows and **import from 0.21** — [sections 9](#9-bulk-loading) and [14](#14-migrating-from-021).
23
27
 
24
28
  Requirements:
@@ -28,7 +32,8 @@ Requirements:
28
32
  - Node.js 22 or newer; the package is an ES module.
29
33
  - One runtime dependency: `postgres`. Zod 4 is an optional peer dependency: only the `letopis/model` subpath needs it,
30
34
  and `import { connect } from 'letopis'` works without it. For development without PostgreSQL there are the optional
31
- `@electric-sql/pglite` and `@electric-sql/pglite-socket`.
35
+ `@electric-sql/pglite` and `@electric-sql/pglite-socket`. The MCP server (`letopis/mcp` and `letopis mcp`) needs
36
+ the optional `@modelcontextprotocol/server`, version 2.4 or newer in the 2.x line.
32
37
 
33
38
  ## Contents
34
39
 
@@ -154,8 +159,8 @@ service key again while the installation still has a valid API key. Next, sectio
154
159
 
155
160
  There are three tables. `entity` is the current state: one row per live entity, including the rows of system classes
156
161
  (`Class`, `Account`, `Resource`, `member`, `rule`). `log` is the journal: a full snapshot of every version with author,
157
- time, kind of operation and hash. `secret` holds credentials and sessions: the application sees none of its rows; only
158
- the sign-in functions work with it.
162
+ time, kind of operation and hash. `secret` holds credentials, sessions and projector checkpoints: the application sees
163
+ none of its rows; only the sign-in and projector functions work with it.
159
164
 
160
165
  Classes are entities too. A class description is stored in a `Class` row; the database also keeps the expanded
161
166
  description (with ancestors applied) and the compiled data check there. The version hash is a generated column: sha256 of the
@@ -191,23 +196,27 @@ is built from the journal.
191
196
 
192
197
  ### Signals
193
198
 
194
- A write reports changes through `NOTIFY` on the schema channel. Raw SQL sends the signal from a trigger, and PostgreSQL
195
- delivers it on commit. Library transactions defer their signals: the process notifier sends them every 100 ms
196
- (`notifyIntervalMs`) and retries the ones that failed to send. What a write signal carries is set by the installation
197
- mode `up({ signals })`:
199
+ A write reports changes through `NOTIFY`. The channel is the full name of the installation schema, version included,
200
+ for example `v2.shop`; because of the dot, SQL takes the name in double quotes: `LISTEN "v2.shop"`. A raw SQL write on
201
+ its own connection sends the signal from a trigger, and PostgreSQL delivers it on commit. Library transactions,
202
+ `db.sql` and `db.transaction()` included, defer their signals: the process notifier sends them every 100 ms
203
+ (`notifyIntervalMs`) and retries the ones that failed to send. What a write signal carries (its payload) is set by the
204
+ installation mode `up({ signals })`:
198
205
 
199
- | Mode | Write signal | What any role that can connect to the database sees |
206
+ | Mode | Write signal payload | What any role that can connect to the database sees |
200
207
  |---|---|---|
201
208
  | `class` (default) | the class name: `Booking` | class names and write times |
202
- | `tenant` | the class name and a tenant token: `Booking@3f9c…` | also tenant pseudonyms |
203
- | `opaque` | a class token and a tenant token | only write times and frequency per pseudonym |
209
+ | `tenant` | `Class@tenant token`: `Booking@3f9c…` | also tenant pseudonyms |
210
+ | `opaque` | `class token@tenant token`: `9a1e…@3f9c…` | only write times and frequency per pseudonym |
204
211
  | `none` | no signals | nothing |
205
212
 
206
- A revoked session sends `session:<start of the token hash>`, a reset sends `*`; signals carry no ids and no data. A
207
- token is a keyed HMAC: only a process with the signal key can match a pseudonym to a name, and the key is given to
208
- services (the `letopis.signals` permission). On signals, the class registry is re-read, the result cache is cleared
209
- and subscriptions wake up; what the mode changes for the cache and subscriptions is in
210
- [section 12](#12-change-subscription-and-cache).
213
+ A token is 24 hexadecimal characters of a keyed HMAC: only a process with the signal key can match a pseudonym to a
214
+ name, and the key is given to services (the `letopis.signals` permission). Besides write signals, there are service
215
+ ones: a revoked session sends `session:<16 hex>` (the first 16 hexadecimal characters of the sha256 of its token), a
216
+ session reset sends `session:*`, and a reset of a tenant or of the whole installation sends `*` (a reset of the whole
217
+ installation also sends `session:*`). In the `none` mode there are none of them either. Signals carry no ids and no
218
+ data. On signals, the class registry is re-read, the result cache is cleared and subscriptions wake up; what the mode
219
+ changes for the cache and subscriptions is in [section 12](#12-change-subscription-and-cache).
211
220
 
212
221
  ## 3. Installation and connection
213
222
 
@@ -265,8 +274,8 @@ const ANNA = r.tenant!.user! // its first user
265
274
  The application works as a LOGIN role that is a member of `letopis_app` only:
266
275
  `create role shop_app login password '…'; grant letopis_app to shop_app;`. Then RLS and triggers apply to it, as to
267
276
  any raw SQL from this role. `connect()` rejects a role that bypasses RLS with the error `bypass_rls`. The admin
268
- connection (`allowBypassRls: true`) is needed only for the loader, verification and maintenance; keep it in a separate
269
- process, not in the web application.
277
+ connection (`allowBypassRls: true`) is needed only for the loader, projectors, aggregate repair (`rebuildAggregates`),
278
+ verification and maintenance; keep it in a separate process, not in the web application.
270
279
 
271
280
  ### First tenant and user
272
281
 
@@ -339,7 +348,8 @@ through `as()` share the pool with the original one: close the original one.
339
348
 
340
349
  ### Admin connection
341
350
 
342
- The loader needs a connection of a role that can become the owner, with `allowBypassRls: true`. `verify()` and
351
+ The loader, projectors and `rebuildAggregates` need a connection of a role that can become the owner, with
352
+ `allowBypassRls: true`. `verify()` and
343
353
  `maintain()` work on it too, and also for a system administrator on a regular connection. The admin connection prints
344
354
  the warning "RLS не действует" ("RLS does not apply"); this is expected. The session sets the tenant it works with. A
345
355
  separate loading process is better off with a service key:
@@ -353,12 +363,23 @@ const adm = await connect({ dsn: ADMIN, schema: r.schema, token: r.tenant!.token
353
363
 
354
364
  ### Upgrading the package and schema
355
365
 
356
- The schema has an engine revision (currently 5). If a new package version brings a new revision, `connect()` only warns
366
+ The schema has an engine revision (currently 6). If a new package version brings a new revision, `connect()` only warns
357
367
  and names the command. It is `up()` that refuses: a lagging schema without `upgrade: true`, and a schema newer than the
358
368
  package. So a startup script that calls `up()` on every start will stop after a package update until it gets
359
369
  `upgrade: true`. `up({ dsn, schema, upgrade: true })` applies the migrations from `sql/migrate/` and re-applies the
360
370
  functions; the data is kept.
361
371
 
372
+ The first `up()` after moving to 1.2 builds the journal index `log_class_tenant_at (class, tenant, at)` instead of
373
+ `log_class_at`: an `asOf` and `withDeleted` slice by class no longer reads the journal of other tenants with the same
374
+ class name. While the index is being built, writes to the journal wait. On a large installation, build it in advance
375
+ without stopping writes: `create index concurrently log_class_tenant_at on "v2.<schema>".log (class, tenant, at)` as
376
+ the owner role — then `up()` only drops the old one.
377
+
378
+ The `upgrade` test checks upgrading 1.1.0 and 1.0.1 schemas. The old package installs the schema and the old engine
379
+ writes the data, then the current version's `up()` runs. After it, the versions and hashes written by the old engine
380
+ are in place, `verify({ data: true })` is clean, and the schema catalog matches a clean installation. A 1.1.0 schema is
381
+ upgraded with a plain `up()`, since its revision is the same. A 1.0.1 schema is upgraded with `up({ upgrade: true })`.
382
+
362
383
  ### Development without PostgreSQL: PGlite
363
384
 
364
385
  `up({ pglite: true })` starts PGlite (PostgreSQL 18 on WebAssembly) with pgcrypto and a socket on a free port. It needs
@@ -488,13 +509,78 @@ name of the class for a chain step, for example a name in another language.
488
509
 
489
510
  - `history: false`: no journal is kept for the class: no versions, no point-in-time reads, no restore and no
490
511
  subscription events. A retention policy is `history: { all: '1 day', daily: '3 months', … }`
491
- ([section 11](#11-history-and-point-in-time-reads)).
512
+ ([section 11](#11-history-and-point-in-time-reads)). Only someone with the `READ` and `DELETE` permissions without
513
+ conditions on all rows of the class and its descendants may change `history` of an existing class: `maintain()` thins
514
+ the journal by the policy, and `false` turns it off.
492
515
  - `ownerDefault: 'actor'`: the owner of new rows is the user, not the tenant (the default is `'tenant'`). This is
493
516
  useful for the rule "everyone sees their own records".
494
517
  - `cache: { ttl }`: reads of this class alone are cached without `.cache()` ([section 12](#12-change-subscription-and-cache)).
495
518
  - `meta`: free-form metadata for the application: a title, an icon, a color, an order in a menu. Up to 64 KB,
496
519
  inherited key by key (`null` removes an inherited value), visible to anyone who has a session. For an end, use
497
520
  `.meta()`.
521
+ - `derivedBy: 'projector-name'`: a derived class: only this projector writes its rows ([Projectors](#projectors)).
522
+ The name is lowercase Latin letters, digits and `_.:-` (the first character is a letter or a digit), up to 64
523
+ characters. The option is inherited, and a descendant does not change it. The ends of a derived class are only
524
+ `onDelete: 'cascade'` or `'unset'`: a derived row does not prevent deleting its target. Nothing refers to the rows
525
+ of a derived class. The application, raw SQL and the loader do not write to such a class: writing, editing and
526
+ deleting its rows gives `derived_class`. The derived class itself (own or inherited) is created, edited and deleted
527
+ only by an administrator — an admin connection or an account with the System permission `letopis.projector`; for an
528
+ application session it is `acl_denied` ([Projectors](#projectors)).
529
+ - `aggregate: { field: { sum | count, of, via, where } }`: an aggregate in the database: the database itself keeps
530
+ the sum or count of source rows ([Aggregates in the database](#aggregates-in-the-database)).
531
+
532
+ ### Aggregates in the database
533
+
534
+ An aggregate is a field that the database itself maintains: the sum of a field or the number of live rows of a
535
+ source class whose end leads to the row. It is needed where the total must match the source even for raw SQL: a
536
+ wallet balance, the number of order lines.
537
+
538
+ ```ts
539
+ const Wallet = hub('Wallet', { number: z.string(), balance: z.number().int().min(0), deposits: z.number().int().optional() }, {
540
+ key: ['number'],
541
+ aggregate: {
542
+ balance: { sum: 'amount', of: 'Movement', via: 'Wallet' }, // sum of movements
543
+ deposits: { count: true, of: 'Movement', via: 'Wallet', where: { properties: { amount: { minimum: 1 } } } }, // number of top-ups
544
+ },
545
+ })
546
+ const Movement = link('Movement', { amount: z.number().int() }, { ends: { Wallet } })
547
+ ```
548
+
549
+ - `sum: 'field'` is the sum of a numeric field of the source, `count: true` is the number of rows. `of` is the source
550
+ class (a name or a model) together with its descendants, `via` is its single end that leads to the target row.
551
+ `where` is a JSON Schema over the source data: rows that do not pass it are not counted.
552
+ - Only live rows of the same tenant as the target are counted. A reference to a System row is not counted: a shared
553
+ directory does not count the data of all tenants. An aggregate of a tenant class, including one inherited from a
554
+ System class, counts only classes of its own tenant: a System class named `of` is not counted, while the tenant's
555
+ own descendant of a System class is. An aggregate of a System class counts System classes and the classes of the
556
+ target row's tenant.
557
+ - A trigger recalculates the field after every statement: writes by the library, raw SQL, a delete cascade,
558
+ `restore`, `reclass`, `rekey`, the loader, account deletion. The operations of one statement add up to one increment
559
+ of the target, and the class rule (`min(0)`) is checked on the total: an operation that would take the balance below
560
+ zero is rejected with `invalid_data`. Every change of the target is its new version with the author and reason of
561
+ the operation.
562
+ - Only the database writes the field. A value in `create`, `update`, `upsert`, `rekey`, `reclass` or raw SQL is an
563
+ `aggregate_field` error. A new row gets the total of already existing sources, usually 0. Replacing the data as a
564
+ whole without the field keeps the previous value. The loader does not write a passed value. No permission to write
565
+ the target is needed: the permission to write the source is enough.
566
+ - The database checks the description (`invalid_class`): the field is in the schema, it is numeric and not part of the
567
+ key; the `via` end is single and accepts the target class; the `sum` field is a number; aggregates do not form a
568
+ cycle; the target and the source are not system or derived classes. A source class is described before the
569
+ aggregate: an aggregate over a class that does not exist yet is refused. For a tenant class, the source is looked up
570
+ among the tenant's classes. `syncModels` describes sources before targets itself. The `of` class is not deleted while
571
+ an aggregate of a class of the same tenant names it: first the aggregate is removed from the target's description.
572
+ - Rows get into an aggregate only with the knowledge of someone who reads them all: the field also counts rows that
573
+ the author cannot see. Only someone who reads all rows of the source and its descendants without conditions can
574
+ describe or change an aggregate. Only someone who reads all rows of a class and its descendants without conditions,
575
+ both before the change and after it, can make the class a source (a new descendant of a source, a `parent` change).
576
+ Otherwise `acl_denied`.
577
+ - The option is inherited, and a descendant replaces it as a whole. Adding or changing an aggregate is a tightening:
578
+ writes to the target and to the sources wait while the field is recalculated for all rows.
579
+ - `db.rebuildAggregates(class)` recalculates the field from the current rows, for repair. A version is written only
580
+ where the value has changed. `db.verify({ data: true })` shows discrepancies. It needs an admin connection.
581
+ - A raw `INSERT … ON CONFLICT DO UPDATE SET data = excluded.data` for a target is rejected: the insert trigger has
582
+ already set the field in `excluded.data`. Write `excluded.data - array['balance']`, and the field stays as it was;
583
+ the library's `upsert` does this itself.
498
584
 
499
585
  ### Writing the model to the database
500
586
 
@@ -636,6 +722,13 @@ without reads or locks in the application. Rules:
636
722
  - an `inc` marker cannot come from JSON: `{ "$inc": 5 }` that came from the network stays data;
637
723
  - store money as an integer number of minor units (cents), no greater than 2^53 − 1.
638
724
 
725
+ A counter that must match the source even for raw SQL is easier to keep as an aggregate
726
+ ([Aggregates in the database](#aggregates-in-the-database)): the database recalculates it itself, and you cannot forget
727
+ an `inc`. An aggregate and `inc` cost the same: operations on one object go one after another under the lock of its
728
+ row, and each writes its version. On the benchmark machine, one object took 150–190 operations per second this way, no
729
+ matter how many writers were waiting for it ([section 17](#17-performance)). For hotter counters, use a projector that
730
+ folds a batch ([Projectors](#projectors)): the total lags behind the source, but writers do not wait for each other.
731
+
639
732
  ### Protection against concurrent edits: `{ rev }`
640
733
 
641
734
  <!-- run: guide -->
@@ -666,9 +759,12 @@ const [vip] = await db.Customer(olga).reclass('Vip', { name: 'Olga', email: 'olg
666
759
  - `rekey(patch)` sets a new key value: a row with a new id, all references moved, the old row deleted. If a referencing
667
760
  row includes this end in its own key, its id changes too, and the move goes on. The journal links the old and the new
668
761
  id (`moved`).
669
- - `db.rekeyClass(class, key)` sets a new key for all rows of the family, in one transaction.
670
- - `reclass(class, data)` changes the class; the data is optional, and if given, it is given whole. If the new class has
671
- no key, the id is kept (as for `vip` above); if it has one, the id is computed anew and the references are moved.
762
+ - `db.rekeyClass(class, key)` sets a new key for all rows of the family, in one transaction. For a derived class and
763
+ for a class with a derived descendant — `derived_class`: only its projector writes the rows of a derived class.
764
+ - `reclass(class, data)` changes the class; the data is optional, and if given, it is given whole. The row leaves its
765
+ previous class, so it needs the `DELETE` permission on it, and `WRITE` on the new class and on every row whose
766
+ references the move rewrites, at any depth. If the new class has no key, the id is kept (as for `vip` above); if
767
+ it has one, the id is computed anew and the references are moved.
672
768
  The roles of referencing rows must accept the new class, otherwise `reclass_denied`. A raw `UPDATE … SET class`
673
769
  passes only if neither the old nor the new class has a key (otherwise `use_reclass`).
674
770
  - Deleting, changing the class and changing the key work only in `read committed`: library transactions always open at
@@ -820,7 +916,8 @@ const avgPrice = await db.Shop(shop).Product().avg('data.price')
820
916
  const byStatus = await db.Order().countBy('data.status') // { new: 1 }
821
917
  ```
822
918
 
823
- The database computes aggregates over the entities of the last step.
919
+ The database computes aggregates over the entities of the last step. `limit(n)` and `offset(n)` before `countBy()`
920
+ apply to the groups: `db.Order().limit(3).countBy('data.status')` gives the three most frequent values.
824
921
 
825
922
  ### Paths
826
923
 
@@ -887,8 +984,12 @@ await db.Customer(temp).purge({ confirm: true }).rows() // history erased
887
984
  ```
888
985
 
889
986
  `purge` works only for deleted objects (a live one gives `not_deleted`) and erases their whole history; in place of
890
- version 1, a `purge` record stays with the number of erased versions. The history of system classes is not erased. To
891
- delete an account, use `db.auth.purgeAccount` ([section 10](#10-users-sign-in-and-permissions)).
987
+ version 1, a `purge` record stays with the number of erased versions. The history of system classes is not erased.
988
+ While a projector whose `from` sources include the class of the object ([“Projectors”](#projectors)) has not read
989
+ the deletion, `purge` refuses (`projector_pending`): the projector would never see an erased tombstone, and the
990
+ object's contribution would stay in the totals. So a deletion and a `purge` in one transaction do not go through in
991
+ such a class. Likewise, `db.trimHistory` refuses while the projector has not read the first remaining version: it
992
+ would no longer find the previous version for `before`. To delete an account, use `db.auth.purgeAccount` ([section 10](#10-users-sign-in-and-permissions)).
892
993
 
893
994
  ## 8. Transactions, batches and money
894
995
 
@@ -1031,6 +1132,67 @@ try {
1031
1132
  - A transfer is two operations in one `db.begin()`, with the wallets edited in id order: then there are no deadlocks.
1032
1133
  - Insufficient funds are told apart from other data errors by `e.issues`: the field `balance`, the rule `minimum`.
1033
1134
 
1135
+ ### Recipe: balance as an aggregate
1136
+
1137
+ The same wallet, but the database itself keeps the balance: `balance` is an aggregate, the sum of `amount` over the
1138
+ live operations ([Aggregates in the database](#aggregates-in-the-database)). An operation is one `Movement` record;
1139
+ there is no `inc` increment in the code, so it cannot be forgotten. The balance stays correct even when raw SQL or the
1140
+ loader writes the operations and a cascade deletes them.
1141
+
1142
+ ```ts
1143
+ const Wallet = hub('Wallet', { number: z.string(), balance: z.number().int().min(0) }, {
1144
+ ends: { Customer }, key: ['number'],
1145
+ aggregate: { balance: { sum: 'amount', of: 'Movement', via: 'Wallet' } }, // the balance is the sum of live operations
1146
+ })
1147
+ const Movement = link('Movement', { opId: z.string(), amount: z.number().int() },
1148
+ { ends: { Wallet }, key: ['Wallet', 'opId'] }) // the operation key gives idempotency
1149
+
1150
+ /** Operation: an amount above zero tops up, below zero debits; the database recalculates the balance in the same transaction. */
1151
+ async function operate(db: Db, wallet: string, opId: string, amount: number) {
1152
+ try {
1153
+ await db.Wallet(wallet).Movement().create({ opId, amount }).rows()
1154
+ return 'done'
1155
+ } catch (e) {
1156
+ if (e instanceof LetopisError && e.code === 'exists') return 'repeat' // a repeat after a lost response
1157
+ throw e // invalid_data: the balance would go below zero (min(0))
1158
+ }
1159
+ }
1160
+
1161
+ /** Adjustment "balance = target": an operation for the difference, with the wallet locked until commit. */
1162
+ async function adjust(db: Db, wallet: string, opId: string, target: number) {
1163
+ const tx = await db.begin()
1164
+ try {
1165
+ const w = (await tx.Wallet(wallet).forUpdate().first())!
1166
+ await tx.Wallet(wallet).Movement().create({ opId, amount: target - (w.data.balance as number) }).rows()
1167
+ await tx.commit()
1168
+ return 'done'
1169
+ } catch (e) {
1170
+ await tx.rollback()
1171
+ if (e instanceof LetopisError && e.code === 'exists') return 'repeat' // an adjustment with this opId is already written
1172
+ throw e
1173
+ }
1174
+ }
1175
+
1176
+ const [wallet] = await db.Customer(olga).Wallet().create({ number: 'W-1' }).rows() // balance is 0: the database sets it
1177
+ await operate(db, wallet.id, 'op-1', 10_000) // balance 10,000
1178
+ await operate(db, wallet.id, 'op-2', -3_000) // balance 7,000
1179
+ await adjust(db, wallet.id, 'op-3', 5_000) // an operation of −2,000, balance 5,000
1180
+ ```
1181
+
1182
+ - The model here is its own, and the block is not part of the example program: there the balance grows through `inc`,
1183
+ and an aggregate field does not change that way.
1184
+ - `balance` is passed neither to `create` nor to `update` (`aggregate_field`), so an adjustment is an operation too,
1185
+ not an `update` under `{ rev }`. Writing an operation needs no permission to write the wallet: the permission to
1186
+ write `Movement` is enough.
1187
+ - Insufficient funds are `invalid_data`, as with `inc`: `e.issues` holds the field `balance` and the rule `minimum`.
1188
+ Check a repeat as `same()` in the recipe above does: `exists` only says that an operation with this `opId` already
1189
+ exists.
1190
+ - Editing and deleting an operation change the balance too: the database subtracts the previous value. It is still
1191
+ better to cancel an operation with a reverse one, so that the history of the wallet stays complete.
1192
+ - Every operation locks the wallet row and writes its version, so the operations of one wallet go one after another,
1193
+ as with `inc`: about 150 per second on the benchmark machine ([section 17](#17-performance)). If one object gets
1194
+ more, compute the total with a projector instead ([Recipe: daily revenue](#recipe-daily-revenue)).
1195
+
1034
1196
  ## 9. Bulk loading
1035
1197
 
1036
1198
  ### When you need the loader
@@ -1138,7 +1300,8 @@ problem. Its `detail`: `rows` is the first 100 violations with the row number, c
1138
1300
  `limit` is the list limit (100), `complete` means the checks that found violations went through all rows. Row codes:
1139
1301
  the codes of the database checks, and also `duplicate` (a repeated id), `exists` (the object already exists), `deleted`
1140
1302
  (an object with this id existed before) and `class_not_loadable` (the loader does not accept `Class` rows: classes are
1141
- set by `syncModels`).
1303
+ set by `syncModels`). The loader rejects rows of derived classes and references to them with the code
1304
+ `derived_class`: only a projector writes such rows.
1142
1305
 
1143
1306
  Violations are searched for in stages: parsing the rows (classes, ids, repeats), permissions on classes
1144
1307
  (`load({ as })`), data and ends in batches, and at the end of a portion the targets outside the load, owners and the
@@ -1298,6 +1461,33 @@ cut down to 999, so System rules with a weight of 1000 or more are an administra
1298
1461
  outweigh. Tenant rules apply to the tenant's rows, System rules apply everywhere. Permissions apply to raw SQL too: rows
1299
1462
  without permission are invisible, and an edit changes nothing or fails with `acl_denied`.
1300
1463
 
1464
+ A class mask in a `READ`, `WRITE` or `DELETE` object matches the descendants of the class too, by the names of the
1465
+ ancestors and their aliases. So an edit of a class that changes `parent`, `alias` or `derivedBy` needs the `READ`,
1466
+ `WRITE` and `DELETE` permissions without conditions on all rows of the class and of each of its descendants,
1467
+ otherwise `acl_denied`: the `WRITE` permission on `Class` alone is not enough to put the rows of a class under other
1468
+ rules. An alias does not match the name of another class (including `Class` and the system ones) or another class's
1469
+ alias in the tenant and System, and the name of a new class does not match another class's alias: otherwise a rule
1470
+ written for one class would act on the rows of another too. A violation gives `invalid_class`.
1471
+
1472
+ By the same names, the rules also find the old versions of rows in the journal. So a class is deleted only when it has
1473
+ no rows, no descendants and no history of rows: `purge()` erases the history of deleted rows first. For a System class,
1474
+ the rows, descendants and history of a tenant that has its own class with the same name do not count: they belong to
1475
+ that tenant's class. A class deleted with history before is created again only with the same `parent`, `alias`,
1476
+ `derivedBy` and `history` — otherwise `invalid_class`: the old versions of its rows would fall under other rules or a
1477
+ different thinning. The rules also find old versions by the aliases of ancestors. So while deleted descendants of a
1478
+ class keep row history, the class does not change its alias (`invalid_class`), and once deleted it is created again
1479
+ only with the same description. If the journal holds rows of a class with this name but not its tombstone (for
1480
+ example, rows of a System class deleted in 1.1), a class with this name is not created until `purge()` erases the
1481
+ history.
1482
+
1483
+ A derived class (`derivedBy`) is created, edited and deleted only by an administrator: an admin connection or an
1484
+ account with the System permission `letopis.projector` (the system administrator has it through the `letopis.*`
1485
+ mask). The projector writes totals over all source rows of the tenant into such a class, without read permissions,
1486
+ so a user who created it on their own would read totals over rows they may not read. A class that has derived
1487
+ descendants is also edited only by an administrator: editing an ancestor changes the description of the descendant
1488
+ (fields, key, `history`). The database itself sets the `aggregatedBy` mark (the class became an aggregate source) on a
1489
+ derived class too: it does not change its fields or key.
1490
+
1301
1491
  ### Granting a permission
1302
1492
 
1303
1493
  Example: Oleg is a shop employee with the `staff` role, and employees may not delete products.
@@ -1370,6 +1560,10 @@ Every version has an author, an agent, a time, a number, a kind of operation (`c
1370
1560
  operation of a block, from `db.begin({ reason })` and `db.batch(name, { reason })`; in raw SQL, from the
1371
1561
  `letopis.reason` setting of the transaction. A deletion version has `$deleted`.
1372
1562
 
1563
+ Without `sort()`, versions go by id and version number (with `follow`, by time). `sort()` sets the order of versions
1564
+ too, and `limit()` limits them in the query itself: `db.Product(kettle).sort('at', 'desc').limit(5).versions()` gives
1565
+ the five latest versions, newest first.
1566
+
1373
1567
  ### Point-in-time read
1374
1568
 
1375
1569
  <!-- run: guide -->
@@ -1425,16 +1619,57 @@ const [b] = await db.Order(id).blame() // every field of th
1425
1619
  await db.Order(id).blame({ at: '2026-10-01', limit: 200 }) // the version at a moment; scan 200 versions
1426
1620
  ```
1427
1621
 
1428
- - `diff()` returns one item per object of the step: `{ id, class, from, to, changes }`, where a change is
1429
- `{ path, kind, before, after }`, `kind` is `added`, `removed` or `changed`, and the path starts with the section of
1430
- the row: `data`, `links`, `tags`, `owner`, `class`. Data objects are compared by keys, arrays as a whole; a
1431
- multi-valued end and tags element by element. If the object did not exist at `from`, every path is `added`; if it
1432
- was deleted at `to`, `to.deleted` is `true`.
1433
- - `blame()` returns `{ id, class, rev, complete, fields }`: for every leaf, the version that changed it last,
1434
- `author`, `agent`, `op`, `reason` and `exact`. `exact: false` means there are gaps between the current version and
1435
- that one (the history is thinned, trimmed or partly invisible): a missing version may have changed the leaf too.
1436
- `complete: true` means the whole history was scanned without gaps; by default at most the 1000 latest versions are
1437
- scanned.
1622
+ Both terminals return an array, one item per object of the step. This is what the answers look like for a kettle
1623
+ whose price was lowered with a reason:
1624
+
1625
+ ```ts
1626
+ import type { BlameResult, DiffResult } from 'letopis'
1627
+
1628
+ // db.Product(kettle).blame(): version 2 changed the price, the other leaves date from the creation (version 1)
1629
+ const blamed: BlameResult = {
1630
+ id: '6f1c…', class: 'Product', rev: 2, complete: true,
1631
+ fields: [ // in path order
1632
+ { path: ['class'], rev: 1, at: '2026-10-27T09:15:02.118204+00:00', author: '<Anna id>', agent: '<service id>',
1633
+ op: 'create', exact: true },
1634
+ { path: ['data', 'price'], rev: 2, at: '2026-10-27T09:20:41.502617+00:00', author: '<Anna id>', agent: '<service id>',
1635
+ op: 'update', reason: 'promotion discount', exact: true },
1636
+ // ['data', 'sku'], ['data', 'title'], ['links', 'Shop'], ['owner'] — also version 1, like ['class']
1637
+ ],
1638
+ }
1639
+ // db.Product(kettle).diff(): previous version → latest
1640
+ const diffed: DiffResult = {
1641
+ id: '6f1c…', class: 'Product',
1642
+ from: { rev: 1, at: '2026-10-27T09:15:02.118204+00:00' },
1643
+ to: { rev: 2, at: '2026-10-27T09:20:41.502617+00:00' },
1644
+ changes: [{ path: ['data', 'price'], kind: 'changed', before: 2500, after: 2000 }],
1645
+ }
1646
+ ```
1647
+
1648
+ `agent` here is the service that writes on behalf of Anna (`db = await svc.as(…)`); a write without impersonation has
1649
+ none.
1650
+
1651
+ - `diff()` returns an array of `DiffResult`: `{ id, class, from, to, changes }`. `from` and `to` are the compared
1652
+ versions `{ rev, at }`. They have `id` if the version belongs to another id (`follow: true` across `reclass` or
1653
+ `rekey`), and `to` has `deleted: true` if the object was deleted at that moment; then `changes` is empty. `from` is
1654
+ `null` if the object did not exist yet: every path is `added`. If there is no version at all, `class` and `to` are
1655
+ `null` too.
1656
+ - A change, `DiffChange`, is `{ path, kind, before, after }`. `path` is an array of segments, not a dotted string;
1657
+ the first segment is the section of the row: `data`, `links`, `tags`, `owner`, `class`. `kind` is `added` (there
1658
+ was no value), `removed` (the value is gone) or `changed`. A missing value is omitted: `added` has no `before`,
1659
+ `removed` has no `after`. Data objects are compared by keys, arrays as a whole. A multi-valued end and tags are
1660
+ sets: one change per id or tag, the path is `['links', role]` or `['tags']`, and the id or tag itself is in
1661
+ `before` or `after`.
1662
+ - `blame()` returns an array of `BlameResult`: `{ id, class, rev, complete, fields }`, plus `deleted: true` if that
1663
+ version is a tombstone. An object with no versions at that moment gives no item. `fields` is an array of
1664
+ `BlameField`, one item per leaf of the version, ordered by path: `{ path, rev, at, author, op, exact }` and, when
1665
+ present, `agent`, `reason`, `id`. A leaf is a scalar or an array in the data, an empty object, the role of a
1666
+ single-valued end, every id of a multi-valued end (`['links', role, id]`), every tag (`['tags', tag]`), `owner` and
1667
+ `class`. `rev`, `at`, `author`, `agent`, `op` and `reason` come from the version that changed the leaf last. Empty
1668
+ `agent` and `reason` are omitted; `id` is present only if it is a version of the former id (the history went
1669
+ through `reclass` or `rekey`).
1670
+ - `exact: false` means there are gaps between the current version and the found one (the history is thinned,
1671
+ trimmed or partly invisible): a missing version may have changed the leaf too. `complete: true` means the whole
1672
+ history was scanned without gaps; by default at most the 1000 latest versions are scanned.
1438
1673
  - The database computes both with the caller's rights: raw SQL gets the same, `select "v2.shop".blame('<id>')`,
1439
1674
  `select "v2.shop".diff_versions('<id>')`.
1440
1675
  - The journal column `changed` (the paths changed by a version, which make `blame` fast) is not granted to the
@@ -1484,9 +1719,14 @@ for await (const ev of sub) { // ev: { cursor, row, id, class, op, r
1484
1719
 
1485
1720
  - `from` is `'now'` (the default: only changes after subscribing), `'start'` (the whole journal) or a saved cursor.
1486
1721
  `db.watch()` returns at once, and the subscription takes the "now" moment with its first query to the database: a
1487
- write made right after `db.watch()` may commit earlier and miss the stream. That is why the example above reads from
1488
- `'start'`.
1489
- - `classes`: only these classes with their descendants; `pollMs`: fallback polling (1000 ms).
1722
+ write made right after `db.watch()` may commit earlier and miss the stream. `await sub.ready()` waits for that
1723
+ moment: a write after it comes to the stream. The example above reads from `'start'`.
1724
+ - Calls of `sub.next()` go in turn: parallel calls get different events. A call abandoned by a timeout
1725
+ (`Promise.race`) still gets its event — keep its promise and wait for it again.
1726
+ - `classes`: only these classes with their descendants; `pollMs`: fallback polling (1000 ms). A row that left these
1727
+ classes through a class change without an id change arrives as a deletion event in the previous class (`op:
1728
+ 'delete'`, `row` is the previous version with `$deleted`), even if the subscriber cannot see the new class. The
1729
+ event comes if the subscriber could see the previous version.
1490
1730
  - Events come without losses or repeats, under the reader's permissions, in the order of transaction numbers, and
1491
1731
  within a transaction in the order of writing. This is not the commit order: compare versions of one object by `rev`.
1492
1732
  - A long write transaction in any database of the cluster delays the stream: `await sub.lag()` shows the lag in ms.
@@ -1496,6 +1736,215 @@ for await (const ev of sub) { // ev: { cursor, row, id, class, op, r
1496
1736
  - A subscription without losses is a saved cursor: store `ev.cursor` of the last processed event and resume from it
1497
1737
  after a restart.
1498
1738
 
1739
+ ### Projectors
1740
+
1741
+ A projector builds rows of derived classes from the journal: summaries, denormalized cards and anything else that may
1742
+ be computed with a lag behind the writes. The application writes the source classes (`from`), and only the projector
1743
+ writes the rows of the `into` classes. Each `into` class is marked with the option `derivedBy` holding the projector
1744
+ name ([Class settings](#class-settings)). Only an administrator creates derived classes: sync them with an admin
1745
+ connection holding a tenant session — `adm` from section 3 (`connect({ dsn: ADMIN, schema, token, allowBypassRls:
1746
+ true })`) or `as()` of a service on such a connection; in the CLI, `letopis sync` with the `--dsn` of the admin role.
1747
+ For an application session it is `acl_denied`. In the example below, the projector counts product sales from the
1748
+ order lines `line`.
1749
+
1750
+ ```ts
1751
+ const ProductSales = hub('ProductSales', { qty: z.number().int(), revenue: z.number().int() }, {
1752
+ ends: { Product: one(Product).onDelete('cascade') }, key: ['Product'], derivedBy: 'product-sales', history: false,
1753
+ }) // the derived class is in the model; syncModels — on an admin connection
1754
+
1755
+ const sales = adm.projector('product-sales', { // adm is an admin connection
1756
+ from: ['line'], // source classes with descendants; they must keep a journal
1757
+ into: ['ProductSales'], // the derived classes of this projector
1758
+ version: 1, // the handle logic changed: a new version and a rebuild
1759
+ withBefore: true, // c.before is the previous version of the object or null
1760
+ handle(changes, out) {
1761
+ for (const c of changes) {
1762
+ if (c.op === 'purge' || c.op === 'trim' || c.op === 'reset') continue // service changes, not versions
1763
+ const was = c.before // the previous version of the order line: subtract it
1764
+ const now = c.row.$deleted ? null : c.row // the new version: add it; a deletion has none
1765
+ if (was) out.inc('ProductSales', { Product: was.links.Product },
1766
+ { qty: -Number(was.data.qty), revenue: -Number(was.data.qty) * Number(was.data.price) }, { tenant: c.tenant })
1767
+ if (now) out.inc('ProductSales', { Product: now.links.Product },
1768
+ { qty: Number(now.data.qty), revenue: Number(now.data.qty) * Number(now.data.price) }, { tenant: c.tenant })
1769
+ }
1770
+ },
1771
+ })
1772
+ const report = await sales.run({ once: true }) // catch up with the journal: { batches, changes, created, updated, … }
1773
+ const state = await sales.status() // { name, state, version, cursor, at, lag, failed, rebuildingTenants }
1774
+ await sales.rebuild() // rebuild from the current state; { chunked: true } works in batches
1775
+ await sales.rebuild({ tenant: tenantId }) // rebuild one tenant, for example a dropped one (failed)
1776
+ await sales.rebuild({ tenant: tenantId, chunked: true }) // the same in batches: meanwhile the journal tail goes on for all
1777
+ sales.run().catch((err) => console.error('the projector stopped', err)) // follow the journal until stop()
1778
+ sales.close() // stop() after the current batch and unsubscribe from signals
1779
+ ```
1780
+
1781
+ - A projector needs an admin connection: `connect({ allowBypassRls: true })` with a role that may become the owner of
1782
+ the schema. Otherwise you get `admin_required`, and inside `db.begin()` you get `invalid_query`. The projector reads
1783
+ the journal of all tenants.
1784
+ - The first `run()` rebuilds the projection from the current state: the journal is thinned by the retention policy
1785
+ and cut by `purge`, so the projection cannot be assembled from the start of the journal.
1786
+ - Each batch is one transaction: the journal changes, `handle`, the written results and the cursor move are committed
1787
+ together. A break in the middle of a batch rolls the whole batch back, and the retry starts from the same cursor, so
1788
+ every change is applied exactly once. A projector has one writer: a second process of the same projector skips the
1789
+ batch (the `skipped` counter of the report), and `rebuild()` started while another process writes a batch gives
1790
+ `projector_busy`.
1791
+ - A batch is split by tenant: `handle` gets the changes of one tenant, and its results are written under a savepoint
1792
+ of its own. If the database rejects writing the results of a tenant (the data do not pass the description of its
1793
+ derived class, the target of an end is not found and so on) or `out` rejects a row or a delta built from its data
1794
+ (a delta that is not a number), only its changes are rolled back: the tenant drops out, and the others are written
1795
+ in the same batch. A dropped tenant gets into `status().failed` — `{ [tenant id]: { cursor, code, message } }`, and
1796
+ `message` names the tenant and the row id — and its changes are skipped until it is rebuilt. The derived rows of a
1797
+ dropped tenant no longer change: a tenant dropped in `run()` keeps them as they were before the failed batch, and a
1798
+ tenant dropped in the middle of a rebuild is left without them — the partly built rows are deleted. An exception
1799
+ from `handle`, a breach of the `out` limits and a failure of the environment (a deadlock, a cancelled query, a
1800
+ broken connection) roll the whole batch back, and `run()` is rejected. So catch `run()` without `once`: after an
1801
+ error it stops.
1802
+ - A change (`ProjectorChange`) is `cursor`, `id`, `class`, `tenant`, `op`, `rev`, `at`, `author`, `agent`, `reason`,
1803
+ `moved` and `row`, the version from the journal (for a deletion, a tombstone with `$deleted`). During a rebuild, the
1804
+ changes are rows of the current state: `rebuild: true`, `op` is `create`, `cursor` is empty.
1805
+ - A class change that keeps the id (classes without a key: a raw `UPDATE … SET class` and `reclass()`) is written to
1806
+ the journal as a version of the new class. If it takes the row out of the `from` family, the projector gets a
1807
+ deletion: `op` is `delete`, `row` is the last version of the row in the family with `$deleted`, and `rev`, `at` and
1808
+ the author are those of the class change. A row that enters the family this way comes as a `reclass` change with
1809
+ `before` = `null`.
1810
+ - With `withBefore`, a change has `before`, the previous version of the same id. `before` is `null` if there is no
1811
+ previous version (a creation and the first version of a new id after `reclass` and `rekey`), if the previous
1812
+ version is a deletion (this happens after `restore`) or lies outside the `from` family, and for every change of a
1813
+ rebuild. With `reclass` and `rekey`, the former data are subtracted through the tombstone of the former id: it comes
1814
+ as a change with `$deleted`, its `before` is the last version of the former id, and `moved` of both changes points
1815
+ to the other id. So the fold of the example gives the same sum after `rekey`. The service changes `purge`, `trim`
1816
+ and `reset` have no `before` field at all. So subtract `before` only after checking it for `null`, as in the example
1817
+ above.
1818
+ - The service journal records `purge`, `trim` and `reset` also come as changes, and their `rev` is 1. So check `op`
1819
+ first, and only then compare `rev`.
1820
+ - `out.upsert(class, row, { tenant })` creates a row or replaces it entirely by the class key.
1821
+ `out.inc(class, row, deltas, { tenant })` creates a row or completes it and adds the numbers without reading the row
1822
+ (missing fields count from 0). `out.delete(class, key or id, { tenant })` deletes a row and does nothing if there is
1823
+ none. `await out.get(class, key or id, { tenant })` reads the current row together with the writes of the batch.
1824
+ `out` writes only to the `into` classes and only to the tenant of the changes passed to `handle`; otherwise you get
1825
+ `derived_class` or `tenant_denied`, and the batch is rolled back.
1826
+ - `out` adds up the increments of a row given by key fields only (or by `$id` alone) before sending them to the
1827
+ database, as long as there is no `upsert`, `delete` or `get` between them: one edit of a row per batch, and the class
1828
+ rule is checked on the total of its increments. `inc` does not fold a row with other fields (data, `$tags`,
1829
+ `$owner`): its fields edit the row, and the order of such edits is kept.
1830
+ - A row for `out` is in the loader format: fields and end roles at the top level, the service keys `$tags`, `$owner`
1831
+ and `$id` (`$id` is needed only by a class without a key). The value of a role is the id of the target. An edit that
1832
+ changes nothing gives no version. Versions of derived rows are written by System with the agent
1833
+ `letopis projector <name>`; the journal is kept according to the `history` of the class, and the change signal is
1834
+ always sent.
1835
+ - `rebuild()` rebuilds the projection atomically, in one transaction: until it commits, readers see the previous rows,
1836
+ but the subscription horizon stands still. `rebuild({ chunked: true })` works in batches: the derived rows are
1837
+ deleted at once, the data stays incomplete until the rebuild ends, and `status().state` is `'rebuilding'`. The next
1838
+ start continues an interrupted batch rebuild. `run()` itself, on the first start of a new projection and with
1839
+ `onExpired: 'rebuild'`, rebuilds in batches: the subscription horizon of the cluster stands still no longer than a
1840
+ batch. Meanwhile a second process of the same projector waits or continues the same rebuild instead of starting it
1841
+ anew. A rebuild deletes the rows of the participants only, and only of the classes of their `into` families, and
1842
+ starts all participants anew: the marks of dropped tenants are removed, and a tenant whose data fail again drops
1843
+ out again.
1844
+ - `rebuild({ tenant: id })` rebuilds one tenant in one transaction: it deletes its derived rows, builds them from its
1845
+ current source rows and removes its mark in `status().failed`; the projector does not apply again the changes the
1846
+ rebuild has already counted. Fix the data or the class description of the tenant, then rebuild it. If the database
1847
+ rejects the write again, the rebuild is rolled back with this error, and the tenant stays dropped. No projection yet
1848
+ or a batch rebuild in progress gives `rebuild_required`; a tenant that is not a participant gives `tenant_denied`.
1849
+ With `chunked: true` — in batches, each in a transaction of its own with its own snapshot, while the journal tail goes
1850
+ on for all: the tail skips the tenant's changes above the last batch (a batch will read them) and, in a passed batch,
1851
+ those whose transaction is visible in its snapshot. The tenant's derived rows are deleted at once and are incomplete
1852
+ until the end of the rebuild (`status().rebuildingTenants`); the rebuild waits out a lock held by `run()` of another
1853
+ process. An interrupted rebuild (the process stopped) is continued by `run()` of any process, with or without
1854
+ `onExpired`.
1855
+ - `status().state` describes the projection in the database, not a process: `'new'` means there is no projection yet,
1856
+ and the first `run()` or `rebuild()` builds it; `'running'` means the projection is built and kept up with the
1857
+ journal (it does not mean that some process is executing `run()` right now); `'rebuilding'` means a batch rebuild
1858
+ is in progress or was interrupted.
1859
+ - `version` is the version of the `handle` logic. If the projection was built by another version, `run()` throws
1860
+ `rebuild_required`, and a rebuild with `rebuild()` is needed.
1861
+ - If the cursor is older than the `all` tier of the retention policy of the sources and there are changes after it,
1862
+ some changes may have been thinned out. Each tenant has a tier of its own: it is the description of its classes. If
1863
+ the cursor is older than the tier of every participant, `run()` throws `cursor_expired`, and with
1864
+ `onExpired: 'rebuild'` the projector rebuilds entirely, in batches. If it is older than the tier of only some of
1865
+ them, they drop out (`status().failed`, code `cursor_expired`) and the others go on, and with
1866
+ `onExpired: 'rebuild'` the dropped tenants rebuild themselves, in batches: the rebuild state lies in the checkpoint,
1867
+ and `run()` of any process continues it; a lock held by another process is only a retry on the next pass, and a data
1868
+ error drops the tenant with its code, without retries. The tier of a rebuilt tenant counts from the start of its
1869
+ rebuild until the cursor catches up with it: a lagging projector does not drop it again. A projector without new
1870
+ changes does not expire. `status().lag` is the age of the oldest unprocessed change in ms.
1871
+ - A projector works with the tenants in which every `into` class is marked with its `derivedBy`; it skips the changes
1872
+ of other tenants, so a class with the same name and no mark in another tenant does not get in its way. No such tenant
1873
+ at all — `derived_class`. Only an administrator may mark the classes of a tenant, so it is the administrator who
1874
+ chooses the participants: a tenant user cannot join a projector on their own. The option `tenants: [id, …]`
1875
+ is an explicit list of participants: the projector works only with these tenants (their `into` classes must be
1876
+ marked too) and ignores the marks of the others. The sources are the `from` families of these tenants; a class
1877
+ cannot be a source and a result at the same time. A projector does not see the changes of a class with
1878
+ `history: false`: it has no journal. So a tenant with such a class among the sources drops out with the code
1879
+ `invalid_class`, and the projector does not build its derived rows. Turn the journal of the class on and rebuild the
1880
+ tenant with `rebuild({ tenant })`.
1881
+ - `batch` is how many changes go into a batch (500), `pollMs` is fallback polling (1000 ms): source signals wake the
1882
+ projector earlier.
1883
+ - `lockTimeout` is how long a projector transaction waits for another transaction's row lock (5000 ms; 0 — no limit).
1884
+ Otherwise a transaction of any tenant holding a total row or an end target would hold the batch of all tenants.
1885
+ Longer than that, the batch rolls back, the cursor stays, and `run()` does not reject but skips the pass
1886
+ (`skipped`) and repeats it.
1887
+ - Every edit that `out` sends to the database is a separate edit of a row, and the speed of a projector is set by the
1888
+ number of edits, not by the number of changes. The folding of `inc` brings the edits of a row down to one per batch,
1889
+ so the handle from the example above (`out.inc` for every change) caught up with 6 thousand changes per second with
1890
+ `batch` 500 and 33 thousand with `batch` 2000 on the benchmark machine ([section 17](#17-performance)): the larger
1891
+ the batch, the more increments of one row fold into one edit.
1892
+
1893
+ ### Recipe: daily revenue
1894
+
1895
+ An aggregate in the database and a projector solve similar tasks in different ways:
1896
+
1897
+ | | Aggregate in the database | Projector |
1898
+ |---|---|---|
1899
+ | when it is computed | in the same transaction as the write of the source | after the commit, with a lag (`status().lag`) |
1900
+ | what it computes | the sum of a field or the number of source rows for one target row | anything: summaries by day and by group, denormalized cards |
1901
+ | matches the source | always, raw SQL included | when the projector has caught up with the journal |
1902
+ | cost of a write | every operation locks the target row and writes its version, so the writers of one target queue up | writers do not wait for each other: the projector applies changes in batches after the commit |
1903
+ | rule over the total | `min(0)` rejects an operation that would take the total beyond the rule | the total does not block writes: a data error drops the tenant out of the projection |
1904
+ | what to run | nothing: the database keeps the field | a process on an admin connection |
1905
+
1906
+ A wallet balance that must not go below zero is an aggregate
1907
+ ([Recipe: balance as an aggregate](#recipe-balance-as-an-aggregate)). The revenue of a shop by day is a total over many
1908
+ sales that may lag a little, which is a job for a projector:
1909
+
1910
+ ```ts
1911
+ const Sale = hub('Sale', { day: z.iso.date(), amount: z.number().int().min(0) }, { ends: { Shop } }) // a sale: day and amount
1912
+ const DayRevenue = hub('DayRevenue', { day: z.iso.date(), revenue: z.number().int(), sales: z.number().int() }, {
1913
+ ends: { Shop: one(Shop).onDelete('cascade') }, key: ['Shop', 'day'], derivedBy: 'day-revenue', history: false,
1914
+ })
1915
+ await syncModels(db, [Sale], { apply: true })
1916
+ await syncModels(adm, [DayRevenue], { apply: true }) // the derived class is created by an administrator: adm holds a tenant session
1917
+
1918
+ const revenue = adm.projector('day-revenue', {
1919
+ from: ['Sale'], into: ['DayRevenue'], version: 1, withBefore: true,
1920
+ handle(changes, out) {
1921
+ for (const c of changes) {
1922
+ if (c.op === 'purge' || c.op === 'trim' || c.op === 'reset') continue // service changes
1923
+ const was = c.before // the previous version of the sale: subtract it from its day
1924
+ const now = c.row.$deleted ? null : c.row // the new one: add it to its day; a deletion has none
1925
+ if (was) out.inc('DayRevenue', { Shop: was.links.Shop, day: was.data.day },
1926
+ { revenue: -Number(was.data.amount), sales: -1 }, { tenant: c.tenant })
1927
+ if (now) out.inc('DayRevenue', { Shop: now.links.Shop, day: now.data.day },
1928
+ { revenue: Number(now.data.amount), sales: 1 }, { tenant: c.tenant })
1929
+ }
1930
+ },
1931
+ })
1932
+ revenue.run().catch((err) => console.error('the revenue projector stopped', err)) // follows the journal until stop()
1933
+ const week = await db.Shop(shop).DayRevenue().sort('data.day', 'desc').limit(7).rows() // the last seven days
1934
+ ```
1935
+
1936
+ - The day is a field of the sale, not the version time `at`: with the version time, an edit, `restore` or `rekey`
1937
+ would move the amount of the sale to its own day.
1938
+ - `out.inc` adds without reading the row, so the total does not depend on the order of changes: the journal gives
1939
+ transactions in the order of their first write, not of their commit. Editing a sale subtracts the previous amount
1940
+ from the previous day and adds the new one to the new day; a deletion only subtracts. A day with no sales left stays
1941
+ as a row of zeros. `out` folds the increments of one day in a batch into one edit.
1942
+ - `DayRevenue` has `history: false`: the derived rows do not double the journal, and a rebuild assembles them without
1943
+ history anyway. `Sale` needs a journal: a projector does not see the changes of a class with `history: false`.
1944
+ - The application reads `DayRevenue` rows like any class, and only the projector writes them: `create` or `update` of
1945
+ such a row gives `derived_class`. The total lags behind the sales until the next batch: write signals wake the
1946
+ projector, and without them it polls every `pollMs`.
1947
+
1499
1948
  ### Result cache
1500
1949
 
1501
1950
  <!-- run: guide -->
@@ -1542,11 +1991,11 @@ const report = await adm.maintain() // { sessions, codes, thinned, batches,
1542
1991
  PostgreSQL signal queue: if it is more than half full, you get a warning. The system administrator, a role with the
1543
1992
  `letopis.maintain` permission or an admin connection can call it. An external scheduler runs it (`letopis maintain`),
1544
1993
  or the library itself does: `connect({ maintain: '1h' })`. Of several processes, only one works: each batch is its own
1545
- transaction, and the first thing it does is take a transaction-level advisory lock (`pg_try_advisory_xact_lock`),
1546
- which is released when the batch commits. If another process holds the lock, the call answers `skipped: true` before
1547
- the first batch, and between batches it stops (that process continues) and reports what it has done with
1548
- `more: true`. There is no session-level lock, so `maintain()` works behind any pool, including PgBouncer in
1549
- transaction mode.
1994
+ transaction, and the first thing it does is take the maintenance lock — a row of the `secret` table that neither the
1995
+ application role nor any other database role can take; the lock is released when the batch commits. If another
1996
+ process holds the lock, the call answers `skipped: true` before the first batch, and between batches it stops (that
1997
+ process continues) and reports what it has done with `more: true`. There is no session-level lock, so `maintain()`
1998
+ works behind any pool, including PgBouncer in transaction mode.
1550
1999
 
1551
2000
  ### Reset: `reset()`
1552
2001
 
@@ -1571,8 +2020,157 @@ const modelText = describeText(model) // the same as text: for the application's
1571
2020
  ```
1572
2021
 
1573
2022
  `db.describe()` returns a description of the classes available to the session: fields, ends, keys, policies and
1574
- metadata, plus the multi-field conditions `anyOf` and `oneOf` with inheritance applied, when a class has them. From
1575
- the command line: `letopis describe --format text --out llms.txt`.
2023
+ metadata, plus the multi-field conditions `anyOf` and `oneOf` with inheritance applied, the derived class mark
2024
+ `derivedBy` and the aggregates `aggregate`, when a class has them. From the command line:
2025
+ `letopis describe --format text --out llms.txt`.
2026
+
2027
+ ### Agents and MCP
2028
+
2029
+ An agent — Claude, Cursor or your own — works with the data through MCP tools: it learns the model, reads with
2030
+ chains, looks at the history and, if allowed, writes. The MCP server acts on behalf of one user and only with that
2031
+ user's rights: every call is a chain under the user's session, and the database checks rights and tenants. The agent
2032
+ has no raw SQL.
2033
+
2034
+ | Tool | What it does |
2035
+ |---|---|
2036
+ | `letopis_model` | the model: classes, fields (JSON Schema), the `anyOf` and `oneOf` conditions, ends, keys, aliases, metadata; with `class`, one class in detail |
2037
+ | `letopis_query` | reading with a JSON chain: the start is `class` and `filter`, `id` or `ids`; `steps` are hops by roles and classes; `asOf`, `withDeleted`, `sort`, `limit`, `cursor`; the terminal is `rows`, `first`, `ids`, `count`, `sum`, `avg`, `min`, `max` or `countBy` |
2038
+ | `letopis_history` | `versions`, `diff` or `blame` of one object |
2039
+ | `letopis_write` | `create`, `update` (only with `rev`), `upsert`, `delete`, only with `write: true` (`letopis mcp --write`) |
2040
+
2041
+ A `letopis_query` request is the same chain as `db.Product({ price: lt(1000) }).Shop().sort('data.name').rows()`, written as JSON:
2042
+
2043
+ ```json
2044
+ { "class": "Product", "filter": { "price": { "$lt": 1000 } }, "steps": [{ "step": "Shop" }], "sort": { "field": "data.name" } }
2045
+ ```
2046
+
2047
+ A filter value means equality, a nested object addresses the fields of nested objects, `{ "Role": id }` is an end.
2048
+ Operators: `$ne`, `$gt`, `$gte`, `$lt`, `$lte`, `$between`, `$in`, `$like`, `$ilike`, `$starts`, `$ends`, `$has`,
2049
+ `$hasAny`, `$hasAll`, `$exists`, `$isNull`, `$not`; OR is `{ "$or": [filter, …] }`. An unknown class, role, parameter
2050
+ or operator gives an `isError` answer with a hint of similar names. There are at most 16 `steps` and at most 64
2051
+ conditions in the `$or` of one filter; otherwise `invalid_query`. These limits do not bound the time of a query: set
2052
+ `statement_timeout` for the database role the MCP server connects as (`alter role … set statement_timeout = '5s'`).
2053
+
2054
+ The answer is `structuredContent` and its text copy. `structuredContent` is `{ notice, data }`: `notice` is a constant
2055
+ text of the package saying that `data` holds database data, not instructions, and `data` is the answer itself. The
2056
+ shape of `data` of `letopis_query` depends on the terminal:
2057
+
2058
+ | Terminal | `structuredContent.data` |
2059
+ |---|---|
2060
+ | `rows` (default) | `{ rows, count, more, cursor? }` |
2061
+ | `ids` | `{ ids, count, more, cursor? }` |
2062
+ | `first` | `{ row }`, a row or `null` |
2063
+ | `count` | `{ count }` |
2064
+ | `countBy` | `{ counts, truncated? }` |
2065
+ | `sum`, `avg`, `min`, `max` | `{ value }` |
2066
+
2067
+ For `rows` and `ids`, `count` is the number of rows on this page, not the total. A page holds at most `limit`,
2068
+ `maxRows` rows (100) and `maxBytes` bytes of row JSON (50,000); a row that is itself larger than `maxBytes` comes
2069
+ alone on its page. `more: true` means there are more rows, and then the same request with `cursor` returns the next
2070
+ page. `counts` is `{ value: number }`, the most frequent values; if the groups did not fit into the limits,
2071
+ `truncated: true` comes. `countBy` reads at most `maxRows` + 1 groups from the database. The
2072
+ field of an aggregate and of `countBy` is `field: 'data.<path>'`. Sorting is `sort: { field, dir }`: `field` is `at`,
2073
+ `rev`, `id` or `data.<path>`, `dir` is `'asc'` (default) or `'desc'`; without `sort`, rows go by `id`. `sort`, `limit`
2074
+ and `cursor` work only with `rows`, `first` and `ids`.
2075
+
2076
+ `letopis_model` without `class` answers (in `data`) `{ schema, tenant, classes }`, the same classes as in `db.describe()`. If such
2077
+ a model is larger than `maxBytes`, a brief one comes: `brief: true`, and the classes have only the names of their
2078
+ fields and ends; `letopis_model` with `class` returns each class in detail. `letopis_history` takes `class`, `id` and
2079
+ `kind`. `versions` answers `{ versions, more }`, the latest versions within the answer limits, from the oldest to the
2080
+ newest; `diff` with `from` and `to` (a version number or a moment) answers `{ diff }`; `blame` with `at` and `limit`
2081
+ answers `{ blame }`. If the changes or the leaves did not fit, the answer is cut, and it has `truncated: true` and
2082
+ `total`. `follow` works with `versions` and `diff`.
2083
+
2084
+ A tool refusal is `isError: true` with a `structuredContent` of the form `{ notice, error }`, where `error` is `{ code, message }`, plus `similar` (similar names),
2085
+ `detail` (details) and `sqlstate` (the PostgreSQL code of an error not from letopis) when present. A parameter that
2086
+ does not match the tool input schema (a wrong type, a value outside the list, a missing required one) is rejected by
2087
+ the SDK before letopis: then it is `isError: true` with text only, without `structuredContent`.
2088
+
2089
+ In the text of the answer, everything that came from the database (rows, names and descriptions of classes, error
2090
+ texts, hints) sits inside `<untrusted-data-…>` with a random suffix and a note that this is data, not instructions.
2091
+ Tool descriptions and the server's `instructions` are fixed text of the package: tenants write the `meta` of classes,
2092
+ so it only appears in results.
2093
+
2094
+ Writing. Without `write`, the `letopis_write` tool is not in the tool list, and a call to it is an SDK protocol error
2095
+ (`Tool letopis_write not found`), not a tool answer with `isError`. The ends of a write are set by
2096
+ `links: { Role: id | [id] | null }`, and `create`, `update` and `upsert` answer `{ rows }`. `update` takes only the
2097
+ `rev` of the version that was read: someone else's version gives `conflict`, and no update is lost. `delete` without
2098
+ `confirm` returns a preview of the cascade and the operation code: `{ preview, operation }`. With `confirm: true`, a
2099
+ human confirms the deletion in the client through elicitation (in protocol revision 2025 as a request to the client,
2100
+ in 2026-07-28 as an `input_required` answer), the confirmation is bound to the hash of the operation, and the answer
2101
+ is `{ deleted, operation }`. In revision 2026-07-28, the `requestState` signature with which the client repeats the
2102
+ call is also bound to the user of the server and is valid for 10 minutes: a signature of another user's server and an
2103
+ expired one give the refusal `confirmation_required`. The confirmation window shows the canonical class name and the ids of the targets (the
2104
+ first 10, then «и ещё K», that is "and K more"); a tenant writes the alias and the `meta` of a class, so they do not
2105
+ get into the window. An operation code sent by the agent is not a confirmation; a client without elicitation gets the
2106
+ refusal `confirmation_required`. Every write gets the reason `mcp: letopis_write`, and `blame()` shows it. By default,
2107
+ `letopis_write` does not create, edit or delete the permission classes, `rule`, `Resource`, `member`, `Account`,
2108
+ `Class` and their descendants (rules, memberships, class descriptions): the refusal is `acl_denied`. An agent under an
2109
+ injection in the data would grant a stranger a membership or a rule without a human confirmation. Only the option
2110
+ `allowAclWrite: true` allows it.
2111
+
2112
+ **stdio.** `letopis mcp` is a stdio server for MCP clients. It needs the SDK package: `npm i @modelcontextprotocol/server`
2113
+ (an optional peer dependency). The client passes the connection string, the schema and the token as flags or as the
2114
+ variables `LETOPIS_DSN`, `LETOPIS_SCHEMA`, `LETOPIS_TOKEN`; `--write` turns writing on. Only a user session token is
2115
+ accepted, and the application issues it: `svc.auth.sessionFor(user, { ttl })`, while `auth.switch(tenant)` sets the
2116
+ working tenant of the session. The command rejects the service key (`--api-key`): it grants all `auth.*` rights and
2117
+ would lie on the agent's machine. Without a token and with `--api-key`, the command exits with code 2 and writes the
2118
+ reason to stderr. The command also rejects the token of a session in the System tenant (this is how a system
2119
+ administrator works): it has no `allowSystemAdmin` and `allowAclWrite` options. Client configuration (Claude Desktop,
2120
+ Claude Code, Cursor):
2121
+
2122
+ ```json
2123
+ {
2124
+ "mcpServers": {
2125
+ "letopis": {
2126
+ "command": "npx",
2127
+ "args": ["letopis", "mcp"],
2128
+ "env": {
2129
+ "LETOPIS_DSN": "postgres://shop_app:app-secret@localhost:5432/shop",
2130
+ "LETOPIS_SCHEMA": "v2.shop",
2131
+ "LETOPIS_TOKEN": "<user session token>"
2132
+ }
2133
+ }
2134
+ }
2135
+ }
2136
+ ```
2137
+
2138
+ **HTTP.** There is no built-in HTTP server. `createMcpServer(db, { write, maxRows, maxBytes })` from `letopis/mcp`
2139
+ returns an MCP server, and the application plugs it into its own `createMcpHandler` handler from the SDK with its own
2140
+ OAuth 2.1 authorization: the application checks the request token and finds the letopis user by it. The connection
2141
+ is a user session (`connect({ token })`) or the facade `svc.as(user)`: the service key stays in the application
2142
+ process. A connection that bypasses RLS is refused with `bypass_rls`; the service root without `as()` and a connection
2143
+ without a session, with `user_session_required`; the facade of an explicit transaction `db.begin()`, with
2144
+ `invalid_query`. The server also refuses a session in the System tenant with `user_session_required` unless the option
2145
+ `allowSystemAdmin: true` is passed: this is how a system administrator works, and their `admin` role grants rights to
2146
+ the accounts, memberships and rules of the installation.
2147
+
2148
+ <!-- run: mcp -->
2149
+ ```ts
2150
+ import { up, connect } from 'letopis'
2151
+ import { createMcpServer } from 'letopis/mcp'
2152
+ import { createMcpHandler } from '@modelcontextprotocol/server'
2153
+
2154
+ const ADMIN = 'postgres://postgres:secret@localhost:5432/shop'
2155
+ const APP = 'postgres://shop_app:app-secret@localhost:5432/shop'
2156
+ const r = await up({ dsn: ADMIN, schema: 'shop', quiet: true,
2157
+ tenant: { name: 'Chamomile', user: { name: 'Anna', login: 'anna@shop.ru', password: 'Chamomile-2026!' } } })
2158
+ const svc = await connect({ dsn: APP, schema: r.schema, apiKey: r.serviceKey! })
2159
+
2160
+ // authInfo is the result of the application's check of the request token; it puts the letopis account id into extra
2161
+ const mcp = createMcpHandler(async ({ authInfo }) =>
2162
+ createMcpServer(await svc.as(String(authInfo?.extra?.account), { tenant: r.tenant!.tenant })))
2163
+ // in the application's HTTP handler, after checking the token: return mcp.fetch(request, { authInfo })
2164
+ ```
2165
+
2166
+ The handler calls the factory on every HTTP request, so `createMcpServer` checks the connection every time (one query
2167
+ to the database).
2168
+
2169
+ The handler can be checked without a network: the SDK client from the `@modelcontextprotocol/client` package reaches
2170
+ it through the application's `fetch`: `new StreamableHTTPClientTransport(new URL('http://localhost/mcp'), { fetch:
2171
+ (url, init) => mcp.fetch(new Request(url, init), { authInfo }) })`. This is how the `docs` test checks the example
2172
+ above (the `mcp` scenario in `lib/test/docs.test.ts` of the repository): without `write` the client sees three tools,
2173
+ and `letopis_model` answers on behalf of Anna in the shop.
1576
2174
 
1577
2175
  ### Raw SQL
1578
2176
 
@@ -1599,6 +2197,8 @@ letopis maintain --dsn <dsn> --schema <v2.name> [--token <token> | --api-key <ke
1599
2197
  letopis describe --dsn <dsn> --schema <v2.name> (--token <token> | --api-key <key>) [--format json|text] [--out <file>]
1600
2198
  letopis import --from <0.21 dsn> --from-schema <v1.name> --dsn <dsn> --schema <v2.name> [--tenant <name>] [--keyless <Class>]…
1601
2199
  [--rename <Old>=<New>]… [--update] [--commit-every <N>] [--partition <name>] [--initial]
2200
+ letopis mcp --dsn <dsn> --schema <v2.name> --token <user token> [--write]
2201
+ (or LETOPIS_DSN, LETOPIS_SCHEMA, LETOPIS_TOKEN; the service key --api-key is not accepted)
1602
2202
  ```
1603
2203
 
1604
2204
  ### Monitoring
@@ -1668,9 +2268,10 @@ The full machine-readable list of exports, methods and error texts is `docs/api-
1668
2268
 
1669
2269
  | Entry point | What it exports |
1670
2270
  |---|---|
1671
- | `letopis` | `connect`, `up`, `createTenant`; operators `ne`, `gt`, `gte`, `lt`, `lte`, `between`, `inList`, `like`, `ilike`, `starts`, `ends`, `has`, `hasAny`, `hasAll`, `exists`, `isNull`, `not`, `or`, `isOp`; `inc`, `isInc`, `cursorOf`; `idOf`, `uuidv5`, `v5Name`, `v5Id`; `LetopisError`, `ValidationError`, `fromDbError`; `syncModels`, `generateTypes`, `schemaToTs`; `compileValidator`, `applyDefaults`, `Registry`; `totpCode`, `hashPassword`, `verifyHash`; `describeText`; constants `DDL_REVISION`, `RESERVED_CLASS_NAMES`, `CHAIN_METHODS`, `DB_METHODS`; types `Db`, `DbTx`, `TypedDb`, `Chain`, `Row`, `Filter`, `Cursor`, `ConnectOptions`, `UpOptions`, `LoadOptions`, `LoadReport`, `WatchOptions`, `WatchEvent`, `AuthApi`, `AclApi` and others |
1672
- | `letopis/model` | `hub`, `link`, `one`, `many`, `End`, types `Infer`, `InferLinks`, `Model`, `ClassDef`, `ClassMeta`, `HistoryPolicy`; `normalizeSchema`, `translatePattern`, type `JsonSchema` |
1673
- | CLI `letopis` | `sync`, `types`, `load`, `verify`, `maintain`, `describe`, `import` ([section 13](#13-operations)) |
2271
+ | `letopis` | `connect`, `up`, `createTenant`; operators `ne`, `gt`, `gte`, `lt`, `lte`, `between`, `inList`, `like`, `ilike`, `starts`, `ends`, `has`, `hasAny`, `hasAll`, `exists`, `isNull`, `not`, `or`, `isOp`; `inc`, `isInc`, `cursorOf`; `idOf`, `uuidv5`, `v5Name`, `v5Id`; `LetopisError`, `ValidationError`, `fromDbError`; `syncModels`, `generateTypes`, `schemaToTs`; `compileValidator`, `applyDefaults`, `Registry`; `totpCode`, `hashPassword`, `verifyHash`; `describeText`; constants `DDL_REVISION`, `RESERVED_CLASS_NAMES`, `CHAIN_METHODS`, `DB_METHODS`; types `Db`, `DbTx`, `TypedDb`, `Chain`, `Row`, `Filter`, `Cursor`, `ConnectOptions`, `UpOptions`, `LoadOptions`, `LoadReport`, `WatchOptions`, `WatchEvent`, `AuthApi`, `AclApi`, `DiffResult`, `DiffChange`, `BlameResult`, `BlameField`, `Projector`, `ProjectorOptions`, `ProjectorChange`, `ProjectorOut`, `ProjectorRow`, `ProjectorReport`, `ProjectorStatus`, `ProjectorFailure` ([Projector](#projector)), `AggregateInfo` and others |
2272
+ | `letopis/model` | `hub`, `link`, `one`, `many`, `End`, types `Infer`, `InferLinks`, `Model`, `ClassDef`, `ClassMeta`, `HistoryPolicy`, `AggregateDef`; `normalizeSchema`, `translatePattern`, type `JsonSchema` |
2273
+ | `letopis/mcp` | `createMcpServer(db, options)`: an MCP SDK server with the letopis tools; the type `McpOptions`: `write` adds the `letopis_write` tool (absent by default), `maxRows` is the rows in an answer (100, from 1 to 10,000), `maxBytes` is the bytes in an answer (50,000, from 1,000 to 50,000,000); `allowAclWrite` lets `letopis_write` write the permission classes too (by default `acl_denied`); `allowSystemAdmin` accepts a session in the System tenant (by default `user_session_required`); a value out of range gives `invalid_data`. Needs the `@modelcontextprotocol/server` package ([section 13](#agents-and-mcp)) |
2274
+ | CLI `letopis` | `sync`, `types`, `load`, `verify`, `maintain`, `describe`, `import`, `mcp` ([section 13](#13-operations)); `mcp` takes `--dsn`, `--schema`, `--token` (or `LETOPIS_DSN`, `LETOPIS_SCHEMA`, `LETOPIS_TOKEN`) and `--write` |
1674
2275
 
1675
2276
  Removed 0.21 names (`uuidv7`, `LETOPIS_NS` and the methods listed below) throw `removed` with the replacement.
1676
2277
 
@@ -1685,6 +2286,8 @@ Removed 0.21 names (`uuidv7`, `LETOPIS_NS` and the methods listed below) throw `
1685
2286
  | `begin()`, `commit(tr)`, `rollback(tr)`, `lock(…keys)` | an explicit transaction ([section 8](#8-transactions-batches-and-money)) |
1686
2287
  | `batch(name)` | a queue of plans: `run()`, `discard()`, `size()` |
1687
2288
  | `load(input, options)` | the loader ([section 9](#9-bulk-loading)) |
2289
+ | `projector(name, options)` | a projector from the journal: `run()`, `rebuild({ chunked, tenant })`, `status()`, `stop()`, `close()` ([Projectors](#projectors)) |
2290
+ | `rebuildAggregates(class)` | repair of the aggregates of a class and its descendants in all tenants: the field anew from live sources, a version only for corrected rows; → their number ([Aggregates in the database](#aggregates-in-the-database)) |
1688
2291
  | `watch(options)`, `cache` | subscription; result cache (`clear()`, `stats()`) |
1689
2292
  | `` sql`…` ``, `transaction(fn)` | raw SQL under the session and RLS |
1690
2293
  | `idOf(class, key)` | the id of an object of a class with a key, in the session tenant |
@@ -1702,7 +2305,7 @@ Removed 0.21 names (`uuidv7`, `LETOPIS_NS` and the methods listed below) throw `
1702
2305
  | modifiers | `limit(n)`, `offset(n)`, `sort(field, 'asc' \| 'desc')`, `after(cursor)`, `asOf(moment)`, `withDeleted()`, `deep(max)`, `exact()`, `alias(name)`, `tags(…)`, `owner(…)`, `entity(…)`, `forUpdate()`, `cache({ ttl })` |
1703
2306
  | verbs | `create(data)`, `update(patch, { rev })`, `upsert(data)`, `delete({ confirm })`, `anonymize(fields)`, `reclass(class, data)`, `rekey(patch)`, `restore()`, `purge({ confirm })` |
1704
2307
  | slots | after a verb: `.RoleName.set(value)`, `.RoleName.unset()`, `.RoleName.add(value)`, `.RoleName.remove(value)` |
1705
- | terminals | `rows()`, `first()`, `ids()`, `count({ paths })`, `paths()`, `versions({ follow })`, `sum(field)`, `avg(field)`, `min(field)`, `max(field)`, `countBy(field)` |
2308
+ | terminals | `rows()`, `first()`, `ids()`, `count({ paths })`, `paths()`, `versions({ follow })`, `diff({ from, to, follow })`, `blame({ at, limit })`, `sum(field)`, `avg(field)`, `min(field)`, `max(field)`, `countBy(field)` |
1706
2309
 
1707
2310
  ### `db.auth`
1708
2311
 
@@ -1729,6 +2332,23 @@ its own key, but not its other credentials; nobody can revoke the last active ke
1729
2332
  `TOTP` method. The credential kind `kind` is `^[A-Z][A-Z0-9_]{1,31}$` (`PASSWORD`, `EMAIL`,
1730
2333
  `TELEGRAM` …); `SESSION`, `APIKEY`, `KEYSECRET`, `OTP`, `TOTP` and `KEY` are taken.
1731
2334
 
2335
+ ### Projector
2336
+
2337
+ `db.projector(name, options)` works only on an admin connection ([Projectors](#projectors)). The name is lowercase
2338
+ Latin letters, digits and `_.:-`, up to 64 characters; it is also the `derivedBy` of the derived classes. Wrong options
2339
+ give `invalid_data`.
2340
+
2341
+ | Member | What it does |
2342
+ |---|---|
2343
+ | options `ProjectorOptions` | `from`, `into`: classes with descendants; `version`: an integer from 1; `handle(changes, out)`; `withBefore`; `onExpired: 'rebuild' \| 'error'` (`'rebuild'` rebuilds in batches); `batch`: changes in a batch (500, from 1 to 10,000); `pollMs`: fallback polling (1000 ms); `lockTimeout`: waiting for another transaction's row lock (5000 ms, 0 — no limit); `tenants`: ids of the participant tenants |
2344
+ | `run({ once })` | catch up with the journal (`once`) or follow it until `stop()` → `ProjectorReport` |
2345
+ | `rebuild({ chunked, tenant })` | rebuild from the current state: atomically, in batches or for one tenant → `ProjectorReport` |
2346
+ | `status()` | → `ProjectorStatus`: `{ name, state, version, cursor, at, lag, failed, rebuildingTenants }`; `failed` is `{ [tenant id]: ProjectorFailure }`, where `ProjectorFailure` is `{ cursor, code, message }` |
2347
+ | `stop()`, `close()` | stop `run()` after the current batch; the same plus unsubscribing from signals |
2348
+ | `ProjectorReport` | `{ batches, changes, created, updated, deleted, unchanged, skipped, rebuilds }` |
2349
+ | `ProjectorChange` | `{ cursor, id, class, tenant, op, rev, at, author, row }` and, when present, `agent`, `reason`, `moved`, `before`, `rebuild` |
2350
+ | `ProjectorOut` | `upsert(class, row, { tenant })`, `inc(class, row, deltas, { tenant })`, `delete(class, key or id, { tenant })`, `get(class, key or id, { tenant })` → `Row \| null`; a row is a `ProjectorRow` in the loader format |
2351
+
1732
2352
  ### Result row
1733
2353
 
1734
2354
  | Field | Meaning |
@@ -1754,8 +2374,8 @@ database does not accept a link with such a name (`reserved_names()`; the instal
1754
2374
  `owner`, `forUpdate`, `cache`, `rekey`, `diff`, `blame`; guards of the removed API: `run`, `execute`, `account`;
1755
2375
  - the `db` root: `as`, `begin`, `commit`, `rollback`, `lock`, `batch`, `watch`, `close`, `registry`, `sql`, `load`,
1756
2376
  `verify`, `maintain`, `reset`, `describe`, `auth`, `acl`, `idOf`, `refresh`, `transaction`, `flushSignals`,
1757
- `schema`, `token`, `tenant`, `trimHistory`, `rekeyClass`, `cache`; guards: `reloadSchema`, `accounts`,
1758
- `credentials`, `resources`, `rules`, `compact`;
2377
+ `schema`, `token`, `tenant`, `trimHistory`, `rekeyClass`, `cache`, `projector`, `rebuildAggregates`; guards: `reloadSchema`,
2378
+ `accounts`, `credentials`, `resources`, `rules`, `compact`;
1759
2379
  - batch: `discard`, `size`.
1760
2380
 
1761
2381
  ## 16. Errors
@@ -1775,16 +2395,19 @@ Database codes. Raw SQL gets them too: SQLSTATE class `LT`, the details are JSON
1775
2395
  | `acl_denied` | LT006 | denied by the access rules |
1776
2396
  | `id_mismatch`, `id_from_db` | LT007, LT008 | the id is not v5 of the key; the id of a class without a key is assigned by the database |
1777
2397
  | `immutable_key` | LT009 | a key field, the id, the tenant and the owner do not change (a new key is `rekey`) |
1778
- | `invalid_class` | LT010 | the class description is rejected, the class is not found or is abstract |
2398
+ | `invalid_class` | LT010 | the class description is rejected, the class is not found or is abstract; a class with rows, descendants or a history of rows is not deleted |
1779
2399
  | `use_reclass`, `reclass_denied` | LT011, LT012 | changing the class with `update` when the old or the new class has a key: the id changes, so use `reclass()`; the roles of referencing rows do not accept the new class |
1780
2400
  | `tightening_conflict` | LT013 | rows prevent tightening the description |
1781
2401
  | `isolation_level` | LT014 | deleting, changing the class and changing the key work only in `read committed` |
1782
2402
  | `recreated_in_statement` | LT015 | an id was deleted and created again in one statement |
1783
- | `cursor_expired` | LT016 | the subscription cursor is older than the `all` tier of the retention policy |
2403
+ | `cursor_expired` | LT016 | the cursor of a subscription or a projector is older than the `all` tier of the retention policy |
1784
2404
  | `delete_restricted` | LT017 | referencing rows prevent the deletion (`onDelete: 'restrict'`) |
1785
2405
  | `not_deleted` | LT018 | `restore` and `purge` work only for deleted objects |
1786
2406
  | `reset_disabled` | LT019 | reset is disabled in the installation (`allowReset`) |
1787
2407
  | `confirm_mismatch` | LT020 | the reset confirmation does not match the schema name |
2408
+ | `derived_class` | LT021 | a row of a derived class (`derivedBy`) is written, edited or deleted by someone other than its projector; an end leads to such a row or to a derived class; a projector writes to a class outside its `into` or has found no tenant in which every `into` class is marked with its `derivedBy` |
2409
+ | `aggregate_field` | LT022 | the application passed or changed an aggregate field (`aggregate`): the database calculates it |
2410
+ | `projector_pending` | LT023 | `purge` of a deleted object whose deletion a projector of its class has not read yet, or `trimHistory` whose first remaining version it has not read yet: run the projector first |
1788
2411
 
1789
2412
  Library codes:
1790
2413
 
@@ -1801,10 +2424,14 @@ Library codes:
1801
2424
  | `no_key` | `upsert`, `rekey` or `idOf` on a class without a key |
1802
2425
  | `no_path` | there is no hop between the classes of neighboring steps |
1803
2426
  | `invalid_query` | the chain or the API is used incorrectly (a step modifier `deep`, `exact`, `alias`, `tags` or `owner` after a verb, a slot not after a verb, an `after()` cursor with a value of a different type than the `sort` field, without `v` or with an `id` that is not a uuid, and so on) |
1804
- | `admin_required` | the loader needs an admin connection |
2427
+ | `admin_required` | the loader, a projector and `rebuildAggregates` need an admin connection |
2428
+ | `rebuild_required` | the projection was built by another version of the projector (`version`): a rebuild with `rebuild()` is needed; `rebuild({ tenant })` while there is no projection or a batch rebuild is in progress |
2429
+ | `projector_busy` | a rebuild cannot start: the projector is running in another process right now |
1805
2430
  | `not_empty` | the initial load (`initial`) goes only into an empty schema |
1806
2431
  | `class_not_loadable` | the loader does not accept `Class` rows |
1807
- | `bypass_rls` | `connect()` with a role that bypasses RLS, without `allowBypassRls` |
2432
+ | `bypass_rls` | `connect()` with a role that bypasses RLS, without `allowBypassRls`; `createMcpServer` with a connection that bypasses RLS |
2433
+ | `user_session_required` | `createMcpServer` and `letopis mcp`: a user session is needed — the connection has no session, it is the service root without `as()`, a session in the System tenant without `allowSystemAdmin`, or the command line got the service key or no token |
2434
+ | `confirmation_required` | `letopis_write` (MCP): a deletion without a human confirmation — the client has no elicitation, the human declined, the answer belongs to another operation, or the data changed after the preview |
1808
2435
  | `removed` | a removed 0.21 API; the text gives the replacement |
1809
2436
 
1810
2437
  A PostgreSQL permission denial (`42501`) comes, for example, on an `upsert` over someone else's row that you may not
@@ -1847,9 +2474,26 @@ load in 77 s (13,902 rows/s, 1.4 GB of WAL). Most of the difference comes from t
1847
2474
  the load costs 36 s, building them at the end about 5 s. Preparing rows in the loader itself (checks, ends, ids) runs
1848
2475
  alongside the `COPY` of the previous batch and now sets the pace: about 60 s of the 77.
1849
2476
 
2477
+ Derived data in 1.2 (`cd lib && npx tsx bench/derived.bench.mjs`, 2026-10-27, PostgreSQL 18.6, one machine; for
2478
+ aggregates, the median of three rounds; the projector rows are a repeat of series 2 and 3 after the folding of `inc` in
2479
+ `out`, `BENCH_SERIES=2,3`):
2480
+
2481
+ | Measurement | 1.2 | For comparison |
2482
+ |---|---|---|
2483
+ | an operation with an aggregate, one writer | 94 operations/s, p50 10.7 ms | without an aggregate 137 and 6.9 ms; the `inc` recipe 69 |
2484
+ | operations on one wallet with an aggregate, 20 writers | 152 operations/s, p95 0.47 s | without an aggregate 1,987; the `inc` recipe 190 |
2485
+ | operations on 20 wallets with an aggregate, 20 writers | 1,379 operations/s | without an aggregate 2,023 |
2486
+ | projector catch-up, `out.inc` for every change | 6,076 and 33,395 changes/s (`batch` 500 and 2000) | before the folding of `inc` in `out`: 992 and 1,203 |
2487
+ | rebuild of 75 thousand rows | atomic 13.3–13.5 s, in batches 13.0–13.2 s | before the folding of `inc` in `out`: 101 s and 58–63 s |
2488
+ | delay of a subscription event during a rebuild | atomic up to 13.4 s, in batches up to 0.21 s | without a rebuild up to 0.12 s |
2489
+
2490
+ The aggregate triggers fire on every write statement, but for classes without aggregates the measurement does not see
2491
+ their cost: the p50 difference is from −0.28 to +0.21 ms. The benchmark against 1.0.1 (`bench/vs-101.bench.mjs`, four
2492
+ runs) showed no write or read regressions against 1.1: two outliers did not repeat in separate checks.
2493
+
1850
2494
  ## 18. Limitations
1851
2495
 
1852
- This section lists what letopis 1.1 deliberately does not do or does with caveats, and where you need to watch the
2496
+ This section lists what letopis 1.3 deliberately does not do or does with caveats, and where you need to watch the
1853
2497
  behavior.
1854
2498
 
1855
2499
  **Platform**
@@ -1935,13 +2579,22 @@ behavior.
1935
2579
  attempts.
1936
2580
  - When users link a taken identifier themselves, the answer is `acl_denied`, so a user with a recent sign-in learns
1937
2581
  this way that the identifier is taken. Permission changes in all tenants are serialized through one counter row.
1938
- - The number of invisible rows that block a deletion, and the existence of an account by id (as a reference target),
1939
- are not hidden.
2582
+ - Refusal texts name the class, role and id of a referencing row only if the actor can see it: a cascade delete,
2583
+ unsetting an end, rewriting references on a class or key change, and a `parent` change conflict
2584
+ (`tightening_conflict`). The number of invisible rows that block a deletion or a
2585
+ class change (`reclass_denied`), and the existence of an account by id (as a reference target), are not hidden.
2586
+ - Tightening a class description and `rekeyClass` check all rows of the family, so they need the `READ` permission
2587
+ without conditions on all rows of the class and its descendants: otherwise a trial tightening (`maximum`, `enum`,
2588
+ `pattern`) would answer with the number of invisible rows that do not match it. A system administrator who tightens
2589
+ a System class gets the number of violating rows of other tenants (`others`): this is the installation level.
1940
2590
  - Class metadata (`meta`) is visible to anyone who has a session: anyone can read `Class` rows. Secrets do not belong
1941
2591
  there.
1942
- - An application role can slow the installation down for everyone: `LOCK TABLE entity`, advisory locks with
1943
- predictable keys, fake `NOTIFY` signals (they only clear the cache), frequent permission changes (one counter row per
1944
- installation). The global journal number reveals the overall write activity.
2592
+ - An application role can slow the installation down for everyone: `LOCK TABLE entity`, advisory locks with the keys
2593
+ of the application's `lock()`, fake `NOTIFY` signals (they only clear the cache), frequent permission changes (one
2594
+ counter row per installation). The global journal number reveals the overall write activity. Internal locks — class
2595
+ families, projectors and maintenance — are rows of the `secret` table, not advisory locks: neither the application
2596
+ role nor any other database role can take them. A family writer passes the tightening gate in a subtransaction — one
2597
+ per transaction for the first write to the family.
1945
2598
 
1946
2599
  **History, signals, cache**
1947
2600
 
@@ -1969,6 +2622,12 @@ behavior.
1969
2622
  - The `NOTIFY` queue (8 GB) is shared by the whole cluster: a stuck listener fills it up, and then transactions with
1970
2623
  `NOTIFY` fail on commit. The library reads signals right away, and `maintain()` warns when the queue is more than
1971
2624
  half full.
2625
+ - PostgreSQL commits transactions with `NOTIFY` one at a time: a commit holds the cluster-wide lock of the signal queue
2626
+ together with the disk write. A raw SQL write on its own connection sends the signal from a trigger in the same
2627
+ transaction, so concurrent raw SQL writers commit one after another: on the benchmark machine, 20 such writers did
2628
+ about 520 inserts per second. Library transactions, `db.sql` and `db.transaction()` included, commit the data
2629
+ without `NOTIFY` and send the signals later, separately, so their writers do not wait in this queue: 20 library
2630
+ writers did about 2,000 operations per second.
1972
2631
  - The subscription horizon is shared by the whole cluster: a long write transaction delays the events of all
1973
2632
  subscribers (`sub.lag()`); long operations are better done in batches.
1974
2633
  - Between processes, the result cache goes stale for the signal sending interval (100 ms) plus delivery, and for the
@@ -1977,6 +2636,67 @@ behavior.
1977
2636
  - A "from now" subscription cursor carries a snapshot of the subscription moment until it passes that boundary: such a
1978
2637
  cursor is longer than a regular one.
1979
2638
 
2639
+ **Projectors**
2640
+
2641
+ - A projector is an admin process: it works only on an admin connection, reads the journal of all tenants bypassing
2642
+ RLS and writes its results as the owner role. Keep it in a separate process, like the loader. A participant of a
2643
+ projector is a tenant in which all `into` classes are marked with its `derivedBy`; only an administrator may mark a
2644
+ class (the `letopis.projector` permission).
2645
+ - The totals of a derived class are seen by everyone whom the tenant rules give `READ` on it — even by those who do not
2646
+ read the source rows: decide who reads it with rules on the derived class itself. With `history: true`, its journal
2647
+ shows the change of every batch, that is, the contribution of single source rows too.
2648
+ - Only their projector writes the rows of derived classes: the application, raw SQL and the loader get
2649
+ `derived_class`, and ends may not refer to such rows.
2650
+ - An atomic rebuild is one long write transaction: while it runs, the subscription horizon stands still, and events
2651
+ are delayed for the subscribers and projectors of all processes. On the benchmark machine, a rebuild of 75 thousand
2652
+ rows took 13 s, and the subscriber got no event until it ended. That is why `run()` itself rebuilds in batches, and
2653
+ only an explicit `rebuild()` runs an atomic rebuild.
2654
+ - A batch rebuild holds the horizon only for the time of a batch (in the same measurements, events were delayed by up
2655
+ to 0.2 s), but the derived data is incomplete until it ends: the rows are deleted at once and reappear as the
2656
+ rebuild goes on.
2657
+ - A projector sees only the changes from the journal, so a class with `history: false` cannot be a source: a tenant
2658
+ with such a source drops out of the projector (`invalid_class`) until the journal of the class is turned on and the
2659
+ tenant is rebuilt.
2660
+ - Thinning the journal older than the `all` tier leads to a rebuild: a projector that lags behind by more than that
2661
+ tier gets `cursor_expired` or, with `onExpired: 'rebuild'`, rebuilds itself. The tier is per tenant: if it has
2662
+ expired only for some participants, only they drop out and rebuild.
2663
+ - A projector works only on a schema with the engine revision the library expects; on a schema of another revision it
2664
+ answers `invalid_data`.
2665
+ - `purge()` waits for projectors: while a projector whose sources include the class of the object, and which has a
2666
+ derived class in the object's tenant, has not read the deletion, `purge` answers `projector_pending`. A projector
2667
+ that is no longer run holds `purge` of its sources while the tenant has its derived classes: run it or delete them.
2668
+ `trimHistory` waits the same way: while the projector has not read the first remaining version, trimming answers
2669
+ `projector_pending`.
2670
+
2671
+ **Aggregates in the database**
2672
+
2673
+ - An aggregate queues the writers of one object: every operation locks the target row and writes its version. On the
2674
+ benchmark machine, 20 writers of one wallet with an aggregate did 152 operations per second (p95 is 0.47 s), without
2675
+ an aggregate 1,987. For hot counters, use a projector that folds a batch.
2676
+ - `reclass` and `rekey` accumulate contributions until the end of the move, while a delete cascade and the loader apply
2677
+ them per statement: the class rule is checked on the total of each cascade statement and of each loader batch. So a
2678
+ load in which debits come in a batch before top-ups gets `invalid_data`, although the total of the whole load is
2679
+ valid. Every batch that touches a target writes its version.
2680
+ - A tenant aggregate does not count rows of a System class, and the field of a System row counts only System sources.
2681
+
2682
+ **Agents and MCP**
2683
+
2684
+ - The `<untrusted-data-…>` wrapper is a mitigation, not a protection: the model may follow the text inside it. The
2685
+ protection is the rights in the database: the agent reads and writes only what its user is allowed to. In
2686
+ `structuredContent`, the database data lies in `data` and `error` next to the `notice`, without marks: a client that
2687
+ gives the model that instead of the text passes the data with the note but without a random mark.
2688
+ - By default the server only reads; writing is `write: true` (`--write`); deletion is only with a human confirmation.
2689
+ The server writes the permission classes only with `allowAclWrite` and accepts a session in the System tenant only
2690
+ with `allowSystemAdmin`; `create`, `update` and `upsert` of the other classes need no human confirmation.
2691
+ - The HTTP handler serves revision 2025 statelessly: the per-request instance does not know the client's
2692
+ capabilities, so deletion through it is refused (`confirmation_required`). Delete over stdio or in revision
2693
+ 2026-07-28.
2694
+ - A confirmation in revision 2026-07-28 is signed with a process key: a retried call that lands in another process of
2695
+ the application (several instances behind a load balancer) is refused with `confirmation_required`.
2696
+ - `versions` returns the latest versions within `maxRows` and `maxBytes` (at most `maxRows` + 1 versions are read
2697
+ from the database) and `more`; earlier ones, through `diff` and `blame` with `from`, `to`, `at`. A mode where the
2698
+ agent writes code against the API (code mode) is not part of the package.
2699
+
1980
2700
  **Import from 0.21**
1981
2701
 
1982
2702
  - 1.0 is a new major version, not a continuation of 0.21: `count()`, `paths()`, `create`, roles, multiple ends and
@@ -1994,7 +2714,8 @@ This section is for those who develop letopis itself. Run: `cd lib && npm test`.
1994
2714
  # connect — connecting under a session, refusing roles that bypass RLS, the registry, the deferred signal, error translation (stage 2)
1995
2715
  # context — sessions, service sign-in by key, the RLS policies of stage 1 (stage 1)
1996
2716
  # access-rest — the remaining rows of stage 5: loading on behalf of a user, balance No. 20, 22, 29, upsert and { rev } under ACL, inlining of acl_ok (stage 5)
1997
- # acl — ACL permissions: the 0.21 suite ported, row conditions, weight and default deny, cascade, deny on a descendant, deep, an invisible target, raw SQL versus the library (stage 5)
2717
+ # aggregate — aggregates in the database: 20 writers of one wallet, raw SQL, a cascade, reclass, restore, the loader, account deletion, restore and rekey of the target, a description change under writes and race No. 18, aggregate_field, repair and verify, a System row (plan 1.1, stage 4b)
2718
+ # acl — ACL permissions: the 0.21 suite ported, row conditions, weight and default deny, cascade, deny on a descendant, deep, an invisible target, raw SQL versus the library; a change of parent and alias of a class only with permissions on all its rows, an alias is not the name of another class, a name is not another class's alias (stage 5; security review No. 2)
1998
2719
  # acl-parity — 1000 random rule sets: the database permission plan and endpoint decisions versus decide from 0.21 (stage 5)
1999
2720
  # auth — credentials (password, key, key + secret, OTP, TOTP, identity), timing equalization, case, session revocation, recent sign-in (stage 5)
2000
2721
  # balance — top-up and debit: idempotency by opId, races, deadlock, connection drop (TCP proxy), adjustment under { rev } and forUpdate (stage 4)
@@ -2004,16 +2725,16 @@ This section is for those who develop letopis itself. Run: `cd lib && npm test`.
2004
2725
  # chain-write — writing with chains: create, update, upsert, delete with preview, anonymize, reclass, slots, inc, { rev }, fan-out, plan rollback (stage 4)
2005
2726
  # demo — the booking demo domain on Zod: seed from models, writing data, id by key, agreement between Zod and the database (stage 2)
2006
2727
  # docs — documentation examples are executed: the marked blocks of the README and the cheatsheet (Russian and English) run as scenarios against PostgreSQL 18 (stage 8)
2007
- # engine-attacks — raw SQL attacks from the application role: the rows of §7.3.1 for stages 1 and 3 with specific outcomes; the signal key without permission, flooding with fake signals, Class included (stages 1, 3; plan 1.1, stage 2 and checkpoint A)
2008
- # engine-class — classes as entities: metaschema, inheritance, recomputing descendants, tightening, metadata (stage 1)
2728
+ # engine-attacks — raw SQL attacks from the application role: the rows of §7.3.1 for stages 1 and 3 with specific outcomes; the signal key without permission, flooding with fake signals, Class included; writing to a derived class and a reference to it, a projector checkpoint through the credential functions; an agent through MCP without --write and without a human confirmation; a change of parent, alias and derivedBy of a class with the WRITE permission on Class alone, deleting and re-creating a class with a history of rows, a trial tightening as an oracle, reclass without DELETE and rewriting references deeper than the first level, a retention policy with the WRITE permission on Class alone; an agent through MCP grants a membership or a rule, a system administrator session; a tenant user creates a derived class on their own, `rekeyClass` of an ancestor of a derived class; moving a row in a way that would rewrite references of derived class rows; the application role holds advisory locks with the former keys of internal locks, while lock rows are out of its reach; a forged note of the family locks already taken (stages 1, 3; plan 1.1, stages 2, 4a, 5, checkpoint A, security review No. 2, checkpoint B and the work after it, security review No. 3)
2729
+ # engine-class — classes as entities: metaschema, inheritance, recomputing descendants, tightening, metadata; a class with a history of rows is deleted only after purge() (stage 1; the work after checkpoint B)
2009
2730
  # engine-write — the write pipeline in raw SQL: v5 ids, defaults, merge, inc, deletion, resurrection, journal, signals (stage 1)
2010
- # engine-races — races on two connections in both orders, without timed pauses (stage 1)
2011
- # explain — chain indexes from the application role: outside, the primary key and (class, tenant); inside find_*(), GIN (stage 3)
2012
- # find-ids-parity — 30 chains through find_*() and through a direct read under RLS give the same answers (stage 3)
2731
+ # engine-races — races on two connections in both orders, without timed pauses; tightening during continuous writes to the family does not starve; locking families whose roots differ only in case, and a planted row of kind LOCK (stage 1; security review No. 3)
2732
+ # explain — chain indexes from the application role: outside, the primary key and (class, tenant); inside find_*(), GIN; an asOf slice by class does not read another tenant's journal (stage 3; the work after checkpoint B)
2733
+ # find-ids-parity — 30 chains through find_*() and through a direct read under RLS give the same answers; own memberships in other tenants and the Class rows of System — now, asOf and withDeleted (stage 3; the work after checkpoint B)
2013
2734
  # hash — the version hash chain, journal tampering and gaps, verify() (stage 1)
2014
2735
  # harness — the test harness: starting the database, a schema per file, two connection strings (stage 0)
2015
2736
  # jsonschema — compiling JSON Schema to jsonpath and checking data in the database (stage 1)
2016
- # impersonation — impersonation: letopis.account only for a service with the permission, target restrictions, author and agent in the journal, sessionFor (stage 5)
2737
+ # impersonation — impersonation: letopis.account only for a service with the permission, target restrictions, author and agent in the journal, sessionFor; the db.as() registry is one per tenant, does not empty, is re-read on a signal of its tenant (stage 5; the work after checkpoint B); the facade registry is not shared with the root, an empty registry is not cached (security review No. 3 leftovers)
2017
2738
  # install — the installer and roles: a repeated installation, two installations in one database, three tables (stage 1)
2018
2739
  # member — memberships: working in a tenant by roles, auth.switch, leaving with ownership transfer, system rows, tenant roles without System permissions (stage 5)
2019
2740
  # model — Zod models, the JSON Schema normalizer, pattern translation, agreement with the database metaschema (stage 2)
@@ -2037,11 +2758,14 @@ This section is for those who develop letopis itself. Run: `cd lib && npm test`.
2037
2758
  # cache — result cache: hits, own and others' writes, the key with the user, permissions, revoked sessions, listener drop, ttl, class marking, without LISTEN, a race, db.sql, the limit of signal counters (stage 6; plan 1.1, checkpoint A)
2038
2759
  # maintain — maintenance: expired sessions and codes, permissions, one process under a lock, schedule (stage 6)
2039
2760
  # up — the full installer: fresh installation, re-apply, custom seeds, automatic database creation, UTF8, version 17, upgrade with migrations, PGlite, the first tenant and user (up({ tenant }), createTenant) (stages 6, 8)
2761
+ # upgrade — schema upgrade from 1.1.0 (re-apply) and from 1.0.1 (upgrade: true, migration 0006) with the letopis-11 and letopis-1 packages: versions and hashes written by the old engine, verify(), the catalog against a clean installation, lock rows, the application, a subscription and a projector over old data, a repeated up() (work after 1.2.0, stage 1)
2040
2762
  # scenario-salon — end-to-end salon scenario: 21 acts from 0.21 (salon) with a coverage matrix of the 1.0 API (stage 6)
2041
2763
  # telemetry — onQuery and slowMs, describe() and class metadata with inheritance, describeText (stage 6)
2042
2764
  # history — diff and blame over the log, the reason of a change for verbs, db.begin() and batches; a version hidden by an ACL condition, the changed column, a missing version, row_changes over 20 thousand targets (plan 1.1, stage 1 and checkpoint A)
2043
2765
  # signals — signal modes: the payload seen by a foreign role, cache precision per tenant, System, permission classes, a process without the key, the loader, subscriptions, the key, none, coalescing, the server version, a class change, a resend, the mode after up() and reset({ all }) (plan 1.1, stage 2)
2044
- # initial-load — the initial load: equality with the regular loader, structure and trigger modes, system rows, a non-empty schema, an insert during the start, rollback, MULTI_INSERT in WAL, commitEvery, import --initial (plan 1.1, stage 3)
2766
+ # initial-load — the initial load: equality with the regular loader, structure and trigger modes, system rows, a non-empty schema, an insert during the start, rollback, MULTI_INSERT in WAL, commitEvery, import --initial, aggregates (agg_fill) (plan 1.1, stages 3 and 4b)
2767
+ # projector — projectors from the journal: catching up, exactly once on a break, one writer, a long transaction, a transaction on the boundary of the rebuild snapshot, no writes to derived classes and no references to them, the bounds of out, atomic and batch rebuilds during source writes, purge of a source, lag and cursor_expired, the credential functions; the version, the name, an admin connection; a race of two processes; a data error of one tenant drops out only this tenant (failed, rebuild({ tenant }), the tenants option), the checkpoint is only a System row, a rebuild touches only the rows of participants and into families, before after rekey, the all tier and the journal of sources per tenant; folding of out.inc by key in a batch, batch rebuilds on the automatic paths of run() and a second process during them; rebuilding one tenant in batches while the tail goes on; onExpired in batches in another process, a data error without retries, the tier from the start of the rebuild; an empty last pass of a batch rebuild, a tenant that drops out in the middle of its rebuild keeps no partial rows, a class change that keeps the id out of the source family and into it, purge of an unread tombstone (plan 1.1, stage 4a; security review No. 2; checkpoint B and the work after it; security review No. 3)
2768
+ # mcp — MCP tools: the model and meta only in results, tenant rights, the JSON chain and hints of similar names, row and byte limits with a cursor, the untrusted-data wrapper, writing only with write, update only with rev, deletion with a human confirmation (elicitation in revisions 2025 and 2026-07-28), the reason in blame, credentials, tool schemas, letopis without the SDK; letopis mcp over stdio, the HTTP handler; the confirmation window without the alias and meta, countBy and versions limited in the query itself, limits of steps and $or conditions; the permission classes only with allowAclWrite, a session in the System tenant only with allowSystemAdmin; structuredContent with a note, a requestState signature of the server user with an expiry (plan 1.1, stage 5; security review No. 2; checkpoint B and the work after it)
2045
2769
  # import — import from 0.21 with a fixture of a real 0.21: key violation, --keyless, per-class counters, verify, self-reference, repeat, ids without a key, a dangling reference, $$strict and fields starting with $, update, credentials, CLI (stage 7)
2046
2770
  # security-final — the final security review (stage 8, §7.10 No. 8.4): a family lock by the tenant of the root, account_purge without permission and restore/purge of a foreign id give neither a leak nor a lock; the functions of stage 6, log_refs_at, journal-based find_* reads and watch_read stay within their own tenant; NOTIFY carries only names or mode tokens; the cache key distinguishes the token, the actor and the tenant (stage 8)
2047
2771
  ```