letopis 1.4.2 → 3.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (123) hide show
  1. package/AGENT-CHEATSHEET.md +516 -398
  2. package/CHANGELOG.md +739 -0
  3. package/LICENSE +75 -21
  4. package/LICENSE-COMMERCIAL.md +49 -0
  5. package/README.md +1312 -1190
  6. package/api-contract.json +381 -0
  7. package/dist/acl.d.ts +81 -1
  8. package/dist/acl.js +281 -13
  9. package/dist/admin.d.ts +11 -141
  10. package/dist/admin.js +28 -186
  11. package/dist/auth.d.ts +20 -5
  12. package/dist/auth.js +80 -21
  13. package/dist/cache.d.ts +8 -1
  14. package/dist/cache.js +30 -8
  15. package/dist/chain.d.ts +46 -48
  16. package/dist/chain.js +556 -223
  17. package/dist/cli.js +73 -96
  18. package/dist/demo/booking.d.ts +352 -36
  19. package/dist/describe.d.ts +47 -0
  20. package/dist/describe.js +195 -0
  21. package/dist/errors.d.ts +77 -13
  22. package/dist/errors.js +231 -30
  23. package/dist/index.d.ts +62 -74
  24. package/dist/index.js +312 -155
  25. package/dist/internal.d.ts +7 -0
  26. package/dist/jsonschema.js +18 -17
  27. package/dist/load.d.ts +29 -17
  28. package/dist/load.js +235 -190
  29. package/dist/log.d.ts +121 -0
  30. package/dist/log.js +633 -0
  31. package/dist/mcp.d.ts +5 -7
  32. package/dist/mcp.js +194 -225
  33. package/dist/messages/access.d.ts +90 -0
  34. package/dist/messages/access.js +113 -0
  35. package/dist/messages/api.d.ts +237 -0
  36. package/dist/messages/api.js +304 -0
  37. package/dist/messages/boot.d.ts +201 -0
  38. package/dist/messages/boot.js +256 -0
  39. package/dist/messages/codes.d.ts +172 -0
  40. package/dist/messages/codes.js +173 -0
  41. package/dist/messages/index.d.ts +98 -0
  42. package/dist/messages/index.js +121 -0
  43. package/dist/messages/log.d.ts +229 -0
  44. package/dist/messages/log.js +102 -0
  45. package/dist/messages/mcp.d.ts +306 -0
  46. package/dist/messages/mcp.js +387 -0
  47. package/dist/messages/model.d.ts +256 -0
  48. package/dist/messages/model.js +314 -0
  49. package/dist/messages/query.d.ts +141 -0
  50. package/dist/messages/query.js +171 -0
  51. package/dist/messages/sql-access.d.ts +70 -0
  52. package/dist/messages/sql-access.js +87 -0
  53. package/dist/messages/sql-core.d.ts +164 -0
  54. package/dist/messages/sql-core.js +198 -0
  55. package/dist/messages/sql-time.d.ts +86 -0
  56. package/dist/messages/sql-time.js +111 -0
  57. package/dist/messages/sql-write.d.ts +246 -0
  58. package/dist/messages/sql-write.js +270 -0
  59. package/dist/messages/time.d.ts +23 -0
  60. package/dist/messages/time.js +29 -0
  61. package/dist/messages/types.d.ts +27 -0
  62. package/dist/messages/types.js +5 -0
  63. package/dist/messages/write.d.ts +452 -0
  64. package/dist/messages/write.js +572 -0
  65. package/dist/model.d.ts +77 -60
  66. package/dist/model.js +43 -34
  67. package/dist/ops.d.ts +35 -0
  68. package/dist/ops.js +101 -0
  69. package/dist/pglite.d.ts +0 -9
  70. package/dist/pglite.js +3 -1
  71. package/dist/registry.d.ts +3 -19
  72. package/dist/registry.js +3 -16
  73. package/dist/signals.d.ts +1 -5
  74. package/dist/signals.js +1 -5
  75. package/dist/sql.d.ts +9 -1
  76. package/dist/sql.js +175 -107
  77. package/dist/sync.d.ts +10 -4
  78. package/dist/sync.js +90 -17
  79. package/dist/tx.d.ts +21 -2
  80. package/dist/tx.js +76 -13
  81. package/dist/typed.d.ts +252 -53
  82. package/dist/types.d.ts +93 -90
  83. package/dist/types.js +17 -14
  84. package/dist/up.d.ts +56 -35
  85. package/dist/up.js +229 -83
  86. package/dist/uuid.d.ts +8 -0
  87. package/dist/uuid.js +14 -6
  88. package/dist/validate.d.ts +13 -1
  89. package/dist/validate.js +82 -33
  90. package/dist/watch.d.ts +10 -16
  91. package/dist/watch.js +36 -77
  92. package/dist/write.d.ts +87 -17
  93. package/dist/write.js +441 -206
  94. package/llms.txt +48 -31
  95. package/package.json +13 -14
  96. package/sql/10-core.sql +7 -50
  97. package/sql/15-errors.sql +64 -10
  98. package/sql/20-context.sql +18 -9
  99. package/sql/30-validate.sql +79 -30
  100. package/sql/40-class.sql +41 -115
  101. package/sql/50-acl.sql +8 -13
  102. package/sql/60-write.sql +220 -676
  103. package/sql/70-read.sql +11 -13
  104. package/sql/80-auth.sql +109 -42
  105. package/sql/90-time.sql +108 -947
  106. package/sql/95-seed.system.sql +7 -14
  107. package/sql/99-revision.sql +6 -1
  108. package/sql/README.md +32 -24
  109. package/sql/migrate/0009-restore-rev.tenants-name.sql +10 -0
  110. package/sql/migrate/0010-impersonation-no-account.sql +5 -0
  111. package/sql/migrate/0011-key-lock.otp-enabled.sql +7 -0
  112. package/sql/migrate/README.md +18 -0
  113. package/AGENT-CHEATSHEET.en.md +0 -488
  114. package/MIGRATION.md +0 -190
  115. package/README.en.md +0 -2910
  116. package/dist/import.d.ts +0 -185
  117. package/dist/import.js +0 -820
  118. package/dist/project.d.ts +0 -359
  119. package/dist/project.js +0 -1258
  120. package/sql/75-history.sql +0 -436
  121. package/sql/88-derived.sql +0 -1236
  122. package/sql/migrate/0006-changed-paths.sql +0 -3
  123. package/sql/migrate/0007-secret-account-fk.sql +0 -12
@@ -1,488 +0,0 @@
1
- # letopis 1.4 — agent cheatsheet
2
-
3
- One page instead of [README.en.md](README.en.md), for an AI agent that writes code on top of the package. It ships in
4
- the package next to the README; in Russian: [Русская версия](AGENT-CHEATSHEET.md). Migration from 0.21:
5
- [MIGRATION.md](MIGRATION.md) (in Russian); the machine-readable catalog of the API and errors is `docs/api-contract.json`
6
- in the repository. The examples below go in order and form one program; the `docs` test runs it against PostgreSQL 18,
7
- so the code is checked against the package.
8
-
9
- ## The model in a minute
10
-
11
- - PostgreSQL 18+, Node 22+. Three tables in the installation schema `v2.<name>`: `entity` (the current version of each
12
- entity), `log` (the journal of all versions), `secret` (credentials and sessions). Classes, accounts and access rules
13
- are `entity` rows too.
14
- - Checks are done by **the database**: triggers (data against JSON Schema, references, keys, the journal) and RLS
15
- policies (tenant and access rules). Raw SQL from the application role behaves the same way as the library.
16
- - A hub (`hub`) is an entity; a link (`link`) is an entity with ends. Ends are in `row.links`, fields are in `row.data`.
17
- - id: for a class with a key it is a UUID v5 of (class, key values) within the tenant, and it can be computed in
18
- advance (`db.idOf`); for a class without a key the database assigns the id. Only the key gives uniqueness.
19
- - Tenant = account. Rows belong to the **session** tenant; rows of other tenants are not visible. Access rules are
20
- always on and **deny** by default.
21
-
22
- ## Working cycle
23
-
24
- ### 1. Installation: `up()` and the application role
25
-
26
- <!-- run -->
27
- ```ts
28
- import { up, connect, syncModels, inc, gt, or, cursorOf } from 'letopis'
29
-
30
- const ADMIN = 'postgres://postgres:pass@localhost:5432/app' // superuser or a role with CREATEROLE
31
- const r = await up({ dsn: ADMIN, schema: 'myapp', // schema 'v2.myapp'
32
- tenant: { name: 'Salon', user: { name: 'Olga', login: 'olga@salon.ru', password: 'Lilac-2026!' } } })
33
- // r: { schema, created, upgraded, revision, serviceKey, migrations, analyzed, tenant, … }
34
- // r.serviceKey ('lts_…') is the service key; it is issued at the first installation (again only if the installation has
35
- // no API keys left except revoked ones). Save it.
36
- // r.tenant: { tenant, user, token, created } — the tenant "Salon" and its first user Olga
37
- ```
38
- - `up()` waits for the database, creates it, checks for PostgreSQL 18+ and UTF8, and installs the roles `letopis_owner`
39
- and `letopis_app`. Options: `fresh` (recreate the schema), `upgrade` (apply migrations), `seeds` (the name of a
40
- package seed, for example `'booking'` for the demo domain, or a path to your own `.sql`), `history` (retention
41
- policy), `allowReset`, `serviceKey`, `quiet`, `pglite: true` (a database for development without PostgreSQL; then
42
- `connect({ dsn: r.pglite.dsn, listen: false, max: 1, allowBypassRls: true, … })` — a superuser connection, RLS
43
- does not apply on it).
44
- - `signals` — what `NOTIFY` signals show to any role that can connect to the database. The channel is the full
45
- schema name (`LISTEN "v2.myapp"`); the write signal payload: `'class'` (default) — `Class`; `'tenant'` —
46
- `Class@tenant token` (the cache is cleared only in the tenant of the write); `'opaque'` — `class token@tenant
47
- token`; `'none'` — no signals (the cache works only with `ttlOnly`, subscriptions poll the log). Service signals:
48
- `session:<16 hex>` is a revoked session, `session:*` and `*` are resets (`reset`). The mode is stored in the
49
- schema: `up()` without the option keeps it; after a change, restart the processes. The signal key (precise cache
50
- clearing) goes to services — the `letopis.signals` permission. The `up()` result also has `signals` and `warnings`
51
- (PostgreSQL below 18.2 — upgrade).
52
- - After a package update, call `up()`: only it re-applies the schema functions. `connect()` warns if the engine stamp
53
- in the schema comment (package version and a hash of its SQL) does not match the package: an older schema — `up()`,
54
- a newer one — update the package.
55
- - `tenant` — the first tenant and user without a seed of your own: the tenant gets the rule "authenticated users may
56
- do everything" (`grantAll`, on by default, weight 10); the user gets a membership with the roles `user.roles`
57
- (`['owner']` by default), a password for signing in with the login `login`, and a session in the tenant. Repeating
58
- the call with the same name and login duplicates nothing and does not change the roles of an existing membership (the
59
- ids are v5 of `tenant:<name>` and `user:<login>`). Another tenant: `createTenant(postgres(ADMIN), 'v2.myapp', { name,
60
- user })` (needs an admin connection).
61
- - The application connects with an **application LOGIN role** that is a member of `letopis_app` only:
62
- `create role myapp login password '…'; grant letopis_app to myapp;`. Connecting as a superuser is the error
63
- `bypass_rls` (RLS would not apply); an admin connection is possible only with an explicit `allowBypassRls: true`.
64
- - Accounts live in the System tenant; the system administrator (a member of System with the `admin` role; your own
65
- installation seed or the loader on an admin connection appoints one) creates them with the chain `admin.Account().create({ name, categories })`. Your own seed (`seeds: ['./my.sql']`, the owner functions
66
- `sys_create`, `sys_grant_all`, the markers `@SCHEMA@`, `@SYSTEM@`) remains for special cases.
67
-
68
- ### 2. Sign-in: service, password, session
69
-
70
- <!-- run -->
71
- ```ts
72
- const APP = 'postgres://myapp:…@localhost:5432/app'
73
- const svc = await connect({ dsn: APP, schema: 'v2.myapp', apiKey: r.serviceKey! }) // service: key → session
74
- const SALON = r.tenant!.tenant, OLGA = r.tenant!.user!
75
- await svc.auth.setPassword({ account: SALON, identifier: 'salon@example.com', password: 'Lilac-2026!', confirmed: true })
76
- const login = await svc.auth.verifyPassword({ identifier: 'salon@example.com', password: 'Lilac-2026!' })
77
- // login: { account, token } | null (a wrong password or an unconfirmed credential gives null, not an error)
78
- const db = await connect({ dsn: APP, schema: 'v2.myapp', token: login!.token })
79
- db.tenant // the session tenant: by default the account's own tenant (here SALON)
80
- ```
81
- - `connect({ dsn | sql, schema: 'v2.<name>', token | apiKey, max, listen, cache, onQuery, slowMs, maintain,
82
- allowBypassRls, models })`. `schema` is the **full** name with `v2.`.
83
- - `db.auth`: `setPassword`, `verifyPassword`, `issueApiKey`, `verifyApiKey`, `issueKeySecret`, `verifyKeySecret`,
84
- `enrollTotp`, `verifyTotp`, `totpEnabled`, `issueOtp`, `verifyOtp`, `link`, `lookup`, `credentials`, `revokeCredential`,
85
- `setFlags`, `sessionFor(account, { ttl })`, `session(token)`, `refresh`, `revoke`, `revokeAll`, `switch(tenant)`,
86
- `tenants()`, `purgeAccount`. Password, OTP and TOTP checks, issuing sessions, session details by token and
87
- confirming credentials are called by the **service** (the `auth.*` permissions); a regular user gets `acl_denied`.
88
- Signatures: `issueApiKey({ account, name?, ttl? })` → `{ key, id }`; `verifyApiKey(key)` → `{ token, account } | null`;
89
- `sessionFor(account, { ttl })` → a token string; then `connect({ dsn, schema, token })`, and to work in another
90
- tenant through a membership, `db.auth.switch(tenant)` (without it, the session is in the account's own tenant). The
91
- token for the CLI (`--token`) is the same `token` from `verifyPassword`/`verifyApiKey`/`sessionFor`. An admin
92
- connection (`allowBypassRls`) prints the warning "RLS не действует" (RLS is not in effect); this is expected.
93
- - Second factor: `verifyPassword({ identifier, password, totp })` and `lookup({ kind, identifier, totp })` check the
94
- TOTP code after the first check (the `auth.totp` permission; the failure counter and lockout are shared with
95
- `verifyTotp`) → a session with the methods `[kind, 'TOTP']`, with which a user with TOTP changes their own
96
- credentials. A wrong or repeated code and a code for an account without TOTP give `null`; without `totp`, the code
97
- is not checked.
98
- - Service keys: with a session of its own key, the service issues and revokes its own keys itself (`issueApiKey`,
99
- `issueKeySecret` with `account` set to the service id; `revokeCredential(id)`). Revoking a key revokes only the
100
- sessions of that key, so rotation has no downtime: new key → processes onto it → revoke the old one. The last active
101
- key of a service cannot be revoked (`acl_denied`); to disable a service, set `enabled: false` on its account.
102
- - Session by token: `svc.auth.session(token)` → `{ account, tenant, member, expiresAt, loginAt, methods } | null`.
103
- The service (the `auth.session` permission) learns whose session it is, which tenant it works in and whether the
104
- membership there is intact, without connecting with the token and without extending the session. A wrong, revoked
105
- or expired token and a disabled account give `null`. This is how a web service checks the session from a cookie at
106
- the start of every request: reading under a revoked session gives no error, the answers are simply empty.
107
- - Working in another tenant through a membership: `await user.auth.tenants()` → `[{ tenant, roles }]`, then
108
- `await user.auth.switch(tenant)` (only from the connection root). Impersonation by a service:
109
- `const u = await svc.as(account, { tenant, owner })` — the database checks it; in the journal, `author` is the user
110
- and `agent` is the service. `db.as()` is **asynchronous**.
111
-
112
- ### 3. Models in Zod and `letopis sync`
113
-
114
- <!-- run -->
115
- ```ts
116
- // models.ts
117
- import { z } from 'zod'
118
- import { hub, link, one, many } from 'letopis/model'
119
- export const Shop = hub('Shop', { name: z.string(), city: z.string().optional() }, { key: ['name'] })
120
- export const Wallet = hub('Wallet', { number: z.string(), balance: z.number().int().min(0).default(0) },
121
- { key: ['number'], ends: { Shop } }) // a bare model in ends = one(Shop), a required end
122
- export const Payment = link('Payment', { opId: z.string(), amount: z.number().int() },
123
- { key: ['Wallet', 'opId'], ends: { Wallet } }) // key from an end and a field: idempotency by opId
124
- export const Note = hub('Note', { text: z.string() }, {}) // no key: the database assigns the id
125
- ```
126
- - Full form: `hub(name, { alias, extends, abstract, ends, fields: z.object(…) | { fields }, key, onDelete, history,
127
- ownerDefault: 'tenant' | 'actor', cache: { ttl }, meta, derivedBy, aggregate })`; `link` works the same way.
128
- `derivedBy` and `aggregate` are in sections 12 and 13 below. `extends` also works in the short form, but TypeScript
129
- infers the data types of a descendant only in the full form.
130
- - Ends: `one(A)` (required), `one(A, B)` (union), `one('Name')` (target by name, for a class that refers to itself),
131
- `.optional()`, `.onDelete('restrict' | 'cascade' | 'unset')` (default `restrict`; `unset` only for an optional
132
- end), `many(A, { min, max, onDelete })`.
133
- - A key is made of fields (required strings) and single required ends. The key is inherited from `extends`.
134
-
135
- Models into the database (classes are written to the session tenant; a tightening that existing rows block gives `tightening_conflict`):
136
- <!-- run -->
137
- ```ts
138
- const synced = await syncModels(db, [Shop, Wallet, Payment, Note], { apply: true })
139
- // synced: { applied, created, updated, unchanged, skipped, lost } (lost: refine and anything else JSON Schema cannot express)
140
- ```
141
- ```sh
142
- npx letopis sync ./models.js --dsn "$APP" --schema v2.myapp --token "$TOKEN" # dry run (.ts needs Node 22.18+)
143
- npx letopis sync ./models.js --dsn "$APP" --schema v2.myapp --token "$TOKEN" --apply # apply
144
- npx letopis types --dsn "$APP" --schema v2.myapp --token "$TOKEN" --out types.ts # class types from the database
145
- ```
146
- Chain types: `connect({ …, models: [Shop, Wallet, Payment, Note] })` — the models are used only for types.
147
-
148
- ### 4. Writing with chains
149
-
150
- A verb is part of a plan; the plan is executed by the **terminal** (`rows`, `first`, `ids`, `count` …). The whole plan is one transaction.
151
- <!-- run -->
152
- ```ts
153
- const [shop] = await db.Shop().create({ name: 'Center', city: 'Kazan' }).rows()
154
- const [w] = await db.Shop(shop).Wallet().create({ number: 'W-1' }).rows() // the Shop end comes from the path
155
- const [n] = await db.Note().create({ text: 'hello' }).rows()
156
- await db.Note(n.id).update({ text: 'bye' }).rows() // patch merge, new version
157
- const [up1] = await db.Shop().upsert({ name: 'Center', city: 'Ufa' }).rows() // full replacement; up1.$upsert:
158
- // 'created' | 'updated' | 'unchanged'
159
- // money: an increment in the database, without a read; Payment with the key [Wallet, opId]: a repeat gives exists
160
- await db.Wallet(w).Payment().create({ opId: 'op-1', amount: 500 }).Wallet().update({ balance: inc(500) }).rows()
161
- await db.Wallet(w).update({ balance: inc(-300) }).first() // balance 200; a result below minimum (0) is invalid_data
162
- const cur = (await db.Wallet(w).first())!
163
- await db.Wallet(w).update({ balance: 0 }, { rev: cur.rev }).first() // strict mode: someone else's edit gives conflict
164
- // end slots after create / update / upsert: .Role.set(x) .unset(), and for many also .add(x) .remove(x)
165
- const [west] = await db.Shop().create({ name: 'West' }).rows()
166
- await db.Wallet(w).update({}).Shop.set(west).rows()
167
- const [w2] = await db.Shop(shop).Wallet().create({ number: 'W-2' }).rows()
168
- const [w200] = await db.Wallet(w2).rekey({ number: 'W-200' }).rows() // new key → new id, references are moved
169
- await db.Note(n.id).anonymize(['text']).rows()
170
- const b = db.batch('init'); b.Shop().create({ name: 'A' }); b.Shop().create({ name: 'B' })
171
- const batchRows = await b.run() // Row[][] in one transaction; b.size(), b.discard()
172
- ```
173
- - `inc(n, { start })` — only in `update`; a missing field is an error (or `start` is used). The marker cannot come
174
- from JSON.
175
- - `{ rev }` — only when the step has one target (`rev_ambiguous`). If fewer rows are affected than the step found, the
176
- plan is rolled back with `target_not_found` (not visible), `conflict` (`rev` differs) or `acl_denied`.
177
-
178
- ### 5. Reading with a chain
179
-
180
- <!-- run -->
181
- ```ts
182
- await db.Shop({ city: 'Ufa' }).Wallet().rows() // Row[]: { id, class, rev, tenant, owner, links, data, tags, at, author, op, … }
183
- const rich = await db.Wallet({ balance: gt(100) }).count() // the number of ENTITIES; for paths, count({ paths: true })
184
- const paid = await db.Wallet(w).Payment().sum('data.amount') // sum | avg | min | max (null when empty), countBy
185
- await db.Shop(west).Wallet().Payment().paths() // [{ Shop: Row, Wallet: Row, Payment: Row }]
186
- await db.Wallet(w).first() // Row | null
187
- const page = await db.Wallet().sort('data.number', 'desc').limit(20).rows()
188
- const next = await db.Wallet().sort('data.number', 'desc').after(cursorOf(page.at(-1)!, 'data.number')).limit(20).rows() // keyset
189
- await db.Note().withDeleted().rows() // including deleted ones ($deleted: true)
190
- await db.Wallet(or({ number: 'W-1' }, { number: 'W-200' })).ids()
191
- ```
192
- - Step filter: an id, a list of ids, a result row, an object of fields nested to any depth, `or(…)`, the operators
193
- `ne gt gte lt lte between inList like ilike starts ends has hasAny hasAll exists isNull not`, end roles
194
- `{ Shop: id }`, `{ Shop: null }`. Fields are in `row.data`, not at the root of the row.
195
- - A step is a class name, an alias or an end role; a step by a parent class also returns descendants (`.exact()` —
196
- only its own class); `.deep(max)` — a tree (`$depth`); `.tags()`, `.owner()`, `.alias(name)`, `db.entity(row)`.
197
- - `after(cursorOf(row, field))` requires `sort()` by the same field; it pages `rows()`, `versions()` and `paths()` (the
198
- cursor of a path is `cursorOf(path, field)`, the keys of a path follow the order of the steps). `count()`,
199
- aggregates, `diff()`, `blame()` ignore the cursor.
200
-
201
- ### 6. Deletion with a preview
202
-
203
- <!-- run -->
204
- ```ts
205
- const preview = await db.Shop(shop).delete().rows() // deletes nothing: the whole cascade closure,
206
- // each row has $action: 'delete' | 'unset' | 'restrict' and $depth (0 is the target; unset and restrict do not have it). Wallet refers to Shop
207
- // with the default onDelete 'restrict': in the preview it has $action 'restrict', and delete({ confirm: true }) would give delete_restricted
208
- await db.Note(n.id).delete({ confirm: true }).rows() // tombstones (row.$deleted); the cascade follows the onDelete of the ends
209
- await db.Note(n.id).restore().rows() // bring back a deleted entity (same id); a live one gives not_deleted
210
- await db.Note(n.id).delete({ confirm: true }).rows()
211
- await db.Note(n.id).purge({ confirm: true }).rows() // erase the history of a deleted entity; without confirm, a preview
212
- ```
213
-
214
- ### 7. Loader
215
-
216
- Only an **admin** connection (`allowBypassRls` and the right to become the owner), otherwise `admin_required`:
217
- <!-- run -->
218
- ```ts
219
- const adm = await connect({ dsn: ADMIN, schema: 'v2.myapp', token: login!.token, allowBypassRls: true })
220
- const rep = await adm.load({
221
- Shop: [{ name: 'North' }],
222
- Wallet: [{ number: 'W-9', Shop: { $key: ['North'] } }], // reference: id | { $key: [...] } | { $ext }
223
- }, { existing: 'skip' }) // duplicates 'error'|'first'|'last', existing 'error'|'skip'|'update', commitEvery, withHistory, verify, tenant, as
224
- // rep: { rows, loaded, updated, skipped, duplicates, byClass, external, portions, ms }
225
- ```
226
- Initial load: `adm.load(rows, { initial: true })`, `letopis load --initial`, `letopis import --initial` — only into an
227
- empty schema (otherwise `not_empty`), in one transaction; about 2.2 times faster than a regular load on a million
228
- objects; no `commitEvery`, `existing` or `{ as }`.
229
-
230
- NDJSON files (one object per line; all files form one load):
231
- ```sh
232
- npx letopis load shops.ndjson wallets.ndjson --dsn "$ADMIN" --schema v2.myapp --token "$TOKEN" [--class Wallet] \
233
- [--duplicates first] [--existing skip] [--commit-every 100000] [--verify] [--initial]
234
- # {"$class":"Shop","name":"South"}
235
- # {"$class":"Wallet","number":"W-10","Shop":{"$key":["South"]}}
236
- # {"$class":"Note","text":"x","$ext":"n1"} ← $ext is the row's external key for references { "$ext": "n1" }
237
- ```
238
- An error carries the code of the first problem; `err.detail.rows` is the first 100 violations (line number, code,
239
- details), `err.detail.total` is how many there are in all, `err.detail.truncated` means the list is cut. A polymorphic
240
- reference needs `$class`. `$tags` are the row's tags, `$owner` is its owner.
241
-
242
- ### 8. Access rules: granting a permission
243
-
244
- A rule = a group (a `Resource` of category `ACCOUNT`) → an object (`API` — an endpoint, `READ`/`WRITE`/`DELETE` — rows
245
- of a class with a condition) with `allow`/`deny` and a weight. One rule decides: the highest weight; on a tie, deny.
246
- A tenant rule weight above 999 is cut down to 999. Rows of access rules in a tenant can be written by whoever has
247
- `WRITE` on them (here, the tenant itself, through the "may do everything" rule).
248
- <!-- run -->
249
- ```ts
250
- // Olga's membership in the salon was created by up({ tenant }) with the role owner; we make her an employee with the role staff
251
- await db.member(db.idOf('member', { User: OLGA, Tenant: SALON })).update({ roles: ['staff'] }).rows()
252
- const [staff] = await db.Resource().create({ alias: 'staff:ACCOUNT', category: 'ACCOUNT', pattern: { roles: '{staff}' } }).rows()
253
- const [api] = await db.Resource().create({ alias: 'reports:API', category: 'API', pattern: { endpoint: 'shop.reports.*' } }).rows()
254
- await db.rule().create({ permission: 'allow', weight: 50 }).Group.set(staff).Object.set(api).rows()
255
- const [del] = await db.Resource().create({ alias: 'wallet:DELETE', category: 'DELETE', pattern: { class: 'Wallet' } }).rows()
256
- await db.rule().create({ permission: 'deny', weight: 50 }).Group.set(staff).Object.set(del).rows() // stronger than "may do everything" (10)
257
- // check on behalf of Olga (the service acts on her behalf in the salon)
258
- const olga = await svc.as(OLGA, { tenant: SALON })
259
- const canReport = await olga.acl.check('shop.reports.view') // { allow: true, rule: { weight: 50, … }, code?, message? }
260
- const canDelete = await olga.acl.checkData('Wallet', 'DELETE') // { allow: false } | { allow: true, filter? }
261
- ```
262
- - A membership is a `member` row with the ends `User` and `Tenant` and the key `[User, Tenant]`: a new membership is
263
- `db.member().create({ roles }).User.set(account).Tenant.set(tenant)` if it does not exist yet (otherwise `exists`).
264
- - Group: `{ categories: '{Staff}' }` (account categories) or `{ roles: '{staff}' }` (membership roles). A mask
265
- (`categories`, `roles`, `endpoint`, `class`) is **a string only**: the database rejects an array (`['staff']`), a
266
- number or `null` with `invalid_data`.
267
- Object condition: `{ class: 'Wallet', owner: '$actor' }`, `{ class: 'Wallet', data: { … } }`; a class mask includes
268
- descendants, `'{rule,Resource,member}'` means several classes. `deny` with `{ code, message }` is a custom endpoint
269
- refusal.
270
- - The database recomputes access rights by itself; to remove a permission, use
271
- `db.rule(id).delete({ confirm: true }).rows()`.
272
-
273
- ### 9. Subscription and cache
274
-
275
- <!-- run -->
276
- ```ts
277
- const sub = db.watch({ from: 'start', classes: ['Wallet'] }) // from: 'now' (default) | 'start' | a cursor; pollMs; leaving classes — op 'delete'
278
- const lagMs = await sub.lag() // lag, ms
279
- await db.Wallet(w).update({ balance: inc(1) }).rows()
280
- let cursor = ''
281
- for await (const ev of sub) { // ev: { cursor, row, id, class, op, rev, at }
282
- cursor = ev.cursor // to resume: db.watch({ from: cursor })
283
- if (ev.id === w.id && ev.row.data.balance === 1) break // break or sub.close() stops it
284
- }
285
- const shops = await db.Shop().cache({ ttl: 300 }).rows() // result cache; reset by your own writes and by signals
286
- db.cache.stats(); db.cache.clear()
287
- ```
288
- Events arrive without losses or duplicates, under RLS; a cursor older than the `all` tier of the retention policy gives
289
- `cursor_expired`. With `from: 'now'` the subscription sees only what was committed after it started.
290
-
291
- ### 10. As-of snapshot and history
292
-
293
- <!-- run -->
294
- ```ts
295
- const was = await db.Wallet(w).asOf('2026-10-01T00:00:00Z').first() // as it was at that moment (an ISO string or a Date)
296
- await db.Shop(shop).Wallet().asOf(new Date('2026-10-01')).rows()
297
- const history = await db.Wallet(w).versions() // all versions; the deletion version has $deleted
298
- await db.Wallet(w200).versions({ follow: true }) // also across id changes (reclass, rekey)
299
- await db.Wallet(w).diff() // what changed: previous version → latest
300
- await db.Wallet(w).blame() // who, when and why (verb option reason) changed fields
301
- ```
302
- `diff()` is an array of `DiffResult`: `{ id, class, from, to, changes }`; `from` and `to` are `{ rev, at, id? }`, `to`
303
- also has `deleted?` (then `changes` is empty), `from: null` means the object did not exist then. A change, `DiffChange`,
304
- is `{ path, kind: 'added' | 'removed' | 'changed', before?, after? }`. `blame()` is an array of `BlameResult`:
305
- `{ id, class, rev, complete, fields }`; `fields` are the `BlameField` leaves in path order:
306
- `{ path, rev, at, author, op, exact, agent?, reason?, id? }`. `path` is an array of segments: `['data', 'price']`;
307
- empty `agent`, `reason`, `before`, `after` are omitted. The types come from `letopis`.
308
- A class with `history: false` does not provide `versions`, `asOf`, `restore` or subscription events.
309
-
310
- ### 11. Import from 0.21
311
-
312
- The 1.0 schema is installed in advance (`up()` without the demo seed); the connections are admin connections. The
313
- source is the 0.21 tables (`Schema`, `Entity`, `Account`, `Credential`, `Resource`, `Rule`) in a `v1.<name>` schema of
314
- any database. The repository has the fixture `lib/test/fixtures/v021.sql`, a dump of these tables with the marker
315
- `<SCHEMA>`: replace the marker and run the file.
316
- ```ts
317
- import postgres from 'postgres'; import { readFile } from 'node:fs/promises'
318
- const sql = postgres(ADMIN)
319
- await sql.unsafe((await readFile('lib/test/fixtures/v021.sql', 'utf8')).replaceAll('<SCHEMA>', 'v1.fixture'))
320
- await up({ dsn: ADMIN, schema: 'imported' }) // v2.imported
321
- ```
322
- ```sh
323
- npx letopis import --from "$ADMIN" --from-schema v1.fixture --dsn "$ADMIN" --schema v2.imported \
324
- --tenant Import --keyless Service
325
- ```
326
- - `--tenant <name>` — the tenant for the System data of 0.21; `--keyless <Class>` — a class whose data violates the
327
- key (without it, importing the fixture stops: two 0.21 objects give one 1.0 id); `--rename Old=New` — names taken by
328
- 1.0 methods; `--update` — append new versions; `--commit-every N` — commit in portions; `--partition <name>` — the
329
- 0.21 source partition (default `entity`).
330
- - The report is JSON at the end of the output (before it come the progress lines `letopis.import: …`): `classes`,
331
- `notes`, `source`/`target` (objects, versions and live ones by class in 0.21 and 1.0), `tenants`, `accounts`,
332
- `credentials`, `resources`, `rules`, `objects`, `uuidFields`, `droppedFields`, `ownerLost`, `verify`, `ms`; the exit
333
- code is 1 if `verify` failed. Repeating the import adds what is missing, without duplicates.
334
-
335
- ### 12. Projectors
336
-
337
- A projector builds rows of derived classes from the journal: summaries, cards, anything that may be computed with a
338
- lag. A derived class is marked with `derivedBy: 'projector-name'`, and only this projector writes its rows. The
339
- derived class itself (own or inherited) and its ancestors are created, edited and deleted only by an administrator:
340
- `syncModels` on an admin connection holding a tenant session (`connect({ dsn: ADMIN, schema, token, allowBypassRls: true })`) or an
341
- account with the System permission `letopis.projector`; for an application session it is `acl_denied`. Everyone whom
342
- the rules give `READ` on a derived class sees its totals, even without permissions on the source rows.
343
- ```ts
344
- const p = adm.projector('daily', { from: ['Payment'], into: ['DayTotal'], version: 1, withBefore: true,
345
- handle(changes, out) { /* out.inc, out.upsert, out.delete, await out.get: into classes, { tenant: c.tenant } */ } })
346
- await p.run({ once: true }) // catch up with the journal; run() follows it until stop(); rebuild(), status(), close()
347
- p.run().catch(report) // run() without once needs a catch: after a projector error it stops
348
- ```
349
- Only an admin connection (otherwise `admin_required`). A batch is one transaction, every change is applied exactly
350
- once, there is one writer (`rebuild()` while another process writes a batch gives `projector_busy`). The first
351
- start rebuilds the projection from the current state, in batches. `purge`, `trim` and `reset` come as changes with `rev` 1, so check `op` first.
352
- Another `version` gives `rebuild_required`; a cursor older than the `all` tier gives `cursor_expired`
353
- (`onExpired: 'rebuild'` rebuilds by itself, in batches); the tier is per tenant: if it has expired for some participants, only
354
- they drop out. A tenant with a source with `history: false` drops out (`invalid_class`). `c.before` (with
355
- `withBefore`) is the previous version of the same id or `null`: `null` for a creation, for the first version of a new
356
- id after `reclass` and `rekey` (the tombstone of the former id subtracts the former data), during a rebuild and when
357
- the previous version is a deletion (after `restore`) or lies outside the `from` family; `purge`, `trim` and `reset`
358
- have no such field (`undefined`), so subtract `before` only after checking that it is truthy: `if (c.before)`. A class change that keeps the id (a class
359
- without a key) and takes the row out of the `from` family comes as a deletion (`op` `delete`, `row` is the last
360
- version in the family with `$deleted`). `handle` gets the changes of one tenant. An error of its data (the
361
- database rejected its results, or `out` rejected its row or delta) rolls back only this tenant: it drops out
362
- (`status().failed`: `{ id: { cursor, code, message } }`) until `rebuild({ tenant: id })` (`{ tenant, chunked: true }` — in batches while the tail goes on for all, an interrupted one is continued by any `run()`;
363
- `status().rebuildingTenants`), and the others go on; with `onExpired: 'rebuild'` a tenant dropped by the tier rebuilds
364
- itself in batches, in any process;
365
- `tenants: [id, …]` is an explicit list of participants.
366
- `status().state`: `'new'` means there is no projection yet, `'running'` means the projection is
367
- built and kept up (not "a process is running now"), `'rebuilding'` means a batch rebuild is in progress or was
368
- interrupted.
369
- A daily total is a fold: subtract the previous version (`c.before`), add the new one (`c.row` unless it is `$deleted`)
370
- with `out.inc('DayTotal', { day }, { total: ±amount }, { tenant: c.tenant })`; `DayTotal` has the key `['day']`,
371
- `derivedBy: 'daily'` and `history: false`. Take the day from the data of the row, not from `at`: every edit changes the
372
- version time. `out` itself folds the increments of one row by key into one edit per batch (as long as there is no
373
- `upsert`, `delete` or `get` between them) and checks the class rule on their total. The full recipe is in
374
- [the README, "Recipe: daily revenue"](README.en.md#recipe-daily-revenue).
375
-
376
- ### 13. Aggregates in the database
377
-
378
- The database itself keeps a sum or a count of source rows — for totals that must match even for raw SQL:
379
- ```ts
380
- const Wallet = hub('Wallet', { number: z.string(), balance: z.number().int().min(0) }, {
381
- key: ['number'], aggregate: { balance: { sum: 'amount', of: 'Payment', via: 'Wallet' } } }) // or count: true; where is a JSON Schema
382
- await db.Wallet(w).Payment().create({ opId: 'op-9', amount: 500 }).rows() // the balance grew by itself, no inc
383
- ```
384
- A trigger recalculates the field after every statement (the library, raw SQL, a cascade, `restore`, `reclass`,
385
- `rekey`, the loader); going below `minimum` is `invalid_data`, every change is a version of the target. Live rows of
386
- the same tenant are counted (a System row is not). Repair: `adm.rebuildAggregates('Wallet')` on an admin connection;
387
- discrepancies: `verify({ data: true })`. Only someone who reads all rows of the source and its descendants without
388
- conditions can describe or change an aggregate, otherwise `acl_denied`. The source is described before the aggregate:
389
- an aggregate over a class that does not exist yet is refused (`invalid_class`), and the `of` class is not deleted while an
390
- aggregate names it; `syncModels` puts sources before targets itself. A hot counter (more than ~150 writes per
391
- second per object) is a projector that folds a batch, not an aggregate.
392
-
393
- ### 14. Agents and MCP
394
-
395
- MCP tools for an agent, under a user session (needs `npm i @modelcontextprotocol/server`):
396
- ```ts
397
- import { createMcpServer } from 'letopis/mcp'
398
- const server = await createMcpServer(await svc.as(userId)) // letopis_model, letopis_query, letopis_history; { write: true } adds letopis_write
399
- // stdio: letopis mcp --dsn … --schema v2.x --token <user token> [--write]; HTTP: createMcpHandler(() => createMcpServer(…)) from the SDK
400
- ```
401
- Only a user session or `svc.as(user)`: a connection that bypasses RLS gets `bypass_rls`, the service root without
402
- `as()` and a session in the System tenant (without `allowSystemAdmin`) get `user_session_required`; `letopis_write`
403
- does not write the permission classes without `allowAclWrite` (`acl_denied`); the command line does not accept
404
- `--api-key` but also takes `--dsn`,
405
- `--schema` and `--token` from `LETOPIS_DSN`, `LETOPIS_SCHEMA`, `LETOPIS_TOKEN`. Reading is a JSON chain
406
- (`class`, `filter` with `$gt`, `$in`, `$or` and so on, `steps`, `sort: { field, dir: 'asc' | 'desc' }`, `cursor`),
407
- with no SQL; an answer holds up to `maxRows` rows (100) and `maxBytes` bytes (50,000) from the `createMcpServer`
408
- options, then comes a `cursor`. `structuredContent` is `{ notice, data }` (`notice` is a constant note that this is
409
- data, not instructions); `data` by terminal: `rows` — `{ rows, count, more, cursor? }`, `ids` —
410
- `{ ids, count, more, cursor? }` (`count` is the rows on the page), `first` — `{ row }`, `count` — `{ count }`,
411
- `countBy` — `{ counts, truncated? }`, `sum`, `avg`, `min`, `max` —
412
- `{ value }`. A refusal is `isError: true` with a `structuredContent` of the form `{ notice, error }` (`error` is
413
- `{ code, message, detail?, sqlstate?, similar? }`). Writing only
414
- with `write: true` (without it `letopis_write` is not in the list, and a call is an SDK protocol error): `update` with
415
- `rev`; `delete` without `confirm` is a preview, with `confirm` a human confirms it (elicitation), and without that it
416
- is `confirmation_required`. Data from the database in an answer sits inside `<untrusted-data-…>`.
417
-
418
- ### Operations
419
-
420
- `db.verify({ data: true })` and `letopis verify` — hash chains, the anchor; `db.maintain()` and
421
- `connect({ maintain: '1h' })` — sessions, codes, journal thinning; `db.trimHistory(target, moment)`;
422
- `db.reset({ level, confirm, tenant })` — with `up({ allowReset: true })`; `db.describe()` /
423
- `letopis describe --format text`. Close each connection once; connections from `svc.as()` share the pool with `svc`:
424
- <!-- run -->
425
- ```ts
426
- await adm.close(); await db.close(); await svc.close()
427
- ```
428
-
429
- ## Pitfalls in 1.0
430
-
431
- 1. **`create` with a taken key is the error `exists`** (`err.detail.id`), not an overwrite. To replace the whole
432
- object, use `upsert`, and only for a class with a key (otherwise `no_key`). For a partial edit, use `update`.
433
- 2. **`update` never creates.** If the step found nothing, the result is empty with no error, so check its length;
434
- without a filter and without `data.id`, all rows of the step are edited. `target_not_found` comes with `{ rev }` or
435
- if the row was deleted between the lookup and the edit.
436
- 3. **Verbs return a chain.** Without a terminal (`.rows()`, `.first()` …) nothing runs. `delete()` without
437
- `{ confirm: true }` is only a preview; `purge` works only on deleted entities (`not_deleted`) and once a projector
438
- of the class has read the deletion (`projector_pending`); `trimHistory`, once the projector has read the versions.
439
- 4. **Money and counters**: `update({ balance: inc(n) })` — the database adds inside the statement, without a read and
440
- without lost updates; the `minimum` check in the schema catches a drop below zero. For a total computed from what
441
- you read, use `{ rev }` (re-read and retry on `conflict`) or `forUpdate()` in `db.begin()`. Idempotency comes from
442
- an operation key (`[Wallet, opId]`): a repeat gives `exists`.
443
- 5. **`db.begin()` is `read committed`.** Two reads in one transaction can see different data. To lock what you read:
444
- `await tr.Wallet(w).forUpdate().first()` (only in `db.begin()`, otherwise `tx_required`; only with `rows`, `first`,
445
- `ids`, otherwise `lock_unsupported`). `tr.lock(...keys)` takes an advisory lock until the end of the transaction.
446
- 6. **An error breaks an explicit transaction.** After any error (including `exists` and `conflict`), every call and
447
- `commit()` throw the original error, and `commit()` rolls back in that case; `rollback()` succeeds. Pattern:
448
- `const tr = await db.begin(); try { …; await tr.commit() } catch (e) { await tr.rollback(); throw e }`.
449
- `db.begin()` has no retries: on `40P01`, repeat the whole block. Operations from the pool retry `40001`/`40P01`
450
- by themselves.
451
- 7. **`commit_unknown`** — the connection dropped during `COMMIT`: the transaction may have been committed. Do not
452
- retry blindly; repeat the same operation with the same key (`exists` means "already written").
453
- 8. **The loader, a projector and `rebuildAggregates` need an admin connection** (`allowBypassRls: true`, the owner
454
- role), otherwise `admin_required`; inside `db.begin()` they give `invalid_query`. A regular application uses the
455
- application role, without bypassing RLS.
456
- 9. **Data is under RLS.** Without a session (`token`/`apiKey`) a write gives `no_session`; a row of another tenant or
457
- an invisible row cannot be told apart from a missing one: the step does not find it, and a reference to it gives
458
- `target_not_found`; without a permission you get an empty result or `acl_denied`. `` db.sql`…` `` and raw SQL also run under the session and RLS.
459
- 10. **The key is immutable.** Editing a key field, the id, the tenant or the owner gives `immutable_key`; a new key is
460
- `rekey(patch)` (or `db.rekeyClass(class, key)`), and that is a new id. Changing the class is
461
- `reclass(class, data)`, not `update` (`use_reclass`). Deletion, `reclass` and `rekey` run only in
462
- `read committed` (`isolation_level`).
463
- 11. **Classes live in the session tenant.** `letopis sync` with a tenant's token writes the classes to that tenant;
464
- other tenants do not see them. Classes shared by all tenants live in System (the installation seed or an import).
465
- 12. **Do not pass the id of a class with a key**: it is computed (`db.idOf('Shop', { name: 'Center' })`); a different
466
- id gives `id_mismatch`, and an id for a class without a key gives `id_from_db`.
467
- 13. **Method names are reserved**: a class named `count`, `sort`, `set`, `run`, `account`, `sql`, `auth` … cannot be
468
- reached as a step (the list is `RESERVED_CLASS_NAMES`). The removed 0.21 API (`run()`, `account()`, `watch(cb)`,
469
- `reloadSchema()`, the facades `db.accounts` …) throws `removed` with the replacement — see
470
- [MIGRATION.md](MIGRATION.md) (in Russian).
471
- 14. **Access rules deny by default.** A new tenant without rules sees nothing; `up({ tenant })` and `createTenant`
472
- (`grantAll`) are the quick start. The `auth.*` permissions (passwords, sessions, impersonation) belong to the
473
- service and the system administrator.
474
- 15. **Errors:** `err.code` is a lowercase code (`exists`, `conflict`, `invalid_data` …), the text is
475
- `letopis: <code>: …`; data errors (`ValidationError`) have `err.issues: [{ path, keyword, message }]`.
476
- 16. **A subscription with `'now'` starts when the database answers.** `db.watch()` returns immediately, and the
477
- subscription takes the "now" moment with its first query to the database. A write made right after `db.watch()`
478
- can be committed earlier and miss the stream. If you need a guarantee — `await sub.ready()` before the write, a
479
- saved cursor or `'start'`. Parallel `next()` calls go in turn and get different events.
480
- 17. **The terminal returns the rows of the last step.** `…Payment().create(…).Wallet().update(…).rows()` returns the
481
- `Wallet` rows, not the created payment; if you need the payment, end the chain on it.
482
- 18. **A batch queue belongs to its facade.** `db.batch('x')` on the root, on each `await db.as(…)` and on each
483
- `db.begin()` are different queues; a plan runs on behalf of whoever queued it. Collect plans through one facade
484
- object.
485
- 19. **Only its projector writes the rows of a derived class.** The application, raw SQL and the loader get
486
- `derived_class`; ends may not refer to such rows either. To recompute them, use the projector's `rebuild()`.
487
- 20. **Only the database writes an aggregate field.** A value in `create`, `update`, `upsert`, `rekey`, `reclass` and raw
488
- SQL is `aggregate_field`; do not pass it. A raw upsert of a target: `set data = excluded.data - array['balance']`.