toiljs 0.0.115 → 0.0.117

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (326) hide show
  1. package/CHANGELOG.md +10 -0
  2. package/README.md +16 -7
  3. package/TYPESCRIPT_LAW.md +1946 -2073
  4. package/build/backend/.tsbuildinfo +1 -1
  5. package/build/cli/.tsbuildinfo +1 -1
  6. package/build/cli/index.js +2375 -258
  7. package/build/client/.tsbuildinfo +1 -1
  8. package/build/client/dev/devtools.js +1 -1
  9. package/build/client/dev/error-overlay.js +1 -1
  10. package/build/client/routing/mount.js +1 -1
  11. package/build/compiler/.tsbuildinfo +1 -1
  12. package/build/compiler/auth-emails.js +7 -3
  13. package/build/compiler/emails.js +1 -1
  14. package/build/compiler/index.d.ts +1 -0
  15. package/build/compiler/index.js +25 -4
  16. package/build/compiler/pages.js +2 -3
  17. package/build/compiler/prerender.d.ts +3 -8
  18. package/build/compiler/prerender.js +88 -127
  19. package/build/compiler/ssg.js +3 -4
  20. package/build/compiler/template-build.js +3 -4
  21. package/build/compiler/toil-docs.generated.js +6 -5
  22. package/build/compiler/vite.js +3 -6
  23. package/build/devserver/.tsbuildinfo +1 -1
  24. package/build/devserver/db/database.js +7 -3
  25. package/build/devserver/db/types.js +1 -1
  26. package/build/devserver/production.js +4 -3
  27. package/build/devserver/runtime/crypto.d.ts +1 -1
  28. package/build/io/.tsbuildinfo +1 -1
  29. package/build/logger/.tsbuildinfo +1 -1
  30. package/build/shared/.tsbuildinfo +1 -1
  31. package/examples/basic/client/routes/analytics.tsx +7 -5
  32. package/examples/basic/server/tsconfig.json +1 -3
  33. package/package.json +45 -44
  34. package/presets/no-uint8array-tostring.js +120 -153
  35. package/presets/oxlint.d.ts +5 -0
  36. package/presets/oxlint.js +224 -0
  37. package/server/runtime/tsconfig.json +1 -3
  38. package/tsconfig.json +2 -11
  39. package/.github/ISSUE_TEMPLATE/bug_report.md +0 -38
  40. package/.github/ISSUE_TEMPLATE/bug_report.yml +0 -90
  41. package/.github/ISSUE_TEMPLATE/config.yml +0 -8
  42. package/.github/ISSUE_TEMPLATE/feature_request.md +0 -20
  43. package/.github/PULL_REQUEST_TEMPLATE.md +0 -43
  44. package/.github/changelog-config.json +0 -45
  45. package/.github/dependabot.yml +0 -27
  46. package/.github/workflows/ci.yml +0 -222
  47. package/.vscode/settings.json +0 -10
  48. package/docs/README.md +0 -107
  49. package/docs/auth/README.md +0 -104
  50. package/docs/auth/configuration.md +0 -105
  51. package/docs/auth/emails.md +0 -92
  52. package/docs/auth/extending.md +0 -202
  53. package/docs/auth/how-it-works.md +0 -138
  54. package/docs/auth/usage.md +0 -188
  55. package/docs/backend/README.md +0 -143
  56. package/docs/backend/data.md +0 -351
  57. package/docs/backend/rest.md +0 -402
  58. package/docs/backend/rpc.md +0 -226
  59. package/docs/background/README.md +0 -114
  60. package/docs/background/daemons.md +0 -276
  61. package/docs/background/derive.md +0 -179
  62. package/docs/cli/README.md +0 -379
  63. package/docs/concepts/ai-guide.md +0 -132
  64. package/docs/concepts/config.md +0 -416
  65. package/docs/concepts/decorators.md +0 -127
  66. package/docs/concepts/security.md +0 -108
  67. package/docs/concepts/tiers.md +0 -166
  68. package/docs/concepts/types.md +0 -216
  69. package/docs/database/README.md +0 -143
  70. package/docs/database/capacity.md +0 -350
  71. package/docs/database/counters.md +0 -174
  72. package/docs/database/documents.md +0 -277
  73. package/docs/database/events.md +0 -307
  74. package/docs/database/membership.md +0 -246
  75. package/docs/database/setup.md +0 -155
  76. package/docs/database/unique.md +0 -216
  77. package/docs/database/views.md +0 -246
  78. package/docs/frontend/README.md +0 -104
  79. package/docs/frontend/components.md +0 -286
  80. package/docs/frontend/data-fetching.md +0 -260
  81. package/docs/frontend/images.md +0 -148
  82. package/docs/frontend/metadata.md +0 -344
  83. package/docs/frontend/navigation.md +0 -296
  84. package/docs/frontend/rendering.md +0 -236
  85. package/docs/frontend/routing.md +0 -346
  86. package/docs/frontend/scripts.md +0 -118
  87. package/docs/frontend/search.md +0 -191
  88. package/docs/frontend/styling.md +0 -147
  89. package/docs/frontend/toil-global.md +0 -216
  90. package/docs/getting-started/README.md +0 -81
  91. package/docs/getting-started/create-project.md +0 -131
  92. package/docs/getting-started/deploy.md +0 -101
  93. package/docs/getting-started/first-app.md +0 -215
  94. package/docs/getting-started/installation.md +0 -124
  95. package/docs/getting-started/migrating.md +0 -125
  96. package/docs/getting-started/project-structure.md +0 -163
  97. package/docs/introduction/README.md +0 -66
  98. package/docs/introduction/design-principles.md +0 -81
  99. package/docs/introduction/distributed.md +0 -91
  100. package/docs/introduction/how-it-works.md +0 -121
  101. package/docs/introduction/hyperscale.md +0 -70
  102. package/docs/introduction/modern-stack.md +0 -85
  103. package/docs/introduction/vs-other-frameworks.md +0 -64
  104. package/docs/introduction/why-toil.md +0 -99
  105. package/docs/llms.txt +0 -95
  106. package/docs/realtime/README.md +0 -102
  107. package/docs/realtime/channels.md +0 -211
  108. package/docs/realtime/streams.md +0 -369
  109. package/docs/services/README.md +0 -60
  110. package/docs/services/analytics.md +0 -279
  111. package/docs/services/caching.md +0 -175
  112. package/docs/services/cookies.md +0 -235
  113. package/docs/services/crypto.md +0 -209
  114. package/docs/services/email.md +0 -289
  115. package/docs/services/environment.md +0 -141
  116. package/docs/services/ratelimit.md +0 -174
  117. package/docs/services/time.md +0 -85
  118. package/eslint.config.js +0 -57
  119. package/examples/basic/client/routes/docs/[...slug].tsx +0 -12
  120. package/presets/eslint.js +0 -88
  121. package/src/backend/index.ts +0 -204
  122. package/src/cli/configure.ts +0 -363
  123. package/src/cli/create.ts +0 -924
  124. package/src/cli/db.ts +0 -158
  125. package/src/cli/diagnostics.ts +0 -665
  126. package/src/cli/doctor.ts +0 -982
  127. package/src/cli/features.ts +0 -170
  128. package/src/cli/index.ts +0 -326
  129. package/src/cli/notify.ts +0 -120
  130. package/src/cli/proc.ts +0 -56
  131. package/src/cli/ui.ts +0 -169
  132. package/src/cli/update.ts +0 -248
  133. package/src/cli/updates.ts +0 -92
  134. package/src/cli/validate.ts +0 -31
  135. package/src/cli/version-check.ts +0 -125
  136. package/src/client/auth.ts +0 -869
  137. package/src/client/channel/channel.ts +0 -146
  138. package/src/client/components/Form.tsx +0 -65
  139. package/src/client/components/Image.tsx +0 -160
  140. package/src/client/components/Script.tsx +0 -113
  141. package/src/client/components/Slot.tsx +0 -21
  142. package/src/client/dev/devtools.tsx +0 -1089
  143. package/src/client/dev/error-overlay.tsx +0 -225
  144. package/src/client/errors.ts +0 -11
  145. package/src/client/head/head.ts +0 -182
  146. package/src/client/head/metadata.ts +0 -110
  147. package/src/client/index.ts +0 -130
  148. package/src/client/navigation/Link.tsx +0 -108
  149. package/src/client/navigation/NavLink.tsx +0 -86
  150. package/src/client/navigation/navigation.ts +0 -235
  151. package/src/client/navigation/prefetch.ts +0 -169
  152. package/src/client/navigation/scroll.ts +0 -53
  153. package/src/client/routing/Router.tsx +0 -216
  154. package/src/client/routing/action.ts +0 -122
  155. package/src/client/routing/error-boundary.tsx +0 -43
  156. package/src/client/routing/hooks.ts +0 -140
  157. package/src/client/routing/lazy.ts +0 -77
  158. package/src/client/routing/loader.ts +0 -381
  159. package/src/client/routing/match.ts +0 -47
  160. package/src/client/routing/mount.tsx +0 -90
  161. package/src/client/routing/params-context.ts +0 -10
  162. package/src/client/routing/slot-context.ts +0 -7
  163. package/src/client/rpc.ts +0 -97
  164. package/src/client/search/search.ts +0 -189
  165. package/src/client/search/use-page-search.ts +0 -73
  166. package/src/client/ssr/markers.tsx +0 -172
  167. package/src/client/stream/client.ts +0 -237
  168. package/src/client/types.ts +0 -73
  169. package/src/compiler/auth-emails.ts +0 -312
  170. package/src/compiler/config.ts +0 -369
  171. package/src/compiler/docs.ts +0 -127
  172. package/src/compiler/email-preview.ts +0 -314
  173. package/src/compiler/emails.ts +0 -404
  174. package/src/compiler/fonts.ts +0 -89
  175. package/src/compiler/generate.ts +0 -471
  176. package/src/compiler/image-blur.ts +0 -42
  177. package/src/compiler/image-report.ts +0 -88
  178. package/src/compiler/index.ts +0 -1362
  179. package/src/compiler/pages.ts +0 -49
  180. package/src/compiler/plugin.ts +0 -391
  181. package/src/compiler/prerender.ts +0 -263
  182. package/src/compiler/routes.ts +0 -170
  183. package/src/compiler/seo.ts +0 -415
  184. package/src/compiler/sri.ts +0 -144
  185. package/src/compiler/ssg.ts +0 -177
  186. package/src/compiler/ssr-codegen.ts +0 -85
  187. package/src/compiler/template-build.ts +0 -676
  188. package/src/compiler/template.ts +0 -262
  189. package/src/compiler/toil-docs.generated.ts +0 -77
  190. package/src/compiler/vite.ts +0 -245
  191. package/src/devserver/analytics/index.ts +0 -317
  192. package/src/devserver/config/dotenv.ts +0 -102
  193. package/src/devserver/config/env.ts +0 -30
  194. package/src/devserver/config/ratelimit.ts +0 -123
  195. package/src/devserver/daemon/catalog.ts +0 -120
  196. package/src/devserver/daemon/cron.ts +0 -87
  197. package/src/devserver/daemon/host.ts +0 -134
  198. package/src/devserver/daemon/index.ts +0 -369
  199. package/src/devserver/daemon/runtime.ts +0 -48
  200. package/src/devserver/db/catalog.ts +0 -109
  201. package/src/devserver/db/database.ts +0 -1834
  202. package/src/devserver/db/derives.ts +0 -121
  203. package/src/devserver/db/index.ts +0 -20
  204. package/src/devserver/db/routeKinds.ts +0 -191
  205. package/src/devserver/db/types.ts +0 -168
  206. package/src/devserver/email/caps.ts +0 -0
  207. package/src/devserver/email/config.ts +0 -141
  208. package/src/devserver/email/console.ts +0 -348
  209. package/src/devserver/email/index.ts +0 -155
  210. package/src/devserver/email/providers.ts +0 -131
  211. package/src/devserver/email/status.ts +0 -23
  212. package/src/devserver/email/validate.ts +0 -37
  213. package/src/devserver/email/wire.ts +0 -55
  214. package/src/devserver/http/cache.ts +0 -0
  215. package/src/devserver/http/envelope.ts +0 -157
  216. package/src/devserver/http/proxy.ts +0 -172
  217. package/src/devserver/http/runtime.ts +0 -387
  218. package/src/devserver/index.ts +0 -43
  219. package/src/devserver/production-ipc.ts +0 -63
  220. package/src/devserver/production-worker.ts +0 -83
  221. package/src/devserver/production.ts +0 -718
  222. package/src/devserver/runtime/crypto.ts +0 -587
  223. package/src/devserver/runtime/host.ts +0 -427
  224. package/src/devserver/runtime/module.ts +0 -442
  225. package/src/devserver/server.ts +0 -251
  226. package/src/devserver/ssr.ts +0 -208
  227. package/src/devserver/stream/catalog.ts +0 -106
  228. package/src/devserver/stream/host.ts +0 -42
  229. package/src/devserver/stream/index.ts +0 -301
  230. package/src/devserver/stream/manager.ts +0 -167
  231. package/src/devserver/stream/router.ts +0 -101
  232. package/src/devserver/stream/wire.ts +0 -58
  233. package/src/devserver/stream/ws.ts +0 -78
  234. package/src/devserver/wasm/sections.ts +0 -59
  235. package/src/devserver/wasm/surface.ts +0 -115
  236. package/src/io/FastMap.ts +0 -151
  237. package/src/io/FastSet.ts +0 -113
  238. package/src/io/codec.ts +0 -362
  239. package/src/io/index.ts +0 -10
  240. package/src/io/lengths.ts +0 -14
  241. package/src/io/types.ts +0 -19
  242. package/src/logger/index.ts +0 -22
  243. package/src/shared/index.ts +0 -46
  244. package/test/analytics-dev.test.ts +0 -113
  245. package/test/assembly/aspect-shim.ts +0 -33
  246. package/test/assembly/cookie.spec.ts +0 -303
  247. package/test/assembly/example.spec.ts +0 -22
  248. package/test/assembly/ssr.spec.ts +0 -95
  249. package/test/assembly.test.ts +0 -59
  250. package/test/auth-emails.test.ts +0 -118
  251. package/test/built-ssr.test.ts +0 -148
  252. package/test/channel.test.ts +0 -21
  253. package/test/config-host-port.test.ts +0 -59
  254. package/test/configure.test.ts +0 -94
  255. package/test/daemon-build.test.ts +0 -278
  256. package/test/daemon-catalog.test.ts +0 -349
  257. package/test/daemon-emulation.test.ts +0 -256
  258. package/test/db.test.ts +0 -0
  259. package/test/devserver-database.test.ts +0 -1102
  260. package/test/devserver-email.test.ts +0 -306
  261. package/test/devserver-pqauth.test.ts +0 -93
  262. package/test/devserver-secrets.test.ts +0 -63
  263. package/test/devserver.test.ts +0 -259
  264. package/test/doctor.test.ts +0 -298
  265. package/test/dom/Image.test.tsx +0 -144
  266. package/test/dom/Link.test.tsx +0 -67
  267. package/test/dom/NavLink.test.tsx +0 -37
  268. package/test/dom/Script.test.tsx +0 -48
  269. package/test/dom/action.test.tsx +0 -146
  270. package/test/dom/error-overlay.test.tsx +0 -44
  271. package/test/dom/loader.test.tsx +0 -121
  272. package/test/dom/navigation.test.ts +0 -59
  273. package/test/dom/revalidate.test.tsx +0 -38
  274. package/test/dom/route-head.test.tsx +0 -56
  275. package/test/dom/router-loading.test.tsx +0 -44
  276. package/test/dom/scroll.test.ts +0 -56
  277. package/test/dom/slot.test.tsx +0 -131
  278. package/test/dom/use-metadata.test.tsx +0 -58
  279. package/test/dom/view-transitions.test.tsx +0 -53
  280. package/test/dynamic-routing.test.ts +0 -132
  281. package/test/email-console.test.ts +0 -86
  282. package/test/email-preview.test.ts +0 -73
  283. package/test/emails.test.ts +0 -58
  284. package/test/errors.test.ts +0 -21
  285. package/test/example-guestbook.test.ts +0 -123
  286. package/test/features.test.ts +0 -149
  287. package/test/fixtures/bignum-wire/spec.ts +0 -25
  288. package/test/fixtures/daemon-app.ts +0 -65
  289. package/test/fixtures/stream-echo.ts +0 -26
  290. package/test/fixtures/stream-gate.ts +0 -28
  291. package/test/fixtures/stream-trap.ts +0 -18
  292. package/test/fixtures/stream-typed.ts +0 -41
  293. package/test/fonts.test.ts +0 -28
  294. package/test/global-setup.ts +0 -17
  295. package/test/head.test.ts +0 -42
  296. package/test/image-blur.test.ts +0 -50
  297. package/test/io.test.ts +0 -117
  298. package/test/metadata.test.ts +0 -40
  299. package/test/navlink.test.ts +0 -28
  300. package/test/pages.test.ts +0 -105
  301. package/test/placeholder.test.ts +0 -9
  302. package/test/pqauth-e2e.test.ts +0 -216
  303. package/test/pqauth-email.test.ts +0 -801
  304. package/test/prerender.test.ts +0 -116
  305. package/test/prettier-plugin.test.ts +0 -46
  306. package/test/routes.test.ts +0 -76
  307. package/test/rpc-bignum-wire.test.ts +0 -164
  308. package/test/rpc-dispatch.test.ts +0 -132
  309. package/test/rpc-kinds.test.ts +0 -18
  310. package/test/rpc.test.ts +0 -66
  311. package/test/search.test.ts +0 -114
  312. package/test/seo.test.ts +0 -175
  313. package/test/slot-layouts.test.ts +0 -69
  314. package/test/sri.test.ts +0 -129
  315. package/test/ssg.test.ts +0 -36
  316. package/test/ssr-hydration.test.tsx +0 -122
  317. package/test/ssr-render.test.ts +0 -197
  318. package/test/ssr-template.test.tsx +0 -482
  319. package/test/stream-emulation.test.ts +0 -479
  320. package/test/ui.test.ts +0 -40
  321. package/test/update.test.ts +0 -58
  322. package/test/validate.test.ts +0 -42
  323. package/test/version-check.test.ts +0 -130
  324. package/tests/data-parity/generated-parity.ts +0 -99
  325. package/tests/data-parity/parity.ts +0 -80
  326. package/tests/data-parity/spec.ts +0 -46
@@ -1,277 +0,0 @@
1
- # Documents
2
-
3
- The **Documents** family is ToilDB's general-purpose record store: you keep a value under a key and look it up, update it, or delete it by that key. It is the family you will use most, and the one to reach for whenever the other six do not obviously fit.
4
-
5
- ## What and why
6
-
7
- A **Documents collection** maps a `@data` key to a `@data` value: one value per key, which you can read, replace, and remove. Think users by user id, posts by post id, orders by order number. If you are storing "a thing with fields that I look up and update by its id," this is the family.
8
-
9
- Declare one by typing a `@collection` as `Documents<Key, Value>`:
10
-
11
- ```ts
12
- @data
13
- class UserId {
14
- id: string = '';
15
- constructor(id: string = '') { this.id = id; }
16
- }
17
-
18
- @data
19
- class User {
20
- id: string = '';
21
- name: string = '';
22
- email: string = '';
23
- score: u64 = 0;
24
- }
25
-
26
- @database
27
- class AppDb {
28
- @collection static users: Documents<UserId, User>;
29
- }
30
- ```
31
-
32
- ## The operations
33
-
34
- Here is every operation, its shape, and what it gives back. `K` is your key type, `V` your value type.
35
-
36
- | Operation | Signature | Returns | Use it to |
37
- | --- | --- | --- | --- |
38
- | `get` | `get(key: K): V \| null` | the value, or `null` if absent | read one record |
39
- | `require` | `require(key: K): V` | the value; **traps** if absent | read a record you are sure exists |
40
- | `getMany` | `getMany(keys: K[]): Array<V \| null>` | one entry per key, in order, each value or `null` | read several records in one call |
41
- | `exists` | `exists(key: K): bool` | `true` if the record is present | check presence without reading the value |
42
- | `create` | `create(key: K, value: V): bool` | `true` if inserted, `false` if the key was already taken | add a **new** record without overwriting |
43
- | `patch` | `patch(key: K, value: V): V` | the newly stored value; **traps** if the record is absent | replace an **existing** record's value |
44
- | `upsert` | `upsert(key: K, value: V): UpsertResult` | `UpsertResult.Created` or `.Updated`; `.Conflict` on a `@unique` collision (nothing written) | create **or** overwrite in one op (last-writer-wins) |
45
- | `enqueue` | `enqueue(key: K, value: V): bool` | `true` if applied, `false` if a concurrent write won first or the record is absent | a version-checked (compare-and-swap) overwrite of an existing record |
46
- | `delete` | `delete(key: K): void` | nothing (idempotent) | remove a record |
47
- | `getDelete` | `getDelete(key: K): V \| null` | the value that was there, or `null`; removes it atomically | consume a record exactly once |
48
-
49
- Which kind of function may call which operation is covered in [Setup](./setup.md#how-access-is-gated-query-action-and-friends). In short: reads (`get`, `getMany`, `exists`) work anywhere; writes (`create`, `patch`, `upsert`, `enqueue`, `delete`, `getDelete`) need an **Action** (a `@post` route or an `@action`).
50
-
51
- ### Reading: `get`, `require`, `exists`, `getMany`
52
-
53
- `get` is the everyday read. It returns the value or `null`, so you handle "not found" explicitly:
54
-
55
- ```ts
56
- const user = AppDb.users.get(new UserId('u_123'));
57
- if (user == null) {
58
- return Response.notFound();
59
- }
60
- // user is a fully typed User here
61
- ```
62
-
63
- `require` is `get` for the case where absence is a bug, not a normal outcome: it returns the value directly and traps (aborts the request) if the record is missing. Use it only when you have already guaranteed the record exists.
64
-
65
- `exists` answers "is there a record here?" without paying to decode the value. It is handy as a cheap precondition:
66
-
67
- ```ts
68
- if (AppDb.users.exists(new UserId(name))) {
69
- // username already registered, do not overwrite
70
- }
71
- ```
72
-
73
- `getMany` reads several keys in a single operation. You hand it an array of keys; you get back an array the same length and in the same order, each entry either the value or `null` for a key that was absent. Reach for it instead of a loop of `get` calls when you already know the handful of keys you need.
74
-
75
- ```ts
76
- const ids = [new UserId('a'), new UserId('b'), new UserId('c')];
77
- const found: Array<User | null> = AppDb.users.getMany(ids);
78
- // found[0] lines up with ids[0], and so on; each is a User or null
79
- ```
80
-
81
- `getMany` is a **bounded batch of point reads**, not a scan: the number of keys you may pass is capped by the request budget, and it never walks the whole collection. There is no "get all records" operation on a request path, by design (an unbounded scan could fan out across a huge collection). If you need "the latest N of something," model it as [Events](./events.md) or precompute a [View](./views.md).
82
-
83
- ### Writing: `create` vs `upsert` vs `patch` vs `enqueue`
84
-
85
- These four all put a value under a key, but they differ in one important way each. Choosing correctly is the heart of using this family.
86
-
87
- ```mermaid
88
- flowchart TD
89
- START["I want to write a record"] --> Q1{"Must this write fail if another<br/>request changed the record<br/>since I read it?"}
90
- Q1 -->|"Yes, guard against a lost update"| ENQUEUE["enqueue<br/>(version-checked CAS;<br/>false = retry)"]
91
- Q1 -->|"No, last write wins"| Q2{"Does the record<br/>already exist?"}
92
- Q2 -->|"Must be new (never clobber)"| CREATE["create<br/>(false if the key is taken)"]
93
- Q2 -->|"May or may not exist"| UPSERT["upsert<br/>(create-or-overwrite in one op)"]
94
- Q2 -->|"Definitely exists"| PATCH["patch<br/>(overwrite; returns the value;<br/>traps if absent)"]
95
- ```
96
-
97
- **`create` inserts a new record.** It only writes if the key is free. If the key already has a record, `create` does nothing and returns `false`. This is your tool for "sign up a new user" or "claim this order id," where accidentally overwriting an existing record would be a bug. Because every key is serialized at its home (see [eventual consistency](./README.md#eventual-consistency-in-plain-words)), `create` is race-safe: if two requests create the same key at the same instant, exactly one gets `true` and the other gets `false`.
98
-
99
- ```ts
100
- const ok = AppDb.users.create(new UserId(input.id), input);
101
- if (!ok) {
102
- return Response.text('that id is taken', 409);
103
- }
104
- ```
105
-
106
- **`patch` overwrites an existing record** and returns the value now stored. The record **must already exist**: `patch` on a missing key traps (aborts the request), so create it first. Despite the name, `patch` replaces the whole value; there is no field-level partial update, so read the current value, change the fields you want, and patch the whole thing back:
107
-
108
- ```ts
109
- const current = AppDb.users.get(new UserId('u_123'));
110
- if (current == null) return Response.notFound();
111
- current.score = current.score + 10;
112
- const saved: User = AppDb.users.patch(new UserId('u_123'), current);
113
- // saved is what is now stored
114
- ```
115
-
116
- **`enqueue` is a version-checked overwrite (a compare-and-swap).** Like `patch`, it replaces the whole value of an **existing** record, but it does so *only if the record has not changed since you read it*. A **compare-and-swap** (CAS) is exactly that: "write my new value, but only if the current value is still the one I saw." It returns a `bool`: `true` means your write was applied; `false` means either a concurrent write changed the record first (someone else beat you to it) or the record is absent. A `false` is **not an error**; it is the signal to **re-read and try again**. This approach is called **optimistic concurrency**: rather than locking the record, you assume nobody else will touch it, and you simply re-run the update on the rare occasion someone did.
117
-
118
- Reach for `enqueue` when several requests may update the *same* record at once and you must not silently lose any of their changes. A plain `patch` cannot promise that: two overlapping patches clobber each other (the last writer wins and the earlier update just vanishes). The intended pattern for `enqueue` is always a read-modify-CAS **retry loop**:
119
-
120
- ```ts
121
- const key = new UserId('u_123');
122
- for (let attempt = 0; attempt < 5; attempt++) {
123
- const current = AppDb.users.get(key); // 1. read the current value
124
- if (current == null) return Response.notFound();
125
- current.score = current.score + 10; // 2. modify your copy
126
- if (AppDb.users.enqueue(key, current)) { // 3. try to commit it
127
- return Response.text('ok'); // true: applied, we are done
128
- }
129
- // false: someone wrote between our get and our enqueue.
130
- // Loop: re-read the now-newer value and reapply the change on top of it.
131
- }
132
- return Response.text('too much contention, try again later', 409);
133
- ```
134
-
135
- Because every retry re-reads the latest value, the two updates **compose** (both `+10`s land) instead of one silently overwriting the other. If you do not need this guard (only one writer touches the key, or last-write-wins is genuinely fine), a plain `patch` is simpler and also hands you the stored value back directly.
136
-
137
- **`upsert` creates the record or overwrites it, in one operation.** It is the "just store this value, I do not care whether a row was already there" call, and it is the right tool for a save that runs repeatedly (a profile edit, a settings blob) where the first save inserts and every later save replaces. It returns an `UpsertResult` telling you which happened:
138
-
139
- ```ts
140
- const key = new UserId('u_123');
141
- const result = AppDb.users.upsert(key, user);
142
- // result is UpsertResult.Created (row was absent) or UpsertResult.Updated (overwrote)
143
- ```
144
-
145
- Before `upsert`, that meant two operations, `create` then `patch`:
146
-
147
- ```ts
148
- // the old two-op idiom - upsert replaces it
149
- if (!AppDb.users.create(key, user)) {
150
- AppDb.users.patch(key, user);
151
- }
152
- ```
153
-
154
- `upsert` collapses both into a single write, so the common "already exists" path costs one round trip instead of two.
155
-
156
- Like `patch`, it is a **last-writer-wins** overwrite, not a compare-and-swap: concurrent upserts never fail or retry, the later one simply wins. That makes it correct for writing a **whole value**, and wrong for a read-modify-write of accumulating state (a running total, a like count) where a concurrent write would be silently lost, use `enqueue`'s retry loop or a [Counter](./counters.md) there. On a collection with a `@unique` field, `upsert` returns `UpsertResult.Conflict` (and writes nothing) when the value's unique field is already held by a **different** record, the one case it cannot resolve by overwriting:
157
-
158
- ```ts
159
- const outcome = AppDb.handles.upsert(key, profile);
160
- if (outcome == UpsertResult.Conflict) {
161
- return Response.text('that handle is taken', 409);
162
- }
163
- ```
164
-
165
- ### Removing: `delete` and `getDelete`
166
-
167
- `delete` removes a record. It is **idempotent**: deleting a key that is already gone is not an error, it just does nothing. So you can call it without first checking that the record exists.
168
-
169
- ```ts
170
- AppDb.users.delete(new UserId('u_123'));
171
- ```
172
-
173
- `getDelete` is the atomic **fetch-and-remove**: in one indivisible step it reads the current value and deletes it, returning what it removed (or `null` if there was nothing). "Atomic" here means no other request can slip in between the read and the delete, so exactly one caller can ever receive a given value. That makes it the right tool for **consume-once** data: one-time login challenges, single-use invite codes, password-reset tokens. The PQ-auth demo uses it to consume a login challenge exactly once, so a challenge cannot be replayed:
174
-
175
- ```ts
176
- const challenge = AppDb.challenges.getDelete(new ChallengeId(cid));
177
- if (challenge == null) return fail(); // unknown, already used, or expired
178
- // ...verify against challenge...
179
- ```
180
-
181
- If two requests race to `getDelete` the same key, only one gets the value; the other gets `null`. That is the guarantee a plain `get` then `delete` cannot give you, because two racers could both `get` the value before either `delete`s it.
182
-
183
- ## A full worked example: a small CRUD entity
184
-
185
- Putting it together, here is a complete `notes` resource: create, read, update, and delete, backed by a Documents collection.
186
-
187
- ```ts
188
- import { Response, RouteContext } from 'toiljs/server/runtime';
189
-
190
- // ---- key + value ----
191
- @data
192
- class NoteId {
193
- id: string = '';
194
- constructor(id: string = '') { this.id = id; }
195
- }
196
-
197
- @data
198
- class Note {
199
- id: string = '';
200
- title: string = '';
201
- body: string = '';
202
- updatedAt: u64 = 0;
203
- }
204
-
205
- // ---- database ----
206
- @database
207
- class NotesDb {
208
- @collection static notes: Documents<NoteId, Note>;
209
- }
210
-
211
- // ---- routes ----
212
- @rest('notes')
213
- class Notes {
214
- // GET /notes/:id -> read one (Query: read-only)
215
- @get('/:id')
216
- public read(ctx: RouteContext): Response {
217
- const note = NotesDb.notes.get(new NoteId(ctx.param('id')));
218
- if (note == null) return Response.notFound();
219
- return Response.json(note.toJSON().toString());
220
- }
221
-
222
- // POST /notes -> create a new one, refusing a duplicate id (Action: may write)
223
- @post('/')
224
- public create(input: Note): Response {
225
- input.updatedAt = <u64>(Date.now() / 1000);
226
- if (!NotesDb.notes.create(new NoteId(input.id), input)) {
227
- return Response.text('id already exists', 409);
228
- }
229
- return Response.json(input.toJSON().toString());
230
- }
231
-
232
- // POST /notes/:id -> overwrite an existing note (Action)
233
- @post('/:id')
234
- public update(input: Note, ctx: RouteContext): Response {
235
- const key = new NoteId(ctx.param('id'));
236
- if (!NotesDb.notes.exists(key)) return Response.notFound();
237
- input.id = ctx.param('id');
238
- input.updatedAt = <u64>(Date.now() / 1000);
239
- const saved = NotesDb.notes.patch(key, input);
240
- return Response.json(saved.toJSON().toString());
241
- }
242
-
243
- // POST /notes/:id/delete -> remove one (Action)
244
- @post('/:id/delete')
245
- public remove(ctx: RouteContext): Response {
246
- NotesDb.notes.delete(new NoteId(ctx.param('id')));
247
- return Response.text('deleted');
248
- }
249
- }
250
- ```
251
-
252
- That is a complete persistent CRUD entity. Run it under `toiljs dev` and the notes survive across requests; deploy it and the same code stores them worldwide on the edge.
253
-
254
- ## Consistency notes
255
-
256
- Documents follows ToilDB's general model (see [the overview](./README.md#eventual-consistency-in-plain-words)):
257
-
258
- - **Writes to one key are serialized at that key's home**, so `create` is race-safe and `patch`/`upsert`/`enqueue`/`getDelete` never corrupt a record under concurrency. `upsert` is last-writer-wins like `patch`: serialization means the writes do not tear, but the later one still overwrites the earlier, so it does not by itself prevent a lost update (use `enqueue` for that).
259
- - **Reads are eventually consistent across regions.** Right after a write, a read from a far-away region may briefly still see the old value (or, for a just-created record, not see it yet). The copies converge within moments.
260
- - Because `patch` replaces the whole value, two updates to *different* fields of the same record can clobber each other if they overlap (read-modify-write races). `enqueue`'s version check is exactly the guard against that: use the read-modify-CAS retry loop shown above so a lost update turns into a retry instead of silent data loss. If you find yourself contending on one hot record a lot, a counter or a set is often a better fit than a Documents value. See [Counters](./counters.md) and [Membership](./membership.md).
261
-
262
- ## Gotchas
263
-
264
- - **`patch` requires an existing record.** Calling it on a missing key traps the request. Use `create` for new records, or `upsert` when the record may or may not exist yet.
265
- - **`patch` replaces the whole value.** There is no field-level merge; read, modify, and write back the full value.
266
- - **`enqueue` returning `false` is not a failure.** It means a concurrent write beat you to the record (or the record is absent), so re-read and retry in a loop; never ignore the return value or treat `false` as a hard error. `enqueue` also does not hand back the stored value; use `patch` when you want the value returned and last-write-wins is acceptable.
267
- - **`upsert` is last-writer-wins, not a merge.** It writes the whole value you pass, so it is for storing a complete value, not a read-modify-write of one field of a shared record. Two overlapping upserts do not error, but the earlier one is overwritten (lost). When several writers contend on one record and must not lose each other's changes, use `enqueue`'s retry loop or a [Counter](./counters.md).
268
- - **No "get all."** There is no scan on the request path. Use `getMany` for known keys, and [Events](./events.md) or a [View](./views.md) for "the latest N."
269
- - **`getDelete`, not `get` + `delete`, for consume-once.** Only `getDelete` guarantees exactly one caller receives the value.
270
-
271
- ## Related
272
-
273
- - [ToilDB overview](./README.md): the seven families and how to choose.
274
- - [Setup](./setup.md): declaring the collection and which function kinds may write.
275
- - [Data types (`@data`)](../backend/data.md): keys and values.
276
- - [Counters](./counters.md): when you are really just counting.
277
- - [Events](./events.md) and [Views](./views.md): for "the latest N" and precomputed reads.
@@ -1,307 +0,0 @@
1
- # Events (append-only logs)
2
-
3
- An `Events` collection is an **append-only log**: a growing list of immutable
4
- facts under a key. You add events to the end and read the newest ones back. You
5
- never edit or delete an event once it is written.
6
-
7
- ## What and why
8
-
9
- An "append-only log" means exactly two things happen to it:
10
-
11
- 1. You **append** a new event (it goes on the end).
12
- 2. You **read** events back (newest first).
13
-
14
- That is it. There is no "update event 5" and no "delete event 3". Each event is a
15
- permanent record of something that happened: "Ada signed the guestbook", "order
16
- 1234 was refunded", "user logged in from a new device".
17
-
18
- Use an `Events` log when you care about the **history**, not just the latest
19
- state:
20
-
21
- - **Activity feeds:** "what happened in this room, newest first".
22
- - **Audit trails:** a tamper-resistant record of every important action.
23
- - **Event sourcing:** treat the log as the source of truth and compute other
24
- values (totals, summaries, leaderboards) from it.
25
-
26
- If you only ever need the *current* value of something (a user's profile, a
27
- shopping cart), use [Documents](./documents.md) instead. If you only need a
28
- running total (likes, page views), use a [Counter](./counters.md). Events are
29
- for keeping the full list of what happened.
30
-
31
- ```mermaid
32
- flowchart LR
33
- A["append(user, event)"] --> L[("Event log for user<br/>(oldest ... newest)")]
34
- B["append(user, event)"] --> L
35
- L --> R["latest(user, 20)<br/>returns newest 20"]
36
- ```
37
-
38
- ## The type
39
-
40
- An `Events` collection has two type parameters: the **key** (which log) and the
41
- **event** (what each entry stores).
42
-
43
- ```ts
44
- Events<K, V>
45
- ```
46
-
47
- - `K` is the key type: it picks *which* log you are appending to. One key, one
48
- log. For a per-user activity feed, the key is the user id.
49
- - `V` is the event type: the shape of a single entry. Both `K` and `V` are
50
- [`@data`](../concepts/types.md) classes so they can be stored as bytes.
51
-
52
- You declare it as a `@collection` field inside a `@database` class:
53
-
54
- ```ts
55
- @data
56
- class UserKey {
57
- id: string = '';
58
- constructor(id: string = '') { this.id = id; }
59
- }
60
-
61
- @data
62
- class Activity {
63
- action: string = ''; // 'login', 'post.create', 'comment', ...
64
- detail: string = '';
65
- at: u64 = 0; // unix seconds
66
- }
67
-
68
- @database
69
- class FeedDb {
70
- @collection static activity: Events<UserKey, Activity>;
71
- }
72
- ```
73
-
74
- The field name (`activity`) names the collection; the class name (`FeedDb`) names
75
- the database. You reach the log through `FeedDb.activity`.
76
-
77
- ## Operations
78
-
79
- There are three operations, exactly matching the [toilscript](../concepts/decorators.md)
80
- API. Their exact signatures:
81
-
82
- | Operation | Signature | What it does |
83
- | --- | --- | --- |
84
- | `append` | `append(key: K, event: V): void` | Add one event to the end of the log. |
85
- | `appendOnce` | `appendOnce(key: K, eventId: string, event: V): bool` | Add one event, but only if that `eventId` was never seen before. |
86
- | `latest` | `latest(key: K, limit: i32): V[]` | Read up to `limit` events, newest first. |
87
- | `since` | `since(key: K, limit: i32): V[]` | Read the NEXT batch of events past a `@derive`'s checkpoint, oldest first. For folding a growing log incrementally. |
88
-
89
- ### `append`
90
-
91
- The everyday operation. It adds one event and returns nothing.
92
-
93
- ```ts
94
- FeedDb.activity.append(
95
- new UserKey('ada'),
96
- new Activity('login', 'from Paris', now),
97
- );
98
- ```
99
-
100
- `append` is a **write**, so you may call it from an action handler (`@post`,
101
- `@put`, `@patch`, `@del`) but not from a read-only `@get`. See
102
- [decorators](../concepts/decorators.md) for what each handler kind may do.
103
-
104
- ### `appendOnce` (idempotent append)
105
-
106
- `appendOnce` adds an event **at most once** for a given id. "Idempotent" means
107
- running it more than once has the same effect as running it once: the extra calls
108
- change nothing.
109
-
110
- Why you want this: networks retry. A client (or another service) may send the
111
- same request twice because the first reply got lost. With plain `append`, that
112
- double-delivery writes the event twice. With `appendOnce`, you pass a stable
113
- `eventId` (a string you choose that is unique for that logical event), and the
114
- log records it only once.
115
-
116
- ```ts
117
- // `orderId` uniquely identifies this order event. If the request is retried,
118
- // the second call is a no-op and the log still has exactly one entry.
119
- const first = FeedDb.activity.appendOnce(
120
- new UserKey('ada'),
121
- `order-refunded:${orderId}`, // the idempotency id
122
- new Activity('order.refund', orderId, now),
123
- );
124
- // first === true -> this call appended the event
125
- // first === false -> a previous call already appended it; nothing happened now
126
- ```
127
-
128
- The return value tells you which happened:
129
-
130
- - `true`: this call appended the event (it was the first with that id).
131
- - `false`: a call with the same id already appended it, so this was a no-op.
132
-
133
- Pick an `eventId` that is truly unique per event: an order id, a payment id, a
134
- message uuid. Do not reuse one id for two different events, or the second will be
135
- silently dropped as a "duplicate".
136
-
137
- ### `latest`
138
-
139
- Reads the newest events back, up to a limit, newest first.
140
-
141
- ```ts
142
- const recent = FeedDb.activity.latest(new UserKey('ada'), 20);
143
- // recent[0] is the most recent event, recent[1] the next, ...
144
- ```
145
-
146
- `latest` is a **scan**: it can walk many rows. Scans are barred on the request
147
- path, so you **cannot** call `latest` from a `@get` or a `@post`. This is a
148
- deliberate guardrail: a log can grow without bound, and a per-request scan would
149
- get slower and more expensive as the log grows.
150
-
151
- Instead you scan the log **off** the request path, in a
152
- [`@derive`](../background/derive.md), and publish a small precomputed
153
- [View](./views.md) that your routes read cheaply. The next section shows the full
154
- pattern.
155
-
156
- ### `since` (incremental read for a `@derive`)
157
-
158
- `latest` rescans the tail every time. For a log that grows without bound, that gets slower and slower. `since`
159
- reads only the events you have **not folded yet**: the next batch past a saved cursor, oldest first, up to a
160
- limit. It is how a [`@derive`](../background/derive.md) folds a huge log incrementally instead of rescanning
161
- it from the start on every change.
162
-
163
- You do not pass or manage the cursor. The host owns it: it seeds `since` from a durable checkpoint, advances
164
- it as it hands you events, and saves it after your fold's view publish lands. So you just loop until the
165
- batch is empty:
166
-
167
- ```ts
168
- @derive
169
- rollup(): void {
170
- const key = new StatsKey('global');
171
- const view = StatsDb.summary.get(key) ?? new Summary(); // the running view
172
- let batch = StatsDb.events.since(key, 500);
173
- while (batch.length > 0) { // drain in bounded batches
174
- for (let i = 0; i < batch.length; i++) view.apply(batch[i]);
175
- batch = StatsDb.events.since(key, 500);
176
- }
177
- StatsDb.summary.publish(key, view);
178
- }
179
- ```
180
-
181
- Two rules:
182
-
183
- - **`@derive` or `@job` only.** Like `latest`, `since` is a scan, so it is barred on the request path (`@get`
184
- / `@post`). Calling it there is rejected.
185
- - **Your fold must be idempotent per event.** A rare crash-recovery case (a "healed" event that arrives at an
186
- older position) makes the host re-read the log from the start that one run, so applying an event twice must
187
- not change the result. Use set-style updates (`view.byId[e.id] = e.value`), not blind accumulation
188
- (`view.count += 1`). If your fold cannot be idempotent, use `latest` and recompute instead.
189
-
190
- `since` is the efficient path; `latest` (recompute) is the simple path. See [`@derive`](../background/derive.md)
191
- for the full picture.
192
-
193
- ## Worked example: an activity feed
194
-
195
- Here is the whole loop: an action appends an event, a `@derive` folds the log
196
- into a view, and a `@get` serves the view.
197
-
198
- ```ts
199
- import { Activity } from '../models/Activity';
200
- import { UserKey } from '../models/UserKey';
201
- import { FeedView } from '../models/FeedView';
202
- import { NewActivity } from '../models/NewActivity';
203
-
204
- @database
205
- class FeedDb {
206
- // The source of truth: every activity, appended forever.
207
- @collection static activity: Events<UserKey, Activity>;
208
- // The precomputed snapshot the GET serves: the newest 20 activities.
209
- @collection static feed: View<UserKey, FeedView>;
210
-
211
- // A @derive runs OFF the request path, so it is allowed to scan the log
212
- // (`latest`) and publish the view. The runtime runs it right after an append,
213
- // so the view reflects the new event on the next read.
214
- @derive
215
- rebuild(): void {
216
- const key = new UserKey('ada'); // (real code would loop per active user)
217
- const view = new FeedView();
218
- view.items = FeedDb.activity.latest(key, 20);
219
- FeedDb.feed.publish(key, view);
220
- }
221
- }
222
-
223
- @rest('feed')
224
- class Feed {
225
- // GET reads the precomputed view: a single keyed read, NOT a scan.
226
- @get('/')
227
- public list(): FeedView {
228
- const view = FeedDb.feed.get(new UserKey('ada'));
229
- return view == null ? new FeedView() : view;
230
- }
231
-
232
- // POST appends one activity. The @derive republishes `feed` right after.
233
- @post('/')
234
- public record(input: NewActivity): FeedView {
235
- const at = <u64>(Date.now() / 1000);
236
- FeedDb.activity.append(new UserKey('ada'), new Activity(input.action, input.detail, at));
237
- // We do NOT scan here (that would be barred). We just acknowledge; the GET
238
- // serves the updated list from the view the derive rebuilds.
239
- return new FeedView();
240
- }
241
- }
242
- ```
243
-
244
- The two `@data` models:
245
-
246
- ```ts
247
- @data
248
- export class Activity {
249
- action: string = '';
250
- detail: string = '';
251
- at: u64 = 0;
252
- }
253
-
254
- @data
255
- export class FeedView {
256
- items: Activity[] = [];
257
- }
258
- ```
259
-
260
- The key idea: the **log** is the durable history, and the **view** is a small,
261
- read-cheap snapshot derived from it. Reads hit the view; the log grows quietly in
262
- the background.
263
-
264
- ## Ordering and consistency
265
-
266
- ToilDB is a **worldwide** database: your data lives in many regions at once. A
267
- few honest details about what that means for events:
268
-
269
- - **Ordering within a log is stable.** Each key (each log) has one "home" location
270
- where appends are serialized and stamped with an increasing sequence number. So
271
- `latest` always returns a consistent newest-first order for that key.
272
- - **`latest` is newest first.** Index `0` is the most recent event.
273
- - **Reads can lag slightly across regions.** An append is applied at the log's
274
- home first, then copied out to other regions in the background (this is called
275
- *asynchronous replication*, and the result is *eventual consistency*: given a
276
- little time, every region converges to the same log). A read served from a far
277
- region may briefly miss the very newest event. A `@derive` runs right after the
278
- append that triggered it, so the view it publishes reflects that write.
279
- - **`appendOnce` dedups by your id.** The at-most-once guarantee is anchored to
280
- the `eventId` you pass, so retries are safe even across a flaky network.
281
- - **A log grows without bound.** Appending never shrinks it. Keep events small,
282
- and read them through a bounded `latest(key, N)` in a derive rather than trying
283
- to hold the whole log in memory.
284
-
285
- ## Gotchas
286
-
287
- - **You cannot read the log from a route.** `latest` is a scan and is only legal
288
- in a `@derive` or a `@job`. If you try to call it from a `@get`/`@post`, the
289
- compiler rejects it. Read a [View](./views.md) from your route instead.
290
- - **Events are immutable.** There is no "edit" or "delete an event". If a fact
291
- changes, append a new event that records the change (for example, an
292
- `order.refund` event, not an edit to the original `order.create`).
293
- - **Choose `eventId` carefully for `appendOnce`.** It must be unique per logical
294
- event. Reusing an id drops the new event as a duplicate.
295
- - **Do not treat a counter as a log.** A [Counter](./counters.md) gives you a
296
- running total but forgets the individual events. If you need the list, use
297
- `Events`.
298
-
299
- ## Related
300
-
301
- - [Documents](./documents.md): mutable, one value per key (edit in place).
302
- - [Counters](./counters.md): a single running total per key.
303
- - [Views](./views.md): the precomputed snapshot your routes read.
304
- - [`@derive`](../background/derive.md): folds an event log into a View, off the
305
- request path.
306
- - [Data types (`@data`)](../concepts/types.md): how keys and events are stored.
307
- - [Decorators](../concepts/decorators.md): which handler kinds may append vs scan.