toiljs 0.0.85 → 0.0.87

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (132) hide show
  1. package/CHANGELOG.md +5 -0
  2. package/README.md +2 -2
  3. package/build/cli/.tsbuildinfo +1 -1
  4. package/build/cli/index.js +303 -293
  5. package/build/compiler/.tsbuildinfo +1 -1
  6. package/build/compiler/config.d.ts +2 -0
  7. package/build/compiler/config.js +1 -0
  8. package/build/compiler/docs.js +8 -24
  9. package/build/compiler/generate.js +4 -2
  10. package/build/compiler/index.d.ts +1 -1
  11. package/build/compiler/index.js +69 -6
  12. package/build/compiler/toil-docs.generated.js +64 -22
  13. package/build/devserver/.tsbuildinfo +1 -1
  14. package/build/devserver/analytics/index.js +7 -3
  15. package/build/devserver/db/database.d.ts +4 -0
  16. package/build/devserver/db/database.js +43 -1
  17. package/build/devserver/db/index.d.ts +1 -1
  18. package/build/devserver/db/index.js +1 -1
  19. package/build/devserver/db/types.d.ts +3 -0
  20. package/build/devserver/db/types.js +2 -0
  21. package/build/devserver/runtime/module.js +4 -1
  22. package/docs/README.md +104 -65
  23. package/docs/auth/README.md +102 -0
  24. package/docs/auth/configuration.md +94 -0
  25. package/docs/auth/extending.md +202 -0
  26. package/docs/auth/how-it-works.md +138 -0
  27. package/docs/auth/usage.md +188 -0
  28. package/docs/backend/README.md +143 -0
  29. package/docs/backend/data.md +351 -0
  30. package/docs/backend/rest.md +402 -0
  31. package/docs/backend/rpc.md +226 -0
  32. package/docs/background/README.md +114 -0
  33. package/docs/background/daemons.md +230 -0
  34. package/docs/background/derive.md +179 -0
  35. package/docs/cli/README.md +377 -0
  36. package/docs/concepts/config.md +416 -0
  37. package/docs/concepts/decorators.md +127 -0
  38. package/docs/concepts/security.md +108 -0
  39. package/docs/concepts/tiers.md +166 -0
  40. package/docs/concepts/types.md +216 -0
  41. package/docs/database/README.md +143 -0
  42. package/docs/database/capacity.md +350 -0
  43. package/docs/database/counters.md +174 -0
  44. package/docs/database/documents.md +255 -0
  45. package/docs/database/events.md +307 -0
  46. package/docs/database/membership.md +246 -0
  47. package/docs/database/setup.md +155 -0
  48. package/docs/database/unique.md +216 -0
  49. package/docs/database/views.md +246 -0
  50. package/docs/frontend/README.md +101 -0
  51. package/docs/frontend/data-fetching.md +243 -0
  52. package/docs/frontend/images.md +148 -0
  53. package/docs/frontend/metadata.md +344 -0
  54. package/docs/frontend/rendering.md +236 -0
  55. package/docs/frontend/routing.md +344 -0
  56. package/docs/frontend/scripts.md +118 -0
  57. package/docs/frontend/search.md +191 -0
  58. package/docs/frontend/styling.md +147 -0
  59. package/docs/getting-started/README.md +81 -0
  60. package/docs/getting-started/create-project.md +131 -0
  61. package/docs/getting-started/deploy.md +101 -0
  62. package/docs/getting-started/first-app.md +215 -0
  63. package/docs/getting-started/installation.md +106 -0
  64. package/docs/getting-started/migrating.md +125 -0
  65. package/docs/getting-started/project-structure.md +163 -0
  66. package/docs/introduction/README.md +41 -0
  67. package/docs/introduction/design-principles.md +55 -0
  68. package/docs/introduction/distributed.md +74 -0
  69. package/docs/introduction/how-it-works.md +74 -0
  70. package/docs/introduction/hyperscale.md +51 -0
  71. package/docs/introduction/modern-stack.md +36 -0
  72. package/docs/introduction/vs-other-frameworks.md +48 -0
  73. package/docs/introduction/why-toil.md +93 -0
  74. package/docs/llms.txt +90 -0
  75. package/docs/realtime/README.md +102 -0
  76. package/docs/realtime/channels.md +211 -0
  77. package/docs/realtime/streams.md +369 -0
  78. package/docs/services/README.md +60 -0
  79. package/docs/services/analytics.md +268 -0
  80. package/docs/services/caching.md +175 -0
  81. package/docs/services/cookies.md +235 -0
  82. package/docs/services/crypto.md +209 -0
  83. package/docs/services/email.md +289 -0
  84. package/docs/services/environment.md +141 -0
  85. package/docs/services/ratelimit.md +174 -0
  86. package/docs/services/time.md +85 -0
  87. package/examples/basic/client/routes/analytics.tsx +13 -12
  88. package/examples/basic/server/models/SiteAnalytics.ts +29 -17
  89. package/examples/basic/server/routes/Analytics.ts +15 -17
  90. package/examples/basic/server/routes/UserId.ts +22 -0
  91. package/package.json +15 -11
  92. package/scripts/gen-toil-docs.mjs +24 -35
  93. package/server/auth/AuthController.ts +336 -0
  94. package/server/auth/AuthUser.ts +23 -0
  95. package/server/auth/index.ts +16 -0
  96. package/server/globals/auth.ts +31 -0
  97. package/server/globals/userid.ts +107 -0
  98. package/src/compiler/config.ts +13 -0
  99. package/src/compiler/docs.ts +16 -33
  100. package/src/compiler/generate.ts +6 -2
  101. package/src/compiler/index.ts +114 -6
  102. package/src/compiler/toil-docs.generated.ts +64 -22
  103. package/src/devserver/analytics/index.ts +10 -4
  104. package/src/devserver/db/database.ts +67 -1
  105. package/src/devserver/db/index.ts +1 -0
  106. package/src/devserver/db/types.ts +13 -0
  107. package/src/devserver/runtime/module.ts +7 -0
  108. package/test/analytics-dev.test.ts +2 -1
  109. package/test/devserver-database.test.ts +113 -0
  110. package/docs/auth-todo.md +0 -149
  111. package/docs/auth.md +0 -322
  112. package/docs/caching.md +0 -115
  113. package/docs/cli.md +0 -17
  114. package/docs/client.md +0 -39
  115. package/docs/cookies.md +0 -457
  116. package/docs/crypto.md +0 -130
  117. package/docs/daemon.md +0 -123
  118. package/docs/data.md +0 -131
  119. package/docs/derive.md +0 -159
  120. package/docs/email.md +0 -326
  121. package/docs/environment.md +0 -97
  122. package/docs/getting-started.md +0 -128
  123. package/docs/index.md +0 -30
  124. package/docs/ratelimit.md +0 -95
  125. package/docs/routing.md +0 -259
  126. package/docs/rpc.md +0 -149
  127. package/docs/server.md +0 -61
  128. package/docs/ssr.md +0 -632
  129. package/docs/streams.md +0 -178
  130. package/docs/styling.md +0 -22
  131. package/docs/tiers.md +0 -133
  132. package/docs/time.md +0 -43
@@ -0,0 +1,246 @@
1
+ # Membership (sets and relationships)
2
+
3
+ A `Membership` collection stores **sets**: unordered groups of members under a
4
+ key. You add and remove members, ask whether a member is in the set, and list a
5
+ set's members. It is the natural fit for many-to-many relationships like group
6
+ members, followers, and tags.
7
+
8
+ ## What and why
9
+
10
+ A "set" is a collection with no duplicates and no order: a member is either in it
11
+ or not. `Membership` gives you one set per key. The key names the set; each member
12
+ is one item in it.
13
+
14
+ This maps directly onto **many-to-many relationships**, where each thing on one
15
+ side can relate to many things on the other:
16
+
17
+ - **Group members:** the key is a group id, the members are user ids.
18
+ - **Followers:** the key is a user id, the members are the ids of their followers.
19
+ - **Tags:** the key is an article id, the members are tag names.
20
+ - **Access control:** the key is a resource id, the members are the users allowed
21
+ in.
22
+
23
+ You could try to store these as an array field inside a [Document](./documents.md)
24
+ (for example `group.memberIds: string[]`). That works for **small, rarely-changed**
25
+ sets, but it has real limits:
26
+
27
+ | | Array in a Document | `Membership` set |
28
+ | --- | --- | --- |
29
+ | Add/remove one member | Read the whole array, edit it, write it all back | One direct `add`/`remove`, no read-modify-write |
30
+ | Concurrent edits | Two writers can clobber each other's array | Each `add`/`remove` is its own operation |
31
+ | Big sets | The whole array is loaded on every read/write | Members are stored separately; you read a bounded page |
32
+ | "Is X a member?" | Load the array and search it | One direct `contains` check |
33
+
34
+ Rule of thumb: a **handful** of stable items (a user's two or three roles) is fine
35
+ as an array in a Document. A set that **grows** or is **edited concurrently**
36
+ (followers, group members, tags across many articles) should be a `Membership`.
37
+
38
+ ```mermaid
39
+ flowchart LR
40
+ K["Group key: 'eng'"] --> S[("Membership set")]
41
+ S --> M1["ada"]
42
+ S --> M2["grace"]
43
+ S --> M3["linus"]
44
+ ```
45
+
46
+ ## The type
47
+
48
+ ```ts
49
+ Membership<K, M>
50
+ ```
51
+
52
+ - `K` is the key type: it picks *which* set. For group membership, the key is the
53
+ group id.
54
+ - `M` is the member type: the shape of one member. For user membership, that is a
55
+ user id. Both are [`@data`](../concepts/types.md) classes.
56
+
57
+ Declare it as a `@collection` field on a `@database` class:
58
+
59
+ ```ts
60
+ @data
61
+ class GroupKey {
62
+ group: string = '';
63
+ constructor(group: string = '') { this.group = group; }
64
+ }
65
+
66
+ @data
67
+ class Member {
68
+ userId: string = '';
69
+ constructor(userId: string = '') { this.userId = userId; }
70
+ }
71
+
72
+ @database
73
+ class GroupsDb {
74
+ @collection static members: Membership<GroupKey, Member>;
75
+ }
76
+ ```
77
+
78
+ ## Operations
79
+
80
+ Four operations, matching the toilscript API exactly. Exact signatures:
81
+
82
+ | Operation | Signature | What it does |
83
+ | --- | --- | --- |
84
+ | `contains` | `contains(key: K, member: M): bool` | Is `member` in the set? |
85
+ | `add` | `add(key: K, member: M): void` | Put `member` in the set (idempotent). |
86
+ | `remove` | `remove(key: K, member: M): void` | Take `member` out of the set (idempotent). |
87
+ | `list` | `list(key: K, limit: i32): M[]` | Up to `limit` members of the set. |
88
+
89
+ ### `contains`
90
+
91
+ A direct membership check. It is a keyed read, so it is allowed from any handler,
92
+ including a read-only `@get`.
93
+
94
+ ```ts
95
+ if (GroupsDb.members.contains(new GroupKey('eng'), new Member('ada'))) {
96
+ // ada is in the 'eng' group
97
+ }
98
+ ```
99
+
100
+ ### `add` and `remove`
101
+
102
+ Both are **writes**, so you call them from an action handler (`@post`, `@put`,
103
+ `@patch`, `@del`), not from a `@get`. Both are **idempotent**: "idempotent"
104
+ means doing it again has no extra effect.
105
+
106
+ - `add` on a member already in the set: no change, no error.
107
+ - `remove` on a member not in the set: no change, no error.
108
+
109
+ ```ts
110
+ GroupsDb.members.add(new GroupKey('eng'), new Member('ada')); // ada joins
111
+ GroupsDb.members.add(new GroupKey('eng'), new Member('ada')); // no-op, still one 'ada'
112
+ GroupsDb.members.remove(new GroupKey('eng'), new Member('linus')); // linus leaves (or no-op)
113
+ ```
114
+
115
+ Because they are idempotent, you do not need to check `contains` before calling
116
+ them. Just `add` to join and `remove` to leave.
117
+
118
+ ### `list`
119
+
120
+ Returns up to `limit` members of a set.
121
+
122
+ ```ts
123
+ const roster = GroupsDb.members.list(new GroupKey('eng'), 100);
124
+ ```
125
+
126
+ `list` is a **scan** (it can walk many rows), so it is barred on the request
127
+ path: you **cannot** call it from a `@get` or a `@post`. Like reading a whole
128
+ event log, listing a whole set belongs off the request path, in a
129
+ [`@derive`](../background/derive.md) or a `@job`, which publishes a
130
+ [View](./views.md) your routes read. The [worked example](#worked-example-group-membership)
131
+ below shows this.
132
+
133
+ `list` returns up to `limit` members. A set can be larger than one page, so treat
134
+ the result as "a bounded page of members", not necessarily "every member".
135
+
136
+ ## Worked example: group membership
137
+
138
+ A user joins or leaves a group from a route; a derive lists the roster into a
139
+ view; a route reads the view. Membership checks happen directly in the route.
140
+
141
+ ```ts
142
+ import { GroupKey } from '../models/GroupKey';
143
+ import { Member } from '../models/Member';
144
+ import { Roster } from '../models/Roster';
145
+ import { JoinRequest } from '../models/JoinRequest';
146
+
147
+ @database
148
+ class GroupsDb {
149
+ // The set: who is in each group.
150
+ @collection static members: Membership<GroupKey, Member>;
151
+ // A precomputed roster for the GET (listing is a scan, barred on routes).
152
+ @collection static roster: View<GroupKey, Roster>;
153
+
154
+ @derive
155
+ rebuild(): void {
156
+ const key = new GroupKey('eng');
157
+ const r = new Roster();
158
+ r.users = GroupsDb.members.list(key, 500); // scan, allowed in a derive
159
+ GroupsDb.roster.publish(key, r);
160
+ }
161
+ }
162
+
163
+ @rest('groups')
164
+ class Groups {
165
+ // GET the roster from the view (a keyed read, not a scan).
166
+ @get('/eng')
167
+ public list(): Roster {
168
+ const r = GroupsDb.roster.get(new GroupKey('eng'));
169
+ return r == null ? new Roster() : r;
170
+ }
171
+
172
+ // Direct membership check: `contains` is a keyed read, legal in a GET.
173
+ @get('/eng/is-member')
174
+ public isMember(userId: string): bool {
175
+ return GroupsDb.members.contains(new GroupKey('eng'), new Member(userId));
176
+ }
177
+
178
+ // Join: `add` is idempotent, so joining twice is harmless.
179
+ @post('/eng/join')
180
+ public join(input: JoinRequest): bool {
181
+ GroupsDb.members.add(new GroupKey('eng'), new Member(input.userId));
182
+ return true; // the @derive rebuilds the roster view right after
183
+ }
184
+
185
+ // Leave: `remove` is idempotent, so leaving when not a member is harmless.
186
+ @post('/eng/leave')
187
+ public leave(input: JoinRequest): bool {
188
+ GroupsDb.members.remove(new GroupKey('eng'), new Member(input.userId));
189
+ return true;
190
+ }
191
+ }
192
+ ```
193
+
194
+ The models:
195
+
196
+ ```ts
197
+ @data
198
+ export class Member {
199
+ userId: string = '';
200
+ }
201
+
202
+ @data
203
+ export class Roster {
204
+ users: Member[] = [];
205
+ }
206
+ ```
207
+
208
+ The split is the point: `contains`/`add`/`remove` act on a single member and are
209
+ legal on the request path; `list` scans the whole set and lives in a derive that
210
+ feeds a view.
211
+
212
+ ## Consistency
213
+
214
+ - **Add and remove serialize at the set's home.** Each set key has one home
215
+ location where its writes are applied in order, so an `add` followed by a
216
+ `remove` of the same member lands in that order and the set ends up correct.
217
+ - **Reads can lag slightly across regions.** ToilDB is worldwide. A change made
218
+ at a set's home is copied to other regions in the background (asynchronous
219
+ replication, giving eventual consistency: every region converges after a short
220
+ delay). A `contains` or a `list` served from a far region may briefly miss a
221
+ just-made change.
222
+ - **`add`/`remove` are idempotent**, so retries are safe: re-adding an existing
223
+ member or re-removing an absent one does nothing.
224
+ - **A set can grow without bound.** Adding never shrinks it. Read it through a
225
+ bounded `list(key, N)` in a derive, not all at once.
226
+
227
+ ## Gotchas
228
+
229
+ - **You cannot `list` from a route.** `list` is a scan, legal only in a `@derive`
230
+ or `@job`. To show a roster on a page, publish it to a [View](./views.md) from a
231
+ derive and `get` that view in the route.
232
+ - **`list` returns a bounded page.** A large set may have more members than your
233
+ `limit`. Do not assume the returned array is the entire set.
234
+ - **Sets are unordered.** Do not rely on the order `list` returns members in.
235
+ - **Use `Membership` for growth or concurrency.** A small, stable set is fine as
236
+ an array in a Document. Reach for `Membership` when the set grows, is edited
237
+ concurrently, or you frequently ask "is X a member?".
238
+
239
+ ## Related
240
+
241
+ - [Documents](./documents.md): when a small array field is enough (and when it is
242
+ not).
243
+ - [Views](./views.md): where a roster/list belongs so routes can read it cheaply.
244
+ - [`@derive`](../background/derive.md): runs `list` off the request path.
245
+ - [Data types (`@data`)](../concepts/types.md): how set keys and members are stored.
246
+ - [Decorators](../concepts/decorators.md): which handler kinds may add/remove vs scan.
@@ -0,0 +1,155 @@
1
+ # Declaring a database
2
+
3
+ You set up ToilDB by **declaring** it in code: a `@database` class listing your collections, plus the `@data` key and value types those collections use. There is no schema file, no migration command, and no connection string.
4
+
5
+ ## What and why
6
+
7
+ A **database declaration** tells toiljs three things at compile time: which collections exist, which family each one is, and what key and value types it stores. The compiler bakes that list into your server's WebAssembly, and both the dev server and the production edge read it back to set up your collections automatically. You declare; the platform provisions.
8
+
9
+ Reach for this on day one: before you can read or write any data, you need a `@database` with at least one `@collection`.
10
+
11
+ ## How: the three pieces
12
+
13
+ A working database is always these three pieces together.
14
+
15
+ 1. **`@data` key and value types.** Plain classes, tagged `@data`, that describe what you store.
16
+ 2. **A `@database` class** whose static fields are `@collection`s, each typed by a family.
17
+ 3. **A handler** (a `@rest` route or an RPC method) that reads and writes those collections.
18
+
19
+ Here is a complete, minimal example: a store of users you look up by id.
20
+
21
+ ```ts
22
+ // A @data key: how you address one user.
23
+ @data
24
+ class UserId {
25
+ id: string = '';
26
+ constructor(id: string = '') { this.id = id; }
27
+ }
28
+
29
+ // A @data value: what you store for each user.
30
+ @data
31
+ class User {
32
+ id: string = '';
33
+ name: string = '';
34
+ score: u64 = 0;
35
+ }
36
+
37
+ // The database declaration. Each @collection is one collection,
38
+ // typed by its family (here, Documents) with <Key, Value>.
39
+ @database
40
+ class AppDb {
41
+ @collection static users: Documents<UserId, User>;
42
+ }
43
+
44
+ // A route that reads and writes the collection.
45
+ @rest('users')
46
+ class Users {
47
+ @get('/:id')
48
+ public getUser(ctx: RouteContext): User {
49
+ const user = AppDb.users.get(new UserId(ctx.param('id')));
50
+ return user == null ? new User() : user;
51
+ }
52
+
53
+ @post('/')
54
+ public createUser(input: User): User {
55
+ AppDb.users.create(new UserId(input.id), input);
56
+ return input;
57
+ }
58
+ }
59
+ ```
60
+
61
+ That is the whole setup. Run `toiljs dev` and `AppDb.users` works immediately.
62
+
63
+ ### The `@data` types
64
+
65
+ Both the key and the value are `@data` classes. `@data` is what makes a class storable: the compiler synthesizes a binary codec (pack to bytes, unpack from bytes) so ToilDB can persist it. The rules that matter here:
66
+
67
+ - **Give every field a default** (`= ''`, `= 0`, and so on). The decoder builds an empty instance and fills it, so it needs defaults.
68
+ - **A value type must be default-constructible** (creatable with `new User()` and no arguments). Keys often add a convenience constructor, as `UserId` does above, so you can write `new UserId('abc')`.
69
+ - Fields may be numbers (`u8`..`u256`, `i8`..`i256`, `f32`, `f64`), `bool`, `string`, another `@data` class, or an array of any of these.
70
+
71
+ The full reference, including how the same type becomes a typed client type, is on the [data types page](../backend/data.md).
72
+
73
+ ### The `@database` class and `@collection` fields
74
+
75
+ ```ts
76
+ @database
77
+ class AppDb {
78
+ @collection static users: Documents<UserId, User>;
79
+ @collection static likes: Counter<UserId>;
80
+ }
81
+ ```
82
+
83
+ - `@database` marks the class as a database. You can have more than one `@database` class; each is a separate namespace of collections.
84
+ - Each `@collection` is one collection. Declare it as a **`static`** field with **no initializer**: you write the type, and the compiler wires up the actual handle for you.
85
+ - The field's **type is its family**: `Documents<K, V>`, `Counter<K>`, `Events<K, V>`, `Unique<K, V>`, `Membership<K, M>`, `Capacity<K>`, or `View<K, V>`. The compiler reads the family straight from this type, so getting it right here is how you pick a family (see [Choosing a family](./README.md#choosing-a-family-the-decision-guide)).
86
+
87
+ You reach a collection through the class, statically: `AppDb.users.get(...)`, `AppDb.likes.add(...)`. There is nothing to instantiate.
88
+
89
+ ### Reaching collections from a handler
90
+
91
+ Any backend function can read and write collections by referencing them on the database class. What that function is allowed to do depends on its **kind**, covered next.
92
+
93
+ ## How access is gated: `@query`, `@action`, and friends
94
+
95
+ ToilDB will not let every function do everything. Each backend function runs as one **function kind**, and each kind is allowed a different slice of database operations. This is a safety rail: a read-only endpoint physically cannot write, and an expensive scan cannot run on the hot request path.
96
+
97
+ You rarely write these decorators by hand, because routes get a sensible kind automatically:
98
+
99
+ - A **`@get`** route (a safe, read-only HTTP method) runs as a **Query**.
100
+ - A **`@post`** route (a mutating method) runs as an **Action**.
101
+ - A plain RPC **`@remote`** method defaults to a **Query** (read-only) because it has no HTTP method to infer from. Tag it `@action` if it writes.
102
+
103
+ You can override the default with `@query` or `@action` on the method when the automatic choice is wrong (for example, a `@get` that genuinely needs to write, though that is unusual).
104
+
105
+ What each kind may do:
106
+
107
+ | Kind | Set by | May do | May **not** do |
108
+ | --- | --- | --- | --- |
109
+ | **Query** | `@get`, plain `@remote`, or `@query` | Point reads: `get`, `getMany`, `exists`, `lookup`, `contains`, counter `get`, view `get`, capacity `available`. | Any write. Any scan. |
110
+ | **Action** | `@post` or `@action` | Everything a Query can, plus bounded writes: `create`, `patch`, `delete`, `getDelete`, `enqueue`, `append`, `appendOnce`, counter `add`, membership `add`/`remove`, unique `claim`/`release`, capacity `reserve`/`confirm`/`cancel`. | Scans. Publishing a View. |
111
+ | **Derive / Job** | `@derive` / `@job` (background work) | Reads including **scans** (`latest`, membership `list`), plus `publish` a View. | (Run off the request path; see below.) |
112
+
113
+ Two rules trip people up, so they are worth stating plainly:
114
+
115
+ - **Scans are barred from request handlers.** Reading "the newest N events" (`events.latest`) or "the members of this set" (`membership.list`) can fan out across many rows, so a `@get` or `@post` cannot call them. Do the scan in a `@derive` (a small function that recomputes a snapshot off the request path) and have the request read the snapshot. See [Views](./views.md) and [@derive](../background/derive.md).
116
+ - **Only a `@derive` or `@job` may `publish` a View.** Requests read views; background work writes them.
117
+
118
+ Both gates are enforced twice: the compiler rejects an illegal call at build time, and the edge rejects it again at runtime, so a hand-edited module cannot sneak past. For the decorator catalog, see [Decorators](../concepts/decorators.md).
119
+
120
+ ## No manual provisioning: how it actually gets set up
121
+
122
+ You never create a table or run a migration. Here is the machinery, so the "it just works" is not a mystery.
123
+
124
+ When toilscript compiles your backend, it scans every `@database` class and writes a small catalog into the `.wasm` file: for each collection, its name, its family, its key and value type names, and the value's schema version. This catalog rides *inside* the compiled module.
125
+
126
+ Then, wherever your backend runs, the host reads that catalog once at startup and builds your collections to match:
127
+
128
+ - Under **`toiljs dev`**, the host is an in-process, in-memory emulator. It reads the catalog and stands up all seven families in memory. This is a development store: single process, single tenant, and cleared when you restart. It exists so you can build and test against real ToilDB behavior with no services to run.
129
+ - On the **Dacely edge**, the host is the real ToilDB, backed by a globally distributed ScyllaDB cluster. It reads the *same* catalog and serves the *same* operations, now durable and worldwide.
130
+
131
+ Because both sides read the same catalog, **the same code runs unchanged in dev and in production.** There is no connection string to swap and no provisioning step to run.
132
+
133
+ ```mermaid
134
+ flowchart TD
135
+ SRC["@database AppDb<br/>@collection static users: Documents...<br/>@data UserId / User"]
136
+ SRC -->|toilscript compile| WASM["server.wasm<br/>(with an embedded<br/>toildb catalog section)"]
137
+ WASM -->|toiljs dev reads the catalog| DEV["In-memory dev store<br/>(one process, cleared on restart)"]
138
+ WASM -->|edge reads the same catalog| EDGE[("ToilDB on the Dacely edge<br/>(ScyllaDB, worldwide, durable)")]
139
+ ```
140
+
141
+ ## Gotchas
142
+
143
+ - **Declare collections as `static` fields with no initializer.** Do not try to `new` a collection or assign a handle yourself; the compiler owns that.
144
+ - **Every `@data` field needs a default, and value types must be default-constructible.** A missing default fails to compile.
145
+ - **Dev data is not durable.** The `toiljs dev` store lives in memory and resets on restart. Do not rely on it to persist across dev runs; that is what the edge is for.
146
+ - **Changing a `@data` type is a format change.** Reordering fields or changing a field's type changes the stored layout. Add new fields at the end, and use a migration when you evolve a stored type. See [data types](../backend/data.md).
147
+ - **Pick the family at the type.** The family is read from the collection's declared type, so `Counter<K>` versus `Documents<K, V>` is a real, load-bearing choice, not a hint. Revisit [Choosing a family](./README.md#choosing-a-family-the-decision-guide) if unsure.
148
+
149
+ ## Related
150
+
151
+ - [ToilDB overview](./README.md): the seven families and how to choose.
152
+ - [Documents](./documents.md): the general-purpose record family (a good first collection).
153
+ - [Data types (`@data`)](../backend/data.md): keys, values, and the codec.
154
+ - [Decorators](../concepts/decorators.md): `@database`, `@collection`, `@query`, `@action`, `@derive`.
155
+ - [@derive](../background/derive.md): recompute snapshots and run scans off the request path.
@@ -0,0 +1,216 @@
1
+ # Unique
2
+
3
+ The **Unique** family enforces that a value is claimed by only one owner across the entire world. It is how you make usernames, email addresses, and URL slugs one-of-a-kind, safely, even when two people try to grab the same one at the same instant.
4
+
5
+ ## What and why
6
+
7
+ A **Unique collection** maps a `@data` **claim key** (the thing that must be unique, like a username) to a `@data` **owner value** (who or what claimed it, like a user id). At any moment a claim key is either **unclaimed** or **owned by exactly one owner**. The family gives you three operations: look up who owns a key, claim a key, and release it.
8
+
9
+ Reach for Unique whenever a value must be globally singular:
10
+
11
+ - usernames or handles
12
+ - email addresses
13
+ - URL slugs or workspace names
14
+ - any "reserve this name for me" scenario
15
+
16
+ Why not just check a Documents record first? Because a check-then-write has a race: two requests can both check "is `alice` free?", both see yes, and both create it. Unique closes that gap. The claim is decided at the key's single **home** (see [consistency](./README.md#eventual-consistency-in-plain-words)), where claims are processed one at a time, so exactly one of the two racers wins.
17
+
18
+ Declare one by typing a `@collection` as `Unique<ClaimKey, OwnerValue>`:
19
+
20
+ ```ts
21
+ @data
22
+ class Username {
23
+ name: string = '';
24
+ constructor(name: string = '') { this.name = name; }
25
+ }
26
+
27
+ @data
28
+ class OwnerId {
29
+ userId: string = '';
30
+ constructor(userId: string = '') { this.userId = userId; }
31
+ }
32
+
33
+ @database
34
+ class AppDb {
35
+ @collection static usernames: Unique<Username, OwnerId>;
36
+ }
37
+ ```
38
+
39
+ ## The operations
40
+
41
+ `K` is the claim-key type, `V` the owner value type.
42
+
43
+ | Operation | Signature | Returns | Use it to |
44
+ | --- | --- | --- | --- |
45
+ | `lookup` | `lookup(key: K): V \| null` | the current owner, or `null` if unclaimed | find out who owns a name |
46
+ | `claim` | `claim(key: K, value: V): ClaimResult<V>` | a `ClaimResult` (see below) | try to take a name for an owner |
47
+ | `release` | `release(key: K, value: V): void` | nothing; **traps** if you are not the owner | give a name back |
48
+
49
+ `lookup` is a read, so it works in any function. `claim` and `release` are writes, so they need an **Action** (a `@post` route or an `@action`); see [Setup](./setup.md#how-access-is-gated-query-action-and-friends).
50
+
51
+ ### `ClaimResult`
52
+
53
+ `claim` returns a small object that tells you what happened:
54
+
55
+ ```ts
56
+ class ClaimResult<V> {
57
+ claimed: bool; // true if YOU own the key now
58
+ owner: V | null; // when claimed is false, who owns it instead
59
+ }
60
+ ```
61
+
62
+ - **`claimed == true`**: you own the key. This covers both a fresh claim and an **idempotent re-claim**: if you claim a key you already own (same owner value), you still get `true`, so retrying a claim is safe. `owner` is `null` in this case.
63
+ - **`claimed == false`**: someone else got there first. `owner` is their value, so you can tell the user "that name is taken."
64
+
65
+ ### `lookup`
66
+
67
+ `lookup` just reads the current owner without changing anything:
68
+
69
+ ```ts
70
+ const owner = AppDb.usernames.lookup(new Username('alice'));
71
+ if (owner == null) {
72
+ // 'alice' is free
73
+ } else {
74
+ // owner.userId currently holds 'alice'
75
+ }
76
+ ```
77
+
78
+ Note that `lookup` is subject to [eventual consistency](./README.md#eventual-consistency-in-plain-words): a claim made moments ago in another region may not show up in a far-away `lookup` yet. Do not use `lookup` as your uniqueness guard. `lookup` is for display ("this name is taken by ..."); the real guarantee comes from `claim`, which is decided at the home and cannot race.
79
+
80
+ ### `claim`
81
+
82
+ `claim` is the operation that actually enforces uniqueness. You pass the key and the owner value:
83
+
84
+ ```ts
85
+ const result = AppDb.usernames.claim(new Username('alice'), new OwnerId('u_123'));
86
+ if (result.claimed) {
87
+ // 'alice' is now yours
88
+ } else {
89
+ // taken; result.owner is the current owner
90
+ }
91
+ ```
92
+
93
+ Because claims are serialized at the key's home, this is race-safe. If two requests anywhere in the world call `claim('alice', ...)` at the same moment, the home applies them in order: the first gets `claimed: true`, the second gets `claimed: false` with the first as `owner`. There is no window where both win.
94
+
95
+ ### `release`
96
+
97
+ `release` gives a claim back so the key becomes available again. Only the **current owner** may release: you pass both the key and the owner value, and if that value is not the current owner, `release` **traps** (aborts the request). This prevents one user from releasing another user's name.
98
+
99
+ ```ts
100
+ AppDb.usernames.release(new Username('alice'), new OwnerId('u_123'));
101
+ ```
102
+
103
+ Release when a name is being changed or an account is deleted, so the name returns to the pool.
104
+
105
+ ## The claim / release lifecycle
106
+
107
+ A claim is a small state machine: unclaimed, owned, and back to unclaimed.
108
+
109
+ ```mermaid
110
+ stateDiagram-v2
111
+ [*] --> Unclaimed
112
+ Unclaimed --> Owned: claim(key, me) -> claimed: true
113
+ Owned --> Owned: claim(key, me) again -> claimed: true (idempotent)
114
+ Owned --> Owned: claim(key, someoneElse) -> claimed: false, owner: me
115
+ Owned --> Unclaimed: release(key, me)
116
+ ```
117
+
118
+ The key insight: **claiming is the guard, releasing is cleanup.** You claim to reserve, you release to free. A claim that is never released stays owned forever (there is no automatic expiry; if you want time-limited holds, that is what the [Capacity](./capacity.md) family's TTL reservations are for).
119
+
120
+ ## Worked example: reserving a username on signup
121
+
122
+ The safe signup pattern is: claim the username first, then create the account, and if creating the account fails, release the claim so the name is not stranded.
123
+
124
+ ```ts
125
+ import { Response } from 'toiljs/server/runtime';
126
+
127
+ @data
128
+ class Username {
129
+ name: string = '';
130
+ constructor(name: string = '') { this.name = name; }
131
+ }
132
+
133
+ @data
134
+ class OwnerId {
135
+ userId: string = '';
136
+ constructor(userId: string = '') { this.userId = userId; }
137
+ }
138
+
139
+ @data
140
+ class UserId {
141
+ id: string = '';
142
+ constructor(id: string = '') { this.id = id; }
143
+ }
144
+
145
+ @data
146
+ class User {
147
+ id: string = '';
148
+ username: string = '';
149
+ }
150
+
151
+ @data
152
+ class SignupInput {
153
+ username: string = '';
154
+ userId: string = '';
155
+ }
156
+
157
+ @database
158
+ class AppDb {
159
+ @collection static usernames: Unique<Username, OwnerId>;
160
+ @collection static users: Documents<UserId, User>;
161
+ }
162
+
163
+ @rest('signup')
164
+ class Signup {
165
+ // POST /signup (Action: may claim and write)
166
+ @post('/')
167
+ public signup(input: SignupInput): Response {
168
+ const owner = new OwnerId(input.userId);
169
+
170
+ // 1. Claim the username. This is the uniqueness guard.
171
+ const claim = AppDb.usernames.claim(new Username(input.username), owner);
172
+ if (!claim.claimed) {
173
+ return Response.text('username taken', 409);
174
+ }
175
+
176
+ // 2. Create the account. If this fails, undo the claim so the name is free.
177
+ const user = new User();
178
+ user.id = input.userId;
179
+ user.username = input.username;
180
+ if (!AppDb.users.create(new UserId(input.userId), user)) {
181
+ AppDb.usernames.release(new Username(input.username), owner);
182
+ return Response.text('could not create account', 409);
183
+ }
184
+
185
+ return Response.json(user.toJSON().toString());
186
+ }
187
+ }
188
+ ```
189
+
190
+ Two things make this correct:
191
+
192
+ - **The claim happens before the account write**, so uniqueness is decided up front.
193
+ - **The claim is released if the follow-up write fails**, so a failed signup does not permanently burn a username.
194
+
195
+ Because `claim` is idempotent for the same owner, a client that retries the whole request after a network hiccup does not get a false "taken" error: the second `claim('alice', u_123)` returns `claimed: true` again.
196
+
197
+ ## Consistency notes
198
+
199
+ - **`claim` and `release` are strongly consistent at the key's home.** Two callers can never both own the same key; the home serializes claims. This is the whole point of the family.
200
+ - **`lookup` is eventually consistent.** It reads a possibly-nearby copy, so a very recent claim from elsewhere may not appear yet. Never gate uniqueness on `lookup`; gate it on the result of `claim`.
201
+ - **Claims do not expire.** A claim stays until someone releases it. Use [Capacity](./capacity.md) if you need holds that auto-release after a timeout.
202
+
203
+ ## Gotchas
204
+
205
+ - **Do not use `lookup` as the uniqueness check.** Its answer can be stale. Call `claim` and trust its `claimed` flag.
206
+ - **Release on failure and on account deletion.** A claim you forget to release stays owned forever, quietly blocking that name.
207
+ - **`release` traps for a non-owner.** Pass the correct owner value, or you abort the request. If you are not sure you own it, `lookup` first (for display) but expect `release` to enforce ownership regardless.
208
+ - **The owner value is data you choose.** Store enough in it (a user id, say) to know who holds the claim, so you can display and release it later.
209
+
210
+ ## Related
211
+
212
+ - [ToilDB overview](./README.md): the seven families and how to choose.
213
+ - [Setup](./setup.md): declaring the collection and which function kinds may claim.
214
+ - [Documents](./documents.md): the account record you create alongside a claim.
215
+ - [Capacity](./capacity.md): time-limited holds that auto-release (a different kind of "reserve").
216
+ - [Data types (`@data`)](../backend/data.md): the claim key and owner value.