toiljs 0.0.85 → 0.0.87
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +5 -0
- package/README.md +2 -2
- package/build/cli/.tsbuildinfo +1 -1
- package/build/cli/index.js +303 -293
- package/build/compiler/.tsbuildinfo +1 -1
- package/build/compiler/config.d.ts +2 -0
- package/build/compiler/config.js +1 -0
- package/build/compiler/docs.js +8 -24
- package/build/compiler/generate.js +4 -2
- package/build/compiler/index.d.ts +1 -1
- package/build/compiler/index.js +69 -6
- package/build/compiler/toil-docs.generated.js +64 -22
- package/build/devserver/.tsbuildinfo +1 -1
- package/build/devserver/analytics/index.js +7 -3
- package/build/devserver/db/database.d.ts +4 -0
- package/build/devserver/db/database.js +43 -1
- package/build/devserver/db/index.d.ts +1 -1
- package/build/devserver/db/index.js +1 -1
- package/build/devserver/db/types.d.ts +3 -0
- package/build/devserver/db/types.js +2 -0
- package/build/devserver/runtime/module.js +4 -1
- package/docs/README.md +104 -65
- package/docs/auth/README.md +102 -0
- package/docs/auth/configuration.md +94 -0
- package/docs/auth/extending.md +202 -0
- package/docs/auth/how-it-works.md +138 -0
- package/docs/auth/usage.md +188 -0
- package/docs/backend/README.md +143 -0
- package/docs/backend/data.md +351 -0
- package/docs/backend/rest.md +402 -0
- package/docs/backend/rpc.md +226 -0
- package/docs/background/README.md +114 -0
- package/docs/background/daemons.md +230 -0
- package/docs/background/derive.md +179 -0
- package/docs/cli/README.md +377 -0
- package/docs/concepts/config.md +416 -0
- package/docs/concepts/decorators.md +127 -0
- package/docs/concepts/security.md +108 -0
- package/docs/concepts/tiers.md +166 -0
- package/docs/concepts/types.md +216 -0
- package/docs/database/README.md +143 -0
- package/docs/database/capacity.md +350 -0
- package/docs/database/counters.md +174 -0
- package/docs/database/documents.md +255 -0
- package/docs/database/events.md +307 -0
- package/docs/database/membership.md +246 -0
- package/docs/database/setup.md +155 -0
- package/docs/database/unique.md +216 -0
- package/docs/database/views.md +246 -0
- package/docs/frontend/README.md +101 -0
- package/docs/frontend/data-fetching.md +243 -0
- package/docs/frontend/images.md +148 -0
- package/docs/frontend/metadata.md +344 -0
- package/docs/frontend/rendering.md +236 -0
- package/docs/frontend/routing.md +344 -0
- package/docs/frontend/scripts.md +118 -0
- package/docs/frontend/search.md +191 -0
- package/docs/frontend/styling.md +147 -0
- package/docs/getting-started/README.md +81 -0
- package/docs/getting-started/create-project.md +131 -0
- package/docs/getting-started/deploy.md +101 -0
- package/docs/getting-started/first-app.md +215 -0
- package/docs/getting-started/installation.md +106 -0
- package/docs/getting-started/migrating.md +125 -0
- package/docs/getting-started/project-structure.md +163 -0
- package/docs/introduction/README.md +41 -0
- package/docs/introduction/design-principles.md +55 -0
- package/docs/introduction/distributed.md +74 -0
- package/docs/introduction/how-it-works.md +74 -0
- package/docs/introduction/hyperscale.md +51 -0
- package/docs/introduction/modern-stack.md +36 -0
- package/docs/introduction/vs-other-frameworks.md +48 -0
- package/docs/introduction/why-toil.md +93 -0
- package/docs/llms.txt +90 -0
- package/docs/realtime/README.md +102 -0
- package/docs/realtime/channels.md +211 -0
- package/docs/realtime/streams.md +369 -0
- package/docs/services/README.md +60 -0
- package/docs/services/analytics.md +268 -0
- package/docs/services/caching.md +175 -0
- package/docs/services/cookies.md +235 -0
- package/docs/services/crypto.md +209 -0
- package/docs/services/email.md +289 -0
- package/docs/services/environment.md +141 -0
- package/docs/services/ratelimit.md +174 -0
- package/docs/services/time.md +85 -0
- package/examples/basic/client/routes/analytics.tsx +13 -12
- package/examples/basic/server/models/SiteAnalytics.ts +29 -17
- package/examples/basic/server/routes/Analytics.ts +15 -17
- package/examples/basic/server/routes/UserId.ts +22 -0
- package/package.json +15 -11
- package/scripts/gen-toil-docs.mjs +24 -35
- package/server/auth/AuthController.ts +336 -0
- package/server/auth/AuthUser.ts +23 -0
- package/server/auth/index.ts +16 -0
- package/server/globals/auth.ts +31 -0
- package/server/globals/userid.ts +107 -0
- package/src/compiler/config.ts +13 -0
- package/src/compiler/docs.ts +16 -33
- package/src/compiler/generate.ts +6 -2
- package/src/compiler/index.ts +114 -6
- package/src/compiler/toil-docs.generated.ts +64 -22
- package/src/devserver/analytics/index.ts +10 -4
- package/src/devserver/db/database.ts +67 -1
- package/src/devserver/db/index.ts +1 -0
- package/src/devserver/db/types.ts +13 -0
- package/src/devserver/runtime/module.ts +7 -0
- package/test/analytics-dev.test.ts +2 -1
- package/test/devserver-database.test.ts +113 -0
- package/docs/auth-todo.md +0 -149
- package/docs/auth.md +0 -322
- package/docs/caching.md +0 -115
- package/docs/cli.md +0 -17
- package/docs/client.md +0 -39
- package/docs/cookies.md +0 -457
- package/docs/crypto.md +0 -130
- package/docs/daemon.md +0 -123
- package/docs/data.md +0 -131
- package/docs/derive.md +0 -159
- package/docs/email.md +0 -326
- package/docs/environment.md +0 -97
- package/docs/getting-started.md +0 -128
- package/docs/index.md +0 -30
- package/docs/ratelimit.md +0 -95
- package/docs/routing.md +0 -259
- package/docs/rpc.md +0 -149
- package/docs/server.md +0 -61
- package/docs/ssr.md +0 -632
- package/docs/streams.md +0 -178
- package/docs/styling.md +0 -22
- package/docs/tiers.md +0 -133
- package/docs/time.md +0 -43
|
@@ -0,0 +1,246 @@
|
|
|
1
|
+
# Views (materialized views)
|
|
2
|
+
|
|
3
|
+
A `View` collection is a **precomputed, read-optimized result** that you publish
|
|
4
|
+
once and read many times. Reads are a single cheap keyed lookup, no scanning, no
|
|
5
|
+
recomputing.
|
|
6
|
+
|
|
7
|
+
## What and why
|
|
8
|
+
|
|
9
|
+
A "materialized view" is a fancy name for a simple idea: instead of computing an
|
|
10
|
+
answer every time someone asks, you compute it **once**, store the finished
|
|
11
|
+
answer, and just hand it out on each request. "Materialized" means the result is
|
|
12
|
+
made real and saved (as opposed to computed on the fly). "View" means it is a
|
|
13
|
+
read-friendly projection of your underlying data.
|
|
14
|
+
|
|
15
|
+
Think of a leaderboard. Computing "the top 10 players by score" means scanning
|
|
16
|
+
every player and sorting. You do not want to do that on every page load. So you
|
|
17
|
+
compute it in the background, save the finished top-10 list as a view, and every
|
|
18
|
+
page load just reads that saved list. Fast, cheap, and it does not get slower as
|
|
19
|
+
you add players.
|
|
20
|
+
|
|
21
|
+
Use a `View` when:
|
|
22
|
+
|
|
23
|
+
- A page needs a result that is **expensive to compute** (a scan, a sort, a fold
|
|
24
|
+
over many rows).
|
|
25
|
+
- That result is **read far more often than it changes** (a home page, a feed, a
|
|
26
|
+
leaderboard, a "latest N" list, a rendered summary).
|
|
27
|
+
|
|
28
|
+
The rule of thumb: if a route wants "the newest N of something" or "the top N of
|
|
29
|
+
something", that is a scan, and scans are barred on the request path. A `View`
|
|
30
|
+
is how you serve that result without scanning per request.
|
|
31
|
+
|
|
32
|
+
```mermaid
|
|
33
|
+
flowchart LR
|
|
34
|
+
subgraph off["Off the request path (@derive)"]
|
|
35
|
+
S[("Source data<br/>events / counters / records")] --> D["@derive: scan + compute"]
|
|
36
|
+
D --> P["publish(key, result)"]
|
|
37
|
+
end
|
|
38
|
+
P --> V[("View")]
|
|
39
|
+
subgraph on["On the request path (@get)"]
|
|
40
|
+
G["view.get(key)"] --> V
|
|
41
|
+
end
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
## The type
|
|
45
|
+
|
|
46
|
+
A `View` has two type parameters: the **key** (which view) and the **value**
|
|
47
|
+
(the precomputed result).
|
|
48
|
+
|
|
49
|
+
```ts
|
|
50
|
+
View<K, V>
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
- `K` is the key type: it picks *which* view to read or publish. For a single
|
|
54
|
+
global leaderboard, the key can be a fixed constant like `'main'`.
|
|
55
|
+
- `V` is the value type: the finished result you serve. Both are
|
|
56
|
+
[`@data`](../concepts/types.md) classes.
|
|
57
|
+
|
|
58
|
+
Declare it as a `@collection` field inside a `@database` class, alongside the
|
|
59
|
+
sources it is built from:
|
|
60
|
+
|
|
61
|
+
```ts
|
|
62
|
+
@database
|
|
63
|
+
class BoardDb {
|
|
64
|
+
@collection static scores: Events<GameKey, ScoreEvent>; // a source
|
|
65
|
+
@collection static board: View<GameKey, Leaderboard>; // the view
|
|
66
|
+
}
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
## Operations
|
|
70
|
+
|
|
71
|
+
A `View` has exactly two operations you use directly (plus `require`, a
|
|
72
|
+
convenience wrapper). Exact signatures:
|
|
73
|
+
|
|
74
|
+
| Operation | Signature | What it does |
|
|
75
|
+
| --- | --- | --- |
|
|
76
|
+
| `get` | `get(key: K): V \| null` | Read the published view, or `null` if nothing has been published yet. |
|
|
77
|
+
| `require` | `require(key: K): V` | Like `get`, but traps (aborts the request) if nothing is published. |
|
|
78
|
+
| `publish` | `publish(key: K, value: V): void` | Overwrite the view for `key` with a fresh result. |
|
|
79
|
+
|
|
80
|
+
### `get`
|
|
81
|
+
|
|
82
|
+
A plain keyed read. It is cheap and is allowed from **any** handler, including a
|
|
83
|
+
read-only `@get` route.
|
|
84
|
+
|
|
85
|
+
```ts
|
|
86
|
+
const board = BoardDb.board.get(new GameKey('main'));
|
|
87
|
+
if (board == null) {
|
|
88
|
+
// Nothing published yet (e.g. brand-new game). Serve an empty default.
|
|
89
|
+
return new Leaderboard();
|
|
90
|
+
}
|
|
91
|
+
return board;
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
Always handle the `null` case: until a `@derive` (or `@job`) has published at
|
|
95
|
+
least once, `get` returns `null`.
|
|
96
|
+
|
|
97
|
+
### `publish`
|
|
98
|
+
|
|
99
|
+
Overwrites the stored view with a new value. This is how the view gets its
|
|
100
|
+
content.
|
|
101
|
+
|
|
102
|
+
```ts
|
|
103
|
+
BoardDb.board.publish(new GameKey('main'), freshBoard);
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
`publish` is **restricted**: you may only call it from a
|
|
107
|
+
[`@derive`](../background/derive.md) or a `@job` (a background task), never from a
|
|
108
|
+
request handler (`@get`/`@post`/...). The compiler enforces this, and so does the
|
|
109
|
+
edge at runtime. The reason: a view is meant to be maintained off the request
|
|
110
|
+
path from the source of truth, not written ad hoc by whichever request happens to
|
|
111
|
+
run.
|
|
112
|
+
|
|
113
|
+
### How `publish` differs from a Documents write
|
|
114
|
+
|
|
115
|
+
A `View`'s `publish` looks like writing a value, but it is not the same as a
|
|
116
|
+
[Documents](./documents.md) `patch`/`create`. The differences matter:
|
|
117
|
+
|
|
118
|
+
| | `Documents` write (`create` / `patch`) | `View` `publish` |
|
|
119
|
+
| --- | --- | --- |
|
|
120
|
+
| Who may call it | Any action handler (`@post`, ...) | Only a `@derive` or `@job` |
|
|
121
|
+
| Meaning | The record IS the source of truth | The view is a **copy** derived from a source |
|
|
122
|
+
| Version control | You may version-check (optimistic concurrency) | The host assigns the version; a later publish always wins (last writer wins) |
|
|
123
|
+
| Read path | Standard keyed read | A read-optimized fast path (views are heavily read) |
|
|
124
|
+
| Who owns correctness | You (each writer edits the true value) | The derive (it recomputes from the source, so the view always converges) |
|
|
125
|
+
|
|
126
|
+
In short: a Document is the truth; a View is a saved snapshot of a computation
|
|
127
|
+
over the truth. If the view is ever wrong or stale, the derive just recomputes and
|
|
128
|
+
republishes it. You never hand-edit a view from a route.
|
|
129
|
+
|
|
130
|
+
## Automatic maintenance with `@derive`
|
|
131
|
+
|
|
132
|
+
You rarely call `publish` by hand. The normal pattern is a
|
|
133
|
+
[`@derive`](../background/derive.md): a method on your `@database` class that
|
|
134
|
+
reads the sources, builds the value, and publishes it. The runtime runs it for
|
|
135
|
+
you:
|
|
136
|
+
|
|
137
|
+
- **Right after a write to a source.** When a request writes one of the
|
|
138
|
+
database's source collections (an `append`, a `counter.add`, a record write),
|
|
139
|
+
the database's derives run right after the response is produced. So the view
|
|
140
|
+
reflects the new data on the next read.
|
|
141
|
+
- **On box load.** When the server starts or reloads, the views are rebuilt from
|
|
142
|
+
their sources before the first read is served.
|
|
143
|
+
|
|
144
|
+
You never call a derive yourself, and a derive's own `publish` never re-triggers
|
|
145
|
+
it. See [`@derive`](../background/derive.md) for the full rules.
|
|
146
|
+
|
|
147
|
+
## Worked example: a leaderboard
|
|
148
|
+
|
|
149
|
+
A game appends score events; a derive folds them into a top-10 list; a route
|
|
150
|
+
serves that list with a single `get`.
|
|
151
|
+
|
|
152
|
+
```ts
|
|
153
|
+
import { ScoreEvent } from '../models/ScoreEvent';
|
|
154
|
+
import { Leaderboard } from '../models/Leaderboard';
|
|
155
|
+
import { GameKey } from '../models/GameKey';
|
|
156
|
+
import { NewScore } from '../models/NewScore';
|
|
157
|
+
|
|
158
|
+
@database
|
|
159
|
+
class BoardDb {
|
|
160
|
+
// The source of truth: every score, appended as it happens.
|
|
161
|
+
@collection static scores: Events<GameKey, ScoreEvent>;
|
|
162
|
+
// The materialized view: the current top entries, ready to serve.
|
|
163
|
+
@collection static board: View<GameKey, Leaderboard>;
|
|
164
|
+
|
|
165
|
+
// Off the request path: scan the recent scores, sort, publish the top 10.
|
|
166
|
+
@derive
|
|
167
|
+
rebuild(): void {
|
|
168
|
+
const key = new GameKey('main');
|
|
169
|
+
const recent = BoardDb.scores.latest(key, 500); // a scan, allowed in a derive
|
|
170
|
+
const board = new Leaderboard();
|
|
171
|
+
board.entries = topTen(recent); // your own sort/aggregate
|
|
172
|
+
BoardDb.board.publish(key, board);
|
|
173
|
+
}
|
|
174
|
+
}
|
|
175
|
+
|
|
176
|
+
@rest('leaderboard')
|
|
177
|
+
class LeaderboardRoutes {
|
|
178
|
+
// GET reads the precomputed view: one keyed read, no scan.
|
|
179
|
+
@get('/')
|
|
180
|
+
public top(): Leaderboard {
|
|
181
|
+
const board = BoardDb.board.get(new GameKey('main'));
|
|
182
|
+
return board == null ? new Leaderboard() : board;
|
|
183
|
+
}
|
|
184
|
+
|
|
185
|
+
// POST records a score. The @derive rebuilds `board` right after.
|
|
186
|
+
@post('/')
|
|
187
|
+
public submit(input: NewScore): Leaderboard {
|
|
188
|
+
BoardDb.scores.append(new GameKey('main'), new ScoreEvent(input.player, input.score));
|
|
189
|
+
return new Leaderboard(); // ack; the GET serves the updated board from the view
|
|
190
|
+
}
|
|
191
|
+
}
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
The models:
|
|
195
|
+
|
|
196
|
+
```ts
|
|
197
|
+
@data
|
|
198
|
+
export class ScoreEvent {
|
|
199
|
+
player: string = '';
|
|
200
|
+
score: u64 = 0;
|
|
201
|
+
}
|
|
202
|
+
|
|
203
|
+
@data
|
|
204
|
+
export class Leaderboard {
|
|
205
|
+
entries: ScoreEvent[] = [];
|
|
206
|
+
}
|
|
207
|
+
```
|
|
208
|
+
|
|
209
|
+
A "latest N" view is the same shape: the derive calls `latest(key, N)` and
|
|
210
|
+
publishes the list. See the [Events](./events.md) page for that variant end to
|
|
211
|
+
end.
|
|
212
|
+
|
|
213
|
+
## Consistency
|
|
214
|
+
|
|
215
|
+
- **Last writer wins.** The host assigns each `publish` a version, and a later
|
|
216
|
+
publish always supersedes an earlier one. Because a derive recomputes the whole
|
|
217
|
+
view from the source of truth, the view converges to a correct snapshot; you do
|
|
218
|
+
not need to coordinate concurrent publishes.
|
|
219
|
+
- **A view can be briefly stale.** ToilDB is worldwide, and a view is published
|
|
220
|
+
at its home then copied to other regions in the background (asynchronous
|
|
221
|
+
replication). A read from a far region can lag the newest publish by a moment.
|
|
222
|
+
This is usually fine for the things views hold (feeds, leaderboards, summaries):
|
|
223
|
+
a leaderboard that is a second behind is still a good leaderboard.
|
|
224
|
+
- **`get` returns `null` until the first publish.** Always default the empty case.
|
|
225
|
+
|
|
226
|
+
## Gotchas
|
|
227
|
+
|
|
228
|
+
- **`publish` is derive/job only.** You cannot publish from a route. If you need
|
|
229
|
+
to update a view in response to a request, write the *source* (append an event,
|
|
230
|
+
add to a counter) from the action and let the `@derive` republish.
|
|
231
|
+
- **Handle `null` from `get`.** A never-published view reads as `null`, not as an
|
|
232
|
+
empty value. Return a sensible default.
|
|
233
|
+
- **A view is a copy, not the truth.** Never store data *only* in a view. Keep
|
|
234
|
+
the real data in a source family (Documents, Events, Counters) and treat the
|
|
235
|
+
view as a disposable, recomputable projection.
|
|
236
|
+
- **Keep views bounded.** A view value is read whole on every request, so build
|
|
237
|
+
it from a bounded read (top N, latest N, a total), not the entire history.
|
|
238
|
+
|
|
239
|
+
## Related
|
|
240
|
+
|
|
241
|
+
- [`@derive`](../background/derive.md): the normal way a view is maintained.
|
|
242
|
+
- [Events](./events.md): the append-only log a view is often folded from.
|
|
243
|
+
- [Counters](./counters.md): another common source for a view (totals).
|
|
244
|
+
- [Documents](./documents.md): mutable source-of-truth records (and how a write
|
|
245
|
+
differs from a publish).
|
|
246
|
+
- [Data types (`@data`)](../concepts/types.md): how view keys and values are stored.
|
|
@@ -0,0 +1,101 @@
|
|
|
1
|
+
# Frontend
|
|
2
|
+
|
|
3
|
+
Your toiljs frontend is a React app with file-based routing that runs in the browser, and can also be rendered ahead of time on the server for a fast first paint and good SEO.
|
|
4
|
+
|
|
5
|
+
If you have written React before, everything here is familiar React: components, hooks, JSX. toiljs adds the parts a plain React app makes you wire up yourself: a router, data loading, `<head>` and SEO management, an image component, and a typed client for calling your backend. You get them for free and you do not import most of them, they live on a global called `Toil`.
|
|
6
|
+
|
|
7
|
+
## What "frontend" means here
|
|
8
|
+
|
|
9
|
+
A toiljs project has three top-level folders. The frontend is the first two:
|
|
10
|
+
|
|
11
|
+
- **`client/`** is your React app: pages, components, and styles. This is what runs in the user's browser.
|
|
12
|
+
- **`shared/`** is a typed bridge that toiljs generates for you. It lets the browser call your backend with full type safety (see [Fetching data](./data-fetching.md)).
|
|
13
|
+
- **`server/`** is your backend. It compiles to WebAssembly and runs on the edge. That is a separate section (see [Backend](../backend/README.md)).
|
|
14
|
+
|
|
15
|
+
Inside `client/`, the important pieces are:
|
|
16
|
+
|
|
17
|
+
| Path | What it is |
|
|
18
|
+
| --- | --- |
|
|
19
|
+
| `client/toil.tsx` | The entry file. It imports your routes and global styles and mounts the app. |
|
|
20
|
+
| `client/routes/` | Your pages. One file per URL (file-based routing). See [Routing](./routing.md). |
|
|
21
|
+
| `client/layout.tsx` | The root layout that wraps every page (a header, a footer, and so on). |
|
|
22
|
+
| `client/components/` | Your own reusable React components. |
|
|
23
|
+
| `client/styles/` | Your CSS. See [Styling](./styling.md). |
|
|
24
|
+
| `client/public/` | Static files served as-is (`favicon.ico`, images, `robots.txt`). |
|
|
25
|
+
|
|
26
|
+
The entry file is tiny, and you rarely touch it:
|
|
27
|
+
|
|
28
|
+
```tsx
|
|
29
|
+
// client/toil.tsx
|
|
30
|
+
import { routes, layout, notFound, globalError, slots } from 'toiljs/routes';
|
|
31
|
+
import './styles/main.css';
|
|
32
|
+
|
|
33
|
+
Toil.mount(routes, layout, notFound, globalError, slots);
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
`toiljs/routes` is a virtual module: the compiler scans `client/routes/` and generates the route table for you, so you never hand-maintain a list of pages. Adding a file under `client/routes/` adds a page.
|
|
37
|
+
|
|
38
|
+
## SPA, but with a server head start
|
|
39
|
+
|
|
40
|
+
By default a toiljs app is a single-page application (SPA). The word "SPA" means the browser loads one HTML shell, then JavaScript builds every page and swaps between them without a full reload. That makes navigation instant, but it has a classic weakness: the very first load shows a blank page until the JavaScript runs, and simple crawlers that do not run JavaScript see nothing useful.
|
|
41
|
+
|
|
42
|
+
toiljs closes that gap in two ways, both optional and both explained in [Rendering and SSR](./rendering.md):
|
|
43
|
+
|
|
44
|
+
1. **Build-time prerendering.** At build time toiljs bakes each route's `<head>` (title, description, Open Graph, and so on) into real HTML, so crawlers and link-preview bots see correct metadata even without running your code.
|
|
45
|
+
2. **Edge server-side rendering (SSR).** For routes you opt in with `export const ssr = true`, the edge fills in real first-paint HTML for the page body, then the browser "hydrates" it (attaches React to the already-drawn markup instead of redrawing it).
|
|
46
|
+
|
|
47
|
+
Here is the full life of one request, from a cold link click to a warm client-side app:
|
|
48
|
+
|
|
49
|
+
```mermaid
|
|
50
|
+
sequenceDiagram
|
|
51
|
+
participant U as User's browser
|
|
52
|
+
participant E as Dacely edge
|
|
53
|
+
U->>E: GET /some-page
|
|
54
|
+
E-->>U: HTML (baked head + first-paint body if ssr=true)
|
|
55
|
+
Note over U: First paint appears immediately
|
|
56
|
+
U->>E: fetch the JS bundle + route chunk
|
|
57
|
+
Note over U: React "hydrates": attaches to the existing HTML
|
|
58
|
+
Note over U: Page is now interactive
|
|
59
|
+
U->>U: Click a link -> client-side navigation (no reload)
|
|
60
|
+
Note over U: Only the new route's data + chunk are fetched
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
The key idea: the server gets you a correct first paint fast, and from then on the app runs entirely in the browser, fetching only small route chunks and data as you navigate.
|
|
64
|
+
|
|
65
|
+
## The `Toil` global
|
|
66
|
+
|
|
67
|
+
Most of the client API lives on a global object called `Toil`, so route files need no imports for the common things. A few examples you will meet across these pages:
|
|
68
|
+
|
|
69
|
+
```tsx
|
|
70
|
+
Toil.Link // a client-side navigation link
|
|
71
|
+
Toil.NavLink // a Link that knows when it is "active"
|
|
72
|
+
Toil.useParams() // read dynamic URL params, e.g. { id } for /blog/[id]
|
|
73
|
+
Toil.useLoaderData // read data your route's loader fetched
|
|
74
|
+
Toil.Image // an <img> that avoids layout shift and lazy-loads
|
|
75
|
+
Toil.useHead // set the <title> and <meta> tags
|
|
76
|
+
Server.REST.* // the typed fetch client for your backend
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
`Server` is also global (it is the typed backend client, see [Fetching data](./data-fetching.md)). Everything on `Toil` is fully typed: your editor autocompletes it, because toiljs generates a `toil-env.d.ts` that maps `Toil` onto the `toiljs/client` package.
|
|
80
|
+
|
|
81
|
+
The same fast data utilities your backend uses are available in client code too, as bare globals with no import: `FastMap` and `FastSet` (high-performance map and set collections), and `DataWriter` / `DataReader` (a compact binary codec for encoding and decoding buffers). They are handed to you the same way `Toil` and `Server` are, so you can write `new DataWriter()` straight in a component. See [Data types](../backend/data.md) for the codec and when to reach for it.
|
|
82
|
+
|
|
83
|
+
## The frontend pages
|
|
84
|
+
|
|
85
|
+
Read them in roughly this order:
|
|
86
|
+
|
|
87
|
+
- **[Routing](./routing.md)**: turn files into URLs. Index, nested, and dynamic pages; layouts and templates; active links; and navigating in code.
|
|
88
|
+
- **[Rendering and SSR](./rendering.md)**: what renders on the server versus in the browser, how hydration works, and the current SSR limitations.
|
|
89
|
+
- **[Styling](./styling.md)**: plain CSS, preprocessors (Sass / Less / Stylus), and Tailwind.
|
|
90
|
+
- **[Images](./images.md)**: the `Toil.Image` component, automatic blur placeholders, and how it stops layout shift.
|
|
91
|
+
- **[Metadata and SEO](./metadata.md)**: set the page title, description, and social-share tags per route.
|
|
92
|
+
- **[Fetching data](./data-fetching.md)**: call your backend with the generated typed clients, submit forms, and read who is logged in.
|
|
93
|
+
- **[Scripts](./scripts.md)**: load external or inline `<script>` tags with a loading strategy, using `Toil.Script`.
|
|
94
|
+
- **[Search](./search.md)**: the built-in, statically-baked page search and command palette (`usePageSearch`).
|
|
95
|
+
|
|
96
|
+
## Related
|
|
97
|
+
|
|
98
|
+
- [Getting started](../getting-started/README.md): install toiljs and create a project.
|
|
99
|
+
- [Project structure](../getting-started/project-structure.md): the full folder layout.
|
|
100
|
+
- [Backend overview](../backend/README.md): the `server/` side your frontend talks to.
|
|
101
|
+
- [The CLI](../cli/README.md): `toiljs dev`, `toiljs build`, and every flag.
|
|
@@ -0,0 +1,243 @@
|
|
|
1
|
+
# Fetching data
|
|
2
|
+
|
|
3
|
+
Your React frontend and your toiljs backend live in one project, so toiljs generates a **typed client** that lets the browser call your server with full type safety and no hand-written `fetch` boilerplate. This page covers loading data for a page, calling the backend directly, submitting forms, and reading who is logged in.
|
|
4
|
+
|
|
5
|
+
## The generated `Server` client
|
|
6
|
+
|
|
7
|
+
When you build the server, toiljs writes a file at `shared/server.ts` that contains your `@data` classes and a typed description of every backend endpoint. Importing anything from `shared/server` attaches the runtime clients to a global called `Server`. From then on you call your backend through `Server`, fully typed, with editor autocomplete.
|
|
8
|
+
|
|
9
|
+
There are two surfaces under `Server`:
|
|
10
|
+
|
|
11
|
+
- **`Server.REST.<controller>.<route>(args)`**: a real, typed `fetch` client for your `@rest` HTTP controllers. This is the working, recommended way to call the backend today.
|
|
12
|
+
- **`Server.<service>.<method>(args)` and `Server.<remote>(args)`**: the typed RPC surface for `@service` / `@remote` functions.
|
|
13
|
+
|
|
14
|
+
Both are generated from your server code, so if you rename a route or change an argument type, the call site is a compile error until you fix it.
|
|
15
|
+
|
|
16
|
+
## Calling a REST endpoint
|
|
17
|
+
|
|
18
|
+
`Server.REST` mirrors your `@rest` controllers. If your backend has a `players` controller with routes on it, you call them like this:
|
|
19
|
+
|
|
20
|
+
```tsx
|
|
21
|
+
import { NewPlayer, ScoreDelta } from 'shared/server';
|
|
22
|
+
|
|
23
|
+
// POST /players with a typed @data body -> typed Promise<Player>
|
|
24
|
+
const player = await Server.REST.players.create({ body: new NewPlayer('Ada') });
|
|
25
|
+
|
|
26
|
+
// POST /players/:id/score with a path param AND a body
|
|
27
|
+
const updated = await Server.REST.players.addScore({
|
|
28
|
+
params: { id: 1 },
|
|
29
|
+
body: new ScoreDelta(5n),
|
|
30
|
+
});
|
|
31
|
+
|
|
32
|
+
// GET /leaderboard -> typed Promise<Standings>
|
|
33
|
+
const board = await Server.REST.leaderboard.top();
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
The single argument is an object with up to four optional parts:
|
|
37
|
+
|
|
38
|
+
| Key | What it is |
|
|
39
|
+
| --- | --- |
|
|
40
|
+
| `params` | Path parameters, e.g. `{ id: 1 }` for a `/players/:id` route. |
|
|
41
|
+
| `body` | The request body, usually a `@data` class instance. |
|
|
42
|
+
| `query` | Query-string values. |
|
|
43
|
+
| `headers` | Extra request headers. |
|
|
44
|
+
|
|
45
|
+
Return values are typed and decoded for you. A route that returns a `@data` type hands you the parsed class instance. A route that returns a raw `Response` hands you the raw fetch `Response`, so you can inspect the status and headers yourself:
|
|
46
|
+
|
|
47
|
+
```tsx
|
|
48
|
+
// This route returns a Response, so you get the raw fetch Response.
|
|
49
|
+
const res = await Server.REST.players.get({ params: { id: 1 } });
|
|
50
|
+
if (!res.ok) {
|
|
51
|
+
console.log('status', res.status);
|
|
52
|
+
} else {
|
|
53
|
+
const p = await res.json();
|
|
54
|
+
}
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
`@data` classes are the typed values that cross the wire. You import them from `shared/server` and construct them normally (`new NewPlayer('Ada')`). See [Data types](../backend/data.md) for how they are defined on the server.
|
|
58
|
+
|
|
59
|
+
### Handling errors
|
|
60
|
+
|
|
61
|
+
A failed call throws. The global `parseError` helper turns any caught value into a readable message, which is handy in a `catch`:
|
|
62
|
+
|
|
63
|
+
```tsx
|
|
64
|
+
try {
|
|
65
|
+
const board = await Server.REST.leaderboard.top();
|
|
66
|
+
} catch (err) {
|
|
67
|
+
console.error(parseError(err));
|
|
68
|
+
}
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
## Loading data for a page (loaders)
|
|
72
|
+
|
|
73
|
+
Calling the backend from a button handler (as above) is fine for actions. For the data a page needs to *render*, use a route **loader** instead of a `useEffect`. A loader runs on navigation, in parallel with loading the page's code, and the page suspends (showing its `loading.tsx`) until the data is ready:
|
|
74
|
+
|
|
75
|
+
```tsx
|
|
76
|
+
// client/routes/blog/[id].tsx
|
|
77
|
+
export const loader = async ({ params }: Toil.LoaderArgs) => {
|
|
78
|
+
return Server.REST.blog.get({ params: { id: params.id } });
|
|
79
|
+
};
|
|
80
|
+
|
|
81
|
+
export default function BlogPost() {
|
|
82
|
+
const post = Toil.useLoaderData<typeof loader>();
|
|
83
|
+
return <article><h1>{post.title}</h1></article>;
|
|
84
|
+
}
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
`useLoaderData<typeof loader>()` is fully typed from the loader's return. This keeps data fetching declarative and out of effects, and it is what lets server-rendered pages seed the browser with the server's data so hydration stays clean.
|
|
88
|
+
|
|
89
|
+
### Caching and revalidating loader data
|
|
90
|
+
|
|
91
|
+
Loader results are cached by URL. You control how long with `export const revalidate`:
|
|
92
|
+
|
|
93
|
+
| `revalidate` value | Behavior |
|
|
94
|
+
| --- | --- |
|
|
95
|
+
| `0` (default) | Re-run the loader on every navigation to the route. |
|
|
96
|
+
| a number `n` | Reuse cached data for `n` seconds, then refetch. |
|
|
97
|
+
| `false` | Cache until you invalidate it manually. |
|
|
98
|
+
|
|
99
|
+
After a mutation (say you just POSTed a new comment), refresh the current page's data with `revalidate()` or the router:
|
|
100
|
+
|
|
101
|
+
```tsx
|
|
102
|
+
const router = Toil.useRouter();
|
|
103
|
+
await Server.REST.comments.add({ body: new NewComment(text) });
|
|
104
|
+
router.revalidate(); // refetch the active route's loader
|
|
105
|
+
// or target another route: router.revalidate('/posts');
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
`router.refresh()` re-runs the current loader and clears the cache; `router.revalidate(href)` invalidates a specific route.
|
|
109
|
+
|
|
110
|
+
### Invalidating loader data by hand
|
|
111
|
+
|
|
112
|
+
`revalidate()` and the `<Toil.Form>` `revalidate` prop are both built on one lower-level primitive: `Toil.invalidateLoaderData(href?)`. It drops cached loader data so the next render refetches it, but it does **not** trigger the re-render itself (that is the difference from `revalidate()`, which does both):
|
|
113
|
+
|
|
114
|
+
- `invalidateLoaderData()` with no argument clears every route's cached data.
|
|
115
|
+
- `invalidateLoaderData('/posts')` clears just that one route's entry.
|
|
116
|
+
|
|
117
|
+
You rarely need it directly; `revalidate()` (which pairs it with a re-render) is what you usually want. Reach for the primitive when you want to mark data stale *now* but let a later navigation do the actual refetch:
|
|
118
|
+
|
|
119
|
+
```tsx
|
|
120
|
+
// After a background sync finished, mark the dashboard stale so its
|
|
121
|
+
// next visit refetches, without disturbing the page the user is on.
|
|
122
|
+
Toil.invalidateLoaderData('/dashboard');
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
Signature: `invalidateLoaderData(href?: string): void`.
|
|
126
|
+
|
|
127
|
+
## Forms
|
|
128
|
+
|
|
129
|
+
For the common "submit a form, then refresh the page's data" loop, use `Toil.Form`. It runs an async action on submit (no page reload), tracks pending and error state, and revalidates the route's loader data on success:
|
|
130
|
+
|
|
131
|
+
```tsx
|
|
132
|
+
import { NewMessage } from 'shared/server';
|
|
133
|
+
|
|
134
|
+
export default function Guestbook() {
|
|
135
|
+
const entries = Toil.useLoaderData<typeof loader>();
|
|
136
|
+
|
|
137
|
+
const sign = async (data: FormData) => {
|
|
138
|
+
const author = String(data.get('author'));
|
|
139
|
+
const message = String(data.get('message'));
|
|
140
|
+
await Server.REST.guestbook.sign({ body: new NewMessage(author, message) });
|
|
141
|
+
};
|
|
142
|
+
|
|
143
|
+
return (
|
|
144
|
+
<Toil.Form action={sign} resetOnSuccess>
|
|
145
|
+
{({ pending }) => (
|
|
146
|
+
<>
|
|
147
|
+
<input name="author" />
|
|
148
|
+
<input name="message" />
|
|
149
|
+
<button disabled={pending}>Sign</button>
|
|
150
|
+
</>
|
|
151
|
+
)}
|
|
152
|
+
</Toil.Form>
|
|
153
|
+
);
|
|
154
|
+
}
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
Key `Toil.Form` props:
|
|
158
|
+
|
|
159
|
+
| Prop | Default | What it does |
|
|
160
|
+
| --- | --- | --- |
|
|
161
|
+
| `action` | (required) | Runs on submit, receiving the form's `FormData`. May be async. |
|
|
162
|
+
| `revalidate` | `true` (current route) | Which loader data to refetch after a successful submit. |
|
|
163
|
+
| `resetOnSuccess` | `false` | Clear the form fields after success. |
|
|
164
|
+
| `onSuccess` / `onError` | | Callbacks for the two outcomes. |
|
|
165
|
+
|
|
166
|
+
Passing a **render function** as `children` gives you live submit state (`pending`, `error`), so you can disable the button while the request is in flight. On success, `Form` revalidates the loader, so the page's data updates automatically without a manual refetch.
|
|
167
|
+
|
|
168
|
+
## Mutations without a form: `useAction`
|
|
169
|
+
|
|
170
|
+
`Toil.Form` is convenient sugar over a more general primitive, `Toil.useAction`. Use `useAction` for any write that is not a form submit: a delete button, a "like" toggle, a "mark as read" action. It runs your async function on demand and tracks the whole lifecycle (`pending`, `error`, `data`), then revalidates the affected loader data on success so the page reflects the change, exactly like `Form` does.
|
|
171
|
+
|
|
172
|
+
You call it with the function to run and an optional options object, and it hands back a handle:
|
|
173
|
+
|
|
174
|
+
```tsx
|
|
175
|
+
export default function PostRow({ id }: { id: number }) {
|
|
176
|
+
const del = Toil.useAction(
|
|
177
|
+
(postId: number) => Server.REST.posts.remove({ params: { id: postId } }),
|
|
178
|
+
{ revalidate: true, onError: (e) => alert(parseError(e)) },
|
|
179
|
+
);
|
|
180
|
+
|
|
181
|
+
return (
|
|
182
|
+
<button disabled={del.pending} onClick={() => void del.run(id)}>
|
|
183
|
+
{del.pending ? 'Deleting...' : 'Delete'}
|
|
184
|
+
</button>
|
|
185
|
+
);
|
|
186
|
+
}
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
The handle it returns:
|
|
190
|
+
|
|
191
|
+
| Field | What it is |
|
|
192
|
+
| --- | --- |
|
|
193
|
+
| `run(input)` | Runs the action. Resolves to the result, or `undefined` if it threw. The error is captured in `error` rather than rejected, so a fire-and-forget `onClick` cannot leak an unhandled rejection. |
|
|
194
|
+
| `pending` | `true` while a run is in flight. Drive a spinner or a disabled button off this. |
|
|
195
|
+
| `error` | The error from the last failed run, or `undefined`. |
|
|
196
|
+
| `data` | The value the last successful run returned, or `undefined`. |
|
|
197
|
+
| `reset()` | Clear `pending` / `error` / `data` back to idle. |
|
|
198
|
+
|
|
199
|
+
The options mirror `Toil.Form`:
|
|
200
|
+
|
|
201
|
+
| Option | Default | What it does |
|
|
202
|
+
| --- | --- | --- |
|
|
203
|
+
| `revalidate` | `true` (current route) | Which loader data to refetch after a successful run: `true` (the active route), an `href` or array of hrefs (those routes), or `false` (none). |
|
|
204
|
+
| `onSuccess` | | Called with the action's return value after a successful run. |
|
|
205
|
+
| `onError` | | Called with the thrown value when the action fails. |
|
|
206
|
+
|
|
207
|
+
`Toil.Form` is just this hook wired to a `<form>`'s submit event, with the form's `FormData` passed as the input. Anything a form does, you can do by hand with `useAction`.
|
|
208
|
+
|
|
209
|
+
## Reading who is logged in
|
|
210
|
+
|
|
211
|
+
To render "logged in as ..." you need the current user. toiljs generates a `getUser()` helper (from the backend's `@user` surface) that reads the current user from a readable session cookie with no network round-trip. It is instant, which makes it perfect for display, but it is **untrusted**: a value read on the client can be forged, so never gate anything security-sensitive on it. For a trusted check, call a guarded backend route that re-verifies the signed session.
|
|
212
|
+
|
|
213
|
+
The full auth flow (post-quantum login, sessions, `getUser()`, and guarding routes) is its own guide:
|
|
214
|
+
|
|
215
|
+
- [Auth usage](../auth/usage.md): reading the session, `getUser()`, and guarding pages.
|
|
216
|
+
|
|
217
|
+
## RPC (`@service` / `@remote`)
|
|
218
|
+
|
|
219
|
+
Alongside REST, toiljs generates a typed RPC surface for `@service` methods and free `@remote` functions. These read like plain function calls with no URL:
|
|
220
|
+
|
|
221
|
+
```tsx
|
|
222
|
+
const n = await Server.ping(10); // a free @remote
|
|
223
|
+
const count = await Server.stats.playerCount(); // a @service method
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
The types come straight from your server, so `Server.ping` knows it takes a number and returns a number. Under the hood each call encodes its arguments and POSTs them to a single reserved endpoint (`/__toil_rpc`) with a compact method id, then decodes the typed result. Both the local dev server and the production edge dispatch this endpoint, so RPC works end to end.
|
|
227
|
+
|
|
228
|
+
If a call throws an "unavailable" error, it means the generated client has not attached yet: build the server (`npm run build:server`) to regenerate `shared/server.ts`, and import from `shared/server` so the client loads (see the gotchas below). See [Typed RPC](../backend/rpc.md) for the backend side and when to choose RPC over REST.
|
|
229
|
+
|
|
230
|
+
## Gotchas
|
|
231
|
+
|
|
232
|
+
- **Import from `shared/server` to attach the clients.** `Server.REST` (and the RPC client) attach when you import from `shared/server`. Importing your `@data` classes from there does it naturally; if `Server.REST.foo()` throws an "unavailable" error, make sure the server has been built (`npm run build:server`) and that `shared/server` is imported.
|
|
233
|
+
- **The server runs a fresh instance per request.** In the examples, in-memory writes are previews that do not persist. To keep data, write it to the database (see [Database](../database/README.md)). This is a backend property, but it explains why a create returns a value that is not there on the next request.
|
|
234
|
+
- **Fetch page data in a `loader`, not `useEffect`.** A loader runs in parallel with the route chunk and integrates with `loading.tsx`, suspense, caching, and SSR hydration. A `useEffect` fetch runs only after the page mounts (slower, and invisible to the server).
|
|
235
|
+
- **`getUser()` is display-only.** It is fast because it does not verify anything. Do real authorization on the server.
|
|
236
|
+
|
|
237
|
+
## Related
|
|
238
|
+
|
|
239
|
+
- [Routing](./routing.md): loaders, `loading.tsx`, and navigation.
|
|
240
|
+
- [Backend HTTP routes](../backend/rest.md): the `@rest` controllers behind `Server.REST`.
|
|
241
|
+
- [Typed RPC](../backend/rpc.md): the `@service` / `@remote` surface behind `Server`.
|
|
242
|
+
- [Data types](../backend/data.md): the `@data` classes that cross the wire.
|
|
243
|
+
- [Auth usage](../auth/usage.md): sessions and `getUser()`.
|