@tapedb/tapedb 0.0.0-stage → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/API.md +301 -0
- package/LICENSE +21 -0
- package/LIMITATIONS.md +34 -0
- package/PRIVACY.md +119 -0
- package/README.md +720 -2
- package/RELEASING.md +117 -0
- package/SECURITY.md +121 -0
- package/dist/chunk-3AL32Q4K.js +241 -0
- package/dist/chunk-3AL32Q4K.js.map +1 -0
- package/dist/chunk-CYDXFAEB.js +33 -0
- package/dist/chunk-CYDXFAEB.js.map +1 -0
- package/dist/chunk-UYZJAMBR.js +27 -0
- package/dist/chunk-UYZJAMBR.js.map +1 -0
- package/dist/chunk-VWW5OS5B.js +841 -0
- package/dist/chunk-VWW5OS5B.js.map +1 -0
- package/dist/cli.d.ts +44 -0
- package/dist/cli.js +1501 -0
- package/dist/cli.js.map +1 -0
- package/dist/client-DgSnn9BE.d.ts +646 -0
- package/dist/index.d.ts +24 -0
- package/dist/index.js +3007 -0
- package/dist/index.js.map +1 -0
- package/dist/react.d.ts +41 -0
- package/dist/react.js +59 -0
- package/dist/react.js.map +1 -0
- package/dist/schema-CVcGZJfR.d.ts +691 -0
- package/dist/schema-entry-CpO_IGFv.d.ts +522 -0
- package/dist/schema.d.ts +2 -0
- package/dist/schema.js +22 -0
- package/dist/schema.js.map +1 -0
- package/dist/worker.d.ts +437 -0
- package/dist/worker.js +9270 -0
- package/dist/worker.js.map +1 -0
- package/package.json +93 -4
package/API.md
ADDED
|
@@ -0,0 +1,301 @@
|
|
|
1
|
+
# TapeDB API
|
|
2
|
+
|
|
3
|
+
This is the public API of TapeDB and what it promises. It is the reference;
|
|
4
|
+
[README.md](README.md) is the tour.
|
|
5
|
+
|
|
6
|
+
## Stability
|
|
7
|
+
|
|
8
|
+
Every part of the API is in one of three states:
|
|
9
|
+
|
|
10
|
+
- **Stable.** Changes only in a major release, except to fix a bug or a
|
|
11
|
+
security problem. Additions (new optional options, new exports, new error
|
|
12
|
+
codes) may arrive in a minor release.
|
|
13
|
+
- **Experimental.** Marked `@experimental` in the types and below. Works and
|
|
14
|
+
is tested, but its shape may change in a minor release, with a CHANGELOG
|
|
15
|
+
entry.
|
|
16
|
+
- **Internal.** Everything not listed here. Not exported; may change at any
|
|
17
|
+
time.
|
|
18
|
+
|
|
19
|
+
Before 1.0, a breaking change to a stable part bumps the minor version
|
|
20
|
+
(0.x → 0.x+1) and is listed in the CHANGELOG under "Breaking".
|
|
21
|
+
|
|
22
|
+
The surface is checked: `test/api-surface.test.ts` compares every export,
|
|
23
|
+
every signature (including the types those signatures use), the CLI usage and
|
|
24
|
+
the set of error codes against [api/surface.json](api/surface.json), and fails
|
|
25
|
+
on any difference. A deliberate change updates this file and the CHANGELOG,
|
|
26
|
+
then records the new surface with `bun run api:update`.
|
|
27
|
+
|
|
28
|
+
## Entry points
|
|
29
|
+
|
|
30
|
+
| Import | Runs in | Contents |
|
|
31
|
+
| --- | --- | --- |
|
|
32
|
+
| `@tapedb/tapedb` | browser, Node, Workers | the client, schema builders, ids, roles |
|
|
33
|
+
| `@tapedb/tapedb/schema` | anywhere | only what a schema file needs |
|
|
34
|
+
| `@tapedb/tapedb/react` | browser (React 19 peer) | hooks |
|
|
35
|
+
| `@tapedb/tapedb/worker` | Cloudflare Workers | `defineTapeDB` and claim helpers |
|
|
36
|
+
| `npx @tapedb/tapedb` | Node 22.18+ | the CLI (a command, not an import) |
|
|
37
|
+
|
|
38
|
+
`@tapedb/tapedb` has no runtime dependencies and never imports React.
|
|
39
|
+
|
|
40
|
+
## `@tapedb/tapedb/schema` — stable
|
|
41
|
+
|
|
42
|
+
`schema`, `model`, `t` (field builders), `role`, `filter`, `check`,
|
|
43
|
+
`fileCheck`, `attachedTo`, and the types `Schema`, `Row`, `Where`,
|
|
44
|
+
`FieldFilter`, `OrderBy`, `Include`, `CreateData`, `UpdateData`,
|
|
45
|
+
`DeleteBehaviour`, `ScalarKind`, `Access`, `AccessAuth`, `CheckContext`,
|
|
46
|
+
`FileAccess`, `FileCheckContext`, `FileInfo`.
|
|
47
|
+
|
|
48
|
+
`schema(models, { files? })`: `files` is `{ read?, write?, delete? }`, each a
|
|
49
|
+
`role(...)`, `fileCheck(({ auth, file, existing }) => boolean)` or
|
|
50
|
+
`attachedTo(model, (path) => id | null)`. `file` is a `FileInfo`
|
|
51
|
+
(`{ path, size, contentType, uploadedBy }`, `uploadedBy` the uploader's claim subject; for a write, the incoming file,
|
|
52
|
+
and `existing` the one it replaces). `attachedTo` reads need the row readable
|
|
53
|
+
by the caller; writes and deletes need the caller allowed to update it. Defaults:
|
|
54
|
+
read `member`, write and delete `editor`. A rule that throws denies.
|
|
55
|
+
|
|
56
|
+
## `@tapedb/tapedb` — client
|
|
57
|
+
|
|
58
|
+
Everything in `@tapedb/tapedb/schema`, plus:
|
|
59
|
+
|
|
60
|
+
**`createClient({ schema, url, workspace, userId?, getToken?, onWriteRejected? })`** →
|
|
61
|
+
`Client`. Without `getToken` the client asks the development claim route for a
|
|
62
|
+
token, which only `tapedb dev` (or a gated test Worker) serves. Browser clients
|
|
63
|
+
using the default IndexedDB outbox must supply `userId` matching the claim subject
|
|
64
|
+
(`dev-user` in development). Close and recreate clients on account changes;
|
|
65
|
+
legacy unscoped queues are retained but never automatically replayed.
|
|
66
|
+
|
|
67
|
+
Per model, `db.<model>` is a `ModelClient`:
|
|
68
|
+
|
|
69
|
+
| Method | Returns | |
|
|
70
|
+
| --- | --- | --- |
|
|
71
|
+
| `findMany(args?)` | `Query<Row[]>` | `where`, `orderBy`, `take`, `skip`, `include`, `select`; no `take` reads all rows |
|
|
72
|
+
| `findFirst(args?)` | `Query<Row \| null>` | |
|
|
73
|
+
| `findUnique({ where: { id }, include?, select? })` | `Query<Row \| null>` | |
|
|
74
|
+
| `findPage(args?)` | `PageQuery<Row>` | cursor pages in a stable order; `loadMore()` |
|
|
75
|
+
| `count({ where? })` | `Query<number>` | |
|
|
76
|
+
| `aggregate(args)` | `Query<…>` | **experimental**: count, sum, avg, min, max |
|
|
77
|
+
| `groupBy(args)` | `Query<…[]>` | **experimental**: the same per group, at most 1000 groups |
|
|
78
|
+
| `create({ data })` | `Write` | nested create of related rows |
|
|
79
|
+
| `update({ where: { id }, data })` | `Write` | |
|
|
80
|
+
| `upsert({ where: { id }, create, update })` | `Write` | |
|
|
81
|
+
| `delete({ where: { id } })` | `Write` | |
|
|
82
|
+
| `updateMany({ where, data })` | `Promise<{ count, queued? }>` | every matching row in one transaction, however many; commits whole or not at all. Applied to the cached rows it certainly matches and queued offline like any write; `queued: true` while it waits |
|
|
83
|
+
| `deleteMany({ where })` | `Promise<{ count }>` | as above |
|
|
84
|
+
|
|
85
|
+
A `Query` is awaitable, and live through `subscribe(listener)`, which receives
|
|
86
|
+
a `LiveState` (`{ data, isLoading, error }`) and returns an unsubscribe
|
|
87
|
+
function; `snapshot()` gives the current state. Reads go through the caller's
|
|
88
|
+
read rules. A `Write` is awaitable (it resolves to a `WriteResult`), is applied
|
|
89
|
+
optimistically and is queued while offline; a write the server refuses for
|
|
90
|
+
good reaches `onWriteRejected` with a `WriteRejection`.
|
|
91
|
+
|
|
92
|
+
On the client itself:
|
|
93
|
+
|
|
94
|
+
| Member | |
|
|
95
|
+
| --- | --- |
|
|
96
|
+
| `$transaction(writes)` | several writes, applied atomically → `Promise<WriteResult[]>` |
|
|
97
|
+
| `$files` | `FilesClient`: `upload(blob, { path })` → `StorageObject` (any size: past 16 MiB it goes up as a multipart upload of 16 MiB parts, abandoned if a part fails), `download(path)` → `Blob`, `delete(path)`, `list({ prefix? })` → `StoredFile[]` (the files the caller may read, each with `uploadedBy`) |
|
|
98
|
+
| `$presence(room)` | `PresenceRoom`: `set(presence)`, `cursor(cursor)`, `drag(drag)`, `clearDrag()`, `subscribe(listener)` with `PresencePeer[]`, `leave()` |
|
|
99
|
+
| `$broadcast(topic)` | **experimental**: `BroadcastTopic`: `send(message)` → `boolean | "queued"` (`"queued"` while the socket is down: sent after reconnecting, up to 100 per topic for 60 s; `false` when that queue is full), `subscribe((message, from) => …, { onError? })`, which never hears your own messages; never stored, never delivered late |
|
|
100
|
+
| `$across(workspaceIds, { getToken? })` | `Across`: per model `findMany`, `findFirst`, `findPage` and `count` over those workspaces, each with its own claim (`getToken(workspaceId)`, required unless the client uses the development claim route), merged in `orderBy` order; rows carry `workspaceId`; a page `cursor` holds a position per workspace. Results are `AcrossQuery` (awaitable, `subscribe`, `snapshot`, usable with `useLive`); live states add `errors` by workspace. A failed workspace rejects a one-off read with `AcrossError` (`errors`, `partial`) |
|
|
101
|
+
| `$export()` | `AsyncIterable<ExportRecord>`: every row and file entry the claim may read |
|
|
102
|
+
| `$history({ model, id })` | `Promise<HistoryEntry[]>`: each recorded change to the row, oldest first, as `{ seq, at, sub, role, action }` with `action` `"insert"`, `"update"` or `"delete"`; ADMIN or higher; needs `audit: true` on the Worker |
|
|
103
|
+
| `$flush()` | waits until queued writes are acknowledged |
|
|
104
|
+
| `$close()` | closes the socket |
|
|
105
|
+
|
|
106
|
+
Also exported: `id()` (time-ordered ids), `Query`, `PageQuery`, `Write`,
|
|
107
|
+
`WORKSPACE_ROLE_LEVELS`, `hasRole`, `hasRoleLevel`, and the
|
|
108
|
+
types `Client`, `ClientConfig`, `ModelClient`, `FindArgs`, `PageArgs`,
|
|
109
|
+
`Page`, `Result`, `LiveState`, `WriteResult`, `HistoryEntry`, `Across`, `AcrossModel`, `AcrossOptions`, `AcrossQuery`, `AcrossLiveState`, `AcrossError`, `QueryError`,
|
|
110
|
+
`WriteRejection`, `PresenceRoom`, `PresencePeer`, `RoleClaim`,
|
|
111
|
+
`WorkspaceRole`, `ExportRecord`, and **experimental** `BroadcastTopic` and
|
|
112
|
+
`TopicSender`.
|
|
113
|
+
|
|
114
|
+
## `@tapedb/tapedb/react` — hooks
|
|
115
|
+
|
|
116
|
+
| Hook | Returns |
|
|
117
|
+
| --- | --- |
|
|
118
|
+
| `useLive(query)` | `LiveState`: `{ data, isLoading, error }`; for an `AcrossQuery`, `AcrossLiveState` with `errors` too |
|
|
119
|
+
| `useLivePage(pageQuery)` | `LivePage`: `{ rows, hasMore, isLoading, error, loadMore }` |
|
|
120
|
+
| `usePresence(db, room, presence?)` | `PresencePeer[]`; publishes `presence` while mounted |
|
|
121
|
+
| `useBroadcast(db, topic, onMessage?)` | **experimental**: a `send(message) => boolean \| "queued"` function |
|
|
122
|
+
|
|
123
|
+
## `@tapedb/tapedb/worker`
|
|
124
|
+
|
|
125
|
+
**`defineTapeDB(options)`** → `{ worker, WorkspaceRoom }`. Export `worker` as
|
|
126
|
+
the default and `WorkspaceRoom` under that name (`tapedb init` writes this).
|
|
127
|
+
|
|
128
|
+
| Option | |
|
|
129
|
+
| --- | --- |
|
|
130
|
+
| `schema` | the schema |
|
|
131
|
+
| `migrations` | generated by `tapedb migrate dev` (`AppMigration[]`) |
|
|
132
|
+
| `identity` | `ClaimIdentity`: `getSubject(headers)` → `{ sub } \| null`, `getRole(workspaceId, sub)` → role or `null`, optional `limits` (`ClaimLimits`: `{ ip?, subject? }`, each `{ requests, perSeconds }` or `false`; defaults `{ requests: 300, perSeconds: 60 }` per IP before the session lookup and `{ requests: 30, perSeconds: 60 }` per subject after it; `false` turns both off), and optional `allowRequest(key)` → `boolean`, checked after the IP limit with `key` = `claim:<cf-connecting-ip>` (false → 429 `rate_limited`) |
|
|
133
|
+
| `devClaims` | serve the development claim route; refused in production |
|
|
134
|
+
| `retention` | `{ days }`: how long deleted or overwritten values may stay in history; default 30, at least 2 |
|
|
135
|
+
| `audit` | `true` records the writer (`sub`, role) of every write and keeps a per-row history for `$history`, within `retention`; default off |
|
|
136
|
+
| `topics` | **experimental**: `TopicOptions` `{ authorize?, replaySeconds? }`: a `TopicAuthorizer` for broadcast topics, and the seconds of each topic kept for listeners that reconnect (default 60, `0` keeps none) |
|
|
137
|
+
|
|
138
|
+
Also exported: `signRoleClaim(secret, { sub, workspaceId, role, iss?, aud?,
|
|
139
|
+
ttlSeconds? }, now?)` → `{ token, expiresAt, expiresIn }` and
|
|
140
|
+
`verifyRoleClaim(token, env, expectedWorkspaceId)` → `RoleClaim`, for minting
|
|
141
|
+
and checking claims outside the claim route (for example a backend that calls
|
|
142
|
+
export or erase), `ClaimError`,
|
|
143
|
+
`MAX_UPLOAD_BYTES` (25 MiB, the most one upload request carries; files of any size go up in parts), `WORKSPACE_ROLE_LEVELS`, and the types
|
|
144
|
+
`DefineTapeDBOptions`, `TapeDBWorker`, `AppMigration`, `ClaimIdentity`,
|
|
145
|
+
`Env`, `TapeDBSchema`, `BundledWorkspaceMigration`, and the type-only
|
|
146
|
+
`TapeDBRoom` name for the class returned by `defineTapeDB`. These named types
|
|
147
|
+
allow consumers to emit declarations without leaking private names.
|
|
148
|
+
`ModelOptions` is also exported from `@tapedb/tapedb/schema`.
|
|
149
|
+
`TopicAction` and `TopicAuthorizer` are **experimental**.
|
|
150
|
+
|
|
151
|
+
### Bindings and settings (`Env`)
|
|
152
|
+
|
|
153
|
+
| Name | Kind | |
|
|
154
|
+
| --- | --- | --- |
|
|
155
|
+
| `WORKSPACE` | Durable Object namespace | class `WorkspaceRoom` |
|
|
156
|
+
| `DATA` | R2 bucket | log segments and snapshots |
|
|
157
|
+
| `FILES` | R2 bucket, optional | uploads |
|
|
158
|
+
| `TAPEDB_CLAIM_SECRET` | secret | HS256 key for claims, at least 32 bytes |
|
|
159
|
+
| `TAPEDB_KEY_SECRET` | secret | stable wrapping and HMAC key for personal fields; required when the schema has personal fields |
|
|
160
|
+
| `TAPEDB_CLAIM_ISSUER`, `TAPEDB_CLAIM_AUDIENCE` | var, optional | checked when set |
|
|
161
|
+
| `ENVIRONMENT` | var | `production` refuses the development claim route |
|
|
162
|
+
| `TAPEDB_MOVE_FROM` | var, optional | source jurisdiction while moving: `none`, `eu` or `fedramp`; pair with `DATA_FROM` and `FILES_FROM`, and `WORKSPACE_FROM` for a different namespace |
|
|
163
|
+
| `TAPEDB_JURISDICTION` | var, optional | `eu` or `fedramp`; relocate existing workspaces with `tapedb move` |
|
|
164
|
+
| `TAPEDB_DEV_CLAIMS` | var or secret | development claim route on; never in production |
|
|
165
|
+
| `TAPEDB_DEV_CLAIMS_TOKEN` | secret, optional | the development claim route then needs `x-tapedb-dev-token` |
|
|
166
|
+
|
|
167
|
+
## HTTP
|
|
168
|
+
|
|
169
|
+
All engine routes need `Authorization: Bearer <claim>` for the workspace in
|
|
170
|
+
the path. Bodies and answers are JSON unless stated; errors are
|
|
171
|
+
`{ ok: false, error: { code, message } }`.
|
|
172
|
+
|
|
173
|
+
| Route | |
|
|
174
|
+
| --- | --- |
|
|
175
|
+
| `GET /status`, `GET /health` | the Worker's health; no claim |
|
|
176
|
+
| `POST /_tapedb/claim` | `{ workspaceId }` → `{ token, workspaceId, roleLevel, expiresAt, expiresIn }` from the identity adapter; the development route answers `{ token, workspaceId, expiresIn }` |
|
|
177
|
+
| `POST /engine/workspaces/:id/transact` | a transaction: `{ txId, clientTxId?, issuedAt?, ops }` |
|
|
178
|
+
| `POST /engine/workspaces/:id/query` | a one-off read |
|
|
179
|
+
| `GET /engine/workspaces/:id/schema` | the schema document the workspace runs |
|
|
180
|
+
| `GET /engine/workspaces/:id/status` | the workspace's status, including `durability: { seq, flushedSeq, snapshotSeq, lastFlushError }`: every write at or below `flushedSeq` is in R2 |
|
|
181
|
+
| `GET /engine/workspaces/:id/export` | NDJSON export (below) |
|
|
182
|
+
| `GET /engine/workspaces/:id/history?model=…&id=…` | one row's recorded changes → `{ ok, data: HistoryEntry[] }`; ADMIN or higher; 409 `audit_disabled` without `audit: true` |
|
|
183
|
+
| `POST /engine/workspaces/:id/import` | owner-only atomic import of a consistent NDJSON export into an empty workspace; copies metadata, not file bytes |
|
|
184
|
+
| `POST /engine/workspaces/:id/move` | owner-only resumable move to `{ to: "eu" | "fedramp" | "none" }` matching the deployed configuration; preserves receipt ids and removes source data after switching traffic |
|
|
185
|
+
| `DELETE /engine/workspaces/:id` | erase the workspace; owner only; answers 410 afterwards |
|
|
186
|
+
| `GET`/`HEAD`/`PUT`/`DELETE /engine/workspaces/:id/files/:path` | file bytes; a `PUT` carries at most `MAX_UPLOAD_BYTES` |
|
|
187
|
+
| `POST /engine/workspaces/:id/files/:path?uploads` | start a multipart upload (the request's `content-type` is the file's) → `{ uploadId }`; EDITOR or higher, like every write below |
|
|
188
|
+
| `PUT …/files/:path?uploadId=…&partNumber=n` | store part n (1–10000, at most `MAX_UPLOAD_BYTES`; all but the last the same size, at least 5 MiB) → `{ partNumber, etag }` |
|
|
189
|
+
| `POST …/files/:path?uploadId=…` | `{ parts: [{ partNumber, etag }] }` completes it and records the file, like a `PUT` → 201 `{ key, size }` |
|
|
190
|
+
| `DELETE …/files/:path?uploadId=…` | abandon it (R2 also drops unfinished uploads after 7 days). Nothing is listed or readable before completion |
|
|
191
|
+
| `GET /engine/workspaces/:id/files` | the file index; MEMBER or higher, or with file rules any claim, filtered to readable files. The index changes only through the byte routes (other methods: 405) |
|
|
192
|
+
| `GET /engine/workspaces/:id/websocket` | the live socket (below) |
|
|
193
|
+
|
|
194
|
+
**Retries.** A transaction with `clientTxId` is applied at most once, without
|
|
195
|
+
an age limit. The receipt keeps its id, sequence and time permanently; its
|
|
196
|
+
operation hash is removed after half the history retention (15 days by
|
|
197
|
+
default). While the hash exists, different operations under the same id return
|
|
198
|
+
`conflict`; after its removal, the id acknowledges the original transaction
|
|
199
|
+
without applying the submitted operations. The client sends `issuedAt` for you.
|
|
200
|
+
An upgrade or restore from state with missing receipts preserves a fixed
|
|
201
|
+
boundary: unknown retries with `issuedAt` at or before it return
|
|
202
|
+
`retry_expired` (410). Known receipts are acknowledged even before that boundary.
|
|
203
|
+
|
|
204
|
+
|
|
205
|
+
**Export format, version 1.** One JSON object per line: a
|
|
206
|
+
`{"type":"header","format":"tapedb-export","version":1,…}` line, then `row`
|
|
207
|
+
and `file` lines, then `end` (with `consistent`) or `error`. A later format
|
|
208
|
+
changes `version`; readers check it.
|
|
209
|
+
|
|
210
|
+
## Live socket
|
|
211
|
+
|
|
212
|
+
`GET /engine/workspaces/:id/websocket` with the subprotocols `tapedb` and
|
|
213
|
+
`tapedb.claim.<claim>` (never a URL parameter). JSON messages:
|
|
214
|
+
|
|
215
|
+
- client → server: `query:subscribe`, `query:unsubscribe`, `presence:set`,
|
|
216
|
+
`presence:clear`, `cursor:set`, `drag:set`, `drag:clear`, `ping`, and
|
|
217
|
+
**experimental** `topic:join`, `topic:leave`, `topic:send`
|
|
218
|
+
- server → client: `session:ready`, `session:error`, `pong`, `query:data`,
|
|
219
|
+
`query:diff`, `query:error`, `query:unsubscribed`, `presence:update`,
|
|
220
|
+
`cursor:update`, `drag:update`, `error`, and **experimental** `topic:message`, `topic:error`
|
|
221
|
+
|
|
222
|
+
Messages have no size bound of TapeDB's own. A frame is at most
|
|
223
|
+
Cloudflare's 32 MiB; the client sends a message longer than 256 Ki
|
|
224
|
+
characters as frames of that size:
|
|
225
|
+
|
|
226
|
+
- `{ type: "part", id, index, last, data }`: slices of the message's JSON,
|
|
227
|
+
which the workspace stores until the last and then runs as one message.
|
|
228
|
+
Parts out of order answer `error` `part_out_of_order` and drop the message.
|
|
229
|
+
- for `topic:send`, `{ type: "topic:send", topic, part: { id, index, last },
|
|
230
|
+
data }`: slices of the topic message's JSON, relayed to listeners as
|
|
231
|
+
`topic:message` with the same `part` and `data` and joined by the
|
|
232
|
+
receiving client. The workspace never holds the whole message.
|
|
233
|
+
|
|
234
|
+
Topics (**experimental**): every `topic:message` carries the topic's `seq`,
|
|
235
|
+
and `origin` when the sender gave one (`topic:send { origin }`, a stable id
|
|
236
|
+
of the sending client, so it can skip its own messages when they are
|
|
237
|
+
replayed). `topic:join { topic, since? }` answers `topic:joined { topic, seq }`
|
|
238
|
+
after replaying the kept messages with a sequence above `since` (the last
|
|
239
|
+
`topics.replaySeconds`, default 60, at most 1,000 per topic; engine
|
|
240
|
+
migration `tapedb_0009_topic_buffer`).
|
|
241
|
+
|
|
242
|
+
A socket's live queries, presence and topics are kept in its attachment
|
|
243
|
+
while they fit (12 KiB) and in the workspace's storage past that (engine
|
|
244
|
+
migration `tapedb_0007_socket_state`), so a socket may hold any number of
|
|
245
|
+
them. The protocol belongs to the client; use it directly only if you must.
|
|
246
|
+
|
|
247
|
+
## CLI
|
|
248
|
+
|
|
249
|
+
`npx @tapedb/tapedb init | migrate dev|rollback|status|check | dev | studio | deploy |
|
|
250
|
+
export | erase | move`. The exact commands and flags are in `npx @tapedb/tapedb --help`
|
|
251
|
+
and are part of the checked surface. `deploy` refuses when `ENVIRONMENT` is not
|
|
252
|
+
`production`, when `TAPEDB_DEV_CLAIMS` is set, when an R2 bucket is outside
|
|
253
|
+
`TAPEDB_JURISDICTION`, or when `migrate check` fails (no migrations, schema
|
|
254
|
+
changes without a migration, or a stale generated migrations file).
|
|
255
|
+
|
|
256
|
+
## Error codes
|
|
257
|
+
|
|
258
|
+
Every code the engine and Worker answer with is listed in
|
|
259
|
+
[api/surface.json](api/surface.json) (`errorCodes`) and is stable: a code is
|
|
260
|
+
never repurposed. New codes may be added in a minor release. The ones an app
|
|
261
|
+
usually handles:
|
|
262
|
+
|
|
263
|
+
| Code | Status | Meaning |
|
|
264
|
+
| --- | --- | --- |
|
|
265
|
+
| `permission_denied` | 403 | a write or read rule refused |
|
|
266
|
+
| `insufficient_role` | 403 | the claim's role is too low |
|
|
267
|
+
| `not_a_member` | 403 | the claim route found no role for the caller in that workspace |
|
|
268
|
+
| `rate_limited` | 429 | a claim limit or `identity.allowRequest` refused a claim request; built-in limits send `Retry-After` |
|
|
269
|
+
| `unique_violation`, `foreign_key_violation` | 409 | a constraint refused the write |
|
|
270
|
+
| `conflict` | 409 | a `clientTxId` reused with different operations |
|
|
271
|
+
| `retry_expired` | 410 | an unknown retry before the fixed receipt boundary of a legacy upgrade or restore; send it as a new write if it still applies |
|
|
272
|
+
| `workspace_moving` | 409 | writes are frozen during a move; keep them queued and retry |
|
|
273
|
+
| `workspace_moved` | 308 | the old workspace moved; use the move-aware destination Worker |
|
|
274
|
+
| `workspace_importing` | 409 | an import is in progress |
|
|
275
|
+
| `audit_disabled` | 409 | `$history` on a Worker without `audit: true` |
|
|
276
|
+
| `workspace_erased` | 410 | the workspace was erased |
|
|
277
|
+
| `workspace_recovering` | 503 | the workspace is being restored from R2; retry after `Retry-After` seconds. The client retries on its own for up to a minute |
|
|
278
|
+
| `invalid_claim`, `claim_expired`, `invalid_signature`, `missing_token`, `wrong_workspace` | 401/403 | the claim |
|
|
279
|
+
| `jurisdiction_mismatch` | 500 | the Worker reached a room outside `TAPEDB_JURISDICTION` |
|
|
280
|
+
|
|
281
|
+
## Personal data
|
|
282
|
+
|
|
283
|
+
`t.string().personal({ subject: "self" | "<string field>" })` and
|
|
284
|
+
`t.json().personal(...)` encrypt values under a subject key. Personal output
|
|
285
|
+
fields are nullable and erased fields appear in the row's `$erased: string[]`.
|
|
286
|
+
Personal fields support selection and access rules; filters, ordering,
|
|
287
|
+
aggregates, indexes and uniqueness are refused.
|
|
288
|
+
|
|
289
|
+
`db.$erasePerson(subject): Promise<void>` uses
|
|
290
|
+
`DELETE /engine/workspaces/:id/subjects/:subject` with an owner claim. URL-encode
|
|
291
|
+
the subject. Success is `{ ok: true }`; repeated erasure succeeds. The key is
|
|
292
|
+
removed from R2, making existing ciphertext unreadable. Later writes create a
|
|
293
|
+
new independent key. Ordinary fields and files are unchanged.
|
|
294
|
+
|
|
295
|
+
Workers with personal fields require `TAPEDB_KEY_SECRET`, a stable random secret
|
|
296
|
+
of at least 32 characters. Moves require the same secret at both destinations.
|
|
297
|
+
Existing plaintext rows are encrypted in resumable batches before reads resume.
|
|
298
|
+
Historical plaintext copies retain their original retention windows.
|
|
299
|
+
|
|
300
|
+
Broadcast `send` returns `false` for JSON messages exceeding 4 MiB, before
|
|
301
|
+
queueing or sending them. Multipart transmission obeys the same message bound.
|
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Brandon Diaz
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/LIMITATIONS.md
ADDED
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
# Design and release gates
|
|
2
|
+
|
|
3
|
+
## Design
|
|
4
|
+
|
|
5
|
+
- **Cloudflare only.** TapeDB is a library for Workers, Durable Objects and R2.
|
|
6
|
+
It has no hosted control plane; you deploy and operate it in your own account.
|
|
7
|
+
- **One writer per workspace.** Each workspace is one Durable Object, so
|
|
8
|
+
transactions are serializable without locks. A transaction or join stays in
|
|
9
|
+
one workspace, and there is no active-active setup. Reads span workspaces
|
|
10
|
+
with `db.$across`.
|
|
11
|
+
- **Identity is yours.** TapeDB verifies short-lived claims and works with the
|
|
12
|
+
auth you already have, as an ORM does; it does not manage users, sessions,
|
|
13
|
+
passwords or memberships.
|
|
14
|
+
- **Personal fields use erasable keys.** Deleting a subject key makes its
|
|
15
|
+
ciphertext unreadable in SQL, logs, snapshots and recovery copies. Ordinary
|
|
16
|
+
plaintext and copies from before encryption follow the history retention
|
|
17
|
+
and Cloudflare's 30-day recovery window.
|
|
18
|
+
- **Migrations are files you review.** Destructive changes need
|
|
19
|
+
`--accept-data-loss`; renames need an explicit rename; new required fields
|
|
20
|
+
need a default or must be optional. `migrate rollback` writes the migration
|
|
21
|
+
back to an earlier schema; a drop cannot be undone, because its values are
|
|
22
|
+
gone. Review generated SQL before deploying.
|
|
23
|
+
|
|
24
|
+
## Release gates
|
|
25
|
+
|
|
26
|
+
Every release passes the gates in [RELEASING.md](RELEASING.md): CI on every
|
|
27
|
+
change, a clean secret and history scan, a security review of the claim,
|
|
28
|
+
workspace, file and erase paths, a packed artifact exercised from a clean
|
|
29
|
+
project, an independent source review, and deployed-runtime durability,
|
|
30
|
+
recovery, migration, erasure and offline persistence checks.
|
|
31
|
+
|
|
32
|
+
Socket JSON messages are bounded to 4 MiB, with at most 1,024 parts. Pending
|
|
33
|
+
assemblies allow eight IDs and 8 MiB per socket, and 32 MiB per workspace. Topic
|
|
34
|
+
replay is additionally bounded to 8 MiB across the workspace.
|
package/PRIVACY.md
ADDED
|
@@ -0,0 +1,119 @@
|
|
|
1
|
+
# Privacy and GDPR
|
|
2
|
+
|
|
3
|
+
What tapedb stores about the people using your app, where, for how long, and
|
|
4
|
+
how to answer their requests. You are the controller of that data. tapedb is
|
|
5
|
+
a library running in your Cloudflare account, and Cloudflare is your
|
|
6
|
+
processor.
|
|
7
|
+
|
|
8
|
+
## What is stored, and where
|
|
9
|
+
|
|
10
|
+
| Where | What | How long |
|
|
11
|
+
| --- | --- | --- |
|
|
12
|
+
| Durable Object SQLite, one per workspace | Your models' rows | Until deleted |
|
|
13
|
+
| | `_tx_log`: each write's operations (with values) and the time. The writer's `sub` and role only with `audit: true` | Covered rows deleted once older than half the retention (15 days by default) |
|
|
14
|
+
| | `_tx_audit`, only with `audit: true`: for each changed row, the model, row id, sequence, time, action and the writer's `sub` and role | Deleted once older than half the retention (15 days by default); gone from R2 copies within the full retention |
|
|
15
|
+
| | `_tx_receipts`: random client transaction id, sequence and time; an operation hash while recent (an HMAC under `TAPEDB_KEY_SECRET` when the schema has personal fields) | Id, sequence and time for the workspace lifetime; hash removed after half the retention (15 days by default). Superseded R2 copies leave within the full retention |
|
|
16
|
+
| | `_files`: path, size, content type and uploader (the claim's `sub`) of each upload | Until the file is deleted |
|
|
17
|
+
| R2 `DATA` bucket | Log segments (the `_tx_log` lines), snapshots of every table, a manifest | Superseded copies deleted within the retention (30 days by default) |
|
|
18
|
+
| R2 `DATA` key prefix | Per-subject AES-GCM keys wrapped with `TAPEDB_KEY_SECRET`, named by subject HMAC | Until subject or workspace erasure |
|
|
19
|
+
| R2 `FILES` bucket | Uploaded bytes | Deleted with the file, at once |
|
|
20
|
+
| Socket attachments, and `_socket_state` / `_socket_parts` past 12 KiB | A connected client's `sub`, role, subscriptions, presence state and joined topics; a message arriving in parts until its last part | While the socket is open |
|
|
21
|
+
| Broadcast topics | Messages | Not stored: delivered to connected sockets only |
|
|
22
|
+
| `_topic_buffer` (Durable Object SQLite) | Recent broadcast messages and their sender's `sub` | `topics.replaySeconds` (60 s by default; 1,000 messages per topic at most) |
|
|
23
|
+
| Claim limiter Durable Objects | A token count per HMAC of a caller's IP or subject (never the IP or subject itself) | Deleted once the bucket has refilled: a minute by default |
|
|
24
|
+
| Worker logs | Workspace ids, counts, error messages. No row values, claim tokens or user ids | Your Workers Logs or Logpush settings |
|
|
25
|
+
|
|
26
|
+
Receipts contain no row values or writer identity. Storage grows by one small
|
|
27
|
+
receipt per transaction with a client id. Use a random id for each new write;
|
|
28
|
+
receipt ids are not user ids. Workspace erasure removes the receipts too.
|
|
29
|
+
|
|
30
|
+
Claim tokens are never put in a URL. HTTP requests carry them in
|
|
31
|
+
`Authorization`, and WebSocket upgrades in `Sec-WebSocket-Protocol`. If you
|
|
32
|
+
turn on request-header logging in Cloudflare, exclude those two headers.
|
|
33
|
+
|
|
34
|
+
By default tapedb does not record who made each row write. Files are the
|
|
35
|
+
exception: their uploader is kept for file rules. `defineTapeDB({ audit: true })`
|
|
36
|
+
records the writer of every write (see [README: Audit](README.md#audit)); that
|
|
37
|
+
history is bounded by the retention, read only by admins and owners, and not
|
|
38
|
+
part of `$export()`. A writer's entries leave every copy within the
|
|
39
|
+
retention, which is how an erasure request for them is answered; erasing the
|
|
40
|
+
workspace removes them at once. Use an opaque user id as
|
|
41
|
+
the claim's `sub`, not an email address: the audit stores it as given.
|
|
42
|
+
|
|
43
|
+
## Retention
|
|
44
|
+
|
|
45
|
+
A value deleted or overwritten at time T is gone from every copy tapedb
|
|
46
|
+
keeps by T + retention: the local log, R2 log segments and older snapshots.
|
|
47
|
+
That holds even in a workspace that sees few writes (see
|
|
48
|
+
[README: History retention](README.md#history-retention)). Configure it with:
|
|
49
|
+
|
|
50
|
+
```ts
|
|
51
|
+
defineTapeDB({ schema, retention: { days: 30 } }); // at least 2
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
Two things sit outside tapedb's control:
|
|
55
|
+
|
|
56
|
+
- **Durable Object point-in-time recovery.** Cloudflare keeps 30 days of
|
|
57
|
+
restorable history for every SQLite-backed Durable Object. tapedb never
|
|
58
|
+
restores from it, but until it ages out, code running in your account could.
|
|
59
|
+
For ordinary plaintext the effective bound is the greater of retention and 30 days; personal ciphertext becomes unreadable when its R2 subject key is deleted. The
|
|
60
|
+
default retention is 30 days to match.
|
|
61
|
+
- **R2 bucket lock rules.** A lock rule on the `DATA` or `FILES` bucket stops
|
|
62
|
+
tapedb from deleting anything it covers. Don't add one to tapedb's buckets.
|
|
63
|
+
|
|
64
|
+
## Answering requests
|
|
65
|
+
|
|
66
|
+
**Access and portability (Art. 15, 20).** `db.$export()`, or
|
|
67
|
+
`tapedb export --workspace <id>`, streams every row and file entry the claim
|
|
68
|
+
may read as NDJSON, through the same read rules as any query. Run it with
|
|
69
|
+
the person's own claim to get what they may see, or with an owner's claim and
|
|
70
|
+
filter to the rows about them.
|
|
71
|
+
|
|
72
|
+
**Rectification (Art. 16).** Update the rows. The old values leave the
|
|
73
|
+
history within the retention.
|
|
74
|
+
|
|
75
|
+
**Erasure of a person (Art. 17).** Mark personal string and JSON fields with
|
|
76
|
+
`.personal({ subject: "self" })` or a string subject field. An owner calls
|
|
77
|
+
`db.$erasePerson(subject)` to delete the subject key from R2. SQL, transaction
|
|
78
|
+
logs, snapshots and Cloudflare recovery copies hold ciphertext that can no
|
|
79
|
+
longer be decrypted; reads return `null` with `$erased` field names. Delete
|
|
80
|
+
ordinary fields and files separately. Plaintext stored before encryption was
|
|
81
|
+
enabled still follows the history retention and Cloudflare's 30-day window.
|
|
82
|
+
Exports contain decrypted values: their copies and any external key backups
|
|
83
|
+
need their own erasure. The wrapping secret must stay stable, including moves;
|
|
84
|
+
losing it loses access to every personal field.
|
|
85
|
+
|
|
86
|
+
**Erasure of a workspace (a whole tenant).** `tapedb erase --workspace <id>
|
|
87
|
+
--confirm <id>`, or `DELETE /engine/workspaces/:id` with an owner's claim,
|
|
88
|
+
deletes the workspace's R2 history, its files and its Durable Object
|
|
89
|
+
storage. The workspace answers `410` afterwards (see
|
|
90
|
+
[README: Erasing a workspace](README.md#erasing-a-workspace)).
|
|
91
|
+
|
|
92
|
+
## Where the data lives
|
|
93
|
+
|
|
94
|
+
By default Cloudflare places each workspace's Durable Object near its first
|
|
95
|
+
request, and R2 buckets wherever they were created.
|
|
96
|
+
`tapedb init --jurisdiction eu` keeps the Durable Objects and both buckets in
|
|
97
|
+
Cloudflare's EU jurisdiction, where they run and store data only in the EU,
|
|
98
|
+
and `tapedb deploy` refuses a config where the two disagree (see
|
|
99
|
+
[README: Data residency](README.md#data-residency)). `tapedb move` relocates
|
|
100
|
+
existing workspaces, including their rows, receipts, snapshots, log segments
|
|
101
|
+
and indexed file bytes. The source remains readable while it is frozen;
|
|
102
|
+
after the destination is ready, traffic switches and source objects and
|
|
103
|
+
SQLite storage are removed. A routing tombstone contains only the workspace
|
|
104
|
+
id and destination jurisdiction. Cloudflare's point-in-time recovery window
|
|
105
|
+
still applies to the original Durable Object's plaintext data.
|
|
106
|
+
|
|
107
|
+
Jurisdiction controls where the data is stored and processed. Cloudflare,
|
|
108
|
+
Inc. is still the processor, so your transfer assessment and Cloudflare's
|
|
109
|
+
data processing addendum still apply.
|
|
110
|
+
|
|
111
|
+
## Security
|
|
112
|
+
|
|
113
|
+
- Claims are short-lived HS256 tokens your Worker verifies on every request.
|
|
114
|
+
The engine never mints them except through the development claim route,
|
|
115
|
+
which `ENVIRONMENT=production` refuses and `tapedb deploy` will not ship.
|
|
116
|
+
- Every engine route needs a verified claim before a workspace is touched.
|
|
117
|
+
Read and write rules apply inside the engine, and erasing needs an owner.
|
|
118
|
+
- Data reaches Cloudflare over TLS and is encrypted at rest by Cloudflare in
|
|
119
|
+
Durable Object storage and R2.
|