@jskit-ai/agent-docs 0.1.283 → 0.1.284
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/guide/agent/app-extras/assistant.md +14 -2
- package/guide/agent/app-extras/realtime.md +38 -1
- package/guide/agent/app-setup/authentication.md +64 -14
- package/guide/agent/app-setup/console.md +8 -0
- package/guide/agent/app-setup/database-layer.md +23 -103
- package/guide/agent/app-setup/existing-application-migration.md +105 -72
- package/guide/agent/app-setup/multi-homing.md +94 -32
- package/guide/agent/app-setup/quickstart.md +1 -1
- package/guide/agent/app-setup/run-an-accounts-hub.md +1 -1
- package/guide/agent/app-setup/upgrading-jskit.md +8 -3
- package/guide/agent/app-setup/users.md +8 -5
- package/guide/agent/framework/application-operations.md +1 -1
- package/guide/agent/framework/crud-operations.md +63 -45
- package/guide/agent/framework/data-access.md +1031 -0
- package/guide/agent/framework/sanity-checks.md +26 -27
- package/guide/agent/framework/typed-features.md +114 -247
- package/guide/agent/index.md +2 -0
- package/package.json +6 -3
- package/patterns/feature-package/PATTERN.md +25 -17
- package/patterns/feature-package/example/booking-engine/package.json +1 -1
- package/patterns/feature-package/example/variations/InvoiceRollupFeature.js +6 -7
- package/patterns/feature-package/example/variations/invoiceResource.js +15 -0
- package/patterns/feature-package/example/variations/invoiceRollupActions.js +4 -4
- package/patterns/feature-package/example/variations/ownedTableRepository.js +27 -0
- package/patterns/feature-package/example/variations/types.js +1 -1
- package/patterns/minimal-foundation/example/AGENTS.md +4 -0
- package/patterns/minimal-foundation/example/package.json +4 -4
- package/patterns/minimal-foundation/example/server.js +4 -1
- package/patterns/shell-foundation/example/AGENTS.md +4 -0
- package/patterns/shell-foundation/example/package.json +5 -5
- package/patterns/shell-foundation/example/packages/main/package.json +1 -1
- package/patterns/shell-foundation/example/server.js +4 -1
- package/reference/autogen/KERNEL_MAP.md +2 -1
- package/reference/autogen/PATTERN_INDEX.md +70 -59
- package/reference/autogen/README.md +1 -1
- package/reference/autogen/packages/agent-docs.md +7 -3
- package/reference/autogen/packages/agent-runtime.md +48 -11
- package/reference/autogen/packages/architecture-inspector.md +6 -2
- package/reference/autogen/packages/assistant-core.md +4 -4
- package/reference/autogen/packages/assistant-runtime.md +33 -13
- package/reference/autogen/packages/auth-provider-local-core.md +3 -1
- package/reference/autogen/packages/auth-provider-local-db-core.md +65 -14
- package/reference/autogen/packages/auth-provider-oidc-core.md +5 -1
- package/reference/autogen/packages/auth-provider-supabase-core.md +1 -1
- package/reference/autogen/packages/connectors-core.md +22 -2
- package/reference/autogen/packages/console-core.md +9 -5
- package/reference/autogen/packages/crud-core.md +38 -173
- package/reference/autogen/packages/database-runtime.md +8 -32
- package/reference/autogen/packages/http-runtime.md +3 -3
- package/reference/autogen/packages/http-web.md +12 -0
- package/reference/autogen/packages/kernel.md +25 -2
- package/reference/autogen/packages/oidc-issuer-core.md +4 -0
- package/reference/autogen/packages/payments-core.md +29 -6
- package/reference/autogen/packages/realtime.md +15 -6
- package/reference/autogen/packages/resource-core.md +30 -5
- package/reference/autogen/packages/resource-crud-core.md +30 -34
- package/reference/autogen/packages/resource-runtime.md +198 -0
- package/reference/autogen/packages/rewarded-core.md +16 -10
- package/reference/autogen/packages/shell-web.md +1 -0
- package/reference/autogen/packages/users-core.md +26 -14
- package/reference/autogen/packages/workspaces-core.md +17 -10
- package/reference/autogen/tooling/config-eslint.md +69 -0
- package/reference/autogen/tooling/jskit-catalog.md +36 -1
- package/reference/autogen/tooling/jskit-release.md +3 -0
- package/skills/jskit/SKILL.md +4 -1
- package/skills/jskit/references/app-operations.md +1 -1
- package/skills/jskit/references/crud-operations.md +62 -44
- package/skills/jskit/references/data-access.md +1026 -0
- package/skills/jskit/references/database-layer.md +411 -0
- package/skills/jskit/references/existing-application-migration.md +105 -72
- package/skills/jskit/references/heuristics/INDEX.md +6 -4
- package/skills/jskit/references/heuristics/crud-authoring.md +22 -14
- package/skills/jskit/references/heuristics/data-access.md +19 -0
- package/skills/jskit/references/heuristics/filters.md +6 -6
- package/skills/jskit/references/heuristics/row-policies.md +40 -16
- package/skills/jskit/references/heuristics/server-search.md +30 -42
- package/skills/jskit/references/heuristics/table-resource-fields.md +78 -0
- package/skills/jskit/references/heuristics/typed-features.md +3 -3
- package/skills/jskit/references/multi-homing.md +94 -32
- package/skills/jskit/references/pattern-index.md +70 -59
- package/skills/jskit/references/patterns/app/minimal-foundation/example/AGENTS.md +4 -0
- package/skills/jskit/references/patterns/app/minimal-foundation/example/package.json +4 -4
- package/skills/jskit/references/patterns/app/minimal-foundation/example/server.js +4 -1
- package/skills/jskit/references/patterns/app/shell-foundation/example/AGENTS.md +4 -0
- package/skills/jskit/references/patterns/app/shell-foundation/example/package.json +5 -5
- package/skills/jskit/references/patterns/app/shell-foundation/example/packages/main/package.json +1 -1
- package/skills/jskit/references/patterns/app/shell-foundation/example/server.js +4 -1
- package/skills/jskit/references/patterns/assistant/assistant-surface/PATTERN.md +5 -2
- package/skills/jskit/references/patterns/auth/accounts-hub/example/package.json +3 -2
- package/skills/jskit/references/patterns/auth/auth-surface/PATTERN.md +15 -0
- package/skills/jskit/references/patterns/auth/oidc-sign-in/example/package.json +2 -2
- package/skills/jskit/references/patterns/auth/supabase-auth/example/package.json +1 -1
- package/skills/jskit/references/patterns/connectors/calendar-cli/example/package.json +4 -4
- package/skills/jskit/references/patterns/crud/crud-contract/PATTERN.md +133 -0
- package/skills/jskit/references/patterns/crud/crud-contract/example/bookResource.js +74 -0
- package/skills/jskit/references/patterns/crud/{json-api-resource-package → crud-feature-package}/PATTERN.md +69 -37
- package/skills/jskit/references/patterns/crud/{json-api-resource-package → crud-feature-package}/example/packages/books/package.json +5 -4
- package/skills/jskit/references/patterns/crud/crud-feature-package/example/packages/books/src/server/BooksFeature.js +13 -0
- package/skills/jskit/references/patterns/crud/crud-feature-package/example/packages/books/src/shared/bookResource.js +74 -0
- package/skills/jskit/references/patterns/crud/crud-feature-package/example/packages/books/src/shared/index.js +1 -0
- package/skills/jskit/references/patterns/crud/crud-screen-set/PATTERN.md +26 -8
- package/skills/jskit/references/patterns/crud/crud-screen-set/example/books/BookEditPage.vue +2 -2
- package/skills/jskit/references/patterns/crud/crud-screen-set/example/books/BookListPage.vue +2 -2
- package/skills/jskit/references/patterns/crud/crud-screen-set/example/books/BookNewPage.vue +2 -2
- package/skills/jskit/references/patterns/crud/crud-screen-set/example/books/BookViewPage.vue +3 -3
- package/skills/jskit/references/patterns/database/mysql-application/PATTERN.md +9 -4
- package/skills/jskit/references/patterns/database/mysql-application/example/package.json +2 -1
- package/skills/jskit/references/patterns/database/mysql-application/example/scripts/prepare-database.js +8 -0
- package/skills/jskit/references/patterns/database/postgres-application/PATTERN.md +8 -4
- package/skills/jskit/references/patterns/database/postgres-application/example/package.json +2 -1
- package/skills/jskit/references/patterns/database/postgres-application/example/scripts/prepare-database.js +8 -0
- package/skills/jskit/references/patterns/database/table-resource/PATTERN.md +117 -0
- package/skills/jskit/references/patterns/database/table-resource/example/orderResource.js +25 -0
- package/skills/jskit/references/patterns/realtime/realtime-application/PATTERN.md +7 -0
- package/skills/jskit/references/patterns/realtime/realtime-application/example/package.json +2 -2
- package/skills/jskit/references/patterns/rewards/google-rewarded/PATTERN.md +1 -1
- package/skills/jskit/references/patterns/server/feature-package/PATTERN.md +25 -17
- package/skills/jskit/references/patterns/server/feature-package/example/booking-engine/package.json +1 -1
- package/skills/jskit/references/patterns/server/feature-package/example/variations/InvoiceRollupFeature.js +6 -7
- package/skills/jskit/references/patterns/server/feature-package/example/variations/invoiceResource.js +15 -0
- package/skills/jskit/references/patterns/server/feature-package/example/variations/invoiceRollupActions.js +4 -4
- package/skills/jskit/references/patterns/server/feature-package/example/variations/ownedTableRepository.js +27 -0
- package/skills/jskit/references/patterns/server/feature-package/example/variations/types.js +1 -1
- package/skills/jskit/references/patterns/users/user-administration-server/PATTERN.md +59 -10
- package/skills/jskit/references/patterns/users/user-administration-server/example/packages/users/package.json +5 -4
- package/skills/jskit/references/patterns/users/user-administration-server/example/packages/users/src/server/UsersFeature.js +7 -5
- package/skills/jskit/references/patterns/users/user-administration-server/example/packages/users/src/shared/index.js +1 -1
- package/skills/jskit/references/patterns/users/user-administration-server/example/packages/users/src/shared/userCrud.js +21 -0
- package/skills/jskit/references/patterns/users/user-administration-server/example/packages/users-workspace/package.json +11 -5
- package/skills/jskit/references/patterns/users/user-administration-server/example/packages/users-workspace/src/server/UsersWorkspaceFeature.js +14 -5
- package/skills/jskit/references/patterns/users/user-administration-server/example/packages/users-workspace/src/server/workspaceMembersOnly.js +52 -0
- package/skills/jskit/references/patterns/users/user-administration-server/example/packages/users-workspace/src/shared/index.js +1 -1
- package/skills/jskit/references/patterns/users/user-administration-server/example/packages/users-workspace/src/shared/userCrud.js +21 -0
- package/skills/jskit/references/patterns/workspaces/workspace-server/PATTERN.md +9 -9
- package/skills/jskit/references/patterns/workspaces/workspace-surfaces/PATTERN.md +16 -0
- package/skills/jskit/references/patterns/workspaces/workspace-surfaces/example/src/pages/home/workspaces/index.vue +2 -0
- package/skills/jskit/references/sanity-checks.md +27 -28
- package/skills/jskit/references/typed-features.md +114 -247
- package/patterns/feature-package/example/variations/customKnexRepository.js +0 -40
- package/reference/autogen/packages/json-rest-api-core.md +0 -66
- package/skills/jskit/references/heuristics/crud-repository-mapping.md +0 -140
- package/skills/jskit/references/patterns/crud/json-api-resource-package/example/packages/books/src/server/BooksFeature.js +0 -13
- package/skills/jskit/references/patterns/crud/json-api-resource-package/example/packages/books/src/shared/bookResource.js +0 -83
- package/skills/jskit/references/patterns/crud/json-api-resource-package/example/packages/books/src/shared/index.js +0 -1
- package/skills/jskit/references/patterns/crud/resource-contract/PATTERN.md +0 -155
- package/skills/jskit/references/patterns/crud/resource-contract/example/bookResource.js +0 -87
- package/skills/jskit/references/patterns/server/feature-package/example/variations/customKnexRepository.js +0 -40
- package/skills/jskit/references/patterns/users/user-administration-server/example/packages/users/src/shared/userResource.js +0 -69
- package/skills/jskit/references/patterns/users/user-administration-server/example/packages/users-workspace/src/shared/userResource.js +0 -56
- /package/skills/jskit/references/patterns/crud/{json-api-resource-package → crud-feature-package}/example/migrations/20260815000000_books.cjs +0 -0
- /package/skills/jskit/references/patterns/crud/{json-api-resource-package → crud-feature-package}/example/packages/books/src/server/types.js +0 -0
|
@@ -0,0 +1,1026 @@
|
|
|
1
|
+
<!-- Generated from `packages/agent-docs/site/guide/framework/data-access.md`. -->
|
|
2
|
+
|
|
3
|
+
# Data access
|
|
4
|
+
|
|
5
|
+
Read this before writing code that reads or writes a database table, with or
|
|
6
|
+
without screens. [Typed features](typed-features.md) covers how
|
|
7
|
+
the code around it is shaped; this page covers how it reaches the data.
|
|
8
|
+
|
|
9
|
+
## The resource is the only door
|
|
10
|
+
|
|
11
|
+
Every table has exactly one **table resource**. Only the feature that owns the
|
|
12
|
+
table reads or writes it, through the resource's handle. Ownership, the row
|
|
13
|
+
policy, validation, ownership stamping, audits and change events run only on
|
|
14
|
+
that path. Knex reaches the table past all of them, and putting it in a file
|
|
15
|
+
named "repository" protects nothing.
|
|
16
|
+
|
|
17
|
+
Bad: a repository raised a version counter on a record id it was handed, with
|
|
18
|
+
no workspace predicate, so one workspace could change another workspace's
|
|
19
|
+
booking.
|
|
20
|
+
|
|
21
|
+
Good: the owner's repository calls `bookings.patch(id, { version }, options)`,
|
|
22
|
+
which changes the row only when the caller's scope owns it.
|
|
23
|
+
|
|
24
|
+
## The handle's calls
|
|
25
|
+
|
|
26
|
+
One table, one table resource, one owner feature. The owner reaches its table
|
|
27
|
+
through its handle's calls, and every call takes the call's `options` last:
|
|
28
|
+
|
|
29
|
+
| Call | Does |
|
|
30
|
+
| --- | --- |
|
|
31
|
+
| `query({ filters, sort, page, include }, options)` | one page: `{ records, page }` |
|
|
32
|
+
| `queryAll({ filters, sort, include }, options)` | every matching row, for jobs, id lists and one parent's rows |
|
|
33
|
+
| `get(id, params, options)` | one record |
|
|
34
|
+
| `aggregate({ measures, groupBy, filters, sort }, options)` | totals, grouped by fields or date buckets |
|
|
35
|
+
| `create(data, options)`, `findOrCreate({ id, ...data }, options)` | one record; a keyed one, created once |
|
|
36
|
+
| `patch(id, changes, options)`, `delete(id, options)` | one record |
|
|
37
|
+
| `createMany(rows, options)` | many records, or an upsert with `onConflict` |
|
|
38
|
+
| `patchMany(rows, options)` | many records, each with its own changes |
|
|
39
|
+
| `patchWhere(filters, changes, options)`, `deleteWhere(filters, options)` | every matching row |
|
|
40
|
+
| `lock(id, txOptions)`, `lockMany(ids, txOptions)`, `lockWhere(filters, txOptions)` | rows locked until the transaction ends |
|
|
41
|
+
| `lockOrCreate({ id, ...initial }, options, work)` | a keyed record locked, created first when missing, for `work` |
|
|
42
|
+
| `claim({ filters, sort, take }, txOptions)` | the next rows no other worker holds |
|
|
43
|
+
| `withTransaction(options, work)` | one transaction; `work` gets its `txOptions` |
|
|
44
|
+
| `scopedQuery(params, options)` | a read-only builder for a report ([Layer 5](#layer-5-reports-the-resource-cannot-express)) |
|
|
45
|
+
|
|
46
|
+
```js
|
|
47
|
+
// How many invoices of each status the workspace issued each month, and their total.
|
|
48
|
+
function totalsByMonth(options) {
|
|
49
|
+
return invoices.aggregate({
|
|
50
|
+
groupBy: ["status", { field: "issuedAt", bucket: "month", as: "month" }],
|
|
51
|
+
measures: { invoices: { count: "*" }, amount: { sum: "amount" } },
|
|
52
|
+
sort: ["month"]
|
|
53
|
+
}, options);
|
|
54
|
+
}
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
## Words
|
|
58
|
+
|
|
59
|
+
| Word | Meaning |
|
|
60
|
+
| --- | --- |
|
|
61
|
+
| table resource | The one description of one table: fields and their columns, ownership, relationships, search and lock version. `defineTableResource()`, shared with the client. |
|
|
62
|
+
| owner feature | The feature whose `owns` lists the table resource. It is the only feature that reads or writes the table. |
|
|
63
|
+
| handle | What the owner gets for each resource it owns. Every method applies the scope of the call's `options.context`. |
|
|
64
|
+
| repository | The owner's module that holds every query, built on its handles. Its methods have domain names. A plain CRUD feature's repository is its table's handle itself. |
|
|
65
|
+
| service | Orchestration. It calls repositories (its own, or another feature's through that feature's capability: an owner API, or a CRUD feature's `repository`) and other features' services. It never builds a query (no Knex, no `scopedQuery`) and never reaches another feature's table any other way. |
|
|
66
|
+
| owner API | The capability through which a table's owner lets other features read and write its records, such as `users.profiles`. |
|
|
67
|
+
| CRUD contract | The optional HTTP and screen surface on a table resource: `defineCrudContract()`. |
|
|
68
|
+
| CRUD feature | The feature `defineCrudFeature()` builds from a CRUD contract: its repository, service, actions, routes and screens. |
|
|
69
|
+
| ownership | Who owns a row, required in every table resource: `public`, `workspace`, `user`, `workspace_user`, or `workspace_or_shared` (a workspace's own rows, or rows shared with every workspace when the owner is empty). |
|
|
70
|
+
| row policy | A server-only rule narrowing which rows a context sees, applied to every statement, joins, includes and locks included. |
|
|
71
|
+
| scoped query | The read-only Knex builder `scopedQuery()` returns, with ownership and the row policy already applied. |
|
|
72
|
+
| system scope | Why a feature works across every workspace and user, declared with `systemScope`. |
|
|
73
|
+
| system context | The context of that work, from `systemContext(reason)`; `forWorkspace`, `forUser` and `forWorkspaceUser` narrow it. |
|
|
74
|
+
| `txOptions` | What `withTransaction(options, work)` hands `work`: only the call's context and its transaction, `{ context, trx }`, passed to every call inside. |
|
|
75
|
+
| record | What a handle answers: `{ id, ...fields }`, relationships as ids, included records nested. |
|
|
76
|
+
|
|
77
|
+
## Layer 1: the table resource
|
|
78
|
+
|
|
79
|
+
Every table your app owns has one table resource. The feature that owns the
|
|
80
|
+
table lists it in `owns`, and only that feature's setup receives its handle.
|
|
81
|
+
Every read and write goes through the handle. On every statement the handle
|
|
82
|
+
adds the ownership predicate and the row policy, validates what you write and
|
|
83
|
+
stamps the workspace or user on new rows. A table whose schema stores
|
|
84
|
+
`createdAt` and `updatedAt` as `dateTime` fields gets them stamped by every
|
|
85
|
+
write: both on a create (`createMany` too), `updatedAt` on every change
|
|
86
|
+
(`patch`, `patchMany`, `patchWhere`, an upsert merge, which keeps the stored
|
|
87
|
+
`createdAt`). The stamps replace what any other caller sends, and a CRUD
|
|
88
|
+
contract offers them as output only; only a system context (an import, a
|
|
89
|
+
migration, a seed) sets its own. Pass the call's `options` (its context and
|
|
90
|
+
transaction) to every handle call; a call without a context throws instead of
|
|
91
|
+
guessing.
|
|
92
|
+
|
|
93
|
+
```js
|
|
94
|
+
// shared/receivalResource.js: the table's facts, shared with the client
|
|
95
|
+
const receivalResource = defineTableResource({
|
|
96
|
+
name: "receivals",
|
|
97
|
+
tableName: "receivals",
|
|
98
|
+
ownership: "workspace",
|
|
99
|
+
schema: {
|
|
100
|
+
status: { type: "string", required: true, search: true },
|
|
101
|
+
arrivedAt: { type: "dateTime", nullable: true }
|
|
102
|
+
}
|
|
103
|
+
});
|
|
104
|
+
|
|
105
|
+
// server/ReceivalsFeature.js: the owner
|
|
106
|
+
const ReceivalsFeature = defineFeature({
|
|
107
|
+
id: "receivals",
|
|
108
|
+
owns: [receivalResource, { resource: receivalNoteResource, rowPolicy: authorOrManager }],
|
|
109
|
+
provides: { receivals: "receivals" },
|
|
110
|
+
setup: ({ resources }) => ({ receivals: createReceivalsService(createReceivalsRepository(resources)) })
|
|
111
|
+
});
|
|
112
|
+
|
|
113
|
+
/**
|
|
114
|
+
* server/receivalsRepository.js: every query, by domain name.
|
|
115
|
+
* @param {{ receivals: import("@jskit-ai/resource-runtime/server/types").ResourceHandle }} handles
|
|
116
|
+
*/
|
|
117
|
+
function createReceivalsRepository({ receivals }) {
|
|
118
|
+
/**
|
|
119
|
+
* One page of the open receivals.
|
|
120
|
+
* @param {{ size: number, after?: string }} page
|
|
121
|
+
* @param {import("@jskit-ai/resource-runtime/server/types").ResourceCallOptions} options
|
|
122
|
+
*/
|
|
123
|
+
function listOpen(page, options) {
|
|
124
|
+
return receivals.query({ filters: { status: "open" }, page }, options);
|
|
125
|
+
}
|
|
126
|
+
|
|
127
|
+
/**
|
|
128
|
+
* @param {string} id
|
|
129
|
+
* @param {string} arrivedAt
|
|
130
|
+
* @param {import("@jskit-ai/resource-runtime/server/types").ResourceCallOptions} options
|
|
131
|
+
*/
|
|
132
|
+
function markArrived(id, arrivedAt, options) {
|
|
133
|
+
return receivals.patch(id, { status: "arrived", arrivedAt }, options);
|
|
134
|
+
}
|
|
135
|
+
|
|
136
|
+
return Object.freeze({
|
|
137
|
+
withTransaction: receivals.withTransaction,
|
|
138
|
+
listOpen,
|
|
139
|
+
markArrived
|
|
140
|
+
});
|
|
141
|
+
}
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
A package whose features own tables or declare `systemScope` lists
|
|
145
|
+
`runtime.resources` in its descriptor's `capabilities.requires`, or boot
|
|
146
|
+
refuses it; the feature never requires it itself, since it receives its
|
|
147
|
+
handles.
|
|
148
|
+
|
|
149
|
+
Reads take `(params, options)` and writes `(data, options)`: `get(id, {},
|
|
150
|
+
options)`. A call's settings ride in its options, added on the call they are
|
|
151
|
+
for and spread from the options the method received: `{ ...options, missing:
|
|
152
|
+
"skip" }`, `{ ...options, returning: "id" }`. A method passes on only the
|
|
153
|
+
options it received, never a setting it added for one of its own calls, and
|
|
154
|
+
another table gets only `{ context, trx }`: `passedOn(options)` from
|
|
155
|
+
`@jskit-ai/resource-runtime/server`. An owner API, called with another
|
|
156
|
+
feature's options, hands each of its calls `passedOn(options)` plus the
|
|
157
|
+
settings that call needs, never the caller's options whole. `withTransaction`
|
|
158
|
+
hands its work only the context and the transaction, and so does a handle to
|
|
159
|
+
its hooks and its own nested calls.
|
|
160
|
+
|
|
161
|
+
`txOptions` is `{ context, trx }` and nothing else: every other key of the
|
|
162
|
+
method's `options`, the application's own included (a flag that skips an
|
|
163
|
+
email, a `limit`, a per-request cache), is gone inside the work. Read what the
|
|
164
|
+
work needs from `options` itself, never from `txOptions`. A method that wraps
|
|
165
|
+
a write on its own table in `withTransaction` passes that write `{ ...options,
|
|
166
|
+
...txOptions }`, or the caller's `returning`, `missing` or `format` never
|
|
167
|
+
reaches it and the method answers something its caller did not ask for.
|
|
168
|
+
Another table's call still gets `txOptions`:
|
|
169
|
+
|
|
170
|
+
```js
|
|
171
|
+
const addLines = (rows, options) => lines.withTransaction(options, async (txOptions) => {
|
|
172
|
+
const added = await lines.createMany(rows, { ...options, ...txOptions });
|
|
173
|
+
await orders.patch(rows[0].orderId, { linesChangedAt: new Date() }, txOptions);
|
|
174
|
+
return added;
|
|
175
|
+
});
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
A filter key is one of the resource's `search: true` fields or a
|
|
179
|
+
`searchSchema` entry. `id` is every handle's own filter (a resource never
|
|
180
|
+
declares an `id` search entry) and takes one id or a list; an empty list
|
|
181
|
+
matches no row. Another list of values needs an entry with `filterOperator:
|
|
182
|
+
"in"`. A `null` value matches the rows where the field is null: `filters: {
|
|
183
|
+
archivedAt: null }`.
|
|
184
|
+
|
|
185
|
+
A filter on another table's field (`searchSchema: { supplierName: { type:
|
|
186
|
+
"string", actualField: "suppliers.name" } }`) needs that field declared
|
|
187
|
+
`indexed: true` on its own table resource, with a database index on its column,
|
|
188
|
+
even when the join is on the other table's id; `own()` refuses it at startup
|
|
189
|
+
otherwise, naming both.
|
|
190
|
+
|
|
191
|
+
A table keyed by a `uuid` or `string` primary key takes its ids as text; an
|
|
192
|
+
`owns` entry's `normalizeId` is only for an id written some other way.
|
|
193
|
+
|
|
194
|
+
A read answers every field unless `fields` names the ones it needs (a
|
|
195
|
+
belongs-to field by its own key, such as `customerId`, or its relationship);
|
|
196
|
+
the locks (`lock`, `lockMany`, `lockWhere` and `claim`) take it in their
|
|
197
|
+
options, `{ ...txOptions, fields }`.
|
|
198
|
+
Each belongs-to field a read answers costs a check that its related record is
|
|
199
|
+
visible, so a hot path names its fields. A collection relation with
|
|
200
|
+
`hydrateOnList: false` or `hydrateOnView: false` loads its rows on that kind of
|
|
201
|
+
read only when the read includes it.
|
|
202
|
+
|
|
203
|
+
A `hidden: true` field (an ownership id, a bearer token, a storage key) never
|
|
204
|
+
reaches a request: no CRUD route or JSON:API document (`format: "jsonapi"`)
|
|
205
|
+
carries it, even when `fields` names it. The owner's own code reads one by
|
|
206
|
+
naming it in a handle read's `fields`, `get(id, { fields: ["publicToken"] },
|
|
207
|
+
options)` or `{ ...txOptions, fields }` for a lock, which costs one more
|
|
208
|
+
statement per 1,000 records, scoped as the read is; a read without `fields`
|
|
209
|
+
never answers it. It may be an id, string, integer, number or boolean, and
|
|
210
|
+
naming any other kind refuses the read. The repository a CRUD feature gives
|
|
211
|
+
other features, and its own actions, refuses a read naming a hidden field, and
|
|
212
|
+
any `scopedQuery()` (its builder carries every column), and every method it
|
|
213
|
+
holds, `decorateRepository`'s included, answers without the hidden fields of the
|
|
214
|
+
tables its owner owns, unless the call runs in a system context. So the owner
|
|
215
|
+
checks a token in a method `decorateRepository` adds, with the handle it
|
|
216
|
+
receives, and answers only the outcome. Never unhide a field to read it, and
|
|
217
|
+
never read it with `scopedQuery`.
|
|
218
|
+
|
|
219
|
+
`create` and `patch` answer the written record by default, read back through
|
|
220
|
+
the row policy. Server code that does not use it passes `returning: "id"` or
|
|
221
|
+
`"none"`, which skips that read; `delete` answers the id. One rule covers a
|
|
222
|
+
record that is missing or outside the caller's scope: `get`, `patch`, `delete`
|
|
223
|
+
and `lock` throw json-rest-api's not-found error (a 404), and with
|
|
224
|
+
`missing: "skip"` they answer `null` instead, without failing the caller's
|
|
225
|
+
transaction. A row keyed by an id the code computes (a fingerprint, a
|
|
226
|
+
conversation id) is read or created once with `findOrCreate({ id,
|
|
227
|
+
...attributes }, options)`, outside the transaction that then locks it: inside
|
|
228
|
+
one, losing the race to create it fails the whole transaction. Compute such an
|
|
229
|
+
id from the owner too (the workspace or user id in the fingerprint): an id
|
|
230
|
+
another scope's row holds answers a missing record. That answer still says the
|
|
231
|
+
id is taken, so `findOrCreate` and `lockOrCreate` take only ids the server
|
|
232
|
+
computes or generates, never an id a request supplies.
|
|
233
|
+
|
|
234
|
+
A write that the caller's row policy then hides fails, because its read-back
|
|
235
|
+
finds nothing. The `create` or `patch` answers json-rest-api's 404
|
|
236
|
+
(`REST_API_RESOURCE`, "Resource not found") and its transaction rolls back. A
|
|
237
|
+
staff member who sees only the bookings they run cannot book a colleague this
|
|
238
|
+
way. Decide which way the rule goes. Either the policy lets the writer see
|
|
239
|
+
what they may write, or the action refuses the write before it runs, with a
|
|
240
|
+
403 or 422 that says why.
|
|
241
|
+
|
|
242
|
+
You never write `where workspace_id = ?`, so you cannot forget it. The row
|
|
243
|
+
policy and write hooks go in the `owns` entry on the server, never in the
|
|
244
|
+
shared definition, which ships to the browser. Another feature's contribution
|
|
245
|
+
to a row policy can only narrow it. A row policy adds one predicate correlated
|
|
246
|
+
with the row being read, on indexed columns, and reads the owner's other
|
|
247
|
+
tables through `ownTable(name)`, a read-only subquery, never by table name. It
|
|
248
|
+
never loads everything the subject was ever linked to into an `IN` list, and
|
|
249
|
+
never calls another owner's check once per item: it runs on every read, write
|
|
250
|
+
read-back and lookup, so every request pays what it costs. A policy is told
|
|
251
|
+
who the call runs for as `caller`: `{ kind: "user", userId }` for a signed-in
|
|
252
|
+
request, `{ kind: "anonymous" }` for one by no one, and `{ kind: "system" }` or
|
|
253
|
+
`{ kind: "job" }` (a context `forWorkspace` and its kin derived), each with its
|
|
254
|
+
`reason`, for background work. A job names a user only as `actorId`, the
|
|
255
|
+
person it was derived for (below), so never read a missing actor as an
|
|
256
|
+
anonymous request.
|
|
257
|
+
|
|
258
|
+
```js
|
|
259
|
+
// good: one predicate on the row being read
|
|
260
|
+
function noticeVisibilityPolicy({ query, caller, column, value, ownTable }) {
|
|
261
|
+
if (caller.kind !== "user") {
|
|
262
|
+
return caller.kind !== "anonymous";
|
|
263
|
+
}
|
|
264
|
+
query.where(column("recipientId"), value("recipientId", caller.userId)).whereExists(
|
|
265
|
+
ownTable("sources").select(1).whereColumn("sources.id", column("sourceId")).where("sources.status", "published")
|
|
266
|
+
);
|
|
267
|
+
return true;
|
|
268
|
+
}
|
|
269
|
+
|
|
270
|
+
// bad: every source the recipient was ever notified about, checked one by one
|
|
271
|
+
async function noticeVisibilityPolicy({ query, caller, context, column, value }) {
|
|
272
|
+
const readable = [];
|
|
273
|
+
for (const sourceId of await noticeHistory.listSourceIds(caller.userId)) {
|
|
274
|
+
if (await sources.canRead(sourceId, context)) readable.push(sourceId);
|
|
275
|
+
}
|
|
276
|
+
query.whereIn(column("sourceId"), readable.map((id) => value("sourceId", id)));
|
|
277
|
+
return true;
|
|
278
|
+
}
|
|
279
|
+
```
|
|
280
|
+
|
|
281
|
+
A rule that depends on the caller's role reads `context.permissions` with
|
|
282
|
+
`hasPermission(context.permissions, "bookings.view-all")` from
|
|
283
|
+
`@jskit-ai/kernel/shared/support`, never with `includes()`. A role may hold
|
|
284
|
+
`*` (an owner does) or `bookings.*`, and `includes()` misses both, so the
|
|
285
|
+
owner would see only the rows a narrower rule leaves.
|
|
286
|
+
|
|
287
|
+
```js
|
|
288
|
+
import { hasPermission } from "@jskit-ai/kernel/shared/support";
|
|
289
|
+
|
|
290
|
+
// good: staff see the bookings they run; bookings.view-all, the owner's * included, sees them all
|
|
291
|
+
function staffSeeTheirOwnBookings({ query, caller, context, column, value }) {
|
|
292
|
+
if (caller.kind !== "user") {
|
|
293
|
+
return caller.kind !== "anonymous";
|
|
294
|
+
}
|
|
295
|
+
if (hasPermission(context.permissions, "bookings.view-all")) {
|
|
296
|
+
return true;
|
|
297
|
+
}
|
|
298
|
+
query.where(column("staffUserId"), value("staffUserId", caller.userId));
|
|
299
|
+
return true;
|
|
300
|
+
}
|
|
301
|
+
|
|
302
|
+
// bad: an owner holding "*" sees only the bookings they run
|
|
303
|
+
if (context.permissions.includes("bookings.view-all")) return true;
|
|
304
|
+
```
|
|
305
|
+
|
|
306
|
+
A policy trusts `caller`. In a workspace request it also trusts what the
|
|
307
|
+
server resolved from the route's workspace slug and the caller's membership
|
|
308
|
+
before any handle ran:
|
|
309
|
+
- `context.workspace.id`;
|
|
310
|
+
- `context.membership.roleSid`;
|
|
311
|
+
- `context.permissions`, the role's permissions in that workspace.
|
|
312
|
+
|
|
313
|
+
On a surface whose access policy sets `requireWorkspaceMembership: false`, a
|
|
314
|
+
caller who is not a member has `context.membership` null and
|
|
315
|
+
`context.permissions` empty. Never decide on a value from the request's input.
|
|
316
|
+
|
|
317
|
+
A row policy hides a row everywhere, also as the target of a visible record's
|
|
318
|
+
link: an order whose supplier the reader cannot see reads `supplierId: null`.
|
|
319
|
+
A form that saves the order back sends that null, and the write keeps the
|
|
320
|
+
stored link: a null for a link whose target exists but is hidden from the
|
|
321
|
+
writer counts as unchanged. Send that null, never the hidden id: a write naming
|
|
322
|
+
a target the writer cannot see is refused as a missing target, since accepting
|
|
323
|
+
the right hidden id would tell the writer which record is linked. The link is
|
|
324
|
+
still unreadable to that reader, so a row policy says who may see a row at all;
|
|
325
|
+
narrowing what one screen lists is a filter on that screen's read.
|
|
326
|
+
|
|
327
|
+
```js
|
|
328
|
+
// bad: a row policy to list only the suppliers a user added; every order whose
|
|
329
|
+
// supplier someone else added now reads supplierId: null for that user
|
|
330
|
+
owns: [{ resource: supplierResource, rowPolicy: addedByActor }]
|
|
331
|
+
|
|
332
|
+
// good: the screen's read filters; orders keep their suppliers
|
|
333
|
+
function listAddedByMe(params, options) {
|
|
334
|
+
return suppliers.query({ ...params, filters: { ...params.filters, addedBy: resolveActorId(options.context) } }, options);
|
|
335
|
+
}
|
|
336
|
+
```
|
|
337
|
+
|
|
338
|
+
## Layer 2: the CRUD surface
|
|
339
|
+
|
|
340
|
+
When people list, view, create, edit or delete the rows over HTTP, add a CRUD
|
|
341
|
+
contract to the same table resource. It says which operations exist and which
|
|
342
|
+
fields each one takes and returns; `defineCrudFeature` builds the standard
|
|
343
|
+
repository, service, actions, routes and screens from it. A table without
|
|
344
|
+
screens skips this layer and still uses Layer 1. A table never gets a second
|
|
345
|
+
resource for its CRUD. [CRUD operations](crud-operations.md)
|
|
346
|
+
covers this layer in depth.
|
|
347
|
+
|
|
348
|
+
```js
|
|
349
|
+
const receivalCrud = defineCrudContract(receivalResource, {
|
|
350
|
+
namespace: "receivals",
|
|
351
|
+
apiAccess: "authenticated",
|
|
352
|
+
operations: ["list", "view", "create", "patch"],
|
|
353
|
+
fields: {
|
|
354
|
+
status: { output: "required", create: "required", patch: "optional" },
|
|
355
|
+
arrivedAt: { output: "optional" }
|
|
356
|
+
}
|
|
357
|
+
});
|
|
358
|
+
```
|
|
359
|
+
|
|
360
|
+
A contract shows and accepts only the fields it names, and so does an included
|
|
361
|
+
record: it shows its id, its relation's `labelKey`, and only what the including
|
|
362
|
+
contract lists, a record it links to only when the list names that
|
|
363
|
+
relationship, at any depth. Another table's contract never shapes it, since
|
|
364
|
+
that contract may be served to another surface or behind another permission:
|
|
365
|
+
|
|
366
|
+
```js
|
|
367
|
+
fields: {
|
|
368
|
+
ownerId: { output: { required: false, includeFields: ["displayName"] } },
|
|
369
|
+
// ?include=pet,pet.breed shows the pet's name and its breed's name
|
|
370
|
+
petId: { output: { required: false, includeFields: ["name", { breed: ["name"] }] } }
|
|
371
|
+
}
|
|
372
|
+
```
|
|
373
|
+
A `decorateService` that also needs the record it wrote (for an audit or a
|
|
374
|
+
history row) reads it from its own answer, `simplifyJsonApiResult(answer)`
|
|
375
|
+
from `@jskit-ai/http-runtime/shared`, never with a second read; a relationship
|
|
376
|
+
whose field key is not its name plus `Id` passes
|
|
377
|
+
`{ lookupFieldMap: { owner: "ownerUserId" } }`.
|
|
378
|
+
A second surface on the same table (a customer's view beside the staff's) is
|
|
379
|
+
a second CRUD contract on the same resource, never a second resource, with
|
|
380
|
+
its own namespace. One plain feature owns the table and provides its handle.
|
|
381
|
+
Each surface is a `defineCrudFeature` that serves its contract through that
|
|
382
|
+
handle with `ownerRepository`. A CRUD feature that owns the table provides
|
|
383
|
+
`{ repository, crud, service }` rather than a handle, so it cannot be another
|
|
384
|
+
one's `ownerRepository`.
|
|
385
|
+
|
|
386
|
+
```js
|
|
387
|
+
const ServicesFeature = defineFeature({
|
|
388
|
+
id: "app.services",
|
|
389
|
+
owns: [serviceResource],
|
|
390
|
+
provides: { services: "app.services" },
|
|
391
|
+
setup: ({ resources }) => ({ services: resources.services })
|
|
392
|
+
});
|
|
393
|
+
const ServiceAdminFeature = defineCrudFeature({
|
|
394
|
+
crud: serviceAdminCrud,
|
|
395
|
+
surface: "admin",
|
|
396
|
+
ownerRepository: "app.services",
|
|
397
|
+
scope: WORKSPACE_CRUD_SCOPE,
|
|
398
|
+
relativePath: "/services"
|
|
399
|
+
});
|
|
400
|
+
const ServiceCatalogFeature = defineCrudFeature({
|
|
401
|
+
crud: serviceCatalogCrud,
|
|
402
|
+
surface: "app",
|
|
403
|
+
ownerRepository: "app.services",
|
|
404
|
+
scope: WORKSPACE_CRUD_SCOPE,
|
|
405
|
+
relativePath: "/service-catalog"
|
|
406
|
+
});
|
|
407
|
+
```
|
|
408
|
+
|
|
409
|
+
`WORKSPACE_CRUD_SCOPE` is from
|
|
410
|
+
`@jskit-ai/workspaces-core/server/support/workspaceCrud`; the two contracts'
|
|
411
|
+
namespaces differ because `crud.contracts` refuses two contracts under one
|
|
412
|
+
namespace. A field
|
|
413
|
+
fragment several resources spread into their schemas carries only the field:
|
|
414
|
+
each contract that shows it states its operations in its own `fields`.
|
|
415
|
+
|
|
416
|
+
## Layer 3: many rows at once
|
|
417
|
+
|
|
418
|
+
When one action creates, changes or deletes many rows, use the handle's
|
|
419
|
+
many-row operations: `createMany`, `patchMany`, `patchWhere` and `deleteWhere`.
|
|
420
|
+
Every filter takes `id`, one id or a list, so `patchWhere({ id: ids },
|
|
421
|
+
changes, options)` and `deleteWhere({ id: ids }, options)` change a known
|
|
422
|
+
list. Each checks every row against the resource first, reads which of the
|
|
423
|
+
rows the caller may touch, then writes each chunk with one statement, in one
|
|
424
|
+
transaction. Audits and change events run once for the batch, so the
|
|
425
|
+
statements are the same for 10 rows as for 200. `patchWhere` and
|
|
426
|
+
`deleteWhere` check the scope and filters on each row once they hold its lock,
|
|
427
|
+
so a row gone meanwhile, or changed so that it no longer matches, is skipped,
|
|
428
|
+
and they take no `missing`. That makes them a compare-and-set on their filters
|
|
429
|
+
on every database: of two writers whose filters both matched a row (open to
|
|
430
|
+
booked), only one writes it, and the other's answer says so
|
|
431
|
+
(`returning: "ids"` lists only the rows it wrote). The one that waited for the
|
|
432
|
+
other's lock leaves the row out; SQLite gets there by failing the second writer
|
|
433
|
+
with `SQLITE_BUSY`, and MariaDB with `innodb_snapshot_isolation` fails the
|
|
434
|
+
waiter with errno 1020 ("Record has changed since last read"), which a write
|
|
435
|
+
that owns its transaction runs again. For one row,
|
|
436
|
+
`patch(id, changes, { ...options, onlyIf: { status: "open" } })` does the same.
|
|
437
|
+
A filter on another table (a search field whose `actualField` reaches a
|
|
438
|
+
related table) is refused on MySQL and MariaDB (`RestApiValidationError`, rule
|
|
439
|
+
`selection_cross_table`), even when it matches no row. A search entry's own
|
|
440
|
+
`applyFilter` that reads another table through a subquery is not detected, and
|
|
441
|
+
is no compare-and-set there either. Filter on the row's own columns instead, for example
|
|
442
|
+
on the link after locking its targets with `lockMany` in the same transaction
|
|
443
|
+
(Layer 4), or move the value onto the row itself for `onlyIf`, or use a
|
|
444
|
+
`claim`. A many-row write answers
|
|
445
|
+
nothing unless `returning` asks: `"ids"`, `"records"`, or `"count"`, which
|
|
446
|
+
reads nothing back.
|
|
447
|
+
|
|
448
|
+
Bad: a batch intake patched each record through the resource: 705 statements
|
|
449
|
+
for 10 records, 11,915 for 200. Written with `patchMany`, it sends the same
|
|
450
|
+
number at both sizes.
|
|
451
|
+
|
|
452
|
+
An upsert (`createMany(rows, { ...options, onConflict })`) on an owned table
|
|
453
|
+
names the ownership field among its conflict keys, or it throws: otherwise it
|
|
454
|
+
could update or hide another workspace's row. `merge` is refused under a row
|
|
455
|
+
policy; use `ignore`, then `patchMany`. A table with no owners and no row
|
|
456
|
+
policy, keyed by a natural id such as an email, may key the upsert on the id
|
|
457
|
+
alone: `onConflict: { keys: ["id"], action: "merge" }`, with an id on every
|
|
458
|
+
row, creates the record or replaces the fields given in one statement. Of two
|
|
459
|
+
upserts at once the later stays, so a counter that must count both is created
|
|
460
|
+
plainly and its duplicate retried instead. Inserting and catching the
|
|
461
|
+
duplicate works only outside a transaction: inside one, the failed insert
|
|
462
|
+
fails the whole transaction, so use an upsert there.
|
|
463
|
+
|
|
464
|
+
A transaction whose insert loses a race for a unique key (another request with
|
|
465
|
+
the same idempotency key committed first) runs once more:
|
|
466
|
+
`retryLostInsertRace(options, () => repository.withTransaction(options, work))`
|
|
467
|
+
from `@jskit-ai/database-runtime/shared`. The database reports the duplicate
|
|
468
|
+
only once the winner has committed, so the second run's lookup by the key
|
|
469
|
+
finds the winner's row. A second duplicate is final, and inside a caller's
|
|
470
|
+
transaction nothing runs again: the caller's transaction has failed, and the
|
|
471
|
+
caller's own retry runs the whole of it. Never write this retry by hand.
|
|
472
|
+
|
|
473
|
+
```js
|
|
474
|
+
// A repeated payment request answers the first one's record.
|
|
475
|
+
const recordPayment = (input, options) => retryLostInsertRace(options, () => payments.withTransaction(options, async (txOptions) => {
|
|
476
|
+
const [paid] = await payments.queryAll({ filters: { requestKey: input.requestKey } }, txOptions);
|
|
477
|
+
return paid ?? payments.create(input, txOptions);
|
|
478
|
+
}));
|
|
479
|
+
```
|
|
480
|
+
|
|
481
|
+
## Layer 4: locks
|
|
482
|
+
|
|
483
|
+
When two requests could change the same rows at once (allocating stock,
|
|
484
|
+
numbering documents, moving items into a batch), lock the rows first, inside
|
|
485
|
+
the transaction: `lock(id, txOptions)`, `lockMany(ids, txOptions)` or
|
|
486
|
+
`lockWhere(filters, txOptions)`. The handle locks only rows the caller may
|
|
487
|
+
see, in id order, so nobody locks another workspace's rows: a missing or
|
|
488
|
+
hidden id throws not-found, as for `get`, or with `missing: "skip"` is left
|
|
489
|
+
out. A check-then-write that must not race (is the drier empty? then load it)
|
|
490
|
+
locks the record first, so another lock or write of it waits until the
|
|
491
|
+
transaction ends. Locks cover existing rows only: number with a unique index
|
|
492
|
+
or a counter row taken with `lock(id)`. Every lock needs a transaction; SQLite
|
|
493
|
+
runs one transaction at a time, so there is no row lock to take. Take locks in
|
|
494
|
+
one order on every path, a parent before its children.
|
|
495
|
+
|
|
496
|
+
On MySQL and MariaDB a locking read locks every row it examines, so two rules
|
|
497
|
+
apply there. Lock ids taken from your own scoped reads, never a list straight
|
|
498
|
+
from a request: every id listed is locked, a hidden one too, before it is left
|
|
499
|
+
out. And `lockWhere` needs an index that starts with the ownership columns (on
|
|
500
|
+
a table with no owners, with a column the filters compare); without one it
|
|
501
|
+
refuses with `RestApiBulkWriteUnsupportedError` naming the index to add in a
|
|
502
|
+
migration.
|
|
503
|
+
|
|
504
|
+
A worker taking the next job uses `claim({ filters, sort, take }, txOptions)`,
|
|
505
|
+
which skips rows another worker holds. A record that serializes work on a key
|
|
506
|
+
and may not exist yet (an account's balance, a connection to a provider), on a
|
|
507
|
+
table with no owners, takes `lockOrCreate({ id, ...initial }, options, work)`:
|
|
508
|
+
it locks the record, creating it from the row first when it is missing, and
|
|
509
|
+
runs `work(record, txOptions)` in the same transaction, so failed work leaves
|
|
510
|
+
no record behind. On its own it runs work once, so work may call a provider:
|
|
511
|
+
two first uses of one id at once can collide on the insert, and the loser runs
|
|
512
|
+
again before its work starts.
|
|
513
|
+
|
|
514
|
+
A record that a person edits from a form it read earlier carries a version
|
|
515
|
+
number: the table resource names it with `lockVersionField`, an integer field
|
|
516
|
+
over a non-null integer column with `defaultTo: 0`.
|
|
517
|
+
`patch(id, changes, { ...options, lockVersion })` then locks the row, applies
|
|
518
|
+
`changes` only while it still holds that version, and advances it. A record
|
|
519
|
+
whose version moved on rejects with a 409 coded
|
|
520
|
+
`crud.document.lockVersionConflict` ("This record changed. Reload and try
|
|
521
|
+
again.") and nothing is written; a missing or hidden record follows the
|
|
522
|
+
missing-row rule. The changes leave the version field out. Every update
|
|
523
|
+
advances the version, a plain `patch`, `patchMany`, `patchWhere` or upsert
|
|
524
|
+
merge too, so a form that read version 3 and saves after a background job's
|
|
525
|
+
plain patch of the same record (now version 4) gets the 409 and reloads, even
|
|
526
|
+
though the job changed other fields. `patch(id, changes, { ...options, onlyIf:
|
|
527
|
+
{ status: "open" } })` is the same check on stored values, answering `null`
|
|
528
|
+
when they differ.
|
|
529
|
+
|
|
530
|
+
## Layer 5: reports the resource cannot express
|
|
531
|
+
|
|
532
|
+
Some reads are reports, not records: totals by status and month, the newest
|
|
533
|
+
row per group, distinct active days. Try `aggregate` (counts, sums, buckets by
|
|
534
|
+
day, week, month or year) and an include with `strategy: "window"` (newest per
|
|
535
|
+
group) first. A window include is declared on the table resource's
|
|
536
|
+
relationship, never in the query, and read with `include: ["snapshots"]`:
|
|
537
|
+
|
|
538
|
+
```js
|
|
539
|
+
relationships: {
|
|
540
|
+
snapshots: { type: "hasMany", target: "snapshots", foreignKey: "booking_id", include: { strategy: "window", limit: 1, orderBy: ["-revision"] } }
|
|
541
|
+
}
|
|
542
|
+
```
|
|
543
|
+
|
|
544
|
+
When neither can express it, the owner's repository builds it from
|
|
545
|
+
`scopedQuery()`, the table's builder with ownership and the row policy already
|
|
546
|
+
applied:
|
|
547
|
+
|
|
548
|
+
```js
|
|
549
|
+
// Knex: report - the days each member was active, counted per member
|
|
550
|
+
async function activeDays(memberIds, options) {
|
|
551
|
+
const { query } = await visits.scopedQuery({}, options);
|
|
552
|
+
return query
|
|
553
|
+
.select({ memberId: visits.column("memberId") })
|
|
554
|
+
.countDistinct({ days: visits.raw("DATE(??)", [visits.column("visitedAt")]) })
|
|
555
|
+
.whereIn(visits.column("memberId"), memberIds)
|
|
556
|
+
.groupBy(visits.column("memberId"));
|
|
557
|
+
}
|
|
558
|
+
```
|
|
559
|
+
|
|
560
|
+
- Each report is one named method in its owner's repository file
|
|
561
|
+
(`*Repository.js`, `repository.js`, or a file under `repositories/`;
|
|
562
|
+
`jskit/knex-reason` refuses `scopedQuery()` anywhere else), under
|
|
563
|
+
`// Knex: <kind> - <reason>`. The kinds:
|
|
564
|
+
- `totals`: sums or counts `aggregate` cannot group;
|
|
565
|
+
- `report`: rows shaped across tables;
|
|
566
|
+
- `newest-per-group`: one row per group, such as each customer's latest
|
|
567
|
+
visit;
|
|
568
|
+
- `subquery`: a scoped set another query joins;
|
|
569
|
+
- `hot-path`: a measured path the handle makes too slow, with its JSKIT
|
|
570
|
+
ticket (`// Knex: hot-path #123 - <reason>`);
|
|
571
|
+
- `runtime-tables`: tables people define while the app runs, which no table
|
|
572
|
+
resource in the code describes.
|
|
573
|
+
- `column()` names columns; `raw` takes one expression as a string literal,
|
|
574
|
+
with its values as bindings.
|
|
575
|
+
- The builder only reads, and only its own table is scoped. A `.join()` of a
|
|
576
|
+
table by name, or a table named inside `raw()`, reads that table's rows past
|
|
577
|
+
its owner and row policy. Nothing at run time stops it: `jskit/no-direct-knex`
|
|
578
|
+
and `jskit/sql-literal` flag it, and `expectScoped` catches it only when the
|
|
579
|
+
test seeds another owner's rows the join matches. Join another table only as
|
|
580
|
+
that table's scoped subquery.
|
|
581
|
+
- When a report needs another owner's rows side by side, that owner adds a
|
|
582
|
+
method answering its scoped rows as a subquery with its own column names,
|
|
583
|
+
and the report joins it in one statement. The method answers
|
|
584
|
+
`{ query, column }` from its `scopedQuery({ alias }, options)`, with
|
|
585
|
+
`column(field)` naming its columns under that alias, wrapped, since a
|
|
586
|
+
builder is thenable and an async function would run it. The report shapes
|
|
587
|
+
that builder (select, group), names it with `.as()` and joins it, so every
|
|
588
|
+
owner's scope and row policy stay applied. The report lives with the main
|
|
589
|
+
table's owner, or in a feature that requires both. A join source carries
|
|
590
|
+
every column, hidden ones too, so an owner answers one only to a feature
|
|
591
|
+
that may read them.
|
|
592
|
+
|
|
593
|
+
```js
|
|
594
|
+
// billing's owner API: its scoped invoices as a join source, with their column names
|
|
595
|
+
async function invoiceSource(alias, options) {
|
|
596
|
+
const { query } = await invoices.scopedQuery({ alias }, options);
|
|
597
|
+
return { query, column: (field) => invoices.column(field, alias) };
|
|
598
|
+
}
|
|
599
|
+
|
|
600
|
+
// Knex: report - open tasks and billed totals per project, joined on billing's scoped invoices
|
|
601
|
+
async function projectSummary(options) {
|
|
602
|
+
const source = await billing.invoiceSource("i", options);
|
|
603
|
+
const billed = source.query
|
|
604
|
+
.select({ projectId: source.column("projectId") })
|
|
605
|
+
.sum({ billed: source.column("amount") })
|
|
606
|
+
.groupBy(source.column("projectId"))
|
|
607
|
+
.as("b");
|
|
608
|
+
const { query } = await tasks.scopedQuery({ alias: "t" }, options);
|
|
609
|
+
const projectId = tasks.column("projectId", "t");
|
|
610
|
+
return query
|
|
611
|
+
.leftJoin(billed, "b.projectId", projectId)
|
|
612
|
+
.select({ projectId })
|
|
613
|
+
.count({ open: tasks.column("id", "t") })
|
|
614
|
+
.max({ billed: "b.billed" })
|
|
615
|
+
.groupBy(projectId);
|
|
616
|
+
}
|
|
617
|
+
```
|
|
618
|
+
|
|
619
|
+
- When each owner can answer on its own (keys in, grouped totals out), the
|
|
620
|
+
report is a feature of its own that calls each owner's method and merges the
|
|
621
|
+
answers in memory: one statement per owner, never a call per row.
|
|
622
|
+
- Give each report a scope test and a flat-query test (see [Tests](#tests)).
|
|
623
|
+
|
|
624
|
+
## Reads shaped for the screen
|
|
625
|
+
|
|
626
|
+
- A list reads one page with `query`; it needs `page.size`, at most the
|
|
627
|
+
resource's `queryMaxLimit`. A scrolling list passes the previous page's
|
|
628
|
+
`page.next` as `page.after`; numbered pages, or restoring a known page, pass
|
|
629
|
+
`page.number` (from 1) and answer `page: { size, number, hasMore, next }`.
|
|
630
|
+
`page.number` reads with OFFSET, which slows as the rows grow, so anything
|
|
631
|
+
that grows pages with `page.after`. Each page is one statement, with no
|
|
632
|
+
count: never walk pages to reach one.
|
|
633
|
+
- A screen list never uses `queryAll`.
|
|
634
|
+
- Related rows come by id list (`filters: { id: ids }`, or an `in` filter on
|
|
635
|
+
another field) or `include`, grouped in memory. Never a query per row, however
|
|
636
|
+
the loop is spelled (`for`, `map` with `Promise.all`, a helper called per
|
|
637
|
+
item).
|
|
638
|
+
- `queryAll` reads every matching row, in cursor pages that skip and repeat
|
|
639
|
+
none. Use it for id lists, one parent's rows and jobs: a read its callers
|
|
640
|
+
treat as complete, which a `query` would silently cut at one page. In a
|
|
641
|
+
request it stops past the request row limit (10,000 rows) with a 400 coded
|
|
642
|
+
`RESOURCE_ROW_LIMIT`.
|
|
643
|
+
|
|
644
|
+
```js
|
|
645
|
+
// good: one page of invoices, then all of that page's lines in one read
|
|
646
|
+
const { records, page: next } = await invoices.query({ sort: ["-issuedAt"], page }, options);
|
|
647
|
+
const lines = await invoiceLines.queryAll({ filters: { invoiceIds: records.map((invoice) => invoice.id) } }, options);
|
|
648
|
+
const linesByInvoice = Map.groupBy(lines, (line) => line.invoiceId);
|
|
649
|
+
|
|
650
|
+
// bad: one more read, and its row policy, for every invoice
|
|
651
|
+
for (const invoice of records) {
|
|
652
|
+
invoice.lines = await invoiceLines.queryAll({ filters: { invoiceIds: [invoice.id] } }, options);
|
|
653
|
+
}
|
|
654
|
+
|
|
655
|
+
// good: a delivery job reads every active subscriber
|
|
656
|
+
const subscribers = await subscriptions.queryAll({ filters: { newsletterId, status: "active" } }, options);
|
|
657
|
+
|
|
658
|
+
// bad: the first page only, whatever its size
|
|
659
|
+
const { records: firstPage } = await subscriptions.query({ filters: { newsletterId, status: "active" }, page: { size: 20 } }, options);
|
|
660
|
+
```
|
|
661
|
+
|
|
662
|
+
Bad: a list read 100-row pages in a loop for each of 40 related sets.
|
|
663
|
+
|
|
664
|
+
## Background work
|
|
665
|
+
|
|
666
|
+
Work over the whole population runs as a background job, never inside one
|
|
667
|
+
person's request. Its feature declares why with `systemScope`, and its setup
|
|
668
|
+
receives `systemContext(reason)`, which names its reason at every call:
|
|
669
|
+
|
|
670
|
+
```js
|
|
671
|
+
const SettlementFeature = defineFeature({
|
|
672
|
+
id: "leagues.settlement",
|
|
673
|
+
owns: [cohortResource, memberResource],
|
|
674
|
+
systemScope: "settle finished weekly leagues for every user",
|
|
675
|
+
setup: ({ resources, systemContext }) => ({ settlement: createSettlement({ resources, systemContext }) })
|
|
676
|
+
});
|
|
677
|
+
```
|
|
678
|
+
|
|
679
|
+
- Prefer `systemContext(reason).forWorkspace(id)`, `.forUser(id)` or
|
|
680
|
+
`.forWorkspaceUser(workspaceId, userId)`: each workspace or user is then one
|
|
681
|
+
unit of work, scoped as usual, in a transaction of its own. An event it
|
|
682
|
+
publishes carries that workspace's or user's scope, as a request there would,
|
|
683
|
+
so it reaches that audience, never everyone. An action it runs with
|
|
684
|
+
`actions.execute({ actionId, input, context })` keeps the context's kind:
|
|
685
|
+
inside it, a handle call still reads as the system context or job.
|
|
686
|
+
- Work a job does for one person (a request it carries out later) passes
|
|
687
|
+
`{ actorId }` last: `forWorkspace(id, { actorId: requestedBy })`. It is for
|
|
688
|
+
attribution only: audit fields read it with `resolveAuditActorId(context)`
|
|
689
|
+
(the signed-in user, else the person a job acts for) and the row policy as
|
|
690
|
+
`caller.actorId`. It never changes which rows the job reaches and signs no
|
|
691
|
+
one in: `resolveActorId` and every `authenticated` check still see no one.
|
|
692
|
+
Write audit fields with `resolveAuditActorId`, and decide access with
|
|
693
|
+
`resolveActorId`.
|
|
694
|
+
- A job finds its units with one `aggregate` grouped by the ownership field,
|
|
695
|
+
which a system context may group by (and count distinct) even when the table
|
|
696
|
+
hides it, then runs each unit scoped to it. A request's aggregate still
|
|
697
|
+
refuses a hidden field:
|
|
698
|
+
|
|
699
|
+
```js
|
|
700
|
+
async function settleDue(options) {
|
|
701
|
+
const due = await cohorts.aggregate({ measures: { cohorts: { count: "id" } }, groupBy: ["workspaceId"], filters: { status: "finished" } }, options);
|
|
702
|
+
for (const { workspaceId } of due) {
|
|
703
|
+
await settleWorkspace({ context: systemContext("settle finished weekly leagues for every user").forWorkspace(workspaceId) });
|
|
704
|
+
}
|
|
705
|
+
}
|
|
706
|
+
```
|
|
707
|
+
- A job that must learn each row's owner across workspaces names the hidden
|
|
708
|
+
ownership field in its read's `fields`, `queryAll({ filters, fields:
|
|
709
|
+
["workspaceId", "email"] }, { context })`, as a system context may do through
|
|
710
|
+
another feature's CRUD repository too (see [Layer
|
|
711
|
+
1](#layer-1-the-table-resource)). Never unhide it for the job.
|
|
712
|
+
- A request uses its own context. The only exception is a console report
|
|
713
|
+
behind a console permission, with bounded output.
|
|
714
|
+
- `jskit check` lists every system scope and every `systemContext` call with
|
|
715
|
+
its reason, read from the string literal at the call: a reason kept in a
|
|
716
|
+
constant reads as none.
|
|
717
|
+
- A system context comes only from `systemScope`, and a context's scope only
|
|
718
|
+
from the request or from `forWorkspace`, `forUser` and `forWorkspaceUser`:
|
|
719
|
+
lint refuses `createSystemContext` and hand-written `scopeValues`, and a
|
|
720
|
+
context naming two workspaces or two users throws.
|
|
721
|
+
- A job that runs on a schedule is a feature whose `boot()` starts a timer and
|
|
722
|
+
whose `shutdown()` stops it; JSKIT ships no scheduler. See
|
|
723
|
+
[typed features, Owned state](typed-features.md#owned-state).
|
|
724
|
+
Every server runs the timer, so a run takes its rows with `claim()`.
|
|
725
|
+
|
|
726
|
+
Bad: a job read every record of every workspace inside a request path, every
|
|
727
|
+
hour.
|
|
728
|
+
|
|
729
|
+
## Transactions
|
|
730
|
+
|
|
731
|
+
Open a transaction with the repository's `withTransaction(options, work)` and
|
|
732
|
+
pass the `txOptions` it gives `work` to every call inside, spread over the
|
|
733
|
+
method's own `options` for the write the caller's settings are for. Never read the
|
|
734
|
+
transaction out of `options` by hand. A handle refuses every other
|
|
735
|
+
transaction, a raw Knex one or a savepoint, for reads as for writes. Inside an
|
|
736
|
+
open transaction it joins it; otherwise it opens one and runs events and other
|
|
737
|
+
after-effects after commit. Any failed write inside it fails the whole
|
|
738
|
+
transaction.
|
|
739
|
+
|
|
740
|
+
A deadlock victim's transaction (MySQL and MariaDB errno 1213, PostgreSQL
|
|
741
|
+
`40P01`), and one MariaDB fails as "Record has changed since last read" (errno
|
|
742
|
+
1020, with `innodb_snapshot_isolation` on), runs its whole work again, up to
|
|
743
|
+
three more times, so the work must
|
|
744
|
+
be safe to repeat: an HTTP call or an email goes after commit, or into an
|
|
745
|
+
outbox row written inside it. Work that cannot be repeated passes `{ ...options, retry: false }`, and work that must
|
|
746
|
+
commit before its caller goes on passes `{ ...options, outermost: true }`.
|
|
747
|
+
Code never asks whether it runs in a transaction: `withTransaction` joins the
|
|
748
|
+
caller's, `outermost` refuses one, and `lock`, `lockMany`, `lockWhere` and
|
|
749
|
+
`claim` refuse to run outside one; `lockOrCreate` opens its own (and retries a
|
|
750
|
+
lost insert race only then).
|
|
751
|
+
|
|
752
|
+
When the work throws, the transaction rolls back and the caller gets the
|
|
753
|
+
work's own error, with its status and code, reporting `transactionOutcome:
|
|
754
|
+
"rolledBack"`, so `retryLostInsertRace` (Layer 3) also runs again work that
|
|
755
|
+
turned the duplicate into its own 409. A failed write inside the work, or a failed commit, reaches
|
|
756
|
+
the caller as json-rest-api's `RestApiWriteError` with the transaction's
|
|
757
|
+
outcome. It carries the failure's own `code` and `status` (a driver's
|
|
758
|
+
`ER_CHECKREAD`, a not-found's 404), and `REST_API_WRITE` only when the failure
|
|
759
|
+
has no code. Classify an error by its `code` or `status`,
|
|
760
|
+
never with `instanceof`: a second copy of a package makes a second class.
|
|
761
|
+
|
|
762
|
+
A database error never reaches a client. Its message carries the statement
|
|
763
|
+
and its values, so JSKIT's HTTP handler and the assistant's stream answer one,
|
|
764
|
+
thrown as it is or as a wrapper's cause, with a stand-in
|
|
765
|
+
(`hideDatabaseError` in `@jskit-ai/kernel/server/runtime`) and log it in full:
|
|
766
|
+
409 `kernel.database.concurrentWrite` when the database refused the transaction for a concurrent one
|
|
767
|
+
(a deadlock or lock wait whose retries ran out, a row changed since the
|
|
768
|
+
transaction's snapshot, a serialization failure), which the client may send
|
|
769
|
+
again, else 500. To give a duplicate key or another database refusal its own
|
|
770
|
+
answer, throw an `AppError` (a `ConflictError`) with the driver's error as its
|
|
771
|
+
`cause`: an `AppError` keeps its answer whatever it wraps.
|
|
772
|
+
|
|
773
|
+
```js
|
|
774
|
+
const allocate = (input, options) => repository.withTransaction(options, async (txOptions) => {
|
|
775
|
+
const invoices = await repository.lockOpenInvoices(input.invoiceIds, txOptions);
|
|
776
|
+
return repository.recordAllocations(allocationsFor(invoices, input.amount), txOptions);
|
|
777
|
+
});
|
|
778
|
+
```
|
|
779
|
+
|
|
780
|
+
## Tables you don't own
|
|
781
|
+
|
|
782
|
+
Ask the owner: its service, or for a JSKIT table its owner API, with the same
|
|
783
|
+
`options`. JSKIT's packages expose their tables this way: users
|
|
784
|
+
(`users.profiles`, `users.settings`), workspaces (`workspaces`,
|
|
785
|
+
`workspaces.memberships`), local sign-in (`auth.local.accounts`,
|
|
786
|
+
`auth.local.sessions`, `auth.local.recovery`) and the console
|
|
787
|
+
(`console.settings`). A realtime audience reaches its users the same way: its
|
|
788
|
+
`audienceUsers({ event, options })` asks the owners and answers user ids, in the
|
|
789
|
+
event's scope: for a customer's or a public surface's event, the workspace the
|
|
790
|
+
request resolved, never one its payload names (see [Realtime](/guide/app-extras/realtime)).
|
|
791
|
+
|
|
792
|
+
An app that had a table resource of its own over a JSKIT table (such as
|
|
793
|
+
`users`) deletes it: its CRUD contract is built on the package's table resource
|
|
794
|
+
(users-core's `userProfileTableResource`) and served through the owner with
|
|
795
|
+
`defineCrudFeature({ crud, ownerRepository: "users.profiles" })` (the
|
|
796
|
+
`users/user-administration-server` pattern). The app's own facts about a
|
|
797
|
+
person (a staff role, a preference) live in the app's own table keyed by the
|
|
798
|
+
user id, owned by the app with ownership `user` or `workspace_user`, never as
|
|
799
|
+
new columns on `users`.
|
|
800
|
+
|
|
801
|
+
A surface over a table you don't own that may show only some of its rows (a
|
|
802
|
+
people picker limited to those who share a workspace with the caller) asks the
|
|
803
|
+
owner for exactly that read, as one owner API method that narrows with one
|
|
804
|
+
predicate over the owner's own tables. It never contributes the rule to the
|
|
805
|
+
owner's row policy, which narrows every read of the table (sign-in's too), and
|
|
806
|
+
never passes an id list of everyone the caller may see: an id list is fine for
|
|
807
|
+
the rows one screen shows, not for a rule over thousands of people. An owner
|
|
808
|
+
without such a method gets one added at the owner; until then the surface uses
|
|
809
|
+
the owner's own listing for that scope, such as
|
|
810
|
+
`workspaces.memberships.listActiveByWorkspaceId(workspaceId, options)`, which
|
|
811
|
+
reads the members with their users in one statement.
|
|
812
|
+
|
|
813
|
+
Bad: an app wrote sign-in sessions and memberships with Knex; the framework's
|
|
814
|
+
own checks and audits never ran.
|
|
815
|
+
|
|
816
|
+
## Two features that need each other
|
|
817
|
+
|
|
818
|
+
When a parent's owner and its children's owners each need the other (the
|
|
819
|
+
parent's service reads its children, and a child's write must advance the
|
|
820
|
+
parent's version), split what both need into a feature of its own, so
|
|
821
|
+
capabilities point one way:
|
|
822
|
+
|
|
823
|
+
- The parent table's owner keeps the table and provides a small owner API,
|
|
824
|
+
such as `versions.advance(id, txOptions)`, and requires none of the
|
|
825
|
+
children.
|
|
826
|
+
- Each child's owner requires that API and calls it in its own write's
|
|
827
|
+
transaction, from its repository's write method. An `owns` entry's
|
|
828
|
+
`afterWrite` hook cannot do it: it is declared with the table, before any
|
|
829
|
+
capability arrives.
|
|
830
|
+
- The aggregate that reads or changes them together (the parent with its
|
|
831
|
+
children) is a third feature that requires both.
|
|
832
|
+
|
|
833
|
+
A version that moves on with no field changed is the parent's
|
|
834
|
+
`lockVersionField`: `lockMany(ids, txOptions)` locks the parents, and
|
|
835
|
+
`patchMany(ids.map((id) => ({ id })), { ...txOptions, returning: "none" })`
|
|
836
|
+
advances each by one, in one statement, whatever the field is called. An
|
|
837
|
+
advance is an update, so it stamps the parent's `updatedAt` too: the parent as
|
|
838
|
+
a whole changed, and a reader that compares either one sees it. A condition
|
|
839
|
+
named with no value, such as `lockVersion: record.lockVersion` on a field
|
|
840
|
+
called `version`, is refused (`options.lockVersion is undefined`), never run
|
|
841
|
+
without it.
|
|
842
|
+
|
|
843
|
+
## Check and judge
|
|
844
|
+
|
|
845
|
+
Before calling data work done, apply the
|
|
846
|
+
[sanity checks](sanity-checks.md#query-counts-and-batches) with
|
|
847
|
+
JSKIT's tools: `expectFlatQueries` for statements that grow with rows,
|
|
848
|
+
`expectScoped` for every read and write path, and a look at each `// Knex:`
|
|
849
|
+
method, which `aggregate` or an include might do instead.
|
|
850
|
+
|
|
851
|
+
## Tests
|
|
852
|
+
|
|
853
|
+
Each data path gets two one-line tests on the real test database, with the
|
|
854
|
+
helpers from `@jskit-ai/resource-runtime/test`:
|
|
855
|
+
|
|
856
|
+
```js
|
|
857
|
+
import { expectEveryTableOwned, expectFlatQueries, expectScoped, startTestFeatures } from "@jskit-ai/resource-runtime/test";
|
|
858
|
+
|
|
859
|
+
test("allocation stays in its workspace", () => expectScoped({ environment: process.env, handles: [invoices, payments], systemContext, owners: [workspaceA, workspaceB], seed: seedInvoices, run: (owner) => billing.allocate(input, owner) }));
|
|
860
|
+
test("allocation is flat", () => expectFlatQueries({ environment: process.env, knex, seed: seedInvoices, run: (seeded) => billing.allocate(inputFor(seeded), seeded.options) }));
|
|
861
|
+
```
|
|
862
|
+
|
|
863
|
+
`expectFlatQueries` (from `@jskit-ai/resource-runtime/test`) seeds each size,
|
|
864
|
+
counts the statements the path sends, and fails when a larger size sends more,
|
|
865
|
+
with the counts and the statements repeated most; the later chunks a many-row
|
|
866
|
+
write reports are allowed. `expectScoped` runs the path as owner A before and
|
|
867
|
+
after seeding owner B and fails when the answer changes, so it catches a report
|
|
868
|
+
that sums, counts or joins B's rows, not only one that returns them; it also
|
|
869
|
+
fails when the path changed or removed a row of B's. The path runs twice, so it
|
|
870
|
+
answers the same for the same rows: a write answers what repeats, such as the
|
|
871
|
+
ids it changed, not a fresh id or time.
|
|
872
|
+
|
|
873
|
+
An owner is whatever `seed` and `run` receive. On a workspace path it is a
|
|
874
|
+
signed-in member's options, built as the request pipeline builds them:
|
|
875
|
+
`{ context: { actor: { id: userId }, workspace: { id: workspaceId }, permissions } }`,
|
|
876
|
+
where `permissions` holds the member's role permissions. Seed with a job's
|
|
877
|
+
context, `{ context: systemContext(reason).forWorkspace(workspaceId) }`. A row
|
|
878
|
+
policy sees that context as `caller.kind` `"job"`, never as the member.
|
|
879
|
+
|
|
880
|
+
A data test never skips: without `TEST_DB_NAME` it fails
|
|
881
|
+
(`requireTestDatabase(process.env)`). Give `expectFlatQueries` the test's
|
|
882
|
+
`knex` whenever the features run on it (`startTestFeatures`,
|
|
883
|
+
`createTestResourceRuntime`). Without `knex` it counts through a server the
|
|
884
|
+
test booted (`countRuntimeQueries(work)` from
|
|
885
|
+
`@jskit-ai/database-runtime/server/testDatabases`). A check that measures
|
|
886
|
+
nothing never passes: "The path sent no statement at any size" means pass
|
|
887
|
+
`knex`, or the path never reached the database.
|
|
888
|
+
|
|
889
|
+
`createTestResourceRuntime({ knex })` gives a test the resource runtime its
|
|
890
|
+
features own their tables through: `runtime.own(featureId, [resource])`
|
|
891
|
+
answers the handles, and `runtime.declareSystemScope(featureId, reason)` the
|
|
892
|
+
`systemContext` function a test reads every owner's rows with. One more test
|
|
893
|
+
proves every table has an owner:
|
|
894
|
+
`expectEveryTableOwned({ knex, runtime, infrastructureTables, environment })`
|
|
895
|
+
fails, naming them, when the migrated test database holds a table no feature
|
|
896
|
+
owns and no package keeps as infrastructure. JSKIT's packages export their
|
|
897
|
+
infrastructure tables for it: `LOCAL_AUTH_INFRASTRUCTURE_TABLES`
|
|
898
|
+
(`@jskit-ai/auth-provider-local-db-core/server/lib/index`),
|
|
899
|
+
`OIDC_CLIENT_INFRASTRUCTURE_TABLES` (`@jskit-ai/auth-provider-oidc-core/server/lib/index`)
|
|
900
|
+
and `OIDC_ISSUER_INFRASTRUCTURE_TABLES` (`@jskit-ai/oidc-issuer-core/server`).
|
|
901
|
+
`runtime` knows only the features the test started, so start every owner of a
|
|
902
|
+
migrated table, JSKIT's own included, and pass `runtime: resources` from
|
|
903
|
+
`startTestFeatures`. Local sign-in's tables belong to
|
|
904
|
+
`AuthLocalDatabaseBackendProvider`
|
|
905
|
+
(`@jskit-ai/auth-provider-local-db-core/server/providers/AuthLocalDatabaseBackendProvider`,
|
|
906
|
+
given `inputs: { "runtime.database": { knex } }`).
|
|
907
|
+
`LOCAL_AUTH_INFRASTRUCTURE_TABLES` lists only the identities table it keeps
|
|
908
|
+
outside resources.
|
|
909
|
+
|
|
910
|
+
A test of a feature and the owner APIs it requires starts them as the server
|
|
911
|
+
does, without HTTP: `startTestFeatures({ knex, features, requires, inputs,
|
|
912
|
+
createTables })` orders the features by their requires, owns their tables
|
|
913
|
+
through a test resource runtime, runs every boot (so `crud.contracts` is
|
|
914
|
+
sealed) and answers `{ capabilities, resources, stop }`. JSKIT's owners start
|
|
915
|
+
beside yours: `UsersExtensionsProvider` and `UsersIdentityProvider` give
|
|
916
|
+
`users.profiles`, and `WorkspaceRecordsFeature`
|
|
917
|
+
(`@jskit-ai/workspaces-core/test/WorkspaceRecordsFeature`) gives `workspaces`
|
|
918
|
+
and `workspaces.memberships` without the workspace routes. Never import another
|
|
919
|
+
package's repositories to build its owner API by hand.
|
|
920
|
+
|
|
921
|
+
A `defineCrudFeature` also requires `crud.contracts` and `runtime.http`, which
|
|
922
|
+
an application's server supplies. In a test, add `CrudContractsProvider`
|
|
923
|
+
(`@jskit-ai/crud-core/server/crudContracts`) to `features` and give
|
|
924
|
+
`inputs: { "runtime.http": { router: { register() {} } } }`. The routes are
|
|
925
|
+
then registered nowhere, and the test calls the feature's `service` and
|
|
926
|
+
`repository` directly.
|
|
927
|
+
|
|
928
|
+
```js
|
|
929
|
+
const { capabilities, stop } = await startTestFeatures({
|
|
930
|
+
knex,
|
|
931
|
+
features: [RosterFeature, WorkspaceRecordsFeature, UsersIdentityProvider, UsersExtensionsProvider],
|
|
932
|
+
requires: { roster: "roster" }
|
|
933
|
+
});
|
|
934
|
+
t.after(stop);
|
|
935
|
+
```
|
|
936
|
+
|
|
937
|
+
A test of the routes themselves boots the application's server and signs
|
|
938
|
+
people in over `app.inject`. Keep one cookie jar per person and send the
|
|
939
|
+
session's `csrfToken` as `csrf-token` with every write. The session answer
|
|
940
|
+
carries no user id, so the test finds each person by email
|
|
941
|
+
(`users.profiles.findByEmail(email)`).
|
|
942
|
+
|
|
943
|
+
```js
|
|
944
|
+
const app = await createServer({ runtimeEnv: { ...resolveRuntimeEnv(), AUTH_LOCAL_EMAIL_CONFIRMATION: "false" } });
|
|
945
|
+
|
|
946
|
+
async function signUp(email, password) {
|
|
947
|
+
const cookies = new Map();
|
|
948
|
+
const send = async (options) => {
|
|
949
|
+
const cookie = [...cookies].map(([name, value]) => `${name}=${value}`).join("; ");
|
|
950
|
+
const response = await app.inject({ ...options, headers: { ...options.headers, cookie } });
|
|
951
|
+
for (const header of [response.headers["set-cookie"] ?? []].flat()) {
|
|
952
|
+
const [name, ...value] = header.split(";")[0].split("=");
|
|
953
|
+
cookies.set(name.trim(), value.join("="));
|
|
954
|
+
}
|
|
955
|
+
return response;
|
|
956
|
+
};
|
|
957
|
+
const csrf = async () => (await send({ method: "GET", url: "/api/session" })).json().csrfToken;
|
|
958
|
+
await send({ method: "POST", url: "/api/register", headers: { "csrf-token": await csrf() }, payload: { email, password } });
|
|
959
|
+
await send({ method: "POST", url: "/api/login", headers: { "csrf-token": await csrf() }, payload: { email, password } });
|
|
960
|
+
return send;
|
|
961
|
+
}
|
|
962
|
+
```
|
|
963
|
+
|
|
964
|
+
## Hard rules
|
|
965
|
+
|
|
966
|
+
These are always wrong, and lint reports them as errors in all code but the
|
|
967
|
+
client's, wherever it sits; tests, migrations and seeds are outside them.
|
|
968
|
+
`eslint-disable` cannot turn them off.
|
|
969
|
+
|
|
970
|
+
### Lint rules
|
|
971
|
+
|
|
972
|
+
| Rule | Reports | Section |
|
|
973
|
+
| --- | --- | --- |
|
|
974
|
+
| `jskit/no-direct-knex` | a table reached by name with Knex, a `.knex` read or `knex` destructured, a transaction opened by hand | [The resource is the only door](#the-resource-is-the-only-door) |
|
|
975
|
+
| `jskit/no-transaction-reader` | `options.trx`, `trx \|\| knex`, a `trx = knex` default, `trx` destructured from options | [Transactions](#transactions) |
|
|
976
|
+
| `jskit/no-hand-scope` | a hand-written `where` on `workspace_id` or `workspaceId` | [Layer 1: the table resource](#layer-1-the-table-resource) |
|
|
977
|
+
| `jskit/sql-literal` | raw SQL built from strings or templates instead of one literal with bindings | [Layer 5: reports the resource cannot express](#layer-5-reports-the-resource-cannot-express) |
|
|
978
|
+
| `jskit/no-hand-context` | a system context made with `createSystemContext`, a context's `scopeValues` written by hand | [Background work](#background-work) |
|
|
979
|
+
| `jskit/knex-reason` | a `scopedQuery()` report outside a named repository method under `// Knex: <kind> - <reason>` | [Layer 5: reports the resource cannot express](#layer-5-reports-the-resource-cannot-express) |
|
|
980
|
+
|
|
981
|
+
### Exceptions
|
|
982
|
+
|
|
983
|
+
A module whose job is data access itself (a database runtime, a migration
|
|
984
|
+
runner, test-database setup, an operator's data import or cutover tool, a
|
|
985
|
+
database console that runs the person's own SQL) may opt out of
|
|
986
|
+
`jskit/no-direct-knex`, `jskit/no-transaction-reader`, `jskit/no-hand-scope`
|
|
987
|
+
and `jskit/sql-literal` as a whole, with one line per rule before its first
|
|
988
|
+
line of code:
|
|
989
|
+
|
|
990
|
+
```js
|
|
991
|
+
// readable-check: allow no-direct-knex - infrastructure: the test databases' snapshot and restore
|
|
992
|
+
```
|
|
993
|
+
|
|
994
|
+
In a file excused from `jskit/no-direct-knex`, raw SQL runs on Knex itself,
|
|
995
|
+
so `jskit/sql-literal` lets it hold a statement; it is still one literal with
|
|
996
|
+
its values as bindings. Only a module that runs SQL it is given by design (a
|
|
997
|
+
console's query runner) opts out of `jskit/sql-literal` itself, in that one
|
|
998
|
+
header line with its reason; never with a per-line disable. `jskit check`
|
|
999
|
+
lists every such file for review. `jskit/no-hand-context` and
|
|
1000
|
+
`jskit/knex-reason` take no exception.
|
|
1001
|
+
|
|
1002
|
+
A data import or cutover tool (loading another system's records into the app's
|
|
1003
|
+
tables before go-live, say) is the one application module that commonly takes
|
|
1004
|
+
it. Check and judge, for each file:
|
|
1005
|
+
|
|
1006
|
+
- it runs only as an operator's command; nothing a request can reach imports
|
|
1007
|
+
it;
|
|
1008
|
+
- it bypasses ownership and row policies on purpose, so every read and write
|
|
1009
|
+
names its workspace explicitly;
|
|
1010
|
+
- it writes in one transaction per batch, so a failed batch leaves nothing
|
|
1011
|
+
half-loaded;
|
|
1012
|
+
- its statements per row are still measured, as any write path's are: a batch
|
|
1013
|
+
of 200 rows costs about what a batch of 10 does.
|
|
1014
|
+
|
|
1015
|
+
`jskit check` also fails:
|
|
1016
|
+
|
|
1017
|
+
- a table with two table resources;
|
|
1018
|
+
- a system scope or `systemContext` call without a reason;
|
|
1019
|
+
- an infrastructure exception or a feature's system scope that the app's
|
|
1020
|
+
checked-in `jskit-data-access.json` does not list, so each new one is
|
|
1021
|
+
reviewed: `{ "infrastructure": [{ "file", "rule" }], "systemScopes":
|
|
1022
|
+
[{ "feature", "systemScope" }] }`;
|
|
1023
|
+
- a module that requires `runtime.database` or `runtime.resources` without
|
|
1024
|
+
the infrastructure exception: features own their tables with `owns`;
|
|
1025
|
+
- an `eslint-disable` that names a data-access rule, or no rule, outside
|
|
1026
|
+
client code.
|