@jskit-ai/agent-docs 0.1.275 → 0.1.276
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/guide/agent/app-setup/database-layer.md +30 -0
- package/guide/agent/framework/typed-features.md +207 -0
- package/package.json +1 -1
- package/patterns/minimal-foundation/example/package.json +1 -1
- package/patterns/shell-foundation/example/package.json +1 -1
- package/reference/autogen/PATTERN_INDEX.md +3 -3
- package/skills/jskit/SKILL.md +6 -0
- package/skills/jskit/references/heuristics/INDEX.md +2 -2
- package/skills/jskit/references/heuristics/row-policies.md +10 -0
- package/skills/jskit/references/heuristics/typed-features.md +12 -0
- package/skills/jskit/references/heuristics/ui-testing.md +9 -5
- package/skills/jskit/references/pattern-index.md +3 -3
- package/skills/jskit/references/patterns/app/minimal-foundation/example/package.json +1 -1
- package/skills/jskit/references/patterns/app/shell-foundation/example/package.json +1 -1
- package/skills/jskit/references/typed-features.md +207 -0
|
@@ -365,6 +365,36 @@ Do not add migration receipts, sync ledgers, generator provenance, or dialect
|
|
|
365
365
|
questionnaires. The installed graph, migration source, environment, and
|
|
366
366
|
database migration table are sufficient.
|
|
367
367
|
|
|
368
|
+
## Integration test databases
|
|
369
|
+
|
|
370
|
+
Disposable integration and migration tests use `TEST_DB_NAME`. A test run
|
|
371
|
+
prepares that schema once and restores it from a stored snapshot; it never
|
|
372
|
+
rebuilds a database by replaying the whole migration history:
|
|
373
|
+
|
|
374
|
+
- The snapshot holds the schema as the engine reports it (each table's
|
|
375
|
+
`SHOW CREATE TABLE` on MySQL and MariaDB, `pg_dump --schema-only` on
|
|
376
|
+
PostgreSQL), the migration ledger rows and the seed rows. Its key is a digest
|
|
377
|
+
of the migration files that the application's `migrations/` and its installed
|
|
378
|
+
packages supply, and of the seed source.
|
|
379
|
+
- When that digest changes, the harness migrates one database once, outside
|
|
380
|
+
the timed tests, and stores a new snapshot. Otherwise it restores the stored
|
|
381
|
+
one. On PostgreSQL, `CREATE DATABASE <name> TEMPLATE <prepared>` copies a
|
|
382
|
+
prepared database that has no open connections.
|
|
383
|
+
- A test that runs migrations up or down, or otherwise changes the schema,
|
|
384
|
+
leaves its database dirty. The harness restores the snapshot before another
|
|
385
|
+
test uses that database, instead of running `migrate.latest()` over the
|
|
386
|
+
whole history.
|
|
387
|
+
- Only the fresh-schema check in [Verification](#verification) replays the
|
|
388
|
+
whole history, as its own step.
|
|
389
|
+
- The harness reports each phase (digest, restore or rebuild, seed) with its
|
|
390
|
+
duration, and the suite has a time budget, so slow preparation is visible
|
|
391
|
+
instead of hiding inside the next test that waits for it. A slow phase is
|
|
392
|
+
fixed, never absorbed by a longer time limit.
|
|
393
|
+
|
|
394
|
+
Endpoints and batch operations get their own database tests at realistic row
|
|
395
|
+
counts with a fixed query budget; see the typed features guide's
|
|
396
|
+
[budgets at realistic counts](/guide/framework/typed-features#budgets-at-realistic-counts).
|
|
397
|
+
|
|
368
398
|
## Browser test preparation
|
|
369
399
|
|
|
370
400
|
Keep the test schema and migration history between routine browser runs. The
|
|
@@ -724,6 +724,107 @@ const rows = extractJsonRestCollectionRows(await api.resources.workspaceMembersh
|
|
|
724
724
|
}, createJsonRestContext(options.context)));
|
|
725
725
|
```
|
|
726
726
|
|
|
727
|
+
### One read per table
|
|
728
|
+
|
|
729
|
+
**A read that builds a list reads each table once for the whole list: one
|
|
730
|
+
query per table with an id-list filter, the rows grouped in memory. It never
|
|
731
|
+
reads one row, or runs one lookup, per item of the list.**
|
|
732
|
+
|
|
733
|
+
- Collect the ids from the first read, then read each related table once with
|
|
734
|
+
an id-list filter: a `searchSchema` entry
|
|
735
|
+
`{ type: "array", actualField: "<column>", filterOperator: "in" }`, read
|
|
736
|
+
through `queryAllJsonRestCollectionRows()`, or the resource's `include`.
|
|
737
|
+
Group the rows by id with `Map.groupBy()`.
|
|
738
|
+
- `getDocumentById()`, `queryDocuments()`, a service `get` or another owner's
|
|
739
|
+
single-record read inside a loop over the list is a read per row, however
|
|
740
|
+
the loop is spelled (`for`, `map` with `Promise.all`, a helper called per
|
|
741
|
+
item). So is a single-record helper reused for a list.
|
|
742
|
+
- Work derived from a row (hashing, preparing a form, assembling a snapshot)
|
|
743
|
+
runs once per distinct input in the call, not once per row it appears in.
|
|
744
|
+
- The list's query count follows its tables and pages (100 rows a page unless
|
|
745
|
+
the resource sets `queryMaxLimit`), never its rows: three hundred invoices
|
|
746
|
+
and their lines take a handful of queries, not three hundred more (see
|
|
747
|
+
[Budgets at realistic counts](#budgets-at-realistic-counts)).
|
|
748
|
+
|
|
749
|
+
Why: a read per row costs nothing with three test rows and seconds with a few
|
|
750
|
+
hundred real ones, and every lookup and visibility check inside it repeats too.
|
|
751
|
+
|
|
752
|
+
```js
|
|
753
|
+
// good: one read per table, however many invoices there are
|
|
754
|
+
const invoiceRows = await queryAllJsonRestCollectionRows(api.resources.invoices, {
|
|
755
|
+
queryParams: { filters: { workspace: workspaceId } },
|
|
756
|
+
transaction: resolveTransaction(options),
|
|
757
|
+
format: "plain"
|
|
758
|
+
}, options.context);
|
|
759
|
+
const lineRows = await queryAllJsonRestCollectionRows(api.resources.invoiceLines, {
|
|
760
|
+
queryParams: { filters: { invoiceIds: invoiceRows.map((invoice) => invoice.id) } },
|
|
761
|
+
transaction: resolveTransaction(options),
|
|
762
|
+
format: "plain"
|
|
763
|
+
}, options.context);
|
|
764
|
+
const linesByInvoice = Map.groupBy(lineRows, (line) => line.invoiceId);
|
|
765
|
+
return invoiceRows.map((invoice) => ({ ...invoice, lines: linesByInvoice.get(invoice.id) || [] }));
|
|
766
|
+
|
|
767
|
+
// bad: one more query, and its lookups and row policy, for every invoice
|
|
768
|
+
for (const invoice of invoiceRows) {
|
|
769
|
+
invoice.lines = await invoiceLines.queryDocuments({ invoiceId: invoice.id }, options);
|
|
770
|
+
}
|
|
771
|
+
```
|
|
772
|
+
|
|
773
|
+
### Bounded access checks
|
|
774
|
+
|
|
775
|
+
**A row policy costs what the rows its query reads cost, never what the
|
|
776
|
+
subject's stored history costs, and a write decides access once instead of
|
|
777
|
+
paying a read policy for every row it writes.**
|
|
778
|
+
|
|
779
|
+
- A `rowPolicy` adds one predicate to the query it is given: a join, `EXISTS`
|
|
780
|
+
or subquery correlated with the row being read, on indexed columns. It never
|
|
781
|
+
loads everything the subject was ever linked to (every source, notice or
|
|
782
|
+
grant in their history) to build an `IN` list, and never calls another
|
|
783
|
+
owner's access check once per item of that history. The
|
|
784
|
+
[row policies heuristic](https://github.com/mobily-enterprises/jskit-ai/blob/main/packages/agent-docs/heuristics/row-policies.md)
|
|
785
|
+
has the policy contract and its package composition.
|
|
786
|
+
- The policy runs on every single-record read, and a CRUD write includes one:
|
|
787
|
+
`createDocument()` and `patchDocumentById()` write with `returning: "full"`
|
|
788
|
+
and read the record back through the policy, and `patch` and `delete` first
|
|
789
|
+
look the record up through it. A write therefore pays the read policy for
|
|
790
|
+
each row it writes.
|
|
791
|
+
- A write path decides access once, inside its transaction. A batch write (a
|
|
792
|
+
delivery, an import, a fan-out) whose per-row read-back would pay an
|
|
793
|
+
expensive policy writes through a repository method added with
|
|
794
|
+
`decorateRepository`, on `resolveRepoClient(database.knex, options)`,
|
|
795
|
+
setting the same columns the resource would, its ownership columns
|
|
796
|
+
included, and reading back only what it returns.
|
|
797
|
+
- Its test runs one read and one write for a subject with a short history and
|
|
798
|
+
with a realistic long one, and asserts the same query count.
|
|
799
|
+
|
|
800
|
+
Why: a check sized by history grows with every record the subject receives,
|
|
801
|
+
so the same write that takes milliseconds in a test takes seconds a year later.
|
|
802
|
+
|
|
803
|
+
```js
|
|
804
|
+
// good: one predicate on the row being read
|
|
805
|
+
function noticeVisibilityPolicy({ query, context, column, value, db }) {
|
|
806
|
+
const userId = resolveActorId(context);
|
|
807
|
+
if (!userId) return false;
|
|
808
|
+
query.where(column("recipientId"), value("recipientId", userId)).whereExists(
|
|
809
|
+
db("sources").select(1)
|
|
810
|
+
.whereColumn("sources.id", column("sourceId"))
|
|
811
|
+
.where("sources.status", "published")
|
|
812
|
+
);
|
|
813
|
+
return true;
|
|
814
|
+
}
|
|
815
|
+
|
|
816
|
+
// bad: every source the recipient was ever notified about, checked one by one
|
|
817
|
+
async function noticeVisibilityPolicy({ query, context, column, value }) {
|
|
818
|
+
const sourceIds = await noticeHistory.listSourceIds(resolveActorId(context));
|
|
819
|
+
const readable = [];
|
|
820
|
+
for (const sourceId of sourceIds) {
|
|
821
|
+
if (await sources.canRead(sourceId, context)) readable.push(sourceId);
|
|
822
|
+
}
|
|
823
|
+
query.whereIn(column("sourceId"), readable.map((id) => value("sourceId", id)));
|
|
824
|
+
return true;
|
|
825
|
+
}
|
|
826
|
+
```
|
|
827
|
+
|
|
727
828
|
### Routes
|
|
728
829
|
|
|
729
830
|
**A route does HTTP work only (status, cookies, redirects) and runs an action
|
|
@@ -910,6 +1011,112 @@ export function resetInvoiceCacheForTests() {
|
|
|
910
1011
|
}
|
|
911
1012
|
```
|
|
912
1013
|
|
|
1014
|
+
### Budgets at realistic counts
|
|
1015
|
+
|
|
1016
|
+
**A list, batch or write path has a database test at the row counts and
|
|
1017
|
+
history a real workspace holds, and that test fails above a fixed query
|
|
1018
|
+
budget, one that a read per row or a history-sized check cannot meet.**
|
|
1019
|
+
|
|
1020
|
+
- Seed hundreds of records and long per-person histories, not one to three
|
|
1021
|
+
rows: a read per row and a history-sized policy cost nothing at three.
|
|
1022
|
+
- Count the queries with the test Knex instance's `query` event and assert a
|
|
1023
|
+
ceiling: the count the correct implementation needs at that size, which a
|
|
1024
|
+
read per row overshoots by hundreds. A path that pages nothing (one read,
|
|
1025
|
+
one write) asserts the same count at a short and a long history. A query
|
|
1026
|
+
count is stable in CI; also report the time the call took.
|
|
1027
|
+
- Meet the budget by fixing the reads ([One read per
|
|
1028
|
+
table](#one-read-per-table), [Bounded access checks](#bounded-access-checks)),
|
|
1029
|
+
never by shrinking the data or raising the budget.
|
|
1030
|
+
|
|
1031
|
+
Why: without a budget at real counts, a path that does one read per row ships,
|
|
1032
|
+
and its first sign is a request that takes seconds in production.
|
|
1033
|
+
|
|
1034
|
+
```js
|
|
1035
|
+
// good
|
|
1036
|
+
test("the invoice list reads each table once", async () => {
|
|
1037
|
+
await seedInvoices(knex, { workspace, count: 300, linesEach: 5 });
|
|
1038
|
+
let queries = 0;
|
|
1039
|
+
const count = () => { queries += 1; };
|
|
1040
|
+
knex.on("query", count);
|
|
1041
|
+
try {
|
|
1042
|
+
await invoices.listWithLines(workspace, { context });
|
|
1043
|
+
} finally {
|
|
1044
|
+
knex.off("query", count);
|
|
1045
|
+
}
|
|
1046
|
+
// A page of each table at a time; one read per invoice would add 300.
|
|
1047
|
+
assert.ok(queries <= 40, `${queries} queries`);
|
|
1048
|
+
});
|
|
1049
|
+
|
|
1050
|
+
// bad: three invoices and no budget, so a read per invoice passes
|
|
1051
|
+
test("the invoice list has its lines", async () => {
|
|
1052
|
+
await seedInvoices(knex, { workspace, count: 3, linesEach: 1 });
|
|
1053
|
+
assert.equal((await invoices.listWithLines(workspace, { context })).length, 3);
|
|
1054
|
+
});
|
|
1055
|
+
```
|
|
1056
|
+
|
|
1057
|
+
### Time limits are not fixes
|
|
1058
|
+
|
|
1059
|
+
**A slow test, suite, preparation or request is a defect to find, never a time
|
|
1060
|
+
limit to raise. Measure where the time goes, fix that cause, and keep the
|
|
1061
|
+
default limits.**
|
|
1062
|
+
|
|
1063
|
+
- Raising `--test-timeout`, a test's `{ timeout }`, Playwright's `timeout`,
|
|
1064
|
+
`test.setTimeout()` or an `expect` timeout, adding retries, or calling a
|
|
1065
|
+
request in a loop until it answers hides the cause and lets it grow.
|
|
1066
|
+
- Read the preparation phases' durations and the query counts first; the
|
|
1067
|
+
cause is usually a read per row, a history-sized policy, or a database
|
|
1068
|
+
prepared by replaying migrations ([Test databases are prepared
|
|
1069
|
+
once](#test-databases-are-prepared-once)).
|
|
1070
|
+
- A longer limit is right only for work that is long by nature, such as a
|
|
1071
|
+
fixture of deliberate size, and the test names that work.
|
|
1072
|
+
- A time limit still bounds every run, so a hang ends; it is not a speed
|
|
1073
|
+
setting.
|
|
1074
|
+
|
|
1075
|
+
Why: a raised limit absorbs today's slowness and every later one, until a run
|
|
1076
|
+
takes hours and nobody knows which change made it slow.
|
|
1077
|
+
|
|
1078
|
+
```js
|
|
1079
|
+
// good: the slow step was found and fixed, so the default limit holds
|
|
1080
|
+
test("delivers a batch of notices", async () => {
|
|
1081
|
+
await deliveries.processBatch(workspace, { context });
|
|
1082
|
+
});
|
|
1083
|
+
|
|
1084
|
+
// bad
|
|
1085
|
+
test("delivers a batch of notices", { timeout: 600_000 }, async () => {
|
|
1086
|
+
await deliveries.processBatch(workspace, { context });
|
|
1087
|
+
});
|
|
1088
|
+
```
|
|
1089
|
+
|
|
1090
|
+
### Test databases are prepared once
|
|
1091
|
+
|
|
1092
|
+
**A test run prepares each database once, from a stored snapshot of its
|
|
1093
|
+
schema, migration ledger and seed rows checked by a digest of their sources,
|
|
1094
|
+
and never replays the whole migration history inside the run.**
|
|
1095
|
+
|
|
1096
|
+
- The snapshot is rebuilt only when the digest of the migration and seed
|
|
1097
|
+
sources changes, once and outside the timed tests; every disposable
|
|
1098
|
+
database, and every database a test migrated up or down, is restored from
|
|
1099
|
+
it.
|
|
1100
|
+
- The full migration history runs once, in its own fresh-schema check
|
|
1101
|
+
([database verification](https://mobily-enterprises.github.io/jskit-ai/guide/app-setup/database-layer#verification)).
|
|
1102
|
+
- The harness reports each preparation phase and its duration, and the suite
|
|
1103
|
+
has a time budget, so slow preparation shows instead of hiding inside the
|
|
1104
|
+
next test that waits for it.
|
|
1105
|
+
- The database layer guide's [integration test
|
|
1106
|
+
databases](https://mobily-enterprises.github.io/jskit-ai/guide/app-setup/database-layer#integration-test-databases)
|
|
1107
|
+
gives the snapshot's contents per engine.
|
|
1108
|
+
|
|
1109
|
+
Why: the migration history only grows, so a run that replays it whenever a
|
|
1110
|
+
schema changed gets slower with every migration the application adds.
|
|
1111
|
+
|
|
1112
|
+
```js
|
|
1113
|
+
// good: the application's test support restores the snapshot its digest names
|
|
1114
|
+
before(async () => restoreTestDatabase(knex, await testDatabaseSnapshot()));
|
|
1115
|
+
|
|
1116
|
+
// bad: every file replays every migration
|
|
1117
|
+
before(() => knex.migrate.latest());
|
|
1118
|
+
```
|
|
1119
|
+
|
|
913
1120
|
## Files and layout
|
|
914
1121
|
|
|
915
1122
|
### Size limits
|
package/package.json
CHANGED
|
@@ -15,7 +15,7 @@ A concrete Fastify, Vue, and JSKIT application foundation for products that do n
|
|
|
15
15
|
|
|
16
16
|
- Id: `app/minimal-foundation`
|
|
17
17
|
- Keywords: `app`, `fastify`, `foundation`, `minimal`, `server`, `vite`, `vue`
|
|
18
|
-
- Owner: `@jskit-ai/agent-docs@0.1.
|
|
18
|
+
- Owner: `@jskit-ai/agent-docs@0.1.276`
|
|
19
19
|
- Read: [PATTERN.md](../../skills/jskit/references/patterns/app/minimal-foundation/PATTERN.md)
|
|
20
20
|
- Examples: [example/](../../skills/jskit/references/patterns/app/minimal-foundation/example/)
|
|
21
21
|
- Requires: `@jskit-ai/http-runtime`, `@jskit-ai/kernel`
|
|
@@ -26,7 +26,7 @@ A concrete JSKIT web application foundation with responsive shell navigation, se
|
|
|
26
26
|
|
|
27
27
|
- Id: `app/shell-foundation`
|
|
28
28
|
- Keywords: `app`, `foundation`, `material`, `navigation`, `placements`, `shell`, `vite`, `vue`
|
|
29
|
-
- Owner: `@jskit-ai/agent-docs@0.1.
|
|
29
|
+
- Owner: `@jskit-ai/agent-docs@0.1.276`
|
|
30
30
|
- Read: [PATTERN.md](../../skills/jskit/references/patterns/app/shell-foundation/PATTERN.md)
|
|
31
31
|
- Examples: [example/](../../skills/jskit/references/patterns/app/shell-foundation/example/)
|
|
32
32
|
- Requires: `@jskit-ai/http-runtime`, `@jskit-ai/kernel`, `@jskit-ai/shell-web`
|
|
@@ -345,7 +345,7 @@ Define a server feature through explicit capabilities and first-class actions, a
|
|
|
345
345
|
|
|
346
346
|
- Id: `server/feature-package`
|
|
347
347
|
- Keywords: `actions`, `feature`, `json-rest`, `knex`, `orchestration`, `package`, `provider`, `repository`, `routes`, `server`
|
|
348
|
-
- Owner: `@jskit-ai/agent-docs@0.1.
|
|
348
|
+
- Owner: `@jskit-ai/agent-docs@0.1.276`
|
|
349
349
|
- Read: [PATTERN.md](../../skills/jskit/references/patterns/server/feature-package/PATTERN.md)
|
|
350
350
|
- Examples: [example/](../../skills/jskit/references/patterns/server/feature-package/example/)
|
|
351
351
|
- Requires: `@jskit-ai/kernel`
|
package/skills/jskit/SKILL.md
CHANGED
|
@@ -121,3 +121,9 @@ Run focused tests for a slice and the application's `npm run verify` for a
|
|
|
121
121
|
whole changeset. Use the current-state owners in application operations and
|
|
122
122
|
Playwright for affected UI. There is no current JSKIT Doctor command; ignore
|
|
123
123
|
old CLI authoring-history diagnoses. Never expose environment values.
|
|
124
|
+
|
|
125
|
+
Tests and requests take the time their work needs, no more: a list reads each
|
|
126
|
+
table once, a row policy is bounded by the rows it reads, list and batch paths
|
|
127
|
+
have a query budget tested at realistic row counts, and test databases restore
|
|
128
|
+
a stored snapshot instead of replaying migrations. Find why a check is slow;
|
|
129
|
+
never raise its time limit. Typed features' repository and Tests rules say how.
|
|
@@ -42,9 +42,9 @@ How to use it:
|
|
|
42
42
|
- [server-search.md](./server-search.md)
|
|
43
43
|
- `actualField`, `storage.writeSerializer`, `storage.virtual`, computed field, virtual field, projection, `createCrudResourceRuntime`, datetime write serialization, temporalPrecision, json-rest-schema dates
|
|
44
44
|
- [crud-repository-mapping.md](./crud-repository-mapping.md)
|
|
45
|
-
- row policy, permission filtering, visibility before pagination, recursive CTE, descendants, package cycle, `RowPolicyPlugin
|
|
45
|
+
- row policy, permission filtering, visibility before pagination, recursive CTE, descendants, package cycle, `RowPolicyPlugin`, row policy cost, slow writes through a policy, access check per history item
|
|
46
46
|
- [row-policies.md](./row-policies.md)
|
|
47
|
-
- typed features, server feature, provider, service, repository, collaborator, `typeof`, `= null`, `setProvider`, `register`, `input || {}`, `payload = {}`, `async execute`, `Object.freeze`, `requires`, `optional`, `types.js`, `RuntimeKnex`, `silentLogger`, `create<Name>Actions`, `ActionSpecifications`, request context, `api.resources.x.query`, every row, list stops at 20 rows, `queryAllJsonRestCollectionRows`, `jskit/*` lint findings, `readable-check: allow`, `optional operation:`, `process.env`, `AppError` code, `Object.assign(new Error`, `idempotency`, `…ForTests` export, file or function size
|
|
47
|
+
- typed features, server feature, provider, service, repository, collaborator, `typeof`, `= null`, `setProvider`, `register`, `input || {}`, `payload = {}`, `async execute`, `Object.freeze`, `requires`, `optional`, `types.js`, `RuntimeKnex`, `silentLogger`, `create<Name>Actions`, `ActionSpecifications`, request context, `api.resources.x.query`, every row, list stops at 20 rows, `queryAllJsonRestCollectionRows`, `jskit/*` lint findings, `readable-check: allow`, `optional operation:`, `process.env`, `AppError` code, `Object.assign(new Error`, `idempotency`, `…ForTests` export, file or function size, read per row, N+1 queries, slow endpoint, query budget, realistic row counts, slow tests, raised timeout, `migrate.latest()` in tests, test database snapshot
|
|
48
48
|
- [typed-features.md](./typed-features.md)
|
|
49
49
|
- client features, composable, `use<Name>`, Vue component, client provider, props, emits, `inject`, `provide`, `ref`, `reactive`, `watch`, `addEventListener`, `setInterval`, module state, `Object.freeze` on a composable, `?.()` in a template, `jskit/typed-inject`, `jskit/collaborator-props`, `jskit/fixed-result-shape`, `jskit/released-listeners`, `jskit/no-module-state`, `jskit/no-frozen-vue-state`, `clientFeaturesConfig`, `shared resource:`
|
|
50
50
|
- [client-features.md](./client-features.md)
|
|
@@ -8,6 +8,7 @@ Use when:
|
|
|
8
8
|
- filtering after pagination produces empty or incomplete pages
|
|
9
9
|
- another package grants access to a resource but a reverse dependency would create a cycle
|
|
10
10
|
- the user mentions `RowPolicyPlugin`, recursive CTEs, descendants, or permission-filtered counts
|
|
11
|
+
- reads or writes of a resource with a policy get slower as records or a person's history grow
|
|
11
12
|
|
|
12
13
|
Read first:
|
|
13
14
|
|
|
@@ -36,6 +37,13 @@ Package composition:
|
|
|
36
37
|
- the owner seals the contributions and builds the policy during `boot()`
|
|
37
38
|
- do not use module-global mutable registries
|
|
38
39
|
|
|
40
|
+
Cost:
|
|
41
|
+
|
|
42
|
+
- a policy's cost follows the rows its query reads: add a join, `EXISTS` or subquery correlated with the row through `column()` and `value()`, on indexed columns
|
|
43
|
+
- never load the subject's whole history (every source, notice or grant they were ever linked to) to build an `IN` list, and never call another owner's access check per item of it; another package's grant is a predicate contribution
|
|
44
|
+
- the policy runs on every single-record read, including the read-back after `createDocument()` and `patchDocumentById()` (`returning: "full"`) and the lookup before `patch` and `delete`, so a write pays it once per row it writes
|
|
45
|
+
- a write decides access once, inside its transaction; a batch write that would pay an expensive read-back per row follows the typed features guide's [Bounded access checks](../typed-features.md#bounded-access-checks)
|
|
46
|
+
|
|
39
47
|
Child-resource rules:
|
|
40
48
|
|
|
41
49
|
- a self-referencing relationship targets the same resource and receives the same row policy
|
|
@@ -50,6 +58,7 @@ Testing:
|
|
|
50
58
|
- assert hidden direct records behave as not found
|
|
51
59
|
- assert children and includes do not leak records
|
|
52
60
|
- assert package and provider dependencies stay one-way
|
|
61
|
+
- run one read and one write for a subject with a short history and with a realistic long one (hundreds of linked rows), and assert the same query count
|
|
53
62
|
|
|
54
63
|
AnyAPI boundary:
|
|
55
64
|
|
|
@@ -65,3 +74,4 @@ Avoid:
|
|
|
65
74
|
- adding a default policy file to every CRUD
|
|
66
75
|
- returning a Knex builder or promise from a synchronous grouped contribution
|
|
67
76
|
- swallowing a policy error and retrying without mandatory visibility
|
|
77
|
+
- a policy that reads the subject's history or calls an access check per history item
|
|
@@ -65,6 +65,10 @@ ends with the same link.
|
|
|
65
65
|
- [One reader per fact](../typed-features.md#one-reader-per-fact)
|
|
66
66
|
- `api.resources.x.query(…)` with no `page` for a list, `queryDocuments(query)` with no `limit` in a service, a list that stops at 20 rows
|
|
67
67
|
- [Every row](../typed-features.md#every-row)
|
|
68
|
+
- `await getDocumentById(…)`, `queryDocuments(…)` or a service `get` inside a loop or a `Promise.all` over a list, a single-record helper reused for a list, N+1 queries, a slow list endpoint
|
|
69
|
+
- [One read per table](../typed-features.md#one-read-per-table)
|
|
70
|
+
- a `rowPolicy` that loads a person's whole history or calls an access check per item, `createDocument` per row of a batch, a write that gets slower as history grows
|
|
71
|
+
- [Bounded access checks](../typed-features.md#bounded-access-checks)
|
|
68
72
|
- a route that queries storage, `boot*.js` holding routes
|
|
69
73
|
- [Routes](../typed-features.md#routes)
|
|
70
74
|
- `new AppError(status, message)` with no code, `new NotFoundError("…")`, one code for two sentences (`jskit/app-error-code`)
|
|
@@ -81,6 +85,12 @@ ends with the same link.
|
|
|
81
85
|
- [Tests are not callers](../typed-features.md#tests-are-not-callers)
|
|
82
86
|
- `…ForTests`, `__reset…`, `testOnly…` exports (`jskit/no-test-only-export`)
|
|
83
87
|
- [No test-only exports](../typed-features.md#no-test-only-exports)
|
|
88
|
+
- a list, batch or write test with one to three rows, no query count, a slow endpoint first seen with real data
|
|
89
|
+
- [Budgets at realistic counts](../typed-features.md#budgets-at-realistic-counts)
|
|
90
|
+
- a raised `{ timeout }`, `test.setTimeout()`, `--test-timeout` or Playwright `timeout`, retries or a request loop that absorbs slowness, a slow suite
|
|
91
|
+
- [Time limits are not fixes](../typed-features.md#time-limits-are-not-fixes)
|
|
92
|
+
- `knex.migrate.latest()` in each test file, a template database rebuilt by migrating, test preparation slower than its tests
|
|
93
|
+
- [Test databases are prepared once](../typed-features.md#test-databases-are-prepared-once)
|
|
84
94
|
- a file over 750 lines, a function over 600 (`max-lines`, `max-lines-per-function`), a data module
|
|
85
95
|
- [Size limits](../typed-features.md#size-limits), [Data modules](../typed-features.md#data-modules)
|
|
86
96
|
- two properties on one line in `Object.freeze({ … })` or a token map (`jskit/one-property-per-line`)
|
|
@@ -100,3 +110,5 @@ Avoid:
|
|
|
100
110
|
- module-level registries
|
|
101
111
|
- refusals without their own code
|
|
102
112
|
- excusing a finding without a reason in its `readable-check: allow` comment
|
|
113
|
+
- a read per row of a list, or an access check sized by stored history
|
|
114
|
+
- raising a time limit instead of finding why a check is slow
|
|
@@ -65,11 +65,15 @@ in the regression: one test's teardown must not erase another test's migrations.
|
|
|
65
65
|
|
|
66
66
|
Do not drop/recreate the database, replay completed migrations, build every
|
|
67
67
|
module fixture, or run a production frontend build for each focused check.
|
|
68
|
-
Choose small fixture sets for the selected suite
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
68
|
+
Choose small fixture sets for the selected suite; realistic row counts belong
|
|
69
|
+
in the database tests that hold each endpoint's query budget. Fresh-schema
|
|
70
|
+
migration tests and production-artifact tests are separate operations. Measure
|
|
71
|
+
migration, fixture, and server startup time and fix the slow phase; never retry
|
|
72
|
+
a slow launch, or raise its deadline or a spec's timeout, to absorb it. A page
|
|
73
|
+
that needs a longer timeout to load is a slow endpoint to fix (typed features'
|
|
74
|
+
Time limits are not fixes). When a host supplies a managed target, use its
|
|
75
|
+
scoped suite command so one startup serves the whole batch and cleanup
|
|
76
|
+
restores normal Preview.
|
|
73
77
|
|
|
74
78
|
## Keep browser evidence focused
|
|
75
79
|
|
|
@@ -15,7 +15,7 @@ A concrete Fastify, Vue, and JSKIT application foundation for products that do n
|
|
|
15
15
|
|
|
16
16
|
- Id: `app/minimal-foundation`
|
|
17
17
|
- Keywords: `app`, `fastify`, `foundation`, `minimal`, `server`, `vite`, `vue`
|
|
18
|
-
- Owner: `@jskit-ai/agent-docs@0.1.
|
|
18
|
+
- Owner: `@jskit-ai/agent-docs@0.1.276`
|
|
19
19
|
- Read: [PATTERN.md](patterns/app/minimal-foundation/PATTERN.md)
|
|
20
20
|
- Examples: [example/](patterns/app/minimal-foundation/example/)
|
|
21
21
|
- Requires: `@jskit-ai/http-runtime`, `@jskit-ai/kernel`
|
|
@@ -26,7 +26,7 @@ A concrete JSKIT web application foundation with responsive shell navigation, se
|
|
|
26
26
|
|
|
27
27
|
- Id: `app/shell-foundation`
|
|
28
28
|
- Keywords: `app`, `foundation`, `material`, `navigation`, `placements`, `shell`, `vite`, `vue`
|
|
29
|
-
- Owner: `@jskit-ai/agent-docs@0.1.
|
|
29
|
+
- Owner: `@jskit-ai/agent-docs@0.1.276`
|
|
30
30
|
- Read: [PATTERN.md](patterns/app/shell-foundation/PATTERN.md)
|
|
31
31
|
- Examples: [example/](patterns/app/shell-foundation/example/)
|
|
32
32
|
- Requires: `@jskit-ai/http-runtime`, `@jskit-ai/kernel`, `@jskit-ai/shell-web`
|
|
@@ -345,7 +345,7 @@ Define a server feature through explicit capabilities and first-class actions, a
|
|
|
345
345
|
|
|
346
346
|
- Id: `server/feature-package`
|
|
347
347
|
- Keywords: `actions`, `feature`, `json-rest`, `knex`, `orchestration`, `package`, `provider`, `repository`, `routes`, `server`
|
|
348
|
-
- Owner: `@jskit-ai/agent-docs@0.1.
|
|
348
|
+
- Owner: `@jskit-ai/agent-docs@0.1.276`
|
|
349
349
|
- Read: [PATTERN.md](patterns/server/feature-package/PATTERN.md)
|
|
350
350
|
- Examples: [example/](patterns/server/feature-package/example/)
|
|
351
351
|
- Requires: `@jskit-ai/kernel`
|
|
@@ -719,6 +719,107 @@ const rows = extractJsonRestCollectionRows(await api.resources.workspaceMembersh
|
|
|
719
719
|
}, createJsonRestContext(options.context)));
|
|
720
720
|
```
|
|
721
721
|
|
|
722
|
+
### One read per table
|
|
723
|
+
|
|
724
|
+
**A read that builds a list reads each table once for the whole list: one
|
|
725
|
+
query per table with an id-list filter, the rows grouped in memory. It never
|
|
726
|
+
reads one row, or runs one lookup, per item of the list.**
|
|
727
|
+
|
|
728
|
+
- Collect the ids from the first read, then read each related table once with
|
|
729
|
+
an id-list filter: a `searchSchema` entry
|
|
730
|
+
`{ type: "array", actualField: "<column>", filterOperator: "in" }`, read
|
|
731
|
+
through `queryAllJsonRestCollectionRows()`, or the resource's `include`.
|
|
732
|
+
Group the rows by id with `Map.groupBy()`.
|
|
733
|
+
- `getDocumentById()`, `queryDocuments()`, a service `get` or another owner's
|
|
734
|
+
single-record read inside a loop over the list is a read per row, however
|
|
735
|
+
the loop is spelled (`for`, `map` with `Promise.all`, a helper called per
|
|
736
|
+
item). So is a single-record helper reused for a list.
|
|
737
|
+
- Work derived from a row (hashing, preparing a form, assembling a snapshot)
|
|
738
|
+
runs once per distinct input in the call, not once per row it appears in.
|
|
739
|
+
- The list's query count follows its tables and pages (100 rows a page unless
|
|
740
|
+
the resource sets `queryMaxLimit`), never its rows: three hundred invoices
|
|
741
|
+
and their lines take a handful of queries, not three hundred more (see
|
|
742
|
+
[Budgets at realistic counts](#budgets-at-realistic-counts)).
|
|
743
|
+
|
|
744
|
+
Why: a read per row costs nothing with three test rows and seconds with a few
|
|
745
|
+
hundred real ones, and every lookup and visibility check inside it repeats too.
|
|
746
|
+
|
|
747
|
+
```js
|
|
748
|
+
// good: one read per table, however many invoices there are
|
|
749
|
+
const invoiceRows = await queryAllJsonRestCollectionRows(api.resources.invoices, {
|
|
750
|
+
queryParams: { filters: { workspace: workspaceId } },
|
|
751
|
+
transaction: resolveTransaction(options),
|
|
752
|
+
format: "plain"
|
|
753
|
+
}, options.context);
|
|
754
|
+
const lineRows = await queryAllJsonRestCollectionRows(api.resources.invoiceLines, {
|
|
755
|
+
queryParams: { filters: { invoiceIds: invoiceRows.map((invoice) => invoice.id) } },
|
|
756
|
+
transaction: resolveTransaction(options),
|
|
757
|
+
format: "plain"
|
|
758
|
+
}, options.context);
|
|
759
|
+
const linesByInvoice = Map.groupBy(lineRows, (line) => line.invoiceId);
|
|
760
|
+
return invoiceRows.map((invoice) => ({ ...invoice, lines: linesByInvoice.get(invoice.id) || [] }));
|
|
761
|
+
|
|
762
|
+
// bad: one more query, and its lookups and row policy, for every invoice
|
|
763
|
+
for (const invoice of invoiceRows) {
|
|
764
|
+
invoice.lines = await invoiceLines.queryDocuments({ invoiceId: invoice.id }, options);
|
|
765
|
+
}
|
|
766
|
+
```
|
|
767
|
+
|
|
768
|
+
### Bounded access checks
|
|
769
|
+
|
|
770
|
+
**A row policy costs what the rows its query reads cost, never what the
|
|
771
|
+
subject's stored history costs, and a write decides access once instead of
|
|
772
|
+
paying a read policy for every row it writes.**
|
|
773
|
+
|
|
774
|
+
- A `rowPolicy` adds one predicate to the query it is given: a join, `EXISTS`
|
|
775
|
+
or subquery correlated with the row being read, on indexed columns. It never
|
|
776
|
+
loads everything the subject was ever linked to (every source, notice or
|
|
777
|
+
grant in their history) to build an `IN` list, and never calls another
|
|
778
|
+
owner's access check once per item of that history. The
|
|
779
|
+
[row policies heuristic](heuristics/row-policies.md)
|
|
780
|
+
has the policy contract and its package composition.
|
|
781
|
+
- The policy runs on every single-record read, and a CRUD write includes one:
|
|
782
|
+
`createDocument()` and `patchDocumentById()` write with `returning: "full"`
|
|
783
|
+
and read the record back through the policy, and `patch` and `delete` first
|
|
784
|
+
look the record up through it. A write therefore pays the read policy for
|
|
785
|
+
each row it writes.
|
|
786
|
+
- A write path decides access once, inside its transaction. A batch write (a
|
|
787
|
+
delivery, an import, a fan-out) whose per-row read-back would pay an
|
|
788
|
+
expensive policy writes through a repository method added with
|
|
789
|
+
`decorateRepository`, on `resolveRepoClient(database.knex, options)`,
|
|
790
|
+
setting the same columns the resource would, its ownership columns
|
|
791
|
+
included, and reading back only what it returns.
|
|
792
|
+
- Its test runs one read and one write for a subject with a short history and
|
|
793
|
+
with a realistic long one, and asserts the same query count.
|
|
794
|
+
|
|
795
|
+
Why: a check sized by history grows with every record the subject receives,
|
|
796
|
+
so the same write that takes milliseconds in a test takes seconds a year later.
|
|
797
|
+
|
|
798
|
+
```js
|
|
799
|
+
// good: one predicate on the row being read
|
|
800
|
+
function noticeVisibilityPolicy({ query, context, column, value, db }) {
|
|
801
|
+
const userId = resolveActorId(context);
|
|
802
|
+
if (!userId) return false;
|
|
803
|
+
query.where(column("recipientId"), value("recipientId", userId)).whereExists(
|
|
804
|
+
db("sources").select(1)
|
|
805
|
+
.whereColumn("sources.id", column("sourceId"))
|
|
806
|
+
.where("sources.status", "published")
|
|
807
|
+
);
|
|
808
|
+
return true;
|
|
809
|
+
}
|
|
810
|
+
|
|
811
|
+
// bad: every source the recipient was ever notified about, checked one by one
|
|
812
|
+
async function noticeVisibilityPolicy({ query, context, column, value }) {
|
|
813
|
+
const sourceIds = await noticeHistory.listSourceIds(resolveActorId(context));
|
|
814
|
+
const readable = [];
|
|
815
|
+
for (const sourceId of sourceIds) {
|
|
816
|
+
if (await sources.canRead(sourceId, context)) readable.push(sourceId);
|
|
817
|
+
}
|
|
818
|
+
query.whereIn(column("sourceId"), readable.map((id) => value("sourceId", id)));
|
|
819
|
+
return true;
|
|
820
|
+
}
|
|
821
|
+
```
|
|
822
|
+
|
|
722
823
|
### Routes
|
|
723
824
|
|
|
724
825
|
**A route does HTTP work only (status, cookies, redirects) and runs an action
|
|
@@ -905,6 +1006,112 @@ export function resetInvoiceCacheForTests() {
|
|
|
905
1006
|
}
|
|
906
1007
|
```
|
|
907
1008
|
|
|
1009
|
+
### Budgets at realistic counts
|
|
1010
|
+
|
|
1011
|
+
**A list, batch or write path has a database test at the row counts and
|
|
1012
|
+
history a real workspace holds, and that test fails above a fixed query
|
|
1013
|
+
budget, one that a read per row or a history-sized check cannot meet.**
|
|
1014
|
+
|
|
1015
|
+
- Seed hundreds of records and long per-person histories, not one to three
|
|
1016
|
+
rows: a read per row and a history-sized policy cost nothing at three.
|
|
1017
|
+
- Count the queries with the test Knex instance's `query` event and assert a
|
|
1018
|
+
ceiling: the count the correct implementation needs at that size, which a
|
|
1019
|
+
read per row overshoots by hundreds. A path that pages nothing (one read,
|
|
1020
|
+
one write) asserts the same count at a short and a long history. A query
|
|
1021
|
+
count is stable in CI; also report the time the call took.
|
|
1022
|
+
- Meet the budget by fixing the reads ([One read per
|
|
1023
|
+
table](#one-read-per-table), [Bounded access checks](#bounded-access-checks)),
|
|
1024
|
+
never by shrinking the data or raising the budget.
|
|
1025
|
+
|
|
1026
|
+
Why: without a budget at real counts, a path that does one read per row ships,
|
|
1027
|
+
and its first sign is a request that takes seconds in production.
|
|
1028
|
+
|
|
1029
|
+
```js
|
|
1030
|
+
// good
|
|
1031
|
+
test("the invoice list reads each table once", async () => {
|
|
1032
|
+
await seedInvoices(knex, { workspace, count: 300, linesEach: 5 });
|
|
1033
|
+
let queries = 0;
|
|
1034
|
+
const count = () => { queries += 1; };
|
|
1035
|
+
knex.on("query", count);
|
|
1036
|
+
try {
|
|
1037
|
+
await invoices.listWithLines(workspace, { context });
|
|
1038
|
+
} finally {
|
|
1039
|
+
knex.off("query", count);
|
|
1040
|
+
}
|
|
1041
|
+
// A page of each table at a time; one read per invoice would add 300.
|
|
1042
|
+
assert.ok(queries <= 40, `${queries} queries`);
|
|
1043
|
+
});
|
|
1044
|
+
|
|
1045
|
+
// bad: three invoices and no budget, so a read per invoice passes
|
|
1046
|
+
test("the invoice list has its lines", async () => {
|
|
1047
|
+
await seedInvoices(knex, { workspace, count: 3, linesEach: 1 });
|
|
1048
|
+
assert.equal((await invoices.listWithLines(workspace, { context })).length, 3);
|
|
1049
|
+
});
|
|
1050
|
+
```
|
|
1051
|
+
|
|
1052
|
+
### Time limits are not fixes
|
|
1053
|
+
|
|
1054
|
+
**A slow test, suite, preparation or request is a defect to find, never a time
|
|
1055
|
+
limit to raise. Measure where the time goes, fix that cause, and keep the
|
|
1056
|
+
default limits.**
|
|
1057
|
+
|
|
1058
|
+
- Raising `--test-timeout`, a test's `{ timeout }`, Playwright's `timeout`,
|
|
1059
|
+
`test.setTimeout()` or an `expect` timeout, adding retries, or calling a
|
|
1060
|
+
request in a loop until it answers hides the cause and lets it grow.
|
|
1061
|
+
- Read the preparation phases' durations and the query counts first; the
|
|
1062
|
+
cause is usually a read per row, a history-sized policy, or a database
|
|
1063
|
+
prepared by replaying migrations ([Test databases are prepared
|
|
1064
|
+
once](#test-databases-are-prepared-once)).
|
|
1065
|
+
- A longer limit is right only for work that is long by nature, such as a
|
|
1066
|
+
fixture of deliberate size, and the test names that work.
|
|
1067
|
+
- A time limit still bounds every run, so a hang ends; it is not a speed
|
|
1068
|
+
setting.
|
|
1069
|
+
|
|
1070
|
+
Why: a raised limit absorbs today's slowness and every later one, until a run
|
|
1071
|
+
takes hours and nobody knows which change made it slow.
|
|
1072
|
+
|
|
1073
|
+
```js
|
|
1074
|
+
// good: the slow step was found and fixed, so the default limit holds
|
|
1075
|
+
test("delivers a batch of notices", async () => {
|
|
1076
|
+
await deliveries.processBatch(workspace, { context });
|
|
1077
|
+
});
|
|
1078
|
+
|
|
1079
|
+
// bad
|
|
1080
|
+
test("delivers a batch of notices", { timeout: 600_000 }, async () => {
|
|
1081
|
+
await deliveries.processBatch(workspace, { context });
|
|
1082
|
+
});
|
|
1083
|
+
```
|
|
1084
|
+
|
|
1085
|
+
### Test databases are prepared once
|
|
1086
|
+
|
|
1087
|
+
**A test run prepares each database once, from a stored snapshot of its
|
|
1088
|
+
schema, migration ledger and seed rows checked by a digest of their sources,
|
|
1089
|
+
and never replays the whole migration history inside the run.**
|
|
1090
|
+
|
|
1091
|
+
- The snapshot is rebuilt only when the digest of the migration and seed
|
|
1092
|
+
sources changes, once and outside the timed tests; every disposable
|
|
1093
|
+
database, and every database a test migrated up or down, is restored from
|
|
1094
|
+
it.
|
|
1095
|
+
- The full migration history runs once, in its own fresh-schema check
|
|
1096
|
+
([database verification](https://mobily-enterprises.github.io/jskit-ai/guide/app-setup/database-layer#verification)).
|
|
1097
|
+
- The harness reports each preparation phase and its duration, and the suite
|
|
1098
|
+
has a time budget, so slow preparation shows instead of hiding inside the
|
|
1099
|
+
next test that waits for it.
|
|
1100
|
+
- The database layer guide's [integration test
|
|
1101
|
+
databases](https://mobily-enterprises.github.io/jskit-ai/guide/app-setup/database-layer#integration-test-databases)
|
|
1102
|
+
gives the snapshot's contents per engine.
|
|
1103
|
+
|
|
1104
|
+
Why: the migration history only grows, so a run that replays it whenever a
|
|
1105
|
+
schema changed gets slower with every migration the application adds.
|
|
1106
|
+
|
|
1107
|
+
```js
|
|
1108
|
+
// good: the application's test support restores the snapshot its digest names
|
|
1109
|
+
before(async () => restoreTestDatabase(knex, await testDatabaseSnapshot()));
|
|
1110
|
+
|
|
1111
|
+
// bad: every file replays every migration
|
|
1112
|
+
before(() => knex.migrate.latest());
|
|
1113
|
+
```
|
|
1114
|
+
|
|
908
1115
|
## Files and layout
|
|
909
1116
|
|
|
910
1117
|
### Size limits
|