@lovrozagar/oat 0.10.0 → 0.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +25 -2006
- package/dist/cli.js +283 -59
- package/dist/config/define-config.d.ts +8 -0
- package/dist/config/define-config.d.ts.map +1 -1
- package/dist/config/define-config.js.map +1 -1
- package/dist/config/validate.d.ts +12 -0
- package/dist/config/validate.d.ts.map +1 -0
- package/dist/config/validate.js +86 -0
- package/dist/config/validate.js.map +1 -0
- package/dist/conformance/effects.d.ts +9 -0
- package/dist/conformance/effects.d.ts.map +1 -0
- package/dist/conformance/effects.js +276 -0
- package/dist/conformance/effects.js.map +1 -0
- package/dist/conformance/expected-failures.d.ts +18 -0
- package/dist/conformance/expected-failures.d.ts.map +1 -0
- package/dist/conformance/expected-failures.js +48 -0
- package/dist/conformance/expected-failures.js.map +1 -0
- package/dist/conformance/feature-gate.d.ts +7 -0
- package/dist/conformance/feature-gate.d.ts.map +1 -0
- package/dist/conformance/feature-gate.js +428 -0
- package/dist/conformance/feature-gate.js.map +1 -0
- package/dist/conformance/fuzz.d.ts +108 -0
- package/dist/conformance/fuzz.d.ts.map +1 -0
- package/dist/conformance/fuzz.js +246 -0
- package/dist/conformance/fuzz.js.map +1 -0
- package/dist/conformance/leaks.d.ts +10 -0
- package/dist/conformance/leaks.d.ts.map +1 -0
- package/dist/conformance/leaks.js +23 -0
- package/dist/conformance/leaks.js.map +1 -0
- package/dist/conformance/pool.d.ts +49 -0
- package/dist/conformance/pool.d.ts.map +1 -0
- package/dist/conformance/pool.js +75 -0
- package/dist/conformance/pool.js.map +1 -0
- package/dist/conformance/rate-limit.d.ts +7 -0
- package/dist/conformance/rate-limit.d.ts.map +1 -0
- package/dist/conformance/rate-limit.js +372 -0
- package/dist/conformance/rate-limit.js.map +1 -0
- package/dist/conformance/scope.d.ts +15 -0
- package/dist/conformance/scope.d.ts.map +1 -0
- package/dist/conformance/scope.js +78 -0
- package/dist/conformance/scope.js.map +1 -0
- package/dist/conformance/seed-contract.d.ts +11 -0
- package/dist/conformance/seed-contract.d.ts.map +1 -0
- package/dist/conformance/seed-contract.js +680 -0
- package/dist/conformance/seed-contract.js.map +1 -0
- package/dist/conformance/shapes.d.ts +35 -0
- package/dist/conformance/shapes.d.ts.map +1 -0
- package/dist/conformance/shapes.js +162 -0
- package/dist/conformance/shapes.js.map +1 -0
- package/dist/conformance/specs.d.ts +21 -0
- package/dist/conformance/specs.d.ts.map +1 -0
- package/dist/conformance/specs.js +306 -0
- package/dist/conformance/specs.js.map +1 -0
- package/dist/conformance/suite.d.ts +209 -0
- package/dist/conformance/suite.d.ts.map +1 -0
- package/dist/conformance/suite.js +1538 -0
- package/dist/conformance/suite.js.map +1 -0
- package/dist/conformance/unique.d.ts +7 -0
- package/dist/conformance/unique.d.ts.map +1 -0
- package/dist/conformance/unique.js +586 -0
- package/dist/conformance/unique.js.map +1 -0
- package/dist/conformance/worker.d.ts +3 -0
- package/dist/conformance/worker.d.ts.map +1 -0
- package/dist/conformance/worker.js +27 -0
- package/dist/conformance/worker.js.map +1 -0
- package/dist/index.d.ts +8 -3
- package/dist/index.js +5 -2
- package/dist/reference/defects.d.ts +71 -1
- package/dist/reference/defects.d.ts.map +1 -1
- package/dist/reference/defects.js +74 -8
- package/dist/reference/defects.js.map +1 -1
- package/dist/reference/dialect.d.ts +6 -4
- package/dist/reference/dialect.d.ts.map +1 -1
- package/dist/reference/dialect.js +1 -0
- package/dist/reference/dialect.js.map +1 -1
- package/dist/reference/http.d.ts +17 -25
- package/dist/reference/http.d.ts.map +1 -1
- package/dist/reference/http.js +574 -224
- package/dist/reference/http.js.map +1 -1
- package/dist/reference/listing.d.ts +51 -0
- package/dist/reference/listing.d.ts.map +1 -0
- package/dist/reference/listing.js +256 -0
- package/dist/reference/listing.js.map +1 -0
- package/dist/reference/model.d.ts +35 -1
- package/dist/reference/model.d.ts.map +1 -1
- package/dist/reference/model.js +42 -0
- package/dist/reference/model.js.map +1 -1
- package/dist/reference/query.d.ts +132 -43
- package/dist/reference/query.d.ts.map +1 -1
- package/dist/reference/query.js +397 -315
- package/dist/reference/query.js.map +1 -1
- package/dist/reference/shapes.d.ts +60 -0
- package/dist/reference/shapes.d.ts.map +1 -0
- package/dist/reference/shapes.js +142 -0
- package/dist/reference/shapes.js.map +1 -0
- package/dist/reference/spec.d.ts +9 -2
- package/dist/reference/spec.d.ts.map +1 -1
- package/dist/reference/spec.js +291 -71
- package/dist/reference/spec.js.map +1 -1
- package/dist/reference/store-api.d.ts +14 -43
- package/dist/reference/store-api.d.ts.map +1 -1
- package/dist/reference/store-api.js +8 -58
- package/dist/reference/store-api.js.map +1 -1
- package/dist/reference/stores/memory.d.ts +12 -15
- package/dist/reference/stores/memory.d.ts.map +1 -1
- package/dist/reference/stores/memory.js +22 -70
- package/dist/reference/stores/memory.js.map +1 -1
- package/dist/reference/stores/postgres.d.ts +11 -23
- package/dist/reference/stores/postgres.d.ts.map +1 -1
- package/dist/reference/stores/postgres.js +53 -334
- package/dist/reference/stores/postgres.js.map +1 -1
- package/dist/reference/stores/sql-compile.d.ts +36 -0
- package/dist/reference/stores/sql-compile.d.ts.map +1 -0
- package/dist/reference/stores/sql-compile.js +95 -0
- package/dist/reference/stores/sql-compile.js.map +1 -0
- package/dist/reference/stores/sqlite.d.ts +12 -27
- package/dist/reference/stores/sqlite.d.ts.map +1 -1
- package/dist/reference/stores/sqlite.js +54 -355
- package/dist/reference/stores/sqlite.js.map +1 -1
- package/dist/report/console.d.ts.map +1 -1
- package/dist/report/console.js +23 -9
- package/dist/report/console.js.map +1 -1
- package/dist/report/matrix.d.ts +2 -0
- package/dist/report/matrix.d.ts.map +1 -1
- package/dist/report/matrix.js +63 -10
- package/dist/report/matrix.js.map +1 -1
- package/dist/report/render.d.ts +10 -1
- package/dist/report/render.d.ts.map +1 -1
- package/dist/report/render.js +75 -13
- package/dist/report/render.js.map +1 -1
- package/dist/runtime/async.d.ts +16 -3
- package/dist/runtime/async.d.ts.map +1 -1
- package/dist/runtime/async.js +24 -19
- package/dist/runtime/async.js.map +1 -1
- package/dist/runtime/auth.d.ts +8 -9
- package/dist/runtime/auth.d.ts.map +1 -1
- package/dist/runtime/auth.js +60 -62
- package/dist/runtime/auth.js.map +1 -1
- package/dist/runtime/body.d.ts +15 -3
- package/dist/runtime/body.d.ts.map +1 -1
- package/dist/runtime/body.js +61 -12
- package/dist/runtime/body.js.map +1 -1
- package/dist/runtime/check-costs.d.ts +15 -0
- package/dist/runtime/check-costs.d.ts.map +1 -0
- package/dist/runtime/check-costs.js +114 -0
- package/dist/runtime/check-costs.js.map +1 -0
- package/dist/runtime/check-ids.d.ts +15 -0
- package/dist/runtime/check-ids.d.ts.map +1 -0
- package/dist/runtime/check-ids.js +140 -0
- package/dist/runtime/check-ids.js.map +1 -0
- package/dist/runtime/checks.d.ts +88 -20
- package/dist/runtime/checks.d.ts.map +1 -1
- package/dist/runtime/checks.js +3288 -1947
- package/dist/runtime/checks.js.map +1 -1
- package/dist/runtime/client.d.ts +112 -0
- package/dist/runtime/client.d.ts.map +1 -1
- package/dist/runtime/client.js +271 -39
- package/dist/runtime/client.js.map +1 -1
- package/dist/runtime/cookies.d.ts +16 -2
- package/dist/runtime/cookies.d.ts.map +1 -1
- package/dist/runtime/cookies.js +83 -22
- package/dist/runtime/cookies.js.map +1 -1
- package/dist/runtime/exchanges.d.ts +4 -5
- package/dist/runtime/exchanges.d.ts.map +1 -1
- package/dist/runtime/exchanges.js +33 -55
- package/dist/runtime/exchanges.js.map +1 -1
- package/dist/runtime/exit.d.ts +40 -0
- package/dist/runtime/exit.d.ts.map +1 -0
- package/dist/runtime/exit.js +48 -0
- package/dist/runtime/exit.js.map +1 -0
- package/dist/runtime/feature-gate.d.ts +0 -9
- package/dist/runtime/feature-gate.d.ts.map +1 -1
- package/dist/runtime/feature-gate.js +0 -21
- package/dist/runtime/feature-gate.js.map +1 -1
- package/dist/runtime/finding.d.ts +49 -8
- package/dist/runtime/finding.d.ts.map +1 -1
- package/dist/runtime/finding.js +51 -8
- package/dist/runtime/finding.js.map +1 -1
- package/dist/runtime/fixture.d.ts +35 -28
- package/dist/runtime/fixture.d.ts.map +1 -1
- package/dist/runtime/fixture.js +92 -731
- package/dist/runtime/fixture.js.map +1 -1
- package/dist/runtime/generate.d.ts +63 -0
- package/dist/runtime/generate.d.ts.map +1 -0
- package/dist/runtime/generate.js +635 -0
- package/dist/runtime/generate.js.map +1 -0
- package/dist/runtime/headers.d.ts +6 -0
- package/dist/runtime/headers.d.ts.map +1 -0
- package/dist/runtime/headers.js +16 -0
- package/dist/runtime/headers.js.map +1 -0
- package/dist/runtime/path.d.ts +7 -0
- package/dist/runtime/path.d.ts.map +1 -0
- package/dist/runtime/path.js +44 -0
- package/dist/runtime/path.js.map +1 -0
- package/dist/runtime/payloads.d.ts +2 -0
- package/dist/runtime/payloads.d.ts.map +1 -1
- package/dist/runtime/payloads.js +2 -1
- package/dist/runtime/payloads.js.map +1 -1
- package/dist/runtime/rate-limit.d.ts +6 -4
- package/dist/runtime/rate-limit.d.ts.map +1 -1
- package/dist/runtime/rate-limit.js +22 -33
- package/dist/runtime/rate-limit.js.map +1 -1
- package/dist/runtime/redact.d.ts +24 -0
- package/dist/runtime/redact.d.ts.map +1 -0
- package/dist/runtime/redact.js +98 -0
- package/dist/runtime/redact.js.map +1 -0
- package/dist/runtime/run.d.ts +34 -2
- package/dist/runtime/run.d.ts.map +1 -1
- package/dist/runtime/run.js +374 -138
- package/dist/runtime/run.js.map +1 -1
- package/dist/runtime/runs.d.ts +5 -0
- package/dist/runtime/runs.d.ts.map +1 -1
- package/dist/runtime/runs.js +7 -1
- package/dist/runtime/runs.js.map +1 -1
- package/dist/runtime/teardown.d.ts +17 -4
- package/dist/runtime/teardown.d.ts.map +1 -1
- package/dist/runtime/teardown.js +49 -16
- package/dist/runtime/teardown.js.map +1 -1
- package/dist/runtime/transcript.d.ts +34 -2
- package/dist/runtime/transcript.d.ts.map +1 -1
- package/dist/runtime/transcript.js +119 -24
- package/dist/runtime/transcript.js.map +1 -1
- package/dist/runtime/unique.d.ts +2 -11
- package/dist/runtime/unique.d.ts.map +1 -1
- package/dist/runtime/unique.js +27 -53
- package/dist/runtime/unique.js.map +1 -1
- package/dist/runtime/validate.d.ts +13 -2
- package/dist/runtime/validate.d.ts.map +1 -1
- package/dist/runtime/validate.js +51 -31
- package/dist/runtime/validate.js.map +1 -1
- package/dist/runtime/wait.d.ts +2 -1
- package/dist/runtime/wait.d.ts.map +1 -1
- package/dist/runtime/wait.js +2 -36
- package/dist/runtime/wait.js.map +1 -1
- package/dist/runtime/world.d.ts +24 -1
- package/dist/runtime/world.d.ts.map +1 -1
- package/dist/runtime/world.js +101 -26
- package/dist/runtime/world.js.map +1 -1
- package/dist/spec/collection.d.ts +8 -2
- package/dist/spec/collection.d.ts.map +1 -1
- package/dist/spec/collection.js +47 -7
- package/dist/spec/collection.js.map +1 -1
- package/dist/spec/extensions.d.ts +11 -1
- package/dist/spec/extensions.d.ts.map +1 -1
- package/dist/spec/extensions.js +10 -2
- package/dist/spec/extensions.js.map +1 -1
- package/dist/spec/graph.d.ts +46 -0
- package/dist/spec/graph.d.ts.map +1 -1
- package/dist/spec/graph.js +175 -11
- package/dist/spec/graph.js.map +1 -1
- package/dist/spec/load.d.ts +25 -5
- package/dist/spec/load.d.ts.map +1 -1
- package/dist/spec/load.js +160 -59
- package/dist/spec/load.js.map +1 -1
- package/dist/spec/query-capabilities.d.ts +4 -0
- package/dist/spec/query-capabilities.d.ts.map +1 -1
- package/dist/spec/query-capabilities.js +6 -0
- package/dist/spec/query-capabilities.js.map +1 -1
- package/dist/spec/schema.d.ts +20 -0
- package/dist/spec/schema.d.ts.map +1 -0
- package/dist/spec/schema.js +118 -0
- package/dist/spec/schema.js.map +1 -0
- package/dist/spec/tag-schema.d.ts +12 -0
- package/dist/spec/tag-schema.d.ts.map +1 -0
- package/dist/spec/tag-schema.js +144 -0
- package/dist/spec/tag-schema.js.map +1 -0
- package/dist/spec/types.js +17 -1
- package/dist/spec/types.js.map +1 -1
- package/docs/api.md +59 -0
- package/docs/assumptions.md +99 -0
- package/docs/checks.md +183 -0
- package/docs/commands.md +214 -0
- package/docs/configuration.md +650 -0
- package/docs/model.md +184 -0
- package/docs/reference-defects.md +94 -0
- package/docs/reports.md +211 -0
- package/docs/tags.md +403 -0
- package/package.json +3 -1
package/README.md
CHANGED
|
@@ -30,47 +30,17 @@ Those are disagreements between **projections of the same fact**. That is the ma
|
|
|
30
30
|
|
|
31
31
|
There is no ground-truth database. A filter and its negation must partition the set. A page walk must cover the collection without gaps or dupes. List, item, and `id.eq.` must show the same field. One root cause is one finding; checks that depend on a broken primitive are `BLOCKED`, not a page of copies.
|
|
32
32
|
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
- [
|
|
38
|
-
- [
|
|
39
|
-
- [
|
|
40
|
-
- [
|
|
41
|
-
- [
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
- [oat doctor](#oat-doctor)
|
|
45
|
-
- [oat plan](#oat-plan)
|
|
46
|
-
- [oat serve](#oat-serve)
|
|
47
|
-
- [oat conformance](#oat-conformance)
|
|
48
|
-
- [Configuration](#configuration)
|
|
49
|
-
- [Top-level fields](#top-level-fields)
|
|
50
|
-
- [Principals](#principals)
|
|
51
|
-
- [Auth flows](#auth-flows)
|
|
52
|
-
- [Harvested principal](#harvested-principal)
|
|
53
|
-
- [Secondary origins](#secondary-origins)
|
|
54
|
-
- [Hooks](#hooks)
|
|
55
|
-
- [Uploads](#uploads)
|
|
56
|
-
- [Environment interpolation](#environment-interpolation)
|
|
57
|
-
- [Loading TypeScript configs](#loading-typescript-configs)
|
|
58
|
-
- [How the model is derived](#how-the-model-is-derived)
|
|
59
|
-
- [Seeding](#seeding)
|
|
60
|
-
- [Query roles and grammars](#query-roles-and-grammars)
|
|
61
|
-
- [Pagination and envelopes](#pagination-and-envelopes)
|
|
62
|
-
- [OpenAPI meta tags](#openapi-meta-tags)
|
|
63
|
-
- [Checks](#checks)
|
|
64
|
-
- [Verdicts, skips, and exit codes](#verdicts-skips-and-exit-codes)
|
|
65
|
-
- [Reports](#reports)
|
|
66
|
-
- [Progress logs](#progress-logs)
|
|
67
|
-
- [Programmatic API](#programmatic-api)
|
|
68
|
-
- [CI](#ci)
|
|
69
|
-
- [Reference defects (`oat serve --defects`)](#reference-defects-oat-serve---defects)
|
|
70
|
-
- [Limits and non-features](#limits-and-non-features)
|
|
71
|
-
- [Compared to schema fuzzers](#compared-to-schema-fuzzers)
|
|
72
|
-
- [Labs](#labs)
|
|
73
|
-
- [License](#license)
|
|
33
|
+
## Reference
|
|
34
|
+
|
|
35
|
+
- [Commands](docs/commands.md) — `run`, `--ops`, `doctor`, `plan`, `serve`, `conformance`
|
|
36
|
+
- [Configuration](docs/configuration.md) — complete configs, every field, principals, auth flows, hooks, uploads
|
|
37
|
+
- [How oat reads a document](docs/model.md) — entities, identity, seeding, query grammars, pagination
|
|
38
|
+
- [OpenAPI meta tags](docs/tags.md) — every `x-*` tag and what it unlocks
|
|
39
|
+
- [Checks and verdicts](docs/checks.md) — what each check needs and asserts, verdicts, exit codes
|
|
40
|
+
- [Reports](docs/reports.md) — `oat-report.json`, `matrix.json`, repro scripts, progress logs
|
|
41
|
+
- [Programmatic API](docs/api.md)
|
|
42
|
+
- [Assumptions](docs/assumptions.md) — every assumption oat makes about an API, and how to declare an exception
|
|
43
|
+
- [Reference defects](docs/reference-defects.md) — the named lies `oat serve --defects` can tell
|
|
74
44
|
|
|
75
45
|
## Install
|
|
76
46
|
|
|
@@ -97,7 +67,7 @@ Unknown commands, unknown flags, and missing required flags exit `2`.
|
|
|
97
67
|
2. **Model** entities by inverting `x-invalidate` (or path heuristics) into a read surface per entity.
|
|
98
68
|
3. **Authenticate** every configured principal (static headers and/or an auth flow). Credentials refresh on a countdown from `exp` (default 30s buffer) before every dispatch and each async poll.
|
|
99
69
|
4. **Seed** a cohort of records per entity, in parent-before-child order, using each entity's create operation.
|
|
100
|
-
5. **Test** the matrix
|
|
70
|
+
5. **Test** the matrix entity by entity: foundations first, then composition, writes, isolation, declared effects. Entities that can see each other's writes — a parent and its children, a route one invalidates, an effect one lands on another — run in series; entities that share nothing run side by side. Checks inside an entity stay ordered.
|
|
101
71
|
6. **Teardown** everything the run created, unless `--keep-fixtures` / `keepFixtures: true`.
|
|
102
72
|
|
|
103
73
|
The first principal is the writer. Isolation needs a second principal with different `roots`. A rank lattice needs two or more principals that share `roots` and differ in `rank`. Invite checks need `x-invite` plus a peer with `inviteAs`.
|
|
@@ -137,1979 +107,28 @@ oat run --config oat.config.ts
|
|
|
137
107
|
|
|
138
108
|
`doctor` is the adoption command. It runs offline against the spec alone and reports every coverage gap, naming the tag that would close it.
|
|
139
109
|
|
|
140
|
-
##
|
|
141
|
-
|
|
142
|
-
These are copy-paste starting points. Real configs import `defineConfig` from `@lovrozagar/oat`. Files inside this repository import from `../dist/index.js` because they live in the source tree.
|
|
143
|
-
|
|
144
|
-
### Static API keys, two tenants (smallest useful)
|
|
145
|
-
|
|
146
|
-
```ts
|
|
147
|
-
// oat.config.ts
|
|
148
|
-
import { defineConfig } from "@lovrozagar/oat"
|
|
149
|
-
|
|
150
|
-
export default defineConfig({
|
|
151
|
-
spec: "https://api.example.com/openapi.json",
|
|
152
|
-
baseUrl: "https://api.example.com",
|
|
153
|
-
principals: [
|
|
154
|
-
{
|
|
155
|
-
id: "alpha",
|
|
156
|
-
headers: { authorization: "Bearer ${API_TOKEN}" },
|
|
157
|
-
roots: { project_id: "${PROJECT_A}" },
|
|
158
|
-
},
|
|
159
|
-
{
|
|
160
|
-
id: "beta",
|
|
161
|
-
headers: { authorization: "Bearer ${API_TOKEN_B}" },
|
|
162
|
-
roots: { project_id: "${PROJECT_B}" },
|
|
163
|
-
},
|
|
164
|
-
],
|
|
165
|
-
})
|
|
166
|
-
```
|
|
167
|
-
|
|
168
|
-
```bash
|
|
169
|
-
export API_TOKEN=… API_TOKEN_B=… PROJECT_A=proj_a PROJECT_B=proj_b
|
|
170
|
-
oat run --config oat.config.ts
|
|
171
|
-
```
|
|
172
|
-
|
|
173
|
-
One principal is enough to seed and run CRUD / query / schema checks. The second principal, with different `roots`, is what makes `tenant.*` run. Without it those checks are **did not apply**, not a pass.
|
|
174
|
-
|
|
175
|
-
Shipped as `labs/minimal.config.ts` (and in the npm package).
|
|
176
|
-
|
|
177
|
-
### JSON config
|
|
178
|
-
|
|
179
|
-
Same object. `${NAME}` is interpolated after load. There is no `defineConfig` wrapper.
|
|
180
|
-
|
|
181
|
-
```json
|
|
182
|
-
{
|
|
183
|
-
"spec": "https://api.example.com/openapi.json",
|
|
184
|
-
"baseUrl": "https://api.example.com",
|
|
185
|
-
"principals": [
|
|
186
|
-
{
|
|
187
|
-
"id": "alpha",
|
|
188
|
-
"headers": { "authorization": "Bearer ${API_TOKEN}" },
|
|
189
|
-
"roots": { "project_id": "${PROJECT_A}" }
|
|
190
|
-
}
|
|
191
|
-
],
|
|
192
|
-
"seed": 42,
|
|
193
|
-
"cohortSize": 7,
|
|
194
|
-
"outDir": "./.oat/runs"
|
|
195
|
-
}
|
|
196
|
-
```
|
|
197
|
-
|
|
198
|
-
```bash
|
|
199
|
-
oat run --config oat.config.json
|
|
200
|
-
```
|
|
201
|
-
|
|
202
|
-
### Demo server (operation-id login)
|
|
203
|
-
|
|
204
|
-
Shipped as `labs/local.config.ts`. Points at `oat serve`.
|
|
205
|
-
|
|
206
|
-
```ts
|
|
207
|
-
import { defineConfig } from "@lovrozagar/oat"
|
|
208
|
-
|
|
209
|
-
export default defineConfig({
|
|
210
|
-
spec: "/v1/openapi/spec",
|
|
211
|
-
baseUrl: "http://127.0.0.1:8787",
|
|
212
|
-
seed: 42,
|
|
213
|
-
principals: [
|
|
214
|
-
{
|
|
215
|
-
id: "alpha",
|
|
216
|
-
roots: { project_id: "proj_alpha" },
|
|
217
|
-
auth: {
|
|
218
|
-
credentialFrom: "$.access_token",
|
|
219
|
-
steps: [{ operationId: "auth.token", body: { key: "key_alpha" } }],
|
|
220
|
-
},
|
|
221
|
-
},
|
|
222
|
-
{
|
|
223
|
-
id: "beta",
|
|
224
|
-
roots: { project_id: "proj_beta" },
|
|
225
|
-
auth: {
|
|
226
|
-
credentialFrom: "$.access_token",
|
|
227
|
-
steps: [{ operationId: "auth.token", body: { key: "key_beta" } }],
|
|
228
|
-
},
|
|
229
|
-
},
|
|
230
|
-
],
|
|
231
|
-
})
|
|
232
|
-
```
|
|
233
|
-
|
|
234
|
-
`spec: "/v1/openapi/spec"` is resolved against `baseUrl`. `--base-url` on the CLI overrides the origin without editing the file.
|
|
235
|
-
|
|
236
|
-
### Two tenants plus a same-tenant rank lattice
|
|
237
|
-
|
|
238
|
-
See [Principals](#principals). Isolation keys off `roots`. Rank keys off `rank` with shared `roots`.
|
|
239
|
-
|
|
240
|
-
## Commands
|
|
241
|
-
|
|
242
|
-
```
|
|
243
|
-
oat run --config <file> test a live backend and write a report
|
|
244
|
-
oat doctor --spec <url|file> what oat can and cannot test, and why
|
|
245
|
-
oat plan --spec <url|file> print the derived model (offline)
|
|
246
|
-
oat serve [--defects A,B] run the demo API
|
|
247
|
-
oat conformance self-test: injected defects vs detection
|
|
248
|
-
oat help
|
|
249
|
-
```
|
|
250
|
-
|
|
251
|
-
`--spec` for `doctor` / `plan` can be replaced by `--config` (the spec is read from the config). `--json` makes those two commands emit machine-readable output. `--base-url` on `doctor` / `plan` is only used to resolve a relative spec path.
|
|
252
|
-
|
|
253
|
-
`--untagged` is a **serve** flag (and a conformance concern). It is not a `run` flag.
|
|
254
|
-
|
|
255
|
-
### `oat run`
|
|
256
|
-
|
|
257
|
-
Requires `--config`. CLI flags override the same field in the config when both are set.
|
|
258
|
-
|
|
259
|
-
| flag | default | meaning |
|
|
260
|
-
| ------------------------------------------ | -------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
261
|
-
| `--config` | required | module or JSON file, default export |
|
|
262
|
-
| `--base-url` | `config.baseUrl` | backend origin |
|
|
263
|
-
| `--ops` | `config.ops` or every operation | comma-separated operationIds to grade; everything else is support. `*` globs, `<originId>:` prefixes. See [Targeted runs](#targeted-runs---ops) |
|
|
264
|
-
| `--only` | `config.only` or all entities | comma-separated entity names as `oat plan` prints them (singularised); each grades every operation it owns. Joins `--ops` as a union |
|
|
265
|
-
| `--seed` | `config.seed` or `1` | fixture generation seed (reproducible) |
|
|
266
|
-
| `--out` | `config.outDir` or `./.oat/runs` | history root; each run writes `<out>/<datetime>/` and updates `latest` |
|
|
267
|
-
| `--max-in-flight` | `config.maxInFlight` or `4` | HTTP requests allowed at once |
|
|
268
|
-
| `--keep-fixtures` | `config.keepFixtures` or false | do not DELETE what the run created |
|
|
269
|
-
| `--quiet` | false | no stderr progress; files under `--out` still update |
|
|
270
|
-
| `--save-exchanges` / `--no-save-exchanges` | on unless `--profile cheap` | persist every HTTP exchange under the run dir (`exchanges.jsonl`, `exchanges/`, `blobs/`). Does **not** change in-memory transcript RAM |
|
|
271
|
-
|
|
272
|
-
**Exit codes:** `0` no defects, `1` at least one root-cause finding (`BACKEND_BUG`, `SPEC_BUG`, `SECURITY`, `AMBIGUITY`) **or** the run stopped because the network never came back **or**, on a targeted run, a target that did not end `held`, `2` usage error (missing `--config`, no principals, unknown flag, an `--ops` / `--only` name that matches nothing, a target the profile excludes). `COVERAGE_GAP` and `BLOCKED` do not fail a full run.
|
|
273
|
-
|
|
274
|
-
Example:
|
|
275
|
-
|
|
276
|
-
```bash
|
|
277
|
-
oat run --config oat.config.ts --only store,product --out .oat/runs/prod
|
|
278
|
-
```
|
|
279
|
-
|
|
280
|
-
`--only store,product` matches the **entity names** from `oat plan`, not path segments. `/v1/stores` is usually the entity `store`. An unknown name exits `2` with the nearest names.
|
|
281
|
-
|
|
282
|
-
A run with no principals exits `2`. Isolation checks then need a second principal; they are skipped, not failed, when only one is present.
|
|
283
|
-
|
|
284
|
-
### Targeted runs (`--ops`)
|
|
285
|
-
|
|
286
|
-
A full run grades every operation. After changing two endpoints you usually want those two, not the whole API:
|
|
287
|
-
|
|
288
|
-
```bash
|
|
289
|
-
oat run --config oat.config.ts --ops table.create,row.list
|
|
290
|
-
oat run --config oat.config.ts --ops 'row.*' # glob within an operationId
|
|
291
|
-
oat run --config oat.config.ts --ops cdn:asset.get # an operation on the origin with id "cdn"
|
|
292
|
-
oat plan --config oat.config.ts --ops table.create # dry run: what would be graded, queued, called
|
|
293
|
-
```
|
|
294
|
-
|
|
295
|
-
**Targets** are graded: every check whose subjects include a target runs the full matrix against it. **Support** operations are called and never graded: principal sign-up, parent creates, the cohort create behind a list, reads used as an oracle, teardown deletes. A list route cannot be tested without rows in it, so its create still runs, but a defect in that create is out of this run's scope.
|
|
296
|
-
|
|
297
|
-
How the scope is decided:
|
|
298
|
-
|
|
299
|
-
- Every check declares its **subjects**, the operations whose contract it judges (`filter.*` judges the list route, `patch.minimality` the update, `effects.declared-effect-occurs` each `x-effects` operation). A check runs when one of its subjects is a target. Others are **out of scope**: not run, not skipped.
|
|
300
|
-
- An entity is seeded only when one of its checks grades a target. Declared edges pull in the other direction too: targeting a read route that another entity's create names in `x-invalidate` queues that entity, so its invalidation check grades the route.
|
|
301
|
-
- Checks that judge several operations (`effects.*`, `async.*`, `response.status-is-documented`, `schema.error-response-matches-document`, `invalidation.declared-route-changes`) invoke and judge only the targeted ones. An untargeted `x-effects` upload is never POSTed.
|
|
302
|
-
- Every finding names the operations it judges (`operations` in `oat-report.json`).
|
|
303
|
-
- An operation counts as graded only when a check exercised it. A check that judges after the fact (`response.status-is-documented` reads the transcript) grades only the operations it saw. A check that reports a coverage gap for an operation did not grade it. `--only <entity>` therefore surfaces every action no check can reach as `untested`.
|
|
304
|
-
|
|
305
|
-
Each target ends with one status:
|
|
306
|
-
|
|
307
|
-
| status | meaning |
|
|
308
|
-
| -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
309
|
-
| `held` | at least one check graded it and none failed |
|
|
310
|
-
| `failed` | a defect finding is attributed to it |
|
|
311
|
-
| `inconclusive` | checks ran and none could decide |
|
|
312
|
-
| `blocked` | every check was suppressed, or a support operation it needed failed (named, with evidence) |
|
|
313
|
-
| `untested` | nothing exercised it. The reason says why (`unmodeled`, `entity-not-testable`, `coverage gap: …`, `no check applied: needs …`, `no check exercised it`) |
|
|
314
|
-
|
|
315
|
-
A targeted run exits `0` only when every target is `held`. A target nothing could grade must not read as a pass.
|
|
316
|
-
|
|
317
|
-
Validation happens before any principal signs up: a name that matches nothing exits `2` with the nearest operationIds; an exact target the active `--profile` excludes exits `2`; a glob or `--only` match the profile excludes is dropped and listed. An origin no target names is not run.
|
|
318
|
-
|
|
319
|
-
A targeted run is for iterating. It grades what you named and what those operations observably affect through declared edges. It does not catch a regression in an untargeted operation caused by shared code, so run the full suite before a release.
|
|
320
|
-
|
|
321
|
-
### `oat doctor`
|
|
322
|
-
|
|
323
|
-
Offline. Loads the document, builds the model, prints coverage.
|
|
324
|
-
|
|
325
|
-
```bash
|
|
326
|
-
oat doctor --spec https://api.example.com/openapi.json
|
|
327
|
-
oat doctor --config oat.config.ts --json
|
|
328
|
-
oat doctor --spec ./openapi.yaml --base-url https://api.example.com
|
|
329
|
-
```
|
|
330
|
-
|
|
331
|
-
Human output:
|
|
332
|
-
|
|
333
|
-
- `trackable` — entities with an identity and a read surface
|
|
334
|
-
- `listable` — those that also have a list (query checks need this)
|
|
335
|
-
- tags that are absent, and the checks each tag would unlock
|
|
336
|
-
- tags that would **sharpen** checks that already run (`x-query`, `x-tenant`)
|
|
337
|
-
- per-operation gaps (`x-entity` could not be inferred, assumed tenant param, …)
|
|
338
|
-
- external `$ref`s that were not fetched
|
|
339
|
-
|
|
340
|
-
`--json` shape:
|
|
341
|
-
|
|
342
|
-
```json
|
|
343
|
-
{
|
|
344
|
-
"blocking": 1,
|
|
345
|
-
"entities": 12,
|
|
346
|
-
"trackableEntities": 10,
|
|
347
|
-
"testableEntities": 10,
|
|
348
|
-
"listableEntities": 8,
|
|
349
|
-
"roots": ["organization_id"],
|
|
350
|
-
"externalRefs": ["https://example.com/shared.yaml"],
|
|
351
|
-
"gaps": [{ "operationId": "table.list", "tag": "x-query", "detail": "…" }]
|
|
352
|
-
}
|
|
353
|
-
```
|
|
354
|
-
|
|
355
|
-
Exit `1` if there are **blocking** gaps: entities that are not trackable (no identity / no read), or the document has roots oat cannot create. Advisory gaps (missing `x-query`, no `x-async`) print and still exit `0`.
|
|
356
|
-
|
|
357
|
-
### `oat plan`
|
|
358
|
-
|
|
359
|
-
Offline. Prints the derived entity graph, operations, and query capability.
|
|
360
|
-
|
|
361
|
-
```bash
|
|
362
|
-
oat plan --spec ./openapi.yaml
|
|
363
|
-
oat plan --config oat.config.ts --json
|
|
364
|
-
```
|
|
365
|
-
|
|
366
|
-
Human columns:
|
|
367
|
-
|
|
368
|
-
```
|
|
369
|
-
entity CLRUD ident read surface
|
|
370
|
-
store CLRU· id 2 route(s) (inferred)
|
|
371
|
-
GET /v1/stores
|
|
372
|
-
GET /v1/stores/{store_id}
|
|
373
|
-
```
|
|
374
|
-
|
|
375
|
-
`CLRUD` is Create / List / Read / Update / Delete. `·` means that slot is missing. `ident` is the identity property. Read surface is declared (`x-invalidate`) or inferred (sibling collection/item routes).
|
|
376
|
-
|
|
377
|
-
`--json` is `{ entities, operations, roots }` — the full `SpecModel` maps, including conventions, query capability, async, invite, and path params. Use this when you need to know what oat will call something.
|
|
378
|
-
|
|
379
|
-
With `--ops` / `--only` (flag or config), `plan` prints the targeted scope instead: each target with its entity and the checks that would grade it, untestable targets with the reason, the entities that would be queued, and the support operations predicted from the graph. `--json` adds a `scope` object with the same fields. An unknown name exits `2`.
|
|
380
|
-
|
|
381
|
-
### `oat serve`
|
|
382
|
-
|
|
383
|
-
In-process demo API. Same fixture as conformance.
|
|
384
|
-
|
|
385
|
-
```bash
|
|
386
|
-
oat serve
|
|
387
|
-
oat serve --defects STALE_LIST,PATCH_REPLACES
|
|
388
|
-
oat serve --backend sqlite --dialect classic
|
|
389
|
-
oat serve --untagged
|
|
390
|
-
```
|
|
391
|
-
|
|
392
|
-
| flag | default | meaning |
|
|
393
|
-
| ------------ | ----------- | -------------------------------------------------------------------------------------- |
|
|
394
|
-
| `--backend` | `memory` | `memory` \| `sqlite` \| `postgres` |
|
|
395
|
-
| `--dialect` | `postgrest` | `postgrest` \| `classic` \| `linked` \| `jsonapi` \| `plain` |
|
|
396
|
-
| `--defects` | none | comma-separated names from [Reference defects](#reference-defects-oat-serve---defects) |
|
|
397
|
-
| `--untagged` | false | serve the same API behind a spec with every `x-*` tag stripped |
|
|
398
|
-
|
|
399
|
-
Printed keys: `key_alpha` (tenant `proj_alpha`), `key_beta` (tenant `proj_beta`). Spec: `{url}/v1/openapi/spec`. Stop with ctrl-c.
|
|
400
|
-
|
|
401
|
-
`labs/local.config.ts` is written for this server.
|
|
402
|
-
|
|
403
|
-
Dialects are **reference-backend shapes**, not something you configure against your API. They exist so conformance proves checks read the document rather than one fixture's spelling:
|
|
404
|
-
|
|
405
|
-
| dialect | filter | sort | select | page model | envelope |
|
|
406
|
-
| ----------- | --------------------------- | ---------- | ---------------- | ------------------- | --------------------------------- |
|
|
407
|
-
| `postgrest` | `filter=status.eq.active` | `name.asc` | `select=id,name` | `page` + `cursor` | entity-named + `count`/`hasMore` |
|
|
408
|
-
| `classic` | `filter=status=eq:active` | `sort=` | `fields=` | `page` + `per_page` | `{ data, total_count, has_more }` |
|
|
409
|
-
| `linked` | postgrest | dotted | `fields=` | `offset` + `limit` | raw array + `Link: rel=next` |
|
|
410
|
-
| `jsonapi` | postgrest | `-name` | `fields[table]=` | `page` + `size` | `{ data, total, has_more }` |
|
|
411
|
-
| `plain` | `?status=active` (equality) | `name:asc` | `fields=` | `page` + `limit` | `{ items, total, has_more }` |
|
|
412
|
-
|
|
413
|
-
`postgres` needs a server on the default `postgres` database (local, default `postgres` driver connection). `sqlite` needs Node's `node:sqlite` (`--experimental-sqlite` on Node 22). Missing backends fail at serve time rather than falling back.
|
|
414
|
-
|
|
415
|
-
### `oat conformance`
|
|
416
|
-
|
|
417
|
-
Self-test. Not for your API. Injects named defects into the reference backend and asserts oat reports the matching check.
|
|
418
|
-
|
|
419
|
-
```bash
|
|
420
|
-
# this repo
|
|
421
|
-
npm test
|
|
422
|
-
|
|
423
|
-
# after install, from a checkout with --experimental-sqlite if you want sqlite
|
|
424
|
-
oat conformance
|
|
425
|
-
oat conformance --backend memory --dialect plain
|
|
426
|
-
oat conformance --fuzz 300 --max-defects 12 --seed 7
|
|
427
|
-
oat conformance --precision 60 --backend memory
|
|
428
|
-
oat conformance --parser
|
|
429
|
-
oat conformance --backend d1
|
|
430
|
-
oat conformance --only STALE_LIST,PATCH_REPLACES
|
|
431
|
-
```
|
|
432
|
-
|
|
433
|
-
| flag | meaning |
|
|
434
|
-
| ----------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
|
|
435
|
-
| `--backend` | `memory` \| `sqlite` \| `postgres` \| `d1`. Default: every _local_ backend that is available. `d1` is never default — it is remote |
|
|
436
|
-
| `--dialect` | pin one shape; default runs postgrest on each backend plus classic/linked/jsonapi/plain on memory |
|
|
437
|
-
| `--fuzz [n]` | random _sets_ of defects (default 25 if flag is bare) |
|
|
438
|
-
| `--max-defects` | cap per fuzz combination (default 4) |
|
|
439
|
-
| `--precision [n]` | vary _data_ against a correct backend; any finding is a false positive (default 50 if flag is bare) |
|
|
440
|
-
| `--seed` | replay a fuzz/precision run |
|
|
441
|
-
| `--parser` | only the hostile-document + example-spec + tag-unlock suites |
|
|
442
|
-
| `--only` | restrict injected defects (comma-separated `STALE_LIST,…`) |
|
|
443
|
-
|
|
444
|
-
A default `oat conformance` (no `--fuzz` / `--precision` / `--parser`) also runs a 40-case combination smoke on memory after the one-at-a-time matrix.
|
|
445
|
-
|
|
446
|
-
D1 needs `CLOUDFLARE_ACCOUNT_ID`, `CLOUDFLARE_D1_DATABASE_ID`, `CLOUDFLARE_API_TOKEN`. Postgres needs a reachable server on the default connection (`database: "postgres"`). Missing backends are skipped with a printed reason, not treated as a pass.
|
|
447
|
-
|
|
448
|
-
`--parser` still always runs first (hostile documents, `labs/annotated-openapi.yaml` model lock, tag-unlock map). Exit `1` if any parser, matrix, fuzz, or precision case fails.
|
|
449
|
-
|
|
450
|
-
## Configuration
|
|
451
|
-
|
|
452
|
-
Two inputs, always: **the spec** and **a config file**. Backend-specific knowledge lives in `x-*` tags and this file. A backend adopts oat by adding tags, not by adapting to oat.
|
|
453
|
-
|
|
454
|
-
A config is:
|
|
455
|
-
|
|
456
|
-
- `.ts` / `.js` / `.mjs` with `export default defineConfig({ ... })`, or
|
|
457
|
-
- `.json` with the same object.
|
|
458
|
-
|
|
459
|
-
Named export without `default` is also accepted (`module.default ?? module`).
|
|
460
|
-
|
|
461
|
-
### Top-level fields
|
|
462
|
-
|
|
463
|
-
```ts
|
|
464
|
-
import { defineConfig } from "@lovrozagar/oat"
|
|
465
|
-
|
|
466
|
-
export default defineConfig({
|
|
467
|
-
spec: "https://api.example.com/openapi.json", // URL or filesystem path; JSON or YAML
|
|
468
|
-
baseUrl: "https://api.example.com",
|
|
469
|
-
principals: [/* at least one; see below */],
|
|
470
|
-
hooks: {/* optional */},
|
|
471
|
-
uploads: { pool: ["./fixtures/**/*"], eachMax: 24 },
|
|
472
|
-
globalHeaders: { "x-request-id": "oat" }, // sent on every request; oat does not inspect them
|
|
473
|
-
roots: { org_id: "org_shared" }, // path params oat cannot create; also declarable via x-root
|
|
474
|
-
seed: 42, // fixture generation; a failing run with the same seed is identical
|
|
475
|
-
cohortSize: 12, // records created per entity (default 7)
|
|
476
|
-
maxInFlight: 4, // HTTP in flight
|
|
477
|
-
ops: ["store.create", "product.*"], // grade only these operationIds; the rest is support
|
|
478
|
-
only: ["store", "product"], // grade every operation these entities own (union with ops)
|
|
479
|
-
keepFixtures: false,
|
|
480
|
-
outDir: "./.oat/runs",
|
|
481
|
-
saveExchanges: true, // default on unless --profile cheap; --quiet does not turn this off
|
|
482
|
-
network: { retries: 4, waitMs: 60_000 }, // fetch-threw: retry, then wait for the link
|
|
483
|
-
outOfBand: { attempts: 20, initialMs: 1000, maxMs: 8000 },
|
|
484
|
-
origins: [{ id: "cdn", baseUrl: "https://cdn.example.com", spec: "https://cdn.example.com/openapi.json" }],
|
|
485
|
-
query: {
|
|
486
|
-
operators: ["eq", "neq", "gt", "gte", "lt", "lte", "in", "nin", "like", "ilike", "is"],
|
|
487
|
-
emptyIn: "match-none",
|
|
488
|
-
maxInValues: 100,
|
|
489
|
-
searchEmpty: "match-all",
|
|
490
|
-
sort: { nulls: ["first", "last"], maxKeys: 3 },
|
|
491
|
-
select: { unknown: "reject" },
|
|
492
|
-
},
|
|
493
|
-
entities: {
|
|
494
|
-
row: {
|
|
495
|
-
query: {
|
|
496
|
-
identityFilter: "_id",
|
|
497
|
-
filterable: [{ field: "_id", type: "string", ops: ["eq", "neq", "in"] }],
|
|
498
|
-
},
|
|
499
|
-
},
|
|
500
|
-
},
|
|
501
|
-
})
|
|
502
|
-
```
|
|
503
|
-
|
|
504
|
-
| field | required | default | notes |
|
|
505
|
-
| --------------- | -------- | --------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
506
|
-
| `spec` | yes | | See [Spec loading](#spec-loading) |
|
|
507
|
-
| `baseUrl` | yes | | Primary origin. OpenAPI `servers[]` is ignored |
|
|
508
|
-
| `principals` | yes | | Non-empty. First is the writer |
|
|
509
|
-
| `hooks` | no | | See [Hooks](#hooks) |
|
|
510
|
-
| `uploads` | no | | `pool` globs; optional `each` (operationId → globs) and `eachMax`. JSON configs may set all three |
|
|
511
|
-
| `globalHeaders` | no | `{}` | Merged first. `resolveHeaders` then caller headers then auth |
|
|
512
|
-
| `origins` | no | `[]` | Extra `{ id, baseUrl, spec }` hosts. Auth JWT is reused. Do not merge those routes into `spec` |
|
|
513
|
-
| `outOfBand` | no | `{ attempts: 6, initialMs: 200, maxMs: 3000 }` | Backoff for `resolveOutOfBand` and `resolvePrincipalAuth`. See [Hooks](#hooks) |
|
|
514
|
-
| `roots` | no | `{}` | Shared path params (merged with each principal's `roots`) |
|
|
515
|
-
| `seed` | no | `1` | Integer. Same seed → same fixture bodies |
|
|
516
|
-
| `cohortSize` | no | `7` | Sliced from the 7 built-in variants. Larger repeats the pattern |
|
|
517
|
-
| `maxInFlight` | no | `4` | Across the whole run |
|
|
518
|
-
| `only` | no | all | Entity names from `oat plan` |
|
|
519
|
-
| `keepFixtures` | no | `false` | Skip DELETE at the end |
|
|
520
|
-
| `outDir` | no | `./.oat/runs` | History root. Each run writes `<outDir>/<datetime>/` and updates `latest`. Also writes `principals.json` after acquire |
|
|
521
|
-
| `saveExchanges` | no | on unless `profile` is `cheap` | Persist every HTTP exchange under the run dir. `--save-exchanges` / `--no-save-exchanges` override. `--quiet` does not |
|
|
522
|
-
| `network` | no | `{ retries: 4, waitMs: 60000, requestTimeoutMs: 180000 }` | When `fetch` throws (offline / DNS / reset / timeout): retry, then wait once for the link. Not a 5xx policy. `requestTimeoutMs: 0` disables the per-attempt cap |
|
|
523
|
-
| `query` | no | | Global query-catalog defaults. Overlay after `x-query`. Does not invent operators |
|
|
524
|
-
| `entities` | no | | Per-entity overlays. This release only reads `query`. Unknown names are ignored; `doctor` warns |
|
|
525
|
-
|
|
526
|
-
`spec` may be a path relative to `baseUrl` (`/v1/openapi/spec`) or an absolute URL or a file.
|
|
527
|
-
|
|
528
|
-
CLI `--base-url`, `--ops`, `--only`, `--seed`, `--out`, `--max-in-flight`, `--keep-fixtures`, `--save-exchanges` / `--no-save-exchanges` override these when passed.
|
|
529
|
-
|
|
530
|
-
### Spec loading
|
|
531
|
-
|
|
532
|
-
Resolved in this order, never by guessing the string's "look":
|
|
533
|
-
|
|
534
|
-
1. Absolute `http(s)://` or `file://` — used as given.
|
|
535
|
-
2. A path that exists on disk, relative to the working directory.
|
|
536
|
-
3. Anything else, when `baseUrl` is known — resolved against it (`/v1/openapi/spec`, `openapi.json`).
|
|
537
|
-
|
|
538
|
-
JSON if the first non-space character is `{` or `[`, otherwise YAML. Empty files error. A JSON document with more opening than closing brackets is diagnosed as truncated (proxy / download limit), not as a syntax error.
|
|
539
|
-
|
|
540
|
-
OpenAPI 3.0 and 3.1 both work. oat reads `paths`, operations, parameters, request/response JSON schemas, and `x-*` extensions. It does not require a particular `openapi:` version string.
|
|
541
|
-
|
|
542
|
-
Internal `$ref`s are dereferenced. External `$ref`s stay unresolved and show up in `oat doctor` / the JSON `externalRefs` list.
|
|
543
|
-
|
|
544
|
-
### Principals
|
|
545
|
-
|
|
546
|
-
```ts
|
|
547
|
-
{
|
|
548
|
-
id: "alpha", // required, stable name in reports
|
|
549
|
-
headers: { authorization: "Bearer …" }, // static; enough for a long-lived key
|
|
550
|
-
auth: { /* AuthFlow — see below */ },
|
|
551
|
-
roots: { org_id: "org_alpha" }, // this principal's tenant / path params
|
|
552
|
-
rootsFromFlow: { org_id: "orgId" }, // take path params from values the auth flow bound
|
|
553
|
-
role: "owner", // free-form label in reports
|
|
554
|
-
rank: 2, // higher can do everything a lower rank can; default 0
|
|
555
|
-
inviteAs: "key_beta", // how an owner names this principal in an invite body
|
|
556
|
-
}
|
|
557
|
-
```
|
|
558
|
-
|
|
559
|
-
Rules that matter:
|
|
560
|
-
|
|
561
|
-
- **Isolation** (`tenant.*`) needs two principals whose `roots` differ.
|
|
562
|
-
- **Rank** (`auth.rank-is-monotonic`) needs two principals with the _same_ `roots` and different `rank`.
|
|
563
|
-
- **Invite** (`auth.invite-grants-then-revokes`) needs `x-invite` on the spec and a _different-tenant_ principal with `inviteAs` set.
|
|
564
|
-
- Extra principals are not ignored. Isolation picks the first different-`roots` peer. Rank uses the same-tenant pair.
|
|
565
|
-
- `headers` and `auth` compose: static headers are sent, then the flow's credential header is merged on top.
|
|
566
|
-
- A principal with only `headers` (no `auth`) never hits a login route.
|
|
567
|
-
- `auth` may be a harvested credential instead of a step chain: `{ fromHook: "oauth-google" }`. See [Harvested principal](#harvested-principal).
|
|
568
|
-
|
|
569
|
-
Example — two tenants plus a same-tenant lattice:
|
|
570
|
-
|
|
571
|
-
```ts
|
|
572
|
-
principals: [
|
|
573
|
-
{
|
|
574
|
-
id: "alpha",
|
|
575
|
-
role: "owner",
|
|
576
|
-
rank: 2,
|
|
577
|
-
auth: {
|
|
578
|
-
credentialFrom: "$.access_token",
|
|
579
|
-
steps: [{ operationId: "auth.token", body: { key: "key_alpha" } }],
|
|
580
|
-
},
|
|
581
|
-
roots: { org_id: "org_alpha" },
|
|
582
|
-
},
|
|
583
|
-
{
|
|
584
|
-
id: "alpha_member",
|
|
585
|
-
role: "member",
|
|
586
|
-
rank: 1,
|
|
587
|
-
auth: {
|
|
588
|
-
credentialFrom: "$.access_token",
|
|
589
|
-
steps: [{ operationId: "auth.token", body: { key: "key_alpha_member" } }],
|
|
590
|
-
},
|
|
591
|
-
roots: { org_id: "org_alpha" },
|
|
592
|
-
},
|
|
593
|
-
{
|
|
594
|
-
id: "beta",
|
|
595
|
-
role: "owner",
|
|
596
|
-
rank: 2,
|
|
597
|
-
inviteAs: "key_beta",
|
|
598
|
-
auth: {
|
|
599
|
-
credentialFrom: "$.access_token",
|
|
600
|
-
steps: [{ operationId: "auth.token", body: { key: "key_beta" } }],
|
|
601
|
-
},
|
|
602
|
-
roots: { org_id: "org_beta" },
|
|
603
|
-
},
|
|
604
|
-
]
|
|
605
|
-
```
|
|
606
|
-
|
|
607
|
-
### Auth flows
|
|
608
|
-
|
|
609
|
-
```ts
|
|
610
|
-
auth: {
|
|
611
|
-
steps: [ /* register / verify — first acquire only */ ],
|
|
612
|
-
credentialFrom: "$.access_token", // JSON path in the last (or saved) response
|
|
613
|
-
expiresInFrom: "$.expires_in", // lifetime in seconds
|
|
614
|
-
header: "authorization", // default
|
|
615
|
-
template: "Bearer {credential}", // default
|
|
616
|
-
assumeTtlMs: 3600000, // used only if neither expiresInFrom nor JWT exp is present
|
|
617
|
-
refreshBufferMs: 30_000, // optional; default 30s. Proactive when expiresAt - now <= this
|
|
618
|
-
refresh: { // signup flows must set this — re-running steps is not a refresh
|
|
619
|
-
steps: [
|
|
620
|
-
{
|
|
621
|
-
operationId: "auth.refreshToken",
|
|
622
|
-
body: { refresh_token: "{refreshToken}" },
|
|
623
|
-
saveAs: {
|
|
624
|
-
credential: "$.access_token",
|
|
625
|
-
refreshToken: "$.refresh_token",
|
|
626
|
-
},
|
|
627
|
-
},
|
|
628
|
-
],
|
|
629
|
-
},
|
|
630
|
-
}
|
|
631
|
-
```
|
|
632
|
-
|
|
633
|
-
Expiry is `expiresInFrom` (seconds) → JWT `exp` claim → `assumeTtlMs`. `assumeTtlMs` is only the fallback lifetime when nothing else revealed expiry — it is not used for the refresh threshold when `expiresAt` is known.
|
|
634
|
-
|
|
635
|
-
Refresh is countdown-based (`expiresAt - refreshBufferMs`, default 30s) before every dispatch and before each async poll. Signup flows declare `auth.refresh` (refresh-token operation). Re-running `steps` is only the fallback when `refresh` is omitted (API-key / token-exchange principals). A register-like first hop without `refresh` fails closed (`AUTH_REFRESH_REQUIRED`) rather than signing up again.
|
|
636
|
-
|
|
637
|
-
One 401 → force refresh + single retry with live headers. A second 401 is evidence. 5xx / 429 are never a refresh trigger. `expiresAt === null` (static-header principal) never proactive-refreshes.
|
|
638
|
-
|
|
639
|
-
Each step is one of:
|
|
640
|
-
|
|
641
|
-
**Operation step** (prefer this — survives the path moving):
|
|
642
|
-
|
|
643
|
-
```ts
|
|
644
|
-
{
|
|
645
|
-
operationId: "auth.token",
|
|
646
|
-
body: { key: "${API_KEY}" },
|
|
647
|
-
headers: { "x-extra": "1" },
|
|
648
|
-
query: { realm: "test" },
|
|
649
|
-
saveAs: { credential: "$.access_token", refreshToken: "$.refresh_token" },
|
|
650
|
-
saveClaimsFrom: { token: "$.access_token", bind: { orgId: "orgs.0.oid" } },
|
|
651
|
-
bind: { address: "user@example.test" }, // literals, with {name} interpolation
|
|
652
|
-
expect: [200], // default: any 2xx
|
|
653
|
-
}
|
|
654
|
-
```
|
|
655
|
-
|
|
656
|
-
**Request step** (when the document has no auth operations, or the hop is not in the spec):
|
|
657
|
-
|
|
658
|
-
```ts
|
|
659
|
-
{
|
|
660
|
-
method: "POST",
|
|
661
|
-
path: "/v1/auth/register/email",
|
|
662
|
-
body: { email: "{address}", password: "…" },
|
|
663
|
-
bind: { address: "oat-alpha@example.test" },
|
|
664
|
-
}
|
|
665
|
-
```
|
|
666
|
-
|
|
667
|
-
`path` is joined to this origin's `baseUrl`, unless it is an absolute `http://` or `https://` URL — then oat dispatches to that URL as a recorded exchange (method, URL, status, redirects, response headers). That is how a consume page on the app origin is an auth step when `baseUrl` is the API. Unknown host is allowed. `redirect` defaults to `"follow"` with a per-request cookie jar, so a 303 that sets `Set-Cookie` still leaves that cookie visible to `saveAs`. `redirect: "manual"` stops at the first 3xx (`expect: [303]` then bind from that response).
|
|
668
|
-
|
|
669
|
-
**Out-of-band step** (email link, OTP — oat cannot collect this itself):
|
|
670
|
-
|
|
671
|
-
```ts
|
|
672
|
-
{ outOfBand: { address: "{address}", kind: "email-verify", as: "verifyLink" } }
|
|
673
|
-
```
|
|
674
|
-
|
|
675
|
-
The hook returns a string. If that string is a URL, a later RequestStep must GET it — a hook-side `fetch` is not a recorded exchange.
|
|
676
|
-
|
|
677
|
-
Later steps interpolate `{name}` from the flow scope, `saveAs` addresses included (`cookie:rt.{userId}` reads a per-user cookie). `saveAs` addresses:
|
|
678
|
-
|
|
679
|
-
- `$.foo.bar` / `$.orgs.0.id` — JSON body (dot + numeric index only; no JSON Pointer, no filters)
|
|
680
|
-
- `cookie:<name>` — that cookie on `Set-Cookie` for this hop, including followed hops
|
|
681
|
-
- `header:<name>` — a non-set-cookie response header, case-insensitive
|
|
682
|
-
|
|
683
|
-
Missing cookie / header / JSON path fails the auth step closed. Bind `saveAs.refreshToken` from `$.refresh_token` or `cookie:refresh` so `{refreshToken}` interpolates in `auth.refresh`. `saveClaimsFrom.token` is a JSON path into the response body (`$.access_token`) or a scope key already bound by `saveAs` (`credential`). Signature is not verified — oat is reading its own credential. `rootsFromFlow` maps path parameter names to those bound keys.
|
|
684
|
-
|
|
685
|
-
`bind` on a step runs **before** the request. `saveAs` then `saveClaimsFrom` run **after**. `credentialFrom` is read from the last HTTP response unless a step already saved `credential`. `outOfBand.as = "credential"` still wins over `credentialFrom`.
|
|
686
|
-
|
|
687
|
-
If a step's status is not acceptable, auth fails the run (not a finding): `oat: principal "alpha" failed at auth step 2 (POST /v1/…)`.
|
|
688
|
-
|
|
689
|
-
A complete register → mailed GET → cookie session example:
|
|
690
|
-
|
|
691
|
-
```ts
|
|
692
|
-
function signUp(email: string): AuthFlow {
|
|
693
|
-
return {
|
|
694
|
-
credentialFrom: "$.access_token",
|
|
695
|
-
expiresInFrom: "$.access_token_expires_in",
|
|
696
|
-
header: "cookie",
|
|
697
|
-
template: "session={credential}",
|
|
698
|
-
refresh: {
|
|
699
|
-
steps: [
|
|
700
|
-
{
|
|
701
|
-
body: { refresh_token: "{refreshToken}" },
|
|
702
|
-
method: "POST",
|
|
703
|
-
path: "/v1/auth/refresh",
|
|
704
|
-
saveAs: { credential: "$.access_token", refreshToken: "$.refresh_token" },
|
|
705
|
-
},
|
|
706
|
-
],
|
|
707
|
-
},
|
|
708
|
-
steps: [
|
|
709
|
-
{
|
|
710
|
-
bind: { address: email },
|
|
711
|
-
body: { email, password: "…" },
|
|
712
|
-
method: "POST",
|
|
713
|
-
path: "/v1/auth/register/email",
|
|
714
|
-
saveAs: { credential: "$.access_token", refreshToken: "$.refresh_token" },
|
|
715
|
-
},
|
|
716
|
-
{ outOfBand: { address: email, as: "verifyLink", kind: "email-verify" } },
|
|
717
|
-
{
|
|
718
|
-
method: "GET",
|
|
719
|
-
path: "{verifyLink}", // absolute URL; not joined to baseUrl
|
|
720
|
-
expect: [200, 303],
|
|
721
|
-
saveAs: {
|
|
722
|
-
credential: "cookie:session",
|
|
723
|
-
refreshToken: "cookie:refresh",
|
|
724
|
-
},
|
|
725
|
-
saveClaimsFrom: {
|
|
726
|
-
token: "credential",
|
|
727
|
-
bind: { orgId: "orgs.0.oid", projectId: "orgs.0.pids.0" },
|
|
728
|
-
},
|
|
729
|
-
},
|
|
730
|
-
],
|
|
731
|
-
}
|
|
732
|
-
}
|
|
733
|
-
```
|
|
734
|
-
|
|
735
|
-
When the user path **is** POST-token, keep that chain — it is still valid:
|
|
736
|
-
|
|
737
|
-
```ts
|
|
738
|
-
{ outOfBand: { address: email, as: "verifyToken", kind: "email-verify" } },
|
|
739
|
-
{
|
|
740
|
-
body: { token: "{verifyToken}" },
|
|
741
|
-
method: "POST",
|
|
742
|
-
path: "/v1/auth/email/verify",
|
|
743
|
-
saveAs: { credential: "$.access_token", refreshToken: "$.refresh_token" },
|
|
744
|
-
saveClaimsFrom: { token: "$.access_token", bind: { orgId: "orgs.0.oid" } },
|
|
745
|
-
}
|
|
746
|
-
```
|
|
747
|
-
|
|
748
|
-
```ts
|
|
749
|
-
principals: [
|
|
750
|
-
{
|
|
751
|
-
id: "alpha",
|
|
752
|
-
auth: signUp("oat-alpha@example.test"),
|
|
753
|
-
rootsFromFlow: { organization_id: "orgId", project_id: "projectId" },
|
|
754
|
-
},
|
|
755
|
-
]
|
|
756
|
-
```
|
|
757
|
-
|
|
758
|
-
The address used for `teardownPrincipal` is `scope.address` or `scope.email` (set via `bind: { address }` or `bind: { email }`).
|
|
759
|
-
|
|
760
|
-
### Harvested principal
|
|
761
|
-
|
|
762
|
-
When the credential is produced outside oat (a human finishes Google OAuth on a harvest page, a pair lands in KV), do not make oat drive `authorize` / `callback`:
|
|
763
|
-
|
|
764
|
-
```ts
|
|
765
|
-
{
|
|
766
|
-
id: "google-user",
|
|
767
|
-
auth: { fromHook: "oauth-google" },
|
|
768
|
-
}
|
|
769
|
-
```
|
|
770
|
-
|
|
771
|
-
oat polls `hooks.resolvePrincipalAuth("oauth-google")` with the same `outOfBand` backoff as mail. Return `{ credential, refreshToken?, expiresIn? }` or `null` to retry. Refresh re-calls the hook (401 and countdown). oat does not speak OAuth.
|
|
772
|
-
|
|
773
|
-
### Secondary origins
|
|
774
|
-
|
|
775
|
-
One run, one primary `baseUrl`. A CDN (or any second host) keeps its own OpenAPI.
|
|
776
|
-
|
|
777
|
-
```ts
|
|
778
|
-
export default defineConfig({
|
|
779
|
-
spec: "https://api.example.com/openapi.json",
|
|
780
|
-
baseUrl: "https://api.example.com",
|
|
781
|
-
principals: [/* acquire JWT on the API */],
|
|
782
|
-
origins: [{ id: "cdn", baseUrl: "https://cdn.example.com", spec: "https://cdn.example.com/openapi.json" }],
|
|
783
|
-
})
|
|
784
|
-
```
|
|
785
|
-
|
|
786
|
-
After primary auth, oat snapshots the principals, binds those credentials to the other host, and runs the matrix against that document. Auth steps may set `origin: "cdn"` to send one hop to a named origin during acquire.
|
|
787
|
-
|
|
788
|
-
Do not merge CDN routes into the API gateway document.
|
|
789
|
-
|
|
790
|
-
A second `defineConfig` can reuse the first run's snapshot instead:
|
|
791
|
-
|
|
792
|
-
```ts
|
|
793
|
-
import { defineConfig, loadPersistedPrincipals } from "@lovrozagar/oat"
|
|
794
|
-
|
|
795
|
-
export default defineConfig({
|
|
796
|
-
spec: "https://cdn.example.com/openapi.json",
|
|
797
|
-
baseUrl: "https://cdn.example.com",
|
|
798
|
-
principals: loadPersistedPrincipals("./.oat/runs/latest/principals.json"),
|
|
799
|
-
})
|
|
800
|
-
```
|
|
801
|
-
|
|
802
|
-
The CLI writes `principals.json` into each run folder (and `.oat/runs/latest/principals.json` via the `latest` symlink).
|
|
803
|
-
|
|
804
|
-
### Hooks
|
|
805
|
-
|
|
806
|
-
```ts
|
|
807
|
-
hooks: {
|
|
808
|
-
// Return null to retry (attempt is 1-based). oat backs off until a value arrives.
|
|
809
|
-
// `scope` is the flow at this step; `headers` are the live principal credential headers.
|
|
810
|
-
resolveOutOfBand: async ({ address, kind, attempt, scope, headers }) => {
|
|
811
|
-
const link = await readMailCatcher(address, kind)
|
|
812
|
-
return link // URL or token string, or null
|
|
813
|
-
},
|
|
814
|
-
// Remove a principal this run provisioned. `ctx` is the last live credential.
|
|
815
|
-
teardownPrincipal: async (address, { credential, headers }) => {
|
|
816
|
-
await fetch(`https://api.example.com/v1/auth/account`, { method: "DELETE", headers })
|
|
817
|
-
},
|
|
818
|
-
// Return a file to send, `{ fields }` to replace the whole request, or null to fall through.
|
|
819
|
-
resolveUpload: async ({ operationId, field, contentMediaType }) => {
|
|
820
|
-
if (operationId === "extract.once" && field === "file") {
|
|
821
|
-
const bytes = await Deno.readFile("./invoices/known.pdf")
|
|
822
|
-
return { bytes, filename: "known.pdf", mediaType: "application/pdf" }
|
|
823
|
-
}
|
|
824
|
-
return null
|
|
825
|
-
},
|
|
826
|
-
// After globalHeaders, before auth. Return null to add nothing.
|
|
827
|
-
resolveHeaders: async ({ operationId, method, url }) => {
|
|
828
|
-
if (operationId === "auth.register" || operationId === "auth.login") {
|
|
829
|
-
return { "cf-turnstile-response": await harvestTurnstile() }
|
|
830
|
-
}
|
|
831
|
-
return null
|
|
832
|
-
},
|
|
833
|
-
// Replace a generated JSON field. Null keeps the generator.
|
|
834
|
-
resolveInput: async ({ operationId, field }) => {
|
|
835
|
-
if (operationId === "billing.subscribe" && field === "payment_method_id") {
|
|
836
|
-
return process.env.STRIPE_TEST_PM
|
|
837
|
-
}
|
|
838
|
-
return null
|
|
839
|
-
},
|
|
840
|
-
// Harvested OAuth pair. Null retries with the outOfBand backoff.
|
|
841
|
-
resolvePrincipalAuth: async (fromHook) => {
|
|
842
|
-
if (fromHook !== "oauth-google") return null
|
|
843
|
-
const pair = await readHarvestedGoogle()
|
|
844
|
-
return pair === null ? null : { credential: pair.access_token, refreshToken: pair.refresh_token, expiresIn: pair.expires_in }
|
|
845
|
-
},
|
|
846
|
-
// After seed: add or replace any axis of the query catalog. Null keeps the merge.
|
|
847
|
-
resolveQueryCapabilities: async ({ entity, get }) => {
|
|
848
|
-
if (entity !== "row") return null
|
|
849
|
-
const body = await get("table.get")
|
|
850
|
-
return { filterable: /* harvest from body */ [] }
|
|
851
|
-
},
|
|
852
|
-
// Optional extra stop condition while x-wait polls.
|
|
853
|
-
awaitSideEffect: async ({ operationId, record }) => {
|
|
854
|
-
if (operationId !== "webhook.deliver") return null
|
|
855
|
-
return Array.isArray((record as { items?: unknown }).items) && (record as { items: unknown[] }).items.length > 0
|
|
856
|
-
? true
|
|
857
|
-
: null
|
|
858
|
-
},
|
|
859
|
-
}
|
|
860
|
-
```
|
|
861
|
-
|
|
862
|
-
Without `resolveOutOfBand`, an `outOfBand` step cannot complete. oat polls the hook; the hook must not sleep. Returning `""` is treated like `null`. The hook is not a recorded hop: if the human GETs a mailed URL, a later RequestStep must GET that URL so it appears in `exchanges.jsonl` and can `expect` / `saveAs`.
|
|
863
|
-
|
|
864
|
-
Default schedule (0.6.2, unchanged unless `outOfBand` is set): **6** attempts, first sleep **200** ms, doubling, cap **3000** ms. oat sleeps after every miss, including the last, so the worst-case wait is
|
|
865
|
-
|
|
866
|
-
`200 + 400 + 800 + 1600 + 3000 + 3000 = 9000` ms.
|
|
867
|
-
|
|
868
|
-
That is too short for real mail (often 10–60 s) and for a human finishing Google OAuth or a Turnstile harvest. Configure it:
|
|
869
|
-
|
|
870
|
-
```ts
|
|
871
|
-
outOfBand: { attempts: 20, initialMs: 1000, maxMs: 8000 }
|
|
872
|
-
// worst case: 1000 + 2000 + 4000 + 8000×17 = 143000 ms
|
|
873
|
-
```
|
|
874
|
-
|
|
875
|
-
Worst-case wait is `sum_{i=0}^{attempts-1} min(initialMs × 2^i, maxMs)`. `worstCaseWaitMs()` from the package computes it. Existing configs that omit `outOfBand` do not slow down.
|
|
876
|
-
|
|
877
|
-
Without `teardownPrincipal`, provisioned accounts are reported as leftover rather than cascade-deleted. Per-record DELETE still runs for seeded rows when a delete (or `x-cleanup`) exists. The hook is called with the last live credential (`ctx.credential` / `ctx.headers`), not an already-cleared one, so authenticated delete-account is expressible without a tester-key god route. One-argument JavaScript callbacks still run.
|
|
878
|
-
|
|
879
|
-
`resolveHeaders` is called on every dispatch (including the 401 retry). Merge order: `globalHeaders` → hook → per-request headers → principal credential. Use `ctx.operationId` / `ctx.method` / `ctx.url` to attach a one-shot captcha only on captcha ops. oat does not speak Turnstile.
|
|
880
|
-
|
|
881
|
-
`resolveInput` is the JSON twin of `resolveUpload`. Return a value to replace that field (`payment_method_id` on `billing.subscribe`); `null` keeps the generator.
|
|
882
|
-
|
|
883
|
-
`resolveQueryCapabilities` runs once per entity after seed. `get(operationId)` (or `"GET /path"`) uses the seeded parent scope so a follow-up read can harvest dynamic columns. A provided `filterable` / `sortable` / `searchable` / `selectable` list **replaces** that axis; omitted axes stay. JSON configs have no hook.
|
|
884
|
-
|
|
885
|
-
`resolvePrincipalAuth` and `awaitSideEffect` are documented below.
|
|
886
|
-
|
|
887
|
-
Worked `query` overlay for a generic PostgREST-shaped API (not a named product):
|
|
888
|
-
|
|
889
|
-
```ts
|
|
890
|
-
export default defineConfig({
|
|
891
|
-
spec: "https://api.example.com/openapi.json",
|
|
892
|
-
baseUrl: "https://api.example.com",
|
|
893
|
-
principals: [{ id: "alpha", headers: { authorization: `Bearer ${process.env.TOKEN}` } }],
|
|
894
|
-
query: {
|
|
895
|
-
operators: ["eq", "neq", "gt", "gte", "lt", "lte", "in", "nin", "like", "ilike", "is"],
|
|
896
|
-
operatorsByType: {
|
|
897
|
-
string: ["eq", "neq", "like", "ilike", "in", "nin", "is"],
|
|
898
|
-
number: ["eq", "neq", "gt", "gte", "lt", "lte", "in", "nin", "is"],
|
|
899
|
-
date: ["eq", "neq", "gt", "gte", "lt", "lte", "is"],
|
|
900
|
-
boolean: ["eq", "neq", "is"],
|
|
901
|
-
},
|
|
902
|
-
aliases: { ne: "neq" },
|
|
903
|
-
emptyIn: "match-none",
|
|
904
|
-
maxInValues: 100,
|
|
905
|
-
maxFilterConditions: 20,
|
|
906
|
-
searchEmpty: "match-all",
|
|
907
|
-
sort: { nulls: ["first", "last"], maxKeys: 3 },
|
|
908
|
-
select: { nested: false, unknown: "reject" },
|
|
909
|
-
},
|
|
910
|
-
})
|
|
911
|
-
```
|
|
912
|
-
|
|
913
|
-
New surface still has to be listed. oat will not silently enable `in` / `ilike` / `is` / `contains` / search modes / `nullsfirst` on every PostgREST-shaped document.
|
|
914
|
-
|
|
915
|
-
### Uploads
|
|
916
|
-
|
|
917
|
-
Multipart and binary parts are filled in this order:
|
|
918
|
-
|
|
919
|
-
1. `hooks.resolveUpload` — a non-null `UploadFile` wins. `{ fields }` that includes a file part replaces the whole request. `{ fields }` that omits the file part overlays scalars and keeps the each / pool / dummy bytes.
|
|
920
|
-
2. `uploads.each` fixture for this invocation, when that operation is listed.
|
|
921
|
-
3. `uploads.pool` — first file whose extension / sniffed type matches the part's `contentMediaType`. Same `seed` + field + index → same pick. Ops not in `each` stay pick-one.
|
|
922
|
-
4. A tiny dummy with sniffable magic (`%PDF-1.1`, 1×1 PNG, empty zip, …). Unknown types become 16 octet-stream bytes, not a skip.
|
|
923
|
-
|
|
924
|
-
`uploads.each` is a matrix, not a source. `operationId → globs` means that operation is invoked once per matched file (after `eachMax`). Same seed does **not** collapse `each`. A hook that ignores `request.fixture` and always returns the same file will send that file N times.
|
|
925
|
-
|
|
926
|
-
Each invocation is still a citeable hop (method, URL, status, fixture name, content-addressed body). Live `FormData` / `Blob` payloads are **not** kept on `Client.transcript` for the rest of the run — they are replaced with `{ sha256, bytes, mediaType }` (and multipart `{ parts }`) after the hop is journaled, or immediately when the journal is off. Fan-out therefore multiplies requests, not retained fixture bytes.
|
|
927
|
-
|
|
928
|
-
```ts
|
|
929
|
-
export default defineConfig({
|
|
930
|
-
spec: "openapi.json",
|
|
931
|
-
baseUrl: "https://api.example.com",
|
|
932
|
-
principals: [/* … */],
|
|
933
|
-
uploads: {
|
|
934
|
-
pool: ["./fixtures/**/*"],
|
|
935
|
-
each: {
|
|
936
|
-
"extract.once": ["./fixtures/**/*"],
|
|
937
|
-
"extract.stream": ["./fixtures/**/*"],
|
|
938
|
-
},
|
|
939
|
-
eachMax: 24,
|
|
940
|
-
},
|
|
941
|
-
hooks: {
|
|
942
|
-
resolveUpload: async ({ operationId, field, fixture }) => {
|
|
943
|
-
if (operationId === "extract.once" && field === "file") {
|
|
944
|
-
return { fields: { columns: "vendor,date,amount" } }
|
|
945
|
-
}
|
|
946
|
-
return null
|
|
947
|
-
},
|
|
948
|
-
},
|
|
949
|
-
})
|
|
950
|
-
```
|
|
951
|
-
|
|
952
|
-
| case | outcome |
|
|
953
|
-
| ------------------------------- | ----------------------------------------------------------- |
|
|
954
|
-
| `each` omitted | pick-one (today) |
|
|
955
|
-
| glob matches 0 files | warn once, no extra invocations, fall through to pool/dummy |
|
|
956
|
-
| one path in the list is missing | drop that slot, warn once, never `BACKEND_BUG` |
|
|
957
|
-
| `eachMax` < match count | first `eachMax` after sort, warn that it capped |
|
|
958
|
-
| fixture unreadable | drop that slot, warn once, not `BACKEND_BUG` |
|
|
959
|
-
|
|
960
|
-
A missing pool path warns once and falls through. An empty pool match uses a dummy. The run does not fail. JSON configs may set `pool`, `each`, and `eachMax`. `resolveUpload` is TypeScript.
|
|
961
|
-
|
|
962
|
-
`--profile cheap` (or any profile that excludes the op) still drops the whole family. `each` does not punch through a profile.
|
|
963
|
-
|
|
964
|
-
Findings from an `each` invocation carry `fixture: "invoice.pdf"` and render as `extract.once · invoice.pdf`. One 5xx is one finding on that file.
|
|
965
|
-
|
|
966
|
-
oat does not OCR. It sends bytes and checks HTTP / JSON. A 200 with empty extract rows is not automatically a backend defect. A 4xx because the dummy is “not a real invoice” is not automatically a backend defect if the dummy matched the declared `contentMediaType`.
|
|
967
|
-
|
|
968
|
-
Prefer `multipart/form-data` when the operation documents it, even if JSON is also listed. Text form fields still use the string generator (`format` / `pattern` / `maxLength`). A part that is either text or file is sent as a file when the schema is binary.
|
|
969
|
-
|
|
970
|
-
### Environment interpolation
|
|
971
|
-
|
|
972
|
-
After the module loads, every string in the config is scanned for `${NAME}`:
|
|
973
|
-
|
|
974
|
-
```ts
|
|
975
|
-
headers: {
|
|
976
|
-
authorization: "Bearer ${API_TOKEN}"
|
|
977
|
-
}
|
|
978
|
-
```
|
|
979
|
-
|
|
980
|
-
If `API_TOKEN` is unset, oat exits with an error. Do not commit secrets; put them in the environment.
|
|
981
|
-
|
|
982
|
-
Template literals in a `.ts` config (`Bearer ${process.env.API_TOKEN}`) are evaluated by Node _before_ oat sees the object. Either style works; `${NAME}` is what a `.json` config can use.
|
|
983
|
-
|
|
984
|
-
Names match `[A-Z0-9_]+` case-insensitively.
|
|
110
|
+
## CI
|
|
985
111
|
|
|
986
|
-
|
|
112
|
+
This repository runs [`.github/workflows/ci.yml`](./.github/workflows/ci.yml) on every push and pull request to `main`: format, lint, typecheck, then `npm test` (conformance on memory, sqlite and a Postgres service, the built-in combination smoke with a fresh fuzz seed per run, and a precision pass). D1 and the live labs Workers are not in that job — D1 needs Cloudflare credentials.
|
|
987
113
|
|
|
988
|
-
|
|
114
|
+
The package is [`@lovrozagar/oat` on npm](https://www.npmjs.com/package/@lovrozagar/oat). That URL is the repository website. GitHub Releases match npm versions. Pushing a tag `vX.Y.Z` (same as `package.json` `version`) runs [`.github/workflows/release.yml`](./.github/workflows/release.yml): test, `npm publish` via trusted publishing, then a GitHub Release. Configure the trusted publisher once on npm (package Settings → Trusted Publisher → GitHub Actions, workflow `release.yml`, no environment). Do not publish to GitHub Packages — people install from the public npm registry.
|
|
989
115
|
|
|
990
|
-
|
|
116
|
+
Against **your** API:
|
|
991
117
|
|
|
992
|
-
```
|
|
993
|
-
|
|
118
|
+
```yaml
|
|
119
|
+
# GitHub Actions sketch
|
|
120
|
+
- run: npm i -D @lovrozagar/oat
|
|
121
|
+
- run: oat doctor --spec "$SPEC_URL"
|
|
122
|
+
- run: oat run --config oat.config.ts
|
|
123
|
+
env:
|
|
124
|
+
API_TOKEN: ${{ secrets.API_TOKEN }}
|
|
125
|
+
API_TOKEN_B: ${{ secrets.API_TOKEN_B }}
|
|
994
126
|
```
|
|
995
127
|
|
|
996
|
-
|
|
997
|
-
|
|
998
|
-
`oat plan` is this model. Checks never see raw paths; they see entities, actions, and query **roles**.
|
|
999
|
-
|
|
1000
|
-
### Entity name and action
|
|
1001
|
-
|
|
1002
|
-
**Explicit:** `x-entity: { name, action, identity? }` on the operation.
|
|
1003
|
-
|
|
1004
|
-
**Heuristic** (when the tag is absent):
|
|
1005
|
-
|
|
1006
|
-
1. Split the path on `/`. Ignore `{param}` segments.
|
|
1007
|
-
2. The last non-parameter segment is the noun. `v1` / `v2` / `vN` is never a noun.
|
|
1008
|
-
3. Singularise that noun (`stores` → `store`, `batches` → `batch`). Irregulars include `people→person`, `categories→category`, `campuses→campus`, `statuses→status`, `children→child`, `companies→company`, `addresses→address`, `indices→index`, `queries→query`, `properties→property`, `entities→entity`, `inboxes→inbox`. Endings `us|ss|is|os|as|ics|ews|ess|ous|sis` are left alone (`status` stays `status`).
|
|
1009
|
-
4. If a non-parameter segment follows the noun (`/rows/aggregate`, `/tables/{id}/restore`), action is `action`.
|
|
1010
|
-
5. Otherwise: `GET` collection → `list`, `GET` item → `read`, `POST` collection → `create`, `POST` item → `action`, `PUT`/`PATCH` → `update`, `DELETE` → `delete`.
|
|
1011
|
-
|
|
1012
|
-
If no noun can be found, the operation is untracked and `doctor` records an `x-entity` gap.
|
|
1013
|
-
|
|
1014
|
-
`--only` and report entity names are these singular names.
|
|
1015
|
-
|
|
1016
|
-
### Identity
|
|
1017
|
-
|
|
1018
|
-
`x-entity.identity` wins. Else the first of `id`, `uuid`, `slug`, `key`, `name` that is **required** on the item schema, else the first of those that exists, else the trailing path-param suffix (`{table_id}` → `id`). Without an identity the entity is not trackable.
|
|
1019
|
-
|
|
1020
|
-
### Read surface
|
|
1021
|
-
|
|
1022
|
-
The set of `GET` routes through which an instance is visible.
|
|
1023
|
-
|
|
1024
|
-
- Declared: every `"METHOD /path"` in any `x-invalidate` that refers to this entity.
|
|
1025
|
-
- Inferred: sibling collection and item routes on the same path prefix as a mutator.
|
|
1026
|
-
|
|
1027
|
-
`invalidation.declared-route-changes` only runs when a mutator's `x-invalidate` names **another** entity's route.
|
|
1028
|
-
|
|
1029
|
-
### Generated / immutable / soft-delete / tenant
|
|
1030
|
-
|
|
1031
|
-
See [OpenAPI meta tags](#openapi-meta-tags). Fallbacks:
|
|
1032
|
-
|
|
1033
|
-
- `readOnly: true` counts as generated (omitted from create bodies).
|
|
1034
|
-
- No immutability testing without `x-immutable`.
|
|
1035
|
-
- Tenant param: `x-tenant` or a path param matching `org|organization|tenant|workspace|account|project|app` + optional `_id`/`_slug`. Inferred tenants make a cross-tenant read `AMBIGUITY`, not `SECURITY`. With neither a tag nor an inferred name, the check does not apply.
|
|
1036
|
-
|
|
1037
|
-
### Idempotency
|
|
1038
|
-
|
|
1039
|
-
No meta tag. If create declares a header matching `Idempotency-Key` / `Idempotence-Key` / `X-Idempotency-Key` (spaces ignored, case-insensitive), `idempotency.replay-does-not-duplicate` runs.
|
|
1040
|
-
|
|
1041
|
-
## Seeding
|
|
1042
|
-
|
|
1043
|
-
Per entity, oat POSTs the create body built from the request schema (JSON, multipart, or urlencoded).
|
|
1044
|
-
|
|
1045
|
-
Default cohort is **7** records, one of each variant, sliced by `cohortSize`:
|
|
1046
|
-
|
|
1047
|
-
| variant | what it is for |
|
|
1048
|
-
| --------------- | -------------------------------------------------- |
|
|
1049
|
-
| `baseline` | `"Quarterly Report N"` |
|
|
1050
|
-
| `lexical-first` | sorts first (`"aaa first alphabetically"`) |
|
|
1051
|
-
| `lexical-last` | sorts last (`"zzz last alphabetically"`) |
|
|
1052
|
-
| `null-heavy` | `null` on every nullable field |
|
|
1053
|
-
| `unicode` | `"äöüß čćžšđ 日本語 中文 한글 привет مرحبا 🙂"` |
|
|
1054
|
-
| `metacharacter` | `"100% _off_ *everything*"` — LIKE / escape probes |
|
|
1055
|
-
| `boundary` | empty / maxLength / numeric `maximum` |
|
|
1056
|
-
|
|
1057
|
-
Numbers use the ladder `1, 2, 5, 10, 20, 50, 100` so **lexical order ≠ numeric order** (otherwise a TEXT compare looks correct). Enums walk `index % enum.length`. `readOnly` / `x-generated` fields are omitted. Required fields that cannot be generated get a type fallback (`0`, `false`, `[]`, `{}`, `"value"`). Arrays honour `minItems` (never send `[]` when `minItems ≥ 1`). Nested objects stop at depth 4.
|
|
1058
|
-
|
|
1059
|
-
Empty schemas (`{}`, `true`, `additionalProperties: {}`) and cyclic `$ref`s after inlining stop the walk — they become a scalar or `{}`, never another object descent. A `RangeError` during generation is a `COVERAGE_GAP` on that entity naming the operationId and JSON pointer (`fixture generation overflow on table.create (/)`), not `blocked by unknown`.
|
|
1060
|
-
|
|
1061
|
-
String fields honour `format`, `pattern`, and `minLength` together: `email` → `oat-{variant}-{index}@example.test`, `uri` / `url` → `https://example.test/...`, `uuid` → a fixed-shape UUID, `pattern` → a string that matches (or the field is omitted / the entity is a gap). A generated string is padded to `minLength` (repeat the last character) without breaking `pattern`. `"Quarterly Report N"` is only used when the document does not constrain the string. `minLength` greater than `maxLength` omits an optional field and records `missingRequired` on a required one.
|
|
1062
|
-
|
|
1063
|
-
An operation with `x-invite` is not `entity.create`. oat does not POST a generated invitee. The invite check sends `granteeField` = the peer's `inviteAs`. Missing `inviteAs` is a coverage gap naming the tag.
|
|
1064
|
-
|
|
1065
|
-
A create whose `operationId` appears in any principal `auth.steps`, or that declares `x-fresh-principal`, is not seeded. Those rows were provisioned by the auth flow.
|
|
1066
|
-
|
|
1067
|
-
Parent path parameters are created first (depth-first through the owning entity's create). Config / principal `roots` fill parameters oat cannot create.
|
|
1068
|
-
|
|
1069
|
-
A create that returns `>= 300` on the first variant fails the entity (downstream checks `BLOCKED`). Later variants that fail just shorten the cohort — a partial cohort is still used.
|
|
1070
|
-
|
|
1071
|
-
HTTP 429 is retried first — `Retry-After` or exponential backoff, up to five times — on seed, checks, and teardown, whether or not the operation declared `x-rate-limit`. The first 429 is never a seed failure. A leftover 429 after those retries is a gap, not a backend defect.
|
|
1072
|
-
|
|
1073
|
-
A 402 / plan-limit (`payment_required`, `*_plan_limit`) on create is not a backend defect when the same-tenant list already has a row — typically an earlier `x-effects` create that filled a free-plan quota. oat reuses that id so children (a `row` after extract created a `table`) can still seed. It does not invent records. Write-path checks on the adopted entity stand down.
|
|
1074
|
-
|
|
1075
|
-
A documented feature-gate 403 is the exception: if create declares `x-feature-gate` and the body is `vars.type: feature_gate` (and `vars.feature` matches the tag when present), oat records a `COVERAGE_GAP` naming the tag rather than a seed defect. See [`x-feature-gate`](#x-feature-gate).
|
|
1076
|
-
|
|
1077
|
-
When create is tagged `x-unique`, a first-variant seed **409** with a nonempty same-tenant list **adopts** that row (`world.seed` `COVERAGE_GAP` naming `x-unique` — the create could not insert). Unique-conflict checks still run against that row. Write-path oracles that need a body oat submitted stay skipped. 409 with an empty list stays `BLOCKED` (`could not seed`). A 409 without the tag is still today's seed failure. The seed 409 itself is not `create.unique-conflict-rejected` passing — that check is the explicit second POST. See [`x-unique`](#x-unique). Later variants that 409 only shorten the cohort. Generated values for unique body columns differ across variants (a suffix) without weakening `maxLength` / `pattern`; if the document cannot express two distinct values, oat records a gap and skips extra variants.
|
|
1078
|
-
|
|
1079
|
-
`--seed` / `seed` makes the bodies identical across runs. It does not make server-assigned ids identical.
|
|
1080
|
-
|
|
1081
|
-
Teardown DELETEs created rows (or the `x-cleanup` route) newest-first. Failures and missing delete routes are printed as leftovers, not as check findings. `keepFixtures: true` skips this.
|
|
1082
|
-
|
|
1083
|
-
## Query roles and grammars
|
|
1084
|
-
|
|
1085
|
-
Checks do not look for a parameter _named_ `filter`. They resolve **roles** from aliases, then write values in the grammar the document demonstrates.
|
|
1086
|
-
|
|
1087
|
-
| role | aliases (normalised: case, `_` / `-`, `perPage` → `per_page`) |
|
|
1088
|
-
| ----------------- | ----------------------------------------------------------------------------------- |
|
|
1089
|
-
| filter | `filter`, `where`, `query`, `conditions` |
|
|
1090
|
-
| order | `order`, `order_by`, `sort`, `sort_by`, `ordering` |
|
|
1091
|
-
| select | `select`, `fields`, `field`, `include_fields`, `projection`, `only` |
|
|
1092
|
-
| search | `q`, `search`, `query_text`, `term`, `keyword`, `text` |
|
|
1093
|
-
| search mode | `search_mode`, `searchmode`, `search_type`, `mode` (only if a search role exists) |
|
|
1094
|
-
| limit (page size) | `limit`, `per_page`, `page_size`, `pagesize`, `count`, `max_results`, `top`, `size` |
|
|
1095
|
-
| page | `page`, `page_number`, `pagenum`, `p` |
|
|
1096
|
-
| offset | `offset`, `skip`, `start`, `from` |
|
|
1097
|
-
| cursor | `cursor`, `after`, `starting_after`, `next`, `page_token`, `continuation` |
|
|
1098
|
-
|
|
1099
|
-
A bracketed suffix is a _value_ in the name (`fields[articles]`, `filter[status]`), not part of the role. `count` is a page size only when it looks like one (has `maximum` or a default); otherwise it is treated as a total.
|
|
1100
|
-
|
|
1101
|
-
A bounded integer with a default that matches no alias is still taken as page size. A 1-based integer with no maximum is taken as page number.
|
|
1102
|
-
|
|
1103
|
-
**Filter grammars** — how oat **writes** a term:
|
|
1104
|
-
|
|
1105
|
-
| name | example | `and` / `or` |
|
|
1106
|
-
| ----------- | ---------------------------------------------------------------------------------------------------- | ---------------------------------- |
|
|
1107
|
-
| `postgrest` | `status.eq.active`, `name.neq.x`, `id.in.(a,b)`, `name.ilike.FOO`, `note.is.null`, `tags.contains.x` | `and(a.eq.1,b.eq.2)`, `or(...)` |
|
|
1108
|
-
| `colon` | `filter=status=eq:active` (comma-joined terms; no grouping) | not expressible; those checks skip |
|
|
1109
|
-
| `equality` | `?status=active` (one query param per field). Only `eq` is expressible | not expressible |
|
|
1110
|
-
|
|
1111
|
-
Operators the **postgrest** writer can emit: `eq`, `ne`, `neq`, `gt`, `gte`, `lt`, `lte`, `in`, `nin`, `like`, `ilike`, `is`, `contains`. Colon stays on `eq` / `neq` / `gt` / `gte` / `lt` / `lte` / `like`. Equality is `eq` only. Anything a grammar cannot write becomes **did not apply**, not a failed request.
|
|
1112
|
-
|
|
1113
|
-
A check that needs `in`, `ilike`, `is`, `contains`, a search mode, or `nullsfirst` / `nullslast` also needs that capability **declared**. oat does not infer new operators, modes, or nulls tokens onto an API that never listed them.
|
|
1114
|
-
|
|
1115
|
-
**Sort grammars:** `name.asc` (dotted), `-name` (prefixed / JSON:API; ascending is the bare name), `name:asc` (colon), `name asc` (spaced). Dotted (PostgREST-shaped) can also emit `name.asc.nullsfirst` / `name.desc.nullslast` when the capability map allows that token. Colon / prefixed / spaced stay as they are.
|
|
1116
|
-
|
|
1117
|
-
**Select grammars:** `id,name` (csv) or `fields[table]=id,name` (bracketed). If the parameter is already named `fields[articles]`, that name is used verbatim.
|
|
1118
|
-
|
|
1119
|
-
### How a grammar is inferred
|
|
1120
|
-
|
|
1121
|
-
`x-query.grammar` wins when it is `postgrest` | `colon` | `equality`.
|
|
1122
|
-
|
|
1123
|
-
Otherwise oat concatenates the filter/order/select parameter's `example`, `examples`, and `description`:
|
|
1124
|
-
|
|
1125
|
-
- Filter: `/postgrest/i` or `field.op.value` / `status.eq.active` → `postgrest`; `status=eq:` → `colon`; else `equality`. A free-text `filter` string that still looks like equality produces an `x-query` gap telling you to declare the grammar.
|
|
1126
|
-
- Sort: `name.asc` → dotted; `name:desc` → colon; `name desc` → spaced; leading `-field` → prefixed; else dotted.
|
|
1127
|
-
- Select: parameter name or description contains `fields[…]` → bracketed; else csv.
|
|
1128
|
-
|
|
1129
|
-
Without `x-query`, if a filter/order/select/search role resolves, oat assumes **every scalar** is filterable/sortable/selectable (`string` / `number` / `integer` / `boolean`, including nullable unions). Searchable-without-tag is further narrowed to names matching `name|title|slug|label|description|email`. `doctor` warns. Pagination-only lists stay uncovered.
|
|
1130
|
-
|
|
1131
|
-
That heuristic runs **only** for an axis that was not tagged and not set in config. An explicit empty claim — `filterable: []`, `searchable: null`, `selectable: []`, `sortable: []` — is a claim of none. oat will not infer scalars over it.
|
|
1132
|
-
|
|
1133
|
-
Searchable / filterable / sortable / selectable from the tag are used as given. `maxLimit` is also taken from a page-size parameter's `schema.maximum` when the tag omits it.
|
|
1134
|
-
|
|
1135
|
-
### Declared or skip
|
|
1136
|
-
|
|
1137
|
-
Every list check consults one **effective capability map** per entity. Precedence, unchanged in spirit:
|
|
1138
|
-
|
|
1139
|
-
| order | source | what it contributes |
|
|
1140
|
-
| ----- | ------------------------------- | ---------------------------------------------------------------------------------------------------- |
|
|
1141
|
-
| 1 | `x-query` on the list operation | fields, structured rows, operators, modes, caps, harvest |
|
|
1142
|
-
| 2 | `config.entities[name].query` | union of fields; overlay of ops / modes / caps |
|
|
1143
|
-
| 3 | `config.query` | global defaults for the same keys |
|
|
1144
|
-
| 4 | heuristic | today's scalars / name-regex searchable — **only if that axis was not tagged and not set in config** |
|
|
1145
|
-
| 5 | skip | the check does not apply |
|
|
1146
|
-
|
|
1147
|
-
`hooks.resolveQueryCapabilities` runs after 1–3 (and after any `*From` harvest) and may add or replace any axis. Return `null` to keep the merge. Called once per entity after seed, with the seeded parent scope so a follow-up GET can run.
|
|
1148
|
-
|
|
1149
|
-
Missing capability → the check does not apply / unresolved. Never `BACKEND_BUG` for an undeclared op, mode, or nulls option.
|
|
1150
|
-
|
|
1151
|
-
## Pagination and envelopes
|
|
1152
|
-
|
|
1153
|
-
Three page models, all first-class:
|
|
1154
|
-
|
|
1155
|
-
- **Page number** (`page` + `limit` roles).
|
|
1156
|
-
- **Offset** (`offset` + `limit`). Checks that say "page 3" translate to `offset = (page - 1) * size` (size defaults to 20 only for that translation).
|
|
1157
|
-
- **Cursor** (`cursor` role + envelope `nextCursor` or a `Link: rel=next` header).
|
|
1158
|
-
|
|
1159
|
-
`hasMore` is taken from the body (`hasMore`, `has_more`, `hasNextPage`, `more`) **or** from a documented `Link` response header. Under Link pagination, **absence** of `rel="next"` means no more pages.
|
|
1160
|
-
|
|
1161
|
-
Collection shape is derived from the success JSON schema, not from hardcoded wrapper names:
|
|
1162
|
-
|
|
1163
|
-
- Response `type: array` → the body is the list (`key: null`).
|
|
1164
|
-
- Otherwise the array property whose items are objects, skipping sidecar names `error(s)`, `warning(s)`, `message(s)`, `meta`, `links`. Resource-named envelopes (`{ tables: [...] }`) work.
|
|
1165
|
-
- Sibling keys become envelope fields:
|
|
1166
|
-
|
|
1167
|
-
| role | accepted property names |
|
|
1168
|
-
| ---------- | ----------------------------------------------------------- |
|
|
1169
|
-
| total | `count`, `total`, `totalCount`, `total_count`, `totalItems` |
|
|
1170
|
-
| hasMore | `hasMore`, `has_more`, `hasNextPage`, `more` |
|
|
1171
|
-
| nextCursor | `nextCursor`, `next_cursor`, `cursor`, `next`, `endCursor` |
|
|
1172
|
-
| page | `page`, `pageNumber`, `page_number`, `offset` |
|
|
1173
|
-
| limit | `limit`, `perPage`, `per_page`, `pageSize`, `page_size` |
|
|
1174
|
-
|
|
1175
|
-
Success schema is the first JSON media type on responses `200`, `201`, `202`, `2XX`, or `default`. Request schema is taken from `requestBody` preferring `multipart/form-data`, then `application/x-www-form-urlencoded`, then JSON. A success response that lists `text/event-stream` is a stream: oat consumes it to the end and does not treat the raw body as a JSON schema defect. Media type is the stream tag — there is no `x-stream`.
|
|
1176
|
-
|
|
1177
|
-
## OpenAPI meta tags
|
|
1178
|
-
|
|
1179
|
-
Vendor-neutral `x-*` extensions. Every one is optional. Precedence: **explicit tag → heuristic → skip with a coverage gap**.
|
|
1180
|
-
|
|
1181
|
-
A complete document with every tag in place is shipped as `labs/annotated-openapi.yaml` (also in the npm package). `oat conformance` asserts the derived model matches that file.
|
|
1182
|
-
|
|
1183
|
-
```bash
|
|
1184
|
-
oat plan --spec node_modules/@lovrozagar/oat/labs/annotated-openapi.yaml
|
|
1185
|
-
oat doctor --spec node_modules/@lovrozagar/oat/labs/annotated-openapi.yaml
|
|
1186
|
-
```
|
|
1187
|
-
|
|
1188
|
-
What each tag **unlocks** (otherwise the check cannot run):
|
|
1189
|
-
|
|
1190
|
-
| tag | checks unlocked |
|
|
1191
|
-
| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
1192
|
-
| `x-async` | `async.reaches-terminal-state`, `async.receipt-identifies-the-job` |
|
|
1193
|
-
| `x-effects` | `effects.declared-effect-occurs` |
|
|
1194
|
-
| `x-immutable` | `patch.immutable-field-rejected` |
|
|
1195
|
-
| `x-invalidate` | `invalidation.declared-route-changes` (when the list names another entity) |
|
|
1196
|
-
| `x-query` | `spec.declared-filterable-is-filterable`, `spec.declared-sortable-is-sortable`, `spec.declared-selectable-is-selectable`, plus the declared-or-skip catalog checks (`in` / `ilike` / `is` / nulls / caps / …) |
|
|
1197
|
-
| `x-soft-delete` | `softdelete.absent-from-default-list` |
|
|
1198
|
-
| `x-invite` | `auth.invite-grants-then-revokes` |
|
|
1199
|
-
| `x-wait` | `effects.side-effect-arrives` |
|
|
1200
|
-
| `x-unique` | `create.unique-conflict-rejected`, `update.unique-conflict-rejected` |
|
|
1201
|
-
|
|
1202
|
-
What each tag **sharpens** (the check already runs, but the verdict changes):
|
|
1203
|
-
|
|
1204
|
-
| tag | without it |
|
|
1205
|
-
| ---------- | --------------------------------------------------------------------------------------------------------------------------- |
|
|
1206
|
-
| `x-query` | every scalar is probed, including columns you never indexed — expect findings you will dismiss |
|
|
1207
|
-
| `x-tenant` | inferred tenant: a cross-tenant read is `AMBIGUITY`, not `SECURITY`. No tenant tagged or inferred: the check does not apply |
|
|
1208
|
-
|
|
1209
|
-
### `x-invalidate`
|
|
1210
|
-
|
|
1211
|
-
```yaml
|
|
1212
|
-
x-invalidate:
|
|
1213
|
-
- GET /v1/projects/{project_id}/tables
|
|
1214
|
-
- GET /v1/projects/{project_id}/tables/{table_id}
|
|
1215
|
-
```
|
|
1216
|
-
|
|
1217
|
-
`string[]` of `"METHOD /path"`. Colon or brace path params; oat normalises to brace. Method is uppercased.
|
|
1218
|
-
|
|
1219
|
-
This is the entity graph. Inverted, it is each entity's read surface. Highest-value tag.
|
|
1220
|
-
|
|
1221
|
-
**Fallback:** pair a mutator with sibling collection/item routes on the same path prefix. Misses cross-entity effects.
|
|
1222
|
-
|
|
1223
|
-
**Unlocks:** `invalidation.declared-route-changes` (when the list names _another_ entity's route).
|
|
1224
|
-
|
|
1225
|
-
### `x-entity`
|
|
1226
|
-
|
|
1227
|
-
```yaml
|
|
1228
|
-
x-entity:
|
|
1229
|
-
name: table
|
|
1230
|
-
action: create # create | list | read | update | delete | action
|
|
1231
|
-
identity: id
|
|
1232
|
-
```
|
|
1233
|
-
|
|
1234
|
-
Overrides path-segment inference. `identity` is required when the item schema has no `id` (or `uuid` / `slug` / `key` / `name`).
|
|
1235
|
-
|
|
1236
|
-
**Fallback:** deepest plural segment + HTTP verb. See [How the model is derived](#how-the-model-is-derived).
|
|
1237
|
-
|
|
1238
|
-
### `x-invite`
|
|
1239
|
-
|
|
1240
|
-
```yaml
|
|
1241
|
-
x-invite:
|
|
1242
|
-
invite: table.invite
|
|
1243
|
-
accept: invite.accept
|
|
1244
|
-
revoke: table.revoke
|
|
1245
|
-
granteeField: key
|
|
1246
|
-
tokenPointer: $.token
|
|
1247
|
-
grantPointer: $.grant_id
|
|
1248
|
-
tokenFrom: response # or outOfBand
|
|
1249
|
-
tokenKind: org-invite # only when tokenFrom is outOfBand; default `${entity}-invite`
|
|
1250
|
-
acceptFrom: token # default. `link` = GET the OOB URL instead of POSTing accept JSON
|
|
1251
|
-
```
|
|
1252
|
-
|
|
1253
|
-
Put this on the invite operation. Config must give the invitee `inviteAs`. Defaults if omitted: `granteeField: key`, `tokenPointer: $.token`, `grantPointer: $.grant_id`, `tokenFrom: response`, `acceptFrom: token`. All three of `invite` / `accept` / `revoke` (operationIds) are required or the tag is ignored. `oat doctor` / `oat plan` print the accept mode.
|
|
1254
|
-
|
|
1255
|
-
`tokenFrom: response` (default) reads the accept token from the invite HTTP body at `tokenPointer`. Keep this for backends that still put the token in JSON.
|
|
1256
|
-
|
|
1257
|
-
`tokenFrom: outOfBand` ignores the response token and calls `resolveOutOfBand({ address: inviteAs, kind, scope, headers })` after the invite POST. `kind` is `tokenKind` or `${entity}-invite` (`org` → `org-invite`, `project` → `project-invite`). Use this when the live profile must accept only the mailed token.
|
|
1258
|
-
|
|
1259
|
-
`acceptFrom: token` (default) stuffs that string into the documented accept JSON / path as today. `acceptFrom: link` requires the string to be an absolute `http(s)` URL: oat GETs it (recorded, cookie-jar follow). The `accept` operationId is unused for that hop; `revoke` still uses the documented revoke operation. 2xx/3xx that leaves the grant readable continues the timeline; 4xx, 404, or an unreadable grant is a finding. A config that still POSTs `{ token }` with `tokenFrom: outOfBand` and no `acceptFrom` does not change meaning.
|
|
1260
|
-
|
|
1261
|
-
An invite operation is **not** the entity's fixture create, even when it is `POST` on the collection. oat will not seed it with a generated email. The invite check (and only that check) creates the grant, using `inviteAs` as `granteeField`. The check still runs when there is no non-invite create, as long as an item or list route exists.
|
|
1262
|
-
|
|
1263
|
-
Timeline asserted: cannot read → invite → still cannot → accept → can → revoke → cannot.
|
|
1264
|
-
|
|
1265
|
-
Accept and revoke send the documented JSON request body when the operation declares one, filled from the invite token / grant id (and other known scope values). A path-only accept (`POST /invites/{token}` with no body) stays path-only — oat does not invent a body.
|
|
1266
|
-
|
|
1267
|
-
**Fallback:** the check does not run.
|
|
1268
|
-
|
|
1269
|
-
### `x-query`
|
|
1270
|
-
|
|
1271
|
-
String arrays still work. Structured rows are optional. Either, not both required.
|
|
1272
|
-
|
|
1273
|
-
```yaml
|
|
1274
|
-
x-query:
|
|
1275
|
-
grammar: postgrest # postgrest | colon | equality
|
|
1276
|
-
filterable: [id, name, created_at]
|
|
1277
|
-
sortable: [name, created_at]
|
|
1278
|
-
searchable: [name, slug]
|
|
1279
|
-
selectable: [id, name, created_at]
|
|
1280
|
-
maxLimit: 100
|
|
1281
|
-
defaultOrder: created_at.desc
|
|
1282
|
-
stableTiebreak: id
|
|
1283
|
-
|
|
1284
|
-
# structured (optional) — unlocks per-field ops / types / nulls
|
|
1285
|
-
filterable:
|
|
1286
|
-
- { field: id, type: string, ops: [eq, neq, in, nin] }
|
|
1287
|
-
- { field: name, type: string, ops: [eq, neq, like, ilike, in] }
|
|
1288
|
-
- { field: created_at, type: date, ops: [eq, gt, gte, lt, lte, is] }
|
|
1289
|
-
- { field: tags, type: array, ops: [contains, eq, is] }
|
|
1290
|
-
sortable:
|
|
1291
|
-
- { field: name, type: string }
|
|
1292
|
-
- { field: created_at, type: date, nulls: [first, last] }
|
|
1293
|
-
|
|
1294
|
-
operators: [eq, ne, neq, gt, gte, lt, lte, in, nin, like, ilike, is, contains]
|
|
1295
|
-
operatorsByType:
|
|
1296
|
-
string: [eq, ne, neq, like, ilike, in, nin, is]
|
|
1297
|
-
number: [eq, ne, neq, gt, gte, lt, lte, in, nin, is]
|
|
1298
|
-
date: [eq, ne, neq, gt, gte, lt, lte, is]
|
|
1299
|
-
enum: [eq, ne, neq, in, nin, is]
|
|
1300
|
-
boolean: [eq, ne, neq, is]
|
|
1301
|
-
array: [contains, eq, is]
|
|
1302
|
-
aliases: { ne: neq }
|
|
1303
|
-
identityFilter: _id # filter field when it differs from the JSON identity
|
|
1304
|
-
emptyIn: match-none # reject | match-none; undeclared → empty-in check skips
|
|
1305
|
-
maxInValues: 100
|
|
1306
|
-
maxFilterConditions: 20
|
|
1307
|
-
searchModes: [keyword] # free-form; undeclared → mode checks skip
|
|
1308
|
-
searchEmpty: match-all # ignore | match-all | reject
|
|
1309
|
-
sortNulls: [first, last]
|
|
1310
|
-
maxSortKeys: 3
|
|
1311
|
-
selectNested: true # `rel(col)` grammar; undeclared → nested check skips
|
|
1312
|
-
selectUnknown: reject # reject | ignore; undeclared → unknown-select check skips
|
|
1313
|
-
|
|
1314
|
-
# harvest any axis from another GET — no baked path
|
|
1315
|
-
filterableFrom: { operationId: table.get, path: $.columns[*].name, typePath: $.columns[*].type, typeMap: { text: string, int: number } }
|
|
1316
|
-
sortableFrom: { operationId: table.get, path: $.columns[*].name }
|
|
1317
|
-
searchableFrom: { operationId: table.get, path: $.columns[?(@.searchable==true)].name }
|
|
1318
|
-
selectableFrom: { operationId: table.get, path: $.columns[*].name }
|
|
1319
|
-
```
|
|
1320
|
-
|
|
1321
|
-
`defaultOrder` / `stableTiebreak` / `maxLimit` stay. Harvest keys are generic JSON paths (`$`, dots, `[n]`, `[*]`, `[?(@.key==value)]`). Absent → no harvest.
|
|
1322
|
-
|
|
1323
|
-
String arrays + `operatorsByType` is enough for most APIs. Per-field `ops` / `nulls` close the list that field actually accepts.
|
|
1324
|
-
|
|
1325
|
-
**Fallback:** if those roles resolve and the axis was not tagged and not set in config, treat every scalar as capable and warn. Empty tagged `filterable: []` / `searchable: null` / `selectable: []` is an explicit claim of none.
|
|
1326
|
-
|
|
1327
|
-
**Unlocks:** `spec.declared-filterable-is-filterable`, `spec.declared-sortable-is-sortable`, `spec.declared-selectable-is-selectable`. Structured extras unlock the matching catalog checks (`in` = union of `eq`, `ilike` vs `like`, `is.null`, illegal op, empty `in`, caps, nulls, search empty/modes, unknown select, nested `rel(col)`). Sharpens every other query check (without the tag they probe columns you may not have indexed).
|
|
1328
|
-
|
|
1329
|
-
`oat doctor` prints, per entity, the effective map (all four axes, ops, modes, caps, harvest/hook) and which new checks will apply. Unknown `config.entities` names are ignored and warned.
|
|
1330
|
-
|
|
1331
|
-
### `x-async`
|
|
1332
|
-
|
|
1333
|
-
```yaml
|
|
1334
|
-
x-async:
|
|
1335
|
-
poll: "GET /v1/projects/{project_id}/batches/{batch_id}"
|
|
1336
|
-
idFrom: batch_id # or $.id
|
|
1337
|
-
until: "status.in.complete,partial,failed"
|
|
1338
|
-
successWhen: "status.eq.complete"
|
|
1339
|
-
timeoutMs: 120000
|
|
1340
|
-
pollIntervalMs: 2000
|
|
1341
|
-
```
|
|
1342
|
-
|
|
1343
|
-
`x-async` means **POST is a receipt, then poll**. It is not how you mark a stream. A stream is a success response that lists `text/event-stream`.
|
|
1344
|
-
|
|
1345
|
-
When the start response is JSON, oat polls `poll` until `until` matches, then treats that payload as the result. `poll` may be an operationId or `"GET /path/{id}"`. Defaults: `timeoutMs: 120000`, `pollIntervalMs: 2000`.
|
|
1346
|
-
|
|
1347
|
-
When the start response is `text/event-stream`, oat parses `event:` / `data:` frames. `data` is JSON when it starts with `{` / `[`. `idFrom` is resolved against **each event's JSON `data`**, first hit wins — `$.batch_id` on `event: batch` data `{ "batch_id": "…" }` works. A frame named `complete` / `error`, or one whose `data` matches `until`, is the terminal record; `successWhen` is applied to it and oat does **not** GET the poll route. Poll only if the stream ended without a terminal frame **and** `idFrom` resolved.
|
|
1348
|
-
|
|
1349
|
-
A spec may leave both `x-async` and `text/event-stream` on the same operation. The stream is still the result; `x-async` only supplies `idFrom` / `until` / the poll fallback.
|
|
1350
|
-
|
|
1351
|
-
`until` / `successWhen` use the same `field.op.value` predicates as filters (`eq`, `in`, …).
|
|
1352
|
-
|
|
1353
|
-
**Fallback:** treated as synchronous; async checks are `COVERAGE_GAP`. A 2xx stream without `x-async` is consumed and recorded; async checks do not run.
|
|
1354
|
-
|
|
1355
|
-
### `x-effects`
|
|
1356
|
-
|
|
1357
|
-
```yaml
|
|
1358
|
-
x-effects:
|
|
1359
|
-
- { entity: table, op: create }
|
|
1360
|
-
- { entity: row, op: append, min: 1 }
|
|
1361
|
-
```
|
|
1362
|
-
|
|
1363
|
-
`op`: `create` | `append` | `update` | `delete` | `replace`. Each item is `{ entity, op, count?, min? }`.
|
|
1364
|
-
|
|
1365
|
-
- `count` is an **exact** cardinality delta on that entity's list.
|
|
1366
|
-
- `min` is **at-least** (`delta >= min` and `added.length >= min`). Use this when the child count is data-dependent (an extract that appends 1 / 2 / 5 rows).
|
|
1367
|
-
- Omit both → `count: 1` (invite / create).
|
|
1368
|
-
- Set both on one item → rejected at load / `oat doctor` (`x-effects` gap). The item is not checked.
|
|
1369
|
-
|
|
1370
|
-
After a write that declares `create` on A, oat binds the new A id from the write response (`table_id`, or the entity identity) or from A's list delta — the same adopt idea as a 402 plan-limit reuse. Later items in **that same** `x-effects` array whose child list is under A (`GET .../tables/{table_id}/rows`) fill the path with that id. `x-wait` after the same write uses the same bound id. No tester hook.
|
|
1371
|
-
|
|
1372
|
-
An extract-shaped write (create table + append ≥1 row) fails when the new table's row list is empty, and passes when it has 5.
|
|
1373
|
-
|
|
1374
|
-
**Fallback:** derived from `x-entity.action` for this entity only.
|
|
1375
|
-
|
|
1376
|
-
### `x-wait`
|
|
1377
|
-
|
|
1378
|
-
```yaml
|
|
1379
|
-
x-wait:
|
|
1380
|
-
operationId: inbox.list
|
|
1381
|
-
until: $.items.0
|
|
1382
|
-
timeoutMs: 30000
|
|
1383
|
-
pollIntervalMs: 1000
|
|
1384
|
-
```
|
|
1385
|
-
|
|
1386
|
-
Put this on the **write**. After that write succeeds, oat polls `operationId` until `until` (JSON path `$.items.0` or JSON pointer `/items/0`) is non-empty, or `hooks.awaitSideEffect` returns `true`. Default `timeoutMs` is **30s**. Timeout is a **finding** (`effects.side-effect-arrives`), not a coverage gap.
|
|
1387
|
-
|
|
1388
|
-
Use this for queue consumers and webhook inboxes (1–30 s), not for the same request. `x-effects` still asserts cardinality; `x-wait` asserts “this other GET eventually has a body”. If the poll path needs a parent id the write created, oat binds it the same way `x-effects` does.
|
|
1389
|
-
|
|
1390
|
-
**Fallback:** the check does not run.
|
|
1391
|
-
|
|
1392
|
-
### `x-soft-delete`
|
|
1393
|
-
|
|
1394
|
-
```yaml
|
|
1395
|
-
x-soft-delete: deleted_at
|
|
1396
|
-
```
|
|
1397
|
-
|
|
1398
|
-
On any operation of the entity (commonly DELETE). Tombstone, not remove. Without it, a correct soft-delete looks like a bug (the row is still GET-able).
|
|
1399
|
-
|
|
1400
|
-
### `x-immutable` / `x-generated`
|
|
1401
|
-
|
|
1402
|
-
```yaml
|
|
1403
|
-
x-immutable: [id, project_id, created_at]
|
|
1404
|
-
x-generated: [id, created_at, updated_at]
|
|
1405
|
-
```
|
|
1406
|
-
|
|
1407
|
-
Generated fields are omitted from create bodies and expected in responses. Immutable fields must reject or ignore PATCH.
|
|
1408
|
-
|
|
1409
|
-
**Fallback:** `readOnly: true` counts as generated. No immutability testing without the tag.
|
|
1410
|
-
|
|
1411
|
-
### `x-tenant`
|
|
1412
|
-
|
|
1413
|
-
```yaml
|
|
1414
|
-
x-tenant: project_id
|
|
1415
|
-
```
|
|
1416
|
-
|
|
1417
|
-
Path parameter that scopes the operation.
|
|
1418
|
-
|
|
1419
|
-
**Fallback:** regex over `{organization_id}`, `{project_id}`, `{tenant_id}`, `{workspace_id}`, `{app_slug}`, `{org_id}`, `{account_id}`, … (`org|organization|tenant|workspace|account|project|app` + optional `_id`/`_slug`). Without the tag, a matching path parameter still infers a tenant and a cross-tenant read is `AMBIGUITY`, not `SECURITY`. Omitting `x-tenant` and not naming a tenant path parameter means the check does not apply.
|
|
1420
|
-
|
|
1421
|
-
### `x-root`
|
|
1422
|
-
|
|
1423
|
-
```yaml
|
|
1424
|
-
# on a path parameter, not an operation
|
|
1425
|
-
x-root: true
|
|
1426
|
-
```
|
|
1427
|
-
|
|
1428
|
-
This resource has no create endpoint; supply it in config `roots` / principal `roots`.
|
|
1429
|
-
|
|
1430
|
-
**Fallback:** inferred when a path param has no create op; everything beneath is `UNSEEDABLE`.
|
|
1431
|
-
|
|
1432
|
-
### `x-cleanup`
|
|
1433
|
-
|
|
1434
|
-
```yaml
|
|
1435
|
-
x-cleanup: "DELETE /v1/projects/{project_id}/tables/{table_id}"
|
|
1436
|
-
```
|
|
1437
|
-
|
|
1438
|
-
Teardown route when the entity has no discoverable delete. Without it, leftover records are reported at end of run.
|
|
1439
|
-
|
|
1440
|
-
### `x-cost` / `x-destructive` / `x-idempotent` / `x-fresh-principal`
|
|
1441
|
-
|
|
1442
|
-
```yaml
|
|
1443
|
-
x-cost: high # low | medium | high
|
|
1444
|
-
x-destructive: true
|
|
1445
|
-
x-idempotent: true
|
|
1446
|
-
x-fresh-principal: true
|
|
1447
|
-
```
|
|
1448
|
-
|
|
1449
|
-
`x-cost` and `x-destructive` are consulted by `--profile` (below). `x-idempotent` and `x-fresh-principal` are parsed onto the operation model for the `plan`/`doctor` output but not yet consulted anywhere else. Replay safety is tested from a documented `Idempotency-Key` header (`idempotency.replay-does-not-duplicate`), not from `x-idempotent`.
|
|
1450
|
-
|
|
1451
|
-
### `x-feature-gate`
|
|
1452
|
-
|
|
1453
|
-
```yaml
|
|
1454
|
-
x-feature-gate: webhooks # or custom_domain, audit_log, …
|
|
1455
|
-
```
|
|
1456
|
-
|
|
1457
|
-
Plan key this operation is sold behind. Honey/apps emit it as route meta → OpenAPI. `doctor` / `plan` show it. A non-string or missing tag is ignored (`null`).
|
|
1458
|
-
|
|
1459
|
-
This is not cost. `--profile cheap` skips expensive operations; a free principal hitting `webhook.create` is the real product. Skipping the op in a profile would hide the gate.
|
|
1460
|
-
|
|
1461
|
-
A 403 is a **documented feature-gate denial** only when all of:
|
|
1462
|
-
|
|
1463
|
-
1. The operation has `x-feature-gate: <string>`.
|
|
1464
|
-
2. HTTP status is 403.
|
|
1465
|
-
3. JSON body `vars.type === "feature_gate"`.
|
|
1466
|
-
4. `vars.feature` equals the tag **when `vars.feature` is a string**. Absent `vars.feature` is enough together with (3). A string that disagrees with the tag is backend/tag drift — still a seed failure.
|
|
1467
|
-
|
|
1468
|
-
`error_key` / `feature_name` / `required_plan` are i18n/product and are not required. A typical body:
|
|
1469
|
-
|
|
1470
|
-
```json
|
|
1471
|
-
{
|
|
1472
|
-
"success": false,
|
|
1473
|
-
"status": 403,
|
|
1474
|
-
"status_key": "forbidden",
|
|
1475
|
-
"error_key": "forbidden",
|
|
1476
|
-
"vars": {
|
|
1477
|
-
"type": "feature_gate",
|
|
1478
|
-
"feature": "webhooks",
|
|
1479
|
-
"feature_name": "Webhooks",
|
|
1480
|
-
"current_plan": "free",
|
|
1481
|
-
"required_plan": "pro"
|
|
1482
|
-
}
|
|
1483
|
-
}
|
|
1484
|
-
```
|
|
1485
|
-
|
|
1486
|
-
A gated create is a **coverage gap, not a fail**. oat degrades the entity the same way `--profile` excluding `create` does: `COVERAGE_GAP` on `world.seed` naming `x-feature-gate: <key>` (and the plan vars if present); remaining checks that needed a seeded row `did not apply` / `BLOCKED` because of that gap, not a page of "returned 403" copies. If the list route already has rows, read-only checks still run.
|
|
1487
|
-
|
|
1488
|
-
A later write (`create` after seed, `update`, action) that returns a documented gate 403 is the same: the check that needed a 2xx is `COVERAGE_GAP` / did not apply, citing the tag — not `SECURITY`, not a schema defect. `validation.*` and `schema.error-response-matches-document` still apply: the 403 body must match the documented 403 schema. A gate 403 with an undeclared shape is still drift.
|
|
1489
|
-
|
|
1490
|
-
Every 403 is not a feature gate. No tag, or a tag that disagrees with `vars.feature`, stays a `SeedError`.
|
|
1491
|
-
|
|
1492
|
-
**Fallback:** none. Without the tag, a 403 is a failed create.
|
|
1493
|
-
|
|
1494
|
-
### `x-unique`
|
|
1495
|
-
|
|
1496
|
-
```yaml
|
|
1497
|
-
x-unique:
|
|
1498
|
-
- [email]
|
|
1499
|
-
- [workspace_id, slug]
|
|
1500
|
-
# or: { columns: [email] }
|
|
1501
|
-
# or a single string[] meaning one set: [email]
|
|
1502
|
-
```
|
|
1503
|
-
|
|
1504
|
-
Column sets that must stay unique. Honey/apps emit this as route meta → OpenAPI. `doctor` / `plan` show the effective sets per entity the way they show `x-feature-gate`.
|
|
1505
|
-
|
|
1506
|
-
Accepted shapes: a list of column sets, `{ columns: [...] }` objects, and a single `string[]` meaning one set. Each set is a non-empty list of column names; empty sets are dropped. Malformed / non-array → treat as absent (`null`), unique checks do not run, `doctor` records an `x-unique` gap. `[]` after filtering → explicit none, checks do not run. No tag → unique checks do not run, `doctor` says so.
|
|
1507
|
-
|
|
1508
|
-
Do not infer uniqueness from 409s, unique-looking names, or JSON Schema `uniqueItems`. Index names and product `error_key` / `vars.type` are not required; HTTP **409** is the unique-conflict class.
|
|
1509
|
-
|
|
1510
|
-
`create.unique-conflict-rejected` — after a known row (seeded or adopted), a second create with the same unique-set values is 409, list cardinality does not grow, and a documented 409 body still matches the 409 schema. **2xx is `BACKEND_BUG`** (evidence: both exchanges, unique columns, list before/after when list resolved; tear down a returned id). Probe each probeable set separately (one 2xx fails the check; do not collide every set in one request). Probeable = at least one column on the create JSON or form body; path/tenant/`x-generated` columns may fill set identity from scope, but a set with no body columns is skipped, not failed. Do not reuse `Idempotency-Key` / `Idempotence-Key` / `X-Idempotency-Key` (omit, or send a fresh key if the document requires it). 402 / documented feature-gate 403 on the probe is `COVERAGE_GAP`, not a unique pass. Other 4xx / 5xx is not a unique pass. Verdict is never `SECURITY`. Leftover/extra rows do not skip the check.
|
|
1511
|
-
|
|
1512
|
-
`update.unique-conflict-rejected` — PATCH a **different** row onto another row's unique-set values is 409 (same scoring). PATCH of a row's own unique values unchanged is not this check. Skip update sets whose columns are all `x-immutable`.
|
|
1513
|
-
|
|
1514
|
-
**Fallback:** none. Without the tag, unique checks do not run, and a seed 409 is a failed create.
|
|
1515
|
-
|
|
1516
|
-
### `--profile` — cost gating
|
|
1517
|
-
|
|
1518
|
-
```bash
|
|
1519
|
-
oat run --config oat.config.ts --profile cheap
|
|
1520
|
-
```
|
|
1521
|
-
|
|
1522
|
-
A profile restricts which operations a run is allowed to touch, filtering on `x-cost` and `x-destructive`. Two exist without being declared anywhere: `full` (no gating — the default, today's behaviour if you never mention a profile) and `cheap` (`{ maxCost: "low" }`). Reach for a profile when some operations are expensive to call every run — an extraction endpoint billed per request, a bulk job, anything you don't want fired on every `oat run`.
|
|
1523
|
-
|
|
1524
|
-
Anything more specific than a cost band is a named entry in config:
|
|
1525
|
-
|
|
1526
|
-
```ts
|
|
1527
|
-
export default defineConfig({
|
|
1528
|
-
// ...
|
|
1529
|
-
profiles: {
|
|
1530
|
-
// skip everything above "low" cost, same as the built-in "cheap"
|
|
1531
|
-
cheap: { maxCost: "low" },
|
|
1532
|
-
// skip destructive operations and two specific extraction endpoints
|
|
1533
|
-
safe: { excludeDestructive: true, exclude: ["report.extract", "report.summarize"] },
|
|
1534
|
-
},
|
|
1535
|
-
profile: "safe", // --profile on the CLI overrides this
|
|
1536
|
-
})
|
|
1537
|
-
```
|
|
1538
|
-
|
|
1539
|
-
An excluded operation never silently narrows what gets reported. Excluding an entity's `create` degrades that entity the same way a real seeding failure does — a `COVERAGE_GAP` naming the reason, then read-only checks run against whatever the list route already returns; `BLOCKED` if nothing exists to fall back on. Excluding `read`/`update`/`delete` individually stands down just the checks that need that one operation.
|
|
1540
|
-
|
|
1541
|
-
Exclusion applies to every invocation, not only create/read/update/delete. `effects.declared-effect-occurs` and `async.reaches-terminal-state` drop ops the profile forbids and record `profile.skip` — they do not POST `extract.once` under `--profile cheap`. A check whose only targets were excluded did not apply / names the profile; it never looks like the backend returned 500.
|
|
1542
|
-
|
|
1543
|
-
The run summary states what a profile skipped: `skipped 12 operation(s) under --profile cheap (12 high-cost)` — the same principle as `did not apply` for a coverage gap: a report that only shows what ran invites the reader to assume the rest was verified.
|
|
1544
|
-
|
|
1545
|
-
`--profile cheap` also defaults the exchange journal **off**. `--save-exchanges` turns it back on; `--quiet` does not change it.
|
|
1546
|
-
|
|
1547
|
-
### `x-rate-limit` — pacing oat's own traffic
|
|
1548
|
-
|
|
1549
|
-
```yaml
|
|
1550
|
-
x-rate-limit: { category: ai, rps: 3 }
|
|
1551
|
-
```
|
|
1552
|
-
|
|
1553
|
-
Groups operations sharing one throughput budget (`category`) and optionally the rate itself (`rps`). Without this, oat's matrix can hammer a `login` or paid-inference route past its real limit, collect 429s, and report them as backend defects — exactly the false-positive class that erodes trust fastest. With it, requests to that category are paced through a token bucket before they fire.
|
|
1554
|
-
|
|
1555
|
-
Tags are the **proactive** path. HTTP 429 is the **reactive** path and is honoured even when the operation has no `x-rate-limit` at all (untagged JWT writes, teardown DELETEs). oat reads `Retry-After` (delta-seconds or HTTP-date), otherwise backs off 1s, 2s, 4s… capped at 30s, and retries the same request up to five times. The wait is fed into the matching bucket, or into an implicit `untagged-write` bucket so the next call does not immediately re-trip.
|
|
1556
|
-
|
|
1557
|
-
A 429 is only ever reported when the request that drew it was demonstrably under the _declared_ rate — oat's own bucket had a free token, so it did not have to wait for one. A 429 that arrived only after oat's bucket made the request wait means oat's own rate model was too generous, which is paced around, never reported. A 429 against a config-supplied or implicit rate is never a finding.
|
|
1558
|
-
|
|
1559
|
-
`config.rateLimits` is checked first and needs no tag at all — it is what keeps oat usable against a backend that has not adopted `x-rate-limit` yet, or against an environment (staging, usually) whose real limit differs from what the document claims for production. A 429 against a config-supplied rate is never a finding: it is the operator's own belief about the environment, not a claim the API made.
|
|
1560
|
-
|
|
1561
|
-
```ts
|
|
1562
|
-
export default defineConfig({
|
|
1563
|
-
// ...
|
|
1564
|
-
rateLimits: [
|
|
1565
|
-
{ match: "POST /v1/auth/login", rps: 2 },
|
|
1566
|
-
{ match: "auth.login", rps: 2 }, // operationId works too
|
|
1567
|
-
{ match: "category:ai", rps: 1 }, // overrides every x-rate-limit-tagged "ai" operation at once
|
|
1568
|
-
],
|
|
1569
|
-
})
|
|
1570
|
-
```
|
|
1571
|
-
|
|
1572
|
-
## Checks
|
|
1573
|
-
|
|
1574
|
-
96 checks. A check that cannot run says so (`did not apply` + `needs`). A check that depends on a broken primitive is `BLOCKED`. A check that ran and stopped is inconclusive, not a pass.
|
|
1575
|
-
|
|
1576
|
-
Order is fixed (foundations first) so cascade suppression has a cause to point at. Mutating checks run alone; read-only checks may share in-flight requests under `maxInFlight`.
|
|
1577
|
-
|
|
1578
|
-
`depends` is the `dependsOn` list: if any of those already failed **for this entity**, this check is `BLOCKED` rather than reported as a second defect. Suppression is transitive.
|
|
1579
|
-
|
|
1580
|
-
| id | asserts | needs | depends |
|
|
1581
|
-
| ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------- | --------------------------------------- |
|
|
1582
|
-
| `list.read-after-write` | a just-created record appears on the list | create + a seeded record | — |
|
|
1583
|
-
| `create.persists-submitted-fields` | every writable field sent on create is echoed | create that echoes the record | `list.read-after-write` |
|
|
1584
|
-
| `payload.string-survives` | a documented-valid string is stored exactly; 4xx after an ASCII control is a fail | update or create+delete, item GET, unconstrained string | `list.read-after-write` |
|
|
1585
|
-
| `create.status-matches-document` | create status is one the document declared | create | — |
|
|
1586
|
-
| `response.status-is-documented` | every non-create exchange returns a status that operation names (`default` ≠ 201) | a modeled non-create operation oat invoked | — |
|
|
1587
|
-
| `schema.success-response-matches-document` | create body validates against the success schema | success schema on create | `create.status-matches-document` |
|
|
1588
|
-
| `schema.error-response-matches-document` | an error body validates against the documented error schema | error schema on the item route | — |
|
|
1589
|
-
| `pagination.limit-bounds-page-size` | page size ≤ the requested limit | page-size _role_ (aliases include `limit`, `per_page`, …) | `list.read-after-write` |
|
|
1590
|
-
| `pagination.limit-respects-documented-max` | requesting more than `maxLimit` does not return more | declared `maxLimit` and a larger cohort | `pagination.limit-bounds-page-size` |
|
|
1591
|
-
| `pagination.has-more-is-accurate` | `hasMore` / `Link rel=next` matches whether another page exists | page-forward + `hasMore` or `Link rel=next` | `pagination.limit-bounds-page-size` |
|
|
1592
|
-
| `pagination.page-walk-covers-set` | walking pages covers the collection with no gaps or dupes | page or offset + ≥3 records | `pagination.limit-bounds-page-size` |
|
|
1593
|
-
| `pagination.cursor-agrees-with-page` | cursor walk and page walk yield the same set | both cursor and page | `pagination.limit-bounds-page-size` |
|
|
1594
|
-
| `filter.unknown-field-rejected` | a filter on a field that does not exist is not silently ignored | a filter expression | — |
|
|
1595
|
-
| `filter.equality-selects-exactly-one` | `id.eq.<one>` returns that one record | equality on the identity | `list.read-after-write` |
|
|
1596
|
-
| `filter.zero-match-returns-none` | a filter that matches nothing returns an empty page, not the whole set | same | `list.read-after-write` |
|
|
1597
|
-
| `filter.negation-partitions-the-set` | `eq` ∪ `neq` = whole set, intersection empty | eq and neq | `list.read-after-write`, equality |
|
|
1598
|
-
| `filter.and-composes-as-intersection` | `and(A,B)` = A ∩ B | two filterable fields + AND | equality / list |
|
|
1599
|
-
| `filter.or-composes-as-union` | `or(A,B)` = A ∪ B | `or()` — **postgrest grammar only** | equality / list |
|
|
1600
|
-
| `filter.like-metacharacters-escaped` | `%` `_` `*` in a value are literals, not wildcards | like operator | `list.read-after-write` |
|
|
1601
|
-
| `filter.numeric-comparison-is-numeric` | `gt`/`lt` on a number uses numeric order, not TEXT (`1,10,2`) | numeric field + a filter param | `list.read-after-write`, unknown-field |
|
|
1602
|
-
| `filter.in-is-union-of-eq` | `in.(a,b)` = `eq.a` ∪ `eq.b` | field allows `in`; ≥2 distinct values | equality |
|
|
1603
|
-
| `filter.nin-complements-in` | `in` ∩ `nin` empty; union = set minus nulls | field allows `in` and `nin` | `in` |
|
|
1604
|
-
| `filter.gte-is-gt-or-eq` | `gte.x` = `gt.x` ∪ `eq.x` | field allows `gte` and `gt`; ordered type | numeric comparison |
|
|
1605
|
-
| `filter.lte-is-lt-or-eq` | `lte.x` = `lt.x` ∪ `eq.x` | field allows `lte` and `lt` | numeric comparison |
|
|
1606
|
-
| `filter.ordered-triple-partitions` | `lt` ∪ `eq` ∪ `gt` = set minus nulls; pairwise disjoint | field allows all three | numeric comparison |
|
|
1607
|
-
| `filter.ilike-is-case-insensitive` | case-flipped value: `ilike` matches; `like` does not (unresolved if both match) | field allows both | like |
|
|
1608
|
-
| `filter.is-null-selects-nulls` | `is.null` = nulls; `is.notnull` = complement | field allows `is`; cohort has a null | foundations |
|
|
1609
|
-
| `filter.contains-membership` | `contains.<el>` = records whose array value includes `el` | array field or ops include `contains` | foundations |
|
|
1610
|
-
| `filter.nested-and-or-distributes` | `and(A,or(B,C))` = (A∩B) ∪ (A∩C) | postgrest grammar; ≥2 filterable fields | `and` / `or` |
|
|
1611
|
-
| `filter.alias-matches-canonical` | each declared alias token selects the same id-set as its target | `aliases` non-empty | equality |
|
|
1612
|
-
| `filter.illegal-op-rejected` | one op **not** in that field's `ops` is 4xx | closed `ops` list | unknown-field |
|
|
1613
|
-
| `filter.empty-in` | `in.()` is 4xx (`reject`) or zero rows (`match-none`) | `emptyIn` set; field allows `in` | `in` |
|
|
1614
|
-
| `filter.in-over-limit-rejected` | `in` list of `maxInValues+1` is 4xx | `maxInValues` set | foundations |
|
|
1615
|
-
| `filter.condition-cap-rejected` | `maxFilterConditions+1` `eq` terms is 4xx | `maxFilterConditions` set | foundations |
|
|
1616
|
-
| `spec.declared-filterable-ops-accepted` | every declared field × every op in that field's `ops` returns <400 | closed `ops` | declared-filterable |
|
|
1617
|
-
| `spec.declared-filterable-illegal-op-rejected` | one illegal op per declared field is 4xx | closed `ops` | illegal-op |
|
|
1618
|
-
| `error.malformed-filter-not-5xx` | garbage filter text is 4xx, never 5xx | a filter expression | — |
|
|
1619
|
-
| `query.filter-selects-from-whole-set` | a filter is applied to the collection, not to the current page | filterable + ≥3 records | list / walk |
|
|
1620
|
-
| `sort.order-is-applied` | requesting a sort actually rearranges the page | order + a sortable field | `pagination.limit-bounds-page-size` |
|
|
1621
|
-
| `sort.reverse-symmetry` | desc is the reverse of asc (nulls included) | order + asc/desc | order-is-applied |
|
|
1622
|
-
| `sort.unknown-field-rejected` | `order` on an undeclared field is 4xx, not silent ignore | order param | — |
|
|
1623
|
-
| `sort.numeric-order-is-numeric` | `1,10,2` sorts as numbers, not text | numeric sortable field whose lexical order disagrees | order-is-applied |
|
|
1624
|
-
| `sort.nulls-first-last` | `nullsfirst` / `nullslast` put nulls at the start / end of asc | declared nulls token; dotted grammar; cohort has a null | order + reverse |
|
|
1625
|
-
| `sort.multi-key-tiebreak` | ties on the first key are ordered by the second | ≥2 sortable fields; `maxKeys` absent or ≥2 | order + reverse |
|
|
1626
|
-
| `sort.default-order-applied` | omitting `order` matches `defaultOrder` | `defaultOrder` set; walk complete | order + walk |
|
|
1627
|
-
| `sort.stable-tiebreak` | the same `order` twice yields the same sequence | `stableTiebreak` set | order-is-applied |
|
|
1628
|
-
| `spec.declared-sortable-nulls-accepted` | each declared nulls token returns <400 | some field/global declares `nulls` | order-is-applied |
|
|
1629
|
-
| `search.q-narrows-result` | a search term that matches one record does not return the whole set | search param + searchable fields | `list.read-after-write` |
|
|
1630
|
-
| `search.tokens-and` | `q=a b` = intersection (AND). Sameness of AND and OR passes | searchable fields; two tokens that split the cohort | `q-narrows` |
|
|
1631
|
-
| `search.case-insensitive` | case-flipped token matches the same set (unresolved if the backend is case-sensitive) | searchable field with a letter | `q-narrows` |
|
|
1632
|
-
| `search.empty-q` | `q=` is ignore / match-all / reject as declared | `searchEmpty` set | `q-narrows` |
|
|
1633
|
-
| `search.undeclared-field-not-required` | extra recall on a non-searchable field is not `SEARCH_IGNORED` | ≥1 searchable and ≥1 non-searchable string | `q-narrows` |
|
|
1634
|
-
| `search.mode-accepted` | each declared `searchModes` value is <400 on the mode param | `searchModes` set and a mode role | — |
|
|
1635
|
-
| `search.modes-differ` | two modes may differ; exact sameness is unresolved, not a fail | ≥2 `searchModes`; a mode param | `mode-accepted` |
|
|
1636
|
-
| `select.projection-honoured` | `select=id,name` does not return undeclared fields | select param | — |
|
|
1637
|
-
| `select.requested-fields-present` | every name in `select=` appears on each returned item | select param; ≥1 selectable field | projection |
|
|
1638
|
-
| `select.unknown-field-rejected` | unknown select name is 4xx (`reject`) or dropped (`ignore`) | `select.unknown` set | projection |
|
|
1639
|
-
| `select.nested-honoured` | `rel(col)` returns `rel` as an object/array carrying only `col` | `select.nested` and a named relation | projection |
|
|
1640
|
-
| `count.consistent-with-returned-page` | envelope total ≥ rows on this page, and is not zero when the page is not | envelope total | `list.read-after-write` |
|
|
1641
|
-
| `count.matches-filtered-set` | filtered total equals the size of the filtered walk | total + a filter | list, equality |
|
|
1642
|
-
| `query.axes-compose` | filter + sort together: filter still holds on the sorted page | filterable + sortable | filter + sort foundations |
|
|
1643
|
-
| `query.filter-and-select-compose` | filter + select together | filterable + select | same |
|
|
1644
|
-
| `query.search-and-filter-compose` | search + filter together | filterable + search | same |
|
|
1645
|
-
| `query.filter-sort-select-compose` | filter + sort + select | filter + sort + select | same |
|
|
1646
|
-
| `query.filter-search-sort-compose` | filter + search + sort | filter + search + sort | same |
|
|
1647
|
-
| `query.filter-search-select-compose` | filter + search + select | filter + search + select | same |
|
|
1648
|
-
| `query.sort-and-select-compose` | sort + select: order holds; extras dropped; requested fields present | sortable + selectable | sort + select |
|
|
1649
|
-
| `query.search-and-select-compose` | search + select: search membership holds; extras dropped | searchable + selectable | search + select |
|
|
1650
|
-
| `query.search-and-sort-compose` | search + sort: search membership holds; remaining rows ordered | searchable + sortable | search + sort |
|
|
1651
|
-
| `query.filter-search-sort-select-compose` | all four: filter ∩ search; order holds; extras dropped | all four axes declared | the triples |
|
|
1652
|
-
| `spec.declared-filterable-is-filterable` | every `x-query.filterable` field actually accepts a filter | `x-query` naming filterable fields | filter foundations |
|
|
1653
|
-
| `spec.declared-sortable-is-sortable` | every `x-query.sortable` field actually accepts a sort | `x-query` naming sortable fields | sort foundations |
|
|
1654
|
-
| `spec.declared-selectable-is-selectable` | every `x-query.selectable` field actually accepts a select | `x-query` naming selectable fields | select |
|
|
1655
|
-
| `tenant.item-not-readable-cross-tenant` | principal B cannot GET principal A's item | second principal, different `roots`, and a tagged or inferred tenant | — |
|
|
1656
|
-
| `tenant.denial-does-not-reveal-existence` | 404 vs 403 (or equivalent) does not distinguish "exists other tenant" from "missing" | second principal, and a tagged or inferred tenant | `tenant.item-not-readable-cross-tenant` |
|
|
1657
|
-
| `tenant.filter-does-not-bypass-scope` | `filter=id.eq.<other tenant>` does not return that row | second principal, a filter, and a tagged or inferred tenant | `query.filter-selects-from-whole-set` |
|
|
1658
|
-
| `auth.rank-is-monotonic` | a lower rank cannot do what a higher rank is denied | two same-tenant principals at different `rank` | `list.read-after-write` |
|
|
1659
|
-
| `auth.invite-grants-then-revokes` | invite → accept grants; revoke takes it back | `x-invite` + peer with `inviteAs` | list, cross-tenant |
|
|
1660
|
-
| `create.unique-conflict-rejected` | a second create colliding a documented unique set is 409, not 2xx; the list does not grow | `x-unique` with a probeable create body column and a known row | — |
|
|
1661
|
-
| `update.unique-conflict-rejected` | PATCHing a **different** row onto another row's unique-set values is 409, not 2xx | `x-unique`, update, two known rows; skip all-`x-immutable` sets | — |
|
|
1662
|
-
| `patch.immutable-field-rejected` | PATCHing an `x-immutable` field is rejected or ignored | `x-immutable` | — |
|
|
1663
|
-
| `softdelete.absent-from-default-list` | a soft-deleted row is gone from the default list | `x-soft-delete` | `list.read-after-write` |
|
|
1664
|
-
| `invalidation.declared-route-changes` | after a write, the other entity's listed route actually changes | `x-invalidate` naming another entity | list, persist |
|
|
1665
|
-
| `effects.declared-effect-occurs` | `x-effects` cardinality delta is observed on the named list (`count` exact, `min` at-least; nested child lists bind the created parent id) | `x-effects` | `list.read-after-write` |
|
|
1666
|
-
| `effects.side-effect-arrives` | after the write, the named GET’s JSON path is occupied before `timeoutMs` | `x-wait` | `list.read-after-write` |
|
|
1667
|
-
| `async.reaches-terminal-state` | `x-async` reaches `until` (poll, or a terminal SSE frame) before `timeoutMs` | `x-async` | — |
|
|
1668
|
-
| `async.receipt-identifies-the-job` | `idFrom` on the receipt (JSON object or SSE event JSON) resolves to a pollable job | `x-async` + `idFrom` | — |
|
|
1669
|
-
| `patch.minimality` | PATCH `{ name }` does not clear other writable fields | update + item route | — |
|
|
1670
|
-
| `idempotency.replay-does-not-duplicate` | same Idempotency-Key + same body does not create a second row | create + documented Idempotency-Key header | list, persist |
|
|
1671
|
-
| `delete.absent-record-returns-404` | DELETE of a missing id is 404, not 200 | delete | — |
|
|
1672
|
-
| `concurrency.no-lost-update` | two PATCHes to different fields do not clobber each other | update + two writable strings | persist + patch |
|
|
1673
|
-
| `validation.enum-enforced` | a value outside the enum is rejected | enum in the request schema | — |
|
|
1674
|
-
| `validation.max-length-enforced` | a string over `maxLength` is rejected | maxLength | — |
|
|
1675
|
-
| `validation.required-enforced` | omitting a required field is rejected | required field | — |
|
|
1676
|
-
| `validation.content-type-enforced` | a wrong Content-Type is 415 when 415 is documented | documented 415 | — |
|
|
1677
|
-
| `consistency.projections-agree` | list, item, and filtered views of the same field agree | item route + a comparable field | list + persist + filter |
|
|
1678
|
-
|
|
1679
|
-
On a typical untagged CRUD document (create, list, item, `page`/`limit`, maybe `sort`):
|
|
1680
|
-
|
|
1681
|
-
- Foundations, PATCH/delete, and schema checks usually run.
|
|
1682
|
-
- The query matrix runs when filter/order/select/search roles resolve.
|
|
1683
|
-
- Isolation runs only if you configured two principals.
|
|
1684
|
-
- Spec-as-adversary and tagged behaviour never run — `doctor` says so.
|
|
1685
|
-
|
|
1686
|
-
Worked examples of what a finding looks like:
|
|
1687
|
-
|
|
1688
|
-
```text
|
|
1689
|
-
BACKEND_BUG table created row missing from GET /tables
|
|
1690
|
-
list.read-after-write
|
|
1691
|
-
|
|
1692
|
-
BACKEND_BUG product PATCH { name } cleared description
|
|
1693
|
-
patch.minimality
|
|
1694
|
-
|
|
1695
|
-
SECURITY table GET /tables/{id} readable with the other tenant's key
|
|
1696
|
-
tenant.item-not-readable-cross-tenant
|
|
1697
|
-
|
|
1698
|
-
SPEC_BUG table x-query.filterable lists "ghost"; filter=ghost.eq.x is 400
|
|
1699
|
-
spec.declared-filterable-is-filterable
|
|
1700
|
-
|
|
1701
|
-
BACKEND_BUG table in() is not the union of the equalities it lists
|
|
1702
|
-
filter.in-is-union-of-eq
|
|
1703
|
-
|
|
1704
|
-
BACKEND_BUG table an illegal filter operator is accepted
|
|
1705
|
-
filter.illegal-op-rejected
|
|
1706
|
-
|
|
1707
|
-
COVERAGE_GAP batch no x-async; receipt treated as the result
|
|
1708
|
-
async.reaches-terminal-state
|
|
1709
|
-
```
|
|
1710
|
-
|
|
1711
|
-
## Verdicts, skips, and exit codes
|
|
1712
|
-
|
|
1713
|
-
| verdict | meaning | fails `oat run`? |
|
|
1714
|
-
| -------------- | -------------------------------------------------------------------- | ---------------- |
|
|
1715
|
-
| `BACKEND_BUG` | the handler is wrong | yes |
|
|
1716
|
-
| `SPEC_BUG` | the document is wrong (or disagrees with the handler) | yes |
|
|
1717
|
-
| `SECURITY` | isolation/authz failure and `x-tenant` (or equivalent) was declared | yes |
|
|
1718
|
-
| `AMBIGUITY` | same evidence, but the tenant boundary was only inferred | yes |
|
|
1719
|
-
| `COVERAGE_GAP` | the check could not run; the report names the missing tag or surface | no |
|
|
1720
|
-
| `BLOCKED` | a check this one depends on already failed | no |
|
|
1721
|
-
|
|
1722
|
-
Separate from findings:
|
|
1723
|
-
|
|
1724
|
-
- **did not apply** — entity never had what `needs` lists (printed in the report, not a pass). Example: no `select` parameter → `select.projection-honoured` did not apply.
|
|
1725
|
-
- **inconclusive** — the check ran and stopped (empty listing, probe 4xx, no shared filterable field, …). Not a pass. The report prints the reason.
|
|
1726
|
-
|
|
1727
|
-
Coverage is split **never** (zero entities could run it) vs **partial** (ran on some entities, skipped on others). A clean run still prints both. "Nothing found" and "nothing was looked for" are different.
|
|
1728
|
-
|
|
1729
|
-
Cascade suppression is transitive: one root cause is one finding, not a page of consequences. A blocked check has **not** been verified; re-run after the cause is fixed.
|
|
1730
|
-
|
|
1731
|
-
`oat run` exit `1` if any finding has a failing verdict. Gaps and blocked entries do not fail a full run. A targeted run (`--ops`) also exits `1` when any target did not end `held`.
|
|
1732
|
-
|
|
1733
|
-
Console (stdout) after a run:
|
|
1734
|
-
|
|
1735
|
-
```
|
|
1736
|
-
50 checks · 5 entities · 842 requests · 41.2s · p95 90ms · 4 checks did not apply
|
|
1737
|
-
|
|
1738
|
-
BACKEND DEFECTS (1)
|
|
1739
|
-
product PATCH { name } also cleared description
|
|
1740
|
-
patch.minimality
|
|
1741
|
-
|
|
1742
|
-
DID NOT APPLY — no entity had what these need
|
|
1743
|
-
async.reaches-terminal-state needs an operation declaring x-async
|
|
1744
|
-
|
|
1745
|
-
leftover teardown printed next
|
|
1746
|
-
report / matrix / graph / progress paths
|
|
1747
|
-
```
|
|
1748
|
-
|
|
1749
|
-
## Reports
|
|
1750
|
-
|
|
1751
|
-
Written under `--out` (default `./.oat/runs`). Each invocation creates a UTC timestamp folder and points `latest` at it:
|
|
1752
|
-
|
|
1753
|
-
```
|
|
1754
|
-
.oat/runs/2026-08-18T12-00-00Z/
|
|
1755
|
-
.oat/runs/latest -> 2026-08-18T12-00-00Z
|
|
1756
|
-
```
|
|
1757
|
-
|
|
1758
|
-
`--out` / `outDir` replace the root, not the leaf — `--out .oat/runs/prod` writes `.oat/runs/prod/<datetime>/`.
|
|
1759
|
-
|
|
1760
|
-
| file | |
|
|
1761
|
-
| ---------------------------- | -------------------------------------------------------------------------------------------------------------------------- |
|
|
1762
|
-
| `oat-report.md` | human report: scope, per-operation status, summary, findings with request/response excerpts, coverage, latency p50/p95/max |
|
|
1763
|
-
| `oat-report.json` | same data for CI. Summary only — full exchanges are not inlined here |
|
|
1764
|
-
| `matrix.html` | visual matrix of entities × checks |
|
|
1765
|
-
| `matrix.json` | the same graph (AI-friendly), including a mermaid string |
|
|
1766
|
-
| `issue-repro/*.sh` | one executable `curl` script per finding that has evidence. Directory is omitted when the run is clean |
|
|
1767
|
-
| `progress.log` | logfmt, one event per line, never truncated |
|
|
1768
|
-
| `progress.jsonl` | same events as JSON |
|
|
1769
|
-
| `progress.tsv` | same columns, tab-separated. `req_id` is the join key to the exchange journal |
|
|
1770
|
-
| `progress.json` | latest snapshot only (overwritten ~1s) |
|
|
1771
|
-
| `exchanges.jsonl` | one line per request (`seq`, `requestId`, method, url, status, bytes, …). Greppable |
|
|
1772
|
-
| `exchanges/<requestId>.json` | full exchange: status, headers, described bodies. Missing id → `seq-<n>.json`. Duplicate id → `-<seq>` |
|
|
1773
|
-
| `blobs/<sha256>` | content-addressed file parts and oversized / binary bodies |
|
|
1774
|
-
|
|
1775
|
-
The journal is oat `Exchange` JSON, not HAR (HAR export is out of scope). Default **on** unless `--profile cheap`. `--no-save-exchanges` skips `exchanges/` entirely. `--quiet` does not. **`--no-save-exchanges` does not reduce transcript RAM** — it only skips the disk journal. The in-memory transcript always drops live `FormData` / `Blob` / `ArrayBuffer` / multi-MiB strings after each hop, keeping citeable metadata and content-addressed body descriptors. Reconstruct bytes from `blobs/<sha256>` when a check genuinely needs them.
|
|
1776
|
-
|
|
1777
|
-
Bodies are described, never base64 multipart. JSON / text inline up to 256 KiB, then `blobs/<sha256>`. `FormData` file parts and binary downloads (`application/pdf`, spreadsheet, image, `octet-stream`) are always blobs; the same fixture bytes POSTed N times share one file. SSE is parsed incrementally into `{ event, data }[]` frames — the raw `text/event-stream` concatenation is not retained on the transcript.
|
|
1778
|
-
|
|
1779
|
-
Redaction is on by default (not opt-in), applied on write:
|
|
1780
|
-
|
|
1781
|
-
- Headers: `authorization`, `cookie`, `set-cookie`, `proxy-authorization`, and any `x-*-key` / `x-*-secret` / `x-ia-tester-key` (case-insensitive).
|
|
1782
|
-
- JSON bodies, any depth: `access_token`, `refresh_token`, `id_token`, `password`, `token`, `secret`, `api_key` → `"<redacted>"`.
|
|
1783
|
-
|
|
1784
|
-
Every report names its scope. The console prints `scope: full · 141 of 212 operations graded` or `scope: targeted · 2 operations (--ops …)` followed by one line per target. `oat-report.md` has a **Scope** line and an **Operations** table: on a full run, every operation in the document with its status (so operations no check ever grades are visible); on a targeted run, the targets only, followed by **Support operations** with call and non-2xx counts. In `matrix.html` / `matrix.json` a check a targeted run did not ask for is `out-of-scope`, distinct from `skipped` (not applicable).
|
|
1785
|
-
|
|
1786
|
-
Join `progress.tsv` `req_id` to `exchanges.jsonl` `requestId` (response `x-request-id` / `request-id` / `x-correlation-id` / `correlation-id` wins; else what oat sent). `oat-report.md` includes one line such as `1841 exchanges → exchanges/`.
|
|
1787
|
-
|
|
1788
|
-
### `oat-report.json`
|
|
1789
|
-
|
|
1790
|
-
```json
|
|
1791
|
-
{
|
|
1792
|
-
"backend": "https://api.example.com",
|
|
1793
|
-
"generatedAt": "2026-08-15T12:00:00.000Z",
|
|
1794
|
-
"durationMs": 41200,
|
|
1795
|
-
"requests": 842,
|
|
1796
|
-
"entitiesTested": ["store", "product"],
|
|
1797
|
-
"scope": {
|
|
1798
|
-
"mode": "targeted",
|
|
1799
|
-
"requested": { "ops": ["product.update"], "only": [] },
|
|
1800
|
-
"operations": [
|
|
1801
|
-
{
|
|
1802
|
-
"operationId": "product.update",
|
|
1803
|
-
"entity": "product",
|
|
1804
|
-
"status": "failed",
|
|
1805
|
-
"checks": {
|
|
1806
|
-
"held": ["concurrency.no-lost-update"],
|
|
1807
|
-
"failed": ["patch.minimality"],
|
|
1808
|
-
"suppressed": [],
|
|
1809
|
-
"inconclusive": [],
|
|
1810
|
-
"gaps": []
|
|
1811
|
-
},
|
|
1812
|
-
"findings": 1,
|
|
1813
|
-
"reason": null
|
|
1814
|
-
}
|
|
1815
|
-
],
|
|
1816
|
-
"support": [{ "operationId": "product.create", "calls": 9, "non2xx": 0 }],
|
|
1817
|
-
"excluded": [],
|
|
1818
|
-
"originsSkipped": []
|
|
1819
|
-
},
|
|
1820
|
-
"checksOutOfScope": [{ "check": "filter.in-is-union-of-eq", "entity": "product" }],
|
|
1821
|
-
"checksRun": ["list.read-after-write", "patch.minimality"],
|
|
1822
|
-
"checksSkipped": [{ "check": "async.reaches-terminal-state", "entity": "store", "needs": "…" }],
|
|
1823
|
-
"checksSuppressed": [{ "check": "query.axes-compose", "entity": "store", "because": "list.read-after-write" }],
|
|
1824
|
-
"inconclusive": [{ "check": "filter.and-composes-as-intersection", "entity": "store", "reason": "…" }],
|
|
1825
|
-
"summary": { "BACKEND_BUG": 1 },
|
|
1826
|
-
"coverage": {
|
|
1827
|
-
"neverApplied": ["async.reaches-terminal-state"],
|
|
1828
|
-
"partial": [{ "check": "select.projection-honoured", "ran": 1, "skipped": 1 }]
|
|
1829
|
-
},
|
|
1830
|
-
"latency": {
|
|
1831
|
-
"p50": 12,
|
|
1832
|
-
"p95": 90,
|
|
1833
|
-
"max": 400,
|
|
1834
|
-
"slowest": { "method": "GET", "path": "/v1/products" }
|
|
1835
|
-
},
|
|
1836
|
-
"findings": [
|
|
1837
|
-
{
|
|
1838
|
-
"check": "patch.minimality",
|
|
1839
|
-
"verdict": "BACKEND_BUG",
|
|
1840
|
-
"entity": "product",
|
|
1841
|
-
"summary": "PATCH { name } also cleared description",
|
|
1842
|
-
"detail": "…",
|
|
1843
|
-
"evidence": [
|
|
1844
|
-
{
|
|
1845
|
-
"method": "PATCH",
|
|
1846
|
-
"url": "https://api.example.com/v1/products/p1",
|
|
1847
|
-
"status": 200,
|
|
1848
|
-
"requestBody": { "name": "x" },
|
|
1849
|
-
"responseBody": { "name": "x", "description": null }
|
|
1850
|
-
}
|
|
1851
|
-
]
|
|
1852
|
-
}
|
|
1853
|
-
]
|
|
1854
|
-
}
|
|
1855
|
-
```
|
|
1856
|
-
|
|
1857
|
-
Latency is reported, never asserted. oat has no baseline for "too slow".
|
|
1858
|
-
|
|
1859
|
-
Gate in CI on process exit code, or on `findings` whose `verdict` is not `COVERAGE_GAP` / `BLOCKED`.
|
|
1860
|
-
|
|
1861
|
-
### `matrix.json`
|
|
1862
|
-
|
|
1863
|
-
```json
|
|
1864
|
-
{
|
|
1865
|
-
"kind": "oat.matrix",
|
|
1866
|
-
"version": 2,
|
|
1867
|
-
"baseUrl": "…",
|
|
1868
|
-
"generatedAt": "…",
|
|
1869
|
-
"thesis": "…",
|
|
1870
|
-
"summary": "…",
|
|
1871
|
-
"index": { "entityCount": 5, "failed": ["product"], "parents": ["store"], "crossClaims": 1, "inbound": {} },
|
|
1872
|
-
"counts": { "failed": 1, "blocked": 0, "held": 40, "skipped": 14 },
|
|
1873
|
-
"entities": [
|
|
1874
|
-
{
|
|
1875
|
-
"name": "product",
|
|
1876
|
-
"identity": "id",
|
|
1877
|
-
"readSurface": ["GET /v1/products", "GET /v1/products/{id}"],
|
|
1878
|
-
"counts": { "failed": 1, "blocked": 0, "held": 20, "skipped": 5 },
|
|
1879
|
-
"roots": ["store_id"],
|
|
1880
|
-
"nodes": [
|
|
1881
|
-
{
|
|
1882
|
-
"id": "product/patch.minimality",
|
|
1883
|
-
"group": "product",
|
|
1884
|
-
"layer": "axis",
|
|
1885
|
-
"status": "failed",
|
|
1886
|
-
"verdict": "BACKEND_BUG",
|
|
1887
|
-
"summary": "…"
|
|
1888
|
-
}
|
|
1889
|
-
]
|
|
1890
|
-
}
|
|
1891
|
-
],
|
|
1892
|
-
"invalidate": [
|
|
1893
|
-
{
|
|
1894
|
-
"fromEntity": "product",
|
|
1895
|
-
"fromOp": "product.create",
|
|
1896
|
-
"toEntity": "store",
|
|
1897
|
-
"toRoute": "GET /v1/stores/{id}",
|
|
1898
|
-
"cross": true
|
|
1899
|
-
}
|
|
1900
|
-
],
|
|
1901
|
-
"edges": [{ "from": "product/list.read-after-write", "to": "product/patch.minimality", "kind": "dependsOn" }],
|
|
1902
|
-
"mermaid": "flowchart LR\n…"
|
|
1903
|
-
}
|
|
1904
|
-
```
|
|
1905
|
-
|
|
1906
|
-
Cell `status`: `held` (passed), `failed`, `blocked`, `skipped`. Edge `kind`: `dependsOn` (cascade) or `uses` (a composition check built from a single-axis check).
|
|
1907
|
-
|
|
1908
|
-
### `issue-repro/<entity>-<check>.sh`
|
|
1909
|
-
|
|
1910
|
-
Created only when a finding has HTTP evidence. Not created on a clean run.
|
|
1911
|
-
|
|
1912
|
-
```bash
|
|
1913
|
-
#!/usr/bin/env bash
|
|
1914
|
-
# Generated by oat. Set TOKEN to a valid credential before running.
|
|
1915
|
-
# product — PATCH { name } also cleared description
|
|
1916
|
-
set -u
|
|
1917
|
-
BASE="${BASE:-https://api.example.com}"
|
|
1918
|
-
TOKEN="${TOKEN:?set TOKEN to a valid credential}"
|
|
1919
|
-
|
|
1920
|
-
# step 1 — observed 200
|
|
1921
|
-
curl -sS -X PATCH "$BASE/v1/products/p1" \
|
|
1922
|
-
-H "authorization: Bearer $TOKEN" \
|
|
1923
|
-
-H "content-type: application/json" \
|
|
1924
|
-
-d '{"name":"x"}'
|
|
1925
|
-
```
|
|
1926
|
-
|
|
1927
|
-
`authorization`, `cookie`, and `x-api-key` are redacted to `$TOKEN`. Replay needs a live credential; ids in the script are whatever the failing run created (gone if teardown ran). Use `--keep-fixtures` when you want to replay against the same rows.
|
|
1928
|
-
|
|
1929
|
-
## Progress logs
|
|
1930
|
-
|
|
1931
|
-
`progress.log` starts with a glossary. Fields:
|
|
1932
|
-
|
|
1933
|
-
| key | |
|
|
1934
|
-
| -------------------------- | -------------------------------------------------------------------------------- |
|
|
1935
|
-
| `ts` | ISO-8601 UTC when the line was written |
|
|
1936
|
-
| `status` | `ok`, `stall` (`idle_ms` ≥ 15000 after the last call returned), or `in_flight` |
|
|
1937
|
-
| `done` / `total` | entity index / how many entities |
|
|
1938
|
-
| `phase` | `load` \| `auth` \| `seed` \| `test` \| `teardown` \| `done` |
|
|
1939
|
-
| `entity` / `check` | current entity and check id |
|
|
1940
|
-
| `msg` | phase note. In-flight is `status`, not `msg` |
|
|
1941
|
-
| `req` / `find` | request count, finding count |
|
|
1942
|
-
| `method` / `path` / `http` | last completed call, or the in-flight call when `status=in_flight` (`http=-`) |
|
|
1943
|
-
| `last_ms` | duration of the last completed call. `-` while `in_flight` |
|
|
1944
|
-
| `idle_ms` | while `in_flight`: ms since this request started. otherwise ms since it returned |
|
|
1945
|
-
| `elapsed_ms` | wall clock since start |
|
|
1946
|
-
|
|
1947
|
-
`--quiet` keeps the files and drops stderr.
|
|
1948
|
-
|
|
1949
|
-
Every HTTP call emits a **start** line (`status=in_flight`, `http=-`, `last_ms=-`) when `fetch` is issued, then a **completed** line when it returns (`http=200`, `last_ms` ≈ wall time, `idle_ms=0`). The 5 s heartbeat keeps the in-flight path and climbs `idle_ms` until the call returns; after return it climbs on the completed call (and becomes `stall` at 15 s). A 300 s `POST /extract` therefore stays `/extract` in the log, not the previous refresh or GET.
|
|
1950
|
-
|
|
1951
|
-
Lines are also written when the phase/entity/check/message changes, or every 2 s, or on `load`/`done`. `progress.json` is rewritten about once a second.
|
|
1952
|
-
|
|
1953
|
-
If `idle_ms` climbs through a long poll (`x-async`) that is expected. If it climbs on a simple GET while `status` is not `in_flight`, the process or the network is stuck. Live `run` aborts a single `fetch` after `network.requestTimeoutMs` (default **180s**). A hung `text/event-stream` becomes `net.unreachable` (`kind=timeout`), not an unbounded buffer on the transcript. Set `requestTimeoutMs: 0` only if you intend to wait for the socket.
|
|
1954
|
-
|
|
1955
|
-
Bun and Node both honour that timeout. If oat still runs away with RAM (a bug, or a caller holding the process open), treat the process like any other prefer-kill target for `earlyoom` / systemd — Bun is often **not** in a default `--prefer` list.
|
|
1956
|
-
|
|
1957
|
-
## Programmatic API
|
|
1958
|
-
|
|
1959
|
-
```ts
|
|
1960
|
-
import {
|
|
1961
|
-
defineConfig,
|
|
1962
|
-
loadConfig,
|
|
1963
|
-
run,
|
|
1964
|
-
loadSpec,
|
|
1965
|
-
dereference,
|
|
1966
|
-
buildModel,
|
|
1967
|
-
renderJson,
|
|
1968
|
-
renderMarkdown,
|
|
1969
|
-
} from "@lovrozagar/oat"
|
|
1970
|
-
|
|
1971
|
-
const config = defineConfig({
|
|
1972
|
-
spec: "./openapi.yaml",
|
|
1973
|
-
baseUrl: "https://api.example.com",
|
|
1974
|
-
principals: [{ id: "alpha", headers: { authorization: `Bearer ${process.env.API_TOKEN}` } }],
|
|
1975
|
-
})
|
|
1976
|
-
|
|
1977
|
-
const result = await run({
|
|
1978
|
-
spec: config.spec,
|
|
1979
|
-
baseUrl: config.baseUrl,
|
|
1980
|
-
principals: config.principals,
|
|
1981
|
-
seed: 1,
|
|
1982
|
-
maxInFlight: 4,
|
|
1983
|
-
onProgress: (snap) => {
|
|
1984
|
-
// snap.phase, snap.entity, snap.check, snap.inflight, snap.last, snap.message, …
|
|
1985
|
-
},
|
|
1986
|
-
})
|
|
1987
|
-
|
|
1988
|
-
// result.findings, result.checksSkipped, result.checksSuppressed,
|
|
1989
|
-
// result.inconclusive, result.entitiesTested, result.teardown, result.created,
|
|
1990
|
-
// result.scope (per-operation status), result.checksOutOfScope
|
|
1991
|
-
// pass `ops: ["product.update"]` to run() for a targeted run; ScopeError on an unknown name
|
|
1992
|
-
```
|
|
1993
|
-
|
|
1994
|
-
`loadConfig(path)` loads `.ts` / `.js` / `.mjs` / `.json` the same way the CLI does. The CLI then expands `${NAME}` in every string. `defineConfig` is an identity function for typing; it does not interpolate. If you call `run()` with an in-process object, resolve secrets yourself (template literals, `process.env`) before passing it.
|
|
1995
|
-
|
|
1996
|
-
`run(options)` does not write files. The CLI writes reports after `run` returns. To produce the same artifacts, call `renderMarkdown` / `renderJson` / `renderMatrixHtml` / `renderMatrixGraph` / `renderRepros` with a `ReportInput` (`findings`, `model`, `client`, `baseUrl`, `entitiesTested`, `checksRun`, `startedAt`, `durationMs`, plus optional skip/suppress/inconclusive lists).
|
|
1997
|
-
|
|
1998
|
-
Offline:
|
|
1999
|
-
|
|
2000
|
-
```ts
|
|
2001
|
-
const doc = await loadSpec("./openapi.yaml")
|
|
2002
|
-
const { doc: resolved, externalRefs } = dereference(doc)
|
|
2003
|
-
const model = buildModel(resolved)
|
|
2004
|
-
```
|
|
2005
|
-
|
|
2006
|
-
Types exported: `OatConfig`, `Principal`, `AuthFlow`, `AuthRefresh`, `AuthStep`, `HookAuth`, `Hooks`, `Uploads`, `UploadRequest`, `UploadFile`, `HeaderRequest`, `InputRequest`, `OriginSpec`, `OutOfBandConfig`, `OutOfBandRequest`, `TeardownPrincipalContext`, `RunOptions`, `RunResult`, `Finding`, `Verdict`, `Actor`, `SpecModel`, `EntityModel`, `OperationModel`, `OpenApiDocument`, `AuthRefreshRequiredError`, `loadPersistedPrincipals`, `worstCaseWaitMs`, `allocateRunDir`, `DEFAULT_RUNS_ROOT`, matrix types.
|
|
2007
|
-
|
|
2008
|
-
## CI
|
|
2009
|
-
|
|
2010
|
-
This repository runs [`.github/workflows/ci.yml`](./.github/workflows/ci.yml) on every push and pull request to `main`: format, lint, typecheck, then `npm test` (conformance on memory + sqlite, plus the built-in combination smoke). D1, live labs Workers, and postgres are not in that job — D1 needs Cloudflare credentials, and the postgres reference backend is not yet a green CI gate.
|
|
2011
|
-
|
|
2012
|
-
The package is [`@lovrozagar/oat` on npm](https://www.npmjs.com/package/@lovrozagar/oat). That URL is the repository website. GitHub Releases match npm versions. Pushing a tag `vX.Y.Z` (same as `package.json` `version`) runs [`.github/workflows/release.yml`](./.github/workflows/release.yml): test, `npm publish` via trusted publishing, then a GitHub Release. Configure the trusted publisher once on npm (package Settings → Trusted Publisher → GitHub Actions, workflow `release.yml`, no environment). Do not publish to GitHub Packages — people install from the public npm registry.
|
|
2013
|
-
|
|
2014
|
-
Against **your** API:
|
|
2015
|
-
|
|
2016
|
-
```yaml
|
|
2017
|
-
# GitHub Actions sketch
|
|
2018
|
-
- run: npm i -D @lovrozagar/oat
|
|
2019
|
-
- run: oat doctor --spec "$SPEC_URL"
|
|
2020
|
-
- run: oat run --config oat.config.ts
|
|
2021
|
-
env:
|
|
2022
|
-
API_TOKEN: ${{ secrets.API_TOKEN }}
|
|
2023
|
-
API_TOKEN_B: ${{ secrets.API_TOKEN_B }}
|
|
2024
|
-
```
|
|
2025
|
-
|
|
2026
|
-
Gate on exit code 1 (defects). Read `.oat/runs/latest/oat-report.json` if you need to classify verdicts.
|
|
128
|
+
Gate on exit code 1 (defects); treat 3 as "the run itself failed", not as a pass or a defect. Read `.oat/runs/latest/oat-report.json` if you need to classify verdicts.
|
|
2027
129
|
|
|
2028
130
|
Wipe leftover rows on a shared database between runs if a previous `--keep-fixtures` or a crashed teardown left data. Leftover rows make numeric/filter checks look like type bugs (`1, 10, 2` from old TEXT-sorted leftovers mixed with a fresh numeric cohort).
|
|
2029
131
|
|
|
2030
|
-
## Reference defects (`oat serve --defects`)
|
|
2031
|
-
|
|
2032
|
-
Comma-separated. Each is one named lie the demo API can tell. Primary check is what conformance asserts; extras in parentheses are accepted additional symptoms of the same lie.
|
|
2033
|
-
|
|
2034
|
-
| defect | primary check | the lie |
|
|
2035
|
-
| ------------------------------------------- | ------------------------------------------ | ------------------------------------------------------ |
|
|
2036
|
-
| `STALE_LIST` | `list.read-after-write` | create succeeds, list does not show the row |
|
|
2037
|
-
| `CREATE_DROPS_FIELD` | `create.persists-submitted-fields` | a submitted field is dropped |
|
|
2038
|
-
| `STRING_PAYLOAD_MANGLED` | `payload.string-survives` | non-ASCII / surrounding whitespace stripped on write |
|
|
2039
|
-
| `RESPONSE_STATUS_UNDECLARED` | `response.status-is-documented` | PATCH returns 201 when the document names 200 |
|
|
2040
|
-
| `CREATED_201_AS_200` | `create.status-matches-document` | create returns 200 when the spec says 201 |
|
|
2041
|
-
| `RESPONSE_SCHEMA_DRIFT` | `schema.success-response-matches-document` | success body does not match the schema |
|
|
2042
|
-
| `ERROR_SCHEMA_DRIFT` | `schema.error-response-matches-document` | error body does not match the schema |
|
|
2043
|
-
| `LIMIT_IGNORED` | `pagination.limit-bounds-page-size` | `limit` is accepted and ignored |
|
|
2044
|
-
| `LIMIT_EXCEEDS_MAX` | `pagination.limit-respects-documented-max` | documented max is not capped |
|
|
2045
|
-
| `HASMORE_ALWAYS_FALSE` | `pagination.has-more-is-accurate` | `hasMore` is always false |
|
|
2046
|
-
| `OFF_BY_ONE_PAGE` | `pagination.page-walk-covers-set` | page walk skips or repeats |
|
|
2047
|
-
| `UNSTABLE_SORT` | `pagination.page-walk-covers-set` | default order is not a total order |
|
|
2048
|
-
| `CURSOR_DRIFT` | `pagination.cursor-agrees-with-page` | cursor and page disagree |
|
|
2049
|
-
| `FILTER_IGNORED` | `filter.unknown-field-rejected` | unknown filter field is ignored |
|
|
2050
|
-
| `FILTER_EQ_NOT_APPLIED` | `filter.equality-selects-exactly-one` | equality filter is ignored |
|
|
2051
|
-
| `EMPTY_RESULT_RETURNS_ALL` | `filter.zero-match-returns-none` | empty match returns the whole set |
|
|
2052
|
-
| `NEQ_DROPS_NULLS` | `filter.negation-partitions-the-set` | `neq` drops nulls so the partition leaks |
|
|
2053
|
-
| `FILTER_GROUP_COMBINATOR_SWAPPED` | `filter.and-composes-as-intersection` | `and`/`or` are swapped |
|
|
2054
|
-
| `LIKE_UNESCAPED` | `filter.like-metacharacters-escaped` | `%`/`_` are wildcards in values |
|
|
2055
|
-
| `NUMERIC_COMPARED_AS_TEXT` | `filter.numeric-comparison-is-numeric` | numbers compared as strings |
|
|
2056
|
-
| `ERROR_500_ON_BAD_FILTER` | `error.malformed-filter-not-5xx` | bad filter is 500 |
|
|
2057
|
-
| `FILTER_AFTER_PAGINATION` | `query.filter-selects-from-whole-set` | filter applied after the page is cut |
|
|
2058
|
-
| `ORDER_IGNORED` | `sort.order-is-applied` | sort param is ignored |
|
|
2059
|
-
| `SORT_DESC_DROPS_NULLS` | `sort.reverse-symmetry` | desc drops nulls |
|
|
2060
|
-
| `SEARCH_IGNORED` | `search.q-narrows-result` | search param is ignored |
|
|
2061
|
-
| `SELECT_IGNORED` | `select.projection-honoured` | select param is ignored |
|
|
2062
|
-
| `COUNT_ALWAYS_ZERO` | `count.consistent-with-returned-page` | total is always 0 |
|
|
2063
|
-
| `COUNT_IGNORES_FILTER` | `count.matches-filtered-set` | total ignores the filter |
|
|
2064
|
-
| `FILTER_DROPPED_WHEN_SORTED` | `query.axes-compose` | sort drops the filter |
|
|
2065
|
-
| `FILTER_DROPPED_WHEN_SELECTED` | `query.filter-and-select-compose` | select drops the filter |
|
|
2066
|
-
| `FILTER_DROPPED_WHEN_SEARCHED` | `query.search-and-filter-compose` | search drops the filter |
|
|
2067
|
-
| `FILTER_DROPPED_WHEN_SORTED_AND_SELECTED` | `query.filter-sort-select-compose` | the triple drops the filter |
|
|
2068
|
-
| `FILTER_DROPPED_WHEN_SORTED_AND_SEARCHED` | `query.filter-search-sort-compose` | the triple drops the filter |
|
|
2069
|
-
| `FILTER_DROPPED_WHEN_SEARCHED_AND_SELECTED` | `query.filter-search-select-compose` | the triple drops the filter |
|
|
2070
|
-
| `SPEC_OVERCLAIMS_FILTERABLE` | `spec.declared-filterable-is-filterable` | `x-query` lists a field that 400s |
|
|
2071
|
-
| `SPEC_OVERCLAIMS_SORTABLE` | `spec.declared-sortable-is-sortable` | same for sort |
|
|
2072
|
-
| `SPEC_OVERCLAIMS_SELECTABLE` | `spec.declared-selectable-is-selectable` | same for select |
|
|
2073
|
-
| `FILTER_IN_FIRST_ONLY` | `filter.in-is-union-of-eq` | `in.(a,b)` matches only the first value |
|
|
2074
|
-
| `FILTER_GTE_IS_GT` | `filter.gte-is-gt-or-eq` | `gte` is compiled as `gt` |
|
|
2075
|
-
| `FILTER_ILIKE_IS_LIKE` | `filter.ilike-is-case-insensitive` | `ilike` is compiled as `like` |
|
|
2076
|
-
| `FILTER_IS_NULL_MATCHES_ALL` | `filter.is-null-selects-nulls` | `is.null` matches every row |
|
|
2077
|
-
| `FILTER_ILLEGAL_OP_IGNORED` | `filter.illegal-op-rejected` | an operator outside the allowlist is ignored |
|
|
2078
|
-
| `SORT_NUMERIC_AS_TEXT` | `sort.numeric-order-is-numeric` | a numeric field is ordered lexicographically |
|
|
2079
|
-
| `SORT_MULTI_KEY_IGNORED` | `sort.multi-key-tiebreak` | the second sort key is ignored |
|
|
2080
|
-
| `SELECT_FIELD_MISSING` | `select.requested-fields-present` | a requested select field is dropped |
|
|
2081
|
-
| `CROSS_TENANT_READ` | `tenant.item-not-readable-cross-tenant` | item GET is global by id |
|
|
2082
|
-
| `EXISTENCE_LEAK_VIA_STATUS` | `tenant.denial-does-not-reveal-existence` | 403 vs 404 reveals the other tenant's row |
|
|
2083
|
-
| `TENANT_LEAK_VIA_FILTER` | `tenant.filter-does-not-bypass-scope` | filter drops the tenant predicate |
|
|
2084
|
-
| `ROLE_MONOTONICITY_BROKEN` | `auth.rank-is-monotonic` | a lower rank can do more |
|
|
2085
|
-
| `INVITE_NEVER_GRANTS` | `auth.invite-grants-then-revokes` | accept does not grant |
|
|
2086
|
-
| `REVOKE_IGNORED` | `auth.invite-grants-then-revokes` | revoke leaves the grant |
|
|
2087
|
-
| `IMMUTABLE_WRITABLE` | `patch.immutable-field-rejected` | immutable fields accept writes |
|
|
2088
|
-
| `SOFT_DELETE_LEAK` | `softdelete.absent-from-default-list` | tombstone stays on the default list |
|
|
2089
|
-
| `PARENT_PROJECTION_STALE` | `invalidation.declared-route-changes` | child write does not bump the parent |
|
|
2090
|
-
| `EFFECT_NOT_APPLIED` | `effects.declared-effect-occurs` | declared cardinality delta does not happen |
|
|
2091
|
-
| `ASYNC_NEVER_COMPLETES` | `async.reaches-terminal-state` | job stays pending |
|
|
2092
|
-
| `ASYNC_RECEIPT_MISSING_ID` | `async.receipt-identifies-the-job` | receipt has no id |
|
|
2093
|
-
| `PATCH_REPLACES` | `patch.minimality` | PATCH is implemented as replace |
|
|
2094
|
-
| `IDEMPOTENCY_IGNORED` | `idempotency.replay-does-not-duplicate` | Idempotency-Key is ignored |
|
|
2095
|
-
| `UNIQUE_NOT_ENFORCED` | `create.unique-conflict-rejected` | a duplicate unique-set write is 2xx instead of 409 |
|
|
2096
|
-
| `DELETE_MISSING_OK` | `delete.absent-record-returns-404` | DELETE missing returns 200 |
|
|
2097
|
-
| `CONCURRENT_WRITE_LOST` | `concurrency.no-lost-update` | full-row write clobbers a parallel PATCH |
|
|
2098
|
-
| `ENUM_NOT_VALIDATED` | `validation.enum-enforced` | enum is not enforced |
|
|
2099
|
-
| `MAXLENGTH_NOT_VALIDATED` | `validation.max-length-enforced` | maxLength is not enforced |
|
|
2100
|
-
| `REQUIRED_NOT_VALIDATED` | `validation.required-enforced` | required is not enforced |
|
|
2101
|
-
| `CONTENT_TYPE_NOT_ENFORCED` | `validation.content-type-enforced` | wrong Content-Type is accepted |
|
|
2102
|
-
| `LIST_DETAIL_DISAGREE` | `consistency.projections-agree` | list and item show different values |
|
|
2103
|
-
| `COLUMN_NAME_MISMATCH` | `create.persists-submitted-fields` | SQL identifier does not match the field (SQL backends) |
|
|
2104
|
-
| `COLLATION_INCONSISTENT` | `pagination.cursor-agrees-with-page` | cursor order ≠ page order (SQL) |
|
|
2105
|
-
|
|
2106
|
-
```bash
|
|
2107
|
-
oat serve --defects STALE_LIST,PATCH_REPLACES
|
|
2108
|
-
oat run --config labs/local.config.ts --base-url <url>
|
|
2109
|
-
```
|
|
2110
|
-
|
|
2111
|
-
`COLUMN_NAME_MISMATCH`, `NUMERIC_COMPARED_AS_TEXT`, `COLLATION_INCONSISTENT`, and `CONCURRENT_WRITE_LOST` are SQL-only in conformance (the in-memory store cannot exhibit them).
|
|
2112
|
-
|
|
2113
132
|
## Limits and non-features
|
|
2114
133
|
|
|
2115
134
|
These are deliberate. An agent should not invent a flag for them.
|