ata-validator 1.21.0 → 1.22.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +42 -0
- package/README.md +94 -12
- package/compat.js +12 -1
- package/index.d.ts +31 -0
- package/index.js +46 -17
- package/lib/data-positions.js +10 -0
- package/lib/enrich-error.js +89 -26
- package/lib/formats.js +158 -30
- package/lib/interpreter.js +20 -1
- package/lib/js-compiler.js +53 -2
- package/lib/levenshtein.js +19 -7
- package/lib/plan-compiler.js +38 -0
- package/lib/pointer.js +46 -0
- package/lib/retry-message.js +26 -2
- package/lib/suggestions.js +27 -24
- package/lib/version.js +1 -1
- package/package.json +10 -10
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,48 @@
|
|
|
2
2
|
|
|
3
3
|
All notable changes to ata-validator are documented here. The format follows [Keep a Changelog](https://keepachangelog.com/), and this project adheres to semantic versioning.
|
|
4
4
|
|
|
5
|
+
## 1.22.1 - 2026-09-16
|
|
6
|
+
|
|
7
|
+
### Fixed
|
|
8
|
+
|
|
9
|
+
- `validateJSON()` did not return on a document that ends inside an array. `validateJSON('[')` was one byte, any schema, the default configuration, and it never came back. On a rejection the error carries a caret, so a map from pointer to position is built from the raw text; its array loop asked for the next element without checking that the input had ended, `walk` returned without moving, and nothing else could close the array. `'[}'` hung the same way, because a character that begins no value leaves the position where it was. The object loop escaped only by accident: an unterminated string throws, and `lib/data-position-cache.js` catches throws. It cannot catch a loop. Both container loops now stop at the end of the input, and the array loop stops when an iteration does not advance, so a truncated document still yields a map of what was there.
|
|
10
|
+
- A regression here is an infinite loop, which an ordinary test cannot report because it never reaches a failure. `tests/test_malformed_json_termination.js` runs its corpus in a child process under a watchdog, so a hang becomes a timeout and a red test. The corpus is every prefix of ten documents plus characters inserted where they cannot begin a value: 1793 inputs, 7172 verdicts. It was confirmed to fail before this change and pass after.
|
|
11
|
+
- `tests/fuzz_positions.js` had never fed malformed text to this walker; it fuzzes schema positions with `JSON.stringify` output, which is always well formed. Nothing else changes: on the same five-field schema, `validate()` on a valid document, `validate()` on an invalid one with the errors read, and `validateJSON()` on both are all within measurement noise of 1.22.0.
|
|
12
|
+
|
|
13
|
+
- `unevaluatedProperties: false` reported the boolean schema underneath the keyword instead of the keyword itself: `keyword: 'not'`, empty `params`, and an `instancePath` pointing at the offending member rather than the object holding it. Nothing in that error said which key was unexpected, so a caller branching on `params.unevaluatedProperty` to flag unknown keys saw nothing at all. It now reports `unevaluatedProperties` on the object, with `params.unevaluatedProperty` naming the key, one error per key. `unevaluatedItems: false` had the same shape and now reports one error for the array, with `params.limit` at the first unevaluated position. Both engines were changed together and `tests/test_unevaluated_error_shape.js` holds them to it. Reported by a config linter reading `err.params.unevaluatedProperty` through `ata-validator/compat`.
|
|
14
|
+
- `verbose: true` attached only `parentSchema`. It now also attaches `data`, the value the error points at, and `schema`, the failing keyword's own value, which is the whole set the option is supposed to carry. Without `data` a caller had to walk the document by the instance path itself. The default error shape is unchanged: the two fields appear only when `verbose` is on, including on the errors `ata-validator/compat` synthesises for an `if` wrapper.
|
|
15
|
+
|
|
16
|
+
### Changed
|
|
17
|
+
|
|
18
|
+
- `additionalProperties: false` no longer compiles to a comparison per declared name. It was one comparison per name per key of the document, which is quadratic in the number of properties: a 1000-property schema spent 75 µs on a document where the same schema without the keyword took 3, and doubling the properties quadrupled that. Above 128 declared names the generated code now reads a name lookup built once per compiled function; below that the comparison chain stays, because it allocates nothing and the two are the same within noise there.
|
|
19
|
+
- Cross-process medians, one variant per process, properties of type string: 250 names 8.76 µs to 5.49, 500 names 26.29 to 12.47. With no per-property work, so the membership test is all that is measured: 1000 names 77.3 µs to 18.1, 2000 names 273.6 to 38.9.
|
|
20
|
+
- Measured and not taken: `for...in` over the document instead of an indexed loop over `Object.keys` was about a quarter slower at every size, even though it allocates no key array.
|
|
21
|
+
- `tests/test_additional_properties_scaling.js` guards this structurally, on the emitted source: the lookup is present above the threshold, absent below it, and declared outside the function that runs per call. It asserts no timing, because a wall-clock ratio was not a guard here: the quadratic shape measured 7.7x for four times the properties on the machine the test was written on, where a CI runner measured 8.3x for the fixed code. The comparison chain short-circuits as soon as a key matches, so the ratio never separated the two. Found by a config linter whose schema is generated from a project's environment variables, so its property count is whatever the project has.
|
|
22
|
+
|
|
23
|
+
## 1.22.0 - 2026-09-14
|
|
24
|
+
|
|
25
|
+
### Fixed
|
|
26
|
+
|
|
27
|
+
- A schema validated once with `isValidObject()` lost its generated error function for the rest of the process. The verdict-only fast path compiles that one function and seeds the shared compile cache with the other two set to null, meaning "not built yet"; the full compile read those nulls as "the compiler declined this schema" and never tried again, so errors fell back to the interpreted engine or the addon. Every server hits that order, because most documents are valid. Cache entries now record whether they are complete. Verdicts and error content were never affected, and `tests/test_compile_cache_order.js` runs both orders in separate processes and holds the engine and the error list equal.
|
|
28
|
+
- Measured on a five-field schema with an enum and a format, after validating one valid document: `validate(bad).errors` 3.89 µs to 0.66 µs, the Standard Schema path 3.52 µs to 0.07 µs. With no addon installed at all (`ATA_NO_NATIVE=1`), where the fallback was the interpreter rather than the addon, 1.05 µs to 0.66 µs and 0.42 µs to 0.07 µs.
|
|
29
|
+
- `engine()` reported `closure` for a schema that compiles, once a valid document had been validated first. It now reports `codegen` in either order.
|
|
30
|
+
|
|
31
|
+
### Changed
|
|
32
|
+
|
|
33
|
+
- The official test suite submodule moved 60 commits forward, to 2026-09-13. All three dialects stay at zero regressions and the case counts rose: Draft 2020-12 1299 to 1301, draft 7 927 to 929, the v1 dialect 1133 to 1135, identical with code generation blocked. The buffer path agrees with `validate()` on all 3365 of them. Every figure in the README, the docs and the contributor notes was remeasured and updated together, and the floors in `tests/test_no_eval.js` were raised so the new cases cannot be lost silently.
|
|
34
|
+
|
|
35
|
+
- `uri` is one walk over the string instead of three, and reads its character classes out of tables instead of comparison chains. It was the most expensive thing an ordinary document carried: on a schema with five formats over a 1.6 KB payload, format assertion was 94 percent of the validation cost and `uri` was most of that, while the structural check of the whole nested document was 43 ns. The authority's landmarks, the last `@`, the colons after it and whether a bracket came before it, are now recorded during the character scan rather than by cutting the authority out with `slice` and asking the copy for `lastIndexOf` and two regular expressions.
|
|
36
|
+
- `format: uri` on one string, through a compiled validator: 92 to 42 ns. The same product schema's document: 697 to 423 ns. End to end, where `JSON.parse` is the other 75 percent: 2.98 to 2.69 µs.
|
|
37
|
+
- Measured, and not taken: nine `===` tests per character cost more than the loop they guarded, a lookup table is 2.3x that chain, and a three-regex formulation of the whole rule landed at 65 ns against the table walk's 41.
|
|
38
|
+
- The code generator hoists the walk once per compiled function and calls it, rather than inlining it at each call site where the tables would have to be rebuilt. This is the shape `email` already had. `tests/test_uri_helper_parity.js` holds the function, the generated text and a compiled validator to the same answer over 35,677 strings built from the walk's own boundaries.
|
|
39
|
+
|
|
40
|
+
- Enriching a rejection no longer reads more of the document than the diagnostic it produces. `received` describes the offending value in at most 60 characters, and it used to find out whether a value fitted by serialising it whole, so a `required` error on a large payload cost a `JSON.stringify` of that payload, once per error. Sizing is now bounded by those 60 characters and stops as soon as they are spent. Enriching one error on a document holding a thousand rows: 61.5 µs to 160 ns, and flat in the size of the document rather than linear. On the schema-benchmarks product schema, where `validate()` reports 16 errors, reading `.errors` went from 9.84 to 5.10 µs (48 percent less) on the same machine, separate processes. Verdict paths, `isValidObject()` and the Standard Schema bridge are unchanged.
|
|
41
|
+
- An object too large to show reports its shape rather than its serialised size: `[object, 5 keys]` where it used to say `[object, ~0.1KB]`. This is the only change to error output; every value that printed its body before prints the same body now, including nested objects, arrays and values with their own `toJSON`.
|
|
42
|
+
- The error's value is resolved from its pointer once and handed to both the `received` repr and the suggestion sources. `lib/suggestions.js` had a second pointer walk that split the string and mapped over the segments; both paths now use `resolvePointer` in `lib/pointer.js`.
|
|
43
|
+
- A `required` typo hint is offered on containers of up to 64 keys. Past that the nearest key stops being evidence of a typo, which is the cap `suggestEnumTypo` already applied to its candidate list, and the scan is a distance computation per key.
|
|
44
|
+
- `levenshtein` reads characters by code and swaps its two rows through a temporary. The destructured swap it used built an array per row. About 9 percent faster on the key pairs the suggestion sources compare.
|
|
45
|
+
- `tests/test_enrich_received.js` holds enrichment to within 4x across a hundredfold increase in document volume. Before this change the same measurement was 122x.
|
|
46
|
+
|
|
5
47
|
## 1.17.2 - 2026-09-13
|
|
6
48
|
|
|
7
49
|
### Changed
|
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# ata-validator
|
|
2
2
|
|
|
3
|
-
JSON Schema validation
|
|
3
|
+
JSON Schema validation that compiles for speed and still runs where code generation is blocked. The compiled and interpreted engines answer identically, at 100% of the official suite in both modes, across Draft 2020-12, draft 7 and the [JSON Schema v1 dialect](#dialects). First-class TypeScript inference, and `ata build` emits a standalone module that imports nothing.
|
|
4
4
|
|
|
5
5
|
[](https://www.npmjs.com/package/ata-validator)
|
|
6
6
|
[](LICENSE)
|
|
@@ -20,11 +20,11 @@ The `ata-validator` package itself is pure JavaScript. The native accelerator (s
|
|
|
20
20
|
npm install ata-validator --omit=optional
|
|
21
21
|
```
|
|
22
22
|
|
|
23
|
-
or set `ATA_NO_NATIVE=1` at runtime. Typical schemas compile to specialized JS; shapes the compiler cannot represent (some `$dynamicRef`, cyclic `$ref`, unusual keyword interactions) fall back to an interpreted engine, so every schema validates in every environment. The pure-JS setup scores the same on the official suite as the native one,
|
|
23
|
+
or set `ATA_NO_NATIVE=1` at runtime. Typical schemas compile to specialized JS; shapes the compiler cannot represent (some `$dynamicRef`, cyclic `$ref`, unusual keyword interactions) fall back to an interpreted engine, so every schema validates in every environment. The pure-JS setup scores the same on the official suite as the native one, 1301 of 1301 Draft 2020-12 cases. Only the buffer and parallel APIs (`isValid` on raw buffers, `countValid`, `batchIsValid`, `validateAndParse`) need the native engine and say so with a clear error.
|
|
24
24
|
|
|
25
|
-
Those four now agree with `validate()` on every case of the official suite,
|
|
25
|
+
Those four now agree with `validate()` on every case of the official suite, 3365 across three dialects. The native walker behind them does not handle every shape (`contains`, `unevaluatedProperties`, `patternProperties`, tuple `items`, cross-document `$ref`, a few formats), so for schemas using one of those the buffer APIs parse the bytes and answer through `validate()`; the list is in `lib/buffer-gate.js`. Typical request schemas stay on the zero-copy path. `npm test` holds the disagreement count at zero.
|
|
26
26
|
|
|
27
|
-
Where `new Function` is refused altogether, on Cloudflare Workers, Deno Deploy or under a strict Content-Security-Policy, ata drops to the interpreted engine and scores the same
|
|
27
|
+
Where `new Function` is refused altogether, on Cloudflare Workers, Deno Deploy or under a strict Content-Security-Policy, ata drops to the interpreted engine and scores the same 1301 of 1301 with code generation blocked. No flags, and on Workers no `nodejs_compat` either. See [docs/edge-runtimes.md](docs/edge-runtimes.md).
|
|
28
28
|
|
|
29
29
|
Node has a switch for exactly that environment, so this takes thirty seconds to check for yourself, on ata or on whatever you use today:
|
|
30
30
|
|
|
@@ -97,6 +97,55 @@ run on the interpreted engine. That engine compiles each schema into a tree of c
|
|
|
97
97
|
on a six-field object schema a passing payload costs about 136 ns against 8 ns for the same
|
|
98
98
|
shape on the compiled path, both measured warm in one run on the same machine.
|
|
99
99
|
|
|
100
|
+
## Schemas and models
|
|
101
|
+
|
|
102
|
+
If you ask a model for structured output and validate the result, the schema is
|
|
103
|
+
part of the prompt and the validation error is part of the retry. Two functions
|
|
104
|
+
produce those strings.
|
|
105
|
+
|
|
106
|
+
```js
|
|
107
|
+
import { describeSchema, toRetryMessage, Validator } from 'ata-validator'
|
|
108
|
+
|
|
109
|
+
describeSchema(schema)
|
|
110
|
+
// output: object
|
|
111
|
+
// status: one of "AWAITING_CLEARANCE", "PART_SETTLED", "CLOSED_OUT"
|
|
112
|
+
// region_code: one of "R11", "R24", "R37", "R52"
|
|
113
|
+
// amount: integer, at least 1
|
|
114
|
+
// no other fields
|
|
115
|
+
|
|
116
|
+
const r = new Validator(schema).validate(fromTheModel)
|
|
117
|
+
if (!r.valid) toRetryMessage(r.errors)
|
|
118
|
+
// /status: expected one of ["AWAITING_CLEARANCE", "PART_SETTLED", "CLOSED_OUT"], found "partial"
|
|
119
|
+
// /region_code: expected one of ["R11", "R24", "R37", "R52"], found "HH"
|
|
120
|
+
// /amount: expected ≥1, found 0
|
|
121
|
+
// /: unknown property "extra"
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
Whether this is worth anything depends on the schema, and the measurement that
|
|
125
|
+
says so is
|
|
126
|
+
[public](https://github.com/mertcanaltin/retry-message-experiment), with the
|
|
127
|
+
harness and the raw results.
|
|
128
|
+
|
|
129
|
+
On constraints a model can work out from the document, a pattern, a date format,
|
|
130
|
+
a currency in uppercase, it changes nothing: 40 of 40 documents recovered on one
|
|
131
|
+
retry with the conventional error text, and 40 of 40 with the detailed one.
|
|
132
|
+
|
|
133
|
+
On constraints it cannot work out, an `enum` of internal codes while the
|
|
134
|
+
document says "paid in part" and "Hamburg office", the conventional text
|
|
135
|
+
recovered 0 of 30 and the detailed text 30 of 30. The model does not give up
|
|
136
|
+
when it is not told the values, it invents plausible ones, so no number of
|
|
137
|
+
retries closes it.
|
|
138
|
+
|
|
139
|
+
Stating the constraints up front does the same job earlier. Across four prompts
|
|
140
|
+
on the same fixtures, first attempts that validate: nothing 0 of 30, a lean
|
|
141
|
+
field list 0 of 30, a careful hand-written description 30 of 30 on the inferable
|
|
142
|
+
fixture and 0 of 30 on the opaque one, and `describeSchema` 30 and 23. A person
|
|
143
|
+
writing prose matches the generated description wherever prose works. It is the
|
|
144
|
+
enumerated values a person leaves out.
|
|
145
|
+
|
|
146
|
+
One model, Claude Haiku 4.5, one schema shape, 30 to 40 documents per arm, no
|
|
147
|
+
temperature control. Read it as a finding, not a law.
|
|
148
|
+
|
|
100
149
|
## Error messages
|
|
101
150
|
|
|
102
151
|
ata's error output is compiler-grade: each error carries a stable code, an inline source frame pointing at the schema file, and another pointing at the offending bytes in the request payload. Renderers ship in three styles:
|
|
@@ -153,6 +202,33 @@ Schemas without an `errorMessage` keyword pay nothing: the override pass is only
|
|
|
153
202
|
|
|
154
203
|
For consumers who built log dashboards on the v0.14 error shape, `new Validator(schema, { richErrors: false })` returns the legacy shape exactly. For high-throughput paths, `abortEarly: true` continues to short-circuit; the returned error carries `code: 'ATA9000'` and no enrichment.
|
|
155
204
|
|
|
205
|
+
## Which path, and where
|
|
206
|
+
|
|
207
|
+
There are two ways to run ata and they suit different places. Measuring the wrong one
|
|
208
|
+
is the most common way to get a misleading number out of this library.
|
|
209
|
+
|
|
210
|
+
| | compiled with `ata build` | runtime `new Validator(schema)` |
|
|
211
|
+
|---|---|---|
|
|
212
|
+
| In a bundle, gzipped | **1.9 KB** | 75.6 KB |
|
|
213
|
+
| Time to a served request | **3.5 ms** | 10.7 ms |
|
|
214
|
+
| Schema known when | build time | any time |
|
|
215
|
+
|
|
216
|
+
The bundle row is a ten-field user schema built with
|
|
217
|
+
`bun build --minify --target=browser`. The startup row is a Hono route on Bun 1.4, best
|
|
218
|
+
of seven, against 3.6 ms for the same app doing no validation at all, so the compiled
|
|
219
|
+
path costs nothing measurable to start. The runtime
|
|
220
|
+
figure is what it is because a schema that arrives at run time can use any keyword, so
|
|
221
|
+
the whole engine has to be there. The compiled module imports nothing and contains only
|
|
222
|
+
the checks your schema asks for.
|
|
223
|
+
|
|
224
|
+
**On a server, use whichever fits your schemas.** 76 KB of JavaScript on a Node or Bun
|
|
225
|
+
process is not a cost anyone notices, and the runtime API is the simpler thing to reach
|
|
226
|
+
for. Speed is the same either way once warm.
|
|
227
|
+
|
|
228
|
+
**In a browser, on an edge runtime, or anywhere cold starts are charged, compile.** This
|
|
229
|
+
is where the difference is the whole story, and it is also where `new Function` is often
|
|
230
|
+
blocked outright, which the compiled module does not need.
|
|
231
|
+
|
|
156
232
|
## When to use the runtime API instead
|
|
157
233
|
|
|
158
234
|
`ata build` is for schemas you know at build time. If your schemas are user-supplied at runtime (form builders, no-code platforms, dynamic API ingestion), use the runtime API:
|
|
@@ -374,7 +450,7 @@ const v = new Validator(schema, {
|
|
|
374
450
|
|
|
375
451
|
### Build-time compile (`ata compile`)
|
|
376
452
|
|
|
377
|
-
The `ata` CLI turns a JSON Schema file into a self-contained JavaScript module. No runtime dependency on `ata-validator`, so only the generated validator ships to the browser. Typical output is
|
|
453
|
+
The `ata` CLI turns a JSON Schema file into a self-contained JavaScript module. No runtime dependency on `ata-validator`, so only the generated validator ships to the browser. Typical output is about 2 KB gzipped for a ten-field schema, against 74 KB for the runtime bundled for the browser.
|
|
378
454
|
|
|
379
455
|
```bash
|
|
380
456
|
npx ata compile schemas/user.json -o src/generated/user.validator.mjs
|
|
@@ -404,7 +480,7 @@ CLI options:
|
|
|
404
480
|
| `-o, --output <file>` | `<schema>.validator.mjs` | Output path |
|
|
405
481
|
| `-f, --format <fmt>` | `esm` | `esm` or `cjs` |
|
|
406
482
|
| `--name <TypeName>` | from filename | Root type name in the `.d.ts` |
|
|
407
|
-
| `--abort-early` | off | Generate the stub-error variant
|
|
483
|
+
| `--abort-early` | off | Generate the stub-error variant, about a third of the source |
|
|
408
484
|
| `--no-types` | off | Skip the `.d.mts` / `.d.cts` output |
|
|
409
485
|
|
|
410
486
|
For a project with many schemas, `ata build <glob>` compiles them all in one command:
|
|
@@ -415,13 +491,19 @@ npx ata build 'schemas/*.json' --out-dir build/validators --check
|
|
|
415
491
|
|
|
416
492
|
Run with `--watch` during development for incremental rebuilds.
|
|
417
493
|
|
|
418
|
-
|
|
494
|
+
Bundle sizes for a 10-field user schema, minified and gzipped, measured with
|
|
495
|
+
`bun build --minify --target=browser`:
|
|
419
496
|
|
|
420
|
-
|
|
|
497
|
+
| What the app imports | Size | Notes |
|
|
421
498
|
|---|---|---|
|
|
422
|
-
| `ata-validator`
|
|
423
|
-
| `
|
|
424
|
-
| `
|
|
499
|
+
| `Validator` from `ata-validator` | 74.1 KB | The compiler ships with it, because a runtime schema can use any keyword |
|
|
500
|
+
| `isValid` from the compiled module | **1.9 KB** | Nothing else is reachable, so the error collector is dropped |
|
|
501
|
+
| `validate` from the compiled module | **3.6 KB** | Adds the detailed error collector |
|
|
502
|
+
|
|
503
|
+
`--abort-early` makes the generated source about three times smaller, and after
|
|
504
|
+
bundling it makes no difference: importing only `isValid` already leaves the error
|
|
505
|
+
collector unreachable, and a bundler drops it. Use the flag to cut the file on disk,
|
|
506
|
+
not to cut what ships.
|
|
425
507
|
|
|
426
508
|
Programmatic API if you prefer to script it:
|
|
427
509
|
|
|
@@ -560,7 +642,7 @@ Both are implemented in the interpreted engine, so a v1 schema that uses `$dynam
|
|
|
560
642
|
|
|
561
643
|
### Known limitations
|
|
562
644
|
|
|
563
|
-
Running the whole Draft 2020-12 suite with nothing excluded, `format` and `default` under specification semantics (`assertFormat: false`, `useDefaults: false`), gives
|
|
645
|
+
Running the whole Draft 2020-12 suite with nothing excluded, `format` and `default` under specification semantics (`assertFormat: false`, `useDefaults: false`), gives 1301 of 1301 cases. Draft 7 gives 929 of 929. The v1 dialect gives 1135 of 1135. `npm run test:suite` reproduces all three.
|
|
564
646
|
|
|
565
647
|
Areas that remain deliberate scope decisions for 1.x:
|
|
566
648
|
|
package/compat.js
CHANGED
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
'use strict';
|
|
2
2
|
|
|
3
|
+
const { resolvePointer } = require('./lib/pointer.js');
|
|
4
|
+
|
|
3
5
|
// Drop-in for the default validator's class. `const Ajv = require('ata-validator/compat')`
|
|
4
6
|
// and the rest of the file stays as it was: `new Ajv(opts)`, `compile`,
|
|
5
7
|
// `validate`, `addSchema`, `getSchema`, `removeSchema`, `addFormat`,
|
|
@@ -174,7 +176,16 @@ class Ata {
|
|
|
174
176
|
validate.errors = null;
|
|
175
177
|
return true;
|
|
176
178
|
}
|
|
177
|
-
|
|
179
|
+
let errors = shaper === null ? result.errors : shaper(result.errors, data);
|
|
180
|
+
// Errors the shaper synthesises, the `if` wrapper among them, are built
|
|
181
|
+
// from the group rather than carried up from the validator, so verbose
|
|
182
|
+
// mode has to fill in the value they point at here or they would be the
|
|
183
|
+
// only ones missing it.
|
|
184
|
+
if (this.opts.verbose) {
|
|
185
|
+
errors = errors.map((e) => (e && !('data' in e)
|
|
186
|
+
? { ...e, data: resolvePointer(data, e.instancePath || '', undefined) }
|
|
187
|
+
: e));
|
|
188
|
+
}
|
|
178
189
|
validate.errors = shaper === null && !allErrors ? errors.slice(0, 1) : errors;
|
|
179
190
|
return false;
|
|
180
191
|
};
|
package/index.d.ts
CHANGED
|
@@ -72,6 +72,37 @@ export function renderPretty(errors: RichValidationError[], opts?: PrettyOptions
|
|
|
72
72
|
export function renderCompact(errors: RichValidationError[], opts?: CompactOptions): string;
|
|
73
73
|
export function renderJSON(errors: RichValidationError[], opts?: JSONRenderOptions): string;
|
|
74
74
|
|
|
75
|
+
/** Options for {@link describeSchema}. */
|
|
76
|
+
export interface DescribeSchemaOptions {
|
|
77
|
+
/** What to call the top level in the description. Defaults to `output`. */
|
|
78
|
+
name?: string;
|
|
79
|
+
}
|
|
80
|
+
|
|
81
|
+
/**
|
|
82
|
+
* The schema's constraints as prose, for the prompt that asks a model to
|
|
83
|
+
* produce a document. Values a model cannot infer, an `enum` of internal codes
|
|
84
|
+
* most of all, are the ones worth stating: measured on one fixture, naming them
|
|
85
|
+
* took first-attempt validity from 0 of 30 to 23 of 30, where a careful
|
|
86
|
+
* hand-written description scored 0 on exactly those fields.
|
|
87
|
+
* https://github.com/mertcanaltin/retry-message-experiment
|
|
88
|
+
*/
|
|
89
|
+
export function describeSchema(schema: object, opts?: DescribeSchemaOptions): string;
|
|
90
|
+
|
|
91
|
+
/** Options for {@link toRetryMessage}. */
|
|
92
|
+
export interface RetryMessageOptions {
|
|
93
|
+
/** Most errors to include. Defaults to 20, because a model does not act on a long list. */
|
|
94
|
+
limit?: number;
|
|
95
|
+
}
|
|
96
|
+
|
|
97
|
+
/**
|
|
98
|
+
* The errors as a string to send back to a model, naming what was expected and
|
|
99
|
+
* what arrived. On constraints a model can infer from context this changes
|
|
100
|
+
* nothing; where a constraint is an internal vocabulary it was the difference
|
|
101
|
+
* between 0 of 30 and 30 of 30 recovered on one retry.
|
|
102
|
+
* https://github.com/mertcanaltin/retry-message-experiment
|
|
103
|
+
*/
|
|
104
|
+
export function toRetryMessage(errors: RichValidationError[], opts?: RetryMessageOptions): string;
|
|
105
|
+
|
|
75
106
|
/** A user-supplied format checker. Receives the candidate value, returns true if valid. */
|
|
76
107
|
export type FormatChecker = (value: string) => boolean;
|
|
77
108
|
|
package/index.js
CHANGED
|
@@ -458,13 +458,17 @@ Object.defineProperty(RichRejection.prototype, 'errors', {
|
|
|
458
458
|
const enrich = this._enrich;
|
|
459
459
|
let raw = this._result.errors || [];
|
|
460
460
|
if (raw.length > 1) raw = sortErrorsBySchemaOrder(this._root, raw);
|
|
461
|
-
|
|
462
|
-
|
|
461
|
+
// One options object for the whole list, not one per error.
|
|
462
|
+
const opts = enrich && raw.length
|
|
463
|
+
? {
|
|
463
464
|
data: this._data,
|
|
464
465
|
positions: this._positions,
|
|
465
466
|
schemaPositions: self._schemaPositions,
|
|
466
467
|
schemaFile: self._source ? self._source.path : undefined,
|
|
467
|
-
}
|
|
468
|
+
}
|
|
469
|
+
: null;
|
|
470
|
+
const cached = opts
|
|
471
|
+
? raw.map((e) => enrich(e, opts))
|
|
468
472
|
// The v0.14 shape is a fixed key set; the ordering key the
|
|
469
473
|
// generated code carries is dropped from it here.
|
|
470
474
|
: raw.map(stripOrdinal);
|
|
@@ -1005,7 +1009,12 @@ class Validator {
|
|
|
1005
1009
|
// there.
|
|
1006
1010
|
if (this._v1Dynamic || this._usesKeywords || !codegenAvailable()) {
|
|
1007
1011
|
jsFn = null; jsCombinedFn = null; jsErrFn = null;
|
|
1008
|
-
|
|
1012
|
+
// `full` separates an entry that holds every compiled function from one
|
|
1013
|
+
// the verdict-only fast path seeded, where `combined` and `errFn` are null
|
|
1014
|
+
// because nothing has tried to build them yet. Both halves of that
|
|
1015
|
+
// distinction are null, and reading the second as the first costs this
|
|
1016
|
+
// schema its generated error function for the life of the process.
|
|
1017
|
+
} else if (cached && cached.full && !_forceNapi) {
|
|
1009
1018
|
jsFn = cached.jsFn;
|
|
1010
1019
|
jsCombinedFn = cached.combined;
|
|
1011
1020
|
jsErrFn = cached.errFn;
|
|
@@ -1019,13 +1028,13 @@ class Validator {
|
|
|
1019
1028
|
_isCodegen = !!_cgFn;
|
|
1020
1029
|
this._engine = _cgFn ? 'codegen' : jsFn ? 'closure' : null;
|
|
1021
1030
|
if (!uf) {
|
|
1022
|
-
_compileCache.set(mapKey, { jsFn, combined: jsCombinedFn, errFn: jsErrFn, isCodegen: _isCodegen });
|
|
1031
|
+
_compileCache.set(mapKey, { jsFn, combined: jsCombinedFn, errFn: jsErrFn, isCodegen: _isCodegen, full: true });
|
|
1023
1032
|
}
|
|
1024
1033
|
} else {
|
|
1025
1034
|
jsFn = null; jsCombinedFn = null; jsErrFn = null;
|
|
1026
1035
|
}
|
|
1027
1036
|
this._jsFn = jsFn;
|
|
1028
|
-
if (this._engine === undefined) this._engine = cached ? (cached.isCodegen ? 'codegen' : jsFn ? 'closure' : null) : null;
|
|
1037
|
+
if (this._engine === undefined) this._engine = (cached && cached.full) ? (cached.isCodegen ? 'codegen' : jsFn ? 'closure' : null) : null;
|
|
1029
1038
|
|
|
1030
1039
|
// Data mutators -- try codegen first (12x faster), fallback to closure arrays.
|
|
1031
1040
|
// Follow cross-refs so coercion/defaults/removeAdditional see the referenced
|
|
@@ -1224,19 +1233,36 @@ class Validator {
|
|
|
1224
1233
|
}
|
|
1225
1234
|
: (data) => (jsFn(data) ? VALID_RESULT : errFn(data));
|
|
1226
1235
|
}
|
|
1227
|
-
// Verbose mode: populate parentSchema on each error
|
|
1228
|
-
//
|
|
1236
|
+
// Verbose mode: populate parentSchema, schema and data on each error, the
|
|
1237
|
+
// three fields the default error shape carries under the same option.
|
|
1238
|
+
// `data` is the value the error points at; without it a caller has to
|
|
1239
|
+
// walk the document by the instance path itself, which is what one
|
|
1240
|
+
// migration ended up writing by hand. Errors may be frozen, so clone
|
|
1241
|
+
// them with the extra fields.
|
|
1229
1242
|
if (this._verbose) {
|
|
1230
1243
|
const inner = this.validate;
|
|
1231
1244
|
const root = this._schemaObj;
|
|
1245
|
+
const { resolvePointer } = require('./lib/pointer.js');
|
|
1232
1246
|
this.validate = (data) => {
|
|
1233
1247
|
const result = inner(data);
|
|
1234
1248
|
if (result && !result.valid && result.errors) {
|
|
1235
|
-
const enriched = result.errors.map((err) =>
|
|
1236
|
-
err
|
|
1237
|
-
|
|
1238
|
-
|
|
1239
|
-
|
|
1249
|
+
const enriched = result.errors.map((err) => {
|
|
1250
|
+
if (!err || err.parentSchema !== undefined) return err;
|
|
1251
|
+
const parentSchema = resolveSchemaByPath(root, err.schemaPath);
|
|
1252
|
+
// The last segment of the schema path is the keyword that
|
|
1253
|
+
// failed, so its value on the parent is that keyword's schema.
|
|
1254
|
+
const sp = typeof err.schemaPath === 'string' ? err.schemaPath : '';
|
|
1255
|
+
const last = sp.slice(sp.lastIndexOf('/') + 1).replace(/~1/g, '/').replace(/~0/g, '~');
|
|
1256
|
+
const keywordSchema = (parentSchema !== null && typeof parentSchema === 'object' && last)
|
|
1257
|
+
? parentSchema[last]
|
|
1258
|
+
: undefined;
|
|
1259
|
+
return {
|
|
1260
|
+
...err,
|
|
1261
|
+
parentSchema,
|
|
1262
|
+
schema: keywordSchema,
|
|
1263
|
+
data: resolvePointer(data, err.instancePath, undefined),
|
|
1264
|
+
};
|
|
1265
|
+
});
|
|
1240
1266
|
return { valid: false, errors: enriched };
|
|
1241
1267
|
}
|
|
1242
1268
|
return result;
|
|
@@ -1549,12 +1575,13 @@ class Validator {
|
|
|
1549
1575
|
// Declaration order, as validate() applies it; the text path
|
|
1550
1576
|
// used to enrich in emission order.
|
|
1551
1577
|
const ordered = result.errors.length > 1 ? sortErrorsBySchemaOrder(this._schemaObj, result.errors) : result.errors;
|
|
1552
|
-
const
|
|
1578
|
+
const enrichOpts = {
|
|
1553
1579
|
data: parsedData,
|
|
1554
1580
|
positions,
|
|
1555
1581
|
schemaPositions: this._schemaPositions,
|
|
1556
1582
|
schemaFile: this._source ? this._source.path : undefined,
|
|
1557
|
-
}
|
|
1583
|
+
};
|
|
1584
|
+
const enriched = ordered.map((e) => enrich(e, enrichOpts));
|
|
1558
1585
|
if (enriched.length > 1) attachRelated(enriched);
|
|
1559
1586
|
attachDiagnosticSource(enriched, {
|
|
1560
1587
|
data: parsedData,
|
|
@@ -1776,9 +1803,11 @@ class Validator {
|
|
|
1776
1803
|
this._jsFn = jsFn;
|
|
1777
1804
|
if (jsFn) {
|
|
1778
1805
|
this.isValidObject = jsFn;
|
|
1779
|
-
//
|
|
1806
|
+
// A partial entry: the verdict function is real, the other two are not
|
|
1807
|
+
// built yet rather than declined. `full: false` says so, so the next
|
|
1808
|
+
// caller that needs errors compiles them instead of inheriting nulls.
|
|
1780
1809
|
if (!uf) {
|
|
1781
|
-
if (!cached) _compileCache.set(mapKey, { jsFn, combined: null, errFn: null });
|
|
1810
|
+
if (!cached) _compileCache.set(mapKey, { jsFn, combined: null, errFn: null, full: false });
|
|
1782
1811
|
else cached.jsFn = jsFn;
|
|
1783
1812
|
}
|
|
1784
1813
|
}
|
package/lib/data-positions.js
CHANGED
|
@@ -63,6 +63,10 @@ function buildDataPositionMap (input) {
|
|
|
63
63
|
i++;
|
|
64
64
|
while (true) {
|
|
65
65
|
skipWs();
|
|
66
|
+
// The input is not required to be valid JSON: this map is built to
|
|
67
|
+
// put a caret on a syntax error, so a document that stops in the
|
|
68
|
+
// middle of a container is the normal case, not an impossible one.
|
|
69
|
+
if (i >= n) break;
|
|
66
70
|
if (text.charCodeAt(i) === 0x7d) { i++; break; }
|
|
67
71
|
if (text.charCodeAt(i) === 0x2c) { i++; continue; }
|
|
68
72
|
skipWs();
|
|
@@ -88,9 +92,15 @@ function buildDataPositionMap (input) {
|
|
|
88
92
|
let idx = 0;
|
|
89
93
|
while (true) {
|
|
90
94
|
skipWs();
|
|
95
|
+
if (i >= n) break;
|
|
91
96
|
if (text.charCodeAt(i) === 0x5d) { i++; break; }
|
|
92
97
|
if (text.charCodeAt(i) === 0x2c) { i++; continue; }
|
|
98
|
+
const before = i;
|
|
93
99
|
walk(path.concat([String(idx)]));
|
|
100
|
+
// A character that begins no value at all, a stray `}` for instance,
|
|
101
|
+
// leaves the position where it was, and a loop that does not advance
|
|
102
|
+
// does not end. `validateJSON('[')` hung here on one byte.
|
|
103
|
+
if (i === before) break;
|
|
94
104
|
idx++;
|
|
95
105
|
}
|
|
96
106
|
} else if (ch === 0x22) {
|
package/lib/enrich-error.js
CHANGED
|
@@ -2,9 +2,70 @@
|
|
|
2
2
|
|
|
3
3
|
const { CODES, codeFor, fromNative } = require('./error-codes');
|
|
4
4
|
const { suggestFor } = require('./suggestions');
|
|
5
|
+
const { resolvePointer } = require('./pointer');
|
|
6
|
+
|
|
7
|
+
// A pointer that walked out of the document: no value to report, not even the
|
|
8
|
+
// word 'undefined', which is what an absent key reports.
|
|
9
|
+
const MISSING = Symbol('ata.received.missing');
|
|
5
10
|
|
|
6
11
|
const DOC_BASE = 'https://ata-validator.com/e/';
|
|
7
12
|
|
|
13
|
+
// The number of characters JSON.stringify would emit for `v`, or -1 as soon as
|
|
14
|
+
// that passes `budget` or the value holds something a plain walk cannot size.
|
|
15
|
+
// The point is the bound: `reprValue` only ever shows a body of 60 characters,
|
|
16
|
+
// so sizing one never needs to read further than that, and a `required` error
|
|
17
|
+
// on a large document used to serialise the whole container to find out it was
|
|
18
|
+
// too big. Keys are counted unescaped, which makes the result a lower bound,
|
|
19
|
+
// and the caller rechecks the real length on the string it ends up building.
|
|
20
|
+
function jsonSizeWithin (v, budget) {
|
|
21
|
+
if (budget < 0) return -1;
|
|
22
|
+
if (v === null) return 4;
|
|
23
|
+
const t = typeof v;
|
|
24
|
+
if (t === 'number') return Number.isFinite(v) ? String(v).length : 4;
|
|
25
|
+
if (t === 'boolean') return v ? 4 : 5;
|
|
26
|
+
if (t === 'string') return v.length + 2 > budget ? -1 : v.length + 2;
|
|
27
|
+
if (t !== 'object') return -1;
|
|
28
|
+
// A value with its own toJSON serialises to something only that method can
|
|
29
|
+
// say. Sizing it exactly would mean running it, and a Date's toJSON alone
|
|
30
|
+
// costs more than serialising the object around it, so this returns the
|
|
31
|
+
// smallest thing it could produce. Undershooting is safe: the caller only
|
|
32
|
+
// uses the estimate to decide whether to try, and rechecks the real length.
|
|
33
|
+
if (typeof v.toJSON === 'function') return 2;
|
|
34
|
+
if (Array.isArray(v)) {
|
|
35
|
+
let n = 2;
|
|
36
|
+
for (let i = 0; i < v.length; i++) {
|
|
37
|
+
n += i === 0 ? 0 : 1;
|
|
38
|
+
if (n > budget) return -1;
|
|
39
|
+
const c = jsonSizeWithin(v[i], budget - n);
|
|
40
|
+
if (c < 0) return -1;
|
|
41
|
+
n += c;
|
|
42
|
+
if (n > budget) return -1;
|
|
43
|
+
}
|
|
44
|
+
return n;
|
|
45
|
+
}
|
|
46
|
+
const proto = Object.getPrototypeOf(v);
|
|
47
|
+
if (proto !== Object.prototype && proto !== null) return -1;
|
|
48
|
+
let n = 2;
|
|
49
|
+
let first = true;
|
|
50
|
+
// for-in rather than Object.keys: the prototype is checked above, so there
|
|
51
|
+
// is nothing inherited to enumerate, and this walk runs per error.
|
|
52
|
+
for (const k in v) {
|
|
53
|
+
const val = v[k];
|
|
54
|
+
// Keys JSON drops cost nothing, but they also cost nothing to skip.
|
|
55
|
+
if (val === undefined || typeof val === 'function' || typeof val === 'symbol') continue;
|
|
56
|
+
// Charge the key before reading the value, so a budget already spent
|
|
57
|
+
// returns without descending into it.
|
|
58
|
+
n += k.length + 3 + (first ? 0 : 1);
|
|
59
|
+
if (n > budget) return -1;
|
|
60
|
+
first = false;
|
|
61
|
+
const c = jsonSizeWithin(val, budget - n);
|
|
62
|
+
if (c < 0) return -1;
|
|
63
|
+
n += c;
|
|
64
|
+
if (n > budget) return -1;
|
|
65
|
+
}
|
|
66
|
+
return n;
|
|
67
|
+
}
|
|
68
|
+
|
|
8
69
|
function reprValue (v) {
|
|
9
70
|
if (v === undefined) return 'undefined';
|
|
10
71
|
if (v === null) return 'null';
|
|
@@ -16,13 +77,21 @@ function reprValue (v) {
|
|
|
16
77
|
if (t === 'number' || t === 'boolean') return String(v);
|
|
17
78
|
if (Array.isArray(v)) return `[array, ${v.length} items]`;
|
|
18
79
|
if (t === 'object') {
|
|
80
|
+
if (jsonSizeWithin(v, 60) >= 0) {
|
|
81
|
+
try {
|
|
82
|
+
const s = JSON.stringify(v);
|
|
83
|
+
if (s !== undefined && s.length <= 60) return s;
|
|
84
|
+
} catch {
|
|
85
|
+
return '[object, unserializable]';
|
|
86
|
+
}
|
|
87
|
+
}
|
|
88
|
+
let n;
|
|
19
89
|
try {
|
|
20
|
-
|
|
21
|
-
if (s.length <= 60) return s;
|
|
22
|
-
return `[object, ~${(s.length / 1024).toFixed(1)}KB]`;
|
|
90
|
+
n = Object.keys(v).length;
|
|
23
91
|
} catch {
|
|
24
92
|
return '[object, unserializable]';
|
|
25
93
|
}
|
|
94
|
+
return `[object, ${n} ${n === 1 ? 'key' : 'keys'}]`;
|
|
26
95
|
}
|
|
27
96
|
return `[${t}]`;
|
|
28
97
|
}
|
|
@@ -45,28 +114,14 @@ function expectedFor (err) {
|
|
|
45
114
|
}
|
|
46
115
|
}
|
|
47
116
|
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
// pass on the segments that carry no `~`.
|
|
117
|
+
// The value the error's pointer resolves to, or MISSING when the pointer walks
|
|
118
|
+
// out of the document. Resolved once per error and handed to both the
|
|
119
|
+
// `received` repr and the suggestion sources, which used to walk it each.
|
|
120
|
+
function resolveAt (err, data) {
|
|
121
|
+
if (!data && data !== 0 && data !== false) return MISSING;
|
|
54
122
|
const p = err.instancePath || err.path || '';
|
|
55
|
-
if (!p) return
|
|
56
|
-
|
|
57
|
-
let cur = data;
|
|
58
|
-
let i = p.charCodeAt(0) === 47 ? 1 : 0; // 47 is '/'
|
|
59
|
-
for (;;) {
|
|
60
|
-
let j = p.indexOf('/', i);
|
|
61
|
-
if (j === -1) j = len;
|
|
62
|
-
let seg = p.slice(i, j);
|
|
63
|
-
if (seg.indexOf('~') !== -1) seg = seg.replace(/~1/g, '/').replace(/~0/g, '~');
|
|
64
|
-
if (cur == null) return undefined;
|
|
65
|
-
cur = cur[seg];
|
|
66
|
-
if (j === len) break;
|
|
67
|
-
i = j + 1;
|
|
68
|
-
}
|
|
69
|
-
return reprValue(cur);
|
|
123
|
+
if (!p) return data;
|
|
124
|
+
return resolvePointer(data, p, MISSING);
|
|
70
125
|
}
|
|
71
126
|
|
|
72
127
|
/**
|
|
@@ -146,13 +201,15 @@ function enrich (rawErr, opts) {
|
|
|
146
201
|
const meta = CODES[code];
|
|
147
202
|
const path = rawErr.instancePath != null ? rawErr.instancePath : (rawErr.path || '');
|
|
148
203
|
|
|
204
|
+
const at = data !== undefined ? resolveAt(rawErr, data) : MISSING;
|
|
205
|
+
|
|
149
206
|
const out = {
|
|
150
207
|
code,
|
|
151
208
|
message: rawErr.message || (meta && meta.headline) || 'validation failed',
|
|
152
209
|
keyword,
|
|
153
210
|
path,
|
|
154
211
|
expected: expectedFor(rawErr),
|
|
155
|
-
received:
|
|
212
|
+
received: at === MISSING ? undefined : reprValue(at),
|
|
156
213
|
schemaPath: rawErr.schemaPath,
|
|
157
214
|
docUrl: DOC_BASE + code,
|
|
158
215
|
// Back-compat aliases (additive, present in both rich and legacy paths)
|
|
@@ -162,6 +219,12 @@ function enrich (rawErr, opts) {
|
|
|
162
219
|
parentSchema: rawErr.parentSchema,
|
|
163
220
|
};
|
|
164
221
|
|
|
222
|
+
// Verbose mode's two other fields. Carried only when the raw error has them,
|
|
223
|
+
// so the default error shape does not grow two undefined keys for everyone
|
|
224
|
+
// who never asked for verbose.
|
|
225
|
+
if ('data' in rawErr) out.data = rawErr.data;
|
|
226
|
+
if ('schema' in rawErr) out.schema = rawErr.schema;
|
|
227
|
+
|
|
165
228
|
// oneOf/anyOf collapse: preserve the nested branch errors so the pretty
|
|
166
229
|
// renderer can surface the closest variant's diagnostics.
|
|
167
230
|
if (rawErr.branchErrors) out.branchErrors = rawErr.branchErrors;
|
|
@@ -187,7 +250,7 @@ function enrich (rawErr, opts) {
|
|
|
187
250
|
// Suggestion attachment runs last so it can read `received`, `params`, and
|
|
188
251
|
// `keyword` from the enriched shape. `data` is the full input object so the
|
|
189
252
|
// required-typo source can scan sibling keys.
|
|
190
|
-
const sugg = suggestFor(out,
|
|
253
|
+
const sugg = suggestFor(out, data, at === MISSING ? undefined : at);
|
|
191
254
|
if (sugg) out.suggestion = sugg;
|
|
192
255
|
|
|
193
256
|
const detail = detailFor(rawErr, out);
|