ata-validator 1.7.0 → 1.7.2

package/CHANGELOG.md

@@ -2,6 +2,35 @@
 
 All notable changes to ata-validator are documented here. The format follows [Keep a Changelog](https://keepachangelog.com/), and this project adheres to semantic versioning.
 
+## 1.7.2 - 2026-08-24
+
+### Fixed
+
+- `enum` compared object members by their serialized form, so `{"b": 2, "a": 1}` was rejected against `enum: [{"a": 1, "b": 2}]`. JSON Schema compares instances by value: two objects with the same members are equal whatever order their keys were written in. All three code generation paths did this, so they agreed with each other and disagreed with the interpreted engine, which has always compared structurally. `const` was already correct but reached the answer by building a canonical string on every call. Both now use one structural comparison hoisted per compiled function. The official suite does not cover reordered keys for `enum`, and the entry-point agreement test compares the generators against each other, so neither would have caught this; `tests/test_value_equality.js` pins the answer through every engine that can produce one and runs as part of `npm test`.
+- A draft-07 document passed through the `schemas` option was rejected with "Schema in schemas option must have $id" whenever it carried `$ref` next to `$id`. Draft-07 ignores every keyword sitting beside `$ref`, so normalization drops them and `$id` goes with them, which is the right reading for evaluation: the reference resolves against the URI the document was retrieved from, not the `$id` it declares. It is the wrong reading for registration, where `$id` is the only name the caller gave the document. The identity is now read from what the caller passed when normalization has dropped it, so such a document registers under its `$id` in the array form and is addressable by both its retrieval URI and its `$id` in the map form. A document that declares no `$id` at all is still refused, and a fragment-only `$id` is still an anchor rather than a document name. This is what made all 23 remote-reference cases of the draft-07 suite error out in the Bowtie compliance report; they now answer, and answer correctly.
+
+### Changed
+
+- The closure-tree compiler now covers every schema the interpreted engine accepts. It previously declined `unevaluatedProperties`/`unevaluatedItems` and `$dynamicRef` in any schema with more than one resource, which left 192 of the 987 suite schemas walking the generic evaluator; that count is now zero. Annotations flow through the compiled tree: the compiler decides at compile time which nodes anyone reads annotations from, and those nodes carry a record that in-place children write into directly, with a failed child rolled back by truncating the record rather than by allocating a fresh one per child. `$dynamicRef` resolves against the dynamic scope computed at compile time, so multi-resource schemas no longer search it per call. A schema whose compiled graph has no cycle drops the recursion guard and the `(schema, data)` stack entirely. An unresolvable `$ref` compiles into the same runtime rejection the evaluator produces instead of declining the schema.
+- Measured on the official suite with prebuilt validators, several runs each on the same machine: draft 2020-12 went from 94 ns to 77-80 ns per case overall, and the cases that route to the interpreted engine from about 240 ns to about 125 ns when they accept and 235 ns to 160 ns when they reject. Draft 7 went from 59 ns to 49 ns overall. The metaschema-validation case, the worst outlier in the suite, went from 5.5 µs to about 1.4 µs. On a longer steady-state loop over the same suite, where construction is not part of what is timed, draft 2020-12 went from 46 ns to 43 ns per case and draft 7 from 33 ns to 29 ns.
+- The value-level keywords of a schema node are compiled into closures over their own operands instead of being answered by a generic evaluator that re-reads a dozen presence flags off the plan on every call. The type of the value is computed once per node rather than once per keyword group, and a node whose keywords never need it does not compute it at all.
+- `uniqueItems` on arrays whose item type is not known ahead of time no longer builds a canonicalizing closure and a `Set` on every call. The helpers are hoisted once per compiled function; arrays of twelve items or fewer compare pairwise with a structural equality that ignores key order and allocates nothing, and longer arrays keep the `Set`, using canonical strings only once an object actually appears. Over the draft 7 suite the `uniqueItems` cases went from 125 ns to 24 ns each.
+- `ARCHITECTURE.md` documents the interpreted engine and the closure-tree compiler, and no longer claims `$dynamicRef` falls back to the native engine, which stopped being true in 1.7.1.
+
+## 1.7.1 - 2026-08-23
+
+### Added
+
+- A closure-tree compiler for the interpreted engine. A schema the code generator declines is compiled into a tree of plain closures, one per schema node, with every keyword branch decided at compile time and `$ref` targets resolved once; no source generation and no `new Function`, so it works under a CSP and on Workers. Scope: schemas without `unevaluatedProperties`/`unevaluatedItems`, and `$dynamicRef` only in single-resource schemas; everything else keeps the generic evaluator. `tests/test_plan_compiler.js` holds the compiled tree to byte-identical verdicts and errors against the evaluator over 2,864 suite cases.
+
+### Changed
+
+- Plans that check only value-level keywords (most leaves of any schema) skip the evaluator's prologue entirely; `$ref` resolutions are cached with their planned target on the plan itself; the dynamic scope is pushed and popped in place instead of copied per resource. Rejection results are a small class with the `errors` accessor on the prototype, since defining a getter inside an object literal builds a closure and an accessor property on every rejection, which was the single largest cost on the rejection path.
+- The interpreted engine gained a verdict-only mode: `isValidObject()` and the internal fast checks walk the schema without constructing a single error object, message string or scratch array. On an interpreter-routed schema the boolean check dropped from 894 ns to 70 ns.
+- Object validation for `$dynamicRef` schemas no longer routes to the native engine. The interpreted engine has scored the same on every `$dynamicRef` case of the suite since the dynamic-scope fix in 1.7.0, needs no addon, and carries the verdict-only mode; the suite's `$dynamicRef` rejects dropped from about 1.7 µs to the interpreter's cost.
+- String length bounds decide from the UTF-16 length where possible: a string's code point count always sits between half its length and its length, so `minLength`/`maxLength` only count code points inside the narrow band where the answer is genuinely uncertain. The surrogate test is a single wraparound compare. The code generator already worked this way; the interpreter and the closure path now match it.
+- Errors are paid for when read, not when produced. `validate()` answers the verdict from the fastest engine for the schema and materializes `errors` through a cached getter on first access; declaration-order sorting and enrichment (received value, suggestions, source frames) moved with it into one presentation layer. A caller that only reads `.valid`, which is every gateway check, no longer pays for error construction at all. The output of `.errors` is byte-for-byte what it was. Measured on a suite-shaped benchmark of prebuilt validators over 1,052 mixed valid and invalid cases, `validate().valid` went from 778 ns to about 150 ns per call, ahead of every error-capable validator we measured, and a rejection that never has its errors read now costs less than `abortEarly` mode used to. One observable edge: mutating the data between `validate()` and the first read of `.errors` now reflects the mutated data in the errors, and if the mutation makes the data valid the errors fall back to a single generic entry.
+
 ## 1.7.0 - 2026-08-23
 
 ### Fixed

package/README.md

@@ -64,9 +64,10 @@ passes, and a rejection costs more than a verdict: ata builds an error carrying
 offending value, a documentation link and a suggestion. On a five-field object schema a
 passing payload costs about 17 ns and a rejected one about 155 ns. `abortEarly: true` or
 `isValidObject()` skips that work when only the verdict matters. Schemas ata declines to
-compile, mostly cross-document `$ref`, `$dynamicRef` and `unevaluated*`, run on the
-interpreted engine: about 343 ns for a passing payload on a six-field object schema, eight
-times the compiled path, measured warm on the same machine.
+compile, mostly cross-document `$ref`, `$dynamicRef` and the harder `unevaluated*` shapes,
+run on the interpreted engine. That engine compiles each schema into a tree of closures, so
+on a six-field object schema a passing payload costs about 136 ns against 8 ns for the same
+shape on the compiled path, both measured warm in one run on the same machine.
 
 ## Error messages
 
@@ -169,6 +170,9 @@ v.isValidJSON('{"name": "Mert", "email": "mert@example.com"}'); // true
 // Buffer input (zero-copy, raw NAPI)
 v.isValid(Buffer.from('{"name": "Mert", "email": "mert@example.com"}'));
 
+// Which engine answers this schema: 'codegen', 'closure', 'native' or 'interpreter'
+v.engine(); // 'codegen'
+
 // Parallel batch - multi-core, NDJSON, 13.4M items/sec
 const ndjson = Buffer.from(lines.join('\n'));
 v.isValidParallel(ndjson);  // bool[]
@@ -400,6 +404,12 @@ const { toStandaloneModule } = require('ata-validator/build');
 fs.writeFileSync('./user.validator.mjs', toStandaloneModule(schema, { format: 'esm' }));
 ```
 
+Custom format functions either get their source embedded (the default, refused
+at build time with a named error when the function would not survive
+serialization) or, with `formatMode: 'inject'`, are supplied at load time
+through a `setFormats()` export the module carries. `docs/API.md` has the
+details.
+
 **Fastify startup, 10 route schemas, from a cold process to the first validated request:
 ajv 19.6 ms, ata 3.1 ms, no build step required.** ata registers in 1.1 ms of that and
 compiles on the first request, so counting only registration would overstate the gap.

package/index.d.ts

@@ -403,9 +403,9 @@ export interface Validator<T = unknown> {
   isValidObject(data: unknown): data is T;
   /**
    * Which engine answers `validate()` for this schema: 'codegen' (generated
-   * JS), 'closure' (the closure compiler), 'native' (the C++ engine, only
-   * for some $dynamicRef schemas) or 'interpreter'. The verdict is the same
-   * on every engine; the cost is not. A diagnostic, not a configuration.
+   * JS), 'closure' (the closure compiler) or 'interpreter'; 'native' is
+   * reserved. The verdict is the same on every engine; the cost is not.
+   * A diagnostic, not a configuration.
    */
   engine(): 'codegen' | 'closure' | 'native' | 'interpreter';
 

package/index.js

@@ -314,6 +314,34 @@ const ABORT_EARLY_RESULT = Object.freeze({
 
 // Above this size, simdjson On Demand (selective field access) beats JSON.parse
 // (which must materialize the full JS object tree). Buffer.from + NAPI ~2x faster.
+
+// Rejection result with errors materialized on first read. The accessor
+// lives on the prototype so constructing one is a plain allocation; an
+// object-literal getter would create a closure and define an accessor
+// property on every rejection, which showed up as the single largest cost
+// on the rejection path. `toJSON` keeps JSON.stringify output identical to
+// the eager shape. Note for tests: deepStrictEqual against a plain object
+// compares prototypes; read `.errors` and compare that.
+class LazyRejection {
+  constructor(build, data) {
+    this.valid = false;
+    this._build = build;
+    this._data = data;
+    this._errors = null;
+  }
+  toJSON() {
+    return { valid: false, errors: this.errors };
+  }
+}
+Object.defineProperty(LazyRejection.prototype, 'errors', {
+  enumerable: true,
+  configurable: true,
+  get() {
+    if (this._errors === null) this._errors = this._build(this._data);
+    return this._errors;
+  },
+});
+
 const SIMDJSON_THRESHOLD = 8192;
 
 // Resolve a JSON Schema path like "#/properties/name/type" to the schema object
@@ -440,6 +468,23 @@ function _normalizeCallerSchema(s, inheritDraft7) {
   return JSON.stringify(copy) === str ? s : copy
 }
 
+// The identity a document is registered under. Draft-07 ignores every
+// keyword sitting next to `$ref`, so normalization drops them, `$id` among
+// them: that is the right reading for evaluation, where the reference
+// resolves against the retrieval URI rather than the declared `$id`. It is
+// the wrong reading for registration, since `$id` is how the caller names
+// the document. So the identity is read from the normalized copy first and
+// from what the caller passed second. A bare-fragment `$id` is a draft-07
+// anchor rather than a document identity, and normalization has already
+// turned it into `$anchor`, so it is not used here.
+function declaredId(original, normalized) {
+  const n = normalized && typeof normalized === 'object' ? normalized.$id : undefined
+  if (typeof n === 'string' && n !== '') return n
+  const o = original && typeof original === 'object' ? original.$id : undefined
+  if (typeof o === 'string' && o !== '' && o[0] !== '#') return o
+  return undefined
+}
+
 // `inheritDraft7` is true when the root schema is draft-07: a retrieved
 // document that declares no dialect is read under the root's draft.
 function buildSchemaMap(schemas, inheritDraft7) {
@@ -448,7 +493,7 @@ function buildSchemaMap(schemas, inheritDraft7) {
   if (Array.isArray(schemas)) {
     for (const s of schemas) {
       const normalized = _normalizeCallerSchema(s, inheritDraft7)
-      const id = normalized.$id
+      const id = declaredId(s, normalized)
       if (!id) throw new Error('Schema in schemas option must have $id')
       map.set(id, normalized)
     }
@@ -459,7 +504,8 @@ function buildSchemaMap(schemas, inheritDraft7) {
       // under and by the $id it declares. Registering only the $id makes
       // references to the retrieval URI unresolvable.
       map.set(key, normalized)
-      if (normalized.$id && normalized.$id !== key) map.set(normalized.$id, normalized)
+      const id = declaredId(s, normalized)
+      if (id && id !== key) map.set(id, normalized)
     }
   }
   return map
@@ -904,6 +950,10 @@ class Validator {
         } catch {}
       }
 
+      // The boolean engine is the verdict authority for these paths; the
+      // final lazy wrapper uses it to skip error construction entirely.
+      if (!hasDynRef || _isCodegen) this._fastVerdict = preprocess ? null : jsFn;
+
       if (options.abortEarly && jsFn && !hasDynRef) {
         // abortEarly: do NOT enrich. Skip position lookups, suggestions, source maps.
         // This is the perf-critical path for edge gateways. The richErrors wrap
@@ -920,10 +970,22 @@ class Validator {
           ? (data) => { preprocess(data); return _fn(data) ? _R : _efn(data); }
           : (data) => _fn(data) ? _R : _efn(data);
       } else if (hasDynRef) {
-        // $dynamicRef without codegen: delegate to native C++ (interpretive path unreliable)
+        // $dynamicRef without codegen: the interpreted engine. It scores the
+        // same on the suite's $dynamicRef cases as the native walker since the
+        // dynamic-scope fix, needs no addon, and gets the verdict-only mode.
+        if (!_interp) {
+          const { createInterpreter } = require('./lib/interpreter');
+          _interp = createInterpreter(schemaObj, {
+            schemaMap: this._schemaMap.size > 0 ? this._schemaMap : null,
+            formats: this._userFormats,
+            v1: isV1Dialect(schemaObj),
+          });
+        }
+        const interp = _interp;
+        this._fastVerdict = preprocess ? null : (d) => interp.isValid(d);
         this.validate = preprocess
-          ? (data) => { preprocess(data); return errFn(data); }
-          : errFn;
+          ? (data) => { preprocess(data); return interp.validate(data); }
+          : (data) => interp.validate(data);
       } else if (jsFn && jsFn._hybridFactory) {
         // Zero-wrapper: hybridFactory bakes VALID_RESULT + errFn into a single function
         // No arrow function wrapper, no ternary, one function call
@@ -1098,15 +1160,11 @@ class Validator {
       // propertyDependencies exists only in the interpreted engine, so a schema
       // using it goes there even when it also uses $dynamicRef.
       const _hasPropDeps = this._schemaStr.includes('"propertyDependencies"')
+      // $dynamicRef used to delegate to the native validateJSON path here;
+      // the interpreted engine now scores the same on those cases, carries
+      // the verdict-only mode, and works without the addon.
       let _validate;
-      if (_hasDynRef && !_hasUneval && !_hasPropDeps && !this._v1Dynamic) {
-        // validateJSON is the C++ path with full anchor-map support; the NAPI
-        // direct V8 `validate` path has no anchor maps.
-        this._engine = 'native';
-        _validate = (data) => this._compiled.validateJSON(JSON.stringify(data));
-        this.validateJSON = (jsonStr) => this._compiled.validateJSON(jsonStr);
-        this.isValidJSON = (jsonStr) => this._compiled.isValidJSON(jsonStr);
-      } else {
+      {
         const { createInterpreter } = require('./lib/interpreter');
         const interp = createInterpreter(schemaObj, {
           schemaMap: this._schemaMap.size > 0 ? this._schemaMap : null,
@@ -1115,6 +1173,7 @@ class Validator {
         });
         this._engine = 'interpreter';
         _validate = (data) => interp.validate(data);
+        this._fastVerdict = preprocess ? null : (d) => interp.isValid(d);
         this.validateJSON = (jsonStr) => {
           try {
             return _validate(JSON.parse(jsonStr));
@@ -1130,7 +1189,9 @@ class Validator {
             return _validate(data);
           }
         : _validate;
-      this.isValidObject = (data) => _validate(data).valid;
+      this.isValidObject = this._fastVerdict
+        ? this._fastVerdict
+        : (data) => _validate(data).valid;
       this.validateAndParse = (jsonStr) => this._compiled.validateAndParse(jsonStr);
       {
         const slot = this._fastSlot;
@@ -1173,11 +1234,14 @@ class Validator {
         v1: isV1Dialect(schemaObj),
       });
       this._engine = 'interpreter';
+      if (!preprocess) this._fastVerdict = (d) => interp.isValid(d);
       const run = preprocess
         ? (data) => { preprocess(data); return interp.validate(data); }
         : (data) => interp.validate(data);
       this.validate = run;
-      this.isValidObject = (data) => run(data).valid;
+      this.isValidObject = this._fastVerdict
+        ? this._fastVerdict
+        : (data) => run(data).valid;
       this.validateJSON = (jsonStr) => {
         try {
           return run(JSON.parse(jsonStr));
@@ -1188,42 +1252,44 @@ class Validator {
       this.isValidJSON = (jsonStr) => this.validateJSON(jsonStr).valid;
     }
 
-    // Declaration-order errors: whichever engine produced them, multi-error
-    // results are sorted by the schema's keyword declaration order before
-    // enrichment. Single-error and abortEarly results pass through untouched.
+    // Error presentation, one lazy layer: declaration-order sorting, rich
+    // enrichment (received value, suggestions, source frames, docUrl), or the
+    // raw v0.14 shape under `richErrors: false`. All of it is work a caller
+    // that only reads `.valid` never sees, so it runs on first access to
+    // `.errors` and is cached. One wrapper, one allocation per rejection.
     if (this.validate) {
       const inner = this.validate;
+      const enrich = this._richErrors ? require('./lib/enrich-error').enrich : null;
       const root = this._schemaObj;
+      const self = this;
       this.validate = (data) => {
         const result = inner(data);
-        if (result && !result.valid && result.errors && result.errors.length > 1 && result !== ABORT_EARLY_RESULT) {
-          return { valid: false, errors: sortErrorsBySchemaOrder(root, result.errors) };
-        }
-        return result;
-      };
-    }
-
-    // richErrors enrichment: layered on top of whichever validate path was
-    // bound above. Verbose's parentSchema flows through because enrich()
-    // copies it. Opt-out (`richErrors: false`) leaves the raw v0.14 shape.
-    if (this._richErrors && this.validate) {
-      const inner = this.validate;
-      const enrich = require('./lib/enrich-error').enrich;
-      this.validate = (data) => {
-        const result = inner(data);
-        if (result && !result.valid && result.errors && result.errors.length) {
-          // abortEarly returns the shared ATA9000 stub; preserve it as-is so the
-          // perf fast path stays allocation-free and the documented code stays stable.
-          if (result === ABORT_EARLY_RESULT) return result;
-          const positions = (this._lastRawInput != null) ? this._posCache.get(this._lastRawInput) : null;
-          const enriched = result.errors.map((e) => enrich(e, {
-            data,
-            positions,
-            schemaPositions: this._schemaPositions,
-            schemaFile: this._source ? this._source.path : undefined,
-          }));
-          if (positions) this._posCache.reset();
-          return { valid: false, errors: enriched };
+        // abortEarly returns the shared ATA9000 stub; preserve it as-is so the
+        // perf fast path stays allocation-free and the documented code stays stable.
+        if (result && result.valid === false && result !== ABORT_EARLY_RESULT) {
+          // Positions come from the raw input when validateJSON set one;
+          // resolved eagerly since the cache is reset per call.
+          const positions = (enrich && self._lastRawInput != null) ? self._posCache.get(self._lastRawInput) : null;
+          if (positions) self._posCache.reset();
+          let cached = null;
+          return {
+            valid: false,
+            get errors() {
+              if (cached === null) {
+                let raw = result.errors || [];
+                if (raw.length > 1) raw = sortErrorsBySchemaOrder(root, raw);
+                cached = (enrich && raw.length)
+                  ? raw.map((e) => enrich(e, {
+                      data,
+                      positions,
+                      schemaPositions: self._schemaPositions,
+                      schemaFile: self._source ? self._source.path : undefined,
+                    }))
+                  : raw;
+              }
+              return cached;
+            },
+          };
         }
         return result;
       };
@@ -1231,7 +1297,7 @@ class Validator {
       // validateJSON also enriches: set _lastRawInput so the position cache
       // can lazily build a map for dataFrame attachment. Only validateJSON
       // wires this — validate(data) takes a pre-parsed object, by design.
-      if (this.validateJSON) {
+      if (this._richErrors && this.validateJSON) {
         const innerJson = this.validateJSON;
         this.validateJSON = (jsonStr) => {
           this._lastRawInput = jsonStr;
@@ -1335,6 +1401,34 @@ class Validator {
       }
     }
 
+    // Errors are paid for when read, not when produced. The full pipeline
+    // above (error codegen, enrichment, custom messages, verbose) stays
+    // intact, but validate() now answers the verdict from the boolean
+    // engine and materializes `errors` through a getter on first access.
+    // A caller that only reads `.valid`, which is every gateway check and
+    // every benchmark, skips error construction entirely; a caller that
+    // reads `.errors` pays once and the result is cached. Skipped when the
+    // schema coerces or defaults (preprocess mutates before the verdict),
+    // under abortEarly (already a frozen stub), and for $dynamicRef (the
+    // boolean engine is not the authority there).
+    if (this._fastVerdict && !preprocess && !options.abortEarly && this.validate) {
+      const _full = this.validate;
+      const _fast = this._fastVerdict;
+      const EMPTY_ERRORS = Object.freeze([]);
+      const _buildErrors = (data) => {
+        const r = _full(data);
+        return (r && r.valid === false && r.errors && r.errors.length)
+          ? r.errors
+          // The data changed between the verdict and this read; keep the
+          // verdict and say so rather than inventing a specific error.
+          : [{ keyword: 'validation', instancePath: '', schemaPath: '#', params: {}, message: 'schema validation failed' }];
+      };
+      this.validate = (data) => {
+        if (_fast(data)) return { valid: true, data, errors: EMPTY_ERRORS };
+        return new LazyRejection(_buildErrors, data);
+      };
+    }
+
     // The buffer APIs answer from the native walker, which disagrees with
     // validate() on shapes listed in lib/buffer-gate.js. For those schemas
     // every buffer entry point goes through validate() instead.

package/lib/aot.js

@@ -21,7 +21,7 @@ const SAFE_REGEX_SOURCE = require('./safe-regex-source');
 const _CP_LEN_SOURCE = `function _cpLen(s) {
   const len = s.length;
   for (let i = 0; i < len; i++) {
-    if (s.charCodeAt(i) >= 0xD800 && s.charCodeAt(i) <= 0xDBFF) {
+    if (((s.charCodeAt(i) - 0xD800) >>> 0) < 0x400) {
       let n = 0; for (const _ of s) n++; return n;
     }
   }