ata-validator 0.4.14 → 0.4.16

package/README.md

@@ -2,21 +2,47 @@
 
 Ultra-fast JSON Schema validator powered by [simdjson](https://github.com/simdjson/simdjson). Multi-core parallel validation, RE2 regex, codegen bytecode engine. Standard Schema V1 compatible.
 
-**[ata-validator.com](https://ata-validator.com)**
+**[ata-validator.com](https://ata-validator.com)** | **[API Docs](docs/API.md)** | **[Contributing](CONTRIBUTING.md)**
 
 ## Performance
 
-### Single-Document Validation
+### Simple Schema (5 properties, type + format + range checks)
 
 | Scenario | ata | ajv | |
 |---|---|---|---|
-| **validate(obj)** valid | 25.5M ops/sec | 19.3M ops/sec | **ata 1.3x faster** |
-| **validate(obj)** invalid | 17.7M ops/sec | 13.5M ops/sec | **ata 1.3x faster** |
-| **isValidObject(obj)** | 39.5M ops/sec | 17.6M ops/sec | **ata 2.2x faster** |
-| **Constructor cold start** | 1.28M ops/sec | 812 ops/sec | **ata 1,580x faster** |
-| **First validation** | 396K ops/sec | 880 ops/sec | **ata 450x faster** |
+| **validate(obj)** valid | 28ns | 104ns | **ata 3.6x faster** |
+| **validate(obj)** invalid | 79ns | 108ns | **ata 2.3x faster** |
+| **isValidObject(obj)** | 28ns | 102ns | **ata 3.7x faster** |
+| **Schema compilation** | 554ns | 1.21ms | **ata 2,184x faster** |
+| **First validation** | 1.70μs | 1.18ms | **ata 719x faster** |
 
-> validate(obj) numbers are isolated single-schema benchmarks. Multi-schema benchmark overhead reduces throughput; real-world numbers depend on workload.
+### Complex Schema (patternProperties + dependentSchemas + propertyNames + additionalProperties)
+
+| Scenario | ata | ajv | |
+|---|---|---|---|
+| **validate(obj)** valid | 20ns | 121ns | **ata 5.9x faster** |
+| **validate(obj)** invalid | 53ns | 196ns | **ata 3.2x faster** |
+| **isValidObject(obj)** | 20ns | 128ns | **ata 5.9x faster** |
+
+### Cross-Schema `$ref` (multi-schema with `$id` registry)
+
+| Scenario | ata | ajv | |
+|---|---|---|---|
+| **validate(obj)** valid | 17ns | 25ns | **ata 1.5x faster** |
+| **validate(obj)** invalid | 34ns | 54ns | **ata 1.6x faster** |
+
+> Measured with [mitata](https://github.com/evanwashere/mitata) on Apple M4 Pro (process-isolated). [Benchmark code](benchmark/bench_complex_mitata.mjs)
+
+### vs Ecosystem (Zod, Valibot, TypeBox)
+
+| Scenario | ata | ajv | typebox | zod | valibot |
+|---|---|---|---|---|---|
+| **validate (valid)** | **13ns** | 37ns | 48ns | 328ns | 316ns |
+| **validate (invalid)** | **35ns** | 104ns | 4ns | 11.7μs | 838ns |
+| **compilation** | **533ns** | 1.14ms | 52μs | — | — |
+| **first validation** | **1.3μs** | 1.07ms | 53μs | — | — |
+
+> Different categories: ata/ajv/typebox are JSON Schema validators, zod/valibot are schema-builder DSLs. [Benchmark code](benchmark/bench_all_mitata.mjs)
 
 ### Large Data - JS Object Validation
 
@@ -39,11 +65,11 @@ Ultra-fast JSON Schema validator powered by [simdjson](https://github.com/simdjs
 
 ### How it works
 
-**Combined single-pass validator**: ata compiles schemas into a single function that validates and collects errors in one pass. Valid data returns `VALID_RESULT` with zero allocation. Invalid data collects errors inline - no double validation, no try/catch (3.3x V8 deopt). Lazy compilation defers all work to first usage - constructor is near-zero cost.
+**Combined single-pass validator**: ata compiles schemas into a single function that validates and collects errors in one pass. Valid data returns `VALID_RESULT` with zero allocation. Invalid data collects errors inline with pre-allocated frozen error objects - no double validation, no try/catch (3.3x V8 deopt). Lazy compilation defers all work to first usage - constructor is near-zero cost.
 
-**JS codegen**: Schemas are compiled to monolithic JS functions (like ajv). Supported keywords: `type`, `required`, `properties`, `items`, `enum`, `const`, `allOf`, `anyOf`, `oneOf`, `not`, `if/then/else`, `uniqueItems`, `contains`, `prefixItems`, `additionalProperties`, `dependentRequired`, `$ref` (local), `minimum/maximum`, `minLength/maxLength`, `pattern`, `format`.
+**JS codegen**: Schemas are compiled to monolithic JS functions (like ajv). Full keyword support including `patternProperties`, `dependentSchemas`, `propertyNames`, cross-schema `$ref` with `$id` registry, and Draft 7 auto-detection. charCodeAt prefix matching replaces regex for simple patterns (4x faster). Merged key iteration loops (patternProperties + propertyNames + additionalProperties in a single `for..in`).
 
-**V8 TurboFan optimizations**: Destructuring batch reads, `undefined` checks instead of `in` operator, context-aware type guard elimination, property hoisting to local variables, tiered uniqueItems (nested loop for small arrays).
+**V8 TurboFan optimizations**: Destructuring batch reads, `undefined` checks instead of `in` operator, context-aware type guard elimination, property hoisting to local variables, tiered uniqueItems (nested loop for small arrays), inline key comparison for small property sets (no Set.has overhead).
 
 **Adaptive simdjson**: For large documents (>8KB) with selective schemas, simdjson On Demand seeks only the needed fields - skipping irrelevant data at GB/s speeds.
 
@@ -53,8 +79,11 @@ Ultra-fast JSON Schema validator powered by [simdjson](https://github.com/simdjs
 
 ## When to use ata
 
-- **High-throughput `validate(obj)`** - 25.5M ops/sec valid, 17.7M ops/sec invalid
-- **Serverless / cold starts** - 1,580x faster constructor, 450x faster first validation
+- **High-throughput `validate(obj)`** - 5.9x faster than ajv on complex schemas, 27x faster than zod
+- **Complex schemas** - `patternProperties`, `dependentSchemas`, `propertyNames` all inline JS codegen (5.9x faster than ajv)
+- **Multi-schema projects** - cross-schema `$ref` with `$id` registry, `addSchema()` API
+- **Draft 7 migration** - auto-detects `$schema`, normalizes Draft 7 keywords transparently
+- **Serverless / cold starts** - 2,184x faster compilation, 719x faster first validation
 - **Security-sensitive apps** - RE2 regex, immune to ReDoS attacks
 - **Batch/streaming validation** - NDJSON log processing, data pipelines (2.6x faster)
 - **Standard Schema V1** - native support for Fastify v5, tRPC, TanStack
@@ -62,12 +91,14 @@ Ultra-fast JSON Schema validator powered by [simdjson](https://github.com/simdjs
 
 ## When to use ajv
 
-- **Schemas with `patternProperties`, `dependentSchemas`** - these bypass JS codegen and hit the slower NAPI path
 - **100% spec compliance needed** - ajv covers more edge cases (ata: 98.4%)
+- **`$dynamicRef` / `unevaluatedProperties`** - not yet supported in ata
 
 ## Features
 
-- **Hybrid validator**: 25.5M ops/sec valid, 17.7M ops/sec invalid - codegen + single-pass error collection. No try/catch, no double pass. Schema compilation cache for repeated schemas
+- **Hybrid validator**: 5.9x faster than ajv valid, 3.2x faster invalid on complex schemas - jsFn boolean guard for valid path (zero allocation), combined codegen with pre-allocated errors for invalid path. Schema compilation cache for repeated schemas
+- **Cross-schema `$ref`**: `schemas` option and `addSchema()` API. Compile-time resolution with `$id` registry, zero runtime overhead
+- **Draft 7 support**: Auto-detects `$schema` field, normalizes `dependencies`/`additionalItems`/`definitions` transparently
 - **Multi-core**: Parallel validation across all CPU cores - 13.4M validations/sec
 - **simdjson**: SIMD-accelerated JSON parsing at GB/s speeds, adaptive On Demand for large docs
 - **RE2 regex**: Linear-time guarantees, immune to ReDoS attacks (2391x faster on pathological input)
@@ -122,12 +153,36 @@ v.isValidParallel(ndjson);  // bool[]
 v.countValid(ndjson);        // number
 ```
 
+### Cross-Schema `$ref`
+
+```javascript
+const addressSchema = {
+  $id: 'https://example.com/address',
+  type: 'object',
+  properties: { street: { type: 'string' }, city: { type: 'string' } },
+  required: ['street', 'city']
+};
+
+const v = new Validator({
+  type: 'object',
+  properties: {
+    name: { type: 'string' },
+    address: { $ref: 'https://example.com/address' }
+  }
+}, { schemas: [addressSchema] });
+
+// Or use addSchema()
+const v2 = new Validator(mainSchema);
+v2.addSchema(addressSchema);
+```
+
 ### Options
 
 ```javascript
 const v = new Validator(schema, {
   coerceTypes: true,       // "42" → 42 for integer fields
   removeAdditional: true,  // strip properties not in schema
+  schemas: [otherSchema],  // cross-schema $ref registry
 });
 ```
 
@@ -215,6 +270,27 @@ auto result = ata::validate(schema, R"({"name": "Mert"})");
 
 ## Building from Source
 
+### Development prerequisites
+
+Native builds require C/C++ toolchain support and the following libraries:
+
+- `re2`
+- `abseil`
+- `mimalloc`
+
+Install them before running `npm install` / `npm run build`:
+
+```bash
+# macOS (Homebrew)
+brew install re2 abseil mimalloc
+```
+
+```bash
+# Ubuntu/Debian (apt)
+sudo apt-get update
+sudo apt-get install -y libre2-dev libabsl-dev libmimalloc-dev
+```
+
 ```bash
 # C++ library + tests
 cmake -B build
@@ -234,6 +310,7 @@ npm run test:suite
 
 MIT
 
-## Author
+## Authors
 
 [Mert Can Altin](https://github.com/mertcanaltin)
+[Daniel Lemire](https://github.com/lemire)

package/index.d.ts

@@ -12,6 +12,7 @@ export interface ValidationResult {
 export interface ValidatorOptions {
   coerceTypes?: boolean;
   removeAdditional?: boolean;
+  schemas?: Record<string, object> | object[];
 }
 
 export interface StandardSchemaV1Props {
@@ -27,6 +28,9 @@ export interface StandardSchemaV1Props {
 export class Validator {
   constructor(schema: object | string, options?: ValidatorOptions);
 
+  /** Add a schema to the validator */
+  addSchema(schema: object): void;
+
   /** Validate data — returns result with errors. Applies defaults, coerceTypes, removeAdditional. */
   validate(data: unknown): ValidationResult;
 

package/index.js

@@ -5,6 +5,7 @@ const {
   compileToJSCodegenWithErrors,
   compileToJSCombined,
 } = require("./lib/js-compiler");
+const { normalizeDraft7 } = require("./lib/draft7");
 
 // Extract default values from a schema tree. Returns a function that applies
 // defaults to an object in-place (mutates), or null if no defaults exist.
@@ -205,6 +206,58 @@ function collectRemovals(schema, actions, path) {
   }
 }
 
+// Generate a fast preprocess function via codegen instead of closure arrays
+function buildPreprocessCodegen(schema, options) {
+  if (typeof schema !== 'object' || schema === null || !schema.properties) return null;
+  const lines = [];
+  const props = schema.properties;
+  const keys = Object.keys(props);
+
+  // removeAdditional: inline key check
+  if (options.removeAdditional && schema.additionalProperties === false) {
+    const checks = keys.map(k => `_k!==${JSON.stringify(k)}`).join('&&');
+    lines.push(`for(var _k in d)if(${checks})delete d[_k]`);
+  }
+
+  // coerceTypes: inline per property
+  if (options.coerceTypes) {
+    for (const [key, prop] of Object.entries(props)) {
+      if (!prop || typeof prop !== 'object' || !prop.type) continue;
+      const t = Array.isArray(prop.type) ? null : prop.type;
+      if (!t) continue;
+      const k = JSON.stringify(key);
+      if (t === 'integer') {
+        lines.push(`if(typeof d[${k}]==='string'){var _n=Number(d[${k}]);if(d[${k}]!==''&&Number.isInteger(_n))d[${k}]=_n}`);
+        lines.push(`if(typeof d[${k}]==='boolean')d[${k}]=d[${k}]?1:0`);
+      } else if (t === 'number') {
+        lines.push(`if(typeof d[${k}]==='string'){var _n=Number(d[${k}]);if(d[${k}]!==''&&!isNaN(_n))d[${k}]=_n}`);
+        lines.push(`if(typeof d[${k}]==='boolean')d[${k}]=d[${k}]?1:0`);
+      } else if (t === 'string') {
+        lines.push(`if(typeof d[${k}]==='number'||typeof d[${k}]==='boolean')d[${k}]=String(d[${k}])`);
+      } else if (t === 'boolean') {
+        lines.push(`if(d[${k}]==='true'||d[${k}]==='1')d[${k}]=true`);
+        lines.push(`if(d[${k}]==='false'||d[${k}]==='0')d[${k}]=false`);
+      }
+    }
+  }
+
+  // defaults: inline per property
+  for (const [key, prop] of Object.entries(props)) {
+    if (prop && typeof prop === 'object' && prop.default !== undefined) {
+      const k = JSON.stringify(key);
+      const def = JSON.stringify(prop.default);
+      lines.push(`if(!(${k} in d))d[${k}]=${def}`);
+    }
+  }
+
+  if (lines.length === 0) return null;
+  try {
+    return new Function('d', lines.join('\n'));
+  } catch {
+    return null;
+  }
+}
+
 // Schema compilation cache: same schema string -> reuse compiled functions
 const _compileCache = new Map();
 
@@ -233,13 +286,35 @@ function createPaddedBuffer(jsonStr) {
   return { buffer: padded, length: jsonBuf.length };
 }
 
+function buildSchemaMap(schemas) {
+  if (!schemas) return null
+  const map = new Map()
+  if (Array.isArray(schemas)) {
+    for (const s of schemas) {
+      normalizeDraft7(s)
+      const id = s.$id
+      if (!id) throw new Error('Schema in schemas option must have $id')
+      map.set(id, s)
+    }
+  } else {
+    for (const [key, s] of Object.entries(schemas)) {
+      normalizeDraft7(s)
+      map.set(s.$id || key, s)
+    }
+  }
+  return map
+}
+
 class Validator {
   constructor(schema, opts) {
     const options = opts || {};
-    const schemaStr =
-      typeof schema === "string" ? schema : JSON.stringify(schema);
     const schemaObj = typeof schema === "string" ? JSON.parse(schema) : schema;
 
+    // Draft 7 normalization — convert keywords to 2020-12 equivalents in-place
+    normalizeDraft7(schemaObj);
+
+    const schemaStr = JSON.stringify(schemaObj);
+
     this._schemaStr = schemaStr;
     this._schemaObj = schemaObj;
     this._options = options;
@@ -251,6 +326,9 @@ class Validator {
     this._preprocess = null;
     this._applyDefaults = null;
 
+    // Schema map for cross-schema $ref resolution
+    this._schemaMap = buildSchemaMap(options.schemas) || new Map();
+
     // Lazy stubs: trigger compilation on first call, then re-dispatch
     this.validate = (data) => {
       this._ensureCompiled();
@@ -303,40 +381,45 @@ class Validator {
     const options = this._options;
 
     // Check cache first -- reuse compiled functions for same schema
-    const cached = _compileCache.get(this._schemaStr);
+    const sm = this._schemaMap.size > 0 ? this._schemaMap : null;
+    const mapKey = this._schemaMap.size > 0
+      ? this._schemaStr + '\0' + [...this._schemaMap.keys()].sort().join('\0')
+      : this._schemaStr;
+    const cached = _compileCache.get(mapKey);
     let jsFn, jsCombinedFn, jsErrFn;
     if (cached && !process.env.ATA_FORCE_NAPI) {
       jsFn = cached.jsFn;
       jsCombinedFn = cached.combined;
       jsErrFn = cached.errFn;
     } else if (!process.env.ATA_FORCE_NAPI) {
-      jsFn = compileToJSCodegen(schemaObj) || compileToJS(schemaObj);
-      jsCombinedFn = compileToJSCombined(schemaObj, VALID_RESULT);
-      jsErrFn = compileToJSCodegenWithErrors(schemaObj);
-      _compileCache.set(this._schemaStr, { jsFn, combined: jsCombinedFn, errFn: jsErrFn });
+      jsFn = compileToJSCodegen(schemaObj, sm) || compileToJS(schemaObj, null, sm);
+      jsCombinedFn = compileToJSCombined(schemaObj, VALID_RESULT, sm);
+      jsErrFn = compileToJSCodegenWithErrors(schemaObj, sm);
+      _compileCache.set(mapKey, { jsFn, combined: jsCombinedFn, errFn: jsErrFn });
     } else {
       jsFn = null; jsCombinedFn = null; jsErrFn = null;
     }
     this._jsFn = jsFn;
 
-    // Data mutators -- applied in-place before validation
-    const applyDefaults = buildDefaultsApplier(schemaObj);
-    const applyCoerce = options.coerceTypes ? buildCoercer(schemaObj) : null;
-    const applyRemove = options.removeAdditional
-      ? buildRemover(schemaObj)
-      : null;
-    this._applyDefaults = applyDefaults;
-
-    // Combine all mutators into a single pre-validation step
-    const mutators = [applyRemove, applyCoerce, applyDefaults].filter(Boolean);
-    const preprocess =
-      mutators.length === 0
-        ? null
-        : mutators.length === 1
-          ? mutators[0]
-          : (data) => {
-              for (let i = 0; i < mutators.length; i++) mutators[i](data);
-            };
+    // Data mutators -- try codegen first (12x faster), fallback to closure arrays
+    let preprocess = buildPreprocessCodegen(schemaObj, options);
+    if (!preprocess) {
+      const applyDefaults = buildDefaultsApplier(schemaObj);
+      const applyCoerce = options.coerceTypes ? buildCoercer(schemaObj) : null;
+      const applyRemove = options.removeAdditional
+        ? buildRemover(schemaObj)
+        : null;
+      const mutators = [applyRemove, applyCoerce, applyDefaults].filter(Boolean);
+      preprocess =
+        mutators.length === 0
+          ? null
+          : mutators.length === 1
+            ? mutators[0]
+            : (data) => {
+                for (let i = 0; i < mutators.length; i++) mutators[i](data);
+              };
+    }
+    this._applyDefaults = preprocess;
     this._preprocess = preprocess;
 
     // Detect if schema is "selective" -- doesn't recurse into arrays/deep objects.
@@ -394,7 +477,15 @@ class Validator {
         } catch {}
       }
 
-      if (safeCombinedFn) {
+      if (safeCombinedFn && jsFn) {
+        // Hybrid: jsFn boolean guard for valid (fast, no allocation), combined for invalid
+        this.validate = preprocess
+          ? (data) => {
+              preprocess(data);
+              return jsFn(data) ? VALID_RESULT : safeCombinedFn(data);
+            }
+          : (data) => jsFn(data) ? VALID_RESULT : safeCombinedFn(data);
+      } else if (safeCombinedFn) {
         this.validate = preprocess
           ? (data) => {
               preprocess(data);
@@ -494,25 +585,50 @@ class Validator {
   _ensureNative() {
     if (this._nativeReady) return;
     this._nativeReady = true;
-    this._compiled = new native.CompiledSchema(this._schemaStr);
-    this._fastSlot = native.fastRegister(this._schemaStr);
+    let nativeSchemaStr = this._schemaStr;
+    if (this._schemaMap.size > 0) {
+      const merged = JSON.parse(this._schemaStr);
+      if (!merged.$defs) merged.$defs = {};
+      for (const [id, s] of this._schemaMap) {
+        merged.$defs['__ext_' + id.replace(/[^a-zA-Z0-9]/g, '_')] = s;
+      }
+      nativeSchemaStr = JSON.stringify(merged);
+    }
+    this._compiled = new native.CompiledSchema(nativeSchemaStr);
+    this._fastSlot = native.fastRegister(nativeSchemaStr);
+  }
+
+  addSchema(schema) {
+    if (this._initialized) {
+      throw new Error('Cannot add schema after compilation — call addSchema() before validate()')
+    }
+    if (!schema || !schema.$id) {
+      throw new Error('Schema must have $id')
+    }
+    // Apply Draft 7 normalization if needed
+    normalizeDraft7(schema)
+    this._schemaMap.set(schema.$id, schema)
   }
 
   _ensureCodegen() {
     if (this._jsFn) return;
     if (process.env.ATA_FORCE_NAPI) return;
-    const cached = _compileCache.get(this._schemaStr);
+    const sm = this._schemaMap.size > 0 ? this._schemaMap : null;
+    const mapKey = this._schemaMap.size > 0
+      ? this._schemaStr + '\0' + [...this._schemaMap.keys()].sort().join('\0')
+      : this._schemaStr;
+    const cached = _compileCache.get(mapKey);
     if (cached && cached.jsFn) {
       this._jsFn = cached.jsFn;
       this.isValidObject = cached.jsFn;
       return;
     }
-    const jsFn = compileToJSCodegen(this._schemaObj) || compileToJS(this._schemaObj);
+    const jsFn = compileToJSCodegen(this._schemaObj, sm) || compileToJS(this._schemaObj, null, sm);
     this._jsFn = jsFn;
     if (jsFn) {
       this.isValidObject = jsFn;
       // seed cache with codegen, combined/errFn filled later by _ensureCompiled
-      if (!cached) _compileCache.set(this._schemaStr, { jsFn, combined: null, errFn: null });
+      if (!cached) _compileCache.set(mapKey, { jsFn, combined: null, errFn: null });
       else cached.jsFn = jsFn;
     }
   }

package/lib/draft7.js

@@ -0,0 +1,82 @@
+'use strict'
+
+const DRAFT7_SCHEMAS = new Set([
+  'http://json-schema.org/draft-07/schema#',
+  'http://json-schema.org/draft-07/schema',
+])
+
+function isDraft7(schema) {
+  return !!(schema && schema.$schema && DRAFT7_SCHEMAS.has(schema.$schema))
+}
+
+function normalizeDraft7(schema) {
+  if (!isDraft7(schema)) return schema
+  _normalize(schema)
+  return schema
+}
+
+function _normalize(schema) {
+  if (typeof schema !== 'object' || schema === null) return
+
+  // definitions → $defs
+  if (schema.definitions && !schema.$defs) {
+    schema.$defs = schema.definitions
+    delete schema.definitions
+  }
+
+  // dependencies → dependentSchemas + dependentRequired
+  if (schema.dependencies) {
+    for (const [key, value] of Object.entries(schema.dependencies)) {
+      if (Array.isArray(value)) {
+        if (!schema.dependentRequired) schema.dependentRequired = {}
+        schema.dependentRequired[key] = value
+      } else {
+        if (!schema.dependentSchemas) schema.dependentSchemas = {}
+        schema.dependentSchemas[key] = value
+      }
+    }
+    delete schema.dependencies
+  }
+
+  // items (array form) → prefixItems + items/additionalItems swap
+  if (Array.isArray(schema.items)) {
+    schema.prefixItems = schema.items
+    if (schema.additionalItems !== undefined) {
+      schema.items = schema.additionalItems
+      delete schema.additionalItems
+    } else {
+      delete schema.items
+    }
+  }
+
+  // Recurse into object-valued sub-schemas
+  const objSubs = ['properties', 'patternProperties', '$defs', 'definitions', 'dependentSchemas']
+  for (const key of objSubs) {
+    if (schema[key] && typeof schema[key] === 'object') {
+      for (const v of Object.values(schema[key])) {
+        if (typeof v === 'object' && v !== null) _normalize(v)
+      }
+    }
+  }
+
+  // Recurse into array-valued sub-schemas
+  const arrSubs = ['allOf', 'anyOf', 'oneOf', 'prefixItems']
+  for (const key of arrSubs) {
+    if (Array.isArray(schema[key])) {
+      for (const s of schema[key]) {
+        if (typeof s === 'object' && s !== null) _normalize(s)
+      }
+    }
+  }
+
+  // Recurse into single sub-schemas
+  const singleSubs = ['items', 'contains', 'not', 'if', 'then', 'else',
+                       'additionalProperties', 'propertyNames']
+  for (const key of singleSubs) {
+    if (typeof schema[key] === 'object' && schema[key] !== null) {
+      _normalize(schema[key])
+    }
+  }
+}
+
+module.exports = { isDraft7, normalizeDraft7 }