@ata-project/unplugin 0.5.0 → 0.7.0

package/README.md

@@ -151,6 +151,7 @@ Other sources (`.json` without the `.schema` suffix, `.js`, `.ts`) produce
 | `root` | from the bundler | Where the globs resolve from. Vite's root, Webpack's and Rspack's `context` and esbuild's `absWorkingDir` are read; Rollup and Rolldown need it passed. |
 | `alias` | Vite's `resolve.alias` | Import aliases inside `.ts` schema files. tsconfig `paths` work without configuration. |
 | `compileAway` | `true` | Replace `new Validator(schema)` in your code with a validator compiled at build time, where that gives the same results. See below. Needs ata-validator 1.36.0; with an older version the runtime stays, and a warning is printed only if you set the option yourself. |
+| `sharedErrors` | `'auto'` | Where a compiled validator's error detail comes from. A compact module carries its verdict and its schema and takes its errors from `ata-validator/error-runtime`, which the bundle holds once. `'auto'` scans the project's sources at build start (skipping `node_modules`, build output and test files), adds up what every replaceable call would save, and uses the runtime when the total pays for it; a file the scan did not see decides alone, which takes a schema of several hundred properties. Measured on projects of 1 to 40 SchemaStore schemas, no bundle came out bigger than with `false`, and 40 schemas went from 724 KB to 405 KB gzipped, on ata-validator 1.51.0. `true` writes every schema that allows it compact, and `false` never. The scan adds little to the build, measured in separate processes with esbuild: 84 to 85 ms for three validators, 681 to 715 ms for forty. Errors and verdicts are the same either way; a rejected document costs more, since its errors come from an interpreted walk. Needs ata-validator 1.50.0; with an older one every module stays self-contained, as before. |
 
 ## compileAway: keep `new Validator`, drop the compiler
 
@@ -179,7 +180,8 @@ away this way.
 
 Where the code only ever calls `isValidObject()` or `isValidJSON()`, the plugin
 uses a smaller wrapper without the error machinery (needs ata-validator 1.40.0):
-a small app that only asks for a boolean bundles to 2.1 KB gzipped.
+a small app that only asks for a boolean bundles to 2.6 KB gzipped, against 12.1 KB with
+the full wrapper and 102.0 KB for the runtime, on 0.5.0 with ata-validator 1.40.0.
 
 The replacement answers `validate()`, `isValidObject()`, `validateJSON()` and
 `isValidJSON()` as a `Validator` with default options does: the same verdicts,
@@ -194,7 +196,11 @@ otherwise:
 - `Validator` is a named import from `ata-validator`;
 - the schema argument is an object literal, a top-level `const` bound to one,
   the default import of a relative `.json` file, or `defineSchema(...)` around
-  one of those;
+  one of those; calls to the `t` builder from `ata-validator/t` whose arguments are
+  such values are run at build time, since the builder returns plain JSON Schema,
+  and are marked pure so a schema only the replaced call used leaves the bundle
+  with the builder (`t.refine` and `t.recursive`, which take a function, stay on
+  the runtime);
 - there is no second argument, or it is `{ useDefaults: false }` (or
   `{ useDefaults: true }`), written as a literal or a top-level `const`; that
   one option needs ata-validator 1.37.0, and older versions keep such calls on

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@ata-project/unplugin",
-  "version": "0.5.0",
+  "version": "0.7.0",
   "description": "Build-time JSON Schema for Vite, Webpack, Rollup, Rolldown, esbuild and Rspack. compileAway takes the runtime compiler out of the bundle.",
   "type": "module",
   "main": "./src/index.js",
@@ -79,7 +79,7 @@
   "devDependencies": {
     "@ata-project/keywords": "^0.3.3",
     "@rspack/core": "^1.7.0",
-    "ata-validator": "^1.40.0",
+    "ata-validator": "^1.51.0",
     "esbuild": "^0.28.0",
     "rolldown": "^1.2.8",
     "rollup": "^4.60.0",

package/src/compile-away.js

@@ -24,11 +24,15 @@
 
 import fs from 'node:fs'
 import path from 'node:path'
+import zlib from 'node:zlib'
 import { parse } from '@babel/parser'
 import MagicString from 'magic-string'
 
 const METHODS = new Set(['validate', 'isValidObject', 'validateJSON', 'isValidJSON'])
 const NOT_STATIC = Symbol('not static')
+// The builders of ata-validator/t that take only values. refine and
+// recursive take a function and stay on the runtime.
+const T_PURE = new Set(['string', 'number', 'integer', 'boolean', 'null', 'literal', 'const', 'enum', 'array', 'tuple', 'record', 'object', 'optional', 'union', 'intersect', 'ref', 'any', 'unknown', 'never', 'pick', 'omit', 'partial', 'required', 'composite'])
 
 // Keys that hold child nodes. Types are left out on purpose: they are erased
 // before anything runs, so a name used in a type annotation says nothing about
@@ -134,9 +138,30 @@ function staticValue(node, ctx, seen = new Set()) {
       seen.delete(node.name)
       return v
     }
-    case 'CallExpression':
+    case 'CallExpression': {
       if (node.callee.type === 'Identifier' && node.callee.name === ctx.defineSchema && node.arguments.length === 1) return staticValue(node.arguments[0], ctx, seen)
+      // `t.object(...)` and the rest of the builder from ata-validator/t. It
+      // returns plain JSON Schema and holds no state, so a call whose
+      // arguments the build can read gives here what it gives at run time.
+      // The builders that take a function are left out.
+      const c = node.callee
+      if (ctx.t && ctx.tName && c.type === 'MemberExpression' && !c.computed && c.object.type === 'Identifier' &&
+          c.object.name === ctx.tName && ctx.declared.get(ctx.tName) === 1 && c.property.type === 'Identifier' && T_PURE.has(c.property.name) &&
+          typeof ctx.t[c.property.name] === 'function') {
+        const args = []
+        for (const a of node.arguments) {
+          if (a.type === 'SpreadElement') return NOT_STATIC
+          const v = staticValue(a, ctx, seen)
+          if (v === NOT_STATIC) return NOT_STATIC
+          args.push(v)
+        }
+        let v
+        try { v = ctx.t[c.property.name](...args) } catch { return NOT_STATIC }
+        ctx.tCalls.add(node)
+        return v
+      }
       return NOT_STATIC
+    }
   }
   return NOT_STATIC
 }
@@ -198,8 +223,84 @@ const verdictOnly = (methods) => [...methods].every((m) => VERDICT_METHODS.has(m
 
 // A verdict-only module hands out isValid alone, so the error function inside
 // it is unreferenced and the bundler drops it.
+// A compact module takes its errors from this runtime, which a bundle holds
+// once. A module is written compact only when it alone saves more than the
+// runtime adds, so the choice never makes a bundle bigger, whatever else the
+// bundle holds and in whatever order files are transformed. The bar is the
+// module's gzipped saving as this transform sees it (unminified source), and
+// it was set from minified esbuild bundles of one schema each: the runtime
+// paid for itself from about 22000 bytes of that saving (a 300-property
+// schema lost 1826 bytes at 19430, a 400-property one gained 1937 at 25127).
+// Below it, a schema stays self-contained; `sharedErrors: true` writes every
+// module compact, for an application with many schemas.
+const ERROR_RUNTIME = 'ata-validator/error-runtime'
+const ERROR_RUNTIME_COST = 24000
+const RUNTIME_IMPORT = /^import \{ createErrors as _ataCreateErrors(?:, compileSafe as _ataCompileSafe)? \} from [^\n]*\n/m
+
+// The scan and the transform weigh the same modules, and the module sources
+// come back as the same strings from the session's compile cache: each is
+// gzipped once per build (clearSizes runs at build start).
+const sizes = new Map()
+// The scan parses every source the transform parses again: once per build.
+const asts = new Map()
+function gzipSize(src) {
+  let n = sizes.get(src)
+  if (n === undefined) { n = zlib.gzipSync(src).length; if (sizes.size > 4096) sizes.clear(); sizes.set(src, n) }
+  return n
+}
+export function clearSizes() { sizes.clear(); asts.clear() }
+
+// The compact module when it is the choice for `mode`, else null:
+//   true      wherever the schema allows it
+//   'smaller' where it is smaller than the self-contained module (the build
+//             scan found that the application as a whole pays for the runtime)
+//   'auto'    where this module alone pays for the runtime
+//   false     never
+// With `collect`, nothing is chosen: the saving is recorded for the scan.
+function compactModuleFor(schema, full, ata, mode, collect) {
+  if (mode === false || !ata.compiledSharedErrors) return null
+  let src = null
+  try { src = ata.compiledModuleFor(schema, { format: 'esm', errorRuntime: ERROR_RUNTIME }) } catch { src = null }
+  if (!src || !RUNTIME_IMPORT.test(src)) return null
+  if (mode === true) return src
+  // The scan collects the pairs and weighs them all at once, in parallel
+  // (sharedErrorsPay); the transform finds the sizes cached.
+  if (collect) { collect.push([full, src]); return null }
+  const saving = gzipSize(full) - gzipSize(src)
+  if (mode === 'smaller') return saving > 0 ? src : null
+  return saving > ERROR_RUNTIME_COST ? src : null
+}
+
+// Whether an application pays for the shared runtime: the saving of every
+// replaceable call in its sources, against the runtime's cost. `sources` is a
+// list of [code, id]. A call the transform would not replace adds nothing.
+// Gzipping a module of a hundred kilobytes is the scan's largest cost; the
+// asynchronous zlib runs on libuv's thread pool, so the modules of a build
+// are weighed in parallel rather than one after the other.
+function gzipSizeAsync(src) {
+  const n = sizes.get(src)
+  if (n !== undefined) return Promise.resolve(n)
+  return new Promise((resolve, reject) => zlib.gzip(src, (err, out) => {
+    if (err) { reject(err); return }
+    if (sizes.size > 4096) sizes.clear()
+    sizes.set(src, out.length)
+    resolve(out.length)
+  }))
+}
+
+export async function sharedErrorsPay(sources, ata) {
+  const collect = []
+  for (const [code, id] of sources) {
+    try { compileAway(code, id, ata, { sharedErrors: 'auto', collect }) } catch {}
+  }
+  const savings = await Promise.all(collect.map(async ([full, compact]) => (await gzipSizeAsync(full)) - (await gzipSizeAsync(compact))))
+  let saving = 0
+  for (const x of savings) if (x > 0) saving += x
+  return { saving, pays: saving > ERROR_RUNTIME_COST, calls: collect.length }
+}
+
 function inlineModule(src, index, verdict) {
-  const body = src.split('\n').filter((l) => !/^export\s/.test(l)).join('\n')
+  const body = src.replace(RUNTIME_IMPORT, '').split('\n').filter((l) => !/^export\s/.test(l)).join('\n')
   return `const __ataCompiled${index} = (() => {\n${body}\nreturn ${verdict ? '{ isValid }' : '{ validate, isValid }'};\n})();\n`
 }
 
@@ -218,16 +319,26 @@ function compiledOptionsArg(node, ctx, supported) {
   return opts.useDefaults === false ? '{"useDefaults":false}' : ''
 }
 
-export function compileAway(code, id, ata) {
+export function compileAway(code, id, ata, options) {
+  const sharedErrors = options && options.sharedErrors !== undefined ? options.sharedErrors : 'auto'
+  const collect = options && Array.isArray(options.collect) ? options.collect : null
   const { compiledModuleFor, compiledSchemaFor } = ata
   const supportedOptions = Array.isArray(ata.compiledOptions) ? ata.compiledOptions : []
   if (!code.includes('ata-validator')) return null
-  let ast
-  try {
-    ast = parse(code, { sourceType: 'module', plugins: ['typescript', 'jsx'], errorRecovery: false })
-  } catch {
-    return null
+  // The scan keeps the tree it parsed for the transform, which takes it and
+  // lets it go, so a long dev session holds no old versions of a file.
+  let ast = asts.get(code)
+  if (ast !== undefined) {
+    if (!collect) asts.delete(code)
+  } else {
+    try {
+      ast = parse(code, { sourceType: 'module', plugins: ['typescript', 'jsx'], errorRecovery: false })
+    } catch {
+      ast = null
+    }
+    if (collect) asts.set(code, ast)
   }
+  if (ast === null) return null
   let validatorName = null
   let importDecl = null
   let defineSchema = null
@@ -244,6 +355,15 @@ export function compileAway(code, id, ata) {
     }
   }
   if (!validatorName) return null
+  let tName = null
+  for (const stmt of ast.program.body) {
+    if (stmt.type !== 'ImportDeclaration' || stmt.source.value !== 'ata-validator/t' || stmt.importKind === 'type') continue
+    for (const sp of stmt.specifiers) {
+      if (sp.type !== 'ImportSpecifier' || sp.importKind === 'type') continue
+      const imported = sp.imported.type === 'Identifier' ? sp.imported.name : sp.imported.value
+      if (imported === 't') tName = sp.local.name
+    }
+  }
   let withKeywords = null
   for (const stmt of ast.program.body) {
     if (stmt.type !== 'ImportDeclaration' || stmt.source.value !== '@ata-project/keywords' || stmt.importKind === 'type') continue
@@ -256,12 +376,14 @@ export function compileAway(code, id, ata) {
 
   const { declared, refs } = scan(ast)
   if (declared.get(validatorName) !== 1) return null
-  const ctx = { declared, refs, constants: constantsOf(ast, id), defineSchema, withKeywords, extendChecks: ata.compiledExtendChecks === true }
+  const ctx = { declared, refs, constants: constantsOf(ast, id), defineSchema, withKeywords, extendChecks: ata.compiledExtendChecks === true, t: ata.t || null, tName, tCalls: new Set() }
 
   const s = new MagicString(code)
   const modules = []
   const done = new Set()
   const wrappers = { full: false, verdict: false }
+  let usesRuntime = false
+  let usesSafeRe = false
   for (const r of refs) {
     if (r.node.name !== validatorName) continue
     const expr = r.parent
@@ -281,6 +403,10 @@ export function compileAway(code, id, ata) {
     if (!src) continue
     const index = modules.length
     const verdict = ata.compiledVerdict === true && verdictOnly(binding.methods)
+    // Code that never reads errors gets the verdict wrapper and no errors at
+    // all, so a compact module would only add the runtime.
+    const compact = verdict ? null : compactModuleFor(schema, src, ata, sharedErrors, collect)
+    if (compact) { src = compact; usesRuntime = true; if (compact.includes('_ataCompileSafe')) usesSafeRe = true }
     modules.push(inlineModule(src, index, verdict))
     const wrap = verdict ? '__ataFromCompiledVerdict' : '__ataFromCompiled'
     if (verdict) wrappers.verdict = true; else wrappers.full = true
@@ -310,8 +436,15 @@ export function compileAway(code, id, ata) {
     const clause = [def ? text(def) : null, named.length ? `{ ${named.map(text).join(', ')} }` : null].filter(Boolean).join(', ')
     s.overwrite(importDecl.start, importDecl.end, `import ${clause} from ${code.slice(importDecl.source.start, importDecl.source.end)}`)
   }
+  // The builder calls read here are pure, and marked so: a schema constant
+  // only the replaced call used, and the builder with it, can then leave the
+  // bundle.
+  for (const call of ctx.tCalls) s.appendLeft(call.start, '/*#__PURE__*/ ')
   if (wrappers.verdict) s.prepend(`import { fromCompiledVerdict as __ataFromCompiledVerdict } from 'ata-validator/compiled-verdict';\n`)
   if (wrappers.full) s.prepend(`import { fromCompiled as __ataFromCompiled } from 'ata-validator/compiled';\n`)
+  // compileSafe is the pattern engine a compact module with a pattern the
+  // platform RegExp does not run takes from the runtime instead of embedding.
+  if (usesRuntime) s.prepend(`import { createErrors as _ataCreateErrors${usesSafeRe ? ', compileSafe as _ataCompileSafe' : ''} } from '${ERROR_RUNTIME}';\n`)
   s.appendLeft(lastImportEnd, '\n' + modules.join(''))
   return { code: s.toString(), map: s.generateMap({ hires: true, source: id, includeContent: true }), replaced: done.size }
 }

package/src/core.js

@@ -74,6 +74,12 @@ async function loadAta() {
   // withKeywords from @ata-project/keywords registers one; from 1.40.0. Asked
   // of a wrapper itself, so an ata-validator without it keeps such calls on
   // the runtime.
+  // The ata-validator/t builder, run at build time on calls whose arguments
+  // the build can read.
+  let t = null
+  if (canCompileAway) {
+    try { const m = await import('ata-validator/t'); t = (m.default ?? m).t ?? m.t ?? null } catch { t = null }
+  }
   let compiledExtendChecks = false
   if (canCompileAway) {
     try {
@@ -82,8 +88,16 @@ async function loadAta() {
       compiledExtendChecks = typeof fromCompiled({ validate: () => ({ valid: true, errors: [] }), isValid: () => true }, {}, { useDefaults: false })._extendChecks === 'function'
     } catch { compiledExtendChecks = false }
   }
+  // Compact modules, from ata-validator 1.50.0: the verdict and the schema,
+  // with errors from ata-validator/error-runtime, which the bundle then holds
+  // once for every compact module in it.
+  let compiledSharedErrors = false
+  if (canCompileAway) {
+    try { compiledSharedErrors = typeof (await import('ata-validator/error-runtime')).createErrors === 'function' } catch { compiledSharedErrors = false }
+  }
   return {
     ...api,
+    compiledSharedErrors,
     toStandaloneModule: build.toStandaloneModule,
     compiledModuleFor: canCompileAway ? build.compiledModuleFor : null,
     compiledSchemaFor: canCompileAway ? build.compiledSchemaFor : null,
@@ -92,6 +106,7 @@ async function loadAta() {
     compiledOptions: canCompileAway && Array.isArray(build.compiledOptions) ? build.compiledOptions : [],
     compiledVerdict,
     compiledExtendChecks,
+    t,
   }
 }
 

package/src/index.d.ts

@@ -30,6 +30,16 @@ export interface Options {
    * Default: `true`; `false` turns it off.
    */
   compileAway?: boolean
+  /**
+   * Where a replaced validator's error detail comes from. `'auto'` (default)
+   * scans the project's sources at build start and writes modules compact
+   * (their verdict and schema, with errors from `ata-validator/error-runtime`,
+   * held once by the bundle) when all of them together save more than the
+   * runtime adds; `true` writes every module that can be compact; `false`
+   * never.
+   * Needs ata-validator 1.50.0.
+   */
+  sharedErrors?: 'auto' | boolean
 }
 
 export interface CompileResult {

package/src/index.js

@@ -10,10 +10,50 @@
 import path from 'node:path'
 import { createUnplugin } from 'unplugin'
 import { createSession, normalizeAlias, compile, __internal } from './core.js'
-import { compileAway } from './compile-away.js'
+import fs from 'node:fs'
+import { compileAway, sharedErrorsPay, clearSizes } from './compile-away.js'
 
 const SOURCE = /\.[cm]?[jt]sx?$/
 
+// Source files the application may import, for the sharedErrors scan: what
+// the transform would see, without node_modules, build output, tests and
+// dot-directories. Only files that mention ata-validator are kept.
+const SKIP_DIRS = new Set(['node_modules', 'dist', 'build', 'out', 'coverage', '.git', '.next', '.nuxt', '.output', '.svelte-kit', '.turbo', '.cache'])
+const TEST_FILE = /\.(test|spec)\.[cm]?[jt]sx?$/
+function scanSources(root) {
+  const out = []
+  const walk = (dir, depth) => {
+    if (depth > 12) return
+    let entries
+    try { entries = fs.readdirSync(dir, { withFileTypes: true }) } catch { return }
+    for (const e of entries) {
+      if (e.name.startsWith('.') || SKIP_DIRS.has(e.name)) continue
+      const p = path.join(dir, e.name)
+      if (e.isDirectory()) { if (e.name !== '__tests__' && e.name !== 'test' && e.name !== 'tests') walk(p, depth + 1); continue }
+      if (!e.isFile() || !SOURCE.test(e.name) || TEST_FILE.test(e.name)) continue
+      let code
+      try { if (fs.statSync(p).size > 1 << 20) continue; code = fs.readFileSync(p, 'utf8') } catch { continue }
+      if (code.includes('ata-validator')) out.push([code, p])
+    }
+  }
+  walk(root, 0)
+  return out
+}
+
+// The scan and the transform compile the same schemas: one compile each.
+function cachedApi(session, api) {
+  if (session.cachedApi && session.cachedApiFor === api) return session.cachedApi
+  const memo = new Map()
+  const compiledModuleFor = (schema, opts) => {
+    const key = JSON.stringify(schema) + '\u0000' + JSON.stringify(opts || {})
+    if (!memo.has(key)) memo.set(key, api.compiledModuleFor(schema, opts))
+    return memo.get(key)
+  }
+  session.cachedApi = { ...api, compiledModuleFor }
+  session.cachedApiFor = api
+  return session.cachedApi
+}
+
 export const unpluginFactory = (userOptions = {}) => {
   const session = createSession(userOptions)
 
@@ -25,6 +65,26 @@ export const unpluginFactory = (userOptions = {}) => {
       const { files, results } = await session.compileAll()
       const changed = results.filter((r) => r.changed).length
       session.logger?.info?.(`[unplugin-ata] compiled ${files.length} schema(s), ${changed} file(s) written`)
+      // sharedErrors 'auto' decides for the whole application, not file by
+      // file: the sources are scanned once here, and the runtime is used when
+      // every replaceable call together saves more than it costs.
+      session.sharedDecision = undefined
+      clearSizes()
+      session.scanned = null
+      if (userOptions.compileAway !== false && (userOptions.sharedErrors === undefined || userOptions.sharedErrors === 'auto')) {
+        const api = await session.api()
+        if (api.compiledModuleFor && api.compiledSharedErrors) {
+          const sources = scanSources(session.root || process.cwd())
+          session.scanned = new Set(sources.map(([, id]) => path.resolve(id)))
+          const { pays, saving, calls } = await sharedErrorsPay(sources, cachedApi(session, api))
+          // When the whole application does not pay for the runtime, no single
+          // call does either: the transform skips the compact attempt. A file
+          // the scan did not see (outside the root) decides alone, by the
+          // per-module rule, which never makes a bundle bigger either.
+          session.sharedDecision = pays ? 'smaller' : false
+          if (calls > 0) session.logger?.info?.(`[unplugin-ata] ${calls} validator(s) can share one error runtime, saving ${saving} bytes gzipped against its cost: ${pays ? 'shared' : 'kept self-contained'}`)
+        }
+      }
     },
 
     async watchChange(id) {
@@ -55,7 +115,9 @@ export const unpluginFactory = (userOptions = {}) => {
         }
         return null
       }
-      const out = compileAway(code, id.split('?')[0], api)
+      const file = id.split('?')[0]
+      const scanned = session.sharedDecision !== undefined && session.scanned && session.scanned.has(path.resolve(file))
+      const out = compileAway(code, file, cachedApi(session, api), { sharedErrors: scanned ? session.sharedDecision : userOptions.sharedErrors })
       return out ? { code: out.code, map: out.map } : null
     },