yuku-analyzer 0.5.37 → 0.5.39

package/README.md

@@ -1,15 +1,15 @@
 # yuku-analyzer
 
-Full semantic analysis for JavaScript and TypeScript: scopes, symbols, resolved references, closures, and cross-file module linking. Powered by [Yuku](https://github.com/yuku-toolchain/yuku), written in Zig.
+Full semantic analysis for JavaScript and TypeScript: scopes, symbols, resolved references, closures, and cross-file module linking, computed natively in Zig and queried as plain JavaScript objects. Powered by [Yuku](https://github.com/yuku-toolchain/yuku).
 
-The usual options are a hand-rolled scope tracker (fragile, per-file, re-bugged in every tool) or the TypeScript compiler (correct, but hundreds of milliseconds per file and a compiler-sized dependency). This is the fast path between them. One native call per file, then every query is plain JavaScript with zero per-query FFI cost, sub-millisecond on a typical file.
+**No single library gives you all of this.** Scopes and resolved references mean `eslint-scope` or `@typescript-eslint/scope-manager`. Cross-file go-to-definition means the TypeScript compiler or `ts-morph`. A parser sits under both. `yuku-analyzer` is all of them in one native pass behind one API.
+
+**At native speed.** Up to ~15× faster per file than `eslint-scope`, `@typescript-eslint/scope-manager`, and `@babel/traverse`, with zero per-query cost after the single native call. Stitch those separate tools together yourself and the gap only widens: each re-walks the AST, you re-parse to resolve across files, and you keep the indexes between them in sync by hand. `yuku-analyzer` pays all of that once, in Zig.
 
 ```bash
-npm install yuku-analyzer yuku-parser
+npm install yuku-analyzer
 ```
 
-`yuku-parser` is a peer dependency.
-
 ## Quick start
 
 ```js
@@ -35,9 +35,7 @@ console.log(def.module.path); // "lib.ts"
 console.log(def.symbol.has(SymbolFlags.Const)); // true
 ```
 
-## Why this exists
-
-JavaScript tooling that needs real semantics has had two options: track scopes by hand during a walk (fragile, incomplete, and re-implemented in every tool), or embed a full language service (heavy). `yuku-analyzer` is the missing middle: the exact scope and binding model of a production compiler, exposed to JavaScript as a small, fast object graph.
+## How it works
 
 Nothing semantic is reimplemented in JavaScript. The binder, scope tree, reference resolution, and module records are computed by the same well-tested native analyzer that powers the rest of Yuku, then shipped to JavaScript as one compact buffer that the JS side only decodes, through lazy zero-copy views. That handoff is usually where native tooling stalls: either every query pays an FFI round trip, or the semantics get a hand-written JS twin that slowly drifts. Here there is one implementation and one crossing, so it cannot drift, and the corner cases arrive already correct: catch-clause scope sharing, named function expression scopes, `var` hoist targets, TS declaration merging, value space versus type space, and write detection through destructuring patterns.
 
@@ -117,7 +115,9 @@ module.dependents;      // modules that import this file
 analyzer.diagnostics;   // e.g. "Module './lib.ts' has no export 'helpr'"
 ```
 
-Linking is automatic: every cross-file surface relinks on demand after files change. Call `analyzer.link()` explicitly if you want to control when the work happens.
+Linking is automatic: every cross-file surface relinks on demand after files change. Re-adding a path replaces its module with a new object and relinks, and removing it relinks too. Call `analyzer.link()` explicitly if you want to control when the work happens.
+
+Module records and linking model ECMAScript and TypeScript module syntax, so CommonJS `require` and `module.exports` are ordinary code rather than records and take no part in linking. Per-file scopes, symbols, and references are still computed for CommonJS sources.
 
 Module resolution is pluggable. The default resolves relative specifiers among added files with standard extension and index probing. Pass your own resolver for anything else:
 
@@ -135,7 +135,7 @@ Concretely, on an Apple M-series machine: parsing plus complete semantic analysi
 
 ## TypeScript
 
-Everything is fully typed. Visitor handlers receive exact node types, AST types come from `yuku-parser`, and the semantic surface (`Module`, `Scope`, `Symbol`, `Reference`, `Import`, `Export`, `Capture`) is exported:
+Everything is fully typed. Visitor handlers receive exact node types, and the semantic surface (`Module`, `Scope`, `Symbol`, `Reference`, `Import`, `Export`, `Capture`) is exported:
 
 ```ts
 import type { Module, Symbol, Capture } from "yuku-analyzer";

package/analyzer.js

@@ -2,9 +2,7 @@ import { Module, SymbolFlags } from "./module.js";
 
 const RESOLVE_EXTENSIONS = [".tsx", ".ts", ".jsx", ".js", ".mts", ".mjs", ".cts", ".cjs"];
 
-// the third resolution outcome of ResolveExport (16.2.1.7.2.2): more
-// than one `export *` declaration supplies the name through different
-// bindings. distinct from null (not found / circular).
+// name supplied ambiguously by multiple export * (ResolveExport, 16.2.1.7.2.2)
 const AMBIGUOUS = Symbol("ambiguous");
 
 export class Analyzer {
@@ -13,6 +11,10 @@ export class Analyzer {
   #diagnostics = [];
   #dirty = false;
   #linking = false;
+  // origin defining symbol -> importing local symbols that resolve to it,
+  // across every module. built once per link so referencesOf is a lookup
+  // rather than a per-call walk over every import in the graph.
+  #importersByOrigin = new Map();
 
   constructor(options = {}) {
     this.#resolve = options.resolve ?? defaultResolve(this.#modules);
@@ -44,10 +46,7 @@ export class Analyzer {
     return this.#diagnostics;
   }
 
-  // the static half of spec linking. resolves every module request to an
-  // added module, wires the dependency graph, and reports binding
-  // validation failures as diagnostics rather than throwing. unresolvable
-  // specifiers are skipped so a partial graph still links.
+  // static spec linking, reports failures as diagnostics rather than throwing
   link() {
     this.#dirty = false;
     this.#linking = true;
@@ -74,18 +73,40 @@ export class Analyzer {
         record._resolved = this.#resolveModule(record.specifier, module, false);
         if (record._resolved === null) continue;
         this.#wire(module, record._resolved);
-        // `export * as ns` (fromName null) trivially resolves and bare
-        // `export *` conflicts are deferred to use sites, per spec
+        // export * as ns resolves trivially, bare export * conflicts deferred to use sites
         if (record.fromName !== null) {
           this.#validate(module, record, record.fromName, "Re-export");
         }
       }
     }
+    this.#indexImporters();
     this.#linking = false;
   }
 
-  // reports a record whose name does not resolve, or resolves
-  // ambiguously, against its (already resolved) source module
+  // buckets every importing local under the origin symbol it resolves to,
+  // computing each definitionOf once. referencesOf then reads the bucket
+  // instead of re-walking the whole import graph on every call.
+  #indexImporters() {
+    this.#importersByOrigin = new Map();
+    for (const module of this.#modules.values()) {
+      for (const record of module.imports) {
+        const local = record.local;
+        if (local === null) continue;
+        const definition = this.definitionOf(local);
+        // a symbol object is unique to its (module, id), so the origin
+        // symbol alone keys the bucket; the module is implied.
+        if (definition === null || definition.symbol === null) continue;
+        let importers = this.#importersByOrigin.get(definition.symbol);
+        if (importers === undefined) {
+          importers = [];
+          this.#importersByOrigin.set(definition.symbol, importers);
+        }
+        importers.push(local);
+      }
+    }
+  }
+
+  // report a record whose name fails to resolve or resolves ambiguously
   #validate(module, record, name, what) {
     const resolution = this.#resolveExport(record._resolved, name, []);
     if (resolution !== null && resolution !== AMBIGUOUS) return;
@@ -102,10 +123,7 @@ export class Analyzer {
     });
   }
 
-  // follows an import binding to its defining module and symbol via
-  // ResolveExport (InitializeEnvironment step 7.c). namespace
-  // resolutions yield { module, symbol: null }; unresolved or ambiguous
-  // chains yield null.
+  // follow an import binding to its defining module and symbol (ResolveExport, InitializeEnvironment 7.c)
   definitionOf(symbol) {
     this._ensureLinked();
     let module = symbol.module;
@@ -138,19 +156,14 @@ export class Analyzer {
     for (const reference of origin.symbol.references) {
       out.push({ module: origin.module, reference });
     }
-    for (const module of this.#modules.values()) {
-      if (module === origin.module) continue;
-      for (const record of module.imports) {
-        if (record.local === null) continue;
-        const definition = this.definitionOf(record.local);
-        if (
-          definition !== null &&
-          definition.symbol === origin.symbol &&
-          definition.module === origin.module
-        ) {
-          for (const reference of record.local.references) {
-            out.push({ module, reference });
-          }
+    const importers = this.#importersByOrigin.get(origin.symbol);
+    if (importers !== undefined) {
+      for (const local of importers) {
+        // a self-import resolving back to the origin module is already
+        // covered by the direct references above
+        if (local.module === origin.module) continue;
+        for (const reference of local.references) {
+          out.push({ module: local.module, reference });
         }
       }
     }
@@ -185,21 +198,8 @@ export class Analyzer {
     if (!to._dependents.includes(from)) to._dependents.push(from);
   }
 
-  // ResolveExport(exportName, resolveSet), 16.2.1.7.2.2: resolves a
-  // name to the binding that defines it, following named re-export and
-  // `export *` chains. returns { module, symbol, namespace } (the
-  // spec's ResolvedBinding, with namespace: true for its namespace
-  // [[BindingName]]), null when unresolvable or circular, or AMBIGUOUS.
-  //
-  // local and indirect entries live in one name-keyed map (_exportMap);
-  // duplicate export names are an early error, so the spec's step 5/6
-  // partition cannot change the outcome. ParseModule step 10.a.ii
-  // (rewriting an export of an imported binding through its original
-  // module) is applied at resolution time instead of collection time.
+  // ResolveExport, 16.2.1.7.2.2
   #resolveExport(module, exportName, resolveSet) {
-    // step 3: a repeated (module, exportName) pair is a circular import
-    // request. the pair key still permits re-entering a module for a
-    // different name, which renaming re-export chains legitimately do.
     for (const entry of resolveSet) {
       if (entry.module === module && entry.exportName === exportName) return null;
     }
@@ -210,8 +210,7 @@ export class Analyzer {
       if (direct.specifier === null) {
         const local = direct.local;
         if (local !== null && (local.flags & SymbolFlags.Import) !== 0) {
-          // export of an imported binding: resolve through the original
-          // module (the ParseModule rewrite)
+          // export of an imported binding resolves through the original module (ParseModule 10.a.ii)
           const record = module._importOfSymbol(local.id);
           if (record !== undefined) {
             const imported =
@@ -221,22 +220,19 @@ export class Analyzer {
             return this.#resolveExport(imported, record.name, resolveSet);
           }
         }
-        // step 5: the module provides the direct binding
         return { module, symbol: local, namespace: false };
       }
       const next = direct._resolved ?? this.#resolveModule(direct.specifier, module, true);
       if (next === null) return null;
-      // step 6: `export * as ns` binds the namespace, named re-exports
-      // follow the chain under the source name
+      // namespace re-export binds the namespace, named ones follow the chain
       if (direct.isNamespaceReexport) return { module: next, symbol: null, namespace: true };
       return this.#resolveExport(next, direct.fromName, resolveSet);
     }
 
-    // step 7: "default" is never satisfied through `export *`
+    // default never crosses export *
     if (exportName === "default") return null;
 
-    // steps 8-9: identical star bindings collapse, different ones are
-    // ambiguous
+    // identical star bindings collapse, otherwise ambiguous
     let starResolution = null;
     for (const star of module._starExports()) {
       const next = star._resolved ?? this.#resolveModule(star.specifier, module, true);

package/decode.js

@@ -1275,6 +1275,8 @@ function decode(buffer, source) {
       }
     }
     let o = ((dp + 3) & ~3) >> 2;
+    if (o + 8 > _u32.length)
+      throw new RangeError("yuku-analyzer: truncated semantic sub-header");
     const scopeCount = _u32[o], symbolCount = _u32[o + 1],
           referenceCount = _u32[o + 2], declNodeCount = _u32[o + 3],
           importCount = _u32[o + 4], exportCount = _u32[o + 5],
@@ -1294,6 +1296,8 @@ function decode(buffer, source) {
     o += exportCount * 10;
     const nodeScopes = _u32.subarray(o, o + nodeScopeCount);
     o += nodeScopeCount;
+    if (o > _u32.length)
+      throw new RangeError("yuku-analyzer: truncated semantic sections");
     const _id = (v) => v === NULL ? null : v;
     return (_semView = {
       scope: {

package/engine.js

@@ -55,8 +55,7 @@ export class WalkContext {
     this._removed = true;
   }
   insertBefore(node) {
-    // advance past the insert so the current node keeps its turn and the
-    // new sibling is not visited
+    // advance past the insert so the current node keeps its turn
     this.#insert(node, 0);
     this._frame.i++;
   }
@@ -86,10 +85,6 @@ function createDispatch(visitors) {
   return { enter, leave, typed: (type) => concrete.get(type) };
 }
 
-/**
- * Walk an AST depth-first, dispatching to typed visitors and mutating
- * in place. Returns the root.
- */
 export function walk(root, visitors, state) {
   _walk(root, visitors, state, new WalkContext());
   return root;
@@ -107,8 +102,6 @@ export function _walk(root, visitors, state, ctx) {
     ctx._frame = frame;
   }
 
-  // swaps the current node in its parent slot and continues the walk on
-  // the replacement
   function applyReplace(parent, key, list, frame) {
     const next = ctx._replacement;
     ctx._replacement = null;

package/index.d.ts

@@ -388,7 +388,10 @@ interface Module {
   symbolOf(node: Node): Symbol | null;
   /** The reference recorded for an identifier node, or null. */
   referenceOf(node: Node): Reference | null;
-  /** The innermost scope whose extent contains `node`. */
+  /**
+   * The innermost scope whose extent contains `node`, or the module's
+   * root scope for a node not produced by this module's analysis.
+   */
   scopeOf(node: Node): Scope;
   /**
    * The node that structurally contains `node`. Null at the program
@@ -432,7 +435,7 @@ interface Module {
 
   /** Collects every node of the given type(s), in source order. */
   findAll<K extends NodeType>(type: K): NodeOfType<K>[];
-  findAll<K extends NodeType>(types: readonly K[]): NodeOfType<K>[];
+  findAll<K extends NodeType>(types: Iterable<K>): NodeOfType<K>[];
 
   /** Import records, in source order. */
   readonly imports: Import[];

package/module.js

@@ -1,6 +1,4 @@
-// One analyzed file, the AST plus the scope/symbol/reference/import/
-// export object graph, built lazily over the decoded analyzer buffer.
-
+// one analyzed file, its AST plus a lazily built semantic graph
 import binding from "./binding.js";
 import { decode, SymbolFlags } from "./decode.js";
 import { walkModule } from "./walk.js";
@@ -316,8 +314,7 @@ export class Module {
     return this.scopes[this.#sem.nodeScope(index)];
   }
 
-  // the node that structurally contains `node`, or null at the program
-  // root or for a node not part of this module's AST.
+  // structural parent, or null at the root or for a foreign node
   parentOf(node) {
     const index = this.#r.indexOf(node);
     if (index === undefined) return null;
@@ -333,11 +330,7 @@ export class Module {
     return null;
   }
 
-  // GetExportedNames(exportStarSet), 16.2.1.7.2.1: every name exported
-  // directly or through `export *` chains. ambiguous star names are not
-  // filtered (spec note), "default" never crosses a star boundary, and
-  // circular `export *` terminates via exportStarSet. TS `export =` /
-  // `export as namespace` are not ESM export names and do not appear.
+  // GetExportedNames, 16.2.1.7.2.1
   exportedNames(exportStarSet = new Set()) {
     if (exportStarSet.has(this)) return [];
     exportStarSet.add(this);
@@ -352,14 +345,7 @@ export class Module {
     return [...names];
   }
 
-  // the free variables of a function: every binding referenced inside
-  // `fn` (nested functions included, value positions only) that is
-  // declared outside its scope subtree, deduplicated by symbol with
-  // `isWritten` OR-ed across the references. only bindings count.
-  // `this`, `arguments`, and unresolved/global names carry no symbol
-  // and never appear. module-scope and import bindings count like any
-  // other outer binding. computed method keys evaluate in the enclosing
-  // scope, outside the function node's span, so they are not captures.
+  // outer bindings a function closes over, deduped by symbol
   capturesOf(fn) {
     const index = this.#r.indexOf(fn);
     if (index === undefined) {
@@ -447,11 +433,7 @@ export class Module {
     return this.#symbolReferences[symbolId];
   }
 
-  // the spec's export-entry partition (ParseModule, 16.2.1.7.1): a
-  // name-keyed map of the local + indirect entries and the star-entry
-  // list (`export *` only). name-less records (`export *`, TS
-  // `export =` / `export as namespace`) stay out of the map. duplicate
-  // names are an early error; in recovery trees the first record wins.
+  // export-entry partition (ParseModule, 16.2.1.7.1)
   _exportMap() {
     if (this.#exportMap === null) {
       const map = new Map();

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "yuku-analyzer",
-  "version": "0.5.37",
-  "description": "High-performance JavaScript/TypeScript semantic analyzer: per-file scopes, symbols, references, and cross-file module linking",
+  "version": "0.5.39",
+  "description": "Full JavaScript and TypeScript semantic analysis: scopes, symbols, resolved references, closures, and cross-file module linking, computed natively in Zig and queried as plain JavaScript objects",
   "license": "MIT",
   "repository": {
     "type": "git",
@@ -21,17 +21,17 @@
     "decode.js"
   ],
   "optionalDependencies": {
-    "@yuku-analyzer/binding-linux-x64-gnu": "0.5.37",
-    "@yuku-analyzer/binding-linux-arm64-gnu": "0.5.37",
-    "@yuku-analyzer/binding-linux-arm-gnu": "0.5.37",
-    "@yuku-analyzer/binding-linux-x64-musl": "0.5.37",
-    "@yuku-analyzer/binding-linux-arm64-musl": "0.5.37",
-    "@yuku-analyzer/binding-linux-arm-musl": "0.5.37",
-    "@yuku-analyzer/binding-darwin-x64": "0.5.37",
-    "@yuku-analyzer/binding-darwin-arm64": "0.5.37",
-    "@yuku-analyzer/binding-win32-x64": "0.5.37",
-    "@yuku-analyzer/binding-win32-arm64": "0.5.37",
-    "@yuku-analyzer/binding-freebsd-x64": "0.5.37"
+    "@yuku-analyzer/binding-linux-x64-gnu": "0.5.39",
+    "@yuku-analyzer/binding-linux-arm64-gnu": "0.5.39",
+    "@yuku-analyzer/binding-linux-arm-gnu": "0.5.39",
+    "@yuku-analyzer/binding-linux-x64-musl": "0.5.39",
+    "@yuku-analyzer/binding-linux-arm64-musl": "0.5.39",
+    "@yuku-analyzer/binding-linux-arm-musl": "0.5.39",
+    "@yuku-analyzer/binding-darwin-x64": "0.5.39",
+    "@yuku-analyzer/binding-darwin-arm64": "0.5.39",
+    "@yuku-analyzer/binding-win32-x64": "0.5.39",
+    "@yuku-analyzer/binding-win32-arm64": "0.5.39",
+    "@yuku-analyzer/binding-freebsd-x64": "0.5.39"
   },
   "keywords": [
     "analyzer",
@@ -49,6 +49,6 @@
     "typescript"
   ],
   "dependencies": {
-    "@yuku-toolchain/types": "0.5.32"
+    "@yuku-toolchain/types": "0.5.37"
   }
 }