eyeprolog 1.3.67 → 1.3.68

package/README.md

@@ -159,12 +159,14 @@ variable representatives and ordinary Prolog backtracking, and the REPL projects
 module `attribute_goals//1` hooks as residual goals. See
 [`examples/attributed-variables.pl`](examples/attributed-variables.pl).
 
-This is the first migration stage toward reusing Markus Triska's MIT-licensed
+The second migration stage toward reusing Markus Triska's MIT-licensed
 [Scryer `library(clpz)`](https://github.com/mthom/scryer-prolog/blob/master/src/lib/clpz.pl)
-instead of maintaining a separate constraint solver in `src/clpz.js`. The
-current CLP(Z) implementation remains active while compatibility gaps such as
-Scryer's compile-time term/goal expansion and supporting libraries are closed.
-The staged plan and exact compatibility boundary are documented in
+is now in place as well. Normal-profile loading supports user-defined
+`term_expansion/2` and `goal_expansion/2`, clause-list generation, and
+`expand_term/2` for processor DCG lowering, including the qualified hook shape
+used by Scryer libraries. The current CLP(Z) implementation remains active while
+the remaining Scryer support-library surface is brought across. The staged plan
+and exact compatibility boundary are documented in
 [`src/CLPZ-MIGRATION.md`](src/CLPZ-MIGRATION.md).
 
 Recursion through negation is explicit. EyeProlog provides `tnot/1` for

package/package.json

@@ -3,7 +3,7 @@
   "publishConfig": {
     "access": "public"
   },
-  "version": "1.3.67",
+  "version": "1.3.68",
   "description": "EyeProlog turns facts and rules into answers and proofs.",
   "type": "module",
   "main": "./index.js",

package/src/ARCHITECTURE.md

@@ -7,8 +7,8 @@ higher-level frontends.
 
 1. **Kernel representation and syntax** — `term.js`, `number-value.js`,
    `syntax-scan.js`, `parser.js`, `write.js`, `errors.js`.
-2. **Program preparation** — `program.js` plus `program-analysis.js` and
-   `program-indexing.js`. Static recursion/Datalog/WFS classification lives in
+2. **Program preparation** — `program.js`, `source-expansion.js`, plus
+   `program-analysis.js` and `program-indexing.js`. Static recursion/Datalog/WFS classification lives in
    `program-analysis.js`; compact clauses and candidate indexes live in
    `program-indexing.js`.
 3. **Execution** — `solver.js`, `cleanup.js`, `io.js`, `datalog.js`, `wfs.js`,
@@ -48,6 +48,16 @@ needed by Scryer's `library(atts)` and `library(clpz)`. The current CLP(Z)-speci
 `Env._clpz` store remains transitional until the Scryer Prolog implementation can
 replace it without losing test or performance parity; see `src/CLPZ-MIGRATION.md`.
 
+`source-expansion.js` is the explicit compile-time execution boundary. When an
+already-loaded `term_expansion/2` or `goal_expansion/2` hook exists, program
+preparation invokes it with a fresh bounded `Solver` against the partially built
+program. `solver.js` therefore imports clause-selection primitives directly from
+`program-indexing.js` rather than importing the `program.js` facade. This is the
+one deliberate preparation-to-execution service edge and keeps the JavaScript
+import graph acyclic while avoiding a second meta-interpreter solely for source
+expansion. `expansion-builtins.js` provides the lower-level `expand_term/2` DCG
+service without depending on `Solver`.
+
 The JavaScript runtime stays flat directly under `src/`; the existing `src/lib/`
 contains Prolog library sources rather than JavaScript runtime modules. The architecture
 test rejects JavaScript import cycles under `src/`. Cleanup lifecycle hooks are

package/src/CLPZ-MIGRATION.md

@@ -32,20 +32,40 @@ attributes, post-binding goals, backtracking, `term_attributed_variables/2`, and
 REPL projection through the conventional module `attribute_goals//1` hook.
 `examples/attributed-variables.pl` is the runnable source-level example.
 
-## What still prevents a drop-in Scryer `clpz.pl`
+## Stage 2: source expansion substrate
+
+This stage is implemented. Normal-profile source loading now invokes already
+loaded module-local and `user` `term_expansion/2` and `goal_expansion/2` hooks
+left-to-right. A term expansion may yield one clause or a list of clauses, and
+`expand_term/2` exposes the processor's DCG lowering so a hook can generate the
+ordinary clauses used by Scryer's CLP(Z) parser generators. Goal expansion is
+recursive across control constructs and preserves sharing with variables in the
+surrounding clause.
+
+Module-qualified hook clauses such as `user:goal_expansion/2` are also compiled
+with the correct split between procedure ownership and lexical body context. A
+hook defined from a library module therefore lives in `user` while its
+unqualified helper calls still resolve in the defining library. The compiler
+executes hooks with a fresh bounded solver against the partially built program;
+`solver.js` imports clause-selection helpers directly from `program-indexing.js`,
+so this compile-time service remains acyclic. Strict ISO Part 1 mode does not run
+these extension hooks.
 
-The upstream solver is not only an attributed-variable client. It also relies on
-compile-time facilities that EyeProlog does not yet provide generically:
+The regression corpus covers clause-list generation, variable-sharing goal
+expansion, qualified hook lexical context, and the Scryer pattern where a
+`term_expansion/2` hook calls `expand_term/2` to generate a DCG predicate.
+
+## What still prevents a drop-in Scryer `clpz.pl`
 
-- user-defined `term_expansion/2` and `goal_expansion/2` during module loading;
-- the generated predicates at the end of Scryer's `clpz.pl` and its custom Duo
-  DCG `++>` expansion;
-- the supporting Scryer library surface (`assoc`, `pairs`, `between`, `dcgs`,
-  `terms`, `error`, `si`, `freeze`, `arithmetic`, `debug`, and `format`) at the
-  interfaces expected by that source;
+The remaining compatibility work is the supporting Scryer library surface
+(`assoc`, `pairs`, `between`, `dcgs`, `terms`, `error`, `si`, `freeze`,
+`arithmetic`, `debug`, and `format`) at the interfaces expected by that source,
+plus any small builtin gaps exposed while loading those modules. The generated
+predicates and custom Duo DCG `++>` expansion are no longer VM/compiler
+blockers: Scryer's own `term_expansion/2` can now perform that lowering.
 
-These are compatibility gaps, not reasons to add more CLP(Z)-specific code to
-`term.js` or `solver.js`.
+These are reusable library compatibility gaps, not reasons to add more
+CLP(Z)-specific code to `term.js` or `solver.js`.
 
 ## Migration rule
 

package/src/expansion-builtins.js

@@ -0,0 +1,99 @@
+// Generic source-expansion helpers exposed to Prolog code.
+//
+// expand_term/2 provides the system part of source expansion that libraries
+// such as Scryer's CLP(Z) use while generating predicates. User-defined
+// term_expansion/2 and goal_expansion/2 are invoked by program preparation in
+// source-expansion.js; keeping this builtin independent of Solver avoids an
+// import cycle through the standard-library registry.
+
+import {
+  ATOM, COMPOUND, Env, atom, compound, deref, flattenConjunction, unify,
+} from './term.js';
+import { expandDcgRuleClause } from './dcg.js';
+import { PrologError } from './errors.js';
+
+function conjunctionFromGoals(goals) {
+  if (goals.length === 0) return atom('true');
+  let body = goals[goals.length - 1];
+  for (let index = goals.length - 2; index >= 0; index--) {
+    body = compound(',', [goals[index], body]);
+  }
+  return body;
+}
+
+export function clauseToSourceTerm(clause) {
+  if (clause == null || clause.kind === 'quad') return null;
+  if (!Array.isArray(clause.body) || clause.body.length === 0) return clause.head;
+  return compound(':-', [clause.head, conjunctionFromGoals(clause.body)]);
+}
+
+export function sourceTermToClauses(term, env = new Env()) {
+  term = deref(term, env);
+  const list = sourceTermList(term, env);
+  if (list != null) {
+    const clauses = [];
+    for (const item of list) clauses.push(...sourceTermToClauses(item, env));
+    return clauses;
+  }
+  if (term.type === COMPOUND && term.name === ':-' && term.arity === 2) {
+    const head = deref(term.args[0], env);
+    const body = deref(term.args[1], env);
+    return [{ head, body: body.type === ATOM && body.name === 'true' ? [] : flattenConjunction(body) }];
+  }
+  if (term.type === COMPOUND && term.name === ':-' && term.arity === 1) {
+    return [{ head: term, body: [] }];
+  }
+  if (term.type === COMPOUND && term.name === '?-' && term.arity >= 1) {
+    return [{ head: term, body: [] }];
+  }
+  if (term.type !== ATOM && term.type !== COMPOUND) {
+    throw new PrologError('type_error(callable)', term);
+  }
+  return [{ head: term, body: [] }];
+}
+
+function sourceTermList(term, env) {
+  const items = [];
+  const seen = new Set();
+  let cursor = deref(term, env);
+  while (cursor.type === COMPOUND && cursor.name === '.' && cursor.arity === 2) {
+    if (seen.has(cursor)) throw new PrologError('type_error(list)', term);
+    seen.add(cursor);
+    items.push(deref(cursor.args[0], env));
+    cursor = deref(cursor.args[1], env);
+  }
+  if (cursor.type === ATOM && cursor.name === '[]') return items;
+  return null;
+}
+
+export function systemExpandTerm(term, module = 'user') {
+  const resolved = deref(term, new Env());
+  if (resolved.type === COMPOUND && resolved.name === '-->' && resolved.arity === 2) {
+    const expanded = expandDcgRuleClause({ head: resolved, body: [] }, module);
+    if (expanded == null) return resolved;
+    return clauseToSourceTerm(expanded);
+  }
+  return resolved;
+}
+
+function* expandTermBuiltin({ goal, env }) {
+  const input = deref(goal.args[0], env);
+  if (input.type === 'var') throw new PrologError('instantiation_error');
+  const expanded = systemExpandTerm(input, goal.module ?? 'user');
+  const next = env.clone();
+  if (unify(goal.args[1], expanded, next)) yield next;
+}
+
+function* expandGoalBuiltin({ goal, env }) {
+  const input = deref(goal.args[0], env);
+  if (input.type === 'var') throw new PrologError('instantiation_error');
+  const next = env.clone();
+  if (unify(goal.args[1], input, next)) yield next;
+}
+
+export const expansionBuiltins = {
+  register(registry) {
+    registry.add('expand_term', 2, expandTermBuiltin, { deterministic: true, eyePrologLibrary: true });
+    registry.add('expand_goal', 2, expandGoalBuiltin, { deterministic: true, eyePrologLibrary: true });
+  },
+};

package/src/parser.js

@@ -1119,6 +1119,32 @@ class Parser {
         head = items.pop();
         while (items.length > 0) head = compound(',', [items.pop(), head]);
       }
+      // The clause grammar keeps priority-1200 neck/DCG tokens outside the
+      // initial head parse. A user-defined operator at priority 1199 or 1200,
+      // however, can itself be the outermost source term. Scryer's CLP(Z) uses
+      // exactly this shape for its private `++>` Duo-DCG rules. Fold such a
+      // custom low-strength infix operator back into the source term while
+      // leaving the processor-defined :-, -->, and ?- dispatch below intact.
+      const topLevelOperator = this.operatorTokenName();
+      const topLevelInfo = topLevelOperator == null ? null : this.infixOperators.get(topLevelOperator);
+      if (topLevelInfo && topLevelInfo.precedence < 3 &&
+          ![':-', '-->', '?-'].includes(topLevelOperator)) {
+        this.advance();
+        const right = this.parseTerm(
+          topLevelInfo.associativity === 'right' ? topLevelInfo.precedence : topLevelInfo.precedence + 1,
+          true,
+          true,
+          false,
+        );
+        head = compound(topLevelOperator, [head, right]);
+        if (topLevelInfo.associativity === 'none') {
+          const nextOperator = this.operatorTokenName();
+          if (this.infixOperators.get(nextOperator)?.precedence === topLevelInfo.precedence) {
+            throw new Error(`parse line ${this.token.line}: non-associative operator ${topLevelOperator} requires parentheses`);
+          }
+        }
+      }
+
       // Parentheses and other ordinary term syntax may hide the surface ?-
       // token from the program-level dispatch above.  Once the complete head
       // term has been parsed, recognize the same ?-/1 or ?-/2 structure here.

package/src/program.js

@@ -21,6 +21,7 @@ import {
   eyePrologInteropLibraryModules,
 } from './standard-library.js';
 import { expandDcgRuleClause } from './dcg.js';
+import { expandClauseGoals, expandSourceClause } from './source-expansion.js';
 import {
   CompactBinaryClause, clauseBodyLength, clauseHasCut, compactHeadArgName, compactHeadArgType, modulePredicateKey,
   indexCompactOne, indexOne, isCompactBinaryClause, makeArgumentIndex, rebuildGroupIndexes, termContainsCut, termHasNoVariables,
@@ -1069,18 +1070,56 @@ function loadSourceIntoBuilder(builder, source, options, ensured, loadedModules,
     builder.addClauses(batch);
     batch.length = 0;
   };
-  const accept = (clause) => {
+
+  const processClause = (inputClause) => {
+    let clause = inputClause;
     if (clause?.kind === 'quad') {
       flush();
       clause.module = context.module;
       builder.addClauses([clause]);
       return;
     }
+
+    // User source expansion is a normal-profile compile-time facility. Hooks
+    // execute against the clauses that have already been prepared, matching
+    // Prolog's left-to-right source loading model. Strict ISO Part 1 keeps its
+    // processor-defined preparation semantics unchanged.
+    if (!builder.program.strictIso) {
+      const expansion = expandSourceClause(builder.program, clause, context.module, {
+        module: context.module,
+        filename: options.filename ?? '<input>',
+      });
+      if (expansion.expanded) {
+        for (const generated of expansion.clauses) processPreparedClause(generated);
+        return;
+      }
+    }
+    processPreparedClause(clause);
+  };
+
+  const processPreparedClause = (inputClause) => {
+    let clause = inputClause;
+    const lexicalModule = context.module;
+
     // Grammar-rule expansion belongs to ISO/IEC TS 13211-3 rather than the
     // Part 1 strict-core language.  In strict core mode -->/2 remains the
     // ordinary predefined operator term from Table 7 and is not rewritten.
-    const grammarClause = builder.program.strictIso ? null : expandDcgRuleClause(clause, context.module);
+    const grammarClause = builder.program.strictIso ? null : expandDcgRuleClause(clause, lexicalModule);
     if (grammarClause) clause = grammarClause;
+
+    // A source module qualifier on a clause head selects the procedure owner,
+    // while unqualified calls in the body retain the lexical module of the
+    // source text. This distinction is essential for conventional hooks such
+    // as user:term_expansion/2 defined from library modules.
+    clause = normalizeQualifiedClauseHead(clause, lexicalModule);
+
+    if (!builder.program.strictIso && !isDirectiveClause(clause)) {
+      expandClauseGoals(builder.program, clause, lexicalModule, {
+        module: lexicalModule,
+        filename: options.filename ?? '<input>',
+      });
+    }
+
     const moduleDeclaration = moduleDirective(clause);
     if (moduleDeclaration) {
       flush();
@@ -1121,10 +1160,20 @@ function loadSourceIntoBuilder(builder, source, options, ensured, loadedModules,
       clause.module ??= context.module;
       clause.textUnit ??= context.textUnit;
       if (!isDirectiveClause(clause)) {
-        for (const goal of clause.body) annotateGoalModule(goal, clause.module);
+        const bodyModule = clause.lexicalModule ?? lexicalModule;
+        for (const goal of clause.body) annotateGoalModule(goal, bodyModule);
+      }
+      const isExpansionHook = !isDirectiveClause(clause) &&
+        clause.head?.arity === 2 && ['term_expansion', 'goal_expansion'].includes(clause.head?.name);
+      if (isExpansionHook) {
+        // A hook becomes visible to the very next source term, so it cannot
+        // wait in the large streaming build batch.
+        flush();
+        builder.addClauses([clause]);
+      } else {
+        batch.push(clause);
+        if (batch.length >= PROGRAM_BUILD_BATCH_SIZE) flush();
       }
-      batch.push(clause);
-      if (batch.length >= PROGRAM_BUILD_BATCH_SIZE) flush();
       return;
     }
     flush();
@@ -1152,15 +1201,32 @@ function loadSourceIntoBuilder(builder, source, options, ensured, loadedModules,
       batch[batch.length - 1].textUnit = context.textUnit;
       if (batch.length >= PROGRAM_BUILD_BATCH_SIZE) flush();
     };
-    const parsed = tryParseClausesFastInto(source, accept, acceptBinary, options);
+    const parsed = tryParseClausesFastInto(source, processClause, acceptBinary, options);
     if (parsed) flush();
     return parsed;
   }
-  parseClausesInto(source, options, accept);
+  parseClausesInto(source, options, processClause);
   flush();
   return true;
 }
 
+function normalizeQualifiedClauseHead(clause, lexicalModule = 'user') {
+  if (clause == null || clause.kind === 'quad' || isDirectiveClause(clause)) return clause;
+  const head = clause.head;
+  if (head?.type !== COMPOUND || head.name !== ':' || head.arity !== 2) return clause;
+  const qualifier = head.args[0];
+  const qualifiedHead = head.args[1];
+  if (qualifier.type === VAR) throw new PrologError('instantiation_error');
+  if (qualifier.type !== ATOM) throw new PrologError('type_error(atom)', qualifier);
+  if (qualifiedHead.type !== ATOM && qualifiedHead.type !== COMPOUND) {
+    throw new PrologError('type_error(callable)', qualifiedHead);
+  }
+  clause.head = qualifiedHead;
+  clause.module = qualifier.name;
+  clause.lexicalModule = lexicalModule;
+  return clause;
+}
+
 function includeDirective(clause) {
   if (isCompactBinaryClause(clause) || !isDirectiveClause(clause)) return null;
   const directive = clause.head.args[0];

package/src/solver.js

@@ -8,7 +8,7 @@ import {
 import { numberValueKey, sameNumberValue } from './number-value.js';
 import { PrologError, getStrictIsoRegistry } from './iso.js';
 import { getEyePrologRegistry } from './standard-library.js';
-import { selectClauseCandidates, selectClauseCandidatesForValues, selectGroundClauseCandidates } from './program.js';
+import { selectClauseCandidates, selectClauseCandidatesForValues, selectGroundClauseCandidates } from './program-indexing.js';
 import { StreamManager } from './io.js';
 import { clpzStateConsistent } from './clpz.js';
 import { hardHeapLimit, softHeapLimit, usedHeapSize } from './platform.js';

package/src/source-expansion.js

@@ -0,0 +1,173 @@
+// User-defined source expansion for the normal EyeProlog profile.
+//
+// This deliberately sits beside program preparation rather than inside the
+// term kernel. A small, fresh Solver executes already-loaded expansion hooks
+// against the partially built Program. The solver itself imports clause-index
+// helpers directly, so this compiler service does not introduce a module cycle.
+
+import {
+  ATOM, COMPOUND, Env, Term, VAR, atom, compound, deref, variable, variantTerms,
+} from './term.js';
+import { modulePredicateKey } from './program-indexing.js';
+import { Solver } from './solver.js';
+import { clauseToSourceTerm, sourceTermToClauses } from './expansion-builtins.js';
+import { PrologError } from './errors.js';
+
+let expansionFresh = 0;
+const MAX_EXPANSION_DEPTH = 64;
+const MAX_EXPANSION_INFERENCES = 1_000_000;
+
+function exactHookGroup(program, module, name) {
+  return program.groups.get(modulePredicateKey(module, name, 2)) ?? null;
+}
+
+function hookModules(module) {
+  return module === 'user' ? ['user'] : [module, 'user'];
+}
+
+function inputVariableNames(term) {
+  const names = new Set();
+  const stack = [term];
+  const seen = new Set();
+  while (stack.length > 0) {
+    const current = stack.pop();
+    if (current == null || seen.has(current)) continue;
+    seen.add(current);
+    if (current.type === VAR) {
+      names.add(current.name);
+      continue;
+    }
+    if (current.type === COMPOUND) {
+      for (const arg of current.args) stack.push(arg);
+    }
+  }
+  return names;
+}
+
+function copyHookResult(term, env, inputNames) {
+  const rootNames = new Map();
+  for (const name of inputNames) {
+    const root = deref(variable(name), env);
+    if (root.type === VAR && !rootNames.has(root.name)) rootNames.set(root.name, name);
+  }
+  const copy = (value) => {
+    const resolved = deref(value, env);
+    if (resolved.type === VAR) return variable(rootNames.get(resolved.name) ?? resolved.name);
+    const copied = resolved.type === COMPOUND && resolved.arity === 0
+      ? atom(resolved.name)
+      : new Term(resolved.type, resolved.name, resolved.args.map(copy));
+    if (resolved.module != null) copied.module = resolved.module;
+    return copied;
+  };
+  return copy(term);
+}
+
+function runHook(program, module, name, input, loadContext = null) {
+  for (const owner of hookModules(module)) {
+    const group = exactHookGroup(program, owner, name);
+    if (group == null || group.clauses.length === 0) continue;
+
+    const output = variable(`\u0000source-expansion:${++expansionFresh}`);
+    const goal = compound(name, [input, output]);
+    goal.module = owner;
+    const solver = new Solver(program, {
+      solutionLimit: 1,
+      maxInferences: MAX_EXPANSION_INFERENCES,
+      sourceLoadContext: loadContext,
+    });
+    const inputNames = inputVariableNames(input);
+    for (const answer of solver.solve([goal], new Env())) {
+      return copyHookResult(output, answer, inputNames);
+    }
+  }
+  return null;
+}
+
+function sourceTermsAreVariants(left, right) {
+  return variantTerms(left, new Env(), right, new Env());
+}
+
+function expandTermRecursively(program, module, term, loadContext, depth) {
+  if (depth > MAX_EXPANSION_DEPTH) throw new PrologError('resource_error(source_expansion)');
+  const expanded = runHook(program, module, 'term_expansion', term, loadContext);
+  if (expanded == null) return sourceTermToClauses(term);
+
+  const clauses = sourceTermToClauses(expanded);
+  if (clauses.length === 1 && sourceTermsAreVariants(term, clauseToSourceTerm(clauses[0]))) return clauses;
+  const result = [];
+  for (const clause of clauses) {
+    const generatedTerm = clauseToSourceTerm(clause);
+    result.push(...expandTermRecursively(program, module, generatedTerm, loadContext, depth + 1));
+  }
+  return result;
+}
+
+export function expandSourceClause(program, clause, module = 'user', loadContext = null) {
+  const sourceTerm = clauseToSourceTerm(clause);
+  if (sourceTerm == null) return { expanded: false, clauses: [clause] };
+  if (exactHookGroup(program, module, 'term_expansion') == null &&
+      (module === 'user' || exactHookGroup(program, 'user', 'term_expansion') == null)) {
+    return { expanded: false, clauses: [clause] };
+  }
+  const first = runHook(program, module, 'term_expansion', sourceTerm, loadContext);
+  if (first == null) return { expanded: false, clauses: [clause] };
+
+  const firstClauses = sourceTermToClauses(first);
+  if (firstClauses.length === 1 && sourceTermsAreVariants(sourceTerm, clauseToSourceTerm(firstClauses[0]))) {
+    return { expanded: false, clauses: [clause] };
+  }
+  const result = [];
+  for (const generated of firstClauses) {
+    const generatedTerm = clauseToSourceTerm(generated);
+    result.push(...expandTermRecursively(program, module, generatedTerm, loadContext, 1));
+  }
+  return { expanded: true, clauses: result };
+}
+
+function isCallable(term) {
+  return term?.type === ATOM || term?.type === COMPOUND;
+}
+
+function expandGoalChildren(program, goal, module, loadContext, depth) {
+  if (goal.type !== COMPOUND) return goal;
+
+  if (goal.name === ':' && goal.arity === 2) {
+    const qualifier = deref(goal.args[0], new Env());
+    if (qualifier.type === VAR) return goal;
+    if (qualifier.type !== ATOM) return goal;
+    const child = expandGoalRecursively(program, goal.args[1], qualifier.name, loadContext, depth + 1);
+    return compound(':', [goal.args[0], child]);
+  }
+
+  if ([',', ';', '->'].includes(goal.name) && goal.arity === 2) {
+    return compound(goal.name, [
+      expandGoalRecursively(program, goal.args[0], module, loadContext, depth + 1),
+      expandGoalRecursively(program, goal.args[1], module, loadContext, depth + 1),
+    ]);
+  }
+  if (goal.name === '\\+' && goal.arity === 1) {
+    return compound('\\+', [expandGoalRecursively(program, goal.args[0], module, loadContext, depth + 1)]);
+  }
+  return goal;
+}
+
+function expandGoalRecursively(program, goal, module, loadContext, depth = 0) {
+  if (depth > MAX_EXPANSION_DEPTH) throw new PrologError('resource_error(source_expansion)');
+  const resolved = deref(goal, new Env());
+  if (!isCallable(resolved)) return resolved;
+
+  const expanded = runHook(program, module, 'goal_expansion', resolved, loadContext);
+  if (expanded != null) {
+    if (!isCallable(expanded)) throw new PrologError('type_error(callable)', expanded);
+    if (sourceTermsAreVariants(resolved, expanded)) return expandGoalChildren(program, resolved, module, loadContext, depth);
+    return expandGoalRecursively(program, expanded, module, loadContext, depth + 1);
+  }
+  return expandGoalChildren(program, resolved, module, loadContext, depth);
+}
+
+export function expandClauseGoals(program, clause, lexicalModule = 'user', loadContext = null) {
+  if (!Array.isArray(clause.body) || clause.body.length === 0) return clause;
+  clause.body = clause.body.map((goal) =>
+    expandGoalRecursively(program, goal, lexicalModule, loadContext));
+  return clause;
+}

package/src/standard-library.js

@@ -5,6 +5,7 @@
 import { PrologError, createDefaultRegistry, eyePrologLibraryBuiltins } from './iso.js';
 import { clpzBuiltins } from './clpz.js';
 import { attsBuiltins } from './atts.js';
+import { expansionBuiltins } from './expansion-builtins.js';
 import { registerCleanupBuiltins } from './cleanup.js';
 import { fs, isNode, memoryStatistics } from './platform.js';
 import { ATOM, VAR, atom, deref, numberTerm, unify } from './term.js';
@@ -188,6 +189,7 @@ export function createEyePrologRegistry() {
   registerCleanupBuiltins(registry);
   eyePrologLibraryBuiltins.register(registry);
   attsBuiltins.register(registry);
+  expansionBuiltins.register(registry);
   clpzBuiltins.register(registry);
   registry.eyePrologLibrary = true;
   return registry;

package/test/run-regression.mjs

@@ -185,6 +185,50 @@ probe.
           'Scryer-style operator exports and multi-attribute directives load');
       },
     },
+    {
+      name: 'user source expansion supports Scryer-style generated predicates and DCGs',
+      run: () => {
+        const termSource = `term_expansion(make_generated, [generated(one),generated(two)]).
+make_generated.
+`;
+        const generated = Program.parse(termSource, { sourceMetadata: false });
+        assertEqual(generated.findGroup('generated', 1, 'user')?.clauses.length, 2,
+          'term_expansion/2 may emit a list of clauses');
+        assertEqual(generated.findGroup('make_generated', 0, 'user'), null,
+          'the source marker is replaced by its expansion');
+
+        const dcgSource = `term_expansion(make_dcg, Clause) :-
+    expand_term((generated_nt --> [a]), Clause).
+make_dcg.
+check :- phrase(generated_nt, [a]).
+`;
+        assertIncludes(run(dcgSource, { goal: 'check' }).stdout, 'check.',
+          'expand_term/2 exposes system DCG expansion to a source hook');
+
+        const duoSource = `:- op(1200, xfx, ++>).
+term_expansion((Head ++> Body), (Head :- Body)).
+generated_duo(ok) ++> true.
+`;
+        assertEqual(run(duoSource, { goal: 'generated_duo(ok)' }).stats.completed_goal_lists, 1,
+          'custom Duo-DCG-style operators can be lowered by term_expansion/2');
+
+        const goalSource = `:- module(expansion_probe, [check/0]).
+helper(ok).
+user:goal_expansion(old(X), new(X)) :- helper(ok).
+new(ok).
+check :- old(X), X = ok.
+`;
+        const goalProgram = Program.parse(goalSource, { sourceMetadata: false });
+        const check = goalProgram.findGroup('check', 0, 'expansion_probe')?.clauses[0];
+        assertEqual(check?.body[0]?.name, 'new', 'goal_expansion/2 rewrites a later source goal');
+        assertEqual(check?.body[0]?.args[0]?.name, check?.body[1]?.args[0]?.name,
+          'goal expansion preserves sharing with the surrounding clause');
+        assertEqual(goalProgram.findGroup('goal_expansion', 2, 'user')?.clauses[0]?.body[0]?.module,
+          'expansion_probe', 'qualified hook clauses retain their lexical body module');
+        assertIncludes(run(goalSource, { goal: 'expansion_probe:check' }).stdout, 'expansion_probe:check.',
+          'a user hook defined by a library module executes in its lexical module');
+      },
+    },
     {
       name: 'write_term variable_names/1 errors preserve the instantiated option culprit (issue #69)',
       run: () => {
@@ -4785,7 +4829,7 @@ answer(ok) :-
         assertEqual(Boolean(registry.get('is', 2)), true, 'ISO is/2 exists');
         assertEqual(Boolean(registry.get('append', 3)), false, 'append/3 is not ISO core');
         assertEqual(library.eyePrologLibrary, true, 'complete registry marker');
-        assertEqual(library.defs.size, 171, 'EyeProlog registry contains ISO definitions, cleanup controls, observability extensions, WFS tnot/1, and private library adapters');
+        assertEqual(library.defs.size, 173, 'EyeProlog registry contains ISO definitions, cleanup controls, observability extensions, WFS tnot/1, and private library adapters');
         assertEqual(Boolean(registry.get('phrase', 2)), true, 'Part 3 phrase/2 exists');
         assertEqual(Boolean(registry.get('phrase', 3)), true, 'Part 3 phrase/3 exists');
         assertEqual(registry.get('statistics', 0), null, 'statistics/0 is absent from the ISO registry');

package/the-art-of-eyeprolog.md

@@ -6424,11 +6424,15 @@ Hamiltonian circuits, three-way comparison, and domain reflection. Its existing
 `src/clpz.js` store is now explicitly transitional: the attributed-variable layer
 is intended to support staged reuse of Markus Triska's MIT-licensed
 [Scryer `clpz.pl`](https://github.com/mthom/scryer-prolog/blob/master/src/lib/clpz.pl).
-The remaining blockers are compile-time term/goal expansion and the supporting
-library surface used by that source; the migration state is tracked in
-[`src/CLPZ-MIGRATION.md`](src/CLPZ-MIGRATION.md). Until parity is reached, the JS
-kernel remains the active implementation so existing CLP(Z) behavior does not
-regress.
+The generic compile-time expansion blocker is now closed: normal-profile source
+loading runs module-local and `user` `term_expansion/2` and `goal_expansion/2`
+hooks, accepts clause-list expansions, preserves surrounding variable sharing,
+and provides `expand_term/2` for processor DCG lowering. This is sufficient for
+Scryer's generated-predicate and Duo-DCG expansion strategy. The remaining
+blocker is the supporting Scryer library surface used by that source; the
+migration state is tracked in [`src/CLPZ-MIGRATION.md`](src/CLPZ-MIGRATION.md).
+Until parity is reached, the JS kernel remains the active implementation so
+existing CLP(Z) behavior does not regress.
 
 Alongside its interop entries `call_nth/2`, `time/1`, and `.../2`,
 `library(iso_ext)` also exports EyeProlog's extension relations `countall/2`,