document-compute.js 1.1.0 → 1.2.1

package/README.md

@@ -102,9 +102,20 @@ solveFor(xSquared, 4, "x", {}, { method: "newton", initialGuess: 3 }); // 2, via
 
 ## Deviations from the issue
 
-Two things #573 asks for are not here in full. One was closed at adoption: `Quantity` and `FormulaBindings` (with `Interval` and `EvaluationValue`) are typed contracts in `document-schema.js`'s `src/math.ts`, beside `MathExpression` itself, exactly as the issue proposes — evaluation inputs are validated schemas like everything else in that package, and this package imports them from there the same way it imports `MathExpression`, `DimensionVector`, and `ExactRational`, rather than carrying package-local definitions.
+One thing #573 asks for was closed at adoption rather than built here: `Quantity` and `FormulaBindings` (with `Interval` and `EvaluationValue`) are typed contracts in `document-schema.js`'s `src/math.ts`, beside `MathExpression` itself, exactly as the issue proposes — evaluation inputs are validated schemas like everything else in that package, and this package imports them from there the same way it imports `MathExpression`, `DimensionVector`, and `ExactRational`, rather than carrying package-local definitions.
 
-The one that remains: **the worked-example test harness** the issue describes as its differentiator — measuring the fraction of a real document corpus's formulae whose evaluation reproduces the document's own stated answer. That harness needs real document corpora with stated formulae and answers, which this package has no access to build against; what ships here instead is thorough unit-level coverage of `evaluate`, the unit/dimension model, interval arithmetic, and `solveFor` in isolation (`src/compute/*.test.ts`). It is tracked as a follow-up to this package, not silently dropped.
+## The worked-example differential harness
+
+[ExaDev/documents.js#794](https://github.com/ExaDev/documents.js/issues/794) split #573's own stated differentiator — measuring the fraction of a real document's formulae whose evaluation reproduces the document's own stated answer — into its own package once `evaluate`/`solveFor` themselves existed to measure against. `src/harness/worked-example.ts` and `src/harness/corpus.ts` are that harness:
+
+- `runWorkedExampleSequence(formulas, symbolTable?, options?)` walks a document-ordered sequence of already-lowered `ContentFormula` values and recognises the "givens, a formula, a stated result" shape a worked example actually has: a **definition** (`F = m \times a` — the right-hand side still mentions a symbol), a **binding** (`m = 2 kg` — a fully closed "given"), and a **stated result** (`F = 6 N` — structurally identical to a binding, but restating a symbol a definition is waiting on). A definition's own right-hand side is evaluated against whatever bindings are current when its stated result is reached, not a snapshot taken when the definition line first appeared — the common real document states the general law first, then the specific numbers, then the answer. Every outcome is one of `match`, `mismatch`, `gap` (naming a specific `WorkedExampleGap` — `unbound-symbol`, `unknown-unit`, `incompatible-dimensions`, `division-by-zero`, `unsupported-construct`, `numeric-domain`, `non-convergent-solve` — one per `compute/errors.ts` class), or `unresolved` (a definition the document never restated an answer for). Comparison is by relative tolerance (`1e-3` default), not exact equality, since a worked example's own stated answer is conventionally rounded.
+- `collectFormulas(document)`/`runCorpus(documents, options?)` extract the formula sequence out of a wordprocessing `ContentDocument`'s block flow (table cells included) and run the harness over a whole corpus at once, aggregating one combined coverage fraction plus every document's own outcomes; `formatCorpusReport` renders the result as plain text for a CLI/console caller.
+
+Scoped to point-valued (`Quantity`) answers: every value this harness computes comes from evaluating a closed statement with no bindings, which `evaluate` cannot turn into an `Interval` (an `Interval` only ever arises by binding a symbol to one) — a genuinely interval-valued worked example (`0.87 <= cos(phi) <= 1`, #573's own illustration of interval arithmetic) has no representation in this "symbol = expression" equality grammar at all, since neither `MathExpression` nor `documents.js`'s LaTeX lowering has a compound-inequality-to-range reading, and is out of scope for this pass rather than silently mishandled.
+
+`src/harness/corpus.test.ts` proves the whole pipeline end to end — markdown text through `markdown-codec`'s `$$` block recognition and `documents.js`'s `lowerMarkdownMath` (the "LaTeX lowering" #794 names as the natural source of worked examples) into this harness — against a small, hand-authored starter corpus. `markdown-codec` and `documents.js` are **devDependencies only**: both sit above this package in the family's own dependency order (see the monorepo root README's package table), so neither can be a runtime dependency here without a cycle, which is exactly why this package remains "not wired into the conversion pipeline" at runtime even though its own test suite now exercises that pipeline. A large real-world corpus (the issue's own stated differentiator at scale) is not included — gathering one is a data-curation task, not a code one — but is a straightforward local addition: point a `test/corpus/` directory (gitignored, matching `pdf-codec`'s own `test:corpus` convention) at real markdown documents with worked examples and feed `readMarkdownContent` → `lowerMarkdownMath` → `runCorpus` the same way `corpus.test.ts` does.
+
+While building this harness's own fixtures, a real bug surfaced in `documents.js`'s LaTeX lowering: `F = m \times a` (the textbook-standard way to write almost any formula) lowers to `(F = m) \times a` rather than `F = (m \times a)`, because the lowering folds relational and arithmetic operators at the same precedence with no notion that `=` should bind loosest — filed as [ExaDev/documents.js#812](https://github.com/ExaDev/documents.js/issues/812). This package's own fixtures work around it with an explicit braced right-hand side (`F = {m \times a}`, which lowers correctly), since fixing the lowering itself is out of scope for this package.
 
 ## Out of scope
 
@@ -112,7 +123,6 @@ Quoting the issue's own scope line directly: **this is deliberately not a CAS in
 
 - **Symbolic algebra** — exact rearrangement of an expression emitted back out as LaTeX, simplification, integration. `solveFor` finds a root numerically; it never isolates the unknown algebraically.
 - **A SymPy sidecar or any other symbolic-engine adapter.** The issue names this as the eventual home for symbolic work, behind an evaluator interface this package does not define or stub.
-- **The worked-example test harness** — see Deviations from the issue above.
 - **Matrix-valued evaluation.** `MathExpression`'s `'matrix'` node exists in the grammar `document-schema.js` defines, but this evaluator only ever produces scalar `Quantity`/`Interval` values; a `'matrix'` node throws `UnsupportedExpressionError` rather than being silently misevaluated.
 - **A general interval rule for `pow`/`sqrt`/the trigonometric operators.** These are implemented for `Quantity` only; applied to an `Interval` operand they throw `UnsupportedExpressionError` rather than guessing at a range a non-monotonic or sign-dependent function would need real analysis to get right.
 

package/dist/index.cjs

@@ -277,7 +277,7 @@ function absInterval(a) {
 }
 //#endregion
 //#region src/compute/evaluate.ts
-const EMPTY_SYMBOL_TABLE$1 = {
+const EMPTY_SYMBOL_TABLE$2 = {
 	symbols: [],
 	units: []
 };
@@ -287,7 +287,7 @@ function isInterval(value) {
 function toInterval(value) {
 	return isInterval(value) ? value : pointInterval(value.magnitude, value.dimension);
 }
-function asQuantity(value, context) {
+function asQuantity$1(value, context) {
 	if (isInterval(value)) throw new UnsupportedExpressionError(context, "this position requires a plain Quantity, not an Interval");
 	return value;
 }
@@ -323,7 +323,7 @@ const UNARY_OPERATORS = {
 	"math:cos": { quantity: cosQuantity },
 	"math:tan": { quantity: tanQuantity }
 };
-function evaluate(expression, bindings, context = EMPTY_SYMBOL_TABLE$1) {
+function evaluate(expression, bindings, context = EMPTY_SYMBOL_TABLE$2) {
 	switch (expression.kind) {
 		case "num": return quantity(rationalToNumber(toRational(expression)), {});
 		case "qty": return evaluateQty(expression, context);
@@ -375,13 +375,13 @@ function evaluateApp(node, bindings, context) {
 	}
 	if (node.operator === "math:pow") {
 		const [base, exponent] = expectTwoArgs(args, "'math:pow'");
-		return powQuantity(asQuantity(base, "evaluate"), asQuantity(exponent, "evaluate"));
+		return powQuantity(asQuantity$1(base, "evaluate"), asQuantity$1(exponent, "evaluate"));
 	}
 	throw new UnsupportedExpressionError("evaluate", `unknown operator '${node.operator}'`);
 }
 function evaluateBinder(node, bindings, context) {
-	const lower = asQuantity(evaluate(node.lower, bindings, context), `evaluate:${node.kind}`);
-	const upper = asQuantity(evaluate(node.upper, bindings, context), `evaluate:${node.kind}`);
+	const lower = asQuantity$1(evaluate(node.lower, bindings, context), `evaluate:${node.kind}`);
+	const upper = asQuantity$1(evaluate(node.upper, bindings, context), `evaluate:${node.kind}`);
 	if (!isDimensionless(lower.dimension) || !isDimensionless(upper.dimension)) throw new IncompatibleDimensionsError(`math:${node.kind}`, lower.dimension, upper.dimension, "binder bounds must be dimensionless");
 	if (!Number.isInteger(lower.magnitude) || !Number.isInteger(upper.magnitude)) throw new UnsupportedExpressionError(`evaluate:${node.kind}`, "binder bounds must evaluate to integers");
 	let accumulator = node.kind === "sum" ? quantity(0, {}) : quantity(1, {});
@@ -390,7 +390,7 @@ function evaluateBinder(node, bindings, context) {
 			...bindings,
 			[node.binder]: quantity(i, {})
 		};
-		const bodyValue = asQuantity(evaluate(node.body, bodyBindings, context), `evaluate:${node.kind}`);
+		const bodyValue = asQuantity$1(evaluate(node.body, bodyBindings, context), `evaluate:${node.kind}`);
 		accumulator = node.kind === "sum" ? addQuantities(accumulator, bodyValue) : multiplyQuantities(accumulator, bodyValue);
 	}
 	return accumulator;
@@ -400,7 +400,7 @@ function evaluateBinder(node, bindings, context) {
 const DEFAULT_TOLERANCE = 1e-9;
 const DEFAULT_MAX_ITERATIONS = 100;
 const DEFAULT_DERIVATIVE_STEP = 1e-6;
-const EMPTY_SYMBOL_TABLE = {
+const EMPTY_SYMBOL_TABLE$1 = {
 	symbols: [],
 	units: []
 };
@@ -414,7 +414,7 @@ function residualFn(expression, targetValue, unknownSymbol, bindings, context, d
 		return result.magnitude - targetValue;
 	};
 }
-function solveFor(expression, targetValue, unknownSymbol, bindings, options = {}, context = EMPTY_SYMBOL_TABLE) {
+function solveFor(expression, targetValue, unknownSymbol, bindings, options = {}, context = EMPTY_SYMBOL_TABLE$1) {
 	const method = options.method ?? "bisection";
 	const tolerance = options.tolerance ?? DEFAULT_TOLERANCE;
 	const maxIterations = options.maxIterations ?? DEFAULT_MAX_ITERATIONS;
@@ -456,6 +456,214 @@ function newton(f, initialGuess, tolerance, maxIterations, h) {
 	throw new NonConvergentSolveError("newton", maxIterations, `residual still exceeds tolerance ${tolerance} after ${maxIterations} iterations`);
 }
 //#endregion
+//#region src/harness/worked-example.ts
+const DEFAULT_RELATIVE_TOLERANCE = .001;
+const EMPTY_BINDINGS = {};
+const EMPTY_SYMBOL_TABLE = {
+	symbols: [],
+	units: []
+};
+function asEquality(expression) {
+	if (expression.kind !== "app" || expression.operator !== "math:eq") return;
+	const [lhs, rhs] = expression.args;
+	if (lhs === void 0 || rhs === void 0 || lhs.kind !== "sym") return;
+	return {
+		targetSymbol: lhs.id,
+		rhs
+	};
+}
+function containsSymbol(expression) {
+	switch (expression.kind) {
+		case "sym": return true;
+		case "num":
+		case "qty":
+		case "unparsed": return false;
+		case "app": return expression.args.some(containsSymbol);
+		case "sum":
+		case "prod": return containsSymbol(expression.lower) || containsSymbol(expression.upper) || containsSymbol(expression.body);
+		case "matrix": return expression.rows.some((row) => row.some(containsSymbol));
+	}
+}
+function asQuantity(value) {
+	if (isInterval(value)) throw new UnsupportedExpressionError("runWorkedExampleSequence", "this harness compares point-valued Quantity answers only; a symbol resolving to a range (Interval) has no stated-answer comparison defined yet");
+	return value;
+}
+function gapFromError(error) {
+	if (error instanceof UnboundSymbolError) return "unbound-symbol";
+	if (error instanceof UnknownUnitError) return "unknown-unit";
+	if (error instanceof IncompatibleDimensionsError) return "incompatible-dimensions";
+	if (error instanceof DivisionByZeroError) return "division-by-zero";
+	if (error instanceof UnsupportedExpressionError) return "unsupported-construct";
+	if (error instanceof NumericDomainError) return "numeric-domain";
+	if (error instanceof NonConvergentSolveError) return "non-convergent-solve";
+	return "other-evaluation-error";
+}
+function errorMessage(error) {
+	return error instanceof Error ? error.message : String(error);
+}
+function withinTolerance(actual, expected, relativeTolerance) {
+	if (expected === 0) return Math.abs(actual) <= relativeTolerance;
+	return Math.abs(actual - expected) / Math.abs(expected) <= relativeTolerance;
+}
+function resultsMatch(actual, expected, relativeTolerance) {
+	return dimensionsEqual(actual.dimension, expected.dimension) && withinTolerance(actual.magnitude, expected.magnitude, relativeTolerance);
+}
+function runWorkedExampleSequence(formulas, symbolTable = EMPTY_SYMBOL_TABLE, options) {
+	const relativeTolerance = options?.relativeTolerance ?? DEFAULT_RELATIVE_TOLERANCE;
+	const bindings = {};
+	const outcomes = [];
+	let pending;
+	const closeUnresolved = () => {
+		if (pending === void 0) return;
+		outcomes.push({
+			outcome: "unresolved",
+			targetSymbol: pending.targetSymbol,
+			message: `"${pending.targetSymbol}" was defined but the sequence never restated it as a closed numeric result before ending or being superseded by another definition`
+		});
+		pending = void 0;
+	};
+	for (const formula of formulas) {
+		if (formula.content === void 0) continue;
+		const equality = asEquality(formula.content);
+		if (equality === void 0) continue;
+		const { targetSymbol, rhs } = equality;
+		if (containsSymbol(rhs)) {
+			closeUnresolved();
+			pending = {
+				targetSymbol,
+				rhs
+			};
+			continue;
+		}
+		let closedValue;
+		try {
+			closedValue = asQuantity(evaluate(rhs, EMPTY_BINDINGS, symbolTable));
+		} catch (error) {
+			outcomes.push({
+				outcome: "gap",
+				gap: gapFromError(error),
+				targetSymbol,
+				message: errorMessage(error)
+			});
+			continue;
+		}
+		if (pending?.targetSymbol === targetSymbol) {
+			const { rhs: definitionRhs } = pending;
+			pending = void 0;
+			let actual;
+			try {
+				actual = asQuantity(evaluate(definitionRhs, bindings, symbolTable));
+			} catch (error) {
+				outcomes.push({
+					outcome: "gap",
+					gap: gapFromError(error),
+					targetSymbol,
+					message: errorMessage(error)
+				});
+				continue;
+			}
+			outcomes.push(resultsMatch(actual, closedValue, relativeTolerance) ? {
+				outcome: "match",
+				targetSymbol,
+				expected: closedValue,
+				actual
+			} : {
+				outcome: "mismatch",
+				targetSymbol,
+				expected: closedValue,
+				actual
+			});
+			continue;
+		}
+		bindings[targetSymbol] = closedValue;
+	}
+	closeUnresolved();
+	const matched = outcomes.filter((o) => o.outcome === "match").length;
+	const mismatched = outcomes.filter((o) => o.outcome === "mismatch").length;
+	const gaps = outcomes.filter((o) => o.outcome === "gap").length;
+	const unresolved = outcomes.filter((o) => o.outcome === "unresolved").length;
+	return {
+		outcomes,
+		total: outcomes.length,
+		matched,
+		mismatched,
+		gaps,
+		unresolved,
+		coverage: matched + mismatched === 0 ? void 0 : matched / (matched + mismatched)
+	};
+}
+//#endregion
+//#region src/harness/corpus.ts
+function collectFormulasFromBlocks(blocks, out) {
+	for (const block of blocks) {
+		if (block.kind === "table") {
+			for (const row of block.rows) for (const cell of row.cells) collectFormulasFromBlocks(cell.blocks, out);
+			continue;
+		}
+		if (block.kind === "embeddedObject" && block.objectKind === "formula") {
+			const embedded = block.document;
+			if (embedded.kind === "formula") out.push(embedded.formula);
+		}
+	}
+}
+function collectFormulas(document) {
+	if (document.kind !== "wordprocessing") return [];
+	const out = [];
+	for (const section of document.sections) collectFormulasFromBlocks(section.blocks, out);
+	return out;
+}
+function runCorpus(documents, options) {
+	const reports = documents.map(({ label, document }) => {
+		const symbolTable = document.symbolTable ?? {
+			symbols: [],
+			units: []
+		};
+		return {
+			label,
+			report: runWorkedExampleSequence(collectFormulas(document), symbolTable, options)
+		};
+	});
+	const matched = sumBy(reports, (r) => r.report.matched);
+	const mismatched = sumBy(reports, (r) => r.report.mismatched);
+	const gaps = sumBy(reports, (r) => r.report.gaps);
+	const unresolved = sumBy(reports, (r) => r.report.unresolved);
+	return {
+		documents: reports,
+		total: sumBy(reports, (r) => r.report.total),
+		matched,
+		mismatched,
+		gaps,
+		unresolved,
+		coverage: matched + mismatched === 0 ? void 0 : matched / (matched + mismatched)
+	};
+}
+function sumBy(items, project) {
+	return items.reduce((total, item) => total + project(item), 0);
+}
+function formatOutcome(outcome) {
+	switch (outcome.outcome) {
+		case "match": return `match: ${outcome.targetSymbol}`;
+		case "mismatch": return `MISMATCH: ${outcome.targetSymbol} -- expected ${formatEvaluationResult(outcome.expected)}, got ${formatEvaluationResult(outcome.actual)}`;
+		case "gap": return `GAP (${outcome.gap}): ${outcome.targetSymbol} -- ${outcome.message}`;
+		case "unresolved": return `unresolved: ${outcome.targetSymbol} -- ${outcome.message}`;
+	}
+}
+function formatEvaluationResult(value) {
+	if (value.kind === "interval") return `[${value.min}, ${value.max}]`;
+	return `${value.magnitude}`;
+}
+function formatCorpusReport(report) {
+	const lines = [];
+	for (const { label, report: documentReport } of report.documents) {
+		const coverageText = documentReport.coverage === void 0 ? "no stated answers" : `${(documentReport.coverage * 100).toFixed(1)}% (${documentReport.matched}/${documentReport.matched + documentReport.mismatched})`;
+		lines.push(`${label}: ${coverageText}`);
+		for (const outcome of documentReport.outcomes) if (outcome.outcome !== "match") lines.push(`  ${formatOutcome(outcome)}`);
+	}
+	const combinedText = report.coverage === void 0 ? "no stated answers in corpus" : `${(report.coverage * 100).toFixed(1)}% (${report.matched}/${report.matched + report.mismatched}), ${report.gaps} gap(s), ${report.unresolved} unresolved`;
+	lines.push(`TOTAL: ${combinedText}`);
+	return lines.join("\n");
+}
+//#endregion
 exports.DivisionByZeroError = DivisionByZeroError;
 exports.IncompatibleDimensionsError = IncompatibleDimensionsError;
 exports.NonConvergentSolveError = NonConvergentSolveError;
@@ -468,6 +676,7 @@ exports.absQuantity = absQuantity;
 exports.addIntervals = addIntervals;
 exports.addQuantities = addQuantities;
 exports.addRational = addRational;
+exports.collectFormulas = collectFormulas;
 exports.cosQuantity = cosQuantity;
 exports.dimensionExponent = dimensionExponent;
 exports.dimensionToString = dimensionToString;
@@ -477,6 +686,7 @@ exports.divideIntervals = divideIntervals;
 exports.divideQuantities = divideQuantities;
 exports.divideRational = divideRational;
 exports.evaluate = evaluate;
+exports.formatCorpusReport = formatCorpusReport;
 exports.interval = interval;
 exports.isDimensionless = isDimensionless;
 exports.isInterval = isInterval;
@@ -490,6 +700,8 @@ exports.pointInterval = pointInterval;
 exports.powQuantity = powQuantity;
 exports.quantity = quantity;
 exports.rationalToNumber = rationalToNumber;
+exports.runCorpus = runCorpus;
+exports.runWorkedExampleSequence = runWorkedExampleSequence;
 exports.scaleDimension = scaleDimension;
 exports.sinQuantity = sinQuantity;
 exports.solveFor = solveFor;

package/dist/index.d.cts

@@ -1,4 +1,4 @@
-import { DimensionVector, ExactRational, FormulaBindings, Interval, MathExpression, Quantity, SiBaseDimension, SymbolTable } from "document-schema.js";
+import { ContentDocument, ContentFormula, DimensionVector, ExactRational, FormulaBindings, Interval, MathExpression, Quantity, SiBaseDimension, SymbolTable } from "document-schema.js";
 //#region src/compute/rational.d.ts
 interface Rational {
   readonly n: bigint;
@@ -103,4 +103,66 @@ interface SolveForOptions {
 }
 declare function solveFor(expression: MathExpression, targetValue: number, unknownSymbol: string, bindings: FormulaBindings, options?: SolveForOptions, context?: SymbolTable): number;
 //#endregion
-export { DivisionByZeroError, EvaluationResult, IncompatibleDimensionsError, NonConvergentSolveError, NumericDomainError, Rational, SolveForOptions, SolveMethod, UnboundSymbolError, UnknownUnitError, UnsupportedExpressionError, absInterval, absQuantity, addIntervals, addQuantities, addRational, cosQuantity, dimensionExponent, dimensionToString, dimensionsEqual, divideDimensions, divideIntervals, divideQuantities, divideRational, evaluate, interval, isDimensionless, isInterval, multiplyDimensions, multiplyIntervals, multiplyQuantities, multiplyRational, negateInterval, negateQuantity, pointInterval, powQuantity, quantity, rationalToNumber, scaleDimension, sinQuantity, solveFor, sqrtQuantity, subtractIntervals, subtractQuantities, subtractRational, tanQuantity, toExactRational, toRational };
+//#region src/harness/worked-example.d.ts
+type WorkedExampleGap = "unbound-symbol" | "unknown-unit" | "incompatible-dimensions" | "division-by-zero" | "unsupported-construct" | "numeric-domain" | "non-convergent-solve" | "other-evaluation-error";
+interface WorkedExampleMatch {
+  readonly outcome: "match";
+  readonly targetSymbol: string;
+  readonly expected: Quantity;
+  readonly actual: Quantity;
+}
+interface WorkedExampleMismatch {
+  readonly outcome: "mismatch";
+  readonly targetSymbol: string;
+  readonly expected: Quantity;
+  readonly actual: Quantity;
+}
+interface WorkedExampleGapResult {
+  readonly outcome: "gap";
+  readonly gap: WorkedExampleGap;
+  readonly targetSymbol: string;
+  readonly message: string;
+}
+interface WorkedExampleUnresolved {
+  readonly outcome: "unresolved";
+  readonly targetSymbol: string;
+  readonly message: string;
+}
+type WorkedExampleOutcome = WorkedExampleMatch | WorkedExampleMismatch | WorkedExampleGapResult | WorkedExampleUnresolved;
+interface WorkedExampleReport {
+  readonly outcomes: readonly WorkedExampleOutcome[];
+  readonly total: number;
+  readonly matched: number;
+  readonly mismatched: number;
+  readonly gaps: number;
+  readonly unresolved: number;
+  readonly coverage: number | undefined;
+}
+interface WorkedExampleOptions {
+  readonly relativeTolerance?: number;
+}
+declare function runWorkedExampleSequence(formulas: readonly ContentFormula[], symbolTable?: SymbolTable, options?: WorkedExampleOptions): WorkedExampleReport;
+//#endregion
+//#region src/harness/corpus.d.ts
+declare function collectFormulas(document: ContentDocument): readonly ContentFormula[];
+interface CorpusDocument {
+  readonly label: string;
+  readonly document: ContentDocument;
+}
+interface CorpusDocumentReport {
+  readonly label: string;
+  readonly report: WorkedExampleReport;
+}
+interface CorpusReport {
+  readonly documents: readonly CorpusDocumentReport[];
+  readonly total: number;
+  readonly matched: number;
+  readonly mismatched: number;
+  readonly gaps: number;
+  readonly unresolved: number;
+  readonly coverage: number | undefined;
+}
+declare function runCorpus(documents: readonly CorpusDocument[], options?: WorkedExampleOptions): CorpusReport;
+declare function formatCorpusReport(report: CorpusReport): string;
+//#endregion
+export { CorpusDocument, CorpusDocumentReport, CorpusReport, DivisionByZeroError, EvaluationResult, IncompatibleDimensionsError, NonConvergentSolveError, NumericDomainError, Rational, SolveForOptions, SolveMethod, UnboundSymbolError, UnknownUnitError, UnsupportedExpressionError, WorkedExampleGap, WorkedExampleGapResult, WorkedExampleMatch, WorkedExampleMismatch, WorkedExampleOptions, WorkedExampleOutcome, WorkedExampleReport, WorkedExampleUnresolved, absInterval, absQuantity, addIntervals, addQuantities, addRational, collectFormulas, cosQuantity, dimensionExponent, dimensionToString, dimensionsEqual, divideDimensions, divideIntervals, divideQuantities, divideRational, evaluate, formatCorpusReport, interval, isDimensionless, isInterval, multiplyDimensions, multiplyIntervals, multiplyQuantities, multiplyRational, negateInterval, negateQuantity, pointInterval, powQuantity, quantity, rationalToNumber, runCorpus, runWorkedExampleSequence, scaleDimension, sinQuantity, solveFor, sqrtQuantity, subtractIntervals, subtractQuantities, subtractRational, tanQuantity, toExactRational, toRational };

package/dist/index.d.ts

@@ -1,4 +1,4 @@
-import { DimensionVector, ExactRational, FormulaBindings, Interval, MathExpression, Quantity, SiBaseDimension, SymbolTable } from "document-schema.js";
+import { ContentDocument, ContentFormula, DimensionVector, ExactRational, FormulaBindings, Interval, MathExpression, Quantity, SiBaseDimension, SymbolTable } from "document-schema.js";
 //#region src/compute/rational.d.ts
 interface Rational {
   readonly n: bigint;
@@ -103,4 +103,66 @@ interface SolveForOptions {
 }
 declare function solveFor(expression: MathExpression, targetValue: number, unknownSymbol: string, bindings: FormulaBindings, options?: SolveForOptions, context?: SymbolTable): number;
 //#endregion
-export { DivisionByZeroError, EvaluationResult, IncompatibleDimensionsError, NonConvergentSolveError, NumericDomainError, Rational, SolveForOptions, SolveMethod, UnboundSymbolError, UnknownUnitError, UnsupportedExpressionError, absInterval, absQuantity, addIntervals, addQuantities, addRational, cosQuantity, dimensionExponent, dimensionToString, dimensionsEqual, divideDimensions, divideIntervals, divideQuantities, divideRational, evaluate, interval, isDimensionless, isInterval, multiplyDimensions, multiplyIntervals, multiplyQuantities, multiplyRational, negateInterval, negateQuantity, pointInterval, powQuantity, quantity, rationalToNumber, scaleDimension, sinQuantity, solveFor, sqrtQuantity, subtractIntervals, subtractQuantities, subtractRational, tanQuantity, toExactRational, toRational };
+//#region src/harness/worked-example.d.ts
+type WorkedExampleGap = "unbound-symbol" | "unknown-unit" | "incompatible-dimensions" | "division-by-zero" | "unsupported-construct" | "numeric-domain" | "non-convergent-solve" | "other-evaluation-error";
+interface WorkedExampleMatch {
+  readonly outcome: "match";
+  readonly targetSymbol: string;
+  readonly expected: Quantity;
+  readonly actual: Quantity;
+}
+interface WorkedExampleMismatch {
+  readonly outcome: "mismatch";
+  readonly targetSymbol: string;
+  readonly expected: Quantity;
+  readonly actual: Quantity;
+}
+interface WorkedExampleGapResult {
+  readonly outcome: "gap";
+  readonly gap: WorkedExampleGap;
+  readonly targetSymbol: string;
+  readonly message: string;
+}
+interface WorkedExampleUnresolved {
+  readonly outcome: "unresolved";
+  readonly targetSymbol: string;
+  readonly message: string;
+}
+type WorkedExampleOutcome = WorkedExampleMatch | WorkedExampleMismatch | WorkedExampleGapResult | WorkedExampleUnresolved;
+interface WorkedExampleReport {
+  readonly outcomes: readonly WorkedExampleOutcome[];
+  readonly total: number;
+  readonly matched: number;
+  readonly mismatched: number;
+  readonly gaps: number;
+  readonly unresolved: number;
+  readonly coverage: number | undefined;
+}
+interface WorkedExampleOptions {
+  readonly relativeTolerance?: number;
+}
+declare function runWorkedExampleSequence(formulas: readonly ContentFormula[], symbolTable?: SymbolTable, options?: WorkedExampleOptions): WorkedExampleReport;
+//#endregion
+//#region src/harness/corpus.d.ts
+declare function collectFormulas(document: ContentDocument): readonly ContentFormula[];
+interface CorpusDocument {
+  readonly label: string;
+  readonly document: ContentDocument;
+}
+interface CorpusDocumentReport {
+  readonly label: string;
+  readonly report: WorkedExampleReport;
+}
+interface CorpusReport {
+  readonly documents: readonly CorpusDocumentReport[];
+  readonly total: number;
+  readonly matched: number;
+  readonly mismatched: number;
+  readonly gaps: number;
+  readonly unresolved: number;
+  readonly coverage: number | undefined;
+}
+declare function runCorpus(documents: readonly CorpusDocument[], options?: WorkedExampleOptions): CorpusReport;
+declare function formatCorpusReport(report: CorpusReport): string;
+//#endregion
+export { CorpusDocument, CorpusDocumentReport, CorpusReport, DivisionByZeroError, EvaluationResult, IncompatibleDimensionsError, NonConvergentSolveError, NumericDomainError, Rational, SolveForOptions, SolveMethod, UnboundSymbolError, UnknownUnitError, UnsupportedExpressionError, WorkedExampleGap, WorkedExampleGapResult, WorkedExampleMatch, WorkedExampleMismatch, WorkedExampleOptions, WorkedExampleOutcome, WorkedExampleReport, WorkedExampleUnresolved, absInterval, absQuantity, addIntervals, addQuantities, addRational, collectFormulas, cosQuantity, dimensionExponent, dimensionToString, dimensionsEqual, divideDimensions, divideIntervals, divideQuantities, divideRational, evaluate, formatCorpusReport, interval, isDimensionless, isInterval, multiplyDimensions, multiplyIntervals, multiplyQuantities, multiplyRational, negateInterval, negateQuantity, pointInterval, powQuantity, quantity, rationalToNumber, runCorpus, runWorkedExampleSequence, scaleDimension, sinQuantity, solveFor, sqrtQuantity, subtractIntervals, subtractQuantities, subtractRational, tanQuantity, toExactRational, toRational };

package/dist/index.js

@@ -276,7 +276,7 @@ function absInterval(a) {
 }
 //#endregion
 //#region src/compute/evaluate.ts
-const EMPTY_SYMBOL_TABLE$1 = {
+const EMPTY_SYMBOL_TABLE$2 = {
 	symbols: [],
 	units: []
 };
@@ -286,7 +286,7 @@ function isInterval(value) {
 function toInterval(value) {
 	return isInterval(value) ? value : pointInterval(value.magnitude, value.dimension);
 }
-function asQuantity(value, context) {
+function asQuantity$1(value, context) {
 	if (isInterval(value)) throw new UnsupportedExpressionError(context, "this position requires a plain Quantity, not an Interval");
 	return value;
 }
@@ -322,7 +322,7 @@ const UNARY_OPERATORS = {
 	"math:cos": { quantity: cosQuantity },
 	"math:tan": { quantity: tanQuantity }
 };
-function evaluate(expression, bindings, context = EMPTY_SYMBOL_TABLE$1) {
+function evaluate(expression, bindings, context = EMPTY_SYMBOL_TABLE$2) {
 	switch (expression.kind) {
 		case "num": return quantity(rationalToNumber(toRational(expression)), {});
 		case "qty": return evaluateQty(expression, context);
@@ -374,13 +374,13 @@ function evaluateApp(node, bindings, context) {
 	}
 	if (node.operator === "math:pow") {
 		const [base, exponent] = expectTwoArgs(args, "'math:pow'");
-		return powQuantity(asQuantity(base, "evaluate"), asQuantity(exponent, "evaluate"));
+		return powQuantity(asQuantity$1(base, "evaluate"), asQuantity$1(exponent, "evaluate"));
 	}
 	throw new UnsupportedExpressionError("evaluate", `unknown operator '${node.operator}'`);
 }
 function evaluateBinder(node, bindings, context) {
-	const lower = asQuantity(evaluate(node.lower, bindings, context), `evaluate:${node.kind}`);
-	const upper = asQuantity(evaluate(node.upper, bindings, context), `evaluate:${node.kind}`);
+	const lower = asQuantity$1(evaluate(node.lower, bindings, context), `evaluate:${node.kind}`);
+	const upper = asQuantity$1(evaluate(node.upper, bindings, context), `evaluate:${node.kind}`);
 	if (!isDimensionless(lower.dimension) || !isDimensionless(upper.dimension)) throw new IncompatibleDimensionsError(`math:${node.kind}`, lower.dimension, upper.dimension, "binder bounds must be dimensionless");
 	if (!Number.isInteger(lower.magnitude) || !Number.isInteger(upper.magnitude)) throw new UnsupportedExpressionError(`evaluate:${node.kind}`, "binder bounds must evaluate to integers");
 	let accumulator = node.kind === "sum" ? quantity(0, {}) : quantity(1, {});
@@ -389,7 +389,7 @@ function evaluateBinder(node, bindings, context) {
 			...bindings,
 			[node.binder]: quantity(i, {})
 		};
-		const bodyValue = asQuantity(evaluate(node.body, bodyBindings, context), `evaluate:${node.kind}`);
+		const bodyValue = asQuantity$1(evaluate(node.body, bodyBindings, context), `evaluate:${node.kind}`);
 		accumulator = node.kind === "sum" ? addQuantities(accumulator, bodyValue) : multiplyQuantities(accumulator, bodyValue);
 	}
 	return accumulator;
@@ -399,7 +399,7 @@ function evaluateBinder(node, bindings, context) {
 const DEFAULT_TOLERANCE = 1e-9;
 const DEFAULT_MAX_ITERATIONS = 100;
 const DEFAULT_DERIVATIVE_STEP = 1e-6;
-const EMPTY_SYMBOL_TABLE = {
+const EMPTY_SYMBOL_TABLE$1 = {
 	symbols: [],
 	units: []
 };
@@ -413,7 +413,7 @@ function residualFn(expression, targetValue, unknownSymbol, bindings, context, d
 		return result.magnitude - targetValue;
 	};
 }
-function solveFor(expression, targetValue, unknownSymbol, bindings, options = {}, context = EMPTY_SYMBOL_TABLE) {
+function solveFor(expression, targetValue, unknownSymbol, bindings, options = {}, context = EMPTY_SYMBOL_TABLE$1) {
 	const method = options.method ?? "bisection";
 	const tolerance = options.tolerance ?? DEFAULT_TOLERANCE;
 	const maxIterations = options.maxIterations ?? DEFAULT_MAX_ITERATIONS;
@@ -455,4 +455,212 @@ function newton(f, initialGuess, tolerance, maxIterations, h) {
 	throw new NonConvergentSolveError("newton", maxIterations, `residual still exceeds tolerance ${tolerance} after ${maxIterations} iterations`);
 }
 //#endregion
-export { DivisionByZeroError, IncompatibleDimensionsError, NonConvergentSolveError, NumericDomainError, UnboundSymbolError, UnknownUnitError, UnsupportedExpressionError, absInterval, absQuantity, addIntervals, addQuantities, addRational, cosQuantity, dimensionExponent, dimensionToString, dimensionsEqual, divideDimensions, divideIntervals, divideQuantities, divideRational, evaluate, interval, isDimensionless, isInterval, multiplyDimensions, multiplyIntervals, multiplyQuantities, multiplyRational, negateInterval, negateQuantity, pointInterval, powQuantity, quantity, rationalToNumber, scaleDimension, sinQuantity, solveFor, sqrtQuantity, subtractIntervals, subtractQuantities, subtractRational, tanQuantity, toExactRational, toRational };
+//#region src/harness/worked-example.ts
+const DEFAULT_RELATIVE_TOLERANCE = .001;
+const EMPTY_BINDINGS = {};
+const EMPTY_SYMBOL_TABLE = {
+	symbols: [],
+	units: []
+};
+function asEquality(expression) {
+	if (expression.kind !== "app" || expression.operator !== "math:eq") return;
+	const [lhs, rhs] = expression.args;
+	if (lhs === void 0 || rhs === void 0 || lhs.kind !== "sym") return;
+	return {
+		targetSymbol: lhs.id,
+		rhs
+	};
+}
+function containsSymbol(expression) {
+	switch (expression.kind) {
+		case "sym": return true;
+		case "num":
+		case "qty":
+		case "unparsed": return false;
+		case "app": return expression.args.some(containsSymbol);
+		case "sum":
+		case "prod": return containsSymbol(expression.lower) || containsSymbol(expression.upper) || containsSymbol(expression.body);
+		case "matrix": return expression.rows.some((row) => row.some(containsSymbol));
+	}
+}
+function asQuantity(value) {
+	if (isInterval(value)) throw new UnsupportedExpressionError("runWorkedExampleSequence", "this harness compares point-valued Quantity answers only; a symbol resolving to a range (Interval) has no stated-answer comparison defined yet");
+	return value;
+}
+function gapFromError(error) {
+	if (error instanceof UnboundSymbolError) return "unbound-symbol";
+	if (error instanceof UnknownUnitError) return "unknown-unit";
+	if (error instanceof IncompatibleDimensionsError) return "incompatible-dimensions";
+	if (error instanceof DivisionByZeroError) return "division-by-zero";
+	if (error instanceof UnsupportedExpressionError) return "unsupported-construct";
+	if (error instanceof NumericDomainError) return "numeric-domain";
+	if (error instanceof NonConvergentSolveError) return "non-convergent-solve";
+	return "other-evaluation-error";
+}
+function errorMessage(error) {
+	return error instanceof Error ? error.message : String(error);
+}
+function withinTolerance(actual, expected, relativeTolerance) {
+	if (expected === 0) return Math.abs(actual) <= relativeTolerance;
+	return Math.abs(actual - expected) / Math.abs(expected) <= relativeTolerance;
+}
+function resultsMatch(actual, expected, relativeTolerance) {
+	return dimensionsEqual(actual.dimension, expected.dimension) && withinTolerance(actual.magnitude, expected.magnitude, relativeTolerance);
+}
+function runWorkedExampleSequence(formulas, symbolTable = EMPTY_SYMBOL_TABLE, options) {
+	const relativeTolerance = options?.relativeTolerance ?? DEFAULT_RELATIVE_TOLERANCE;
+	const bindings = {};
+	const outcomes = [];
+	let pending;
+	const closeUnresolved = () => {
+		if (pending === void 0) return;
+		outcomes.push({
+			outcome: "unresolved",
+			targetSymbol: pending.targetSymbol,
+			message: `"${pending.targetSymbol}" was defined but the sequence never restated it as a closed numeric result before ending or being superseded by another definition`
+		});
+		pending = void 0;
+	};
+	for (const formula of formulas) {
+		if (formula.content === void 0) continue;
+		const equality = asEquality(formula.content);
+		if (equality === void 0) continue;
+		const { targetSymbol, rhs } = equality;
+		if (containsSymbol(rhs)) {
+			closeUnresolved();
+			pending = {
+				targetSymbol,
+				rhs
+			};
+			continue;
+		}
+		let closedValue;
+		try {
+			closedValue = asQuantity(evaluate(rhs, EMPTY_BINDINGS, symbolTable));
+		} catch (error) {
+			outcomes.push({
+				outcome: "gap",
+				gap: gapFromError(error),
+				targetSymbol,
+				message: errorMessage(error)
+			});
+			continue;
+		}
+		if (pending?.targetSymbol === targetSymbol) {
+			const { rhs: definitionRhs } = pending;
+			pending = void 0;
+			let actual;
+			try {
+				actual = asQuantity(evaluate(definitionRhs, bindings, symbolTable));
+			} catch (error) {
+				outcomes.push({
+					outcome: "gap",
+					gap: gapFromError(error),
+					targetSymbol,
+					message: errorMessage(error)
+				});
+				continue;
+			}
+			outcomes.push(resultsMatch(actual, closedValue, relativeTolerance) ? {
+				outcome: "match",
+				targetSymbol,
+				expected: closedValue,
+				actual
+			} : {
+				outcome: "mismatch",
+				targetSymbol,
+				expected: closedValue,
+				actual
+			});
+			continue;
+		}
+		bindings[targetSymbol] = closedValue;
+	}
+	closeUnresolved();
+	const matched = outcomes.filter((o) => o.outcome === "match").length;
+	const mismatched = outcomes.filter((o) => o.outcome === "mismatch").length;
+	const gaps = outcomes.filter((o) => o.outcome === "gap").length;
+	const unresolved = outcomes.filter((o) => o.outcome === "unresolved").length;
+	return {
+		outcomes,
+		total: outcomes.length,
+		matched,
+		mismatched,
+		gaps,
+		unresolved,
+		coverage: matched + mismatched === 0 ? void 0 : matched / (matched + mismatched)
+	};
+}
+//#endregion
+//#region src/harness/corpus.ts
+function collectFormulasFromBlocks(blocks, out) {
+	for (const block of blocks) {
+		if (block.kind === "table") {
+			for (const row of block.rows) for (const cell of row.cells) collectFormulasFromBlocks(cell.blocks, out);
+			continue;
+		}
+		if (block.kind === "embeddedObject" && block.objectKind === "formula") {
+			const embedded = block.document;
+			if (embedded.kind === "formula") out.push(embedded.formula);
+		}
+	}
+}
+function collectFormulas(document) {
+	if (document.kind !== "wordprocessing") return [];
+	const out = [];
+	for (const section of document.sections) collectFormulasFromBlocks(section.blocks, out);
+	return out;
+}
+function runCorpus(documents, options) {
+	const reports = documents.map(({ label, document }) => {
+		const symbolTable = document.symbolTable ?? {
+			symbols: [],
+			units: []
+		};
+		return {
+			label,
+			report: runWorkedExampleSequence(collectFormulas(document), symbolTable, options)
+		};
+	});
+	const matched = sumBy(reports, (r) => r.report.matched);
+	const mismatched = sumBy(reports, (r) => r.report.mismatched);
+	const gaps = sumBy(reports, (r) => r.report.gaps);
+	const unresolved = sumBy(reports, (r) => r.report.unresolved);
+	return {
+		documents: reports,
+		total: sumBy(reports, (r) => r.report.total),
+		matched,
+		mismatched,
+		gaps,
+		unresolved,
+		coverage: matched + mismatched === 0 ? void 0 : matched / (matched + mismatched)
+	};
+}
+function sumBy(items, project) {
+	return items.reduce((total, item) => total + project(item), 0);
+}
+function formatOutcome(outcome) {
+	switch (outcome.outcome) {
+		case "match": return `match: ${outcome.targetSymbol}`;
+		case "mismatch": return `MISMATCH: ${outcome.targetSymbol} -- expected ${formatEvaluationResult(outcome.expected)}, got ${formatEvaluationResult(outcome.actual)}`;
+		case "gap": return `GAP (${outcome.gap}): ${outcome.targetSymbol} -- ${outcome.message}`;
+		case "unresolved": return `unresolved: ${outcome.targetSymbol} -- ${outcome.message}`;
+	}
+}
+function formatEvaluationResult(value) {
+	if (value.kind === "interval") return `[${value.min}, ${value.max}]`;
+	return `${value.magnitude}`;
+}
+function formatCorpusReport(report) {
+	const lines = [];
+	for (const { label, report: documentReport } of report.documents) {
+		const coverageText = documentReport.coverage === void 0 ? "no stated answers" : `${(documentReport.coverage * 100).toFixed(1)}% (${documentReport.matched}/${documentReport.matched + documentReport.mismatched})`;
+		lines.push(`${label}: ${coverageText}`);
+		for (const outcome of documentReport.outcomes) if (outcome.outcome !== "match") lines.push(`  ${formatOutcome(outcome)}`);
+	}
+	const combinedText = report.coverage === void 0 ? "no stated answers in corpus" : `${(report.coverage * 100).toFixed(1)}% (${report.matched}/${report.matched + report.mismatched}), ${report.gaps} gap(s), ${report.unresolved} unresolved`;
+	lines.push(`TOTAL: ${combinedText}`);
+	return lines.join("\n");
+}
+//#endregion
+export { DivisionByZeroError, IncompatibleDimensionsError, NonConvergentSolveError, NumericDomainError, UnboundSymbolError, UnknownUnitError, UnsupportedExpressionError, absInterval, absQuantity, addIntervals, addQuantities, addRational, collectFormulas, cosQuantity, dimensionExponent, dimensionToString, dimensionsEqual, divideDimensions, divideIntervals, divideQuantities, divideRational, evaluate, formatCorpusReport, interval, isDimensionless, isInterval, multiplyDimensions, multiplyIntervals, multiplyQuantities, multiplyRational, negateInterval, negateQuantity, pointInterval, powQuantity, quantity, rationalToNumber, runCorpus, runWorkedExampleSequence, scaleDimension, sinQuantity, solveFor, sqrtQuantity, subtractIntervals, subtractQuantities, subtractRational, tanQuantity, toExactRational, toRational };

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "document-compute.js",
-  "version": "1.1.0",
+  "version": "1.2.1",
   "description": "Units-typed, tree-walking evaluator for document-schema.js's MathExpression -- exact-rational unit conversion, interval arithmetic, and bisection/Newton numeric solve-for, the compute package for the documents.js family.",
   "type": "module",
   "repository": {
@@ -65,14 +65,16 @@
   },
   "packageManager": "pnpm@11.6.0",
   "dependencies": {
-    "document-schema.js": "^5.1.0"
+    "document-schema.js": "^5.2.0"
   },
   "devDependencies": {
     "@arethetypeswrong/cli": "^0.18.5",
     "@cloudflare/vitest-pool-workers": "^0.20.1",
     "@types/node": "^26.1.2",
+    "documents.js": "^6.1.3",
     "eslint": "^10.8.0",
     "husky": "^9.1.7",
+    "markdown-codec": "^6.1.1",
     "publint": "^0.3.21",
     "tsdown": "^0.22.13",
     "turbo": "^2.10.8",