@bldrs-ai/conway 1.323.1029 → 1.325.1034

package/README.md

@@ -178,7 +178,7 @@ Every PR is gated on two checks defined in `.github/workflows/build.yml`:
 | Job | What it does |
 |---|---|
 | `build` | `yarn install`, WASM + TS compile (WASM cached on the `conway-geom` submodule SHA), `yarn test`, `yarn lint`. |
-| `run-ifc-regression` | `needs: build`. Reuses the same WASM cache. Runs the regression batch against the pinned `test-models` tag, posts a per-PR comment with `failed.csv` / `errors.csv` summaries, and uploads the candidate npm tarball as a workflow artifact. |
+| `run-ifc-regression` | `needs: build`. Reuses the same WASM cache. Runs the regression batch against the pinned `test-models` tag, posts a per-PR comment with `failed.csv` / `errors.csv` / perf summaries, and uploads the candidate npm tarball + `perf.csv` as workflow artifacts. |
 
 A merge to `main` re-runs those two jobs and then chains into `auto-publish` (see [Releases](#releases) below).
 
@@ -186,9 +186,11 @@ A merge to `main` re-runs those two jobs and then chains into `auto-publish` (se
 
 The same batch the regression CI job runs can be invoked locally — see [regression/README.md](regression/README.md) for digest / verbose / batch modes and the catalog of model fixtures. The CI pinned tag is `TEST_MODELS_TAG` near the top of `build.yml`.
 
-## Performance benchmarks (local only today)
+## Performance benchmarks
 
-Perf isn't wired into CI yet. The current flow is a local `scripts/performance_test.sh` against a checkout of [headless-three](https://github.com/bldrs-ai/headless-three) cross-linked via `yarn link` against the working tree; see [scripts/README.md](scripts/README.md). Wiring perf into CI is tracked in #314 — see [Roadmap](#roadmap).
+**Tier 1 — Conway-only perf in CI (live).** Every regression run emits a `perf.csv` of `parseTimeMs / geometryTimeMs / totalTimeMs / rssMb / heapUsedMb / heapTotalMb` per model. The top-10 slowest are posted in the PR comment; the full CSV is uploaded as a workflow artifact. This piggybacks on the existing regression batch so cost is ~0 extra runner minutes.
+
+**Tier 2 — full headless-three perf (planned, #314).** A `perf-three` job on `push: main` that runs `scripts/benchmark.cjs` against a clone of [headless-three](https://github.com/bldrs-ai/headless-three), installing the candidate Conway tarball as H3's `@bldrs-ai/conway` dep. Captures PNG renders + per-model timings via H3's rendering server, including a parallel job for `test-models-private`. Tracked in #314.
 
 
 # Releases
@@ -278,27 +280,23 @@ follow-ups to deepen its coverage. Big items first.
 
 ### #314 — Performance benchmarks in CI
 
-Today perf only runs locally before a major release, against a
-[headless-three](https://github.com/bldrs-ai/headless-three) checkout
-cross-linked via `yarn link`. That makes perf regressions easy to miss
-and impossible to gate merges on.
-
-Wiring perf into CI as a third job (`needs: build`, parallel to
-`run-ifc-regression`) is the biggest open improvement. The blocker is
-deciding how to host the H3 dependency in the runner — three
-candidates:
-
-- **Vendor H3 as a submodule** of conway. Tight coupling but simplest
-  build chain.
-- **Pre-built H3 container image** that the perf job pulls. Looser
-  coupling, slower runner startup, more infra.
-- **A Conway-only perf harness** that bypasses H3 entirely — measure
-  parse + geometry timings directly off the CLI. Less faithful to the
-  Share end-to-end picture but the easiest to land.
-
-Once perf runs in CI, the comment posted on each PR should include a
-delta vs. main so regressions show up alongside the regression-batch
-results.
+**Tier 1 — Conway-only perf (done):** the regression batch now emits a
+per-model `perf.csv` (parse/geometry/total time + RSS/heap MB),
+piggybacking on the existing run so there's no extra runner cost. Top-10
+slowest models appear in each PR comment; the full CSV uploads as an
+artifact. See "Performance benchmarks" above.
+
+**Tier 2 — full headless-three perf (next):** an end-to-end render-and-time
+job that runs `scripts/benchmark.cjs` against a clone of
+[headless-three](https://github.com/bldrs-ai/headless-three). To avoid the
+local `yarn link` toolchain, the job installs the candidate Conway tarball
+(produced by `run-ifc-regression`) as H3's `@bldrs-ai/conway` dep and pins
+`H3_TAG` in `build.yml`. Gated to `push: main` to keep PR runtime down
+while still exercising every release. A sibling job runs against
+`test-models-private` using `secrets.TEST_MODELS_PRIVATE_TOKEN`, never on
+PR events (so forks can't leak the secret), and uploads results as
+org-scoped artifacts. Delta vs. the previous release lands in the
+auto-publish summary.
 
 ### #315 — Bump pinned `TEST_MODELS_TAG`
 

package/compiled/examples/browser-bundled.cjs

@@ -69608,7 +69608,7 @@ var IfcStepParser = class extends StepParser {
 IfcStepParser.Instance = new IfcStepParser();
 
 // compiled/src/version/version.js
-var versionString = "Conway Web-Ifc Shim v1.323.1029";
+var versionString = "Conway Web-Ifc Shim v1.325.1034";
 
 // compiled/src/statistics/statistics.js
 var Statistics = class {

package/compiled/examples/cli-bundled.cjs

@@ -85869,7 +85869,7 @@ var IfcSceneBuilder = class {
 };
 
 // compiled/src/version/version.js
-var versionString = "Conway Web-Ifc Shim v1.323.1029";
+var versionString = "Conway Web-Ifc Shim v1.325.1034";
 
 // compiled/src/statistics/statistics.js
 var Statistics = class {

package/compiled/examples/validator-bundled.cjs

@@ -69606,7 +69606,7 @@ var IfcStepParser = class extends StepParser {
 IfcStepParser.Instance = new IfcStepParser();
 
 // compiled/src/version/version.js
-var versionString = "Conway Web-Ifc Shim v1.323.1029";
+var versionString = "Conway Web-Ifc Shim v1.325.1034";
 
 // compiled/src/statistics/statistics.js
 var Statistics = class {

package/compiled/src/ifc/ifc_regression_batch_main.js

@@ -126,6 +126,56 @@ async function runDiff(ifcFolder, outputFolder, target, diffOutputPath, isDryRun
         await exec(`git checkout -- "${outputFolder}"`, { cwd: ifcFolder });
     }
 }
+/**
+ * Aggregate all per-file `*.perf.csv` rows in perfDir into a single CSV at
+ * outputCsvPath, sorted by file name. Files are written by individual child
+ * regression runs (one row + one header per file); we keep the header from
+ * the first file we read and concatenate the data rows from the rest. The
+ * input directory is removed afterwards.
+ *
+ * @param perfDir Directory the children wrote their per-file perf CSVs to.
+ * @param outputCsvPath Aggregate perf CSV destination.
+ */
+async function aggregatePerfCsvs(perfDir, outputCsvPath) {
+    const entries = await fsPromises.readdir(perfDir, { withFileTypes: true });
+    const perfFiles = entries
+        .filter((d) => d.isFile() && d.name.endsWith('.perf.csv'))
+        .map((d) => path.join(perfDir, d.name));
+    if (perfFiles.length === 0) {
+        console.warn(`No per-file perf CSVs found in ${perfDir}; skipping aggregate.`);
+        return;
+    }
+    const rows = [];
+    let header;
+    for (const perfFile of perfFiles) {
+        const contents = await fsPromises.readFile(perfFile, 'utf8');
+        const lines = contents.split(/\r?\n/).filter((l) => l.length > 0);
+        if (lines.length < 2) {
+            continue;
+        }
+        if (header === undefined) {
+            header = lines[0];
+        }
+        // Each per-file CSV has exactly one data row (lines[1]); the first column
+        // is the file name which we use for stable sorting.
+        const dataLine = lines[1];
+        const firstComma = dataLine.indexOf(',');
+        const fileKey = firstComma >= 0 ? dataLine.slice(0, firstComma) : dataLine;
+        rows.push({ file: fileKey, line: dataLine });
+    }
+    rows.sort((a, b) => a.file.localeCompare(b.file));
+    // Defensive: if every child wrote a malformed CSV (lines.length < 2 for all),
+    // header stays undefined and rows stays empty. Bail rather than write the
+    // literal "undefined" as a CSV header.
+    if (rows.length === 0 || header === undefined) {
+        console.warn(`No usable perf rows in ${perfDir} (${perfFiles.length} file(s) checked); ` +
+            `skipping aggregate write.`);
+        return;
+    }
+    const body = rows.map((r) => r.line).join('\n');
+    await fsPromises.writeFile(outputCsvPath, `${header}\n${body}\n`);
+    console.log(`Wrote aggregate perf CSV: ${outputCsvPath} (${rows.length} rows)`);
+}
 let totalTime = 0; // To keep track of the running total time
 /**
  * Run a regression test digest for a file.
@@ -133,11 +183,13 @@ let totalTime = 0; // To keep track of the running total time
  * @param filePath
  * @param outputPath
  * @param maxTimeout
+ * @param perfPath Optional path the child should write a one-row perf CSV to.
  */
-async function runForFile(filePath, outputPath, maxTimeout) {
+async function runForFile(filePath, outputPath, maxTimeout, perfPath) {
     const MAX_TIMEOUT_MS = maxTimeout;
     const startTime = Date.now(); // Start time
-    const safeExecCommand = `node --experimental-specifier-resolution=node ./compiled/src/ifc/ifc_regression_main.js -d "${filePath}" "${outputPath}"`;
+    const perfFlag = perfPath ? ` --perf "${perfPath}"` : '';
+    const safeExecCommand = `node --experimental-specifier-resolution=node ./compiled/src/ifc/ifc_regression_main.js -d${perfFlag} "${filePath}" "${outputPath}"`;
     console.log(`Current File: ${filePath}`);
     // Use safeExecWithCancellation, will kill the process if it takes longer than MAX_TIMEOUT_MS.
     const process = await safeExecWithCancellation(safeExecCommand, MAX_TIMEOUT_MS);
@@ -235,8 +287,9 @@ function getSystemMemoryUsagePercent() {
  * @param failedLines
  * @param memUtilization
  * @param maxTimeout
+ * @param perfDir If set, the child writes its perf CSV here as <basename>.perf.csv.
  */
-async function processIFCFilesInParallel(ifcFiles, outputPath, errorLines, fileLines, failedLines, memUtilization, maxTimeout) {
+async function processIFCFilesInParallel(ifcFiles, outputPath, errorLines, fileLines, failedLines, memUtilization, maxTimeout, perfDir) {
     const concurrencyLimit = os.cpus().length;
     console.log(`Concurrency: ${concurrencyLimit} threads - Max Timeout: ${maxTimeout} ms`);
     const limit = pLimit(concurrencyLimit);
@@ -251,7 +304,10 @@ async function processIFCFilesInParallel(ifcFiles, outputPath, errorLines, fileL
         }
         activeTasks++;
         console.log(`Starting task for "${path.basename(ifcPath)}". Active tasks: ${activeTasks}`);
-        const fileResults = await runForFile(ifcPath, path.join(outputPath, path.basename(ifcPath, '.ifc')), maxTimeout);
+        const perfChildPath = perfDir ?
+            path.join(perfDir, `${path.basename(ifcPath, '.ifc')}.perf.csv`) :
+            undefined;
+        const fileResults = await runForFile(ifcPath, path.join(outputPath, path.basename(ifcPath, '.ifc')), maxTimeout, perfChildPath);
         activeTasks--;
         console.log(`Completed task for "${path.basename(ifcPath)}". Active tasks: ${activeTasks}`);
         return { ifcPath, fileResults };
@@ -286,8 +342,9 @@ async function processIFCFilesInParallel(ifcFiles, outputPath, errorLines, fileL
  * @param fileLines
  * @param failedLines
  * @param maxTimeout
+ * @param perfDir If set, the child writes its perf CSV here as <basename>.perf.csv.
  */
-async function recursiveWalk(parentPath, excludeRegex, outputPath, errorLines, fileLines, failedLines, maxTimeout) {
+async function recursiveWalk(parentPath, excludeRegex, outputPath, errorLines, fileLines, failedLines, maxTimeout, perfDir) {
     const items = await fsPromises.readdir(parentPath, { withFileTypes: true });
     items.sort((a, b) => (a.name > b.name ? 1 : -1));
     for (const item of items) {
@@ -296,10 +353,13 @@ async function recursiveWalk(parentPath, excludeRegex, outputPath, errorLines, f
             continue;
         }
         if (item.isDirectory()) {
-            await recursiveWalk(resolved, excludeRegex, outputPath, errorLines, fileLines, failedLines, maxTimeout);
+            await recursiveWalk(resolved, excludeRegex, outputPath, errorLines, fileLines, failedLines, maxTimeout, perfDir);
         }
         else if (path.extname(resolved).toLowerCase() === '.ifc') {
-            const fileResults = await runForFile(resolved, path.join(outputPath, path.basename(resolved, '.ifc')), maxTimeout);
+            const perfChildPath = perfDir ?
+                path.join(perfDir, `${path.basename(resolved, '.ifc')}.perf.csv`) :
+                undefined;
+            const fileResults = await runForFile(resolved, path.join(outputPath, path.basename(resolved, '.ifc')), maxTimeout, perfChildPath);
             if (fileResults.type === 'Run') {
                 if (fileResults.errorLines !== void 0) {
                     errorLines.push(...fileResults.errorLines);
@@ -368,6 +428,13 @@ const args = yargs(process.argv.slice(SKIP_PARAMS))
         alias: 'timeout',
         default: 150000,
     });
+    yargs2.option('perf', {
+        describe: 'Output path for aggregate perf CSV. Each child writes a one-row ' +
+            '<basename>.perf.csv to a temp dir; on completion they are merged ' +
+            'into this file, sorted by file name. Disabled when unset.',
+        type: 'string',
+        default: '',
+    });
     yargs2.positional('model_folder', {
         describe: 'Folder containing IFC files, recursively walked',
         type: 'string',
@@ -388,10 +455,20 @@ const args = yargs(process.argv.slice(SKIP_PARAMS))
     const doParallel = argv['parallel'] ?? false; // <--- read the parallel flag
     const memUtilization = argv['mem-utilization'];
     const maxTimeout = argv['timeout'];
+    const perfOutputPath = (argv['perf'] ?? '').trim();
     if (changes.length === 0) {
         changes = path.join(outputPath, 'changes');
     }
     await fsPromises.mkdir(outputPath, { recursive: true });
+    // When perf is requested, children write their one-row CSVs into a
+    // throwaway temp dir; we aggregate and clean up afterwards. Keeping
+    // them out of outputPath avoids the runDiff step picking up
+    // machine-specific timings as "changes" against the test-models
+    // checked-in baselines.
+    let perfTmpDir;
+    if (perfOutputPath.length > 0) {
+        perfTmpDir = await fsPromises.mkdtemp(path.join(os.tmpdir(), 'conway-perf-'));
+    }
     const mainPath = path.join(outputPath, 'main.csv');
     const errorPath = path.join(outputPath, 'errors.csv');
     const failedPath = path.join(outputPath, 'failed.csv');
@@ -404,16 +481,30 @@ const args = yargs(process.argv.slice(SKIP_PARAMS))
         // 1) Collect all IFC files first
         const allIFCFiles = await collectIFCFiles(ifcFolder, excludeRegex);
         // 2) Process them in parallel
-        await processIFCFilesInParallel(allIFCFiles, outputPath, errorLines, fileLines, failedLines, memUtilization, maxTimeout);
+        await processIFCFilesInParallel(allIFCFiles, outputPath, errorLines, fileLines, failedLines, memUtilization, maxTimeout, perfTmpDir);
     }
     else {
         console.log('Processing in serial mode...');
-        await recursiveWalk(ifcFolder, excludeRegex, outputPath, errorLines, fileLines, failedLines, maxTimeout);
+        await recursiveWalk(ifcFolder, excludeRegex, outputPath, errorLines, fileLines, failedLines, maxTimeout, perfTmpDir);
     }
     // Write out results
     await fsPromises.writeFile(mainPath, `file,hash,errors\n${fileLines.join('')}`);
     await fsPromises.writeFile(errorPath, `${errorCSVHeader}\n${errorLines.join('')}`);
     await fsPromises.writeFile(failedPath, `file,code,signal\n${failedLines.join('')}`);
+    // Aggregate per-child perf rows (if requested) before runDiff so the
+    // run completes deterministically even when the git diff step is
+    // skipped or fails.
+    if (perfTmpDir) {
+        try {
+            await aggregatePerfCsvs(perfTmpDir, perfOutputPath);
+        }
+        catch (e) {
+            console.error('Failed to aggregate perf CSVs:', e);
+        }
+        finally {
+            await fsPromises.rm(perfTmpDir, { recursive: true, force: true });
+        }
+    }
     // If user wants a git diff
     await runDiff(ifcFolder, outputPath, target, changes, dryRun);
     // ---- New: log total runtime

package/compiled/src/ifc/ifc_regression_main.js

@@ -36,6 +36,50 @@ function csvSafeString(from) {
     }
     return from;
 }
+// Bytes per megabyte for memory-stat formatting in the perf CSV.
+// eslint-disable-next-line no-magic-numbers
+const BYTES_PER_MB = 1024 * 1024;
+// Fixed-point precision for perf MB values.
+// eslint-disable-next-line no-magic-numbers
+const PERF_MB_PRECISION = 2;
+/**
+ * Write a single-row per-file perf CSV at the given path. Memory snapshot is
+ * taken at call time; for the OK path this is right after geometry extraction
+ * — close to peak. No-op when perfPath is empty.
+ *
+ * Memory semantics: `rssMb` includes the conway-geom WASM heap (mmap'd into
+ * the process) and is the load-bearing memory metric for geometry work.
+ * `heapUsedMb` / `heapTotalMb` are V8-only — useful for tracking JS-side
+ * allocation pressure but they exclude WASM-side buffers. Expect a large
+ * gap between rss and heap on geometry-heavy models.
+ *
+ * @param perfPath Path to write the CSV to. Empty string disables.
+ * @param ifcFile Source IFC file path (basename used as the row key).
+ * @param status OK or FAIL.
+ * @param parseTimeMs Parse stage duration in ms.
+ * @param geometryTimeMs Geometry extraction duration in ms.
+ * @param totalTimeMs Sum of parse + geometry in ms.
+ */
+async function writePerfCsvIfRequested(perfPath, ifcFile, status, parseTimeMs, geometryTimeMs, totalTimeMs) {
+    if (perfPath.length === 0) {
+        return;
+    }
+    const mem = process.memoryUsage();
+    const rssMb = (mem.rss / BYTES_PER_MB).toFixed(PERF_MB_PRECISION);
+    const heapUsedMb = (mem.heapUsed / BYTES_PER_MB).toFixed(PERF_MB_PRECISION);
+    const heapTotalMb = (mem.heapTotal / BYTES_PER_MB).toFixed(PERF_MB_PRECISION);
+    const fileName = csvSafeString(path.basename(ifcFile));
+    const header = "file,status,parseTimeMs,geometryTimeMs,totalTimeMs,rssMb,heapUsedMb,heapTotalMb\n";
+    const row = `${fileName},${status},${parseTimeMs},${geometryTimeMs},${totalTimeMs},` +
+        `${rssMb},${heapUsedMb},${heapTotalMb}\n`;
+    try {
+        await fsPromises.writeFile(perfPath, header + row);
+    }
+    catch (e) {
+        // Perf is best-effort; never fail the regression run because of a perf write.
+        console.error(`Failed to write perf CSV at ${perfPath}:`, e);
+    }
+}
 /**
  * Display errors and dump errors errors to stderr
  *
@@ -94,6 +138,12 @@ function doWork() {
             alias: "v",
             default: false,
         });
+        yargs2.option("perf", {
+            describe: "Write a single-row perf CSV (parse/geometry/total time + memory) at this path",
+            type: "string",
+            alias: "p",
+            default: "",
+        });
         yargs2.positional("filename", { describe: "IFC File Paths", type: "string" });
         yargs2.positional("output", { describe: "Output path", type: "string" });
     }, async (argv) => {
@@ -104,6 +154,7 @@ function doWork() {
         const strict = argv["strict"] ?? false;
         const digest = argv["digest"] ?? false;
         const verbose = argv["verbose"] ?? false;
+        const perfPath = argv["perf"] ?? "";
         try {
             indexIfcBuffer = fs.readFileSync(ifcFile);
         }
@@ -119,6 +170,7 @@ function doWork() {
         }
         const parser = IfcStepParser.Instance;
         const bufferInput = new ParsingBuffer(indexIfcBuffer);
+        const parseStartMs = Date.now();
         const result0 = parser.parseHeader(bufferInput)[1];
         switch (result0) {
             case ParseResult.COMPLETE:
@@ -155,11 +207,20 @@ function doWork() {
                 break;
             default:
         }
+        const parseEndMs = Date.now();
+        const parseTimeMs = parseEndMs - parseStartMs;
         if (model === void 0) {
+            await writePerfCsvIfRequested(perfPath, ifcFile, "FAIL", parseTimeMs, 0, parseTimeMs);
             return;
         }
         model.nullOnErrors = !strict;
+        const geomStartMs = Date.now();
         const result = await geometryExtraction(model);
+        const geomEndMs = Date.now();
+        const geometryTimeMs = geomEndMs - geomStartMs;
+        const totalTimeMs = geomEndMs - parseStartMs;
+        const perfStatus = result === void 0 ? "FAIL" : "OK";
+        await writePerfCsvIfRequested(perfPath, ifcFile, perfStatus, parseTimeMs, geometryTimeMs, totalTimeMs);
         if (result === void 0) {
             Logger.error("Couldn't extract geometry");
         }

package/compiled/src/version/version.js

@@ -1,2 +1,2 @@
-const versionString = 'Conway Web-Ifc Shim v1.323.1029';
+const versionString = 'Conway Web-Ifc Shim v1.325.1034';
 export { versionString };