@swiftbrowser/wasm-compiler 0.0.0-stage → 0.8.0

package/README.md

@@ -1,3 +1,300 @@
-# Temporary Holding Version
+# @swiftbrowser/wasm-compiler
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+The Swift compiler and linker built as WebAssembly, and what runs them:
+Milestone 1 of [the in-browser compiler plan](../../docs/in-browser-compiler.md).
+Not published yet.
+
+- `src/driver.ts` plans the jobs `swiftc` would run for
+  `wasm32-unknown-wasip1`: one `swift-frontend` job per source, then
+  `wasm-ld`. Pure TypeScript, so the browser can use it.
+- `src/wasm.ts` reads the autolink entries out of the objects, in place of
+  `swift-autolink-extract`.
+- `src/compile.ts` runs those jobs on a host. `src/node` has two hosts: the
+  native toolchain, and the WebAssembly tools under Node's WASI.
+- `src/cli.ts` is `swiftbrowser-wasmc`.
+- `toolchain/` downloads the inputs and builds the WebAssembly tools.
+
+## The CLI
+
+```sh
+npm run build --workspace packages/wasm-compiler
+node packages/wasm-compiler/bin/swiftbrowser-wasmc.js \
+  --sdk <swift-sdk>/wasm32-unknown-wasip1 --tools $WASMC_WORK/build/tools \
+  --module-cache ~/.cache/swiftbrowser-wasmc \
+  --module-name Hello -o hello.wasm --run main.swift
+```
+
+`toolchain/install-tools.sh` puts `swift-frontend.wasm` and `wasm-ld.wasm`
+in `$WASMC_WORK/build/tools`. The first compile builds Clang's modules for
+SwiftShims and wasi-libc into the module cache; later ones reuse them.
+
+`--native-toolchain <dir>` runs the same jobs with a native Swift toolchain
+instead, which is how the driver is checked on a machine with Swift.
+
+## Where it stands
+
+Milestone 1 works: the CLI compiles `Sources/HelloWasm/main.swift` with
+`swift-frontend.wasm`, links it with `wasm-ld.wasm`, both under Node's WASI,
+and the result prints its line under Node and wasmtime. Multi-file modules
+and diagnostics (shown with host paths) work too.
+
+| | Time |
+|---|---|
+| `swift-frontend.wasm -c`, HelloWasm, empty module cache | 1.55 s |
+| the same, module cache warm | 1.35 s |
+| `wasm-ld.wasm` | 0.61 s |
+| the whole CLI run | 2.0–2.3 s |
+
+The same host, native tools: 0.07 s and 0.19 s (below). Most of the
+WebAssembly time is V8 compiling the modules lazily as functions are first
+called; wasmtime compiles the whole frontend up front, about 45 s per run.
+
+What came out of the check against native (steps 1.3 and 1.4): the frontend's
+object, linked by the native `wasm-ld`, runs; its bytes match the native
+frontend's but for 16 bytes the native one randomizes too. `wasm-ld.wasm`
+and the native `wasm-ld` write byte-identical executables.
+
+The tools' sizes (brotli at quality 11 with a 16 MB window):
+
+| | Raw | gzip | brotli |
+|---|---|---|---|
+| `swift-frontend.wasm`, assertions, not stripped | 168.3 MB | 40.7 MB | 25.0 MB |
+| `swift-frontend.wasm`, no assertions, stripped | 118.8 MB | 35.0 MB | 21.5 MB |
+| `wasm-ld.wasm`, not stripped | 54.4 MB | 13.8 MB | 8.7 MB |
+| `wasm-ld.wasm`, stripped | 39.9 MB | 12.4 MB | 7.8 MB |
+
+Stripping (`STRIP=ON toolchain/install-tools.sh`) does most of it: the name
+section is 43 MB of the frontend. Turning off assertions saves 7 MB raw.
+`wasm-opt -Os` on the stripped linker saves 8 MB raw but almost nothing
+compressed (7.8 → 7.7 MB brotli), so it is not used. The no-assertions
+frontend compiles hello world in 1.15–1.3 s under Node.
+
+**The in-memory host** (`memoryHost` in `src/memory-host.ts`, over
+`@bjorn3/browser_wasi_shim`, which is what the browser will use) runs the
+same compile, link and program in about 1.3 s, building Clang's modules
+in memory as it goes. `toolchain/record-sdk-files.mjs` drives it and logs
+which SDK files the tools open: for hello world, 53 of the SDK's 3324 files,
+46.6 MB, listed in `sdk-manifests/hello.json`. Most of that is
+`Swift.swiftmodule` (17 MB), `libswiftCore.a` (12 MB) and `Swift.swiftdoc`
+(4.5 MB, opened alongside the module; whether a compile can do without it
+is for Milestone 3's download work).
+
+Things a 32-bit host ran into, all fixed in `toolchain/patches`:
+
+- Types that assume pointers have three spare low bits (`PointerIntPair`
+  with 3 bits, custom `PointerLikeTypeTraits`) get explicit `alignas` or a
+  plain struct on 32-bit.
+- `BridgedSwiftObject`'s refcount field is 32 bits on a 32-bit host, as the
+  Swift object header is.
+- Clang's `ZeroFilledMemoryBuffer` uses `calloc` instead of `mmap`, and
+  IRGen frees the SIL module on the same thread, since there are none.
+
+Node 22's V8 crashes in the garbage collector with fast API calls when a
+module this size runs under WASI; `src/node/wasi-host.ts` turns them off
+(`--no-turbo-fast-api-calls`).
+
+## Apps (Milestone 2)
+
+`SWIFTBROWSER_COMPILER=wasm` (or `auto`; `packages/cli/src/compiler.ts`)
+makes the CLI (`packages/cli`) build an app with these tools instead of
+`swift build`, for `build` and `dev` alike, and type-check it for `check`
+(`packages/cli/src/wasmCompiler.ts`), with no Swift install: the CLI
+downloads the Swift SDK for WebAssembly itself and uses the released kit,
+whose macros come built as WebAssembly. It reproduces SwiftPM's release build
+of the staged package: one whole-module `-O` frontend job against the kit's
+modules, Foundation's trimmed ICU data as an object written in TypeScript
+(`dataObject` in `src/object.ts`, in place of clang assembling `.incbin`),
+and a `wasm-ld` link with the app's exports and the kit's archives, linked
+whole. Package dependencies, copied for the shim or plain, are compiled
+module by module first (`packages/cli/src/wasmPackages.ts`), only the
+targets the app's products need. Without SwiftPM their manifests run on
+these tools: each is compiled against PackageDescription built for
+WebAssembly (`toolchain/build-manifest-api.sh`, shipped as `manifest-api/`
+in @swiftbrowser/wasm-tools), linked with Foundation, and run under WASI
+the way SwiftPM runs it (`packages/cli/src/manifest.ts`). C targets
+compile with clang built as WebAssembly (`clang.wasm`, from the same LLVM
+build as `wasm-ld`; `compileC` in `src/compile.ts`). Macro targets in
+packages are linked as WebAssembly executables, like the kit's macros, and
+handed to the host as they are built (`WasiHost.addPlugin`).
+
+Every example, built this way and run headless, gives the same op stream as
+its native build (`scripts/snapshot-test.sh`'s check). With tools release
+`swift-6.4.0-r3`:
+
+| Example | Macros | Frontend (`-O`, whole module) | `wasm-ld` | Snapshot |
+|---|---|---|---|---|
+| Counter, 3 files | | 3.7 s, 130 MB | 2.8 s, 409 MB | 111 ops, same |
+| Filters | | 5.4 s, 137 MB | 2.8 s, 409 MB | 145 ops, same |
+| Gallery | | 9.0 s, 144 MB | 3.0 s, 410 MB | 360 ops, same |
+| PhotoPicker | | 4.8 s, 137 MB | 2.8 s, 413 MB | 61 ops, same |
+| FoodTruck, 57 files | | 36.8 s, 227 MB | 2.7 s, 415 MB | 4064 ops, same |
+| Settings | `@Observable` | 7.2 s, 143 MB | 2.8 s, 410 MB | 154 ops, same |
+| Todos | `@Model`, `@Query` | 7.3 s, 142 MB | 2.7 s, 408 MB | 198 ops, same |
+| Journal | SwiftData | 8.0 s, 145 MB | 2.7 s, 409 MB | 85 ops, same |
+| Bookshelf | SwiftData, `#Predicate` | 9.0 s, 149 MB | 2.8 s, 415 MB | 341 ops, same |
+| CupcakeCorner | `@Observable` | 6.9 s, 144 MB | 2.6 s, 409 MB | 104 ops, same |
+| HackingWithSwift, 16 files | SwiftData, `@Observable` | 15.3 s, 172 MB | 2.7 s, 410 MB | 1182 ops, same |
+| Fidelity | `@Observable`, `@Entry` | 17.9 s, 168 MB | 2.7 s, 412 MB | 635 ops, same |
+| HotProspects + CodeScanner | SwiftData | 7.8 s + 4.2 s, 149 MB | 3.3 s, 424 MB | 167 ops, same |
+
+Time is wall time under Node on the 4-core container (module cache warm);
+memory is each tool's linear memory when it exited, which is its peak. The
+native frontend compiles Counter in 1.1 s (below), so the WebAssembly one
+runs at about a third of native speed. Linking the kit's archives whole
+costs about 3 s and 410 MB whatever the app, which is cheap enough that
+static linking stays (Milestone 4's dynamic linking is not needed for
+time). In the dev server, Counter builds in 6.3 s and rebuilds on save.
+
+### Macros
+
+A native compiler runs a macro plugin as a process. Here the plugins are
+WebAssembly too (`toolchain/build-plugins.sh`: the shim's SwiftDataMacros,
+the toolchain's ObservationMacros and SwiftMacros, the standard
+library's, each given a `@main`, and swift-testing's TestingMacros), built with
+swift-syntax for WASI, and the host runs them in the compiler's own thread
+(`src/plugins.ts`): the compiler, patched to ask the host instead of
+spawning (`PluginRegistry.cpp`), imports `plugin_spawn`, `plugin_write`,
+`plugin_read` and `plugin_close` from a `swiftbrowser` module; the host
+calls the plugin's `swift_wasm_macro_v1_pump` export once per message, with
+the message on its stdin and the reply taken from its stdout. Everything is
+synchronous, so it works the same under Node and in a browser worker; the
+e2e tests expand `@Observable` on both hosts, and CI does on a runner
+without Swift. A compiler is named a plugin with
+`-load-plugin-executable /plugins/ObservationMacros.wasm#ObservationMacros`;
+the CLIs find the plugins in `plugins/` next to the tools.
+
+Expanding macros needs the compiler built with swift-syntax and ASTGen
+(`SWIFT_SYNTAX=ON toolchain/build-swift.sh`): swift-syntax as static
+libraries, the regex parser from swift-experimental-string-processing, and
+no plugin server or shared-library macros on WASI. The frontend grows from
+168 to 187 MB (27.3 MB brotli); each plugin is about 41 MB (7.2 MB brotli).
+
+### The module cache
+
+The first compile that imports Foundation builds Clang modules for
+SwiftShims, the WASI libc and Foundation's C parts: 13 MB of PCMs. On disk
+(the CLIs) they stay in the module cache between builds. In memory (the
+browser) a session starts empty, so `memoryHost({ moduleCache })` starts
+from a prebuilt cache and `moduleCacheFiles()` returns the one a session
+ends with. Every file in memory has the same timestamp, so a cache built
+for the same files and flags validates in any session and nothing is
+rebuilt; the e2e tests check that only the pruning timestamp changes.
+`toolchain/prebuild-module-cache.mjs` builds one by type-checking a source
+that imports Foundation and Observation (the cache directory is a hash of
+the flags, and a type-check and a compile share it): 8 modules, 13.0 MB,
+3.3 MB brotli, for the playground to ship with the SDK, built with the
+flags it compiles with.
+
+On the in-memory host, a program importing Foundation runs its frontend in
+2.8 s with an empty cache and 1.0 s with the prebuilt one. Before
+`writeAt` in `src/memory-host.ts`, the empty cache took 12.6 s:
+browser_wasi_shim grows a file to exactly the size of each write, so
+writing a 10 MB PCM in small writes copied it quadratically; writes now go
+into a buffer that doubles.
+
+### What the examples found
+
+- FoodTruck: Swift sized `debug_value` and `alloc_stack` instructions
+  without the padding before their 8-byte-aligned trailing debug-expression
+  elements, which SROA then wrote past on a 32-bit host (fixed from tools
+  release `swift-6.4.0-r2`).
+- HotProspects, built twice: LLVM maps a file read-only with
+  `MAP_NORESERVE` to compare an output with the one it replaces, which
+  wasi-libc's emulated `mmap` rejects, so rewriting an unchanged
+  `.swiftmodule` failed (fixed in `toolchain/patches/llvm-project`).
+- The in-memory host: `browser_wasi_shim` reports readlink as unsupported
+  rather than "not a link", and grants directories no rights, both of which
+  the compiler's checks on a plugin's path trip over; `src/memory-host.ts`
+  answers as a real WASI host does.
+
+## Building the tools
+
+The scripts in `toolchain/` keep everything under `$WASMC_WORK` (about 30 GB),
+in this order:
+
+```sh
+export WASMC_WORK=~/wasmc
+toolchain/fetch.sh               # Swift 6.4.0, its Swift SDK for Wasm, wasmtime; the sources, patched
+toolchain/build-native-tools.sh  # llvm-tblgen and clang-tblgen for the build machine
+toolchain/build-sysroot.sh       # the Swift SDK's sysroot, with a libc++ that has threads
+toolchain/build-resources.sh     # the Swift SDK's resources, with a libc module map C++ interop can use
+toolchain/build-llvm.sh          # LLVM, Clang and LLD for a wasm32-wasip1 host
+toolchain/check-llvm.sh          # clang.wasm and lld.wasm compile and link a C program that runs
+toolchain/build-cmark.sh         # swift-cmark for the same host
+toolchain/build-swift.sh         # swift-frontend for the same host, and the LLVM libraries it links
+toolchain/build-plugins.sh       # the macro plugins, as WebAssembly
+toolchain/build-manifest-api.sh  # PackageDescription and CompilerPluginSupport for WebAssembly, for manifests
+toolchain/install-tools.sh       # swift-frontend.wasm, wasm-ld.wasm, the plugins and manifest-api into $WASMC_WORK/build/tools
+```
+
+`toolchain/publish-tools.sh pack`, then `upload`, compresses the tools with
+brotli and puts them in Cloudflare R2 (bucket `swiftbrowser-wasm-tools`,
+under the release `versions.sh` names), and writes
+`packages/wasm-tools/tools.json`, which pins them. `@swiftbrowser/wasm-tools`
+(`packages/wasm-tools`) is how everything else gets them: its `fetch.mjs`
+downloads them from R2 into the package and checks them, the npm release
+publishes the package with them, and the `WebAssembly compiler` workflow
+(`.github/workflows/wasm-compiler.yml`) uses them to compile HelloWasm on a
+runner without Swift. The CLI takes them from that package when neither
+`--tools` nor `SWIFTBROWSER_WASM_TOOLS` is set. To use the published tools
+instead of building them:
+
+```sh
+CLOUDFLARE_API_TOKEN=... CLOUDFLARE_ACCOUNT_ID=... node packages/wasm-tools/fetch.mjs
+```
+
+Then, to record the SDK files a compile opens, on the in-memory host:
+
+```sh
+npm run build --workspace packages/wasm-compiler
+ulimit -s unlimited
+node --stack-size=65500 packages/wasm-compiler/toolchain/record-sdk-files.mjs \
+  --sdk <swift-sdk>/wasm32-unknown-wasip1 --tools $WASMC_WORK/build/tools \
+  --source Sources/HelloWasm/main.swift --out hello.json
+```
+
+The native toolchain's clang compiles everything against the Swift SDK's
+WASI sysroot, so the compiler's C++ and its Swift parts share one libc and
+one libc++. The compiler's Swift parts (SwiftCompilerSources, mandatory
+since Swift 6) are cross-compiled by the native `swiftc` with C++ interop.
+`toolchain/patches` holds the changes to LLVM, Swift and swift-syntax;
+after editing the sources under `$WASMC_WORK/src`, `toolchain/save-patches.sh`
+writes them back, each ending its description with the base it applies to.
+That base is pinned in `toolchain/versions.sh`: `SWIFT_TAG` and, per source
+tree, the commit the tag names, which `fetch.sh` checks every clone
+against. Moving to another Swift release means changing both, then
+rebasing the patches.
+
+## Tests
+
+`npm run test:unit` checks the driver against the jobs the native driver and
+clang run, recorded in `test/fixtures` by `toolchain/record-invocations.mjs`.
+With `WASMC_WORK` set, `test/e2e.test.ts` also compiles, links and runs hello
+world with the native tools, and with the WebAssembly tools once
+`$WASMC_WORK/build/tools` holds them.
+
+## Native baseline
+
+Step 1.1 measured the native Swift 6.4.0 frontend and `wasm-ld` for
+`wasm32-unknown-wasip1` (`toolchain/measure.py`: wall time and peak resident
+memory), on a 4-core container that was also building LLVM, so the times
+are on the slow side. The examples are staged by the CLI and compiled
+against its kit, with the toolchain's macro plugins (`@Observable` needs
+ObservationMacros).
+
+| Input | Step | Time | Peak memory |
+|---|---|---|---|
+| HelloWasm | `-typecheck` | 0.06 s | 101 MB |
+| HelloWasm | `-c -Onone` | 0.07 s | 114 MB |
+| HelloWasm | `-c -O` | 0.14 s | 129 MB |
+| HelloWasm | `wasm-ld` | 0.19 s | 119 MB |
+| Counter, 3 files | `-typecheck`, whole module | 0.13 s | 138 MB |
+| Counter | `-c -Onone`, one primary file | 0.20 s | 153 MB |
+| Counter | `-c -O -wmo` | 1.11 s | 228 MB |
+| Settings, 3 files | `-typecheck`, whole module | 0.30 s | 166 MB |
+| Settings | `-c -O -wmo` | 1.61 s | 200 MB |
+| HackingWithSwift, 16 files | `-typecheck`, whole module | 0.87 s | 186 MB |
+| HackingWithSwift | `-c -O -wmo` | 5.19 s | 245 MB |
+
+A compile stays far below the 4 GB a 32-bit WebAssembly memory allows, so
+the compiler is built for wasm32; Memory64 is not needed.

package/bin/swiftbrowser-wasmc.js

@@ -0,0 +1,18 @@
+#!/usr/bin/env node
+// Entry point of the `swiftbrowser-wasmc` command (see ../src/cli.ts).
+import { main } from '../dist/cli.js';
+
+// Node marks its WASI as experimental; every run would say so.
+process.removeAllListeners('warning');
+process.on('warning', (warning) => {
+  if (warning.name === 'ExperimentalWarning' && warning.message.includes('WASI')) return;
+  console.warn(warning);
+});
+
+main(process.argv.slice(2)).then(
+  (code) => process.exit(code),
+  (error) => {
+    console.error(error instanceof Error ? error.message : String(error));
+    process.exit(1);
+  },
+);

package/dist/cli.d.ts

@@ -0,0 +1 @@
+export declare function main(argv: string[]): Promise<number>;

package/dist/cli.js

@@ -0,0 +1,147 @@
+// `swiftbrowser-wasmc`: compiles Swift sources into a WASI executable with
+// the WebAssembly-hosted Swift compiler and linker, no native Swift needed.
+// Milestone 1 of docs/in-browser-compiler.md. `--native-toolchain` runs the
+// same jobs with a native toolchain instead, for comparison.
+import { copyFile, mkdtemp, readFile, rm } from 'node:fs/promises';
+import { tmpdir } from 'node:os';
+import path from 'node:path';
+import { parseArgs } from 'node:util';
+import { compile } from './compile.js';
+import { nativeHost } from './node/native-host.js';
+import { findPlugins, packagedToolsDir, toolFile } from './node/tools.js';
+import { runWasiCommand, wasiHost } from './node/wasi-host.js';
+const USAGE = `usage: swiftbrowser-wasmc [options] <file.swift>...
+
+Compiles Swift sources for wasm32-unknown-wasip1 with the Swift compiler and
+linker built as WebAssembly, run under Node's WASI.
+
+  -o, --output FILE         the executable to write (default: <module>.wasm)
+      --typecheck           type-check only
+      --module-name NAME    the module name (default: main)
+      --sdk DIR             the Swift SDK's wasm32-unknown-wasip1 directory
+                            (default: $SWIFTBROWSER_WASM_SDK)
+      --tools DIR           the directory holding swift-frontend.wasm and
+                            wasm-ld.wasm, or their .wasm.br (default:
+                            $SWIFTBROWSER_WASM_TOOLS, then the installed
+                            @swiftbrowser/wasm-tools); macro plugins are
+                            taken from its plugins/ directory
+      --module-cache DIR    keep Clang's module cache here between runs
+                            (default: a temporary directory per run)
+      --native-toolchain DIR
+                            run a native toolchain's tools instead
+      --run                 run the executable under Node's WASI after linking
+  -v, --verbose             print each tool invocation and how long it took
+  -h, --help`;
+export async function main(argv) {
+    const { values, positionals } = parseArgs({
+        args: argv,
+        allowPositionals: true,
+        options: {
+            output: { type: 'string', short: 'o' },
+            typecheck: { type: 'boolean' },
+            'module-name': { type: 'string' },
+            sdk: { type: 'string' },
+            tools: { type: 'string' },
+            'module-cache': { type: 'string' },
+            'native-toolchain': { type: 'string' },
+            run: { type: 'boolean' },
+            verbose: { type: 'boolean', short: 'v' },
+            help: { type: 'boolean', short: 'h' },
+        },
+    });
+    if (values.help) {
+        console.log(USAGE);
+        return 0;
+    }
+    if (positionals.length === 0) {
+        console.error(USAGE);
+        return 2;
+    }
+    const sdkRoot = values.sdk ?? process.env['SWIFTBROWSER_WASM_SDK'];
+    if (!sdkRoot)
+        throw new Error('no Swift SDK: pass --sdk or set SWIFTBROWSER_WASM_SDK');
+    const moduleName = values['module-name'] ?? 'main';
+    const output = values.typecheck ? undefined : path.resolve(values.output ?? `${moduleName}.wasm`);
+    const sources = positionals.map((file) => path.resolve(file));
+    const work = await mkdtemp(path.join(tmpdir(), 'wasmc-work-'));
+    try {
+        let host;
+        // Guest directory → host directory, to show diagnostics with real paths.
+        let guestToHost = [];
+        let guestSources;
+        let guestWork;
+        let frontendFlags = [];
+        if (values['native-toolchain']) {
+            host = nativeHost({ toolchain: path.resolve(values['native-toolchain']), sdkRoot: path.resolve(sdkRoot) });
+            guestSources = sources;
+            guestWork = work;
+        }
+        else {
+            const toolsDir = values.tools ?? process.env['SWIFTBROWSER_WASM_TOOLS'] ?? packagedToolsDir();
+            // Each source directory is mounted once, at /src/0, /src/1, ...
+            const sourceDirs = [...new Set(sources.map((file) => path.dirname(file)))];
+            const mounts = { '/sdk': path.resolve(sdkRoot), '/work': work };
+            sourceDirs.forEach((dir, i) => (mounts[`/src/${i}`] = dir));
+            guestSources = sources.map((file) => `/src/${sourceDirs.indexOf(path.dirname(file))}/${path.basename(file)}`);
+            guestToHost = Object.entries(mounts).filter(([guest]) => guest.startsWith('/src/'));
+            guestWork = '/work';
+            // The macro plugins next to the tools (plugins/), mounted at /plugins.
+            const plugins = Object.entries(findPlugins(toolsDir));
+            if (plugins.length > 0)
+                mounts['/plugins'] = path.dirname(plugins[0][1]);
+            const guestPlugin = (file) => `/plugins/${path.basename(file)}`;
+            frontendFlags = plugins.flatMap(([name, file]) => ['-load-plugin-executable', `${guestPlugin(file)}#${name}`]);
+            host = wasiHost({
+                tools: { 'swift-frontend': toolFile(toolsDir, 'swift-frontend'), 'wasm-ld': toolFile(toolsDir, 'wasm-ld') },
+                mounts,
+                sdkRoot: '/sdk',
+                plugins: Object.fromEntries(plugins.map(([, file]) => [guestPlugin(file), file])),
+                ...(values['module-cache'] ? { moduleCacheDir: path.resolve(values['module-cache']) } : {}),
+            });
+        }
+        if (values.verbose)
+            console.error(`using ${host.description}`);
+        let started = performance.now();
+        let current = null;
+        const finishStep = () => {
+            if (values.verbose && current)
+                console.error(`  ${current}: ${((performance.now() - started) / 1000).toFixed(2)} s`);
+        };
+        const result = await compile(host, {
+            moduleName,
+            sources: guestSources,
+            objectDir: guestWork,
+            ...(output ? { output: `${guestWork}/${moduleName}.wasm` } : {}),
+            ...(frontendFlags.length > 0 ? { frontendFlags } : {}),
+            onStep: (step) => {
+                finishStep();
+                current = step.tool;
+                started = performance.now();
+                if (values.verbose)
+                    console.error(`${step.tool} ${step.args.join(' ')}`);
+            },
+        });
+        finishStep();
+        if (result.output) {
+            let output = result.output;
+            for (const [guest, hostDir] of guestToHost)
+                output = output.split(`${guest}/`).join(`${hostDir}/`);
+            process.stderr.write(output);
+        }
+        if (!result.ok)
+            return 1;
+        if (!output)
+            return 0;
+        await copyFile(path.join(work, `${moduleName}.wasm`), output);
+        if (values.run) {
+            const module = await WebAssembly.compile(await readFile(output));
+            const run = await runWasiCommand({ module, args: [moduleName], mounts: {} });
+            process.stdout.write(run.output);
+            return run.exitCode;
+        }
+        return 0;
+    }
+    finally {
+        await rm(work, { recursive: true, force: true });
+    }
+}

package/dist/compile.d.ts

@@ -0,0 +1,69 @@
+import type { Host } from './host.js';
+export interface CompileRequest {
+    moduleName: string;
+    /** Source paths as the host's tools see them. */
+    sources: readonly string[];
+    /** Where objects go, as the tools see it. */
+    objectDir: string;
+    /** The executable to write, as the tools see it; omit to type-check only. */
+    output?: string;
+    /**
+     * Compile a library module instead of an executable: objects and this
+     * `.swiftmodule` (as the tools see it), for a later compile to import
+     * (`-I` its directory) and a later link to take the objects. Runs one
+     * whole-module job and links nothing.
+     */
+    emitModule?: string;
+    /**
+     * With `emitModule`, whether the module's sources are a library's (the
+     * default); false keeps a `main.swift`'s top-level code, for an executable
+     * module built for its tests to import.
+     */
+    parseAsLibrary?: boolean;
+    /** One frontend job for the whole module (`FrontendOptions.wholeModule`). */
+    wholeModule?: boolean;
+    /** The build's own frontend flags (`FrontendOptions.flags`). */
+    frontendFlags?: readonly string[];
+    /** Objects to link besides the module's own, as the tools see them. */
+    objects?: readonly string[];
+    /** The build's own linker flags (`LinkOptions.flags`). */
+    linkerFlags?: readonly string[];
+    /** Called with each step as it starts, for progress and timing. */
+    onStep?: (step: {
+        tool: string;
+        args: readonly string[];
+    }) => void;
+}
+export interface CompileResult {
+    ok: boolean;
+    /** Every tool's output, in order: the diagnostics. */
+    output: string;
+    /** Each step that ran: how long it took and, for a WebAssembly tool, its peak memory. */
+    steps: CompileStep[];
+    /** The objects the frontend wrote, as the tools see them. */
+    objects: string[];
+}
+export interface CompileStep {
+    tool: string;
+    exitCode: number;
+    seconds: number;
+    memoryBytes?: number;
+}
+export declare function compile(host: Host, request: CompileRequest): Promise<CompileResult>;
+export interface CompileCRequest {
+    /** C or C++ sources, as the tools see them. */
+    sources: readonly string[];
+    /** Where objects go: one per source, named for its path below `root`. */
+    objectDir: string;
+    /** The directory sources are named relative to, for their objects' names. */
+    root: string;
+    /** clang flags for every source (`ClangOptions.flags`). */
+    flags?: readonly string[];
+    /** Called with each step as it starts. */
+    onStep?: (step: {
+        tool: string;
+        args: readonly string[];
+    }) => void;
+}
+/** A C target's sources compiled with clang, one job per source, as SwiftPM builds one. */
+export declare function compileC(host: Host, request: CompileCRequest): Promise<CompileResult>;

package/dist/compile.js

@@ -0,0 +1,89 @@
+// What `swiftc -static-stdlib -o app.wasm *.swift` does, as jobs on a host:
+// the frontend once per source, the autolink entries read from the objects,
+// then wasm-ld.
+import { clangJob, frontendJobs, linkJob } from './driver.js';
+import { extractAutolink } from './wasm.js';
+export async function compile(host, request) {
+    const library = request.emitModule !== undefined;
+    const jobs = frontendJobs({
+        sdk: host.sdk,
+        moduleName: request.moduleName,
+        sources: request.sources,
+        action: request.output === undefined && !library ? 'typecheck' : 'compile',
+        objectDir: request.objectDir,
+        ...(host.plugins ? { plugins: host.plugins } : {}),
+        ...(host.moduleCachePath ? { moduleCachePath: host.moduleCachePath } : {}),
+        ...(request.wholeModule || library ? { wholeModule: true } : {}),
+        flags: [
+            ...(request.frontendFlags ?? []),
+            ...(library ? [...(request.parseAsLibrary === false ? [] : ['-parse-as-library']), '-emit-module-path', request.emitModule] : []),
+        ],
+    });
+    const objects = jobs.flatMap((job) => job.outputs);
+    let output = '';
+    const steps = [];
+    const run = async (job) => {
+        request.onStep?.(job);
+        const started = performance.now();
+        const result = await host.run(job.tool, job.args);
+        output += result.output;
+        steps.push({
+            tool: job.tool,
+            exitCode: result.exitCode,
+            seconds: (performance.now() - started) / 1000,
+            ...(result.memoryBytes !== undefined ? { memoryBytes: result.memoryBytes } : {}),
+        });
+        return result;
+    };
+    for (const job of jobs) {
+        if ((await run(job)).exitCode !== 0)
+            return { ok: false, output, steps, objects };
+    }
+    if (request.output === undefined || library)
+        return { ok: true, output, steps, objects };
+    // The libraries the objects ask for, the build's own objects (other
+    // modules' among them) as much as the module's.
+    const linked = [...objects, ...(request.objects ?? [])];
+    const autolink = extractAutolink(await Promise.all(linked.map((object) => host.readFile(object))));
+    const staticExecutableArgs = new TextDecoder().decode(await host.readFile(host.sdk.staticExecutableArgs));
+    const link = linkJob({
+        sdk: host.sdk,
+        objects: linked,
+        autolink,
+        staticExecutableArgs,
+        output: request.output,
+        ...(request.linkerFlags ? { flags: request.linkerFlags } : {}),
+    });
+    const result = await run(link);
+    return { ok: result.exitCode === 0, output, steps, objects };
+}
+/** A C target's sources compiled with clang, one job per source, as SwiftPM builds one. */
+export async function compileC(host, request) {
+    let output = '';
+    const steps = [];
+    const objects = [];
+    for (const source of request.sources) {
+        const relative = source.startsWith(`${request.root}/`) ? source.slice(request.root.length + 1) : source.split('/').pop();
+        const job = clangJob({
+            sdk: host.sdk,
+            source,
+            output: `${request.objectDir}/${relative.replace(/\//g, '_')}.o`,
+            ...(request.flags ? { flags: request.flags } : {}),
+            cxx: /\.(cc|cpp|cxx|c\+\+)$/.test(source),
+        });
+        request.onStep?.(job);
+        const started = performance.now();
+        const result = await host.run(job.tool, job.args);
+        output += result.output;
+        steps.push({
+            tool: job.tool,
+            exitCode: result.exitCode,
+            seconds: (performance.now() - started) / 1000,
+            ...(result.memoryBytes !== undefined ? { memoryBytes: result.memoryBytes } : {}),
+        });
+        if (result.exitCode !== 0)
+            return { ok: false, output, steps, objects };
+        objects.push(...job.outputs);
+    }
+    return { ok: true, output, steps, objects };
+}