@johnhenry/sitefile 0.0.0-stage → 0.0.0

package/CHANGELOG.md

@@ -0,0 +1,48 @@
+# Changelog
+
+## 0.0.0 — first npm release (2026-11-06)
+
+`@johnhenry/sitefile` is a new package: Sitefile was never published to npm
+under any name, and `0.0.0` is its first version. It joins the @johnhenry
+family as its static build orchestrator, next to `@johnhenry/fileable` (file
+trees in JSX) and `@johnhenry/packfile` (the Web Bundle reader and server).
+
+- **Published as `@johnhenry/sitefile`** (was `sitefile`, `private: true`,
+  `0.1.0` in `package.json` only). The tarball ships `lib/src/` (compiled by
+  `prepack`, no source maps), the README, `LICENSE` and this file; it has no
+  runtime dependencies. The Rust port stays at `0.1.0` in `rust/Cargo.toml`
+  and is not published (johnhenry/sitefile#69).
+- **MIT `LICENSE` file added.** `package.json` and `rust/Cargo.toml` already
+  said MIT. See johnhenry/sitefile#69.
+- **Node floor raised from 22 to 26** (`engines`, `.nvmrc`, CI), the family
+  floor. The suite passed unchanged on Node 26.
+- **CI on every push and pull request** (johnhenry/sitefile#67): typecheck,
+  tests and the examples smoke test (`npm run examples`), the Rust
+  fmt/build/test (clippy non-blocking until triaged), conformance between the
+  two implementations with the release binary, the VS Code extension and
+  tree-sitter grammar tests, actionlint, and the family's workflow lint.
+  `publish.yml` publishes from `main` through npm trusted publishing.
+- **Interop with `@johnhenry/packfile` is tested**
+  (`test/packfile-interop.test.ts`, packfile as a devDependency only): packfile
+  reads a Sitefile `.wbn` back into exactly the exported files and serves it,
+  and its `fromArchive()` reads the same bundle once gzipped. The other
+  direction, importing gzip-wrapped bundles from fileable or packfile with
+  `AS tree`, is johnhenry/sitefile#141.
+- **Fixed: a replaced archive destination went unnoticed on Linux.** Before
+  renaming the new archive into place, `exportFile` (and the Rust
+  `export_file`) checked that the existing destination was still the file
+  they inspected by device and inode number only. Linux often gives a file
+  created right after an unlink the same inode number, so a destination
+  swapped mid-export passed the check and was replaced. The check now also
+  compares size and nanosecond `ctime`. Found by the new CI's first Linux
+  run; the existing test only ever passed on macOS. Fixed in 0e0a1a1 (#142).
+- **Fixed: a case-only collision inside one directory had no source line.**
+  Two names that differ only in case in a directory read by `COPY` (possible
+  on a case-sensitive file system) failed with `E_PATH_COLLISION` but without
+  the instruction's line; both implementations now report it. Its test only
+  runs on case-sensitive file systems, so it had never run. Fixed in 0e0a1a1
+  (#142).
+- **Docs:** README badges, `## Contents`, `## Install`, `## Adding a new
+  plugin`, `## Security model` (the former "Safety defaults", with what stays
+  the user's responsibility), `## Family`; `examples/README.md`;
+  `AGENTS.md` reshaped into the family playbook with `CLAUDE.md` as a symlink.

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 John Henry
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,476 @@
-# Temporary Holding Version
+# Sitefile
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+[![npm version](https://img.shields.io/npm/v/%40johnhenry%2Fsitefile.svg)](https://www.npmjs.com/package/@johnhenry/sitefile)
+[![CI](https://github.com/johnhenry/sitefile/actions/workflows/ci.yml/badge.svg)](https://github.com/johnhenry/sitefile/actions/workflows/ci.yml)
+[![license](https://img.shields.io/npm/l/%40johnhenry%2Fsitefile.svg)](LICENSE)
+
+Full documentation: [opensource.johnhenry.me/sitefile](https://opensource.johnhenry.me/sitefile/)
+
+Docker-inspired static artifact builder. Compose immutable filesystem trees from
+local and remote sources, render templates, and export a deployable directory —
+no container runtime, no shell, no runtime dependencies.
+
+```
+# examples/hello/Sitefile
+ARG TITLE = "Hello"
+LET GREETING = "Welcome to ${TITLE}."
+COPY ./public/ /
+RENDER --escape=html ./templates/index.html TO /index.html
+EXPORT / AS directory
+```
+
+```bash
+sitefile build examples/hello --build-arg TITLE="My site"
+```
+
+This writes `examples/hello/dist/index.html` (with the title substituted) and
+`examples/hello/dist/style.css`.
+
+## Contents
+
+- [Install](#install)
+- [Status](#status)
+- [CLI](#cli)
+- [Variants: `sitefile bake`](#variants-sitefile-bake)
+- [Watch mode](#watch-mode)
+- [Profiling a build](#profiling-a-build)
+- [Images and servers](#images-and-servers)
+- [Export formats](#export-formats)
+- [Rust implementation](#rust-implementation)
+- [Editor support](#editor-support)
+- [Docs](#docs)
+- [Development](#development)
+- [Adding a new plugin](#adding-a-new-plugin)
+- [Security model](#security-model)
+- [Family](#family)
+- [License](#license)
+
+## Install
+
+```bash
+npm install -g @johnhenry/sitefile   # the `sitefile` command (TypeScript implementation)
+npx @johnhenry/sitefile build .      # or without installing
+```
+
+Requires Node.js 26 or later; the npm package has no runtime dependencies.
+The Rust binary (faster, and the one that provides `sitefile lsp` for editors)
+builds from this repository: `cargo install --path rust`. Prebuilt binaries are
+not published yet (johnhenry/sitefile#69).
+
+`@johnhenry/sitefile` is a new package: it was never published to npm under any
+name, and `0.0.0` is its first version.
+
+The library API is the same engine the CLI uses (`import { … } from
+"@johnhenry/sitefile"`); see [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md).
+
+## Status
+
+Milestones **M0** (local vertical slice), **M1** (remote inputs) and **M2**
+(trees and stages: multi-stage builds, `COPY --from`, archive imports,
+`.sitefileignore`; see `examples/multistage`) are implemented and tested; see [docs/TASKS.md](docs/TASKS.md). Requires Node.js 22+,
+no runtime dependencies.
+
+| Instruction | Supported |
+| --- | --- |
+| `FROM scratch\|<dir>\|<stage>\|image <ref> [AT <dir>] [AS name]` | yes; several FROMs make a multi-stage build, the last stage is exported; `image` starts from a site image's files (a theme: `examples/theme`, `examples/themed-site`), locked in `Sitefile.lock` when it comes from a registry |
+| `IMPORT name FROM <dir>\|git+https://…#ref:dir AS sitefile [ARG NAME=value]…` | yes; the stages of another Sitefile, local or from Git, as `name+<stage>` in `FROM` and `--from` (like Earthly targets; `examples/shared-stages`), with its own ARGs and paths, pinned in `Sitefile.lock` |
+| `ARG NAME[: type] [= value]` | yes, `string`/`number`/`boolean`, `--build-arg` |
+| `LET NAME[: type] = value` | yes |
+| `IMPORT name FROM <path or https URL> [AS file\|text\|json\|yaml\|csv\|markdown\|wbn\|tree] [INTEGRITY sha256:…] [SORT BY key [DESC]] [SCHEMA <file>] [DRAFTS true\|false]` | yes; `AS tree` unpacks tar, tar.gz, zip and Web Bundles; `AS markdown` reads posts with frontmatter (`SCHEMA` checks it against a YAML/JSON schema); `AS yaml`/`csv`/`wbn` give data for templates |
+| `COPY [--overwrite] [--from=<stage>] <source> <dest>` | yes; source is a local path, `name:/path`, `<<EOF` inline content, or with `--from` an absolute path in an earlier stage |
+| `RENDER [--escape=html] [--overwrite] [--from=<stage>] <template> TO <dest> [FOR item IN list [BY key] [PAGE n [FIRST <dest>]]]` | yes; template is a file or `<<EOF` inline body; `${NAME}`, `${json.path}`, `${#each}`, `${#if}`, `${x\|raw}`; `FOR` renders one page per element, `BY` one per tag, `PAGE` paginates |
+| `PLUGIN name FROM ./plugins/x.mjs\|x.wasm [INTEGRITY sha256:…]` | yes; a custom renderer: a sandboxed JS module (runs only with `--allow-plugin name`) or an import-free WebAssembly module (no flag needed; see [docs/PLUGINS.md](docs/PLUGINS.md), `examples/wasm/`) |
+| `TRANSFORM [--include/--exclude=<pattern>] [<source>] WITH name TO <dest> [OPTIONS k=v …] [FOR item IN list]` | yes; e.g. text → animated SVG (`examples/motion/`), responsive images (`examples/image/`); a directory source runs once per file (`TO /img/thumbs/*.png`) |
+| `TOOL name` + `TRANSFORM … WITH name TO <dest> [OPTIONS …] [FOR …] ARGS …` | yes; run an external program (e.g. `ffmpeg`) you allow with `--allow-exec name`: argv only, no shell, temp files, scrubbed environment (see [docs/PLUGINS.md](docs/PLUGINS.md)) |
+| `SITEMAP BASE <url> [TO <dest>]` | yes; a sitemaps.org `/sitemap.xml` of the `.html` pages so far |
+| `FEED <import> BASE <url> LINK <path> [TO <dest>] [TITLE <text>]` | yes; an Atom `/feed.xml` from Markdown posts (`title` and `date` required) |
+| `REDIRECT <from> <to> [STATUS 301\|302\|303\|307\|308]`, `HEADER <pattern> <Name>: <value>` | yes; `/_redirects` and `/_headers` for Netlify / Cloudflare Pages (and `sitefile serve`) |
+| `CHECKLINKS [--warn] [--external]` | yes; fails (or warns) on links to files that do not exist, with file and line; `--external` requests http(s) links under the network policy |
+| `FINGERPRINT [--overwrite] <pattern>… [MANIFEST <dest>]` | yes; renames assets to `name.<hash>.ext` and rewrites the HTML and CSS that reference them, plus a JSON manifest |
+| `EXPORT <path> [AS directory\|zip\|tar\|tgz\|wbn\|oci] [BASE <https-url>] [AT <root>]` | yes; archives are deterministic, `wbn` is an unsigned Web Bundle (needs `BASE` or `--base-url`), `oci` is a single-layer OCI image of the files (under `AT`, default `/srv/www`) |
+
+Templates can also live in the Sitefile itself (see `examples/inline/`):
+
+```
+RENDER --escape=html <<HTML TO /index.html
+<h1>${TITLE}</h1>
+HTML
+```
+
+A blog from Markdown posts with frontmatter (see `examples/blog/`):
+
+```
+ARG DRAFTS: boolean = false                          # --build-arg DRAFTS=true previews drafts
+IMPORT posts FROM ./posts AS markdown SORT BY date DESC SCHEMA ./post.schema.yaml DRAFTS ${DRAFTS}
+RENDER --escape=html ./templates/index.html TO /page/${page.number}/index.html FOR page IN posts PAGE 10 FIRST /index.html
+RENDER --escape=html ./templates/post.html TO /posts/${post.slug}/index.html FOR post IN posts
+RENDER --escape=html ./templates/tag.html TO /tags/${tag.key}/index.html FOR tag IN posts BY tags
+FEED posts BASE ${SITE_URL} LINK /posts/${item.slug}/index.html TITLE ${SITE_TITLE}
+SITEMAP BASE ${SITE_URL}
+FINGERPRINT /style.css MANIFEST /asset-manifest.json
+CHECKLINKS
+REDIRECT /feed /feed.xml
+HEADER /*.css Cache-Control: public, max-age=3600
+```
+
+Data files are readable too (see `examples/data/`):
+
+```
+IMPORT site FROM ./site.yaml AS yaml                 # ${site.nav.0.title}
+IMPORT team FROM ./team.csv AS csv SORT BY joined    # FOR person IN team
+IMPORT bundle FROM ./vendor/hello.wbn AS wbn         # ${#each bundle.entries as e}
+IMPORT hello FROM ./vendor/hello.wbn AS tree         # COPY hello:/ /hello/
+```
+
+Custom renderers are plugins that run in a sandboxed Node process (see `examples/motion/`):
+
+```
+PLUGIN motion FROM ./plugins/motion.mjs
+TRANSFORM ./title.txt WITH motion TO /title.svg OPTIONS style=typewriter
+TRANSFORM WITH motion TO /cards/${post.slug}.svg OPTIONS text=${post.title} FOR post IN posts
+```
+
+A JavaScript plugin is code from the repository you are building, so it runs only when you allow it:
+
+```
+sitefile build --allow-plugin motion     # or --allow-plugins for every PLUGIN
+```
+
+WebAssembly plugins (`PLUGIN … FROM ./x.wasm`) import nothing, can only compute, and need no flag (see `examples/wasm/`).
+
+Images: the first-party image plugin (`plugins/image/image.wasm`, WebAssembly) decodes PNG, JPEG, GIF and WebP, resizes with a fixed Lanczos-3 filter and writes PNG, JPEG or lossless WebP, with a metadata file listing each variant's path, width, height and a ready `srcset` (see [docs/PLUGINS.md](docs/PLUGINS.md#the-image-plugin), `examples/image/`):
+
+```
+PLUGIN image FROM ./plugins/image.wasm INTEGRITY sha256:…
+TRANSFORM ./photos/sky.jpg WITH image TO /img/ OPTIONS widths=400,800,1200 quality=80
+```
+
+External programs are explicit opt-ins: declare a `TOOL`, and allow it on the command line.
+
+```
+TOOL ffmpeg
+TRANSFORM WITH ffmpeg TO /intro.mp4 ARGS -f lavfi -i "color=c=black:s=1280x720:d=3" -fflags +bitexact -map_metadata -1 {output}
+```
+
+```
+sitefile build --allow-exec ffmpeg
+```
+
+TRANSFORM results are cached in `$SITEFILE_HOME/cache/transform` (`--no-cache` to skip; `sitefile cache info|prune`).
+CI runs and teammates can share them through an OCI registry: `--cache-to <registry/name:tag>` pushes the results a
+build used, `--cache-from <ref>` reuses them (see [docs/CLI.md](docs/CLI.md#registry-cache)).
+
+## CLI
+
+```
+sitefile build [context] [-o dist] [--format directory|zip|tar|tgz|wbn] [--base-url URL]
+               [--build-arg K=V | --build-arg K] [--build-arg-file vars.json|.yaml|.env] [--force]
+               [--frozen-lockfile | --update-lock] [--strict-pins] [--allow-http] [--allow-private-network]
+               [--allow-hardlinks] [--allow-exec name[=/abs/path]]... [--allow-plugin name]... [--allow-plugins]
+               [--no-cache] [--cache-from registry/name:tag]... [--cache-to registry/name:tag] [--watch] [--target stage] [--profile trace.json]
+sitefile validate [context] [--json] [--target stage]
+sitefile inspect  [context] [--json] [--target stage]
+sitefile graph    [context] [--json | --dot]   # dependency graph of imports, stages and operations
+sitefile why <path> [context] [--json]         # which instructions wrote an exported file, and what overwrote it
+sitefile vendor   [context] [--vendor-dir vendor/sitefile]   # download remote inputs for offline builds
+sitefile outdated [context] [--json]     # compare remote imports and BASE images with their pins (read-only)
+sitefile bake [targets...] [-f sitefile.bake.yaml] [--print [--json]] [-j n]   # build several variants
+
+# images and servers (see below and docs/CLI.md)
+sitefile build [context] -t name[:tag]        # also store the build as an image
+sitefile images | image inspect|diff|prune | tag | rmi | save
+sitefile cache info | cache prune [--older-than days]   # TRANSFORM result cache
+sitefile cache du                                       # cache sizes; SITEFILE_CACHE_MAX bounds them (default 1G)
+sitefile serve [-d] [--name n] [-p [host:]port] [--spa] <image|./dir>
+sitefile serve --watch [-p [host:]port] [--spa] [context]   # build in memory, serve, live reload
+sitefile servers [-a] | server stop|rm [-f]|restart|logs [-f] <name>
+```
+
+`--target <stage>` (like `docker build --target`) stops after a named stage and exports that stage's whole
+tree, in the EXPORT format but without the EXPORT path or `_redirects`/`_headers`; an unknown name is a usage
+error listing the stages. See [docs/CLI.md](docs/CLI.md#build-target---target-stage).
+`sitefile graph` and `sitefile why` are described in [docs/CLI.md](docs/CLI.md#explaining-a-build-graph-and-why).
+
+## Variants: `sitefile bake`
+
+Like `docker buildx bake`, `sitefile bake` builds several variants of a site (locales, staging and production)
+from one bake file, `sitefile.bake.yaml` (or `.yml`, `.json`), with the outputs side by side:
+
+```yaml
+groups:
+  default: [site]
+targets:
+  site:
+    matrix:                         # one target per combination: site-en-staging, site-en-production, ...
+      locale: [en, fr]
+      env: [staging, production]
+    arg-files: ["env/${env}.env"]   # ${name} is a matrix value
+    args: {LOCALE: "${locale}"}
+    output: dist/${env}/${locale}
+```
+
+```bash
+cd examples/bake && sitefile bake       # four builds, up to 4 at a time (-j), sharing the TRANSFORM cache
+sitefile bake --print                   # the resolved targets (--json for tools), without building
+```
+
+Targets also take `context`, `file`, `target` (a stage), `format`, `tags` and `inherits`. Every target runs even
+when one fails (then the exit code is 1); outputs that overlap or a tag used twice are refused before anything is
+built. See [docs/CLI.md](docs/CLI.md#variants-sitefile-bake).
+
+## Watch mode
+
+```bash
+sitefile build --watch examples/hello     # rebuild dist/ whenever the context, Sitefile or a --build-arg-file changes
+sitefile serve --watch examples/hello     # build in memory, serve on 127.0.0.1, reload open pages after each rebuild
+```
+
+Rebuilds are debounced (100 ms) and changes made during a build become one follow-up build. The output
+directory, `.git`, `Sitefile.lock` and the store are not watched. A failed rebuild prints the error and keeps the
+last good output (exports are swapped in atomically; `serve --watch` keeps serving the last good build). Live
+reload is a small script added to HTML responses only under `serve --watch`, listening on the reserved
+`/__sitefile/reload` event stream; it is never written to exports. Rebuilds are incremental: steps whose inputs
+are unchanged (by content digest) are reused from the previous build, in memory, and the result is identical to a
+clean build (`SITEFILE_NO_INCREMENTAL=1` turns it off). See [docs/CLI.md](docs/CLI.md#watch-mode).
+
+## Profiling a build
+
+```bash
+sitefile build examples/blog --profile trace.json   # open in ui.perfetto.dev or chrome://tracing
+```
+
+`--profile` writes a Chrome trace of the build: parsing, each import, plugin start, stage, operation, TRANSFORM call
+(with its cache hit or miss), the BASE pull and the export, so slow steps stand out. See
+[docs/CLI.md](docs/CLI.md#profiling---profile).
+
+## Images and servers
+
+Builds can be kept in a local, Docker-like **image store**, which is a standard
+[OCI image layout](https://github.com/opencontainers/image-spec/blob/main/image-layout.md)
+in `$SITEFILE_HOME` (default `~/.sitefile`). An image is the site's files as one
+layer; its ID is the manifest digest, and building identical output gives the same ID.
+
+```bash
+sitefile build examples/hello -t hello:1          # no -o: stored only, nothing written to the context
+sitefile images                                    # REPOSITORY TAG IMAGE ID CREATED FILES SIZE
+sitefile save hello:1 -o hello.oci.tar             # or .zip / .tar.gz / .wbn / a directory
+sitefile serve -d --name site -p 8080 hello:1      # serve it, in the background
+sitefile servers; sitefile server logs site; sitefile server stop site; sitefile server rm site
+```
+
+`serve` serves GET/HEAD only on `127.0.0.1` (never `0.0.0.0` unless you ask for it in `-p`), with ETags,
+correct MIME types and no directory listings; it serves `/404.html` for missing paths, `/about` from
+`/about.html`, and honours the site's `_redirects` and `_headers` (exact paths and `/*` splats). `EXPORT / AS oci` (or `build -o site.oci.tar`) writes the
+same OCI image without the store; on its own it contains files only, but `EXPORT / AS oci BASE nginx:alpine
+AT /usr/share/nginx/html` (or `--base`) builds a runnable image on top of a pulled base, and `sitefile push` / `pull`
+move images to and from any OCI registry. `--provenance` and `--sbom` attach a SLSA provenance statement and a
+CycloneDX file list to the image as OCI 1.1 referrers (deterministic, pushed with it; the image ID does not change). See
+[docs/CLI.md](docs/CLI.md).
+
+## Export formats
+
+```bash
+sitefile build examples/hello --format zip     # examples/hello/dist.zip
+sitefile build examples/hello -o hello.tar.gz      # format from the extension
+sitefile build examples/hello --format wbn --base-url https://example.com/
+```
+
+Archives are reproducible: sorted entries, fixed timestamps (or `SOURCE_DATE_EPOCH`),
+uid/gid 0, modes 0644/0755. See [docs/LANGUAGE.md](docs/LANGUAGE.md#export-formats).
+
+## Rust implementation
+
+`rust/` is a full port, maintained alongside the TypeScript version and checked
+against it by `node conformance/run.mjs` (identical output on every example and
+case). The Rust binary is the shipped product. TypeScript is the reference
+specification and the library API, and both get every feature (see
+[docs/RUST.md](docs/RUST.md#roles-decided-in-82)). It starts in ~4 ms instead of ~70 ms and is 1.3–2× faster on large
+sites, with a fraction of the memory; see [docs/RUST.md](docs/RUST.md) and
+[bench/README.md](bench/README.md).
+
+```bash
+(cd rust && cargo build --release) && rust/target/release/sitefile build examples/blog
+```
+
+## Editor support
+
+Highlighting works on its own; the language features (diagnostics,
+completion, hover, go to definition, references, rename, outline) come from
+the Rust binary's language server, `sitefile lsp`
+([docs/LSP.md](docs/LSP.md)), so put it on `PATH` with
+`cargo install --path rust` or point the editor at it.
+
+- **VS Code**: [editors/vscode/](editors/vscode/), a TextMate grammar plus a
+  language client that starts `sitefile lsp` (`sitefile.server.path`). Not
+  published; `(cd editors/vscode && npm install && npm run package)`, then
+  `code --install-extension editors/vscode/sitefile-0.2.0.vsix`.
+- **Zed**: [editors/zed/](editors/zed/), tree-sitter highlighting and the
+  `sitefile-lsp` language server; **zed: install dev extension** on
+  `editors/zed`.
+- **Neovim**: [editors/neovim/](editors/neovim/), tree-sitter queries and a
+  `vim.lsp.config` snippet for `sitefile lsp`.
+- **Helix**: [editors/helix/](editors/helix/), queries and a `languages.toml`
+  with the grammar and the `sitefile-lsp` server.
+
+The grammar behind Zed, Neovim and Helix is
+[editors/tree-sitter-sitefile/](editors/tree-sitter-sitefile/). Without an
+extension, `"files.associations": { "Sitefile": "dockerfile" }` is a usable
+stopgap in VS Code.
+
+## Docs
+
+- [docs/CLI.md](docs/CLI.md) — every command, the image store layout, reference grammar
+- [docs/LANGUAGE.md](docs/LANGUAGE.md) — syntax and the exact semantics chosen for v0.1
+- [docs/RUST.md](docs/RUST.md) — the Rust port: layout, parity checks, known differences
+- [docs/LSP.md](docs/LSP.md) — the language server for editors (`sitefile lsp`, Rust binary)
+- [docs/PLUGINS.md](docs/PLUGINS.md) — writing renderer plugins, and the sandbox
+- [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) — invariants and module layout
+- [docs/MILESTONES.md](docs/MILESTONES.md), [docs/TASKS.md](docs/TASKS.md)
+- [AGENTS.md](AGENTS.md) — the verification loop and rules for anyone (human or agent) changing this repo
+
+## Development
+
+```bash
+npm ci
+npm test                 # compile with tsc, then run node:test suites
+npm run typecheck
+(cd rust && cargo build --release && cargo test --release)
+npm run conformance      # both implementations on every example and case
+```
+
+From a clone, `node lib/src/cli.js` runs the TypeScript CLI after `npm run build`.
+See [AGENTS.md](AGENTS.md) for the full verification loop CI runs.
+
+## Adding a new plugin
+
+A plugin is a custom renderer that `TRANSFORM` calls. Write one ES module in its
+own folder that exports `transform(request)` and returns the bytes of the output
+file (or `{ files }` for several), declare it with `PLUGIN`, and allow it when you
+build:
+
+```js
+// plugins/upper/upper.mjs
+export const apiVersion = 1;
+export async function transform(request) {
+  return { bytes: (request.input?.text ?? '').toUpperCase() };
+}
+```
+
+```
+PLUGIN upper FROM ./plugins/upper/upper.mjs
+TRANSFORM ./title.txt WITH upper TO /title.txt
+```
+
+```bash
+sitefile build --allow-plugin upper
+```
+
+A WebAssembly plugin that imports nothing needs no flag. The request shape, the
+directory form (one call per file), the ABI for `.wasm` plugins, caching and the
+first-party image plugin are in [docs/PLUGINS.md](docs/PLUGINS.md); a complete
+example is `examples/motion/`. Run an external program instead with `TOOL` and
+`--allow-exec` (same document).
+
+## Security model
+
+A Sitefile can come from someone else (a Git URL, a theme image, a shared
+stage), so building one is treated as running untrusted input. Templates,
+imports and WebAssembly plugins are data; JavaScript plugins and external
+tools are code, and run only when you name them on the command line.
+
+**What Sitefile guarantees:**
+
+- **Local reads stay inside the build context.** No `..`, absolute paths or
+  symlinks out; hard-linked files are refused (`E_HARDLINK`) unless
+  `--allow-hardlinks`, since a hard link can alias a file outside the context.
+  A `.sitefileignore` (gitignore-style subset) at the context root excludes files
+  from directory copies; explicitly named files are never ignored.
+- **Remote imports are `https:` only, to public addresses only** — checked at
+  connect time and on every redirect (SSRF), size- and time-limited, never
+  decompressed or executed. `--allow-http` and `--allow-private-network` are the
+  only ways to widen this.
+- **Pins are enforced.** Unpinned URLs are recorded in `Sitefile.lock` and must
+  match on later builds; `--strict-pins` refuses any remote import or BASE image
+  that is not pinned (`INTEGRITY`, `@sha256:` or a lockfile entry).
+- **Git imports run no `git`.** `IMPORT … FROM git+https://host/repo.git#ref:path`
+  speaks protocol v2 over the same HTTPS client and policy, checks every object's
+  SHA-1, records the commit in `Sitefile.lock`, keeps symlinks inside the tree and
+  refuses submodules ([docs/LANGUAGE.md](docs/LANGUAGE.md#git-sources-githttps)).
+- **Secrets go only where they are sent.** `--secret id=token,env=API_TOKEN` with
+  `IMPORT … SECRET token AS BEARER` (or `AS HEADER X-Api-Key`) sends the secret
+  only to that import's own origin, and never into the output, caches,
+  `Sitefile.lock`, vendor files, provenance, traces or messages.
+- **Nothing executes unless you allow it by name.** A JavaScript `PLUGIN` runs
+  only with `--allow-plugin <name>`, in its own Node process under Node's
+  permission model (read access to its own folder only, no writes, network,
+  processes or environment); a `TOOL` runs only with `--allow-exec <name>`, argv
+  only, no shell, scrubbed environment. The Sitefile cannot allow itself. A
+  WebAssembly plugin must import nothing (checked before the build starts) and
+  runs with a time and memory limit, so it needs no flag
+  ([docs/PLUGINS.md](docs/PLUGINS.md#trust-model---allow-plugin)).
+- **No silent overwrites.** Writing different content to an existing path fails
+  unless `--overwrite` is given. Two output paths that differ only in case
+  (`/A.html`, `/a.html`) always fail (`E_PATH_COLLISION`), and every path is
+  Unicode NFC, so a build gives the same files and image IDs on macOS, Windows
+  and Linux.
+- **Exports are atomic.** A directory export is staged in a temporary sibling and
+  swapped in; a non-empty destination outside the context is only replaced with
+  `--force`. Archive exports (`zip`, `tar`, `tgz`, `wbn`) write one file via a
+  temporary sibling and rename; `--out` may not be the context, its parent, root
+  or home, nor a directory or symlink, and an existing file outside the context
+  needs `--force`.
+- **Offline builds are verified.** `sitefile vendor` stores every remote import
+  (and the BASE image's blobs) under `vendor/sitefile/` by content digest; builds
+  then read them from there, verified, with no network access
+  ([docs/LANGUAGE.md](docs/LANGUAGE.md#vendoring-sitefile-vendor)).
+- **`sitefile serve` listens on `127.0.0.1`**, GET/HEAD only, no directory
+  listings, unless `-p` names another address.
+
+**What is still yours:**
+
+- **What you allow is trusted code.** Node calls its permission model a "seat
+  belt" for trusted code, not a defence against deliberately malicious code.
+  `--allow-plugin`/`--allow-plugins` on a Sitefile you did not write runs its
+  author's JavaScript on your machine; `--allow-exec` runs a program on your
+  `PATH` with the arguments the Sitefile chose. Read them first.
+- **The content of what you build.** `RENDER --escape=html` escapes template
+  values; `${x|raw}` and templates without it do not. Markdown and data you
+  import from elsewhere end up in your pages as written.
+- **Unpinned remote content.** The lockfile catches a change; it does not tell
+  you whether the first version you locked was the one you meant. Use
+  `INTEGRITY`, `@sha256:` and `--strict-pins` for builds that must be
+  reproducible.
+- **Web Bundles are unsigned.** `EXPORT … AS wbn` writes an unsigned b2 bundle;
+  signing (for example for an Isolated Web App) happens outside Sitefile.
+- **Widening flags are yours to justify.** `--allow-http`,
+  `--allow-private-network`, `--allow-hardlinks`, `--force` and `serve -p
+  0.0.0.0:…` each turn off one of the guarantees above for that build.
+
+## Family
+
+Sitefile is the build orchestrator of the @johnhenry family's static tools. It
+runs the pipeline (pinned imports, stages, templates, sandboxed plugins, checks,
+exports); its siblings write file trees in code and serve the result. It has no
+runtime dependency on either: they meet at file formats, a directory or a `.wbn`
+Web Bundle.
+
+- **[`@johnhenry/fileable`](https://github.com/johnhenry/fileable)** -- writes
+  file trees in JSX (`Dir`, `File`, `Rm`), for scaffolding and codegen. A
+  fileable build's output directory, or its `encode="zip"` archive, is an
+  ordinary Sitefile input: `COPY ./generated/ /`, or `IMPORT gen FROM
+  ./site.zip AS tree` and `COPY gen:/ /` (both give the same output). Its
+  `encode="wbn"` archive is gzip around a Web Bundle, which `AS tree` does not
+  read yet (johnhenry/sitefile#141). Neither package depends on the other.
+- **[`@johnhenry/packfile`](https://github.com/johnhenry/packfile)** -- the
+  family's Web Bundle reader and in-memory server. `EXPORT / AS wbn BASE <url>`
+  writes a plain b2 Web Bundle that packfile's `fromWebBundle(buffer, { baseURL })`
+  reads back into exactly the exported files, and `createRouter()` serves as a
+  `(Request) => Response` handler. packfile's own archive format
+  (`fromArchive()`) is gzip around the same bundle under
+  `https://packfile.invalid/`: gzip a Sitefile bundle exported with that BASE and
+  `fromArchive()` reads it as is. `test/packfile-interop.test.ts` checks all
+  three; `@johnhenry/packfile` is a devDependency for that test only.
+
+## License
+
+[MIT](LICENSE)

package/lib/src/archive-writers.d.ts

@@ -0,0 +1,31 @@
+import { crc32 } from './deflate.js';
+import type { Tree } from './tree.js';
+/**
+ * Deterministic archive writers. The same tree always yields byte-identical
+ * output: entries are sorted by path, there are
+ * no directory entries, uid/gid are 0, there are no user/group names, modes are
+ * the tree's 0644/0755, and every timestamp is fixed. Compression uses the
+ * deterministic encoder in deflate.ts (shared with the Rust port), so zip,
+ * tar.gz and OCI layer bytes are the same in both implementations.
+ */
+export interface WriterOptions {
+    /**
+     * Seconds since the Unix epoch used for every entry timestamp (the
+     * `SOURCE_DATE_EPOCH` convention). Default: 1980-01-01 00:00 for zip (the DOS
+     * epoch) and 0 for tar. The gzip header mtime is always 0.
+     */
+    readonly epoch?: number | undefined;
+    /** tar only: directory entries (relative, no slashes at the ends, e.g. `srv`, `srv/www`) written before the files. */
+    readonly dirs?: readonly string[] | undefined;
+}
+/** Read `SOURCE_DATE_EPOCH`; undefined if unset, E_USAGE if malformed. */
+export declare function sourceDateEpoch(env?: NodeJS.ProcessEnv): number | undefined;
+export { crc32 };
+/** Zip (no zip64). Deflate when it shrinks the data, otherwise stored. */
+export declare function writeZip(tree: Tree, options?: WriterOptions): Uint8Array;
+/** POSIX ustar; long names use the prefix field, then a pax `path` extended header. */
+export declare function writeTar(tree: Tree, options?: WriterOptions): Uint8Array;
+/** gzip of the tar, header mtime 0 and no filename, so the stream is reproducible. */
+export declare function writeTgz(tree: Tree, options?: WriterOptions): Uint8Array;
+/** Reproducible gzip of an already-written tar (see deflate.ts for the exact format). */
+export declare function gzipTar(tar: Uint8Array): Uint8Array;