@kungfu-tech/buildchain 0.0.0-bootstrap.0 → 2.0.13-alpha.10

package/docs/lifecycle-protocol.md

@@ -0,0 +1,422 @@
+# Lifecycle Protocol
+
+Buildchain uses `buildchain.toml` as the v2 repository configuration format.
+The file is optional for simple JavaScript repositories, but it is the preferred
+way to describe release version state and lifecycle commands when a project is
+not a plain pnpm, npm, or yarn workspace.
+
+Only TOML is supported in v2. YAML, JSON, and JavaScript config files are not
+loaded.
+
+## Minimal File
+
+```toml
+schema = 1
+
+[version]
+required = true
+
+[[version.files]]
+type = "json"
+path = "package.json"
+key = "version"
+
+[lifecycle.verify]
+commands = [
+  "pnpm run check",
+]
+```
+
+`schema = 1` is required. Buildchain fails closed when the schema is missing or
+unknown.
+
+## Version State
+
+Version state is the source file evidence that matches a release tag. During
+promotion, Buildchain writes the selected release or prerelease version into the
+configured files, verifies the resulting tree, creates a source version commit,
+then moves exact and floating refs.
+
+Supported version file types:
+
+| Type | Use case | Required fields |
+| --- | --- | --- |
+| `json` | `package.json`, JSON manifests | `path`, `key` |
+| `toml` | `pyproject.toml`, other TOML manifests | `path`, `key` |
+| `regex` | `CMakeLists.txt`, `conanfile.py`, plain version files | `path`, `pattern`, `replacement` |
+
+`key` is a dotted key path:
+
+```toml
+[[version.files]]
+type = "toml"
+path = "pyproject.toml"
+key = "project.version"
+```
+
+Regex files must expose the current version through a named capture group called
+`version`:
+
+```toml
+[[version.files]]
+type = "regex"
+path = "CMakeLists.txt"
+pattern = 'project\([^)]* VERSION (?<version>[^ )]+)'
+replacement = '${version}'
+```
+
+If `version.required = true`, promotion fails when no configured version files
+are available.
+
+### Anchored Manual Versions
+
+Some repositories do not derive their package version from the Buildchain
+release tag. `libnode` is the canonical example: the package version is anchored
+to an explicitly selected upstream Node.js release such as `22.22.3-kf.0`, while
+the channel line may be `release/v22/v22.22`.
+
+Those repositories can opt into anchored manual semantics:
+
+```toml
+[version]
+required = true
+strategy = "anchored"
+next = "manual"
+manifest = "libnode.release.json"
+
+[[version.files]]
+type = "json"
+path = "package.json"
+key = "version"
+```
+
+With `strategy = "anchored"` and `next = "manual"`:
+
+- Buildchain validates the configured version files and anchor manifest, but it
+  does not rewrite those files to the Buildchain release tag.
+- `lifecycle.verify` is the project-owned truth gate. It should compare the
+  package version, anchor manifest, and upstream source/submodule state.
+- release promotion still creates the exact/floating production refs for the
+  current line;
+- release promotion does not auto-create the next alpha branch or tag;
+- the action output `next-anchor-required` is `true`, signaling that the next
+  upstream anchor line must be created explicitly by the repository.
+
+The configured anchor manifest must be JSON or TOML. Buildchain does not
+interpret project-specific field names; it only loads the manifest and exposes
+its top-level fields to validation summaries and lifecycle environment:
+
+```text
+BUILDCHAIN_VERSION_STRATEGY=anchored
+BUILDCHAIN_VERSION_NEXT=manual
+BUILDCHAIN_ANCHOR_MANIFEST=libnode.release.json
+BUILDCHAIN_ANCHOR_MANIFEST_JSON={"nodeTag":"v22.22.3",...}
+```
+
+The upstream anchor decision remains outside Buildchain. A future line such as
+`dev/v24/v24.xx` should be created by an explicit repository workflow or human
+decision after the upstream version has been selected and checked in.
+
+## Lifecycle Stages
+
+Lifecycle stages are declarative shell commands. A stage can use exactly one of:
+
+- `command`: one shell command;
+- `commands`: multiple shell commands run in order;
+- `script`: a multiline shell script.
+
+Any command failure fails the stage. `timeout_minutes`, `retries`, `shell`, and
+`env` can be attached to a stage.
+
+During version-state verification, Buildchain also sets `BUILDCHAIN_VERSION` to
+the release or prerelease version being verified.
+
+```toml
+[lifecycle.install]
+timeout_minutes = 10
+retries = 3
+commands = [
+  "pnpm install --frozen-lockfile",
+]
+
+[lifecycle.build]
+commands = [
+  "pnpm run build",
+  "pnpm run package",
+]
+
+[lifecycle.verify]
+shell = "bash"
+script = """
+set -euo pipefail
+pnpm run check
+git diff --check
+"""
+```
+
+Shared environment variables can be declared once:
+
+```toml
+[lifecycle.env]
+PYTHONUNBUFFERED = "1"
+```
+
+Stage-specific environment variables override shared lifecycle environment:
+
+```toml
+[lifecycle.test]
+command = "pytest"
+
+[lifecycle.test.env]
+PYTHONPATH = "src"
+```
+
+### Publish Stage
+
+`lifecycle.publish` is the project-owned side-effect stage. It may call npm,
+PyPI, Conan, CMake packaging scripts, Docker/OCI registries, S3 uploaders, or
+any other publisher. Buildchain does not assume the tool; it assumes the
+evidence contract.
+
+```toml
+[lifecycle.publish]
+script = """
+set -euo pipefail
+python scripts/publish_wheels.py
+node scripts/publish-images.mjs
+node scripts/write-publish-evidence.mjs
+"""
+```
+
+When `actions/promote-buildchain-ref` runs with `publish-transaction: "true"`,
+the publish stage receives:
+
+```text
+BUILDCHAIN_VERSION
+BUILDCHAIN_CHANNEL
+BUILDCHAIN_SOURCE_SHA
+BUILDCHAIN_TARGET_REF
+BUILDCHAIN_RELEASE_STATE
+BUILDCHAIN_EVIDENCE_DIR
+BUILDCHAIN_RELEASE_SHA
+BUILDCHAIN_RELEASE_MATERIAL_SHA
+BUILDCHAIN_PUBLISH_TOOLING_SHA
+BUILDCHAIN_PUBLISH_EVIDENCE
+```
+
+The stage must write publish evidence JSON. Buildchain validates that evidence
+before exact tags and floating refs move. In GitHub Actions, the promotion
+action also persists `state.json` and `evidence.json` to
+`refs/heads/buildchain/release-state/<version>` so fresh runners can recover
+without local workspace residue. See
+[`docs/publish-transaction.md`](publish-transaction.md) for the state machine,
+evidence schema, and recovery commands.
+
+## Promotion Semantics
+
+`actions/promote-buildchain-ref` consumes `version.files`, `lifecycle.verify`,
+and optionally `lifecycle.publish`.
+
+The verify stage runs after Buildchain has applied the generated version-state
+changes to the local checkout, and before it creates release commits or moves
+refs. After the command finishes, Buildchain checks that only declared
+version-state files changed. This prevents verification from quietly adding
+extra source changes to the release commit.
+
+On protected alpha and release branches, the generated version-state commit is
+merged through a normal pull request. This keeps review requirements,
+conversation resolution, strict status checks, and admin enforcement intact.
+After that PR lands, Buildchain verifies that the version-state PR changed only
+declared version files from the legal channel-promotion parent before it moves
+tags.
+
+The action input `verification-command` remains supported. When it is provided,
+it overrides `lifecycle.verify` for that invocation.
+
+## Migration Preflight
+
+Heavy repositories can validate their Buildchain declaration before they are
+ready to run the real build. `actions/validate-config` checks that
+`buildchain.toml` parses, configured version-state files exist, configured
+version keys are strings, and required lifecycle stage names are declared.
+For web-surface repositories it also validates `project`, `channels`, `deploy`,
+`retention`, and `security` declarations.
+
+It does not run lifecycle commands. This is useful for repositories such as
+`libnode`, where `lifecycle.build` represents an expensive multi-platform native
+build and the first migration milestone is to prove the release metadata and
+lifecycle protocol without consuming build runners.
+
+```yaml
+- uses: kungfu-systems/buildchain/actions/validate-config@v2
+  with:
+    require-version-state: "true"
+    require-lifecycle-stages: "install,build,verify"
+```
+
+Web-surface repositories can use the same action without requiring version
+state:
+
+```yaml
+- uses: kungfu-systems/buildchain/actions/validate-config@v2
+  with:
+    require-lifecycle-stages: "build,verify"
+```
+
+The action exposes project and deploy metadata through outputs such as
+`project-type`, `project-site`, `channels`, and `deploy-adapters-json`.
+
+## Web-Surface Projects
+
+`project.type = "web-surface"` is for sites, docs, browser apps, and operator
+consoles whose release object is a deployed surface, not a package version.
+
+```toml
+schema = 1
+
+[project]
+type = "web-surface"
+name = "site-kungfu-tech"
+site = "kungfu-tech"
+
+[channels.preview]
+url_pattern = "https://{alias}.preview.kungfu.tech"
+visibility = "ephemeral"
+noindex = true
+
+[channels.staging]
+url = "https://staging.kungfu.tech"
+visibility = "protected"
+requires_auth = true
+noindex = true
+promotable = true
+
+[channels.production]
+url = "https://kungfu.tech"
+visibility = "public"
+canonical = true
+noindex = false
+
+[deploy.production]
+adapter = "aws-s3-cloudfront"
+bucket = "kungfu-tech-production"
+artifact_path = "dist"
+secret_refs = ["AWS_ROLE_ARN"]
+```
+
+See [Web-surface deployments](web-surface-deployments.md) for the manifest,
+preview alias, retention, cleanup, and dry-run deploy contract.
+
+## Examples
+
+### Node Workspace
+
+```toml
+schema = 1
+
+[version]
+required = true
+
+[[version.files]]
+type = "json"
+path = "package.json"
+key = "version"
+
+[lifecycle.verify]
+commands = [
+  "pnpm run check",
+]
+```
+
+### Python Package
+
+```toml
+schema = 1
+
+[version]
+required = true
+
+[[version.files]]
+type = "toml"
+path = "pyproject.toml"
+key = "project.version"
+
+[lifecycle.install]
+command = "python -m pip install -e .[test]"
+
+[lifecycle.build]
+command = "python -m build"
+
+[lifecycle.verify]
+commands = [
+  "python -m build",
+  "pytest",
+]
+```
+
+### CMake and Conan
+
+```toml
+schema = 1
+
+[[version.files]]
+type = "regex"
+path = "CMakeLists.txt"
+pattern = 'project\([^)]* VERSION (?<version>[^ )]+)'
+replacement = '${version}'
+
+[lifecycle.configure]
+commands = [
+  "conan install . --build=missing",
+  "cmake -S . -B build -DCMAKE_BUILD_TYPE=Release",
+]
+
+[lifecycle.build]
+command = "cmake --build build --config Release"
+
+[lifecycle.verify]
+commands = [
+  "cmake --build build --config Release",
+  "ctest --test-dir build --output-on-failure",
+]
+```
+
+### Docker Image
+
+```toml
+schema = 1
+
+[[version.files]]
+type = "json"
+path = "package.json"
+key = "version"
+
+[lifecycle.build]
+command = "docker build -f Dockerfile -t kungfutrader/example:${BUILDCHAIN_VERSION} ."
+
+[lifecycle.verify]
+command = "docker build -f Dockerfile -t kungfutrader/example:verify ."
+```
+
+Docker publishing is an external side effect and should be gated by a release
+workflow after version-state promotion has been verified.
+
+## Design Boundaries
+
+The lifecycle protocol is also the command source for the reusable build
+surface. `.github/workflows/.build.yml` runs `lifecycle.install`,
+`lifecycle.build`, and `lifecycle.verify` by default, while allowing callers to
+override each stage with explicit workflow inputs. The underlying
+`actions/run-lifecycle` action can be used directly by repositories that need a
+custom workflow but still want Buildchain's lifecycle and deterministic manifest
+contract.
+
+Buildchain lifecycle commands are data, not executable configuration files.
+They make release behavior reviewable in pull requests and keep the release
+fact chain simple:
+
+1. choose the channel branch and release line;
+2. generate a source version commit from declared version files;
+3. verify that exact tree;
+4. move exact tags and floating refs only after verification succeeds;
+5. run publish or deployment side effects in separately gated workflows.

package/docs/publish-transaction.md

@@ -0,0 +1,285 @@
+# Publish Transaction
+
+Buildchain release promotion is not just tag movement. A release can also publish
+external artifacts: npm packages, Python wheels, OCI images, binary archives,
+metadata manifests, or site deployment records. Those side effects are harder
+than Git refs because most registries are append-only: a failed rerun must know
+which artifacts already exist, which are still missing, and whether any existing
+artifact conflicts with the release material.
+
+Buildchain v2 models that work as a release transaction.
+
+## Why This Exists
+
+The old ABV workflow made Git refs the visible release authority. That was
+enough when "release" meant "create a version commit, move tags, and let
+downstream jobs react." It is not enough when a single publish run must also
+upload packages and images.
+
+The failure mode to avoid is:
+
+1. publish an external artifact;
+2. fail before moving the exact release tag or floating channel refs;
+3. rerun from a new job id with no memory of the artifact;
+4. either republish something different or move refs without proving the
+   already-published artifact matches the release.
+
+The transaction gives reruns a stable identity and a machine-readable state so
+Buildchain can resume safely. The identity is:
+
+```text
+repository + version + source_sha + target_ref
+```
+
+It is not the GitHub Actions run id.
+
+## Durable State
+
+`actions/promote-buildchain-ref` stores release transaction state in a
+machine-managed Git branch:
+
+```text
+refs/heads/buildchain/release-state/<version>
+```
+
+The branch contains:
+
+```text
+state.json
+evidence.json   # present after publish evidence exists
+```
+
+The local `.buildchain/release-state/...` and
+`.buildchain/release-evidence/...` files are working copies. They are useful for
+local inspection and lifecycle commands, but they are not the durable truth for
+GitHub-hosted reruns. On action startup, Buildchain reads the durable state ref
+first, restores the local working copies, and only then decides whether to
+publish, repair, or finalize.
+
+Every meaningful state transition is written to the durable ref before public
+release refs move. If the durable write fails, the action fails closed.
+
+Durable release-state refs reserve their exact version even when the public exact
+tag was never created. If a later machine run sees a failed or repair-required
+state for `vX.Y.Z-alpha.N` and cannot resume it with the same transaction
+identity, alpha version selection must advance to the next prerelease instead
+of reusing or overwriting that failed transaction slot.
+
+## Lifecycle
+
+Repositories declare publish work in `buildchain.toml`:
+
+```toml
+[lifecycle.publish]
+commands = [
+  "python scripts/publish_wheels.py",
+  "node scripts/publish-images.mjs",
+  "node scripts/write-publish-evidence.mjs",
+]
+```
+
+`actions/promote-buildchain-ref` runs `lifecycle.publish` only when
+`publish-transaction: "true"` is set or when a `publish-command` input is
+provided. The action sets:
+
+```text
+BUILDCHAIN_VERSION
+BUILDCHAIN_CHANNEL
+BUILDCHAIN_SOURCE_SHA
+BUILDCHAIN_TARGET_REF
+BUILDCHAIN_RELEASE_STATE
+BUILDCHAIN_EVIDENCE_DIR
+BUILDCHAIN_RELEASE_SHA
+BUILDCHAIN_RELEASE_MATERIAL_SHA
+BUILDCHAIN_PUBLISH_TOOLING_SHA
+BUILDCHAIN_PUBLISH_EVIDENCE
+```
+
+Buildchain itself uses this contract for npm publishing:
+
+```toml
+[lifecycle.publish]
+command = "node scripts/npm-publish-transaction.mjs"
+```
+
+That script validates that `package.json` matches `BUILDCHAIN_VERSION`, runs
+`npm publish --access public --tag <alpha|latest>` through npm Trusted
+Publishing, and writes npm artifact evidence before the promotion action moves
+public refs.
+
+`BUILDCHAIN_RELEASE_MATERIAL_SHA` is the source material whose artifacts must
+match. `BUILDCHAIN_PUBLISH_TOOLING_SHA` identifies the publishing code. A repair
+run may change tooling, but material drift fails closed.
+
+## Evidence
+
+The publish lifecycle must write JSON evidence. Buildchain validates common
+fields and required artifact identities before final refs move.
+
+```json
+{
+  "schema": 1,
+  "version": "2.0.11",
+  "channel": "release",
+  "source_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
+  "release_sha": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
+  "target_ref": "release/v2/v2.0",
+  "release_material_sha": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
+  "publish_tooling_sha": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
+  "artifacts": [
+    {
+      "group": "node",
+      "kind": "npm",
+      "name": "@kungfu-systems/example",
+      "ref": "2.0.11",
+      "digest": "sha256:..."
+    },
+    {
+      "group": "image",
+      "kind": "oci",
+      "name": "ghcr.io/kungfu-systems/example",
+      "ref": "2.0.11",
+      "digest": "sha256:..."
+    }
+  ]
+}
+```
+
+The generic contract is intentionally small:
+
+- `version`, `channel`, `source_sha`, `release_sha`, and `target_ref` must match
+  the promotion run;
+- required artifacts must appear in evidence;
+- evidence used by a GitHub-hosted rerun must either be stored in the durable
+  state ref or be reconstructed by a machine-verifiable consumer command;
+- existing artifacts with the same identity and digest are accepted on rerun;
+- missing artifacts can be published by the next run;
+- an existing artifact with a different digest puts the transaction into
+  `repair_required`.
+
+Artifact identity is `group + kind + name + ref`. A required artifact that omits
+`group` matches any group with the same `kind + name + ref`.
+
+## Registry Truth Contract
+
+Buildchain owns transaction orchestration, finalization ordering, durable state,
+and generic evidence validation. It does not embed registry clients for npm,
+PyPI, GHCR/OCI, GitHub Releases, S3, Conan, CMake packaging, or project-specific
+download pages.
+
+Consumer `lifecycle.publish` commands own registry truth. A valid consumer stage
+must be idempotent and machine-verifiable:
+
+- inspect the target registry before publishing;
+- accept an existing exact artifact only when version, identity, digest, and
+  release-source binding match;
+- publish missing required artifacts;
+- reject conflicting existing artifacts and write evidence that lets Buildchain
+  move the transaction to `repair_required`;
+- write `BUILDCHAIN_PUBLISH_EVIDENCE` after every successful inspect/publish
+  cycle;
+- leave floating aliases such as npm dist-tags, PyPI stable markers, OCI
+  floating tags, GitHub Release "published" status, or download-page stable
+  links to a finalization step after Buildchain evidence validation.
+
+The first-class adapter surface is command-based. Projects may wrap npm, PyPI,
+GHCR/OCI, GitHub Release assets, archives, SBOMs, provenance, or checksums
+however they need, as long as they emit the common evidence contract.
+
+## States
+
+The state machine is:
+
+```text
+prepared -> publishing -> published -> finalizing -> complete
+                    |            |            |
+                    v            v            v
+              publish_failed  repair_required failed_permanently
+                    |
+                    v
+                abandoned
+```
+
+Supported states:
+
+| State | Meaning |
+| --- | --- |
+| `prepared` | Transaction identity was created, but publish has not started. |
+| `publishing` | Publish lifecycle is running or may have been interrupted. |
+| `publish_failed` | Publish command failed before valid evidence was produced. |
+| `published` | Evidence is valid; refs have not necessarily finalized. |
+| `finalizing` | Buildchain is moving exact/floating refs or needs a later run to do it. |
+| `complete` | Required evidence is valid and refs have finalized. |
+| `repair_required` | Existing evidence or artifact state conflicts with expected release material. |
+| `abandoned` | A human or controlled process abandoned this transaction, usually because a newer version supersedes it. |
+| `failed_permanently` | Recovery should not continue without explicit override. |
+
+`repair_required`, `abandoned`, and `failed_permanently` fail closed unless the
+operator passes an explicit override. That override is for controlled repair
+runs, not normal retry behavior.
+
+## Ref Ordering
+
+When publish transactions are enabled, promotion order is:
+
+1. verify target source and governance;
+2. create or reuse the version-state release commit;
+3. acquire or resume the release transaction;
+4. run `lifecycle.publish` or accept already-valid evidence;
+5. validate evidence and required artifacts;
+6. move exact release/prerelease tag;
+7. move floating tags and channel refs;
+8. mark the transaction `complete`.
+
+If protected branches require a generated version-state PR, the transaction can
+stop in `finalizing` and output `finalization-needed=true`. A later run can
+resume from the same transaction state and complete ref movement without
+republishing matching artifacts.
+
+If finalization fails after an exact Git tag is created, the next run reads the
+durable `finalizing` state, verifies the exact tag points at the recorded
+release SHA, and retries the remaining floating refs. An exact tag at a
+different SHA is a material conflict and blocks recovery.
+
+## CLI Recovery
+
+Local recovery commands operate on the same state/evidence files:
+
+```bash
+node scripts/release-transaction.mjs inspect --version v2.0.11
+node scripts/release-transaction.mjs recover --version v2.0.11
+node scripts/release-transaction.mjs finalize --version v2.0.11
+node scripts/release-transaction.mjs abort --version v2.0.11 --superseded-by v2.0.12
+```
+
+The CLI is a diagnostic and local repair surface. It reports the durable
+`state_ref`, but remote durable-ref writes and public Git ref finalization are
+owned by `actions/promote-buildchain-ref`, because that action runs inside the
+same governed GitHub permissions and branch-protection checks as release
+promotion. In other words, CLI `finalize` can mark the local transaction state
+complete after valid evidence; the machine-operated public finalization path is
+to rerun the promotion action.
+
+When no state file exists, creation commands also require:
+
+```bash
+--repository kungfu-systems/buildchain \
+--source-sha <sha> \
+--release-sha <sha> \
+--target-ref release/v2/v2.0 \
+--channel release
+```
+
+## Build-Images Follow-Up
+
+`build-images` should consume this contract rather than inventing a separate
+workflow rule. The expected integration shape is:
+
+- image build writes OCI digests into publish evidence;
+- required image families are passed through `publish-required-artifacts-json`;
+- reruns check GHCR or the target registry and accept existing images only when
+  tag and digest match;
+- preview or alpha image tags remain non-stable until the transaction evidence
+  validates;
+- production image aliases move only after all required image artifacts are
+  present and the Buildchain exact release tag has finalized.