wimcast 0.1.0__tar.gz

wimcast-0.1.0/CHANGELOG.md

@@ -0,0 +1,97 @@
+# Changelog
+
+All notable changes to this project are documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [0.1.0] - 2026-10-21
+
+Initial release.
+
+### Added
+
+- `wimcast build`: full build pipeline (extract, export, winpe, bootfiles,
+  publish, manifest, verify) over a `sources/<arch>/<name>/` tree.
+- `wimcast doctor`: runtime contract preflight (Python, wimlib-imagex,
+  pycdlib; conditional 7zz, rsync, qemu), never installs anything.
+- `wimcast probe`: ISO inspection without building (arch, editions,
+  indices, sizes, ISO backend used).
+- ISO reading via pycdlib with automatic `7zz` fallback backend; external
+  members are streamed and bounded by a timeout.
+- wimlib facade: `wiminfo --xml` (UTF-16LE), export, update, verify.
+- Content-addressed cache with sqlite index, flock-guarded and atomic.
+- Per-edition WIM export with LZX compression and `--boot` support.
+- boot.wim customisation: overlay, `\Drivers\`, `startnet.cmd` drvload
+  loop before `wpeinit`.
+- `[winpe] source = "file"` extension point for externally prepared
+  WinPE WIMs, with sha256 validation.
+- Boot files: pinned wimboot download (2.9.0) with SHA-256 verification,
+  bootmgr, boot.sdi, BCD, EFI.
+- Atomic publish into `finalized/` (temp file + `os.replace`) with stale
+  file pruning.
+- `images.ini` manifest (frozen v1 contract) and `build.json` (rich,
+  written after verification with real artifacts and stage durations).
+- INF parser (decorated sections, `%strings%`, CRLF, UTF-16).
+- Boot.wim driver injection via file placement + `drvload` loop (Level A).
+- Install.wim driver trees published under `sources/drivers/` and applied
+  at deploy time (Linux has no DISM).
+- Parallel scheduler: `[jobs] cpu` semaphore bounds concurrent wimlib
+  invocations across sources.
+- GitLab CI: `lint -> test (3.12, 3.13) -> build -> publish`; tagged
+  releases publish to PyPI via Trusted Publishing (OIDC, no token).
+- `scripts/smoke.py`: QEMU boot test that requires the `startnet.cmd`
+  serial banner.
+- Design docs: behaviour, ci-contract, inventory, output-contract,
+  compatibility, parity, benchmarks, runbook.
+- User docs: README and `docs/foreman-integration.md`. Example
+  configuration at `ci/examples/wimcast.toml`.
+
+### Changed
+
+- Configuration is strict: unknown sections/keys, invalid TOML, bad
+  digests and invalid values fail with an actionable `ConfigError`.
+- `[jobs] io` was removed; `[jobs] cpu` is the only parallelism knob.
+- Default `[winpe] overlay_dir` is unset (an overlay is optional).
+- Packaging uses a single version source (`wimcast.__version__`), PEP 639
+  license metadata and typed-package markers.
+
+### Fixed
+
+- Missing default WinPE overlay no longer makes a default build fail.
+- Edition filenames are collision-free and case-insensitively unique,
+  even when `INSTALLATIONTYPE` values collide.
+- WIM metadata names are sanitized before becoming paths or `images.ini`
+  entries (path traversal, control characters, reserved names).
+- wimboot downloads are checksum-verified and cached under the digest.
+- `7zz` fallback extracts/streams the exact member instead of a
+  hard-coded path.
+- Stale files in `finalized/` are pruned after a rebuild.
+- CLI catches known build errors instead of printing raw tracebacks and
+  always closes the cache.
+- Paths expand `~`; config-relative and temporary paths are no longer
+  silently cwd-dependent in manifests.
+- `wimlib-imagex update` sends commands over stdin (a single invocation
+  accepts one `--command`).
+- `ExternalReader` has list/stream timeouts and kills its process group.
+- Quoted wimlib update paths, so directories with spaces work.
+- INF decoding falls back to replacement; unreadable INFs are skipped
+  with a warning.
+- `ScheduleError`, `WinpeError` and `BootfilesError` now return their
+  message from `str(exc)`.
+- Duplicate `Artifact` models, dead scheduler IO semaphore and unused
+  publish helpers were removed.
+- CI removed unreachable runner tags that left pipelines pending.
+
+### Security
+
+- WIM metadata, edition names and driver reports are sanitized before
+  use in paths, manifests or terminal output.
+- Downloads and cached blobs are integrity-checked (SHA-256).
+- External tools run without a shell, with captured output and bounded
+  timeouts; timeouts kill the whole process group.
+
+[Unreleased]: https://gitlab.com/6admin.io/misc/wimcast/-/compare/v0.1.0...main
+[0.1.0]: https://gitlab.com/6admin.io/misc/wimcast/-/tags/v0.1.0

wimcast-0.1.0/CONTRIBUTING.md

@@ -0,0 +1,186 @@
+# Contributing to wimcast
+
+Thanks for contributing. This project is deliberately small and strictly
+governed: every rule below exists to keep it that way.
+
+## Architecture
+
+### Pipeline
+
+One source leaf (`sources/<arch>/<name>/*.iso`) produces one
+`finalized/<arch>/<name>/` tree:
+
+```
+probe -> hash -> extract -> info -> filter -> export(per edition) -> winpe
+      -> bootfiles -> publish -> manifest -> verify
+```
+
+Every expensive stage is keyed in the content-addressed cache
+(`[cache] directory`); a hit skips the work entirely and only republishes
+the files. The scheduler runs multiple sources concurrently, bounded by
+`[jobs] cpu`.
+
+### Module map
+
+| Module | Responsibility |
+|--------|----------------|
+| `config.py` | TOML to frozen dataclasses, explicit validation. Every site decision (paths, versions, job limits) lives here, never in code |
+| `doctor.py` | runtime contract preflight: the single source of truth for what the tool needs. Adding a dependency without a `doctor` entry is a bug |
+| `run.py` | subprocess wrapper: `Runner` protocol (injected in tests), `SubprocessRunner`, `ToolError` |
+| `model.py` | core dataclasses: `Source`, `WimImage`, `IsoMedia`, `Artifact` |
+| `iso.py` | ISO reading: pycdlib primary, `7zz` fallback |
+| `wim.py` | wimlib facade: info, export, update, verify |
+| `winpe.py` | boot.wim customisation (overlay, drivers, startnet.cmd) |
+| `inf.py` | INF parser (decorated sections, `%strings%`, CRLF, UTF-16) |
+| `drivers.py` | driver discovery, WinPE injection, publication |
+| `bootfiles.py` | wimboot, bootmgr, boot.sdi, BCD, EFI |
+| `publish.py` | atomic writes into `finalized/` |
+| `manifest.py` | `images.ini` (frozen v1 contract, parsed by Foreman) + `build.json` (audit, never parsed) |
+| `cache.py` | content-addressed store + sqlite index |
+| `sched.py` | parallel source scheduler bounded by `[jobs] cpu` |
+
+### Key contracts
+
+- `finalized/` layout and `images.ini` schema are frozen for v1: see
+  `docs/output-contract.md`. Changing them is a MAJOR bump.
+- `images.ini` and WIM naming are parity-sensitive: covered by golden
+  tests and by QEMU smoke tests (`scripts/smoke.py`).
+- `build.json` is additive only, never parsed by Foreman.
+
+## Ground rules
+
+- **KISS / YAGNI / DRY**. The simplest thing that works, no speculative
+  generality. An interface with one implementation and no foreseeable second
+  one is not accepted.
+- **Security by default**: validate inputs at the boundary, never hardcode
+  secrets, never log sensitive data, fail loud in development.
+- **No new dependencies** without a written reason and a `doctor` entry.
+  If it is not in `wimcast doctor`, adding it is a bug.
+
+## Development setup
+
+```bash
+git clone git@gitlab.com:6admin.io/misc/wimcast.git
+cd wimcast
+python3.12 -m venv .venv
+source .venv/bin/activate
+pip install -e ".[dev]"
+wimcast doctor
+```
+
+Note: `--system-site-packages` is only needed if the registry feature
+(`hivex`, not on PyPI) is ever re-enabled. It is not needed today.
+
+## Quality gates
+
+Every contribution must pass, locally or in CI:
+
+```bash
+ruff check .
+ruff format --check .
+mypy wimcast/
+pytest
+```
+
+- **Ruff** for lint and format (config in `pyproject.toml`, line length 100).
+- **mypy strict** for typing. `tests/` is excluded from strict checking.
+- **pytest** for tests. Unit tests must not require external tools: the
+  `Runner` is injected. Fixture tests may use `wimcapture` and `xorriso`
+  when the machine has them; media must never be committed to the repo
+  (licensing).
+
+## Commit convention
+
+[Conventional Commits](https://www.conventionalcommits.org/) 1.0.0:
+
+```
+<type>(<optional scope>): <description>
+
+[optional body]
+
+[optional footer]
+```
+
+Types: `feat`, `fix`, `docs`, `style`, `refactor`, `perf`, `test`, `chore`,
+`ci`, `revert`. Scopes (optional, encouraged): `config`, `doctor`, `iso`,
+`wim`, `winpe`, `drivers`, `inf`, `cache`, `publish`, `manifest`, `sched`,
+`cli`, `docs`, `ci`.
+
+Examples:
+
+```
+feat(wim): add LZX export with configurable thread count
+fix(iso): handle UDF paths with Unicode characters
+docs: explain wimboot version pinning
+chore(ci): pin wimboot in Containerfile
+```
+
+## Versioning
+
+[SemVer](https://semver.org/) 2.0.0:
+
+- **MAJOR**: breaking changes to the CLI, config schema, or output layout.
+- **MINOR**: backward-compatible functionality.
+- **PATCH**: backward-compatible fixes.
+
+The `layout_version` key in `wimcast.toml` tracks output compatibility
+separately from the package version. Changing the default output layout is a
+MAJOR bump even if the code diff is small.
+
+## Release process
+
+1. Update `CHANGELOG.md` (Keep a Changelog format), move `[Unreleased]`
+   entries into a dated release section.
+2. Bump `__version__` in `wimcast/__init__.py` (the only version source;
+   `pyproject.toml` reads it dynamically).
+3. **Wait for CI to pass on `main`** (lint + test-3.12/3.13 + build green).
+4. Build and validate the artifacts locally:
+
+```bash
+python -m pip install -e ".[release]"
+python -m build
+twine check --strict dist/*
+```
+
+5. Tag the exact commit, push the tag, and let the `publish` job upload via
+   PyPI Trusted Publishing:
+
+```bash
+git tag v<version>
+git push origin v<version>
+```
+
+6. Verify the release on PyPI, then install the published wheel in a clean
+   environment and run `wimcast --version` and `wimcast doctor`.
+
+Never tag a broken build. If CI fails, fix first, tag after. The `publish`
+job also refuses a tag that does not match `wimcast.__version__`.
+
+## Testing policy
+
+- New code ships with tests.
+- Behaviour changes update existing tests in the same commit.
+- Parity-sensitive code (manifest, publish, naming) is covered by golden
+  tests on `images.ini` and by QEMU smoke tests (`scripts/smoke.py`)
+  against reference trees.
+- Windows media never enters the repository or any public artifact store.
+
+## Branching model (simplified gitflow)
+
+- **`main`** is always deployable. No direct commits.
+- **`feat/*`** branches for features; **`fix/*`** for bug fixes.
+- **`hotfix/*`** for urgent production fixes (branch from `main`, merge back).
+- Open a merge request against `main`, reference the issue, keep the diff
+  focused.
+- The operator reviews every MR; no self-merge.
+- Tag `main` with `v<semver>` at release time (see Release process above).
+
+## Docs
+
+- Design decisions are recorded in `docs/`, not only in commit messages.
+- `docs/compatibility.md` and `docs/parity.md` are living documents: update
+  them when you touch ISO backends, drivers, or output behaviour.
+
+## Questions
+
+Open an issue: https://gitlab.com/6admin.io/misc/wimcast/-/issues

wimcast-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 6admin.io
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

wimcast-0.1.0/MANIFEST.in

@@ -0,0 +1,8 @@
+include LICENSE
+include README.md
+include CHANGELOG.md
+include CONTRIBUTING.md
+recursive-include docs *.md
+recursive-include ci/examples *
+recursive-include scripts *.py
+recursive-include tests *.py

wimcast-0.1.0/PKG-INFO

@@ -0,0 +1,229 @@
+Metadata-Version: 2.4
+Name: wimcast
+Version: 0.1.0
+Summary: Linux-native Windows image builder for Foreman. Drop-in replacement for WIMan/GenerateWIM.ps1.
+Author: 6admin.io
+License-Expression: MIT
+Project-URL: Homepage, https://gitlab.com/6admin.io/misc/wimcast
+Project-URL: Source, https://gitlab.com/6admin.io/misc/wimcast
+Project-URL: Issues, https://gitlab.com/6admin.io/misc/wimcast/-/issues
+Project-URL: Changelog, https://gitlab.com/6admin.io/misc/wimcast/-/blob/main/CHANGELOG.md
+Project-URL: Documentation, https://gitlab.com/6admin.io/misc/wimcast/-/blob/main/docs/runbook.md
+Keywords: foreman,wim,wimlib,winpe,pxe,provisioning
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: System Administrators
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: System :: Installation/Setup
+Classifier: Topic :: System :: Systems Administration
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pycdlib>=1.20.0
+Provides-Extra: dev
+Requires-Dist: pytest<9,>=8; extra == "dev"
+Requires-Dist: pytest-asyncio<2,>=0.23; extra == "dev"
+Requires-Dist: pytest-cov<8,>=5; extra == "dev"
+Requires-Dist: mypy<3,>=1.10; extra == "dev"
+Requires-Dist: ruff<1,>=0.5; extra == "dev"
+Provides-Extra: release
+Requires-Dist: build<2,>=1.2; extra == "release"
+Requires-Dist: twine<7,>=6.2; extra == "release"
+Dynamic: license-file
+
+# wimcast
+
+[![Python versions](https://img.shields.io/pypi/pyversions/wimcast)](https://pypi.org/project/wimcast/)
+[![License](https://img.shields.io/badge/license-MIT-blue.svg)](https://gitlab.com/6admin.io/misc/wimcast/-/blob/main/LICENSE)
+[![Code style](https://img.shields.io/badge/code%20style-ruff-000000)](https://docs.astral.sh/ruff/)
+[![Types](https://img.shields.io/badge/types-mypy--strict-blue)](https://mypy-lang.org/)
+[![SemVer](https://img.shields.io/badge/semver-2.0.0-brightgreen)](https://semver.org/)
+[![Conventional Commits](https://img.shields.io/badge/Conventional%20Commits-1.0.0-yellow.svg)](https://conventionalcommits.org)
+
+Build Windows images for [Foreman](https://theforeman.org) on Linux.
+
+wimcast turns Windows ISOs into a Foreman-ready HTTP tree of per-edition WIM
+files. It is a drop-in replacement for [WIMan](https://github.com/UXabre/WIMan)
+/ `GenerateWIM.ps1`, running **unprivileged on Linux**: no Windows host, no
+root, no `mount(2)`, no FUSE, no ADK.
+
+```
+sources/<arch>/<name>/*.iso   -->   finalized/<arch>/<name>/
+                                        sources/
+                                          boot.wim
+                                          ServerStandard.wim
+                                          ServerDatacenter.wim
+                                          images.ini
+                                          drivers/...
+                                        wimboot
+                                        bootmgr, bootmgr.efi
+                                        boot/boot.sdi, boot/bcd
+                                        build.json
+```
+
+## Why wimcast
+
+| Problem | wimcast solution |
+|---------|------------------|
+| WIMan needs Windows, the ADK and an admin shell | Any Linux machine, unprivileged |
+| Serial builds, one ISO at a time | Parallel scheduler with CPU semaphores |
+| Downloads `latest` wimboot on every build (non-reproducible) | Pinned, checksummed wimboot |
+| Full re-export of unchanged ISOs | Content-addressed cache: unchanged ISO skips re-export |
+| Driver tree never published (production bug in WIMan) | `sources/drivers/` published for deploy-time DISM |
+
+## Quick start
+
+```bash
+uv tool install wimcast   # or: pipx install wimcast
+
+wimcast doctor                 # check prerequisites (safe, installs nothing)
+wimcast probe sources/         # inspect your ISOs before building
+wimcast build sources/         # build the finalized/ tree
+ls finalized/                  # ready to serve over HTTP
+```
+
+That's it. Three commands from zero to a deployable tree.
+
+## Installation
+
+```bash
+uv tool install wimcast
+# or: pipx install wimcast
+# or: python3.12 -m pip install --user wimcast
+```
+
+Debian/Ubuntu:
+
+```bash
+sudo apt install wimtools
+uv tool install wimcast   # pycdlib is installed as a dependency
+```
+
+`wimcast doctor` tells you exactly what is missing, and whether it is
+required or only conditional.
+
+## Deploy to Foreman
+
+The full walkthrough (deployment repo, CI pipeline, web server, Foreman
+setup with Ansible) is in [docs/foreman-integration.md](https://gitlab.com/6admin.io/misc/wimcast/-/blob/main/docs/foreman-integration.md).
+
+The 30-second version:
+
+1. **Create a deployment repo** with your ISOs:
+
+   ```
+   deploy-repo/
+     sources/<arch>/<name>/*.iso     # your ISOs
+     .gitlab-ci.yml                  # copy ci/examples/deploy-pipeline.yml
+     wimcast.toml                    # optional, see ci/examples/wimcast.toml
+   ```
+
+2. **Configure 4 CI/CD variables**: `WEB_SERVER`, `WEB_SERVER_URL`,
+   `WEB_SERVER_USER`, `SSH_PRIVATE_KEY_B64`.
+3. **Push**: the pipeline runs `init -> build -> deploy -> verify` and your
+   `finalized/` tree lands on your web server.
+4. **Configure Foreman** (media, OS, provisioning template): see the
+   Ansible playbook in [docs/foreman-integration.md](https://gitlab.com/6admin.io/misc/wimcast/-/blob/main/docs/foreman-integration.md).
+
+Ready-to-copy files:
+
+| File | Purpose |
+|------|---------|
+| [ci/examples/deploy-pipeline.yml](https://gitlab.com/6admin.io/misc/wimcast/-/blob/main/ci/examples/deploy-pipeline.yml) | GitLab CI for the deployment repo |
+| [ci/examples/wimcast.toml](https://gitlab.com/6admin.io/misc/wimcast/-/blob/main/ci/examples/wimcast.toml) | Example configuration |
+
+## Commands
+
+| Command | What it does |
+|---------|--------------|
+| `wimcast doctor` | Check the runtime contract (required + conditional tools) |
+| `wimcast probe [path]` | Inspect ISO(s) without building: backend, WIM paths, per-index image metadata |
+| `wimcast build [path]` | Build all sources into `./finalized` |
+
+Global flags: `--version`, `--config <path>`, `-v/--verbose`.
+
+## Configuration
+
+`wimcast.toml` in the working directory, all values optional. A missing
+file yields the defaults; an invalid value is an error.
+
+```toml
+[winpe]
+source = "iso"          # "iso" (default): boot.wim index 1 from the ISO
+# source = "file"       # externally prepared WinPE (e.g. ADK-built)
+# path   = "assets/boot-winpe-amd64.wim"
+# sha256 = "..."
+iso_index = 1           # boot.wim index to export as WinPE base
+
+[editions]
+allowed = []            # substring match on image name; e.g. ["Standard", "Datacenter"]
+
+[output]
+filename_scheme = "installationtype"   # or "editionid" or "install"
+
+[wimboot]
+version = "2.9.0"       # pinned version, verified by checksum
+# sha256 = "..."        # required for an unpinned version; overrides the built-in digest
+
+[jobs]
+cpu = 0                 # concurrent wimlib-imagex calls (0 = auto-detect)
+
+[cache]
+directory = ".cache"    # content-addressed store; unset = no cache
+
+[tools]
+wimlib_imagex = "wimlib-imagex"   # override binary path if not in PATH
+sevenzip = "7z"                   # ISO fallback backend
+```
+
+wimboot is downloaded once from the pin's GitHub release and then cached.
+The built-in SHA-256 is checked before use; override `[wimboot] sha256` to
+pin a different release, including offline mirrors.
+
+## Requirements
+
+| Tool | Minimum | Package | Required |
+|------|---------|---------|----------|
+| Python | 3.12 | python3 | yes |
+| `wimlib-imagex` | 1.13 | `wimtools` | yes |
+| `pycdlib` | 1.20 | pip dependency | yes |
+| `7zz` | - | `p7zip-full` | no (ISO fallback backend) |
+| `rsync` | - | `rsync` | no (deployment pipeline) |
+| `qemu-system-x86_64` | - | `qemu-system` | no (smoke test) |
+
+## Drivers
+
+- **boot.wim (WinPE)**: INF trees from `winpe/drivers/` are added into the
+  WIM under `\Drivers\` and loaded with `drvload` in `startnet.cmd` before
+  `wpeinit`.
+- **install.wim (installed OS)**: the driver tree is published into
+  `finalized/<arch>/<name>/sources/drivers/` and applied at deploy time by
+  `dism /image:W:\ /add-driver /recurse` inside WinPE. Offline injection
+  into `install.wim` needs DISM, so it is not done on Linux.
+
+## Windows versions
+
+Supported target: Windows Server 2016/2019/2022/2025 and Windows 10/11
+(`amd64`). `wimcast probe` reports the backend actually used per ISO; the
+live test matrix is maintained in
+[docs/compatibility.md](https://gitlab.com/6admin.io/misc/wimcast/-/blob/main/docs/compatibility.md).
+
+## Non-goals (deliberate)
+
+- **Offline update integration** (`.msu`/`.cab`): `updates/` is warned about
+  and ignored. Patching happens after deployment (Ansible/WSUS/GPO).
+- **Offline driver injection into `install.wim`**: drivers are published as
+  a tree and injected by DISM inside WinPE at deploy time.
+- **ISO authoring**: the output is an HTTP tree, not bootable media.
+
+## Contributing
+
+See [CONTRIBUTING.md](https://gitlab.com/6admin.io/misc/wimcast/-/blob/main/CONTRIBUTING.md): architecture, module map, quality
+gates, conventions, release process.
+
+## License
+
+[MIT](https://gitlab.com/6admin.io/misc/wimcast/-/blob/main/LICENSE)