valise-sh 0.1.0 → 0.2.0

package/docs/module-abi.md

@@ -19,7 +19,8 @@ Required fields:
 | `shells` | Space-separated supported shell identifiers. |
 | `platforms` | Space-separated supported platform identifiers. |
 | `side_effect` | Startup/install/update side-effect class. |
-| `load` or `load_hook` | Exactly one load target: a relative module body path or a validated shell function hook. |
+| `load` | Repository-relative path to the module body. |
+| `requires` | Required when the module is a **meta module** (see below). |
 
 Optional fields:
 
@@ -31,6 +32,37 @@ Optional fields:
 | `updates` | Space-separated update identifiers registered by the consuming project. |
 | `version` | Optional module version used to reject version skew across roots. |
 
+## Meta modules
+
+A **meta module** declares dependencies and has no body. It is the natural
+way to write an aggregator: instead of an empty shell file whose only job is
+to name a dependency, the aggregator is metadata alone.
+
+A meta module has `requires=` with at least one entry and **no** `load=`.
+The loader and the compiler treat it as a dependency-only
+record: the module itself has nothing to source, but its required closure
+loads normally and in the right order. A meta module must leave `exports=`
+empty; it has no body to define an exported function. Metadata that declares
+an export on a meta module is rejected.
+
+```text
+name=profile_main
+version=1.0.0
+side_effect=read-only
+requires=contrib_path contrib_editor
+optional=
+shells=bash zsh ksh
+platforms=any
+exports=
+updates=
+```
+
+At least one of `load` or `requires` is required: a module with neither a
+body nor dependencies is malformed.
+
+`load` and `requires` are **not** mutually exclusive. A module that declares
+both loads its own body *and* its dependencies, in dependency-first order.
+
 ## Identifiers and allowed values
 
 Module names, dependency names, and exported shell functions must match
@@ -41,13 +73,22 @@ non-empty list containing only whitespace is malformed. Required `shells` and
 `platforms` lists must contain at least one parsed value. Empty identifiers,
 whitespace inside one identifier, path separators, quotes, shell metacharacters,
 glob characters, and control characters are invalid. Invalid identifiers are
-rejected before the loader can build function names or compatibility
-bridge hooks.
+rejected before the loader can build function names or source a body.
 
 Accepted `shells` values are `sh`, `bash`, `zsh`, `ksh`, and `fish`.
-`shells=sh` means the module body is POSIX-shell syntax and may be loaded by
-the POSIX core from `sh`, `bash`, `zsh`, or `ksh` adapters. `fish` selects the
-native Fish surface under `lib/fish/`; it never sources POSIX shell bodies.
+`shells=sh` means the module body is POSIX-shell syntax, and it is loaded by
+**every POSIX shell** — `sh`, `bash`, `zsh`, `ksh`, and dash — because those
+shells all run POSIX syntax. The engine classifies a running shell as `bash`,
+`zsh`, or `ksh` when it can prove it, and as `sh` otherwise (dash, BusyBox
+ash, and any other POSIX shell); `shells=sh` matches both the `sh` class and
+the specific classes, so a `shells=sh` module loads in all of them. Write
+portable bodies for `shells=sh` and you cover every POSIX shell at once.
+
+`fish` is the exception: it selects the native Fish surface under
+`lib/fish/`, and the Fish loader never sources POSIX shell bodies. A module
+whose `shells=` list does not contain `fish` is skipped under Fish rather
+than failed — a directly requested module fails closed in strict mode.
+
 The engine ships a generic Fish adapter at `entrypoints/fish/profile.fish` as
 the reference implementation of this handshake: it sets
 `_sp_fish_loader_bootstrap_mode` and `_sp_fish_loader_bootstrap_seeds`, then sources `loader.fish`. The loader
@@ -125,7 +166,7 @@ The supported composition is a primary consumer root plus one
 version-pinned layer root per provider.
 The provider tree is vendored under `layer/<name>/` of the release
 (preserving its internal `module.d/` + `modules/` layout), and a single
-manifest covers the whole closure, so `verify-active` checks consumer
+manifest covers the whole closure, so `release verify` checks consumer
 and provider bytes together before anything is sourced.
 
 - Pinning: record the provider source and revision beside the vendored
@@ -181,6 +222,7 @@ Supported `side_effect` values are:
 - `update-network` — performs explicit update/check network behavior.
 - `install-time` — belongs only in installer flows.
 - `local-secret` — reads local secret references at runtime only.
+- `local-state` — creates or caches state under the XDG state directory.
 - `work-only` — work/project-specific behavior that must be opt-in.
 
 `requires` dependencies must exist and load before the declaring module. Missing
@@ -211,11 +253,30 @@ and `tests/fish-runtime.test.sh` respectively:
   quarantines them and continues. Unlike the POSIX fallback, quarantined
   members are not loaded.
 
+### Writing a cross-platform profile
+
+`requires` means the module cannot work without the dependency, so a
+dependency that does not support the host blocks the module and fails strict
+mode. Use `optional` for platform-specific modules, so one profile works on
+every host:
+
+```text
+name=profile_main
+requires=common_tools
+optional=mac_tools linux_tools
+```
+
+Only the module for the running platform loads. The other is skipped without a
+message, because its absence is expected. The POSIX loader and compiled plans
+follow the same rules.
+
 `exports` lists public commands or shell functions the module declares. The loader
 uses this list for post-load export checks and failure cleanup; project catalogs
 and wrapper manifests use it for documentation and registration. An exported name
-must be defined by the load hook/body before the module is considered
-successfully loaded.
+must be defined by the module body before the module is considered
+successfully loaded. An export must name a function, so shell reserved words
+(`case`, `do`, `done`, `elif`, `else`, `esac`, `fi`, `for`, `if`, `in`,
+`then`, `until`, `while`, and `time`) are rejected.
 
 `updates` lists update registrations declared by the module. Each value is an ABI
 identifier that maps one-to-one to a row in the project update manifest. The
@@ -238,12 +299,19 @@ engine update mechanism.
 
 The ordinary source phase rejects `generated-file`, `update-network`, and
 `install-time` modules before sourcing their bodies. `read-only`, `path-env`,
-`local-secret`, and `work-only` are permitted by the engine; choosing whether
-to load a `work-only` module is the consuming profile's responsibility.
+`local-secret`, `local-state`, and `work-only` are permitted by the engine;
+choosing whether to load a `work-only` module is the consuming profile's
+responsibility.
+
+`local-state` is the permitted class for a module that creates a cache or
+directory under the XDG state directory at source time (`mkdir -p` under
+`$XDG_STATE_HOME`). It is the only write class the ordinary source phase
+allows. A module that writes anywhere else must declare `generated-file`,
+which the source phase rejects.
 
 The enforcement hook is
 `sp_metadata_validate_source_side_effect <side_effect>`. The loader calls it
-before ordinary source invokes a load path or compatibility load hook.
+before ordinary source invokes a load path.
 
 ## Capability layer available to modules
 
@@ -255,6 +323,17 @@ lookup, OS detection, canonical path containment, symlink-component rejection,
 Homebrew path discovery, secure local-bin permission checks, and background
 process launch.
 
+**The helpers stay available after the profile loads.** A compiled plan
+sources the capability prelude and leaves it in place, so a function defined
+by a module can call `sp_cap_os` or any other `sp_cap_*` helper long after
+the plan finished loading. The compiled plan and the loader agree on this;
+neither cleans the helpers up at the end of a load.
+
+A module that needs a portable operation should call the `sp_cap_*` helper
+rather than reaching for a platform-specific command directly. That keeps the
+module working on every platform the engine supports, and keeps the
+platform-specific code in one reviewed library.
+
 Mutation paths must fail closed when a capability cannot prove safety. Update
 destinations must combine canonical containment with symlink-component rejection
 before writing, cooldown logic must use the mtime helper instead of raw
@@ -276,13 +355,11 @@ the loader after metadata validation, dependency sorting, shell/platform matchin
 and phase side-effect checks have passed. The ABI has no unload hook: the loader
 does not implement module rollback or unload semantics.
 
-Bridge-backed records include `load_hook=_shell_module_<name>` instead of a
-`load` path. `load` and `load_hook` are mutually exclusive; at least one is
-required. A `load_hook` value must match the same identifier grammar as exported
-functions. The bridge hook is invoked only after the same identifier, dependency,
-shell/platform, and phase checks that protect file-backed modules.
+A meta module has no `load`; it has `requires` and no body. Nothing is
+sourced for it, but its dependency closure loads and its dependencies'
+export checks still apply.
 
-Module bodies and bridge hooks must treat `_spl_*`, `_sp_loader_*`, and
+Module bodies must treat `_spl_*`, `_sp_loader_*`, and
 `_SP_LOADER_*` names as loader-private reserved namespaces. Native Fish
 transactions additionally reserve `_sp_fish_loader_scope_before_*`,
 `__sp_fish_loader_scope_before_*`, `_sp_fish_loader_scope_after_*`, and
@@ -295,8 +372,14 @@ module and must never use module-controlled private state for recursive cleanup.
 Modules must also not define shell functions that shadow critical shell control
 utilities (`command`, `unset`, `readonly`, `return`, or `eval`) or mark those
 functions readonly. Those names are reserved because the loader must recover
-control after sourcing a module; direct definitions in file-backed modules and
-inspectable bridge hooks are rejected before the module body or hook is invoked.
+control after sourcing a module; direct definitions in module bodies are
+rejected before the body is invoked.
+
+After a body runs, the loader restores the caller's physical directory, logical
+`PWD` and `OLDPWD`, `IFS`, and the shell options it snapshotted. A body that
+makes any of those variables readonly is rejected as caller-state modification:
+the loader fails the module without aborting the caller. Cleanup resolves `rm`
+from a fixed absolute path, so a module that changes `PATH` cannot prevent it.
 
 Failure handling is selected once by the loader caller with
 `VALISE_MODE=strict|permissive`; it is not a per-module metadata promise.
@@ -309,21 +392,6 @@ Permissive mode reports the affected module and continues with unrelated modules
 paths, `..` path segments, spaces, quotes, shell metacharacters, and glob
 characters are invalid. Reading metadata must not source these paths.
 
-## Compatibility bridge
-
-The legacy flat-gist modules expose dependencies by executing `_shell_module_*`
-functions with `_SHMOD_GET_DEPS=1`. That behavior is compatibility evidence, not
-the target discovery model. The bridge serializes each legacy module into the
-same metadata shape with:
-
-- `name=<legacy-name>`
-- `requires=<legacy _SHMOD_DEPS>`
-- `load_hook=_shell_module_<legacy-name>`
-
-The bridge exists to preserve current dependency edges and permissive warning
-behavior while the loader migrates to metadata. It must not be used to justify
-executing arbitrary new module bodies during discovery.
-
 ## Validation implementation
 
 `lib/metadata.sh` provides the current metadata parser and validator:
@@ -332,9 +400,5 @@ executing arbitrary new module bodies during discovery.
 - `sp_metadata_validate_identifier <identifier>` validates ABI identifiers.
 - `sp_metadata_validate_source_side_effect <side_effect>` rejects phase-unsafe
   classes before ordinary startup can source a module body.
-- `sp_metadata_legacy_bridge <name> <requires> <side-effect> <exports>` emits a
-  normalized compatibility record for an existing `_shell_module_*` module. Bridge
-  records include empty `updates=` unless a later migration task explicitly maps
-  legacy update registration.
 
 The executable contract is `tests/loader/metadata-abi.test.sh`.

package/docs/release-policy.md

@@ -2,7 +2,7 @@
 
 ## Current status
 
-`valise-sh` is at **0.1.0**. Before 1.0, the CLI, module ABI, and release
+`valise-sh` is at **0.2.0**. Before 1.0, the CLI, module ABI, and release
 format do not have a backward-compatibility guarantee. Consumers should pin
 updates, review release notes, and run their profile's tests before upgrading.
 
@@ -19,26 +19,121 @@ The compatibility contract has three separate parts:
 
 A change affecting one of these is a compatibility change even if the other
 surfaces remain unchanged. Before 1.0, incompatible changes may occur between
-minor releases; release notes must identify affected consumers and migration
-steps. After 1.0, use semantic versioning: patch for compatible fixes,
+minor releases, and release notes must state what consumers have to change.
+After 1.0, use semantic versioning: patch for compatible fixes,
 minor for backward-compatible features, and major for incompatible changes.
 
+## What changed in 0.2.0
+
+`0.1.0` had no external users, so `0.2.0` provides no migration path for its
+CLI or shell integration. The items below describe the 0.2.0 contract; each is
+also recorded in [CHANGELOG.md](../CHANGELOG.md).
+
+### The registry is a generated artifact
+
+`shell/release/module-registry.tsv` is generated from the project's metadata
+by `valise registry`, and `valise build` regenerates it while `valise registry
+--check` validates it. A hand-written registry is no longer the source of truth, and a
+project needs no registry file at all to build: it keeps its `.meta` files,
+their bodies, the bundle roots, and the variants. A committed registry is
+accepted as input, but it is overwritten on the next build.
+
+The row order is fixed and does not depend on the caller's locale: the
+primary modules of `shell/module.d/` first, then the layer modules of each
+`shell/layers/<provider>/module.d/`, each group in C-locale name order.
+
+### The metadata ABI gained fields and classes
+
+- **Meta modules.** Metadata with `requires=` and no `load` is a
+  dependency-only aggregator. It loads successfully without a body. `load`
+  and `requires` may be combined.
+- **`side_effect=local-state`.** A source-safe class for modules that write
+  under the XDG state directory. It loads in the ordinary phase where
+  `generated-file` still fails.
+- **`shells=sh` covers every POSIX shell.** A module that declares `shells=sh`
+  loads under `sh`, `dash`, `bash`, `zsh`, and `ksh`, and is skipped under
+  Fish unless it is requested directly in strict mode.
+
+### Compiled plans follow the loader
+
+A compiled plan uses the loader's dependency rules. The required closure is
+the bundle's roots plus their transitive `requires`. A module whose required
+dependency is unavailable, failed, or skipped for the platform does not run.
+Strict mode stops at the first failure in that closure and returns nonzero;
+permissive mode reports each failure, continues with unrelated modules, and
+returns zero. An optional dependency loads first when it is available for the
+shell and variant; its absence never blocks the module that declared it. A
+cross-platform profile declares platform-specific modules with `optional=`.
+A meta module that declares exports is a compile error, and a module that
+fails has its declared exports removed from the function namespace.
+
+### Capabilities are available at runtime
+
+The `sp_cap_*` helpers remain sourced and callable after a compiled plan or a
+loader load finishes. A module may call `sp_cap_os` and the other helpers
+from its own functions. A plan that removed them at the end of the load left
+module functions unusable.
+
+### Startup integration is managed
+
+`valise install` writes a managed block into `~/.profile` and into each
+further startup file it manages: the existing `~/.bash_profile` or
+`~/.bash_login`, `~/.bashrc` and `~/.zshrc` when that shell is current or the
+file exists, and `~/.zprofile` when zsh is on `PATH` or the file exists. It
+does not create `~/.bash_profile` or `~/.bash_login`, because either would
+stop bash from reading `~/.profile`. A Fish `conf.d` hook is written when
+valise finds a Fish executable in a trusted location. `--no-integration`
+writes no startup file and no Fish hook. `valise apply` uses the same writer
+after building and installing.
+
+Each POSIX file receives the block between `# BEGIN valise managed block` and
+`# END valise managed block`. A repeat install replaces the block in place
+and preserves the rest of the file. An unpaired marker is refused and stops
+the install; a file already processed keeps its updated block. A symlinked
+startup file is reported and left untouched.
+
+### Commands
+
+The command surface has one command per job. `valise init` takes `posix` or
+`fish`; with no argument it prints the Fish hook when `$SHELL` names fish and
+the POSIX block otherwise. `valise new`, `valise module`, `valise build`,
+`valise apply`, `valise status`, and `valise registry` are new. `valise build`
+builds a release tree, and `valise apply` builds, installs, writes the shell
+integration, and verifies the result.
+
+Installed releases are managed by `valise release list|use|prune|verify`.
+`variant list`, `variant use`, and `verify-active` are gone; `variant` now
+refers only to a build variant. `valise doctor` gained `--pre-install` and
+`--no-integration`; it reads the startup files the running shell actually
+reads and exits nonzero when the active release is missing or fails
+verification, or when no startup file carries the block. `valise
+explain-bundles` and `valise layer-lint` no longer use Python.
+
 ## Release checklist
 
 Before tagging or publishing a release:
 
 1. Update `package.json` and `bin/valise` version together; add a dated entry
-   to [CHANGELOG.md](../CHANGELOG.md).
+   to [CHANGELOG.md](../CHANGELOG.md). The CLI's test suite asserts that the
+   two versions match, so a bump that updates one and forgets the other
+   fails the build.
 2. Run `make check-fast`, then run `make VALISE_REQUIRE_TOOLING=1 check`
    locally or use the GitHub Actions job. Keep the successful strict run as
    release evidence for every lane, including Fish, zsh, BusyBox, dash, and
    ksh.
 3. Run the README isolated quickstart from a clean checkout and verify that
-   install, `verify-active`, and example module loading work in the temporary
+   install, `release verify`, and example module loading work in the temporary
    home. Confirm cleanup removes the entire demo state.
 4. Inspect the result of `npm pack --dry-run` and install/test the packed
    tarball. The CLI's tests currently cover pack contents, installation, and
-   module loading.
+   module loading. The probe in `tests/cli-posix.test.sh` builds a tree from
+   the real `files` whitelist, plants build directories at more than one
+   depth under `examples/`, and requires the tarball to exclude every one of
+   them while keeping the surrounding example sources. `package.json` keeps
+   them out with `!examples/**/build/`, because npm applies `files` without
+   consulting `.gitignore` and `valise apply` writes build output into
+   `examples/profile-project/`. The `**` is load-bearing: a single `*` still
+   packs a build directory nested one level deeper.
 5. Review the changed CLI, module ABI, release manifest, docs, and examples;
    check `git diff --check` and doc links.
 6. Publish only through the chosen distribution channel. The repository
@@ -48,3 +143,12 @@ Before tagging or publishing a release:
 
 Do not claim a platform or shell has passed unless its runtime lane actually
 ran. An explicit skip is useful development evidence, not release evidence.
+
+## Test isolation
+
+Every suite runs in a quarantined environment: a fresh temporary `HOME`,
+controlled `TMPDIR` and `PATH`, and cleared startup hooks. A suite must also
+pass when it is run directly rather than through `make`, because the Makefile
+pins `TMPDIR=/tmp` and that can hide a defect that a developer's own
+environment exposes. macOS sets `TMPDIR` to a path ending in a slash, so a
+path comparison in a suite must resolve both sides before comparing them.

package/docs/shared-modules.md

@@ -1,8 +1,9 @@
 # Shared modules: governance and index
 
 This policy covers the shared modules shipped here and the requirements for
-provider repositories. It defines the layer namespace, review expectations,
-and how modules are indexed, deprecated, and removed.
+provider repositories. It defines the layer namespace, what belongs in a
+layer, review expectations, and how modules are indexed, deprecated, and
+removed.
 
 ## Namespace and ownership
 
@@ -24,6 +25,121 @@ review responsibility.
   discover what exists; they read this index to discover what the engine
   ships.
 
+### Use an owner segment
+
+The `contrib_` prefix is the only part the tools enforce. Nothing
+mechanically distinguishes one provider from another, so two providers that
+both ship `contrib_path_utils` fail closed against each other with a
+same-name conflict, and neither can be swapped for the other.
+
+**Use a provider segment after the prefix: `contrib_<owner>_<thing>`.**
+
+```
+contrib_acme_widget_setup
+contrib_acme_widget_apply
+contrib_globex_path_helpers
+```
+
+This is a recommendation, not a rule the tools check, but it is what keeps
+two providers from colliding. A layer should use one owner segment
+consistently across all of its modules, so a consumer can see at a glance
+which provider owns what.
+
+If you must ship a module whose name another provider might already use,
+prefix it with your owner segment anyway — a name collision between two
+vendored providers is otherwise unresolvable without renaming one of them.
+
+## What belongs in a layer
+
+A layer is a vendorable set of modules: something another person can take
+unchanged, use, and maintain on their own terms. The practical test is
+simple: **could another person vendor this module unchanged and still be
+happy with it?**
+
+A layer module must have:
+
+- **No employer material.** No account IDs, hostnames, cloud profiles,
+  project names, or anything else tied to one organization.
+- **No personal configuration.** No hard-coded home paths, no chosen
+  editor, no particular toolchain, model, or theme.
+- **No dependency outside the layer**, apart from the engine's own modules.
+  A layer module may require an engine module, but it may not require a
+  module that lives in the consumer's primary root — that dependency cannot
+  be satisfied when someone else vendors the layer.
+
+Everything else stays in the primary root: environment setup, completions,
+tool wrappers, work-specific modules, and the aggregators.
+
+`valise layer-lint <layer-name>` enforces the mechanical half of this. It
+checks that the layer has a `module.d/` directory, that every metadata file
+is a regular file with a safe mode, that every module name carries the
+`contrib_` prefix, that the metadata validates as data, that the version is
+present, that the load target exists as a regular file, and that every
+dependency resolves inside the layer or to an engine module. It needs no
+Python.
+
+### 1. Separate generic engines from opinionated presets
+
+A reusable mechanism and the opinionated defaults built on it are different
+modules with different audiences, and they belong in different files.
+
+The engine is the part anyone might want. The preset is a list of specific
+choices that only some users want. Ship them as two modules so a consumer
+can take the engine without the presets.
+
+```
+contrib_gitignore_engine   # the matching logic
+contrib_gitignore_preset   # one opinionated list of paths
+```
+
+A consumer who wants their own path list takes the engine and writes their
+own preset. A consumer who wants the shipped defaults takes both. If the
+engine and the preset were one module, every consumer would be forced to
+either accept the opinionated list or fork the whole thing.
+
+### 2. Avoid grab-bag modules
+
+A module's dependencies and `shells=` are the union of its parts. Splitting
+one module into several narrow ones lets each part declare only what it
+actually needs.
+
+The failure this avoids is concrete: a helper that used `local` was bundled
+into a general-purpose utilities module, and because `local` is not POSIX,
+that one function excluded ksh for **every** function in the module. The
+module's `shells=` list had to be the union, so one non-portable helper
+cost the whole module its ksh support.
+
+Split instead:
+
+```
+contrib_owner_cli_quote     # POSIX, no dependencies
+contrib_owner_cli_local     # needs a non-POSIX shell
+```
+
+Now the first module keeps `shells=sh bash zsh ksh` and the second declares
+what it actually needs. The same rule applies to dependencies: a module that
+pulls in a heavy dependency should be separate from one that does not.
+
+### 3. Use owner namespaces
+
+Covered above: `contrib_<owner>_<thing>`, one segment per provider, used
+consistently.
+
+### 4. Runtime capability use is allowed
+
+A layer module **may** call `sp_cap_*` helpers at run time. A compiled plan
+sources the capability prelude and leaves it in place, so a function the
+module defines can call `sp_cap_os`, `sp_cap_cpu_count`, or any other helper
+long after the profile finished loading. The loader behaves the same way.
+
+This is a change from earlier engine behavior, where the helpers were
+removed after loading and a module function that called one broke. If you
+have a layer that avoided the helpers to work around that, you can now use
+them directly.
+
+Prefer the `sp_cap_*` helpers over open-coding platform-specific commands.
+See [contrib-modules.md](contrib-modules.md) for the list.
+
 ## Index of engine-shipped shared modules
 
 | Module | Version | Exports | Source | Status |
@@ -42,6 +158,10 @@ The example provider tree in this repository
 | `contrib_shared_paths` | 1.0.0 | `contrib_shared_paths_setup` | `examples/shared/modules/shared/contrib_shared_paths.sh` | active |
 | `contrib_shared_umask` | 1.0.0 | `contrib_shared_umask_secure` | `examples/shared/modules/shared/contrib_shared_umask.sh` | active |
 
+These engine-shipped modules predate the owner-segment recommendation and
+keep their existing names; renaming them would break every consumer that
+already vendors them. New modules should use the owner-segment form.
+
 Update the index in the same change that adds, renames, deprecates, or
 removes a shared module. Provider repositories should maintain a corresponding
 index for their own layer modules.
@@ -53,6 +173,8 @@ Every change to a shared module passes the same floor:
 1. The engine gate: `make check-fast` during iteration and
    `make VALISE_REQUIRE_TOOLING=1 check` before a release claim (strict
    tooling is automatic when CI sets `CI=true` or `GITHUB_ACTIONS=true`).
+   These targets live in the source repository; the npm package ships the
+   runtime, the docs, and the examples, not the Makefile.
 2. The authoring checklist in
    [contrib-modules.md](contrib-modules.md): metadata validates, the
    module loads strict from a clean environment, behavior is idempotent