dotagents-cli 0.8.2__tar.gz → 0.9.0__tar.gz

{dotagents_cli-0.8.2 → dotagents_cli-0.9.0}/CHANGELOG.md

@@ -7,6 +7,182 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
 
 ## [Unreleased]
 
+## [0.9.0] - 2026-11-04
+
+dotagents implements the [Agent Overlay Specification](https://boriken-dev.github.io/agent-overlay-spec/)
+(AOS, edition `0.1.0-draft`) and passes its conformance vectors.
+
+### Changed (breaking)
+
+- An overlay's manifest is `overlay.cfg` (TOML by default; `#!json` or `#!yaml` on
+  its first line for JSON or YAML, the latter with the `yaml` extra). `overlay.toml`
+  is not read any more; an overlay that still has one gets a warning naming it.
+- A manifest that does not validate against AOS is ignored whole, with a warning
+  naming where it is wrong, and `overlays add` / `sync` refuse it. It used to keep
+  its valid fields and drop the others. The small built-in TOML reader is gone: below
+  Python 3.11 manifests need `tomli` (a declared dependency, vendored in the `.pyz`).
+- An overlay's Python modules go in `lib/python/`, prepended to `PYTHONPATH`; a
+  bare `lib/` is not on any path any more. `bin/` and `lib/python/` count only when
+  they hold a file.
+- A `bin/<name>.py` no longer gets a launcher by its name: a command is a
+  `[scripts]` entry (`curl = "bin/curl.py"`). `bin/` itself stays on `PATH`.
+- An overlay's setup is the first `[[setup]]` variant it declares for the OS, else
+  one `setup.<ext>` at its root -- the first, in the order `.py`, `.js`, `.mjs`,
+  `.cjs`, then `.ps1` on Windows, then `.bash`, `.sh` elsewhere, that this machine
+  can run -- taken to need the network. It is told its phase as its first argument
+  and `AOS_SETUP_PHASE`: `install` on the first `add`, `sync` on every `add` or
+  `sync` after one succeeded, and `remove` when `overlays remove` deletes the
+  overlay (new: setup used to run with no argument, and never on removal). A script
+  must treat a phase it does not know as success. It runs after the overlay's
+  dependencies are installed, in the overlay's own environment, with
+  `AOS_OVERLAY_ROOT`, `AOS_OVERLAY_NAME` and `AOS_OVERLAY_STATE` (where it writes,
+  kept across syncs, emptied before `install` and deleted on removal; sessions see
+  it as `<NAME>_OVERLAY_STATE`). `AGENTS_OVERLAY_DIR` is gone (it is
+  `AOS_OVERLAY_ROOT`). `setup_timeout` is now a variant's `timeout`.
+- An overlay with dependencies or a setup is activated only for the digest its last
+  preparation succeeded for on this machine (recorded in
+  `<store>/overlays/.dotagents-state/<name>/<os>.prepared.json`). One dotagents finds
+  at another digest -- an install that failed, a copy made without dotagents, or one
+  installed with `--no-setup` -- is refused until `overlays sync` prepares it; a
+  failed sync of the digest already prepared keeps it active. A failed `remove`
+  phase is reported and the removal goes ahead.
+- An overlay's `env.py` runs as an AOS env script: one `env.<ext>`, chosen in the
+  same order, with the overlay's own environment and no arguments, a 60-second
+  timeout, printing one JSON object (over one or several lines) or one object per
+  line. An overlay's root `env` file is read as a dotenv file -- `NAME=value`
+  lines with `${NAME}` substituted, no shell -- not sourced by bash. An overlay's `pre.env.py`,
+  `pre.env` and `local.env` mean nothing; the stores keep all of theirs. A store's
+  `pre.*` files run before every overlay, its `env.py` / `env` / `local.env` after.
+- An overlay's `rules/<name>.md` files are always-on rules files by default,
+  before the ones `rules` lists.
+- `<NAME>_OVERLAY_ROOT` always names the overlay copy in play; an inherited value
+  used to hold.
+- The overlays of every store are applied in one composition order, `(priority,
+  name)`, as AOS orders them: a project's overlay no longer comes before a user
+  store's overlay because it is the project's (a same-named one still shadows the
+  store's copy). Every store's own `bin/` and `lib/` lead all the overlays' dirs on
+  `PATH` and `PYTHONPATH`, and `dotagents path` and `AGENTS_PYTHONPATH` list them in
+  that order.
+- An overlay's root `env` is applied before its `env.py` (which used to run first),
+  and its `lib/python/` is on the session's `PYTHONPATH` for every Python, also when
+  the store venv layers it (it used to reach only `$AGENTS_PYTHON` then).
+- An overlay's `CONTEXT.md` comes after the stores' `AGENTS.md` in `context`, as AOS
+  orders the base context first.
+- Dependencies install by default from `lib/python/requirements.txt` and
+  `lib/node/package.json`, on `add` and every `sync`, before setup, into an
+  environment: `shared` (the default, one per runtime per store; dotagents' store
+  venv layers the shared Python one), a name, or `isolated`, chosen with
+  `environment` in a `[[runtime.python]]` / `[[runtime.node]]` variant, which may
+  also name another file (`requirements`, `lock` with `lock_digest`, `package`). A
+  root `requirements.txt` means nothing by its name, and overlays' packages are no
+  longer installed into the venv itself. A requirements file is a lock when every
+  requirement has a `--hash` (hashed in part: refused), installed with
+  `--require-hashes --no-deps --only-binary=:all:`; a `package.json` with a
+  `package-lock.json` beside it installs with `npm ci`, else `npm install`. Every
+  install is reported `deps-network`, and an unpinned file `unlocked-dependencies`.
+  Overlays of one environment install together and the environment is rebuilt from
+  the remaining members on every add, sync and removal; two members pinning one
+  package to two versions are both refused, the package named. `lib/node/` goes on
+  `NODE_PATH`, each environment's packages after the overlay's `lib/<runtime>/`.
+  `[defaults] deps = false` turns the default files off.
+- The installed copy holds exactly the overlay's files, so it keeps the overlay's tree
+  digest: the install record moved beside it, to
+  `<store>/overlays/.dotagents-install/<name>.json` (with the commit and the tree
+  digest), and caches and setup output inside an installed overlay are removed by
+  `add` / `sync`. `--prune` is gone, and so are the `.gitignore` / `.ignore` rules
+  that protected files inside an install (`dotagents._ignore`).
+- `overlays add` / `sync` refuse an overlay before writing anything when its tree has
+  a symbolic link or a name AOS refuses, its git commit holds a link or a submodule,
+  or this machine lacks what its `runtime`, setup or scripts need (an OS, a command,
+  an interpreter, a Python or Node version). A launcher is made only for python
+  scripts; an overlay declaring a script for another interpreter is refused.
+- MCP: a stdio server starts in its overlay's root; `env_passthrough` and
+  `headers_env` pass a variable only when the store grants it; a `command` with a
+  separator is a path inside the overlay. An overlay with an invalid `[mcp]` table
+  declares nothing, instead of stopping `dotagents mcp` from starting.
+- An overlay source refuses `::` in a git location, a host starting with `-`, a git
+  URL scheme other than https, http, ssh, git or file, and a `#path` that is not a
+  relative path.
+
+### Added
+
+- `env`, `env_files` (AOS or dotenv files), `paths` (prepend or append, per OS),
+  `env_scripts`, `context`, `skills`, `runtime`, `setup`, `harness` and `defaults` in
+  `overlay.cfg`. Each field's defaults come first and what it declares adds to them;
+  an entry naming a default's file or directory replaces that default entry, in its
+  place; `[defaults]` turns one off (`bin`, `lib`, `env`, `env_scripts`, `setup`,
+  `context`, `skills`, `rules = false`; any other key is refused). Env values can be
+  secret references, resolved only when granted.
+- Substitution in env values: a text value in `env`, an `aos` or `dotenv` env file
+  or an MCP server's `env` reads other variables as `${NAME}` and its operators
+  (`${NAME:-default}`, `${NAME:+alt}`, `${#NAME}`, slices, replacements, trims,
+  case), `$$` for a `$`. Each value is substituted once, when it is set, in the order
+  of application: a `dotenv` file line by line, one mapping's entries against the
+  environment from before it, with the overlay's own `AOS_OVERLAY_ROOT`,
+  `AOS_OVERLAY_NAME` and `AOS_OVERLAY_STATE` readable. An undefined name without a
+  default, or a placeholder reading a variable that holds a secret, refuses the
+  overlay's environment; a single-quoted `dotenv` value is kept as written.
+- Pins, grants and settings in a store's registry document (`dotagents.toml`,
+  `overlays` form): `[overlays.<name>] pin = "sha256:..."` refuses any other tree
+  digest, at install and at every activation; `grants` lets an overlay read a host
+  variable, a named secret (`[secrets.<name>]`) or a file under `secrets_dir`;
+  `env_scripts = false` stops an overlay's env scripts. A registry entry may be a
+  table with a `source`.
+- Weakenings (`unpinned`, `env-sensitive`, `env-passthrough`, `secret-read`,
+  `env-script`, `script-not-run`, `setup-network`, `network-not-enforced`,
+  `no-timeout`, `unlocked-dependencies`, `insecure-transport`) are reported each
+  time an overlay is activated, and by `overlays show`, which also prints the tree
+  digest, the pin, what this machine lacks and the declarations.
+- Setup and env scripts run with their interpreter found on `PATH` (never a `.cmd` or
+  `.bat`), arguments as a list, and in a process group or job object, so nothing they
+  start outlives them. Python run for an overlay keeps its bytecode under
+  `<user store>/.cache/pycache` (`PYTHONPYCACHEPREFIX`).
+- `[harness.<id>]`: `context` for that harness, and `skills = false` to keep the
+  overlay's skills out of that harness's own skills dir.
+- Git checkouts hold the blobs' bytes (no line-ending or filter conversion), so a
+  digest computed here agrees with any other AOS implementation.
+
+### Upgrading an existing store
+
+1. In each overlay you author, rename `overlay.toml` to `overlay.cfg`, and move its
+   Python modules from `lib/` to `lib/python/` (fix anything that names the old
+   path), and a root `requirements.txt` to `lib/python/requirements.txt` (or declare
+   it as `[[runtime.python]] requirements = "requirements.txt"`). Declare what is
+   not a default: a `bin/<name>.py` command
+   as `[scripts] <name> = "bin/<name>.py"`, `setup_timeout` as a `[[setup]]`
+   entry's `timeout`. Make a root `env` file plain `NAME=value` lines (or an
+   `env.py`); write a literal `${` in one as `$${`, or single-quote the value. Have
+   setup write only into `$AOS_OVERLAY_STATE`, and accept its phase (`install`,
+   `sync`, `remove`) as its first argument, exiting 0 for one it does not know.
+2. Install the new dotagents, then `dotagents overlays sync -g` (and in each project
+   with overlays, `dotagents overlays sync`): it installs the `overlay.cfg` overlays,
+   moves the install records beside them, builds their dependency environments and
+   runs each setup in its `install` phase (nothing is recorded as prepared yet;
+   until this sync, an overlay with dependencies or a setup is refused). To drop the
+   packages an older dotagents installed into the store venv itself, delete
+   `<store>/.pyvenv/` just before this sync, which makes it again; deleting it later
+   takes the dependency environments with it, so sync again after.
+3. Optionally pin what you trust: `dotagents overlays show <name>` prints its digest;
+   put it in the store's `dotagents.toml` as `[overlays.<name>] pin = "..."`.
+
+## [0.8.3] - 2026-10-30
+
+### Added
+
+- `dotagents mcp` starts the stdio MCP gateway and can forward tools from
+  overlay-declared stdio and Streamable HTTP servers. Forwarded names default
+  to `<skill-or-server-name>.<tool>`; `--prefix` accepts a `{name}` template.
+  `dotagents.mcp.refresh` reloads active overlay declarations and reconnects
+  servers without restarting dotagents.
+  Overlays declare servers and skill mappings in `overlay.toml` under `[mcp]`;
+  Streamable HTTP requires the `mcp-http` extra, and credentials are read from
+  named environment variables.
+
+### Changed
+
+- Requires duho 0.7.9 (its `ToolProvider` serves the forwarded tools), also in
+  the scope venvs and the `.pyz`.
+
 ## [0.8.2] - 2026-10-25
 
 ### Fixed
@@ -1567,7 +1743,9 @@ Patch release: the PATH/POSIX-conversion fix above (the only change since 0.3.0)
   affected module sources (`dotagents.cli`, `duho.presets`) to extracted temp files
   before dispatch; a no-op for a plain install.
 
-[Unreleased]: https://github.com/jose-pr/dotagents/compare/v0.8.2...HEAD
+[Unreleased]: https://github.com/jose-pr/dotagents/compare/v0.9.0...HEAD
+[0.9.0]: https://github.com/jose-pr/dotagents/compare/v0.8.3...v0.9.0
+[0.8.3]: https://github.com/jose-pr/dotagents/compare/v0.8.2...v0.8.3
 [0.8.2]: https://github.com/jose-pr/dotagents/compare/v0.8.1...v0.8.2
 [0.8.1]: https://github.com/jose-pr/dotagents/compare/v0.8.0...v0.8.1
 [0.8.0]: https://github.com/jose-pr/dotagents/compare/v0.7.6...v0.8.0

{dotagents_cli-0.8.2 → dotagents_cli-0.9.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: dotagents-cli
-Version: 0.8.2
+Version: 0.9.0
 Summary: The dotagents CLI: install and manage a portable ~/.agents config for AI coding agents
 Project-URL: Homepage, https://github.com/jose-pr/dotagents/
 Project-URL: Repository, https://github.com/jose-pr/dotagents
@@ -24,7 +24,7 @@ Classifier: Topic :: Software Development :: Libraries :: Python Modules
 Classifier: Typing :: Typed
 Requires-Python: >=3.9
 Requires-Dist: distlib<0.5,>=0.4.0
-Requires-Dist: duho<0.8,>=0.7.5
+Requires-Dist: duho<0.8,>=0.7.9
 Requires-Dist: pathapi<0.2,>=0.1.0
 Requires-Dist: tomli>=1.1.0; python_version < '3.11'
 Provides-Extra: dev
@@ -33,12 +33,15 @@ Requires-Dist: hatchling>=1.27; extra == 'dev'
 Requires-Dist: pathapi-uri<0.2,>=0.1.0; extra == 'dev'
 Requires-Dist: pytest; extra == 'dev'
 Requires-Dist: pyyaml; extra == 'dev'
+Requires-Dist: syshttp<0.2,>=0.1.0; extra == 'dev'
 Provides-Extra: docs
 Requires-Dist: mkdocs-material>=9.5; extra == 'docs'
 Requires-Dist: mkdocs>=1.5; extra == 'docs'
 Requires-Dist: mkdocstrings[python]>=0.24; extra == 'docs'
 Provides-Extra: http
 Requires-Dist: pathapi-uri<0.2,>=0.1.0; extra == 'http'
+Provides-Extra: mcp-http
+Requires-Dist: syshttp<0.2,>=0.1.0; extra == 'mcp-http'
 Provides-Extra: repo
 Requires-Dist: pathapi-uri<0.2,>=0.1.0; extra == 'repo'
 Requires-Dist: repopath[uri]<0.2,>=0.1.0; extra == 'repo'
@@ -111,8 +114,10 @@ tree: they are swappable payloads, not part of the tool. Name that branch as a r
 This repo's own working material (decisions, findings, plans) lives **privately** in
 its untracked `.agents/` — like every project, `.agents/` is never tracked or pushed.
 
-Each overlay's `<name>/overlay.toml` carries a `name`/`description`/`requires`/`routing`
-manifest read by the `dotagents overlays` subcommand, which manages overlays by name
+Overlays follow the [Agent Overlay Specification](https://boriken-dev.github.io/agent-overlay-spec/)
+(AOS): each may carry an `overlay.cfg` manifest (`name`/`description`/`requires`/`routing`,
+and what it needs that is not a default: environment, path additions, setup,
+runtime), read by the `dotagents overlays` subcommand, which manages overlays by name
 (`add`/`remove`/`list`/`sync`/`show`) — see [Managing overlays](#managing-overlays) below.
 
 ## Install
@@ -211,13 +216,21 @@ installed first, its `routing`/`rules` merge into `AGENTS.md`'s managed block, a
 store's shared `skills/` so every agent sees the same skills.
 
 - `add` and `sync` keep an installed overlay exactly as its source ships it: changed
-  files are replaced and files the source does not ship are removed, anything edited
-  there backed up under `<store>/install_backup/` first. What the `.gitignore` /
-  `.ignore` at the overlay's root match (setup output, local state) and tool caches are left
-  alone; `--prune` clears them too. A fresh `add` whose setup script fails is rolled
-  back.
+  files are replaced and everything else is removed (tool caches and setup output
+  included), anything edited there backed up under `<store>/install_backup/` first,
+  so the copy keeps the overlay's AOS tree digest. A source is refused before
+  anything is written when its manifest does not validate, this machine lacks what
+  its `runtime` needs, or the store pins it to another digest. A fresh `add` whose
+  preparation (dependencies, then setup) fails is rolled back.
+- An overlay's packages install by default: `lib/python/requirements.txt` (a lock when
+  every requirement has a `--hash`) and `lib/node/package.json` (locked by the
+  `package-lock.json` beside it), on `add` and every `sync`, before setup. They go
+  into the store's **shared** environment, a named one, or the overlay's own
+  (`[[runtime.python]] environment = "isolated"`); two overlays pinning one package
+  to two versions in one environment are both refused, naming the package.
+  `[defaults] deps = false` turns this off.
 - Each store has a Python venv, `<store>/.pyvenv/<version>-<os>-<arch>/`, where skills
-  run (`$AGENTS_PYTHON`): duho, every overlay's `requirements.txt` and every overlay
+  run (`$AGENTS_PYTHON`): duho, the store's shared Python environment and every agent
   `lib/` import there, layered project > user > system. pip comes from the
   interpreter, else its pinned wheel (cache, `AGENTS_PIP_SOURCE`, your index mirror,
   or a `.pyz` built `--with-pip`).
@@ -230,24 +243,29 @@ store's shared `skills/` so every agent sees the same skills.
 - `list` and `show` flag a `requires` no installed overlay provides; `show` names the
   repo an installed overlay came from.
 
-**Overlay setup scripts.** An overlay may ship an **idempotent** `setup.py` at its root
-(the recommended form: it runs under the same Python that runs dotagents, so it works on
-every OS). After `add`/`sync` copies the overlay
-in, dotagents runs the script automatically — so anything a human would otherwise
-hand-follow (PATH/lib wiring, self-registration) is one script the tool runs, not a doc.
-Presence of the script is the opt-in; skip it with `--no-setup`. The contract for
-authors:
-
-- **Idempotent** — safe to run on every `add`/`sync`; check-then-act, never blindly append.
-- **cwd** is the installed overlay dir (`<store>/overlays/<name>/`), so reference
-  your own files by relative path.
-- **Env** carries `AGENTS_HOME` (the user store — never hardcode `~/.agents`),
-  `AGENTS_SCOPE_ROOT` (the store the overlay is installed into: the user store with
-  `-g`, the project's `.agents` otherwise), `AGENTS_SCOPE` (`user` or `project`) and
-  `AGENTS_OVERLAY_DIR` (your own installed dir).
-- A **non-zero exit fails the install** with a clear error (not a silent skip). For any
-  outward or irreversible action the *script* must confirm first — the runner invokes a
-  script you chose to install; it does not second-guess it.
+**Overlay setup.** An overlay is activated only for the digest its last preparation
+(dependencies, then setup) succeeded for on this machine; `overlays sync` prepares it.
+An overlay's setup is the `[[setup]]` variant its `overlay.cfg`
+declares for this OS (interpreter, args, timeout, network), else one `setup.<ext>` at its
+root, the first this machine can run (`.py`, `.js`, `.mjs`, `.cjs`, then `.ps1` on
+Windows, `.bash`, `.sh` elsewhere), taken to need the network. dotagents runs it
+after `add`/`sync` copies the overlay in and before `remove` deletes it, telling it the
+**phase** as its first argument and `$AOS_SETUP_PHASE`: `install` the first time, `sync`
+on every add or sync after that, `remove` on removal. `--no-setup` skips it. The
+contract for authors:
+
+- **Idempotent**, and a phase it does not know is success with nothing to do (exit 0).
+- An `install` or `sync` that fails keeps the overlay from being activated until a
+  setup succeeds; a `remove` that fails is reported and the removal goes ahead.
+- **cwd** is the installed overlay root; **write only into `$AOS_OVERLAY_STATE`**, a
+  directory outside the root, kept across syncs and deleted on removal, that sessions
+  see as `$<NAME>_OVERLAY_STATE`.
+- **Env** is your environment, every overlay's root variable, the overlay's own
+  declarations, `AOS_OVERLAY_ROOT` / `AOS_OVERLAY_NAME` / `AOS_OVERLAY_STATE`, and
+  `AGENTS_HOME` (the user store — never hardcode `~/.agents`), `AGENTS_SCOPE_ROOT`,
+  `AGENTS_SCOPE`.
+- A **non-zero exit or the timeout fails the install** with a clear error, and
+  everything the step started is stopped with it.
 
 ### Downloadable `dotagents.pyz`
 

{dotagents_cli-0.8.2 → dotagents_cli-0.9.0}/README.md

@@ -55,8 +55,10 @@ tree: they are swappable payloads, not part of the tool. Name that branch as a r
 This repo's own working material (decisions, findings, plans) lives **privately** in
 its untracked `.agents/` — like every project, `.agents/` is never tracked or pushed.
 
-Each overlay's `<name>/overlay.toml` carries a `name`/`description`/`requires`/`routing`
-manifest read by the `dotagents overlays` subcommand, which manages overlays by name
+Overlays follow the [Agent Overlay Specification](https://boriken-dev.github.io/agent-overlay-spec/)
+(AOS): each may carry an `overlay.cfg` manifest (`name`/`description`/`requires`/`routing`,
+and what it needs that is not a default: environment, path additions, setup,
+runtime), read by the `dotagents overlays` subcommand, which manages overlays by name
 (`add`/`remove`/`list`/`sync`/`show`) — see [Managing overlays](#managing-overlays) below.
 
 ## Install
@@ -155,13 +157,21 @@ installed first, its `routing`/`rules` merge into `AGENTS.md`'s managed block, a
 store's shared `skills/` so every agent sees the same skills.
 
 - `add` and `sync` keep an installed overlay exactly as its source ships it: changed
-  files are replaced and files the source does not ship are removed, anything edited
-  there backed up under `<store>/install_backup/` first. What the `.gitignore` /
-  `.ignore` at the overlay's root match (setup output, local state) and tool caches are left
-  alone; `--prune` clears them too. A fresh `add` whose setup script fails is rolled
-  back.
+  files are replaced and everything else is removed (tool caches and setup output
+  included), anything edited there backed up under `<store>/install_backup/` first,
+  so the copy keeps the overlay's AOS tree digest. A source is refused before
+  anything is written when its manifest does not validate, this machine lacks what
+  its `runtime` needs, or the store pins it to another digest. A fresh `add` whose
+  preparation (dependencies, then setup) fails is rolled back.
+- An overlay's packages install by default: `lib/python/requirements.txt` (a lock when
+  every requirement has a `--hash`) and `lib/node/package.json` (locked by the
+  `package-lock.json` beside it), on `add` and every `sync`, before setup. They go
+  into the store's **shared** environment, a named one, or the overlay's own
+  (`[[runtime.python]] environment = "isolated"`); two overlays pinning one package
+  to two versions in one environment are both refused, naming the package.
+  `[defaults] deps = false` turns this off.
 - Each store has a Python venv, `<store>/.pyvenv/<version>-<os>-<arch>/`, where skills
-  run (`$AGENTS_PYTHON`): duho, every overlay's `requirements.txt` and every overlay
+  run (`$AGENTS_PYTHON`): duho, the store's shared Python environment and every agent
   `lib/` import there, layered project > user > system. pip comes from the
   interpreter, else its pinned wheel (cache, `AGENTS_PIP_SOURCE`, your index mirror,
   or a `.pyz` built `--with-pip`).
@@ -174,24 +184,29 @@ store's shared `skills/` so every agent sees the same skills.
 - `list` and `show` flag a `requires` no installed overlay provides; `show` names the
   repo an installed overlay came from.
 
-**Overlay setup scripts.** An overlay may ship an **idempotent** `setup.py` at its root
-(the recommended form: it runs under the same Python that runs dotagents, so it works on
-every OS). After `add`/`sync` copies the overlay
-in, dotagents runs the script automatically — so anything a human would otherwise
-hand-follow (PATH/lib wiring, self-registration) is one script the tool runs, not a doc.
-Presence of the script is the opt-in; skip it with `--no-setup`. The contract for
-authors:
-
-- **Idempotent** — safe to run on every `add`/`sync`; check-then-act, never blindly append.
-- **cwd** is the installed overlay dir (`<store>/overlays/<name>/`), so reference
-  your own files by relative path.
-- **Env** carries `AGENTS_HOME` (the user store — never hardcode `~/.agents`),
-  `AGENTS_SCOPE_ROOT` (the store the overlay is installed into: the user store with
-  `-g`, the project's `.agents` otherwise), `AGENTS_SCOPE` (`user` or `project`) and
-  `AGENTS_OVERLAY_DIR` (your own installed dir).
-- A **non-zero exit fails the install** with a clear error (not a silent skip). For any
-  outward or irreversible action the *script* must confirm first — the runner invokes a
-  script you chose to install; it does not second-guess it.
+**Overlay setup.** An overlay is activated only for the digest its last preparation
+(dependencies, then setup) succeeded for on this machine; `overlays sync` prepares it.
+An overlay's setup is the `[[setup]]` variant its `overlay.cfg`
+declares for this OS (interpreter, args, timeout, network), else one `setup.<ext>` at its
+root, the first this machine can run (`.py`, `.js`, `.mjs`, `.cjs`, then `.ps1` on
+Windows, `.bash`, `.sh` elsewhere), taken to need the network. dotagents runs it
+after `add`/`sync` copies the overlay in and before `remove` deletes it, telling it the
+**phase** as its first argument and `$AOS_SETUP_PHASE`: `install` the first time, `sync`
+on every add or sync after that, `remove` on removal. `--no-setup` skips it. The
+contract for authors:
+
+- **Idempotent**, and a phase it does not know is success with nothing to do (exit 0).
+- An `install` or `sync` that fails keeps the overlay from being activated until a
+  setup succeeds; a `remove` that fails is reported and the removal goes ahead.
+- **cwd** is the installed overlay root; **write only into `$AOS_OVERLAY_STATE`**, a
+  directory outside the root, kept across syncs and deleted on removal, that sessions
+  see as `$<NAME>_OVERLAY_STATE`.
+- **Env** is your environment, every overlay's root variable, the overlay's own
+  declarations, `AOS_OVERLAY_ROOT` / `AOS_OVERLAY_NAME` / `AOS_OVERLAY_STATE`, and
+  `AGENTS_HOME` (the user store — never hardcode `~/.agents`), `AGENTS_SCOPE_ROOT`,
+  `AGENTS_SCOPE`.
+- A **non-zero exit or the timeout fails the install** with a clear error, and
+  everything the step started is stopped with it.
 
 ### Downloadable `dotagents.pyz`
 

{dotagents_cli-0.8.2 → dotagents_cli-0.9.0}/docs/guide/authoring.md

@@ -1,7 +1,15 @@
 # Authoring an overlay
 
 An overlay is just a directory with the files you want installed into a scope, plus an
-optional `overlay.toml` manifest. Anyone can write one; name its parent directory (a
+optional `overlay.cfg` manifest: an
+[AOS](https://boriken-dev.github.io/agent-overlay-spec/) overlay, readable by any
+consumer of the format. The basics need no manifest: `bin/` goes on `PATH`,
+`lib/python/` on `PYTHONPATH`, `skills/<name>/SKILL.md` are skills, `CONTEXT.md` is
+context, `rules/<name>.md` are always-on rules, an `env` file (dotenv) and one
+`env.<ext>` set variables, and one `setup.<ext>` is the setup. Declare only what is
+not a default -- a declaration adds to the defaults, and one naming a default's file
+replaces that default -- and turn a default off in `[defaults]` (`bin`, `lib`, `env`,
+`env_scripts`, `setup`, `context`, `skills`, `rules = false`). Anyone can write one; name its parent directory (a
 directory of overlays), a registry, or a git repo with `--repo` — see the
 [overlays guide](overlays.md). Keep the source outside `<store>/overlays/`:
 `overlays remove` deletes the installed directory outright.
@@ -10,12 +18,12 @@ directory of overlays), a registry, or a git repo with `--repo` — see the
 
 ```
 my-overlay/
-  overlay.toml
+  overlay.cfg
   kb/MY_TOPIC.md
 ```
 
 ```toml
-# my-overlay/overlay.toml
+# my-overlay/overlay.cfg (TOML; `#!json` or `#!yaml` on the first line for those)
 name = "my-overlay"
 description = "My team's conventions for topic X."
 routing = [
@@ -64,37 +72,70 @@ yours needs it.
 
 ## Libraries and launchers
 
-Put importable Python in `lib/`. Every Python a session starts sees every overlay's
-`lib/` on `PYTHONPATH` (`dotagents env` puts them there, highest precedence first,
-and lists them in `$AGENTS_PYTHONPATH`), so a skill script or another overlay can
-`import` your module. So can your setup script and your command modules, which
-dotagents starts with the libs in place whatever shell it runs from. A `lib/` module
-comes before site-packages and the standard library: do not reuse a package name you
-do not mean to replace.
+Put importable Python in `lib/python/` (a bare `lib/` means nothing). Every Python a
+session starts sees it on `PYTHONPATH` (prepended, and listed in
+`$AGENTS_PYTHONPATH`), so a skill script or another overlay can `import` your module.
+So can your command modules, which dotagents starts with the libs in place whatever
+shell it runs from, and the setup of an overlay that `requires` yours. Another
+directory goes there with a path addition:
 
-## Commands on PATH: `bin/*.py` and `[scripts]`
+```toml
+[[paths]]
+variable = "PYTHONPATH"
+path = "src"
+position = "append"      # or "prepend", the default
+os = ["linux", "macos"]  # optional
+```
+
+An overlay's packages are `lib/python/requirements.txt` and `lib/node/package.json`,
+installed by default on `add` and every `sync`, before setup. A requirements file is a
+lock when every requirement carries a `--hash` (one hashed in part is refused); a
+`package.json` is locked by the `package-lock.json` beside it (`npm ci`; without it
+`npm install`, reported `unlocked-dependencies`). A variant can name another file or
+another environment:
+
+```toml
+[[runtime.python]]
+version = "3.9"
+requirements = "deps/requirements.txt"   # or lock = "locks/all.txt" + lock_digest = "sha256:..."
+environment = "isolated"                 # "shared" (default), a name, or "isolated"
 
-A Python command needs no launcher of your own. Every top-level `bin/<name>.py`
-gets one named `<name>`: `<name>.exe` on Windows (distlib's `ScriptMaker`, the
+[[runtime.node]]
+package = "web/package.json"
+```
+
+Overlays in one shared or named environment install together; if two pin one package
+to two versions, neither is activated. Choose `isolated` (or a name of your own) to
+keep yours apart. `lib/python/` is on `PYTHONPATH` and `lib/node/` on `NODE_PATH`,
+each before the environment's packages; `[defaults] deps = false` turns the default
+files off.
+
+## Commands on PATH: `[scripts]`
+
+A Python command needs no launcher of your own. Name it under `[scripts]` and it
+gets one named after the key; `bin/` itself is only on `PATH`, and no file there is a
+command by its name. Each launcher is `<name>.exe` on Windows (distlib's `ScriptMaker`, the
 launcher pip gives console scripts) and a `/bin/sh` script elsewhere, made in the
 agents' venv, in `<venv>/overlays/`, whenever `overlays add` / `sync` / `remove` or
 `init` brings the venv up to date. `dotagents env` puts that directory first on
-`PATH` for the venv `$AGENTS_PYTHON` names, so `bin/curl.py` answers to `curl`
-ahead of the system's, and drops any other venv's.
+`PATH` for the venv `$AGENTS_PYTHON` names, so with `curl = "bin/curl.py"` the
+overlay's `curl` answers ahead of the system's, and drops any other venv's.
 
-The launcher runs your script under the venv's interpreter, as `python
-bin/<name>.py` would, found when it runs through `$<NAME>_OVERLAY_ROOT`: editing
-the script or moving the store needs nothing regenerated, only a new interpreter
-does. Unset, or with the script missing, it exits 2 and says so. A file whose name
-starts with `_` is not a command. Name other scripts, rename one, or opt one out
-in `overlay.toml`:
+The launcher runs your script under the venv's interpreter, as `python <script>`
+would, found when it runs through `$<NAME>_OVERLAY_ROOT`: editing the script or
+moving the store needs nothing regenerated, only a new interpreter does. Unset, or
+with the script missing, it exits 2 and says so.
 
 ```toml
 [scripts]
-tool = "lib/tool/main.py"   # a launcher `tool` for another path
-helper = false              # none for bin/helper.py
+curl = "bin/curl.py"
+tool = "lib/tool/main.py"
+check = { path = "tools/check.py", interpreter = "python", os = ["windows"] }
 ```
 
+dotagents launches python scripts; an overlay naming another interpreter for a
+script is refused by `overlays add`.
+
 Two overlays with one script name is an error naming both. A launcher no overlay
 has any more is deleted at the next sync.
 
@@ -108,23 +149,61 @@ for what it starts, if anything needs the libs: its own `lib/`, then
 `$AGENTS_PYTHONPATH`, then the caller's, leaving empty parts out (an empty entry
 means the current directory). The net overlay's `bin/curl.py` does this.
 
-## Setup scripts
-
-Ship an idempotent `setup.py` at the overlay root to run install-time wiring
-automatically — the recommended, OS-agnostic form (it runs under the same Python as
-dotagents, so it works on every platform). See the contract in
-[Overlays → Setup scripts](overlays.md#setup-scripts). The essentials:
-
-- idempotent, check-then-act;
-- cwd is your installed overlay dir;
-- the environment is the scope's assembled `dotagents env` (every overlay's `lib/` on
-  `PYTHONPATH`, `bin/` on `PATH`), plus the user store (`AGENTS_HOME`), the store you are installed
-  into (`AGENTS_SCOPE_ROOT`, with `AGENTS_SCOPE` = `user` / `project`) and your own
-  installed dir (`AGENTS_OVERLAY_DIR`) — never hardcode a home path;
-- it has 300 seconds unless `setup_timeout` in `overlay.toml` (or the user's
-  `--setup-timeout`) says otherwise; `0` means no limit;
+## Environment
+
+An `env` file at the root (`NAME=value` lines, optionally `export `; `${NAME}` is
+substituted from the lines above and the environment, a single-quoted value is kept as
+written)
+and an `env.py` (or `.js`, `.mjs`, `.cjs`, `.ps1` on Windows, `.bash`, `.sh` elsewhere: the first this machine can run) set your
+overlay's variables by default; `[defaults] env = false` turns off the `env` file
+and `env_scripts = false` the script. The script prints one JSON object (across
+several lines if it likes) or one per line (`{"NAME": "value"}`, `null` to unset),
+and gets no arguments unless an `[[env_scripts]]` entry naming it gives some.
+Static values are better declared:
+
+```toml
+[env]
+TOOL_MODE = "read-only"
+TOOL_CONFIG = { path = "conf/tool.toml" }            # an absolute path in the overlay
+SYSHTTP_PREFIX = { value = "AGENTS_", if_unset = true }
+DOCS_TOKEN = { provider = "env", key = "DOCS_TOKEN" } # needs the store's grant
+```
+
+No overlay sets `AOS_*`, `*_OVERLAY_ROOT` or `*_OVERLAY_STATE`. A secret is never a
+value in the overlay: reference it, and the user grants it (see
+[Overlays → Pins, grants and env scripts](overlays.md#pins-grants-and-env-scripts-the-stores-own-settings)).
+
+## Setup
+
+A `setup.py` (or `.js`, `.mjs`, `.cjs`, `.ps1` on Windows, `.bash`, `.sh` elsewhere) at the root is your setup -- taken to need the network unless a `[[setup]]` entry naming it says `network = false` -- or declare
+variants per OS with `[[setup]]`. See the contract in
+[Overlays → Setup](overlays.md#setup). The essentials:
+
+- idempotent; it gets the phase as its first argument and `$AOS_SETUP_PHASE`:
+  `install` the first time, `sync` on every add or sync after that, `remove` before
+  the overlay is deleted; a phase it does not know is success (exit 0); your
+  dependencies are installed before it runs;
+- cwd is your installed overlay root; write only into `$AOS_OVERLAY_STATE`, kept
+  across syncs;
+- the environment is the user's, the overlays' root variables, your own declarations,
+  `AOS_OVERLAY_ROOT` / `_NAME` / `_STATE`, the user store (`AGENTS_HOME`) and the
+  store you are installed into (`AGENTS_SCOPE_ROOT`, `AGENTS_SCOPE`);
+- 300 seconds unless the variant's `timeout` (or the user's `--setup-timeout`) says
+  otherwise; `0` means no limit;
+- declare `network = true` when it needs the network: it is reported;
 - a non-zero exit fails the install loudly; confirm any irreversible action yourself.
 
+## Needs
+
+Say what must exist where your overlay goes; `overlays add` refuses a machine that
+lacks it, naming what is missing:
+
+```toml
+[runtime]
+os = ["linux", "macos", "windows"]
+commands = ["git", { name = "gh", os = ["linux", "macos"] }]
+```
+
 ## Custom commands
 
 A **command module** is a `*.py` file defining a `duho` command class with a
@@ -195,7 +274,8 @@ a good tool:
 ## Skills
 
 Put `skills/<skill-name>/` directories in your overlay to publish shared skills into
-the scope on install. See [Overlays → Skills](overlays.md#skills).
+the scope on install (`skills = [...]` lists them when not all are). See
+[Overlays → Skills](overlays.md#skills).
 
 A skill that relies on commands says so, in its own text: which `dotagents` commands
 it uses, that they are available as MCP tools (`dotagents.<command>`) when the