opsmith-cli 0.6.0__tar.gz → 0.6.1__tar.gz

{opsmith_cli-0.6.0 → opsmith_cli-0.6.1}/CHANGELOG.md

@@ -4,6 +4,18 @@ All notable changes to Opsmith are recorded here. The format follows
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project follows
 [semantic versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.6.1] - 2026-10-01
+
+### Fixed
+
+- `opsmith run` no longer runs a slow command more than once on AWS. The command was held open
+  as one SSM execution. Any command that took longer than a minute hit the connection's timeout,
+  and the connection then executed it again from the start, up to three more times, while the
+  earlier runs carried on. The command now starts as a background job on the machine, and
+  Opsmith checks on it every ten seconds until it finishes. Starting the job is never retried. A
+  command may run for up to 24 hours, after which it is stopped. GCP runs the same way, so a
+  dropped IAP tunnel no longer ends the run.
+
 ## [0.6.0] - 2026-09-21
 
 Coding-harness integration, phase 1 of the

{opsmith_cli-0.6.0 → opsmith_cli-0.6.1}/CLAUDE.md

@@ -254,7 +254,7 @@ anything structural.
   ownership, the testing standard.
 - `docs/specs/` — nine phase specs. Phase 0 (headless core) is split into seven parts, `0a`–`0g`,
   each its own merge unit; its `README.md` is the index and records which acceptance criterion each
-  part proves.
+  part proves. Phase 2 is split the same way, into eight parts, `2a`–`2h`.
 - `docs/reference/` — how the system works today. Written in the present tense, maintained.
 
 A spec is written before the work and stops changing once it ships; do not amend one to match what

{opsmith_cli-0.6.0 → opsmith_cli-0.6.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: opsmith-cli
-Version: 0.6.0
+Version: 0.6.1
 Summary: Opsmith is an AI devops engineer in your terminal
 License-Expression: GPL-3.0-only
 License-File: LICENSE.txt
@@ -9,10 +9,9 @@ Classifier: Environment :: Console
 Classifier: Intended Audience :: Developers
 Classifier: Programming Language :: Python
 Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
 Classifier: Topic :: Software Development
 Requires-Python: >=3.12
 Requires-Dist: ansible==14.3.1

{opsmith_cli-0.6.0 → opsmith_cli-0.6.1}/docs/notes/2026-09-04-migration-plan.md

@@ -10,7 +10,7 @@ The specs live in [`../specs/`](../specs/), one per phase:
 |------:|------|------------|------|----------|
 | 0 | [Headless core](../specs/2026-09-04-phase-0-headless-core/) | – | L | 0.5.0, shipped |
 | 1 | [Coding-harness integration](../specs/2026-09-04-phase-1-harness-integration.md) | 0 | M | 0.6.0 |
-| 2 | [Service model v2 and deterministic rendering](../specs/2026-09-04-phase-2-service-model-v2.md) | 0 | XL | 1.0.0 |
+| 2 | [Service model v2 and deterministic rendering](../specs/2026-09-04-phase-2-service-model-v2/) | 0 | XL | 1.0.0 |
 | 3 | [Customization layer](../specs/2026-09-04-phase-3-customization.md) | 0, 2 | M | 1.0.0, with phase 2 |
 | 4 | [Remote state and config sync](../specs/2026-09-04-phase-4-remote-state.md) | 0; 3 recommended first | M | 1.1.0 |
 | 5 | [Recipes](../specs/2026-09-04-phase-5-recipes.md) | 2, 3, 4 | L | 1.2.0 |
@@ -22,7 +22,7 @@ Phase 0 is specified as seven parts inside [its spec folder](../specs/2026-09-04
 
 **The phases were renumbered on 2026-09-20**, when harness integration moved from last to first. A phase number is its position here, so moving the work meant moving the number; the map from the old numbers is under [renumbering](#renumbering), and the shipped phase 0 specs still use the old ones.
 
-Sizes are relative effort, not dates. Phases 4 and 6 can run in parallel with their neighbours once their dependencies have landed. Phases 2 and 3 ship in one release: the first deterministic release overwrites hand-edited compose files, and phase 3 provides the override file where those edits belong.
+Sizes are relative effort, not dates. Phases 4 and 6 can run in parallel with their neighbours once their dependencies have landed. Phases 2 and 3 ship in one release: the first `release` or `update` after upgrading overwrites compose files edited on the machine, and phase 3 provides the override file where those edits belong.
 
 Harness integration goes first because almost all of it is already possible: the skill is instructions, its two generated references come from the Typer app and the pydantic schema that phase 0 already exposes, and the installer is file copying. What it cannot describe yet — the ownership manifest, the reference grammar, recipes, the template registry — is added by the phase that builds each one, under [definition of done](#definition-of-done-for-a-phase). The alternative was shipping the agent surface last, which would mean the tool spent its whole 0.x series being the thing agents could not drive.
 
@@ -53,7 +53,7 @@ without it.
 
 Every one of the planned capabilities is blocked by the same properties of the code today:
 
-1. **The model is used where code belongs.** `MonolithicDeploymentStrategy._generate_docker_compose` asks it to merge compose snippets and invent env values on every attempt, `_deploy_validate_docker_compose` asks it to judge success from container logs on every release, and `_select_virtual_machine_type` asks it to size a VM with no declared resource needs. Those steps are rendering and arithmetic, not judgment: they are slow, non-deterministic, untestable, and tied to one strategy. The model stays where judgment is needed, on analysis, generation, estimation and diagnosis.
+1. **The model is used where code belongs.** `MonolithicDeploymentStrategy._generate_docker_compose` asks it to merge compose snippets and invent env values on every attempt, and `_select_virtual_machine_type` asks it to size a VM with no declared resource needs. Those steps are rendering and arithmetic, not judgment: they are slow, non-deterministic, untestable, and tied to one strategy. The model stays where judgment is needed, on analysis, generation, estimation, diagnosis and deciding whether a deployment came up, though `_deploy_validate_docker_compose` has it decide that from container logs alone, with no container states to hold it to.
 2. **Every flow is interactive.** `inquirer` prompts live in `main.py`, `service_detector.py`, `deployment_strategies/monolithic.py` and inside `cloud_providers/aws.py` and `gcp.py`. An agent driving opsmith through a shell cannot answer them, and nothing can be tested end to end.
 3. **State is local-only.** Terraform state is gitignored by `GitRepo.ensure_gitignore`, so git never protected it; the only copy is on the user's disk.
 4. **Analysis is a pre-computed map.** `repo_map.py` is an aider port with the ranking removed, capped at a small token budget, and regenerated on every Dockerfile attempt.
@@ -84,7 +84,7 @@ Principles that hold across phases:
 
 - **Core is UI-free.** Nothing outside `opsmith/cli/` imports `inquirer`, `typer`, or `rich` prompt helpers. A test enforces it from phase 0 on.
 - **Every interaction has a key.** Strategies ask, confirm, wait and notify through one interaction API with stable keys, so answers can come from a terminal, flags, a file, env vars or a driving agent, and every command is safe to run again after it stops for an answer or an external action.
-- **The model does the judgment, code does the rendering.** A configured model is a requirement of the tool, not an option. It analyses repositories, generates Dockerfiles, estimates what the config does not declare, and explains failures. Rendering artifacts and checking that a deployment came up are deterministic code, so results are reproducible and testable. A coding harness may also author config and Dockerfiles itself and have opsmith validate them.
+- **The model does the judgment, code does the rendering.** A configured model is a requirement of the tool, not an option. It analyses repositories, generates Dockerfiles, estimates what the config does not declare, judges whether a deployment came up, and repairs the compose file when that is what failed. Rendering artifacts is deterministic code, so results are reproducible and testable. The judgment works over facts that code gathers (container states, health, restarts), and it cannot overrule them to call a failed container a success. A coding harness may also author config and Dockerfiles itself and have opsmith validate them.
 - **Strategies render and plan, the model estimates.** Given the service model, the bindings and a capacity estimate, a strategy produces artifacts and a machine plan deterministically. Same input, same output. How many machines and of what kind is the strategy's decision: one VM for monolithic, node pools for a future Kubernetes strategy.
 - **Recipes are configuration, not a new format.** A recipe is a partial `deployments.yml` that any strategy can render.
 - **State lives in the user's cloud account.** A per-app bucket holds Terraform state and a copy of the config, with locking.
@@ -187,7 +187,7 @@ Stable keys used by the interaction API. Flags map onto these, and an answers fi
 | `env.domain_email` | `env create`, `update` | `--domain-email` |
 | `env.domain.<slug>` | `env create`, `update` | `--domain slug=host` |
 | `env.state_backend` | `env create` (phase 4) | `--state cloud\|local` |
-| `envvar.<KEY>` | compose env confirmation | `--env-var KEY=VALUE` |
+| `envvar.<KEY>` | compose env confirmation; `env vars` (phase 2) | `--env-var KEY=VALUE` |
 | `build_env.<slug>.<KEY>` | frontend build env | `--build-env slug:KEY=VALUE` |
 | `dns.confirm` | the DNS confirmation as it stands today, over every record at once | `--answer dns.confirm=true` |
 | `dns.<slug>` | `wait_for` once the records are known, replacing `dns.confirm` | run again after creating the records; answered by key only where the strategy cannot verify |
@@ -195,8 +195,8 @@ Stable keys used by the interaction API. Flags map onto these, and an answers fi
 | `run.service`, `run.command` | `run` | positional |
 | `delete.confirm` | `destroy` | `--answer delete.confirm=DELETE` |
 | `update.confirm_infra_changes` | `update` | `--answer update.confirm_infra_changes=true` |
-| `dockerfile.edit`, `compose.edit` | fix editors | exit 4 headless, with the path and the validate command |
-| `config.upgrade` | first save after schema upgrade (phase 2) | `--answer config.upgrade=true` |
+| `update.confirm_compose_upgrade` | `update`, on the first render of an environment deployed before phase 2, when that render would drop something (phase 2) | `--answer update.confirm_compose_upgrade=true` |
+| `dockerfile.edit`, `compose.edit` | fix editors | exit 4 headless, with the path and the validate command || `config.upgrade` | first save after schema upgrade (phase 2) | `--answer config.upgrade=true` |
 | `recipe.input.<KEY>` | `recipe add` (phase 5) | `--input KEY=VALUE` |
 | `template.reset.confirm`, `template.adopt.confirm` | `template reset`, `template adopt` (phase 3) | `--answer template.reset.confirm=true`, `--answer template.adopt.confirm=true` |
 
@@ -252,7 +252,7 @@ directory, or a build context that never read it.
 
 - **Rewriting the CLI in JavaScript.** Ansible is pip-installed today and would become a user prerequisite; after phase 0 the UI layer is thin and its language barely matters. Revisit only if Ansible is replaced.
 - **Compose files as the recipe format.** Recipes are expressed in the `deployments.yml` schema so any strategy can render them.
-- **LLM-rendered docker-compose.** Rendering becomes deterministic in phase 2.
+- **LLM-rendered docker-compose.** Rendering becomes deterministic in phase 2. After a failed deploy the model may repair the deployed file, which releases keep deploying until the next `update` renders from the config again; how a repair survives that is left to phase 3.
 - **An LLM-free mode.** The model is part of the tool and is always configured. Steps that are pure rendering or arithmetic are deterministic for reproducibility, not to remove the model.
 - **A Kubernetes strategy.** Out of scope, but the service model and the capacity-planning hook in phase 2 are designed so one can be added without schema changes: it would implement `plan_capacity` to return node pools instead of one VM.
 
@@ -281,9 +281,9 @@ names "phase 5". Read every one of them through the left column of this table.
 
 Tracked here, resolved inside the phase that owns them.
 
-- Phase 2: whether `FULL_STACK` remains a distinct service type or becomes `BACKEND_API` with a route on `/`.
 - Phase 2: replicas under the monolithic strategy. Current answer: honoured through compose `deploy.replicas` for services without volumes; services with volumes stay at one replica.
 - Phase 4: one state bucket per cloud account versus one per environment. Current answer: per account, keyed by app slug.
 - Phase 5: policy for recipe slugs colliding with detected services. Current answer: fail with a `--slug-prefix` hint.
 - Phase 1: exact skill directories per harness at implementation time; conventions are still moving.
 - Phase 5: whether recipes may ship overlays or override files. Current answer: no, recipes use `files/` only.
+- Phase 3: how the model's repair of a failed deploy (phase 2) survives an `update`, how a fix whose cause is in `deployments.yml` is proposed, confirmed and applied there, and whether the working directory's compose file stays committed now that `release` deploys it (a checkout with an older copy would deploy that). Current answer: the repair is written to that file, which releases deploy until the next `update` renders from the config, and its notice names the field.

opsmith_cli-0.6.1/docs/specs/2026-09-04-phase-2-service-model-v2/2a-schema-v2-and-upgrade.md

@@ -0,0 +1,188 @@
+# Phase 2a: Schema v2 and the v1 upgrade
+
+**Goal:** `deployments.yml` gains schema version 2, with every field the later parts render, the renamed service types and the structural validation rules, and a v1 config is upgraded in memory on load and rewritten on the first save.
+**Depends on:** phase 0.
+**Size:** M.
+**Lands as:** the first part of phase 2; the phase ships as 1.0.0 with phase 3 once 2h lands.
+
+## Scope
+
+1. The schema v2 models in `opsmith/types.py`, and `schema_version`.
+2. The service types renamed, with `FULL_STACK` merged into `WEB_SERVICE`.
+3. The structural validation rules, enforced by `config validate`.
+4. `upgrade_config`, the upgrade of deployed snapshots, and the first save with a backup.
+5. Change detection over the whole service and infra models.
+
+## Non-goals
+
+- Honouring the new fields. Routes, volumes, files, healthchecks, init jobs, resources and image sources are accepted and validated here, and take effect in the part that renders or runs them: 2c and 2d, 2f, 2g and 2h.
+- References and `secret()` (2b), and detection writing them (2d).
+
+## Design
+
+### Schema v2
+
+`DeploymentConfig` gains `schema_version: int = 2`. New and changed models in `opsmith/types.py`:
+
+```python
+class BuildSource(BaseModel):
+    kind: Literal["build"] = "build"
+    context: str = "."                      # build context relative to repo root
+    dockerfile: Optional[str] = None        # default .opsmith/docker/<slug>/Dockerfile
+
+class ImageSource(BaseModel):
+    kind: Literal["image"] = "image"
+    image: str                              # "odoo", "ghcr.io/org/app"
+    tag: str = "latest"
+    platforms: List[str] = ["linux/amd64"]  # used for architecture selection
+
+class Route(BaseModel):
+    path_prefix: str = "/"
+    port: int
+    strip_prefix: bool = False
+
+class VolumeMount(BaseModel):
+    name: str                               # [a-z0-9_]+, unique per config
+    path: str                               # absolute path inside the container
+    backup: bool = False                    # consumed by phase 8
+
+class FileMount(BaseModel):
+    path: str                               # absolute path inside the container
+    content: Optional[str] = None           # template, resolved with the render context
+    source: Optional[str] = None            # path under .opsmith/files/ or a recipe's files/ dir; exclusive with content
+    mode: str = "0644"
+
+class HealthCheck(BaseModel):
+    http_path: Optional[str] = None
+    cmd: Optional[List[str]] = None         # exclusive with http_path
+    port: Optional[int] = None
+    interval_s: int = 10
+    timeout_s: int = 5
+    retries: int = 5
+    start_period_s: int = 30
+
+class Resources(BaseModel):
+    """What one instance of the service needs at light load. Scaled per environment by the capacity plan."""
+    min_ram_gb: float
+    recommended_ram_gb: Optional[float] = None
+    min_cpu: float = 0.25
+    storage_gb: Optional[float] = None       # for services with volumes
+
+class InitJob(BaseModel):
+    name: str
+    command: List[str]
+    when: Literal["first_deploy", "every_release"] = "every_release"
+
+class EnvVarConfig(BaseModel):
+    key: str
+    is_secret: bool = False
+    value: Optional[str] = None             # template; resolved at render time; never prompted
+    default_value: Optional[str] = None     # prompted with this default when value is None
+
+class ServiceInfo(BaseModel):
+    name_slug: str
+    source: Union[BuildSource, ImageSource] = Field(default_factory=BuildSource, discriminator="kind")
+    service_type: ServiceTypeEnum
+    language: Optional[str] = None          # required when source.kind == "build"
+    language_version: Optional[str] = None
+    framework: Optional[str] = None
+    service_port: Optional[int] = None
+    build_cmd / build_dir / build_path      # unchanged, STATIC_SITE only
+    routes: List[Route] = []
+    env_vars: List[EnvVarConfig] = []
+    volumes: List[VolumeMount] = []
+    files: List[FileMount] = []
+    command: Optional[List[str]] = None
+    entrypoint: Optional[List[str]] = None
+    healthcheck: Optional[HealthCheck] = None
+    resources: Optional[Resources] = None
+    init_jobs: List[InitJob] = []
+    depends_on: List[str] = []              # service slugs or infra instance names
+
+class InfrastructureDependency(BaseModel):
+    instance: str                           # default: provider value; unique per config
+    dependency_type: DependencyTypeEnum
+    provider: InfrastructureProviderEnum
+    version: str = "latest"
+    settings: Dict[str, str] = {}           # provider-specific: username, database, extra env
+```
+
+`ServiceTypeEnum` is renamed for what each type still decides once `routes` carry exposure. `FULL_STACK` is merged into the type that replaces `BACKEND_API`, because the two already rendered the same way and no Dockerfile template told them apart:
+
+| v1 | v2 | Meaning |
+|----|----|---------|
+| `BACKEND_API`, `FULL_STACK` | `WEB_SERVICE` | a container that listens on a port and serves requests, whether it is exposed through routes or only reached by other services |
+| `BACKEND_WORKER` | `WORKER` | a container that listens on no port, such as a queue consumer or a scheduler |
+| `FRONTEND` | `STATIC_SITE` | built into static files and served from a CDN, with no container |
+
+- The meanings go into the `service_type` field's description, which detection reads through the generated field list. So an internal API is a `WEB_SERVICE` although it is not public, and a Next.js or Nuxt app that renders on a server is a `WEB_SERVICE`, not a `STATIC_SITE`.
+- The type selects the Dockerfile template, and `python_backend_api` and `python_backend_worker` become `python_web_service` and `python_worker`. For a `STATIC_SITE` it also selects the CDN path.
+- Slugs are not renamed. Detection builds them from the type, so new services get names such as `python_web_service_1`, but an existing slug names compose services, Dockerfile directories and CDN state, and stays as it is.
+- Event step names such as `frontend` are part of the event schema, and keep their names too.
+
+### Validation rules
+
+Enforced by `config validate`. [2b](2b-references-secrets-and-providers.md) adds the three rules about references, `secret()` and duplicate env var keys.
+
+- A service is **exposed** iff `routes` is non-empty. `STATIC_SITE` services never have routes or containers; they keep the CDN path.
+- `route.port` must be `service_port` or another port the container serves; duplicates of `path_prefix` within a service are errors.
+- `volumes[].name` unique across the config; `files[].path` unique per service; `content` xor `source`.
+- `depends_on` entries must exist as a service slug or an infra instance.
+- `language` required for build sources.
+- Infra `instance` unique; `provider` must be compatible with `dependency_type` (existing `COMPATIBLE_PROVIDERS`).
+
+### Upgrade from v1
+
+`opsmith/core/config.py::upgrade_config(data: dict) -> dict`, applied on load when `schema_version` is missing:
+
+- every service gets `source: {kind: build}`;
+- `BACKEND_API` and `FULL_STACK` with `service_port` get `routes: [{path_prefix: "/", port: service_port}]`; `BACKEND_WORKER` gets none;
+- `service_type` is renamed as in the table above: `BACKEND_API` and `FULL_STACK` become `WEB_SERVICE`, `BACKEND_WORKER` becomes `WORKER`, and `FRONTEND` becomes `STATIC_SITE`. Slugs are left alone;
+- infra deps get `instance = provider`;
+- every container service gets `depends_on` listing every infra instance, which is what the LLM-generated compose files declared;
+- env vars keep `default_value`; no `value` is invented.
+- `MonolithicDeploymentState.deployed_services` snapshots are upgraded the same way on load so change detection does not report a spurious full change.
+
+The first save after an upgrade asks `config.upgrade` (`--answer config.upgrade=true` headless) and writes `.opsmith/deployments.v1.bak.yml`.
+
+### Change detection
+
+`_detect_configuration_changes` compares the full `model_dump` of each service and infra instance, keyed by slug and instance, and reports changed field names. Because the deployed snapshots are upgraded the same way as the config, an environment deployed before this part reports no spurious change. [2d](2d-deterministic-deploys.md) adds the compose renderer's version to what counts as a change.
+
+### Until 2d: the LLM compose path
+
+The model still writes the compose file until 2d replaces it, from per-type snippets that it looks up by `service_type`. So the snippets are renamed with the types: `services/backend_api.yml` becomes `services/web_service.yml`, `services/backend_worker.yml` becomes `services/worker.yml`, and `services/full_stack.yml`, which differed from the API snippet only by a trailing blank line, is deleted. Domains are still asked for by type, now `WEB_SERVICE` and `STATIC_SITE`, until 2d asks by routes.
+
+### Detection
+
+The hand-written field list in `REPO_ANALYSIS_PROMPT_TEMPLATE` gets the new type names with their meanings, and asks for `routes` on web services, so a freshly detected config is exposed the same way an upgraded one is. 2d replaces the list with one generated from the schema, together with the instructions for references and `secret()`.
+
+## Code changes by file
+
+| File | Change |
+|------|--------|
+| `opsmith/types.py` | the v2 models; `schema_version`; `ServiceTypeEnum` renamed, with `FULL_STACK` merged; structural validators; deployed snapshots upgraded on load |
+| `opsmith/core/config.py` | load with upgrade, save with backup |
+| `opsmith/deployment_strategies/monolithic.py` | the type renames; change detection over the full models |
+| `opsmith/core/operations.py`, `opsmith/service_detector.py` | the type renames, including `ROUTED_SERVICE_TYPES`, `BUILDABLE_SERVICE_TYPES` and the slugs detection builds |
+| `opsmith/templates/docker_compose_snippets/services/` | the per-type snippets renamed; `full_stack.yml` deleted |
+| `opsmith/templates/dockerfiles/` | `python_backend_api` and `python_backend_worker` renamed to `python_web_service` and `python_worker` |
+| `opsmith/prompts.py` | the detection field list's type names and `routes` |
+| `README.md` | schema v2 documentation |
+
+## Acceptance criteria
+
+1. A v1 project loads, validates, and deploys without edits; the first save writes a v2 file and a backup (phase acceptance criterion 1, which every later part keeps true).
+2. An environment deployed before this part reports no service or infra changes on `update`.
+3. `config validate` rejects a violation of each structural rule with the path to the offending field.
+
+## Tests
+
+- `test_types_v2.py`: validators, discriminator, uniqueness rules.
+- `test_config_upgrade.py`: v1 fixtures upgrade to expected v2 dicts, including the type renames and the `FULL_STACK` merge with slugs unchanged; backup written once.
+- Change detection: an upgraded config against its upgraded snapshots reports no change.
+
+## Harness surface
+
+- `references/config-schema.md` regenerates for schema v2. The freshness test fails until it is committed.
+- `SKILL.md`: the hand-authoring section gets the renamed service types. The skill must say that a v1 config is upgraded in memory and rewritten on the next save, because a harness will meet both shapes in repositories it did not write.

opsmith_cli-0.6.1/docs/specs/2026-09-04-phase-2-service-model-v2/2b-references-secrets-and-providers.md

@@ -0,0 +1,154 @@
+# Phase 2b: References, secrets and provider specs
+
+**Goal:** env values and file contents can reference infrastructure, services and domains through one grammar, app secrets are asked for with `secret()`, and what opsmith knows about each infra provider lives in one registry, all checked by `config validate` without a strategy.
+**Depends on:** 2a.
+**Size:** M.
+**Lands as:** a part of phase 2; the phase ships as 1.0.0 with phase 3 once 2h lands.
+
+## Scope
+
+1. The reference grammar and the resolvers in `opsmith/core/render.py`.
+2. `secret()` and its formats, and the rules for which secrets are generated.
+3. The infra provider registry, `opsmith/infra/providers.py`.
+4. The monolithic bindings.
+5. The validation rules for references, `secret()` and duplicate env var keys.
+
+## Non-goals
+
+- Rendering ([2c](2c-compose-renderer-and-previews.md)), and generating or asking for anything during a deploy ([2d](2d-deterministic-deploys.md)). This part is functions and their tests; no command deploys with them yet.
+- Detection writing references or `secret()` (2d). Until then only a hand-written config holds them, and `config validate` accepts them.
+
+## Design
+
+### Reference grammar and bindings
+
+Templates use Jinja2 with `StrictUndefined`, expression-only, plus one function, `secret(length=32, format='alnum')`, described below. Allowed roots:
+
+| Root | Attributes | Bound by |
+|------|-----------|----------|
+| `infra.<instance>` | `host`, `port`, `username`, `password`, `database`, `url` | strategy |
+| `services.<slug>` | `host`, `port` | strategy |
+| `domains.<slug>` | string | environment config |
+| `inputs.<KEY>` | string | recipe inputs (phase 5); empty until then |
+| `env` | environment name | config |
+| `app` | app slug | config |
+
+Core types in `opsmith/core/render.py`:
+
+```python
+class InfraBinding(BaseModel):
+    host: str; port: int; username: Optional[str]; password: Optional[str]; database: Optional[str]; url: Optional[str]
+
+class ServiceBinding(BaseModel):
+    host: str; port: Optional[int]
+
+class RenderContext(BaseModel):
+    infra: Dict[str, InfraBinding]; services: Dict[str, ServiceBinding]; domains: Dict[str, str]
+    inputs: Dict[str, str]; env: str; app: str
+
+def validate_references(config: DeploymentConfig) -> list[ReferenceError]
+def generated_env_vars(config: DeploymentConfig) -> Dict[str, EnvVarConfig]       # env vars whose value calls secret(), by key
+def generate_value(env_var: EnvVarConfig) -> str                                  # its literal text around one fresh secret()
+def secret_keys(config: DeploymentConfig) -> set[str]                             # env keys holding a secret, declared or derived
+def resolve_env_vars(service: ServiceInfo, ctx: RenderContext, answers: Dict[str, str]) -> Dict[str, str]
+def resolve_files(service: ServiceInfo, ctx: RenderContext) -> Dict[str, str]     # container path -> content
+```
+
+Generation is only for credentials opsmith owns: an infra instance's password, and an app secret that the config asks for with `secret()`. Each is generated once, the first time the environment is deployed, and read back from then on. A third-party key is an env var with `is_secret: true` and no `value`. It is always prompted as `envvar.<KEY>` and never generated, because no generated value is ever right for it.
+
+Which of the two an env var is gets decided when the config is written, and the deploy only reads the answer. The model decides at detection (2d), a person can change it in the `setup` review, and a harness or a recipe that writes the config decides for itself. The test is whether any random value of the right format would work:
+
+- A value the app creates and checks itself gets `secret()`. Examples: Django's `SECRET_KEY`, a JWT signing key, a session or encryption key.
+- A value someone else issues is prompted. Examples: a Stripe key, an OAuth client secret, a webhook secret, an SMTP password, a Sentry DSN.
+- When in doubt it is prompted, because the two mistakes do not cost the same. A generated third-party key deploys and only fails later, at runtime; a prompted app secret costs one question.
+
+`secret()` may appear once in an env var's `value`, with nothing around it but literal text: `"{{ secret() }}"`, or `"base64:{{ secret(32, format='base64') }}"` for Laravel's `APP_KEY`. The env var's key names the whole resolved value, and that value is what is stored and read back. That is why `config validate` rejects a `secret()` beside another reference, a second `secret()` in one value, and any `secret()` in `files[].content`, where there is no key to store it under. `format` sets the shape:
+
+| `format` | Produces | Example |
+|----------|----------|---------|
+| `alnum` (default) | `length` letters and digits, safe unquoted in URLs, shells and `.env` files | Django `SECRET_KEY`: `secret(50)` |
+| `hex` | `length` random bytes as hex | Rails `SECRET_KEY_BASE`: `secret(64, format='hex')` |
+| `base64` | `length` random bytes as standard base64 | Laravel `APP_KEY`: `base64:` followed by `secret(32, format='base64')` |
+| `urlsafe` | `length` random bytes as URL-safe base64, padded | a Fernet key: `secret(32, format='urlsafe')` |
+
+`length` counts characters for `alnum` and random bytes for the other three, because keys in those encodings are specified in bytes. A secret that needs a shape none of these produces is prompted instead.
+
+Generation happens before rendering, never in it. The strategy generates every secret that has no persisted value, meaning infra credentials under its binding keys and each env var in `generated_env_vars(config)`, made with `generate_value`, and records each through `ctx.answers` as `envvar.<KEY>` with `secret=True`. A run that stops before the `.env` reaches the machine therefore keeps what it generated, and once the machine's `.env` is fetched back, `adopt_env_file` makes it the source of truth, as it already is for every other secret. The renderer receives all of these in `answers` and only reads them. 2d wires this into `env create` and `update`.
+
+A value is a secret when its env var declares `is_secret`, or when its template reads an infra `password`, an infra `url` whose provider's `url_template` carries the password, or `secret()`. `secret_keys(config)` derives that set with the same walk `validate_references` does. It exists because a `DATABASE_URL` built from `{{ infra.postgresql.url }}` holds the password whether or not the config marks it secret.
+
+### Infra provider specs
+
+Provider knowledge moves out of prompts and snippets into one registry, `opsmith/infra/providers.py`:
+
+```python
+class InfraProviderSpec(BaseModel):
+    provider: InfrastructureProviderEnum
+    default_port: int
+    url_template: Optional[str]         # "postgresql://{username}:{password}@{host}:{port}/{database}"
+    default_username: Optional[str]     # "{app}" or fixed
+    default_database: Optional[str]
+    secret_env: Dict[str, str]          # legacy env key -> binding attribute, e.g. {"POSTGRES_PASSWORD": "password"}
+    default_ram_gb: float               # for sizing
+    default_cpu: float
+    image_for(version, arch) -> str
+    settings_keys: list[str]            # accepted keys in InfrastructureDependency.settings
+```
+
+Values for the eight providers follow the existing snippets (postgresql 5432, mysql 3306, mongodb 27017, redis 6379, rabbitmq 5672, kafka 9092, elasticsearch 9200, weaviate 8080) with the sizing defaults from the current machine-requirements prompt.
+
+`secret_env` records the env keys the current compose snippets already use, because the env file on every deployed VM holds secrets under exactly these names:
+
+| Provider | `secret_env` |
+|----------|--------------|
+| postgresql | `POSTGRES_PASSWORD` → password |
+| mysql | `MYSQL_PASSWORD` → password, `MYSQL_ROOT_PASSWORD` → root_password |
+| mongodb | `MONGO_INITDB_ROOT_USERNAME` → username, `MONGO_INITDB_ROOT_PASSWORD` → password |
+| rabbitmq | `RABBITMQ_DEFAULT_USER` → username, `RABBITMQ_DEFAULT_PASS` → password |
+| redis, kafka, elasticsearch, weaviate | none; `password` binds to null and the URL carries no credentials |
+
+### Monolithic bindings
+
+```
+infra.<instance>.host      = <instance>            (compose service key)
+infra.<instance>.port      = spec.default_port
+infra.<instance>.username  = settings.username, else the persisted secret for the provider's username key, else spec.default_username
+infra.<instance>.password  = persisted secret under the provider's password key from secret_env
+infra.<instance>.database  = settings.database or spec.default_database
+infra.<instance>.url       = spec.url_template formatted
+services.<slug>.host       = <slug>
+services.<slug>.port       = service_port
+```
+
+Secret env keys are the legacy names from `secret_env` for the default instance of a provider, the one whose instance name equals the provider value, and `<INSTANCE_UPPER>_<KEY>` for any additional instance, for example `ODOO_DB_POSTGRES_PASSWORD`. A persisted value is always read from the env file fetched from the VM before a new one is generated, so an upgraded environment keeps the credentials its database was initialised with.
+
+### Validation rules
+
+Added to the structural rules from [2a](2a-schema-v2-and-upgrade.md), and enforced by `config validate`:
+
+- Every `{{ ... }}` reference in `env_vars[].value` and `files[].content` must resolve against the reference grammar above (checked without a strategy).
+- `secret()` may appear once in an env var's `value`, with nothing around it but literal text: never beside another reference, never twice in one value, never in `files[].content`. Its `format` must be `alnum`, `hex`, `base64` or `urlsafe`.
+- An env var key declared by more than one service must have the same `value` in each. The monolithic strategy writes one `.env` that every service reads as `KEY=${KEY}`, so two services whose `DATABASE_URL` points at different instances would otherwise both get whichever was written last.
+
+## Code changes by file
+
+| File | Change |
+|------|--------|
+| `opsmith/core/render.py` | new: bindings, reference grammar, `secret()` and its formats, resolvers, secret classification and the secrets to generate |
+| `opsmith/infra/providers.py` | new: the provider spec registry |
+| `opsmith/core/config.py` | the `validate_references` hook, and the `secret()` and duplicate-key rules |
+| `opsmith/deployment_strategies/monolithic.py` | the monolithic bindings, building a `RenderContext` for an environment |
+
+## Acceptance criteria
+
+1. `config validate` rejects a bad reference such as `{{ infra.nope.url }}` with the path to the offending env var, a `secret()` beside another reference, twice in one value or in a file, an unknown `format`, and one env var key declared by two services with different values (phase acceptance criterion 6).
+2. `secret(32, format='urlsafe')` decodes as URL-safe base64 to 32 bytes, as a Fernet key must, and `"base64:{{ secret(32, format='base64') }}"` has the shape of a Laravel `APP_KEY` (the shapes in phase acceptance criterion 18; 2d proves the rest).
+
+## Tests
+
+- `test_render.py`: reference validation, `resolve_env_vars` precedence, `secret()` stability with persisted answers, the `secret()` placement, format and duplicate-key rules, the shape of each format's output, and derived secret keys.
+- The provider registry: every provider has a spec, and each `secret_env` names the keys the v1 snippets use.
+
+## Harness surface
+
+None. Nothing a harness writes with references or `secret()` is honoured until 2d, which documents them.

opsmith_cli-0.6.1/docs/specs/2026-09-04-phase-2-service-model-v2/2c-compose-renderer-and-previews.md

@@ -0,0 +1,91 @@
+# Phase 2c: Compose renderer and preview commands
+
+**Goal:** a pure renderer turns the config, an environment's bindings and its answers into a compose file, an env file and mounted files, and three commands preview what it produces, before any deploy uses it.
+**Depends on:** 2b.
+**Size:** L.
+**Lands as:** a part of phase 2; the phase ships as 1.0.0 with phase 3 once 2h lands.
+
+## Scope
+
+1. `ComposeRenderer` and `ComposeBundle`, with golden tests.
+2. The compose templates: `base.yml`, `services/container.yml.j2`, and infra snippets that take `instance` and `settings`.
+3. `opsmith compose render`, `compose diff` and `compose validate`.
+
+## Non-goals
+
+- Deploying with the renderer ([2d](2d-deterministic-deploys.md)). Until then the LLM compose path deploys, and the previews show what 2d will deploy.
+- `deploy.replicas`, which [2g](2g-capacity-planning.md) adds from the capacity plan.
+
+## Design
+
+### Compose renderer
+
+`opsmith/deployment_strategies/compose_renderer.py`:
+
+```python
+class ComposeBundle(BaseModel):
+    compose_yaml: str
+    env_file: Dict[str, str]                 # KEY -> value, secrets included
+    secret_keys: List[str]                   # env_file keys holding a secret, from secret_keys(config)
+    files: Dict[str, str]                    # relative path under files/ -> content
+    init_jobs: List[Tuple[str, InitJob]]     # (service slug, job)
+
+class ComposeRenderer:
+    def render(self, config, environment, env_state, images: Dict[str, str], answers: Dict[str, str]) -> ComposeBundle
+```
+
+Rules:
+
+- Build a dict, not text. Load `base.yml` (traefik, logging anchor, network), add one entry per container service from a single template `services/container.yml.j2`, one per infra instance from the provider spec, then `yaml.safe_dump`.
+- Container service entry: `image` (from `images[slug]` for build sources, `image:tag` for image sources), `command`, `entrypoint`, `environment` as `KEY=${KEY}` references for every resolved env var, `volumes` for named volumes and `./files/<slug>/<n>:<path>:ro` for files, `healthcheck` translated to compose form, `depends_on`, `restart: always`, the logging anchor, and traefik labels per route: router `<slug>-r<i>` with rule ``Host(`<domain>`) && PathPrefix(`<prefix>`)``, service `<slug>-r<i>` pointing at `route.port`, plus the existing https redirect and security-headers middlewares. A service with routes but no domain in the environment is a validation error before rendering.
+- Infra entry: compose key is the instance name; env from provider spec and `settings`; password from the persisted secret; named volume `<instance>-data`.
+- Top-level `volumes:` lists every named volume once.
+- `service_type` no longer affects rendering; it selects Dockerfile templates and, for `STATIC_SITE`, the CDN path only. The per-type snippets stay in place for the LLM path until 2d deletes them.
+- `.env` composition: infra passwords and `secret()` values, resolved `value` templates, then the env vars with no `value`. All of them arrive in `answers`, and the renderer asks nothing and generates nothing: 2d generates the secrets and asks for the rest before rendering.
+
+The renderer is pure: same inputs, same bundle. Golden tests cover it.
+
+Override files are not merged by the renderer. Phase 3 hands them to Compose, which merges them at deploy time, so the renderer stays pure and overrides survive upgrades.
+
+The golden render of an upgraded v1 config is what proves compatibility guarantees 1, 3 and 5, and the key names in guarantee 2, listed in the [README](README.md#compatibility-with-existing-environments).
+
+### Preview commands
+
+Three of the commands the [harness spec](../2026-09-04-phase-1-harness-integration.md) lists ship here, because each one needs this phase's renderer and none of them could be written before it:
+
+```
+opsmith compose render   --env NAME      # rendered docker-compose.yml plus env keys, secret values masked
+opsmith compose diff     --env NAME      # rendered file against the one currently on the VM
+opsmith compose validate --env NAME      # renders, then runs `docker compose config` locally
+```
+
+They also give the first deterministic `update` something to be previewed with. `compose diff` fetches the remote compose file with `_fetch_remote_deployment_files` and prints a unified diff without deploying. Each command returns a typed result, so the generated `commands.md` names its model.
+
+`compose render` masks values by `ComposeBundle.secret_keys` rather than by `is_secret` alone, because its output is read by agents. None of the three generates, prompts or writes to the working directory, whose `docker-compose.yml` is what `release` deploys. A secret not yet generated, and an env var not yet answered, are shown as placeholders, so previewing a new environment creates nothing.
+
+## Code changes by file
+
+| File | Change |
+|------|--------|
+| `opsmith/deployment_strategies/compose_renderer.py` | new |
+| `opsmith/templates/docker_compose_snippets/` | `services/container.yml.j2`; infra snippets take `instance` and `settings`, which the LLM path passes as the provider until 2d |
+| `opsmith/core/operations.py`, `opsmith/core/results.py` | the three preview operations and their typed results |
+| `opsmith/cli/commands/` | `compose render`, `compose diff` and `compose validate` |
+
+## Acceptance criteria
+
+1. A fixture environment deployed from a v1 config with a captured LLM-style compose file, whose env file holds the legacy secret keys, renders a compose file with identical volume names, infra service keys and secret env keys, and `compose diff` shows only label and formatting changes (phase acceptance criterion 7).
+2. `compose diff --env dev` prints the difference between the rendered file and the file fetched from the VM without deploying (phase acceptance criterion 8).
+3. `compose render` masks a `DATABASE_URL` whose value is `{{ infra.postgresql.url }}` although the config does not mark it secret (the masking in phase acceptance criterion 15; 2d proves the rest).
+4. None of the three commands generates a secret, asks a question or writes to the working directory.
+
+## Tests
+
+- `test_compose_renderer.py`: golden files for api+worker+postgres+redis, image-only service with routes and files, two routes on one service; a renderer that never generates.
+- `test_compat_v1_environment.py`: golden render of an upgraded v1 config asserting legacy volume names, infra keys, secret keys and `depends_on`; `compose diff` against a captured LLM-generated file.
+- The preview commands: masking by derived secret keys, placeholders for what is not generated or answered yet, and nothing written.
+
+## Harness surface
+
+- `references/commands.md` regenerates for the `compose` commands.
+- `SKILL.md`: the validate loop gains `compose render`, `compose diff` and `compose validate`.