composable-data-stack 0.12.0__tar.gz → 0.14.0__tar.gz

{composable_data_stack-0.12.0/composable_data_stack.egg-info → composable_data_stack-0.14.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: composable-data-stack
-Version: 0.12.0
+Version: 0.14.0
 Summary: CLI for validating and composing composable data stack profiles
 Author: Ronald Hensbergen
 License-Expression: Apache-2.0
@@ -32,7 +32,7 @@ Provides-Extra: dev
 Requires-Dist: coverage>=7.6; extra == "dev"
 Requires-Dist: bandit>=1.7; extra == "dev"
 Requires-Dist: pip-audit>=2.7; extra == "dev"
-Requires-Dist: ruff==0.17.0; extra == "dev"
+Requires-Dist: ruff==0.17.1; extra == "dev"
 Dynamic: license-file
 
 # 🚀 Composable Data Stack (CDS)
@@ -813,9 +813,23 @@ Common errors from `cds validate`, `cds plan`, and `cds render`, and how to fix
 | `[E081] ... Required secret "CDS_X_PASSWORD" not found in environment` | A secret marked `required: true` in the profile's `spec.secrets.values` is missing from the shell environment or the `.env` file in the current working directory. | Run `cds init <profile>` to generate `.env` in the project root, set the missing `CDS_*` variable, or export it directly before running the command. |
 | `[E041] ... Contract ref "x.y" points to unknown module "x"` | A `consumes` binding's `contractRef` refers to a module ID that isn't defined in the profile. | Check `spec.modules` for the correct module `id`, and confirm the contract ref follows `<module-id>.<contract-name>`. |
 | `[E041] ... but it does not provide "<contract-name>"` | The referenced module exists, but its `spec.provides` list doesn't expose that contract name. | Check the producing module's `module.yaml` for the contracts it actually provides, and fix the consumer's `contractRef` to match. |
+| `[E129] ... Certificate ref ... is not defined in spec.certificates.values` (or `malformed`) | A module config value references `certificates.<alias>.cert`/`.key`, but the alias isn't declared under `spec.certificates.values` -- or the reference is bare/has an unknown trailing part instead of `.cert` or `.key`. | Declare the alias under `spec.certificates.values` with its `certEnv`/`keyEnv` names, or fix the reference to name one half of a declared pair. CDS only maps names; the certificate/key material itself is supplied at runtime by the external secret/cert provider. |
 | `[E042] ... Contract kind mismatch` | The consumer expects one contract kind (e.g. `sql-database`) but the producer exposes a different kind. | Point the binding at a module that provides the expected contract kind, or update the consumer's expected kind if the mismatch is intentional. |
 | `[E043] ... recorded as unsupported in the compatibility registry` | The binding's `kind` matches, but this exact provider/consumer pairing is explicitly recorded as `unsupported` in `cli/resources/compatibility-registry.json` -- it's known NOT to work together despite matching shape. | Use a different provider/consumer pairing for this contract, or if the pairing has since been fixed/verified, update its `status` to `tested` in the registry (see [docs/architecture.md § Secrets and contract resolution](docs/architecture.md#secrets-and-contract-resolution)). |
+| `[E128] ... requires a TLS-protected endpoint (tls: required), but ... does not advertise one` | A module's `consumes` entry declares `tls: required` (issue #577), but the bound provider's contract spec does not set `tls: true` (absent, `false`, or a `${config.<path>}` reference that does not resolve to `true` against the producer's profile config). | Point the binding at a provider that advertises TLS, or have the provider set `tls: true` in its provide entry's contract spec. Drop `tls: required` only if the binding is genuinely allowed to run unencrypted. |
+
+| `[E130] ... Duplicate backup policy for workload class "<class>"` | `spec.backup.policies` declares more than one policy for the same workload class. | Keep one policy per workload class. |
+| `[E131] ... Backup encryption key ref ... is not defined in spec.secrets.values` | `spec.backup.encryption.keyFrom` references a `secrets.<alias>` that the profile does not declare. | Declare the alias under `spec.secrets.values` with its `env` name, or fix the reference. |
+| `[E132] ... Backup target contractRef ...` | The pinned `spec.backup.target.contractRef` is malformed, points to an unknown module or contract, is not a `backup-target` provider, or (in production) the provider does not advertise `encrypted: true` / `supportsRestoreVerification: true`. | Point at a module providing a `backup-target` contract that advertises the required capabilities, or drop the pin to leave the storage provider open. |
+| `[E133] ... Production profiles must declare spec.backup ...` / `... missing workload classes: ...` | A `metadata.environment: production` profile has no `spec.backup` block, or lacks a policy for a workload class it runs (warehouse, orchestration/BI/identity/secrets metadata, or configuration). | Declare encrypted, restore-verified backup policies for every workload class the profile runs; see `docs/backup-restore.md`. |
+| `[E134] ... Production backups must be encrypted ...` / `... must be declared required: true` | A production profile has no `spec.backup.encryption.keyFrom`, or the referenced secret alias is declared `required: false` so the key can be missing at runtime. | Declare the encryption key alias with `required: true` so a missing key fails closed at startup. |
+| `[E135] ... Production backup policy "<class>" must declare restoreVerification` | A production backup policy has no `restoreVerification` block. | Add `restoreVerification.frequencyDays` to every production backup policy.|
+| `[E136] ... Reverse-proxy route host ...` (unknown target / no resolvable http-service endpoint / backend address not a rendered compose service / entrypoint not declared by the proxy / undeclared certificate alias / duplicate route host) | A `config.routes` entry on a reverse-proxy provider names a module that is missing, does not provide an `http-service` contract with host/port (or a port outside 1-65535), advertises a backend address that matches no rendered compose service (multi-service modules must advertise `serviceName`), names a `<module-id>.<contract-name>` contract the target does not provide, names an entrypoint the proxy's `config.entryPoints` does not declare, references an undeclared certificate alias, or duplicates the host of an earlier route on the same proxy (#581). | Fix the route's `target`/`certificate`/`host`/`entryPoints` (a bare module id routes to its first http-service contract; `<module-id>.<contract-name>` picks a specific one; entrypoints must match the proxy's declared `config.entryPoints` keys), or advertise the serving compose service via `serviceName` in the target's http-service provide entry. |
 | `[E103] ... config.image.tag is required ... when config.image.source is "registry"` | A module instance sets `config.image.source: registry` without also setting `config.image.tag`. | Set `config.image.tag` to a version, or switch back to `config.image.source: build`. |
+| `[E139] ... without an explicit host authentication advertisement` / `... host authentication method "trust"` | A `metadata.environment: production` profile runs a `sql-database` provider whose contract does not advertise a host authentication method, or advertises `trust` (issue #206). | Configure the provider's authentication explicitly (e.g. postgres `config.authentication.hostAuth: scram-sha-256`); trust authentication is prohibited in production profiles. |
+| `[E137] ... accepts unauthenticated clients` | A production profile runs a `cache-service` provider that does not advertise `authentication: required` (no password credential configured). | Supply a cache password through a `secrets.<alias>` reference (e.g. keydb `config.passwordFrom`), declared `required: true` in `spec.secrets.values`. |
+| `[E138] ... Credential field ... must reference a secret alias` / `... is empty` / `... uses a ${CDS_*} profile input` / `... contains a plaintext value` | A credential-named config field of a production database/cache provider is not a `secrets.<alias>` reference, or its alias is not declared `required: true` (issue #206). | Supply the credential through a secret reference declared `required: true` so a missing value fails closed at startup. |
+| `[W105] ... legacy md5 host authentication` | A production `sql-database` provider advertises `md5`, which validates but is a deprecated password method. | Prefer `scram-sha-256` for production profiles. |
 | `[W097] ... config.image.tag is "latest" with config.image.source "registry"` | `config.image.tag: latest` under `source: registry` still validates, but drifts silently between deploys instead of pinning a reproducible version. | Pin an explicit tag from `publish-images.yml`'s output (or `tests/fixtures/signed-images.json`) instead of `latest`. |
 
 All diagnostics print with their error code and YAML path (e.g. `spec.modules[1].config`), so search the profile file for that path to find the exact line to fix.

{composable_data_stack-0.12.0 → composable_data_stack-0.14.0}/README.md

@@ -776,9 +776,23 @@ Common errors from `cds validate`, `cds plan`, and `cds render`, and how to fix
 | `[E081] ... Required secret "CDS_X_PASSWORD" not found in environment` | A secret marked `required: true` in the profile's `spec.secrets.values` is missing from the shell environment or the `.env` file in the current working directory. | Run `cds init <profile>` to generate `.env` in the project root, set the missing `CDS_*` variable, or export it directly before running the command. |
 | `[E041] ... Contract ref "x.y" points to unknown module "x"` | A `consumes` binding's `contractRef` refers to a module ID that isn't defined in the profile. | Check `spec.modules` for the correct module `id`, and confirm the contract ref follows `<module-id>.<contract-name>`. |
 | `[E041] ... but it does not provide "<contract-name>"` | The referenced module exists, but its `spec.provides` list doesn't expose that contract name. | Check the producing module's `module.yaml` for the contracts it actually provides, and fix the consumer's `contractRef` to match. |
+| `[E129] ... Certificate ref ... is not defined in spec.certificates.values` (or `malformed`) | A module config value references `certificates.<alias>.cert`/`.key`, but the alias isn't declared under `spec.certificates.values` -- or the reference is bare/has an unknown trailing part instead of `.cert` or `.key`. | Declare the alias under `spec.certificates.values` with its `certEnv`/`keyEnv` names, or fix the reference to name one half of a declared pair. CDS only maps names; the certificate/key material itself is supplied at runtime by the external secret/cert provider. |
 | `[E042] ... Contract kind mismatch` | The consumer expects one contract kind (e.g. `sql-database`) but the producer exposes a different kind. | Point the binding at a module that provides the expected contract kind, or update the consumer's expected kind if the mismatch is intentional. |
 | `[E043] ... recorded as unsupported in the compatibility registry` | The binding's `kind` matches, but this exact provider/consumer pairing is explicitly recorded as `unsupported` in `cli/resources/compatibility-registry.json` -- it's known NOT to work together despite matching shape. | Use a different provider/consumer pairing for this contract, or if the pairing has since been fixed/verified, update its `status` to `tested` in the registry (see [docs/architecture.md § Secrets and contract resolution](docs/architecture.md#secrets-and-contract-resolution)). |
+| `[E128] ... requires a TLS-protected endpoint (tls: required), but ... does not advertise one` | A module's `consumes` entry declares `tls: required` (issue #577), but the bound provider's contract spec does not set `tls: true` (absent, `false`, or a `${config.<path>}` reference that does not resolve to `true` against the producer's profile config). | Point the binding at a provider that advertises TLS, or have the provider set `tls: true` in its provide entry's contract spec. Drop `tls: required` only if the binding is genuinely allowed to run unencrypted. |
+
+| `[E130] ... Duplicate backup policy for workload class "<class>"` | `spec.backup.policies` declares more than one policy for the same workload class. | Keep one policy per workload class. |
+| `[E131] ... Backup encryption key ref ... is not defined in spec.secrets.values` | `spec.backup.encryption.keyFrom` references a `secrets.<alias>` that the profile does not declare. | Declare the alias under `spec.secrets.values` with its `env` name, or fix the reference. |
+| `[E132] ... Backup target contractRef ...` | The pinned `spec.backup.target.contractRef` is malformed, points to an unknown module or contract, is not a `backup-target` provider, or (in production) the provider does not advertise `encrypted: true` / `supportsRestoreVerification: true`. | Point at a module providing a `backup-target` contract that advertises the required capabilities, or drop the pin to leave the storage provider open. |
+| `[E133] ... Production profiles must declare spec.backup ...` / `... missing workload classes: ...` | A `metadata.environment: production` profile has no `spec.backup` block, or lacks a policy for a workload class it runs (warehouse, orchestration/BI/identity/secrets metadata, or configuration). | Declare encrypted, restore-verified backup policies for every workload class the profile runs; see `docs/backup-restore.md`. |
+| `[E134] ... Production backups must be encrypted ...` / `... must be declared required: true` | A production profile has no `spec.backup.encryption.keyFrom`, or the referenced secret alias is declared `required: false` so the key can be missing at runtime. | Declare the encryption key alias with `required: true` so a missing key fails closed at startup. |
+| `[E135] ... Production backup policy "<class>" must declare restoreVerification` | A production backup policy has no `restoreVerification` block. | Add `restoreVerification.frequencyDays` to every production backup policy.|
+| `[E136] ... Reverse-proxy route host ...` (unknown target / no resolvable http-service endpoint / backend address not a rendered compose service / entrypoint not declared by the proxy / undeclared certificate alias / duplicate route host) | A `config.routes` entry on a reverse-proxy provider names a module that is missing, does not provide an `http-service` contract with host/port (or a port outside 1-65535), advertises a backend address that matches no rendered compose service (multi-service modules must advertise `serviceName`), names a `<module-id>.<contract-name>` contract the target does not provide, names an entrypoint the proxy's `config.entryPoints` does not declare, references an undeclared certificate alias, or duplicates the host of an earlier route on the same proxy (#581). | Fix the route's `target`/`certificate`/`host`/`entryPoints` (a bare module id routes to its first http-service contract; `<module-id>.<contract-name>` picks a specific one; entrypoints must match the proxy's declared `config.entryPoints` keys), or advertise the serving compose service via `serviceName` in the target's http-service provide entry. |
 | `[E103] ... config.image.tag is required ... when config.image.source is "registry"` | A module instance sets `config.image.source: registry` without also setting `config.image.tag`. | Set `config.image.tag` to a version, or switch back to `config.image.source: build`. |
+| `[E139] ... without an explicit host authentication advertisement` / `... host authentication method "trust"` | A `metadata.environment: production` profile runs a `sql-database` provider whose contract does not advertise a host authentication method, or advertises `trust` (issue #206). | Configure the provider's authentication explicitly (e.g. postgres `config.authentication.hostAuth: scram-sha-256`); trust authentication is prohibited in production profiles. |
+| `[E137] ... accepts unauthenticated clients` | A production profile runs a `cache-service` provider that does not advertise `authentication: required` (no password credential configured). | Supply a cache password through a `secrets.<alias>` reference (e.g. keydb `config.passwordFrom`), declared `required: true` in `spec.secrets.values`. |
+| `[E138] ... Credential field ... must reference a secret alias` / `... is empty` / `... uses a ${CDS_*} profile input` / `... contains a plaintext value` | A credential-named config field of a production database/cache provider is not a `secrets.<alias>` reference, or its alias is not declared `required: true` (issue #206). | Supply the credential through a secret reference declared `required: true` so a missing value fails closed at startup. |
+| `[W105] ... legacy md5 host authentication` | A production `sql-database` provider advertises `md5`, which validates but is a deprecated password method. | Prefer `scram-sha-256` for production profiles. |
 | `[W097] ... config.image.tag is "latest" with config.image.source "registry"` | `config.image.tag: latest` under `source: registry` still validates, but drifts silently between deploys instead of pinning a reproducible version. | Pin an explicit tag from `publish-images.yml`'s output (or `tests/fixtures/signed-images.json`) instead of `latest`. |
 
 All diagnostics print with their error code and YAML path (e.g. `spec.modules[1].config`), so search the profile file for that path to find the exact line to fix.

composable_data_stack-0.14.0/cli/confinement.py

@@ -0,0 +1,219 @@
+"""Declarative runtime confinement (seccomp, AppArmor, SELinux) for Compose services.
+
+A profile declares a default under `spec.security.runtime.confinement`; a
+module instance may override individual keys under `security.confinement`.
+The effective value is rendered into each service's `security_opt`.
+"""
+from __future__ import annotations
+
+import os
+import re
+from pathlib import Path
+from typing import Any
+
+from .diagnostics import Diagnostic
+
+INVALID_CONFINEMENT_CODE = "E127"
+CONFINEMENT_KEYS = ("seccomp", "apparmor", "selinux")
+
+_APPARMOR_NAME_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._/-]{0,127}$")
+_DRIVE_RE = re.compile(r"^[A-Za-z]:")
+_SELINUX_LABEL_RE = re.compile(r"^(user|role|type|level):[A-Za-z0-9_.,:-]+$")
+
+
+def profile_confinement(profile_or_plan: dict[str, Any] | None) -> dict[str, Any]:
+    """Return the profile-wide confinement block from a profile or plan."""
+    if not isinstance(profile_or_plan, dict):
+        return {}
+    security = profile_or_plan.get("security")
+    if security is None:
+        security = (profile_or_plan.get("spec") or {}).get("security")
+    runtime = (security or {}).get("runtime") if isinstance(security, dict) else None
+    block = (runtime or {}).get("confinement") if isinstance(runtime, dict) else None
+    return block if isinstance(block, dict) else {}
+
+
+def plan_declares_confinement(plan: dict[str, Any] | None) -> bool:
+    """True when the profile or any module instance sets confinement."""
+    if profile_confinement(plan):
+        return True
+    return any(
+        isinstance((module.get("security") or {}).get("confinement"), dict)
+        and (module.get("security") or {}).get("confinement")
+        for module in (plan or {}).get("modules") or []
+        if isinstance(module, dict)
+    )
+
+
+def effective_confinement(
+    defaults: dict[str, Any] | None, override: dict[str, Any] | None
+) -> dict[str, Any]:
+    """Profile defaults with the module instance's keys layered on top."""
+    merged = {key: value for key, value in (defaults or {}).items() if key in CONFINEMENT_KEYS}
+    for key, value in (override or {}).items():
+        if key in CONFINEMENT_KEYS:
+            merged[key] = value
+    return merged
+
+
+def validate_confinement(
+    block: Any, path: str, base_dir: Path | None = None
+) -> list[Diagnostic]:
+    """Check one confinement block; `base_dir` anchors `file:` seccomp profiles."""
+    if block is None:
+        return []
+    if not isinstance(block, dict):
+        return [_error(f"{path} must be an object.", path)]
+    diagnostics: list[Diagnostic] = []
+    for key in block:
+        if key not in CONFINEMENT_KEYS:
+            diagnostics.append(_error(
+                f"Unknown confinement key '{key}'; supported keys: {', '.join(CONFINEMENT_KEYS)}.",
+                f"{path}.{key}",
+            ))
+
+    seccomp = block.get("seccomp")
+    if seccomp is not None:
+        diagnostics.extend(_validate_seccomp(seccomp, f"{path}.seccomp", base_dir))
+    apparmor = block.get("apparmor")
+    if apparmor is not None and not (
+        isinstance(apparmor, str)
+        and (apparmor in {"runtime/default", "unconfined"} or _APPARMOR_NAME_RE.match(apparmor))
+    ):
+        diagnostics.append(_error(
+            "apparmor must be 'runtime/default', 'unconfined' or the name of a profile "
+            "already loaded on the host (letters, digits, '.', '_', '-', '/').",
+            f"{path}.apparmor",
+        ))
+    selinux = block.get("selinux")
+    if selinux is not None:
+        diagnostics.extend(_validate_selinux(selinux, f"{path}.selinux"))
+    return diagnostics
+
+
+def _validate_seccomp(value: Any, path: str, base_dir: Path | None) -> list[Diagnostic]:
+    if value in {"default", "unconfined"}:
+        return []
+    if not isinstance(value, str) or not value.startswith("file:") or len(value) == 5:
+        return [_error(
+            "seccomp must be 'default', 'unconfined' or 'file:<path to a JSON profile "
+            "relative to the profile directory>'.",
+            path,
+        )]
+    relative = value[5:]
+    if base_dir is None:
+        return [_error(
+            "seccomp file profiles need a known profile directory to resolve against.", path
+        )]
+    # Reject rooted and drive-qualified paths explicitly: they are not
+    # "absolute" to pathlib on every platform (e.g. "/x" on Windows).
+    if (
+        Path(relative).is_absolute()
+        or relative.startswith(("/", "\\"))
+        or _DRIVE_RE.match(relative)
+        or ".." in Path(relative).parts
+    ):
+        return [_error(
+            "seccomp file profiles must be relative paths inside the profile directory.", path
+        )]
+    target = (base_dir / relative).resolve()
+    if not target.is_relative_to(base_dir.resolve()):
+        return [_error(
+            "seccomp file profile resolves outside the profile directory.", path
+        )]
+    if not target.is_file():
+        return [_error(f"seccomp profile '{relative}' does not exist.", path)]
+    return []
+
+
+def _validate_selinux(value: Any, path: str) -> list[Diagnostic]:
+    if value == "disable":
+        return []
+    if not isinstance(value, list) or not value or not all(isinstance(v, str) for v in value):
+        return [_error(
+            "selinux must be 'disable' or a non-empty list of labels such as "
+            "'type:container_t' or 'level:s0:c100,c200'.",
+            path,
+        )]
+    bad = [label for label in value if not _SELINUX_LABEL_RE.match(label)]
+    if bad:
+        return [_error(
+            f"Invalid SELinux label(s): {', '.join(bad)}; expected user:, role:, type: or level: values.",
+            path,
+        )]
+    kinds = [label.split(":", 1)[0] for label in value]
+    duplicates = sorted({k for k in kinds if kinds.count(k) > 1})
+    if duplicates:
+        return [_error(
+            f"Duplicate SELinux label kind(s): {', '.join(duplicates)}; "
+            "each of user:, role:, type: and level: may appear at most once.",
+            path,
+        )]
+    return []
+
+
+def confinement_security_opts(
+    confinement: dict[str, Any], compose_dir: Path, base_dir: Path | None
+) -> list[str]:
+    """Translate a validated confinement block into Compose `security_opt` entries.
+
+    Raises ValueError when a seccomp file cannot be expressed relative to
+    `compose_dir` (for example on a different Windows drive).
+    """
+    opts: list[str] = []
+    seccomp = confinement.get("seccomp")
+    if seccomp == "unconfined":
+        opts.append("seccomp=unconfined")
+    elif isinstance(seccomp, str) and seccomp.startswith("file:"):
+        target = (base_dir or compose_dir) / seccomp[5:]
+        opts.append(f"seccomp:{Path(os.path.relpath(target, compose_dir)).as_posix()}")
+    apparmor = confinement.get("apparmor")
+    if apparmor == "runtime/default":
+        opts.append("apparmor=docker-default")
+    elif isinstance(apparmor, str):
+        opts.append(f"apparmor={apparmor}")
+    selinux = confinement.get("selinux")
+    if selinux == "disable":
+        opts.append("label=disable")
+    elif isinstance(selinux, list):
+        opts.extend(f"label={label}" for label in selinux)
+    return opts
+
+
+def apply_confinement(
+    service: dict[str, Any],
+    confinement: dict[str, Any],
+    compose_dir: Path,
+    base_dir: Path | None,
+    path: str = "security.confinement",
+) -> list[Diagnostic]:
+    """Set the declared confinement on a service, replacing module-set values per key."""
+    if not confinement:
+        return []
+    try:
+        new_opts = confinement_security_opts(confinement, compose_dir, base_dir)
+    except ValueError:
+        return [_error(
+            "seccomp file profile cannot be referenced relative to the generated "
+            "docker-compose.yml (different drive); render into the profile's drive.",
+            f"{path}.seccomp",
+        )]
+    existing = [str(opt) for opt in service.get("security_opt") or []]
+    replaced = {
+        "seccomp": ("seccomp=", "seccomp:"),
+        "apparmor": ("apparmor=", "apparmor:"),
+        "selinux": ("label=", "label:"),
+    }
+    prefixes = tuple(p for key in confinement for p in replaced.get(key, ()))
+    kept = [opt for opt in existing if not opt.startswith(prefixes)] if prefixes else existing
+    # `default` seccomp means "engine default", so it only clears module-set values.
+    merged = kept + new_opts
+    if merged:
+        service["security_opt"] = merged
+    else:
+        service.pop("security_opt", None)
+    return []
+
+
+def _error(message: str, path: str) -> Diagnostic:
+    return Diagnostic(level="error", code=INVALID_CONFINEMENT_CODE, message=message, path=path)

{composable_data_stack-0.12.0 → composable_data_stack-0.14.0}/cli/getter.py

@@ -664,7 +664,7 @@ def _extract_dockerfile_instruction_sources(
             continue
         if _contains_template(source):
             continue
-        if Path(source).is_absolute():
+        if Path(source).is_absolute() or source.startswith(("/", "\\")):
             continue
         matches = list(context_path.glob(source))
         if not matches and "*" not in source and "?" not in source and "[" not in source:

{composable_data_stack-0.12.0 → composable_data_stack-0.14.0}/cli/k8s_renderer.py

@@ -28,6 +28,7 @@ from typing import Any
 
 import yaml
 
+from .confinement import plan_declares_confinement
 from .diagnostics import Diagnostic
 from .renderer import (
     _build_context,
@@ -37,6 +38,12 @@ from .renderer import (
     _resolve_profile_dir,
     _resolve_project_root,
     _substitute_values,
+    apply_production_gates,
+)
+from .secret_leak import (
+    check_generated_text,
+    declared_secret_env_names,
+    resolved_secret_values,
 )
 from .utils import _atomic_write
 
@@ -73,12 +80,14 @@ class K8sRenderError(Exception):
 def render_helm(
     plan: dict[str, Any],
     output_dir: str | None = None,
+    env_file: str | None = None,
 ) -> tuple[dict[str, str], list[Diagnostic]]:
     """
     Render a Helm chart from a resolved plan.
 
     Returns a mapping of chart-relative path -> file contents, plus diagnostics.
-    Files are written to `output_dir` when given.
+    Files are written to `output_dir` when given, unless an error was found
+    (including a resolved secret value in any chart file, CDS-SEC-030).
     """
     diagnostics: list[Diagnostic] = []
     metadata = plan.get("metadata", {})
@@ -103,6 +112,21 @@ def render_helm(
                 path=f"module:{module_id}.implementation.kubernetes",
             )
         )
+
+    if plan_declares_confinement(plan):
+        diagnostics.append(
+            Diagnostic(
+                level="warning",
+                code="W127",
+                message=(
+                    "Runtime confinement (seccomp/AppArmor/SELinux) is only rendered for "
+                    "the compose target; it has no effect on the Helm chart. "
+                    "Kubernetes security contexts are tracked under #65."
+                ),
+                path="spec.security.runtime.confinement",
+            )
+        )
+
     if unsupported:
         return {}, diagnostics
 
@@ -110,6 +134,28 @@ def render_helm(
     # ${k8s.service.*} can point forward to a workload declared later.
     service_names = _collect_service_names(renderable)
     diagnostics.extend(_check_binding_services(renderable, service_names))
+
+    # The built-in backup capability (config.backup.enabled, issue #665) is
+    # only implemented for the compose target: there is no Kubernetes
+    # sidecar workload yet. Fail closed instead of rendering a chart whose
+    # database is silently unprotected.
+    for module in renderable:
+        backup_config = (module.get("config") or {}).get("backup")
+        if isinstance(backup_config, dict) and backup_config.get("enabled") is True:
+            diagnostics.append(
+                Diagnostic(
+                    level="error",
+                    code="E142",
+                    message=(
+                        f'Module "{module["id"]}" enables config.backup.enabled, but the '
+                        'backup capability is not implemented for the "helm" render target '
+                        "(no Kubernetes sidecar workload exists yet). Render this profile "
+                        "with --target=compose, or disable backup for Kubernetes deploys."
+                    ),
+                    path=f"module:{module['id']}.config.backup.enabled",
+                )
+            )
+
     if any(diagnostic.level == "error" for diagnostic in diagnostics):
         return {}, diagnostics
 
@@ -134,8 +180,13 @@ def render_helm(
             "service": _service_context(module, service_names)
         }
         _release_scope_binding_hosts(context, service_names)
-        compose_services = deepcopy(
-            module["implementation"].get("compose", {}).get("services", {})
+        # The helm target is treated as production-only for now: Kubernetes
+        # deploys always render the production variants, regardless of the
+        # profile's metadata.environment. The learning (easy-install)
+        # variants exist for the compose target.
+        compose_services = apply_production_gates(
+            deepcopy(module["implementation"].get("compose", {}).get("services", {})),
+            True,
         )
         _merge_init_db_env(compose_services, module, secrets)
 
@@ -155,6 +206,7 @@ def render_helm(
             service_names=service_names,
             profile_dir=profile_dir,
             project_root=project_root,
+            is_production=True,
         )
         values["modules"][module_id] = {
             "config": _values_config(module.get("config", {})),
@@ -183,6 +235,11 @@ def render_helm(
 
     diagnostics.extend(_check_unresolved(files))
     diagnostics.extend(_check_secret_references(files, secret_keys))
+    secret_values = resolved_secret_values(declared_secret_env_names(plan), env_file)
+    for relative_path, content in files.items():
+        diagnostics.extend(check_generated_text(
+            content, f"{output_dir or 'chart'}/{relative_path}", secret_values
+        ))
 
     if output_dir and not any(
         diagnostic.level == "error" for diagnostic in diagnostics
@@ -207,6 +264,7 @@ def _render_module(
     service_names: dict[str, str],
     profile_dir: Path | None,
     project_root: Path | None,
+    is_production: bool = False,
 ) -> tuple[
     dict[str, Any],
     list[tuple[str, dict[str, Any]]],
@@ -220,12 +278,16 @@ def _render_module(
     secret_keys: set[str] = set()
     module_values: dict[str, Any] = {}
 
-    volume_specs = _substitute_values(deepcopy(k8s.get("volumes", {})), context)
+    volume_specs = apply_production_gates(
+        _substitute_values(deepcopy(k8s.get("volumes", {})), context), is_production
+    )
+    configmap_specs = apply_production_gates(
+        _substitute_values(deepcopy(k8s.get("configMaps", {})), context), is_production
+    )
     for volume_spec in volume_specs.values():
         enabled_from = volume_spec.get("enabledFrom")
         if isinstance(enabled_from, str):
             volume_spec["enabledFrom"] = _resolve_expr(enabled_from, context)
-    configmap_specs = _substitute_values(deepcopy(k8s.get("configMaps", {})), context)
 
     # ConfigMaps are module-scoped: one manifest, mounted by whichever
     # containers name it.
@@ -293,6 +355,7 @@ def _render_module(
                 pod_volumes,
                 pvcs,
                 is_init=True,
+                is_production=is_production,
             )
             secret_keys |= keys
             init_containers.append(container)
@@ -316,6 +379,7 @@ def _render_module(
                 pod_volumes,
                 pvcs,
                 is_init=False,
+                is_production=is_production,
             )
             secret_keys |= keys
             containers.append(container)
@@ -420,6 +484,7 @@ def _translate_container(
     pod_volumes: dict[str, dict[str, Any]],
     pvcs: dict[str, dict[str, Any]],
     is_init: bool,
+    is_production: bool = False,
 ) -> tuple[dict[str, Any], dict, dict, set[str]]:
     spec = _substitute_values(deepcopy(spec), context)
     spec.pop("enabledFrom", None)
@@ -483,6 +548,7 @@ def _translate_container(
         pvcs,
         overrides.get("dropVolumeMounts", []) or [],
         context=context,
+        is_production=is_production,
     )
     if mounts:
         container["volumeMounts"] = mounts
@@ -718,6 +784,7 @@ def _translate_volumes(
     pvcs: dict[str, dict[str, Any]],
     drop_mounts: list[str],
     context: dict[str, Any] | None = None,
+    is_production: bool = False,
 ) -> list[dict[str, Any]]:
     mounts: list[dict[str, Any]] = []
     dropped = {str(d).rstrip("/") for d in drop_mounts}
@@ -733,6 +800,9 @@ def _translate_volumes(
         mounts.append({"name": name, "mountPath": path})
 
     for entry in spec.get("volumes", []) or []:
+        if isinstance(entry, dict) and isinstance(entry.get("production"), bool):
+            if entry["production"] != is_production:
+                continue
         source, target, read_only, kind = _parse_volume_entry(entry)
         if target is None or target.rstrip("/") in dropped:
             continue

{composable_data_stack-0.12.0 → composable_data_stack-0.14.0}/cli/main.py

@@ -2,6 +2,8 @@
 from __future__ import annotations
 
 import argparse
+import contextlib
+import io
 import json
 import logging
 import os
@@ -48,6 +50,11 @@ from .planner import build_plan
 from .preflight import preflight_passed, run_preflight
 from .renderer import render_compose
 from .report import build_compliance_report
+from .secret_leak import (
+    CLI_OUTPUT_REDACTED_CODE,
+    RedactingStream,
+    resolved_secret_values,
+)
 from .security import PrecomputedRender, run_security_validation
 from .security_common import (
     COMPLIANCE_CATEGORIES,
@@ -1016,6 +1023,10 @@ def _write_env_file(
     )
     lines.append("")
     _atomic_write(output_path, "\n".join(lines))
+    # Holds secrets: keep it owner-only even if _atomic_write's temp-file
+    # creation ever changes (CDS-SEC-032). No-op semantics on Windows.
+    if os.name != "nt":
+        output_path.chmod(0o600)
 
 
 def _cds_version() -> str:
@@ -1320,7 +1331,7 @@ def _run_image_verification(
         return [_unverifiable_image_finding(f"Image verification failed unexpectedly: {e}")]
 
 
-def _render_helm_chart(plan, output_dir, force):
+def _render_helm_chart(plan, output_dir, force, env_file=None):
     """
     Render the Helm chart for a plan and report where it landed.
 
@@ -1341,7 +1352,7 @@ def _render_helm_chart(plan, output_dir, force):
                 "like a CDS-generated chart. Pass --force to overwrite it."
             )
             return 1, []
-    files, diags = render_helm(plan, output_dir=str(target_dir))
+    files, diags = render_helm(plan, output_dir=str(target_dir), env_file=env_file)
     return 0, diags
 
 
@@ -1366,10 +1377,74 @@ def _maybe_notify_update() -> None:
         _LOGGER.debug("Automatic update notice failed", exc_info=True)
 
 
+_REDACTED_OUTPUT_COMMANDS = frozenset({"validate", "plan", "test", "security"})
+_output_redaction: tuple[Any, Any, set[str]] | None = None
+
+
+def _install_output_redaction(args: argparse.Namespace) -> None:
+    """CDS-SEC-071: redact resolved secret values from this command's stdout/stderr.
+
+    Runs for commands that print plans, diagnostics or findings, unless the
+    user explicitly asked for unredacted output with --reveal-secrets.
+    Failing to work out the secret set must not leave output unprotected, so
+    it falls back to the secret-named CDS_* variables alone.
+    """
+    global _output_redaction
+    if args.command not in _REDACTED_OUTPUT_COMMANDS or getattr(args, "reveal_secrets", False):
+        return
+    declared: set[str] = set()
+    env_file = None
+    try:
+        # The command resolves the profile itself and prints any warning;
+        # don't print it a second time from this pre-pass.
+        with contextlib.redirect_stdout(io.StringIO()):
+            profile_path = resolve_profile_path(args.profile)
+            env_file = str(resolve_env_file_path(profile_path))
+            declared = _collect_profile_env_vars(profile_path, getattr(args, "environment", None))[1]
+    except Exception:  # noqa: BLE001 - redaction must never break the command itself
+        _LOGGER.debug("Could not collect declared secrets for output redaction", exc_info=True)
+    secret_values = resolved_secret_values(declared, env_file)
+    if not secret_values:
+        return
+    hits: set[str] = set()
+    _output_redaction = (sys.stdout, sys.stderr, hits)
+    sys.stdout = RedactingStream(sys.stdout, secret_values, hits)
+    sys.stderr = RedactingStream(sys.stderr, secret_values, hits)
+
+
+def _finish_output_redaction() -> None:
+    global _output_redaction
+    if _output_redaction is None:
+        return
+    original_stdout, original_stderr, hits = _output_redaction
+    _output_redaction = None
+    for stream in (sys.stdout, sys.stderr):
+        if isinstance(stream, RedactingStream):
+            stream.finish()
+        else:
+            stream.flush()
+    sys.stdout, sys.stderr = original_stdout, original_stderr
+    if hits:
+        print(
+            Diagnostic(
+                level="warning",
+                code=CLI_OUTPUT_REDACTED_CODE,
+                message=(
+                    "[CDS-SEC-071] Output contained the resolved value of "
+                    f"{', '.join(sorted(hits))}; it was redacted. Find where the literal "
+                    "value is being copied into config and use a secrets.<alias> reference instead."
+                ),
+                path="cli-output",
+            ).format(),
+            file=sys.stderr,
+        )
+
+
 def main() -> int:
     try:
         return _run()
     finally:
+        _finish_output_redaction()
         _maybe_notify_update()
 
 
@@ -1897,6 +1972,8 @@ def _run() -> int:
     if args.command in {"validate", "render", "up", "down", "test", "state", "security"}:
         args.target = getattr(args, "target", None) or load_saved_target() or "compose"
     
+    _install_output_redaction(args)
+
     if args.command == "validate":
         try:
             profile_path = resolve_profile_path(args.profile)
@@ -2158,7 +2235,7 @@ def _run() -> int:
 
             env_file = str(resolve_env_file_path(str(source_profile)))
             if args.target == "helm":
-                code, render_diags = _render_helm_chart(plan, output_path, args.force)
+                code, render_diags = _render_helm_chart(plan, output_path, args.force, env_file=env_file)
                 all_diags = render_diags
                 if code != 0:
                     _record_audit("render", str(source_profile), "failure", plan=plan)
@@ -2222,7 +2299,7 @@ def _run() -> int:
                 )
 
             if args.target == "helm":
-                code, render_diags = _render_helm_chart(plan, output_path, args.force)
+                code, render_diags = _render_helm_chart(plan, output_path, args.force, env_file=env_file)
                 all_diags = all_diags + render_diags
                 if code != 0:
                     _record_audit("render", profile_or_plan, "failure", environment=args.environment, plan=plan)
@@ -2302,7 +2379,7 @@ def _run() -> int:
         if args.target == "helm":
             project_root = resolve_project_root(profile_path)
             chart_dir = (Path(args.chart_dir) if args.chart_dir else project_root / "chart").resolve()
-            code, render_diags = _render_helm_chart(plan, chart_dir, force=True)
+            code, render_diags = _render_helm_chart(plan, chart_dir, force=True, env_file=env_file)
             all_diags.extend(render_diags)
             if code != 0 or has_errors(all_diags):
                 print_diagnostics(all_diags)