composable-data-stack 0.14.0__tar.gz → 0.15.1__tar.gz

{composable_data_stack-0.14.0/composable_data_stack.egg-info → composable_data_stack-0.15.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: composable-data-stack
-Version: 0.14.0
+Version: 0.15.1
 Summary: CLI for validating and composing composable data stack profiles
 Author: Ronald Hensbergen
 License-Expression: Apache-2.0
@@ -28,11 +28,12 @@ Provides-Extra: completion
 Requires-Dist: argcomplete<4,>=3.0.0; extra == "completion"
 Provides-Extra: test
 Requires-Dist: jinja2>=3.0; extra == "test"
+Requires-Dist: sqlalchemy>=2.0; extra == "test"
 Provides-Extra: dev
 Requires-Dist: coverage>=7.6; extra == "dev"
 Requires-Dist: bandit>=1.7; extra == "dev"
 Requires-Dist: pip-audit>=2.7; extra == "dev"
-Requires-Dist: ruff==0.17.1; extra == "dev"
+Requires-Dist: ruff==0.17.3; extra == "dev"
 Dynamic: license-file
 
 # 🚀 Composable Data Stack (CDS)
@@ -531,6 +532,8 @@ This runs `validate` → `plan` → `render` → `docker compose build` → `doc
 
 `docker compose build` output is streamed to the terminal (grouped by image) and also persisted to a log file (default: `.cds/logs/up-<profile>-<timestamp>.log`; override with `--log-file`). `docker compose up` always runs detached: in the default mode its output goes only to the log file while the terminal shows the live state view; with `--detach` it is streamed to the terminal like the build output. Once the stack starts, `cds up` polls `docker compose ps` and redraws the same grouped/colored view as `cds state` until every service is healthy/running/exited or a timeout is hit (default 180s; override with `--timeout`; disable colored labels with `--no-color`). Container logs from that point on go only to the log file, not the terminal, since the terminal is showing the live state view. `cds up` exits `0` once the stack settles, or `1` on timeout or an unhealthy service.
 
+The rendered `docker-compose.yml` is a standard Compose file, so the stack also runs without `cds` — `docker compose -f docker-compose.yml up -d` works on its own from the project root. See [docs/underlying-commands.md](docs/underlying-commands.md) for the full mapping of every `cds` lifecycle command (`render` / `up` / `state` / `down`, Compose and Helm) to its underlying `docker compose` / `helm` / `kubectl` commands.
+
 Add `--detach` (or `-d`) to skip the live state view and return as soon as the stack starts, e.g. for scripting or CI:
 
 ```bash
@@ -817,6 +820,7 @@ Common errors from `cds validate`, `cds plan`, and `cds render`, and how to fix
 | `[E042] ... Contract kind mismatch` | The consumer expects one contract kind (e.g. `sql-database`) but the producer exposes a different kind. | Point the binding at a module that provides the expected contract kind, or update the consumer's expected kind if the mismatch is intentional. |
 | `[E043] ... recorded as unsupported in the compatibility registry` | The binding's `kind` matches, but this exact provider/consumer pairing is explicitly recorded as `unsupported` in `cli/resources/compatibility-registry.json` -- it's known NOT to work together despite matching shape. | Use a different provider/consumer pairing for this contract, or if the pairing has since been fixed/verified, update its `status` to `tested` in the registry (see [docs/architecture.md § Secrets and contract resolution](docs/architecture.md#secrets-and-contract-resolution)). |
 | `[E128] ... requires a TLS-protected endpoint (tls: required), but ... does not advertise one` | A module's `consumes` entry declares `tls: required` (issue #577), but the bound provider's contract spec does not set `tls: true` (absent, `false`, or a `${config.<path>}` reference that does not resolve to `true` against the producer's profile config). | Point the binding at a provider that advertises TLS, or have the provider set `tls: true` in its provide entry's contract spec. Drop `tls: required` only if the binding is genuinely allowed to run unencrypted. |
+| `[E154] ... expects database engine ... but contract ref ... advertises engine ...` / `... advertises no "engine" in its spec` | A module's `consumes` entry declares `engineFrom: spec.config.<path>` (issue #682), but the bound provider's `sql-database` contract advertises a different `engine`, or none at all -- the consumer would speak the wrong wire protocol to the bound database (e.g. Keycloak's `KC_DB` vs the bound provider). The expected engine is read from the profile config at that path, falling back to the `configSchema` default when unset. | Set the consumer's config field (e.g. keycloak `metadataDatabase.driver`) to the provider's advertised engine, bind a provider advertising the expected engine, or have the provider set `engine` in its provide entry's contract spec. Drop `engineFrom` only if the consumer genuinely accepts any engine. |
 
 | `[E130] ... Duplicate backup policy for workload class "<class>"` | `spec.backup.policies` declares more than one policy for the same workload class. | Keep one policy per workload class. |
 | `[E131] ... Backup encryption key ref ... is not defined in spec.secrets.values` | `spec.backup.encryption.keyFrom` references a `secrets.<alias>` that the profile does not declare. | Declare the alias under `spec.secrets.values` with its `env` name, or fix the reference. |
@@ -824,13 +828,39 @@ Common errors from `cds validate`, `cds plan`, and `cds render`, and how to fix
 | `[E133] ... Production profiles must declare spec.backup ...` / `... missing workload classes: ...` | A `metadata.environment: production` profile has no `spec.backup` block, or lacks a policy for a workload class it runs (warehouse, orchestration/BI/identity/secrets metadata, or configuration). | Declare encrypted, restore-verified backup policies for every workload class the profile runs; see `docs/backup-restore.md`. |
 | `[E134] ... Production backups must be encrypted ...` / `... must be declared required: true` | A production profile has no `spec.backup.encryption.keyFrom`, or the referenced secret alias is declared `required: false` so the key can be missing at runtime. | Declare the encryption key alias with `required: true` so a missing key fails closed at startup. |
 | `[E135] ... Production backup policy "<class>" must declare restoreVerification` | A production backup policy has no `restoreVerification` block. | Add `restoreVerification.frequencyDays` to every production backup policy.|
-| `[E136] ... Reverse-proxy route host ...` (unknown target / no resolvable http-service endpoint / backend address not a rendered compose service / entrypoint not declared by the proxy / undeclared certificate alias / duplicate route host) | A `config.routes` entry on a reverse-proxy provider names a module that is missing, does not provide an `http-service` contract with host/port (or a port outside 1-65535), advertises a backend address that matches no rendered compose service (multi-service modules must advertise `serviceName`), names a `<module-id>.<contract-name>` contract the target does not provide, names an entrypoint the proxy's `config.entryPoints` does not declare, references an undeclared certificate alias, or duplicates the host of an earlier route on the same proxy (#581). | Fix the route's `target`/`certificate`/`host`/`entryPoints` (a bare module id routes to its first http-service contract; `<module-id>.<contract-name>` picks a specific one; entrypoints must match the proxy's declared `config.entryPoints` keys), or advertise the serving compose service via `serviceName` in the target's http-service provide entry. |
+| `[E136] ... Reverse-proxy route ...` (malformed or unknown contractRef / no resolvable http-service endpoint / backend address not a rendered compose service / entrypoint not declared by the proxy / undeclared certificate alias / route with neither host nor pathPrefix / duplicate route rule) | A `config.routes` entry on a reverse-proxy provider has a `contractRef` that is not `<module-id>.<contract-name>` or names a module that is missing, or that module does not provide the named `http-service` contract with host/port (or a port outside 1-65535), advertises a backend address that matches no rendered compose service (multi-service modules must advertise `serviceName`), names an entrypoint the proxy's `config.entryPoints` does not declare, references an undeclared certificate alias, names neither `host` nor `pathPrefix`, or duplicates an earlier route's rule on the same proxy (#581, #883). | Fix the route's `contractRef`/`certificate`/`host`/`pathPrefix`/`entryPoints` (entrypoints must match the proxy's declared `config.entryPoints` keys), or advertise the serving compose service via `serviceName` in the target's http-service provide entry. |
+| `[E150] ... Provided contract ... advertises network host ...` | A module's provided `http-service`, `sql-database`, `cache-service`, `object-storage`, `lineage-sink`, `log-sink`, `metrics-sink`, or `trace-sink` contract advertises a network host (the `host` field, the `endpoint`'s or `connectionUri`'s hostname part, or a `serviceName` mapped through the renderer's service-naming rule; a scheme-less `host:port` value contributes its host part, #1036) that matches no service the render produced for that module -- no compose service on the compose target, no rendered Kubernetes Service on the helm target -- so consumers would resolve a hostname nothing answers (#1010, #1029, #1035). | Compose: instantiate the module with an id matching its compose service name, advertise the serving service via a `serviceName` field in the provide entry, or build the endpoint from `${service.dns}` or `${service.services.<name>}`. Helm: render the serving workload with a `service` block (or as a `StatefulSet`, which gets a governing Service), advertise the serving workload via a `serviceName` field, or build the endpoint from `${service.dns}`. |
+| `[E151] ... Raw manifest ... must be an object with apiVersion, kind, and metadata.name` / `... must declare non-empty string apiVersion, kind, and metadata.name` | A raw manifest under a module's `implementation.kubernetes.manifests` is malformed -- a plan loaded from a file bypasses the module schema, so the Helm renderer fails closed instead of shipping a broken manifest (issue #767). | Fix the manifest shape in the module (or the saved plan): every entry needs string `apiVersion`/`kind` and a `metadata.name`. |
 | `[E103] ... config.image.tag is required ... when config.image.source is "registry"` | A module instance sets `config.image.source: registry` without also setting `config.image.tag`. | Set `config.image.tag` to a version, or switch back to `config.image.source: build`. |
 | `[E139] ... without an explicit host authentication advertisement` / `... host authentication method "trust"` | A `metadata.environment: production` profile runs a `sql-database` provider whose contract does not advertise a host authentication method, or advertises `trust` (issue #206). | Configure the provider's authentication explicitly (e.g. postgres `config.authentication.hostAuth: scram-sha-256`); trust authentication is prohibited in production profiles. |
+| `[E140] ... configures webServer.mode "development"` | A production profile sets an HTTP-serving module's `config.webServer.mode` to `development`, selecting its built-in development web server (issue #216). | Set `config.webServer.mode` to `production` (or remove the override) so the service runs behind its production process manager; see `docs/serving-modes.md`. |
 | `[E137] ... accepts unauthenticated clients` | A production profile runs a `cache-service` provider that does not advertise `authentication: required` (no password credential configured). | Supply a cache password through a `secrets.<alias>` reference (e.g. keydb `config.passwordFrom`), declared `required: true` in `spec.secrets.values`. |
 | `[E138] ... Credential field ... must reference a secret alias` / `... is empty` / `... uses a ${CDS_*} profile input` / `... contains a plaintext value` | A credential-named config field of a production database/cache provider is not a `secrets.<alias>` reference, or its alias is not declared `required: true` (issue #206). | Supply the credential through a secret reference declared `required: true` so a missing value fails closed at startup. |
 | `[W105] ... legacy md5 host authentication` | A production `sql-database` provider advertises `md5`, which validates but is a deprecated password method. | Prefer `scram-sha-256` for production profiles. |
+| `[E148] ... RBAC rule ... must be a mapping with non-empty "apiGroups", "resources", and "verbs" lists` / `... The "serviceAccount" block ... must be a mapping ...` | A module's `implementation.kubernetes.serviceAccount` block (or one of its `rbac` rules) is malformed -- a plan loaded from a file bypasses the module schema, so the Helm renderer fails closed instead of shipping a broken Role (#75). | Fix the `serviceAccount`/`rbac` shape in the module (or the saved plan). |
+| `[E149] ... The "networkPolicy"/... block ... must be a mapping with optional "ingress"/"egress" lists ...` | A module's custom NetworkPolicy allow rules are malformed (plan-file bypass of the module schema) (#75). | Fix the `networkPolicy` shape; each rule must be a mapping. |
+| `[E152] ... waitFor gate of workload "<module>/<workload>" names host "<host>", which the module's default-deny NetworkPolicy does not allow egress to ...` | On the Helm target, a `waitFor` gate names a host that is neither a Service of the same module emitted by this render (a workload disabled via `enabledFrom` gets no Service), the endpoint of a provider the module consumes, nor the target of one of the module's reverse-proxy routes, and no custom `networkPolicy.egress` ipBlock rule or declared `implementation.kubernetes.egress` entry covers the host:port, so the default-deny NetworkPolicy blocks the init container's connection and it would only hang until its timeout (#1031, #1033, #1023). | Bind the host's module through a `consumes` contract, route to it via `config.routes`, gate on a same-module `${k8s.service.*}` Service, declare the destination under `implementation.kubernetes.egress` (an IP/CIDR with the gate's port), or add a custom `networkPolicy.egress` ipBlock rule covering the host and the gate's port (a DNS-name host cannot be expressed in a NetworkPolicy and stays denied). |
+| `[E153] ... External egress entry of module \"<module>\" at ... is invalid ...` | A declared external egress entry under `implementation.kubernetes.egress` (or `workloads.<name>.egress`) is malformed -- plan-file bypass of the module schema (#1023): exactly one of `host` (IP literal; DNS names are rejected) / `cidr` (network form, no host bits), `except` CIDRs inside the `cidr`, `port`/`ports` within 1-65535, `protocol` TCP/UDP/SCTP, and all `${config.*}` expressions must resolve. | Fix the entry shape in the module (or the saved plan); see `docs/kubernetes.md` ("Declaring external egress"). |
+| `[W128] ... carries no numeric port and provider module ... advertises no service ports ...` | A contract binding carries no port evidence, so its generated NetworkPolicy egress rule allows all ports to the provider (#75). | Declare the port in the contract so the rule stays port-scoped. |
+| `[W129] ... Reverse-proxy route ... cannot be derived as a NetworkPolicy edge ...` | On the Helm target, a `config.routes` entry of a reverse-proxy module has a `contractRef` naming an unknown module, a module without the named `http-service` provide, or a target whose http-service contract carries no usable port evidence (absent, or outside 1-65535, and the target advertises no service ports), so no port-scoped NetworkPolicy edge can be derived and the proxy's default-deny policy keeps the target unreachable (#1033, #883). | Fix the route's `contractRef` (`<module-id>.<contract-name>` naming an existing http-service provide) and declare a valid `port` in that contract, or add a custom `networkPolicy.egress` rule for the target. |
+| `[W130] ... renders resources for operator "<name>" (CRD <crd>), which CDS does not install itself ...` | On the Helm target, a module declares an out-of-band operator under `implementation.kubernetes.operators` (e.g. the CloudNativePG operator of `modules-experimental/warehouse/postgres-operator`): CDS renders the CRs but never installs the operator, so the chart only reconciles once the operator is on the cluster (#768). The warning names the install command; the same note is in the chart's `NOTES.txt`. | Run the install command from the warning (e.g. `helm repo add cnpg https://cloudnative-pg.github.io/charts && helm upgrade --install cnpg --namespace cnpg-system --create-namespace cnpg/cloudnative-pg`), verify with `kubectl get crd clusters.postgresql.cnpg.io`, then re-run; see `docs/kubernetes.md` ("Operator-managed modules"). |
+| `[W131] ... provides open-data-provider contract ... with classification ... which is outside the recommended DCAT-AP-NL data-theme vocabulary ...` | A module provides an `open-data-provider` contract whose `classification` is free text outside the recommended standard vocabulary (the DCAT-AP-NL data themes: AGRI, ECON, EDUC, ENER, ENVI, GEOJ, GOVE, HEAH, INTR, JUST, REGI, SOCI, TRAN, or the full EU data-theme authority URI), so the dataset is harder to compare and find across domains (issue #845). The value still renders; this is a warning only. | Set `classification` to a theme from the DCAT-AP-NL data-theme vocabulary (e.g. in the postgres module's `config.dataProvider.classification`); see `shared/contracts/open-data-provider.yaml` and `docs/fds-alignment.md`. |
+| `[E155] ... needs operator "<name>", but its CRD "<crd>" is not present on the cluster ...` | `cds up --target helm` checked the cluster (`kubectl get crd <crd>`) and the operator a module depends on is confirmed absent, so `helm upgrade --install` would fail with "no matches for kind" (#768). | Install the operator with the command named in the error (e.g. `helm upgrade --install cnpg --namespace cnpg-system --create-namespace cnpg/cloudnative-pg` after `helm repo add cnpg https://cloudnative-pg.github.io/charts`), then re-run `cds up --target helm`. |
+| `[E156] ... forEach marker of manifest ... resolved ... to nothing / to a <type>, but fan-out needs a list / produced the duplicate metadata.name ...` | A raw manifest under `implementation.kubernetes.manifests` declares `forEach: <expr>`, but the expression does not resolve to a list in the plan, or two fanned-out copies resolve to the same `metadata.name` (they would collide in the release namespace and in the chart's template file names) (issue #1170). | Point `forEach` at a list-valued config path (e.g. `config.services`), and reference `${item...}` in the manifest's `metadata.name` so every fanned-out copy is uniquely named. |
+| `[E157] ... Duplicate datasetId ...` | `spec.dataSharing.datasets` declares the same `datasetId` more than once; each disclosed dataset must be declared once (issue #844). | Remove the duplicate entry, or give each entry a distinct `datasetId`. |
+| `[E158] ... Dataset ... names module ... which is not an enabled module instance ...` | A `spec.dataSharing.datasets` entry's `module` names a module id that is not an enabled module instance in the profile (issue #844). | Point `module` at an existing, enabled module instance id from `spec.modules`, or drop the field. |
+| `[E159] ... enrolls service "<name>" in its service-mesh config (config.services[<i>].name), but no module instance with that id exists in the profile ...` | At plan time, a module providing the `service-mesh` contract has a `config.services[].name` that matches no module instance id of the profile (a typo such as `warehose` for `warehouse`, or a disabled module) (issue #1179). The rendered per-service `AuthorizationPolicy` would select zero pods via the `cds.dev/module` label -- a silent no-op allow policy. | Fix the enrollment to name an existing module instance id (the error lists the planned ids), add the module with that id, or remove the entry from `config.services`; enable a disabled module you want to enroll. |
+| `[E161] ... Unknown module id '<id>' ...` / `... addresses/traverses a non-scalar value ...` / `... matches a resolved secret value ...` | A persisted `cds config set module.<id>.config.<path> <value>` override in `.cds/config.json` or the shared `cds.shared.json` (issue #1220) names a module id that is not in the resolved profile, points at a dict/list instead of a scalar leaf, or tries to store a resolved secret value. The same code is emitted at load time when a hand-edited `config.json` breaks, so `validate`/`plan`/`render` fail loudly instead of silently ignoring the override. | Fix the id or path (only `module.<id>.config.<scalar leaf>` keys are accepted), remove the override with `cds config unset [--shared] module.<id>.config.<path>`, or store a `secrets.<alias>` reference instead of a secret value; see [docs/cli-reference.md](docs/cli-reference.md#local-module-config-overrides-moduleidconfigpath). |
+| `[W132] ... enrolls service "<name>" in its service-mesh config, but the module renders no Kubernetes workload pods ...` | On the Helm target, a known enrollment names a module with no rendered workloads, such as a manifest-only module or one whose workloads are all disabled by `enabledFrom` (issue #1198). If enabled, the per-service `AuthorizationPolicy` would select zero pods via the `cds.dev/module` label. | Enroll a module that renders at least one Kubernetes workload, or enable one of its gated workloads; remove the enrollment if no pod workload is intended. |
 | `[W097] ... config.image.tag is "latest" with config.image.source "registry"` | `config.image.tag: latest` under `source: registry` still validates, but drifts silently between deploys instead of pinning a reproducible version. | Pin an explicit tag from `publish-images.yml`'s output (or `tests/fixtures/signed-images.json`) instead of `latest`. |
+| `[E127] ... Unknown confinement key ...` / `... must be 'default', 'unconfined' or 'file:<path>' ...` / `... Invalid SELinux label(s) ...` | A `security.runtime.confinement` block (or a module instance's confinement) is malformed: an unsupported key, an invalid `seccomp`/`apparmor`/`selinux` value, or a `file:` seccomp profile path that is missing or resolves outside the profile directory. | Use `seccomp: default`/`unconfined`/`file:<relative path inside the profile directory>`, `apparmor: runtime/default`/`unconfined`/`<host profile name>`, and `selinux: disable` or a `label: value` list; see `docs/runtime-confinement.md`. |
+| `[W071] ... has kind "docker-compose" but no "compose" definition` | A module implementation declares the `docker-compose` kind without an `implementation.compose` block, so the renderer skips the module. | Add the `implementation.compose` block (services, volumes) to the module's `module.yaml`, or set a supported implementation `kind`. |
+| `[W072] ... Container "<name>" in Kubernetes workload "<module>/<workload>" has no resource requests or limits` | Rendering the Helm target: a container or init container listed in a Kubernetes workload has no matching entry under the workload's `resources`. | Add a `resources.<container>` entry with `requests` and `limits` to the workload in the module's Kubernetes implementation (see `docs/kubernetes.md`). |
+| `[W090] ... Invalid .env line format: "<line>" (expected KEY=VALUE)` | A line in the `.env` file is neither blank, a `#` comment, nor in `KEY=VALUE` form, so it is ignored. | Fix the line to `KEY=VALUE` form, comment it out with `#`, or remove it. |
+| `[W098] ... Applied plaintext endpoint exposure waiver for production profile (modules: ...)` | A production profile's `spec.security.waivers.plaintextEndpointExposure` waived the CDS-SEC-074 plaintext-endpoint findings; CDS reports the waiver so it stays visible in security output. | Confirm the waiver is still justified and its `reason` is accurate; see `docs/security-rules.md` (Warnings). |
+| `[W099] ... plaintextEndpointExposure is set but has no effect for profile class ...` | `spec.security.waivers.plaintextEndpointExposure` is declared on a non-prod profile, where CDS-SEC-074 and its waiver only apply to prod-class profiles. | Remove the waiver from the non-prod profile, or move it to the prod profile (overlay) it is meant for. |
+| `[W103] ... Applied runtime policy waiver for production profile (checks: ...)` | A production profile's `spec.security.waivers.runtimePolicy` suppressed CDS-SEC-090 violations; CDS reports the waiver so it stays visible in security output. | Confirm the waiver is still justified and its `reason` is accurate; see `docs/security-rules.md` (Warnings). |
+| `[W104] ... runtimePolicy is set but has no effect for profile class ...` | `spec.security.waivers.runtimePolicy` is declared on a non-prod profile, where CDS-SEC-090 and its waiver only apply to prod-class profiles. | Remove the waiver from the non-prod profile, or move it to the prod profile (overlay) it is meant for. |
 
 All diagnostics print with their error code and YAML path (e.g. `spec.modules[1].config`), so search the profile file for that path to find the exact line to fix.
 
@@ -971,6 +1001,8 @@ Please read these first:
 - [CONTRIBUTING.md](CONTRIBUTING.md)
 - [docs/maintainer-merge-policy.md](docs/maintainer-merge-policy.md)
 - [SECURITY.md](SECURITY.md) — how to report a vulnerability
+- [ACCESSIBILITY.md](ACCESSIBILITY.md) — our accessibility commitment and how
+  to report a barrier
 - [LEGAL.md](LEGAL.md) — license, code of conduct, security, support, and
   regulatory-compliance documentation index
 - [CHANGELOG.md](CHANGELOG.md)
@@ -985,11 +1017,15 @@ Development helper tools are located in the gitignored `tools/` directory. See
 
 - [Quickstart](README.md#-quickstart) — get running in 5 minutes
 - [CLI Reference](docs/cli-reference.md) — full command/flag table and `[profile]` resolution rules
+- [Underlying Commands](docs/underlying-commands.md) — the exact `docker compose` / `helm` / `kubectl` commands behind `cds render` / `up` / `state` / `down`, for running the stack without the `cds` CLI
 - [How CDS Works](docs/how-it-works.md) — C4 + sequence diagrams of the validate → plan → render engine
-- [Kubernetes Target](docs/kubernetes.md): Helm rendering and isolated local k3s workflow
+- [Kubernetes Target](docs/kubernetes.md): Helm rendering, Docker Compose migration guide, and isolated local k3s workflow — with ready-made example profiles (`profiles/local-kubernetes-example`, `profiles/production-kubernetes-example` with its `prod` overlay)
 - [From Docker Compose to CDS Profile](docs/from-docker-to-cds-profile.md) — complete transformation guide
 - [Architecture](docs/architecture.md) — design and core concepts
 - [Modules](docs/modules.md) — how to structure reusable components
+- [Repository Layout](docs/repo-layout.md) — best practice for keeping platform assets (modules, profiles) separate from project code (Dagster pipelines, dbt/dlt, database scripts)
+- [Workload Runtime API](docs/runtime-api.md) — `cds-runtime`: named database connections for Python workloads
+- [Production Hardening Checklist](docs/production-hardening-checklist.md) — control-to-owner mapping and the operator verification procedure for a production profile
 - [Roadmap](docs/roadmap.md) — planned features and milestones
 
 ---

{composable_data_stack-0.14.0 → composable_data_stack-0.15.1}/README.md

@@ -494,6 +494,8 @@ This runs `validate` → `plan` → `render` → `docker compose build` → `doc
 
 `docker compose build` output is streamed to the terminal (grouped by image) and also persisted to a log file (default: `.cds/logs/up-<profile>-<timestamp>.log`; override with `--log-file`). `docker compose up` always runs detached: in the default mode its output goes only to the log file while the terminal shows the live state view; with `--detach` it is streamed to the terminal like the build output. Once the stack starts, `cds up` polls `docker compose ps` and redraws the same grouped/colored view as `cds state` until every service is healthy/running/exited or a timeout is hit (default 180s; override with `--timeout`; disable colored labels with `--no-color`). Container logs from that point on go only to the log file, not the terminal, since the terminal is showing the live state view. `cds up` exits `0` once the stack settles, or `1` on timeout or an unhealthy service.
 
+The rendered `docker-compose.yml` is a standard Compose file, so the stack also runs without `cds` — `docker compose -f docker-compose.yml up -d` works on its own from the project root. See [docs/underlying-commands.md](docs/underlying-commands.md) for the full mapping of every `cds` lifecycle command (`render` / `up` / `state` / `down`, Compose and Helm) to its underlying `docker compose` / `helm` / `kubectl` commands.
+
 Add `--detach` (or `-d`) to skip the live state view and return as soon as the stack starts, e.g. for scripting or CI:
 
 ```bash
@@ -780,6 +782,7 @@ Common errors from `cds validate`, `cds plan`, and `cds render`, and how to fix
 | `[E042] ... Contract kind mismatch` | The consumer expects one contract kind (e.g. `sql-database`) but the producer exposes a different kind. | Point the binding at a module that provides the expected contract kind, or update the consumer's expected kind if the mismatch is intentional. |
 | `[E043] ... recorded as unsupported in the compatibility registry` | The binding's `kind` matches, but this exact provider/consumer pairing is explicitly recorded as `unsupported` in `cli/resources/compatibility-registry.json` -- it's known NOT to work together despite matching shape. | Use a different provider/consumer pairing for this contract, or if the pairing has since been fixed/verified, update its `status` to `tested` in the registry (see [docs/architecture.md § Secrets and contract resolution](docs/architecture.md#secrets-and-contract-resolution)). |
 | `[E128] ... requires a TLS-protected endpoint (tls: required), but ... does not advertise one` | A module's `consumes` entry declares `tls: required` (issue #577), but the bound provider's contract spec does not set `tls: true` (absent, `false`, or a `${config.<path>}` reference that does not resolve to `true` against the producer's profile config). | Point the binding at a provider that advertises TLS, or have the provider set `tls: true` in its provide entry's contract spec. Drop `tls: required` only if the binding is genuinely allowed to run unencrypted. |
+| `[E154] ... expects database engine ... but contract ref ... advertises engine ...` / `... advertises no "engine" in its spec` | A module's `consumes` entry declares `engineFrom: spec.config.<path>` (issue #682), but the bound provider's `sql-database` contract advertises a different `engine`, or none at all -- the consumer would speak the wrong wire protocol to the bound database (e.g. Keycloak's `KC_DB` vs the bound provider). The expected engine is read from the profile config at that path, falling back to the `configSchema` default when unset. | Set the consumer's config field (e.g. keycloak `metadataDatabase.driver`) to the provider's advertised engine, bind a provider advertising the expected engine, or have the provider set `engine` in its provide entry's contract spec. Drop `engineFrom` only if the consumer genuinely accepts any engine. |
 
 | `[E130] ... Duplicate backup policy for workload class "<class>"` | `spec.backup.policies` declares more than one policy for the same workload class. | Keep one policy per workload class. |
 | `[E131] ... Backup encryption key ref ... is not defined in spec.secrets.values` | `spec.backup.encryption.keyFrom` references a `secrets.<alias>` that the profile does not declare. | Declare the alias under `spec.secrets.values` with its `env` name, or fix the reference. |
@@ -787,13 +790,39 @@ Common errors from `cds validate`, `cds plan`, and `cds render`, and how to fix
 | `[E133] ... Production profiles must declare spec.backup ...` / `... missing workload classes: ...` | A `metadata.environment: production` profile has no `spec.backup` block, or lacks a policy for a workload class it runs (warehouse, orchestration/BI/identity/secrets metadata, or configuration). | Declare encrypted, restore-verified backup policies for every workload class the profile runs; see `docs/backup-restore.md`. |
 | `[E134] ... Production backups must be encrypted ...` / `... must be declared required: true` | A production profile has no `spec.backup.encryption.keyFrom`, or the referenced secret alias is declared `required: false` so the key can be missing at runtime. | Declare the encryption key alias with `required: true` so a missing key fails closed at startup. |
 | `[E135] ... Production backup policy "<class>" must declare restoreVerification` | A production backup policy has no `restoreVerification` block. | Add `restoreVerification.frequencyDays` to every production backup policy.|
-| `[E136] ... Reverse-proxy route host ...` (unknown target / no resolvable http-service endpoint / backend address not a rendered compose service / entrypoint not declared by the proxy / undeclared certificate alias / duplicate route host) | A `config.routes` entry on a reverse-proxy provider names a module that is missing, does not provide an `http-service` contract with host/port (or a port outside 1-65535), advertises a backend address that matches no rendered compose service (multi-service modules must advertise `serviceName`), names a `<module-id>.<contract-name>` contract the target does not provide, names an entrypoint the proxy's `config.entryPoints` does not declare, references an undeclared certificate alias, or duplicates the host of an earlier route on the same proxy (#581). | Fix the route's `target`/`certificate`/`host`/`entryPoints` (a bare module id routes to its first http-service contract; `<module-id>.<contract-name>` picks a specific one; entrypoints must match the proxy's declared `config.entryPoints` keys), or advertise the serving compose service via `serviceName` in the target's http-service provide entry. |
+| `[E136] ... Reverse-proxy route ...` (malformed or unknown contractRef / no resolvable http-service endpoint / backend address not a rendered compose service / entrypoint not declared by the proxy / undeclared certificate alias / route with neither host nor pathPrefix / duplicate route rule) | A `config.routes` entry on a reverse-proxy provider has a `contractRef` that is not `<module-id>.<contract-name>` or names a module that is missing, or that module does not provide the named `http-service` contract with host/port (or a port outside 1-65535), advertises a backend address that matches no rendered compose service (multi-service modules must advertise `serviceName`), names an entrypoint the proxy's `config.entryPoints` does not declare, references an undeclared certificate alias, names neither `host` nor `pathPrefix`, or duplicates an earlier route's rule on the same proxy (#581, #883). | Fix the route's `contractRef`/`certificate`/`host`/`pathPrefix`/`entryPoints` (entrypoints must match the proxy's declared `config.entryPoints` keys), or advertise the serving compose service via `serviceName` in the target's http-service provide entry. |
+| `[E150] ... Provided contract ... advertises network host ...` | A module's provided `http-service`, `sql-database`, `cache-service`, `object-storage`, `lineage-sink`, `log-sink`, `metrics-sink`, or `trace-sink` contract advertises a network host (the `host` field, the `endpoint`'s or `connectionUri`'s hostname part, or a `serviceName` mapped through the renderer's service-naming rule; a scheme-less `host:port` value contributes its host part, #1036) that matches no service the render produced for that module -- no compose service on the compose target, no rendered Kubernetes Service on the helm target -- so consumers would resolve a hostname nothing answers (#1010, #1029, #1035). | Compose: instantiate the module with an id matching its compose service name, advertise the serving service via a `serviceName` field in the provide entry, or build the endpoint from `${service.dns}` or `${service.services.<name>}`. Helm: render the serving workload with a `service` block (or as a `StatefulSet`, which gets a governing Service), advertise the serving workload via a `serviceName` field, or build the endpoint from `${service.dns}`. |
+| `[E151] ... Raw manifest ... must be an object with apiVersion, kind, and metadata.name` / `... must declare non-empty string apiVersion, kind, and metadata.name` | A raw manifest under a module's `implementation.kubernetes.manifests` is malformed -- a plan loaded from a file bypasses the module schema, so the Helm renderer fails closed instead of shipping a broken manifest (issue #767). | Fix the manifest shape in the module (or the saved plan): every entry needs string `apiVersion`/`kind` and a `metadata.name`. |
 | `[E103] ... config.image.tag is required ... when config.image.source is "registry"` | A module instance sets `config.image.source: registry` without also setting `config.image.tag`. | Set `config.image.tag` to a version, or switch back to `config.image.source: build`. |
 | `[E139] ... without an explicit host authentication advertisement` / `... host authentication method "trust"` | A `metadata.environment: production` profile runs a `sql-database` provider whose contract does not advertise a host authentication method, or advertises `trust` (issue #206). | Configure the provider's authentication explicitly (e.g. postgres `config.authentication.hostAuth: scram-sha-256`); trust authentication is prohibited in production profiles. |
+| `[E140] ... configures webServer.mode "development"` | A production profile sets an HTTP-serving module's `config.webServer.mode` to `development`, selecting its built-in development web server (issue #216). | Set `config.webServer.mode` to `production` (or remove the override) so the service runs behind its production process manager; see `docs/serving-modes.md`. |
 | `[E137] ... accepts unauthenticated clients` | A production profile runs a `cache-service` provider that does not advertise `authentication: required` (no password credential configured). | Supply a cache password through a `secrets.<alias>` reference (e.g. keydb `config.passwordFrom`), declared `required: true` in `spec.secrets.values`. |
 | `[E138] ... Credential field ... must reference a secret alias` / `... is empty` / `... uses a ${CDS_*} profile input` / `... contains a plaintext value` | A credential-named config field of a production database/cache provider is not a `secrets.<alias>` reference, or its alias is not declared `required: true` (issue #206). | Supply the credential through a secret reference declared `required: true` so a missing value fails closed at startup. |
 | `[W105] ... legacy md5 host authentication` | A production `sql-database` provider advertises `md5`, which validates but is a deprecated password method. | Prefer `scram-sha-256` for production profiles. |
+| `[E148] ... RBAC rule ... must be a mapping with non-empty "apiGroups", "resources", and "verbs" lists` / `... The "serviceAccount" block ... must be a mapping ...` | A module's `implementation.kubernetes.serviceAccount` block (or one of its `rbac` rules) is malformed -- a plan loaded from a file bypasses the module schema, so the Helm renderer fails closed instead of shipping a broken Role (#75). | Fix the `serviceAccount`/`rbac` shape in the module (or the saved plan). |
+| `[E149] ... The "networkPolicy"/... block ... must be a mapping with optional "ingress"/"egress" lists ...` | A module's custom NetworkPolicy allow rules are malformed (plan-file bypass of the module schema) (#75). | Fix the `networkPolicy` shape; each rule must be a mapping. |
+| `[E152] ... waitFor gate of workload "<module>/<workload>" names host "<host>", which the module's default-deny NetworkPolicy does not allow egress to ...` | On the Helm target, a `waitFor` gate names a host that is neither a Service of the same module emitted by this render (a workload disabled via `enabledFrom` gets no Service), the endpoint of a provider the module consumes, nor the target of one of the module's reverse-proxy routes, and no custom `networkPolicy.egress` ipBlock rule or declared `implementation.kubernetes.egress` entry covers the host:port, so the default-deny NetworkPolicy blocks the init container's connection and it would only hang until its timeout (#1031, #1033, #1023). | Bind the host's module through a `consumes` contract, route to it via `config.routes`, gate on a same-module `${k8s.service.*}` Service, declare the destination under `implementation.kubernetes.egress` (an IP/CIDR with the gate's port), or add a custom `networkPolicy.egress` ipBlock rule covering the host and the gate's port (a DNS-name host cannot be expressed in a NetworkPolicy and stays denied). |
+| `[E153] ... External egress entry of module \"<module>\" at ... is invalid ...` | A declared external egress entry under `implementation.kubernetes.egress` (or `workloads.<name>.egress`) is malformed -- plan-file bypass of the module schema (#1023): exactly one of `host` (IP literal; DNS names are rejected) / `cidr` (network form, no host bits), `except` CIDRs inside the `cidr`, `port`/`ports` within 1-65535, `protocol` TCP/UDP/SCTP, and all `${config.*}` expressions must resolve. | Fix the entry shape in the module (or the saved plan); see `docs/kubernetes.md` ("Declaring external egress"). |
+| `[W128] ... carries no numeric port and provider module ... advertises no service ports ...` | A contract binding carries no port evidence, so its generated NetworkPolicy egress rule allows all ports to the provider (#75). | Declare the port in the contract so the rule stays port-scoped. |
+| `[W129] ... Reverse-proxy route ... cannot be derived as a NetworkPolicy edge ...` | On the Helm target, a `config.routes` entry of a reverse-proxy module has a `contractRef` naming an unknown module, a module without the named `http-service` provide, or a target whose http-service contract carries no usable port evidence (absent, or outside 1-65535, and the target advertises no service ports), so no port-scoped NetworkPolicy edge can be derived and the proxy's default-deny policy keeps the target unreachable (#1033, #883). | Fix the route's `contractRef` (`<module-id>.<contract-name>` naming an existing http-service provide) and declare a valid `port` in that contract, or add a custom `networkPolicy.egress` rule for the target. |
+| `[W130] ... renders resources for operator "<name>" (CRD <crd>), which CDS does not install itself ...` | On the Helm target, a module declares an out-of-band operator under `implementation.kubernetes.operators` (e.g. the CloudNativePG operator of `modules-experimental/warehouse/postgres-operator`): CDS renders the CRs but never installs the operator, so the chart only reconciles once the operator is on the cluster (#768). The warning names the install command; the same note is in the chart's `NOTES.txt`. | Run the install command from the warning (e.g. `helm repo add cnpg https://cloudnative-pg.github.io/charts && helm upgrade --install cnpg --namespace cnpg-system --create-namespace cnpg/cloudnative-pg`), verify with `kubectl get crd clusters.postgresql.cnpg.io`, then re-run; see `docs/kubernetes.md` ("Operator-managed modules"). |
+| `[W131] ... provides open-data-provider contract ... with classification ... which is outside the recommended DCAT-AP-NL data-theme vocabulary ...` | A module provides an `open-data-provider` contract whose `classification` is free text outside the recommended standard vocabulary (the DCAT-AP-NL data themes: AGRI, ECON, EDUC, ENER, ENVI, GEOJ, GOVE, HEAH, INTR, JUST, REGI, SOCI, TRAN, or the full EU data-theme authority URI), so the dataset is harder to compare and find across domains (issue #845). The value still renders; this is a warning only. | Set `classification` to a theme from the DCAT-AP-NL data-theme vocabulary (e.g. in the postgres module's `config.dataProvider.classification`); see `shared/contracts/open-data-provider.yaml` and `docs/fds-alignment.md`. |
+| `[E155] ... needs operator "<name>", but its CRD "<crd>" is not present on the cluster ...` | `cds up --target helm` checked the cluster (`kubectl get crd <crd>`) and the operator a module depends on is confirmed absent, so `helm upgrade --install` would fail with "no matches for kind" (#768). | Install the operator with the command named in the error (e.g. `helm upgrade --install cnpg --namespace cnpg-system --create-namespace cnpg/cloudnative-pg` after `helm repo add cnpg https://cloudnative-pg.github.io/charts`), then re-run `cds up --target helm`. |
+| `[E156] ... forEach marker of manifest ... resolved ... to nothing / to a <type>, but fan-out needs a list / produced the duplicate metadata.name ...` | A raw manifest under `implementation.kubernetes.manifests` declares `forEach: <expr>`, but the expression does not resolve to a list in the plan, or two fanned-out copies resolve to the same `metadata.name` (they would collide in the release namespace and in the chart's template file names) (issue #1170). | Point `forEach` at a list-valued config path (e.g. `config.services`), and reference `${item...}` in the manifest's `metadata.name` so every fanned-out copy is uniquely named. |
+| `[E157] ... Duplicate datasetId ...` | `spec.dataSharing.datasets` declares the same `datasetId` more than once; each disclosed dataset must be declared once (issue #844). | Remove the duplicate entry, or give each entry a distinct `datasetId`. |
+| `[E158] ... Dataset ... names module ... which is not an enabled module instance ...` | A `spec.dataSharing.datasets` entry's `module` names a module id that is not an enabled module instance in the profile (issue #844). | Point `module` at an existing, enabled module instance id from `spec.modules`, or drop the field. |
+| `[E159] ... enrolls service "<name>" in its service-mesh config (config.services[<i>].name), but no module instance with that id exists in the profile ...` | At plan time, a module providing the `service-mesh` contract has a `config.services[].name` that matches no module instance id of the profile (a typo such as `warehose` for `warehouse`, or a disabled module) (issue #1179). The rendered per-service `AuthorizationPolicy` would select zero pods via the `cds.dev/module` label -- a silent no-op allow policy. | Fix the enrollment to name an existing module instance id (the error lists the planned ids), add the module with that id, or remove the entry from `config.services`; enable a disabled module you want to enroll. |
+| `[E161] ... Unknown module id '<id>' ...` / `... addresses/traverses a non-scalar value ...` / `... matches a resolved secret value ...` | A persisted `cds config set module.<id>.config.<path> <value>` override in `.cds/config.json` or the shared `cds.shared.json` (issue #1220) names a module id that is not in the resolved profile, points at a dict/list instead of a scalar leaf, or tries to store a resolved secret value. The same code is emitted at load time when a hand-edited `config.json` breaks, so `validate`/`plan`/`render` fail loudly instead of silently ignoring the override. | Fix the id or path (only `module.<id>.config.<scalar leaf>` keys are accepted), remove the override with `cds config unset [--shared] module.<id>.config.<path>`, or store a `secrets.<alias>` reference instead of a secret value; see [docs/cli-reference.md](docs/cli-reference.md#local-module-config-overrides-moduleidconfigpath). |
+| `[W132] ... enrolls service "<name>" in its service-mesh config, but the module renders no Kubernetes workload pods ...` | On the Helm target, a known enrollment names a module with no rendered workloads, such as a manifest-only module or one whose workloads are all disabled by `enabledFrom` (issue #1198). If enabled, the per-service `AuthorizationPolicy` would select zero pods via the `cds.dev/module` label. | Enroll a module that renders at least one Kubernetes workload, or enable one of its gated workloads; remove the enrollment if no pod workload is intended. |
 | `[W097] ... config.image.tag is "latest" with config.image.source "registry"` | `config.image.tag: latest` under `source: registry` still validates, but drifts silently between deploys instead of pinning a reproducible version. | Pin an explicit tag from `publish-images.yml`'s output (or `tests/fixtures/signed-images.json`) instead of `latest`. |
+| `[E127] ... Unknown confinement key ...` / `... must be 'default', 'unconfined' or 'file:<path>' ...` / `... Invalid SELinux label(s) ...` | A `security.runtime.confinement` block (or a module instance's confinement) is malformed: an unsupported key, an invalid `seccomp`/`apparmor`/`selinux` value, or a `file:` seccomp profile path that is missing or resolves outside the profile directory. | Use `seccomp: default`/`unconfined`/`file:<relative path inside the profile directory>`, `apparmor: runtime/default`/`unconfined`/`<host profile name>`, and `selinux: disable` or a `label: value` list; see `docs/runtime-confinement.md`. |
+| `[W071] ... has kind "docker-compose" but no "compose" definition` | A module implementation declares the `docker-compose` kind without an `implementation.compose` block, so the renderer skips the module. | Add the `implementation.compose` block (services, volumes) to the module's `module.yaml`, or set a supported implementation `kind`. |
+| `[W072] ... Container "<name>" in Kubernetes workload "<module>/<workload>" has no resource requests or limits` | Rendering the Helm target: a container or init container listed in a Kubernetes workload has no matching entry under the workload's `resources`. | Add a `resources.<container>` entry with `requests` and `limits` to the workload in the module's Kubernetes implementation (see `docs/kubernetes.md`). |
+| `[W090] ... Invalid .env line format: "<line>" (expected KEY=VALUE)` | A line in the `.env` file is neither blank, a `#` comment, nor in `KEY=VALUE` form, so it is ignored. | Fix the line to `KEY=VALUE` form, comment it out with `#`, or remove it. |
+| `[W098] ... Applied plaintext endpoint exposure waiver for production profile (modules: ...)` | A production profile's `spec.security.waivers.plaintextEndpointExposure` waived the CDS-SEC-074 plaintext-endpoint findings; CDS reports the waiver so it stays visible in security output. | Confirm the waiver is still justified and its `reason` is accurate; see `docs/security-rules.md` (Warnings). |
+| `[W099] ... plaintextEndpointExposure is set but has no effect for profile class ...` | `spec.security.waivers.plaintextEndpointExposure` is declared on a non-prod profile, where CDS-SEC-074 and its waiver only apply to prod-class profiles. | Remove the waiver from the non-prod profile, or move it to the prod profile (overlay) it is meant for. |
+| `[W103] ... Applied runtime policy waiver for production profile (checks: ...)` | A production profile's `spec.security.waivers.runtimePolicy` suppressed CDS-SEC-090 violations; CDS reports the waiver so it stays visible in security output. | Confirm the waiver is still justified and its `reason` is accurate; see `docs/security-rules.md` (Warnings). |
+| `[W104] ... runtimePolicy is set but has no effect for profile class ...` | `spec.security.waivers.runtimePolicy` is declared on a non-prod profile, where CDS-SEC-090 and its waiver only apply to prod-class profiles. | Remove the waiver from the non-prod profile, or move it to the prod profile (overlay) it is meant for. |
 
 All diagnostics print with their error code and YAML path (e.g. `spec.modules[1].config`), so search the profile file for that path to find the exact line to fix.
 
@@ -934,6 +963,8 @@ Please read these first:
 - [CONTRIBUTING.md](CONTRIBUTING.md)
 - [docs/maintainer-merge-policy.md](docs/maintainer-merge-policy.md)
 - [SECURITY.md](SECURITY.md) — how to report a vulnerability
+- [ACCESSIBILITY.md](ACCESSIBILITY.md) — our accessibility commitment and how
+  to report a barrier
 - [LEGAL.md](LEGAL.md) — license, code of conduct, security, support, and
   regulatory-compliance documentation index
 - [CHANGELOG.md](CHANGELOG.md)
@@ -948,11 +979,15 @@ Development helper tools are located in the gitignored `tools/` directory. See
 
 - [Quickstart](README.md#-quickstart) — get running in 5 minutes
 - [CLI Reference](docs/cli-reference.md) — full command/flag table and `[profile]` resolution rules
+- [Underlying Commands](docs/underlying-commands.md) — the exact `docker compose` / `helm` / `kubectl` commands behind `cds render` / `up` / `state` / `down`, for running the stack without the `cds` CLI
 - [How CDS Works](docs/how-it-works.md) — C4 + sequence diagrams of the validate → plan → render engine
-- [Kubernetes Target](docs/kubernetes.md): Helm rendering and isolated local k3s workflow
+- [Kubernetes Target](docs/kubernetes.md): Helm rendering, Docker Compose migration guide, and isolated local k3s workflow — with ready-made example profiles (`profiles/local-kubernetes-example`, `profiles/production-kubernetes-example` with its `prod` overlay)
 - [From Docker Compose to CDS Profile](docs/from-docker-to-cds-profile.md) — complete transformation guide
 - [Architecture](docs/architecture.md) — design and core concepts
 - [Modules](docs/modules.md) — how to structure reusable components
+- [Repository Layout](docs/repo-layout.md) — best practice for keeping platform assets (modules, profiles) separate from project code (Dagster pipelines, dbt/dlt, database scripts)
+- [Workload Runtime API](docs/runtime-api.md) — `cds-runtime`: named database connections for Python workloads
+- [Production Hardening Checklist](docs/production-hardening-checklist.md) — control-to-owner mapping and the operator verification procedure for a production profile
 - [Roadmap](docs/roadmap.md) — planned features and milestones
 
 ---

{composable_data_stack-0.14.0 → composable_data_stack-0.15.1}/cli/audit_log.py

@@ -16,6 +16,7 @@ recorded (e.g. "CDS_DB_PASSWORD"), the same `${CDS_*}` placeholder
 convention used by the planner/renderer (see `cli/secrets.py`) -- never
 secret values.
 """
+
 from __future__ import annotations
 
 import json
@@ -25,6 +26,8 @@ from datetime import UTC, datetime
 from pathlib import Path
 from typing import Any
 
+from .security_common import SymlinkedWriteError, ensure_no_symlinked_parent
+
 _LOG_FILE = Path(".cds") / "audit-log.jsonl"
 
 # Set to disable the audit log without touching project config, e.g. for a
@@ -36,13 +39,12 @@ _TRUTHY = {"1", "true", "yes", "on"}
 def audit_log_path(project_root: Path | None = None) -> Path:
     """Return the path audit entries are appended to for `project_root`.
 
-    No `cli.main` call site currently passes an explicit `project_root`, so
-    this defaults to `Path.cwd()` rather than the `resolve_project_root
-    (profile_path)` convention used elsewhere (e.g. for rendered Compose
-    output). That means audit entries land in whatever directory the user
-    happened to invoke `cds` from, not necessarily the profile's own
-    project root (e.g. when `CDS_PROFILE_PATH` points at a separate
-    profiles checkout). Tracked by #822.
+    `cli.main` call sites pass the profile-anchored project root
+    (`resolve_project_root(profile_path)` -- the same resolution that
+    decides where rendered Compose output goes), so entries for a given
+    profile consistently land in one `.cds/audit-log.jsonl` regardless of
+    the cwd `cds` was invoked from (#822). With no `project_root` (direct
+    programmatic use), this falls back to `Path.cwd()`.
     """
     root = (project_root or Path.cwd()).expanduser().resolve()
     return root / _LOG_FILE
@@ -141,6 +143,13 @@ def record_entry(
         entry["details"] = details
 
     try:
+        # Never append through a symlinked .cds directory: mkdir() and
+        # open() both follow the link, so the entry would land outside the
+        # project tree (issue #1225, same write-through-symlink class as
+        # the lifecycle-log guard in cli/main.py and the tracking manifest
+        # in cli/getter.py). audit_log_path() always returns the default
+        # .cds/audit-log.jsonl, so this guard covers the default path only.
+        ensure_no_symlinked_parent(log_path, what="audit log directory")
         log_path.parent.mkdir(parents=True, exist_ok=True)
         if log_path.is_symlink():
             # Never follow/overwrite-through a symlink planted at the log
@@ -148,8 +157,11 @@ def record_entry(
             log_path.unlink()
         with open(log_path, "a", encoding="utf-8") as f:
             f.write(json.dumps(entry, sort_keys=True) + "\n")
-    except OSError as exc:
-        print(f"WARNING Could not write audit log entry to {log_path}: {exc}", file=sys.stderr)
+    except (OSError, SymlinkedWriteError) as exc:
+        print(
+            f"WARNING Could not write audit log entry to {log_path}: {exc}",
+            file=sys.stderr,
+        )
         return None
 
     return log_path

{composable_data_stack-0.14.0 → composable_data_stack-0.15.1}/cli/composer.py

@@ -186,12 +186,18 @@ def compose_profile(
         )
         return None, diagnostics
 
-    providers, existing_ids, provider_diags = _collect_provided_contracts(profile, profile_dir, modules_root)
+    providers, existing_ids, provider_diags = _collect_provided_contracts(
+        profile, profile_dir, modules_root
+    )
     diagnostics.extend(provider_diags)
     if any(d.level == "error" for d in diagnostics):
         return None, diagnostics
 
-    metadata = module_def.get("metadata") if isinstance(module_def.get("metadata"), dict) else {}
+    metadata = (
+        module_def.get("metadata")
+        if isinstance(module_def.get("metadata"), dict)
+        else {}
+    )
     resolved_id = instance_id or metadata.get("name")
     if not isinstance(resolved_id, str) or not resolved_id.strip():
         diagnostics.append(
@@ -275,7 +281,9 @@ def compose_profile(
                 )
                 continue
         else:
-            candidates = sorted(ref for ref, kind in providers.items() if kind == contract_kind)
+            candidates = sorted(
+                ref for ref, kind in providers.items() if kind == contract_kind
+            )
             if len(candidates) == 1:
                 chosen = candidates[0]
             elif not candidates:
@@ -321,7 +329,9 @@ def compose_profile(
     if any(d.level == "error" for d in diagnostics):
         return None, diagnostics
 
-    secret_values = deepcopy((profile.get("spec", {}).get("secrets") or {}).get("values") or {})
+    secret_values = deepcopy(
+        (profile.get("spec", {}).get("secrets") or {}).get("values") or {}
+    )
     for alias in sorted(set(_iter_secret_aliases(config))):
         if alias in secret_values:
             continue
@@ -359,7 +369,9 @@ def compose_profile(
     merged["spec"]["modules"].append(new_instance)
 
     if secret_values:
-        secrets_block = merged["spec"].setdefault("secrets", {"provider": {"type": "env"}})
+        secrets_block = merged["spec"].setdefault(
+            "secrets", {"provider": {"type": "env"}}
+        )
         secrets_block.setdefault("provider", {"type": "env"})
         secrets_block["values"] = secret_values
 

{composable_data_stack-0.14.0 → composable_data_stack-0.15.1}/cli/confinement.py

@@ -4,6 +4,7 @@ A profile declares a default under `spec.security.runtime.confinement`; a
 module instance may override individual keys under `security.confinement`.
 The effective value is rendered into each service's `security_opt`.
 """
+
 from __future__ import annotations
 
 import os
@@ -49,7 +50,9 @@ def effective_confinement(
     defaults: dict[str, Any] | None, override: dict[str, Any] | None
 ) -> dict[str, Any]:
     """Profile defaults with the module instance's keys layered on top."""
-    merged = {key: value for key, value in (defaults or {}).items() if key in CONFINEMENT_KEYS}
+    merged = {
+        key: value for key, value in (defaults or {}).items() if key in CONFINEMENT_KEYS
+    }
     for key, value in (override or {}).items():
         if key in CONFINEMENT_KEYS:
             merged[key] = value
@@ -67,10 +70,12 @@ def validate_confinement(
     diagnostics: list[Diagnostic] = []
     for key in block:
         if key not in CONFINEMENT_KEYS:
-            diagnostics.append(_error(
-                f"Unknown confinement key '{key}'; supported keys: {', '.join(CONFINEMENT_KEYS)}.",
-                f"{path}.{key}",
-            ))
+            diagnostics.append(
+                _error(
+                    f"Unknown confinement key '{key}'; supported keys: {', '.join(CONFINEMENT_KEYS)}.",
+                    f"{path}.{key}",
+                )
+            )
 
     seccomp = block.get("seccomp")
     if seccomp is not None:
@@ -78,13 +83,18 @@ def validate_confinement(
     apparmor = block.get("apparmor")
     if apparmor is not None and not (
         isinstance(apparmor, str)
-        and (apparmor in {"runtime/default", "unconfined"} or _APPARMOR_NAME_RE.match(apparmor))
+        and (
+            apparmor in {"runtime/default", "unconfined"}
+            or _APPARMOR_NAME_RE.match(apparmor)
+        )
     ):
-        diagnostics.append(_error(
-            "apparmor must be 'runtime/default', 'unconfined' or the name of a profile "
-            "already loaded on the host (letters, digits, '.', '_', '-', '/').",
-            f"{path}.apparmor",
-        ))
+        diagnostics.append(
+            _error(
+                "apparmor must be 'runtime/default', 'unconfined' or the name of a profile "
+                "already loaded on the host (letters, digits, '.', '_', '-', '/').",
+                f"{path}.apparmor",
+            )
+        )
     selinux = block.get("selinux")
     if selinux is not None:
         diagnostics.extend(_validate_selinux(selinux, f"{path}.selinux"))
@@ -95,16 +105,21 @@ def _validate_seccomp(value: Any, path: str, base_dir: Path | None) -> list[Diag
     if value in {"default", "unconfined"}:
         return []
     if not isinstance(value, str) or not value.startswith("file:") or len(value) == 5:
-        return [_error(
-            "seccomp must be 'default', 'unconfined' or 'file:<path to a JSON profile "
-            "relative to the profile directory>'.",
-            path,
-        )]
+        return [
+            _error(
+                "seccomp must be 'default', 'unconfined' or 'file:<path to a JSON profile "
+                "relative to the profile directory>'.",
+                path,
+            )
+        ]
     relative = value[5:]
     if base_dir is None:
-        return [_error(
-            "seccomp file profiles need a known profile directory to resolve against.", path
-        )]
+        return [
+            _error(
+                "seccomp file profiles need a known profile directory to resolve against.",
+                path,
+            )
+        ]
     # Reject rooted and drive-qualified paths explicitly: they are not
     # "absolute" to pathlib on every platform (e.g. "/x" on Windows).
     if (
@@ -113,14 +128,17 @@ def _validate_seccomp(value: Any, path: str, base_dir: Path | None) -> list[Diag
         or _DRIVE_RE.match(relative)
         or ".." in Path(relative).parts
     ):
-        return [_error(
-            "seccomp file profiles must be relative paths inside the profile directory.", path
-        )]
+        return [
+            _error(
+                "seccomp file profiles must be relative paths inside the profile directory.",
+                path,
+            )
+        ]
     target = (base_dir / relative).resolve()
     if not target.is_relative_to(base_dir.resolve()):
-        return [_error(
-            "seccomp file profile resolves outside the profile directory.", path
-        )]
+        return [
+            _error("seccomp file profile resolves outside the profile directory.", path)
+        ]
     if not target.is_file():
         return [_error(f"seccomp profile '{relative}' does not exist.", path)]
     return []
@@ -129,26 +147,36 @@ def _validate_seccomp(value: Any, path: str, base_dir: Path | None) -> list[Diag
 def _validate_selinux(value: Any, path: str) -> list[Diagnostic]:
     if value == "disable":
         return []
-    if not isinstance(value, list) or not value or not all(isinstance(v, str) for v in value):
-        return [_error(
-            "selinux must be 'disable' or a non-empty list of labels such as "
-            "'type:container_t' or 'level:s0:c100,c200'.",
-            path,
-        )]
+    if (
+        not isinstance(value, list)
+        or not value
+        or not all(isinstance(v, str) for v in value)
+    ):
+        return [
+            _error(
+                "selinux must be 'disable' or a non-empty list of labels such as "
+                "'type:container_t' or 'level:s0:c100,c200'.",
+                path,
+            )
+        ]
     bad = [label for label in value if not _SELINUX_LABEL_RE.match(label)]
     if bad:
-        return [_error(
-            f"Invalid SELinux label(s): {', '.join(bad)}; expected user:, role:, type: or level: values.",
-            path,
-        )]
+        return [
+            _error(
+                f"Invalid SELinux label(s): {', '.join(bad)}; expected user:, role:, type: or level: values.",
+                path,
+            )
+        ]
     kinds = [label.split(":", 1)[0] for label in value]
     duplicates = sorted({k for k in kinds if kinds.count(k) > 1})
     if duplicates:
-        return [_error(
-            f"Duplicate SELinux label kind(s): {', '.join(duplicates)}; "
-            "each of user:, role:, type: and level: may appear at most once.",
-            path,
-        )]
+        return [
+            _error(
+                f"Duplicate SELinux label kind(s): {', '.join(duplicates)}; "
+                "each of user:, role:, type: and level: may appear at most once.",
+                path,
+            )
+        ]
     return []
 
 
@@ -193,11 +221,13 @@ def apply_confinement(
     try:
         new_opts = confinement_security_opts(confinement, compose_dir, base_dir)
     except ValueError:
-        return [_error(
-            "seccomp file profile cannot be referenced relative to the generated "
-            "docker-compose.yml (different drive); render into the profile's drive.",
-            f"{path}.seccomp",
-        )]
+        return [
+            _error(
+                "seccomp file profile cannot be referenced relative to the generated "
+                "docker-compose.yml (different drive); render into the profile's drive.",
+                f"{path}.seccomp",
+            )
+        ]
     existing = [str(opt) for opt in service.get("security_opt") or []]
     replaced = {
         "seccomp": ("seccomp=", "seccomp:"),
@@ -205,7 +235,11 @@ def apply_confinement(
         "selinux": ("label=", "label:"),
     }
     prefixes = tuple(p for key in confinement for p in replaced.get(key, ()))
-    kept = [opt for opt in existing if not opt.startswith(prefixes)] if prefixes else existing
+    kept = (
+        [opt for opt in existing if not opt.startswith(prefixes)]
+        if prefixes
+        else existing
+    )
     # `default` seccomp means "engine default", so it only clears module-set values.
     merged = kept + new_opts
     if merged:
@@ -216,4 +250,6 @@ def apply_confinement(
 
 
 def _error(message: str, path: str) -> Diagnostic:
-    return Diagnostic(level="error", code=INVALID_CONFINEMENT_CODE, message=message, path=path)
+    return Diagnostic(
+        level="error", code=INVALID_CONFINEMENT_CODE, message=message, path=path
+    )

{composable_data_stack-0.14.0 → composable_data_stack-0.15.1}/cli/constants.py

@@ -4,5 +4,6 @@
 # RecursionError / stack overflow.
 MAX_NESTING_DEPTH = 100
 
+
 class MaxNestingDepthExceeded(Exception):
     """Raised when a recursive structure exceeds MAX_NESTING_DEPTH."""

{composable_data_stack-0.14.0 → composable_data_stack-0.15.1}/cli/diagnostics.py

@@ -4,7 +4,7 @@ from dataclasses import dataclass
 
 @dataclass
 class Diagnostic:
-    level: str   # "error" | "warning"
+    level: str  # "error" | "warning"
     code: str
     message: str
     path: str