datasecops-cli 0.6.2__tar.gz → 0.7.0b2__tar.gz

{datasecops_cli-0.6.2 → datasecops_cli-0.7.0b2}/.gitignore

@@ -219,4 +219,5 @@ __marimo__/
 
 
 legacy_framework/
-plans/
+plans/
+.cortex/

datasecops_cli-0.7.0b2/AGENTS.md

@@ -0,0 +1,79 @@
+# Instructions for AI agents (and humans) working on the DataSecOps CLI
+
+The `datasecops` CLI (Click) is the developer and CI companion to the Orbit (DataSecOps Framework)
+Snowflake Native App (`monitorial-io/monitorial.orbit.nativeapp`). This repository also owns
+`src/datasecops_model`, the DCM model package that the app vendors. See `DEVELOPMENT.md` for setup,
+project structure and publishing.
+
+## Git workflow
+
+- **Never commit to `main`.** Every change goes on a branch and reaches `main` through a pull request.
+  Keep one branch per open PR; ongoing DCM work goes on `feature/dcm-integration` (PR #31) until it
+  is merged.
+- Check `git status` and `git branch --show-current` before changing anything.
+- Others push to shared branches too (e.g. version bumps for beta releases). If a push is rejected,
+  `git fetch`, look at the new commits, `git rebase origin/<branch>`, re-run the tests, then push.
+  Never force-push a shared branch without asking.
+- Version numbers in `pyproject.toml` and releases (`cli-v*` tags) are the maintainers' call - don't
+  bump or tag unless asked.
+- Commit only what you changed; never commit `.venv/`, `__pycache__/`, build output or secrets.
+- Commit messages: a short imperative subject, a body that says why, and the CoCo footer:
+
+  ```
+  Generated with [Snowflake CoCo](https://docs.snowflake.com/en/user-guide/cortex-code/cortex-code)
+
+  Co-authored-by: Snowflake CoCo <noreply@snowflake.com>
+  ```
+
+## Tests
+
+```bash
+.venv/bin/python -m pytest -q
+```
+
+Run them before every commit and report the real results. Add tests next to the existing ones
+(`tests/test_dcm_model.py` for the model package, `tests/test_dcm_orchestrator.py` for `dcm`
+commands, `tests/test_model_commands.py` for `model` commands, `tests/test_main.py` for Click wiring).
+
+## `datasecops_model` (shared with the native app)
+
+- Keep it dependency-light (pydantic; Jinja only lazily) - it also runs inside Snowflake stored
+  procedures from the app's `app.zip`.
+- The schema (`datasecops.dcm/v1`) uses `extra="forbid"`: new fields must be optional with defaults so
+  existing model files keep loading. A model without `scope` is a full model; a scoped file is one
+  repository's slice (see `scope.py`).
+- After changing it: commit here first, then vendor it into the app on an app branch and run the app's
+  tests:
+
+  ```bash
+  .venv/bin/python scripts/vendor_model.py <app-repo>/src
+  ```
+
+  `VENDOR.json` records this repository's commit, so vendor after committing. Never edit the app's copy.
+- Starter model defaults: only `DATAOPS_ADMIN`, `DATA_ENGINEER_ADMIN` and `DEVELOPERS`, no warehouses.
+  ORGADMIN / GLOBALORGADMIN are optional Snowflake roles (not extracted; validation warns).
+- `extract.to_warehouses` drops values equal to the `warehouse()` macro defaults, except the grant maps
+  (`grants`, `application_grants`), which must stay dictionaries.
+
+## CLI conventions
+
+- Commands are Click groups in `src/datasecops_cli/main.py`; logic lives in `services/`.
+- Options are consistent across commands: `-p/--project` (repeatable; names, types or `all`),
+  `--target`, `--connection/-c`, `--check` for CI checks that must not write files.
+- Commands must work non-interactively in CI (`CI`, `GITHUB_ACTIONS`, `TF_BUILD`, `BUILD_BUILDID`):
+  never prompt there; `dcm deploy` stops for manual scripts only at a terminal.
+- Native app procedures are called through `SnowflakeService.call_procedure`, which only allows names
+  in `ALLOWED_PROCEDURES`: add new ones there.
+- Say **CoCo**, not "Cortex Code" (the `cortex` command name stays).
+
+## Documentation
+
+When commands or behaviour change, update in the same change: `docs/dcm.md` (DCM), `README.md`, and
+an entry under `## [Unreleased]` in `CHANGELOG.md`.
+
+## Working style
+
+- Read the code before changing it, keep changes to what was asked, and flag wider changes.
+- Ask before outward-facing or hard-to-reverse actions that weren't requested (opening or merging PRs,
+  tagging releases, publishing, running DDL/DML against an account).
+- At the end, say what changed, what was tested (with the results) and what is still to do.

{datasecops_cli-0.6.2 → datasecops_cli-0.7.0b2}/CHANGELOG.md

@@ -2,6 +2,86 @@
 
 All notable changes to the DataSecOps CLI are documented in this file.
 
+## [Unreleased]
+
+### Fixed
+
+- **Default app database renamed to `MONITORIAL_ORBIT`** — the `datasecops setup` suggested app-database name (and matching docs examples) was `DATA_ENGINEERS_DATASECOPS_FRAMEWORK`; it now matches the native app's actual installed name.
+- **`dbt_project.yml` generator matches the source-based project spec** — added `docs-paths`, fixed `analysis-paths` (was `analyses`), added top-level `seeds:`/`tests:` blocks, `+static_analysis`/`+meta`/`+tags` at the model root, per-object-type `+static_analysis` (stored procedures/UDFs off, tasks baseline), the commented-out post-hook/on-run-start/on-run-end guidance, and the data-governance `vars` block.
+- **`profiles.yml` deploy targets now run as the framework's CI identity** — every target except `local-dev` previously used the local developer's Snowflake user with each target's own role/warehouse. They now use the CI Username/Role/Warehouse from the native app's Deployment settings (new `ProjectSettings.ci_username`/`ci_role`/`ci_warehouse`, falling back to the previous behavior when unset), matching what the CI pipeline actually deploys with.
+- **`datasecops setup` could silently default to GitHub for a non-GitHub platform** — `PROJECT`/`SOURCE_CONTROL`/`WORK_ITEMS` framework config and profile/`profiles.yml` resolution were one single `try/except`; any failure after connecting (including just the `SOURCE_CONTROL` fetch itself) discarded already-fetched config and silently fell back to `SourceControl()`'s default platform (`GitHub`) with no indication why, so the MCP setup step could prompt for the wrong PAT. Each framework config fetch is now independent, so one failing can't blank out another; a missing/failed `SOURCE_CONTROL` config, or one missing `source_control_platform` specifically, now warns explicitly which platform it's defaulting to.
+- **`local-dev` could be overwritten by a same-named framework target, or by the CI identity on a plain `dbt run`** — `generate_profiles_yml` no longer lets a framework target literally named `local-dev` clobber the dedicated external-browser output, and the profile's default target is now always `local-dev` (previously it followed whatever the native app marked as its own default, which could be a deploy-only target running as the CI identity). `_run_interactive` and bootstrap's `dbt deps` step now always run as `local-dev` instead of trusting the framework's default target.
+- **Re-running `datasecops setup` for an already-known profile never refreshed its deployment targets** — a CI identity change in the native app had no effect until `profiles.yml` was deleted. Setup now always refreshes the CI/deployment targets (while preserving an existing `local-dev` target if already customized), and skips writing `profiles.yml` entirely if the `PROJECT` framework config couldn't be loaded (previously it wrote incorrect defaults).
+- **Hardened `dbt_project.yml` generation against unsafe characters and a YAML boolean-coercion bug** — project/profile name, dbt version and warehouse name are now validated against a safe identifier pattern before being substituted into the raw YAML template (closing a potential injection via a newline or quote in native-app-configured values), and `static_analysis: off` is now quoted so YAML doesn't resolve it to the boolean `false`.
+
+### Docs
+
+- **Documented setting up `~/.dbt/profiles.yml`** — `datasecops bootstrap` writes a generated `profiles.yml` inside the dbt project folder as a reference only; dbt itself reads from `~/.dbt` (per the native app's `profile_dir` setting), which getting-started.md now explains, including the minimal `local-dev` block a developer needs there.
+
+## [0.7.0b1] - 2026-10-26
+
+### Added
+
+- **Shared DCM model (`datasecops_model`)** — one versioned JSON document (`config/dcm_model.json`, schema `datasecops.dcm/v1`) describing every DCM project (type, dir, `depends_on`, targets with deploy roles), the security model (role groups with environment fan-out, SCIM/UGE prefixes, diagram layout) and the warehouses (the `warehouse()` macro arguments, grants, application grants, budgets). The same package is vendored into the DataSecOps native app (`scripts/vendor_model.py`), so the app and the CLI validate, render and draw identically.
+- **`datasecops model` commands** — `import` (build the model from existing hand-written DCM projects), `validate` (hierarchy: cycles, orphans, unknown roles, duplicate grants, SCIM typos, bootstrap mode; cross-project: warehouse grantees exist, `depends_on` acyclic, unique DCM identifiers), `render [--check] [--scaffold] [--manifest]` (generates `roles.sql` / `warehouses.sql` / `manifest.yml` that still use the projects' macros, so the rendered SQL is statement-identical), `scripts [--previous REF] [--ado]` (pre/post auto/manual scripts incl. application grants, budgets, account privileges and REVOKEs for removed privileges), `diagram [--check]` (grid / snowflake-only / entra-links / entra-ace-map SVGs + markdown), `extract [--diff|--merge|--replace]` (current roles, role grants and warehouses via SHOW commands, env-suffixed roles folded back into environment groups), `pull` / `push` (native app ↔ local file), `schema`.
+- **Multi-project DCM** — `--project` is repeatable and accepts project names, types (`security` / `platform` / `custom`) or `all`; projects always run in `depends_on` order and a failed project skips its dependants only. Each target's `deploy_role` is passed to `snow` as `--role`.
+- **`datasecops dcm deploy`** — generates `out/scripts` per project, refuses to deploy while a manual ACCOUNTADMIN pre-deploy script is pending (unless `--pre-manual-done`), runs `pre_auto.sql` → `snow dcm deploy --alias <target>-<build>-<sha>` → records the deployment (with the CI identity) → `post_auto.sql`.
+- **`datasecops dcm plan --summary plan.md --fail-on-changes`** — runs `snow dcm plan --save-output`, summarises `out/plan/plan_result.json` (counts per operation, DROP / REVOKE warnings, manual steps) for PR comments.
+- **Shared environments** in the DCM model: `environments` are defined once and every project inherits them (`<database>.<schema_name>.<OBJECT_NAME><object_suffix>`, owned by the project `owner`); per-project `targets` are now overrides only. Older models load unchanged; `convert_to_environments` (app: *Use shared environments*) lifts identical targets.
+- **`datasecops dcm pipelines`** — downloads the DCM pipelines defined in the native app's Pipelines page (multi-project steps `dcm_cli_setup`, `dcm_model_validate`, `dcm_render_check`, `dcm_generate_scripts`, `dcm_plan_all`, `dcm_deploy_all`; defaults *DCM Pull Request* and *DCM Release* = build → manual pre gate → deploy → manual post gate). `datasecops download dcm` writes the same files. Pipelines are no longer generated locally.
+- **Per-project DCM pipelines** — each DCM pipeline deploys a single project (projects usually live in their own repositories); the app's all-projects *DCM Pull Request* / *DCM Release* pipelines are gone. `datasecops dcm pipelines` writes the app's pipelines (*DCM PR validate* / *DCM deploy*, assigned to DCM projects in the app) once per assigned project as `.github/workflows/dcm-<project>-<file>` / `.azuredevops/dcm-<project>-<file>`, the same paths the app pushes; `--project` limits them to some projects. Their unassigned generic rendering is never written.
+- **One model file per repository** — each repository's `config/dcm_model.json` holds only its part of the model, marked by a `scope` block: a security project gets the RBAC model (plus warehouse names), a platform project the warehouses (plus role names, so grants are still validated), a custom project only its project. `datasecops model pull` writes the current repository's slice (`-p NAME` to choose, `--full` for the whole model), `model push` merges a slice back into the app's model, `model extract` only compares the sections the file holds and `model diagram --check` passes when no security project is selected. Files without `scope` are the full model, as before (`datasecops_model.scope`: `scoped_model`, `merge_scoped`).
+- **`dcm deploy --sync-model`** — step 7 updates the native app's model from the deployed model file after a successful deploy of a project's default target (`api.sync_deployed_model`). The app only applies it when its own part (the repository's projects and RBAC or warehouses) is unchanged since it last exchanged it with git; otherwise the deploy warns and the drift is left for someone to reconcile. Never fails the deploy. The *DCM deploy* pipeline passes it.
+- **Step-by-step `dcm deploy`** — each project's deploy prints numbered steps (generate scripts, manual pre-deploy script, `pre_auto.sql`, `snow dcm deploy`, `post_auto.sql`, manual post-deploy script). At a terminal and in the DCM Operations menu it stops at a manual ACCOUNTADMIN script, names the file and waits for `done` (`show` prints it, `stop` ends the deploy); in CI or with `--no-input` it never waits. The menu no longer asks about manual scripts up front.
+- **`datasecops dcm list` / `dcm deployments`**; the DCM menu works on every project in the model and its "download" option pulls the model from the app or extracts it from Snowflake.
+- **DCM predeploy/postdeploy SQL scripts** — `datasecops dcm deploy` now runs numbered `.sql` scripts from `<project>/predeploy/` before `snow dcm deploy` and `<project>/postdeploy/` after it succeeds (sorted by filename, executed via `snow sql -f`). A failing predeploy script aborts the deploy for that project; a failing postdeploy script is reported (and fails the overall exit code) but doesn't block the already-recorded deployment status. Both folders are optional (missing/empty = no-op) and are also runnable independently via new `datasecops dcm predeploy`/`datasecops dcm postdeploy` subcommands and matching `DCM Operations` menu options. `BootstrapService`'s DCM scaffolding now creates empty `predeploy/`/`postdeploy/` folders alongside `definitions/`. (These folders are now the legacy auto layout: `pre_deployment/{auto,manual}` and `post_deployment/{auto,manual}` are preferred.)
+
+### Changed
+
+- **Starter model holds only the framework's own roles** — DATAOPS_ADMIN, DATA_ENGINEER_ADMIN (the `warehouse()` macro's admin role, so the SYSADMIN admin-role override is gone) and DEVELOPERS, and no warehouses; everything else is imported from Snowflake.
+- **Clearer diagram layout errors** — instead of "grouping 'None' is not listed in diagram_groups", the message names the roles without a diagram box and says to set a Grouping, or names the missing box and the roles that use it. Diagram boxes carry `data-box` so the app can drop dragged roles on them.
+- **ORGADMIN / GLOBALORGADMIN are optional Snowflake roles** — they only exist in some accounts (the organization account), so `model extract` / Import from Snowflake no longer records grants to or from them, and a model that references them validates with a warning ("the deploy fails where it does not exist") instead of an unknown-role or bootstrap error.
+
+- **No built-in default DCM target** — `dcm_default_target` (and the app DCM config's `default_target`) no longer default to `DEV`; without one a project uses its `default_target`, else its first target.
+
+- **Bootstrap scaffolds real DCM projects** from the native app's DCM model (or a starter model using the app's DCM naming convention): valid `manifest.yml` files, generated definitions, the framework macros and the deployment folders, instead of comment-only manifests.
+- **`SnowflakeService.call_procedure` only calls allow-listed native app procedures.**
+- **MCP setup no longer writes tokens into `mcp.json`** — servers read `SNOWFLAKE_PAT`, `GITHUB_PERSONAL_ACCESS_TOKEN` and `AZURE_DEVOPS_PAT` from the environment.
+- **Azure DevOps DCM pipelines are written to `.azuredevops/`** (was `.azuredevops/workflows/`).
+- Removed the "one security + one platform project per target" assumption from project resolution and the MCP DCM tools; added the `dcm_get_model` MCP tool.
+
+- **Migrated the non-interactive command tree from `argparse` to Click** — `setup`, `bootstrap`, `download`, and `dcm` (`init`/`analyze`/`plan`/`deploy`) are now a `click` group/subcommands instead of a hand-rolled `argparse.ArgumentParser`. CLI flags, defaults, and exit codes are unchanged; `--help` output is now Click-formatted. The interactive menu system (`menus/*.py`) is unaffected.
+- **Colored output now uses `click.secho`/`click.style` instead of `colorama`** — fixes ANSI escape codes leaking into non-TTY output (e.g. CI/CD logs for `datasecops download`/`datasecops dcm`), since Click auto-detects when stdout isn't a terminal and strips color codes.
+- **`DcmService` now validates the Snowflake connection name** before using it in any `snow`/`snow sql` subprocess call, closing the one dynamic value that wasn't already covered by `_validate_token` (target/alias/project identifiers already were).
+- **`tests/test_main.py` now exercises the CLI via `click.testing.CliRunner`** instead of calling `_build_parser()` directly, giving more faithful coverage of the actual command-line entry point.
+
+### Dependencies
+
+- Added `click>=8.1`, `jinja2>=3.1`.
+- Removed `colorama` (no longer used).
+
+### Fixed
+
+- **Role hierarchy diagrams: built-in admin roles** — the built-in Snowflake hierarchy (ACCOUNTADMIN > SYSADMIN / SECURITYADMIN > USERADMIN) now sets the rows inside a box, so the admin roles draw as a tree instead of one row with dashed arrows running through each other; a built-in edge the model already has is not drawn twice.
+
+## [0.7.0] - 2026-09-07
+
+### Added
+
+- **DCM (Database Change Management) support** — new `DcmService` wraps `snow dcm` commands (analyze, plan, deploy, create, list-deployments) for the security/platform two-project model.
+- **DCM Operations menu** — new interactive menu (init, analyze, plan, deploy, status, download, list projects) accessible from the main menu when `dcm_enabled: true` is set in `.datasecops.yml`.
+- **`datasecops dcm` subcommand** — non-interactive `init`/`analyze`/`plan`/`deploy` commands with `--target`, `--project`, and `--connection` flags for CI/CD pipelines.
+- **DCM MCP tools** — `dcm_list_projects`, `dcm_get_project_status`, and `dcm_get_deployment_targets` exposed to AI coding assistants.
+- **DCM config fields** — `.datasecops.yml` now supports `dcm_enabled`, `dcm_security_dir`, `dcm_platform_dir`, and `dcm_default_target`.
+- **DCM bootstrap scaffolding** — bootstrap can now optionally scaffold a `security/`/`platform/` DCM project structure from the native app's DCM configuration.
+- **`datasecops download dcm`** — downloads DCM configuration and DCM-tagged pipeline templates from the native app for CI/CD use.
+- **`dcm init` now actually registers a project** — requires `--project security|platform` and `--identifier` and calls `snow dcm create`, rather than only printing instructions.
+- **Input validation on `snow dcm` invocations** — project identifiers, targets, and aliases are validated against a safe character set before being passed to `subprocess.run`.
+
+### Fixed
+
+- fixed `docs serve` command to target folder correctly
+
 ## [0.6.2] - 2026-09-07
 
 ### Changed
@@ -25,8 +105,8 @@ All notable changes to the DataSecOps CLI are documented in this file.
 
 ### Changed
 
-- **Skill target renamed** — "Cortex Code" target label renamed to "CoCo" to match the native app UI naming convention.
-- **Downloads menu label** — "Cortex Code skills" renamed to "AI coding skills".
+- **Skill target renamed** — "CoCo" target label renamed to "CoCo" to match the native app UI naming convention.
+- **Downloads menu label** — "CoCo skills" renamed to "AI coding skills".
 - **Shared conversion helpers** — extracted `_strip_frontmatter()` and `_prepare_body()` to eliminate duplication across format converters.
 - **Robust frontmatter stripping** — uses line-boundary matching (`\n---`) instead of simple `split('---', 2)` to avoid false matches in skill body content.
 - **Cursor description quoting** — `.mdc` description values are now properly quoted and sanitized (newlines removed, quotes escaped).
@@ -70,7 +150,7 @@ All notable changes to the DataSecOps CLI are documented in this file.
 
 - **AzureDevOps pipelines download to correct folder** — `download_pipelines` now writes AzureDevOps pipeline files to `.azuredevops/workflows/` instead of the project root, matching the platform convention (similar to how GitHub pipelines are written to `.github/workflows/`).
 - **MCP server setup recognises all AzureDevOps platform variants** — the initial setup flow in `main.py` now uses case-insensitive matching for the platform name (`"AzureDevOps"` and `"Azure DevOps"` both work), consistent with the menu paths.
-- **Cortex Code MCP server uses absolute venv path** — the `datasecops-framework` MCP server entry in `.snowflake/cortex/mcp.json` now uses the absolute path to the project's `.venv/Scripts/datasecops-mcp` (Windows) or `.venv/bin/datasecops-mcp` (Unix) instead of the bare `datasecops-mcp` command, fixing "Process exited with code 1" errors in Cortex Code Desktop when the venv isn't on the system PATH.
+- **CoCo MCP server uses absolute venv path** — the `datasecops-framework` MCP server entry in `.snowflake/cortex/mcp.json` now uses the absolute path to the project's `.venv/Scripts/datasecops-mcp` (Windows) or `.venv/bin/datasecops-mcp` (Unix) instead of the bare `datasecops-mcp` command, fixing "Process exited with code 1" errors in CoCo Desktop when the venv isn't on the system PATH.
 
 ## [0.5.4] - 2026-06-05
 
@@ -96,9 +176,9 @@ All notable changes to the DataSecOps CLI are documented in this file.
 
 ### Changed
 
-- **Cortex Code MCP config is now project-level** — MCP server registration for Cortex Code now writes `.snowflake/cortex/mcp.json` in the project root instead of calling `cortex mcp add`. This enables project-scoped MCP servers in Cortex Code Desktop and removes the dependency on the `cortex` binary being installed.
-- **All MCP servers included in Cortex Code config** — the datasecops-framework, GitHub, Azure DevOps, and dbt MCP servers are all written to the project-level config file with `"type": "stdio"` field.
-- **Cortex Code always available as an AI tool option** — removed the `shutil.which("cortex")` check since we no longer need the cortex binary for MCP configuration.
+- **CoCo MCP config is now project-level** — MCP server registration for CoCo now writes `.snowflake/cortex/mcp.json` in the project root instead of calling `cortex mcp add`. This enables project-scoped MCP servers in CoCo Desktop and removes the dependency on the `cortex` binary being installed.
+- **All MCP servers included in CoCo config** — the datasecops-framework, GitHub, Azure DevOps, and dbt MCP servers are all written to the project-level config file with `"type": "stdio"` field.
+- **CoCo always available as an AI tool option** — removed the `shutil.which("cortex")` check since we no longer need the cortex binary for MCP configuration.
 - **Expanded `_ALLOWED_PROCEDURES`** — added `get_sqlfluff_config_file` and `get_documentation` to the MCP connection allowlist.
 
 ### Fixed
@@ -112,7 +192,7 @@ All notable changes to the DataSecOps CLI are documented in this file.
 
 ### Added
 
-- **MCP server CLI arguments** — `datasecops-mcp` now accepts `--connection-name` and `--app-database` arguments, eliminating the dependency on `.datasecops.yml` being in the working directory. This fixes connection failures when AI tools (Cortex Code, Cursor, etc.) launch the MCP server from a different working directory.
+- **MCP server CLI arguments** — `datasecops-mcp` now accepts `--connection-name` and `--app-database` arguments, eliminating the dependency on `.datasecops.yml` being in the working directory. This fixes connection failures when AI tools (CoCo, Cursor, etc.) launch the MCP server from a different working directory.
 - **`init_snowflake_service()` function** — new public API in `datasecops_mcp.connection` to pre-initialize the Snowflake service singleton with explicit parameters.
 
 ### Changed
@@ -212,22 +292,22 @@ All notable changes to the DataSecOps CLI are documented in this file.
 - **Configuration menu** — new `[4] configure` top-level menu consolidating tool installation and project setup:
   - Install dbt-core & dbt-snowflake from framework-pinned versions
   - Install SQLFluff from framework-pinned versions
-  - Configure MCP servers for VS Code, Cursor, Claude Code, or Cortex Code
+  - Configure MCP servers for VS Code, Cursor, Claude Code, or CoCo
   - Initialize a new dbt project with framework profiles
-  - Update Cortex Code to the latest version (`cortex update`)
-- **Cortex Code agent connection** — new `cortex_connection` field in `.datasecops.yml` to configure the Cortex Code agent connection separately from the SQL connection. Selected during setup, checked on launch with a prompt to update if mismatched.
+  - Update CoCo to the latest version (`cortex update`)
+- **CoCo agent connection** — new `cortex_connection` field in `.datasecops.yml` to configure the CoCo agent connection separately from the SQL connection. Selected during setup, checked on launch with a prompt to update if mismatched.
 - **profiles.yml generation during setup** — setup now connects to Snowflake, fetches framework PROJECT settings and targets, and creates the profile entry in `profiles.yml` at the framework-defined `profile_dir` location. Creates directory and file if missing, preserves existing profiles.
 - **Profile name resolution during setup** — setup reads `profile_name` from `dbt_project.yml` if it exists, otherwise fetches profiles from the native app (auto-selects single profile, prompts for multiple).
-- **MCP server configuration during setup** — setup now offers to configure MCP servers for AI tools (VS Code, Cursor, Claude Code, Cortex Code) with platform auto-detected from framework SOURCE_CONTROL config.
-- **Cortex Code and Node.js prerequisite checks** — setup now checks for `cortex` and `node` on PATH alongside existing `dbtf`, `gh`, and `az` checks.
-- **Auto-install prerequisites in setup scripts** — `setup.ps1` and `setup.sh` now offer to install missing tools: uv (auto), dbt Fusion (auto), Cortex Code, GitHub CLI (`winget`/`brew`/`apt`), Azure CLI (`winget`/`brew`/`curl`).
+- **MCP server configuration during setup** — setup now offers to configure MCP servers for AI tools (VS Code, Cursor, Claude Code, CoCo) with platform auto-detected from framework SOURCE_CONTROL config.
+- **CoCo and Node.js prerequisite checks** — setup now checks for `cortex` and `node` on PATH alongside existing `dbtf`, `gh`, and `az` checks.
+- **Auto-install prerequisites in setup scripts** — `setup.ps1` and `setup.sh` now offer to install missing tools: uv (auto), dbt Fusion (auto), CoCo, GitHub CLI (`winget`/`brew`/`apt`), Azure CLI (`winget`/`brew`/`curl`).
 
 ### Changed
 
 - **Main menu renumbered** — configure `[4]`, downloads `[5]`, bootstrap `[6]`, update pkgs `[7]`
 - **Install options moved to configuration menu** — removed `[14] install lint` from development menu and `[6] install` from SQLFluff lint submenu; removed `[6] new project` and `[7] mcp servers` from downloads menu
 - **Setup prompt defaults to yes** — `Run setup now?` now defaults to `y` with a branded header
-- **Cortex Code launch sets both connections** — runs `cortex connections set` for SQL and `cortex --connection` for agent, with mismatch detection against `settings.json`
+- **CoCo launch sets both connections** — runs `cortex connections set` for SQL and `cortex --connection` for agent, with mismatch detection against `settings.json`
 
 ### Fixed
 
@@ -241,18 +321,18 @@ All notable changes to the DataSecOps CLI are documented in this file.
 
 ### Added
 
-- **Launch Cortex Code from main menu** — new option `[3] cortex code` launches the Cortex Code AI coding agent directly from the CLI. Shows an error with install link if `cortex` is not on PATH.
-- **Automatic skill registration with Cortex Code** — after installing skills to `~/.cortex/skills/`, the CLI now runs `cortex skill add` to register the directory so skills are immediately discoverable as `$skill-name` commands. Previously skills were written to disk but not registered.
+- **Launch CoCo from main menu** — new option `[3] cortex code` launches the CoCo AI coding agent directly from the CLI. Shows an error with install link if `cortex` is not on PATH.
+- **Automatic skill registration with CoCo** — after installing skills to `~/.cortex/skills/`, the CLI now runs `cortex skill add` to register the directory so skills are immediately discoverable as `$skill-name` commands. Previously skills were written to disk but not registered.
 
 ### Changed
 
-- **Main menu renumbered** — downloads moved to `[4]`, bootstrap to `[5]`, update pkgs to `[6]` to accommodate the new Cortex Code option at `[3]`
+- **Main menu renumbered** — downloads moved to `[4]`, bootstrap to `[5]`, update pkgs to `[6]` to accommodate the new CoCo option at `[3]`
 
 ## [0.4.0] - 2026-05-16
 
 ### Added
 
-- **Framework-aligned project scaffolding** — `datasecops bootstrap` now generates `dbt_project.yml` directly from the native app configuration instead of running `dbtf init`. The generated project matches the `setup-dbt-project` Cortex Code skill specification, including:
+- **Framework-aligned project scaffolding** — `datasecops bootstrap` now generates `dbt_project.yml` directly from the native app configuration instead of running `dbtf init`. The generated project matches the `setup-dbt-project` CoCo skill specification, including:
   - Model sections based on profile's `MODEL_TYPES` (sources, dimensional_modelling, domains, cortex_models)
   - Object sections with doc node colors (stored_procedures, tasks, UDFs)
   - `dispatch` with framework package search order
@@ -271,7 +351,7 @@ All notable changes to the DataSecOps CLI are documented in this file.
   - `get_support_contacts` — team contact details
 - **New native app stored procedures** — `api.get_model_definitions()`, `api.get_object_definitions()`, `api.get_source_definitions()`, `api.get_access_groups()`, `api.get_support_contacts()` for CLI/MCP consumption
 - **New SnowflakeService methods** — both CLI and MCP Snowflake services expose the new stored procedures
-- **Multi-IDE skill installation** — skills now install to Cursor (`.cursor/rules/{id}.mdc` with frontmatter) and Claude Code (`.claude/commands/{id}.md`) in addition to Cortex Code. Bootstrap and downloads menu prompt users to select target IDE(s) via a multi-select prompt (`[1] Cortex Code`, `[2] Cursor`, `[3] Claude Code`, `[a] All`)
+- **Multi-IDE skill installation** — skills now install to Cursor (`.cursor/rules/{id}.mdc` with frontmatter) and Claude Code (`.claude/commands/{id}.md`) in addition to CoCo. Bootstrap and downloads menu prompt users to select target IDE(s) via a multi-select prompt (`[1] CoCo`, `[2] Cursor`, `[3] Claude Code`, `[a] All`)
 - **`select_multiple_from_list` display helper** — new utility for comma-separated multi-select input with "all" shortcut
 
 ### Changed
@@ -417,7 +497,7 @@ All notable changes to the DataSecOps CLI are documented in this file.
 
 ### Added
 
-- **MCP Server** — new `datasecops-mcp` server exposes framework governance config to AI coding tools (VS Code, Cursor, Cortex Code, Claude Code)
+- **MCP Server** — new `datasecops-mcp` server exposes framework governance config to AI coding tools (VS Code, Cursor, CoCo, Claude Code)
   - `get_branching_rules` — branch types, naming conventions, ticket requirements
   - `get_linting_rules` — SQLFluff rules from the native app
   - `get_dbt_packages` — approved packages with pinned versions
@@ -426,7 +506,7 @@ All notable changes to the DataSecOps CLI are documented in this file.
   - `get_project_settings` — project paths, execution mode, targets
   - `get_project_profiles` — all registered dbt projects
   - `get_deployment_targets` — environment targets with roles and warehouses
-  - `get_available_skills` — Cortex Code skills metadata
+  - `get_available_skills` — CoCo skills metadata
   - `get_default_macros` — default macros for a project profile
   - `validate_branch_name` — validates branch names against framework conventions
   - `get_deployment_workflow` — environment promotion flow
@@ -442,7 +522,7 @@ All notable changes to the DataSecOps CLI are documented in this file.
   5. Downloads `packages.yml` with approved versions
   6. Runs `dbt deps`
   7. Checks SQLFluff version requirements
-  8. Installs Cortex Code skills
+  8. Installs CoCo skills
   - Also available from main menu as option [4]
 
 - **Macro downloads** — new `api.get_macros_for_profile` integration
@@ -454,7 +534,7 @@ All notable changes to the DataSecOps CLI are documented in this file.
   - Check for Node.js (needed for GitHub/Azure DevOps MCP servers)
   - Install with MCP support (`datasecops-cli[mcp]`)
   - Optionally install `dbt-mcp`
-  - Interactively configure MCP servers for VS Code, Cursor, or Cortex Code
+  - Interactively configure MCP servers for VS Code, Cursor, or CoCo
   - Support GitHub and Azure DevOps PAT configuration
 
 - **MCP server integration guide** (`mcp-servers.json`) — example config for running framework, dbt, GitHub, and Azure DevOps MCP servers together
@@ -468,7 +548,7 @@ All notable changes to the DataSecOps CLI are documented in this file.
 
 ### Documentation
 
-- `docs/getting-started.md` — full setup guide for CLI + MCP servers with VS Code, Cursor, Cortex Code
+- `docs/getting-started.md` — full setup guide for CLI + MCP servers with VS Code, Cursor, CoCo
 - `docs/mcp-server.md` — MCP server reference with tool documentation, architecture, and usage examples
 - `README.md` — updated with MCP server section, recommended server stack, and new documentation links
 
@@ -480,7 +560,7 @@ All notable changes to the DataSecOps CLI are documented in this file.
 - Interactive menu-driven interface for dbt development, git operations, and configuration downloads
 - dbt commands via dbt Fusion subprocess (run, build, test, lint, compile, snapshot, freshness, docs, seed, deps, clean, debug, list, retry)
 - Git operations via GitPython (branching, commit, push, pull, rebase, squash, deploy, cherry-pick)
-- Configuration downloads from native app (SQLFluff rules, CI/CD pipelines, dbt packages, Cortex Code skills)
+- Configuration downloads from native app (SQLFluff rules, CI/CD pipelines, dbt packages, CoCo skills)
 - Framework-enforced branch naming conventions
 - Environment branch deployment (dev/test/prod)
 - SQLFluff linting with framework-managed rules

{datasecops_cli-0.6.2 → datasecops_cli-0.7.0b2}/DEVELOPMENT.md

@@ -13,7 +13,7 @@ Before running setup, ensure you have:
 
 Optional but recommended:
 
-- **dbt Fusion** (or dbt-core with dbt-snowflake) — required for dbt commands
+- **dbt 2.x** (`pip install dbt`) or **dbt Core 1.x** (`pip install dbt-core dbt-snowflake`) — required for dbt commands
 - **Cortex Code** — required for skill downloads
 
 ## Quick Start
@@ -38,7 +38,7 @@ chmod +x setup.sh
 The script will prompt you for:
 
 - **Connection name** — the name of your Snowflake connection from `~/.snowflake/connections.toml`
-- **Native app database name** — the database where the DataSecOps Framework app is installed (e.g. `DATA_ENGINEERS_DATASECOPS_FRAMEWORK`)
+- **Native app database name** — the database where the DataSecOps Framework app is installed (e.g. `MONITORIAL_ORBIT`)
 
 ### 2. Activate the virtual environment
 
@@ -75,12 +75,12 @@ This connects to the native app, loads your project configuration, and presents
 
 ### Development Menu
 
-Run dbt commands via dbt Fusion:
+Run dbt commands via dbt 2.x or dbt Core 1.x (configured engine):
 
 - **run** — full run, modified only (`state:modified+`), or specific models
 - **build** — run + test in one command
 - **test** — all tests or specific selectors
-- **lint** — SQLFluff lint/fix on modified files or entire project
+- **lint** — SQLFluff lint/fix (dbt Core 1.x) or `dbt lint`/`dbt lint --fix` (dbt 2.x) on modified files or entire project
 - **deps** — install dbt packages
 - **seed** — load seed data
 - **compile** — compile models
@@ -120,7 +120,7 @@ Created by the setup script in your project root. Contains:
 
 ```yaml
 connection_name: "my_connection"    # Snowflake connection name
-app_database: "DATA_ENGINEERS_DATASECOPS_FRAMEWORK"  # Native app database
+app_database: "MONITORIAL_ORBIT"  # Native app database
 profile_name: ""                    # Auto-detected from dbt_project.yml
 ```
 

{datasecops_cli-0.6.2 → datasecops_cli-0.7.0b2}/PKG-INFO

@@ -1,12 +1,13 @@
 Metadata-Version: 2.5
 Name: datasecops-cli
-Version: 0.6.2
+Version: 0.7.0b2
 Summary: DataSecOps Framework CLI for Snowflake Native App
 License-Expression: MIT
 License-File: LICENSE
 Requires-Python: >=3.10
-Requires-Dist: colorama>=0.4
+Requires-Dist: click>=8.1
 Requires-Dist: gitpython>=3.1
+Requires-Dist: jinja2>=3.1
 Requires-Dist: pydantic>=2.0
 Requires-Dist: pyyaml>=6.0
 Requires-Dist: snowflake-connector-python>=3.0
@@ -27,10 +28,11 @@ The DataSecOps CLI is the local developer companion to the **Data Engineers Data
 
 This CLI connects to the native app and gives developers:
 
-- **dbt development commands** — run, build, test, lint, and manage dbt projects via dbt Fusion
+- **dbt development commands** — run, build, test, lint, and manage dbt projects via dbt 2.x or dbt Core 1.x
 - **Source control operations** — branching, committing, rebasing, and deploying via GitPython with naming conventions enforced by the framework
-- **Configuration downloads** — pull SQLFluff rules, CI/CD pipelines, dbt packages, and Cortex Code skills from the native app to your local project
-- **MCP server** — expose framework governance rules to AI coding assistants (VS Code, Cursor, Cortex Code, Claude Code)
+- **DCM projects** — validate, render, plan and deploy any number of Snowflake DCM projects from the shared DCM model (`config/dcm_model.json`), in dependency order with each project's deploy role
+- **Configuration downloads** — pull SQLFluff rules, CI/CD pipelines, dbt packages, and CoCo skills from the native app to your local project
+- **MCP server** — expose framework governance rules to AI coding assistants (VS Code, Cursor, CoCo, Claude Code)
 
 ## Installation
 
@@ -54,8 +56,8 @@ Requires Python 3.10 or later.
 
 Optional:
 
-- **dbt Fusion** — required for dbt commands (`dbtf`). Install from https://docs.getdbt.com/docs/core/installation
-- **Cortex Code** for skill downloads
+- **dbt** — required for dbt commands (`dbt`). Install via `pip install dbt` (dbt 2.x) or `pip install dbt-core dbt-snowflake` (dbt Core 1.x), matching the engine configured in the native app's Project Settings.
+- **CoCo** for skill downloads
 - **Node.js 18+** for GitHub/Azure DevOps MCP servers
 
 ## Quick Start
@@ -82,7 +84,8 @@ datasecops
 |------|-------------|
 | **Development** | dbt run, build, test, lint (SQLFluff), deps, seed, compile, snapshot, freshness, docs |
 | **Git** | Branch create/checkout/delete, commit & push, rebase, squash, deploy to environment branches, cherry-pick, reset branch from main |
-| **Downloads** | SQLFluff config, CI/CD pipelines (GitHub Actions / Azure DevOps), dbt packages, Cortex Code skills |
+| **DCM Operations** | Plan, deploy, status, download and list the DCM projects in the model |
+| **Downloads** | SQLFluff config, CI/CD pipelines (GitHub Actions / Azure DevOps), dbt packages, CoCo skills |
 
 ## Non-Interactive Mode (CI/CD)
 
@@ -117,7 +120,7 @@ Your pipeline needs two things:
 
    ```yaml
    connection_name: "ci"
-   app_database: "DATA_ENGINEERS_DATASECOPS_FRAMEWORK"
+   app_database: "MONITORIAL_ORBIT"
    ```
 
 2. **A Snowflake connection** in `~/.snowflake/connections.toml` for the CI service account:
@@ -143,6 +146,21 @@ Your pipeline needs two things:
 
 The exit code is `0` on success, `1` if any download fails.
 
+## DCM projects
+
+```bash
+datasecops model pull                       # native app -> config/dcm_model.json (this repo's projects; --full = all)
+datasecops model validate
+datasecops model render --check             # generated definitions are current
+datasecops dcm plan -p security --target PROD --summary dcm-plan.md
+datasecops dcm deploy -p security --target PROD
+datasecops dcm pipelines                    # the app-defined pipelines assigned to each project
+```
+
+Each DCM project usually lives in its own repository, and its pipelines (*DCM PR validate*, *DCM
+deploy*) plan and deploy that single project. See [docs/dcm.md](docs/dcm.md) for the model, every
+command, the deployment scripts and the pipelines.
+
 ## MCP Server
 
 The package includes an MCP (Model Context Protocol) server that exposes your framework's governance configuration to AI coding assistants. Instead of static skill files, the MCP server gives AI tools live access to your native app's current rules.
@@ -158,7 +176,7 @@ The package includes an MCP (Model Context Protocol) server that exposes your fr
 
 ### Setup for AI Tools
 
-**Cortex Code** — add to `.snowflake/cortex/mcp.json` in your project root:
+**CoCo** — add to `.snowflake/cortex/mcp.json` in your project root:
 
 ```json
 {
@@ -246,7 +264,7 @@ The CLI reads from a `.datasecops.yml` file in your project root (created by the
 
 ```yaml
 connection_name: "my_connection"
-app_database: "DATA_ENGINEERS_DATASECOPS_FRAMEWORK"
+app_database: "MONITORIAL_ORBIT"
 ```
 
 Project profiles, linting rules, pipeline templates, and deployment targets are all managed centrally in the native app and pulled down by the CLI.

{datasecops_cli-0.6.2 → datasecops_cli-0.7.0b2}/README.md

@@ -8,10 +8,11 @@ The DataSecOps CLI is the local developer companion to the **Data Engineers Data
 
 This CLI connects to the native app and gives developers:
 
-- **dbt development commands** — run, build, test, lint, and manage dbt projects via dbt Fusion
+- **dbt development commands** — run, build, test, lint, and manage dbt projects via dbt 2.x or dbt Core 1.x
 - **Source control operations** — branching, committing, rebasing, and deploying via GitPython with naming conventions enforced by the framework
-- **Configuration downloads** — pull SQLFluff rules, CI/CD pipelines, dbt packages, and Cortex Code skills from the native app to your local project
-- **MCP server** — expose framework governance rules to AI coding assistants (VS Code, Cursor, Cortex Code, Claude Code)
+- **DCM projects** — validate, render, plan and deploy any number of Snowflake DCM projects from the shared DCM model (`config/dcm_model.json`), in dependency order with each project's deploy role
+- **Configuration downloads** — pull SQLFluff rules, CI/CD pipelines, dbt packages, and CoCo skills from the native app to your local project
+- **MCP server** — expose framework governance rules to AI coding assistants (VS Code, Cursor, CoCo, Claude Code)
 
 ## Installation
 
@@ -35,8 +36,8 @@ Requires Python 3.10 or later.
 
 Optional:
 
-- **dbt Fusion** — required for dbt commands (`dbtf`). Install from https://docs.getdbt.com/docs/core/installation
-- **Cortex Code** for skill downloads
+- **dbt** — required for dbt commands (`dbt`). Install via `pip install dbt` (dbt 2.x) or `pip install dbt-core dbt-snowflake` (dbt Core 1.x), matching the engine configured in the native app's Project Settings.
+- **CoCo** for skill downloads
 - **Node.js 18+** for GitHub/Azure DevOps MCP servers
 
 ## Quick Start
@@ -63,7 +64,8 @@ datasecops
 |------|-------------|
 | **Development** | dbt run, build, test, lint (SQLFluff), deps, seed, compile, snapshot, freshness, docs |
 | **Git** | Branch create/checkout/delete, commit & push, rebase, squash, deploy to environment branches, cherry-pick, reset branch from main |
-| **Downloads** | SQLFluff config, CI/CD pipelines (GitHub Actions / Azure DevOps), dbt packages, Cortex Code skills |
+| **DCM Operations** | Plan, deploy, status, download and list the DCM projects in the model |
+| **Downloads** | SQLFluff config, CI/CD pipelines (GitHub Actions / Azure DevOps), dbt packages, CoCo skills |
 
 ## Non-Interactive Mode (CI/CD)
 
@@ -98,7 +100,7 @@ Your pipeline needs two things:
 
    ```yaml
    connection_name: "ci"
-   app_database: "DATA_ENGINEERS_DATASECOPS_FRAMEWORK"
+   app_database: "MONITORIAL_ORBIT"
    ```
 
 2. **A Snowflake connection** in `~/.snowflake/connections.toml` for the CI service account:
@@ -124,6 +126,21 @@ Your pipeline needs two things:
 
 The exit code is `0` on success, `1` if any download fails.
 
+## DCM projects
+
+```bash
+datasecops model pull                       # native app -> config/dcm_model.json (this repo's projects; --full = all)
+datasecops model validate
+datasecops model render --check             # generated definitions are current
+datasecops dcm plan -p security --target PROD --summary dcm-plan.md
+datasecops dcm deploy -p security --target PROD
+datasecops dcm pipelines                    # the app-defined pipelines assigned to each project
+```
+
+Each DCM project usually lives in its own repository, and its pipelines (*DCM PR validate*, *DCM
+deploy*) plan and deploy that single project. See [docs/dcm.md](docs/dcm.md) for the model, every
+command, the deployment scripts and the pipelines.
+
 ## MCP Server
 
 The package includes an MCP (Model Context Protocol) server that exposes your framework's governance configuration to AI coding assistants. Instead of static skill files, the MCP server gives AI tools live access to your native app's current rules.
@@ -139,7 +156,7 @@ The package includes an MCP (Model Context Protocol) server that exposes your fr
 
 ### Setup for AI Tools
 
-**Cortex Code** — add to `.snowflake/cortex/mcp.json` in your project root:
+**CoCo** — add to `.snowflake/cortex/mcp.json` in your project root:
 
 ```json
 {
@@ -227,7 +244,7 @@ The CLI reads from a `.datasecops.yml` file in your project root (created by the
 
 ```yaml
 connection_name: "my_connection"
-app_database: "DATA_ENGINEERS_DATASECOPS_FRAMEWORK"
+app_database: "MONITORIAL_ORBIT"
 ```
 
 Project profiles, linting rules, pipeline templates, and deployment targets are all managed centrally in the native app and pulled down by the CLI.