databricks-schema 0.7.0__tar.gz → 0.10.0__tar.gz

{databricks_schema-0.7.0 → databricks_schema-0.10.0}/.gitignore

@@ -14,4 +14,3 @@ wheels/
 
 # Claude Code
 .claude/*
-!.claude/skills/

{databricks_schema-0.7.0 → databricks_schema-0.10.0}/CLAUDE.md

@@ -2,7 +2,7 @@
 
 ## Project
 
-`databricks-schema` — CLI + Python library that extracts, diffs, and generates SQL for Databricks Unity Catalog schemas stored as YAML or JSON files (one file per schema). Pydantic v2 models serve as the intermediate representation for future bidirectional sync.
+`databricks-schema` — CLI + Python library that extracts, diffs, and generates SQL for Databricks Unity Catalog schemas stored as YAML or JSON files (one file per schema) or DBML (one file per catalog). Pydantic v2 models serve as the intermediate representation for future bidirectional sync.
 
 ## Key commands
 
@@ -18,9 +18,10 @@ uv run databricks-schema --help
 
 ```
 databricks_schema/
-  models.py      # Pydantic v2 models: Catalog, Schema, Table, Column, PrimaryKey, ForeignKey
+  models.py      # Pydantic v2 models: Catalog, Schema, Table, Column, PrimaryKey, ForeignKey (Schema/Table carry grants)
   extractor.py   # CatalogExtractor — wraps databricks-sdk
-  yaml_io.py     # schema/catalog to/from YAML and JSON; _strip_empty removes None + empty collections
+  yaml_io.py     # schema/catalog to/from YAML and JSON; _strip_empty removes None + empty collections; load_schema_dir (all formats)
+  dbml_io.py     # schema/catalog to/from DBML; own writer, PyDBML grammar for parsing
   diff.py        # diff_schemas / diff_catalogs / diff_catalog_with_dir / diff_schema_dirs; FieldChange, ColumnDiff, TableDiff, SchemaDiff, CatalogDiff
   sql_gen.py     # schema_diff_to_sql — pure SQL generation from SchemaDiff; no SDK/IO
   cli.py         # argparse CLI: extract, diff, generate-sql, validate, diff-files, list-catalogs, list-schemas
@@ -29,15 +30,17 @@ tests/
   test_models.py
   test_extractor.py  # all SDK calls mocked with MagicMock
   test_yaml_io.py
+  test_dbml_io.py
   test_diff.py       # pure model comparison, no SDK calls
   test_sql_gen.py    # pure SQL generation tests, no SDK calls
   test_validate.py
-.claude/skills/databricks-schema-cli/SKILL.md  # Agent Skill: teaches an AI agent to drive the CLI
+skills/databricks-schema-cli/SKILL.md  # Agent Skill: teaches an AI agent to drive the CLI
 ```
 
 ## Conventions
 
 - Package manager: `uv`; do not use `pip` directly
+- Version bumps: use `uv version --bump <major|minor|patch>` (updates `pyproject.toml` and `uv.lock` together) — do not hand-edit the version string
 - Ruff: select E, W, F, I, UP; line-length 100; target py311
 - Use `X | None` not `Optional[X]`; use `datetime.UTC` not `timezone.utc`
 - `from __future__ import annotations` in every module
@@ -48,20 +51,27 @@ tests/
 - Column order in YAML = SDK position (None → 9999)
 - `_strip_empty`: removes `None` and empty `dict`/`list`; preserves `False`, `0`, empty strings
 - `yaml_io.py` has parallel YAML and JSON functions (`schema_to_yaml`/`schema_to_json`, etc.)
-- `diff_catalog_with_dir` accepts `fmt: Literal["yaml", "json"] = "yaml"` to select file format
+- `diff_catalog_with_dir` / `diff_schema_dirs` / `load_schema_dir` take `fmt: SchemaFormat` (`"yaml" | "json" | "dbml"`, default `"yaml"`)
+- DBML is one document per catalog (`extract -f dbml -o DIR` writes `DIR/<catalog>.dbml`): Catalog → `Project`, Schema → `TableGroup` (always emitted), PK → `indexes { (...) [pk] }`, FK → `Ref`
+- DBML has no key/value metadata: tags/grants/owner/table_type/storage_location are trailing `@key: value` lines in notes (`@tags`/`@grants` = JSON objects); writer output must round-trip exactly and stay parseable by `@dbml/core`
+- PyDBML is optional (`databricks-schema[dbml]` extra; also in the `dev` group): `dbml_io` imports it in a top-level `try/except ImportError` setting `PYDBML_AVAILABLE`; writing DBML must never need it, readers raise `DbmlDependencyError` (CLI exits 2)
+- `dbml_io` parses via PyDBML's grammar (`pydbml.definitions`) but resolves refs itself, so refs to tables outside the file are kept; never build a PyDBML `Database` (it rejects such refs and reflows note text) and never use PyDBML's renderer (it leaves complex types unquoted)
+- For DBML directories, `--schema` filters match schema names (YAML/JSON match file stems)
 - CLI: catalog is a required positional argument (not a flag)
-- `extract` `--format`/`-f` selects output format (`yaml` default, `json` opt-in); dest is `fmt`
-- `diff` auto-detects format from files present in the directory; exits 2 on mixed YAML+JSON
+- `extract` `--format`/`-f` selects output format (`yaml` default, `json`/`dbml` opt-in); dest is `fmt`
+- `diff` auto-detects format from files present in the directory; exits 2 on mixed formats
 - `diff`'s second positional arg is a directory if one exists at that path, else a second catalog name (`diff_catalogs`); first `catalog` arg is always the live side
 - `TableType` is re-exported from `databricks.sdk.service.catalog` — do not redefine it
 - `--include-metadata` flag (on `extract`, `diff`, `generate-sql`, `diff-files`) adds `owner` to output/comparison; `extract` additionally includes `storage_location` in its output (not compared by `diff`/`generate-sql`/`diff-files`); excluded by default
 - `--include-tags` flag (on `extract`, `diff`, `generate-sql`) enables Unity Catalog tag lookups; excluded by default
+- `--include-grants` flag (on `extract`, `diff`, `generate-sql`, `diff-files`) adds `grants: dict[str, list[str]]` (principal → sorted privileges) on `Schema`/`Table` via `grants.list()` — direct grants only, never inherited/effective ones; diff functions take `include_grants` and ignore grants otherwise; a grants `FieldChange` holds only differing principals
+- `generate-sql` emits `GRANT` for stored-not-live privileges and `REVOKE` for live-not-stored; `REVOKE` is destructive and follows `--allow-drop`
 - `--quiet`/`-q` flag (on `extract`, `diff`, `generate-sql`, `diff-files`) raises the log level to suppress progress messages; errors always print
 - `diff` command exits 0 (no changes) or 1 (differences found) — useful in CI
 - Diff result types are dataclasses (not Pydantic); comparison functions are pure (no SDK calls)
-- `generate-sql` auto-detects format from files; exits 2 on mixed YAML+JSON or empty directory
+- `generate-sql` auto-detects format from files; exits 2 on mixed formats or empty directory; invalid DBML exits 2
 - `generate-sql` destructive statements commented out by default; `--allow-drop` emits real DROPs
 - `sql_gen.py` is pure (no SDK, no I/O); diff direction: `FieldChange.old` = stored (target), `.new` = live
 - FK refs in SQL use same catalog as the source table (`ref_schema` + `ref_table` from model)
 - Unsupported field changes (`table_type`) emit `-- TODO: unsupported change: …` comments
-- When CLI commands, flags, or output shape change, update `.claude/skills/databricks-schema-cli/SKILL.md` to match
+- When CLI commands, flags, or output shape change, update `skills/databricks-schema-cli/SKILL.md` to match

{databricks_schema-0.7.0 → databricks_schema-0.10.0}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.4
+Metadata-Version: 2.5
 Name: databricks-schema
-Version: 0.7.0
+Version: 0.10.0
 Summary: Databricks Unity Catalog schema extractor
 Project-URL: Homepage, https://github.com/frizzleqq/databricks-schema
 Project-URL: Repository, https://github.com/frizzleqq/databricks-schema
@@ -18,14 +18,16 @@ Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
 Requires-Python: >=3.11
-Requires-Dist: databricks-sdk>=0.64.0
+Requires-Dist: databricks-sdk>=0.121.0
 Requires-Dist: pydantic>=2.0
 Requires-Dist: pyyaml>=6.0
+Provides-Extra: dbml
+Requires-Dist: pydbml<2,>=1.2.1; extra == 'dbml'
 Description-Content-Type: text/markdown
 
 # databricks-schema
 
-A CLI tool and Python library that uses the Databricks SDK to extract and diff Unity Catalog schemas as YAML files. It can also generate Databricks Spark SQL to apply schema changes across catalogs.
+A CLI tool and Python library that uses the Databricks SDK to extract and diff Unity Catalog schemas as YAML, JSON or [DBML](https://dbml.dbdiagram.io/docs/) files. It can also generate Databricks Spark SQL to apply schema changes across catalogs.
 
 ## Getting Started
 
@@ -53,23 +55,94 @@ databricks-schema diff test_catalog ./schemas/
 databricks-schema generate-sql test_catalog ./schemas/ --output-dir ./migrations/
 ```
 
-The YAML files act as a version-controllable snapshot of your schema. The `diff` command exits with code `1` when differences are found, making it suitable for CI pipelines.
+## Databricks Authentication
+
+The tool uses the [Databricks SDK](https://github.com/databricks/databricks-sdk-py) for auth. Configure it via environment variables:
+
+```bash
+export DATABRICKS_HOST=https://<databricks-url>
+export DATABRICKS_TOKEN=<your-personal-access-token>
+```
+
+Or use a [Databricks CLI profile](https://docs.databricks.com/dev-tools/cli/profiles.html) (`~/.databrickscfg`) — the SDK will pick it up automatically.
+
+You can also pass credentials directly as flags using `--host` / `--token` below.
+
+## Development
+
+Requires Python 3.11+ and [uv](https://github.com/astral-sh/uv).
+
+```bash
+git clone https://github.com/frizzleqq/databricks-schema
+cd databricks-schema
+uv sync --all-groups  # includes pytest and ruff
+```
+
+```bash
+# Run tests
+uv run pytest
+
+# Lint
+uv run ruff check databricks_schema/ tests/
+
+# Format
+uv run ruff format databricks_schema/ tests/
+```
+
+## Agent Skill
+
+This repo ships an [Agent Skill](https://code.claude.com/docs/en/skills) at
+[`skills/databricks-schema-cli/SKILL.md`](skills/databricks-schema-cli/SKILL.md)
+that teaches an AI coding agent (e.g. Claude Code) how to use the `databricks-schema` CLI to
+explore, snapshot, diff, and generate migration SQL for a live Unity Catalog. Copy the skill into
+your own project and it can use the CLI directly instead of writing ad-hoc Databricks SDK calls
+or queries.
+
+The skill assumes the `databricks-schema` command is on `PATH`. Install it as a standalone tool
+with [uv](https://github.com/astral-sh/uv):
+
+```bash
+uv tool install databricks-schema
+```
+
+To use the skill in your own project without cloning this repo, download it directly into your
+project's `.claude/skills/`:
+
+```bash
+mkdir -p .claude/skills/databricks-schema-cli
+curl -o .claude/skills/databricks-schema-cli/SKILL.md \
+  https://raw.githubusercontent.com/frizzleqq/databricks-schema/main/skills/databricks-schema-cli/SKILL.md
+```
+
+For agentic harnesses other than Claude Code that look under `.agents/skills/` instead:
+
+```bash
+mkdir -p .agents/skills/databricks-schema-cli
+curl -o .agents/skills/databricks-schema-cli/SKILL.md \
+  https://raw.githubusercontent.com/frizzleqq/databricks-schema/main/skills/databricks-schema-cli/SKILL.md
+```
 
 ## Output Format
 
-Each schema is written to `{output-dir}/{schema-name}.yaml`. Fields with no value (null comments, empty tag dicts, empty FK lists) are omitted. Use `--format json` to write `.json` files with the same structure.
+Each schema is written to `{output-dir}/{schema-name}.yaml` if `--output-dir` is specified. Fields with no value (null comments, empty tag dicts, empty FK lists) are omitted. Use `--format json` to write `.json` files with the same structure.
 
 ```yaml
 name: main
 comment: Main production schema
 tags:
   env: prod
+grants:
+  analysts:
+    - USE_SCHEMA
 tables:
   - name: users
     table_type: MANAGED
     comment: User accounts
     tags:
       domain: identity
+    grants:
+      analysts:
+        - SELECT
     columns:
       - name: id
         data_type: bigint
@@ -93,74 +166,74 @@ tables:
           - id
 ```
 
-Without `--output-dir`, `extract` prints a single `Catalog` document to stdout instead — the same schema shape nested under a `schemas` list, with the catalog name as a top-level field:
-
-```yaml
-name: prod_catalog
-schemas:
-  - name: main
-    comment: Main production schema
-    tags:
-      env: prod
-    tables: [...]
-```
-
-## Authentication
+### DBML
 
-The tool uses the [Databricks SDK](https://github.com/databricks/databricks-sdk-py) for auth. Configure it via environment variables:
+Writing DBML works out of the box; reading it back (`diff`, `generate-sql`, `validate`, `diff-files`, `catalog_from_dbml`) needs the optional `dbml` extra, which installs [PyDBML](https://github.com/Vanderhoof/PyDBML):
 
 ```bash
-export DATABRICKS_HOST=https://<databricks-url>
-export DATABRICKS_TOKEN=<your-personal-access-token>
+pip install 'databricks-schema[dbml]'
+uv tool install 'databricks-schema[dbml]'
 ```
 
-Or use a [Databricks CLI profile](https://docs.databricks.com/dev-tools/cli/profiles.html) (`~/.databrickscfg`) — the SDK will pick it up automatically.
+Without it, those commands exit `2` with a message saying how to install it.
 
-You can also pass credentials directly as flags (see `--host` / `--token` below).
+Use `--format dbml` to write the catalog as a single [DBML](https://dbml.dbdiagram.io/docs/) document — `{output-dir}/{catalog}.dbml` with `--output-dir`, or stdout without. Unlike YAML/JSON, DBML is one file per catalog, so that foreign keys between schemas resolve within the file. Paste it into [dbdiagram.io](https://dbdiagram.io) for an ER diagram, or point `diff`, `generate-sql`, `validate` and `diff-files` at the directory just like a YAML/JSON one.
 
-## Agent Skill
+```dbml
+Project "prod_catalog" {
+    database_type: 'Databricks'
+}
 
-This repo ships an [Agent Skill](https://code.claude.com/docs/en/skills) at
-[`.claude/skills/databricks-schema-cli/SKILL.md`](.claude/skills/databricks-schema-cli/SKILL.md)
-that teaches an AI coding agent (e.g. Claude Code) how to use the `databricks-schema` CLI to
-explore, snapshot, diff, and generate migration SQL for a live Unity Catalog. Point an agent at
-this repo (or copy the skill into your own project) and it can use the CLI directly instead of
-writing ad-hoc Databricks SDK calls.
+TableGroup "main" [note: 'Main production schema'] {
+    "main"."users"
+}
 
-The skill assumes the `databricks-schema` command is on `PATH`. Install it as a standalone tool
-with [uv](https://github.com/astral-sh/uv):
+Table "main"."users" {
+    "id" bigint [not null, note: 'Primary key']
+    "email" string
+    "org_id" bigint
+    "settings" "struct<theme:string,beta:boolean>"
 
-```bash
-uv tool install databricks-schema
-```
+    indexes {
+        ("id") [pk, name: 'pk_users']
+    }
 
-To use the skill in your own project without cloning this repo, download it directly:
+    Note: '''User accounts
 
-```bash
-mkdir -p .claude/skills/databricks-schema-cli
-curl -o .claude/skills/databricks-schema-cli/SKILL.md \
-  https://raw.githubusercontent.com/frizzleqq/databricks-schema/main/.claude/skills/databricks-schema-cli/SKILL.md
+@table_type: MANAGED
+@tags: {"domain": "identity"}'''
+}
+
+Ref "fk_org": "main"."users"."org_id" > "orgs"."organizations"."id"
 ```
 
-## Development
+How the model maps to DBML:
 
-Requires Python 3.11+ and [uv](https://github.com/astral-sh/uv).
+| Model | DBML |
+|---|---|
+| Catalog | `Project` (comment in its `Note`) |
+| Schema | `TableGroup` named after the schema (comment in its `note`); written for every schema, so empty schemas survive |
+| Table / column comment | `Note` / `note:` |
+| `nullable: false` | `not null` |
+| Primary key | `indexes { (...) [pk, name: '...'] }` |
+| Foreign key | `Ref "<name>": ... > ...` |
+| Complex types (`struct<...>`, `map<...>`) | double-quoted type |
+| `tags`, `grants`, `owner`, `table_type`, `storage_location` | trailing `@key: value` lines in the note (`@tags` and `@grants` are JSON objects) |
 
-```bash
-git clone https://github.com/frizzleqq/databricks-schema
-cd databricks-schema
-uv sync --all-groups  # includes pytest and ruff
-```
+Everything round-trips: reading a written file gives back exactly the extracted model. Hand-written DBML (e.g. exported from dbdiagram) is accepted too — tables without a schema land in `public`, column `pk` and inline `ref:` settings are understood, and constructs with no Unity Catalog equivalent (enums, sticky notes, other indexes, defaults, many-to-many refs, TableGroups that don't match a schema) are ignored. A directory may hold several `.dbml` files, but each schema may be defined in only one of them.
 
-```bash
-# Run tests
-uv run pytest
+### Catalog document
 
-# Lint
-uv run ruff check databricks_schema/ tests/
+Without `--output-dir`, `extract` prints a single `Catalog` document to stdout instead — the same schema shape nested under a `schemas` list, with the catalog name as a top-level field:
 
-# Format
-uv run ruff format databricks_schema/ tests/
+```yaml
+name: prod_catalog
+schemas:
+  - name: main
+    comment: Main production schema
+    tags:
+      env: prod
+    tables: [...]
 ```
 
 ## CLI Usage
@@ -169,9 +242,27 @@ uv run ruff format databricks_schema/ tests/
 databricks-schema [OPTIONS] COMMAND [ARGS]...
 ```
 
-`extract`, `diff`, `generate-sql`, and `diff-files` print progress messages (e.g. `Extracting
-catalog 'X'...`) to stderr as they run. Pass `--quiet` / `-q` to suppress them; results and
-errors are unaffected.
+`extract`, `diff`, `generate-sql`, and `diff-files` print progress messages (e.g. `Extracting catalog 'X'...`) to stderr as they run. Pass `--quiet` / `-q` to suppress them;
+results and errors are unaffected.
+
+### `list-catalogs`
+
+List all accessible catalogs:
+
+```bash
+databricks-schema list-catalogs
+```
+
+### `list-schemas`
+
+List schemas in a catalog:
+
+```bash
+databricks-schema list-schemas <catalog>
+```
+
+Both print one name per line by default. Pass `--format json` / `-f json` (or `--format yaml`) to
+get the names as a JSON array or YAML list instead.
 
 ### `extract`
 
@@ -181,7 +272,11 @@ Extract all schemas from a catalog to YAML files:
 databricks-schema extract <catalog> --output-dir ./schemas/
 ```
 
-Use `--format json` to write `.json` files instead of `.yaml`.
+Print to stdout instead of writing files (no `--output-dir`) — the catalog and its schemas are printed as a single `Catalog` document (`name` + `schemas: [...]`):
+
+```bash
+databricks-schema extract <catalog>
+```
 
 Extract specific schemas only:
 
@@ -196,16 +291,16 @@ databricks-schema extract <catalog>.<schema>
 databricks-schema extract <catalog>.<schema>.<table>
 ```
 
-Print to stdout instead of writing files (no `--output-dir`) — the catalog and its matching schemas are printed as a single `Catalog` document (`name` + `schemas: [...]`):
+Include Unity Catalog tags in the output (extra API calls per entity; excluded by default):
 
 ```bash
-databricks-schema extract <catalog> --schema main
+databricks-schema extract <catalog> --output-dir ./schemas/ --include-tags
 ```
 
-Include Unity Catalog tags in the output (extra API calls per entity; excluded by default):
+Include grants (principal → privileges) on schemas and tables. Only grants made directly on each object are extracted — privileges inherited from a parent catalog or schema are left out, so they don't show up repeatedly on every child (one extra API call per schema/table; excluded by default):
 
 ```bash
-databricks-schema extract <catalog> --output-dir ./schemas/ --include-tags
+databricks-schema extract <catalog> --output-dir ./schemas/ --include-grants
 ```
 
 Include additional metadata (`owner`, `storage_location`) in the output:
@@ -214,7 +309,13 @@ Include additional metadata (`owner`, `storage_location`) in the output:
 databricks-schema extract <catalog> --output-dir ./schemas/ --include-metadata
 ```
 
-Control the number of parallel workers (default: 4):
+Write a single DBML file (`./schemas/<catalog>.dbml`) instead — see [DBML](#dbml):
+
+```bash
+databricks-schema extract <catalog> --output-dir ./schemas/ --format dbml
+```
+
+Control the number of parallel workers (default: 4), useful for large schemas with many tags:
 
 ```bash
 databricks-schema extract <catalog> --output-dir ./schemas/ --workers 8
@@ -222,13 +323,27 @@ databricks-schema extract <catalog> --output-dir ./schemas/ --workers 8
 
 ### `diff`
 
-Compare the live catalog against previously extracted schema files (format auto-detected from the directory — YAML or JSON, not mixed):
+Compare the live catalog against previously extracted schema files (format auto-detected from the directory — YAML, JSON or DBML, not mixed):
 
 ```bash
 databricks-schema diff <catalog> ./schemas/
 ```
 
-Or compare two live catalogs directly, e.g. dev against prod — no local files needed. The second argument is treated as a directory if one exists at that path, otherwise as a catalog name:
+Exits with code `0` if no differences are found, `1` if there are — making it suitable for CI pipelines. Output example:
+
+```
+~ Schema: main [MODIFIED]
+  ~ Table: users [MODIFIED]
+    ~ Column: score [MODIFIED]
+        data_type: 'int' -> 'double'
+    + Column: phone [ADDED]
+  + Table: events [ADDED]
+- Schema: legacy [REMOVED]
+```
+
+Markers: `+` added in catalog, `-` removed from catalog, `~` modified.
+
+Or compare two live catalogs directly, e.g. dev against prod. The second argument is treated as a directory if one exists at that path, otherwise as a catalog name:
 
 ```bash
 databricks-schema diff dev_catalog prod_catalog
@@ -252,23 +367,22 @@ Include `owner` in the comparison (excluded by default):
 databricks-schema diff <catalog> ./schemas/ --include-metadata
 ```
 
-Exits with code `0` if no differences are found, `1` if there are — making it suitable for CI pipelines. Output example:
+Include direct grants in the comparison (excluded by default; without it, grants in the files are ignored):
 
-```
-~ Schema: main [MODIFIED]
-  ~ Table: users [MODIFIED]
-    ~ Column: score [MODIFIED]
-        data_type: 'int' -> 'double'
-    + Column: phone [ADDED]
-  + Table: events [ADDED]
-- Schema: legacy [REMOVED]
+```bash
+databricks-schema diff <catalog> ./schemas/ --include-grants
 ```
 
-Markers: `+` added in catalog, `-` removed from catalog, `~` modified.
+Get the same comparison as structured JSON or YAML instead of the `+`/`-`/`~` tree (exit codes unchanged) — useful for scripts or agents:
+
+```bash
+databricks-schema diff <catalog> ./schemas/ --format json
+databricks-schema diff <catalog> ./schemas/ --format yaml
+```
 
 ### `generate-sql`
 
-Generate Databricks Spark SQL statements to bring the live catalog in line with local schema files (format auto-detected, YAML or JSON, not mixed). Statements are printed to stdout by default:
+Generate Databricks Spark SQL statements to bring the live catalog in line with local schema files (format auto-detected — YAML, JSON or DBML, not mixed). Statements are printed to stdout by default:
 
 ```bash
 databricks-schema generate-sql <catalog> ./schemas/
@@ -280,7 +394,7 @@ Write one `.sql` file per schema to a directory instead:
 databricks-schema generate-sql <catalog> ./schemas/ --output-dir ./migrations/
 ```
 
-Destructive statements (`DROP SCHEMA`, `DROP TABLE`, `DROP COLUMN`) are emitted as SQL comments by default. Pass `--allow-drop` to emit them as executable statements:
+Destructive statements (`DROP SCHEMA`, `DROP TABLE`, `DROP COLUMN`, `REVOKE`) are emitted as SQL comments by default. Pass `--allow-drop` to emit them as executable statements:
 
 ```bash
 databricks-schema generate-sql <catalog> ./schemas/ --allow-drop
@@ -304,6 +418,12 @@ Include `owner` in the comparison (excluded by default):
 databricks-schema generate-sql <catalog> ./schemas/ --include-metadata
 ```
 
+Include direct grants in the comparison, emitting `GRANT` for privileges in the files but not live and (commented-out unless `--allow-drop`) `REVOKE` for privileges live but not in the files:
+
+```bash
+databricks-schema generate-sql <catalog> ./schemas/ --include-grants
+```
+
 ### `validate`
 
 Validate local schema files for structural integrity (no Databricks connection needed):
@@ -312,29 +432,28 @@ Validate local schema files for structural integrity (no Databricks connection n
 databricks-schema validate ./schemas/
 ```
 
+Pass `--format json` or `--format yaml` for an `{"issues": [...]}` document instead of `OK`/`ERROR:` text.
+
 ### `diff-files`
 
 Compare two local directories of schema files (no Databricks connection needed). Same output,
-exit codes, and `--schema` / `--include-metadata` flags as `diff`:
+exit codes, `--format json`/`yaml`, and `--schema` / `--include-metadata` / `--include-grants` flags as `diff`:
 
 ```bash
 databricks-schema diff-files ./schemas-prod/ ./schemas-test/
 ```
 
-### `list-catalogs`
-
-List all accessible catalogs:
-
-```bash
-databricks-schema list-catalogs
-```
-
-### `list-schemas`
+### `json-schema`
 
-List schemas in a catalog:
+Print the JSON Schema for one of the CLI's structured output shapes — no Databricks connection
+needed. Useful before parsing `--format json`/`yaml` output (or `extract`'s output)
+programmatically. Add `--format yaml` to render the schema itself as YAML:
 
 ```bash
-databricks-schema list-schemas <catalog>
+databricks-schema json-schema catalog    # extract's stdout Catalog document
+databricks-schema json-schema schema     # one per-schema extract file
+databricks-schema json-schema diff       # diff / diff-files --format json/yaml
+databricks-schema json-schema validate   # validate --format json/yaml
 ```
 
 ## Python Library Usage
@@ -353,6 +472,10 @@ catalog = extractor.extract_catalog("my_catalog", schema_filter=["main", "raw"])
 # Include Unity Catalog tags (extra API calls per entity; excluded by default)
 catalog = extractor.extract_catalog("my_catalog", include_tags=True)
 
+# Include direct (non-inherited) grants on schemas and tables; compare them with
+# diff_schemas(..., include_grants=True) and friends
+catalog = extractor.extract_catalog("my_catalog", include_grants=True)
+
 # Include additional metadata (owner, storage_location)
 catalog = extractor.extract_catalog("my_catalog", include_metadata=True)
 
@@ -363,7 +486,12 @@ yaml_text = catalog_to_yaml(catalog)
 schema = schema_from_yaml(open("schemas/main.yaml").read())
 print(schema.tables[0].columns)
 
-# Compare live catalog against local YAML files
+# DBML: one document per catalog
+from databricks_schema import catalog_from_dbml, catalog_to_dbml
+Path("schemas/my_catalog.dbml").write_text(catalog_to_dbml(catalog))
+catalog = catalog_from_dbml(Path("schemas/my_catalog.dbml").read_text())
+
+# Compare live catalog against local YAML files (fmt="json" / fmt="dbml" for other formats)
 result = diff_catalog_with_dir(catalog, Path("./schemas/"))
 if result.has_changes:
     for schema_diff in result.schemas: