modelith-dbt 0.1.0__tar.gz

modelith_dbt-0.1.0/.claude/launch.json

@@ -0,0 +1,17 @@
+{
+  "version": "0.0.1",
+  "configurations": [
+    {
+      "name": "canvas",
+      "runtimeExecutable": "uv",
+      "runtimeArgs": ["run", "mdl", "serve", "-m", "model", "--port", "4800"],
+      "port": 4800
+    },
+    {
+      "name": "ibor",
+      "runtimeExecutable": "uv",
+      "runtimeArgs": ["run", "mdl", "serve", "-m", "/tmp/pension-ibor/model", "--port", "4802"],
+      "port": 4802
+    }
+  ]
+}

modelith_dbt-0.1.0/.gitignore

@@ -0,0 +1,26 @@
+__pycache__/
+*.pyc
+.venv/
+.pytest_cache/
+.ruff_cache/
+.hypothesis/
+*.egg-info/
+dist/
+build/
+.coverage
+htmlcov/
+# generated dbt projects live under target/ by default
+target/
+# canvas toolchain (the production build IS committed, under packages/server/.../static)
+canvas/node_modules/
+# vscode extension toolchain (source is committed; artifacts are not)
+vscode/node_modules/
+vscode/dist/
+vscode/*.vsix
+
+# local demo workspace (own git repo inside; never part of modelith)
+demo/
+
+# duckdb build artifacts
+*.duckdb
+*.duckdb.wal

modelith_dbt-0.1.0/LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

modelith_dbt-0.1.0/PKG-INFO

@@ -0,0 +1,269 @@
+Metadata-Version: 2.5
+Name: modelith-dbt
+Version: 0.1.0
+Summary: Ontology-anchored, git-native data modeling tool for dbt-core teams
+License: Apache-2.0
+License-File: LICENSE
+Requires-Python: >=3.11
+Requires-Dist: dbt-artifacts-parser>=0.5
+Requires-Dist: fastapi>=0.110
+Requires-Dist: jinja2>=3.1
+Requires-Dist: pydantic>=2.6
+Requires-Dist: pygls>=2.0
+Requires-Dist: pyshacl>=0.25
+Requires-Dist: rdflib>=7.0
+Requires-Dist: ruamel-yaml>=0.18
+Requires-Dist: typer>=0.12
+Requires-Dist: uvicorn>=0.29
+Description-Content-Type: text/markdown
+
+# Modelith
+
+Ontology-anchored, git-native data modeling for dbt teams. Design your model as an
+entity-relationship diagram, generate contract-enforced dbt from it, reverse an
+existing warehouse back into a model, and catch drift before it ships. The model
+lives in git as plain YAML, so every surface (the `mdl` CLI, the web canvas, the
+VS Code extension, the drift bot) is a client and none of them owns state.
+
+[![tests](https://img.shields.io/badge/tests-215%20passing-brightgreen)](#development)
+[![python](https://img.shields.io/badge/python-3.11%2B-blue)](pyproject.toml)
+[![dbt](https://img.shields.io/badge/dbt--core-1.7%2B-orange)](https://www.getdbt.com/)
+[![license](https://img.shields.io/badge/license-Apache--2.0-lightgrey)](#license)
+[![code style: ruff](https://img.shields.io/badge/lint-ruff-black)](https://docs.astral.sh/ruff/)
+
+![Modelith canvas: a seven-entity pension IBoR model with crow's-foot relationships](docs/assets/canvas.png)
+
+## Why
+
+Warehouse teams keep their meaning in three disconnected places: an ER tool that
+never sees production, dbt SQL that drifts from the design, and a governance catalog
+nobody edits. Modelith puts one model in git and makes every tool read and write it.
+You get a real data modeler (entities, attributes, keys, domains, relationships,
+subtypes) that generates dbt you can actually run, verifies the warehouse still
+matches the model, and reads a legacy dbt project (or an erwin export) back into a
+clean logical model.
+
+Nothing here is a mockup. Every capability below is exercised by the test suite and,
+where it touches SQL, by a real `dbt build` against DuckDB.
+
+## Install
+
+```bash
+uv tool install modelith-dbt    # puts `mdl` on your PATH
+# or: pipx install modelith-dbt
+mdl --help
+```
+
+Modelith runs in the same environment as dbt-core (Python 3.11+).
+
+## Quickstart
+
+```bash
+mdl init my-model --name my_model          # scaffold a model repo
+mdl new entity customer -m my-model        # add an entity (mints ULIDs)
+mdl validate -m my-model                   # schema, refs, naming, ontology
+mdl generate -m my-model -o warehouse      # emit contract-enforced dbt
+mdl serve -m my-model                       # open the visual canvas
+```
+
+`mdl generate` writes dbt models with protected regions, a `schema.yml` carrying
+contract constraints, and a three-way merge on regeneration, so your hand edits and
+the generated blocks both survive.
+
+```text
+$ mdl validate -m my-model
+validation passed
+
+$ mdl generate -m my-model -o warehouse
+  created          models/customer.sql
+  created          models/schema.yml
+wrote 2 files to warehouse
+```
+
+## The visual canvas
+
+`mdl serve` opens an ER canvas in the browser (also embeddable as a VS Code webview).
+It is a full editor, not a viewer: drag between entities to draw a relationship, edit
+attributes and keys inline, align a term to an ontology, and commit from a git panel.
+Every edit is a typed, comment-preserving mutation with optimistic concurrency, so two
+people can work the same repo without clobbering each other.
+
+![The entity inspector, showing attributes, named keys, an enumerated domain, user-defined properties, and relationships](docs/assets/inspector.png)
+
+The inspector above shows one entity carrying an enumerated domain (`asset_class`),
+a named primary key and a unique key, user-defined properties, and its relationships,
+all first-class in the model.
+
+## In VS Code
+
+Install the extension from `vscode/modelith-vscode-0.1.0.vsix`:
+
+```bash
+code --install-extension vscode/modelith-vscode-0.1.0.vsix
+```
+
+The extension does not bundle its own copy of the canvas. It launches `mdl serve`
+and embeds the live canvas, so whatever the CLI understands, the editor shows. The
+model files stay in git; the extension is just another client over them.
+
+**Side by side: YAML, SQL, and the canvas.** Right-click a model file in the
+Explorer (or in the open editor) and choose **Modelith: Open Model Preview to the
+Side**. The canvas opens in a split beside your file and follows the active editor:
+open `instrument.yaml` and the canvas centers on that entity; switch to a generated
+`.sql` file and the preview tracks it. Save the YAML and the preview re-renders. You
+read and edit the model as text on the left, watch the diagram update on the right,
+and keep the generated dbt SQL a tab away, all in one window.
+
+![Split view in VS Code: the model YAML on the left, the live canvas preview on the right](docs/assets/vscode-split.png)
+
+For a full editing session, **Modelith: Open Canvas** opens the editable canvas as
+its own tab (drag-to-connect relationships, inline attribute and key editing,
+git commit panel). The preview-to-the-side is the read-along companion while you work
+in text; the full canvas is where you drive structural edits.
+
+What the extension adds on top of the canvas:
+
+- **Diagnostics on save.** `mdl validate` runs when you save a model YAML and surfaces
+  `MDL-*` findings in the Problems panel, mapped to the file that declares the issue.
+- **Language server.** `mdl lsp` drives drift and contract diagnostics on the generated
+  dbt files, hover cards (glossary term, ontology IRI, owner), and code actions (adopt a
+  column, lift a model, unmanage, declare a relationship).
+- **Commands.** Generate the dbt project, check drift, lint and fix naming, scaffold a new
+  entity, vendor an ontology, emit the semantic layer, all from the command palette.
+- **YAML completion.** JSON Schemas exported from the model are registered with the Red Hat
+  YAML extension, so authoring the YAML by hand is schema-checked and autocompleted.
+- **Devcontainer-ready.** The extension declares `extensionKind: ["workspace"]`, so in a
+  devcontainer the server and the `mdl` toolchain run next to dbt and your warehouse
+  credentials, not on the laptop.
+
+Detection resolves `mdl` in order: an explicit `modelith.mdlPath` setting, a project
+`.venv`, `mdl` on PATH, the active conda or virtualenv, then the common per-user install
+locations. A standard `uv tool install modelith-dbt` needs no configuration.
+
+## What it models
+
+Modelith represents the core data-modeling taxonomy as first-class, git-tracked objects:
+
+| Concept | Support |
+|---|---|
+| Conceptual / logical / physical layers | Separate object kinds, referenced by immutable ULID |
+| Entities and attributes | Name, domain, role (business key / surrogate / attribute / measure), nullability |
+| Relationships | Four cardinalities, identifying vs non-identifying, optionality, crow's-foot rendering |
+| Named keys | Primary, alternate, unique, and index key groups with ordered, composite members |
+| Domains and reference data | Reusable domains, inline enumerations, and shared code sets that emit dbt `accepted_values` tests |
+| Subtypes and supertypes | Category clusters with a discriminator and a physical materialization strategy (single-table or table-per-subtype) |
+| User-defined properties | Extensible metadata on any object, flowing through to dbt `meta` |
+| Subject areas | Diagram partitioning with color grouping on the canvas |
+| Ontology alignment | Four-layer stack (industry / core / domain / specialised), SKOS predicates, FIBO out of the box |
+| Design patterns | SCD2 and Data Vault (hub / link / satellite), emitting working SQL, not stubs |
+
+## What it does
+
+**Forward engineering.** Generate dbt-core models with enforced contracts, primary and
+unique key constraints, relationship tests, and platform-specific types for DuckDB,
+Snowflake, Redshift, Iceberg, and Trino. Regeneration runs a three-way merge so hand
+edits survive.
+
+**Reverse engineering.** Point `mdl reverse` at a compiled dbt project (`manifest.json`
+plus `catalog.json`) and get a logical model back. It excludes staging and intermediate
+models, collapses SCD2 column triples into a pattern, strips surrogate keys, detects Data
+Vault structures, and infers relationships from tests and naming, recording every decision
+in a reviewable ledger.
+
+**Drift detection.** `mdl drift` compares the committed model to a compiled warehouse and
+classifies each difference as breaking, additive, or cosmetic, with a CI gate mode and a
+reconcile mode. A 400-model breaking change classifies in under thirty seconds.
+
+**Governance sync.** A neutral governance graph maps to an external catalog through a
+customer-owned Jinja profile. A Collibra adapter ships, along with OpenLineage emission
+and a conformance kit that validates a bespoke mapping in CI.
+
+**Semantic layer.** Emit MetricFlow semantic models and metrics, or OSI (version-isolated),
+with joinability and fan-out validation.
+
+**Collaboration.** A structural, ULID-keyed git merge driver lets two people add different
+attributes to the same entity and merge cleanly. A change classifier routes pull requests
+to the right reviewers, a debt valve records engineer-owned SQL with an expiry, and a
+git-native glossary app lets subject-matter experts propose definitions as pull requests
+without ever seeing git or a CLI.
+
+## Surfaces
+
+| Surface | What it is |
+|---|---|
+| `mdl` CLI | The full command set: init, validate, lint, generate, reverse, drift, serve, glossary, ontology, emit, export, import, gov, and more |
+| Web canvas | `mdl serve` opens the ER editor; state stays in git |
+| VS Code extension | Canvas beside your YAML (follows the active editor), full canvas tab, diagnostics on save, generate / drift / lint commands, YAML completion, devcontainer-ready. See [In VS Code](#in-vs-code). |
+| Language server | `mdl lsp` (one server for VS Code, Cursor, Windsurf, JetBrains, and CI): drift and contract diagnostics on the dbt files, hover cards, code actions |
+| Glossary app | `mdl glossary` serves a narrow, git-native glossary surface for subject-matter experts |
+
+## CLI reference
+
+```text
+mdl init [--workspace] [--git-hooks]              scaffold a model repo or full topology
+mdl new entity|term|subject-area <name>           add an object (ULIDs minted)
+mdl delete entity <name> [--cascade]              remove an object, safely
+mdl validate [--format json]                      schema, refs, ontology, naming
+mdl lint [--fix]                                  naming-standards lint
+mdl generate [--target] [--dry-run]               emit the dbt project
+mdl reverse --project <manifest|schema.yml>       lift a dbt project into a model
+mdl drift --manifest <m> [--check|--reconcile]    compare model to compiled warehouse
+mdl serve [--read-only]                           web canvas + read API
+mdl glossary [--read-only]                        SME glossary app
+mdl ontology search|check|promote|vendor          vocabulary and alignment lifecycle
+mdl emit semantic --format metricflow|osi         semantic layer
+mdl export json-schema|rdf|shacl                  interchange out
+mdl import osi|erwin                              interchange in
+mdl gov plan|apply|pull|publish|import            catalog sync
+mdl classify | unmanage | debt | decisions        collaboration and review
+```
+
+## Layout
+
+```
+packages/
+  core/           IR, ULID identity, YAML round-trip, validator, merge engine (no in-repo deps)
+  emit-dbt/       dbt-core emitter, platform adapters, SCD2 macros
+  reverse/        manifest + catalog reader, drift, reverse engineering, decision ledger, erwin import
+  ontology/       vocabulary registry, four-layer validation, RDF/OWL + SHACL export
+  emit-semantic/  MetricFlow + OSI, joinability validation
+  governance/     governance graph, Jinja mapping DSL, adapter SPI, conformance kit, OpenLineage
+  adapters/
+    collibra/     Collibra governance adapter
+  server/         read API (FastAPI) + hosts the canvas build
+  lsp/            language server (pygls)
+  cli/            mdl
+canvas/           web canvas source (Vite + React + React Flow)
+vscode/           VS Code extension (TypeScript + esbuild)
+profiles/         CI workflow templates and reference governance profiles
+```
+
+The layering rule is enforced: `core` depends on nothing else in the repo, and
+`emit-dbt` depends only on `core`.
+
+## Development
+
+```bash
+uv sync
+uv run pytest                 # 215 tests
+uv run ruff check packages/
+```
+
+Build the canvas and the extension (Node 20):
+
+```bash
+cd canvas && npm install && npm run build      # emits into the server static dir
+cd vscode && npm install && npm run build && npm run package   # produces the .vsix
+```
+
+## Documentation
+
+- [`docs/adoption-guide.md`](docs/adoption-guide.md): a step-by-step runbook for taking a
+  team from an empty repo to a working practice (platform, then SMEs, architects, engineers).
+- [`docs/collaboration-model.md`](docs/collaboration-model.md): the operating model, review
+  routes, and the merge driver.
+- [`modelith-spec.md`](modelith-spec.md): the full build specification.
+
+## License
+
+Apache-2.0.