dataproduct-builder-skills 1.2.1__tar.gz

dataproduct_builder_skills-1.2.1/PKG-INFO

@@ -0,0 +1,23 @@
+Metadata-Version: 2.4
+Name: dataproduct-builder-skills
+Version: 1.2.1
+Summary: Agent skills for designing and building Vulcan/DataOS data products — run via uvx/pipx to scaffold .cursor/.claude/.codex/.github skills and docs into any project.
+License: MIT
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+
+# dataproduct-builder-skills (Python)
+
+Python port of the `dataproduct-builder-skills` npm scaffolder — run it via `uvx`/`pipx` instead of `npx`.
+
+```bash
+uvx --from . dataproduct-builder-skills snowflake
+# or
+pipx run --spec . dataproduct-builder-skills snowflake
+```
+
+Before building/testing, vendor the shared `skills/` and `dpbs-docs/` content from the repo root:
+
+```bash
+python3 scripts/vendor_data.py
+```

dataproduct_builder_skills-1.2.1/README.md

@@ -0,0 +1,15 @@
+# dataproduct-builder-skills (Python)
+
+Python port of the `dataproduct-builder-skills` npm scaffolder — run it via `uvx`/`pipx` instead of `npx`.
+
+```bash
+uvx --from . dataproduct-builder-skills snowflake
+# or
+pipx run --spec . dataproduct-builder-skills snowflake
+```
+
+Before building/testing, vendor the shared `skills/` and `dpbs-docs/` content from the repo root:
+
+```bash
+python3 scripts/vendor_data.py
+```

dataproduct_builder_skills-1.2.1/pyproject.toml

@@ -0,0 +1,20 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "dataproduct-builder-skills"
+version = "1.2.1"
+description = "Agent skills for designing and building Vulcan/DataOS data products — run via uvx/pipx to scaffold .cursor/.claude/.codex/.github skills and docs into any project."
+readme = "README.md"
+license = { text = "MIT" }
+requires-python = ">=3.9"
+
+[project.scripts]
+dataproduct-builder-skills = "dataproduct_builder_skills.cli:main"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+dataproduct_builder_skills = ["data/**/*"]

dataproduct_builder_skills-1.2.1/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

dataproduct_builder_skills-1.2.1/src/dataproduct_builder_skills/cli.py

@@ -0,0 +1,215 @@
+#!/usr/bin/env python3
+"""Scaffolds Vulcan/DataOS agent skills + docs into the current project.
+
+Python port of bin/create.js — same behavior, same prompts, same output
+layout, so it can be run via `uvx`/`pipx run` alongside the npm package's
+`npx` flow.
+"""
+import shutil
+import sys
+from pathlib import Path
+
+RESET, GREEN, CYAN, YELLOW, BOLD, DIM, RED = (
+    "\x1b[0m", "\x1b[32m", "\x1b[36m", "\x1b[33m", "\x1b[1m", "\x1b[2m", "\x1b[31m",
+)
+
+
+def log(msg=""):
+    print(msg)
+
+
+def ok(msg):
+    log(f"  {GREEN}✓{RESET}  {msg}")
+
+
+def info(msg):
+    log(f"  {CYAN}→{RESET}  {msg}")
+
+
+def warn(msg):
+    log(f"  {YELLOW}!{RESET}  {msg}")
+
+
+def err(msg):
+    log(f"  {RED}✗{RESET}  {msg}")
+
+
+DATA_DIR = Path(__file__).resolve().parent / "data"
+
+ALLOWED_ENGINES = ["databricks", "fabric", "mssql", "postgres", "snowflake", "spark", "trino"]
+
+IDE_OPTIONS = [
+    ("Cursor", ".cursor"),
+    ("Claude Code", ".claude"),
+    ("Codex", ".codex"),
+    ("VS Code (Copilot)", ".github"),
+]
+
+
+def sync_dir(src: Path, dest: Path):
+    """Mirrors src into dest — copies/overwrites everything, and deletes any
+    dest entry no longer present in src."""
+    dest.mkdir(parents=True, exist_ok=True)
+    src_names = {p.name for p in src.iterdir()}
+
+    for entry in dest.iterdir():
+        if entry.name not in src_names or entry.name == ".git":
+            if entry.is_dir():
+                shutil.rmtree(entry)
+            else:
+                entry.unlink()
+
+    for entry in src.iterdir():
+        if entry.name == ".git":
+            continue
+        d = dest / entry.name
+        if entry.is_dir():
+            sync_dir(entry, d)
+        else:
+            shutil.copyfile(entry, d)
+
+
+def count_files(d: Path) -> int:
+    return sum(1 for p in d.rglob("*") if p.is_file())
+
+
+def rewrite_path_references(d: Path, old: str, new: str):
+    for p in d.rglob("*.md"):
+        content = p.read_text(encoding="utf-8")
+        if old in content:
+            p.write_text(content.replace(old, new), encoding="utf-8")
+
+
+def prompt_ides() -> list[str]:
+    log(f"{BOLD}Which IDE(s) are you using?{RESET} {DIM}(comma-separated for multiple, e.g. 1,2){RESET}")
+    log()
+    for i, (label, _) in enumerate(IDE_OPTIONS, 1):
+        log(f"  {DIM}{i}{RESET}  {label}")
+    log(f"  {DIM}{len(IDE_OPTIONS) + 1}{RESET}  All")
+    log()
+
+    answer = input(f"Enter number(s) (1–{len(IDE_OPTIONS) + 1}): ").strip()
+
+    try:
+        nums = [int(s.strip()) for s in answer.split(",")]
+    except ValueError:
+        nums = []
+
+    if not nums or any(n < 1 or n > len(IDE_OPTIONS) + 1 for n in nums):
+        err(f'Invalid selection "{answer}". Use numbers 1–{len(IDE_OPTIONS) + 1}, comma-separated.')
+        sys.exit(1)
+
+    if (len(IDE_OPTIONS) + 1) in nums:
+        folders = [folder for _, folder in IDE_OPTIONS]
+    else:
+        folders = [IDE_OPTIONS[n - 1][1] for n in nums]
+
+    labels = ", ".join(label for label, folder in IDE_OPTIONS if folder in folders)
+    log()
+    info(f"IDE(s): {BOLD}{labels}{RESET}")
+    log()
+    return folders
+
+
+def main():
+    target_dir = Path.cwd()
+    examples_dir = DATA_DIR / "dpbs-docs" / "vulcan-examples"
+
+    valid_engines = (
+        [e for e in ALLOWED_ENGINES if (examples_dir / e).exists()]
+        if examples_dir.exists()
+        else []
+    )
+
+    cli_engine = sys.argv[1].lower() if len(sys.argv) > 1 else None
+    if cli_engine and cli_engine not in valid_engines:
+        err(f'Unknown engine: "{cli_engine}"')
+        log()
+        log(f"  Available engines: {', '.join(valid_engines)}")
+        log()
+        sys.exit(1)
+
+    log()
+    log(f"{BOLD}dataproduct-builder-skills{RESET} — scaffolding skills + docs")
+    log()
+
+    ide_folders = prompt_ides()
+
+    if cli_engine:
+        engine = cli_engine
+        info(f"Engine: {BOLD}{engine}{RESET} (from CLI argument)")
+    else:
+        engine = None
+        info(f"Engine: {BOLD}all{RESET} (default)")
+    log()
+
+    # ── skills ──
+    skills_src = DATA_DIR / "skills"
+    if not skills_src.exists():
+        warn("skills/ directory not found in package — skipping")
+    else:
+        skills = [p.name for p in skills_src.iterdir() if p.is_dir()]
+        for ide_folder in ide_folders:
+            for skill in skills:
+                src = skills_src / skill
+                dest = target_dir / ide_folder / "skills" / skill
+                existed = dest.exists()
+                sync_dir(src, dest)
+                rewrite_path_references(dest, "dpbs-docs/dataos", "dpbs-docs/vulcan-docs")
+                ok(f"{'updated' if existed else 'created'}  {ide_folder}/skills/{skill}/")
+
+    # ── dpbs-docs (non-examples) ──
+    docs_src = DATA_DIR / "dpbs-docs"
+    docs_dest = target_dir / "dpbs-docs"
+
+    if not docs_src.exists():
+        warn("dpbs-docs/ directory not found in package — skipping")
+    else:
+        docs_dest.mkdir(parents=True, exist_ok=True)
+        for entry in docs_src.iterdir():
+            if not entry.is_dir() or entry.name == "vulcan-examples":
+                continue
+            dest = docs_dest / entry.name
+            existed = dest.exists()
+            sync_dir(entry, dest)
+            n = count_files(entry)
+            ok(f"{'updated' if existed else 'created'}  dpbs-docs/{entry.name}/  ({n} file{'' if n == 1 else 's'})")
+
+        loose_src = [p for p in docs_src.iterdir() if p.is_file()]
+        loose_names = {p.name for p in loose_src}
+        for entry in list(docs_dest.iterdir()):
+            if entry.is_file() and entry.name not in loose_names:
+                entry.unlink()
+        for entry in loose_src:
+            dest = docs_dest / entry.name
+            existed = dest.exists()
+            shutil.copyfile(entry, dest)
+            ok(f"{'updated' if existed else 'created'}  dpbs-docs/{entry.name}")
+
+    # ── dpbs-docs/vulcan-examples (filtered or all) ──
+    if examples_dir.exists():
+        engines_to_copy = [engine] if engine else valid_engines
+        for eng in engines_to_copy:
+            src = examples_dir / eng
+            dest = docs_dest / "vulcan-examples" / eng
+            existed = dest.exists()
+            sync_dir(src, dest)
+            n = count_files(dest)
+            ok(f"{'updated' if existed else 'created'}  dpbs-docs/vulcan-examples/{eng}/  ({n} file{'' if n == 1 else 's'})")
+
+    log()
+    log(f"{GREEN}{BOLD}Done!{RESET}  Your project now has:")
+    log()
+    for ide_folder in ide_folders:
+        info(f"{ide_folder}/skills/design-data-product/")
+        info(f"{ide_folder}/skills/build-data-product/")
+    info(f"dpbs-docs/vulcan-examples/{engine or '{all engines}'}/")
+    info(f"dpbs-docs/vulcan-*.whl  — install: pip install \"dpbs-docs/vulcan-*.whl[{engine or 'ENGINE'}]\"")
+    log()
+    log("Ask the agent to use the skills — e.g.:")
+    log(f'  {CYAN}"design a data product for daily revenue by customer segment"{RESET}')
+    log()
+
+
+if __name__ == "__main__":
+    main()

dataproduct_builder_skills-1.2.1/src/dataproduct_builder_skills/data/dpbs-docs/dataos-philosophy/about.md

@@ -0,0 +1,37 @@
+# About
+
+## **Why "*Vulcan*"**
+
+We're a semantics-core product, so the name isn't incidental. ***Vulcan*** is the Roman god of the forge. Not of fire in the abstract, but of **fire put to work**. The forge is where raw ore becomes something precise and load-bearing.
+
+That's the same transformation the Data Product Stack or *Vulcan* performs: raw SQL and Python go in, certified data products come out. The forge doesn't just heat things, but shapes them under controlled conditions, removes impurities, and produces something you can depend on structurally.
+
+Scholars trace *Vulcanus* back to its roots, meaning lustre, brilliance, or glare.
+
+> *Vulcan* is named for that quality: not the fire itself, but what fire reveals.
+
+Visibility into what's running, what's queued, what failed, and why. A blacksmith doesn't guess whether the metal is ready because he can see it. Vulcan gives data teams the same ability: to see exactly what's happening at every step, before bad data reaches anyone who acts on it.
+
+---
+
+## What is *Vulcan*
+
+*Vulcan* is a data product stack built for developers with a mission to enable data for direct business impact. The stack covers the **full journey from raw code to business-ready data products.** It handles CI/CD, testing, data quality enforcement, and API generation in **one coherent system**.
+
+---
+
+## **The *Vulcan* Mental Model**
+
+Think of *Vulcan* as a contract system. Each model in your project makes implicit promises: that it will produce a certain shape of data, that its dependencies are satisfied, and that the output meets defined quality thresholds.
+
+*Vulcan* makes those promises explicit and verifiable. When a model passes all its checks, it **becomes a "certified" data product**. That certification is what allows downstream consumers, whether APIs, semantic models, or business users, to trust what they receive **without inspecting how it was produced**.
+
+This means you can reason about any part of the system in isolation.
+
+> A semantic model doesn't need to know how its source models are implemented, only that they've been validated. An API doesn't need to know what the semantic layer ingested, only that it passed.
+
+This is the same principle that makes interfaces useful in software engineering: you hide the implementation and expose a verified contract.
+
+## What's In It for Developers
+
+[https://tmdc-io.github.io/vulcan-book/](https://tmdc-io.github.io/vulcan-book/)

dataproduct_builder_skills-1.2.1/src/dataproduct_builder_skills/data/dpbs-docs/dataos-philosophy/components/advanced-features/index.md

@@ -0,0 +1,4 @@
+# Advanced Features
+
+- [Macros](macros.md)
+- [Signals](signals.md)

dataproduct_builder_skills-1.2.1/src/dataproduct_builder_skills/data/dpbs-docs/dataos-philosophy/components/advanced-features/macros.md

@@ -0,0 +1,24 @@
+# Macros
+
+## **Symbiotic Pillars: Abstraction and Reusability**
+
+SQL is deliberately simple, which makes it powerful. It is readable, reviewable, and accessible to people who are not professional software engineers. But simplicity has a cost when your models start to share logic.
+
+At a small scale, it is tolerable, but as the project grows, it becomes the primary source of inconsistency. Reusability is one of the most important capabilities in data management: the more reusable you make any logic or data asset, the simpler and more cost-effective the data product becomes. Macros are how *Vulcan* applies that principle at the transformation layer.
+
+The philosophy is the same one that makes good software maintainable: don't embed logic directly in every place it is needed, encode it once where it can be trusted and referenced everywhere it is relevant.
+
+When the business rule changes, you change it in one place. Every model that references it picks up the change automatically, without a search-and-replace across the project. This is the same principle and design pattern that powers the semantic layer: define metrics once, use them everywhere, own them centrally.
+
+> With Macros, the pattern is applied one layer down, where the transformation logic that feeds those metrics is built. **Even before modeling begins, existing assets and logic should be discovered for reusability.**
+
+Macros make the mechanism that makes transformation logic discoverable and reusable by design rather than by convention. The two systems *Vulcan* supports, native Vulcan macros and Jinja, serve different working styles, but both exist in service of the same goal: logic that lives in one authoritative place, carries forward cleanly as the project scales, and never has to be reconciled across the files that duplicated it.
+
+## Macros in *Vulcan*
+
+See the reference documentation for:
+
+- [Overview](https://tmdc-io.github.io/vulcan-book/components/advanced-features/macros/overview/)
+- [Variables](https://tmdc-io.github.io/vulcan-book/components/advanced-features/macros/variables/)
+- [Built-in Macros](https://tmdc-io.github.io/vulcan-book/components/advanced-features/macros/built_in/)
+- [Jinja](https://tmdc-io.github.io/vulcan-book/components/advanced-features/macros/jinja/)

dataproduct_builder_skills-1.2.1/src/dataproduct_builder_skills/data/dpbs-docs/dataos-philosophy/components/advanced-features/signals.md

@@ -0,0 +1,17 @@
+# Signals
+
+## **Event-Driven Data Products**
+
+The default mental model for data pipelines is a chain of schedules. Model A runs at midnight, Model B runs at 1 AM, Model C runs at 2 AM: each one assumes the previous one finished and the data is ready. This is a pipeline-first design: execution order is defined by time, not by the state of the data itself. It works until it doesn't.
+
+When something is off, the entire chain produces incomplete output or fails, because nothing in the design checked whether the data was actually ready. The **schedule was a proxy for readiness**.
+
+But what happens when you put data in the middle instead of the clock? Rather than asking "has enough time passed?", a *Signal* asks "is the data in the right state?"
+
+That inversion is not a scheduling feature anymore, but an architectural shift in what the model treats as its **execution precondition**. The model becomes responsive to its inputs rather than oblivious to them.
+
+It waits not because a cron told it to wait, but because the data it depends on has not yet **met the condition it declared**. When that condition is met, execution proceeds. Not a moment before or after. This is what it means to build a data product that is genuinely data-driven.
+
+## Signals Overview
+
+[https://tmdc-io.github.io/vulcan-book/components/advanced-features/signals/](https://tmdc-io.github.io/vulcan-book/components/advanced-features/signals/)

dataproduct_builder_skills-1.2.1/src/dataproduct_builder_skills/data/dpbs-docs/dataos-philosophy/components/audits.md

@@ -0,0 +1,20 @@
+# Audits
+
+## **Observability vs. Auditability**
+
+Auditability is what converts observability from a reporting tool into a quality guarantee.
+
+- Observability is the ability to see what your system is doing over time. It answers the question: *What is happening?*
+- Auditability is the ability to verify that your data meets a defined standard before it is used. It answers a different question: *Is this data correct enough to act on?*
+
+Both matter, but they operate at different points in the data lifecycle, and substituting one for the other is how bad data reaches production with a green light.
+
+Most data teams have observability. They have monitoring, alerting, and lineage graphs (even though not end-to-end) that tell them when a pipeline runs and whether it succeeded. What they often don't have is auditability: a formal, enforced standard that the data itself must meet before it is promoted.
+
+The distinction is not subtle in its consequences. Observability tells you the pipeline ran. Auditability tells you the output was valid. Passive vs Active. A pipeline can run successfully and produce wrong data. An audit catches that while a monitoring dashboard does not.
+
+*Vulcan*'s audit layer sits at precisely that gap, between execution and promotion, and it is blocking by design. Of course, blocking is not convenient and may cause intermediate downstream stress, but it's better than a system that continues on bad data, which is a system that cannot be trusted. It's a loss of long-term trust as well as prolonged downstream distress.
+
+## Audits in *Vulcan*
+
+[https://tmdc-io.github.io/vulcan-book/components/audits/audits/](https://tmdc-io.github.io/vulcan-book/components/audits/audits/)

dataproduct_builder_skills-1.2.1/src/dataproduct_builder_skills/data/dpbs-docs/dataos-philosophy/components/checks.md

@@ -0,0 +1,27 @@
+# Checks
+
+## **Guardrails as Platform Enforcement**
+
+There is a measurable difference between a team that follows good practices because they care and a team that follows good practices because the platform makes it structurally difficult to do otherwise.
+
+- The first depends on **individual discipline, which is unscalable and unevenly distributed**.
+- The second is a property of the system, which **holds regardless of who** is on the team, what they know, or how much time they have.
+
+Checks are how *Vulcan* builds the second quality: Instead of blocking enforcement like audits, it ensures persistent, instrumented visibility that makes the cost of **ignoring quality problems progressively harder to justify**.
+
+The data product doctrine's argument about defensive design is useful here. A defensive system delegates the brunt of mistrust to the foundational platform, so the users operating on top of it can trust what they receive.
+
+Checks are the mechanism at the quality layer. They are the platform's way of watching what data teams would otherwise have to watch manually and surfacing problems without requiring anyone to think to look.
+
+The value compounds as the suite of checks grows: each new check is a commitment encoded in the platform rather than buried in someone's head. When that person leaves, **the commitment stays**. When the team grows, **new members inherit it automatically**. Almost poetic justice.
+
+The non-blocking nature of checks is also a deliberate design choice.
+
+- *Audits* block because certain quality violations are structurally incompatible with downstream trust.
+- *Checks* watch because many quality questions require context, like historical trends, proportional thresholds, and statistical distributions that cannot be reduced to a binary pass or fail at the moment of execution.
+
+A check that flags a 40% drop in row count is not failing a pipeline but surfacing a signal that the pipeline owner is now accountable for investigating. That accountability (the feedback loop between the person who caused the problem and the system that detected it) is exactly what the data product doctrine sticks to when it argues that productisation fixes the cost-value asymmetry in data pipelines.
+
+## Checks on Vulcan
+
+[https://tmdc-io.github.io/vulcan-book/components/checks/checks/](https://tmdc-io.github.io/vulcan-book/components/checks/checks/)

dataproduct_builder_skills-1.2.1/src/dataproduct_builder_skills/data/dpbs-docs/dataos-philosophy/components/index.md

@@ -0,0 +1,8 @@
+# Components
+
+- [Model](model/index.md)
+- [Tests](tests.md)
+- [Audits](audits.md)
+- [Checks](checks.md)
+- [Semantics](semantics/index.md)
+- [Advanced Features](advanced-features/index.md)

dataproduct_builder_skills-1.2.1/src/dataproduct_builder_skills/data/dpbs-docs/dataos-philosophy/components/model/index.md

@@ -0,0 +1,4 @@
+# Model
+
+- [Overview](overview.md)
+- [Types](types/index.md)

dataproduct_builder_skills-1.2.1/src/dataproduct_builder_skills/data/dpbs-docs/dataos-philosophy/components/model/overview.md

@@ -0,0 +1,28 @@
+# Overview
+
+## **Models as Units of Value**
+
+Most data teams think of a model as a transformation: SQL that reads from one place and writes to another. That framing is technically accurate but insufficient.
+
+A transformation is a process, but **a model, in the data product sense, is a unit of value**: a self-contained artifact with a name, an owner, a schema, a schedule, and a defined purpose it was built to serve. It is the difference between code that runs and code that can be depended upon.
+
+When you define a model in *Vulcan,* with its name, grain, tags, terms, column descriptions, and materialization strategy, you are making the model legible to everyone who comes after you: the analyst who builds a dashboard on top of it, the engineer who inherits it, the system that catalogs it, and the consumer who **depends on it without knowing how it was built**.
+
+A transformation that nobody can find, trust, or build on is infrastructure that generates no value. A model that is named, described, owned, and validated is a unit that compounds by getting reused, extended, and composed into higher-order products. That compounding is where the data team's leverage and value attribution come alive.
+
+## **Model as a Contract**
+
+Every model you ship makes an implicit promise to its downstream consumers: this schema will be here, these columns will mean what they say, this data will arrive on schedule. Most data teams have never made that promise explicitly, and so when the schema changes, the grain shifts, or the pipeline breaks, **the consumer finds out the hard way and bears the burden of resolution**.
+
+The model was always a contract, it just wasn't written down. *Vulcan*'s model structure is how you write it down.
+
+- The `MODEL` block is comparable to a contract header.
+- The `name` and `grain` define the identity of what you're delivering.
+- The `kind` and `cron` define the terms of delivery.
+- The `column_descriptions`, `column_tags`, and `column_terms` define what each field means, so consumers don't have to reverse-engineer intent from column names.
+
+The validation layer (tests, assertions, checks) is what backs the contract with enforcement rather than goodwill. A model that passes its assertions is a certified output that a downstream consumer can depend on without inspecting its internals. That is what separates a model from a query, and a data product from a pipeline.
+
+## Model in Vulcan
+
+[https://tmdc-io.github.io/vulcan-book/components/model/overview/](https://tmdc-io.github.io/vulcan-book/components/model/overview/)

dataproduct_builder_skills-1.2.1/src/dataproduct_builder_skills/data/dpbs-docs/dataos-philosophy/components/model/types/index.md

@@ -0,0 +1,10 @@
+# Types
+
+- [Multi-Type Advantage](multi-type-advantage.md)
+
+Model types in Vulcan:
+
+- [SQL](https://tmdc-io.github.io/vulcan-book/components/model/types/sql_models/)
+- [Python](https://tmdc-io.github.io/vulcan-book/components/model/types/python_models/)
+- [External](https://tmdc-io.github.io/vulcan-book/components/model/types/external_models/)
+- [Managed](https://tmdc-io.github.io/vulcan-book/components/model/types/managed_models/)

dataproduct_builder_skills-1.2.1/src/dataproduct_builder_skills/data/dpbs-docs/dataos-philosophy/components/model/types/multi-type-advantage.md

@@ -0,0 +1,20 @@
+# Multi-Type Advantage
+
+## **Why Polyglot Modeling Matters**
+
+The history of data tooling is a history of **forced choices.**
+
+Pick a warehouse, learn its dialect. Pick a transformation framework, write everything in its paradigm. The implicit assumption behind every one of these choices was that a single language, if expressive enough, should be sufficient for everything a data team needs to do.
+
+That assumption has never been true, and the technical debt of teams trying to make it true is visible in every SQL file that has a CASE WHEN stack where a function call should be, and every Python script stitched onto the side of a pipeline because the transformation layer couldn't handle what the business actually needed.
+
+The four model types in *Vulcan* (SQL, Python, External, and Managed) are not four ways to do the same thing. They are four distinct tools for four distinct categories of work, and the **ability to mix them in a single project** is what makes the difference between a stack that fits your problem and one you're constantly working around.
+
+- **SQL** is the right tool for set-based transformations against data that already exists in your warehouse: joins, aggregations, filters, window functions.
+- **Python** is the right tool when the transformation requires logic that SQL handles poorly or not at all: calling an external API, running an ML model, applying complex business rules that would take a hundred lines of SQL to express imperfectly.
+- **External models** are how you bring data that exists outside your warehouse into *Vulcan*'s dependency graph without rewriting the systems that produce it.
+- **Managed models** give you control over data *Vulcan* owns end-to-end, without an external source driving it.
+
+The point of polyglot modeling is not to give you more options, but to **expose the right tool for each unit of work**, so you **stop bending your logic to fit your tooling**.
+
+A data team that can write the model in the language the problem calls for, and have all of it participate in the same validation barrier, the same dependency graph, the same CI/CD pipeline, is a team that spends its time on the problem by saving time that would otherwise be spent on the constraints of the stack.

dataproduct_builder_skills-1.2.1/src/dataproduct_builder_skills/data/dpbs-docs/dataos-philosophy/components/semantics/business-metrics.md

@@ -0,0 +1,30 @@
+# Business Metrics
+
+## Working with Economic Consequences
+
+Every business metric is ultimately a claim about economic reality. When a metric is defined, it is to formalize the numbers that drive decisions: where to invest, what to cut, which customers to prioritise, and whether the business is growing or contracting.
+
+The economic weight of that is considerable. A metric that is wrong, inconsistently defined, or drifting undetected from its intended calculation is a decision-quality problem. And decisions made confidently on wrong numbers have a cost that accumulates (again, mostly undetected for a long time) in budgets allocated to the wrong segments, in growth levers left unpulled, in problems diagnosed too late because the signal was corrupted at source.
+
+This is the key argument for defining metrics in the semantic layer rather than embedding them in dashboards, reports, or ad hoc SQL.
+
+- When a metric exists in a dashboard, its **economic consequences are local:** confined to whoever reads that dashboard.
+- When it exists in the semantic layer, backed by validated models and owned by a named team, its economic consequences are explicit, and its definition is a shared organisational commitment.
+
+Changing the definition of "active customer" is no longer a one-line SQL edit that shifts a chart undetected; it is a versioned change to a contract that downstream consumers depend on, communicated and reviewed before it ships.
+
+The `meta` fields (business owner, calculation method, benchmark) are the answer to the question every decision-maker should be asking before they act on a number: who is accountable for this, how was it derived, and what should I expect it to be?
+
+### Enabling AI with Measurable Economic Returns
+
+Today's semantic layer powers human consumers: analysts querying dashboards, product managers reviewing KPIs, and finance teams pulling reports. Tomorrow's semantic layer must also be legible to AI agents that need to act on data, not just observe it.
+
+An agent reasoning over whether to approve a discount, trigger a reorder, or escalate an anomaly needs to know not just the current value of a metric but what it means, who owns it, how it was calculated, and whether it is trustworthy.
+
+The `meta` fields on a business metric are the beginning of the context layer that separates metrics an agent can act on from metrics it can only read.
+
+Every business metric defined in *Vulcan* today is a node in the context architecture of tomorrow: a unit of meaning that the organisation can build on, reason from, and eventually activate across every system that needs to understand what the business knows about itself.
+
+## Business Metrics on *Vulcan*
+
+[https://tmdc-io.github.io/vulcan-book/components/semantics/business_metrics/](https://tmdc-io.github.io/vulcan-book/components/semantics/business_metrics/)

dataproduct_builder_skills-1.2.1/src/dataproduct_builder_skills/data/dpbs-docs/dataos-philosophy/components/semantics/index.md

@@ -0,0 +1,5 @@
+# Semantics
+
+- [Overview](overview.md)
+- [Semantic Models](semantic-models.md)
+- [Business Metrics](business-metrics.md)

dataproduct_builder_skills-1.2.1/src/dataproduct_builder_skills/data/dpbs-docs/dataos-philosophy/components/semantics/overview.md

@@ -0,0 +1,24 @@
+# Overview
+
+## **Semantic Layer as Business Interface**
+
+The semantic layer emerged because the alternatives failed in sequence.
+
+- Data cubes pre-calculated everything and left no room for novel queries: every new business question required a new engineering cycle.
+- BI tools freed analysts from those boundaries but trapped semantics inside vendor ecosystems: a metric defined in Tableau was inaccessible to a product team building a custom application, and the same revenue definition had to be rebuilt from scratch across every tool that needed it.
+
+The fundamental problem was about where semantics was hosted and whether it could be reused.
+
+A semantic layer solves this by **decoupling context from the systems that consume it**. You define meaning once, at a layer between the physical data and every surface that needs to understand it (dashboards, APIs, applications, AI models), and every consumer draws from the same source.
+
+But a semantic layer placed directly over unvalidated physical data replicates its problems at a higher altitude. [Semantic mistrust](https://moderndata101.substack.com/i/144240703/be-wary-of-semantic-mistrust), the condition where metrics look credible but are built on unstable foundations, is **the semantic layer's most dangerous risk**.
+
+A sales analyst who finds the right measure and gets a wrong answer has a problem that is harder to diagnose than one who cannot find the measure at all.
+
+This is why *Vulcan*'s semantic layer sits downstream of the validation barrier, not parallel to it. The models feeding the semantic layer have been tested, audited, and certified. The dimensions, measures, and metrics built on top of them inherit that certification.
+
+The semantic layer is not just a business interface, but the point where validated data products become accessible to every consumer, in every tool, through every API, without re-deriving from scratch.
+
+## Semantic Layer Overview
+
+[https://tmdc-io.github.io/vulcan-book/components/semantics/overview/](https://tmdc-io.github.io/vulcan-book/components/semantics/overview/)

dataproduct_builder_skills-1.2.1/src/dataproduct_builder_skills/data/dpbs-docs/dataos-philosophy/components/semantics/semantic-models.md

@@ -0,0 +1,15 @@
+# Semantic Models
+
+Physical models are organised around engineering constraints (normalization, incremental processing, storage efficiency) while business understanding is organised around entities, relationships, and concepts that span multiple tables and exist in a context that no single schema can express.
+
+A semantic model is the artifact that reconstructs that context. It maps what engineers optimised for computation onto what analysts need for comprehension.
+
+The absence of context boundaries means every table, metric, and dimension floats in isolation, becoming fragments of truth without a shared narrative. Cataloguing these fragments tells you what exists, but not why it matters, how it relates, or what it means in a business domain.
+
+Semantic models address this at the layer where it can actually be resolved: at the point of definition, **before consumption begins**. When you define an `active_customers` segment once in a semantic model, you are not just saving analysts from writing the same WHERE clause, but you are making a deduction: this is what "active" means in this domain, and **this definition is authoritative.**
+
+Every consumer who queries against that segment receives the same answer, derived from the same logic, without negotiating what "active" means with a colleague first. The semantic model is where institutional knowledge about data, the kind that currently lives in Slack threads and tribal memory, gets formalised into something the system can enforce and every consumer can trust.
+
+## Semantic Models in *Vulcan*
+
+[https://tmdc-io.github.io/vulcan-book/components/semantics/models/](https://tmdc-io.github.io/vulcan-book/components/semantics/models/)

dataproduct_builder_skills-1.2.1/src/dataproduct_builder_skills/data/dpbs-docs/dataos-philosophy/components/tests.md

@@ -0,0 +1,33 @@
+# Tests
+
+## What Tests Actually Guarantee
+
+Data that cannot be trusted does not get used, or worse, it gets used without anyone knowing it shouldn't be. The cost of the second scenario is a business decision made confidently on numbers that were wrong.
+
+Tests exist to make that scenario structurally unlikely by making correctness a **property that the system enforces rather than assumes**. Not by making engineers more careful.
+
+Tests can be described as a safety net, which is accurate but undersells the mechanism. A test is a formal statement about what a model promises to produce. When you write input fixtures and expected outputs, you are not just checking whether the SQL runs; you are ensuring the contract the model holds with its consumers.
+
+> Every test that passes is **evidence** that the contract is intact.
+
+Every test that fails is the system catching a broken promise before a consumer does. That distinction (catching it before the consumer, not after) is what converts testing from a development practice into a trust infrastructure.
+
+It is surprising how the simplest quality assurance factors go missing when it comes to data handling. And it is **often because of the nature of data,** which can be independently reproduced, viewed, duplicated, sliced and diced, or downloaded in disconnected units.
+
+But data products change that. Trust, for a data product, is the feature that **determines whether business users build on your data or route around it.**
+
+## Who Owns the Test Owns the Contract
+
+**Quality is Product Responsibility**. But in a pipeline-first world, quality is someone else's problem. The pipeline runs, the data lands, and if something is wrong, the analyst who finds it three days later raises a ticket. A model without tests is a model that asks its consumers to absorb the risk of its author's assumptions.
+
+The feedback loop between the person who caused the problem and the person who experienced it is long, indirect, and **structurally designed to decouple accountability from consequence**.
+
+Tests close that loop by making the model author responsible for **correctness at the point of authorship** instead of at the point of discovery.
+
+This is what the data product doctrine means when it says ownership extends beyond code. A data product owner who does not test their models has made an implicit choice: they have decided that the cost of writing tests is higher than the cost of their consumers encountering bad data.
+
+That calculation is almost always wrong, and it becomes more wrong as the number of consumers grows. Tests are not overhead on top of the work, but are the work and the part that converts a transformation into a deliverable someone can depend on.
+
+## Tests in *Vulcan*
+
+[https://tmdc-io.github.io/vulcan-book/components/tests/tests/](https://tmdc-io.github.io/vulcan-book/components/tests/tests/)