model2data 1.13.0__tar.gz → 1.16.0__tar.gz

{model2data-1.13.0/model2data.egg-info → model2data-1.16.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: model2data
-Version: 1.13.0
+Version: 1.16.0
 Summary: Turn a data model into realistic synthetic data, a runnable dbt project and metrics with known values
 Author: JB Analytica
 License-Expression: MIT
@@ -89,9 +89,12 @@ emails, not `Lorem ipsum`. Type a column with any Faker provider (`billing_count
 come from one country. → [The model file](https://github.com/JB-Analytica/model2data/blob/main/docs/model-file.md)
 
 **Foreign keys that resolve.** Tables are generated in dependency order and every foreign key
-points at a parent row that exists — onto a primary key or a unique one. Composite primary and
-unique keys hold, and a column that repeats a parent's value through a foreign key
-(`orders.customer_email`) holds the email of the order's own customer.
+points at a parent row that exists — onto a primary key or a unique one — and
+`after: customers.created_at` keeps an order on or after the sign-up of the customer it points
+at (`validate` suggests it where a date could fall before its parent's). Integer primary keys
+number the rows 1..N, in order. Composite primary and unique keys hold, and a column that repeats a
+parent's value through a foreign key (`orders.customer_email`) holds the email of the order's
+own customer.
 → [The model file](https://github.com/JB-Analytica/model2data/blob/main/docs/model-file.md)
 
 **A dbt project that runs.** Seeds, staging models that `ref()` them, `not_null`, `unique`,
@@ -115,8 +118,7 @@ byte-identical. → [The determinism promise](https://github.com/JB-Analytica/mo
 
 **When things happen.** `--business-hours`, `--growth` and `--seasonality` shape every timestamp
 (weekdays and working hours, a trend, a Q4 peak), or one column at a time. `updated_at` never
-lands before `created_at`, `after` orders any two columns, and `when` gives `shipped_at` a value
-on shipped and delivered orders only. → [When things happen](https://github.com/JB-Analytica/model2data/blob/main/docs/time-shapes.md)
+lands before `created_at`, `after` orders any two columns, or a column after a parent's, and `when` gives `shipped_at` a value on shipped and delivered orders only. → [When things happen](https://github.com/JB-Analytica/model2data/blob/main/docs/time-shapes.md)
 
 **How the data is spread.** `--skew` lets a few customers place most of the orders; `weights`,
 `true_rate`, `null_rate` and `distinct` shape a column; `distribution` draws a number from a
@@ -131,6 +133,9 @@ orphans, nulls, invalid values, messy text, late-arriving rows. Every run with d
 orders, average order value. The run writes each metric's value over the generated data to
 `metric_values.json`, overall and per month, a dbt test per metric that must reproduce it on the
 warehouse, and an [Apache Ossie](https://github.com/apache/ossie) semantic model under `osi/`.
+New in 1.14.0: `--lightdash` writes the same metrics into the staging models' YAML as
+[Lightdash](https://www.lightdash.com/) reads them, filters, joins and ratios included, so the
+dashboard's revenue can be checked against the number the run already knows.
 → [Metrics with known values](https://github.com/JB-Analytica/model2data/blob/main/docs/metrics.md)
 
 **Validate in CI.** `model2data validate` checks models and metrics files and prints GitHub
@@ -272,7 +277,7 @@ definition goes in; nothing but synthetic data comes out.
   `accepted_values` fail for the right reason.
 - **A team defining metrics once.** Write revenue in one metrics file, check every number the
   semantic layer or dashboard shows against `metric_values.json`, let `dbt build` prove the SQL,
-  and hand the definitions on as Apache Ossie.
+  and hand the definitions on as Apache Ossie or straight to Lightdash.
 
 ## How it works
 
@@ -298,7 +303,8 @@ model (.model2data.yml or DBML) + metrics file (optional)
    model's hints, table and column `description:` fields from the model, and a profile for
    DuckDB (zero-config, file-based) or Postgres.
 4. **Compute metrics.** With `--metrics`, computes each metric over the generated data and writes
-   its known value, a dbt test that checks it, and the Apache Ossie file.
+   its known value, a dbt test that checks it, and the Apache Ossie file; with `--lightdash`,
+   Lightdash metrics and dimensions in the staging models' YAML.
 
 ---
 

{model2data-1.13.0 → model2data-1.16.0}/README.md

@@ -42,9 +42,12 @@ emails, not `Lorem ipsum`. Type a column with any Faker provider (`billing_count
 come from one country. → [The model file](docs/model-file.md)
 
 **Foreign keys that resolve.** Tables are generated in dependency order and every foreign key
-points at a parent row that exists — onto a primary key or a unique one. Composite primary and
-unique keys hold, and a column that repeats a parent's value through a foreign key
-(`orders.customer_email`) holds the email of the order's own customer.
+points at a parent row that exists — onto a primary key or a unique one — and
+`after: customers.created_at` keeps an order on or after the sign-up of the customer it points
+at (`validate` suggests it where a date could fall before its parent's). Integer primary keys
+number the rows 1..N, in order. Composite primary and unique keys hold, and a column that repeats a
+parent's value through a foreign key (`orders.customer_email`) holds the email of the order's
+own customer.
 → [The model file](docs/model-file.md)
 
 **A dbt project that runs.** Seeds, staging models that `ref()` them, `not_null`, `unique`,
@@ -68,8 +71,7 @@ byte-identical. → [The determinism promise](docs/generating.md#the-determinism
 
 **When things happen.** `--business-hours`, `--growth` and `--seasonality` shape every timestamp
 (weekdays and working hours, a trend, a Q4 peak), or one column at a time. `updated_at` never
-lands before `created_at`, `after` orders any two columns, and `when` gives `shipped_at` a value
-on shipped and delivered orders only. → [When things happen](docs/time-shapes.md)
+lands before `created_at`, `after` orders any two columns, or a column after a parent's, and `when` gives `shipped_at` a value on shipped and delivered orders only. → [When things happen](docs/time-shapes.md)
 
 **How the data is spread.** `--skew` lets a few customers place most of the orders; `weights`,
 `true_rate`, `null_rate` and `distinct` shape a column; `distribution` draws a number from a
@@ -84,6 +86,9 @@ orphans, nulls, invalid values, messy text, late-arriving rows. Every run with d
 orders, average order value. The run writes each metric's value over the generated data to
 `metric_values.json`, overall and per month, a dbt test per metric that must reproduce it on the
 warehouse, and an [Apache Ossie](https://github.com/apache/ossie) semantic model under `osi/`.
+New in 1.14.0: `--lightdash` writes the same metrics into the staging models' YAML as
+[Lightdash](https://www.lightdash.com/) reads them, filters, joins and ratios included, so the
+dashboard's revenue can be checked against the number the run already knows.
 → [Metrics with known values](docs/metrics.md)
 
 **Validate in CI.** `model2data validate` checks models and metrics files and prints GitHub
@@ -225,7 +230,7 @@ definition goes in; nothing but synthetic data comes out.
   `accepted_values` fail for the right reason.
 - **A team defining metrics once.** Write revenue in one metrics file, check every number the
   semantic layer or dashboard shows against `metric_values.json`, let `dbt build` prove the SQL,
-  and hand the definitions on as Apache Ossie.
+  and hand the definitions on as Apache Ossie or straight to Lightdash.
 
 ## How it works
 
@@ -281,7 +286,8 @@ flowchart LR
    model's hints, table and column `description:` fields from the model, and a profile for
    DuckDB (zero-config, file-based) or Postgres.
 4. **Compute metrics.** With `--metrics`, computes each metric over the generated data and writes
-   its known value, a dbt test that checks it, and the Apache Ossie file.
+   its known value, a dbt test that checks it, and the Apache Ossie file; with `--lightdash`,
+   Lightdash metrics and dimensions in the staging models' YAML.
 
 ---
 

{model2data-1.13.0 → model2data-1.16.0}/README_PYPI.md

@@ -42,9 +42,12 @@ emails, not `Lorem ipsum`. Type a column with any Faker provider (`billing_count
 come from one country. → [The model file](https://github.com/JB-Analytica/model2data/blob/main/docs/model-file.md)
 
 **Foreign keys that resolve.** Tables are generated in dependency order and every foreign key
-points at a parent row that exists — onto a primary key or a unique one. Composite primary and
-unique keys hold, and a column that repeats a parent's value through a foreign key
-(`orders.customer_email`) holds the email of the order's own customer.
+points at a parent row that exists — onto a primary key or a unique one — and
+`after: customers.created_at` keeps an order on or after the sign-up of the customer it points
+at (`validate` suggests it where a date could fall before its parent's). Integer primary keys
+number the rows 1..N, in order. Composite primary and unique keys hold, and a column that repeats a
+parent's value through a foreign key (`orders.customer_email`) holds the email of the order's
+own customer.
 → [The model file](https://github.com/JB-Analytica/model2data/blob/main/docs/model-file.md)
 
 **A dbt project that runs.** Seeds, staging models that `ref()` them, `not_null`, `unique`,
@@ -68,8 +71,7 @@ byte-identical. → [The determinism promise](https://github.com/JB-Analytica/mo
 
 **When things happen.** `--business-hours`, `--growth` and `--seasonality` shape every timestamp
 (weekdays and working hours, a trend, a Q4 peak), or one column at a time. `updated_at` never
-lands before `created_at`, `after` orders any two columns, and `when` gives `shipped_at` a value
-on shipped and delivered orders only. → [When things happen](https://github.com/JB-Analytica/model2data/blob/main/docs/time-shapes.md)
+lands before `created_at`, `after` orders any two columns, or a column after a parent's, and `when` gives `shipped_at` a value on shipped and delivered orders only. → [When things happen](https://github.com/JB-Analytica/model2data/blob/main/docs/time-shapes.md)
 
 **How the data is spread.** `--skew` lets a few customers place most of the orders; `weights`,
 `true_rate`, `null_rate` and `distinct` shape a column; `distribution` draws a number from a
@@ -84,6 +86,9 @@ orphans, nulls, invalid values, messy text, late-arriving rows. Every run with d
 orders, average order value. The run writes each metric's value over the generated data to
 `metric_values.json`, overall and per month, a dbt test per metric that must reproduce it on the
 warehouse, and an [Apache Ossie](https://github.com/apache/ossie) semantic model under `osi/`.
+New in 1.14.0: `--lightdash` writes the same metrics into the staging models' YAML as
+[Lightdash](https://www.lightdash.com/) reads them, filters, joins and ratios included, so the
+dashboard's revenue can be checked against the number the run already knows.
 → [Metrics with known values](https://github.com/JB-Analytica/model2data/blob/main/docs/metrics.md)
 
 **Validate in CI.** `model2data validate` checks models and metrics files and prints GitHub
@@ -225,7 +230,7 @@ definition goes in; nothing but synthetic data comes out.
   `accepted_values` fail for the right reason.
 - **A team defining metrics once.** Write revenue in one metrics file, check every number the
   semantic layer or dashboard shows against `metric_values.json`, let `dbt build` prove the SQL,
-  and hand the definitions on as Apache Ossie.
+  and hand the definitions on as Apache Ossie or straight to Lightdash.
 
 ## How it works
 
@@ -251,7 +256,8 @@ model (.model2data.yml or DBML) + metrics file (optional)
    model's hints, table and column `description:` fields from the model, and a profile for
    DuckDB (zero-config, file-based) or Postgres.
 4. **Compute metrics.** With `--metrics`, computes each metric over the generated data and writes
-   its known value, a dbt test that checks it, and the Apache Ossie file.
+   its known value, a dbt test that checks it, and the Apache Ossie file; with `--lightdash`,
+   Lightdash metrics and dimensions in the staging models' YAML.
 
 ---
 

{model2data-1.13.0 → model2data-1.16.0}/model2data/cli.py

@@ -29,6 +29,7 @@ from model2data.defects import (
 )
 from model2data.generate.core import (
     generate_data_from_dbml,
+    get_capped_tables,
     get_cyclic_tables,
     get_unresolved_composite_keys,
 )
@@ -50,13 +51,16 @@ from model2data.metrics import (
     SPEC_VERSION as METRICS_SPEC_VERSION,
 )
 from model2data.metrics import (
+    LightdashExport,
     Metrics,
     MetricsError,
     is_metrics_file,
     known_values,
     metrics_stem,
     sibling_model,
+    to_lightdash,
     to_ossie,
+    write_lightdash,
     write_metric_tests,
 )
 from model2data.metrics import load as load_metrics
@@ -181,7 +185,7 @@ app = typer.Typer(
     cls=_GenerateByDefault,
     help=(
         "model2data: Generate analytics-ready datasets from a data model.\n\n"
-        "Given a model -- a .model2data.yml document (spec 0.4.0), the same as JSON,\n"
+        "Given a model -- a .model2data.yml document (spec 0.5.0), the same as JSON,\n"
         "or a DBML file -- this tool produces:\n"
         "• Synthetic but realistic data\n"
         "• A runnable dbt project scaffold\n"
@@ -398,8 +402,9 @@ def main(
         "warn",
         "--hint-tests",
         help=(
-            "Write the model's generation hints as dbt tests (min/max range, after, "
-            "null_rate, distinct, when, grain) at this severity: error, warn or off. They "
+            "Write the model's generation hints as dbt tests (min/max range, after "
+            "(also a parent's column), null_rate, distinct, when, grain) at this severity: "
+            "error, warn or off. They "
             "describe intent, so they warn by default rather than break a first dbt "
             "build on real data."
         ),
@@ -471,6 +476,16 @@ def main(
             "without it. Not picked up from beside the model: pass it."
         ),
     ),
+    lightdash: bool = typer.Option(
+        False,
+        "--lightdash",
+        help=(
+            "Add Lightdash metrics and dimensions to the staging models' YAML\n"
+            "(models/staging/stg_*.yml, under config.meta): the --metrics file's metrics,\n"
+            "or without one the metrics the model implies. Without it those files are\n"
+            "unchanged."
+        ),
+    ),
 ):
     """
     Generate synthetic data and a dbt project from a model.
@@ -726,6 +741,13 @@ def main(
     # Metrics: only added files, computed from what the seeds hold
     # -------------------------
     metrics_count: Optional[int] = None
+    lightdash_export = None
+    if _given(lightdash):
+        typer.echo("🔦 Adding Lightdash metrics and dimensions to the staging models...")
+        lightdash_export = to_lightdash(
+            resolve_metrics(model, metrics_doc, model_name=_model_stem(file))
+        )
+        write_lightdash(dest, lightdash_export)
     if metrics_doc is not None and metrics_file is not None:
         typer.echo("📏 Computing each metric's known value and its dbt test...")
         semantic = resolve_metrics(model, metrics_doc, model_name=_model_stem(file))
@@ -752,6 +774,7 @@ def main(
     cyclic_tables = get_cyclic_tables()
     unresolved_keys = get_unresolved_composite_keys()
     duplicate_unique = get_duplicate_unique_columns()
+    capped_tables = get_capped_tables()
 
     typer.echo("\n📊 Summary")
     typer.echo(f"  Tables generated:        {len(generated_tables)}")
@@ -790,6 +813,13 @@ def main(
         )
         for label in duplicate_unique:
             typer.echo(f"    - {label}")
+    if capped_tables:
+        typer.echo(
+            "  ⚠️  Tables generated with fewer rows than asked, so a unique reference "
+            "keeps one row per parent value:"
+        )
+        for label in capped_tables:
+            typer.echo(f"    - {label}")
 
     if metrics_count is not None:
         typer.echo(
@@ -797,6 +827,13 @@ def main(
             f"osi/{project_name}.yml, data-tests/metrics/)"
         )
 
+    if lightdash_export is not None:
+        typer.echo(
+            f"  Lightdash:               {len(lightdash_export.metrics)} metrics "
+            "(models/staging/stg_*.yml, config.meta)"
+        )
+        _print_losses("Not expressible in Lightdash, left out", lightdash_export.lossiness)
+
     if report is not None:
         _print_defects(report)
 
@@ -810,12 +847,13 @@ def main(
 
 
 def _spec_of(file: Path) -> str:
-    """The spec a conforming model is written against: 0.4.0, 0.3.0, or 0.2.0 (DBML without
-    `when` included)."""
+    """The spec a conforming model is written against: 0.5.0, 0.4.0, 0.3.0, or 0.2.0 (DBML
+    without `when` or a 0.5.0 `after` included)."""
     version = str(load(file).version)
-    return (
-        "0.4.0" if version.startswith("0.4") else "0.3.0" if version.startswith("0.3") else "0.2.0"
-    )
+    for minor in ("0.5", "0.4", "0.3"):
+        if version.startswith(minor):
+            return f"{minor}.0"
+    return "0.2.0"
 
 
 def _print_defects(report: DefectsReport) -> None:
@@ -907,11 +945,12 @@ def validate_command(
         ),
     ),
 ):
-    """Check that models conform to spec 0.4.0 (or 0.2.x, 0.3.x), printing every issue with its path.
+    """Check that models conform to spec 0.5.0 (or 0.2.x-0.4.x), printing every issue with its path.
 
     Takes one or more files and/or --glob patterns. A metrics file (*.metrics.yml) is checked
     against metrics spec 0.1.0 and against its model. Exits 1 when any file has an error;
-    warnings are printed, and the file conforms.
+    warnings are printed, and the file conforms. A warning that a child's date `can fall
+    before` its parent's names the `after: <table>.<column>` to add.
     """
     if output_format not in ("text", "github"):
         raise typer.BadParameter("must be 'text' or 'github'", param_hint="--format")
@@ -1002,7 +1041,8 @@ def convert_command(
     force: bool = typer.Option(False, "--force", help="Overwrite the output file if it exists."),
 ):
     """Convert a model -- DBML, typically -- to a .model2data.yml document (spec 0.2.0, or
-    0.3.0 when it has defects, 0.4.0 when it has `when`)."""
+    0.3.0 when it has defects, 0.4.0 when it has `when`, 0.5.0 when an `after` is a list or
+    names a parent's column)."""
     try:
         model = load(file)
     except ModelError as error:
@@ -1074,7 +1114,14 @@ metrics_app = typer.Typer(
 )
 app.add_typer(metrics_app, name="metrics")
 
-_METRICS_EXPORTS = ("ossie",)
+_METRICS_EXPORTS = ("ossie", "lightdash")
+
+
+def _print_losses(heading: str, losses: list) -> None:
+    if losses:
+        typer.echo(f"ℹ️  {heading}:")
+        for loss in losses:
+            typer.echo(f"  - {loss}")
 
 
 def _metrics_inputs(file: Path, metrics_file: Optional[Path]) -> Any:
@@ -1138,7 +1185,14 @@ def metrics_list_command(
 def metrics_export_command(
     file: Path = _MODEL_OPTION,
     metrics_file: Optional[Path] = _METRICS_OPTION,
-    to: str = typer.Option("ossie", "--to", help="The format: 'ossie' (Apache Ossie 0.1.1, YAML)."),
+    to: str = typer.Option(
+        "ossie",
+        "--to",
+        help=(
+            "The format: 'ossie' (Apache Ossie 0.1.1, YAML) or 'lightdash' (dbt model "
+            "properties with Lightdash's config.meta, YAML)."
+        ),
+    ),
     output: Optional[Path] = typer.Option(  # noqa: B008
         None,
         "--output",
@@ -1153,6 +1207,11 @@ def metrics_export_command(
     Ossie datasets read the staging models of the dbt project `generate` writes
     (staging.stg_<table>). What Ossie 0.1.1 cannot say is listed at the top of the file,
     and carried in custom_extensions.
+
+    Lightdash's file describes those staging models (stg_<table>) as a dbt properties
+    file. dbt reads a model's properties from one file only, so in a project `generate`
+    wrote, use `generate --lightdash` instead: it merges the same meta into the staging
+    models' own YAML. What Lightdash cannot say is listed at the top of the file.
     """
     to = (_given(to) or "ossie").lower()
     if to not in _METRICS_EXPORTS:
@@ -1160,7 +1219,7 @@ def metrics_export_command(
             f"Choose one of: {', '.join(_METRICS_EXPORTS)}.", param_hint="--to"
         )
     semantic = _metrics_inputs(file, _given(metrics_file))
-    export = to_ossie(semantic)
+    export = to_ossie(semantic) if to == "ossie" else to_lightdash(semantic)
     text = export.to_yaml()
     output = _given(output)
     if output is None:
@@ -1170,8 +1229,11 @@ def metrics_export_command(
         typer.echo(f"❌ {output} already exists. Use --force to overwrite it.")
         raise typer.Exit(1)
     output.write_text(text, encoding="utf-8")
+    if isinstance(export, LightdashExport):
+        typer.echo(f"✅ Wrote {output} (Lightdash, dbt YAML, {len(export.metrics)} metrics)")
+        _print_losses("Not expressible in Lightdash, left out", export.lossiness)
+        return
     typer.echo(f"✅ Wrote {output} (Apache Ossie 0.1.1, {len(semantic.metrics)} metrics)")
-    if export.lossiness:
-        typer.echo("ℹ️  Not expressible in Ossie 0.1.1 itself, carried in custom_extensions:")
-        for loss in export.lossiness:
-            typer.echo(f"  - {loss}")
+    _print_losses(
+        "Not expressible in Ossie 0.1.1 itself, carried in custom_extensions", export.lossiness
+    )

{model2data-1.13.0 → model2data-1.16.0}/model2data/dbt/hint_tests.py

@@ -12,11 +12,17 @@ What becomes a test, and what does not:
 |---------------------------|----------------------------------------|-----------|
 | `generate.min` / `max`    | `model2data_between`                   | none      |
 | `generate.after: other`   | `model2data_not_before`                | none      |
+| `generate.after: table.column` | `model2data_not_before_parent`    | none      |
 | `generate.null_rate`      | `model2data_max_null_share`            | yes       |
 | `generate.distinct: n`    | `model2data_max_distinct`              | none      |
 | `generate.when`           | `model2data_when`                      | none      |
 | table `grain`             | `model2data_unique_combination`        | none      |
 
+An `after` listing several columns writes one test for each. A parent's column
+(`after: customers.created_at`, see `model2data.generate.parents`) is tested
+through the foreign key that reaches it, so that test needs the model's
+references: a caller passing bare tables passes `refs` too.
+
 A column with `when` and `null_rate` has its null share tested among the rows
 `when` matches (`model2data_when_max_null_share`), since only those rows can
 hold a value, and its `model2data_when` test lets those rows be null.
@@ -28,8 +34,9 @@ constraint a row can break, so they produce no test.
 
 The generic tests live in `macros/model2data_hint_tests.sql` of the generated
 project (see `write_hint_macros`), and the `when` ones in
-`macros/model2data_when_tests.sql`, written only for a model that has one, so
-a project without `when` is what it was: plain SQL, no dbt package.
+`macros/model2data_when_tests.sql` and the parent one in
+`macros/model2data_parent_tests.sql`, each written only for a model that has
+one, so a project without either is what it was: plain SQL, no dbt package.
 """
 
 from __future__ import annotations
@@ -41,6 +48,9 @@ from pathlib import Path
 from typing import Any, Literal, Optional, Union
 
 from model2data.generate import kinds
+from model2data.generate.core import parent_rules_for
+from model2data.generate.parents import ParentRule
+from model2data.generate.timeline import same_row_after
 from model2data.model.engine import EngineInputs, to_engine
 from model2data.model.types import Model
 from model2data.parse.dbml import ColumnDef, TableDef
@@ -53,6 +63,8 @@ MACROS_FILE = "model2data_hint_tests.sql"
 _MACROS_SOURCE = Path(__file__).parent / "templates" / "hint_macros" / MACROS_FILE
 WHEN_MACROS_FILE = "model2data_when_tests.sql"
 _WHEN_MACROS_SOURCE = Path(__file__).parent / "templates" / "hint_macros" / WHEN_MACROS_FILE
+PARENT_MACROS_FILE = "model2data_parent_tests.sql"
+_PARENT_MACROS_SOURCE = Path(__file__).parent / "templates" / "hint_macros" / PARENT_MACROS_FILE
 
 
 @dataclass(frozen=True)
@@ -64,6 +76,8 @@ class HintTest:
     test's name, `arguments` its parameters as dbt's `arguments:` block, and
     `hint` the hint it comes from (`min`, `max`, `after`, `null_rate`,
     `distinct`, `when`, `grain`). `min` and `max` of one column share one test.
+    `parent` is the `(table, column)` a test looks rows up in: the parent's key
+    for `model2data_not_before_parent`, None for every other test.
     """
 
     table: str
@@ -71,6 +85,7 @@ class HintTest:
     test: str
     arguments: dict[str, Any] = field(default_factory=dict)
     hint: str = ""
+    parent: Optional[tuple[str, str]] = None
 
     def to_dbt(self, severity: str = "warn") -> dict[str, dict[str, Any]]:
         """The entry for a `tests:` list: `{name: {arguments: ..., config: {severity: ...}}}`."""
@@ -83,12 +98,16 @@ def hint_tests_for(
     model_or_tables: Union[Model, EngineInputs, Mapping[str, TableDef]],
     *,
     tolerance: float = DEFAULT_TOLERANCE,
+    refs: Optional[list[dict]] = None,
 ) -> list[HintTest]:
     """The tests implied by the hints of a model, in table then column order.
 
     Takes a `Model`, the `EngineInputs` of one, or the `tables` dict (as
     `to_engine(model).tables`, or the renamed tables the CLI hands
     `generate_dbt_yml`). Table-level tests follow the table's column tests.
+    With `tables`, `refs` are the references as `to_engine` gives them; without
+    them no test of a parent's column is written (a `Model` or `EngineInputs`
+    brings its own).
 
     `tolerance` is the absolute slack of the statistical test (the null share
     may be `null_rate + tolerance`); hard constraints take none. Severity is
@@ -97,18 +116,25 @@ def hint_tests_for(
     if tolerance < 0:
         raise ValueError("tolerance must not be negative")
     if isinstance(model_or_tables, Model):
-        tables: Mapping[str, TableDef] = to_engine(model_or_tables).tables
+        inputs = to_engine(model_or_tables)
+        tables: Mapping[str, TableDef] = inputs.tables
+        refs = inputs.refs
     elif isinstance(model_or_tables, EngineInputs):
         tables = model_or_tables.tables
+        refs = model_or_tables.refs
     else:
         tables = model_or_tables
+    rules = parent_rules_for(dict(tables), refs) if refs else {}
 
     found: list[HintTest] = []
     for key, table in tables.items():
         by_name = {column.name: column for column in table.columns}
         names = set(by_name)
+        parents = {rule.column: rule for rule in rules.get(key, [])}
         for column in table.columns:
             found.extend(_column_tests(key, column, by_name, tolerance))
+            if column.name in parents:
+                found.extend(_parent_tests(key, parents[column.name]))
         grain = (table.note or {}).get("grain")
         if grain and all(name in names for name in grain):
             found.append(
@@ -146,8 +172,7 @@ def _column_tests(
                 )
             )
 
-    after = note.get("after")
-    if isinstance(after, str) and after in kinds_of and after != column.name:
+    for after in [a for a in same_row_after(note) if a in kinds_of and a != column.name]:
         arguments: dict[str, Any] = {"other": after}
         # The generator compares a date with a timestamp at day granularity.
         if (
@@ -202,6 +227,31 @@ def _column_tests(
     return tests
 
 
+def _parent_tests(table: str, rule: ParentRule) -> list[HintTest]:
+    """One `model2data_not_before_parent` test per parent column the `after` names."""
+    tests = []
+    for link in rule.links:
+        arguments: dict[str, Any] = {
+            "foreign_key": link.fk,
+            "to": f"ref('stg_{link.parent_table}')",
+            "field": link.parent_key,
+            "parent_column": link.parent_column,
+        }
+        if link.by_day:
+            arguments["granularity"] = "day"
+        tests.append(
+            HintTest(
+                table,
+                rule.column,
+                "model2data_not_before_parent",
+                arguments,
+                "after",
+                parent=(link.parent_table, link.parent_key),
+            )
+        )
+    return tests
+
+
 def _when_conditions(when: Any, kinds_of: dict[str, ColumnDef]) -> Optional[dict[str, list]]:
     """A `when` hint as the test's `conditions`: an enum's values as the member text it holds."""
     if not isinstance(when, dict) or not when or not all(name in kinds_of for name in when):
@@ -212,13 +262,14 @@ def _when_conditions(when: Any, kinds_of: dict[str, ColumnDef]) -> Optional[dict
     }
 
 
-def write_hint_macros(dest: Path, *, when: bool = False) -> Path:
+def write_hint_macros(dest: Path, *, when: bool = False, parents: bool = False) -> Path:
     """Write the generic tests into `dest/macros/`, and return the file's path.
 
     A project whose YAML holds `hint_tests_for` tests needs this file next to
     it; `generate_dbt_yml` writes it itself when it emits any. With `when`, the
     file of the `when` tests is written beside it (`WHEN_MACROS_FILE`), which a
-    project holding a `model2data_when` test needs too.
+    project holding a `model2data_when` test needs too; with `parents`, the
+    file of `model2data_not_before_parent` (`PARENT_MACROS_FILE`).
     """
     macros = dest / "macros"
     macros.mkdir(parents=True, exist_ok=True)
@@ -226,4 +277,6 @@ def write_hint_macros(dest: Path, *, when: bool = False) -> Path:
     target.write_text(_MACROS_SOURCE.read_text())
     if when:
         (macros / WHEN_MACROS_FILE).write_text(_WHEN_MACROS_SOURCE.read_text())
+    if parents:
+        (macros / PARENT_MACROS_FILE).write_text(_PARENT_MACROS_SOURCE.read_text())
     return target

model2data-1.16.0/model2data/dbt/templates/hint_macros/model2data_parent_tests.sql

@@ -0,0 +1,37 @@
+{#-
+  The generic test model2data writes for a cross-table `after` (`after:
+  customers.created_at` on orders.order_date): a row is not dated before the
+  parent row its foreign key points at. Written beside
+  model2data_hint_tests.sql, and only by a model that has such a hint. Plain
+  ANSI SQL, runs on DuckDB and Postgres. Returns the rows that break it.
+  `column_name` arrives already quoted by the schema YAML; `foreign_key`,
+  `field` (the parent's key) and `parent_column` (the column `after` names)
+  are raw names and are quoted here. A row whose foreign key is null, or points
+  at no parent row, is left to the not_null and relationships tests, and a key
+  several parent rows share (left to the unique test) counts its earliest one.
+  `granularity='day'` compares a date with a timestamp by day.
+-#}
+
+{% test model2data_not_before_parent(model, column_name, foreign_key, to, field, parent_column, granularity=none) %}
+{%- set mine = 'child.' ~ column_name -%}
+{%- set theirs = 'parent.parent_value' -%}
+{%- if granularity == 'day' -%}
+  {%- set mine = 'cast(' ~ mine ~ ' as date)' -%}
+  {%- set theirs = 'cast(' ~ theirs ~ ' as date)' -%}
+{%- endif %}
+select
+  child.{{ column_name }} as value,
+  parent.parent_value
+from {{ model }} as child
+join (
+  select
+    {{ adapter.quote(field) }} as parent_key,
+    min({{ adapter.quote(parent_column) }}) as parent_value
+  from {{ to }}
+  where {{ adapter.quote(parent_column) }} is not null
+  group by {{ adapter.quote(field) }}
+) as parent
+  on child.{{ adapter.quote(foreign_key) }} = parent.parent_key
+where child.{{ column_name }} is not null
+  and {{ mine }} < {{ theirs }}
+{% endtest %}