model2data 0.2.2__tar.gz → 0.3.0__tar.gz

{model2data-0.2.2 → model2data-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: model2data
-Version: 0.2.2
+Version: 0.3.0
 Summary: Generate analytics-ready datasets from DBML models
 Requires-Python: >=3.10
 Description-Content-Type: text/markdown
@@ -11,6 +11,8 @@ Requires-Dist: faker>=37.12.0
 Requires-Dist: pandas>=2.3.3
 Requires-Dist: pyyaml>=6.0.3
 Requires-Dist: typer>=0.20.0
+Provides-Extra: postgres
+Requires-Dist: dbt-postgres>=1.5.0; extra == "postgres"
 Provides-Extra: dev
 Requires-Dist: pytest; extra == "dev"
 Requires-Dist: pytest-cov; extra == "dev"
@@ -44,8 +46,8 @@ Building analytics or testing dbt pipelines often requires realistic data, but u
 ## How it works (high level)
 
 1. **Parse DBML**: Reads your database schema from a DBML file, extracting tables, columns, types, and relationships.
-2. **Generate Data**: Uses Faker and custom logic to create realistic synthetic data, respecting foreign keys and constraints.
-3. **Scaffold dbt Project**: Creates a dbt project with seeds (CSV files), staging models, profiles, and tests, ready to run with DuckDB.
+2. **Generate Data**: Uses Faker and custom logic to create realistic synthetic data, respecting foreign keys and constraints. Column names are matched against common patterns (`email`, `first_name`, `city`, `phone`, `company`, ...) so a column called `email` gets real-looking emails instead of generic text.
+3. **Scaffold dbt Project**: Creates a dbt project with seeds (CSV files), staging models, profiles, and tests, ready to run with DuckDB or Postgres.
 
 ---
 
@@ -64,7 +66,7 @@ We provide an example Hacker News dataset in `examples/hackernews.dbml`.
 Generate a project with synthetic data:
 
 ```bash
-model2data generate --file examples/hackernews.dbml --rows 200 --seed 42
+model2data --file examples/hackernews.dbml --rows 200 --seed 42
 ```
 
 This creates a `dbt_hackernews/` folder with your data and dbt setup.
@@ -80,6 +82,17 @@ dbt run
 
 Your analytics-ready dataset is now in DuckDB!
 
+To target Postgres instead, install the extra and pass `--adapter postgres`:
+
+```bash
+pip install "model2data[postgres]"
+model2data --file examples/hackernews.dbml --rows 200 --seed 42 --adapter postgres
+```
+
+Connection details are read from environment variables (`MODEL2DATA_PG_HOST`, `MODEL2DATA_PG_PORT`, `MODEL2DATA_PG_USER`, `MODEL2DATA_PG_PASSWORD`, `MODEL2DATA_PG_DATABASE`), defaulting to `localhost:5432` with a `postgres`/`postgres` user for local development.
+
+After generation, the CLI prints a short summary — tables and rows generated, relationships found in the DBML, and any columns that fell back to generic placeholder text because neither their type nor name could be matched.
+
 ---
 
 ## Generated dbt project structure
@@ -102,20 +115,20 @@ dbt_{project_name}/
 ├── macros/
 │   └── generate_schema_name.sql
 ├── dbt_project.yml
-├── profiles.yml  # DuckDB config
-└── {project_name}.duckdb
+├── profiles.yml  # DuckDB or Postgres config, depending on --adapter
+└── {project_name}.duckdb  # DuckDB adapter only
 ```
 
 - **Seeds**: CSV files with generated synthetic data.
 - **Staging Models**: Basic dbt models that load from seeds.
 - **Sources & Tests**: YAML configs defining sources and basic tests (not_null, unique).
-- **Profiles**: Pre-configured for DuckDB with schema handling.
+- **Profiles**: Pre-configured for DuckDB (file-based) or Postgres (via env vars), with schema handling.
 
 ---
 
 ## Design decisions / non-goals
 
-- **DuckDB Default**: Chosen for its zero-config, file-based nature, making it easy to get started without database setup. Other adapters can be configured manually.
+- **DuckDB Default**: Chosen for its zero-config, file-based nature, making it easy to get started without database setup. Postgres is supported via `--adapter postgres`; other adapters can be configured manually.
 - **dbt Integration**: Leverages dbt's transformation capabilities for a familiar workflow in analytics engineering.
 - **Synthetic Data**: Uses deterministic generation for reproducibility; not intended for production use or as a replacement for real data.
 - **Non-goals**: This is not a data migration tool, ETL pipeline, or real-time data generator. It focuses on static, synthetic datasets for testing and prototyping.
@@ -126,17 +139,20 @@ dbt_{project_name}/
 
 - Supports basic DBML features; complex constraints or advanced SQL types may not be fully handled.
 - Synthetic data generation is heuristic-based and may not perfectly mimic real-world distributions or edge cases.
-- Currently optimized for DuckDB; other databases require manual profile adjustments.
+- DuckDB and Postgres are supported today; other databases require manual profile adjustments.
 - No support for incremental models or advanced dbt features in generated projects.
 
 ---
 
 ## Roadmap
 
-- Support for additional database adapters (e.g., Snowflake, BigQuery).
-- Enhanced data type handling and custom generators.
-- Integration with more dbt features like incremental models.
-- Web-based DBML editor and data preview.
+- [x] Postgres adapter support (`--adapter postgres`)
+- [x] Name-aware synthetic data (email, name, address, phone, etc. instead of generic text)
+- [x] Post-run generation summary (tables, rows, relationships, unmapped columns)
+- [ ] Additional database adapters (e.g., Snowflake, BigQuery).
+- [ ] Enhanced data type handling and custom generators.
+- [ ] Integration with more dbt features like incremental models.
+- [ ] Web-based DBML editor and data preview.
 
 ---
 

{model2data-0.2.2 → model2data-0.3.0}/README.md

@@ -20,8 +20,8 @@ Building analytics or testing dbt pipelines often requires realistic data, but u
 ## How it works (high level)
 
 1. **Parse DBML**: Reads your database schema from a DBML file, extracting tables, columns, types, and relationships.
-2. **Generate Data**: Uses Faker and custom logic to create realistic synthetic data, respecting foreign keys and constraints.
-3. **Scaffold dbt Project**: Creates a dbt project with seeds (CSV files), staging models, profiles, and tests, ready to run with DuckDB.
+2. **Generate Data**: Uses Faker and custom logic to create realistic synthetic data, respecting foreign keys and constraints. Column names are matched against common patterns (`email`, `first_name`, `city`, `phone`, `company`, ...) so a column called `email` gets real-looking emails instead of generic text.
+3. **Scaffold dbt Project**: Creates a dbt project with seeds (CSV files), staging models, profiles, and tests, ready to run with DuckDB or Postgres.
 
 ---
 
@@ -40,7 +40,7 @@ We provide an example Hacker News dataset in `examples/hackernews.dbml`.
 Generate a project with synthetic data:
 
 ```bash
-model2data generate --file examples/hackernews.dbml --rows 200 --seed 42
+model2data --file examples/hackernews.dbml --rows 200 --seed 42
 ```
 
 This creates a `dbt_hackernews/` folder with your data and dbt setup.
@@ -56,6 +56,17 @@ dbt run
 
 Your analytics-ready dataset is now in DuckDB!
 
+To target Postgres instead, install the extra and pass `--adapter postgres`:
+
+```bash
+pip install "model2data[postgres]"
+model2data --file examples/hackernews.dbml --rows 200 --seed 42 --adapter postgres
+```
+
+Connection details are read from environment variables (`MODEL2DATA_PG_HOST`, `MODEL2DATA_PG_PORT`, `MODEL2DATA_PG_USER`, `MODEL2DATA_PG_PASSWORD`, `MODEL2DATA_PG_DATABASE`), defaulting to `localhost:5432` with a `postgres`/`postgres` user for local development.
+
+After generation, the CLI prints a short summary — tables and rows generated, relationships found in the DBML, and any columns that fell back to generic placeholder text because neither their type nor name could be matched.
+
 ---
 
 ## Generated dbt project structure
@@ -78,20 +89,20 @@ dbt_{project_name}/
 ├── macros/
 │   └── generate_schema_name.sql
 ├── dbt_project.yml
-├── profiles.yml  # DuckDB config
-└── {project_name}.duckdb
+├── profiles.yml  # DuckDB or Postgres config, depending on --adapter
+└── {project_name}.duckdb  # DuckDB adapter only
 ```
 
 - **Seeds**: CSV files with generated synthetic data.
 - **Staging Models**: Basic dbt models that load from seeds.
 - **Sources & Tests**: YAML configs defining sources and basic tests (not_null, unique).
-- **Profiles**: Pre-configured for DuckDB with schema handling.
+- **Profiles**: Pre-configured for DuckDB (file-based) or Postgres (via env vars), with schema handling.
 
 ---
 
 ## Design decisions / non-goals
 
-- **DuckDB Default**: Chosen for its zero-config, file-based nature, making it easy to get started without database setup. Other adapters can be configured manually.
+- **DuckDB Default**: Chosen for its zero-config, file-based nature, making it easy to get started without database setup. Postgres is supported via `--adapter postgres`; other adapters can be configured manually.
 - **dbt Integration**: Leverages dbt's transformation capabilities for a familiar workflow in analytics engineering.
 - **Synthetic Data**: Uses deterministic generation for reproducibility; not intended for production use or as a replacement for real data.
 - **Non-goals**: This is not a data migration tool, ETL pipeline, or real-time data generator. It focuses on static, synthetic datasets for testing and prototyping.
@@ -102,17 +113,20 @@ dbt_{project_name}/
 
 - Supports basic DBML features; complex constraints or advanced SQL types may not be fully handled.
 - Synthetic data generation is heuristic-based and may not perfectly mimic real-world distributions or edge cases.
-- Currently optimized for DuckDB; other databases require manual profile adjustments.
+- DuckDB and Postgres are supported today; other databases require manual profile adjustments.
 - No support for incremental models or advanced dbt features in generated projects.
 
 ---
 
 ## Roadmap
 
-- Support for additional database adapters (e.g., Snowflake, BigQuery).
-- Enhanced data type handling and custom generators.
-- Integration with more dbt features like incremental models.
-- Web-based DBML editor and data preview.
+- [x] Postgres adapter support (`--adapter postgres`)
+- [x] Name-aware synthetic data (email, name, address, phone, etc. instead of generic text)
+- [x] Post-run generation summary (tables, rows, relationships, unmapped columns)
+- [ ] Additional database adapters (e.g., Snowflake, BigQuery).
+- [ ] Enhanced data type handling and custom generators.
+- [ ] Integration with more dbt features like incremental models.
+- [ ] Web-based DBML editor and data preview.
 
 ---
 

{model2data-0.2.2 → model2data-0.3.0}/model2data/cli.py

@@ -13,9 +13,12 @@ from model2data.dbt.project import (
 )
 from model2data.dbt.tests import generate_dbt_yml
 from model2data.generate.core import generate_data_from_dbml
+from model2data.generate.faker import get_unmapped_columns, reset_stats
 from model2data.parse.dbml import parse_dbml
 from model2data.utils import normalize_identifier
 
+SUPPORTED_ADAPTERS = ("duckdb", "postgres")
+
 app = typer.Typer(
     help=(
         "model2data: Generate analytics-ready datasets from DBML models.\n\n"
@@ -67,11 +70,27 @@ def main(
         "--force",
         help="Overwrite the destination directory if it already exists.",
     ),
+    adapter: str = typer.Option(
+        "duckdb",
+        "--adapter",
+        "-a",
+        help=f"dbt warehouse adapter to target. One of: {', '.join(SUPPORTED_ADAPTERS)}.",
+    ),
 ):
     """
     Generate synthetic data and a dbt project from a DBML model.
     """
 
+    # -------------------------
+    # Validate adapter
+    # -------------------------
+    adapter = adapter.lower()
+    if adapter not in SUPPORTED_ADAPTERS:
+        typer.echo(
+            f"❌ Unsupported adapter '{adapter}'. Choose one of: {', '.join(SUPPORTED_ADAPTERS)}."
+        )
+        raise typer.Exit(1)
+
     # -------------------------
     # Deterministic seed
     # -------------------------
@@ -108,6 +127,7 @@ def main(
     # Generate synthetic data
     # -------------------------
     typer.echo("🧮 Generating synthetic datasets from DBML definitions...")
+    reset_stats()
     generated_tables = generate_data_from_dbml(
         tables=tables,
         refs=refs,
@@ -132,12 +152,29 @@ def main(
     typer.echo("🧪 Generating dbt yml with tests...")
     generate_dbt_yml(dest, tables, refs, project_name)
 
-    typer.echo("🪪 Ensuring dbt profile exists...")
-    create_profiles_yml(dest, profile_name)
+    typer.echo(f"🪪 Ensuring dbt profile exists ({adapter})...")
+    create_profiles_yml(dest, profile_name, adapter=adapter)
 
     # Keep original DBML for reference
     shutil.copy(file, dest / file.name)
 
+    # -------------------------
+    # Summary
+    # -------------------------
+    total_rows = sum(len(df) for df in generated_tables.values())
+    unmapped = get_unmapped_columns()
+
+    typer.echo("\n📊 Summary")
+    typer.echo(f"  Tables generated:        {len(generated_tables)}")
+    typer.echo(f"  Rows generated:          {total_rows}")
+    typer.echo(f"  Relationships in DBML:   {len(refs)}")
+    if unmapped:
+        typer.echo(f"  Columns using generic fallback text: {len(unmapped)}")
+        for col_name, data_type in unmapped:
+            typer.echo(f"    - {col_name} ({data_type})")
+    else:
+        typer.echo("  Columns using generic fallback text: 0")
+
     # -------------------------
     # Done
     # -------------------------

{model2data-0.2.2 → model2data-0.3.0}/model2data/dbt/project.py

@@ -24,7 +24,7 @@ def create_project_scaffold(dest: Path, project_name: str, profile_name: str) ->
     )
 
     # Copy over any macros from templates
-    template_macros_dir = Path("model2data/dbt/templates/macros")
+    template_macros_dir = TEMPLATES_DIR / "macros"
     if template_macros_dir.exists():
         for macro_file in template_macros_dir.glob("*.sql"):
             target_file = dest / "macros" / macro_file.name
@@ -53,7 +53,7 @@ from {{{{ source('raw', '{table_name}') }}}}
             model_file.write_text(sql_content)
 
 
-def create_profiles_yml(dest: Path, profile_name: str) -> None:
+def create_profiles_yml(dest: Path, profile_name: str, adapter: str = "duckdb") -> None:
     profiles_file = dest / "profiles.yml"
     if profiles_file.exists():
         content = profiles_file.read_text()
@@ -62,7 +62,7 @@ def create_profiles_yml(dest: Path, profile_name: str) -> None:
     _render_template(
         template_name="profiles.yml.jinja",
         output_path=profiles_file,
-        context={"profile_name": profile_name},
+        context={"profile_name": profile_name, "adapter": adapter},
     )
 
 

model2data-0.3.0/model2data/dbt/templates/dbt_project.yml.jinja

@@ -0,0 +1,29 @@
+name: '{{ project_name }}'
+version: '1.0'
+config-version: 2
+
+profile: '{{ profile_name }}'
+
+model-paths: ["models"]
+analysis-paths: ["analyses"]
+test-paths: ["data-tests"]
+seed-paths: ["seeds"]
+macro-paths: ["macros"]
+snapshot-paths: ["snapshots"]
+
+target-path: "target"
+clean-targets:
+  - "target"
+  - "dbt_packages"
+
+models:
+  {{ project_name }}:
+    staging:
+      +schema: staging
+      +materialized: view
+    marts:
+      +schema: marts
+      +materialized: table
+
+seeds:
+  +schema: raw

model2data-0.3.0/model2data/dbt/templates/macros/generate_schema_name.sql

@@ -0,0 +1,9 @@
+{% macro generate_schema_name(custom_schema_name, node) %}
+
+{%- set default_schema = target.schema -%}
+{%- if custom_schema_name is none -%}
+    {{ default_schema }}
+{%- else -%}
+    {{ custom_schema_name | trim }}
+{%- endif -%}
+{%- endmacro %}

model2data-0.3.0/model2data/dbt/templates/profiles.yml.jinja

@@ -0,0 +1,18 @@
+{{ profile_name }}:
+  target: dev
+  outputs:
+    dev:
+{% if adapter == "postgres" %}
+      type: postgres
+      threads: 4
+      host: "{% raw %}{{ env_var('MODEL2DATA_PG_HOST', 'localhost') }}{% endraw %}"
+      port: "{% raw %}{{ env_var('MODEL2DATA_PG_PORT', '5432') | as_number }}{% endraw %}"
+      user: "{% raw %}{{ env_var('MODEL2DATA_PG_USER', 'postgres') }}{% endraw %}"
+      pass: "{% raw %}{{ env_var('MODEL2DATA_PG_PASSWORD', 'postgres') }}{% endraw %}"
+      dbname: "{% raw %}{{ env_var('MODEL2DATA_PG_DATABASE', 'postgres') }}{% endraw %}"
+      schema: "{{ profile_name }}"
+{% else %}
+      type: duckdb
+      threads: 1
+      path: "{{ profile_name }}.duckdb"
+{% endif %}

model2data-0.3.0/model2data/generate/faker.py

@@ -0,0 +1,221 @@
+from __future__ import annotations
+
+import random
+import re
+import uuid
+from datetime import datetime, timedelta
+from typing import Callable, Optional
+
+import pandas as pd
+from faker import Faker
+
+from model2data.parse.dbml import ColumnDef
+
+fake = Faker()
+
+
+# ---------------------------------------------------------
+# Column-name -> Faker provider inference
+# ---------------------------------------------------------
+# Ordered most-specific first: a column like "first_name" must match
+# "first_name" before any looser pattern gets a chance. There is
+# deliberately no generic "name" pattern, since "product_name" or
+# "company_name" would otherwise be filled with a person's name.
+_NAME_PATTERNS: list[tuple[str, Callable[[], object]]] = [
+    ("first_name", lambda: fake.first_name()),
+    ("last_name", lambda: fake.last_name()),
+    ("full_name", lambda: fake.name()),
+    ("user_name", lambda: fake.user_name()),
+    ("username", lambda: fake.user_name()),
+    ("password", lambda: fake.password()),
+    ("email", lambda: fake.email()),
+    ("phone", lambda: fake.phone_number()),
+    ("mobile", lambda: fake.phone_number()),
+    ("fax", lambda: fake.phone_number()),
+    ("street", lambda: fake.street_address()),
+    ("address", lambda: fake.address().replace("\n", ", ")),
+    ("city", lambda: fake.city()),
+    ("province", lambda: fake.state()),
+    ("state", lambda: fake.state()),
+    ("country", lambda: fake.country()),
+    ("zip", lambda: fake.postcode()),
+    ("postal", lambda: fake.postcode()),
+    ("homepage", lambda: fake.url()),
+    ("website", lambda: fake.url()),
+    ("url", lambda: fake.url()),
+    ("domain", lambda: fake.domain_name()),
+    ("employer", lambda: fake.company()),
+    ("company", lambda: fake.company()),
+    ("job_title", lambda: fake.job()),
+    ("ip_address", lambda: fake.ipv4()),
+    ("colour", lambda: fake.color_name()),
+    ("color", lambda: fake.color_name()),
+    ("currency", lambda: fake.currency_code()),
+    ("latitude", lambda: fake.latitude()),
+    ("longitude", lambda: fake.longitude()),
+    ("slug", lambda: fake.slug()),
+    ("avatar", lambda: fake.image_url()),
+    ("image", lambda: fake.image_url()),
+    ("bio", lambda: fake.text(max_nb_chars=160)),
+    ("description", lambda: fake.text(max_nb_chars=160)),
+    ("comment", lambda: fake.text(max_nb_chars=160)),
+    ("summary", lambda: fake.text(max_nb_chars=160)),
+]
+
+# Column/table introspection helpers used by both generation and the
+# CLI's post-run summary, so the two stay in sync.
+_stats_state: dict[str, list[tuple[str, str]]] = {"unmapped": []}
+
+
+def reset_stats() -> None:
+    """Clear the record of columns that fell back to generic text."""
+    _stats_state["unmapped"] = []
+
+
+def get_unmapped_columns() -> list[tuple[str, str]]:
+    """Return (column_name, data_type) pairs generated with a generic fallback."""
+    return list(_stats_state["unmapped"])
+
+
+def _infer_by_name(column_name: str) -> Optional[Callable[[], object]]:
+    normalized = re.sub(r"[^a-z0-9]+", "_", column_name.lower())
+    padded = f"_{normalized}_"
+    for pattern, generator in _NAME_PATTERNS:
+        if f"_{pattern}_" in padded:
+            return generator
+    return None
+
+
+# ---------------------------------------------------------
+# Public API
+# ---------------------------------------------------------
+def generate_column_values(
+    column: ColumnDef,
+    row_count: int,
+    fk_series: Optional[pd.Series] = None,
+    ensure_unique: bool = False,
+) -> list:
+    """
+    Generate synthetic values for a single column.
+    Respects FKs, uniqueness, and optional min/max hints in column notes.
+    """
+    if fk_series is not None and not fk_series.empty:
+        fk_values = fk_series.tolist()
+        return [random.choice(fk_values) for _ in range(row_count)]
+
+    dtype = column.data_type.lower()
+    base_type = dtype.split("(")[0].strip()
+    values: list = []
+
+    # Extract min/max from note if present
+    min_val = None
+    max_val = None
+    if column.note:
+        min_val = column.note.get("min")
+        max_val = column.note.get("max")
+
+    # -----------------------------------------------------
+    # UUIDs / hashes
+    # -----------------------------------------------------
+    if "uuid" in base_type or "hash" in base_type:
+        values = [str(uuid.uuid4()) for _ in range(row_count)]
+
+    # -----------------------------------------------------
+    # Integers
+    # -----------------------------------------------------
+    elif any(key in base_type for key in ["int", "integer", "bigint", "smallint"]):
+        # Use note values if present, otherwise defaults
+        if min_val is None:
+            min_val = 0
+        if max_val is None:
+            max_val = 100
+        values = [random.randint(min_val, max_val) for _ in range(row_count)]
+
+    # -----------------------------------------------------
+    # Floats / decimals
+    # -----------------------------------------------------
+    elif any(key in base_type for key in ["decimal", "numeric", "float", "double"]):
+        if min_val is None:
+            min_val = 0
+        if max_val is None:
+            max_val = 10_000
+        values = [round(random.uniform(min_val, max_val), 2) for _ in range(row_count)]
+
+    # -----------------------------------------------------
+    # Booleans
+    # -----------------------------------------------------
+    elif "boolean" in base_type or "bool" in base_type:
+        values = [random.choice([True, False]) for _ in range(row_count)]
+
+    # -----------------------------------------------------
+    # Dates
+    # -----------------------------------------------------
+    elif "date" in base_type and "time" not in base_type:
+        values = [fake.date_between(start_date="-2y", end_date="today") for _ in range(row_count)]
+
+    elif "time" in base_type and "stamp" not in base_type:
+        values = [fake.time() for _ in range(row_count)]
+
+    elif any(key in base_type for key in ["timestamp", "datetime"]):
+        values = [_random_datetime().isoformat(sep=" ") for _ in range(row_count)]
+
+    # -----------------------------------------------------
+    # Untyped / generic string columns: infer intent from the
+    # column name first (email, city, phone...), then fall back
+    # to a literal Faker provider name, then to a generic value.
+    # -----------------------------------------------------
+    else:
+        name_generator = _infer_by_name(column.name)
+        if name_generator is not None:
+            values = [name_generator() for _ in range(row_count)]
+            values = _deduplicate(values, name_generator) if ensure_unique else values
+        else:
+            try:
+                values = [fake.format(base_type) for _ in range(row_count)]
+            except (AttributeError, TypeError):
+                if column.name.lower().endswith("_id") or ensure_unique:
+                    values = [str(uuid.uuid4()) for _ in range(row_count)]
+                else:
+                    _stats_state["unmapped"].append((column.name, column.data_type))
+                    values = [fake.sentence(nb_words=3) for _ in range(row_count)]
+
+    # -----------------------------------------------------
+    # Nullability
+    # -----------------------------------------------------
+    if "not null" not in column.settings:
+        null_fraction = max(0, min(0.2, 1 - (row_count / (row_count + 50))))
+        sample_size = int(row_count * null_fraction)
+        if sample_size:
+            for idx in random.sample(range(row_count), k=sample_size):
+                values[idx] = None
+
+    return values
+
+
+# ---------------------------------------------------------
+# Internal helpers
+# ---------------------------------------------------------
+def _deduplicate(values: list, generator: Callable[[], object], max_attempts: int = 20) -> list:
+    """
+    Best-effort de-duplication for name-inferred values (e.g. unique emails).
+    Retries collisions a bounded number of times, then accepts remaining
+    duplicates rather than looping forever on a small value space.
+    """
+    seen: set = set()
+    result = []
+    for value in values:
+        attempts = 0
+        while value in seen and attempts < max_attempts:
+            value = generator()
+            attempts += 1
+        seen.add(value)
+        result.append(value)
+    return result
+
+
+def _random_datetime(start_days: int = -365, end_days: int = 0) -> datetime:
+    start = datetime.now() + timedelta(days=start_days)
+    end = datetime.now() + timedelta(days=end_days)
+    delta = end - start
+    random_second = random.randint(0, int(delta.total_seconds()))
+    return start + timedelta(seconds=random_second)

{model2data-0.2.2 → model2data-0.3.0}/model2data/parse/dbml.py

@@ -101,6 +101,7 @@ def parse_dbml(dbml_path: Path) -> tuple[dict[str, TableDef], list[dict]]:
 
     current_table: Optional[TableDef] = None
     in_indexes_block = False
+    in_note_block = False
     note_block_depth = 0
     in_ref_block = False
 
@@ -140,12 +141,20 @@ def parse_dbml(dbml_path: Path) -> tuple[dict[str, TableDef], list[dict]]:
                 if cleaned.endswith("}"):
                     in_indexes_block = False
                 continue
+            if in_note_block:
+                if cleaned.endswith("}"):
+                    in_note_block = False
+                continue
             if cleaned.startswith("}"):
                 tables[current_table.name] = current_table
                 current_table = None
                 continue
             if cleaned.startswith("Note:"):
                 continue
+            # Multi-line table note: `Note {` ... `}` (not a column definition)
+            if cleaned.startswith("Note") and cleaned.rstrip().endswith("{"):
+                in_note_block = True
+                continue
 
             col_match = re.match(
                 r'^(".*?"|`.*?`|[A-Za-z_][\w]*)\s+(.+?)(?:\s+\[(.+)\])?$',

{model2data-0.2.2 → model2data-0.3.0}/model2data.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: model2data
-Version: 0.2.2
+Version: 0.3.0
 Summary: Generate analytics-ready datasets from DBML models
 Requires-Python: >=3.10
 Description-Content-Type: text/markdown
@@ -11,6 +11,8 @@ Requires-Dist: faker>=37.12.0
 Requires-Dist: pandas>=2.3.3
 Requires-Dist: pyyaml>=6.0.3
 Requires-Dist: typer>=0.20.0
+Provides-Extra: postgres
+Requires-Dist: dbt-postgres>=1.5.0; extra == "postgres"
 Provides-Extra: dev
 Requires-Dist: pytest; extra == "dev"
 Requires-Dist: pytest-cov; extra == "dev"
@@ -44,8 +46,8 @@ Building analytics or testing dbt pipelines often requires realistic data, but u
 ## How it works (high level)
 
 1. **Parse DBML**: Reads your database schema from a DBML file, extracting tables, columns, types, and relationships.
-2. **Generate Data**: Uses Faker and custom logic to create realistic synthetic data, respecting foreign keys and constraints.
-3. **Scaffold dbt Project**: Creates a dbt project with seeds (CSV files), staging models, profiles, and tests, ready to run with DuckDB.
+2. **Generate Data**: Uses Faker and custom logic to create realistic synthetic data, respecting foreign keys and constraints. Column names are matched against common patterns (`email`, `first_name`, `city`, `phone`, `company`, ...) so a column called `email` gets real-looking emails instead of generic text.
+3. **Scaffold dbt Project**: Creates a dbt project with seeds (CSV files), staging models, profiles, and tests, ready to run with DuckDB or Postgres.
 
 ---
 
@@ -64,7 +66,7 @@ We provide an example Hacker News dataset in `examples/hackernews.dbml`.
 Generate a project with synthetic data:
 
 ```bash
-model2data generate --file examples/hackernews.dbml --rows 200 --seed 42
+model2data --file examples/hackernews.dbml --rows 200 --seed 42
 ```
 
 This creates a `dbt_hackernews/` folder with your data and dbt setup.
@@ -80,6 +82,17 @@ dbt run
 
 Your analytics-ready dataset is now in DuckDB!
 
+To target Postgres instead, install the extra and pass `--adapter postgres`:
+
+```bash
+pip install "model2data[postgres]"
+model2data --file examples/hackernews.dbml --rows 200 --seed 42 --adapter postgres
+```
+
+Connection details are read from environment variables (`MODEL2DATA_PG_HOST`, `MODEL2DATA_PG_PORT`, `MODEL2DATA_PG_USER`, `MODEL2DATA_PG_PASSWORD`, `MODEL2DATA_PG_DATABASE`), defaulting to `localhost:5432` with a `postgres`/`postgres` user for local development.
+
+After generation, the CLI prints a short summary — tables and rows generated, relationships found in the DBML, and any columns that fell back to generic placeholder text because neither their type nor name could be matched.
+
 ---
 
 ## Generated dbt project structure
@@ -102,20 +115,20 @@ dbt_{project_name}/
 ├── macros/
 │   └── generate_schema_name.sql
 ├── dbt_project.yml
-├── profiles.yml  # DuckDB config
-└── {project_name}.duckdb
+├── profiles.yml  # DuckDB or Postgres config, depending on --adapter
+└── {project_name}.duckdb  # DuckDB adapter only
 ```
 
 - **Seeds**: CSV files with generated synthetic data.
 - **Staging Models**: Basic dbt models that load from seeds.
 - **Sources & Tests**: YAML configs defining sources and basic tests (not_null, unique).
-- **Profiles**: Pre-configured for DuckDB with schema handling.
+- **Profiles**: Pre-configured for DuckDB (file-based) or Postgres (via env vars), with schema handling.
 
 ---
 
 ## Design decisions / non-goals
 
-- **DuckDB Default**: Chosen for its zero-config, file-based nature, making it easy to get started without database setup. Other adapters can be configured manually.
+- **DuckDB Default**: Chosen for its zero-config, file-based nature, making it easy to get started without database setup. Postgres is supported via `--adapter postgres`; other adapters can be configured manually.
 - **dbt Integration**: Leverages dbt's transformation capabilities for a familiar workflow in analytics engineering.
 - **Synthetic Data**: Uses deterministic generation for reproducibility; not intended for production use or as a replacement for real data.
 - **Non-goals**: This is not a data migration tool, ETL pipeline, or real-time data generator. It focuses on static, synthetic datasets for testing and prototyping.
@@ -126,17 +139,20 @@ dbt_{project_name}/
 
 - Supports basic DBML features; complex constraints or advanced SQL types may not be fully handled.
 - Synthetic data generation is heuristic-based and may not perfectly mimic real-world distributions or edge cases.
-- Currently optimized for DuckDB; other databases require manual profile adjustments.
+- DuckDB and Postgres are supported today; other databases require manual profile adjustments.
 - No support for incremental models or advanced dbt features in generated projects.
 
 ---
 
 ## Roadmap
 
-- Support for additional database adapters (e.g., Snowflake, BigQuery).
-- Enhanced data type handling and custom generators.
-- Integration with more dbt features like incremental models.
-- Web-based DBML editor and data preview.
+- [x] Postgres adapter support (`--adapter postgres`)
+- [x] Name-aware synthetic data (email, name, address, phone, etc. instead of generic text)
+- [x] Post-run generation summary (tables, rows, relationships, unmapped columns)
+- [ ] Additional database adapters (e.g., Snowflake, BigQuery).
+- [ ] Enhanced data type handling and custom generators.
+- [ ] Integration with more dbt features like incremental models.
+- [ ] Web-based DBML editor and data preview.
 
 ---
 

{model2data-0.2.2 → model2data-0.3.0}/model2data.egg-info/SOURCES.txt

@@ -13,6 +13,9 @@ model2data.egg-info/top_level.txt
 model2data/dbt/__init__.py
 model2data/dbt/project.py
 model2data/dbt/tests.py
+model2data/dbt/templates/dbt_project.yml.jinja
+model2data/dbt/templates/profiles.yml.jinja
+model2data/dbt/templates/macros/generate_schema_name.sql
 model2data/generate/__init__.py
 model2data/generate/core.py
 model2data/generate/faker.py
@@ -25,4 +28,5 @@ tests/test_dbml_parser.py
 tests/test_dbt_naming.py
 tests/test_dbt_project.py
 tests/test_dbt_tests.py
+tests/test_faker_name_inference.py
 tests/test_generation.py

{model2data-0.2.2 → model2data-0.3.0}/model2data.egg-info/requires.txt

@@ -14,3 +14,6 @@ black>=23.0.0
 ty>=0.0.4
 types-pyyaml
 poethepoet>=0.38.0
+
+[postgres]
+dbt-postgres>=1.5.0

{model2data-0.2.2 → model2data-0.3.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "model2data"
-version = "0.2.2"
+version = "0.3.0"
 description = "Generate analytics-ready datasets from DBML models"
 readme = "README.md"
 requires-python = ">=3.10"
@@ -18,6 +18,9 @@ dependencies = [
 ]
 
 [project.optional-dependencies]
+postgres = [
+  "dbt-postgres>=1.5.0",
+]
 dev = [
   "pytest",
   "pytest-cov",
@@ -36,6 +39,12 @@ model2data = "model2data.cli:app"
 where = ["."]
 include = ["model2data*"]
 
+[tool.setuptools.package-data]
+model2data = [
+    "dbt/templates/**/*",
+    "examples/**/*",
+]
+
 # Ruff configuration (fast Python linter)
 [tool.ruff]
 target-version = "py39"

{model2data-0.2.2 → model2data-0.3.0}/tests/test_cli.py

@@ -108,6 +108,89 @@ def test_cli_no_tables_found(tmp_path):
     assert "❌ No tables found in the provided DBML file." in result.stdout
 
 
+def test_cli_rejects_unsupported_adapter(tmp_path):
+    """Test CLI exits cleanly on an unknown --adapter value."""
+    dbml_file = tmp_path / "test.dbml"
+    dbml_file.write_text(
+        """
+    Table users {
+        id int [pk]
+    }
+    """
+    )
+
+    original_cwd = os.getcwd()
+    try:
+        os.chdir(tmp_path)
+        result = runner.invoke(
+            app,
+            ["--file", str(dbml_file), "--adapter", "snowflake"],
+        )
+    finally:
+        os.chdir(original_cwd)
+
+    assert result.exit_code == 1
+    assert "Unsupported adapter" in result.stdout
+
+
+def test_cli_postgres_adapter_generates_postgres_profile(tmp_path):
+    """Test that --adapter postgres produces a postgres profiles.yml."""
+    dbml_file = tmp_path / "test.dbml"
+    dbml_file.write_text(
+        """
+    Table users {
+        id int [pk]
+        email varchar
+    }
+    """
+    )
+
+    original_cwd = os.getcwd()
+    try:
+        os.chdir(tmp_path)
+        result = runner.invoke(
+            app,
+            ["--file", str(dbml_file), "--rows", "10", "--adapter", "postgres"],
+        )
+    finally:
+        os.chdir(original_cwd)
+
+    assert result.exit_code == 0
+    profiles_yml = tmp_path / "dbt_test" / "profiles.yml"
+    content = profiles_yml.read_text()
+    assert "type: postgres" in content
+    assert "env_var('MODEL2DATA_PG_HOST'" in content
+
+
+def test_cli_prints_generation_summary(tmp_path):
+    """Test that the post-run summary reports table/row/relationship counts."""
+    dbml_file = tmp_path / "test.dbml"
+    dbml_file.write_text(
+        """
+    Table users {
+        id int [pk]
+        weird_field custom_unmapped_type
+    }
+    """
+    )
+
+    original_cwd = os.getcwd()
+    try:
+        os.chdir(tmp_path)
+        result = runner.invoke(
+            app,
+            ["--file", str(dbml_file), "--rows", "10"],
+        )
+    finally:
+        os.chdir(original_cwd)
+
+    assert result.exit_code == 0
+    assert "📊 Summary" in result.stdout
+    assert "Tables generated:        1" in result.stdout
+    assert "Rows generated:          10" in result.stdout
+    assert "weird_field" in result.stdout
+
+
 def test_cli_with_custom_name(tmp_path):
     """Test CLI with custom project name."""
     dbml_file = tmp_path / "test.dbml"

{model2data-0.2.2 → model2data-0.3.0}/tests/test_dbml_parser.py

@@ -628,6 +628,30 @@ name varchar
     assert len(tables["t"].columns) == 2
 
 
+def test_note_curly_brace_block_inside_table(tmp_path):
+    """Test the `Note { ... }` block form (distinct from `Note: '...'`).
+
+    Regression test: this form used to fall through to column parsing and
+    produce a spurious column named "Note" with data_type "{".
+    """
+    dbml_file = tmp_path / "note_curly.dbml"
+    dbml_file.write_text(
+        """Table t {
+id int
+name varchar
+Note {
+    'Describes this table across multiple lines'
+}
+email varchar
+}
+"""
+    )
+    tables, refs = parse_dbml(dbml_file)
+    columns = tables["t"].columns
+    assert len(columns) == 3
+    assert all(col.name != "Note" for col in columns)
+
+
 def test_indexes_block(tmp_path):
     """Test indexes block is properly ignored."""
     dbml_file = tmp_path / "with_indexes.dbml"

{model2data-0.2.2 → model2data-0.3.0}/tests/test_dbt_project.py

@@ -3,6 +3,7 @@ import tempfile
 from pathlib import Path
 
 import pytest
+import yaml
 
 from model2data.dbt.project import (
     TEMPLATES_DIR,
@@ -37,6 +38,21 @@ def test_create_project_scaffold_creates_directories(temp_dir):
     assert (temp_dir / "snapshots").exists()
 
 
+def test_create_project_scaffold_copies_schema_macro_regardless_of_cwd(temp_dir, monkeypatch):
+    """Regression test: macro copy used to rely on a CWD-relative path, so it
+    silently copied nothing when model2data was run from anywhere outside the
+    repo checkout (e.g. after a normal `pip install`). Without the macro,
+    generated projects fail at `dbt run` with a schema-not-found error.
+    """
+    monkeypatch.chdir(temp_dir.parent)
+
+    create_project_scaffold(temp_dir, "test_project", "test_profile")
+
+    macro_file = temp_dir / "macros" / "generate_schema_name.sql"
+    assert macro_file.exists()
+    assert "generate_schema_name" in macro_file.read_text()
+
+
 def test_create_project_scaffold_creates_dbt_project_yml(temp_dir):
     """Test that dbt_project.yml is created with correct content."""
     project_name = "my_analytics"
@@ -180,6 +196,45 @@ def test_create_profiles_yml_appends_if_different_profile(temp_dir):
     assert profiles_file.exists()
 
 
+def test_create_profiles_yml_defaults_to_duckdb(temp_dir):
+    """Test that omitting adapter still produces a duckdb profile."""
+    profile_name = "duck_profile"
+
+    create_profiles_yml(temp_dir, profile_name)
+
+    content = (temp_dir / "profiles.yml").read_text()
+    parsed = yaml.safe_load(content)
+    assert parsed[profile_name]["outputs"]["dev"]["type"] == "duckdb"
+    assert parsed[profile_name]["outputs"]["dev"]["path"] == f"{profile_name}.duckdb"
+
+
+def test_create_profiles_yml_postgres_adapter(temp_dir):
+    """Test that the postgres adapter renders a valid, env-var-driven profile."""
+    profile_name = "pg_profile"
+
+    create_profiles_yml(temp_dir, profile_name, adapter="postgres")
+
+    content = (temp_dir / "profiles.yml").read_text()
+    parsed = yaml.safe_load(content)
+    dev_output = parsed[profile_name]["outputs"]["dev"]
+
+    assert dev_output["type"] == "postgres"
+    assert dev_output["schema"] == profile_name
+    # Connection details are sourced from env vars, not hardcoded secrets.
+    assert "env_var('MODEL2DATA_PG_HOST'" in content
+    assert "env_var('MODEL2DATA_PG_PORT'" in content
+    assert "env_var('MODEL2DATA_PG_USER'" in content
+    assert "env_var('MODEL2DATA_PG_PASSWORD'" in content
+    assert "env_var('MODEL2DATA_PG_DATABASE'" in content
+
+
+def test_create_profiles_yml_rejects_no_hardcoded_duckdb_type_for_postgres(temp_dir):
+    """Guard against the postgres branch accidentally falling through to duckdb."""
+    create_profiles_yml(temp_dir, "mixed_profile", adapter="postgres")
+    content = (temp_dir / "profiles.yml").read_text()
+    assert "type: duckdb" not in content
+
+
 def test_render_template_creates_output(temp_dir):
     """Test that _render_template creates output file with rendered content."""
     # This test assumes templates exist in the templates directory

{model2data-0.2.2 → model2data-0.3.0}/tests/test_dbt_tests.py

@@ -33,6 +33,7 @@ def test_dbt_tests_generation(tmp_path, monkeypatch):
         seed=42,
         name="test_project",
         force=True,
+        adapter="duckdb",
     )
 
     project_dir = tmp_path / "dbt_test_project"

model2data-0.3.0/tests/test_faker_name_inference.py

@@ -0,0 +1,79 @@
+"""Tests for column-name-based Faker inference in generate.faker."""
+
+import re
+
+from model2data.generate.faker import (
+    _deduplicate,
+    generate_column_values,
+    get_unmapped_columns,
+    reset_stats,
+)
+from model2data.parse.dbml import ColumnDef
+
+EMAIL_RE = re.compile(r"^[^@\s]+@[^@\s]+\.[^@\s]+$")
+
+
+class TestNameInference:
+    def test_email_column_gets_real_emails(self):
+        col = ColumnDef(name="email", data_type="varchar", settings={"not null"})
+        values = generate_column_values(col, row_count=20)
+        assert all(EMAIL_RE.match(v) for v in values)
+
+    def test_first_and_last_name_are_distinct_providers(self):
+        first = generate_column_values(
+            ColumnDef(name="first_name", data_type="varchar", settings={"not null"}),
+            row_count=10,
+        )
+        last = generate_column_values(
+            ColumnDef(name="last_name", data_type="varchar", settings={"not null"}),
+            row_count=10,
+        )
+        # Different providers should not consistently produce identical output.
+        assert first != last
+
+    def test_generic_product_name_is_not_treated_as_a_person_name(self):
+        # "name" alone is intentionally unmapped so "product_name" style
+        # columns don't get filled with people's names.
+        col = ColumnDef(name="product_name", data_type="varchar", settings={"not null"})
+        values = generate_column_values(col, row_count=5)
+        assert len(values) == 5
+        assert all(isinstance(v, str) for v in values)
+
+    def test_city_and_country_columns(self):
+        city_values = generate_column_values(
+            ColumnDef(name="city", data_type="varchar", settings={"not null"}), row_count=5
+        )
+        country_values = generate_column_values(
+            ColumnDef(name="country", data_type="varchar", settings={"not null"}), row_count=5
+        )
+        assert all(isinstance(v, str) and v for v in city_values)
+        assert all(isinstance(v, str) and v for v in country_values)
+
+    def test_unmatched_type_and_name_is_tracked_in_stats(self):
+        reset_stats()
+        col = ColumnDef(name="misc_notes_field", data_type="weird_custom_type")
+        generate_column_values(col, row_count=3)
+        unmapped = get_unmapped_columns()
+        assert ("misc_notes_field", "weird_custom_type") in unmapped
+
+    def test_matched_name_is_not_tracked_as_unmapped(self):
+        reset_stats()
+        col = ColumnDef(name="email", data_type="varchar", settings={"not null"})
+        generate_column_values(col, row_count=3)
+        assert get_unmapped_columns() == []
+
+    def test_ensure_unique_deduplicates_name_inferred_values(self):
+        col = ColumnDef(name="email", data_type="varchar", settings={"not null", "pk"})
+        values = generate_column_values(col, row_count=50, ensure_unique=True)
+        assert len(values) == len(set(values)) == 50
+
+    def test_deduplicate_retries_on_collision(self):
+        # Value space of 2 with 3 requested values forces at least one retry.
+        pool = iter(["a", "a", "b", "c", "a"])
+        result = _deduplicate(["a", "a", "b"], generator=lambda: next(pool))
+        assert len(result) == len(set(result)) == 3
+
+    def test_deduplicate_gives_up_after_max_attempts_on_tiny_value_space(self):
+        # Only one possible value: dedup can't succeed, but must terminate.
+        result = _deduplicate(["x", "x", "x"], generator=lambda: "x", max_attempts=3)
+        assert result == ["x", "x", "x"]

model2data-0.2.2/model2data/generate/faker.py

@@ -1,122 +0,0 @@
-from __future__ import annotations
-
-import random
-import uuid
-from datetime import datetime, timedelta
-from typing import Optional
-
-import pandas as pd
-from faker import Faker
-
-from model2data.parse.dbml import ColumnDef
-
-fake = Faker()
-
-
-# ---------------------------------------------------------
-# Public API
-# ---------------------------------------------------------
-def generate_column_values(
-    column: ColumnDef,
-    row_count: int,
-    fk_series: Optional[pd.Series] = None,
-    ensure_unique: bool = False,
-) -> list:
-    """
-    Generate synthetic values for a single column.
-    Respects FKs, uniqueness, and optional min/max hints in column notes.
-    """
-    if fk_series is not None and not fk_series.empty:
-        fk_values = fk_series.tolist()
-        return [random.choice(fk_values) for _ in range(row_count)]
-
-    dtype = column.data_type.lower()
-    base_type = dtype.split("(")[0].strip()
-    values: list = []
-
-    # Extract min/max from note if present
-    min_val = None
-    max_val = None
-    if column.note:
-        min_val = column.note.get("min")
-        max_val = column.note.get("max")
-
-    # -----------------------------------------------------
-    # UUIDs / hashes
-    # -----------------------------------------------------
-    if "uuid" in base_type or "hash" in base_type:
-        values = [str(uuid.uuid4()) for _ in range(row_count)]
-
-    # -----------------------------------------------------
-    # Integers
-    # -----------------------------------------------------
-    elif any(key in base_type for key in ["int", "integer", "bigint", "smallint"]):
-        # Use note values if present, otherwise defaults
-        if min_val is None:
-            min_val = 0
-        if max_val is None:
-            max_val = 100
-        values = [random.randint(min_val, max_val) for _ in range(row_count)]
-
-    # -----------------------------------------------------
-    # Floats / decimals
-    # -----------------------------------------------------
-    elif any(key in base_type for key in ["decimal", "numeric", "float", "double"]):
-        if min_val is None:
-            min_val = 0
-        if max_val is None:
-            max_val = 10_000
-        values = [round(random.uniform(min_val, max_val), 2) for _ in range(row_count)]
-
-    # -----------------------------------------------------
-    # Booleans
-    # -----------------------------------------------------
-    elif "boolean" in base_type or "bool" in base_type:
-        values = [random.choice([True, False]) for _ in range(row_count)]
-
-    # -----------------------------------------------------
-    # Dates
-    # -----------------------------------------------------
-    elif "date" in base_type and "time" not in base_type:
-        values = [fake.date_between(start_date="-2y", end_date="today") for _ in range(row_count)]
-
-    elif "time" in base_type and "stamp" not in base_type:
-        values = [fake.time() for _ in range(row_count)]
-
-    elif any(key in base_type for key in ["timestamp", "datetime"]):
-        values = [_random_datetime().isoformat(sep=" ") for _ in range(row_count)]
-
-    # -----------------------------------------------------
-    # Fallback to Faker providers
-    # -----------------------------------------------------
-    else:
-        try:
-            values = [fake.format(base_type) for _ in range(row_count)]
-        except (AttributeError, TypeError):
-            if column.name.lower().endswith("_id") or ensure_unique:
-                values = [str(uuid.uuid4()) for _ in range(row_count)]
-            else:
-                values = [fake.sentence(nb_words=3) for _ in range(row_count)]
-
-    # -----------------------------------------------------
-    # Nullability
-    # -----------------------------------------------------
-    if "not null" not in column.settings:
-        null_fraction = max(0, min(0.2, 1 - (row_count / (row_count + 50))))
-        sample_size = int(row_count * null_fraction)
-        if sample_size:
-            for idx in random.sample(range(row_count), k=sample_size):
-                values[idx] = None
-
-    return values
-
-
-# ---------------------------------------------------------
-# Internal helpers
-# ---------------------------------------------------------
-def _random_datetime(start_days: int = -365, end_days: int = 0) -> datetime:
-    start = datetime.now() + timedelta(days=start_days)
-    end = datetime.now() + timedelta(days=end_days)
-    delta = end - start
-    random_second = random.randint(0, int(delta.total_seconds()))
-    return start + timedelta(seconds=random_second)