pipelantic 0.2.0__tar.gz → 0.3.0__tar.gz

{pipelantic-0.2.0 → pipelantic-0.3.0}/CHANGELOG.md

@@ -5,6 +5,44 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.3.0] - 2026-07-16
+
+### Added
+
+- `Data` as the preferred thin facade over ContractModel (`DataContractModel`
+  remains as a deprecated alias)
+- Multi-phase validation (structural, reference, semantic, policy, capability)
+  with diagnostic actions and source symbols
+- `Profile` templates, `SecretRef`, scoped registries, and capability negotiation
+- Immutable secret-free `PipelinePlan` IR (`pipelantic.plan/1`) with slicing,
+  explain output, and canonical fingerprints
+- Schema-drift models (`NormalizedSchema`, observations, changes, `DriftImpact`)
+- Portable reliability/intent models (freshness, write/materialization intents,
+  idempotency, evidence schemas)
+- JSON Schemas for profiles, project config, and PipelinePlan
+- `pipelantic` CLI: `validate`, `inspect`, `plan`, and `plan explain`
+  (plus `plan --explain` alias, `--nodes`, and mutual exclusion for
+  `--run-one` / `--run-until`)
+- `Pipeline.plan()` / `Pipeline.explain_plan()`
+
+### Fixed
+
+- `run_until` selects declaration-order prefix (not only upstream closure)
+- Unknown plan selections fail closed with `PMPLAN501`
+- Multi-engine regions and cross-engine durable artifact strategies
+- Profile snapshot included in plan fingerprint
+- Strict policy requires registered transformation implementations
+- Subpipeline validation inherits parent planning context
+- Plan JSON round-trip restores `secret_ref` and verifies fingerprints
+- Schema-only DTCS ports map `datetime`/`decimal`/`binary` correctly
+- Diff APIs return diagnostics for malformed toolkit input
+
+### Changed
+
+- Package version and public status advance to 0.3 (Validation and Pipeline Plan IR)
+- Explicit `click` dependency for the CLI plan command group (Typer 0.15+ no
+  longer pulls it in transitively)
+
 ## [0.2.0] - 2026-07-16
 
 ### Added
@@ -48,5 +86,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - uv + ruff toolchain, MkDocs documentation site, shared GitHub Actions
   checks, and tag-triggered PyPI release
 
+[0.3.0]: https://github.com/eddiethedean/pipelantic/releases/tag/v0.3.0
 [0.2.0]: https://github.com/eddiethedean/pipelantic/releases/tag/v0.2.0
 [0.1.0]: https://github.com/eddiethedean/pipelantic/releases/tag/v0.1.0

{pipelantic-0.2.0 → pipelantic-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pipelantic
-Version: 0.2.0
+Version: 0.3.0
 Summary: Typed, contract-driven data pipeline modeling for Python.
 Project-URL: Homepage, https://github.com/eddiethedean/pipelantic
 Project-URL: Documentation, https://github.com/eddiethedean/pipelantic/tree/main/docs
@@ -20,11 +20,13 @@ Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
 Classifier: Typing :: Typed
 Requires-Python: >=3.11
+Requires-Dist: click<9,>=8
 Requires-Dist: contractmodel>=0.1.2
 Requires-Dist: dpcs<1,>=0.13
 Requires-Dist: dtcs<1,>=0.11
 Requires-Dist: packaging>=24
 Requires-Dist: pydantic<3,>=2.12
+Requires-Dist: typer<1,>=0.15
 Description-Content-Type: text/markdown
 
 ![Pipelantic banner](https://raw.githubusercontent.com/eddiethedean/pipelantic/main/docs/theme/assets/pipelantic-banner.png)
@@ -40,11 +42,11 @@ Typed, contract-driven data pipeline modeling for Python.
 
 ## Status
 
-**0.2.0 — Contract Interoperability**
+**0.3.0 — Validation and Pipeline Plan IR**
 
-Pipelantic provides the typed modeling kernel plus ODCS/DTCS/DPCS generate and
-load, deterministic contract bundles, and compatibility hooks. Planning and
-execution plugins arrive in later milestones.
+Pipelantic provides the typed modeling kernel, contract interoperability, and
+an immutable secret-free `PipelinePlan`. Planning is pure (no secret
+resolution or execution). Local runtime arrives in later milestones.
 
 See the [hosted documentation](https://pipelantic.readthedocs.io/) for the
 full design,
@@ -78,8 +80,8 @@ the `dev` dependency group (pytest, ruff, mkdocs) by default.
 Tag a version that matches `src/pipelantic/_version.py`, then push the tag:
 
 ```bash
-git tag v0.2.0
-git push origin v0.2.0
+git tag v0.3.0
+git push origin v0.3.0
 ```
 
 GitHub Actions runs checks and publishes to PyPI using the `PYPI_API_TOKEN`
@@ -88,17 +90,16 @@ repository secret.
 ## Quick example
 
 ```python
-from contractmodel import ContractModel as DataContractModel
-from pipelantic import Input, Output, Pipeline, Sink, Source, Transformation
+from pipelantic import Data, Input, Output, Pipeline, Sink, Source, Transformation
 
 
-class RawCustomer(DataContractModel):
+class RawCustomer(Data):
     customer_id: int
     first_name: str
     last_name: str
 
 
-class Customer(DataContractModel):
+class Customer(Data):
     customer_id: int
     full_name: str
 

{pipelantic-0.2.0 → pipelantic-0.3.0}/README.md

@@ -11,11 +11,11 @@ Typed, contract-driven data pipeline modeling for Python.
 
 ## Status
 
-**0.2.0 — Contract Interoperability**
+**0.3.0 — Validation and Pipeline Plan IR**
 
-Pipelantic provides the typed modeling kernel plus ODCS/DTCS/DPCS generate and
-load, deterministic contract bundles, and compatibility hooks. Planning and
-execution plugins arrive in later milestones.
+Pipelantic provides the typed modeling kernel, contract interoperability, and
+an immutable secret-free `PipelinePlan`. Planning is pure (no secret
+resolution or execution). Local runtime arrives in later milestones.
 
 See the [hosted documentation](https://pipelantic.readthedocs.io/) for the
 full design,
@@ -49,8 +49,8 @@ the `dev` dependency group (pytest, ruff, mkdocs) by default.
 Tag a version that matches `src/pipelantic/_version.py`, then push the tag:
 
 ```bash
-git tag v0.2.0
-git push origin v0.2.0
+git tag v0.3.0
+git push origin v0.3.0
 ```
 
 GitHub Actions runs checks and publishes to PyPI using the `PYPI_API_TOKEN`
@@ -59,17 +59,16 @@ repository secret.
 ## Quick example
 
 ```python
-from contractmodel import ContractModel as DataContractModel
-from pipelantic import Input, Output, Pipeline, Sink, Source, Transformation
+from pipelantic import Data, Input, Output, Pipeline, Sink, Source, Transformation
 
 
-class RawCustomer(DataContractModel):
+class RawCustomer(Data):
     customer_id: int
     first_name: str
     last_name: str
 
 
-class Customer(DataContractModel):
+class Customer(Data):
     customer_id: int
     full_name: str
 

{pipelantic-0.2.0 → pipelantic-0.3.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "pipelantic"
-version = "0.2.0"
+version = "0.3.0"
 description = "Typed, contract-driven data pipeline modeling for Python."
 readme = "README.md"
 license = "MIT"
@@ -23,8 +23,13 @@ dependencies = [
     "packaging>=24",
     "dtcs>=0.11,<1",
     "dpcs>=0.13,<1",
+    "typer>=0.15,<1",
+    "click>=8,<9",
 ]
 
+[project.scripts]
+pipelantic = "pipelantic.cli:run"
+
 [project.urls]
 Homepage = "https://github.com/eddiethedean/pipelantic"
 Documentation = "https://github.com/eddiethedean/pipelantic/tree/main/docs"

pipelantic-0.3.0/src/pipelantic/__init__.py

@@ -0,0 +1,189 @@
+"""Pipelantic — typed, contract-driven data pipeline modeling.
+
+0.3 adds multi-phase validation, profiles, SecretRef, scoped registries,
+schema-drift and reliability models, and an immutable PipelinePlan IR.
+"""
+
+from __future__ import annotations
+
+import warnings
+from typing import Any
+
+from pipelantic._version import __version__
+from pipelantic.capabilities import CapabilityDecision, PluginCapabilities
+from pipelantic.contracts import Data, load_data_contract, write_odcs
+from pipelantic.diagnostics import (
+    Diagnostic,
+    DiagnosticAction,
+    Severity,
+    SourceLocation,
+    ValidationReport,
+)
+from pipelantic.exceptions import (
+    ModelDefinitionError,
+    PipelanticError,
+    PipelineValidationError,
+)
+from pipelantic.interchange import (
+    ArtifactProvenance,
+    ContractBundle,
+    ProvenanceKind,
+    diff_data_contracts,
+    diff_pipelines,
+    diff_transformations,
+    generate_contracts,
+    graphs_equivalent,
+    load_bundle,
+    normalize_pipeline,
+    write_contracts,
+)
+from pipelantic.model import Edge, LogicalGraph, Node, NodeKind
+from pipelantic.pipeline import Pipeline, Sink, Source, SubpipelineInstance
+from pipelantic.plan import (
+    ArtifactRef,
+    ArtifactStrategy,
+    PipelinePlan,
+    explain_plan,
+    plan_pipeline,
+)
+from pipelantic.policy import ValidationPolicy
+from pipelantic.ports import Input, Output, Parameter
+from pipelantic.profile import (
+    Profile,
+    development_profile,
+    load_profile,
+    production_profile,
+    resolve_profile,
+    test_profile,
+    write_profile,
+)
+from pipelantic.refs import OutputRef
+from pipelantic.registry import (
+    BindingDescriptor,
+    ImplementationDescriptor,
+    PlanningContext,
+    PluginDescriptor,
+    RegistryBundle,
+    builtin_stub_registry,
+)
+from pipelantic.reliability import (
+    BackfillDeclaration,
+    FreshnessExpectation,
+    IdempotencyDeclaration,
+    MaterializationIntent,
+    MaterializationMode,
+    PartitionCompletenessExpectation,
+    ReconciliationDeclaration,
+    ReliabilityEvidence,
+    RepairDeclaration,
+    RetrySafetyDeclaration,
+    WriteIntent,
+    WriteMode,
+)
+from pipelantic.schema_drift import (
+    DriftImpact,
+    NormalizedSchema,
+    SchemaChange,
+    SchemaChangeSet,
+    SchemaObservation,
+    diff_contract_schemas,
+    diff_normalized_schemas,
+    normalize_schema_from_model,
+)
+from pipelantic.secrets import SecretRef
+from pipelantic.transformation import ImplementationRecord, Step, Transformation
+
+__all__ = [
+    "ArtifactProvenance",
+    "ArtifactRef",
+    "ArtifactStrategy",
+    "BackfillDeclaration",
+    "BindingDescriptor",
+    "CapabilityDecision",
+    "ContractBundle",
+    "Data",
+    "DataContractModel",
+    "Diagnostic",
+    "DiagnosticAction",
+    "DriftImpact",
+    "Edge",
+    "FreshnessExpectation",
+    "IdempotencyDeclaration",
+    "ImplementationDescriptor",
+    "ImplementationRecord",
+    "Input",
+    "LogicalGraph",
+    "MaterializationIntent",
+    "MaterializationMode",
+    "ModelDefinitionError",
+    "Node",
+    "NodeKind",
+    "NormalizedSchema",
+    "Output",
+    "OutputRef",
+    "Parameter",
+    "PartitionCompletenessExpectation",
+    "PipelanticError",
+    "Pipeline",
+    "PipelinePlan",
+    "PipelineValidationError",
+    "PlanningContext",
+    "PluginCapabilities",
+    "PluginDescriptor",
+    "Profile",
+    "ProvenanceKind",
+    "ReconciliationDeclaration",
+    "RegistryBundle",
+    "ReliabilityEvidence",
+    "RepairDeclaration",
+    "RetrySafetyDeclaration",
+    "SchemaChange",
+    "SchemaChangeSet",
+    "SchemaObservation",
+    "SecretRef",
+    "Severity",
+    "Sink",
+    "Source",
+    "SourceLocation",
+    "Step",
+    "SubpipelineInstance",
+    "Transformation",
+    "ValidationPolicy",
+    "ValidationReport",
+    "WriteIntent",
+    "WriteMode",
+    "__version__",
+    "builtin_stub_registry",
+    "development_profile",
+    "diff_contract_schemas",
+    "diff_data_contracts",
+    "diff_normalized_schemas",
+    "diff_pipelines",
+    "diff_transformations",
+    "explain_plan",
+    "generate_contracts",
+    "graphs_equivalent",
+    "load_bundle",
+    "load_data_contract",
+    "load_profile",
+    "normalize_pipeline",
+    "normalize_schema_from_model",
+    "plan_pipeline",
+    "production_profile",
+    "resolve_profile",
+    "test_profile",
+    "write_contracts",
+    "write_odcs",
+    "write_profile",
+]
+
+
+def __getattr__(name: str) -> Any:
+    if name == "DataContractModel":
+        warnings.warn(
+            "DataContractModel is deprecated; use pipelantic.Data instead.",
+            DeprecationWarning,
+            stacklevel=2,
+        )
+        return Data
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

{pipelantic-0.2.0 → pipelantic-0.3.0}/src/pipelantic/_version.py

@@ -1,3 +1,3 @@
 """Package version."""
 
-__version__ = "0.2.0"
+__version__ = "0.3.0"

pipelantic-0.3.0/src/pipelantic/capabilities.py

@@ -0,0 +1,134 @@
+"""Plugin capability declarations and negotiation results."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from enum import StrEnum
+from typing import Any
+
+
+class CapabilityDecision(StrEnum):
+    """Outcome of comparing required vs available capabilities."""
+
+    SUPPORTED = "supported"
+    FALLBACK = "fallback"
+    UNSUPPORTED = "unsupported"
+
+
+@dataclass(frozen=True, slots=True)
+class PluginCapabilities:
+    """Declared capabilities of a plugin or engine."""
+
+    engine: str
+    async_execution: bool = False
+    streaming: bool = False
+    transactions: bool = False
+    checkpoints: bool = False
+    sql: bool = False
+    spark: bool = False
+    dataframe: bool = True
+    secret_provider: bool = False
+    extras: frozenset[str] = field(default_factory=frozenset)
+
+    def supports(self, requirement: str) -> bool:
+        """Return True when this capability set covers ``requirement``."""
+        known = {
+            "async": self.async_execution,
+            "async_execution": self.async_execution,
+            "streaming": self.streaming,
+            "transactions": self.transactions,
+            "checkpoints": self.checkpoints,
+            "sql": self.sql,
+            "spark": self.spark,
+            "dataframe": self.dataframe,
+            "secret_provider": self.secret_provider,
+        }
+        if requirement in known:
+            return known[requirement]
+        return requirement in self.extras
+
+    def to_dict(self) -> dict[str, Any]:
+        """Serialize capabilities."""
+        return {
+            "engine": self.engine,
+            "async_execution": self.async_execution,
+            "streaming": self.streaming,
+            "transactions": self.transactions,
+            "checkpoints": self.checkpoints,
+            "sql": self.sql,
+            "spark": self.spark,
+            "dataframe": self.dataframe,
+            "secret_provider": self.secret_provider,
+            "extras": sorted(self.extras),
+        }
+
+
+@dataclass(frozen=True, slots=True)
+class CapabilityNegotiation:
+    """Record of a capability check for one requirement."""
+
+    requirement: str
+    engine: str
+    decision: CapabilityDecision
+    fallback_engine: str | None = None
+    message: str | None = None
+
+    def to_dict(self) -> dict[str, Any]:
+        """Serialize negotiation record."""
+        return {
+            "requirement": self.requirement,
+            "engine": self.engine,
+            "decision": self.decision.value,
+            "fallback_engine": self.fallback_engine,
+            "message": self.message,
+        }
+
+
+def negotiate_capabilities(
+    *,
+    requirements: list[str],
+    available: PluginCapabilities,
+    fallback: PluginCapabilities | None = None,
+    allow_fallback: bool = False,
+) -> list[CapabilityNegotiation]:
+    """Negotiate required capabilities against an available engine.
+
+    Unsupported requirements fail closed unless ``allow_fallback`` is True and
+    a fallback engine covers the requirement.
+    """
+    results: list[CapabilityNegotiation] = []
+    for requirement in requirements:
+        if available.supports(requirement):
+            results.append(
+                CapabilityNegotiation(
+                    requirement=requirement,
+                    engine=available.engine,
+                    decision=CapabilityDecision.SUPPORTED,
+                )
+            )
+            continue
+        if allow_fallback and fallback is not None and fallback.supports(requirement):
+            results.append(
+                CapabilityNegotiation(
+                    requirement=requirement,
+                    engine=available.engine,
+                    decision=CapabilityDecision.FALLBACK,
+                    fallback_engine=fallback.engine,
+                    message=(
+                        f"Requirement {requirement!r} unsupported by "
+                        f"{available.engine}; using fallback {fallback.engine}."
+                    ),
+                )
+            )
+            continue
+        results.append(
+            CapabilityNegotiation(
+                requirement=requirement,
+                engine=available.engine,
+                decision=CapabilityDecision.UNSUPPORTED,
+                message=(
+                    f"Requirement {requirement!r} unsupported by {available.engine}."
+                ),
+            )
+        )
+    return results