etlantic 0.10.0__tar.gz

etlantic-0.10.0/.gitignore

@@ -0,0 +1,24 @@
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.egg-info/
+.eggs/
+dist/
+build/
+site/
+.venv/
+venv/
+.env
+
+# Tools
+.pytest_cache/
+.ruff_cache/
+.mypy_cache/
+.coverage
+htmlcov/
+.DS_Store
+examples/_file_storage_out/
+examples/_generated_*.py
+tmp_cli_contracts/
+tmp_/

etlantic-0.10.0/CHANGELOG.md

@@ -0,0 +1,382 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.10.0] - 2026-07-17
+
+### Added
+
+- Optional `etlantic-sparkforge` migration adapter (IR → Pipeline / Profile)
+- SparkForge debug/run-mode mapping to `RunSelection` / `RunIntent` / `DebugSession`
+- SparkForge-shaped result normalization to `PipelineRunReport` with redaction
+- Write / Delta capability compatibility helpers (fail closed)
+- Representative ecommerce IR fixture and semantic parity suite
+- Migration guide `docs/11_DEVELOPMENT/MIGRATION_0_9_TO_0_10.md`
+- Runnable docs guides for Airflow compile and SparkForge adapter
+- Docs status SSOT via CAPABILITIES + expanded `scripts/check_docs.py` guards
+
+### Changed
+
+- Package version advances to 0.10 (SparkForge Migration Preview)
+- Plugin packages require `etlantic>=0.10.0,<1.0`
+- SparkForge adoption checklist prerequisites marked complete
+- Release workflow paces PyPI uploads (10 minutes between packages)
+- Install docs prefer from-source until `v0.10.0` wheels exist on PyPI
+- Agent guidance lists `etlantic.viz` and optional `etlantic-airflow` for compile
+
+### Fixed
+
+- Schema history refuses row-like metadata by key (no substring false positives)
+- Core `redact_message` / `redact_value` cover HTTPS basic-auth and string leaves
+- SparkForge report adapter redacts free-text errors, Bearer tokens, and DSNs
+- `strict_delta=False` emits warnings for missing Delta capabilities (PMSF323)
+- SparkForge IR missing step names emit `PMSF310` instead of raising `KeyError`
+- Retry policy mapping clamps `max_attempts` to at least 1
+- Unknown validation policy names fail closed (`KeyError`) instead of inventing
+  empty policies
+- Testing helpers harden write-semantics, orchestrator secret scans, and missing
+  secret fail-closed behavior
+- Adapter stops importing private `_PipelineNamespace`
+- Docs SSOT: Graphviz/HTML/lineage, Spark, Airflow, and Mermaid status drift
+
+### Upgrade notes
+
+- Install `etlantic-sparkforge` (or `etlantic[sparkforge]`) for the adapter
+- ETLantic core remains free of bronze / silver / gold types
+- Full SparkForge engine retirement remains a progressive path (see migration guide)
+- Prefer `git tag v0.10.0 && git push origin v0.10.0` (not `git push --tags`)
+
+## [0.9.0] - 2026-07-17
+
+### Added
+
+- CLI wrappers: `compile`, `generate`, `diff`, `plugin list|info`, schema family,
+  reliability ops, viz exporters, and `report compare`
+- `Profile.plugin_allowlist` with version pins; production fail-closed trust
+- `FileSchemaHistoryProvider`, SARIF/GitHub diagnostic renderers
+- Reliability provider protocols (quality, statistical, reconciliation, env)
+- Observability protocols, JSON console logger, optional OpenTelemetry adapter
+- `FileReportStore` and report comparison helpers
+- Graphviz DOT / HTML lineage / JSON lineage exporters (`etlantic.viz`)
+- Optional packages `etlantic-keyring` and `etlantic-sqlmodel`
+- IDE command/result JSON schemas (`etlantic.ide`)
+- Notebook / IPython display helpers (`etlantic.notebook`)
+- Agent guidance generators + `scripts/check_agent_guidance.py`
+- Expanded `etlantic.testing` (orchestrator, secrets, write-semantics parity)
+- Migration guide `docs/11_DEVELOPMENT/MIGRATION_0_8_TO_0_9.md`
+
+### Changed
+
+- Package version advances to 0.9 (Tooling, SDK, and Ecosystem Readiness)
+- Plugin packages require `etlantic>=0.9.0,<1.0`
+- Graphviz / HTML / lineage docs promoted out of Future Design
+
+### Upgrade notes
+
+- Production profiles should set `plugin_allowlist` explicitly
+- Schema history stores fingerprints only — never source rows
+- Core remains free of Airflow, Spark, SQL drivers, SQLModel, keyring, and OTel
+  unless extras / optional packages are installed
+
+## [0.8.0] - 2026-07-17
+
+### Added
+
+- Versioned orchestration protocol (`etlantic.orchestration/1`) with compile,
+  artifact transport, retry-safety, and lifecycle correlation models
+- Independently installable `etlantic-airflow` reference compiler
+- `compile_plan(...)` and `PipelinePlan.compile(target=...)`
+- `Profile.schedule`, `Profile.execution`, and
+  `required_orchestrator_capabilities`
+- Entry-point group `etlantic.orchestrator_plugins`
+- Runnable example `examples/airflow_compile.py`
+- Migration guide `docs/11_DEVELOPMENT/MIGRATION_0_7_TO_0_8.md`
+
+### Changed
+
+- Package version advances to 0.8 (External Orchestration)
+- Plugin packages (polars/pandas/sql/pyspark/airflow) require
+  `etlantic>=0.8.0,<0.9`
+
+### Upgrade notes
+
+- Install `etlantic-airflow` to enable Airflow DAG compilation
+- Core remains free of Airflow, PySpark, and SQL driver dependencies
+- Unsupported orchestrator semantics fail compilation visibly (`PMORCH3xx`)
+
+## [0.7.0] - 2026-07-17
+
+### Added
+
+- Versioned Spark execution protocol (`etlantic.spark/1`) with dataset refs,
+  lazy region compile/execute, write intents, and UDF policy
+- `Profile.spark_engine`, `spark_udf_policy`, `spark_streaming`, and
+  `required_spark_capabilities`
+- Independently installable `etlantic-pyspark` plugin with local Spark provider
+- Lazy Spark region fusion preserving logical step identities
+- Contract ↔ Spark schema mapping that diagnoses lossy/unknown mappings
+- Delta-compatible append/overwrite/merge write intents (fail-closed)
+- Structured Streaming foundation types marked **experimental**
+- Entry-point groups `etlantic.spark_plugins` and `etlantic.spark_providers`
+- Migration guide `docs/11_DEVELOPMENT/MIGRATION_0_6_TO_0_7.md`
+
+### Changed
+
+- Package version advances to 0.7 (Distributed Spark Execution)
+- Planner prefers `spark_engine` over `sql_engine` / `dataframe_engine` when set
+- Plugin packages (polars/pandas/sql/pyspark) require `etlantic>=0.7.0,<0.8`
+
+### Upgrade notes
+
+- Install `etlantic-pyspark` to enable Spark execution
+- Core remains free of PySpark and Delta dependencies
+- Streaming APIs are experimental; batch Spark is the production path
+
+## [0.6.1] - 2026-07-16
+
+### Changed
+
+- Renamed the project, Python package, CLI, plugins, documentation, and
+  distribution artifacts from Pipelantic to ETLantic
+- Updated package discovery, plugin entry points, schemas, environment
+  variables, examples, and repository URLs to use the `etlantic` namespace
+
+## [0.6.0] - 2026-07-16
+
+### Added
+
+- Versioned SQL execution protocol (`etlantic.sql/1`) with relation refs,
+  typed expressions, write intents, and dialect capability negotiation
+- `Profile.sql_engine` for selecting a SQL backend during planning
+- Independently installable `etlantic-sql` PostgreSQL reference plugin
+  (SQLAlchemy Core; SQLite usable for local demos)
+- SQL→SQL execution without intermediate Python row fetches when fusion is
+  preserved
+- Fail-closed merge and capability checks when a dialect cannot honor required
+  semantics
+- Entry-point discovery group `etlantic.sql_plugins`
+- Conformance helpers under `etlantic.testing` for SQL expressions
+
+### Changed
+
+- Package version advances to 0.6 (SQL-Native Execution)
+- Planning contexts for `sql` auto-require SQL capabilities when
+  `Profile.sql_engine` is set
+
+### Upgrade notes
+
+- Install `etlantic-sql` separately; core stays driver-free
+- Register SQL implementations with
+  `@TransformationClass.implementation("sql")`
+- Set `Profile(sql_engine="sql")` to select the SQL backend
+- Configure connections via `ETLANTIC_SQL_URL` (or plugin-specific wiring)
+- Missing plugins or unsupported capabilities (e.g. MERGE without keys) fail
+  at validation/planning
+
+## [0.5.0] - 2026-07-16
+
+### Added
+
+- Versioned dataframe execution protocol (`etlantic.dataframe/1`) with
+  materialize → invoke → normalize → validate → metrics → cleanup phases
+- Expanded capability vocabulary: eager, lazy, Arrow import/export, zero-copy,
+  schema inspection, invalid-row separation, cancellation, thread-safety
+- Planner recording of engine, plugin version, capabilities, collection points,
+  conversion boundaries, ownership, and validation policy on `PipelinePlan`
+- Runtime delegation of Polars/Pandas steps through the dataframe protocol
+  without reselecting engines
+- Independently installable `etlantic-polars` (eager + LazyFrame preservation)
+  and `etlantic-pandas` (eager, CoW/ownership isolation)
+- Optional Arrow interchange helpers (PyArrow imported only when available)
+- Entry-point discovery group `etlantic.dataframe_plugins`
+- Conformance helpers in `etlantic.testing`
+- uv workspace packaging for core + dataframe plugins
+
+### Changed
+
+- Package version advances to 0.5 (Dataframe Execution)
+- Built-in `local` registry plugin is a runtime/records path (`dataframe=False`),
+  not a dataframe engine
+- Planning contexts for `polars`/`pandas` auto-require dataframe capabilities
+
+### Upgrade notes
+
+- Install `etlantic-polars` or `etlantic-pandas` separately; core stays
+  engine-free
+- Register dataframe implementations with
+  `@TransformationClass.implementation("polars")` or `"pandas"`
+- Set `Profile.dataframe_engine` to `"polars"` or `"pandas"` to select a backend
+- Missing plugins or unsupported capabilities (e.g. Pandas + lazy) fail at
+  validation/planning
+
+### Fixed
+
+- Default `Pipeline.run` / CLI planning now uses plugins discovered on
+  `PipelineRuntime` (and entry-point discovery for plan-only contexts)
+- Fan-out / unconsumed / cross-engine ports map to in-memory strategies instead
+  of durable records; durable conversion only follows durable strategies
+- Per-port collect and validation; quarantine/reject populate invalid artifacts
+  and metrics
+- Schema observation uses plugin `inspect_schema` for native frames
+- Durable `ArtifactStore` refuses native frames/LazyFrames (fail closed)
+- Discovery load failures emit warnings; LazyFrame dtype checks without row
+  collect; invoke kwargs prefer inputs over colliding parameters
+- Core `from_arrow_table` no longer imports Polars/Pandas
+
+## [0.4.0] - 2026-07-16
+
+### Upgrade notes
+
+- Register local executable transformations with
+  `@TransformationClass.implementation("local")`.
+- Use the `development` profile for the built-in local runtime examples.
+- Pandas, Polars, SQL, Spark, and Airflow plugins are not part of this release.
+
+### Known limitations
+
+- The local report store is process-scoped.
+- The package remains alpha and 0.x releases may contain breaking changes.
+
+### Added
+
+- Local async-first runtime that executes `PipelinePlan`s in-process
+- `Pipeline.run` / `Pipeline.arun` / `Pipeline.debug` entry points
+- `RunIntent`, `RunSelection`, `RunRequest`, and materialization/retry/timeout
+  policies
+- `PipelineRuntime` with lifespan, middleware, resource injection, callbacks,
+  and outbound event types
+- Runtime-only `SecretValue` with env and mounted-file `SecretProvider`s,
+  bounded cache, and fail-closed resolution (planning remains secret-free)
+- Storage bindings: `memory`, `callable`, `json`, `csv`, and `null` (no-write)
+- Versioned `PipelineRunReport` with text/JSON/HTML renderers and in-process
+  report store
+- Structured logging with central secret redaction and lifecycle/security events
+- `SchemaDriftPolicy` observation hooks and local reliability helpers
+  (freshness, partition completeness, retry-safety, backfill/repair/no-write)
+- CLI: `etlantic run` and `etlantic report show|export`
+- Hard dependency on `anyio` for structured concurrency
+
+### Fixed
+
+- Durable materialization no longer always writes workspace files; strategy
+  comes from the plan/request
+- Missing transformation implementations fail closed (no silent identity
+  fallback or engine swap)
+- Cancellation uses `anyio` cancelled-exc handling and returns a partial report
+- Exception and diagnostic messages are redacted before entering reports/logs
+- Binding `SecretRef` values are passed into storage context; unresolved
+  secrets still fail closed
+- Plan `retry_max_attempts` / `timeout_seconds` merge into default `RunRequest`
+  policies
+- Planner default binding provider is `memory` (`local`/`python` remain aliases)
+- Binding overrides resolve registry descriptors for provider/location/secret
+- Retry honors backoff/`retry_on` and retry-safety declarations; `CONTINUE`
+  soft-skips without failing the step status as hard failure
+- Schema observations fingerprint observed data (not only the contract class)
+- CLI `run` prints embedded reports on `PipelineExecutionError`
+- Debug session invalidation clears shared artifact/memory state across reruns
+- Resource `Inject` wiring, outbound `Emit` capture, report lineage, and
+  run/region lifespan helpers
+
+### Changed
+
+- Package version and public status advance to 0.4 (Local Runtime and
+  Operational Model)
+- `SecretRef` lives under the `etlantic.secrets` package alongside runtime
+  secret resolution types
+
+## [0.3.0] - 2026-07-16
+
+### Added
+
+- `Data` as the preferred thin facade over ContractModel (`DataContractModel`
+  remains as a deprecated alias)
+- Multi-phase validation (structural, reference, semantic, policy, capability)
+  with diagnostic actions and source symbols
+- `Profile` templates, `SecretRef`, scoped registries, and capability negotiation
+- Immutable secret-free `PipelinePlan` IR (`etlantic.plan/1`) with slicing,
+  explain output, and canonical fingerprints
+- Schema-drift models (`NormalizedSchema`, observations, changes, `DriftImpact`)
+- Portable reliability/intent models (freshness, write/materialization intents,
+  idempotency, evidence schemas)
+- JSON Schemas for profiles, project config, and PipelinePlan
+- `etlantic` CLI: `validate`, `inspect`, `plan`, and `plan explain`
+  (plus `plan --explain` alias, `--nodes`, and mutual exclusion for
+  `--run-one` / `--run-until`)
+- `Pipeline.plan()` / `Pipeline.explain_plan()`
+
+### Fixed
+
+- `run_until` selects declaration-order prefix (not only upstream closure)
+- Unknown plan selections fail closed with `PMPLAN501`
+- Multi-engine regions and cross-engine durable artifact strategies
+- Profile snapshot included in plan fingerprint
+- Strict policy requires registered transformation implementations
+- Subpipeline validation inherits parent planning context
+- Plan JSON round-trip restores `secret_ref` and verifies fingerprints
+- Schema-only DTCS ports map `datetime`/`decimal`/`binary` correctly
+- Diff APIs return diagnostics for malformed toolkit input
+
+### Changed
+
+- Package version and public status advance to 0.3 (Validation and Pipeline Plan IR)
+- Explicit `click` dependency for the CLI plan command group (Typer 0.15+ no
+  longer pulls it in transitively)
+
+## [0.2.0] - 2026-07-16
+
+### Added
+
+- Contract interoperability for ODCS (via ContractModel), DTCS, and DPCS
+- `Transformation.to_dtcs` / `from_dtcs` and `Pipeline.to_dpcs` / `from_dpcs`
+- Deterministic `ContractBundle` generation via `generate_contracts` /
+  `write_contracts` / `load_bundle`
+- ODCS facades `load_data_contract` and `write_odcs`
+- Diff hooks: `diff_data_contracts`, `diff_transformations`, `diff_pipelines`
+- Supported-version policy, bounded safe loaders, and source-aware diagnostics
+- Dependencies on the published `dtcs` and `dpcs` toolkits
+
+### Fixed
+
+- Map ContractModel `number`/`datetime` fields to DTCS `decimal`/`datetime`
+- Emit transformation parameters as `etlantic:parameters` (toolkit-valid)
+- Round-trip DPCS step parameter overrides
+- Treat matching published ODCS ids as compatible in `PMPIPE210` checks
+- Recursively validate nested subpipelines
+- Clear stale cyclic graph-build errors on fresh `build_graph()`
+- Fail closed on DPCS diff incompatible categories and bad DTCS parse input
+- Detect published-id / filename slug collisions during bundle generation
+- Resolve `odcs:`-prefixed contract registry keys in `from_dtcs`
+- Expose `ValidationReport.has_errors` for the documented validation UX
+- Correct getting-started imports to use `etlantic.DataContractModel`
+  (ContractModel does not export `DataContractModel`)
+
+## [0.1.0] - 2026-07-16
+
+### Added
+
+- First public release as **ETLantic** (PyPI package `etlantic`)
+- Typed modeling kernel for authoring pipelines without an execution backend
+- `Transformation`, `Input`, `Output`, and `Parameter` port annotations
+- `Pipeline`, `Source`, `Step`, `Sink`, and subpipeline composition
+- Typed `OutputRef` wiring with stable node and port identities
+- Structural validation diagnostics (cycles, missing refs, incompatible ports)
+- Logical graph inspection and Mermaid diagram generation
+- ContractModel integration boundary via `DataContractModel` alias
+- uv + ruff toolchain, MkDocs documentation site, shared GitHub Actions
+  checks, and tag-triggered PyPI release
+
+[0.10.0]: https://github.com/eddiethedean/etlantic/releases/tag/v0.10.0
+[0.9.0]: https://github.com/eddiethedean/etlantic/releases/tag/v0.9.0
+[0.8.0]: https://github.com/eddiethedean/etlantic/releases/tag/v0.8.0
+[0.7.0]: https://github.com/eddiethedean/etlantic/releases/tag/v0.7.0
+[0.6.1]: https://github.com/eddiethedean/etlantic/releases/tag/v0.6.1
+[0.6.0]: https://github.com/eddiethedean/etlantic/releases/tag/v0.6.0
+[0.5.0]: https://github.com/eddiethedean/etlantic/releases/tag/v0.5.0
+[0.4.0]: https://github.com/eddiethedean/etlantic/releases/tag/v0.4.0
+[0.3.0]: https://github.com/eddiethedean/etlantic/releases/tag/v0.3.0
+[0.2.0]: https://github.com/eddiethedean/etlantic/releases/tag/v0.2.0
+[0.1.0]: https://github.com/eddiethedean/etlantic/releases/tag/v0.1.0

etlantic-0.10.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Odo Matthews
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

etlantic-0.10.0/PKG-INFO

@@ -0,0 +1,235 @@
+Metadata-Version: 2.4
+Name: etlantic
+Version: 0.10.0
+Summary: Typed, contract-driven data pipeline modeling for Python.
+Project-URL: Homepage, https://github.com/eddiethedean/etlantic
+Project-URL: Documentation, https://github.com/eddiethedean/etlantic/tree/main/docs
+Project-URL: Repository, https://github.com/eddiethedean/etlantic
+Project-URL: Issues, https://github.com/eddiethedean/etlantic/issues
+Project-URL: Changelog, https://github.com/eddiethedean/etlantic/blob/main/CHANGELOG.md
+Author-email: Odo Matthews <odosmatthews@gmail.com>
+License-Expression: MIT
+License-File: LICENSE
+Keywords: data-contracts,etl,pipelines,pydantic,typed
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Typing :: Typed
+Requires-Python: >=3.11
+Requires-Dist: anyio<5,>=4
+Requires-Dist: click<9,>=8
+Requires-Dist: contractmodel>=0.1.2
+Requires-Dist: dpcs<1,>=0.13
+Requires-Dist: dtcs<1,>=0.11
+Requires-Dist: packaging>=24
+Requires-Dist: pydantic<3,>=2.12
+Requires-Dist: typer<1,>=0.15
+Provides-Extra: airflow
+Requires-Dist: etlantic-airflow==0.10.0; extra == 'airflow'
+Provides-Extra: arrow
+Requires-Dist: pyarrow>=14; extra == 'arrow'
+Provides-Extra: dataframes
+Requires-Dist: etlantic-pandas==0.10.0; extra == 'dataframes'
+Requires-Dist: etlantic-polars==0.10.0; extra == 'dataframes'
+Provides-Extra: keyring
+Requires-Dist: etlantic-keyring==0.10.0; extra == 'keyring'
+Provides-Extra: observability
+Requires-Dist: opentelemetry-api<2,>=1.36; extra == 'observability'
+Provides-Extra: otel
+Requires-Dist: opentelemetry-api<2,>=1.36; extra == 'otel'
+Provides-Extra: pandas
+Requires-Dist: etlantic-pandas==0.10.0; extra == 'pandas'
+Provides-Extra: polars
+Requires-Dist: etlantic-polars==0.10.0; extra == 'polars'
+Provides-Extra: postgresql
+Requires-Dist: etlantic-sql==0.10.0; extra == 'postgresql'
+Provides-Extra: pyspark
+Requires-Dist: etlantic-pyspark==0.10.0; extra == 'pyspark'
+Provides-Extra: spark
+Requires-Dist: etlantic-pyspark==0.10.0; extra == 'spark'
+Provides-Extra: sparkforge
+Requires-Dist: etlantic-sparkforge==0.10.0; extra == 'sparkforge'
+Provides-Extra: sql
+Requires-Dist: etlantic-sql==0.10.0; extra == 'sql'
+Provides-Extra: sqlmodel
+Requires-Dist: etlantic-sqlmodel==0.10.0; extra == 'sqlmodel'
+Description-Content-Type: text/markdown
+
+![ETLantic banner](https://raw.githubusercontent.com/eddiethedean/etlantic/main/docs/theme/assets/etlantic-banner.png)
+
+# ETLantic
+
+Catch incompatible data-pipeline wiring before you process data.
+
+Define datasets, transformations, and pipelines as typed Python classes.
+Validate and plan them once. Run locally today; swap Polars, Pandas, or SQL
+backends without rewriting the logical pipeline.
+
+**Status:** Alpha **0.10.0** — local runtime + optional
+Polars/Pandas/SQL/PySpark/Airflow plugins. Structured Streaming is
+experimental.
+
+> **Install note:** Until a matching `v0.10.0` tag is published to PyPI, prefer
+> installing from source (below). Anonymous `pip install etlantic` may fail if
+> wheels are not yet uploaded.
+
+## Install (from source — works today)
+
+Requires Python 3.11+ and [uv](https://docs.astral.sh/uv/).
+
+```bash
+git clone https://github.com/eddiethedean/etlantic.git
+cd etlantic
+uv sync
+uv run python -c "import etlantic; print(etlantic.__version__)"
+uv run python examples/quickstart.py
+```
+
+### Optional when wheels are on PyPI
+
+```bash
+pip install etlantic
+# optional engines / compilers
+pip install etlantic-polars etlantic-pandas etlantic-sql
+pip install etlantic-pyspark etlantic-airflow
+pip install etlantic-keyring etlantic-sqlmodel etlantic-sparkforge
+```
+
+## Quick example
+
+```python
+from etlantic import (
+    Data,
+    Input,
+    Output,
+    Pipeline,
+    PipelineRuntime,
+    Sink,
+    Source,
+    Transformation,
+)
+
+
+class RawCustomer(Data):
+    customer_id: int
+    first_name: str
+    last_name: str
+
+
+class Customer(Data):
+    customer_id: int
+    full_name: str
+
+
+class NormalizeCustomers(Transformation):
+    customers: Input[RawCustomer]
+    result: Output[Customer]
+
+
+class CustomerPipeline(Pipeline):
+    raw: Source[RawCustomer] = Source(binding="customer_source")
+    normalized = NormalizeCustomers.step(customers=raw)
+    curated: Sink[Customer] = Sink(
+        input=normalized.result,
+        binding="customer_sink",
+    )
+
+
+@NormalizeCustomers.implementation("local")
+def normalize(customers: list[RawCustomer]) -> list[Customer]:
+    return [
+        Customer(
+            customer_id=row.customer_id,
+            full_name=f"{row.first_name} {row.last_name}",
+        )
+        for row in customers
+    ]
+
+
+CustomerPipeline.validate(profile="development").raise_for_errors()
+
+runtime = PipelineRuntime()
+runtime.memory.seed(
+    "customer_source",
+    [RawCustomer(customer_id=1, first_name="Ada", last_name="Lovelace")],
+)
+run_report = CustomerPipeline.run(profile="development", runtime=runtime)
+print(runtime.memory.get("customer_sink"))
+```
+
+Run the complete tested version at
+[examples/quickstart.py](examples/quickstart.py).
+
+## Current capability boundary
+
+| Capability | 0.10 |
+|---|---|
+| Typed modeling, validation, contracts, and planning | Available |
+| Local Python execution and run reports | Available |
+| Memory, callable, JSON, CSV, and no-write storage | Available |
+| Polars and Pandas dataframe plugins | Available (`etlantic-polars` / `etlantic-pandas`) |
+| SQL plugin | Available (`etlantic-sql`) |
+| PySpark plugin + local provider | Available (`etlantic-pyspark`) |
+| Structured Streaming | Experimental |
+| Airflow orchestrator compiler | Available (`etlantic-airflow`) |
+| CLI compile / generate / schema / SARIF | Available |
+| Plugin allowlists / keyring / SQLModel extras | Available |
+| SparkForge migration adapter | Available (`etlantic-sparkforge`) |
+
+**Next design line:** releases 0.11-0.15 are planned to add a PySpark-inspired
+portable transformation language, followed by Polars, PySpark, Pandas, and safe
+SQL compilers. This is documented future design, not part of the 0.10 API. See
+the [portable transformation design](docs/04_TRANSFORMATIONS/PORTABLE_TRANSFORMATIONS.md)
+and [roadmap](docs/11_DEVELOPMENT/ROADMAP.md).
+
+## Documentation
+
+- [Getting Started](docs/01_GETTING_STARTED/README.md) (start here)
+- [Capabilities and Limitations](docs/01_GETTING_STARTED/CAPABILITIES.md)
+- [Quickstart](docs/01_GETTING_STARTED/QUICKSTART.md)
+- [Evaluator brief](docs/01_GETTING_STARTED/EVALUATOR.md)
+- [Core Concepts](docs/02_FOUNDATIONS/CORE_CONCEPTS.md)
+- [Architecture](docs/02_FOUNDATIONS/ARCHITECTURE.md)
+- [Roadmap](docs/11_DEVELOPMENT/ROADMAP.md)
+
+Build the docs locally with `uv run mkdocs serve` (hosted site TBD).
+
+## Development
+
+Requires [uv](https://docs.astral.sh/uv/).
+
+```bash
+uv sync
+uv run pytest
+uv run ruff check .
+uv run ruff format .
+```
+
+`uv sync` creates `.venv`, installs the package in editable mode, and installs
+the `dev` dependency group (pytest, ruff, mkdocs) by default.
+
+## Release
+
+The first upload of each new package name counts against PyPI’s new-project
+rate limit (`429 Too many new projects created`). Release CI waits **10 minutes**
+between package publishes. See
+[Release Process](docs/11_DEVELOPMENT/RELEASE_PROCESS.md).
+
+Tag a version that matches `src/etlantic/_version.py` (and every
+`packages/*/pyproject.toml`), then push **only that tag**:
+
+```bash
+git tag -a v0.10.0 -m "ETLantic 0.10.0"
+git push origin v0.10.0
+```
+
+Do not use `git push --tags`. GitHub Actions runs checks and publishes to PyPI
+using the `PYPI_API_TOKEN` repository secret.
+
+## License
+
+MIT