proccompy 0.4.1__tar.gz

proccompy-0.4.1/.gitignore

@@ -0,0 +1,14 @@
+.venv/
+__pycache__/
+*.pyc
+*.egg-info/
+.pytest_cache/
+dist/
+build/
+.idea/
+.vscode/
+scratch/
+scratch/
+data/
+scratch/
+data/

proccompy-0.4.1/CHANGELOG.md

@@ -0,0 +1,129 @@
+# Changelog
+
+All notable changes to proccompy are documented here.
+
+The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.4.1] — 2026-05-25
+
+### Fixed
+- **Declared `pyarrow` as a required dependency.** It's needed at runtime
+  for polars↔DuckDB data exchange but was missing from `pyproject.toml`
+  in 0.4.0. Users installing from PyPI in a clean environment would hit
+  `ModuleNotFoundError: No module named 'pyarrow'` on the first
+  `compare()` call. Detected during TestPyPI launch verification.
+
+## [0.4.0] — 2026-05-25
+
+First public release on PyPI.
+
+### Added
+- PyPI metadata: keywords, trove classifiers, project URLs pointing to GitHub.
+
+### Notes
+- API and CLI surface are unchanged from 0.3.1; this is purely a packaging
+  release for PyPI launch.
+
+## [0.3.1] — 2026-05-25
+
+### Added
+- **Parquet directory input** in the CLI. Pass a folder of `part_*.parquet`
+  files (Spark/Athena-style output) as either `BASE` or `COMPARE`; polars
+  reads them as one logical dataset.
+- **Hive partitioning** is on by default for directory input — subdirectories
+  like `year=2024/month=01/` produce `year` and `month` columns usable as
+  ID columns. Disable with `--no-hive`.
+
+### Tests
+- 11 new tests covering simple multi-file directories, Hive-partitioned
+  directories, dir-vs-file comparisons, and schema-mismatch errors.
+
+## [0.3.0] — 2026-05-25
+
+### Added
+- **`proccompy` command-line interface** for shell-level use, with no
+  Python required.
+  - Reads parquet, CSV, and TSV files (format inferred from extension,
+    overridable with `--format`).
+  - Per-column tolerances via repeated `--tolerance "col=method:value"`
+    flags (`abs`, `pct`, `exact`).
+  - Multi-column ID specification via comma-separated `--id "col1,col2"`.
+  - Structured output flags: `--lst`, `--text`, `--summary-csv`,
+    `--diff-parquet`.
+  - Exit code **0** if datasets match (within tolerances), **1** if they
+    differ — usable as a CI/cron validation gate.
+- `click >= 8.0` dependency for the CLI.
+
+### Tests
+- 36 new CLI tests using `click.testing.CliRunner`.
+
+## [0.2.1] — 2026-05-24
+
+### Added
+- **Per-column difference statistics** on `CompareResult.summary()`:
+  - `mean_diff`, `std_diff`, `min_diff` (signed)
+  - `n_base_greater`, `n_compare_greater` (directional bias counts)
+- New `.lst` section: "Statistics for Numeric Differences" rendered
+  beneath the variables-with-unequal-values table.
+- Statistics are computed over **matched non-null observation pairs**,
+  matching SAS PROC COMPARE `STATS` semantics — independent of whether
+  tolerance accepted the diff.
+
+### Tests
+- 15 new tests covering bias direction, mixed-sign drift, single-row
+  edge cases, and stats-when-within-tolerance.
+
+## [0.2.0] — 2026-05-23
+
+### Added
+- **`CompareResult.to_lst()`** — SAS-style `.lst` report writer:
+  - 132-column fixed width, ASCII only
+  - Form-feed page breaks between major sections
+  - Page headers with procedure name, datetime, page number
+  - Per-observation comparison detail (Base / Compare / Diff / % Diff)
+  - Capped at `n_sample` observations (default 50)
+- **`CompareResult.to_text()`** — write the terminal-style report to a
+  UTF-8 text file.
+
+### Tests
+- 20 new tests for `.lst` rendering, ASCII compliance, page-break
+  conventions, and edge cases.
+
+## [0.1.1] — 2026-05-22
+
+Initial release.
+
+### Added
+- **DuckDB-based comparison engine.** `FULL OUTER JOIN` keyed on
+  `id_columns` with per-column unequal expressions. Streams on disk,
+  not memory-bound.
+- **Per-column tolerances**: `T.exact()`, `T.absolute(value)`,
+  `T.percent(value)`.
+- **Null-safe join** via SQL `IS NOT DISTINCT FROM`. No sentinel-string
+  hacks.
+- **Strict duplicate-key detection** — raises `ValueError` with counts
+  per side.
+- **Pandas, Polars, and DuckDB relation input**.
+- `CompareResult` accessor methods:
+  - `summary()` — per-column polars DataFrame
+  - `unequal_rows()` / `base_only_rows()` / `compare_only_rows()`
+  - `diff_dataset()` — SAS `OUT=` equivalent with `_TYPE_` ∈ {BASE,
+    COMPARE, DIF, PERCENT} and `_OBS_`
+  - `joined()` — full joined frame for custom analysis
+  - `assert_matches()` — raises `AssertionError` with structured
+    failure detail, for pytest / CI use
+  - `report()` — terminal-style structured report
+
+### Tests
+- 21 tests covering identical frames, value diffs, tolerances, nulls,
+  row presence, duplicate-key detection, dtype mismatches, multi-column
+  IDs, pandas input, OUT= dataset, and report rendering.
+
+[0.4.1]: https://github.com/kulbshar/proccompy/releases/tag/v0.4.1
+[0.4.0]: https://github.com/kulbshar/proccompy/releases/tag/v0.4.0
+[0.3.1]: https://github.com/kulbshar/proccompy/releases/tag/v0.3.1
+[0.3.0]: https://github.com/kulbshar/proccompy/releases/tag/v0.3.0
+[0.2.1]: https://github.com/kulbshar/proccompy/releases/tag/v0.2.1
+[0.2.0]: https://github.com/kulbshar/proccompy/releases/tag/v0.2.0
+[0.1.1]: https://github.com/kulbshar/proccompy/releases/tag/v0.1.1

proccompy-0.4.1/LICENSE

@@ -0,0 +1,202 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright [yyyy] [name of copyright owner]
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

proccompy-0.4.1/PKG-INFO

@@ -0,0 +1,284 @@
+Metadata-Version: 2.4
+Name: proccompy
+Version: 0.4.1
+Summary: Structured comparison reports for tabular data in Python — modeled on SAS PROC COMPARE.
+Project-URL: Homepage, https://github.com/kulbshar/proccompy
+Project-URL: Repository, https://github.com/kulbshar/proccompy
+Project-URL: Issues, https://github.com/kulbshar/proccompy/issues
+Project-URL: Changelog, https://github.com/kulbshar/proccompy/blob/main/CHANGELOG.md
+Project-URL: Documentation, https://github.com/kulbshar/proccompy#readme
+Author-email: Kulbhushan Sharma <sharma.kul.1982@gmail.com>
+License: Apache-2.0
+License-File: LICENSE
+Keywords: comparison,data-engineering,dataframe,duckdb,etl,pandas,polars,proc-compare,sas,validation
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Information Technology
+Classifier: Intended Audience :: Science/Research
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Information Analysis
+Classifier: Topic :: Software Development :: Quality Assurance
+Classifier: Topic :: Software Development :: Testing
+Classifier: Topic :: Utilities
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Requires-Dist: click>=8.0
+Requires-Dist: duckdb>=0.10.0
+Requires-Dist: polars>=1.0.0
+Requires-Dist: pyarrow>=10.0.0
+Provides-Extra: pandas
+Requires-Dist: pandas>=2.0.0; extra == 'pandas'
+Provides-Extra: test
+Requires-Dist: pandas>=2.0.0; extra == 'test'
+Requires-Dist: pytest>=7.0; extra == 'test'
+Description-Content-Type: text/markdown
+
+# proccompy
+
+**Structured comparison reports for tabular data in Python.**
+
+If you've ever needed to prove that two DataFrames hold the same data — after a pipeline rewrite, between source and ETL output, across a database migration, or in a daily regression check — you've probably found that `pandas.DataFrame.compare()` shows you cell-level diffs, `assert_frame_equal` gives you pass/fail, and neither tells you the thing you actually want to know: **where do they differ, by how much, in which direction, and does it matter?**
+
+proccompy produces a structured diagnostic report that answers those questions in the order you'd ask them. It's modeled on SAS PROC COMPARE — a 40-year-old procedure that data analysts trust because it makes "what changed?" a 30-second scan instead of an afternoon of investigation.
+
+```python
+import polars as pl
+import proccompy as pc
+from proccompy import T
+
+base    = pl.read_parquet("production.parquet")
+rewrite = pl.read_parquet("new_implementation.parquet")
+
+result = pc.compare(
+    base, rewrite,
+    id_columns="account_id",
+    tolerances={
+        "dollar_amount": T.absolute(0.01),   # rounding-level diffs are OK
+        "rate":          T.percent(0.001),   # but >0.1% relative is not
+    },
+)
+
+result.to_lst("validation.lst")     # SAS-style listing for archival / review
+result.assert_matches()              # raises AssertionError if they don't — for CI
+```
+
+## Why this exists
+
+The Python ecosystem has many ways to *find* differences between two DataFrames. What it doesn't have is a structured report that helps you *interpret* them.
+
+- `pandas.DataFrame.compare()` returns a multi-index DataFrame of cell-level diffs. No summary, no row-presence breakdown, no statistics.
+- `pandas.testing.assert_frame_equal` raises on first difference. Tells you something is wrong, not what.
+- [`datacompy`](https://github.com/capitalone/datacompy) has a report, but it's a flat text dump optimized for a single screen. No per-column difference statistics, no SAS conventions, no `.lst` output, no streaming/out-of-core support.
+- Schema-validation tools (Great Expectations, Pandera) check one frame against rules. They don't compare two frames.
+
+The result: every team building a data pipeline ends up writing their own ad-hoc comparison script. That script's report is always a little worse than PROC COMPARE's because PROC COMPARE has had four decades of refinement.
+
+proccompy brings that report to Python natively — running on polars, pandas, or DuckDB; pluggable into pytest; producing `.lst` files SAS analysts already know how to read.
+
+## Who is this for
+
+**Data engineers doing pipeline validation.** You're rewriting an implementation, migrating a database, or proving that yesterday's output matches today's. You want tolerance per column (because floating-point drift isn't "wrong"), the differences summarized and broken down, and a structured object you can assert on in CI.
+
+**ETL teams comparing output to source.** You need a recurring check that your pipeline faithfully reproduces source data, with the diagnostic depth to tell rounding drift from real corruption.
+
+**SAS analysts who miss the PROC COMPARE listing.** You moved to Python (willingly or not) and want a comparison report your boss will accept — `.lst` format, with the same sections you've read for 20 years.
+
+**Testing frameworks for data pipelines.** You're building (or using) dbt tests, custom assertion harnesses, regression suites. proccompy gives you a clean programmatic `CompareResult` to embed: structured fields, pytest-friendly.
+
+## What the report tells you
+
+A real PROC COMPARE listing isn't arbitrary — it's a diagnostic protocol. Each section answers a question, in the order you'd ask it:
+
+1. **Dataset Summary** — are we comparing what I think we're comparing?
+2. **Variables Summary** — do the schemas match?
+3. **Observation Summary** — do the same keys exist on both sides?
+4. **Values Comparison Summary** — how broken is this, in one glance?
+5. **Variables with Unequal Values** — which columns are responsible?
+6. **Statistics for Numeric Differences** — bias direction, spread, magnitude
+7. **Comparison Results for Observations** — show me the actual numbers
+
+You can scan a 50-page `.lst` in 30 seconds and decide whether to investigate or close the ticket. That's the value.
+
+## Install
+
+```bash
+pip install proccompy
+```
+
+Requires Python 3.10+. Pulls in `duckdb` and `polars`. Add `pandas` only if you need it as input.
+
+## Usage
+
+### The basic comparison
+
+```python
+import polars as pl
+import proccompy as pc
+
+base    = pl.DataFrame({"id": [1, 2, 3], "x": [10.0, 20.0, 30.0]})
+compare = pl.DataFrame({"id": [1, 2, 3], "x": [10.0, 20.5, 30.0]})
+
+result = pc.compare(base, compare, id_columns="id")
+
+result.matches             # → False
+result.summary()           # → per-column polars DataFrame
+print(result.report())     # → terminal-style report
+```
+
+Inputs can be polars DataFrames, pandas DataFrames, or DuckDB relations.
+
+### Tolerances per column
+
+```python
+from proccompy import T
+
+result = pc.compare(
+    base, compare,
+    id_columns="account_id",
+    tolerances={
+        "dollar_amount": T.absolute(0.01),   # within 1 cent → equal
+        "rate":          T.percent(0.001),   # within 0.1% relative → equal
+        "category":      T.exact(),          # exact match required (default)
+    },
+)
+```
+
+Three tolerance types: `T.exact()`, `T.absolute(value)`, `T.percent(value)`. Per-column. Defaults to exact for columns you don't specify.
+
+### Multiple ID columns
+
+```python
+result = pc.compare(
+    base, compare,
+    id_columns=["sector", "naics_code", "year"],
+)
+```
+
+### SAS-style .lst output
+
+```python
+result.to_lst("validation.lst")
+```
+
+Produces a paginated, fixed-width, ASCII-only `.lst` file with form-feed page breaks and SAS-style headers. Looks like real PROC COMPARE output. Open it in any text editor; attach it to a ticket; share it with anyone who used to write SAS.
+
+### Assertion for CI
+
+```python
+def test_pipeline_rewrite_matches_production():
+    base = pl.read_parquet("expected.parquet")
+    actual = run_pipeline()
+    pc.compare(base, actual, id_columns="id").assert_matches()
+```
+
+Raises `AssertionError` with structured per-column failure detail. Drops into any pytest suite.
+
+### The structured result
+
+`CompareResult` exposes everything as DataFrames you can post-process:
+
+```python
+result.summary()             # per-column summary
+result.unequal_rows()        # both-side rows with at least one diff
+result.base_only_rows()      # rows only in base
+result.compare_only_rows()   # rows only in compare
+result.diff_dataset()        # SAS OUT= equivalent: BASE / COMPARE / DIF / PERCENT
+result.joined()              # full joined frame for custom analysis
+```
+
+### Command line
+
+proccompy installs a `proccompy` console script for shell-level use — no Python required:
+
+```bash
+# Simple: compare two parquet files, report to stdout, exit 0 or 1
+proccompy expected.parquet actual.parquet --id account_id
+
+# With tolerances and multiple ID columns
+proccompy old.csv new.csv --id "sector,year" \
+    --tolerance "amount=abs:0.01" \
+    --tolerance "rate=pct:0.001"
+
+# Pipe everything to disk for archival / CI artifacts
+proccompy a.parquet b.parquet --id id \
+    --lst report.lst \
+    --summary-csv summary.csv \
+    --diff-parquet diffs.parquet \
+    --quiet
+```
+
+Exit codes: **0** if the datasets match (within any specified tolerances), **1** if they differ in any compared value, row presence, or column overlap. Use as a CI gate:
+
+```bash
+proccompy expected.parquet actual.parquet --id id --quiet && deploy.sh
+```
+
+Supports parquet, CSV, and TSV input; format is inferred from extension or set with `--format`. Run `proccompy --help` for the full reference.
+
+### Per-column difference statistics
+
+For numeric columns, the summary includes diagnostic statistics computed over matched non-null pairs (SAS PROC COMPARE STATS semantics):
+
+```python
+summary = result.summary()
+# Columns: column, base_dtype, compare_dtype, n_compared, n_unequal,
+#          n_null_mismatch, max_abs_diff, mean_diff, std_diff, min_diff,
+#          n_base_greater, n_compare_greater
+```
+
+These let you distinguish three failure modes that `max_abs_diff` alone can't:
+
+| mean_diff | std_diff | n_base_gt | n_compare_gt | Diagnosis |
+|---|---|---|---|---|
+| `+2.0` | `0.0` | `0` | `195` | Constant positive offset (unit conversion?) |
+| `0.0` | `5.1` | `10` | `10` | No bias but high spread (real drift) |
+| `+0.1` | `0.0` | `0` | `195` | Small uniform drift, within tolerance |
+
+## Alternatives — when to use what
+
+| | Best for | Limitation |
+|---|---|---|
+| `pandas.DataFrame.compare()` | Quick cell-level diff in a notebook | No summary, no report, no tolerance |
+| `pandas.testing.assert_frame_equal` | Strict pass/fail in tests | Stops at first diff, no diagnostic depth |
+| [`datacompy`](https://github.com/capitalone/datacompy) | Flat text report for one-screen review | Memory-bound, global-only tolerance, no `.lst`, no per-column diff stats |
+| Great Expectations, Pandera | Schema/distribution validation on one frame | Don't compare two frames |
+| **proccompy** | Structured diagnostic report for pipeline validation | Out-of-core via DuckDB engine only (Spark engine not yet implemented) |
+
+If you need a one-line cell diff, use `pandas.compare()`. If you need a structured report you can attach to a PR, a Jira ticket, or a SAS analyst's email, use proccompy.
+
+## What's under the hood
+
+- **DuckDB engine.** The comparison is a `FULL OUTER JOIN` keyed on `id_columns` with per-column unequal expressions. DuckDB streams this on disk, so you're not memory-bound even for hundreds of millions of rows.
+- **Polars I/O.** Frames register zero-copy via Arrow; results come back as polars DataFrames.
+- **Null-safe.** Uses SQL `IS NOT DISTINCT FROM` semantics. No sentinel-string hacks.
+- **Strict duplicate-key detection.** If your ID columns aren't unique, you'll get a clear error with counts, not silent wrong answers.
+- **Apache 2.0 licensed.**
+
+## Roadmap
+
+- **v0.4** — BY groups (stratified comparison summaries)
+- **v0.5** — `result.to_html()` self-contained HTML report
+- **v1.0** — pytest plugin, full documentation site, PyPI release
+
+Stable: top-level `pc.compare()`, `T.exact/absolute/percent`, `CompareResult` accessor methods, `.lst` format, CLI flags and exit codes.
+
+Subject to change: `_joined` internal structure, exact `.lst` column widths.
+
+## Contributing
+
+Issues and PRs welcome. Run the test suite with `pytest tests/`. Currently 92 tests; please add one for any new behavior.
+
+## License
+
+Apache 2.0. See `LICENSE`.
+
+---
+
+Built by [Kulbhushan Sharma](https://github.com/kulbshar) because the PROC COMPARE report was the SAS feature I missed most after switching to Python.