compara 0.5.0__py3-none-win_amd64.whl

compara-0.5.0.dist-info/METADATA

@@ -0,0 +1,157 @@
+Metadata-Version: 2.1
+Name: compara
+Version: 0.5.0
+Summary: Compare two tables (CSV, NDJSON, Parquet) on a composite key: counts, per-column statistics and a self-contained HTML report.
+Home-page: https://github.com/andrey-usa/compara
+Project-URL: Source, https://github.com/andrey-usa/compara
+Project-URL: Changelog, https://github.com/andrey-usa/compara/blob/HEAD/CHANGELOG.md
+License: MIT
+Keywords: csv,parquet,diff,compare,data-quality
+Classifier: Programming Language :: Rust
+Classifier: Environment :: Console
+Classifier: License :: OSI Approved :: MIT License
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+
+# compara
+
+Compare two tables on a composite key and get the counts, the per-column
+statistics and a self-contained HTML report. Key columns, compared columns and
+normalisation rules are parameters, so one tool serves every recurring
+comparison: a nightly export against yesterday's, a CSV against the Parquet a
+warehouse emits, a migration's source against its target.
+
+- Reads **CSV, newline-delimited JSON and Parquet**, and the two sides need not
+  be in the same format.
+- Ten million rows × 20 columns in about a second on four cores
+  ([numbers below](#speed)).
+- One static binary; the report is a single HTML file with nothing to host.
+- Exit codes made for pipelines: `0` identical, `1` differences, `2` error,
+  `3` duplicate keys (with `--fail-on-dups`).
+
+## Install
+
+Prebuilt binaries for Linux (x86_64, aarch64), macOS (arm64, x86_64) and
+Windows (x64) are on the [releases page](https://github.com/andrey-usa/compara/releases),
+and the same binaries through package managers:
+
+| | |
+|---|---|
+| Cargo | `cargo install --locked compara` |
+| npm | `npm install -g compara` (or `npx compara …`) |
+| pip / uv | `pip install compara` · `uv tool install compara` |
+| Homebrew | `brew install andrey-usa/tap/compara` |
+| Scoop | `scoop bucket add andrey-usa https://github.com/andrey-usa/scoop-bucket` then `scoop install compara` |
+
+## Use
+
+```sh
+compara compare a.csv b.csv -k account_id,txn_id -i updated_at
+compara compare a.csv b.csv -k account_id,txn_id --trim --tolerance 0.005
+compara compare export.csv warehouse.parquet -k id --summary
+compara compare a.csv b.csv --profile orders --json summary.json --export-dir out/
+compara columns data.parquet          # column names (reads only the footer)
+compara head data.ndjson -n 5         # first rows, aligned
+```
+
+`compare` writes `<a>__vs__<b>.html` (or `--out PATH`): counts, per-column
+statistics, and the changed, added, removed and duplicate rows, each section
+capped at `--max-rows` (counts are always exact). [`examples/report.html`](examples/report.html)
+is one, made from the two files next to it:
+
+```sh
+compara compare examples/orders_2026-08.csv examples/orders_2026-09.csv -k order_id,line_no -i updated_at
+```
+
+| Option | Effect |
+|---|---|
+| `-k`, `--key` | composite key, required (or from a profile) |
+| `-c`, `--compare` | columns to diff; default every column present in both files except the key |
+| `-i`, `--ignore` | columns to skip (timestamps, run ids) |
+| `--trim`, `--ignore-case`, `--empty-is-null` | normalisation before comparing (applies to key and values) |
+| `--tolerance` | absolute numeric tolerance where both sides parse as numbers |
+| `--delimiter`, `--encoding` | override auto-detection (CSV only) |
+| `--threads` | how wide the default engine runs; the default is every core |
+| `--memory` | memory to plan for (e.g. `8G`); decides when large files are joined in one pass instead of two |
+| `--max-rows` | rows embedded per report section (default 50 000) |
+| `--export-dir` | full, uncapped changed/added/removed CSVs |
+| `--json` | also write a JSON summary (counts and column stats; schema in `docs/contract.schema.json`) |
+| `--summary` | print the counts and write nothing |
+| `--engine` | `auto` (default), `turbo`, `sortmerge` or `native` |
+| `--no-compress` | plain JSON payload in the report, for pre-2023 browsers |
+| `--fail-on-dups` | exit 3 when either file has duplicate keys |
+
+Duplicate keys are counted and listed per file; the first occurrence of each
+key takes part in the join.
+
+**Profiles.** A `compara.toml` in the working directory (or
+`~/.config/compara/compara.toml`, or `--config PATH`) names recurring
+comparisons: key, compared and ignored columns, normalisation, engine. See
+[`compara.example.toml`](compara.example.toml). The command line overrides the
+profile.
+
+## Speed
+
+Ten million rows × 20 columns, keyed on `(account_id, txn_id)`, `--ignore
+updated_at`, one GitHub Actions runner (AMD EPYC 7763, 4 vCPU / 16 GB), best of
+five ([run 36672430118](https://github.com/andrey-usa/data-comparison-polyglot/actions/runs/36672430118)).
+Each cell is wall time · CPU time · memory above the mapped input.
+
+| Format | Input a side | compara |
+|---|---:|---|
+| CSV | 3,509 MB | 1.20 s · 4.0 s · 591 MB |
+| ndjson | 8,487 MB | 2.79 s · 10.4 s · 592 MB |
+| Parquet | 2,074 MB | 1.41 s · 4.7 s · 1,342 MB |
+
+compara started as the Rust port in
+[data-comparison-polyglot](https://github.com/andrey-usa/data-comparison-polyglot),
+where the same comparison is implemented in C, C++, Rust and Zig and measured
+side by side; that repository has the full tables and how they are produced.
+
+## Engines
+
+Three backends, one result contract: every engine returns identical counts and
+column statistics for the same input, and the test suite asserts it cell for
+cell. `--engine auto` takes the first one that can load the input, in this
+order.
+
+| Engine | Implementation | Memory model | Use it for |
+|---|---|---|---|
+| `turbo` | mapped file, byte-level index; CSV, JSON and Parquet | off-heap bytes, index in memory | the default, and the fastest |
+| `sortmerge` | external sort, merge join | bounded memory, spills to disk | files past what memory can index |
+| `native` | over the `csv` crate | in-memory, row-oriented | the dependency-light baseline |
+
+All three read values as text, with no type inference, so `1.0` and `1` stay
+different unless a tolerance is set, and all three treat an empty field as
+absent whether or not it is quoted.
+
+### Input formats
+
+`turbo` decides what a file is by its content rather than its name: a Parquet
+magic number, a leading `{`, or a CSV header.
+
+| Format | How it is read |
+|---|---|
+| CSV | mapped, scanned eight bytes at a time; a field is an offset and a length |
+| newline-delimited JSON | the same, addressed by key rather than by column number; `\uXXXX` is decoded, so an escaped and a literal character compare equal |
+| Parquet | pages decoded into an arena; dictionary-encoded columns point at the dictionary entry, so a repeated value costs eight bytes a row |
+
+The Parquet reader is written here rather than taken from a crate, to keep that
+representation instead of converting to Arrow arrays. It reads PLAIN,
+dictionary (`PLAIN_DICTIONARY`, `RLE_DICTIONARY`), RLE booleans and the
+version-2 delta encodings; uncompressed, snappy, gzip, zstd and LZ4-raw pages;
+data pages v1 and v2; definition levels for optional columns. It refuses, by
+name, nested or repeated schemas, `BYTE_STREAM_SPLIT`, LZO and Brotli.
+
+A typed Parquet value is rendered to text by a fixed rule: byte arrays as their
+bytes, booleans `true`/`false`, integers and decimals in full, floats in the
+shortest round-trip form, dates `YYYY-MM-DD`, timestamps ISO 8601.
+`tests/fixtures/formats/typed.csv` is that rule written down.
+
+## Contributing
+
+See [CONTRIBUTING.md](CONTRIBUTING.md).
+
+## License
+
+[MIT](LICENSE)

compara-0.5.0.dist-info/RECORD

@@ -0,0 +1,5 @@
+compara-0.5.0.data/scripts/compara.exe,sha256=AzTd77MJI-Oq6yOrTeiIHejXFVGu4JgG5V2MOMX0_Gk,1940480
+compara-0.5.0.dist-info/METADATA,sha256=JUdUX5ffJ5LBqBdse3vQAWjw68WR05lIAOEfzXKA8LI,7499
+compara-0.5.0.dist-info/WHEEL,sha256=g1aQ65onkFiWyh7QjiXnWXLnCV9xiZTddNDcljDxYOM,88
+compara-0.5.0.dist-info/licenses/LICENSE,sha256=RSeVx-hjXtE643Nsk8gnOcomD9Wdf4PAXy7J7fSi4pQ,1067
+compara-0.5.0.dist-info/RECORD,,

compara-0.5.0.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: package.py
+Root-Is-Purelib: false
+Tag: py3-none-win_amd64

compara-0.5.0.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 andrey-usa
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.