sas2parquet 0.1.9__tar.gz → 0.2.0__tar.gz

{sas2parquet-0.1.9 → sas2parquet-0.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: sas2parquet
-Version: 0.1.9
+Version: 0.2.0
 Summary: SAS → Parquet Hybrid Converter & Validator
 License: MIT
 License-File: LICENSE
@@ -36,102 +36,119 @@ Description-Content-Type: text/markdown
 [![Python versions](https://img.shields.io/pypi/pyversions/sas2parquet.svg)](https://pypi.org/project/sas2parquet/)
 [![License](https://img.shields.io/pypi/l/sas2parquet.svg)](LICENCE)
 
-**The ultimate SAS (.sas7bdat) to Parquet converter** - handles problematic files that fail with standard tools. Automatic encoding detection, intelligent type inference, schema repair, and pixel-perfect validation.
+**The ultimate SAS (.sas7bdat) to Parquet converter** — built to handle files that fail with standard tools.
+
+`sas2parquet` automatically detects encodings, repairs schemas, infers correct data types, and performs **pixel-perfect validation** between SAS and Parquet outputs.
+
+---
 
 ## ✨ Features
 
 | Feature | Description |
-|---------|-------------|
-| 🔄 **Auto Encoding** | Detects UTF-8, Latin1, CP1252 from metadata/fallback |
-| 🧠 **Smart Types** | Infers datetime, numeric, string with 20+ fallback attempts |
-| ✅ **Validation** | Compares SAS vs Parquet chunk-by-chunk (numeric + string) |
+|-------|-------------|
+| 🔄 **Auto Encoding** | Detects UTF-8, Latin1, CP1252 from metadata or fallback |
+| 🧠 **Smart Types** | Infers datetime, numeric, string with 20+ retry strategies |
+| ✅ **Validation** | Chunk-by-chunk comparison (metadata, counts, values) |
 | 📊 **Memory Safe** | Chunked processing (96GB RAM optimized, configurable) |
-| 💾 **ZSTD** | Level 6 compression for maximum efficiency |
-| 📝 **Detailed Logs** | Mismatch reports + full conversion trace |
-| 🎯 **Two Modes** | Single file OR recursive directory processing |
+| 💾 **ZSTD Compression** | Level-6 ZSTD for efficient Parquet storage |
+| 📝 **Detailed Logs** | Full conversion trace + mismatch reports |
+| 🎯 **Two Modes** | Single file or recursive directory processing |
+
+---
 
-## Quick Start
+## 🚀 Quick Start
 
 ### Install
 ```bash
 pip install sas2parquet
 ```
 
-### Single File
+---
+
+## ✅ Usage
+
+### Convert a directory (recommended)
+
 ```bash
-sas2parquet input.sas output.parquet
+sas2parquet path/to/sasdata/
 ```
 
-### Batch Directory (Recommended)
+- Converts **all `.sas7bdat` files recursively**
+- Creates `parquetdata/` and `logging/` next to `sasdata/`
+
+---
+
+### Convert a single file
+
 ```bash
-sas2parquet --dir-mode
+sas2parquet path/to/file.sas7bdat
 ```
 
-## 📁 Directory Mode (Default Workflow)
+Output (default):
+```text
+path/to/file.parquet
+```
+
+---
+
+### Specify output location
+
+#### Directory mode — custom output directory
+```bash
+sas2parquet path/to/sasdata/ --out path/to/parquetdata/
+```
+
+#### File mode — custom output file
+```bash
+sas2parquet path/to/file.sas7bdat --out path/to/output.parquet
+```
 
-### How it works
+---
 
-- You provide a `sasdata/` directory containing all `.sas7bdat` files (including nested subfolders).
-- The tool automatically creates a `parquetdata/` directory in the same parent folder as `sasdata/`.
-- All files are converted to Parquet and written into `parquetdata/`, mirroring the original folder structure.
+### Custom log directory (directory mode)
+
+```bash
+sas2parquet path/to/sasdata/ --log-dir path/to/logs/
+```
+
+---
+
+## 📁 Directory Mode Behavior
 
 ```text
 your-project/
-├── sasdata/              # ← Put your .sas7bdat files here
+├── sasdata/
 │   ├── file1.sas7bdat
 │   └── subfolder/
 │       └── nested.sas7bdat
-├── parquetdata/          # ← AUTO-CREATED (mirrors sasdata/)
+├── parquetdata/
 │   ├── file1.parquet
-│   └── subfolder/
+│   └── subfolder_parquet/
 │       └── nested.parquet
-└── logging/             # ← AUTO-CREATED (detailed logs)
+└── logging/
     └── conversion_20260205_1145.log
 ```
 
-Just run:
-```bash
-sas2parquet --dir-mode
-```
+---
 
 ## 🛠️ CLI Reference
+
 ```bash
 sas2parquet --help
 ```
 
-```text
-usage: sas2parquet [-h] [--dir-mode] [sas_file] [parquet_file]
-
-Robust SAS to Parquet converter with validation
-```
-
-## 📊 Example Output
-```text
-🚀 SAS → Parquet Hybrid Fix & Validate (full folder)
-Found 3 files.
-============================================================
-...
-```
+---
 
 ## ⚙️ Configuration (Advanced)
 
-Edit `src/sas2parquet/convert.py` constants:
+Edit constants in:
 
-```python
-AVAILABLE_RAM_GB = 96
-RAM_USAGE_FACTOR = 0.5
-ZSTD_COMPRESSION_LEVEL = 6
-MIN_CHUNK_SIZE = 100_000
-MAX_CHUNK_SIZE = 10_000_000
+```text
+src/sas2parquet/convert.py
 ```
 
-## 🧪 Validation Details
-Each file undergoes 4-stage validation:
-1. Metadata
-2. Exact counts
-3. Column order
-4. Value comparison
+---
 
 ## 📄 License
-MIT License
 
+MIT License

sas2parquet-0.2.0/README.md

@@ -0,0 +1,122 @@
+# sas2parquet
+
+[![PyPI version](https://badge.fury.io/py/sas2parquet.svg)](https://pypi.org/project/sas2parquet/)
+[![Python versions](https://img.shields.io/pypi/pyversions/sas2parquet.svg)](https://pypi.org/project/sas2parquet/)
+[![License](https://img.shields.io/pypi/l/sas2parquet.svg)](LICENCE)
+
+**The ultimate SAS (.sas7bdat) to Parquet converter** — built to handle files that fail with standard tools.
+
+`sas2parquet` automatically detects encodings, repairs schemas, infers correct data types, and performs **pixel-perfect validation** between SAS and Parquet outputs.
+
+---
+
+## ✨ Features
+
+| Feature | Description |
+|-------|-------------|
+| 🔄 **Auto Encoding** | Detects UTF-8, Latin1, CP1252 from metadata or fallback |
+| 🧠 **Smart Types** | Infers datetime, numeric, string with 20+ retry strategies |
+| ✅ **Validation** | Chunk-by-chunk comparison (metadata, counts, values) |
+| 📊 **Memory Safe** | Chunked processing (96GB RAM optimized, configurable) |
+| 💾 **ZSTD Compression** | Level-6 ZSTD for efficient Parquet storage |
+| 📝 **Detailed Logs** | Full conversion trace + mismatch reports |
+| 🎯 **Two Modes** | Single file or recursive directory processing |
+
+---
+
+## 🚀 Quick Start
+
+### Install
+```bash
+pip install sas2parquet
+```
+
+---
+
+## ✅ Usage
+
+### Convert a directory (recommended)
+
+```bash
+sas2parquet path/to/sasdata/
+```
+
+- Converts **all `.sas7bdat` files recursively**
+- Creates `parquetdata/` and `logging/` next to `sasdata/`
+
+---
+
+### Convert a single file
+
+```bash
+sas2parquet path/to/file.sas7bdat
+```
+
+Output (default):
+```text
+path/to/file.parquet
+```
+
+---
+
+### Specify output location
+
+#### Directory mode — custom output directory
+```bash
+sas2parquet path/to/sasdata/ --out path/to/parquetdata/
+```
+
+#### File mode — custom output file
+```bash
+sas2parquet path/to/file.sas7bdat --out path/to/output.parquet
+```
+
+---
+
+### Custom log directory (directory mode)
+
+```bash
+sas2parquet path/to/sasdata/ --log-dir path/to/logs/
+```
+
+---
+
+## 📁 Directory Mode Behavior
+
+```text
+your-project/
+├── sasdata/
+│   ├── file1.sas7bdat
+│   └── subfolder/
+│       └── nested.sas7bdat
+├── parquetdata/
+│   ├── file1.parquet
+│   └── subfolder_parquet/
+│       └── nested.parquet
+└── logging/
+    └── conversion_20260205_1145.log
+```
+
+---
+
+## 🛠️ CLI Reference
+
+```bash
+sas2parquet --help
+```
+
+---
+
+## ⚙️ Configuration (Advanced)
+
+Edit constants in:
+
+```text
+src/sas2parquet/convert.py
+```
+
+---
+
+## 📄 License
+
+MIT License

{sas2parquet-0.1.9 → sas2parquet-0.2.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "sas2parquet"
-version = "0.1.9"
+version = "0.2.0"
 description = "SAS → Parquet Hybrid Converter & Validator"
 readme = "README.md"
 authors = ["Zaman Ziabakhshganji <zaman.ganji@gmail.com>"]

{sas2parquet-0.1.9 → sas2parquet-0.2.0}/src/sas2parquet/cli.py

@@ -1,5 +1,8 @@
 #!/usr/bin/env python
-"""CLI entrypoint for sas2parquet."""
+"""
+CLI entrypoint for sas2parquet.
+"""
+
 import argparse
 import sys
 from pathlib import Path
@@ -13,41 +16,83 @@ except importlib.metadata.PackageNotFoundError:
 from .convert import main as convert_dir, reconvert_file_ultimate
 
 
-def main():
-    parser = argparse.ArgumentParser(prog="sas2parquet", description="SAS to Parquet converter")
-    parser.add_argument("--version", action="version", version=f"%(prog)s {__version__}")
+def main() -> None:
+    parser = argparse.ArgumentParser(
+        prog="sas2parquet",
+        description="SAS to Parquet converter with validation",
+    )
+
+    parser.add_argument(
+        "--version",
+        action="version",
+        version=f"%(prog)s {__version__}",
+    )
+
+    # Backward compatibility (optional)
+    parser.add_argument(
+        "--dir-mode",
+        "-d",
+        action="store_true",
+        help="(Backward compatible) Treat input as a directory. "
+             "If no path is provided, defaults to ./sasdata",
+    )
 
     parser.add_argument(
         "path",
-        help="Path to a .sas7bdat file OR a directory containing SAS files (recursively)."
+        nargs="?",
+        help="Path to a .sas7bdat file OR a directory containing SAS files (recursively).",
     )
+
     parser.add_argument(
-        "--out", "-o",
+        "--out",
+        "-o",
+        default=None,
         help="Output Parquet file (file mode) OR output directory (dir mode). "
              "If omitted, dir mode uses sibling 'parquetdata/'.",
-        default=None
     )
+
     parser.add_argument(
         "--log-dir",
-        help="Directory where logs are written (dir mode). If omitted, uses sibling 'logging/'.",
-        default=None
+        default=None,
+        help="Directory where logs are written (dir mode). "
+             "If omitted, uses sibling 'logging/'.",
     )
 
     args = parser.parse_args()
 
-    p = Path(args.path).expanduser().resolve()
+    # -----------------------------
+    # Resolve input path
+    # -----------------------------
+    if args.dir_mode and args.path is None:
+        # Old behavior: default to ./sasdata
+        p = Path("sasdata").expanduser().resolve()
+    else:
+        if args.path is None:
+            parser.print_help()
+            sys.exit(1)
+        p = Path(args.path).expanduser().resolve()
+
     if not p.exists():
         print(f"❌ Path not found: {p}")
         sys.exit(2)
 
+    # -----------------------------
     # Directory mode
+    # -----------------------------
     if p.is_dir():
         out_dir = Path(args.out).expanduser().resolve() if args.out else None
         log_dir = Path(args.log_dir).expanduser().resolve() if args.log_dir else None
-        rc = convert_dir(p, parquet_output_dir=out_dir, log_dir=log_dir)
+
+        rc = convert_dir(
+            p,
+            parquet_output_dir=out_dir,
+            log_dir=log_dir,
+        )
         sys.exit(rc)
 
+    # -----------------------------
     # File mode
+    # -----------------------------
     if p.is_file():
         if p.suffix.lower() != ".sas7bdat":
             print(f"❌ Not a .sas7bdat file: {p.name}")

sas2parquet-0.1.9/README.md

@@ -1,104 +0,0 @@
-# sas2parquet
-
-[![PyPI version](https://badge.fury.io/py/sas2parquet.svg)](https://pypi.org/project/sas2parquet/)
-[![Python versions](https://img.shields.io/pypi/pyversions/sas2parquet.svg)](https://pypi.org/project/sas2parquet/)
-[![License](https://img.shields.io/pypi/l/sas2parquet.svg)](LICENCE)
-
-**The ultimate SAS (.sas7bdat) to Parquet converter** - handles problematic files that fail with standard tools. Automatic encoding detection, intelligent type inference, schema repair, and pixel-perfect validation.
-
-## ✨ Features
-
-| Feature | Description |
-|---------|-------------|
-| 🔄 **Auto Encoding** | Detects UTF-8, Latin1, CP1252 from metadata/fallback |
-| 🧠 **Smart Types** | Infers datetime, numeric, string with 20+ fallback attempts |
-| ✅ **Validation** | Compares SAS vs Parquet chunk-by-chunk (numeric + string) |
-| 📊 **Memory Safe** | Chunked processing (96GB RAM optimized, configurable) |
-| 💾 **ZSTD** | Level 6 compression for maximum efficiency |
-| 📝 **Detailed Logs** | Mismatch reports + full conversion trace |
-| 🎯 **Two Modes** | Single file OR recursive directory processing |
-
-## Quick Start
-
-### Install
-```bash
-pip install sas2parquet
-```
-
-### Single File
-```bash
-sas2parquet input.sas output.parquet
-```
-
-### Batch Directory (Recommended)
-```bash
-sas2parquet --dir-mode
-```
-
-## 📁 Directory Mode (Default Workflow)
-
-### How it works
-
-- You provide a `sasdata/` directory containing all `.sas7bdat` files (including nested subfolders).
-- The tool automatically creates a `parquetdata/` directory in the same parent folder as `sasdata/`.
-- All files are converted to Parquet and written into `parquetdata/`, mirroring the original folder structure.
-
-```text
-your-project/
-├── sasdata/              # ← Put your .sas7bdat files here
-│   ├── file1.sas7bdat
-│   └── subfolder/
-│       └── nested.sas7bdat
-├── parquetdata/          # ← AUTO-CREATED (mirrors sasdata/)
-│   ├── file1.parquet
-│   └── subfolder/
-│       └── nested.parquet
-└── logging/             # ← AUTO-CREATED (detailed logs)
-    └── conversion_20260205_1145.log
-```
-
-Just run:
-```bash
-sas2parquet --dir-mode
-```
-
-## 🛠️ CLI Reference
-```bash
-sas2parquet --help
-```
-
-```text
-usage: sas2parquet [-h] [--dir-mode] [sas_file] [parquet_file]
-
-Robust SAS to Parquet converter with validation
-```
-
-## 📊 Example Output
-```text
-🚀 SAS → Parquet Hybrid Fix & Validate (full folder)
-Found 3 files.
-============================================================
-...
-```
-
-## ⚙️ Configuration (Advanced)
-
-Edit `src/sas2parquet/convert.py` constants:
-
-```python
-AVAILABLE_RAM_GB = 96
-RAM_USAGE_FACTOR = 0.5
-ZSTD_COMPRESSION_LEVEL = 6
-MIN_CHUNK_SIZE = 100_000
-MAX_CHUNK_SIZE = 10_000_000
-```
-
-## 🧪 Validation Details
-Each file undergoes 4-stage validation:
-1. Metadata
-2. Exact counts
-3. Column order
-4. Value comparison
-
-## 📄 License
-MIT License