lidar-converter 0.2.0__tar.gz

lidar_converter-0.2.0/CHANGELOG.md

@@ -0,0 +1,83 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+### Added
+- `requirements.txt` aligned with `pyproject.toml` runtime dependencies
+- `TESTING_GUIDE.md` with manual CLI examples, SDK / `pip` notes, and link to **[lidar-multi-convert](https://github.com/Param-Patel-o5/lidar-multi-convert)**; accurate `Lidar_Converter/CLI_README.md`
+
+### Changed
+- CLI program name in help text: `lidar-converter` (matches the PyPI package and console script)
+- **`pyproject.toml` `[project.urls]`** and docs now use the GitHub repo **`Param-Patel-o5/lidar-multi-convert`** (PyPI name stays `lidar-converter`)
+- Removed unused `click` dependency (CLI uses `argparse`)
+- README: project layout, install clone URL, SDK vs PCAP table, testing references
+
+### Removed
+- Duplicate `[tool.flake8]` block from `pyproject.toml` (use `.flake8` at repo root)
+- `[tool.pytest.ini_options]` from `pyproject.toml` (no bundled test suite)
+
+## [0.2.0] - 2024-11-18
+
+### Added
+- **Hesai LiDAR Support** - Full support for Hesai sensors (PandarXT-32, PandarXT-16, Pandar64, Pandar40P, PandarQT)
+  - Automatic vendor detection with 48% confidence scoring
+  - PCAP packet parsing based on official Hesai documentation
+  - Support for all output formats (LAS, LAZ, PCD, BIN, CSV)
+  - Dynamic packet structure parsing for flexibility across models
+  - Reads channel count, block count, and distance unit from packet headers
+  - Adaptive block size calculation for different Hesai models
+- **LAZ Compression** - Real compression now working for all vendors
+  - Added `lazrs>=0.8.0` dependency for LAZ compression
+  - Compression ratio: 60-80% smaller than uncompressed LAS
+  - Verified compression for Ouster, Velodyne, Livox, and Hesai
+  - Typical compression: 3-15 bytes/point (vs 28-30 for uncompressed)
+- **PyPI Package Configuration** - Professional package setup for PyPI publishing
+  - MIT License file
+  - Contributing guidelines (CONTRIBUTING.md)
+  - Code of Conduct (CODE_OF_CONDUCT.md)
+  - Comprehensive tool configurations (black, pytest, flake8, mypy)
+  - Optional dependencies for development and documentation
+  - PyPI badges in README
+- Livox LiDAR support (Avia, Horizon, Tele-15, Mid-40/70, HAP)
+- Multiple output format support (PCD, BIN, CSV) in addition to LAS/LAZ
+- Batch conversion capability
+- Health check command
+- Comprehensive CLI with rich output formatting
+
+### Changed
+- Updated README.md with Hesai support information and PyPI installation instructions
+- Updated vendor detection to include Hesai magic bytes (0xEEFF)
+- Updated UDP port detection to recognize Hesai port (2368)
+- Improved packet size detection for Hesai (861 bytes)
+- Enhanced documentation with Hesai packet structure details
+- Improved vendor detection with multi-method approach
+- Enhanced error handling and logging
+- Optimized performance with configurable scan limits
+- Fixed package discovery in pyproject.toml for proper subpackage inclusion
+
+### Fixed
+- Syntax error in __init__.py (missing closing quote in __all__)
+- LAZ compression now works properly (was creating uncompressed files before)
+- Hesai packet parsing now correctly reads data from byte 12 (not byte 42)
+- Fixed block size calculation to use header information
+- Improved point filtering to remove invalid/noise points
+
+## [0.1.0] - 2025-11-01
+
+### Added
+- Initial release
+- Ouster LiDAR support (OS-0, OS-1, OS-2, OS-Dome series)
+- Velodyne LiDAR support (VLP-16, VLP-32C, HDL-32E, HDL-64E, VLS-128)
+- Automatic vendor detection
+- LAS/LAZ output format support
+- Command-line interface
+- Python API
+
+[Unreleased]: https://github.com/Param-Patel-o5/lidar-multi-convert/compare/v0.2.0...HEAD
+[0.2.0]: https://github.com/Param-Patel-o5/lidar-multi-convert/compare/v0.1.0...v0.2.0
+[0.1.0]: https://github.com/Param-Patel-o5/lidar-multi-convert/releases/tag/v0.1.0

lidar_converter-0.2.0/CONTRIBUTING.md

@@ -0,0 +1,104 @@
+# Contributing to LiDAR Converter
+
+Thank you for your interest in contributing to LiDAR Converter! We appreciate your effort.
+
+## Development Setup
+
+### Prerequisites
+- Python 3.8 or higher
+- Git
+
+### Setup Instructions
+
+1. **Fork and clone the repository:**
+```bash
+git clone https://github.com/Param-Patel-o5/lidar-multi-convert.git
+cd lidar-multi-convert
+```
+
+2. **Create a virtual environment:**
+```bash
+python -m venv venv
+source venv/bin/activate  # On Windows: venv\Scripts\activate
+```
+
+3. **Install in editable mode with dev dependencies:**
+```bash
+pip install -e ".[dev]"
+```
+
+## Running Tests
+
+Run all tests:
+```bash
+pytest
+```
+
+Run with coverage report:
+```bash
+pytest --cov=Lidar_Converter --cov-report=html
+```
+
+## Code Quality
+
+We maintain consistent code quality using:
+
+### Format with Black
+```bash
+black Lidar_Converter/
+black tests/
+```
+
+### Lint with Flake8
+```bash
+flake8 Lidar_Converter/
+flake8 tests/
+```
+
+### Type checking with mypy
+```bash
+mypy Lidar_Converter/
+```
+
+### Run all checks
+```bash
+black Lidar_Converter/ tests/ && flake8 Lidar_Converter/ tests/ && mypy Lidar_Converter/ && pytest
+```
+
+## Making Changes
+
+1. **Create a feature branch:**
+```bash
+git checkout -b feature/your-feature-name
+```
+
+2. **Make your changes and commit:**
+```bash
+git add .
+git commit -m "feat: description of your changes"
+```
+
+Follow conventional commit format:
+- `feat:` for new features
+- `fix:` for bug fixes
+- `docs:` for documentation
+- `test:` for test additions
+- `refactor:` for code refactoring
+- `style:` for formatting changes
+
+3. **Push and create a Pull Request:**
+```bash
+git push origin feature/your-feature-name
+```
+
+## Pull Request Guidelines
+
+- Include a clear description of changes
+- Reference related issues (e.g., "Fixes #123")
+- Ensure all tests pass locally
+- Update documentation if needed
+- Follow code style guidelines
+
+## Questions?
+
+Feel free to open an issue or discussion in the repository.

lidar_converter-0.2.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Param Patel
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

lidar_converter-0.2.0/Lidar_Converter/CLI_README.md

@@ -0,0 +1,113 @@
+# lidar-converter — command-line interface
+
+The installed console script is **`lidar-converter`** (defined in `pyproject.toml`).  
+Global options (`--log-level`, `--config`, `--output-format`, etc.) must appear **before** the subcommand.
+
+## Install
+
+```bash
+pip install lidar-converter
+# or from source:
+pip install -e .
+```
+
+Verify:
+
+```bash
+lidar-converter --help
+lidar-converter --log-level ERROR --output-format json health
+```
+
+## Subcommands
+
+| Command | Purpose |
+|---------|---------|
+| `convert` | One input file → one output file (format from `-o` extension or `-f`) |
+| `batch` | Many files in a directory (shared output format) |
+| `detect` | Vendor detection only |
+| `health` | SDK / wrapper availability |
+| `test` | Run pipeline test on one file (detect + wrapper test conversion) |
+
+There is **no** `info` subcommand and **no** `--version` flag in the current CLI (use `pip show lidar-converter` for the package version).
+
+## `convert`
+
+**Syntax:**
+
+```bash
+lidar-converter convert INPUT_FILE -o OUTPUT_FILE [options]
+```
+
+**Important:** output path is **` -o ` / `--output`**, not a second positional argument.
+
+**Examples:**
+
+```bash
+# LAS (extension chooses format)
+lidar-converter convert data.pcap -o out.las --max-scans 10
+
+# LAZ, PCD, BIN, CSV
+lidar-converter convert data.pcap -o out.laz --max-scans 10
+lidar-converter convert data.pcap -o out.pcd --max-scans 10
+lidar-converter convert data.pcap -o out.bin --max-scans 10
+lidar-converter convert data.pcap -o out.csv --max-scans 10
+
+# Explicit format (optional)
+lidar-converter convert data.pcap -o out.file --format laz
+
+# Ouster: metadata JSON beside PCAP (same stem) or pass explicitly
+lidar-converter convert scan.pcap -o out.las -c scan.json --max-scans 10
+```
+
+**Common options:**
+
+- `-o`, `--output` — output file path  
+- `-f`, `--format` — `las`, `laz`, `pcd`, `bin`, `csv` (optional if extension is clear)  
+- `--max-scans` — limit scans for faster tests (default in CLI: 1000)  
+- `-c`, `--calibration` — calibration / metadata file  
+- `--validate` — validate output after conversion  
+- `--no-intensity` — drop intensity  
+
+## `batch`
+
+```bash
+lidar-converter batch INPUT_DIR -o OUTPUT_DIR [--pattern "*.pcap"] [-r] [--max-scans 10]
+```
+
+Output filenames are derived from input stems; format comes from `-f` or config.
+
+## `detect`
+
+```bash
+lidar-converter --output-format json detect file.pcap
+```
+
+## `health`
+
+```bash
+lidar-converter --output-format json health
+```
+
+Use JSON on Windows if the Rich tables fail due to console encoding (see `TESTING_GUIDE.md`).
+
+## `test`
+
+```bash
+lidar-converter test file.pcap
+```
+
+## Configuration file
+
+Optional JSON (merged with defaults), searched in order:
+
+1. Path from `--config`  
+2. `~/.lidar_converter.json`  
+3. `./.lidar_converter.json`  
+
+Environment overrides: `LIDAR_OUTPUT_FORMAT`, `LIDAR_OUTPUT_DIR`, `LIDAR_LOG_LEVEL`.
+
+## Further reading
+
+- Repository root: `README.md`, `TESTING_GUIDE.md`  
+- Source on GitHub: [Param-Patel-o5/lidar-multi-convert](https://github.com/Param-Patel-o5/lidar-multi-convert) (PyPI: `lidar-converter`)  
+- Wrappers: `Lidar_Converter/Wrappers/README.md`

lidar_converter-0.2.0/Lidar_Converter/Wrappers/README.md

@@ -0,0 +1,126 @@
+# Vendor Wrapper Modules
+
+This directory contains vendor-specific wrappers that provide a unified interface for LiDAR conversion across different manufacturers.
+
+## Architecture
+
+All vendor wrappers follow the same architecture pattern:
+
+```
+BaseVendorWrapper (abstract base class)
+    ├── OusterWrapper ✅ (implemented - Python SDK)
+    ├── VelodyneWrapper ✅ (implemented - dpkt parsing)
+    ├── HesaiWrapper (planned)
+    ├── LivoxWrapper (planned)
+    ├── RIEGLWrapper (planned)
+    └── SICKWrapper (planned)
+```
+
+## Design Principles
+
+1. **Unified Interface**: All wrappers implement the same abstract methods
+2. **Abstraction**: Vendor-specific details are hidden from converter.py
+3. **Robustness**: Comprehensive error handling and validation
+4. **Extensibility**: Easy to add new vendors following the same pattern
+5. **Documentation**: Clear docstrings and usage examples
+
+## Usage
+
+### Basic Usage
+
+```python
+from Wrappers import OusterWrapper, VelodyneWrapper
+
+# Ouster conversion (requires ouster-sdk)
+ouster_wrapper = OusterWrapper()
+if ouster_wrapper.sdk_available:
+    result = ouster_wrapper.convert_to_las(
+        input_path="ouster_data.pcap",
+        output_path="output.las",
+        calibration_file="metadata.json"  # Required for Ouster
+    )
+
+# Velodyne conversion (uses dpkt fallback)
+velodyne_wrapper = VelodyneWrapper()
+if velodyne_wrapper.sdk_available:
+    result = velodyne_wrapper.convert_to_las(
+        input_path="velodyne_data.pcap",
+        output_path="output.las",
+        sensor_model="VLP-16"  # Optional, auto-detected if not provided
+    )
+
+# Check results
+if result["success"]:
+    print(f"Converted {result['points_converted']} points")
+    print(f"Time: {result['conversion_time']:.2f}s")
+    print(f"Vendor: {result.get('vendor', 'unknown')}")
+else:
+    print(f"Error: {result['error']}")
+```
+
+### Integration with converter.py
+
+```python
+from converter import LiDARConverter
+
+# Automatic vendor detection and conversion
+converter = LiDARConverter()
+result = converter.convert("file.pcap", "output.las")
+
+# Manual wrapper usage
+from detector import VendorDetector
+from Wrappers import OusterWrapper, VelodyneWrapper
+
+detector = VendorDetector()
+detection_result = detector.detect_vendor("file.pcap")
+
+if detection_result["success"]:
+    vendor = detection_result["vendor_name"]
+    
+    if vendor == "ouster":
+        wrapper = OusterWrapper()
+    elif vendor == "velodyne":
+        wrapper = VelodyneWrapper()
+    
+    if wrapper.sdk_available:
+        result = wrapper.convert_to_las("file.pcap", "output.las")
+```
+
+## Wrapper Interface
+
+All wrappers must implement:
+
+- `get_vendor_name()` → str
+- `validate_sdk_installation()` → dict
+- `convert_to_las(input_path, output_path, **kwargs)` → dict
+- `convert(input_path, output_format, output_path, **kwargs)` → dict
+- `get_vendor_info()` → dict
+- `validate_conversion(input_path, output_path)` → bool
+
+## Adding New Vendors
+
+To add a new vendor wrapper:
+
+1. Create `{vendor}_wrapper.py` in this directory
+2. Inherit from `BaseVendorWrapper`
+3. Implement all abstract methods
+4. Follow the same error handling and logging patterns
+5. Add to `__init__.py` exports
+6. Document vendor-specific requirements
+
+See `ouster_wrapper.py` and `velodyne_wrapper.py` as reference implementations.
+
+## Current Implementations
+
+### OusterWrapper
+- **SDK**: Uses official Ouster Python SDK (ouster-sdk)
+- **Requirements**: Requires JSON metadata file
+- **Sensors**: OS-0, OS-1, OS-2, OS-Dome series (16/32/64/128 channels)
+- **Features**: Full SDK integration with coordinate transformations
+
+### VelodyneWrapper  
+- **SDK**: Uses dpkt for PCAP parsing (no vendor SDK required)
+- **Requirements**: Only PCAP file needed
+- **Sensors**: VLP-16, VLP-32C, HDL-32E, HDL-64E, VLS-128
+- **Features**: Manual packet parsing with coordinate conversion
+

lidar_converter-0.2.0/Lidar_Converter/Wrappers/__init__.py

@@ -0,0 +1,30 @@
+"""
+Vendor-specific wrapper modules for LiDAR conversion.
+
+This package contains vendor-specific wrappers that abstract SDK complexity
+behind a unified interface. All wrappers inherit from BaseVendorWrapper and
+provide consistent method signatures for seamless integration with converter.py.
+
+Supported Vendors:
+    - Ouster: Full implementation
+    - Velodyne: Full implementation
+    - Hesai: Full implementation
+    - Livox: Full implementation
+    - RIEGL: Planned
+    - SICK: Planned
+"""
+
+from .base_wrapper import BaseVendorWrapper
+from .ouster_wrapper import OusterWrapper
+from .velodyne_wrapper import VelodyneWrapper
+from .livox_wrapper import LivoxWrapper
+from .hesai_wrapper import HesaiWrapper
+
+__all__ = [
+    "BaseVendorWrapper",
+    "OusterWrapper",
+    "VelodyneWrapper",
+    "LivoxWrapper",
+    "HesaiWrapper",
+]
+