pycstructdataparser-lib 1.0.0__tar.gz → 1.0.2__tar.gz

pycstructdataparser_lib-1.0.2/PKG-INFO

@@ -0,0 +1,299 @@
+Metadata-Version: 2.4
+Name: pycstructdataparser-lib
+Version: 1.0.2
+Summary: Parse C/C++ struct definitions from header text and serialize/deserialize binary data with ctypes — no compilation needed
+Author: NGC13009
+License-Expression: GPL-3.0-only
+Project-URL: Homepage, https://github.com/NGC13009/cStructDataParser
+Project-URL: Source, https://github.com/NGC13009/cStructDataParser
+Project-URL: Tracker, https://github.com/NGC13009/cStructDataParser/issues
+Keywords: ctypes,C struct,binary,serialization,deserialization,header parser,pack,unpack,c-struct,binary-parser,embedded,protocol,pragma-pack,bitfield,struct-parser
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: System :: Archiving :: Packaging
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+# pycstructdataparser-lib
+
+<p align="center">
+  <a href="https://pypi.org/project/pycstructdataparser-lib/"><img src="https://img.shields.io/pypi/v/pycstructdataparser-lib" alt="PyPI"></a>
+  <a href="https://pypi.org/project/pycstructdataparser-lib/"><img src="https://img.shields.io/pypi/pyversions/pycstructdataparser-lib" alt="Python Versions"></a>
+  <a href="LICENSE"><img src="https://img.shields.io/badge/License-GPLv3-blue.svg" alt="License: GPL v3"></a>
+</p>
+
+<p align="center">
+  [<a href="https://github.com/NGC13009/pycstructdataparser_lib/blob/main/README_CN.md">中文说明书</a>] | English
+</p>
+
+**Automatically parse struct definitions directly from C/C++ header text and process binary data in Python** — No compilation required, no manual `ctypes.Structure` subclass writing needed, zero boilerplate code. Seamless data transfer between C/C++ and Python with automated data unpacking/packing.
+
+## Overview
+
+Specifically, this package uses Python's built-in `ctypes` module to parse struct definitions from C/C++ `.h` files and handle their corresponding binary data. You only need to provide the struct definition text from the header file to accomplish binary data serialization (pack) and deserialization (unpack).
+
+With this module, users no longer need to rewrite Python CRUD code each time data field definitions change — packing/unpacking methods can be generated directly from `.h` files.
+
+Ideal for data communication between Python and C/C++ programs, such as:
+
+- TCP/UDP bitstream message transmission
+- Local binary data file read/write (`.bin` files, `.dat` files, etc.)
+- Project scenarios where data protocols change frequently
+- Development stages with rapid protocol iteration
+
+## Quick Start
+
+Install directly from PyPI:
+
+```bash
+pip install pycstructdataparser-lib
+```
+
+Then it's ready to use. The following code demonstrates the complete core workflow:
+
+> Define struct → Parse → Pack → Unpack
+
+```python
+from pycstructdataparser_lib import CStructParser, pack, unpack
+
+# 1. Prepare C struct definition text
+header = '''
+#pragma pack(push, 1)
+typedef struct {
+    unsigned char id;
+    unsigned int  value;
+    float         data[4];
+} MyStruct;
+#pragma pack(pop)
+'''
+
+# 2. Parse the header file and register the struct type
+parser = CStructParser()
+parser.parse(header)
+
+# 3. Pack a Python dictionary into binary bytes (serialization)
+original = {'id': 10, 'value': 12345, 'data': [1.0, 2.0, 3.0, 4.0]}
+raw = pack(parser, 'MyStruct', original)
+print(f"pack output: {len(raw)} bytes")
+
+# 4. Unpack binary bytes back into a Python dictionary (deserialization)
+restored = unpack(parser, 'MyStruct', raw)
+print(restored)
+```
+
+Expected output:
+
+```powershell
+pack output: 21 bytes
+{'id': 10, 'value': 12345, 'data': [1.0, 2.0, 3.0, 4.0]}
+```
+
+## Features
+
+- **Bitfield Support**: Parses bitfield declarations in the form `unsigned char field : 2;`
+- **C Header Parsing**: Recognizes `typedef struct { … } TypeName;` blocks, automatically extracting struct definitions
+- **`#pragma pack` Support**: Correctly handles `#pragma pack(push, N)` and `#pragma pack(pop)` alignment stack
+- **Array Support**: Handles fixed-size arrays, e.g., `Type array[10];`
+- **Nested Structs**: Supports struct types used as fields within other structs
+- **Serialization (pack)**: Converts Python dictionaries into C struct binary bytes
+- **Deserialization (unpack)**: Parses raw bytes back into nested Python dictionaries
+- **Strict Mode**: Optionally validates that all non-bitfield fields are present during packing
+
+## Installation Methods
+
+### Method 1: PyPI Installation (Recommended)
+
+```bash
+pip install pycstructdataparser-lib
+```
+
+### Method 2: Local Use from GitHub Repository
+
+After cloning the repository, you can install it in editable mode or add the repository directory to the Python search path.
+
+**Editable Mode Installation (Recommended)**:
+
+```bash
+git clone https://github.com/NGC13009/pycstructdataparser_lib.git
+cd pycstructdataparser_lib
+pip install -e .
+```
+
+**Adding the Path Directly in Code** (no installation required):
+
+```python
+import sys
+sys.path.insert(0, '/path/to/pycstructdataparser_lib')
+from pycstructdataparser_lib import CStructParser, pack, unpack
+```
+
+> This repository uses a flat module layout — the root directory (containing `__init__.py`) is itself the package directory. Placing this directory in the Python package search path (`sys.path`) allows normal imports via `from pycstructdataparser_lib import ...`.
+> Known Issue⚠: Due to this flat structure, the source code may require minor adjustments before use, such as renaming the folder, removing the `pycstructdataparser_lib.py` file, and migrating its content into `__init__.py`.
+
+### Method 3: Install via .whl Package
+
+Download the latest `.whl` file from the [Release page](https://github.com/NGC13009/pycstructdataparser_lib/releases), then run:
+
+```bash
+pip install pycstructdataparser_lib-*.whl
+```
+
+### Method 4: Manual Build
+
+```bash
+pip install build  # Build toolchain
+git clone https://github.com/NGC13009/pycstructdataparser_lib.git
+cd pycstructdataparser_lib
+python -m build
+pip install dist/pycstructdataparser_lib-*.whl
+```
+
+## Usage Guide
+
+### Core API
+
+| Function / Method | Description |
+|------------------|-------------|
+| `CStructParser()` | Creates a parser instance |
+| `parser.parse(header_content)` | Parses a C/C++ header string, registering the struct types defined within |
+| `parser.get_struct_class(name)` | Retrieves the corresponding `ctypes.Structure` subclass by type name |
+| `parser.struct_classes` | Returns a dictionary of all registered struct classes (property) |
+| `pack(parser, struct_name, data_dict, strict=True)` | Serializes a Python dictionary into a binary byte sequence |
+| `unpack(parser, struct_name, data)` | Deserializes a binary byte sequence into a Python dictionary |
+
+### Basic Usage: Directly Processing Header Strings and Binary Streams
+
+The core workflow of this library is: pass the struct definition text from a C header file to `CStructParser.parse()`, then use `pack()` and `unpack()` to convert between Python dictionaries and binary bytes. See the "Quick Start" example above for details.
+
+### Reading/Writing Local Binary Files
+
+Using `test_roundtrip.py` as a reference, a typical workflow proceeds as follows:
+
+1. Read a header file → Parse it with `CStructParser`
+2. Construct a Python dictionary as test data
+3. Call `pack()` to serialize the dictionary into binary bytes, then write to a `.bin` file
+4. Read bytes from the `.bin` file, call `unpack()` to restore them into a dictionary
+5. Compare the original data with the restored data to verify consistency
+
+For detailed implementation, refer to the [`test_roundtrip.py`](test_roundtrip.py) file in this repository. This file also demonstrates operations such as saving a dictionary as a JSON file and packing after loading from JSON.
+
+### Application in Network Communication
+
+In TCP/UDP programming, the output of `pack()` can be sent directly to a socket, and bytes received from a socket can be passed directly to `unpack()`. Since the library's inputs and outputs are all of `bytes` type, it is naturally compatible with standard socket interfaces. Business-specific logic (such as handling message boundaries and byte order negotiation) must be implemented by the user according to the actual protocol.
+
+## API Reference
+
+### `CStructParser`
+
+```python
+class CStructParser:
+    def __init__(self) -> None: ...
+    def parse(self, header_content: str) -> None: ...
+    def get_struct_class(self, name: str) -> type: ...
+    @property
+    def struct_classes(self) -> Dict[str, type]: ...
+```
+
+**`parse(header_content)`**
+
+Parses a C/C++ header string, extracting all `typedef struct { … } Name;` blocks and dynamically creating a `ctypes.Structure` subclass for each struct.
+
+Supported features:
+
+- C-style comments (`//` and `/* … */`)
+- `#pragma pack(push, N)` / `#pragma pack(pop)` alignment directives
+- Standard C basic types, bitfields, fixed-size arrays
+- Nested named structs (forward references are handled preferentially)
+
+**`get_struct_class(name)`**
+
+Returns the `ctypes.Structure` subclass corresponding to the specified name. Raises `ValueError` if the name is not registered.
+
+**`struct_classes`**
+
+Returns a dictionary of all registered struct classes, with struct names as keys and corresponding `ctypes.Structure` subclasses as values.
+
+### `pack(parser, struct_name, data_dict, strict=True)`
+
+```python
+def pack(
+    parser: CStructParser,
+    struct_name: str,
+    data_dict: Dict[str, Any],
+    strict: bool = True,
+) -> bytes: ...
+```
+
+Serializes a Python dictionary into binary bytes corresponding to a C struct.
+
+- `parser`: A `CStructParser` instance on which `parse()` has been called.
+- `struct_name`: The target struct type name.
+- `data_dict`: A dictionary mapping field names to field values.
+- `strict`: If `True` (default), raises `ValueError` when non-bitfield fields are missing; if `False`, missing fields are silently filled with 0.
+
+### `unpack(parser, struct_name, data)`
+
+```python
+def unpack(
+    parser: CStructParser,
+    struct_name: str,
+    data: bytes,
+) -> Dict[str, Any]: ...
+```
+
+Deserializes raw binary bytes into a nested Python dictionary.
+
+- `parser`: A `CStructParser` instance on which `parse()` has been called.
+- `struct_name`: The target struct type name.
+- `data`: The binary byte sequence to decode.
+
+## Supported C Type Mapping
+
+| C Type | ctypes Type |
+|--------|-------------|
+| `char`, `signed char` | `c_byte` |
+| `unsigned char` | `c_ubyte` |
+| `short`, `short int` | `c_short` |
+| `unsigned short` | `c_ushort` |
+| `int`, `signed int` | `c_int` |
+| `unsigned int` | `c_uint` |
+| `long`, `long int` | `c_long` |
+| `unsigned long` | `c_ulong` |
+| `long long`, `long long int` | `c_longlong` |
+| `unsigned long long` | `c_ulonglong` |
+| `float` | `c_float` |
+| `double` | `c_double` |
+| `long double` | `c_longdouble` |
+| `int8_t` .. `uint64_t` | `c_int8` .. `c_uint64` |
+| `float32_t` | `c_float` |
+| `float64_t` | `c_double` |
+
+## Comparison with Common Approaches
+
+| Feature | `struct` module | `ctypes.Structure` | `construct` library | cStructDataParser |
+|---------|:---:|:---:|:---:|:---:|
+| Direct parsing from header text | ❌ | ❌ | ❌ | ✅ |
+| Automatic `#pragma pack` alignment | ❌ | ❌ | ❌ | ✅ |
+| Bitfield support | ❌ | ✅ | ✅ | ✅ |
+| Nested structs | ❌ | Manual writing required | ✅ | ✅ |
+| No external dependencies (pure Python) | ✅ | ✅ | ✅ | ✅ |
+
+## Known Limitations
+
+- Currently only supports **little-endian** byte order; big-endian scenarios are not yet handled
+- If the device uses big-endian order, additional byte order processing is needed before/after calling `pack()` / `unpack()`
+
+## License
+
+This project is licensed under [GPLv3.0](https://www.gnu.org/licenses/gpl-3.0.en.html).
+[NGC13009/pycstructdataparser_lib.git](https://github.com/NGC13009/pycstructdataparser_lib.git)

pycstructdataparser_lib-1.0.2/README.md

@@ -0,0 +1,273 @@
+# pycstructdataparser-lib
+
+<p align="center">
+  <a href="https://pypi.org/project/pycstructdataparser-lib/"><img src="https://img.shields.io/pypi/v/pycstructdataparser-lib" alt="PyPI"></a>
+  <a href="https://pypi.org/project/pycstructdataparser-lib/"><img src="https://img.shields.io/pypi/pyversions/pycstructdataparser-lib" alt="Python Versions"></a>
+  <a href="LICENSE"><img src="https://img.shields.io/badge/License-GPLv3-blue.svg" alt="License: GPL v3"></a>
+</p>
+
+<p align="center">
+  [<a href="https://github.com/NGC13009/pycstructdataparser_lib/blob/main/README_CN.md">中文说明书</a>] | English
+</p>
+
+**Automatically parse struct definitions directly from C/C++ header text and process binary data in Python** — No compilation required, no manual `ctypes.Structure` subclass writing needed, zero boilerplate code. Seamless data transfer between C/C++ and Python with automated data unpacking/packing.
+
+## Overview
+
+Specifically, this package uses Python's built-in `ctypes` module to parse struct definitions from C/C++ `.h` files and handle their corresponding binary data. You only need to provide the struct definition text from the header file to accomplish binary data serialization (pack) and deserialization (unpack).
+
+With this module, users no longer need to rewrite Python CRUD code each time data field definitions change — packing/unpacking methods can be generated directly from `.h` files.
+
+Ideal for data communication between Python and C/C++ programs, such as:
+
+- TCP/UDP bitstream message transmission
+- Local binary data file read/write (`.bin` files, `.dat` files, etc.)
+- Project scenarios where data protocols change frequently
+- Development stages with rapid protocol iteration
+
+## Quick Start
+
+Install directly from PyPI:
+
+```bash
+pip install pycstructdataparser-lib
+```
+
+Then it's ready to use. The following code demonstrates the complete core workflow:
+
+> Define struct → Parse → Pack → Unpack
+
+```python
+from pycstructdataparser_lib import CStructParser, pack, unpack
+
+# 1. Prepare C struct definition text
+header = '''
+#pragma pack(push, 1)
+typedef struct {
+    unsigned char id;
+    unsigned int  value;
+    float         data[4];
+} MyStruct;
+#pragma pack(pop)
+'''
+
+# 2. Parse the header file and register the struct type
+parser = CStructParser()
+parser.parse(header)
+
+# 3. Pack a Python dictionary into binary bytes (serialization)
+original = {'id': 10, 'value': 12345, 'data': [1.0, 2.0, 3.0, 4.0]}
+raw = pack(parser, 'MyStruct', original)
+print(f"pack output: {len(raw)} bytes")
+
+# 4. Unpack binary bytes back into a Python dictionary (deserialization)
+restored = unpack(parser, 'MyStruct', raw)
+print(restored)
+```
+
+Expected output:
+
+```powershell
+pack output: 21 bytes
+{'id': 10, 'value': 12345, 'data': [1.0, 2.0, 3.0, 4.0]}
+```
+
+## Features
+
+- **Bitfield Support**: Parses bitfield declarations in the form `unsigned char field : 2;`
+- **C Header Parsing**: Recognizes `typedef struct { … } TypeName;` blocks, automatically extracting struct definitions
+- **`#pragma pack` Support**: Correctly handles `#pragma pack(push, N)` and `#pragma pack(pop)` alignment stack
+- **Array Support**: Handles fixed-size arrays, e.g., `Type array[10];`
+- **Nested Structs**: Supports struct types used as fields within other structs
+- **Serialization (pack)**: Converts Python dictionaries into C struct binary bytes
+- **Deserialization (unpack)**: Parses raw bytes back into nested Python dictionaries
+- **Strict Mode**: Optionally validates that all non-bitfield fields are present during packing
+
+## Installation Methods
+
+### Method 1: PyPI Installation (Recommended)
+
+```bash
+pip install pycstructdataparser-lib
+```
+
+### Method 2: Local Use from GitHub Repository
+
+After cloning the repository, you can install it in editable mode or add the repository directory to the Python search path.
+
+**Editable Mode Installation (Recommended)**:
+
+```bash
+git clone https://github.com/NGC13009/pycstructdataparser_lib.git
+cd pycstructdataparser_lib
+pip install -e .
+```
+
+**Adding the Path Directly in Code** (no installation required):
+
+```python
+import sys
+sys.path.insert(0, '/path/to/pycstructdataparser_lib')
+from pycstructdataparser_lib import CStructParser, pack, unpack
+```
+
+> This repository uses a flat module layout — the root directory (containing `__init__.py`) is itself the package directory. Placing this directory in the Python package search path (`sys.path`) allows normal imports via `from pycstructdataparser_lib import ...`.
+> Known Issue⚠: Due to this flat structure, the source code may require minor adjustments before use, such as renaming the folder, removing the `pycstructdataparser_lib.py` file, and migrating its content into `__init__.py`.
+
+### Method 3: Install via .whl Package
+
+Download the latest `.whl` file from the [Release page](https://github.com/NGC13009/pycstructdataparser_lib/releases), then run:
+
+```bash
+pip install pycstructdataparser_lib-*.whl
+```
+
+### Method 4: Manual Build
+
+```bash
+pip install build  # Build toolchain
+git clone https://github.com/NGC13009/pycstructdataparser_lib.git
+cd pycstructdataparser_lib
+python -m build
+pip install dist/pycstructdataparser_lib-*.whl
+```
+
+## Usage Guide
+
+### Core API
+
+| Function / Method | Description |
+|------------------|-------------|
+| `CStructParser()` | Creates a parser instance |
+| `parser.parse(header_content)` | Parses a C/C++ header string, registering the struct types defined within |
+| `parser.get_struct_class(name)` | Retrieves the corresponding `ctypes.Structure` subclass by type name |
+| `parser.struct_classes` | Returns a dictionary of all registered struct classes (property) |
+| `pack(parser, struct_name, data_dict, strict=True)` | Serializes a Python dictionary into a binary byte sequence |
+| `unpack(parser, struct_name, data)` | Deserializes a binary byte sequence into a Python dictionary |
+
+### Basic Usage: Directly Processing Header Strings and Binary Streams
+
+The core workflow of this library is: pass the struct definition text from a C header file to `CStructParser.parse()`, then use `pack()` and `unpack()` to convert between Python dictionaries and binary bytes. See the "Quick Start" example above for details.
+
+### Reading/Writing Local Binary Files
+
+Using `test_roundtrip.py` as a reference, a typical workflow proceeds as follows:
+
+1. Read a header file → Parse it with `CStructParser`
+2. Construct a Python dictionary as test data
+3. Call `pack()` to serialize the dictionary into binary bytes, then write to a `.bin` file
+4. Read bytes from the `.bin` file, call `unpack()` to restore them into a dictionary
+5. Compare the original data with the restored data to verify consistency
+
+For detailed implementation, refer to the [`test_roundtrip.py`](test_roundtrip.py) file in this repository. This file also demonstrates operations such as saving a dictionary as a JSON file and packing after loading from JSON.
+
+### Application in Network Communication
+
+In TCP/UDP programming, the output of `pack()` can be sent directly to a socket, and bytes received from a socket can be passed directly to `unpack()`. Since the library's inputs and outputs are all of `bytes` type, it is naturally compatible with standard socket interfaces. Business-specific logic (such as handling message boundaries and byte order negotiation) must be implemented by the user according to the actual protocol.
+
+## API Reference
+
+### `CStructParser`
+
+```python
+class CStructParser:
+    def __init__(self) -> None: ...
+    def parse(self, header_content: str) -> None: ...
+    def get_struct_class(self, name: str) -> type: ...
+    @property
+    def struct_classes(self) -> Dict[str, type]: ...
+```
+
+**`parse(header_content)`**
+
+Parses a C/C++ header string, extracting all `typedef struct { … } Name;` blocks and dynamically creating a `ctypes.Structure` subclass for each struct.
+
+Supported features:
+
+- C-style comments (`//` and `/* … */`)
+- `#pragma pack(push, N)` / `#pragma pack(pop)` alignment directives
+- Standard C basic types, bitfields, fixed-size arrays
+- Nested named structs (forward references are handled preferentially)
+
+**`get_struct_class(name)`**
+
+Returns the `ctypes.Structure` subclass corresponding to the specified name. Raises `ValueError` if the name is not registered.
+
+**`struct_classes`**
+
+Returns a dictionary of all registered struct classes, with struct names as keys and corresponding `ctypes.Structure` subclasses as values.
+
+### `pack(parser, struct_name, data_dict, strict=True)`
+
+```python
+def pack(
+    parser: CStructParser,
+    struct_name: str,
+    data_dict: Dict[str, Any],
+    strict: bool = True,
+) -> bytes: ...
+```
+
+Serializes a Python dictionary into binary bytes corresponding to a C struct.
+
+- `parser`: A `CStructParser` instance on which `parse()` has been called.
+- `struct_name`: The target struct type name.
+- `data_dict`: A dictionary mapping field names to field values.
+- `strict`: If `True` (default), raises `ValueError` when non-bitfield fields are missing; if `False`, missing fields are silently filled with 0.
+
+### `unpack(parser, struct_name, data)`
+
+```python
+def unpack(
+    parser: CStructParser,
+    struct_name: str,
+    data: bytes,
+) -> Dict[str, Any]: ...
+```
+
+Deserializes raw binary bytes into a nested Python dictionary.
+
+- `parser`: A `CStructParser` instance on which `parse()` has been called.
+- `struct_name`: The target struct type name.
+- `data`: The binary byte sequence to decode.
+
+## Supported C Type Mapping
+
+| C Type | ctypes Type |
+|--------|-------------|
+| `char`, `signed char` | `c_byte` |
+| `unsigned char` | `c_ubyte` |
+| `short`, `short int` | `c_short` |
+| `unsigned short` | `c_ushort` |
+| `int`, `signed int` | `c_int` |
+| `unsigned int` | `c_uint` |
+| `long`, `long int` | `c_long` |
+| `unsigned long` | `c_ulong` |
+| `long long`, `long long int` | `c_longlong` |
+| `unsigned long long` | `c_ulonglong` |
+| `float` | `c_float` |
+| `double` | `c_double` |
+| `long double` | `c_longdouble` |
+| `int8_t` .. `uint64_t` | `c_int8` .. `c_uint64` |
+| `float32_t` | `c_float` |
+| `float64_t` | `c_double` |
+
+## Comparison with Common Approaches
+
+| Feature | `struct` module | `ctypes.Structure` | `construct` library | cStructDataParser |
+|---------|:---:|:---:|:---:|:---:|
+| Direct parsing from header text | ❌ | ❌ | ❌ | ✅ |
+| Automatic `#pragma pack` alignment | ❌ | ❌ | ❌ | ✅ |
+| Bitfield support | ❌ | ✅ | ✅ | ✅ |
+| Nested structs | ❌ | Manual writing required | ✅ | ✅ |
+| No external dependencies (pure Python) | ✅ | ✅ | ✅ | ✅ |
+
+## Known Limitations
+
+- Currently only supports **little-endian** byte order; big-endian scenarios are not yet handled
+- If the device uses big-endian order, additional byte order processing is needed before/after calling `pack()` / `unpack()`
+
+## License
+
+This project is licensed under [GPLv3.0](https://www.gnu.org/licenses/gpl-3.0.en.html).
+[NGC13009/pycstructdataparser_lib.git](https://github.com/NGC13009/pycstructdataparser_lib.git)

{pycstructdataparser_lib-1.0.0 → pycstructdataparser_lib-1.0.2}/ctype_map_def.py

@@ -10,10 +10,6 @@
 import ctypes
 from typing import Any, Dict, List, Optional, Tuple
 
-# ============================================================================
-# Type mapping — C type name → ctypes type
-# ============================================================================
-
 # Character types
 _CHAR_TYPES = {
     'char': ctypes.c_byte,