pydantic-construct 0.1.0__tar.gz

pydantic_construct-0.1.0/PKG-INFO

@@ -0,0 +1,204 @@
+Metadata-Version: 2.4
+Name: pydantic-construct
+Version: 0.1.0
+Summary: Interfaces between construct and Pydantic to allow typed serializing to bytes.
+Keywords: pydantic,construct,binary,serializing
+Author: Jay184
+Author-email: Jay184 <me@jay0.mozmail.com>
+License-Expression: 0BSD
+Requires-Dist: construct>=2.10.70
+Requires-Dist: construct-typing>=0.7.0
+Requires-Dist: pydantic>=2.12.5
+Maintainer: Jay184
+Maintainer-email: Jay184 <me@jay0.mozmail.com>
+Requires-Python: >=3.11
+Project-URL: Issues, https://github.com/Jay184/pydantic-construct/issues
+Project-URL: Repository, https://github.com/Jay184/pydantic-construct.git
+Description-Content-Type: text/markdown
+
+# Pydantic-Construct
+
+**Pydantic-Construct** integrates Pydantic with `construct` to provide **typed binary serialization and parsing** using standard Pydantic models.
+
+Define binary layouts declaratively with type annotations, while keeping Pydantic’s validation and serialization.
+
+---
+
+## Features
+
+* Declarative binary schemas via `Annotated`
+* `model_dump_bytes()` / `model_validate_bytes()`
+* Nested models
+* Computed fields with ordering control
+* Mode-aware field omission (`json`, `python`, `binary`)
+* Async parsing from streams
+
+---
+
+## Installation
+
+```bash
+pip install pydantic-construct
+```
+
+---
+
+## Quick Start
+
+Minimal example:
+
+```python
+from typing import Annotated
+from construct import Int32ul
+from pydantic_construct import ConstructModel
+
+class Model(ConstructModel):
+    x: Annotated[int, Int32ul]
+
+m = Model(x=123)
+
+data = m.model_dump_bytes()
+parsed = Model.model_validate_bytes(data)
+
+assert data == b"\x7B\x00\x00\x00"
+assert parsed.x == 123
+```
+
+With padding (ignored outside binary mode):
+
+```python
+from typing import Annotated
+from pydantic_construct import ConstructModel, OmitInMode
+from construct import Padding, Int32ul
+
+class Model(ConstructModel):
+    x: Annotated[int, Int32ul]
+    pad: Annotated[bytes | None, Padding(4), OmitInMode({"json", "python"})] = None
+```
+
+---
+
+## Core Concepts
+
+### Binary Fields
+
+Each field must define a `construct` type via `Annotated`:
+
+```python
+x: Annotated[int, Int32ul]
+```
+
+---
+
+### Mode-Based Omission
+
+Exclude fields depending on serialization mode:
+
+```python
+from typing import Annotated
+from pydantic_construct import OmitInMode
+from construct import Padding
+
+pad: Annotated[
+    bytes | None,
+    Padding(4),
+    OmitInMode({"json", "python"})
+]
+```
+
+Modes:
+
+* `"python"`
+* `"json"`
+* `"binary"`
+
+---
+
+### Nested Models
+
+```python
+from typing import Annotated
+from pydantic_construct import ConstructModel
+from construct import Int32ul
+
+class Header(ConstructModel):
+    length: Annotated[int, Int32ul]
+
+class Packet(ConstructModel):
+    header: Header
+```
+
+---
+
+### Computed Fields (Binary)
+
+Computed fields can participate in binary serialization if they return `Annotated[..., Construct]`.
+
+```python
+from typing import Annotated
+from pydantic_construct import ConstructModel, binary_after
+from pydantic import computed_field
+from construct import Int32ul
+
+class Example(ConstructModel):
+    x: Annotated[int, Int32ul]
+
+    @computed_field
+    @property
+    @binary_after("x")
+    def checksum(self) -> Annotated[int, Int32ul]:
+        return self.x ^ 0xFFFFFFFF
+```
+
+Positioning:
+
+* `@binary_after("field")`
+* `@binary_before("field")`
+
+---
+
+## API Overview
+
+### Serialize to Binary
+
+```python
+data = model.model_dump_bytes()
+```
+
+---
+
+### Parse from Binary
+
+```python
+model = Model.model_validate_bytes(data)
+```
+
+---
+
+### Async Stream Parsing
+
+```python
+model = await Model.model_validate_reader(reader)
+```
+
+---
+
+## Design Notes
+
+* A `construct.Struct` is generated at class creation time
+* Field order is deterministic and includes computed fields
+* Multiple `ConstructModel` roots in inheritance are disallowed
+
+---
+
+## Constraints
+
+* Every field must define a `construct` type
+* Computed fields must return `Annotated[..., Construct]`
+* Binary layout must be deterministic
+
+---
+
+## License
+
+0BSD

pydantic_construct-0.1.0/README.md

@@ -0,0 +1,186 @@
+# Pydantic-Construct
+
+**Pydantic-Construct** integrates Pydantic with `construct` to provide **typed binary serialization and parsing** using standard Pydantic models.
+
+Define binary layouts declaratively with type annotations, while keeping Pydantic’s validation and serialization.
+
+---
+
+## Features
+
+* Declarative binary schemas via `Annotated`
+* `model_dump_bytes()` / `model_validate_bytes()`
+* Nested models
+* Computed fields with ordering control
+* Mode-aware field omission (`json`, `python`, `binary`)
+* Async parsing from streams
+
+---
+
+## Installation
+
+```bash
+pip install pydantic-construct
+```
+
+---
+
+## Quick Start
+
+Minimal example:
+
+```python
+from typing import Annotated
+from construct import Int32ul
+from pydantic_construct import ConstructModel
+
+class Model(ConstructModel):
+    x: Annotated[int, Int32ul]
+
+m = Model(x=123)
+
+data = m.model_dump_bytes()
+parsed = Model.model_validate_bytes(data)
+
+assert data == b"\x7B\x00\x00\x00"
+assert parsed.x == 123
+```
+
+With padding (ignored outside binary mode):
+
+```python
+from typing import Annotated
+from pydantic_construct import ConstructModel, OmitInMode
+from construct import Padding, Int32ul
+
+class Model(ConstructModel):
+    x: Annotated[int, Int32ul]
+    pad: Annotated[bytes | None, Padding(4), OmitInMode({"json", "python"})] = None
+```
+
+---
+
+## Core Concepts
+
+### Binary Fields
+
+Each field must define a `construct` type via `Annotated`:
+
+```python
+x: Annotated[int, Int32ul]
+```
+
+---
+
+### Mode-Based Omission
+
+Exclude fields depending on serialization mode:
+
+```python
+from typing import Annotated
+from pydantic_construct import OmitInMode
+from construct import Padding
+
+pad: Annotated[
+    bytes | None,
+    Padding(4),
+    OmitInMode({"json", "python"})
+]
+```
+
+Modes:
+
+* `"python"`
+* `"json"`
+* `"binary"`
+
+---
+
+### Nested Models
+
+```python
+from typing import Annotated
+from pydantic_construct import ConstructModel
+from construct import Int32ul
+
+class Header(ConstructModel):
+    length: Annotated[int, Int32ul]
+
+class Packet(ConstructModel):
+    header: Header
+```
+
+---
+
+### Computed Fields (Binary)
+
+Computed fields can participate in binary serialization if they return `Annotated[..., Construct]`.
+
+```python
+from typing import Annotated
+from pydantic_construct import ConstructModel, binary_after
+from pydantic import computed_field
+from construct import Int32ul
+
+class Example(ConstructModel):
+    x: Annotated[int, Int32ul]
+
+    @computed_field
+    @property
+    @binary_after("x")
+    def checksum(self) -> Annotated[int, Int32ul]:
+        return self.x ^ 0xFFFFFFFF
+```
+
+Positioning:
+
+* `@binary_after("field")`
+* `@binary_before("field")`
+
+---
+
+## API Overview
+
+### Serialize to Binary
+
+```python
+data = model.model_dump_bytes()
+```
+
+---
+
+### Parse from Binary
+
+```python
+model = Model.model_validate_bytes(data)
+```
+
+---
+
+### Async Stream Parsing
+
+```python
+model = await Model.model_validate_reader(reader)
+```
+
+---
+
+## Design Notes
+
+* A `construct.Struct` is generated at class creation time
+* Field order is deterministic and includes computed fields
+* Multiple `ConstructModel` roots in inheritance are disallowed
+
+---
+
+## Constraints
+
+* Every field must define a `construct` type
+* Computed fields must return `Annotated[..., Construct]`
+* Binary layout must be deterministic
+
+---
+
+## License
+
+0BSD

pydantic_construct-0.1.0/pyproject.toml

@@ -0,0 +1,38 @@
+[project]
+name = "pydantic-construct"
+version = "0.1.0"
+description = "Interfaces between construct and Pydantic to allow typed serializing to bytes."
+keywords = ["pydantic", "construct", "binary", "serializing"]
+authors = [
+  {name = "Jay184", email = "me@jay0.mozmail.com"},
+]
+maintainers = [
+  {name = "Jay184", email = "me@jay0.mozmail.com"},
+]
+license = "0BSD"
+readme = "README.md"
+requires-python = ">=3.11"
+dependencies = [
+    "construct>=2.10.70",
+    "construct-typing>=0.7.0",
+    "pydantic>=2.12.5",
+]
+
+[project.urls]
+Repository = "https://github.com/Jay184/pydantic-construct.git"
+Issues = "https://github.com/Jay184/pydantic-construct/issues"
+
+[dependency-groups]
+dev = [
+    "coverage>=7.13.5",
+    "invoke>=2.2.1",
+    "mypy>=1.19.1",
+    "pytest>=9.0.2",
+    "pytest-randomly>=4.0.1",
+    "hypothesis>=6.151.10",
+    "ruff>=0.15.8",
+]
+
+[build-system]
+requires = ["uv_build>=0.8.12,<0.9.0"]
+build-backend = "uv_build"

pydantic_construct-0.1.0/src/pydantic_construct/__init__.py

@@ -0,0 +1,15 @@
+from .base import (
+    ConstructModel,
+    OmitInMode,
+    Mode,
+    binary_before,
+    binary_after,
+)
+
+__all__ = [
+    "ConstructModel",
+    "OmitInMode",
+    "Mode",
+    "binary_before",
+    "binary_after",
+]

pydantic_construct-0.1.0/src/pydantic_construct/base.py

@@ -0,0 +1,304 @@
+from typing import ClassVar, Any, Self, Literal, Callable, Iterable
+from typing import Annotated, get_origin, get_args
+from typing_extensions import Buffer
+from functools import lru_cache
+from asyncio import StreamReader
+
+import dataclasses
+
+from pydantic import BaseModel, model_serializer, main
+from construct import Construct, Container, Struct
+from pydantic_core.core_schema import SerializerFunctionWrapHandler, SerializationInfo
+
+
+def extract_construct(annotation):
+    """Extract Construct instance from Annotated[...]"""
+    if get_origin(annotation) is Annotated:
+        base_type, *metadata = get_args(annotation)
+
+        for meta in metadata:
+            if isinstance(meta, Construct):
+                return meta
+
+    return None
+
+
+def binary_after(field_name: str):
+    def decorator(func):
+        setattr(func, "__binary_after__", field_name)
+        return func
+    return decorator
+
+
+def binary_before(field_name: str):
+    def decorator(func):
+        setattr(func, "__binary_before__", field_name)
+        return func
+    return decorator
+
+
+Mode = Literal["python", "json", "binary"]
+
+
+@dataclasses.dataclass
+class OmitInMode:
+    modes: set[Mode] = dataclasses.field(default_factory=lambda: {"json"})
+
+    def __init__(self, modes: Mode | Iterable[Mode] = "json"):
+        if isinstance(modes, str):
+            self.modes = {modes}
+        else:
+            self.modes = set(modes)
+
+    def matches(self, current_mode: str) -> bool:
+        return current_mode in self.modes
+
+
+class ConstructModel(BaseModel):
+    # Dynamically generated struct
+    struct: ClassVar[Struct]
+    _computed_subcons: ClassVar[dict[str, Construct]]
+
+    # Cached order for computed fields
+    _binary_final_order: ClassVar[list[str]]
+    _binary_ordered_struct: ClassVar[Struct]
+
+    @model_serializer(mode="wrap")
+    def exclude_omissions(
+        self,
+        handler: SerializerFunctionWrapHandler,
+        info: SerializationInfo,
+    ) -> dict[str, object]:
+        serialized = handler(self)
+        cls = type(self)
+
+        return {
+            name: value
+            for name, value in serialized.items()
+            if not cls._is_omitted_in_mode(name, info.mode)
+        }
+
+    @classmethod
+    def __pydantic_init_subclass__(cls, **kwargs):
+        """Hook for Pydantic subclass initialization."""
+        super().__pydantic_init_subclass__(**kwargs)
+
+        roots = cls._get_construct_roots()
+
+        if len(roots) > 1:
+            raise TypeError(
+                f"{cls.__name__} has multiple ConstructModel roots: "
+                f"{[r.__name__ for r in roots]}. "
+                "This leads to ambiguous binary layout. Use composition instead."
+            )
+
+        subcons = {}
+        computed_subcons = {}
+
+        for name, field in cls.model_fields.items():
+            if cls._is_omitted_in_mode(name, mode="binary"):
+                continue
+
+            construct_type = None
+
+            if isinstance(field.annotation, type) and issubclass(field.annotation, ConstructModel):
+                construct_type = field.annotation.struct
+            else:
+                for meta in field.metadata:
+                    if isinstance(meta, Construct):
+                        construct_type = meta
+                        break
+
+            if construct_type is None:
+                raise TypeError(
+                    f"Field '{name}' must be Annotated with a Construct type"
+                )
+
+            subcons[name] = construct_type
+
+        for name, field in cls.model_computed_fields.items():
+            if cls._is_omitted_in_mode(name, mode="binary"):
+                continue
+
+            construct_type = extract_construct(field.return_type)
+
+            if construct_type is None:
+                raise TypeError(
+                    f"Computed field '{name}' must return Annotated[..., Construct]"
+                )
+
+            computed_subcons[name] = construct_type
+
+        cls.struct = Struct(**subcons)
+        cls._computed_subcons = computed_subcons
+
+        base_order = list(subcons.keys())
+        final_order = base_order.copy()
+
+        for name, cons in computed_subcons.items():
+            func = getattr(cls, name).fget
+            after = getattr(func, "__binary_after__", None)
+            before = getattr(func, "__binary_before__", None)
+
+            if after and before:
+                raise TypeError(f"{name} cannot have both before and after")
+            if after:
+                idx = final_order.index(after) + 1
+            elif before:
+                idx = final_order.index(before)
+            else:
+                idx = len(final_order)
+            final_order.insert(idx, name)
+
+        cls._binary_final_order = final_order
+
+        # Build a cached Struct for serialization
+        ordered_subcons = []
+        for name in final_order:
+            if name in subcons:
+                ordered_subcons.append(name / subcons[name])
+            elif name in computed_subcons:
+                ordered_subcons.append(name / computed_subcons[name])
+
+        cls._binary_ordered_struct = Struct(*ordered_subcons)
+
+    @classmethod
+    def _get_construct_roots(cls: type):
+        roots = set()
+
+        for base in cls.__mro__:
+            if (
+                isinstance(base, type)
+                and issubclass(base, ConstructModel)
+                and base is not ConstructModel
+            ):
+                # Check if this base has a ConstructModel parent (excluding base class)
+                has_construct_parent = any(
+                    issubclass(parent, ConstructModel)
+                    and parent is not ConstructModel
+                    for parent in base.__bases__
+                )
+
+                if not has_construct_parent:
+                    roots.add(base)
+
+        return roots
+
+    @classmethod
+    def _is_omitted_in_mode(cls, name: str, mode: Mode | str) -> bool:
+        print(name, mode)
+        return any(
+            isinstance(meta, OmitInMode) and meta.matches(mode)
+            for meta in cls._get_field_metadata(name)
+        )
+
+    @classmethod
+    @lru_cache
+    def _get_field_metadata(cls, name: str):
+        # Regular field
+        if name in cls.model_fields:
+            return cls.model_fields[name].metadata
+
+        # Computed field
+        if name in cls.model_computed_fields:
+            annotation = cls.model_computed_fields[name].return_type
+            if get_origin(annotation) is Annotated:
+                return get_args(annotation)[1:]
+            return ()
+
+        # Unknown field
+        return ()
+
+    @classmethod
+    def model_validate_bytes(
+        cls,
+        obj: bytes | bytearray | Buffer,
+        *,
+        strict: bool | None = None,
+        extra: main.ExtraValues | None = None,
+        from_attributes: bool | None = None,
+        context: Any | None = None,
+        by_alias: bool | None = None,
+        by_name: bool | None = None,
+    ) -> Self:
+        parsed: Container = cls.struct.parse(obj)
+
+        filtered = {
+            k: v for k, v in dict(parsed).items()
+            if not k.startswith("_")
+        }
+
+        return cls.model_validate(
+            filtered,
+            strict=strict,
+            extra=extra,
+            from_attributes=from_attributes,
+            context=context,
+            by_alias=by_alias,
+            by_name=by_name,
+        )
+
+    def model_dump_bytes(
+        self,
+        *,
+        mode: Literal["json", "python", "binary"] | str = "binary",
+        include: main.IncEx | None = None,
+        exclude: main.IncEx | None = None,
+        context: Any | None = None,
+        by_alias: bool | None = None,
+        exclude_unset: bool = False,
+        exclude_defaults: bool = False,
+        exclude_none: bool = False,
+        exclude_computed_fields: bool = False,
+        round_trip: bool = False,
+        warnings: bool | Literal["none", "warn", "error"] = True,
+        fallback: Callable[[Any], Any] | None = None,
+        serialize_as_any: bool = False,
+    ) -> bytes:
+        data = self.model_dump(
+            mode=mode,
+            include=include,
+            exclude=exclude,
+            context=context,
+            by_alias=by_alias,
+            exclude_unset=exclude_unset,
+            exclude_defaults=exclude_defaults,
+            exclude_none=exclude_none,
+            exclude_computed_fields=exclude_computed_fields,
+            round_trip=round_trip,
+            warnings=warnings,
+            fallback=fallback,
+            serialize_as_any=serialize_as_any,
+        )
+
+        # Only include fields that exist in struct
+        # noinspection PyProtectedMember
+        values = {
+            k: v for k, v in data.items()
+            if k in self.struct._subcons or k in self._computed_subcons
+        }
+
+        return self._binary_ordered_struct.build(values)
+
+    @classmethod
+    async def model_validate_reader(
+        cls,
+        obj: StreamReader,
+        *,
+        strict: bool | None = None,
+        extra: main.ExtraValues | None = None,
+        from_attributes: bool | None = None,
+        context: Any | None = None,
+        by_alias: bool | None = None,
+        by_name: bool | None = None,
+    ) -> Self:
+        data = await obj.readexactly(cls.struct.sizeof())
+        return cls.model_validate_bytes(
+            data,
+            strict=strict,
+            extra=extra,
+            from_attributes=from_attributes,
+            context=context,
+            by_alias=by_alias,
+            by_name=by_name,
+        )