firstclass 0.0.2__tar.gz → 0.0.3__tar.gz

{firstclass-0.0.2 → firstclass-0.0.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: firstclass
-Version: 0.0.2
+Version: 0.0.3
 Summary: dataclass with bytes, ctypes, C struct, JSON and XML serialization
 Requires-Python: >=3.10
 Description-Content-Type: text/markdown
@@ -47,11 +47,24 @@ Packet.sizeof()                    # 36
 | `char_array[N]` | `char x[N]` | N (UTF-8, NUL-padded) |
 | `bytes_t[N]` | `uint8_t x[N]` | N |
 | `array_t[T, N]` | `T x[N]` | N * size(T) |
+| `enum_t[Color, uint8_t]` | `uint8_t x` (+ `enum Color {..}` in header) | storage size |
+| `bitfield[uint8_t, 3]` | `uint8_t x : 3` | shares a storage unit |
 | another `@firstclass` class | `struct Name` | its size |
 
 Values are checked: out of range ints, over-long strings and wrong lengths raise
 `ValueError`; wrong Python types raise `TypeError`. Nothing is truncated or coerced silently.
 
+**Enums.** Int-valued `Enum`/`IntEnum`/`IntFlag` classes. `enum_t[Color, uint8_t]` picks the width;
+a bare `Color` annotation is 32-bit like a C `int` (unsigned unless a value is negative). Values
+are stored, and written to JSON/XML, as ints; reading an unknown value raises `ValueError`.
+`c_header()` emits `enum Color { Color_RED = 1, ... };`.
+
+**Bitfields.** `bitfield[uint8_t, 3]` is 3 bits of a `uint8_t` unit, like `uint8_t x : 3`.
+Neighbouring bitfields with the same storage type share a unit while they fit; otherwise a new
+unit starts. Little-endian fills from the lowest bit, big-endian from the highest (what ctypes
+does). Signed storage (`int8_t`) gives signed fields. Mixing storage types in one run is not
+portable across C compilers; keep a run on one type. No arrays of bitfields.
+
 Raw `ctypes` types work as annotations too (packed, same layout as a `_pack_ = 1` Structure):
 `ctypes.c_int`, `c_uint64`, `c_size_t`, `c_bool`, `c_float`, `c_double`, `c_char`, `c_uint16 * 4`,
 `c_char * 8`. Pointers (`c_void_p`, `POINTER(T)`, `c_char_p`) are stored as the address (an `int`,
@@ -64,6 +77,7 @@ address only means something inside the same process. Sizes follow the platform
 | Method | |
 |---|---|
 | `obj.to_bytes()` / `Cls.from_bytes(data)` | binary; `from_bytes` needs exactly `sizeof()` bytes |
+| `Cls.unpack_from(buffer, offset=0)` | like `struct.unpack_from`: read one object at `offset`, trailing bytes are fine; any buffer type |
 | `Cls.sizeof()` | size in bytes |
 | `obj.to_dict()` / `Cls.from_dict(d)` | JSON-compatible dict (bytes as hex) |
 | `obj.to_json(**kw)` / `Cls.from_json(s)` | `kw` goes to `json.dumps` |
@@ -71,6 +85,33 @@ address only means something inside the same process. Sizes follow the platform
 | `obj.to_ctypes()` / `Cls.from_ctypes(c)` / `Cls.ctypes_type()` | `ctypes.Structure` with matching layout |
 | `Cls.c_header()` | C source for the struct(s), nested ones first |
 
+## Import C/C++ headers
+
+```python
+from firstclass import from_header          # also: firstclass.from_header
+
+t = from_header("protocol.h")               # or from_header(text="struct P { ... };")
+pkt = t.Packet(1, True, ...)                # structs -> @firstclass classes
+t.Color.RED                                 # enums / enum class -> IntEnum
+t.constants["MAX_LEN"]                      # integer #defines, consts, enum members
+t.skipped                                   # {"name": "reason"} for anything not convertible
+```
+
+Understood: `struct`/`class`/`typedef struct`, nested and anonymous-member structs, `enum` and
+`enum class : type` (explicit values, implicit increments, constant expressions), bitfields, arrays
+(multi-dimensional, sizes from `#define`/enum/`const`/`sizeof`), `char[N]` -> `char_array`,
+`uint8_t[N]` -> `bytes_t`, pointers and function pointers (stored as addresses), `typedef`/`using`,
+`#define`/`#if`/`#ifdef`/`#elif` (preset macros with `defines={...}`), `namespace`, `extern "C"`,
+`#pragma pack(push/pop/N)` and `__attribute__((packed))`. Layout follows C: natural alignment
+padding unless the struct is packed; force it with `align=True/False`. Options: `byteorder`,
+`long_size` (size of C `long`; default is this machine's).
+
+Skipped with a reason instead of failing: unions, flexible array members, unnamed padding
+bitfields, inheritance, virtual members, unknown types (`#include`d headers are not read), `pack(2)`
+/`pack(4)`, `aligned(N)`, `long double`, empty structs, anything depending on a skipped struct.
+Plain C enums are 32-bit like C unless `: type` is given. C field names that are Python keywords
+(`class`, `from`, ...) get a trailing underscore (`class_`).
+
 ## Layout options
 
 ```python
@@ -95,7 +136,8 @@ in a comment.
 
 ## Example
 
-`python examples/example.py` shows bytes, JSON, XML, ctypes and the C header from one class.
+`python examples/example.py` defines a class in Python, then imports `examples/protocol.h` (enums, `enum class`,
+bitfields, `#pragma pack` structs) and uses the generated classes.
 
 ## Develop
 

{firstclass-0.0.2 → firstclass-0.0.3}/README.md

@@ -40,11 +40,24 @@ Packet.sizeof()                    # 36
 | `char_array[N]` | `char x[N]` | N (UTF-8, NUL-padded) |
 | `bytes_t[N]` | `uint8_t x[N]` | N |
 | `array_t[T, N]` | `T x[N]` | N * size(T) |
+| `enum_t[Color, uint8_t]` | `uint8_t x` (+ `enum Color {..}` in header) | storage size |
+| `bitfield[uint8_t, 3]` | `uint8_t x : 3` | shares a storage unit |
 | another `@firstclass` class | `struct Name` | its size |
 
 Values are checked: out of range ints, over-long strings and wrong lengths raise
 `ValueError`; wrong Python types raise `TypeError`. Nothing is truncated or coerced silently.
 
+**Enums.** Int-valued `Enum`/`IntEnum`/`IntFlag` classes. `enum_t[Color, uint8_t]` picks the width;
+a bare `Color` annotation is 32-bit like a C `int` (unsigned unless a value is negative). Values
+are stored, and written to JSON/XML, as ints; reading an unknown value raises `ValueError`.
+`c_header()` emits `enum Color { Color_RED = 1, ... };`.
+
+**Bitfields.** `bitfield[uint8_t, 3]` is 3 bits of a `uint8_t` unit, like `uint8_t x : 3`.
+Neighbouring bitfields with the same storage type share a unit while they fit; otherwise a new
+unit starts. Little-endian fills from the lowest bit, big-endian from the highest (what ctypes
+does). Signed storage (`int8_t`) gives signed fields. Mixing storage types in one run is not
+portable across C compilers; keep a run on one type. No arrays of bitfields.
+
 Raw `ctypes` types work as annotations too (packed, same layout as a `_pack_ = 1` Structure):
 `ctypes.c_int`, `c_uint64`, `c_size_t`, `c_bool`, `c_float`, `c_double`, `c_char`, `c_uint16 * 4`,
 `c_char * 8`. Pointers (`c_void_p`, `POINTER(T)`, `c_char_p`) are stored as the address (an `int`,
@@ -57,6 +70,7 @@ address only means something inside the same process. Sizes follow the platform
 | Method | |
 |---|---|
 | `obj.to_bytes()` / `Cls.from_bytes(data)` | binary; `from_bytes` needs exactly `sizeof()` bytes |
+| `Cls.unpack_from(buffer, offset=0)` | like `struct.unpack_from`: read one object at `offset`, trailing bytes are fine; any buffer type |
 | `Cls.sizeof()` | size in bytes |
 | `obj.to_dict()` / `Cls.from_dict(d)` | JSON-compatible dict (bytes as hex) |
 | `obj.to_json(**kw)` / `Cls.from_json(s)` | `kw` goes to `json.dumps` |
@@ -64,6 +78,33 @@ address only means something inside the same process. Sizes follow the platform
 | `obj.to_ctypes()` / `Cls.from_ctypes(c)` / `Cls.ctypes_type()` | `ctypes.Structure` with matching layout |
 | `Cls.c_header()` | C source for the struct(s), nested ones first |
 
+## Import C/C++ headers
+
+```python
+from firstclass import from_header          # also: firstclass.from_header
+
+t = from_header("protocol.h")               # or from_header(text="struct P { ... };")
+pkt = t.Packet(1, True, ...)                # structs -> @firstclass classes
+t.Color.RED                                 # enums / enum class -> IntEnum
+t.constants["MAX_LEN"]                      # integer #defines, consts, enum members
+t.skipped                                   # {"name": "reason"} for anything not convertible
+```
+
+Understood: `struct`/`class`/`typedef struct`, nested and anonymous-member structs, `enum` and
+`enum class : type` (explicit values, implicit increments, constant expressions), bitfields, arrays
+(multi-dimensional, sizes from `#define`/enum/`const`/`sizeof`), `char[N]` -> `char_array`,
+`uint8_t[N]` -> `bytes_t`, pointers and function pointers (stored as addresses), `typedef`/`using`,
+`#define`/`#if`/`#ifdef`/`#elif` (preset macros with `defines={...}`), `namespace`, `extern "C"`,
+`#pragma pack(push/pop/N)` and `__attribute__((packed))`. Layout follows C: natural alignment
+padding unless the struct is packed; force it with `align=True/False`. Options: `byteorder`,
+`long_size` (size of C `long`; default is this machine's).
+
+Skipped with a reason instead of failing: unions, flexible array members, unnamed padding
+bitfields, inheritance, virtual members, unknown types (`#include`d headers are not read), `pack(2)`
+/`pack(4)`, `aligned(N)`, `long double`, empty structs, anything depending on a skipped struct.
+Plain C enums are 32-bit like C unless `: type` is given. C field names that are Python keywords
+(`class`, `from`, ...) get a trailing underscore (`class_`).
+
 ## Layout options
 
 ```python
@@ -88,7 +129,8 @@ in a comment.
 
 ## Example
 
-`python examples/example.py` shows bytes, JSON, XML, ctypes and the C header from one class.
+`python examples/example.py` defines a class in Python, then imports `examples/protocol.h` (enums, `enum class`,
+bitfields, `#pragma pack` structs) and uses the generated classes.
 
 ## Develop
 

{firstclass-0.0.2 → firstclass-0.0.3}/firstclass/__init__.py

@@ -1,10 +1,13 @@
 from ._core import firstclass
+from ._header import from_header
 from ._specs import (
     array_t,
+    bitfield,
     bool_t,
     bytes_t,
     char_array,
     char_t,
+    enum_t,
     float32_t,
     float64_t,
     int8_t,
@@ -18,9 +21,11 @@ from ._specs import (
 )
 
 __all__ = [
-    "firstclass",
+    "firstclass", "from_header",
     "uint8_t", "uint16_t", "uint32_t", "uint64_t",
     "int8_t", "int16_t", "int32_t", "int64_t",
     "float32_t", "float64_t", "bool_t", "char_t",
-    "char_array", "bytes_t", "array_t",
+    "char_array", "bytes_t", "array_t", "bitfield", "enum_t",
 ]
+
+firstclass.from_header = from_header  # also reachable as the decorator's attribute

{firstclass-0.0.2 → firstclass-0.0.3}/firstclass/_core.py

@@ -8,10 +8,10 @@ import sys
 import typing
 import xml.etree.ElementTree as ET
 
-from ._specs import ArrayOf, Nested, Spec, resolve_spec
+from ._specs import ArrayOf, BitField, EnumSpec, Nested, Spec, resolve_spec
 
 _METHODS = (
-    "to_bytes", "from_bytes", "to_dict", "from_dict", "to_json", "from_json",
+    "to_bytes", "from_bytes", "unpack_from", "to_dict", "from_dict", "to_json", "from_json",
     "to_xml", "from_xml", "to_ctypes", "from_ctypes", "ctypes_type", "c_header", "sizeof",
 )
 
@@ -50,16 +50,35 @@ class Schema:
                 )
 
     def layout(self):
+        """-> (items, total_size, widest_alignment).
+
+        An item is ("field", index, offset) or ("bits", storage_spec, offset, [(index, shift, bits)]).
+        """
         if self._layout is None:
             self.require_binary()
-            off, widest, offsets = 0, 1, []
-            for _, spec in self.fields:
+            msb = self.bo == ">"
+            off, widest, items, group = 0, 1, [], None  # group = [storage, used_bits, members]
+            for i, (_, spec) in enumerate(self.fields):
+                if isinstance(spec, BitField):
+                    st, unit = spec.storage, spec.storage.size() * 8
+                    if group is None or group[0].name != st.name or group[1] + spec.bits > unit:
+                        a = st.align() if self.aligned else 1
+                        off = _round_up(off, a)
+                        widest = max(widest, a)
+                        group = [st, 0, []]
+                        items.append(("bits", st, off, group[2]))
+                        off += st.size()
+                    shift = unit - group[1] - spec.bits if msb else group[1]
+                    group[2].append((i, shift, spec.bits))
+                    group[1] += spec.bits
+                    continue
+                group = None
                 a = spec.align() if self.aligned else 1
                 off = _round_up(off, a)
-                offsets.append(off)
+                items.append(("field", i, off))
                 off += spec.size()
                 widest = max(widest, a)
-            self._layout = (offsets, _round_up(off, widest), widest)
+            self._layout = (items, _round_up(off, widest), widest)
         return self._layout
 
     def size(self) -> int:
@@ -81,35 +100,71 @@ class Schema:
         return {f.name: f for f, _ in self.fields}  # small; fields are cached
 
     def _ctx(self, f, exc):
-        return type(exc)(f"{self.cls.__name__}.{f.name}: {exc}")
+        kind = TypeError if isinstance(exc, TypeError) else ValueError  # e.g. UnicodeDecodeError -> ValueError
+        return kind(f"{self.cls.__name__}.{f.name}: {exc}")
 
     # -- bytes ---------------------------------------------------------------
 
     def pack(self, obj) -> bytes:
-        offsets, total, _ = self.layout()
+        items, total, _ = self.layout()
         buf = bytearray(total)
-        for (f, spec), off in zip(self.fields, offsets):
-            try:
-                raw = spec.pack(getattr(obj, f.name), self.bo)
-            except (TypeError, ValueError) as e:
-                raise self._ctx(f, e) from e
-            buf[off:off + len(raw)] = raw
+        order = "little" if self.bo == "<" else "big"
+        for item in items:
+            if item[0] == "field":
+                f, spec = self.fields[item[1]]
+                try:
+                    raw = spec.pack(getattr(obj, f.name), self.bo)
+                except (TypeError, ValueError) as e:
+                    raise self._ctx(f, e) from e
+                buf[item[2]:item[2] + len(raw)] = raw
+                continue
+            _, st, off, members = item
+            unit = 0
+            for i, shift, bits in members:
+                f, spec = self.fields[i]
+                try:
+                    unit |= (spec.check(getattr(obj, f.name)) & ((1 << bits) - 1)) << shift
+                except (TypeError, ValueError) as e:
+                    raise self._ctx(f, e) from e
+            buf[off:off + st.size()] = unit.to_bytes(st.size(), order)
         return bytes(buf)
 
     def unpack(self, buf, off: int = 0):
-        offsets, total, _ = self.layout()
+        items, total, _ = self.layout()
         if len(buf) - off < total:
             raise ValueError(f"{self.cls.__name__} needs {total} bytes, got {len(buf) - off}")
+        order = "little" if self.bo == "<" else "big"
         kw = {}
-        for (f, spec), o in zip(self.fields, offsets):
-            try:
-                kw[f.name] = spec.unpack(buf, off + o, self.bo)
-            except (TypeError, ValueError) as e:
-                raise self._ctx(f, e) from e
+        for item in items:
+            if item[0] == "field":
+                f, spec = self.fields[item[1]]
+                try:
+                    kw[f.name] = spec.unpack(buf, off + item[2], self.bo)
+                except (TypeError, ValueError) as e:
+                    raise self._ctx(f, e) from e
+                continue
+            _, st, o, members = item
+            unit = int.from_bytes(bytes(buf[off + o:off + o + st.size()]), order)
+            for i, shift, bits in members:
+                f, spec = self.fields[i]
+                v = (unit >> shift) & ((1 << bits) - 1)
+                if spec.signed and v >> (bits - 1):
+                    v -= 1 << bits
+                kw[f.name] = v
         return self._build(kw)
 
+    @staticmethod
+    def _as_bytes_like(data):
+        return data if isinstance(data, (bytes, bytearray)) else memoryview(data).cast("B")
+
+    def unpack_from(self, buffer, offset: int = 0):
+        """Like struct.unpack_from: read one object at `offset`; extra bytes after it are fine."""
+        if isinstance(offset, bool) or not isinstance(offset, int) or offset < 0:
+            raise ValueError(f"offset must be a non-negative int, got {offset!r}")
+        return self.unpack(self._as_bytes_like(buffer), offset)
+
     def from_bytes(self, data):
-        data = memoryview(data).cast("B") if not isinstance(data, (bytes, bytearray)) else data
+        data = self._as_bytes_like(data)
         if len(data) != self.size():
             raise ValueError(f"{self.cls.__name__} needs exactly {self.size()} bytes, got {len(data)}")
         return self.unpack(data, 0)
@@ -160,11 +215,15 @@ class Schema:
         if self._ctype is None:
             self.require_binary()
             base = ctypes.LittleEndianStructure if self.bo == "<" else ctypes.BigEndianStructure
-            ns = {"_fields_": [(f.name, spec.ctype(self.bo)) for f, spec in self.fields]}
+            ns = {"_fields_": [
+                (f.name, spec.ctype(self.bo), spec.bits) if isinstance(spec, BitField)
+                else (f.name, spec.ctype(self.bo))
+                for f, spec in self.fields
+            ]}
             if not self.aligned:
                 ns["_pack_"] = 1
-                if sys.version_info >= (3, 14):
-                    ns["_layout_"] = "ms"
+            if sys.version_info >= (3, 14):
+                ns["_layout_"] = "ms"  # our bitfield rules; also silences the _pack_ warning
             self._ctype = type(self.cls.__name__ + "_c", (base,), ns)
         return self._ctype
 
@@ -197,6 +256,12 @@ class Schema:
             seen.add(schema.cls)
             for dep in schema._nested_schemas():
                 emit(dep)
+            for _, spec in schema.fields:
+                while isinstance(spec, ArrayOf):
+                    spec = spec.elem
+                if isinstance(spec, EnumSpec) and spec.cls not in seen:
+                    seen.add(spec.cls)
+                    out.append(spec.c_definition())
             lines = [f"/* byteorder: {schema.byteorder} */"]
             if not schema.aligned:
                 lines.append("#pragma pack(push, 1)")
@@ -219,6 +284,8 @@ def firstclass(cls=None, /, *, byteorder: str = "little", align: bool = False, *
     """
 
     def wrap(cls):
+        if not isinstance(cls, type):
+            raise TypeError(f"@firstclass needs a class, got {type(cls).__name__}")
         dc = dataclasses.dataclass(cls, **dataclass_kwargs)
         clash = [f.name for f in dataclasses.fields(dc) if f.name in _METHODS or f.name == "__firstclass__"]
         if clash:
@@ -227,12 +294,13 @@ def firstclass(cls=None, /, *, byteorder: str = "little", align: bool = False, *
         dc.__firstclass__ = s
         dc.to_bytes = lambda self: s.pack(self)
         dc.from_bytes = staticmethod(s.from_bytes)
+        dc.unpack_from = staticmethod(s.unpack_from)
         dc.to_dict = lambda self: s.to_dict(self)
         dc.from_dict = staticmethod(s.from_dict)
         dc.to_json = lambda self, **kw: json.dumps(s.to_dict(self), **kw)
         dc.from_json = staticmethod(lambda text: s.from_dict(json.loads(text)))
         dc.to_xml = lambda self, indent=None: _to_xml(s, self, indent)
-        dc.from_xml = staticmethod(lambda text: s.read_xml(ET.fromstring(text)))
+        dc.from_xml = staticmethod(lambda text: s.read_xml(_parse_xml(text)))
         dc.to_ctypes = lambda self: s.to_ctypes(self)
         dc.from_ctypes = staticmethod(s.from_ctypes)
         dc.ctypes_type = staticmethod(s.ctypes_type)
@@ -243,6 +311,13 @@ def firstclass(cls=None, /, *, byteorder: str = "little", align: bool = False, *
     return wrap if cls is None else wrap(cls)
 
 
+def _parse_xml(text):
+    try:
+        return ET.fromstring(text)
+    except ET.ParseError as e:
+        raise ValueError(f"malformed XML: {e}") from e
+
+
 def _to_xml(schema: Schema, obj, indent) -> str:
     root = ET.Element(schema.cls.__name__)
     schema.fill_xml(obj, root)