microspec-py 0.1.1__tar.gz

microspec_py-0.1.1/LICENSE.TXT

@@ -0,0 +1,7 @@
+Copyright 2026 Meehai
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

microspec_py-0.1.1/PKG-INFO

@@ -0,0 +1,97 @@
+Metadata-Version: 2.1
+Name: microspec-py
+Version: 0.1.1
+Summary: microspec: micro protocol parser and data validator from a simple JSON spec
+Home-page: https://gitlab.com/meehai/microspec
+License: MIT
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+License-File: LICENSE.TXT
+Requires-Dist: numpy>=2.2.0
+Requires-Dist: loggez>=0.8
+Provides-Extra: dev
+Requires-Dist: pytest>=8.4; extra == "dev"
+
+# microspec
+
+A micro specification parser and data validator for TCP protocols. The protocol can be defined either inside the code or as a standalone JSON which can be loaded via `Protocol.from_dict(...)`. Then, all the data payloads for all the defined endpoints are simply validated via `protocol.validate_endpoint(endpoint, payload) -> ValidationError | None`.
+
+Usage:
+
+- Via pip: `pip install microspec-py` (the PyPI name is `microspec-py`; the import name is `microspec`)
+- From source code:
+```bash
+git clone https://gitlab.com/meehai/microspec                 # clone the source code
+cd microspec                                                  # go in the cloned directory
+python -m venv .venv && source .venv/bin/activate             # make a virtual env, optional but useful
+python -m pip install -e .                                    # install microspec in this virtual env
+python -m pytest test/                                        # run the unit & integration tests to verify installation
+python microspec/microspec.py test/integration/protocol.json  # smoke run: parse + validate the bundled spec
+```
+
+Docs: [meehai.gitlab.io/microspec](https://meehai.gitlab.io/microspec/) — built by
+[`docs/build_docs.sh`](docs/build_docs.sh) (pdoc; no sphinx/config). Build locally with
+`bash docs/build_docs.sh` and open the printed `file://` link.
+
+## Usage
+
+Protocol:
+```json
+{
+  "move": {
+    "input":  {"control_input": {"dtype": "float32", "shape": [6], "range": [-100, 100]}},
+    "output": {"status": {"dtype": "str_enum", "enum": ["move_applied"]}, "new_state": {"dtype": "dict"}}
+  }
+}
+```
+
+```python
+import json
+from microspec import Protocol
+
+# Can also be defined here manually via the `Endpoint`, `Field` classes and `Dtype` enum from the library.
+protocol = Protocol.from_dict(json.load(open("test/integration/protocol.json")), n_max_robots=10)
+
+err = protocol.validate_endpoint("move", {"control_input": [5, 5, 5, 3, 3, 3]})
+if err is not None: # err is of type ValidationError (has .error, .endpoint, .field for context)
+  raise ValueError(f"payload is not valid: {err.endpoint}: {err.error}")
+```
+
+## Spec format
+
+Each command is an `input` / `output` map of `name -> field`. Errors are not per-command: every
+endpoint shares one error shape, declared once as `Protocol`'s `error_field` (default `Field("error",
+Dtype.STR)`).
+
+```json
+{
+  "move": {
+    "input":  {"control_input": {"dtype": "float32", "shape": [6], "range": [-100, 100]}},
+    "output": {"status": {"dtype": "str_enum", "enum": ["move_applied"]}, "new_state": {"dtype": "dict"}}
+  },
+  "robot_get_state": {
+    "input":  {"robot_ix": {"dtype": "int32", "range": [0, "${n_max_robots}"]}},
+    "output": {"robot": {"dtype": "dict"}}
+  }
+}
+```
+
+## Field schema
+
+| key       | applies to            | meaning                                                        |
+|-----------|-----------------------|----------------------------------------------------------------|
+| `dtype`   | required              | `str` `int32` `float32` `bool` `dict` `bytes` `str_enum` `int_enum` |
+| `shape`   | array dtypes          | e.g. `[6]`; `null` = free first axis (`[null, 6]`)             |
+| `range`   | `int32` / `float32`   | `[min, max]`, inclusive (NaN always rejected; ±Inf only if outside the range) |
+| `enum`    | `str_enum` / `int_enum` | non-empty list of allowed values (required for enum dtypes)  |
+| `min_len` | arrays with free axis | minimum length of the `null` axis                              |
+| `fields`  | `dict`                | optional nested `name -> field` map; omit for an opaque dict   |
+
+Notes: a `shape` key means "array" (numpy, exact dtype — `np.float32`, not `int64`); scalars have no
+`shape`. A `dict` with `fields` is validated recursively (keys must match exactly, nested arrays are
+list→ndarray converted); without `fields` it is opaque (any dict passes). `${var}` is a single bare
+variable filled at parse time (e.g. `n_max_robots`). The spec file is plain **JSON**.
+
+## Public API
+
+`Protocol`, `Endpoint`, `Field`, `Dtype`, `ValidationError`. Everything else is internal.

microspec_py-0.1.1/README.md

@@ -0,0 +1,83 @@
+# microspec
+
+A micro specification parser and data validator for TCP protocols. The protocol can be defined either inside the code or as a standalone JSON which can be loaded via `Protocol.from_dict(...)`. Then, all the data payloads for all the defined endpoints are simply validated via `protocol.validate_endpoint(endpoint, payload) -> ValidationError | None`.
+
+Usage:
+
+- Via pip: `pip install microspec-py` (the PyPI name is `microspec-py`; the import name is `microspec`)
+- From source code:
+```bash
+git clone https://gitlab.com/meehai/microspec                 # clone the source code
+cd microspec                                                  # go in the cloned directory
+python -m venv .venv && source .venv/bin/activate             # make a virtual env, optional but useful
+python -m pip install -e .                                    # install microspec in this virtual env
+python -m pytest test/                                        # run the unit & integration tests to verify installation
+python microspec/microspec.py test/integration/protocol.json  # smoke run: parse + validate the bundled spec
+```
+
+Docs: [meehai.gitlab.io/microspec](https://meehai.gitlab.io/microspec/) — built by
+[`docs/build_docs.sh`](docs/build_docs.sh) (pdoc; no sphinx/config). Build locally with
+`bash docs/build_docs.sh` and open the printed `file://` link.
+
+## Usage
+
+Protocol:
+```json
+{
+  "move": {
+    "input":  {"control_input": {"dtype": "float32", "shape": [6], "range": [-100, 100]}},
+    "output": {"status": {"dtype": "str_enum", "enum": ["move_applied"]}, "new_state": {"dtype": "dict"}}
+  }
+}
+```
+
+```python
+import json
+from microspec import Protocol
+
+# Can also be defined here manually via the `Endpoint`, `Field` classes and `Dtype` enum from the library.
+protocol = Protocol.from_dict(json.load(open("test/integration/protocol.json")), n_max_robots=10)
+
+err = protocol.validate_endpoint("move", {"control_input": [5, 5, 5, 3, 3, 3]})
+if err is not None: # err is of type ValidationError (has .error, .endpoint, .field for context)
+  raise ValueError(f"payload is not valid: {err.endpoint}: {err.error}")
+```
+
+## Spec format
+
+Each command is an `input` / `output` map of `name -> field`. Errors are not per-command: every
+endpoint shares one error shape, declared once as `Protocol`'s `error_field` (default `Field("error",
+Dtype.STR)`).
+
+```json
+{
+  "move": {
+    "input":  {"control_input": {"dtype": "float32", "shape": [6], "range": [-100, 100]}},
+    "output": {"status": {"dtype": "str_enum", "enum": ["move_applied"]}, "new_state": {"dtype": "dict"}}
+  },
+  "robot_get_state": {
+    "input":  {"robot_ix": {"dtype": "int32", "range": [0, "${n_max_robots}"]}},
+    "output": {"robot": {"dtype": "dict"}}
+  }
+}
+```
+
+## Field schema
+
+| key       | applies to            | meaning                                                        |
+|-----------|-----------------------|----------------------------------------------------------------|
+| `dtype`   | required              | `str` `int32` `float32` `bool` `dict` `bytes` `str_enum` `int_enum` |
+| `shape`   | array dtypes          | e.g. `[6]`; `null` = free first axis (`[null, 6]`)             |
+| `range`   | `int32` / `float32`   | `[min, max]`, inclusive (NaN always rejected; ±Inf only if outside the range) |
+| `enum`    | `str_enum` / `int_enum` | non-empty list of allowed values (required for enum dtypes)  |
+| `min_len` | arrays with free axis | minimum length of the `null` axis                              |
+| `fields`  | `dict`                | optional nested `name -> field` map; omit for an opaque dict   |
+
+Notes: a `shape` key means "array" (numpy, exact dtype — `np.float32`, not `int64`); scalars have no
+`shape`. A `dict` with `fields` is validated recursively (keys must match exactly, nested arrays are
+list→ndarray converted); without `fields` it is opaque (any dict passes). `${var}` is a single bare
+variable filled at parse time (e.g. `n_max_robots`). The spec file is plain **JSON**.
+
+## Public API
+
+`Protocol`, `Endpoint`, `Field`, `Dtype`, `ValidationError`. Everything else is internal.

microspec_py-0.1.1/microspec/__init__.py

@@ -0,0 +1,4 @@
+"""init file"""
+from .microspec import ValidationError, Dtype, Field, Endpoint, Protocol
+
+__all__ = ["ValidationError", "Dtype", "Field", "Endpoint", "Protocol"]

microspec_py-0.1.1/microspec/microspec.py

@@ -0,0 +1,386 @@
+#!/usr/bin/env python3
+"""microspec.py - Micro protocol parser and data validator from a simple JSON-based specification."""
+from __future__ import annotations
+import math
+from dataclasses import dataclass
+from enum import StrEnum
+from typing import Any
+import numpy as np
+
+MAX_SHAPE = 1_000_000_000
+
+@dataclass
+class Endpoint:
+    """An endpoint of the protocol. It has two sets of fields: input and output. Each field has a schema (dtype, etc)"""
+    name: str
+    input: dict[str, Field]
+    output: dict[str, Field]
+    description: str
+
+    def to_dict(self) -> dict:
+        """dict representation of this endpoint"""
+        return {"input": {k: v.to_dict() for k, v in self.input.items()},
+                "output": {k: v.to_dict() for k, v in self.output.items()},
+                "description": self.description}
+
+    @staticmethod
+    def from_dict(name: str, data: dict, **kwargs) -> Endpoint:
+        """create an endpoint from dict data"""
+        return _parse_endpoint(name, endpoint_data=data, **kwargs)
+
+    def __hash__(self):
+        return hash(self.name)
+
+    def __repr__(self):
+        return f"{self.name}({','.join(self.input.keys())}) -> ({','.join(self.output.keys())})"
+
+@dataclass
+class Field:
+    """A field has a single required parameters (dtype) and a bunch of optional ones, some required based on dtype"""
+    dtype: Dtype
+    shape: tuple[int | None, *tuple[int, ...]] | None = None  # (1, 2, 3, ) or (None, 1, 2, 3., ..),
+                                                              # valid for float32, int32, bool, str
+    range: tuple[int, int] | None = None                      # valid for float32, int32
+    enum: list[str] | list[int] | None = None                 # valid for int_enum or str_enum
+    min_len: int | None = None                                # valid only if shape[0] is None (e.g. batched array)
+    fields: dict[str, Field] | None = None                    # valid only for dict and is optional for opaque dicts
+
+    def to_dict(self) -> dict:
+        """dict representation of this field"""
+        res = {"dtype": self.dtype.value}
+        if self.shape is not None:
+            res["shape"] = list(self.shape)  # tuple -> list so from_dict(to_dict()) round-trips (parser wants a list)
+        if self.range is not None:
+            res["range"] = self.range
+        if self.enum is not None:
+            res["enum"] = self.enum
+        if self.min_len is not None:
+            res["min_len"] = self.min_len
+        if self.fields is not None:
+            res["fields"] = {k: v.to_dict() for k, v in self.fields.items()}
+        return res
+
+class Dtype(StrEnum):
+    """All allowed data types by this protocol"""
+    STR      = "str"
+    INT32    = "int32"
+    FLOAT32  = "float32"
+    BOOL     = "bool"
+    DICT     = "dict"
+    BYTES    = "bytes"
+    STR_ENUM = "str_enum"
+    INT_ENUM = "int_enum"
+
+@dataclass
+class ValidationError:
+    """Validation error is a dataclass for returning errors from validation (field + data + error msg)"""
+    field: Field | None
+    data: Any
+    error: str
+    endpoint: Endpoint | None = None
+
+class ParseError(ValueError):
+    """custom exception for parsing errors"""
+
+# parsing
+
+def _interp(item: str | Any, **kwargs) -> Any:
+    """value interpolation from ${variable_name} to an actual value provided via kwargs"""
+    if isinstance(item, str) and item[0:2] == "${" and item[-1] == "}":
+        if item[2:-1] not in kwargs:
+            raise ParseError(f"{item} not in {kwargs=}")
+        return kwargs[item[2:-1]]
+    return item
+
+def _parse_field(name: str, field_data: dict[str, Any], **kwargs) -> Field:
+    if "dtype" not in field_data:
+        raise ParseError(f"{name=}: 'dtype' not in {list(field_data.keys())}")
+    res = {}
+    for k, v in field_data.items():
+        if k == "dtype":
+            res[k] = Dtype(_interp(v, **kwargs))
+
+        elif k == "shape":
+            # shape can be of the following structure: (1, 2, 3) or (None, 1, 2, 3)
+            if res["dtype"] not in (Dtype.INT32, Dtype.FLOAT32, Dtype.STR, Dtype.BOOL, Dtype.INT_ENUM, Dtype.STR_ENUM):
+                raise ParseError(f"{name=} {k=}: invalid dtype for shape: {res['dtype']}")
+            if not (isinstance(v, list) and len(v) > 0):
+                raise ParseError(f"{name=} {k=}: must be a non-empty list: {v}")
+            v_interp = [_interp(_v, **kwargs) for _v in v]
+            if not isinstance(v[0], (int, type(None))):
+                raise ParseError(f"{name=} {k=}: first dim must be int or None: {v_interp}")
+            v_checked = v_interp[1:] if v[0] is None else v_interp # check all if v[0] is not None otherwise >=1 dim
+            if any(not isinstance(_v, int) or isinstance(_v, bool) or _v <= 0 or _v >= MAX_SHAPE for _v in v_checked):
+                raise ParseError(f"{name=} {k=}: dims must be positive ints < {MAX_SHAPE}: {v_interp}")
+            res[k] = tuple(v_interp)
+
+        elif k == "enum":
+            if res["dtype"] not in (Dtype.INT_ENUM, Dtype.STR_ENUM):
+                raise ParseError(f"{name=} {k=}: invalid dtype for enum: {res['dtype']}")
+            if isinstance(v, str): # for the special case of enum: "${data}" -> enum: [data1, data2,...]
+                v = _interp(v, **kwargs)
+            if not isinstance(v, list):
+                raise ParseError(f"{name=} {k=}: must be a list: {v}")
+            v_interp = [_interp(_v, **kwargs) for _v in v]
+            if len(v_interp) == 0:
+                raise ParseError(f"{name=} {k=}: must be a non-empty list: {v_interp}")
+            if res["dtype"] == Dtype.INT_ENUM:
+                if not all(isinstance(_v, int) and not isinstance(_v, bool) for _v in v_interp):
+                    raise ParseError(f"{name=} {k=}: all values must be ints: {v_interp}")
+            else: # Dtype.STR_ENUM
+                if not all(isinstance(_v, str) for _v in v_interp):
+                    raise ParseError(f"{name=} {k=}: all values must be strs: {v_interp}")
+            res[k] = v_interp
+
+        elif k == "range":
+            if res["dtype"] not in (Dtype.INT32, Dtype.FLOAT32):
+                raise ParseError(f"{name=} {k=}: invalid dtype for range: {res['dtype']}")
+            if not isinstance(v, list):
+                raise ParseError(f"{name=} {k=}: must be a list: {v}")
+            if len(v) != 2: # TODO: or of array shape so we can do range [[-1000, 1000], [-500, 500]] for (2, ) array.
+                raise ParseError(f"{name=} {k=}: must be of length 2: {v}")
+            v_interp = [_interp(_v, **kwargs) for _v in v]
+            if not v_interp[0] <= v_interp[1]:
+                raise ParseError(f"{name=} {k=}: low must be <= high: {v_interp}")
+            if res["dtype"] == Dtype.INT32:
+                if not all(isinstance(_v, int) for _v in v_interp):
+                    raise ParseError(f"{name=} {k=}: all values must be ints: {v_interp}")
+            else: # float32
+                if not all(isinstance(_v, (int, float)) and not math.isnan(_v) for _v in v_interp):
+                    raise ParseError(f"{name=} {k=}: all values must be non-NaN numbers: {v_interp}")
+            res[k] = v_interp
+
+        elif k == "min_len":
+            if "shape" not in res:
+                raise ParseError(f"{name=} {k=}: 'shape' not in {res}. Maybe define shape before min_len?")
+            if res["shape"][0] is not None:
+                raise ParseError(f"{name=} {k=} First shape is not None ({res['shape']})")
+            v_interp = _interp(v, **kwargs)
+            if not isinstance(v_interp, int) or v_interp <= 0:
+                raise ParseError(f"{name=} {k=} Must be a positive integer: {v_interp}")
+            res[k] = v_interp
+
+        elif k == "fields":
+            if res["dtype"] != Dtype.DICT:
+                raise ParseError(f"{name=} {k=}: invalid dtype for fields: {res['dtype']}")
+            if not isinstance(v, dict):
+                raise TypeError(f"{name=} {k=}: Expected dict, got {type(v)}")
+            if len(v) == 0:
+                raise ParseError(f"{name=} {k=}: 0 fields provided. For opaque dicts, just don't provide 'fields' key")
+            res[k] = {k2: _parse_field(name=k2, field_data=v2, **kwargs) for k2, v2 in v.items()}
+
+        else:
+            raise NotImplementedError(k, v)
+
+    # checking dtype invariants
+    if res["dtype"] == Dtype.INT_ENUM and "enum" not in res:
+        raise ParseError(f"{name=}: int_enum requires 'enum': {res}")
+    if res["dtype"] == Dtype.STR_ENUM and "enum" not in res:
+        raise ParseError(f"{name=}: str_enum requires 'enum': {res}")
+
+    return Field(**res)
+
+def _parse_endpoint(name: str, endpoint_data: dict[str, Any], **kwargs) -> Endpoint:
+    if endpoint_data.keys() != (expected := {"input", "output", "description"}):
+        raise ParseError(f"{name=}: endpoint must have keys {expected}, got {list(endpoint_data.keys())}")
+    res = {"name": name, "input": {}, "output": {}, "description": endpoint_data["description"]}
+    for ioe in ["input", "output"]:
+        for k, v in endpoint_data[ioe].items():
+            res[ioe][k] = _parse_field(name=k, field_data=v, **kwargs)
+    return Endpoint(**res)
+
+# validation
+
+def _validate_array_and_convert_to_numpy_if_needed(field: Field, data: Any) \
+        -> tuple[Any | np.ndarray, ValidationError | None]:
+    """Converts a data item to numpy array if the field is arr. Returns tuple: converted (or not) data + error/None"""
+    if field.shape is None: # not an array
+        return data, None
+
+    if field.dtype in (Dtype.STR, Dtype.STR_ENUM):
+        dtype = "object"
+    elif field.dtype == Dtype.INT_ENUM:
+        dtype = "int32"
+    else: # int32, float32, bool, etc.
+        dtype = field.dtype
+
+    try:
+        arr_data = np.array(data, dtype=dtype)
+    except (ValueError, TypeError) as e:
+        return data, ValidationError(None, data, error=f"Exception at array conversion: {str(e)}")
+    return arr_data, None
+
+def _validate_array(field: Field, data: np.ndarray) -> ValidationError | None:
+    """validates a field that is known to be an numpy array"""
+    if not isinstance(data, np.ndarray) or len(data.shape) == 0:
+        _type = type(data.item() if isinstance(data, np.ndarray) and len(data.shape) == 0 else data)
+        return ValidationError(field, data, error=f"Expected np.ndarray, got {_type}")
+
+    if field.dtype in (Dtype.STR, Dtype.STR_ENUM):
+        if data.dtype != "object":
+            return ValidationError(field, data, error=f"Expected dtype=object, got {data.dtype}.")
+    elif field.dtype == Dtype.INT_ENUM:
+        if data.dtype != "int32":
+            return ValidationError(field, data, error=f"Expected dtype=int32, got {data.dtype}.")
+    elif data.dtype != field.dtype:
+        return ValidationError(field, data, error=f"Expected dtype={field.dtype}, got {data.dtype}")
+
+    expected_shape = field.shape if field.shape[0] is not None else (len(data), *field.shape[1:])
+    if data.shape != expected_shape:
+        return ValidationError(field, data, error=f"Expected shape={field.shape}, got {data.shape}")
+
+    if field.min_len is not None and len(data) < field.min_len:
+        return ValidationError(field, data, error=f"Expected min len={field.min_len}, got {len(data)}")
+
+    if field.dtype == Dtype.STR:
+        pass
+
+    elif field.dtype == Dtype.INT32:
+        if field.range is not None:
+            if (data < field.range[0]).any() or (data > field.range[1]).any():
+                return ValidationError(field, data, error=f"Out of range: {data} not in {field.range}")
+
+    elif field.dtype == Dtype.FLOAT32:
+        if np.isnan(data).any():
+            return ValidationError(field, data, error=f"NaN not allowed: {data}")
+        if field.range is not None:
+            if (data < field.range[0]).any() or (data > field.range[1]).any():
+                return ValidationError(field, data, error=f"Out of range: {data} not in {field.range}")
+
+    elif field.dtype == Dtype.BOOL:
+        if np.isin(data, field.enum, invert=True).any():
+            pass
+
+    elif field.dtype in (Dtype.INT_ENUM, Dtype.STR_ENUM):
+        if np.isin(data, field.enum, invert=True).any():
+            return ValidationError(field, data, error=f"One or more of {data} not in expected values: {field.enum}")
+
+    else:
+        raise NotImplementedError(field, data)
+
+    return None
+
+def _validate_field(field: Field, data: Any) -> ValidationError | None:
+    if field.shape is not None: # array types are handled by calling this function recursively again
+        return _validate_array(field, data)
+
+    # guaranteed to be scalars below
+
+    if field.dtype == Dtype.STR:
+        if not isinstance(data, str):
+            return ValidationError(field, data, error=f"Expected {field.dtype.value}, got {type(data)}")
+        return None
+
+    elif field.dtype == Dtype.INT32:
+        if not isinstance(data, int) or isinstance(data, bool): # isinstance(True, int) == True python -_-
+            return ValidationError(field, data, error=f"Expected {field.dtype.value}, got {type(data)}")
+        if field.range is not None:
+            if not field.range[0] <= data <= field.range[1]:
+                return ValidationError(field, data, error=f"Out of range: {data} not in {field.range}")
+        return None
+
+    elif field.dtype == Dtype.FLOAT32:
+        if not isinstance(data, float):
+            return ValidationError(field, data, error=f"Expected {field.dtype.value}, got {type(data)}")
+        if math.isnan(data):
+            return ValidationError(field, data, error=f"NaN not allowed: {data}")
+        if field.range is not None:
+            if not field.range[0] <= data <= field.range[1]:
+                return ValidationError(field, data, error=f"Out of range: {data} not in {field.range}")
+        return None
+
+    elif field.dtype == Dtype.BOOL:
+        if not isinstance(data, bool):
+            return ValidationError(field, data, error=f"Expected {field.dtype.value}, got {type(data)}")
+        return None
+
+    elif field.dtype == Dtype.DICT:
+        if not isinstance(data, dict):
+            return ValidationError(field, data, error=f"Expected {field.dtype.value}, got {type(data)}")
+
+        # recursively validate each field if the dict is not opaque.
+        if field.fields is not None:
+            if (field_keys := field.fields.keys()) != data.keys():
+                return ValidationError(field, data, error=f"Wrong data keys. Expected: {field_keys}, got: {list(data)}")
+
+            for k, nested_field in field.fields.items():
+                data[k], err = _validate_array_and_convert_to_numpy_if_needed(nested_field, data[k])
+                if err is not None:
+                    return err
+
+                if (err := _validate_field(field=nested_field, data=data[k])) is not None:
+                    return err
+        return None
+
+    elif field.dtype == Dtype.BYTES:
+        if not isinstance(data, bytes):
+            return ValidationError(field, data, error=f"Expected {field.dtype.value}, got {type(data)}")
+        return None
+
+    elif field.dtype == Dtype.STR_ENUM:
+        if not isinstance(data, str):
+            return ValidationError(field, data, error=f"Expected {field.dtype.value}, got {type(data)}")
+        if data not in field.enum:
+            return ValidationError(field, data, error=f"Data {data} not in expected values: {field.enum}")
+        return None
+
+    elif field.dtype == Dtype.INT_ENUM:
+        if not isinstance(data, int) or isinstance(data, bool): # isinstance(True, int) == True python -_-
+            return ValidationError(field, data, error=f"Expected {field.dtype.value}, got {type(data)}")
+        if data not in field.enum:
+            return ValidationError(field, data, error=f"Data {data} not in expected values: {field.enum}")
+        return None
+
+    else:
+        raise NotImplementedError(field, data)
+
+def _validate_endpoint(endpoint: Endpoint, data: dict[str, Any], mode: str) -> ValidationError | None:
+    """validates and endpoint call with some data"""
+    if mode not in ("input", "output"):
+        raise ValueError(f"Unknown mode: {mode}. Expected 'input' or 'output'")
+    if not isinstance(data, dict):
+        return ValidationError(field=None, data=data, error=f"Expected dict, got {type(data)}", endpoint=endpoint)
+    endpoint_fields = endpoint.input if mode == "input" else endpoint.output
+    # in robosim we get "cmd" as part of the message, but that's not part of the spec, so we need it to not exact match
+    if (diff := (names := set(endpoint_fields.keys())).difference(data.keys())) != set():
+        return ValidationError(field=None, data=data, endpoint=endpoint,
+                               error=f"Not all keys provided. Expected: {names}. Missing: {diff}")
+    for k, field in endpoint_fields.items():
+        data[k], err = _validate_array_and_convert_to_numpy_if_needed(field, data[k])
+        if err is not None:
+            err.endpoint = endpoint
+            return err
+
+        if (err := _validate_field(field, data[k])) is not None:
+            err.endpoint = endpoint
+            return err
+    return None
+
+class Protocol:
+    """The protocol class. A protocol is a dict of endpoints, each with some fields that follow some schema/rules"""
+    def __init__(self, endpoints: dict[str, Endpoint], error_field: Field | None = None):
+        self.endpoints = endpoints
+        self.error_field = error_field or Field(dtype=Dtype.STR)
+
+    def validate_endpoint(self, endpoint: str, data: dict[str, Any], mode: str = "input") -> ValidationError | None:
+        """validates and endpoint call with some data. Note: data is auto-converted to numpy arrays, if array fields!"""
+        check = _validate_endpoint(endpoint=self.endpoints[endpoint], data=data, mode=mode)
+        if check is not None and mode == "output": # maybe the data is not validated because it's an error
+            try:
+                if _validate_field(self.error_field, data.get("error")) is None:
+                    return None
+            except Exception:
+                pass
+        return check
+
+    @staticmethod
+    def from_dict(data: dict[str, Any], **kwargs) -> Protocol:
+        """creates a protocol from a dict"""
+        return Protocol(endpoints={k: Endpoint.from_dict(name=k, data=v, **kwargs) for k, v in data.items()})
+
+    def to_dict(self) -> dict[str, Any]:
+        """saves the protocol as a dict"""
+        return {k: v.to_dict() for k, v in self.endpoints.items()}
+
+    def __repr__(self):
+        return f"[Protocol] Endpoints ({len(self.endpoints)}):\n- {'\n- '.join(map(repr, self.endpoints.values()))}"

microspec_py-0.1.1/microspec_py.egg-info/PKG-INFO

@@ -0,0 +1,97 @@
+Metadata-Version: 2.1
+Name: microspec-py
+Version: 0.1.1
+Summary: microspec: micro protocol parser and data validator from a simple JSON spec
+Home-page: https://gitlab.com/meehai/microspec
+License: MIT
+Requires-Python: >=3.12
+Description-Content-Type: text/markdown
+License-File: LICENSE.TXT
+Requires-Dist: numpy>=2.2.0
+Requires-Dist: loggez>=0.8
+Provides-Extra: dev
+Requires-Dist: pytest>=8.4; extra == "dev"
+
+# microspec
+
+A micro specification parser and data validator for TCP protocols. The protocol can be defined either inside the code or as a standalone JSON which can be loaded via `Protocol.from_dict(...)`. Then, all the data payloads for all the defined endpoints are simply validated via `protocol.validate_endpoint(endpoint, payload) -> ValidationError | None`.
+
+Usage:
+
+- Via pip: `pip install microspec-py` (the PyPI name is `microspec-py`; the import name is `microspec`)
+- From source code:
+```bash
+git clone https://gitlab.com/meehai/microspec                 # clone the source code
+cd microspec                                                  # go in the cloned directory
+python -m venv .venv && source .venv/bin/activate             # make a virtual env, optional but useful
+python -m pip install -e .                                    # install microspec in this virtual env
+python -m pytest test/                                        # run the unit & integration tests to verify installation
+python microspec/microspec.py test/integration/protocol.json  # smoke run: parse + validate the bundled spec
+```
+
+Docs: [meehai.gitlab.io/microspec](https://meehai.gitlab.io/microspec/) — built by
+[`docs/build_docs.sh`](docs/build_docs.sh) (pdoc; no sphinx/config). Build locally with
+`bash docs/build_docs.sh` and open the printed `file://` link.
+
+## Usage
+
+Protocol:
+```json
+{
+  "move": {
+    "input":  {"control_input": {"dtype": "float32", "shape": [6], "range": [-100, 100]}},
+    "output": {"status": {"dtype": "str_enum", "enum": ["move_applied"]}, "new_state": {"dtype": "dict"}}
+  }
+}
+```
+
+```python
+import json
+from microspec import Protocol
+
+# Can also be defined here manually via the `Endpoint`, `Field` classes and `Dtype` enum from the library.
+protocol = Protocol.from_dict(json.load(open("test/integration/protocol.json")), n_max_robots=10)
+
+err = protocol.validate_endpoint("move", {"control_input": [5, 5, 5, 3, 3, 3]})
+if err is not None: # err is of type ValidationError (has .error, .endpoint, .field for context)
+  raise ValueError(f"payload is not valid: {err.endpoint}: {err.error}")
+```
+
+## Spec format
+
+Each command is an `input` / `output` map of `name -> field`. Errors are not per-command: every
+endpoint shares one error shape, declared once as `Protocol`'s `error_field` (default `Field("error",
+Dtype.STR)`).
+
+```json
+{
+  "move": {
+    "input":  {"control_input": {"dtype": "float32", "shape": [6], "range": [-100, 100]}},
+    "output": {"status": {"dtype": "str_enum", "enum": ["move_applied"]}, "new_state": {"dtype": "dict"}}
+  },
+  "robot_get_state": {
+    "input":  {"robot_ix": {"dtype": "int32", "range": [0, "${n_max_robots}"]}},
+    "output": {"robot": {"dtype": "dict"}}
+  }
+}
+```
+
+## Field schema
+
+| key       | applies to            | meaning                                                        |
+|-----------|-----------------------|----------------------------------------------------------------|
+| `dtype`   | required              | `str` `int32` `float32` `bool` `dict` `bytes` `str_enum` `int_enum` |
+| `shape`   | array dtypes          | e.g. `[6]`; `null` = free first axis (`[null, 6]`)             |
+| `range`   | `int32` / `float32`   | `[min, max]`, inclusive (NaN always rejected; ±Inf only if outside the range) |
+| `enum`    | `str_enum` / `int_enum` | non-empty list of allowed values (required for enum dtypes)  |
+| `min_len` | arrays with free axis | minimum length of the `null` axis                              |
+| `fields`  | `dict`                | optional nested `name -> field` map; omit for an opaque dict   |
+
+Notes: a `shape` key means "array" (numpy, exact dtype — `np.float32`, not `int64`); scalars have no
+`shape`. A `dict` with `fields` is validated recursively (keys must match exactly, nested arrays are
+list→ndarray converted); without `fields` it is opaque (any dict passes). `${var}` is a single bare
+variable filled at parse time (e.g. `n_max_robots`). The spec file is plain **JSON**.
+
+## Public API
+
+`Protocol`, `Endpoint`, `Field`, `Dtype`, `ValidationError`. Everything else is internal.

microspec_py-0.1.1/microspec_py.egg-info/SOURCES.txt

@@ -0,0 +1,10 @@
+LICENSE.TXT
+README.md
+setup.py
+microspec/__init__.py
+microspec/microspec.py
+microspec_py.egg-info/PKG-INFO
+microspec_py.egg-info/SOURCES.txt
+microspec_py.egg-info/dependency_links.txt
+microspec_py.egg-info/requires.txt
+microspec_py.egg-info/top_level.txt

microspec_py-0.1.1/microspec_py.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

microspec_py-0.1.1/microspec_py.egg-info/requires.txt

@@ -0,0 +1,5 @@
+numpy>=2.2.0
+loggez>=0.8
+
+[dev]
+pytest>=8.4

microspec_py-0.1.1/microspec_py.egg-info/top_level.txt

@@ -0,0 +1 @@
+microspec

microspec_py-0.1.1/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

microspec_py-0.1.1/setup.py

@@ -0,0 +1,37 @@
+"""setup.py -- note use setuptools==73.0.1; older versions fuck up the data files, newer versions include resources."""
+from pathlib import Path
+from setuptools import setup, find_packages
+
+NAME = "microspec-py"  # 'microspec' is squatted on PyPI; the import name is still `microspec`
+VERSION = "0.1.1"
+DESCRIPTION = "microspec: micro protocol parser and data validator from a simple JSON spec"
+URL = "https://gitlab.com/meehai/microspec"
+
+CWD = Path(__file__).absolute().parent
+with open(CWD/"README.md", "r", encoding="utf-8") as fh:
+    long_description = fh.read()
+
+REQUIRED_CORE = [
+    "numpy>=2.2.0",
+    "loggez>=0.8",
+]
+
+REQUIRED_DEV = [
+    "pytest>=8.4",
+]
+
+setup(
+    name=NAME,
+    version=VERSION,
+    description=DESCRIPTION,
+    long_description=long_description,
+    long_description_content_type="text/markdown",
+    url=URL,
+    packages=find_packages(),
+    install_requires=REQUIRED_CORE,
+    extras_require={"dev": REQUIRED_DEV},
+    dependency_links=[],
+    license="MIT",
+    python_requires=">=3.12",
+    scripts=[], # cli/xxx in the future
+)