structwire 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- structwire-0.1.0/CONTRIBUTING.md +89 -0
- structwire-0.1.0/LICENSE +21 -0
- structwire-0.1.0/MANIFEST.in +10 -0
- structwire-0.1.0/PKG-INFO +148 -0
- structwire-0.1.0/README.md +106 -0
- structwire-0.1.0/RELEASING.md +31 -0
- structwire-0.1.0/SECURITY.md +13 -0
- structwire-0.1.0/docs/API.md +138 -0
- structwire-0.1.0/docs/AUTOSPAN_ADOPTION.md +41 -0
- structwire-0.1.0/docs/COMPATIBILITY.md +92 -0
- structwire-0.1.0/docs/DESIGN.md +51 -0
- structwire-0.1.0/docs/INTEGRATION_MODEL.md +68 -0
- structwire-0.1.0/docs/SCHEMA.md +169 -0
- structwire-0.1.0/examples/automotive_bindings.yaml +36 -0
- structwire-0.1.0/examples/ble_payload.py +14 -0
- structwire-0.1.0/examples/can_like_signal.py +12 -0
- structwire-0.1.0/examples/catalog_demo.py +19 -0
- structwire-0.1.0/examples/codecs.yaml +59 -0
- structwire-0.1.0/examples/diagnostic_record.py +13 -0
- structwire-0.1.0/examples/uart_bindings.yaml +13 -0
- structwire-0.1.0/pyproject.toml +75 -0
- structwire-0.1.0/scripts/check_dist.py +144 -0
- structwire-0.1.0/scripts/check_readme.py +91 -0
- structwire-0.1.0/scripts/verify_install.py +124 -0
- structwire-0.1.0/setup.cfg +4 -0
- structwire-0.1.0/src/structwire/__init__.py +33 -0
- structwire-0.1.0/src/structwire/catalog.py +84 -0
- structwire-0.1.0/src/structwire/codec.py +506 -0
- structwire-0.1.0/src/structwire/errors.py +17 -0
- structwire-0.1.0/src/structwire/formatting.py +38 -0
- structwire-0.1.0/src/structwire/loaders.py +234 -0
- structwire-0.1.0/src/structwire/model.py +330 -0
- structwire-0.1.0/src/structwire/py.typed +0 -0
- structwire-0.1.0/src/structwire.egg-info/PKG-INFO +148 -0
- structwire-0.1.0/src/structwire.egg-info/SOURCES.txt +43 -0
- structwire-0.1.0/src/structwire.egg-info/dependency_links.txt +1 -0
- structwire-0.1.0/src/structwire.egg-info/requires.txt +18 -0
- structwire-0.1.0/src/structwire.egg-info/top_level.txt +1 -0
- structwire-0.1.0/tests/test_check_readme.py +78 -0
- structwire-0.1.0/tests/test_codec.py +607 -0
- structwire-0.1.0/tests/test_compilation.py +138 -0
- structwire-0.1.0/tests/test_examples.py +123 -0
- structwire-0.1.0/tests/test_loaders.py +547 -0
- structwire-0.1.0/tests/test_properties.py +126 -0
- structwire-0.1.0/tests/test_schema.py +204 -0
|
@@ -0,0 +1,89 @@
|
|
|
1
|
+
# Contributing
|
|
2
|
+
|
|
3
|
+
Keep StructWire independent of transports, devices and test frameworks. New features
|
|
4
|
+
should describe binary layout semantics; see [design](docs/DESIGN.md).
|
|
5
|
+
|
|
6
|
+
## Development
|
|
7
|
+
|
|
8
|
+
Create and activate a virtual environment from this checkout (Unix/macOS):
|
|
9
|
+
|
|
10
|
+
```bash
|
|
11
|
+
python3 -m venv .venv
|
|
12
|
+
source .venv/bin/activate
|
|
13
|
+
python -m pip install -e '.[dev]'
|
|
14
|
+
python -m pytest
|
|
15
|
+
python -m ruff check .
|
|
16
|
+
python -m ruff format --check .
|
|
17
|
+
python -m mypy src/structwire
|
|
18
|
+
python -m compileall -q src examples tests scripts
|
|
19
|
+
python scripts/check_readme.py
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
On Windows, create the environment with `py -3 -m venv .venv` and activate it
|
|
23
|
+
in PowerShell with `.venv\Scripts\Activate.ps1`, then use the same `python` commands.
|
|
24
|
+
The README checker renders the PyPI description and checks local documentation
|
|
25
|
+
links and anchors.
|
|
26
|
+
|
|
27
|
+
Codec changes need encode/decode tests, meaningful round trips and malformed-input
|
|
28
|
+
coverage. Document byte order, width and failure behavior. Update the
|
|
29
|
+
[compatibility contract](docs/COMPATIBILITY.md) when changing precision, canonical
|
|
30
|
+
output, or schema validation. Add boundary and differential/property tests for
|
|
31
|
+
numeric changes.
|
|
32
|
+
|
|
33
|
+
Pytest executes the four Python examples and every Python block in the README and
|
|
34
|
+
`docs/`. YAML snippets are parsed and their StructWire schemas compiled; application
|
|
35
|
+
routing examples are checked for valid nested layouts. The example routing files
|
|
36
|
+
are checked against the supplied catalog. Property tests compare integer and float
|
|
37
|
+
bytes with Python's standard library, exercise shared bit fields, and test scaled
|
|
38
|
+
integer conversion with biases. Targeted cases cover malformed inputs, duplicate
|
|
39
|
+
schemas, and precision loss.
|
|
40
|
+
|
|
41
|
+
## Coverage
|
|
42
|
+
|
|
43
|
+
CI reports branch coverage for all `structwire` modules without a percentage gate
|
|
44
|
+
or custom exclusions. To inspect the same report locally:
|
|
45
|
+
|
|
46
|
+
```bash
|
|
47
|
+
python -m coverage run --branch --source=structwire -m pytest
|
|
48
|
+
python -m coverage report -m
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
The report measures the main test process; subprocess example and resource-limit
|
|
52
|
+
checks run separately and are not counted. Review missing behavior, not only the
|
|
53
|
+
percentage. Pathological compilation tests use a memory-limited subprocess on
|
|
54
|
+
Linux; portable allocation and overlap tests also run on the other platforms.
|
|
55
|
+
|
|
56
|
+
## Distribution checks
|
|
57
|
+
|
|
58
|
+
Build into an empty `dist/` directory; move any previous output aside first.
|
|
59
|
+
|
|
60
|
+
```bash
|
|
61
|
+
python -m build
|
|
62
|
+
python -m twine check --strict dist/*.whl dist/*.tar.gz
|
|
63
|
+
python scripts/check_dist.py dist
|
|
64
|
+
python scripts/verify_install.py dist
|
|
65
|
+
python scripts/verify_install.py dist --distribution sdist
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
`build` creates the sdist and builds the wheel from it. The configured build and
|
|
69
|
+
development tools support SPDX license metadata and core metadata version 2.4.
|
|
70
|
+
`check_dist.py` verifies metadata, license/typing files, the single-source version,
|
|
71
|
+
public links, and matching package bytes in both artifacts. It prints artifact hashes.
|
|
72
|
+
|
|
73
|
+
`verify_install.py` creates a fresh environment outside the checkout, installs the
|
|
74
|
+
selected distribution and test dependencies, checks imports and metadata under
|
|
75
|
+
`python -I`, then runs the full suite from the extracted source distribution.
|
|
76
|
+
Tests exercise the installed package without a source-tree import path.
|
|
77
|
+
|
|
78
|
+
CI runs quality checks and builds once, then tests that wheel on CPython 3.11–3.14
|
|
79
|
+
on Linux, Windows, and macOS. The source installation is tested independently.
|
|
80
|
+
See [releasing](RELEASING.md) for publication using those artifacts.
|
|
81
|
+
|
|
82
|
+
## Source material
|
|
83
|
+
|
|
84
|
+
Contribute only source, schemas and fixtures that you have the right to publish
|
|
85
|
+
under MIT. Prefer synthetic examples and public protocol specifications. Keep
|
|
86
|
+
private/customer payload databases, bench configuration and device secrets out
|
|
87
|
+
of this repository. Customer and project mappings belong in their private plugins
|
|
88
|
+
unless publication rights are explicit. Automated checks cannot establish those
|
|
89
|
+
rights; the domain-neutral API never requires publishing customer protocol data.
|
structwire-0.1.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Nidhal Abidi
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
include CONTRIBUTING.md
|
|
2
|
+
include RELEASING.md
|
|
3
|
+
include SECURITY.md
|
|
4
|
+
recursive-include docs *.md
|
|
5
|
+
recursive-include examples *.py *.yaml
|
|
6
|
+
recursive-include tests *.py
|
|
7
|
+
recursive-include scripts *.py
|
|
8
|
+
global-exclude __pycache__ *.py[cod]
|
|
9
|
+
prune artifacts
|
|
10
|
+
prune .github
|
|
@@ -0,0 +1,148 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: structwire
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: A small schema-driven binary payload codec for Python.
|
|
5
|
+
Author: Nidhal Abidi
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/nidops/structwire
|
|
8
|
+
Project-URL: Documentation, https://github.com/nidops/structwire/blob/master/docs/SCHEMA.md
|
|
9
|
+
Project-URL: Source, https://github.com/nidops/structwire
|
|
10
|
+
Project-URL: Issues, https://github.com/nidops/structwire/issues
|
|
11
|
+
Keywords: binary,codec,schema,embedded,payload,bitfield,yaml
|
|
12
|
+
Classifier: Development Status :: 3 - Alpha
|
|
13
|
+
Classifier: Intended Audience :: Developers
|
|
14
|
+
Classifier: Operating System :: OS Independent
|
|
15
|
+
Classifier: Programming Language :: Python :: 3
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
20
|
+
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
|
21
|
+
Classifier: Typing :: Typed
|
|
22
|
+
Requires-Python: >=3.11
|
|
23
|
+
Description-Content-Type: text/markdown
|
|
24
|
+
License-File: LICENSE
|
|
25
|
+
Requires-Dist: PyYAML>=6.0
|
|
26
|
+
Provides-Extra: test
|
|
27
|
+
Requires-Dist: pytest>=8; extra == "test"
|
|
28
|
+
Requires-Dist: hypothesis>=6; extra == "test"
|
|
29
|
+
Requires-Dist: readme-renderer[md]>=44; extra == "test"
|
|
30
|
+
Provides-Extra: dev
|
|
31
|
+
Requires-Dist: build>=1.2; extra == "dev"
|
|
32
|
+
Requires-Dist: coverage>=7; extra == "dev"
|
|
33
|
+
Requires-Dist: mypy>=1.13; extra == "dev"
|
|
34
|
+
Requires-Dist: pytest>=8; extra == "dev"
|
|
35
|
+
Requires-Dist: hypothesis>=6; extra == "dev"
|
|
36
|
+
Requires-Dist: ruff>=0.8; extra == "dev"
|
|
37
|
+
Requires-Dist: twine>=6.2; extra == "dev"
|
|
38
|
+
Requires-Dist: packaging>=24.2; extra == "dev"
|
|
39
|
+
Requires-Dist: readme-renderer[md]>=44; extra == "dev"
|
|
40
|
+
Requires-Dist: types-PyYAML>=6.0.12; extra == "dev"
|
|
41
|
+
Dynamic: license-file
|
|
42
|
+
|
|
43
|
+
# StructWire
|
|
44
|
+
|
|
45
|
+
StructWire converts fixed-layout binary payloads to Python values and back.
|
|
46
|
+
Use it when a device message, file record or test fixture has a known byte layout.
|
|
47
|
+
Define fields in Python or YAML, then reuse a compiled codec to encode and decode.
|
|
48
|
+
Your application supplies the bytes and handles transport I/O and routing.
|
|
49
|
+
|
|
50
|
+
## Install
|
|
51
|
+
|
|
52
|
+
Python 3.11 or newer is required. Install from the source checkout:
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
git clone https://github.com/nidops/structwire.git
|
|
56
|
+
cd structwire
|
|
57
|
+
python -m pip install .
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
Version `0.1.0` is an alpha API. PyYAML is installed automatically for YAML loading.
|
|
61
|
+
|
|
62
|
+
## Quick start
|
|
63
|
+
|
|
64
|
+
```python
|
|
65
|
+
from structwire import Field, Layout
|
|
66
|
+
|
|
67
|
+
codec = Layout(
|
|
68
|
+
Field("temperature_c", "i16le", scale=0.1),
|
|
69
|
+
Field("state", "u8"),
|
|
70
|
+
).compile()
|
|
71
|
+
|
|
72
|
+
payload = codec.encode({"temperature_c": 23.5, "state": 1})
|
|
73
|
+
assert payload == bytes.fromhex("eb 00 01")
|
|
74
|
+
assert codec.decode(payload) == {"temperature_c": 23.5, "state": 1}
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
Compile a layout once and reuse its codec.
|
|
78
|
+
|
|
79
|
+
## YAML example
|
|
80
|
+
|
|
81
|
+
Save this as `codecs.yaml`:
|
|
82
|
+
|
|
83
|
+
```yaml
|
|
84
|
+
version: 1
|
|
85
|
+
codecs:
|
|
86
|
+
node_status:
|
|
87
|
+
size: 4
|
|
88
|
+
fields:
|
|
89
|
+
- name: mode
|
|
90
|
+
type: enum
|
|
91
|
+
values: {0: asleep, 1: awake, 2: fault}
|
|
92
|
+
- name: supply_voltage
|
|
93
|
+
type: uint16_le
|
|
94
|
+
offset: 2
|
|
95
|
+
scale: 0.001
|
|
96
|
+
unit: V
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
```python
|
|
100
|
+
from structwire import catalog_from_yaml
|
|
101
|
+
|
|
102
|
+
catalog = catalog_from_yaml("codecs.yaml")
|
|
103
|
+
payload = bytes.fromhex("01 00 88 13")
|
|
104
|
+
values = catalog.decode("node_status", payload)
|
|
105
|
+
assert values == {"mode": "awake", "supply_voltage": 5.0}
|
|
106
|
+
assert catalog.encode("node_status", values) == payload
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
Offsets and sizes are in bytes. Here, byte 1 is unused and encodes as zero.
|
|
110
|
+
`unit` is display metadata; it does not change the conversion.
|
|
111
|
+
|
|
112
|
+
## Supported layouts
|
|
113
|
+
|
|
114
|
+
- Signed and unsigned 8/16/24/32/64-bit integers, with explicit byte order.
|
|
115
|
+
- IEEE-754 32/64-bit floats, BCD, fixed ASCII, bytes and MAC addresses.
|
|
116
|
+
- Enums, named flags, booleans and masked sub-fields.
|
|
117
|
+
- Numeric scaling and bias, explicit offsets and optional total payload size.
|
|
118
|
+
- Immutable layouts and reusable compiled codecs, with exact-length decoding by default.
|
|
119
|
+
- Named catalogs and single-layout loaders for YAML or Python mappings.
|
|
120
|
+
|
|
121
|
+
## Limitations
|
|
122
|
+
|
|
123
|
+
Unscaled integers preserve their full range, including 64-bit values. Scaling uses
|
|
124
|
+
Python floats, so some values cannot be represented exactly. Integer encoding
|
|
125
|
+
rejects values between raw counts, allowing small rounding errors; decoding raises
|
|
126
|
+
`DecodeError` if scaling loses the original count. For exact arithmetic, decode
|
|
127
|
+
unscaled integers and convert them in your application.
|
|
128
|
+
|
|
129
|
+
Encoding zeroes unused bytes and bits, encodes booleans as 0 or 1, and writes only
|
|
130
|
+
declared masks from flags mappings. Arbitrary payloads may therefore change after
|
|
131
|
+
decoding and re-encoding. See the
|
|
132
|
+
[compatibility contract](https://github.com/nidops/structwire/blob/master/docs/COMPATIBILITY.md)
|
|
133
|
+
for precise round-trip and floating-point guarantees.
|
|
134
|
+
|
|
135
|
+
Variable-length fields, nested/repeated layouts and checksums are not supported.
|
|
136
|
+
|
|
137
|
+
Schemas are trusted configuration. Safe YAML loading does not impose resource
|
|
138
|
+
limits; applications must bound untrusted schema and payload sizes. See
|
|
139
|
+
[schema resource limits](https://github.com/nidops/structwire/blob/master/docs/SCHEMA.md#trusted-schemas-and-resource-limits).
|
|
140
|
+
|
|
141
|
+
## Documentation and examples
|
|
142
|
+
|
|
143
|
+
- [Python API](https://github.com/nidops/structwire/blob/master/docs/API.md)
|
|
144
|
+
- [Schema reference](https://github.com/nidops/structwire/blob/master/docs/SCHEMA.md)
|
|
145
|
+
- [Examples](https://github.com/nidops/structwire/tree/master/examples)
|
|
146
|
+
- [Questions and bug reports](https://github.com/nidops/structwire/issues)
|
|
147
|
+
|
|
148
|
+
MIT. See [LICENSE](https://github.com/nidops/structwire/blob/master/LICENSE).
|
|
@@ -0,0 +1,106 @@
|
|
|
1
|
+
# StructWire
|
|
2
|
+
|
|
3
|
+
StructWire converts fixed-layout binary payloads to Python values and back.
|
|
4
|
+
Use it when a device message, file record or test fixture has a known byte layout.
|
|
5
|
+
Define fields in Python or YAML, then reuse a compiled codec to encode and decode.
|
|
6
|
+
Your application supplies the bytes and handles transport I/O and routing.
|
|
7
|
+
|
|
8
|
+
## Install
|
|
9
|
+
|
|
10
|
+
Python 3.11 or newer is required. Install from the source checkout:
|
|
11
|
+
|
|
12
|
+
```bash
|
|
13
|
+
git clone https://github.com/nidops/structwire.git
|
|
14
|
+
cd structwire
|
|
15
|
+
python -m pip install .
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
Version `0.1.0` is an alpha API. PyYAML is installed automatically for YAML loading.
|
|
19
|
+
|
|
20
|
+
## Quick start
|
|
21
|
+
|
|
22
|
+
```python
|
|
23
|
+
from structwire import Field, Layout
|
|
24
|
+
|
|
25
|
+
codec = Layout(
|
|
26
|
+
Field("temperature_c", "i16le", scale=0.1),
|
|
27
|
+
Field("state", "u8"),
|
|
28
|
+
).compile()
|
|
29
|
+
|
|
30
|
+
payload = codec.encode({"temperature_c": 23.5, "state": 1})
|
|
31
|
+
assert payload == bytes.fromhex("eb 00 01")
|
|
32
|
+
assert codec.decode(payload) == {"temperature_c": 23.5, "state": 1}
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
Compile a layout once and reuse its codec.
|
|
36
|
+
|
|
37
|
+
## YAML example
|
|
38
|
+
|
|
39
|
+
Save this as `codecs.yaml`:
|
|
40
|
+
|
|
41
|
+
```yaml
|
|
42
|
+
version: 1
|
|
43
|
+
codecs:
|
|
44
|
+
node_status:
|
|
45
|
+
size: 4
|
|
46
|
+
fields:
|
|
47
|
+
- name: mode
|
|
48
|
+
type: enum
|
|
49
|
+
values: {0: asleep, 1: awake, 2: fault}
|
|
50
|
+
- name: supply_voltage
|
|
51
|
+
type: uint16_le
|
|
52
|
+
offset: 2
|
|
53
|
+
scale: 0.001
|
|
54
|
+
unit: V
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
```python
|
|
58
|
+
from structwire import catalog_from_yaml
|
|
59
|
+
|
|
60
|
+
catalog = catalog_from_yaml("codecs.yaml")
|
|
61
|
+
payload = bytes.fromhex("01 00 88 13")
|
|
62
|
+
values = catalog.decode("node_status", payload)
|
|
63
|
+
assert values == {"mode": "awake", "supply_voltage": 5.0}
|
|
64
|
+
assert catalog.encode("node_status", values) == payload
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
Offsets and sizes are in bytes. Here, byte 1 is unused and encodes as zero.
|
|
68
|
+
`unit` is display metadata; it does not change the conversion.
|
|
69
|
+
|
|
70
|
+
## Supported layouts
|
|
71
|
+
|
|
72
|
+
- Signed and unsigned 8/16/24/32/64-bit integers, with explicit byte order.
|
|
73
|
+
- IEEE-754 32/64-bit floats, BCD, fixed ASCII, bytes and MAC addresses.
|
|
74
|
+
- Enums, named flags, booleans and masked sub-fields.
|
|
75
|
+
- Numeric scaling and bias, explicit offsets and optional total payload size.
|
|
76
|
+
- Immutable layouts and reusable compiled codecs, with exact-length decoding by default.
|
|
77
|
+
- Named catalogs and single-layout loaders for YAML or Python mappings.
|
|
78
|
+
|
|
79
|
+
## Limitations
|
|
80
|
+
|
|
81
|
+
Unscaled integers preserve their full range, including 64-bit values. Scaling uses
|
|
82
|
+
Python floats, so some values cannot be represented exactly. Integer encoding
|
|
83
|
+
rejects values between raw counts, allowing small rounding errors; decoding raises
|
|
84
|
+
`DecodeError` if scaling loses the original count. For exact arithmetic, decode
|
|
85
|
+
unscaled integers and convert them in your application.
|
|
86
|
+
|
|
87
|
+
Encoding zeroes unused bytes and bits, encodes booleans as 0 or 1, and writes only
|
|
88
|
+
declared masks from flags mappings. Arbitrary payloads may therefore change after
|
|
89
|
+
decoding and re-encoding. See the
|
|
90
|
+
[compatibility contract](https://github.com/nidops/structwire/blob/master/docs/COMPATIBILITY.md)
|
|
91
|
+
for precise round-trip and floating-point guarantees.
|
|
92
|
+
|
|
93
|
+
Variable-length fields, nested/repeated layouts and checksums are not supported.
|
|
94
|
+
|
|
95
|
+
Schemas are trusted configuration. Safe YAML loading does not impose resource
|
|
96
|
+
limits; applications must bound untrusted schema and payload sizes. See
|
|
97
|
+
[schema resource limits](https://github.com/nidops/structwire/blob/master/docs/SCHEMA.md#trusted-schemas-and-resource-limits).
|
|
98
|
+
|
|
99
|
+
## Documentation and examples
|
|
100
|
+
|
|
101
|
+
- [Python API](https://github.com/nidops/structwire/blob/master/docs/API.md)
|
|
102
|
+
- [Schema reference](https://github.com/nidops/structwire/blob/master/docs/SCHEMA.md)
|
|
103
|
+
- [Examples](https://github.com/nidops/structwire/tree/master/examples)
|
|
104
|
+
- [Questions and bug reports](https://github.com/nidops/structwire/issues)
|
|
105
|
+
|
|
106
|
+
MIT. See [LICENSE](https://github.com/nidops/structwire/blob/master/LICENSE).
|
|
@@ -0,0 +1,31 @@
|
|
|
1
|
+
# Releasing StructWire
|
|
2
|
+
|
|
3
|
+
Package version lives in `src/structwire/__init__.py`; setuptools derives metadata
|
|
4
|
+
from it. Keep this as the single version source.
|
|
5
|
+
|
|
6
|
+
## Setup
|
|
7
|
+
|
|
8
|
+
Configure [trusted publishing](https://docs.pypi.org/trusted-publishers/)
|
|
9
|
+
separately on TestPyPI and PyPI for project `structwire`, repository
|
|
10
|
+
`nidops/structwire`, and workflow `publish.yml`. Use GitHub environments `testpypi`
|
|
11
|
+
and `pypi` respectively, restricted to release tags. For a new project, register
|
|
12
|
+
a pending publisher; this does not reserve the package name. Confirm account
|
|
13
|
+
ownership and public documentation links.
|
|
14
|
+
|
|
15
|
+
## Release
|
|
16
|
+
|
|
17
|
+
1. Update the package version and documentation.
|
|
18
|
+
2. Run the [development checks](CONTRIBUTING.md#development) and
|
|
19
|
+
[distribution checks](CONTRIBUTING.md#distribution-checks).
|
|
20
|
+
3. From a clean checkout, tag the reviewed `master` commit as `vX.Y.Z` after all
|
|
21
|
+
supported CI jobs pass. Verify the version with
|
|
22
|
+
`python scripts/check_dist.py dist --tag vX.Y.Z`, substituting the actual version.
|
|
23
|
+
4. Push the tag, then manually run **Publish** in GitHub Actions with that tag selected.
|
|
24
|
+
|
|
25
|
+
The workflow reruns CI, uploads verified artifacts to TestPyPI, checks the downloaded
|
|
26
|
+
wheel's hash and installation, then publishes the same artifacts to PyPI and verifies
|
|
27
|
+
again. Pushes alone do not publish. Check the package page after completion.
|
|
28
|
+
|
|
29
|
+
If publication fails, rerun failed jobs in the same run while its `release-dist`
|
|
30
|
+
artifact is available. Never overwrite tags or uploaded files, or hide mismatches
|
|
31
|
+
with `skip-existing`. Changed release contents require a new version.
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
# Security policy
|
|
2
|
+
|
|
3
|
+
StructWire parses binary payloads using caller-supplied schemas. Report vulnerabilities
|
|
4
|
+
privately through the repository's security-advisory feature rather than a public issue.
|
|
5
|
+
Only the latest released `0.x` version receives fixes.
|
|
6
|
+
|
|
7
|
+
Schemas are trusted application configuration. YAML loading uses safe constructors
|
|
8
|
+
and rejects duplicate mapping keys, without arbitrary Python object construction;
|
|
9
|
+
it does not limit input size, nesting or resource consumption. Applications own
|
|
10
|
+
access control and resource limits for schemas and payloads. Bound widths, offsets,
|
|
11
|
+
layout sizes and field/table counts before accepting untrusted schemas. Encoding
|
|
12
|
+
still allocates a complete payload even when a sparse layout compiles cheaply.
|
|
13
|
+
See [schema resource limits](docs/SCHEMA.md#trusted-schemas-and-resource-limits).
|
|
@@ -0,0 +1,138 @@
|
|
|
1
|
+
# Python API
|
|
2
|
+
|
|
3
|
+
All names below are exported from `structwire`.
|
|
4
|
+
See the [schema reference](SCHEMA.md) for field kinds and options and the
|
|
5
|
+
[compatibility contract](COMPATIBILITY.md) for precision and canonical encoding.
|
|
6
|
+
|
|
7
|
+
## Fields and layouts
|
|
8
|
+
|
|
9
|
+
`Field(name, kind, ...)` describes one field. Optional arguments are `length`, `offset`, `mask`,
|
|
10
|
+
`shift`, `scale`, `bias`, `values`, `bits`, `byteorder`, `description` and `unit`.
|
|
11
|
+
Python constructors use these canonical names; the YAML key aliases `type`,
|
|
12
|
+
`bitmask`, `bitshift` and `physical_offset` belong to the loaders. Kind aliases
|
|
13
|
+
such as `uint16_le` work in both APIs.
|
|
14
|
+
|
|
15
|
+
`Field.width` reports its storage size in bytes. `Field.effective_byteorder` reports the
|
|
16
|
+
byte order used by integer, float, enum, flag, boolean and MAC conversions.
|
|
17
|
+
BCD always stores its most significant decimal digits first; ASCII and bytes
|
|
18
|
+
retain their byte sequence. `Field.signed` identifies signed integer kinds.
|
|
19
|
+
Fields are immutable and copy their enum/flag tables into read-only mappings.
|
|
20
|
+
Non-empty `values` is valid only for enums and non-empty `bits` only for flags;
|
|
21
|
+
empty mappings are accepted on other kinds so descriptions can be reloaded.
|
|
22
|
+
Numeric kinds take precedence over a conflicting `byteorder`; see the
|
|
23
|
+
[byte-order policy](SCHEMA.md#field-keys).
|
|
24
|
+
|
|
25
|
+
`Layout(*fields, name="", description="", size=None)` stores an ordered tuple of
|
|
26
|
+
fields. `size` can add reserved bytes but cannot truncate a field. An empty layout
|
|
27
|
+
is valid, with size zero unless a size is supplied. Layouts are immutable.
|
|
28
|
+
|
|
29
|
+
`layout.compile()` returns a `Codec`. Compilation resolves offsets and rejects
|
|
30
|
+
overlapping writers and undersized layouts with `SchemaError`. Reuse the compiled
|
|
31
|
+
codec when processing multiple payloads. `layout.encode(...)` and `layout.decode(...)`
|
|
32
|
+
are convenience methods that compile on each call and accept the same arguments
|
|
33
|
+
as the codec methods.
|
|
34
|
+
|
|
35
|
+
```python
|
|
36
|
+
from structwire import Field, Layout
|
|
37
|
+
|
|
38
|
+
codec = Layout(
|
|
39
|
+
Field("counter", "u16le"),
|
|
40
|
+
Field("state", "u8"),
|
|
41
|
+
name="status",
|
|
42
|
+
size=4,
|
|
43
|
+
).compile()
|
|
44
|
+
assert codec.size == 4
|
|
45
|
+
assert codec.fields[0].width == 2
|
|
46
|
+
assert codec.encode({"counter": 0x1234, "state": 1}) == b"\x34\x12\x01\x00"
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
## Compiled codecs
|
|
50
|
+
|
|
51
|
+
`Codec(layout)` is equivalent to `layout.compile()`. Its `layout`, `fields` and
|
|
52
|
+
`size` properties expose the source layout, ordered fields and exact payload size.
|
|
53
|
+
Codecs keep no mutable state between calls and can be reused by independent callers.
|
|
54
|
+
|
|
55
|
+
| Method | Behavior |
|
|
56
|
+
|---|---|
|
|
57
|
+
| `decode(payload, *, strict=True)` | Accepts `bytes`, `bytearray` or `memoryview`; returns a new dictionary of named values. Exact length is required by default. |
|
|
58
|
+
| `decode(payload, strict=False)` | Ignores trailing bytes; short payloads still raise `DecodeError`. |
|
|
59
|
+
| `encode(values, *, require_all=True)` | Accepts a mapping and returns new `bytes`. All fields are required by default; unknown names always raise `EncodeError`. |
|
|
60
|
+
| `encode(values, require_all=False)` | Leaves omitted fields' storage bits zero, rather than applying a physical-value default. |
|
|
61
|
+
| `describe()` | Returns a new dictionary with layout metadata and resolved field offsets, widths, kinds, byte order and enum/flag tables. |
|
|
62
|
+
|
|
63
|
+
Inputs are not modified. Encoding zeroes unrepresented bytes and bits. A zeroed
|
|
64
|
+
omitted field need not decode as physical zero: scaling, bias or enum labels can
|
|
65
|
+
give those bits a different meaning. Descriptions can be serialized with
|
|
66
|
+
`json.dumps` and loaded again with `layout_from_mapping`; modifying a description
|
|
67
|
+
does not change the codec.
|
|
68
|
+
|
|
69
|
+
## Catalogs and loaders
|
|
70
|
+
|
|
71
|
+
`Catalog(layouts)` accepts a non-empty mapping of names to `Layout` or `Codec`
|
|
72
|
+
objects and compiles layouts immediately. A layout's non-empty name must match its
|
|
73
|
+
catalog key. Catalogs implement the read-only `Mapping[str, Codec]` interface:
|
|
74
|
+
indexing, iteration, `len`, `keys`, `values`, `items` and `get` are available.
|
|
75
|
+
`catalog.names` returns names in declaration order. An unknown name raises `KeyError`.
|
|
76
|
+
Invalid containers or entries raise `SchemaError`.
|
|
77
|
+
|
|
78
|
+
`catalog.encode(name, values, *, require_all=True)` and
|
|
79
|
+
`catalog.decode(name, payload, *, strict=True)` delegate to that codec.
|
|
80
|
+
`catalog.describe()` returns a fresh mapping from names to codec descriptions.
|
|
81
|
+
|
|
82
|
+
```python
|
|
83
|
+
from structwire import Catalog, Field, Layout
|
|
84
|
+
|
|
85
|
+
catalog = Catalog({"status": Layout(Field("counter", "u8"))})
|
|
86
|
+
assert catalog.names == ("status",)
|
|
87
|
+
assert catalog.encode("status", {"counter": 7}) == b"\x07"
|
|
88
|
+
assert catalog.decode("status", b"\x07") == {"counter": 7}
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
| Loader | Input and result |
|
|
92
|
+
|---|---|
|
|
93
|
+
| `layout_from_mapping(data, *, default_name="")` | A single layout mapping; returns an uncompiled `Layout`. Uses `default_name` when `name` is omitted. A non-empty `default_name` must match an explicit layout name. |
|
|
94
|
+
| `layout_from_yaml(path)` | A UTF-8 YAML file containing a single layout; returns an uncompiled `Layout`. |
|
|
95
|
+
| `catalog_from_mapping(data)` | A versioned catalog mapping; returns a compiled `Catalog`. |
|
|
96
|
+
| `catalog_from_yaml(path)` | A UTF-8 YAML file containing a catalog; returns a compiled `Catalog`. |
|
|
97
|
+
|
|
98
|
+
File loaders accept a string or `pathlib.Path` path, not YAML text.
|
|
99
|
+
Loading a single layout validates its field definitions;
|
|
100
|
+
call `compile()` to also validate overlaps and total size. Catalog loading performs
|
|
101
|
+
both steps. See the [schema document shapes](SCHEMA.md).
|
|
102
|
+
|
|
103
|
+
## Formatting
|
|
104
|
+
|
|
105
|
+
`format_mac_address(data, *, byteorder="big")` accepts exactly six bytes as `bytes`,
|
|
106
|
+
`bytearray` or `memoryview` and returns uppercase colon-separated hexadecimal.
|
|
107
|
+
`byteorder="little"` reverses the address bytes. Invalid input raises `ValueError`.
|
|
108
|
+
|
|
109
|
+
`format_value(value)` converts byte buffers to uppercase, space-separated hex
|
|
110
|
+
(or `"<empty>"`), recursively visiting dictionaries, lists and tuples. Other values
|
|
111
|
+
are returned unchanged. `format_record(record)` applies it to each value in a
|
|
112
|
+
dictionary and returns a new dictionary. These helpers do not apply units or
|
|
113
|
+
change numeric values.
|
|
114
|
+
|
|
115
|
+
```python
|
|
116
|
+
from structwire import format_mac_address, format_record
|
|
117
|
+
|
|
118
|
+
assert format_mac_address(bytes.fromhex("01 23 45 67 89 AB")) == "01:23:45:67:89:AB"
|
|
119
|
+
assert format_record({"raw": b"\x01\xab", "count": 2}) == {"raw": "01 AB", "count": 2}
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
## Errors
|
|
123
|
+
|
|
124
|
+
`StructWireError` is the common base for `SchemaError`, `EncodeError` and
|
|
125
|
+
`DecodeError`. All three specific exceptions also inherit from `ValueError`.
|
|
126
|
+
|
|
127
|
+
- `SchemaError`: invalid field definitions, schema mappings or YAML, ambiguous
|
|
128
|
+
overlaps, unsupported versions or an undersized layout.
|
|
129
|
+
- `EncodeError`: non-mapping input, unknown or missing fields, invalid values, values
|
|
130
|
+
outside storage limits, off-grid scaled integers or finite values that overflow inverse scaling.
|
|
131
|
+
- `DecodeError`: invalid payload type or length, malformed ASCII/BCD, scaled integer
|
|
132
|
+
precision loss or finite float values that overflow scaling.
|
|
133
|
+
|
|
134
|
+
File access errors remain normal `OSError` exceptions.
|
|
135
|
+
BCD values that exceed Python's integer/string conversion limit raise `EncodeError`
|
|
136
|
+
or `DecodeError`.
|
|
137
|
+
The type annotations describe the supported public inputs; unrelated Python
|
|
138
|
+
programming errors are not uniformly converted to StructWire exceptions.
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
# Optional AutoSpan adoption
|
|
2
|
+
|
|
3
|
+
An AutoSpan project plugin can use StructWire to encode and decode payloads.
|
|
4
|
+
StructWire has no AutoSpan dependency, and using it does not require AutoSpan core
|
|
5
|
+
to depend on it.
|
|
6
|
+
|
|
7
|
+
Suggested responsibilities for this optional integration:
|
|
8
|
+
|
|
9
|
+
| Component | Responsibility |
|
|
10
|
+
|---|---|
|
|
11
|
+
| AutoSpan core | Test planning, execution and evidence |
|
|
12
|
+
| StructWire | Payload layouts, encoding, decoding and value formatting |
|
|
13
|
+
| Driver or adapter | Transport, protocol and device operations |
|
|
14
|
+
| Project plugin | Device configuration, routing and payload definitions |
|
|
15
|
+
|
|
16
|
+
## Project payload catalog
|
|
17
|
+
|
|
18
|
+
A project-specific diagnostic or signal catalog should own the mapping from its identifier to a
|
|
19
|
+
StructWire `Codec`:
|
|
20
|
+
|
|
21
|
+
```python
|
|
22
|
+
from structwire import Field, Layout
|
|
23
|
+
|
|
24
|
+
SOFTWARE_VERSION = Layout(
|
|
25
|
+
Field("major", "bcd"),
|
|
26
|
+
Field("minor", "bcd"),
|
|
27
|
+
Field("patch", "bcd"),
|
|
28
|
+
).compile()
|
|
29
|
+
|
|
30
|
+
PAYLOADS = {
|
|
31
|
+
0x1001: SOFTWARE_VERSION,
|
|
32
|
+
}
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
The protocol layer remains responsible for extracting or adding service identifiers and headers.
|
|
36
|
+
StructWire receives only the payload region whose binary layout it owns.
|
|
37
|
+
|
|
38
|
+
## Integration rule
|
|
39
|
+
|
|
40
|
+
Declare the StructWire dependency in the plugin that uses it. For named catalogs
|
|
41
|
+
and inline layouts, see the [integration examples](INTEGRATION_MODEL.md).
|