struct-frame 0.0.29__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- struct_frame/__init__.py +11 -0
- struct_frame/__main__.py +8 -0
- struct_frame/base.py +85 -0
- struct_frame/c_gen.py +252 -0
- struct_frame/generate.py +607 -0
- struct_frame/gql_gen.py +207 -0
- struct_frame/py_gen.py +213 -0
- struct_frame/ts_gen.py +260 -0
- struct_frame-0.0.29.dist-info/METADATA +311 -0
- struct_frame-0.0.29.dist-info/RECORD +12 -0
- struct_frame-0.0.29.dist-info/WHEEL +4 -0
- struct_frame-0.0.29.dist-info/licenses/LICENSE +21 -0
|
@@ -0,0 +1,311 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: struct-frame
|
|
3
|
+
Version: 0.0.29
|
|
4
|
+
Summary: A framework for serializing data with headers
|
|
5
|
+
Project-URL: Homepage, https://github.com/mylonics/struct-frame
|
|
6
|
+
Project-URL: Issues, https://github.com/mylonics/struct-frame/issues
|
|
7
|
+
Author-email: Rijesh Augustine <rijesh@mylonics.com>
|
|
8
|
+
License-Expression: MIT
|
|
9
|
+
License-File: LICENSE
|
|
10
|
+
Classifier: Operating System :: OS Independent
|
|
11
|
+
Classifier: Programming Language :: Python :: 3
|
|
12
|
+
Requires-Python: >=3.8
|
|
13
|
+
Requires-Dist: proto-schema-parser>=1.4.5
|
|
14
|
+
Requires-Dist: structured-classes>=3.1.0
|
|
15
|
+
Description-Content-Type: text/markdown
|
|
16
|
+
|
|
17
|
+
|
|
18
|
+
# Struct Frame
|
|
19
|
+
|
|
20
|
+
A multi-language code generation framework that converts Protocol Buffer (.proto) files into serialization/deserialization code for C, TypeScript, Python, and GraphQL. It provides framing and parsing utilities for structured message communication.
|
|
21
|
+
|
|
22
|
+
## Quick Start
|
|
23
|
+
|
|
24
|
+
### Installation
|
|
25
|
+
```bash
|
|
26
|
+
# Install Python dependencies
|
|
27
|
+
pip install proto-schema-parser structured-classes
|
|
28
|
+
|
|
29
|
+
# Install Node.js dependencies (for TypeScript)
|
|
30
|
+
npm install
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
### Basic Usage
|
|
34
|
+
```bash
|
|
35
|
+
# Generate code for all languages
|
|
36
|
+
PYTHONPATH=src python3 src/main.py examples/myl_vehicle.proto --build_c --build_ts --build_py --build_gql
|
|
37
|
+
|
|
38
|
+
# Run comprehensive test suite
|
|
39
|
+
python test_all.py
|
|
40
|
+
|
|
41
|
+
# Generated files will be in the generated/ directory
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
### Test Suite
|
|
45
|
+
|
|
46
|
+
The project includes a comprehensive test suite that validates code generation, compilation, and serialization across all supported languages:
|
|
47
|
+
|
|
48
|
+
```bash
|
|
49
|
+
# Run all tests
|
|
50
|
+
python test_all.py
|
|
51
|
+
|
|
52
|
+
# Run with verbose output
|
|
53
|
+
python tests/run_tests.py --verbose
|
|
54
|
+
|
|
55
|
+
# Skip specific languages
|
|
56
|
+
python tests/run_tests.py --skip-ts --skip-c
|
|
57
|
+
|
|
58
|
+
# Generate code only (no compilation/execution)
|
|
59
|
+
python tests/run_tests.py --generate-only
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
See `tests/README.md` for detailed test documentation.
|
|
63
|
+
|
|
64
|
+
### Language-Specific Examples
|
|
65
|
+
|
|
66
|
+
#### Python
|
|
67
|
+
```bash
|
|
68
|
+
python src/main.py examples/myl_vehicle.proto --build_py
|
|
69
|
+
# Use generated Python classes directly
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
#### TypeScript
|
|
73
|
+
```bash
|
|
74
|
+
python src/main.py examples/myl_vehicle.proto --build_ts
|
|
75
|
+
npx tsc examples/index.ts --outDir generated/
|
|
76
|
+
node generated/examples/index.js
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
#### C
|
|
80
|
+
```bash
|
|
81
|
+
python src/main.py examples/myl_vehicle.proto --build_c
|
|
82
|
+
gcc examples/main.c -I generated/c -o main
|
|
83
|
+
./main
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
#### GraphQL
|
|
87
|
+
```bash
|
|
88
|
+
python src/main.py examples/myl_vehicle.proto --build_gql
|
|
89
|
+
# Use generated .graphql schema files
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
## Feature Compatibility Matrix
|
|
93
|
+
|
|
94
|
+
| Feature | C | TypeScript | Python | GraphQL | Status |
|
|
95
|
+
|---------|---|------------|--------|---------|--------|
|
|
96
|
+
| **Core Types** | ✓ | ✓ | ✓ | ✓ | Stable |
|
|
97
|
+
| **String** | ✓ | ✓ | ✓ | ✓ | Stable |
|
|
98
|
+
| **Enums** | ✓ | ✓ | ✓ | ✓ | Stable |
|
|
99
|
+
| **Nested Messages** | ✓ | ✓ | ✓ | ✓ | Stable |
|
|
100
|
+
| **Message IDs** | ✓ | ✓ | ✓ | N/A | Stable |
|
|
101
|
+
| **Message Serialization** | ✓ | ✓ | ✓ | N/A | Stable |
|
|
102
|
+
| **Flatten** | N/A | N/A | ✓ | ✓ | Partial |
|
|
103
|
+
| **Arrays** | ✓ | ✓ | ✓ | ✓ | Stable |
|
|
104
|
+
|
|
105
|
+
**Legend:**
|
|
106
|
+
- **✓** - Feature works as documented
|
|
107
|
+
- **Partial** - Basic functionality works, some limitations
|
|
108
|
+
- **✗** - Feature not yet available
|
|
109
|
+
- **N/A** - Not applicable for this language
|
|
110
|
+
|
|
111
|
+
## Project Structure
|
|
112
|
+
|
|
113
|
+
- `src/struct_frame/` - Core code generation framework
|
|
114
|
+
- `generate.py` - Main parser and validation logic
|
|
115
|
+
- `*_gen.py` - Language-specific code generators
|
|
116
|
+
- `boilerplate/` - Runtime libraries for each language
|
|
117
|
+
- `examples/` - Example .proto files and usage demos
|
|
118
|
+
- `main.c` - C API demonstration (encoding/decoding, parsing)
|
|
119
|
+
- `index.ts` - TypeScript API demonstration (similar functionality)
|
|
120
|
+
- `*.proto` - Protocol Buffer definitions for examples
|
|
121
|
+
- `generated/` - Output directory for generated code (git-ignored)
|
|
122
|
+
|
|
123
|
+
## Protocol Buffer Schema Reference
|
|
124
|
+
|
|
125
|
+
### Supported Data Types
|
|
126
|
+
|
|
127
|
+
| Type | Size (bytes) | Description | Range/Notes |
|
|
128
|
+
|------|--------------|-------------|-------------|
|
|
129
|
+
| **Integers** |
|
|
130
|
+
| `int8` | 1 | Signed 8-bit integer | -128 to 127 |
|
|
131
|
+
| `uint8` | 1 | Unsigned 8-bit integer | 0 to 255 |
|
|
132
|
+
| `int16` | 2 | Signed 16-bit integer | -32,768 to 32,767 |
|
|
133
|
+
| `uint16` | 2 | Unsigned 16-bit integer | 0 to 65,535 |
|
|
134
|
+
| `int32` | 4 | Signed 32-bit integer | -2.1B to 2.1B |
|
|
135
|
+
| `uint32` | 4 | Unsigned 32-bit integer | 0 to 4.3B |
|
|
136
|
+
| `int64` | 8 | Signed 64-bit integer | Large integers |
|
|
137
|
+
| `uint64` | 8 | Unsigned 64-bit integer | Large positive integers |
|
|
138
|
+
| **Floating Point** |
|
|
139
|
+
| `float` | 4 | Single precision (IEEE 754) | 7 decimal digits |
|
|
140
|
+
| `double` | 8 | Double precision (IEEE 754) | 15-17 decimal digits |
|
|
141
|
+
| **Other** |
|
|
142
|
+
| `bool` | 1 | Boolean value | `true` or `false` |
|
|
143
|
+
| `string` | Variable | UTF-8 encoded string | Length-prefixed |
|
|
144
|
+
| `EnumType` | 1 | Custom enumeration | Defined in .proto |
|
|
145
|
+
| `MessageType` | Variable | Nested message | User-defined structure |
|
|
146
|
+
|
|
147
|
+
> **Note:** All types use little-endian byte order for cross-platform compatibility.
|
|
148
|
+
|
|
149
|
+
### Array Support
|
|
150
|
+
|
|
151
|
+
Arrays (repeated fields) support all data types - primitives, enums, and messages across all target languages.
|
|
152
|
+
|
|
153
|
+
| Array Type | Syntax | Memory Usage | Use Case |
|
|
154
|
+
|------------|--------|--------------|----------|
|
|
155
|
+
| **Fixed** | `repeated type field = N [size=X];` | `sizeof(type) * X` | Matrices, buffers (always full) |
|
|
156
|
+
| **Bounded** | `repeated type field = N [max_size=X];` | 1 byte (count) + `sizeof(type) * X` | Dynamic lists with limits |
|
|
157
|
+
| **String Arrays** | `repeated string field = N [max_size=X, element_size=Y];` | 1 byte (count) + `X * Y` bytes | Text collections with size limits |
|
|
158
|
+
|
|
159
|
+
```proto
|
|
160
|
+
message ArrayExample {
|
|
161
|
+
repeated float matrix = 1 [size=9]; // 3x3 matrix (always 9 elements)
|
|
162
|
+
repeated string names = 2 [max_size=10, element_size=32]; // Up to 10 strings, each max 32 chars
|
|
163
|
+
repeated int32 values = 3 [max_size=100]; // Up to 100 integers (variable count)
|
|
164
|
+
}
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
**Generated Output** (all languages now supported):
|
|
168
|
+
- **Python**: `matrix: list[float]`, `names: list[str]`, `values: list[int]`
|
|
169
|
+
- **C**: `float matrix[9]`, `struct { uint8_t count; char data[10][32]; } names`
|
|
170
|
+
- **TypeScript**: `Array('matrix', 'Float32LE', 9)`, `Array('names_data', 'String', 10)`
|
|
171
|
+
- **GraphQL**: `matrix: [Float!]!`, `names: [String!]!`, `values: [Int!]!`
|
|
172
|
+
|
|
173
|
+
> **Important**: String arrays require both `max_size` (or `size`) AND `element_size` parameters because they are "arrays of arrays" - you need to specify both how many strings AND the maximum size of each individual string. This ensures predictable memory layout and prevents buffer overflows.
|
|
174
|
+
|
|
175
|
+
### String Type
|
|
176
|
+
|
|
177
|
+
Strings are a special case of bounded character arrays with built-in UTF-8 encoding and null-termination handling across all target languages.
|
|
178
|
+
|
|
179
|
+
| String Type | Syntax | Memory Usage | Use Case |
|
|
180
|
+
|-------------|--------|--------------|----------|
|
|
181
|
+
| **Fixed String** | `string field = N [size=X];` | `X` bytes | Fixed-width text fields |
|
|
182
|
+
| **Variable String** | `string field = N [max_size=X];` | 1 byte (length) + `X` bytes | Text with known maximum length |
|
|
183
|
+
|
|
184
|
+
```proto
|
|
185
|
+
message StringExample {
|
|
186
|
+
string device_name = 1 [size=16]; // Exactly 16 characters (pad with nulls)
|
|
187
|
+
string description = 2 [max_size=256]; // Up to 256 characters (length-prefixed)
|
|
188
|
+
string error_msg = 3 [max_size=128]; // Up to 128 characters for error messages
|
|
189
|
+
}
|
|
190
|
+
```
|
|
191
|
+
|
|
192
|
+
**String Benefits:**
|
|
193
|
+
- **Simplified Schema**: No need to specify `repeated uint8` for text data
|
|
194
|
+
- **Automatic Encoding**: UTF-8 encoding/decoding handled by generators
|
|
195
|
+
- **Null Handling**: Proper null-termination and padding for fixed strings
|
|
196
|
+
- **Type Safety**: Clear distinction between binary data and text
|
|
197
|
+
- **Cross-Language**: Consistent string handling across C, TypeScript, and Python
|
|
198
|
+
|
|
199
|
+
### Message Options
|
|
200
|
+
|
|
201
|
+
**Message ID (`msgid`)** - Required for serializable messages:
|
|
202
|
+
```proto
|
|
203
|
+
message MyMessage {
|
|
204
|
+
option msgid = 42; // Must be unique within package (0-65535)
|
|
205
|
+
string content = 1;
|
|
206
|
+
}
|
|
207
|
+
```
|
|
208
|
+
|
|
209
|
+
### Field Options
|
|
210
|
+
|
|
211
|
+
**Flatten (`flatten=true`)** - Merge nested message fields into parent:
|
|
212
|
+
```proto
|
|
213
|
+
message Position {
|
|
214
|
+
double lat = 1;
|
|
215
|
+
double lon = 2;
|
|
216
|
+
}
|
|
217
|
+
|
|
218
|
+
message Status {
|
|
219
|
+
Position pos = 1 [flatten=true]; // lat, lon become direct fields
|
|
220
|
+
float battery = 2;
|
|
221
|
+
}
|
|
222
|
+
```
|
|
223
|
+
|
|
224
|
+
**Array Options** - Control array behavior:
|
|
225
|
+
```proto
|
|
226
|
+
message Data {
|
|
227
|
+
repeated int32 fixed_buffer = 1 [size=256]; // Always 256 integers
|
|
228
|
+
repeated int32 var_buffer = 2 [max_size=256]; // Up to 256 integers
|
|
229
|
+
repeated string messages = 3 [max_size=10, element_size=64]; // Up to 10 strings, each max 64 chars
|
|
230
|
+
string device_name = 4 [size=32]; // Always 32 characters
|
|
231
|
+
string description = 5 [max_size=256]; // Up to 256 characters
|
|
232
|
+
}
|
|
233
|
+
```
|
|
234
|
+
|
|
235
|
+
## Complete Example
|
|
236
|
+
|
|
237
|
+
```proto
|
|
238
|
+
package sensor_system;
|
|
239
|
+
|
|
240
|
+
enum SensorType {
|
|
241
|
+
TEMPERATURE = 0;
|
|
242
|
+
HUMIDITY = 1;
|
|
243
|
+
PRESSURE = 2;
|
|
244
|
+
}
|
|
245
|
+
|
|
246
|
+
message Position {
|
|
247
|
+
double lat = 1;
|
|
248
|
+
double lon = 2;
|
|
249
|
+
float alt = 3;
|
|
250
|
+
}
|
|
251
|
+
|
|
252
|
+
message SensorReading {
|
|
253
|
+
option msgid = 1;
|
|
254
|
+
|
|
255
|
+
uint32 device_id = 1;
|
|
256
|
+
int64 timestamp = 2;
|
|
257
|
+
SensorType type = 3;
|
|
258
|
+
|
|
259
|
+
// Device name (fixed 16-character string)
|
|
260
|
+
string device_name = 4 [size=16];
|
|
261
|
+
|
|
262
|
+
// Sensor location (flattened)
|
|
263
|
+
Position location = 5 [flatten=true];
|
|
264
|
+
|
|
265
|
+
// Measurement values (up to 8 readings)
|
|
266
|
+
repeated float values = 6 [max_size=8];
|
|
267
|
+
|
|
268
|
+
// Calibration matrix (always 3x3 = 9 elements)
|
|
269
|
+
repeated float calibration = 7 [size=9];
|
|
270
|
+
|
|
271
|
+
// Error message (up to 128 characters)
|
|
272
|
+
string error_msg = 8 [max_size=128];
|
|
273
|
+
|
|
274
|
+
bool valid = 9;
|
|
275
|
+
}
|
|
276
|
+
|
|
277
|
+
message DeviceStatus {
|
|
278
|
+
option msgid = 2;
|
|
279
|
+
|
|
280
|
+
uint32 device_id = 1;
|
|
281
|
+
repeated SensorReading recent_readings = 2 [max_size=10];
|
|
282
|
+
float battery_level = 3;
|
|
283
|
+
}
|
|
284
|
+
```
|
|
285
|
+
|
|
286
|
+
## Schema Validation Rules
|
|
287
|
+
|
|
288
|
+
- **Message IDs**: Must be unique within package (0-65535)
|
|
289
|
+
- **Field numbers**: Must be unique within message
|
|
290
|
+
- **Array requirements**: All `repeated` fields must specify `[size=X]` (fixed) or `[max_size=X]` (bounded)
|
|
291
|
+
- **String requirements**: All `string` fields must specify `[size=X]` (fixed) or `[max_size=X]` (variable)
|
|
292
|
+
- **String array requirements**: `repeated string` fields must specify both array size AND `[element_size=Y]`
|
|
293
|
+
- **Flatten constraints**: No field name collisions after flattening
|
|
294
|
+
- **Size limits**: Arrays limited to 255 elements maximum
|
|
295
|
+
|
|
296
|
+
## Code Generation
|
|
297
|
+
|
|
298
|
+
```bash
|
|
299
|
+
# Generate all languages
|
|
300
|
+
python src/main.py schema.proto --build_c --build_ts --build_py --build_gql
|
|
301
|
+
|
|
302
|
+
# Language-specific paths
|
|
303
|
+
python src/main.py schema.proto --build_py --py_path output/python/
|
|
304
|
+
python src/main.py schema.proto --build_c --c_path output/c/
|
|
305
|
+
python src/main.py schema.proto --build_ts --ts_path output/typescript/
|
|
306
|
+
python src/main.py schema.proto --build_gql --gql_path output/graphql/
|
|
307
|
+
```
|
|
308
|
+
|
|
309
|
+
## Additional Documentation
|
|
310
|
+
|
|
311
|
+
- **[Array Implementation Guide](ARRAY_IMPLEMENTATION.md)** - Comprehensive documentation of array features, syntax, and generated code examples across all languages
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
struct_frame/__init__.py,sha256=m9hfIbcgCYvt-fIeYrJRQyk6I7rq35jSdZIhrMmZntg,374
|
|
2
|
+
struct_frame/__main__.py,sha256=tIybnBeFHvwiwVhodVOSnxhne5AX_80mtXBx4rneSB4,143
|
|
3
|
+
struct_frame/base.py,sha256=1Z_0vMkwz0X8r2hIVLv5yuhwwD929LwNMzVKBqFxxac,2012
|
|
4
|
+
struct_frame/c_gen.py,sha256=4verO94bPBgxOGb4lkhal2RLWI_308ywbV0-54Mxz64,9539
|
|
5
|
+
struct_frame/generate.py,sha256=8YL4MJ--vvOsv5MFqGjKSATSulanEqX_Qf0Jew6x7UU,23416
|
|
6
|
+
struct_frame/gql_gen.py,sha256=jDLsyZJmfpVBpdXE06mixZ3D01NAmRioGJqk_FoQt90,7543
|
|
7
|
+
struct_frame/py_gen.py,sha256=EQqw8u2o-FszROviQyQCpF45bbx0KqjdqVBefpyDqH4,7932
|
|
8
|
+
struct_frame/ts_gen.py,sha256=glT9eA--j_8vaR8tHQ0ekDg7udIVgE9FGA4MGAwiCsA,10103
|
|
9
|
+
struct_frame-0.0.29.dist-info/METADATA,sha256=qEwaFx4oKSDaOzQVygIo-kY8iNBldaQXjmfZCxgo63I,10831
|
|
10
|
+
struct_frame-0.0.29.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
|
|
11
|
+
struct_frame-0.0.29.dist-info/licenses/LICENSE,sha256=UjbLtGfcHCIqJg9UzEVGoNW8fyX4Ah9ZbsuAmJ_vhmk,1094
|
|
12
|
+
struct_frame-0.0.29.dist-info/RECORD,,
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2025 Rijesh Augustine
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|