struct-frame 0.0.24__tar.gz → 0.0.28__tar.gz

struct_frame-0.0.28/.github/copilot-instructions.md

@@ -0,0 +1,126 @@
+# struct-frame Repository
+struct-frame is a multi-language code generation framework that takes Protocol Buffer (.proto) files and generates serialization/deserialization code for C, TypeScript, and Python. It provides framing and parsing utilities for structured message communication.
+
+Always reference these instructions first and fallback to search or bash commands only when you encounter unexpected information that does not match the info here.
+
+## Working Effectively
+
+### Prerequisites and Dependencies
+- Install Python dependencies:
+  - `python3 -m pip install proto-schema-parser structured-classes`
+- Install Node.js dependencies:
+  - `npm install` -- takes ~1 second
+- GCC is available for C compilation
+
+### Core Build Commands
+- **NEVER CANCEL**: All commands below complete in under 2 seconds unless specified
+- Generate code from proto files:
+  - `PYTHONPATH=src python3 src/main.py [proto_file] --build_c --build_ts --build_py --c_path gen/c --ts_path gen/ts --py_path gen/py`
+  - Takes ~0.1 seconds to complete
+- Compile TypeScript:
+  - `npx tsc --project tsconfig.json` -- takes ~2 seconds
+- Python module works via:
+  - `PYTHONPATH=src python3 -c "import struct_frame; struct_frame.main()"`
+
+### Known Working Components
+- **Python code generator**: FULLY FUNCTIONAL
+  - Reads .proto files and generates code for all three target languages
+  - CLI interface works correctly
+  - Code generation completes successfully
+  - Generated Python files can be imported and used
+- **TypeScript compilation**: PARTIALLY FUNCTIONAL
+  - TypeScript files compile without errors
+  - Generated code has runtime issues (missing method definitions in enum handling)
+  - Use with caution for actual execution
+- **C code generation**: GENERATES BUT HAS ISSUES
+  - Headers are generated but have compilation errors
+  - Conflicts between different API versions in generated macros
+  - Example code in examples/main.c is incompatible with generated headers
+
+### Running Tests and Validation
+- **No formal test suite exists** in the repository
+- Manual validation is required for all components (Python, TypeScript, and C)
+- Test Python code generation by running the main command and verifying output
+
+### Build Times and Timeouts
+- Python dependencies install: ~30 seconds (may fail due to network timeouts)
+- Code generation: ~0.1 seconds - NEVER CANCEL
+- npm install: ~1 second - NEVER CANCEL  
+- TypeScript compilation: ~2 seconds - NEVER CANCEL
+- All operations are very fast, no long builds
+
+## Validation Scenarios
+- **ALWAYS test Python code generation** after making changes to the core generator
+- **Test with the provided examples/myl_vehicle.proto file** as the reference example
+- **Validate that generated Python files import successfully**
+- **DO NOT rely on TypeScript or C compilation** for validation due to known runtime/compilation issues
+- Manually validate core functionality by generating code and checking output
+
+## Common Issues and Workarounds
+- **Python package build fails**: Network timeouts are common - use `PYTHONPATH=src` approach instead
+- **TypeScript runtime errors**: Generated code calls undefined methods like `.myl_vehicle_type()` - this is a code generation bug
+- **C compilation fails**: Generated headers have macro conflicts and syntax errors (C99 vs C++ style initialization)
+- **Example code is outdated**: examples/main.c and examples/index.ts examples don't match current generated code APIs
+
+## Repository Structure
+```
+/
+├── src/                      # Source code directory
+│   ├── main.py              # CLI entry point
+│   └── struct_frame/        # Python code generator (WORKING)
+│       ├── generate.py      # Main generation logic
+│       ├── c_gen.py         # C code generator  
+│       ├── ts_gen.py        # TypeScript code generator
+│       ├── py_gen.py        # Python code generator
+│       └── boilerplate/     # Template files for each language
+├── examples/                # Example files directory
+│   ├── myl_vehicle.proto    # Example proto file
+│   ├── index.ts             # TypeScript example (INCOMPATIBLE with generated code)
+│   └── main.c               # C example (INCOMPATIBLE with generated code)
+├── package.json             # Node.js dependencies
+├── tsconfig.json            # TypeScript configuration
+├── pyproject.toml           # Python package configuration
+└── gen/                     # Generated code output directory
+```
+
+## Critical Warnings
+- **DO NOT expect C or TypeScript examples to compile/run** - they are incompatible with current generator output
+- **DO NOT attempt Python package builds** - they fail due to network issues, use direct module execution
+- **ALWAYS use PYTHONPATH=src** when running Python components
+- **Generated code has API compatibility issues** between languages and with examples
+
+## Quick Start for New Developers
+1. Install dependencies: `python3 -m pip install proto-schema-parser structured-classes && npm install`
+2. Generate code: `PYTHONPATH=src python3 src/main.py examples/myl_vehicle.proto --build_py --py_path gen/py`  
+3. Validate: Check that generated files are created in gen/py directory
+4. For development: Always test Python generation, ignore C/TypeScript runtime errors
+
+## Common Tasks Reference
+
+### Repository Root Contents
+```
+.clang-format       # C formatting config
+.github/            # GitHub configuration including copilot-instructions.md
+.gitignore         # Git ignore rules
+DEVGUIDE.md        # Basic development guide (minimal)
+LICENSE            # MIT license
+README.md          # Basic setup instructions (minimal)
+TODO               # Single item: "Check if message id is repeated"
+examples/          # Example files directory
+├── index.ts       # TypeScript example (broken)
+├── main.c         # C example (broken) 
+└── myl_vehicle.proto  # Proto definition example
+package.json       # Node.js config
+package-lock.json  # Node.js lockfile
+pyproject.toml     # Python package config
+src/               # Source code directory
+tsconfig.json      # TypeScript config
+```
+
+### Example proto file content (examples/myl_vehicle.proto)
+Contains definitions for vehicle communication messages including position, pose, heartbeat with message IDs and field types.
+
+### Working Python Generation Command
+```bash
+PYTHONPATH=src python3 src/main.py examples/myl_vehicle.proto --build_py --py_path gen/py
+```

struct_frame-0.0.28/.gitignore

@@ -0,0 +1,38 @@
+# Build outputs and temporary files
+ts_out/
+node_modules/
+.vscode/
+main.exe
+dist/
+__pycache__/
+
+# Generated code directories
+generated/
+gen/
+tests/generated/
+
+# Legacy generated output directories (cleanup)
+c/
+py/
+ts/
+
+# Test artifacts
+tests/c/*.exe
+tests/c/*.bin
+tests/py/*.bin
+tests/ts/*.bin
+*_test_data.bin
+
+# Debug and temporary files
+debug*.c
+debug*.exe
+error.txt
+*.log
+InstallationLog.txt
+
+# IDE and OS files
+.DS_Store
+Thumbs.db
+*.swp
+*.swo
+*~

struct_frame-0.0.28/AI_TESTING_GUIDE.md

@@ -0,0 +1,205 @@
+# AI Agent Guide for struct-frame Testing
+
+This document provides guidance for AI agents working with the struct-frame project's test suite. It explains how to efficiently update, debug, and extend the testing framework.
+
+## Test Architecture Overview
+
+The struct-frame project uses a comprehensive multi-language test suite located in the `tests/` directory:
+
+```
+tests/
+├── run_tests.py              # Main test orchestrator
+├── proto/                    # Protocol buffer test definitions  
+├── c/                        # C language test programs
+├── ts/                       # TypeScript test programs
+├── py/                       # Python test programs
+└── generated/                # Generated code output (git-ignored)
+```
+
+### Entry Points
+
+- **Primary**: `python test_all.py` (project root)
+- **Direct**: `python tests/run_tests.py` (detailed options)
+
+## Key Components
+
+### 1. Test Runner (`tests/run_tests.py`)
+
+The TestRunner class orchestrates all testing activities:
+- **Code Generation**: Runs struct-frame generator on proto files
+- **Compilation**: Compiles generated C/TypeScript code  
+- **Execution**: Runs test programs for each language
+- **Cross-Language**: Verifies binary compatibility between languages
+
+**Key Methods:**
+- `test_code_generation()`: Generates code for all proto files
+- `test_c_compilation()`: Compiles C tests using GCC
+- `run_c_tests()`: Executes compiled C test programs  
+- `run_python_tests()`: Executes Python test scripts
+- `print_summary()`: Reports comprehensive results
+
+### 2. Protocol Definitions (`tests/proto/`)
+
+Test proto files covering different aspects:
+- **`basic_types.proto`**: All primitive data types (int8-int64, float, double, bool, string)
+- **`comprehensive_arrays.proto`**: Fixed arrays, bounded arrays, string arrays, enum arrays, nested message arrays
+- **`nested_messages.proto`**: Message composition, enums, flatten attribute
+- **`serialization_test.proto`**: Cross-language compatibility testing
+
+### 3. Language-Specific Tests
+
+Each language directory contains parallel test implementations:
+- **`test_basic_types.*`**: Serialization/deserialization of primitive types
+- **`test_arrays.*`**: Array operations and memory layout verification
+- **`test_serialization.*`**: Cross-language binary compatibility
+
+## Common AI Agent Tasks
+
+### Adding New Test Cases
+
+1. **Create Proto Definition**:
+   ```proto
+   // tests/proto/new_feature.proto
+   package new_feature;
+   
+   message TestMessage {
+     option msgid = 205;  // Use next available ID
+     // Add your test fields
+   }
+   ```
+
+2. **Implement Language Tests**:
+   - Copy existing test structure from `test_basic_types.*`
+   - Modify to test your specific feature
+   - Ensure all languages have parallel implementations
+
+3. **Update Test Runner**:
+   - Add proto file to `proto_files` list in `test_code_generation()`
+   - Add test category if needed in `TestRunner.results` dictionary
+
+### Debugging Test Failures
+
+#### C Compilation Issues
+- **Missing GCC**: Test runner gracefully skips if GCC unavailable
+- **Header Issues**: Check generated files in `tests/generated/c/`
+- **Boilerplate Problems**: Verify `copy_boilerplate_files()` execution
+- **C++ Syntax**: Look for aggregate initialization (`{}`) instead of C-compatible `{0}`
+
+#### Python Test Issues  
+- **Import Errors**: Check `PYTHONPATH` includes generated directory
+- **Module Missing**: Verify code generation completed successfully
+- **Constructor Issues**: Structured classes require positional arguments, not keyword arguments
+
+#### TypeScript Issues
+- **Compilation**: Check if `tsc` is available and `npm install` completed
+- **Runtime Errors**: Generated TypeScript may have method call issues (known limitation)
+
+#### Cross-Language Compatibility
+- **Binary Files**: Tests create `*_test_data.bin` files for compatibility checking
+- **Serialization Format**: All languages must produce identical binary output
+- **Checksum Validation**: Fletcher checksum algorithm must match across implementations
+
+### Performance Optimization
+
+The test suite is designed for speed:
+- **Code Generation**: ~0.1 seconds per proto file
+- **C Compilation**: ~1-2 seconds per test file
+- **Python Tests**: ~0.5 seconds per test
+- **Total Runtime**: Usually under 5 seconds
+
+**Never cancel operations** - they complete quickly and interruption can leave partial state.
+
+### Known Issues and Workarounds
+
+#### C Code Generation
+- **Issue**: C++ aggregate initialization syntax in generated C code
+- **Fix**: Update boilerplate files to use `= {0}` instead of `{}`
+- **Location**: `src/struct_frame/boilerplate/c/*.h`
+
+#### Checksum Validation
+- **Issue**: Mismatch between encoding and validation checksum calculation
+- **Fix**: Ensure both use same data length (`msg_size + 1` to include message ID)
+- **Location**: `basic_frame_validate_packet()` function
+
+#### Floating Point Comparisons
+- **Issue**: Exact equality fails due to precision
+- **Fix**: Use tolerance-based comparison with `fabsf()` and `FLOAT_TOLERANCE`
+- **Pattern**: `assert(fabsf(actual - expected) < FLOAT_TOLERANCE)`
+
+#### Array Serialization
+- **Issue**: Complex array structures may serialize only partial data
+- **Limitation**: Known limitation in struct-frame serialization system
+- **Workaround**: Basic types work perfectly; arrays have architectural limitations
+
+### Test Result Interpretation
+
+The test runner provides detailed results categorized by:
+- **Code Generation**: Should always pass (validates proto parsing)
+- **Compilation**: May fail if compilers unavailable (GCC, TSC)
+- **Basic Types**: Should pass for all languages (core functionality)
+- **Arrays**: Python passes; C/TS may have limitations
+- **Serialization**: Tests cross-language binary compatibility
+
+**Success Criteria:**
+- **80%+ pass rate**: Full success
+- **30%+ with core working**: Partial success (acceptable)
+- **<30% pass rate**: Needs attention
+
+### Environment Dependencies
+
+#### Required Always
+- Python 3.8+ with `proto-schema-parser` and `structured-classes`
+
+#### Optional (graceful degradation)
+- **GCC**: For C compilation and execution
+- **TypeScript/Node.js**: For TypeScript compilation and execution
+- **MinGW** (Windows): Provides GCC compiler
+
+#### Directory Structure
+- Tests automatically create `tests/generated/` for output
+- Boilerplate files copied from `src/struct_frame/boilerplate/`
+- Binary test files written to language-specific directories
+
+### Extending the Framework
+
+#### Adding New Languages
+1. Create boilerplate directory: `src/struct_frame/boilerplate/newlang/`
+2. Implement generator: `src/struct_frame/newlang_gen.py`
+3. Add test directory: `tests/newlang/`
+4. Update test runner with compilation and execution logic
+
+#### Adding New Features
+1. Update proto definitions with new syntax
+2. Implement in all language generators
+3. Add comprehensive test cases
+4. Verify cross-language compatibility
+
+### Best Practices for AI Agents
+
+1. **Always run full test suite** after making changes
+2. **Test incrementally** - add one feature at a time
+3. **Verify all languages** - don't assume changes work universally  
+4. **Check boilerplate files** - many issues stem from template problems
+5. **Use verbose mode** when debugging: `python tests/run_tests.py --verbose`
+6. **Preserve working tests** - ensure changes don't break existing functionality
+
+### Quick Reference Commands
+
+```bash
+# Full test suite
+python test_all.py
+
+# Code generation only
+python tests/run_tests.py --generate-only
+
+# Skip problematic languages
+python tests/run_tests.py --skip-c --skip-ts
+
+# Debug specific issue
+python tests/run_tests.py --verbose
+
+# Clean and retry
+rm -rf tests/generated/ && python test_all.py
+```
+
+This testing framework ensures struct-frame maintains high quality across all supported languages while providing clear feedback for development and debugging.

struct_frame-0.0.28/DEVGUIDE.md

@@ -0,0 +1,37 @@
+
+### Installing
+``` py -m pip install --upgrade build twine```
+
+### Building
+Update version in pyproject.toml if needed
+```py -m build```
+
+### Uploading
+```py -m twine upload dist/*```
+
+```py -m build; py -m twine upload dist/*```
+
+
+### Running Locally
+Install dependencies
+```py -m pip install proto-schema-parser```
+
+Run module with example
+```python src/main.py examples/myl_vehicle.proto --build_c --build_ts --build_py --build_gql```
+
+The generated files will be placed in the `generated/` directory with subdirectories for each language (`c/`, `ts/`, `py/`, `gql/`). GraphQL schemas are written with a `.graphql` extension.
+
+### Testing Examples
+After generating code, you can test the examples:
+
+TypeScript:
+```bash
+npx tsc examples/index.ts --outDir generated/
+node generated/examples/index.js
+```
+
+C:
+```bash
+gcc examples/main.c -I generated/c -o main
+./main
+```