knots-test-complexity 1.11.0__py3-none-win_amd64.whl

knots_test_complexity-1.11.0.dist-info/METADATA

@@ -0,0 +1,447 @@
+Metadata-Version: 2.4
+Name: knots-test-complexity
+Version: 1.11.0
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Topic :: Software Development :: Quality Assurance
+Classifier: Topic :: Software Development :: Testing
+Summary: Test quality analyzer — validates unit tests have sufficient complexity and boundary coverage (companion to knots)
+Keywords: testing,complexity,static-analysis,coverage
+Author: Brandon Arrendondo
+License: MIT
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
+Project-URL: Homepage, https://github.com/brandon-arrendondo/knots
+Project-URL: Repository, https://github.com/brandon-arrendondo/knots
+
+# knots-test-complexity
+
+A Rust-based test quality analyzer for C projects that validates unit tests have sufficient complexity to thoroughly exercise source code.
+
+## Motivation
+
+Traditional code coverage metrics (line, branch, function) can be misleading. A test can achieve 100% branch coverage with simple assertions while missing critical edge cases like:
+
+- **Overflow scenarios**: `uint16_t` timer wrapping at 65535
+- **Boundary conditions**: Off-by-one errors at array bounds
+- **State transitions**: Complex state machines with temporal dependencies
+- **Error paths**: Multiple error conditions not all tested
+
+This tool enforces that tests have sufficient **cyclomatic complexity** to exercise all logical paths and **boundary value testing** to catch edge cases.
+
+## Philosophy
+
+> "A test with lower complexity than its source code is likely not testing all scenarios."
+
+### Example: The Overflow Bug
+
+```c
+// Source: Cyclomatic Complexity = 2
+uint16_t timer_ms = 0;
+
+void periodic_1ms() {
+    timer_ms++;  // Overflows at 65535!
+}
+
+bool is_timeout(uint16_t start_ms, uint16_t duration_ms) {
+    return (timer_ms - start_ms) >= duration_ms;
+}
+```
+
+**Traditional Coverage** (100% line, 100% branch):
+```c
+void test_timeout() {
+    timer_ms = 0;
+    TEST_ASSERT_TRUE(is_timeout(0, 100));   // Happy path
+    TEST_ASSERT_FALSE(is_timeout(0, 1));    // Boundary
+}
+// PASSES coverage but MISSES overflow bug!
+```
+
+**Tool Enforcement** - Would detect:
+- Test complexity (2) barely meets source complexity (2)
+- Missing boundary tests: 0, 65535, wrap-around scenarios
+- Missing state variation: timer_ms at different values
+
+**Better Tests** (Higher Complexity):
+```c
+void test_timeout_boundaries() {
+    // Boundary: timer at 0
+    timer_ms = 0;
+    TEST_ASSERT_TRUE(is_timeout(0, 100));
+
+    // Boundary: timer near max
+    timer_ms = 65530;
+    TEST_ASSERT_TRUE(is_timeout(65520, 100));
+
+    // CRITICAL: Overflow scenario
+    timer_ms = 5;  // Wrapped from 65535
+    TEST_ASSERT_TRUE(is_timeout(65530, 100));  // Catches overflow!
+
+    // Multiple start/duration combinations
+    for (int i = 0; i < 5; i++) {
+        test_scenario(scenarios[i]);
+    }
+}
+// Higher complexity test catches the bug!
+```
+
+## Features
+
+### Core Metrics
+
+1. **Test-to-Source Complexity Ratio**
+   - Aggregate cyclomatic complexity of all test functions
+   - Compare to aggregate complexity of source functions
+   - Default threshold: 70% (configurable)
+
+2. **Boundary Value Detection**
+   - Detects integer types: `uint8_t`, `uint16_t`, `uint32_t`, `int8_t`, etc.
+   - Identifies range checks: `if (x > MAX)`, `if (x < MIN)`
+   - Counts required boundary tests
+   - Validates tests cover: MIN, MIN-1, MAX, MAX+1
+
+3. **State Variable Tracking** (Future Enhancement)
+   - Identifies `static`, `volatile`, and global variables
+   - Requires multiple test scenarios per state variable
+   - Validates state transitions are tested
+
+### Output Modes
+
+- **Warning Mode** (default): Reports violations but doesn't fail pre-commit
+- **Error Mode**: Fails pre-commit on violations
+- **Verbose Mode**: Shows detailed per-function complexity breakdown
+
+## Building
+
+```bash
+cargo build --release --workspace
+```
+
+The binary will be at `target/release/knots-test-complexity`
+
+## Installation
+
+### From Source
+
+```bash
+cd knots
+cargo build --release --workspace
+# Binary is at target/release/knots-test-complexity
+```
+
+Add to your PATH or copy to a location in your PATH.
+
+### Pre-Commit Integration
+
+Add to your project's `.pre-commit-config.yaml`:
+
+```yaml
+repos:
+  - repo: https://github.com/brandon-arrendondo/knots
+    rev: v0.3.0  # Use specific version tag
+    hooks:
+      - id: test-complexity
+        args:
+          - --threshold=0.70
+          - --boundary-threshold=0.80
+          - --level=warn
+          - --framework=ceedling
+          - --test-dir=Test
+```
+
+**Configuration Options:**
+
+- `--threshold=0.70`: Minimum test-to-source complexity ratio (default: 0.70 = 70%)
+- `--level=warn`: Enforcement level (`warn` or `error`, default: `warn`)
+- `--no-check-boundaries`: Disable boundary value detection (enabled by default)
+- `--verbose`: Show detailed per-file analysis
+
+**Example: Strict Enforcement**
+```yaml
+args:
+  - --threshold=0.80
+  - --level=error
+  - --verbose
+```
+
+**Example: Warning Only (No Boundaries)**
+```yaml
+args:
+  - --threshold=0.70
+  - --level=warn
+  - --no-check-boundaries
+```
+
+## Usage
+
+### Command Line
+
+Analyze a test file and its corresponding source:
+
+```bash
+knots-test-complexity Test/test_battery_service.c Core/Src/modules/battery_service/battery_service.c
+```
+
+With verbose output:
+
+```bash
+knots-test-complexity -v Test/test_battery_service.c Core/Src/modules/battery_service/battery_service.c
+```
+
+With custom thresholds:
+
+```bash
+knots-test-complexity \
+  --threshold=0.70 \
+  --boundary-threshold=0.80 \
+  --level=error \
+  Test/test_timer.c Core/Src/timer.c
+```
+
+### Output Example
+
+```
+Analyzing Test Quality: test_battery_service.c
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+
+Source File: battery_service.c
+  Functions: 15
+  Total Cyclomatic Complexity: 87
+  Boundary Values Detected: 12
+    - uint8_t variables: 4 (boundaries: 0, 255)
+    - Range checks: 8 (if (x > MAX), etc.)
+
+Test File: test_battery_service.c
+  Functions: 58
+  Total Cyclomatic Complexity: 74
+  Boundary Tests Found: 15
+
+Complexity Analysis:
+  Test/Source Ratio: 85% ✓ (threshold: 70%)
+  Test Complexity: 74
+  Source Complexity: 87
+  Ratio: 74/87 = 0.85
+
+Boundary Analysis:
+  Required Boundary Tests: 12
+  Found Boundary Tests: 15 ✓
+  Coverage: 125%
+
+Result: ✓ PASS
+
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+```
+
+### Failure Example
+
+```
+Analyzing Test Quality: test_lin_comm_service.c
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+
+Source File: lin_comm_service.c
+  Functions: 8
+  Total Cyclomatic Complexity: 54
+  Boundary Values Detected: 8
+
+Test File: test_lin_comm_service.c
+  Functions: 12
+  Total Cyclomatic Complexity: 28
+  Boundary Tests Found: 3
+
+Complexity Analysis:
+  Test/Source Ratio: 52% ✗ (threshold: 70%)
+  Test Complexity: 28
+  Source Complexity: 54
+  Ratio: 28/54 = 0.52
+
+Boundary Analysis:
+  Required Boundary Tests: 8
+  Found Boundary Tests: 3 ✗
+  Missing Boundaries:
+    - rxByteCounter: 0, 3, 11, 12 (RX_HEADER_SIZE boundaries)
+    - rxFrameId: 0, 0x3F (FRAME_MASK boundary)
+
+Recommendations:
+  1. Add tests for edge cases and error paths
+  2. Test boundary conditions: 0, max values, overflow
+  3. Add state transition tests (4 static variables detected)
+  4. Consider parametrized tests or loops in test code
+
+Result: ✗ FAIL (--level=error)
+
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+```
+
+## Algorithm Details
+
+### 1. Complexity Ratio Calculation
+
+```
+For each test file:
+  1. Parse test file with tree-sitter-c
+  2. Calculate cyclomatic complexity for all test functions
+     - Count: if, while, for, switch, &&, ||, ?:
+     - Include test helper functions
+  3. Sum total test complexity
+
+  4. Find corresponding source file
+  5. Calculate cyclomatic complexity for all source functions
+  6. Sum total source complexity
+
+  7. Calculate ratio = test_complexity / source_complexity
+  8. Compare ratio >= threshold (default 70%)
+
+  9. Report: PASS or FAIL with recommendations
+```
+
+### 2. Boundary Value Detection
+
+```
+For source file:
+  1. Find all integer type declarations
+     - uint8_t → boundaries: 0, 255
+     - uint16_t → boundaries: 0, 65535
+     - int8_t → boundaries: -128, 127
+
+  2. Find all range checks
+     - if (x > MAX) → test MAX, MAX+1
+     - if (x < MIN) → test MIN-1, MIN
+     - if (x >= threshold) → test threshold-1, threshold
+
+  3. Count required boundary tests
+
+For test file:
+  1. Find all numeric literals in assertions
+  2. Match literals to source boundaries
+  3. Count covered boundaries
+
+  4. Report: boundary_coverage = found / required
+  5. Warn if coverage < 100%
+```
+
+### 3. Test Helper Function Handling
+
+Test helpers ARE included in complexity calculation:
+
+```c
+// Helper complexity counts!
+void simulate_frame(uint8_t id) {
+    setup_mocks();
+    if (id == SPECIAL) {  // +1 complexity
+        special_handling();
+    }
+    verify_results();
+}
+
+// Test also counts
+void test_multiple_frames() {
+    for (int i = 0; i < 10; i++) {  // +1 complexity
+        simulate_frame(i);  // Helper complexity included
+    }
+}
+// Total test complexity = 2 (loop + helper's if)
+```
+
+This encourages well-structured tests with reusable helpers.
+
+## Integration with knots
+
+`knots-test-complexity` complements `knots`:
+
+| Tool | Purpose | Applied To | Metric |
+|------|---------|------------|--------|
+| **knots** | Source code quality | Production code (`.c`, `.h`) | McCabe & Cognitive complexity per function |
+| **knots-test-complexity** | Test quality | Test code (`test_*.c`) | Aggregate complexity ratio & boundary coverage |
+
+**Example Combined Workflow:**
+
+```yaml
+- repo: local
+  hooks:
+    # Check source code complexity (per-function limits)
+    - id: knots
+      name: Code Complexity Check
+      entry: hooks/pre-commit-wrapper.sh
+      language: script
+      files: \.(c|h)$
+      exclude: ^Test/
+      args: [--mccabe-threshold=15, --cognitive-threshold=15]
+
+    # Check test quality (aggregate complexity ratio)
+    - id: test-complexity
+      name: Test Quality Check
+      entry: hooks/test-complexity-wrapper.sh
+      language: script
+      files: ^Test/test_.*\.c$
+      args: [--threshold=70, --level=warn]
+```
+
+### Future: Unified Tool
+
+A future enhancement could merge both tools:
+
+```bash
+# Unified complexity tool
+complexity-check --source <file.c> --test <test_file.c>
+  --source-mccabe-max=15
+  --source-cognitive-max=15
+  --test-ratio-min=0.70
+  --check-boundaries
+```
+
+## Dependencies
+
+```toml
+[dependencies]
+tree-sitter = "0.22"
+tree-sitter-c = "0.21"
+anyhow = "1.0"
+clap = { version = "4.5", features = ["derive"] }
+regex = "1.10"
+```
+
+## Project Structure
+
+```
+knots-test-complexity/
+├── Cargo.toml                          # Rust project manifest
+├── README.md                           # This file
+├── src/
+│   ├── main.rs                        # CLI entry point
+│   ├── analyzer.rs                    # Test quality analyzer
+│   ├── boundary.rs                    # Boundary value detector
+│   └── reporter.rs                    # Output formatting
+└── examples/
+    ├── test_timer_good.c              # Example: sufficient complexity
+    ├── test_timer_bad.c               # Example: insufficient complexity
+    └── README.md                      # Example documentation
+```
+
+## Testing
+
+Run the test suite:
+
+```bash
+cargo test
+```
+
+Run with verbose output:
+
+```bash
+cargo test -- --nocapture
+```
+
+## License
+
+MIT License. See LICENSE file.
+
+## See Also
+
+- **knots**: Source code complexity analyzer
+- **pmccabe**: Industry-standard McCabe complexity tool (validation reference)
+- **Cognitive Complexity**: [SonarSource specification](https://www.sonarsource.com/resources/cognitive-complexity/)
+- **Mutation Testing**: Alternative approach for test quality (future consideration)
+

knots_test_complexity-1.11.0.dist-info/RECORD

@@ -0,0 +1,5 @@
+knots_test_complexity-1.11.0.data/scripts/knots-test-complexity.exe,sha256=nkfqNx9_E6JmI1qr0Cq1RSaBM5hENFBViw7URfFpiA8,11164160
+knots_test_complexity-1.11.0.dist-info/METADATA,sha256=RXJcV_eP_6iAWPkhk_VIoyqdFxtB73JQq_1ZGl01U00,12908
+knots_test_complexity-1.11.0.dist-info/WHEEL,sha256=2zDlIYIdD4m4N3p5DVEG3iJhGLdhsBQgdH-FqVkAur8,94
+knots_test_complexity-1.11.0.dist-info/sboms/knots-test-complexity.cyclonedx.json,sha256=2T8Oevq-Nha23xUb0Qco6Zu6pWwqTReCWUwJvuADwhg,57590
+knots_test_complexity-1.11.0.dist-info/RECORD,,

knots_test_complexity-1.11.0.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: maturin (1.14.1)
+Root-Is-Purelib: false
+Tag: py3-none-win_amd64