phigrade 1.0.0__tar.gz

phigrade-1.0.0/LICENSE

@@ -0,0 +1,7 @@
+Copyright 2026 Matt Gormley, Jacob Rast, and contributors to this project
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

phigrade-1.0.0/PKG-INFO

@@ -0,0 +1,300 @@
+Metadata-Version: 2.3
+Name: phigrade
+Version: 1.0.0
+Summary: A lightweight autograder where tests run locally and data is stored remotely
+License: MIT
+Keywords: autograder,education,grading
+Author: Matt Gormley
+Author-email: mgormley@cs.cmu.edu
+Requires-Python: >=3.12,<4.0
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Provides-Extra: dev
+Requires-Dist: fastapi (>=0.115.13,<0.116.0)
+Requires-Dist: numpy (>=2.0.0,<3.0.0)
+Requires-Dist: omegaconf (>=2.3.0,<3.0.0)
+Requires-Dist: pydantic (>=2.11.7,<3.0.0)
+Requires-Dist: pytest (>=8.4.1,<9.0.0) ; extra == "dev"
+Requires-Dist: requests (>=2.32.4,<3.0.0)
+Requires-Dist: tinydb (>=4.8.2,<5.0.0)
+Requires-Dist: uvicorn (>=0.34.3,<0.35.0)
+Project-URL: Repository, https://github.com/mld-instructors/phigrade
+Description-Content-Type: text/markdown
+
+# PhiGrade
+
+PhiGrade is a lightweight autograder framework designed for educational environments where tests run locally and results are stored remotely. It provides a unique approach to automated grading by executing student code locally while maintaining centralized result management.
+
+## Key Features
+
+- **Local Test Execution**: Tests run on the student's machine, ensuring security and scalability
+- **Dual-Mode Operation**: Teacher mode for storing references, student mode for comparison
+- **Multiple Comparison Types**: Support for exact equality (`phigrade_isequal`) and numpy array tolerance (`phigrade_allclose`)
+- **Flexible Deployment**: Local development server or remote backend integration
+- **Call Count Tracking**: Multiple assertions per test function with unique identification
+
+## Quick Start
+
+### Installation
+
+```bash
+# Install with Poetry
+poetry install
+
+# Or with pip
+pip install -e .
+```
+
+### Basic Usage
+
+1. **Create a configuration file** (`phigrade.yaml`):
+```yaml
+assignment_id: "homework_1"
+use_local_server: true
+teacher_mode: false
+timeout_seconds: 5
+```
+
+2. **Write test functions**:
+```python
+from phigrade.phigrade import phigrade_isequal, phigrade_allclose, weight
+import numpy as np
+
+@weight(5.0)
+def test_basic_math():
+    result = 2 + 2
+    phigrade_isequal(result)  # Compares against stored reference
+    return result
+
+@weight(10.0)
+def test_numpy_array():
+    result = np.array([1.0, 2.0, 3.0])
+    phigrade_allclose(result, rtol=1e-5, atol=1e-8)
+    return result
+```
+
+3. **Run tests**:
+```bash
+poetry run python your_test_file.py
+```
+
+## Architecture
+
+### Dual-Mode Workflow
+
+**Teacher Mode** (`teacher_mode: true`):
+- Instructors run tests on reference implementations
+- Results are stored as "correct" references with comparison metadata
+- Each test function with `@weight` decorator stores its output
+
+**Student Mode** (`teacher_mode: false`):
+- Students run the same test functions on their implementations
+- Their outputs are compared against stored references
+- Automatic scoring based on comparison results
+
+### Comparison Types
+
+#### Exact Equality (`phigrade_isequal`)
+```python
+@weight(3.0)
+def test_string_processing():
+    result = process_text("hello")
+    phigrade_isequal(result)  # Exact match required
+```
+
+#### Numpy Array Tolerance (`phigrade_allclose`)
+```python
+@weight(8.0)
+def test_numerical_computation():
+    result = np.array([1.0, 2.0, 3.0])
+    phigrade_allclose(result, rtol=1e-5, atol=1e-8, equal_nan=False)
+```
+
+### Multiple Calls Per Test
+
+```python
+@weight(15.0)
+def test_multiple_operations():
+    # Each call gets unique identifier: test_multiple_operations@0, @1, @2
+    result1 = operation_a()
+    phigrade_isequal(result1)
+    
+    result2 = operation_b()
+    phigrade_isequal(result2)
+    
+    array_result = operation_c()
+    phigrade_allclose(array_result, rtol=0.01)
+```
+
+## Configuration
+
+### Local Development Mode
+
+```yaml
+assignment_id: "my_assignment"
+use_local_server: true
+teacher_mode: false
+timeout_seconds: 10
+```
+
+### Remote Backend Mode
+
+```yaml
+assignment_id: "my_assignment"
+use_local_server: false
+server_url: "https://your-phigrade-backend.com"
+api_key: "your_api_key_here"
+teacher_mode: false
+timeout_seconds: 30
+```
+
+## Data Flow
+
+```
+┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
+│   Teacher Mode  │    │                  │    │  Student Mode   │
+│                 │    │  PhiGrade Server │    │                 │
+│ Store Reference │───▶│                  │◀───│ Submit Results  │
+│    Results      │    │  (Local/Remote)  │    │ Get Comparison  │
+└─────────────────┘    └──────────────────┘    └─────────────────┘
+```
+
+1. **Teacher Phase**: Instructor runs tests, stores reference outputs and comparison configuration
+2. **Student Phase**: Students run tests, submit outputs for comparison against references
+3. **Scoring**: Automatic scoring based on comparison results (exact match or tolerance-based)
+
+## Development
+
+### Running Tests
+
+```bash
+# Run all tests
+poetry run pytest
+
+# Run specific test categories
+poetry run pytest tests/test_phigrade_isequal_local.py  # Equality tests
+poetry run pytest tests/test_phigrade_allclose_local.py # Array tolerance tests
+poetry run pytest tests/test_local_server.py          # Local server tests
+poetry run pytest tests/test_real_backend_integration.py # Backend integration
+
+# Verbose output
+poetry run pytest -v
+```
+
+### Local Development Server
+
+```bash
+# Start local server manually
+poetry run python -m phigrade.app
+
+# Server starts automatically when use_local_server: true
+```
+
+### Project Structure
+
+```
+phigrade/
+├── phigrade/
+│   ├── __init__.py
+│   ├── phigrade.py          # Main API (decorators, assertions)
+│   ├── app.py             # Local FastAPI server
+│   ├── server.py          # Server management
+│   ├── compare.py         # Comparison logic
+│   └── config.py          # Configuration handling
+├── tests/
+│   ├── test_phigrade_isequal_local.py    # Equality comparison tests
+│   ├── test_phigrade_allclose_local.py   # Array tolerance tests
+│   ├── test_local_server.py             # Local server API tests
+│   └── test_real_backend_integration.py # Backend integration tests
+├── examples/              # Example usage
+├── phigrade.yaml          # Configuration file
+└── README.md
+```
+
+## Examples
+
+### Complete Teacher Workflow
+
+```python
+# teacher_tests.py
+from phigrade.phigrade import phigrade_isequal, phigrade_allclose, weight
+import numpy as np
+
+@weight(5.0)
+def test_factorial():
+    """Test factorial function"""
+    result = factorial(5)
+    phigrade_isequal(result)  # Stores 120 as reference
+    return result
+
+@weight(10.0)
+def test_matrix_multiplication():
+    """Test matrix operations"""
+    A = np.array([[1, 2], [3, 4]])
+    B = np.array([[2, 0], [1, 2]])
+    result = A @ B
+    phigrade_allclose(result, rtol=1e-10)  # Stores array with tight tolerance
+    return result
+```
+
+### Complete Student Workflow
+
+```python
+# student_tests.py (same function names)
+from phigrade.phigrade import phigrade_isequal, phigrade_allclose, weight
+import numpy as np
+
+@weight(5.0)
+def test_factorial():
+    """Student implementation"""
+    result = my_factorial(5)  # Student's implementation
+    phigrade_isequal(result)   # Compares against teacher's stored result
+    return result
+
+@weight(10.0)
+def test_matrix_multiplication():
+    """Student implementation"""
+    A = np.array([[1, 2], [3, 4]])
+    B = np.array([[2, 0], [1, 2]])
+    result = my_matrix_mult(A, B)  # Student's implementation
+    phigrade_allclose(result, rtol=1e-10)  # Compares with tolerance
+    return result
+```
+
+## API Reference
+
+### Decorators
+
+#### `@weight(points: float)`
+Decorates test functions to assign point values and enable phigrade tracking.
+
+### Assertion Functions
+
+#### `phigrade_isequal(output: Any) -> None`
+Compares output using exact equality (`==`).
+
+#### `phigrade_allclose(output: np.ndarray, rtol=1e-05, atol=1e-08, equal_nan=False) -> None`
+Compares numpy arrays with tolerance using `np.allclose()`.
+
+### Configuration
+
+#### `PhiGradeConfig`
+Loads configuration from YAML files with validation and type checking.
+
+## Contributing
+
+1. Fork the repository
+2. Create a feature branch
+3. Add tests for new functionality
+4. Ensure all tests pass: `poetry run pytest`
+5. Submit a pull request
+
+## License
+
+This project is licensed under the MIT License - see the LICENSE file for details.
+
+## Related Projects
+
+- **phigrade-backend**: Centralized backend server for multi-user deployments
+- **phigrade-frontend**: Web interface for instructors and administrators

phigrade-1.0.0/README.md

@@ -0,0 +1,276 @@
+# PhiGrade
+
+PhiGrade is a lightweight autograder framework designed for educational environments where tests run locally and results are stored remotely. It provides a unique approach to automated grading by executing student code locally while maintaining centralized result management.
+
+## Key Features
+
+- **Local Test Execution**: Tests run on the student's machine, ensuring security and scalability
+- **Dual-Mode Operation**: Teacher mode for storing references, student mode for comparison
+- **Multiple Comparison Types**: Support for exact equality (`phigrade_isequal`) and numpy array tolerance (`phigrade_allclose`)
+- **Flexible Deployment**: Local development server or remote backend integration
+- **Call Count Tracking**: Multiple assertions per test function with unique identification
+
+## Quick Start
+
+### Installation
+
+```bash
+# Install with Poetry
+poetry install
+
+# Or with pip
+pip install -e .
+```
+
+### Basic Usage
+
+1. **Create a configuration file** (`phigrade.yaml`):
+```yaml
+assignment_id: "homework_1"
+use_local_server: true
+teacher_mode: false
+timeout_seconds: 5
+```
+
+2. **Write test functions**:
+```python
+from phigrade.phigrade import phigrade_isequal, phigrade_allclose, weight
+import numpy as np
+
+@weight(5.0)
+def test_basic_math():
+    result = 2 + 2
+    phigrade_isequal(result)  # Compares against stored reference
+    return result
+
+@weight(10.0)
+def test_numpy_array():
+    result = np.array([1.0, 2.0, 3.0])
+    phigrade_allclose(result, rtol=1e-5, atol=1e-8)
+    return result
+```
+
+3. **Run tests**:
+```bash
+poetry run python your_test_file.py
+```
+
+## Architecture
+
+### Dual-Mode Workflow
+
+**Teacher Mode** (`teacher_mode: true`):
+- Instructors run tests on reference implementations
+- Results are stored as "correct" references with comparison metadata
+- Each test function with `@weight` decorator stores its output
+
+**Student Mode** (`teacher_mode: false`):
+- Students run the same test functions on their implementations
+- Their outputs are compared against stored references
+- Automatic scoring based on comparison results
+
+### Comparison Types
+
+#### Exact Equality (`phigrade_isequal`)
+```python
+@weight(3.0)
+def test_string_processing():
+    result = process_text("hello")
+    phigrade_isequal(result)  # Exact match required
+```
+
+#### Numpy Array Tolerance (`phigrade_allclose`)
+```python
+@weight(8.0)
+def test_numerical_computation():
+    result = np.array([1.0, 2.0, 3.0])
+    phigrade_allclose(result, rtol=1e-5, atol=1e-8, equal_nan=False)
+```
+
+### Multiple Calls Per Test
+
+```python
+@weight(15.0)
+def test_multiple_operations():
+    # Each call gets unique identifier: test_multiple_operations@0, @1, @2
+    result1 = operation_a()
+    phigrade_isequal(result1)
+    
+    result2 = operation_b()
+    phigrade_isequal(result2)
+    
+    array_result = operation_c()
+    phigrade_allclose(array_result, rtol=0.01)
+```
+
+## Configuration
+
+### Local Development Mode
+
+```yaml
+assignment_id: "my_assignment"
+use_local_server: true
+teacher_mode: false
+timeout_seconds: 10
+```
+
+### Remote Backend Mode
+
+```yaml
+assignment_id: "my_assignment"
+use_local_server: false
+server_url: "https://your-phigrade-backend.com"
+api_key: "your_api_key_here"
+teacher_mode: false
+timeout_seconds: 30
+```
+
+## Data Flow
+
+```
+┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
+│   Teacher Mode  │    │                  │    │  Student Mode   │
+│                 │    │  PhiGrade Server │    │                 │
+│ Store Reference │───▶│                  │◀───│ Submit Results  │
+│    Results      │    │  (Local/Remote)  │    │ Get Comparison  │
+└─────────────────┘    └──────────────────┘    └─────────────────┘
+```
+
+1. **Teacher Phase**: Instructor runs tests, stores reference outputs and comparison configuration
+2. **Student Phase**: Students run tests, submit outputs for comparison against references
+3. **Scoring**: Automatic scoring based on comparison results (exact match or tolerance-based)
+
+## Development
+
+### Running Tests
+
+```bash
+# Run all tests
+poetry run pytest
+
+# Run specific test categories
+poetry run pytest tests/test_phigrade_isequal_local.py  # Equality tests
+poetry run pytest tests/test_phigrade_allclose_local.py # Array tolerance tests
+poetry run pytest tests/test_local_server.py          # Local server tests
+poetry run pytest tests/test_real_backend_integration.py # Backend integration
+
+# Verbose output
+poetry run pytest -v
+```
+
+### Local Development Server
+
+```bash
+# Start local server manually
+poetry run python -m phigrade.app
+
+# Server starts automatically when use_local_server: true
+```
+
+### Project Structure
+
+```
+phigrade/
+├── phigrade/
+│   ├── __init__.py
+│   ├── phigrade.py          # Main API (decorators, assertions)
+│   ├── app.py             # Local FastAPI server
+│   ├── server.py          # Server management
+│   ├── compare.py         # Comparison logic
+│   └── config.py          # Configuration handling
+├── tests/
+│   ├── test_phigrade_isequal_local.py    # Equality comparison tests
+│   ├── test_phigrade_allclose_local.py   # Array tolerance tests
+│   ├── test_local_server.py             # Local server API tests
+│   └── test_real_backend_integration.py # Backend integration tests
+├── examples/              # Example usage
+├── phigrade.yaml          # Configuration file
+└── README.md
+```
+
+## Examples
+
+### Complete Teacher Workflow
+
+```python
+# teacher_tests.py
+from phigrade.phigrade import phigrade_isequal, phigrade_allclose, weight
+import numpy as np
+
+@weight(5.0)
+def test_factorial():
+    """Test factorial function"""
+    result = factorial(5)
+    phigrade_isequal(result)  # Stores 120 as reference
+    return result
+
+@weight(10.0)
+def test_matrix_multiplication():
+    """Test matrix operations"""
+    A = np.array([[1, 2], [3, 4]])
+    B = np.array([[2, 0], [1, 2]])
+    result = A @ B
+    phigrade_allclose(result, rtol=1e-10)  # Stores array with tight tolerance
+    return result
+```
+
+### Complete Student Workflow
+
+```python
+# student_tests.py (same function names)
+from phigrade.phigrade import phigrade_isequal, phigrade_allclose, weight
+import numpy as np
+
+@weight(5.0)
+def test_factorial():
+    """Student implementation"""
+    result = my_factorial(5)  # Student's implementation
+    phigrade_isequal(result)   # Compares against teacher's stored result
+    return result
+
+@weight(10.0)
+def test_matrix_multiplication():
+    """Student implementation"""
+    A = np.array([[1, 2], [3, 4]])
+    B = np.array([[2, 0], [1, 2]])
+    result = my_matrix_mult(A, B)  # Student's implementation
+    phigrade_allclose(result, rtol=1e-10)  # Compares with tolerance
+    return result
+```
+
+## API Reference
+
+### Decorators
+
+#### `@weight(points: float)`
+Decorates test functions to assign point values and enable phigrade tracking.
+
+### Assertion Functions
+
+#### `phigrade_isequal(output: Any) -> None`
+Compares output using exact equality (`==`).
+
+#### `phigrade_allclose(output: np.ndarray, rtol=1e-05, atol=1e-08, equal_nan=False) -> None`
+Compares numpy arrays with tolerance using `np.allclose()`.
+
+### Configuration
+
+#### `PhiGradeConfig`
+Loads configuration from YAML files with validation and type checking.
+
+## Contributing
+
+1. Fork the repository
+2. Create a feature branch
+3. Add tests for new functionality
+4. Ensure all tests pass: `poetry run pytest`
+5. Submit a pull request
+
+## License
+
+This project is licensed under the MIT License - see the LICENSE file for details.
+
+## Related Projects
+
+- **phigrade-backend**: Centralized backend server for multi-user deployments
+- **phigrade-frontend**: Web interface for instructors and administrators

phigrade-1.0.0/phigrade/__init__.py

@@ -0,0 +1,17 @@
+# Import all functions from phigrade module
+from .phigrade import (
+    weight,
+    phigrade_allclose,
+    phigrade_isequal,
+    PhiGradeConfig,
+    load_phigrade_config,
+)
+
+# Make everything available at package level
+__all__ = [
+    'weight',
+    'phigrade_allclose',
+    'phigrade_isequal',
+    'PhiGradeConfig',
+    'load_phigrade_config',
+]