@hivehub/rulebook 1.2.0

package/templates/languages/PYTHON.md

@@ -0,0 +1,682 @@
+<!-- PYTHON:START -->
+# Python Project Rules
+
+## Agent Automation Commands
+
+**CRITICAL**: Execute these commands after EVERY implementation (see AGENT_AUTOMATION module for full workflow).
+
+```bash
+# Complete quality check sequence:
+ruff format --check .      # Format check
+ruff check .               # Linting
+mypy .                     # Type checking
+pytest                     # All tests (100% pass required)
+pytest --cov               # Coverage check (95%+ required)
+
+# Security audit:
+pip-audit                  # Vulnerability scan
+pip list --outdated        # Check outdated deps
+```
+
+## Python Version
+
+**CRITICAL**: Use Python 3.11+ for modern features and performance.
+
+- **Minimum Version**: Python 3.11+
+- **Recommended**: Python 3.12+
+- **Type Hints**: Required for all public APIs
+
+### Formatting
+
+- Use `ruff format` (fast, modern) or `black` (traditional)
+- Line length: 100 characters (configurable)
+- Consistent formatting across entire project
+- Format before committing
+
+Configuration in `pyproject.toml`:
+```toml
+[tool.ruff]
+line-length = 100
+target-version = "py311"
+
+[tool.ruff.format]
+quote-style = "double"
+indent-style = "space"
+```
+
+### Linting
+
+- Use `ruff check` (fast, comprehensive) or `ruff` + `flake8`
+- Fix all linting errors before committing
+- Document any disabled rules with justification
+
+Configuration in `pyproject.toml`:
+```toml
+[tool.ruff.lint]
+select = ["E", "F", "I", "N", "W", "UP", "B", "A", "C4", "SIM"]
+ignore = ["E501"]  # Line too long (handled by formatter)
+
+[tool.ruff.lint.per-file-ignores]
+"tests/*" = ["S101"]  # Allow assert in tests
+```
+
+### Type Checking
+
+- Use `mypy` for static type checking
+- All public APIs must have type hints
+- Use `typing` module for complex types
+- Gradual typing allowed for legacy code
+
+Configuration in `pyproject.toml`:
+```toml
+[tool.mypy]
+python_version = "3.11"
+strict = true
+warn_return_any = true
+warn_unused_configs = true
+disallow_untyped_defs = true
+```
+
+Example:
+```python
+from typing import Optional, List, Dict, Any
+
+def process_data(
+    input_data: str,
+    options: Optional[Dict[str, Any]] = None
+) -> List[str]:
+    """Process input data and return results."""
+    # Implementation
+    return []
+```
+
+### Testing
+
+- **Framework**: pytest
+- **Location**: `/tests` directory
+- **Coverage**: Must meet project threshold (default 95%)
+- **Fixtures**: Use pytest fixtures for setup/teardown
+- **Parametrize**: Use `@pytest.mark.parametrize` for multiple test cases
+
+Example test structure:
+```python
+import pytest
+from mymodule import process_data
+
+@pytest.fixture
+def sample_data():
+    """Provide sample data for tests."""
+    return "test input"
+
+def test_process_data_valid_input(sample_data):
+    """Test process_data with valid input."""
+    result = process_data(sample_data)
+    assert result == ["expected"]
+
+@pytest.mark.parametrize("input_val,expected", [
+    ("hello", ["HELLO"]),
+    ("world", ["WORLD"]),
+])
+def test_process_data_parametrized(input_val, expected):
+    """Test multiple input cases."""
+    result = process_data(input_val)
+    assert result == expected
+```
+
+### Test Categories: S2S and Slow Tests
+
+**CRITICAL**: Tests must be categorized based on execution time and dependencies.
+
+#### Test Time Limits
+
+- **Fast Tests**: Must complete in ≤ 10-20 seconds
+- **Slow Tests**: Any test taking > 10-20 seconds must be marked as slow
+- **S2S Tests**: Tests requiring active server/database must be isolated and run on-demand
+
+#### S2S (Server-to-Server) Tests
+
+**Tests that require active servers, databases, or external services must be isolated using pytest markers.**
+
+**Implementation**:
+
+1. **Mark S2S tests with pytest markers**:
+```python
+import pytest
+import os
+
+# Regular fast test (always runs)
+def test_local_computation():
+    """Fast test, no external dependencies."""
+    result = compute_locally("input")
+    assert result == "expected"
+
+# S2S test (only runs with -m s2s)
+@pytest.mark.s2s
+def test_database_connection():
+    """Requires active database server."""
+    db = connect_to_database()
+    # ... test implementation
+
+@pytest.mark.s2s
+def test_api_integration():
+    """Requires active API server."""
+    client = create_api_client()
+    # ... test implementation
+```
+
+2. **Configure `pytest.ini` or `pyproject.toml`**:
+```ini
+# pytest.ini
+[pytest]
+markers =
+    s2s: Server-to-server tests requiring active services
+    slow: Slow tests taking > 20 seconds
+```
+
+Or in `pyproject.toml`:
+```toml
+[tool.pytest.ini_options]
+markers = [
+    "s2s: Server-to-server tests requiring active services",
+    "slow: Slow tests taking > 20 seconds",
+]
+```
+
+3. **Run tests**:
+```bash
+# Regular tests (excludes S2S)
+pytest
+
+# Include S2S tests (requires active servers)
+pytest -m s2s
+
+# Run all tests including S2S
+pytest -m "not slow"  # Fast + S2S, excludes slow
+```
+
+#### Slow Tests
+
+**Tests that take > 10-20 seconds must be marked and run separately.**
+
+**Implementation**:
+
+1. **Mark slow tests with pytest markers**:
+```python
+import pytest
+
+# Fast test (always runs)
+def test_quick_operation():
+    """Completes in < 1 second."""
+    result = quick_compute("input")
+    assert result == "expected"
+
+# Slow test (only runs with -m slow)
+@pytest.mark.slow
+def test_heavy_computation():
+    """Takes 30+ seconds."""
+    # Heavy processing, large dataset, etc.
+    result = process_large_dataset()
+    assert result is not None
+
+@pytest.mark.slow
+def test_large_file_processing():
+    """Processes large files, takes > 20 seconds."""
+    result = process_file("large_file.dat")
+    assert result.success
+```
+
+2. **Run tests**:
+```bash
+# Regular tests (excludes slow and S2S)
+pytest -m "not slow and not s2s"
+
+# Include slow tests
+pytest -m slow
+
+# Run all tests
+pytest -m ""  # Empty marker means all tests
+```
+
+3. **Add pytest configuration for timeouts**:
+```python
+# conftest.py
+import pytest
+
+@pytest.fixture(autouse=True)
+def configure_timeouts(request):
+    """Configure timeouts based on test markers."""
+    if 'slow' in request.keywords:
+        request.node.add_marker(pytest.mark.timeout(300))  # 5 minutes
+    elif 's2s' in request.keywords:
+        request.node.add_marker(pytest.mark.timeout(60))  # 1 minute
+    else:
+        request.node.add_marker(pytest.mark.timeout(20))  # 20 seconds
+```
+
+4. **Add scripts in `pyproject.toml` or `setup.py`**:
+```toml
+[tool.poetry.scripts]
+test = "pytest -m 'not slow and not s2s'"
+test-s2s = "pytest -m s2s"
+test-slow = "pytest -m slow"
+test-all = "pytest"
+```
+
+#### Best Practices
+
+- ✅ **Always run fast tests** in CI/CD by default
+- ✅ **Isolate S2S tests** - never run them in standard test suite
+- ✅ **Mark slow tests** - prevent CI/CD timeouts
+- ✅ **Document requirements** - specify which servers/services are needed for S2S tests
+- ✅ **Use timeouts** - Set appropriate timeouts: `@pytest.mark.timeout(60)`
+- ✅ **Use pytest markers** - `@pytest.mark.s2s` and `@pytest.mark.slow`
+- ✅ **Skip conditionally** - `@pytest.mark.skipif(not os.getenv('RUN_S2S_TESTS'), reason='S2S tests disabled')`
+- ❌ **Never mix** fast and slow/S2S tests in same test run
+- ❌ **Never require** external services for standard test suite
+- ❌ **Never exceed** 10-20 seconds for regular tests
+
+## Dependency Management
+
+**CRITICAL**: Use modern dependency management tools.
+
+### Recommended: Poetry
+
+```toml
+[tool.poetry]
+name = "myproject"
+version = "0.1.0"
+description = ""
+authors = ["Your Name <you@example.com>"]
+
+[tool.poetry.dependencies]
+python = "^3.11"
+requests = "^2.31.0"
+
+[tool.poetry.group.dev.dependencies]
+pytest = "^7.4.0"
+mypy = "^1.5.0"
+ruff = "^0.1.0"
+```
+
+Commands:
+```bash
+poetry install              # Install dependencies
+poetry add requests         # Add dependency
+poetry add --group dev pytest  # Add dev dependency
+poetry update               # Update dependencies
+```
+
+### Alternative: pip-tools
+
+```
+# requirements.in
+requests>=2.31.0
+pydantic>=2.0.0
+
+# requirements-dev.in
+-r requirements.in
+pytest>=7.4.0
+mypy>=1.5.0
+```
+
+Commands:
+```bash
+pip-compile requirements.in
+pip-compile requirements-dev.in
+pip-sync requirements-dev.txt
+```
+
+### Dependency Guidelines
+
+1. **Check for latest versions**:
+   - Use Context7 MCP tool if available
+   - Check PyPI: `pip index versions <package>`
+   - Review changelog for breaking changes
+
+2. **Version pinning**:
+   - ✅ Pin exact versions in applications
+   - ✅ Use ranges in libraries (`>=1.0,<2.0`)
+   - ✅ Keep dependencies updated regularly
+   - ❌ Don't use outdated packages with security issues
+
+## Error Handling
+
+- Use specific exception types
+- Create custom exceptions when needed
+- Document exceptions in docstrings
+- Never use bare `except:`
+
+Example:
+```python
+class ValidationError(Exception):
+    """Raised when data validation fails."""
+    
+    def __init__(self, message: str, field: str):
+        super().__init__(message)
+        self.field = field
+
+def validate_data(data: dict[str, Any]) -> None:
+    """
+    Validate input data.
+    
+    Args:
+        data: The data to validate
+        
+    Raises:
+        ValidationError: If validation fails
+    """
+    if not isinstance(data, dict):
+        raise ValidationError("Data must be a dictionary", "data")
+```
+
+## Documentation
+
+- **Docstrings**: Google or NumPy style
+- **Type hints**: Required for public APIs
+- **README**: Include installation and usage
+- **API docs**: Consider Sphinx for large projects
+
+Example (Google style):
+```python
+def process_data(input_data: str, options: dict[str, Any] | None = None) -> list[str]:
+    """
+    Process input data and return results.
+    
+    Args:
+        input_data: The input string to process
+        options: Optional processing options
+        
+    Returns:
+        A list of processed strings
+        
+    Raises:
+        ValidationError: If input_data is empty
+        
+    Examples:
+        >>> process_data("hello")
+        ['HELLO']
+        >>> process_data("world", {"lowercase": True})
+        ['world']
+    """
+    # Implementation
+    return []
+```
+
+## Project Structure
+
+```
+project/
+├── pyproject.toml      # Project metadata and dependencies
+├── README.md           # Project overview (allowed in root)
+├── CHANGELOG.md        # Version history (allowed in root)
+├── AGENTS.md          # AI assistant rules (allowed in root)
+├── LICENSE            # Project license (allowed in root)
+├── CONTRIBUTING.md    # Contribution guidelines (allowed in root)
+├── CODE_OF_CONDUCT.md # Code of conduct (allowed in root)
+├── SECURITY.md        # Security policy (allowed in root)
+├── src/
+│   └── mypackage/
+│       ├── __init__.py
+│       ├── module.py
+│       └── py.typed    # PEP 561 marker for type hints
+├── tests/              # Test files
+│   ├── __init__.py
+│   └── test_module.py
+└── docs/               # Documentation
+```
+
+## Async Programming
+
+- Use `asyncio` for async code
+- Type hints: `async def func() -> Coroutine`
+- Testing: Use `pytest-asyncio`
+
+Example:
+```python
+import asyncio
+from typing import List
+
+async def fetch_data(url: str) -> dict[str, Any]:
+    """Fetch data asynchronously."""
+    # Implementation
+    return {}
+
+async def main() -> None:
+    """Main async function."""
+    results = await asyncio.gather(
+        fetch_data("url1"),
+        fetch_data("url2"),
+    )
+    print(results)
+
+if __name__ == "__main__":
+    asyncio.run(main())
+```
+
+## CI/CD Requirements
+
+Must include GitHub Actions workflows for:
+
+1. **Testing** (`python-test.yml`):
+   - Test on ubuntu-latest, windows-latest, macos-latest
+   - Test on Python 3.11, 3.12
+   - Upload coverage reports
+
+2. **Linting** (`python-lint.yml`):
+   - Format check: `ruff format --check .`
+   - Lint: `ruff check .`
+   - Type check: `mypy .`
+
+3. **Security** (`python-security.yml`):
+   - Check for vulnerabilities: `pip-audit`
+   - Scan dependencies: `safety check`
+
+## Package Publication
+
+### Publishing to PyPI
+
+**Prerequisites:**
+1. Create account at https://pypi.org
+2. Enable 2FA for security
+3. Configure trusted publishing (recommended) or create API token
+4. For trusted publishing: Add GitHub as publisher in PyPI settings
+
+**pyproject.toml Configuration:**
+
+```toml
+[build-system]
+requires = ["setuptools>=68.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "your-package-name"
+version = "1.0.0"
+description = "A short description of your package"
+readme = "README.md"
+requires-python = ">=3.11"
+license = {text = "MIT"}
+authors = [
+    {name = "Your Name", email = "your.email@example.com"}
+]
+keywords = ["your", "keywords"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+]
+dependencies = [
+    "requests>=2.31.0",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=7.4.0",
+    "pytest-cov>=4.1.0",
+    "ruff>=0.1.0",
+    "mypy>=1.7.0",
+    "black>=23.12.0",
+]
+
+[project.urls]
+Homepage = "https://github.com/your-org/your-package"
+Documentation = "https://your-package.readthedocs.io"
+Repository = "https://github.com/your-org/your-package"
+"Bug Tracker" = "https://github.com/your-org/your-package/issues"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+your_package = ["py.typed"]
+```
+
+### PEP 625 Package Naming Convention
+
+**CRITICAL**: Package names must be normalized according to PEP 625.
+
+PyPI requires source distribution filenames to use normalized package names (underscores instead of hyphens).
+
+**Naming Rules:**
+
+1. **Package name in `pyproject.toml`**: Use underscores (`_`)
+   ```toml
+   [project]
+   name = "my_package_name"  # Correct
+   # NOT: name = "my-package-name"  # Will cause deprecation warning
+   ```
+
+2. **Package directory**: Must match with underscores
+   ```
+   src/
+   └── my_package_name/     # Correct
+       ├── __init__.py
+       └── ...
+   ```
+
+3. **Import statement**: Uses underscores
+   ```python
+   import my_package_name
+   from my_package_name import something
+   ```
+
+4. **Distribution filename**: Will be `my_package_name-1.0.0.tar.gz` ✅
+
+**Common Issue:**
+
+If you use hyphens in the package name, PyPI will reject new uploads:
+```toml
+# ❌ WRONG - Will fail PEP 625 compliance
+[project]
+name = "my-package-name"
+
+# Result: my-package-name-1.0.0.tar.gz (non-compliant)
+# PyPI Error: "Filename does not contain normalized project name"
+```
+
+**Correct Approach:**
+```toml
+# ✅ CORRECT - PEP 625 compliant
+[project]
+name = "my_package_name"
+
+# Result: my_package_name-1.0.0.tar.gz (compliant)
+# PyPI: Accepts upload without warnings
+```
+
+**Migration from Hyphenated Names:**
+
+If you previously published with hyphens:
+
+1. Update `pyproject.toml` and `setup.py` to use underscores
+2. Existing uploads remain on PyPI (no action needed)
+3. Future uploads will use normalized name
+4. PyPI will automatically redirect:
+   - `pip install my-package-name` → works (auto-normalized)
+   - `pip install my_package_name` → works (canonical form)
+5. Import statement unchanged: `import my_package_name`
+
+**Reference**: [PEP 625 - File name of a Source Distribution](https://peps.python.org/pep-0625/)
+
+**Publishing Workflow:**
+
+1. Update version in pyproject.toml
+2. Update CHANGELOG.md
+3. Run quality checks:
+   ```bash
+   ruff check .
+   ruff format --check .
+   mypy .
+   pytest
+   ```
+4. Build package:
+   ```bash
+   python -m build
+   twine check dist/*
+   ```
+5. Test on Test PyPI (optional):
+   ```bash
+   twine upload --repository testpypi dist/*
+   ```
+6. Create git tag: `git tag v1.0.0 && git push --tags`
+7. GitHub Actions automatically publishes to PyPI
+8. Or manual publish: `twine upload dist/*`
+
+**Publishing Checklist:**
+
+- ✅ All tests passing (`pytest`)
+- ✅ Type checking passes (`mypy .`)
+- ✅ Linting passes (`ruff check .`)
+- ✅ Code formatted (`ruff format .`)
+- ✅ Version updated in pyproject.toml
+- ✅ CHANGELOG.md updated
+- ✅ README.md up to date
+- ✅ LICENSE file present
+- ✅ **Package name uses underscores (PEP 625 compliant)**
+- ✅ `py.typed` marker for type hints
+- ✅ Package builds successfully (`python -m build`)
+- ✅ Package checks pass (`twine check dist/*`)
+- ✅ Manifest complete (`check-manifest`)
+- ✅ **Verify dist filename**: `my_package-1.0.0.tar.gz` (underscores) ✅
+
+**Trusted Publishing (Recommended):**
+
+PyPI trusted publishing eliminates the need for API tokens:
+
+1. Go to PyPI → Your Account → Publishing
+2. Add a new pending publisher:
+   - PyPI Project Name: `your-package-name`
+   - Owner: `your-github-org`
+   - Repository: `your-repo-name`
+   - Workflow: `python-publish.yml`
+   - Environment: `release` (optional)
+
+3. GitHub Actions will authenticate automatically using OIDC
+
+**Versioning:**
+
+Use semantic versioning and consider:
+- **Automated versioning**: Use tools like `bump2version` or `setuptools_scm`
+- **Version from git tags**: Configure `setuptools_scm` in pyproject.toml:
+
+```toml
+[build-system]
+requires = ["setuptools>=68.0", "setuptools_scm>=8.0"]
+
+[tool.setuptools_scm]
+version_file = "src/your_package/_version.py"
+```
+
+**Type Hints:**
+
+Include `py.typed` marker for PEP 561 compliance:
+```bash
+touch src/your_package/py.typed
+```
+
+This tells type checkers your package includes type information.
+
+<!-- PYTHON:END -->
+