httpr 0.1.17__tar.gz

httpr-0.1.17/.github/copilot-instructions.md

@@ -0,0 +1,240 @@
+# httpr - AI Coding Agent Instructions
+
+## Project Overview
+
+**httpr** is a high-performance HTTP client for Python built in Rust using PyO3 and reqwest. It's designed as a drop-in replacement for `httpx` and `requests` with significantly better performance through Rust's native speed.
+
+### Directory Structure
+
+```
+httpr/
+├── .github/           # GitHub workflows and configurations
+│   ├── workflows/     # CI/CD pipelines (testing, building, releasing)
+│   └── copilot-instructions.md  # This file
+├── src/              # Rust source code
+│   ├── lib.rs        # Main RClient class, PyO3 module definition
+│   ├── response.rs   # Response object with CaseInsensitiveHeaderMap
+│   ├── traits.rs     # Python↔Rust type conversion traits
+│   └── utils.rs      # CA certificates, encoding detection
+├── httpr/            # Python package
+│   ├── __init__.py   # Client and AsyncClient classes
+│   ├── httpr.pyi     # Type stubs for IDE support
+│   └── py.typed      # PEP 561 marker for type checking
+├── tests/            # Python tests using pytest
+├── benchmark/        # Performance benchmarking suite
+├── docs/             # MkDocs documentation
+├── Cargo.toml        # Rust dependencies and build config
+├── pyproject.toml    # Python project metadata and dependencies
+└── README.md         # User-facing documentation
+```
+
+### Architecture
+
+- **Rust Core** (`src/`): PyO3-based HTTP client wrapping reqwest
+  - `lib.rs`: Main `RClient` class with sync request handling via Tokio single-threaded runtime
+  - `response.rs`: `Response` object with `CaseInsensitiveHeaderMap` for HTTP headers
+  - `traits.rs`: Conversion traits between Python/Rust types (IndexMap ↔ HeaderMap)
+  - `utils.rs`: CA certificate loading, encoding detection
+- **Python Wrapper** (`httpr/`): Thin Python layer over Rust bindings
+  - `__init__.py`: `Client` (sync) and `AsyncClient` classes with context manager support
+  - `AsyncClient` uses `asyncio.run_in_executor()` to wrap sync Rust calls - NOT native async
+  - `httpr.pyi`: Type stubs for IDE support
+- **Build System**: Maturin (PyO3 build tool) compiles Rust → Python wheels
+
+### Key Architectural Decisions
+
+1. **Single Tokio Runtime**: Uses `LazyLock<Runtime>` with `new_current_thread()` - all async operations run on one thread
+2. **Async is Sync**: `AsyncClient` runs sync Rust code in thread executor, NOT native async Rust
+3. **Case-Insensitive Headers**: Custom `CaseInsensitiveHeaderMap` struct maintains original casing while allowing case-insensitive lookups (required for HTTP/2)
+4. **Zero Python Dependencies**: All functionality in Rust; no runtime Python dependencies
+
+### Key Dependencies
+
+**Rust (Cargo.toml)**:
+- `pyo3` (0.23.4): Python bindings, enables calling Rust from Python
+- `reqwest` (0.12): HTTP client library, provides core networking functionality
+- `tokio` (1.43): Async runtime for Rust (single-threaded mode)
+- `anyhow`: Error handling with context
+- `indexmap` + `foldhash`: Fast hash maps for headers/params
+- `rustls-tls`: TLS implementation (no OpenSSL dependency)
+- Compression: `gzip`, `brotli`, `zstd`, `deflate` support
+
+**Python (pyproject.toml)**:
+- **Zero runtime dependencies** - all functionality in Rust
+- Dev dependencies: `pytest`, `pytest-asyncio`, `mypy`, `ruff`, `maturin`
+- Supports Python 3.9+
+
+## Development Workflows
+
+### Building & Testing
+
+```bash
+# Install development dependencies (includes maturin)
+uv sync --extra dev
+
+# Build Rust extension in development mode (fast compilation)
+uv run maturin develop
+
+# Run tests (requires httpbin.org access)
+uv run pytest tests/
+
+# Type checking
+uv run mypy httpr/
+
+# Linting
+uv run ruff check httpr/
+```
+
+### Benchmarking
+
+```bash
+cd benchmark/
+uv run uvicorn server:app  # Terminal 1: Start test server
+uv run python benchmark.py  # Terminal 2: Run benchmarks
+```
+
+The benchmark compares httpr against requests, httpx, curl_cffi, pycurl, tls_client, and aiohttp.
+
+### Release Process
+
+1. Tag with version: `git tag v0.1.x`
+2. Push tag: `git push origin v0.1.x`
+3. CI auto-builds wheels for Linux (x86_64, aarch64, armv7), macOS (Intel/ARM), Windows
+4. Maturin publishes to PyPI
+
+### CI/CD
+
+**Workflows** (`.github/workflows/`):
+- `CI.yml`: Main CI pipeline - runs tests, linting, type checking on pull requests
+- `mkdocs.yml`: Documentation deployment to GitHub Pages
+- Platform matrix testing: Linux (multiple architectures), macOS (Intel/ARM), Windows
+- Automated wheel building via `maturin` for all platforms
+- Test suite runs against live httpbin.org endpoints
+
+### Debugging
+
+**Rust Debugging**:
+```bash
+# Enable Rust logging
+RUST_LOG=debug uv run pytest tests/test_client.py -v
+
+# Build with debug symbols (slower but debuggable)
+uv run maturin develop --release=false
+```
+
+**Python Debugging**:
+```python
+# Standard Python debugging works
+import httpr
+client = httpr.Client()
+breakpoint()  # Python debugger
+response = client.get("https://httpbin.org/get")
+```
+
+**Common Issues**:
+- "Module not found" after Rust changes → Run `uv run maturin develop`
+- SSL errors → Check `HTTPR_CA_BUNDLE` environment variable
+- Test failures → httpbin.org may be down; tests use `@retry()` decorator
+- Import errors → Ensure virtual environment is activated
+
+## Project-Specific Conventions
+
+### Python-Rust Interface Patterns
+
+- **All Python params converted to strings**: `Client.request()` converts param values to strings before passing to Rust
+- **Method validation in Python**: HTTP method validation happens in Python wrapper, not Rust
+- **Optional chaining**: Client stores default `auth`, `params`, `headers` - per-request values override via `.or()`
+
+### Type System
+
+- Rust uses `IndexMap<String, String, RandomState>` (foldhash) for dicts
+- Python type hints via `TypedDict` (`RequestParams`, `ClientRequestParams`)
+- Use `Unpack` for `**kwargs` typing (supports Python 3.9+ via typing_extensions)
+
+### Error Handling
+
+- Rust errors bubble up via `anyhow::Result` → PyO3 exceptions
+- No fine-grained error types yet (all raise generic `Exception`)
+- Tests use `@retry()` decorator to handle flaky httpbin.org responses
+
+### Testing Patterns
+
+```python
+# Sync tests
+def test_client_feature():
+    client = httpr.Client(auth=("user", "pass"))
+    response = client.get("https://httpbin.org/anything")
+    assert response.status_code == 200
+
+# Async tests
+@pytest.mark.asyncio
+async def test_asyncclient_feature():
+    async with httpr.AsyncClient() as client:
+        response = await client.get("https://httpbin.org/anything")
+        assert response.status_code == 200
+```
+
+Tests depend on httpbin.org for validation - use `@retry()` wrapper for flaky network requests.
+
+## Critical Implementation Details
+
+### SSL/TLS Configuration
+
+- CA certs loaded via `HTTPR_CA_BUNDLE` env var (set before client init)
+- `ca_cert_file` param internally sets `HTTPR_CA_BUNDLE`
+- mTLS via `client_pem` parameter (PEM format identity file)
+- `verify=False` enables `danger_accept_invalid_certs()`
+
+### Headers Behavior
+
+- Headers are **lowercased** internally (HTTP/2 requirement)
+- `client.headers` getter excludes `Cookie` header
+- `client.cookies` getter/setter extracts from `Cookie` header
+- Use `CaseInsensitiveHeaderMap` in Rust for lookups
+
+### Request Body Handling
+
+- Mutually exclusive: `content` (bytes), `data` (form), `json` (JSON), `files` (multipart)
+- `data` and `json` use `pythonize::depythonize()` for Python → Rust serde_json::Value
+- `files` dict maps field names to file paths (auto-reads and sends as multipart)
+
+### Proxy Support
+
+- Set via `proxy` param or `HTTPR_PROXY` env var
+- Changing `client.proxy` rebuilds entire reqwest client (expensive operation)
+
+## Common Tasks
+
+### Adding New Request Parameters
+
+1. Add to `RequestParams` TypedDict in `httpr.pyi`
+2. Add parameter to `RClient.request()` signature in `src/lib.rs`
+3. Handle parameter in request builder chain
+4. Update docstrings in both Python and Rust
+
+### Modifying Response Properties
+
+1. Add Rust implementation in `src/response.rs`
+2. Add `#[pymethods]` getter/method
+3. Add type hint to `Response` class in `httpr.pyi`
+
+### Performance Optimization
+
+- Check `Cargo.toml` for reqwest feature flags (enable compression codecs, http2, etc.)
+- Profile with `cargo flamegraph` on Rust side
+- Benchmark against other clients using `benchmark/benchmark.py`
+
+## What NOT to Do
+
+- Don't add Python dependencies (defeats "zero dependencies" goal)
+- Don't use native async Rust in request path (breaks single-threaded runtime model)
+- Don't modify headers case-sensitivity behavior (HTTP/2 spec requirement)
+- Don't remove reqwest compression features (users expect automatic gzip/brotli)
+- Don't skip `maturin develop` after Rust changes (Python won't see updates)
+
+## Resources
+
+- [PyO3 User Guide](https://pyo3.rs/)
+- [reqwest docs](https://docs.rs/reqwest/)
+- [Maturin Guide](https://www.maturin.rs/)
+- Benchmark results: `benchmark.jpg` in repo root

httpr-0.1.17/.github/workflows/CI.yml

@@ -0,0 +1,386 @@
+name: CI
+
+on:
+  push:
+    tags:
+      - "*"
+  pull_request:
+  schedule:
+    # Run weekly on Sundays at midnight UTC
+    - cron: "0 0 * * 0"
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+jobs:
+  # Fast test job - runs on PRs with Python version matrix
+  test:
+    runs-on: ubuntu-22.04
+    if: github.event_name == 'pull_request'
+    strategy:
+      matrix:
+        python-version: ["3.10", "3.11", "3.12", "3.13"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v7
+        with:
+          python-version: ${{ matrix.python-version }}
+          enable-cache: false
+      - name: Build and test
+        run: |
+          uv sync --extra dev
+          uv run maturin develop
+          uv run pytest
+
+  # Docs build test - runs on PRs to catch broken docs early
+  docs:
+    runs-on: ubuntu-latest
+    if: github.event_name == 'pull_request'
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v7
+        with:
+          python-version: "3.12"
+          enable-cache: false
+      - name: Build docs
+        run: |
+          uv sync --extra docs
+          uv run mkdocs build --strict
+
+  # Build jobs - run on tags, weekly schedule, or manual dispatch
+  linux:
+    runs-on: ${{ matrix.platform.runner }}
+    if: github.event_name != 'pull_request'
+    strategy:
+      matrix:
+        platform:
+          - runner: ubuntu-22.04
+            target: x86_64
+          - runner: ubuntu-22.04
+            target: x86
+          - runner: ubuntu-24.04-arm
+            target: aarch64
+          - runner: ubuntu-22.04
+            target: armv7
+          # - runner: ubuntu-22.04
+          #   target: s390x
+          # - runner: ubuntu-22.04
+          #   target: ppc64le
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v7
+        with:
+          python-version: "3.12"
+          enable-cache: false
+      # Only modify pyproject.toml temporarily for this job if running from a tag
+      - name: Update version in pyproject.toml
+        if: ${{ startsWith(github.ref, 'refs/tags/') }}
+        run: |
+          TAG_VERSION=${GITHUB_REF#refs/tags/v}
+          uvx --with toml python .github/workflows/set_version.py $TAG_VERSION
+      - name: Build wheels
+        uses: PyO3/maturin-action@v1
+        with:
+          target: ${{ matrix.platform.target }}
+          args: --release --out dist
+          sccache: ${{ !startsWith(github.ref, 'refs/tags/') }}
+          manylinux: auto
+      - name: Build free-threaded wheels
+        uses: PyO3/maturin-action@v1
+        with:
+          target: ${{ matrix.platform.target }}
+          args: --release --out dist -i python3.13t
+          sccache: ${{ !startsWith(github.ref, 'refs/tags/') }}
+          manylinux: auto
+      - name: Upload wheels
+        uses: actions/upload-artifact@v4
+        with:
+          name: wheels-linux-${{ matrix.platform.target }}
+          path: dist
+      - name: pytest
+        if: ${{ matrix.platform.target == 'x86_64' || matrix.platform.target == 'aarch64' }}
+        shell: bash
+        env:
+          RUSTC_WRAPPER: ""
+        run: |
+          set -e
+          uv sync --extra dev --find-links dist --reinstall
+          uv run pytest
+      - name: pytest
+        if: ${{ matrix.platform.target == 'armv7' }}
+        uses: uraimo/run-on-arch-action@v2
+        with:
+          arch: ${{ matrix.platform.target }}
+          distro: ubuntu22.04
+          githubToken: ${{ github.token }}
+          install: |
+            apt-get update
+            apt-get install -y --no-install-recommends python3 python3-pip python3-dev build-essential libffi-dev
+            pip3 install -U pip
+          run: |
+            set -e
+            pip3 install httpr --find-links dist --no-index --force-reinstall
+            pip3 install certifi pytest pytest-asyncio pytest-httpbin mypy ruff maturin trustme
+            pytest
+
+  musllinux:
+    runs-on: ${{ matrix.platform.runner }}
+    if: github.event_name != 'pull_request'
+    strategy:
+      matrix:
+        platform:
+          - runner: ubuntu-22.04
+            target: x86_64
+          - runner: ubuntu-22.04
+            target: x86
+          - runner: ubuntu-24.04-arm
+            target: aarch64
+          # - runner: ubuntu-22.04
+          #   target: armv7
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v7
+        with:
+          python-version: "3.12"
+          enable-cache: false
+      # Only modify pyproject.toml temporarily for this job if running from a tag
+      - name: Update version in pyproject.toml
+        if: ${{ startsWith(github.ref, 'refs/tags/') }}
+        run: |
+          TAG_VERSION=${GITHUB_REF#refs/tags/v}
+          uvx --with toml python .github/workflows/set_version.py $TAG_VERSION
+      - name: Build wheels
+        uses: PyO3/maturin-action@v1
+        with:
+          target: ${{ matrix.platform.target }}
+          args: --release --out dist
+          sccache: ${{ !startsWith(github.ref, 'refs/tags/') }}
+          manylinux: musllinux_1_2
+      - name: Build free-threaded wheels
+        uses: PyO3/maturin-action@v1
+        with:
+          target: ${{ matrix.platform.target }}
+          args: --release --out dist -i python3.13t
+          sccache: ${{ !startsWith(github.ref, 'refs/tags/') }}
+          manylinux: musllinux_1_2
+      - name: Upload wheels
+        uses: actions/upload-artifact@v4
+        with:
+          name: wheels-musllinux-${{ matrix.platform.target }}
+          path: dist
+      - name: pytest
+        if: ${{ matrix.platform.target == 'x86_64' || matrix.platform.target == 'aarch64' }}
+        uses: addnab/docker-run-action@v3
+        with:
+          image: alpine:latest
+          options: -v ${{ github.workspace }}:/io -w /io
+          run: |
+            set -e
+            apk add py3-pip py3-virtualenv python3-dev build-base libffi-dev
+            python3 -m virtualenv .venv
+            source .venv/bin/activate
+            pip install httpr --find-links dist --no-index --force-reinstall
+            pip install certifi pytest pytest-asyncio pytest-httpbin mypy ruff maturin trustme
+            pytest
+
+  windows:
+    runs-on: ${{ matrix.platform.runner }}
+    if: github.event_name != 'pull_request'
+    strategy:
+      matrix:
+        platform:
+          - runner: windows-latest
+            target: x64
+          - runner: windows-latest
+            target: x86
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+          architecture: ${{ matrix.platform.target }}
+      - uses: astral-sh/setup-uv@v7
+        with:
+          python-version: "3.12"
+          enable-cache: false
+      # Only modify pyproject.toml temporarily for this job if running from a tag
+      - name: Update version in pyproject.toml
+        if: ${{ startsWith(github.ref, 'refs/tags/') }}
+        shell: pwsh
+        run: |
+          $TAG_VERSION = $env:GITHUB_REF -replace 'refs/tags/v', ''
+          uvx --with toml python .github/workflows/set_version.py $TAG_VERSION
+      - name: Build wheels
+        uses: PyO3/maturin-action@v1
+        with:
+          target: ${{ matrix.platform.target }}
+          args: --release --out dist -i python3.12
+          sccache: ${{ !startsWith(github.ref, 'refs/tags/') }}
+      - uses: actions/setup-python@v5
+        with:
+          python-version: 3.13t
+          architecture: ${{ matrix.platform.target }}
+      - name: Build free-threaded wheels
+        uses: PyO3/maturin-action@v1
+        with:
+          target: ${{ matrix.platform.target }}
+          args: --release --out dist -i python3.13t
+          sccache: ${{ !startsWith(github.ref, 'refs/tags/') }}
+      - name: Upload wheels
+        uses: actions/upload-artifact@v4
+        with:
+          name: wheels-windows-${{ matrix.platform.target }}
+          path: dist
+      - name: pytest
+        shell: bash
+        env:
+          RUSTC_WRAPPER: ""
+        run: |
+          set -e
+          uv sync --extra dev --find-links dist --reinstall
+          uv run pytest
+
+  macos:
+    runs-on: ${{ matrix.platform.runner }}
+    if: github.event_name != 'pull_request'
+    strategy:
+      matrix:
+        platform:
+          - runner: macos-15
+            target: x86_64
+          - runner: macos-14
+            target: aarch64
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v7
+        with:
+          python-version: "3.12"
+          enable-cache: false
+      # Only modify pyproject.toml temporarily for this job if running from a tag
+      - name: Update version in pyproject.toml
+        if: ${{ startsWith(github.ref, 'refs/tags/') }}
+        run: |
+          TAG_VERSION=${GITHUB_REF#refs/tags/v}
+          uvx --with toml python .github/workflows/set_version.py $TAG_VERSION
+      - name: Build wheels
+        uses: PyO3/maturin-action@v1
+        with:
+          target: ${{ matrix.platform.target }}
+          args: --release --out dist
+          sccache: ${{ !startsWith(github.ref, 'refs/tags/') }}
+      - name: Build free-threaded wheels
+        uses: PyO3/maturin-action@v1
+        with:
+          target: ${{ matrix.platform.target }}
+          args: --release --out dist -i python3.13t
+          sccache: ${{ !startsWith(github.ref, 'refs/tags/') }}
+      - name: Upload wheels
+        uses: actions/upload-artifact@v4
+        with:
+          name: wheels-macos-${{ matrix.platform.target }}
+          path: dist
+      - name: pytest
+        env:
+          RUSTC_WRAPPER: ""
+        run: |
+          set -e
+          uv sync --extra dev --find-links dist --reinstall
+          uv run pytest
+
+  sdist:
+    runs-on: ubuntu-latest
+    if: github.event_name != 'pull_request'
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v7
+        with:
+          python-version: "3.12"
+          enable-cache: false
+      # Only modify pyproject.toml temporarily for this job if running from a tag
+      - name: Update version in pyproject.toml
+        if: ${{ startsWith(github.ref, 'refs/tags/') }}
+        run: |
+          TAG_VERSION=${GITHUB_REF#refs/tags/v}
+          uvx --with toml python .github/workflows/set_version.py $TAG_VERSION
+      - name: Build sdist
+        uses: PyO3/maturin-action@v1
+        with:
+          command: sdist
+          args: --out dist
+      - name: Upload sdist
+        uses: actions/upload-artifact@v4
+        with:
+          name: wheels-sdist
+          path: dist
+
+  release:
+    name: Release
+    environment: pypi
+    runs-on: ubuntu-latest
+    if: ${{ startsWith(github.ref, 'refs/tags/') || github.event_name == 'workflow_dispatch' }}
+    needs: [linux, musllinux, windows, macos, sdist]
+    permissions:
+      # Use to sign the release artifacts
+      id-token: write
+      # Used to upload release artifacts
+      contents: write
+      # Used to generate artifact attestation
+      attestations: write
+    steps:
+      - uses: actions/download-artifact@v4
+      - name: Generate artifact attestation
+        uses: actions/attest-build-provenance@v2
+        with:
+          subject-path: "wheels-*/*"
+      - name: Publish to PyPI
+        if: ${{ startsWith(github.ref, 'refs/tags/') }}
+        uses: PyO3/maturin-action@v1
+        env:
+          MATURIN_PYPI_TOKEN: ${{ secrets.PYPI_API_TOKEN }}
+        with:
+          command: upload
+          args: --non-interactive --skip-existing wheels-*/*
+
+  benchmark:
+    permissions:
+      contents: write
+    runs-on: ubuntu-latest
+    needs: [linux]
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+      - uses: astral-sh/setup-uv@v7
+        with:
+          python-version: "3.12"
+          enable-cache: false
+      - name: Download wheels
+        uses: actions/download-artifact@v4
+        with:
+          name: wheels-linux-x86_64
+      - name: Install dependencies
+        run: uv sync --package benchmark --find-links ./ --reinstall
+      - name: Start Uvicorn server
+        run: |
+          uv run --package benchmark uvicorn benchmark.server:app --host 0.0.0.0 --port 8000 &
+          sleep 10
+      - name: Run benchmark
+        run: uv run --package benchmark python benchmark/benchmark.py
+      - name: Generate and commit benchmark image
+        if: startsWith(github.ref, 'refs/tags/') || github.event_name == 'schedule'
+        run: |
+          uv run --package benchmark python benchmark/generate_image.py
+          git config --global user.name 'github-actions[bot]'
+          git config --global user.email '41898282+github-actions[bot]@users.noreply.github.com'
+          git add *.jpg
+          if ! git diff --quiet --cached; then
+            if [ "${{ github.event_name }}" = "schedule" ]; then
+              git commit -m "Update benchmark image (weekly)"
+            else
+              git commit -m "Update benchmark image for ${{ github.ref_name }}"
+            fi
+            git push
+          else
+            echo "No changes to commit"
+          fi