ccproxy-api 0.1.3__tar.gz → 0.1.5__tar.gz

ccproxy_api-0.1.5/CHANGELOG.md

@@ -0,0 +1,255 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+### Added
+
+- **Advanced Session and Pool Management**:
+    - Implemented a robust session-aware pool for persistent Claude SDK connections, significantly improving performance and maintaining conversation continuity.
+    - Introduced a hybrid pooling system that automatically transfers clients from a general pool to the session pool upon receiving a session ID.
+    - Developed a queue-based streaming architecture to efficiently handle and broadcast messages to multiple listeners, improving session management and disconnection handling.
+- **Enhanced Observability and Logging**:
+    - Upgraded logging capabilities to include detailed session metadata, providing deeper insights into session lifecycle and reuse.
+    - Implemented a pool monitoring system to track the health and performance of both general and session-based connection pools.
+    - Reduced logging noise by adjusting log levels for operational SDK messages, focusing on essential access logs, warnings, and errors.
+- **Improved Configuration and Control**:
+    - Introduced a `builtin_permissions` flag to provide granular control over the built-in permission handling infrastructure (MCP and SSE).
+    - Implemented configurable system prompt injection modes (`minimal` and `full`) to customize how the Claude Code identity is presented in requests.
+- **Robust Streaming and Header Management**:
+    - Implemented `StreamingResponseWithLogging` for unified and consistent access logging across all streaming endpoints.
+    - Ensured critical upstream headers (e.g., `cf-ray`, `anthropic-ratelimit-*`) are correctly forwarded in SSE streaming responses.
+
+### Changed
+
+- **Default Behavior**:
+    - The Claude SDK connection pool is now disabled by default, requiring an explicit opt-in for safer and more predictable behavior.
+- **Architectural Improvements**:
+    - Refactored the application's startup and shutdown logic into a modular, component-based architecture for better maintainability and testability.
+    - Renamed `SessionContext` to `SessionClient` for improved clarity and consistency in the session pooling implementation.
+- **Testing Infrastructure**:
+    - Reorganized the entire test suite into a hierarchical structure (`unit` and `integration`) to improve navigation and maintainability.
+    - Migrated from legacy test fixtures to a more flexible and maintainable factory pattern for creating test clients and application instances.
+
+### Fixed
+
+- **Session and Streaming Stability**:
+    - Eliminated critical race conditions and `AttributeError` exceptions in the session pool and stream handling logic.
+    - Replaced fragile `asyncio.sleep` calls with a robust, event-based synchronization mechanism to prevent timing-related failures.
+    - Implemented a more accurate message-based stale detection mechanism to prevent the incorrect termination of active sessions.
+- **Resource Management**:
+    - Corrected several resource leak issues by improving stream interruption handling, ensuring that hanging sessions are properly cleaned up.
+- **Header and Content Formatting**:
+    - Resolved an issue that prevented the forwarding of upstream headers in SSE streaming responses.
+    - Fixed a formatting bug in the OpenAI adapter that caused message content to be improperly concatenated.
+
+
+### Added
+
+- **Configurable Permission Infrastructure**: Added `builtin_permissions` configuration flag to control built-in permission handling infrastructure:
+  - New `builtin_permissions` flag (default: true) in ClaudeSettings for granular control
+  - CLI support with `--builtin-permissions/--no-builtin-permissions` options
+  - When disabled: No MCP server setup, no SSE permission endpoints, no permission service initialization
+  - When enabled: Full built-in permission infrastructure with smart MCP server merging
+  - Users can configure custom MCP servers independently when built-in infrastructure is disabled
+  - Maintains full backward compatibility with existing configurations
+- **Claude SDK Pool Mode**: Implemented connection pooling for Claude Code SDK clients to improve request performance:
+  - Maintains a pool of pre-initialized Claude Code instances to eliminate startup overhead
+  - Reduces request latency by reusing established connections
+  - Pool mode is disabled by default and can be enabled via configuration
+  - **Limitations**: Pool mode does not support dynamic Claude options (max_tokens, model changes, etc.)
+  - Pool instances are shared across requests with identical configurations
+- **Session-Aware Connection Pooling**: Added advanced session-based pooling for persistent conversation context:
+  - Session pools maintain dedicated Claude SDK clients per session ID for conversation continuity
+  - Configurable session TTL (time-to-live) with automatic cleanup of idle sessions
+  - Session pool settings include max sessions, idle threshold, and cleanup intervals
+  - Automatic connection recovery for unhealthy sessions when enabled
+  - Session interruption support for graceful handling of canceled requests
+  - Separate from the general connection pool - can be used independently or together
+  - Configuration via `claude.session_pool` settings with sensible defaults
+- **Claude Detection Service**: Implemented automatic Claude CLI header and system prompt detection at startup:
+  - Automatically detects current Claude CLI version and extracts real headers/system prompt
+  - Caches detection results per version to avoid repeated startup delays
+  - Falls back to hardcoded values when detection fails
+- **Detection Models**: Added Pydantic models for Claude detection data:
+  - `ClaudeCodeHeaders` - Structured header extraction with field aliases
+  - `SystemPromptData` - System prompt content with cache control
+  - `ClaudeCacheData` - Complete cached detection data with version tracking
+
+### Changed
+
+- **Configuration Updates**: Enhanced Claude settings with new pool configuration options:
+  - Added `use_client_pool` boolean flag to enable general connection pooling
+  - Added `pool_settings` for configuring general pool behavior (size, timeouts, health checks)
+  - Added `session_pool` settings for session-aware pooling configuration
+  - Session pool enabled by default with 1-hour TTL and automatic cleanup
+- **HTTP Request Transformation**: Enhanced request transformers to use detected Claude CLI headers and system prompt:
+  - Dynamically uses detected headers when available, falls back to hardcoded when not
+  - System prompt injection now uses detected Claude Code system prompt
+  - Added app_state parameter propagation for detection data access
+- **Request Transformer Architecture**: Refactored transformers to support dynamic header and prompt injection:
+  - Added proxy_mode parameter to RequestTransformer base class
+  - Enhanced transform methods to accept app_state for detection data access
+  - Improved header creation logic to distinguish between detected and fallback headers
+- **Test Organization Cleanup**: Finalized test suite migration and removed obsolete migration documentation:
+  - Removed obsolete `MIGRATION_GUIDE.md` files from factories, fixtures, and auth directories
+  - Cleaned up `conftest.py` by removing backward compatibility aliases for fixture names
+  - Updated fixture references to use direct imports from fixture modules
+  - Simplified FastAPI factory test organization by removing legacy compatibility layer
+  - Modernized fixture naming convention throughout test files (internal_claude_sdk_service vs claude_service)
+  - Removed unused factory fixtures and consolidated client creation patterns
+- **Test Organization**: Migrated test suite from flat structure to organized hierarchy:
+  - Tests now organized under `tests/unit/` and `tests/integration/` directories
+  - Unit tests categorized by component: `api/`, `services/`, `auth/`, `config/`, `utils/`, `cli/`
+  - Integration tests moved to dedicated `tests/integration/` directory
+  - Enhanced factory pattern with `FastAPIAppFactory` for flexible test app creation
+  - Improved fixture organization with dedicated `tests/fixtures/` structure
+- **Configuration Cleanup**: Removed unused `ConfigLoader` class and simplified configuration management
+- **Logging Optimization**: Reduced permission service log verbosity from INFO to DEBUG level for cleaner production logs
+
+### Infrastructure
+
+- **Test Structure**: Added `.gitignore` for test artifacts and coverage reports
+- **Documentation**: Updated `TESTING.md` with new test organization and examples
+- **Cache Directory**: Added automatic creation of `~/.cache/ccproxy/` for detection data persistence
+- **Session Pool Components**: Added new modules for session management:
+  - `ccproxy/claude_sdk/session_pool.py` - Core session pool implementation
+  - `ccproxy/claude_sdk/session_client.py` - Session-aware client wrapper
+  - `ccproxy/claude_sdk/manager.py` - Unified pool management with metrics integration
+- **Test Coverage**: Added comprehensive tests for session pool functionality:
+  - Unit tests for session lifecycle, cleanup, and recovery
+  - Integration tests for end-to-end session pooling behavior
+
+## [0.1.4] - 2025-05-28
+
+### Fixed
+
+- **Pydantic Compatibility**: Fixed TypeError in model_dump_json() call by removing invalid separators parameter (issue #5)
+
+## [0.1.3] - 2025-07-25
+
+### Added
+
+- **Version Update Checking**: Automatic version checking against GitHub releases with configurable intervals (default 12 hours) and startup checks
+- **MCP Server Integration**: Added Model Context Protocol (MCP) server functionality with permission checking tools for Claude Code integration
+- **Permission System**: Implemented comprehensive permission management with REST API endpoints and Server-Sent Events (SSE) streaming for real-time permission requests
+- **Request/Response Logging**: Added comprehensive logging middleware with configurable verbosity levels (`CCPROXY_VERBOSE_API`, `CCPROXY_REQUEST_LOG_DIR`)
+- **Claude SDK Custom Content Blocks**: Added support for `system_message`, `tool_use_sdk`, and `tool_result_sdk` content blocks with full metadata preservation
+- **Model Mapping Utilities**: Centralized model provider abstraction with unified mapping logic in `ccproxy/utils/models_provider.py`
+- **Terminal Permission Handler**: Interactive permission workflow handler for CLI-based permission management
+- **Claude SDK Field Rendering**: Added flexible content handling with `forward`, `formatted`, and `ignore` rendering options for Claude SDK fields
+
+### Changed
+
+- **Claude SDK Integration**: Refactored to use native ThinkingBlock models from Claude Code SDK
+- **Models Endpoint**: Centralized `/v1/models` endpoint implementation to eliminate code duplication across routes
+- **OpenAI Adapter**: Enhanced with improved modularization and streaming architecture
+- **Logging System**: Migrated to canonical structlog pattern for structured, consistent logging
+- **SSE Streaming**: Improved Server-Sent Events format with comprehensive examples and better error handling
+
+### Fixed
+
+- **SDK Double Content**: Resolved duplicate content issue in Claude SDK message processing
+- **Error Handling**: Enhanced error handling throughout Claude SDK message processing pipeline
+- **Type Safety**: Improved type checking across permission system components
+- **Permission Handler**: Fixed lazy initialization issues in terminal permission handler
+
+## [0.1.2] - 2025-07-22
+
+### Added
+
+- **Permission Mode Support**: Restored `--permission-mode` option supporting default, acceptEdits, and bypassPermissions modes
+- **Custom Permission Tool**: Restored `--permission-prompt-tool-name` option to specify custom permission tool names
+- **Permission Response Models**: Added `PermissionToolAllowResponse` and `PermissionToolDenyResponse` models with proper JSON serialization
+
+### Changed
+
+- **Message Formatting**: Modified `MessageConverter.combine_text_parts()` to join text parts with newlines instead of spaces, preserving formatting in multi-line content
+- **Settings Integration**: Enhanced OptionsHandler to apply default Claude options from settings before API parameters
+- **Configuration**: Extended settings to persist permission_mode and permission_prompt_tool_name
+
+### Fixed
+
+- **Claude SDK Options**: Integrated Settings dependency into ClaudeSDKService to support configuration-based options
+
+## [0.1.1] - 2025-07-22
+
+### Added
+
+- **Conditional Authentication**: API endpoints now support optional authentication - when `SECURITY__AUTH_TOKEN` is configured, authentication is enforced; when not configured, the proxy runs in open mode.
+- **Startup Validation**: Added comprehensive validation checks during application startup:
+  - Validates OAuth credentials and warns about expired tokens
+  - Checks for Claude CLI binary availability with installation instructions
+  - Logs token expiration time and subscription type when valid
+- **Default Command**: The `serve` command is now the default - running `ccproxy` without subcommands automatically starts the server.
+- **Alternative Entry Point**: Added `ccproxy-api` as an alternative command-line entry point.
+
+### Changed
+
+- **Authentication Variable**: Renamed environment variable from `AUTH_TOKEN` to `SECURITY__AUTH_TOKEN` for better namespace organization and clarity.
+- **Credential Priority**: Reordered credential search paths to prioritize ccproxy-specific credentials before Claude CLI paths.
+- **CLI Syntax**: Migrated all CLI parameters to modern Annotated syntax for better type safety and IDE support.
+- **Pydantic v2**: Updated all models to use Pydantic v2 configuration syntax (`model_config` instead of `Config` class).
+- **Documentation**: Improved Aider integration docs with correct API endpoint URLs and added installation options (uv, pipx).
+
+### Fixed
+
+- **Authentication Separation**: Fixed critical issue where auth token was incorrectly used for both client and upstream authentication - now client auth token is separate from OAuth credentials.
+- **URL Paths**: Fixed documentation to use `/api` endpoints for Aider compatibility instead of SDK mode paths.
+- **Default Values**: Fixed default values for list parameters in CLI (docker_env, docker_volume, docker_arg).
+
+### Removed
+
+- **Status Endpoints**: Removed redundant `/status` endpoints from both Claude SDK and proxy routes.
+- **Permission Tool**: Removed Claude permission tool functionality and related CLI options (`--permission-mode`, `--permission-prompt-tool-name`) that are no longer needed.
+- **Deprecated Options**: Removed references to deprecated permission_mode and permission_prompt_tool_name from documentation.
+
+## [0.1.0] - 2025-07-21
+
+This is the initial public release of the CCProxy API.
+
+### Added
+
+#### Core Functionality
+
+- **Personal Claude Access**: Enables using a personal Claude Pro, Team, or Enterprise subscription as an API endpoint, without needing separate API keys.
+- **OAuth2 Authentication**: Implements the official Claude OAuth2 flow for secure, local authentication.
+- **Local Proxy Server**: Runs a lightweight FastAPI server on your local machine.
+- **HTTP/HTTPS Proxy Support**: Full support for routing requests through an upstream HTTP or HTTPS proxy.
+
+#### API & Compatibility
+
+- **Dual API Support**: Provides full compatibility with both Anthropic and OpenAI API specifications.
+- **Anthropic Messages API**: Native support for the Anthropic Messages API at `/v1/chat/completions`.
+- **OpenAI Chat Completions API**: Compatibility layer for the OpenAI Chat Completions API at `/openai/v1/chat/completions`.
+- **Request/Response Translation**: Seamlessly translates requests and responses between OpenAI and Anthropic formats.
+- **Streaming Support**: Real-time streaming for both Anthropic and OpenAI-compatible endpoints.
+- **Model Endpoints**: Lists available models via `/v1/models` and `/openai/v1/models`.
+- **Health Check**: A `/health` endpoint for monitoring the proxy's status.
+
+#### Configuration & CLI
+
+- **Unified `ccproxy` CLI**: A single, user-friendly command-line interface for managing the proxy.
+- **TOML Configuration**: Configure the server using a `config.toml` file with JSON Schema validation.
+- **Keyring Integration**: Securely stores and manages OAuth credentials in the system's native keyring.
+- **`generate-token` Command**: A CLI command to manually generate and manage API tokens.
+- **Systemd Integration**: Includes a setup script and service template for running the proxy as a systemd service in production environments.
+- **Docker Support**: A `Dockerfile` and `docker-compose.yml` for running the proxy in an isolated containerized environment.
+
+#### Security
+
+- **Local-First Design**: All processing and authentication happens locally; no conversation data is stored or transmitted to third parties.
+- **Credential Security**: OAuth tokens are stored securely in the system keyring, not in plaintext files.
+- **Header Stripping**: Automatically removes client-side `Authorization` headers to prevent accidental key leakage.
+
+#### Developer Experience
+
+- **Comprehensive Documentation**: Includes a quick start guide, API reference, and setup instructions.
+- **Pre-commit Hooks**: Configured for automated code formatting and linting to ensure code quality.
+- **Modern Tooling**: Uses `uv` for package management and `devenv` for a reproducible development environment.
+- **Extensive Test Suite**: Includes unit, integration, and benchmark tests to ensure reliability.
+- **Rich Logging**: Structured and colorized logging for improved readability during development and debugging.

{ccproxy_api-0.1.3 → ccproxy_api-0.1.5}/PKG-INFO

@@ -1,12 +1,12 @@
 Metadata-Version: 2.4
 Name: ccproxy-api
-Version: 0.1.3
+Version: 0.1.5
 Summary: API server that provides an Anthropic and OpenAI compatible interface over Claude Code, allowing to use your Claude OAuth account or over the API.
 License-File: LICENSE
 Requires-Python: >=3.11
 Requires-Dist: aiofiles>=24.1.0
 Requires-Dist: aiosqlite>=0.21.0
-Requires-Dist: claude-code-sdk>=0.0.14
+Requires-Dist: claude-code-sdk>=0.0.19
 Requires-Dist: duckdb-engine>=0.17.0
 Requires-Dist: duckdb>=1.1.0
 Requires-Dist: fastapi-mcp
@@ -182,6 +182,33 @@ CLAUDE__SDK_MESSAGE_MODE=formatted ccproxy
 CLAUDE__PRETTY_FORMAT=false ccproxy
 ```
 
+## Claude SDK Pool Mode
+
+CCProxy supports connection pooling for Claude Code SDK clients to improve request performance by maintaining a pool of pre-initialized Claude instances.
+
+### Benefits
+- **Reduced Latency**: Eliminates Claude Code startup overhead on each request
+- **Improved Performance**: Reuses established connections for faster response times
+- **Resource Efficiency**: Maintains a configurable pool size to balance performance and resource usage
+
+### Usage
+Pool mode is disabled by default and can be enabled using the CLI flag:
+
+```bash
+# Enable pool mode with default settings
+ccproxy --sdk-enable-pool
+
+# Configure pool size (default: 3)
+ccproxy --sdk-enable-pool --sdk-pool-size 5
+```
+
+### Limitations
+- **No Dynamic Options**: Pool instances cannot change Claude options (max_tokens, model, etc.) after initialization
+- **Shared Configuration**: All requests using the pool must use identical Claude configuration
+- **Memory Usage**: Each pool instance consumes additional memory
+
+Pool mode is most effective for high-frequency requests with consistent configuration requirements.
+
 ## Using with Aider
 
 CCProxy works seamlessly with Aider and other AI coding assistants:

{ccproxy_api-0.1.3 → ccproxy_api-0.1.5}/README.md

@@ -150,6 +150,33 @@ CLAUDE__SDK_MESSAGE_MODE=formatted ccproxy
 CLAUDE__PRETTY_FORMAT=false ccproxy
 ```
 
+## Claude SDK Pool Mode
+
+CCProxy supports connection pooling for Claude Code SDK clients to improve request performance by maintaining a pool of pre-initialized Claude instances.
+
+### Benefits
+- **Reduced Latency**: Eliminates Claude Code startup overhead on each request
+- **Improved Performance**: Reuses established connections for faster response times
+- **Resource Efficiency**: Maintains a configurable pool size to balance performance and resource usage
+
+### Usage
+Pool mode is disabled by default and can be enabled using the CLI flag:
+
+```bash
+# Enable pool mode with default settings
+ccproxy --sdk-enable-pool
+
+# Configure pool size (default: 3)
+ccproxy --sdk-enable-pool --sdk-pool-size 5
+```
+
+### Limitations
+- **No Dynamic Options**: Pool instances cannot change Claude options (max_tokens, model, etc.) after initialization
+- **Shared Configuration**: All requests using the pool must use identical Claude configuration
+- **Memory Usage**: Each pool instance consumes additional memory
+
+Pool mode is most effective for high-frequency requests with consistent configuration requirements.
+
 ## Using with Aider
 
 CCProxy works seamlessly with Aider and other AI coding assistants:

ccproxy_api-0.1.5/SESSION_POOL_FIX_PLAN.md

@@ -0,0 +1,118 @@
+# Session Pool Stale Detection Fix Plan
+
+## Issue Analysis
+
+The session pool incorrectly marks completed streams as "stale" using a fixed 10-second threshold (`idle_seconds > 10.0` in session_pool.py:184). This causes unnecessary interruptions of already-completed streams.
+
+## Root Cause
+
+1. **Incorrect stale detection logic**: Uses 10s timeout instead of proper message-based lifecycle tracking
+2. **No stream completion detection**: Pool doesn't detect when streams finish with ResultMessage
+3. **Missing first chunk timing**: No distinction between waiting-for-first-chunk vs ongoing streams
+
+## Solution: Message-Based Stream Lifecycle Tracking
+
+### Step 1: Enhance Stream Handle with Message Tracking
+- Add message type tracking in `StreamHandle` class
+- Track `SystemMessage(init)` reception (first chunk received)
+- Track `ResultMessage` reception (stream completed)
+- Track timestamps for timeout calculations
+
+### Step 2: Update Session Pool Stale Detection Logic
+Replace current logic in `session_pool.py:182-184` with:
+```python
+# Stream is stale if:
+# - No SystemMessage received within 3 seconds (first chunk timeout)
+# - SystemMessage received but no activity for 60 seconds (ongoing timeout)
+# - Never check stale for completed streams (ResultMessage received)
+```
+
+### Step 3: Implementation Details
+
+**In `stream_handle.py`:**
+- Add fields: `first_chunk_received_at`, `completed_at`, `has_result_message`
+- Track message types in stream worker message processing
+- Expose completion status to session pool
+
+**In `session_pool.py`:**
+- Update stale detection in `get_session_client()` method (lines 182-184)
+- Implement proper timeout logic:
+  - 3s timeout if no first chunk
+  - 60s timeout if first chunk received but not completed
+  - Skip timeout check for completed streams
+
+### Step 4: Message Flow Integration
+- Hook into existing message processing in `stream_worker.py`
+- Update handle when `SystemMessage(subtype=init)` received
+- Update handle when `ResultMessage` received
+- No special handling for slash commands (generic solution)
+
+## Files to Modify
+
+1. `ccproxy/claude_sdk/stream_handle.py` - Add message lifecycle tracking
+2. `ccproxy/claude_sdk/session_pool.py` - Fix stale detection logic
+3. `ccproxy/claude_sdk/stream_worker.py` - Update message tracking in worker
+
+## Testing
+
+- Unit tests for new timeout logic
+- Integration tests with slash commands and regular messages
+- Verify 3s/60s timeout behavior
+
+## Implementation Order
+
+1. ✅ Write plan to file
+2. ✅ Update StreamHandle with message tracking
+3. ✅ Update StreamWorker to notify handle of message types
+4. ✅ Update SessionPool stale detection logic
+5. ✅ Test the fix
+6. ✅ Fix Claude SDK client timeout conflict
+7. ✅ Fix unhandled StreamTimeoutError exception
+
+## Implementation Summary
+
+The fix has been successfully implemented with the following changes:
+
+### StreamHandle (`ccproxy/claude_sdk/stream_handle.py`)
+- Added message lifecycle tracking fields: `_first_chunk_received_at`, `_completed_at`, `_has_result_message`, `_last_activity_at`
+- Added methods: `on_first_chunk_received()`, `on_message_received()`, `on_completion()`
+- Added properties: `is_completed`, `has_first_chunk`, `idle_seconds`
+- Added `is_stale()` method with proper timeout logic:
+  - 3s timeout if no SystemMessage received
+  - 60s timeout if SystemMessage received but no ResultMessage
+  - Never stale for completed streams
+
+### StreamWorker (`ccproxy/claude_sdk/stream_worker.py`)
+- Added `stream_handle` parameter to constructor
+- Added message tracking logic to notify handle of:
+  - All message activity (updates last activity time)
+  - SystemMessage with init subtype (first chunk received)
+  - ResultMessage (stream completion)
+
+### SessionPool (`ccproxy/claude_sdk/session_pool.py`)
+- Replaced fixed 10-second stale detection with message-based timeout logic:
+  - First chunk timeout (3s): Terminate session client
+  - Ongoing timeout (60s): Interrupt stream but keep session
+- Added differentiated timeout handling with proper logging
+
+### ClaudeClient (`ccproxy/claude_sdk/client.py`)
+- Modified first chunk timeout handling to check if session pool is enabled
+- When session pool is enabled, logs timeout but lets session pool handle cleanup
+- When session pool is disabled, maintains original interrupt behavior
+- Prevents conflicting timeout handling between client and session pool
+
+### Testing
+- All existing unit tests pass
+- Type checking passes with mypy
+- Linting passes with ruff
+- Custom validation tests confirm proper timeout behavior
+- Session pool integration tests pass
+
+## Message Flow
+
+```
+Stream Start → StreamWorker processes SystemMessage(init) → StreamHandle.on_first_chunk()
+Stream Processing → StreamWorker processes content messages → Continue
+Stream End → StreamWorker processes ResultMessage → StreamHandle.on_completion()
+Session Pool → Check handle.is_completed() before stale detection
+```