pytest-fastcollect 0.5.2__cp312-cp312-musllinux_1_2_i686.whl

pytest_fastcollect-0.5.2.dist-info/METADATA

@@ -0,0 +1,747 @@
+Metadata-Version: 2.4
+Name: pytest-fastcollect
+Version: 0.5.2
+Classifier: Framework :: Pytest
+Classifier: Programming Language :: Rust
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Programming Language :: Python :: Implementation :: CPython
+Classifier: Programming Language :: Python :: Implementation :: PyPy
+Classifier: Topic :: Software Development :: Testing
+Requires-Dist: pytest>=7.0.0
+Requires-Dist: pytest-benchmark>=4.0.0 ; extra == 'dev'
+Requires-Dist: hypothesis>=6.0.0 ; extra == 'dev'
+Provides-Extra: dev
+License-File: LICENSE
+Summary: A high-performance pytest plugin that replaces test collection with a Rust-based implementation
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
+
+# pytest-fastcollect
+
+[![CI](https://github.com/Samrhan/pytest-fastcollect/actions/workflows/CI.yml/badge.svg)](https://github.com/Samrhan/pytest-fastcollect/actions/workflows/CI.yml)
+[![codecov](https://codecov.io/gh/Samrhan/pytest-fastcollect/branch/main/graph/badge.svg)](https://codecov.io/gh/Samrhan/pytest-fastcollect)
+[![PyPI version](https://badge.fury.io/py/pytest-fastcollect.svg)](https://badge.fury.io/py/pytest-fastcollect)
+[![Python versions](https://img.shields.io/pypi/pyversions/pytest-fastcollect.svg)](https://pypi.org/project/pytest-fastcollect/)
+
+A high-performance pytest plugin that uses Rust to accelerate test collection. This plugin leverages `rustpython-parser` to parse Python test files in parallel, with incremental caching to skip unchanged files.
+
+**Performance**: Up to **2.4x faster** collection on large projects (tested on Django's 1977 test files). Best for codebases with 200+ test files.
+
+## Features
+
+- 🦀 **Rust-Powered Parsing**: Uses `rustpython-parser` for blazing-fast Python AST parsing
+- ⚡ **Parallel Processing**: Leverages Rayon for parallel file processing
+- 💾 **Incremental Caching**: Caches parsed results with file modification tracking
+- 🎯 **Smart Filtering**: Pre-filters test files before pytest's collection phase
+- 🔧 **Drop-in Replacement**: Works as a pytest plugin with no code changes required
+- 🎛️ **Configurable**: Enable/disable fast collection and caching with command-line flags
+- 📈 **Scales with Size**: Performance improvements scale with project size (2-4x on 500+ files)
+
+## Installation
+
+### From Source
+
+```bash
+# Clone the repository
+git clone https://github.com/yourusername/pytest-fastcollect.git
+cd pytest-fastcollect
+
+# Create a virtual environment
+python -m venv .venv
+source .venv/bin/activate  # On Windows: .venv\Scripts\activate
+
+# Install maturin and build
+pip install maturin
+maturin develop --release
+
+# Or install in production mode
+maturin build --release
+pip install target/wheels/pytest_fastcollect-*.whl
+```
+
+### Requirements
+
+- Python 3.9+ (supports Python 3.9, 3.10, 3.11, 3.12, 3.13, 3.14)
+- Rust 1.70+
+- pytest 7.0+
+
+## Usage
+
+### Should I Use This Plugin?
+
+Not sure if pytest-fastcollect will help your project? Run the built-in benchmark:
+
+```bash
+pytest --benchmark-collect
+```
+
+This will:
+- ⏱️  Measure collection time **with** and **without** the plugin
+- 📊 Analyze your project size and structure
+- 💡 Provide a clear **recommendation** with actionable advice
+- 🎯 Show expected time savings
+
+**Example output:**
+```
+======================================================================
+pytest-fastcollect Benchmark
+======================================================================
+
+Analyzing your test suite to determine if pytest-fastcollect is beneficial...
+
+📊 Project Stats:
+   Test files: 245
+   Test items: 1,892
+
+⚡ Benchmark 1: WITH pytest-fastcollect
+   Running collection with Rust acceleration... Done! (0.342s)
+
+🐌 Benchmark 2: WITHOUT pytest-fastcollect
+   Running standard pytest collection... Done! (1.567s)
+
+======================================================================
+📈 Results
+======================================================================
+
+⏱️  Collection Time:
+   Standard pytest:      1.567s
+   With fastcollect:     0.342s
+   Time saved:           1.225s
+   Speedup:              4.58x
+
+💡 Recommendation:
+   ⭐⭐⭐ EXCELLENT
+   pytest-fastcollect provides SIGNIFICANT speedup (4.6x faster)!
+   ✅ Highly recommended for your project.
+   ✅ You'll save 1.2s on every test run.
+
+📦 Project Size Analysis:
+   Your project is MEDIUM-LARGE (245 files).
+   ✅ Good fit for pytest-fastcollect.
+======================================================================
+```
+
+### Basic Usage
+
+Once installed, the plugin is automatically activated for all pytest runs:
+
+```bash
+# Run pytest as normal - fast collection is enabled by default
+pytest
+
+# Collect tests only (useful for benchmarking)
+pytest --collect-only
+
+# Disable fast collection
+pytest --no-fast-collect
+
+# Clear cache and reparse all files
+pytest --fastcollect-clear-cache --collect-only
+
+# Disable caching (always parse)
+pytest --no-fastcollect-cache
+
+# Benchmark: Test if pytest-fastcollect is beneficial for your project
+pytest --benchmark-collect
+
+# Experimental: Parallel module import (2.33x faster on pytest itself!)
+pytest --parallel-import --parallel-workers=4
+
+# Production-Ready: Collection Daemon (instant re-collection)
+pytest --daemon-start tests/        # Start daemon
+pytest --daemon-status              # Check status
+pytest --daemon-stop                # Stop daemon
+pytest --daemon-health              # Health check
+
+# Run benchmarks
+python benchmark.py --synthetic
+python benchmark_incremental.py  # Shows cache effectiveness
+python benchmark_parallel.py     # Test parallel import performance
+```
+
+### Configuration Options
+
+- `--use-fast-collect`: Enable Rust-based fast collection (default: True)
+- `--no-fast-collect`: Disable fast collection and use standard pytest collection
+- `--fastcollect-cache`: Enable incremental caching (default: True)
+- `--no-fastcollect-cache`: Disable caching and parse all files
+- `--fastcollect-clear-cache`: Clear the cache before collection
+- `--benchmark-collect`: **[Recommended]** Benchmark to test if the plugin is beneficial for your project
+- `--parallel-import`: **[Experimental]** Pre-import modules in parallel (default: False)
+- `--parallel-workers=N`: Number of parallel import workers (default: CPU count)
+- `--daemon-start`: **[Production-Ready]** Start collection daemon for instant re-collection
+- `--daemon-stop`: Stop the collection daemon gracefully
+- `--daemon-status`: Show comprehensive daemon status (PID, uptime, cached modules, metrics)
+- `--daemon-health`: Check daemon health and diagnostics
+
+## Architecture
+
+### System Overview
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                          pytest CLI                              │
+└──────────────────────┬──────────────────────────────────────────┘
+                       │
+                       ▼
+┌─────────────────────────────────────────────────────────────────┐
+│                   pytest-fastcollect Plugin                      │
+│  ┌────────────────────────────────────────────────────────────┐ │
+│  │  1. pytest_configure Hook (Early Initialization)           │ │
+│  │     - Create cache instance                                │ │
+│  │     - Start daemon (if --daemon-start)                     │ │
+│  │     - Check cache for existing data                        │ │
+│  └────────────────────────────────────────────────────────────┘ │
+│                       │                                          │
+│                       ▼                                          │
+│  ┌────────────────────────────────────────────────────────────┐ │
+│  │  2. Rust FastCollector (Parallel AST Parsing)             │ │
+│  │     - Walk directory tree (exclude .git, venv, etc.)       │ │
+│  │     - Parse Python AST with rustpython-parser              │ │
+│  │     - Extract tests, markers, mtimes (parallel via Rayon)  │ │
+│  │     - Return metadata to Python                            │ │
+│  └────────────────────────────────────────────────────────────┘ │
+│                       │                                          │
+│                       ▼                                          │
+│  ┌────────────────────────────────────────────────────────────┐ │
+│  │  3. Caching Layer (CollectionCache)                        │ │
+│  │     - Check file mtimes against cache                      │ │
+│  │     - Return cached data for unchanged files               │ │
+│  │     - Store new results in .pytest_cache/                  │ │
+│  │     - Track cache hits/misses for stats                    │ │
+│  └────────────────────────────────────────────────────────────┘ │
+│                       │                                          │
+│                       ▼                                          │
+│  ┌────────────────────────────────────────────────────────────┐ │
+│  │  4. Filter Logic (Optional: -k/-m filters)                 │ │
+│  │     - Apply keyword filters to test names                  │ │
+│  │     - Apply marker filters to test decorators              │ │
+│  │     - Select only files with matching tests                │ │
+│  │     - Skip importing non-matching files                    │ │
+│  └────────────────────────────────────────────────────────────┘ │
+│                       │                                          │
+│                       ▼                                          │
+│  ┌────────────────────────────────────────────────────────────┐ │
+│  │  5. pytest_ignore_collect Hook (File Filtering)            │ │
+│  │     - Consulted for each file/directory pytest encounters  │ │
+│  │     - Return True to skip files not in collected data      │ │
+│  │     - Let pytest handle actual test collection             │ │
+│  └────────────────────────────────────────────────────────────┘ │
+└───────────────────────────┬─────────────────────────────────────┘
+                            │
+                            ▼
+                  ┌─────────────────────┐
+                  │  pytest Collection  │
+                  │  (Standard Process) │
+                  └─────────────────────┘
+```
+
+### How It Works
+
+1. **Rust Collector (`FastCollector`)**:
+   - Walks the directory tree to find Python test files
+   - Uses `rustpython-parser` to parse each file's AST in parallel
+   - Extracts file modification times for cache validation
+   - Returns collected metadata to Python
+
+2. **Incremental Caching**:
+   - Caches parsed test data with file mtimes in `.pytest_cache/v/fastcollect/`
+   - On subsequent runs, checks file mtimes
+   - Only reparses files that have changed
+   - Shows cache statistics (hits/misses) after collection
+
+3. **pytest Plugin Integration**:
+   - Hooks into pytest's `pytest_ignore_collect` to filter files
+   - Uses cached data when available
+   - Falls back to standard collection on errors
+
+4. **File Detection**:
+   - Test files: `test_*.py` or `*_test.py`
+   - Ignored directories: `.git`, `__pycache__`, `.tox`, `.venv`, `venv`, `.eggs`, `*.egg-info`
+
+### Components
+
+```
+pytest-fastcollect/
+├── src/
+│   └── lib.rs                    # Rust implementation (FastCollector)
+├── pytest_fastcollect/
+│   ├── __init__.py               # Python package init
+│   ├── plugin.py                 # pytest plugin hooks
+│   ├── cache.py                  # Incremental caching layer
+│   ├── daemon.py                 # Collection daemon server
+│   ├── daemon_client.py          # Daemon client communication
+│   └── filter.py                 # Selective import filtering
+├── tests/
+│   └── sample_tests/             # Sample tests for validation
+├── benchmark.py                  # Performance benchmarking
+├── benchmark_incremental.py      # Cache effectiveness benchmark
+├── benchmark_parallel.py         # Parallel import benchmarking
+├── Cargo.toml                    # Rust dependencies
+└── pyproject.toml                # Python package metadata
+```
+
+## Benchmarks
+
+### v0.4.0 - Selective Import (Latest) ⭐
+
+**THE BREAKTHROUGH**: Selective import based on `-k` and `-m` filters!
+
+**How it works**:
+1. Parse all test files with Rust (parallel, fast)
+2. Extract test names and markers from AST
+3. Apply `-k` and `-m` filters BEFORE importing modules
+4. Only import files containing matching tests
+5. Result: Massive speedups for filtered runs!
+
+**Benchmark Results** (100 files, 10 tests/file):
+```
+Scenario                        Time      Speedup
+Full collection (no filter)     0.98s     baseline
+With -k filter (10% files)      0.57s     1.71x faster ⚡
+With -m filter (20% files)      0.64s     1.55x faster ⚡
+Combined filters                0.55s     1.78x faster ⚡
+```
+
+**Real-World Impact**:
+```bash
+# Before v0.4.0: imports ALL 100 test files
+pytest -k test_user  # 0.98s
+
+# With v0.4.0: imports only 10 matching files
+pytest -k test_user  # 0.57s (1.71x faster!)
+```
+
+**When it helps most**:
+- ✅ Running specific tests: `pytest -k test_user_login`
+- ✅ Running marked tests: `pytest -m smoke`
+- ✅ Development workflow (constantly filtering tests)
+- ✅ CI/CD with test splits
+- ✅ Large test suites with good organization
+
+**Key Features**:
+- Marker detection from decorators (`@pytest.mark.slow`)
+- Keyword matching (function names, class names, file names)
+- Supports `and`, `or`, `not` in expressions
+- Shows file selection stats with `-v`
+- Fully compatible with pytest's filter syntax
+
+### Multi-Project Real-World Benchmarks 📊
+
+Tested on **5 popular Python projects** to validate real-world performance:
+
+| Project | Files | Baseline | FastCollect | Speedup | Grade |
+|---------|-------|----------|-------------|---------|-------|
+| **Django** | ~1977 | 10.85s | 4.49s | **2.42x** | ⚡⚡⚡ Excellent |
+| **SQLAlchemy** | ~219 | 0.68s | 0.63s | **1.07x** | ✓ Minor |
+| **Pytest** | ~108 | 2.40s | 2.54s | **0.94x** | ⚠️ Overhead |
+| **Requests** | ~9 | 0.61s | 0.54s | **1.13x** | ✓ Minor |
+| **Flask** | ~22 | 0.55s | 0.55s | **1.00x** | → Neutral |
+
+**Key Finding**: Performance scales with project size! 🚀
+
+**Selective Import Performance** (additional speedup with `-k` filters):
+- **Pytest**: Up to **2.75x faster** with specific filters (`-k test_basic`)
+- **Django**: Additional **1.32x faster** with keyword filters
+- **Small projects**: Minimal additional benefit
+
+**Break-Even Analysis**:
+- ✅ **Large projects (500+ files)**: **2-4x speedup** - highly recommended
+- ⚠️ **Medium projects (100-300 files)**: **0.9-1.5x** - evaluate first
+- → **Small projects (< 50 files)**: **~1.0x** - not necessary
+
+**Bottom Line**: pytest-fastcollect is **ideal for large codebases** (200+ test files) where collection time becomes a bottleneck. For projects with < 50 files, the overhead roughly equals the benefit.
+
+📄 See [REALWORLD_BENCHMARKS.md](REALWORLD_BENCHMARKS.md) for comprehensive analysis across all projects.
+
+### Parallel Import (Experimental) ⚡⚡⚡
+
+**NEW**: Pre-import test modules in parallel for additional speedup!
+
+```bash
+pytest --parallel-import --parallel-workers=4
+```
+
+**Benchmark Results** (with --parallel-import):
+
+| Project | Baseline | With Parallel | Speedup | Grade |
+|---------|----------|---------------|---------|-------|
+| **Pytest** | 2.40s | 1.03s | **2.33x faster** | ⚡⚡⚡ Excellent |
+| **SQLAlchemy** | 0.69s | 0.64s | **1.07x faster** | ✓ Minor |
+| **Django** | 4.80s | 4.90s | **0.98x slower** | ⚠️ Overhead |
+
+**Key Finding**: Parallel import works great for projects with **simple, independent test modules** (like pytest itself!), but can hurt projects with complex interdependent imports (like Django).
+
+**When to use**:
+- ✅ Medium projects (100-300 files) with simple imports → **2-2.5x speedup**
+- ⚠️ Projects with complex imports → **Benchmark first**
+- ❌ Small projects (< 100 files) → **Overhead not worth it**
+
+**Optimal configuration**: 4 workers seems to be the sweet spot for most projects.
+
+**ProcessPoolExecutor** (experimental):
+```bash
+# Bypass GIL with true process parallelism
+pytest --parallel-import --use-processes --parallel-workers=4
+```
+
+**Results**: ProcessPoolExecutor tested but **not recommended**
+- ❌ Slower than ThreadPoolExecutor in most cases (0.88-1.10x)
+- Process overhead > GIL bypass benefit
+- Must import twice (subprocess + main process)
+- ThreadPoolExecutor remains the better choice
+
+📄 See [PARALLEL_IMPORT_RESULTS.md](PARALLEL_IMPORT_RESULTS.md) for threading details.
+📄 See [PROCESS_POOL_RESULTS.md](PROCESS_POOL_RESULTS.md) for process pool analysis.
+
+### Collection Daemon (Production-Ready) 🚀
+
+**Production-Ready**: Long-running daemon process that keeps test modules imported in memory for instant re-collection!
+
+```bash
+# Start the daemon (imports all modules once)
+pytest --daemon-start tests/
+
+# Check daemon status
+pytest --daemon-status
+
+# Check daemon health
+pytest --daemon-health
+
+# Stop the daemon
+pytest --daemon-stop
+```
+
+**Expected Performance**:
+- First run: ~10s (cold start, imports all modules)
+- Subsequent runs: ~0.01s (instant! modules already in memory)
+- **100-1000x speedup** on subsequent test runs
+
+**Production Features**:
+- ✅ Robust daemon server with Unix socket communication
+- ✅ Module pre-importing and caching in memory
+- ✅ Start/stop/status/health management commands
+- ✅ Comprehensive error handling and logging
+- ✅ Input validation and security checks
+- ✅ Connection management and rate limiting
+- ✅ Metrics tracking and monitoring
+- ✅ Graceful shutdown handling
+- ✅ Automatic retry with exponential backoff
+- ✅ Production-grade logging with rotation
+- ⏳ Full pytest collection integration (Phase 2)
+- ⏳ File watching for auto-reload (Phase 3)
+
+**Architecture**:
+- Long-running Python process
+- Unix socket IPC for client-daemon communication
+- Keeps modules in `sys.modules` across pytest runs
+- Background process management with forking
+- Per-project daemon instances (separate socket per root)
+
+**When to use**:
+- 🎯 **TDD workflows**: Constantly re-running tests during development
+- 🎯 **Watch mode**: Instant collection on file changes
+- 🎯 **Large codebases**: Where collection time > 5 seconds
+- 🎯 **Development environments**: Optimized for rapid iteration
+- ⚠️ **Not for CI/CD**: Designed for development, not one-shot runs
+
+**Production-Ready Features**:
+- ✅ Unix/Linux support (uses Unix domain sockets)
+- ✅ Comprehensive error handling and recovery
+- ✅ Security: Input validation and path checking
+- ✅ Monitoring: Health checks and metrics
+- ✅ Logging: Structured logs with automatic rotation
+- ✅ Resource management: Connection limits and timeouts
+- ✅ Graceful shutdown and cleanup
+- ✅ Comprehensive test coverage
+
+**Remaining Limitations**:
+- Phase 1: Infrastructure ready, pytest collection integration in progress
+- Manual daemon management (start/stop) - automation coming in Phase 2
+- File watching not yet implemented - planned for Phase 3
+
+📄 See [COLLECTION_DAEMON_PLAN.md](COLLECTION_DAEMON_PLAN.md) for full implementation roadmap.
+
+### Django Real-World Benchmark 🚀
+
+**The ultimate test**: Django's massive test suite with **~1977 Python test files**!
+
+**Full Collection Performance**:
+```
+Scenario                        Time      Speedup
+Baseline (no plugin)            36.59s    -
+FastCollect                     9.16s     3.99x faster ⚡⚡⚡
+```
+
+**Selective Import Performance**:
+```
+Filter Type                     Time      Speedup vs Full
+Full collection                 9.16s     baseline
+-k test_get                     4.12s     2.22x faster ⚡⚡
+-k test_forms                   3.80s     2.41x faster ⚡⚡
+-k "test_view or test_model"    4.19s     2.19x faster ⚡⚡
+```
+
+**Combined Impact**: FastCollect + Selective Import = **9.6x faster** than baseline pytest!
+- Baseline: 36.59s → FastCollect + filter: 3.80s
+
+**Key Takeaways**:
+- ✅ **4x faster** on full collection (real-world production codebase)
+- ✅ **2-2.4x additional speedup** with keyword filters
+- ✅ **Nearly 10x overall** when combining both optimizations
+- ✅ **Production-ready** on Django's complex test infrastructure
+- ✅ **Zero configuration** required - works out of the box
+
+📄 See [DJANGO_BENCHMARK_RESULTS.md](DJANGO_BENCHMARK_RESULTS.md) for detailed analysis.
+
+### v0.3.0 - Better Integration
+
+**Architecture Improvements**:
+```
+Early initialization:             Collection happens in pytest_configure
+File filtering:                   Simplified pytest_ignore_collect hook
+Collection overhead:              Reduced hook call complexity
+Code quality:                     Cleaner separation of concerns
+```
+
+**Performance** (comparable to v0.2.0):
+- Maintains incremental caching benefits (~5% improvement on warm cache)
+- No duplicate collection issues
+- Cleaner initialization flow
+
+**Key Changes**:
+- Moved Rust collection from lazy (first `pytest_ignore_collect` call) to eager (in `pytest_configure`)
+- Simplified `pytest_ignore_collect` to only use pre-collected data
+- Removed custom collector class to avoid pytest hook conflicts
+- More predictable and testable architecture
+
+### v0.2.0 - Incremental Caching
+
+**Incremental Collection Benchmark (500 files, 20 tests/file, 5 files modified)**:
+```
+Cold start (no cache):              3.34s  (baseline)
+Warm cache (no changes):            3.18s  (1.05x faster)
+Incremental (5 files changed):      3.50s  (cache + reparse 1%)
+
+Cache effectiveness:
+  - 100% cache hit rate on unchanged files
+  - Only modified files are reparsed
+  - ~5% improvement on warm cache
+  - Persistent across pytest runs
+```
+
+**Cache Statistics** (displayed after collection with `-v`):
+```
+FastCollect Cache: 2 files from cache, 0 parsed (100.0% hit rate)
+```
+
+### v0.1.0 - File Filtering Only
+
+**Synthetic Benchmarks**:
+- **200 files, 50 tests/file**: ~1.01x speedup
+- **500 files, 100 tests/file**: ~1.01x speedup
+
+**Real-World (pandas, 969 test files)**:
+- File filtering: 0.75x (slower due to overhead)
+
+### Performance Analysis
+
+**Why Limited Speedup in Pure Collection?**
+
+1. **Pytest Import Bottleneck**: Even with caching, pytest must import Python modules to get actual function/class objects
+2. **File Filtering Overhead**: The `pytest_ignore_collect` hook is called for every path
+3. **Duplicate Work**: Both Rust (for discovery) and Python (for import) process files
+
+**Where Caching Provides Real Value:**
+
+1. ✅ **Incremental Workflows**: Only reparse modified files (5-10% improvement)
+2. ✅ **Large Codebases**: Faster discovery in deep directory trees
+3. ✅ **CI/CD Pipelines**: Cache persists across runs
+4. ✅ **Development Workflow**: Repeated `pytest --collect-only` calls
+5. ✅ **Watch Mode**: Quick re-collection when files change
+
+### Recent Improvements (v0.2.0)
+
+✅ **Incremental Caching** - Cache parsed results with file modification tracking
+- Persists to `.pytest_cache/v/fastcollect/cache.json`
+- Only reparses files that have changed
+- Shows cache statistics after collection
+- ~5% improvement on repeated runs
+
+### Future Optimizations
+
+To achieve even greater speedup:
+
+1. **Direct Item Creation**: Create pytest `Item` objects directly from Rust (complex, see IMPLEMENTATION_NOTES.md)
+2. **Lazy Loading**: Only parse files when their tests are actually executed
+3. **Better Integration**: Use pytest's lower-level APIs to bypass standard collection
+4. **Parallel Imports**: Import Python modules in parallel
+
+## Technical Details
+
+### Rust Dependencies
+
+- `pyo3`: Python bindings for Rust
+- `rustpython-parser`: Python AST parser in Rust
+- `walkdir`: Recursive directory traversal
+- `rayon`: Data parallelism library
+
+### Python API
+
+```python
+from pytest_fastcollect import FastCollector
+
+# Create a collector for a directory
+collector = FastCollector("/path/to/tests")
+
+# Collect all tests (basic mode)
+results = collector.collect()
+# Returns: {"file_path": [{"name": "test_foo", "line": 10, "type": "Function"}, ...]}
+
+# Collect with metadata (includes file mtimes for caching)
+metadata = collector.collect_with_metadata()
+# Returns: {"file_path": {"mtime": 1234567890.0, "items": [...]}}
+```
+
+## Development
+
+### Building from Source
+
+```bash
+# Development build (faster compilation, slower runtime)
+maturin develop
+
+# Release build (slower compilation, faster runtime)
+maturin develop --release
+
+# Build wheel
+maturin build --release
+```
+
+### Running Tests
+
+```bash
+# Run sample tests
+pytest tests/sample_tests -v
+
+# Run with fast collection disabled
+pytest tests/sample_tests --no-fast-collect -v
+
+# Collect only (no execution)
+pytest tests/sample_tests --collect-only
+
+# View cache statistics
+pytest tests/sample_tests --collect-only -v
+```
+
+### Running Benchmarks
+
+```bash
+# Synthetic benchmark with custom parameters
+python benchmark.py --synthetic --num-files 200 --tests-per-file 100
+
+# Incremental caching benchmark (shows cache effectiveness)
+python benchmark_incremental.py
+
+# Benchmark on a real project
+python benchmark.py --project /path/to/project
+
+# Run all benchmarks
+python benchmark.py --all
+```
+
+### Cache Management
+
+```bash
+# Clear the cache
+pytest --fastcollect-clear-cache --collect-only
+
+# Disable caching for one run
+pytest --no-fastcollect-cache
+
+# View cache contents
+cat .pytest_cache/v/fastcollect/cache.json
+```
+
+## Contributing
+
+Contributions are welcome! Areas for improvement:
+
+1. **Performance Optimization**: Implement direct `Item` creation from Rust data
+2. **Advanced Caching**: Add file content hashing for more reliable cache validation
+3. **Test Discovery**: Support more complex test patterns (fixtures, parameterization)
+4. **Configuration**: Add support for custom test patterns and ignore rules
+5. **Documentation**: Add more examples and use cases
+
+## License
+
+This project is licensed under the MIT License - see the LICENSE file for details.
+
+## Acknowledgments
+
+- Built with [PyO3](https://pyo3.rs/) for seamless Rust-Python integration
+- Uses [RustPython Parser](https://github.com/RustPython/RustPython) for Python AST parsing
+- Inspired by the need for faster test collection in large Python codebases
+
+## Technical Notes
+
+### Why Rust?
+
+- **Speed**: Rust's zero-cost abstractions and lack of GIL make it ideal for CPU-intensive parsing
+- **Parallelism**: Rayon makes it trivial to parse files in parallel
+- **Safety**: Rust's type system ensures memory safety without garbage collection overhead
+
+### Current Limitations
+
+1. **Limited Pure Collection Speedup**: Pytest still needs to import modules (~5% improvement)
+2. **Simple Test Detection**: Only detects `test_*` functions and `Test*` classes
+3. **No Fixture Support**: Doesn't analyze pytest fixtures or dependencies
+4. **No Parametrization**: Doesn't expand parametrized tests
+
+### Changelog
+
+#### v0.5.0 (Current)
+- 🚀 **Production-Ready Daemon**: Collection daemon upgraded from experimental to production-ready
+- 🔒 **Security**: Comprehensive input validation and path checking to prevent attacks
+- 📊 **Monitoring**: Health checks, metrics tracking, and detailed diagnostics
+- 📝 **Logging**: Structured logging with automatic rotation (10MB files, 5 backups)
+- 🔄 **Reliability**: Automatic retries with exponential backoff
+- 🛡️ **Error Handling**: Comprehensive error handling and recovery mechanisms
+- 🔗 **Connection Management**: Rate limiting, timeouts, and proper resource cleanup
+- ✅ **Testing**: Comprehensive unit and integration tests for daemon
+- 📚 **Documentation**: Complete troubleshooting guide and best practices
+- 🎯 **Health Endpoint**: New `--daemon-health` command for diagnostics
+- 📏 **Benchmark Tool**: New `--benchmark-collect` to test if plugin is beneficial for your project
+
+#### v0.3.0
+- 🏗️ **Better Integration**: Refactored plugin architecture for cleaner code
+- ⚡ Early initialization in `pytest_configure` instead of lazy loading
+- 🔧 Simplified `pytest_ignore_collect` hook to only use cached data
+- 🐛 Fixed duplicate collection issues from custom collector conflicts
+- 📊 Maintains all caching benefits from v0.2.0
+- 🧹 Cleaner separation of concerns and more predictable behavior
+
+#### v0.2.0
+- ✨ Added incremental caching with file modification tracking
+- 💾 Cache persists to `.pytest_cache/v/fastcollect/cache.json`
+- 📊 Shows cache statistics after collection
+- 🚀 ~5% improvement on repeated runs with warm cache
+- 📝 Added `benchmark_incremental.py` for cache effectiveness testing
+
+#### v0.1.0
+- 🎉 Initial release
+- 🦀 Rust-based parallel AST parsing
+- 🎯 File filtering via `pytest_ignore_collect` hook
+- ⚡ Parallel processing with Rayon
+- 📚 Comprehensive documentation
+
+## Contact
+
+For issues, questions, or contributions, please open an issue on GitHub.
+