pytest-fastcollect 0.5.1__cp314-cp314t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- pytest_fastcollect/__init__.py +12 -0
- pytest_fastcollect/cache.py +171 -0
- pytest_fastcollect/constants.py +89 -0
- pytest_fastcollect/daemon.py +881 -0
- pytest_fastcollect/daemon_client.py +581 -0
- pytest_fastcollect/filter.py +204 -0
- pytest_fastcollect/plugin.py +601 -0
- pytest_fastcollect/py.typed +0 -0
- pytest_fastcollect/pytest_fastcollect.cpython-314t-powerpc64le-linux-gnu.so +0 -0
- pytest_fastcollect/socket_strategy.py +217 -0
- pytest_fastcollect-0.5.1.dist-info/METADATA +686 -0
- pytest_fastcollect-0.5.1.dist-info/RECORD +15 -0
- pytest_fastcollect-0.5.1.dist-info/WHEEL +5 -0
- pytest_fastcollect-0.5.1.dist-info/entry_points.txt +2 -0
- pytest_fastcollect-0.5.1.dist-info/licenses/LICENSE +21 -0
|
@@ -0,0 +1,686 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: pytest-fastcollect
|
|
3
|
+
Version: 0.5.1
|
|
4
|
+
Classifier: Framework :: Pytest
|
|
5
|
+
Classifier: Programming Language :: Rust
|
|
6
|
+
Classifier: Programming Language :: Python :: 3
|
|
7
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
8
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
9
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
10
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
11
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
12
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
13
|
+
Classifier: Programming Language :: Python :: Implementation :: CPython
|
|
14
|
+
Classifier: Programming Language :: Python :: Implementation :: PyPy
|
|
15
|
+
Classifier: Topic :: Software Development :: Testing
|
|
16
|
+
Requires-Dist: pytest>=7.0.0
|
|
17
|
+
Requires-Dist: pytest-benchmark>=4.0.0 ; extra == 'dev'
|
|
18
|
+
Provides-Extra: dev
|
|
19
|
+
License-File: LICENSE
|
|
20
|
+
Summary: A high-performance pytest plugin that replaces test collection with a Rust-based implementation
|
|
21
|
+
Requires-Python: >=3.9
|
|
22
|
+
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
|
|
23
|
+
|
|
24
|
+
# pytest-fastcollect
|
|
25
|
+
|
|
26
|
+
[](https://github.com/Samrhan/pytest-fastcollect/actions/workflows/CI.yml)
|
|
27
|
+
[](https://codecov.io/gh/Samrhan/pytest-fastcollect)
|
|
28
|
+
[](https://badge.fury.io/py/pytest-fastcollect)
|
|
29
|
+
[](https://pypi.org/project/pytest-fastcollect/)
|
|
30
|
+
|
|
31
|
+
A high-performance pytest plugin that uses Rust to accelerate test collection. This plugin leverages `rustpython-parser` to parse Python test files in parallel, with incremental caching to skip unchanged files.
|
|
32
|
+
|
|
33
|
+
**Performance**: Up to **2.4x faster** collection on large projects (tested on Django's 1977 test files). Best for codebases with 200+ test files.
|
|
34
|
+
|
|
35
|
+
## Features
|
|
36
|
+
|
|
37
|
+
- ๐ฆ **Rust-Powered Parsing**: Uses `rustpython-parser` for blazing-fast Python AST parsing
|
|
38
|
+
- โก **Parallel Processing**: Leverages Rayon for parallel file processing
|
|
39
|
+
- ๐พ **Incremental Caching**: Caches parsed results with file modification tracking
|
|
40
|
+
- ๐ฏ **Smart Filtering**: Pre-filters test files before pytest's collection phase
|
|
41
|
+
- ๐ง **Drop-in Replacement**: Works as a pytest plugin with no code changes required
|
|
42
|
+
- ๐๏ธ **Configurable**: Enable/disable fast collection and caching with command-line flags
|
|
43
|
+
- ๐ **Scales with Size**: Performance improvements scale with project size (2-4x on 500+ files)
|
|
44
|
+
|
|
45
|
+
## Installation
|
|
46
|
+
|
|
47
|
+
### From Source
|
|
48
|
+
|
|
49
|
+
```bash
|
|
50
|
+
# Clone the repository
|
|
51
|
+
git clone https://github.com/yourusername/pytest-fastcollect.git
|
|
52
|
+
cd pytest-fastcollect
|
|
53
|
+
|
|
54
|
+
# Create a virtual environment
|
|
55
|
+
python -m venv .venv
|
|
56
|
+
source .venv/bin/activate # On Windows: .venv\Scripts\activate
|
|
57
|
+
|
|
58
|
+
# Install maturin and build
|
|
59
|
+
pip install maturin
|
|
60
|
+
maturin develop --release
|
|
61
|
+
|
|
62
|
+
# Or install in production mode
|
|
63
|
+
maturin build --release
|
|
64
|
+
pip install target/wheels/pytest_fastcollect-*.whl
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
### Requirements
|
|
68
|
+
|
|
69
|
+
- Python 3.8+
|
|
70
|
+
- Rust 1.70+
|
|
71
|
+
- pytest 7.0+
|
|
72
|
+
|
|
73
|
+
## Usage
|
|
74
|
+
|
|
75
|
+
### Should I Use This Plugin?
|
|
76
|
+
|
|
77
|
+
Not sure if pytest-fastcollect will help your project? Run the built-in benchmark:
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
pytest --benchmark-collect
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
This will:
|
|
84
|
+
- โฑ๏ธ Measure collection time **with** and **without** the plugin
|
|
85
|
+
- ๐ Analyze your project size and structure
|
|
86
|
+
- ๐ก Provide a clear **recommendation** with actionable advice
|
|
87
|
+
- ๐ฏ Show expected time savings
|
|
88
|
+
|
|
89
|
+
**Example output:**
|
|
90
|
+
```
|
|
91
|
+
======================================================================
|
|
92
|
+
pytest-fastcollect Benchmark
|
|
93
|
+
======================================================================
|
|
94
|
+
|
|
95
|
+
Analyzing your test suite to determine if pytest-fastcollect is beneficial...
|
|
96
|
+
|
|
97
|
+
๐ Project Stats:
|
|
98
|
+
Test files: 245
|
|
99
|
+
Test items: 1,892
|
|
100
|
+
|
|
101
|
+
โก Benchmark 1: WITH pytest-fastcollect
|
|
102
|
+
Running collection with Rust acceleration... Done! (0.342s)
|
|
103
|
+
|
|
104
|
+
๐ Benchmark 2: WITHOUT pytest-fastcollect
|
|
105
|
+
Running standard pytest collection... Done! (1.567s)
|
|
106
|
+
|
|
107
|
+
======================================================================
|
|
108
|
+
๐ Results
|
|
109
|
+
======================================================================
|
|
110
|
+
|
|
111
|
+
โฑ๏ธ Collection Time:
|
|
112
|
+
Standard pytest: 1.567s
|
|
113
|
+
With fastcollect: 0.342s
|
|
114
|
+
Time saved: 1.225s
|
|
115
|
+
Speedup: 4.58x
|
|
116
|
+
|
|
117
|
+
๐ก Recommendation:
|
|
118
|
+
โญโญโญ EXCELLENT
|
|
119
|
+
pytest-fastcollect provides SIGNIFICANT speedup (4.6x faster)!
|
|
120
|
+
โ
Highly recommended for your project.
|
|
121
|
+
โ
You'll save 1.2s on every test run.
|
|
122
|
+
|
|
123
|
+
๐ฆ Project Size Analysis:
|
|
124
|
+
Your project is MEDIUM-LARGE (245 files).
|
|
125
|
+
โ
Good fit for pytest-fastcollect.
|
|
126
|
+
======================================================================
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
### Basic Usage
|
|
130
|
+
|
|
131
|
+
Once installed, the plugin is automatically activated for all pytest runs:
|
|
132
|
+
|
|
133
|
+
```bash
|
|
134
|
+
# Run pytest as normal - fast collection is enabled by default
|
|
135
|
+
pytest
|
|
136
|
+
|
|
137
|
+
# Collect tests only (useful for benchmarking)
|
|
138
|
+
pytest --collect-only
|
|
139
|
+
|
|
140
|
+
# Disable fast collection
|
|
141
|
+
pytest --no-fast-collect
|
|
142
|
+
|
|
143
|
+
# Clear cache and reparse all files
|
|
144
|
+
pytest --fastcollect-clear-cache --collect-only
|
|
145
|
+
|
|
146
|
+
# Disable caching (always parse)
|
|
147
|
+
pytest --no-fastcollect-cache
|
|
148
|
+
|
|
149
|
+
# Benchmark: Test if pytest-fastcollect is beneficial for your project
|
|
150
|
+
pytest --benchmark-collect
|
|
151
|
+
|
|
152
|
+
# Experimental: Parallel module import (2.33x faster on pytest itself!)
|
|
153
|
+
pytest --parallel-import --parallel-workers=4
|
|
154
|
+
|
|
155
|
+
# Production-Ready: Collection Daemon (instant re-collection)
|
|
156
|
+
pytest --daemon-start tests/ # Start daemon
|
|
157
|
+
pytest --daemon-status # Check status
|
|
158
|
+
pytest --daemon-stop # Stop daemon
|
|
159
|
+
pytest --daemon-health # Health check
|
|
160
|
+
|
|
161
|
+
# Run benchmarks
|
|
162
|
+
python benchmark.py --synthetic
|
|
163
|
+
python benchmark_incremental.py # Shows cache effectiveness
|
|
164
|
+
python benchmark_parallel.py # Test parallel import performance
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
### Configuration Options
|
|
168
|
+
|
|
169
|
+
- `--use-fast-collect`: Enable Rust-based fast collection (default: True)
|
|
170
|
+
- `--no-fast-collect`: Disable fast collection and use standard pytest collection
|
|
171
|
+
- `--fastcollect-cache`: Enable incremental caching (default: True)
|
|
172
|
+
- `--no-fastcollect-cache`: Disable caching and parse all files
|
|
173
|
+
- `--fastcollect-clear-cache`: Clear the cache before collection
|
|
174
|
+
- `--benchmark-collect`: **[Recommended]** Benchmark to test if the plugin is beneficial for your project
|
|
175
|
+
- `--parallel-import`: **[Experimental]** Pre-import modules in parallel (default: False)
|
|
176
|
+
- `--parallel-workers=N`: Number of parallel import workers (default: CPU count)
|
|
177
|
+
- `--daemon-start`: **[Production-Ready]** Start collection daemon for instant re-collection
|
|
178
|
+
- `--daemon-stop`: Stop the collection daemon gracefully
|
|
179
|
+
- `--daemon-status`: Show comprehensive daemon status (PID, uptime, cached modules, metrics)
|
|
180
|
+
- `--daemon-health`: Check daemon health and diagnostics
|
|
181
|
+
|
|
182
|
+
## Architecture
|
|
183
|
+
|
|
184
|
+
### How It Works
|
|
185
|
+
|
|
186
|
+
1. **Rust Collector (`FastCollector`)**:
|
|
187
|
+
- Walks the directory tree to find Python test files
|
|
188
|
+
- Uses `rustpython-parser` to parse each file's AST in parallel
|
|
189
|
+
- Extracts file modification times for cache validation
|
|
190
|
+
- Returns collected metadata to Python
|
|
191
|
+
|
|
192
|
+
2. **Incremental Caching**:
|
|
193
|
+
- Caches parsed test data with file mtimes in `.pytest_cache/v/fastcollect/`
|
|
194
|
+
- On subsequent runs, checks file mtimes
|
|
195
|
+
- Only reparses files that have changed
|
|
196
|
+
- Shows cache statistics (hits/misses) after collection
|
|
197
|
+
|
|
198
|
+
3. **pytest Plugin Integration**:
|
|
199
|
+
- Hooks into pytest's `pytest_ignore_collect` to filter files
|
|
200
|
+
- Uses cached data when available
|
|
201
|
+
- Falls back to standard collection on errors
|
|
202
|
+
|
|
203
|
+
4. **File Detection**:
|
|
204
|
+
- Test files: `test_*.py` or `*_test.py`
|
|
205
|
+
- Ignored directories: `.git`, `__pycache__`, `.tox`, `.venv`, `venv`, `.eggs`, `*.egg-info`
|
|
206
|
+
|
|
207
|
+
### Components
|
|
208
|
+
|
|
209
|
+
```
|
|
210
|
+
pytest-fastcollect/
|
|
211
|
+
โโโ src/
|
|
212
|
+
โ โโโ lib.rs # Rust implementation (FastCollector)
|
|
213
|
+
โโโ pytest_fastcollect/
|
|
214
|
+
โ โโโ __init__.py # Python package init
|
|
215
|
+
โ โโโ plugin.py # pytest plugin hooks
|
|
216
|
+
โ โโโ cache.py # Incremental caching layer
|
|
217
|
+
โ โโโ daemon.py # Collection daemon server
|
|
218
|
+
โ โโโ daemon_client.py # Daemon client communication
|
|
219
|
+
โ โโโ filter.py # Selective import filtering
|
|
220
|
+
โโโ tests/
|
|
221
|
+
โ โโโ sample_tests/ # Sample tests for validation
|
|
222
|
+
โโโ benchmark.py # Performance benchmarking
|
|
223
|
+
โโโ benchmark_incremental.py # Cache effectiveness benchmark
|
|
224
|
+
โโโ benchmark_parallel.py # Parallel import benchmarking
|
|
225
|
+
โโโ Cargo.toml # Rust dependencies
|
|
226
|
+
โโโ pyproject.toml # Python package metadata
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
## Benchmarks
|
|
230
|
+
|
|
231
|
+
### v0.4.0 - Selective Import (Latest) โญ
|
|
232
|
+
|
|
233
|
+
**THE BREAKTHROUGH**: Selective import based on `-k` and `-m` filters!
|
|
234
|
+
|
|
235
|
+
**How it works**:
|
|
236
|
+
1. Parse all test files with Rust (parallel, fast)
|
|
237
|
+
2. Extract test names and markers from AST
|
|
238
|
+
3. Apply `-k` and `-m` filters BEFORE importing modules
|
|
239
|
+
4. Only import files containing matching tests
|
|
240
|
+
5. Result: Massive speedups for filtered runs!
|
|
241
|
+
|
|
242
|
+
**Benchmark Results** (100 files, 10 tests/file):
|
|
243
|
+
```
|
|
244
|
+
Scenario Time Speedup
|
|
245
|
+
Full collection (no filter) 0.98s baseline
|
|
246
|
+
With -k filter (10% files) 0.57s 1.71x faster โก
|
|
247
|
+
With -m filter (20% files) 0.64s 1.55x faster โก
|
|
248
|
+
Combined filters 0.55s 1.78x faster โก
|
|
249
|
+
```
|
|
250
|
+
|
|
251
|
+
**Real-World Impact**:
|
|
252
|
+
```bash
|
|
253
|
+
# Before v0.4.0: imports ALL 100 test files
|
|
254
|
+
pytest -k test_user # 0.98s
|
|
255
|
+
|
|
256
|
+
# With v0.4.0: imports only 10 matching files
|
|
257
|
+
pytest -k test_user # 0.57s (1.71x faster!)
|
|
258
|
+
```
|
|
259
|
+
|
|
260
|
+
**When it helps most**:
|
|
261
|
+
- โ
Running specific tests: `pytest -k test_user_login`
|
|
262
|
+
- โ
Running marked tests: `pytest -m smoke`
|
|
263
|
+
- โ
Development workflow (constantly filtering tests)
|
|
264
|
+
- โ
CI/CD with test splits
|
|
265
|
+
- โ
Large test suites with good organization
|
|
266
|
+
|
|
267
|
+
**Key Features**:
|
|
268
|
+
- Marker detection from decorators (`@pytest.mark.slow`)
|
|
269
|
+
- Keyword matching (function names, class names, file names)
|
|
270
|
+
- Supports `and`, `or`, `not` in expressions
|
|
271
|
+
- Shows file selection stats with `-v`
|
|
272
|
+
- Fully compatible with pytest's filter syntax
|
|
273
|
+
|
|
274
|
+
### Multi-Project Real-World Benchmarks ๐
|
|
275
|
+
|
|
276
|
+
Tested on **5 popular Python projects** to validate real-world performance:
|
|
277
|
+
|
|
278
|
+
| Project | Files | Baseline | FastCollect | Speedup | Grade |
|
|
279
|
+
|---------|-------|----------|-------------|---------|-------|
|
|
280
|
+
| **Django** | ~1977 | 10.85s | 4.49s | **2.42x** | โกโกโก Excellent |
|
|
281
|
+
| **SQLAlchemy** | ~219 | 0.68s | 0.63s | **1.07x** | โ Minor |
|
|
282
|
+
| **Pytest** | ~108 | 2.40s | 2.54s | **0.94x** | โ ๏ธ Overhead |
|
|
283
|
+
| **Requests** | ~9 | 0.61s | 0.54s | **1.13x** | โ Minor |
|
|
284
|
+
| **Flask** | ~22 | 0.55s | 0.55s | **1.00x** | โ Neutral |
|
|
285
|
+
|
|
286
|
+
**Key Finding**: Performance scales with project size! ๐
|
|
287
|
+
|
|
288
|
+
**Selective Import Performance** (additional speedup with `-k` filters):
|
|
289
|
+
- **Pytest**: Up to **2.75x faster** with specific filters (`-k test_basic`)
|
|
290
|
+
- **Django**: Additional **1.32x faster** with keyword filters
|
|
291
|
+
- **Small projects**: Minimal additional benefit
|
|
292
|
+
|
|
293
|
+
**Break-Even Analysis**:
|
|
294
|
+
- โ
**Large projects (500+ files)**: **2-4x speedup** - highly recommended
|
|
295
|
+
- โ ๏ธ **Medium projects (100-300 files)**: **0.9-1.5x** - evaluate first
|
|
296
|
+
- โ **Small projects (< 50 files)**: **~1.0x** - not necessary
|
|
297
|
+
|
|
298
|
+
**Bottom Line**: pytest-fastcollect is **ideal for large codebases** (200+ test files) where collection time becomes a bottleneck. For projects with < 50 files, the overhead roughly equals the benefit.
|
|
299
|
+
|
|
300
|
+
๐ See [REALWORLD_BENCHMARKS.md](REALWORLD_BENCHMARKS.md) for comprehensive analysis across all projects.
|
|
301
|
+
|
|
302
|
+
### Parallel Import (Experimental) โกโกโก
|
|
303
|
+
|
|
304
|
+
**NEW**: Pre-import test modules in parallel for additional speedup!
|
|
305
|
+
|
|
306
|
+
```bash
|
|
307
|
+
pytest --parallel-import --parallel-workers=4
|
|
308
|
+
```
|
|
309
|
+
|
|
310
|
+
**Benchmark Results** (with --parallel-import):
|
|
311
|
+
|
|
312
|
+
| Project | Baseline | With Parallel | Speedup | Grade |
|
|
313
|
+
|---------|----------|---------------|---------|-------|
|
|
314
|
+
| **Pytest** | 2.40s | 1.03s | **2.33x faster** | โกโกโก Excellent |
|
|
315
|
+
| **SQLAlchemy** | 0.69s | 0.64s | **1.07x faster** | โ Minor |
|
|
316
|
+
| **Django** | 4.80s | 4.90s | **0.98x slower** | โ ๏ธ Overhead |
|
|
317
|
+
|
|
318
|
+
**Key Finding**: Parallel import works great for projects with **simple, independent test modules** (like pytest itself!), but can hurt projects with complex interdependent imports (like Django).
|
|
319
|
+
|
|
320
|
+
**When to use**:
|
|
321
|
+
- โ
Medium projects (100-300 files) with simple imports โ **2-2.5x speedup**
|
|
322
|
+
- โ ๏ธ Projects with complex imports โ **Benchmark first**
|
|
323
|
+
- โ Small projects (< 100 files) โ **Overhead not worth it**
|
|
324
|
+
|
|
325
|
+
**Optimal configuration**: 4 workers seems to be the sweet spot for most projects.
|
|
326
|
+
|
|
327
|
+
**ProcessPoolExecutor** (experimental):
|
|
328
|
+
```bash
|
|
329
|
+
# Bypass GIL with true process parallelism
|
|
330
|
+
pytest --parallel-import --use-processes --parallel-workers=4
|
|
331
|
+
```
|
|
332
|
+
|
|
333
|
+
**Results**: ProcessPoolExecutor tested but **not recommended**
|
|
334
|
+
- โ Slower than ThreadPoolExecutor in most cases (0.88-1.10x)
|
|
335
|
+
- Process overhead > GIL bypass benefit
|
|
336
|
+
- Must import twice (subprocess + main process)
|
|
337
|
+
- ThreadPoolExecutor remains the better choice
|
|
338
|
+
|
|
339
|
+
๐ See [PARALLEL_IMPORT_RESULTS.md](PARALLEL_IMPORT_RESULTS.md) for threading details.
|
|
340
|
+
๐ See [PROCESS_POOL_RESULTS.md](PROCESS_POOL_RESULTS.md) for process pool analysis.
|
|
341
|
+
|
|
342
|
+
### Collection Daemon (Production-Ready) ๐
|
|
343
|
+
|
|
344
|
+
**Production-Ready**: Long-running daemon process that keeps test modules imported in memory for instant re-collection!
|
|
345
|
+
|
|
346
|
+
```bash
|
|
347
|
+
# Start the daemon (imports all modules once)
|
|
348
|
+
pytest --daemon-start tests/
|
|
349
|
+
|
|
350
|
+
# Check daemon status
|
|
351
|
+
pytest --daemon-status
|
|
352
|
+
|
|
353
|
+
# Check daemon health
|
|
354
|
+
pytest --daemon-health
|
|
355
|
+
|
|
356
|
+
# Stop the daemon
|
|
357
|
+
pytest --daemon-stop
|
|
358
|
+
```
|
|
359
|
+
|
|
360
|
+
**Expected Performance**:
|
|
361
|
+
- First run: ~10s (cold start, imports all modules)
|
|
362
|
+
- Subsequent runs: ~0.01s (instant! modules already in memory)
|
|
363
|
+
- **100-1000x speedup** on subsequent test runs
|
|
364
|
+
|
|
365
|
+
**Production Features**:
|
|
366
|
+
- โ
Robust daemon server with Unix socket communication
|
|
367
|
+
- โ
Module pre-importing and caching in memory
|
|
368
|
+
- โ
Start/stop/status/health management commands
|
|
369
|
+
- โ
Comprehensive error handling and logging
|
|
370
|
+
- โ
Input validation and security checks
|
|
371
|
+
- โ
Connection management and rate limiting
|
|
372
|
+
- โ
Metrics tracking and monitoring
|
|
373
|
+
- โ
Graceful shutdown handling
|
|
374
|
+
- โ
Automatic retry with exponential backoff
|
|
375
|
+
- โ
Production-grade logging with rotation
|
|
376
|
+
- โณ Full pytest collection integration (Phase 2)
|
|
377
|
+
- โณ File watching for auto-reload (Phase 3)
|
|
378
|
+
|
|
379
|
+
**Architecture**:
|
|
380
|
+
- Long-running Python process
|
|
381
|
+
- Unix socket IPC for client-daemon communication
|
|
382
|
+
- Keeps modules in `sys.modules` across pytest runs
|
|
383
|
+
- Background process management with forking
|
|
384
|
+
- Per-project daemon instances (separate socket per root)
|
|
385
|
+
|
|
386
|
+
**When to use**:
|
|
387
|
+
- ๐ฏ **TDD workflows**: Constantly re-running tests during development
|
|
388
|
+
- ๐ฏ **Watch mode**: Instant collection on file changes
|
|
389
|
+
- ๐ฏ **Large codebases**: Where collection time > 5 seconds
|
|
390
|
+
- ๐ฏ **Development environments**: Optimized for rapid iteration
|
|
391
|
+
- โ ๏ธ **Not for CI/CD**: Designed for development, not one-shot runs
|
|
392
|
+
|
|
393
|
+
**Production-Ready Features**:
|
|
394
|
+
- โ
Unix/Linux support (uses Unix domain sockets)
|
|
395
|
+
- โ
Comprehensive error handling and recovery
|
|
396
|
+
- โ
Security: Input validation and path checking
|
|
397
|
+
- โ
Monitoring: Health checks and metrics
|
|
398
|
+
- โ
Logging: Structured logs with automatic rotation
|
|
399
|
+
- โ
Resource management: Connection limits and timeouts
|
|
400
|
+
- โ
Graceful shutdown and cleanup
|
|
401
|
+
- โ
Comprehensive test coverage
|
|
402
|
+
|
|
403
|
+
**Remaining Limitations**:
|
|
404
|
+
- Phase 1: Infrastructure ready, pytest collection integration in progress
|
|
405
|
+
- Manual daemon management (start/stop) - automation coming in Phase 2
|
|
406
|
+
- File watching not yet implemented - planned for Phase 3
|
|
407
|
+
|
|
408
|
+
๐ See [COLLECTION_DAEMON_PLAN.md](COLLECTION_DAEMON_PLAN.md) for full implementation roadmap.
|
|
409
|
+
|
|
410
|
+
### Django Real-World Benchmark ๐
|
|
411
|
+
|
|
412
|
+
**The ultimate test**: Django's massive test suite with **~1977 Python test files**!
|
|
413
|
+
|
|
414
|
+
**Full Collection Performance**:
|
|
415
|
+
```
|
|
416
|
+
Scenario Time Speedup
|
|
417
|
+
Baseline (no plugin) 36.59s -
|
|
418
|
+
FastCollect 9.16s 3.99x faster โกโกโก
|
|
419
|
+
```
|
|
420
|
+
|
|
421
|
+
**Selective Import Performance**:
|
|
422
|
+
```
|
|
423
|
+
Filter Type Time Speedup vs Full
|
|
424
|
+
Full collection 9.16s baseline
|
|
425
|
+
-k test_get 4.12s 2.22x faster โกโก
|
|
426
|
+
-k test_forms 3.80s 2.41x faster โกโก
|
|
427
|
+
-k "test_view or test_model" 4.19s 2.19x faster โกโก
|
|
428
|
+
```
|
|
429
|
+
|
|
430
|
+
**Combined Impact**: FastCollect + Selective Import = **9.6x faster** than baseline pytest!
|
|
431
|
+
- Baseline: 36.59s โ FastCollect + filter: 3.80s
|
|
432
|
+
|
|
433
|
+
**Key Takeaways**:
|
|
434
|
+
- โ
**4x faster** on full collection (real-world production codebase)
|
|
435
|
+
- โ
**2-2.4x additional speedup** with keyword filters
|
|
436
|
+
- โ
**Nearly 10x overall** when combining both optimizations
|
|
437
|
+
- โ
**Production-ready** on Django's complex test infrastructure
|
|
438
|
+
- โ
**Zero configuration** required - works out of the box
|
|
439
|
+
|
|
440
|
+
๐ See [DJANGO_BENCHMARK_RESULTS.md](DJANGO_BENCHMARK_RESULTS.md) for detailed analysis.
|
|
441
|
+
|
|
442
|
+
### v0.3.0 - Better Integration
|
|
443
|
+
|
|
444
|
+
**Architecture Improvements**:
|
|
445
|
+
```
|
|
446
|
+
Early initialization: Collection happens in pytest_configure
|
|
447
|
+
File filtering: Simplified pytest_ignore_collect hook
|
|
448
|
+
Collection overhead: Reduced hook call complexity
|
|
449
|
+
Code quality: Cleaner separation of concerns
|
|
450
|
+
```
|
|
451
|
+
|
|
452
|
+
**Performance** (comparable to v0.2.0):
|
|
453
|
+
- Maintains incremental caching benefits (~5% improvement on warm cache)
|
|
454
|
+
- No duplicate collection issues
|
|
455
|
+
- Cleaner initialization flow
|
|
456
|
+
|
|
457
|
+
**Key Changes**:
|
|
458
|
+
- Moved Rust collection from lazy (first `pytest_ignore_collect` call) to eager (in `pytest_configure`)
|
|
459
|
+
- Simplified `pytest_ignore_collect` to only use pre-collected data
|
|
460
|
+
- Removed custom collector class to avoid pytest hook conflicts
|
|
461
|
+
- More predictable and testable architecture
|
|
462
|
+
|
|
463
|
+
### v0.2.0 - Incremental Caching
|
|
464
|
+
|
|
465
|
+
**Incremental Collection Benchmark (500 files, 20 tests/file, 5 files modified)**:
|
|
466
|
+
```
|
|
467
|
+
Cold start (no cache): 3.34s (baseline)
|
|
468
|
+
Warm cache (no changes): 3.18s (1.05x faster)
|
|
469
|
+
Incremental (5 files changed): 3.50s (cache + reparse 1%)
|
|
470
|
+
|
|
471
|
+
Cache effectiveness:
|
|
472
|
+
- 100% cache hit rate on unchanged files
|
|
473
|
+
- Only modified files are reparsed
|
|
474
|
+
- ~5% improvement on warm cache
|
|
475
|
+
- Persistent across pytest runs
|
|
476
|
+
```
|
|
477
|
+
|
|
478
|
+
**Cache Statistics** (displayed after collection with `-v`):
|
|
479
|
+
```
|
|
480
|
+
FastCollect Cache: 2 files from cache, 0 parsed (100.0% hit rate)
|
|
481
|
+
```
|
|
482
|
+
|
|
483
|
+
### v0.1.0 - File Filtering Only
|
|
484
|
+
|
|
485
|
+
**Synthetic Benchmarks**:
|
|
486
|
+
- **200 files, 50 tests/file**: ~1.01x speedup
|
|
487
|
+
- **500 files, 100 tests/file**: ~1.01x speedup
|
|
488
|
+
|
|
489
|
+
**Real-World (pandas, 969 test files)**:
|
|
490
|
+
- File filtering: 0.75x (slower due to overhead)
|
|
491
|
+
|
|
492
|
+
### Performance Analysis
|
|
493
|
+
|
|
494
|
+
**Why Limited Speedup in Pure Collection?**
|
|
495
|
+
|
|
496
|
+
1. **Pytest Import Bottleneck**: Even with caching, pytest must import Python modules to get actual function/class objects
|
|
497
|
+
2. **File Filtering Overhead**: The `pytest_ignore_collect` hook is called for every path
|
|
498
|
+
3. **Duplicate Work**: Both Rust (for discovery) and Python (for import) process files
|
|
499
|
+
|
|
500
|
+
**Where Caching Provides Real Value:**
|
|
501
|
+
|
|
502
|
+
1. โ
**Incremental Workflows**: Only reparse modified files (5-10% improvement)
|
|
503
|
+
2. โ
**Large Codebases**: Faster discovery in deep directory trees
|
|
504
|
+
3. โ
**CI/CD Pipelines**: Cache persists across runs
|
|
505
|
+
4. โ
**Development Workflow**: Repeated `pytest --collect-only` calls
|
|
506
|
+
5. โ
**Watch Mode**: Quick re-collection when files change
|
|
507
|
+
|
|
508
|
+
### Recent Improvements (v0.2.0)
|
|
509
|
+
|
|
510
|
+
โ
**Incremental Caching** - Cache parsed results with file modification tracking
|
|
511
|
+
- Persists to `.pytest_cache/v/fastcollect/cache.json`
|
|
512
|
+
- Only reparses files that have changed
|
|
513
|
+
- Shows cache statistics after collection
|
|
514
|
+
- ~5% improvement on repeated runs
|
|
515
|
+
|
|
516
|
+
### Future Optimizations
|
|
517
|
+
|
|
518
|
+
To achieve even greater speedup:
|
|
519
|
+
|
|
520
|
+
1. **Direct Item Creation**: Create pytest `Item` objects directly from Rust (complex, see IMPLEMENTATION_NOTES.md)
|
|
521
|
+
2. **Lazy Loading**: Only parse files when their tests are actually executed
|
|
522
|
+
3. **Better Integration**: Use pytest's lower-level APIs to bypass standard collection
|
|
523
|
+
4. **Parallel Imports**: Import Python modules in parallel
|
|
524
|
+
|
|
525
|
+
## Technical Details
|
|
526
|
+
|
|
527
|
+
### Rust Dependencies
|
|
528
|
+
|
|
529
|
+
- `pyo3`: Python bindings for Rust
|
|
530
|
+
- `rustpython-parser`: Python AST parser in Rust
|
|
531
|
+
- `walkdir`: Recursive directory traversal
|
|
532
|
+
- `rayon`: Data parallelism library
|
|
533
|
+
|
|
534
|
+
### Python API
|
|
535
|
+
|
|
536
|
+
```python
|
|
537
|
+
from pytest_fastcollect import FastCollector
|
|
538
|
+
|
|
539
|
+
# Create a collector for a directory
|
|
540
|
+
collector = FastCollector("/path/to/tests")
|
|
541
|
+
|
|
542
|
+
# Collect all tests (basic mode)
|
|
543
|
+
results = collector.collect()
|
|
544
|
+
# Returns: {"file_path": [{"name": "test_foo", "line": 10, "type": "Function"}, ...]}
|
|
545
|
+
|
|
546
|
+
# Collect with metadata (includes file mtimes for caching)
|
|
547
|
+
metadata = collector.collect_with_metadata()
|
|
548
|
+
# Returns: {"file_path": {"mtime": 1234567890.0, "items": [...]}}
|
|
549
|
+
```
|
|
550
|
+
|
|
551
|
+
## Development
|
|
552
|
+
|
|
553
|
+
### Building from Source
|
|
554
|
+
|
|
555
|
+
```bash
|
|
556
|
+
# Development build (faster compilation, slower runtime)
|
|
557
|
+
maturin develop
|
|
558
|
+
|
|
559
|
+
# Release build (slower compilation, faster runtime)
|
|
560
|
+
maturin develop --release
|
|
561
|
+
|
|
562
|
+
# Build wheel
|
|
563
|
+
maturin build --release
|
|
564
|
+
```
|
|
565
|
+
|
|
566
|
+
### Running Tests
|
|
567
|
+
|
|
568
|
+
```bash
|
|
569
|
+
# Run sample tests
|
|
570
|
+
pytest tests/sample_tests -v
|
|
571
|
+
|
|
572
|
+
# Run with fast collection disabled
|
|
573
|
+
pytest tests/sample_tests --no-fast-collect -v
|
|
574
|
+
|
|
575
|
+
# Collect only (no execution)
|
|
576
|
+
pytest tests/sample_tests --collect-only
|
|
577
|
+
|
|
578
|
+
# View cache statistics
|
|
579
|
+
pytest tests/sample_tests --collect-only -v
|
|
580
|
+
```
|
|
581
|
+
|
|
582
|
+
### Running Benchmarks
|
|
583
|
+
|
|
584
|
+
```bash
|
|
585
|
+
# Synthetic benchmark with custom parameters
|
|
586
|
+
python benchmark.py --synthetic --num-files 200 --tests-per-file 100
|
|
587
|
+
|
|
588
|
+
# Incremental caching benchmark (shows cache effectiveness)
|
|
589
|
+
python benchmark_incremental.py
|
|
590
|
+
|
|
591
|
+
# Benchmark on a real project
|
|
592
|
+
python benchmark.py --project /path/to/project
|
|
593
|
+
|
|
594
|
+
# Run all benchmarks
|
|
595
|
+
python benchmark.py --all
|
|
596
|
+
```
|
|
597
|
+
|
|
598
|
+
### Cache Management
|
|
599
|
+
|
|
600
|
+
```bash
|
|
601
|
+
# Clear the cache
|
|
602
|
+
pytest --fastcollect-clear-cache --collect-only
|
|
603
|
+
|
|
604
|
+
# Disable caching for one run
|
|
605
|
+
pytest --no-fastcollect-cache
|
|
606
|
+
|
|
607
|
+
# View cache contents
|
|
608
|
+
cat .pytest_cache/v/fastcollect/cache.json
|
|
609
|
+
```
|
|
610
|
+
|
|
611
|
+
## Contributing
|
|
612
|
+
|
|
613
|
+
Contributions are welcome! Areas for improvement:
|
|
614
|
+
|
|
615
|
+
1. **Performance Optimization**: Implement direct `Item` creation from Rust data
|
|
616
|
+
2. **Advanced Caching**: Add file content hashing for more reliable cache validation
|
|
617
|
+
3. **Test Discovery**: Support more complex test patterns (fixtures, parameterization)
|
|
618
|
+
4. **Configuration**: Add support for custom test patterns and ignore rules
|
|
619
|
+
5. **Documentation**: Add more examples and use cases
|
|
620
|
+
|
|
621
|
+
## License
|
|
622
|
+
|
|
623
|
+
This project is licensed under the MIT License - see the LICENSE file for details.
|
|
624
|
+
|
|
625
|
+
## Acknowledgments
|
|
626
|
+
|
|
627
|
+
- Built with [PyO3](https://pyo3.rs/) for seamless Rust-Python integration
|
|
628
|
+
- Uses [RustPython Parser](https://github.com/RustPython/RustPython) for Python AST parsing
|
|
629
|
+
- Inspired by the need for faster test collection in large Python codebases
|
|
630
|
+
|
|
631
|
+
## Technical Notes
|
|
632
|
+
|
|
633
|
+
### Why Rust?
|
|
634
|
+
|
|
635
|
+
- **Speed**: Rust's zero-cost abstractions and lack of GIL make it ideal for CPU-intensive parsing
|
|
636
|
+
- **Parallelism**: Rayon makes it trivial to parse files in parallel
|
|
637
|
+
- **Safety**: Rust's type system ensures memory safety without garbage collection overhead
|
|
638
|
+
|
|
639
|
+
### Current Limitations
|
|
640
|
+
|
|
641
|
+
1. **Limited Pure Collection Speedup**: Pytest still needs to import modules (~5% improvement)
|
|
642
|
+
2. **Simple Test Detection**: Only detects `test_*` functions and `Test*` classes
|
|
643
|
+
3. **No Fixture Support**: Doesn't analyze pytest fixtures or dependencies
|
|
644
|
+
4. **No Parametrization**: Doesn't expand parametrized tests
|
|
645
|
+
|
|
646
|
+
### Changelog
|
|
647
|
+
|
|
648
|
+
#### v0.5.0 (Current)
|
|
649
|
+
- ๐ **Production-Ready Daemon**: Collection daemon upgraded from experimental to production-ready
|
|
650
|
+
- ๐ **Security**: Comprehensive input validation and path checking to prevent attacks
|
|
651
|
+
- ๐ **Monitoring**: Health checks, metrics tracking, and detailed diagnostics
|
|
652
|
+
- ๐ **Logging**: Structured logging with automatic rotation (10MB files, 5 backups)
|
|
653
|
+
- ๐ **Reliability**: Automatic retries with exponential backoff
|
|
654
|
+
- ๐ก๏ธ **Error Handling**: Comprehensive error handling and recovery mechanisms
|
|
655
|
+
- ๐ **Connection Management**: Rate limiting, timeouts, and proper resource cleanup
|
|
656
|
+
- โ
**Testing**: Comprehensive unit and integration tests for daemon
|
|
657
|
+
- ๐ **Documentation**: Complete troubleshooting guide and best practices
|
|
658
|
+
- ๐ฏ **Health Endpoint**: New `--daemon-health` command for diagnostics
|
|
659
|
+
- ๐ **Benchmark Tool**: New `--benchmark-collect` to test if plugin is beneficial for your project
|
|
660
|
+
|
|
661
|
+
#### v0.3.0
|
|
662
|
+
- ๐๏ธ **Better Integration**: Refactored plugin architecture for cleaner code
|
|
663
|
+
- โก Early initialization in `pytest_configure` instead of lazy loading
|
|
664
|
+
- ๐ง Simplified `pytest_ignore_collect` hook to only use cached data
|
|
665
|
+
- ๐ Fixed duplicate collection issues from custom collector conflicts
|
|
666
|
+
- ๐ Maintains all caching benefits from v0.2.0
|
|
667
|
+
- ๐งน Cleaner separation of concerns and more predictable behavior
|
|
668
|
+
|
|
669
|
+
#### v0.2.0
|
|
670
|
+
- โจ Added incremental caching with file modification tracking
|
|
671
|
+
- ๐พ Cache persists to `.pytest_cache/v/fastcollect/cache.json`
|
|
672
|
+
- ๐ Shows cache statistics after collection
|
|
673
|
+
- ๐ ~5% improvement on repeated runs with warm cache
|
|
674
|
+
- ๐ Added `benchmark_incremental.py` for cache effectiveness testing
|
|
675
|
+
|
|
676
|
+
#### v0.1.0
|
|
677
|
+
- ๐ Initial release
|
|
678
|
+
- ๐ฆ Rust-based parallel AST parsing
|
|
679
|
+
- ๐ฏ File filtering via `pytest_ignore_collect` hook
|
|
680
|
+
- โก Parallel processing with Rayon
|
|
681
|
+
- ๐ Comprehensive documentation
|
|
682
|
+
|
|
683
|
+
## Contact
|
|
684
|
+
|
|
685
|
+
For issues, questions, or contributions, please open an issue on GitHub.
|
|
686
|
+
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
pytest_fastcollect-0.5.1.dist-info/METADATA,sha256=2wI1pAqBl7B8fDhllPYxkyZLPOvRqhWjqCmk9bmnLWU,26070
|
|
2
|
+
pytest_fastcollect-0.5.1.dist-info/WHEEL,sha256=bx89wP1GetWIcDwYk_3q9qMfIrdlquq9NrEvvgZlRuA,151
|
|
3
|
+
pytest_fastcollect-0.5.1.dist-info/entry_points.txt,sha256=avD7vzLBXlmW0XzgjsdUI6GC6oO576XUSwNI4ny-YG8,49
|
|
4
|
+
pytest_fastcollect-0.5.1.dist-info/licenses/LICENSE,sha256=fUq8yKGFAGr5FOA8nAAe57puhN9LxPxlV8uKYSdoALo,1090
|
|
5
|
+
pytest_fastcollect/__init__.py,sha256=pJtiIfoME-UDbCjJ9pmg_5TfoIUx0cLiibwaOQ5CPMw,369
|
|
6
|
+
pytest_fastcollect/cache.py,sha256=quqhge83vZfBmX4gaj3et-Z3zaeIjgLIy5XV5of0I9Q,5823
|
|
7
|
+
pytest_fastcollect/constants.py,sha256=crQp4tBKipWT1CP6aX5xBRnwsErv5y1dCU6S4N7SwJY,2122
|
|
8
|
+
pytest_fastcollect/daemon.py,sha256=eyOpmBN5dLtk4nJzf-RULg2SU3dKmdprN_AaqN_we2U,31988
|
|
9
|
+
pytest_fastcollect/daemon_client.py,sha256=1NBRS8KtHkLXGiFwQtkMXIxMOs7jJKxbRfloHgygm7g,17989
|
|
10
|
+
pytest_fastcollect/filter.py,sha256=BkgS4thuRauj8PnrKIncGcYEvp6-OesDpNFtKwpuNx0,6918
|
|
11
|
+
pytest_fastcollect/plugin.py,sha256=m6M4P49nTSjbrKwfEG9jm_DgfSqcDq1BQEXlkZ5_zME,24219
|
|
12
|
+
pytest_fastcollect/py.typed,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
|
|
13
|
+
pytest_fastcollect/pytest_fastcollect.cpython-314t-powerpc64le-linux-gnu.so,sha256=WWdMUTV7LV-F9sSj5B_cD1PDVp8T8hUydoZvJSrtbKY,4990456
|
|
14
|
+
pytest_fastcollect/socket_strategy.py,sha256=zmkLB6KMg6Yc1C8mU7WclyWF6ifPbPq0e_DADh0aPuE,7255
|
|
15
|
+
pytest_fastcollect-0.5.1.dist-info/RECORD,,
|