robotframework-testselection 0.1.0__tar.gz → 0.3.0__tar.gz

{robotframework_testselection-0.1.0 → robotframework_testselection-0.3.0}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.4
 Name: robotframework-testselection
-Version: 0.1.0
-Summary: Vector-based diverse test case selection for Robot Framework
+Version: 0.3.0
+Summary: Vector-based diverse test case selection for Robot Framework and pytest
 Project-URL: Homepage, https://github.com/manykarim/robotframework-testselection
 Project-URL: Documentation, https://github.com/manykarim/robotframework-testselection#readme
 Project-URL: Repository, https://github.com/manykarim/robotframework-testselection
@@ -10,8 +10,9 @@ Project-URL: Changelog, https://github.com/manykarim/robotframework-testselectio
 Author: Many Kasiriha
 License: Apache-2.0
 License-File: LICENSE
-Keywords: diversity,embeddings,machine-learning,nlp,robotframework,test-selection,testing
+Keywords: diversity,embeddings,machine-learning,nlp,pytest,robotframework,test-selection,testing
 Classifier: Development Status :: 3 - Alpha
+Classifier: Framework :: Pytest
 Classifier: Framework :: Robot Framework
 Classifier: Framework :: Robot Framework :: Library
 Classifier: Framework :: Robot Framework :: Tool
@@ -58,7 +59,7 @@ Description-Content-Type: text/markdown
 
 # robotframework-testselection
 
-Vector-based diverse test case selection for Robot Framework. Embeds test cases as semantic vectors and selects maximally diverse subsets to reduce test suite execution time while preserving coverage breadth.
+Vector-based diverse test case selection for Robot Framework and pytest. Embeds test cases as semantic vectors and selects maximally diverse subsets to reduce test suite execution time while preserving coverage breadth.
 
 ## How It Works
 
@@ -101,6 +102,8 @@ uv sync --extra vectorize
 uv sync --extra all
 ```
 
+The pytest plugin is included in the base install and activates when you pass `--diverse-k` to pytest.
+
 ### Dependency Groups
 
 | Group | Packages | Purpose |
@@ -158,6 +161,62 @@ testcase-select vectorize \
   --datadriver-csv tests/data/login.csv tests/data/search.csv
 ```
 
+### Passing Robot Framework Options
+
+All arguments after `--` are forwarded directly to `robot`. This lets you combine diversity selection with any Robot Framework option — variables, tag filters, log levels, listeners, etc.
+
+```bash
+# Pass variables and set log level
+testcase-select run --suite tests/ --k 20 \
+  -- --variable ENV:staging --variable USER:admin --loglevel DEBUG
+
+# Use Robot's own tag filtering on top of diversity selection
+testcase-select execute --suite tests/ --selection sel.json \
+  -- --include smoke --exclude manual
+
+# Add metadata and custom output name
+testcase-select run --suite tests/ --k 30 \
+  -- --metadata Version:2.1 --name "Smoke Regression"
+
+# Set variables file and debug log
+testcase-select run --suite tests/ --k 50 \
+  -- --variablefile config/env_staging.py --debugfile debug.log
+```
+
+This works with both `run` and `execute` subcommands, including during graceful fallback (when selection fails and all tests are run).
+
+### pytest Support
+
+The diversity selection algorithm also works with pytest test suites, via a pytest plugin:
+
+```bash
+# Select 20 most diverse tests
+pytest --diverse-k=20 tests/
+
+# With custom strategy and seed
+pytest --diverse-k=30 --diverse-strategy=fps_multi --diverse-seed=123 tests/
+
+# Filter by markers before selection
+pytest --diverse-k=20 --diverse-include-markers slow integration tests/
+
+# Group parametrized tests (select at group level)
+pytest --diverse-k=20 --diverse-group-parametrize tests/
+```
+
+Or via the `testcase-select` CLI:
+
+```bash
+# Full pipeline with pytest
+testcase-select run --framework pytest --suite tests/ --k 20 --strategy fps
+
+# Stage-by-stage
+testcase-select vectorize --framework pytest --suite tests/ --output ./artifacts/
+testcase-select select --artifacts ./artifacts/ --k 20
+testcase-select execute --framework pytest --suite tests/ --selection selected_tests.json
+```
+
+The pytest plugin is installed automatically and activated by `--diverse-k`. It uses AST-based text representation with sentence embeddings for test diversity analysis.
+
 ### Direct Robot Framework Integration
 
 You can also use the components directly with `robot`:
@@ -323,6 +382,11 @@ src/TestSelection/
     prerun_modifier.py # DiversePreRunModifier (SuiteVisitor)
     listener.py        # DiverseDataDriverListener (Listener v3)
     runner.py          # ExecutionRunner
+  pytest/              # pytest integration
+    plugin.py          # pytest plugin (pytest11 entry point)
+    collector.py       # Programmatic test collection
+    text_builder.py    # AST-based text representation
+    runner.py          # pytest execution
   pipeline/            # Orchestration layer
     vectorize.py       # Stage 1 orchestrator
     select.py          # Stage 2 orchestrator
@@ -377,6 +441,68 @@ uv run pytest -m integration
 uv run pytest -m benchmark
 ```
 
+## Coverage Retention Analysis
+
+Measured on [robotframework-doctestlibrary](https://github.com/manykarim/robotframework-doctestlibrary) (5,045 statements in the DocTest package). All strategies use `all-MiniLM-L6-v2` embeddings (384-dim), seed=42.
+
+### pytest (594 tests, 70.1% full coverage)
+
+| Strategy | Selection | Tests | Coverage | Retention | Speedup |
+|---|---|---|---|---|---|
+| *(full suite)* | 100% | 594 | 70.1% | baseline | 1.0x |
+| **fps** | **50%** | **297** | **65.9%** | **93.9%** | **3.4x** |
+| dpp | 50% | 297 | 65.9% | 93.9% | 3.5x |
+| kmedoids | 50% | 297 | 65.8% | 93.8% | 3.1x |
+| fps_multi | 50% | 297 | 65.7% | 93.7% | 3.0x |
+| facility | 50% | 297 | 64.9% | 92.6% | 2.9x |
+| random | 50% | 297 | 64.2% | 91.5% | 2.3x |
+| **facility** | **20%** | **119** | **58.6%** | **83.6%** | **7.5x** |
+| kmedoids | 20% | 119 | 58.3% | 83.1% | 6.7x |
+| fps | 20% | 119 | 56.1% | 80.0% | 6.7x |
+| fps_multi | 20% | 119 | 53.8% | 76.8% | 8.0x |
+| random | 20% | 119 | 53.2% | 75.8% | 5.4x |
+| dpp | 20% | 119 | 47.6% | 67.8% | 6.0x |
+| **fps_multi** | **10%** | **60** | **47.7%** | **67.9%** | **18.5x** |
+| kmedoids | 10% | 60 | 46.9% | 66.9% | 14.9x |
+| facility | 10% | 60 | 46.8% | 66.8% | 11.5x |
+| random | 10% | 60 | 44.3% | 63.1% | 10.3x |
+| dpp | 10% | 60 | 42.9% | 61.1% | 13.7x |
+| fps | 10% | 60 | 40.6% | 57.9% | 21.4x |
+
+### Robot Framework (126 tests, 63.5% full coverage)
+
+| Strategy | Selection | Tests | Coverage | Retention | Speedup |
+|---|---|---|---|---|---|
+| *(full suite)* | 100% | 126 | 63.5% | baseline | 1.0x |
+| **fps_multi** | **50%** | **63** | **61.6%** | **97.0%** | **1.5x** |
+| facility | 50% | 63 | 61.6% | 96.9% | 1.5x |
+| fps | 50% | 63 | 61.2% | 96.4% | 1.6x |
+| kmedoids | 50% | 63 | 60.5% | 95.3% | 1.5x |
+| dpp | 50% | 63 | 58.5% | 92.0% | 1.7x |
+| random | 50% | 63 | 57.2% | 90.1% | 2.3x |
+| **facility** | **20%** | **26** | **51.8%** | **81.5%** | **4.4x** |
+| kmedoids | 20% | 26 | 47.5% | 74.8% | 5.5x |
+| random | 20% | 26 | 46.0% | 72.5% | 5.7x |
+| fps | 20% | 26 | 45.3% | 71.4% | 3.1x |
+| fps_multi | 20% | 26 | 45.3% | 71.4% | 2.9x |
+| dpp | 20% | 26 | 43.9% | 69.0% | 4.6x |
+| **kmedoids** | **10%** | **13** | **42.3%** | **66.6%** | **11.3x** |
+| dpp | 10% | 13 | 40.9% | 64.3% | 10.7x |
+| facility | 10% | 13 | 40.2% | 63.3% | 12.9x |
+| random | 10% | 13 | 37.6% | 59.2% | 11.3x |
+| fps_multi | 10% | 13 | 37.5% | 59.1% | 10.7x |
+| fps | 10% | 13 | 33.2% | 52.3% | 8.3x |
+
+### Key Findings
+
+- **50% selection retains 92-97% of coverage** across all strategies — the diversity algorithms select tests maximizing behavioral breadth, with minimal redundancy loss
+- **Facility location excels at 20% selection** — 83.6% retention on pytest, 81.5% on Robot Framework, making it the best choice for aggressive smoke-test suites
+- **kmedoids matches facility at 20%** on pytest (83.1% retention) and leads at 10% on Robot Framework (66.6%)
+- **fps_multi leads at 10% pytest** (67.9% retention) where initial-point sensitivity matters most
+- **DPP performs well at 50%** (93.9% retention, matching FPS) but drops at 20% due to FPS fallback for large k values
+- **All diversity strategies outperform random** at 50% by 2-7 percentage points of retention
+- At 50% robot selection, fps_multi achieves **97% retention** — meaning half the tests contribute almost no incremental coverage
+
 ## Performance
 
 Benchmarked on 384-dimensional normalized vectors:

{robotframework_testselection-0.1.0 → robotframework_testselection-0.3.0}/README.md

@@ -1,6 +1,6 @@
 # robotframework-testselection
 
-Vector-based diverse test case selection for Robot Framework. Embeds test cases as semantic vectors and selects maximally diverse subsets to reduce test suite execution time while preserving coverage breadth.
+Vector-based diverse test case selection for Robot Framework and pytest. Embeds test cases as semantic vectors and selects maximally diverse subsets to reduce test suite execution time while preserving coverage breadth.
 
 ## How It Works
 
@@ -43,6 +43,8 @@ uv sync --extra vectorize
 uv sync --extra all
 ```
 
+The pytest plugin is included in the base install and activates when you pass `--diverse-k` to pytest.
+
 ### Dependency Groups
 
 | Group | Packages | Purpose |
@@ -100,6 +102,62 @@ testcase-select vectorize \
   --datadriver-csv tests/data/login.csv tests/data/search.csv
 ```
 
+### Passing Robot Framework Options
+
+All arguments after `--` are forwarded directly to `robot`. This lets you combine diversity selection with any Robot Framework option — variables, tag filters, log levels, listeners, etc.
+
+```bash
+# Pass variables and set log level
+testcase-select run --suite tests/ --k 20 \
+  -- --variable ENV:staging --variable USER:admin --loglevel DEBUG
+
+# Use Robot's own tag filtering on top of diversity selection
+testcase-select execute --suite tests/ --selection sel.json \
+  -- --include smoke --exclude manual
+
+# Add metadata and custom output name
+testcase-select run --suite tests/ --k 30 \
+  -- --metadata Version:2.1 --name "Smoke Regression"
+
+# Set variables file and debug log
+testcase-select run --suite tests/ --k 50 \
+  -- --variablefile config/env_staging.py --debugfile debug.log
+```
+
+This works with both `run` and `execute` subcommands, including during graceful fallback (when selection fails and all tests are run).
+
+### pytest Support
+
+The diversity selection algorithm also works with pytest test suites, via a pytest plugin:
+
+```bash
+# Select 20 most diverse tests
+pytest --diverse-k=20 tests/
+
+# With custom strategy and seed
+pytest --diverse-k=30 --diverse-strategy=fps_multi --diverse-seed=123 tests/
+
+# Filter by markers before selection
+pytest --diverse-k=20 --diverse-include-markers slow integration tests/
+
+# Group parametrized tests (select at group level)
+pytest --diverse-k=20 --diverse-group-parametrize tests/
+```
+
+Or via the `testcase-select` CLI:
+
+```bash
+# Full pipeline with pytest
+testcase-select run --framework pytest --suite tests/ --k 20 --strategy fps
+
+# Stage-by-stage
+testcase-select vectorize --framework pytest --suite tests/ --output ./artifacts/
+testcase-select select --artifacts ./artifacts/ --k 20
+testcase-select execute --framework pytest --suite tests/ --selection selected_tests.json
+```
+
+The pytest plugin is installed automatically and activated by `--diverse-k`. It uses AST-based text representation with sentence embeddings for test diversity analysis.
+
 ### Direct Robot Framework Integration
 
 You can also use the components directly with `robot`:
@@ -265,6 +323,11 @@ src/TestSelection/
     prerun_modifier.py # DiversePreRunModifier (SuiteVisitor)
     listener.py        # DiverseDataDriverListener (Listener v3)
     runner.py          # ExecutionRunner
+  pytest/              # pytest integration
+    plugin.py          # pytest plugin (pytest11 entry point)
+    collector.py       # Programmatic test collection
+    text_builder.py    # AST-based text representation
+    runner.py          # pytest execution
   pipeline/            # Orchestration layer
     vectorize.py       # Stage 1 orchestrator
     select.py          # Stage 2 orchestrator
@@ -319,6 +382,68 @@ uv run pytest -m integration
 uv run pytest -m benchmark
 ```
 
+## Coverage Retention Analysis
+
+Measured on [robotframework-doctestlibrary](https://github.com/manykarim/robotframework-doctestlibrary) (5,045 statements in the DocTest package). All strategies use `all-MiniLM-L6-v2` embeddings (384-dim), seed=42.
+
+### pytest (594 tests, 70.1% full coverage)
+
+| Strategy | Selection | Tests | Coverage | Retention | Speedup |
+|---|---|---|---|---|---|
+| *(full suite)* | 100% | 594 | 70.1% | baseline | 1.0x |
+| **fps** | **50%** | **297** | **65.9%** | **93.9%** | **3.4x** |
+| dpp | 50% | 297 | 65.9% | 93.9% | 3.5x |
+| kmedoids | 50% | 297 | 65.8% | 93.8% | 3.1x |
+| fps_multi | 50% | 297 | 65.7% | 93.7% | 3.0x |
+| facility | 50% | 297 | 64.9% | 92.6% | 2.9x |
+| random | 50% | 297 | 64.2% | 91.5% | 2.3x |
+| **facility** | **20%** | **119** | **58.6%** | **83.6%** | **7.5x** |
+| kmedoids | 20% | 119 | 58.3% | 83.1% | 6.7x |
+| fps | 20% | 119 | 56.1% | 80.0% | 6.7x |
+| fps_multi | 20% | 119 | 53.8% | 76.8% | 8.0x |
+| random | 20% | 119 | 53.2% | 75.8% | 5.4x |
+| dpp | 20% | 119 | 47.6% | 67.8% | 6.0x |
+| **fps_multi** | **10%** | **60** | **47.7%** | **67.9%** | **18.5x** |
+| kmedoids | 10% | 60 | 46.9% | 66.9% | 14.9x |
+| facility | 10% | 60 | 46.8% | 66.8% | 11.5x |
+| random | 10% | 60 | 44.3% | 63.1% | 10.3x |
+| dpp | 10% | 60 | 42.9% | 61.1% | 13.7x |
+| fps | 10% | 60 | 40.6% | 57.9% | 21.4x |
+
+### Robot Framework (126 tests, 63.5% full coverage)
+
+| Strategy | Selection | Tests | Coverage | Retention | Speedup |
+|---|---|---|---|---|---|
+| *(full suite)* | 100% | 126 | 63.5% | baseline | 1.0x |
+| **fps_multi** | **50%** | **63** | **61.6%** | **97.0%** | **1.5x** |
+| facility | 50% | 63 | 61.6% | 96.9% | 1.5x |
+| fps | 50% | 63 | 61.2% | 96.4% | 1.6x |
+| kmedoids | 50% | 63 | 60.5% | 95.3% | 1.5x |
+| dpp | 50% | 63 | 58.5% | 92.0% | 1.7x |
+| random | 50% | 63 | 57.2% | 90.1% | 2.3x |
+| **facility** | **20%** | **26** | **51.8%** | **81.5%** | **4.4x** |
+| kmedoids | 20% | 26 | 47.5% | 74.8% | 5.5x |
+| random | 20% | 26 | 46.0% | 72.5% | 5.7x |
+| fps | 20% | 26 | 45.3% | 71.4% | 3.1x |
+| fps_multi | 20% | 26 | 45.3% | 71.4% | 2.9x |
+| dpp | 20% | 26 | 43.9% | 69.0% | 4.6x |
+| **kmedoids** | **10%** | **13** | **42.3%** | **66.6%** | **11.3x** |
+| dpp | 10% | 13 | 40.9% | 64.3% | 10.7x |
+| facility | 10% | 13 | 40.2% | 63.3% | 12.9x |
+| random | 10% | 13 | 37.6% | 59.2% | 11.3x |
+| fps_multi | 10% | 13 | 37.5% | 59.1% | 10.7x |
+| fps | 10% | 13 | 33.2% | 52.3% | 8.3x |
+
+### Key Findings
+
+- **50% selection retains 92-97% of coverage** across all strategies — the diversity algorithms select tests maximizing behavioral breadth, with minimal redundancy loss
+- **Facility location excels at 20% selection** — 83.6% retention on pytest, 81.5% on Robot Framework, making it the best choice for aggressive smoke-test suites
+- **kmedoids matches facility at 20%** on pytest (83.1% retention) and leads at 10% on Robot Framework (66.6%)
+- **fps_multi leads at 10% pytest** (67.9% retention) where initial-point sensitivity matters most
+- **DPP performs well at 50%** (93.9% retention, matching FPS) but drops at 20% due to FPS fallback for large k values
+- **All diversity strategies outperform random** at 50% by 2-7 percentage points of retention
+- At 50% robot selection, fps_multi achieves **97% retention** — meaning half the tests contribute almost no incremental coverage
+
 ## Performance
 
 Benchmarked on 384-dimensional normalized vectors:

{robotframework_testselection-0.1.0 → robotframework_testselection-0.3.0}/pyproject.toml

@@ -5,7 +5,7 @@ build-backend = "hatchling.build"
 [project]
 name = "robotframework-testselection"
 dynamic = ["version"]
-description = "Vector-based diverse test case selection for Robot Framework"
+description = "Vector-based diverse test case selection for Robot Framework and pytest"
 readme = "README.md"
 requires-python = ">=3.10"
 license = { text = "Apache-2.0" }
@@ -14,6 +14,7 @@ authors = [
 ]
 keywords = [
     "robotframework",
+    "pytest",
     "testing",
     "test-selection",
     "diversity",
@@ -26,6 +27,7 @@ classifiers = [
     "Framework :: Robot Framework",
     "Framework :: Robot Framework :: Library",
     "Framework :: Robot Framework :: Tool",
+    "Framework :: Pytest",
     "Intended Audience :: Developers",
     "License :: OSI Approved :: Apache Software License",
     "Operating System :: OS Independent",
@@ -76,6 +78,9 @@ all = [
 [project.scripts]
 testcase-select = "TestSelection.cli:main"
 
+[project.entry-points.pytest11]
+diverse-selection = "TestSelection.pytest.plugin"
+
 [tool.hatch.version]
 path = "src/TestSelection/__init__.py"
 
@@ -128,3 +133,9 @@ module = [
     "robot.*",
 ]
 ignore_missing_imports = true
+
+[dependency-groups]
+dev = [
+    "coverage>=7.13.4",
+    "pytest-cov>=7.0.0",
+]

{robotframework_testselection-0.1.0 → robotframework_testselection-0.3.0}/src/TestSelection/__init__.py

@@ -1,3 +1,3 @@
 """Vector-based diverse test case selection for Robot Framework."""
 
-__version__ = "0.1.0"
+__version__ = "0.3.0"