lovranran-mcp-test-runner 0.1.0__tar.gz

lovranran_mcp_test_runner-0.1.0/.github/workflows/ci.yml

@@ -0,0 +1,21 @@
+name: CI
+
+on:
+  push:
+    branches: ["main"]
+  pull_request:
+    branches: ["main"]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+      - uses: astral-sh/setup-uv@v5
+      - run: uv sync --extra dev
+      - run: uv run ruff check .
+      - run: uv run mypy
+      - run: uv run pytest

lovranran_mcp_test_runner-0.1.0/.gitignore

@@ -0,0 +1,5 @@
+.mypy_cache/
+.pytest_cache/
+.ruff_cache/
+.venv/
+__pycache__/

lovranran_mcp_test_runner-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Haichuan Zhou
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

lovranran_mcp_test_runner-0.1.0/PKG-INFO

@@ -0,0 +1,99 @@
+Metadata-Version: 2.4
+Name: lovranran-mcp-test-runner
+Version: 0.1.0
+Summary: MCP server for running and normalizing local test results.
+License-Expression: MIT
+License-File: LICENSE
+Requires-Python: >=3.11
+Requires-Dist: fastmcp>=2.0
+Requires-Dist: pydantic>=2.0
+Provides-Extra: dev
+Requires-Dist: mypy>=1.10; extra == 'dev'
+Requires-Dist: pytest-cov>=5.0; extra == 'dev'
+Requires-Dist: pytest-json-report>=1.5; extra == 'dev'
+Requires-Dist: pytest>=8.0; extra == 'dev'
+Requires-Dist: ruff>=0.5; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# mcp-test-runner
+
+<!-- mcp-name: io.github.LovRanRan/mcp-test-runner -->
+
+MCP server for deterministic local test execution and normalized test result reporting.
+
+`mcp-test-runner` is the verification layer for codebase onboarding agents. It exposes focused MCP tools for running pytest and Jest, parsing test output, and summarizing coverage so downstream agents can mark claims as verified, unverified, or contradicted by real execution.
+
+## Codebase Onboarding Stack
+
+`mcp-test-runner` is the verification layer in a three-server MCP tool stack for Project 6 `wayfinder`, a codebase onboarding agent.
+
+- [`mcp-repo-mapper`](https://github.com/LovRanRan/mcp-repo-mapper) maps repository structure, languages, entry points, framework evidence, and Python dependency edges.
+- [`mcp-ast-explorer`](https://github.com/LovRanRan/mcp-ast-explorer) provides symbol-grounded Python definition, signature, reference, call-chain, and class-hierarchy lookups.
+- [`mcp-test-runner`](https://github.com/LovRanRan/mcp-test-runner) runs local pytest/Jest checks and coverage summaries so agent claims can be verified against execution.
+
+In `wayfinder`, this server turns high-risk code understanding claims into `verified`, `unverified`, or `contradicted` evidence from real test execution.
+
+## Status
+
+This repository is a Python-first v1 MCP test runner. It supports bounded pytest execution, Jest command execution, single-test targeting, pytest coverage summaries, and normalized pytest/Jest JSON parsing.
+
+The server does not use an LLM. Test results come from subprocess execution and structured parser output.
+
+## Tools
+
+| Tool | Purpose |
+| --- | --- |
+| `health()` | Returns `ok` for smoke checks. |
+| `run_pytest(path, test_filter?, timeout_seconds?, cpu_seconds?, memory_mb?)` | Run pytest in a bounded working directory and return raw command output. |
+| `run_jest(path, test_filter?, timeout_seconds?, cpu_seconds?, memory_mb?)` | Run Jest through `npx jest` and return raw command output. |
+| `run_single_test(path, test_id, framework?, timeout_seconds?, cpu_seconds?, memory_mb?)` | Run one pytest node id or one Jest test-name pattern. |
+| `parse_test_output(stdout, framework)` | Normalize test runner JSON output. |
+| `get_coverage_summary(path, framework?, timeout_seconds?, cpu_seconds?, memory_mb?)` | Return pytest-cov JSON coverage totals. |
+
+## Supported Scope
+
+- Python 3.11+ package.
+- FastMCP 2.x server.
+- pytest execution with JSON report output via `pytest-json-report`.
+- Jest command execution via `npx jest --json --outputFile`.
+- pytest and Jest JSON normalization into one `TestRunResult` schema.
+- Single pytest node id execution and Jest test-name targeting.
+- pytest-cov coverage summary from `.coverage.json`.
+- Subprocess timeout plus POSIX `resource.setrlimit` CPU / memory caps.
+
+## Current Limitations
+
+- v1 does not use Docker. It uses cwd validation, subprocess timeouts, and POSIX resource limits.
+- Resource limits require a POSIX platform that supports `resource.setrlimit`.
+- Jest execution expects Node.js plus project-local or `npx`-resolvable Jest.
+- Jest single-test targeting uses `--testNamePattern`; it does not parse Jest file-specific node ids.
+- Coverage summary is pytest-only in v1.
+- Tools return raw command output for execution; call `parse_test_output` to normalize framework JSON.
+
+## Local Development
+
+Install dependencies:
+
+```bash
+uv sync --extra dev
+```
+
+The PyPI distribution name is `lovranran-mcp-test-runner` because `mcp-test-runner` is already taken on PyPI. The installed console script remains `mcp-test-runner`.
+
+Run the MCP server:
+
+```bash
+uv run mcp-test-runner
+```
+
+Run verification:
+
+```bash
+uv run ruff check .
+uv run mypy
+uv run pytest
+```
+
+## License
+
+MIT

lovranran_mcp_test_runner-0.1.0/README.md

@@ -0,0 +1,82 @@
+# mcp-test-runner
+
+<!-- mcp-name: io.github.LovRanRan/mcp-test-runner -->
+
+MCP server for deterministic local test execution and normalized test result reporting.
+
+`mcp-test-runner` is the verification layer for codebase onboarding agents. It exposes focused MCP tools for running pytest and Jest, parsing test output, and summarizing coverage so downstream agents can mark claims as verified, unverified, or contradicted by real execution.
+
+## Codebase Onboarding Stack
+
+`mcp-test-runner` is the verification layer in a three-server MCP tool stack for Project 6 `wayfinder`, a codebase onboarding agent.
+
+- [`mcp-repo-mapper`](https://github.com/LovRanRan/mcp-repo-mapper) maps repository structure, languages, entry points, framework evidence, and Python dependency edges.
+- [`mcp-ast-explorer`](https://github.com/LovRanRan/mcp-ast-explorer) provides symbol-grounded Python definition, signature, reference, call-chain, and class-hierarchy lookups.
+- [`mcp-test-runner`](https://github.com/LovRanRan/mcp-test-runner) runs local pytest/Jest checks and coverage summaries so agent claims can be verified against execution.
+
+In `wayfinder`, this server turns high-risk code understanding claims into `verified`, `unverified`, or `contradicted` evidence from real test execution.
+
+## Status
+
+This repository is a Python-first v1 MCP test runner. It supports bounded pytest execution, Jest command execution, single-test targeting, pytest coverage summaries, and normalized pytest/Jest JSON parsing.
+
+The server does not use an LLM. Test results come from subprocess execution and structured parser output.
+
+## Tools
+
+| Tool | Purpose |
+| --- | --- |
+| `health()` | Returns `ok` for smoke checks. |
+| `run_pytest(path, test_filter?, timeout_seconds?, cpu_seconds?, memory_mb?)` | Run pytest in a bounded working directory and return raw command output. |
+| `run_jest(path, test_filter?, timeout_seconds?, cpu_seconds?, memory_mb?)` | Run Jest through `npx jest` and return raw command output. |
+| `run_single_test(path, test_id, framework?, timeout_seconds?, cpu_seconds?, memory_mb?)` | Run one pytest node id or one Jest test-name pattern. |
+| `parse_test_output(stdout, framework)` | Normalize test runner JSON output. |
+| `get_coverage_summary(path, framework?, timeout_seconds?, cpu_seconds?, memory_mb?)` | Return pytest-cov JSON coverage totals. |
+
+## Supported Scope
+
+- Python 3.11+ package.
+- FastMCP 2.x server.
+- pytest execution with JSON report output via `pytest-json-report`.
+- Jest command execution via `npx jest --json --outputFile`.
+- pytest and Jest JSON normalization into one `TestRunResult` schema.
+- Single pytest node id execution and Jest test-name targeting.
+- pytest-cov coverage summary from `.coverage.json`.
+- Subprocess timeout plus POSIX `resource.setrlimit` CPU / memory caps.
+
+## Current Limitations
+
+- v1 does not use Docker. It uses cwd validation, subprocess timeouts, and POSIX resource limits.
+- Resource limits require a POSIX platform that supports `resource.setrlimit`.
+- Jest execution expects Node.js plus project-local or `npx`-resolvable Jest.
+- Jest single-test targeting uses `--testNamePattern`; it does not parse Jest file-specific node ids.
+- Coverage summary is pytest-only in v1.
+- Tools return raw command output for execution; call `parse_test_output` to normalize framework JSON.
+
+## Local Development
+
+Install dependencies:
+
+```bash
+uv sync --extra dev
+```
+
+The PyPI distribution name is `lovranran-mcp-test-runner` because `mcp-test-runner` is already taken on PyPI. The installed console script remains `mcp-test-runner`.
+
+Run the MCP server:
+
+```bash
+uv run mcp-test-runner
+```
+
+Run verification:
+
+```bash
+uv run ruff check .
+uv run mypy
+uv run pytest
+```
+
+## License
+
+MIT

lovranran_mcp_test_runner-0.1.0/pyproject.toml

@@ -0,0 +1,46 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/mcp_test_runner"]
+
+[project]
+name = "lovranran-mcp-test-runner"
+version = "0.1.0"
+description = "MCP server for running and normalizing local test results."
+readme = "README.md"
+license = "MIT"
+requires-python = ">=3.11"
+dependencies = [
+    "fastmcp>=2.0",
+    "pydantic>=2.0",
+]
+
+[project.optional-dependencies]
+dev = [
+    "mypy>=1.10",
+    "pytest>=8.0",
+    "pytest-cov>=5.0",
+    "pytest-json-report>=1.5",
+    "ruff>=0.5",
+]
+
+[project.scripts]
+mcp-test-runner = "mcp_test_runner.server:main"
+
+[tool.ruff]
+line-length = 100
+
+[tool.ruff.lint]
+select = ["E", "F", "I", "UP", "B", "SIM"]
+
+[tool.mypy]
+python_version = "3.11"
+strict = true
+mypy_path = "src"
+packages = ["mcp_test_runner"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+pythonpath = ["src"]

lovranran_mcp_test_runner-0.1.0/pyrightconfig.json

@@ -0,0 +1,16 @@
+{
+  "venvPath": ".",
+  "venv": ".venv",
+  "pythonVersion": "3.11",
+  "typeCheckingMode": "strict",
+  "executionEnvironments": [
+    {
+      "root": "src",
+      "extraPaths": ["src"]
+    },
+    {
+      "root": "tests",
+      "extraPaths": ["src"]
+    }
+  ]
+}

lovranran_mcp_test_runner-0.1.0/server.json

@@ -0,0 +1,21 @@
+{
+  "$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json",
+  "name": "io.github.LovRanRan/mcp-test-runner",
+  "title": "MCP Test Runner",
+  "description": "Local test execution and coverage summaries for onboarding agents.",
+  "repository": {
+    "url": "https://github.com/LovRanRan/mcp-test-runner",
+    "source": "github"
+  },
+  "version": "0.1.0",
+  "packages": [
+    {
+      "registryType": "pypi",
+      "identifier": "lovranran-mcp-test-runner",
+      "version": "0.1.0",
+      "transport": {
+        "type": "stdio"
+      }
+    }
+  ]
+}

lovranran_mcp_test_runner-0.1.0/src/mcp_test_runner/__init__.py

@@ -0,0 +1,5 @@
+"""MCP test runner package."""
+
+__all__ = ["__version__"]
+
+__version__ = "0.1.0"

lovranran_mcp_test_runner-0.1.0/src/mcp_test_runner/coverage.py

@@ -0,0 +1,55 @@
+import json
+from pathlib import Path
+
+from mcp_test_runner.parsers import _as_mapping, _int_count
+from mcp_test_runner.runner import ResourceLimits, run_command
+from mcp_test_runner.schemas import CoverageSummary
+
+
+def _float_count(value: object) -> float:
+    if isinstance(value, int | float):
+        return float(value)
+    if isinstance(value, str):
+        return float(value)
+    return 0.0
+
+
+def build_coverage_command(framework: str = "pytest") -> list[str]:
+    if framework != "pytest":
+        raise ValueError(f"Unsupported framework: {framework}")
+
+    return [
+        "pytest",
+        "--cov=.",
+        "--cov-report=json:.coverage.json",
+    ]
+
+
+def get_coverage_summary(
+    path: str | Path,
+    framework: str = "pytest",
+    timeout_seconds: float = 10.0,
+    resource_limits: ResourceLimits | None = None,
+) -> CoverageSummary:
+    root = Path(path).resolve()
+    result = run_command(
+        build_coverage_command(framework),
+        cwd=root,
+        timeout_seconds=timeout_seconds,
+        resource_limits=resource_limits,
+    )
+    if result.timed_out:
+        raise RuntimeError("Coverage command timed out")
+    if result.exit_code != 0:
+        raise RuntimeError(result.stderr or result.stdout or "Coverage command failed")
+
+    report_path = root / ".coverage.json"
+    payload = _as_mapping(json.loads(report_path.read_text(encoding="utf-8")))
+    totals = _as_mapping(payload.get("totals"))
+    return CoverageSummary(
+        framework=framework,
+        covered_lines=_int_count(totals.get("covered_lines")),
+        total_lines=_int_count(totals.get("num_statements")),
+        percent_covered=_float_count(totals.get("percent_covered")),
+        report_path=str(report_path),
+    )

lovranran_mcp_test_runner-0.1.0/src/mcp_test_runner/jest_runner.py

@@ -0,0 +1,31 @@
+from pathlib import Path
+
+from mcp_test_runner.runner import CommandResult, ResourceLimits, run_command
+
+
+def build_jest_command(test_filter: str | None = None) -> list[str]:
+    command = [
+        "npx",
+        "jest",
+        "--json",
+        "--outputFile=.jest-report.json",
+    ]
+
+    if test_filter:
+        command.extend(["--testNamePattern", test_filter])
+
+    return command
+
+
+def run_jest(
+    path: str | Path,
+    test_filter: str | None = None,
+    timeout_seconds: float = 10.0,
+    resource_limits: ResourceLimits | None = None,
+) -> CommandResult:
+    return run_command(
+        build_jest_command(test_filter),
+        cwd=path,
+        timeout_seconds=timeout_seconds,
+        resource_limits=resource_limits,
+    )

lovranran_mcp_test_runner-0.1.0/src/mcp_test_runner/parsers.py

@@ -0,0 +1,109 @@
+import json
+from collections.abc import Mapping
+from typing import cast
+
+from mcp_test_runner.schemas import TestFailure, TestRunResult
+
+
+def _as_mapping(value: object) -> dict[str, object]:
+    if isinstance(value, Mapping):
+        mapping = cast(Mapping[object, object], value)
+        return {str(key): item for key, item in mapping.items()}
+    return {}
+
+
+def _as_list(value: object) -> list[object]:
+    if isinstance(value, list):
+        return cast(list[object], value)
+    return []
+
+
+def _optional_str(value: object) -> str | None:
+    if value is None:
+        return None
+    return str(value)
+
+
+def _optional_int(value: object) -> int | None:
+    if isinstance(value, int):
+        return value
+    return None
+
+
+def _int_count(value: object) -> int:
+    if isinstance(value, int):
+        return value
+    if isinstance(value, str):
+        return int(value)
+    return 0
+
+
+def _parse_pytest_output(stdout: str) -> TestRunResult:
+    payload = _as_mapping(json.loads(stdout))
+    summary = _as_mapping(payload.get("summary"))
+    tests = _as_list(payload.get("tests"))
+
+    failures: list[TestFailure] = []
+    for test in tests:
+        test_payload = _as_mapping(test)
+        if test_payload.get("outcome") != "failed":
+            continue
+        call = _as_mapping(test_payload.get("call"))
+        crash = _as_mapping(call.get("crash"))
+
+        failures.append(
+            TestFailure(
+                test_id=str(test_payload.get("nodeid", "")),
+                message=str(crash.get("message", "")),
+                file=_optional_str(test_payload.get("file")),
+                line=_optional_int(test_payload.get("line")),
+                traceback=_optional_str(call.get("longrepr")),
+            )
+        )
+
+    return TestRunResult(
+        passed=_int_count(summary.get("passed")),
+        failed=_int_count(summary.get("failed")),
+        skipped=_int_count(summary.get("skipped")),
+        failures=failures,
+    )
+
+
+def _parse_jest_output(stdout: str) -> TestRunResult:
+    payload = _as_mapping(json.loads(stdout))
+    test_results = _as_list(payload.get("testResults"))
+
+    failures: list[TestFailure] = []
+    for test_file in test_results:
+        test_file_payload = _as_mapping(test_file)
+        file_name = _optional_str(test_file_payload.get("name"))
+        assertions = _as_list(test_file_payload.get("assertionResults"))
+        for assertion in assertions:
+            assertion_payload = _as_mapping(assertion)
+            if assertion_payload.get("status") != "failed":
+                continue
+            messages = _as_list(assertion_payload.get("failureMessages"))
+            message = "\n".join(str(message_part) for message_part in messages)
+            failures.append(
+                TestFailure(
+                    test_id=str(assertion_payload.get("fullName", "")),
+                    message=message,
+                    file=file_name,
+                    traceback=message or None,
+                )
+            )
+
+    return TestRunResult(
+        passed=_int_count(payload.get("numPassedTests")),
+        failed=_int_count(payload.get("numFailedTests")),
+        skipped=_int_count(payload.get("numPendingTests")),
+        failures=failures,
+    )
+
+
+def parse_test_output(stdout: str, framework: str) -> TestRunResult:
+    if framework == "pytest":
+        return _parse_pytest_output(stdout)
+    if framework == "jest":
+        return _parse_jest_output(stdout)
+    raise ValueError(f"Unsupported framework: {framework}")

lovranran_mcp_test_runner-0.1.0/src/mcp_test_runner/pytest_runner.py

@@ -0,0 +1,36 @@
+from pathlib import Path
+
+from mcp_test_runner.runner import CommandResult, ResourceLimits, run_command
+
+
+def build_pytest_command(
+    test_filter: str | None = None,
+    test_id: str | None = None,
+) -> list[str]:
+    command = [
+        "pytest",
+        "--json-report",
+        "--json-report-file=.pytest-report.json",
+    ]
+
+    if test_id:
+        command.append(test_id)
+
+    if test_filter:
+        command.extend(["-k", test_filter])
+
+    return command
+
+
+def run_pytest(
+    path: str | Path,
+    test_filter: str | None = None,
+    timeout_seconds: float = 10.0,
+    resource_limits: ResourceLimits | None = None,
+) -> CommandResult:
+    return run_command(
+        build_pytest_command(test_filter),
+        cwd=path,
+        timeout_seconds=timeout_seconds,
+        resource_limits=resource_limits,
+    )

lovranran_mcp_test_runner-0.1.0/src/mcp_test_runner/runner.py

@@ -0,0 +1,92 @@
+import subprocess
+from collections.abc import Callable
+from contextlib import suppress
+from pathlib import Path
+
+from pydantic import BaseModel
+
+
+class CommandResult(BaseModel):
+    exit_code: int
+    stdout: str
+    stderr: str
+    timed_out: bool = False
+
+
+class ResourceLimits(BaseModel):
+    cpu_seconds: int | None = None
+    memory_mb: int | None = None
+
+
+def _output_to_text(output: str | bytes | None) -> str:
+    if output is None:
+        return ""
+    if isinstance(output, bytes):
+        return output.decode("utf-8", errors="replace")
+    return output
+
+
+def _apply_resource_limits(limits: ResourceLimits) -> None:
+    import resource
+
+    if limits.cpu_seconds is not None:
+        resource.setrlimit(
+            resource.RLIMIT_CPU,
+            (limits.cpu_seconds, limits.cpu_seconds),
+        )
+
+    if limits.memory_mb is not None:
+        memory_bytes = limits.memory_mb * 1024 * 1024
+        # RLIMIT_AS is platform-dependent; keep CPU limits and timeouts active.
+        with suppress(OSError, ValueError):
+            resource.setrlimit(
+                resource.RLIMIT_AS,
+                (memory_bytes, memory_bytes),
+            )
+
+
+def _build_preexec_fn(limits: ResourceLimits | None) -> Callable[[], None] | None:
+    if limits is None:
+        return None
+
+    def apply_limits() -> None:
+        _apply_resource_limits(limits)
+
+    return apply_limits
+
+
+def run_command(
+    command: list[str],
+    cwd: str | Path,
+    timeout_seconds: float = 10.0,
+    resource_limits: ResourceLimits | None = None,
+) -> CommandResult:
+    resolved_cwd = Path(cwd).resolve()
+
+    if not resolved_cwd.exists() or not resolved_cwd.is_dir():
+        raise ValueError(f"cwd must be an existing directory: {resolved_cwd}")
+
+    try:
+        completed = subprocess.run(
+            command,
+            cwd=resolved_cwd,
+            timeout=timeout_seconds,
+            capture_output=True,
+            text=True,
+            check=False,
+            preexec_fn=_build_preexec_fn(resource_limits),
+        )
+    except subprocess.TimeoutExpired as exc:
+        return CommandResult(
+            exit_code=-1,
+            stdout=_output_to_text(exc.stdout),
+            stderr=_output_to_text(exc.stderr),
+            timed_out=True,
+        )
+
+    return CommandResult(
+        exit_code=completed.returncode,
+        stdout=completed.stdout,
+        stderr=completed.stderr,
+        timed_out=False,
+    )

lovranran_mcp_test_runner-0.1.0/src/mcp_test_runner/schemas.py

@@ -0,0 +1,28 @@
+from pydantic import BaseModel, Field
+
+
+class TestFailure(BaseModel):
+    test_id: str
+    message: str
+    file: str | None = None
+    line: int | None = None
+    traceback: str | None = None
+
+
+def empty_failures() -> list[TestFailure]:
+    return []
+
+
+class TestRunResult(BaseModel):
+    passed: int = 0
+    failed: int = 0
+    skipped: int = 0
+    failures: list[TestFailure] = Field(default_factory=empty_failures)
+
+
+class CoverageSummary(BaseModel):
+    framework: str
+    covered_lines: int
+    total_lines: int
+    percent_covered: float
+    report_path: str | None = None