specterqa 0.3.0__tar.gz

specterqa-0.3.0/.github/ISSUE_TEMPLATE/bug_report.yml

@@ -0,0 +1,83 @@
+name: Bug Report
+description: Report a bug in SpecterQA
+title: "[Bug]: "
+labels: ["bug", "triage"]
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Thanks for reporting a bug! Please fill out the sections below so we can reproduce and fix the issue.
+
+  - type: textarea
+    id: description
+    attributes:
+      label: Description
+      description: A clear and concise description of what the bug is.
+    validations:
+      required: true
+
+  - type: textarea
+    id: steps
+    attributes:
+      label: Steps to reproduce
+      description: How can we reproduce this behavior?
+      placeholder: |
+        1. Create a persona YAML with ...
+        2. Run `specterqa run ...`
+        3. See error ...
+    validations:
+      required: true
+
+  - type: textarea
+    id: expected
+    attributes:
+      label: Expected behavior
+      description: What did you expect to happen?
+    validations:
+      required: true
+
+  - type: textarea
+    id: actual
+    attributes:
+      label: Actual behavior
+      description: What actually happened? Include error messages or screenshots if applicable.
+    validations:
+      required: true
+
+  - type: input
+    id: specterqa-version
+    attributes:
+      label: SpecterQA version
+      description: "Output of `specterqa --version` or `pip show specterqa`"
+      placeholder: "0.1.0"
+    validations:
+      required: true
+
+  - type: input
+    id: python-version
+    attributes:
+      label: Python version
+      description: "Output of `python --version`"
+      placeholder: "3.12.0"
+    validations:
+      required: true
+
+  - type: dropdown
+    id: os
+    attributes:
+      label: Operating system
+      options:
+        - Linux
+        - macOS
+        - Windows
+        - Other
+    validations:
+      required: true
+
+  - type: textarea
+    id: additional
+    attributes:
+      label: Additional context
+      description: Any other context, logs, or screenshots about the problem.
+    validations:
+      required: false

specterqa-0.3.0/.github/ISSUE_TEMPLATE/feature_request.yml

@@ -0,0 +1,51 @@
+name: Feature Request
+description: Suggest a new feature or improvement for SpecterQA
+title: "[Feature]: "
+labels: ["enhancement"]
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Thanks for suggesting an improvement! Please describe your idea below.
+
+  - type: textarea
+    id: description
+    attributes:
+      label: Description
+      description: A clear and concise description of the feature you'd like.
+    validations:
+      required: true
+
+  - type: textarea
+    id: use-case
+    attributes:
+      label: Use case
+      description: What problem does this solve? Who benefits from it?
+      placeholder: |
+        As a QA engineer, I want to ... so that ...
+    validations:
+      required: true
+
+  - type: textarea
+    id: proposed-solution
+    attributes:
+      label: Proposed solution
+      description: How do you envision this working? Include API ideas, CLI flags, YAML config examples, etc.
+    validations:
+      required: false
+
+  - type: textarea
+    id: alternatives
+    attributes:
+      label: Alternatives considered
+      description: Have you considered other approaches? What are the trade-offs?
+    validations:
+      required: false
+
+  - type: textarea
+    id: additional
+    attributes:
+      label: Additional context
+      description: Any other context, mockups, or references.
+    validations:
+      required: false

specterqa-0.3.0/.github/pull_request_template.md

@@ -0,0 +1,25 @@
+## Summary
+
+<!-- What does this PR do? Why is it needed? Link related issues with "Closes #123". -->
+
+## Changes
+
+<!-- Bullet list of the key changes in this PR. -->
+
+-
+
+## Test plan
+
+<!-- How did you verify this works? Include commands, screenshots, or test output. -->
+
+- [ ] Tests pass locally (`pytest tests/ -v`)
+- [ ] Linting passes (`ruff check src/`)
+- [ ] Formatting passes (`ruff format --check src/`)
+
+## Checklist
+
+- [ ] I have read the [contributing guide](../CONTRIBUTING.md)
+- [ ] My code follows the project's style (ruff-enforced)
+- [ ] I have added tests for new functionality
+- [ ] I have updated documentation if needed
+- [ ] All new and existing tests pass

specterqa-0.3.0/.github/workflows/ci.yml

@@ -0,0 +1,51 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  lint:
+    name: Lint
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Install dependencies
+        run: pip install -e ".[dev,mcp]"
+
+      - name: Ruff check
+        run: ruff check src/
+
+      - name: Ruff format check
+        run: ruff format --check src/
+
+  test:
+    name: Test (Python ${{ matrix.python-version }})
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.10", "3.11", "3.12"]
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Install dependencies
+        run: pip install -e ".[dev,mcp]"
+
+      - name: Run tests
+        run: pytest tests/ -v --tb=short

specterqa-0.3.0/.github/workflows/publish.yml

@@ -0,0 +1,31 @@
+name: Publish to PyPI
+
+on:
+  release:
+    types: [published]
+
+permissions:
+  id-token: write
+
+jobs:
+  publish:
+    name: Build and publish to PyPI
+    runs-on: ubuntu-latest
+    environment:
+      name: pypi
+      url: https://pypi.org/p/specterqa
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Install build tools
+        run: pip install build
+
+      - name: Build package
+        run: python -m build
+
+      - name: Publish to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1

specterqa-0.3.0/.gitignore

@@ -0,0 +1,57 @@
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+*.egg-info/
+*.egg
+
+# Virtual environments
+.venv/
+env/
+venv/
+ENV/
+
+# IDE
+.vscode/
+.idea/
+*.swp
+*.swo
+*~
+
+# Testing
+.pytest_cache/
+.coverage
+htmlcov/
+.mypy_cache/
+.ruff_cache/
+
+# Environment variables
+.env
+.env.local
+.env.*.local
+
+# GhostQA project config and evidence
+.ghostqa/config.yaml
+.ghostqa/evidence/
+evidence/
+**/evidence/**/*.png
+
+# OS
+.DS_Store
+Thumbs.db

specterqa-0.3.0/.mcpregistry_github_token

@@ -0,0 +1 @@
+ghu_W3yptWq9TzcU9hjKG9YQSuy4XA5OEt3tuVe9

specterqa-0.3.0/.mcpregistry_registry_token

@@ -0,0 +1 @@
+{"token":"eyJhbGciOiJFZERTQSIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJtY3AtcmVnaXN0cnkiLCJleHAiOjE3NzE4MzAwMDgsIm5iZiI6MTc3MTgyOTcwOCwiaWF0IjoxNzcxODI5NzA4LCJhdXRoX21ldGhvZCI6ImdpdGh1Yi1hdCIsImF1dGhfbWV0aG9kX3N1YiI6IlN5bmNUZWtMTEMiLCJwZXJtaXNzaW9ucyI6W3siYWN0aW9uIjoicHVibGlzaCIsInJlc291cmNlIjoiaW8uZ2l0aHViLlN5bmNUZWtMTEMvKiJ9XX0.t0WySIpPwOm0CcdJrNreFdLjZAV1y_aQ2VIA8Ujc_c2Hv7vYxUeI4DBRO-HQnQNi3kYBMYbC8L5ay_f4BXxgBQ","expires_at":1771830008}

specterqa-0.3.0/.well-known/agent.json

@@ -0,0 +1,106 @@
+{
+  "name": "SpecterQA",
+  "description": "AI behavioral testing for web and native applications. AI personas navigate your app in real browser sessions and report UX issues, broken flows, and errors — no test scripts required.",
+  "version": "0.3.0",
+  "url": "https://github.com/SyncTek-LLC/specterqa",
+  "homepage": "https://github.com/SyncTek-LLC/specterqa",
+  "license": "MIT",
+  "capabilities": [
+    "behavioral_testing",
+    "ux_analysis",
+    "vision_testing",
+    "cost_tracking",
+    "mcp"
+  ],
+  "invocation": {
+    "cli": {
+      "command": "specterqa run -p <product>",
+      "install": "pip install specterqa",
+      "json_output_flag": "--output json",
+      "exit_codes": {
+        "0": "all_passed",
+        "1": "test_failure",
+        "2": "config_error",
+        "3": "infra_error"
+      }
+    },
+    "python": {
+      "import": "from specterqa.engine.orchestrator import SpecterQAOrchestrator",
+      "package": "specterqa",
+      "entry_class": "specterqa.engine.orchestrator.SpecterQAOrchestrator"
+    },
+    "mcp": {
+      "command": "specterqa-mcp",
+      "transport": "stdio",
+      "config_example": {
+        "specterqa": {
+          "command": "specterqa-mcp",
+          "args": []
+        }
+      }
+    }
+  },
+  "cost_model": {
+    "type": "per_run",
+    "basis": "anthropic_api_usage",
+    "typical_range_usd": [0.30, 1.50],
+    "budget_enforcement": true,
+    "default_budget_usd": 5.00,
+    "notes": "specterqa_run MCP calls are synchronous and may take 45-300 seconds. Budget cap enforced per run."
+  },
+  "tools": [
+    {
+      "name": "specterqa_run",
+      "description": "Run behavioral tests on a product. AI personas navigate the app and report UX issues. Synchronous — may take 45-300s. Incurs Anthropic API costs (default budget: $5.00).",
+      "inputs": {
+        "product": "string (required) — product name matching .specterqa/products/<name>.yaml",
+        "journey": "string (optional) — journey ID; runs all journeys if omitted",
+        "level": "string (optional) — smoke | standard | thorough",
+        "budget": "float (optional) — cost cap in USD, default 5.00",
+        "directory": "string (optional) — project root directory, defaults to CWD"
+      },
+      "outputs": "JSON with passed, run_id, step_reports, findings, cost_usd, cost_summary"
+    },
+    {
+      "name": "specterqa_list_products",
+      "description": "List configured test products and their available journeys.",
+      "inputs": {
+        "directory": "string (optional) — project root directory, defaults to CWD"
+      },
+      "outputs": "Array of product configs with product name, display name, and journey IDs"
+    },
+    {
+      "name": "specterqa_get_results",
+      "description": "Retrieve full structured results from a previous run by run ID.",
+      "inputs": {
+        "run_id": "string (required) — run ID in format GQA-RUN-YYYYMMDD-HHMMSS-xxxx",
+        "directory": "string (optional) — project root directory, defaults to CWD"
+      },
+      "outputs": "Full run result JSON including step reports, findings, screenshots paths, and cost breakdown"
+    },
+    {
+      "name": "specterqa_init",
+      "description": "Initialize a new SpecterQA project directory with sample product, persona, and journey configs.",
+      "inputs": {
+        "directory": "string (optional) — target directory for initialization, defaults to CWD"
+      },
+      "outputs": "List of created file paths"
+    }
+  ],
+  "requirements": {
+    "python": ">=3.10",
+    "anthropic_api_key": true,
+    "playwright": ">=1.48.0",
+    "notes": "Run 'specterqa install' after pip install to download Playwright browsers"
+  },
+  "output_schema": {
+    "run_result": {
+      "passed": "boolean — top-level pass/fail",
+      "run_id": "string — unique run identifier",
+      "step_reports": "array — per-step pass/fail, duration, error",
+      "findings": "array — UX issues found, each with severity, category, description, step_id",
+      "cost_usd": "float — total API cost for this run",
+      "cost_summary": "object — detailed cost breakdown with budget status"
+    }
+  }
+}

specterqa-0.3.0/CHANGELOG.md

@@ -0,0 +1,76 @@
+# Changelog
+
+All notable changes to SpecterQA are documented here.
+
+Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
+Versions follow [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+---
+
+## [0.3.0] — 2026-02-23
+
+### Changed
+
+- **Rebrand: GhostQA -> SpecterQA.** Package renamed from `ghostqa` to `specterqa`. All imports, CLI commands, MCP tool names, environment variables, and project directory paths updated accordingly. Key changes:
+  - CLI: `ghostqa` -> `specterqa`, `ghostqa-mcp` -> `specterqa-mcp`
+  - Python imports: `from ghostqa` -> `from specterqa`
+  - Classes: `GhostQAConfig` -> `SpecterQAConfig`, `GhostQAOrchestrator` -> `SpecterQAOrchestrator`
+  - MCP tools: `ghostqa_run` -> `specterqa_run`, `ghostqa_list_products` -> `specterqa_list_products`, etc.
+  - Env vars: `GHOSTQA_ALLOWED_DIRS` -> `SPECTERQA_ALLOWED_DIRS`, `GHOSTQA_PROJECT_DIR` -> `SPECTERQA_PROJECT_DIR`, etc.
+  - Project directory: `.ghostqa/` -> `.specterqa/`
+  - Repository URL: `github.com/SyncTek-LLC/ghostqa` -> `github.com/SyncTek-LLC/specterqa`
+
+---
+
+## [0.2.2] — 2026-02-23
+
+### Changed
+
+- Added MCP Registry verification metadata to README (`mcp-name` comment)
+- Added `server.json` for Official MCP Registry submission
+
+---
+
+## [0.2.1] — 2026-02-23
+
+### Security
+
+- **[CRITICAL] Command injection fix (GHSA-SPECTERQA-001):** Removed the `check_command` field from the product YAML schema. The field allowed arbitrary shell commands to be embedded in a YAML service definition and executed by the SpecterQA process. Precondition service checks are now limited to TCP connectivity and HTTP health endpoint checks, which are safe. No workaround exists for v0.2.0 and earlier other than removing `check_command` from all product YAMLs. See [SECURITY_ADVISORY.md](SECURITY_ADVISORY.md) for full details.
+- **MCP directory traversal fix (SPECTERQA_ALLOWED_DIRS):** The MCP server's `specterqa_run` tool now enforces an allowlist for the `directory` parameter when the `SPECTERQA_ALLOWED_DIRS` environment variable is set. Any path not under an allowed prefix is rejected with a structured error response. When the variable is unset, all paths remain accessible (permissive default for local development). Users running the MCP server in shared or multi-user environments should set this variable. See the [README Security section](README.md#security) and [docs/for-agents.md](docs/for-agents.md) for configuration guidance.
+- **Credential scrubbing in run artifacts:** Run result JSON files and log output now automatically scrub known credential patterns (API keys, bearer tokens, passwords, connection strings) before writing to disk. Evidence directories are safer to share and commit as CI artifacts.
+- **API key template removed from sample config:** The `specterqa init` sample configuration previously included a placeholder `anthropic_api_key: sk-ant-...` in `.specterqa/config.yaml`. This has been replaced with an environment variable reference (`${ANTHROPIC_API_KEY}`) to prevent accidental key commits.
+
+### Added
+
+- **`--plain` mode for CI and screen readers:** `specterqa run --plain` disables Rich formatting, colors, and progress spinners and emits plain text to stdout. Useful in CI environments where Rich markup becomes noise, and for accessibility tools that consume terminal output.
+- **`--fail-on-severity` threshold flag:** `specterqa run --fail-on-severity high` causes the run to exit with a non-zero status if any finding at or above the specified severity level is recorded, even if all steps technically passed. Accepted values: `low`, `medium`, `high`, `critical`, `block`. Default behavior (exit 0 on step pass) is unchanged.
+- **`specterqa validate` command:** Validates product, persona, and journey YAML files against the SpecterQA schema without running a test. Reports schema errors, missing required fields, and deprecated fields (including `check_command`, which now produces a schema error). Useful in CI as a pre-flight check.
+- **`.well-known/agent.json` — A2A Agent Card:** SpecterQA now ships an Agent Card at `.well-known/agent.json` following the [Agent-to-Agent (A2A) protocol](https://google.github.io/A2A/). This allows A2A-compatible agent hosts to discover SpecterQA's capabilities, available tools, input/output schemas, and cost model without prior out-of-band configuration.
+- **108 new MCP server tests (248 total):** The MCP server test suite has been expanded from 140 to 248 tests, covering the `SPECTERQA_ALLOWED_DIRS` allowlist enforcement, credential scrubbing behavior, `specterqa validate` tool, the `--plain` and `--fail-on-severity` CLI flags, and edge cases in the `specterqa_run` tool response schema.
+
+### Changed
+
+- The `services.*.check_command` field in product YAMLs is now a schema error. Existing configs containing this field will be rejected by `specterqa validate` and produce a startup error on `specterqa run`. Remove the field and use `health_endpoint` or TCP port checks instead.
+- The `specterqa init` sample product YAML no longer includes a `check_command` example.
+
+---
+
+## [0.2.0] — 2026-02-22
+
+Initial public release.
+
+- Persona-based behavioral testing via YAML-configured personas and journeys
+- Claude vision model integration (Haiku for simple actions, Sonnet for complex reasoning)
+- Playwright-based browser automation with headless support
+- macOS native app testing via Accessibility API (pyobjc optional extra)
+- iOS Simulator testing via simctl (pyobjc optional extra)
+- Budget enforcement with per-run, per-day, and per-month caps
+- JUnit XML output for CI integration
+- JSON structured output (`--output json`) for programmatic consumption
+- Evidence collection: screenshots, findings, cost breakdown, run-result.json per run
+- Stuck detection with model escalation (Haiku → Sonnet → abort)
+- Template variables in journey steps (`{{persona.credentials.email}}`)
+- MCP server (`specterqa-mcp`) exposing `specterqa_run`, `specterqa_list_products`, `specterqa_get_results`, `specterqa_init`
+- Tiered model routing with optional local Ollama fallback for zero-cost simple actions
+- Federated protocol: swap in custom `AIDecider` or `ActionExecutor` implementations
+- 140 unit tests across config, cost tracking, report generation, MCP server, and CLI

specterqa-0.3.0/CONTRIBUTING.md

@@ -0,0 +1,94 @@
+# Contributing to SpecterQA
+
+Thanks for your interest in contributing to SpecterQA! This guide will help you get started.
+
+## Development Setup
+
+1. **Clone the repository**
+
+   ```bash
+   git clone https://github.com/SyncTek-LLC/specterqa.git
+   cd specterqa
+   ```
+
+2. **Create a virtual environment**
+
+   ```bash
+   python -m venv .venv
+   source .venv/bin/activate  # Linux/macOS
+   # .venv\Scripts\activate   # Windows
+   ```
+
+3. **Install in development mode**
+
+   ```bash
+   pip install -e ".[dev]"
+   ```
+
+4. **Install Playwright browsers** (needed for integration tests)
+
+   ```bash
+   playwright install chromium
+   ```
+
+## Running Tests
+
+```bash
+# Run all unit tests
+pytest tests/ -v
+
+# Run with coverage
+pytest tests/ -v --cov=src/specterqa --cov-report=term-missing
+
+# Skip integration tests (which require a running app and API key)
+pytest tests/ -v -m "not integration"
+```
+
+## Code Quality
+
+We use [Ruff](https://docs.astral.sh/ruff/) for linting and formatting.
+
+```bash
+# Check for lint issues
+ruff check src/
+
+# Auto-fix lint issues
+ruff check src/ --fix
+
+# Check formatting
+ruff format --check src/
+
+# Auto-format
+ruff format src/
+```
+
+Type checking with mypy:
+
+```bash
+mypy src/specterqa
+```
+
+## Submitting Changes
+
+1. **Fork the repository** and create a branch from `main`.
+2. **Make your changes.** Add tests for new functionality.
+3. **Ensure all checks pass:**
+   ```bash
+   ruff check src/
+   ruff format --check src/
+   pytest tests/ -v
+   ```
+4. **Open a pull request** against `main`. Fill out the PR template.
+5. A maintainer will review your PR. Address any feedback, then it will be merged.
+
+## Reporting Bugs
+
+Use the [bug report template](https://github.com/SyncTek-LLC/specterqa/issues/new?template=bug_report.yml) on GitHub Issues.
+
+## Requesting Features
+
+Use the [feature request template](https://github.com/SyncTek-LLC/specterqa/issues/new?template=feature_request.yml) on GitHub Issues.
+
+## License
+
+By contributing to SpecterQA, you agree that your contributions will be licensed under the [MIT License](LICENSE). All contributions are subject to this project's license terms.

specterqa-0.3.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 SyncTek LLC
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.