agentic-qa-maestro 0.1.0__tar.gz

agentic_qa_maestro-0.1.0/.github/workflows/ci.yml

@@ -0,0 +1,32 @@
+name: CI
+
+on:
+  push:
+    branches: [main, develop]
+  pull_request:
+    branches: [main, develop]
+
+jobs:
+  lint:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - run: pip install ruff
+      - run: ruff check .
+      - run: ruff format --check .
+
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.10", "3.11", "3.12"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+      - run: pip install -e ".[dev]"
+      - run: pytest --cov=agentic_qa_maestro --cov-report=term-missing

agentic_qa_maestro-0.1.0/.github/workflows/release.yml

@@ -0,0 +1,47 @@
+name: Release to PyPI
+
+on:
+  release:
+    types: [published]
+
+permissions:
+  id-token: write
+  contents: read
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Install build tools
+        run: pip install build
+
+      - name: Build package
+        run: python -m build
+
+      - name: Upload artifacts
+        uses: actions/upload-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+  publish-pypi:
+    needs: build
+    runs-on: ubuntu-latest
+    environment: pypi
+    permissions:
+      id-token: write
+    steps:
+      - name: Download artifacts
+        uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+      - name: Publish to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1

agentic_qa_maestro-0.1.0/.gitignore

@@ -0,0 +1,38 @@
+# Python
+__pycache__/
+*.py[cod]
+*$py.class
+*.egg-info/
+
+# App settings (may contain config)
+.settings.json
+dist/
+build/
+*.egg
+
+# Environment
+.env
+.venv/
+venv/
+env/
+
+# IDE
+.idea/
+.vscode/
+*.swp
+*.swo
+
+# Testing
+.pytest_cache/
+htmlcov/
+.coverage
+.coverage.*
+
+# OS
+.DS_Store
+Thumbs.db
+
+# Test evidence screenshots (keep folder, ignore contents)
+test_evidence/*
+!test_evidence/.gitkeep
+*.png

agentic_qa_maestro-0.1.0/ARCHITECTURE.md

@@ -0,0 +1,254 @@
+# Architecture — Agentic QA Maestro
+
+> Multi-agent QA automation powered by [Microsoft Agent Framework](https://github.com/microsoft/agent-framework).
+
+---
+
+## High-Level Overview
+
+```mermaid
+graph TD
+    subgraph UI["User Interface"]
+        CLI["CLI (main)"]
+        WebUI["Web UI (FastAPI)"]
+    end
+
+    subgraph Core["QAMaestro (main.py)"]
+        Entry["Entry point — loads config, creates model clients & agents,<br/>dispatches to chosen execution mode<br/><br/>Modes: chat | run | pipeline"]
+    end
+
+    subgraph Teams["Team Orchestration"]
+        GC["GroupChat Team<br/>(group_chat_team.py)<br/>Dynamic orchestrator selects<br/>the next agent at each turn"]
+        SEQ["Sequential Team<br/>(sequential_team.py)<br/>Deterministic pipeline runs<br/>agents in fixed order"]
+    end
+
+    subgraph Agents["Agent Layer"]
+        direction LR
+        Orch["Orchestrator"]
+        JIRA["JIRA Agent"]
+        Browser["Browser Agent"]
+        API["API Agent"]
+        Research["Research Agent"]
+        TestRunner["Test Runner"]
+    end
+
+    subgraph Tools["Tool Layer"]
+        direction LR
+        BT["Browser Tools<br/>(Playwright)"]
+        JT["JIRA Tools<br/>(REST/httpx)"]
+        LT["Local Tools<br/>(pytest, …)"]
+        MCP["MCP Servers<br/>(external tools)"]
+    end
+
+    CLI --> Entry
+    WebUI --> Entry
+    Entry --> GC
+    Entry --> SEQ
+    GC --> Orch
+    SEQ --> Orch
+    Orch --> BT
+    Orch --> JT
+    Orch --> LT
+    Orch --> MCP
+```
+
+---
+
+## Execution Modes
+
+```mermaid
+graph TD
+    UR["User Request"] --> Chat["Chat<br/>(interactive loop)"]
+    UR --> Run["Run<br/>(one-shot task)"]
+    UR --> Pipeline["Pipeline<br/>(sequential steps)"]
+
+    Chat --> GC["GroupChat Orchestrator<br/>(dynamic agent select)"]
+    Run --> GC
+    Pipeline --> ST["Sequential Team<br/>(fixed order)"]
+```
+
+| Mode | Orchestration | Use Case |
+|------|---------------|----------|
+| **Chat** | GroupChat (dynamic) | Exploratory QA — user types tasks interactively |
+| **Run** | GroupChat (dynamic) | One-shot task execution |
+| **Pipeline** | Sequential (fixed) | Reproducible E2E pipelines in defined order |
+
+---
+
+## Data Flow
+
+```mermaid
+graph TD
+    YAML["application.yaml"] --> Config["AppConfig (config.py)<br/>• env var substitution<br/>• self-referencing templates"]
+
+    Config --> Models["Model Clients<br/>(azure_openai.py)"]
+    Config --> AgentDefs["Agent Defs<br/>(factory.py)"]
+    Config --> TeamCfg["Team Config<br/>(teams/*.py)"]
+
+    Models --> Agents["Agent Framework Agent instances"]
+    AgentDefs --> Agents
+    TeamCfg --> Agents
+
+    Agents --> ToolFns["@tool functions<br/>(browser, jira, local, mcp)"]
+
+    ToolFns --> PW["Playwright<br/>(browser)"]
+    ToolFns --> JIRAAPI["JIRA API<br/>(httpx)"]
+    ToolFns --> Pytest["pytest<br/>(subprocess)"]
+```
+
+---
+
+## Agent Roles
+
+```mermaid
+graph TD
+    Orch["Orchestrator<br/>Coordinates the full<br/>E2E pipeline phases"] --> JIRA["JIRA Agent<br/>• Fetch issues<br/>• Extract criteria<br/>• Create bugs<br/>• Comment<br/>• Transition"]
+    Orch --> Browser["Browser Agent<br/>• Start browser<br/>• Nav, click, fill<br/>• Screenshots<br/>• AAD auth"]
+    Orch --> API["API Agent<br/>• REST calls<br/>• Contract test"]
+    Orch --> Research["Research Agent<br/>• Web search<br/>• Doc lookup"]
+    Orch --> TestRunner["Test Runner<br/>• pytest execute<br/>• Collect results<br/>• Analyze output"]
+```
+
+---
+
+## E2E Pipeline Phases
+
+```mermaid
+graph LR
+    P1["Phase 1<br/>Requirement Analysis<br/><br/>JIRA Agent<br/>fetches story + criteria"] --> P2["Phase 2<br/>App Discovery<br/><br/>Browser Agent<br/>explores UI via Playwright"]
+    P2 --> P3["Phase 3<br/>Test Case Generation<br/><br/>Orchestrator<br/>creates tests from criteria"]
+    P3 --> P4["Phase 4<br/>Test Execution<br/><br/>Browser Agent + Test Runner<br/>run tests"]
+    P4 --> P5["Phase 5<br/>Bug Reporting<br/><br/>JIRA Agent<br/>files bugs, posts summary"]
+    P5 --> P6["Phase 6<br/>Cleanup<br/><br/>Browser Agent<br/>closes browser"]
+```
+
+---
+
+## Component Details
+
+### Configuration (`config.py`)
+
+Loads `application.yaml` with two substitution passes:
+
+1. **Environment variables**: `${env:AZURE_OPENAI_API_KEY}` → value from env
+2. **Self-references**: `${this:models.default.endpoint}` → value from config
+
+Exposes typed sections: `.models`, `.agents`, `.teams`, `.mcp_servers`, `.observability`, `.web_ui`
+
+### Model Client Factory (`models/azure_openai.py`)
+
+- Creates `OpenAIChatCompletionClient` instances for Azure OpenAI
+- Supports API key or `DefaultAzureCredential` (Azure AD)
+- Auto-detects SSL certificate bundles for corporate proxy environments
+
+### Agent Factory (`agents/factory.py`)
+
+- Maps YAML agent definitions → Agent Framework `Agent` objects
+- Assigns tools and model clients per agent
+- Provides default system prompts for each role
+
+### Tool Layer
+
+| Module | Tools | External Dependency |
+|--------|-------|---------------------|
+| `browser_tools.py` | `start_browser`, `open_url`, `click`, `fill`, `select_option`, `wait_for_selector`, `get_text`, `get_page_content`, `screenshot`, `check_browser`, `close_browser`, `authenticate_aad` | Playwright (Chromium) |
+| `jira_tools.py` | `jira_get_issue`, `jira_add_comment`, `jira_get_comments`, `jira_search_issues`, `jira_create_issue`, `jira_create_bug`, `jira_transition_issue` | JIRA REST API (httpx) |
+| `local_tools.py` | `get_current_time`, `run_pytest`, `collect_pytest_tests` | pytest (subprocess) |
+
+### Observability (`observability/tracing.py`)
+
+```mermaid
+graph TD
+    ST["setup_tracing()"] --> Console["Console Exporter"]
+    ST --> OTLP["OTLP (gRPC)"]
+    ST --> AzMon["Azure Monitor Exporter"]
+```
+
+- OpenTelemetry tracing with configurable exporters
+- Structured Python logging with suppression of noisy libraries
+
+### Web UI (`web_ui/app.py`)
+
+```mermaid
+graph LR
+    Browser["Browser"] --> FastAPI["FastAPI + Jinja2"]
+    FastAPI --> GET_root["GET / → Dashboard (HTML)"]
+    FastAPI --> POST_chat["POST /api/chat → Run group chat"]
+    FastAPI --> GET_status["GET /api/status → Agent status"]
+    FastAPI --> GET_history["GET /api/history → Run history"]
+```
+
+---
+
+## Design Patterns
+
+| Pattern | Where | Purpose |
+|---------|-------|---------|
+| **Factory** | `create_model_clients_from_config()`, `create_agents_from_config()`, `create_*_team()` | Decouple creation from config |
+| **Singleton** | Browser state in `browser_tools.py` (module-level globals + single-thread executor) | One browser instance, thread-safe |
+| **Strategy** | AAD auth methods (`header`, `token_url`, `easyauth`, `msal_cache`); orchestration modes | Swap behavior at runtime |
+| **Decorator** | `@tool(approval_mode=...)` on all tool functions | Register functions as Agent Framework tools |
+| **Template Method** | `AppConfig._apply_substitutions()` | Recursive config resolution |
+
+---
+
+## Directory Structure
+
+```
+agentic-qa-maestro/
+├── ARCHITECTURE.md            ← You are here
+├── application.yaml           # All config: models, agents, teams, observability
+├── pyproject.toml             # Package metadata & dependencies
+├── example.env                # Template for environment variables
+│
+├── agentic_qa_maestro/
+│   ├── main.py                # QAMaestro class & CLI entry point
+│   ├── config.py              # YAML loader with env/self-ref substitution
+│   │
+│   ├── agents/
+│   │   └── factory.py         # Agent creation from config
+│   │
+│   ├── models/
+│   │   └── azure_openai.py    # Azure OpenAI client factory
+│   │
+│   ├── teams/
+│   │   ├── group_chat_team.py # GroupChatBuilder orchestration
+│   │   └── sequential_team.py # SequentialBuilder pipelines
+│   │
+│   ├── tools/
+│   │   ├── browser_tools.py   # Playwright browser automation
+│   │   ├── jira_tools.py      # JIRA REST API client
+│   │   └── local_tools.py     # pytest runner, time utilities
+│   │
+│   ├── observability/
+│   │   └── tracing.py         # OpenTelemetry + logging setup
+│   │
+│   └── web_ui/
+│       ├── app.py             # FastAPI dashboard
+│       └── templates/
+│           └── index.html     # Dark-theme chat UI
+│
+├── scripts/
+│   ├── run_e2e_pipeline.py    # Full 6-phase E2E runner
+│   └── run_jira_pipeline.py   # Lightweight JIRA-only runner
+│
+└── tests/
+    └── unit/                  # Unit tests
+```
+
+---
+
+## Technology Stack
+
+| Layer | Technology |
+|-------|-----------|
+| Agent Framework | Microsoft Agent Framework |
+| LLM Provider | Azure OpenAI (model configurable via application.yaml) |
+| Browser Automation | Playwright (Chromium) |
+| Issue Tracking | JIRA (REST API via httpx) |
+| Test Execution | pytest (subprocess) |
+| Web Framework | FastAPI + Uvicorn + Jinja2 |
+| Observability | OpenTelemetry (console / OTLP / Azure Monitor) |
+| Auth | Azure Identity (DefaultAzureCredential / API keys) |
+| Config | YAML + python-dotenv |
+| Language | Python 3.10+ |

agentic_qa_maestro-0.1.0/CODE_OF_CONDUCT.md

@@ -0,0 +1,28 @@
+# Contributor Covenant Code of Conduct
+
+## Our Pledge
+We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, religion, or sexual identity and orientation.
+
+We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community.
+
+## Our Standards
+Examples of behavior that contributes to a positive environment:
+- Demonstrating empathy and kindness
+- Being respectful of differing opinions
+- Accepting constructive criticism
+- Focusing on what is best for the community
+- Showing gratitude
+
+Examples of unacceptable behavior:
+- The use of sexualized language or imagery
+- Trolling, insulting or derogatory comments
+- Personal or political attacks
+- Public or private harassment
+- Publishing others’ private information
+- Other conduct which could reasonably be considered inappropriate
+
+## Enforcement
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the maintainers at creation-platform@amadeus.com. All complaints will be reviewed and investigated and will result in a response deemed necessary and appropriate to the circumstances.
+
+## Attribution
+This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org/), version 2.1.

agentic_qa_maestro-0.1.0/CONTRIBUTING.md

@@ -0,0 +1,31 @@
+# Contributing to Agentic QA Maestro
+
+Thank you for your interest in contributing! Please follow these guidelines to help us maintain a collaborative and high-quality project.
+
+## How to Contribute
+- Fork the repository and create your branch from `main` or `develop`.
+- Use descriptive branch names (e.g., `feature/your-feature`, `bugfix/issue-123`).
+- Write clear, concise commit messages.
+- Ensure your code passes all tests and lints (see below).
+- Submit a pull request (PR) with a clear description of your changes.
+
+## Code of Conduct
+By participating, you agree to abide by the [Code of Conduct](CODE_OF_CONDUCT.md).
+
+## Development Setup
+- Install dependencies: `pip install -e .[dev]`
+- Run tests: `pytest`
+- Lint code: `ruff .`
+
+## Reporting Issues
+- Use GitHub Issues for bugs, feature requests, and questions.
+- Provide as much detail as possible (steps to reproduce, logs, screenshots).
+
+## Review Process
+- All PRs require at least one approval.
+- Maintain backward compatibility when possible.
+- Add or update documentation as needed.
+
+## Community
+- Be respectful and constructive in all interactions.
+- For major changes, open an issue to discuss your proposal first.

agentic_qa_maestro-0.1.0/GETTINGSTARTED.md

@@ -0,0 +1,135 @@
+# Quick Start
+
+## Prerequisites
+
+- Python 3.10, 3.11, or 3.12
+- Access to Azure OpenAI (endpoint + API key or Azure AD credentials)
+- JIRA instance with API token (for JIRA workflows)
+
+## Installation
+
+```bash
+# Install the released CLI
+pip install agentic-qa-maestro
+
+# Or install it with uv
+uv tool install agentic-qa-maestro
+```
+
+Use `pip` if you want the package in your current Python environment. Use `uv tool`
+if you want an isolated CLI installation.
+
+## Configuration
+
+```bash
+# Scaffold local runtime files
+qa-maestro init
+```
+
+This creates:
+
+- `application.yaml`
+- `.env`
+- `app_flows/example-app.yaml`
+
+Edit `.env` with your credentials:
+
+```dotenv
+# Azure OpenAI (mandatory)
+DEFAULT_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
+DEFAULT_OPENAI_DEPLOYMENT=your-deployment-name
+AZURE_OPENAI_API_KEY=your-api-key-here
+DEFAULT_OPENAI_API_VERSION=2024-12-01-preview
+
+# JIRA (mandatory)
+JIRA_BASE_URL=https://your-jira-instance.atlassian.net
+JIRA_API_TOKEN=your-jira-api-token
+```
+
+All secrets are loaded via environment variables — never hardcode them in config files.
+
+## Browser Setup
+
+Install Playwright's Chromium binary once before running the full E2E pipeline:
+
+```bash
+playwright install chromium
+```
+
+## Run Tests
+
+```bash
+qa-maestro --help
+```
+
+## Run a JIRA Pipeline (Analysis Only)
+
+Fetches the JIRA ticket, generates test cases, and posts them as a comment:
+
+```bash
+qa-maestro --ticket SACP-282967
+```
+
+## Run the Full E2E Pipeline
+
+The E2E pipeline executes all 6 phases against a live application. The `--url`, `--username`, and `--password` flags are **mandatory**.
+
+```bash
+qa-maestro \
+  --ticket YOUR-TICKET-ID \
+  --url "https://your-app.example.com/login" \
+  --username "your-username" \
+  --password "your-password"
+```
+
+### E2E Pipeline Phases
+
+| Phase | Name | Description |
+|-------|------|-------------|
+| 1 | **Requirement Analysis** | Fetch JIRA ticket, extract acceptance criteria |
+| 2 | **App Discovery** | Launch browser, navigate to URL, discover UI elements and selectors |
+| 3 | **Test Case Generation** | Generate test cases from requirements + discovered UI |
+| 4 | **Test Execution** | Execute tests against the live app, take screenshots as evidence |
+| 5 | **Bug Reporting** | Create JIRA bug tickets for failures, post summary comment |
+| 6 | **Cleanup** | Close browser, output final status |
+
+### Optional: Azure AD Authentication
+
+For apps behind Azure AD, pass additional auth flags:
+
+```bash
+qa-maestro \
+  --ticket YOUR-TICKET-ID \
+  --url "https://your-app.example.com" \
+  --username "your-username" \
+  --password "your-password" \
+  --tenant-id "your-tenant-id" \
+  --client-id "your-client-id" \
+  --client-secret "your-client-secret" \
+  --scope "api://your-scope/.default"
+```
+
+## Start the Web UI
+
+```bash
+uvicorn agentic_qa_maestro.web_ui.app:app --port 8000
+```
+
+Open http://localhost:8000 in your browser.
+
+## CLI Usage
+
+```bash
+qa-maestro --help
+```
+
+## Development Setup
+
+If you want to work from a local source checkout instead of the released package,
+use the contributor workflow from [CONTRIBUTING.md](CONTRIBUTING.md).
+
+## Next Steps
+
+- Review `application.yaml` for agent/team/model configuration
+- See [ARCHITECTURE.md](ARCHITECTURE.md) for system design details
+- See [CONTRIBUTING.md](CONTRIBUTING.md) to contribute