qaas-python 0.0.1__tar.gz → 0.0.2__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- qaas_python-0.0.2/.gitignore +45 -0
- qaas_python-0.0.2/PKG-INFO +559 -0
- qaas_python-0.0.2/README.md +524 -0
- qaas_python-0.0.2/config/targets/corvid-09-19.yaml +56 -0
- qaas_python-0.0.2/docs/src/loops.html +119 -0
- qaas_python-0.0.2/docs/src/render.py +73 -0
- qaas_python-0.0.2/docs/src/roster.html +202 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/pyproject.toml +43 -9
- qaas_python-0.0.2/src/qaas/__init__.py +20 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/adapters/tracker.py +392 -53
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/adapters/vcs.py +260 -18
- qaas_python-0.0.2/src/qaas/browser.py +99 -0
- qaas_python-0.0.2/src/qaas/cli.py +3189 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/config.py +311 -17
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/defaults/config/agents/api.yaml +1 -1
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/defaults/config/agents/architect.yaml +1 -1
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/defaults/config/agents/auditor.yaml +1 -1
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/defaults/config/agents/browser.yaml +1 -1
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/defaults/config/agents/dba.yaml +1 -1
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/defaults/config/agents/fixer.yaml +26 -3
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/defaults/config/agents/guide.yaml +1 -1
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/defaults/config/agents/load.yaml +1 -1
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/defaults/config/agents/mapper.yaml +1 -1
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/defaults/config/agents/reporter.yaml +1 -1
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/defaults/config/agents/reproducer.yaml +1 -1
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/defaults/config/agents/reviewer.yaml +1 -1
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/defaults/config/agents/socket.yaml +1 -1
- qaas_python-0.0.2/src/qaas/defaults/config/agents/synthesizer.yaml +32 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/defaults/config/agents/triage.yaml +6 -2
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/defaults/config/agents/verifier.yaml +1 -1
- qaas_python-0.0.2/src/qaas/defaults/config/system.yaml +107 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/envelope.py +16 -3
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/envfile.py +58 -7
- qaas_python-0.0.2/src/qaas/guardrails.py +1341 -0
- qaas_python-0.0.2/src/qaas/importgraph.py +792 -0
- qaas_python-0.0.2/src/qaas/mcp/context.py +163 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/mcp/contract_diff.py +132 -35
- qaas_python-0.0.2/src/qaas/mcp/defect_memory.py +1069 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/mcp/env_control.py +380 -52
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/mcp/envelope_server.py +95 -4
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/mcp/test_runner.py +620 -48
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/mcp/tracker.py +288 -46
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/mcp/vcs.py +193 -22
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/paths.py +84 -13
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/adversarial-review/SKILL.md +1 -1
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/minimal-diff-discipline/SKILL.md +2 -2
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/routing-rules/SKILL.md +12 -3
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/verification-protocol/SKILL.md +1 -1
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/prompts/ARCHITECT.md +37 -4
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/prompts/DBA.md +8 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/prompts/FIXER.md +6 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/prompts/LOAD.md +9 -1
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/prompts/SOCKET.md +8 -0
- qaas_python-0.0.2/src/qaas/prompts/SYNTHESIZER.md +88 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/prompts/VERIFIER.md +3 -3
- qaas_python-0.0.2/src/qaas/quota.py +108 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/registry.py +184 -12
- qaas_python-0.0.2/src/qaas/router.py +1834 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/runner.py +55 -5
- qaas_python-0.0.2/src/qaas/sandbox.py +219 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/scorecard.py +215 -11
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/sdk_compat.py +24 -6
- qaas_python-0.0.2/src/qaas/store.py +569 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/target.py +86 -5
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/tasks.py +159 -8
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/trace.py +211 -16
- qaas_python-0.0.2/src/qaas/ui/__init__.py +53 -0
- qaas_python-0.0.2/src/qaas/ui/config_view.py +508 -0
- qaas_python-0.0.2/src/qaas/ui/config_write.py +373 -0
- qaas_python-0.0.2/src/qaas/ui/serve.py +150 -0
- qaas_python-0.0.2/src/qaas/ui/server.py +925 -0
- qaas_python-0.0.2/src/qaas/ui/state.py +995 -0
- qaas_python-0.0.2/src/qaas/ui/static/app.css +772 -0
- qaas_python-0.0.2/src/qaas/ui/static/app.js +1239 -0
- qaas_python-0.0.2/src/qaas/ui/static/index.html +140 -0
- qaas_python-0.0.2/target-app/.gitignore +4 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/api/app/routes/invoices.py +12 -2
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/api/app/routes/orders.py +14 -3
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/api/app/schemas.py +2 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/api/migrations/001_init.sql +21 -2
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/defects.yaml +129 -3
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/docker-compose.yml +0 -1
- {qaas_python-0.0.1 → qaas_python-0.0.2}/tests/adapters/test_github_vcs.py +60 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/tests/adapters/test_jira_tracker.py +309 -6
- qaas_python-0.0.2/tests/conftest.py +112 -0
- qaas_python-0.0.2/tests/fake_anthropic.py +130 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/tests/mcp/conftest.py +10 -2
- qaas_python-0.0.2/tests/mcp/test_context.py +41 -0
- qaas_python-0.0.2/tests/mcp/test_contract_release_fixes.py +240 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/tests/mcp/test_defect_memory.py +114 -4
- qaas_python-0.0.2/tests/mcp/test_defect_memory_migration.py +308 -0
- qaas_python-0.0.2/tests/mcp/test_defect_memory_targets.py +289 -0
- qaas_python-0.0.2/tests/mcp/test_defect_memory_unmerged.py +153 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/tests/mcp/test_env_control.py +104 -4
- qaas_python-0.0.2/tests/mcp/test_env_release_fixes.py +292 -0
- qaas_python-0.0.2/tests/mcp/test_runner_release_fixes.py +273 -0
- qaas_python-0.0.2/tests/mcp/test_sdk_wire.py +108 -0
- qaas_python-0.0.2/tests/mcp/test_test_runner.py +645 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/tests/mcp/test_tracker.py +40 -23
- qaas_python-0.0.2/tests/mcp/test_tracker_policy.py +295 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/tests/mcp/test_vcs.py +231 -0
- qaas_python-0.0.2/tests/support.py +80 -0
- qaas_python-0.0.2/tests/test_browser.py +195 -0
- qaas_python-0.0.2/tests/test_cli.py +780 -0
- qaas_python-0.0.2/tests/test_cli_preflight.py +226 -0
- qaas_python-0.0.2/tests/test_cli_state_root.py +208 -0
- qaas_python-0.0.2/tests/test_cli_user_errors.py +369 -0
- qaas_python-0.0.2/tests/test_config.py +606 -0
- qaas_python-0.0.2/tests/test_credentials.py +148 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/tests/test_envfile.py +43 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/tests/test_guardrails.py +457 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/tests/test_hooks_and_skills.py +52 -0
- qaas_python-0.0.2/tests/test_importgraph.py +588 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/tests/test_paths.py +112 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/tests/test_prompt_overrides.py +2 -3
- qaas_python-0.0.2/tests/test_quota.py +166 -0
- qaas_python-0.0.2/tests/test_router.py +1790 -0
- qaas_python-0.0.2/tests/test_sandbox.py +215 -0
- qaas_python-0.0.2/tests/test_sandbox_e2e.py +113 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/tests/test_scorecard.py +282 -6
- {qaas_python-0.0.1 → qaas_python-0.0.2}/tests/test_store.py +50 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/tests/test_target_profile_is_honoured.py +84 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/tests/test_trace.py +39 -6
- qaas_python-0.0.2/tests/test_tracker_local_store.py +152 -0
- qaas_python-0.0.2/tests/ui/__init__.py +0 -0
- qaas_python-0.0.2/tests/ui/conftest.py +180 -0
- qaas_python-0.0.2/tests/ui/test_config_view.py +153 -0
- qaas_python-0.0.2/tests/ui/test_extra.py +60 -0
- qaas_python-0.0.2/tests/ui/test_hardening.py +449 -0
- qaas_python-0.0.2/tests/ui/test_liveness.py +346 -0
- qaas_python-0.0.2/tests/ui/test_rate_limited_cards.py +100 -0
- qaas_python-0.0.2/tests/ui/test_replay.py +145 -0
- qaas_python-0.0.2/tests/ui/test_server.py +510 -0
- qaas_python-0.0.2/tests/ui/test_state.py +392 -0
- qaas_python-0.0.2/tests/ui/test_stream.py +241 -0
- qaas_python-0.0.1/.gitignore +0 -24
- qaas_python-0.0.1/ARCHITECTURE.md +0 -476
- qaas_python-0.0.1/BUILD_PLAN.md +0 -306
- qaas_python-0.0.1/CHANGELOG.md +0 -38
- qaas_python-0.0.1/CLAUDE.md +0 -351
- qaas_python-0.0.1/PKG-INFO +0 -429
- qaas_python-0.0.1/README.md +0 -399
- qaas_python-0.0.1/src/qaas/cli.py +0 -1757
- qaas_python-0.0.1/src/qaas/defaults/config/system.yaml +0 -64
- qaas_python-0.0.1/src/qaas/guardrails.py +0 -589
- qaas_python-0.0.1/src/qaas/mcp/context.py +0 -78
- qaas_python-0.0.1/src/qaas/mcp/defect_memory.py +0 -495
- qaas_python-0.0.1/src/qaas/router.py +0 -581
- qaas_python-0.0.1/src/qaas/store.py +0 -323
- qaas_python-0.0.1/tests/conftest.py +0 -54
- qaas_python-0.0.1/tests/mcp/test_test_runner.py +0 -339
- qaas_python-0.0.1/tests/support.py +0 -29
- qaas_python-0.0.1/tests/test_cli.py +0 -197
- qaas_python-0.0.1/tests/test_config.py +0 -239
- qaas_python-0.0.1/tests/test_router.py +0 -630
- {qaas_python-0.0.1 → qaas_python-0.0.2}/LICENSE +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/config/targets/corvid.yaml +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/adapters/__init__.py +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/discover.py +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/mcp/__init__.py +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/.claude-plugin/plugin.json +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/a11y-audit/SKILL.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/api-surface-extraction/SKILL.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/authz-matrix-check/SKILL.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/console-error-triage/SKILL.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/contract-test-generation/SKILL.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/dedupe-strategy/SKILL.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/environment-pinning/SKILL.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/error-taxonomy/SKILL.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/exploratory-ui-walk/SKILL.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/failing-test-authoring/SKILL.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/flake-detection/SKILL.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/form-state-probe/SKILL.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/openapi-diff/SKILL.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/ownership-resolution/SKILL.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/product-task-graph/SKILL.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/regression-risk-scoring/SKILL.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/regression-suite-selection/SKILL.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/repo-cartography/SKILL.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/repro-minimisation/SKILL.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/rollback-plan-authoring/SKILL.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/root-cause-vs-symptom/SKILL.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/severity-rubric/SKILL.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/test-first-fix/SKILL.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/test-quality-audit/SKILL.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/ticket-writer/SKILL.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/plugin/skills/verdict-reporting/SKILL.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/prompts/API.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/prompts/AUDITOR.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/prompts/BROWSER.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/prompts/GUIDE.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/prompts/MAPPER.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/prompts/REPORTER.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/prompts/REPRODUCER.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/prompts/REVIEWER.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/prompts/TRIAGE.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/src/qaas/prompts/_shared.md +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/CODEOWNERS +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/api/Dockerfile +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/api/app/__init__.py +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/api/app/auth.py +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/api/app/config.py +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/api/app/db.py +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/api/app/errors.py +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/api/app/main.py +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/api/app/models.py +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/api/app/routes/__init__.py +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/api/app/routes/auth.py +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/api/app/routes/stream.py +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/api/pyproject.toml +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/api/seed/fixtures.sql +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/openapi.yaml +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/web/.gitignore +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/web/Dockerfile +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/web/index.html +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/web/package-lock.json +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/web/package.json +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/web/src/api.ts +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/web/src/components/Button.tsx +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/web/src/components/Layout.tsx +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/web/src/components/SearchInput.tsx +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/web/src/main.tsx +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/web/src/routes/CheckoutReview.tsx +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/web/src/routes/Login.tsx +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/web/src/routes/NewOrder.tsx +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/web/src/routes/OrderDetail.tsx +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/web/src/routes/OrdersList.tsx +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/web/src/styles.css +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/web/tsconfig.json +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/target-app/web/vite.config.ts +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/tests/mcp/test_contract_diff.py +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/tests/target_app/test_seeded_defects.py +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/tests/test_discover.py +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/tests/test_envelope.py +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/tests/test_registry.py +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/tests/test_skills_actually_load.py +0 -0
- {qaas_python-0.0.1 → qaas_python-0.0.2}/tests/test_user_mcp_servers.py +0 -0
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
.qaas/
|
|
2
|
+
.venv/
|
|
3
|
+
__pycache__/
|
|
4
|
+
*.pyc
|
|
5
|
+
.pytest_cache/
|
|
6
|
+
node_modules/
|
|
7
|
+
.DS_Store
|
|
8
|
+
.env
|
|
9
|
+
all_you_need_to_run_qaas_harness.md
|
|
10
|
+
# Run debris — agents write these while exploring; they are not source.
|
|
11
|
+
.playwright-mcp/
|
|
12
|
+
/*.png
|
|
13
|
+
|
|
14
|
+
# Agent worktrees — real git worktrees, never repo content.
|
|
15
|
+
.claude/worktrees/
|
|
16
|
+
|
|
17
|
+
# Build artifacts.
|
|
18
|
+
dist/
|
|
19
|
+
|
|
20
|
+
# Agent evidence screenshots land here because put_artifact takes only text.
|
|
21
|
+
target-app/*.png
|
|
22
|
+
|
|
23
|
+
# Personal notes, not published.
|
|
24
|
+
tutorial/
|
|
25
|
+
|
|
26
|
+
# REPRODUCER writes failing tests into the target's qa/repro/ during a run.
|
|
27
|
+
# They are run output, reproducible from the ledger, and they accumulate one
|
|
28
|
+
# file per finding across every run -- not source, and not reviewable as a diff.
|
|
29
|
+
target-app/qa/
|
|
30
|
+
|
|
31
|
+
# Internal and working documents. They live in this checkout and are not
|
|
32
|
+
# published: README.md is the repository's and the package's one document.
|
|
33
|
+
/CLAUDE.md
|
|
34
|
+
/MANUAL.md
|
|
35
|
+
/UNDERSTANDING_QAAS.md
|
|
36
|
+
/ARCHITECTURE.md
|
|
37
|
+
/BUILD_PLAN.md
|
|
38
|
+
/PROVIDERS_PLAN.md
|
|
39
|
+
/RELEASE_0_0_2_PLAN.md
|
|
40
|
+
/CHANGELOG.md
|
|
41
|
+
/qa-agent-system-architecture.md
|
|
42
|
+
/docs/*.md
|
|
43
|
+
|
|
44
|
+
# Probe scripts an early run wrote into this repository's own root.
|
|
45
|
+
/qa/
|
|
@@ -0,0 +1,559 @@
|
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
|
+
Name: qaas-python
|
|
3
|
+
Version: 0.0.2
|
|
4
|
+
Summary: A harness that runs sixteen governed Claude Code sessions over your repo: find, reproduce, file, fix, verify
|
|
5
|
+
Project-URL: Homepage, https://github.com/allaabdella2-us/qa-multi-agent-system
|
|
6
|
+
Project-URL: Repository, https://github.com/allaabdella2-us/qa-multi-agent-system
|
|
7
|
+
Project-URL: Documentation, https://github.com/allaabdella2-us/qa-multi-agent-system#readme
|
|
8
|
+
Project-URL: Issues, https://github.com/allaabdella2-us/qa-multi-agent-system/issues
|
|
9
|
+
Project-URL: Changelog, https://github.com/allaabdella2-us/qa-multi-agent-system#whats-new-in-002
|
|
10
|
+
Author: Alla Abdella
|
|
11
|
+
License-Expression: MIT
|
|
12
|
+
License-File: LICENSE
|
|
13
|
+
Keywords: agents,bug-finding,claude,llm,multi-agent,qa,testing
|
|
14
|
+
Classifier: Development Status :: 4 - Beta
|
|
15
|
+
Classifier: Intended Audience :: Developers
|
|
16
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
19
|
+
Classifier: Topic :: Software Development :: Quality Assurance
|
|
20
|
+
Classifier: Topic :: Software Development :: Testing
|
|
21
|
+
Requires-Python: >=3.12
|
|
22
|
+
Requires-Dist: claude-agent-sdk>=0.2.159
|
|
23
|
+
Requires-Dist: pydantic>=2.11
|
|
24
|
+
Requires-Dist: pyyaml>=6.0
|
|
25
|
+
Requires-Dist: rich>=13.9
|
|
26
|
+
Requires-Dist: typer>=0.16
|
|
27
|
+
Provides-Extra: dev
|
|
28
|
+
Requires-Dist: pytest-asyncio>=0.24; extra == 'dev'
|
|
29
|
+
Requires-Dist: pytest>=8.3; extra == 'dev'
|
|
30
|
+
Provides-Extra: ui
|
|
31
|
+
Requires-Dist: sse-starlette>=2.1; extra == 'ui'
|
|
32
|
+
Requires-Dist: starlette>=1.6; extra == 'ui'
|
|
33
|
+
Requires-Dist: uvicorn>=0.30; extra == 'ui'
|
|
34
|
+
Description-Content-Type: text/markdown
|
|
35
|
+
|
|
36
|
+
<p align="center">
|
|
37
|
+
<img src="https://raw.githubusercontent.com/allaabdella2-us/qa-multi-agent-system/main/docs/logo.jpg" alt="qaas" width="140">
|
|
38
|
+
</p>
|
|
39
|
+
|
|
40
|
+
<h1 align="center">qaas</h1>
|
|
41
|
+
|
|
42
|
+
<p align="center">
|
|
43
|
+
<em>Sixteen governed Claude Code sessions that read your repository, find
|
|
44
|
+
defects, reproduce them into failing tests, file them, fix them and verify
|
|
45
|
+
the fix.</em>
|
|
46
|
+
</p>
|
|
47
|
+
|
|
48
|
+
<p align="center">
|
|
49
|
+
<a href="https://pypi.org/project/qaas-python/"><img src="https://img.shields.io/pypi/v/qaas-python.svg" alt="PyPI"></a>
|
|
50
|
+
<img src="https://img.shields.io/badge/python-3.12%2B-blue.svg" alt="Python 3.12+">
|
|
51
|
+
<img src="https://img.shields.io/badge/license-MIT-green.svg" alt="MIT">
|
|
52
|
+
</p>
|
|
53
|
+
|
|
54
|
+
---
|
|
55
|
+
|
|
56
|
+
## Contents
|
|
57
|
+
|
|
58
|
+
1. [What it is](#what-it-is)
|
|
59
|
+
2. [Install](#install)
|
|
60
|
+
3. [Quick start](#quick-start)
|
|
61
|
+
4. [How a run works](#how-a-run-works)
|
|
62
|
+
5. [Run modes](#run-modes)
|
|
63
|
+
6. [What you get](#what-you-get)
|
|
64
|
+
7. [Why it is safe to point at your code](#why-it-is-safe-to-point-at-your-code)
|
|
65
|
+
8. [Pointing it at your own project](#pointing-it-at-your-own-project)
|
|
66
|
+
9. [Filing into Jira](#filing-into-jira)
|
|
67
|
+
10. [Opening pull requests on GitHub](#opening-pull-requests-on-github)
|
|
68
|
+
11. [The dashboard](#the-dashboard)
|
|
69
|
+
12. [Does it actually find things?](#does-it-actually-find-things)
|
|
70
|
+
13. [Customising it](#customising-it)
|
|
71
|
+
14. [Command reference](#command-reference)
|
|
72
|
+
15. [Troubleshooting](#troubleshooting)
|
|
73
|
+
16. [What's new in 0.0.2](#whats-new-in-002)
|
|
74
|
+
17. [Roadmap and contributing](#roadmap-and-contributing)
|
|
75
|
+
|
|
76
|
+
---
|
|
77
|
+
|
|
78
|
+
## What it is
|
|
79
|
+
|
|
80
|
+
`qaas` is a harness **around Claude Code**, not a program that calls an API.
|
|
81
|
+
|
|
82
|
+
Each agent is a real Claude Code session: its own process, its own context, its
|
|
83
|
+
own tool allowlist, its own budget. A Python state machine — the router — runs
|
|
84
|
+
them in order and can refuse any tool call any of them makes. That is what makes
|
|
85
|
+
the per-agent cost, the context boundary and the write-permission matrix real
|
|
86
|
+
rather than bookkeeping: they are separate operating-system processes with
|
|
87
|
+
separate permissions.
|
|
88
|
+
|
|
89
|
+
<p align="center">
|
|
90
|
+
<img src="https://raw.githubusercontent.com/allaabdella2-us/qa-multi-agent-system/main/docs/roster.png" alt="The sixteen agents by phase" width="880">
|
|
91
|
+
</p>
|
|
92
|
+
|
|
93
|
+
| Agent | Phase | What it is for |
|
|
94
|
+
|---|---|---|
|
|
95
|
+
| **MAPPER** | Map | Services, routes, schema and ownership → the shared system map |
|
|
96
|
+
| **ARCHITECT** | Discover | Dependency cycles, layering violations, dead code |
|
|
97
|
+
| **API** | Discover | Contract drift, missing authorization, inconsistent errors |
|
|
98
|
+
| **BROWSER** | Discover | Real UI journeys, driven in a browser |
|
|
99
|
+
| **DBA** | Discover | Constraints the application assumes and the database does not enforce |
|
|
100
|
+
| **AUDITOR** | Discover | Authorization, committed secrets, vulnerable dependencies |
|
|
101
|
+
| **SOCKET** | Discover | WebSocket auth, reconnect, backpressure |
|
|
102
|
+
| **GUIDE** | Discover | Whether a person can actually find and use the feature |
|
|
103
|
+
| **LOAD** | Discover | N+1 queries, hot paths, unbounded results |
|
|
104
|
+
| **SYNTHESIZER** | Synthesise | The defect that is two findings until someone joins them |
|
|
105
|
+
| **TRIAGE** | File | Dedupe, score, file — the only agent that writes to a tracker |
|
|
106
|
+
| **REPRODUCER** | Verify loop | Minimise it, write the failing test, measure the flake |
|
|
107
|
+
| **VERIFIER** | Verify loop | Re-run the original test → VERIFIED / NOT_FIXED / REGRESSED |
|
|
108
|
+
| **FIXER** | Verify loop | The smallest fix, on its own branch |
|
|
109
|
+
| **REVIEWER** | Verify loop | Adversarial review: root cause or symptom? |
|
|
110
|
+
| **REPORTER** | Report | What was found, what recurred, what nothing reached |
|
|
111
|
+
|
|
112
|
+
---
|
|
113
|
+
|
|
114
|
+
## Install
|
|
115
|
+
|
|
116
|
+
```bash
|
|
117
|
+
pip install qaas-python # the CLI and the import package are both `qaas`
|
|
118
|
+
pip install "qaas-python[ui]" # adds `qaas dashboard`
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
**Requirements**
|
|
122
|
+
|
|
123
|
+
| | |
|
|
124
|
+
|---|---|
|
|
125
|
+
| Python | 3.12 or newer |
|
|
126
|
+
| Model access | Signed in to Claude Code (`claude`), or `ANTHROPIC_API_KEY` set. The Claude Code binary ships with the SDK — nothing else to install. |
|
|
127
|
+
| Browser agents *(optional)* | `npx -y @playwright/mcp@0.0.82 install-browser chrome-for-testing` — only for BROWSER and GUIDE. Needs Node.js. |
|
|
128
|
+
| Shell sandbox | Built in on macOS. On Linux: `sudo apt-get install bubblewrap socat`. |
|
|
129
|
+
| Demo app *(optional)* | Docker, for `target-app/` in this repository |
|
|
130
|
+
|
|
131
|
+
`qaas doctor` checks all of these for the target you point it at and says what
|
|
132
|
+
is missing.
|
|
133
|
+
|
|
134
|
+
---
|
|
135
|
+
|
|
136
|
+
## Quick start
|
|
137
|
+
|
|
138
|
+
```bash
|
|
139
|
+
cd your-project
|
|
140
|
+
qaas init . # writes .qaas/ and a target profile
|
|
141
|
+
qaas doctor # what this target makes possible
|
|
142
|
+
qaas run --mode pr-check --dry-run # the whole plan: no API call, no money
|
|
143
|
+
qaas run --mode pr-check # a real run
|
|
144
|
+
qaas show <run-id> # what it found, what it cost
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
Or point it at any repository without setting anything up first:
|
|
148
|
+
|
|
149
|
+
```bash
|
|
150
|
+
qaas run --repo https://github.com/someone/their-project --mode nightly
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
`--dry-run` renders every agent's model, turn cap, tool allowlist and prompt size
|
|
154
|
+
without spending anything. Run it before the first paid run, and after any change
|
|
155
|
+
to a prompt or a policy.
|
|
156
|
+
|
|
157
|
+
---
|
|
158
|
+
|
|
159
|
+
## How a run works
|
|
160
|
+
|
|
161
|
+
```
|
|
162
|
+
map -> discover -> synthesise -> file -> verify loop (per ticket) -> report
|
|
163
|
+
MAPPER API/BROWSER/… SYNTHESIZER TRIAGE REPRODUCER -> VERIFIER REPORTER
|
|
164
|
+
FIXER -> REVIEWER -> VERIFIER
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
1. **Map.** MAPPER reads the repository once and publishes a versioned system
|
|
168
|
+
map every other agent reads.
|
|
169
|
+
2. **Discover.** The discovery agents run concurrently. Each is a specialist
|
|
170
|
+
that can only read — the finder never fixes.
|
|
171
|
+
3. **Synthesise.** SYNTHESIZER reads every finding and joins the ones that are
|
|
172
|
+
one defect seen from two sides (an endpoint with no org filter *and* a table
|
|
173
|
+
with no owning-org constraint is one cross-tenant leak, not two minor issues).
|
|
174
|
+
4. **File.** TRIAGE dedupes against everything filed before, scores, and files
|
|
175
|
+
tickets.
|
|
176
|
+
5. **Verify loop, one ticket at a time.** REPRODUCER writes the failing test;
|
|
177
|
+
VERIFIER confirms it fails; FIXER writes the smallest fix on its own branch;
|
|
178
|
+
REVIEWER reviews it adversarially; VERIFIER re-runs the original test. A
|
|
179
|
+
ticket ends **VERIFIED**, or is handed to a human — never looped forever.
|
|
180
|
+
6. **Report.** REPORTER summarises what was found, fixed, and left.
|
|
181
|
+
|
|
182
|
+
**Escalation is a designed ending, not a failure.** Some tickets are a product
|
|
183
|
+
question no model can answer — *this fix is correct, but it breaks the orders
|
|
184
|
+
page, which has no paging: ship now or hold?* The run says so and stops working
|
|
185
|
+
that ticket until you answer:
|
|
186
|
+
|
|
187
|
+
```bash
|
|
188
|
+
qaas escalations # what is waiting on you
|
|
189
|
+
qaas answer QAAS-60 --decision proceed --note "add paging in the same PR"
|
|
190
|
+
qaas answer QAAS-31 --decision hold --note "outside FIXER's paths; mine"
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
An answer records one line in the run's ledger and dispatches nothing; your note
|
|
194
|
+
reaches FIXER and REVIEWER verbatim on the next cycle.
|
|
195
|
+
|
|
196
|
+
**Stopping and resuming.** Every run can be resumed with
|
|
197
|
+
`qaas run --mode <mode> --run-id <run-id>`. A resume skips everything already
|
|
198
|
+
done — the map, finished agents, filed tickets, verified tickets, and tickets
|
|
199
|
+
waiting on your answer — and keeps the run's ticket cap.
|
|
200
|
+
|
|
201
|
+
**Provider usage limits.** When Claude Code reports a session limit and says
|
|
202
|
+
when it resets ("resets 3:50pm"), the run **waits for the reset and retries the
|
|
203
|
+
agent it stopped**, as long as the reset falls inside the run's own time limit.
|
|
204
|
+
Nothing else is dispatched into the limit while it waits. If it cannot wait, it
|
|
205
|
+
stops cleanly and prints the resume command.
|
|
206
|
+
|
|
207
|
+
---
|
|
208
|
+
|
|
209
|
+
## Run modes
|
|
210
|
+
|
|
211
|
+
| Mode | Agents | Wall clock | Files tickets | Use it for |
|
|
212
|
+
|---|---|---|---|---|
|
|
213
|
+
| `pr-check` | 8 | 15 min | yes | Every pull request |
|
|
214
|
+
| `nightly` | 13 | 2 h | yes | A scheduled sweep: find, reproduce, file |
|
|
215
|
+
| `incident` | 1 | 10 min | no | A quick diagnostic |
|
|
216
|
+
| `fix-cycle` | 3 | 4 h | yes | Fix and verify tickets already filed |
|
|
217
|
+
| `full-loop` | 16 | 8 h | yes | Everything, end to end |
|
|
218
|
+
|
|
219
|
+
Useful flags on `qaas run`:
|
|
220
|
+
|
|
221
|
+
| Flag | What it does |
|
|
222
|
+
|---|---|
|
|
223
|
+
| `--dry-run` | Build every agent's options and stop. Free. |
|
|
224
|
+
| `--only API --only DBA` | Restrict the run to named agents |
|
|
225
|
+
| `--target <name>` / `--repo <path-or-url>` | Choose what to run against |
|
|
226
|
+
| `--run-id <id>` | Resume a run |
|
|
227
|
+
| `--ticket QAAS-12` | Restrict a `fix-cycle` to named tickets |
|
|
228
|
+
| `--from-board "Ready for Fix"` | Take the tickets from a Jira column: drag a card there and the next run picks it up |
|
|
229
|
+
| `--dashboard` | Serve the live dashboard alongside the run |
|
|
230
|
+
|
|
231
|
+
---
|
|
232
|
+
|
|
233
|
+
## What you get
|
|
234
|
+
|
|
235
|
+
- **A run ledger** — every tool call, refusal, escalation and verdict, append-only,
|
|
236
|
+
on disk under `.qaas/runs/<run-id>/`. `qaas trace <run-id> --follow` streams
|
|
237
|
+
it; `qaas dashboard` renders it.
|
|
238
|
+
- **Defect envelopes, not prose.** Agents never hand each other paragraphs. One
|
|
239
|
+
validated type crosses every boundary, with two gates built into it: a finding
|
|
240
|
+
needs evidence, and it needs confidence above the filing threshold. Findings
|
|
241
|
+
that fail are held, not filed.
|
|
242
|
+
- **Failing tests, not bug reports.** REPRODUCER writes the shortest test that
|
|
243
|
+
makes the defect appear, runs it several times to measure flake, and commits it
|
|
244
|
+
on its own branch. A finding it cannot reproduce is demoted.
|
|
245
|
+
- **Tickets** in Jira or in a local file-backed tracker, deduplicated across runs
|
|
246
|
+
by a structural fingerprint that ignores prose, line numbers and timestamps.
|
|
247
|
+
- **Fixes** on their own branches, reviewed adversarially and verified against
|
|
248
|
+
the original failing test before anything is closed. **Nothing merges** —
|
|
249
|
+
merging is a human decision and there is no code path to it.
|
|
250
|
+
- **A memory across runs.** A defect seen before is recognised; one whose fix
|
|
251
|
+
was merged and that comes back is reported as a regression. A fix that is
|
|
252
|
+
verified but not yet merged is not mistaken for one.
|
|
253
|
+
|
|
254
|
+
---
|
|
255
|
+
|
|
256
|
+
## Why it is safe to point at your code
|
|
257
|
+
|
|
258
|
+
Every agent's permissions are declared in its config and **enforced in Python
|
|
259
|
+
before the tool runs** — not requested in a prompt.
|
|
260
|
+
|
|
261
|
+
- **Discovery agents cannot write at all.**
|
|
262
|
+
- **FIXER writes to source and cannot touch the test that defines success**, nor
|
|
263
|
+
migrations, auth, payment paths, secrets or CI config. It is also bounded by a
|
|
264
|
+
diff budget (files and lines) per ticket.
|
|
265
|
+
- **The shell cannot go around the rules.** Commands are parsed for what they
|
|
266
|
+
write; a command that mutates and whose destination cannot be resolved is
|
|
267
|
+
refused. `git push --force`, pushes to protected branches and merges are
|
|
268
|
+
refused outright.
|
|
269
|
+
- **The shell runs in an OS sandbox** (Seatbelt on macOS, bubblewrap on Linux):
|
|
270
|
+
writes only inside the checkout and a private temp directory, no reads of
|
|
271
|
+
`~/.ssh`, `~/.aws`, `~/.config/gh` or other credential stores, and network
|
|
272
|
+
only to your application.
|
|
273
|
+
- **Your credentials stay yours.** Agents get an environment with Jira, GitHub
|
|
274
|
+
and other tokens blanked; the target's own test suite runs with an allowlisted
|
|
275
|
+
environment.
|
|
276
|
+
- **Nothing is loaded from the target's settings.** A repository cloned from a URL
|
|
277
|
+
cannot inject hooks or MCP servers into the process holding your API key.
|
|
278
|
+
- **Security findings are never filed where everyone can read them** — they need a
|
|
279
|
+
restricted Jira project, and a fix for one is committed but never pushed.
|
|
280
|
+
|
|
281
|
+
Every refusal is logged with its reason, and the agent adapts; the turn does not
|
|
282
|
+
die.
|
|
283
|
+
|
|
284
|
+
---
|
|
285
|
+
|
|
286
|
+
## Pointing it at your own project
|
|
287
|
+
|
|
288
|
+
A **target profile** makes the system portable: nothing in any prompt names a
|
|
289
|
+
specific application. `qaas init` writes one by reading your repository and tells
|
|
290
|
+
you what it guessed.
|
|
291
|
+
|
|
292
|
+
```yaml
|
|
293
|
+
name: my-service
|
|
294
|
+
root: .
|
|
295
|
+
layout:
|
|
296
|
+
backend: [api]
|
|
297
|
+
frontend: [web]
|
|
298
|
+
spec: openapi.yaml
|
|
299
|
+
environment:
|
|
300
|
+
mode: compose # none | external | compose
|
|
301
|
+
api_url: http://localhost:8000
|
|
302
|
+
web_url: http://localhost:5173
|
|
303
|
+
auth:
|
|
304
|
+
mode: login
|
|
305
|
+
login_endpoint: POST /v1/auth/login
|
|
306
|
+
roles:
|
|
307
|
+
admin: {username: admin@example.test, password_env: DEMO_ADMIN_PASSWORD}
|
|
308
|
+
```
|
|
309
|
+
|
|
310
|
+
| `environment.mode` | Meaning |
|
|
311
|
+
|---|---|
|
|
312
|
+
| `none` | Static reads only |
|
|
313
|
+
| `external` | Agents may exercise the running app but never reset it — someone else may rely on it |
|
|
314
|
+
| `compose` | This run owns the lifecycle: it may build, seed, reset and tear down |
|
|
315
|
+
|
|
316
|
+
Credentials are never written into a profile; it names environment variables.
|
|
317
|
+
|
|
318
|
+
**Set the diff budget on a real codebase.** FIXER ships with a five-file,
|
|
319
|
+
150-line limit on how much production code it may change before a human sees it.
|
|
320
|
+
In a codebase whose defects are duplication, a correct fix touches every
|
|
321
|
+
duplicate. Raise (or lower) it per target:
|
|
322
|
+
|
|
323
|
+
```yaml
|
|
324
|
+
diff_budget: {max_diff_files: 12, max_diff_lines: 400}
|
|
325
|
+
```
|
|
326
|
+
|
|
327
|
+
When a fix genuinely exceeds the budget, the run escalates saying it is *wider
|
|
328
|
+
than the envelope, not wrong*.
|
|
329
|
+
|
|
330
|
+
---
|
|
331
|
+
|
|
332
|
+
## Filing into Jira
|
|
333
|
+
|
|
334
|
+
```bash
|
|
335
|
+
# .qaas/.env — read by every command; anything already exported wins
|
|
336
|
+
JIRA_BASE_URL=https://you.atlassian.net
|
|
337
|
+
JIRA_EMAIL=bot@example.com
|
|
338
|
+
JIRA_API_TOKEN=... # an API token, not a password
|
|
339
|
+
JIRA_PROJECT_KEY=QA
|
|
340
|
+
JIRA_SECURITY_PROJECT_KEY=QASEC # optional: a restricted project for security findings
|
|
341
|
+
```
|
|
342
|
+
|
|
343
|
+
```bash
|
|
344
|
+
QAAS_TRACKER=jira qaas tracker-check # validates everything, creates nothing
|
|
345
|
+
QAAS_TRACKER=jira qaas run --mode nightly
|
|
346
|
+
qaas board # this repository's board
|
|
347
|
+
```
|
|
348
|
+
|
|
349
|
+
- **Use a dedicated bot account** with *Browse*, *Create*, *Transition* and
|
|
350
|
+
*Link* permissions on the project. Jira Cloud only.
|
|
351
|
+
- **`tracker-check` first.** It confirms the credentials, the project and issue
|
|
352
|
+
type, maps your workflow's statuses, and prints the exact request it *would*
|
|
353
|
+
send.
|
|
354
|
+
- **One view per repository.** Every ticket carries a `repo-<target>` label, and
|
|
355
|
+
`qaas board` finds or creates a saved filter (and a board, where the project
|
|
356
|
+
allows one) over exactly that label.
|
|
357
|
+
- **Dedupe across runs.** Tickets carry a fingerprint label, so next week's run
|
|
358
|
+
recognises a defect it already filed instead of filing it again.
|
|
359
|
+
- **Security findings are refused** unless `JIRA_SECURITY_PROJECT_KEY` names a
|
|
360
|
+
restricted project: filing a vulnerability where the whole company can read it
|
|
361
|
+
is a disclosure with no undo.
|
|
362
|
+
|
|
363
|
+
`.qaas/.env` is gitignored by `qaas init`. The default tracker is `local`, which
|
|
364
|
+
writes tickets as JSON under `.qaas/tickets/` so you can see what *would* be filed.
|
|
365
|
+
|
|
366
|
+
---
|
|
367
|
+
|
|
368
|
+
## Opening pull requests on GitHub
|
|
369
|
+
|
|
370
|
+
The default is `vcs: local`: fixes are committed to branches in your checkout and
|
|
371
|
+
nothing is pushed. To have FIXER push its branch and open a **draft** pull
|
|
372
|
+
request, set `vcs: github` in `.qaas/config/system.yaml` (or `QAAS_VCS=github`)
|
|
373
|
+
and authenticate the GitHub CLI with `gh auth login`. Protected branches can
|
|
374
|
+
never be pushed to, security fixes are never pushed, and nothing is ever merged.
|
|
375
|
+
|
|
376
|
+
---
|
|
377
|
+
|
|
378
|
+
## The dashboard
|
|
379
|
+
|
|
380
|
+
```bash
|
|
381
|
+
pip install "qaas-python[ui]"
|
|
382
|
+
qaas dashboard # the latest run, at http://127.0.0.1:7777
|
|
383
|
+
qaas dashboard <run-id> # a specific run
|
|
384
|
+
qaas run --mode nightly --dashboard # serve it alongside a run
|
|
385
|
+
```
|
|
386
|
+
|
|
387
|
+
<p align="center">
|
|
388
|
+
<img src="https://raw.githubusercontent.com/allaabdella2-us/qa-multi-agent-system/main/docs/dashboard.png" alt="The dashboard over a full sixteen-agent run: every phase complete, agent cards by layer, the timeline, findings and the ledger" width="880">
|
|
389
|
+
</p>
|
|
390
|
+
|
|
391
|
+
It shows the phases, one card per agent (cost, findings, refusals), the
|
|
392
|
+
findings, tickets, escalations and a live feed of the ledger — and the
|
|
393
|
+
installation's configuration.
|
|
394
|
+
|
|
395
|
+
**Replay a finished run** as if it were live: pick a speed and click **▶ replay**
|
|
396
|
+
in the header, or open `http://127.0.0.1:7777/?replay=60#<run-id>`. At ×60 a
|
|
397
|
+
five-hour run plays back in about two minutes, agent by agent, finding by
|
|
398
|
+
finding, ticket by ticket.
|
|
399
|
+
|
|
400
|
+
| Card | Meaning |
|
|
401
|
+
|---|---|
|
|
402
|
+
| Green — *done* | The agent's latest invocation succeeded |
|
|
403
|
+
| Red — *failed* | The agent failed |
|
|
404
|
+
| Amber — *waiting* | A provider usage limit is being waited out; the card says when it retries |
|
|
405
|
+
| Amber — *rate-limited* | The run stopped on a usage limit it could not wait out; the card shows the resume command |
|
|
406
|
+
| Amber — *interrupted* | The process running it was stopped before it finished |
|
|
407
|
+
|
|
408
|
+
It binds to localhost only and reads the ledger; the one thing it can write is
|
|
409
|
+
*tuning* — which model an agent runs, a turn cap, a threshold — never an agent's
|
|
410
|
+
permissions.
|
|
411
|
+
|
|
412
|
+
---
|
|
413
|
+
|
|
414
|
+
## Does it actually find things?
|
|
415
|
+
|
|
416
|
+
`qaas score` answers that with numbers.
|
|
417
|
+
|
|
418
|
+
This repository ships `target-app/`, a deliberately buggy FastAPI + React
|
|
419
|
+
application, and `target-app/defects.yaml`, a golden ledger of exactly what is
|
|
420
|
+
wrong with it — plus correct-but-suspicious code, so precision is measured rather
|
|
421
|
+
than assumed. Matching is deterministic, not model-judged.
|
|
422
|
+
|
|
423
|
+
```bash
|
|
424
|
+
cd target-app && docker compose up -d
|
|
425
|
+
qaas run --mode nightly
|
|
426
|
+
qaas score # recall, precision, false positives — per agent
|
|
427
|
+
qaas sweep --min-precision 0.7 # run + score + exit non-zero below the gate (the cron line)
|
|
428
|
+
```
|
|
429
|
+
|
|
430
|
+
The latest full-loop run on the demo app found **19 of 21** seeded defects
|
|
431
|
+
(90% recall), filed three tickets and verified two fixes, for about $66.
|
|
432
|
+
|
|
433
|
+
---
|
|
434
|
+
|
|
435
|
+
## Customising it
|
|
436
|
+
|
|
437
|
+
**Agents are data.** An agent is a prompt file plus a YAML file; adding one needs
|
|
438
|
+
no Python.
|
|
439
|
+
|
|
440
|
+
```bash
|
|
441
|
+
qaas prompts list # every prompt and which layer it came from
|
|
442
|
+
qaas prompts eject API # copy it into .qaas/prompts/ to edit
|
|
443
|
+
qaas prompts diff # what you changed against what ships
|
|
444
|
+
```
|
|
445
|
+
|
|
446
|
+
**Config layers the ordinary way:** `--config` beats your project's `.qaas/config/`,
|
|
447
|
+
which beats what ships. Agent files, prompts and skills merge by name, so changing
|
|
448
|
+
one agent is one dropped-in file. `.qaas/config/overrides.yaml` changes single
|
|
449
|
+
values without forking anything:
|
|
450
|
+
|
|
451
|
+
```yaml
|
|
452
|
+
thresholds:
|
|
453
|
+
max_tickets_per_run: 10 # stop filing after this many tickets per run
|
|
454
|
+
min_confidence_to_file: 0.7 # file fewer, surer findings
|
|
455
|
+
quota_wait_max_s: 21600 # longest to wait for a usage limit to reset (0 = never)
|
|
456
|
+
agents:
|
|
457
|
+
FIXER: {max_budget_usd: 20, max_turns: 120}
|
|
458
|
+
```
|
|
459
|
+
|
|
460
|
+
**Sandbox:** `sandbox.mode` in `system.yaml` is `auto` (sandbox where the OS can),
|
|
461
|
+
`required` (refuse to run unsandboxed) or `off`.
|
|
462
|
+
|
|
463
|
+
---
|
|
464
|
+
|
|
465
|
+
## Command reference
|
|
466
|
+
|
|
467
|
+
| Command | What it does |
|
|
468
|
+
|---|---|
|
|
469
|
+
| `qaas init <repo>` | Scaffold `.qaas/` and a target profile |
|
|
470
|
+
| `qaas targets` | List the target profiles this project can see |
|
|
471
|
+
| `qaas doctor [--target X]` | What a target makes possible, and what is missing |
|
|
472
|
+
| `qaas validate` | Check config, prompts and tool allowlists — no API call |
|
|
473
|
+
| `qaas run --mode M` | Run. Costs money unless `--dry-run` |
|
|
474
|
+
| `qaas runs` | Recent runs with cost and finding count |
|
|
475
|
+
| `qaas show <run-id>` | One run's findings, tickets, escalations and cost |
|
|
476
|
+
| `qaas trace <run-id> [--follow]` | The ledger as a timeline |
|
|
477
|
+
| `qaas map` | The system map MAPPER produced |
|
|
478
|
+
| `qaas dashboard [<run-id>]` | The live run view in a browser |
|
|
479
|
+
| `qaas escalations` | What is blocked on a human, across runs |
|
|
480
|
+
| `qaas answer <TICKET> --decision proceed\|hold --note "..."` | Answer an escalation |
|
|
481
|
+
| `qaas score [<run-id>]` | Recall and precision against the golden ledger |
|
|
482
|
+
| `qaas sweep` | Run, score, and fail below a precision gate |
|
|
483
|
+
| `qaas board` | Show or create this repository's Jira board |
|
|
484
|
+
| `qaas tracker-check` | Validate the tracker configuration; creates nothing |
|
|
485
|
+
| `qaas prompts list / eject / diff` | Inspect and override agent prompts |
|
|
486
|
+
|
|
487
|
+
Every command takes `--help`.
|
|
488
|
+
|
|
489
|
+
---
|
|
490
|
+
|
|
491
|
+
## Troubleshooting
|
|
492
|
+
|
|
493
|
+
| Symptom | Fix |
|
|
494
|
+
|---|---|
|
|
495
|
+
| BROWSER and GUIDE are skipped with "browser … is not installed" | `npx -y @playwright/mcp@0.0.82 install-browser chrome-for-testing` |
|
|
496
|
+
| `qaas doctor` says the sandbox is unavailable (Linux) | `sudo apt-get install bubblewrap socat` |
|
|
497
|
+
| A run stopped on "session limit" | Wait for the reset, then run the resume command it printed. Runs wait by themselves when the reset falls inside their time limit. |
|
|
498
|
+
| "no target profile" | Run `qaas init .` in the project, or pass `--target` / `--repo` |
|
|
499
|
+
| A fix escalated as "wider than the envelope" | Raise `diff_budget` in the target profile, or split the ticket |
|
|
500
|
+
| Security findings are not filed | Set `JIRA_SECURITY_PROJECT_KEY` to a restricted project |
|
|
501
|
+
| Anything else | `qaas doctor`, then `qaas trace <run-id>` — every refusal says why |
|
|
502
|
+
|
|
503
|
+
---
|
|
504
|
+
|
|
505
|
+
## What's new in 0.0.2
|
|
506
|
+
|
|
507
|
+
- **Every agent runs Claude Opus 5.5 with the 1M-token context**
|
|
508
|
+
(`claude-opus-5-5[1m]`), which needs `claude-agent-sdk` 0.2.159 or newer — the
|
|
509
|
+
first whose bundled Claude Code knows the model. Change one agent's model in
|
|
510
|
+
`.qaas/config/overrides.yaml` or from the dashboard's configuration view.
|
|
511
|
+
- **Sixteen agents** — ARCHITECT, DBA, AUDITOR, SOCKET, GUIDE, LOAD and
|
|
512
|
+
SYNTHESIZER join the roster, and a verify loop (REPRODUCER → VERIFIER → FIXER →
|
|
513
|
+
REVIEWER → VERIFIER) that runs per ticket.
|
|
514
|
+
- **An OS sandbox around every agent's shell**, credentials blanked in agents'
|
|
515
|
+
environments, and a shell-command guardrail that git's global options, wrappers
|
|
516
|
+
and indirection can no longer get around.
|
|
517
|
+
- **Escalations you can answer** (`qaas escalations`, `qaas answer`), and
|
|
518
|
+
`--from-board` to take work from a Jira column.
|
|
519
|
+
- **Resumable runs that do only what is left**, including the ticket cap, and
|
|
520
|
+
**usage limits waited out** instead of failing the run.
|
|
521
|
+
- **A browser check** before BROWSER and GUIDE run, a headless browser, and the
|
|
522
|
+
correct install command.
|
|
523
|
+
- **Memory that knows merged from verified:** a fix waiting to be merged is not
|
|
524
|
+
reported as a regression when the defect is seen again.
|
|
525
|
+
- **The dashboard** — phases, agent cards, findings, tickets, escalations and a
|
|
526
|
+
configuration view; rate-limited agents drawn amber, never as failures.
|
|
527
|
+
- **Per-agent scoring** with `qaas score`, and `qaas sweep` as a precision gate.
|
|
528
|
+
|
|
529
|
+
---
|
|
530
|
+
|
|
531
|
+
## Roadmap and contributing
|
|
532
|
+
|
|
533
|
+
`qaas` runs on the Claude Agent SDK today. Making the model a *choice* is the
|
|
534
|
+
next step, and the most useful place for help: a provider interface in the
|
|
535
|
+
runner, a second implementation behind it, local models for the agents that do
|
|
536
|
+
not need a frontier one, and a scored per-agent model comparison with
|
|
537
|
+
`qaas score` as the referee.
|
|
538
|
+
|
|
539
|
+
Issues and pull requests are welcome — especially a new agent (a prompt plus a
|
|
540
|
+
YAML file), or simply running `qaas` against your own repository and reporting
|
|
541
|
+
what it got wrong.
|
|
542
|
+
|
|
543
|
+
```bash
|
|
544
|
+
git clone https://github.com/allaabdella2-us/qa-multi-agent-system
|
|
545
|
+
cd qa-multi-agent-system
|
|
546
|
+
uv venv && uv pip install -e ".[dev]"
|
|
547
|
+
|
|
548
|
+
pytest -q # 1609 tests, offline, free, no API key
|
|
549
|
+
qaas validate # config, prompts and allowlists cohere
|
|
550
|
+
qaas run --mode pr-check --dry-run # every agent's options assemble
|
|
551
|
+
```
|
|
552
|
+
|
|
553
|
+
Those three are the whole CI gate. The default test run makes no API calls and
|
|
554
|
+
touches no network. If you change a seeded defect in `target-app/`, change its
|
|
555
|
+
entry in `defects.yaml` in the same commit.
|
|
556
|
+
|
|
557
|
+
---
|
|
558
|
+
|
|
559
|
+
**Status:** 0.0.2, beta. **License:** MIT.
|