qaas-python 0.3.1__py3-none-any.whl → 0.3.2__py3-none-any.whl

{qaas_python-0.3.1.dist-info → qaas_python-0.3.2.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: qaas-python
-Version: 0.3.1
+Version: 0.3.2
 Summary: A multi-agent QA system: finds real defects, reproduces them, files tickets, fixes them, and proves the fix
 Project-URL: Homepage, https://github.com/allaabdella2-us/qa-multi-agent-system
 Project-URL: Repository, https://github.com/allaabdella2-us/qa-multi-agent-system
@@ -310,32 +310,28 @@ the refusal and adapts.
 
 ---
 
-## 📈 Benchmarks, honestly
+## 📈 How you know it works
 
-Against the bundled demo app and its golden ledger:
+`qaas` ships with a **deliberately buggy demo app** and a golden ledger recording
+every defect seeded into it — plus **planted non-defects**: correct-but-suspicious
+code that a careless reader would report.
 
-Two runs against the demo app, scored automatically:
-
-| metric | run A (full loop) | run B (discovery only) |
-|---|--:|--:|
-| 🎯 recall | **81%** — 13 of 16 | **69%** — 11 of 16 |
-| 🔇 precision | **100%** — 0 FP | **92%** — 1 FP |
-| 🏷️ severity agreement | **100%** | **100%** |
-
-**Both numbers are shown on purpose.** A single figure would be the flattering
-one, and it would not survive contact with a second run. These are stochastic
-agents: the two runs did not find the same 11–13 defects — run B caught a
-contrast failure run A missed, and missed three run A found. Expect variance of
-this order.
+```bash
+cd target-app && docker compose up -d
+qaas run --mode nightly && qaas score
+```
 
-The ledger's `not_defects` section plants **correct-but-suspicious** code, so
-precision is measured rather than assumed.
+`qaas score` reports recall, precision and severity agreement against that
+ledger. That matters more than any number this page could print: it means a
+change to a prompt, a threshold or a model has **a measurement attached** rather
+than an opinion.
 
 > [!CAUTION]
-> **Treat this as a floor, not a proof.** Seeded defects are easier than real
-> ones and the system was calibrated against them. Two runs is not a sample.
-> These numbers show the loop works end to end and does not spray false
-> positives — not that it will find the hard bug in your codebase.
+> **Seeded defects are easier than real ones**, and this system was calibrated
+> against them. A benchmark shows the loop works end to end and does not spray
+> false positives. It does not show that it will find the hard bug in your
+> codebase. Run it on something you know well and judge it on what it finds
+> there.
 
 ---
 

{qaas_python-0.3.1.dist-info → qaas_python-0.3.2.dist-info}/RECORD

@@ -88,8 +88,8 @@ qaas/prompts/USHER.md,sha256=HfJxJMQym81qHHMy3Tyv5YeeTbvU9iTFWCMgeasyzpM,5144
 qaas/prompts/VAULT.md,sha256=Ansowimx-wMbinQkn9_gGgZFmvmMFMc9v_N8s2Ahws4,2915
 qaas/prompts/WARDEN.md,sha256=39KsbwWRwe7mtjYEik1owBIDgu4qyD52UHG_T_7gTMw,2950
 qaas/prompts/_shared.md,sha256=lN_s_rAmakhGuyRuWItC--iyy-Er6ohT_dzGeE_g-fo,2620
-qaas_python-0.3.1.dist-info/METADATA,sha256=3b2Aykp4GBfpUaqjfC7Se3w8eJzIJzmw4mmdu-bY054,16684
-qaas_python-0.3.1.dist-info/WHEEL,sha256=zOwg4jB6zX2kU910N-cMawjivD6tO8NEWvE12je1bVk,87
-qaas_python-0.3.1.dist-info/entry_points.txt,sha256=6UScfruyhP9N_xGx3tXJGkaoAiB36dkINuKyOH6OkK4,38
-qaas_python-0.3.1.dist-info/licenses/LICENSE,sha256=pHWke5oMtv7PLjIQbN6hRa31J0AKj51VCd5TCTUbbX0,1069
-qaas_python-0.3.1.dist-info/RECORD,,
+qaas_python-0.3.2.dist-info/METADATA,sha256=IgEwLQXmnu1Qs68t5hXgac4qmsuN1mJLvulKj0VzQr8,16455
+qaas_python-0.3.2.dist-info/WHEEL,sha256=zOwg4jB6zX2kU910N-cMawjivD6tO8NEWvE12je1bVk,87
+qaas_python-0.3.2.dist-info/entry_points.txt,sha256=6UScfruyhP9N_xGx3tXJGkaoAiB36dkINuKyOH6OkK4,38
+qaas_python-0.3.2.dist-info/licenses/LICENSE,sha256=pHWke5oMtv7PLjIQbN6hRa31J0AKj51VCd5TCTUbbX0,1069
+qaas_python-0.3.2.dist-info/RECORD,,