@bugzy-ai/bugzy 1.17.0 → 1.18.0

package/dist/cli/index.js

@@ -6053,7 +6053,19 @@ After analyzing test results, triage each failure to determine if it's a product
 
 **IMPORTANT: Do NOT report bugs without triaging first.**
 
-For each failed test:
+### 1. Check Failure Classification
+
+**Before triaging any failure**, read \`new_failures\` from the latest \`test-runs/*/manifest.json\`:
+
+| \`new_failures\` State | Action |
+|------------------------|--------|
+| Non-empty array | Only triage failures listed in \`new_failures\`. Do not investigate, fix, or create issues for \`known_failures\`. |
+| Empty array | No new failures to triage. Output "0 new failures to triage" and skip the rest of this step. |
+| Field missing | Fall back: triage all failed tests (backward compatibility with older reporter versions). |
+
+### 2. Triage Each Failure
+
+For each failed test (from \`new_failures\` or all failures if field is missing):
 
 1. **Read failure details** from JSON report (error message, stack trace)
 2. **Classify the failure:**
@@ -6082,14 +6094,22 @@ For each failed test:
 - Broken navigation flows
 - Validation not working as expected
 
-**Document Classification:**
+### 3. Document Results
+
 \`\`\`markdown
-### Failure Triage
+### Failure Triage Summary
+
+**New failures triaged: N** | **Known failures skipped: M**
 
 | Test ID | Test Name | Classification | Reason |
 |---------|-----------|---------------|--------|
 | TC-001 | Login test | TEST ISSUE | Selector brittle - uses CSS instead of role |
 | TC-002 | Checkout | PRODUCT BUG | 500 error on form submit |
+
+#### Skipped Known Failures
+| Test ID | Test Name | Last Passed Run |
+|---------|-----------|-----------------|
+| TC-003 | Search | 20260210-103045 |
 \`\`\``,
   tags: ["execution", "triage", "analysis"]
 };
@@ -6658,10 +6678,36 @@ npx tsx reporters/parse-results.ts --input <file-or-url> [--timestamp <existing>
         }
       ]
     }
+  ],
+  "new_failures": [
+    {
+      "id": "<test case id>",
+      "name": "<test name>",
+      "error": "<error message or null>",
+      "lastPassedRun": "<timestamp of last passing run or null>"
+    }
+  ],
+  "known_failures": [
+    {
+      "id": "<test case id>",
+      "name": "<test name>",
+      "error": "<error message or null>",
+      "lastPassedRun": null
+    }
   ]
 }
 \`\`\`
-4. For each failed test, create:
+4. **Classify failures** \u2014 after building the manifest, classify each failed test as new or known:
+   - Read \`BUGZY_FAILURE_LOOKBACK\` env var (default: 5)
+   - List previous \`test-runs/*/manifest.json\` files sorted by timestamp descending (skip current run)
+   - For each failed test in the manifest:
+     - If it passed in any of the last N runs \u2192 \`new_failures\` (include the timestamp of the last passing run in \`lastPassedRun\`)
+     - If it failed in ALL of the last N runs \u2192 \`known_failures\`
+     - If the test doesn't exist in any previous run \u2192 \`new_failures\` (new test)
+     - If no previous runs exist at all (first run) \u2192 all failures go to \`new_failures\`
+   - Write the \`new_failures\` and \`known_failures\` arrays into the manifest
+
+5. For each failed test, create:
    - Directory: \`test-runs/{timestamp}/{testCaseId}/exec-1/\`
    - File: \`test-runs/{timestamp}/{testCaseId}/exec-1/result.json\` containing:
 \`\`\`json
@@ -6673,8 +6719,8 @@ npx tsx reporters/parse-results.ts --input <file-or-url> [--timestamp <existing>
   "testFile": "<file path if available>"
 }
 \`\`\`
-5. Print the manifest path to stdout
-6. Exit code 0 on success, non-zero on failure
+6. Print the manifest path to stdout
+7. Exit code 0 on success, non-zero on failure
 
 **Incremental mode** (\`--timestamp\` + \`--test-id\` provided):
 1. Read existing \`test-runs/{timestamp}/manifest.json\`