memento-mori-jester 0.1.64 → 0.1.66

package/CHANGELOG.md

@@ -4,6 +4,18 @@ All notable changes to Memento Mori Jester are tracked here.
 
 ## Unreleased
 
+## 0.1.66
+
+- Added real-world preset fixtures for node, python, infra, and AI slices, growing the corpus to 160 fixtures.
+- Added Kubernetes delete coverage for the infra preset, including a second firing and a read-only near-miss so thin and quiet-pass coverage stay clean.
+- Refreshed demo, roadmap, fixture docs, and release notes for the expanded low-count preset coverage.
+
+## 0.1.65
+
+- Added matched-pass fixtures for low-severity `vibes-based-plan` and `handwave-final` rule boundaries.
+- Added `passEligibleRulesWithoutPassCases` to `npm run fixtures:report -- --json` so curation only asks for pass-case coverage where a matched rule can genuinely remain a pass.
+- Updated fixture report curation to move on from impossible hard-rule pass cases to real-world preset collection, with docs refreshed for the 154-fixture corpus.
+
 ## 0.1.64
 
 - Added second firing fixtures for the remaining built-in destructive-command, final-answer, and configured billing-domain thin examples.

package/README.md

@@ -501,7 +501,7 @@ Use the false-positive template for noisy cautions or blocks. Include `jester su
 
 Maintainers can use [docs/MAINTAINER_TRIAGE.md](docs/MAINTAINER_TRIAGE.md) to turn useful false-positive reports into redacted fixtures.
 Run `npm run fixtures:check` before merging fixture changes; it catches duplicate IDs, missing rule metadata, weak descriptions, unsafe-looking content, and duplicate content.
-Run `npm run fixtures:report` to see fixture coverage by rule, rule family, preset slice, kind, verdict, quiet-pass boundaries, and curation-next guidance before choosing the next fixture.
+Run `npm run fixtures:report` to see fixture coverage by rule, rule family, preset slice, kind, verdict, quiet-pass boundaries, feasible pass-case gaps, and curation-next guidance before choosing the next fixture.
 
 For vulnerabilities, private code exposure, or credential-handling concerns, follow [SECURITY.md](SECURITY.md) instead of opening a public issue with sensitive details.
 

package/ROADMAP.md

@@ -6,6 +6,8 @@ Memento Mori Jester is usable today as a CLI, MCP server, GitHub Action, and git
 
 ## Recently Shipped
 
+- Real-world low-count preset fixture batch in v0.1.66, adding node, python, infra, and AI examples while keeping thin, quiet-pass, and feasible pass-case gaps at zero.
+- Feasible pass-case fixture curation in v0.1.65, adding matched-pass examples for low-severity tone/planning rules and stopping curation from asking for impossible pass cases on hard rules.
 - Final thin-rule fixture precision pass in v0.1.64, clearing all remaining thin coverage gaps across built-in, structural, custom, configured sensitive-domain, and blocked-command rule families.
 - Framework custom-rule fixture precision pass in v0.1.63, clearing custom-rule thin coverage and reducing total thin fixture coverage from 16 rules to 7.
 - AI/API custom-rule fixture precision pass in v0.1.62, reducing total thin fixture coverage from 21 rules to 16 while keeping review behavior unchanged.
@@ -53,7 +55,7 @@ Memento Mori Jester is usable today as a CLI, MCP server, GitHub Action, and git
 
 ## Product Ideas
 
-- Add benign pass-case evidence for rules that currently have match coverage and quiet-pass coverage but no explicit matched pass cases.
+- Collect real-world reports for the next lowest-count preset slices: security, node, python, and web.
 - Add more framework-specific false-positive examples from real reports so tuning guidance keeps getting sharper.
 - Add a Markdown export for fixture reports so maintainers can paste coverage snapshots into issues or release notes.
 

package/docs/DEMO.md

@@ -192,8 +192,8 @@ Project config: none loaded
 Fixture tuning evidence:
 Support: limited
 Confidence: medium
-Total fixtures checked: 152
-Weighted fixtures checked: 294.9
+Total fixtures checked: 160
+Weighted fixtures checked: 305.9
 Matching fixtures: 11
 Weighted matches: 23
 Expected-match weight: 18
@@ -202,7 +202,7 @@ Edge-case matches: 0
 Quiet-pass fixtures: 5
 Quiet-pass weight: 3.6
 By kind: command 0, plan 5, diff 5, final 1
-Fixture coverage: 11/152 (7.8% weighted)
+Fixture coverage: 11/160 (7.5% weighted)
 By verdict: pass 0, caution 3, block 8
 Matched fixture samples:
   infra-public-ingress-block: Public ingress should block in low-risk-tolerance infra repos.
@@ -353,9 +353,9 @@ Preset packs:
 
 ## 13. Review Fixtures
 
-The fixture suite in `examples/fixtures/preset-review-cases.json` captures small real-usage examples with expected `pass`, `caution`, or `block` verdicts. It also includes quiet-pass `absentRuleIds` examples that prove noisy rules stay silent for safe near-misses, stack-specific coverage for every built-in preset, quiet-pass boundaries across built-in, structural, custom, and preset/config-derived rules, second firing examples for preset blocked-command rules, and second examples for AI/API, framework custom, built-in, and configured sensitive-domain rules. These examples are run by `npm test`, so preset tuning changes stay visible.
+The fixture suite in `examples/fixtures/preset-review-cases.json` captures small real-usage examples with expected `pass`, `caution`, or `block` verdicts. It also includes matched-pass examples for low-severity rules, quiet-pass `absentRuleIds` examples that prove noisy rules stay silent for safe near-misses, stack-specific coverage for every built-in preset, quiet-pass boundaries across built-in, structural, custom, and preset/config-derived rules, second firing examples for preset blocked-command rules, second examples for AI/API, framework custom, built-in, and configured sensitive-domain rules, and real-world low-count preset examples. These examples are run by `npm test`, so preset tuning changes stay visible.
 
-Maintainers can run `npm run fixtures:report` to see coverage by verdict, kind, preset, rule family, and preset slice. The report also includes a `Curation next` section that points at the next useful fixture batch, such as thin rules, no-pass evidence, rule-family gaps, or lower-count presets.
+Maintainers can run `npm run fixtures:report` to see coverage by verdict, kind, preset, rule family, and preset slice. The report also includes a `Curation next` section that points at the next useful fixture batch, such as thin rules, feasible pass-case evidence, rule-family gaps, or lower-count presets.
 
 Maintainers can use `docs/MAINTAINER_TRIAGE.md` to turn useful false-positive reports into redacted fixture cases.
 

package/docs/MAINTAINER_TRIAGE.md

@@ -83,7 +83,7 @@ node .\dist\cli.js tune coverage
 ```
 
 5. Fix any duplicate IDs, missing expected rule metadata, weak descriptions, unsafe content, or duplicate content reported by `fixtures:check`.
-6. Use `fixtures:report` to check whether the change improves pass-case, quiet-pass, preset, kind, rule-family, or verdict coverage. Start with the report's `Curation next` section when deciding which fixture batch to add first.
+6. Use `fixtures:report` to check whether the change improves feasible pass-case, quiet-pass, preset, kind, rule-family, or verdict coverage. Start with the report's `Curation next` section when deciding which fixture batch to add first.
 7. Check whether support/confidence changed in the expected direction.
 8. If the fixture changes verdict behavior, mention the exact rule impact in `CHANGELOG.md`.
 

package/docs/PRODUCTION_READINESS.md

@@ -53,7 +53,7 @@ This checklist defines what "production grade" means for Memento Mori Jester rig
 - `SECURITY.md` routes vulnerability reports away from public issues and asks for redacted diagnostics.
 - `docs/MAINTAINER_TRIAGE.md` explains how to turn useful false-positive reports into fixture coverage before changing rule logic.
 - `npm run fixtures:check` validates fixture IDs, metadata, unsafe-looking content, duplicate content, and explicit expected/absent rule intent.
-- `npm run fixtures:report` shows fixture coverage by rule, rule family, preset slice, kind, verdict, and quiet-pass rule boundaries so maintainers can pick the next fixture target.
+- `npm run fixtures:report` shows fixture coverage by rule, rule family, preset slice, kind, verdict, quiet-pass rule boundaries, and feasible pass-case gaps so maintainers can pick the next fixture target.
 - npm publish has a manual workflow fallback, but the normal release path is tag-driven trusted publishing.
 
 ## Static Guard
@@ -74,5 +74,5 @@ This checklist defines what "production grade" means for Memento Mori Jester rig
 
 ## Known Next Gaps
 
-- Continue expanding pass-case and quiet-pass fixtures from real-world usage so false-positive tuning remains evidence-based.
+- Continue expanding real-world preset fixtures and false-positive examples so tuning remains evidence-based.
 - Add more framework-specific false-positive examples as people report real noisy cases.

package/docs/RELEASE_NOTES_v0.1.65.md

@@ -0,0 +1,35 @@
+# Memento Mori Jester v0.1.65
+
+This release makes fixture curation more precise. The report no longer asks maintainers to add impossible matched-pass examples for hard-block or high-severity rules. It also adds real matched-pass examples for the two low-severity rules that still needed them.
+
+## What Changed
+
+- Added 2 fixture cases, growing the corpus from 152 to 154 fixtures.
+- Added matched-pass examples for:
+  - `vibes-based-plan`
+  - `handwave-final`
+- Added `gaps.passEligibleRulesWithoutPassCases` to `npm run fixtures:report -- --json`.
+- Updated `Curation next` so pass-case coverage only appears when pass-eligible low-severity rules still need examples.
+- With the new fixtures, feasible pass-case gaps are now empty and curation moves to real-world preset collection.
+
+## Public Interface
+
+- No CLI command changes.
+- No config schema changes.
+- No rule matching, scoring, or verdict behavior changes.
+- No MCP, playground, GitHub Action, or npm publishing changes.
+- Fixture report JSON includes one additional stable gap field: `passEligibleRulesWithoutPassCases`.
+
+## Release Validation
+
+```powershell
+npm.cmd test
+npm.cmd run demo:svg:check
+npm.cmd run fixtures:report
+npm.cmd run fixtures:report -- --json
+npm.cmd run pack:dry
+git diff --check
+node .\dist\cli.js tune coverage --no-config
+node .\dist\cli.js tune risky-domain --json --no-config
+git diff | node .\dist\cli.js diff --fail-on block --subject "v0.1.65 feasible pass-case curation"
+```

package/docs/RELEASE_NOTES_v0.1.66.md

@@ -0,0 +1,37 @@
+# Memento Mori Jester v0.1.66
+
+This release follows the fixture report's real-world preset curation guidance. It adds practical examples for low-count node, python, infra, and AI slices while preserving the cleaned-up coverage baseline from v0.1.64 and v0.1.65.
+
+## What Changed
+
+- Added 6 fixture cases, growing the corpus from 154 to 160 fixtures.
+- Added quiet real-world examples for:
+  - Node focused test commands.
+  - Python focused pytest commands.
+  - AI retrieved-context schema validation plans.
+- Added infra Kubernetes delete coverage for:
+  - `blocked-command-kubectl-delete`
+  - `custom-infra-production-change`
+- Added a read-only `kubectl get` near-miss so the new infra rules immediately have quiet-pass boundary evidence.
+- Kept thin rule coverage, quiet-pass gaps, and feasible pass-case gaps at zero.
+
+## Public Interface
+
+- No CLI command changes.
+- No config schema changes.
+- No rule matching, scoring, or verdict behavior changes.
+- No MCP, playground, GitHub Action, or npm publishing changes.
+
+## Release Validation
+
+```powershell
+npm.cmd test
+npm.cmd run demo:svg:check
+npm.cmd run fixtures:report
+npm.cmd run fixtures:report -- --json
+npm.cmd run pack:dry
+git diff --check
+node .\dist\cli.js tune coverage --no-config
+node .\dist\cli.js tune risky-domain --json --no-config
+git diff | node .\dist\cli.js diff --fail-on block --subject "v0.1.66 low-count preset fixtures"
+```

package/examples/fixtures/README.md

@@ -16,10 +16,12 @@ Maintainer triage guidance lives in [docs/MAINTAINER_TRIAGE.md](../../docs/MAINT
 - AI preset eval-skipping and model-output execution risks.
 - Quiet-pass boundaries for thin custom, configured sensitive-domain, and preset blocked-command rules.
 - Quiet-pass boundaries for built-in and structural rules such as missing verification, TypeScript suppressions, large removals, wildcard operations, destructive commands, and untested finals.
+- Matched-pass examples for low-severity rules where a single finding should stay below caution.
 - Second firing examples for preset blocked-command rules and high-value stack-specific sensitive-domain rules.
 - Second firing examples for AI and API custom rules around provider keys, model-output execution, raw SQL, and webhook signature checks.
 - Second firing examples for remaining framework custom rules across security, infra, node, python, and web presets.
 - Second firing examples for remaining built-in and configured sensitive-domain thin rules, leaving no thin coverage gaps.
+- Real-world low-count preset examples for node, python, infra, and AI workflows.
 - Fixture metadata like `weight` and `edgeCase` to support precision-weighted tuning evidence.
 
 ## Local Check
@@ -52,6 +54,6 @@ Do not add secrets, private code, customer data, complete logs, or machine-speci
 
 `npm run fixtures:check` validates duplicate IDs, missing expected rule metadata, weak descriptions, unsafe-looking fixture content, and duplicate content before the fixture suite becomes tuning evidence.
 
-`npm run fixtures:report` summarizes coverage by rule, rule family, preset slice, review kind, verdict, and quiet-pass rule boundaries. Use it to find rules without pass-case coverage, rules without quiet-pass coverage, thin rule coverage, preset/kind gaps, quiet pass fixtures, and the next curation target.
+`npm run fixtures:report` summarizes coverage by rule, rule family, preset slice, review kind, verdict, and quiet-pass rule boundaries. Use it to find rules without pass-case coverage, pass-eligible rules without pass-case coverage, rules without quiet-pass coverage, thin rule coverage, preset/kind gaps, quiet pass fixtures, and the next curation target.
 
-The `Curation next` section is a maintainer shortcut: start there when deciding whether the next fixture batch should focus on thin rules, no-pass evidence, a specific rule family, or lower-count presets. The `--json` output includes the same `ruleFamilySlices`, `presetSlices`, and `curationNext` fields for scripts.
+The `Curation next` section is a maintainer shortcut: start there when deciding whether the next fixture batch should focus on thin rules, feasible pass-case evidence, a specific rule family, or lower-count presets. The `--json` output includes the same `ruleFamilySlices`, `presetSlices`, `passEligibleRulesWithoutPassCases`, and `curationNext` fields for scripts.

package/examples/fixtures/preset-review-cases.json

@@ -557,6 +557,18 @@
       "vibes-based-plan"
     ]
   },
+  {
+    "id": "plan-vibes-low-severity-pass",
+    "preset": "default",
+    "kind": "plan",
+    "description": "A lone vague phrase should be recorded as a low-severity matched pass.",
+    "content": "This should work.",
+    "weight": 1,
+    "expectedVerdict": "pass",
+    "expectedRuleIds": [
+      "vibes-based-plan"
+    ]
+  },
   {
     "id": "plan-skip-tests",
     "preset": "default",
@@ -656,6 +668,18 @@
       "handwave-final"
     ]
   },
+  {
+    "id": "final-handwave-low-severity-pass",
+    "preset": "default",
+    "kind": "final",
+    "description": "A lone hand-wavy phrase should be recorded as a low-severity matched pass.",
+    "content": "Looks good.",
+    "expectedVerdict": "pass",
+    "weight": 1,
+    "expectedRuleIds": [
+      "handwave-final"
+    ]
+  },
   {
     "id": "diff-test-removal-caution",
     "preset": "default",
@@ -1353,6 +1377,46 @@
       "blocked-command-terraform-destroy"
     ]
   },
+  {
+    "id": "infra-kubectl-delete-command-block",
+    "preset": "infra",
+    "kind": "command",
+    "description": "Kubernetes delete commands should block through the infra preset blocked-command path.",
+    "content": "kubectl delete deployment checkout-api -n staging",
+    "expectedVerdict": "block",
+    "weight": 2,
+    "expectedRuleIds": [
+      "blocked-command-kubectl-delete",
+      "custom-infra-production-change"
+    ]
+  },
+  {
+    "id": "infra-kubectl-delete-job-command-block",
+    "preset": "infra",
+    "kind": "command",
+    "description": "A second Kubernetes delete command should keep kubectl delete coverage from being thin.",
+    "content": "kubectl delete job stale-worker -n staging",
+    "expectedVerdict": "block",
+    "weight": 2,
+    "expectedRuleIds": [
+      "blocked-command-kubectl-delete",
+      "custom-infra-production-change"
+    ]
+  },
+  {
+    "id": "infra-kubectl-get-command-pass",
+    "preset": "infra",
+    "kind": "command",
+    "description": "Read-only Kubernetes inspection should not trip infra delete/apply checks.",
+    "content": "kubectl get deployment checkout-api -n staging",
+    "expectedVerdict": "pass",
+    "weight": 2,
+    "expectedRuleIds": [],
+    "absentRuleIds": [
+      "blocked-command-kubectl-delete",
+      "custom-infra-production-change"
+    ]
+  },
   {
     "id": "sec-chmod-recursive-secrets-command-block",
     "preset": "security",
@@ -1602,6 +1666,20 @@
       "configured-sensitive-domain-production"
     ]
   },
+  {
+    "id": "node-test-run-command-pass",
+    "preset": "node",
+    "kind": "command",
+    "description": "Focused Node test commands should give the node preset another realistic quiet command.",
+    "content": "node --test test/cli.test.js",
+    "expectedVerdict": "pass",
+    "weight": 1,
+    "expectedRuleIds": [],
+    "absentRuleIds": [
+      "blocked-command-npm-publish-force",
+      "custom-node-install-script-change"
+    ]
+  },
   {
     "id": "python-json-loader-plan-pass",
     "preset": "python",
@@ -1628,6 +1706,20 @@
       "custom-python-eval-exec"
     ]
   },
+  {
+    "id": "python-pytest-command-pass",
+    "preset": "python",
+    "kind": "command",
+    "description": "Focused pytest commands should give the python preset another realistic quiet command.",
+    "content": "python -m pytest tests/test_loader.py",
+    "expectedVerdict": "pass",
+    "weight": 1,
+    "expectedRuleIds": [],
+    "absentRuleIds": [
+      "blocked-command-pip-install-break-system-packages",
+      "custom-python-eval-exec"
+    ]
+  },
   {
     "id": "web-safe-render-plan-pass",
     "preset": "web",
@@ -1768,6 +1860,20 @@
       "configured-sensitive-domain-eval"
     ]
   },
+  {
+    "id": "ai-retrieved-context-schema-plan-pass",
+    "preset": "ai",
+    "kind": "plan",
+    "description": "Schema-checking retrieved context should give the AI preset another realistic quiet plan.",
+    "content": "Validate retrieved context against a schema before allowlisted dispatch.",
+    "expectedVerdict": "pass",
+    "weight": 1,
+    "expectedRuleIds": [],
+    "absentRuleIds": [
+      "custom-ai-user-controlled-tool-dispatch",
+      "custom-ai-model-output-execution"
+    ]
+  },
   {
     "id": "sec-tls-verification-plan-pass",
     "preset": "security",

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "memento-mori-jester",
-  "version": "0.1.64",
+  "version": "0.1.66",
   "description": "A local court-jester sidecar for AI coding agents: review plans, commands, diffs, and final claims before they get too pleased with themselves.",
   "type": "module",
   "repository": {

package/scripts/report-fixtures.mjs

@@ -13,6 +13,16 @@ const structuralRuleIds = new Set([
   "missing-verification-step",
   "wildcard-file-operation"
 ]);
+const passEligibleRuleIds = new Set([
+  "confidence-theater",
+  "console-log",
+  "handwave-final",
+  "large-removal",
+  "missing-verification-step",
+  "temporary-marker",
+  "vibes-based-plan",
+  "wildcard-file-operation"
+]);
 const ruleFamilyOrder = [
   "built-in",
   "structural",
@@ -173,6 +183,10 @@ function buildFixtureReport(rawFixtures) {
       .filter((entry) => entry.passCases === 0)
       .sort((a, b) => b.total - a.total || a.ruleId.localeCompare(b.ruleId))
       .map(ruleGapSummary),
+    passEligibleRulesWithoutPassCases: ruleSummaries
+      .filter((entry) => passEligibleRuleIds.has(entry.ruleId) && entry.passCases === 0)
+      .sort((a, b) => b.total - a.total || a.ruleId.localeCompare(b.ruleId))
+      .map(ruleGapSummary),
     rulesWithoutQuietPassCoverage: ruleSummaries
       .filter((entry) => entry.quietPassCases === 0)
       .sort((a, b) => b.total - a.total || a.ruleId.localeCompare(b.ruleId))
@@ -232,6 +246,8 @@ function renderFixtureReport(report) {
   );
 
   lines.push(...formatRuleGaps(report.gaps.rulesWithoutPassCases));
+  lines.push("", "Pass-eligible rules without pass-case coverage:");
+  lines.push(...formatRuleGaps(report.gaps.passEligibleRulesWithoutPassCases));
   lines.push("", "Rules without quiet-pass coverage:");
   lines.push(...formatRuleGaps(report.gaps.rulesWithoutQuietPassCoverage));
   lines.push("", "Quiet-pass rule coverage:");
@@ -467,13 +483,13 @@ function buildCurationNext(gaps, ruleFamilySlices, presetSlices) {
     });
   }
 
-  if (gaps.rulesWithoutPassCases.length > 0) {
+  if (gaps.passEligibleRulesWithoutPassCases.length > 0) {
     items.push({
       priority: "medium",
       area: "pass-case-coverage",
-      title: "Add benign or docs-only examples for rules with no pass-case evidence",
-      count: gaps.rulesWithoutPassCases.length,
-      ruleIds: gaps.rulesWithoutPassCases.slice(0, 8).map((entry) => entry.ruleId)
+      title: "Add benign matched examples for low-severity rules with no pass-case evidence",
+      count: gaps.passEligibleRulesWithoutPassCases.length,
+      ruleIds: gaps.passEligibleRulesWithoutPassCases.slice(0, 8).map((entry) => entry.ruleId)
     });
   }