memento-mori-jester 0.1.55 → 0.1.57

package/CHANGELOG.md

@@ -4,6 +4,18 @@ All notable changes to Memento Mori Jester are tracked here.
 
 ## Unreleased
 
+## 0.1.57
+
+- Added web, API, infra, and AI preset fixture coverage across the remaining plan, command, and final review-kind gaps.
+- Updated fixture-report regression coverage so every preset now has plan, command, diff, and final examples.
+- Refreshed fixture and demo docs with the expanded 90-fixture corpus.
+
+## 0.1.56
+
+- Added node, python, and security preset fixture coverage across plan, command, diff, and final review kinds.
+- Updated fixture-report regression coverage so those three presets cannot silently fall back to empty preset/kind coverage.
+- Refreshed demo and fixture docs with the expanded 80-fixture corpus.
+
 ## 0.1.55
 
 - Added the first targeted quiet-pass fixture batch for noisy high-signal rules including `risky-domain`, `done-without-evidence`, `package-install-script`, `secret-material`, `sensitive-env-change`, `test-removal`, `skip-tests`, `vibes-based-plan`, `chmod-777`, and `privileged-command`.

package/ROADMAP.md

@@ -6,6 +6,8 @@ Memento Mori Jester is usable today as a CLI, MCP server, GitHub Action, and git
 
 ## Recently Shipped
 
+- Completed preset-kind fixture coverage in v0.1.57 so `default`, `node`, `python`, `web`, `api`, `infra`, `ai`, and `security` now all have plan, command, diff, and final examples.
+- Node, python, and security preset-kind fixture coverage in v0.1.56 so those preset slices now have plan, command, diff, and final examples.
 - Targeted quiet-pass fixture batch in v0.1.55 for noisy high-signal rules, plus quiet-pass evidence in `jester tune` and `npm run fixtures:report`.
 - Fixture coverage report generator in v0.1.54 for rule, preset, review-kind, verdict, and pass-case gaps.
 - Published-package fixture validator fix in v0.1.53 so `npm run fixtures:check` works outside a source checkout.
@@ -44,8 +46,8 @@ Memento Mori Jester is usable today as a CLI, MCP server, GitHub Action, and git
 
 ## Product Ideas
 
+- Add quiet-pass boundaries for thin custom/preset rules that still only have one-sided fixture evidence.
 - Add more framework-specific false-positive examples from real reports so tuning guidance keeps getting sharper.
-- Add preset/kind fixture batches for currently empty `node`, `python`, and `security` coverage slices.
 
 ## Quality And Safety
 

package/docs/DEMO.md

@@ -192,27 +192,28 @@ Project config: none loaded
 Fixture tuning evidence:
 Support: limited
 Confidence: medium
-Total fixtures checked: 68
-Weighted fixtures checked: 131.6
-Matching fixtures: 8
-Weighted matches: 17
-Expected-match weight: 14
+Total fixtures checked: 90
+Weighted fixtures checked: 169.6
+Matching fixtures: 9
+Weighted matches: 19
+Expected-match weight: 16
 Unexpected-match weight: 3
 Edge-case matches: 0
-Quiet-pass fixtures: 4
-Quiet-pass weight: 2.6
-By kind: command 0, plan 3, diff 4, final 1
-Fixture coverage: 8/68 (12.9% weighted)
-By verdict: pass 0, caution 3, block 5
+Quiet-pass fixtures: 5
+Quiet-pass weight: 3.6
+By kind: command 0, plan 4, diff 4, final 1
+Fixture coverage: 9/90 (11.2% weighted)
+By verdict: pass 0, caution 3, block 6
 Matched fixture samples:
   infra-public-ingress-block: Public ingress should block in low-risk-tolerance infra repos.
+  node-plan-production-mode-block: Node production-mode planning should cover node-specific and sensitive-domain signals.
   plan-missing-verification-step: Implementation plan without verification steps should trigger the structural rule.
   sec-secret-material-openai: Hard-coded OpenAI-like token should map to the secret-material rule.
   universal-risky-domain-auth-caution-2: Auth callback changes should keep the broad risky-domain signal covered when verification is present.
-  universal-risky-domain-billing-final: Billing changes in final responses should remain covered when evidence is supplied.
 Quiet-pass fixture samples:
   ai-docs-only-transcript-pass: Docs-only AI setup notes should stay quiet when they do not include concrete dangerous patterns.
   api-docs-only-auth-pass: Docs-only API setup notes should not warn just because they mention auth and production.
+  sec-final-dependency-notes-pass: A verified dependency-note final answer should give the security preset a quiet final case.
   universal-risky-domain-docs-pass: Documentation-only sensitive-domain vocabulary should stay quiet when no code behavior changes.
   web-docs-only-browser-storage-pass: Docs-only web guidance should not warn just because it mentions browser storage or redirects.
 
@@ -352,7 +353,7 @@ Preset packs:
 
 ## 13. Review Fixtures
 
-The fixture suite in `examples/fixtures/preset-review-cases.json` captures small real-usage examples with expected `pass`, `caution`, or `block` verdicts. It also includes quiet-pass `absentRuleIds` examples that prove noisy rules stay silent for safe near-misses. These examples are run by `npm test`, so preset tuning changes stay visible.
+The fixture suite in `examples/fixtures/preset-review-cases.json` captures small real-usage examples with expected `pass`, `caution`, or `block` verdicts. It also includes quiet-pass `absentRuleIds` examples that prove noisy rules stay silent for safe near-misses, plus stack-specific coverage for node, python, and security preset surfaces. These examples are run by `npm test`, so preset tuning changes stay visible.
 
 Maintainers can use `docs/MAINTAINER_TRIAGE.md` to turn useful false-positive reports into redacted fixture cases.
 

package/docs/RELEASE_NOTES_v0.1.56.md

@@ -0,0 +1,50 @@
+# Memento Mori Jester v0.1.56
+
+This release fills the largest preset-kind fixture gap left by the coverage report: `node`, `python`, and `security` now each have plan, command, diff, and final review examples.
+
+## Changes
+
+- Added 12 preset fixtures:
+  - node plan, command, diff, and final cases,
+  - python plan, command, diff, and final cases,
+  - security plan, command, diff, and final cases.
+- Covered stack-specific rules such as:
+  - `custom-node-env-production-change`
+  - `blocked-command-npm-publish-force`
+  - `custom-node-install-script-change`
+  - `custom-python-pickle-load`
+  - `blocked-command-pip-install-break-system-packages`
+  - `custom-python-eval-exec`
+  - `custom-insecure-tls-disabled`
+  - `blocked-command-chmod-r-777`
+  - `custom-broad-cors`
+- Updated fixture report regression tests so `node`, `python`, and `security` cannot silently return to empty preset-kind coverage.
+- Refreshed demo and fixture docs for the 80-fixture corpus.
+
+## Public Interface Changes
+
+- No CLI command, MCP tool, config schema, GitHub Action, release workflow, rule matching, or verdict behavior changed.
+- Fixture evidence changes are data-only: `jester tune` and `fixtures:report` now have more preset-backed examples to report.
+
+## Release Validation
+
+```powershell
+npm.cmd test
+npm.cmd run demo:svg:check
+npm.cmd run fixtures:report
+npm.cmd run fixtures:report -- --json
+npm.cmd run pack:dry
+git diff --check
+node .\dist\cli.js tune risky-domain
+node .\dist\cli.js tune coverage
+git diff | node .\dist\cli.js diff --fail-on block --subject "v0.1.56 stack preset fixture coverage"
+```
+
+## Post-Release Smoke
+
+```powershell
+npm.cmd view memento-mori-jester version --silent
+npx.cmd -y memento-mori-jester@latest doctor
+npx.cmd -y memento-mori-jester@latest tune risky-domain
+npx.cmd -y memento-mori-jester@latest tune coverage
+```

package/docs/RELEASE_NOTES_v0.1.57.md

@@ -0,0 +1,48 @@
+# Memento Mori Jester v0.1.57
+
+This release completes the preset-kind fixture grid left after v0.1.56. Every built-in preset now has plan, command, diff, and final review examples in the fixture suite.
+
+## Changes
+
+- Added 10 preset fixtures:
+  - web plan, command, and final cases,
+  - API plan, command, and final cases,
+  - infra plan and final cases,
+  - AI command and final cases.
+- Covered stack-specific rules such as:
+  - `custom-web-unsafe-html-injection`
+  - `custom-web-public-secret-name`
+  - `custom-api-webhook-signature-disabled`
+  - `blocked-command-prisma-migrate-reset-force`
+  - `custom-infra-iam-wildcard-permission`
+  - `custom-ai-public-provider-key`
+  - `custom-ai-evals-skipped`
+- Updated fixture report regression tests so any future preset-kind gap is caught directly.
+- Refreshed demo and fixture docs for the 90-fixture corpus.
+
+## Public Interface Changes
+
+- No CLI command, MCP tool, config schema, GitHub Action, release workflow, rule matching, or verdict behavior changed.
+- Fixture evidence changes are data-only: `jester tune` and `fixtures:report` now have fuller preset-backed examples to report.
+
+## Release Validation
+
+```powershell
+npm.cmd test
+npm.cmd run demo:svg:check
+npm.cmd run fixtures:report
+npm.cmd run fixtures:report -- --json
+npm.cmd run pack:dry
+git diff --check
+node .\dist\cli.js tune risky-domain --no-config
+node .\dist\cli.js tune coverage --no-config
+git diff | node .\dist\cli.js diff --fail-on block --subject "v0.1.57 remaining preset fixture coverage"
+```
+
+## Post-Release Smoke
+
+```powershell
+npm.cmd view memento-mori-jester version --silent
+npx.cmd -y memento-mori-jester@latest doctor --no-config
+npx.cmd -y memento-mori-jester@latest tune coverage --no-config
+```

package/examples/fixtures/README.md

@@ -9,9 +9,10 @@ Maintainer triage guidance lives in [docs/MAINTAINER_TRIAGE.md](../../docs/MAINT
 ## What They Cover
 
 - Documentation-only diffs that should stay quiet.
+- Plan, command, diff, and final coverage for every built-in preset.
 - Web preset client-exposed key and browser-storage risks.
-- API preset CORS and raw SQL risks.
-- Infra preset public exposure risks.
+- API preset CORS, raw SQL, webhook, and migration-command risks.
+- Infra preset public exposure, IAM wildcard, and destructive command risks.
 - AI preset eval-skipping and model-output execution risks.
 - Fixture metadata like `weight` and `edgeCase` to support precision-weighted tuning evidence.
 

package/examples/fixtures/preset-review-cases.json

@@ -42,6 +42,45 @@
       "configured-sensitive-domain-session"
     ]
   },
+  {
+    "id": "web-unsafe-html-plan-block",
+    "preset": "web",
+    "kind": "plan",
+    "description": "Unsafe HTML injection planning should cover the web preset plan surface.",
+    "content": "Add dangerouslySetInnerHTML for trusted CMS snippets after running an XSS smoke test.",
+    "expectedVerdict": "block",
+    "weight": 2,
+    "expectedRuleIds": [
+      "custom-web-unsafe-html-injection",
+      "configured-sensitive-domain-dangerouslysetinnerhtml",
+      "configured-sensitive-domain-innerhtml"
+    ]
+  },
+  {
+    "id": "web-public-secret-command-block",
+    "preset": "web",
+    "kind": "command",
+    "description": "Client-exposed public secret names should also block on command review surfaces.",
+    "content": "echo NEXT_PUBLIC_PAYMENT_SECRET_KEY",
+    "expectedVerdict": "block",
+    "weight": 2,
+    "expectedRuleIds": [
+      "custom-web-public-secret-name"
+    ]
+  },
+  {
+    "id": "web-final-route-copy-pass",
+    "preset": "web",
+    "kind": "final",
+    "description": "A verified web final answer should give the web preset a quiet final case.",
+    "content": "Updated route copy and ran npm test.",
+    "expectedVerdict": "pass",
+    "weight": 1,
+    "expectedRuleIds": [],
+    "absentRuleIds": [
+      "done-without-evidence"
+    ]
+  },
   {
     "id": "api-docs-only-auth-pass",
     "preset": "api",
@@ -82,6 +121,44 @@
       "custom-api-raw-sql-user-input"
     ]
   },
+  {
+    "id": "api-webhook-signature-plan-block",
+    "preset": "api",
+    "kind": "plan",
+    "description": "Webhook signature bypass planning should cover the API preset plan surface.",
+    "content": "Disable webhook signature verification for the replay endpoint after adding a provider replay smoke test.",
+    "expectedVerdict": "block",
+    "weight": 2,
+    "expectedRuleIds": [
+      "custom-api-webhook-signature-disabled",
+      "configured-sensitive-domain-webhook"
+    ]
+  },
+  {
+    "id": "api-migrate-reset-command-block",
+    "preset": "api",
+    "kind": "command",
+    "description": "Destructive Prisma reset commands should block through the API preset.",
+    "content": "prisma migrate reset --force",
+    "expectedVerdict": "block",
+    "weight": 2,
+    "expectedRuleIds": [
+      "blocked-command-prisma-migrate-reset-force"
+    ]
+  },
+  {
+    "id": "api-final-endpoint-notes-pass",
+    "preset": "api",
+    "kind": "final",
+    "description": "A verified API final answer should give the API preset a quiet final case.",
+    "content": "Updated endpoint notes and ran npm test.",
+    "expectedVerdict": "pass",
+    "weight": 1,
+    "expectedRuleIds": [],
+    "absentRuleIds": [
+      "done-without-evidence"
+    ]
+  },
   {
     "id": "infra-public-ingress-block",
     "preset": "infra",
@@ -107,6 +184,32 @@
       "blocked-command-terraform-destroy"
     ]
   },
+  {
+    "id": "infra-iam-wildcard-plan-block",
+    "preset": "infra",
+    "kind": "plan",
+    "description": "IAM wildcard planning should cover the infra preset plan surface.",
+    "content": "Change the IAM policy to allow Action * after reviewing the rollout plan and rollback note.",
+    "expectedVerdict": "block",
+    "weight": 2,
+    "expectedRuleIds": [
+      "custom-infra-iam-wildcard-permission",
+      "configured-sensitive-domain-iam"
+    ]
+  },
+  {
+    "id": "infra-final-chart-notes-pass",
+    "preset": "infra",
+    "kind": "final",
+    "description": "A verified infra final answer should give the infra preset a quiet final case.",
+    "content": "Updated chart notes and ran npm test.",
+    "expectedVerdict": "pass",
+    "weight": 1,
+    "expectedRuleIds": [],
+    "absentRuleIds": [
+      "done-without-evidence"
+    ]
+  },
   {
     "id": "ai-docs-only-transcript-pass",
     "preset": "ai",
@@ -149,6 +252,31 @@
       "custom-ai-model-output-execution"
     ]
   },
+  {
+    "id": "ai-public-key-command-block",
+    "preset": "ai",
+    "kind": "command",
+    "description": "Client-exposed AI provider key names should also block on command review surfaces.",
+    "content": "echo NEXT_PUBLIC_GEMINI_API_TOKEN",
+    "expectedVerdict": "block",
+    "weight": 2,
+    "expectedRuleIds": [
+      "custom-ai-public-provider-key"
+    ]
+  },
+  {
+    "id": "ai-evals-skipped-final-block",
+    "preset": "ai",
+    "kind": "final",
+    "description": "Skipping evals in final answers should cover the AI preset final surface.",
+    "content": "Skip evals for the prompt update after manual review.",
+    "expectedVerdict": "block",
+    "weight": 2,
+    "expectedRuleIds": [
+      "custom-ai-evals-skipped",
+      "configured-sensitive-domain-eval"
+    ]
+  },
   {
     "id": "sec-secret-material-openai",
     "preset": "default",
@@ -853,5 +981,163 @@
     "absentRuleIds": [
       "privileged-command"
     ]
+  },
+  {
+    "id": "node-plan-production-mode-block",
+    "preset": "node",
+    "kind": "plan",
+    "description": "Node production-mode planning should cover node-specific and sensitive-domain signals.",
+    "content": "Set NODE_ENV=production for the smoke run after npm run build.",
+    "expectedVerdict": "block",
+    "weight": 2,
+    "expectedRuleIds": [
+      "risky-domain",
+      "custom-node-env-production-change",
+      "configured-sensitive-domain-production"
+    ]
+  },
+  {
+    "id": "node-publish-force-command-block",
+    "preset": "node",
+    "kind": "command",
+    "description": "Forced npm publishing should stay blocked by the node preset.",
+    "content": "npm publish --force",
+    "expectedVerdict": "block",
+    "weight": 2,
+    "expectedRuleIds": [
+      "blocked-command-npm-publish-force",
+      "configured-sensitive-domain-npm-publish"
+    ]
+  },
+  {
+    "id": "node-install-script-diff-block",
+    "preset": "node",
+    "kind": "diff",
+    "description": "Node install lifecycle script diffs should cover both built-in and node preset checks.",
+    "content": "diff --git a/package.json b/package.json\n--- a/package.json\n+++ b/package.json\n@@ -3,6 +3,7 @@\n   \"scripts\": {\n+    \"postinstall\": \"node scripts/setup.js\",\n     \"test\": \"node --test\"\n   }\n",
+    "expectedVerdict": "block",
+    "weight": 2,
+    "expectedRuleIds": [
+      "package-install-script",
+      "custom-node-install-script-change",
+      "configured-sensitive-domain-postinstall"
+    ]
+  },
+  {
+    "id": "node-final-package-metadata-pass",
+    "preset": "node",
+    "kind": "final",
+    "description": "A verified node metadata final answer should give the node preset a quiet final case.",
+    "content": "Updated package metadata and ran npm test.",
+    "expectedVerdict": "pass",
+    "weight": 1,
+    "expectedRuleIds": [],
+    "absentRuleIds": [
+      "done-without-evidence"
+    ]
+  },
+  {
+    "id": "python-pickle-plan-block",
+    "preset": "python",
+    "kind": "plan",
+    "description": "Python pickle usage in a plan should cover the python deserialization preset rule.",
+    "content": "Refactor the loader to call pickle.load for trusted fixture data.",
+    "expectedVerdict": "block",
+    "weight": 2,
+    "expectedRuleIds": [
+      "custom-python-pickle-load",
+      "missing-verification-step"
+    ]
+  },
+  {
+    "id": "python-break-system-command-block",
+    "preset": "python",
+    "kind": "command",
+    "description": "Break-system-package installs should stay blocked by the python preset.",
+    "content": "pip install --break-system-packages django",
+    "expectedVerdict": "block",
+    "weight": 2,
+    "expectedRuleIds": [
+      "blocked-command-pip-install-break-system-packages"
+    ]
+  },
+  {
+    "id": "python-eval-diff-caution",
+    "preset": "python",
+    "kind": "diff",
+    "description": "Python eval usage in a diff should cover the dynamic-execution preset rule.",
+    "content": "diff --git a/src/loader.py b/src/loader.py\n--- a/src/loader.py\n+++ b/src/loader.py\n@@ -1 +1,2 @@\n+result = eval(user_value)\n",
+    "expectedVerdict": "caution",
+    "weight": 2,
+    "expectedRuleIds": [
+      "custom-python-eval-exec"
+    ]
+  },
+  {
+    "id": "python-final-pyproject-pass",
+    "preset": "python",
+    "kind": "final",
+    "description": "A verified pyproject metadata final answer should give the python preset a quiet final case.",
+    "content": "Updated pyproject metadata and ran pytest.",
+    "expectedVerdict": "pass",
+    "weight": 1,
+    "expectedRuleIds": [],
+    "absentRuleIds": [
+      "done-without-evidence"
+    ]
+  },
+  {
+    "id": "sec-disable-tls-plan-block",
+    "preset": "security",
+    "kind": "plan",
+    "description": "TLS verification disabling should stay blocked by the security preset.",
+    "content": "Set NODE_TLS_REJECT_UNAUTHORIZED=0 for local debug after documenting the exception.",
+    "expectedVerdict": "block",
+    "weight": 2,
+    "expectedRuleIds": [
+      "custom-insecure-tls-disabled",
+      "configured-sensitive-domain-auth"
+    ]
+  },
+  {
+    "id": "sec-chmod-command-block",
+    "preset": "security",
+    "kind": "command",
+    "description": "Broad chmod in a security preset command should include blocked-command and permission signals.",
+    "content": "chmod -R 777 ./secrets",
+    "expectedVerdict": "block",
+    "weight": 2,
+    "expectedRuleIds": [
+      "blocked-command-chmod-r-777",
+      "chmod-777",
+      "configured-sensitive-domain-secrets"
+    ]
+  },
+  {
+    "id": "sec-broad-cors-diff-block",
+    "preset": "security",
+    "kind": "diff",
+    "description": "Wildcard CORS diffs should cover the security preset broad-CORS rule.",
+    "content": "diff --git a/src/cors.ts b/src/cors.ts\n--- a/src/cors.ts\n+++ b/src/cors.ts\n@@ -1 +1,2 @@\n+app.use(cors({ origin: \"*\" }));\n",
+    "expectedVerdict": "block",
+    "weight": 2,
+    "expectedRuleIds": [
+      "custom-broad-cors",
+      "configured-sensitive-domain-cors"
+    ]
+  },
+  {
+    "id": "sec-final-dependency-notes-pass",
+    "preset": "security",
+    "kind": "final",
+    "description": "A verified dependency-note final answer should give the security preset a quiet final case.",
+    "content": "Updated dependency notes and ran npm test.",
+    "expectedVerdict": "pass",
+    "weight": 1,
+    "expectedRuleIds": [],
+    "absentRuleIds": [
+      "risky-domain",
+      "done-without-evidence"
+    ]
   }
 ]

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "memento-mori-jester",
-  "version": "0.1.55",
+  "version": "0.1.57",
   "description": "A local court-jester sidecar for AI coding agents: review plans, commands, diffs, and final claims before they get too pleased with themselves.",
   "type": "module",
   "repository": {