memento-mori-jester 0.1.81 → 0.1.83
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +12 -0
- package/README.md +4 -0
- package/ROADMAP.md +4 -2
- package/docs/CLI.md +2 -0
- package/docs/DEMO.md +23 -9
- package/docs/FRAMEWORK_TUNING.md +29 -0
- package/docs/GETTING_STARTED.md +1 -1
- package/docs/PRODUCTION_READINESS.md +3 -0
- package/docs/RELEASE.md +2 -1
- package/docs/RELEASE_NOTES_v0.1.82.md +54 -0
- package/docs/RELEASE_NOTES_v0.1.83.md +54 -0
- package/examples/fixtures/README.md +1 -0
- package/examples/fixtures/preset-review-cases.json +91 -0
- package/examples/tuning/README.md +23 -0
- package/examples/tuning/framework-tuning-cookbook.json +96 -0
- package/package.json +3 -2
- package/scripts/check-framework-tuning.mjs +214 -0
- package/scripts/check-production-readiness.mjs +23 -0
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,18 @@ All notable changes to Memento Mori Jester are tracked here.
|
|
|
4
4
|
|
|
5
5
|
## Unreleased
|
|
6
6
|
|
|
7
|
+
## 0.1.83
|
|
8
|
+
|
|
9
|
+
- Added `examples/tuning/framework-tuning-cookbook.json`, a small checked cookbook that maps framework-shaped noisy-rule reports to exact `jester tune <rule> --json` commands and fixture IDs.
|
|
10
|
+
- Added `examples/tuning/README.md` and linked it from README, CLI docs, getting-started docs, demo docs, and the framework tuning guide.
|
|
11
|
+
- Added `npm run framework:tuning:check` and wired it into `npm test` and production-readiness checks so cookbook recipes stay aligned with docs and fixtures.
|
|
12
|
+
|
|
13
|
+
## 0.1.82
|
|
14
|
+
|
|
15
|
+
- Added six real-world quiet-pass fixtures for FastAPI dependency injection, frozen `uv` syncs, docs-only Terraform and Helm guidance, redacted Gitleaks scans, and Next.js workspace linting.
|
|
16
|
+
- Added `docs/FRAMEWORK_TUNING.md` to map common framework-specific false-positive reports to useful `jester tune <rule>` commands and fixture IDs.
|
|
17
|
+
- Refreshed demo, fixture docs, roadmap, and release notes for the 222-fixture corpus.
|
|
18
|
+
|
|
7
19
|
## 0.1.81
|
|
8
20
|
|
|
9
21
|
- Added a lightweight repo-local landing page under `site/index.html` that reuses the social card, demo video, start command, and public project links.
|
package/README.md
CHANGED
|
@@ -315,6 +315,8 @@ jester tune coverage --json
|
|
|
315
315
|
|
|
316
316
|
`jester tune coverage` shows the fixture support and confidence signal for every rule, including suggested next actions such as adding coverage, reviewing surprise matches, checking quiet-pass boundaries, or leaving a healthy signal alone.
|
|
317
317
|
|
|
318
|
+
For stack-shaped noise, see [Framework Tuning Examples](docs/FRAMEWORK_TUNING.md). It maps common Next.js, Vite React, FastAPI, Terraform/Kubernetes, security-scan, and AI/MCP false-positive reports to the `jester tune <rule>` command and fixture IDs worth checking first. The checked [framework tuning cookbook](examples/tuning) turns those rows into copy-paste recipes and a machine-readable JSON file.
|
|
319
|
+
|
|
318
320
|
Disable a noisy rule by adding its id to `disabledRules` in `jester.config.json`:
|
|
319
321
|
|
|
320
322
|
```json
|
|
@@ -426,6 +428,8 @@ More setup examples:
|
|
|
426
428
|
- [Agent Setup](docs/AGENTS.md)
|
|
427
429
|
- [MCP Tool Reference](docs/MCP_TOOLS.md)
|
|
428
430
|
- [GitHub Actions](docs/GITHUB_ACTIONS.md)
|
|
431
|
+
- [Framework Tuning Examples](docs/FRAMEWORK_TUNING.md)
|
|
432
|
+
- [Framework Tuning Cookbook](examples/tuning)
|
|
429
433
|
- [Demo Script](docs/DEMO.md)
|
|
430
434
|
- [Promo Share Kit](promo)
|
|
431
435
|
- [Examples](examples)
|
package/ROADMAP.md
CHANGED
|
@@ -6,6 +6,8 @@ Memento Mori Jester is usable today as a CLI, MCP server, GitHub Action, and git
|
|
|
6
6
|
|
|
7
7
|
## Recently Shipped
|
|
8
8
|
|
|
9
|
+
- Framework tuning cookbook in v0.1.83, adding checked copy-paste recipes and a machine-readable JSON map for stack-shaped noisy-rule reports.
|
|
10
|
+
- Framework tuning examples and quiet-pass fixture curation in v0.1.82, adding six safe real-world examples plus a guide for framework-shaped noisy-rule reports.
|
|
9
11
|
- Repo-local landing page in v0.1.81, adding a static one-page share surface plus deterministic link checks.
|
|
10
12
|
- Social preview card in v0.1.80, adding a deterministic 1200x630 promo card plus generation and freshness checks.
|
|
11
13
|
- Promo freshness check in v0.1.79, verifying the current demo video, share-kit stills, docs, and fixture evidence numbers before public posting.
|
|
@@ -70,8 +72,8 @@ Memento Mori Jester is usable today as a CLI, MCP server, GitHub Action, and git
|
|
|
70
72
|
|
|
71
73
|
## Product Ideas
|
|
72
74
|
|
|
73
|
-
- Collect real-world reports
|
|
74
|
-
- Add
|
|
75
|
+
- Collect real-world reports and fold the strongest redacted cases into more framework tuning cookbook recipes.
|
|
76
|
+
- Add a small fixture issue template checklist that asks for the nearest framework tuning cookbook recipe and redacted `jester tune <rule-id> --json` output.
|
|
75
77
|
- Add a hosted-page option or GitHub Pages instructions once the static page has settled.
|
|
76
78
|
|
|
77
79
|
## Quality And Safety
|
package/docs/CLI.md
CHANGED
|
@@ -158,6 +158,8 @@ Use `jester tune <id>` when the question is practical: should this noisy rule be
|
|
|
158
158
|
|
|
159
159
|
When filing a false-positive issue, include redacted `jester summary` output and `jester tune <rule-id> --json` output when possible.
|
|
160
160
|
|
|
161
|
+
For stack-shaped reports, see [Framework Tuning Examples](FRAMEWORK_TUNING.md). It points common Next.js, Vite React, FastAPI, Terraform/Kubernetes, security-scan, and AI/MCP noisy-rule reports at the relevant `jester tune <rule-id>` command and fixture IDs. The checked [framework tuning cookbook](../examples/tuning) keeps copy-paste recipes and fixture IDs in one machine-readable place.
|
|
162
|
+
|
|
161
163
|
Use `jester tune coverage` when maintaining the rule set. It ranks every rule by fixture support and confidence, shows expected vs unexpected fixture weight, and suggests the next maintenance action for each rule.
|
|
162
164
|
|
|
163
165
|
`jester tune` now also includes fixture evidence:
|
package/docs/DEMO.md
CHANGED
|
@@ -209,17 +209,17 @@ Before muting:
|
|
|
209
209
|
Fixture tuning evidence:
|
|
210
210
|
Support: limited
|
|
211
211
|
Confidence: medium
|
|
212
|
-
Total fixtures checked:
|
|
213
|
-
Weighted fixtures checked:
|
|
212
|
+
Total fixtures checked: 222
|
|
213
|
+
Weighted fixtures checked: 421.8
|
|
214
214
|
Matching fixtures: 11
|
|
215
215
|
Weighted matches: 23
|
|
216
216
|
Expected-match weight: 18
|
|
217
217
|
Unexpected-match weight: 5
|
|
218
218
|
Edge-case matches: 0
|
|
219
|
-
Quiet-pass fixtures:
|
|
220
|
-
Quiet-pass weight:
|
|
219
|
+
Quiet-pass fixtures: 8
|
|
220
|
+
Quiet-pass weight: 5.55
|
|
221
221
|
By kind: command 0, plan 5, diff 5, final 1
|
|
222
|
-
Fixture coverage: 11/
|
|
222
|
+
Fixture coverage: 11/222 (5.5% weighted)
|
|
223
223
|
By verdict: pass 0, caution 3, block 8
|
|
224
224
|
Matched fixture samples:
|
|
225
225
|
infra-public-ingress-block: Public ingress should block in low-risk-tolerance infra repos.
|
|
@@ -230,9 +230,9 @@ Matched fixture samples:
|
|
|
230
230
|
Quiet-pass fixture samples:
|
|
231
231
|
ai-docs-only-transcript-pass: Docs-only AI setup notes should stay quiet when they do not include concrete dangerous patterns.
|
|
232
232
|
api-docs-only-auth-pass: Docs-only API setup notes should not warn just because they mention auth and production.
|
|
233
|
+
infra-helm-values-docs-pass: Docs-only Helm values guidance should stay quiet around infra sensitive-domain warnings.
|
|
234
|
+
infra-terraform-plan-docs-pass: Docs-only Terraform plan review guidance should not trip infra sensitive-domain warnings.
|
|
233
235
|
sec-final-dependency-notes-pass: A verified dependency-note final answer should give the security preset a quiet final case.
|
|
234
|
-
universal-risky-domain-docs-pass: Documentation-only sensitive-domain vocabulary should stay quiet when no code behavior changes.
|
|
235
|
-
web-docs-only-browser-storage-pass: Docs-only web guidance should not warn just because it mentions browser storage or redirects.
|
|
236
236
|
|
|
237
237
|
Commands:
|
|
238
238
|
jester rule risky-domain
|
|
@@ -367,12 +367,26 @@ Preset packs:
|
|
|
367
367
|
|
|
368
368
|
## 13. Review Fixtures
|
|
369
369
|
|
|
370
|
-
The fixture suite in `examples/fixtures/preset-review-cases.json` captures small real-usage examples with expected `pass`, `caution`, or `block` verdicts. It also includes matched-pass examples for low-severity rules, quiet-pass `absentRuleIds` examples that prove noisy rules stay silent for safe near-misses, stack-specific coverage for every built-in preset, quiet-pass boundaries across built-in, structural, custom, and preset/config-derived rules, second firing examples for preset blocked-command rules, second examples for AI/API, framework custom, built-in, and configured sensitive-domain rules, AI tool-dispatch examples with safe allowlist/schema boundaries, and real-world low-count preset examples across node, python, web, infra, AI, and security slices. Recent quiet-pass examples cover typechecks, prebuild scripts, mypy, dataclass parsing, CodeQL, Dependabot limits, form validation, accessibility copy, read-only Kubernetes inspection, Docker disk usage, Terraform linting, public-IP hardening changes, npm audit/outdated/ci, development-mode Node commands, package export maps, workspace test scripts, Bandit, pip-audit, coverage/pytest, Trivy filesystem scans, npm audit, TLS verification-enabled diffs, safe text rendering, allowlisted target paths, public analytics IDs, model-check commands, tool allowlist checks, public model-name config, API schema parsing, query-builder filters, enabled rate limiting, read-only Prisma migration diffs, signed-webhook docs, OpenAPI schema docs, Pydantic parsing, Pyright checks, SBOM generation, vulnerability-report docs, escaped React rendering, session-cookie docs, model regression checks,
|
|
370
|
+
The fixture suite in `examples/fixtures/preset-review-cases.json` captures small real-usage examples with expected `pass`, `caution`, or `block` verdicts. It also includes matched-pass examples for low-severity rules, quiet-pass `absentRuleIds` examples that prove noisy rules stay silent for safe near-misses, stack-specific coverage for every built-in preset, quiet-pass boundaries across built-in, structural, custom, and preset/config-derived rules, second firing examples for preset blocked-command rules, second examples for AI/API, framework custom, built-in, and configured sensitive-domain rules, AI tool-dispatch examples with safe allowlist/schema boundaries, and real-world low-count preset examples across node, python, web, infra, AI, and security slices. Recent quiet-pass examples cover typechecks, prebuild scripts, mypy, dataclass parsing, CodeQL, Dependabot limits, form validation, accessibility copy, read-only Kubernetes inspection, Docker disk usage, Terraform linting, public-IP hardening changes, npm audit/outdated/ci, development-mode Node commands, package export maps, workspace test scripts, Bandit, pip-audit, coverage/pytest, Trivy filesystem scans, npm audit, TLS verification-enabled diffs, safe text rendering, allowlisted target paths, public analytics IDs, model-check commands, tool allowlist checks, public model-name config, API schema parsing, query-builder filters, enabled rate limiting, read-only Prisma migration diffs, signed-webhook docs, OpenAPI schema docs, Pydantic parsing, Pyright checks, SBOM generation, vulnerability-report docs, escaped React rendering, session-cookie docs, model regression checks, static action allowlists, FastAPI dependency injection, frozen `uv` syncs, docs-only Terraform and Helm guidance, redacted Gitleaks scans, and Next.js workspace linting. These examples are run by `npm test`, so preset tuning changes stay visible.
|
|
371
371
|
|
|
372
372
|
Maintainers can run `npm run fixtures:report` to see coverage by verdict, kind, preset, rule family, and preset slice. The report also includes a `Curation next` section that points at the next useful fixture batch, such as thin rules, feasible pass-case evidence, rule-family gaps, or lower-count presets. Use `npm run fixtures:report -- --markdown` for a paste-ready version of the same snapshot.
|
|
373
373
|
|
|
374
374
|
Maintainers can use `docs/MAINTAINER_TRIAGE.md` to turn useful false-positive reports into redacted fixture cases.
|
|
375
375
|
|
|
376
|
-
## 14. Framework
|
|
376
|
+
## 14. Framework Tuning Cookbook
|
|
377
|
+
|
|
378
|
+
For real repos with stack-shaped noisy rules, use [docs/FRAMEWORK_TUNING.md](FRAMEWORK_TUNING.md) and the checked cookbook in [examples/tuning](../examples/tuning).
|
|
379
|
+
|
|
380
|
+
The cookbook maps recipe IDs such as `next-vite-public-config`, `terraform-kubernetes-docs-only`, and `ai-mcp-tooling` to the exact `jester tune <rule-id> --json` commands and fixture IDs worth comparing first.
|
|
381
|
+
|
|
382
|
+
Maintainers can run:
|
|
383
|
+
|
|
384
|
+
```powershell
|
|
385
|
+
npm.cmd run framework:tuning:check
|
|
386
|
+
```
|
|
387
|
+
|
|
388
|
+
That validates [framework-tuning-cookbook.json](../examples/tuning/framework-tuning-cookbook.json) against this guide, the cookbook README, and `examples/fixtures/preset-review-cases.json`.
|
|
389
|
+
|
|
390
|
+
## 15. Framework CI Examples
|
|
377
391
|
|
|
378
392
|
The workflow examples in `examples/ci` show copy-paste GitHub Actions setups for Next.js, Vite React, Express API, FastAPI, Terraform/Kubernetes, and AI MCP repos. Each workflow uploads SARIF and writes the readable Jester job summary.
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
# Framework Tuning Examples
|
|
2
|
+
|
|
3
|
+
Use this when a rule is noisy in a real project and you want the smallest evidence-backed next step before muting it.
|
|
4
|
+
|
|
5
|
+
For copy-pasteable recipes, see [examples/tuning](../examples/tuning). The machine-readable cookbook is [framework-tuning-cookbook.json](../examples/tuning/framework-tuning-cookbook.json), and `npm run framework:tuning:check` keeps it aligned with this guide and the fixture suite.
|
|
6
|
+
|
|
7
|
+
Start with the rule that actually fired:
|
|
8
|
+
|
|
9
|
+
```powershell
|
|
10
|
+
jester summary --kind <command|plan|diff|final> "<redacted minimal input>"
|
|
11
|
+
jester tune <rule-id> --json
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
Then compare the output with the nearest fixture-backed examples below. If your case is closer to the safe examples than the risky examples, add a redacted fixture before changing a rule.
|
|
15
|
+
|
|
16
|
+
| Cookbook recipe | Stack | Common noisy rule | Useful tune command | Safe fixture examples |
|
|
17
|
+
| --- | --- | --- | --- | --- |
|
|
18
|
+
| `next-vite-public-config` | Next.js / Vite React | Public but non-secret frontend names or harmless workspace commands | `jester tune custom-web-public-secret-name --json` or `jester tune custom-node-install-script-change --json` | `web-public-analytics-env-command-pass`, `node-next-lint-command-pass` |
|
|
19
|
+
| `fastapi-python-execution-boundary` | FastAPI / Python | Typed dependency injection, schema parsing, or locked dependency sync being confused with dynamic execution | `jester tune custom-python-eval-exec --json` or `jester tune custom-python-pickle-load --json` | `python-fastapi-dependency-diff-pass`, `python-pydantic-parse-diff-pass`, `python-uv-sync-frozen-command-pass` |
|
|
20
|
+
| `terraform-kubernetes-docs-only` | Terraform / Kubernetes / Helm | Docs-only infrastructure guidance mentioning sensitive tool names | `jester tune risky-domain --json` or `jester tune configured-sensitive-domain-terraform --json` | `infra-terraform-plan-docs-pass`, `infra-helm-values-docs-pass`, `infra-kubectl-describe-command-pass` |
|
|
21
|
+
| `security-scan-reporting` | Security scanning | Redacted scanner output or SBOM/report generation being confused with secret material | `jester tune secret-material --json` or `jester tune custom-insecure-tls-disabled --json` | `sec-gitleaks-redacted-command-pass`, `sec-sbom-command-pass`, `sec-vulnerability-report-docs-pass` |
|
|
22
|
+
| `ai-mcp-tooling` | AI / MCP tools | Static allowlists, model checks, or public model names being confused with unsafe tool dispatch or provider keys | `jester tune custom-ai-user-controlled-tool-dispatch --json` or `jester tune custom-ai-public-provider-key --json` | `ai-tool-registry-allowlist-diff-pass`, `ai-model-regression-command-pass`, `ai-public-model-env-diff-pass` |
|
|
23
|
+
|
|
24
|
+
## What To Do With The Result
|
|
25
|
+
|
|
26
|
+
- If `fixtureEvidence.quietPassFixtures` already contains a close match, prefer a local config mute over changing global rules.
|
|
27
|
+
- If the safe case is missing but the report is minimal and redacted, add a new pass fixture with `absentRuleIds`.
|
|
28
|
+
- If the rule fired on a genuinely dangerous command, secret, broad permission, production-impacting change, or user-controlled execution path, fix the underlying change instead of muting it.
|
|
29
|
+
- If the example depends on private code, customer data, credentials, or internal URLs, redact it before adding a fixture or opening a public issue.
|
package/docs/GETTING_STARTED.md
CHANGED
|
@@ -106,7 +106,7 @@ npx -y memento-mori-jester@latest bootstrap --preset node
|
|
|
106
106
|
|
|
107
107
|
Then tell them to open `MEMENTO_MORI.md`.
|
|
108
108
|
|
|
109
|
-
For copy-paste agent and hook examples, see [examples](../examples). For stack-specific config examples, see [preset example packs](../examples/presets) for Next.js, Vite React, Express API, FastAPI, Terraform/Kubernetes, and AI MCP repos. For copy-paste CI workflows, see [framework CI examples](../examples/ci). For concrete pass, caution, and block cases, see [review fixtures](../examples/fixtures).
|
|
109
|
+
For copy-paste agent and hook examples, see [examples](../examples). For stack-specific config examples, see [preset example packs](../examples/presets) for Next.js, Vite React, Express API, FastAPI, Terraform/Kubernetes, and AI MCP repos. For copy-paste CI workflows, see [framework CI examples](../examples/ci). For concrete pass, caution, and block cases, see [review fixtures](../examples/fixtures). For stack-shaped noisy-rule reports, see [framework tuning examples](FRAMEWORK_TUNING.md) and the checked [framework tuning cookbook](../examples/tuning).
|
|
110
110
|
|
|
111
111
|
## Need Help?
|
|
112
112
|
|
|
@@ -52,11 +52,13 @@ This checklist defines what "production grade" means for Memento Mori Jester rig
|
|
|
52
52
|
- Package metadata points bug reports at the GitHub issues page.
|
|
53
53
|
- `jester doctor --json`, `jester config validate`, and `jester rules` are the first troubleshooting commands.
|
|
54
54
|
- `jester tune`, `jester tune coverage`, and the fixture suite give maintainers a way to inspect noisy rules before changing defaults.
|
|
55
|
+
- [FRAMEWORK_TUNING.md](FRAMEWORK_TUNING.md) maps common stack-specific false-positive reports to the relevant `jester tune <rule-id>` evidence and fixture IDs, while [examples/tuning](../examples/tuning) provides checked copy-paste recipes.
|
|
55
56
|
- GitHub issue templates collect bug reports, false-positive reports, and feature requests with the diagnostic context maintainers need.
|
|
56
57
|
- `SECURITY.md` routes vulnerability reports away from public issues and asks for redacted diagnostics.
|
|
57
58
|
- `docs/MAINTAINER_TRIAGE.md` explains how to turn useful false-positive reports into fixture coverage before changing rule logic.
|
|
58
59
|
- `npm run fixtures:check` validates fixture IDs, metadata, unsafe-looking content, duplicate content, and explicit expected/absent rule intent.
|
|
59
60
|
- `npm run fixtures:report` shows fixture coverage by rule, rule family, preset slice, kind, verdict, quiet-pass rule boundaries, and feasible pass-case gaps so maintainers can pick the next fixture target; `npm run fixtures:report -- --markdown` produces a paste-ready maintainer snapshot.
|
|
61
|
+
- `npm run framework:tuning:check` keeps the framework tuning guide, cookbook JSON, cookbook README, and fixture IDs aligned.
|
|
60
62
|
- `npm run promo:card` regenerates the deterministic social preview card, and `npm run promo:check` verifies current repo-local promo assets against the current fixture evidence before maintainers post or refresh the demo.
|
|
61
63
|
- `npm run site:check` verifies the static landing page before maintainers post or host it.
|
|
62
64
|
- npm publish has a manual workflow fallback, but the normal release path is tag-driven trusted publishing.
|
|
@@ -74,6 +76,7 @@ This checklist defines what "production grade" means for Memento Mori Jester rig
|
|
|
74
76
|
- maintainer triage docs exist and link noisy-rule reports back to fixture coverage.
|
|
75
77
|
- fixture authoring checks are wired into `npm test`.
|
|
76
78
|
- fixture coverage reports are wired into `npm test`.
|
|
79
|
+
- framework tuning cookbook checks are wired into `npm test`.
|
|
77
80
|
- promo freshness checks are wired into `npm test`.
|
|
78
81
|
- site checks are wired into `npm test`.
|
|
79
82
|
|
package/docs/RELEASE.md
CHANGED
|
@@ -12,6 +12,7 @@ npm.cmd run fixtures:check
|
|
|
12
12
|
npm.cmd run fixtures:report
|
|
13
13
|
npm.cmd run fixtures:report -- --json
|
|
14
14
|
npm.cmd run fixtures:report -- --markdown
|
|
15
|
+
npm.cmd run framework:tuning:check
|
|
15
16
|
npm.cmd run promo:card:check
|
|
16
17
|
npm.cmd run promo:check
|
|
17
18
|
npm.cmd run site:check
|
|
@@ -24,7 +25,7 @@ Move the current changelog bullets into a matching version section and add `docs
|
|
|
24
25
|
## 2. Tag And Push
|
|
25
26
|
|
|
26
27
|
```powershell
|
|
27
|
-
git add package.json package-lock.json CHANGELOG.md docs/RELEASE_NOTES_v0.1.x.md docs/PRODUCTION_READINESS.md docs/MAINTAINER_TRIAGE.md SECURITY.md .github/ISSUE_TEMPLATE
|
|
28
|
+
git add package.json package-lock.json CHANGELOG.md docs/RELEASE_NOTES_v0.1.x.md docs/PRODUCTION_READINESS.md docs/MAINTAINER_TRIAGE.md docs/FRAMEWORK_TUNING.md examples/tuning scripts/check-framework-tuning.mjs SECURITY.md .github/ISSUE_TEMPLATE
|
|
28
29
|
git commit -m "Release v0.1.x"
|
|
29
30
|
git tag -a v0.1.x -m "Memento Mori Jester v0.1.x"
|
|
30
31
|
git push origin main
|
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
# Memento Mori Jester v0.1.82
|
|
2
|
+
|
|
3
|
+
## Summary
|
|
4
|
+
|
|
5
|
+
This release continues the core quality track after the landing-page release. It adds a small fixture-backed false-positive batch and a framework tuning guide so maintainers can compare real noisy reports with safe examples before muting or changing rules.
|
|
6
|
+
|
|
7
|
+
## What Changed
|
|
8
|
+
|
|
9
|
+
- Added six quiet-pass fixtures, growing the corpus from 216 to 222 cases:
|
|
10
|
+
- `python-fastapi-dependency-diff-pass`
|
|
11
|
+
- `python-uv-sync-frozen-command-pass`
|
|
12
|
+
- `infra-terraform-plan-docs-pass`
|
|
13
|
+
- `infra-helm-values-docs-pass`
|
|
14
|
+
- `sec-gitleaks-redacted-command-pass`
|
|
15
|
+
- `node-next-lint-command-pass`
|
|
16
|
+
- Added `docs/FRAMEWORK_TUNING.md`.
|
|
17
|
+
- Linked framework tuning guidance from README, CLI docs, and getting-started docs.
|
|
18
|
+
- Refreshed demo, promo-source evidence counts, site proof count, fixture docs, changelog, and roadmap.
|
|
19
|
+
|
|
20
|
+
## Public Interface
|
|
21
|
+
|
|
22
|
+
- No CLI command changes.
|
|
23
|
+
- No MCP tool changes.
|
|
24
|
+
- No config schema changes.
|
|
25
|
+
- No review rule, scoring, matching, or verdict behavior changes.
|
|
26
|
+
- No GitHub Action or release workflow changes.
|
|
27
|
+
- Fixture/tuning evidence changes only because the corpus is larger.
|
|
28
|
+
|
|
29
|
+
## Release Validation
|
|
30
|
+
|
|
31
|
+
```powershell
|
|
32
|
+
npm.cmd test
|
|
33
|
+
npm.cmd run demo:svg:check
|
|
34
|
+
npm.cmd run promo:card:check
|
|
35
|
+
npm.cmd run promo:check
|
|
36
|
+
npm.cmd run fixtures:report
|
|
37
|
+
npm.cmd run fixtures:report -- --json
|
|
38
|
+
npm.cmd run fixtures:report -- --markdown
|
|
39
|
+
npm.cmd run site:check
|
|
40
|
+
npm.cmd run pack:dry
|
|
41
|
+
git diff --check
|
|
42
|
+
node .\dist\cli.js tune risky-domain --json --no-config
|
|
43
|
+
node .\dist\cli.js tune coverage --no-config
|
|
44
|
+
git diff | node .\dist\cli.js diff --fail-on block --subject "v0.1.82 framework tuning fixtures"
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
Expected:
|
|
48
|
+
|
|
49
|
+
- fixture report shows `Fixtures: 222`,
|
|
50
|
+
- no thin rule coverage,
|
|
51
|
+
- no preset/kind gaps,
|
|
52
|
+
- no rules without quiet-pass coverage,
|
|
53
|
+
- `promo:check` passes with 222 fixtures and 8 risky-domain quiet-pass examples,
|
|
54
|
+
- GitHub Release and npm Publish complete from the `v0.1.82` tag.
|
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
# Memento Mori Jester v0.1.83
|
|
2
|
+
|
|
3
|
+
## Summary
|
|
4
|
+
|
|
5
|
+
This release makes the framework tuning guide easier to execute from real repos. It adds a tiny checked cookbook that maps common stack-shaped noisy-rule reports to exact `jester tune <rule-id> --json` commands and fixture IDs.
|
|
6
|
+
|
|
7
|
+
## What Changed
|
|
8
|
+
|
|
9
|
+
- Added `examples/tuning/framework-tuning-cookbook.json`.
|
|
10
|
+
- Added `examples/tuning/README.md` with copy-paste recipes for:
|
|
11
|
+
- Next.js / Vite React,
|
|
12
|
+
- FastAPI / Python,
|
|
13
|
+
- Terraform / Kubernetes / Helm,
|
|
14
|
+
- security scanning,
|
|
15
|
+
- AI / MCP tools.
|
|
16
|
+
- Added `npm run framework:tuning:check`.
|
|
17
|
+
- Wired the cookbook checker into `npm test` and production-readiness checks.
|
|
18
|
+
- Linked the cookbook from README, CLI docs, getting-started docs, demo docs, release docs, production-readiness docs, and `docs/FRAMEWORK_TUNING.md`.
|
|
19
|
+
|
|
20
|
+
## Public Interface
|
|
21
|
+
|
|
22
|
+
- No CLI command changes.
|
|
23
|
+
- No MCP tool changes.
|
|
24
|
+
- No config schema changes.
|
|
25
|
+
- No review rule, scoring, matching, or verdict behavior changes.
|
|
26
|
+
- No GitHub Action or release workflow changes.
|
|
27
|
+
- Fixture count remains `222`; this release adds checked cookbook assets, not new review cases.
|
|
28
|
+
|
|
29
|
+
## Release Validation
|
|
30
|
+
|
|
31
|
+
```powershell
|
|
32
|
+
npm.cmd test
|
|
33
|
+
npm.cmd run demo:svg:check
|
|
34
|
+
npm.cmd run promo:card:check
|
|
35
|
+
npm.cmd run promo:check
|
|
36
|
+
npm.cmd run framework:tuning:check
|
|
37
|
+
npm.cmd run fixtures:report
|
|
38
|
+
npm.cmd run fixtures:report -- --json
|
|
39
|
+
npm.cmd run fixtures:report -- --markdown
|
|
40
|
+
npm.cmd run site:check
|
|
41
|
+
npm.cmd run pack:dry
|
|
42
|
+
git diff --check
|
|
43
|
+
node .\dist\cli.js tune coverage --no-config
|
|
44
|
+
git diff | node .\dist\cli.js diff --fail-on block --subject "v0.1.83 framework tuning cookbook"
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
Expected:
|
|
48
|
+
|
|
49
|
+
- `framework:tuning:check` passes for five recipes,
|
|
50
|
+
- fixture report still shows `Fixtures: 222`,
|
|
51
|
+
- no thin rule coverage,
|
|
52
|
+
- no preset/kind gaps,
|
|
53
|
+
- no rules without quiet-pass coverage,
|
|
54
|
+
- GitHub Release and npm Publish complete from the `v0.1.83` tag.
|
|
@@ -30,6 +30,7 @@ Maintainer triage guidance lives in [docs/MAINTAINER_TRIAGE.md](../../docs/MAINT
|
|
|
30
30
|
- Quiet-pass examples for safe text rendering, allowlisted target paths, public analytics IDs, model-check commands, tool allowlist checks, and public model-name config.
|
|
31
31
|
- Quiet-pass examples for API schema parsing, query-builder filters, enabled rate limiting, read-only Prisma migration diffs, signed-webhook docs, and OpenAPI schema docs.
|
|
32
32
|
- Quiet-pass examples for Pydantic parsing, Pyright checks, SBOM generation, vulnerability-report docs, escaped React rendering, session-cookie docs, model regression checks, and static action allowlists.
|
|
33
|
+
- Quiet-pass examples for FastAPI dependency injection, frozen `uv` syncs, docs-only Terraform and Helm guidance, redacted Gitleaks scans, and Next.js workspace linting.
|
|
33
34
|
- Fixture metadata like `weight` and `edgeCase` to support precision-weighted tuning evidence.
|
|
34
35
|
|
|
35
36
|
## Local Check
|
|
@@ -2859,5 +2859,96 @@
|
|
|
2859
2859
|
"custom-ai-user-controlled-tool-dispatch",
|
|
2860
2860
|
"custom-ai-model-output-execution"
|
|
2861
2861
|
]
|
|
2862
|
+
},
|
|
2863
|
+
{
|
|
2864
|
+
"id": "python-fastapi-dependency-diff-pass",
|
|
2865
|
+
"preset": "python",
|
|
2866
|
+
"kind": "diff",
|
|
2867
|
+
"description": "FastAPI dependency injection should not look like Python dynamic execution or pickle loading.",
|
|
2868
|
+
"content": "diff --git a/app/dependencies.py b/app/dependencies.py\n--- a/app/dependencies.py\n+++ b/app/dependencies.py\n@@ -1 +1,2 @@\n+current_user: User = Depends(get_current_user)\n",
|
|
2869
|
+
"expectedVerdict": "pass",
|
|
2870
|
+
"weight": 2,
|
|
2871
|
+
"expectedRuleIds": [],
|
|
2872
|
+
"absentRuleIds": [
|
|
2873
|
+
"custom-python-eval-exec",
|
|
2874
|
+
"custom-python-pickle-load"
|
|
2875
|
+
]
|
|
2876
|
+
},
|
|
2877
|
+
{
|
|
2878
|
+
"id": "python-uv-sync-frozen-command-pass",
|
|
2879
|
+
"preset": "python",
|
|
2880
|
+
"kind": "command",
|
|
2881
|
+
"description": "Frozen uv dependency sync for tests should not trip unsafe pip install or dynamic execution rules.",
|
|
2882
|
+
"content": "uv sync --frozen --group test",
|
|
2883
|
+
"expectedVerdict": "pass",
|
|
2884
|
+
"weight": 2,
|
|
2885
|
+
"expectedRuleIds": [],
|
|
2886
|
+
"absentRuleIds": [
|
|
2887
|
+
"blocked-command-pip-install-break-system-packages",
|
|
2888
|
+
"custom-python-eval-exec",
|
|
2889
|
+
"custom-python-pickle-load"
|
|
2890
|
+
]
|
|
2891
|
+
},
|
|
2892
|
+
{
|
|
2893
|
+
"id": "infra-terraform-plan-docs-pass",
|
|
2894
|
+
"preset": "infra",
|
|
2895
|
+
"kind": "diff",
|
|
2896
|
+
"description": "Docs-only Terraform plan review guidance should not trip infra sensitive-domain warnings.",
|
|
2897
|
+
"content": "diff --git a/docs/TERRAFORM_PLAN.md b/docs/TERRAFORM_PLAN.md\n--- a/docs/TERRAFORM_PLAN.md\n+++ b/docs/TERRAFORM_PLAN.md\n@@ -1 +1,2 @@\n+Document how to review terraform plan output and attach rollback notes before apply.\n",
|
|
2898
|
+
"expectedVerdict": "pass",
|
|
2899
|
+
"edgeCase": true,
|
|
2900
|
+
"weight": 1,
|
|
2901
|
+
"expectedRuleIds": [],
|
|
2902
|
+
"absentRuleIds": [
|
|
2903
|
+
"configured-sensitive-domain-terraform",
|
|
2904
|
+
"custom-infra-production-change",
|
|
2905
|
+
"risky-domain"
|
|
2906
|
+
]
|
|
2907
|
+
},
|
|
2908
|
+
{
|
|
2909
|
+
"id": "infra-helm-values-docs-pass",
|
|
2910
|
+
"preset": "infra",
|
|
2911
|
+
"kind": "diff",
|
|
2912
|
+
"description": "Docs-only Helm values guidance should stay quiet around infra sensitive-domain warnings.",
|
|
2913
|
+
"content": "diff --git a/docs/HELM_VALUES.md b/docs/HELM_VALUES.md\n--- a/docs/HELM_VALUES.md\n+++ b/docs/HELM_VALUES.md\n@@ -1 +1,2 @@\n+Record which helm values are environment labels and which require change approval.\n",
|
|
2914
|
+
"expectedVerdict": "pass",
|
|
2915
|
+
"edgeCase": true,
|
|
2916
|
+
"weight": 1,
|
|
2917
|
+
"expectedRuleIds": [],
|
|
2918
|
+
"absentRuleIds": [
|
|
2919
|
+
"configured-sensitive-domain-helm",
|
|
2920
|
+
"custom-infra-production-change",
|
|
2921
|
+
"risky-domain"
|
|
2922
|
+
]
|
|
2923
|
+
},
|
|
2924
|
+
{
|
|
2925
|
+
"id": "sec-gitleaks-redacted-command-pass",
|
|
2926
|
+
"preset": "security",
|
|
2927
|
+
"kind": "command",
|
|
2928
|
+
"description": "Redacted gitleaks scans should not be mistaken for leaked secret material.",
|
|
2929
|
+
"content": "gitleaks detect --redact --no-git -r reports/gitleaks.json",
|
|
2930
|
+
"expectedVerdict": "pass",
|
|
2931
|
+
"weight": 2,
|
|
2932
|
+
"expectedRuleIds": [],
|
|
2933
|
+
"absentRuleIds": [
|
|
2934
|
+
"secret-material",
|
|
2935
|
+
"blocked-command-chmod-r-777",
|
|
2936
|
+
"custom-insecure-tls-disabled"
|
|
2937
|
+
]
|
|
2938
|
+
},
|
|
2939
|
+
{
|
|
2940
|
+
"id": "node-next-lint-command-pass",
|
|
2941
|
+
"preset": "node",
|
|
2942
|
+
"kind": "command",
|
|
2943
|
+
"description": "Next.js workspace lint commands should stay quiet around node publish and install-script checks.",
|
|
2944
|
+
"content": "npm run lint --workspace apps/next-web -- --max-warnings=0",
|
|
2945
|
+
"expectedVerdict": "pass",
|
|
2946
|
+
"weight": 2,
|
|
2947
|
+
"expectedRuleIds": [],
|
|
2948
|
+
"absentRuleIds": [
|
|
2949
|
+
"blocked-command-npm-publish-force",
|
|
2950
|
+
"custom-node-install-script-change",
|
|
2951
|
+
"custom-node-env-production-change"
|
|
2952
|
+
]
|
|
2862
2953
|
}
|
|
2863
2954
|
]
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
# Framework Tuning Cookbook
|
|
2
|
+
|
|
3
|
+
These small recipes pair the [Framework Tuning Examples](../../docs/FRAMEWORK_TUNING.md) guide with checked, fixture-backed commands. Use them when a real repo reports a noisy rule and you want the smallest evidence-backed next step before changing config.
|
|
4
|
+
|
|
5
|
+
The machine-readable source is [framework-tuning-cookbook.json](framework-tuning-cookbook.json). It is checked by `npm run framework:tuning:check`, so recipe commands and fixture IDs stay aligned with the public guide and fixture suite.
|
|
6
|
+
|
|
7
|
+
## Recipes
|
|
8
|
+
|
|
9
|
+
| Recipe | Stack | Preset | Tune commands | Fixture examples |
|
|
10
|
+
| --- | --- | --- | --- | --- |
|
|
11
|
+
| `next-vite-public-config` | Next.js / Vite React | `web` | `jester tune custom-web-public-secret-name --json`; `jester tune custom-node-install-script-change --json` | `web-public-analytics-env-command-pass`, `node-next-lint-command-pass` |
|
|
12
|
+
| `fastapi-python-execution-boundary` | FastAPI / Python | `python` | `jester tune custom-python-eval-exec --json`; `jester tune custom-python-pickle-load --json` | `python-fastapi-dependency-diff-pass`, `python-pydantic-parse-diff-pass`, `python-uv-sync-frozen-command-pass` |
|
|
13
|
+
| `terraform-kubernetes-docs-only` | Terraform / Kubernetes / Helm | `infra` | `jester tune risky-domain --json`; `jester tune configured-sensitive-domain-terraform --json` | `infra-terraform-plan-docs-pass`, `infra-helm-values-docs-pass`, `infra-kubectl-describe-command-pass` |
|
|
14
|
+
| `security-scan-reporting` | Security scanning | `security` | `jester tune secret-material --json`; `jester tune custom-insecure-tls-disabled --json` | `sec-gitleaks-redacted-command-pass`, `sec-sbom-command-pass`, `sec-vulnerability-report-docs-pass` |
|
|
15
|
+
| `ai-mcp-tooling` | AI / MCP tools | `ai` | `jester tune custom-ai-user-controlled-tool-dispatch --json`; `jester tune custom-ai-public-provider-key --json` | `ai-tool-registry-allowlist-diff-pass`, `ai-model-regression-command-pass`, `ai-public-model-env-diff-pass` |
|
|
16
|
+
|
|
17
|
+
## How To Use A Recipe
|
|
18
|
+
|
|
19
|
+
1. Run `jester summary --kind <command|plan|diff|final> "<redacted minimal input>"` and copy the rule id that fired.
|
|
20
|
+
2. Run the nearest recipe command, such as `jester tune custom-ai-user-controlled-tool-dispatch --json`.
|
|
21
|
+
3. Compare `fixtureEvidence.quietPassFixtures` and sample fixture descriptions with the local hit.
|
|
22
|
+
4. If the local case is safe but missing from the fixture suite, add a redacted pass fixture before loosening a rule.
|
|
23
|
+
5. If the local case includes a real secret, destructive command, broad permission, skipped eval, or user-controlled execution path, fix the change instead of muting the rule.
|
|
@@ -0,0 +1,96 @@
|
|
|
1
|
+
[
|
|
2
|
+
{
|
|
3
|
+
"id": "next-vite-public-config",
|
|
4
|
+
"stack": "Next.js / Vite React",
|
|
5
|
+
"preset": "web",
|
|
6
|
+
"when": "Public-but-non-secret frontend names or harmless workspace commands look similar to browser secret or install-script risks.",
|
|
7
|
+
"commands": [
|
|
8
|
+
"jester tune custom-web-public-secret-name --json",
|
|
9
|
+
"jester tune custom-node-install-script-change --json"
|
|
10
|
+
],
|
|
11
|
+
"fixtures": [
|
|
12
|
+
"web-public-analytics-env-command-pass",
|
|
13
|
+
"node-next-lint-command-pass"
|
|
14
|
+
],
|
|
15
|
+
"next": [
|
|
16
|
+
"Compare fixtureEvidence.quietPassFixtures with the redacted local hit.",
|
|
17
|
+
"Prefer a local disabledRules entry only after repeated harmless hits in the same repo."
|
|
18
|
+
]
|
|
19
|
+
},
|
|
20
|
+
{
|
|
21
|
+
"id": "fastapi-python-execution-boundary",
|
|
22
|
+
"stack": "FastAPI / Python",
|
|
23
|
+
"preset": "python",
|
|
24
|
+
"when": "Typed dependency injection, schema parsing, or locked dependency sync is mistaken for dynamic execution or unsafe deserialization.",
|
|
25
|
+
"commands": [
|
|
26
|
+
"jester tune custom-python-eval-exec --json",
|
|
27
|
+
"jester tune custom-python-pickle-load --json"
|
|
28
|
+
],
|
|
29
|
+
"fixtures": [
|
|
30
|
+
"python-fastapi-dependency-diff-pass",
|
|
31
|
+
"python-pydantic-parse-diff-pass",
|
|
32
|
+
"python-uv-sync-frozen-command-pass"
|
|
33
|
+
],
|
|
34
|
+
"next": [
|
|
35
|
+
"Compare the local change with the safe FastAPI and Pydantic fixture descriptions first.",
|
|
36
|
+
"Add a redacted pass fixture before loosening Python execution or pickle-shaped rules."
|
|
37
|
+
]
|
|
38
|
+
},
|
|
39
|
+
{
|
|
40
|
+
"id": "terraform-kubernetes-docs-only",
|
|
41
|
+
"stack": "Terraform / Kubernetes / Helm",
|
|
42
|
+
"preset": "infra",
|
|
43
|
+
"when": "Docs-only infrastructure guidance mentions Terraform, Helm, or other sensitive infra words without changing deploy behavior.",
|
|
44
|
+
"commands": [
|
|
45
|
+
"jester tune risky-domain --json",
|
|
46
|
+
"jester tune configured-sensitive-domain-terraform --json"
|
|
47
|
+
],
|
|
48
|
+
"fixtures": [
|
|
49
|
+
"infra-terraform-plan-docs-pass",
|
|
50
|
+
"infra-helm-values-docs-pass",
|
|
51
|
+
"infra-kubectl-describe-command-pass"
|
|
52
|
+
],
|
|
53
|
+
"next": [
|
|
54
|
+
"Check that the local hit is documentation-only before muting broad sensitive-domain noise.",
|
|
55
|
+
"Keep destructive command, IAM widening, public exposure, and deploy-changing rules active."
|
|
56
|
+
]
|
|
57
|
+
},
|
|
58
|
+
{
|
|
59
|
+
"id": "security-scan-reporting",
|
|
60
|
+
"stack": "Security scanning",
|
|
61
|
+
"preset": "security",
|
|
62
|
+
"when": "Redacted scanner output, SBOM generation, or reporting docs look like exposed secret material or skipped verification.",
|
|
63
|
+
"commands": [
|
|
64
|
+
"jester tune secret-material --json",
|
|
65
|
+
"jester tune custom-insecure-tls-disabled --json"
|
|
66
|
+
],
|
|
67
|
+
"fixtures": [
|
|
68
|
+
"sec-gitleaks-redacted-command-pass",
|
|
69
|
+
"sec-sbom-command-pass",
|
|
70
|
+
"sec-vulnerability-report-docs-pass"
|
|
71
|
+
],
|
|
72
|
+
"next": [
|
|
73
|
+
"Verify the report uses placeholders or scanner-generated summaries rather than live tokens.",
|
|
74
|
+
"If a real credential appears, rotate it and treat the Jester hit as correct."
|
|
75
|
+
]
|
|
76
|
+
},
|
|
77
|
+
{
|
|
78
|
+
"id": "ai-mcp-tooling",
|
|
79
|
+
"stack": "AI / MCP tools",
|
|
80
|
+
"preset": "ai",
|
|
81
|
+
"when": "Static allowlists, model checks, or public model names are mistaken for unsafe tool dispatch, skipped evals, or provider keys.",
|
|
82
|
+
"commands": [
|
|
83
|
+
"jester tune custom-ai-user-controlled-tool-dispatch --json",
|
|
84
|
+
"jester tune custom-ai-public-provider-key --json"
|
|
85
|
+
],
|
|
86
|
+
"fixtures": [
|
|
87
|
+
"ai-tool-registry-allowlist-diff-pass",
|
|
88
|
+
"ai-model-regression-command-pass",
|
|
89
|
+
"ai-public-model-env-diff-pass"
|
|
90
|
+
],
|
|
91
|
+
"next": [
|
|
92
|
+
"Compare the local tool path with the allowlist fixture before changing an AI preset rule.",
|
|
93
|
+
"Keep concrete model-output execution and client-exposed provider-key hits treated as dangerous."
|
|
94
|
+
]
|
|
95
|
+
}
|
|
96
|
+
]
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "memento-mori-jester",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.83",
|
|
4
4
|
"description": "A local court-jester sidecar for AI coding agents: review plans, commands, diffs, and final claims before they get too pleased with themselves.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"repository": {
|
|
@@ -40,12 +40,13 @@
|
|
|
40
40
|
"build": "tsc -p tsconfig.json",
|
|
41
41
|
"start": "node dist/server.js",
|
|
42
42
|
"start:mcp": "node dist/server.js",
|
|
43
|
-
"test": "npm run build && node scripts/run-tests.mjs && npm run fixtures:check && npm run fixtures:report && npm run promo:check && npm run site:check && npm run production:check",
|
|
43
|
+
"test": "npm run build && node scripts/run-tests.mjs && npm run fixtures:check && npm run fixtures:report && npm run framework:tuning:check && npm run promo:check && npm run site:check && npm run production:check",
|
|
44
44
|
"doctor": "node dist/cli.js doctor",
|
|
45
45
|
"demo:svg": "node scripts/render-demo-svg.mjs",
|
|
46
46
|
"demo:svg:check": "node scripts/render-demo-svg.mjs --check",
|
|
47
47
|
"fixtures:check": "node scripts/check-fixtures.mjs",
|
|
48
48
|
"fixtures:report": "node scripts/report-fixtures.mjs",
|
|
49
|
+
"framework:tuning:check": "node scripts/check-framework-tuning.mjs",
|
|
49
50
|
"promo:card": "node scripts/render-social-card.mjs",
|
|
50
51
|
"promo:card:check": "node scripts/render-social-card.mjs --check",
|
|
51
52
|
"promo:check": "node scripts/check-promo-freshness.mjs",
|
|
@@ -0,0 +1,214 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
import { readFileSync } from "node:fs";
|
|
3
|
+
import { join } from "node:path";
|
|
4
|
+
|
|
5
|
+
const root = process.cwd();
|
|
6
|
+
const cookbookPath = "examples/tuning/framework-tuning-cookbook.json";
|
|
7
|
+
const cookbookReadmePath = "examples/tuning/README.md";
|
|
8
|
+
const guidePath = "docs/FRAMEWORK_TUNING.md";
|
|
9
|
+
const fixturesPath = "examples/fixtures/preset-review-cases.json";
|
|
10
|
+
const failures = [];
|
|
11
|
+
|
|
12
|
+
const allowedPresets = new Set(["default", "node", "python", "web", "api", "infra", "ai", "security"]);
|
|
13
|
+
const unsafeContentPatterns = [
|
|
14
|
+
{ name: "private key block", pattern: /-----BEGIN [A-Z ]*PRIVATE KEY-----/ },
|
|
15
|
+
{ name: "OpenAI-looking secret key", pattern: /\bsk-(?:proj-)?[A-Za-z0-9_-]{20,}\b/ },
|
|
16
|
+
{ name: "Anthropic-looking secret key", pattern: /\bsk-ant-[A-Za-z0-9_-]{20,}\b/ },
|
|
17
|
+
{ name: "GitHub-looking token", pattern: /\bgh[pousr]_[A-Za-z0-9_]{20,}\b/ },
|
|
18
|
+
{ name: "AWS access key id", pattern: /\bAKIA[0-9A-Z]{16}\b/ },
|
|
19
|
+
{ name: "Slack-looking token", pattern: /\bxox[baprs]-[A-Za-z0-9-]{20,}\b/ },
|
|
20
|
+
{ name: "absolute Unix home path", pattern: /(?:^|[\s"'`])\/(?:Users|home)\/[A-Za-z0-9._-]+/ },
|
|
21
|
+
{ name: "absolute Windows user path", pattern: /[A-Za-z]:\\Users\\[A-Za-z0-9._-]+\\/ }
|
|
22
|
+
];
|
|
23
|
+
|
|
24
|
+
function read(path) {
|
|
25
|
+
return readFileSync(join(root, path), "utf8");
|
|
26
|
+
}
|
|
27
|
+
|
|
28
|
+
function readJson(path) {
|
|
29
|
+
return JSON.parse(read(path));
|
|
30
|
+
}
|
|
31
|
+
|
|
32
|
+
function checkString(value, label, options = {}) {
|
|
33
|
+
if (typeof value !== "string" || value.trim().length === 0) {
|
|
34
|
+
failures.push(`${label} should be a non-empty string.`);
|
|
35
|
+
return;
|
|
36
|
+
}
|
|
37
|
+
|
|
38
|
+
if (options.kebab && !/^[a-z0-9]+(?:-[a-z0-9]+)*$/.test(value)) {
|
|
39
|
+
failures.push(`${label} should use stable kebab-case.`);
|
|
40
|
+
}
|
|
41
|
+
|
|
42
|
+
if (options.minLength && value.trim().length < options.minLength) {
|
|
43
|
+
failures.push(`${label} should be at least ${options.minLength} characters.`);
|
|
44
|
+
}
|
|
45
|
+
}
|
|
46
|
+
|
|
47
|
+
function checkStringArray(value, label, options = {}) {
|
|
48
|
+
if (!Array.isArray(value) || value.length === 0) {
|
|
49
|
+
failures.push(`${label} should be a non-empty array.`);
|
|
50
|
+
return [];
|
|
51
|
+
}
|
|
52
|
+
|
|
53
|
+
const seen = new Set();
|
|
54
|
+
for (const [index, item] of value.entries()) {
|
|
55
|
+
if (typeof item !== "string" || item.trim().length === 0) {
|
|
56
|
+
failures.push(`${label}[${index}] should be a non-empty string.`);
|
|
57
|
+
continue;
|
|
58
|
+
}
|
|
59
|
+
|
|
60
|
+
if (seen.has(item)) {
|
|
61
|
+
failures.push(`${label} repeats ${item}.`);
|
|
62
|
+
}
|
|
63
|
+
seen.add(item);
|
|
64
|
+
|
|
65
|
+
if (options.pattern && !options.pattern.test(item)) {
|
|
66
|
+
failures.push(`${label}[${index}] should match ${options.description}. Saw ${item}.`);
|
|
67
|
+
}
|
|
68
|
+
}
|
|
69
|
+
|
|
70
|
+
return value.filter((item) => typeof item === "string" && item.trim().length > 0);
|
|
71
|
+
}
|
|
72
|
+
|
|
73
|
+
function requireIncludes(content, file, value, description) {
|
|
74
|
+
if (!content.includes(value)) {
|
|
75
|
+
failures.push(`${file} should include ${description}: ${value}`);
|
|
76
|
+
}
|
|
77
|
+
}
|
|
78
|
+
|
|
79
|
+
let cookbook;
|
|
80
|
+
let fixtures;
|
|
81
|
+
try {
|
|
82
|
+
cookbook = readJson(cookbookPath);
|
|
83
|
+
fixtures = readJson(fixturesPath);
|
|
84
|
+
} catch (error) {
|
|
85
|
+
console.error(`Could not parse framework tuning inputs: ${error instanceof Error ? error.message : String(error)}`);
|
|
86
|
+
process.exit(1);
|
|
87
|
+
}
|
|
88
|
+
|
|
89
|
+
const cookbookRaw = read(cookbookPath);
|
|
90
|
+
const cookbookReadme = read(cookbookReadmePath);
|
|
91
|
+
const frameworkGuide = read(guidePath);
|
|
92
|
+
|
|
93
|
+
for (const [file, content] of [
|
|
94
|
+
[cookbookPath, cookbookRaw],
|
|
95
|
+
[cookbookReadmePath, cookbookReadme],
|
|
96
|
+
[guidePath, frameworkGuide]
|
|
97
|
+
]) {
|
|
98
|
+
for (const unsafe of unsafeContentPatterns) {
|
|
99
|
+
if (unsafe.pattern.test(content)) {
|
|
100
|
+
failures.push(`${file} appears to contain ${unsafe.name}; tuning examples should stay public and redacted.`);
|
|
101
|
+
}
|
|
102
|
+
}
|
|
103
|
+
}
|
|
104
|
+
|
|
105
|
+
if (!Array.isArray(cookbook)) {
|
|
106
|
+
failures.push(`${cookbookPath} should contain a JSON array.`);
|
|
107
|
+
} else if (cookbook.length < 5) {
|
|
108
|
+
failures.push(`${cookbookPath} should contain at least five framework recipes.`);
|
|
109
|
+
}
|
|
110
|
+
|
|
111
|
+
if (!Array.isArray(fixtures)) {
|
|
112
|
+
failures.push(`${fixturesPath} should contain a JSON array.`);
|
|
113
|
+
fixtures = [];
|
|
114
|
+
}
|
|
115
|
+
|
|
116
|
+
const fixtureIds = new Set();
|
|
117
|
+
const fixtureRuleIds = new Set();
|
|
118
|
+
for (const fixture of fixtures) {
|
|
119
|
+
if (typeof fixture?.id === "string") {
|
|
120
|
+
fixtureIds.add(fixture.id);
|
|
121
|
+
}
|
|
122
|
+
for (const field of ["expectedRuleIds", "absentRuleIds"]) {
|
|
123
|
+
if (Array.isArray(fixture?.[field])) {
|
|
124
|
+
for (const ruleId of fixture[field]) {
|
|
125
|
+
if (typeof ruleId === "string") {
|
|
126
|
+
fixtureRuleIds.add(ruleId);
|
|
127
|
+
}
|
|
128
|
+
}
|
|
129
|
+
}
|
|
130
|
+
}
|
|
131
|
+
}
|
|
132
|
+
|
|
133
|
+
requireIncludes(frameworkGuide, guidePath, "framework-tuning-cookbook.json", "the cookbook JSON link");
|
|
134
|
+
requireIncludes(frameworkGuide, guidePath, "examples/tuning", "the tuning examples link");
|
|
135
|
+
requireIncludes(cookbookReadme, cookbookReadmePath, "framework-tuning-cookbook.json", "the cookbook JSON link");
|
|
136
|
+
requireIncludes(cookbookReadme, cookbookReadmePath, "npm run framework:tuning:check", "the checker command");
|
|
137
|
+
|
|
138
|
+
const ids = new Set();
|
|
139
|
+
const commandPattern = /^jester tune ([a-z0-9]+(?:-[a-z0-9]+)*) --json$/;
|
|
140
|
+
|
|
141
|
+
for (const [index, recipe] of (Array.isArray(cookbook) ? cookbook : []).entries()) {
|
|
142
|
+
const label = typeof recipe?.id === "string" ? recipe.id : `recipe[${index}]`;
|
|
143
|
+
|
|
144
|
+
if (!recipe || typeof recipe !== "object" || Array.isArray(recipe)) {
|
|
145
|
+
failures.push(`${label} should be an object.`);
|
|
146
|
+
continue;
|
|
147
|
+
}
|
|
148
|
+
|
|
149
|
+
checkString(recipe.id, `${label}.id`, { kebab: true });
|
|
150
|
+
checkString(recipe.stack, `${label}.stack`);
|
|
151
|
+
checkString(recipe.preset, `${label}.preset`);
|
|
152
|
+
checkString(recipe.when, `${label}.when`, { minLength: 40 });
|
|
153
|
+
|
|
154
|
+
if (typeof recipe.id === "string") {
|
|
155
|
+
if (ids.has(recipe.id)) {
|
|
156
|
+
failures.push(`${label}.id is duplicated.`);
|
|
157
|
+
}
|
|
158
|
+
ids.add(recipe.id);
|
|
159
|
+
}
|
|
160
|
+
|
|
161
|
+
if (!allowedPresets.has(recipe.preset)) {
|
|
162
|
+
failures.push(`${label}.preset should be one of: ${[...allowedPresets].join(", ")}.`);
|
|
163
|
+
}
|
|
164
|
+
|
|
165
|
+
const commands = checkStringArray(recipe.commands, `${label}.commands`, {
|
|
166
|
+
pattern: commandPattern,
|
|
167
|
+
description: "`jester tune <rule-id> --json`"
|
|
168
|
+
});
|
|
169
|
+
const fixturesForRecipe = checkStringArray(recipe.fixtures, `${label}.fixtures`, {
|
|
170
|
+
pattern: /^[a-z0-9]+(?:-[a-z0-9]+)*$/,
|
|
171
|
+
description: "a kebab-case fixture id"
|
|
172
|
+
});
|
|
173
|
+
checkStringArray(recipe.next, `${label}.next`);
|
|
174
|
+
|
|
175
|
+
for (const command of commands) {
|
|
176
|
+
const match = command.match(commandPattern);
|
|
177
|
+
const ruleId = match?.[1];
|
|
178
|
+
if (ruleId && !fixtureRuleIds.has(ruleId)) {
|
|
179
|
+
failures.push(`${label}.commands references ${ruleId}, but no fixture currently names that rule in expectedRuleIds or absentRuleIds.`);
|
|
180
|
+
}
|
|
181
|
+
|
|
182
|
+
requireIncludes(frameworkGuide, guidePath, command, `${label} command`);
|
|
183
|
+
requireIncludes(cookbookReadme, cookbookReadmePath, command, `${label} command`);
|
|
184
|
+
}
|
|
185
|
+
|
|
186
|
+
for (const fixtureId of fixturesForRecipe) {
|
|
187
|
+
if (!fixtureIds.has(fixtureId)) {
|
|
188
|
+
failures.push(`${label}.fixtures references missing fixture ${fixtureId}.`);
|
|
189
|
+
}
|
|
190
|
+
|
|
191
|
+
requireIncludes(frameworkGuide, guidePath, fixtureId, `${label} fixture id`);
|
|
192
|
+
requireIncludes(cookbookReadme, cookbookReadmePath, fixtureId, `${label} fixture id`);
|
|
193
|
+
}
|
|
194
|
+
|
|
195
|
+
if (typeof recipe.id === "string") {
|
|
196
|
+
requireIncludes(frameworkGuide, guidePath, recipe.id, `${label} recipe id`);
|
|
197
|
+
requireIncludes(cookbookReadme, cookbookReadmePath, recipe.id, `${label} recipe id`);
|
|
198
|
+
}
|
|
199
|
+
|
|
200
|
+
if (typeof recipe.stack === "string") {
|
|
201
|
+
requireIncludes(frameworkGuide, guidePath, recipe.stack, `${label} stack`);
|
|
202
|
+
requireIncludes(cookbookReadme, cookbookReadmePath, recipe.stack, `${label} stack`);
|
|
203
|
+
}
|
|
204
|
+
}
|
|
205
|
+
|
|
206
|
+
if (failures.length > 0) {
|
|
207
|
+
console.error("Framework tuning check failed:");
|
|
208
|
+
for (const failure of failures) {
|
|
209
|
+
console.error(`- ${failure}`);
|
|
210
|
+
}
|
|
211
|
+
process.exit(1);
|
|
212
|
+
}
|
|
213
|
+
|
|
214
|
+
console.log(`Framework tuning check passed for ${Array.isArray(cookbook) ? cookbook.length : 0} recipes.`);
|
|
@@ -61,12 +61,14 @@ for (const path of [
|
|
|
61
61
|
"docs/RELEASE.md",
|
|
62
62
|
"docs/TRUSTED_PUBLISHING.md",
|
|
63
63
|
"docs/PRODUCTION_READINESS.md",
|
|
64
|
+
"docs/FRAMEWORK_TUNING.md",
|
|
64
65
|
"docs/MAINTAINER_TRIAGE.md",
|
|
65
66
|
`docs/RELEASE_NOTES_${tag}.md`,
|
|
66
67
|
"action.yml",
|
|
67
68
|
"scripts/check-promo-freshness.mjs",
|
|
68
69
|
"scripts/render-social-card.mjs",
|
|
69
70
|
"scripts/check-site.mjs",
|
|
71
|
+
"scripts/check-framework-tuning.mjs",
|
|
70
72
|
"scripts/check-fixtures.mjs",
|
|
71
73
|
"scripts/report-fixtures.mjs",
|
|
72
74
|
".github/ISSUE_TEMPLATE/bug_report.yml",
|
|
@@ -80,6 +82,8 @@ for (const path of [
|
|
|
80
82
|
"examples/github-code-scanning.yml",
|
|
81
83
|
"examples/ci/README.md",
|
|
82
84
|
"examples/presets/README.md",
|
|
85
|
+
"examples/tuning/README.md",
|
|
86
|
+
"examples/tuning/framework-tuning-cookbook.json",
|
|
83
87
|
"examples/fixtures/preset-review-cases.json",
|
|
84
88
|
"site/index.html"
|
|
85
89
|
]) {
|
|
@@ -99,6 +103,8 @@ requireText("README.md", /MAINTAINER_TRIAGE\.md/, "maintainer triage guide link"
|
|
|
99
103
|
requireText("README.md", /fixtures:check/, "fixture authoring check guidance");
|
|
100
104
|
requireText("README.md", /fixtures:report/, "fixture coverage report guidance");
|
|
101
105
|
requireText("README.md", /fixtures:report -- --markdown/, "Markdown fixture report guidance");
|
|
106
|
+
requireText("README.md", /FRAMEWORK_TUNING\.md/, "framework tuning guide link");
|
|
107
|
+
requireText("README.md", /examples\/tuning/, "framework tuning cookbook link");
|
|
102
108
|
requireText("README.md", /License: PolyForm Noncommercial/, "the noncommercial license badge");
|
|
103
109
|
requireText("docs/PRODUCTION_READINESS.md", /npm package/i, "npm package readiness");
|
|
104
110
|
requireText("docs/PRODUCTION_READINESS.md", /GitHub Action/i, "GitHub Action readiness");
|
|
@@ -112,9 +118,17 @@ requireText("docs/PRODUCTION_READINESS.md", /MAINTAINER_TRIAGE\.md/, "maintainer
|
|
|
112
118
|
requireText("docs/PRODUCTION_READINESS.md", /fixtures:check/, "fixture authoring check readiness");
|
|
113
119
|
requireText("docs/PRODUCTION_READINESS.md", /fixtures:report/, "fixture coverage report readiness");
|
|
114
120
|
requireText("docs/PRODUCTION_READINESS.md", /fixtures:report -- --markdown/, "Markdown fixture report readiness");
|
|
121
|
+
requireText("docs/PRODUCTION_READINESS.md", /framework:tuning:check/, "framework tuning cookbook readiness");
|
|
115
122
|
requireText("docs/PRODUCTION_READINESS.md", /quiet-pass/, "quiet-pass fixture readiness");
|
|
116
123
|
requireText("docs/CLI.md", /jester doctor --json/, "doctor JSON CLI docs");
|
|
117
124
|
requireText("docs/CLI.md", /quiet-pass fixture/, "quiet-pass fixture CLI docs");
|
|
125
|
+
requireText("docs/CLI.md", /FRAMEWORK_TUNING\.md/, "framework tuning CLI link");
|
|
126
|
+
requireText("docs/CLI.md", /examples\/tuning/, "framework tuning cookbook CLI link");
|
|
127
|
+
requireText("docs/FRAMEWORK_TUNING.md", /Next\.js/, "Next.js framework tuning example");
|
|
128
|
+
requireText("docs/FRAMEWORK_TUNING.md", /FastAPI/, "FastAPI framework tuning example");
|
|
129
|
+
requireText("docs/FRAMEWORK_TUNING.md", /Terraform/, "Terraform framework tuning example");
|
|
130
|
+
requireText("docs/FRAMEWORK_TUNING.md", /jester tune <rule-id> --json/, "framework tuning command guidance");
|
|
131
|
+
requireText("docs/FRAMEWORK_TUNING.md", /framework-tuning-cookbook\.json/, "framework tuning cookbook JSON link");
|
|
118
132
|
requireText("docs/MAINTAINER_TRIAGE.md", /doctor --json/, "doctor JSON triage prompt");
|
|
119
133
|
requireText("docs/MAINTAINER_TRIAGE.md", /tune <rule-id> --json/, "tune JSON triage prompt");
|
|
120
134
|
requireText("docs/MAINTAINER_TRIAGE.md", /preset-review-cases\.json/, "fixture suite link");
|
|
@@ -125,6 +139,10 @@ requireText("examples/fixtures/README.md", /Adding A Fixture From A Report/, "fi
|
|
|
125
139
|
requireText("examples/fixtures/README.md", /fixtures:check/, "fixture authoring check guidance");
|
|
126
140
|
requireText("examples/fixtures/README.md", /fixtures:report/, "fixture coverage report guidance");
|
|
127
141
|
requireText("examples/fixtures/README.md", /fixtures:report -- --markdown/, "Markdown fixture report guidance");
|
|
142
|
+
requireText("examples/tuning/README.md", /framework-tuning-cookbook\.json/, "framework tuning cookbook JSON link");
|
|
143
|
+
requireText("examples/tuning/README.md", /jester tune <rule-id> --json|jester tune [a-z0-9-]+ --json/, "framework tuning command guidance");
|
|
144
|
+
requireText("examples/tuning/framework-tuning-cookbook.json", /next-vite-public-config/, "Next/Vite tuning recipe");
|
|
145
|
+
requireText("examples/tuning/framework-tuning-cookbook.json", /ai-mcp-tooling/, "AI/MCP tuning recipe");
|
|
128
146
|
requireText("scripts/check-fixtures.mjs", /duplicated/, "duplicate fixture id check");
|
|
129
147
|
requireText("scripts/check-fixtures.mjs", /unsafeContentPatterns/, "unsafe fixture content checks");
|
|
130
148
|
forbidText("scripts/check-fixtures.mjs", /src\/config\.ts|src\/types\.ts/, "source-only fixture validator dependencies");
|
|
@@ -134,14 +152,19 @@ requireText("scripts/report-fixtures.mjs", /quietPassRuleCoverage/, "quiet-pass
|
|
|
134
152
|
requireText("scripts/report-fixtures.mjs", /presetKindGaps/, "preset and kind gap report");
|
|
135
153
|
requireText("scripts/report-fixtures.mjs", /--markdown/, "Markdown fixture report output");
|
|
136
154
|
forbidText("scripts/report-fixtures.mjs", /src\/config\.ts|src\/types\.ts/, "source-only fixture report dependencies");
|
|
155
|
+
requireText("scripts/check-framework-tuning.mjs", /framework-tuning-cookbook\.json/, "framework tuning cookbook check");
|
|
156
|
+
requireText("scripts/check-framework-tuning.mjs", /preset-review-cases\.json/, "framework tuning fixture alignment");
|
|
157
|
+
requireText("scripts/check-framework-tuning.mjs", /unsafeContentPatterns/, "unsafe tuning content checks");
|
|
137
158
|
requireText("package.json", /"fixtures:check": "node scripts\/check-fixtures\.mjs"/, "fixture authoring check script");
|
|
138
159
|
requireText("package.json", /"fixtures:report": "node scripts\/report-fixtures\.mjs"/, "fixture coverage report script");
|
|
160
|
+
requireText("package.json", /"framework:tuning:check": "node scripts\/check-framework-tuning\.mjs"/, "framework tuning cookbook check script");
|
|
139
161
|
requireText("package.json", /"promo:card": "node scripts\/render-social-card\.mjs"/, "social card render script");
|
|
140
162
|
requireText("package.json", /"promo:card:check": "node scripts\/render-social-card\.mjs --check"/, "social card stale check script");
|
|
141
163
|
requireText("package.json", /"promo:check": "node scripts\/check-promo-freshness\.mjs"/, "promo freshness check script");
|
|
142
164
|
requireText("package.json", /"site:check": "node scripts\/check-site\.mjs"/, "site check script");
|
|
143
165
|
requireText("package.json", /npm run fixtures:check/, "fixture authoring check in npm test");
|
|
144
166
|
requireText("package.json", /npm run fixtures:report/, "fixture coverage report in npm test");
|
|
167
|
+
requireText("package.json", /npm run framework:tuning:check/, "framework tuning cookbook check in npm test");
|
|
145
168
|
requireText("package.json", /npm run promo:check/, "promo freshness check in npm test");
|
|
146
169
|
requireText("package.json", /npm run site:check/, "site check in npm test");
|
|
147
170
|
requireText("scripts/check-promo-freshness.mjs", /--require-package-version/, "optional strict package-version promo check");
|