memento-mori-jester 0.1.67 → 0.1.68
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +6 -0
- package/ROADMAP.md +2 -1
- package/docs/DEMO.md +4 -4
- package/docs/RELEASE_NOTES_v0.1.68.md +35 -0
- package/examples/fixtures/README.md +1 -0
- package/examples/fixtures/preset-review-cases.json +50 -0
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,12 @@ All notable changes to Memento Mori Jester are tracked here.
|
|
|
4
4
|
|
|
5
5
|
## Unreleased
|
|
6
6
|
|
|
7
|
+
## 0.1.68
|
|
8
|
+
|
|
9
|
+
- Added AI preset fixtures for user-controlled tool dispatch from request body and URL search parameter inputs.
|
|
10
|
+
- Added quiet-pass AI near-misses for explicit tool allowlists and schema-validated model data, growing the corpus to 170 fixtures.
|
|
11
|
+
- Refreshed demo, roadmap, fixture docs, and release notes for the AI tool-dispatch coverage pass.
|
|
12
|
+
|
|
7
13
|
## 0.1.67
|
|
8
14
|
|
|
9
15
|
- Added six real-world quiet-pass fixtures for security, web, node, and python preset slices, growing the corpus to 166 fixtures.
|
package/ROADMAP.md
CHANGED
|
@@ -6,6 +6,7 @@ Memento Mori Jester is usable today as a CLI, MCP server, GitHub Action, and git
|
|
|
6
6
|
|
|
7
7
|
## Recently Shipped
|
|
8
8
|
|
|
9
|
+
- AI tool-dispatch fixture curation in v0.1.68, adding request-body and URL-parameter caution examples plus allowlist/schema quiet-pass boundaries.
|
|
9
10
|
- Security/web/node/python preset fixture curation in v0.1.67, adding real-world quiet-pass examples while keeping thin, quiet-pass, feasible pass-case, and preset-kind gaps at zero.
|
|
10
11
|
- Real-world low-count preset fixture batch in v0.1.66, adding node, python, infra, and AI examples while keeping thin, quiet-pass, and feasible pass-case gaps at zero.
|
|
11
12
|
- Feasible pass-case fixture curation in v0.1.65, adding matched-pass examples for low-severity tone/planning rules and stopping curation from asking for impossible pass cases on hard rules.
|
|
@@ -56,7 +57,7 @@ Memento Mori Jester is usable today as a CLI, MCP server, GitHub Action, and git
|
|
|
56
57
|
|
|
57
58
|
## Product Ideas
|
|
58
59
|
|
|
59
|
-
- Collect real-world reports for the next lowest-count preset slices:
|
|
60
|
+
- Collect real-world reports for the next lowest-count preset slices: node, python, security, then web.
|
|
60
61
|
- Add more framework-specific false-positive examples from real reports so tuning guidance keeps getting sharper.
|
|
61
62
|
- Add a Markdown export for fixture reports so maintainers can paste coverage snapshots into issues or release notes.
|
|
62
63
|
|
package/docs/DEMO.md
CHANGED
|
@@ -192,8 +192,8 @@ Project config: none loaded
|
|
|
192
192
|
Fixture tuning evidence:
|
|
193
193
|
Support: limited
|
|
194
194
|
Confidence: medium
|
|
195
|
-
Total fixtures checked:
|
|
196
|
-
Weighted fixtures checked:
|
|
195
|
+
Total fixtures checked: 170
|
|
196
|
+
Weighted fixtures checked: 325.9
|
|
197
197
|
Matching fixtures: 11
|
|
198
198
|
Weighted matches: 23
|
|
199
199
|
Expected-match weight: 18
|
|
@@ -202,7 +202,7 @@ Edge-case matches: 0
|
|
|
202
202
|
Quiet-pass fixtures: 5
|
|
203
203
|
Quiet-pass weight: 3.6
|
|
204
204
|
By kind: command 0, plan 5, diff 5, final 1
|
|
205
|
-
Fixture coverage: 11/
|
|
205
|
+
Fixture coverage: 11/170 (7.1% weighted)
|
|
206
206
|
By verdict: pass 0, caution 3, block 8
|
|
207
207
|
Matched fixture samples:
|
|
208
208
|
infra-public-ingress-block: Public ingress should block in low-risk-tolerance infra repos.
|
|
@@ -353,7 +353,7 @@ Preset packs:
|
|
|
353
353
|
|
|
354
354
|
## 13. Review Fixtures
|
|
355
355
|
|
|
356
|
-
The fixture suite in `examples/fixtures/preset-review-cases.json` captures small real-usage examples with expected `pass`, `caution`, or `block` verdicts. It also includes matched-pass examples for low-severity rules, quiet-pass `absentRuleIds` examples that prove noisy rules stay silent for safe near-misses, stack-specific coverage for every built-in preset, quiet-pass boundaries across built-in, structural, custom, and preset/config-derived rules, second firing examples for preset blocked-command rules, second examples for AI/API, framework custom, built-in, and configured sensitive-domain rules, and real-world low-count preset examples across node, python, web, infra, AI, and security slices. These examples are run by `npm test`, so preset tuning changes stay visible.
|
|
356
|
+
The fixture suite in `examples/fixtures/preset-review-cases.json` captures small real-usage examples with expected `pass`, `caution`, or `block` verdicts. It also includes matched-pass examples for low-severity rules, quiet-pass `absentRuleIds` examples that prove noisy rules stay silent for safe near-misses, stack-specific coverage for every built-in preset, quiet-pass boundaries across built-in, structural, custom, and preset/config-derived rules, second firing examples for preset blocked-command rules, second examples for AI/API, framework custom, built-in, and configured sensitive-domain rules, AI tool-dispatch examples with safe allowlist/schema boundaries, and real-world low-count preset examples across node, python, web, infra, AI, and security slices. These examples are run by `npm test`, so preset tuning changes stay visible.
|
|
357
357
|
|
|
358
358
|
Maintainers can run `npm run fixtures:report` to see coverage by verdict, kind, preset, rule family, and preset slice. The report also includes a `Curation next` section that points at the next useful fixture batch, such as thin rules, feasible pass-case evidence, rule-family gaps, or lower-count presets.
|
|
359
359
|
|
|
@@ -0,0 +1,35 @@
|
|
|
1
|
+
# Memento Mori Jester v0.1.68
|
|
2
|
+
|
|
3
|
+
This release strengthens the AI preset fixture slice with concrete tool-dispatch examples. It keeps review behavior unchanged and only improves the evidence used by fixture reports and tuning context.
|
|
4
|
+
|
|
5
|
+
## What Changed
|
|
6
|
+
|
|
7
|
+
- Added 4 fixture cases, growing the corpus from 166 to 170 fixtures.
|
|
8
|
+
- Added two caution fixtures for `custom-ai-user-controlled-tool-dispatch`:
|
|
9
|
+
- Tool names taken from request body input.
|
|
10
|
+
- Tool names taken from URL search parameters.
|
|
11
|
+
- Added quiet-pass AI near-misses for:
|
|
12
|
+
- Explicit tool allowlists.
|
|
13
|
+
- Schema validation of model data instead of executing model output.
|
|
14
|
+
- Kept thin rule coverage, quiet-pass gaps, feasible pass-case gaps, and preset/kind gaps at zero.
|
|
15
|
+
|
|
16
|
+
## Public Interface
|
|
17
|
+
|
|
18
|
+
- No CLI command changes.
|
|
19
|
+
- No config schema changes.
|
|
20
|
+
- No rule matching, scoring, or verdict behavior changes.
|
|
21
|
+
- No MCP, playground, GitHub Action, or npm publishing changes.
|
|
22
|
+
|
|
23
|
+
## Release Validation
|
|
24
|
+
|
|
25
|
+
```powershell
|
|
26
|
+
npm.cmd test
|
|
27
|
+
npm.cmd run demo:svg:check
|
|
28
|
+
npm.cmd run fixtures:report
|
|
29
|
+
npm.cmd run fixtures:report -- --json
|
|
30
|
+
npm.cmd run pack:dry
|
|
31
|
+
git diff --check
|
|
32
|
+
node .\dist\cli.js tune coverage --no-config
|
|
33
|
+
node .\dist\cli.js tune risky-domain --json --no-config
|
|
34
|
+
git diff | node .\dist\cli.js diff --fail-on block --subject "v0.1.68 AI tool-dispatch fixtures"
|
|
35
|
+
```
|
|
@@ -14,6 +14,7 @@ Maintainer triage guidance lives in [docs/MAINTAINER_TRIAGE.md](../../docs/MAINT
|
|
|
14
14
|
- API preset CORS, raw SQL, webhook, and migration-command risks.
|
|
15
15
|
- Infra preset public exposure, IAM wildcard, and destructive command risks.
|
|
16
16
|
- AI preset eval-skipping and model-output execution risks.
|
|
17
|
+
- AI preset user-controlled tool-dispatch risks and safe allowlist/schema boundaries.
|
|
17
18
|
- Quiet-pass boundaries for thin custom, configured sensitive-domain, and preset blocked-command rules.
|
|
18
19
|
- Quiet-pass boundaries for built-in and structural rules such as missing verification, TypeScript suppressions, large removals, wildcard operations, destructive commands, and untested finals.
|
|
19
20
|
- Matched-pass examples for low-severity rules where a single finding should stay below caution.
|
|
@@ -2145,5 +2145,55 @@
|
|
|
2145
2145
|
"custom-python-eval-exec",
|
|
2146
2146
|
"custom-python-pickle-load"
|
|
2147
2147
|
]
|
|
2148
|
+
},
|
|
2149
|
+
{
|
|
2150
|
+
"id": "ai-user-controlled-tool-diff-caution",
|
|
2151
|
+
"preset": "ai",
|
|
2152
|
+
"kind": "diff",
|
|
2153
|
+
"description": "Tool dispatch from request body input should warn in AI preset repos.",
|
|
2154
|
+
"content": "diff --git a/src/tools.ts b/src/tools.ts\n--- a/src/tools.ts\n+++ b/src/tools.ts\n@@ -1 +1,2 @@\n+const selectedTool = req.body.tool; await callTool(selectedTool, req.body.args);\n",
|
|
2155
|
+
"expectedVerdict": "caution",
|
|
2156
|
+
"weight": 2,
|
|
2157
|
+
"expectedRuleIds": [
|
|
2158
|
+
"custom-ai-user-controlled-tool-dispatch"
|
|
2159
|
+
]
|
|
2160
|
+
},
|
|
2161
|
+
{
|
|
2162
|
+
"id": "ai-searchparams-tool-diff-caution",
|
|
2163
|
+
"preset": "ai",
|
|
2164
|
+
"kind": "diff",
|
|
2165
|
+
"description": "Tool dispatch from URL search parameters should warn without an allowlist.",
|
|
2166
|
+
"content": "diff --git a/src/tool-router.ts b/src/tool-router.ts\n--- a/src/tool-router.ts\n+++ b/src/tool-router.ts\n@@ -1 +1,2 @@\n+const toolName = searchParams.get(\"name\"); await callTool(toolName, {});\n",
|
|
2167
|
+
"expectedVerdict": "caution",
|
|
2168
|
+
"weight": 2,
|
|
2169
|
+
"expectedRuleIds": [
|
|
2170
|
+
"custom-ai-user-controlled-tool-dispatch"
|
|
2171
|
+
]
|
|
2172
|
+
},
|
|
2173
|
+
{
|
|
2174
|
+
"id": "ai-tool-allowlist-diff-pass",
|
|
2175
|
+
"preset": "ai",
|
|
2176
|
+
"kind": "diff",
|
|
2177
|
+
"description": "Explicit AI tool allowlists should not trip user-controlled dispatch checks.",
|
|
2178
|
+
"content": "diff --git a/src/tools.ts b/src/tools.ts\n--- a/src/tools.ts\n+++ b/src/tools.ts\n@@ -1 +1,2 @@\n+const allowedTools = new Set([\"search_docs\", \"summarize\"]);\n",
|
|
2179
|
+
"expectedVerdict": "pass",
|
|
2180
|
+
"weight": 2,
|
|
2181
|
+
"expectedRuleIds": [],
|
|
2182
|
+
"absentRuleIds": [
|
|
2183
|
+
"custom-ai-user-controlled-tool-dispatch"
|
|
2184
|
+
]
|
|
2185
|
+
},
|
|
2186
|
+
{
|
|
2187
|
+
"id": "ai-schema-validation-diff-pass",
|
|
2188
|
+
"preset": "ai",
|
|
2189
|
+
"kind": "diff",
|
|
2190
|
+
"description": "Schema validation of model data should stay quiet around model-output execution checks.",
|
|
2191
|
+
"content": "diff --git a/src/output.ts b/src/output.ts\n--- a/src/output.ts\n+++ b/src/output.ts\n@@ -1 +1,2 @@\n+const parsed = ResponseSchema.parse(responseJson);\n",
|
|
2192
|
+
"expectedVerdict": "pass",
|
|
2193
|
+
"weight": 2,
|
|
2194
|
+
"expectedRuleIds": [],
|
|
2195
|
+
"absentRuleIds": [
|
|
2196
|
+
"custom-ai-model-output-execution"
|
|
2197
|
+
]
|
|
2148
2198
|
}
|
|
2149
2199
|
]
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "memento-mori-jester",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.68",
|
|
4
4
|
"description": "A local court-jester sidecar for AI coding agents: review plans, commands, diffs, and final claims before they get too pleased with themselves.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"repository": {
|