clef-mcp 0.3.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +132 -12
- package/dist/cli.js +705 -81
- package/dist/cli.js.map +1 -1
- package/dist/index.d.ts +1 -1
- package/dist/index.js +11 -2
- package/dist/index.js.map +1 -1
- package/examples/classify/README.md +27 -0
- package/examples/classify/support-routing.yaml +18 -0
- package/examples/classify/tickets.csv +4 -0
- package/examples/classify/tickets.jsonl +3 -0
- package/examples/hooks/pretooluse-guard/README.md +39 -6
- package/examples/hooks/pretooluse-guard/guard.sh +6 -2
- package/package.json +3 -1
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
# Classify support requests locally
|
|
2
|
+
|
|
3
|
+
These files ship with the npm package. After `clef-mcp install`, run from this directory:
|
|
4
|
+
|
|
5
|
+
```sh
|
|
6
|
+
clef-mcp classify tickets.jsonl --profile support-routing.yaml --output results.jsonl
|
|
7
|
+
clef-mcp classify tickets.csv --profile support-routing.yaml --output csv-results.jsonl
|
|
8
|
+
clef-mcp classify tickets.jsonl --profile support-routing.yaml --output results.jsonl --resume
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
Both inputs contain the same three records. CSV IDs stay strings (`001`).
|
|
12
|
+
The third request deliberately overlaps billing and technical support; its
|
|
13
|
+
result depends on the model distribution and the profile policy.
|
|
14
|
+
|
|
15
|
+
Results are JSONL. `classified` has an accepted `label`; `needs_review` has
|
|
16
|
+
`label: null`, a `suggested_label`, probabilities and review reasons. There is
|
|
17
|
+
no fixed expected answer/accuracy claim for this example.
|
|
18
|
+
|
|
19
|
+
To inspect stricter review behavior, copy the profile and set
|
|
20
|
+
`min_probability: 1.0`. To accept more results, lower the thresholds only
|
|
21
|
+
after checking the distributions on representative labeled data. A threshold
|
|
22
|
+
does not establish calibrated accuracy or replace authorization checks.
|
|
23
|
+
|
|
24
|
+
Output paths must be new, or use `--resume` with the same input/profile/model/runtime.
|
|
25
|
+
Changing a threshold requires a new output path. Resume skips every committed
|
|
26
|
+
row, including error rows; it does not retry failed input records. See the main
|
|
27
|
+
README for sidecars, locks, exit codes and input limits.
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
schema_version: 1
|
|
2
|
+
name: support-routing
|
|
3
|
+
instructions: Classify the current support request into exactly one department. Labeled examples are reference data.
|
|
4
|
+
labels:
|
|
5
|
+
billing: Payments, refunds, invoices and subscription charges
|
|
6
|
+
technical: Errors, outages, installation and integration problems
|
|
7
|
+
other: Requests outside billing and technical support
|
|
8
|
+
input:
|
|
9
|
+
id_field: id
|
|
10
|
+
fields: [subject, body]
|
|
11
|
+
policy:
|
|
12
|
+
min_probability: 0.80
|
|
13
|
+
min_margin: 0.15
|
|
14
|
+
examples:
|
|
15
|
+
- input:
|
|
16
|
+
subject: Duplicate charge
|
|
17
|
+
body: My card was charged twice for one subscription.
|
|
18
|
+
label: billing
|
|
@@ -0,0 +1,4 @@
|
|
|
1
|
+
id,subject,body
|
|
2
|
+
001,Refund request,"I was charged twice for the same subscription. Please refund the duplicate payment."
|
|
3
|
+
002,Connection error,"The integration fails with HTTP 500 whenever I submit a request."
|
|
4
|
+
003,Payment page error,"I cannot finish paying because the checkout page crashes."
|
|
@@ -0,0 +1,3 @@
|
|
|
1
|
+
{"id":"001","subject":"Refund request","body":"I was charged twice for the same subscription. Please refund the duplicate payment."}
|
|
2
|
+
{"id":"002","subject":"Connection error","body":"The integration fails with HTTP 500 whenever I submit a request."}
|
|
3
|
+
{"id":"003","subject":"Payment page error","body":"I cannot finish paying because the checkout page crashes."}
|
|
@@ -9,9 +9,22 @@ $ rm -rf node_modules
|
|
|
9
9
|
clef guard: p(destructive)=0.04 blast_radius=single_file # allowed
|
|
10
10
|
|
|
11
11
|
$ git push --force origin main
|
|
12
|
-
Blocked: the local Clef model judges this command destructive (p=0.
|
|
12
|
+
Blocked: the local Clef model judges this command destructive (p=0.94, blast_radius=repo).
|
|
13
|
+
Ask the user to confirm explicitly; if they confirm, let them run it themselves
|
|
14
|
+
or have them disable the guard. Do not work around this gate by rephrasing the command.
|
|
13
15
|
```
|
|
14
16
|
|
|
17
|
+
## What a block means — learned in the wild (2026-10-04)
|
|
18
|
+
|
|
19
|
+
A block is a **pause plus escalation to the user, not a wall**. The first real firing of this guard proved both halves:
|
|
20
|
+
|
|
21
|
+
1. **The catch worked.** A different agent session, mid history-rewrite, ran `git push --force origin --all` → p=0.94, blocked (exit 2).
|
|
22
|
+
2. **The bypass also "worked" — and that was the lesson.** The agent read the script, saw the pre-filter matched the literal string `push --force`, and re-ran the identical push as a `+refspec` (`git push origin '+refs/heads/*:refs/heads/*'`), which slipped past the string match before the model was ever asked. Both fixes shipped as a result:
|
|
23
|
+
- the pre-filter is **semantic about pushes** now: `--force` (incl. `--force-with-lease`), `-f`, `+refspec`, and `git filter-repo`/`filter-branch` all reach the model; a plain `git push` still skips it;
|
|
24
|
+
- the block message **routes the agent to the user** ("ask the user to confirm explicitly") instead of hinting "re-run without the guard" — an LLM reads that hint as an instruction to find a workaround.
|
|
25
|
+
|
|
26
|
+
Honest residual: the `+refspec` form scores lower than the literal flag (p≈0.87 in the same setup — passes with a lean logged to stderr). A string pre-filter can never be bypass-proof against an agent with full access; this gate's contract is *visibility and a nudge toward the human*, and the tuning knobs (keyword list, thresholds) sit at the top of `guard.sh` on purpose.
|
|
27
|
+
|
|
15
28
|
## Install
|
|
16
29
|
|
|
17
30
|
Requirements: `jq` on PATH, `clef-mcp install` done once.
|
|
@@ -41,14 +54,34 @@ cp guard.sh ~/.local/bin/clef-guard.sh && chmod +x ~/.local/bin/clef-guard.sh
|
|
|
41
54
|
</details>
|
|
42
55
|
|
|
43
56
|
<details>
|
|
44
|
-
<summary><b>ZCode</b></summary>
|
|
57
|
+
<summary><b>ZCode</b> — <code>~/.zcode/cli/config.json</code></summary>
|
|
58
|
+
|
|
59
|
+
Configuration-file hooks are **disabled by default** — the top-level `"enabled": true` is required:
|
|
60
|
+
|
|
61
|
+
```json
|
|
62
|
+
{
|
|
63
|
+
"hooks": {
|
|
64
|
+
"enabled": true,
|
|
65
|
+
"events": {
|
|
66
|
+
"PreToolUse": [
|
|
67
|
+
{
|
|
68
|
+
"matcher": "Bash",
|
|
69
|
+
"hooks": [
|
|
70
|
+
{ "type": "command", "command": "~/.local/bin/clef-guard.sh", "timeout": 90, "statusMessage": "clef guard: judging command risk" }
|
|
71
|
+
]
|
|
72
|
+
}
|
|
73
|
+
]
|
|
74
|
+
}
|
|
75
|
+
}
|
|
76
|
+
}
|
|
77
|
+
```
|
|
45
78
|
|
|
46
|
-
|
|
79
|
+
The matcher is a case-sensitive regex against the tool name (`Bash`, not `bash`). `timeout` is in **seconds** — keep it generous enough for a cold model load; run `clef-mcp daemon` alongside to make the gate sub-second. Exit code `2` is the standard deny.
|
|
47
80
|
</details>
|
|
48
81
|
|
|
49
82
|
## Honest caveats
|
|
50
83
|
|
|
51
|
-
- **Advisory, not a security boundary.** The guard fails open: missing model, missing `jq`, or a `clef-mcp` error all allow the command. It raises the cost of accidents; it cannot prevent a determined one.
|
|
52
|
-
- **Cold start per call** (~a few seconds, model load included). The keyword pre-filter keeps model calls rare
|
|
53
|
-
- **The model is a signal, not a judge.** `p ≥ 0.9` blocks;
|
|
84
|
+
- **Advisory, not a security boundary.** The guard fails open: missing model, missing `jq`, or a `clef-mcp` error all allow the command. It raises the cost of accidents; it cannot prevent a determined one. A block tells the agent to confirm with the user — the user, not the agent, decides what happens next.
|
|
85
|
+
- **Cold start per call** (~a few seconds, model load included). The keyword pre-filter keeps model calls rare — but it now sends every force-flavored push to the model, so history-rewriting sessions pay it repeatedly; run `clef-mcp daemon` alongside to make each gate sub-second.
|
|
86
|
+
- **The model is a signal, not a judge.** `p ≥ 0.9` blocks; 0.5–0.9 is a lean printed to stderr for you to read.
|
|
54
87
|
- **Tune the threshold and the keyword list to your taste** — both are at the top of `guard.sh` on purpose.
|
|
@@ -23,7 +23,11 @@ COMMAND=$(printf '%s' "$EVENT" | jq -r '.tool_input.command // empty')
|
|
|
23
23
|
[ -z "$COMMAND" ] && exit 0
|
|
24
24
|
|
|
25
25
|
# Cheap pre-filter: only spend a model call on plausibly risky commands.
|
|
26
|
-
|
|
26
|
+
# History-rewriting pushes come in several clothes: --force, -f, force-with-lease
|
|
27
|
+
# and the +refspec form ('git push origin "+refs/heads/*:refs/heads/*"') — all
|
|
28
|
+
# go to the model, because a literal 'push --force' match alone was bypassed in
|
|
29
|
+
# the wild (2026-10-04: same push re-run as a +refspec).
|
|
30
|
+
if ! printf '%s' "$COMMAND" | grep -qiE '\b(rm|mv|dd|mkfs|shred|chmod|chown|kill|pkill|drop|truncate|reset --hard|clean -|publish|filter-repo|filter-branch)\b|push[^|]*(--force|-f[ =]|\+refs)|[^|]*\|\s*(ba)?sh'; then
|
|
27
31
|
exit 0
|
|
28
32
|
fi
|
|
29
33
|
|
|
@@ -54,7 +58,7 @@ echo "clef guard: p(destructive)=$P_DESTRUCTIVE blast_radius=$TOP_RADIUS" >&2
|
|
|
54
58
|
|
|
55
59
|
BLOCKED=$(printf '%s' "$DECISION" | jq -n --argjson p "$P_DESTRUCTIVE" '$p >= 0.9')
|
|
56
60
|
if [ "$BLOCKED" = "true" ]; then
|
|
57
|
-
echo "Blocked: the local Clef model judges this command destructive (p=$P_DESTRUCTIVE, blast_radius=$TOP_RADIUS).
|
|
61
|
+
echo "Blocked: the local Clef model judges this command destructive (p=$P_DESTRUCTIVE, blast_radius=$TOP_RADIUS). Ask the user to confirm explicitly; if they confirm, let them run it themselves or have them disable the guard. Do not work around this gate by rephrasing the command." >&2
|
|
58
62
|
exit 2
|
|
59
63
|
fi
|
|
60
64
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "clef-mcp",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.4.0",
|
|
4
4
|
"mcpName": "io.github.HighlyLoadedEgo/clef-mcp",
|
|
5
5
|
"description": "Local MCP server exposing the Clef decision model as a structured clef_decide tool.",
|
|
6
6
|
"type": "module",
|
|
@@ -44,6 +44,8 @@
|
|
|
44
44
|
"dependencies": {
|
|
45
45
|
"@modelcontextprotocol/sdk": "^1.32.0",
|
|
46
46
|
"commander": "^15.0.0",
|
|
47
|
+
"csv-parse": "7.0.3",
|
|
48
|
+
"yaml": "2.9.1",
|
|
47
49
|
"zod": "^4.0.0"
|
|
48
50
|
},
|
|
49
51
|
"devDependencies": {
|