vigiles 5.0.1 → 5.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +15 -9
- package/dist/adapters/claude-code/adapter.js +1 -0
- package/dist/adapters/claude-code/agent-runtime.d.ts +30 -6
- package/dist/adapters/claude-code/agent-runtime.js +66 -37
- package/dist/adapters/claude-code/dialect.js +37 -2
- package/dist/adapters/claude-code/effect-region.d.ts +9 -0
- package/dist/adapters/claude-code/effect-region.js +45 -0
- package/dist/adapters/claude-code/layout.js +3 -0
- package/dist/adapters/claude-code/skill-runtime.d.ts +25 -0
- package/dist/adapters/claude-code/skill-runtime.js +48 -0
- package/dist/adapters/codex/adapter.js +3 -0
- package/dist/adapters/codex/eval.d.ts +94 -0
- package/dist/adapters/codex/eval.js +227 -0
- package/dist/adapters/codex/layout.js +3 -0
- package/dist/adapters/opencode/adapter.js +1 -0
- package/dist/adapters/opencode/layout.js +3 -0
- package/dist/check.d.ts +8 -0
- package/dist/check.js +27 -3
- package/dist/cli.js +712 -21
- package/dist/codex.d.ts +1 -0
- package/dist/codex.js +3 -0
- package/dist/core/adapter.d.ts +10 -0
- package/dist/core/bash-effects.d.ts +41 -0
- package/dist/core/bash-effects.js +405 -0
- package/dist/core/compile.d.ts +3 -1
- package/dist/core/compile.js +169 -74
- package/dist/core/description-overlap.d.ts +27 -0
- package/dist/core/description-overlap.js +53 -0
- package/dist/core/dialect.d.ts +18 -0
- package/dist/core/effects.d.ts +172 -0
- package/dist/core/effects.js +245 -0
- package/dist/core/frontmatter-read.d.ts +25 -0
- package/dist/core/frontmatter-read.js +138 -0
- package/dist/core/hook-events.d.ts +34 -0
- package/dist/core/hook-events.js +48 -0
- package/dist/core/layout.d.ts +6 -0
- package/dist/core/mcp-config.d.ts +20 -0
- package/dist/core/mcp-config.js +40 -0
- package/dist/core/mcp-hook.d.ts +35 -0
- package/dist/core/mcp-hook.js +70 -0
- package/dist/core/mcp-tool.d.ts +50 -0
- package/dist/core/mcp-tool.js +61 -0
- package/dist/core/orphans.js +21 -0
- package/dist/core/spec.d.ts +142 -3
- package/dist/core/spec.js +48 -0
- package/dist/core/tool-contract.d.ts +68 -0
- package/dist/core/tool-contract.js +113 -0
- package/dist/core/types.d.ts +91 -2
- package/dist/core/validate.js +23 -1
- package/dist/eval.d.ts +69 -13
- package/dist/eval.js +106 -51
- package/dist/harness-test.d.ts +7 -0
- package/dist/harness-test.js +19 -7
- package/dist/leaderboard.d.ts +2 -0
- package/dist/leaderboard.js +63 -3
- package/dist/optimize.d.ts +74 -0
- package/dist/optimize.js +94 -0
- package/dist/plugin-loader.d.ts +1 -0
- package/dist/plugin-loader.js +71 -18
- package/dist/scaffold-test.d.ts +30 -0
- package/dist/scaffold-test.js +158 -0
- package/dist/scan-behavioral.d.ts +73 -0
- package/dist/scan-behavioral.js +150 -0
- package/dist/scan.d.ts +166 -1
- package/dist/scan.js +622 -55
- package/dist/score-explainer.d.ts +69 -0
- package/dist/score-explainer.js +169 -0
- package/dist/test-coverage.d.ts +7 -0
- package/dist/test-coverage.js +39 -24
- package/package.json +2 -1
- package/skills/{migrate-to-spec → adopt-spec}/SKILL.md +4 -6
- package/skills/edit-spec/SKILL.md +1 -1
package/README.md
CHANGED
|
@@ -30,19 +30,18 @@ deterministic layer for it, and does two independent things — adopt either, or
|
|
|
30
30
|
| **🧪 Test** | Hooks, skills, and subagents are code. vigiles tests they _do their job_ — and almost all of it is **deterministic, no API key**; the real-model evals run on your **Claude subscription**, not metered tokens. **[→](docs/harness-testing.md)** |
|
|
31
31
|
|
|
32
32
|
Pick the one that hurts today. **Works with Claude Code and Codex**
|
|
33
|
-
([`vigiles/codex`](docs/harnesses.md))
|
|
34
|
-
[
|
|
33
|
+
([`vigiles/codex`](docs/harnesses.md)), and you can
|
|
34
|
+
[teach it your own harness](docs/authoring-an-adapter.md).
|
|
35
35
|
|
|
36
36
|
## Quick start
|
|
37
37
|
|
|
38
38
|
**Paste into Claude Code or Codex:**
|
|
39
39
|
|
|
40
40
|
```text
|
|
41
|
-
|
|
42
|
-
references and show me what's stale, then write
|
|
43
|
-
of my hooks or skills.
|
|
44
|
-
|
|
45
|
-
whether to enforce strictly (--strict).
|
|
41
|
+
Set up vigiles in this repo with good defaults (lint + test, non-interactive).
|
|
42
|
+
Verify my CLAUDE.md / AGENTS.md references and show me what's stale, then write
|
|
43
|
+
and run a harness test for one of my hooks or skills. Ask me first before gating
|
|
44
|
+
it in CI, adding a real-model eval, or enforcing strictly (--strict).
|
|
46
45
|
```
|
|
47
46
|
|
|
48
47
|
Or do it yourself:
|
|
@@ -95,6 +94,12 @@ File paths, scripts, and code symbols — plus linter rules across **7 catalogs*
|
|
|
95
94
|
step up to a typed `.spec.ts` (compiled to CLAUDE.md, compiler-grade) when you want
|
|
96
95
|
it. **[Full guide →](docs/verifying-instruction-files.md)**
|
|
97
96
|
|
|
97
|
+
**Same cross-reference, any plugin.** `npx vigiles scan` checks a plugin's
|
|
98
|
+
_contracts_ — every subagent tool, `mcp__server__tool`, `mcp_tool` hook, hook
|
|
99
|
+
event, and script path actually **exists and resolves**, not just parses (valid
|
|
100
|
+
YAML ≠ a tool that's real). A **superset of Anthropic's `claude plugin validate`**,
|
|
101
|
+
no key. **[Audit any plugin →](docs/cli.md#scan-dir)**
|
|
102
|
+
|
|
98
103
|
## ② Test — does your harness do its job?
|
|
99
104
|
|
|
100
105
|
A hook can be wired wrong. A skill's description can fail to trigger — or hijack
|
|
@@ -132,8 +137,9 @@ That's why you can eval your harness on every change, not just once.
|
|
|
132
137
|
|
|
133
138
|
## More
|
|
134
139
|
|
|
135
|
-
- **[
|
|
136
|
-
- **[
|
|
140
|
+
- **[Plugin health leaderboard →](docs/cli.md#scan-dir)** — point `scan` at a marketplace (e.g. `wshobson/agents`) and it ranks every plugin by structural health (0–100, A–F), worst issues first — still **no key**. Add `--trigger` for the model-gated column: do the skills actually fire?
|
|
141
|
+
- **[CLI & GitHub Action →](docs/cli.md)** — every command, the Action (inputs / output / versioning), and the Claude Code plugin. The full **[lint rules matrix →](docs/verifying-instruction-files.md#the-validation-rules--the-full-matrix)** lives with the linting guide.
|
|
142
|
+
- **[Skills →](docs/skills.md)** — consumer skills installed as a Claude Code plugin: `/plugin marketplace add zernie/vigiles` then `/plugin install vigiles@vigiles` (or let `vigiles init` do it). The model-invocable ones (`test-harness`, `strengthen`, `edit-spec`) fire on their own — ask _"test my skills"_, _"strengthen my rules"_, or _"add a rule to CLAUDE.md"_ and the agent reaches for them; `adopt-spec` and `linter-docs` are user-invoked.
|
|
137
143
|
- **[Docs index →](docs/README.md)** · **[Research →](research/README.md)** · **[Related tools →](docs/related-tools.md)** (ast-grep, Dependency Cruiser, Ruler, rulesync).
|
|
138
144
|
- Companion to [Feedback Loop Is All You Need](https://zernie.com/blog/feedback-loop-is-all-you-need).
|
|
139
145
|
|
|
@@ -22,6 +22,7 @@
|
|
|
22
22
|
* (mirrors `.vigiles/active-skill.json`). The decision logic below is
|
|
23
23
|
* harness-agnostic and fully testable.
|
|
24
24
|
*/
|
|
25
|
+
import type { PurityLevel } from "../../core/effects.js";
|
|
25
26
|
/**
|
|
26
27
|
* Parse an agent's allowed-tools contract from its compiled markdown.
|
|
27
28
|
*
|
|
@@ -31,6 +32,24 @@
|
|
|
31
32
|
* an empty list (`tools:` with nothing after it) means "no tools allowed".
|
|
32
33
|
*/
|
|
33
34
|
export declare function parseAgentTools(markdown: string): string[] | null;
|
|
35
|
+
/**
|
|
36
|
+
* Parse a comma/array tool list under an arbitrary frontmatter `key` (e.g.
|
|
37
|
+
* `tools:` or `disallowedTools:`) via the shared lenient reader
|
|
38
|
+
* (core/frontmatter-read.ts): a real YAML parse (so `key: [Read, "Bash"]` is a
|
|
39
|
+
* native array and `key: Read, Bash` a comma scalar) with a regex salvage when
|
|
40
|
+
* the block is malformed — the rail still reads the contract. `null` when the key
|
|
41
|
+
* is absent (inherits all), `[]` when present-but-empty (no tools). Shared by the
|
|
42
|
+
* rail (`tools:`) and the `disallowed-tools-contract` scan/lint.
|
|
43
|
+
*/
|
|
44
|
+
export declare function parseAgentToolList(markdown: string, key: string): string[] | null;
|
|
45
|
+
/**
|
|
46
|
+
* Parse the declared purity floor from a compiled agent's `.md` — the
|
|
47
|
+
* `<!-- vigiles:purity:LEVEL -->` marker `compile` emits (see `purityMarker`).
|
|
48
|
+
* Returns null when no marker is present (the unit declared no floor, so the
|
|
49
|
+
* purity gate imposes no constraint). The single source of truth the runtime
|
|
50
|
+
* gate reads, exactly like `tools:` for the tool-contract rail.
|
|
51
|
+
*/
|
|
52
|
+
export declare function parseAgentPurity(markdown: string): PurityLevel | null;
|
|
34
53
|
export interface PreToolDecision {
|
|
35
54
|
/** Whether the tool call is allowed (true) or blocked (false). */
|
|
36
55
|
readonly allow: boolean;
|
|
@@ -54,11 +73,16 @@ export declare function clearActiveAgent(cwd: string): void;
|
|
|
54
73
|
/** The path of the active agent's compiled `.md`, or null when none is active. */
|
|
55
74
|
export declare function readActiveAgent(cwd: string): string | null;
|
|
56
75
|
/**
|
|
57
|
-
* PreToolUse-hook decision. If an agent is active,
|
|
58
|
-
*
|
|
59
|
-
*
|
|
60
|
-
*
|
|
61
|
-
*
|
|
76
|
+
* PreToolUse-hook decision. If an agent is active, enforce BOTH deterministic
|
|
77
|
+
* rails its compiled `.md` declares, in order:
|
|
78
|
+
*
|
|
79
|
+
* 1. the tool-contract rail (`tools:`) — allow only listed tools;
|
|
80
|
+
* 2. the purity gate (`vigiles:purity:`) — allow only calls within the declared
|
|
81
|
+
* effect floor, refining `Bash` by the live `command` (`decidePurityGate`).
|
|
82
|
+
*
|
|
83
|
+
* The first to deny wins, feeding its reason back to the model. With no active
|
|
84
|
+
* agent (or one that declared neither contract), always allow — the rails only
|
|
85
|
+
* constrain agents that opted in.
|
|
62
86
|
*/
|
|
63
|
-
export declare function evaluatePreToolUse(cwd: string, tool: string): PreToolDecision;
|
|
87
|
+
export declare function evaluatePreToolUse(cwd: string, tool: string, command?: string): PreToolDecision;
|
|
64
88
|
//# sourceMappingURL=agent-runtime.d.ts.map
|
|
@@ -25,6 +25,8 @@
|
|
|
25
25
|
*/
|
|
26
26
|
Object.defineProperty(exports, "__esModule", { value: true });
|
|
27
27
|
exports.parseAgentTools = parseAgentTools;
|
|
28
|
+
exports.parseAgentToolList = parseAgentToolList;
|
|
29
|
+
exports.parseAgentPurity = parseAgentPurity;
|
|
28
30
|
exports.decidePreToolUse = decidePreToolUse;
|
|
29
31
|
exports.setActiveAgent = setActiveAgent;
|
|
30
32
|
exports.clearActiveAgent = clearActiveAgent;
|
|
@@ -32,28 +34,13 @@ exports.readActiveAgent = readActiveAgent;
|
|
|
32
34
|
exports.evaluatePreToolUse = evaluatePreToolUse;
|
|
33
35
|
const node_fs_1 = require("node:fs");
|
|
34
36
|
const node_path_1 = require("node:path");
|
|
37
|
+
const frontmatter_read_js_1 = require("../../core/frontmatter-read.js");
|
|
38
|
+
const effects_js_1 = require("../../core/effects.js");
|
|
39
|
+
const dialect_js_1 = require("./dialect.js");
|
|
40
|
+
const effect_region_js_1 = require("./effect-region.js");
|
|
35
41
|
// ---------------------------------------------------------------------------
|
|
36
42
|
// Parse the tool contract from a compiled agent .md
|
|
37
43
|
// ---------------------------------------------------------------------------
|
|
38
|
-
/** Extract the YAML frontmatter block (between the first pair of `---` fences). */
|
|
39
|
-
function extractFrontmatter(markdown) {
|
|
40
|
-
const lines = markdown.split("\n");
|
|
41
|
-
let start = -1;
|
|
42
|
-
for (let i = 0; i < lines.length; i++) {
|
|
43
|
-
if (lines[i].trim() === "---") {
|
|
44
|
-
start = i;
|
|
45
|
-
break;
|
|
46
|
-
}
|
|
47
|
-
}
|
|
48
|
-
if (start === -1)
|
|
49
|
-
return null;
|
|
50
|
-
for (let i = start + 1; i < lines.length; i++) {
|
|
51
|
-
if (lines[i].trim() === "---") {
|
|
52
|
-
return lines.slice(start + 1, i).join("\n");
|
|
53
|
-
}
|
|
54
|
-
}
|
|
55
|
-
return null;
|
|
56
|
-
}
|
|
57
44
|
/**
|
|
58
45
|
* Parse an agent's allowed-tools contract from its compiled markdown.
|
|
59
46
|
*
|
|
@@ -63,16 +50,31 @@ function extractFrontmatter(markdown) {
|
|
|
63
50
|
* an empty list (`tools:` with nothing after it) means "no tools allowed".
|
|
64
51
|
*/
|
|
65
52
|
function parseAgentTools(markdown) {
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
53
|
+
return parseAgentToolList(markdown, "tools");
|
|
54
|
+
}
|
|
55
|
+
/**
|
|
56
|
+
* Parse a comma/array tool list under an arbitrary frontmatter `key` (e.g.
|
|
57
|
+
* `tools:` or `disallowedTools:`) via the shared lenient reader
|
|
58
|
+
* (core/frontmatter-read.ts): a real YAML parse (so `key: [Read, "Bash"]` is a
|
|
59
|
+
* native array and `key: Read, Bash` a comma scalar) with a regex salvage when
|
|
60
|
+
* the block is malformed — the rail still reads the contract. `null` when the key
|
|
61
|
+
* is absent (inherits all), `[]` when present-but-empty (no tools). Shared by the
|
|
62
|
+
* rail (`tools:`) and the `disallowed-tools-contract` scan/lint.
|
|
63
|
+
*/
|
|
64
|
+
function parseAgentToolList(markdown, key) {
|
|
65
|
+
return (0, frontmatter_read_js_1.frontmatterList)((0, frontmatter_read_js_1.readFrontmatter)(markdown), key);
|
|
66
|
+
}
|
|
67
|
+
const PURITY_RE = /<!--\s*vigiles:purity:(pure|bounded|unrestricted)\s*-->/;
|
|
68
|
+
/**
|
|
69
|
+
* Parse the declared purity floor from a compiled agent's `.md` — the
|
|
70
|
+
* `<!-- vigiles:purity:LEVEL -->` marker `compile` emits (see `purityMarker`).
|
|
71
|
+
* Returns null when no marker is present (the unit declared no floor, so the
|
|
72
|
+
* purity gate imposes no constraint). The single source of truth the runtime
|
|
73
|
+
* gate reads, exactly like `tools:` for the tool-contract rail.
|
|
74
|
+
*/
|
|
75
|
+
function parseAgentPurity(markdown) {
|
|
76
|
+
const m = PURITY_RE.exec(markdown);
|
|
77
|
+
return m ? m[1] : null;
|
|
76
78
|
}
|
|
77
79
|
/**
|
|
78
80
|
* Decide whether `tool` is allowed under an agent's tool contract. Pure, so the
|
|
@@ -128,20 +130,47 @@ function readActiveAgent(cwd) {
|
|
|
128
130
|
// PreToolUse-hook decision
|
|
129
131
|
// ---------------------------------------------------------------------------
|
|
130
132
|
/**
|
|
131
|
-
* PreToolUse-hook decision. If an agent is active,
|
|
132
|
-
*
|
|
133
|
-
*
|
|
134
|
-
*
|
|
135
|
-
*
|
|
133
|
+
* PreToolUse-hook decision. If an agent is active, enforce BOTH deterministic
|
|
134
|
+
* rails its compiled `.md` declares, in order:
|
|
135
|
+
*
|
|
136
|
+
* 1. the tool-contract rail (`tools:`) — allow only listed tools;
|
|
137
|
+
* 2. the purity gate (`vigiles:purity:`) — allow only calls within the declared
|
|
138
|
+
* effect floor, refining `Bash` by the live `command` (`decidePurityGate`).
|
|
139
|
+
*
|
|
140
|
+
* The first to deny wins, feeding its reason back to the model. With no active
|
|
141
|
+
* agent (or one that declared neither contract), always allow — the rails only
|
|
142
|
+
* constrain agents that opted in.
|
|
136
143
|
*/
|
|
137
|
-
function evaluatePreToolUse(cwd, tool) {
|
|
144
|
+
function evaluatePreToolUse(cwd, tool, command) {
|
|
138
145
|
const agentPath = readActiveAgent(cwd);
|
|
139
146
|
if (!agentPath)
|
|
140
147
|
return { allow: true, message: "" };
|
|
141
148
|
const full = (0, node_path_1.resolve)(cwd, agentPath);
|
|
142
149
|
if (!(0, node_fs_1.existsSync)(full))
|
|
143
150
|
return { allow: true, message: "" };
|
|
144
|
-
const
|
|
145
|
-
|
|
151
|
+
const md = (0, node_fs_1.readFileSync)(full, "utf-8");
|
|
152
|
+
// 1) Tool-contract rail — the declared allowlist.
|
|
153
|
+
const rail = decidePreToolUse(parseAgentTools(md), tool);
|
|
154
|
+
if (!rail.allow)
|
|
155
|
+
return rail;
|
|
156
|
+
// 2) Purity gate — the declared effect floor, refined by the live command.
|
|
157
|
+
// If an effect boundary is declared, tighten to "pure" outside it and apply
|
|
158
|
+
// the declared purity (or "unrestricted") inside.
|
|
159
|
+
const purity = parseAgentPurity(md);
|
|
160
|
+
const boundary = (0, effect_region_js_1.hasEffectBoundary)(md);
|
|
161
|
+
if (boundary) {
|
|
162
|
+
const effective = (0, effect_region_js_1.readEffectActive)(cwd)
|
|
163
|
+
? (purity ?? "unrestricted")
|
|
164
|
+
: "pure";
|
|
165
|
+
const gate = (0, effects_js_1.decidePurityGate)(effective, tool, command, dialect_js_1.claudeCodeDialect);
|
|
166
|
+
if (!gate.allow)
|
|
167
|
+
return gate;
|
|
168
|
+
}
|
|
169
|
+
else if (purity) {
|
|
170
|
+
const gate = (0, effects_js_1.decidePurityGate)(purity, tool, command, dialect_js_1.claudeCodeDialect);
|
|
171
|
+
if (!gate.allow)
|
|
172
|
+
return gate;
|
|
173
|
+
}
|
|
174
|
+
return { allow: true, message: "" };
|
|
146
175
|
}
|
|
147
176
|
//# sourceMappingURL=agent-runtime.js.map
|
|
@@ -9,9 +9,13 @@ exports.claudeCodeDialect = {
|
|
|
9
9
|
"Read",
|
|
10
10
|
"Write",
|
|
11
11
|
"Edit",
|
|
12
|
+
"MultiEdit",
|
|
12
13
|
"Bash",
|
|
14
|
+
"BashOutput",
|
|
15
|
+
"KillBash",
|
|
13
16
|
"Grep",
|
|
14
17
|
"Glob",
|
|
18
|
+
"LS",
|
|
15
19
|
"WebSearch",
|
|
16
20
|
"WebFetch",
|
|
17
21
|
"NotebookEdit",
|
|
@@ -30,12 +34,24 @@ exports.claudeCodeDialect = {
|
|
|
30
34
|
"WaitForMcpServers",
|
|
31
35
|
],
|
|
32
36
|
mcpToolPattern: /^mcp__[a-z0-9_-]+__[a-z0-9_-]+$/i,
|
|
37
|
+
// Claude Code's own built-in MCP server: the IDE integration provides
|
|
38
|
+
// `mcp__ide__getDiagnostics` / `mcp__ide__executeCode` at runtime without any
|
|
39
|
+
// plugin declaring it, so a contract that lists those must NOT be flagged as
|
|
40
|
+
// referencing an undeclared server (the mcp-tool-resolves allowlist).
|
|
41
|
+
knownMcpServers: ["ide"],
|
|
42
|
+
// The real Claude Code hook events. (Was wrong: PreSession/PostSession don't
|
|
43
|
+
// exist; SessionStart/SessionEnd/Stop/SubagentStop/UserPromptSubmit/PreCompact
|
|
44
|
+
// were missing — verified against the events real plugins register.)
|
|
33
45
|
hookEvents: [
|
|
34
46
|
"PreToolUse",
|
|
35
47
|
"PostToolUse",
|
|
36
|
-
"
|
|
37
|
-
"PostSession",
|
|
48
|
+
"UserPromptSubmit",
|
|
38
49
|
"Notification",
|
|
50
|
+
"Stop",
|
|
51
|
+
"SubagentStop",
|
|
52
|
+
"PreCompact",
|
|
53
|
+
"SessionStart",
|
|
54
|
+
"SessionEnd",
|
|
39
55
|
],
|
|
40
56
|
// Claude Code natively reads CLAUDE.md only — it does NOT auto-load AGENTS.md
|
|
41
57
|
// (anthropics/claude-code#34235 is open; AGENTS.md works solely via an
|
|
@@ -47,5 +63,24 @@ exports.claudeCodeDialect = {
|
|
|
47
63
|
// Claude Code reads the full SKILL.md frontmatter set (description,
|
|
48
64
|
// disable-model-invocation, argument-hint, …).
|
|
49
65
|
skillFrontmatter: "claude-code",
|
|
66
|
+
// Tools that produce side effects in Claude Code. The complement — the
|
|
67
|
+
// read-only tools — are: Read, Grep, Glob, LS, ToolSearch (and LSP/Agent
|
|
68
|
+
// which are not in the subagent catalog). Bash is side-effecting because
|
|
69
|
+
// `cat` and `rm -rf` are the same tool at the tool-name level — the
|
|
70
|
+
// sandbox is the only closure for subprocess effects.
|
|
71
|
+
sideEffectingTools: [
|
|
72
|
+
"Bash",
|
|
73
|
+
"BashOutput",
|
|
74
|
+
"KillBash",
|
|
75
|
+
"Edit",
|
|
76
|
+
"MultiEdit",
|
|
77
|
+
"Write",
|
|
78
|
+
"NotebookEdit",
|
|
79
|
+
"WebFetch",
|
|
80
|
+
"WebSearch",
|
|
81
|
+
"Skill",
|
|
82
|
+
"Task",
|
|
83
|
+
"TodoWrite",
|
|
84
|
+
],
|
|
50
85
|
};
|
|
51
86
|
//# sourceMappingURL=dialect.js.map
|
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
/** Record that the agent has entered an effect boundary. */
|
|
2
|
+
export declare function setEffectActive(cwd: string): void;
|
|
3
|
+
/** Clear the effect-active marker (the agent exited the effect boundary). */
|
|
4
|
+
export declare function clearEffectActive(cwd: string): void;
|
|
5
|
+
/** True iff the agent is currently inside an effect boundary. Tolerates malformed file. */
|
|
6
|
+
export declare function readEffectActive(cwd: string): boolean;
|
|
7
|
+
/** True iff the compiled markdown declares an effect boundary (`<!-- vigiles:effect -->`). */
|
|
8
|
+
export declare function hasEffectBoundary(markdown: string): boolean;
|
|
9
|
+
//# sourceMappingURL=effect-region.d.ts.map
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
3
|
+
exports.setEffectActive = setEffectActive;
|
|
4
|
+
exports.clearEffectActive = clearEffectActive;
|
|
5
|
+
exports.readEffectActive = readEffectActive;
|
|
6
|
+
exports.hasEffectBoundary = hasEffectBoundary;
|
|
7
|
+
/**
|
|
8
|
+
* Effect-boundary position-aware state tracking.
|
|
9
|
+
* Mirrors active-unit tracking (.vigiles/active-agent.json / .vigiles/active-skill.json).
|
|
10
|
+
* The PreToolUse hook reads this file to decide whether the agent is inside an
|
|
11
|
+
* effect boundary (set by `vigiles effect-enter`) or outside it.
|
|
12
|
+
*/
|
|
13
|
+
const node_fs_1 = require("node:fs");
|
|
14
|
+
const node_path_1 = require("node:path");
|
|
15
|
+
const EFFECT_ACTIVE_PATH = ".vigiles/effect-active.json";
|
|
16
|
+
/** Record that the agent has entered an effect boundary. */
|
|
17
|
+
function setEffectActive(cwd) {
|
|
18
|
+
const p = (0, node_path_1.resolve)(cwd, EFFECT_ACTIVE_PATH);
|
|
19
|
+
(0, node_fs_1.mkdirSync)((0, node_path_1.dirname)(p), { recursive: true });
|
|
20
|
+
(0, node_fs_1.writeFileSync)(p, JSON.stringify({ active: true }) + "\n");
|
|
21
|
+
}
|
|
22
|
+
/** Clear the effect-active marker (the agent exited the effect boundary). */
|
|
23
|
+
function clearEffectActive(cwd) {
|
|
24
|
+
const p = (0, node_path_1.resolve)(cwd, EFFECT_ACTIVE_PATH);
|
|
25
|
+
if ((0, node_fs_1.existsSync)(p))
|
|
26
|
+
(0, node_fs_1.rmSync)(p);
|
|
27
|
+
}
|
|
28
|
+
/** True iff the agent is currently inside an effect boundary. Tolerates malformed file. */
|
|
29
|
+
function readEffectActive(cwd) {
|
|
30
|
+
const p = (0, node_path_1.resolve)(cwd, EFFECT_ACTIVE_PATH);
|
|
31
|
+
if (!(0, node_fs_1.existsSync)(p))
|
|
32
|
+
return false;
|
|
33
|
+
try {
|
|
34
|
+
const parsed = JSON.parse((0, node_fs_1.readFileSync)(p, "utf-8"));
|
|
35
|
+
return parsed.active === true;
|
|
36
|
+
}
|
|
37
|
+
catch {
|
|
38
|
+
return false;
|
|
39
|
+
}
|
|
40
|
+
}
|
|
41
|
+
/** True iff the compiled markdown declares an effect boundary (`<!-- vigiles:effect -->`). */
|
|
42
|
+
function hasEffectBoundary(markdown) {
|
|
43
|
+
return markdown.includes("<!-- vigiles:effect -->");
|
|
44
|
+
}
|
|
45
|
+
//# sourceMappingURL=effect-region.js.map
|
|
@@ -9,6 +9,9 @@ exports.claudeCodeLayout = {
|
|
|
9
9
|
settingsFormat: "json",
|
|
10
10
|
instructionFile: "CLAUDE.md",
|
|
11
11
|
surfaceDirs: ["skills", "agents", "commands"],
|
|
12
|
+
skillDir: "skills",
|
|
13
|
+
agentDir: "agents",
|
|
14
|
+
commandDir: "commands",
|
|
12
15
|
materializeRoot: ".claude",
|
|
13
16
|
pluginRootToken: "${CLAUDE_PLUGIN_ROOT}",
|
|
14
17
|
mcpConfigFile: ".mcp.json",
|
|
@@ -20,6 +20,7 @@
|
|
|
20
20
|
* (`vigiles run-skill`). It is not a silent hook and runs nothing the spec
|
|
21
21
|
* didn't declare as a gate.
|
|
22
22
|
*/
|
|
23
|
+
import type { PurityLevel } from "../../core/effects.js";
|
|
23
24
|
export type RuntimeGate = {
|
|
24
25
|
readonly kind: "cmd";
|
|
25
26
|
readonly command: string;
|
|
@@ -86,6 +87,30 @@ export declare function setActiveSkill(cwd: string, skillPath: string): void;
|
|
|
86
87
|
export declare function clearActiveSkill(cwd: string): void;
|
|
87
88
|
/** The path of the active skill, or null when none is in progress. */
|
|
88
89
|
export declare function readActiveSkill(cwd: string): string | null;
|
|
90
|
+
/**
|
|
91
|
+
* Parse the declared purity floor from a compiled SKILL.md — the
|
|
92
|
+
* `<!-- vigiles:purity:LEVEL -->` marker `compileSkill` emits (see
|
|
93
|
+
* `purityMarker` in compile.ts). Returns null when no marker is present (the
|
|
94
|
+
* skill declared no floor, so the purity gate imposes no constraint). Mirrors
|
|
95
|
+
* `parseAgentPurity` in agent-runtime.ts.
|
|
96
|
+
*/
|
|
97
|
+
export declare function parseSkillPurity(markdown: string): PurityLevel | null;
|
|
98
|
+
/** A runtime allow/deny decision (mirrors PreToolDecision in agent-runtime.ts). */
|
|
99
|
+
export interface SkillPreToolDecision {
|
|
100
|
+
/** Whether the tool call is allowed (true) or blocked (false). */
|
|
101
|
+
readonly allow: boolean;
|
|
102
|
+
/** Message fed back to the model on a block; empty on allow. */
|
|
103
|
+
readonly message: string;
|
|
104
|
+
}
|
|
105
|
+
/**
|
|
106
|
+
* PreToolUse purity gate for skills. Mirrors `evaluatePreToolUse` in
|
|
107
|
+
* agent-runtime.ts, but enforces ONLY the purity floor — skills have no
|
|
108
|
+
* tools-allowlist rail (that's a separate, future concern). If a skill is
|
|
109
|
+
* active and declares a `vigiles:purity:` marker, `decidePurityGate` checks
|
|
110
|
+
* the live call (refining `Bash` by the concrete command via `isReadOnlyBash`).
|
|
111
|
+
* With no active skill, a missing `.md`, or no purity marker, always allows.
|
|
112
|
+
*/
|
|
113
|
+
export declare function evaluateSkillPreToolUse(cwd: string, tool: string, command?: string): SkillPreToolDecision;
|
|
89
114
|
export interface StopDecision {
|
|
90
115
|
/** Whether the agent may stop (true) or must keep working (false). */
|
|
91
116
|
readonly allow: boolean;
|
|
@@ -30,10 +30,15 @@ exports.runSkillGates = runSkillGates;
|
|
|
30
30
|
exports.setActiveSkill = setActiveSkill;
|
|
31
31
|
exports.clearActiveSkill = clearActiveSkill;
|
|
32
32
|
exports.readActiveSkill = readActiveSkill;
|
|
33
|
+
exports.parseSkillPurity = parseSkillPurity;
|
|
34
|
+
exports.evaluateSkillPreToolUse = evaluateSkillPreToolUse;
|
|
33
35
|
exports.evaluateStopHook = evaluateStopHook;
|
|
34
36
|
const node_child_process_1 = require("node:child_process");
|
|
35
37
|
const node_fs_1 = require("node:fs");
|
|
36
38
|
const node_path_1 = require("node:path");
|
|
39
|
+
const effects_js_1 = require("../../core/effects.js");
|
|
40
|
+
const dialect_js_1 = require("./dialect.js");
|
|
41
|
+
const effect_region_js_1 = require("./effect-region.js");
|
|
37
42
|
const STEP_RE = /^###\s+Step\s+(\d+)/;
|
|
38
43
|
const GATE_CMD_RE = /<!--\s*vigiles:gate\s+"([^"]*)"(?:\s+retry:(\d+))?\s*-->/;
|
|
39
44
|
const GATE_FILE_RE = /<!--\s*vigiles:gate\s+file:(\S+)\s*-->/;
|
|
@@ -257,6 +262,49 @@ function readActiveSkill(cwd) {
|
|
|
257
262
|
return null;
|
|
258
263
|
}
|
|
259
264
|
}
|
|
265
|
+
// ---------------------------------------------------------------------------
|
|
266
|
+
// Skill purity gate (mirrors agent-runtime.ts parseAgentPurity / evaluatePreToolUse)
|
|
267
|
+
// ---------------------------------------------------------------------------
|
|
268
|
+
const PURITY_RE = /<!--\s*vigiles:purity:(pure|bounded|unrestricted)\s*-->/;
|
|
269
|
+
/**
|
|
270
|
+
* Parse the declared purity floor from a compiled SKILL.md — the
|
|
271
|
+
* `<!-- vigiles:purity:LEVEL -->` marker `compileSkill` emits (see
|
|
272
|
+
* `purityMarker` in compile.ts). Returns null when no marker is present (the
|
|
273
|
+
* skill declared no floor, so the purity gate imposes no constraint). Mirrors
|
|
274
|
+
* `parseAgentPurity` in agent-runtime.ts.
|
|
275
|
+
*/
|
|
276
|
+
function parseSkillPurity(markdown) {
|
|
277
|
+
const m = PURITY_RE.exec(markdown);
|
|
278
|
+
return m ? m[1] : null;
|
|
279
|
+
}
|
|
280
|
+
/**
|
|
281
|
+
* PreToolUse purity gate for skills. Mirrors `evaluatePreToolUse` in
|
|
282
|
+
* agent-runtime.ts, but enforces ONLY the purity floor — skills have no
|
|
283
|
+
* tools-allowlist rail (that's a separate, future concern). If a skill is
|
|
284
|
+
* active and declares a `vigiles:purity:` marker, `decidePurityGate` checks
|
|
285
|
+
* the live call (refining `Bash` by the concrete command via `isReadOnlyBash`).
|
|
286
|
+
* With no active skill, a missing `.md`, or no purity marker, always allows.
|
|
287
|
+
*/
|
|
288
|
+
function evaluateSkillPreToolUse(cwd, tool, command) {
|
|
289
|
+
const skillPath = readActiveSkill(cwd);
|
|
290
|
+
if (!skillPath)
|
|
291
|
+
return { allow: true, message: "" };
|
|
292
|
+
const full = (0, node_path_1.resolve)(cwd, skillPath);
|
|
293
|
+
if (!(0, node_fs_1.existsSync)(full))
|
|
294
|
+
return { allow: true, message: "" };
|
|
295
|
+
const md = (0, node_fs_1.readFileSync)(full, "utf-8");
|
|
296
|
+
const purity = parseSkillPurity(md);
|
|
297
|
+
const boundary = (0, effect_region_js_1.hasEffectBoundary)(md);
|
|
298
|
+
if (boundary) {
|
|
299
|
+
const effective = (0, effect_region_js_1.readEffectActive)(cwd)
|
|
300
|
+
? (purity ?? "unrestricted")
|
|
301
|
+
: "pure";
|
|
302
|
+
return (0, effects_js_1.decidePurityGate)(effective, tool, command, dialect_js_1.claudeCodeDialect);
|
|
303
|
+
}
|
|
304
|
+
if (!purity)
|
|
305
|
+
return { allow: true, message: "" };
|
|
306
|
+
return (0, effects_js_1.decidePurityGate)(purity, tool, command, dialect_js_1.claudeCodeDialect);
|
|
307
|
+
}
|
|
260
308
|
/**
|
|
261
309
|
* Stop-hook decision. If a skill is active and declares a result gate, run it:
|
|
262
310
|
* allow the stop only when the gate passes; otherwise block and tell the model
|
|
@@ -29,6 +29,9 @@ exports.codexAdapter = {
|
|
|
29
29
|
referenceVerification: true,
|
|
30
30
|
harnessTesting: true,
|
|
31
31
|
shellHooks: true,
|
|
32
|
+
// Codex `[agents]` is a concurrency table, not a subagent tool-contract file
|
|
33
|
+
// — the subagent-surface rules report n/a here (a deliberate non-goal).
|
|
34
|
+
subagents: false,
|
|
32
35
|
},
|
|
33
36
|
dialect: dialect_js_1.codexDialect,
|
|
34
37
|
layout: layout_js_1.codexLayout,
|
|
@@ -0,0 +1,94 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Codex EVAL-tier transport — the runner + trace parser that
|
|
3
|
+
* `measureTriggerRate`/`runEval` dispatch to via the `ModelOutputParser` seam.
|
|
4
|
+
*
|
|
5
|
+
* SCHEMA: CONFIRMED against real `codex exec --json` (codex-cli 0.139.0, ChatGPT
|
|
6
|
+
* auth). The stream is the thread/item model:
|
|
7
|
+
*
|
|
8
|
+
* {"type":"thread.started","thread_id":"…"}
|
|
9
|
+
* {"type":"turn.started"}
|
|
10
|
+
* {"type":"item.started","item":{"id":"item_0","type":"command_execution",…}} // mid-flight
|
|
11
|
+
* {"type":"item.completed","item":{"id":"item_0","type":"command_execution","command":"…","aggregated_output":"…","exit_code":0}}
|
|
12
|
+
* {"type":"item.completed","item":{"id":"item_1","type":"agent_message","text":"…"}}
|
|
13
|
+
* {"type":"turn.completed","usage":{"input_tokens":…,"cached_input_tokens":…,"output_tokens":…}}
|
|
14
|
+
*
|
|
15
|
+
* So: assistant text = `item.completed` with `item.type:"agent_message"` →
|
|
16
|
+
* `item.text`; a tool call = `item.type:"command_execution"` → `item.command`;
|
|
17
|
+
* usage rides `turn.completed`. We count `item.completed` ONLY (an `item.started`
|
|
18
|
+
* carries the same `id` mid-flight — counting both double-counts).
|
|
19
|
+
*
|
|
20
|
+
* THE SKILL FINDING: Codex has NO discrete "skill selected" event (its CLI has no
|
|
21
|
+
* Skill-tool concept). When a skill triggers, the model READS the skill's
|
|
22
|
+
* `SKILL.md` via a `command_execution` (`sed/cat … skills/<name>/SKILL.md`) and
|
|
23
|
+
* usually says so in an `agent_message`. So "did skill X fire" on Codex is not a
|
|
24
|
+
* clean trace event like Claude's `Skill` tool_use — it's detected by the
|
|
25
|
+
* SKILL.md read (`codexSkillFired`). Best-effort by nature (a cached skill might
|
|
26
|
+
* not be re-read); pair with a behavioral/judged check for certainty.
|
|
27
|
+
*/
|
|
28
|
+
import type { ParsedModelRun, AgentRunArgs, RunOut, EvalDriver } from "../../eval.js";
|
|
29
|
+
import type { ToolCall } from "../../core/harness-driver.js";
|
|
30
|
+
/** Parse `codex exec --json` stdout into the common trace fields (confirmed schema). */
|
|
31
|
+
export declare function parseCodexEvalRun(out: {
|
|
32
|
+
stdout: string;
|
|
33
|
+
}): ParsedModelRun;
|
|
34
|
+
/**
|
|
35
|
+
* The error message if the run errored or was rate-limited (an `error` /
|
|
36
|
+
* `turn.failed` event), else null. CRITICAL for the eval tier: an errored turn
|
|
37
|
+
* must NOT be scored as a clean "skill didn't fire" miss — dogfooding hit a Codex
|
|
38
|
+
* usage limit ("You've hit your usage limit…") whose `error` event left an empty
|
|
39
|
+
* trace that `codexSkillFired` read as recall 0. A caller should skip/retry an
|
|
40
|
+
* errored run, not count it. (The Claude path has `isRateLimited` + backoff; this
|
|
41
|
+
* is the Codex equivalent detector.)
|
|
42
|
+
*/
|
|
43
|
+
export declare function codexRunError(out: {
|
|
44
|
+
stdout: string;
|
|
45
|
+
}): string | null;
|
|
46
|
+
/**
|
|
47
|
+
* Did Codex activate skill `name` on this run? Detected by the SKILL.md read —
|
|
48
|
+
* Codex has no discrete skill-selection event, so when a skill triggers the model
|
|
49
|
+
* reads its `…/<name>/SKILL.md` via a `command_execution`. Best-effort (a cached
|
|
50
|
+
* skill might not be re-read); for the trigger-rate `fired` predicate over Codex.
|
|
51
|
+
*/
|
|
52
|
+
export declare function codexSkillFired(run: {
|
|
53
|
+
toolCalls: readonly ToolCall[];
|
|
54
|
+
}, name: string): boolean;
|
|
55
|
+
/**
|
|
56
|
+
* Materialize a (Claude-shaped) plugin dir's skills into `<cwd>/.codex/skills/` —
|
|
57
|
+
* where Codex actually discovers them (validated live: codex reads
|
|
58
|
+
* `<cwd>/.codex/skills/<name>/SKILL.md`). This is the Codex analog of Claude's
|
|
59
|
+
* `--plugin-dir`: `measureTriggerRate` hands the runner a `pluginDir` (the
|
|
60
|
+
* stubbed/packaged skills), and the Codex runner installs them here before the
|
|
61
|
+
* turn. Pure fs — unit-testable without a binary.
|
|
62
|
+
*/
|
|
63
|
+
export declare function installCodexSkills(pluginDir: string, cwd: string): number;
|
|
64
|
+
/**
|
|
65
|
+
* The Codex eval-tier `AgentRunner`: install the run's skills into `.codex/skills`
|
|
66
|
+
* (Codex's discovery path, vs Claude's `--plugin-dir`), then drive a real
|
|
67
|
+
* `codex exec --json` turn. The seam `measureTriggerRate(spec, { evalDriver:
|
|
68
|
+
* codexEvalDriver })` dispatches through.
|
|
69
|
+
*/
|
|
70
|
+
export declare function codexEvalAgentRunner(args: AgentRunArgs): Promise<RunOut>;
|
|
71
|
+
/**
|
|
72
|
+
* The Codex eval driver — pass to `measureTriggerRate(spec, { evalDriver:
|
|
73
|
+
* codexEvalDriver })` to run a trigger-rate eval natively on `codex exec`. Pair
|
|
74
|
+
* the spec's `fired` with `codexSkillFired` (Codex has no Skill-tool event).
|
|
75
|
+
*/
|
|
76
|
+
export declare const codexEvalDriver: EvalDriver;
|
|
77
|
+
/**
|
|
78
|
+
* Spawn real `codex exec --json` for the eval tier (real model, the user's codex
|
|
79
|
+
* auth — NOT the mock). CONFIRMED flags (codex 0.139.0): `--json` for the event
|
|
80
|
+
* stream, `--skip-git-repo-check` for a bare cwd, the approvals/sandbox bypass so
|
|
81
|
+
* the turn runs unattended, `-C <cwd>` for the working dir, prompt as the trailing
|
|
82
|
+
* positional, and stdin = /dev/null (`stdio: ["ignore",…]`) — codex otherwise
|
|
83
|
+
* blocks on "Reading additional input from stdin…". Needs ChatGPT/API auth +
|
|
84
|
+
* network egress to the model backend.
|
|
85
|
+
*/
|
|
86
|
+
export declare function codexEvalRunner(args: {
|
|
87
|
+
task: string;
|
|
88
|
+
cwd: string;
|
|
89
|
+
timeoutMs: number;
|
|
90
|
+
}): {
|
|
91
|
+
code: number;
|
|
92
|
+
stdout: string;
|
|
93
|
+
};
|
|
94
|
+
//# sourceMappingURL=eval.d.ts.map
|