vigiles 2.6.0 → 3.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +9 -0
- package/README.md +74 -129
- package/action.yml +143 -7
- package/dist/action-gate.d.ts +1 -1
- package/dist/action-gate.js +1 -1
- package/dist/adapter-conformance.d.ts +30 -0
- package/dist/adapter-conformance.js +153 -0
- package/dist/adapter-registry.d.ts +42 -0
- package/dist/adapter-registry.js +55 -0
- package/dist/adapter.d.ts +26 -0
- package/dist/adapter.js +16 -0
- package/dist/adapters/claude-code/adapter.d.ts +3 -0
- package/dist/adapters/claude-code/adapter.js +46 -0
- package/dist/{agent-result.d.ts → adapters/claude-code/agent-result.d.ts} +1 -1
- package/dist/adapters/claude-code/dialect.d.ts +13 -0
- package/dist/adapters/claude-code/dialect.js +51 -0
- package/dist/adapters/claude-code/egress-entry.d.ts +2 -0
- package/dist/adapters/claude-code/egress-entry.js +115 -0
- package/dist/adapters/claude-code/egress.d.ts +114 -0
- package/dist/adapters/claude-code/egress.js +276 -0
- package/dist/{eval-cache.d.ts → adapters/claude-code/eval-cache.d.ts} +1 -1
- package/dist/{eval-cache.js → adapters/claude-code/eval-cache.js} +1 -1
- package/dist/{eval.d.ts → adapters/claude-code/eval.d.ts} +28 -4
- package/dist/{eval.js → adapters/claude-code/eval.js} +61 -25
- package/dist/{harness-test.d.ts → adapters/claude-code/harness-test.d.ts} +33 -37
- package/dist/{harness-test.js → adapters/claude-code/harness-test.js} +124 -43
- package/dist/adapters/claude-code/hook-protocol.d.ts +10 -0
- package/dist/adapters/claude-code/hook-protocol.js +10 -0
- package/dist/adapters/claude-code/layout.d.ts +8 -0
- package/dist/adapters/claude-code/layout.js +18 -0
- package/dist/{mock-model.d.ts → adapters/claude-code/mock-model.d.ts} +2 -24
- package/dist/adapters/claude-code/model-mock.d.ts +11 -0
- package/dist/adapters/claude-code/model-mock.js +10 -0
- package/dist/adapters/claude-code/plugin-loader.d.ts +25 -0
- package/dist/adapters/claude-code/plugin-loader.js +19 -0
- package/dist/{run-hook.d.ts → adapters/claude-code/run-hook.d.ts} +50 -3
- package/dist/{run-hook.js → adapters/claude-code/run-hook.js} +178 -18
- package/dist/adapters/claude-code/run-scripts.d.ts +52 -0
- package/dist/adapters/claude-code/run-scripts.js +150 -0
- package/dist/adapters/claude-code/runtime.d.ts +16 -0
- package/dist/adapters/claude-code/runtime.js +39 -0
- package/dist/{sandbox.d.ts → adapters/claude-code/sandbox.d.ts} +10 -0
- package/dist/{sandbox.js → adapters/claude-code/sandbox.js} +5 -3
- package/dist/{skill-driver.d.ts → adapters/claude-code/skill-driver.d.ts} +1 -1
- package/dist/adapters/codex/adapter.d.ts +3 -0
- package/dist/adapters/codex/adapter.js +49 -0
- package/dist/adapters/codex/dialect.d.ts +10 -0
- package/dist/adapters/codex/dialect.js +30 -0
- package/dist/adapters/codex/driver.d.ts +20 -0
- package/dist/adapters/codex/driver.js +89 -0
- package/dist/adapters/codex/hook-protocol.d.ts +10 -0
- package/dist/adapters/codex/hook-protocol.js +18 -0
- package/dist/adapters/codex/layout.d.ts +16 -0
- package/dist/adapters/codex/layout.js +18 -0
- package/dist/adapters/codex/mock-model.d.ts +52 -0
- package/dist/adapters/codex/mock-model.js +210 -0
- package/dist/adapters/codex/model-mock.d.ts +11 -0
- package/dist/adapters/codex/model-mock.js +10 -0
- package/dist/adapters/codex/runtime.d.ts +38 -0
- package/dist/adapters/codex/runtime.js +62 -0
- package/dist/adapters/opencode/adapter.d.ts +3 -0
- package/dist/adapters/opencode/adapter.js +48 -0
- package/dist/adapters/opencode/dialect.d.ts +10 -0
- package/dist/adapters/opencode/dialect.js +36 -0
- package/dist/adapters/opencode/layout.d.ts +9 -0
- package/dist/adapters/opencode/layout.js +25 -0
- package/dist/adapters/opencode/model-mock.d.ts +10 -0
- package/dist/adapters/opencode/model-mock.js +10 -0
- package/dist/adapters/opencode/runtime.d.ts +9 -0
- package/dist/adapters/opencode/runtime.js +21 -0
- package/dist/claude-code.d.ts +8 -2
- package/dist/claude-code.js +8 -2
- package/dist/cli-flags.d.ts +22 -0
- package/dist/cli-flags.js +38 -0
- package/dist/cli.js +406 -206
- package/dist/codex.d.ts +20 -0
- package/dist/codex.js +36 -0
- package/dist/community-skills.d.ts +1 -1
- package/dist/community-skills.js +2 -2
- package/dist/core/adapter.d.ts +88 -0
- package/dist/core/adapter.js +3 -0
- package/dist/{compile.d.ts → core/compile.d.ts} +11 -2
- package/dist/{compile.js → core/compile.js} +68 -53
- package/dist/core/compose.d.ts +79 -0
- package/dist/core/compose.js +145 -0
- package/dist/core/dialect.d.ts +51 -0
- package/dist/core/dialect.js +3 -0
- package/dist/core/harness-driver.d.ts +134 -0
- package/dist/core/harness-driver.js +3 -0
- package/dist/core/hook-protocol.d.ts +28 -0
- package/dist/core/hook-protocol.js +3 -0
- package/dist/core/layout.d.ts +43 -0
- package/dist/core/layout.js +3 -0
- package/dist/core/model-mock.d.ts +27 -0
- package/dist/core/model-mock.js +3 -0
- package/dist/core/refs.d.ts +62 -0
- package/dist/{refs.js → core/refs.js} +49 -23
- package/dist/core/runtime.d.ts +38 -0
- package/dist/core/runtime.js +3 -0
- package/dist/{types.d.ts → core/types.d.ts} +29 -0
- package/dist/{validate.d.ts → core/validate.d.ts} +1 -1
- package/dist/{validate.js → core/validate.js} +41 -4
- package/dist/e2e.d.ts +19 -0
- package/dist/e2e.js +39 -0
- package/dist/harness-assert.d.ts +29 -14
- package/dist/harness-assert.js +37 -10
- package/dist/integration.d.ts +16 -0
- package/dist/integration.js +32 -0
- package/dist/leaderboard.d.ts +33 -0
- package/dist/leaderboard.js +107 -0
- package/dist/linting.d.ts +2 -2
- package/dist/linting.js +2 -2
- package/dist/plugin-loader.d.ts +6 -3
- package/dist/plugin-loader.js +101 -49
- package/dist/scan.d.ts +49 -0
- package/dist/scan.js +177 -0
- package/dist/setup-plan.d.ts +51 -0
- package/dist/setup-plan.js +85 -0
- package/dist/skill-test.d.ts +1 -1
- package/dist/skill-test.js +1 -1
- package/dist/test-coverage.d.ts +71 -0
- package/dist/test-coverage.js +228 -0
- package/dist/testing.d.ts +3 -3
- package/dist/testing.js +3 -3
- package/dist/unit.d.ts +17 -0
- package/dist/unit.js +36 -0
- package/hooks/refs-nudge.sh +24 -0
- package/package.json +30 -15
- package/skills/audit-feedback-loop/SKILL.md +76 -0
- package/skills/edit-spec/SKILL.md +131 -0
- package/skills/enforce-rules-format/SKILL.md +71 -0
- package/skills/generate-logo/SKILL.md +103 -0
- package/skills/generate-rule/SKILL.md +64 -0
- package/skills/linter-docs/clippy.md +241 -0
- package/skills/linter-docs/eslint.md +384 -0
- package/skills/linter-docs/pylint.md +288 -0
- package/skills/linter-docs/rubocop.md +277 -0
- package/skills/linter-docs/ruff.md +187 -0
- package/skills/linter-docs/stylelint.md +247 -0
- package/skills/migrate-to-spec/SKILL.md +126 -0
- package/skills/pr-to-lint-rule/SKILL.md +97 -0
- package/skills/strengthen/SKILL.md +168 -0
- package/skills/test-harness/SKILL.md +157 -0
- package/dist/action.d.ts +0 -7
- package/dist/action.js +0 -180
- package/dist/refs.d.ts +0 -44
- package/dist/run-scripts.d.ts +0 -20
- package/dist/run-scripts.js +0 -70
- /package/dist/{agent-result.js → adapters/claude-code/agent-result.js} +0 -0
- /package/dist/{agent-runtime.d.ts → adapters/claude-code/agent-runtime.d.ts} +0 -0
- /package/dist/{agent-runtime.js → adapters/claude-code/agent-runtime.js} +0 -0
- /package/dist/{egress-proxy.d.ts → adapters/claude-code/egress-proxy.d.ts} +0 -0
- /package/dist/{egress-proxy.js → adapters/claude-code/egress-proxy.js} +0 -0
- /package/dist/{eval-baseline.d.ts → adapters/claude-code/eval-baseline.d.ts} +0 -0
- /package/dist/{eval-baseline.js → adapters/claude-code/eval-baseline.js} +0 -0
- /package/dist/{judge.d.ts → adapters/claude-code/judge.d.ts} +0 -0
- /package/dist/{judge.js → adapters/claude-code/judge.js} +0 -0
- /package/dist/{mock-entry.d.ts → adapters/claude-code/mock-entry.d.ts} +0 -0
- /package/dist/{mock-entry.js → adapters/claude-code/mock-entry.js} +0 -0
- /package/dist/{mock-model.js → adapters/claude-code/mock-model.js} +0 -0
- /package/dist/{skill-driver.js → adapters/claude-code/skill-driver.js} +0 -0
- /package/dist/{skill-runtime.d.ts → adapters/claude-code/skill-runtime.d.ts} +0 -0
- /package/dist/{skill-runtime.js → adapters/claude-code/skill-runtime.js} +0 -0
- /package/dist/{stats.d.ts → adapters/claude-code/stats.d.ts} +0 -0
- /package/dist/{stats.js → adapters/claude-code/stats.js} +0 -0
- /package/dist/{compile-generator.d.ts → core/compile-generator.d.ts} +0 -0
- /package/dist/{compile-generator.js → core/compile-generator.js} +0 -0
- /package/dist/{coverage.d.ts → core/coverage.d.ts} +0 -0
- /package/dist/{coverage.js → core/coverage.js} +0 -0
- /package/dist/{doc-refs.d.ts → core/doc-refs.d.ts} +0 -0
- /package/dist/{doc-refs.js → core/doc-refs.js} +0 -0
- /package/dist/{evolve.d.ts → core/evolve.d.ts} +0 -0
- /package/dist/{evolve.js → core/evolve.js} +0 -0
- /package/dist/{frontmatter.d.ts → core/frontmatter.d.ts} +0 -0
- /package/dist/{frontmatter.js → core/frontmatter.js} +0 -0
- /package/dist/{generate-schema.d.ts → core/generate-schema.d.ts} +0 -0
- /package/dist/{generate-schema.js → core/generate-schema.js} +0 -0
- /package/dist/{generate-types.d.ts → core/generate-types.d.ts} +0 -0
- /package/dist/{generate-types.js → core/generate-types.js} +0 -0
- /package/dist/{hash.d.ts → core/hash.d.ts} +0 -0
- /package/dist/{hash.js → core/hash.js} +0 -0
- /package/dist/{inline.d.ts → core/inline.d.ts} +0 -0
- /package/dist/{inline.js → core/inline.js} +0 -0
- /package/dist/{integrity.d.ts → core/integrity.d.ts} +0 -0
- /package/dist/{integrity.js → core/integrity.js} +0 -0
- /package/dist/{linters.d.ts → core/linters.d.ts} +0 -0
- /package/dist/{linters.js → core/linters.js} +0 -0
- /package/dist/{mcp.d.ts → core/mcp.d.ts} +0 -0
- /package/dist/{mcp.js → core/mcp.js} +0 -0
- /package/dist/{orphans.d.ts → core/orphans.d.ts} +0 -0
- /package/dist/{orphans.js → core/orphans.js} +0 -0
- /package/dist/{proofs.d.ts → core/proofs.d.ts} +0 -0
- /package/dist/{proofs.js → core/proofs.js} +0 -0
- /package/dist/{session.d.ts → core/session.d.ts} +0 -0
- /package/dist/{session.js → core/session.js} +0 -0
- /package/dist/{sidecar.d.ts → core/sidecar.d.ts} +0 -0
- /package/dist/{sidecar.js → core/sidecar.js} +0 -0
- /package/dist/{spec.d.ts → core/spec.d.ts} +0 -0
- /package/dist/{spec.js → core/spec.js} +0 -0
- /package/dist/{symbols.d.ts → core/symbols.d.ts} +0 -0
- /package/dist/{symbols.js → core/symbols.js} +0 -0
- /package/dist/{test-utils.d.ts → core/test-utils.d.ts} +0 -0
- /package/dist/{test-utils.js → core/test-utils.js} +0 -0
- /package/dist/{types.js → core/types.js} +0 -0
- /package/{.claude-plugin/hooks → hooks}/post-edit.sh +0 -0
- /package/{.claude-plugin/hooks → hooks}/pre-edit.sh +0 -0
- /package/{.claude-plugin/hooks → hooks}/session-start.sh +0 -0
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Untested-surface detection: the third gap detector, alongside orphan-docs.
|
|
3
|
+
*
|
|
4
|
+
* Stale-ref detection catches specs pointing at files that vanished.
|
|
5
|
+
* Orphan-docs catches docs nothing references. This catches harness *surfaces*
|
|
6
|
+
* — skills, subagents, and hooks — that ship without a test or eval. A surface
|
|
7
|
+
* with no test is a probabilistic-compliance gap hiding in the deterministic
|
|
8
|
+
* layer: nothing measures whether it still does what it claims.
|
|
9
|
+
*
|
|
10
|
+
* Two detectors decide "tested", OR'd, so a test placed ANYWHERE counts:
|
|
11
|
+
* 1. colocation — a `*.{harness,eval}.mjs` next to the surface (the zero-config
|
|
12
|
+
* convention the warning suggests): `skills/foo/*.eval.mjs`,
|
|
13
|
+
* `agents/bar.harness.mjs`, `hooks/pre-edit.harness.mjs`.
|
|
14
|
+
* 2. content-reference — any discovered test (incl. `*.test.ts`) that names the
|
|
15
|
+
* surface by PATH (`skills/foo`, `hooks/pre-edit.sh`) or NAMESPACE
|
|
16
|
+
* (`vigiles:foo`). Not bare-name — too fuzzy.
|
|
17
|
+
*
|
|
18
|
+
* Warning-by-default (a nudge, not a gate). User-invoked skills
|
|
19
|
+
* (`disable-model-invocation: true`) can't auto-trigger, so they're exempt by
|
|
20
|
+
* default; flip `includeUserInvokedSkills` to demand an outcome test for them.
|
|
21
|
+
* Per-surface opt-out: a `vigiles:ignore-test` marker in the surface file.
|
|
22
|
+
*/
|
|
23
|
+
export type SurfaceKind = "skill" | "agent" | "hook";
|
|
24
|
+
export interface Surface {
|
|
25
|
+
readonly kind: SurfaceKind;
|
|
26
|
+
/** Repo-relative path to the surface file (SKILL.md / agent .md / hook script). */
|
|
27
|
+
readonly path: string;
|
|
28
|
+
/** Stable name: skill dir, agent basename, or hook script basename. */
|
|
29
|
+
readonly name: string;
|
|
30
|
+
/** Substrings a test may reference to "cover" this surface (path / namespace). */
|
|
31
|
+
readonly tokens: readonly string[];
|
|
32
|
+
/** True for skills with `disable-model-invocation: true`. */
|
|
33
|
+
readonly userInvoked: boolean;
|
|
34
|
+
}
|
|
35
|
+
export interface UntestedReport {
|
|
36
|
+
/** Total surfaces considered (after exemptions). */
|
|
37
|
+
readonly total: number;
|
|
38
|
+
readonly covered: readonly Surface[];
|
|
39
|
+
readonly untested: readonly Surface[];
|
|
40
|
+
/** Surfaces skipped (user-invoked skills, `vigiles:ignore-test`). */
|
|
41
|
+
readonly exempt: number;
|
|
42
|
+
}
|
|
43
|
+
export interface TestCoverageOptions {
|
|
44
|
+
/** Repository root. Defaults to `process.cwd()`. */
|
|
45
|
+
readonly basePath?: string;
|
|
46
|
+
/** Scan skills under `skills/` and `.claude/skills/`. Default true. */
|
|
47
|
+
readonly skills?: boolean;
|
|
48
|
+
/** Scan subagents under `agents/` and `.claude/agents/`. Default true. */
|
|
49
|
+
readonly agents?: boolean;
|
|
50
|
+
/** Scan hook scripts referenced from plugin.json / settings.json. Default true. */
|
|
51
|
+
readonly hooks?: boolean;
|
|
52
|
+
/** Require a test for user-invoked (disable-model-invocation) skills. Default false. */
|
|
53
|
+
readonly includeUserInvokedSkills?: boolean;
|
|
54
|
+
/** Globs of test files that count as coverage. */
|
|
55
|
+
readonly testGlobs?: readonly string[];
|
|
56
|
+
/** Extra ignore globs (added to node_modules/dist/.git/.vigiles). */
|
|
57
|
+
readonly exclude?: readonly string[];
|
|
58
|
+
}
|
|
59
|
+
/**
|
|
60
|
+
* Find harness surfaces (skills / agents / hooks) that no test or eval covers.
|
|
61
|
+
* A surface is covered by a colocated `*.{harness,eval}.mjs` OR any discovered
|
|
62
|
+
* test that references its path/namespace. User-invoked skills are exempt unless
|
|
63
|
+
* `includeUserInvokedSkills` is set; any surface with a `vigiles:ignore-test`
|
|
64
|
+
* marker is exempt.
|
|
65
|
+
*/
|
|
66
|
+
export declare function findUntestedSurfaces(options?: TestCoverageOptions): UntestedReport;
|
|
67
|
+
/** Suggested colocated test path for an untested surface (shown in the warning). */
|
|
68
|
+
export declare function suggestedTestPath(surface: Surface): string;
|
|
69
|
+
/** Format an untested-surface report as human-readable text. */
|
|
70
|
+
export declare function formatUntestedReport(report: UntestedReport): string;
|
|
71
|
+
//# sourceMappingURL=test-coverage.d.ts.map
|
|
@@ -0,0 +1,228 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
/**
|
|
3
|
+
* Untested-surface detection: the third gap detector, alongside orphan-docs.
|
|
4
|
+
*
|
|
5
|
+
* Stale-ref detection catches specs pointing at files that vanished.
|
|
6
|
+
* Orphan-docs catches docs nothing references. This catches harness *surfaces*
|
|
7
|
+
* — skills, subagents, and hooks — that ship without a test or eval. A surface
|
|
8
|
+
* with no test is a probabilistic-compliance gap hiding in the deterministic
|
|
9
|
+
* layer: nothing measures whether it still does what it claims.
|
|
10
|
+
*
|
|
11
|
+
* Two detectors decide "tested", OR'd, so a test placed ANYWHERE counts:
|
|
12
|
+
* 1. colocation — a `*.{harness,eval}.mjs` next to the surface (the zero-config
|
|
13
|
+
* convention the warning suggests): `skills/foo/*.eval.mjs`,
|
|
14
|
+
* `agents/bar.harness.mjs`, `hooks/pre-edit.harness.mjs`.
|
|
15
|
+
* 2. content-reference — any discovered test (incl. `*.test.ts`) that names the
|
|
16
|
+
* surface by PATH (`skills/foo`, `hooks/pre-edit.sh`) or NAMESPACE
|
|
17
|
+
* (`vigiles:foo`). Not bare-name — too fuzzy.
|
|
18
|
+
*
|
|
19
|
+
* Warning-by-default (a nudge, not a gate). User-invoked skills
|
|
20
|
+
* (`disable-model-invocation: true`) can't auto-trigger, so they're exempt by
|
|
21
|
+
* default; flip `includeUserInvokedSkills` to demand an outcome test for them.
|
|
22
|
+
* Per-surface opt-out: a `vigiles:ignore-test` marker in the surface file.
|
|
23
|
+
*/
|
|
24
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
25
|
+
exports.findUntestedSurfaces = findUntestedSurfaces;
|
|
26
|
+
exports.suggestedTestPath = suggestedTestPath;
|
|
27
|
+
exports.formatUntestedReport = formatUntestedReport;
|
|
28
|
+
const node_fs_1 = require("node:fs");
|
|
29
|
+
const node_path_1 = require("node:path");
|
|
30
|
+
const glob_1 = require("glob");
|
|
31
|
+
// ---------------------------------------------------------------------------
|
|
32
|
+
// Internals
|
|
33
|
+
// ---------------------------------------------------------------------------
|
|
34
|
+
const DEFAULT_TEST_GLOBS = [
|
|
35
|
+
"**/*.harness.mjs",
|
|
36
|
+
"**/*.eval.mjs",
|
|
37
|
+
"**/*.test.ts",
|
|
38
|
+
"**/*.test.mts",
|
|
39
|
+
"**/*.test.cts",
|
|
40
|
+
"**/*.test.js",
|
|
41
|
+
"**/*.test.mjs",
|
|
42
|
+
"**/*.test.cjs",
|
|
43
|
+
];
|
|
44
|
+
const DEFAULT_IGNORE = [
|
|
45
|
+
"node_modules/**",
|
|
46
|
+
"dist/**",
|
|
47
|
+
".vigiles/**",
|
|
48
|
+
".git/**",
|
|
49
|
+
];
|
|
50
|
+
const IGNORE_MARKER = "vigiles:ignore-test";
|
|
51
|
+
const SCRIPT_RE = /[\w./${}@-]+\.(?:sh|mjs|cjs|js|ts|py|rb)/g;
|
|
52
|
+
function read(path) {
|
|
53
|
+
try {
|
|
54
|
+
return (0, node_fs_1.readFileSync)(path, "utf-8");
|
|
55
|
+
}
|
|
56
|
+
catch {
|
|
57
|
+
return "";
|
|
58
|
+
}
|
|
59
|
+
}
|
|
60
|
+
/** A SKILL.md is user-invoked when its frontmatter sets disable-model-invocation. */
|
|
61
|
+
function isUserInvoked(content) {
|
|
62
|
+
return /^\s*disable-model-invocation:\s*true\s*$/m.test(content);
|
|
63
|
+
}
|
|
64
|
+
function discoverSkills(basePath, ignore) {
|
|
65
|
+
const out = [];
|
|
66
|
+
const found = (0, glob_1.globSync)(["skills/*/SKILL.md", ".claude/skills/*/SKILL.md"], {
|
|
67
|
+
cwd: basePath,
|
|
68
|
+
ignore,
|
|
69
|
+
});
|
|
70
|
+
for (const path of found.sort()) {
|
|
71
|
+
const name = (0, node_path_1.basename)((0, node_path_1.dirname)(path));
|
|
72
|
+
const content = read((0, node_path_1.join)(basePath, path));
|
|
73
|
+
if (content.includes(IGNORE_MARKER))
|
|
74
|
+
continue;
|
|
75
|
+
out.push({
|
|
76
|
+
kind: "skill",
|
|
77
|
+
path,
|
|
78
|
+
name,
|
|
79
|
+
tokens: [`skills/${name}`, `:${name}`],
|
|
80
|
+
userInvoked: isUserInvoked(content),
|
|
81
|
+
});
|
|
82
|
+
}
|
|
83
|
+
return out;
|
|
84
|
+
}
|
|
85
|
+
function discoverAgents(basePath, ignore) {
|
|
86
|
+
const out = [];
|
|
87
|
+
const found = (0, glob_1.globSync)(["agents/*.md", ".claude/agents/*.md"], {
|
|
88
|
+
cwd: basePath,
|
|
89
|
+
ignore,
|
|
90
|
+
});
|
|
91
|
+
for (const path of found.sort()) {
|
|
92
|
+
if (path.endsWith(".spec.ts"))
|
|
93
|
+
continue;
|
|
94
|
+
const content = read((0, node_path_1.join)(basePath, path));
|
|
95
|
+
if (content.includes(IGNORE_MARKER))
|
|
96
|
+
continue;
|
|
97
|
+
const name = (0, node_path_1.basename)(path, ".md");
|
|
98
|
+
const dir = (0, node_path_1.dirname)(path);
|
|
99
|
+
out.push({
|
|
100
|
+
kind: "agent",
|
|
101
|
+
path,
|
|
102
|
+
name,
|
|
103
|
+
tokens: [`${dir}/${name}`],
|
|
104
|
+
userInvoked: false,
|
|
105
|
+
});
|
|
106
|
+
}
|
|
107
|
+
return out;
|
|
108
|
+
}
|
|
109
|
+
/** Hook-script paths referenced from a manifest's `hooks` block (file hooks only). */
|
|
110
|
+
function hookScripts(basePath, manifest) {
|
|
111
|
+
if (!(0, node_fs_1.existsSync)((0, node_path_1.join)(basePath, manifest)))
|
|
112
|
+
return [];
|
|
113
|
+
let hooks;
|
|
114
|
+
try {
|
|
115
|
+
hooks = JSON.parse(read((0, node_path_1.join)(basePath, manifest)))
|
|
116
|
+
.hooks;
|
|
117
|
+
}
|
|
118
|
+
catch {
|
|
119
|
+
return [];
|
|
120
|
+
}
|
|
121
|
+
if (hooks === undefined)
|
|
122
|
+
return [];
|
|
123
|
+
const text = JSON.stringify(hooks);
|
|
124
|
+
const scripts = new Set();
|
|
125
|
+
for (const m of text.matchAll(SCRIPT_RE)) {
|
|
126
|
+
const rel = m[0]
|
|
127
|
+
.replace("${CLAUDE_PLUGIN_ROOT}/", "")
|
|
128
|
+
.replace(/^\$\{CLAUDE_PLUGIN_ROOT\}/, "")
|
|
129
|
+
.replace(/^\.\//, "");
|
|
130
|
+
if ((0, node_fs_1.existsSync)((0, node_path_1.join)(basePath, rel)))
|
|
131
|
+
scripts.add(rel);
|
|
132
|
+
}
|
|
133
|
+
return [...scripts];
|
|
134
|
+
}
|
|
135
|
+
function discoverHooks(basePath) {
|
|
136
|
+
const scripts = new Set();
|
|
137
|
+
for (const m of [
|
|
138
|
+
".claude-plugin/plugin.json",
|
|
139
|
+
".claude/settings.json",
|
|
140
|
+
".claude/settings.local.json",
|
|
141
|
+
]) {
|
|
142
|
+
for (const s of hookScripts(basePath, m))
|
|
143
|
+
scripts.add(s);
|
|
144
|
+
}
|
|
145
|
+
return [...scripts].sort().map((path) => ({
|
|
146
|
+
kind: "hook",
|
|
147
|
+
path,
|
|
148
|
+
name: (0, node_path_1.basename)(path).replace(/\.[^.]+$/, ""),
|
|
149
|
+
tokens: [path],
|
|
150
|
+
userInvoked: false,
|
|
151
|
+
}));
|
|
152
|
+
}
|
|
153
|
+
function discoverTests(basePath, globs, ignore) {
|
|
154
|
+
const found = (0, glob_1.globSync)([...globs], { cwd: basePath, ignore });
|
|
155
|
+
return found.map((path) => ({ path, content: read((0, node_path_1.join)(basePath, path)) }));
|
|
156
|
+
}
|
|
157
|
+
/** Colocated: a test inside a skill dir, or a name-prefixed sibling of an agent/hook. */
|
|
158
|
+
function isColocated(surface, testPath) {
|
|
159
|
+
if (surface.kind === "skill") {
|
|
160
|
+
return testPath.startsWith(`${(0, node_path_1.dirname)(surface.path)}/`);
|
|
161
|
+
}
|
|
162
|
+
return ((0, node_path_1.dirname)(testPath) === (0, node_path_1.dirname)(surface.path) &&
|
|
163
|
+
(0, node_path_1.basename)(testPath).startsWith(`${surface.name}.`));
|
|
164
|
+
}
|
|
165
|
+
function isCovered(surface, tests) {
|
|
166
|
+
for (const t of tests) {
|
|
167
|
+
if (t.path === surface.path)
|
|
168
|
+
continue;
|
|
169
|
+
if (isColocated(surface, t.path))
|
|
170
|
+
return true;
|
|
171
|
+
if (surface.tokens.some((tok) => t.content.includes(tok)))
|
|
172
|
+
return true;
|
|
173
|
+
}
|
|
174
|
+
return false;
|
|
175
|
+
}
|
|
176
|
+
// ---------------------------------------------------------------------------
|
|
177
|
+
// Public API
|
|
178
|
+
// ---------------------------------------------------------------------------
|
|
179
|
+
/**
|
|
180
|
+
* Find harness surfaces (skills / agents / hooks) that no test or eval covers.
|
|
181
|
+
* A surface is covered by a colocated `*.{harness,eval}.mjs` OR any discovered
|
|
182
|
+
* test that references its path/namespace. User-invoked skills are exempt unless
|
|
183
|
+
* `includeUserInvokedSkills` is set; any surface with a `vigiles:ignore-test`
|
|
184
|
+
* marker is exempt.
|
|
185
|
+
*/
|
|
186
|
+
function findUntestedSurfaces(options = {}) {
|
|
187
|
+
const basePath = options.basePath ?? process.cwd();
|
|
188
|
+
const ignore = [...DEFAULT_IGNORE, ...(options.exclude ?? [])];
|
|
189
|
+
const globs = options.testGlobs ?? DEFAULT_TEST_GLOBS;
|
|
190
|
+
const surfaces = [];
|
|
191
|
+
if (options.skills !== false)
|
|
192
|
+
surfaces.push(...discoverSkills(basePath, ignore));
|
|
193
|
+
if (options.agents !== false)
|
|
194
|
+
surfaces.push(...discoverAgents(basePath, ignore));
|
|
195
|
+
if (options.hooks !== false)
|
|
196
|
+
surfaces.push(...discoverHooks(basePath));
|
|
197
|
+
const considered = surfaces.filter((s) => !s.userInvoked || s.kind !== "skill" || options.includeUserInvokedSkills);
|
|
198
|
+
const exempt = surfaces.length - considered.length;
|
|
199
|
+
const tests = discoverTests(basePath, globs, ignore);
|
|
200
|
+
const covered = [];
|
|
201
|
+
const untested = [];
|
|
202
|
+
for (const s of considered) {
|
|
203
|
+
(isCovered(s, tests) ? covered : untested).push(s);
|
|
204
|
+
}
|
|
205
|
+
return { total: considered.length, covered, untested, exempt };
|
|
206
|
+
}
|
|
207
|
+
/** Suggested colocated test path for an untested surface (shown in the warning). */
|
|
208
|
+
function suggestedTestPath(surface) {
|
|
209
|
+
if (surface.kind === "skill") {
|
|
210
|
+
return `${(0, node_path_1.dirname)(surface.path)}/${surface.name}.eval.mjs`;
|
|
211
|
+
}
|
|
212
|
+
return `${(0, node_path_1.dirname)(surface.path)}/${surface.name}.harness.mjs`;
|
|
213
|
+
}
|
|
214
|
+
/** Format an untested-surface report as human-readable text. */
|
|
215
|
+
function formatUntestedReport(report) {
|
|
216
|
+
if (report.untested.length === 0) {
|
|
217
|
+
const tail = report.exempt > 0 ? ` (${String(report.exempt)} exempt)` : "";
|
|
218
|
+
return `✓ all ${String(report.total)} surface(s) have a test or eval${tail}`;
|
|
219
|
+
}
|
|
220
|
+
const lines = [
|
|
221
|
+
`⚠ ${String(report.untested.length)} surface(s) with no test or eval:`,
|
|
222
|
+
];
|
|
223
|
+
for (const s of report.untested) {
|
|
224
|
+
lines.push(` ${s.kind} ${s.path} — add e.g. ${suggestedTestPath(s)}`);
|
|
225
|
+
}
|
|
226
|
+
return lines.join("\n");
|
|
227
|
+
}
|
|
228
|
+
//# sourceMappingURL=test-coverage.js.map
|
package/dist/testing.d.ts
CHANGED
|
@@ -5,8 +5,8 @@
|
|
|
5
5
|
* separate from `vigiles/claude-code` so this surface can stay harness-agnostic as
|
|
6
6
|
* more harnesses are added. Granular paths (`vigiles/run-hook`, etc.) still work.
|
|
7
7
|
*/
|
|
8
|
-
export * from "./run-hook.js";
|
|
9
|
-
export * from "./harness-test.js";
|
|
10
|
-
export * from "./eval.js";
|
|
8
|
+
export * from "./adapters/claude-code/run-hook.js";
|
|
9
|
+
export * from "./adapters/claude-code/harness-test.js";
|
|
10
|
+
export * from "./adapters/claude-code/eval.js";
|
|
11
11
|
export * from "./harness-assert.js";
|
|
12
12
|
//# sourceMappingURL=testing.d.ts.map
|
package/dist/testing.js
CHANGED
|
@@ -21,8 +21,8 @@ Object.defineProperty(exports, "__esModule", { value: true });
|
|
|
21
21
|
* separate from `vigiles/claude-code` so this surface can stay harness-agnostic as
|
|
22
22
|
* more harnesses are added. Granular paths (`vigiles/run-hook`, etc.) still work.
|
|
23
23
|
*/
|
|
24
|
-
__exportStar(require("./run-hook.js"), exports);
|
|
25
|
-
__exportStar(require("./harness-test.js"), exports);
|
|
26
|
-
__exportStar(require("./eval.js"), exports);
|
|
24
|
+
__exportStar(require("./adapters/claude-code/run-hook.js"), exports);
|
|
25
|
+
__exportStar(require("./adapters/claude-code/harness-test.js"), exports);
|
|
26
|
+
__exportStar(require("./adapters/claude-code/eval.js"), exports);
|
|
27
27
|
__exportStar(require("./harness-assert.js"), exports);
|
|
28
28
|
//# sourceMappingURL=testing.js.map
|
package/dist/unit.d.ts
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* `vigiles/unit` — the **no-capability** harness-testing surface.
|
|
3
|
+
*
|
|
4
|
+
* Importing from here is a contract: this test needs **no `claude`, no model, no
|
|
5
|
+
* bubblewrap, no network** — only a process. It is the cheap base of the pyramid:
|
|
6
|
+
* `runHook` (pipe a synthesized event to a hook and read the block/allow
|
|
7
|
+
* decision), the bare `Trace` predicates, the `assert*` helpers, and the pure
|
|
8
|
+
* parsers. A `*.test.ts` should import **only** from here (enforced by lint).
|
|
9
|
+
*
|
|
10
|
+
* Higher tiers re-export this one — dependencies point downward only
|
|
11
|
+
* (e2e → integration → unit), never up. See `vigiles/integration` and
|
|
12
|
+
* `vigiles/e2e`.
|
|
13
|
+
*/
|
|
14
|
+
export * from "./harness-assert.js";
|
|
15
|
+
export { runHook, parseHookOutput, decideHook, } from "./adapters/claude-code/run-hook.js";
|
|
16
|
+
export type { HookInput, HookOutput, HookRunResult, RunHookOptions, } from "./adapters/claude-code/run-hook.js";
|
|
17
|
+
//# sourceMappingURL=unit.d.ts.map
|
package/dist/unit.js
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
|
|
3
|
+
if (k2 === undefined) k2 = k;
|
|
4
|
+
var desc = Object.getOwnPropertyDescriptor(m, k);
|
|
5
|
+
if (!desc || ("get" in desc ? !m.__esModule : desc.writable || desc.configurable)) {
|
|
6
|
+
desc = { enumerable: true, get: function() { return m[k]; } };
|
|
7
|
+
}
|
|
8
|
+
Object.defineProperty(o, k2, desc);
|
|
9
|
+
}) : (function(o, m, k, k2) {
|
|
10
|
+
if (k2 === undefined) k2 = k;
|
|
11
|
+
o[k2] = m[k];
|
|
12
|
+
}));
|
|
13
|
+
var __exportStar = (this && this.__exportStar) || function(m, exports) {
|
|
14
|
+
for (var p in m) if (p !== "default" && !Object.prototype.hasOwnProperty.call(exports, p)) __createBinding(exports, m, p);
|
|
15
|
+
};
|
|
16
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
17
|
+
exports.decideHook = exports.parseHookOutput = exports.runHook = void 0;
|
|
18
|
+
/**
|
|
19
|
+
* `vigiles/unit` — the **no-capability** harness-testing surface.
|
|
20
|
+
*
|
|
21
|
+
* Importing from here is a contract: this test needs **no `claude`, no model, no
|
|
22
|
+
* bubblewrap, no network** — only a process. It is the cheap base of the pyramid:
|
|
23
|
+
* `runHook` (pipe a synthesized event to a hook and read the block/allow
|
|
24
|
+
* decision), the bare `Trace` predicates, the `assert*` helpers, and the pure
|
|
25
|
+
* parsers. A `*.test.ts` should import **only** from here (enforced by lint).
|
|
26
|
+
*
|
|
27
|
+
* Higher tiers re-export this one — dependencies point downward only
|
|
28
|
+
* (e2e → integration → unit), never up. See `vigiles/integration` and
|
|
29
|
+
* `vigiles/e2e`.
|
|
30
|
+
*/
|
|
31
|
+
__exportStar(require("./harness-assert.js"), exports);
|
|
32
|
+
var run_hook_js_1 = require("./adapters/claude-code/run-hook.js");
|
|
33
|
+
Object.defineProperty(exports, "runHook", { enumerable: true, get: function () { return run_hook_js_1.runHook; } });
|
|
34
|
+
Object.defineProperty(exports, "parseHookOutput", { enumerable: true, get: function () { return run_hook_js_1.parseHookOutput; } });
|
|
35
|
+
Object.defineProperty(exports, "decideHook", { enumerable: true, get: function () { return run_hook_js_1.decideHook; } });
|
|
36
|
+
//# sourceMappingURL=unit.js.map
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
# PostToolUse hook — nudge the agent to express references in instruction files
|
|
3
|
+
# (CLAUDE.md / AGENTS.md / SKILL.md) as vigiles marks, so `vigiles audit` can
|
|
4
|
+
# actually verify them. Non-blocking by default; set the `unmarked-refs` rule to
|
|
5
|
+
# "error" in .vigilesrc.json to turn the nudge into a hard block, or to false to
|
|
6
|
+
# disable it. Runs as its OWN PostToolUse entry so its stdout stays clean JSON.
|
|
7
|
+
|
|
8
|
+
set -uo pipefail
|
|
9
|
+
|
|
10
|
+
INPUT=$(cat)
|
|
11
|
+
|
|
12
|
+
# No npx / not a Node project → nothing to do, never disrupt the edit.
|
|
13
|
+
command -v npx >/dev/null 2>&1 || exit 0
|
|
14
|
+
[ -f package.json ] || exit 0
|
|
15
|
+
|
|
16
|
+
printf '%s' "$INPUT" | npx vigiles refs-hook
|
|
17
|
+
status=${PIPESTATUS[1]}
|
|
18
|
+
|
|
19
|
+
# Propagate ONLY an explicit block (exit 2). Swallow tool/setup errors so a
|
|
20
|
+
# missing dep or transient failure never blocks the agent's edit.
|
|
21
|
+
if [ "$status" -eq 2 ]; then
|
|
22
|
+
exit 2
|
|
23
|
+
fi
|
|
24
|
+
exit 0
|
package/package.json
CHANGED
|
@@ -1,28 +1,33 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "vigiles",
|
|
3
|
-
"version": "
|
|
3
|
+
"version": "3.0.0",
|
|
4
4
|
"description": "Compile .spec.ts files to instruction files (CLAUDE.md, AGENTS.md) with linter cross-referencing",
|
|
5
5
|
"bin": {
|
|
6
6
|
"vigiles": "dist/cli.js"
|
|
7
7
|
},
|
|
8
|
-
"main": "./dist/spec.js",
|
|
9
|
-
"types": "./dist/spec.d.ts",
|
|
8
|
+
"main": "./dist/core/spec.js",
|
|
9
|
+
"types": "./dist/core/spec.d.ts",
|
|
10
10
|
"exports": {
|
|
11
|
-
".": "./dist/spec.js",
|
|
11
|
+
".": "./dist/core/spec.js",
|
|
12
|
+
"./unit": "./dist/unit.js",
|
|
13
|
+
"./integration": "./dist/integration.js",
|
|
14
|
+
"./e2e": "./dist/e2e.js",
|
|
12
15
|
"./linting": "./dist/linting.js",
|
|
13
16
|
"./testing": "./dist/testing.js",
|
|
14
17
|
"./claude-code": "./dist/claude-code.js",
|
|
15
|
-
"./
|
|
16
|
-
"./
|
|
17
|
-
"./
|
|
18
|
-
"./
|
|
19
|
-
"./
|
|
18
|
+
"./codex": "./dist/codex.js",
|
|
19
|
+
"./adapter": "./dist/adapter.js",
|
|
20
|
+
"./spec": "./dist/core/spec.js",
|
|
21
|
+
"./compile": "./dist/core/compile.js",
|
|
22
|
+
"./linters": "./dist/core/linters.js",
|
|
23
|
+
"./eval": "./dist/adapters/claude-code/eval.js",
|
|
24
|
+
"./harness-test": "./dist/adapters/claude-code/harness-test.js",
|
|
20
25
|
"./harness-assert": "./dist/harness-assert.js",
|
|
21
|
-
"./run-hook": "./dist/run-hook.js",
|
|
22
|
-
"./plugin-loader": "./dist/plugin-loader.js",
|
|
23
|
-
"./mcp": "./dist/mcp.js",
|
|
24
|
-
"./judge": "./dist/judge.js",
|
|
25
|
-
"./mock-model": "./dist/mock-model.js",
|
|
26
|
+
"./run-hook": "./dist/adapters/claude-code/run-hook.js",
|
|
27
|
+
"./plugin-loader": "./dist/adapters/claude-code/plugin-loader.js",
|
|
28
|
+
"./mcp": "./dist/core/mcp.js",
|
|
29
|
+
"./judge": "./dist/adapters/claude-code/judge.js",
|
|
30
|
+
"./mock-model": "./dist/adapters/claude-code/mock-model.js",
|
|
26
31
|
"./vitest": {
|
|
27
32
|
"types": "./dist/vitest.d.mts",
|
|
28
33
|
"default": "./dist/vitest.mjs"
|
|
@@ -38,6 +43,9 @@
|
|
|
38
43
|
"!dist/**/*.test.d.ts",
|
|
39
44
|
"action.yml",
|
|
40
45
|
".claude-plugin",
|
|
46
|
+
"hooks",
|
|
47
|
+
"skills",
|
|
48
|
+
"!skills/**/*.spec.ts",
|
|
41
49
|
"README.md",
|
|
42
50
|
"LICENSE"
|
|
43
51
|
],
|
|
@@ -49,7 +57,10 @@
|
|
|
49
57
|
"fmt": "prettier --write .",
|
|
50
58
|
"fmt:check": "prettier --check .",
|
|
51
59
|
"demo": "npm run build && bash examples/demo/run.sh",
|
|
52
|
-
"test:
|
|
60
|
+
"test:unit": "npm run build && vitest run --project unit",
|
|
61
|
+
"test:integration": "npm run build && vitest run --project integration",
|
|
62
|
+
"test:e2e": "npm run build && vitest run --project e2e",
|
|
63
|
+
"test:cli-e2e": "bash test/e2e/run.sh",
|
|
53
64
|
"test:harness": "npm run build && node dist/cli.js test",
|
|
54
65
|
"test:eval": "npm run build && node dist/cli.js eval",
|
|
55
66
|
"test:vitest": "npm run build && vitest run --project runners",
|
|
@@ -66,6 +77,8 @@
|
|
|
66
77
|
"@typescript-eslint/parser": "^8.58.0",
|
|
67
78
|
"@vitest/coverage-v8": "^4.1.8",
|
|
68
79
|
"eslint": "^10.1.0",
|
|
80
|
+
"eslint-import-resolver-typescript": "^4.4.5",
|
|
81
|
+
"eslint-plugin-boundaries": "^6.0.2",
|
|
69
82
|
"eslint-plugin-sonarjs": "^4.0.2",
|
|
70
83
|
"globals": "^17.4.0",
|
|
71
84
|
"jest": "^30.4.2",
|
|
@@ -91,11 +104,13 @@
|
|
|
91
104
|
"@ast-grep/lang-ruby": "^0.0.7",
|
|
92
105
|
"@ast-grep/lang-rust": "^0.0.7",
|
|
93
106
|
"@ast-grep/napi": "^0.43.0",
|
|
107
|
+
"@iarna/toml": "^2.2.5",
|
|
94
108
|
"@jackchuka/mdschema": "^0.12.8",
|
|
95
109
|
"cosmiconfig": "^9.0.1",
|
|
96
110
|
"glob": "^13.0.6",
|
|
97
111
|
"js-yaml": "^4.1.0",
|
|
98
112
|
"minimatch": "^10.0.1",
|
|
113
|
+
"ts-essentials": "^10.2.1",
|
|
99
114
|
"typescript": "^5.9.3"
|
|
100
115
|
}
|
|
101
116
|
}
|
|
@@ -0,0 +1,76 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: audit-feedback-loop
|
|
3
|
+
description: Scan the current repo and score its feedback loop maturity for AI-assisted development
|
|
4
|
+
disable-model-invocation: true
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
Scan the current repository and score its feedback loop maturity for AI-assisted development.
|
|
8
|
+
|
|
9
|
+
## Instructions
|
|
10
|
+
|
|
11
|
+
Analyze this repository and score its **feedback loop maturity** using the levels below. Check for each signal, then output a summary report.
|
|
12
|
+
|
|
13
|
+
### Maturity Levels
|
|
14
|
+
|
|
15
|
+
**Level 0 — Vibes**
|
|
16
|
+
No CI config, no linter rules, no CLAUDE.md. The AI agent is flying blind.
|
|
17
|
+
|
|
18
|
+
**Level 1 — Guardrails**
|
|
19
|
+
Has CI + standard linters, but no custom rules. The agent gets basic feedback but can't learn project-specific conventions.
|
|
20
|
+
|
|
21
|
+
**Level 2 — Architecture as Code**
|
|
22
|
+
Has custom lint rules, CLAUDE.md rules have enforcement annotations. The agent gets rich, project-specific feedback.
|
|
23
|
+
|
|
24
|
+
**Level 3 — The Organism**
|
|
25
|
+
Has CI + custom rules + screenshot/visual tests + observability + scheduled agent tasks. The entire development loop is instrumented.
|
|
26
|
+
|
|
27
|
+
### Signals to Check
|
|
28
|
+
|
|
29
|
+
Scan the repository for the following and note which exist:
|
|
30
|
+
|
|
31
|
+
1. **CI Configuration**: Look for `.github/workflows/`, `.circleci/`, `Jenkinsfile`, `.gitlab-ci.yml`, `bitbucket-pipelines.yml`, `.travis.yml`, etc.
|
|
32
|
+
2. **Linter Config** (language-aware):
|
|
33
|
+
- **JS/TS**: `eslint.config.*`, `.eslintrc*`, `biome.json`, `.prettierrc*`, `deno.json`
|
|
34
|
+
- **Python**: `pyproject.toml` (look for `[tool.ruff]`, `[tool.pylint]`, `[tool.flake8]`), `setup.cfg`, `.flake8`, `ruff.toml`
|
|
35
|
+
- **Rust**: `clippy.toml`, `.clippy.toml`, `rustfmt.toml`
|
|
36
|
+
- **Go**: `.golangci.yml`, `.golangci.yaml`
|
|
37
|
+
- **Ruby**: `.rubocop.yml`
|
|
38
|
+
- **Java/Kotlin**: `checkstyle.xml`, `pmd.xml`, `detekt.yml`
|
|
39
|
+
3. **Custom Lint Rules**: Look for custom plugins, rule directories, or inline rule definitions in linter configs
|
|
40
|
+
- JS/TS: `eslint-plugin-*`, `eslint-rules/` directories
|
|
41
|
+
- Python: custom Ruff/Pylint plugins, AST-based checks
|
|
42
|
+
- Rust: custom Clippy lints
|
|
43
|
+
- Go: custom analyzers
|
|
44
|
+
4. **CLAUDE.md**: Check if `CLAUDE.md` exists at the repo root
|
|
45
|
+
5. **CLAUDE.md Enforcement**: Check if using vigiles v2 specs (`CLAUDE.md.spec.ts` exists) or v1 annotations (`**Enforced by:**` in CLAUDE.md). v2 specs = higher maturity.
|
|
46
|
+
6. **Type-Safe Specs**: Check for `CLAUDE.md.spec.ts` or `*.spec.ts` files — indicates typed spec compilation via vigiles v2
|
|
47
|
+
7. **Generated Types**: Check for `.vigiles/generated.d.ts` — indicates linter rules are type-checked at authoring time
|
|
48
|
+
8. **Screenshot/Visual Tests**: Look for Playwright (`playwright.config.*`), Cypress (`cypress.config.*`), Chromatic, Percy, BackstopJS configs
|
|
49
|
+
9. **Observability**: Search for imports/usage of `@sentry/`, `dd-trace`, `@datadog/`, `newrelic`, `@opentelemetry/`, `sentry_sdk`, `structlog`, `tracing` (Rust), `opentelemetry` in source files
|
|
50
|
+
10. **Scheduled Agent Tasks**: Look for cron patterns in CI configs, `.github/workflows/` with `schedule:` triggers, or references to scheduled Claude Code tasks
|
|
51
|
+
|
|
52
|
+
### Output Format
|
|
53
|
+
|
|
54
|
+
```
|
|
55
|
+
## Feedback Loop Audit
|
|
56
|
+
|
|
57
|
+
**Repository:** <repo name>
|
|
58
|
+
**Primary language(s):** <detected languages>
|
|
59
|
+
**Score: Level X — <Name>**
|
|
60
|
+
|
|
61
|
+
### Signals Found
|
|
62
|
+
- [x] CI Configuration: <details>
|
|
63
|
+
- [ ] Custom Lint Rules: not found
|
|
64
|
+
- [x] CLAUDE.md: found, 5 enforced / 2 guidance / 1 missing
|
|
65
|
+
...
|
|
66
|
+
|
|
67
|
+
### Recommendations
|
|
68
|
+
1. <Most impactful next step to level up>
|
|
69
|
+
2. <Second recommendation>
|
|
70
|
+
3. <Third recommendation>
|
|
71
|
+
|
|
72
|
+
### How to Level Up
|
|
73
|
+
<Specific, actionable advice for reaching the next maturity level>
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
Be specific about file paths and what you found. Give actionable recommendations tailored to the project's language and toolchain.
|