@holmes-lab/holmes-kit 0.3.2 → 0.3.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +35 -0
- package/README.md +58 -6
- package/dist/.build-id +1 -1
- package/dist/holmes/review/excerpt-align.d.ts +14 -0
- package/dist/holmes/review/excerpt-align.js +51 -0
- package/dist/holmes/review/run-replay.d.ts +2 -0
- package/dist/holmes/review/run-replay.js +6 -1
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,41 @@ All notable changes to this project will be documented in this file.
|
|
|
4
4
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
|
+
<!-- @implements A-SPEC-209 -->
|
|
8
|
+
## [0.3.4] - 2026-09-01
|
|
9
|
+
|
|
10
|
+
Docs: a friendly, accurate setup guide for the opt-in semantic tiers.
|
|
11
|
+
|
|
12
|
+
### Added
|
|
13
|
+
|
|
14
|
+
- README Quickstart step 4: enabling the semantic layer — tier `local` (install
|
|
15
|
+
`@xenova/transformers`, no egress) and tier `cloud` (`gemini-embedding-001`): get a key from
|
|
16
|
+
Google AI Studio, store it out-of-tree via `holmes-kit semantic-key set` (stdin-only, 0600 /
|
|
17
|
+
macOS keychain, value never printed), env-var alternatives for CI
|
|
18
|
+
(`HOLMES_SEMANTIC_API_KEY` / `GEMINI_API_KEY` / `GOOGLE_API_KEY`), revocation via
|
|
19
|
+
`semantic-key unset`, and the security stance (no project-tree key source; agent
|
|
20
|
+
self-consent is gated).
|
|
21
|
+
|
|
22
|
+
### Internal (measurement plumbing, no default-path change)
|
|
23
|
+
|
|
24
|
+
- `alignExcerpt` request-aligned excerptor + `dumpBodies.mode` (A-SPEC-500.1) — feeds the
|
|
25
|
+
judgment-precision experiment REQ-500; default excerpts stay byte-identical.
|
|
26
|
+
|
|
27
|
+
<!-- @implements A-SPEC-209 -->
|
|
28
|
+
## [0.3.3] - 2026-08-31
|
|
29
|
+
|
|
30
|
+
Docs-only: the README the npm page shows catches up with 0.3.x.
|
|
31
|
+
|
|
32
|
+
### Changed
|
|
33
|
+
|
|
34
|
+
- README rewritten to 0.3.x: the 3-tier semantic layer (consent ladder + measured recalls), the
|
|
35
|
+
graded impact surface and its pre-registered baselines, the measured-not-claimed grading against
|
|
36
|
+
the modeled-human band (independent blind-judge P 0.727 = 81% of the band), approval-dialog
|
|
37
|
+
expiry visibility, the corrected Codex wiring path (`.codex/config.toml`), the Windows row
|
|
38
|
+
corrected from a CI claim to the 2026-08-31 field validation, a zero-install `npx` one-shot row,
|
|
39
|
+
and the stale-global-shim shadowing warning (field-measured dual global roots on Windows).
|
|
40
|
+
No code changes.
|
|
41
|
+
|
|
7
42
|
<!-- @implements A-SPEC-209 -->
|
|
8
43
|
## [0.3.2] - 2026-08-31
|
|
9
44
|
|
package/README.md
CHANGED
|
@@ -14,11 +14,15 @@
|
|
|
14
14
|
|
|
15
15
|
---
|
|
16
16
|
|
|
17
|
-
### 🛡️ Currently Supported Features (v0.
|
|
17
|
+
### 🛡️ Currently Supported Features (v0.3.x Production Features)
|
|
18
18
|
|
|
19
19
|
- 📋 **Requirements & Specification Governance**: Strict **"No Spec, No Code"** enforcement with 4-tier spec chain traceability (`REQ ➔ H-SPEC ➔ A-SPEC ➔ T-SPEC`) and `// @implements A-SPEC-XXX` line 1 code anchors.
|
|
20
|
+
- 🧠 **3-Tier Semantic Layer** *(new in 0.3.0)*: knowledge-graph semantic search with an explicit consent ladder — `none` (default, **zero egress**), `local` (bge-m3, no egress, optional module), `cloud` (gemini-embedding-001, opt-in via `GEMINI_API_KEY`). Measured on 305 traceability cases: recall 0.486 (lexical) → 0.667 (local) → **0.887 (cloud)**; on lexical-zero requests: 0% → 52% → **92%**. Surfaced only additively — rerank, evidence (`semCos`), and `semanticAlternates` — never as a hard filter.
|
|
21
|
+
- 🎯 **Graded Impact Surface** *(new in 0.3.0)*: `rankedImpact` (personalized-PageRank over the spec/code graph) beat its pre-registered naive baseline on **both recall and precision across 3 corpora (×1.6–×17)** — the necessary condition for any better-than-a-person phrasing, measured before claimed.
|
|
20
22
|
- 🐞 **Causal Defect Localization & CPG**: AST Code Property Graph (CPG) & Dataflow Taint reachability analysis across 7 languages (TS/JS, Python, Go, Rust, Java, C/C++, C#).
|
|
21
|
-
-
|
|
23
|
+
- 📏 **Measured, Not Claimed** *(new in 0.3.x)*: performance is judged against a pre-registered modeled-human band (R 0.67–0.78 / P ≈0.9±). Current official grade: **band entry on recall; division-of-labor precision 0.727 = 81% of the modeled human — reproduced by an independent context-free judge on a fresh blind window.** No superhuman claims until both metrics exceed the band.
|
|
24
|
+
- 🧪 **Self-Healing & Diagnostic Doctor**: Automated integrity checks and self-healing auto-fix remediation (`holmes-kit doctor --fix` & `spec_remediate`) — wiring-handshake checks run on Windows natively as of 0.3.2.
|
|
25
|
+
- 🔔 **Approval UX** *(new in 0.3.1)*: in-session approval dialogs forewarn their 120s deadline and, on expiry, the refusal says exactly where the decision went (`npx holmes-kit approve` out-of-band queue) — no more silently dead dialogs.
|
|
22
26
|
- 🚦 **CI/CD Governance Gate Runner**: Non-interactive headless CI/CD build gate (`holmes-kit ci`) for GitHub Actions and GitLab CI pipelines.
|
|
23
27
|
- 📊 **Automated RTM & Taint Heatmap**: Interactive standalone HTML/SVG report generation (`generateRtmHeatmap`) for spec coverage and security dataflow reachability.
|
|
24
28
|
- 🤖 **CLI-First AI Harness Matrix**: Native process hook gating for Claude Code, Antigravity CLI (AGY), Codex CLI, and Google Antigravity SDK.
|
|
@@ -42,7 +46,7 @@ Holmes-Kit prioritizes **CLI-based AI Coding Agents** where OS-level process hoo
|
|
|
42
46
|
| :--- | :--- | :--- |
|
|
43
47
|
| 🤖 **Claude Code CLI** | 🥇 Tier 1 (Native) | OS PreToolUse & Stop hooks (`.claude/settings.local.json`), MCP server (`.mcp.json`) |
|
|
44
48
|
| 🚀 **Antigravity CLI (AGY)** | 🥇 Tier 1 (Native) | AGY Hooks (`hooks.json`), MCP config (`.agents/mcp_config.json`), Governance Skills |
|
|
45
|
-
| 💻 **Codex CLI / Agentic Shell** | 🥇 Tier 1 (Native) | Codex MCP integration (`.codex/
|
|
49
|
+
| 💻 **Codex CLI / Agentic Shell** | 🥇 Tier 1 (Native) | Codex MCP integration (`.codex/config.toml`), plugin-packaged gate hooks (installed via Codex plugin marketplace) |
|
|
46
50
|
| 🧩 **Google Antigravity SDK** | 🥇 Tier 1 (Native) | Autonomous Agent SDK bindings and cryptographic provenance verification |
|
|
47
51
|
|
|
48
52
|
> **Note**: Holmes-Kit focuses strictly on CLI-based autonomous agents to guarantee 100% deterministic OS hook gating (`deny` enforcement) before file modifications occur.
|
|
@@ -69,10 +73,18 @@ Add `-DryRun` to see what it would do without installing. Exit codes: `0` ok / a
|
|
|
69
73
|
| Which situation are you in? | Privileges | Command |
|
|
70
74
|
|---|---|---|
|
|
71
75
|
| **Using it in one project** (most people) | none | `npm install --save-dev @holmes-lab/holmes-kit` |
|
|
76
|
+
| **Zero-install one-shot** (try it first) | none | `npx -y @holmes-lab/holmes-kit init` — npx fetches and runs, nothing to install beforehand |
|
|
72
77
|
| Company-managed PC / restricted account | none | same — no system directory is touched |
|
|
73
78
|
| CI / container | none | same, plus `--prefer-online` right after a release |
|
|
74
79
|
| CLI across many projects (`-g`) | depends | run `npm config get prefix` first — see below |
|
|
75
80
|
|
|
81
|
+
**Stale global shadowing** *(field-measured on Windows, 2026-08-31)*: an old global install makes the
|
|
82
|
+
bare `holmes-kit` command run the OLD version while `npx holmes-kit` runs the local one — and
|
|
83
|
+
Windows has TWO global roots (`C:\Program Files\nodejs` and `%APPDATA%\npm`), so `npm uninstall -g`
|
|
84
|
+
against one root can leave a live shim in the other. If `holmes-kit --version` and
|
|
85
|
+
`npx holmes-kit --version` disagree, run `where.exe holmes-kit` (Windows) / `which -a holmes-kit`
|
|
86
|
+
and remove the stale shim; prefer the `npx` form day-to-day.
|
|
87
|
+
|
|
76
88
|
**Before `npm install -g`**: if `npm config get prefix` names a protected directory
|
|
77
89
|
(`C:\Program Files\nodejs`, `/usr/local`), `-g` dies with `EPERM` **before any package file
|
|
78
90
|
arrives** — no package version can fix that, and elevation is the wrong fix (it runs native
|
|
@@ -96,7 +108,7 @@ npx holmes-kit init # drop the npx prefix if you installed with -g
|
|
|
96
108
|
? Select the AI Agent harnesses to wire into this project:
|
|
97
109
|
[X] 🤖 Claude Code (.claude/settings.local.json, .mcp.json)
|
|
98
110
|
[X] 🚀 Antigravity CLI (AGY) (.agents/mcp_config.json, hooks.json, skills)
|
|
99
|
-
[ ] 💻 Codex CLI (.codex/
|
|
111
|
+
[ ] 💻 Codex CLI (.codex/config.toml)
|
|
100
112
|
```
|
|
101
113
|
|
|
102
114
|
### 3. Verify Health
|
|
@@ -105,6 +117,46 @@ npx holmes-kit doctor
|
|
|
105
117
|
```
|
|
106
118
|
*If everything is green, your project is governed and ready for AI pair-programming!*
|
|
107
119
|
|
|
120
|
+
### 4. Optional: Enable the Semantic Layer (Tier `local` / Tier `cloud`)
|
|
121
|
+
|
|
122
|
+
Out of the box Holmes-Kit runs tier **`none`** — lexical + citation + graph search, **zero
|
|
123
|
+
egress**. Two opt-in tiers raise recall on requests your vocabulary can't reach (measured on 305
|
|
124
|
+
traceability cases — see the feature list above):
|
|
125
|
+
|
|
126
|
+
**Tier `local` — no egress, no account.** Install the optional embedding runtime next to
|
|
127
|
+
holmes-kit and the local model (`bge-m3`) is picked up automatically:
|
|
128
|
+
```bash
|
|
129
|
+
npm install --save-dev @xenova/transformers
|
|
130
|
+
npx holmes-kit doctor # → semantic tier: local (no egress)
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
**Tier `cloud` — highest recall, explicit consent (`gemini-embedding-001`).** Setting a key IS
|
|
134
|
+
the consent act: with a key present, repository-derived text is sent to Google's embedding API.
|
|
135
|
+
|
|
136
|
+
1. Get a Gemini API key (Google AI Studio → <https://aistudio.google.com/apikey>; the free tier
|
|
137
|
+
is enough to try it).
|
|
138
|
+
2. Store it **outside your project tree** with the built-in command — the key rides **stdin,
|
|
139
|
+
never argv**, lands in `~/.holmes/credentials.json` with `0600` permissions, and no output
|
|
140
|
+
ever contains the value:
|
|
141
|
+
```bash
|
|
142
|
+
npx holmes-kit semantic-key set # hidden prompt on a TTY; or: echo "$KEY" | npx holmes-kit semantic-key set
|
|
143
|
+
npx holmes-kit semantic-key status # shows the key's SOURCE only, never the value
|
|
144
|
+
npx holmes-kit doctor # → semantic tier: cloud (egress: on)
|
|
145
|
+
```
|
|
146
|
+
On macOS the key prefers the system keychain; elsewhere the `0600` file is the store.
|
|
147
|
+
Environment variables also work and take precedence (`HOLMES_SEMANTIC_API_KEY` dedicated, or
|
|
148
|
+
the ecosystem-compatible `GEMINI_API_KEY` / `GOOGLE_API_KEY`) — useful for CI. Prefer
|
|
149
|
+
`semantic-key set` on workstations: it keeps the key out of shell history, `.env` files, and
|
|
150
|
+
the repository.
|
|
151
|
+
3. To revoke consent at any time:
|
|
152
|
+
```bash
|
|
153
|
+
npx holmes-kit semantic-key unset # clears the stored key; tier falls back to local/none
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
> 🔒 **Never** commit a key, pass it as a CLI argument, or put it in a file inside the project
|
|
157
|
+
> tree. Holmes-Kit's credential chain has **no project-tree source by design**, and agents are
|
|
158
|
+
> gated from setting keys on their own — consent stays a human act.
|
|
159
|
+
|
|
108
160
|
---
|
|
109
161
|
|
|
110
162
|
## 🔄 Daily Workflow (How It Works)
|
|
@@ -181,11 +233,11 @@ Holmes-Kit embeds native AST & Code Property Graph (D-CPG) analyzers to track ca
|
|
|
181
233
|
| **macOS** | Apple Silicon (arm64) / Intel (x64) | ✅ Tier 1 | macOS 12+ (Full hook enforcement) |
|
|
182
234
|
| **Linux** | x86_64 / arm64 | ✅ Tier 1 | Ubuntu, Debian, Fedora, Arch, RHEL |
|
|
183
235
|
| **Windows (WSL2)** | x86_64 | ✅ Tier 1 | WSL2 Ubuntu/Debian recommended |
|
|
184
|
-
| **Windows Native** | x86_64 | ✅ Tier 1 | Windows 10/11 (Node.js 20
|
|
236
|
+
| **Windows Native** | x86_64 | ✅ Tier 1 | Windows 10/11 (Node.js 20+; prebuilt natives, no build tools needed in the common case). **Field-validated 2026-08-31** on a real user machine: registry install, natives (better-sqlite3 + 7 tree-sitter grammars), both OS gates, MCP handshake (30 tools), interactive init TUI, out-of-band approval channel (doctor 25 PASS; the 4 false FAILs it also showed were doctor's own win32 spawn bug, fixed in 0.3.2). See ADR-015 (platform tier is decided by executed verification — internal decision record) for tier criteria and residual risks (NTFS 8.3 names, reserved device names, 260-char paths; no Windows CI yet) |
|
|
185
237
|
|
|
186
238
|
> **Runtime Requirement**: Node.js `>= 20.0.0` (LTS recommended)
|
|
187
239
|
>
|
|
188
|
-
> **How a tier is decided**: by the verification that actually executes, not by declaration. A platform is Tier 1 only while its gate verdicts are exercised by the suite; if that stops being true it is demoted and the demotion is recorded. See
|
|
240
|
+
> **How a tier is decided**: by the verification that actually executes, not by declaration. A platform is Tier 1 only while its gate verdicts are exercised by the suite; if that stops being true it is demoted and the demotion is recorded. See ADR-015 (platform tier is decided by executed verification — internal decision record).
|
|
189
241
|
|
|
190
242
|
---
|
|
191
243
|
|
package/dist/.build-id
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
|
|
1
|
+
68b5982-mthe8y11
|
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Request-aligned excerpting for the judgment-measurement dump. The excerpt ladder
|
|
3
|
+
* (S-505→S-516) showed excerpt information content DOMINATES the agent-judge's precision, and
|
|
4
|
+
* the shipped head-slice starves the judge exactly when the relevant region sits mid-file —
|
|
5
|
+
* imports and module prose fill the window and the judge degrades to filename guessing.
|
|
6
|
+
*
|
|
7
|
+
* The alignment is deliberately dumb and PURE: slide a chars-wide window at half-stride and keep
|
|
8
|
+
* the one containing the most DISTINCT request tokens (presence, not frequency — one dense
|
|
9
|
+
* mention must not outvote three different matched concepts). Ties keep the EARLIEST window and
|
|
10
|
+
* zero overlap falls back to the head slice, so where there is no signal the behavior is
|
|
11
|
+
* byte-identical to the status quo. Every return is a verbatim substring — the judge must only
|
|
12
|
+
* ever see real code.
|
|
13
|
+
*/
|
|
14
|
+
export declare function alignExcerpt(text: string, request: string, chars: number): string;
|
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
// @implements A-SPEC-500.1
|
|
3
|
+
/**
|
|
4
|
+
* Request-aligned excerpting for the judgment-measurement dump. The excerpt ladder
|
|
5
|
+
* (S-505→S-516) showed excerpt information content DOMINATES the agent-judge's precision, and
|
|
6
|
+
* the shipped head-slice starves the judge exactly when the relevant region sits mid-file —
|
|
7
|
+
* imports and module prose fill the window and the judge degrades to filename guessing.
|
|
8
|
+
*
|
|
9
|
+
* The alignment is deliberately dumb and PURE: slide a chars-wide window at half-stride and keep
|
|
10
|
+
* the one containing the most DISTINCT request tokens (presence, not frequency — one dense
|
|
11
|
+
* mention must not outvote three different matched concepts). Ties keep the EARLIEST window and
|
|
12
|
+
* zero overlap falls back to the head slice, so where there is no signal the behavior is
|
|
13
|
+
* byte-identical to the status quo. Every return is a verbatim substring — the judge must only
|
|
14
|
+
* ever see real code.
|
|
15
|
+
*/
|
|
16
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
17
|
+
exports.alignExcerpt = alignExcerpt;
|
|
18
|
+
const tokensOf = (request) => {
|
|
19
|
+
const seen = new Set();
|
|
20
|
+
for (const m of request.toLowerCase().match(/[a-z0-9_$가-힣]+/g) ?? []) {
|
|
21
|
+
if (m.length >= 3)
|
|
22
|
+
seen.add(m);
|
|
23
|
+
}
|
|
24
|
+
return [...seen];
|
|
25
|
+
};
|
|
26
|
+
function alignExcerpt(text, request, chars) {
|
|
27
|
+
if (text.length <= chars)
|
|
28
|
+
return text;
|
|
29
|
+
const tokens = tokensOf(request);
|
|
30
|
+
const head = text.slice(0, chars);
|
|
31
|
+
if (tokens.length === 0)
|
|
32
|
+
return head;
|
|
33
|
+
const stride = Math.ceil(chars / 2);
|
|
34
|
+
let best = head;
|
|
35
|
+
let bestCount = 0;
|
|
36
|
+
for (let at = 0; at < text.length; at += stride) {
|
|
37
|
+
const window = text.slice(at, at + chars);
|
|
38
|
+
const lower = window.toLowerCase();
|
|
39
|
+
let count = 0;
|
|
40
|
+
for (const t of tokens)
|
|
41
|
+
if (lower.includes(t))
|
|
42
|
+
count++;
|
|
43
|
+
if (count > bestCount) {
|
|
44
|
+
bestCount = count;
|
|
45
|
+
best = window;
|
|
46
|
+
}
|
|
47
|
+
if (at + chars >= text.length)
|
|
48
|
+
break;
|
|
49
|
+
}
|
|
50
|
+
return bestCount === 0 ? head : best;
|
|
51
|
+
}
|
|
@@ -241,6 +241,8 @@ export declare function runReplay(corpus: ReplayCorpus, limit: number, opts?: {
|
|
|
241
241
|
chars: number;
|
|
242
242
|
/** @implements A-SPEC-490 — drop leading license blocks before excerpting; default false keeps S-507 bytes. */
|
|
243
243
|
skipLicense?: boolean;
|
|
244
|
+
/** @implements A-SPEC-500.1 — 'aligned' excerpts the request-densest window instead of the head; default 'head' keeps prior bytes. */
|
|
245
|
+
mode?: 'head' | 'aligned';
|
|
244
246
|
};
|
|
245
247
|
}): Promise<ReplayResult>;
|
|
246
248
|
/**
|
|
@@ -64,6 +64,7 @@ const localize_1 = require("../rtm/localize");
|
|
|
64
64
|
// @implements A-SPEC-483 — content-level verification: the candidate's BODY, read from the
|
|
65
65
|
// materialized parent tree.
|
|
66
66
|
const content_verify_1 = require("./content-verify");
|
|
67
|
+
const excerpt_align_1 = require("./excerpt-align");
|
|
67
68
|
// @implements A-SPEC-467 — the associative-recall arm: measured beside the product, never inside it.
|
|
68
69
|
const assoc_arm_1 = require("../assoc/assoc-arm");
|
|
69
70
|
// @implements A-SPEC-470 — the impact axis's Phase-B external yardstick, always scored.
|
|
@@ -265,7 +266,11 @@ async function runReplay(corpus, limit, opts = {}) {
|
|
|
265
266
|
const raw = fs.readFileSync(path.join(dest, f), 'utf8');
|
|
266
267
|
// @implements A-SPEC-490 — the informative excerpt: license blocks first, then cut.
|
|
267
268
|
const text = opts.dumpBodies.skipLicense === true ? (0, content_verify_1.skipLicenseHeader)(raw) : raw;
|
|
268
|
-
|
|
269
|
+
// @implements A-SPEC-500.1 — 'aligned' picks the request-densest window; the
|
|
270
|
+
// default stays the head slice, byte-identical to the pre-mode path.
|
|
271
|
+
bodies[f] = opts.dumpBodies.mode === 'aligned'
|
|
272
|
+
? (0, excerpt_align_1.alignExcerpt)(text, c.subject, opts.dumpBodies.chars)
|
|
273
|
+
: text.slice(0, opts.dumpBodies.chars);
|
|
269
274
|
}
|
|
270
275
|
catch { /* unreadable candidate: omitted */ }
|
|
271
276
|
}
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"//": "@implements A-SPEC-209",
|
|
3
3
|
"name": "@holmes-lab/holmes-kit",
|
|
4
|
-
"version": "0.3.
|
|
4
|
+
"version": "0.3.4",
|
|
5
5
|
"description": "Holmes-Kit — deterministic Agentic Software Engineering (ASE) harness with causal traceability (spec chain + D-CPG + RTM + phase guardrail)",
|
|
6
6
|
"main": "dist/holmes/mcp/server.js",
|
|
7
7
|
"types": "dist/holmes/mcp/server.d.ts",
|