claude-spotter 1.5.2 → 1.5.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +23 -0
- package/README.ja.md +11 -22
- package/README.md +17 -29
- package/bin/spotter.mjs +2 -2
- package/package.json +1 -1
- package/src/cli/codex-hook-cmd.mjs +2 -118
- package/src/cli/doctor.mjs +1 -1
- package/src/cli/factory-diagnostics.mjs +1 -1
- package/src/cli/install.mjs +3 -3
- package/src/daemon/daemon.mjs +1 -24
- package/src/dashboard/render.mjs +13 -5
- package/src/hooks/user-prompt.mjs +2 -112
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,28 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 1.5.4 — 2026-08-05
|
|
4
|
+
|
|
5
|
+
- **Throughlineを提案監査の実行条件から撤去。** Claude / CodexのUserPromptSubmitは、
|
|
6
|
+
`auditorContext`のmode、Throughlineのfreshness、取得成否に関係なく、現在のuser promptと
|
|
7
|
+
host-local tool catalogで監査する。daemonも旧`audit:false`やcontext payloadで監査を止めず、
|
|
8
|
+
Throughline本文を監査AIへ渡さない。
|
|
9
|
+
- **評価文脈だけを独立取得。** 提案が出た時のThroughline `observer-read`は改善用証拠として
|
|
10
|
+
一度だけ取得し、失敗時は評価文脈だけをunavailableにする。監査結果、親へのtool提案、
|
|
11
|
+
成功Hook eventには影響させず、retryやbackground回収も追加しない。
|
|
12
|
+
- **停止を正常passへ偽装しない。** 旧版で`context_disabled` / `context_not_fresh` /
|
|
13
|
+
`auditor_context`となっていた経路を削除し、以後の監査成功turnは提案なしの場合も評価母数へ入る。
|
|
14
|
+
install / doctor / READMEもThroughlineを`evaluation context`として表示する。
|
|
15
|
+
- **検証。** disabled / stale / provider error / legacy payload / observer-read failureを含む
|
|
16
|
+
Claude・Codex回帰テストとfull suite 588件(586 pass / 2 platform skip)を通過した。
|
|
17
|
+
|
|
18
|
+
## 1.5.3 — 2026-08-04
|
|
19
|
+
|
|
20
|
+
- **dashboardの難解な集計略号を廃止。** 概要cardとproject/tool別内訳の
|
|
21
|
+
`S/P/I/C/A/M`を、対象ターン、ツール提案あり、提案ツール数、利用判定済み、
|
|
22
|
+
実際に使用、判定不能の日本語表示へ置き換える。集計schemaとCLIの内部キーは変更しない。
|
|
23
|
+
- **率の意味を式ではなく言葉で表示。** 提案率は「ツール提案あり ÷ 対象ターン」、
|
|
24
|
+
採用率は「実際に使用 ÷ 利用判定済み」と併記する。
|
|
25
|
+
|
|
3
26
|
## 1.5.2 — 2026-08-04
|
|
4
27
|
|
|
5
28
|
- **Mac LaunchAgentでNodeを解決。** `spotter`の絶対pathだけでなく、env shebangが使う
|
package/README.ja.md
CHANGED
|
@@ -156,11 +156,16 @@ flowchart LR
|
|
|
156
156
|
|
|
157
157
|
両者に共通するのは **「主体に頼らない仕組み」**。併用できます。
|
|
158
158
|
|
|
159
|
-
### Throughline
|
|
159
|
+
### Throughlineの提案時評価文脈(任意)
|
|
160
160
|
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
`
|
|
161
|
+
Spotterの提案AIはThroughlineを使わず、Throughlineの導入・設定・freshness・取得成否に関係なく
|
|
162
|
+
UserPromptSubmitごとに監査します。Throughlineは、提案が出た時に改善分析用の別文脈を
|
|
163
|
+
`observer-read`で一度だけ記録する任意経路です。取得できなくても`context_unavailable`として記録するだけで、
|
|
164
|
+
監査や親への助言には影響しません。
|
|
165
|
+
|
|
166
|
+
`spotter install`がPATH上のThroughlineを絶対パスへ解決できる場合、この評価証拠の取得経路を既定で設定します。
|
|
167
|
+
既存互換の`--auditor-context`名はmarker設定に残っていますが、監査のON/OFFは制御しません。
|
|
168
|
+
評価文脈の取得だけを無効化するには次を実行します。
|
|
164
169
|
|
|
165
170
|
```bash
|
|
166
171
|
spotter install -y --auditor-context disabled
|
|
@@ -183,24 +188,8 @@ spotter install -y --auditor-context throughline `
|
|
|
183
188
|
--throughline-arg 'C:\absolute\path\to\throughline\bin\throughline.mjs'
|
|
184
189
|
```
|
|
185
190
|
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
AI 呼出し候補になり、それ以外の status では AI を呼びません。enabled connector の障害は hidden fallback
|
|
189
|
-
ではなく固定 warning として出します。
|
|
190
|
-
|
|
191
|
-
Throughline から渡すのは fresh な完了済み L2 user/assistant pair だけです。直近 2 pair (N=2)、各 body は
|
|
192
|
-
600 文字、合計は 4,000 文字に制限します。Spotter は Throughline の L2、`reason`、`raw` を親へ反射せず、
|
|
193
|
-
親には安全な catalog tool ID から作る固定・非命令形の助言だけを渡します。`spotter doctor` は command / args
|
|
194
|
-
を表示せず、auditor-context mode と固定の availability detail だけを表示します。
|
|
195
|
-
|
|
196
|
-
v2 model-matrix では context choice を明示できます。
|
|
197
|
-
|
|
198
|
-
```bash
|
|
199
|
-
spotter auditor model-matrix --fixtures test/fixtures/auditor-model-matrix.v2.json \
|
|
200
|
-
--recent-turns 2 --body-cap 600
|
|
201
|
-
```
|
|
202
|
-
|
|
203
|
-
評価結論はN=2 / 600です。既定ONは確定済みで、7日・30 fresh resultの実運用測定は精度改善に使います。
|
|
191
|
+
`spotter doctor`はこの経路を`evaluation context`として表示し、command / argsや会話本文は表示しません。
|
|
192
|
+
observer snapshotは端末内の評価SQLiteにだけ保存され、network送信、retry、background回収は行いません。
|
|
204
193
|
|
|
205
194
|
## よく使うコマンド
|
|
206
195
|
|
package/README.md
CHANGED
|
@@ -156,12 +156,18 @@ The audited catalog is host-local: Claude uses `<project>/.spotter/tool-db.json`
|
|
|
156
156
|
|
|
157
157
|
Both share the principle of **"don't rely on the primary agent to do it itself."** They compose well — you can run them together.
|
|
158
158
|
|
|
159
|
-
### Throughline
|
|
159
|
+
### Optional Throughline evidence for proposal evaluation
|
|
160
160
|
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
161
|
+
Spotter's proposal auditor does not use Throughline. Every UserPromptSubmit
|
|
162
|
+
audit runs independently of Throughline installation, configuration, freshness, or read
|
|
163
|
+
failures. When Spotter emits a proposal, the evaluation recorder may call
|
|
164
|
+
Throughline `observer-read` once to save separate improvement evidence. A failed
|
|
165
|
+
read is recorded as `context_unavailable` and never changes auditing or parent advice.
|
|
166
|
+
|
|
167
|
+
When `spotter install` resolves Throughline on PATH to an absolute executable, it
|
|
168
|
+
configures this evaluation-evidence path by default. The legacy option name
|
|
169
|
+
`--auditor-context` remains for marker compatibility but no longer controls whether
|
|
170
|
+
auditing runs. Disable only the evidence capture with:
|
|
165
171
|
|
|
166
172
|
```bash
|
|
167
173
|
spotter install -y --auditor-context disabled
|
|
@@ -185,28 +191,9 @@ spotter install -y --auditor-context throughline `
|
|
|
185
191
|
--throughline-arg 'C:\absolute\path\to\throughline\bin\throughline.mjs'
|
|
186
192
|
```
|
|
187
193
|
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
eligible for an AI call; every other status skips AI. An enabled connector
|
|
192
|
-
failure becomes a fixed warning, not a hidden fallback.
|
|
193
|
-
|
|
194
|
-
Throughline contributes only fresh, completed L2 user/assistant pairs: two
|
|
195
|
-
recent pairs (N=2), each body capped at 600 characters and 4,000 characters in
|
|
196
|
-
total. Spotter never reflects Throughline L2, `reason`, or `raw` to the parent.
|
|
197
|
-
The parent receives only fixed non-imperative advice built from safe catalog tool
|
|
198
|
-
IDs. `spotter doctor` displays the auditor-context mode and a fixed availability
|
|
199
|
-
detail without printing its command or arguments.
|
|
200
|
-
|
|
201
|
-
The v2 model-matrix can make the context choice explicit:
|
|
202
|
-
|
|
203
|
-
```bash
|
|
204
|
-
spotter auditor model-matrix --fixtures test/fixtures/auditor-model-matrix.v2.json \
|
|
205
|
-
--recent-turns 2 --body-cap 600
|
|
206
|
-
```
|
|
207
|
-
|
|
208
|
-
The evaluated setting is N=2 / 600. Default-on is final; the 7-day,
|
|
209
|
-
30-fresh-result sample is used only to identify precision improvements.
|
|
194
|
+
`spotter doctor` reports this as `evaluation context` without printing commands,
|
|
195
|
+
arguments, or conversation text. Observer snapshots stay in the terminal-local
|
|
196
|
+
evaluation SQLite. Spotter adds no network upload, retry, or background recovery.
|
|
210
197
|
|
|
211
198
|
## Common commands
|
|
212
199
|
|
|
@@ -256,8 +243,9 @@ spotter uninstall # remove hooks from this project (leaves ~/.spotter int
|
|
|
256
243
|
|
|
257
244
|
The dashboard is local-first. Every terminal reads its own `~/.spotter/evaluation.db`; the hub
|
|
258
245
|
keeps only a static device-to-upstream map and does not copy evaluation data into a cloud database.
|
|
259
|
-
The device view shows
|
|
260
|
-
non-adopted cases, and the two
|
|
246
|
+
The device view shows Japanese labels for every evaluation metric, proposal and adoption rates
|
|
247
|
+
with their numerator and denominator, project/tool breakdowns, non-adopted cases, and the two
|
|
248
|
+
separately captured context sources. The hub checks health only
|
|
261
249
|
when the device list is requested, so an offline terminal is isolated without a background monitor
|
|
262
250
|
or retry queue.
|
|
263
251
|
|
package/bin/spotter.mjs
CHANGED
|
@@ -28,8 +28,8 @@ Usage:
|
|
|
28
28
|
[--throughline-command ABS] [--throughline-arg VALUE]
|
|
29
29
|
register hooks in <cwd>/.claude/settings.json
|
|
30
30
|
and create <cwd>/.spotter/marker.json
|
|
31
|
-
(Throughline on PATH enables
|
|
32
|
-
|
|
31
|
+
(Throughline on PATH enables proposal-time
|
|
32
|
+
evaluation evidence by default; it never gates auditing)
|
|
33
33
|
(run inside each project you want audited)
|
|
34
34
|
spotter install --user [-y] legacy: register globally in ~/.claude/settings.json
|
|
35
35
|
(NOT RECOMMENDED — fires for every Claude Code session
|
package/package.json
CHANGED
|
@@ -30,10 +30,6 @@ import {
|
|
|
30
30
|
hookEventsPath,
|
|
31
31
|
summarizeHookEvents,
|
|
32
32
|
} from '../core/hook-event-log.mjs';
|
|
33
|
-
import {
|
|
34
|
-
loadAuditorContext,
|
|
35
|
-
readProjectAuditorContextConfig,
|
|
36
|
-
} from '../core/auditor-context.mjs';
|
|
37
33
|
import { observeRuntimeErrorIsolatedSafe } from '../core/runtime-error-store.mjs';
|
|
38
34
|
import { createEvaluationStore } from '../core/evaluation-store.mjs';
|
|
39
35
|
import { loadEvaluationObserverContext } from '../core/evaluation-context.mjs';
|
|
@@ -130,8 +126,6 @@ export async function runCodexUserPromptSubmitHook({
|
|
|
130
126
|
createAuditorBackendFn = createAuditorBackend,
|
|
131
127
|
recordHookEventFn = appendCodexHookEvent,
|
|
132
128
|
discardLegacyPendingFn = discardLegacyPending,
|
|
133
|
-
readAuditorContextConfigFn = readProjectAuditorContextConfig,
|
|
134
|
-
loadAuditorContextFn = loadAuditorContext,
|
|
135
129
|
writeOutput = (text) => process.stdout.write(text),
|
|
136
130
|
writeError = (text) => process.stderr.write(text),
|
|
137
131
|
runtimeErrorObserver = async () => ({ collected: false, reason: 'observer_not_configured' }),
|
|
@@ -147,12 +141,10 @@ export async function runCodexUserPromptSubmitHook({
|
|
|
147
141
|
const projectRoot = findSpotterMarker(input.cwd);
|
|
148
142
|
if (!projectRoot) return;
|
|
149
143
|
const startedAt = Date.now();
|
|
150
|
-
let contextDurationMs = null;
|
|
151
|
-
let auditorContextConfig;
|
|
152
144
|
|
|
153
145
|
const prompt = requireString(input, 'prompt');
|
|
154
146
|
const sessionId = codexSessionId(input);
|
|
155
|
-
const recordEvaluation = async ({ auditStatus, proposedToolIds = [], backend = null, model = null
|
|
147
|
+
const recordEvaluation = async ({ auditStatus, proposedToolIds = [], backend = null, model = null }) => {
|
|
156
148
|
if (!sessionId) {
|
|
157
149
|
reportEvaluationFailure(writeError);
|
|
158
150
|
return;
|
|
@@ -167,7 +159,6 @@ export async function runCodexUserPromptSubmitHook({
|
|
|
167
159
|
projectRoot,
|
|
168
160
|
host: 'codex',
|
|
169
161
|
sessionId,
|
|
170
|
-
config,
|
|
171
162
|
recordedAtMs: proposedAtMs,
|
|
172
163
|
});
|
|
173
164
|
}
|
|
@@ -182,7 +173,7 @@ export async function runCodexUserPromptSubmitHook({
|
|
|
182
173
|
sessionId,
|
|
183
174
|
auditStatus,
|
|
184
175
|
requestText: prompt,
|
|
185
|
-
auditorSeenContext,
|
|
176
|
+
auditorSeenContext: null,
|
|
186
177
|
observerContextStatus: observerContext.status,
|
|
187
178
|
observerSnapshot: observerContext.snapshot,
|
|
188
179
|
proposedToolIds: proposals.resolvedToolIds,
|
|
@@ -198,97 +189,6 @@ export async function runCodexUserPromptSubmitHook({
|
|
|
198
189
|
}
|
|
199
190
|
};
|
|
200
191
|
const legacyPending = await discardLegacyPendingFn({ projectRoot, sessionId: codexSessionId(input) });
|
|
201
|
-
let context;
|
|
202
|
-
const contextStartedAt = Date.now();
|
|
203
|
-
try {
|
|
204
|
-
const config = await readAuditorContextConfigFn(projectRoot);
|
|
205
|
-
auditorContextConfig = config;
|
|
206
|
-
contextDurationMs = Date.now() - contextStartedAt;
|
|
207
|
-
if (config.mode === 'disabled') {
|
|
208
|
-
await recordEvaluation({ auditStatus: 'skipped' });
|
|
209
|
-
await recordCodexHookEventSafe(recordHookEventFn, {
|
|
210
|
-
projectRoot,
|
|
211
|
-
event: {
|
|
212
|
-
hook: 'UserPromptSubmit',
|
|
213
|
-
status: 'skipped',
|
|
214
|
-
reason: 'context_disabled',
|
|
215
|
-
contextStatus: 'disabled',
|
|
216
|
-
contextDurationMs,
|
|
217
|
-
legacyPendingDiagnostic: legacyPending.diagnostic,
|
|
218
|
-
durationMs: Date.now() - startedAt,
|
|
219
|
-
},
|
|
220
|
-
}, writeError);
|
|
221
|
-
return;
|
|
222
|
-
}
|
|
223
|
-
context = await loadAuditorContextFn({
|
|
224
|
-
config,
|
|
225
|
-
host: 'codex',
|
|
226
|
-
sessionId: requireCodexSessionId(input),
|
|
227
|
-
projectRoot,
|
|
228
|
-
transcriptPath: requireString(input, 'transcript_path'),
|
|
229
|
-
});
|
|
230
|
-
contextDurationMs = Date.now() - contextStartedAt;
|
|
231
|
-
} catch (err) {
|
|
232
|
-
await recordEvaluation({ auditStatus: 'error' });
|
|
233
|
-
await observeRuntimeFailure(runtimeErrorObserver, 'auditor_unavailable');
|
|
234
|
-
contextDurationMs = Date.now() - contextStartedAt;
|
|
235
|
-
const failure = projectBackendFailure(err?.code);
|
|
236
|
-
safeWriteError(writeError, failure.stderr);
|
|
237
|
-
await recordCodexHookEventSafe(recordHookEventFn, {
|
|
238
|
-
projectRoot,
|
|
239
|
-
event: {
|
|
240
|
-
hook: 'UserPromptSubmit',
|
|
241
|
-
status: 'error',
|
|
242
|
-
code: failure.code,
|
|
243
|
-
reason: 'auditor_context',
|
|
244
|
-
contextDurationMs,
|
|
245
|
-
legacyPendingDiagnostic: legacyPending.diagnostic,
|
|
246
|
-
durationMs: Date.now() - startedAt,
|
|
247
|
-
},
|
|
248
|
-
}, writeError);
|
|
249
|
-
writeCodexSystemMessage({ systemMessage: failure.systemMessage, writeOutput });
|
|
250
|
-
return;
|
|
251
|
-
}
|
|
252
|
-
if (context.status === 'unavailable' || context.status === 'schema_mismatch') {
|
|
253
|
-
await recordEvaluation({ auditStatus: 'error' });
|
|
254
|
-
await observeRuntimeFailure(runtimeErrorObserver, 'auditor_unavailable');
|
|
255
|
-
const failure = projectBackendFailure(context.status === 'unavailable'
|
|
256
|
-
? 'E_AUDITOR_CONTEXT_UNAVAILABLE'
|
|
257
|
-
: 'E_AUDITOR_CONTEXT_SCHEMA');
|
|
258
|
-
safeWriteError(writeError, failure.stderr);
|
|
259
|
-
await recordCodexHookEventSafe(recordHookEventFn, {
|
|
260
|
-
projectRoot,
|
|
261
|
-
event: {
|
|
262
|
-
hook: 'UserPromptSubmit',
|
|
263
|
-
status: 'error',
|
|
264
|
-
code: failure.code,
|
|
265
|
-
reason: 'auditor_context_status',
|
|
266
|
-
contextDurationMs,
|
|
267
|
-
legacyPendingDiagnostic: legacyPending.diagnostic,
|
|
268
|
-
durationMs: Date.now() - startedAt,
|
|
269
|
-
},
|
|
270
|
-
}, writeError);
|
|
271
|
-
writeCodexSystemMessage({ systemMessage: failure.systemMessage, writeOutput });
|
|
272
|
-
return;
|
|
273
|
-
}
|
|
274
|
-
if (context.status !== 'fresh') {
|
|
275
|
-
await recordEvaluation({ auditStatus: 'skipped' });
|
|
276
|
-
await recordCodexHookEventSafe(recordHookEventFn, {
|
|
277
|
-
projectRoot,
|
|
278
|
-
event: {
|
|
279
|
-
hook: 'UserPromptSubmit',
|
|
280
|
-
status: 'skipped',
|
|
281
|
-
reason: 'context_not_fresh',
|
|
282
|
-
contextStatus: context.status,
|
|
283
|
-
contextTurns: 0,
|
|
284
|
-
contextChars: 0,
|
|
285
|
-
contextDurationMs,
|
|
286
|
-
legacyPendingDiagnostic: legacyPending.diagnostic,
|
|
287
|
-
durationMs: Date.now() - startedAt,
|
|
288
|
-
},
|
|
289
|
-
}, writeError);
|
|
290
|
-
return;
|
|
291
|
-
}
|
|
292
192
|
|
|
293
193
|
let catalog;
|
|
294
194
|
let backend;
|
|
@@ -301,8 +201,6 @@ export async function runCodexUserPromptSubmitHook({
|
|
|
301
201
|
judgment = await backend.judge({
|
|
302
202
|
stage: 'user_input',
|
|
303
203
|
userInput: prompt,
|
|
304
|
-
recentContext: context.turns,
|
|
305
|
-
contextStatus: 'fresh',
|
|
306
204
|
});
|
|
307
205
|
} catch (err) {
|
|
308
206
|
await recordEvaluation({ auditStatus: 'error', backend: err?.backend ?? null, model: err?.diagnostics?.modelSelection?.effectiveModel ?? null });
|
|
@@ -332,10 +230,6 @@ export async function runCodexUserPromptSubmitHook({
|
|
|
332
230
|
backend: judgment.meta?.backend ?? backend.name ?? 'unknown',
|
|
333
231
|
pass: judgment.pass,
|
|
334
232
|
missingTools: projectToolIds(judgment.findings.map((finding) => finding.toolName)),
|
|
335
|
-
contextStatus: 'fresh',
|
|
336
|
-
contextTurns: context.stats.returnedTurns,
|
|
337
|
-
contextChars: context.stats.chars,
|
|
338
|
-
contextDurationMs,
|
|
339
233
|
...compactCodexModelSelectionForEvent(judgment.meta?.modelSelection),
|
|
340
234
|
legacyPendingDiagnostic: legacyPending.diagnostic,
|
|
341
235
|
backendDurationMs: judgment.meta?.durationMs ?? null,
|
|
@@ -357,8 +251,6 @@ export async function runCodexUserPromptSubmitHook({
|
|
|
357
251
|
proposedToolIds: toolIds,
|
|
358
252
|
backend: judgment.meta?.backend ?? backend.name ?? 'unknown',
|
|
359
253
|
model: judgment.meta?.modelSelection?.effectiveModel ?? null,
|
|
360
|
-
auditorSeenContext: JSON.stringify(context.turns),
|
|
361
|
-
config: auditorContextConfig,
|
|
362
254
|
});
|
|
363
255
|
const advice = projectParentAdvice(toolIds);
|
|
364
256
|
if (advice) writeCodexUserPromptContexts({ contexts: [advice], writeOutput });
|
|
@@ -848,14 +740,6 @@ function codexSessionId(payload) {
|
|
|
848
740
|
return typeof value === 'string' && value.length > 0 ? value : null;
|
|
849
741
|
}
|
|
850
742
|
|
|
851
|
-
function requireCodexSessionId(payload) {
|
|
852
|
-
const value = codexSessionId(payload);
|
|
853
|
-
if (value) return value;
|
|
854
|
-
const err = new Error('session_id is required');
|
|
855
|
-
err.code = 'E_AUDITOR_CONTEXT_INPUT';
|
|
856
|
-
throw err;
|
|
857
|
-
}
|
|
858
|
-
|
|
859
743
|
// v1.4.19: legacy pending migration is handled only by `discardLegacyPending` on
|
|
860
744
|
// UserPromptSubmit. Stop findings are never persisted for a future parent prompt.
|
|
861
745
|
|
package/src/cli/doctor.mjs
CHANGED
|
@@ -78,7 +78,7 @@ export async function runDoctor() {
|
|
|
78
78
|
if (!sidecar.ok) warnings += 1;
|
|
79
79
|
|
|
80
80
|
const auditorContext = await inspectAuditorContextConfiguration({ projectRoot });
|
|
81
|
-
mark(auditorContext.ok, `
|
|
81
|
+
mark(auditorContext.ok, `evaluation context: ${auditorContext.mode}`, auditorContext.detail);
|
|
82
82
|
if (!auditorContext.ok) warnings += 1;
|
|
83
83
|
}
|
|
84
84
|
|
|
@@ -45,7 +45,7 @@ export async function runFactoryDiagnostics({
|
|
|
45
45
|
if (contextMode === null) {
|
|
46
46
|
checks.push(check('throughline_context', 'fail', 'invalid_context_configuration'));
|
|
47
47
|
} else if (contextMode === 'disabled') {
|
|
48
|
-
checks.push(check('throughline_context', 'skipped', '
|
|
48
|
+
checks.push(check('throughline_context', 'skipped', 'evaluation_evidence_disabled'));
|
|
49
49
|
} else {
|
|
50
50
|
const context = await inspectAuditorContextFn({ projectRoot });
|
|
51
51
|
checks.push(context?.ok === true && context?.mode === 'throughline'
|
package/src/cli/install.mjs
CHANGED
|
@@ -102,12 +102,12 @@ export async function runInstall({
|
|
|
102
102
|
};
|
|
103
103
|
await writeFile(markerPath, JSON.stringify(marker, null, 2) + '\n', 'utf8');
|
|
104
104
|
console.log(` wrote ${markerPath}`);
|
|
105
|
-
console.log(`
|
|
105
|
+
console.log(` evaluation context provider: ${preservedAuditorContext.mode} (${preservedAuditorContext.origin})`);
|
|
106
106
|
if (preservedAuditorContext.mode === 'throughline') {
|
|
107
|
-
console.log('
|
|
107
|
+
console.log(' captures bounded proposal-time observer evidence; this never gates auditing; disable evidence capture with:');
|
|
108
108
|
console.log(' spotter install -y --auditor-context disabled');
|
|
109
109
|
} else if (preservedAuditorContext.reason === 'throughline_unavailable') {
|
|
110
|
-
console.log(' Throughline was not found;
|
|
110
|
+
console.log(' Throughline was not found; proposal-time observer evidence is unavailable, but auditing remains active');
|
|
111
111
|
}
|
|
112
112
|
}
|
|
113
113
|
|
package/src/daemon/daemon.mjs
CHANGED
|
@@ -277,30 +277,7 @@ export async function startDaemon({
|
|
|
277
277
|
? payload.observation_id
|
|
278
278
|
: null;
|
|
279
279
|
|
|
280
|
-
|
|
281
|
-
logFn('user_input: audit skipped because fresh context was unavailable');
|
|
282
|
-
return { pass: true, missing_tools: [], reason: 'auditor_context_not_fresh' };
|
|
283
|
-
}
|
|
284
|
-
|
|
285
|
-
const hasContext = payload.context_status !== undefined || payload.recent_context !== undefined;
|
|
286
|
-
if (hasContext && (payload.context_status !== 'fresh' || !Array.isArray(payload.recent_context))) {
|
|
287
|
-
const err = new Error('context-bearing user_input payload must include fresh recent_context');
|
|
288
|
-
err.code = 'E_AUDITOR_CONTEXT_INPUT';
|
|
289
|
-
throw err;
|
|
290
|
-
}
|
|
291
|
-
if (hasContext && auditorBackend.name === 'haiku') {
|
|
292
|
-
const err = new Error('recent conversation context is not supported by the haiku auditor backend');
|
|
293
|
-
err.code = 'E_AUDITOR_CONTEXT_BACKEND_UNSUPPORTED';
|
|
294
|
-
throw err;
|
|
295
|
-
}
|
|
296
|
-
const judgment = await runAuditorJudgment(hasContext
|
|
297
|
-
? {
|
|
298
|
-
stage: 'user_input',
|
|
299
|
-
userInput,
|
|
300
|
-
recentContext: payload.recent_context,
|
|
301
|
-
contextStatus: 'fresh',
|
|
302
|
-
}
|
|
303
|
-
: { stage: 'user_input', userInput });
|
|
280
|
+
const judgment = await runAuditorJudgment({ stage: 'user_input', userInput });
|
|
304
281
|
const result = legacyResultFromJudgment(judgment);
|
|
305
282
|
const meta = judgment.meta ?? {};
|
|
306
283
|
logFn(
|
package/src/dashboard/render.mjs
CHANGED
|
@@ -3,6 +3,14 @@
|
|
|
3
3
|
// opens EvaluationStore or asks Throughline for context.
|
|
4
4
|
|
|
5
5
|
const METRIC_KEYS = ['S', 'P', 'I', 'C', 'A', 'M'];
|
|
6
|
+
const METRIC_LABELS = {
|
|
7
|
+
S: '対象ターン',
|
|
8
|
+
P: 'ツール提案あり',
|
|
9
|
+
I: '提案ツール数',
|
|
10
|
+
C: '利用判定済み',
|
|
11
|
+
A: '実際に使用',
|
|
12
|
+
M: '判定不能',
|
|
13
|
+
};
|
|
6
14
|
|
|
7
15
|
/**
|
|
8
16
|
* Render a complete dashboard document.
|
|
@@ -96,7 +104,7 @@ function renderDeviceContent({ overview, cases, caseDetail, filters, action }) {
|
|
|
96
104
|
return `${renderFilters(filters, action)}
|
|
97
105
|
<section aria-labelledby="overview-heading">
|
|
98
106
|
<div class="section-heading"><h2 id="overview-heading">概要</h2>${renderRates(totals)}</div>
|
|
99
|
-
<div class="metrics" aria-label="評価メトリクス">${METRIC_KEYS.map((key) => `<article class="metric"><span>${key}</span><strong>${escapeHtml(metric(totals, key))}</strong></article>`).join('')}</div>
|
|
107
|
+
<div class="metrics" aria-label="評価メトリクス">${METRIC_KEYS.map((key) => `<article class="metric"><span>${METRIC_LABELS[key]}</span><strong>${escapeHtml(metric(totals, key))}</strong></article>`).join('')}</div>
|
|
100
108
|
</section>
|
|
101
109
|
${renderBreakdown('project-breakdown', 'project別内訳', projects)}
|
|
102
110
|
${renderBreakdown('tool-breakdown', 'tool別内訳', tools)}
|
|
@@ -119,13 +127,13 @@ function renderFilters(filters, action) {
|
|
|
119
127
|
function renderRates(summary) {
|
|
120
128
|
const proposal = rate(metricNumber(summary, 'P'), metricNumber(summary, 'S'));
|
|
121
129
|
const adoption = rate(metricNumber(summary, 'A'), metricNumber(summary, 'C'));
|
|
122
|
-
return `<dl class="rates"><div><dt
|
|
130
|
+
return `<dl class="rates"><div><dt>提案率<small>ツール提案あり ÷ 対象ターン</small></dt><dd>${escapeHtml(proposal)}</dd></div><div><dt>採用率<small>実際に使用 ÷ 利用判定済み</small></dt><dd>${escapeHtml(adoption)}</dd></div></dl>`;
|
|
123
131
|
}
|
|
124
132
|
|
|
125
133
|
function renderBreakdown(id, title, rows) {
|
|
126
134
|
return `<section aria-labelledby="${id}">
|
|
127
135
|
<h2 id="${id}">${title}</h2>
|
|
128
|
-
${rows.length === 0 ? '<p class="empty">該当するデータはありません。</p>' : `<div class="table-wrap"><table><thead><tr><th scope="col">項目</th>${METRIC_KEYS.map((key) => `<th scope="col">${key}</th>`).join('')}<th scope="col">提案率</th><th scope="col">採用率</th></tr></thead><tbody>${rows.map(([label, summary]) => `<tr><th scope="row">${escapeHtml(label)}</th>${METRIC_KEYS.map((key) => `<td>${escapeHtml(metric(summary, key))}</td>`).join('')}<td>${escapeHtml(rate(metricNumber(summary, 'P'), metricNumber(summary, 'S')))}</td><td>${escapeHtml(rate(metricNumber(summary, 'A'), metricNumber(summary, 'C')))}</td></tr>`).join('')}</tbody></table></div>`}
|
|
136
|
+
${rows.length === 0 ? '<p class="empty">該当するデータはありません。</p>' : `<div class="table-wrap"><table><thead><tr><th scope="col">項目</th>${METRIC_KEYS.map((key) => `<th scope="col">${METRIC_LABELS[key]}</th>`).join('')}<th scope="col">提案率</th><th scope="col">採用率</th></tr></thead><tbody>${rows.map(([label, summary]) => `<tr><th scope="row">${escapeHtml(label)}</th>${METRIC_KEYS.map((key) => `<td>${escapeHtml(metric(summary, key))}</td>`).join('')}<td>${escapeHtml(rate(metricNumber(summary, 'P'), metricNumber(summary, 'S')))}</td><td>${escapeHtml(rate(metricNumber(summary, 'A'), metricNumber(summary, 'C')))}</td></tr>`).join('')}</tbody></table></div>`}
|
|
129
137
|
</section>`;
|
|
130
138
|
}
|
|
131
139
|
|
|
@@ -217,8 +225,8 @@ h1 { margin: .25rem 0; font-size: clamp(1.75rem, 4vw, 2.5rem); } h2 { margin-top
|
|
|
217
225
|
.device { align-items: center; border: 1px solid #ccd4dd; border-radius: 999px; color: inherit; display: inline-flex; gap: .45rem; padding: .45rem .7rem; text-decoration: none; }
|
|
218
226
|
.device[aria-current="page"] { border-color: #1c63b8; box-shadow: 0 0 0 2px #b8d6fa; }.device small { color: #52606d; }.status-dot { background: #a03333; border-radius: 50%; height: .55rem; width: .55rem; }.online .status-dot { background: #16803c; }
|
|
219
227
|
.filters { align-items: end; display: flex; flex-wrap: wrap; gap: .75rem; margin: 1.5rem 0; }.filters label { display: grid; font-size: .85rem; gap: .25rem; }.filters input, button { border: 1px solid #aeb8c4; border-radius: .35rem; font: inherit; padding: .45rem; }button { background: #1c63b8; color: white; cursor: pointer; }
|
|
220
|
-
.section-heading { align-items: baseline; display: flex; flex-wrap: wrap; gap: 1rem; justify-content: space-between; }.rates { display: flex; gap: 1.25rem; margin: 0; }.rates div { display: flex; gap: .
|
|
221
|
-
.metrics { display: grid; gap: .75rem; grid-template-columns: repeat(6, minmax(
|
|
228
|
+
.section-heading { align-items: baseline; display: flex; flex-wrap: wrap; gap: 1rem; justify-content: space-between; }.rates { display: flex; gap: 1.25rem; margin: 0; }.rates div { display: flex; gap: .45rem; }.rates dt { color: #52606d; }.rates dt small { display: block; font-size: .72rem; }.rates dd { font-weight: 700; margin: 0; }
|
|
229
|
+
.metrics { display: grid; gap: .75rem; grid-template-columns: repeat(6, minmax(110px, 1fr)); }.metric { background: #fff; border: 1px solid #d7dce2; border-radius: .5rem; padding: .8rem; }.metric span { color: #52606d; display: block; }.metric strong { font-size: 1.5rem; }
|
|
222
230
|
.table-wrap { overflow-x: auto; }table { border-collapse: collapse; min-width: 680px; width: 100%; }th, td { border-bottom: 1px solid #d7dce2; padding: .6rem; text-align: left; vertical-align: top; }thead { background: #eaf0f6; }tbody tr:nth-child(even) { background: #fbfcfd; }
|
|
223
231
|
.case-detail { border-top: 2px solid #1c63b8; margin-top: 2.5rem; }.metadata { display: flex; flex-wrap: wrap; gap: 1rem; }.metadata div { min-width: 12rem; }.metadata dt { color: #52606d; }.metadata dd { margin: .2rem 0; overflow-wrap: anywhere; }pre { background: #1e2935; color: #e6edf3; margin: 0; overflow-x: auto; padding: 1rem; white-space: pre-wrap; word-break: break-word; }
|
|
224
232
|
@media (max-width: 700px) { .dashboard { padding-top: 1rem; }.metrics { grid-template-columns: repeat(3, 1fr); }.filters { align-items: stretch; flex-direction: column; }.filters input, button { width: 100%; }.rates { flex-direction: column; gap: .25rem; } }
|
|
@@ -25,10 +25,6 @@ import { sendRequest, TransportError } from '../daemon/transport.mjs';
|
|
|
25
25
|
import { spawnDaemonAndWaitReady } from './spawn-daemon.mjs';
|
|
26
26
|
import { discardLegacyPending } from './pending-context.mjs';
|
|
27
27
|
import { projectBackendFailure, projectParentAdvice, projectToolIds } from './parent-output-projector.mjs';
|
|
28
|
-
import {
|
|
29
|
-
loadAuditorContext,
|
|
30
|
-
readProjectAuditorContextConfig,
|
|
31
|
-
} from '../core/auditor-context.mjs';
|
|
32
28
|
import { randomUUID } from 'node:crypto';
|
|
33
29
|
import { createEvaluationStore } from '../core/evaluation-store.mjs';
|
|
34
30
|
import { loadEvaluationObserverContext } from '../core/evaluation-context.mjs';
|
|
@@ -41,8 +37,6 @@ export async function runUserPrompt({
|
|
|
41
37
|
sendRequestFn = sendRequest,
|
|
42
38
|
spawnDaemonAndWaitReadyFn = spawnDaemonAndWaitReady,
|
|
43
39
|
discardLegacyPendingFn = discardLegacyPending,
|
|
44
|
-
readAuditorContextConfigFn = readProjectAuditorContextConfig,
|
|
45
|
-
loadAuditorContextFn = loadAuditorContext,
|
|
46
40
|
recordHookEventFn = recordClaudeHookEvent,
|
|
47
41
|
createEvaluationStoreFn = createEvaluationStore,
|
|
48
42
|
loadEvaluationObserverContextFn = loadEvaluationObserverContext,
|
|
@@ -62,10 +56,9 @@ export async function runUserPrompt({
|
|
|
62
56
|
const observationId = randomUUIDFn();
|
|
63
57
|
const projectRoot = findSpotterMarker(input.cwd);
|
|
64
58
|
const startedAt = Date.now();
|
|
65
|
-
let contextDurationMs = null;
|
|
66
59
|
let evaluationWriteFailed = false;
|
|
67
60
|
|
|
68
|
-
const safelyRecordEvaluation = async ({ auditStatus, result = null
|
|
61
|
+
const safelyRecordEvaluation = async ({ auditStatus, result = null }) => {
|
|
69
62
|
if (!projectRoot) return;
|
|
70
63
|
try {
|
|
71
64
|
const proposedToolIds = auditStatus === 'success' && result?.pass !== true
|
|
@@ -77,7 +70,6 @@ export async function runUserPrompt({
|
|
|
77
70
|
projectRoot,
|
|
78
71
|
host: 'claude',
|
|
79
72
|
sessionId,
|
|
80
|
-
config: auditContext?.config,
|
|
81
73
|
recordedAtMs: proposalRecordedAtMs,
|
|
82
74
|
})
|
|
83
75
|
: { status: 'not_requested', snapshot: null };
|
|
@@ -94,7 +86,7 @@ export async function runUserPrompt({
|
|
|
94
86
|
sessionId,
|
|
95
87
|
auditStatus,
|
|
96
88
|
requestText: prompt,
|
|
97
|
-
auditorSeenContext:
|
|
89
|
+
auditorSeenContext: null,
|
|
98
90
|
observerContextStatus: observer.status,
|
|
99
91
|
observerSnapshot: observer.snapshot,
|
|
100
92
|
proposedToolIds,
|
|
@@ -128,7 +120,6 @@ export async function runUserPrompt({
|
|
|
128
120
|
status: 'degraded',
|
|
129
121
|
code: failure.code,
|
|
130
122
|
reason,
|
|
131
|
-
contextDurationMs,
|
|
132
123
|
legacyPendingDiagnostic: legacyPending.diagnostic,
|
|
133
124
|
durationMs: Date.now() - startedAt,
|
|
134
125
|
},
|
|
@@ -151,108 +142,12 @@ export async function runUserPrompt({
|
|
|
151
142
|
}
|
|
152
143
|
};
|
|
153
144
|
|
|
154
|
-
const syncUserInputWithoutAudit = async (contextStatus) => {
|
|
155
|
-
try {
|
|
156
|
-
const response = await sendDaemonPayload({
|
|
157
|
-
user_input: prompt,
|
|
158
|
-
observation_id: observationId,
|
|
159
|
-
audit: false,
|
|
160
|
-
context_status: contextStatus,
|
|
161
|
-
});
|
|
162
|
-
if (response.ok !== true) {
|
|
163
|
-
await degrade({ code: response.error?.code ?? 'E_INTERNAL', reason: 'daemon_state_sync' });
|
|
164
|
-
return false;
|
|
165
|
-
}
|
|
166
|
-
return true;
|
|
167
|
-
} catch (err) {
|
|
168
|
-
await degrade({ code: err?.code, reason: 'daemon_state_sync' });
|
|
169
|
-
return false;
|
|
170
|
-
}
|
|
171
|
-
};
|
|
172
|
-
|
|
173
|
-
let context;
|
|
174
|
-
let auditorContextConfig;
|
|
175
|
-
const contextStartedAt = Date.now();
|
|
176
|
-
try {
|
|
177
|
-
const config = await readAuditorContextConfigFn(projectRoot);
|
|
178
|
-
auditorContextConfig = config;
|
|
179
|
-
contextDurationMs = Date.now() - contextStartedAt;
|
|
180
|
-
if (config.mode === 'disabled') {
|
|
181
|
-
if (!await syncUserInputWithoutAudit('disabled')) {
|
|
182
|
-
await safelyRecordEvaluation({ auditStatus: 'error' });
|
|
183
|
-
return;
|
|
184
|
-
}
|
|
185
|
-
await safelyRecordEvaluation({ auditStatus: 'skipped' });
|
|
186
|
-
await recordHookEventFn({
|
|
187
|
-
projectRoot,
|
|
188
|
-
event: {
|
|
189
|
-
hook: 'UserPromptSubmit',
|
|
190
|
-
status: 'skipped',
|
|
191
|
-
reason: 'context_disabled',
|
|
192
|
-
contextStatus: 'disabled',
|
|
193
|
-
contextDurationMs,
|
|
194
|
-
legacyPendingDiagnostic: legacyPending.diagnostic,
|
|
195
|
-
durationMs: Date.now() - startedAt,
|
|
196
|
-
},
|
|
197
|
-
});
|
|
198
|
-
return;
|
|
199
|
-
}
|
|
200
|
-
context = await loadAuditorContextFn({
|
|
201
|
-
config,
|
|
202
|
-
host: 'claude',
|
|
203
|
-
sessionId,
|
|
204
|
-
projectRoot,
|
|
205
|
-
transcriptPath: requireString(input, 'transcript_path'),
|
|
206
|
-
});
|
|
207
|
-
contextDurationMs = Date.now() - contextStartedAt;
|
|
208
|
-
} catch (err) {
|
|
209
|
-
contextDurationMs = Date.now() - contextStartedAt;
|
|
210
|
-
await degrade({ code: err?.code, reason: 'auditor_context' });
|
|
211
|
-
await safelyRecordEvaluation({ auditStatus: 'error' });
|
|
212
|
-
return;
|
|
213
|
-
}
|
|
214
|
-
|
|
215
|
-
if (context.status === 'unavailable' || context.status === 'schema_mismatch') {
|
|
216
|
-
await degrade({
|
|
217
|
-
code: context.status === 'unavailable'
|
|
218
|
-
? 'E_AUDITOR_CONTEXT_UNAVAILABLE'
|
|
219
|
-
: 'E_AUDITOR_CONTEXT_SCHEMA',
|
|
220
|
-
reason: 'auditor_context_status',
|
|
221
|
-
});
|
|
222
|
-
await safelyRecordEvaluation({ auditStatus: 'error' });
|
|
223
|
-
return;
|
|
224
|
-
}
|
|
225
|
-
if (context.status !== 'fresh') {
|
|
226
|
-
if (!await syncUserInputWithoutAudit(context.status)) {
|
|
227
|
-
await safelyRecordEvaluation({ auditStatus: 'error' });
|
|
228
|
-
return;
|
|
229
|
-
}
|
|
230
|
-
await safelyRecordEvaluation({ auditStatus: 'skipped' });
|
|
231
|
-
await recordHookEventFn({
|
|
232
|
-
projectRoot,
|
|
233
|
-
event: {
|
|
234
|
-
hook: 'UserPromptSubmit',
|
|
235
|
-
status: 'skipped',
|
|
236
|
-
reason: 'context_not_fresh',
|
|
237
|
-
contextStatus: context.status,
|
|
238
|
-
contextTurns: 0,
|
|
239
|
-
contextChars: 0,
|
|
240
|
-
contextDurationMs,
|
|
241
|
-
legacyPendingDiagnostic: legacyPending.diagnostic,
|
|
242
|
-
durationMs: Date.now() - startedAt,
|
|
243
|
-
},
|
|
244
|
-
});
|
|
245
|
-
return;
|
|
246
|
-
}
|
|
247
|
-
|
|
248
145
|
let response;
|
|
249
146
|
try {
|
|
250
147
|
response = await sendDaemonPayload({
|
|
251
148
|
user_input: prompt,
|
|
252
149
|
observation_id: observationId,
|
|
253
150
|
audit: true,
|
|
254
|
-
context_status: 'fresh',
|
|
255
|
-
recent_context: context.turns,
|
|
256
151
|
});
|
|
257
152
|
} catch (err) {
|
|
258
153
|
await degrade({ code: err?.code ?? 'E_RESURRECT_FAILED', reason: 'transport_or_resurrect' });
|
|
@@ -278,7 +173,6 @@ export async function runUserPrompt({
|
|
|
278
173
|
await safelyRecordEvaluation({
|
|
279
174
|
auditStatus: 'success',
|
|
280
175
|
result,
|
|
281
|
-
auditContext: { config: auditorContextConfig, turns: context.turns },
|
|
282
176
|
});
|
|
283
177
|
if (advice) emitAdditionalContext(writeOutput, advice);
|
|
284
178
|
|
|
@@ -289,10 +183,6 @@ export async function runUserPrompt({
|
|
|
289
183
|
status: 'success',
|
|
290
184
|
pass: result.pass === true,
|
|
291
185
|
missingTools: toolIds,
|
|
292
|
-
contextStatus: 'fresh',
|
|
293
|
-
contextTurns: context.stats.returnedTurns,
|
|
294
|
-
contextChars: context.stats.chars,
|
|
295
|
-
contextDurationMs,
|
|
296
186
|
legacyPendingDiagnostic: legacyPending.diagnostic,
|
|
297
187
|
durationMs: Date.now() - startedAt,
|
|
298
188
|
},
|