tink-harness 1.3.0 → 1.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +1 -1
- package/CHANGELOG.md +31 -0
- package/README.ko.md +11 -3
- package/README.md +12 -4
- package/VERSIONING.md +1 -1
- package/bin/install.js +67 -0
- package/commands/cast.md +4 -1
- package/docs/context-budget-ledger.ko.md +2 -0
- package/docs/context-budget-ledger.md +2 -0
- package/docs/context-metrics-evaluator.ko.md +39 -0
- package/docs/context-metrics-evaluator.md +39 -0
- package/docs/context-run-history-rollup.ko.md +36 -0
- package/docs/context-run-history-rollup.md +36 -0
- package/docs/context-run-record-policy.ko.md +50 -0
- package/docs/context-run-record-policy.md +50 -0
- package/docs/context-threshold-status.ko.md +43 -0
- package/docs/context-threshold-status.md +43 -0
- package/docs/pr/2026-06-08-codex-surface-cleanup.ko.md +27 -0
- package/docs/pr/2026-06-08-context-metrics-artifact.ko.md +30 -0
- package/docs/pr/2026-06-08-context-metrics-evaluator.ko.md +28 -0
- package/docs/pr/2026-06-08-context-run-history-rollup.ko.md +27 -0
- package/docs/pr/2026-06-08-context-run-record-policy.ko.md +25 -0
- package/docs/pr/2026-06-08-context-threshold-status.ko.md +25 -0
- package/docs/pr/2026-06-08-v1.5.0.ko.md +30 -0
- package/docs/update-troubleshooting.ko.md +6 -1
- package/docs/update-troubleshooting.md +6 -1
- package/docs/update-verification-recipe.ko.md +3 -0
- package/docs/update-verification-recipe.md +3 -0
- package/docs/work-state.ko.md +7 -3
- package/docs/work-state.md +7 -3
- package/package.json +1 -1
- package/templates/claude/commands/tink/cast.md +4 -1
- package/templates/codex/skills/tink-core/RULES.md +4 -1
- package/templates/tink/schemas/context-metrics-evaluation.schema.json +89 -0
package/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,37 @@ All notable changes to Tink are tracked here.
|
|
|
7
7
|
No unreleased changes yet.
|
|
8
8
|
|
|
9
9
|
|
|
10
|
+
## [1.5.0] - 2026-06-08
|
|
11
|
+
|
|
12
|
+
### Changed
|
|
13
|
+
|
|
14
|
+
- Codex-only install/update now removes repo-local Tink Claude Code command files under `.claude/commands/tink/*.md` so Codex no longer shows them as stale `Source Command Tink ...` entries after a Codex-only refresh.
|
|
15
|
+
- Codex-only install/update now removes the old repo-local `.claude/skills/tink/SKILL.md` Tink surface when it matches the known generated Tink skill, while preserving unknown user-authored content.
|
|
16
|
+
- Update troubleshooting and verification docs now explain when `Source Command Tink ...` is stale and when it is expected because the repo intentionally keeps the Claude Code surface.
|
|
17
|
+
- README and Korean README now highlight the v1.5.0 update behavior for existing Codex users.
|
|
18
|
+
|
|
19
|
+
### Added
|
|
20
|
+
|
|
21
|
+
- Regression coverage for Codex-only update cleanup of repo-local Claude command and skill surfaces.
|
|
22
|
+
- Korean PR history draft for the v1.5.0 release in `docs/pr/2026-06-08-v1.5.0.ko.md`.
|
|
23
|
+
|
|
24
|
+
|
|
25
|
+
## [1.4.0] - 2026-06-08
|
|
26
|
+
|
|
27
|
+
### Added
|
|
28
|
+
|
|
29
|
+
- `context-metrics-evaluation.schema.json` for `.tink/current/context-metrics-evaluation.json`.
|
|
30
|
+
- Context Metrics Evaluator run-state artifact guidance for `/tink:cast` and `$tink:cast`.
|
|
31
|
+
- Fixture-ratio evaluation docs in Korean and English, explaining measured fixture scope versus production telemetry.
|
|
32
|
+
- Test-backed context metrics evaluation fixture that calculates all six context-efficiency metrics at or above the 90% target within fixture scope.
|
|
33
|
+
- Korean PR history draft for the Context Metrics Artifact work in `docs/pr/2026-06-08-context-metrics-artifact.ko.md`.
|
|
34
|
+
|
|
35
|
+
### Changed
|
|
36
|
+
|
|
37
|
+
- Work State Guide now includes `context-metrics-evaluation.json` in the reading order.
|
|
38
|
+
- README and Korean README now link to the Context Metrics Evaluator docs without adding a new command.
|
|
39
|
+
|
|
40
|
+
|
|
10
41
|
## [1.3.0] - 2026-06-08
|
|
11
42
|
|
|
12
43
|
### Added
|
package/README.ko.md
CHANGED
|
@@ -8,7 +8,7 @@ Claude Code와 Codex를 위한 작은 하네스 레이어입니다.
|
|
|
8
8
|
|
|
9
9
|
Tink는 지금 작업에 맞는 하네스를 고르고, 실행 상태를 보이게 만들고, 실제 사용 중 생긴 실패와 피드백으로 하네스 세트를 개선합니다.
|
|
10
10
|
|
|
11
|
-
**최신 릴리스:** v1.
|
|
11
|
+
**최신 릴리스:** v1.5.0 — Codex-only update에서 Codex skill picker에 `Source Command Tink ...`로 보이던 repo-local Claude Tink surface를 정리합니다.
|
|
12
12
|
|
|
13
13
|
[English](README.md) · **한국어**
|
|
14
14
|
|
|
@@ -59,14 +59,22 @@ npx tink-harness@latest update
|
|
|
59
59
|
|
|
60
60
|
업데이트 후 Codex skill, schema, Windows 경고가 이상해 보이면 `docs/update-troubleshooting.ko.md` 또는 `docs/update-troubleshooting.md`를 확인하세요.
|
|
61
61
|
|
|
62
|
-
## 1.
|
|
62
|
+
## 1.5.0에서 달라진 점
|
|
63
|
+
|
|
64
|
+
이번 릴리스는 기존 repo에서 Codex skill picker가 헷갈리게 보이는 문제를 고쳤습니다.
|
|
65
|
+
|
|
66
|
+
- Codex-only `tink-harness update`가 repo-local `.claude/commands/tink/*.md`와 예전 repo-local `.claude/skills/tink/SKILL.md` Tink surface를 정리합니다.
|
|
67
|
+
- 그래서 Codex에서 `$tink:*` action skill만 기대하는 상황에 `Source Command Tink Frog/List/...` 또는 넓은 `Tink` 항목이 같이 보이는 일을 줄입니다.
|
|
68
|
+
- 의도적으로 Claude Code와 Codex를 둘 다 설치한 경우에는 repo-local Claude Code command가 남을 수 있고, 이때 `Source Command Tink ...` 항목은 정상일 수 있습니다. 자세한 내용은 `docs/update-troubleshooting.ko.md` 또는 `docs/update-troubleshooting.md`를 확인하세요.
|
|
69
|
+
|
|
70
|
+
## 1.2.0 이후 기반 개선
|
|
63
71
|
|
|
64
72
|
이번 릴리스는 Tink를 Claude Code와 Codex에서 같은 하네스 레이어로 쓰기 쉽게 정리합니다.
|
|
65
73
|
|
|
66
74
|
- Codex에는 하나의 넓은 `tink` 스킬 대신 `$tink:cast`, `$tink:verify` 같은 action skill만 보이도록 설치됩니다.
|
|
67
75
|
- 비단순 작업은 `context-pack.md`, `context-map.json`, `excluded-context.md`로 어떤 context를 썼고 뺐는지 남깁니다.
|
|
68
76
|
- Repo Signal과 Context Graph Lite는 새 `tink index` 명령을 만들지 않고도 관련 테스트, 스키마, 동기화 파일, 검증 힌트를 고르는 데 쓰입니다.
|
|
69
|
-
- context 효율
|
|
77
|
+
- context 효율 점수화, fixture 비율 계산, run-history rollup, 90% threshold status, 실제 run 기록 경계 기준은 `docs/context-budget-ledger.ko.md`, `docs/context-budget-ledger.md`, `docs/context-metrics-evaluator.ko.md`, `docs/context-metrics-evaluator.md`, `docs/context-run-history-rollup.ko.md`, `docs/context-run-history-rollup.md`, `docs/context-threshold-status.ko.md`, `docs/context-threshold-status.md`, `docs/context-run-record-policy.ko.md`, `docs/context-run-record-policy.md`에서 확인할 수 있습니다.
|
|
70
78
|
- `/tink:verify`와 `$tink:verify`는 같은 Verify Runner 모델을 쓰며 `.tink/current/verification.json`에 검증 증거를 남깁니다.
|
|
71
79
|
- 외부 context는 MCP Safe Profile을 따릅니다. 가장 작은 source handle만 남기고, 신뢰도와 민감도를 표시하며, 위험하거나 너무 넓은 context는 `excluded-context.md`에 따로 기록합니다.
|
|
72
80
|
|
package/README.md
CHANGED
|
@@ -17,14 +17,14 @@
|
|
|
17
17
|
</p>
|
|
18
18
|
|
|
19
19
|
<p>
|
|
20
|
-
<a href="https://github.com/dotoricode/tink-harness/releases/tag/v1.
|
|
20
|
+
<a href="https://github.com/dotoricode/tink-harness/releases/tag/v1.5.0"><img src="https://img.shields.io/github/v/release/dotoricode/tink-harness?label=release&color=2ea44f" alt="GitHub release"></a>
|
|
21
21
|
<a href="https://www.npmjs.com/package/tink-harness"><img src="https://img.shields.io/npm/v/tink-harness?label=npm&color=cb3837" alt="npm version"></a>
|
|
22
22
|
<a href="https://github.com/dotoricode/tink-harness/actions/workflows/ci.yml"><img src="https://img.shields.io/github/actions/workflow/status/dotoricode/tink-harness/ci.yml?branch=main&label=ci" alt="CI"></a>
|
|
23
23
|
<a href="https://github.com/dotoricode/tink-harness/blob/main/LICENSE"><img src="https://img.shields.io/github/license/dotoricode/tink-harness" alt="License"></a>
|
|
24
24
|
<a href="https://github.com/dotoricode/tink-harness/stargazers"><img src="https://img.shields.io/github/stars/dotoricode/tink-harness?style=social" alt="GitHub stars"></a>
|
|
25
25
|
</p>
|
|
26
26
|
|
|
27
|
-
<p><strong>Latest release:</strong> v1.
|
|
27
|
+
<p><strong>Latest release:</strong> v1.5.0 - Codex-only update now removes repo-local Claude Tink surfaces that appeared as <code>Source Command Tink ...</code>.</p>
|
|
28
28
|
|
|
29
29
|
**English** · [한국어](README.ko.md)
|
|
30
30
|
|
|
@@ -124,14 +124,22 @@ To quickly verify the updated install, see `docs/update-verification-recipe.md`
|
|
|
124
124
|
|
|
125
125
|
If an update looks stale or incomplete, see `docs/update-troubleshooting.md` or `docs/update-troubleshooting.ko.md`.
|
|
126
126
|
|
|
127
|
-
## What's new in 1.
|
|
127
|
+
## What's new in 1.5.0
|
|
128
|
+
|
|
129
|
+
This release fixes the Codex skill picker state after updating an existing repo.
|
|
130
|
+
|
|
131
|
+
- Codex-only `tink-harness update` now removes repo-local `.claude/commands/tink/*.md` and the old repo-local `.claude/skills/tink/SKILL.md` Tink surface.
|
|
132
|
+
- This prevents Codex from showing `Source Command Tink Frog/List/...` or a broad repo-local `Tink` entry when the user expects only `$tink:*` action skills.
|
|
133
|
+
- If you intentionally install both Claude Code and Codex surfaces, repo-local Claude Code commands can remain and `Source Command Tink ...` entries may be expected. See `docs/update-troubleshooting.md` or `docs/update-troubleshooting.ko.md`.
|
|
134
|
+
|
|
135
|
+
## Recent foundation from 1.2.0+
|
|
128
136
|
|
|
129
137
|
This release makes Tink work as one harness layer across Claude Code and Codex.
|
|
130
138
|
|
|
131
139
|
- Codex now installs focused `$tink:*` action skills instead of one broad visible `tink` skill, so the picker shows commands like `$tink:cast` and `$tink:verify` cleanly.
|
|
132
140
|
- Non-trivial runs now create context artifacts: `context-pack.md`, `context-map.json`, and `excluded-context.md`.
|
|
133
141
|
- Repo Signals and Context Graph Lite help `/tink:cast` and `$tink:cast` choose relevant tests, schemas, sync partners, and verification hints without adding a new `tink index` command.
|
|
134
|
-
- Context Budget Ledger fields are documented in `docs/context-budget-ledger.md
|
|
142
|
+
- Context Budget Ledger fields, fixture-ratio evaluation, run-history rollup, the 90 percent threshold status, and future real-run record boundaries are documented in `docs/context-budget-ledger.md`, `docs/context-budget-ledger.ko.md`, `docs/context-metrics-evaluator.md`, `docs/context-metrics-evaluator.ko.md`, `docs/context-run-history-rollup.md`, `docs/context-run-history-rollup.ko.md`, `docs/context-threshold-status.md`, `docs/context-threshold-status.ko.md`, `docs/context-run-record-policy.md`, and `docs/context-run-record-policy.ko.md` without adding a new command.
|
|
135
143
|
- `/tink:verify` and `$tink:verify` share one portable Verify Runner model and write compact evidence to `.tink/current/verification.json`.
|
|
136
144
|
- External context now follows the MCP Safe Profile: include only the smallest useful source handle, mark confidence and sensitivity, exclude unsafe context visibly, and connect important claims to verification.
|
|
137
145
|
|
package/VERSIONING.md
CHANGED
package/bin/install.js
CHANGED
|
@@ -369,6 +369,70 @@ function removeLegacyCodexSkill(codexTarget) {
|
|
|
369
369
|
if (!dryRun) fs.rmSync(legacyDir, { recursive: true, force: true });
|
|
370
370
|
}
|
|
371
371
|
|
|
372
|
+
function removeIfExists(base, filePath, label = 'legacy') {
|
|
373
|
+
if (!fs.existsSync(filePath)) return false;
|
|
374
|
+
log.message(`${dryRun ? `would remove ${label}` : `remove ${label}`} ${displayPath(base, filePath)}`);
|
|
375
|
+
recordOperation('removedLegacy', base, filePath);
|
|
376
|
+
if (!dryRun) fs.rmSync(filePath, { recursive: true, force: true });
|
|
377
|
+
return true;
|
|
378
|
+
}
|
|
379
|
+
|
|
380
|
+
function removeRepoLocalClaudeTinkSurface(target) {
|
|
381
|
+
const commandDir = path.join(target, '.claude/commands/tink');
|
|
382
|
+
const flatCommandDir = path.join(target, '.claude/commands');
|
|
383
|
+
const commandFiles = ['setup.md', 'cast.md', 'verify.md', 'list.md', 'frog.md', 'weave.md', 'update.md'];
|
|
384
|
+
for (const name of commandFiles) {
|
|
385
|
+
removeIfExists(target, path.join(commandDir, name), 'repo-local Claude command');
|
|
386
|
+
}
|
|
387
|
+
if (fs.existsSync(commandDir) && fs.readdirSync(commandDir).length === 0) {
|
|
388
|
+
removeIfExists(target, commandDir, 'empty repo-local Claude command dir');
|
|
389
|
+
}
|
|
390
|
+
|
|
391
|
+
const legacyFlatCommands = [
|
|
392
|
+
'tink-setup.md',
|
|
393
|
+
'tink-cast.md',
|
|
394
|
+
'tink-verify.md',
|
|
395
|
+
'tink-list.md',
|
|
396
|
+
'tink-frog.md',
|
|
397
|
+
'tink-weave.md',
|
|
398
|
+
'tink-update.md',
|
|
399
|
+
'tink-forge.md',
|
|
400
|
+
'tink-purge.md',
|
|
401
|
+
'tink-hone.md'
|
|
402
|
+
];
|
|
403
|
+
for (const name of legacyFlatCommands) {
|
|
404
|
+
removeIfExists(target, path.join(flatCommandDir, name), 'repo-local Claude command');
|
|
405
|
+
}
|
|
406
|
+
if (fs.existsSync(flatCommandDir) && fs.readdirSync(flatCommandDir).length === 0) {
|
|
407
|
+
removeIfExists(target, flatCommandDir, 'empty repo-local Claude commands dir');
|
|
408
|
+
}
|
|
409
|
+
|
|
410
|
+
const skillDir = path.join(target, '.claude/skills/tink');
|
|
411
|
+
const skillParentDir = path.join(target, '.claude/skills');
|
|
412
|
+
const skillFile = path.join(skillDir, 'SKILL.md');
|
|
413
|
+
if (!fs.existsSync(skillDir)) {
|
|
414
|
+
if (fs.existsSync(skillParentDir) && fs.readdirSync(skillParentDir).length === 0) {
|
|
415
|
+
removeIfExists(target, skillParentDir, 'empty repo-local Claude skills dir');
|
|
416
|
+
}
|
|
417
|
+
return;
|
|
418
|
+
}
|
|
419
|
+
if (!fs.existsSync(skillFile)) {
|
|
420
|
+
log.message(`keep unknown ${displayPath(target, skillDir)}`);
|
|
421
|
+
recordOperation('keptUnknown', target, skillDir);
|
|
422
|
+
return;
|
|
423
|
+
}
|
|
424
|
+
const text = fs.readFileSync(skillFile, 'utf8');
|
|
425
|
+
if (text.includes('name: tink') && text.includes('# Tink')) {
|
|
426
|
+
removeIfExists(target, skillDir, 'repo-local Claude skill');
|
|
427
|
+
if (fs.existsSync(skillParentDir) && fs.readdirSync(skillParentDir).length === 0) {
|
|
428
|
+
removeIfExists(target, skillParentDir, 'empty repo-local Claude skills dir');
|
|
429
|
+
}
|
|
430
|
+
return;
|
|
431
|
+
}
|
|
432
|
+
log.message(`keep unknown ${displayPath(target, skillDir)}`);
|
|
433
|
+
recordOperation('keptUnknown', target, skillDir);
|
|
434
|
+
}
|
|
435
|
+
|
|
372
436
|
function readJsonFile(filePath, fallback) {
|
|
373
437
|
if (!fs.existsSync(filePath)) return fallback;
|
|
374
438
|
try {
|
|
@@ -417,6 +481,9 @@ function copySelected(scope, components, agent) {
|
|
|
417
481
|
if (includesClaude(agent) && components.includes('commands')) {
|
|
418
482
|
copyTinkCommands(templateRoot, target);
|
|
419
483
|
}
|
|
484
|
+
if (agent === 'codex') {
|
|
485
|
+
removeRepoLocalClaudeTinkSurface(target);
|
|
486
|
+
}
|
|
420
487
|
if (components.includes('skill')) {
|
|
421
488
|
if (includesClaude(agent)) {
|
|
422
489
|
copyDir(path.join(templateRoot, 'claude/skills'), path.join(target, '.claude/skills'), target);
|
package/commands/cast.md
CHANGED
|
@@ -144,6 +144,7 @@ After approval, create `.tink/current/` with these files before doing deeper wor
|
|
|
144
144
|
- `session.json`: lightweight session metadata, especially rule ids already loaded by phase
|
|
145
145
|
- `context-pack.md`: human-readable selected context, including why each item is relevant
|
|
146
146
|
- `context-map.json`: machine-readable included and excluded context with reasons
|
|
147
|
+
- `context-metrics-evaluation.json`: measured or estimated context-efficiency scores, formulas, evidence refs, and limits
|
|
147
148
|
- `excluded-context.md`: notable omitted files, tools, sources, or claims and why they were excluded
|
|
148
149
|
|
|
149
150
|
Create `contract.json` before loading harness bodies. It should be short, factual, and based on the user request plus visible project context:
|
|
@@ -189,9 +190,11 @@ Create context artifacts before deeper implementation work:
|
|
|
189
190
|
- `context-pack.md` should name the user task, selected harnesses, contract summary, loaded rules, selected files/docs, selected external sources, and verification implications.
|
|
190
191
|
- `context-map.json` should contain `task`, `included`, `excluded`, `signals`, and `generated_at`. Each included or excluded entry should include `path` or `source`, `kind`, `reason`, and `confidence`. When external context is selected, also write `external_context[]`.
|
|
191
192
|
- When useful, enrich each context entry with Context Budget Ledger fields: `role`, `cost`, `reuse_signal`, `verification_link`, `staleness`, and `evidence_kind`. Use them to explain why the first context pack is small enough, why excluded context should stay out, and which checks prove selected context matters.
|
|
193
|
+
- `context-metrics-evaluation.json` should contain `run`, `evaluator`, `target_threshold_percent`, `measurement_status`, `scope`, `limits`, and `scores[]`. Each score should include `name`, `score_percent`, `formula`, `numerator`, `denominator`, and `evidence_refs`. If the score is based only on fixtures or the current run, say so in `scope` and `limits`; do not claim production-wide 90% without run-history or telemetry evidence.
|
|
192
194
|
- `excluded-context.md` should make important omissions visible, especially files skipped because they are out of scope, stale, risky, too broad, or unverified external claims.
|
|
193
195
|
|
|
194
196
|
If `.tink/schemas/context-map.schema.json` exists, use it for `context-map.json`. Do not paste the schema into the user response.
|
|
197
|
+
If `.tink/schemas/context-metrics-evaluation.schema.json` exists, use it for `context-metrics-evaluation.json`. Do not paste the schema into the user response.
|
|
195
198
|
|
|
196
199
|
Use deterministic context selection inside cast. Do not create or require a separate `tink index` command for this phase.
|
|
197
200
|
|
|
@@ -401,7 +404,7 @@ A task is trivial only when ALL of the following are true:
|
|
|
401
404
|
15. Run Stitch once before committing to `.tink/current/`. If it triggers, show exactly one proposal before approval. Call `AskUserQuestion` as described in the Interaction policy section.
|
|
402
405
|
16. Ask for explicit approval before non-trivial work.
|
|
403
406
|
17. After approval, read only the selected harness files and any approved run-only draft.
|
|
404
|
-
18. Create `.tink/current/` files from the run state contract, including `contract.json`, `session.json`, `context-pack.md`, `context-map.json`, and `excluded-context.md`.
|
|
407
|
+
18. Create `.tink/current/` files from the run state contract, including `contract.json`, `session.json`, `context-pack.md`, `context-map.json`, `context-metrics-evaluation.json`, and `excluded-context.md`.
|
|
405
408
|
19. Execute the first safe step immediately:
|
|
406
409
|
- inspect relevant files,
|
|
407
410
|
- run a read-only diagnostic,
|
|
@@ -48,6 +48,8 @@ Context Budget Ledger는 Tink가 context를 많이 모으는 대신, 왜 넣었
|
|
|
48
48
|
|
|
49
49
|
실제 telemetry가 없으면 `measurement_status: "estimated"`로 두고 한계를 함께 적는다. 근거 없이 90% 달성으로 표시하지 않는다.
|
|
50
50
|
|
|
51
|
+
fixture에서 비율을 계산할 수 있으면 `measurement_status: "measured"`를 쓸 수 있다. 이때도 scope가 fixture인지 production telemetry인지 분명히 적어야 한다. 자세한 계산 기준은 `docs/context-metrics-evaluator.ko.md`를 본다.
|
|
52
|
+
|
|
51
53
|
## 호환성 기준
|
|
52
54
|
|
|
53
55
|
- Claude Code와 Codex가 같은 schema와 fixture를 읽는다.
|
|
@@ -48,6 +48,8 @@ Entries with `role: "verification_target"` should connect to a command, manual c
|
|
|
48
48
|
|
|
49
49
|
If there is no runtime telemetry yet, mark the scores as `measurement_status: "estimated"` and include the limits. Do not claim 90% without evidence.
|
|
50
50
|
|
|
51
|
+
If fixture ratios can be calculated, `measurement_status: "measured"` is acceptable. The artifact must still state whether the scope is fixture evidence or production telemetry. See `docs/context-metrics-evaluator.md` for the calculation rules.
|
|
52
|
+
|
|
51
53
|
## Compatibility
|
|
52
54
|
|
|
53
55
|
- Claude Code and Codex read the same schema and fixtures.
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
# Context Metrics Evaluator
|
|
2
|
+
|
|
3
|
+
Context Metrics Evaluator는 Context Budget Ledger에 적힌 필드를 실제 비율로 계산하는 테스트 기준이다.
|
|
4
|
+
|
|
5
|
+
이 기능은 새 public command가 아니다. `tink index` 명령, watcher, generated cache, hidden runtime index를 만들지 않는다. `/tink:cast`와 `$tink:cast`는 `.tink/current/context-metrics-evaluation.json`을 run state artifact로 남기고, `tests/fixtures/current-run/context-metrics-evaluation.json`과 `tests/test_templates.py`는 같은 점수를 계산하는지 확인한다.
|
|
6
|
+
|
|
7
|
+
영어판은 `docs/context-metrics-evaluator.md`에 있다.
|
|
8
|
+
|
|
9
|
+
## 왜 필요한가
|
|
10
|
+
|
|
11
|
+
`context-map.json.efficiency_metrics`에 점수를 사람이 직접 적기만 하면, 숫자가 좋아 보여도 근거가 약하다. Evaluator는 fixture를 다시 읽어서 다음을 계산한다.
|
|
12
|
+
|
|
13
|
+
- excluded context가 `role`, `cost`, `reuse_signal`, `staleness`, `reason`을 갖는 비율.
|
|
14
|
+
- included context가 `role`과 `cost`를 갖고, high-cost 항목은 `verification_link`를 갖는 비율.
|
|
15
|
+
- included context가 `role`과 `verification_link`를 함께 갖는 비율.
|
|
16
|
+
- `verification_target` 항목이 실제 verification command나 verification hint와 연결되는 비율.
|
|
17
|
+
- 반복 path-case가 expected context role을 갖는 비율.
|
|
18
|
+
- context-diff 변화가 verification link와 metric impact로 추적되는 비율.
|
|
19
|
+
|
|
20
|
+
## 점수의 의미
|
|
21
|
+
|
|
22
|
+
`fixture-ratio-v1`에서 90% 이상이라는 말은 “예시 artifact가 내부적으로 측정 가능하고 빠진 필드가 거의 없다”는 뜻이다.
|
|
23
|
+
|
|
24
|
+
이것은 아직 “실제 모든 사용자 작업에서 90% 효율을 달성했다”는 뜻이 아니다. production telemetry나 여러 run record가 쌓이기 전까지는 scope를 `fixture`로 제한한다.
|
|
25
|
+
|
|
26
|
+
## 완료 기준
|
|
27
|
+
|
|
28
|
+
- 여섯 지표가 모두 fixture 계산 기준으로 90% 이상이다.
|
|
29
|
+
- `context-map.json.efficiency_metrics.scores[]`와 `context-metrics-evaluation.json`의 점수가 일치한다.
|
|
30
|
+
- 각 점수에는 `formula`, `numerator`, `denominator`, `evidence_refs`, `limit`가 있다.
|
|
31
|
+
- `measurement_status`는 fixture에서 계산되면 `measured`로 둘 수 있지만, 문서에는 한계를 함께 적는다.
|
|
32
|
+
- 설치된 schema는 `.tink/schemas/context-metrics-evaluation.schema.json`이다.
|
|
33
|
+
|
|
34
|
+
## 호환성 기준
|
|
35
|
+
|
|
36
|
+
- Claude Code와 Codex가 같은 artifact를 읽을 수 있어야 한다.
|
|
37
|
+
- macOS와 Windows 모두에서 `npm test`로 검증되어야 한다.
|
|
38
|
+
- 사용자 승인 없이 reusable memory, harness, rule, config를 저장하지 않는다.
|
|
39
|
+
- Sentry와 release evidence bundling은 포함하지 않는다.
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
# Context Metrics Evaluator
|
|
2
|
+
|
|
3
|
+
Context Metrics Evaluator is a test-backed way to calculate the ratios recorded by Context Budget Ledger.
|
|
4
|
+
|
|
5
|
+
It is not a new public command. It must not add a `tink index` command, watcher, generated cache, or hidden runtime index. `/tink:cast` and `$tink:cast` write `.tink/current/context-metrics-evaluation.json` as a run-state artifact, and `tests/fixtures/current-run/context-metrics-evaluation.json` plus `tests/test_templates.py` must calculate the same scores.
|
|
6
|
+
|
|
7
|
+
Korean companion: `docs/context-metrics-evaluator.ko.md`.
|
|
8
|
+
|
|
9
|
+
## Why It Exists
|
|
10
|
+
|
|
11
|
+
If `context-map.json.efficiency_metrics` is only hand-written, the numbers can look better than the evidence. The evaluator re-reads the fixtures and calculates:
|
|
12
|
+
|
|
13
|
+
- Excluded context with `role`, `cost`, `reuse_signal`, `staleness`, and `reason`.
|
|
14
|
+
- Included context with `role` and `cost`, with high-cost entries justified by `verification_link`.
|
|
15
|
+
- Included context with both `role` and `verification_link`.
|
|
16
|
+
- `verification_target` entries linked to known verification commands or hints.
|
|
17
|
+
- Repeated path-cases with expected context roles.
|
|
18
|
+
- Context-diff changes traceable through verification links and metric impacts.
|
|
19
|
+
|
|
20
|
+
## What The Score Means
|
|
21
|
+
|
|
22
|
+
In `fixture-ratio-v1`, a score at or above 90% means the example artifacts are internally measurable and have very few missing fields.
|
|
23
|
+
|
|
24
|
+
It does not mean that every real user run has reached 90% efficiency. Until production telemetry or multiple run records exist, the measurement scope stays `fixture`.
|
|
25
|
+
|
|
26
|
+
## Done Criteria
|
|
27
|
+
|
|
28
|
+
- All six metrics are at or above 90% under the fixture calculation.
|
|
29
|
+
- `context-map.json.efficiency_metrics.scores[]` matches `context-metrics-evaluation.json`.
|
|
30
|
+
- Each score has `formula`, `numerator`, `denominator`, `evidence_refs`, and `limit`.
|
|
31
|
+
- `measurement_status` may be `measured` for fixture calculations, but the docs must state the limit.
|
|
32
|
+
- The installed schema is `.tink/schemas/context-metrics-evaluation.schema.json`.
|
|
33
|
+
|
|
34
|
+
## Compatibility
|
|
35
|
+
|
|
36
|
+
- Claude Code and Codex read the same artifacts.
|
|
37
|
+
- macOS and Windows are both verified through `npm test`.
|
|
38
|
+
- Reusable memory, harness, rule, and config saves still require explicit approval.
|
|
39
|
+
- Sentry and release evidence bundling are out of scope.
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# Context Run History Rollup
|
|
2
|
+
|
|
3
|
+
Context Run History Rollup은 여러 run의 `context-metrics-evaluation.json` 점수를 묶어 90% 목표가 반복 작업에서도 유지되는지 보는 기준이다.
|
|
4
|
+
|
|
5
|
+
이 기능은 새 public command가 아니다. `tink index` 명령, watcher, generated cache, hidden runtime index를 만들지 않는다. 지금 단계에서는 `tests/fixtures/maintenance/context-metrics-rollup.json`과 `tests/test_templates.py`가 같은 rollup 점수를 계산하는지 확인한다.
|
|
6
|
+
|
|
7
|
+
영어판은 `docs/context-run-history-rollup.md`에 있다.
|
|
8
|
+
|
|
9
|
+
## 왜 필요한가
|
|
10
|
+
|
|
11
|
+
current run 하나가 90% 이상이어도 반복 작업 전체가 안정적이라고 말하기는 어렵다. Rollup은 여러 run의 점수를 모아서 다음을 확인한다.
|
|
12
|
+
|
|
13
|
+
- 각 지표의 평균 점수.
|
|
14
|
+
- 각 지표의 최저 점수.
|
|
15
|
+
- 모든 run이 여섯 지표를 빠짐없이 갖는지.
|
|
16
|
+
- 각 지표가 평균과 최저점 모두 90% 이상인지.
|
|
17
|
+
|
|
18
|
+
## 점수의 의미
|
|
19
|
+
|
|
20
|
+
`scope: "run_history"`는 여러 run record를 묶은 값이라는 뜻이다.
|
|
21
|
+
|
|
22
|
+
fixture에 있는 rollup은 production telemetry가 아니다. 실제 `.tink/runs/*` 기록이 충분히 쌓이기 전까지는 “대표 run-history fixture에서 90% 이상”이라고만 말한다.
|
|
23
|
+
|
|
24
|
+
## 완료 기준
|
|
25
|
+
|
|
26
|
+
- 여섯 지표가 모두 rollup 평균 90% 이상이다.
|
|
27
|
+
- 여섯 지표가 모두 run별 최저점 90% 이상이다.
|
|
28
|
+
- 각 score에는 `formula`, `numerator`, `denominator`, `evidence_refs`, `minimum_percent`가 있다.
|
|
29
|
+
- `limits`에 production telemetry가 아님을 명시한다.
|
|
30
|
+
|
|
31
|
+
## 호환성 기준
|
|
32
|
+
|
|
33
|
+
- Claude Code와 Codex가 같은 artifact 이름과 metric 이름을 읽을 수 있어야 한다.
|
|
34
|
+
- macOS와 Windows 모두에서 `npm test`로 검증되어야 한다.
|
|
35
|
+
- 사용자 승인 없이 reusable memory, harness, rule, config를 저장하지 않는다.
|
|
36
|
+
- Sentry와 release evidence bundling은 포함하지 않는다.
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# Context Run History Rollup
|
|
2
|
+
|
|
3
|
+
Context Run History Rollup combines multiple `context-metrics-evaluation.json` scores to check whether the 90% target holds across repeated work.
|
|
4
|
+
|
|
5
|
+
It is not a new public command. It must not add a `tink index` command, watcher, generated cache, or hidden runtime index. At this stage, `tests/fixtures/maintenance/context-metrics-rollup.json` and `tests/test_templates.py` must calculate the same rollup scores.
|
|
6
|
+
|
|
7
|
+
Korean companion: `docs/context-run-history-rollup.ko.md`.
|
|
8
|
+
|
|
9
|
+
## Why It Exists
|
|
10
|
+
|
|
11
|
+
A single current run above 90% is useful, but it does not prove repeated work is stable. The rollup combines several runs and checks:
|
|
12
|
+
|
|
13
|
+
- Average score for each metric.
|
|
14
|
+
- Minimum score for each metric.
|
|
15
|
+
- Whether every run records all six metrics.
|
|
16
|
+
- Whether both average and minimum are at or above 90%.
|
|
17
|
+
|
|
18
|
+
## What The Score Means
|
|
19
|
+
|
|
20
|
+
`scope: "run_history"` means the score combines multiple run records.
|
|
21
|
+
|
|
22
|
+
The fixture rollup is not production telemetry. Until enough real `.tink/runs/*` records exist, describe it as representative run-history fixture evidence only.
|
|
23
|
+
|
|
24
|
+
## Done Criteria
|
|
25
|
+
|
|
26
|
+
- All six metrics have rollup averages at or above 90%.
|
|
27
|
+
- All six metrics have per-run minimums at or above 90%.
|
|
28
|
+
- Each score has `formula`, `numerator`, `denominator`, `evidence_refs`, and `minimum_percent`.
|
|
29
|
+
- `limits` states that the data is not production telemetry.
|
|
30
|
+
|
|
31
|
+
## Compatibility
|
|
32
|
+
|
|
33
|
+
- Claude Code and Codex read the same artifact names and metric names.
|
|
34
|
+
- macOS and Windows are both verified through `npm test`.
|
|
35
|
+
- Reusable memory, harness, rule, and config saves still require explicit approval.
|
|
36
|
+
- Sentry and release evidence bundling are out of scope.
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# Context Run Record Policy
|
|
2
|
+
|
|
3
|
+
Context Run Record Policy는 실제 `.tink/runs/*` 기록을 나중에 rollup할 때 어떤 기록을 써도 되는지 정하는 기준이다.
|
|
4
|
+
|
|
5
|
+
이 문서는 새 public command가 아니다. `tink index` 명령, watcher, generated cache, hidden runtime index를 만들지 않는다. 자동 수집도 하지 않는다.
|
|
6
|
+
|
|
7
|
+
영어 문서는 `docs/context-run-record-policy.md`에 있다.
|
|
8
|
+
|
|
9
|
+
## 왜 필요한가
|
|
10
|
+
|
|
11
|
+
현재 90% 달성 근거는 repository fixture와 대표 run-history fixture 기준이다. 이것만으로 “모든 실제 프로젝트에서 production-wide 90%가 보장된다”고 말하면 안 된다.
|
|
12
|
+
|
|
13
|
+
실제 run 기록으로 넘어가려면 먼저 다음을 정해야 한다.
|
|
14
|
+
|
|
15
|
+
- 어떤 `.tink/runs/*` 기록을 rollup에 포함할 수 있는가.
|
|
16
|
+
- 어떤 정보는 민감하거나 너무 넓어서 제외해야 하는가.
|
|
17
|
+
- metric score가 verification evidence와 연결되어 있는가.
|
|
18
|
+
- Claude Code와 Codex, macOS와 Windows에서 같은 기준으로 검증할 수 있는가.
|
|
19
|
+
|
|
20
|
+
## 포함 가능한 기록
|
|
21
|
+
|
|
22
|
+
포함 가능한 기록은 사용자가 승인한 current-run에서 나온 완료 기록이어야 한다.
|
|
23
|
+
|
|
24
|
+
- run id 또는 run path.
|
|
25
|
+
- 완료 시각.
|
|
26
|
+
- 사용한 surface: Claude Code 또는 Codex.
|
|
27
|
+
- 사용한 platform: macOS 또는 Windows.
|
|
28
|
+
- `context-metrics-evaluation.json` 형태의 여섯 지표 점수.
|
|
29
|
+
- 검증 결과와 check 목록.
|
|
30
|
+
- 짧은 evidence handle.
|
|
31
|
+
- production telemetry인지, fixture인지, 대표 run인지에 대한 limit.
|
|
32
|
+
|
|
33
|
+
## 제외해야 할 기록
|
|
34
|
+
|
|
35
|
+
다음은 run-history rollup에 넣지 않는다.
|
|
36
|
+
|
|
37
|
+
- token, credential, raw private payload.
|
|
38
|
+
- private issue, dashboard, Figma file, discussion 전체 복사본.
|
|
39
|
+
- 별도 승인 없는 `.tink/memory/*`, `.tink/rules/*`, `.tink/harnesses/*` reusable state 변경.
|
|
40
|
+
- Sentry 연동.
|
|
41
|
+
- release evidence bundling.
|
|
42
|
+
|
|
43
|
+
## 완료 기준
|
|
44
|
+
|
|
45
|
+
- 여섯 지표가 모두 기록에 존재한다.
|
|
46
|
+
- metric score마다 근거가 있다.
|
|
47
|
+
- verification result와 checks가 연결되어 있다.
|
|
48
|
+
- limit가 production telemetry인지 아닌지 명확히 말한다.
|
|
49
|
+
- 새 public command, watcher, generated cache, hidden runtime index가 없다.
|
|
50
|
+
- macOS와 Windows 모두 `npm test`로 검증할 수 있다.
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# Context Run Record Policy
|
|
2
|
+
|
|
3
|
+
Context Run Record Policy defines which real `.tink/runs/*` records can later be used for run-history rollup.
|
|
4
|
+
|
|
5
|
+
This is not a new public command. It does not create a `tink index` command, watcher, generated cache, or hidden runtime index. It also does not collect records automatically.
|
|
6
|
+
|
|
7
|
+
Korean documentation is available in `docs/context-run-record-policy.ko.md`.
|
|
8
|
+
|
|
9
|
+
## Why This Exists
|
|
10
|
+
|
|
11
|
+
The current 90 percent evidence is based on repository fixtures and representative run-history fixtures. That is not enough to claim production-wide 90 percent behavior across every real project.
|
|
12
|
+
|
|
13
|
+
Before moving to real run records, Tink needs a clear answer to these questions:
|
|
14
|
+
|
|
15
|
+
- Which `.tink/runs/*` records can be included in a rollup?
|
|
16
|
+
- Which data is sensitive or too broad and must be excluded?
|
|
17
|
+
- Are metric scores linked to verification evidence?
|
|
18
|
+
- Can Claude Code and Codex, on macOS and Windows, verify the same criteria?
|
|
19
|
+
|
|
20
|
+
## Records That Can Be Included
|
|
21
|
+
|
|
22
|
+
Included records must come from an approved current run and represent a completed work record.
|
|
23
|
+
|
|
24
|
+
- Run id or run path.
|
|
25
|
+
- Completion timestamp.
|
|
26
|
+
- Surface: Claude Code or Codex.
|
|
27
|
+
- Platform: macOS or Windows.
|
|
28
|
+
- Six metric scores shaped like `context-metrics-evaluation.json`.
|
|
29
|
+
- Verification result and check list.
|
|
30
|
+
- Short evidence handles.
|
|
31
|
+
- Explicit limits that say whether the record is production telemetry, a fixture, or a representative run.
|
|
32
|
+
|
|
33
|
+
## Records That Must Be Excluded
|
|
34
|
+
|
|
35
|
+
Do not include these in run-history rollup:
|
|
36
|
+
|
|
37
|
+
- Tokens, credentials, or raw private payloads.
|
|
38
|
+
- Full private issue text, whole dashboards, entire Figma files, or complete discussions.
|
|
39
|
+
- Unapproved reusable state changes under `.tink/memory/*`, `.tink/rules/*`, or `.tink/harnesses/*`.
|
|
40
|
+
- Sentry integration.
|
|
41
|
+
- Release evidence bundling.
|
|
42
|
+
|
|
43
|
+
## Completion Criteria
|
|
44
|
+
|
|
45
|
+
- All six metrics are present.
|
|
46
|
+
- Each metric score has evidence.
|
|
47
|
+
- Verification result and checks are linked.
|
|
48
|
+
- Limits clearly state whether the data is production telemetry.
|
|
49
|
+
- No new public command, watcher, generated cache, or hidden runtime index exists.
|
|
50
|
+
- macOS and Windows can verify the criteria with `npm test`.
|
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
# Context Threshold Status
|
|
2
|
+
|
|
3
|
+
Context Threshold Status는 여섯 가지 컨텍스트 효율 지표가 90% 목표를 넘었는지 한눈에 확인하는 상태판이다.
|
|
4
|
+
|
|
5
|
+
현재 상태판은 `tests/fixtures/current-run/context-metrics-evaluation.json`과 `tests/fixtures/maintenance/context-metrics-rollup.json`을 함께 본다. 즉, 단일 current run fixture와 여러 대표 run을 묶은 run-history rollup이 모두 90% 이상인지 확인한다.
|
|
6
|
+
|
|
7
|
+
이 문서는 새 public command가 아니다. `tink index` 명령, watcher, generated cache, hidden runtime index를 만들지 않는다. 자동으로 사용자의 repo 데이터를 수집하지도 않는다.
|
|
8
|
+
|
|
9
|
+
영어 문서는 `docs/context-threshold-status.md`에 있다.
|
|
10
|
+
|
|
11
|
+
## 현재 상태
|
|
12
|
+
|
|
13
|
+
| 지표 | current run | rollup 평균 | rollup 최저 | 상태 |
|
|
14
|
+
| --- | ---: | ---: | ---: | --- |
|
|
15
|
+
| 불필요 context 포함률 감소 | 100% | 97% | 94% | 90% 이상 |
|
|
16
|
+
| 초기 context pack 크기 감소 | 100% | 95% | 92% | 90% 이상 |
|
|
17
|
+
| 리뷰자가 근거 찾는 시간 감소 | 100% | 98% | 96% | 90% 이상 |
|
|
18
|
+
| 검증 누락 탐지율 개선 | 100% | 99% | 98% | 90% 이상 |
|
|
19
|
+
| 반복 작업 context 재사용 정확도 | 100% | 96% | 94% | 90% 이상 |
|
|
20
|
+
| 재작업 가능성 감소 | 100% | 95% | 91% | 90% 이상 |
|
|
21
|
+
|
|
22
|
+
## 왜 필요한가
|
|
23
|
+
|
|
24
|
+
이 상태판이 없으면 90% 이상이라는 말을 어디까지 믿어도 되는지 다시 추리해야 한다.
|
|
25
|
+
|
|
26
|
+
- current run fixture는 지금 만든 artifact가 완전한지 본다.
|
|
27
|
+
- run-history rollup은 반복 작업에서도 점수가 유지되는지 본다.
|
|
28
|
+
- minimum score는 특정 작업 단위가 90% 아래로 떨어지는지 본다.
|
|
29
|
+
|
|
30
|
+
## 한계
|
|
31
|
+
|
|
32
|
+
현재 상태는 repository fixture와 대표 run-history fixture 기준이다. production telemetry가 아니다.
|
|
33
|
+
|
|
34
|
+
따라서 사용자에게 “모든 실제 프로젝트에서 90% 이상 보장”이라고 말하면 안 된다. 실제 `.tink/runs/*` 기록이 충분히 쌓이면 같은 계산식으로 다시 rollup해야 한다.
|
|
35
|
+
|
|
36
|
+
## 완료 기준
|
|
37
|
+
|
|
38
|
+
- 여섯 지표 모두 current run score가 90% 이상이다.
|
|
39
|
+
- 여섯 지표 모두 rollup 평균이 90% 이상이다.
|
|
40
|
+
- 여섯 지표 모두 rollup 최저점이 90% 이상이다.
|
|
41
|
+
- `limits`에 production telemetry가 아니라는 한계를 명시한다.
|
|
42
|
+
- Claude Code와 Codex 모두 같은 artifact 이름과 metric 이름을 읽을 수 있다.
|
|
43
|
+
- macOS와 Windows 모두 `npm test`로 검증할 수 있다.
|
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
# Context Threshold Status
|
|
2
|
+
|
|
3
|
+
Context Threshold Status is a compact status board for checking whether all six context efficiency metrics meet the 90 percent target.
|
|
4
|
+
|
|
5
|
+
The current status compares `tests/fixtures/current-run/context-metrics-evaluation.json` with `tests/fixtures/maintenance/context-metrics-rollup.json`. In plain terms, it checks both the single current-run fixture and the representative run-history rollup.
|
|
6
|
+
|
|
7
|
+
This is not a new public command. It does not create a `tink index` command, watcher, generated cache, or hidden runtime index. It also does not collect user repository data automatically.
|
|
8
|
+
|
|
9
|
+
Korean documentation is available in `docs/context-threshold-status.ko.md`.
|
|
10
|
+
|
|
11
|
+
## Current Status
|
|
12
|
+
|
|
13
|
+
| Metric | Current run | Rollup average | Rollup minimum | Status |
|
|
14
|
+
| --- | ---: | ---: | ---: | --- |
|
|
15
|
+
| unnecessary_context_reduction | 100% | 97% | 94% | >= 90% |
|
|
16
|
+
| initial_context_pack_size_reduction | 100% | 95% | 92% | >= 90% |
|
|
17
|
+
| review_evidence_lookup_time_reduction | 100% | 98% | 96% | >= 90% |
|
|
18
|
+
| verification_omission_detection | 100% | 99% | 98% | >= 90% |
|
|
19
|
+
| repeated_context_reuse_accuracy | 100% | 96% | 94% | >= 90% |
|
|
20
|
+
| rework_probability_reduction | 100% | 95% | 91% | >= 90% |
|
|
21
|
+
|
|
22
|
+
## Why This Exists
|
|
23
|
+
|
|
24
|
+
Without this status board, a reviewer has to infer what the 90 percent claim means from several separate files.
|
|
25
|
+
|
|
26
|
+
- The current-run fixture checks whether the latest artifact shape is complete.
|
|
27
|
+
- The run-history rollup checks whether repeated work stays above the target.
|
|
28
|
+
- The minimum score catches a single work unit falling below 90 percent.
|
|
29
|
+
|
|
30
|
+
## Limits
|
|
31
|
+
|
|
32
|
+
This status is based on repository fixtures and representative run-history fixtures. It is not production telemetry.
|
|
33
|
+
|
|
34
|
+
Do not claim production-wide 90 percent performance until enough real `.tink/runs/*` records are accumulated and rolled up with the same formulas.
|
|
35
|
+
|
|
36
|
+
## Completion Criteria
|
|
37
|
+
|
|
38
|
+
- All six current-run scores are at least 90 percent.
|
|
39
|
+
- All six rollup averages are at least 90 percent.
|
|
40
|
+
- All six rollup minimums are at least 90 percent.
|
|
41
|
+
- `limits` clearly states that this is not production telemetry.
|
|
42
|
+
- Claude Code and Codex can read the same artifact names and metric names.
|
|
43
|
+
- macOS and Windows can verify the status with `npm test`.
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
# Codex-only update에서 repo-local Claude surface 정리
|
|
2
|
+
|
|
3
|
+
## 문제
|
|
4
|
+
|
|
5
|
+
다른 repo에서 `npx tink-harness@latest update` 후 Codex skill picker에 `Source Command Tink Frog/List/...`와 넓은 `Tink` 항목이 보일 수 있었음.
|
|
6
|
+
|
|
7
|
+
이 항목들은 Codex action skill이 아니라 repo-local `.claude/commands/tink/*.md`와 `.claude/skills/tink/SKILL.md`가 Codex에 함께 노출된 것이었음. 사용자는 Codex용 `$tink:*` action skill만 기대하기 때문에 설치가 잘못된 것처럼 보일 수 있었음.
|
|
8
|
+
|
|
9
|
+
## 해결
|
|
10
|
+
|
|
11
|
+
- Codex-only install/update에서는 repo-local Tink Claude command와 Claude skill을 정리하도록 installer를 보강했음.
|
|
12
|
+
- `tink-cast`, `tink-verify`, `tink-list`, `tink-frog`, `tink-weave`, `tink-setup`, `tink-update` Codex action skill은 그대로 유지함.
|
|
13
|
+
- Claude Code와 Codex를 둘 다 선택한 `all` surface에서는 repo-local Claude command가 남을 수 있음을 troubleshooting 문서에 명확히 적었음.
|
|
14
|
+
- update verification recipe에 Codex-only 기대 상태를 추가했음.
|
|
15
|
+
|
|
16
|
+
## 검증
|
|
17
|
+
|
|
18
|
+
- `npm test`
|
|
19
|
+
- `git diff --check`
|
|
20
|
+
- `npm pack --dry-run --json`
|
|
21
|
+
|
|
22
|
+
## 참고
|
|
23
|
+
|
|
24
|
+
- 새 public command를 추가하지 않았음.
|
|
25
|
+
- `tink index`, watcher, generated cache, hidden runtime index를 추가하지 않았음.
|
|
26
|
+
- Sentry와 release evidence bundling은 포함하지 않았음.
|
|
27
|
+
- 다른 repo에서 바로 적용하려면 npm publish가 필요함.
|
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
# Context Metrics Artifact
|
|
2
|
+
|
|
3
|
+
## 문제
|
|
4
|
+
|
|
5
|
+
Context Metrics Evaluator가 fixture 기준으로 여섯 지표를 계산할 수 있게 되었지만, 실제 `/tink:cast`와 `$tink:cast` run state에는 아직 `context-metrics-evaluation.json`이 공식 산출물로 연결되어 있지 않았다. 이 상태에서는 90% 목표를 current run마다 반복 검증하기 어렵다.
|
|
6
|
+
|
|
7
|
+
## 해결
|
|
8
|
+
|
|
9
|
+
- `templates/tink/schemas/context-metrics-evaluation.schema.json`을 추가했다.
|
|
10
|
+
- `/tink:cast`와 `$tink:cast` 지침에 `.tink/current/context-metrics-evaluation.json` 생성 규칙을 추가했다.
|
|
11
|
+
- Work State Guide에 metrics evaluation 읽기 순서를 추가했다.
|
|
12
|
+
- Context Metrics Evaluator 문서를 run-state artifact 기준으로 갱신했다.
|
|
13
|
+
- `v1.4.0` 릴리즈 메타데이터, README, CHANGELOG, VERSIONING을 갱신했다.
|
|
14
|
+
|
|
15
|
+
## 검증
|
|
16
|
+
|
|
17
|
+
- `npm test`
|
|
18
|
+
- `git diff --check`
|
|
19
|
+
- `claude plugin validate .claude-plugin/plugin.json`
|
|
20
|
+
- `claude plugin validate .claude-plugin/marketplace.json`
|
|
21
|
+
- `npm pack --dry-run --json`
|
|
22
|
+
|
|
23
|
+
## 참고
|
|
24
|
+
|
|
25
|
+
- 새 public command는 추가하지 않았다.
|
|
26
|
+
- `tink index` 명령, watcher, generated cache, hidden runtime index를 만들지 않았다.
|
|
27
|
+
- Sentry는 포함하지 않았다.
|
|
28
|
+
- release evidence bundling은 포함하지 않았다.
|
|
29
|
+
- 점수의 scope와 limit를 명시해야 하며, run-history나 telemetry 없이 production-wide 90%를 주장하지 않는다.
|
|
30
|
+
- Claude Code와 Codex, macOS와 Windows 동시 지원을 기준으로 작성했다.
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
# Context Metrics Evaluator
|
|
2
|
+
|
|
3
|
+
## 문제
|
|
4
|
+
|
|
5
|
+
Context Budget Ledger로 context entry에 역할, 비용, 재사용 신호, 검증 연결을 남길 수 있게 되었지만, `efficiency_metrics` 점수는 아직 사람이 적은 값에 가까웠다. 목표 지표를 90% 이상으로 반복 개선하려면 점수가 실제 fixture에서 다시 계산되어야 한다.
|
|
6
|
+
|
|
7
|
+
## 해결
|
|
8
|
+
|
|
9
|
+
- `tests/fixtures/current-run/context-metrics-evaluation.json`을 추가해 여섯 지표의 계산식, 분자, 분모, evidence ref를 기록했다.
|
|
10
|
+
- `context-map.json.efficiency_metrics`를 `fixture-ratio-v1` 기준의 `measured` 점수로 갱신했다.
|
|
11
|
+
- `tests/test_templates.py`가 context-map, context-diff, contract, repo signal, path-case fixture를 다시 읽어 점수를 직접 계산하도록 했다.
|
|
12
|
+
- 반복 path-case에 expected context role을 보강해 재사용 정확도도 계산할 수 있게 했다.
|
|
13
|
+
- 한국어 우선 문서 `docs/context-metrics-evaluator.ko.md`와 영어 companion을 추가했다.
|
|
14
|
+
|
|
15
|
+
## 검증
|
|
16
|
+
|
|
17
|
+
- `npm test`
|
|
18
|
+
- `git diff --check`
|
|
19
|
+
- `npm pack --dry-run --json`
|
|
20
|
+
|
|
21
|
+
## 참고
|
|
22
|
+
|
|
23
|
+
- 새 public command는 추가하지 않았다.
|
|
24
|
+
- `tink index` 명령, watcher, generated cache, hidden runtime index를 만들지 않았다.
|
|
25
|
+
- Sentry는 포함하지 않았다.
|
|
26
|
+
- release evidence bundling은 포함하지 않았다.
|
|
27
|
+
- 이번 점수는 fixture scope의 측정값이며, production telemetry 전체가 90%에 도달했다는 뜻은 아니다.
|
|
28
|
+
- Claude Code와 Codex, macOS와 Windows 동시 지원 기준을 유지했다.
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
# Context Run History Rollup
|
|
2
|
+
|
|
3
|
+
## 문제
|
|
4
|
+
|
|
5
|
+
`context-metrics-evaluation.json`으로 current run 하나의 여섯 지표는 계산할 수 있게 되었지만, 반복 작업 전체가 90% 이상을 유지하는지는 아직 확인하기 어려웠다.
|
|
6
|
+
|
|
7
|
+
## 해결
|
|
8
|
+
|
|
9
|
+
- `tests/fixtures/maintenance/context-metrics-rollup.json`을 추가해 여러 run의 점수를 묶었다.
|
|
10
|
+
- 테스트가 rollup 평균, 최저점, evidence ref, metric 누락 여부를 직접 계산하도록 했다.
|
|
11
|
+
- 한국어 우선 문서 `docs/context-run-history-rollup.ko.md`와 영어 companion을 추가했다.
|
|
12
|
+
- README에는 링크만 추가해 본문을 늘리지 않았다.
|
|
13
|
+
|
|
14
|
+
## 검증
|
|
15
|
+
|
|
16
|
+
- `npm test`
|
|
17
|
+
- `git diff --check`
|
|
18
|
+
- `npm pack --dry-run --json`
|
|
19
|
+
|
|
20
|
+
## 참고
|
|
21
|
+
|
|
22
|
+
- 새 public command는 추가하지 않았다.
|
|
23
|
+
- `tink index` 명령, watcher, generated cache, hidden runtime index를 만들지 않았다.
|
|
24
|
+
- Sentry는 포함하지 않았다.
|
|
25
|
+
- release evidence bundling은 포함하지 않았다.
|
|
26
|
+
- 이번 값은 대표 run-history fixture 기준이며, production telemetry 전체가 90%에 도달했다는 뜻은 아니다.
|
|
27
|
+
- Claude Code와 Codex, macOS와 Windows 동시 지원 기준을 유지했다.
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
# Context Run Record Policy로 실제 run rollup 전 안전 기준 추가
|
|
2
|
+
|
|
3
|
+
## 문제
|
|
4
|
+
|
|
5
|
+
현재 90% 달성 근거는 current-run fixture와 대표 run-history fixture까지 확장되었지만, 실제 `.tink/runs/*` 기록을 어떤 기준으로 rollup에 포함할지는 아직 명확하지 않다. 이 기준 없이 자동 수집이나 hidden cache부터 만들면 사용자 영역을 침범할 수 있다.
|
|
6
|
+
|
|
7
|
+
## 해결
|
|
8
|
+
|
|
9
|
+
- `tests/fixtures/maintenance/context-run-record-policy.json`에 실제 run 기록으로 넘어가기 전 필요한 포함/제외 기준을 정의했다.
|
|
10
|
+
- `docs/context-run-record-policy.ko.md`와 영어 companion 문서를 추가했다.
|
|
11
|
+
- `tests/test_templates.py`가 새 정책 fixture에서 public command, watcher, hidden runtime index, generated cache, Sentry가 제외되어 있는지 검증하게 했다.
|
|
12
|
+
- README에는 관련 문서 링크만 추가했다.
|
|
13
|
+
|
|
14
|
+
## 검증
|
|
15
|
+
|
|
16
|
+
- `npm test`
|
|
17
|
+
- `git diff --check`
|
|
18
|
+
- `npm pack --dry-run --json`
|
|
19
|
+
|
|
20
|
+
## 참고
|
|
21
|
+
|
|
22
|
+
- 새 public command를 추가하지 않았다.
|
|
23
|
+
- `tink index`, watcher, generated cache, hidden runtime index를 추가하지 않았다.
|
|
24
|
+
- 실제 run 기록을 자동 수집하지 않는다.
|
|
25
|
+
- 사용자에게 보이는 설치/명령/스키마 동작 변경이 아니므로 version bump, npm publish, GitHub Release는 진행하지 않는다.
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
# Context Threshold Status로 90% 달성 근거 정리
|
|
2
|
+
|
|
3
|
+
## 문제
|
|
4
|
+
|
|
5
|
+
current-run fixture와 run-history rollup은 각각 존재하지만, 여섯 지표가 90% 이상인지 한곳에서 보기 어렵다. 리뷰자는 current run 점수, rollup 평균, rollup 최저점을 다시 대조해야 한다.
|
|
6
|
+
|
|
7
|
+
## 해결
|
|
8
|
+
|
|
9
|
+
- `tests/fixtures/maintenance/context-threshold-status.json`에 여섯 지표의 current-run 점수, rollup 평균, rollup 최저점을 함께 기록했다.
|
|
10
|
+
- `tests/test_templates.py`가 이 상태판을 기존 fixture 두 개에서 다시 계산해 검증하도록 했다.
|
|
11
|
+
- `docs/context-threshold-status.ko.md`와 영어 companion 문서를 추가해 90% 달성 범위와 한계를 설명했다.
|
|
12
|
+
- README에는 본문을 늘리지 않고 관련 문서 링크만 추가했다.
|
|
13
|
+
|
|
14
|
+
## 검증
|
|
15
|
+
|
|
16
|
+
- `npm test`
|
|
17
|
+
- `git diff --check`
|
|
18
|
+
- `npm pack --dry-run --json`
|
|
19
|
+
|
|
20
|
+
## 참고
|
|
21
|
+
|
|
22
|
+
- 새 public command를 추가하지 않았다.
|
|
23
|
+
- `tink index`, watcher, generated cache, hidden runtime index를 추가하지 않았다.
|
|
24
|
+
- Sentry와 release evidence bundling은 포함하지 않았다.
|
|
25
|
+
- 현재 근거는 production telemetry가 아니라 repository fixture와 대표 run-history fixture다.
|
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
# v1.5.0 릴리스
|
|
2
|
+
|
|
3
|
+
## 문제
|
|
4
|
+
|
|
5
|
+
다른 repo에서 `npx tink-harness@latest update`를 실행한 뒤 Codex skill picker에 `Source Command Tink Frog/List/...`와 넓은 `Tink` 항목이 같이 보이는 문제가 있었음.
|
|
6
|
+
|
|
7
|
+
이 항목들은 새 Codex `$tink:*` action skill이 아니라 repo-local `.claude/commands/tink/*.md`와 `.claude/skills/tink/SKILL.md`가 Codex에 함께 노출된 것이었음. Codex-only 사용자는 `$tink:cast`, `$tink:verify`처럼 필요한 action skill만 보이길 기대하므로 update 결과가 낡거나 잘못된 것처럼 보였음.
|
|
8
|
+
|
|
9
|
+
## 해결
|
|
10
|
+
|
|
11
|
+
- Codex-only install/update에서 repo-local Tink Claude Code command 파일을 정리하도록 했음.
|
|
12
|
+
- Codex-only install/update에서 알려진 생성물 형태의 repo-local `.claude/skills/tink/SKILL.md`도 정리하도록 했음.
|
|
13
|
+
- 사용자 작성 가능성이 있는 알 수 없는 `.claude/skills/tink` 내용은 지우지 않도록 보수적으로 처리했음.
|
|
14
|
+
- Claude Code와 Codex를 둘 다 설치한 경우에는 repo-local Claude command가 남을 수 있음을 troubleshooting 문서에 명확히 적었음.
|
|
15
|
+
- README와 한국어 README에 v1.5.0 최신 릴리스 요약을 반영했음.
|
|
16
|
+
|
|
17
|
+
## 검증
|
|
18
|
+
|
|
19
|
+
- `npm test`
|
|
20
|
+
- `git diff --check`
|
|
21
|
+
- `npm pack --dry-run --json`
|
|
22
|
+
- temp repo에서 Codex-only update smoke 검증
|
|
23
|
+
- GitHub Actions CI 확인 예정
|
|
24
|
+
|
|
25
|
+
## 참고
|
|
26
|
+
|
|
27
|
+
- 새 public command를 추가하지 않았음.
|
|
28
|
+
- `tink index`, watcher, generated cache, hidden runtime index를 추가하지 않았음.
|
|
29
|
+
- Sentry나 release evidence bundling은 포함하지 않았음.
|
|
30
|
+
- 이 수정은 다른 repo에서 `npx tink-harness@latest update`로 받아야 하므로 npm publish가 필요함.
|
|
@@ -19,13 +19,15 @@ README에는 긴 문제 해결 내용을 넣지 않는다. 평소에는 설치
|
|
|
19
19
|
가능한 원인:
|
|
20
20
|
|
|
21
21
|
- 이전 버전의 넓은 Codex `skills/tink/SKILL.md`가 남아 있다.
|
|
22
|
+
- repo-local Claude Code 파일인 `.claude/commands/tink/*.md`가 Codex에서 `Source Command Tink ...`로 보이고 있다.
|
|
23
|
+
- repo-local Claude Code skill인 `.claude/skills/tink/SKILL.md`가 Codex에서 넓은 `Tink` skill로 보이고 있다.
|
|
22
24
|
- `CODEX_HOME`이 예상한 위치와 다르다.
|
|
23
25
|
- Codex UI가 아직 skill 목록을 새로 읽지 않았다.
|
|
24
26
|
|
|
25
27
|
확인:
|
|
26
28
|
|
|
27
29
|
```bash
|
|
28
|
-
npx tink-harness@latest update --yes
|
|
30
|
+
TINK_INSTALL_SURFACES=codex npx tink-harness@latest update --yes
|
|
29
31
|
```
|
|
30
32
|
|
|
31
33
|
그 다음 `Update Result Summary`에서 다음을 본다.
|
|
@@ -37,9 +39,12 @@ npx tink-harness@latest update --yes
|
|
|
37
39
|
기대 상태:
|
|
38
40
|
|
|
39
41
|
- `skills/tink`는 없어야 한다.
|
|
42
|
+
- Codex-only update에서는 `.claude/commands/tink/*.md`와 `.claude/skills/tink/SKILL.md`가 없어야 한다.
|
|
40
43
|
- `tink-cast`, `tink-verify`, `tink-list`, `tink-frog`, `tink-weave`, `tink-setup`, `tink-update`가 있어야 한다.
|
|
41
44
|
- `tink-core/RULES.md`는 내부 공유 규칙으로 존재하지만 picker에 action으로 보이지 않는 것이 정상이다.
|
|
42
45
|
|
|
46
|
+
Claude Code와 Codex를 둘 다 선택해서 설치한 경우에는 repo-local Claude command가 남을 수 있다. 이때 `Source Command Tink ...` 항목은 repo에 Claude Code command가 설치되어 있기 때문에 보이는 것이다.
|
|
47
|
+
|
|
43
48
|
## schema files가 없을 때
|
|
44
49
|
|
|
45
50
|
가능한 원인:
|
|
@@ -19,13 +19,15 @@ After `tink-harness update`, read `Update Result Summary`.
|
|
|
19
19
|
Possible causes:
|
|
20
20
|
|
|
21
21
|
- The old broad Codex `skills/tink/SKILL.md` is still present.
|
|
22
|
+
- Repo-local Claude Code files such as `.claude/commands/tink/*.md` are being shown by Codex as `Source Command Tink ...`.
|
|
23
|
+
- Repo-local Claude Code skill `.claude/skills/tink/SKILL.md` is being shown by Codex as a broad `Tink` skill.
|
|
22
24
|
- `CODEX_HOME` points somewhere unexpected.
|
|
23
25
|
- Codex has not refreshed the skill list yet.
|
|
24
26
|
|
|
25
27
|
Run:
|
|
26
28
|
|
|
27
29
|
```bash
|
|
28
|
-
npx tink-harness@latest update --yes
|
|
30
|
+
TINK_INSTALL_SURFACES=codex npx tink-harness@latest update --yes
|
|
29
31
|
```
|
|
30
32
|
|
|
31
33
|
Then check `Codex target`, `Removed legacy paths`, and `Codex skills` in the summary.
|
|
@@ -33,9 +35,12 @@ Then check `Codex target`, `Removed legacy paths`, and `Codex skills` in the sum
|
|
|
33
35
|
Expected state:
|
|
34
36
|
|
|
35
37
|
- `skills/tink` is gone.
|
|
38
|
+
- `.claude/commands/tink/*.md` and `.claude/skills/tink/SKILL.md` are gone for a Codex-only update.
|
|
36
39
|
- `tink-cast`, `tink-verify`, `tink-list`, `tink-frog`, `tink-weave`, `tink-setup`, and `tink-update` exist.
|
|
37
40
|
- `tink-core/RULES.md` exists as shared internal rules, but should not appear as a user action in the picker.
|
|
38
41
|
|
|
42
|
+
If you intentionally selected both Claude Code and Codex, repo-local Claude commands may remain. In that case `Source Command Tink ...` entries are expected because the repo still has Claude Code commands installed.
|
|
43
|
+
|
|
39
44
|
## Schema Files Are Missing
|
|
40
45
|
|
|
41
46
|
Check that `Install target` is the repo you meant to update.
|
|
@@ -49,6 +49,7 @@ Codex surface를 갱신했다면 다음이 있어야 한다.
|
|
|
49
49
|
|
|
50
50
|
- `skills/tink`는 제거되어야 한다.
|
|
51
51
|
- picker에는 action skill이 보이고, `tink-core`는 공유 규칙으로만 쓰인다.
|
|
52
|
+
- Codex-only update에서는 repo-local `.claude/commands/tink/*.md`와 `.claude/skills/tink/SKILL.md`가 제거되어 Codex에 `Source Command Tink ...` 또는 넓은 repo-local `Tink` 항목이 보이지 않아야 한다.
|
|
52
53
|
|
|
53
54
|
### 3. Claude Code command 확인
|
|
54
55
|
|
|
@@ -64,6 +65,8 @@ Claude Code surface를 갱신했다면 다음 command가 있어야 한다.
|
|
|
64
65
|
|
|
65
66
|
repo 개발 중에는 `npm test`가 3-copy sync를 확인한다.
|
|
66
67
|
|
|
68
|
+
Claude Code와 Codex를 둘 다 갱신했다면 repo-local Claude command가 Codex에서 `Source Command Tink ...`로 보일 수 있다. 이것은 `all` surface에서는 예상 가능한 상태이며, Codex-only update에서는 피한다.
|
|
69
|
+
|
|
67
70
|
### 4. Schema 확인
|
|
68
71
|
|
|
69
72
|
다음 파일이 있어야 한다.
|
|
@@ -49,6 +49,7 @@ Expected state:
|
|
|
49
49
|
|
|
50
50
|
- `skills/tink` is removed.
|
|
51
51
|
- The picker shows action skills.
|
|
52
|
+
- For a Codex-only update, repo-local `.claude/commands/tink/*.md` and `.claude/skills/tink/SKILL.md` are removed so Codex does not show `Source Command Tink ...` or a broad repo-local `Tink` entry.
|
|
52
53
|
- `tink-core` is shared internal guidance, not a user-facing action.
|
|
53
54
|
|
|
54
55
|
### 3. Claude Code Commands
|
|
@@ -65,6 +66,8 @@ For Claude Code, these commands should exist:
|
|
|
65
66
|
|
|
66
67
|
When developing Tink itself, `npm test` verifies the 3-copy command sync rule.
|
|
67
68
|
|
|
69
|
+
If you intentionally refreshed both Claude Code and Codex, these repo-local Claude commands may also appear in Codex as `Source Command Tink ...`. That is expected for the `all` surface and avoided by a Codex-only update.
|
|
70
|
+
|
|
68
71
|
### 4. Schemas
|
|
69
72
|
|
|
70
73
|
These files should exist:
|
package/docs/work-state.ko.md
CHANGED
|
@@ -21,11 +21,13 @@ Tink는 실행 상태를 파일로 남겨서 사람이 빠르게 네 가지를
|
|
|
21
21
|
- 선택된 파일, 문서, 규칙, 외부 source의 사람용 요약을 읽습니다.
|
|
22
22
|
3. `.tink/current/context-map.json`
|
|
23
23
|
- `included`, `excluded`, `signals`, `external_context`를 구조적으로 확인합니다.
|
|
24
|
-
4. `.tink/current/
|
|
24
|
+
4. `.tink/current/context-metrics-evaluation.json`
|
|
25
|
+
- context 효율 점수, 계산식, 분자/분모, evidence ref, 측정 scope와 한계를 확인합니다.
|
|
26
|
+
5. `.tink/current/excluded-context.md`
|
|
25
27
|
- 오래됐거나, 위험하거나, 너무 넓거나, 접근할 수 없거나, 범위 밖이라 제외한 context를 확인합니다.
|
|
26
|
-
|
|
28
|
+
6. `.tink/current/verification.json`
|
|
27
29
|
- pass, fail, blocked, skipped 검증 결과와 최종 report를 확인합니다.
|
|
28
|
-
|
|
30
|
+
7. `.tink/current/notes.md`
|
|
29
31
|
- 마지막 안전 지점, 복구 메모, 짧은 검증 요약을 읽습니다.
|
|
30
32
|
|
|
31
33
|
## Context 읽는 법
|
|
@@ -50,6 +52,8 @@ Tink는 실행 상태를 파일로 남겨서 사람이 빠르게 네 가지를
|
|
|
50
52
|
|
|
51
53
|
자세한 기준은 `docs/context-budget-ledger.ko.md`를 봅니다.
|
|
52
54
|
|
|
55
|
+
`context-metrics-evaluation.json`은 점수가 어떻게 나왔는지 보여줍니다. `scope: "fixture"`나 `scope: "current_run"`이면 해당 범위 안에서만 측정된 값입니다. 여러 run record나 production telemetry가 없으면 전체 사용자 작업이 90%에 도달했다고 말하지 않습니다.
|
|
56
|
+
|
|
53
57
|
`signals[]`에 `kind: "context_graph_rule"`가 있으면 `/tink:cast`나 `$tink:cast`가 changed path를 보고 고른 작은 단서로 읽습니다. `context_graph_lite.rules.claude-command-sync` 같은 안정적인 `source_ref`를 가리키고, 왜 관련 파일을 함께 포함했는지 설명해야 합니다. 이 신호는 cast 내부 선택 근거일 뿐이며 public `tink index` 명령, watcher, generated cache, hidden runtime index를 뜻하지 않습니다.
|
|
54
58
|
|
|
55
59
|
외부 context는 다음 항목을 확인합니다.
|
package/docs/work-state.md
CHANGED
|
@@ -19,11 +19,13 @@ Start here when resuming, reviewing, or handing off a run:
|
|
|
19
19
|
- Read the short human summary of selected files, docs, rules, and external sources.
|
|
20
20
|
3. `.tink/current/context-map.json`
|
|
21
21
|
- Inspect the structured `included`, `excluded`, `signals`, and `external_context` entries.
|
|
22
|
-
4. `.tink/current/
|
|
22
|
+
4. `.tink/current/context-metrics-evaluation.json`
|
|
23
|
+
- Check context-efficiency scores, formulas, numerators, denominators, evidence refs, measurement scope, and limits.
|
|
24
|
+
5. `.tink/current/excluded-context.md`
|
|
23
25
|
- Check what was skipped because it was stale, unsafe, too broad, unavailable, or outside scope.
|
|
24
|
-
|
|
26
|
+
6. `.tink/current/verification.json`
|
|
25
27
|
- Confirm pass, fail, blocked, or skipped checks and the final report.
|
|
26
|
-
|
|
28
|
+
7. `.tink/current/notes.md`
|
|
27
29
|
- Read the last safe point, recovery notes, and compact verification summaries.
|
|
28
30
|
|
|
29
31
|
## How To Read Context
|
|
@@ -48,6 +50,8 @@ When context entries include Context Budget Ledger fields, read them this way:
|
|
|
48
50
|
|
|
49
51
|
See `docs/context-budget-ledger.md` for the detailed rules.
|
|
50
52
|
|
|
53
|
+
`context-metrics-evaluation.json` explains how the score was produced. If `scope` is `fixture` or `current_run`, the value is measured only inside that boundary. Do not claim all user work has reached 90% without run-history or production telemetry evidence.
|
|
54
|
+
|
|
51
55
|
When `signals[]` includes `kind: "context_graph_rule"`, read it as a small changed-path clue selected by `/tink:cast` or `$tink:cast`. It should point to a stable `source_ref` such as `context_graph_lite.rules.claude-command-sync`, explain why related files were included, and stay internal to cast. It must not imply a public `tink index` command, watcher, generated cache, or hidden runtime index.
|
|
52
56
|
|
|
53
57
|
For external context, check:
|
package/package.json
CHANGED
|
@@ -144,6 +144,7 @@ After approval, create `.tink/current/` with these files before doing deeper wor
|
|
|
144
144
|
- `session.json`: lightweight session metadata, especially rule ids already loaded by phase
|
|
145
145
|
- `context-pack.md`: human-readable selected context, including why each item is relevant
|
|
146
146
|
- `context-map.json`: machine-readable included and excluded context with reasons
|
|
147
|
+
- `context-metrics-evaluation.json`: measured or estimated context-efficiency scores, formulas, evidence refs, and limits
|
|
147
148
|
- `excluded-context.md`: notable omitted files, tools, sources, or claims and why they were excluded
|
|
148
149
|
|
|
149
150
|
Create `contract.json` before loading harness bodies. It should be short, factual, and based on the user request plus visible project context:
|
|
@@ -189,9 +190,11 @@ Create context artifacts before deeper implementation work:
|
|
|
189
190
|
- `context-pack.md` should name the user task, selected harnesses, contract summary, loaded rules, selected files/docs, selected external sources, and verification implications.
|
|
190
191
|
- `context-map.json` should contain `task`, `included`, `excluded`, `signals`, and `generated_at`. Each included or excluded entry should include `path` or `source`, `kind`, `reason`, and `confidence`. When external context is selected, also write `external_context[]`.
|
|
191
192
|
- When useful, enrich each context entry with Context Budget Ledger fields: `role`, `cost`, `reuse_signal`, `verification_link`, `staleness`, and `evidence_kind`. Use them to explain why the first context pack is small enough, why excluded context should stay out, and which checks prove selected context matters.
|
|
193
|
+
- `context-metrics-evaluation.json` should contain `run`, `evaluator`, `target_threshold_percent`, `measurement_status`, `scope`, `limits`, and `scores[]`. Each score should include `name`, `score_percent`, `formula`, `numerator`, `denominator`, and `evidence_refs`. If the score is based only on fixtures or the current run, say so in `scope` and `limits`; do not claim production-wide 90% without run-history or telemetry evidence.
|
|
192
194
|
- `excluded-context.md` should make important omissions visible, especially files skipped because they are out of scope, stale, risky, too broad, or unverified external claims.
|
|
193
195
|
|
|
194
196
|
If `.tink/schemas/context-map.schema.json` exists, use it for `context-map.json`. Do not paste the schema into the user response.
|
|
197
|
+
If `.tink/schemas/context-metrics-evaluation.schema.json` exists, use it for `context-metrics-evaluation.json`. Do not paste the schema into the user response.
|
|
195
198
|
|
|
196
199
|
Use deterministic context selection inside cast. Do not create or require a separate `tink index` command for this phase.
|
|
197
200
|
|
|
@@ -401,7 +404,7 @@ A task is trivial only when ALL of the following are true:
|
|
|
401
404
|
15. Run Stitch once before committing to `.tink/current/`. If it triggers, show exactly one proposal before approval. Call `AskUserQuestion` as described in the Interaction policy section.
|
|
402
405
|
16. Ask for explicit approval before non-trivial work.
|
|
403
406
|
17. After approval, read only the selected harness files and any approved run-only draft.
|
|
404
|
-
18. Create `.tink/current/` files from the run state contract, including `contract.json`, `session.json`, `context-pack.md`, `context-map.json`, and `excluded-context.md`.
|
|
407
|
+
18. Create `.tink/current/` files from the run state contract, including `contract.json`, `session.json`, `context-pack.md`, `context-map.json`, `context-metrics-evaluation.json`, and `excluded-context.md`.
|
|
405
408
|
19. Execute the first safe step immediately:
|
|
406
409
|
- inspect relevant files,
|
|
407
410
|
- run a read-only diagnostic,
|
|
@@ -31,7 +31,7 @@ Accept legacy `$tink <action>` spelling for compatibility, but present `$tink:<a
|
|
|
31
31
|
11. Use `request_user_input` for choice prompts when available. Otherwise stop and ask one concise blocking approval question directly in chat. Do not continue until the user answers.
|
|
32
32
|
12. Treat reusable saves as a separate hard approval gate for `.tink/memory/*`, `.tink/harnesses/*`, `.tink/rules/*`, `.tink/config.json`, Codex skill files, and template/plugin files that affect future installs.
|
|
33
33
|
13. Current-run approval never authorizes reusable-state writes. Before saving reusable state, show operation, destination files, exact entry or patch summary, reusable reason, sensitive content excluded, and rollback/removal path.
|
|
34
|
-
14. After approval, create `.tink/current/plan.md`, `checks.md`, `steps.json`, `notes.md`, `answers.md`, `contract.json`, `session.json`, `context-pack.md`, `context-map.json`, and `excluded-context.md`.
|
|
34
|
+
14. After approval, create `.tink/current/plan.md`, `checks.md`, `steps.json`, `notes.md`, `answers.md`, `contract.json`, `session.json`, `context-pack.md`, `context-map.json`, `context-metrics-evaluation.json`, and `excluded-context.md`.
|
|
35
35
|
15. Do not stop at recommendation. Execute the first safe step after run state exists.
|
|
36
36
|
16. Run `$tink:verify` behavior before final when `contract.json` lists required checks.
|
|
37
37
|
17. Store reusable memory or rule updates under `.tink/` only after separate approval.
|
|
@@ -102,10 +102,13 @@ Create run state before deeper work:
|
|
|
102
102
|
- `session.json`: loaded rule ids by phase and lightweight retrieval metadata
|
|
103
103
|
- `.tink/current/context-pack.md`: human-readable selected context and why it matters
|
|
104
104
|
- `.tink/current/context-map.json`: machine-readable included/excluded context and reasons
|
|
105
|
+
- `.tink/current/context-metrics-evaluation.json`: measured or estimated context-efficiency scores, formulas, evidence refs, and limits
|
|
105
106
|
- `.tink/current/excluded-context.md`: notable omitted context and why it was left out
|
|
106
107
|
|
|
107
108
|
When useful, enrich `context-map.json.included[]` and `context-map.json.excluded[]` entries with Context Budget Ledger fields: `role`, `cost`, `reuse_signal`, `verification_link`, `staleness`, and `evidence_kind`. Use them to keep the first context pack small, mark stale or avoid-next-time context, and connect `verification_target` entries to command checks, manual checks, evidence refs, or verification hints. Do not claim any 90% efficiency score without measurement evidence.
|
|
108
109
|
|
|
110
|
+
When writing `context-metrics-evaluation.json`, include `run`, `evaluator`, `target_threshold_percent`, `measurement_status`, `scope`, `limits`, and `scores[]`. Each score should include `name`, `score_percent`, `formula`, `numerator`, `denominator`, and `evidence_refs`. If the score comes only from fixture or current-run evidence, record that scope and do not claim production-wide 90% without run-history or telemetry evidence.
|
|
111
|
+
|
|
109
112
|
When external context is needed for `$tink:cast`, write it through the MCP Safe Profile shape in `context-map.json.external_context[]`. Record `source`, `source_ref`, `kind`, `included`, `excluded`, `reason`, `confidence`, `sensitivity`, and `verification_hint` when useful. Treat Figma, GitHub, and official docs as representative examples, not the only supported sources; Linear, Jira, Supabase, dashboards, API responses, screenshots, attachments, and runbooks can follow the same shape.
|
|
110
113
|
|
|
111
114
|
When repo signal fixtures contain `context_graph_lite.rules[]`, use those rules inside `$tink:cast` to choose the first related context candidates. Match changed paths against `when_paths`, consider `include_paths`, cite selected rules as `context_graph_rule` signals with `source_ref: "context_graph_lite.rules.<name>"`, and connect `signal_refs` to verification hints where relevant. If the fixture provides `context_budget_policy`, use it to assign roles, costs, reuse signals, verification links, staleness, and evidence kinds. Do not create a public `tink index` command, watch process, generated cache, or hidden runtime index.
|
|
@@ -0,0 +1,89 @@
|
|
|
1
|
+
{
|
|
2
|
+
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
|
3
|
+
"$id": "https://github.com/dotoricode/tink-harness/schemas/context-metrics-evaluation.schema.json",
|
|
4
|
+
"title": "Tink current run context metrics evaluation",
|
|
5
|
+
"type": "object",
|
|
6
|
+
"required": ["run", "evaluator", "target_threshold_percent", "measurement_status", "scope", "scores"],
|
|
7
|
+
"properties": {
|
|
8
|
+
"run": {
|
|
9
|
+
"type": "string",
|
|
10
|
+
"description": "Run state path, usually .tink/current."
|
|
11
|
+
},
|
|
12
|
+
"evaluator": {
|
|
13
|
+
"type": "string",
|
|
14
|
+
"description": "Stable evaluator id, such as fixture-ratio-v1."
|
|
15
|
+
},
|
|
16
|
+
"target_threshold_percent": {
|
|
17
|
+
"type": "number",
|
|
18
|
+
"minimum": 0,
|
|
19
|
+
"maximum": 100
|
|
20
|
+
},
|
|
21
|
+
"measurement_status": {
|
|
22
|
+
"type": "string",
|
|
23
|
+
"enum": ["estimated", "measured", "mixed"]
|
|
24
|
+
},
|
|
25
|
+
"scope": {
|
|
26
|
+
"type": "string",
|
|
27
|
+
"enum": ["fixture", "current_run", "run_history", "production_telemetry"]
|
|
28
|
+
},
|
|
29
|
+
"limits": {
|
|
30
|
+
"type": "array",
|
|
31
|
+
"items": { "type": "string" }
|
|
32
|
+
},
|
|
33
|
+
"scores": {
|
|
34
|
+
"type": "array",
|
|
35
|
+
"items": { "$ref": "#/$defs/metric_score" }
|
|
36
|
+
}
|
|
37
|
+
},
|
|
38
|
+
"$defs": {
|
|
39
|
+
"metric_score": {
|
|
40
|
+
"type": "object",
|
|
41
|
+
"required": [
|
|
42
|
+
"name",
|
|
43
|
+
"score_percent",
|
|
44
|
+
"formula",
|
|
45
|
+
"numerator",
|
|
46
|
+
"denominator",
|
|
47
|
+
"evidence_refs"
|
|
48
|
+
],
|
|
49
|
+
"properties": {
|
|
50
|
+
"name": {
|
|
51
|
+
"type": "string",
|
|
52
|
+
"enum": [
|
|
53
|
+
"unnecessary_context_reduction",
|
|
54
|
+
"initial_context_pack_size_reduction",
|
|
55
|
+
"review_evidence_lookup_time_reduction",
|
|
56
|
+
"verification_omission_detection",
|
|
57
|
+
"repeated_context_reuse_accuracy",
|
|
58
|
+
"rework_probability_reduction"
|
|
59
|
+
]
|
|
60
|
+
},
|
|
61
|
+
"score_percent": {
|
|
62
|
+
"type": "number",
|
|
63
|
+
"minimum": 0,
|
|
64
|
+
"maximum": 100
|
|
65
|
+
},
|
|
66
|
+
"formula": {
|
|
67
|
+
"type": "string"
|
|
68
|
+
},
|
|
69
|
+
"numerator": {
|
|
70
|
+
"type": "number",
|
|
71
|
+
"minimum": 0
|
|
72
|
+
},
|
|
73
|
+
"denominator": {
|
|
74
|
+
"type": "number",
|
|
75
|
+
"minimum": 0
|
|
76
|
+
},
|
|
77
|
+
"evidence_refs": {
|
|
78
|
+
"type": "array",
|
|
79
|
+
"items": { "type": "string" }
|
|
80
|
+
},
|
|
81
|
+
"limit": {
|
|
82
|
+
"type": "string"
|
|
83
|
+
}
|
|
84
|
+
},
|
|
85
|
+
"additionalProperties": true
|
|
86
|
+
}
|
|
87
|
+
},
|
|
88
|
+
"additionalProperties": true
|
|
89
|
+
}
|