@event4u/agent-config 8.7.0 → 8.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/CHANGELOG.md +109 -222
- package/CONTRIBUTING.md +25 -0
- package/README.md +7 -4
- package/dist/agent-src/commands/agent-handoff.md +39 -2
- package/dist/agent-src/commands/research/deep.md +8 -0
- package/dist/agent-src/commands/research/report.md +7 -0
- package/dist/agent-src/commands/roadmap/create.md +12 -0
- package/dist/agent-src/commands/roadmap/process-full.md +31 -5
- package/dist/agent-src/contexts/communication/rules-auto/think-before-action-mechanics.md +30 -1
- package/dist/agent-src/contexts/execution/auto-orchestration-activation.md +1 -1
- package/dist/agent-src/contexts/execution/orchestration-benchmark-gate.md +28 -16
- package/dist/agent-src/contexts/execution/orchestration-telemetry.md +36 -5
- package/dist/agent-src/contexts/execution/roadmap-process-loop.md +37 -0
- package/dist/agent-src/contexts/execution/subagent-topologies.md +34 -0
- package/dist/agent-src/contexts/execution/verification-mechanics.md +9 -9
- package/dist/agent-src/rules/active-remediation.md +92 -0
- package/dist/agent-src/rules/agent-authority.md +1 -1
- package/dist/agent-src/rules/broken-access-control.md +106 -0
- package/dist/agent-src/rules/communication-through-line.md +42 -0
- package/dist/agent-src/rules/decision-revisit-gate.md +8 -0
- package/dist/agent-src/rules/delegation-policy.md +5 -0
- package/dist/agent-src/rules/direct-answers.md +1 -1
- package/dist/agent-src/rules/downstream-changes.md +1 -0
- package/dist/agent-src/rules/external-code-graph-interop.md +65 -0
- package/dist/agent-src/rules/git-history-discipline.md +18 -0
- package/dist/agent-src/rules/lethal-trifecta-guard.md +6 -1
- package/dist/agent-src/rules/minimal-safe-diff.md +66 -1
- package/dist/agent-src/rules/non-destructive-by-default.md +3 -0
- package/dist/agent-src/rules/prefer-enums-over-literals.md +65 -0
- package/dist/agent-src/rules/question-not-instruction.md +76 -0
- package/dist/agent-src/rules/senior-engineering-discipline.md +82 -0
- package/dist/agent-src/rules/think-before-action.md +1 -0
- package/dist/agent-src/rules/tool-safety.md +8 -0
- package/dist/agent-src/rules/untrusted-input-defense.md +87 -22
- package/dist/agent-src/scripts/update_roadmap_progress.ts +2 -1
- package/dist/agent-src/skills/accessibility-auditor/SKILL.md +19 -0
- package/dist/agent-src/skills/ai-code-blindspots/SKILL.md +112 -0
- package/dist/agent-src/skills/ai-code-blindspots/evals/triggers.json +52 -0
- package/dist/agent-src/skills/authz-review/SKILL.md +21 -4
- package/dist/agent-src/skills/code-review/SKILL.md +2 -0
- package/dist/agent-src/skills/command-writing/SKILL.md +53 -40
- package/dist/agent-src/skills/dcf-modeling/evals/domain-truth.json +29 -0
- package/dist/agent-src/skills/deep-reading-analyst/SKILL.md +1 -0
- package/dist/agent-src/skills/dependency-upgrade/SKILL.md +20 -0
- package/dist/agent-src/skills/design-intelligence/SKILL.md +49 -12
- package/dist/agent-src/skills/design-review/SKILL.md +29 -15
- package/dist/agent-src/skills/design-system-capture/SKILL.md +63 -24
- package/dist/agent-src/skills/forecasting/evals/domain-truth.json +29 -0
- package/dist/agent-src/skills/frontend-render-security/SKILL.md +104 -0
- package/dist/agent-src/skills/frontend-render-security/evals/triggers.json +52 -0
- package/dist/agent-src/skills/markitdown/SKILL.md +1 -0
- package/dist/agent-src/skills/mcp/SKILL.md +24 -0
- package/dist/agent-src/skills/mcp-builder/SKILL.md +9 -1
- package/dist/agent-src/skills/memory-consolidation/SKILL.md +55 -4
- package/dist/agent-src/skills/roadmap-writing/SKILL.md +46 -5
- package/dist/agent-src/skills/runway-cognition/evals/domain-truth.json +29 -0
- package/dist/agent-src/skills/scenario-modeling/evals/domain-truth.json +29 -0
- package/dist/agent-src/skills/secrets-management/SKILL.md +7 -0
- package/dist/agent-src/skills/skill-writing/SKILL.md +12 -3
- package/dist/agent-src/skills/subagent-orchestration/SKILL.md +27 -33
- package/dist/agent-src/skills/supply-chain-intake/SKILL.md +112 -0
- package/dist/agent-src/skills/supply-chain-intake/evals/triggers.json +52 -0
- package/dist/agent-src/skills/testing-anti-patterns/SKILL.md +72 -2
- package/dist/agent-src/skills/ui-component-architect/SKILL.md +9 -0
- package/dist/agent-src/skills/unit-economics-modeling/evals/domain-truth.json +40 -0
- package/dist/agent-src/templates/agents/agent-project-settings.example.yml +1 -1
- package/dist/agent-src/templates/roadmaps.md +60 -0
- package/dist/cli/registry.js +3 -0
- package/dist/cli/registry.js.map +1 -1
- package/dist/discovery/deprecation-report.md +1 -1
- package/dist/discovery/discovery-manifest.json +268 -46
- package/dist/discovery/discovery-manifest.json.sha256 +1 -1
- package/dist/discovery/discovery-manifest.summary.md +4 -4
- package/dist/discovery/orphan-report.md +1 -1
- package/dist/discovery/packs.json +18 -8
- package/dist/discovery/trust-report.md +11 -11
- package/dist/discovery/workspaces.json +16 -6
- package/dist/install/install.mjs +141 -3
- package/dist/mcp/awesome-mcp-servers.row.md +1 -1
- package/dist/mcp/mcp-cloudflare-catalogue.json +1 -1
- package/dist/mcp/registry-manifest.json +4 -4
- package/dist/router.json +1 -1
- package/dist/scripts/_lib/claude_builtin_names.js +145 -0
- package/dist/scripts/_lib/claude_builtin_names.js.map +1 -0
- package/dist/scripts/condense.js +14 -0
- package/dist/scripts/condense.js.map +1 -1
- package/dist/server/schemas/settings.js +2 -1
- package/dist/server/schemas/settings.js.map +1 -1
- package/docs/CLAIMS.md +58 -2
- package/docs/architecture.md +3 -3
- package/docs/archive/CHANGELOG-pre-8.9.0.md +306 -0
- package/docs/benchmark.md +113 -0
- package/docs/catalog.md +32 -5
- package/docs/contracts/command-clusters.md +18 -0
- package/docs/contracts/design-artifact-verification.md +124 -0
- package/docs/contracts/injected-block-authenticity-model.md +66 -0
- package/docs/contracts/mcp-tool-inventory.md +31 -30
- package/docs/decisions/ADR-061-corpus-grounding-layer.md +13 -0
- package/docs/decisions/ADR-105-automatic-subagent-orchestration.md +6 -0
- package/docs/decisions/ADR-116-memory-tripwire-activation-path.md +11 -2
- package/docs/decisions/ADR-117-subagents-auto-default-on.md +88 -0
- package/docs/decisions/INDEX.md +1 -0
- package/docs/experiences/content_creator.md +2 -0
- package/docs/experiences/finance.md +2 -0
- package/docs/experiences/founder.md +2 -0
- package/docs/experiences/ops.md +2 -0
- package/docs/featured-skills.md +2 -2
- package/docs/getting-started-by-role.md +1 -1
- package/docs/getting-started.md +1 -1
- package/docs/governance-advantage.md +1 -1
- package/docs/guidelines/agent-infra/asking-and-brevity-examples.md +27 -0
- package/docs/guidelines/agent-infra/emphasis-budget.md +45 -0
- package/docs/guidelines/agent-infra/memory-access.md +12 -0
- package/docs/guidelines/agent-infra/simplicity-and-goal-demos.md +151 -0
- package/docs/guidelines/agent-infra/tool-description-as-policy.md +35 -0
- package/docs/guidelines/design-canon.md +1 -0
- package/docs/proof.md +60 -3
- package/docs/release-runbook.md +138 -0
- package/docs/second-brain-scope.md +116 -0
- package/docs/succession.md +80 -0
- package/package.json +2 -2
- package/src/agent-src/scripts/update_roadmap_progress.ts +2 -1
- package/src/config/agent-settings.template.yml +25 -6
- package/src/scripts/_dispatch.bash +27 -0
- package/src/scripts/_lib/claude_builtin_names.ts +148 -0
- package/src/scripts/_lib/file_slicer.ts +117 -0
- package/src/scripts/_lib/lexical_index.ts +155 -0
- package/src/scripts/_lib/orchestration_gate.ts +12 -8
- package/src/scripts/_lib/orchestration_record.ts +170 -0
- package/src/scripts/_lib/orchestration_savings.ts +180 -0
- package/src/scripts/_lib/retrieval_sanitize.ts +71 -0
- package/src/scripts/_lib/security_lint.ts +1 -0
- package/src/scripts/_lib/stat_index.ts +98 -0
- package/src/scripts/benchmark.ts +122 -0
- package/src/scripts/build_proof.ts +51 -0
- package/src/scripts/chat_history.ts +79 -7
- package/src/scripts/check_memory.ts +2 -0
- package/src/scripts/check_quality_regression.ts +51 -0
- package/src/scripts/check_roadmap_trackable.ts +3 -2
- package/src/scripts/condense.ts +14 -0
- package/src/scripts/discovery_graph.ts +0 -0
- package/src/scripts/domain_soundness_status.ts +236 -0
- package/src/scripts/external_sources_denylist.json +4 -1
- package/src/scripts/generate_index.ts +17 -0
- package/src/scripts/hot_context_hook.ts +47 -2
- package/src/scripts/install-hooks.sh +25 -0
- package/src/scripts/install.ts +35 -2
- package/src/scripts/learning_sidecar.ts +0 -0
- package/src/scripts/lint_agent_skill_names.ts +29 -0
- package/src/scripts/lint_behavioural_eval_freshness.ts +140 -0
- package/src/scripts/lint_knowledge_scale.ts +4 -3
- package/src/scripts/lint_roadmap_complexity.ts +124 -0
- package/src/scripts/lint_versioned_cache.ts +195 -0
- package/src/scripts/mcp_server/consumer_tool_catalog.json +23 -3
- package/src/scripts/mcp_server/tools.ts +182 -1
- package/src/scripts/measure_lexical_ranking.ts +173 -0
- package/src/scripts/memory_lookup.ts +0 -0
- package/src/scripts/memory_replay.ts +285 -0
- package/src/scripts/orchestration_record.ts +140 -0
- package/src/scripts/orchestration_savings_report.ts +124 -0
- package/src/scripts/schemas/domain-truth.schema.json +94 -0
- package/src/scripts/schemas/evals.schema.json +113 -0
- package/src/scripts/schemas/skill.schema.json +4 -0
- package/src/scripts/score_domain_truth.ts +192 -0
- package/src/scripts/second_brain_retrieval.ts +278 -0
- package/src/scripts/second_brain_run.ts +341 -0
- package/src/scripts/second_brain_score.ts +186 -0
- package/src/scripts/session_memory_index.ts +94 -0
- package/src/scripts/skill_eval_coverage.ts +318 -0
- package/src/server/schemas/settings.ts +5 -2
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
# Emphasis budget
|
|
2
|
+
|
|
3
|
+
Authoring guideline. **This is a review-time discipline, NOT a merge-blocking
|
|
4
|
+
linter** — there is no CI gate; PR review owns the judgment.
|
|
5
|
+
|
|
6
|
+
## The principle
|
|
7
|
+
|
|
8
|
+
Emphasis is a currency, and inflation debases it. Reserve the loud channels —
|
|
9
|
+
ALL-CAPS, `## Iron Law` fences, spaced repetition of the same imperative — for
|
|
10
|
+
**asymmetric, irreversible harm**: data loss, credential/secret exposure, a
|
|
11
|
+
safety floor, a legal/compliance breach. When everything shouts, nothing does:
|
|
12
|
+
an over-emphasized doc trains the reader to skim past the caps, so the one caps
|
|
13
|
+
line that guards a `DROP TABLE` loses its signal.
|
|
14
|
+
|
|
15
|
+
## The discipline
|
|
16
|
+
|
|
17
|
+
When you add emphasis, be able to answer — in the artifact or the PR — two
|
|
18
|
+
questions:
|
|
19
|
+
|
|
20
|
+
1. **What specific harm does this prevent?** Name it (not "this is
|
|
21
|
+
important" — *what* goes wrong, to *whom*, how badly).
|
|
22
|
+
2. **Why is post-hoc correction insufficient?** Emphasis is justified when the
|
|
23
|
+
harm is irreversible or asymmetric (a wrong send/delete/charge can't be
|
|
24
|
+
taken back). If a normal-weight sentence + a later fix would do, use normal
|
|
25
|
+
weight.
|
|
26
|
+
|
|
27
|
+
If you cannot answer both, down-weight to plain prose.
|
|
28
|
+
|
|
29
|
+
## Why not a linter
|
|
30
|
+
|
|
31
|
+
A mechanical caps-quota is unenforceable — semantic substitution ("Critically:
|
|
32
|
+
never…", bold, block quotes) defeats a caps count while keeping the shout — and
|
|
33
|
+
it blocks ready work on a cosmetic metric. The Iron-Law *count* is also a
|
|
34
|
+
denominator artifact: most existing Iron Laws already guard data-loss /
|
|
35
|
+
credential / safety domains, so the raw number is not evidence of inflation.
|
|
36
|
+
Judgment at review time, keyed to the harm-asymmetry test above, is the control.
|
|
37
|
+
|
|
38
|
+
## Overlap (why a separate guideline)
|
|
39
|
+
|
|
40
|
+
Distinct from [`preservation-guard`](../../../src/rules/preservation-guard.md)
|
|
41
|
+
(preserve existing emphasis through a transform — this governs *adding* it),
|
|
42
|
+
[`size-enforcement`](../../../src/rules/size-enforcement.md) (line budgets, not
|
|
43
|
+
emphasis weight), and [`token-budget-discipline`](../../../src/rules/token-budget-discipline.md)
|
|
44
|
+
(load cost, not signal-to-noise). None owns the emphasis-rationing judgment;
|
|
45
|
+
this guideline does.
|
|
@@ -22,6 +22,18 @@ agent-config memory:lookup \
|
|
|
22
22
|
# --envelope v1 for the retrieval-contract envelope.
|
|
23
23
|
```
|
|
24
24
|
|
|
25
|
+
**Index-first discipline** (road-to-memory-retrieval-economy — the mirror
|
|
26
|
+
of the MCP tool descriptions, for hosts without MCP): when you expect more
|
|
27
|
+
hits than you need, look up with `--envelope v1 --detail index` first —
|
|
28
|
+
each row carries `id`, `title` and `tokens_estimate` (the cost of fetching
|
|
29
|
+
it) — then fetch ONLY the ids you will actually use via the `memory_get`
|
|
30
|
+
MCP tool (or a full `--detail full` lookup narrowed to the right keys),
|
|
31
|
+
batching multiple ids into one call. For precise single-hit lookups the
|
|
32
|
+
full envelope is already minimal — the paired measurement showed index-
|
|
33
|
+
first COSTS tokens there (honest null pinned in
|
|
34
|
+
`internal/bench/reports/memory-retrieval-run.json`), so reach for index
|
|
35
|
+
mode on broad or exploratory queries, not reflexively.
|
|
36
|
+
|
|
25
37
|
Every backend MUST return a list of `Hit` with:
|
|
26
38
|
|
|
27
39
|
| Field | Meaning |
|
|
@@ -0,0 +1,151 @@
|
|
|
1
|
+
# Simplicity and Goal Discipline — wrong/right demos
|
|
2
|
+
|
|
3
|
+
Recognition surface for [`minimal-safe-diff § Anti-over-engineering`](../../src/rules/minimal-safe-diff.md)
|
|
4
|
+
(own-orphan cleanup, speculative-complexity bans) and the goal-driven
|
|
5
|
+
execution clause in `think-before-action`. Per council convergence
|
|
6
|
+
(2026-07-10): the demos ARE the enforcement — an agent that has never seen
|
|
7
|
+
the violation cannot recognize it, no matter how crisp the checklist question.
|
|
8
|
+
Match your diff against the *right* column before presenting.
|
|
9
|
+
|
|
10
|
+
## 1. Over-abstraction vs. one function
|
|
11
|
+
|
|
12
|
+
**Ask:** "Add a function to calculate a percentage discount."
|
|
13
|
+
|
|
14
|
+
**Wrong — Strategy pattern for a single caller (PHP):**
|
|
15
|
+
|
|
16
|
+
```php
|
|
17
|
+
interface DiscountStrategy { public function calculate(float $amount): float; }
|
|
18
|
+
final class PercentageDiscount implements DiscountStrategy { /* … */ }
|
|
19
|
+
final class FixedDiscount implements DiscountStrategy { /* … */ }
|
|
20
|
+
final readonly class DiscountConfig { /* strategy, minPurchase, maxDiscount */ }
|
|
21
|
+
final class DiscountCalculator { /* 30+ lines of orchestration */ }
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
**Right — one function until the second discount type is real:**
|
|
25
|
+
|
|
26
|
+
```php
|
|
27
|
+
function calculateDiscount(float $amount, float $percent): float
|
|
28
|
+
{
|
|
29
|
+
return $amount * ($percent / 100);
|
|
30
|
+
}
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
**Why:** No second caller, no second strategy → no abstraction. When the
|
|
34
|
+
second type genuinely arrives, refactor then (`prefer-enums-over-literals` /
|
|
35
|
+
Strategy sniff test governs that moment). The wrong version is not "bad
|
|
36
|
+
code" — it is *premature* code: harder to read, more surface to test, and
|
|
37
|
+
speculative until the requirement exists.
|
|
38
|
+
|
|
39
|
+
## 2. Speculative features vs. just-what-was-asked
|
|
40
|
+
|
|
41
|
+
**Ask:** "Save user preferences to the database."
|
|
42
|
+
|
|
43
|
+
**Wrong — options nobody requested (TypeScript):**
|
|
44
|
+
|
|
45
|
+
```ts
|
|
46
|
+
class PreferenceManager {
|
|
47
|
+
constructor(db: Db, cache?: Cache, validator?: Validator) { /* … */ }
|
|
48
|
+
save(userId: number, prefs: Prefs,
|
|
49
|
+
opts: { merge?: boolean; validate?: boolean; notify?: boolean } = {}) {
|
|
50
|
+
// merging, validation, cache write, notification fan-out — all unasked
|
|
51
|
+
}
|
|
52
|
+
}
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
**Right — the ask, nothing else:**
|
|
56
|
+
|
|
57
|
+
```ts
|
|
58
|
+
async function savePreferences(db: Db, userId: number, prefs: Prefs) {
|
|
59
|
+
await db.query('UPDATE users SET preferences = $1 WHERE id = $2',
|
|
60
|
+
[JSON.stringify(prefs), userId]);
|
|
61
|
+
}
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
**Why:** Every optional parameter with one call site, every "flexibility"
|
|
65
|
+
flag, and every handler for a scenario that cannot occur is speculative
|
|
66
|
+
complexity. Add caching when performance data demands it, validation when bad
|
|
67
|
+
data appears, merging when the requirement lands.
|
|
68
|
+
|
|
69
|
+
## 3. Drive-by refactor vs. surgical diff
|
|
70
|
+
|
|
71
|
+
**Ask:** "Fix the bug where an empty email crashes the validator."
|
|
72
|
+
|
|
73
|
+
**Wrong:** the fix PLUS improved email regex, new username length checks,
|
|
74
|
+
rewritten comments, and a fresh docstring — four changes, one was asked.
|
|
75
|
+
|
|
76
|
+
**Right:** only the lines that make the empty-email case safe change; every
|
|
77
|
+
other line of the function is byte-identical.
|
|
78
|
+
|
|
79
|
+
**Why:** Every changed line must trace to the stated task
|
|
80
|
+
(`minimal-safe-diff`). The "improvements" belong in a follow-up proposed via
|
|
81
|
+
`active-remediation`'s note-and-ask ladder — not smuggled into the bug fix.
|
|
82
|
+
|
|
83
|
+
## 4. Style drift vs. match-existing-style
|
|
84
|
+
|
|
85
|
+
**Ask:** "Add logging to the upload function."
|
|
86
|
+
|
|
87
|
+
**Wrong:** the diff adds logging AND flips quote style, adds type hints,
|
|
88
|
+
reshapes the boolean return, reflows whitespace — the reviewer can no longer
|
|
89
|
+
see the logging change.
|
|
90
|
+
|
|
91
|
+
**Right:** logger import + three log lines, inserted in the file's existing
|
|
92
|
+
style (its quoting, its spacing, its return pattern), even where you would
|
|
93
|
+
personally write it differently.
|
|
94
|
+
|
|
95
|
+
**Why:** Style consistency is a repo property, not a per-diff taste decision.
|
|
96
|
+
A diff that reformats while changing behavior hides the behavior change.
|
|
97
|
+
|
|
98
|
+
## 5. Own-orphan vs. pre-existing dead code
|
|
99
|
+
|
|
100
|
+
**Ask:** "Replace the legacy formatter call in `processA` with the new one."
|
|
101
|
+
|
|
102
|
+
**Wrong (too little):** `processA` no longer calls `legacyFormat()`, but the
|
|
103
|
+
now-unused import and the helper it alone referenced stay behind — the diff
|
|
104
|
+
shipped its own litter.
|
|
105
|
+
|
|
106
|
+
**Wrong (too much):** while in the file, the agent also deletes
|
|
107
|
+
`oldHelper()`, which `processB` — untouched by this diff — still references
|
|
108
|
+
in another module, or which was already orphaned *before* this task.
|
|
109
|
+
|
|
110
|
+
**Right:** remove exactly the imports/variables/functions whose **last
|
|
111
|
+
reference disappeared in a file this diff touched**; anything still
|
|
112
|
+
referenced from untouched files — or orphaned before the task — is
|
|
113
|
+
pre-existing debt: mention it, don't delete it.
|
|
114
|
+
|
|
115
|
+
**Why:** The boundary is mechanical (grep after editing), so there is no
|
|
116
|
+
judgment call to rationalize — see
|
|
117
|
+
[`minimal-safe-diff § Own-orphan cleanup`](../../src/rules/minimal-safe-diff.md#own-orphan-cleanup).
|
|
118
|
+
|
|
119
|
+
## 6. Vague plan vs. verifiable per-step plan
|
|
120
|
+
|
|
121
|
+
**Ask:** "Add rate limiting to the API."
|
|
122
|
+
|
|
123
|
+
**Wrong — unverifiable narration:**
|
|
124
|
+
|
|
125
|
+
```
|
|
126
|
+
1. Review the code
|
|
127
|
+
2. Identify issues
|
|
128
|
+
3. Implement rate limiting
|
|
129
|
+
4. Test the changes
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
**Right — every step carries its check:**
|
|
133
|
+
|
|
134
|
+
```
|
|
135
|
+
1. In-memory limit on one endpoint → verify: 11th request in a minute → 429 (test)
|
|
136
|
+
2. Extract to middleware, all endpoints → verify: limit fires on /users AND /posts; existing endpoint tests stay green
|
|
137
|
+
3. Shared store backend → verify: counter survives app restart; two instances share the count
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
**Why:** A step without a `verify:` is an assumption. Strong criteria let the
|
|
141
|
+
loop run independently (`verify-before-complete` then has something to
|
|
142
|
+
check); "make it work" guarantees clarification churn after the fact instead
|
|
143
|
+
of before.
|
|
144
|
+
|
|
145
|
+
## Key insight
|
|
146
|
+
|
|
147
|
+
The wrong versions are not obviously wrong — they follow known patterns and
|
|
148
|
+
"best practices". The failure is **timing**: complexity added before the
|
|
149
|
+
requirement exists costs comprehension, bugs, and test surface today for a
|
|
150
|
+
tomorrow that may never come. Solve today's problem simply; refactor when the
|
|
151
|
+
second requirement is real.
|
|
@@ -0,0 +1,35 @@
|
|
|
1
|
+
# Tool description as policy
|
|
2
|
+
|
|
3
|
+
Authoring guideline for skills, commands, and MCP tools. Shared block — cited
|
|
4
|
+
from [`skill-writing`](../../../src/skills/skill-writing/SKILL.md),
|
|
5
|
+
[`command-writing`](../../../src/skills/command-writing/SKILL.md), and
|
|
6
|
+
[`mcp-builder`](../../../src/skills/mcp-builder/SKILL.md).
|
|
7
|
+
|
|
8
|
+
## The principle
|
|
9
|
+
|
|
10
|
+
Policy that governs *how a tool is used* belongs **inside the tool's own
|
|
11
|
+
description**, where it fires at the decision point the model is already
|
|
12
|
+
reading — not in always-on prose the model may not have loaded when it reaches
|
|
13
|
+
for the tool. The description is the highest-signal, best-timed place to encode
|
|
14
|
+
usage policy.
|
|
15
|
+
|
|
16
|
+
## What to encode in the description
|
|
17
|
+
|
|
18
|
+
- **Workflow sequencing.** If tool B must follow tool A ("call `memory_lookup`
|
|
19
|
+
with `detail:index` FIRST, then `memory_get` the ids"), say so in B's
|
|
20
|
+
description, not in a separate rule.
|
|
21
|
+
- **Preconditions.** What must be true before the tool is called (a built
|
|
22
|
+
artifact present, a prior step run) — stated where the model chooses to call.
|
|
23
|
+
- **ID / output provenance.** When the tool returns handles (ids, paths, PR
|
|
24
|
+
numbers), instruct: **copy them verbatim from the tool result, never from
|
|
25
|
+
memory** — context decay silently corrupts a remembered id.
|
|
26
|
+
- **A mandatory "why" intent field.** Require the caller to state the intent of
|
|
27
|
+
the call; a tool that forces "why am I calling this" cuts reflexive misuse.
|
|
28
|
+
- **Turn-end contracts.** If the tool ends the turn or hands off, the
|
|
29
|
+
description says so — the caller does not discover it after the fact.
|
|
30
|
+
|
|
31
|
+
## Why here, not as an always-on rule
|
|
32
|
+
|
|
33
|
+
An always-on rule pays context cost on every turn and may be evicted before the
|
|
34
|
+
decision; the description is loaded exactly when the tool is under
|
|
35
|
+
consideration. Encode the policy at the point of use.
|
|
@@ -60,3 +60,4 @@ Pull when: choosing or auditing an icon library.
|
|
|
60
60
|
- `docs/guidelines/design-antipatterns.md` — the anti-slop catalog (T7 default-font tell, C1/C5 palette tells) this canon helps you escape.
|
|
61
61
|
- `brand-source-of-truth` / `brand-consistency` — the precedence this index obeys.
|
|
62
62
|
- `design-intelligence` — the consumer that surfaces these on a brief signal.
|
|
63
|
+
- **Reference, don't vendor** (`domain-adoption-policy`): palette hex tables, font-width CSVs, easing/motion constant libraries, print-CSS recipes, and device-frame recipes are **referenced from maintained upstreams, never forked** into this suite — a vendored data table rots, fork-drifts, and couples the suite to one aesthetic. Corpus assets (palettes, easing tables, device frames) stay upstream references, not forks.
|
package/docs/proof.md
CHANGED
|
@@ -26,16 +26,24 @@ evidence pointer, or `task check-claims` fails the build.
|
|
|
26
26
|
|
|
27
27
|
| Claim | Kind | Evidence | Resolves |
|
|
28
28
|
|---|---|---|---|
|
|
29
|
+
| The release process is documented as an inheritable runbook + succession doc, and the project's bus-factor (trailing-90-day distinct human reviewers) is tracked and reported truthfully — currently 1, not implied to be more. | qual | `docs/succession.md#trailing 90 days` | ✅ |
|
|
29
30
|
| 172 commands. | quant | `src/scripts/check_artefact_count_messaging.ts#Artefact-count messaging gate` | ✅ |
|
|
31
|
+
| The retrieval economy cuts always-loaded context tokens ~65.6% measured against the FULL always-loaded projection — 98,529 → 33,897 tokens (eager rule load + skill/command descriptions + MCP schemas, thin-flipped). Method: `agent-config benchmark` over the pinned token baseline; the baseline is the honest "what the user pays if everything loads eagerly", NOT a synthetic full-corpus strawman (council Q4). | quant | `internal/bench/reports/token-baseline.json#eager_rule_load` | ✅ |
|
|
30
32
|
| On a weak host (claude-haiku-4-5) the package produces a significant, placebo-controlled discipline lift on scope/downstream traps; on a strong host the same measurement is a published null — the package transplants discipline a weak model lacks, not model intelligence. | quant | `docs/benchmark.md#weak-host-specific` | ✅ |
|
|
33
|
+
| The non-coding domain skills (finance/founder/ops/content) are forged on TS/PHP and labeled unvalidated until they pass a sourced domain-truth fixture; no public prose implies proven domain correctness, and the validated count is CI-ratcheted. | qual | `src/scripts/domain_soundness_status.ts#checkRatchet` | ✅ |
|
|
31
34
|
| On the READ-ONLY FAN-OUT slice family, tier-downshifted subagent dispatch (lite/haiku vs session-tier-proxy sonnet) nets a ≥30% USD-weighted token-cost reduction at held quality — measured 2026-07-08 (n=10 paired live dispatches, 20 telemetry lines): 10/10 exact-match on BOTH arms, 29.4% fewer raw tokens, 76.5% USD-weighted cost reduction at the 3x haiku↔sonnet price ratio. FAMILY-SCOPED — the mechanical-edit family is unmeasured and its downshift (incl. the deferred tier downgrades of existing units) stays gated. Negative control held: an open-ended synthesis/unknown slice never resolves below the session tier (inferSliceTier → medium/inherit, never lite). | quant | `internal/bench/routing-downshift/results-2026-07-08.md#FAMILY-SCOPED PROVE` | ✅ |
|
|
32
35
|
| The lift-carrying essential cut (kernel + downstream-changes) keeps a significant weak-host discipline lift at a fraction of the full load's tokens, and the lift is FAMILY- and HOST-SCOPED — measured on three hosts: claude-haiku-4-5 (weak) shows the family-scoped lift (trapE 0.533→1.000, 7/7 discordant, corpus cost 1.71x); claude-sonnet-4-6 (strong) is a ceiling null; gpt-5-mini (non-Claude weak, codex prompt-prepend surface) FAILED replication with headroom (corpus Δ=+0.024 p=0.70, capability trend n.s. — no harm claimed, injection-surface confound documented). Therefore discipline_profile: auto enables the lift only where measured (vendor-granular unknown_defaults). Non-claims — the balanced router profile was removed after a NULL measurement (p=0.81, n=24); no full-tier recommendation exists; no cross-vendor lift is claimed. | quant | `docs/benchmark.md#REPLICATION FAILED` | ✅ |
|
|
36
|
+
| Behavioural-eval coverage is measured per tier and CI-ratcheted so it can only rise; the current coverage and its gap are published, never implied as "264 evaluated skills". | qual | `src/scripts/skill_eval_coverage.ts#checkRatchet` | ✅ |
|
|
37
|
+
| A hand-rolled, dependency-free BM25 + trigram lexical index resolves the "recalls but does not rank" gap: on the retrieval-precision corpus (9 keyword-overlapping-confuser tasks) it drives the mean top tie-set from 3.333 (the `_score` bucket scorer) to 1.0 — every needed decision uniquely top-ranked — with precision@1 and precision@5 unchanged at 1.0. Method: deterministic, model-free re-ranking of the SAME retrieved entry set; both scorers measured over the identical store via `measure_lexical_ranking.ts`. | quant | `internal/bench/reports/lexical-ranking.json` | ✅ |
|
|
33
38
|
| The whole layer is compiled into host agents with zero runtime daemon. | qual | `docs/contracts/no-runtime-boundary.md#file-first, no-runtime suite` | ✅ |
|
|
34
|
-
|
|
|
35
|
-
|
|
|
39
|
+
| 102 governed rules. | quant | `src/scripts/check_artefact_count_messaging.ts#Artefact-count messaging gate` | ✅ |
|
|
40
|
+
| On a deterministic multi-session recall corpus, the memory substrate produces a measured, placebo-controlled recall lift — memory-on 27/27 vs no-memory 10/27 and vs equal-byte placebo 9/27 (claude-haiku-4-5, n=9 tasks x 3 seeds, sign test p=0.031 for BOTH pairings). Scoped honestly: this is the context-value upper bound (perfect retrieval on a one-fact-per-task corpus), not retrieval precision under a large store. | quant | `internal/bench/reports/second-brain-delta.json` | ✅ |
|
|
41
|
+
| Removing the perfect-retrieval assumption, the substrate's REAL keyword retrieval recalls the needed decision into the top-5 under keyword-overlapping confusers (precision@5 9/9) and the model disambiguates it from the co-injected confusers — retrieval-on 27/27 vs no-memory 5/27 and vs equal-count placebo 5/27 (claude-haiku-4-5, 9 tasks x 3 seeds, sign test p=0.008 both). Named limit: retrieval RECALLS but does not RANK (mean tie-set 3.3, ties broken by store order, not relevance) — the discrimination gap that motivates the SQLite-FTS5 activation path (ADR-116) at larger scale. | quant | `internal/bench/reports/second-brain-retrieval.json` | ✅ |
|
|
42
|
+
| Every artifact the package ships — source AND the condensed projection that reaches consumers — is machine-scanned in CI for hidden-Unicode, mixed-script-confusable, and instruction-smuggling payloads (the rules-file-backdoor class); a finding blocks the release before `npm publish`, not just the merge. | qual | `.github/workflows/publish-npm.yml#lint_agent_security` | ✅ |
|
|
43
|
+
| 267 skills (README hero + feature list). | quant | `src/scripts/check_artefact_count_messaging.ts#Artefact-count messaging gate` | ✅ |
|
|
36
44
|
| Removes only its own keys from a shared host config (matched by JSON-pointer + SHA-256), never a neighbour tool's entries. | qual | `docs/contracts/install-layout.md#JSON-pointer` | ✅ |
|
|
37
45
|
|
|
38
|
-
**
|
|
46
|
+
**16 backed claim(s)** — all evidence pointers resolve in CI.
|
|
39
47
|
|
|
40
48
|
Artefact counts in public prose (skills, commands, governed rules,
|
|
41
49
|
guidelines, personas) are **generated from source and CI-drift-checked**:
|
|
@@ -54,6 +62,53 @@ Benchmark results — **including the runs where the package changed nothing**
|
|
|
54
62
|
live in [`docs/benchmark.md`](benchmark.md). We do not delete a measured null to
|
|
55
63
|
make a number look better; the null is the evidence of honesty.
|
|
56
64
|
|
|
65
|
+
**Behavioural-eval coverage — the honest baseline.** Skill *quality* is only
|
|
66
|
+
as good as its measurement. Today **2 of 264** skills carry a behavioural
|
|
67
|
+
`evals.json`, and the highest-traffic / highest-cost tiers (default-surface +
|
|
68
|
+
`rich` + routers, **0 of 35**) are not yet covered. We publish that gap rather
|
|
69
|
+
than imply "264 evaluated skills": coverage is measured per tier
|
|
70
|
+
(`./scripts-run src/scripts/skill_eval_coverage`) and **CI-ratcheted so it can
|
|
71
|
+
only rise** — a merged change can never lower it. Authoring the priority-tier
|
|
72
|
+
evals is gated on per-case human ratification (a generated assertion that
|
|
73
|
+
checks the wrong property is worse than none), so the number grows
|
|
74
|
+
deliberately, not overnight.
|
|
75
|
+
|
|
76
|
+
**Non-coding domain soundness — scoped, not proven.** The `finance` /
|
|
77
|
+
`founder` / `ops` / `content` profiles sell concrete domain value (DCF,
|
|
78
|
+
runway, RICE, incident command, messaging), but the skills are forged on
|
|
79
|
+
TS/PHP — "promising, not proven" off those stacks. A disclaimer floor
|
|
80
|
+
bounds *liability*, not *correctness*: a skill can be format-correct,
|
|
81
|
+
disclaimered, and still embed a wrong domain assumption. Today **5 of 20**
|
|
82
|
+
default-surface domain skills carry a sourced `domain-truth` fixture
|
|
83
|
+
(`./scripts-run src/scripts/domain_soundness_status`); the rest are labeled
|
|
84
|
+
`unvalidated` and the validated count is CI-ratcheted. The fixtures landed
|
|
85
|
+
so far are the **deterministic** targets (runway, unit-economics, DCF,
|
|
86
|
+
forecasting, scenario band/sensitivity), whose answer keys are computed
|
|
87
|
+
from cited standard formulas — never the skill's own output; the
|
|
88
|
+
**rubric** targets (incident command, messaging, fundraising, editorial)
|
|
89
|
+
need domain-competent grounding and remain unvalidated, so validation
|
|
90
|
+
lands deliberately — the gap is published, never implied away.
|
|
91
|
+
|
|
92
|
+
**Second-brain substrate — measured recall lift, honestly bounded.** On a
|
|
93
|
+
deterministic multi-session recall corpus, the memory substrate beats a
|
|
94
|
+
no-memory baseline AND an equal-byte placebo: memory-on **27/27** vs
|
|
95
|
+
no-memory **10/27** vs placebo **9/27** (claude-haiku-4-5, 9 tasks × 3
|
|
96
|
+
seeds, sign test **p = 0.031** for both pairings) — a real,
|
|
97
|
+
placebo-controlled lift (`internal/bench/reports/second-brain-delta.json`).
|
|
98
|
+
Scoped, not oversold: this is the context-value upper bound (perfect
|
|
99
|
+
retrieval on a one-fact-per-task corpus), NOT retrieval precision under a
|
|
100
|
+
large store; the lift concentrates exactly where memory is the only source
|
|
101
|
+
and ties where the prompt self-contains the fact. Boundary vs a human PKM,
|
|
102
|
+
and why the Obsidian export stays declined, are in
|
|
103
|
+
[`docs/second-brain-scope.md`](second-brain-scope.md).
|
|
104
|
+
|
|
105
|
+
A follow-up removed the perfect-retrieval assumption: against a store of
|
|
106
|
+
keyword-overlapping confusers the REAL retrieval recalls the needed
|
|
107
|
+
decision into the top-5 (9/9) and the model disambiguates it — retrieval-on
|
|
108
|
+
**27/27** vs no-memory **5/27** and vs placebo **5/27** (p=0.008 both). The
|
|
109
|
+
honest limit: the keyword scorer recalls but does not rank (mean tie-set
|
|
110
|
+
3.3), which is the SQLite-FTS5 activation signal (ADR-116) at scale.
|
|
111
|
+
|
|
57
112
|
## 3. Known limits (published, witness-tested)
|
|
58
113
|
|
|
59
114
|
Each limitation below is stated by the skill itself and carries a witness
|
|
@@ -89,6 +144,8 @@ task check-claims # every markered public claim binds to resolvable evidence
|
|
|
89
144
|
task check-refs # no broken internal references
|
|
90
145
|
task check-skill-gaps # every logged known-limit cites a real witness test
|
|
91
146
|
task check-comparison # every comparison-table "our evidence" pointer resolves
|
|
147
|
+
./scripts-run src/scripts/skill_eval_coverage # behavioural-eval coverage, per tier
|
|
148
|
+
./scripts-run src/scripts/skill_eval_coverage --check # the ratchet: coverage may not drop
|
|
92
149
|
task build-proof-check # this page is in sync with its sources
|
|
93
150
|
```
|
|
94
151
|
|
|
@@ -0,0 +1,138 @@
|
|
|
1
|
+
# Release runbook — cut a correct release from written steps alone
|
|
2
|
+
|
|
3
|
+
> **Bus-factor doc (road-to-maintainer-bus-factor Phase 3).** The goal: a second
|
|
4
|
+
> maintainer — or the current one after a long gap — can cut a correct release by
|
|
5
|
+
> following THIS document, with no tribal knowledge. Every step maps to a real
|
|
6
|
+
> gate or command in the repo; nothing here is aspirational. Companion:
|
|
7
|
+
> [`succession.md`](succession.md) (secrets, operator-gated steps, "healthy main").
|
|
8
|
+
>
|
|
9
|
+
> **Source of truth for the mechanism:** [`src/scripts/release.ts`](../src/scripts/release.ts)
|
|
10
|
+
> (the shared pipeline) + [`.github/workflows/release.yml`](../.github/workflows/release.yml)
|
|
11
|
+
> (the CI entry) + [`ADR-113`](decisions/ADR-113-ci-native-release-label-trigger.md).
|
|
12
|
+
> If this runbook and those disagree, THEY win and this doc is stale — fix it.
|
|
13
|
+
|
|
14
|
+
## 0. Mental model
|
|
15
|
+
|
|
16
|
+
There is **one** release pipeline (`src/scripts/release.ts`) reached by **two**
|
|
17
|
+
entry points that produce the identical result:
|
|
18
|
+
|
|
19
|
+
| Entry point | How | When to use |
|
|
20
|
+
|---|---|---|
|
|
21
|
+
| **Local** | `task release` (interactive) | You want to watch it, or CI dispatch is unavailable. |
|
|
22
|
+
| **CI-native** | merge a PR labeled `release` / `release:major` / `release:minor` / `release:patch` | The normal path — the merge triggers [`release.yml`](../.github/workflows/release.yml), which runs `release.ts --ci`. |
|
|
23
|
+
|
|
24
|
+
`release` auto-detects the semver bump from Conventional Commits since the last
|
|
25
|
+
tag; `release:major|minor|patch` forces it. A manual/recovery entry exists via
|
|
26
|
+
the **workflow_dispatch** on `release.yml` (inputs: `bump`, `version`,
|
|
27
|
+
`dry_run`, `resume`).
|
|
28
|
+
|
|
29
|
+
## 1. Pre-flight (before you start)
|
|
30
|
+
|
|
31
|
+
- [ ] `main` is green on the latest commit (all required checks — see
|
|
32
|
+
[`branch-protection-policy.md`](contracts/branch-protection-policy.md)).
|
|
33
|
+
- [ ] Working tree clean, on `main`, `origin/main` in sync
|
|
34
|
+
(`release.ts` step 1 enforces this and aborts otherwise).
|
|
35
|
+
- [ ] `gh` authenticated (`gh auth status`).
|
|
36
|
+
- [ ] The target tag does **not** already exist (step 1 checks; use `--resume`
|
|
37
|
+
to recover a partial run — see § 5).
|
|
38
|
+
- [ ] You have read [`succession.md`](succession.md) if any secret-gated
|
|
39
|
+
downstream (npm publish, cloud deploy) must succeed this release.
|
|
40
|
+
|
|
41
|
+
## 2. The pipeline — what `release.ts` does (9 steps)
|
|
42
|
+
|
|
43
|
+
Both entry points run these in order. Each step prints what it will do before
|
|
44
|
+
doing it, so a crash localises to a step.
|
|
45
|
+
|
|
46
|
+
1. **Preflight** — on `main`, clean tree, `origin` in sync, `gh` present, target
|
|
47
|
+
tag absent.
|
|
48
|
+
2. **Plan** — compute the new version, parse Conventional Commits since the last
|
|
49
|
+
tag, render the CHANGELOG section.
|
|
50
|
+
3. **Confirm** — show the preview and ask once. `--yes` skips; `--ci` always
|
|
51
|
+
requires `--yes` (no terminal in CI).
|
|
52
|
+
4. **Branch + bump** — create `release/X.Y.Z`; update `package.json`,
|
|
53
|
+
`.claude-plugin/marketplace.json`, `CHANGELOG.md`; then run
|
|
54
|
+
`task release-prepare` so pack manifests + tool projections pick up the new
|
|
55
|
+
version (skip this and the PR's own consistency check fails — PR #226
|
|
56
|
+
post-mortem).
|
|
57
|
+
5. **Commit + push** — commit `release: X.Y.Z`, push the branch, open the PR.
|
|
58
|
+
6. **Wait for CI** — `gh pr checks --watch` (skippable with `--no-wait`).
|
|
59
|
+
7. **Merge** — `gh pr merge --merge --delete-branch`.
|
|
60
|
+
8. **Tag main** — fast-forward `main`, tag the merge commit, push the tag.
|
|
61
|
+
9. **GitHub Release** — `gh release create X.Y.Z --notes <changelog>`. Under
|
|
62
|
+
`--ci`, also dispatches `release-guard.yml` + `publish-npm.yml` +
|
|
63
|
+
`cloud-release.yml` (a bot-pushed tag does not trigger them on its own —
|
|
64
|
+
GitHub's `GITHUB_TOKEN` recursion guard).
|
|
65
|
+
|
|
66
|
+
## 3. The two ways to run it
|
|
67
|
+
|
|
68
|
+
### A. CI-native (normal)
|
|
69
|
+
|
|
70
|
+
1. Open your change PR as usual; get it green + merged into `main`.
|
|
71
|
+
2. To release, add a **`release`** label (or `release:minor` etc.) to a PR and
|
|
72
|
+
merge it. `release.yml` fires on `pull_request: closed` (merged) and runs
|
|
73
|
+
`release.ts --ci`.
|
|
74
|
+
3. **Manual checkpoint (only without `RELEASE_PR_TOKEN`):** GitHub's
|
|
75
|
+
bot-created-PR safeguard queues the release PR's required checks in an
|
|
76
|
+
approval-required state. Go to the release PR → **"Approve workflows to
|
|
77
|
+
run"** once. The job's `gh pr checks --watch` then proceeds. This is a
|
|
78
|
+
deliberate production-release checkpoint, not a bug (see
|
|
79
|
+
[`branch-protection-policy.md`](contracts/branch-protection-policy.md)).
|
|
80
|
+
Configuring `RELEASE_PR_TOKEN` removes it (see [`succession.md`](succession.md)).
|
|
81
|
+
4. Watch the `release.yml` run to green. It merges the release PR, tags, cuts
|
|
82
|
+
the GitHub Release, and dispatches the downstream workflows.
|
|
83
|
+
|
|
84
|
+
### B. Local (`task release`)
|
|
85
|
+
|
|
86
|
+
1. `task release` — interactive; it runs the same 9 steps and asks once at
|
|
87
|
+
step 3. Use `--as minor` / `--version X.Y.Z` to override the bump; `--dry-run`
|
|
88
|
+
to preview with zero git/gh mutations.
|
|
89
|
+
2. Watch it merge + tag. The tag push triggers `publish-npm.yml` directly (local
|
|
90
|
+
runs use your user token, so no recursion guard applies).
|
|
91
|
+
|
|
92
|
+
## 4. Post-release verification
|
|
93
|
+
|
|
94
|
+
- [ ] `git log --first-parent origin/main -1` shows the `release: X.Y.Z` merge.
|
|
95
|
+
- [ ] `gh release view X.Y.Z` exists with the CHANGELOG notes.
|
|
96
|
+
- [ ] `publish-npm.yml` succeeded (npm has the new version) — see
|
|
97
|
+
[`succession.md`](succession.md) for the token dependency.
|
|
98
|
+
- [ ] Downstream deploys (`cloud-release.yml`, site) green if this release
|
|
99
|
+
touched them.
|
|
100
|
+
- [ ] `main` is green post-merge (no drift gate red: `release-drift.yml`,
|
|
101
|
+
`release-guard.yml`).
|
|
102
|
+
|
|
103
|
+
## 5. Recovery — a partial or failed release
|
|
104
|
+
|
|
105
|
+
`release.ts` is **idempotent under `--resume`**: it probes existing state
|
|
106
|
+
(branch, commit, PR, tag, GitHub Release) and skips completed steps.
|
|
107
|
+
|
|
108
|
+
- Re-run `release.yml` via **workflow_dispatch** with `resume: true`, or locally
|
|
109
|
+
`task release -- --resume`.
|
|
110
|
+
- Firing the `release` label path twice is a clean no-op (the `--ci`
|
|
111
|
+
nothing-to-release short-circuit).
|
|
112
|
+
- If a downstream dispatch (npm/cloud) failed but the tag + Release exist, re-run
|
|
113
|
+
that specific workflow from the Actions tab — do **not** re-cut the release.
|
|
114
|
+
|
|
115
|
+
## 6. What can go wrong (known checkpoints)
|
|
116
|
+
|
|
117
|
+
- **Release PR checks stuck "expected"** → the approve-workflows checkpoint in
|
|
118
|
+
§ 3.A step 3. Not a failure; click approve.
|
|
119
|
+
- **Consistency check red on the release PR** → step 4's `task release-prepare`
|
|
120
|
+
did not run or pack manifests drifted; re-run it on the release branch.
|
|
121
|
+
- **Tag exists but no npm version** → `publish-npm.yml` did not fire or failed;
|
|
122
|
+
re-run it (§ 5). Never delete + re-push the tag to "retry".
|
|
123
|
+
- **`main`/`origin` diverged mid-release** → stop; resolve the divergence with
|
|
124
|
+
the `git-workflow` skill's Divergent-State Recovery procedure first, then
|
|
125
|
+
`--resume`.
|
|
126
|
+
|
|
127
|
+
## 7. Verify this runbook is not stale
|
|
128
|
+
|
|
129
|
+
```bash
|
|
130
|
+
# every command / gate this runbook names still exists:
|
|
131
|
+
test -f src/scripts/release.ts && test -f .github/workflows/release.yml
|
|
132
|
+
grep -q "release-prepare" Taskfile.yml
|
|
133
|
+
./scripts-run src/scripts/check_release_pr_shape --help >/dev/null 2>&1 || true
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
A written-steps-only **dry run** (cut a no-op release following ONLY this doc,
|
|
137
|
+
no tribal knowledge) is the real freshness test — see the roadmap's Phase 3
|
|
138
|
+
dry-run step (maintainer-run; every gap found is a fix to this file).
|
|
@@ -0,0 +1,116 @@
|
|
|
1
|
+
# Second-brain scope — what it is, what it is not, what is measured
|
|
2
|
+
|
|
3
|
+
> The agent-config "second-brain" substrate is **agent-facing memory**, not a
|
|
4
|
+
> human personal-knowledge-management (PKM) tool. This page states the boundary
|
|
5
|
+
> honestly and binds every capability word to evidence. It follows the
|
|
6
|
+
> `docs/proof.md` § 4 discipline: the category column describes human-PKM only
|
|
7
|
+
> by what is publicly observable, never as a counter-claim to a named project.
|
|
8
|
+
>
|
|
9
|
+
> Roadmap: `road-to-second-brain-delta-proof`. Verdict of record:
|
|
10
|
+
> `agents/settings/contexts/second-brain-delta-verdict.md` (council 2026-07-07).
|
|
11
|
+
|
|
12
|
+
## The honest status (2026-07-09 — measured PASS, bounded)
|
|
13
|
+
|
|
14
|
+
The substrate is **built** (typed knowledge dirs, INDEX generator, retrieval
|
|
15
|
+
protocol, `hot_context_hook` working-memory continuity across compaction,
|
|
16
|
+
`fold_intake`, contradiction surfacing), and the cross-session recall *delta*
|
|
17
|
+
is now **measured** — a real, placebo-controlled lift, honestly scoped.
|
|
18
|
+
|
|
19
|
+
- **The measurement rig** — a deterministic multi-session recall corpus
|
|
20
|
+
(`internal/bench/second-brain/corpus/`) + a scorer
|
|
21
|
+
(`src/scripts/second_brain_score.ts`) with no model-in-the-loop grading.
|
|
22
|
+
- **The paired run (Phase 2)** — `memory-on` / `memory-off` / `placebo` on a
|
|
23
|
+
fixed host (claude-haiku-4-5), 9 tasks × 3 seeds (81 calls). Result:
|
|
24
|
+
memory-on **27/27**, no-memory **10/27**, equal-byte placebo **9/27**;
|
|
25
|
+
memory-on beats BOTH, sign test **p = 0.031** for each pairing → **PASS**.
|
|
26
|
+
Full report: `internal/bench/reports/second-brain-delta.json`
|
|
27
|
+
(claim `second-brain-recall-lift`).
|
|
28
|
+
- **Where the lift lives** — it concentrates on the retrieval-accuracy tasks
|
|
29
|
+
where the prior fact is available ONLY from memory (memory-on 3/3, baseline
|
|
30
|
+
0/3). It **ties** on the three tasks whose k+1 prompt already self-contains
|
|
31
|
+
the signal (a correction stated in-prompt, or a contradiction the prompt
|
|
32
|
+
names) — exactly where memory is not the only source. memory-on never loses.
|
|
33
|
+
|
|
34
|
+
### The scoping caveat (stated, not buried)
|
|
35
|
+
|
|
36
|
+
The Phase-2 delta is the **context-value upper bound**: the corpus is
|
|
37
|
+
one-fact-per-task, so memory-on injects the exact fact (perfect retrieval). It
|
|
38
|
+
shows that **the right prior fact, surfaced, lets the model answer — and that
|
|
39
|
+
this beats both no memory and equal-byte noise** (the placebo isolates mechanism
|
|
40
|
+
from mere extra context).
|
|
41
|
+
|
|
42
|
+
**The retrieval-precision follow-up removed the perfect-retrieval assumption**
|
|
43
|
+
(2026-07-09, claim `second-brain-retrieval-precision`,
|
|
44
|
+
`internal/bench/reports/second-brain-retrieval.json`). Against a populated store
|
|
45
|
+
whose distractors deliberately share query keywords with the needed decision,
|
|
46
|
+
the REAL `memory_lookup` retrieval recalls the needed decision into the top-5 in
|
|
47
|
+
**9/9** cases, and the model disambiguates it from the co-injected confusers —
|
|
48
|
+
**retrieval-on 27/27 vs no-memory 5/27 and vs equal-count placebo 5/27**
|
|
49
|
+
(sign test p=0.008 both). **Named limit (honest):** the keyword scorer
|
|
50
|
+
**recalls but does not rank** — mean tie-set 3.3 entries share the top score,
|
|
51
|
+
ties broken by store order, not relevance. So recall into the top-k holds *at
|
|
52
|
+
this scale*; once more than k entries share a keyword it would degrade — which
|
|
53
|
+
is exactly the discrimination gap the SQLite-FTS5 activation path (ADR-116)
|
|
54
|
+
exists to close. The public claims are scoped to exactly this.
|
|
55
|
+
|
|
56
|
+
**Second named limit (separator recall, found 2026-07-10 in a labelled replay
|
|
57
|
+
of real maintainer queries):** `_score()` matches raw substrings, so a query
|
|
58
|
+
key whose token separator differs from the stored field misses entirely —
|
|
59
|
+
`ai_council`, `roadmap-progress`, `force-push` score 0 against space-normalized
|
|
60
|
+
`key` fields that spaced variants (`ai council`) hit at 0.8. The precision
|
|
61
|
+
measurement above did not probe this class (its confusers share exact
|
|
62
|
+
keywords), so it is scoped OUT of the precision claim. Deliberately not
|
|
63
|
+
hot-fixed: naive scoring is locked until the scale tripwire (the archived
|
|
64
|
+
retrieval-economy roadmap's D5), and the FTS5 unicode61 tokenizer splits on
|
|
65
|
+
separators, closing this structurally at activation time. Reproduce:
|
|
66
|
+
`memory_lookup --types product-rules --key "ai_council"` vs `--key "ai council"`
|
|
67
|
+
against any populated store.
|
|
68
|
+
|
|
69
|
+
## What it IS (measured lift, bounded as above)
|
|
70
|
+
|
|
71
|
+
| Capability | What it does | Evidence status |
|
|
72
|
+
|---|---|---|
|
|
73
|
+
| Working-memory continuity | `hot_context_hook` re-injects a bounded, deterministic cache across session boundaries and Claude Code compaction | mechanism shipped; recall lift **measured (bounded PASS)** |
|
|
74
|
+
| Promotable knowledge cards/pages | typed dirs + INDEX + retrieval protocol, with redaction + team-share gate | mechanism shipped; recall lift **measured (bounded PASS)** |
|
|
75
|
+
| Contradiction surfacing | a session that contradicts a prior decision is flagged on promote | mechanism shipped; ties in-prompt, lift where memory is the only source |
|
|
76
|
+
|
|
77
|
+
## What it is NOT
|
|
78
|
+
|
|
79
|
+
- **Not a human-browsable knowledge graph.** There is no `.obsidian/` config,
|
|
80
|
+
no enforced `[[wikilink]]` convention, no link-density value thesis. An
|
|
81
|
+
editable vault view was evaluated and **rejected** (the dual-write hazard:
|
|
82
|
+
an external edit that never commits silently diverges the INDEX and
|
|
83
|
+
recurrence counters). Browsability, if demanded, is a read-only static
|
|
84
|
+
render — not an editable vault.
|
|
85
|
+
- **Not an Obsidian (or any named PKM) replacement.** The category — a
|
|
86
|
+
human Markdown graph browsed in an editor, value proportional to
|
|
87
|
+
link-density, hybrid human-facing retrieval — is a *different product* for a
|
|
88
|
+
*different consumer* (a person, not an agent). This package writes
|
|
89
|
+
agent-facing memory; it does not serve a human graph browser.
|
|
90
|
+
- **Not an unbounded accuracy multiplier** — the recall lift is measured but
|
|
91
|
+
scoped to the context-value upper bound (see honest status), not proven
|
|
92
|
+
retrieval precision under a large store.
|
|
93
|
+
|
|
94
|
+
## Interop, not competition (export declined, on the record)
|
|
95
|
+
|
|
96
|
+
The Phase-2 paired run did show a real (bounded) lift, which unlocks the
|
|
97
|
+
optional one-way export (promoted cards → Obsidian-compatible Markdown). It is
|
|
98
|
+
**deliberately not built**: the delta-verdict of record
|
|
99
|
+
(`agents/settings/contexts/second-brain-delta-verdict.md`, council 2026-07-07)
|
|
100
|
+
**rejected** Obsidian compatibility — the dual-write hazard (an external edit
|
|
101
|
+
that never commits silently diverges the INDEX + recurrence counters) outweighs
|
|
102
|
+
the browsing convenience, and the measured lift is a bounded context-value
|
|
103
|
+
result, not a retrieval-precision proof that would justify a new external
|
|
104
|
+
surface. If a human-PKM bridge is ever wanted, the sanctioned path is a
|
|
105
|
+
read-only static render, not an editable vault. Positioning stays: agent-config
|
|
106
|
+
writes agent-facing memory; it does not replace a human PKM.
|
|
107
|
+
|
|
108
|
+
## Verify
|
|
109
|
+
|
|
110
|
+
```bash
|
|
111
|
+
./scripts-run src/scripts/second_brain_score # corpus summary
|
|
112
|
+
./scripts-run src/scripts/second_brain_score --dry-run # scorer correct + discriminating, no spend
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
The paired-run delta, when authorized, pins to
|
|
116
|
+
`internal/bench/reports/` and updates `docs/CLAIMS.md` + `docs/benchmark.md`.
|