@tangle-network/agent-knowledge 4.0.1 → 5.0.0

package/AGENTS.md

@@ -2,22 +2,31 @@
 
 Use this package when an agent needs persistent, source-grounded knowledge that improves over time.
 
-## Repo layering — this package depends on agent-eval, never the reverse
+## Package layering
 
 ```
-agent-knowledge (this repo) ─┐
-                             ├──► agent-eval (substrate — the bottom)
-agent-runtime ───────────────┘
+agent-runtime
+      |
+agent-knowledge
+    /       \
+agent-eval  agent-interface
 ```
 
-**Rule: agent-knowledge depends on agent-eval. agent-eval MUST NOT import from agent-knowledge.** No upward imports, no peerDependency declaration in agent-eval pointing at agent-knowledge. Substrate primitives that agent-knowledge needs (`AnalystFinding`, `RunRecord`, optimizer types, release-confidence types) live in agent-eval; this repo consumes them.
+Imports flow downward in this diagram.
+`agent-knowledge` may import `agent-eval` and `agent-interface`, but it must not import `agent-runtime`.
+Agent-powered work enters this package through callbacks.
+`agent-runtime` owns live agent execution and composes this package when a complete agent workflow is needed.
+`agent-eval` must not import `agent-knowledge`.
+Shared run and experiment types that knowledge needs live in `agent-eval`.
 
 Types that stay in THIS repo because they're knowledge-domain-shaped:
 - `KbStore`, `KnowledgeFragment`, `KnowledgeChange`
 - `KnowledgeDiscoveryDispatcher`, source adapters (`createCornellLiiSource`, `createIrsPublicationsSource`)
 - Freshness store + change-detection primitives
 
-**The test for "where does a type live?"** — does this concept make sense WITHOUT persistent knowledge or sourced fragments? If yes, it's substrate (agent-eval). If no, it's a knowledge-domain concept (stays here).
+**The test for "where does a type live?"**
+If the concept makes sense without persistent knowledge or sourced fragments, it belongs in `agent-eval` or `agent-interface`.
+Otherwise, it stays in this package.
 
 ## Rules
 
@@ -66,42 +75,50 @@ The parser rejects absolute paths, `..`, control characters, and writes outside
 
 ## Eval Boundary
 
-Compare candidate knowledge bases on an actual task corpus by running an `@tangle-network/agent-eval` improvement loop (`runImprovementLoop`) over the variants; each run is scored into a `RunRecord`.
+Use a complete `OptimizationMethod` from `@tangle-network/agent-eval` with `runRetrievalImprovementLoop()`, `runRagOptimization()`, `optimizeKnowledgeBasePolicy()`, or `runAgentMemoryImprovement()`.
+The method owns candidate search and resume compatibility.
+This package owns serialized knowledge candidates, real KB or memory adapters, isolated data partitions, and safe activation.
 
 Use `knowledgeReleaseReport()` before promotion. It folds the candidate and baseline `RunRecord[]` (plus optional traces and the gate decision) into `agent-eval` release confidence evidence.
 
 ## Integration Boundaries
 
-- Use `KbStore` for storage. Implement D1 in the consuming app when needed.
-- Use `KnowledgeDiscoveryDispatcher` for research workers. Production apps should wire this to their own swarm/fleet runtime.
+- Use `KbStore` for storage. Applications may provide any durable backend that implements it.
+- Use `KnowledgeDiscoveryDispatcher` for research workers. Applications should connect it to their own runtime.
 - Do not bypass `lint` or `validate` before using generated knowledge in an agent.
 
 ## Pluggable Sources + Freshness + Changes
 
 Agents that need to stay current against external authorities should compose:
 
-- `createCornellLiiSource({ selectors })` — US Code + Wex from law.cornell.edu.
-- `createIrsPublicationsSource({ publications, revenueProcedures })` — IRS index + named pubs.
-- `createStateSosSource({ state, baseUrl, entities })` — generic state SOS adapter.
+- `createCornellLiiSource({ selectors })`: US Code and Wex from law.cornell.edu.
+- `createIrsPublicationsSource({ publications, revenueProcedures })`: IRS index and named publications.
+- `createStateSosSource({ state, baseUrl, entities })`: generic state SOS adapter.
 
-Every fetch returns `KnowledgeFragment[]` with `provenance.verifiable` indicating whether the authority was successfully authenticated. Refuse to cite fragments with `verifiable: false`.
+Every fetch returns `KnowledgeFragment[]` with `provenance.verifiable` indicating whether the configured URL returned an acceptable response and the expected content was extracted.
+This flag does not authenticate the publisher or cryptographically prove the content.
+Refuse to cite fragments with `verifiable: false`.
 
 Track per-tenant freshness with `createFileSystemFreshnessStore({ root })` and re-fetch only when `stale({ workspaceId, sourceId, ttlMs })` returns true.
 
-Diff snapshots with `detectChanges(prev, next)`. Each `KnowledgeChange` carries `affectedDimensions` — pass those to your eval scheduler to re-run only the relevant campaigns.
+Diff snapshots with `detectChanges(prev, next)`.
+Each `KnowledgeChange` carries `affectedDimensions`; pass those to your eval scheduler to run only the relevant campaigns again.
 
 ## Authorship
 
-Do not add `Co-Authored-By:` trailers (or any other AI-attribution lines) to commits, PR descriptions, or other artifacts in this repo. Author = the human running the session. Applies to every contributor, including AI agents and subagents — do not include the default Claude Code template trailer.
+Do not add `Co-Authored-By:` trailers or other AI-attribution lines to commits, PR descriptions, or repository artifacts.
+The author is the human running the session.
 
 ## Comment & doc discipline (no historical narrative)
 
-Comments describe **what the code does and why** — never what it used to do, what it replaced, which audit found a bug, or what the prior version looked like. History belongs in commit messages and PR descriptions, not the source tree.
+Comments describe **what the code does and why**.
+They must not describe what code used to do, what it replaced, which audit found a bug, or what a prior version looked like.
+History belongs in commit messages and PR descriptions.
 
 - Bad: `// replaces the inline retry loop`, `// fix for the silent-zero bug`, `// the 2yr rewrite added this`, `// audit fix`
-- Good: `// value: null when retries exhaust — callers must inspect succeeded`
+- Good: `// value is null when retries exhaust; callers must inspect succeeded`
 
-Applies to docstrings, README sections, SKILL.md, AGENTS.md, CLAUDE.md — anywhere the source tree carries prose.
+This applies anywhere the repository carries prose.
 
 ## No fallbacks. Fail loud.
 
@@ -110,4 +127,3 @@ Sloppy fallbacks corrupt every signal downstream. No silent zeros, no `?? defaul
 External-boundary calls (LLM, network, FS, subprocess) return *typed outcomes* (`{ succeeded, value, error }`). Callers MUST inspect `succeeded` before using `value`. Named, opted-in fallback rotations (`policy.fallbackModels: [...]`) are fine; deep `?? "kimi"` helpers are not.
 
 Full doctrine: `~/dotfiles/claude/AGENTS.md` → "No fallbacks. Fail loud."
-

package/CHANGELOG.md

@@ -1,5 +1,53 @@
 # Changelog
 
+## 5.0.0
+
+### Breaking Changes
+
+- Retrieval improvement now requires independent train, selection, and final scenarios plus an explicit complete `OptimizationMethod`.
+- Serialized retrieval and RAG optimization now requires an immutable `executionRef` covering candidate execution and scoring behavior.
+- Memory configuration improvement now requires a baseline configuration, a complete `OptimizationMethod`, and independent train, selection, and final histories.
+- The RAG lifecycle promotion callback is now `decidePromotion` and runs only after final evidence passes regression, provenance, and cost checks.
+- Knowledge improvement requires an immutable `implementationRef`, separates repeatable development evaluation from single-use final evaluation, and refuses to resume after interrupted final scoring.
+- Memory candidate factories no longer receive scenario, repetition, or seed identity and must report observed external charges through `recordExternalCost()`.
+- Answer-quality hooks require immutable evaluator identity, final scenario identity, and complete cost evidence.
+- Removed the public retrieval and memory proposer-search options; candidate generation and selection now belong to `agent-eval` methods.
+
+### Added
+
+- Added a shared serialized-candidate adapter for running complete `agent-eval` optimization methods with canonical candidate identity and untouched final comparison.
+- Added full RAG configuration optimization and KB maintenance policy optimization.
+- Added direct support for official GEPA and SkillOpt methods through the shared `OptimizationMethod` contract.
+- Added durable per-configuration memory candidate identities to prevent stale result reuse.
+- Added live activation verification so resumed memory runs reject configuration drift.
+- Added private execution contexts that expose memory operations, cancellation, and cost metering without evaluation labels.
+
+### Changed
+
+- Updated `@tangle-network/agent-eval` to `0.126.5` and `@tangle-network/agent-interface` to `0.32.0`.
+- Kept memory provider evaluations resumable and branch-isolated while moving search ownership to the supplied method.
+- Restricted immutable references to lowercase SHA-256 and full Git commit identities.
+
+## 4.1.0
+
+### Added
+
+- Added optional scenario input to `knowledgeReleaseReport()` so a required holdout can prove both scenario and run coverage.
+
+### Changed
+
+- Split knowledge-base improvement and RAG evaluation internals into focused modules while preserving public exports and implementation behavior.
+- Split the knowledge improvement tests into candidate, promotion, activation, and integrity suites with shared setup in one support module.
+- Removed unused development dependencies and internal-only exports.
+- Updated the test runner to Vitest 4 and Node type definitions to 26; TypeScript remains on 5.9 because tsup's declaration bundler does not yet support TypeScript 7.
+- Declared Node types explicitly in TypeScript configuration instead of relying on ambient type discovery.
+- Corrected the package dependency guide and RAG roadmap to reflect runtime-owned agent execution through `runKnowledgeImprovementJob()`.
+- Removed historical commentary and em dashes from repository documentation.
+
+### Fixed
+
+- Replaced the stale versioned HTTP user agent with a stable package identity and added request-header coverage.
+
 ## 4.0.1
 
 ### Changed