@tangle-network/agent-knowledge 1.11.1 → 1.12.0

package/README.md

@@ -7,15 +7,16 @@ This package turns raw sources and generated markdown knowledge into a versionab
 ## Contents
 
 - [Install](#install)
-- [Start here](#start-here) — pick CLI vs programmatic
-- [CLI](#cli) — `init` → `source-add` → `index` → `search` → `lint`
-- [Design](#design) — the invariants (immutable sources, cited claims, deterministic graph)
-- [Benchmark harness](#benchmark-harness) — BEIR/MTEB/qrels, RAG answer, hallucination, KB-improvement cases
-- [Agent-Eval integration](#agent-eval-integration) — retrieval eval + readiness bundles + release reports
-- [Memory adapters](#memory-adapters) — generic memory contract + Neo4j Agent Memory bridge
-- [Research loop](#research-loop) — `runKnowledgeResearchLoop` + control-loop adapter
-- [Runtime integration](#runtime-integration) — how agent runners plug into the pure KB loop
-- [Pluggable knowledge sources](#pluggable-knowledge-sources) — live authorities → eval re-runs
+- [Start here](#start-here): pick CLI vs programmatic
+- [Common uses](#common-uses): wiki, RAG, memory, new KBs, existing KBs, and parallel agents
+- [CLI](#cli): `init` → `source-add` → `index` → `search` → `lint`
+- [Design](#design): the invariants, including immutable sources, cited claims, and deterministic graph output
+- [Benchmark harness](#benchmark-harness): BEIR/MTEB/qrels, RAG answer, hallucination, KB-improvement cases
+- [Agent-Eval integration](#agent-eval-integration): retrieval eval, readiness bundles, and release reports
+- [Memory adapters](#memory-adapters): generic memory contract plus Neo4j Agent Memory bridge
+- [Research loop](#research-loop): `runKnowledgeResearchLoop` plus the control-loop adapter
+- [Runtime integration](#runtime-integration): how agent runners plug into the pure KB loop
+- [Pluggable knowledge sources](#pluggable-knowledge-sources): live authorities → eval re-runs
 
 ## Install
 
@@ -44,6 +45,18 @@ Two ways in, depending on what you're doing:
 
 Storage stays consumer-owned via `KbStore` (`MemoryKbStore`, `FileSystemKbStore`, or your own D1/Postgres). Every primitive below is source-grounded: claims cite immutable source records, and lint fails on un-grounded citations.
 
+## Common Uses
+
+| Use case | What this package owns | Front door |
+|---|---|---|
+| LLM wiki or docs KB | Source records, cited markdown pages, lint, readiness scoring, candidate promotion | `improveKnowledgeBase` or `runKnowledgeResearchLoop` |
+| RAG | Retrieval eval, source-span labels, answer quality checks, missing/stale/noisy-source diagnosis | `runRagKnowledgeImprovementLoop` |
+| Memory DB | Adapter contract that turns memory hits into source-like evidence and benchmark rows | `/memory` plus `runMemoryAdapterBenchmark` |
+| From scratch | Empty KB layout, source ingestion, write-block application, indexing, readiness checks | CLI `init` or `runKnowledgeResearchLoop` |
+| Existing KB | Candidate workspace, resume state, base-hash conflict check, promotion only after evals pass | `improveKnowledgeBase` |
+| Parallel agents | Per-run locks, isolated candidates, `promote: false` handoff, retry with the same `runId` | `improveKnowledgeBase` with runtime workers |
+| One-call agent job | Runtime supervisor plus all KB mechanics above | `runKnowledgeImprovementJob` from `@tangle-network/agent-runtime/knowledge` |
+
 ## CLI
 
 ```bash
@@ -122,7 +135,7 @@ from `@tangle-network/agent-knowledge`.
   `rrfScore` are the raw reciprocal-rank-fusion value (typically 0.01–0.05);
   use them when intent matters or when fusing across queries.
   `normalizedScore` is the same value scaled into [0, 1] relative to the top
-  hit *in this result set* (top hit = 1, others = score / topScore) — use it
+  hit *in this result set* (top hit = 1, others = score / topScore). Use it
   when comparing against natural confidence thresholds. The normalization is
   within-set ranking, not a cross-query absolute confidence.
 - Release confidence uses `@tangle-network/agent-eval` release gates (`evaluateReleaseConfidence`) instead of reimplementing them.
@@ -229,7 +242,7 @@ This is the path for Neo4j Agent Memory, Mem0, Zep, Letta, Graphiti, a vector st
 
 ## Agent-Eval Integration
 
-Use `ragAnswerQualityJudge` or `createRagAnswerQualityHook` when the product already has answer traces and needs SOTA-style RAG scoring without rebuilding metrics.
+Use `ragAnswerQualityJudge` or `createRagAnswerQualityHook` when the product already has answer traces and needs RAG answer scoring without rebuilding metrics.
 The built-in checks are deterministic and general; pass external scores from Ragas, DeepEval, TruLens, RAGChecker, or a custom evaluator when you want model-based judging.
 
 ```ts
@@ -500,13 +513,13 @@ with a differentiated worker/driver split over ONE knowledge base: the `worker`
 does primary research (discovers sources, proposes pages for the open gaps); the
 `driver` verifies each candidate source before it commits, optionally gap-fills
 with its own pass (`driverResearches: true`), and gates on the readiness check.
-Both are yours (no creds) — the loop owns the deterministic mechanics (indexing,
+Both are yours (no creds). The loop owns the deterministic mechanics (indexing,
 applying write blocks, scoring readiness) and stops once no blocking gap remains.
 
 Does the verifying driver actually earn its keep? See
 [docs/two-agent-research-ab.md](docs/two-agent-research-ab.md) for an equal-compute
 A/B (9 ML topics, `glm-5.2`): the two-agent loop admits ~2.33 fewer sources per topic
-at identical coverage — though most of that win is de-duplication, not relevance
+at identical coverage, though most of that win is de-duplication, not relevance
 filtering. Honest caveats and how to reproduce included.
 
 ```ts
@@ -544,7 +557,22 @@ await runTwoAgentResearchLoop({
 
 `agent-knowledge` owns knowledge state and measurement.
 It deliberately does not own an agent runner.
-Live agent orchestration belongs in `@tangle-network/agent-runtime`, which can call `improveKnowledgeBase` by passing an `updateKnowledge` callback:
+Live agent orchestration belongs in `@tangle-network/agent-runtime`.
+Use the one-call runtime job when you want agents, candidate workspaces, readiness checks, promotion, and spend measurement wired together:
+
+```ts
+import { runKnowledgeImprovementJob } from '@tangle-network/agent-runtime/knowledge'
+
+await runKnowledgeImprovementJob({
+  root: './kb',
+  goal: 'Build a grounded onboarding wiki for billing support',
+  readinessSpecs,
+  budget: { maxIterations: 8, maxTokens: 120_000, maxUsd: 10 },
+  backend,
+})
+```
+
+Use `improveKnowledgeBase` directly when your product already owns the agent runner and only needs the pure KB mechanics:
 
 ```ts
 import { improveKnowledgeBase } from '@tangle-network/agent-knowledge'
@@ -574,26 +602,26 @@ score = 0.4 · citation_density
       + 0.2 · gap_coverage
 ```
 
-The output preserves agent intelligence — `items`, `citations`,
+The output preserves agent intelligence: `items`, `citations`,
 `proposedWrites` are typed; `gaps`, `notes`, and any extras the agent
 emitted land in `raw` rather than getting dropped.
 
 ## Pluggable Knowledge Sources
 
 Static knowledge rots. Authorities like Cornell LII, the IRS, and state
-Secretaries of State change without warning — a ruling vacates an FTC
+Secretaries of State change without warning. A ruling vacates an FTC
 non-compete rule, a CFR section renumbers, a state replaces Beverly-Killea
 with RULLCA. The `@tangle-network/agent-knowledge/sources` subpath ships
 three primitives that bridge "live authority" → "eval re-runs":
 
-- `KnowledgeSource` — pluggable contract (`fetch(opts) → KnowledgeFragment[]`).
+- `KnowledgeSource`: pluggable contract (`fetch(opts) → KnowledgeFragment[]`).
   Every fragment carries `provenance` (URL, source-attested timestamp,
   jurisdiction, `verifiable` flag) and `dimensionHints` (which eval
   dimensions a change in this fragment should re-score).
-- `KnowledgeFreshnessStore` — per-`(workspaceId, sourceId)` last-refresh
+- `KnowledgeFreshnessStore`: per-`(workspaceId, sourceId)` last-refresh
   tracker. Filesystem adapter ships in-package; D1 / Postgres adapter
   scaffold is shipped as `createD1FreshnessStoreStub(adapter)`.
-- `detectChanges(prev, next)` — diffs two fragment snapshots, emits
+- `detectChanges(prev, next)`: diffs two fragment snapshots, emits
   `KnowledgeChange[]` tagged with the affected eval dimensions so a cron
   scheduler knows exactly which campaigns to re-run.
 
@@ -623,7 +651,7 @@ const sources = [
     publications: ['p15', 'p17', 'p463'],
     revenueProcedures: [],
   }),
-  // Generic state SOS adapter — one config per state you need tracked.
+  // Generic state SOS adapter: one config per state you need tracked.
   createStateSosSource({
     state: 'CA',
     baseUrl: 'https://www.sos.ca.gov',

package/dist/index.d.ts

@@ -937,6 +937,42 @@ interface KnowledgeDiscoveryDispatcher {
         signal?: AbortSignal;
     }): Promise<DiscoveryResult[]>;
 }
+type DiscoveryLoopStopReason = 'complete' | 'max-rounds' | 'max-tasks' | 'aborted';
+interface DiscoveryLoopRound {
+    round: number;
+    tasks: DiscoveryTask[];
+    results: DiscoveryResult[];
+    queuedFollowUps: DiscoveryTask[];
+}
+interface DiscoveryLoopResult {
+    stopReason: DiscoveryLoopStopReason;
+    tasksDispatched: number;
+    results: DiscoveryResult[];
+    rounds: DiscoveryLoopRound[];
+    /** Tasks retained, not dropped, when a configured limit stops the loop. */
+    pendingTasks: DiscoveryTask[];
+    /** Structurally identical task identities ignored to prevent cycles. */
+    duplicateTaskIds: string[];
+}
+interface RunDiscoveryLoopOptions {
+    dispatcher: KnowledgeDiscoveryDispatcher;
+    initialTasks: readonly DiscoveryTask[];
+    /** Maximum follow-up depth including the initial dispatch. Default 3. */
+    maxRounds?: number;
+    /** Maximum tasks dispatched across all rounds. Default 24. */
+    maxTasks?: number;
+    /** Forwarded to the dispatcher. Default 4. */
+    concurrency?: number;
+    signal?: AbortSignal;
+    onRound?: (round: DiscoveryLoopRound) => Promise<void> | void;
+}
+/**
+ * Dispatch discovery tasks and recursively pursue worker-proposed follow-ups.
+ *
+ * The runner owns only bounded scheduling and accounting. Workers own search,
+ * source verification, and domain-specific candidate shapes.
+ */
+declare function runDiscoveryLoop(options: RunDiscoveryLoopOptions): Promise<DiscoveryLoopResult>;
 declare function createLocalDiscoveryDispatcher(worker: KnowledgeDiscoveryWorker): KnowledgeDiscoveryDispatcher;
 
 interface KnowledgeEventQuery {
@@ -2480,4 +2516,4 @@ declare function normalizeLinkTarget(target: string): string;
 declare function isSafeKnowledgePath(path: string, allowedPrefixes?: string[]): boolean;
 declare function parseKnowledgeWriteBlocks(text: string, allowedPrefixes?: string[]): KnowledgeWriteParseResult;
 
-export { type AdaptiveDecision, type AdaptiveDriverOptions, type AdaptiveResearchDriver, type AdaptiveStats, type AddSourceOptions, type AddSourceTextInput, type ApplyWriteBlocksResult, type BuildEvalKnowledgeBundleOptions, type ChunkingOptions, type ClaimGroundingDriverOptions, type CompanyEvalCase, type D1Adapter, type DedupReason, type DeepQuestion, type DeepQuestionKind, type DefineReadinessSpecInput, type DetectChangesOptions, type DetectChangesResult, type DiscoveryResult, type DiscoveryTask, type DriverResearchContext, type EvalKnowledgeBundleBuildResult, type EvaluateKnowledgeBaseReadinessOptions, type ExpectedGroup, type ExternalRagEvalScore, type FactResult, type FileSystemFreshnessStoreOptions, FileSystemKbStore, type FreshnessKey, type FreshnessMark, type FreshnessRecord, type FreshnessTtl, type GroundClaimOptions, type GroundingResult, type KbStore, KnowledgeBaseCandidateSchema, type KnowledgeBaseQualityOptions, type KnowledgeBaseQualityReport, type KnowledgeBaseReadinessEvaluation, type KnowledgeChange, type KnowledgeChangeKind, type KnowledgeChunk, KnowledgeClaim, type KnowledgeControlLoopAction, type KnowledgeControlLoopActionResult, type KnowledgeControlLoopAdapter, type KnowledgeControlLoopAdapterOptions, type KnowledgeControlLoopState, type KnowledgeDiscoveryDispatcher, type KnowledgeDiscoveryWorker, KnowledgeEvent, type KnowledgeEventQuery, KnowledgeEventSchema, KnowledgeEventType, type KnowledgeExplanation, KnowledgeFragment, type KnowledgeFreshnessStore, type KnowledgeGap, KnowledgeGraph, KnowledgeGraphEdgeSchema, KnowledgeGraphNodeSchema, type KnowledgeImprovementCandidateRecord, type KnowledgeImprovementEvaluationInput, type KnowledgeImprovementEvaluator, type KnowledgeImprovementLifecycleRecord, type KnowledgeImprovementMetric, type KnowledgeImprovementOptions, type KnowledgeImprovementResult, type KnowledgeImprovementRetrievalOptions, type KnowledgeImprovementRunState, type KnowledgeImprovementStatus, type KnowledgeImprovementUpdate, type KnowledgeImprovementUpdateInput, KnowledgeIndex, KnowledgeIndexSchema, type KnowledgeInspection, type KnowledgeLayout, KnowledgeLintFinding, KnowledgePage, KnowledgePageSchema, type KnowledgeProposal, KnowledgeProposalParseError, type KnowledgeReadinessSpec, KnowledgeRelease, type KnowledgeReleaseInput, type KnowledgeReleaseReport, type KnowledgeResearchLoopContext, type KnowledgeResearchLoopDecision, type KnowledgeResearchLoopResult, type KnowledgeResearchLoopStep, KnowledgeSearchResult, KnowledgeWriteBlock, KnowledgeWriteParseResult, type MaterialFact, type MaterialFactLens, type MaterialFactsResult, MemoryKbStore, type ParsedFrontmatter, type ProposeFromFindingsResult, READINESS_SPEC_DEFAULTS, type RagAnswerEvalArtifact, type RagAnswerEvalCase, type RagAnswerEvalScenario, type RagAnswerMetricSummary, type RagAnswerQualityHookOptions, type RagAnswerQualityInput, type RagAnswerQualityJudgeOptions, type RagAnswerQualityResult, type RagCalibrationOptions, type RagCalibrationResult, type RagDiagnosisInput, type RagEvalCitation, type RagEvalClaim, type RagEvalContext, type RagEvalMetricKey, type RagEvalProvider, type RagEvalSlice, type RagGapFinding, type RagGapKind, type RagGapSeverity, type RagKnowledgeAcquisitionInput, type RagKnowledgeImprovementPhase, type RagKnowledgeImprovementPhaseResult, type RagKnowledgeImprovementPhaseStatus, type RagKnowledgeResearchOptions, type RagKnowledgeUpdateInput, type RagKnowledgeUpdateResult, type RagPhaseInputBase, type RagPromotionInput, type RagPromotionResult, type RagRequiredContext, type RejectedSource, type ResearchContribution, type ResearchDriver, type ResearchDrivingDriver, type ResearchDrivingDriverOptions, type ResearchDrivingState, type ResearchDrivingSteer, type ResearchSourceProposal, type ResearchWorker, RetrievalConfig, type RouterClient, RouterError, type RouterUsage, type RunKnowledgeResearchLoopOptions, type RunRagKnowledgeImprovementLoopOptions, type RunRagKnowledgeImprovementLoopResult, RunRetrievalImprovementLoopOptions, RunRetrievalImprovementLoopResult, SCAFFOLD_PAGE_BASENAMES, type SourceAdapter, type SourceAdapterInput, type SourceAdapterOutput, SourceAnchorSchema, type SourceFreshnessInspection, SourceRecord, SourceRecordSchema, SourceRegistry, type SourceVerdict, type SourceVerificationContext, type TangleRouterOptions, type ThesisRunOptions, type ThesisRunResult, type ThesisTaskInput, type TrackedClaim, type TriageClass, type TwoAgentResearchLoopOptions, type TwoAgentResearchLoopResult, type TwoAgentResearchRound, type ValidateKnowledgeOptions, type ValidateKnowledgeResult, type VerifiedResearchLoopOptions, type VerifiedResearchLoopResult, type VerifiedResearchRound, type VerifyingDriverOptions, WIKILINK_REGEX, type WebResearchWorkerOptions, type WebSearchHit, type WorkerClaimDecorationOptions, type WorkerResearchContext, addSourcePath, addSourceText, applyKnowledgeWriteBlocks, applyKnowledgeWriteBlocksFile, buildEvalKnowledgeBundle, buildKnowledgeGraph, buildKnowledgeIndex, calibrateRagAnswerJudge, canonicalizeUrl, chunkMarkdown, citedClaimKey, citedClaimOf, contentKey, createAdaptiveResearchDriver, createClaimDecorator, createClaimGroundingVerifier, createCollectionResearchDriver, createD1FreshnessStoreStub, createFileSystemFreshnessStore, createKnowledgeControlLoopAdapter, createKnowledgeEvent, createLocalDiscoveryDispatcher, createRagAnswerQualityHook, createResearchDrivingDriver, createTangleRouterClient, createVerifyingResearchDriver, createWebResearchWorker, defineReadinessSpec, detectChanges, diagnoseRagAnswerFailure, evaluateKnowledgeBaseReadiness, explainKnowledgeTarget, extractWikilinks, formatFrontmatter, gradeCompanyAgainstText, gradeFactAgainstText, groundClaimInText, hashKnowledgeBase, improveKnowledgeBase, initKnowledgeBase, inspectKnowledgeIndex, investmentThesisSet, isSafeKnowledgePath, isScaffoldPath, kbIndexToText, knowledgeImprovementRunDir, knowledgeImprovementRunId, knowledgeReleaseReport, layoutFor, lensDistribution, lintKnowledgeIndex, loadKnowledgeImprovementState, loadKnowledgePages, loadSourceRegistry, materialFactsSurfaced, materialFactsSurfacedInText, mediaTypeFor, normalizeExternalRagScores, normalizeLinkTarget, parseFrontmatter, parseKnowledgeWriteBlocks, proposeFromFinding, proposeFromFindings, ragAnswerQualityJudge, reciprocalRankFusion, runInvestmentThesisTask, runKnowledgeResearchLoop, runRagKnowledgeImprovementLoop, runTwoAgentResearchLoop, runVerifiedResearchLoop, scoreKnowledgeBaseIndex, scoreRagAnswerArtifact, searchKnowledge, sha256, slugify, sourceMatchesGaps, sourceRegistryPath, stableId, stripFrontmatter, textSourceAdapter, thesisReadinessSpecs, toDeepEvalTestCases, toRagCheckerRecords, toRagasEvaluationRows, toTruLensRecords, tokenizeQuery, totalMaterialFacts, triageSource, validateKnowledgeIndex, withCitedClaim, writeJson, writeKnowledgeIndex, writeSourceRegistry };
+export { type AdaptiveDecision, type AdaptiveDriverOptions, type AdaptiveResearchDriver, type AdaptiveStats, type AddSourceOptions, type AddSourceTextInput, type ApplyWriteBlocksResult, type BuildEvalKnowledgeBundleOptions, type ChunkingOptions, type ClaimGroundingDriverOptions, type CompanyEvalCase, type D1Adapter, type DedupReason, type DeepQuestion, type DeepQuestionKind, type DefineReadinessSpecInput, type DetectChangesOptions, type DetectChangesResult, type DiscoveryLoopResult, type DiscoveryLoopRound, type DiscoveryLoopStopReason, type DiscoveryResult, type DiscoveryTask, type DriverResearchContext, type EvalKnowledgeBundleBuildResult, type EvaluateKnowledgeBaseReadinessOptions, type ExpectedGroup, type ExternalRagEvalScore, type FactResult, type FileSystemFreshnessStoreOptions, FileSystemKbStore, type FreshnessKey, type FreshnessMark, type FreshnessRecord, type FreshnessTtl, type GroundClaimOptions, type GroundingResult, type KbStore, KnowledgeBaseCandidateSchema, type KnowledgeBaseQualityOptions, type KnowledgeBaseQualityReport, type KnowledgeBaseReadinessEvaluation, type KnowledgeChange, type KnowledgeChangeKind, type KnowledgeChunk, KnowledgeClaim, type KnowledgeControlLoopAction, type KnowledgeControlLoopActionResult, type KnowledgeControlLoopAdapter, type KnowledgeControlLoopAdapterOptions, type KnowledgeControlLoopState, type KnowledgeDiscoveryDispatcher, type KnowledgeDiscoveryWorker, KnowledgeEvent, type KnowledgeEventQuery, KnowledgeEventSchema, KnowledgeEventType, type KnowledgeExplanation, KnowledgeFragment, type KnowledgeFreshnessStore, type KnowledgeGap, KnowledgeGraph, KnowledgeGraphEdgeSchema, KnowledgeGraphNodeSchema, type KnowledgeImprovementCandidateRecord, type KnowledgeImprovementEvaluationInput, type KnowledgeImprovementEvaluator, type KnowledgeImprovementLifecycleRecord, type KnowledgeImprovementMetric, type KnowledgeImprovementOptions, type KnowledgeImprovementResult, type KnowledgeImprovementRetrievalOptions, type KnowledgeImprovementRunState, type KnowledgeImprovementStatus, type KnowledgeImprovementUpdate, type KnowledgeImprovementUpdateInput, KnowledgeIndex, KnowledgeIndexSchema, type KnowledgeInspection, type KnowledgeLayout, KnowledgeLintFinding, KnowledgePage, KnowledgePageSchema, type KnowledgeProposal, KnowledgeProposalParseError, type KnowledgeReadinessSpec, KnowledgeRelease, type KnowledgeReleaseInput, type KnowledgeReleaseReport, type KnowledgeResearchLoopContext, type KnowledgeResearchLoopDecision, type KnowledgeResearchLoopResult, type KnowledgeResearchLoopStep, KnowledgeSearchResult, KnowledgeWriteBlock, KnowledgeWriteParseResult, type MaterialFact, type MaterialFactLens, type MaterialFactsResult, MemoryKbStore, type ParsedFrontmatter, type ProposeFromFindingsResult, READINESS_SPEC_DEFAULTS, type RagAnswerEvalArtifact, type RagAnswerEvalCase, type RagAnswerEvalScenario, type RagAnswerMetricSummary, type RagAnswerQualityHookOptions, type RagAnswerQualityInput, type RagAnswerQualityJudgeOptions, type RagAnswerQualityResult, type RagCalibrationOptions, type RagCalibrationResult, type RagDiagnosisInput, type RagEvalCitation, type RagEvalClaim, type RagEvalContext, type RagEvalMetricKey, type RagEvalProvider, type RagEvalSlice, type RagGapFinding, type RagGapKind, type RagGapSeverity, type RagKnowledgeAcquisitionInput, type RagKnowledgeImprovementPhase, type RagKnowledgeImprovementPhaseResult, type RagKnowledgeImprovementPhaseStatus, type RagKnowledgeResearchOptions, type RagKnowledgeUpdateInput, type RagKnowledgeUpdateResult, type RagPhaseInputBase, type RagPromotionInput, type RagPromotionResult, type RagRequiredContext, type RejectedSource, type ResearchContribution, type ResearchDriver, type ResearchDrivingDriver, type ResearchDrivingDriverOptions, type ResearchDrivingState, type ResearchDrivingSteer, type ResearchSourceProposal, type ResearchWorker, RetrievalConfig, type RouterClient, RouterError, type RouterUsage, type RunDiscoveryLoopOptions, type RunKnowledgeResearchLoopOptions, type RunRagKnowledgeImprovementLoopOptions, type RunRagKnowledgeImprovementLoopResult, RunRetrievalImprovementLoopOptions, RunRetrievalImprovementLoopResult, SCAFFOLD_PAGE_BASENAMES, type SourceAdapter, type SourceAdapterInput, type SourceAdapterOutput, SourceAnchorSchema, type SourceFreshnessInspection, SourceRecord, SourceRecordSchema, SourceRegistry, type SourceVerdict, type SourceVerificationContext, type TangleRouterOptions, type ThesisRunOptions, type ThesisRunResult, type ThesisTaskInput, type TrackedClaim, type TriageClass, type TwoAgentResearchLoopOptions, type TwoAgentResearchLoopResult, type TwoAgentResearchRound, type ValidateKnowledgeOptions, type ValidateKnowledgeResult, type VerifiedResearchLoopOptions, type VerifiedResearchLoopResult, type VerifiedResearchRound, type VerifyingDriverOptions, WIKILINK_REGEX, type WebResearchWorkerOptions, type WebSearchHit, type WorkerClaimDecorationOptions, type WorkerResearchContext, addSourcePath, addSourceText, applyKnowledgeWriteBlocks, applyKnowledgeWriteBlocksFile, buildEvalKnowledgeBundle, buildKnowledgeGraph, buildKnowledgeIndex, calibrateRagAnswerJudge, canonicalizeUrl, chunkMarkdown, citedClaimKey, citedClaimOf, contentKey, createAdaptiveResearchDriver, createClaimDecorator, createClaimGroundingVerifier, createCollectionResearchDriver, createD1FreshnessStoreStub, createFileSystemFreshnessStore, createKnowledgeControlLoopAdapter, createKnowledgeEvent, createLocalDiscoveryDispatcher, createRagAnswerQualityHook, createResearchDrivingDriver, createTangleRouterClient, createVerifyingResearchDriver, createWebResearchWorker, defineReadinessSpec, detectChanges, diagnoseRagAnswerFailure, evaluateKnowledgeBaseReadiness, explainKnowledgeTarget, extractWikilinks, formatFrontmatter, gradeCompanyAgainstText, gradeFactAgainstText, groundClaimInText, hashKnowledgeBase, improveKnowledgeBase, initKnowledgeBase, inspectKnowledgeIndex, investmentThesisSet, isSafeKnowledgePath, isScaffoldPath, kbIndexToText, knowledgeImprovementRunDir, knowledgeImprovementRunId, knowledgeReleaseReport, layoutFor, lensDistribution, lintKnowledgeIndex, loadKnowledgeImprovementState, loadKnowledgePages, loadSourceRegistry, materialFactsSurfaced, materialFactsSurfacedInText, mediaTypeFor, normalizeExternalRagScores, normalizeLinkTarget, parseFrontmatter, parseKnowledgeWriteBlocks, proposeFromFinding, proposeFromFindings, ragAnswerQualityJudge, reciprocalRankFusion, runDiscoveryLoop, runInvestmentThesisTask, runKnowledgeResearchLoop, runRagKnowledgeImprovementLoop, runTwoAgentResearchLoop, runVerifiedResearchLoop, scoreKnowledgeBaseIndex, scoreRagAnswerArtifact, searchKnowledge, sha256, slugify, sourceMatchesGaps, sourceRegistryPath, stableId, stripFrontmatter, textSourceAdapter, thesisReadinessSpecs, toDeepEvalTestCases, toRagCheckerRecords, toRagasEvaluationRows, toTruLensRecords, tokenizeQuery, totalMaterialFacts, triageSource, validateKnowledgeIndex, withCitedClaim, writeJson, writeKnowledgeIndex, writeSourceRegistry };

package/dist/index.js

@@ -1006,6 +1006,71 @@ function createCollectionResearchDriver() {
 }
 
 // src/discovery.ts
+import { isDeepStrictEqual } from "util";
+async function runDiscoveryLoop(options) {
+  const maxRounds = positiveInteger(options.maxRounds ?? 3, "maxRounds");
+  const maxTasks = positiveInteger(options.maxTasks ?? 24, "maxTasks");
+  const concurrency = positiveInteger(options.concurrency ?? 4, "concurrency");
+  const known = /* @__PURE__ */ new Map();
+  const pending = [];
+  const duplicateTaskIds = /* @__PURE__ */ new Set();
+  enqueueTasks(options.initialTasks, known, pending, duplicateTaskIds);
+  const rounds = [];
+  const results = [];
+  let tasksDispatched = 0;
+  let aborted = false;
+  while (pending.length > 0 && rounds.length < maxRounds && tasksDispatched < maxTasks) {
+    if (options.signal?.aborted) {
+      aborted = true;
+      break;
+    }
+    const capacity = maxTasks - tasksDispatched;
+    const tasks = pending.splice(0, capacity);
+    let dispatched;
+    try {
+      dispatched = orderCompleteDispatch(
+        tasks,
+        await options.dispatcher.dispatch(tasks, {
+          concurrency,
+          signal: options.signal
+        })
+      );
+    } catch (cause) {
+      if (!options.signal?.aborted) throw cause;
+      pending.unshift(...tasks);
+      aborted = true;
+      break;
+    }
+    tasksDispatched += tasks.length;
+    results.push(...dispatched);
+    const queuedFollowUps = [];
+    for (const result of dispatched) {
+      enqueueTasks(result.followUpTasks ?? [], known, queuedFollowUps, duplicateTaskIds);
+    }
+    pending.push(...queuedFollowUps);
+    const round2 = {
+      round: rounds.length + 1,
+      tasks,
+      results: dispatched,
+      queuedFollowUps
+    };
+    rounds.push(round2);
+    await options.onRound?.(round2);
+    if (options.signal?.aborted) {
+      aborted = true;
+      break;
+    }
+  }
+  const stopReason = aborted ? "aborted" : pending.length === 0 ? "complete" : tasksDispatched >= maxTasks ? "max-tasks" : "max-rounds";
+  return {
+    stopReason,
+    tasksDispatched,
+    results,
+    rounds,
+    pendingTasks: pending,
+    duplicateTaskIds: [...duplicateTaskIds]
+  };
+}
 function createLocalDiscoveryDispatcher(worker) {
   return {
     async dispatch(tasks, options = {}) {
@@ -1020,12 +1085,55 @@ function createLocalDiscoveryDispatcher(worker) {
         }
       }
       await Promise.all(Array.from({ length: Math.min(concurrency, tasks.length) }, runNext));
-      return results.sort(
-        (a, b) => tasks.findIndex((task) => task.id === a.taskId) - tasks.findIndex((task) => task.id === b.taskId)
-      );
+      return orderCompleteDispatch(tasks, results);
     }
   };
 }
+function enqueueTasks(tasks, known, destination, duplicateTaskIds) {
+  for (const task of tasks) {
+    assertTask(task);
+    const prior = known.get(task.id);
+    if (!prior) {
+      known.set(task.id, task);
+      destination.push(task);
+      continue;
+    }
+    if (!isDeepStrictEqual(prior, task)) {
+      throw new Error(`Discovery task id '${task.id}' refers to conflicting tasks`);
+    }
+    duplicateTaskIds.add(task.id);
+  }
+}
+function assertTask(task) {
+  if (!task.id.trim()) throw new Error("Discovery task id must not be empty");
+  if (!task.goal.trim()) throw new Error(`Discovery task '${task.id}' goal must not be empty`);
+}
+function orderCompleteDispatch(tasks, results) {
+  const order = new Map(tasks.map((task, index) => [task.id, index]));
+  const received = /* @__PURE__ */ new Set();
+  for (const result of results) {
+    if (!order.has(result.taskId)) {
+      throw new Error(`Discovery dispatcher returned unknown task '${result.taskId}'`);
+    }
+    if (received.has(result.taskId)) {
+      throw new Error(`Discovery dispatcher returned task '${result.taskId}' more than once`);
+    }
+    received.add(result.taskId);
+  }
+  const missing = tasks.filter((task) => !received.has(task.id)).map((task) => task.id);
+  if (missing.length > 0) {
+    throw new Error(`Discovery dispatcher omitted tasks: ${missing.join(", ")}`);
+  }
+  return [...results].sort(
+    (a, b) => order.get(a.taskId) - order.get(b.taskId)
+  );
+}
+function positiveInteger(value, name) {
+  if (!Number.isInteger(value) || value < 1) {
+    throw new Error(`Discovery loop ${name} must be a positive integer`);
+  }
+  return value;
+}
 
 // src/eval-readiness.ts
 import {
@@ -4794,6 +4902,7 @@ export {
   retrievalHoldoutConfigHash,
   retrievalParameterSweepProposer,
   retrievalRecallJudge,
+  runDiscoveryLoop,
   runInvestmentThesisTask,
   runKnowledgeBenchmarkSuite,
   runKnowledgeResearchLoop,