@kontourai/flow-agents 0.3.0 → 1.0.0

package/kits/knowledge/docs/README.md

@@ -24,6 +24,15 @@ See [`store-contract.md`](store-contract.md) for the full specification. Quick r
 | `raw` | Unprocessed source material — excerpts, transcripts, URLs with notes. |
 | `compiled` | Normalized, editor-reviewed distillations of raw records. |
 | `concept` | Named ideas or principles that other records reference. |
+| `snapshot` | Bounded decision summary for a topic (Addendum A). |
+
+**Record status lifecycle**
+
+| Status | Meaning | Default |
+|---|---|---|
+| `active` | Live, part of the working set. | Yes (records without status field are treated as active). |
+| `implemented` | Decision was shipped; transitional state before archival. | No |
+| `retired` | Excluded from default working-set queries; history preserved. | No |
 
 **Mutation operations**
 
@@ -35,6 +44,8 @@ See [`store-contract.md`](store-contract.md) for the full specification. Quick r
 | `propose` | `agent`, `proposal` (non-empty) |
 | `apply` | `agent`, `new_body` (non-empty), `rationale` (non-empty) |
 | `reject` | `agent`, `reason` (non-empty) |
+| `supersede` | `agent`, `rationale` (non-empty), non-empty `supersededIds` array |
+| `retire` | `agent`, `rationale` (non-empty), `implementedByRef` (when `targetStatus="implemented"`) |
 
 Every mutation throws with `error.code === "MISSING_EVIDENCE"` when required evidence is absent.
 
@@ -127,9 +138,191 @@ and adapter infrastructure remain the foundation.
 
 ---
 
+## Decision Lifecycle — Retiring Records (S7)
+
+Implemented or obsolete records can be retired from the working set via the `knowledge.retire`
+flow. Retirement is **non-destructive**: the record body, links, and creation provenance remain
+intact; the record is simply excluded from the default working set.
+
+### Status transitions
+
+| From | To | Evidence required |
+|---|---|---|
+| `active` | `implemented` | `rationale` (non-empty) + `implementedByRef` (non-empty ref to implementing artifact) |
+| `active` | `retired` | `rationale` (non-empty) |
+| `implemented` | `retired` | `rationale` (non-empty) |
+| `retired` | *(any)* | Invalid — `retired` is terminal |
+
+### Working-set exclusion
+
+Retired records are excluded from:
+
+- `listByType(type)` — default query
+- `listByCategory(category, options)` — default query
+- `defaultSimilarityDetector` — default cluster candidates
+- `createVectorSimilarityDetector` — vector cluster candidates
+
+Add `{ includeRetired: true }` to any query to restore retired records.
+
+`get(id)` **always** returns the full record regardless of status.
+
+### Using the retire flow
+
+```js
+import { KnowledgeFlowRunner } from './adapters/flow-runner/index.js';
+
+const runner = new KnowledgeFlowRunner({ store, workspace });
+
+// Retire a compiled decision record that was implemented
+const result = await runner.retire(compiledId, {
+  targetStatus: 'implemented',
+  rationale: 'REST API shipped in v1.0 (PR #42).',
+  implementedByRef: 'https://github.com/org/repo/pull/42',
+  decision: 'apply',
+});
+
+// Retire an obsolete concept record
+await runner.retire(conceptId, {
+  targetStatus: 'retired',
+  rationale: 'Superseded by new architecture decision in ADR-007.',
+  decision: 'apply',
+});
+
+// Reject a retirement proposal (status unchanged)
+await runner.retire(recordId, {
+  targetStatus: 'retired',
+  rationale: 'Proposing retirement.',
+  decision: 'reject',
+  rejectReason: 'Still needed for reference.',
+});
+```
+
+### Accessing retired records with provenance
+
+```js
+// Always works — returns full record including retirement evidence
+const record = await store.get(retiredId);
+console.log(record.status);           // "retired"
+console.log(record.mutation_log);     // includes retire entry with rationale
+
+// Query all retired records of a type
+const allCompiled = await store.listByType('compiled', { includeRetired: true });
+
+// Get history from snapshot provenance
+const snapshot = await store.get(snapshotId);
+for (const srcId of snapshot.provenance.source_ids) {
+  const src = await store.get(srcId);   // works even if src is retired
+  console.log(src.id, src.status);
+}
+```
+
+---
+
 ## Non-Goals (this iteration)
 
 - Vector/semantic retrieval (parked as I10)
 - Multi-user concurrency
 - Store migrations
 - Personal-KB import (parked as I11)
+
+---
+
+## Similarity Detectors
+
+The `synthesize` and `consolidate` flows accept a pluggable `similarityDetector` option. A
+detector has the signature:
+
+```js
+async (concept: Record, candidates: Record[], store: KnowledgeStoreAdapter) => string[]
+```
+
+It receives the target concept, all compiled candidates, and the store; it returns the IDs of
+candidates that are similar enough to form a cluster. The `KnowledgeFlowRunner` uses the cluster
+as its evidence base — an empty cluster throws `MISSING_EVIDENCE` at the detect-cluster gate.
+
+### Choosing a detector
+
+| Detector | Best for | Tradeoff |
+|---|---|---|
+| `defaultSimilarityDetector` (built-in) | Fast, zero-config. Works well when records share a structured category taxonomy and inter-record wikilinks. | Relies on category prefixes and link-overlap (Jaccard ≥ 0.10). Misses semantic similarity across category boundaries. |
+| `createVectorSimilarityDetector` | Semantic clustering. Finds similar records regardless of how they were categorised. | Requires an embedding backend (ollama by default). Adds latency proportional to cluster size. |
+
+### Vector detector — ollama embedding
+
+The vector adapter lives at `adapters/similarity-vector/index.js`. It is zero-dependency and
+calls ollama's `/api/embed` endpoint via the built-in `fetch`.
+
+```js
+import { createVectorSimilarityDetector } from './adapters/similarity-vector/index.js';
+
+// Default: uses ollama at localhost:11434 with nomic-embed-text
+const detector = createVectorSimilarityDetector();
+
+// Or customise host, model, and threshold:
+const detector = createVectorSimilarityDetector({
+  host: 'http://localhost:11434',
+  model: 'nomic-embed-text',
+  threshold: 0.60,              // cosine similarity cutoff
+});
+
+// Pass to synthesize:
+await runner.synthesize(conceptId, {
+  proposedBody: '...',
+  rationale: '...',
+  similarityDetector: detector,
+});
+```
+
+**Starting ollama:**
+
+```bash
+ollama serve &
+ollama pull nomic-embed-text   # 274 MB, one-time pull
+```
+
+**Threshold guidance:**
+
+The default threshold of `0.60` is validated against `nomic-embed-text` (768-dim). Empirical
+scores observed in the eval suite:
+
+| Pair | Score |
+|---|---|
+| Semantically similar API design texts | ~0.77 |
+| Semantically unrelated (API vs. bread baking) | ~0.41 |
+
+A threshold of `0.60` cleanly separates these two classes. If your domain records are more
+homogeneous (narrow vocabulary, very similar boilerplate) you may need to raise the threshold
+to `0.70–0.80` to avoid over-clustering.
+
+### Fail-closed rationale
+
+The vector detector throws an `Error` with `code="EMBED_FAILURE"` rather than returning `[]`
+when the embedding call fails (network error, HTTP error, malformed response, wrong vector
+count). This is intentional.
+
+A detector that silently returns `[]` on infrastructure failure is indistinguishable from one
+that found no similar records. The result is a misleading `MISSING_EVIDENCE` at the detect-cluster
+gate, which looks like "this concept has no sources" rather than "the embedding service is down".
+
+Failing closed makes the infrastructure problem visible immediately, at the right level, with a
+clear error code. Operators can catch `EMBED_FAILURE` separately from `MISSING_EVIDENCE` and
+route them to different alerting channels.
+
+### Injecting a custom embed function
+
+For tests or alternative providers (OpenAI, Cohere, etc.), pass `embed` directly:
+
+```js
+const detector = createVectorSimilarityDetector({
+  embed: async (texts) => {
+    // texts: string[] — one per record (title + "\n" + body by default)
+    // must return: number[][] — one vector per input text
+    const response = await myEmbeddingAPI.embed(texts);
+    return response.vectors;
+  },
+  threshold: 0.70,
+});
+```
+
+The `embed` function is called once per `synthesize`/`consolidate` call with all texts in a
+single batch (concept first, then candidates).

package/kits/knowledge/docs/store-contract.md

@@ -524,3 +524,199 @@ Superseded snapshots MUST remain queryable:
 
 There is no `"archived"` or `"deleted"` status field; the supersession chain is expressed entirely
 through links and mutation log entries.
+
+---
+
+## Addendum B — Record Status Lifecycle (S7)
+
+### B.1 Status Field
+
+Every record envelope (§1.1) gains an optional `status` field:
+
+| Field | Type | Required | Default | Description |
+|---|---|---|---|---|
+| `status` | `"active"` \| `"implemented"` \| `"retired"` | no | `"active"` | Lifecycle status of the record. Records without a status field are treated as `"active"`. |
+
+`status` is a mutable field but MUST only change via the `retire` mutation op (§B.4).
+Direct field updates via the `update` op MUST NOT change `status`; `update` MUST ignore `status` if
+supplied in the fields argument.
+
+### B.2 Allowed Status Transitions
+
+| From | To | Op | Required Evidence |
+|---|---|---|---|
+| `"active"` | `"implemented"` | `retire` | `implementedByRef` (non-empty, references the implementing artifact/commit/PR) |
+| `"active"` | `"retired"` | `retire` | `rationale` (non-empty, explains obsolescence) |
+| `"implemented"` | `"retired"` | `retire` | `rationale` (non-empty) |
+
+No other transitions are permitted. Attempting an invalid transition MUST throw with
+`error.code === "MISSING_EVIDENCE"` and a human-readable `message`.
+
+Records in `"retired"` status have no further transitions — they are terminal.
+
+### B.3 Working-Set Exclusion
+
+Records with `status === "retired"` are EXCLUDED from the default working set:
+
+- `listByType(type)` returns only non-retired records by default.
+- `listByCategory(category, options?)` returns only non-retired records by default.
+- `defaultSimilarityDetector` considers only non-retired compiled records as candidates.
+- The vector similarity detector (`createVectorSimilarityDetector`) considers only non-retired
+  compiled records as candidates.
+
+All four filtering surfaces accept an `includeRetired: true` option (or equivalent flag on the
+similarity detector) to restore retired records to the result set.
+
+Retired records remain **fully queryable with provenance**:
+- `get(id)` always returns the full record regardless of status.
+- `listByType(type, { includeRetired: true })` returns all records of that type.
+- `listByCategory(category, { includeRetired: true })` returns all matching records.
+- The record's `mutation_log` carries the full retirement evidence.
+
+### B.4 `retire` Mutation Operation
+
+The `retire` op transitions a record from `"active"` or `"implemented"` to the target status.
+It NEVER deletes the record. The record body, links, and provenance remain intact.
+
+**Required fields:**
+
+| Field | Location | Description |
+|---|---|---|
+| `id` | argument | ID of the record to retire. |
+| `targetStatus` | argument | Target status: `"implemented"` or `"retired"`. |
+| `agent` | evidence | Agent performing the retirement. |
+| `rationale` | evidence | Non-empty string explaining why the record is being retired. Required for all target statuses. |
+
+**Conditional evidence fields:**
+
+| Field | Location | Condition | Description |
+|---|---|---|---|
+| `implementedByRef` | evidence | `targetStatus === "implemented"` | Non-empty reference to the implementing artifact (commit SHA, PR URL, issue number, etc.). |
+| `supersededByRef` | evidence | optional for `targetStatus === "retired"` | Reference to a superseding record or artifact. |
+
+**Rejection conditions:**
+- Record with `id` does not exist.
+- `targetStatus` is not `"implemented"` or `"retired"`.
+- Current status transition is invalid (see §B.2).
+- `rationale` is missing or empty.
+- `targetStatus === "implemented"` and `implementedByRef` is missing or empty.
+- Missing `agent` in evidence.
+
+**Post-conditions:**
+- Record `status` is updated to `targetStatus`.
+- Record `updated_at` is refreshed.
+- A mutation log entry (op=`"retire"`) is appended, carrying `targetStatus`, `rationale`,
+  and any supplied `implementedByRef` / `supersededByRef`.
+- The record body, `links`, and creation `provenance` are NOT changed.
+- `get(id)` returns the full record with the updated status.
+- `listByType(type)` (without `includeRetired`) no longer returns this record if
+  `targetStatus === "retired"`.
+
+### B.5 Adapter Contract Extension
+
+The adapter interface (§8) is extended:
+
+```ts
+interface KnowledgeStoreAdapter {
+  // ... existing methods ...
+  retire(id: string, targetStatus: "implemented" | "retired", evidence: RetireEvidence): Promise<void>;
+  listByType(type: RecordType, options?: { includeRetired?: boolean }): Promise<Record[]>;
+  listByCategory(category: string, options?: { prefix?: boolean; includeRetired?: boolean }): Promise<Record[]>;
+}
+```
+
+`RetireEvidence`:
+```ts
+interface RetireEvidence {
+  agent: string;
+  rationale: string;
+  implementedByRef?: string;   // required when targetStatus === "implemented"
+  supersededByRef?: string;    // optional
+  note?: string;
+}
+```
+
+### B.6 Provenance and History Guarantee
+
+Retired records MUST remain reachable from:
+- `get(id)` — always returns the full record.
+- `listByType(type, { includeRetired: true })` and `listByCategory(category, { includeRetired: true })`.
+- Snapshot `provenance.source_ids` — any snapshot that included the record in its cluster before
+  retirement retains the reference intact. The retired record can be retrieved via `get(sourceId)`.
+- The retirement `mutation_log` entry carries the full evidence of why and when the record was
+  retired and by whom.
+
+There is no deletion of records. Physical purge (if ever needed) is a separate, future policy
+hook not defined in this version.
+
+---
+
+## Addendum C — Person Record Type (Entity Cards)
+
+### C.1 `person` Record Type
+
+A `person` record is a first-class entity card for a named individual mentioned in the knowledge base.
+It participates fully in links, the graph index, and status lifecycle like any other record type.
+
+| Field | Notes |
+|---|---|
+| `type` | `"person"` |
+| `title` | The person's full name. Used for exact-match resolution. |
+| `body` | Structured prose with role and/or org: `**Role/Org:** <text>`. Merged on apply during card union. |
+| `tags` | May include `alias:<name>` entries for alternative names (nicknames, initials). |
+| `category` | Dot-separated category. The Obsidian adapter ignores this for routing — person records always land in `people/`. |
+
+### C.2 Link Kinds Extended
+
+| Kind | Direction | Meaning |
+|---|---|---|
+| `"appears-in"` | person → raw\|compiled | Person was mentioned in that record. |
+| `"person"` | compiled → person | Compiled record references a person card. |
+
+### C.3 Obsidian Adapter Layout
+
+The Obsidian store adapter (`adapters/obsidian-store`) places person records under a
+top-level `people/` folder regardless of category, so person cards are vault-global entities.
+
+```
+<storeRoot>/
+  people/
+    dana-smith.md
+    lee-wong.md
+  <category-as-path>/
+    <title-slug>.md      (concept, snapshot)
+    <sourcesDir>/
+      <title-slug>.md   (raw, compiled)
+```
+
+Person cards render an **Appears In** section listing all `appears-in` links as Obsidian wikilinks.
+Notes that reference people render a **People** section listing `person` links.
+
+### C.4 Alias Storage
+
+Aliases are stored in the `tags` array using the prefix `alias:`:
+
+```yaml
+tags: [alias:Dana S., alias:D. Smith]
+```
+
+Resolution checks both `title` and all `alias:*` tags for exact normalised-name match.
+
+### C.5 Entity Extraction (Flow Runner)
+
+The `KnowledgeFlowRunner.extractEntities(compiledId, options)` method:
+
+1. Runs the extractor (default: `defaultEntityExtractor`) against the compiled record and its source raws.
+2. For each mention, resolves via exact-name match (incl. aliases) or creates a new person card.
+3. Near-matches (same surname + initial) create a **separate** card with a `related` link labelled
+   `possible-duplicate` — no auto-merge.
+4. Writes bidirectional links: `person → raw+compiled` (kind `appears-in`) and `compiled → person` (kind `person`).
+
+### C.6 Card Merge
+
+Card merge uses the existing `propose → apply/reject` gate:
+
+- `KnowledgeFlowRunner.mergePerson(primaryId, duplicateId, options)`
+- **apply**: unions body, adds `alias:<duplicate title>` tag to primary, unions `appears-in` links, calls
+  `store.supersede(primaryId, [duplicateId])` to archive the duplicate (supersede-not-delete invariant).
+- **reject**: both cards remain byte-identical.

package/kits/knowledge/evals/contract-suite/suite.test.js

@@ -411,12 +411,17 @@ describe("Knowledge Kit Store Contract Suite", () => {
       );
     });
 
-    test("rejects non-concept concept_id", async () => {
-      const notConcept = await store.create({ type: "raw", title: "NC", body: "nc", category: "test", provenance: { agent: "tester" } });
+    test("propose accepts any record type as target (Addendum B: retire flow needs non-concept targets)", async () => {
+      // Addendum B (S7) extends propose to accept any record type as the target,
+      // enabling the retire flow to attach proposals to compiled/raw/snapshot records.
+      const rawTarget = await store.create({ type: "raw", title: "NC", body: "nc", category: "test", provenance: { agent: "tester" } });
       const pid = await store.create({ type: "raw", title: "P3", body: "p", category: "test", provenance: { agent: "tester" } });
-      await assertMissingEvidence(
-        () => store.propose(notConcept, pid, { agent: "tester", proposal: "change" }),
-        "propose non-concept target"
+      // Should NOT throw — all record types are valid proposal targets
+      await store.propose(rawTarget, pid, { agent: "tester", proposal: "retirement proposal" });
+      const { forward } = await store.getLinks(pid);
+      assert.ok(
+        forward.some((l) => l.target_id === rawTarget && l.kind === "proposes"),
+        "propose on raw record creates proposes link (Addendum B extension)"
       );
     });
 

package/kits/knowledge/evals/entities/demo-acme.js

@@ -0,0 +1,125 @@
+/**
+ * Entity Cards Demo — Acme Example
+ *
+ * Demonstrates the full entity extraction flow:
+ *   1. Capture a raw note with Attendees line (Dana Smith + Lee Wong)
+ *   2. Compile the raw note
+ *   3. Extract person entities → create person cards with bidirectional links
+ *   4. Print Dana's person card verbatim (as Obsidian markdown)
+ *
+ * Run:
+ *   node kits/knowledge/evals/entities/demo-acme.js
+ */
+
+import * as fs from "node:fs";
+import * as os from "node:os";
+import * as path from "node:path";
+import { fileURLToPath } from "node:url";
+
+const __dirname = path.dirname(fileURLToPath(import.meta.url));
+const KIT_ROOT = path.resolve(__dirname, "../../../..");
+
+// Use Obsidian adapter for richest output
+const obsidianPath = path.join(KIT_ROOT, "kits/knowledge/adapters/obsidian-store/index.js");
+const runnerPath   = path.join(KIT_ROOT, "kits/knowledge/adapters/flow-runner/index.js");
+
+const { ObsidianKnowledgeStore } = await import(obsidianPath);
+const { KnowledgeFlowRunner }    = await import(runnerPath);
+
+// ---------------------------------------------------------------------------
+// Setup temp store
+// ---------------------------------------------------------------------------
+
+const storeDir = fs.mkdtempSync(path.join(os.tmpdir(), "acme-entity-demo-"));
+
+const store  = new ObsidianKnowledgeStore({ storeRoot: storeDir, sourcesDir: "meetings" });
+const runner = new KnowledgeFlowRunner({ store, agent: "acme-demo" });
+
+console.log("Store root:", storeDir);
+console.log("");
+
+// ---------------------------------------------------------------------------
+// 1. Capture a raw Acme meeting note
+// ---------------------------------------------------------------------------
+
+const meetingNote = `Acme Q3 Kickoff — 2026-06-12
+
+Attendees: Dana Smith (Acme VP Eng), Lee Wong
+
+Agenda:
+- Q3 roadmap review
+- Engineering capacity planning
+- Open items from Q2 retro
+
+Action Items:
+- Dana Smith to share updated roadmap by Friday
+- Lee Wong to provide capacity estimates
+
+Next meeting: 2026-06-19`;
+
+const { id: rawId } = await runner.capture(meetingNote, {
+  title: "Acme Q3 Kickoff",
+  category: "sales.acme.meetings",
+});
+
+console.log("Captured raw note:", rawId);
+
+// ---------------------------------------------------------------------------
+// 2. Compile the raw note
+// ---------------------------------------------------------------------------
+
+const { id: compiledId } = await runner.compile([rawId], {
+  title: "Compiled: Acme Q3 Kickoff",
+  category: "sales.acme.meetings",
+});
+
+console.log("Compiled note:    ", compiledId);
+
+// ---------------------------------------------------------------------------
+// 3. Extract entities → create person cards
+// ---------------------------------------------------------------------------
+
+const result = await runner.extractEntities(compiledId);
+
+console.log("");
+console.log("Person cards created:");
+for (const pc of result.personCards) {
+  const card = await store.get(pc.cardId);
+  console.log(`  - ${card.title} (id: ${pc.cardId}, created: ${pc.created})`);
+}
+
+// ---------------------------------------------------------------------------
+// 4. Print Dana's person card verbatim (as Obsidian markdown)
+// ---------------------------------------------------------------------------
+
+const danaResult = result.personCards.find((pc) => pc.name === "Dana Smith");
+if (!danaResult) throw new Error("Dana Smith card not found");
+
+// Read the file directly from the vault
+const { default: DefaultKnowledgeStore } = await import(
+  path.join(KIT_ROOT, "kits/knowledge/adapters/default-store/index.js")
+);
+
+// Find the obsidian file for Dana's card
+const pathIndexFile = path.join(storeDir, ".graph-index.json");
+const pathIndex = JSON.parse(fs.readFileSync(pathIndexFile, "utf8"));
+const danaEntry = pathIndex.by_id[danaResult.cardId];
+if (!danaEntry) throw new Error("Dana card not found in path index");
+
+const danaFilePath = path.join(storeDir, danaEntry.path);
+const danaMarkdown = fs.readFileSync(danaFilePath, "utf8");
+
+console.log("");
+console.log("━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━");
+console.log("Dana Smith's person card (verbatim Obsidian markdown):");
+console.log("File:", danaFilePath.replace(storeDir, "<vault>"));
+console.log("━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━");
+console.log(danaMarkdown);
+console.log("━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━");
+
+// ---------------------------------------------------------------------------
+// Cleanup
+// ---------------------------------------------------------------------------
+
+fs.rmSync(storeDir, { recursive: true, force: true });
+console.log("\nDemo complete.");