@sigrun/knowledge 0.0.0-stage → 0.1.1

package/CHANGELOG.md

@@ -0,0 +1,82 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+### Added
+
+- Everything below, released as 0.1.0: the full knowledge extension —
+  store, atomic versioned apply/undo, tools, the curation target, the
+  research flag, recall mirroring and the idle pass.
+
+### Fixed (post-merge follow-ups)
+
+- Idle pass: a kept child type's identity-key rows re-root by its
+  entities, not by the child's own type id — a child type's rows live at
+  its family root (the grandparent), so the old select missed them and a
+  later record for the same key minted a duplicate entity.
+- Mirroring: a rejected `recall.index` call logs one line (the silent
+  skip hid every mirror failure — including the card-date format bug the
+  e2e later caught).
+
+## [0.1.1] - 2026-11-05
+
+### Fixed
+
+- First registry release. 0.1.0 was consumed by the npm stage
+  bootstrap (the name-claiming placeholder) and never published; 0.1.1
+  is the identical tree.
+
+### Added
+
+- Package scaffold: `createKnowledge(host, deps?)` factory returning
+  `defineExtension({ name: "@sigrun/knowledge" })`, arity 1, with lazily
+  opened per-user databases closed on `harnessShutdown`.
+- `src/schema.ts`: the field-schema subset (`FieldType`, `FieldDef`,
+  `TypeDef`, `TypeDefSchema`, `validateValue`).
+- `src/db.ts`: `KnowledgeDb` — types / entities / entity_keys / inquiries /
+  roles / versions / applied / maintenance tables plus
+  `transaction` (BEGIN IMMEDIATE / COMMIT / ROLLBACK) and `close`.
+- `src/config.ts`: `resolveUserSettings` (researchTools glob list with
+  warn-once fallback) and `matchesGlob` (`*` only).
+- `src/apply.ts` / `src/undo.ts` / `src/card.ts`: `applyRecord` (one
+  transaction, whole-record validation, entity de-duplication by
+  identity-key pairs within a type family, idempotent replay by call id),
+  `undoChange` (newest-first row-equality restores, versioned under
+  `undo:<changeId>`, create-archive and skip reporting), `retypeEntities`,
+  and `renderCard` (stale marks past `horizonDays`).
+- `src/tools.ts`: `knowledge_record` (target tool, replay-safe, idempotent
+  by call id; inquiry records the reviewed parent conversation through the
+  fork-ancestry paging lookup) and `knowledge_find` (read tool — batched
+  type/field/inquiry/text queries; text via `recall.search` with LIKE
+  fallback; audience rule first; local-only entities never shown).
+- `src/target.ts`: the `knowledge` curation target — guidance, undo
+  (revert + re-mirror), wantsReview/reviewed on the research flag,
+  `maintain` (the idle pass: re-index pending/stale entities, continue
+  unfinished re-typing, propose one generalization per pass — ≥3
+  overlapping siblings into a shared parent, folding field-less
+  children, ≤50 retypes per pass).
+- `src/research.ts`: the `afterTool` research flag (`<user
+  state>/research.json`) with the 15-minute single-request window.
+- `src/mirror.ts`: recall mirroring after apply/undo — `recall.index`
+  (kind `record`) or `recall.remove` for archived entities; failure
+  leaves `indexedAt` behind the last version for the idle pass.
+- `src/audience.ts`: the audience rule (private copy of recall's).
+- `src/prompts.ts`: the review guidance bullets.
+- `src/maintain.ts`: the idle pass implementation (generalize sibling
+  types into a hierarchy with versioned key re-roots, declined-group
+  memory, collision-drop cleanup).
+
+### Dependencies
+
+- `@sigrun/shared` peer accepts `0.5.0 || 0.6.0 || 0.6.1 || 0.7.0 ||
+  0.8.0` (the runtime value imports landed on 0.5.0; recall made the
+  same widening).
+
+<!-- TODO at release time: pin `@sigrun/sigrun` and `@sigrun/shared` to
+     the releases carrying published hooks and curation targets (core
+     0.9.0 line, when cut). -->

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Michael MacLeod
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,84 @@
-# Temporary Holding Version
+# @sigrun/knowledge
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+Curated per-user knowledge extension for [sigrun](https://github.com/sigrun-ai/sigrun).
+
+This package owns one SQLite store per directory user
+(`<user data dir>/knowledge.db`) holding typed knowledge entities, identity
+keys, inquiries, and the version history that makes every change undoable.
+
+## Tools
+
+- **`knowledge_record`** (curation target tool, `replay: "safe"`,
+  idempotent by call id): one call carries the whole update from a review —
+  the inquiry (question, why, outcome), any new or extended types, and the
+  entities with identity keys, cited fields (`{value, source, observedAt}`
+  where `source` is a recall handle), and per-inquiry roles. Applied
+  atomically; validation errors roll everything back and are reported as
+  text with no `details.curation`. Success returns the one-line summary
+  (`recorded Biscuit's knee: 3 vet clinics`) as the output text and
+  `details.curation: { changeId, summary }` — the undo unit.
+
+  The user is the conversation's acting user (the review fork's owner).
+  The inquiry records the *reviewed parent* conversation, resolved by
+  paging the fork's entries newest-first until an entry stamped with a
+  different conversation id appears.
+
+- **`knowledge_find`** (read tool, never a target tool): batched queries
+  (≤ 10) by `type` (child types included), by `field` comparison
+  (`eq`/`lt`/`gt`/`contains`; money compares amounts within one currency
+  only), by `inquiry` question text, or by free `text`. Text search goes
+  through the optional `recall.search` hook (keeping only knowledge's own
+  hits) and falls back to a `LIKE` scan when recall is absent or fails.
+  Entities with class `local-only` are never shown.
+
+  The audience rule (spec delta 7) runs first: in a shared conversation
+  the tool answers "Not available here." unless every reader may see the
+  user's knowledge — the user's own records are private to their
+  conversations.
+
+## The curation target
+
+The factory result declares one target:
+`{ name: "knowledge", tools: ["knowledge_record"], readTools: ["knowledge_find"] }`.
+
+- `guidance` — the review prompt bullets (record the inquiry and its why;
+  find before creating; cite every field; give fields that go out of date a
+  `horizonDays` — prices ~30, opening hours ~180, addresses none; record
+  the user's verdicts, never the agent's guesses).
+- `undo` — reverts a change (row-level equality: only rows still equal to
+  what the change wrote are restored) and re-mirrors every touched entity.
+- `wantsReview` / `reviewed` — drive the research flag (below).
+- `maintain` — the idle pass: re-indexes up to 50 pending or
+  horizon-stale entities (mirroring them into recall), continues an
+  unfinished re-typing before proposing anything new, and proposes at
+  most ONE generalization per pass — ≥ 3 sibling types whose field
+  names overlap (pairwise Jaccard ≥ 0.5) become children of a new shared
+  parent type; a child left with no distinct fields is folded (its
+  entities re-typed, ≤ 50 per pass, extra values preserved in
+  `attributes`); identity-key rows re-root (versioned) so records keep
+  merging; declined groups are remembered for 30 days. The digest-run
+  budget bounds the pass and its runs.
+
+## The research trigger
+
+An `afterTool` hook watches non-internal conversations with an owner: when
+a tool whose name matches the owner's `researchTools` globs (defaults to
+web search/fetch/crawl tool names) runs, the conversation is flagged in
+`<user state>/research.json`. The curator asks `wantsReview` at settle;
+a flag with no request in the last 15 minutes answers true once (stamping
+the request), `reviewed` clears it, and a review that never completes is
+re-requested after the window. Errors are logged, never thrown, and the
+hook never alters tool results.
+
+The glob list is the per-user `researchTools` setting (`*` is the only
+wildcard), e.g. `["searxng/search*", "web_fetch"]`.
+
+## Recall mirroring (optional dependency)
+
+After each apply or undo, every touched entity is mirrored into recall
+through the published `recall.index` hook (`kind: "record"`, id = entity
+id, title/text/date/sources from the review card); archived entities are
+removed through `recall.remove`. On `ok` the returned class and the index
+instant are stored; on `absent` nothing happens (knowledge works fully
+without recall); on `failed`/`rejected` the entity's `indexedAt` stays
+behind its last version so the next idle pass retries.

package/dist/apply.d.ts

@@ -0,0 +1,56 @@
+import type { KnowledgeDb } from "./db.js";
+import { type TypeDef } from "./schema.js";
+declare const ENTITY_ROLES: readonly ["candidate", "shortlisted", "rejected", "chosen"];
+export interface RecordInput {
+    inquiry?: {
+        id?: string;
+        question: string;
+        why?: string;
+        outcome?: {
+            status: "open" | "decided" | "abandoned";
+            what?: string;
+        };
+    };
+    types?: TypeDef[];
+    entities: {
+        id?: string;
+        type: string;
+        name: string;
+        keys: Record<string, string>;
+        fields: Record<string, {
+            value: unknown;
+            source: string;
+            observedAt: string;
+        }>;
+        attributes?: Record<string, unknown>;
+        role?: (typeof ENTITY_ROLES)[number];
+        reason?: string;
+    }[];
+}
+export interface ApplyResult {
+    changeId: string;
+    summary: string;
+    /** Entity ids created or updated. */
+    touched: string[];
+    errors: string[];
+}
+export interface ApplyContext {
+    conversationId?: number;
+    callId: string;
+    now: number;
+}
+/** Normalize an identity key value for matching within a type family. */
+export declare function normalizeKey(key: string, value: string): string;
+/**
+ * Apply one knowledge record atomically. Any validation error rolls the
+ * whole record back and is reported in `errors`. Replaying an already
+ * applied `callId` returns the stored result unchanged.
+ */
+export declare function applyRecord(db: KnowledgeDb, input: RecordInput, ctx: ApplyContext): ApplyResult;
+/**
+ * Move entities to a different type: field values the new type does not
+ * declare move into attributes, and identity keys re-root at the new
+ * type's family.
+ */
+export declare function retypeEntities(db: KnowledgeDb, entityIds: readonly string[], toType: string, changeId: string, now: number): void;
+export {};