n8n-nodes-engram 0.5.1 → 0.6.0

package/README.md

@@ -25,7 +25,7 @@
 
 <p align="center">
   <a href="#installation">Installation</a> &middot;
-  <a href="#upgrading-from-v04x">Upgrade</a> &middot;
+  <a href="#upgrading-to-v060">Upgrade</a> &middot;
   <a href="#node-reference">Nodes</a> &middot;
   <a href="#storage-backends">Storage</a> &middot;
   <a href="#knowledge-extraction">Extraction</a> &middot;
@@ -101,6 +101,64 @@ npm install n8n-nodes-engram
 
 ---
 
+## Upgrading to v0.6.0
+
+**Installing this release makes the controls available; it does not automatically activate history filtering or final-turn writes.** Existing saved settings retain temperature `0`, History Filter `Off`, Message Write Policy `Legacy`, and Store Tool Audit `Disabled`. Saved Memory v1 nodes remain v1 until explicitly upgraded. The temperature control is available in both v1 and v2; the history/write/query controls require Memory v2.
+
+### Back Up and Install
+
+1. Back up your workflows and n8n instance configuration using your normal recovery procedure. Create current Engram exports and backend backups before changing package or node settings.
+2. **Embedded:** preserve each workflow's graph file while writes are paused, or take a consistent Engram export. Keep the original storage directory, workflow-to-file mapping, and session/group keys. A copied workflow can have a different default storage filename; validate against a separately restored test copy rather than accidentally testing a new empty graph.
+3. **Neo4j:** take a database backup or consistent export using your normal Neo4j recovery procedure. Keep the same database, credentials, and session/group keys for the existing deployment.
+4. Upgrade the existing community-node installation to `n8n-nodes-engram@0.6.0`, restart n8n, refresh the editor, and confirm the installed version. Node.js 20 or newer is required by the package.
+5. For stores already running `0.5.1`, this release requires **no storage schema migration** on either backend. Run **Engram Admin > Monitoring > Diagnostics** to confirm storage health. If upgrading from `0.4.x` or earlier, complete the existing provenance/schema migration instructions separately.
+
+### Activate Deliberately, One Memory Workflow at a Time
+
+Start with one backed-up direct-conversation workflow, then validate before updating the remaining Memory nodes. If it still uses Memory v1, use n8n's node upgrade action in an inactive workflow copy to adopt **Memory v2**. Preserve its intended storage and canonical session key; do not replace session keys or use display-name changes as an identity migration.
+
+| Setting | Initial activation |
+| --- | --- |
+| Extraction Temperature Mode | Select **Provider Default** when the extraction model rejects temperature `0`; this truly omits the request field. Otherwise retain a mode supported by that model. |
+| History Filter | Select **Provenance** to exclude known tool-output, monitor-summary, and audit episodes from context. |
+| Message Write Policy | Select **Final Turns** to apply storage/extraction controls to both normal saves and native bulk tool-assisted saves. |
+| Store Tool Audit | Keep **Disabled** initially. Enable only if audit retention is needed; retained audits stay outside conversation context under Final Turns. |
+| Current Human Query | Map to the same current-human-message expression used by the agent prompt. The default accepts string `chatInput`, then string `input`; a different webhook shape needs an explicit mapping. |
+| Store Human / AI Episodes | Review the intended retention; both default to Enabled in v2. |
+| Extract from Human / AI / System and Tool Content | Human extraction defaults Enabled; AI and System/Tool extraction default Disabled. Review these when upgrading a v1 node. |
+| Human Episode Kind | Keep **Active Human** for direct conversations; use **Passive Human** for observed/curated human ingestion. |
+| Source Message ID / Idempotency Key | Use a stable **per-message/per-turn** identity for retry deduplication. A static session key or phone number is not a turn identity and can collapse distinct messages. |
+
+The context window counts messages/episodes, not user turns or days. With filtering enabled, older eligible messages refill that window within **Maximum History Scan** (default 1000, maximum 10000). Load diagnostics distinguish snapshot overflow from scan-budget truncation. n8n subnode expressions can have different item-resolution behavior from regular nodes; validate mappings for independent conversations rather than assuming a batch resolves each item separately.
+
+### Historical Artifacts: Inspect and Dry-Run First
+
+This release does not delete historical episodes or graph facts. Provenance filtering cannot automatically prove that an unlabelled legacy JSON message is a tool artifact.
+
+1. Use **Engram Explorer > Episode > List/Get** with the exact group, date, and provenance filters to inspect candidate records, their source identities, and `previous_episode_uuid` links. Preserve genuine human JSON, quoted content, and ambiguous messages.
+2. For a bounded preview, use **Engram Admin > Lifecycle > Purge Episodes**, keep **Execution Mode > Dry Run**, and set the same exact group/provenance/date scope with a small **Maximum Episodes** limit. **Malformed Content Rule > Empty Assistant Output** can locate empty assistant records; inspect their neighboring episodes separately. **Content Contains** should use a distinctive reviewed marker, not generic JSON keys. Do not select **Delete** or confirm a purge as part of this upgrade. Dry-run results are preview candidates, not automatic approval for deletion or filtering.
+3. There is no separate History Filter dry-run switch. Preview filtering in an **inactive workflow copy backed by a separate restored store**, compare its Memory load output before/after, and inspect `history.excluded_legacy`, `legacy_candidates`, `returned`, and `scan_truncated`.
+4. If an exact legacy payload is confirmed to be an artifact, calculate its lowercase SHA256 from the **exact UTF-8 episode content**, including whitespace. Select **History Filter > Provenance and Reviewed Legacy** and add that digest to **Reviewed Legacy Content Hashes** (up to 100 comma-separated hashes). A matching hash still requires strict legacy role, adjacency, chain, and source checks. The neighboring empty assistant and reviewed payload are excluded only from context; original records and facts remain intact. Unapproved or ambiguous JSON stays visible.
+
+### Verify Before Expanding the Rollout
+
+- **Extraction:** inspect the Memory save/bulk-write `extraction` outcome and verify expected accepted entities/relationships in Explorer. A green overall chat execution or a valid empty extraction response alone does not prove fact creation.
+- **Older recall:** query a known older accepted fact after its episode has fallen outside the recent-message window. Confirm the configured current query and relevant graph context, including any governance filters.
+- **Tool-assisted conversation:** exercise a real tool, confirm the real human/final-assistant messages are retained, and verify tool results are not new human sources. Inspect bulk write and extraction diagnostics.
+- **Filtered history:** confirm useful messages refill the window, reviewed exclusions match expectations, genuine JSON/quotes remain, and scan truncation is understood. Re-run one explicit turn identity to check retry behavior without reusing that identity for new turns.
+
+After these checks pass, apply the intended controls to the remaining Memory nodes in stages. Extraction and persistence are not a transactional exactly-once pipeline: retrying an already-stored source does not automatically recover extraction interrupted after persistence. Storage-disabled extraction has no durable retry ledger.
+
+### Rollback
+
+Keep the pre-upgrade workflows, exports, and backend backups unchanged until rollout validation finishes. To roll back, pause writes, restore the saved workflow configuration, reinstall `n8n-nodes-engram@0.5.1` through the same installation method, and restart n8n. This returns the prior behavior, including its extraction-model limitations.
+
+No store restore is required merely to undo a non-destructive history filter. However, if **Store Tool Audit** was enabled, `0.5.1` does not implement the new audit-context exclusion and may replay those new records. Review that history before downgrading; keeping `0.6.0` with the intended controls or using a separately restored pre-upgrade store may be preferable. Turning filtering off does not delete audit records.
+
+If a store restore is necessary, use the backed-up embedded file/Engram export or your Neo4j recovery procedure while writes are paused. Reconcile messages created since the backup before replacing live data; restoring an older snapshot can discard newer conversations. Never wipe groups, trim/change session keys, or delete graph facts as an automatic rollback step.
+
+---
+
 ## Upgrading from v0.4.x
 
 Version `0.5.0` adds episode provenance and governance fields. Storage migration and n8n node-version upgrades are separate steps.
@@ -184,6 +242,7 @@ On each conversation turn, Engram Memory:
 | --- | --- | --- |
 | Storage Backend | Embedded (Graphology) or Neo4j | Embedded |
 | Knowledge Extraction | LLM-powered entity extraction | Disabled |
+| Extraction Temperature Mode | Legacy (0), Provider Default (omit), or Custom | Legacy (0) |
 | Store Human Episodes | Persist human messages as episodes (v2) | Enabled |
 | Human Episode Kind | Classify human input as active or passive provenance (v2) | Active Human |
 | Store AI Episodes | Persist assistant replies as episodes (v2) | Enabled |
@@ -191,13 +250,33 @@ On each conversation turn, Engram Memory:
 | Extract from AI Messages | Allow assistant content to produce facts (v2) | Disabled |
 | Semantic Search | Vector embeddings for hybrid search | Disabled |
 | Graph Traversal | BFS enrichment from matched entities | Disabled |
-| Context Window | Recent turns to include | 10 |
+| Context Window | Recent messages/episodes to include | 10 |
 | Max Facts per Query | Facts injected as context | 10 |
 | Min Relevance Score | Threshold for inclusion (0&ndash;1) | 0.5 |
 | Retention Policy | Episode lifecycle management | Forever |
 
 Existing saved Engram Memory nodes retain their original node version and behavior when the package is upgraded. Use n8n's node upgrade action on a workflow copy to adopt v2 controls. Passive ingestion workflows should select **Human Episode Kind > Passive Human**, while direct agent conversations should keep the default **Active Human** classification.
 
+### History Filtering and Final-Turn Writes (Memory v2)
+
+These controls are opt-in. Existing saved settings keep **History Filter > Off** and **Message Write Policy > Legacy**. Upgrade a v1 Memory node through n8n's node upgrade action on a workflow copy before adopting the v2 controls. Session keys and existing episodes are preserved.
+
+- Select **History Filter > Provenance** to exclude tool-output, monitor-summary, and audit episodes from conversation context.
+- Select **Provenance and Reviewed Legacy** to additionally exclude an exact reviewed legacy artifact pair: an empty assistant `[]` immediately chained to an unlabelled legacy human array of response/error objects. The human payload must match a configured **Reviewed Legacy Content Hash**, and source/chain checks must agree. Without that approval, ambiguous JSON remains. Genuine human JSON, quoted messages, and graph fact SystemMessages are retained.
+- **Reviewed Legacy Content Hashes** accepts up to 100 comma-separated lowercase SHA256 hashes of exact UTF-8 episode content, including its whitespace. Review the original episodes before supplying hashes. Filtering changes only the context read; it does not delete records, rewrite chains, or remove graph facts.
+
+The context window counts eligible messages. Engram captures one bounded recent-episode snapshot and scans it in local pages of 100, refilling from older eligible messages. **Maximum History Scan** defaults to 1000 and is capped at 10000; one extra prefetched episode supports boundary evidence and overflow detection. This is bounded snapshot prefetch with local paging, not backend cursor pagination. Load diagnostics show `fetched`, `examined`, `pages`, exclusion counts, and `returned`. `snapshot_overflow` means more history existed than the scan allowance; `scan_truncated` means that allowance prevented filling the window. No new appends are fetched during the scan.
+
+Select **Message Write Policy > Final Turns** to apply the same storage, extraction, and provenance controls when n8n saves a normal turn or calls the native bulk-message tool-save path. Engram retains real human input and the final assistant reply after identified tool calls have closed. Native tool/function messages and intermediate assistant tool calls are classified distinctly and never become human input. **Store Tool Audit** optionally retains those messages as unverified tool-output episodes; they stay outside conversation context even with History Filter set to Off. Tool extraction remains a separate explicit control and is disabled by default.
+
+Set a stable per-turn **Idempotency Key** or human **Source Message ID** to deduplicate retries across both write paths. Final-role keys are independent of intermediate tool count. Repeated identical turns without an explicit identity remain distinct. Existing stored sources are not re-extracted on an ordinary duplicate write; persistence followed by extraction is not a transactional exactly-once guarantee. If storage is disabled, extraction uses transient sources and has no durable retry ledger. Native bulk-save diagnostics report confirmed writes, duplicates, audits, skipped messages, and the extraction outcome.
+
+### Current Query for Empty Memory Loads
+
+Some n8n agent versions call memory loading with empty values. Configure **Current Human Query** to the same current-message expression used by the agent prompt. Its default reads string `chatInput`, then string `input`, and otherwise leaves the query empty. Explicit input passed by the agent takes precedence. Engram does not infer a current query from old history.
+
+The expression is resolved in n8n's subnode item context when memory is supplied. Subnode expression behavior can differ from regular-node per-item evaluation; process independent conversations separately and validate mappings in your workflow. Load diagnostics expose `retrieval.query_source` (`input`, `configured`, or `none`), query availability, and the number of graph fact context messages without including query text. If no current query is available, history still loads but graph search is not attempted. Validate graph recall with a known older accepted fact beyond the recent-message window.
+
 ### Engram Explorer
 
 > Regular node&mdash;CRUD operations on the knowledge graph.
@@ -289,6 +368,32 @@ User message + AI response
 
 Extracted entities and relationships include `attributes.engram_extraction` provenance metadata with source, extraction timestamp, confidence placeholder, review status, and related episode UUIDs where available.
 
+### Extraction Model Compatibility
+
+**Extraction Temperature Mode** applies to every extraction stage in Engram Memory v1 and v2:
+
+- **Legacy (0)** sends temperature `0` and preserves existing workflow behavior.
+- **Provider Default** omits the temperature field. Select this when your model rejects custom temperatures or requires its own default.
+- **Custom** sends the **Extraction Temperature** value. Use a finite number supported by your provider and model; supported ranges vary.
+
+Upgrading the package alone retains Legacy (0). To resolve an unsupported-temperature error, select Provider Default explicitly in the affected Memory node and validate the workflow before activating it. No storage migration or Memory node-version upgrade is required for these settings. Engram does not automatically change models or retry rejected requests with guessed settings. Extraction still uses Chat Completions JSON-object mode; confirm that your provider and model support it.
+
+### Extraction Outcomes
+
+In an execution, open the Engram Memory save output and inspect `extraction`. Conversation history saves before extraction, so an overall successful conversation can still have a failed extraction outcome.
+
+| Status | Meaning |
+| --- | --- |
+| `disabled` | Knowledge extraction is disabled. |
+| `skipped` | No eligible, nonempty source messages were selected. |
+| `succeeded` | Attempted stages completed; finding zero entities is also successful. |
+| `partial` | An optional stage failed, malformed records were excluded, or a required stage failed after some graph changes succeeded. |
+| `failed` | A required stage failed before any graph changes succeeded, or extraction configuration is missing. |
+
+Outcomes include duration, stage status, confirmed mutation counts, filtered/malformed record counts, and bounded error classifications such as `unsupported_parameter`, `authentication`, `rate_limit`, and `timeout`. Update counts include metadata-only updates; edge expirations are counted separately. Partial extraction does not roll back completed graph changes. Provider error bodies and message content are excluded from extraction error diagnostics; normal n8n conversation previews still contain message content.
+
+Extraction outcomes belong to their execution's save output. Engram Admin storage diagnostics do not provide a persistent extraction event history. A failed history preview is reported separately from the completed save. The context window counts messages, not user turns or days; graph recall operates independently of that window.
+
 ---
 
 ## Search

package/dist/descriptions.js

@@ -36,7 +36,7 @@ exports.contextWindowLengthProperty = {
     name: 'contextWindowLength',
     type: 'number',
     default: 5,
-    hint: 'How many past interactions the model receives as context',
+    hint: 'Number of recent messages (episodes) to include, not user turns or days',
 };
 exports.customStoragePathProperty = {
     displayName: 'Custom Storage Path',

package/dist/descriptions.js.map

@@ -1 +1 @@
-{"version":3,"file":"descriptions.js","sourceRoot":"","sources":["../src/descriptions.ts"],"names":[],"mappings":";;;AAEa,QAAA,eAAe,GAAoB;IAC9C,WAAW,EAAE,YAAY;IACzB,IAAI,EAAE,eAAe;IACrB,IAAI,EAAE,SAAS;IACf,OAAO,EAAE;QACP;YAEE,IAAI,EAAE,uCAAuC;YAC7C,KAAK,EAAE,WAAW;YAClB,WAAW,EAAE,2CAA2C;SACzD;QACD;YAEE,IAAI,EAAE,cAAc;YACpB,KAAK,EAAE,WAAW;YAClB,WAAW,EAAE,4EAA4E;SAC1F;KACF;IACD,OAAO,EAAE,WAAW;CACrB,CAAC;AAEW,QAAA,kBAAkB,GAAoB;IACjD,WAAW,EAAE,KAAK;IAClB,IAAI,EAAE,YAAY;IAClB,IAAI,EAAE,QAAQ;IACd,OAAO,EAAE,EAAE;IACX,WAAW,EAAE,kDAAkD;IAC/D,cAAc,EAAE;QACd,IAAI,EAAE;YACJ,aAAa,EAAE,CAAC,WAAW,CAAC;SAC7B;KACF;CACF,CAAC;AAEW,QAAA,2BAA2B,GAAoB;IAC1D,WAAW,EAAE,uBAAuB;IACpC,IAAI,EAAE,qBAAqB;IAC3B,IAAI,EAAE,QAAQ;IACd,OAAO,EAAE,CAAC;IACV,IAAI,EAAE,0DAA0D;CACjE,CAAC;AAEW,QAAA,yBAAyB,GAAoB;IACxD,WAAW,EAAE,qBAAqB;IAClC,IAAI,EAAE,mBAAmB;IACzB,IAAI,EAAE,QAAQ;IACd,OAAO,EAAE,EAAE;IACX,WAAW,EAAE,cAAc;IAE3B,WAAW,EACT,+SAA+S;IACjT,cAAc,EAAE;QACd,IAAI,EAAE;YACJ,OAAO,EAAE,CAAC,UAAU,CAAC;SACtB;KACF;CACF,CAAC"}
+{"version":3,"file":"descriptions.js","sourceRoot":"","sources":["../src/descriptions.ts"],"names":[],"mappings":";;;AAEa,QAAA,eAAe,GAAoB;IAC9C,WAAW,EAAE,YAAY;IACzB,IAAI,EAAE,eAAe;IACrB,IAAI,EAAE,SAAS;IACf,OAAO,EAAE;QACP;YAEE,IAAI,EAAE,uCAAuC;YAC7C,KAAK,EAAE,WAAW;YAClB,WAAW,EAAE,2CAA2C;SACzD;QACD;YAEE,IAAI,EAAE,cAAc;YACpB,KAAK,EAAE,WAAW;YAClB,WAAW,EAAE,4EAA4E;SAC1F;KACF;IACD,OAAO,EAAE,WAAW;CACrB,CAAC;AAEW,QAAA,kBAAkB,GAAoB;IACjD,WAAW,EAAE,KAAK;IAClB,IAAI,EAAE,YAAY;IAClB,IAAI,EAAE,QAAQ;IACd,OAAO,EAAE,EAAE;IACX,WAAW,EAAE,kDAAkD;IAC/D,cAAc,EAAE;QACd,IAAI,EAAE;YACJ,aAAa,EAAE,CAAC,WAAW,CAAC;SAC7B;KACF;CACF,CAAC;AAEW,QAAA,2BAA2B,GAAoB;IAC1D,WAAW,EAAE,uBAAuB;IACpC,IAAI,EAAE,qBAAqB;IAC3B,IAAI,EAAE,QAAQ;IACd,OAAO,EAAE,CAAC;IACV,IAAI,EAAE,yEAAyE;CAChF,CAAC;AAEW,QAAA,yBAAyB,GAAoB;IACxD,WAAW,EAAE,qBAAqB;IAClC,IAAI,EAAE,mBAAmB;IACzB,IAAI,EAAE,QAAQ;IACd,OAAO,EAAE,EAAE;IACX,WAAW,EAAE,cAAc;IAE3B,WAAW,EACT,+SAA+S;IACjT,cAAc,EAAE;QACd,IAAI,EAAE;YACJ,OAAO,EAAE,CAAC,UAAU,CAAC;SACtB;KACF;CACF,CAAC"}

package/dist/extraction/ContradictionDetector.d.ts

@@ -1,3 +1,4 @@
+import { type ExtractionDiagnostics } from './ExtractionDiagnostics';
 import type { LlmClient } from './LlmClient';
 export interface ContradictionResolution {
     isContradiction: boolean;
@@ -5,7 +6,8 @@ export interface ContradictionResolution {
     expiredAt?: string;
 }
 export declare class ContradictionDetector {
+    private diagnostics?;
     private llm;
-    constructor(llm: LlmClient);
+    constructor(llm: LlmClient, diagnostics?: ExtractionDiagnostics | undefined);
     detect(existingFact: string, newFact: string, sourceEntity: string, targetEntity: string, existingEdgeName?: string, newEdgeName?: string): Promise<ContradictionResolution>;
 }

package/dist/extraction/ContradictionDetector.js

@@ -1,13 +1,16 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.ContradictionDetector = void 0;
+const ExtractionDiagnostics_1 = require("./ExtractionDiagnostics");
 const prompts_1 = require("./prompts");
 const temporal_1 = require("../utils/temporal");
 class ContradictionDetector {
-    constructor(llm) {
+    constructor(llm, diagnostics) {
+        this.diagnostics = diagnostics;
         this.llm = llm;
     }
     async detect(existingFact, newFact, sourceEntity, targetEntity, existingEdgeName, newEdgeName) {
+        var _a;
         try {
             const result = await this.llm.chatJson([
                 { role: 'system', content: prompts_1.CONTRADICTION_SYSTEM },
@@ -16,6 +19,11 @@ class ContradictionDetector {
                     content: (0, prompts_1.contradictionUser)(existingFact, newFact, sourceEntity, targetEntity, existingEdgeName, newEdgeName),
                 },
             ]);
+            if (!result ||
+                typeof result.is_contradiction !== 'boolean' ||
+                (result.is_contradiction && typeof result.explanation !== 'string'))
+                throw new ExtractionDiagnostics_1.ExtractionError('invalid_response');
+            (_a = this.diagnostics) === null || _a === void 0 ? void 0 : _a.success('contradictions');
             if (result.is_contradiction) {
                 return {
                     isContradiction: true,
@@ -26,7 +34,7 @@ class ContradictionDetector {
             return { isContradiction: false };
         }
         catch (error) {
-            console.warn('Engram: Contradiction detection failed:', error.message);
+            (0, ExtractionDiagnostics_1.reportExtractionError)('contradictions', error, this.diagnostics);
             return { isContradiction: false };
         }
     }

package/dist/extraction/ContradictionDetector.js.map

@@ -1 +1 @@
-{"version":3,"file":"ContradictionDetector.js","sourceRoot":"","sources":["../../src/extraction/ContradictionDetector.ts"],"names":[],"mappings":";;;AACA,uCAAoE;AACpE,gDAA2C;AAc3C,MAAa,qBAAqB;IAGhC,YAAY,GAAc;QACxB,IAAI,CAAC,GAAG,GAAG,GAAG,CAAC;IACjB,CAAC;IAMD,KAAK,CAAC,MAAM,CACV,YAAoB,EACpB,OAAe,EACf,YAAoB,EACpB,YAAoB,EACpB,gBAAyB,EACzB,WAAoB;QAEpB,IAAI,CAAC;YACH,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAsB;gBAC1D,EAAE,IAAI,EAAE,QAAQ,EAAE,OAAO,EAAE,8BAAoB,EAAE;gBACjD;oBACE,IAAI,EAAE,MAAM;oBACZ,OAAO,EAAE,IAAA,2BAAiB,EACxB,YAAY,EACZ,OAAO,EACP,YAAY,EACZ,YAAY,EACZ,gBAAgB,EAChB,WAAW,CACZ;iBACF;aACF,CAAC,CAAC;YAEH,IAAI,MAAM,CAAC,gBAAgB,EAAE,CAAC;gBAC5B,OAAO;oBACL,eAAe,EAAE,IAAI;oBACrB,WAAW,EAAE,MAAM,CAAC,WAAW;oBAC/B,SAAS,EAAE,IAAA,iBAAM,GAAE;iBACpB,CAAC;YACJ,CAAC;YAED,OAAO,EAAE,eAAe,EAAE,KAAK,EAAE,CAAC;QACpC,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YAEf,OAAO,CAAC,IAAI,CAAC,yCAAyC,EAAG,KAAe,CAAC,OAAO,CAAC,CAAC;YAClF,OAAO,EAAE,eAAe,EAAE,KAAK,EAAE,CAAC;QACpC,CAAC;IACH,CAAC;CACF;AAlDD,sDAkDC"}
+{"version":3,"file":"ContradictionDetector.js","sourceRoot":"","sources":["../../src/extraction/ContradictionDetector.ts"],"names":[],"mappings":";;;AAAA,mEAIiC;AAEjC,uCAAoE;AACpE,gDAA2C;AAc3C,MAAa,qBAAqB;IAGhC,YACE,GAAc,EACN,WAAmC;QAAnC,gBAAW,GAAX,WAAW,CAAwB;QAE3C,IAAI,CAAC,GAAG,GAAG,GAAG,CAAC;IACjB,CAAC;IAMD,KAAK,CAAC,MAAM,CACV,YAAoB,EACpB,OAAe,EACf,YAAoB,EACpB,YAAoB,EACpB,gBAAyB,EACzB,WAAoB;;QAEpB,IAAI,CAAC;YACH,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAsB;gBAC1D,EAAE,IAAI,EAAE,QAAQ,EAAE,OAAO,EAAE,8BAAoB,EAAE;gBACjD;oBACE,IAAI,EAAE,MAAM;oBACZ,OAAO,EAAE,IAAA,2BAAiB,EACxB,YAAY,EACZ,OAAO,EACP,YAAY,EACZ,YAAY,EACZ,gBAAgB,EAChB,WAAW,CACZ;iBACF;aACF,CAAC,CAAC;YACH,IACE,CAAC,MAAM;gBACP,OAAO,MAAM,CAAC,gBAAgB,KAAK,SAAS;gBAC5C,CAAC,MAAM,CAAC,gBAAgB,IAAI,OAAO,MAAM,CAAC,WAAW,KAAK,QAAQ,CAAC;gBAEnE,MAAM,IAAI,uCAAe,CAAC,kBAAkB,CAAC,CAAC;YAChD,MAAA,IAAI,CAAC,WAAW,0CAAE,OAAO,CAAC,gBAAgB,CAAC,CAAC;YAE5C,IAAI,MAAM,CAAC,gBAAgB,EAAE,CAAC;gBAC5B,OAAO;oBACL,eAAe,EAAE,IAAI;oBACrB,WAAW,EAAE,MAAM,CAAC,WAAW;oBAC/B,SAAS,EAAE,IAAA,iBAAM,GAAE;iBACpB,CAAC;YACJ,CAAC;YAED,OAAO,EAAE,eAAe,EAAE,KAAK,EAAE,CAAC;QACpC,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YAEf,IAAA,6CAAqB,EAAC,gBAAgB,EAAE,KAAK,EAAE,IAAI,CAAC,WAAW,CAAC,CAAC;YACjE,OAAO,EAAE,eAAe,EAAE,KAAK,EAAE,CAAC;QACpC,CAAC;IACH,CAAC;CACF;AA5DD,sDA4DC"}

package/dist/extraction/EdgeDeduplicator.d.ts

@@ -1,7 +1,9 @@
+import { type ExtractionDiagnostics } from './ExtractionDiagnostics';
 import type { LlmClient } from './LlmClient';
 export declare class EdgeDeduplicator {
+    private diagnostics?;
     private llm;
-    constructor(llm: LlmClient);
+    constructor(llm: LlmClient, diagnostics?: ExtractionDiagnostics | undefined);
     isDuplicate(existingFact: string, newFact: string, edgeName: string, sourceEntity: string, targetEntity: string): Promise<{
         isDuplicate: boolean;
         mergedFact?: string;

package/dist/extraction/EdgeDeduplicator.js

@@ -1,13 +1,17 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.EdgeDeduplicator = void 0;
+const ExtractionDiagnostics_1 = require("./ExtractionDiagnostics");
 const prompts_1 = require("./prompts");
 class EdgeDeduplicator {
-    constructor(llm) {
+    constructor(llm, diagnostics) {
+        this.diagnostics = diagnostics;
         this.llm = llm;
     }
     async isDuplicate(existingFact, newFact, edgeName, sourceEntity, targetEntity) {
+        var _a, _b;
         if (existingFact.toLowerCase().trim() === newFact.toLowerCase().trim()) {
+            (_a = this.diagnostics) === null || _a === void 0 ? void 0 : _a.success('edge_dedup');
             return { isDuplicate: true };
         }
         try {
@@ -18,17 +22,23 @@ class EdgeDeduplicator {
                     content: (0, prompts_1.edgeDedupUser)(existingFact, newFact, edgeName, sourceEntity, targetEntity),
                 },
             ]);
+            if (!result ||
+                typeof result.is_duplicate !== 'boolean' ||
+                (result.is_duplicate && typeof result.merged_fact !== 'string'))
+                throw new ExtractionDiagnostics_1.ExtractionError('invalid_response');
+            (_b = this.diagnostics) === null || _b === void 0 ? void 0 : _b.success('edge_dedup');
             return {
                 isDuplicate: result.is_duplicate === true,
                 mergedFact: result.is_duplicate ? result.merged_fact : undefined,
             };
         }
         catch (error) {
-            console.warn('Engram: Edge deduplication failed:', error.message);
+            (0, ExtractionDiagnostics_1.reportExtractionError)('edge_dedup', error, this.diagnostics);
             return { isDuplicate: false };
         }
     }
     async isDuplicateCrossName(existingFact, newFact, existingEdgeName, newEdgeName, sourceEntity, targetEntity) {
+        var _a;
         try {
             const result = await this.llm.chatJson([
                 { role: 'system', content: prompts_1.CROSS_NAME_EDGE_DEDUP_SYSTEM },
@@ -37,13 +47,18 @@ class EdgeDeduplicator {
                     content: (0, prompts_1.crossNameEdgeDedupUser)(existingFact, newFact, existingEdgeName, newEdgeName, sourceEntity, targetEntity),
                 },
             ]);
+            if (!result ||
+                typeof result.is_duplicate !== 'boolean' ||
+                (result.is_duplicate && typeof result.merged_fact !== 'string'))
+                throw new ExtractionDiagnostics_1.ExtractionError('invalid_response');
+            (_a = this.diagnostics) === null || _a === void 0 ? void 0 : _a.success('cross_edge_dedup');
             return {
                 isDuplicate: result.is_duplicate === true,
                 mergedFact: result.is_duplicate ? result.merged_fact : undefined,
             };
         }
         catch (error) {
-            console.warn('Engram: Cross-name edge deduplication failed:', error.message);
+            (0, ExtractionDiagnostics_1.reportExtractionError)('cross_edge_dedup', error, this.diagnostics);
             return { isDuplicate: false };
         }
     }

package/dist/extraction/EdgeDeduplicator.js.map

@@ -1 +1 @@
-{"version":3,"file":"EdgeDeduplicator.js","sourceRoot":"","sources":["../../src/extraction/EdgeDeduplicator.ts"],"names":[],"mappings":";;;AACA,uCAKmB;AAOnB,MAAa,gBAAgB;IAG3B,YAAY,GAAc;QACxB,IAAI,CAAC,GAAG,GAAG,GAAG,CAAC;IACjB,CAAC;IAMD,KAAK,CAAC,WAAW,CACf,YAAoB,EACpB,OAAe,EACf,QAAgB,EAChB,YAAoB,EACpB,YAAoB;QAGpB,IAAI,YAAY,CAAC,WAAW,EAAE,CAAC,IAAI,EAAE,KAAK,OAAO,CAAC,WAAW,EAAE,CAAC,IAAI,EAAE,EAAE,CAAC;YACvE,OAAO,EAAE,WAAW,EAAE,IAAI,EAAE,CAAC;QAC/B,CAAC;QAGD,IAAI,CAAC;YACH,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAkB;gBACtD,EAAE,IAAI,EAAE,QAAQ,EAAE,OAAO,EAAE,2BAAiB,EAAE;gBAC9C;oBACE,IAAI,EAAE,MAAM;oBACZ,OAAO,EAAE,IAAA,uBAAa,EAAC,YAAY,EAAE,OAAO,EAAE,QAAQ,EAAE,YAAY,EAAE,YAAY,CAAC;iBACpF;aACF,CAAC,CAAC;YAEH,OAAO;gBACL,WAAW,EAAE,MAAM,CAAC,YAAY,KAAK,IAAI;gBACzC,UAAU,EAAE,MAAM,CAAC,YAAY,CAAC,CAAC,CAAC,MAAM,CAAC,WAAW,CAAC,CAAC,CAAC,SAAS;aACjE,CAAC;QACJ,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YAEf,OAAO,CAAC,IAAI,CAAC,oCAAoC,EAAG,KAAe,CAAC,OAAO,CAAC,CAAC;YAC7E,OAAO,EAAE,WAAW,EAAE,KAAK,EAAE,CAAC;QAChC,CAAC;IACH,CAAC;IAYD,KAAK,CAAC,oBAAoB,CACxB,YAAoB,EACpB,OAAe,EACf,gBAAwB,EACxB,WAAmB,EACnB,YAAoB,EACpB,YAAoB;QAEpB,IAAI,CAAC;YACH,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAkB;gBACtD,EAAE,IAAI,EAAE,QAAQ,EAAE,OAAO,EAAE,sCAA4B,EAAE;gBACzD;oBACE,IAAI,EAAE,MAAM;oBACZ,OAAO,EAAE,IAAA,gCAAsB,EAC7B,YAAY,EACZ,OAAO,EACP,gBAAgB,EAChB,WAAW,EACX,YAAY,EACZ,YAAY,CACb;iBACF;aACF,CAAC,CAAC;YAEH,OAAO;gBACL,WAAW,EAAE,MAAM,CAAC,YAAY,KAAK,IAAI;gBACzC,UAAU,EAAE,MAAM,CAAC,YAAY,CAAC,CAAC,CAAC,MAAM,CAAC,WAAW,CAAC,CAAC,CAAC,SAAS;aACjE,CAAC;QACJ,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YAEf,OAAO,CAAC,IAAI,CAAC,+CAA+C,EAAG,KAAe,CAAC,OAAO,CAAC,CAAC;YACxF,OAAO,EAAE,WAAW,EAAE,KAAK,EAAE,CAAC;QAChC,CAAC;IACH,CAAC;CACF;AAxFD,4CAwFC"}
+{"version":3,"file":"EdgeDeduplicator.js","sourceRoot":"","sources":["../../src/extraction/EdgeDeduplicator.ts"],"names":[],"mappings":";;;AAAA,mEAIiC;AAEjC,uCAKmB;AAOnB,MAAa,gBAAgB;IAG3B,YACE,GAAc,EACN,WAAmC;QAAnC,gBAAW,GAAX,WAAW,CAAwB;QAE3C,IAAI,CAAC,GAAG,GAAG,GAAG,CAAC;IACjB,CAAC;IAMD,KAAK,CAAC,WAAW,CACf,YAAoB,EACpB,OAAe,EACf,QAAgB,EAChB,YAAoB,EACpB,YAAoB;;QAGpB,IAAI,YAAY,CAAC,WAAW,EAAE,CAAC,IAAI,EAAE,KAAK,OAAO,CAAC,WAAW,EAAE,CAAC,IAAI,EAAE,EAAE,CAAC;YACvE,MAAA,IAAI,CAAC,WAAW,0CAAE,OAAO,CAAC,YAAY,CAAC,CAAC;YACxC,OAAO,EAAE,WAAW,EAAE,IAAI,EAAE,CAAC;QAC/B,CAAC;QAGD,IAAI,CAAC;YACH,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAkB;gBACtD,EAAE,IAAI,EAAE,QAAQ,EAAE,OAAO,EAAE,2BAAiB,EAAE;gBAC9C;oBACE,IAAI,EAAE,MAAM;oBACZ,OAAO,EAAE,IAAA,uBAAa,EAAC,YAAY,EAAE,OAAO,EAAE,QAAQ,EAAE,YAAY,EAAE,YAAY,CAAC;iBACpF;aACF,CAAC,CAAC;YACH,IACE,CAAC,MAAM;gBACP,OAAO,MAAM,CAAC,YAAY,KAAK,SAAS;gBACxC,CAAC,MAAM,CAAC,YAAY,IAAI,OAAO,MAAM,CAAC,WAAW,KAAK,QAAQ,CAAC;gBAE/D,MAAM,IAAI,uCAAe,CAAC,kBAAkB,CAAC,CAAC;YAChD,MAAA,IAAI,CAAC,WAAW,0CAAE,OAAO,CAAC,YAAY,CAAC,CAAC;YAExC,OAAO;gBACL,WAAW,EAAE,MAAM,CAAC,YAAY,KAAK,IAAI;gBACzC,UAAU,EAAE,MAAM,CAAC,YAAY,CAAC,CAAC,CAAC,MAAM,CAAC,WAAW,CAAC,CAAC,CAAC,SAAS;aACjE,CAAC;QACJ,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YAEf,IAAA,6CAAqB,EAAC,YAAY,EAAE,KAAK,EAAE,IAAI,CAAC,WAAW,CAAC,CAAC;YAC7D,OAAO,EAAE,WAAW,EAAE,KAAK,EAAE,CAAC;QAChC,CAAC;IACH,CAAC;IAYD,KAAK,CAAC,oBAAoB,CACxB,YAAoB,EACpB,OAAe,EACf,gBAAwB,EACxB,WAAmB,EACnB,YAAoB,EACpB,YAAoB;;QAEpB,IAAI,CAAC;YACH,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAkB;gBACtD,EAAE,IAAI,EAAE,QAAQ,EAAE,OAAO,EAAE,sCAA4B,EAAE;gBACzD;oBACE,IAAI,EAAE,MAAM;oBACZ,OAAO,EAAE,IAAA,gCAAsB,EAC7B,YAAY,EACZ,OAAO,EACP,gBAAgB,EAChB,WAAW,EACX,YAAY,EACZ,YAAY,CACb;iBACF;aACF,CAAC,CAAC;YACH,IACE,CAAC,MAAM;gBACP,OAAO,MAAM,CAAC,YAAY,KAAK,SAAS;gBACxC,CAAC,MAAM,CAAC,YAAY,IAAI,OAAO,MAAM,CAAC,WAAW,KAAK,QAAQ,CAAC;gBAE/D,MAAM,IAAI,uCAAe,CAAC,kBAAkB,CAAC,CAAC;YAChD,MAAA,IAAI,CAAC,WAAW,0CAAE,OAAO,CAAC,kBAAkB,CAAC,CAAC;YAE9C,OAAO;gBACL,WAAW,EAAE,MAAM,CAAC,YAAY,KAAK,IAAI;gBACzC,UAAU,EAAE,MAAM,CAAC,YAAY,CAAC,CAAC,CAAC,MAAM,CAAC,WAAW,CAAC,CAAC,CAAC,SAAS;aACjE,CAAC;QACJ,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YAEf,IAAA,6CAAqB,EAAC,kBAAkB,EAAE,KAAK,EAAE,IAAI,CAAC,WAAW,CAAC,CAAC;YACnE,OAAO,EAAE,WAAW,EAAE,KAAK,EAAE,CAAC;QAChC,CAAC;IACH,CAAC;CACF;AA1GD,4CA0GC"}

package/dist/extraction/EntityDeduplicator.d.ts

@@ -1,7 +1,9 @@
+import { type ExtractionDiagnostics } from './ExtractionDiagnostics';
 import type { LlmClient } from './LlmClient';
 export declare class EntityDeduplicator {
+    private diagnostics?;
     private llm;
-    constructor(llm: LlmClient);
+    constructor(llm: LlmClient, diagnostics?: ExtractionDiagnostics | undefined);
     isDuplicate(newEntity: {
         name: string;
         summary: string;

package/dist/extraction/EntityDeduplicator.js

@@ -1,19 +1,24 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.EntityDeduplicator = void 0;
+const ExtractionDiagnostics_1 = require("./ExtractionDiagnostics");
 const prompts_1 = require("./prompts");
 class EntityDeduplicator {
-    constructor(llm) {
+    constructor(llm, diagnostics) {
+        this.diagnostics = diagnostics;
         this.llm = llm;
     }
     async isDuplicate(newEntity, existingEntity) {
+        var _a, _b, _c;
         if (newEntity.name.toLowerCase().trim() === existingEntity.name.toLowerCase().trim()) {
+            (_a = this.diagnostics) === null || _a === void 0 ? void 0 : _a.success('entity_dedup');
             const mergedSummary = newEntity.summary.length > existingEntity.summary.length
                 ? newEntity.summary
                 : existingEntity.summary;
             return { isDuplicate: true, mergedSummary };
         }
         if (newEntity.entity_type !== existingEntity.entity_type) {
+            (_b = this.diagnostics) === null || _b === void 0 ? void 0 : _b.success('entity_dedup');
             return { isDuplicate: false };
         }
         try {
@@ -24,13 +29,18 @@ class EntityDeduplicator {
                     content: (0, prompts_1.deduplicationUser)(newEntity, existingEntity),
                 },
             ]);
+            if (!result ||
+                typeof result.is_duplicate !== 'boolean' ||
+                (result.is_duplicate && typeof result.merged_summary !== 'string'))
+                throw new ExtractionDiagnostics_1.ExtractionError('invalid_response');
+            (_c = this.diagnostics) === null || _c === void 0 ? void 0 : _c.success('entity_dedup');
             return {
                 isDuplicate: result.is_duplicate === true,
                 mergedSummary: result.is_duplicate ? result.merged_summary : undefined,
             };
         }
         catch (error) {
-            console.warn('Engram: Entity deduplication failed:', error.message);
+            (0, ExtractionDiagnostics_1.reportExtractionError)('entity_dedup', error, this.diagnostics);
             return { isDuplicate: false };
         }
     }

package/dist/extraction/EntityDeduplicator.js.map

@@ -1 +1 @@
-{"version":3,"file":"EntityDeduplicator.js","sourceRoot":"","sources":["../../src/extraction/EntityDeduplicator.ts"],"names":[],"mappings":";;;AACA,uCAAoE;AAOpE,MAAa,kBAAkB;IAG7B,YAAY,GAAc;QACxB,IAAI,CAAC,GAAG,GAAG,GAAG,CAAC;IACjB,CAAC;IAMD,KAAK,CAAC,WAAW,CACf,SAAiE,EACjE,cAAsE;QAGtE,IAAI,SAAS,CAAC,IAAI,CAAC,WAAW,EAAE,CAAC,IAAI,EAAE,KAAK,cAAc,CAAC,IAAI,CAAC,WAAW,EAAE,CAAC,IAAI,EAAE,EAAE,CAAC;YAErF,MAAM,aAAa,GACjB,SAAS,CAAC,OAAO,CAAC,MAAM,GAAG,cAAc,CAAC,OAAO,CAAC,MAAM;gBACtD,CAAC,CAAC,SAAS,CAAC,OAAO;gBACnB,CAAC,CAAC,cAAc,CAAC,OAAO,CAAC;YAC7B,OAAO,EAAE,WAAW,EAAE,IAAI,EAAE,aAAa,EAAE,CAAC;QAC9C,CAAC;QAGD,IAAI,SAAS,CAAC,WAAW,KAAK,cAAc,CAAC,WAAW,EAAE,CAAC;YACzD,OAAO,EAAE,WAAW,EAAE,KAAK,EAAE,CAAC;QAChC,CAAC;QAGD,IAAI,CAAC;YACH,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAsB;gBAC1D,EAAE,IAAI,EAAE,QAAQ,EAAE,OAAO,EAAE,8BAAoB,EAAE;gBACjD;oBACE,IAAI,EAAE,MAAM;oBACZ,OAAO,EAAE,IAAA,2BAAiB,EAAC,SAAS,EAAE,cAAc,CAAC;iBACtD;aACF,CAAC,CAAC;YAEH,OAAO;gBACL,WAAW,EAAE,MAAM,CAAC,YAAY,KAAK,IAAI;gBACzC,aAAa,EAAE,MAAM,CAAC,YAAY,CAAC,CAAC,CAAC,MAAM,CAAC,cAAc,CAAC,CAAC,CAAC,SAAS;aACvE,CAAC;QACJ,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YAEf,OAAO,CAAC,IAAI,CAAC,sCAAsC,EAAG,KAAe,CAAC,OAAO,CAAC,CAAC;YAC/E,OAAO,EAAE,WAAW,EAAE,KAAK,EAAE,CAAC;QAChC,CAAC;IACH,CAAC;CACF;AAlDD,gDAkDC"}
+{"version":3,"file":"EntityDeduplicator.js","sourceRoot":"","sources":["../../src/extraction/EntityDeduplicator.ts"],"names":[],"mappings":";;;AAAA,mEAIiC;AAEjC,uCAAoE;AAOpE,MAAa,kBAAkB;IAG7B,YACE,GAAc,EACN,WAAmC;QAAnC,gBAAW,GAAX,WAAW,CAAwB;QAE3C,IAAI,CAAC,GAAG,GAAG,GAAG,CAAC;IACjB,CAAC;IAMD,KAAK,CAAC,WAAW,CACf,SAAiE,EACjE,cAAsE;;QAGtE,IAAI,SAAS,CAAC,IAAI,CAAC,WAAW,EAAE,CAAC,IAAI,EAAE,KAAK,cAAc,CAAC,IAAI,CAAC,WAAW,EAAE,CAAC,IAAI,EAAE,EAAE,CAAC;YACrF,MAAA,IAAI,CAAC,WAAW,0CAAE,OAAO,CAAC,cAAc,CAAC,CAAC;YAE1C,MAAM,aAAa,GACjB,SAAS,CAAC,OAAO,CAAC,MAAM,GAAG,cAAc,CAAC,OAAO,CAAC,MAAM;gBACtD,CAAC,CAAC,SAAS,CAAC,OAAO;gBACnB,CAAC,CAAC,cAAc,CAAC,OAAO,CAAC;YAC7B,OAAO,EAAE,WAAW,EAAE,IAAI,EAAE,aAAa,EAAE,CAAC;QAC9C,CAAC;QAGD,IAAI,SAAS,CAAC,WAAW,KAAK,cAAc,CAAC,WAAW,EAAE,CAAC;YACzD,MAAA,IAAI,CAAC,WAAW,0CAAE,OAAO,CAAC,cAAc,CAAC,CAAC;YAC1C,OAAO,EAAE,WAAW,EAAE,KAAK,EAAE,CAAC;QAChC,CAAC;QAGD,IAAI,CAAC;YACH,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAsB;gBAC1D,EAAE,IAAI,EAAE,QAAQ,EAAE,OAAO,EAAE,8BAAoB,EAAE;gBACjD;oBACE,IAAI,EAAE,MAAM;oBACZ,OAAO,EAAE,IAAA,2BAAiB,EAAC,SAAS,EAAE,cAAc,CAAC;iBACtD;aACF,CAAC,CAAC;YACH,IACE,CAAC,MAAM;gBACP,OAAO,MAAM,CAAC,YAAY,KAAK,SAAS;gBACxC,CAAC,MAAM,CAAC,YAAY,IAAI,OAAO,MAAM,CAAC,cAAc,KAAK,QAAQ,CAAC;gBAElE,MAAM,IAAI,uCAAe,CAAC,kBAAkB,CAAC,CAAC;YAChD,MAAA,IAAI,CAAC,WAAW,0CAAE,OAAO,CAAC,cAAc,CAAC,CAAC;YAE1C,OAAO;gBACL,WAAW,EAAE,MAAM,CAAC,YAAY,KAAK,IAAI;gBACzC,aAAa,EAAE,MAAM,CAAC,YAAY,CAAC,CAAC,CAAC,MAAM,CAAC,cAAc,CAAC,CAAC,CAAC,SAAS;aACvE,CAAC;QACJ,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YAEf,IAAA,6CAAqB,EAAC,cAAc,EAAE,KAAK,EAAE,IAAI,CAAC,WAAW,CAAC,CAAC;YAC/D,OAAO,EAAE,WAAW,EAAE,KAAK,EAAE,CAAC;QAChC,CAAC;IACH,CAAC;CACF;AA9DD,gDA8DC"}

package/dist/extraction/EntityExtractor.d.ts

@@ -1,3 +1,4 @@
+import { type ExtractionDiagnostics } from './ExtractionDiagnostics';
 import type { LlmClient } from './LlmClient';
 import { type ExtractionSource } from './ExtractionSource';
 export interface ExtractedEntity {
@@ -7,8 +8,9 @@ export interface ExtractedEntity {
     confidence: number | null;
 }
 export declare class EntityExtractor {
+    private diagnostics?;
     private llm;
-    constructor(llm: LlmClient);
+    constructor(llm: LlmClient, diagnostics?: ExtractionDiagnostics | undefined);
     extract(humanMessage: string, aiMessage: string, entityTypes: string[], existingEntities: string[]): Promise<ExtractedEntity[]>;
     extractSources(sources: ExtractionSource[], entityTypes: string[], existingEntities: string[], requireConfidence?: boolean): Promise<ExtractedEntity[]>;
     private extractWithPrompt;

package/dist/extraction/EntityExtractor.js

@@ -1,10 +1,12 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
 exports.EntityExtractor = void 0;
+const ExtractionDiagnostics_1 = require("./ExtractionDiagnostics");
 const prompts_1 = require("./prompts");
 const ExtractionSource_1 = require("./ExtractionSource");
 class EntityExtractor {
-    constructor(llm) {
+    constructor(llm, diagnostics) {
+        this.diagnostics = diagnostics;
         this.llm = llm;
     }
     async extract(humanMessage, aiMessage, entityTypes, existingEntities) {
@@ -14,27 +16,39 @@ class EntityExtractor {
         return this.extractWithPrompt((0, prompts_1.entityExtractionSourcesUser)((0, ExtractionSource_1.formatExtractionSources)(sources), entityTypes, existingEntities), entityTypes, requireConfidence);
     }
     async extractWithPrompt(prompt, entityTypes, requireConfidence) {
+        var _a;
         try {
             const result = await this.llm.chatJson([
                 { role: 'system', content: prompts_1.ENTITY_EXTRACTION_SYSTEM },
                 { role: 'user', content: prompt },
             ]);
-            if (!result.entities || !Array.isArray(result.entities)) {
-                return [];
-            }
+            if (!result || !Array.isArray(result.entities))
+                throw new ExtractionDiagnostics_1.ExtractionError('invalid_response');
+            (_a = this.diagnostics) === null || _a === void 0 ? void 0 : _a.success('entities');
             const allowedTypes = new Set(entityTypes.map((type) => type.toLowerCase().trim()));
             return result.entities
-                .filter((e) => typeof e.name === 'string' &&
-                e.name.trim() !== '' &&
-                typeof e.entity_type === 'string' &&
-                e.entity_type.trim() !== '' &&
-                typeof e.summary === 'string' &&
-                (!requireConfidence ||
+                .filter((e) => {
+                var _a, _b;
+                if (!e ||
+                    typeof e !== 'object' ||
+                    typeof e.name !== 'string' ||
+                    !e.name.trim() ||
+                    typeof e.entity_type !== 'string' ||
+                    !e.entity_type.trim() ||
+                    typeof e.summary !== 'string') {
+                    (_a = this.diagnostics) === null || _a === void 0 ? void 0 : _a.malformed('entities');
+                    return false;
+                }
+                const eligible = (!requireConfidence ||
                     (typeof e.confidence === 'number' &&
                         Number.isFinite(e.confidence) &&
                         e.confidence >= 0 &&
                         e.confidence <= 1)) &&
-                (allowedTypes.size === 0 || allowedTypes.has(e.entity_type.toLowerCase().trim())))
+                    (allowedTypes.size === 0 || allowedTypes.has(e.entity_type.toLowerCase().trim()));
+                if (!eligible)
+                    (_b = this.diagnostics) === null || _b === void 0 ? void 0 : _b.increment('filtered');
+                return eligible;
+            })
                 .map((e) => ({
                 name: e.name.trim(),
                 entity_type: e.entity_type.trim(),
@@ -48,7 +62,7 @@ class EntityExtractor {
             }));
         }
         catch (error) {
-            console.warn('Engram: Entity extraction failed:', error.message);
+            (0, ExtractionDiagnostics_1.reportExtractionError)('entities', error, this.diagnostics);
             return [];
         }
     }

package/dist/extraction/EntityExtractor.js.map

@@ -1 +1 @@
-{"version":3,"file":"EntityExtractor.js","sourceRoot":"","sources":["../../src/extraction/EntityExtractor.ts"],"names":[],"mappings":";;;AACA,uCAImB;AACnB,yDAAoF;AAapF,MAAa,eAAe;IAG1B,YAAY,GAAc;QACxB,IAAI,CAAC,GAAG,GAAG,GAAG,CAAC;IACjB,CAAC;IAED,KAAK,CAAC,OAAO,CACX,YAAoB,EACpB,SAAiB,EACjB,WAAqB,EACrB,gBAA0B;QAE1B,OAAO,IAAI,CAAC,iBAAiB,CAC3B,IAAA,8BAAoB,EAAC,YAAY,EAAE,SAAS,EAAE,WAAW,EAAE,gBAAgB,CAAC,EAC5E,WAAW,EACX,KAAK,CACN,CAAC;IACJ,CAAC;IAED,KAAK,CAAC,cAAc,CAClB,OAA2B,EAC3B,WAAqB,EACrB,gBAA0B,EAC1B,iBAAiB,GAAG,KAAK;QAEzB,OAAO,IAAI,CAAC,iBAAiB,CAC3B,IAAA,qCAA2B,EAAC,IAAA,0CAAuB,EAAC,OAAO,CAAC,EAAE,WAAW,EAAE,gBAAgB,CAAC,EAC5F,WAAW,EACX,iBAAiB,CAClB,CAAC;IACJ,CAAC;IAEO,KAAK,CAAC,iBAAiB,CAC7B,MAAc,EACd,WAAqB,EACrB,iBAA0B;QAE1B,IAAI,CAAC;YACH,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAyB;gBAC7D,EAAE,IAAI,EAAE,QAAQ,EAAE,OAAO,EAAE,kCAAwB,EAAE;gBACrD,EAAE,IAAI,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,EAAE;aAClC,CAAC,CAAC;YAEH,IAAI,CAAC,MAAM,CAAC,QAAQ,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,MAAM,CAAC,QAAQ,CAAC,EAAE,CAAC;gBACxD,OAAO,EAAE,CAAC;YACZ,CAAC;YAED,MAAM,YAAY,GAAG,IAAI,GAAG,CAAC,WAAW,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,CAAC,WAAW,EAAE,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC;YAEnF,OAAO,MAAM,CAAC,QAAQ;iBACnB,MAAM,CACL,CAAC,CAAC,EAAE,EAAE,CACJ,OAAO,CAAC,CAAC,IAAI,KAAK,QAAQ;gBAC1B,CAAC,CAAC,IAAI,CAAC,IAAI,EAAE,KAAK,EAAE;gBACpB,OAAO,CAAC,CAAC,WAAW,KAAK,QAAQ;gBACjC,CAAC,CAAC,WAAW,CAAC,IAAI,EAAE,KAAK,EAAE;gBAC3B,OAAO,CAAC,CAAC,OAAO,KAAK,QAAQ;gBAC7B,CAAC,CAAC,iBAAiB;oBACjB,CAAC,OAAO,CAAC,CAAC,UAAU,KAAK,QAAQ;wBAC/B,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC,UAAU,CAAC;wBAC7B,CAAC,CAAC,UAAU,IAAI,CAAC;wBACjB,CAAC,CAAC,UAAU,IAAI,CAAC,CAAC,CAAC;gBACvB,CAAC,YAAY,CAAC,IAAI,KAAK,CAAC,IAAI,YAAY,CAAC,GAAG,CAAC,CAAC,CAAC,WAAW,CAAC,WAAW,EAAE,CAAC,IAAI,EAAE,CAAC,CAAC,CACpF;iBACA,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;gBACX,IAAI,EAAE,CAAC,CAAC,IAAI,CAAC,IAAI,EAAE;gBACnB,WAAW,EAAE,CAAC,CAAC,WAAW,CAAC,IAAI,EAAE;gBACjC,OAAO,EAAE,CAAC,CAAC,OAAO,CAAC,IAAI,EAAE;gBACzB,UAAU,EACR,OAAO,CAAC,CAAC,UAAU,KAAK,QAAQ;oBAChC,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC,UAAU,CAAC;oBAC7B,CAAC,CAAC,UAAU,IAAI,CAAC;oBACjB,CAAC,CAAC,UAAU,IAAI,CAAC;oBACf,CAAC,CAAC,CAAC,CAAC,UAAU;oBACd,CAAC,CAAC,IAAI;aACX,CAAC,CAAC,CAAC;QACR,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YAEf,OAAO,CAAC,IAAI,CAAC,mCAAmC,EAAG,KAAe,CAAC,OAAO,CAAC,CAAC;YAC5E,OAAO,EAAE,CAAC;QACZ,CAAC;IACH,CAAC;CACF;AAnFD,0CAmFC"}
+{"version":3,"file":"EntityExtractor.js","sourceRoot":"","sources":["../../src/extraction/EntityExtractor.ts"],"names":[],"mappings":";;;AAAA,mEAIiC;AAEjC,uCAImB;AACnB,yDAAoF;AAapF,MAAa,eAAe;IAG1B,YACE,GAAc,EACN,WAAmC;QAAnC,gBAAW,GAAX,WAAW,CAAwB;QAE3C,IAAI,CAAC,GAAG,GAAG,GAAG,CAAC;IACjB,CAAC;IAED,KAAK,CAAC,OAAO,CACX,YAAoB,EACpB,SAAiB,EACjB,WAAqB,EACrB,gBAA0B;QAE1B,OAAO,IAAI,CAAC,iBAAiB,CAC3B,IAAA,8BAAoB,EAAC,YAAY,EAAE,SAAS,EAAE,WAAW,EAAE,gBAAgB,CAAC,EAC5E,WAAW,EACX,KAAK,CACN,CAAC;IACJ,CAAC;IAED,KAAK,CAAC,cAAc,CAClB,OAA2B,EAC3B,WAAqB,EACrB,gBAA0B,EAC1B,iBAAiB,GAAG,KAAK;QAEzB,OAAO,IAAI,CAAC,iBAAiB,CAC3B,IAAA,qCAA2B,EAAC,IAAA,0CAAuB,EAAC,OAAO,CAAC,EAAE,WAAW,EAAE,gBAAgB,CAAC,EAC5F,WAAW,EACX,iBAAiB,CAClB,CAAC;IACJ,CAAC;IAEO,KAAK,CAAC,iBAAiB,CAC7B,MAAc,EACd,WAAqB,EACrB,iBAA0B;;QAE1B,IAAI,CAAC;YACH,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAyB;gBAC7D,EAAE,IAAI,EAAE,QAAQ,EAAE,OAAO,EAAE,kCAAwB,EAAE;gBACrD,EAAE,IAAI,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,EAAE;aAClC,CAAC,CAAC;YAEH,IAAI,CAAC,MAAM,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,MAAM,CAAC,QAAQ,CAAC;gBAAE,MAAM,IAAI,uCAAe,CAAC,kBAAkB,CAAC,CAAC;YAC9F,MAAA,IAAI,CAAC,WAAW,0CAAE,OAAO,CAAC,UAAU,CAAC,CAAC;YAEtC,MAAM,YAAY,GAAG,IAAI,GAAG,CAAC,WAAW,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,CAAC,WAAW,EAAE,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC;YAEnF,OAAO,MAAM,CAAC,QAAQ;iBACnB,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE;;gBACZ,IACE,CAAC,CAAC;oBACF,OAAO,CAAC,KAAK,QAAQ;oBACrB,OAAO,CAAC,CAAC,IAAI,KAAK,QAAQ;oBAC1B,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,EAAE;oBACd,OAAO,CAAC,CAAC,WAAW,KAAK,QAAQ;oBACjC,CAAC,CAAC,CAAC,WAAW,CAAC,IAAI,EAAE;oBACrB,OAAO,CAAC,CAAC,OAAO,KAAK,QAAQ,EAC7B,CAAC;oBACD,MAAA,IAAI,CAAC,WAAW,0CAAE,SAAS,CAAC,UAAU,CAAC,CAAC;oBACxC,OAAO,KAAK,CAAC;gBACf,CAAC;gBACD,MAAM,QAAQ,GACZ,CAAC,CAAC,iBAAiB;oBACjB,CAAC,OAAO,CAAC,CAAC,UAAU,KAAK,QAAQ;wBAC/B,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC,UAAU,CAAC;wBAC7B,CAAC,CAAC,UAAU,IAAI,CAAC;wBACjB,CAAC,CAAC,UAAU,IAAI,CAAC,CAAC,CAAC;oBACvB,CAAC,YAAY,CAAC,IAAI,KAAK,CAAC,IAAI,YAAY,CAAC,GAAG,CAAC,CAAC,CAAC,WAAW,CAAC,WAAW,EAAE,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC;gBACpF,IAAI,CAAC,QAAQ;oBAAE,MAAA,IAAI,CAAC,WAAW,0CAAE,SAAS,CAAC,UAAU,CAAC,CAAC;gBACvD,OAAO,QAAQ,CAAC;YAClB,CAAC,CAAC;iBACD,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;gBACX,IAAI,EAAE,CAAC,CAAC,IAAI,CAAC,IAAI,EAAE;gBACnB,WAAW,EAAE,CAAC,CAAC,WAAW,CAAC,IAAI,EAAE;gBACjC,OAAO,EAAE,CAAC,CAAC,OAAO,CAAC,IAAI,EAAE;gBACzB,UAAU,EACR,OAAO,CAAC,CAAC,UAAU,KAAK,QAAQ;oBAChC,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC,UAAU,CAAC;oBAC7B,CAAC,CAAC,UAAU,IAAI,CAAC;oBACjB,CAAC,CAAC,UAAU,IAAI,CAAC;oBACf,CAAC,CAAC,CAAC,CAAC,UAAU;oBACd,CAAC,CAAC,IAAI;aACX,CAAC,CAAC,CAAC;QACR,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YAEf,IAAA,6CAAqB,EAAC,UAAU,EAAE,KAAK,EAAE,IAAI,CAAC,WAAW,CAAC,CAAC;YAC3D,OAAO,EAAE,CAAC;QACZ,CAAC;IACH,CAAC;CACF;AA9FD,0CA8FC"}

package/dist/extraction/ExtractionDiagnostics.d.ts

@@ -0,0 +1,23 @@
+import { type ExtractionStage, type ExtractionErrorKind, type ExtractionOutcome } from '../schemas';
+export declare class ExtractionError extends Error {
+    readonly kind: ExtractionErrorKind;
+    readonly httpStatus?: number | undefined;
+    constructor(kind: ExtractionErrorKind, httpStatus?: number | undefined);
+}
+export declare function safeExtractionError(error: unknown): ExtractionError;
+export declare function reportExtractionError(stage: ExtractionStage, error: unknown, diagnostics?: ExtractionDiagnostics): void;
+export declare class ExtractionDiagnostics {
+    private readonly started;
+    private readonly outcome;
+    private readonly recordedErrors;
+    private requiredFailure;
+    private optionalFailure;
+    get errorCount(): number;
+    increment(key: keyof ExtractionOutcome['counts']): void;
+    hasFailed(stage: ExtractionStage): boolean;
+    success(stage: ExtractionStage): void;
+    skip(stage: ExtractionStage): void;
+    malformed(stage: ExtractionStage): void;
+    error(stage: ExtractionStage, error: ExtractionError): boolean;
+    finish(reason?: ExtractionOutcome['reason']): ExtractionOutcome;
+}