@semiont/make-meaning 0.5.11 → 0.5.12
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +10 -10
- package/dist/index.d.ts +140 -107
- package/dist/index.js +532 -618
- package/dist/index.js.map +1 -1
- package/dist/smelter-main.js +115 -15
- package/dist/smelter-main.js.map +1 -1
- package/dist/weaver-main.js +10732 -0
- package/dist/weaver-main.js.map +1 -0
- package/package.json +17 -13
package/README.md
CHANGED
|
@@ -20,7 +20,7 @@ Five **access actors** mediate every read and write — the bus-facing interface
|
|
|
20
20
|
|
|
21
21
|
Two **projection pipelines** follow the event log to keep the eventually-consistent read models in sync — addressed by no one, replying to nothing:
|
|
22
22
|
|
|
23
|
-
- **
|
|
23
|
+
- **Weaver** (project) — subscribes to graph-relevant domain events and projects them into the graph database; carried on the KB record (`kb.weaver`) and rebuilt from the event log at startup (`rebuildAll()`)
|
|
24
24
|
- **Smelter** (embed) — standalone embedding pipeline run via `@semiont/make-meaning/smelter-main` (not started by `startMakeMeaning`); subscribes to domain events, reads content from the KB working tree via `WorkerContentTransport`, chunks text, embeds via `@semiont/vectors`, and indexes into the vector store (Qdrant). On startup it reconciles Qdrant against the KS catalog — re-embedding what's missing or stale (every upsert is stamped with the embedded bytes' checksum, so changed content is detected) and deleting orphans — so a wiped Qdrant volume, or events missed while the worker was down, recover by restarting the smelter
|
|
25
25
|
|
|
26
26
|
(The third derived read model — the materialized views — is not pipeline-maintained: the EventStore's `ViewManager` materializes views synchronously inside `appendEvent()` for a read-your-writes guarantee.)
|
|
@@ -60,7 +60,7 @@ await makeMeaning.stop();
|
|
|
60
60
|
|
|
61
61
|
This single call initializes:
|
|
62
62
|
- **KnowledgeSystem** — groups the Knowledge Base and its actors
|
|
63
|
-
- **KnowledgeBase** — groups EventStore, ViewStorage, WorkingTreeStore, GraphDatabase,
|
|
63
|
+
- **KnowledgeBase** — groups EventStore, ViewStorage, WorkingTreeStore, GraphDatabase, Weaver, and optionally VectorStore
|
|
64
64
|
- **Stower** — subscribes to write commands on EventBus
|
|
65
65
|
- **Browser** — subscribes to all KB read queries and directory browse requests on EventBus
|
|
66
66
|
- **Gatherer** — subscribes to annotation and resource gather requests on EventBus; searches vectors for semantically similar passages
|
|
@@ -113,7 +113,7 @@ graph TB
|
|
|
113
113
|
GATHERER["Gatherer<br/>(context assembly)"]
|
|
114
114
|
MATCHER["Matcher<br/>(search/link)"]
|
|
115
115
|
SMELTER["Smelter<br/>(embed pipeline, standalone process)"]
|
|
116
|
-
|
|
116
|
+
WEAVER["Weaver<br/>(graph pipeline)"]
|
|
117
117
|
CTM["CloneTokenManager<br/>(clone)"]
|
|
118
118
|
KB["Knowledge Base"]
|
|
119
119
|
VECTORS["Vector Store<br/>(Qdrant)"]
|
|
@@ -125,7 +125,7 @@ graph TB
|
|
|
125
125
|
MATCHER -->|search| VECTORS
|
|
126
126
|
SMELTER -->|embed & index| VECTORS
|
|
127
127
|
SMELTER -->|read| KB
|
|
128
|
-
|
|
128
|
+
WEAVER -->|project| KB
|
|
129
129
|
CTM -->|query| KB
|
|
130
130
|
end
|
|
131
131
|
|
|
@@ -134,7 +134,7 @@ graph TB
|
|
|
134
134
|
BUS -->|"gather:requested<br/>gather:resource-requested"| GATHERER
|
|
135
135
|
BUS -->|"match:search-requested"| MATCHER
|
|
136
136
|
BUS -->|"domain events:<br/>yield:created, yield:updated<br/>yield:representation-added<br/>mark:added, mark:removed, mark:archived"| SMELTER
|
|
137
|
-
BUS -->|"graph-relevant<br/>domain events"|
|
|
137
|
+
BUS -->|"graph-relevant<br/>domain events"| WEAVER
|
|
138
138
|
BUS -->|"yield:clone-token-requested<br/>yield:clone-resource-requested<br/>yield:clone-create"| CTM
|
|
139
139
|
|
|
140
140
|
STOWER -->|"yield:create-ok, yield:update-ok, yield:move-ok<br/>mark:delete-ok, *-failed replies<br/>(domain events are republished onto the bus<br/>by the EventStore: yield:created, mark:added, ...)"| BUS
|
|
@@ -150,7 +150,7 @@ graph TB
|
|
|
150
150
|
|
|
151
151
|
class BUS bus
|
|
152
152
|
classDef vectorstore fill:#6b8e9d,stroke:#4a6a7a,stroke-width:2px,color:#fff
|
|
153
|
-
class STOWER,BROWSER,GATHERER,MATCHER,SMELTER,
|
|
153
|
+
class STOWER,BROWSER,GATHERER,MATCHER,SMELTER,WEAVER,CTM actor
|
|
154
154
|
class KB kb
|
|
155
155
|
class VECTORS vectorstore
|
|
156
156
|
class Routes,Workers,EBC caller
|
|
@@ -168,7 +168,7 @@ The **Knowledge Base** is an inert store — it has no intelligence, no goals, n
|
|
|
168
168
|
| **Materialized Views** | `ViewStorage` | Denormalized projections for fast reads (materialized synchronously on append) |
|
|
169
169
|
| **Content Store** | `WorkingTreeStore` | Working-tree files addressed by URI |
|
|
170
170
|
| **Graph** | `GraphDatabase` | Eventually consistent relationship projection |
|
|
171
|
-
| **
|
|
171
|
+
| **Weaver** | `Weaver` | Event-to-graph projection pipeline (one of the two pipeline actors; carried on the KB record because `createKnowledgeBase()` constructs and starts it) |
|
|
172
172
|
| **Vectors** *(optional)* | `VectorStore` | Semantic vector index (Qdrant + memory) via `@semiont/vectors` |
|
|
173
173
|
|
|
174
174
|
Its sibling pipeline, the Smelter (event-to-vector projection), is **not** a KB member — it runs as a standalone process via `@semiont/make-meaning/smelter-main`.
|
|
@@ -177,7 +177,7 @@ Its sibling pipeline, the Smelter (event-to-vector projection), is **not** a KB
|
|
|
177
177
|
import { createKnowledgeBase } from '@semiont/make-meaning';
|
|
178
178
|
|
|
179
179
|
const kb = await createKnowledgeBase(eventStore, project, graphDb, eventBus, logger, options);
|
|
180
|
-
// kb.eventStore, kb.views, kb.content, kb.graph, kb.
|
|
180
|
+
// kb.eventStore, kb.views, kb.content, kb.graph, kb.weaver
|
|
181
181
|
// kb.vectors (optional), kb.projectionsDir
|
|
182
182
|
```
|
|
183
183
|
|
|
@@ -221,7 +221,7 @@ This pattern (functional core, imperative shell) is shared with `@semiont/event-
|
|
|
221
221
|
### Knowledge Base
|
|
222
222
|
|
|
223
223
|
- `createKnowledgeBase(eventStore, project, graphDb, eventBus, logger, options?)` — Async factory function
|
|
224
|
-
- `KnowledgeBase` — Interface grouping the KB stores (`eventStore`, `views`, `content`, `graph`, optional `vectors`) plus the `
|
|
224
|
+
- `KnowledgeBase` — Interface grouping the KB stores (`eventStore`, `views`, `content`, `graph`, optional `vectors`) plus the `weaver` pipeline
|
|
225
225
|
|
|
226
226
|
### Actors
|
|
227
227
|
|
|
@@ -232,7 +232,7 @@ This pattern (functional core, imperative shell) is shared with `@semiont/event-
|
|
|
232
232
|
- `CloneTokenManager` — Clone token lifecycle actor (yield domain)
|
|
233
233
|
- `Smelter` / `createSmelterActorStateUnit` / `WorkerContentTransport` — the embedding pipeline, its domain-event fan-in, and the worker-side content transport; wired together by the standalone `@semiont/make-meaning/smelter-main` entry point, and exported for callers that run the pipeline on their own `WorkerBus`
|
|
234
234
|
|
|
235
|
-
The
|
|
235
|
+
The Weaver is not exported — `createKnowledgeBase()` constructs it internally and exposes it as `kb.weaver`.
|
|
236
236
|
|
|
237
237
|
### Operations
|
|
238
238
|
|
package/dist/index.d.ts
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
import { JobQueue } from '@semiont/jobs';
|
|
2
2
|
import { SemiontProject } from '@semiont/core/node';
|
|
3
|
-
import { GraphServiceConfig, VectorsServiceConfig, EmbeddingServiceConfig, EventBus, Logger,
|
|
3
|
+
import { GraphServiceConfig, VectorsServiceConfig, EmbeddingServiceConfig, StateUnit, EventBus, Logger, ResourceId, ResourceDescriptor, AnnotationId, components, ITransport, BaseUrl, ConnectionState, SemiontError, UserDID, EventMap, IContentTransport, PutBinaryRequest, PutBinaryOptions, AccessToken, BusRequestPrimitive, StoredEvent, Annotation, UserId, GatheredContext, ResourceAnnotations, AnnotationCategory, GraphPath, GraphConnection } from '@semiont/core';
|
|
4
4
|
import { EventStore, ViewStorage } from '@semiont/event-sourcing';
|
|
5
5
|
import { WorkingTreeStore } from '@semiont/content';
|
|
6
6
|
import { GraphDatabase } from '@semiont/graph';
|
|
@@ -66,104 +66,79 @@ interface MakeMeaningConfig {
|
|
|
66
66
|
}
|
|
67
67
|
|
|
68
68
|
/**
|
|
69
|
-
*
|
|
70
|
-
*
|
|
71
|
-
*
|
|
72
|
-
*
|
|
73
|
-
*
|
|
74
|
-
*
|
|
75
|
-
*
|
|
76
|
-
*
|
|
77
|
-
*
|
|
78
|
-
*
|
|
79
|
-
*
|
|
80
|
-
*
|
|
81
|
-
*
|
|
82
|
-
*
|
|
83
|
-
*
|
|
84
|
-
*
|
|
85
|
-
*
|
|
86
|
-
*
|
|
87
|
-
*
|
|
88
|
-
*
|
|
89
|
-
* `groupBy(resourceId) + concatMap(...)` is the stream-consumer flavor of
|
|
90
|
-
* per-resource serialization — the same invariant enforced by `Smelter`,
|
|
91
|
-
* `Gatherer`, and (in a different shape) `ViewManager`. See
|
|
92
|
-
* `packages/core/src/serialize-per-key.ts` for the shared primitive used
|
|
93
|
-
* by RPC-style services.
|
|
69
|
+
* WeaveProgress — backend-local fold of `weave:applied` signals
|
|
70
|
+
* (GRAPH-PROJECTION-SYNC P2, D2 = push).
|
|
71
|
+
*
|
|
72
|
+
* The Weaver emits `weave:applied` after applying an event (or a batch's
|
|
73
|
+
* last event) for a resource. This unit folds those signals into a
|
|
74
|
+
* per-resource applied-sequence map and exposes `whenApplied` — the
|
|
75
|
+
* applied-offset barrier: an event-driven await that resolves the moment
|
|
76
|
+
* the graph projection reaches parity with a known sequence (typically the
|
|
77
|
+
* view's `lastSequence`), and rejects with `WeaveProgressTimeout` on the
|
|
78
|
+
* bounded timeout so callers can fall back to the bounded-poll floor.
|
|
79
|
+
*
|
|
80
|
+
* Deliberately transport-blind: it subscribes to the channel, not to the
|
|
81
|
+
* Weaver. In-process the signal rides the core EventBus; after
|
|
82
|
+
* WEAVER-ISOLATION the same channel arrives through the bus gateway and
|
|
83
|
+
* this unit does not change.
|
|
84
|
+
*
|
|
85
|
+
* The map is ephemeral by design — on backend restart it rebuilds lazily
|
|
86
|
+
* from live signals. That loses nothing: a waiter only ever waits for an
|
|
87
|
+
* apply that has not happened yet, and those signals are still to come.
|
|
94
88
|
*/
|
|
95
89
|
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
private coreEventBus;
|
|
100
|
-
private static readonly GRAPH_RELEVANT_EVENTS;
|
|
101
|
-
private static readonly BURST_WINDOW_MS;
|
|
102
|
-
private static readonly MAX_BATCH_SIZE;
|
|
103
|
-
private static readonly IDLE_TIMEOUT_MS;
|
|
104
|
-
private _globalSubscriptions;
|
|
105
|
-
private eventSubject;
|
|
106
|
-
private pipelineSubscription;
|
|
107
|
-
private lastProcessed;
|
|
108
|
-
private readonly logger;
|
|
109
|
-
constructor(eventStore: EventStore, graphDb: GraphDatabase, coreEventBus: EventBus, logger: Logger);
|
|
110
|
-
initialize(): Promise<void>;
|
|
111
|
-
/**
|
|
112
|
-
* Subscribe globally to ALL events, pre-filter to graph-relevant types,
|
|
113
|
-
* and wire through the RxJS burst-buffered pipeline.
|
|
114
|
-
*/
|
|
115
|
-
private subscribeToGlobalEvents;
|
|
116
|
-
/**
|
|
117
|
-
* Wrap applyEventToGraph in try/catch so one failed event doesn't kill the pipeline.
|
|
118
|
-
*/
|
|
119
|
-
private safeApplyEvent;
|
|
120
|
-
private ensureInitialized;
|
|
121
|
-
/**
|
|
122
|
-
* Stop the consumer, flush remaining buffered events, and unsubscribe.
|
|
123
|
-
*/
|
|
124
|
-
stop(): Promise<void>;
|
|
125
|
-
/**
|
|
126
|
-
* Process a batch of events for the same resource.
|
|
127
|
-
* Partitions into consecutive same-type runs for batch optimization.
|
|
128
|
-
*/
|
|
129
|
-
private processBatch;
|
|
130
|
-
/**
|
|
131
|
-
* Batch-optimized processing for consecutive events of the same type.
|
|
132
|
-
* Uses batch graph methods where available, falls back to sequential.
|
|
133
|
-
*/
|
|
134
|
-
private applyBatchByType;
|
|
135
|
-
/**
|
|
136
|
-
* Build a ResourceDescriptor from a resource.created event.
|
|
137
|
-
* Extracted for reuse by both applyEventToGraph and applyBatchByType.
|
|
138
|
-
*/
|
|
139
|
-
private buildResourceDescriptor;
|
|
140
|
-
/**
|
|
141
|
-
* Apply a single event to GraphDB.
|
|
142
|
-
*/
|
|
143
|
-
protected applyEventToGraph(storedEvent: StoredEvent): Promise<void>;
|
|
144
|
-
/**
|
|
145
|
-
* Rebuild entire resource from events.
|
|
146
|
-
* Bypasses the live pipeline — reads directly from event store.
|
|
147
|
-
*/
|
|
148
|
-
rebuildResource(resourceId: ResourceId): Promise<void>;
|
|
90
|
+
interface WeaveProgress extends StateUnit {
|
|
91
|
+
/** Highest applied sequence seen for a resource, if any signal arrived. */
|
|
92
|
+
appliedUpTo(resourceId: string): number | undefined;
|
|
149
93
|
/**
|
|
150
|
-
*
|
|
151
|
-
*
|
|
152
|
-
*
|
|
94
|
+
* Resolve when the Weaver has applied at least `sequenceNumber` for
|
|
95
|
+
* `resourceId` — immediately if the fold already covers it. Rejects with
|
|
96
|
+
* `WeaveProgressTimeout` after `timeoutMs`. After dispose it resolves
|
|
97
|
+
* immediately (inert): barrier callers degrade to their poll floor.
|
|
153
98
|
*/
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
99
|
+
whenApplied(resourceId: string, sequenceNumber: number, timeoutMs: number): Promise<void>;
|
|
100
|
+
}
|
|
101
|
+
|
|
102
|
+
/**
|
|
103
|
+
* SmeltProgress — backend-local fold of `smelt:settled` signals
|
|
104
|
+
* (SMELTER-INDEX-SYNC P1, D1 = push barrier).
|
|
105
|
+
*
|
|
106
|
+
* The Smelter emits `smelt:settled` after deciding a resource's content:
|
|
107
|
+
* `indexed` (embedded + upserted) or `skipped` (media gate, empty text) —
|
|
108
|
+
* keyed by the checksum of the bytes it inspected, and NEVER on transient
|
|
109
|
+
* failures (an error is not a decision; SMELTER-INDEX-SYNC A2). This unit
|
|
110
|
+
* folds those signals per resource and exposes `whenSettled` — the
|
|
111
|
+
* read-your-writes barrier: an event-driven await that resolves the moment
|
|
112
|
+
* the vector projection has settled the exact content generation the caller
|
|
113
|
+
* holds (the view's checksum), and rejects with `SmeltProgressTimeout` on
|
|
114
|
+
* the bounded timeout so callers degrade observably (L4 breadcrumb).
|
|
115
|
+
*
|
|
116
|
+
* Deliberately transport-blind: it subscribes to the channel, not to the
|
|
117
|
+
* Smelter. The signal arrives through the bus gateway from the standalone
|
|
118
|
+
* worker; an in-process Smelter would ride the core EventBus and this unit
|
|
119
|
+
* would not change (the WeaveProgress precedent).
|
|
120
|
+
*
|
|
121
|
+
* The fold is ephemeral by design — on backend restart it rebuilds lazily
|
|
122
|
+
* from live signals. Barrier callers probe the vector store first
|
|
123
|
+
* (SMELTER-INDEX-SYNC A3), so a cold fold only costs waits for resources
|
|
124
|
+
* whose settlement genuinely hasn't been observed yet.
|
|
125
|
+
*/
|
|
126
|
+
|
|
127
|
+
type SmeltOutcome = 'indexed' | 'skipped';
|
|
128
|
+
interface SmeltProgress extends StateUnit {
|
|
129
|
+
/** The latest settlement seen for a resource, if any signal arrived. */
|
|
130
|
+
settledAt(resourceId: string): {
|
|
131
|
+
contentChecksum: string;
|
|
132
|
+
outcome: SmeltOutcome;
|
|
133
|
+
} | undefined;
|
|
163
134
|
/**
|
|
164
|
-
*
|
|
135
|
+
* Resolve with the Smelter's decision once it has settled `resourceId` at
|
|
136
|
+
* exactly `contentChecksum` — immediately if the fold already holds it.
|
|
137
|
+
* Rejects with `SmeltProgressTimeout` after `timeoutMs`. After dispose it
|
|
138
|
+
* resolves `'inert'`: shutdown must not throw through a gather in flight,
|
|
139
|
+
* and callers treat inert as breadcrumb-less degrade.
|
|
165
140
|
*/
|
|
166
|
-
|
|
141
|
+
whenSettled(resourceId: string, contentChecksum: string, timeoutMs: number): Promise<SmeltOutcome | 'inert'>;
|
|
167
142
|
}
|
|
168
143
|
|
|
169
144
|
/**
|
|
@@ -176,7 +151,10 @@ declare class GraphDBConsumer {
|
|
|
176
151
|
* - Materialized Views (fast single-doc queries) — via ViewStorage
|
|
177
152
|
* - Content Store (working-tree files, URI-addressed) — via WorkingTreeStore
|
|
178
153
|
* - Graph (eventually consistent relationship projection) — via GraphDatabase
|
|
179
|
-
* -
|
|
154
|
+
* - WeaveProgress (weave:applied fold — the graph-projection barrier; the
|
|
155
|
+
* Weaver itself runs standalone via @semiont/make-meaning/weaver-main)
|
|
156
|
+
* - SmeltProgress (smelt:settled fold — the vector-projection barrier;
|
|
157
|
+
* SMELTER-INDEX-SYNC, same standalone-actor arrangement as the Weaver)
|
|
180
158
|
* - Vectors (semantic search) — via VectorStore (optional, read-only)
|
|
181
159
|
*
|
|
182
160
|
* The Smelter (event-to-vector projection) runs as an external actor
|
|
@@ -189,7 +167,8 @@ interface KnowledgeBase {
|
|
|
189
167
|
views: ViewStorage;
|
|
190
168
|
content: WorkingTreeStore;
|
|
191
169
|
graph: GraphDatabase;
|
|
192
|
-
|
|
170
|
+
weaveProgress: WeaveProgress;
|
|
171
|
+
smeltProgress: SmeltProgress;
|
|
193
172
|
vectors?: VectorStore;
|
|
194
173
|
projectionsDir: string;
|
|
195
174
|
}
|
|
@@ -288,7 +267,7 @@ declare class Stower {
|
|
|
288
267
|
*
|
|
289
268
|
* `groupBy(resourceId) + concatMap(...)` is the stream-consumer flavor of
|
|
290
269
|
* per-resource serialization — the same invariant enforced by `Smelter`,
|
|
291
|
-
* `
|
|
270
|
+
* `Weaver`, and (in a different shape) `ViewManager`. See
|
|
292
271
|
* `packages/core/src/serialize-per-key.ts` for the shared primitive used
|
|
293
272
|
* by RPC-style services.
|
|
294
273
|
*/
|
|
@@ -457,8 +436,10 @@ declare class CloneTokenManager {
|
|
|
457
436
|
* - cloneTokenManager: token actor — manages resource clone tokens
|
|
458
437
|
*
|
|
459
438
|
* These are the five access actors. Two projection-pipeline actors complete
|
|
460
|
-
* the seven: the
|
|
461
|
-
*
|
|
439
|
+
* the seven, and BOTH run standalone (D4: the projections are part of their
|
|
440
|
+
* stores' stacks, not of the embedding process): the Weaver (weaver-main →
|
|
441
|
+
* graph) and the Smelter (smelter-main → vectors). The backend keeps only
|
|
442
|
+
* the Weaver's `weave:applied` fold (kb.weaveProgress).
|
|
462
443
|
*
|
|
463
444
|
* EventBus, JobQueue, and workers are peers to KnowledgeSystem, not members.
|
|
464
445
|
*/
|
|
@@ -611,6 +592,17 @@ declare class LocalContentTransport implements IContentTransport {
|
|
|
611
592
|
dispose(): void;
|
|
612
593
|
}
|
|
613
594
|
|
|
595
|
+
/**
|
|
596
|
+
* Adapt a raw in-process `EventBus` to the `BusRequestPrimitive` that
|
|
597
|
+
* `busRequest` consumes. Lets backend-internal callers (bootstrap, event
|
|
598
|
+
* replay, linked-data import) use the same confirmed request/reply path as the
|
|
599
|
+
* SDK — `busRequest(asBusRequestPrimitive(eventBus), …)` — instead of
|
|
600
|
+
* hand-rolled `race(domain-event, *-failed, timeout)` blocks. The reply is
|
|
601
|
+
* matched by `correlationId`, so concurrent in-process writes can't cross-match
|
|
602
|
+
* (the latent bug in the old domain-event `race`).
|
|
603
|
+
*/
|
|
604
|
+
declare function asBusRequestPrimitive(eventBus: EventBus): BusRequestPrimitive;
|
|
605
|
+
|
|
614
606
|
/**
|
|
615
607
|
* Handles `mark:create-request` — the bus command for creating an annotation.
|
|
616
608
|
*
|
|
@@ -713,7 +705,7 @@ declare function readEntityTypesProjection(project: SemiontProject): Promise<str
|
|
|
713
705
|
/**
|
|
714
706
|
* SmelterActorStateUnit — domain-event fan-in for the Smelter worker.
|
|
715
707
|
*
|
|
716
|
-
* Subscribes to the
|
|
708
|
+
* Subscribes to the nine smelter-relevant channels on a shared bus and
|
|
717
709
|
* exposes them as a single typed `events$` stream. Transport-neutral —
|
|
718
710
|
* the caller passes a `WorkerBus` (HTTP `ActorStateUnit` today, an in-process
|
|
719
711
|
* bus shim if/when one exists). The state unit does not own the bus and does
|
|
@@ -754,7 +746,7 @@ declare function createSmelterActorStateUnit(options: SmelterActorStateUnitOptio
|
|
|
754
746
|
* Smelter processes events strictly in order per resourceId via
|
|
755
747
|
* `groupBy(resourceId) + concatMap(...)`. This is the stream-consumer
|
|
756
748
|
* flavor of per-resource serialization — the same invariant enforced by
|
|
757
|
-
* `
|
|
749
|
+
* `Weaver`, `Gatherer`, and (in a different shape) `ViewManager`.
|
|
758
750
|
* See `packages/core/src/serialize-per-key.ts` for the shared primitive
|
|
759
751
|
* used by RPC-style services.
|
|
760
752
|
*
|
|
@@ -768,15 +760,18 @@ declare function createSmelterActorStateUnit(options: SmelterActorStateUnitOptio
|
|
|
768
760
|
* Qdrant is an ephemeral projection of the event log. `reconcile()` brings
|
|
769
761
|
* it back in sync at startup — after a wiped volume, or after events missed
|
|
770
762
|
* while the worker was down. It is a planner: it diffs the store against the
|
|
771
|
-
* catalog (over the `browse:*` RPC channels) —
|
|
772
|
-
*
|
|
763
|
+
* catalog (over the `browse:*` RPC channels) — membership, content freshness
|
|
764
|
+
* (via the checksum stamped onto every resource upsert), and tag-stamp
|
|
765
|
+
* freshness (payload-only restamps; tag edits change no bytes) — and
|
|
773
766
|
* enqueues `smelt:*` work items through the same mailbox as live events, so
|
|
774
|
-
* per-resource ordering holds across the two paths (axioms S1/S2/S11/S12
|
|
775
|
-
* `.plans/SMELTER-AXIOMS.md`).
|
|
767
|
+
* per-resource ordering holds across the two paths (axioms S1/S2/S11/S12/S13
|
|
768
|
+
* in `.plans/SMELTER-AXIOMS.md`).
|
|
776
769
|
*/
|
|
777
770
|
|
|
778
771
|
interface ReconcileSummary {
|
|
779
772
|
resourcesEmbedded: number;
|
|
773
|
+
/** Tag-only drift healed by payload restamps — never embedding calls (S13). */
|
|
774
|
+
resourcesRestamped: number;
|
|
780
775
|
resourceVectorsDeleted: number;
|
|
781
776
|
annotationsEmbedded: number;
|
|
782
777
|
annotationVectorsDeleted: number;
|
|
@@ -809,7 +804,7 @@ interface SmelterTiming {
|
|
|
809
804
|
* lanes and batch paths serve both kinds of input.
|
|
810
805
|
*/
|
|
811
806
|
interface SmelterWorkItem {
|
|
812
|
-
type: 'smelt:embed' | 'smelt:purge' | 'smelt:embed-annotation' | 'smelt:purge-annotation';
|
|
807
|
+
type: 'smelt:embed' | 'smelt:restamp' | 'smelt:purge' | 'smelt:embed-annotation' | 'smelt:purge-annotation';
|
|
813
808
|
resourceId: string;
|
|
814
809
|
payload: Record<string, unknown>;
|
|
815
810
|
}
|
|
@@ -852,6 +847,15 @@ declare class Smelter {
|
|
|
852
847
|
/** Returns true if the input was processed without error. */
|
|
853
848
|
private safeProcessEvent;
|
|
854
849
|
private processEvent;
|
|
850
|
+
/**
|
|
851
|
+
* Payload-only stamp refresh: re-read the resource's CURRENT entity types
|
|
852
|
+
* (one code path — `resolveEntityTypes` — so any prior drift self-corrects
|
|
853
|
+
* on first touch) and rewrite the stamp on its existing points. Never calls
|
|
854
|
+
* the embedding provider (S13): content is unchanged by definition on every
|
|
855
|
+
* path that lands here. A resource with no points is a no-op — the stamp
|
|
856
|
+
* rides the next embed.
|
|
857
|
+
*/
|
|
858
|
+
private restampResource;
|
|
855
859
|
private handleResourcePurge;
|
|
856
860
|
/**
|
|
857
861
|
* Resolve a resource's embeddable text: bytes via the content transport,
|
|
@@ -862,6 +866,13 @@ declare class Smelter {
|
|
|
862
866
|
* or is empty — callers skip it.
|
|
863
867
|
*/
|
|
864
868
|
private fetchEmbeddableText;
|
|
869
|
+
/**
|
|
870
|
+
* The Smelter's single outbound signal (SMELTER-AXIOMS D3 as amended by
|
|
871
|
+
* SMELTER-INDEX-SYNC): a per-resource decision report for the barrier
|
|
872
|
+
* fold. Best-effort — waiters degrade to their bounded timeout; a signal
|
|
873
|
+
* failure must never fail the embed.
|
|
874
|
+
*/
|
|
875
|
+
private emitSettled;
|
|
865
876
|
/**
|
|
866
877
|
* Read a resource's current entity types from the materialized view — the
|
|
867
878
|
* authoritative source, updated before the EventBus fires to consumers — so
|
|
@@ -874,7 +885,15 @@ declare class Smelter {
|
|
|
874
885
|
private resolveEntityTypes;
|
|
875
886
|
private embedResource;
|
|
876
887
|
private handleResourceArchived;
|
|
888
|
+
/**
|
|
889
|
+
* Restore what `handleResourceArchived` deleted, from CURRENT state: the
|
|
890
|
+
* resource's vectors (media-gated, full-replace) and its current exact-text
|
|
891
|
+
* annotations — the same catalog read `reconcile()` uses, so the live path
|
|
892
|
+
* and a restart agree (bugs/smelter-misses-unarchive.md).
|
|
893
|
+
*/
|
|
894
|
+
private handleResourceUnarchived;
|
|
877
895
|
private handleAnnotationAdded;
|
|
896
|
+
private indexAnnotation;
|
|
878
897
|
private handleAnnotationRemoved;
|
|
879
898
|
/**
|
|
880
899
|
* Batch-embed chunks from multiple yield:created events in a single
|
|
@@ -1359,6 +1378,20 @@ declare class AnnotationContext {
|
|
|
1359
1378
|
|
|
1360
1379
|
type KnowledgeGraph = components['schemas']['KnowledgeGraph'];
|
|
1361
1380
|
declare class GraphContext {
|
|
1381
|
+
/**
|
|
1382
|
+
* Backoff schedule for the projection-lag grace in `buildKnowledgeGraph`
|
|
1383
|
+
* (GRAPH-PROJECTION-SYNC P1). Total wait is bounded at 375 ms — the Weaver
|
|
1384
|
+
* applies in tens of milliseconds when merely lagging; anything slower is
|
|
1385
|
+
* treated as a real miss.
|
|
1386
|
+
*/
|
|
1387
|
+
private static readonly PROJECTION_LAG_BACKOFF_MS;
|
|
1388
|
+
/**
|
|
1389
|
+
* Bounded wait for the applied-offset barrier (GRAPH-PROJECTION-SYNC P2).
|
|
1390
|
+
* The Weaver applies in tens of milliseconds when merely lagging; a
|
|
1391
|
+
* barrier that hasn't woken in 500 ms means signals have stalled and the
|
|
1392
|
+
* poll floor above owns the remainder.
|
|
1393
|
+
*/
|
|
1394
|
+
private static readonly PROJECTION_BARRIER_TIMEOUT_MS;
|
|
1362
1395
|
/**
|
|
1363
1396
|
* Get all resources referencing this resource (backlinks)
|
|
1364
1397
|
* Requires graph traversal - must use graph database
|
|
@@ -1419,7 +1452,7 @@ declare class LLMContext {
|
|
|
1419
1452
|
* Get comprehensive LLM context for a resource
|
|
1420
1453
|
* Includes: main resource, related resources, annotations, graph, content, summary, references
|
|
1421
1454
|
*/
|
|
1422
|
-
static getResourceContext(resourceId: ResourceId, options: LLMContextOptions, kb: KnowledgeBase, inferenceClient: InferenceClient): Promise<GatheredContext>;
|
|
1455
|
+
static getResourceContext(resourceId: ResourceId, options: LLMContextOptions, kb: KnowledgeBase, inferenceClient: InferenceClient, logger: Logger): Promise<GatheredContext>;
|
|
1423
1456
|
}
|
|
1424
1457
|
|
|
1425
1458
|
/**
|
|
@@ -1442,5 +1475,5 @@ declare function generateResourceSummary(resourceName: string, content: string,
|
|
|
1442
1475
|
*/
|
|
1443
1476
|
declare function generateReferenceSuggestions(referenceTitle: string, client: InferenceClient, entityType?: string, currentContent?: string): Promise<string[] | null>;
|
|
1444
1477
|
|
|
1445
|
-
export { AnnotationContext, AnnotationOperations, BACKUP_FORMAT, Browser, CloneTokenManager, FORMAT_VERSION, Gatherer$1 as Gatherer, GraphContext, LLMContext, LocalContentTransport, LocalTransport, Matcher, ResourceContext, ResourceOperations, Smelter, Stower, bootstrapEntityTypes, createKnowledgeBase, createSmelterActorStateUnit, exportBackup, exportLinkedData, generateReferenceSuggestions, generateResourceSummary, importBackup, importLinkedData, isBackupManifest, readEntityTypesProjection, registerAnnotationAssemblyHandler, registerAnnotationLookupHandlers, registerBindUpdateBodyHandler, registerBusHandlers, registerJobCommandHandlers, startMakeMeaning, stopKnowledgeSystem, validateManifestVersion };
|
|
1478
|
+
export { AnnotationContext, AnnotationOperations, BACKUP_FORMAT, Browser, CloneTokenManager, FORMAT_VERSION, Gatherer$1 as Gatherer, GraphContext, LLMContext, LocalContentTransport, LocalTransport, Matcher, ResourceContext, ResourceOperations, Smelter, Stower, asBusRequestPrimitive, bootstrapEntityTypes, createKnowledgeBase, createSmelterActorStateUnit, exportBackup, exportLinkedData, generateReferenceSuggestions, generateResourceSummary, importBackup, importLinkedData, isBackupManifest, readEntityTypesProjection, registerAnnotationAssemblyHandler, registerAnnotationLookupHandlers, registerBindUpdateBodyHandler, registerBusHandlers, registerJobCommandHandlers, startMakeMeaning, stopKnowledgeSystem, validateManifestVersion };
|
|
1446
1479
|
export type { BackupContentReader, BackupEventStoreReader, BackupExporterOptions, BackupImportResult, BackupImporterOptions, BackupManifestHeader, BackupStreamSummary, BuildContextOptions, ContentBlobResolver, CreateAnnotationResult, CreateResourceInput, CreateResourceResult, KnowledgeBase, KnowledgeSystem, LLMContextOptions, LinkedDataContentReader, LinkedDataExporterOptions, LinkedDataImportResult, LinkedDataImporterOptions, LinkedDataViewReader, ListResourcesFilters, LocalTransportConfig, MakeMeaningConfig, MakeMeaningService, ReconcileState, ReconcileSummary, ReplayStats, SmelterActorStateUnit, SmelterActorStateUnitOptions, SmelterEvent, SmelterInput, SmelterTiming, SmelterWorkItem, UpdateAnnotationBodyResult };
|