@semiont/make-meaning 0.5.10 → 0.5.12
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +13 -13
- package/dist/index.d.ts +159 -109
- package/dist/index.js +700 -702
- package/dist/index.js.map +1 -1
- package/dist/smelter-main.js +115 -15
- package/dist/smelter-main.js.map +1 -1
- package/dist/weaver-main.js +10732 -0
- package/dist/weaver-main.js.map +1 -0
- package/package.json +17 -13
package/README.md
CHANGED
|
@@ -13,14 +13,14 @@ This package implements the actor model from [ACTOR-MODEL.md](../../docs/system/
|
|
|
13
13
|
Five **access actors** mediate every read and write — the bus-facing interface of the Knowledge Base:
|
|
14
14
|
|
|
15
15
|
- **Stower** (write) — the single write gateway to the Knowledge Base; handles all resource and annotation mutations and job lifecycle events
|
|
16
|
-
- **Browser** (read) — handles all KB read queries: resources, annotations, events, annotation history, referenced-by lookups, entity type and tag-schema listing, and directory browse (merging filesystem listings with KB metadata)
|
|
16
|
+
- **Browser** (read) — handles all KB read queries: resources, annotations, events, annotation history, referenced-by lookups, entity type and tag-schema listing, the collaborator directory (the KB's declared software agents, derived from the workers/actors inference config), and directory browse (merging filesystem listings with KB metadata)
|
|
17
17
|
- **Gatherer** (context assembly) — assembles gathered context for annotations (`gather:requested`) and resources (`gather:resource-requested`); searches vectors for semantically similar passages (adds `semanticContext` to `GatheredContext`)
|
|
18
18
|
- **Matcher** (search/link) — context-driven candidate search with multi-source retrieval, composite structural scoring, and optional LLM semantic scoring
|
|
19
19
|
- **CloneTokenManager** (yield) — manages clone token lifecycle for resource cloning
|
|
20
20
|
|
|
21
21
|
Two **projection pipelines** follow the event log to keep the eventually-consistent read models in sync — addressed by no one, replying to nothing:
|
|
22
22
|
|
|
23
|
-
- **
|
|
23
|
+
- **Weaver** (project) — subscribes to graph-relevant domain events and projects them into the graph database; carried on the KB record (`kb.weaver`) and rebuilt from the event log at startup (`rebuildAll()`)
|
|
24
24
|
- **Smelter** (embed) — standalone embedding pipeline run via `@semiont/make-meaning/smelter-main` (not started by `startMakeMeaning`); subscribes to domain events, reads content from the KB working tree via `WorkerContentTransport`, chunks text, embeds via `@semiont/vectors`, and indexes into the vector store (Qdrant). On startup it reconciles Qdrant against the KS catalog — re-embedding what's missing or stale (every upsert is stamped with the embedded bytes' checksum, so changed content is detected) and deleting orphans — so a wiped Qdrant volume, or events missed while the worker was down, recover by restarting the smelter
|
|
25
25
|
|
|
26
26
|
(The third derived read model — the materialized views — is not pipeline-maintained: the EventStore's `ViewManager` materializes views synchronously inside `appendEvent()` for a read-your-writes guarantee.)
|
|
@@ -60,7 +60,7 @@ await makeMeaning.stop();
|
|
|
60
60
|
|
|
61
61
|
This single call initializes:
|
|
62
62
|
- **KnowledgeSystem** — groups the Knowledge Base and its actors
|
|
63
|
-
- **KnowledgeBase** — groups EventStore, ViewStorage, WorkingTreeStore, GraphDatabase,
|
|
63
|
+
- **KnowledgeBase** — groups EventStore, ViewStorage, WorkingTreeStore, GraphDatabase, Weaver, and optionally VectorStore
|
|
64
64
|
- **Stower** — subscribes to write commands on EventBus
|
|
65
65
|
- **Browser** — subscribes to all KB read queries and directory browse requests on EventBus
|
|
66
66
|
- **Gatherer** — subscribes to annotation and resource gather requests on EventBus; searches vectors for semantically similar passages
|
|
@@ -113,7 +113,7 @@ graph TB
|
|
|
113
113
|
GATHERER["Gatherer<br/>(context assembly)"]
|
|
114
114
|
MATCHER["Matcher<br/>(search/link)"]
|
|
115
115
|
SMELTER["Smelter<br/>(embed pipeline, standalone process)"]
|
|
116
|
-
|
|
116
|
+
WEAVER["Weaver<br/>(graph pipeline)"]
|
|
117
117
|
CTM["CloneTokenManager<br/>(clone)"]
|
|
118
118
|
KB["Knowledge Base"]
|
|
119
119
|
VECTORS["Vector Store<br/>(Qdrant)"]
|
|
@@ -125,20 +125,20 @@ graph TB
|
|
|
125
125
|
MATCHER -->|search| VECTORS
|
|
126
126
|
SMELTER -->|embed & index| VECTORS
|
|
127
127
|
SMELTER -->|read| KB
|
|
128
|
-
|
|
128
|
+
WEAVER -->|project| KB
|
|
129
129
|
CTM -->|query| KB
|
|
130
130
|
end
|
|
131
131
|
|
|
132
132
|
BUS -->|"yield:create, yield:update, yield:mv<br/>mark:create, mark:delete, mark:update-body<br/>frame:add-entity-type, frame:add-tag-schema<br/>mark:archive, mark:unarchive, mark:update-entity-types<br/>job:start, job:complete, job:fail"| STOWER
|
|
133
|
-
BUS -->|"browse:resource-requested, browse:resources-requested<br/>browse:annotations-requested, browse:annotation-requested<br/>browse:events-requested, browse:annotation-history-requested<br/>browse:referenced-by-requested, browse:entity-types-requested<br/>browse:tag-schemas-requested, browse:directory-requested"| BROWSER
|
|
133
|
+
BUS -->|"browse:resource-requested, browse:resources-requested<br/>browse:annotations-requested, browse:annotation-requested<br/>browse:events-requested, browse:annotation-history-requested<br/>browse:referenced-by-requested, browse:entity-types-requested<br/>browse:tag-schemas-requested, browse:agents-requested<br/>browse:directory-requested"| BROWSER
|
|
134
134
|
BUS -->|"gather:requested<br/>gather:resource-requested"| GATHERER
|
|
135
135
|
BUS -->|"match:search-requested"| MATCHER
|
|
136
136
|
BUS -->|"domain events:<br/>yield:created, yield:updated<br/>yield:representation-added<br/>mark:added, mark:removed, mark:archived"| SMELTER
|
|
137
|
-
BUS -->|"graph-relevant<br/>domain events"|
|
|
137
|
+
BUS -->|"graph-relevant<br/>domain events"| WEAVER
|
|
138
138
|
BUS -->|"yield:clone-token-requested<br/>yield:clone-resource-requested<br/>yield:clone-create"| CTM
|
|
139
139
|
|
|
140
140
|
STOWER -->|"yield:create-ok, yield:update-ok, yield:move-ok<br/>mark:delete-ok, *-failed replies<br/>(domain events are republished onto the bus<br/>by the EventStore: yield:created, mark:added, ...)"| BUS
|
|
141
|
-
BROWSER -->|"browse:resource-result, browse:resources-result<br/>browse:annotations-result, browse:annotation-result<br/>browse:events-result, browse:annotation-history-result<br/>browse:referenced-by-result, browse:entity-types-result<br/>browse:tag-schemas-result, browse:directory-result"| BUS
|
|
141
|
+
BROWSER -->|"browse:resource-result, browse:resources-result<br/>browse:annotations-result, browse:annotation-result<br/>browse:events-result, browse:annotation-history-result<br/>browse:referenced-by-result, browse:entity-types-result<br/>browse:tag-schemas-result, browse:agents-result<br/>browse:directory-result"| BUS
|
|
142
142
|
GATHERER -->|"gather:complete, gather:failed<br/>gather:resource-complete, gather:resource-failed"| BUS
|
|
143
143
|
MATCHER -->|"match:search-results, match:search-failed"| BUS
|
|
144
144
|
CTM -->|"yield:clone-token-generated<br/>yield:clone-resource-result<br/>yield:clone-created"| BUS
|
|
@@ -150,7 +150,7 @@ graph TB
|
|
|
150
150
|
|
|
151
151
|
class BUS bus
|
|
152
152
|
classDef vectorstore fill:#6b8e9d,stroke:#4a6a7a,stroke-width:2px,color:#fff
|
|
153
|
-
class STOWER,BROWSER,GATHERER,MATCHER,SMELTER,
|
|
153
|
+
class STOWER,BROWSER,GATHERER,MATCHER,SMELTER,WEAVER,CTM actor
|
|
154
154
|
class KB kb
|
|
155
155
|
class VECTORS vectorstore
|
|
156
156
|
class Routes,Workers,EBC caller
|
|
@@ -168,7 +168,7 @@ The **Knowledge Base** is an inert store — it has no intelligence, no goals, n
|
|
|
168
168
|
| **Materialized Views** | `ViewStorage` | Denormalized projections for fast reads (materialized synchronously on append) |
|
|
169
169
|
| **Content Store** | `WorkingTreeStore` | Working-tree files addressed by URI |
|
|
170
170
|
| **Graph** | `GraphDatabase` | Eventually consistent relationship projection |
|
|
171
|
-
| **
|
|
171
|
+
| **Weaver** | `Weaver` | Event-to-graph projection pipeline (one of the two pipeline actors; carried on the KB record because `createKnowledgeBase()` constructs and starts it) |
|
|
172
172
|
| **Vectors** *(optional)* | `VectorStore` | Semantic vector index (Qdrant + memory) via `@semiont/vectors` |
|
|
173
173
|
|
|
174
174
|
Its sibling pipeline, the Smelter (event-to-vector projection), is **not** a KB member — it runs as a standalone process via `@semiont/make-meaning/smelter-main`.
|
|
@@ -177,7 +177,7 @@ Its sibling pipeline, the Smelter (event-to-vector projection), is **not** a KB
|
|
|
177
177
|
import { createKnowledgeBase } from '@semiont/make-meaning';
|
|
178
178
|
|
|
179
179
|
const kb = await createKnowledgeBase(eventStore, project, graphDb, eventBus, logger, options);
|
|
180
|
-
// kb.eventStore, kb.views, kb.content, kb.graph, kb.
|
|
180
|
+
// kb.eventStore, kb.views, kb.content, kb.graph, kb.weaver
|
|
181
181
|
// kb.vectors (optional), kb.projectionsDir
|
|
182
182
|
```
|
|
183
183
|
|
|
@@ -221,7 +221,7 @@ This pattern (functional core, imperative shell) is shared with `@semiont/event-
|
|
|
221
221
|
### Knowledge Base
|
|
222
222
|
|
|
223
223
|
- `createKnowledgeBase(eventStore, project, graphDb, eventBus, logger, options?)` — Async factory function
|
|
224
|
-
- `KnowledgeBase` — Interface grouping the KB stores (`eventStore`, `views`, `content`, `graph`, optional `vectors`) plus the `
|
|
224
|
+
- `KnowledgeBase` — Interface grouping the KB stores (`eventStore`, `views`, `content`, `graph`, optional `vectors`) plus the `weaver` pipeline
|
|
225
225
|
|
|
226
226
|
### Actors
|
|
227
227
|
|
|
@@ -232,7 +232,7 @@ This pattern (functional core, imperative shell) is shared with `@semiont/event-
|
|
|
232
232
|
- `CloneTokenManager` — Clone token lifecycle actor (yield domain)
|
|
233
233
|
- `Smelter` / `createSmelterActorStateUnit` / `WorkerContentTransport` — the embedding pipeline, its domain-event fan-in, and the worker-side content transport; wired together by the standalone `@semiont/make-meaning/smelter-main` entry point, and exported for callers that run the pipeline on their own `WorkerBus`
|
|
234
234
|
|
|
235
|
-
The
|
|
235
|
+
The Weaver is not exported — `createKnowledgeBase()` constructs it internally and exposes it as `kb.weaver`.
|
|
236
236
|
|
|
237
237
|
### Operations
|
|
238
238
|
|
package/dist/index.d.ts
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
import { JobQueue } from '@semiont/jobs';
|
|
2
2
|
import { SemiontProject } from '@semiont/core/node';
|
|
3
|
-
import { GraphServiceConfig, VectorsServiceConfig, EmbeddingServiceConfig, EventBus, Logger,
|
|
3
|
+
import { GraphServiceConfig, VectorsServiceConfig, EmbeddingServiceConfig, StateUnit, EventBus, Logger, ResourceId, ResourceDescriptor, AnnotationId, components, ITransport, BaseUrl, ConnectionState, SemiontError, UserDID, EventMap, IContentTransport, PutBinaryRequest, PutBinaryOptions, AccessToken, BusRequestPrimitive, StoredEvent, Annotation, UserId, GatheredContext, ResourceAnnotations, AnnotationCategory, GraphPath, GraphConnection } from '@semiont/core';
|
|
4
4
|
import { EventStore, ViewStorage } from '@semiont/event-sourcing';
|
|
5
5
|
import { WorkingTreeStore } from '@semiont/content';
|
|
6
6
|
import { GraphDatabase } from '@semiont/graph';
|
|
@@ -49,6 +49,16 @@ interface MakeMeaningConfig {
|
|
|
49
49
|
vectors?: VectorsServiceConfig;
|
|
50
50
|
embedding?: EmbeddingServiceConfig;
|
|
51
51
|
};
|
|
52
|
+
/**
|
|
53
|
+
* The KB's canonical identity domain — the SAME value `/api/tokens/agent`
|
|
54
|
+
* mints agent DIDs from (backend `site.domain`). The agent roster consumes
|
|
55
|
+
* it verbatim, so directory DIDs and work-stamped `generator` DIDs are
|
|
56
|
+
* equal by construction; it is never derived from service topology
|
|
57
|
+
* (.plans/bugs/agent-did-host-skew.md).
|
|
58
|
+
*/
|
|
59
|
+
site?: {
|
|
60
|
+
domain: string;
|
|
61
|
+
};
|
|
52
62
|
/** Per-actor inference config */
|
|
53
63
|
actors?: ActorInferenceConfig;
|
|
54
64
|
/** Per-worker-type inference config */
|
|
@@ -56,104 +66,79 @@ interface MakeMeaningConfig {
|
|
|
56
66
|
}
|
|
57
67
|
|
|
58
68
|
/**
|
|
59
|
-
*
|
|
60
|
-
*
|
|
61
|
-
*
|
|
62
|
-
*
|
|
63
|
-
*
|
|
64
|
-
*
|
|
65
|
-
*
|
|
66
|
-
*
|
|
67
|
-
*
|
|
68
|
-
*
|
|
69
|
-
*
|
|
70
|
-
*
|
|
71
|
-
*
|
|
72
|
-
*
|
|
73
|
-
*
|
|
74
|
-
*
|
|
75
|
-
*
|
|
76
|
-
*
|
|
77
|
-
*
|
|
78
|
-
*
|
|
79
|
-
* `groupBy(resourceId) + concatMap(...)` is the stream-consumer flavor of
|
|
80
|
-
* per-resource serialization — the same invariant enforced by `Smelter`,
|
|
81
|
-
* `Gatherer`, and (in a different shape) `ViewManager`. See
|
|
82
|
-
* `packages/core/src/serialize-per-key.ts` for the shared primitive used
|
|
83
|
-
* by RPC-style services.
|
|
69
|
+
* WeaveProgress — backend-local fold of `weave:applied` signals
|
|
70
|
+
* (GRAPH-PROJECTION-SYNC P2, D2 = push).
|
|
71
|
+
*
|
|
72
|
+
* The Weaver emits `weave:applied` after applying an event (or a batch's
|
|
73
|
+
* last event) for a resource. This unit folds those signals into a
|
|
74
|
+
* per-resource applied-sequence map and exposes `whenApplied` — the
|
|
75
|
+
* applied-offset barrier: an event-driven await that resolves the moment
|
|
76
|
+
* the graph projection reaches parity with a known sequence (typically the
|
|
77
|
+
* view's `lastSequence`), and rejects with `WeaveProgressTimeout` on the
|
|
78
|
+
* bounded timeout so callers can fall back to the bounded-poll floor.
|
|
79
|
+
*
|
|
80
|
+
* Deliberately transport-blind: it subscribes to the channel, not to the
|
|
81
|
+
* Weaver. In-process the signal rides the core EventBus; after
|
|
82
|
+
* WEAVER-ISOLATION the same channel arrives through the bus gateway and
|
|
83
|
+
* this unit does not change.
|
|
84
|
+
*
|
|
85
|
+
* The map is ephemeral by design — on backend restart it rebuilds lazily
|
|
86
|
+
* from live signals. That loses nothing: a waiter only ever waits for an
|
|
87
|
+
* apply that has not happened yet, and those signals are still to come.
|
|
84
88
|
*/
|
|
85
89
|
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
private coreEventBus;
|
|
90
|
-
private static readonly GRAPH_RELEVANT_EVENTS;
|
|
91
|
-
private static readonly BURST_WINDOW_MS;
|
|
92
|
-
private static readonly MAX_BATCH_SIZE;
|
|
93
|
-
private static readonly IDLE_TIMEOUT_MS;
|
|
94
|
-
private _globalSubscriptions;
|
|
95
|
-
private eventSubject;
|
|
96
|
-
private pipelineSubscription;
|
|
97
|
-
private lastProcessed;
|
|
98
|
-
private readonly logger;
|
|
99
|
-
constructor(eventStore: EventStore, graphDb: GraphDatabase, coreEventBus: EventBus, logger: Logger);
|
|
100
|
-
initialize(): Promise<void>;
|
|
101
|
-
/**
|
|
102
|
-
* Subscribe globally to ALL events, pre-filter to graph-relevant types,
|
|
103
|
-
* and wire through the RxJS burst-buffered pipeline.
|
|
104
|
-
*/
|
|
105
|
-
private subscribeToGlobalEvents;
|
|
106
|
-
/**
|
|
107
|
-
* Wrap applyEventToGraph in try/catch so one failed event doesn't kill the pipeline.
|
|
108
|
-
*/
|
|
109
|
-
private safeApplyEvent;
|
|
110
|
-
private ensureInitialized;
|
|
111
|
-
/**
|
|
112
|
-
* Stop the consumer, flush remaining buffered events, and unsubscribe.
|
|
113
|
-
*/
|
|
114
|
-
stop(): Promise<void>;
|
|
115
|
-
/**
|
|
116
|
-
* Process a batch of events for the same resource.
|
|
117
|
-
* Partitions into consecutive same-type runs for batch optimization.
|
|
118
|
-
*/
|
|
119
|
-
private processBatch;
|
|
120
|
-
/**
|
|
121
|
-
* Batch-optimized processing for consecutive events of the same type.
|
|
122
|
-
* Uses batch graph methods where available, falls back to sequential.
|
|
123
|
-
*/
|
|
124
|
-
private applyBatchByType;
|
|
125
|
-
/**
|
|
126
|
-
* Build a ResourceDescriptor from a resource.created event.
|
|
127
|
-
* Extracted for reuse by both applyEventToGraph and applyBatchByType.
|
|
128
|
-
*/
|
|
129
|
-
private buildResourceDescriptor;
|
|
130
|
-
/**
|
|
131
|
-
* Apply a single event to GraphDB.
|
|
132
|
-
*/
|
|
133
|
-
protected applyEventToGraph(storedEvent: StoredEvent): Promise<void>;
|
|
134
|
-
/**
|
|
135
|
-
* Rebuild entire resource from events.
|
|
136
|
-
* Bypasses the live pipeline — reads directly from event store.
|
|
137
|
-
*/
|
|
138
|
-
rebuildResource(resourceId: ResourceId): Promise<void>;
|
|
139
|
-
/**
|
|
140
|
-
* Rebuild entire GraphDB from all events.
|
|
141
|
-
* Uses two-pass approach to ensure all resources exist before creating REFERENCES edges.
|
|
142
|
-
* Bypasses the live pipeline — reads directly from event store.
|
|
143
|
-
*/
|
|
144
|
-
rebuildAll(): Promise<void>;
|
|
90
|
+
interface WeaveProgress extends StateUnit {
|
|
91
|
+
/** Highest applied sequence seen for a resource, if any signal arrived. */
|
|
92
|
+
appliedUpTo(resourceId: string): number | undefined;
|
|
145
93
|
/**
|
|
146
|
-
*
|
|
94
|
+
* Resolve when the Weaver has applied at least `sequenceNumber` for
|
|
95
|
+
* `resourceId` — immediately if the fold already covers it. Rejects with
|
|
96
|
+
* `WeaveProgressTimeout` after `timeoutMs`. After dispose it resolves
|
|
97
|
+
* immediately (inert): barrier callers degrade to their poll floor.
|
|
147
98
|
*/
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
99
|
+
whenApplied(resourceId: string, sequenceNumber: number, timeoutMs: number): Promise<void>;
|
|
100
|
+
}
|
|
101
|
+
|
|
102
|
+
/**
|
|
103
|
+
* SmeltProgress — backend-local fold of `smelt:settled` signals
|
|
104
|
+
* (SMELTER-INDEX-SYNC P1, D1 = push barrier).
|
|
105
|
+
*
|
|
106
|
+
* The Smelter emits `smelt:settled` after deciding a resource's content:
|
|
107
|
+
* `indexed` (embedded + upserted) or `skipped` (media gate, empty text) —
|
|
108
|
+
* keyed by the checksum of the bytes it inspected, and NEVER on transient
|
|
109
|
+
* failures (an error is not a decision; SMELTER-INDEX-SYNC A2). This unit
|
|
110
|
+
* folds those signals per resource and exposes `whenSettled` — the
|
|
111
|
+
* read-your-writes barrier: an event-driven await that resolves the moment
|
|
112
|
+
* the vector projection has settled the exact content generation the caller
|
|
113
|
+
* holds (the view's checksum), and rejects with `SmeltProgressTimeout` on
|
|
114
|
+
* the bounded timeout so callers degrade observably (L4 breadcrumb).
|
|
115
|
+
*
|
|
116
|
+
* Deliberately transport-blind: it subscribes to the channel, not to the
|
|
117
|
+
* Smelter. The signal arrives through the bus gateway from the standalone
|
|
118
|
+
* worker; an in-process Smelter would ride the core EventBus and this unit
|
|
119
|
+
* would not change (the WeaveProgress precedent).
|
|
120
|
+
*
|
|
121
|
+
* The fold is ephemeral by design — on backend restart it rebuilds lazily
|
|
122
|
+
* from live signals. Barrier callers probe the vector store first
|
|
123
|
+
* (SMELTER-INDEX-SYNC A3), so a cold fold only costs waits for resources
|
|
124
|
+
* whose settlement genuinely hasn't been observed yet.
|
|
125
|
+
*/
|
|
126
|
+
|
|
127
|
+
type SmeltOutcome = 'indexed' | 'skipped';
|
|
128
|
+
interface SmeltProgress extends StateUnit {
|
|
129
|
+
/** The latest settlement seen for a resource, if any signal arrived. */
|
|
130
|
+
settledAt(resourceId: string): {
|
|
131
|
+
contentChecksum: string;
|
|
132
|
+
outcome: SmeltOutcome;
|
|
133
|
+
} | undefined;
|
|
153
134
|
/**
|
|
154
|
-
*
|
|
135
|
+
* Resolve with the Smelter's decision once it has settled `resourceId` at
|
|
136
|
+
* exactly `contentChecksum` — immediately if the fold already holds it.
|
|
137
|
+
* Rejects with `SmeltProgressTimeout` after `timeoutMs`. After dispose it
|
|
138
|
+
* resolves `'inert'`: shutdown must not throw through a gather in flight,
|
|
139
|
+
* and callers treat inert as breadcrumb-less degrade.
|
|
155
140
|
*/
|
|
156
|
-
|
|
141
|
+
whenSettled(resourceId: string, contentChecksum: string, timeoutMs: number): Promise<SmeltOutcome | 'inert'>;
|
|
157
142
|
}
|
|
158
143
|
|
|
159
144
|
/**
|
|
@@ -166,7 +151,10 @@ declare class GraphDBConsumer {
|
|
|
166
151
|
* - Materialized Views (fast single-doc queries) — via ViewStorage
|
|
167
152
|
* - Content Store (working-tree files, URI-addressed) — via WorkingTreeStore
|
|
168
153
|
* - Graph (eventually consistent relationship projection) — via GraphDatabase
|
|
169
|
-
* -
|
|
154
|
+
* - WeaveProgress (weave:applied fold — the graph-projection barrier; the
|
|
155
|
+
* Weaver itself runs standalone via @semiont/make-meaning/weaver-main)
|
|
156
|
+
* - SmeltProgress (smelt:settled fold — the vector-projection barrier;
|
|
157
|
+
* SMELTER-INDEX-SYNC, same standalone-actor arrangement as the Weaver)
|
|
170
158
|
* - Vectors (semantic search) — via VectorStore (optional, read-only)
|
|
171
159
|
*
|
|
172
160
|
* The Smelter (event-to-vector projection) runs as an external actor
|
|
@@ -179,7 +167,8 @@ interface KnowledgeBase {
|
|
|
179
167
|
views: ViewStorage;
|
|
180
168
|
content: WorkingTreeStore;
|
|
181
169
|
graph: GraphDatabase;
|
|
182
|
-
|
|
170
|
+
weaveProgress: WeaveProgress;
|
|
171
|
+
smeltProgress: SmeltProgress;
|
|
183
172
|
vectors?: VectorStore;
|
|
184
173
|
projectionsDir: string;
|
|
185
174
|
}
|
|
@@ -235,9 +224,10 @@ interface CreateResourceResult {
|
|
|
235
224
|
declare class Stower {
|
|
236
225
|
private kb;
|
|
237
226
|
private eventBus;
|
|
227
|
+
private project;
|
|
238
228
|
private subscription;
|
|
239
229
|
private readonly logger;
|
|
240
|
-
constructor(kb: KnowledgeBase, eventBus: EventBus, logger: Logger);
|
|
230
|
+
constructor(kb: KnowledgeBase, eventBus: EventBus, project: SemiontProject, logger: Logger);
|
|
241
231
|
initialize(): Promise<void>;
|
|
242
232
|
private handleYieldCreate;
|
|
243
233
|
private handleYieldUpdate;
|
|
@@ -277,7 +267,7 @@ declare class Stower {
|
|
|
277
267
|
*
|
|
278
268
|
* `groupBy(resourceId) + concatMap(...)` is the stream-consumer flavor of
|
|
279
269
|
* per-resource serialization — the same invariant enforced by `Smelter`,
|
|
280
|
-
* `
|
|
270
|
+
* `Weaver`, and (in a different shape) `ViewManager`. See
|
|
281
271
|
* `packages/core/src/serialize-per-key.ts` for the shared primitive used
|
|
282
272
|
* by RPC-style services.
|
|
283
273
|
*/
|
|
@@ -374,6 +364,8 @@ declare class Matcher {
|
|
|
374
364
|
* - browse:referenced-by-requested — find annotations in the KB graph that reference a resource
|
|
375
365
|
* - browse:entity-types-requested — list entity types from the project projection
|
|
376
366
|
* - browse:tag-schemas-requested — list tag schemas from the project projection
|
|
367
|
+
* - browse:agents-requested — the collaborator directory: the KB's declared software
|
|
368
|
+
* agents, derived from the workers + actors inference config (COLLABORATOR-DIRECTORY)
|
|
377
369
|
* - browse:directory-requested — list a project directory, merging fs + ViewStorage
|
|
378
370
|
*/
|
|
379
371
|
|
|
@@ -382,9 +374,10 @@ declare class Browser {
|
|
|
382
374
|
private kb;
|
|
383
375
|
private eventBus;
|
|
384
376
|
private project;
|
|
377
|
+
private config;
|
|
385
378
|
private subscriptions;
|
|
386
379
|
private readonly logger;
|
|
387
|
-
constructor(views: ViewStorage, kb: KnowledgeBase, eventBus: EventBus, project: SemiontProject, logger: Logger);
|
|
380
|
+
constructor(views: ViewStorage, kb: KnowledgeBase, eventBus: EventBus, project: SemiontProject, config: MakeMeaningConfig, logger: Logger);
|
|
388
381
|
initialize(): Promise<void>;
|
|
389
382
|
private handleBrowseResource;
|
|
390
383
|
private handleBrowseResources;
|
|
@@ -395,6 +388,7 @@ declare class Browser {
|
|
|
395
388
|
private handleReferencedBy;
|
|
396
389
|
private handleEntityTypes;
|
|
397
390
|
private handleTagSchemas;
|
|
391
|
+
private handleBrowseAgents;
|
|
398
392
|
private handleBrowseDirectory;
|
|
399
393
|
stop(): Promise<void>;
|
|
400
394
|
}
|
|
@@ -442,8 +436,10 @@ declare class CloneTokenManager {
|
|
|
442
436
|
* - cloneTokenManager: token actor — manages resource clone tokens
|
|
443
437
|
*
|
|
444
438
|
* These are the five access actors. Two projection-pipeline actors complete
|
|
445
|
-
* the seven: the
|
|
446
|
-
*
|
|
439
|
+
* the seven, and BOTH run standalone (D4: the projections are part of their
|
|
440
|
+
* stores' stacks, not of the embedding process): the Weaver (weaver-main →
|
|
441
|
+
* graph) and the Smelter (smelter-main → vectors). The backend keeps only
|
|
442
|
+
* the Weaver's `weave:applied` fold (kb.weaveProgress).
|
|
447
443
|
*
|
|
448
444
|
* EventBus, JobQueue, and workers are peers to KnowledgeSystem, not members.
|
|
449
445
|
*/
|
|
@@ -596,6 +592,17 @@ declare class LocalContentTransport implements IContentTransport {
|
|
|
596
592
|
dispose(): void;
|
|
597
593
|
}
|
|
598
594
|
|
|
595
|
+
/**
|
|
596
|
+
* Adapt a raw in-process `EventBus` to the `BusRequestPrimitive` that
|
|
597
|
+
* `busRequest` consumes. Lets backend-internal callers (bootstrap, event
|
|
598
|
+
* replay, linked-data import) use the same confirmed request/reply path as the
|
|
599
|
+
* SDK — `busRequest(asBusRequestPrimitive(eventBus), …)` — instead of
|
|
600
|
+
* hand-rolled `race(domain-event, *-failed, timeout)` blocks. The reply is
|
|
601
|
+
* matched by `correlationId`, so concurrent in-process writes can't cross-match
|
|
602
|
+
* (the latent bug in the old domain-event `race`).
|
|
603
|
+
*/
|
|
604
|
+
declare function asBusRequestPrimitive(eventBus: EventBus): BusRequestPrimitive;
|
|
605
|
+
|
|
599
606
|
/**
|
|
600
607
|
* Handles `mark:create-request` — the bus command for creating an annotation.
|
|
601
608
|
*
|
|
@@ -698,7 +705,7 @@ declare function readEntityTypesProjection(project: SemiontProject): Promise<str
|
|
|
698
705
|
/**
|
|
699
706
|
* SmelterActorStateUnit — domain-event fan-in for the Smelter worker.
|
|
700
707
|
*
|
|
701
|
-
* Subscribes to the
|
|
708
|
+
* Subscribes to the nine smelter-relevant channels on a shared bus and
|
|
702
709
|
* exposes them as a single typed `events$` stream. Transport-neutral —
|
|
703
710
|
* the caller passes a `WorkerBus` (HTTP `ActorStateUnit` today, an in-process
|
|
704
711
|
* bus shim if/when one exists). The state unit does not own the bus and does
|
|
@@ -739,7 +746,7 @@ declare function createSmelterActorStateUnit(options: SmelterActorStateUnitOptio
|
|
|
739
746
|
* Smelter processes events strictly in order per resourceId via
|
|
740
747
|
* `groupBy(resourceId) + concatMap(...)`. This is the stream-consumer
|
|
741
748
|
* flavor of per-resource serialization — the same invariant enforced by
|
|
742
|
-
* `
|
|
749
|
+
* `Weaver`, `Gatherer`, and (in a different shape) `ViewManager`.
|
|
743
750
|
* See `packages/core/src/serialize-per-key.ts` for the shared primitive
|
|
744
751
|
* used by RPC-style services.
|
|
745
752
|
*
|
|
@@ -753,15 +760,18 @@ declare function createSmelterActorStateUnit(options: SmelterActorStateUnitOptio
|
|
|
753
760
|
* Qdrant is an ephemeral projection of the event log. `reconcile()` brings
|
|
754
761
|
* it back in sync at startup — after a wiped volume, or after events missed
|
|
755
762
|
* while the worker was down. It is a planner: it diffs the store against the
|
|
756
|
-
* catalog (over the `browse:*` RPC channels) —
|
|
757
|
-
*
|
|
763
|
+
* catalog (over the `browse:*` RPC channels) — membership, content freshness
|
|
764
|
+
* (via the checksum stamped onto every resource upsert), and tag-stamp
|
|
765
|
+
* freshness (payload-only restamps; tag edits change no bytes) — and
|
|
758
766
|
* enqueues `smelt:*` work items through the same mailbox as live events, so
|
|
759
|
-
* per-resource ordering holds across the two paths (axioms S1/S2/S11/S12
|
|
760
|
-
* `.plans/SMELTER-AXIOMS.md`).
|
|
767
|
+
* per-resource ordering holds across the two paths (axioms S1/S2/S11/S12/S13
|
|
768
|
+
* in `.plans/SMELTER-AXIOMS.md`).
|
|
761
769
|
*/
|
|
762
770
|
|
|
763
771
|
interface ReconcileSummary {
|
|
764
772
|
resourcesEmbedded: number;
|
|
773
|
+
/** Tag-only drift healed by payload restamps — never embedding calls (S13). */
|
|
774
|
+
resourcesRestamped: number;
|
|
765
775
|
resourceVectorsDeleted: number;
|
|
766
776
|
annotationsEmbedded: number;
|
|
767
777
|
annotationVectorsDeleted: number;
|
|
@@ -794,7 +804,7 @@ interface SmelterTiming {
|
|
|
794
804
|
* lanes and batch paths serve both kinds of input.
|
|
795
805
|
*/
|
|
796
806
|
interface SmelterWorkItem {
|
|
797
|
-
type: 'smelt:embed' | 'smelt:purge' | 'smelt:embed-annotation' | 'smelt:purge-annotation';
|
|
807
|
+
type: 'smelt:embed' | 'smelt:restamp' | 'smelt:purge' | 'smelt:embed-annotation' | 'smelt:purge-annotation';
|
|
798
808
|
resourceId: string;
|
|
799
809
|
payload: Record<string, unknown>;
|
|
800
810
|
}
|
|
@@ -837,6 +847,15 @@ declare class Smelter {
|
|
|
837
847
|
/** Returns true if the input was processed without error. */
|
|
838
848
|
private safeProcessEvent;
|
|
839
849
|
private processEvent;
|
|
850
|
+
/**
|
|
851
|
+
* Payload-only stamp refresh: re-read the resource's CURRENT entity types
|
|
852
|
+
* (one code path — `resolveEntityTypes` — so any prior drift self-corrects
|
|
853
|
+
* on first touch) and rewrite the stamp on its existing points. Never calls
|
|
854
|
+
* the embedding provider (S13): content is unchanged by definition on every
|
|
855
|
+
* path that lands here. A resource with no points is a no-op — the stamp
|
|
856
|
+
* rides the next embed.
|
|
857
|
+
*/
|
|
858
|
+
private restampResource;
|
|
840
859
|
private handleResourcePurge;
|
|
841
860
|
/**
|
|
842
861
|
* Resolve a resource's embeddable text: bytes via the content transport,
|
|
@@ -847,6 +866,13 @@ declare class Smelter {
|
|
|
847
866
|
* or is empty — callers skip it.
|
|
848
867
|
*/
|
|
849
868
|
private fetchEmbeddableText;
|
|
869
|
+
/**
|
|
870
|
+
* The Smelter's single outbound signal (SMELTER-AXIOMS D3 as amended by
|
|
871
|
+
* SMELTER-INDEX-SYNC): a per-resource decision report for the barrier
|
|
872
|
+
* fold. Best-effort — waiters degrade to their bounded timeout; a signal
|
|
873
|
+
* failure must never fail the embed.
|
|
874
|
+
*/
|
|
875
|
+
private emitSettled;
|
|
850
876
|
/**
|
|
851
877
|
* Read a resource's current entity types from the materialized view — the
|
|
852
878
|
* authoritative source, updated before the EventBus fires to consumers — so
|
|
@@ -859,7 +885,15 @@ declare class Smelter {
|
|
|
859
885
|
private resolveEntityTypes;
|
|
860
886
|
private embedResource;
|
|
861
887
|
private handleResourceArchived;
|
|
888
|
+
/**
|
|
889
|
+
* Restore what `handleResourceArchived` deleted, from CURRENT state: the
|
|
890
|
+
* resource's vectors (media-gated, full-replace) and its current exact-text
|
|
891
|
+
* annotations — the same catalog read `reconcile()` uses, so the live path
|
|
892
|
+
* and a restart agree (bugs/smelter-misses-unarchive.md).
|
|
893
|
+
*/
|
|
894
|
+
private handleResourceUnarchived;
|
|
862
895
|
private handleAnnotationAdded;
|
|
896
|
+
private indexAnnotation;
|
|
863
897
|
private handleAnnotationRemoved;
|
|
864
898
|
/**
|
|
865
899
|
* Batch-embed chunks from multiple yield:created events in a single
|
|
@@ -1023,6 +1057,8 @@ interface ReplayStats {
|
|
|
1023
1057
|
|
|
1024
1058
|
interface BackupImporterOptions {
|
|
1025
1059
|
eventBus: EventBus;
|
|
1060
|
+
/** Replay appends historical events directly (facts, not commands). */
|
|
1061
|
+
eventStore: EventStore;
|
|
1026
1062
|
contentStore: WorkingTreeStore;
|
|
1027
1063
|
logger?: Logger;
|
|
1028
1064
|
}
|
|
@@ -1342,6 +1378,20 @@ declare class AnnotationContext {
|
|
|
1342
1378
|
|
|
1343
1379
|
type KnowledgeGraph = components['schemas']['KnowledgeGraph'];
|
|
1344
1380
|
declare class GraphContext {
|
|
1381
|
+
/**
|
|
1382
|
+
* Backoff schedule for the projection-lag grace in `buildKnowledgeGraph`
|
|
1383
|
+
* (GRAPH-PROJECTION-SYNC P1). Total wait is bounded at 375 ms — the Weaver
|
|
1384
|
+
* applies in tens of milliseconds when merely lagging; anything slower is
|
|
1385
|
+
* treated as a real miss.
|
|
1386
|
+
*/
|
|
1387
|
+
private static readonly PROJECTION_LAG_BACKOFF_MS;
|
|
1388
|
+
/**
|
|
1389
|
+
* Bounded wait for the applied-offset barrier (GRAPH-PROJECTION-SYNC P2).
|
|
1390
|
+
* The Weaver applies in tens of milliseconds when merely lagging; a
|
|
1391
|
+
* barrier that hasn't woken in 500 ms means signals have stalled and the
|
|
1392
|
+
* poll floor above owns the remainder.
|
|
1393
|
+
*/
|
|
1394
|
+
private static readonly PROJECTION_BARRIER_TIMEOUT_MS;
|
|
1345
1395
|
/**
|
|
1346
1396
|
* Get all resources referencing this resource (backlinks)
|
|
1347
1397
|
* Requires graph traversal - must use graph database
|
|
@@ -1402,7 +1452,7 @@ declare class LLMContext {
|
|
|
1402
1452
|
* Get comprehensive LLM context for a resource
|
|
1403
1453
|
* Includes: main resource, related resources, annotations, graph, content, summary, references
|
|
1404
1454
|
*/
|
|
1405
|
-
static getResourceContext(resourceId: ResourceId, options: LLMContextOptions, kb: KnowledgeBase, inferenceClient: InferenceClient): Promise<GatheredContext>;
|
|
1455
|
+
static getResourceContext(resourceId: ResourceId, options: LLMContextOptions, kb: KnowledgeBase, inferenceClient: InferenceClient, logger: Logger): Promise<GatheredContext>;
|
|
1406
1456
|
}
|
|
1407
1457
|
|
|
1408
1458
|
/**
|
|
@@ -1425,5 +1475,5 @@ declare function generateResourceSummary(resourceName: string, content: string,
|
|
|
1425
1475
|
*/
|
|
1426
1476
|
declare function generateReferenceSuggestions(referenceTitle: string, client: InferenceClient, entityType?: string, currentContent?: string): Promise<string[] | null>;
|
|
1427
1477
|
|
|
1428
|
-
export { AnnotationContext, AnnotationOperations, BACKUP_FORMAT, Browser, CloneTokenManager, FORMAT_VERSION, Gatherer$1 as Gatherer, GraphContext, LLMContext, LocalContentTransport, LocalTransport, Matcher, ResourceContext, ResourceOperations, Smelter, Stower, bootstrapEntityTypes, createKnowledgeBase, createSmelterActorStateUnit, exportBackup, exportLinkedData, generateReferenceSuggestions, generateResourceSummary, importBackup, importLinkedData, isBackupManifest, readEntityTypesProjection, registerAnnotationAssemblyHandler, registerAnnotationLookupHandlers, registerBindUpdateBodyHandler, registerBusHandlers, registerJobCommandHandlers, startMakeMeaning, stopKnowledgeSystem, validateManifestVersion };
|
|
1478
|
+
export { AnnotationContext, AnnotationOperations, BACKUP_FORMAT, Browser, CloneTokenManager, FORMAT_VERSION, Gatherer$1 as Gatherer, GraphContext, LLMContext, LocalContentTransport, LocalTransport, Matcher, ResourceContext, ResourceOperations, Smelter, Stower, asBusRequestPrimitive, bootstrapEntityTypes, createKnowledgeBase, createSmelterActorStateUnit, exportBackup, exportLinkedData, generateReferenceSuggestions, generateResourceSummary, importBackup, importLinkedData, isBackupManifest, readEntityTypesProjection, registerAnnotationAssemblyHandler, registerAnnotationLookupHandlers, registerBindUpdateBodyHandler, registerBusHandlers, registerJobCommandHandlers, startMakeMeaning, stopKnowledgeSystem, validateManifestVersion };
|
|
1429
1479
|
export type { BackupContentReader, BackupEventStoreReader, BackupExporterOptions, BackupImportResult, BackupImporterOptions, BackupManifestHeader, BackupStreamSummary, BuildContextOptions, ContentBlobResolver, CreateAnnotationResult, CreateResourceInput, CreateResourceResult, KnowledgeBase, KnowledgeSystem, LLMContextOptions, LinkedDataContentReader, LinkedDataExporterOptions, LinkedDataImportResult, LinkedDataImporterOptions, LinkedDataViewReader, ListResourcesFilters, LocalTransportConfig, MakeMeaningConfig, MakeMeaningService, ReconcileState, ReconcileSummary, ReplayStats, SmelterActorStateUnit, SmelterActorStateUnitOptions, SmelterEvent, SmelterInput, SmelterTiming, SmelterWorkItem, UpdateAnnotationBodyResult };
|