@semiont/make-meaning 0.2.30-build.59 → 0.2.30-build.61

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -8,42 +8,26 @@
8
8
 
9
9
  **Making meaning from resources through context assembly, pattern detection, and relationship reasoning.**
10
10
 
11
- This package transforms raw resources into meaningful, interconnected knowledge. It provides the core capabilities for:
11
+ This package transforms raw resources into meaningful, interconnected knowledge through:
12
12
 
13
13
  - **Context Assembly**: Gathering resource metadata, content, and annotations from distributed storage
14
14
  - **Pattern Detection**: AI-powered discovery of semantic patterns (comments, highlights, assessments, tags)
15
- - **Graph Reasoning**: Navigating relationships and connections between resources
15
+ - **Relationship Reasoning**: Navigating connections between resources through graph traversal
16
+ - **Job Workers**: Asynchronous processing of detection tasks with progress tracking
16
17
 
17
- ## Philosophy
18
-
19
- Resources don't exist in isolation. A document becomes meaningful when we understand its annotations, its relationships to other resources, and the patterns within its content. `@semiont/make-meaning` provides the infrastructure to assemble this context from event-sourced storage, detect semantic patterns using AI, and reason about resource relationships through graph traversal.
20
-
21
- This is the "applied meaning-making" layer - it sits between low-level AI primitives ([@semiont/inference](../inference/)) and high-level application orchestration ([apps/backend](../../apps/backend/)).
22
-
23
- ## Installation
18
+ ## Quick Start
24
19
 
25
20
  ```bash
26
21
  npm install @semiont/make-meaning
27
22
  ```
28
23
 
29
- ## Quick Start
30
-
31
24
  ### Assemble Resource Context
32
25
 
33
26
  ```typescript
34
27
  import { ResourceContext } from '@semiont/make-meaning';
35
- import type { EnvironmentConfig } from '@semiont/core';
36
28
 
37
- // Get resource metadata from event-sourced view storage
38
29
  const resource = await ResourceContext.getResourceMetadata(resourceId, config);
39
-
40
- // List all resources with optional filtering
41
- const resources = await ResourceContext.listResources(
42
- { createdAfter: '2024-01-01' },
43
- config
44
- );
45
-
46
- // Add content previews to resource descriptors
30
+ const resources = await ResourceContext.listResources({ createdAfter: '2024-01-01' }, config);
47
31
  const withContent = await ResourceContext.addContentPreviews(resources, config);
48
32
  ```
49
33
 
@@ -62,13 +46,6 @@ const context = await AnnotationContext.buildLLMContext(
62
46
  config,
63
47
  { contextLines: 5 }
64
48
  );
65
-
66
- // Generate AI summary of an annotation
67
- const summary = await AnnotationContext.generateAnnotationSummary(
68
- annotationId,
69
- resourceId,
70
- config
71
- );
72
49
  ```
73
50
 
74
51
  ### Detect Semantic Patterns
@@ -93,15 +70,6 @@ const highlights = await AnnotationDetection.detectHighlights(
93
70
  0.5
94
71
  );
95
72
 
96
- // Detect passages that merit assessment/evaluation
97
- const assessments = await AnnotationDetection.detectAssessments(
98
- resourceId,
99
- config,
100
- 'Evaluate clarity and technical accuracy',
101
- 'constructive',
102
- 0.6
103
- );
104
-
105
73
  // Detect and extract structured tags from text using ontology schemas
106
74
  const tags = await AnnotationDetection.detectTags(
107
75
  resourceId,
@@ -111,41 +79,6 @@ const tags = await AnnotationDetection.detectTags(
111
79
  );
112
80
  ```
113
81
 
114
- ### Structured Tagging with Ontology Schemas
115
-
116
- A powerful use case is **structured tagging** using tag schemas defined in [@semiont/ontology](../ontology/). For example, legal writing can be analyzed using the IRAC framework (Issue, Rule, Application, Conclusion):
117
-
118
- ```typescript
119
- import { AnnotationDetection } from '@semiont/make-meaning';
120
-
121
- // Analyze a legal brief using IRAC schema
122
- const categories = ['issue', 'rule', 'application', 'conclusion'];
123
-
124
- for (const category of categories) {
125
- const tags = await AnnotationDetection.detectTags(
126
- resourceId,
127
- config,
128
- 'irac', // Tag schema from @semiont/ontology
129
- category // Which category to detect
130
- );
131
-
132
- console.log(`Found ${tags.length} ${category} passages`);
133
- }
134
- ```
135
-
136
- **Why this matters:**
137
-
138
- When you tag multiple documents with the same schema (e.g., IRAC for legal briefs, IMRAD for scientific papers), you create a **structured semantic layer** across your corpus:
139
-
140
- - **Rich traversal**: Find all "issue" statements across 100 legal briefs
141
- - **Cross-document analysis**: Compare how different authors structure their "application" sections
142
- - **Context retrieval**: When reading one brief, see related "rule" passages from other cases
143
- - **Graph-based reasoning**: Trace argument patterns across your entire document collection
144
-
145
- This transforms a collection of unstructured documents into a queryable knowledge base organized by domain-specific rhetorical structures.
146
-
147
- See [@semiont/ontology](../ontology/) for available tag schemas and how to define custom schemas.
148
-
149
82
  ### Navigate Resource Relationships
150
83
 
151
84
  ```typescript
@@ -157,473 +90,124 @@ const backlinks = await GraphContext.getBacklinks(resourceId, config);
157
90
  // Find shortest path between two resources
158
91
  const paths = await GraphContext.findPath(fromResourceId, toResourceId, config, 3);
159
92
 
160
- // Get all connections for a resource
161
- const connections = await GraphContext.getResourceConnections(resourceId, config);
162
-
163
93
  // Full-text search across all resources
164
94
  const results = await GraphContext.searchResources('neural networks', config, 10);
165
95
  ```
166
96
 
167
- ## Architecture
168
-
169
- `@semiont/make-meaning` implements a **three-layer architecture**:
97
+ ### Use Job Workers
170
98
 
171
- ```
172
- ┌─────────────────────────────────────────────┐
173
- apps/backend │
174
- Job orchestration, progress tracking, │
175
- HTTP APIs, event emission │
176
- └─────────────────────────────────────────────┘
177
-
178
- ┌─────────────────────────────────────────────┐
179
- │ @semiont/make-meaning │
180
- │ Context assembly, pattern detection, │
181
- │ relationship reasoning │
182
- └─────────────────────────────────────────────┘
183
-
184
- ┌─────────────────────────────────────────────┐
185
- │ @semiont/inference │
186
- │ AI primitives: prompts, parsers, │
187
- │ generateText abstraction │
188
- └─────────────────────────────────────────────┘
189
- ```
190
-
191
- **Key principles:**
192
-
193
- - **Event-sourced context**: Resources and annotations are assembled from event streams via view storage
194
- - **Content-addressed storage**: Content retrieved using checksums, enabling deduplication and caching
195
- - **Graph-backed relationships**: @semiont/graph provides graph traversal for backlinks, paths, and connections
196
- - **Separation of concerns**: Detection logic (make-meaning) is separate from job orchestration (backend)
197
-
198
- See [MAKE-MEANING-PACKAGE.md](../../MAKE-MEANING-PACKAGE.md) for complete architecture documentation.
199
-
200
- ## API Reference
99
+ ```typescript
100
+ import {
101
+ ReferenceDetectionWorker,
102
+ HighlightDetectionWorker,
103
+ GenerationWorker,
104
+ } from '@semiont/make-meaning';
105
+ import { JobQueue } from '@semiont/jobs';
106
+ import { createEventStore } from '@semiont/event-sourcing';
201
107
 
202
- ### ResourceContext
108
+ // Create shared dependencies
109
+ const jobQueue = new JobQueue({ dataDir: './data' });
110
+ await jobQueue.initialize();
111
+ const eventStore = await createEventStore(config);
203
112
 
204
- Provides resource metadata and content assembly from event-sourced storage.
113
+ // Create workers with explicit dependencies
114
+ const referenceWorker = new ReferenceDetectionWorker(jobQueue, config, eventStore);
115
+ const highlightWorker = new HighlightDetectionWorker(jobQueue, config, eventStore);
116
+ const generationWorker = new GenerationWorker(jobQueue, config, eventStore);
205
117
 
206
- ```typescript
207
- class ResourceContext {
208
- /**
209
- * Get resource metadata from view storage
210
- * Implementation: packages/make-meaning/src/resource-context.ts:15-28
211
- */
212
- static async getResourceMetadata(
213
- resourceId: ResourceId,
214
- config: EnvironmentConfig
215
- ): Promise<ResourceDescriptor | null>
216
-
217
- /**
218
- * List resources with optional filtering
219
- * Implementation: packages/make-meaning/src/resource-context.ts:30-48
220
- */
221
- static async listResources(
222
- filters: ListResourcesFilters | undefined,
223
- config: EnvironmentConfig
224
- ): Promise<ResourceDescriptor[]>
225
-
226
- /**
227
- * Add content previews to resource descriptors
228
- * Implementation: packages/make-meaning/src/resource-context.ts:50-77
229
- */
230
- static async addContentPreviews(
231
- resources: ResourceDescriptor[],
232
- config: EnvironmentConfig
233
- ): Promise<Array<ResourceDescriptor & { content: string }>>
234
- }
118
+ // Start workers
119
+ await Promise.all([
120
+ referenceWorker.start(),
121
+ highlightWorker.start(),
122
+ generationWorker.start(),
123
+ ]);
235
124
  ```
236
125
 
237
- **Filters:**
238
- ```typescript
239
- interface ListResourcesFilters {
240
- createdAfter?: string;
241
- createdBefore?: string;
242
- mimeType?: string;
243
- limit?: number;
244
- }
245
- ```
126
+ ## Documentation
246
127
 
247
- ### AnnotationContext
128
+ - **[API Reference](./docs/api-reference.md)** - Complete API documentation for all classes and methods
129
+ - **[Job Workers](./docs/job-workers.md)** - Asynchronous task processing with progress tracking
130
+ - **[Architecture](./docs/architecture.md)** - System design and data flow
131
+ - **[Examples](./docs/examples.md)** - Common use cases and patterns
248
132
 
249
- Consolidated annotation operations including queries, context building, and AI summarization.
133
+ ## Philosophy
250
134
 
251
- ```typescript
252
- class AnnotationContext {
253
- /**
254
- * Build LLM context for an annotation (includes surrounding text)
255
- * Implementation: packages/make-meaning/src/annotation-context.ts:35-120
256
- */
257
- static async buildLLMContext(
258
- annotationUri: AnnotationUri,
259
- resourceId: ResourceId,
260
- config: EnvironmentConfig,
261
- options: BuildContextOptions
262
- ): Promise<AnnotationLLMContextResponse>
263
-
264
- /**
265
- * Get all annotations for a resource, organized by motivation
266
- * Implementation: packages/make-meaning/src/annotation-context.ts:122-172
267
- */
268
- static async getResourceAnnotations(
269
- resourceId: ResourceId,
270
- config: EnvironmentConfig
271
- ): Promise<ResourceAnnotations>
272
-
273
- /**
274
- * Get all annotations for a resource (flat list)
275
- * Implementation: packages/make-meaning/src/annotation-context.ts:174-187
276
- */
277
- static async getAllAnnotations(
278
- resourceId: ResourceId,
279
- config: EnvironmentConfig
280
- ): Promise<Annotation[]>
281
-
282
- /**
283
- * Get a specific annotation by ID
284
- * Implementation: packages/make-meaning/src/annotation-context.ts:189-202
285
- */
286
- static async getAnnotation(
287
- annotationId: AnnotationId,
288
- resourceId: ResourceId,
289
- config: EnvironmentConfig
290
- ): Promise<Annotation | null>
291
-
292
- /**
293
- * List annotations with optional filtering
294
- * Implementation: packages/make-meaning/src/annotation-context.ts:204-225
295
- */
296
- static async listAnnotations(
297
- filters: { resourceId?: ResourceId; type?: AnnotationCategory } | undefined,
298
- config: EnvironmentConfig
299
- ): Promise<Annotation[]>
300
-
301
- /**
302
- * Check if a resource exists in view storage
303
- * Implementation: packages/make-meaning/src/annotation-context.ts:227-236
304
- */
305
- static async resourceExists(
306
- resourceId: ResourceId,
307
- config: EnvironmentConfig
308
- ): Promise<boolean>
309
-
310
- /**
311
- * Get resource statistics (version, last updated)
312
- * Implementation: packages/make-meaning/src/annotation-context.ts:238-254
313
- */
314
- static async getResourceStats(
315
- resourceId: ResourceId,
316
- config: EnvironmentConfig
317
- ): Promise<{
318
- resourceId: ResourceId;
319
- version: number;
320
- updatedAt: string;
321
- }>
322
-
323
- /**
324
- * Get annotation context (surrounding text)
325
- * Implementation: packages/make-meaning/src/annotation-context.ts:256-314
326
- */
327
- static async getAnnotationContext(
328
- annotationId: AnnotationId,
329
- resourceId: ResourceId,
330
- contextBefore: number,
331
- contextAfter: number,
332
- config: EnvironmentConfig
333
- ): Promise<AnnotationContextResponse>
334
-
335
- /**
336
- * Generate AI summary of an annotation
337
- * Implementation: packages/make-meaning/src/annotation-context.ts:316-381
338
- */
339
- static async generateAnnotationSummary(
340
- annotationId: AnnotationId,
341
- resourceId: ResourceId,
342
- config: EnvironmentConfig
343
- ): Promise<ContextualSummaryResponse>
344
- }
345
- ```
135
+ Resources don't exist in isolation. A document becomes meaningful when we understand its annotations, its relationships to other resources, and the patterns within its content. `@semiont/make-meaning` provides the infrastructure to:
346
136
 
347
- **Options:**
348
- ```typescript
349
- interface BuildContextOptions {
350
- contextLines?: number; // Lines of surrounding text (default: 5)
351
- includeMetadata?: boolean; // Include resource metadata (default: true)
352
- }
353
- ```
137
+ 1. **Assemble context** from event-sourced storage
138
+ 2. **Detect patterns** using AI inference
139
+ 3. **Reason about relationships** through graph traversal
354
140
 
355
- ### GraphContext
141
+ This is the "applied meaning-making" layer - it sits between low-level AI primitives ([@semiont/inference](../inference/)) and high-level application orchestration ([apps/backend](../../apps/backend/)).
356
142
 
357
- Provides graph database operations for traversing resource relationships. All operations are delegated to @semiont/graph (which may use Neo4j or other graph database implementations).
143
+ ## Architecture
358
144
 
359
- ```typescript
360
- class GraphContext {
361
- /**
362
- * Get all resources referencing this resource (backlinks)
363
- * Requires graph traversal - uses @semiont/graph
364
- * Implementation: packages/make-meaning/src/graph-context.ts:26-30
365
- */
366
- static async getBacklinks(
367
- resourceId: ResourceId,
368
- config: EnvironmentConfig
369
- ): Promise<Annotation[]>
370
-
371
- /**
372
- * Find shortest path between two resources
373
- * Requires graph traversal - uses @semiont/graph
374
- * Implementation: packages/make-meaning/src/graph-context.ts:36-44
375
- */
376
- static async findPath(
377
- fromResourceId: ResourceId,
378
- toResourceId: ResourceId,
379
- config: EnvironmentConfig,
380
- maxDepth?: number
381
- ): Promise<GraphPath[]>
382
-
383
- /**
384
- * Get resource connections (graph edges)
385
- * Requires graph traversal - uses @semiont/graph
386
- * Implementation: packages/make-meaning/src/graph-context.ts:50-53
387
- */
388
- static async getResourceConnections(
389
- resourceId: ResourceId,
390
- config: EnvironmentConfig
391
- ): Promise<GraphConnection[]>
392
-
393
- /**
394
- * Search resources by name (cross-resource query)
395
- * Requires full-text search - uses @semiont/graph
396
- * Implementation: packages/make-meaning/src/graph-context.ts:59-62
397
- */
398
- static async searchResources(
399
- query: string,
400
- config: EnvironmentConfig,
401
- limit?: number
402
- ): Promise<ResourceDescriptor[]>
403
- }
404
- ```
145
+ Three-layer design separating concerns:
405
146
 
406
- ### AnnotationDetection
147
+ ```mermaid
148
+ graph TB
149
+ Backend["<b>apps/backend</b><br/>Job orchestration, HTTP APIs, streaming"]
150
+ MakeMeaning["<b>@semiont/make-meaning</b><br/>Context assembly, pattern detection,<br/>relationship reasoning, job workers"]
151
+ Inference["<b>@semiont/inference</b><br/>AI primitives: prompts, parsers,<br/>generateText abstraction"]
407
152
 
408
- AI-powered semantic pattern detection. Orchestrates the full pipeline: resource content → AI prompts → response parsing → validated matches.
153
+ Backend --> MakeMeaning
154
+ MakeMeaning --> Inference
409
155
 
410
- ```typescript
411
- class AnnotationDetection {
412
- /**
413
- * Detect passages that merit commentary
414
- * Implementation: packages/make-meaning/src/annotation-detection.ts:27-65
415
- * Uses: MotivationPrompts.buildCommentPrompt, MotivationParsers.parseComments
416
- */
417
- static async detectComments(
418
- resourceId: ResourceId,
419
- config: EnvironmentConfig,
420
- instructions?: string,
421
- tone?: string,
422
- density?: number
423
- ): Promise<CommentMatch[]>
424
-
425
- /**
426
- * Detect passages that should be highlighted
427
- * Implementation: packages/make-meaning/src/annotation-detection.ts:67-101
428
- * Uses: MotivationPrompts.buildHighlightPrompt, MotivationParsers.parseHighlights
429
- */
430
- static async detectHighlights(
431
- resourceId: ResourceId,
432
- config: EnvironmentConfig,
433
- instructions?: string,
434
- density?: number
435
- ): Promise<HighlightMatch[]>
436
-
437
- /**
438
- * Detect passages that merit assessment/evaluation
439
- * Implementation: packages/make-meaning/src/annotation-detection.ts:103-141
440
- * Uses: MotivationPrompts.buildAssessmentPrompt, MotivationParsers.parseAssessments
441
- */
442
- static async detectAssessments(
443
- resourceId: ResourceId,
444
- config: EnvironmentConfig,
445
- instructions?: string,
446
- tone?: string,
447
- density?: number
448
- ): Promise<AssessmentMatch[]>
449
-
450
- /**
451
- * Detect and extract structured tags from text
452
- * Implementation: packages/make-meaning/src/annotation-detection.ts:143-197
453
- * Uses: MotivationPrompts.buildTagPrompt, MotivationParsers.parseTags
454
- */
455
- static async detectTags(
456
- resourceId: ResourceId,
457
- config: EnvironmentConfig,
458
- schemaId: string,
459
- category: string
460
- ): Promise<TagMatch[]>
461
- }
156
+ style Backend fill:#e1f5ff
157
+ style MakeMeaning fill:#fff4e6
158
+ style Inference fill:#f3e5f5
462
159
  ```
463
160
 
464
- **Match types:**
465
- ```typescript
466
- // Re-exported from @semiont/inference for convenience
467
- interface CommentMatch {
468
- exact: string; // The exact text passage
469
- start: number; // Character offset start
470
- end: number; // Character offset end
471
- prefix?: string; // Context before (for fuzzy anchoring)
472
- suffix?: string; // Context after (for fuzzy anchoring)
473
- comment: string; // The AI-generated comment
474
- }
475
-
476
- interface HighlightMatch {
477
- exact: string;
478
- start: number;
479
- end: number;
480
- prefix?: string;
481
- suffix?: string;
482
- }
483
-
484
- interface AssessmentMatch {
485
- exact: string;
486
- start: number;
487
- end: number;
488
- prefix?: string;
489
- suffix?: string;
490
- assessment: string; // The AI-generated assessment
491
- }
492
-
493
- interface TagMatch {
494
- exact: string;
495
- start: number;
496
- end: number;
497
- prefix?: string;
498
- suffix?: string;
499
- category: string; // The tag category
500
- }
501
- ```
161
+ **Key principles:**
502
162
 
503
- **Detection parameters:**
163
+ - **Event-sourced context**: Resources and annotations assembled from event streams
164
+ - **Content-addressed storage**: Content retrieved using checksums (deduplication, caching)
165
+ - **Graph-backed relationships**: @semiont/graph provides traversal for backlinks, paths, connections
166
+ - **Explicit dependencies**: Workers receive JobQueue and EventStore via constructor (no singletons)
504
167
 
505
- - `instructions`: Custom guidance for the AI (e.g., "Focus on technical concepts")
506
- - `tone`: Tone for comments/assessments (e.g., "educational", "constructive", "analytical")
507
- - `density`: Target density 0.0-1.0 (0.5 = ~50% of passages should be detected)
168
+ See [Architecture](./docs/architecture.md) for complete details.
508
169
 
509
- ## Examples
170
+ ## Exports
510
171
 
511
- ### Building Annotation Context for AI
172
+ ### Context Assembly
512
173
 
513
- ```typescript
514
- import { AnnotationContext, ResourceContext } from '@semiont/make-meaning';
174
+ - `ResourceContext` - Resource metadata and content
175
+ - `AnnotationContext` - Annotation queries and context building
176
+ - `GraphContext` - Graph traversal and search
515
177
 
516
- // Get the full context needed for AI to process an annotation
517
- const context = await AnnotationContext.buildLLMContext(
518
- annotationUri,
519
- resourceId,
520
- config,
521
- { contextLines: 10 }
522
- );
178
+ ### Pattern Detection
523
179
 
524
- // context includes:
525
- // - The annotation itself
526
- // - Surrounding text (10 lines before/after)
527
- // - Resource metadata
528
- // - Related annotations in the vicinity
529
- ```
180
+ - `AnnotationDetection` - AI-powered semantic pattern detection
530
181
 
531
- ### Detecting Patterns and Creating Annotations
182
+ ### Job Workers
532
183
 
533
- ```typescript
534
- import { AnnotationDetection } from '@semiont/make-meaning';
535
- import { createEventStore } from '@semiont/event-sourcing';
184
+ - `ReferenceDetectionWorker` - Entity reference detection
185
+ - `GenerationWorker` - AI content generation
186
+ - `HighlightDetectionWorker` - Highlight detection
187
+ - `CommentDetectionWorker` - Comment detection
188
+ - `AssessmentDetectionWorker` - Assessment detection
189
+ - `TagDetectionWorker` - Structured tag detection
536
190
 
537
- // Detect highlights using AI
538
- const highlights = await AnnotationDetection.detectHighlights(
539
- resourceId,
540
- config,
541
- 'Find key definitions and important claims',
542
- 0.6 // Medium density
543
- );
191
+ See [Job Workers](./docs/job-workers.md) for implementation details.
544
192
 
545
- // Create annotations for each detected highlight
546
- const eventStore = await createEventStore(config);
547
- for (const highlight of highlights) {
548
- const annotation = {
549
- '@context': 'http://www.w3.org/ns/anno.jsonld',
550
- type: 'Annotation',
551
- id: generateAnnotationId(),
552
- motivation: 'highlighting',
553
- target: {
554
- type: 'SpecificResource',
555
- source: resourceUri,
556
- selector: [
557
- {
558
- type: 'TextPositionSelector',
559
- start: highlight.start,
560
- end: highlight.end,
561
- },
562
- {
563
- type: 'TextQuoteSelector',
564
- exact: highlight.exact,
565
- prefix: highlight.prefix,
566
- suffix: highlight.suffix,
567
- },
568
- ],
569
- },
570
- body: [],
571
- };
572
-
573
- await eventStore.appendEvent({
574
- type: 'annotation.added',
575
- resourceId,
576
- userId,
577
- version: 1,
578
- payload: { annotation },
579
- });
580
- }
581
- ```
582
-
583
- ### Navigating Resource Relationships
193
+ ### Type Re-exports
584
194
 
585
195
  ```typescript
586
- import { GraphContext, ResourceContext } from '@semiont/make-meaning';
587
-
588
- // Find all resources that link to this one
589
- const backlinks = await GraphContext.getBacklinks(resourceId, config);
590
- console.log(`Found ${backlinks.length} resources linking here`);
591
-
592
- // Find connection path between two resources
593
- const paths = await GraphContext.findPath(sourceId, targetId, config, 3);
594
- if (paths.length > 0) {
595
- console.log(`Shortest path has ${paths[0].nodes.length} nodes`);
596
- }
597
-
598
- // Get all connections for a resource
599
- const connections = await GraphContext.getResourceConnections(resourceId, config);
600
- // connections = [{ from: ResourceId, to: ResourceId, via: AnnotationId }, ...]
196
+ export type {
197
+ CommentMatch,
198
+ HighlightMatch,
199
+ AssessmentMatch,
200
+ TagMatch,
201
+ } from '@semiont/inference';
601
202
  ```
602
203
 
603
204
  ## Configuration
604
205
 
605
- All methods require an `EnvironmentConfig` object with:
206
+ All methods require an `EnvironmentConfig` object:
606
207
 
607
208
  ```typescript
608
- interface EnvironmentConfig {
609
- services: {
610
- backend: {
611
- publicURL: string; // Base URL for resource URIs
612
- };
613
- openai?: {
614
- apiKey: string; // Required for detection methods
615
- model?: string; // Default: 'gpt-4o-mini'
616
- temperature?: number; // Default: 0.7
617
- };
618
- };
619
- storage: {
620
- base: string; // Base path for filesystem storage
621
- };
622
- }
623
- ```
209
+ import type { EnvironmentConfig } from '@semiont/core';
624
210
 
625
- Example:
626
- ```typescript
627
211
  const config: EnvironmentConfig = {
628
212
  services: {
629
213
  backend: {
@@ -652,95 +236,16 @@ const config: EnvironmentConfig = {
652
236
  - **[@semiont/graph](../graph/)**: Neo4j graph database client
653
237
  - **[@semiont/ontology](../ontology/)**: Schema definitions for tags
654
238
  - **[@semiont/inference](../inference/)**: AI primitives (prompts, parsers, generateText)
239
+ - **[@semiont/jobs](../jobs/)**: Job queue and worker base class
655
240
 
656
- ## How Detection Works
657
-
658
- See [packages/inference/README.md](../inference/README.md) for details on the AI pipeline.
659
-
660
- **High-level flow:**
661
-
662
- 1. **Context Assembly**: `ResourceContext.getResourceMetadata()` retrieves resource content
663
- 2. **Prompt Building**: `MotivationPrompts.buildXPrompt()` creates AI prompt with domain knowledge
664
- 3. **AI Inference**: `generateText()` calls OpenAI API with prompt
665
- 4. **Response Parsing**: `MotivationParsers.parseX()` extracts structured matches from response
666
- 5. **Offset Validation**: Parser validates that `start`/`end` offsets match `exact` text in content
667
-
668
- **Example for highlights:**
669
-
670
- ```typescript
671
- // 1. Get content
672
- const resource = await ResourceContext.getResourceMetadata(resourceId, config);
673
- const content = await representationStore.retrieve(resource.contentId);
674
-
675
- // 2. Build prompt
676
- const prompt = MotivationPrompts.buildHighlightPrompt(
677
- content,
678
- 'Find key definitions',
679
- 0.6
680
- );
681
-
682
- // 3. Generate AI response
683
- const response = await generateText(prompt, config);
684
-
685
- // 4. Parse and validate
686
- const highlights = MotivationParsers.parseHighlights(response, content);
687
- // Returns: HighlightMatch[] with validated offsets
688
- ```
689
-
690
- ## Worker Integration
241
+ ## Testing
691
242
 
692
- Detection jobs are orchestrated by workers in [apps/backend/src/jobs/workers/](../../apps/backend/src/jobs/workers/):
693
-
694
- - [highlight-detection-worker.ts](../../apps/backend/src/jobs/workers/highlight-detection-worker.ts) - Delegated detection to `AnnotationDetection.detectHighlights()`
695
- - [comment-detection-worker.ts](../../apps/backend/src/jobs/workers/comment-detection-worker.ts) - Delegated detection to `AnnotationDetection.detectComments()`
696
- - [assessment-detection-worker.ts](../../apps/backend/src/jobs/workers/assessment-detection-worker.ts) - Delegated detection to `AnnotationDetection.detectAssessments()`
697
- - [tag-detection-worker.ts](../../apps/backend/src/jobs/workers/tag-detection-worker.ts) - Delegated detection to `AnnotationDetection.detectTags()`
698
-
699
- Workers handle:
700
- - Job lifecycle (pending → running → completed/failed)
701
- - Progress tracking and event emission
702
- - Annotation creation via event store
703
- - Error handling and retries
704
-
705
- All detection logic lives in `@semiont/make-meaning`, keeping workers focused on orchestration.
706
-
707
- ## Future Direction
708
-
709
- ### Deterministic Reasoning
710
-
711
- Future versions will add deterministic reasoning capabilities alongside AI-powered detection:
712
-
713
- - **Rule-based pattern matching**: Detect annotations using regex, string matching, or custom predicates
714
- - **Ontology-driven inference**: Apply OWL/RDFS reasoning over resource relationships
715
- - **Compositional reasoning**: Combine multiple reasoning strategies (AI + rules + ontology)
716
-
717
- Example (aspirational):
718
-
719
- ```typescript
720
- // AI-powered detection
721
- const aiHighlights = await AnnotationDetection.detectHighlights(resourceId, config);
722
-
723
- // Rule-based detection
724
- const ruleHighlights = await ResourceReasoning.findMatches(resourceId, {
725
- pattern: /\btheorem\b.*\bproof\b/gi,
726
- motivation: 'highlighting',
727
- });
728
-
729
- // Ontology-based reasoning
730
- const inferences = await ResourceReasoning.inferRelationships(resourceId, {
731
- ontology: 'http://example.org/math-ontology',
732
- rules: ['transitive-proof-chain'],
733
- });
243
+ ```bash
244
+ npm test # Run tests
245
+ npm run test:watch # Watch mode
246
+ npm run test:coverage # Coverage report
734
247
  ```
735
248
 
736
- This will enable hybrid approaches where AI handles semantic understanding and deterministic rules handle structural patterns.
737
-
738
- ### Enhanced Context Assembly
739
-
740
- - **Multi-resource context**: Build context spanning multiple related resources
741
- - **Temporal context**: Access historical versions of resources and annotations
742
- - **Provenance tracking**: Track reasoning chains and decision paths
743
-
744
249
  ## License
745
250
 
746
- MIT
251
+ Apache-2.0