@semiont/jobs 0.5.5 → 0.5.6

package/README.md

@@ -10,7 +10,7 @@ Job queue, worker infrastructure, and annotation workers for [Semiont](https://g
 
 ## Architecture Context
 
-Workers run in a separate separate process and connect to the Knowledge System (KS) over HTTP/SSE using `WorkerStateUnit` from `@semiont/api-client`. Workers receive job assignments via SSE push, claim jobs atomically, and emit domain events back to the KS via HTTP. The KS ingests these events onto its EventBus for SSE delivery to the frontend.
+Workers run in a separate process and connect to the Knowledge System (KS) over HTTP/SSE using a `SemiontSession` (from `@semiont/sdk`) driven by a `JobClaimAdapter`. Workers receive job assignments via an SSE `job:queued` subscription, claim jobs atomically, and emit domain events back to the KS via `session.client.transport.emit(...)`. The KS ingests these events onto its EventBus for SSE delivery to the frontend.
 
 ## Installation
 
@@ -19,20 +19,24 @@ npm install @semiont/jobs
 ```
 
 **Dependencies:**
-- `@semiont/core` — Core types, EventBus
-- `@semiont/api-client` — OpenAPI types
+- `@semiont/core` — Core types, `SemiontProject`, EventBus
+- `@semiont/sdk` — `SemiontSession`, `JobClaimAdapter` (worker process)
+- `@semiont/api-client` — HTTP transport, OpenAPI types
 - `@semiont/inference` — InferenceClient for AI operations
+- `@semiont/content` — Content storage URI derivation
+- `@semiont/observability` — Spans and job-outcome metrics
 
 ## Quick Start
 
 ```typescript
-import { JobQueue, type PendingJob, type GenerationParams } from '@semiont/jobs';
-import { EventBus, userId, resourceId, annotationId } from '@semiont/core';
-import { jobId } from '@semiont/api-client';
+import { FsJobQueue, type PendingJob, type GenerationParams } from '@semiont/jobs';
+import { EventBus, userId, resourceId, annotationId, jobId } from '@semiont/core';
+import { SemiontProject } from '@semiont/core/node';
 
-// Initialize
+// Initialize — jobs are stored under project.jobsDir
 const eventBus = new EventBus();
-const jobQueue = new JobQueue({ dataDir: './data' }, logger, eventBus);
+const project = new SemiontProject('/path/to/project');
+const jobQueue = new FsJobQueue(project, logger, eventBus);
 await jobQueue.initialize();
 
 // Create a job
@@ -97,45 +101,30 @@ The `userName`, `userEmail`, and `userDomain` fields are used by workers to buil
 
 ## Annotation Workers
 
-Six workers process different annotation types:
+The worker process (`worker-main.ts` → `startWorkerProcess` in `worker-process.ts`) claims jobs over the bus via a `JobClaimAdapter` and dispatches by `jobType` to a processor function. There are no per-type worker classes; each job type maps to one `process*Job` function:
 
-| Worker | Job Type | Constructor |
-|--------|----------|------------|
-| `ReferenceAnnotationWorker` | `reference-annotation` | `(jobQueue, config, inferenceClient, eventBus, contentFetcher, logger)` |
-| `GenerationWorker` | `generation` | `(jobQueue, config, inferenceClient, eventBus, logger)` |
-| `HighlightAnnotationWorker` | `highlight-annotation` | `(jobQueue, config, inferenceClient, eventBus, contentFetcher, logger)` |
-| `AssessmentAnnotationWorker` | `assessment-annotation` | `(jobQueue, config, inferenceClient, eventBus, contentFetcher, logger)` |
-| `CommentAnnotationWorker` | `comment-annotation` | `(jobQueue, config, inferenceClient, eventBus, contentFetcher, logger)` |
-| `TagAnnotationWorker` | `tag-annotation` | `(jobQueue, config, inferenceClient, eventBus, contentFetcher, logger)` |
+| Job Type | Processor |
+|----------|-----------|
+| `reference-annotation` | `processReferenceJob` |
+| `generation` | `processGenerationJob` |
+| `highlight-annotation` | `processHighlightJob` |
+| `assessment-annotation` | `processAssessmentJob` |
+| `comment-annotation` | `processCommentJob` |
+| `tag-annotation` | `processTagJob` |
 
-Workers emit EventBus commands (`mark:create`, `job:start`, `job:complete`, etc.) — the Stower actor in @semiont/make-meaning handles persistence.
+Detection logic lives in the `AnnotationDetection` class (`src/workers/annotation-detection.ts`); generation synthesis in `generateResourceFromTopic()` (`src/workers/generation/resource-generation.ts`). Each processor fetches content via `session.client.browse.resourceContent(resourceId)`.
 
-## Custom Workers
+Workers emit bus events via `session.client.transport.emit('mark:create' | 'job:start' | 'job:report-progress' | 'job:complete' | 'job:fail', payload)` — the Stower actor in @semiont/make-meaning handles persistence.
 
-```typescript
-import { JobWorker, type AnyJob } from '@semiont/jobs';
-import type { Logger } from '@semiont/core';
-
-class MyWorker extends JobWorker {
-  constructor(jobQueue: JobQueue, logger: Logger) {
-    super(jobQueue, 1000, 5000, logger);
-    //              ^^^^  ^^^^
-    //              poll   error backoff
-  }
+## Adding a Job Type
 
-  protected getWorkerName(): string {
-    return 'MyWorker';
-  }
+Workers are not subclassed. To add a job type:
 
-  protected canProcessJob(job: AnyJob): boolean {
-    return job.metadata.type === 'generation';
-  }
+1. Add the new `JobType` and its params/result/progress types in `src/types.ts`.
+2. Add a `process*Job` function in `src/processors.ts` that runs the inference and returns the annotations/result.
+3. Dispatch the new `jobType` to that processor in `handleJobInner()` in `src/worker-process.ts`.
 
-  protected async executeJob(job: AnyJob): Promise<any> {
-    // Your processing logic — return result object
-  }
-}
-```
+Processors are transport-agnostic: they take content, an `InferenceClient`, the job params, the user id, the `generator` (W3C SoftwareAgent), and an `onProgress` callback, and return annotations plus a result. The worker process handles claiming, content fetching, and lifecycle event emission.
 
 ## Discriminated Unions
 
@@ -182,7 +171,8 @@ Apache-2.0
 
 ## Related Packages
 
-- [`@semiont/core`](../core/) — Domain types, EventBus
-- [`@semiont/api-client`](../api-client/) — OpenAPI types
+- [`@semiont/core`](../core/) — Domain types, `SemiontProject`, EventBus
+- [`@semiont/sdk`](../sdk/) — `SemiontSession`, `JobClaimAdapter`
+- [`@semiont/api-client`](../api-client/) — HTTP transport, OpenAPI types
 - [`@semiont/inference`](../inference/) — AI inference client
 - [`@semiont/make-meaning`](../make-meaning/) — Actor model, Knowledge Base, service orchestration

package/dist/index.d.ts

@@ -415,80 +415,14 @@ declare class FsJobQueue implements JobQueue {
 }
 
 /**
- * Job Worker Base Class
- *
- * Abstract worker that polls the job queue and processes jobs.
- * Subclasses implement specific job processing logic.
- */
-
-declare abstract class JobWorker {
-    private running;
-    private currentJob;
-    private pollIntervalMs;
-    private errorBackoffMs;
-    protected jobQueue: JobQueue;
-    protected logger: Logger;
-    constructor(jobQueue: JobQueue, pollIntervalMs: number | undefined, errorBackoffMs: number | undefined, logger: Logger);
-    /**
-     * Start the worker (polls queue in loop)
-     */
-    start(): Promise<void>;
-    /**
-     * Stop the worker (graceful shutdown)
-     */
-    stop(): Promise<void>;
-    /**
-     * Poll for next job to process
-     */
-    private pollNextJob;
-    /**
-     * Process a job (handles state transitions and error handling)
-     */
-    private processJob;
-    /**
-     * Handle job failure (retry or move to failed)
-     */
-    protected handleJobFailure(job: AnyJob, error: any): Promise<void>;
-    /**
-     * Update job progress (best-effort, doesn't throw)
-     */
-    protected updateJobProgress(job: AnyJob): Promise<void>;
-    /**
-     * Sleep utility
-     */
-    protected sleep(ms: number): Promise<void>;
-    /**
-     * Emit completion event (optional hook for subclasses)
-     * Override this to emit job-specific completion events (e.g., job.completed)
-     */
-    protected emitCompletionEvent(_job: RunningJob<any, any>, _result: any): Promise<void>;
-    /**
-     * Get worker name (for logging)
-     */
-    protected abstract getWorkerName(): string;
-    /**
-     * Check if this worker can process the given job
-     */
-    protected abstract canProcessJob(job: AnyJob): boolean;
-    /**
-     * Execute the job (job-specific logic)
-     * This is where the actual work happens
-     * Return the result object (or void for jobs without results)
-     * Throw an error to trigger retry logic
-     */
-    protected abstract executeJob(job: AnyJob): Promise<any>;
-}
-
-/**
- * Job Processors — extracted from JobWorker subclasses
+ * Job Processors
  *
  * Pure functions that take content + inference client + params,
  * report progress via callback, and return annotations + results.
  *
  * No EventBus, no JobQueue, no side effects except calling inference.
- * Two callers:
- *   1. In-process JobWorker subclasses (existing path)
- *   2. Remote WorkerStateUnit via worker-process.ts (new path)
+ * Driven by the remote worker process (worker-process.ts), which claims
+ * jobs over SSE and dispatches by jobType to these functions.
  */
 
 type Agent = components['schemas']['Agent'];
@@ -641,5 +575,5 @@ declare function generateResourceFromTopic(topic: string, entityTypes: string[],
     content: string;
 }>;
 
-export { AnnotationDetection, FsJobQueue, JobWorker, generateResourceFromTopic, isCancelledJob, isCompleteJob, isFailedJob, isPendingJob, isRunningJob, processAssessmentJob, processCommentJob, processGenerationJob, processHighlightJob, processReferenceJob, processTagJob };
+export { AnnotationDetection, FsJobQueue, generateResourceFromTopic, isCancelledJob, isCompleteJob, isFailedJob, isPendingJob, isRunningJob, processAssessmentJob, processCommentJob, processGenerationJob, processHighlightJob, processReferenceJob, processTagJob };
 export type { AnyJob, AssessmentDetectionJob, AssessmentDetectionParams, AssessmentDetectionProgress, AssessmentDetectionResult, CancelledJob, CommentDetectionJob, CommentDetectionParams, CommentDetectionProgress, CommentDetectionResult, CompleteJob, ContentFetcher, DetectionJob, DetectionParams, DetectionProgress, DetectionResult, FailedJob, GenerationJob, GenerationParams, GenerationResult, HighlightDetectionJob, HighlightDetectionParams, HighlightDetectionProgress, HighlightDetectionResult, JobMetadata, JobQueryFilters, JobQueue, JobStatus, JobType, OnProgress, PendingJob, ProcessorResult, RunningJob, TagDetectionJob, TagDetectionParams, TagDetectionProgress, TagDetectionResult, YieldProgress };