@claritylabs/cl-sdk 3.2.13 → 4.1.0

package/README.md

@@ -24,25 +24,20 @@ npm install @claritylabs/cl-sdk pdf-lib zod
 ## Quick Start
 
 ```typescript
-import { createExtractor } from "@claritylabs/cl-sdk";
+import { buildPageSourceSpans, createExtractor } from "@claritylabs/cl-sdk";
 
 const extractor = createExtractor({
-  generateText: async ({ prompt, system, maxTokens, taskKind, budgetDiagnostics, providerOptions }) => {
-    const result = await yourProvider.generate({ prompt, system, maxTokens, taskKind, budgetDiagnostics, providerOptions });
-    return { text: result.text, usage: result.usage };
-  },
   generateObject: async ({ prompt, system, schema, maxTokens, taskKind, budgetDiagnostics, providerOptions }) => {
-    // Pass providerOptions.pdfBase64 and/or providerOptions.images to your model
     const result = await yourProvider.generateStructured({ prompt, system, schema, maxTokens, taskKind, budgetDiagnostics, providerOptions });
     return { object: result.object, usage: result.usage };
   },
-  concurrency: 3,
-  extractorConcurrency: 4,
-  formatConcurrency: 2,
-  reviewMode: "auto",
 });
 
-const result = await extractor.extract(pdfBase64);
+const sourceSpans = buildPageSourceSpans([
+  { documentId: "policy-123", sourceKind: "policy_pdf", pageNumber: 1, text: pageOneText },
+]);
+
+const result = await extractor.extract(pdfBase64, "policy-123", { sourceSpans });
 console.log(result.sourceTree);          // DocumentSourceNode[] canonical hierarchy
 console.log(result.sourceSpans);         // SourceSpan[] smallest PDF evidence units
 console.log(result.operationalProfile);  // Source-backed facts for policy lists, COIs, compliance
@@ -75,12 +70,14 @@ const sourceSpans = buildPageSourceSpans([
 ]);
 
 const sourceStore = new MemorySourceStore();
-const extractor = createExtractor({ generateText, generateObject, sourceStore });
+const extractor = createExtractor({ generateObject, sourceStore });
 
 const result = await extractor.extract(pdfBase64, "policy-123", { sourceSpans });
 ```
 
-When source spans are available, extraction returns `sourceTree`, `sourceSpans`, `operationalProfile`, `warnings`, `tokenUsage`, and `performanceReport`. The source tree is canonical for policy wording and hierarchy. The extractor groups parser-provided page, title, table, row, and text spans into declarations, policy forms, endorsements, sections, and schedules without running a separate form-inventory, page-map, or source-tree-organizer model call. The operational profile model receives a bounded packet of high-value declaration, schedule, premium, producer, endorsement, and coverage evidence rather than the full document wording. The operational profile contains model-extracted policy metadata, parties, coverage units, nested coverage limit terms, deductibles/retentions, premiums, key dates, and endorsement-support facts, each with `sourceNodeIds` and `sourceSpanIds`. Source-backed identity records such as insurer, producer, contacts, named insured rows, endorsement parties, and standalone insured addresses must include non-empty `sourceSpanIds`. After extraction, a bounded model cleanup pass may keep, drop, or update existing coverage rows and terms by index; the SDK enforces the schema and rejects source IDs that are not present in the current source tree/spans. Endorsement schedules are modeled as whole endorsement coverage units with their own `limits[]` terms instead of unrelated flat rows like `Aggregate Limit`.
+When source spans are available, extraction returns `sourceTree`, `sourceSpans`, `operationalProfile`, `warnings`, `tokenUsage`, and `performanceReport`. The source tree is canonical for policy wording and hierarchy. The extractor groups parser-provided page, title, table, row, and text spans into declarations, policy forms, endorsements, sections, and schedules without running a separate form-inventory, page-map, or source-tree-organizer model call. The operational profile model receives a bounded packet of high-value declaration, schedule, premium, producer, endorsement, and coverage evidence rather than the full document wording. The operational profile contains model-extracted policy metadata, ACORD `linesOfBusiness`, parties, coverage units, nested coverage limit terms, deductibles/retentions, premiums, key dates, endorsement-support facts, and source-backed `declarationFacts` for named-insured identity details such as mailing address, DBA, entity type, tax ID, and additional named insured rows. Each fact keeps `sourceNodeIds` and `sourceSpanIds`. Source-backed identity records such as insurer, producer, contacts, named insured rows, endorsement parties, standalone insured addresses, and declaration facts must include non-empty `sourceSpanIds`. After extraction, a bounded model cleanup pass may keep, drop, or update existing coverage rows and terms by index; the SDK enforces the schema and rejects source IDs that are not present in the current source tree/spans. Endorsement schedules are modeled as whole endorsement coverage units with their own `limits[]` terms instead of unrelated flat rows like `Aggregate Limit`.
+
+CL SDK 4.0 replaces the old `PolicyType` enum and operational-profile `policyTypes` field with ACORD line of business codes. Use `AcordLobCode`, `AcordLobCodeSchema`, `ACORD_LOB_CODES`, `ACORD_LOB_LABELS`, `LEGACY_POLICY_TYPE_TO_LOB`, `PERSONAL_LOB_CODES`, `normalizeOperationalLinesOfBusiness`, and `resolveOperationalProfileLinesOfBusiness`. Serialized legacy profiles with `policyTypes` are still accepted by `PolicyOperationalProfileSchema` and normalize to `linesOfBusiness`.
 
 Store `result.sourceTree` in a retrievable node index and embed node `description` values for search. Keep `result.sourceSpans` as the exact PDF highlighting layer. `result.document` and its `documentOutline` remain compatibility projections for existing host screens; do not treat broad structured policy JSON as canonical extraction truth.
 
@@ -149,7 +146,6 @@ CL-SDK uses deterministic scaffolding with agentic decision points rather than f
 - Source-tree construction stays deterministic from parser spans; the model is used for the operational profile and optional coverage cleanup because those are end-user facts.
 - Referential coverage resolution and application/query workflows still use bounded target-specific actions when those workflows need lookup or explanation steps.
 - Formatting skips the LLM cleanup pass for plain prose and formats long or noisy markdown/table/list content in parallel batches outside the v3 source-span extraction path.
-- `reviewMode: "auto"` skips the expensive LLM review pass when deterministic checks are clean and source spans are available. Use `"always"` for maximum review coverage or `"skip"` when the host owns quality review separately.
 - Application processing plans optional backfill, context auto-fill, document search, batching, reply parsing, lookup, explanations, and next-batch email generation based on current active question state. Conditional fields that are not active are skipped until their parent answers trigger them.
 
 These gates reduce unnecessary provider calls while preserving reliability for edge cases where additional focused extraction or retrieval is needed.

package/dist/application.d.mts

@@ -1,2 +1,2 @@
-export { s as AcroFormMapping, t as AcroFormMappingSchema, u as AnswerParsingResult, v as AnswerParsingResultSchema, w as ApplicationClassifyResult, x as ApplicationClassifyResultSchema, y as ApplicationContextProposal, z as ApplicationContextProposalSchema, C as ApplicationEmailReviewSchema, E as ApplicationField, F as ApplicationFieldSchema, H as ApplicationListFilters, J as ApplicationNextQuestions, K as ApplicationPacket, N as ApplicationPacketAnswer, R as ApplicationPacketAnswerSchema, V as ApplicationPacketSchema, W as ApplicationPipelineConfig, X as ApplicationQualityArtifactSchema, Y as ApplicationQualityIssueSchema, Z as ApplicationQualityReportSchema, _ as ApplicationQualityRoundSchema, $ as ApplicationQuestionCondition, a0 as ApplicationQuestionConditionSchema, a1 as ApplicationQuestionGraph, a2 as ApplicationQuestionGraphSchema, a3 as ApplicationQuestionNode, a4 as ApplicationQuestionNodeSchema, a5 as ApplicationRepeat, a6 as ApplicationRepeatSchema, a7 as ApplicationState, a8 as ApplicationStateSchema, a9 as ApplicationStore, aa as ApplicationTemplate, ab as ApplicationTemplateListFilters, ac as ApplicationTemplateSchema, ad as ApplicationTemplateStore, ae as AutoFillMatch, af as AutoFillMatchSchema, ag as AutoFillResult, ah as AutoFillResultSchema, ak as BackfillProvider, al as BuildApplicationPacketInput, am as BuildApplicationPacketResult, aq as ContextProposalResult, av as CreateApplicationRunInput, aN as FieldExtractionResult, aO as FieldExtractionResultSchema, aP as FieldType, aQ as FieldTypeSchema, aR as FlatPdfPlacement, aS as FlatPdfPlacementSchema, aU as LookupFill, aV as LookupFillResult, aW as LookupFillResultSchema, aX as LookupFillSchema, aY as LookupRequest, aZ as LookupRequestSchema, b7 as ParsedAnswer, b8 as ParsedAnswerSchema, bg as PriorAnswer, bh as ProcessApplicationInput, bi as ProcessApplicationResult, bj as ProcessReplyInput, bk as ProcessReplyResult, bl as QuestionBatchResult, bm as QuestionBatchResultSchema, bp as ReplyIntent, bq as ReplyIntentSchema, bN as applyApplicationAnswers, bO as buildApplicationPacket, bP as buildQuestionGraphFromFields, bQ as createApplicationPipeline, bR as createApplicationRun, bS as extractQuestionGraphFromFields, bT as flattenQuestionGraph, bU as getActiveApplicationFields, bV as getNextApplicationQuestions, bW as normalizeApplicationQuestionGraph, bX as planNextApplicationQuestions, bY as proposeContextWrites, b_ as validateApplicationPacket } from './index-WjheX6Dt.mjs';
+export { s as AcroFormMapping, t as AcroFormMappingSchema, u as AnswerParsingResult, v as AnswerParsingResultSchema, w as ApplicationClassifyResult, x as ApplicationClassifyResultSchema, y as ApplicationContextProposal, z as ApplicationContextProposalSchema, C as ApplicationEmailReviewSchema, E as ApplicationField, F as ApplicationFieldSchema, H as ApplicationListFilters, J as ApplicationNextQuestions, K as ApplicationPacket, N as ApplicationPacketAnswer, R as ApplicationPacketAnswerSchema, V as ApplicationPacketSchema, W as ApplicationPipelineConfig, X as ApplicationQualityArtifactSchema, Y as ApplicationQualityIssueSchema, Z as ApplicationQualityReportSchema, _ as ApplicationQualityRoundSchema, $ as ApplicationQuestionCondition, a0 as ApplicationQuestionConditionSchema, a1 as ApplicationQuestionGraph, a2 as ApplicationQuestionGraphSchema, a3 as ApplicationQuestionNode, a4 as ApplicationQuestionNodeSchema, a5 as ApplicationRepeat, a6 as ApplicationRepeatSchema, a7 as ApplicationState, a8 as ApplicationStateSchema, a9 as ApplicationStore, aa as ApplicationTemplate, ab as ApplicationTemplateListFilters, ac as ApplicationTemplateSchema, ad as ApplicationTemplateStore, ae as AutoFillMatch, af as AutoFillMatchSchema, ag as AutoFillResult, ah as AutoFillResultSchema, ak as BackfillProvider, al as BuildApplicationPacketInput, am as BuildApplicationPacketResult, aq as ContextProposalResult, av as CreateApplicationRunInput, aN as FieldExtractionResult, aO as FieldExtractionResultSchema, aP as FieldType, aQ as FieldTypeSchema, aR as FlatPdfPlacement, aS as FlatPdfPlacementSchema, aU as LookupFill, aV as LookupFillResult, aW as LookupFillResultSchema, aX as LookupFillSchema, aY as LookupRequest, aZ as LookupRequestSchema, bb as ParsedAnswer, bc as ParsedAnswerSchema, bk as PriorAnswer, bl as ProcessApplicationInput, bm as ProcessApplicationResult, bn as ProcessReplyInput, bo as ProcessReplyResult, bp as QuestionBatchResult, bq as QuestionBatchResultSchema, bt as ReplyIntent, bu as ReplyIntentSchema, bR as applyApplicationAnswers, bS as buildApplicationPacket, bT as buildQuestionGraphFromFields, bU as createApplicationPipeline, bV as createApplicationRun, bW as extractQuestionGraphFromFields, bX as flattenQuestionGraph, bY as getActiveApplicationFields, bZ as getNextApplicationQuestions, b_ as normalizeApplicationQuestionGraph, b$ as planNextApplicationQuestions, c0 as proposeContextWrites, c2 as validateApplicationPacket } from './index-BzrY6OMD.mjs';
 import 'zod';

package/dist/application.d.ts

@@ -1,2 +1,2 @@
-export { s as AcroFormMapping, t as AcroFormMappingSchema, u as AnswerParsingResult, v as AnswerParsingResultSchema, w as ApplicationClassifyResult, x as ApplicationClassifyResultSchema, y as ApplicationContextProposal, z as ApplicationContextProposalSchema, C as ApplicationEmailReviewSchema, E as ApplicationField, F as ApplicationFieldSchema, H as ApplicationListFilters, J as ApplicationNextQuestions, K as ApplicationPacket, N as ApplicationPacketAnswer, R as ApplicationPacketAnswerSchema, V as ApplicationPacketSchema, W as ApplicationPipelineConfig, X as ApplicationQualityArtifactSchema, Y as ApplicationQualityIssueSchema, Z as ApplicationQualityReportSchema, _ as ApplicationQualityRoundSchema, $ as ApplicationQuestionCondition, a0 as ApplicationQuestionConditionSchema, a1 as ApplicationQuestionGraph, a2 as ApplicationQuestionGraphSchema, a3 as ApplicationQuestionNode, a4 as ApplicationQuestionNodeSchema, a5 as ApplicationRepeat, a6 as ApplicationRepeatSchema, a7 as ApplicationState, a8 as ApplicationStateSchema, a9 as ApplicationStore, aa as ApplicationTemplate, ab as ApplicationTemplateListFilters, ac as ApplicationTemplateSchema, ad as ApplicationTemplateStore, ae as AutoFillMatch, af as AutoFillMatchSchema, ag as AutoFillResult, ah as AutoFillResultSchema, ak as BackfillProvider, al as BuildApplicationPacketInput, am as BuildApplicationPacketResult, aq as ContextProposalResult, av as CreateApplicationRunInput, aN as FieldExtractionResult, aO as FieldExtractionResultSchema, aP as FieldType, aQ as FieldTypeSchema, aR as FlatPdfPlacement, aS as FlatPdfPlacementSchema, aU as LookupFill, aV as LookupFillResult, aW as LookupFillResultSchema, aX as LookupFillSchema, aY as LookupRequest, aZ as LookupRequestSchema, b7 as ParsedAnswer, b8 as ParsedAnswerSchema, bg as PriorAnswer, bh as ProcessApplicationInput, bi as ProcessApplicationResult, bj as ProcessReplyInput, bk as ProcessReplyResult, bl as QuestionBatchResult, bm as QuestionBatchResultSchema, bp as ReplyIntent, bq as ReplyIntentSchema, bN as applyApplicationAnswers, bO as buildApplicationPacket, bP as buildQuestionGraphFromFields, bQ as createApplicationPipeline, bR as createApplicationRun, bS as extractQuestionGraphFromFields, bT as flattenQuestionGraph, bU as getActiveApplicationFields, bV as getNextApplicationQuestions, bW as normalizeApplicationQuestionGraph, bX as planNextApplicationQuestions, bY as proposeContextWrites, b_ as validateApplicationPacket } from './index-WjheX6Dt.js';
+export { s as AcroFormMapping, t as AcroFormMappingSchema, u as AnswerParsingResult, v as AnswerParsingResultSchema, w as ApplicationClassifyResult, x as ApplicationClassifyResultSchema, y as ApplicationContextProposal, z as ApplicationContextProposalSchema, C as ApplicationEmailReviewSchema, E as ApplicationField, F as ApplicationFieldSchema, H as ApplicationListFilters, J as ApplicationNextQuestions, K as ApplicationPacket, N as ApplicationPacketAnswer, R as ApplicationPacketAnswerSchema, V as ApplicationPacketSchema, W as ApplicationPipelineConfig, X as ApplicationQualityArtifactSchema, Y as ApplicationQualityIssueSchema, Z as ApplicationQualityReportSchema, _ as ApplicationQualityRoundSchema, $ as ApplicationQuestionCondition, a0 as ApplicationQuestionConditionSchema, a1 as ApplicationQuestionGraph, a2 as ApplicationQuestionGraphSchema, a3 as ApplicationQuestionNode, a4 as ApplicationQuestionNodeSchema, a5 as ApplicationRepeat, a6 as ApplicationRepeatSchema, a7 as ApplicationState, a8 as ApplicationStateSchema, a9 as ApplicationStore, aa as ApplicationTemplate, ab as ApplicationTemplateListFilters, ac as ApplicationTemplateSchema, ad as ApplicationTemplateStore, ae as AutoFillMatch, af as AutoFillMatchSchema, ag as AutoFillResult, ah as AutoFillResultSchema, ak as BackfillProvider, al as BuildApplicationPacketInput, am as BuildApplicationPacketResult, aq as ContextProposalResult, av as CreateApplicationRunInput, aN as FieldExtractionResult, aO as FieldExtractionResultSchema, aP as FieldType, aQ as FieldTypeSchema, aR as FlatPdfPlacement, aS as FlatPdfPlacementSchema, aU as LookupFill, aV as LookupFillResult, aW as LookupFillResultSchema, aX as LookupFillSchema, aY as LookupRequest, aZ as LookupRequestSchema, bb as ParsedAnswer, bc as ParsedAnswerSchema, bk as PriorAnswer, bl as ProcessApplicationInput, bm as ProcessApplicationResult, bn as ProcessReplyInput, bo as ProcessReplyResult, bp as QuestionBatchResult, bq as QuestionBatchResultSchema, bt as ReplyIntent, bu as ReplyIntentSchema, bR as applyApplicationAnswers, bS as buildApplicationPacket, bT as buildQuestionGraphFromFields, bU as createApplicationPipeline, bV as createApplicationRun, bW as extractQuestionGraphFromFields, bX as flattenQuestionGraph, bY as getActiveApplicationFields, bZ as getNextApplicationQuestions, b_ as normalizeApplicationQuestionGraph, b$ as planNextApplicationQuestions, c0 as proposeContextWrites, c2 as validateApplicationPacket } from './index-BzrY6OMD.js';
 import 'zod';