factbag 0.0.0-alpha.3 → 0.0.0-alpha.5

package/CHANGELOG.md

@@ -1,5 +1,46 @@
 # factbag
 
+## 0.0.0-alpha.5
+
+### Minor Changes
+
+- 19ac508: **Breaking:** errors have their own entry point, `factbag/exceptions`. `ProgrammerError`, `ExclusiveDefinitionProgrammerError`, `PrimitiveArithmeticProgrammerError`, and `UnsafeUnboundVariableProgrammerError` move there from `factbag/kb`. `UnconstructiveRecursionProgrammerError`, which the solver raised but no entry point exported, is now exported too. `UnsupportedPredicateError` moves there from `factbag/jev/errors` and now extends `ProgrammerError`.
+- 19ac508: **Breaking:** Jev moves from `factbag/jev` to its own package, `@factbag/jev`. Install it alongside `factbag` and `@typesafe-ai/sdk`, which is now a required peer dependency at `^0.7.0`, and import `classifier`, `resolver`, `createConfig`, `Config`, and the `DEFAULT_*` constants from `@factbag/jev`. `factbag` no longer has a Jev entry point or an SDK peer dependency. `@factbag/jev` and `@factbag/clef` are released together with `factbag` and require the same `factbag` version.
+
+  **Breaking:** `@factbag/jev` has a single entry point. `factbag/jev/classifier`, `factbag/jev/resolve`, `factbag/jev/config`, and `factbag/jev/errors` have no replacement subpaths.
+
+  Add `batchSize` to Jev's `Config`: how many questions each request holds. It defaults to 64 and has no upper limit. An invalid `batchSize` throws when the config is created or a run is built.
+
+  Jev's `createConfig` now returns `Normalized & Config`, so `client` is no longer guaranteed on the result.
+
+- dd3730b: **Breaking:** `factbag/guards` and `factbag/query` are removed. Import `guard`, `GuardedKB`, and the other guard exports from `factbag/kb`, and `query`, `queryAll`, `Judgment`, and the goal and support types from `factbag/dsl`.
+
+## 0.0.0-alpha.4
+
+### Minor Changes
+
+- 2814aab: Allow enumerations to hold numbers or dates as well as strings, with one member
+  type per enumeration. Members must be unique; dates compare by instant and are
+  copied on construction. NaN and invalid dates are rejected. Enumeration membership
+  validation at assertion and query time is unchanged.
+- 1f5bb1b: Classify predicates whose fields are number or Date enumerations, alongside
+  string enumerations. Dates are sent to the model as ISO strings and retained as
+  `Date` values in evidence. Open-ended `String`, `Number`, and `Date` fields
+  remain unsupported.
+- 9ca1766: Reorganize classification and resolution to separate evidence workflows from model requests. The workflows own evidence admission and knowledge-base updates; the Jev implementation owns prompts, encoding, batching, and response validation.
+
+  **Breaking changes:** call the classifier with `classifier({ corpus, input, config })`. Classification and resolution jobs now expose only `execute(): Promise<void>`; read the resulting knowledge base from `run.kb`. Job metadata and assessment and resolution reports are removed. Replace `RequestConfig`, `ResolutionConfig`, and nested `AdmissionPolicy` settings with the shared, flat `Config`. Pass an SDK client directly instead of implementing the removed custom client interfaces.
+
+  Classification now executes one job per chunk across all permitted predicates. All responses for the chunk must validate before its evidence is written. Failed jobs can be retried without partial evidence, repeating the chunk's requests; successful repeats add fresh evidence.
+
+  Optionally filter candidates by passage relevance with `config: { routing: true }`. Each chunk first asks which predicates the passage may bear on. For predicates that make claims about supplied entities or values, it can also filter those symbols before combining them into complete candidate facts. `candidateProductCutoff` controls when this extra filtering runs. Filtering defaults to off: false negatives can suppress evidence, including denials, and the extra requests do not guarantee cost savings.
+
+  Expose `createConfig`, `Config`, and shared `DEFAULT_*` constants through `factbag/jev/config`. Normalize settings once and preserve a shared lazy client when a factory-created config is reused across workflows.
+
+  Export `fieldEquals` from `factbag/dsl` and use core equality in Jev: entities compare by constructor and ID, dates by instant. Resolution now groups contradictory evidence about separate instances of the same entity.
+
+  Encode candidate values when building requests, using the original entities and dates instead of construction-time JSON snapshots. Candidate domains remain fixed at run construction.
+
 ## 0.0.0-alpha.3
 
 ### Minor Changes

package/README.md

@@ -32,9 +32,13 @@ the rules again.
 ## Install
 
 ```sh
-npm install factbag@alpha @typesafe-ai/sdk@^0.6.0
+npm install factbag@alpha @factbag/jev@alpha @typesafe-ai/sdk@^0.7.0
 ```
 
+`@factbag/jev` is released with `factbag` and needs the same version. To use
+Cloudflare's Clef models on Workers AI instead, install `@factbag/clef@alpha`, which
+does not need the TypeSafe SDK.
+
 ESM only. Requires Node 22.18+ on the 22.x line, or Node 24 or 26.
 
 Set your TypeSafe API key in the environment:
@@ -63,12 +67,7 @@ into three conditions:
 Declare the facts we'll combine, and describe the claims Jev will assess.
 
 ```ts
-import { and, enumeration, predicate, variable } from "factbag/dsl";
-import { KnowledgeBase } from "factbag/kb";
-import { query } from "factbag/query";
-import { Corpus } from "factbag/corpus";
-import { guard } from "factbag/guards";
-import { classifier } from "factbag/jev/classifier";
+import { enumeration, predicate } from "factbag/dsl";
 
 class Person {
   constructor(
@@ -79,7 +78,7 @@ class Person {
 
 const hasIncomeOver200k = predicate(
   "has_income_over_200k",
-  { person: Person, year: enumeration("2024", "2025") },
+  { person: Person, year: enumeration(2024, 2025) },
   {
     description:
       "The person's total individual annual income exceeded USD 200,000 in the " +
@@ -112,13 +111,16 @@ All three income conditions must hold for this route to qualification. Our
 application also requires verified identity before sending an investor for review.
 
 ```ts
+import { and, variable } from "factbag/dsl";
+import { KnowledgeBase } from "factbag/kb";
+
 const person = variable("person");
 const kb = new KnowledgeBase();
 
 kb.rule(isQualifiedInvestor({ person })).if(
   and(
-    hasIncomeOver200k({ person, year: "2024" }),
-    hasIncomeOver200k({ person, year: "2025" }),
+    hasIncomeOver200k({ person, year: 2024 }),
+    hasIncomeOver200k({ person, year: 2025 }),
     expectsSameIncome({ person }),
   ),
 );
@@ -134,6 +136,8 @@ Identity verification comes directly from our onboarding system. Income evidence
 goes into the corpus for Jev to assess.
 
 ```ts
+import { Corpus } from "factbag/corpus";
+
 const alice = new Person("alice", "Alice");
 kb.entities.add(alice);
 
@@ -160,6 +164,8 @@ The guard starts with no permissions. Permit Jev to assert or deny the two incom
 predicates.
 
 ```ts
+import { guard } from "factbag/kb";
+
 const incomeEvidence = guard(kb)
   .permit("assert", hasIncomeOver200k)
   .permit("deny", hasIncomeOver200k)
@@ -170,7 +176,9 @@ const incomeEvidence = guard(kb)
 ### 6. Ask Jev to assess the income evidence
 
 ```ts
-const run = classifier(corpus, incomeEvidence);
+import { classifier } from "@factbag/jev";
+
+const run = classifier({ corpus, input: incomeEvidence });
 
 for (const job of run) {
   await job.execute(); // Makes a live Jev request.
@@ -183,6 +191,8 @@ The output knowledge base combines the imported fact and rules with the admitted
 evidence.
 
 ```ts
+import { query } from "factbag/dsl";
+
 const qualification = query(run.kb, isQualifiedInvestor({ person: alice }));
 const reviewReadiness = query(run.kb, isReadyForReview({ person: alice }));
 
@@ -282,7 +292,7 @@ Evidence judged superseded is retracted from the resolver's output
 knowledge base.
 
 ```ts
-import { resolver } from "factbag/jev/resolve";
+import { resolver } from "@factbag/jev";
 
 query(run.kb, isQualifiedInvestor({ person: alice })).value; // "conflicted"
 

package/dist/corpus.d.ts

@@ -0,0 +1,112 @@
+/** Application-defined metadata. Values may include native objects and functions. */
+type Metadata = Readonly<Record<string, unknown>>;
+/** A half-open range, measured in JavaScript UTF-16 code units. */
+type TextRange = Readonly<{
+  start: number;
+  end: number;
+}>;
+/** Location of the original artifact; text offsets refer to the stored text, not file bytes. */
+type DocumentSource = Readonly<{
+  uri: string;
+  mediaType?: string;
+}>;
+/** Text supplied by the caller, after any file extraction or normalization. */
+type DocumentInput<M extends Metadata = Metadata> = Readonly<{
+  documentId: string;
+  text: string;
+  title?: string;
+  source?: DocumentSource;
+  metadata?: M;
+}>;
+/** Identifies one immutable document. */
+type DocumentReference = Readonly<{
+  documentId: string;
+}>;
+/** An immutable document. Re-adding its ID with different content is rejected. */
+type CorpusDocument<M extends Metadata = Metadata> = DocumentReference & Readonly<{
+  text: string;
+  title?: string;
+  source?: DocumentSource;
+  /** Shallow snapshot; nested values retain identity and remain caller-owned. */
+  metadata: Readonly<Partial<M>>;
+}>;
+/** A stable source range; its ID depends only on the document and absolute range. */
+type ChunkReference = DocumentReference & Readonly<{
+  id: string;
+  range: TextRange;
+}>;
+/** A source range with text materialized for convenient consumption by models or UIs. */
+type Chunk = ChunkReference & Readonly<{
+  text: string;
+}>;
+/** A resolved chunk together with its complete document. */
+type ResolvedChunk<M extends Metadata = Metadata> = Readonly<{
+  chunk: Chunk;
+  document: CorpusDocument<M>;
+}>;
+/** Synchronous segmentation into source-ordered ranges relative to the supplied text. */
+type Chunker = Readonly<{
+  name: string;
+  split: (text: string) => Iterable<TextRange>;
+}>;
+/** Bounded recursive splitting; size and overlap use grapheme counts unless measure is supplied. */
+type RecursiveChunkOptions = Readonly<{
+  /** Positive integer maximum size of each emitted substring. */
+  size: number;
+  /** Desired maximum overlap, smaller than size. Defaults to zero. */
+  overlap?: number;
+  /** Deterministic nonnegative integer size of a substring, e.g. an exact tokenizer's count. */
+  measure?: (text: string) => number;
+  /** Preferred separators, in order. Defaults to paragraphs, lines, then words. */
+  separators?: readonly string[];
+}>;
+/**
+ * In-memory corpus of immutable documents and addressable source ranges.
+ * {@link Corpus.filter} returns views over the same storage; the filter is evaluated on read.
+ */
+export declare class Corpus<M extends Metadata = Metadata> {
+  #private;
+  /**
+   * Add a document to the shared storage; re-adding identical content is idempotent,
+   * different content throws. On a view, the document is visible only if it matches.
+   */
+  add(input: DocumentInput<M>): CorpusDocument<M>;
+  /** Validate every input, including conflicts, before storing any; then store them in input order. */
+  add(inputs: readonly DocumentInput<M>[]): readonly CorpusDocument<M>[];
+  /** Read a document. Throws when absent or excluded by this view's filter. */
+  document(documentId: string): CorpusDocument<M>;
+  /** List every document matching this view's filter, in insertion order. */
+  documents(): readonly CorpusDocument<M>[];
+  /**
+   * Return a view over the same storage that also requires this predicate. Documents
+   * added later through any view appear when they match.
+   */
+  filter(predicate: (document: CorpusDocument<M>) => boolean): Corpus<M>;
+  /**
+   * Split a document lazily, validating each range as it is consumed. If a later
+   * range is invalid, earlier yielded chunks remain valid and resolvable.
+   */
+  chunk(target: DocumentReference, chunker: Chunker): Iterable<Chunk>;
+  private splitDocument;
+  /** Select an exact range of a document. */
+  slice(target: DocumentReference, selection: TextRange): Chunk;
+  /**
+   * Resolve a chunk ID to its exact text and document. The ID encodes the document and
+   * range, so any valid ID resolves while its document exists; resolution ignores this
+   * view's filter. Throws for malformed IDs, unknown documents and invalid ranges.
+   */
+  resolve(chunkId: string): ResolvedChunk<M>;
+  private store;
+  private requireStored;
+  private requireVisible;
+}
+declare namespace chunkers_d_exports {
+  export { recursive };
+}
+/**
+ * Construct a bounded splitter that prefers paragraph, line, and word boundaries.
+ * Ranges are produced lazily and never split a Unicode grapheme. Custom measures
+ * are checked on exact substrings; a grapheme that cannot fit raises RangeError.
+ */
+declare function recursive(options: RecursiveChunkOptions): Chunker;
+export { type Chunk, type ChunkReference, type Chunker, type CorpusDocument, type DocumentInput, type DocumentReference, type DocumentSource, type Metadata, type RecursiveChunkOptions, type ResolvedChunk, type TextRange, chunkers_d_exports as chunkers };

package/dist/corpus.js

@@ -0,0 +1,266 @@
+import { t as __exportAll } from "./rolldown-runtime-D8jQpDG7.js";
+// ../corpus/src/records.ts
+function record(value) {
+	if (!isRecord(value)) throw new Error("Expected an object");
+	return value;
+}
+function isRecord(value) {
+	return typeof value === "object" && value !== null && !Array.isArray(value);
+}
+function string(value, name) {
+	if (typeof value !== "string") throw new Error(`${name} must be a string`);
+	return value;
+}
+function makeDocument(value) {
+	const input = record(value);
+	const documentId = string(input["documentId"], "documentId");
+	if (documentId.trim().length === 0) throw new Error("documentId must not be empty");
+	const rawMetadata = record(input["metadata"] ?? {});
+	if (Object.getPrototypeOf(rawMetadata) !== Object.prototype && Object.getPrototypeOf(rawMetadata) !== null) throw new Error("Metadata must be a plain object");
+	const metadata = value.metadata === void 0 ? {} : { ...value.metadata };
+	const rawSource = input["source"] === void 0 ? void 0 : record(input["source"]);
+	const source = rawSource === void 0 ? void 0 : Object.freeze({
+		uri: string(rawSource["uri"], "source.uri"),
+		...rawSource["mediaType"] === void 0 ? {} : { mediaType: string(rawSource["mediaType"], "source.mediaType") }
+	});
+	return Object.freeze({
+		documentId,
+		text: string(input["text"], "text"),
+		...input["title"] === void 0 ? {} : { title: string(input["title"], "title") },
+		...source === void 0 ? {} : { source },
+		metadata: Object.freeze(metadata)
+	});
+}
+function makeChunk(document, range) {
+	return Object.freeze({
+		id: `${encodeURIComponent(document.documentId)}#${range.start}-${range.end}`,
+		documentId: document.documentId,
+		range: Object.freeze({
+			start: range.start,
+			end: range.end
+		})
+	});
+}
+/** Parse a chunk ID back into its document and range. Only the canonical spelling is accepted. */
+function parseChunkId(id) {
+	const [, encoded, start, end] = /^([^#]+)#(0|[1-9]\d*)-(0|[1-9]\d*)$/u.exec(id) ?? [];
+	if (encoded === void 0 || start === void 0 || end === void 0) throw new Error(`Invalid chunk ID: ${id}`);
+	let documentId;
+	try {
+		documentId = decodeURIComponent(encoded);
+	} catch {
+		throw new Error(`Invalid chunk ID: ${id}`);
+	}
+	if (encodeURIComponent(documentId) !== encoded) throw new Error(`Invalid chunk ID: ${id}`);
+	return {
+		documentId,
+		range: {
+			start: Number(start),
+			end: Number(end)
+		}
+	};
+}
+// ../corpus/src/text.ts
+function positiveInteger(value, name) {
+	if (!Number.isSafeInteger(value) || value < 1) throw new Error(`${name} must be a positive safe integer`);
+}
+function nonnegativeInteger(value, name) {
+	if (!Number.isSafeInteger(value) || value < 0) throw new Error(`${name} must be a nonnegative safe integer`);
+}
+function graphemeBoundaries(text) {
+	const segmenter = new Intl.Segmenter("en", { granularity: "grapheme" });
+	return [...Array.from(segmenter.segment(text), (segment) => segment.index), text.length];
+}
+function validateRange(range, boundaries) {
+	if (!Number.isSafeInteger(range.start) || !Number.isSafeInteger(range.end) || range.start >= range.end || !boundaries.has(range.start) || !boundaries.has(range.end)) throw new Error("Range must be nonempty, within the text, and on grapheme boundaries");
+}
+function boundaryAt(boundaries, index) {
+	const boundary = boundaries[index];
+	if (boundary === void 0) throw new Error("Invalid text boundary index");
+	return boundary;
+}
+// ../corpus/src/corpus.ts
+/**
+* In-memory corpus of immutable documents and addressable source ranges.
+* {@link Corpus.filter} returns views over the same storage; the filter is evaluated on read.
+*/
+var Corpus = class Corpus {
+	#storage = { documents: /* @__PURE__ */ new Map() };
+	#filter = () => true;
+	add(input) {
+		if (!isBatch(input)) {
+			const document = makeDocument(input);
+			const existing = this.#storage.documents.get(document.documentId);
+			if (existing !== void 0) {
+				assertSameContent(existing, document);
+				return existing;
+			}
+			return this.store(document);
+		}
+		const pending = /* @__PURE__ */ new Map();
+		const documents = input.map((item) => {
+			const document = makeDocument(item);
+			const existing = pending.get(document.documentId) ?? this.#storage.documents.get(document.documentId);
+			if (existing !== void 0) {
+				assertSameContent(existing, document);
+				return existing;
+			}
+			pending.set(document.documentId, document);
+			return document;
+		});
+		for (const document of pending.values()) this.store(document);
+		return Object.freeze(documents);
+	}
+	/** Read a document. Throws when absent or excluded by this view's filter. */
+	document(documentId) {
+		return this.requireVisible({ documentId });
+	}
+	/** List every document matching this view's filter, in insertion order. */
+	documents() {
+		return Object.freeze([...this.#storage.documents.values()].filter(this.#filter));
+	}
+	/**
+	* Return a view over the same storage that also requires this predicate. Documents
+	* added later through any view appear when they match.
+	*/
+	filter(predicate) {
+		const parent = this.#filter;
+		const view = new Corpus();
+		view.#storage = this.#storage;
+		view.#filter = (document) => parent(document) && predicate(document);
+		return view;
+	}
+	/**
+	* Split a document lazily, validating each range as it is consumed. If a later
+	* range is invalid, earlier yielded chunks remain valid and resolvable.
+	*/
+	chunk(target, chunker) {
+		return this.splitDocument(this.requireVisible(target), chunker);
+	}
+	*splitDocument(document, chunker) {
+		const boundaries = new Set(graphemeBoundaries(document.text));
+		let previousStart = -1;
+		for (const range of chunker.split(document.text)) {
+			validateRange(range, boundaries);
+			if (range.start <= previousStart) throw new Error("Chunker ranges must advance in source order");
+			previousStart = range.start;
+			yield materialize(document, makeChunk(document, range));
+		}
+	}
+	/** Select an exact range of a document. */
+	slice(target, selection) {
+		const document = this.requireVisible(target);
+		validateRange(selection, new Set(graphemeBoundaries(document.text)));
+		return materialize(document, makeChunk(document, selection));
+	}
+	/**
+	* Resolve a chunk ID to its exact text and document. The ID encodes the document and
+	* range, so any valid ID resolves while its document exists; resolution ignores this
+	* view's filter. Throws for malformed IDs, unknown documents and invalid ranges.
+	*/
+	resolve(chunkId) {
+		const { documentId, range } = parseChunkId(chunkId);
+		const document = this.requireStored({ documentId });
+		validateRange(range, new Set(graphemeBoundaries(document.text)));
+		return Object.freeze({
+			chunk: materialize(document, makeChunk(document, range)),
+			document
+		});
+	}
+	store(document) {
+		this.#storage.documents.set(document.documentId, document);
+		return document;
+	}
+	requireStored(reference) {
+		const document = this.#storage.documents.get(reference.documentId);
+		if (document === void 0) throw new Error(`Unknown document: ${reference.documentId}`);
+		return document;
+	}
+	requireVisible(reference) {
+		const document = this.requireStored(reference);
+		if (!this.#filter(document)) throw new Error(`Document "${document.documentId}" is excluded by this corpus view`);
+		return document;
+	}
+};
+function materialize(document, reference) {
+	return Object.freeze({
+		...reference,
+		text: document.text.slice(reference.range.start, reference.range.end)
+	});
+}
+function assertSameContent(existing, document) {
+	const keys = Object.keys(existing.metadata);
+	if (existing.text !== document.text || existing.title !== document.title || existing.source?.uri !== document.source?.uri || existing.source?.mediaType !== document.source?.mediaType || keys.length !== Object.keys(document.metadata).length || !keys.every((key) => Object.hasOwn(document.metadata, key) && Object.is(existing.metadata[key], document.metadata[key]))) throw new Error(`Document "${document.documentId}" already exists with different content`);
+}
+function isBatch(input) {
+	return Array.isArray(input);
+}
+// ../corpus/src/chunkers.ts
+var chunkers_exports = /* @__PURE__ */ __exportAll({ recursive: () => recursive });
+/**
+* Construct a bounded splitter that prefers paragraph, line, and word boundaries.
+* Ranges are produced lazily and never split a Unicode grapheme. Custom measures
+* are checked on exact substrings; a grapheme that cannot fit raises RangeError.
+*/
+function recursive(options) {
+	const { size, overlap = 0, measure } = options;
+	const separators = [...options.separators ?? [
+		"\n\n",
+		"\n",
+		" "
+	]];
+	positiveInteger(size, "size");
+	nonnegativeInteger(overlap, "overlap");
+	if (overlap >= size) throw new RangeError("overlap must be smaller than size");
+	if (separators.some((separator) => separator.length === 0)) throw new Error("Separators must be nonempty; grapheme fallback is automatic");
+	return Object.freeze({
+		name: "recursive",
+		*split(text) {
+			const boundaries = graphemeBoundaries(text);
+			const indices = new Map(boundaries.map((offset, index) => [offset, index]));
+			const last = boundaries.length - 1;
+			function count(start, end) {
+				if (measure === void 0) return end - start;
+				const value = measure(text.slice(boundaryAt(boundaries, start), boundaryAt(boundaries, end)));
+				if (!Number.isSafeInteger(value) || value < 0) throw new RangeError("measure must return a nonnegative safe integer");
+				return value;
+			}
+			function boundedEnd(start) {
+				if (measure === void 0) return Math.min(start + size, last);
+				let end = start;
+				while (end < last && count(start, end + 1) <= size) end += 1;
+				if (end === start) throw new RangeError("A single grapheme exceeds the chunk size");
+				return end;
+			}
+			function preferredEnd(start, hardEnd) {
+				if (hardEnd === last) return hardEnd;
+				const startOffset = boundaryAt(boundaries, start);
+				const endOffset = boundaryAt(boundaries, hardEnd);
+				for (const separator of separators) {
+					const offset = text.lastIndexOf(separator, endOffset - separator.length);
+					const candidate = indices.get(offset + separator.length);
+					if (offset >= startOffset && candidate !== void 0 && candidate > start && candidate <= hardEnd) return candidate;
+				}
+				return hardEnd;
+			}
+			function nextStart(start, end) {
+				if (overlap === 0) return end;
+				if (measure === void 0) return Math.max(start + 1, end - overlap);
+				let next = end;
+				while (next > start + 1 && count(next - 1, end) <= overlap) next -= 1;
+				return next;
+			}
+			let start = 0;
+			while (start < last) {
+				const end = preferredEnd(start, boundedEnd(start));
+				yield {
+					start: boundaryAt(boundaries, start),
+					end: boundaryAt(boundaries, end)
+				};
+				if (end === last) return;
+				start = nextStart(start, end);
+			}
+		}
+	});
+}
+export { Corpus, chunkers_exports as chunkers };