factbag 0.0.0-alpha.4 → 0.0.0-alpha.6

package/CHANGELOG.md

@@ -1,5 +1,37 @@
 # factbag
 
+## 0.0.0-alpha.6
+
+### Minor Changes
+
+- 7225a51: Add `firstOfMonth(year, month)`, a date-valued arithmetic term for the first day of a month at UTC midnight, and `yearOf(date)`, a number-valued term for a date's UTC calendar year. Rules can now derive a year's bounds, such as `firstOfMonth(Y, 1)` and `sub(firstOfMonth(add(Y, 1), 1), 86_400_000)`, instead of looking them up in asserted calendar facts. A month outside 1 to 12, or a year that is not a whole number, throws `PrimitiveArithmeticProgrammerError` rather than rolling over. The query API adds `FirstOfMonthGoal` and the `"firstOfMonth"` arithmetic kind.
+- 5114b2f: Show the `citation` and `justification` recorded with a piece of evidence in its explanation, beside its `source`. An evidence reason no longer repeats the claim of the step it supports; it names its own claim only when that differs, as in a conflict, or when the evidence has no provenance. `EvidenceReason.quote`, which was never filled, is replaced by `citation`.
+- eab3679: Explain a rule application with the rule alone, `eligible(P) iff not(active_sanction(P))`, without the trailing `, with P = alice`. The values a rule was applied to are already written into the claim it supports and the claims it rests on.
+- e4b343e: Add a `budget` option to `explainJudgment`: the most claims to explain. The conclusion is always explained, then the claims that decided it before those that were evaluated and changed nothing, and those near the conclusion before those far from it. A claim that rests on claims left out names them, with their values, under `notShown`. Without a budget the whole proof is explained, as before; written as JSON a deep proof repeats each shared claim on every path to it, so give a budget when explaining a judgment in text.
+
+### Patch Changes
+
+- 7f26b45: Write explanation claims with the values their variables hold: `lte(1000, 36900) = true` instead of `lte(I, 36900) = true`, and `spouse(alice, S) = false` instead of `spouse(P, S) = false`. A variable with no value keeps its name. A term that computes a value states it, such as `H = max(1000, 500) = 1000`, instead of ending in `= true`.
+- b7c85ff: Explain a comparison, arithmetic term or aggregate as `computed` instead of `no supporting evidence or rule was found`. When one could not be evaluated because a variable has no value, the explanation names the variable, such as `X has no value`.
+- ed6ec84: Write a date at UTC midnight as a day in explanations, `2015-12-31` instead of `2015-12-31T00:00:00.000Z`. A date with a time of day keeps its full timestamp.
+- 15391f2: Rest a rule application directly on the subgoals of its body in explanations. When a rule's body is an `and` or an `or`, the explanation no longer has a node that restates the body between the rule and its subgoals: the rule's `given` lists the subgoals themselves. A compound nested inside a body is still its own node, and so is a body that is a single atom or a negation.
+- ecbd8d9: Stop naming the solver's temporaries, such as `$tmp0`, in explanations. A nested arithmetic term is written as the term and its value, `mul(1000, 0.15) = 150`; a term that could not be computed stands in for its result wherever that is used, `gte(add(X, 1), 18) = unknown`; and a rule's `with ...` list gives only the rule's own variables.
+
+## 0.0.0-alpha.5
+
+### Minor Changes
+
+- 19ac508: **Breaking:** errors have their own entry point, `factbag/exceptions`. `ProgrammerError`, `ExclusiveDefinitionProgrammerError`, `PrimitiveArithmeticProgrammerError`, and `UnsafeUnboundVariableProgrammerError` move there from `factbag/kb`. `UnconstructiveRecursionProgrammerError`, which the solver raised but no entry point exported, is now exported too. `UnsupportedPredicateError` moves there from `factbag/jev/errors` and now extends `ProgrammerError`.
+- 19ac508: **Breaking:** Jev moves from `factbag/jev` to its own package, `@factbag/jev`. Install it alongside `factbag` and `@typesafe-ai/sdk`, which is now a required peer dependency at `^0.7.0`, and import `classifier`, `resolver`, `createConfig`, `Config`, and the `DEFAULT_*` constants from `@factbag/jev`. `factbag` no longer has a Jev entry point or an SDK peer dependency. `@factbag/jev` and `@factbag/clef` are released together with `factbag` and require the same `factbag` version.
+
+  **Breaking:** `@factbag/jev` has a single entry point. `factbag/jev/classifier`, `factbag/jev/resolve`, `factbag/jev/config`, and `factbag/jev/errors` have no replacement subpaths.
+
+  Add `batchSize` to Jev's `Config`: how many questions each request holds. It defaults to 64 and has no upper limit. An invalid `batchSize` throws when the config is created or a run is built.
+
+  Jev's `createConfig` now returns `Normalized & Config`, so `client` is no longer guaranteed on the result.
+
+- dd3730b: **Breaking:** `factbag/guards` and `factbag/query` are removed. Import `guard`, `GuardedKB`, and the other guard exports from `factbag/kb`, and `query`, `queryAll`, `Judgment`, and the goal and support types from `factbag/dsl`.
+
 ## 0.0.0-alpha.4
 
 ### Minor Changes

package/README.md

@@ -32,9 +32,13 @@ the rules again.
 ## Install
 
 ```sh
-npm install factbag@alpha @typesafe-ai/sdk@^0.6.0
+npm install factbag@alpha @factbag/jev@alpha @typesafe-ai/sdk@^0.7.0
 ```
 
+`@factbag/jev` is released with `factbag` and needs the same version. To use
+Cloudflare's Clef models on Workers AI instead, install `@factbag/clef@alpha`, which
+does not need the TypeSafe SDK.
+
 ESM only. Requires Node 22.18+ on the 22.x line, or Node 24 or 26.
 
 Set your TypeSafe API key in the environment:
@@ -63,12 +67,7 @@ into three conditions:
 Declare the facts we'll combine, and describe the claims Jev will assess.
 
 ```ts
-import { and, enumeration, predicate, variable } from "factbag/dsl";
-import { KnowledgeBase } from "factbag/kb";
-import { query } from "factbag/query";
-import { Corpus } from "factbag/corpus";
-import { guard } from "factbag/guards";
-import { classifier } from "factbag/jev/classifier";
+import { enumeration, predicate } from "factbag/dsl";
 
 class Person {
   constructor(
@@ -112,6 +111,9 @@ All three income conditions must hold for this route to qualification. Our
 application also requires verified identity before sending an investor for review.
 
 ```ts
+import { and, variable } from "factbag/dsl";
+import { KnowledgeBase } from "factbag/kb";
+
 const person = variable("person");
 const kb = new KnowledgeBase();
 
@@ -134,6 +136,8 @@ Identity verification comes directly from our onboarding system. Income evidence
 goes into the corpus for Jev to assess.
 
 ```ts
+import { Corpus } from "factbag/corpus";
+
 const alice = new Person("alice", "Alice");
 kb.entities.add(alice);
 
@@ -160,6 +164,8 @@ The guard starts with no permissions. Permit Jev to assert or deny the two incom
 predicates.
 
 ```ts
+import { guard } from "factbag/kb";
+
 const incomeEvidence = guard(kb)
   .permit("assert", hasIncomeOver200k)
   .permit("deny", hasIncomeOver200k)
@@ -170,6 +176,8 @@ const incomeEvidence = guard(kb)
 ### 6. Ask Jev to assess the income evidence
 
 ```ts
+import { classifier } from "@factbag/jev";
+
 const run = classifier({ corpus, input: incomeEvidence });
 
 for (const job of run) {
@@ -183,6 +191,8 @@ The output knowledge base combines the imported fact and rules with the admitted
 evidence.
 
 ```ts
+import { query } from "factbag/dsl";
+
 const qualification = query(run.kb, isQualifiedInvestor({ person: alice }));
 const reviewReadiness = query(run.kb, isReadyForReview({ person: alice }));
 
@@ -211,33 +221,25 @@ An abbreviated example with supporting evidence for all three claims:
 ```json
 {
   "claim": "is_qualified_investor(alice) = true",
-  "because": "is_qualified_investor(person) if and(...), with person = alice",
+  "because": "is_qualified_investor(person) if and(...)",
   "given": [
     {
-      "claim": "and(...) = true",
-      "given": [
-        {
-          "claim": "has_income_over_200k(alice, 2024) = true",
-          "because": {
-            "claim": "has_income_over_200k(alice, 2024) = true",
-            "source": "alice-2024-federal-tax-return#0-1"
-          }
-        },
-        {
-          "claim": "has_income_over_200k(alice, 2025) = true",
-          "because": {
-            "claim": "has_income_over_200k(alice, 2025) = true",
-            "source": "alice-2025-federal-tax-return#0-1"
-          }
-        },
-        {
-          "claim": "expects_same_income(alice) = true",
-          "because": {
-            "claim": "expects_same_income(alice) = true",
-            "source": "alice-2026-employment-confirmation-letter#0-1"
-          }
-        }
-      ]
+      "claim": "has_income_over_200k(alice, 2024) = true",
+      "because": {
+        "source": "alice-2024-federal-tax-return#0-1"
+      }
+    },
+    {
+      "claim": "has_income_over_200k(alice, 2025) = true",
+      "because": {
+        "source": "alice-2025-federal-tax-return#0-1"
+      }
+    },
+    {
+      "claim": "expects_same_income(alice) = true",
+      "because": {
+        "source": "alice-2026-employment-confirmation-letter#0-1"
+      }
     }
   ]
 }
@@ -282,7 +284,7 @@ Evidence judged superseded is retracted from the resolver's output
 knowledge base.
 
 ```ts
-import { resolver } from "factbag/jev/resolve";
+import { resolver } from "@factbag/jev";
 
 query(run.kb, isQualifiedInvestor({ person: alice })).value; // "conflicted"
 

package/dist/corpus.d.ts

@@ -0,0 +1,112 @@
+/** Application-defined metadata. Values may include native objects and functions. */
+type Metadata = Readonly<Record<string, unknown>>;
+/** A half-open range, measured in JavaScript UTF-16 code units. */
+type TextRange = Readonly<{
+  start: number;
+  end: number;
+}>;
+/** Location of the original artifact; text offsets refer to the stored text, not file bytes. */
+type DocumentSource = Readonly<{
+  uri: string;
+  mediaType?: string;
+}>;
+/** Text supplied by the caller, after any file extraction or normalization. */
+type DocumentInput<M extends Metadata = Metadata> = Readonly<{
+  documentId: string;
+  text: string;
+  title?: string;
+  source?: DocumentSource;
+  metadata?: M;
+}>;
+/** Identifies one immutable document. */
+type DocumentReference = Readonly<{
+  documentId: string;
+}>;
+/** An immutable document. Re-adding its ID with different content is rejected. */
+type CorpusDocument<M extends Metadata = Metadata> = DocumentReference & Readonly<{
+  text: string;
+  title?: string;
+  source?: DocumentSource;
+  /** Shallow snapshot; nested values retain identity and remain caller-owned. */
+  metadata: Readonly<Partial<M>>;
+}>;
+/** A stable source range; its ID depends only on the document and absolute range. */
+type ChunkReference = DocumentReference & Readonly<{
+  id: string;
+  range: TextRange;
+}>;
+/** A source range with text materialized for convenient consumption by models or UIs. */
+type Chunk = ChunkReference & Readonly<{
+  text: string;
+}>;
+/** A resolved chunk together with its complete document. */
+type ResolvedChunk<M extends Metadata = Metadata> = Readonly<{
+  chunk: Chunk;
+  document: CorpusDocument<M>;
+}>;
+/** Synchronous segmentation into source-ordered ranges relative to the supplied text. */
+type Chunker = Readonly<{
+  name: string;
+  split: (text: string) => Iterable<TextRange>;
+}>;
+/** Bounded recursive splitting; size and overlap use grapheme counts unless measure is supplied. */
+type RecursiveChunkOptions = Readonly<{
+  /** Positive integer maximum size of each emitted substring. */
+  size: number;
+  /** Desired maximum overlap, smaller than size. Defaults to zero. */
+  overlap?: number;
+  /** Deterministic nonnegative integer size of a substring, e.g. an exact tokenizer's count. */
+  measure?: (text: string) => number;
+  /** Preferred separators, in order. Defaults to paragraphs, lines, then words. */
+  separators?: readonly string[];
+}>;
+/**
+ * In-memory corpus of immutable documents and addressable source ranges.
+ * {@link Corpus.filter} returns views over the same storage; the filter is evaluated on read.
+ */
+export declare class Corpus<M extends Metadata = Metadata> {
+  #private;
+  /**
+   * Add a document to the shared storage; re-adding identical content is idempotent,
+   * different content throws. On a view, the document is visible only if it matches.
+   */
+  add(input: DocumentInput<M>): CorpusDocument<M>;
+  /** Validate every input, including conflicts, before storing any; then store them in input order. */
+  add(inputs: readonly DocumentInput<M>[]): readonly CorpusDocument<M>[];
+  /** Read a document. Throws when absent or excluded by this view's filter. */
+  document(documentId: string): CorpusDocument<M>;
+  /** List every document matching this view's filter, in insertion order. */
+  documents(): readonly CorpusDocument<M>[];
+  /**
+   * Return a view over the same storage that also requires this predicate. Documents
+   * added later through any view appear when they match.
+   */
+  filter(predicate: (document: CorpusDocument<M>) => boolean): Corpus<M>;
+  /**
+   * Split a document lazily, validating each range as it is consumed. If a later
+   * range is invalid, earlier yielded chunks remain valid and resolvable.
+   */
+  chunk(target: DocumentReference, chunker: Chunker): Iterable<Chunk>;
+  private splitDocument;
+  /** Select an exact range of a document. */
+  slice(target: DocumentReference, selection: TextRange): Chunk;
+  /**
+   * Resolve a chunk ID to its exact text and document. The ID encodes the document and
+   * range, so any valid ID resolves while its document exists; resolution ignores this
+   * view's filter. Throws for malformed IDs, unknown documents and invalid ranges.
+   */
+  resolve(chunkId: string): ResolvedChunk<M>;
+  private store;
+  private requireStored;
+  private requireVisible;
+}
+declare namespace chunkers_d_exports {
+  export { recursive };
+}
+/**
+ * Construct a bounded splitter that prefers paragraph, line, and word boundaries.
+ * Ranges are produced lazily and never split a Unicode grapheme. Custom measures
+ * are checked on exact substrings; a grapheme that cannot fit raises RangeError.
+ */
+declare function recursive(options: RecursiveChunkOptions): Chunker;
+export { type Chunk, type ChunkReference, type Chunker, type CorpusDocument, type DocumentInput, type DocumentReference, type DocumentSource, type Metadata, type RecursiveChunkOptions, type ResolvedChunk, type TextRange, chunkers_d_exports as chunkers };

package/dist/corpus.js

@@ -0,0 +1,266 @@
+import { t as __exportAll } from "./rolldown-runtime-D8jQpDG7.js";
+// ../corpus/src/records.ts
+function record(value) {
+	if (!isRecord(value)) throw new Error("Expected an object");
+	return value;
+}
+function isRecord(value) {
+	return typeof value === "object" && value !== null && !Array.isArray(value);
+}
+function string(value, name) {
+	if (typeof value !== "string") throw new Error(`${name} must be a string`);
+	return value;
+}
+function makeDocument(value) {
+	const input = record(value);
+	const documentId = string(input["documentId"], "documentId");
+	if (documentId.trim().length === 0) throw new Error("documentId must not be empty");
+	const rawMetadata = record(input["metadata"] ?? {});
+	if (Object.getPrototypeOf(rawMetadata) !== Object.prototype && Object.getPrototypeOf(rawMetadata) !== null) throw new Error("Metadata must be a plain object");
+	const metadata = value.metadata === void 0 ? {} : { ...value.metadata };
+	const rawSource = input["source"] === void 0 ? void 0 : record(input["source"]);
+	const source = rawSource === void 0 ? void 0 : Object.freeze({
+		uri: string(rawSource["uri"], "source.uri"),
+		...rawSource["mediaType"] === void 0 ? {} : { mediaType: string(rawSource["mediaType"], "source.mediaType") }
+	});
+	return Object.freeze({
+		documentId,
+		text: string(input["text"], "text"),
+		...input["title"] === void 0 ? {} : { title: string(input["title"], "title") },
+		...source === void 0 ? {} : { source },
+		metadata: Object.freeze(metadata)
+	});
+}
+function makeChunk(document, range) {
+	return Object.freeze({
+		id: `${encodeURIComponent(document.documentId)}#${range.start}-${range.end}`,
+		documentId: document.documentId,
+		range: Object.freeze({
+			start: range.start,
+			end: range.end
+		})
+	});
+}
+/** Parse a chunk ID back into its document and range. Only the canonical spelling is accepted. */
+function parseChunkId(id) {
+	const [, encoded, start, end] = /^([^#]+)#(0|[1-9]\d*)-(0|[1-9]\d*)$/u.exec(id) ?? [];
+	if (encoded === void 0 || start === void 0 || end === void 0) throw new Error(`Invalid chunk ID: ${id}`);
+	let documentId;
+	try {
+		documentId = decodeURIComponent(encoded);
+	} catch {
+		throw new Error(`Invalid chunk ID: ${id}`);
+	}
+	if (encodeURIComponent(documentId) !== encoded) throw new Error(`Invalid chunk ID: ${id}`);
+	return {
+		documentId,
+		range: {
+			start: Number(start),
+			end: Number(end)
+		}
+	};
+}
+// ../corpus/src/text.ts
+function positiveInteger(value, name) {
+	if (!Number.isSafeInteger(value) || value < 1) throw new Error(`${name} must be a positive safe integer`);
+}
+function nonnegativeInteger(value, name) {
+	if (!Number.isSafeInteger(value) || value < 0) throw new Error(`${name} must be a nonnegative safe integer`);
+}
+function graphemeBoundaries(text) {
+	const segmenter = new Intl.Segmenter("en", { granularity: "grapheme" });
+	return [...Array.from(segmenter.segment(text), (segment) => segment.index), text.length];
+}
+function validateRange(range, boundaries) {
+	if (!Number.isSafeInteger(range.start) || !Number.isSafeInteger(range.end) || range.start >= range.end || !boundaries.has(range.start) || !boundaries.has(range.end)) throw new Error("Range must be nonempty, within the text, and on grapheme boundaries");
+}
+function boundaryAt(boundaries, index) {
+	const boundary = boundaries[index];
+	if (boundary === void 0) throw new Error("Invalid text boundary index");
+	return boundary;
+}
+// ../corpus/src/corpus.ts
+/**
+* In-memory corpus of immutable documents and addressable source ranges.
+* {@link Corpus.filter} returns views over the same storage; the filter is evaluated on read.
+*/
+var Corpus = class Corpus {
+	#storage = { documents: /* @__PURE__ */ new Map() };
+	#filter = () => true;
+	add(input) {
+		if (!isBatch(input)) {
+			const document = makeDocument(input);
+			const existing = this.#storage.documents.get(document.documentId);
+			if (existing !== void 0) {
+				assertSameContent(existing, document);
+				return existing;
+			}
+			return this.store(document);
+		}
+		const pending = /* @__PURE__ */ new Map();
+		const documents = input.map((item) => {
+			const document = makeDocument(item);
+			const existing = pending.get(document.documentId) ?? this.#storage.documents.get(document.documentId);
+			if (existing !== void 0) {
+				assertSameContent(existing, document);
+				return existing;
+			}
+			pending.set(document.documentId, document);
+			return document;
+		});
+		for (const document of pending.values()) this.store(document);
+		return Object.freeze(documents);
+	}
+	/** Read a document. Throws when absent or excluded by this view's filter. */
+	document(documentId) {
+		return this.requireVisible({ documentId });
+	}
+	/** List every document matching this view's filter, in insertion order. */
+	documents() {
+		return Object.freeze([...this.#storage.documents.values()].filter(this.#filter));
+	}
+	/**
+	* Return a view over the same storage that also requires this predicate. Documents
+	* added later through any view appear when they match.
+	*/
+	filter(predicate) {
+		const parent = this.#filter;
+		const view = new Corpus();
+		view.#storage = this.#storage;
+		view.#filter = (document) => parent(document) && predicate(document);
+		return view;
+	}
+	/**
+	* Split a document lazily, validating each range as it is consumed. If a later
+	* range is invalid, earlier yielded chunks remain valid and resolvable.
+	*/
+	chunk(target, chunker) {
+		return this.splitDocument(this.requireVisible(target), chunker);
+	}
+	*splitDocument(document, chunker) {
+		const boundaries = new Set(graphemeBoundaries(document.text));
+		let previousStart = -1;
+		for (const range of chunker.split(document.text)) {
+			validateRange(range, boundaries);
+			if (range.start <= previousStart) throw new Error("Chunker ranges must advance in source order");
+			previousStart = range.start;
+			yield materialize(document, makeChunk(document, range));
+		}
+	}
+	/** Select an exact range of a document. */
+	slice(target, selection) {
+		const document = this.requireVisible(target);
+		validateRange(selection, new Set(graphemeBoundaries(document.text)));
+		return materialize(document, makeChunk(document, selection));
+	}
+	/**
+	* Resolve a chunk ID to its exact text and document. The ID encodes the document and
+	* range, so any valid ID resolves while its document exists; resolution ignores this
+	* view's filter. Throws for malformed IDs, unknown documents and invalid ranges.
+	*/
+	resolve(chunkId) {
+		const { documentId, range } = parseChunkId(chunkId);
+		const document = this.requireStored({ documentId });
+		validateRange(range, new Set(graphemeBoundaries(document.text)));
+		return Object.freeze({
+			chunk: materialize(document, makeChunk(document, range)),
+			document
+		});
+	}
+	store(document) {
+		this.#storage.documents.set(document.documentId, document);
+		return document;
+	}
+	requireStored(reference) {
+		const document = this.#storage.documents.get(reference.documentId);
+		if (document === void 0) throw new Error(`Unknown document: ${reference.documentId}`);
+		return document;
+	}
+	requireVisible(reference) {
+		const document = this.requireStored(reference);
+		if (!this.#filter(document)) throw new Error(`Document "${document.documentId}" is excluded by this corpus view`);
+		return document;
+	}
+};
+function materialize(document, reference) {
+	return Object.freeze({
+		...reference,
+		text: document.text.slice(reference.range.start, reference.range.end)
+	});
+}
+function assertSameContent(existing, document) {
+	const keys = Object.keys(existing.metadata);
+	if (existing.text !== document.text || existing.title !== document.title || existing.source?.uri !== document.source?.uri || existing.source?.mediaType !== document.source?.mediaType || keys.length !== Object.keys(document.metadata).length || !keys.every((key) => Object.hasOwn(document.metadata, key) && Object.is(existing.metadata[key], document.metadata[key]))) throw new Error(`Document "${document.documentId}" already exists with different content`);
+}
+function isBatch(input) {
+	return Array.isArray(input);
+}
+// ../corpus/src/chunkers.ts
+var chunkers_exports = /* @__PURE__ */ __exportAll({ recursive: () => recursive });
+/**
+* Construct a bounded splitter that prefers paragraph, line, and word boundaries.
+* Ranges are produced lazily and never split a Unicode grapheme. Custom measures
+* are checked on exact substrings; a grapheme that cannot fit raises RangeError.
+*/
+function recursive(options) {
+	const { size, overlap = 0, measure } = options;
+	const separators = [...options.separators ?? [
+		"\n\n",
+		"\n",
+		" "
+	]];
+	positiveInteger(size, "size");
+	nonnegativeInteger(overlap, "overlap");
+	if (overlap >= size) throw new RangeError("overlap must be smaller than size");
+	if (separators.some((separator) => separator.length === 0)) throw new Error("Separators must be nonempty; grapheme fallback is automatic");
+	return Object.freeze({
+		name: "recursive",
+		*split(text) {
+			const boundaries = graphemeBoundaries(text);
+			const indices = new Map(boundaries.map((offset, index) => [offset, index]));
+			const last = boundaries.length - 1;
+			function count(start, end) {
+				if (measure === void 0) return end - start;
+				const value = measure(text.slice(boundaryAt(boundaries, start), boundaryAt(boundaries, end)));
+				if (!Number.isSafeInteger(value) || value < 0) throw new RangeError("measure must return a nonnegative safe integer");
+				return value;
+			}
+			function boundedEnd(start) {
+				if (measure === void 0) return Math.min(start + size, last);
+				let end = start;
+				while (end < last && count(start, end + 1) <= size) end += 1;
+				if (end === start) throw new RangeError("A single grapheme exceeds the chunk size");
+				return end;
+			}
+			function preferredEnd(start, hardEnd) {
+				if (hardEnd === last) return hardEnd;
+				const startOffset = boundaryAt(boundaries, start);
+				const endOffset = boundaryAt(boundaries, hardEnd);
+				for (const separator of separators) {
+					const offset = text.lastIndexOf(separator, endOffset - separator.length);
+					const candidate = indices.get(offset + separator.length);
+					if (offset >= startOffset && candidate !== void 0 && candidate > start && candidate <= hardEnd) return candidate;
+				}
+				return hardEnd;
+			}
+			function nextStart(start, end) {
+				if (overlap === 0) return end;
+				if (measure === void 0) return Math.max(start + 1, end - overlap);
+				let next = end;
+				while (next > start + 1 && count(next - 1, end) <= overlap) next -= 1;
+				return next;
+			}
+			let start = 0;
+			while (start < last) {
+				const end = preferredEnd(start, boundedEnd(start));
+				yield {
+					start: boundaryAt(boundaries, start),
+					end: boundaryAt(boundaries, end)
+				};
+				if (end === last) return;
+				start = nextStart(start, end);
+			}
+		}
+	});
+}
+export { Corpus, chunkers_exports as chunkers };