@absolutejs/rag 0.18.0 → 0.19.0

package/CHANGELOG.md

@@ -6,6 +6,22 @@ This file is generated by `absolute-changelog` from the entries in
 `changelog/`. Edit an entry, not this file — and add new ones under
 `changelog/unreleased/`.
 
+## 0.19.0 — 2026-09-24
+
+### Added
+
+- **Allow a separately configured review provider and model through the existing host accounting path.** (`ResearchConfig`)
+
+### Fixed
+
+- **Require relevance, entity, time and scope checks before supporting research fields; carry task criteria into review, bound default findings and share evidence space with follow-up sources. Resolve model-selected passage IDs to exact source text instead of asking models to transcribe quotes.** (`bindResearchReview`, `ResearchField`, `createResearch`)
+
+## 0.18.1 — 2026-09-24
+
+### Fixed
+
+- **Accept Elysia 2 and TypeBox 1 research schemas with inferred result types and matching runtime validation while preserving TypeBox 0.34 compatibility.** (`ResearchTask`, `ResearchRuntime`, `ResearchSchema`, `ResearchStatic`, `createResearch`)
+
 ## 0.18.0 — 2026-09-24
 
 ### Added

package/RESEARCH.md

@@ -5,8 +5,7 @@
 ## Configure once
 
 ```ts
-import { Elysia } from "elysia";
-import { Type } from "@sinclair/typebox";
+import { Elysia, t } from "elysia";
 import { anthropic } from "@absolutejs/ai/anthropic";
 import { createBraveSearch } from "@absolutejs/search/brave";
 import {
@@ -21,7 +20,7 @@ const research = createResearch({
   model: process.env.RESEARCH_MODEL!,
   tasks: {
     company: {
-      schema: Type.Object({ name: Type.String(), description: Type.String() }),
+      schema: t.Object({ name: t.String(), description: t.String() }),
     },
     signals: researchMonitorTask,
   },
@@ -45,7 +44,7 @@ const result = await research.run({
 
 `authorizeResearchRequest` is your application's authorization function, not an export of this package. It is required. For multiple tenants, pass `runtime: request => runtimeForAuthorizedTenant(request)`; authorization runs first. Scope tenant caches, workflows and budget ledgers to the same resolved identity. The plugin rejects cross-origin browser POSTs and sends `Cache-Control: no-store`. Do not treat a caller-supplied tenant header as identity.
 
-`run` selects server-registered schemas; browsers cannot override models, limits, schemas or credentials. `extract({schema, instructions}, input)` accepts a typed TypeBox schema directly for trusted server code. The same configured AI provider handles planning, extraction and independent review, with transport retries and schema repair retries disabled.
+`run` selects server-registered schemas; browsers cannot override models, limits, schemas or credentials. `extract({schema, instructions}, input)` accepts JSON-compatible Elysia 2 `t`, TypeBox 1 (`typebox`), or legacy TypeBox 0.34 (`@sinclair/typebox`) schemas directly, preserving result type inference and runtime validation for trusted server code. The same configured AI provider handles planning, extraction and independent review, with transport retries and schema repair retries disabled.
 
 Default limits are four searches, four page reads, two planning rounds, 90 seconds, 32,000 evidence characters, 4,096 output tokens per model call and 128 primitive output fields. Limits are per research call, not per entire batch. Page reading uses the existing hardened public web transport; optional `render` enables your existing browser service. A browser process is not provisioned automatically. Cancellation must also be honored by custom providers and readers.
 
@@ -267,3 +266,42 @@ const mcpTools = toMcpToolRegistry(researchManifest, {
 Use `evaluateResearch` to run representative cases with optional independent human review and total measured cost. Unreviewed acceptance, unknown cost, and cost per accepted finding without accepted findings remain `null`. Publication freshness measures source publication dates; it is not event freshness.
 
 Each result includes `groundingCase`, accepted by the existing `evaluateRAGAnswerGrounding({cases})` function from `@absolutejs/rag/quality`. Citation resolution metrics do not establish factual correctness or discovery recall. Keep approved ground truth and reviewer identity with comparisons, and hold task/model/budget policy constant when switching search providers.
+
+### Research quality and reviewer configuration
+
+Default research returns at most six concise findings. Every supported field must
+have source-bound quotes and positive model checks for task relevance, entity,
+time and scope. Missing or failed checks withhold support even when a quote
+matches. `bindResearchReview` callers must now supply `checks` to promote a field.
+These checks are fallible model judgments, not human or independent verification.
+Task instructions and schema are supplied to both extraction and review.
+Internally, the reviewer selects numbered source passages and the runtime copies
+their text verbatim into citations. Invented or ambiguous references fail closed;
+this removes quote transcription errors without establishing factual entailment.
+
+The evidence budget is shared across sources instead of consumed entirely by
+first results. Fetched pages replace their snippets and receive four times the
+allocation weight of snippet-only sources; short sources donate unused space.
+This preserves more page context while reserving room for follow-up retrieval.
+A maximum of 64 unique sources bounds retained evidence. Truncation can still
+omit necessary context; inspect limitations and field verdicts.
+
+A separate reviewer can be configured without changing planning or extraction:
+
+```ts
+const research = createResearch({
+  search,
+  provider,
+  model: extractionModel,
+  reviewer: { provider: reviewProvider, model: reviewModel },
+});
+```
+
+Omitting `reviewer` uses the main provider/model. The host generation adapter and
+admission/settlement path still apply to review calls. Price review reservations
+and usage with the configured reviewer; do not apply the extraction model's rate
+to a different model. A stronger reviewer does not repair missing retrieval.
+
+An invalid planner schema no longer discards retrieved evidence: extraction and
+review continue without retry, while the result remains partial and the failed
+plan retains unknown settlement. Budget denial and cancellation still stop work.

package/changelog.json

@@ -2,6 +2,45 @@
 	"contract": 1,
 	"name": "@absolutejs/rag",
 	"releases": [
+		{
+			"changes": [
+				{
+					"kind": "added",
+					"summary": "Allow a separately configured review provider and model through the existing host accounting path.",
+					"symbols": [
+						"ResearchConfig"
+					]
+				},
+				{
+					"kind": "fixed",
+					"summary": "Require relevance, entity, time and scope checks before supporting research fields; carry task criteria into review, bound default findings and share evidence space with follow-up sources. Resolve model-selected passage IDs to exact source text instead of asking models to transcribe quotes.",
+					"symbols": [
+						"bindResearchReview",
+						"ResearchField",
+						"createResearch"
+					]
+				}
+			],
+			"date": "2026-09-24",
+			"version": "0.19.0"
+		},
+		{
+			"changes": [
+				{
+					"kind": "fixed",
+					"summary": "Accept Elysia 2 and TypeBox 1 research schemas with inferred result types and matching runtime validation while preserving TypeBox 0.34 compatibility.",
+					"symbols": [
+						"ResearchTask",
+						"ResearchRuntime",
+						"ResearchSchema",
+						"ResearchStatic",
+						"createResearch"
+					]
+				}
+			],
+			"date": "2026-09-24",
+			"version": "0.18.1"
+		},
 		{
 			"changes": [
 				{

package/dist/research/index.js

@@ -124126,14 +124126,31 @@ __export(exports_value2, {
   Patch: () => Patch,
   ValueErrorIterator: () => ValueErrorIterator
 });
+// src/research/schema.ts
+import { Check as Check2 } from "typebox/value";
+var isLegacySchema = (schema) => (Kind in schema);
+var checkResearchValue = (schema, value) => {
+  if (isLegacySchema(schema))
+    return exports_value2.Check(schema, value);
+  return Check2(schema, value);
+};
+
 // src/research/evidence.ts
 import { sourceSupportsQuote } from "@absolutejs/search";
 var ReviewSchema = Type2.Object({
   fields: Type2.Array(Type2.Object({
     path: Type2.String(),
     verdict: Type2.Union(["supported", "unsupported", "conflicting", "unknown"].map((value) => Type2.Literal(value))),
-    citations: Type2.Array(Type2.Object({ sourceId: Type2.String(), quote: Type2.String() }), { maxItems: 8 }),
-    reason: Type2.String()
+    citations: Type2.Array(Type2.Object({ passageId: Type2.String() }), {
+      maxItems: 8
+    }),
+    reason: Type2.String(),
+    checks: Type2.Object({
+      answersQuestion: Type2.Boolean(),
+      correctEntity: Type2.Boolean(),
+      correctTime: Type2.Boolean(),
+      preservesScope: Type2.Boolean()
+    })
   }), { maxItems: 256 })
 });
 var researchLeaves = (value, maxFields = 128) => {
@@ -124163,18 +124180,53 @@ var bindResearchReview = (leaves, reviews, sources) => leaves.map((leaf) => {
     return matching.length === 1 && sourceSupportsQuote(matching[0], citation.quote);
   }) ?? [];
   const valid = !!review && citations.length === review.citations.length;
-  const verdict = !valid || leaf.value === null || !citations.length ? "unknown" : review.verdict;
+  const checksPass = review?.checks && [
+    "answersQuestion",
+    "correctEntity",
+    "correctTime",
+    "preservesScope"
+  ].every((key2) => review.checks?.[key2] === true);
+  const verdict = !valid || leaf.value === null || !citations.length ? "unknown" : review.verdict === "supported" && !checksPass ? "unknown" : review.verdict;
   return {
     ...leaf,
     verdict,
     citations,
-    reason: valid ? review.reason : "Missing, ambiguous, or invalid source-bound review"
+    checks: review?.checks,
+    reason: valid && review.verdict === "supported" && !checksPass ? "Task relevance, entity, time, or scope was not established: " + review.reason : valid ? review.reason : "Missing, ambiguous, or invalid source-bound review"
   };
 });
+var researchPassages = (sources) => sources.map((source) => {
+  const passages = [];
+  for (const excerpt of source.excerpts) {
+    let start = 0;
+    while (start < excerpt.length) {
+      let end = Math.min(start + 600, excerpt.length);
+      if (end < excerpt.length) {
+        const boundary = excerpt.lastIndexOf(`
+`, end);
+        if (boundary > start + 300)
+          end = boundary;
+        if (excerpt.length - end < 12)
+          end = excerpt.length;
+      }
+      passages.push({
+        id: `${source.id}:${passages.length}`,
+        text: excerpt.slice(start, end)
+      });
+      start = end;
+    }
+  }
+  const { excerpts: _excerpts, ...metadata } = source;
+  return { ...metadata, passages };
+});
+var resolveResearchPassages = (citations, sources) => citations.map((citation) => {
+  const matches2 = sources.flatMap((source) => source.passages.filter((passage) => passage.id === citation.passageId).map((passage) => ({ sourceId: source.id, quote: passage.text })));
+  return matches2.length === 1 ? matches2[0] : { sourceId: "", quote: "" };
+});
 
 // src/research/runtime.ts
 var DefaultSchema = Type2.Object({
-  findings: Type2.Array(Type2.String(), { maxItems: 12 })
+  findings: Type2.Array(Type2.String(), { maxItems: 6 })
 });
 var PlanSchema = Type2.Object({
   queries: Type2.Array(Type2.String({ maxLength: 600 }), { maxItems: 8 }),
@@ -124261,8 +124313,8 @@ var createResearch = (config) => {
     };
     const generate2 = async (kind, schema, prompt, payload) => {
       const generated = await operate(kind, () => (config.generateObject ?? generateObjectAI)({
-        provider: config.provider,
-        model: config.model,
+        provider: kind === "review" ? config.reviewer?.provider ?? config.provider : config.provider,
+        model: kind === "review" ? config.reviewer?.model ?? config.model : config.model,
         schema,
         signal,
         contextPolicy: false,
@@ -124273,7 +124325,7 @@ var createResearch = (config) => {
 ${prompt}`,
         messages: [{ role: "user", content: JSON.stringify(payload) }],
         validate: (value) => {
-          if (!exports_value2.Check(schema, value))
+          if (!checkResearchValue(schema, value))
             throw new Error(`Invalid ${kind} output`);
           return value;
         }
@@ -124285,11 +124337,30 @@ ${prompt}`,
     const urls = new Set;
     const queries = new Set;
     let evidenceChars = 0;
+    const sourceText = new Map;
+    const rebalanceEvidence = () => {
+      let remaining = limits.evidenceChars;
+      const weight = (source) => source.contentFetchedAt ? 4 : 1;
+      const ordered = [...result.sources].sort((a, b) => sourceText.get(a.url).length / weight(a) - sourceText.get(b.url).length / weight(b));
+      let remainingWeight = ordered.reduce((sum, source) => sum + weight(source), 0);
+      for (const source of ordered) {
+        const text = sourceText.get(source.url);
+        const allowance = Math.floor(remaining * weight(source) / remainingWeight);
+        remainingWeight -= weight(source);
+        source.excerpts = [text.slice(0, allowance)];
+        remaining -= source.excerpts[0].length;
+      }
+      evidenceChars = limits.evidenceChars - remaining;
+    };
     const addSource = (source) => {
       if (!requiredPhrases.every((phrase) => normalize(`${source.title} ${source.excerpts.join(" ")}`).includes(normalize(phrase))))
         return;
-      if (result.sources.some((existing) => existing.url === source.url) || evidenceChars >= limits.evidenceChars)
+      if (result.sources.some((existing) => existing.url === source.url))
+        return;
+      if (result.sources.length >= 64) {
+        result.limitations.push("Evidence source limit reached");
         return;
+      }
       try {
         const url = new URL(source.url);
         if (!["http:", "https:"].includes(url.protocol) || url.username || url.password)
@@ -124298,15 +124369,16 @@ ${prompt}`,
         return;
       }
       const text = source.excerpts.join(`
-`).slice(0, limits.evidenceChars - evidenceChars);
+`).slice(0, limits.evidenceChars);
       if (!text.trim())
         return;
-      evidenceChars += text.length;
+      sourceText.set(source.url, text);
       result.sources.push({
         ...source,
         id: `s${result.sources.length + 1}`,
         excerpts: [text]
       });
+      rebalanceEvidence();
     };
     const doSearch = async (query2) => {
       for (const bounded of boundedQueries(query2)) {
@@ -124349,12 +124421,9 @@ ${prompt}`,
       if (page.text && page.status !== "error") {
         const prior = result.sources.find((source) => source.url === url);
         if (prior) {
-          const extra = page.text.slice(0, limits.evidenceChars - evidenceChars);
-          if (extra) {
-            prior.excerpts.push(extra);
-            evidenceChars += extra.length;
-            prior.contentFetchedAt = page.fetchedAt;
-          }
+          sourceText.set(prior.url, page.text.slice(0, limits.evidenceChars));
+          prior.contentFetchedAt = page.fetchedAt;
+          rebalanceEvidence();
         } else
           addSource({
             id: "",
@@ -124372,16 +124441,24 @@ ${prompt}`,
       for (let round = 0;round < limits.rounds; round++) {
         if (!result.sources.length && queries.size >= limits.searches)
           break;
-        const plan = await generate2("plan", PlanSchema, "Select follow-up searches for missing evidence and primary-source URLs to read. URLs must come from the supplied search results. done means available evidence addresses the requested schema, not merely that some results exist. Return no actions if further work is not useful.", {
-          query: input.query,
-          schema: task.schema,
-          instructions: task.instructions,
-          sources: result.sources,
-          searched: [...queries],
-          read: [...urls],
-          remainingSearches: limits.searches - queries.size,
-          remainingReads: limits.reads - urls.size
-        });
+        let plan;
+        try {
+          plan = await generate2("plan", PlanSchema, "Select follow-up searches for missing evidence and primary-source URLs to read. For current people or roles, seek the organization\u2019s own leadership/about/team pages; testimonials, package authors, and job listings are not staff identity evidence. For policies, read the full policy or agreement to preserve section scope and exceptions. URLs must come from the supplied search results. done means available evidence addresses the requested schema, not merely that some results exist. Return no actions if further work is not useful.", {
+            query: input.query,
+            schema: task.schema,
+            instructions: task.instructions,
+            sources: result.sources,
+            searched: [...queries],
+            read: [...urls],
+            remainingSearches: limits.searches - queries.size,
+            remainingReads: limits.reads - urls.size
+          });
+        } catch (error) {
+          if (!(error instanceof Error) || error.message !== "Invalid plan output")
+            throw error;
+          result.limitations.push("Invalid planning output; continued with retrieved evidence");
+          break;
+        }
         for (const url of plan.urls)
           await doRead(url);
         for (const query2 of plan.queries)
@@ -124397,15 +124474,30 @@ ${prompt}`,
       }
       if (evidenceChars >= limits.evidenceChars)
         result.limitations.push("Evidence text limit reached");
-      const data = await generate2("extract", task.schema, `Extract only evidence-supported facts for the requested query and schema. ${task.instructions ?? ""}`, { query: input.query, sources: result.sources });
+      const data = await generate2("extract", task.schema, `Extract only facts that directly answer the requested query and schema. Return concise, atomic answers; include only the minimum facts needed to answer the question. For a leadership query, report the named person and current title, not their biography, education, or acquisition history. Do not pad findings with background facts, excluded candidates, or statements that evidence is missing; return an empty findings array when no answer is established. Preserve all material qualifiers: program versus benefit/trial, customer versus partner, eligibility versus automatic approval, dates, exceptions, and attribution of promotional claims. Never generalize a section-specific condition to the whole organization or program. ${task.instructions ?? ""}`, {
+        query: input.query,
+        asOf: result.generatedAt,
+        sources: result.sources
+      });
       const leaves = researchLeaves(data, limits.fields);
       if (!leaves.length) {
         result.data = data;
         result.status = result.limitations.length ? "partial" : "empty";
         return result;
       }
-      const reviewed = await generate2("review", ReviewSchema, "Independently review EVERY JSON pointer and its value. supported requires exact quotes establishing the entire fact for the correct entity and time. A name mention or publication date is insufficient to establish employment, event date, ownership, or buying intent. Check all sources for conflicts. Use conflicting when sources disagree, unsupported for contradicted claims, unknown for missing support. Null is unknown. Quotes must appear in supplied excerpts. Return each pointer once. Never treat this review as independent real-world verification.", { query: input.query, fields: leaves, sources: result.sources });
-      result.fields = bindResearchReview(leaves, reviewed.fields, result.sources);
+      const reviewSources = researchPassages(result.sources);
+      const reviewed = await generate2("review", ReviewSchema, "Independently review EVERY JSON pointer and its value. supported requires exact quotes establishing the entire fact for the correct entity and time. A name mention or publication date is insufficient to establish employment, event date, ownership, or buying intent. Check all sources for conflicts. Use conflicting when sources disagree, unsupported for contradicted claims, unknown for missing support. Null is unknown. Cite only supplied passage IDs; their text will be copied verbatim by the runtime. For each field, explicitly check answersQuestion (directly answers query/schema/instructions, not background or a non-answer), correctEntity, correctTime, and preservesScope. Read surrounding context and headings, not only the matching quote. A benefit/trial condition is not a program-wide rule; customer conditions are not partner conditions; eligible is not approved or automatic. Preserve exceptions, qualifiers and marketing attribution. Every check must pass for supported. A leader\u2019s education or career history does not answer who currently leads a team. Keep each reason under 25 words. Cite only the minimum passages needed to establish the entire field, at most four. Do not invent passage IDs. Read adjacent passages for conditions and exceptions, even when citing only one. Return each pointer once. Never treat this review as independent real-world verification.", {
+        query: input.query,
+        asOf: result.generatedAt,
+        instructions: task.instructions,
+        schema: task.schema,
+        fields: leaves,
+        sources: reviewSources
+      });
+      result.fields = bindResearchReview(leaves, reviewed.fields.map((field) => ({
+        ...field,
+        citations: resolveResearchPassages(field.citations, reviewSources)
+      })), result.sources);
       const complete = result.fields.length > 0 && result.fields.every((field) => field.verdict === "supported");
       result.data = complete ? data : null;
       result.status = complete && !result.limitations.length ? "reviewed" : "partial";
@@ -125273,5 +125365,5 @@ export {
   selectResearchMonitorEvents
 };
 
-//# debugId=815A4FA45BDD337464756E2164756E21
+//# debugId=F9058798C65361FB64756E2164756E21
 //# sourceMappingURL=index.js.map