@falai/agent 1.2.8 → 2.0.0

package/docs/guides/compaction.md

@@ -0,0 +1,163 @@
+---
+title: "Compaction"
+description: "Keep prompts within token limits across long sessions by layering tool-result budgeting, micro-compaction, and LLM summarization in cost order."
+type: guide
+order: 8
+---
+
+# Compaction
+
+> **Where this is introduced:** [Errors](./error-handling.md)
+
+Long-running sessions accumulate history. Tool calls return verbose
+JSON, turns pile up, and at some point the next provider call runs
+out of token budget. Compaction is the framework's answer: a layered
+strategy that trims and summarizes `session.history` before each turn
+so the prompt fits without dropping anything load-bearing. Layers run
+in cost order — character-level truncation first, an LLM summarization
+call only when nothing cheaper closes the gap.
+
+This guide is task-shaped: enable compaction, choose a budget, and
+understand which layer fires when.
+
+## Enable compaction
+
+Compaction is opt-in. Set `AgentOptions.compaction` and the agent
+validates the config at construction time, then runs the engine on
+every `respond` / `respondStream` call before the prompt is composed.
+
+```typescript
+import { createAgent, GeminiProvider } from "@falai/agent";
+
+const agent = createAgent({
+  name: "Concierge",
+  provider: new GeminiProvider({ apiKey: process.env.GEMINI_API_KEY! }),
+  schema: { /* ... */ },
+  flows: [/* ... */],
+
+  compaction: {
+    maxTokens: 32_000,           // total token budget for the prompt
+    compactionThreshold: 0.8,    // fire when history hits 80% of maxTokens (default)
+    preserveRecentCount: 4,      // never touch the last 4 history items (default)
+    maxToolResultChars: 5_000,   // global cap per tool message (default)
+    // enabled: true,            // default true when `compaction` is provided
+  },
+});
+```
+
+The four knobs map onto the
+[`AgentCompactionConfig`](../reference/create-agent.md) interface.
+`maxTokens` is the only required field; the rest fall back to the
+defaults shown above. The agent calls `validateOptions` synchronously
+at construction, so misvalued thresholds (`compactionThreshold` outside
+`[0.5, 0.95]`, `preserveRecentCount < 2`, `maxToolResultChars <= 0`)
+throw immediately rather than silently no-oping at runtime.
+
+A second knob lives on every `Tool`: `maxResultSizeChars`. That value
+is enforced by the tool executor the moment a tool returns, before the
+result enters history at all. It and the agent-level `maxToolResultChars`
+work together — per-tool first, global later.
+
+## How a turn checks the budget
+
+Before each turn, the session manager calls
+`CompactionEngine.checkAndCompact(session.history, options)`. The
+engine estimates the current token count using a character-based
+heuristic (~4 characters per token), compares against
+`maxTokens * compactionThreshold`, and applies the cheapest layer that
+brings history below the threshold. If even the most expensive layer
+cannot, an aggressive truncation fallback removes the oldest items
+until the budget fits.
+
+`preserveRecentCount` is honored by every layer. The trailing N items
+of `session.history` are never modified, summarized, or removed —
+recent turns are the one piece of context the engine refuses to spend.
+
+## Layer 1 — tool-result budgeting
+
+`tool_result_budget` is the cheapest layer. It walks history, finds
+items with `role: "tool"`, and truncates any whose stringified content
+exceeds `maxToolResultChars`. Truncated items get a deterministic
+notice appended:
+
+```text
+[Truncated: 12834 chars total, showing first 5000]
+```
+
+Character-level and synchronous — no LLM call, no network. It fires
+whenever history crosses the threshold and at least one tool message
+is oversized. For agents that orchestrate verbose APIs (search, SQL,
+web fetches) this layer alone usually keeps the prompt in budget.
+
+The per-tool counterpart is `Tool.maxResultSizeChars`. Set it on
+high-volume tools to truncate at execution time, before the result
+ever lands in history. Combine the two: a tight per-tool cap on a
+known-chatty tool, plus a global ceiling at the agent level.
+
+## Layer 2 — micro-compaction
+
+If `tool_result_budget` is not enough, `micro_compact` runs over the
+already-budgeted history. It compresses verbose tool outputs inline
+by collapsing whitespace runs to a single space and trimming the
+edges. Tool results are the only target — user, assistant, and system
+messages pass through unchanged.
+
+Still LLM-free and deterministic. Most effective on tool results that
+contain pretty-printed JSON or multiline text where formatting is
+whitespace-heavy. The recent-N tail (`preserveRecentCount`) is
+preserved verbatim during this pass; the cutoff is `history.length -
+preserveRecentCount`, and only items before that cutoff are
+compressed.
+
+## Layer 3 — LLM summarization
+
+When neither character-level layer brings the prompt under threshold,
+`auto_compact` dispatches a single `provider.generateMessage` call
+asking the agent's own LLM to summarize the older portion. The result
+becomes one synthetic system message:
+
+```text
+[Conversation Summary]
+<summary text from the provider>
+```
+
+That synthetic item replaces every history item before the
+`preserveRecentCount` tail. The recent tail is appended unchanged. If
+the provider call fails — quota, network, any caught exception — the
+engine falls back to `aggressiveTruncate`: walk older messages newest
+to oldest, keep as many as fit under `maxTokens * compactionThreshold`,
+drop the rest. The strategy field on the result still reads
+`"auto_compact"` so callers can see that summarization was attempted.
+
+This layer costs a real provider round-trip. It only fires when
+character-level layers cannot close the gap — in practice, sessions
+with hundreds of long turns or tools that return narrative prose
+rather than structured data.
+
+## When each layer fires
+
+The `compactionThreshold` ratio is the gate for all three layers. Below
+threshold, `checkAndCompact` returns `strategy: "none"` and history
+passes through untouched. At or above threshold, the engine attempts
+each layer in order and stops as soon as the result drops below the
+threshold:
+
+| Estimated tokens | Strategy that fires |
+|------------------|---------------------|
+| `< maxTokens * compactionThreshold` | `none` |
+| `≥ threshold`, oversized tool messages exist | `tool_result_budget` |
+| `≥ threshold` after budgeting | `micro_compact` |
+| `≥ threshold` after micro-compaction | `auto_compact` (LLM call) |
+| LLM call failed | `auto_compact` (truncation fallback) |
+
+The result's `messagesCompacted` count and optional `summary` field
+are logged at `info` level so production logs show which layer fired
+on which turn.
+
+A practical defaults sketch: set `maxTokens` to ~80% of the model's
+context window, leave `compactionThreshold` at the 0.8 default, and
+put a tight per-tool `maxResultSizeChars` on any tool that talks to
+a search API or a database. Pick the budget, set the caps, let the
+layers absorb the noise.
+
+**Next:** [Architecture](../concepts/architecture.md)

package/docs/guides/conditions.md

@@ -0,0 +1,167 @@
+---
+title: "When and if"
+description: "Pick between AI-evaluated `when` strings and code-evaluated `if` predicates, and combine them to save tokens."
+type: guide
+order: 1
+---
+
+# When and if
+
+Conditions decide whether a flow activates, a step runs, an instruction renders, or a branch fires. v2 splits them into two distinct fields with different evaluators:
+
+- `when` — strings the LLM evaluates against intent and the conversation. Costs tokens.
+- `if` — TypeScript predicates the engine evaluates locally. Free.
+
+The split is the same everywhere: `Flow`, `Step`, `Instruction`, and `BranchEntry` all expose `when?` and `if?` with identical semantics. This guide shows how to pick between them, how arrays combine, and what happens when both are set. If you are migrating from v1, the [v1 → v2 migration guide](../migration/v1-to-v2.md) covers the renamed condition fields.
+
+## Pick the right field
+
+Reach for `if` first. It's free, deterministic, and reads clearly in code review.
+
+| Question to answer                                              | Use      |
+| --------------------------------------------------------------- | -------- |
+| Is this user authenticated? Is `data.tier === 'pro'`?           | `if`     |
+| Is the feature flag on? Is the order older than 90 days?        | `if`     |
+| Did the user ask about pricing? Are they expressing frustration? | `when`   |
+| Is the user describing a refund scenario in their own words?    | `when`   |
+
+If the answer lives in `data`, `context`, or `session`, it's `if`. If the answer requires reading the user's intent from natural language, it's `when`.
+
+## `when` — AI strings
+
+`when` accepts a string or an array of strings. Functions are rejected at construction time with `FlowConfigurationError`.
+
+```typescript
+{
+  title: "Refund",
+  when: "the user is requesting a refund",
+}
+```
+
+Multiple strings combine with **AND** semantics — every clause must pass for the condition to match.
+
+```typescript
+{
+  title: "Premium Refund",
+  when: [
+    "the user is requesting a refund",
+    "the user is asking about an enterprise plan",
+  ],
+}
+```
+
+The strings are sent to the LLM as part of the routing or activation prompt. Keep them short, intent-shaped, and free of code-style boolean expressions — `"the user wants to cancel"` lands; `"data.cancelRequested === true"` does not.
+
+## `if` — code predicates
+
+`if` accepts a function or an array of functions. Each predicate receives a `TemplateContext`-shaped argument (`{ context, data, session, history, helpers }`) and returns `boolean | Promise<boolean>`.
+
+```typescript
+{
+  title: "Enterprise",
+  if: ({ context }) => context.tier === "enterprise",
+}
+```
+
+Arrays combine with **AND** semantics — every predicate must return truthy.
+
+```typescript
+{
+  if: [
+    ({ context }) => context.authenticated,
+    ({ data }) => (data.cartTotal ?? 0) > 100,
+  ],
+}
+```
+
+Predicates that throw or reject are caught, logged, and treated as `false` for that evaluation. They never corrupt the session — the worst case is the condition fails to match.
+
+### What's in the predicate context
+
+Predicates receive a context object with the same shape used everywhere templates and conditions evaluate. The fields you'll reach for most:
+
+| Field      | Type                     | Notes                                                            |
+| ---------- | ------------------------ | ---------------------------------------------------------------- |
+| `data`     | `Partial<TData>`         | Collected schema fields. Null-check anything not in `requires`.  |
+| `context`  | `TContext`               | Agent-level ambient context (user, env, services).               |
+| `session`  | `SessionState<TData>`    | Current flow id, current step id, full history.                  |
+| `history`  | `Event[]`                | Read-only conversation history.                                  |
+
+Predicates can be `async`. Awaiting a database lookup or feature-flag service is supported, but remember: every predicate runs every time its host primitive is evaluated. Keep them cheap, or memoize the work in a hook upstream.
+
+```typescript
+{
+  if: async ({ context, data }) =>
+    await context.flags.isEnabled("v2_pricing", data.userId),
+}
+```
+
+## When both are set
+
+Setting both `when` and `if` on the same primitive runs `if` **first**, free. `when` is only sent to the LLM when every `if` predicate passes. The order is deliberate: predicates short-circuit the LLM call when the answer is already disqualified.
+
+```typescript
+{
+  title: "US Pricing",
+  if: ({ context }) => context.country === "US" && context.flags.usPricing,
+  when: "the user is asking about pricing",
+}
+```
+
+In the snippet above, non-US users skip the AI evaluation entirely. The `when` string costs tokens only when the predicate already says "this user is in scope, ask the AI whether they're asking about pricing."
+
+This pattern is the recommended shape any time a condition has both a cheap precondition and an intent classification. Lead with `if` to gate. Use `when` to interpret.
+
+## Where the split lives
+
+The same `when` / `if` shape attaches to four primitives. Semantics are identical in each location:
+
+| Primitive       | Field path              | What it gates                                      |
+| --------------- | ----------------------- | -------------------------------------------------- |
+| Flow            | `FlowOptions.when/if`   | Whether the router selects this flow this turn     |
+| Step            | `StepOptions.when/if`   | Whether the step is reachable in the current flow  |
+| Instruction     | `Instruction.when/if`   | Whether the instruction renders into the prompt    |
+| BranchEntry     | `BranchEntry.when/if`   | Whether this branch entry matches inside `step.branches` |
+
+```typescript
+// Flow scope — gate flow selection
+{ title: "Refund", when: "user wants a refund", if: ({ context }) => context.authenticated }
+
+// Step scope — gate step reachability
+{ id: "verify_payment", when: "user is confirming the order", if: ({ data }) => !!data.cardToken }
+
+// Instruction scope — render only when relevant
+{ kind: "should", when: "user mentions a discount code", prompt: "Validate the code before applying it." }
+
+// Branch scope — pick a successor
+branches: [
+  { if: ({ data }) => data.tier === "enterprise", when: "user wants pricing", then: "enterprise_pricing" },
+  { then: "default_pricing" },
+]
+```
+
+## `step.skip` is the OR companion
+
+One adjacent field rounds out the picture. `step.skip` is **function-only** with **OR** semantics — when any predicate returns truthy, the step is bypassed. Use it for "skip when this field already exists" cases:
+
+```typescript
+{
+  id: "ask_email",
+  collect: ["email"],
+  skip: ({ data }) => !!data.email,
+}
+```
+
+`skip` does not accept strings. There is no AI counterpart — skipping is a code decision by design.
+
+## Quick reference
+
+A short checklist before shipping a condition:
+
+- Does it read a field, flag, or context value? Use `if`.
+- Does it interpret natural language? Use `when`.
+- Multiple clauses that all must pass? Drop them in an array — both fields AND.
+- Need to skip when a value is already collected? `step.skip` (OR semantics).
+- Both fields set? `if` runs first, free; `when` only fires if `if` passes.
+
+**Next:** [Branching](./branching.md)

package/docs/guides/error-handling.md

@@ -0,0 +1,176 @@
+---
+title: "Errors"
+description: "Catch the typed error classes by class, narrow them with instanceof, and recover with the right pattern for each failure mode."
+type: guide
+order: 7
+---
+
+# Errors
+
+`@falai/agent` throws typed `Error` subclasses for every failure mode the framework owns. Catching `Error` is fine for telemetry, but recovering well needs class-level narrowing — a `DataValidationError` wants a re-prompt, a `ToolExecutionError` wants a soft message and maybe a redirect, a `FlowConfigurationError` wants to crash the build. This page is the recipe for each. The full surface lives in the [errors reference](../reference/errors.md).
+
+## The format contract
+
+Every thrown message follows the same shape:
+
+```text
+[<ErrorClass>] <what>: <why>. <how to fix>.
+```
+
+The bracketed prefix mirrors the class name. The colon separates `<what>` from `<why>`. The trailing sentence is `<how to fix>`. The contract is uniform across the five classes the framework throws — `FlowConfigurationError`, `DataValidationError`, `ToolExecutionError`, `ResponseGenerationError`, and `NotImplementedError` — so log lines and `try/catch` blocks parse the same way regardless of which class fired.
+
+The format applies to framework-thrown errors. If you write a custom tool or hook that throws, follow the same shape so downstream handlers stay uniform.
+
+## The five classes
+
+| Class | Thrown when | Recover by |
+|-------|-------------|------------|
+| `FlowConfigurationError` | Misconfigured agent: duplicate ids, unknown `collect` keys, malformed `Directive`, branch targets that don't resolve, auto-step cycles, function on `when`. | Don't. Surface in CI. |
+| `DataValidationError` | A user message produces a value that violates the declared `schema`. | Re-prompt the user for the offending fields. |
+| `ToolExecutionError` | A tool handler throws, all retries fail, or `validateInput` cannot correct invalid args. | User-friendly message, optional retry, optional `agent.dispatch` to a recovery flow. |
+| `ResponseGenerationError` | The provider call fails or the response cannot be parsed. | Backoff retry, fall back to a different provider, or surface a soft failure. |
+| `NotImplementedError` | A reserved option is set to a value this version does not support (e.g. `routerMode: 'embedding'` in v2.0). | Use a supported value. Same posture as `FlowConfigurationError`. |
+
+`FlowConfigurationError`, `ToolExecutionError`, and `NotImplementedError` are exported from `@falai/agent` and matchable with `instanceof`. `DataValidationError` and `ResponseGenerationError` are internal — match them by `error.name`.
+
+## Pattern 1: try/catch + instanceof narrowing
+
+Wrap every `agent.respond` and `agent.respondStream` call site. Narrow by class, return the right user-facing string per branch, rethrow what cannot be recovered.
+
+```typescript
+import {
+  FlowConfigurationError,
+  ToolExecutionError,
+  NotImplementedError,
+} from "@falai/agent";
+
+try {
+  const response = await agent.respond({ history, session });
+  return response.message;
+} catch (err) {
+  if (err instanceof FlowConfigurationError) throw err;     // bug — bubble up
+  if (err instanceof NotImplementedError) throw err;        // config bug — bubble up
+  if (err instanceof ToolExecutionError) {
+    log.warn({ toolId: err.toolId, cause: err.cause }, err.message);
+    return "Sorry — that action failed. Try again in a moment.";
+  }
+  if (err instanceof Error && err.name === "DataValidationError") {
+    return "I need a couple of details cleared up — let's try that again.";
+  }
+  if (err instanceof Error && err.name === "ResponseGenerationError") {
+    return "I'm having trouble reaching the model. Please retry.";
+  }
+  throw err;
+}
+```
+
+The two unrecoverable classes — `FlowConfigurationError` and `NotImplementedError` — get rethrown so the process crashes loudly. The other three return a graceful message. Anything unexpected falls through to the outer rethrow.
+
+## Pattern 2: DataValidationError → re-prompt
+
+A `DataValidationError` carries an `errors: ValidationError[]` array naming the offending fields. Use it to ask the user a targeted clarifying question instead of a generic one.
+
+```typescript
+} catch (err) {
+  if (err instanceof Error && err.name === "DataValidationError") {
+    const fields = (err as { errors: { path: string; message: string }[] }).errors
+      .map((e) => `${e.path} (${e.message})`)
+      .join(", ");
+    return `I couldn't read these from your message: ${fields}. Mind clarifying?`;
+  }
+  // ...
+}
+```
+
+The framework does not retry by itself — the next user turn is the retry. Keep `session` alive between turns so the partial collection survives.
+
+## Pattern 3: ToolExecutionError → message, retry, or dispatch
+
+Tool failures have three shapes. Pick by what the failing tool was doing.
+
+**Soft failure — speak and continue.** A read-only tool that timed out: tell the user, let the conversation move on.
+
+```typescript
+if (err instanceof ToolExecutionError) {
+  log.warn({ toolId: err.toolId }, err.message);
+  return "I couldn't fetch that just now. Want to try again?";
+}
+```
+
+**Optional retry.** Wrap the `respond` call in a small retry loop for transient errors.
+
+```typescript
+for (let attempt = 0; attempt < 2; attempt++) {
+  try {
+    return (await agent.respond({ history, session })).message;
+  } catch (err) {
+    if (err instanceof ToolExecutionError && attempt === 0) {
+      await new Promise((r) => setTimeout(r, 250));
+      continue;
+    }
+    throw err;
+  }
+}
+```
+
+**Recovery flow.** Hard failures — a payment tool that hit a permanent denial — should redirect into a recovery flow rather than echo the error. Dispatch a directive against the session so the *next* turn enters the recovery flow.
+
+```typescript
+if (err instanceof ToolExecutionError && err.toolId === "charge_card") {
+  await agent.dispatch(session, { goTo: "PaymentRecovery" });
+  return "I hit a snag with that payment. Let me walk you through it.";
+}
+```
+
+The dispatched [`Directive`](../reference/directive.md) lands in `session.pendingDirective` and is consumed at the start of the next turn.
+
+## Pattern 4: ResponseGenerationError → backoff or fallback provider
+
+Provider errors are usually transient. Retry with backoff first; fall back to a second provider only if the primary keeps failing.
+
+```typescript
+async function respondWithFallback(history: HistoryItem[], session: Session) {
+  for (let attempt = 0; attempt < 3; attempt++) {
+    try {
+      return await primaryAgent.respond({ history, session });
+    } catch (err) {
+      if (err instanceof Error && err.name === "ResponseGenerationError") {
+        await new Promise((r) => setTimeout(r, 200 * 2 ** attempt));
+        continue;
+      }
+      throw err;
+    }
+  }
+  return fallbackAgent.respond({ history, session });
+}
+```
+
+Both agents share the same `flows`, `tools`, and `schema` — only the `provider` differs. The session is provider-agnostic, so the fallback picks up exactly where the primary left off.
+
+## Pattern 5: FlowConfigurationError → don't catch in production
+
+`FlowConfigurationError` is a *bug*, not a runtime condition. It fires when the agent definition is malformed: a step references a flow id that doesn't exist, two steps share a `collect` key, a branch target points nowhere, an auto-step chain loops. The right place to catch it is the test runner and CI — never the request path.
+
+Smoke-test agent construction in CI so misconfiguration crashes the build rather than the first user request:
+
+```typescript
+import { describe, expect, it } from "bun:test";
+import { FlowConfigurationError } from "@falai/agent";
+import { buildAgent } from "../src/agent";
+
+describe("agent construction", () => {
+  it("builds without configuration errors", () => {
+    expect(() => buildAgent()).not.toThrow(FlowConfigurationError);
+  });
+});
+```
+
+In the request handler, let it bubble all the way up. A 500 plus a loud log is the correct outcome — the next deploy is the fix.
+
+```typescript
+if (err instanceof FlowConfigurationError) throw err;
+```
+
+`NotImplementedError` follows the same posture: it means a reserved option was set to a value this version doesn't support. Read the message, fix the config, ship the fix. Don't catch it.
+
+**Next:** [Compaction](./compaction.md)