@exellix/ai-tasks 8.4.2 → 8.5.0

package/.docs/synthesized-context-strategy-spec.md

@@ -0,0 +1,906 @@
+# Synthesized-Context Execution Strategy — Specification
+
+## The Problem
+
+Today, the DIRECT strategy passes raw memory/narrix content straight to the main task model. The main model receives everything — often far more context than it needs — and must simultaneously understand the domain material *and* perform the task. This wastes tokens on an expensive model doing work that a cheaper model could handle: distilling raw context into task-relevant input.
+
+The aiScoping feature partially addresses this by scoping individual memory paths, but it operates at a field level with separate calls per path. There is no mechanism that synthesizes the *entire available context* (narrix output or memory) into a single, task-aware input — one that is shaped by what the downstream task actually needs to do.
+
+## The Idea
+
+Introduce a new execution strategy — `synthesized-context` — that inserts a **synthesis pre-pass** before the main task execution. A weak/cheap model reads the available context (narrix output when present, memory otherwise) together with the downstream task's own instructions, and produces a **synthesized input** — a compressed, task-relevant representation of everything the main model needs to know.
+
+The main model then runs against this synthesized input instead of raw memory, getting exactly the information it needs in a form it can act on immediately.
+
+This strategy works **with or without Narrix**. It is not a Narrix feature; it is an execution strategy that *leverages* Narrix when available but falls back to memory when not.
+
+---
+
+## Why This Works
+
+| Concern | How this strategy addresses it |
+|---|---|
+| Token waste on the main model | The expensive model receives only what it needs, in pre-digested form |
+| Context overload | The synthesis pass compresses and filters before the main model sees anything |
+| Task alignment | The synthesis prompt includes the downstream task instructions, so the weak model knows *what matters* |
+| Narrix independence | Context source is configurable: narrix-first with memory fallback, or memory-only |
+| Cost control | Synthesis uses a weak/cheap model — the total cost (weak + main) is often less than sending everything to the main model |
+
+---
+
+## Execution pipeline: PRE → main → POST
+
+Execution is a **pipeline** of steps with three phases:
+
+1. **PRE** — zero or more steps that run **before** the main task (e.g. synthesize context, run narrix). Each can mutate request/context for the next step.
+2. **Main** — exactly one step that runs the actual task (e.g. DIRECT: call the skill with the current input and context).
+3. **POST** — zero or more steps that run **after** the main task (e.g. future: validate, enrich response, log). Optional; none defined in this spec.
+
+The synthesized-context feature is a **PRE step**: it runs **before** the real task. The weak model synthesizes context; then the main step runs the skill with that synthesized context as if it had been the normal context. There is no POST step in this change.
+
+The request carries an **array** of steps (see "Execution pipeline API" below). Order of execution: all PRE steps in array order → the single MAIN step → all POST steps in array order. This allows many pre/post strategies over time; this spec only adds one PRE step type.
+
+---
+
+## Strategy identity (synthesized-context as PRE step)
+
+| Property | Value |
+|---|---|
+| Phase | **PRE** (runs before the main task) |
+| Step type | `"synthesized-context"` |
+| Requires `includeContextInPrompt` | **Yes** — the synthesized output is delivered via the context message to the main step |
+
+> **Critical constraint**: This step only works when `includeContextInPrompt: true`. The synthesized output replaces the context markdown that would normally be generated for the main step. If `includeContextInPrompt` is not set, the implementation must either throw a clear error or force it to `true` (configurable via `autoEnableContext`, see config below).
+
+---
+
+## Context Source Resolution
+
+The synthesis pass needs raw material to work with. Where that material comes from depends on **`contextSourcePolicy`** and what's available at runtime.
+
+### Policy table (authoritative)
+
+| `contextSourcePolicy` | Narrix markdown | Web evidence markdown (`buildWebContextEvidenceMarkdown`) | Memory JSON (`jobMemory` / `taskMemory` / `executionMemory`) |
+|----------------------|-----------------|-----------------------------------------------------------|---------------------------------------------------------------|
+| `narrix-web` | Yes (required) | Yes if `executionMemory.webContext` hit | No |
+| `narrix-web-memory` | Yes if attachment/coercion exists | Yes if hit | Yes — **`webContext` is never serialized as raw JSON** (use markdown only when web is included) |
+| `memory-web` | **No** (Narrix fields stripped from bundle before serialize) | Yes if hit | Yes |
+| `memory-only` | No | No | Yes — **`webContext` never appears as raw JSON** |
+| `auto` | — | — | Resolves at runtime: **`narrix-web-memory`** if Narrix output exists; else **`memory-web`** if web scoping hit; else **`memory-only`** |
+
+**Legacy aliases (same behavior):** `narrix-only` → `narrix-web`, `narrix+memory` → `narrix-web-memory`.
+
+**`webEvidence` (optional on `SynthesisConfig`):** Passed to web markdown builder — `preferCleanContent` (default true), `maxSources` (default 5), `dedupeByUrl` (default true), `maxTotalChars`.
+
+**`narrixAttachToField`:** Taken from `request.narrix.attachToField` (default `_narrix`) when stripping Narrix for `memory-web`.
+
+### What "narrix output" means
+
+When narrix is in play (via `request.narrix` pre-processor or `narrix-then-direct` flow), the narrix output is the `NarrixEnrichedAttachment` that would normally go into `executionMemory._narrix` or `taskMemory.narrix`. In synthesized-context mode, instead of injecting it raw, we feed it to the synthesis pass.
+
+### What "memory" means
+
+The enriched memory bundle — `jobMemory`, `taskMemory`, `executionMemory` — after standard enrichment (same bundle that `_executeDirect` builds today). The `bindingDefaultsDb` and other internal fields are cleansed before synthesis, same as today.
+
+---
+
+## The Synthesis Call
+
+### How it runs: AIGateway
+
+The synthesis call is an internal LLM invocation via `AIGateway` — the same mechanism used by `runScopingCall` in aiScoping. It follows the exact same pattern:
+
+```ts
+import { AIGateway } from "@athenices/ai-gateway";
+
+const gateway = new AIGateway();
+const response = await gateway.invoke({
+  jobId: request.jobId ?? "synthesis",
+  agentId: request.agentId ?? "synthesis",
+  instructions: SYNTHESIS_SYSTEM_PROMPT,       // the synthesis template (see below)
+  workingMemory: { input: synthesisUserPrompt }, // "Synthesize now."
+  config: { model: synthesisModel },            // the weak/cheap model
+});
+```
+
+This is a standard gateway call. No new infrastructure needed — the synthesis strategy just makes one extra `gateway.invoke()` before the main task execution.
+
+### Inputs to the synthesis model
+
+The synthesis model must see what the downstream task will actually receive — not raw templates with `{{handlebars}}` placeholders, but the **fully rendered, parsed versions** with memory and variables already populated.
+
+1. **Rendered downstream instructions** — the `.instructions` template for `skillKey`, **parsed with the enriched memory bundle and variables** (the same rendering the gateway would do for the main task). This is the actual system prompt the main model will see.
+2. **Rendered downstream prompt** — the `.prompt` template for `skillKey`, **parsed with the enriched memory bundle, variables, and `request.input`**. This is the actual user message the main model will see.
+3. **Source material** — the resolved context source (narrix output, memory, or both) serialized to structured text
+4. **Synthesis instructions** — the synthesis prompt template (see below)
+
+**Why rendered, not raw?** If the `.instructions` file contains `You are an analyst for {{orgName}}. Review the data in light of {{variables.complianceFramework}}...`, the weak model needs to see `You are an analyst for Acme Corp. Review the data in light of PCI-DSS...` — the actual rendered instructions. Otherwise it cannot know what the downstream task cares about. Same for the prompt: if it says `{{input}}`, the synthesis model must see the actual input JSON, not the placeholder.
+
+In practice, this means the strategy must use the same template rendering pipeline that the gateway/executor uses — `generateContextMarkdown` and the content registry's template engine — to produce the rendered strings *before* passing them to the synthesis prompt. The enriched memory bundle (from step 3 of the execution flow) and `request.variables` and `request.input` are the rendering context.
+
+### The Synthesis Prompt Template
+
+This is the system instruction for the synthesis gateway call (the weak model). It is invariant across tasks — the task-specific parts come from the injected **rendered** instructions and prompt.
+
+**Template location (easy to see and update):** The synthesis system and user prompts are loaded from **project files** so they can be edited without changing code. Recommended layout:
+
+```
+templates/
+  synthesis/
+    system.md      ← synthesis system prompt (placeholders below)
+    user.txt       ← synthesis user prompt (one short line)
+```
+
+- **Resolve order:** (1) Load from `templates/synthesis/system.md` and `templates/synthesis/user.txt` (relative to project root or a configurable base path). (2) If a file is missing or unreadable, fall back to the built-in defaults in code so the feature still works.
+- **Placeholders** in `system.md`: `{{rendered_downstream_instructions}}`, `{{rendered_downstream_prompt}}`, `{{source_material}}`. The implementation replaces these when building the synthesis request.
+- **Override:** Request-level `synthesisConfig.synthesisPromptOverride` can still replace the entire system prompt when provided.
+- **Custom synthesizing guidelines (optional):** When `synthesisConfig.customSynthesizingGuidelines` is set, the implementation **adds** it to the instructions sent to the LLM: after the main template (and after the source material section), insert a section `## Additional guidelines` followed by the guidelines text, then the existing `## Your output` section. The weak model sees the base instructions plus these extra points (e.g. domain rules, what to emphasize, or format preferences). The implementation appends this block when the option is present; no placeholder in the template file is required.
+
+Default content (used when the template file is not present) is given below.
+
+**Default system prompt (fallback):**
+
+```
+You are a context synthesizer. Your job is to read raw context material and produce a focused, synthesized input for a downstream AI task.
+
+## Your constraints
+
+- Use ONLY the provided source material. Do not invent, assume, or hallucinate any facts.
+- Your output will be consumed by another AI model as its primary context. Make it count.
+- Be concise but complete — include everything relevant to the downstream task, exclude everything irrelevant.
+- Preserve factual precision: names, numbers, dates, identifiers, severity levels, statuses — keep them exact.
+- Do not explain what you are doing. Do not add meta-commentary. Just produce the synthesized context.
+
+## What the downstream task needs to do
+
+The downstream task has these instructions (its fully rendered system prompt, with variables and memory already populated):
+
+<downstream_instructions>
+{{rendered_downstream_instructions}}
+</downstream_instructions>
+
+The downstream task will receive this user message (the fully rendered prompt, with input and variables already populated):
+
+<rendered_downstream_prompt>
+{{rendered_downstream_prompt}}
+</rendered_downstream_prompt>
+
+## Source material to synthesize from
+
+<source_material>
+{{source_material}}
+</source_material>
+
+## Your output
+
+Produce a synthesized context document that gives the downstream task exactly what it needs. Structure your output to align with what the downstream task instructions describe and what the rendered prompt contains. The instructions and prompt above are fully rendered — they show exactly what the downstream AI model will see. Use that to determine what information from the source material is relevant and how to organize it.
+```
+
+### The Synthesis User Prompt
+
+**File:** `templates/synthesis/user.txt` (or fallback below).
+
+**Default user prompt (fallback):**
+
+```
+Synthesize the source material above for the downstream task. Output only the synthesized context — nothing else.
+```
+
+### Synthesis Model Configuration
+
+The model used for the `AIGateway.invoke()` synthesis call is configured via `synthesisConfig.modelConfig` on the request. It defaults to the environment variable `SYNTHESIS_MODEL` or the implementation default `gpt-5-nano`.
+
+```ts
+// Resolution order for the model passed to gateway.invoke({ config: { model } }):
+// 1. request.synthesisConfig.modelConfig.model
+// 2. process.env.SYNTHESIS_MODEL
+// 3. fallback (implementation default: "gpt-5-nano")
+```
+
+---
+
+## How the Synthesized Output Reaches the Main Task
+
+The synthesized output replaces the standard context markdown. Here's the flow comparison:
+
+### Today (DIRECT)
+
+```
+enrichMemories → generateContextMarkdown → build enrichedInput.context → executor
+```
+
+### Synthesized-Context
+
+```
+enrichMemories → resolve source material
+              → render downstream .instructions + .prompt (with memory, variables, input)
+              → run synthesis call via AIGateway (weak model)
+              → synthesized output becomes enrichedInput.context
+              → executor (main model)
+```
+
+The main model receives the synthesized output in the same `context` field it would normally receive the generated context markdown. From the executor's perspective, nothing changes — it just gets better, pre-digested context.
+
+### Template-Awareness (the "smart" part)
+
+The key insight: the synthesis model sees the **rendered** downstream instructions and prompt — not raw templates, but the fully parsed versions with memory, variables, and input already substituted. The weak model sees *exactly* what the main model will see as its system prompt and user message. It therefore knows precisely what the main model cares about, what tone it should use, what structure the main model expects, and what input fields are in play.
+
+This means the synthesis output naturally aligns with the actual task execution. The downstream model gets context that *fits* its real prompt.
+
+---
+
+## API Contract
+
+### Execution pipeline (replaces single `executionType`)
+
+Execution is driven by an **array** of steps. Each step has a phase and a type. This is a **breaking change** from the previous single `executionType`; see [BREAKING-CHANGES.md](BREAKING-CHANGES.md) for migration.
+
+```ts
+type ExecutionPhase = "pre" | "main" | "post";
+
+interface ExecutionStep {
+  phase: ExecutionPhase;
+  type: string;       // e.g. "synthesized-context", "direct", "narrix-then-direct"
+  config?: unknown;   // step-specific config (e.g. SynthesisConfig for synthesized-context)
+}
+
+interface RunTaskRequest {
+  // ...
+  /** Pipeline of execution steps. Order: all pre (in order) → single main → all post (in order). Default when omitted: [{ phase: "main", type: "direct" }]. */
+  executionPipeline?: ExecutionStep[];
+  // Deprecated / removed: executionType (use executionPipeline)
+}
+```
+
+- **Exactly one** step must have `phase: "main"`. Typically `{ phase: "main", type: "direct" }`.
+- **PRE** steps run first, in array order. Example: `{ phase: "pre", type: "synthesized-context", config: synthesisConfig }`.
+- **POST** steps run last, in array order. Optional; no built-in post types in this spec.
+- When `executionPipeline` is omitted, behavior is equivalent to `[{ phase: "main", type: "direct" }]`.
+
+Example — synthesized-context (PRE) then direct (main):
+
+```ts
+executionPipeline: [
+  { phase: "pre", type: "synthesized-context", config: { modelConfig: { model: "gpt-5-nano" }, contextSourcePolicy: "auto" } },
+  { phase: "main", type: "direct" },
+],
+includeContextInPrompt: true,
+```
+
+### New fields on `RunTaskRequest` (and step configs)
+
+```ts
+type ContextSourcePolicy =
+  | "auto"
+  | "narrix-only"
+  | "narrix+memory"
+  | "memory-only"
+  | "narrix-web"
+  | "narrix-web-memory"
+  | "memory-web";
+
+interface SynthesisConfig {
+  /** Model configuration for the synthesis call. If omitted, uses SYNTHESIS_MODEL env or fallback. */
+  modelConfig?: ModelConfig;
+
+  /**
+   * What feeds synthesis `source_material`. See policy table in "Context Source Resolution".
+   */
+  contextSourcePolicy?: ContextSourcePolicy;
+
+  /**
+   * Options for serializing web scoper results into markdown in source material.
+   */
+  webEvidence?: {
+    preferCleanContent?: boolean;
+    maxSources?: number;
+    dedupeByUrl?: boolean;
+    maxTotalChars?: number;
+  };
+
+  /**
+   * When true, automatically sets includeContextInPrompt to true if not already set.
+   * When false, throws if includeContextInPrompt is not true.
+   * Default: true
+   */
+  autoEnableContext?: boolean;
+
+  /** 
+   * Optional override for the synthesis system prompt. 
+   * When provided, replaces the default synthesis template entirely.
+   * Must include {{rendered_downstream_instructions}}, {{rendered_downstream_prompt}},
+   * and {{source_material}} placeholders.
+   */
+  synthesisPromptOverride?: string;
+
+  /** Timeout in ms for the synthesis call. Default: 30000. */
+  timeoutMs?: number;
+
+  /** Max output length for synthesis result. Default: no limit. */
+  maxOutputLength?: number;
+
+  /**
+   * Memory paths to include when policy involves memory.
+   * Default: all of jobMemory, taskMemory, executionMemory.
+   * Example: ["jobMemory.customerProfile", "jobMemory.incidents"]
+   */
+  memoryPaths?: string[];
+
+  /**
+   * Optional custom synthesizing guidelines. When provided, they are ADDED to the synthesis
+   * instructions sent to the weak model as additional points to follow (e.g. domain rules,
+   * formatting preferences, or what to emphasize). They are appended after the main template
+   * in a "## Additional guidelines" section so the synthesizer sees both the base instructions
+   * and these extra points.
+   */
+  customSynthesizingGuidelines?: string;
+
+  /**
+   * When true, if the synthesis call fails (timeout, gateway error), run the main step anyway
+   * without synthesized context (same as skipping the pre step). Default: false — synthesis
+   * failure is a failure; no fallback to DIRECT.
+   */
+  fallbackToDirect?: boolean;
+}
+
+interface RunTaskRequest {
+  // ... existing fields ...
+
+  /** Execution pipeline (pre → main → post). When a step has type "synthesized-context", that step's config is SynthesisConfig. */
+  executionPipeline?: ExecutionStep[];
+}
+```
+
+### Usage: PRE step `synthesized-context` then main `direct`
+
+```ts
+const result = await tasks.runTask({
+  skillKey: "tasks/security-risk-summary",
+  executionPipeline: [
+    { phase: "pre", type: "synthesized-context", config: { modelConfig: { model: "gpt-5-nano", temperature: 0.2 }, contextSourcePolicy: "auto" } },
+    { phase: "main", type: "direct" },
+  ],
+  includeContextInPrompt: true,
+  input: { assetId: "a-123", windowDays: 30 },
+
+  // Standard fields — memory is the source material when no narrix
+  jobMemory: {
+    incidents: [ /* ...large array... */ ],
+    assetProfile: { /* ...detailed profile... */ },
+    historicalAlerts: [ /* ...hundreds of alerts... */ ],
+  },
+  taskMemory: { previousFindings: [ /* ... */ ] },
+});
+```
+
+### Usage with Narrix
+
+```ts
+const result = await tasks.runTask({
+  skillKey: "tasks/security-risk-summary",
+  executionPipeline: [
+    { phase: "pre", type: "synthesized-context", config: { modelConfig: { model: "gpt-5-nano" }, contextSourcePolicy: "narrix-only" } },
+    { phase: "main", type: "direct" },
+  ],
+  includeContextInPrompt: true,
+  input: { assetId: "a-123" },
+  narrix: { datasetId: "ds-security" },
+  jobMemory: { record: { /* ... */ } },
+});
+```
+
+### Usage with memory path filtering
+
+```ts
+const result = await tasks.runTask({
+  skillKey: "tasks/incident-triage",
+  executionPipeline: [
+    { phase: "pre", type: "synthesized-context", config: { contextSourcePolicy: "memory-only", memoryPaths: [
+      "jobMemory.currentIncident",
+      "jobMemory.customerContext",
+      "taskMemory.previousAssessments",
+    ] } },
+    { phase: "main", type: "direct" },
+  ],
+  includeContextInPrompt: true,
+  input: { question: "What is the severity?" },
+  jobMemory: {
+    currentIncident: { /* relevant */ },
+    customerContext: { /* relevant */ },
+    hugeIrrelevantDump: { /* ignored because not in memoryPaths */ },
+  },
+});
+```
+
+### Usage with custom synthesizing guidelines
+
+```ts
+const result = await tasks.runTask({
+  skillKey: "tasks/security-risk-summary",
+  executionPipeline: [
+    { phase: "pre", type: "synthesized-context", config: {
+    modelConfig: { model: "gpt-5-nano" },
+    contextSourcePolicy: "auto",
+    customSynthesizingGuidelines:
+      "Emphasize any finding that affects PCI-DSS scope or requires disclosure.\n" +
+      "Keep executive summary to under 200 words.\n" +
+      "Include severity and date for each risk factor.",
+  } },
+    { phase: "main", type: "direct" },
+  ],
+  includeContextInPrompt: true,
+  input: { assetId: "a-123" },
+  jobMemory: { /* ... */ },
+});
+```
+
+The weak model receives the standard synthesis instructions **plus** an "## Additional guidelines" section containing the above text, so it can follow both the base rules and these extra points when producing the synthesized context.
+
+---
+
+## Execution Flow (Normative)
+
+### Full sequence when pipeline has a PRE step of type `synthesized-context`
+
+(The pipeline runs: PRE steps in order → MAIN step → POST steps. Below is the flow for the PRE step "synthesized-context" and the following MAIN step "direct".)
+
+```
+1. VALIDATE
+   ├── executionPipeline has exactly one step with phase "main"
+   ├── for the pre step with type "synthesized-context": step.config (SynthesisConfig) present or use defaults
+   ├── includeContextInPrompt is true (or autoEnableContext forces it)
+   └── if contextSourcePolicy is "narrix-only", verify narrix config exists
+
+2. NARRIX PRE-PROCESSOR (if request.narrix is set)
+   ├── run narrix enrichment (same as today)
+   └── capture narrix attachment (do NOT inject into memory yet)
+
+3. ENRICH MEMORIES
+   └── standard enrichment path (same as _executeDirect today)
+
+4. RESOLVE SOURCE MATERIAL (from pre step's config)
+   ├── based on contextSourcePolicy:
+   │   ├── "auto" → narrix attachment if available, else enriched memory bundle
+   │   ├── "narrix-only" → narrix attachment only
+   │   ├── "narrix+memory" → merge narrix attachment + enriched memory bundle
+   │   └── "memory-only" → enriched memory bundle only
+   ├── if pre step's config.memoryPaths specified → filter memory to only those paths
+   └── serialize to structured text
+
+5. RENDER DOWNSTREAM TASK TEMPLATES
+   ├── resolve {skillKey}.instructions from content registry
+   ├── resolve {skillKey}.prompt from content registry
+   ├── render .instructions with enriched memory bundle + request.variables → rendered_instructions
+   └── render .prompt with enriched memory bundle + request.variables + request.input → rendered_prompt
+   (same template engine the gateway/executor uses — the result is what the main model WOULD see)
+
+6. BUILD SYNTHESIS PROMPT
+   ├── populate synthesis template with:
+   │   ├── {{rendered_downstream_instructions}} → step 5 rendered instructions
+   │   ├── {{rendered_downstream_prompt}} → step 5 rendered prompt
+   │   └── {{source_material}} → step 4 serialized source
+   ├── if pre step's config.customSynthesizingGuidelines is set: append "\n\n## Additional guidelines\n\n" + customSynthesizingGuidelines before the "## Your output" section (or at end of template)
+   └── build synthesis request for AIGateway
+
+7. RUN SYNTHESIS CALL (AIGateway) — model from pre step's config.modelConfig or env
+   ├── gateway = new AIGateway()
+   ├── gateway.invoke({ instructions: synthesisSystemPrompt, workingMemory: { input: synthesisUserPrompt }, config: { model: weakModel } })
+   ├── extract text response (same extractTextFromResponse pattern as runScopingCall)
+   ├── trim + enforce maxOutputLength
+   └── result = synthesized context string
+
+8. BUILD ENRICHED INPUT
+   ├── context = synthesized context string (replaces generateContextMarkdown output)
+   ├── input = request.input (unchanged — the synthesis is in context, not input)
+   └── everything else same as _executeDirect
+
+9. EXECUTE MAIN TASK
+   └── executor.execute(enrichedInput) — main model runs with synthesized context
+
+10. POST-PROCESS
+    ├── liftIntermediateSteps
+    ├── add synthesis step to intermediateSteps:
+    │   { step: 1, id: "synthesis", ok: true, summary: "context synthesized" }
+    └── return result (no POST steps in this spec; pipeline ends after main)
+```
+
+### Error handling
+
+| Failure point | Behavior |
+|---|---|
+| Narrix fails (when narrix is configured) | Throw — same as current narrix pre-processor behavior |
+| Synthesis call fails | By default throw. When `synthesisConfig.fallbackToDirect === true`, run main step without synthesized context instead (no throw). |
+| Synthesis call times out | Throw timeout error |
+| Template fetch fails (instructions/prompt not found) | Fall back to synthesis without downstream template context; log warning |
+| `includeContextInPrompt` not set and `autoEnableContext` is false | Throw configuration error |
+
+---
+
+## Relationship to Existing Patterns
+
+### vs. DIRECT
+
+DIRECT passes raw memory/context to the main model. Synthesized-context adds a pre-pass. Think of it as: `synthesized-context = synthesis(context) → DIRECT`.
+
+### vs. narrix-then-direct
+
+`narrix-then-direct` runs narrix and injects results into taskMemory, then runs DIRECT. `synthesized-context` can *include* narrix in its pipeline, but goes further — it synthesizes the narrix output (and/or memory) before the main call.
+
+### vs. aiScoping
+
+aiScoping scopes individual memory paths with separate LLM calls per path. Synthesized-context synthesizes the *entire* context holistically in one call. They can coexist: aiScoping can run *after* synthesis if both are configured, though in practice you'd typically use one or the other.
+
+### Composability
+
+```
+narrix → synthesized-context → aiScoping → DIRECT execution
+```
+
+All layers are optional and stack. The synthesized-context strategy handles the narrix + synthesis portion, then delegates to the DIRECT path for the final execution (including aiScoping if configured).
+
+---
+
+## Builder Extension
+
+```ts
+class TaskRequestBuilder {
+  // ... existing methods ...
+
+  /** Set the execution pipeline (pre / main / post steps). */
+  withExecutionPipeline(steps: ExecutionStep[]): this {
+    this.request.executionPipeline = steps;
+    return this;
+  }
+
+  /** Add a PRE step: synthesized-context with optional config. Sets includeContextInPrompt true. */
+  withSynthesizedContextPreStep(modelOrConfig?: string | SynthesisConfig): this {
+    const config = typeof modelOrConfig === "string" ? { modelConfig: { model: modelOrConfig } } : modelOrConfig ?? {};
+    const steps = this.request.executionPipeline ?? [{ phase: "main", type: "direct" }];
+    const preSteps = steps.filter(s => s.phase === "pre");
+    const mainStep = steps.find(s => s.phase === "main") ?? { phase: "main" as const, type: "direct" };
+    const postSteps = steps.filter(s => s.phase === "post");
+    this.request.executionPipeline = [
+      ...preSteps,
+      { phase: "pre" as const, type: "synthesized-context", config },
+      mainStep,
+      ...postSteps,
+    ];
+    this.request.includeContextInPrompt = true;
+    return this;
+  }
+}
+```
+
+---
+
+## Synthesis Template Files (project-local, editable)
+
+So that prompts can be changed without code changes, the synthesis system and user prompts are read from the project:
+
+| File | Purpose |
+|---|---|
+| `templates/synthesis/system.md` | System prompt for the synthesis (weak) model. Placeholders: `{{rendered_downstream_instructions}}`, `{{rendered_downstream_prompt}}`, `{{source_material}}`. |
+| `templates/synthesis/user.txt` | User prompt for the synthesis call (e.g. one line: "Synthesize the source material above..."). |
+
+- **Base path:** Resolved relative to process cwd or a configurable base (e.g. `synthesisConfig.templatesBasePath` or env `SYNTHESIS_TEMPLATES_PATH`). Default: project root (cwd) so `templates/synthesis/` is at repo root.
+- **Fallback:** If a file is missing, the implementation uses the built-in default text (see "The Synthesis Prompt Template" and "The Synthesis User Prompt" above).
+- **Override:** `synthesisConfig.synthesisPromptOverride` on the request still replaces the entire system prompt when provided, and takes precedence over the file.
+
+---
+
+## Environment Variables
+
+| Variable | Purpose | Default |
+|---|---|---|
+| `SYNTHESIS_MODEL` | Default model for synthesis calls when not specified in request | `gpt-5-nano` |
+| `SYNTHESIS_TIMEOUT_MS` | Default timeout for synthesis calls | `30000` |
+| `SYNTHESIS_MAX_OUTPUT_LENGTH` | Default max output length | No limit |
+| `SYNTHESIS_TEMPLATES_PATH` | Optional base path for `templates/synthesis/` (system.md, user.txt). When unset, use cwd. | (none — use cwd) |
+
+---
+
+## Template Resolution and Rendering — How the Strategy Gets Downstream Instructions
+
+The strategy needs to produce the **fully rendered** downstream instructions and prompt — what the main model would actually see. This is a two-step process: resolve the raw templates from the content registry, then render them with the actual memory/variables/input.
+
+### Step 1: Resolve raw templates from content registry
+
+```ts
+const skillId = stripPrefix(request.skillKey); // "tasks/foo" → "foo"
+
+// Same resolution chain as the gateway (see SKILL-CONTENT-GUIDE.md)
+let rawInstructions = await contentRegistry.resolve(`${skillId}.instructions`);
+let rawPrompt = await contentRegistry.resolve(`${skillId}.prompt`);
+
+// Fallback: if .instructions missing, try base skillId
+if (!rawInstructions) {
+  rawInstructions = await contentRegistry.resolve(skillId);
+}
+
+// If prompt template missing, use "{{input}}" as default
+if (!rawPrompt) {
+  rawPrompt = "{{input}}";
+}
+```
+
+### Step 2: Render templates with actual data
+
+```ts
+// Use the same template engine the gateway uses (handlebars-style rendering)
+// The rendering context includes everything the main task execution would have:
+const renderContext = {
+  input: request.input,                    // the task input object
+  ...request.variables,                    // orgName, tone, etc.
+  jobMemory: enrichedBundle.jobMemory,     // enriched job memory
+  taskMemory: enrichedBundle.taskMemory,   // enriched task memory
+  executionMemory: enrichedBundle.executionMemory,
+};
+
+const renderedInstructions = templateEngine.render(rawInstructions, renderContext);
+const renderedPrompt = templateEngine.render(rawPrompt, renderContext);
+```
+
+**The result**: `renderedInstructions` is the exact system prompt the main model will see (e.g. `"You are a security analyst for Acme Corp. Assess the risk..."`). `renderedPrompt` is the exact user message (e.g. the full JSON input or a structured question). These rendered strings go into the synthesis prompt so the weak model knows precisely what the main model expects.
+
+This mirrors the existing resolution logic described in the Skill Content Guide — same two-file convention, same fallback chain. The strategy resolves and renders them *before* the synthesis call, rather than letting the gateway resolve them during the main execution.
+
+---
+
+## What the Main Model Sees (Before/After Comparison)
+
+### Before (DIRECT with raw context)
+
+```
+[System] {instructions from .instructions file}
+
+[Context message] 
+## Scoping and discovery
+### Scoping
+**Signals:**
+- HIGH_RISK_VENDOR
+- DATA_BREACH_INDICATOR
+- COMPLIANCE_GAP_PCI
+...50 more signals...
+
+**Stories:**
+- incident-timeline
+- vendor-risk-narrative
+...20 more stories...
+
+### Discovery
+**Signals:**
+...another 30 signals...
+
+[User] {prompt template populated with raw input}
+```
+
+### After (synthesized-context)
+
+```
+[System] {instructions from .instructions file}
+
+[Context message]
+This asset (vendor-acme-prod, third-party vendor) presents elevated security risk 
+based on three converging factors:
+
+1. Active data breach indicator (HIGH severity) detected on 2025-12-15 involving 
+   PII exposure in the vendor's staging environment.
+2. PCI compliance gap identified in Q4 audit — missing requirement 6.5.1 
+   (injection flaws protection).  
+3. Vendor risk score escalated from 62→89 after December incident.
+
+Previous assessment (2025-11-01) rated this vendor as moderate risk. The breach 
+and compliance gap represent material change.
+
+Key identifiers: assetId=a-123, vendorId=v-acme, datasetId=ds-security.
+Window: last 30 days.
+
+[User] {prompt template populated with original input}
+```
+
+The main model can now focus on its actual job — producing the security risk summary — instead of parsing raw signals and stories.
+
+---
+
+## Full Flow Example — One Complete Run
+
+This section walks through a single execution of `executionType: "synthesized-context"` with concrete request data, so you can see exactly what is resolved, what is sent to the weak model, and what the main model receives.
+
+### 1. Incoming request
+
+```ts
+await tasks.runTask({
+  skillKey: "tasks/security-risk-summary",
+  executionPipeline: [
+    { phase: "pre", type: "synthesized-context", config: { modelConfig: { model: "gpt-5-nano", temperature: 0.2 }, contextSourcePolicy: "narrix-only" } },
+    { phase: "main", type: "direct" },
+  ],
+  includeContextInPrompt: true,
+  input: {
+    assetId: "a-123",
+    windowDays: 30,
+    format: "executive-bullet",
+  },
+  variables: {
+    orgName: "Acme Corp",
+    complianceFramework: "PCI-DSS",
+  },
+  narrix: { datasetId: "ds-security" },
+  jobMemory: {
+    record: { /* raw record that narrix will consume */ },
+  },
+});
+```
+
+### 2. What happens inside the pipeline
+
+**Step 1 — Validate**
+
+- `synthesisConfig` present (with defaults applied: `contextSourcePolicy: "narrix-only"`, `autoEnableContext: true`).
+- `includeContextInPrompt` is true.
+- `contextSourcePolicy` is `"narrix-only"` and `request.narrix` is set → valid.
+
+**Step 2 — Narrix pre-processor**
+
+- Narrix runs on `jobMemory.record` (or adapted input) and produces signals/stories.
+- Result is turned into `NarrixEnrichedAttachment` and stored on `request.executionMemory._narrix` (and `request.jobMemory._narrix`).
+- No change to how this works today; the attachment is now available for the synthesis step.
+
+**Step 3 — Enrich memories**
+
+- Standard enrichment: `enrichMemoriesWithScoping("security-risk-summary", "task", memoryBundle)`.
+- Output: `enrichedBundle` (jobMemory, taskMemory, executionMemory with scoping applied, execution cleansed).
+
+**Step 4 — Resolve source material**
+
+- Policy is `"narrix-only"`, so only the narrix attachment is used.
+- Source material is built from `request.executionMemory._narrix` (scoping + discovery signals/stories) and serialized to text, for example:
+
+```
+## Narrix output (scoping)
+Signals: HIGH_RISK_VENDOR, DATA_BREACH_INDICATOR, COMPLIANCE_GAP_PCI, VENDOR_RISK_ESCALATION
+Stories: incident-timeline, vendor-risk-narrative, compliance-gap-audit
+
+## Narrix output (discovery)
+Signals: PII_EXPOSURE_STAGING, ...
+Stories: ...
+```
+
+- This string becomes `source_material` in the synthesis prompt.
+
+**Step 5 — Render downstream task templates**
+
+- Skill id derived from `skillKey`: `"security-risk-summary"`.
+- Content registry is asked for `security-risk-summary.instructions` and `security-risk-summary.prompt`.
+- Raw instructions (example):
+
+```
+You are a security analyst for {{orgName}}. Produce a concise executive risk summary.
+Focus on factors relevant to {{complianceFramework}}. Be factual; do not speculate.
+```
+
+- Raw prompt (example): `{{input}}` or `Summarize risk for asset {{input.assetId}} over the last {{input.windowDays}} days. Output format: {{input.format}}.`
+- Render context: `{ input: request.input, orgName: "Acme Corp", complianceFramework: "PCI-DSS", jobMemory: enrichedBundle.jobMemory, taskMemory: ..., executionMemory: ... }`.
+- **Rendered instructions** (what the main model will see as system prompt):
+
+```
+You are a security analyst for Acme Corp. Produce a concise executive risk summary.
+Focus on factors relevant to PCI-DSS. Be factual; do not speculate.
+```
+
+- **Rendered prompt** (what the main model will see as user message):
+
+```
+Summarize risk for asset a-123 over the last 30 days. Output format: executive-bullet.
+```
+
+- These two strings are passed into the synthesis prompt as `rendered_downstream_instructions` and `rendered_downstream_prompt`.
+
+**Step 6 — Build synthesis prompt**
+
+- Default synthesis system template is filled with:
+  - `{{rendered_downstream_instructions}}` → the rendered instructions above.
+  - `{{rendered_downstream_prompt}}` → the rendered prompt above.
+  - `{{source_material}}` → the serialized narrix output from step 4.
+- Synthesis user message: `"Synthesize the source material above for the downstream task. Output only the synthesized context — nothing else."`
+
+**Step 7 — Run synthesis call (AIGateway)**
+
+- Gateway is invoked with:
+  - `instructions`: the populated synthesis system prompt (with downstream instructions, prompt, and source material).
+  - `workingMemory: { input: synthesisUserPrompt }`.
+  - `config: { model: "gpt-5-nano" }`.
+- Weak model returns plain text, e.g.:
+
+```
+This asset (vendor-acme-prod, third-party vendor) presents elevated security risk based on three converging factors:
+
+1. Active data breach indicator (HIGH severity) detected on 2025-12-15 involving PII exposure in the vendor's staging environment.
+2. PCI compliance gap identified in Q4 audit — missing requirement 6.5.1 (injection flaws protection).
+3. Vendor risk score escalated from 62→89 after December incident.
+
+Previous assessment (2025-11-01) rated this vendor as moderate risk. The breach and compliance gap represent material change.
+
+Key identifiers: assetId=a-123, vendorId=v-acme, datasetId=ds-security. Window: last 30 days.
+```
+
+- This string is trimmed and optionally truncated by `maxOutputLength`; it becomes `synthesizedContext`.
+
+**Step 8 — Build enriched input for main task**
+
+- Same shape as in DIRECT: `enrichedInput = { ...request, skillKey, jobMemory, taskMemory, executionMemory, context, input }`.
+- **Only difference**: `context` is set to `synthesizedContext` (the weak model’s output) instead of `generateContextMarkdown(...)` or raw narrix markdown.
+- `input` is unchanged: `{ assetId: "a-123", windowDays: 30, format: "executive-bullet" }`.
+
+**Step 9 — Execute main task**
+
+- `executor.execute(enrichedInput)` is called.
+- The main model (e.g. GPT-4) receives:
+  - **System**: the same rendered instructions (“You are a security analyst for Acme Corp…”).
+  - **Context message**: the synthesized text above (no raw signals/stories list).
+  - **User**: the same rendered prompt (“Summarize risk for asset a-123…”).
+- The main model produces the executive risk summary using only the synthesized context.
+
+**Step 10 — Post-process**
+
+- Response is returned; `intermediateSteps` is updated so the first step is `{ step: 1, id: "synthesis", ok: true, summary: "context synthesized" }`, and any steps from the main task are renumbered (2, 3, …).
+
+### 3. Same flow without Narrix (memory-only)
+
+If the same request had **no** `narrix` and `contextSourcePolicy: "memory-only"` (or `"auto"` with no narrix):
+
+- Step 2 (narrix) is skipped; there is no `_narrix` attachment.
+- Step 4 uses the **enriched memory bundle** as source: e.g. `jobMemory`, `taskMemory`, `executionMemory` serialized (optionally filtered by `memoryPaths`). For example, `jobMemory` might contain `incidents`, `assetProfile`, `historicalAlerts`.
+- Steps 5–10 are unchanged: the weak model still receives the **rendered** instructions and prompt, but the **source material** is now the serialized memory (e.g. JSON or a markdown summary of those keys) instead of narrix output.
+- The rest of the flow (synthesis call, override context, main task, intermediate steps) is the same.
+
+### 4. Summary diagram
+
+```
+Request (synthesized-context, narrix-only)
+    │
+    ├─► Narrix pre-processor → executionMemory._narrix
+    │
+    ├─► Enrich memories → enrichedBundle
+    │
+    ├─► Resolve source material → "Narrix output: signals/stories..."
+    │
+    ├─► Render templates → "You are a security analyst for Acme Corp...", "Summarize risk for asset a-123..."
+    │
+    ├─► Build synthesis prompt → system + user for weak model
+    │
+    ├─► AIGateway.invoke(weak model) → synthesizedContext string
+    │
+    ├─► enrichedInput.context = synthesizedContext
+    │
+    └─► executor.execute(enrichedInput) → main model sees only synthesized context
+```
+
+This is the full flow as it would look like end-to-end for one run.
+
+---
+
+## Acceptance Criteria
+
+1. Execution uses **execution pipeline** (array of steps: pre → main → post); `executionPipeline` replaces single `executionType`.
+2. PRE step type `"synthesized-context"` is supported; when used, synthesis runs **before** the main task using a configurable weak model.
+3. Synthesis prompt includes rendered downstream task instructions and prompt.
+4. Source material comes from narrix (when available) or memory, controlled by `contextSourcePolicy`.
+5. Synthesized output is delivered to the main model via the context message (`enrichedInput.context`).
+6. Requires `includeContextInPrompt: true` (with configurable auto-enable).
+7. Works with narrix, without narrix, and with memory path filtering; optional `fallbackToDirect`, custom guidelines, template files in project.
+8. Synthesis step appears in `intermediateSteps`; synthesis is **non-streaming** (complete response only).
+9. Builder has `withExecutionPipeline` and `withSynthesizedContextPreStep`.
+10. **README is very clear** about all new abilities and changes: breaking change (link to BREAKING-CHANGES.md), pipeline model, synthesized-context PRE step, SynthesisConfig options, template files, caching (nx-cache), non-streaming, and link to this spec.
+
+---
+
+## Resolved implementation decisions
+
+1. **Synthesis failures and fallback to DIRECT:** **No** by default. Synthesis failure is a failure. A config option (e.g. `fallbackToDirect?: boolean`) may be provided so callers can opt in to falling back to running the main task without synthesized context; **default is false** (no fallback).
+
+2. **Template fallback location:** Fallback content lives **in the project**, not inside the code. The default synthesis prompts are the checked-in files under `templates/synthesis/` (system.md, user.txt). If those files are missing, the implementation may use a minimal in-code fallback or fail with a clear error pointing to the project path; the canonical “default” is the project template files, not strings in source.
+
+3. **Caching:** Yes. Use the **nx-cache** package (already in the project) to cache synthesis results when the same inputs (e.g. source material + task instructions / skillKey + config) are seen again, to reduce cost for batch or repeated runs. Cache key and TTL are implementation-defined.
+
+4. **Streaming:** No at this point. The synthesis call returns a complete response; no streaming. This should be **clearly stated in the README** so callers know synthesis is non-streaming.