@exellix/ai-tasks 7.8.0

package/documenations/web-scoping-in-ai-tasks.md

@@ -0,0 +1,503 @@
+## Web scoping in `@exellix/ai-tasks`
+
+This document explains, in technical detail, **how web scoping is wired into ai-tasks**, how requests opt into it, what data flows through, and what guarantees / failure modes exist. It is intended for engineers integrating with ai-tasks (e.g. graph builders, skill authors) and for maintainers changing the web scoping plumbing.
+
+---
+
+## 1. High-level overview
+
+- **Goal**: Enrich a NARRIX run with **external web context** (via `@exellix/narrix-web-scoper`) and expose that context to local tasks through `executionMemory.webContext`.
+- **Trigger**: A boolean flag on the task request:
+
+  ```ts
+  RunTaskRequest.narrix = {
+    datasetId: "...",
+    enableWebScope: true,   // ← opt-in; default is false/absent
+  };
+  ```
+
+- **Where it runs**:
+  - After a NARRIX run completes successfully (CNI ingested + runner executed).
+  - Implemented in the **task SDK**, in the same flow that invokes the local NARRIX handler.
+- **Where the result lands**:
+
+  ```ts
+  ctx.executionMemory.webContext: WebScoperResult
+  ```
+
+  where `WebScoperResult` is defined by `@exellix/narrix-web-scoper` and is always present when `enableWebScope === true`, even on miss/error.
+
+- **Failure stance**: **Lenient.** If web scoping fails or misses, the NARRIX task still runs and `_narrix` output is still attached. Web scoping only affects `executionMemory.webContext` and observability.
+
+---
+
+## 2. Request contract: enabling web scoping
+
+### 2.1. Request shape
+
+ai-tasks exposes NARRIX as a **local task** in the task SDK. The entry-point for callers is a `RunTaskRequest` that may include a `narrix` section used by the NARRIX pre-processor.
+
+The relevant part of the request is:
+
+```ts
+// src/types/task-types.ts (conceptual)
+export interface NarrixPreProcessorConfig {
+  datasetId: string;
+  attachToField?: string;
+  enableWebScope?: boolean; // ← added for web scoping
+}
+
+export interface RunTaskRequest {
+  skillKey: string;
+  input: unknown;
+  narrix?: NarrixPreProcessorConfig;
+  // ... other fields: executionMemory, variables, etc.
+}
+```
+
+- **`narrix.datasetId`**: required; selects which NARRIX dataset / pack to run.
+- **`narrix.enableWebScope`**:
+  - **`true`** → ai-tasks will attempt a web scoping call after NARRIX completes.
+  - **absent / `false`** → web scoping is **completely skipped** for this request.
+
+### 2.2. Example from a graph builder (worox-graphs)
+
+Downstream (e.g. `worox-graphs`) forwards configuration from node metadata into the NARRIX config:
+
+```jsonc
+{
+  "metadata": {
+    "narrix": {
+      "datasetId": "network.vuln.instances",
+      "enableWebScope": true
+    }
+  }
+}
+```
+
+In TypeScript:
+
+```ts
+narrix: {
+  datasetId: node.taskConfiguration.narrix.datasetId,
+  attachToField: node.taskConfiguration.narrix.attachToField,
+  enableWebScope: node.taskConfiguration.narrix.enableWebScope ?? false,
+}
+```
+
+Only when that `enableWebScope` flag is `true` will ai-tasks invoke the web scoper.
+
+---
+
+## 3. Execution flow inside ai-tasks
+
+### 3.1. NARRIX run
+
+The normal NARRIX pipeline in ai-tasks is:
+
+1. **Ingest** raw input into CNI via `@exellix/narrix-ingest`.
+2. **Run** the NARRIX runner (`@exellix/narrix-runner`) with the CNI and `datasetId`.
+3. **Attach** the NARRIX result to the task’s execution memory.
+
+This produces a **success result** with a shape similar to:
+
+```ts
+export interface NarrixRunOutputSuccess {
+  entity: unknown;
+  signals: unknown;
+  stories: unknown;
+  passes: unknown;
+  meta: unknown;
+  cni?: unknown; // raw CNI (typed more strictly by narrix packages)
+}
+```
+
+The success result is attached under `executionMemory._narrix` (or under `attachToField` when configured).
+
+### 3.2. Web scoping hook
+
+After a successful NARRIX run, the task SDK checks `narrix.enableWebScope`:
+
+1. If **absent/false**:
+   - **No web scoping call is made.**
+   - `executionMemory.webContext` remains `undefined`.
+2. If **true**:
+   - ai-tasks builds scope fields via **`resolveWebScopeQuestionAndTemplates`** in `src/narrix/buildWebScopeScopeInput.ts`, then either calls **`runWebScope()`** (`scopeGeneric` / `ScopeInput`) or **`runWebScopeQuestionPack()`** (`scopeQuestionPack` / explicit questions) from `src/narrix/webScoper.ts`.
+   - The result is normalized to **`WebScoperResult`** and written into `executionMemory.webContext` **regardless of hit/miss/error**.
+
+#### 3.2.1 Search question / templates / explicit questions (ISSUE-006, `@exellix/narrix-web-scoper` 1.2.0+)
+
+Optional fields on **`RunTaskRequest.narrix`**:
+
+| Field | Purpose |
+|--------|--------|
+| **`webScopeTemplates`** | Non-empty array → passed as `ScopeInput.webScopeTemplates`. Uses **athenix-parser** token syntax in the web-scoper; **replaces** normal query planning when set. |
+| **`webScopeObjects`** | Merged on top of an **auto-built** context (CNI `knownFacts`, common `entity.graphized.vulnerability.*` fields, `question` / `baseQuestion`, `entityKey` / `label`). |
+| **`webScopeQuestionTemplate`** | When **`webScopeTemplates` is not set**, optional **Handlebars** string to produce the search `question` from that context plus `input` (full task input). |
+| **`webScopeQuestions`** | When **`webScopeTemplates` is not set**, optional list of explicit scope questions. Non-empty after trim → **`scopeQuestionPack`** with `WebScopeQuestion[]` (`id` generated when omitted). Shape: `Array<{ id?: string; question: string; source?: "manual" \| "ai-driven" }>`. |
+| **`webScoping`** | Optional slice of **`WebScoperConfig.scoping`** from `@exellix/narrix-web-scoper` (snippets, caps, raw content, `maxQueries`, `freshnessDays`, etc.). When non-empty, ai-tasks builds a scoper for that request with `createWebScoper({ enabled: true, searchAdapter, scoping: narrix.webScoping })` so behavior matches upstream docs. Omit or leave empty to use the package default scoper (backward compatible). |
+
+**Precedence:**
+
+1. If **`webScopeTemplates`** is set (after trimming empty entries) → scoper receives **`webScopeTemplates`** + merged **`webScopeObjects`** only (no `question` / **`webScopeQuestions`** from this resolver).
+2. Else if **`webScopeQuestions`** is non-empty after normalization → **`runWebScopeQuestionPack`** with **`questions`** only (no single **`question`** / Handlebars path).
+3. Else if **`webScopeQuestionTemplate`** is set → **`question`** = Handlebars render; on template error, fall back to default enrichment.
+4. Else → **`question`** = `input.question` (or string `input`) **plus** appended **CVE / vendor / product** tokens from context when those tokens are **not** already contained in the base question (case-insensitive).
+
+When there is no base question and no enrichable facts, **`question`** may be omitted (empty resolver result).
+
+Conceptually, the call looks like:
+
+```ts
+import type { ScopeInput, WebScoperResult } from "@exellix/narrix-web-scoper";
+import { runWebScope, runWebScopeQuestionPack } from "../narrix/webScoper.js";
+import { resolveWebScopeQuestionAndTemplates } from "../narrix/buildWebScopeScopeInput.js";
+
+const scopeFields = resolveWebScopeQuestionAndTemplates({
+  narrix: narrixConfig,
+  requestInput: request.input,
+  successResult,
+  entity: /* record medium only */,
+});
+
+const base = {
+  datasetId: narrixConfig.datasetId,
+  subjectId: successResult.entity.entityKey,
+  entityKind: successResult.entity.entityKind,
+  entity: /* record */,
+  cni: successResult.cni,
+};
+
+const webResult: WebScoperResult = scopeFields.packQuestions?.length
+  ? await runWebScopeQuestionPack({ ...base, questions: scopeFields.packQuestions })
+  : await runWebScope({ ...base, ...scopeFields } as ScopeInput);
+
+ctx.executionMemory.webContext = webResult;
+```
+
+The **NARRIX output attachment** (`executionMemory._narrix` or the configured `attachToField`) happens independently of web scoping and is not affected by web scoping failures.
+
+---
+
+## 4. `runWebScope` and `WebScoperResult`
+
+### 4.1. The `runWebScope` helper
+
+`src/narrix/webScoper.ts` provides a thin, testable wrapper around `@exellix/narrix-web-scoper`:
+
+- Maintains a **singleton** `NarrixWebScoper` instance in production.
+- Allows tests to **inject** a fake scoper via `setWebScoperForTesting`.
+- Normalizes failures into a consistent `WebScoperResult`.
+
+The tests in `test/narrix/webScoper.test.ts` show its behavior:
+
+- **Happy path**:
+  - When the injected scoper returns `{ available: true, context, cached: false }`,
+  - `runWebScope` forwards that as-is, and tests assert:
+
+    ```ts
+    assert.strictEqual(result.available, true);
+    assert.deepStrictEqual(result.context, webContext);
+    ```
+
+- **Miss / ineligibility**:
+  - When scoper returns `{ available: false, reason: "notEligible" }`,
+  - `runWebScope` passes this through without throwing:
+
+    ```ts
+    assert.strictEqual(result.available, false);
+    assert.strictEqual(result.reason, "notEligible");
+    ```
+
+- **Error handling**:
+  - When the scoper throws (e.g. network error, adapter unavailable),
+  - `runWebScope` **catches** the error and returns:
+
+    ```ts
+    { available: false, reason: "error", error: "<message>" }
+    ```
+
+  - Tests assert:
+
+    ```ts
+    assert.strictEqual(result.available, false);
+    assert.strictEqual(result.reason, "error");
+    assert.ok(result.error?.includes("search timed out"));
+    ```
+
+### 4.2. Result contract
+
+The effective contract, as used by ai-tasks, is:
+
+```ts
+// Hit
+{
+  available: true;
+  context: WebContext; // rich object with findings, queriesUsed, sources, timestamps, ...
+  cached: boolean;
+}
+
+// Miss or error
+{
+  available: false;
+  reason: WebScoperMissReason | "error";
+  error?: string; // present when reason === "error"
+}
+```
+
+This shape is **stable from the perspective of local tasks**: they only need to check `available` and, optionally, `reason`/`error`.
+
+### 4.3. Source bodies and snippets (`WebSource`)
+
+Upstream behavior is implemented in **`@exellix/narrix-web-scoper`** (and Tavily field mapping in **`@exellix/search-adapter`**). ai-tasks does not map provider JSON; it forwards **`RunTaskRequest.narrix.webScoping`** into the scoper factory when set. Field names and semantics follow the **`@exellix/narrix-web-scoper` README** (current releases use first-class **`providerContent`** / **`providerRawContent`**; **`content`** / **`rawContent`** are legacy mirrors for the same roles).
+
+When **`webScoping.includeSourceSnippets`** is **`true`**, each `WebContext.sources[]` entry may include:
+
+- **`providerContent`** / **`providerRawContent`**: bounded excerpt and optional raw payload from the adapter (when requested).
+- **`content`** / **`rawContent`**: deprecated mirrors of **`providerContent`** / **`providerRawContent`**.
+- **`snippet`**: primary excerpt for the source, chosen via **`webScoping.sourceExcerptFrom`** (default **`providerContent`**, or **`providerRawContent`**, with deprecated aliases **`content`** / **`rawContent`**).
+- **`snippetCharCount`**, **`contentOrigin`**, **`retrievalStage`**, **`contentSource`**, **`score`** / **`rank`**: provenance and metadata when the adapter supplies them (see web-scoper README).
+
+Defaults are backward compatible: **`includeSourceSnippets` defaults to false**, so these fields are omitted unless you opt in (via **`narrix.webScoping`** or the scoper’s own defaults when extended).
+
+Optional caps (only apply when snippets are enabled):
+
+- **`maxSnippetCharsPerSource`**: Unicode cap applied per source to **`providerContent`**, **`providerRawContent`** (and legacy mirrors), and the text chosen for **`snippet`**. When set to a positive number, it is also forwarded as **`snippetMaxChars`** on the shared search request so the adapter can normalize earlier.
+- **`maxTotalWebContextChars`**: additional budget applied **only** to **`WebSource.snippet`** across sources (after each snippet’s per-source cap). It does **not** shrink stored **`providerContent`** / **`providerRawContent`**.
+
+To request raw body text from the provider, set **`webScoping.snippetIncludeRawContent`** (e.g. **`true`** or **`"markdown"`**); it is forwarded as **`includeRawContent`** on search requests. Use **`@exellix/search-adapter`** and **`@exellix/narrix-web-scoper`** versions from the same release family as this repo’s dependencies.
+
+Example:
+
+```ts
+narrix: {
+  datasetId: "network.vuln.instances",
+  enableWebScope: true,
+  webScoping: {
+    includeSourceSnippets: true,
+    maxQueries: 2,
+    maxSnippetCharsPerSource: 4000,
+    snippetIncludeRawContent: false,
+  },
+},
+```
+
+---
+
+## 5. How `executionMemory` is affected
+
+### 5.1. Baseline (without web scoping)
+
+Without web scoping, for a NARRIX task, `executionMemory` will generally contain:
+
+- `executionMemory._narrix` — NARRIX run result (entity, signals, stories, passes, meta, etc.).
+- Any initial `executionMemory` provided by the caller (e.g. `executionMemory.input.raw` for raw records).
+
+`executionMemory.webContext` is **undefined** and should not be relied on.
+
+### 5.2. With `enableWebScope: true`
+
+When `enableWebScope` is true:
+
+- **Always present**:
+
+  ```ts
+  ctx.executionMemory.webContext: WebScoperResult;
+  ```
+
+  even when the scoper returns a miss or throws an error.
+
+- **Never removed / mutated**:
+  - `_narrix` (or the configured attachment field) is **still present and unchanged**, even when web scoping fails.
+  - Tests explicitly assert that `_narrix` remains present in the error path.
+
+Key behaviors from `test/narrix/webScoper.test.ts`:
+
+- When scoper succeeds:
+  - `webContext.available === true`
+  - `_narrix` is still present.
+- When scoper fails:
+  - `_narrix` is **still present**.
+  - `webContext` exists with `available === false` and `reason === "error"`.
+
+### 5.3. Question forwarding and enrichment
+
+When the caller passes a **natural-language question** in `RunTaskRequest.input.question`, the NARRIX pre-processor derives **`ScopeInput.question`** via **`resolveWebScopeQuestionAndTemplates`** (see §3.2.1): the base string is usually that field, **optionally extended** with CVE / vendor / product from CNI `knownFacts` or common `entity.graphized` paths when those tokens are not already in the question. With **`narrix.webScopeTemplates`**, the scoper uses template-driven queries instead and **`question`** may be omitted.
+
+```ts
+await runTask({
+  skillKey: "skills/skill.local:webScopeTestQuestion",
+  narrix: { datasetId: "test.fixture", enableWebScope: true },
+  executionMemory: { input: { raw: { id: "v3" } } },
+  input: { question: "Is this vulnerability critical?" },
+});
+```
+
+Integration tests (with `USE_NARRIX_INGEST=1`) assert the captured `question` when the fixture does not add conflicting CNI facts. Unit tests in the same file cover template precedence, Handlebars `webScopeQuestionTemplate`, and default append behavior.
+
+This allows the scoper to tailor its search and context building to the actual question the LLM or downstream logic is trying to answer, with **entity-specific tokens** when available.
+
+---
+
+## 6. Env and configuration
+
+### 6.1. Required env: Tavily API
+
+The web scoper uses `@exellix/search-adapter` backed by Tavily. It requires:
+
+- **`TAVILY_API_KEY`** — the API key for Tavily search.
+
+Behavior:
+
+- If `TAVILY_API_KEY` is **missing or invalid**:
+  - The scoper returns `available: false` with an appropriate `reason` (e.g. `"error"`).
+  - ai-tasks surfaces this as `executionMemory.webContext` with `available: false`.
+  - The task **continues** normally using NARRIX output only; `_narrix` is still present.
+
+### 6.2. Other relevant flags
+
+- `USE_NARRIX_INGEST`:
+  - When `"1"`, NARRIX ingest + runner are enabled and web scoping is meaningful.
+  - When not set, NARRIX-related flows (including web scoping) may be skipped or disabled in tests/playgrounds.
+
+Optional per-request **`narrix.webScoping`** forwards snippet/cap/raw-content and query limits into `@exellix/narrix-web-scoper` (see §4.3). The main on/off switch remains **`enableWebScope`**.
+
+---
+
+## 7. Error handling and observability
+
+Web scoping is explicitly designed to **never crash the task**:
+
+- Any exception in the scoper is caught by `runWebScope`.
+- The task SDK:
+  - Continues to attach `_narrix` as usual.
+  - Stores a structured error in `executionMemory.webContext` for debugging.
+
+This gives you:
+
+- Reliable signals in logs / memory:
+  - `webContext.available === false`
+  - `webContext.reason === "error"`
+  - `webContext.error` contains a human-readable message.
+- Freedom to treat web scoping as an **optional enhancement** while still observing its behavior in production.
+
+---
+
+## 8. How to consume web scoping from local tasks
+
+From the perspective of a **local task handler**, web scoping is just another field on `ctx.executionMemory`:
+
+```ts
+type LocalTaskHandler = (args: {
+  input: any;
+  ctx: {
+    executionMemory?: {
+      webContext?: WebScoperResult;
+      _narrix?: unknown;
+      // ...
+    };
+    // ...other context fields...
+  };
+}) => Promise<any>;
+```
+
+### 8.1. Recommended consumption pattern
+
+1. **Check for presence**:
+
+   ```ts
+   const webContext = ctx.executionMemory?.webContext;
+   if (!webContext) {
+     // web scoping was not enabled for this task
+   }
+   ```
+
+2. **Branch on `available`**:
+
+   ```ts
+   if (!webContext.available) {
+     // Option A: log / record reason and proceed without web data
+     // Option B: degrade gracefully (e.g. fallback to CNI-only reasoning)
+   } else {
+     // Safe to use webContext.context to enrich reasoning
+   }
+   ```
+
+3. **Do not rely on web scoping for correctness**:
+   - Handlers should behave sensibly even when `webContext` is absent or `available === false`.
+   - Web scoping is an **augmentation**, not a hard dependency.
+
+### 8.2. Example usage (pseudocode)
+
+```ts
+registerLocalTask("skills/skill.local:example", async ({ input, ctx }) => {
+  const narrix = ctx.executionMemory?._narrix;
+  const web = ctx.executionMemory?.webContext;
+
+  if (web?.available) {
+    // Combine NARRIX narrative with fresh web evidence
+    return buildExplanationWithWeb(narrix, web.context);
+  }
+
+  // Fallback: only NARRIX narrative
+  return buildExplanationFromNarrixOnly(narrix);
+});
+```
+
+### 8.3. LLM tasks (`runTask` DIRECT) and synthesized-context
+
+For skills executed through the task SDK with **`includeContextInPrompt: true`**, when NARRIX is in play ai-tasks appends a **Web sources (primary evidence)** markdown section derived from **`executionMemory.webContext`** (when **`available === true`**), using per-source **`providerRawContent`** → **`rawContent`** → **`providerContent`** → **`content`** → **`snippet`**, and labeling **`summary`** / **`findings`** as hints only. The same markdown is merged into source material for the **`synthesized-context`** PRE step under **`narrix-only`** / **`narrix+memory`** (with **`executionMemory.webContext`** stripped from the JSON memory slice when **`narrix+memory`** to avoid duplication). Implementation: **`src/narrix/webContextMarkdown.ts`**. Optional env **`WEB_CONTEXT_MARKDOWN_MAX_CHARS`** caps the size of that markdown string.
+
+---
+
+## 9. Testing strategy and guarantees
+
+ai-tasks contains tests that exercise both **unit-level** and **integration-level** web scoping behavior:
+
+- `test/narrix/webScoper.test.ts`:
+  - Unit-tests **`buildWebScopeTemplateContext`** / **`resolveWebScopeQuestionAndTemplates`** (question enrichment, Handlebars template, `webScopeTemplates` precedence).
+  - Unit-tests `runWebScope` with mocked `NarrixWebScoper`.
+  - Verifies:
+    - Pass-through of `available: true` and `context`.
+    - Stable miss handling (`available: false`, specific `reason`).
+    - Error → `available: false`, `reason: "error"`, populated `error`.
+    - Singleton reset behavior (`resetWebScoperSingleton`, `setWebScoperForTesting`).
+  - Integration with `runTask`:
+    - When `enableWebScope` is **absent**:
+      - `executionMemory.webContext` remains `undefined`.
+    - When `enableWebScope === true` and scoper returns success:
+      - `executionMemory.webContext.available === true`.
+      - `_narrix` is still present.
+    - When scoper throws:
+      - `_narrix` is still present.
+      - `webContext.available === false`, `webContext.reason === "error"`.
+    - When a question is provided (and `webScopeTemplates` is not set):
+      - `ScopeInput.question` equals `request.input.question` when there are no extra CNI/entity tokens to append; otherwise it is the **enriched** string (see §3.2.1).
+    - When `webScopeTemplates` is set:
+      - `ScopeInput.webScopeTemplates` / `webScopeObjects` are passed through as configured (merged with auto-built context).
+
+These tests define the **public contract** of web scoping within ai-tasks: any change that breaks their assertions is a breaking change to web scoping behavior.
+
+---
+
+## 10. Summary of contracts
+
+- **Opt-in**:
+  - `RunTaskRequest.narrix.enableWebScope === true` is required to trigger web scoping.
+- **Data flow**:
+  - NARRIX → success result (`_narrix`) → web scoper via `ScopeInput` → `executionMemory.webContext`.
+- **Consumer surface**:
+  - Local tasks read `ctx.executionMemory.webContext` and `_narrix`.
+- **Failure behavior**:
+  - Web scoping is **non-fatal**.
+  - `_narrix` is always attached on success, independent of web scoping.
+  - `webContext` is always present when enabled, with `available` + `reason`/`error` for observability.
+- **Config**:
+  - Controlled by per-request `enableWebScope`, optional `webScopeTemplates` / `webScopeObjects` / `webScopeQuestionTemplate` / **`webScoping`** on `narrix`, and `TAVILY_API_KEY` env.
+
+With this, integrators should have enough detail to:
+
+- Correctly enable web scoping from graphs / callers.
+- Reliably consume `executionMemory.webContext` in local tasks.
+- Understand and debug web scoping behavior in logs and execution memory without diving into the entire codebase.
+

package/documenations/xynthesis-activix-telemetry.md

@@ -0,0 +1,28 @@
+# Xynthesis Activix telemetry (Mongo)
+
+For **hosts** of `@exellix/ai-tasks` and other orchestrators using **`@exellix/xynthesis`**.
+
+## Symptom
+
+With **`MONGO_URI`** or **`MONGO_LOGS_URI`** set, logs may show Activix init failures mentioning **`activix-collections`**, **`diagnostics.owner`**, or **`collectionRegistry.owner`**. Synthesis calls might still return text while **Mongo telemetry** for xynthesis does not initialize.
+
+## Root cause
+
+**`@x12i/activix`** enables the collection registry by default; that path requires **`diagnostics.owner`** (or **`collectionRegistry.owner`**) on the **`Activix`** constructor. If xynthesis builds Activix **without** `diagnostics`, **`init()`** fails in database mode.
+
+## Required upstream change
+
+In **`exellix/xynthesis`** (`src/activix.ts`, `getActivix`), pass **`diagnostics`** with at least **`owner: "@exellix/xynthesis"`** (and optional **`XYNTHESIS_ACTIVIX_DIAGNOSTICS_OWNER`** override). Details and test ideas: **[`xynthesis-upstream-fixes-checklist.md`](./xynthesis-upstream-fixes-checklist.md)** §1.
+
+## Operational knobs
+
+| Env | Effect |
+|-----|--------|
+| **`XYNTHESIS_ACTIVIX=FALSE`** | Disable xynthesis Mongo telemetry. |
+| **`XYNTHESIS_ACTIVIX_DIAGNOSTICS_OWNER`** | Override **`diagnostics.owner`**. |
+| **`XYNTHESIS_ACTIVIX_COLLECTION`** | Override default collection **`xynthesis-actions`**. |
+
+## Related
+
+- Gateway-side Activix (**`@x12i/ai-gateway`**) is separate from xynthesis’ singleton.
+- ai-tasks **`ACTIVIX_ENABLED`** / **`src/activix/activixClient.ts`** is separate from xynthesis telemetry.

package/documenations/xynthesis-upstream-fixes-checklist.md

@@ -0,0 +1,71 @@
+# `@exellix/xynthesis` — fixes and tests to implement (upstream)
+
+This document lists **what to change and verify in the xynthesis repo** (`exellix/xynthesis`). It is **not** guidance to pin or downgrade xynthesis in consumers; hosts should track **current** published `@exellix/xynthesis` once these items are merged and released.
+
+---
+
+## 1. Activix Mongo init: `diagnostics.owner`
+
+**Symptom:** With `MONGO_URI` / `MONGO_LOGS_URI` set, Activix fails init with messages about **`activix-collections`** / **`diagnostics.owner`** / **`collectionRegistry.owner`**.
+
+**Cause:** `getActivix()` builds `new Activix({ ... })` without **`diagnostics`**, while `@x12i/activix` expects an owning package id for the default collection registry.
+
+**Fix (source: `src/activix.ts`):**
+
+- Pass **`diagnostics: { owner, component, instanceLabel, workerId }`** on the Activix constructor.
+- Default **`owner`** to **`@exellix/xynthesis`**; allow override via env **`XYNTHESIS_ACTIVIX_DIAGNOSTICS_OWNER`** for multi-service Mongo deployments.
+
+**Tests to add in xynthesis:**
+
+- With a test Mongo URI (or mocked store if you have hooks), **`getActivix().init()`** completes without the registry error.
+- Optional: assert legend / registry behavior matches Activix docs.
+
+**Related consumer doc:** [`xynthesis-activix-telemetry.md`](./xynthesis-activix-telemetry.md) (symptom and verification from an app’s perspective).
+
+---
+
+## 2. `runSidekickGatewayCall`: do not call `.trim()` on undefined `jobId` / `taskId`
+
+**Symptom:** `TypeError: Cannot read properties of undefined (reading 'trim')` at `runSidekickGatewayCall` (e.g. `sidekickGateway.js` line ~41) when **`RunSidekickExecConfig.jobId`** or **`taskId`** is missing.
+
+**Reproducer:** Call **`runStructuredSynthesisGatewayCall`** with **`jobId` set but `taskId` omitted** (invalid for the public contract, but easy to do in tests or loose JS).
+
+**Fix (source: `src/sidekickGateway.ts`):**
+
+- Treat missing non-strings as empty before trim, e.g.  
+  `const jobIdTrim = typeof execConfig.jobId === "string" ? execConfig.jobId.trim() : "";`  
+  (same for **`taskId`**).
+- Prefer **after** that: if either is empty, **`throw`** a clear, named error (e.g. *“runSidekickGatewayCall requires non-empty jobId and taskId on RunSidekickExecConfig”*) instead of failing later or only logging a warn.
+
+**Tests to add in xynthesis:**
+
+- Missing **`taskId`** → deterministic error message (no `TypeError` on `trim`).
+- Missing **`jobId`** → same.
+- Valid pair → path proceeds (mock invoker if needed).
+
+---
+
+## 3. Types vs runtime for structured synthesis params
+
+**Fix:** Ensure **`RunStructuredSynthesisParams`** / exported **`RunSidekickExecConfig`** types mark **`jobId`** and **`taskId`** as **required** everywhere they flow into **`runSidekickGatewayCall`**, so TypeScript catches omissions in TS callers.
+
+**Tests:** Compile-time is enough for TS; add a short note in JSDoc that both fields are mandatory for Activix / work-scoped logging.
+
+---
+
+## What is *not* a xynthesis bug (ai-tasks E2E context)
+
+Failures like **`CataloxSkillNotFoundError`** for **`skills/test-intermediate-steps`** or **“could not resolve raw instructions/prompt templates”** for that skill key come from **Catalox / ai-skills content registry**: the test skill must exist in the environment’s catalog, or the E2E must use a skill key that is registered. Track that under **ai-tasks E2E prerequisites** or seed Catalox for CI—not under xynthesis code changes.
+
+---
+
+## Suggested release QA (xynthesis)
+
+1. Activix init against real Mongo URI with registry enabled — **no** owner error.
+2. **`runStructuredSynthesisGatewayCall`** with full **`jobId` + `taskId`** — returns parsed payload (with mocked or real gateway as you do today).
+3. **`runStructuredSynthesisGatewayCall`** with **`taskId` omitted** — **clear thrown error**, not `TypeError` on `trim`.
+
+## Follow-up (same class of bug)
+
+Audit other entrypoints that do **`opts.jobId.trim()`** / **`input.taskId.trim()`** without a type guard (e.g. **`runSynthesisCall.ts`**, **`finalize.ts`**, **`synthesisErrors.ts`**) so missing IDs never produce **`Cannot read properties of undefined`**.
+