@exellix/ai-tasks 8.2.5 → 8.4.0

package/documenations/upstream-feature-requests/xynthesis-orchestrator-invoke-contract-4.2.md

@@ -0,0 +1,175 @@
+# `@exellix/xynthesis` — orchestrator invoke contract fixes (≥ 4.2)
+
+Status: **open** — blocks clean `@exellix/ai-tasks` 8.4+ integration  
+Owner: `@exellix/xynthesis`  
+Filed by: `@exellix/ai-tasks`  
+Consumer: `@exellix/ai-tasks` 8.4.0 (PRE / POST / scoping / utilities / structured synthesis)  
+Pinned in ai-tasks today: `@exellix/xynthesis@4.2.2`
+
+Related (older, partially superseded): [`xynthesis-llm-observability.md`](./xynthesis-llm-observability.md)
+
+---
+
+## Summary
+
+Xynthesis **4.2** moved token budgeting **inside** the package (`executeXynthesisAction` → `resolveEffectiveMaxTokens` + Optimixer). That is the right design.
+
+Remaining gaps for orchestrators (`@exellix/ai-tasks`, graph-engine):
+
+| Orchestrator sends | Package owns internally |
+|--------------------|-------------------------|
+| Wire **model** | — |
+| **outputExpectation** (sizing / density intent for Optimixer) | — |
+| **temperature**, **topP** | — |
+| **reasoningEffort** | — |
+| — | **max completion tokens** (Optimixer) |
+
+**Do not** require hosts to call removed exports (`resolveMaxTokens` from `@exellix/xynthesis/ai-actions`). Hosts must not pre-compute token counts.
+
+---
+
+## Already correct in 4.2.2 (no change needed)
+
+### 1. Token resolution inside `executeXynthesisAction`
+
+- Requires `outputExpectation` on every hop (throws if missing).
+- Calls `resolveEffectiveMaxTokens` (async, Optimixer) with `templateId` derived from `sidekickAction` / `activixActionType` / `trace.kind`.
+- Optional `maxTokens` on the request is only a **ceiling** (`callerMaxTokens`), not the primary API.
+
+### 2. Public export surface
+
+- `@exellix/xynthesis/ai-actions` exports `resolveEffectiveMaxTokens`, `completeOptimixerMaxTokensPrediction`.
+- Sync `resolveMaxTokens` is **not** exported from `ai-actions` (internal legacy paths may still use it).
+
+### 3. Structured gateway
+
+- `runStructuredSynthesisGatewayCall` accepts `outputExpectation` and forwards to sidekick / execute path.
+
+---
+
+## Required fixes
+
+### 1. Accept `reasoningEffort` on all host-facing hop APIs
+
+#### Today
+
+- `reasoningEffort` exists only on **`EffectiveMaxTokensResolutionInput`** (internal to `resolveEffectiveMaxTokens`).
+- **`ExecuteXynthesisActionRequest` has no `reasoningEffort` field.**
+- `executeXynthesisAction` calls Optimixer with `reasoningEffort: input.reasoningEffort ?? "not-applicable"` — hosts cannot override per call.
+
+#### Why it hurts
+
+`@exellix/ai-tasks` unifies tuning in `LlmCallConfig` for every stage. MAIN gets `reasoningEffort` from ai-skills catalog (once fixed); PRE/POST/scoping must be able to send the same knob into xynthesis without forking Catalox-only configuration.
+
+#### Asked behavior
+
+Add optional `reasoningEffort?: OptimixerReasoningEffort` to:
+
+- `ExecuteXynthesisActionRequest`
+- `RunStructuredSynthesisParams` (and `RunSidekickExecConfig` / gateway exec config if separate)
+- `RunXynthesisFinalizeExecConfig`
+- `RunXynthesisAiActionRequest` / config slice if applicable
+
+Forward into `resolveEffectiveMaxTokens({ …, reasoningEffort })`.
+
+**Resolution order:**
+
+1. Request field
+2. Catalox sidekick / action catalog default (if loaded for that template)
+3. `"not-applicable"`
+
+#### Acceptance
+
+- `executeXynthesisAction({ …, reasoningEffort: "high", outputExpectation: { … } })` passes `"high"` to Optimixer predict.
+- `invokeSummary` / `tokenResolution` diagnostics echo effective reasoning effort when available.
+
+---
+
+### 2. Accept and forward `temperature` + `topP` on every public hop (audit)
+
+#### Today
+
+- `ExecuteXynthesisActionRequest` includes `topP`; temperature is often set on `gateway` only.
+- `@exellix/ai-tasks` `runPostStepLlmCall` sets `gateway.temperature` but **does not pass `topP`** today.
+- Older FR in [`xynthesis-llm-observability.md` §1](./xynthesis-llm-observability.md) may be partially addressed — **verify end-to-end** on:
+  - `executeXynthesisAction`
+  - `runStructuredSynthesisGatewayCall` / `runSidekickGatewayCall`
+  - `runXynthesisFinalize`
+
+#### Asked behavior
+
+- Document where `temperature` lives (`gateway` vs top-level) and support **one** canonical shape (or accept both with merge rules).
+- Ensure `topP` is forwarded to Funcx / gateway invoker on **all** paths above.
+- Echo applied values in `invokeSummary.executionMetadata`.
+
+#### Acceptance
+
+- Integration test: `executeXynthesisAction({ topP: 0.8, gateway: { temperature: 0.2, … }, … })` → provider request includes both.
+
+---
+
+### 3. Orchestrator documentation: do not pre-resolve `maxTokens`
+
+#### Today
+
+- Integrators (including ai-tasks 8.4) still import `resolveMaxTokens` from `@exellix/xynthesis/ai-actions` — **removed**, causes runtime `SyntaxError`.
+- README / ai-actions example should steer hosts to:
+  - Pass **`outputExpectation`** (required)
+  - Optionally pass **`maxTokens` only as a rare ceiling** if product keeps that knob
+  - Let **`executeXynthesisAction`** run Optimixer
+
+#### Asked behavior
+
+- **BREAKING / CHANGELOG** entry: `resolveMaxTokens` removed from `ai-actions`; use `executeXynthesisAction` or `resolveEffectiveMaxTokens` for advanced callers.
+- **ai-actions** module doc example updated (already mentions `resolveEffectiveMaxTokens` — expand with `reasoningEffort` once §1 ships).
+
+#### Acceptance
+
+- `import { resolveMaxTokens } from "@exellix/xynthesis/ai-actions"` fails at build time with a clear migration note.
+- Document that **`outputExpectation` is not a token count** — it is Optimixer input; the numeric cap is internal.
+
+---
+
+### 4. Clarify or remove host `maxTokens` on `ExecuteXynthesisActionRequest`
+
+#### Today
+
+- `maxTokens?: number` acts as `callerMaxTokens` ceiling on Optimixer result.
+
+#### Product decision (pick one and document)
+
+**A (recommended for graph-engine parity):** Deprecate host `maxTokens`; ceiling only via catalog / Optimixer constraints. Orchestrators send `outputExpectation` + `reasoningEffort` only.
+
+**B:** Keep `maxTokens` as **optional hard ceiling** for cost guardrails; rename in docs to `maxTokensCeiling` in a future major to avoid confusion.
+
+ai-tasks will follow whichever contract is published; default integration is **no host cap**.
+
+---
+
+## Stale items in `xynthesis-llm-observability.md`
+
+| Old § | Status in 4.2.2 |
+|-------|------------------|
+| §1 temperature / topP | Re-verify all paths; close or narrow if only sidekick gaps remain |
+| §2 `outputExpectation` on `runSynthesisCall` | Superseded by `executeXynthesisAction` requiring `outputExpectation` |
+| § resolveMaxTokens on ai-actions | **Removed** — replace with this doc |
+
+---
+
+## Verification checklist (for xynthesis PR)
+
+- [ ] `ExecuteXynthesisActionRequest.reasoningEffort` (and gateway wrappers).
+- [ ] Optimixer predict receives resolved `reasoningEffort`.
+- [ ] `temperature` + `topP` integration tests on `executeXynthesisAction` + structured gateway.
+- [ ] CHANGELOG: `resolveMaxTokens` not in `ai-actions` exports; migration path documented.
+- [ ] Public contract: hosts send `outputExpectation`, not precomputed `maxTokens`.
+
+---
+
+## After this ships — ai-tasks follow-up
+
+See [`ai-tasks-wrap-up-after-upstream.md`](./ai-tasks-wrap-up-after-upstream.md):
+
+- Remove `resolveMaxTokens` imports and `resolveLlmCallForXynthesis` token math.
+- Pass `outputExpectation` + `temperature` + `topP` + `reasoningEffort` into every `executeXynthesisAction` call.
+- Do not forward `maxTokens` / `maxTokensCap` from `RunTaskRequest`.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@exellix/ai-tasks",
-  "version": "8.2.5",
+  "version": "8.4.0",
   "description": "Task orchestration for the Exellix stack: runTask() with local handlers or LLM-backed execution, task-scoped memory/context enrichment, and executor dispatch via @exellix/ai-skills. ERC-compliant.",
   "type": "module",
   "main": "dist/index.js",
@@ -60,12 +60,13 @@
   },
   "overrides": {
     "@x12i/activix": "$@x12i/activix",
+    "@x12i/ai-profiles": "$@x12i/ai-profiles",
     "@x12i/catalox": "$@x12i/catalox",
     "@x12i/funcx": "$@x12i/funcx",
     "@x12i/logxer": "$@x12i/logxer"
   },
   "dependencies": {
-    "@exellix/ai-skills": "^5.9.7",
+    "@exellix/ai-skills": "^5.9.12",
     "@exellix/memorix-narrix-adapter": "^2.0.0",
     "@exellix/narrix-adapter-chat": "^2.0.0",
     "@exellix/narrix-adapter-docs": "^2.0.0",
@@ -77,13 +78,13 @@
     "@exellix/narrix-ingest": "^2.0.0",
     "@exellix/narrix-runner": "^2.0.0",
     "@exellix/narrix-web-scoper": "^2.0.0",
-    "@exellix/xynthesis": "^4.1.7",
+    "@exellix/xynthesis": "^4.3.0",
     "@x12i/activix": "^8.5.0",
-    "@x12i/ai-profiles": "^1.8.0",
+    "@x12i/ai-profiles": "^2.0.1",
     "@x12i/catalox": "^5.1.3",
     "@x12i/env": "^4.0.1",
     "@x12i/execution-memory-manager": "^1.2.0",
-    "@x12i/funcx": "^4.3.0",
+    "@x12i/funcx": "^4.3.2",
     "@x12i/logxer": "^4.6.0",
     "@x12i/rendrix": "^4.3.0",
     "@x12i/search-adapter": "^1.5.1",