@exellix/ai-tasks 8.2.9 → 8.4.0

package/documenations/upstream-feature-requests/ai-tasks-wrap-up-after-upstream.md

@@ -0,0 +1,126 @@
+# `@exellix/ai-tasks` — wrap-up after upstream invoke-contract fixes
+
+Status: **blocked** on upstream PRs until both packages ship items in:
+
+- [`ai-skills-orchestrator-invoke-contract-5.9.md`](./ai-skills-orchestrator-invoke-contract-5.9.md)
+- [`xynthesis-orchestrator-invoke-contract-4.2.md`](./xynthesis-orchestrator-invoke-contract-4.2.md)
+
+Owner: `@exellix/ai-tasks`  
+Target release: **8.4.x** (or **8.5.0** if breaking `llmCall` surface)
+
+---
+
+## Contract (locked with product)
+
+**ai-tasks sends to lower layers:**
+
+- Model ids via `RunTaskModelConfig` triplet (`preActionModel`, `skillModel`, `postActionModel`) + optional `llmCall.model` overrides.
+- `temperature`, `topP`, **`reasoningEffort`** (once upstream exposes it).
+- **`outputExpectation`** on xynthesis-backed hops only (sizing intent — **not** a token count).
+
+**ai-tasks does not send:**
+
+- `maxTokens` / `maxTokensCap` on `modelConfig`, `runSkill`, `executeXynthesisAction`, or direct gateway repair (unless product later revives an explicit ceiling knob with a new name).
+
+**Lower layers own:**
+
+- MAIN caps → `@exellix/ai-skills` Optimixer + Catalox skill catalog.
+- PRE/POST/scoping/utility caps → `@exellix/xynthesis` `resolveEffectiveMaxTokens` inside `executeXynthesisAction`.
+
+---
+
+## Already done in ai-tasks (this branch)
+
+- [x] `RunTaskRequest.modelConfig` typed as `RunTaskModelConfig` (not ai-skills `ModelConfig` with `maxTokens`).
+- [x] Removed `maxTokens` from `RunTaskModelConfig`; strip legacy key at ai-skills boundary.
+- [x] Stopped mapping `llmCall.maxTokensCap` → `modelConfig.maxTokens` in `buildAiSkillsModelConfigForMain`.
+- [x] Trace repair: `mapGatewayInvokeToTrace` uses `maxTokensRequested` only (not `modelConfig.maxTokens`).
+- [x] README / BREAKING-CHANGES partial updates for modelConfig + MAIN Optimixer.
+
+---
+
+## TODO after upstream ships
+
+### 1. Remove broken / duplicate token budgeting
+
+| File / area | Change |
+|-------------|--------|
+| `src/internal/resolveLlmCallForXynthesis.ts` | Remove `resolveMaxTokens` import; either delete token resolution or reduce to **outputExpectation-only** helper (rename e.g. `resolveLlmOutputExpectationForXynthesis`). |
+| `src/internal/runPostStepLlmCall.ts` | Drop `maxTokens` / `resolveLlmCallForXynthesis` pre-resolution; pass **`outputExpectation`** (required). |
+| `src/synthesis/runStructuredSynthesisRobust.ts` | Repair path: stop `resolveMaxTokens`; do not push `maxTokens` into raw AIGateway `config` unless gateway documents a non-Optimixer escape hatch. |
+| `src/index.ts` | Stop re-exporting `resolveMaxTokens`; export `resolveEffectiveMaxTokens` only if graph-engine needs it (prefer not). |
+| `src/utilities/runUtility.ts` | Remove `maxTokens` forward to xynthesis finalize. |
+| `src/post-steps/resolvePostStepConfig.ts` | Remove env `*_MAX_TOKENS_CAP` wiring if contract is zero host caps (or keep env as deprecated no-op one release). |
+
+### 2. Add `reasoningEffort` to ai-tasks surface
+
+| File | Change |
+|------|--------|
+| `src/types/llmCall.ts` | `reasoningEffort?: OptimixerReasoningEffort` (re-export type from `@x12i/optimixer` or xynthesis). |
+| `src/types/model-config.ts` | Optional tuning field; passthrough in `modelConfigTuningPassthrough` (not stripped). |
+| `src/types/task-types.ts` | Document on `llmCall` and post-step configs. |
+| `buildAiSkillsModelConfigForMain` | Forward to ai-skills `modelConfig` when upstream §1 ships. |
+| `runPostStepLlmCall` / `task-sdk` synthesis | Forward to `executeXynthesisAction` / structured params. |
+| `src/validation/helpers.ts` | Optional enum validation if finite set. |
+| `src/builders/task-request-builder.ts` | Document in `withLlmCall` JSDoc. |
+
+### 3. Fix xynthesis call sites (no new upstream needed except §1–2)
+
+| Call site | Pass |
+|-----------|------|
+| `runPostStepLlmCall` → `executeXynthesisAction` | `outputExpectation`, `temperature`, `topP`, `sidekickAction` / `activixActionType`, `reasoningEffort` (when exists) |
+| PRE structured / markdown synthesis (`task-sdk`) | Same via gateway / `runStructuredSynthesisGatewayCall` params |
+| AI scoping | `outputExpectation` (already defaulted in `runScopingCall`) |
+
+### 4. Remove / deprecate `maxTokensCap` on `LlmCallConfig`
+
+- **Breaking:** remove `maxTokensCap` from `LlmCallConfig`, `RunUtilityRequest.exec`, post-step env fallbacks, validation, README.
+- **Or soft:** mark deprecated in 8.4.1, ignore at runtime, remove in 8.5.0.
+
+Confirm with graph-engine before hard removal.
+
+### 5. Tests & docs
+
+- [ ] `npm run build` + `tsc -p tsconfig.test.json` green (no `resolveMaxTokens` import).
+- [ ] `npm run test` green (mocks via `setSynthesisInvoker` unchanged).
+- [ ] Update [`README.md`](../../README.md) token-budget section (remove `llmCall.maxTokensCap` → `modelConfig.maxTokens` overlay story).
+- [ ] Update [`ai-skills-llm-observability.md`](./ai-skills-llm-observability.md) §2 (stale maxTokens wording).
+- [ ] Update [`xynthesis-llm-observability.md`](./xynthesis-llm-observability.md) superseded sections.
+- [ ] Link all three contract docs from [`BREAKING-CHANGES.md`](../../BREAKING-CHANGES.md) 8.4 section.
+
+---
+
+## Dependency order
+
+```mermaid
+flowchart LR
+  A[ai-skills: reasoningEffort on RunSkillRequest]
+  B[xynthesis: reasoningEffort on ExecuteXynthesisActionRequest]
+  C[xynthesis: verify temperature topP all paths]
+  D[ai-tasks: remove maxTokens forwarding]
+  E[ai-tasks: wire reasoningEffort + outputExpectation]
+  A --> E
+  B --> E
+  C --> E
+  D --> E
+```
+
+Recommended sequence:
+
+1. Merge **xynthesis** + **ai-skills** contract PRs (can be parallel).
+2. Bump `package.json` minimums in ai-tasks.
+3. Execute **this wrap-up** checklist in one ai-tasks PR.
+4. Release note in **8.4.x** or **8.5.0** with graph-engine migration (drop `maxTokensCap` on task payloads if removed).
+
+---
+
+## Quick smoke test after wrap-up
+
+```bash
+npm run build
+npm run test
+# Optional live:
+# npm run test:e2e:synthesis
+```
+
+Expect: no import of `resolveMaxTokens` from `@exellix/xynthesis/ai-actions`; live POST/PRE calls succeed with `outputExpectation` only (no `maxTokens` on request).

package/documenations/upstream-feature-requests/xynthesis-orchestrator-invoke-contract-4.2.md

@@ -0,0 +1,175 @@
+# `@exellix/xynthesis` — orchestrator invoke contract fixes (≥ 4.2)
+
+Status: **open** — blocks clean `@exellix/ai-tasks` 8.4+ integration  
+Owner: `@exellix/xynthesis`  
+Filed by: `@exellix/ai-tasks`  
+Consumer: `@exellix/ai-tasks` 8.4.0 (PRE / POST / scoping / utilities / structured synthesis)  
+Pinned in ai-tasks today: `@exellix/xynthesis@4.2.2`
+
+Related (older, partially superseded): [`xynthesis-llm-observability.md`](./xynthesis-llm-observability.md)
+
+---
+
+## Summary
+
+Xynthesis **4.2** moved token budgeting **inside** the package (`executeXynthesisAction` → `resolveEffectiveMaxTokens` + Optimixer). That is the right design.
+
+Remaining gaps for orchestrators (`@exellix/ai-tasks`, graph-engine):
+
+| Orchestrator sends | Package owns internally |
+|--------------------|-------------------------|
+| Wire **model** | — |
+| **outputExpectation** (sizing / density intent for Optimixer) | — |
+| **temperature**, **topP** | — |
+| **reasoningEffort** | — |
+| — | **max completion tokens** (Optimixer) |
+
+**Do not** require hosts to call removed exports (`resolveMaxTokens` from `@exellix/xynthesis/ai-actions`). Hosts must not pre-compute token counts.
+
+---
+
+## Already correct in 4.2.2 (no change needed)
+
+### 1. Token resolution inside `executeXynthesisAction`
+
+- Requires `outputExpectation` on every hop (throws if missing).
+- Calls `resolveEffectiveMaxTokens` (async, Optimixer) with `templateId` derived from `sidekickAction` / `activixActionType` / `trace.kind`.
+- Optional `maxTokens` on the request is only a **ceiling** (`callerMaxTokens`), not the primary API.
+
+### 2. Public export surface
+
+- `@exellix/xynthesis/ai-actions` exports `resolveEffectiveMaxTokens`, `completeOptimixerMaxTokensPrediction`.
+- Sync `resolveMaxTokens` is **not** exported from `ai-actions` (internal legacy paths may still use it).
+
+### 3. Structured gateway
+
+- `runStructuredSynthesisGatewayCall` accepts `outputExpectation` and forwards to sidekick / execute path.
+
+---
+
+## Required fixes
+
+### 1. Accept `reasoningEffort` on all host-facing hop APIs
+
+#### Today
+
+- `reasoningEffort` exists only on **`EffectiveMaxTokensResolutionInput`** (internal to `resolveEffectiveMaxTokens`).
+- **`ExecuteXynthesisActionRequest` has no `reasoningEffort` field.**
+- `executeXynthesisAction` calls Optimixer with `reasoningEffort: input.reasoningEffort ?? "not-applicable"` — hosts cannot override per call.
+
+#### Why it hurts
+
+`@exellix/ai-tasks` unifies tuning in `LlmCallConfig` for every stage. MAIN gets `reasoningEffort` from ai-skills catalog (once fixed); PRE/POST/scoping must be able to send the same knob into xynthesis without forking Catalox-only configuration.
+
+#### Asked behavior
+
+Add optional `reasoningEffort?: OptimixerReasoningEffort` to:
+
+- `ExecuteXynthesisActionRequest`
+- `RunStructuredSynthesisParams` (and `RunSidekickExecConfig` / gateway exec config if separate)
+- `RunXynthesisFinalizeExecConfig`
+- `RunXynthesisAiActionRequest` / config slice if applicable
+
+Forward into `resolveEffectiveMaxTokens({ …, reasoningEffort })`.
+
+**Resolution order:**
+
+1. Request field
+2. Catalox sidekick / action catalog default (if loaded for that template)
+3. `"not-applicable"`
+
+#### Acceptance
+
+- `executeXynthesisAction({ …, reasoningEffort: "high", outputExpectation: { … } })` passes `"high"` to Optimixer predict.
+- `invokeSummary` / `tokenResolution` diagnostics echo effective reasoning effort when available.
+
+---
+
+### 2. Accept and forward `temperature` + `topP` on every public hop (audit)
+
+#### Today
+
+- `ExecuteXynthesisActionRequest` includes `topP`; temperature is often set on `gateway` only.
+- `@exellix/ai-tasks` `runPostStepLlmCall` sets `gateway.temperature` but **does not pass `topP`** today.
+- Older FR in [`xynthesis-llm-observability.md` §1](./xynthesis-llm-observability.md) may be partially addressed — **verify end-to-end** on:
+  - `executeXynthesisAction`
+  - `runStructuredSynthesisGatewayCall` / `runSidekickGatewayCall`
+  - `runXynthesisFinalize`
+
+#### Asked behavior
+
+- Document where `temperature` lives (`gateway` vs top-level) and support **one** canonical shape (or accept both with merge rules).
+- Ensure `topP` is forwarded to Funcx / gateway invoker on **all** paths above.
+- Echo applied values in `invokeSummary.executionMetadata`.
+
+#### Acceptance
+
+- Integration test: `executeXynthesisAction({ topP: 0.8, gateway: { temperature: 0.2, … }, … })` → provider request includes both.
+
+---
+
+### 3. Orchestrator documentation: do not pre-resolve `maxTokens`
+
+#### Today
+
+- Integrators (including ai-tasks 8.4) still import `resolveMaxTokens` from `@exellix/xynthesis/ai-actions` — **removed**, causes runtime `SyntaxError`.
+- README / ai-actions example should steer hosts to:
+  - Pass **`outputExpectation`** (required)
+  - Optionally pass **`maxTokens` only as a rare ceiling** if product keeps that knob
+  - Let **`executeXynthesisAction`** run Optimixer
+
+#### Asked behavior
+
+- **BREAKING / CHANGELOG** entry: `resolveMaxTokens` removed from `ai-actions`; use `executeXynthesisAction` or `resolveEffectiveMaxTokens` for advanced callers.
+- **ai-actions** module doc example updated (already mentions `resolveEffectiveMaxTokens` — expand with `reasoningEffort` once §1 ships).
+
+#### Acceptance
+
+- `import { resolveMaxTokens } from "@exellix/xynthesis/ai-actions"` fails at build time with a clear migration note.
+- Document that **`outputExpectation` is not a token count** — it is Optimixer input; the numeric cap is internal.
+
+---
+
+### 4. Clarify or remove host `maxTokens` on `ExecuteXynthesisActionRequest`
+
+#### Today
+
+- `maxTokens?: number` acts as `callerMaxTokens` ceiling on Optimixer result.
+
+#### Product decision (pick one and document)
+
+**A (recommended for graph-engine parity):** Deprecate host `maxTokens`; ceiling only via catalog / Optimixer constraints. Orchestrators send `outputExpectation` + `reasoningEffort` only.
+
+**B:** Keep `maxTokens` as **optional hard ceiling** for cost guardrails; rename in docs to `maxTokensCeiling` in a future major to avoid confusion.
+
+ai-tasks will follow whichever contract is published; default integration is **no host cap**.
+
+---
+
+## Stale items in `xynthesis-llm-observability.md`
+
+| Old § | Status in 4.2.2 |
+|-------|------------------|
+| §1 temperature / topP | Re-verify all paths; close or narrow if only sidekick gaps remain |
+| §2 `outputExpectation` on `runSynthesisCall` | Superseded by `executeXynthesisAction` requiring `outputExpectation` |
+| § resolveMaxTokens on ai-actions | **Removed** — replace with this doc |
+
+---
+
+## Verification checklist (for xynthesis PR)
+
+- [ ] `ExecuteXynthesisActionRequest.reasoningEffort` (and gateway wrappers).
+- [ ] Optimixer predict receives resolved `reasoningEffort`.
+- [ ] `temperature` + `topP` integration tests on `executeXynthesisAction` + structured gateway.
+- [ ] CHANGELOG: `resolveMaxTokens` not in `ai-actions` exports; migration path documented.
+- [ ] Public contract: hosts send `outputExpectation`, not precomputed `maxTokens`.
+
+---
+
+## After this ships — ai-tasks follow-up
+
+See [`ai-tasks-wrap-up-after-upstream.md`](./ai-tasks-wrap-up-after-upstream.md):
+
+- Remove `resolveMaxTokens` imports and `resolveLlmCallForXynthesis` token math.
+- Pass `outputExpectation` + `temperature` + `topP` + `reasoningEffort` into every `executeXynthesisAction` call.
+- Do not forward `maxTokens` / `maxTokensCap` from `RunTaskRequest`.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@exellix/ai-tasks",
-  "version": "8.2.9",
+  "version": "8.4.0",
   "description": "Task orchestration for the Exellix stack: runTask() with local handlers or LLM-backed execution, task-scoped memory/context enrichment, and executor dispatch via @exellix/ai-skills. ERC-compliant.",
   "type": "module",
   "main": "dist/index.js",
@@ -66,7 +66,7 @@
     "@x12i/logxer": "$@x12i/logxer"
   },
   "dependencies": {
-    "@exellix/ai-skills": "^5.9.8",
+    "@exellix/ai-skills": "^5.9.12",
     "@exellix/memorix-narrix-adapter": "^2.0.0",
     "@exellix/narrix-adapter-chat": "^2.0.0",
     "@exellix/narrix-adapter-docs": "^2.0.0",
@@ -78,13 +78,13 @@
     "@exellix/narrix-ingest": "^2.0.0",
     "@exellix/narrix-runner": "^2.0.0",
     "@exellix/narrix-web-scoper": "^2.0.0",
-    "@exellix/xynthesis": "^4.1.8",
+    "@exellix/xynthesis": "^4.3.0",
     "@x12i/activix": "^8.5.0",
-    "@x12i/ai-profiles": "^1.8.0",
+    "@x12i/ai-profiles": "^2.0.1",
     "@x12i/catalox": "^5.1.3",
     "@x12i/env": "^4.0.1",
     "@x12i/execution-memory-manager": "^1.2.0",
-    "@x12i/funcx": "^4.3.0",
+    "@x12i/funcx": "^4.3.2",
     "@x12i/logxer": "^4.6.0",
     "@x12i/rendrix": "^4.3.0",
     "@x12i/search-adapter": "^1.5.1",