@exellix/ai-tasks 9.1.0 → 10.0.0

package/documenations/upstream-feature-requests/funcx-gap-analysis-cr-fr.md

@@ -0,0 +1,401 @@
+# `@x12i/funcx` — gap analysis (CRs & FRs for upstream)
+
+**Filed by:** `@exellix/ai-tasks` + `@exellix/xynthesis`  
+**Baseline validated:** `@x12i/funcx` **4.9.13** (live OpenRouter, 2026-06-12)  
+**Evidence:** `temp/funcx-functions/` live runs, `instructions-comparison.md`, `live-run-summary.md`
+
+**Related docs:**
+
+- [**funcx-4.9.9-residual-upstream-cr-fr.md**](../../../archived/ai-tasks/documenations/upstream-feature-requests/superseded/funcx-4.9.9-residual-upstream-cr-fr.md) — **file these to `x12i/funcx`** (copy-paste GitHub issues; blocks deleting consumer workarounds)
+- [funcx-generic-xynthesis-hosting.md](./funcx-generic-xynthesis-hosting.md) — envelope / `run()` baseline
+- [funcx-pre-post-sidekick-actions.md](./funcx-pre-post-sidekick-actions.md) — per-function sidekick specs
+- [funcx-upstream-github-issues-draft.md](../funcx-upstream-github-issues-draft.md) — GitHub issue templates
+- Consumer specs: `temp/funcx-functions/*/instructions-template.md`, `input-object.md`, `output-object.md`
+
+---
+
+## 0. Executive summary
+
+| Area | 4.9.9 status | FuncX work left? |
+|------|--------------|------------------|
+| **Registry** — 10 mapped ids callable via `run()` | **Done** | No |
+| **Prompt content** — match xynthesis / ai-tasks disk templates | **Done** (FR-GEN-1, CR-FUNCX-1) | CI only (FR-GEN-9) |
+| **Output contracts** — especially audit-checklist markdown | **Done @ 4.9.11** | No (CR-FUNCX-2 closed) |
+| **Envelope richness** — audit-merge full template fields | **Done @ 4.9.11** (plain text merge output) | Monitor rich-envelope prompt parity (FR-GEN-11) |
+| **Orchestrator ergonomics** — sampling on `run()` | **Done** (`temperature`, `topP`, `reasoningEffort`) | No |
+| **Types / schemas exported** | **Done** (`SidekickGenericEnvelope`, `getFuncx*Schema`) | FR-GEN-3 publish polish |
+
+**Mono-repo wiring** (FuncX `run()` only in gateway — no flag) — **shipped** in xynthesis + ai-tasks audit (2026-06). See §7.
+
+---
+
+## 1. What FuncX 4.9.8 already delivered (closed items)
+
+Update prior trackers that still say “Open @ 4.4.0”:
+
+| Id | Topic | Status @ 4.9.8 |
+|----|--------|----------------|
+| **FR-PRE-POST-1** | `pre-synthesize` built-in | **Shipped** (alias; `pre/synthesize` normalizes) |
+| **FR-PRE-POST-3** | `post-audit` | **Shipped** |
+| **FR-PRE-POST-4** | `post-fix` | **Shipped** |
+| **FR-PRE-POST-5** | `post-pick-best`, `post-craft-final` | **Shipped** |
+| **FR-PRE-POST-2** (registry only) | `post-audit-checklist`, `post-audit-merge` | **Shipped** (callable; content/contract gaps remain) |
+| **FR-GEN-0** | `run()`, `getRunJsonResult`, `buildAskAttribution`, execution/research generics | **Shipped** |
+| **FR-GEN-6** (registry) | Checklist / merge ids | **Shipped** → content in CR-FUNCX-2 / FR-GEN-11 |
+| **FR-GEN-8** | `strategyArgs` on wire | **Closed (consumer merges into `args`)** |
+
+**Live proof:** `temp/funcx-functions` — 10/10 `run()` success on 4.9.8 (2 transient `fetch failed` on first batch; retries OK).
+
+---
+
+## 2. Gap taxonomy
+
+| Kind | Meaning | File to FuncX as |
+|------|---------|------------------|
+| **CR** | Shipped built-in behaves differently from documented consumer contract | Bug / contract fix |
+| **FR** | New capability or content not yet meeting consumer spec | Enhancement |
+
+---
+
+## 3. Change requests (CR) — fix shipped behavior
+
+### CR-FUNCX-1 — Built-ins use generic prompts, not Catalox / consumer templates
+
+**Priority:** P0  
+**Affects:** All 10 mapped functions  
+**Evidence:** `temp/funcx-functions/instructions-comparison.md`; FuncX `dist/functions/index.js` uses one-line system + full envelope JSON dump for execution/research; post/pre same pattern on 4.9.8.
+
+**Today:**
+
+```
+System: "Produce execution planning output only."
+User:   "You implement the FuncX built-in execution/plan… Payload (JSON): { … }"
+```
+
+**Expected:** Content-backed built-ins render Catalox `fx/{id}/` instructions equivalent to:
+
+- `xynthesis/templates/{actionType}/system-structured.md` + `user-structured.md`
+- `ai-tasks/templates/post-steps/audit/system.md` + `user.txt` / `synthesis.md`
+
+**Consumer spec:** `temp/funcx-functions/*/instructions-template.md`
+
+**Acceptance:**
+
+- [ ] Each of the 10 ids loads Catalox content (not hard-coded generic stub) when `content:primitives:sync` is deployed.
+- [ ] Rendered system+user prompt byte-matches normalized disk template for golden fixture envelopes (FR-GEN-9).
+- [ ] `model` on envelope body still honored.
+
+**Depends on:** FR-GEN-1 (template variable mapping).
+
+---
+
+### CR-FUNCX-2 — `post/audit-checklist` output is JSON; consumer requires markdown
+
+**Priority:** P0  
+**Affects:** `post/audit-checklist`  
+**Evidence:** Live 4.9.8 returned `{ checks: [{ check, kind, passed, feedback }], overall_feedback }`. ai-tasks expects markdown with `### Checks` and `### Overall feedback` (`templates/post-steps/audit/system.md`).
+
+**Expected output (markdown):**
+
+```markdown
+### Checks
+- **Check:** (echo check text)
+- **Kind:** must | should
+- **Passed:** yes | no
+- **Feedback:** …
+
+### Overall feedback
+…
+```
+
+**Acceptance:**
+
+- [ ] Built-in `responseFormat` / instructions enforce markdown sections above (not JSON checklist array).
+- [ ] Or: documented canonical JSON schema + explicit `outputMode: "markdown"|"json"` on envelope `args` with default `markdown` for ai-tasks compatibility.
+- [ ] Golden test vs `ai-tasks/templates/post-steps/audit/` fixtures.
+
+**Related FR:** FR-GEN-3 (publish output schema).
+
+---
+
+### CR-FUNCX-3 — `research/plan-questions` schema vs disk template drift
+
+**Priority:** P1  
+**Affects:** `research/plan-questions`  
+**Evidence:** Disk template requires `"source": "ai-driven"` per question; FuncX schema uses optional `sourceType` (string). Live 4.9.8 sometimes omits both; call still validates.
+
+**Expected (disk / xynthesis):**
+
+```json
+{ "question": "…", "reason": "…", "source": "ai-driven" }
+```
+
+**Acceptance (pick one and document):**
+
+- [ ] **A:** Schema requires `source` literal `"ai-driven"` (match disk).
+- [ ] **B:** Schema standardizes on `sourceType`; export migration note; xynthesis adapter updated (consumer change).
+
+**Consumer spec:** `temp/funcx-functions/research-plan-questions/instructions-template.md`
+
+---
+
+### CR-FUNCX-4 — `execution/plan` prompt vs output schema mismatch
+
+**Priority:** P1  
+**Affects:** `execution/plan`  
+**Evidence:** Live 4.9.6 runs failed `ERR_SCHEMA_INVALID` when model returned `confidence: "high"` and `suggestedContext: {}`. Schema requires `confidence: number`, `suggestedContext: string[]`. Disk template text does not mention these fields.
+
+**Acceptance:**
+
+- [ ] Catalox instructions explicitly require `confidence` (0–1 number) and `suggestedContext` (string array, may be empty `[]`).
+- [ ] Or: relax schema for optional fields with sensible defaults (document in FR-GEN-3).
+- [ ] Fewer than 5% schema failures on golden envelopes with `cheap/default`.
+
+**Consumer spec:** `temp/funcx-functions/execution-plan/output-object.md`
+
+---
+
+### CR-FUNCX-5 — Transient `fetch failed` loses `error.cause`
+
+**Priority:** P2  
+**Affects:** All `run()` OpenRouter hops  
+**Evidence:** Batch live run 8/10 + 2× `fetch failed`; retries succeeded. Error surface is only `message: "fetch failed"`.
+
+**Acceptance:**
+
+- [ ] `run()` / `askJson` errors preserve `cause` (`ECONNRESET`, `ETIMEDOUT`, etc.) on thrown/rejected errors.
+- [ ] Optional retry policy documented for embedders (exponential backoff on transport failures).
+
+---
+
+## 4. Feature requests (FR) — new upstream capability
+
+### FR-GEN-1 — `envelopeToTemplateVariables` (P0)
+
+**Status:** Open  
+**Blocks:** CR-FUNCX-1, FR-GEN-9
+
+**Ask:**
+
+```ts
+export function envelopeToTemplateVariables(
+  envelope: GenericExecutionEnvelope | SidekickGenericEnvelope
+): Record<string, string>;
+```
+
+**Minimum keys** (sidekick + execution):
+
+| Token | Source |
+|-------|--------|
+| `goal`, `question`, `task` | `envelope.goal` |
+| `input_json`, `context_json`, `args_json`, `result_json` | `JSON.stringify(...)` |
+| `strategy_args_json`, `iteration_index` | execution family |
+| `ai_output`, `used_instructions`, `used_prompt` | post sidekick `input` |
+| `local_raw`, `supporting_raw`, `rendered_downstream_*` | pre/synthesize `input` |
+| `candidates_json`, `prior_audit_json`, `metadata_json` | post sidekick |
+| Checklist: `originalInput`, `promptContext`, `candidateOutput`, `mustChecks`, `shouldChecks` | audit-checklist `input` |
+
+Also support nested `input.goal` style for existing `renderInstructionTemplate`.
+
+**Acceptance:** Used by all content-backed xynthesis built-ins before `callAI`.
+
+---
+
+### FR-GEN-9 — Catalox seed ↔ consumer template parity CI (P0)
+
+**Status:** Open  
+**Depends on:** FR-GEN-1
+
+**Ask:**
+
+1. Seed bundle `xynthesisSidekick.seed.json` (or extend `genericExecutionResearch.seed.json`) with all 10 ids.
+2. CI job: for each id, render Catalox instructions with golden envelope → compare hash to xynthesis rendrix / Handlebars output.
+3. Source files listed in `temp/funcx-functions/instructions-comparison.md`.
+
+**Acceptance:** CI fails on prompt drift; `contentVersion` bumped on intentional changes.
+
+---
+
+### FR-GEN-3 — Published output JSON Schema artifacts (P1)
+
+**Status:** Open
+
+**Ask:** Versioned JSON Schema per built-in id, co-located with FuncX package, matching xynthesis parsers:
+
+- `parseAuditPayload`, `parseFixPayload`, `parsePickBestPayload`, `parseCraftFinalPayload`
+- `adaptExecutionPlanResult`, `adaptEvaluateResultResponse`, `adaptResearchPlanQuestionsResult`
+- audit-checklist markdown contract OR JSON schema (per CR-FUNCX-2 resolution)
+
+**Acceptance:** `@exellix/xynthesis` / `@exellix/ai-tasks` can import schemas without duplicating types.
+
+---
+
+### FR-GEN-4 — `RunOptions` sampling passthrough (P1)
+
+**Status:** Open
+
+**Ask:** Add to `run()` options: `temperature`, `topP`, `reasoningEffort` (forwarded to nested `ask()` / OpenRouter).
+
+**Why:** xynthesis `FuncxInvoker.ask()` passes `temperature: 0.2` today; `run()` path cannot match without envelope hacks.
+
+**Acceptance:** Parity test: same sampling on `run()` vs direct `ask()`.
+
+---
+
+### FR-GEN-11 — Rich audit-merge envelope + template (P1)
+
+**Status:** Open (extends FR-PRE-POST-2)
+
+**Ask:** FuncX built-in + Catalox template for `post/audit-merge` accepts full fields from `ai-tasks/templates/post-steps/audit/synthesis.md`:
+
+- `originalInput`, `promptContext`
+- `candidateA` / `candidateB` with `cycle`, `score`, `output`, `failedChecks[]`, `passedChecks[]`
+- `customSynthesisGuidelines` (from `args`)
+
+**Consumer envelope builder:** `ai-tasks/src/post-steps/audit/auditChecklistFuncxEnvelope.ts` (expand merge builder).
+
+**Acceptance:** Merge output quality golden test vs Handlebars `synthesis.md` path.
+
+---
+
+### FR-PRE-POST-7 — Sidekick template context helper (P2)
+
+**Status:** Open  
+**Alternative to:** FR-GEN-1 if FuncX prefers a named helper.
+
+**Ask:** `buildSidekickTemplateContext(envelope)` wrapping FR-GEN-1 for sidekick ids only.
+
+---
+
+### FR-PRE-POST-8 — Export `SidekickGenericEnvelope` type (P2)
+
+**Status:** Open
+
+**Ask:** Export from `@x12i/funcx/functions` (or `@x12i/funcx/contracts`) matching `xynthesis/src/sidekickFuncxEnvelope.ts`.
+
+**Acceptance:** ai-tasks / xynthesis stop duplicating structural types.
+
+---
+
+### FR-GEN-2 — Per-function `args` JSON Schema in `meta.json` (P2)
+
+**Status:** Open (partial by convention)
+
+**Ask:** Document and validate `args` per id: `templateMode`, `customGuidelines`, `maxQuestions`, `allowSkip`, `iterationIndex`, `customAuditGuidelines`, etc.
+
+---
+
+### FR-GEN-5 — `runGenericExecution` embedder helper (P2)
+
+**Status:** Open
+
+**Ask:** Thin wrapper: content resolve → render (FR-GEN-1) → `askJson` → `getRunJsonResult` → `{ value, usage, model }`.
+
+---
+
+### FR-GEN-10 — Deprecate legacy hyphen ids (P3)
+
+**Status:** Open
+
+**Ask:** Mark `ai-tasks-plan-task`, `ai-tasks-optimizer-evaluate` deprecated; alias to `execution-plan`, `execution-evaluate-result`.
+
+---
+
+### FR-GEN-12 — Golden fixtures package (P2)
+
+**Status:** Open
+
+**Ask:** `@x12i/funcx/fixtures` or export from package: golden envelopes + expected prompt hashes + expected JSON outputs for 10 ids (sourced from `temp/funcx-functions`).
+
+---
+
+## 5. Per-function FuncX checklist
+
+| functionId | Registry 4.9.8 | CR / FR to close parity |
+|------------|----------------|-------------------------|
+| `pre/synthesize` | OK | CR-FUNCX-1, FR-GEN-1, FR-GEN-9 |
+| `execution/plan` | OK | CR-FUNCX-1, CR-FUNCX-4, FR-GEN-1, FR-GEN-9 |
+| `research/plan-questions` | OK | CR-FUNCX-1, CR-FUNCX-3, FR-GEN-1, FR-GEN-9 |
+| `execution/evaluate-result` | OK | CR-FUNCX-1, FR-GEN-1, FR-GEN-9 |
+| `post/audit` | OK | CR-FUNCX-1, FR-GEN-1, FR-GEN-9 |
+| `post/fix` | OK | CR-FUNCX-1, FR-GEN-1, FR-GEN-9 (+ `metadata_json` in user template) |
+| `post/pick-best` | OK | CR-FUNCX-1, FR-GEN-1, FR-GEN-9 (+ `metadata_json`) |
+| `post/craft-final` | OK | CR-FUNCX-1, FR-GEN-1, FR-GEN-9 (+ `metadata_json`) |
+| `post/audit-checklist` | OK | **CR-FUNCX-2**, FR-GEN-1, FR-GEN-3, FR-GEN-9 |
+| `post/audit-merge` | OK | CR-FUNCX-1, **FR-GEN-11**, FR-GEN-1, FR-GEN-9 |
+
+---
+
+## 6. Suggested FuncX release slices
+
+### 4.9.9 / 4.10.0 — P0 (production parity blockers)
+
+1. **FR-GEN-1** — `envelopeToTemplateVariables`
+2. **CR-FUNCX-1** — Catalox content for all 10 ids (seed + sync)
+3. **CR-FUNCX-2** — audit-checklist markdown output
+4. **FR-GEN-9** — parity CI
+
+### 4.10.x — P1
+
+5. **CR-FUNCX-3**, **CR-FUNCX-4** — schema / template alignment
+6. **FR-GEN-3** — published output schemas
+7. **FR-GEN-4** — RunOptions sampling
+8. **FR-GEN-11** — audit-merge full template
+
+### 4.11.x — P2/P3
+
+9. FR-PRE-POST-8, FR-GEN-2, FR-GEN-5, FR-GEN-10, FR-GEN-12, CR-FUNCX-5
+
+---
+
+## 7. Mono-repo owner (wired @ 4.9.13 cutover)
+
+| Item | Owner | Status |
+|------|--------|--------|
+| FuncX `run()` in `sidekickGateway.ts`, `runExecutionStrategyAction.ts` | xynthesis | **Done** — `invokeFuncxRun.ts`; no `XYNTHESIS_FUNCX_SIDEKICK` flag |
+| ai-tasks audit checklist/merge via FuncX envelopes | ai-tasks | **Done** — `runAuditFuncxCall.ts`; Handlebars templates removed |
+| Expand `buildAuditMergeFuncxEnvelope` for rich merge fields | ai-tasks | **Done** |
+| Catalox seed from `@x12i/funcx/content-seed` (`contentVersion: 4.9.13`) | xynthesis | **Done** |
+| FR-GEN-9 consumer parity CI before deleting disk templates | xynthesis | **Done** — `sidekickFuncxPromptParity.unit.ts` in `npm test` |
+| Structured-only PRE synthesis (no markdown path) | ai-tasks | **Done** |
+| `parseAuditChecklistOutput` (markdown + JSON object) | ai-tasks | **Done** — `parseAuditOutput.ts` |
+| Live golden e2e with OpenRouter | both | **Out of repo CI** — run `test:integration` / `RUN_SYNTHESIS_E2E=1` locally |
+
+---
+
+## 8. How to file in GitHub (`x12i/funcx`)
+
+| Upstream item | Suggested GitHub label |
+|---------------|------------------------|
+| CR-FUNCX-1 … CR-FUNCX-5 | `bug` or `contract` |
+| FR-GEN-1, FR-GEN-9 | `enhancement`, `P0` |
+| FR-GEN-11, CR-FUNCX-2 | `enhancement`, `ai-tasks-integration` |
+
+Copy bodies from this doc + link `temp/funcx-functions/{folder}/live-run.md` as evidence.
+
+---
+
+## 9. Version note (update peer docs)
+
+| Package | Version | Notes |
+|---------|---------|-------|
+| `@x12i/funcx` | **4.9.9** | Content-backed sidekicks; 9/10 live (transient fetch on `pre/synthesize`) |
+| `@exellix/xynthesis` | **^4.9.9** dep | `invokeFuncxRun` + gateway flag wiring |
+| `@exellix/ai-tasks` | **^4.9.9** dep | Audit FuncX path behind same flag |
+
+---
+
+## 10. Recheck log (4.9.11 → 4.9.13)
+
+**Latest:** `@x12i/funcx` **4.9.13** live OpenRouter, 2026-06-12 (`temp/funcx-functions`)
+
+| Id | Status @ 4.9.13 | Notes |
+|----|-----------------|-------|
+| **CR-FUNCX-5** | **Closed** | `error.cause` on transport errors (4.9.9); embedder retry docs (4.9.12); `RunOptions.transportRetries` (4.9.13). `pre/synthesize` OK with `transportRetries: 2`. |
+| **FR-GEN-3** | **Closed** | `getFuncxOutputSchema` CI (4.9.12); checklist/merge `outputSchemaType: "string"` in fixtures manifest. |
+| **FR-GEN-9** | **Closed** | `sidekickPromptParity.test.ts` + `@x12i/funcx/fixtures` hashes. Consumer parity CI in mono-repo **done** (`sidekickFuncxPromptParity.unit.ts`). |
+| **FR-GEN-11** | **Closed** | Rich merge `prompt.md` + `post-audit-merge-envelope.json` golden fixture with checks arrays. |
+| **CR-FUNCX-2** | **Closed** | Markdown checklist @ 4.9.11+ — unchanged @ 4.9.13. |
+| **CR-FUNCX-1, FR-GEN-1, FR-GEN-4, FR-GEN-12, CR-FUNCX-3, CR-FUNCX-4** | **Closed** | Unchanged. |
+
+**Mono-repo cutover complete @ 4.9.13.** See [funcx-4.9.13-open-items.md](./funcx-4.9.13-open-items.md).

package/documenations/upstream-feature-requests/funcx-pre-post-sidekick-actions.md

@@ -2,6 +2,7 @@
 
 **Filed by:** `@exellix/ai-tasks` + `@exellix/xynthesis`  
 **Baseline:** `@x12i/funcx` **4.4.0** — execution/research generics **shipped**; pre/post ids below are **not** in registry yet  
+**Updated:** See [funcx-gap-analysis-cr-fr.md](./funcx-gap-analysis-cr-fr.md) for **4.9.8** status (10/10 registered; content parity CRs/FRs open).
 **Blocks:** `XYNTHESIS_FUNCX_SIDEKICK=1` production use, ai-tasks audit/polish off Handlebars  
 **Related:** [funcx-generic-xynthesis-hosting.md](./funcx-generic-xynthesis-hosting.md) (4.4.0 baseline + execution FRs)