nduka 1.0.0

package/docs/CLAUDE_STYLE_TOOLING_HANDOVER.md

@@ -0,0 +1,1188 @@
+# Claude-Style Tooling and MCP Handover Guide
+
+Branch context: `work/next-milestone-async-jobs`
+
+Base checkpoint:
+- `context-memory-best-v1`
+- `stable/context-memory-best`
+- `work/from-context-memory-best-v1`
+- Current intended work branch: `work/next-milestone-async-jobs`
+
+This guide is the source-of-truth handover for building Claude-style tool use, async tool jobs, follow-up tool context, and MCP connector support into the WhatsApp automation bot without damaging the currently working normal chat memory behavior.
+
+## 0. Non-negotiable rules
+
+These rules apply to every file, every milestone, every test, and every future handover.
+
+1. Do not hardcode routes, model names, tool names, provider names, test live examples, customer examples, or magic behavior in runtime logic.
+2. Tool selection must be config-driven through the existing tool catalog, connector manifest, MCP registry, and/or settings files.
+3. Normal chat behavior must not be touched unless a test proves the change is required and safe.
+4. Normal chat must keep the current memory/retrieval behavior from the green context-memory branch.
+5. A message must only enter the Claude-style tool lane when a cheap, config-driven detector flags a realistic tool need.
+6. The Claude planner can propose a tool call, but deterministic runtime code must validate it before any execution or async promise.
+7. The model is never the authority on permissions, risk, tool availability, private data, or whether a job was actually started.
+8. Never say "I'll check", "I'll follow up", "I'm looking into it", or any similar future-work promise until the async job has been successfully persisted.
+9. Customer slash commands must not expose owner/admin/tool controls.
+10. Owner-only tools must never be callable by customers.
+11. Private account/payment/fees/balances/due dates must never be answered from normal chat memory.
+12. All errors must be logged directly and cleanly with a reason, correlation ID, chat JID where safe, job ID where relevant, and no secret leakage.
+13. Logs must be useful, short, and structured. Do not dump full raw messages, secrets, tokens, or complete tool outputs into logs.
+14. Every tool call must be schema-validated, policy-checked, audited, and bounded by timeout/retry rules.
+15. Every new branch must start from the current stable context-memory base unless intentionally stated otherwise.
+16. Do not revive the previous failed tool approach. Do not re-add normal-chat tool surface language. Do not route by hardcoded text keywords.
+17. Research before implementation. If docs may have changed, consult official docs first.
+18. If behavior is unclear, inspect the repo before changing it. Do not guess.
+19. Testing should be planned from the beginning, but the user should not be asked to test after every small milestone. Implement the full planned integration, then run `npm run build` and `npm test` at the end. Targeted tests may be run internally when needed to diagnose failures, but the handover must not stop at every milestone asking the user to validate.
+20. This project must stay owner-reviewable. Self-improvement may generate tests and patches, but must not auto-merge or auto-push code without owner approval.
+
+## 1. Research foundation
+
+### 1.1 Claude tool-use style
+
+Claude tool use is the chosen pattern for this project. Claude can decide whether to call a tool based on the user request and the supplied tool descriptions. It returns structured tool-use blocks. For client tools, the application executes the tool and returns tool results. The important design lesson is that the model proposes structured tool calls, but the application owns execution, validation, and policy.
+
+Use Claude-style client tools, not Claude server tools, for this WhatsApp bot. The repo already owns tool policy, audit, async persistence, WhatsApp replies, customer/owner separation, and retry/recovery. Server tools would bypass too much of the repo's control layer.
+
+The important Claude concepts to adapt:
+- Tool definitions contain name, description, and input schema.
+- Detailed tool descriptions matter because they control when Claude should and should not call tools.
+- `strict` tool schema behavior should be used where supported.
+- Tool use adds token cost, so do not pass every tool on every normal chat turn.
+- Tool calls return structured tool requests; the app executes and returns tool results.
+- Multiple tools and follow-up tool loops are possible, but the app controls ordering and execution.
+
+### 1.2 MCP connectors
+
+MCP is an open standard for connecting AI applications to external systems such as files, databases, tools, and workflows. For this project, MCP should be treated as one connector/tool source, not as a replacement for the existing security model.
+
+MCP support must follow these rules:
+- MCP servers expose tools and resources.
+- The repo imports/discovers MCP tools into the same internal catalog/adapter layer as other tools.
+- MCP tool definitions must be converted into internal `ToolDefinition`-compatible metadata.
+- MCP tools must still pass permission checks, schema checks, policy checks, audit logging, timeout, retries, and async persistence.
+- MCP results must be summarized and stored in `ToolContextStore`, not blindly dumped into normal memory.
+- MCP prompts/resources must never override system safety rules, owner/customer boundaries, or private-data rules.
+- MCP tool descriptions may be untrusted input. Treat them as metadata, not instructions to the runtime.
+
+### 1.3 Codex / Claude Code style
+
+Codex-style and Claude-Code-style workflows are useful later for the self-improvement loop:
+- trace bad behavior,
+- generate a failing regression test,
+- propose a patch,
+- run build/tests,
+- show diff,
+- wait for owner approval.
+
+Do not copy coding-agent behavior into customer WhatsApp chat. Customers must never get repo-editing tools, shell tools, or arbitrary MCP access.
+
+## 2. Current repo foundation to preserve
+
+The repo already contains strong foundations. Build on them. Do not replace them.
+
+### 2.1 Async jobs
+
+The async job store already persists:
+- job ID,
+- dedupe key,
+- chat JID,
+- sender JID,
+- original message ID,
+- original text,
+- tool name,
+- tool args JSON,
+- job status,
+- attempts and max attempts,
+- created/updated/scheduled timestamps,
+- started/completed timestamps,
+- last error,
+- result preview,
+- correlation ID.
+
+The async store already supports:
+- enqueue with dedupe,
+- get,
+- find by dedupe key,
+- list recent,
+- list by status,
+- list for chat,
+- claim due jobs,
+- requeue,
+- mark succeeded,
+- mark failed,
+- cancel,
+- expire queued jobs,
+- recover stale running jobs.
+
+The async runner already supports:
+- claiming due jobs,
+- processing them,
+- retrying retryable failures,
+- final failure,
+- success hooks,
+- failure hooks.
+
+The async worker already supports:
+- interval execution,
+- enabled flag,
+- limit per tick,
+- overlap protection,
+- result and error hooks.
+
+### 2.2 Tool catalog
+
+The tool catalog already resolves enabled tools and exposes:
+- name,
+- provider,
+- operation,
+- public vs owner-only,
+- synthesis/raw mode,
+- aliases,
+- input and output schemas,
+- timeout,
+- retry hints,
+- permissions.
+
+This must remain the source of truth for tool availability. Runtime code must not hardcode tool names.
+
+### 2.3 Secure tool runtime
+
+The secure tool runtime already supports:
+- schema validation,
+- output validation,
+- retries,
+- timeout,
+- policy evaluation,
+- audit entries,
+- tenant permission checks.
+
+All new Claude/MCP tool calls must go through this runtime or an equivalent adapter that preserves the same policy and audit guarantees.
+
+### 2.4 Normal chat memory branch
+
+The current green branch fixed the memory issue:
+- no hidden engine history cap,
+- no duplicate current user message,
+- normal chat remembers recent conversation,
+- quoted WhatsApp context is passed into normal chat history,
+- tool language removed from memory-only branch,
+- model routing is settings-backed,
+- no hardcoded model routing.
+
+Do not disturb this path.
+
+## 3. High-level architecture
+
+The new architecture is a separate tool-needed lane.
+
+```
+Incoming WhatsApp message
+  -> existing production runtime gates
+  -> owner/customer/group/private-data checks
+  -> cheap config-driven tool candidate detector
+      -> if no candidate: current normal chat memory path
+      -> if candidate: Claude-style tool planner
+          -> deterministic plan validation
+          -> policy and permission checks
+          -> async job persistence
+          -> optional immediate execution with short deadline
+          -> tool result summarization
+          -> tool context storage
+          -> follow-up handling
+```
+
+The normal chat path must remain:
+
+```
+Incoming message
+  -> existing gates
+  -> no tool candidate
+  -> processWithNormalChatMemory(...)
+  -> existing AI response path
+```
+
+## 4. Key principle: no normal chat regression
+
+The following examples must stay in normal chat and must not trigger Claude tool planning:
+- "Do you remember my friend?"
+- "What did I tell you earlier?"
+- "Can you rewrite this?"
+- "Explain this."
+- "Thanks."
+- "Okay."
+- "How are you?"
+- "What did I say before?"
+- "Continue."
+
+The following may trigger the tool lane only if configured tool metadata supports it:
+- requests for current/latest information,
+- explicit search/fetch requests in natural language,
+- requests that need an external system,
+- requests that depend on a connector/MCP resource,
+- follow-ups referring to recent tool results,
+- owner-approved write/admin actions.
+
+## 5. Two-stage tool detection
+
+### 5.1 Stage 1: cheap configured candidate detector
+
+Create:
+
+```
+src/agent-tools/tool-candidate-detector.ts
+```
+
+Purpose:
+- fast local check before spending Claude tokens,
+- decide if the message is worth tool planning,
+- return top candidate tools only,
+- fall through to normal chat when uncertain.
+
+Inputs:
+- message text,
+- actor role: customer | owner | group,
+- chat type,
+- configured enabled tool summaries,
+- connector/MCP tool summaries,
+- recent tool contexts for this chat,
+- private-data guard result,
+- existing runtime command/gate state.
+
+Output shape:
+
+```ts
+export interface ToolCandidateDetection {
+  shouldPlanWithClaude: boolean;
+  candidateToolNames: string[];
+  reason: string;
+  confidence: number;
+  followupHint?: {
+    possibleToolFollowup: boolean;
+    contextId?: string;
+    kind?: 'answer_from_context' | 'same_tool' | 'new_tool' | 'clarify';
+  };
+}
+```
+
+Rules:
+- No hardcoded tool names.
+- Use tool catalog metadata, aliases, trigger phrases, tags, operation type, and descriptions.
+- Conservative default: if uncertain, return `shouldPlanWithClaude: false`.
+- Follow-up language such as "what about it", "how about it", "what about the second one", "compare that", "open it", or "summarize that" may be a tool follow-up only when there is recent tool context.
+- If there is no recent tool context, vague follow-ups should remain normal chat or ask a normal clarification.
+- Do not route normal memory questions to tools.
+
+### 5.2 Stage 2: Claude tool planner
+
+Create:
+
+```
+src/agent-tools/claude-tool-planner.ts
+```
+
+Purpose:
+- given a candidate subset, ask Claude whether to call a tool, ask a clarification, deny, or let normal chat continue.
+
+Inputs:
+- candidate tools only,
+- actor role and chat type,
+- message text,
+- recent normal memory summary,
+- recent tool context summaries,
+- private-data guard state,
+- allowed max steps,
+- connector/MCP availability.
+
+Output shape:
+
+```ts
+export type ClaudeToolPlannerDecision =
+  | { decision: 'no_tool'; reason: string; confidence: number }
+  | { decision: 'ask_clarifying_question'; reply: string; reason: string; confidence: number }
+  | { decision: 'deny'; reply: string; reason: string; confidence: number }
+  | {
+      decision: 'tool_use';
+      toolCalls: Array<{
+        toolName: string;
+        arguments: Record<string, unknown>;
+        reason: string;
+      }>;
+      reason: string;
+      confidence: number;
+      requiresAsync: boolean;
+      requiresOwnerApproval: boolean;
+    };
+```
+
+Important:
+- Claude planner is not trusted.
+- Its output must be validated locally.
+- Unknown tool names must be rejected.
+- Extra/invalid args must be rejected.
+- Owner-only tools must be rejected for customers.
+- Private account tools require explicit allowed path and/or owner approval.
+
+## 6. Claude tool adapter
+
+Create:
+
+```
+src/agent-tools/claude-tool-adapter.ts
+```
+
+Purpose:
+- convert internal tool definitions/catalog tools/MCP tools into Claude-compatible tool definitions.
+
+Rules:
+- Only pass candidate tools.
+- Include detailed descriptions generated from config, not hardcoded route logic.
+- Include when-to-use and when-not-to-use text from config.
+- Include input schema.
+- Use strict schema mode where supported.
+- Do not expose secrets.
+- Do not expose owner-only tools to customer planning.
+- Do not include disabled tools.
+
+Suggested function:
+
+```ts
+export function toClaudeToolDefinitions(
+  tools: ResolvedCatalogTool[],
+  actorRole: 'customer' | 'owner' | 'group',
+): ClaudeToolDefinition[]
+```
+
+## 7. Tool plan validator
+
+Create:
+
+```
+src/agent-tools/tool-plan-validator.ts
+```
+
+Purpose:
+- deterministic validation of Claude's plan.
+
+Checks:
+1. Tool exists.
+2. Tool is enabled.
+3. Tool is in the candidate subset.
+4. Actor role is allowed.
+5. Chat type is allowed.
+6. Input schema validates.
+7. No extra args unless explicitly allowed.
+8. Secret fields are not model-supplied.
+9. Managed fields are filled by runtime, not the model.
+10. Private data policy is respected.
+11. Tool risk level is allowed.
+12. Owner approval is required where configured.
+13. Async eligibility is respected.
+14. Maximum tool steps is respected.
+15. MCP tool source is trusted/enabled.
+
+Output:
+
+```ts
+export interface ToolPlanValidationResult {
+  ok: boolean;
+  acceptedCalls: ValidatedToolCallPlan[];
+  rejectedCalls: Array<{ toolName?: string; reason: string; code: string }>;
+  requiresOwnerApproval: boolean;
+  requiresAsync: boolean;
+  safeReplyIfRejected?: string;
+}
+```
+
+Logging:
+- `AgentToolPlanValidation` with reason codes.
+- Do not log full args if they include secrets or PII.
+- Include correlation ID.
+
+## 8. Async tool bridge
+
+Create:
+
+```
+src/agent-tools/async-tool-bridge.ts
+```
+
+Purpose:
+- convert validated tool plans into durable async jobs,
+- optionally execute immediately within a short deadline,
+- never promise follow-up before persistence.
+
+Flow:
+
+```
+validated tool call
+  -> build dedupe key
+  -> AsyncJobStore.enqueue()
+  -> if enqueue fails: no promise
+  -> if enqueue succeeds:
+       optionally try immediate execution
+       if quick success: summarize and send final result
+       if still running: send pending acknowledgement
+```
+
+Pending acknowledgement rule:
+- Only after successful `AsyncJobStore.enqueue()`.
+- If deduped existing job exists, reply should say it is already being checked if appropriate.
+- If enqueue fails, say the lookup could not be started.
+
+Suggested output:
+
+```ts
+export type AsyncToolBridgeResult =
+  | { handled: true; mode: 'queued'; reply: string; jobIds: string[] }
+  | { handled: true; mode: 'completed_immediately'; reply: string; jobIds: string[] }
+  | { handled: true; mode: 'failed_to_enqueue'; reply: string }
+  | { handled: false };
+```
+
+## 9. Tool execution processor
+
+Create or complete:
+
+```
+src/agent-tools/async-tool-processor.ts
+```
+
+Purpose:
+- connect `AsyncJobRunner` to `ToolExecutionRuntime`.
+
+Flow:
+
+```
+AsyncJobRecord
+  -> parse tool_args_json
+  -> create ToolCall with tenant/chat/user/correlation
+  -> ToolExecutionRuntime.execute()
+  -> return AsyncJobProcessorResult
+```
+
+Rules:
+- All parse errors must be non-retryable.
+- Provider timeouts may be retryable.
+- Permission/policy errors are non-retryable.
+- Schema errors are non-retryable.
+- Network/provider transient errors may be retryable.
+- Unknown tool is non-retryable.
+- Every failure must have clean `errorCode` and `errorMessage`.
+- No raw secret output in result preview or logs.
+
+## 10. Tool result summarizer
+
+Create:
+
+```
+src/agent-tools/tool-result-summarizer.ts
+```
+
+Purpose:
+- convert raw tool result to WhatsApp-safe answer.
+
+Rules:
+- Keep concise.
+- Cite/source where available.
+- If result is partial, say so.
+- If provider timed out, say so.
+- Do not expose internal tool args.
+- Do not expose secrets.
+- Do not pretend certainty if tool result is ambiguous.
+- Do not claim work was done if tool failed.
+- Store a compact summary in tool context.
+- Optionally store an even shorter memory marker in normal memory, but do not dump raw JSON into normal memory.
+
+## 11. Tool context store
+
+Create:
+
+```
+src/agent-tools/tool-context-store.ts
+```
+
+Purpose:
+- store structured tool results for follow-up reasoning.
+
+Suggested table:
+
+```
+tool_contexts (
+  context_id TEXT PRIMARY KEY,
+  chat_jid TEXT NOT NULL,
+  source_message_id TEXT NOT NULL,
+  correlation_id TEXT NOT NULL,
+  job_id TEXT,
+  tool_name TEXT NOT NULL,
+  tool_args_json TEXT NOT NULL,
+  result_summary TEXT NOT NULL,
+  result_preview TEXT,
+  result_json TEXT,
+  source_urls_json TEXT,
+  citations_json TEXT,
+  entities_json TEXT,
+  created_at INTEGER NOT NULL,
+  expires_at INTEGER,
+  status TEXT NOT NULL
+)
+```
+
+Also consider:
+
+```
+tool_sessions (
+  session_id TEXT PRIMARY KEY,
+  chat_jid TEXT NOT NULL,
+  original_message_id TEXT NOT NULL,
+  correlation_id TEXT NOT NULL,
+  status TEXT NOT NULL,
+  step_count INTEGER NOT NULL,
+  created_at INTEGER NOT NULL,
+  updated_at INTEGER NOT NULL
+)
+
+tool_session_steps (
+  session_id TEXT NOT NULL,
+  step_index INTEGER NOT NULL,
+  context_id TEXT,
+  tool_name TEXT NOT NULL,
+  status TEXT NOT NULL,
+  depends_on_step_index INTEGER,
+  PRIMARY KEY (session_id, step_index)
+)
+```
+
+Rules:
+- Bound context size.
+- Expire old contexts.
+- Keep summaries short.
+- Keep structured metadata for references such as "the second one", "that link", "it", "there", "Saturday", etc.
+- Do not store secrets.
+- Do not store full sensitive account data in ordinary tool context unless explicitly designed and protected.
+
+## 12. Follow-up resolver
+
+Create:
+
+```
+src/agent-tools/tool-followup-resolver.ts
+```
+
+Purpose:
+- understand vague follow-ups after tool results.
+
+Examples:
+- "How about it?"
+- "What about that?"
+- "Open it."
+- "Summarize the second one."
+- "What about Saturday?"
+- "Compare those."
+- "Fetch that page."
+- "Is that still true?"
+
+Resolver inputs:
+- current message,
+- recent tool contexts,
+- recent normal memory,
+- actor role,
+- candidate tools.
+
+Resolver output:
+
+```ts
+export interface ToolFollowupResolution {
+  isToolFollowup: boolean;
+  confidence: number;
+  relatedContextId?: string;
+  relatedContextIds?: string[];
+  followupKind?: 'answer_from_context' | 'call_same_tool' | 'call_new_tool' | 'clarify';
+  candidateToolNames: string[];
+  contextSummary?: string;
+  clarificationReply?: string;
+  reason: string;
+}
+```
+
+Rules:
+- A vague phrase like "how about it" is a tool follow-up only if a recent tool result exists and the reference is clear enough.
+- If no recent tool context exists, it should not become a tool follow-up.
+- If multiple recent results exist and the reference is unclear, ask a clarification.
+- Normal chat references like "Do you remember it?" should use normal memory unless recent tool context strongly dominates.
+- Do not force a tool call if the answer can be given from stored tool context.
+- If the user changes a parameter like date/location/entity, call the same tool again with updated args.
+- If the user asks to fetch/open/inspect a result URL, use a fetch-type tool if configured.
+
+## 13. Tool chaining
+
+V1 should support controlled sequential chaining only.
+
+Rules:
+- Max 2 tool steps per user request in V1.
+- Read-only tools only for chaining.
+- No write/admin tools in a chain without explicit owner approval.
+- No parallel execution in V1.
+- Every step must be stored in a tool session.
+- Each step must have its own validation, policy decision, audit entry, timeout, and result summary.
+- If step 1 fails, do not run step 2.
+- If step 2 needs clarification, ask the user.
+- If Claude proposes more than max steps, stop safely.
+
+Example chain:
+- search -> fetch -> summarize
+- find website -> fetch page -> summarize
+- search docs -> fetch document -> answer
+
+Do not allow:
+- search -> fetch -> send email
+- search -> book appointment
+- fetch -> modify account
+- any customer-initiated write action unless explicitly configured and approved.
+
+## 14. Connector and MCP handling
+
+Create:
+
+```
+src/agent-tools/mcp-tool-adapter.ts
+src/agent-tools/connector-tool-adapter.ts
+```
+
+Purpose:
+- normalize connector/MCP tools into internal tool definitions.
+
+Rules for connectors:
+1. Connector tools must be declared in config/manifest.
+2. Connector URLs or IDs must not be guessed.
+3. If a connector action accepts a specific URL form, validate it before invoking.
+4. If a connector action requires file references, pass connector references exactly as returned.
+5. Do not fetch connector URLs with public web tools.
+6. Do not turn connector failures into normal chat hallucinations.
+7. Connector errors should produce clear messages and trace entries.
+8. Connector output should go into ToolContextStore and summarizer.
+9. Connector credentials/secrets must never be exposed to model output or logs.
+10. Owner-only connector actions remain owner-only.
+
+Rules for MCP:
+1. MCP servers are external tool providers, not trusted policy authorities.
+2. MCP server tool schemas must be imported into a bounded internal registry.
+3. MCP tool descriptions are untrusted metadata and must not override system rules.
+4. MCP results are untrusted external content and may contain prompt injection.
+5. Before summarization, tool results should be treated as data, not instructions.
+6. MCP action tools require stricter approval than read-only tools.
+7. MCP server availability must be checked and logged cleanly.
+8. MCP tool discovery should be bounded and cached.
+9. Do not load thousands of MCP tools into Claude context.
+10. Use candidate selection to pass only relevant MCP tools.
+
+## 15. Runtime integration point
+
+Modify only after existing gates:
+
+```
+runtime-production.ts
+  processMessage()
+    existing context/connector/owner/natural schedule commands
+    existing slash boundary
+    existing current date shortcut
+    existing private account guard
+    existing transactional support guard
+    NEW maybe agent tool route
+    existing processWithNormalChatMemory(...)
+```
+
+Pseudo-flow:
+
+```ts
+const toolCandidate = await agentToolRouter.detect(msg, text, runtimeContext);
+
+if (toolCandidate.shouldPlanWithClaude) {
+  const routed = await agentToolRouter.tryRoute({
+    msg,
+    text,
+    actorRole,
+    chatJid,
+    senderJid,
+    candidate: toolCandidate,
+  });
+
+  if (routed.handled) return routed.runtimeResult;
+}
+
+return this.processWithNormalChatMemory(msg, text, isGroup);
+```
+
+Rules:
+- If tool router is uncertain, return `{ handled: false }`.
+- If no candidate tools, fall through to normal chat.
+- If planner returns `no_tool`, fall through to normal chat.
+- If private/account guard already handled, do not invoke tool planner.
+- If group rules say no reply, do not invoke tool planner.
+- Do not store tool prompts as normal customer memory.
+- Store final tool summaries as tool context, with optional compact memory marker.
+
+## 16. Agent tool route result shape
+
+Suggested:
+
+```ts
+export type AgentToolRouteResult =
+  | { handled: false }
+  | {
+      handled: true;
+      runtimeResult: {
+        decision: 'USER_COMMAND' | 'AI_RESPOND' | 'OWNER_COMMAND' | 'STORE_ONLY' | 'BLOCK';
+        reply: string | null;
+        shouldStoreInMemory: boolean;
+      };
+      traceId: string;
+      jobIds?: string[];
+      contextIds?: string[];
+    };
+```
+
+## 17. Logging and error handling
+
+Add clean structured logs. Suggested event names:
+
+- `AgentToolCandidate`
+- `AgentToolPlannerStart`
+- `AgentToolPlannerResult`
+- `AgentToolPlanRejected`
+- `AgentToolPlanAccepted`
+- `AgentToolJobEnqueued`
+- `AgentToolJobDedupe`
+- `AgentToolImmediateStart`
+- `AgentToolImmediateTimeout`
+- `AgentToolExecutionSucceeded`
+- `AgentToolExecutionFailed`
+- `AgentToolContextStored`
+- `AgentToolFollowupResolved`
+- `AgentToolFollowupClarify`
+- `AgentToolMcpDiscoveryFailed`
+- `AgentToolConnectorFailed`
+- `AgentToolPolicyDenied`
+
+Every log should include:
+- event,
+- correlationId,
+- chatJid hash or JID if existing logging allows,
+- actorRole,
+- candidate count,
+- toolName where safe,
+- reason code,
+- jobId/contextId where available,
+- latencyMs where relevant.
+
+Do not log:
+- API keys,
+- secrets,
+- customer secret fields,
+- raw complete tool results,
+- full private account data,
+- long message bodies,
+- auth tokens.
+
+Errors must be direct:
+- `TOOL_UNKNOWN`
+- `TOOL_DISABLED`
+- `TOOL_NOT_ALLOWED_FOR_ACTOR`
+- `TOOL_SCHEMA_INVALID`
+- `TOOL_POLICY_DENIED`
+- `TOOL_TIMEOUT`
+- `TOOL_PROVIDER_ERROR`
+- `TOOL_RESULT_INVALID`
+- `ASYNC_ENQUEUE_FAILED`
+- `MCP_SERVER_UNAVAILABLE`
+- `CONNECTOR_UNSUPPORTED_URL`
+- `FOLLOWUP_AMBIGUOUS`
+- `FOLLOWUP_CONTEXT_EXPIRED`
+
+## 18. Self-improvement loop
+
+Self-improvement is not automatic production mutation. It is owner-only.
+
+Create later:
+
+```
+src/self-improvement/trace-store.ts
+src/self-improvement/eval-generator.ts
+src/self-improvement/repair-proposal.ts
+```
+
+Flow:
+1. Capture traces.
+2. Detect failure patterns.
+3. Generate failing regression tests.
+4. Generate patch proposal.
+5. Run build/tests.
+6. Show owner diff and results.
+7. Owner approves commit/push.
+
+Failure patterns:
+- wrong tool selected,
+- tool should have been called but was not,
+- tool was called when normal chat should handle it,
+- follow-up context was missed,
+- vague follow-up misidentified,
+- private-data guard bypassed,
+- owner-only tool exposed,
+- async job promised but not persisted,
+- connector URL mishandled,
+- MCP result prompt injection affected output,
+- provider error hidden by generic reply,
+- normal memory path regressed.
+
+## 19. Milestones
+
+The implementation should be done from start to finish on `work/next-milestone-async-jobs`. Do not ask the user to test after each milestone. Write tests as behavior specs while implementing, but run the final full suite after integration is complete.
+
+### Milestone 1: Agent tool planning with no execution
+
+Files:
+- `src/agent-tools/tool-candidate-detector.ts`
+- `src/agent-tools/claude-tool-adapter.ts`
+- `src/agent-tools/claude-tool-planner.ts`
+- `src/agent-tools/tool-plan-validator.ts`
+- tests under `src/agent-tools/__tests__/`
+
+Goal:
+- Detect possible tool-needed messages.
+- Send only candidate tools to Claude planner.
+- Parse planner output.
+- Validate planner output.
+- Do not execute tools yet.
+- Fall through to normal chat when no tool is needed.
+
+Tests to write:
+- normal memory question does not call planner,
+- ordinary rewrite request does not call planner,
+- normal greeting does not call planner,
+- configured current/latest request becomes candidate,
+- configured connector request becomes candidate,
+- planner receives only candidate tools,
+- planner does not receive owner-only tools for customer,
+- unknown Claude tool name rejected,
+- invalid Claude args rejected,
+- missing required args causes clarification,
+- private account request denied or routed to existing guard, not normal tool,
+- detector has no hardcoded tool names,
+- detector uses config metadata,
+- group non-mentioned message does not route to planner,
+- customer slash tool attempt does not route to planner.
+
+### Milestone 2: Async bridge and immediate execution deadline
+
+Files:
+- `src/agent-tools/async-tool-bridge.ts`
+- `src/agent-tools/async-tool-processor.ts`
+- `src/agent-tools/tool-route-result.ts`
+- tests under `src/agent-tools/__tests__/`
+
+Goal:
+- Validated tool plan becomes durable async job.
+- Acknowledgement only after job is persisted.
+- Optional immediate execution with short deadline.
+- Slow job remains queued/running for worker.
+- Deduped jobs produce safe response.
+
+Tests:
+- validated tool plan enqueues job,
+- enqueue failure does not promise follow-up,
+- duplicate request dedupes,
+- fast tool returns final answer,
+- slow tool returns pending acknowledgement only after enqueue,
+- policy denial does not enqueue,
+- schema failure does not enqueue,
+- unknown tool does not enqueue,
+- job stores original message ID/text,
+- job stores correlation ID,
+- errors have reason codes.
+
+### Milestone 3: Tool result summarization and context storage
+
+Files:
+- `src/agent-tools/tool-result-summarizer.ts`
+- `src/agent-tools/tool-context-store.ts`
+- migration/init logic for new tables
+- tests under `src/agent-tools/__tests__/`
+
+Goal:
+- Summarize tool outputs into WhatsApp-safe replies.
+- Store structured tool context for follow-ups.
+- Do not dump raw tool JSON into normal memory.
+- Keep source/citation metadata where available.
+- Bound context size and expiry.
+
+Tests:
+- successful result creates tool context,
+- failed result creates clean failure trace,
+- summary hides secrets,
+- summary handles partial results,
+- summary includes sources when result has sources,
+- large result is bounded,
+- expired context is not used,
+- context lookup is per chat,
+- one chat cannot use another chat's tool context.
+
+### Milestone 4: Follow-up resolver
+
+Files:
+- `src/agent-tools/tool-followup-resolver.ts`
+- tests under `src/agent-tools/__tests__/`
+
+Goal:
+- Vague follow-ups can refer to prior tool results without becoming normal chat.
+- But normal chat must not be misidentified as a tool follow-up.
+
+Tests:
+- "How about it?" after a single recent tool result resolves to that context,
+- "What about the second one?" resolves to second result when metadata exists,
+- "Open it" after search result routes to fetch candidate if configured,
+- "What about Saturday?" after weather result reuses location and changes date,
+- "Summarize that" after tool result answers from context without new tool,
+- vague follow-up with no tool context falls through to normal chat,
+- vague follow-up with multiple possible contexts asks clarification,
+- "Do you remember it?" uses normal memory when no tool context dominates,
+- expired context causes clarification or normal fallback,
+- owner/customer permissions still enforced on follow-up tool calls.
+
+### Milestone 5: Controlled chaining
+
+Files:
+- `src/agent-tools/tool-session-store.ts`
+- `src/agent-tools/tool-chain-runner.ts`
+- tests under `src/agent-tools/__tests__/`
+
+Goal:
+- Sequential read-only chains only.
+- Max 2 tool steps in V1.
+- Every step is validated, audited, and stored.
+- No write/admin chains without owner approval.
+
+Tests:
+- search -> fetch works,
+- chain stores session and steps,
+- chain stops after max steps,
+- step 2 not run if step 1 fails,
+- write/admin tool in chain requires approval,
+- customer cannot chain into owner-only tool,
+- parallel execution disabled in V1,
+- chained result gets one final summary.
+
+### Milestone 6: MCP and connector integration
+
+Files:
+- `src/agent-tools/mcp-tool-adapter.ts`
+- `src/agent-tools/connector-tool-adapter.ts`
+- tests under `src/agent-tools/__tests__/`
+
+Goal:
+- MCP/connector tools become catalog-compatible tool definitions.
+- Tool discovery is bounded and cached.
+- Connector/MCP failures are cleanly handled.
+- MCP result prompt injection is treated as data, not instruction.
+
+Tests:
+- MCP tool schema converts to internal definition,
+- MCP disabled server is ignored,
+- MCP unknown server error is clean,
+- MCP tool prompt injection is not followed,
+- connector URL must match supported schema,
+- connector file references are preserved exactly,
+- connector raw failures do not become normal hallucinations,
+- owner-only connector tool not exposed to customer,
+- candidate detector only sees enabled connector/MCP tools,
+- large MCP catalog is trimmed to candidate subset.
+
+### Milestone 7: Runtime integration
+
+Files:
+- `src/conversation/runtime-production.ts`
+- `src/agent-tools/agent-tool-router.ts`
+- tests under runtime and agent-tools.
+
+Goal:
+- Plug in after existing gates and before `processWithNormalChatMemory`.
+- Normal chat unchanged.
+- Tool route handles only flagged messages.
+- Tool route falls through safely when uncertain.
+
+Tests:
+- normal memory branch still passes existing regression tests,
+- tool-needed branch routes to agent router,
+- no candidate falls through to normal chat,
+- planner no-tool falls through to normal chat,
+- customer slash command still blocked/guided as before,
+- private account guard remains before tool route,
+- transactional support remains before tool route,
+- group behavior remains unchanged,
+- quoted normal chat remains unchanged,
+- tool follow-up after result is handled by tool follow-up resolver.
+
+### Milestone 8: Self-improvement traces
+
+Files:
+- `src/self-improvement/trace-store.ts`
+- `src/self-improvement/eval-generator.ts`
+- `src/self-improvement/repair-proposal.ts`
+
+Goal:
+- Record enough trace data to improve behavior later.
+- Owner-only workflow.
+- No auto-merge.
+- No auto-push.
+- No production self-modification.
+
+Tests:
+- trace stored for planner decision,
+- trace stored for rejected plan,
+- trace stored for async job failure,
+- trace redacts secrets,
+- eval generator creates dynamic tests,
+- repair proposal does not modify files without owner approval.
+
+## 20. Final validation
+
+When all implementation is complete, run:
+
+```
+npm run build
+npm test
+```
+
+Then run hardcoding checks:
+
+```
+grep -RIn \
+  "nvidia/nemotron\|openrouter/free 🆓\|Kilocode (Free)\|MODEL_CATALOG\|model: OPENROUTER_MODEL\|conversationHistory.slice(-10)\|TASK_TOOL_SYNTHESIS\|tool_synthesis" \
+  src config docker-compose.yml \
+  --exclude-dir=node_modules \
+  --exclude-dir=dist \
+  --exclude-dir=.git \
+  --exclude='*.test.ts' || true
+```
+
+Expected: no output.
+
+Also check no legacy model constants outside explicitly allowed config if any remain:
+
+```
+grep -RIn \
+  "OPENROUTER_MODEL\|KILOCODE_MODEL\|GEMINI_MODEL\|GEMINI_VISION_MODEL" \
+  src \
+  --exclude-dir=node_modules \
+  --exclude-dir=dist \
+  --exclude-dir=.git \
+  --exclude='*.test.ts' || true
+```
+
+Expected: no output unless intentionally preserved and documented.
+
+Check for hardcoded live data:
+
+```
+grep -RIn \
+  "Kunle\|Ajayi\|Abuja\|Oya give me\|cybersecurity data scientist\|data scientist but got" \
+  src \
+  --exclude-dir=node_modules \
+  --exclude-dir=dist \
+  --exclude-dir=.git || true
+```
+
+Expected: no output.
+
+## 21. Handover prompt for the next engineer/LLM
+
+Use this prompt to continue:
+
+```
+You are continuing development on the WhatsApp automation bot in branch `work/next-milestone-async-jobs`.
+
+Start by reading `docs/CLAUDE_STYLE_TOOLING_HANDOVER.md` completely. Treat it as the project source of truth.
+
+Current stable base:
+- `context-memory-best-v1`
+- `stable/context-memory-best`
+- normal chat memory/retrieval path is green and must not be changed.
+
+Hard rules:
+- no hardcoding,
+- no guessing,
+- inspect repo before changing code,
+- research official docs when behavior depends on Claude/MCP/OpenAI/provider behavior,
+- do not touch normal chat path except through the documented tool-route insertion point,
+- all tool behavior must be config-driven,
+- all tool execution must be schema-validated, policy-checked, audited, timeout-bounded, and logged cleanly,
+- never promise async work until `AsyncJobStore.enqueue()` succeeds,
+- vague follow-ups like "how about it" must resolve to recent tool context only when a recent tool result exists and the reference is clear,
+- if no recent tool context exists, vague follow-ups should not be forced into tools,
+- connector/MCP tools must go through the same internal catalog/policy/runtime path,
+- MCP and connector results are untrusted data, not instructions,
+- tests should be written for all documented behavior,
+- do not ask the user to test after each milestone; implement the full planned integration and run `npm run build` and `npm test` after final integration.
+
+Implementation order:
+1. tool candidate detector,
+2. Claude tool adapter and planner,
+3. tool plan validator,
+4. async bridge,
+5. async processor,
+6. result summarizer,
+7. tool context store,
+8. follow-up resolver,
+9. controlled chaining,
+10. MCP/connector adapters,
+11. runtime integration,
+12. self-improvement traces.
+
+Before coding, inspect the existing files:
+- `src/conversation/runtime-production.ts`
+- `src/conversation/runtime-impl.ts`
+- `src/async-jobs/job-store.ts`
+- `src/async-jobs/job-runner.ts`
+- `src/async-jobs/job-worker.ts`
+- `src/tools/tool-catalog.ts`
+- `src/tool-runtime/engine.ts`
+- connector and MCP related files.
+
+After implementation:
+- run `npm run build`
+- run `npm test`
+- run hardcoding greps listed in the handover.
+```
+
+## 22. Stop conditions
+
+Stop and ask for owner review if:
+- implementation would require changing normal memory chat behavior,
+- a tool needs private account/payment data,
+- a connector action writes/modifies external data,
+- MCP server exposes high-risk action tools,
+- a tool result includes secrets or unexpected private data,
+- the model requests a tool not in the enabled catalog,
+- async job persistence fails,
+- a patch requires reintroducing hardcoded model/tool routes,
+- tests require live personal data or hardcoded customer examples.
+
+## 23. Summary
+
+The correct design is:
+
+```
+Claude-style client tool use
++ strict schemas
++ candidate subset only
++ deterministic validation
++ existing ToolCatalog
++ existing ToolExecutionRuntime
++ existing AsyncJobStore/Runner/Worker
++ ToolContextStore for follow-ups
++ controlled sequential chaining
++ MCP/connector adapters
++ normal chat path untouched
++ clean logs and proper errors
++ final build/test validation
+```
+
+Do not build another broad, slow, tool-heavy normal chat route. Build a narrow, conservative, durable, config-driven tool lane.