aki-pro-max 2.3.3

package/docs/CORRELATION-DESIGN.md

@@ -0,0 +1,226 @@
+# Native tool-result correlation design (offline, source-derived)
+
+Status: design only; no implementation or live validation.
+
+## 1. Evidence and boundary
+
+### The current HTTP 400 is emitted by this facade
+
+The pinned 9router executor forwards the OpenAI request to `<facade>/v1/chat/completions` with its configured bearer key. `test/9router-executor.integration.test.mjs` then observes the second request returning HTTP 400. The rejecting branch is local: `src/postman-request.mjs:5-12` checks `request.toolResults?.length` and throws `invalid(..., 'unsupported_tool_result_continuation')` before `fetchImpl` is called. The integration assertion `captures.length === 1` proves the rejected second request did not reach the Postman-facing fetch boundary. This design therefore changes the facade; it does not attribute the 400 to 9router and does not require a 9router session header.
+
+### Immutable ZIP native request shape
+
+Reviewed archive: `C:\Users\ADMIN\aicoworker\openclaw\media\outbound\6f139e01-fd99-4f9b-a8a4-2e2777fad843.zip`, SHA-256 `807E29A51F6EE068D9BEFE0112855DF2B244486502DF721D5AF3BC5CD071BDF6`.
+
+`proxypostman-main/src/postman.mjs:113-124` establishes:
+
+- Initial request: `input={chatType:'USER_QUERY', query, conversationId, product:'workspace_v12', toolCallGroupId:null, toolResponses:[]}`.
+- Native continuation starts from saved `conversationId`, then uses `chatType:'TOOL_RESPONSE'`, `query:''`, `useCase:null`.
+- Grouped call: `toolCallGroupId=<upstream group id>` and `toolResponses=[{toolCallId,content,toolResponseSummary,toolResponseStatus}, ...]` (the local `groupId` property is removed).
+- Singular ungrouped call: `toolCallId`, `toolResponse`, `toolResponseSummary`.
+- Multiple calls without one common non-empty group ID are rejected by that source transport rather than submitted as a native batch.
+
+`proxypostman-main/src/bridge.mjs:163-182,329-372` establishes that continuation depends on saved history, `conversationId`, pending calls, and `correlateToolResults`; non-correlatable/transcript cases reset native conversation state. `src/tool-results.mjs` requires exactly one client result for every pending call, no duplicate result IDs, and no unexpected IDs.
+
+### Existing facade limits retained
+
+- Request body: default 1 MiB, configurable 1 KiB–8 MiB (`MAX_BODY_BYTES`).
+- Tools: default 64, configurable 0–256 (`MAX_TOOLS`).
+- One accumulated upstream tool argument value: default 256 KiB, configurable 256 B–4 MiB (`MAX_TOOL_ARGUMENT_BYTES`).
+- Upstream output: default 4 MiB, configurable 1 KiB–16 MiB.
+- Tool names: `[A-Za-z0-9_-]{1,64}`; registered names are namespaced/hashed to at most 64 characters.
+- Supported schema vocabulary is deliberately bounded to `type`, `properties`, `required`, boolean `additionalProperties`, `items`, `enum`, `const`, `anyOf`, `oneOf`, `allOf`, descriptive fields, numeric/string/array bounds, `$schema`, and `$id`. Formats, regex patterns, refs, tuple items, numeric `additionalProperties`, and other keywords remain rejected.
+- Tool result content remains text-only at the OpenAI facade boundary and is still constrained by the overall request body limit. A separate `MAX_TOOL_RESULT_BYTES` should bound each result and the aggregate native continuation body below `MAX_BODY_BYTES`.
+
+## 2. Scope and lifecycle
+
+This is not general conversation persistence. Every request without a valid pending facade-issued tool call is stateless and becomes a fresh upstream `USER_QUERY`. State begins only after an upstream response has supplied a conversation ID and one or more complete, schema-valid tool calls. State ends after their native result continuation is definitively consumed, explicitly expires before dispatch, or reaches a terminal uncertain/cancelled record retained only for replay rejection/audit TTL.
+
+A normal no-tool response stores nothing. A caller cannot opt into state using a header or caller-created ID. No 9router session behavior is assumed.
+
+## 3. Authenticated principal and isolation
+
+Define `principalId = HMAC(serverPrincipalKey, credentialSlotId)` where `credentialSlotId` is the server-side identity of the matched configured ingress API key, not the plaintext key and not `body.user`.
+
+The present single `PROVIDER_API_KEY` configuration creates exactly one principal. It isolates holders of that key from holders of other configured keys only after multi-key configuration exists; it does **not** distinguish fictional end users who share the same key. `body.user`, forwarded IP, and arbitrary headers are not authentication boundaries.
+
+Deployment requirement for mutually untrusted clients: configure a distinct ingress key/credential slot per client. A later implementation may support an explicit key map, with each slot optionally restricted to a fixed workspace/model allowlist. Until then, shared-key clients must be treated as one trust domain.
+
+Lookup is always `(principalId, publicToolCallId)`. An ID that exists for a different principal returns the same generic `unknown_tool_call` response as an absent/forged ID; it must not reveal cross-principal existence.
+
+## 4. Public IDs and internal records
+
+### Public tool call IDs
+
+Never expose the upstream `toolCallId` or `toolCallGroupId` as the OpenAI `assistant.tool_calls[].id`.
+
+Generate each public ID with at least 128 bits from a CSPRNG, for example `call_pmn_` plus base64url of 24 random bytes (192 bits). IDs are facade-issued, unstructured, non-sequential, and carry no tenant, workspace, group, or upstream identifier. Enforce uniqueness in the active + terminal-retention indexes; regenerate on collision.
+
+The response adapter replaces each upstream call ID with its public ID. The private mapping remains server-side.
+
+### Group record
+
+One record per upstream group (or one synthetic group per singular ungrouped call):
+
+```text
+GroupRecord {
+  principalId
+  workspaceId
+  model
+  conversationId
+  upstreamGroupId: string|null
+  groupKey: random internal identifier
+  state: PENDING|INFLIGHT|CONSUMED|EXPIRED|CANCELLED|UNCERTAIN
+  issuedAt, pendingExpiresAt, terminalExpiresAt
+  transcriptHash
+  transcriptPrefixLength
+  toolsHash
+  calls: ordered CallRecord[]
+  acceptedResultsHash: string|null
+  dispatchAttemptId: random|null
+  terminalResponseDigest: string|null
+}
+CallRecord {
+  publicToolCallId
+  upstreamToolCallId
+  upstreamGroupId: string|null
+  index
+  originalName
+  argumentsHash
+  argumentsJson
+  schemaHash
+}
+```
+
+`workspaceId` comes only from server configuration selected for the authenticated credential. `conversationId`, upstream call/group IDs, tool name, arguments, and order come only from validated upstream events. `model` is the exact model used for the initial request.
+
+### Canonical hashes
+
+Use a versioned deterministic serializer (sorted object keys, preserved array order, UTF-8, explicit nulls; no Unicode normalization) and SHA-256 or stronger.
+
+- `transcriptHash`: hash of the exact normalized OpenAI message prefix returned/accepted for the tool-call turn: prior normalized messages plus the facade-produced assistant message containing public IDs, names, and exact argument strings. Exclude the new `role=tool` messages. Store `transcriptPrefixLength` and require exact equality of that prefix on continuation.
+- `toolsHash`: hash of normalized caller tool definitions, original names, schemas, descriptions, and effective `tool_choice` in order.
+- Each `argumentsHash`: exact upstream argument string plus original name and schema hash.
+- `acceptedResultsHash`: ordered canonical tuple of every `(publicToolCallId, content)` accepted for the group.
+
+The hash is an integrity attribute, not a lookup key. The primary lookup remains `(principalId, publicToolCallId)`; after lookup, every binding is checked.
+
+## 5. Continuation validation
+
+A request containing any `role=tool` message takes the continuation path and must never fall back to `USER_QUERY`.
+
+1. Authenticate and derive `principalId`.
+2. Normalize the OpenAI request under existing body/tool/schema limits.
+3. Resolve every distinct `tool_call_id` by `(principalId, publicToolCallId)`.
+4. Require all IDs to resolve to one `GroupRecord`; reject mixed groups.
+5. Require exactly one result for every call in the record, no duplicates and no extras. Grouped native continuation is all-results-only. A singular ungrouped record requires exactly one result.
+6. Require the messages before the result block to equal the stored normalized transcript prefix byte-for-byte after the versioned canonical serialization. Any removed, inserted, reordered, or changed earlier message, tool call, name, arguments, public ID, or assistant content fails `continuation_history_mismatch`.
+7. Require results to form the final contiguous message suffix. No new user/assistant/system/developer message may accompany a pending native result submission. This deliberately avoids ambiguity about whether extra context belongs in `TOOL_RESPONSE`.
+8. Require `request.model === record.model`; otherwise return `continuation_model_mismatch`. Do not route silently to the old or new model.
+9. Require server-selected `workspaceId === record.workspaceId` and current tool definitions/choice hash to equal `toolsHash`; otherwise reject explicitly. Omitting `tools` on continuation is permitted only if normalization substitutes the stored definitions before hashing; the simpler first implementation should require exact resubmission.
+10. Enforce state and expiration transitions atomically, then build native `TOOL_RESPONSE` exclusively from private upstream IDs.
+
+Unknown, hallucinated, expired, replayed, cross-principal, cross-workspace, mixed-group, partial-group, duplicate, model-changed, tool-changed, and history-mismatched submissions are explicit failures. None becomes a new query.
+
+## 6. Native output construction
+
+For a common non-empty upstream group ID:
+
+```json
+{
+  "chatType": "TOOL_RESPONSE",
+  "query": "",
+  "conversationId": "<stored upstream conversationId>",
+  "product": "workspace_v12",
+  "useCase": null,
+  "toolCallGroupId": "<stored upstream group id>",
+  "toolResponses": [
+    {
+      "toolCallId": "<stored upstream call id>",
+      "content": "<OpenAI tool message text>",
+      "toolResponseSummary": "Result for <original tool name>",
+      "toolResponseStatus": "SUCCESS"
+    }
+  ]
+}
+```
+
+For one ungrouped upstream call, omit `toolCallGroupId`/`toolResponses` and set stored `toolCallId`, `toolResponse`, and `toolResponseSummary`. Do not infer or invent group IDs. Result-status mapping is not expressed by standard OpenAI `role=tool`; the bounded first implementation therefore uses `SUCCESS` for accepted text results. Adding an error-status extension requires a separately documented request contract and tests.
+
+## 7. Atomic state machine
+
+All transitions below occur under a per-group mutex/CAS. Multi-ID lookup first resolves records, then locks the one group and revalidates state and expiration.
+
+| Current state | Event | Next state | Required result |
+|---|---|---|---|
+| PENDING | Valid complete result set before expiry | INFLIGHT | Store `acceptedResultsHash` and random `dispatchAttemptId` atomically, then perform exactly one upstream dispatch. |
+| PENDING | Partial/duplicate/extra/mixed result set | PENDING | HTTP 400, no mutation. |
+| PENDING | History/model/tools/workspace mismatch | PENDING | HTTP 409 `continuation_*_mismatch`, no mutation. |
+| PENDING | Unknown/cross-principal public ID | PENDING/none | HTTP 404 `unknown_tool_call`, no information leak. |
+| PENDING | Deadline observed before dispatch claim | EXPIRED | HTTP 410 `tool_call_expired`; retain tombstone. |
+| PENDING | Capacity pressure | PENDING | Never evict; reject admission of newer tool groups instead. |
+| PENDING | Explicit server shutdown/cancel before dispatch | CANCELLED | HTTP 409/503 on later submission; retain tombstone. |
+| INFLIGHT | Same result set retry | INFLIGHT | HTTP 409 `continuation_inflight` plus stable request ID; never dispatch again. |
+| INFLIGHT | Different result set retry | INFLIGHT | HTTP 409 `continuation_conflict`; never dispatch. |
+| INFLIGHT | Upstream `[DONE]` and response committed | CONSUMED | Cache terminal response digest/body within strict size bound for idempotent replay policy. |
+| INFLIGHT | Definite pre-send failure (no bytes/request accepted) | PENDING | Clear dispatch claim only when transport proves no upstream request could have been accepted. |
+| INFLIGHT | Timeout, disconnect, abort, or ambiguous post-send failure | UNCERTAIN | Never automatically retry TOOL_RESPONSE; return `continuation_outcome_uncertain`. |
+| CONSUMED | Same result set retry | CONSUMED | Return cached response if safely retained, else HTTP 409 `tool_call_already_consumed`; never dispatch. |
+| CONSUMED | Different result set retry | CONSUMED | HTTP 409 `continuation_conflict`. |
+| EXPIRED | Any submission | EXPIRED | HTTP 410 `tool_call_expired`. |
+| CANCELLED | Any submission | CANCELLED | HTTP 409 `tool_call_cancelled`. |
+| UNCERTAIN | Any submission | UNCERTAIN | HTTP 409/502 `continuation_outcome_uncertain`; operator reconciliation required, no duplicate side effect. |
+| terminal | Tombstone TTL elapsed | removed | Later lookup returns generic 404 `unknown_tool_call`; no live work is affected. |
+
+`CONSUMED` means the native result submission is consumed, not that subsequent tool-call groups cannot be issued. If the upstream continuation response emits another complete tool group and conversation ID, create a new independent PENDING group with new public IDs and bindings after consuming the prior group.
+
+## 8. TTL and bounded capacity
+
+Suggested configurable starting values (implementation approval required): pending TTL 15 minutes, inflight hard deadline equal to request timeout plus 30 seconds, terminal/tombstone TTL 15 minutes, maximum 1,000 live groups, 4,096 live calls, and a bounded cached terminal response budget.
+
+Rules:
+
+- Expiration is checked on lookup, admission, and periodic sweep.
+- A sweep may transition an overdue PENDING record to EXPIRED and later remove terminal tombstones. It may not silently delete a non-expired PENDING record or any INFLIGHT record.
+- INFLIGHT timeout becomes UNCERTAIN, not EXPIRED/PENDING.
+- When live-group/call/memory capacity is full, reject the **new initial request's tool-call response before exposing unusable public IDs** if possible. For streaming, because headers/chunks may already be emitted, reserve capacity when the first upstream conversation/tool call appears; on reservation failure terminate with explicit `correlation_capacity_exceeded` and do not pretend the calls are continuable.
+- Do not LRU-evict pending groups. Capacity recovery comes from explicit expiry/terminal cleanup or operator action.
+- Record and result byte accounting must be calculated before admission/transition, using configured hard maxima.
+
+## 9. Cancellation and side-effect safety
+
+Client disconnect during the initial tool-call response does not erase an admitted PENDING group; the client may have received IDs before disconnect. It expires normally.
+
+Client disconnect/cancel before the continuation wins the atomic PENDING→INFLIGHT claim leaves PENDING or transitions to CANCELLED only through an explicit server decision. After the claim and possible upstream send, cancellation is not proof the upstream ignored the TOOL_RESPONSE. Any ambiguous interruption becomes UNCERTAIN. The facade must not reset to PENDING and must not automatically submit the same native result again, because continuation can cause the model to request new side-effecting tools and duplicate dispatch would produce divergent actions.
+
+Within one process, the group CAS plus `dispatchAttemptId` guarantees at most one dispatch. This design is explicitly single-process/in-memory; multiple workers or restart durability require a shared transactional store and are not claimed.
+
+## 10. Error contract
+
+Suggested stable codes:
+
+- 404 `unknown_tool_call` — absent, forged, or cross-principal ID.
+- 400 `tool_result_group_mismatch` — mixed groups, duplicate/extra IDs, malformed suffix.
+- 400 `tool_result_incomplete` — not every group call has exactly one result.
+- 409 `continuation_history_mismatch`.
+- 409 `continuation_model_mismatch`.
+- 409 `continuation_tools_mismatch`.
+- 409 `continuation_workspace_mismatch` (generic message; normally impossible under fixed per-key config).
+- 409 `continuation_inflight`.
+- 409 `tool_call_already_consumed` / `continuation_conflict`.
+- 410 `tool_call_expired`.
+- 409 `tool_call_cancelled`.
+- 502 `continuation_outcome_uncertain`.
+- 429/503 `correlation_capacity_exceeded` for new group admission.
+- Retain current 400 `unsupported_tool_result_continuation` only when correlation support is disabled by configuration; never use it for a resolvable but invalid state.
+
+Error bodies and logs must not contain upstream conversation, group, or call IDs, bearer tokens, result contents, or transcript text. Structured logs may include request ID, principal credential-slot identifier/hash, public group fingerprint, state transition, byte counts, and error code.
+
+## 11. Implementation seams (future, not performed)
+
+- Authentication returns a principal/credential configuration rather than only boolean success.
+- Response adaptation allocates group capacity and replaces upstream call IDs before OpenAI output.
+- A dedicated `CorrelationStore` owns records, indexes, locks/CAS, timers, byte accounting, and tombstones.
+- Request normalization separates the stored prefix from the contiguous final tool-result suffix and validates exact group completeness.
+- `buildPostmanRequest` receives a validated continuation object and constructs either source-derived `USER_QUERY` or `TOOL_RESPONSE`; it never decides correlation from untrusted message IDs alone.
+- Tests use source-derived synthetic frames and direct assertions, not claims that fixtures are captured live payloads.

package/docs/FAIL-CLOSED-ISSUE-HARNESS.md

@@ -0,0 +1,21 @@
+# Fail-Closed Issue-Closure Harness
+
+`node scripts/verify-issue-closure.mjs` is the deterministic closure gate for the Claude tool-loop and PMN attribution evidence. It never retries, chooses a fallback, repairs evidence, or treats missing fields as success.
+
+## Enumerated rejection branches
+
+| Boundary | Rejected branch |
+|---|---|
+| Freshness | Prior current-probe attempts are non-zero |
+| Upstream policy | Retry or failover is enabled |
+| Claude selection | Public/selected model differs from the accepted pair |
+| Claude initial phase | Non-200, zero/multiple tool calls, invalid call |
+| Claude arguments | Not an object, non-empty object, parse error, or invalid schema |
+| Claude continuation | Non-200, missing exact marker, non-terminal match, branch error |
+| Cleanup | Probe server cleanup is not proven |
+| Provider mapping | Dedicated PMN connection missing, renamed, or inactive |
+| Negative control | CPX was inactive or no surrounding CPX control row exists |
+| Attribution | Zero/multiple PMN rows, wrong connection/model/status |
+| Marker correlation | Completion-token count differs from the exact four-token marker |
+
+Tests seed each major bad branch and assert rejection in `test/issue-closure-harness.test.mjs`. This prevents a future documentation-only or happy-path-only claim from silently reopening issues #1/#2.