@cartanova/qgrid-cli 2.5.3 → 2.5.5

package/skills/qgrid/references/decision-rationale.md

@@ -49,7 +49,16 @@ Sources:
 Key decisions:
 
 - qgrid tool calling is intentionally implemented through structured-output emulation, not native provider tool use. The provider call returns a structured action with tool calls; the AI SDK loop executes tools and sends follow-up turns.
-- Tools and `jsonSchema` are mutually exclusive at the qgrid dispatcher boundary because both need the provider structured-output slot.
+- Tools and `jsonSchema` share one provider structured-output slot by composing
+  an action envelope on every turn. Its final `answer` branch embeds the user's
+  schema; its `tool_call` branch carries emulated AI SDK calls. This keeps final
+  output constrained even when tools are available but never called.
+- Final output is identified by `action: "answer"`, not by tool presence or turn
+  count. Structured answers are serialized as JSON text for AI SDK
+  `Output.object` parsing, while tools-only callers retain their legacy string
+  answer and tolerant parse fallback.
+- AI SDK `toolChoice` remains outside qgrid's wire contract. Do not imply that
+  qgrid transports or enforces it.
 - Tool-call request logs are a multi-step run: create run, append generate/tool steps, then finish run. This makes AI SDK multi-step behavior visible in the dashboard instead of logging only one opaque completion.
 - Tool results update the existing `tool_call` step by `request_log_id + tool_call_id`. They are not logged as a second unrelated completion row.
 - The server infers single-turn completion versus an open tool run from context and finish reason. The removed caller-selected `logMode` contract must not be recreated.

package/skills/qgrid/references/docs-workflow.md

@@ -40,3 +40,12 @@ When a repo doc establishes or changes durable rationale for a qgrid feature, re
 - `docs/qgrid-doc-map.json` maps each durable topic to code, canonical skill references, and the matching Notion page.
 - Code and tests remain the source of truth. Update the skill and Notion together when a code change alters a documented contract.
 - Keep dated operational snapshots, internal server addresses, and other environment-specific facts in Notion. Keep reusable runtime contracts in the skill.
+
+## Retract pre-release caveats after shipping
+
+Documenting a contract before it ships is fine, but the caveat must be retracted once it does. Wording like "not deployed yet", "based on the working tree", "migration is still untracked", or "package version is still X" becomes a false statement the moment the release lands, and `pnpm qgrid-docs:check` will not catch it — that script validates the skill mirror and `docs/qgrid-doc-map.json` paths only, never Notion body text.
+
+- When you add a pre-release caveat, record the retraction in the release page at the same time. Do not rely on memory.
+- After a release, search the wiki for `미배포`, `배포되지 않았`, `working tree`, `untracked`, `아직`, and the previous version number.
+- Update each page's baseline commit to the release commit. A stale baseline undermines every claim below it.
+- This happened once already: caveats added 2026-07-23 survived the 2.4.9 and 2.5.0 releases and left six pages asserting the opposite of reality for several days.

package/skills/qgrid/references/repo-map.md

@@ -28,7 +28,8 @@ Use this map to orient qgrid work quickly.
 - Cost/usage accounting: `packages/api/src/utils/providers/common/model-cost.ts`.
 - Weighted token selection: `packages/api/src/utils/providers/common/smooth-weighted-round-robin.ts`.
 - Boot order and startup migrations: `packages/api/src/server-bootstrap.ts`, `packages/api/src/startup-migrations.ts`.
-- Tool-call emulation: `packages/api/src/application/qgrid/tool-emulation.ts`.
+- Tool-call schema composition: `packages/api/src/application/qgrid/tool-emulation-schema.ts`.
+- Tool-call response decoding: `packages/api/src/application/qgrid/tool-emulation.ts`.
 
 ## Web/Sonamu hot spots
 

package/skills/qgrid/references/tool-calling-and-multiturn.md

@@ -24,6 +24,11 @@ qgrid supports the AI SDK `tools` interface, but it does not use native provider
 - qgrid maps that structured object back into AI SDK `tool-call` content.
 - The AI SDK executes the actual tool functions and calls qgrid again with tool results.
 
+`generateText` and `streamText` default to `stepCountIs(1)`. When an executable
+tool can run before the final answer, callers must set a bounded `stopWhen`
+(for example `stepCountIs(3)`) or the AI SDK stops after the first tool-call
+step and cannot produce the final text or `Output.object` value.
+
 Do not describe this as OpenAI native function calling, Claude native tool use, or Codex/Claude executing client tools. The tools are client-side AI SDK tools requested through structured-output emulation.
 
 ## Client Flow
@@ -36,7 +41,9 @@ When `options.tools` contains function tools:
 2. Convert them with `toQgridTool`.
 3. Send `tools` in `/api/qgrid/query` or `/api/qgrid/prepareStream`.
 4. Forward the per-call `logger` switch when request logging is disabled.
-5. Do not send `jsonSchema` response format at the same time. The server infers whether the logged request becomes a multi-step run.
+5. When a top-level object response format is present, send `jsonSchema` alongside
+   `tools`. The server composes both schemas and infers whether the logged request
+   becomes a multi-step run.
 
 When qgrid responds with `finishReason: "tool-calls"`:
 
@@ -57,9 +64,10 @@ The registry is keyed per pending run so concurrent `generateText(...tools...)`
 
 ## Server Schema Emulation
 
-`packages/api/src/application/qgrid/tool-emulation.ts` builds the tool-call schema.
+`packages/api/src/application/qgrid/tool-emulation-schema.ts` builds the tool-call schema.
+`tool-emulation.ts` decodes the model's action envelope into qgrid answer/tool-call content.
 
-The schema shape is:
+The tools-only schema shape is:
 
 ```json
 {
@@ -83,7 +91,125 @@ The schema explicitly tells the model:
 
 `qgrid.dispatcher.ts` strictifies this schema through `buildStrictOutputSchema` before it reaches provider dispatchers.
 
-If tools and `jsonSchema` are both present, dispatcher rejects the request.
+## Composing Tools With A User Output Schema
+
+This section explains why `tool-emulation-schema.ts` exists at all. Read it
+before changing schema composition; the mechanics look arbitrary without the
+failure it was built to prevent.
+
+### The problem
+
+AI SDK hands the provider `tools` and `responseFormat` independently on every
+step, and gives no "this is the final turn" flag. Before 2.5.4, qgrid saw tools
+and dropped the user schema, so `answer` stayed a free-form string:
+
+```json
+{ "action": "answer", "answer": "string | null", "toolCalls": "…" }
+```
+
+`strict: true` was in force, but only over that outer envelope. A model could
+therefore return valid JSON with trailing garbage and still satisfy the schema:
+
+```jsonc
+// Envelope-valid. answer is a string, so the trailing marker is legal.
+{ "action": "answer", "answer": "{\"title\":\"...\"}<|proto_end|>", "toolCalls": null }
+```
+
+Measured on the Medpath `translateBatch()` path: 5 of 10 raw Terra responses
+were corrupted this way, while the same workload with a schema and no tools
+failed 0 of 385 times. The model was fine; nothing constrained the final answer.
+
+### The fix, and why it needs rebasing
+
+Embed the user schema in the `answer` branch so the provider's
+constrained-output machinery enforces it — the same machinery that scored
+385/385. The model still chooses `tool_call` to continue the loop or `answer` to
+finish, and qgrid never adds an extra turn to "finalize".
+
+Embedding cannot be a verbatim nest, because `$ref` is document-absolute. `#`
+means the root of the *final* document, not the root of the fragment it was
+written in. Nest a caller schema unchanged and its pointers silently retarget
+the envelope:
+
+```jsonc
+// Caller sends this. "#/$defs/Person" resolves inside their own document.
+{
+  "type": "object",
+  "properties": { "author": { "$ref": "#/$defs/Person" } },
+  "$defs": { "Person": { "type": "object", "properties": { "name": { "type": "string" } } } }
+}
+```
+
+Nested verbatim, `#/$defs/Person` now points at the envelope root, where no
+`Person` exists — a broken reference, or worse, a coincidental match against an
+unrelated definition. Validation would then check the wrong shape and say
+nothing.
+
+So composition does two things: park the caller schema under a reserved name,
+and rewrite every local pointer to the new base.
+
+```jsonc
+{
+  "type": "object",
+  "properties": {
+    "action": { "enum": ["answer", "tool_call"] },
+    // string branch becomes a reference to the caller's schema
+    "answer": { "anyOf": [{ "$ref": "#/$defs/__qgrid_user_output" }, { "type": "null" }] },
+    "toolCalls": { "…": "unchanged" }
+  },
+  "$defs": {
+    "__qgrid_user_output": {
+      "type": "object",
+      "properties": { "author": { "$ref": "#/$defs/__qgrid_user_output/$defs/Person" } },
+      "$defs": { "Person": { "type": "object", "properties": { "name": { "type": "string" } } } }
+    }
+  }
+}
+```
+
+Pointer rewriting is mechanical — `#` gains the namespace prefix:
+
+| Caller writes | Becomes |
+| --- | --- |
+| `#` | `#/$defs/__qgrid_user_output` |
+| `#/$defs/Person` | `#/$defs/__qgrid_user_output/$defs/Person` |
+
+### Why the traversal is exhaustive
+
+A missed position means an un-rebased `$ref` shipping to the provider, so the
+rewriter must visit every place a subschema can hide. JSON Schema puts them in
+three shapes — name→schema maps (`properties`, `$defs`), schema arrays
+(`anyOf`, `prefixItems`), and single slots (`items`, `not`, `if`) — plus Draft-7
+`dependencies`, whose value is either a schema or a list of property names.
+
+That position list is shared: `utils/providers/common/json-schema-keywords.ts`
+owns it, and the rebaser, the normalization scan, and the strict rewrite all
+read from it. They previously kept private copies and had already drifted
+(`additionalItems` was rebased but not scanned). Add a new keyword there, not in
+a consumer.
+
+Consumers still differ in what they *do* per position, and that stays local: the
+strictifier marks positions like `not` and `if` non-normalizable because
+tightening them would change caller semantics.
+
+### What is rejected, and why silence is not an option
+
+Composition fails loudly on `$id`, `id`, `$anchor`, `$dynamicAnchor`,
+`$recursiveAnchor`, `$dynamicRef`, and `$recursiveRef`. Each one breaks the
+premise that a pointer's meaning survives relocation: `$id` declares a new base
+URI so descendant refs stop resolving against the document root, and dynamic
+refs pick their target from runtime scope. There is no correct rewrite, and a
+wrong one would validate the wrong shape without complaint — so callers get an
+HTTP 400 naming the offending path instead.
+
+Non-object top-level schemas are also rejected. Only `type: "object"` roots can
+occupy the `answer` branch.
+
+### Compatibility
+
+`answer` falls back to `{ "type": "string" }` whenever no user schema is present.
+That single branch is why tools-only calls and older SDKs keep working against a
+2.5.4 server unchanged.
 
 ## Provider-Specific Path
 
@@ -111,9 +237,14 @@ In both providers, native provider tools are not the public abstraction. The pub
 `applyToolCallEmulation` handles provider text:
 
 - No tools: return normal text content.
-- Tools present and JSON parse fails: warn and fall back to text with `finishReason: "stop"`.
+- Tools-only mode and JSON parse failure: preserve the legacy warning and text
+  fallback with `finishReason: "stop"`.
+- Tools plus a user schema and malformed or incoherent envelope: return an
+  explicit structured-output error; do not rescue it as successful text.
 - `action: "tool_call"`: validate tool names, generate qgrid `toolCallId`, return `finishReason: "tool-calls"`.
-- `action: "answer"`: return final text with `finishReason: "stop"`.
+- `action: "answer"` in tools-only mode: return the string answer as final text.
+- `action: "answer"` with a user schema: JSON-serialize the structured `answer`
+  as final text so AI SDK can parse and validate `Output.object`.
 
 Unknown tool names throw.
 
@@ -128,6 +259,14 @@ AI SDK multi-step behavior is client-orchestrated:
 
 This is different from a provider-native tool loop. Neither Codex nor Claude Code executes the AI SDK tool functions.
 
+The composed schema is reapplied to tool-result follow-up turns. Final output is
+identified by `action: "answer"` rather than by whether a tool happened to be
+available or called. AI SDK `toolChoice` is not transported or enforced by
+qgrid; the model selects between the envelope's answer and tool-call actions.
+
+The combined contract requires qgrid server 2.5.4 and
+`@cartanova/qgrid-ai-sdk` 2.5.4.
+
 OpenAI/Codex follow-ups can reuse the same Codex thread when `runContext.threadCoord` remains valid. For tool-result follow-up turns, qgrid sends delta input containing tool result text to the existing thread; cold fallback injects full history.
 
 Anthropic follow-ups do not reuse Claude sessions. qgrid starts a fresh Claude process, replays flattened full history, and sends the tool-result continuation input.

package/skills/qgrid/references/verification-and-debugging.md

@@ -30,6 +30,7 @@ AI SDK package `@cartanova/qgrid-ai-sdk`:
 - `pnpm --filter @cartanova/qgrid-ai-sdk test`.
 - `pnpm --filter @cartanova/qgrid-ai-sdk build`.
 - `pnpm --filter @cartanova/qgrid-ai-sdk e2e`.
+- `pnpm --filter @cartanova/qgrid-ai-sdk e2e:tools-output`.
 - `pnpm --filter @cartanova/qgrid-ai-sdk e2e:logger`.
 
 CLI package `@cartanova/qgrid-cli`:
@@ -54,7 +55,7 @@ Choose by affected area:
 | AI SDK logger integration | `packages/ai-sdk/src/logger.test.ts` |
 | qgrid provider routing and strict schema | `packages/api/src/application/qgrid/qgrid.dispatcher.test.ts` |
 | conversation/thread reuse routing | `packages/api/src/application/qgrid/conv-routing.test.ts` |
-| tool-call emulation | `packages/api/src/application/qgrid/tool-emulation.test.ts` |
+| tool-call emulation | `packages/api/src/application/qgrid/tool-emulation.test.ts`, `tool-emulation-schema.test.ts` |
 | request-log run lifecycle | `packages/api/src/application/qgrid/qgrid-run-lifecycle.test.ts` |
 | qgrid frame API behavior | `packages/api/src/application/qgrid/qgrid.frame.test.ts` |
 | token subscriber and LISTEN/NOTIFY handling | `packages/api/src/application/qgrid/token-subscriber.test.ts` |
@@ -101,6 +102,20 @@ AI SDK structured output:
 - Env: `QGRID_URL`, `SMOKE_MODEL`, `SMOKE_CASES`, `SMOKE_REPEAT`.
 - Uses AI SDK 6 `streamText` with `Output.object`; useful for structured-output regressions.
 
+AI SDK tools plus structured output:
+
+- `packages/ai-sdk/e2e/tools-structured-output.ts`
+- This is the public-path release acceptance matrix: `qgrid()` HTTP/SSE,
+  `generateText`/`streamText`, OpenAI/Anthropic, direct final answers, and one
+  client-tool result followed by `Output.object`.
+- It consumes real provider quota and requires the explicit
+  `QGRID_REAL_PROVIDER_ACCEPTANCE=1` opt-in.
+- Uses `QGRID_URL`, `QGRID_ACCEPTANCE_OPENAI_MODEL`, and
+  `QGRID_ACCEPTANCE_ANTHROPIC_MODEL`. OpenAI direct-final coverage defaults to
+  10 generate plus 10 stream attempts and can be changed with
+  `QGRID_ACCEPTANCE_OPENAI_DIRECT_REPEATS_PER_MODE`. Request logs use the stable
+  project name `qgrid-ai-sdk-tools-output-acceptance`.
+
 Image generation:
 
 - `packages/api/scripts/smoke-test-image-generation.ts`