@lenne.tech/nest-server 11.40.0 → 11.41.1

package/migration-guides/11.40.0-to-11.41.0.md

@@ -0,0 +1,118 @@
+# Migration Guide: 11.40.0 → 11.41.0
+
+> **Why a MINOR.** The MAJOR digit in this package tracks the NestJS major (11.x = NestJS 11), so it
+> is not ours to spend — every behaviour change of our own ships as a MINOR. This release changes
+> when the AI module asks an endpoint for JSON, which a project can notice.
+
+## Overview
+
+| Category | Effort | Applies to |
+|----------|--------|-----------|
+| **Behaviour change** | none | Projects using the AI module — three internal calls stop requesting JSON |
+| Bugfix | none | Everyone using the AI module — capability detection and context windows get more accurate |
+| New feature (opt-in) | none | `ai.maxRunMs` |
+| Internal | none | Nobody — a duplicate index declaration was removed |
+
+Most projects update with `pnpm update @lenne.tech/nest-server` and read no further. **Projects that
+do not use the AI module are unaffected by all of it.**
+
+## Quick Migration
+
+```bash
+pnpm update @lenne.tech/nest-server
+```
+
+```bash
+# Do you use the AI module at all? If this is empty, you are done.
+grep -rn "CoreAiModule\|aiConnections\|AiTool" src/ 2>/dev/null
+```
+
+## Behaviour change: JSON mode is decided per PROMPT, not per connection
+
+### What changed
+
+A connection with `supportsJsonResponse: true` used to send `response_format: json_object` on
+**every** completion. It is now sent only when the call actually wants structured output.
+
+That was wrong in both directions, and the second one is why it is worth fixing rather than leaving:
+
+- The flag describes what the ENDPOINT can do, not what a given prompt NEEDS. A capability is not
+  an instruction.
+- It leaked into calls that must not be JSON. The final answer to the user, and the compaction step
+  that summarises a long conversation, both want prose — and both were being asked for JSON. The
+  compaction case is the visible one: it spliced JSON-wrapped summaries into the history that the
+  model then had to read back.
+
+### Do I need to do anything?
+
+**No.** There is deliberately no new option to pass: `jsonResponse` lives on `LlmCompletionOptions`,
+which is the PROVIDER-level contract, and the module sets it for itself. It is not a field on the
+prompt input, and nothing in a consumer project needs to change.
+
+Concretely, the narrowing is three call sites, all of which wanted prose and were being asked for
+JSON:
+
+| Call | Now |
+|------|-----|
+| The agent loop, on a connection with NATIVE tool calling | JSON mode off — the tools carry the structure |
+| The final answer to the user | never JSON |
+| Compaction (summarising a long conversation) | never JSON |
+
+A connection WITHOUT native tool calling is untouched: emulated tool calling is built on
+prompt-driven JSON, so those calls still request it.
+
+The one thing to know is the seam the fix deliberately leaves imperfect, because a project can reach
+it: the gate keys on native tool support, not on what the prompt asks for. A project that overrides
+the `native` prompt slot via `CoreAiSlotService` to request JSON anyway will have it narrowed off.
+That degrades gracefully rather than failing — prompt-driven JSON is the fallback the module is
+built around, and its extractor is lenient — but if you have such an override and see prose where
+you expected an object, this is why.
+
+A custom `ILlmProvider` needs no change either: `jsonResponse` is OPTIONAL, so an implementation
+that ignores it keeps compiling and keeps its previous behaviour.
+
+## Bugfixes (no action required)
+
+| Fix | What it was |
+|-----|-------------|
+| **Context window for two model families** | The known-model table matched by substring, so `ministral` did not match `mistral` and `mistral-medium` had no entry at all. Both fell back to the 8192 default instead of their real 131072 — silently truncating history on models that had 16x the room. Both are now listed |
+| **Capability probes recorded permanent false negatives** | A reasoning model spends output tokens on its thinking phase BEFORE emitting `tool_calls`. With the old few-token probe budget the endpoint answered `200` with `finish_reason: 'length'` and no tool call, which read as "native tools unsupported" — persisted, never re-probed, and the assistant degraded to emulated tool calling for good. The probe now budgets 256 tokens and retries once at 1024 before answering `false` |
+| **A thrown detection re-probed on every prompt** | A detection that threw left the capability `undefined`, and `undefined` is what triggers detection — so a transient endpoint blip fired an extra upstream completion before EVERY user prompt, ahead of the rate limiter and outside budget accounting. A 5-minute per-connection backoff now bounds it |
+| **Duplicate `tenantId` index** | Two AI models declared an index the `mongooseTenantPlugin` already creates, producing Mongoose "Duplicate schema index" warnings. Declaration removed; the index itself is unchanged and created by the plugin as before |
+
+## New: `ai.maxRunMs` (opt-in, off by default)
+
+A wall-clock ceiling for ONE prompt run, checked before each agent-loop iteration.
+
+```typescript
+ai: {
+  maxRunMs: 120000,   // 0 or omitted = no limit (previous behaviour)
+}
+```
+
+**Why it is worth setting.** Without it, a run's only bound is `maxIterations` multiplied by the
+connection's per-call timeout — 8 iterations at the 120 s default is a request that can legitimately
+hold a socket, its message buffer and a request context for 16 minutes, with compaction adding a
+call per iteration on top. Set it to something a client would actually wait for.
+
+A misconfigured value degrades to "no limit" rather than to an expired deadline: the check is
+`maxRunMs > 0`, and a non-numeric value arriving through `NSC__AI__MAX_RUN_MS` yields `NaN`, which
+fails that comparison. See `.claude/rules/configurable-features.md` → Numeric Sentinel, Family A.
+
+## Troubleshooting
+
+**"My final answer used to come back as JSON and now returns prose."** That is the behaviour change
+above, and it is the fix rather than a regression: the final answer was never meant to be JSON. If
+you were parsing it, parse the prose or move the structured part into a tool result, which is what
+tool calling is for.
+
+**"An endpoint that supported tools is suddenly using emulated tool calling."** That is the OLD
+defect, and it persisted the wrong flag. Clear the stored capability on the connection so it is
+re-probed with the new budget; detection now records `true` where it previously recorded a false
+negative.
+
+## Module Documentation
+
+- [AI module README](../src/core/modules/ai/README.md)
+- [AI integration checklist](../src/core/modules/ai/INTEGRATION-CHECKLIST.md)
+- [Configurable features](../.claude/rules/configurable-features.md)

package/migration-guides/11.41.0-to-11.41.1.md

@@ -0,0 +1,114 @@
+# Migration Guide: 11.41.0 → 11.41.1
+
+## Overview
+
+| Category | Effort | Applies to |
+|----------|--------|-----------|
+| **Bugfix** | none | Projects whose AI connection points at a reasoning model — prompts that silently returned nothing now return an answer |
+| Behaviour change | none, but worth knowing | Projects using `ai.budget` — one prompt can now cost two upstream calls, and both are metered |
+| New (read-only) | none | `LlmResponse.finishReason`, `LlmUsage.reasoningTokens` — both optional |
+
+Most projects update with `pnpm update @lenne.tech/nest-server` and read no further. **Projects that
+do not use the AI module are unaffected by all of it.**
+
+## Quick Migration
+
+```bash
+pnpm update @lenne.tech/nest-server
+```
+
+```bash
+# Do you use the AI module at all? If this is empty, you are done.
+grep -rn "CoreAiModule\|aiConnections\|AiTool" src/ 2>/dev/null
+```
+
+## Bugfix: a reasoning model that thinks past its budget now gets a second chance
+
+### What was broken
+
+A reasoning model spends output tokens on its thinking phase **before** it writes a single character
+of the answer. When that phase consumes the whole `max_tokens` allowance, the endpoint answers a
+perfectly ordinary `HTTP 200` carrying `finish_reason: 'length'`, empty content, and
+`reasoning_tokens == completion_tokens`.
+
+`chat()` handed that on as an empty string. Nothing distinguished it from "the model had nothing to
+say", so a consumer with a modest `maxTokens` received nothing — silently, on every prompt, for as
+long as the connection pointed at such a model. Call sites that degrade to a null-fallback reported
+no error at all.
+
+Measured against an OpenAI-compatible hosting endpoint with a 900-token budget:
+
+| Model | Default | With the thinking phase off |
+|-------|---------|-----------------------------|
+| Mistral-Medium-3.5-128B | empty, 900/900 spent thinking | 348 characters, 0.8 s |
+| Qwen3.6-35B-A3B-FP8 | empty, 900/900 spent thinking | 399 characters, 0.8 s |
+| gpt-oss-120b | 537 characters | HTTP 400 — rejects the parameter |
+| Ministral-3-14B-Instruct | 1213 characters | 901 characters |
+
+**Raising the token budget does not help.** The model spends whatever it is given: 900 of 900, 1500
+of 1500, 2048 of 2048, 4096 of 4096 — always with empty content.
+
+### What happens now
+
+When, and only when, a completion comes back with `finish_reason: 'length'`, no content and no tool
+call, the provider retries the identical request once with `reasoning_effort: 'none'`. Everything
+else is unchanged: a truncated answer that HAS content is kept (retrying would discard usable text),
+an empty answer with `finish_reason: 'stop'` is kept (the model chose to say nothing, so no budget
+ran out), and a tool call is kept (the model did answer, in the tool channel).
+
+The parameter cannot be sent pre-emptively — `gpt-oss-120b` rejects it with a `400` while working
+perfectly well without it. When the retry fails for any reason, the original response is returned
+with its `finishReason` and usage intact, and the log names the actual failure.
+
+### Do I need to do anything?
+
+**No.** There is no new configuration, and deliberately so: the behaviour only triggers on a
+response that previously reached you as an empty string, which is not an outcome anyone can want.
+
+## Behaviour change: one prompt can now cost two upstream calls
+
+Worth knowing if you run with `ai.budget` limits or watch provider spend.
+
+On the retry path both calls are genuinely billed by the provider, so **both are reported in
+`usage`** — `promptTokens`, `completionTokens` and `reasoningTokens` are the sum of the two. That is
+what keeps `ai.budget` honest: the accounting sums `totalTokens` from the audit records, so
+reporting only the retry would hide the starved call, which by definition burned the entire
+`max_tokens` allowance, from the very limit meant to bound it.
+
+Two consequences:
+
+- On a model that starves on every prompt, real token spend against a configured limit is roughly
+  double what the same workload cost before — because it always was, and is now measured. If you
+  set `ai.budget.user.maxTokens` against observed usage from 11.41.0, re-check the number.
+- `contextWindow.used` counts the prompt twice on that path, since the same prompt really was sent
+  twice. It is clamped to the window size, so it can saturate at 100 % but never report nonsense.
+
+The retry shares the original call's timeout budget rather than starting a fresh one, so the
+documented ceiling of `maxIterations` × the per-call timeout still holds. A connection whose entire
+`timeoutMs` is below one second never retries.
+
+## New: two optional fields on the provider contract
+
+```ts
+interface LlmResponse {
+  /** 'stop', 'length', 'tool_calls', … — undefined when the backend omits it. */
+  finishReason?: string;
+}
+
+interface LlmUsage {
+  /** Output tokens spent thinking. Part of completionTokens, not additional to it. */
+  reasoningTokens?: number;
+}
+```
+
+Both are optional additions, so every existing `ILlmProvider` implementation and every `LlmResponse`
+literal still compiles unchanged. They exist because without them a caller cannot tell an answer
+apart from a fragment: `length` means the budget ran out mid-flight, so short or empty text is a
+truncation rather than the model's verdict.
+
+## Under the hood
+
+`OpenAiCompatibleProvider` gained two protected seams a subclass can override: `postCompletion()`
+(one request, mapped to `LlmResponse`) and `isReasoningStarved()` (the detection predicate). The SSRF
+egress allowlist (`ai.allowedBaseUrlHosts`) is applied before both calls, as before — the retry
+reuses the already-validated URL and changes only the request body.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@lenne.tech/nest-server",
-  "version": "11.40.0",
+  "version": "11.41.1",
   "description": "Modern, fast, powerful Node.js web framework in TypeScript based on Nest with a GraphQL API and a connection to MongoDB (or other databases).",
   "keywords": [
     "node",

package/src/core/common/interfaces/server-options.interface.ts

@@ -1901,6 +1901,23 @@ export interface IAi {
   /** Maximum number of agent-loop iterations (tool round-trips). @default 5 */
   maxIterations?: number;
 
+  /**
+   * Wall-clock ceiling for ONE prompt run, in milliseconds. Checked before each
+   * agent-loop iteration; once exceeded the run stops and answers with whatever it
+   * has (or the translated "no final answer" message).
+   *
+   * Without it the only bound is `maxIterations` multiplied by the connection's
+   * PER-CALL timeout — e.g. 8 iterations at the 120 s default is a request that can
+   * legitimately occupy a socket, its message buffer and a request context for 16
+   * minutes, and compaction can add a further call per iteration on top. Set this
+   * to something a client would actually wait for.
+   *
+   * `0` or omitted disables the check (previous behaviour).
+   *
+   * @default 0
+   */
+  maxRunMs?: number;
+
   /** Maximum characters of a tool-results payload fed back to the model. @default 12000 */
   maxToolResultChars?: number;
 

package/src/core/modules/ai/README.md

@@ -118,11 +118,32 @@ never probed). Detection runs in two complementary ways:
   warns — the stored value is never changed. OFF by default because it makes outbound calls to
   the LLM endpoints on every boot; also skipped in the ci/e2e runners.
 
-The probe is provider-agnostic best effort: `response_format: json_object` is sent
-(2xx → JSON supported); a trivial tool with `tool_choice: 'required'` is sent (2xx
-returning a `tool_calls` result → native tools supported; a `4xx` or a silent ignore
-→ unsupported). Override `OpenAiCompatibleProvider.detectCapabilities()` for custom
-backends, or implement the optional `ILlmProvider.detectCapabilities()` in your own provider.
+The probe is provider-agnostic best effort, and a 2xx alone is never the verdict —
+a backend that does not implement a parameter typically ignores it and answers
+normally, which would persist a `true` it never earns:
+
+- **JSON:** `response_format: json_object` is sent and the CONTENT must actually
+  parse. A response truncated by the output budget (`finish_reason: 'length'`, empty
+  OR partial) proves nothing and is retried once with a larger budget before the
+  probe settles on `false`.
+- **Native tools:** a trivial tool with `tool_choice: 'required'` is sent; a
+  `tool_calls` result → supported, a `4xx` or a complete answer without tool calls →
+  unsupported, a truncation → the same one retry.
+
+Both probes run concurrently. Override `OpenAiCompatibleProvider.detectCapabilities()`
+for custom backends, or implement the optional `ILlmProvider.detectCapabilities()` in
+your own provider.
+
+> **`supportsJsonResponse` is a CONNECTION flag, but whether an answer must be JSON is
+> a property of the PROMPT.** The JSON output contract is carried only by the
+> `output_contract` / `tool_protocol_emulated` fragments (both `capability: 'emulated'`)
+> and by `plan_protocol`. A **native**-tools run receives none of them and is asked for
+> prose — attaching `response_format` on top is a contradiction the model can only
+> resolve by inventing a shape of its own, which then reaches the user as the answer.
+> The orchestrator therefore decides JSON mode per CALL, not per connection: pass
+> `jsonResponse: false` in `LlmCompletionOptions` from any call whose prompt asks for
+> prose. The option only ever NARROWS — it can never assert JSON mode for a connection
+> whose endpoint was not probed for it.
 
 ### Backend examples (external, local, CLI)
 

package/src/core/modules/ai/core-ai.controller.ts

@@ -96,7 +96,22 @@ export class CoreAiController {
     res.setHeader('Cache-Control', 'no-cache');
     res.setHeader('Connection', 'keep-alive');
     res.setHeader('Content-Type', 'text/event-stream');
+    // Disable proxy-side response buffering (nginx and friends honour this); without
+    // it an intermediary can hold the events until the response ends, which defeats
+    // the whole point of the stream.
+    res.setHeader('X-Accel-Buffering', 'no');
     res.flushHeaders?.();
+
+    // A prompt run can stay silent for a long time — a multi-step turn measured
+    // 30-40 s, and `ai.maxRunMs` allows up to two minutes. Many proxies close an
+    // idle connection at 60 s, which the client then sees as a turn that silently
+    // vanished. Comment frames are ignored by every SSE client and keep the
+    // connection observably alive.
+    const heartbeat = setInterval(() => {
+      res.write(': keep-alive\n\n');
+    }, 15_000);
+    heartbeat.unref?.();
+
     try {
       for await (const event of this.aiService.promptStream(input, serviceOptions)) {
         res.write(`data: ${JSON.stringify(event)}\n\n`);
@@ -104,6 +119,7 @@ export class CoreAiController {
     } catch (err) {
       res.write(`data: ${JSON.stringify({ message: (err as Error).message, type: 'error' })}\n\n`);
     } finally {
+      clearInterval(heartbeat);
       res.end();
     }
   }

package/src/core/modules/ai/interfaces/llm-provider.interface.ts

@@ -100,6 +100,17 @@ export interface LlmToolCall {
 export interface LlmUsage {
   completionTokens?: number;
   promptTokens?: number;
+  /**
+   * Output tokens the model spent THINKING before answering, where the backend
+   * reports them (`completion_tokens_details.reasoning_tokens`). Part of
+   * {@link completionTokens}, not additional to it.
+   *
+   * Worth surfacing because the thinking phase competes with the answer for the
+   * SAME budget: when it equals `completionTokens` the model never got to the
+   * answer, which is a very different failure from a model that had nothing to
+   * say. See {@link LlmResponse.finishReason}.
+   */
+  reasoningTokens?: number;
   totalTokens?: number;
 }
 
@@ -108,6 +119,21 @@ export interface LlmUsage {
  * when omitted.
  */
 export interface LlmCompletionOptions {
+  /**
+   * Set `false` to suppress structured-JSON mode for THIS call even though the
+   * connection advertises `supportsJsonResponse`.
+   *
+   * Narrowing only — it can never switch JSON mode ON for a connection whose
+   * endpoint was not probed for it, because the flag is measured per connection and
+   * asserting it elsewhere is what produces a 4xx nobody expected.
+   *
+   * It exists because `supportsJsonResponse` is CONNECTION state while whether an
+   * answer must be JSON is a property of the PROMPT. A caller that asks for prose —
+   * a summary, a native-tools chat turn, a plan summary — must be able to say so
+   * without rebuilding the connection object around the flag.
+   */
+  jsonResponse?: boolean;
+
   /** Maximum number of tokens to generate. */
   maxTokens?: number;
 
@@ -125,6 +151,19 @@ export interface LlmCompletionOptions {
  * Normalized response of a single LLM completion.
  */
 export interface LlmResponse {
+  /**
+   * Why the model stopped, as reported by the backend (`stop`, `length`,
+   * `tool_calls`, …). Undefined when the backend omits it.
+   *
+   * Without this a caller cannot tell an answer apart from a fragment: `length`
+   * means the output budget ran out mid-flight, so short or empty text is a
+   * truncation and not the model's verdict. Retrying an identical request is
+   * pointless in that case — and actively misleading against a backend that
+   * caches identical prompts, which answers the retry from cache in
+   * milliseconds.
+   */
+  finishReason?: string;
+
   /** Raw provider payload (for debugging/audit, never sent to clients). */
   raw?: unknown;
 

package/src/core/modules/ai/models/core-ai-prompt.model.ts

@@ -89,9 +89,18 @@ export class CoreAiPrompt extends CorePersistenceModel {
 
   /** Tenant id when scope = 'tenant' (set from the creator's tenant at create time). */
   @UnifiedField({
+    // No `index: true`: declaring the `tenantId` PATH is what activates
+    // `mongooseTenantPlugin`, and the plugin then adds `schema.index({ tenantId: 1 })`
+    // itself. Declaring it here as well makes Mongoose log
+    // "Duplicate schema index on {"tenantId":1}" on every boot.
+    //
+    // The `mongoose` key itself MUST stay: `UnifiedField` emits `@Prop` only inside
+    // `if (opts.mongoose)`, so dropping the whole key would remove the schema path —
+    // and `mongooseTenantPlugin` returns early on `!schema.path('tenantId')`, i.e.
+    // the model would lose its tenant filtering entirely.
     description: 'Tenant id (when scope = "tenant")',
     isOptional: true,
-    mongoose: { index: true },
+    mongoose: { type: String },
     roles: RoleEnum.S_USER,
   })
   tenantId?: string = undefined;

package/src/core/modules/ai/models/core-ai-slot.model.ts

@@ -133,9 +133,18 @@ export class CoreAiSlot extends CorePersistenceModel {
    * slot is effectively system-wide.
    */
   @UnifiedField({
+    // No `index: true`: declaring the `tenantId` PATH is what activates
+    // `mongooseTenantPlugin`, and the plugin then adds `schema.index({ tenantId: 1 })`
+    // itself. Declaring it here as well makes Mongoose log
+    // "Duplicate schema index on {"tenantId":1}" on every boot.
+    //
+    // The `mongoose` key itself MUST stay: `UnifiedField` emits `@Prop` only inside
+    // `if (opts.mongoose)`, so dropping the whole key would remove the schema path —
+    // and `mongooseTenantPlugin` returns early on `!schema.path('tenantId')`, i.e.
+    // the model would lose its tenant filtering entirely.
     description: 'Tenant id the slot applies to (auto-set; undefined = system-wide)',
     isOptional: true,
-    mongoose: { index: true },
+    mongoose: { type: String },
     roles: RoleEnum.ADMIN,
   })
   tenantId?: string = undefined;