@lenne.tech/nest-server 11.39.0 → 11.40.0

package/migration-guides/11.39.0-to-11.40.0.md

@@ -0,0 +1,186 @@
+# Migration Guide: 11.39.0 → 11.40.0
+
+> **Why a MINOR for what looks like a patch.** In this package the MAJOR digit tracks the NestJS
+> major (11.x = NestJS 11), so it is not ours to spend — and every breaking change of our own ships
+> as a MINOR instead. This release contains one: a security control that was silently inert starts
+> being enforced, and a deployment that relied on the inert behaviour can stop working. The number
+> of affected projects is small; the rule does not ask how many, it asks whether a working
+> deployment can break.
+
+## Overview
+
+| Category | Effort | Applies to |
+|----------|--------|-----------|
+| **Breaking (behaviour)** | 5 minutes | Projects that set `ai.allowedBaseUrlHosts` **as a string** (i.e. via `NSC__AI__ALLOWED_BASE_URL_HOSTS`) |
+| Bugfix | none | Everyone using the AI module |
+| Internal tooling | none | Nobody — `scripts/` does not ship |
+
+Almost every project can update with `pnpm update @lenne.tech/nest-server` and read no further.
+**One group cannot**, and for them the change is the uncomfortable kind: a security control that was
+silently inert starts working, and a working deployment can stop working as a result.
+
+## Quick Migration
+
+```bash
+# Does this affect you? If both come back empty, you are done.
+grep -r "NSC__AI__ALLOWED_BASE_URL_HOSTS" . --include="*.env*" --include="*.yml" --include="*.yaml" 2>/dev/null
+grep -rn "allowedBaseUrlHosts" src/ 2>/dev/null
+```
+
+## Breaking Change: `ai.allowedBaseUrlHosts` set as a string was never enforced
+
+### What was wrong
+
+`ai.allowedBaseUrlHosts` is the SSRF egress allowlist for AI connection base URLs. It is reachable
+through the framework's own environment mapping:
+
+```bash
+NSC__AI__ALLOWED_BASE_URL_HOSTS=llm.example.com,api.openai.com
+```
+
+`getEnvironmentObject()` turns that into `{ ai: { allowedBaseUrlHosts: '<string>' } }`, and lodash
+`merge` assigns the scalar straight over the configured array.
+
+**And there is no way to avoid that from the environment.** The `NSC__*` reader coerces exactly
+three things — `'true'`, `'false'`, and anything `Number()` accepts — and leaves everything else a
+string. No CSV, JSON array literal or other notation produces an array:
+
+```bash
+NSC__AI__ALLOWED_BASE_URL_HOSTS='["a.example.com"]'   # -> the string '["a.example.com"]'
+NSC__AI__MAX_ITERATIONS=7                             # -> the number 7
+```
+
+So this is not an edge case for people who happened to pick the wrong notation. **If you configured
+the allowlist through the environment at all, it was off — completely, on every deployment, for its
+whole life.** Verified empirically against the built `config.helper.js`, not inferred from the code.
+Reported by the nest-server-starter session, which went looking for why a string arrives in the
+first place.
+
+That gives you a clean dividing line, and it is the only thing you need to check:
+
+| How you set `ai.allowedBaseUrlHosts` | Were you affected? |
+|---|---|
+| `NSC__AI__ALLOWED_BASE_URL_HOSTS` (or any env route) | **Yes — the control was inert** |
+| A real array in `config.env.ts` | No — it worked as documented |
+| Not set at all | No — no restriction was intended |
+
+The guard then did:
+
+```typescript
+if (!Array.isArray(allowedHosts) || !allowedHosts.length) {
+  return;   // read as "no allowlist configured"
+}
+```
+
+So a string was read as **"not configured"** and the check was skipped entirely — no log line, no
+error. An operator who used the canonical `NSC__` spelling (the documented form for every other
+setting) had **no egress restriction at all** while believing the control was on.
+
+### What changes in 11.40.0
+
+A string is now parsed as a comma-separated list, so the setting does what it says.
+
+**This is a behaviour change in the restrictive direction.** If you set it as a string, your
+deployment went from *no restriction* to *enforced*. Any AI connection whose host is not in that
+list now fails with `ServiceUnavailableException` and a WARN naming the host. There is no
+deprecation window, because leaving an SSRF control off for a release cycle is worse than the
+breakage.
+
+### Before upgrading
+
+1. Find the configured value:
+   ```bash
+   grep -r "NSC__AI__ALLOWED_BASE_URL_HOSTS" . --include="*.env*" --include="*.yml" 2>/dev/null
+   ```
+2. List every `baseUrl` your connections actually use — including any seeded via
+   `ai.defaultConnection`, and any added at runtime through `aiConnections`:
+   ```
+   query { aiConnections { name baseUrl enabled } }
+   ```
+3. Confirm every one of those hosts appears in the list. Add the missing ones, or clear the setting
+   entirely if you did not mean to restrict egress:
+   ```bash
+   NSC__AI__ALLOWED_BASE_URL_HOSTS=llm.example.com,api.openai.com,localhost:11434
+   ```
+
+**Unset is still permissive**, unchanged — a local Ollama works out of the box.
+
+### Matching rules
+
+Worth reading once, because they are the likely cause of a surprising refusal:
+
+| Entry | Matches | Does not match |
+|-------|---------|----------------|
+| `llm.example.com` | any port on that host | `llm.example.com.evil.test` |
+| `llm.internal:8080` | exactly that port | `llm.internal:9200` |
+| `example.com:443` | `https://example.com/` (default port) | `http://example.com/` |
+| `LLM.Example.com` | `https://llm.example.com/` | — |
+
+Entries and URLs are both trimmed, lowercased and stripped of a fully-qualifying trailing dot, so
+neither side can win by spelling one DNS name differently.
+
+## Bugfix: the allowlist now covers every outbound path
+
+`probeContextWindow()` (the Ollama `/api/show` probe behind `detectContextWindow()`) reached the
+network **without consulting the allowlist**. It is not an admin-only path: `CoreAiService` calls
+`detectAndPersistCapabilities()` on an ordinary user prompt whenever `contextWindow` is undefined,
+and it runs *before* the AI rate limit.
+
+All three outbound paths — chat completions, the capability probe, and the context-window probe —
+now go through the same check. A refusal there degrades to "context window unknown" and falls back
+to the built-in model table, so nothing breaks for an allowed host.
+
+**No action required**, unless you were relying on the context-window probe reaching a host your
+allowlist excludes — in which case add that host.
+
+## Bugfix: a malformed value is now reported
+
+A value that is neither a list nor a string (a number, a boolean, an object — all reachable through
+`NEST_SERVER_CONFIG`) carries no hostnames, so the allowlist cannot be applied. That was silent.
+It is now logged as an error:
+
+```
+ai.allowedBaseUrlHosts is a number and carries no hostnames — the SSRF egress allowlist is
+NOT active. Use an array or a comma-separated string.
+```
+
+The behaviour is unchanged (permissive); only the silence is gone. From the outside, "misconfigured"
+and "deliberately unset" looked identical, which is what let the original defect survive.
+
+## Not consumer-facing
+
+The rest of this release is repository tooling and does not ship: a new `check:overrides` guard for
+stale pnpm overrides and audit suppressions, a `pnpm peers check` step, audit-count accounting in
+`scripts/check.mjs`, and test-evidence work. `package.json` `files` ships `dist` plus docs, and the
+CLI's vendor transformation copies `src/core/` — neither includes `scripts/`.
+
+Four further tooling fixes landed late in the release, all in the same area and all of the same
+kind — a gate that reported safety it had not established:
+
+| Fix | What it was |
+|-----|-------------|
+| Audit hang guard (`CHECK_AUDIT_TIMEOUT`, default 600s) | `pnpm audit` emits no intermediate output, so the existing idle watchdog structurally could not tell a hang from a healthy slow run. A second, absolute cap now kills it with its own cause |
+| `'unreadable'` degradation | An audit exiting **0** with no parseable tally printed a GREEN tick and a literal `0`, having assessed nothing. Reachable in practice: a pnpm version collision writes its error to stderr and exits 0 |
+| 5xx read from the code field only | The signature matched `\b5\d\d\b` against the whole envelope, so pnpm's own `audited 503 packages` degraded a run that had to block |
+| Reachability probe asked the wrong registry | Both check scripts hardcoded `registry.npmjs.org` while pnpm audits against the **configured** registry. Behind a private registry or proxy that reproduces the very false-green the probe removes: the real registry is unreachable, npmjs.org answers, the run reports "clean" |
+| Steps list contradicted the warning | A degraded audit printed a yellow warning and "NOT CHECKED" — and a **green tick** in the Steps list, which hard-coded one per step |
+| JSONC comment stripping | A regex stripper ate glob patterns out of `tsconfig.tests.json` and `.oxlintrc.json` (2783 bytes, and one whole `overrides` entry). Both files still parsed, so the assertions above them ran green against a mutilated config |
+
+None of them changes shipped behaviour; they are recorded because each one made a **check** claim
+something it had not verified, and that is the class of defect a consumer inherits indirectly — via
+a release that passed a gate which was not looking.
+
+## Troubleshooting
+
+| Symptom | Cause | Fix |
+|---------|-------|-----|
+| AI stopped working after the update; log shows `host "…" is not in ai.allowedBaseUrlHosts` | The allowlist is now enforced where it previously was not | Add the host, or clear the setting |
+| `ServiceUnavailableException` on one connection only | That connection's `baseUrl` host is missing from the list | Add it — check `aiConnections`, not just `defaultConnection` |
+| Error log says the allowlist is `NOT active` | The value is not a list or a string | Use an array, or a comma-separated string |
+| An entry with `:443` stopped matching an `http://` URL | `:443` is the https default, not http's | Write the host without a port, or with the right one |
+
+## Module Documentation
+
+- `src/core/modules/ai/README.md` → "Egress allowlist (`ai.allowedBaseUrlHosts`)"
+- `.claude/rules/configurable-features.md` → "AI Egress Allowlist (SSRF)"
+- `src/core/common/interfaces/server-options.interface.ts` → `IAi.allowedBaseUrlHosts`

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@lenne.tech/nest-server",
-  "version": "11.39.0",
+  "version": "11.40.0",
   "description": "Modern, fast, powerful Node.js web framework in TypeScript based on Nest with a GraphQL API and a connection to MongoDB (or other databases).",
   "keywords": [
     "node",
@@ -25,11 +25,12 @@
     "cf": "pnpm run check:fix",
     "check": "node scripts/check.mjs",
     "check:consumer": "node scripts/check-consumer.mjs",
-    "check:fix": "pnpm install && pnpm run spectaql:sync && pnpm audit --fix && pnpm run format && pnpm run lint:fix && pnpm run typecheck:tests && pnpm run check:swc-tdz && pnpm test && pnpm run build && pnpm run check:manifest && bash scripts/check-server-start.sh",
+    "check:fix": "pnpm install && pnpm run spectaql:sync && pnpm audit --fix && pnpm run check:overrides && pnpm peers check && pnpm run format && pnpm run lint:fix && pnpm run typecheck:tests && pnpm run check:swc-tdz && pnpm test && pnpm run build && pnpm run check:manifest && bash scripts/check-server-start.sh",
     "check:manifest": "node scripts/check-package-manifest.mjs",
     "check:mutations": "node scripts/check-mutations.mjs",
-    "check:naf": "pnpm install && pnpm run spectaql:sync && pnpm run format && pnpm run lint:fix && pnpm run typecheck:tests && pnpm run check:swc-tdz && pnpm test && pnpm run build && pnpm run check:manifest && bash scripts/check-server-start.sh",
-    "check:raw": "pnpm install --frozen-lockfile && pnpm run spectaql:sync && pnpm audit && pnpm run format:check && pnpm run lint && pnpm run typecheck:tests && pnpm run check:swc-tdz && pnpm test && pnpm run build && pnpm run check:manifest && bash scripts/check-server-start.sh",
+    "check:naf": "pnpm install && pnpm run spectaql:sync && pnpm run check:overrides && pnpm peers check && pnpm run format && pnpm run lint:fix && pnpm run typecheck:tests && pnpm run check:swc-tdz && pnpm test && pnpm run build && pnpm run check:manifest && bash scripts/check-server-start.sh",
+    "check:overrides": "node scripts/check-overrides.mjs",
+    "check:raw": "pnpm install --frozen-lockfile && pnpm run spectaql:sync && pnpm audit && pnpm run check:overrides && pnpm peers check && pnpm run format:check && pnpm run lint && pnpm run typecheck:tests && pnpm run check:swc-tdz && pnpm test && pnpm run build && pnpm run check:manifest && bash scripts/check-server-start.sh",
     "check:swc-tdz": "nest build -b swc -p tsconfig.swc-tdz.json && node scripts/check-swc-tdz.mjs",
     "cnaf": "pnpm run check:naf",
     "docs": "pnpm run docs:ci && open http://127.0.0.1:8080/ && open ./public/index.html && compodoc -p tsconfig.json -s ",

package/src/core/common/interfaces/server-options.interface.ts

@@ -1712,9 +1712,25 @@ export interface IAi {
    * bare `hostname`); unset → permissive (so local providers like Ollama on localhost
    * work out of the box). `baseUrl` is admin-only, so this guards a compromised or
    * misconfigured admin, not end-user input.
+   *
+   * Accepts an array OR a comma-separated string. The string form is not a convenience:
+   * it is the shape the framework's own env mapping produces. `NSC__AI__ALLOWED_BASE_URL_HOSTS`
+   * becomes `{ ai: { allowedBaseUrlHosts: '<string>' } }`, and lodash `merge` assigns that
+   * scalar straight over a configured array — so the canonical `NSC__` spelling MUST be
+   * understood here or the control silently switches itself off.
+   *
+   * Entries are trimmed, lowercased and stripped of a fully-qualifying trailing dot, and the
+   * same normalisation is applied to the URL being checked, so neither side can win by
+   * spelling the same DNS name differently. A bare hostname entry matches any port on that
+   * host; an entry naming the scheme's default port (`example.com:443` for https) also matches
+   * the portless URL. A value that is neither an array nor a string carries no hostnames, so
+   * the allowlist is inactive — that case is LOGGED as an error rather than passed over, since
+   * it looks identical to "correctly unset" from the outside.
+   *
    * @example ['llm.example.com', 'localhost:11434']
+   * @example 'llm.example.com,localhost:11434'   // NSC__AI__ALLOWED_BASE_URL_HOSTS
    */
-  allowedBaseUrlHosts?: string[];
+  allowedBaseUrlHosts?: string | string[];
 
   /**
    * Persist an audit record (`aiInteractions`) for every prompt run (admin-readable).

package/src/core/modules/ai/README.md

@@ -159,6 +159,39 @@ calling and executes tools itself through `CrudService` with the caller's permis
 the child runs in a temp dir so no `CLAUDE.md`/settings leak into the context. See
 `ClaudeCliProvider` for the full security model and the optional `ai.claudeCli` config.
 
+## Egress allowlist (`ai.allowedBaseUrlHosts`)
+
+A connection's `baseUrl` decides where the server sends outbound HTTP. It is admin-set, so the
+threat model is a compromised or mistyped admin rather than end-user input — but the request still
+leaves from inside your network, which is what makes it an SSRF surface.
+
+**Unset (the default) means no restriction**, so a local provider works out of the box. When set,
+only the listed hosts are reachable and everything else is refused with
+`ServiceUnavailableException` plus a WARN naming the host:
+
+```typescript
+ai: {
+  allowedBaseUrlHosts: ['llm.example.com', 'localhost:11434'],
+}
+```
+
+```bash
+# Same setting via the canonical env spelling — a comma-separated string is understood
+NSC__AI__ALLOWED_BASE_URL_HOSTS=llm.example.com,localhost:11434
+```
+
+Matching details worth knowing before you debug a refusal:
+
+- A bare hostname entry matches **any port** on that host. An entry that names the scheme's default
+  port (`example.com:443` for https) also matches the portless URL.
+- Entries and URLs are both trimmed, lowercased, and stripped of a fully-qualifying trailing dot, so
+  `LLM.Example.com`, `llm.example.com` and `llm.example.com.` are one host.
+- The check covers **all three** outbound paths: chat completions, the capability probe, and the
+  Ollama context-window probe.
+- A value that is neither a list nor a string carries no hostnames. The allowlist is then inactive
+  and the framework logs an error — from the outside that state is indistinguishable from
+  "correctly unset", which is exactly why it is not silent.
+
 ## Connections (DB configuration)
 
 Connections live in the `aiConnections` collection and are managed by admins via

package/src/core/modules/ai/providers/openai-compatible.provider.ts

@@ -134,8 +134,8 @@ export class OpenAiCompatibleProvider implements ILlmProvider {
    * admin, not an end-user input.
    */
   protected assertBaseUrlAllowed(url: string): void {
-    const allowedHosts = ConfigService.get<string[]>('ai.allowedBaseUrlHosts');
-    if (!Array.isArray(allowedHosts) || !allowedHosts.length) {
+    const allowedHosts = this.resolveAllowedBaseUrlHosts();
+    if (!allowedHosts.length) {
       return;
     }
     let parsed: URL;
@@ -144,7 +144,17 @@ export class OpenAiCompatibleProvider implements ILlmProvider {
     } catch {
       throw new ServiceUnavailableException(ErrorCode.AI_CONNECTION_INVALID_URL);
     }
-    if (!allowedHosts.includes(parsed.host) && !allowedHosts.includes(parsed.hostname)) {
+    // A hostname entry stays host-wide (any port); the extra candidates only make an
+    // operator who was MORE explicit than necessary succeed rather than fail. `URL.host`
+    // omits the default port, so a conscientious `llm.example.com:443` entry would
+    // otherwise never match `https://llm.example.com/` — a lockout whose only symptom is
+    // a WARN and "the AI stopped working". Nothing here widens the set of reachable hosts.
+    const defaultPort = parsed.protocol === 'https:' ? '443' : parsed.protocol === 'http:' ? '80' : '';
+    const candidates = [parsed.host, parsed.hostname];
+    if (defaultPort && parsed.host === parsed.hostname) {
+      candidates.push(`${parsed.hostname}:${defaultPort}`);
+    }
+    if (!candidates.map((candidate) => this.normaliseHostEntry(candidate)).some((c) => allowedHosts.includes(c))) {
       this.logger.warn(
         `AI connection "${this.connection.name}" host "${parsed.host}" is not in ai.allowedBaseUrlHosts`,
       );
@@ -152,6 +162,62 @@ export class OpenAiCompatibleProvider implements ILlmProvider {
     }
   }
 
+  /**
+   * The configured egress allowlist as a lowercase array, whatever shape it has.
+   *
+   * A STRING is split as CSV rather than rejected, because `ai.allowedBaseUrlHosts`
+   * is reachable through the framework's own `NSC__AI__ALLOWED_BASE_URL_HOSTS`
+   * environment mapping: `getEnvironmentObject()` turns that variable into
+   * `{ ai: { allowedBaseUrlHosts: '<string>' } }` and lodash `merge` assigns the
+   * scalar straight over the configured array. A bare `!Array.isArray(...) -> return`
+   * then reads it as "no allowlist configured" and skips the check entirely — so an
+   * operator using the canonical `NSC__` spelling silently disables SSRF egress
+   * control, with no log line and no error. A malformed security setting must be
+   * interpreted or fail CLOSED, never fail open.
+   *
+   * Entries are lowercased because `URL.host` / `URL.hostname` always are; a
+   * differently-cased entry would otherwise fail closed for no stated reason, and
+   * the only symptom would be a WARN log plus "the AI stopped working".
+   *
+   * A value that is NEITHER an array nor a string carries no hostnames and cannot be
+   * interpreted — `NSC__AI__ALLOWED_BASE_URL_HOSTS=0` coerces to a number, and
+   * `NEST_SERVER_CONFIG` can deliver an object. Returning an empty list there is the
+   * only honest answer, but it reopens egress while the operator believes the control
+   * is on, so it is LOGGED every time rather than passed over in silence. That is the
+   * difference between this and the documented unset-is-permissive default: unset is a
+   * decision, a malformed value is an accident nobody is told about.
+   */
+  protected resolveAllowedBaseUrlHosts(): string[] {
+    const configured = ConfigService.get<unknown>('ai.allowedBaseUrlHosts');
+    if (configured === undefined || configured === null) {
+      return [];
+    }
+    const entries = typeof configured === 'string' ? configured.split(',') : configured;
+    if (!Array.isArray(entries)) {
+      this.logger.error(
+        `ai.allowedBaseUrlHosts is a ${typeof configured} and carries no hostnames — the SSRF egress ` +
+          'allowlist is NOT active. Use an array or a comma-separated string.',
+      );
+      return [];
+    }
+    return entries.map((host) => this.normaliseHostEntry(String(host))).filter(Boolean);
+  }
+
+  /**
+   * Lowercase, trim, and drop a fully-qualifying trailing dot.
+   *
+   * Applied to BOTH the allowlist entry and the URL being checked, so neither side can
+   * win by spelling the same DNS name differently. `llm.example.com.` and
+   * `llm.example.com` resolve identically, so treating them as different hosts only ever
+   * produced a confusing refusal — never protection.
+   */
+  protected normaliseHostEntry(value: string): string {
+    return value
+      .trim()
+      .toLowerCase()
+      .replace(/\.(?=$|:)/, '');
+  }
+
   /**
    * Probe the backend to auto-detect capabilities for flags the connection left
    * undefined. Explicit flags are authoritative and are NOT probed. Best effort:
@@ -263,6 +329,13 @@ export class OpenAiCompatibleProvider implements ILlmProvider {
     if (!base.startsWith('http')) {
       return undefined;
     }
+    // The third outbound path from the same admin-controlled baseUrl, and the one the
+    // allowlist used to miss. It is NOT admin-only in practice: CoreAiService calls
+    // detectAndPersistCapabilities() on an ordinary user prompt whenever contextWindow is
+    // undefined, and it runs BEFORE checkRateLimit(). A guard applied to two of three
+    // egress paths is not a guard. Throwing is right here — detectContextWindow() already
+    // wraps this call, so a refusal degrades to "context window unknown".
+    this.assertBaseUrlAllowed(`${base}/api/show`);
     const response = await fetch(`${base}/api/show`, {
       body: JSON.stringify({ name: this.connection.model }),
       headers: { 'Content-Type': 'application/json' },