@nanogpt/private-mode 0.2.8 → 0.2.11

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -5,7 +5,7 @@ OpenAI-compatible localhost proxy for NanoGPT Private Mode with supported TEE mo
5
5
  Requires Node.js 22 or later.
6
6
 
7
7
  ```bash
8
- NANOGPT_API_KEY=sk-your-key npx @nanogpt/private-mode
8
+ NANOGPT_API_KEY=sk-your-key npx --yes @nanogpt/private-mode@latest
9
9
  ```
10
10
 
11
11
  Then point any OpenAI-compatible client at:
@@ -32,6 +32,14 @@ const response = await client.chat.completions.create({
32
32
 
33
33
  The local proxy verifies TEE attestation, encrypts request bodies with EHBP, sends ciphertext through NanoGPT, decrypts encrypted responses locally, and returns normal OpenAI JSON to the calling app.
34
34
 
35
+ The model list is bundled with the installed proxy package. Before the proxy starts listening, it checks that list against NanoGPT's hosted Private Mode catalog and stops with an upgrade command if models were added or retired. This prevents an old package from advertising enclave targets that are no longer available. The running proxy reports its exact version at `GET http://127.0.0.1:8787/v1/private-mode/status`. To print the latest published package version, run:
36
+
37
+ ```bash
38
+ npx --yes @nanogpt/private-mode@latest --version
39
+ ```
40
+
41
+ If an older proxy is already running, stop it before starting the command above. API clients must use the localhost base URL shown below; sending `private/*` model IDs directly to NanoGPT's standard `/api/v1/chat/completions` endpoint does not enable Private Mode.
42
+
35
43
  After a long idle period (five minutes by default), the proxy re-verifies attestation and creates a fresh encrypted transport before the next request. This makes laptop sleep/resume safe without requiring a proxy restart. Set `NANOGPT_PRIVATE_CLIENT_IDLE_RESET_MS` to a positive millisecond value to adjust the idle threshold, or `0` to disable idle resets.
36
44
 
37
45
  NanoGPT can see account identity, selected private model, selected TEE target metadata, timing, sizes, status, and usage metadata. NanoGPT cannot read the prompt or completion body for supported private models.
@@ -44,6 +52,12 @@ In the hosted web app, decrypted Private Mode turns remain in local browser hist
44
52
 
45
53
  Private Mode also requires enough NanoGPT balance and API-key spend-limit headroom before dispatch. For streaming calls, NanoGPT may precharge a conservative reserve before dispatch and refund unused balance after verified usage metadata is available.
46
54
 
55
+ Because the request body is encrypted before NanoGPT receives it, the service cannot inspect the request's `max_tokens` value during the balance precheck. The default streaming reserve therefore includes up to 32,768 output tokens, even when the encrypted request asks for fewer. This is a temporary reserve, not the final charge: unused balance is refunded when verified usage metadata arrives. The encrypted `max_tokens` value still controls generation inside the verified TEE.
56
+
57
+ Private Mode streaming has a 29-minute soft deadline so billing can settle before the route's 30-minute hard runtime limit. If the decrypted upstream stream ends without both a non-null `finish_reason` and the OpenAI `[DONE]` marker, the local proxy emits an SSE error with code `private_stream_incomplete`. Any text received before that error may be partial.
58
+
59
+ Private Mode currently supports Chat Completions only. The Responses API and Batch API are not available through the encrypted Private Mode endpoint. The standard Responses API outside Private Mode supports background requests within its configured 30-minute runtime limit; work that must reliably continue longer needs a durable job system.
60
+
47
61
  Useful local checks:
48
62
 
49
63
  ```text
@@ -54,17 +68,21 @@ GET http://127.0.0.1:8787/v1/private-mode/attestation
54
68
 
55
69
  Supported private model IDs include:
56
70
 
57
- - `private/deepseek-v4-flash`
71
+ - `private/deepseek-v4-flash` - DeepSeek V4 Flash 0731
58
72
  - `private/kimi-k3`
59
- - `private/glm-5-1`
60
- - `private/glm-5-1-thinking`
73
+ - `private/gpt-oss-120b`
74
+ - `private/llama3-3-70b`
61
75
  - `private/glm-5-2`
62
76
  - `private/glm-5-2:thinking`
77
+ - `private/gemma4-31b`
78
+ - `private/gemma4-31b:thinking`
79
+
80
+ Both GLM 5.2 variants are deployed with a 393,216-token total context limit, with prompt and output sharing that window. Kimi K3 Private has a 256,000-token context limit. DeepSeek V4 Flash 0731 Private has a 1,048,576-token total context limit. The model-list extension `context_length` is the combined prompt-and-output window, while `max_output_tokens` is the output ceiling within that same window. These values are not additive.
63
81
 
64
82
  Browser requests are locked down by default. The proxy only accepts same-machine clients and rejects browser `Origin` headers that are not explicitly allowed, so a random website or LAN client cannot spend the local `NANOGPT_API_KEY` while the proxy is running. If a local browser app needs to call the proxy directly, allow that exact origin:
65
83
 
66
84
  ```bash
67
- NANOGPT_API_KEY=sk-your-key npx @nanogpt/private-mode --allow-origin http://localhost:3000
85
+ NANOGPT_API_KEY=sk-your-key npx --yes @nanogpt/private-mode@latest --allow-origin http://localhost:3000
68
86
  ```
69
87
 
70
88
  You can also set `NANOGPT_PRIVATE_ALLOWED_ORIGINS=http://localhost:3000`. Wildcard browser origins are not supported.
@@ -86,13 +104,13 @@ NanoGPT's hosted web app shows a Private Mode receipt on completed Private Mode
86
104
  To independently verify a copied receipt:
87
105
 
88
106
  ```bash
89
- npx @nanogpt/private-mode verify receipt.json
107
+ npx --yes @nanogpt/private-mode@latest verify receipt.json
90
108
  ```
91
109
 
92
110
  You can also pipe JSON on stdin:
93
111
 
94
112
  ```bash
95
- cat receipt.json | npx @nanogpt/private-mode verify
113
+ cat receipt.json | npx --yes @nanogpt/private-mode@latest verify
96
114
  ```
97
115
 
98
116
  The verifier re-fetches attestation material for the receipt's enclave, verifies it locally, and checks the resulting measurements, release digest, and HPKE key against the copied receipt. If an optional attestation bundle hash is present but the freshly fetched bundle serializes differently, the verifier prints a warning instead of failing the core verification.
@@ -1,6 +1,7 @@
1
1
  #!/usr/bin/env node
2
2
 
3
3
  import { parseArgs } from 'node:util';
4
+ import { PRIVATE_MODE_PROXY_VERSION } from '../lib/packageInfo.js';
4
5
  import { startPrivateModeProxy } from '../lib/server.js';
5
6
  import { verifyPrivateModeReceiptCli } from '../lib/verifyReceipt.js';
6
7
 
@@ -14,10 +15,16 @@ const { values, positionals } = parseArgs({
14
15
  },
15
16
  'allow-origin': { type: 'string', multiple: true },
16
17
  help: { type: 'boolean', short: 'h', default: false },
18
+ version: { type: 'boolean', short: 'v', default: false },
17
19
  },
18
20
  allowPositionals: true,
19
21
  });
20
22
 
23
+ if (values.version) {
24
+ process.stdout.write(`${PRIVATE_MODE_PROXY_VERSION}\n`);
25
+ process.exit(0);
26
+ }
27
+
21
28
  if (values.help) {
22
29
  process.stdout.write(`NanoGPT Private Mode Proxy
23
30
 
@@ -32,6 +39,7 @@ Options:
32
39
  --api-base <url> NanoGPT API base. Default: https://nano-gpt.com
33
40
  --allow-origin <origin>
34
41
  Browser origin allowed to call the proxy, for example http://localhost:3000
42
+ --version, -v Print the installed proxy version
35
43
  `);
36
44
  process.exit(0);
37
45
  }
@@ -0,0 +1,76 @@
1
+ const DEFAULT_CATALOG_CHECK_TIMEOUT_MS = 10_000;
2
+
3
+ function normalizeModelIds(models) {
4
+ if (!Array.isArray(models)) return null;
5
+
6
+ const ids = models.map((model) => (
7
+ typeof model?.id === 'string' ? model.id.trim().toLowerCase() : ''
8
+ ));
9
+ if (ids.some((id) => !id)) return null;
10
+
11
+ return new Set(ids);
12
+ }
13
+
14
+ export function comparePrivateModeCatalogs(localModels, hostedCatalog) {
15
+ const localIds = normalizeModelIds(localModels);
16
+ const hostedIds = normalizeModelIds(hostedCatalog?.data);
17
+ if (!localIds || !hostedIds) {
18
+ throw new Error('Private Mode model catalog response was malformed.');
19
+ }
20
+
21
+ return {
22
+ status: 'checked',
23
+ retiredLocalModels: [...localIds].filter((id) => !hostedIds.has(id)).sort(),
24
+ missingLocalModels: [...hostedIds].filter((id) => !localIds.has(id)).sort(),
25
+ };
26
+ }
27
+
28
+ export async function checkHostedPrivateModeCatalog({
29
+ apiBase,
30
+ localModels,
31
+ fetchImpl = fetch,
32
+ timeoutMs = DEFAULT_CATALOG_CHECK_TIMEOUT_MS,
33
+ }) {
34
+ const controller = new AbortController();
35
+ const timeout = setTimeout(() => controller.abort(), timeoutMs);
36
+
37
+ try {
38
+ const response = await fetchImpl(`${apiBase}/api/v1/private/tinfoil/models`, {
39
+ headers: { accept: 'application/json' },
40
+ signal: controller.signal,
41
+ });
42
+ if (!response.ok) {
43
+ return { status: 'unavailable', reason: `HTTP ${response.status}` };
44
+ }
45
+
46
+ return comparePrivateModeCatalogs(localModels, await response.json());
47
+ } catch (error) {
48
+ return {
49
+ status: 'unavailable',
50
+ reason: error instanceof Error ? error.message : String(error),
51
+ };
52
+ } finally {
53
+ clearTimeout(timeout);
54
+ }
55
+ }
56
+
57
+ export function hasPrivateModeCatalogMismatch(result) {
58
+ return result.status === 'checked'
59
+ && (result.retiredLocalModels.length > 0 || result.missingLocalModels.length > 0);
60
+ }
61
+
62
+ export function buildPrivateModeCatalogUnavailableMessage(result, apiBase) {
63
+ return `Could not verify the bundled Private Mode model catalog against ${apiBase}: ${result.reason}. The proxy did not start because catalog compatibility could not be confirmed. Check your network connection and NanoGPT service availability, then retry.`;
64
+ }
65
+
66
+ export function buildPrivateModeCatalogMismatchMessage(result, proxyVersion) {
67
+ const details = [];
68
+ if (result.retiredLocalModels.length > 0) {
69
+ details.push(`retired local models: ${result.retiredLocalModels.join(', ')}`);
70
+ }
71
+ if (result.missingLocalModels.length > 0) {
72
+ details.push(`hosted models missing locally: ${result.missingLocalModels.join(', ')}`);
73
+ }
74
+
75
+ return `The bundled Private Mode model catalog in @nanogpt/private-mode v${proxyVersion} is out of date (${details.join('; ')}). Stop this proxy and restart with: NANOGPT_API_KEY=... npx --yes @nanogpt/private-mode@latest`;
76
+ }
@@ -0,0 +1,6 @@
1
+ export const PRIVATE_MODE_STREAM_ROUTE_MAX_DURATION_MS = 30 * 60 * 1000;
2
+ export const PRIVATE_MODE_STREAM_SOFT_DEADLINE_LEAD_MS = 60 * 1000;
3
+ export const PRIVATE_MODE_STREAM_SETTLEMENT_SAFETY_MS = 30 * 1000;
4
+ export const PRIVATE_MODE_STREAM_SOFT_DEADLINE_MS =
5
+ PRIVATE_MODE_STREAM_ROUTE_MAX_DURATION_MS - PRIVATE_MODE_STREAM_SOFT_DEADLINE_LEAD_MS;
6
+ export const PRIVATE_MODE_DEFAULT_RESERVE_MAX_OUTPUT_TOKENS = 32_768;
@@ -0,0 +1,8 @@
1
+ import { readFileSync } from 'node:fs';
2
+
3
+ const packageJson = JSON.parse(
4
+ readFileSync(new URL('../package.json', import.meta.url), 'utf8'),
5
+ );
6
+
7
+ export const PRIVATE_MODE_PROXY_VERSION = String(packageJson.version);
8
+ export const PRIVATE_MODE_PROXY_VERSION_HEADER = 'x-nanogpt-private-proxy-version';
@@ -14,8 +14,7 @@ const MAX_TOKEN_ALIAS_FIELDS = [
14
14
  ];
15
15
  const KIMI_K3_STANDARD_DEFAULT_MAX_COMPLETION_TOKENS = 16_384;
16
16
  const KIMI_K3_DEFAULT_MAX_COMPLETION_TOKENS = 65_536;
17
- const KIMI_K3_MAX_COMPLETION_TOKENS = 1_048_576;
18
- const KIMI_K3_CONTEXT_WINDOW_TOKENS = 1_048_576;
17
+ const KIMI_K3_PRIVATE_CONTEXT_WINDOW_TOKENS = 256_000;
19
18
  const KIMI_K3_CONTEXT_SAFETY_MARGIN_TOKENS = 10;
20
19
  const KIMI_K3_ESTIMATED_TEXT_BYTES_PER_TOKEN = 3;
21
20
  const KIMI_K3_HIGH_ENTROPY_TEXT_MIN_LENGTH = 2_048;
@@ -549,8 +548,18 @@ function resolvePrivateModeKimiK3ReasoningEffort(body) {
549
548
  return excludesReasoning ? normalizedEffort ?? 'low' : normalizedEffort ?? 'max';
550
549
  }
551
550
 
552
- function applyPrivateModeKimiK3RequestParams(body) {
551
+ function applyPrivateModeKimiK3RequestParams(body, model) {
553
552
  const reasoningEffort = resolvePrivateModeKimiK3ReasoningEffort(body);
553
+ const contextWindowTokens = typeof model.maxInputTokens === 'number'
554
+ && Number.isFinite(model.maxInputTokens)
555
+ && model.maxInputTokens > 0
556
+ ? Math.floor(model.maxInputTokens)
557
+ : KIMI_K3_PRIVATE_CONTEXT_WINDOW_TOKENS;
558
+ const maxCompletionTokens = typeof model.maxOutputTokens === 'number'
559
+ && Number.isFinite(model.maxOutputTokens)
560
+ && model.maxOutputTokens > 0
561
+ ? Math.floor(model.maxOutputTokens)
562
+ : contextWindowTokens;
554
563
  const requestedMaxTokens = typeof body.max_tokens === 'number' && Number.isFinite(body.max_tokens)
555
564
  ? body.max_tokens
556
565
  : undefined;
@@ -559,13 +568,13 @@ function applyPrivateModeKimiK3RequestParams(body) {
559
568
  ? KIMI_K3_DEFAULT_MAX_COMPLETION_TOKENS
560
569
  : KIMI_K3_STANDARD_DEFAULT_MAX_COMPLETION_TOKENS
561
570
  : requestedMaxTokens < 0
562
- ? KIMI_K3_MAX_COMPLETION_TOKENS
571
+ ? maxCompletionTokens
563
572
  : requestedMaxTokens;
564
573
  normalizePrivateModeKimiK3NamedToolChoice(body);
565
574
  const promptTokenEstimate = estimatePrivateModeKimiK3PromptTokens(body);
566
575
  const remainingContext = Math.max(
567
576
  1,
568
- KIMI_K3_CONTEXT_WINDOW_TOKENS -
577
+ contextWindowTokens -
569
578
  promptTokenEstimate -
570
579
  KIMI_K3_CONTEXT_SAFETY_MARGIN_TOKENS,
571
580
  );
@@ -670,13 +679,128 @@ export function createPrivateModeReasoningContentSuppressor() {
670
679
  }
671
680
 
672
681
  export function splitCompleteSseFrames(buffer) {
673
- const parts = buffer.split(/\r?\n\r?\n/);
682
+ const parts = buffer.split(/(?:\r\n|\r|\n){2}/);
674
683
  return {
675
- frames: parts.slice(0, -1).map((frame) => frame.replace(/\r\n/g, '\n')),
684
+ frames: parts.slice(0, -1).map((frame) => frame.replace(/\r\n|\r/g, '\n')),
676
685
  remainder: parts.at(-1) || '',
677
686
  };
678
687
  }
679
688
 
689
+ export function createPrivateModeSseTerminalState(expectedChoiceCount = 1) {
690
+ return {
691
+ expectedChoiceCount: Number.isSafeInteger(expectedChoiceCount) && expectedChoiceCount > 0
692
+ ? expectedChoiceCount
693
+ : 1,
694
+ finishedChoiceIndexes: new Set(),
695
+ sawDone: false,
696
+ };
697
+ }
698
+
699
+ function readPrivateModeSseDataPayload(frame) {
700
+ if (typeof frame !== 'string') return '';
701
+ return frame
702
+ .split('\n')
703
+ .filter((line) => line.startsWith('data:'))
704
+ .map((line) => line.slice('data:'.length).trim())
705
+ .join('\n')
706
+ .trim();
707
+ }
708
+
709
+ export function observePrivateModeSseTerminalFrame(state, frame) {
710
+ if (!isPlainObject(state) || typeof frame !== 'string') return state;
711
+ const payload = readPrivateModeSseDataPayload(frame);
712
+ if (!payload) return state;
713
+ if (payload === '[DONE]') {
714
+ state.sawDone = true;
715
+ return state;
716
+ }
717
+
718
+ try {
719
+ const parsed = JSON.parse(payload);
720
+ if (isPlainObject(parsed) && Array.isArray(parsed.choices)) {
721
+ for (const choice of parsed.choices) {
722
+ if (
723
+ !isPlainObject(choice) ||
724
+ choice.finish_reason === undefined ||
725
+ choice.finish_reason === null
726
+ ) continue;
727
+ const choiceIndex = Number.isSafeInteger(choice.index)
728
+ ? choice.index
729
+ : state.expectedChoiceCount === 1
730
+ ? 0
731
+ : null;
732
+ if (choiceIndex !== null && choiceIndex >= 0 && choiceIndex < state.expectedChoiceCount) {
733
+ state.finishedChoiceIndexes.add(choiceIndex);
734
+ }
735
+ }
736
+ }
737
+ } catch {
738
+ // Other SSE payloads do not contribute to the OpenAI terminal sequence.
739
+ }
740
+ return state;
741
+ }
742
+
743
+ export function isPrivateModeSseTerminalStateComplete(state) {
744
+ return state?.sawDone === true &&
745
+ state?.finishedChoiceIndexes instanceof Set &&
746
+ state.finishedChoiceIndexes.size === state.expectedChoiceCount;
747
+ }
748
+
749
+ export function createPrivateModeSseTerminalGuard(expectedChoiceCount = 1) {
750
+ const state = createPrivateModeSseTerminalState(expectedChoiceCount);
751
+ let emittedDone = false;
752
+ return {
753
+ state,
754
+ observe(frame, forwardedFrame = frame) {
755
+ const doneAlreadyEmitted = emittedDone;
756
+ const containsDone = readPrivateModeSseDataPayload(frame) === '[DONE]';
757
+ observePrivateModeSseTerminalFrame(state, frame);
758
+ // The pump drains transport bytes after client-visible completion so the
759
+ // hosted request can finish settlement. Nothing from that drain belongs
760
+ // on the client side of the canonical terminal marker.
761
+ if (doneAlreadyEmitted) return '';
762
+ // Drop every raw DONE marker so duplicates collapse to one canonical
763
+ // marker. Reasoning suppression may prepend a synthetic visible-content
764
+ // delta, which must remain ahead of that marker.
765
+ const forwardedWithoutDone = containsDone
766
+ ? forwardedFrame
767
+ .split(/(?:\r\n|\r|\n){2}/)
768
+ .filter((forwarded) => readPrivateModeSseDataPayload(forwarded) !== '[DONE]')
769
+ .join('\n\n')
770
+ : forwardedFrame;
771
+
772
+ // A hosted stream can keep transport EOF open while billing settles.
773
+ // Release successful completion as soon as the full SSE terminal
774
+ // sequence is observed, then let the response pump drain the transport.
775
+ if (!emittedDone && isPrivateModeSseTerminalStateComplete(state)) {
776
+ emittedDone = true;
777
+ return [forwardedWithoutDone, 'data: [DONE]'].filter(Boolean).join('\n\n');
778
+ }
779
+ return forwardedWithoutDone;
780
+ },
781
+ hasEmittedDone() {
782
+ return emittedDone;
783
+ },
784
+ finish() {
785
+ if (isPrivateModeSseTerminalStateComplete(state)) {
786
+ if (emittedDone) return { complete: true, frame: '' };
787
+ emittedDone = true;
788
+ return { complete: true, frame: 'data: [DONE]' };
789
+ }
790
+ return {
791
+ complete: false,
792
+ frame: `data: ${JSON.stringify({
793
+ error: {
794
+ message: 'Private Mode stream ended before a complete terminal sequence. The response may be partial; retry the request.',
795
+ type: 'api_error',
796
+ code: 'private_stream_incomplete',
797
+ },
798
+ })}`,
799
+ };
800
+ },
801
+ };
802
+ }
803
+
680
804
  const PRIVATE_MODE_SSE_CHUNK_METADATA_FIELDS = [
681
805
  'id',
682
806
  'object',
@@ -899,7 +1023,7 @@ export function applyPrivateModelRequestMutations(body, model) {
899
1023
  normalizeStreamOptions(body);
900
1024
  applyTinfoilCompatibilityMutations(body, model);
901
1025
  if (isPrivateModeKimiK3Model(model)) {
902
- applyPrivateModeKimiK3RequestParams(body);
1026
+ applyPrivateModeKimiK3RequestParams(body, model);
903
1027
  } else if (model.thinkingMode === 'deepseek-v4') {
904
1028
  clampPrivateModeDeepSeekV4Output(body);
905
1029
  }
@@ -1,5 +1,40 @@
1
1
  const DEFAULT_SECURE_CLIENT_IDLE_RESET_MS = 5 * 60 * 1000;
2
+ const DEFAULT_VERIFICATION_MAX_AGE_MS = 5 * 60 * 1000;
2
3
  const EHBP_RESPONSE_NONCE_HEADER = 'ehbp-response-nonce';
4
+ const REQUIRED_VERIFICATION_STEPS = [
5
+ 'fetchDigest',
6
+ 'verifyCode',
7
+ 'verifyEnclave',
8
+ 'compareMeasurements',
9
+ ];
10
+
11
+ function isNonEmptyString(value) {
12
+ return typeof value === 'string' && value.trim().length > 0;
13
+ }
14
+
15
+ function hasCompleteVerificationEvidence(document) {
16
+ return document?.securityVerified === true
17
+ && isNonEmptyString(document.codeFingerprint)
18
+ && isNonEmptyString(document.enclaveFingerprint)
19
+ && isNonEmptyString(document.releaseDigest)
20
+ && isNonEmptyString(document.hpkePublicKey)
21
+ && REQUIRED_VERIFICATION_STEPS.every(
22
+ (step) => document.steps?.[step]?.status === 'success',
23
+ )
24
+ && !Object.values(document.steps || {}).some((step) => step?.status === 'failed');
25
+ }
26
+
27
+ function isMeasurementMismatchError(error) {
28
+ let current = error;
29
+ for (let depth = 0; depth < 5 && current; depth += 1) {
30
+ const message = String(current.message || current);
31
+ if (/\b(?:code\s+)?measurement mismatch\b|\bcannot compare measurements\b/i.test(message)) {
32
+ return true;
33
+ }
34
+ current = current.cause;
35
+ }
36
+ return false;
37
+ }
3
38
 
4
39
  function readSecureClientIdleResetMs() {
5
40
  const configured = process.env.NANOGPT_PRIVATE_CLIENT_IDLE_RESET_MS?.trim();
@@ -14,9 +49,14 @@ export function createSecureState(apiBase, options = {}) {
14
49
  : readSecureClientIdleResetMs();
15
50
  const loadTinfoil = options.loadTinfoil || (() => import('tinfoil'));
16
51
  const now = options.now || Date.now;
52
+ const verificationMaxAgeMs = Number.isFinite(options.verificationMaxAgeMs)
53
+ && options.verificationMaxAgeMs >= 0
54
+ ? Math.floor(options.verificationMaxAgeMs)
55
+ : DEFAULT_VERIFICATION_MAX_AGE_MS;
17
56
  let clientState = null;
18
57
  let verificationDocument = null;
19
58
  let verificationError = null;
59
+ let verifiedAtMs = null;
20
60
 
21
61
  async function getClient(userCacheSecret = clientState?.userCacheSecret) {
22
62
  if (!userCacheSecret) {
@@ -24,12 +64,18 @@ export function createSecureState(apiBase, options = {}) {
24
64
  }
25
65
  const nowMs = now();
26
66
  const idleMs = clientState ? nowMs - clientState.lastUsedAtMs : 0;
67
+ const verificationAgeMs = verifiedAtMs === null ? 0 : nowMs - verifiedAtMs;
27
68
  if (
28
69
  clientState?.userCacheSecret === userCacheSecret
29
70
  && (
30
71
  idleResetMs === 0
31
72
  || (idleMs >= 0 && idleMs < idleResetMs)
32
73
  )
74
+ && (
75
+ verifiedAtMs === null
76
+ || verificationMaxAgeMs === 0
77
+ || (verificationAgeMs >= 0 && verificationAgeMs < verificationMaxAgeMs)
78
+ )
33
79
  ) {
34
80
  clientState.lastUsedAtMs = nowMs;
35
81
  return clientState.promise;
@@ -40,6 +86,7 @@ export function createSecureState(apiBase, options = {}) {
40
86
  // still be finishing an older in-flight request.
41
87
  verificationDocument = null;
42
88
  verificationError = null;
89
+ verifiedAtMs = null;
43
90
  const nextState = {
44
91
  client: null,
45
92
  lastUsedAtMs: nowMs,
@@ -54,10 +101,19 @@ export function createSecureState(apiBase, options = {}) {
54
101
  transport: 'ehbp',
55
102
  userCacheSecret,
56
103
  });
57
- await client.ready();
58
104
  nextState.client = client;
105
+ try {
106
+ await client.ready();
107
+ } finally {
108
+ if (clientState === nextState) {
109
+ verificationDocument = client.getVerificationDocument();
110
+ }
111
+ }
112
+ if (!hasCompleteVerificationEvidence(verificationDocument)) {
113
+ throw new Error('Private Mode attestation evidence was incomplete.');
114
+ }
59
115
  if (clientState === nextState) {
60
- verificationDocument = client.getVerificationDocument();
116
+ verifiedAtMs = now();
61
117
  verificationError = null;
62
118
  }
63
119
  return client;
@@ -80,7 +136,23 @@ export function createSecureState(apiBase, options = {}) {
80
136
  getClient,
81
137
  markClientUsed(client) {
82
138
  if (!clientState || clientState.client !== client) return false;
83
- clientState.lastUsedAtMs = now();
139
+ const nowMs = now();
140
+ const currentDocument = client.getVerificationDocument();
141
+ if (currentDocument !== verificationDocument) {
142
+ verificationDocument = currentDocument;
143
+ if (hasCompleteVerificationEvidence(currentDocument)) {
144
+ verifiedAtMs = nowMs;
145
+ verificationError = null;
146
+ } else {
147
+ verifiedAtMs = null;
148
+ // Preserve the incomplete document for status visibility, but never
149
+ // reuse a transport after the SDK has invalidated its evidence.
150
+ clientState.lastUsedAtMs = nowMs;
151
+ clientState = null;
152
+ return true;
153
+ }
154
+ }
155
+ clientState.lastUsedAtMs = nowMs;
84
156
  return true;
85
157
  },
86
158
  invalidateClient(client) {
@@ -89,11 +161,49 @@ export function createSecureState(apiBase, options = {}) {
89
161
  clientState = null;
90
162
  verificationDocument = null;
91
163
  verificationError = null;
164
+ verifiedAtMs = null;
92
165
  return true;
93
166
  },
94
167
  getVerificationState() {
168
+ const nowMs = now();
169
+ // Tinfoil 1.1.12 resets its document to "pending" after an exhausted
170
+ // AttestationError retry, so retain the SDK's explicit mismatch signal.
171
+ const mismatch = verificationDocument?.steps?.compareMeasurements?.status === 'failed'
172
+ || isMeasurementMismatchError(verificationError);
173
+ const verified = hasCompleteVerificationEvidence(verificationDocument)
174
+ && !mismatch
175
+ && verifiedAtMs !== null;
176
+ const verificationAgeMs = verifiedAtMs === null ? null : nowMs - verifiedAtMs;
177
+ const clockRolledBack = verificationAgeMs !== null && verificationAgeMs < 0;
178
+ const expiresAtMs = verifiedAtMs === null || verificationMaxAgeMs === 0
179
+ ? null
180
+ : verifiedAtMs + verificationMaxAgeMs;
181
+ const isStale = verified && (
182
+ clockRolledBack
183
+ || (expiresAtMs !== null && nowMs >= expiresAtMs)
184
+ );
185
+ const freshnessSeconds = expiresAtMs === null
186
+ ? null
187
+ : clockRolledBack
188
+ ? 0
189
+ : Math.max(0, Math.floor((expiresAtMs - nowMs) / 1000));
190
+ const verificationStatus = mismatch
191
+ ? 'measurement_mismatch'
192
+ : verified && verifiedAtMs !== null
193
+ ? isStale
194
+ ? 'stale'
195
+ : 'verified'
196
+ : verificationError || verificationDocument
197
+ ? 'verification_failed'
198
+ : 'not_checked';
95
199
  return {
96
- verified: verificationDocument?.securityVerified === true,
200
+ verified,
201
+ verificationStatus,
202
+ verifiedAt: verifiedAtMs === null ? null : new Date(verifiedAtMs).toISOString(),
203
+ expiresAt: expiresAtMs === null
204
+ ? null
205
+ : new Date(expiresAtMs).toISOString(),
206
+ freshnessSeconds,
97
207
  verificationDocument,
98
208
  error: verificationError ? String(verificationError.message || verificationError) : null,
99
209
  };
package/lib/server.js CHANGED
@@ -2,28 +2,45 @@ import { createServer } from 'node:http';
2
2
  import { readFileSync } from 'node:fs';
3
3
 
4
4
  import { buildPrivateModeCacheScopeProof } from './cacheScope.js';
5
+ import {
6
+ buildPrivateModeCatalogMismatchMessage,
7
+ buildPrivateModeCatalogUnavailableMessage,
8
+ checkHostedPrivateModeCatalog,
9
+ hasPrivateModeCatalogMismatch,
10
+ } from './catalogCompatibility.js';
11
+ import {
12
+ PRIVATE_MODE_DEFAULT_RESERVE_MAX_OUTPUT_TOKENS,
13
+ PRIVATE_MODE_STREAM_SOFT_DEADLINE_MS,
14
+ } from './constants.js';
5
15
  import {
6
16
  buildPrivateModeOriginPolicy,
7
17
  getCorsHeadersForRequest,
8
18
  } from './originPolicy.js';
9
19
  import {
10
20
  applyPrivateModelRequestMutations,
11
- buildPrivateModeSseContentDelta,
12
- createPrivateModeReasoningContentSuppressor,
13
21
  shouldSuppressPrivateModelReasoning,
14
- splitCompleteSseFrames,
15
- suppressPrivateModeReasoningFromSseFrame,
16
22
  suppressPrivateModeReasoningFromJsonText,
17
23
  } from './requestTransforms.js';
18
24
  import {
19
25
  normalizePrivateModeUpstreamErrorMessage,
20
26
  readErrorMessage,
21
27
  } from './serverErrorNormalization.js';
28
+ import {
29
+ PRIVATE_MODE_PROXY_VERSION,
30
+ PRIVATE_MODE_PROXY_VERSION_HEADER,
31
+ } from './packageInfo.js';
22
32
  import {
23
33
  createSecureState,
24
34
  fetchWithSecureClientRecovery,
25
35
  } from './secureClientLifecycle.js';
26
- import { buildPrivateModeStatusContract } from './statusContract.js';
36
+ import {
37
+ buildPrivateModeStatusContract,
38
+ buildPublicAttestationSummary,
39
+ } from './statusContract.js';
40
+ import {
41
+ isPrivateModeSseResponse,
42
+ pipePrivateModeSseResponse,
43
+ } from './sseResponsePump.js';
27
44
 
28
45
  const MODELS = JSON.parse(
29
46
  readFileSync(new URL('../models/private-tee.json', import.meta.url), 'utf8'),
@@ -76,15 +93,18 @@ function openAIModelList() {
76
93
  object: 'model',
77
94
  created: model.created,
78
95
  owned_by: model.ownedBy,
96
+ ...(model.maxInputTokens ? { context_length: model.maxInputTokens } : {}),
79
97
  ...(model.maxOutputTokens ? { max_output_tokens: model.maxOutputTokens } : {}),
80
98
  })),
81
99
  };
82
100
  }
83
101
 
84
- export function privateModeStatus(apiBase, secureState, localBase, originPolicy) {
102
+ export function privateModeStatus(apiBase, secureState, localBase, originPolicy, catalogCompatibility) {
103
+ const verificationState = secureState.getVerificationState();
85
104
  return {
86
105
  ok: true,
87
106
  mode: 'private_tee',
107
+ proxy_version: PRIVATE_MODE_PROXY_VERSION,
88
108
  apiBase,
89
109
  local_base_url: localBase,
90
110
  models_path: '/v1/models',
@@ -97,12 +117,19 @@ export function privateModeStatus(apiBase, secureState, localBase, originPolicy)
97
117
  response_body_encrypted: true,
98
118
  streaming: true,
99
119
  streaming_billing: 'precharged_reserve_with_verified_usage_refund',
120
+ // This deadline is enforced by the hosted route, not this process. A
121
+ // local environment override cannot configure the remote service.
122
+ streaming_soft_deadline_ms: PRIVATE_MODE_STREAM_SOFT_DEADLINE_MS,
123
+ default_stream_reserve_max_output_tokens: PRIVATE_MODE_DEFAULT_RESERVE_MAX_OUTPUT_TOKENS,
124
+ responses_api_supported: false,
125
+ batch_api_supported: false,
100
126
  api_local_proxy_required: true,
101
127
  browser_frontend_local_proxy_required: false,
102
- ...buildPrivateModeStatusContract(),
128
+ ...buildPrivateModeStatusContract(verificationState),
103
129
  browser_origins_allowed: originPolicy.allowedOrigins,
104
130
  models: openAIModelList().data,
105
- attestation: secureState.getVerificationState(),
131
+ catalog_compatibility: catalogCompatibility,
132
+ attestation: buildPublicAttestationSummary(verificationState),
106
133
  };
107
134
  }
108
135
 
@@ -174,6 +201,7 @@ async function runPreflight({ apiBase, apiKey, model, req, requestBodyBytes }) {
174
201
  headers: {
175
202
  authorization: `Bearer ${apiKey}`,
176
203
  'content-type': 'application/json',
204
+ [PRIVATE_MODE_PROXY_VERSION_HEADER]: PRIVATE_MODE_PROXY_VERSION,
177
205
  ...copyLocalHeaders(req),
178
206
  },
179
207
  body: JSON.stringify(preflightBody),
@@ -265,7 +293,7 @@ async function handleChatCompletion({ apiBase, apiKey, secureState, req, res, co
265
293
  if (!model) {
266
294
  jsonResponse(res, 400, {
267
295
  error: {
268
- message: `Unsupported private model "${body.model || ''}".`,
296
+ message: `Unsupported private model "${body.model || ''}" in @nanogpt/private-mode v${PRIVATE_MODE_PROXY_VERSION}. Check this proxy's /v1/models endpoint or restart with: NANOGPT_API_KEY=... npx --yes @nanogpt/private-mode@latest`,
269
297
  type: 'invalid_request_error',
270
298
  code: 'model_not_supported',
271
299
  },
@@ -276,6 +304,7 @@ async function handleChatCompletion({ apiBase, apiKey, secureState, req, res, co
276
304
  const suppressReasoning = shouldSuppressPrivateModelReasoning(body, model);
277
305
  applyPrivateModelRequestMutations(body, model);
278
306
  const privateStreamRequested = body.stream === true;
307
+ const expectedChoiceCount = Number.isSafeInteger(body.n) && body.n > 0 ? body.n : 1;
279
308
 
280
309
  const privateRequestBody = JSON.stringify(body);
281
310
  const preflight = await runPreflight({
@@ -312,6 +341,7 @@ async function handleChatCompletion({ apiBase, apiKey, secureState, req, res, co
312
341
  'x-nanogpt-private-model': model.id,
313
342
  'x-nanogpt-private-stream': privateStreamRequested ? 'true' : 'false',
314
343
  'x-nanogpt-private-cache-scope': buildPrivateModeCacheScopeProof(preflight.cacheScope),
344
+ [PRIVATE_MODE_PROXY_VERSION_HEADER]: PRIVATE_MODE_PROXY_VERSION,
315
345
  'x-query-source': 'api',
316
346
  ...copyLocalHeaders(req),
317
347
  },
@@ -360,59 +390,30 @@ async function handleChatCompletion({ apiBase, apiKey, secureState, req, res, co
360
390
  return;
361
391
  }
362
392
 
363
- if (privateStreamRequested && response.body) {
393
+ if (isPrivateModeSseResponse(privateStreamRequested, response)) {
364
394
  res.writeHead(response.status, headers);
365
- const reader = response.body.getReader();
366
- const decoder = suppressReasoning ? new TextDecoder() : null;
367
- const contentSuppressor = suppressReasoning
368
- ? createPrivateModeReasoningContentSuppressor()
369
- : null;
370
- const sseChunkMetadata = {};
371
- let sseBuffer = '';
372
- let streamFailed = false;
373
395
  try {
374
- while (true) {
375
- const { done, value } = await reader.read();
376
- if (done) break;
377
- if (res.destroyed) {
378
- await reader.cancel('client disconnected');
379
- upstreamAbortController.abort();
380
- return;
381
- }
382
- if (!value) continue;
383
- if (!suppressReasoning || !decoder || !contentSuppressor) {
384
- res.write(Buffer.from(value));
385
- continue;
386
- }
387
- sseBuffer += decoder.decode(value, { stream: true });
388
- const { frames, remainder } = splitCompleteSseFrames(sseBuffer);
389
- sseBuffer = remainder;
390
- for (const frame of frames) {
391
- res.write(`${suppressPrivateModeReasoningFromSseFrame(frame, contentSuppressor, sseChunkMetadata)}\n\n`);
392
- }
393
- }
394
- if (suppressReasoning && decoder && contentSuppressor) {
395
- sseBuffer += decoder.decode();
396
- if (sseBuffer) {
397
- res.write(`${suppressPrivateModeReasoningFromSseFrame(sseBuffer, contentSuppressor, sseChunkMetadata)}\n\n`);
398
- }
399
- const flushedContent = contentSuppressor.flush();
400
- if (flushedContent) {
401
- const finalDelta = buildPrivateModeSseContentDelta(flushedContent, sseChunkMetadata);
402
- res.write(`data: ${finalDelta}\n\n`);
403
- }
396
+ const outcome = await pipePrivateModeSseResponse({
397
+ body: response.body,
398
+ res,
399
+ suppressReasoning,
400
+ upstreamAbortController,
401
+ expectedChoiceCount,
402
+ });
403
+ // Successful completion and downstream disconnect still count as
404
+ // activity. Truncated or transport-failed streams keep the prior failure
405
+ // behavior and do not refresh the secure-client lifecycle.
406
+ if (!outcome.transportFailed && !outcome.streamFailed) {
407
+ secureState.markClientUsed(responseClient);
404
408
  }
405
409
  } catch (error) {
406
- streamFailed = true;
407
410
  upstreamAbortController.abort();
408
411
  if (!res.destroyed) {
409
- res.destroy(error instanceof Error ? error : new Error(String(error)));
412
+ process.stderr.write(`Private Mode stream pump failed: ${error instanceof Error ? error.message : String(error)}\n`);
410
413
  }
411
- return;
412
414
  } finally {
413
415
  responseComplete = true;
414
- if (!streamFailed) secureState.markClientUsed(responseClient);
415
- if (!streamFailed && !res.writableEnded && !res.destroyed) res.end();
416
+ if (!res.writableEnded && !res.destroyed) res.end();
416
417
  }
417
418
  return;
418
419
  }
@@ -468,6 +469,19 @@ function handleOptions(req, res, corsHeaders) {
468
469
 
469
470
  export async function startPrivateModeProxy(options) {
470
471
  const apiBase = normalizeApiBase(options.apiBase);
472
+ const catalogCompatibility = await checkHostedPrivateModeCatalog({
473
+ apiBase,
474
+ localModels: MODELS,
475
+ });
476
+ if (catalogCompatibility.status === 'unavailable') {
477
+ throw new Error(buildPrivateModeCatalogUnavailableMessage(catalogCompatibility, apiBase));
478
+ }
479
+ if (hasPrivateModeCatalogMismatch(catalogCompatibility)) {
480
+ throw new Error(buildPrivateModeCatalogMismatchMessage(
481
+ catalogCompatibility,
482
+ PRIVATE_MODE_PROXY_VERSION,
483
+ ));
484
+ }
471
485
  const secureState = createSecureState(apiBase);
472
486
  const localBase = `http://${options.host}:${options.port}/v1`;
473
487
  const originPolicy = buildPrivateModeOriginPolicy(options);
@@ -493,7 +507,7 @@ export async function startPrivateModeProxy(options) {
493
507
  }
494
508
 
495
509
  if (req.method === 'GET' && url.pathname === '/health') {
496
- jsonResponse(res, 200, privateModeStatus(apiBase, secureState, localBase, originPolicy), corsHeaders);
510
+ jsonResponse(res, 200, privateModeStatus(apiBase, secureState, localBase, originPolicy, catalogCompatibility), corsHeaders);
497
511
  return;
498
512
  }
499
513
 
@@ -503,7 +517,7 @@ export async function startPrivateModeProxy(options) {
503
517
  }
504
518
 
505
519
  if (req.method === 'GET' && url.pathname === '/v1/private-mode/status') {
506
- jsonResponse(res, 200, privateModeStatus(apiBase, secureState, localBase, originPolicy), corsHeaders);
520
+ jsonResponse(res, 200, privateModeStatus(apiBase, secureState, localBase, originPolicy, catalogCompatibility), corsHeaders);
507
521
  return;
508
522
  }
509
523
 
@@ -539,7 +553,7 @@ export async function startPrivateModeProxy(options) {
539
553
  }
540
554
 
541
555
  if (req.method === 'GET' && url.pathname === '/') {
542
- textResponse(res, 200, `NanoGPT Private Mode Proxy\nOpenAI base URL: ${localBase}\nStatus: /v1/private-mode/status\n`, corsHeaders);
556
+ textResponse(res, 200, `NanoGPT Private Mode Proxy v${PRIVATE_MODE_PROXY_VERSION}\nOpenAI base URL: ${localBase}\nStatus: /v1/private-mode/status\n`, corsHeaders);
543
557
  return;
544
558
  }
545
559
 
@@ -567,6 +581,7 @@ export async function startPrivateModeProxy(options) {
567
581
 
568
582
  if (options.quiet !== true) {
569
583
  process.stdout.write(`NanoGPT Private Mode Proxy
584
+ Version: ${PRIVATE_MODE_PROXY_VERSION}
570
585
  Local base URL: ${localBase}
571
586
  NanoGPT API: ${apiBase}
572
587
  Models: ${MODELS.map((model) => model.id).join(', ')}
@@ -0,0 +1,163 @@
1
+ import {
2
+ buildPrivateModeSseContentDelta,
3
+ createPrivateModeReasoningContentSuppressor,
4
+ createPrivateModeSseTerminalGuard,
5
+ splitCompleteSseFrames,
6
+ suppressPrivateModeReasoningFromSseFrame,
7
+ } from './requestTransforms.js';
8
+
9
+ const MAX_SSE_FRAME_BYTES = 2 * 1024 * 1024;
10
+ const DEFAULT_DOWNSTREAM_DRAIN_TIMEOUT_MS = 60 * 1000;
11
+
12
+ function assertSseFrameWithinLimit(frame) {
13
+ if (Buffer.byteLength(frame, 'utf8') > MAX_SSE_FRAME_BYTES) {
14
+ throw new Error('Private Mode upstream SSE frame exceeded the proxy buffer limit.');
15
+ }
16
+ }
17
+
18
+ export function isPrivateModeSseResponse(streamRequested, response) {
19
+ const contentType = response?.headers?.get?.('content-type') || '';
20
+ return streamRequested === true
21
+ && Boolean(response?.body)
22
+ && contentType.toLowerCase().includes('text/event-stream');
23
+ }
24
+
25
+ function writeWithBackpressure(res, chunk, timeoutMs) {
26
+ if (res.destroyed || res.writableEnded) return Promise.resolve(false);
27
+ try {
28
+ if (res.write(chunk)) return Promise.resolve(true);
29
+ } catch {
30
+ return Promise.resolve(false);
31
+ }
32
+
33
+ return new Promise((resolve) => {
34
+ let drainTimer;
35
+ const cleanup = () => {
36
+ if (drainTimer) clearTimeout(drainTimer);
37
+ res.off('drain', onDrain);
38
+ res.off('close', onClose);
39
+ res.off('error', onError);
40
+ };
41
+ const settle = (writable) => {
42
+ cleanup();
43
+ resolve(writable);
44
+ };
45
+ const onDrain = () => settle(!res.destroyed && !res.writableEnded);
46
+ const onClose = () => settle(false);
47
+ const onError = () => settle(false);
48
+ res.once('drain', onDrain);
49
+ res.once('close', onClose);
50
+ res.once('error', onError);
51
+ drainTimer = setTimeout(() => settle(false), timeoutMs);
52
+ drainTimer.unref?.();
53
+ });
54
+ }
55
+
56
+ export async function pipePrivateModeSseResponse({
57
+ body,
58
+ res,
59
+ suppressReasoning,
60
+ upstreamAbortController,
61
+ expectedChoiceCount = 1,
62
+ downstreamDrainTimeoutMs = DEFAULT_DOWNSTREAM_DRAIN_TIMEOUT_MS,
63
+ }) {
64
+ let reader;
65
+ const decoder = new TextDecoder();
66
+ const contentSuppressor = suppressReasoning
67
+ ? createPrivateModeReasoningContentSuppressor()
68
+ : null;
69
+ const terminalGuard = createPrivateModeSseTerminalGuard(expectedChoiceCount);
70
+ const sseChunkMetadata = {};
71
+ let sseBuffer = '';
72
+ let transportFailed = false;
73
+
74
+ const writeFrame = async (frame) => {
75
+ if (!frame) return true;
76
+ return writeWithBackpressure(res, `${frame}\n\n`, downstreamDrainTimeoutMs);
77
+ };
78
+ const forwardFrame = async (frame) => {
79
+ const forwarded = suppressReasoning && contentSuppressor
80
+ ? suppressPrivateModeReasoningFromSseFrame(frame, contentSuppressor, sseChunkMetadata)
81
+ : frame;
82
+ return writeFrame(terminalGuard.observe(frame, forwarded));
83
+ };
84
+ const finishStream = async () => {
85
+ // A complete terminal sequence may be released before transport EOF.
86
+ // Never append a defensive suppressor flush after that marker.
87
+ if (contentSuppressor && !terminalGuard.hasEmittedDone()) {
88
+ const flushedContent = contentSuppressor.flush();
89
+ if (flushedContent) {
90
+ const delta = buildPrivateModeSseContentDelta(flushedContent, sseChunkMetadata);
91
+ if (!await writeFrame(`data: ${delta}`)) return null;
92
+ }
93
+ }
94
+ const terminal = terminalGuard.finish();
95
+ if (!await writeFrame(terminal.frame)) return null;
96
+ return terminal;
97
+ };
98
+
99
+ try {
100
+ reader = body.getReader();
101
+ while (true) {
102
+ const { done, value } = await reader.read();
103
+ if (done) break;
104
+ if (res.destroyed || res.writableEnded) {
105
+ await reader.cancel('client disconnected').catch(() => {});
106
+ upstreamAbortController.abort();
107
+ return { streamFailed: false, clientDisconnected: true, transportFailed: false };
108
+ }
109
+ if (!value) continue;
110
+ sseBuffer += decoder.decode(value, { stream: true });
111
+ const { frames, remainder } = splitCompleteSseFrames(sseBuffer);
112
+ sseBuffer = remainder;
113
+ assertSseFrameWithinLimit(sseBuffer);
114
+ for (const frame of frames) {
115
+ assertSseFrameWithinLimit(frame);
116
+ if (!await forwardFrame(frame)) {
117
+ await reader.cancel('client disconnected').catch(() => {});
118
+ upstreamAbortController.abort();
119
+ return { streamFailed: false, clientDisconnected: true, transportFailed: false };
120
+ }
121
+ }
122
+ }
123
+ sseBuffer += decoder.decode();
124
+ if (sseBuffer && !await forwardFrame(sseBuffer)) {
125
+ await reader.cancel('client disconnected').catch(() => {});
126
+ upstreamAbortController.abort();
127
+ return { streamFailed: false, clientDisconnected: true, transportFailed: false };
128
+ }
129
+ } catch (error) {
130
+ transportFailed = true;
131
+ upstreamAbortController.abort();
132
+ process.stderr.write(`Private Mode upstream stream failed: ${error instanceof Error ? error.message : String(error)}\n`);
133
+ }
134
+
135
+ if (res.destroyed || res.writableEnded) {
136
+ upstreamAbortController.abort();
137
+ return { streamFailed: false, clientDisconnected: true, transportFailed };
138
+ }
139
+
140
+ let terminal;
141
+ try {
142
+ terminal = await finishStream();
143
+ } catch (error) {
144
+ transportFailed = true;
145
+ upstreamAbortController.abort();
146
+ process.stderr.write(`Private Mode stream finalization failed: ${error instanceof Error ? error.message : String(error)}\n`);
147
+ if (!terminalGuard.hasEmittedDone()) {
148
+ await writeFrame(`data: ${JSON.stringify({
149
+ error: {
150
+ message: 'Private Mode stream ended before a complete terminal sequence. The response may be partial; retry the request.',
151
+ type: 'api_error',
152
+ code: 'private_stream_incomplete',
153
+ },
154
+ })}`);
155
+ }
156
+ return { streamFailed: true, clientDisconnected: false, transportFailed };
157
+ }
158
+ if (!terminal) {
159
+ upstreamAbortController.abort();
160
+ return { streamFailed: false, clientDisconnected: true, transportFailed };
161
+ }
162
+ return { streamFailed: !terminal.complete, clientDisconnected: false, transportFailed };
163
+ }
@@ -1,3 +1,5 @@
1
+ import { createHash } from 'node:crypto';
2
+
1
3
  export const PRIVATE_MODE_FRONTEND_SUPPORTED_FEATURES = Object.freeze([
2
4
  'text_chat',
3
5
  'streaming',
@@ -43,12 +45,88 @@ export const PRIVATE_MODE_PLAINTEXT_VISIBLE_TO = Object.freeze([
43
45
  'verified_tee_target',
44
46
  ]);
45
47
 
46
- export function buildPrivateModeStatusContract() {
48
+ export const PRIVATE_MODE_KNOWN_LIMITATIONS = Object.freeze([
49
+ 'NanoGPT still observes account, routing, timing, size, status, and usage metadata.',
50
+ 'Provider-reported usage metrics are trusted for billing and are not end-to-end signed.',
51
+ 'A verification receipt does not prove usage correctness, prompt non-retention, or response content authenticity on its own.',
52
+ 'Verification receipts are browser-generated and unsigned; the hashed request identifier is for opaque correlation, not proof of server issuance.',
53
+ 'NanoGPT does not implement attestation-gated KMS release of upstream credentials; the inference enclave is provider-operated.',
54
+ ]);
55
+
56
+ const PRIVATE_MODE_VERIFICATION_STATUSES = new Set([
57
+ 'verified',
58
+ 'unavailable',
59
+ 'stale',
60
+ 'measurement_mismatch',
61
+ 'verification_failed',
62
+ 'unsupported',
63
+ 'not_checked',
64
+ ]);
65
+
66
+ function stringIdentifier(value) {
67
+ return typeof value === 'string' && value.trim() ? value.trim().slice(0, 512) : undefined;
68
+ }
69
+
70
+ function publicKeyFingerprint(value) {
71
+ return typeof value === 'string' && value
72
+ ? createHash('sha256').update(value).digest('hex')
73
+ : undefined;
74
+ }
75
+
76
+ export function buildPublicAttestationSummary(verificationState = {}) {
77
+ return {
78
+ verified: verificationState.verified === true,
79
+ verificationStatus: verificationState.verificationStatus,
80
+ verifiedAt: verificationState.verifiedAt,
81
+ expiresAt: verificationState.expiresAt,
82
+ freshnessSeconds: verificationState.freshnessSeconds,
83
+ error: verificationState.error ? 'Private Mode attestation failed.' : null,
84
+ };
85
+ }
86
+
87
+ export function buildPrivateModeStatusContract(verificationState = {}) {
88
+ const verificationStatus = PRIVATE_MODE_VERIFICATION_STATUSES.has(verificationState.verificationStatus)
89
+ ? verificationState.verificationStatus
90
+ : 'not_checked';
91
+ const document = verificationState.verificationDocument || {};
92
+ const keyFingerprint = publicKeyFingerprint(document.hpkePublicKey);
47
93
  return {
48
94
  frontend_supported_features: [...PRIVATE_MODE_FRONTEND_SUPPORTED_FEATURES],
49
95
  frontend_disabled_features: [...PRIVATE_MODE_FRONTEND_DISABLED_FEATURES],
50
96
  nanogpt_visible: [...PRIVATE_MODE_NANOGPT_VISIBLE],
51
97
  encrypted_from_nanogpt: [...PRIVATE_MODE_ENCRYPTED_FROM_NANOGPT],
52
98
  plaintext_visible_to: [...PRIVATE_MODE_PLAINTEXT_VISIBLE_TO],
99
+ assurance: {
100
+ contract_version: 1,
101
+ assurance_level: 'private_mode',
102
+ transport: 'ehbp',
103
+ verification_status: verificationStatus,
104
+ eligible: verificationStatus === 'verified',
105
+ verified_by: verificationState.verifiedAt || verificationStatus !== 'not_checked'
106
+ ? ['local_proxy:tinfoil-js']
107
+ : [],
108
+ ...(verificationState.verifiedAt ? { verified_at: verificationState.verifiedAt } : {}),
109
+ ...(verificationState.expiresAt ? { expires_at: verificationState.expiresAt } : {}),
110
+ ...(Number.isFinite(verificationState.freshnessSeconds)
111
+ ? { freshness_seconds: Math.max(0, Math.floor(verificationState.freshnessSeconds)) }
112
+ : {}),
113
+ ...(stringIdentifier(document.codeFingerprint)
114
+ ? { expected_measurement: stringIdentifier(document.codeFingerprint) }
115
+ : {}),
116
+ ...(stringIdentifier(document.enclaveFingerprint)
117
+ ? { observed_measurement: stringIdentifier(document.enclaveFingerprint) }
118
+ : {}),
119
+ ...(stringIdentifier(document.releaseDigest)
120
+ ? { release_digest: stringIdentifier(document.releaseDigest) }
121
+ : {}),
122
+ ...(keyFingerprint ? { public_key_fingerprint: keyFingerprint } : {}),
123
+ receipt_available: true,
124
+ verifier: { name: 'tinfoil-js', version: '1.1.12' },
125
+ documentation_url: 'https://nano-gpt.com/private-mode-verification',
126
+ known_limitations: [...PRIVATE_MODE_KNOWN_LIMITATIONS],
127
+ ...(verificationStatus !== 'verified' && verificationStatus !== 'not_checked'
128
+ ? { error_code: verificationStatus }
129
+ : {}),
130
+ },
53
131
  };
54
132
  }
@@ -2,6 +2,9 @@ import { createHash } from 'node:crypto';
2
2
  import { readFile } from 'node:fs/promises';
3
3
  import { fetchAttestationBundle, Verifier } from 'tinfoil';
4
4
 
5
+ const PRIVATE_MODE_ATTESTATION_MAX_AGE_MS = 5 * 60 * 1000;
6
+ const PRIVATE_MODE_RECEIPT_CLOCK_SKEW_MS = 60 * 1000;
7
+
5
8
  function isRecord(value) {
6
9
  return Boolean(value) && typeof value === 'object' && !Array.isArray(value);
7
10
  }
@@ -84,7 +87,7 @@ function assertReceiptFlag(failures, label, value) {
84
87
  }
85
88
 
86
89
  export async function verifyPrivateModeReceipt(receipt) {
87
- if (!isRecord(receipt) || receipt.schemaVersion !== 1) {
90
+ if (!isRecord(receipt) || (receipt.schemaVersion !== 1 && receipt.schemaVersion !== 2)) {
88
91
  throw new Error('Unsupported or invalid Private Mode receipt.');
89
92
  }
90
93
 
@@ -113,6 +116,29 @@ export async function verifyPrivateModeReceipt(receipt) {
113
116
  if (receipt.status !== 'verified') {
114
117
  failures.push(`Receipt status is ${receipt.status || 'missing'}; expected verified.`);
115
118
  }
119
+ if (receipt.schemaVersion === 2) {
120
+ if (receipt.assuranceLevel !== 'private_mode') {
121
+ failures.push('Receipt assurance level is not private_mode.');
122
+ }
123
+ const verifiedAt = Date.parse(receipt.verifiedAt);
124
+ const expiresAt = Date.parse(receipt.expiresAt);
125
+ const now = Date.now();
126
+ if (
127
+ !Number.isFinite(verifiedAt)
128
+ || !Number.isFinite(expiresAt)
129
+ || expiresAt <= verifiedAt
130
+ || expiresAt - verifiedAt > PRIVATE_MODE_ATTESTATION_MAX_AGE_MS
131
+ || verifiedAt > now + PRIVATE_MODE_RECEIPT_CLOCK_SKEW_MS
132
+ ) {
133
+ failures.push('Receipt freshness timestamps are invalid.');
134
+ } else if (expiresAt <= now) {
135
+ warnings.push('Receipt attestation freshness window has expired; fresh evidence was verified again now.');
136
+ }
137
+ const requestIdSha256 = receipt.response?.requestIdSha256;
138
+ if (requestIdSha256 !== undefined && !/^[a-f0-9]{64}$/i.test(requestIdSha256)) {
139
+ failures.push('Request identifier binding is not a SHA-256 digest.');
140
+ }
141
+ }
116
142
 
117
143
  const encryption = isRecord(receipt.encryption) ? receipt.encryption : {};
118
144
  if (encryption.transport !== 'EHBP') {
@@ -126,6 +152,16 @@ export async function verifyPrivateModeReceipt(receipt) {
126
152
  assertRequiredEqual(failures, 'Runtime measurement', doc?.enclaveFingerprint, receipt.verifier?.enclaveFingerprint);
127
153
  assertRequiredEqual(failures, 'Release digest', doc?.releaseDigest, receipt.verifier?.releaseDigest);
128
154
  assertRequiredEqual(failures, 'HPKE public key', doc?.hpkePublicKey, receipt.verifier?.hpkePublicKey);
155
+ if (receipt.schemaVersion === 2) {
156
+ assertRequiredEqual(
157
+ failures,
158
+ 'HPKE public-key fingerprint',
159
+ isNonEmptyString(doc?.hpkePublicKey)
160
+ ? createHash('sha256').update(doc.hpkePublicKey).digest('hex')
161
+ : undefined,
162
+ receipt.verifier?.hpkePublicKeySha256,
163
+ );
164
+ }
129
165
  assertOptionalEqual(warnings, 'Attestation bundle SHA-256', attestationBundleSha256, receipt.verifier?.attestationBundleSha256);
130
166
 
131
167
  return {
@@ -137,6 +173,7 @@ export async function verifyPrivateModeReceipt(receipt) {
137
173
  enclaveFingerprint: doc?.enclaveFingerprint,
138
174
  attestationBundleSha256,
139
175
  verificationDocumentSha256,
176
+ schemaVersion: receipt.schemaVersion,
140
177
  };
141
178
  }
142
179
 
@@ -155,6 +192,7 @@ export async function verifyPrivateModeReceiptCli(positionals, io = process) {
155
192
  }
156
193
 
157
194
  io.stdout.write('Private Mode receipt verified.\n');
195
+ io.stdout.write(`Receipt schema: v${result.schemaVersion}\n`);
158
196
  io.stdout.write(`Enclave: ${result.enclaveURL}\n`);
159
197
  io.stdout.write(`Expected measurement: ${result.codeFingerprint}\n`);
160
198
  io.stdout.write(`Runtime measurement: ${result.enclaveFingerprint}\n`);
@@ -1,11 +1,13 @@
1
1
  [
2
2
  {
3
3
  "id": "private/deepseek-v4-flash",
4
- "name": "DeepSeek V4 Flash Private",
4
+ "name": "DeepSeek V4 Flash 0731 Private",
5
5
  "upstreamModel": "deepseek-v4-flash",
6
6
  "billingModel": "private/deepseek-v4-flash",
7
+ "providerPricingModel": "TEE/deepseek-v4-flash",
7
8
  "teeTargetModel": "deepseek-v4-flash",
8
9
  "thinkingMode": "deepseek-v4",
10
+ "maxInputTokens": 1048576,
9
11
  "maxOutputTokens": 1048576,
10
12
  "created": 1786406400,
11
13
  "ownedBy": "nanogpt-private-mode",
@@ -16,8 +18,10 @@
16
18
  "name": "Kimi K3 Private",
17
19
  "upstreamModel": "kimi-k3",
18
20
  "billingModel": "TEE/kimi-k3",
21
+ "providerPricingModel": "TEE/kimi-k3",
19
22
  "teeTargetModel": "kimi-k3",
20
- "maxOutputTokens": 1048576,
23
+ "maxInputTokens": 256000,
24
+ "maxOutputTokens": 256000,
21
25
  "created": 1786147200,
22
26
  "ownedBy": "nanogpt-private-mode",
23
27
  "aliases": ["TEE/kimi-k3"]
@@ -27,6 +31,7 @@
27
31
  "name": "GPT OSS 120B Private",
28
32
  "upstreamModel": "gpt-oss-120b",
29
33
  "billingModel": "TEE/gpt-oss-120b",
34
+ "providerPricingModel": "openai/gpt-oss-120b",
30
35
  "teeTargetModel": "gpt-oss-120b",
31
36
  "created": 1764547200,
32
37
  "ownedBy": "nanogpt-private-mode",
@@ -37,6 +42,7 @@
37
42
  "name": "Llama 3.3 70B Private",
38
43
  "upstreamModel": "llama3-3-70b",
39
44
  "billingModel": "TEE/llama3-3-70b",
45
+ "providerPricingModel": "meta-llama/llama-3.3-70b-instruct",
40
46
  "teeTargetModel": "llama3-3-70b",
41
47
  "created": 1764547200,
42
48
  "ownedBy": "nanogpt-private-mode",
@@ -47,8 +53,11 @@
47
53
  "name": "GLM 5.2 Private",
48
54
  "upstreamModel": "glm-5-2",
49
55
  "billingModel": "TEE/glm-5-2",
56
+ "providerPricingModel": "TEE/glm-5.2",
50
57
  "teeTargetModel": "glm-5-2",
51
58
  "thinkingMode": "glm-5.2",
59
+ "maxInputTokens": 393216,
60
+ "maxOutputTokens": 131072,
52
61
  "created": 1781827200,
53
62
  "ownedBy": "nanogpt-private-mode",
54
63
  "aliases": ["private/glm-5.2", "TEE/glm-5-2", "TEE/glm-5.2"]
@@ -58,8 +67,11 @@
58
67
  "name": "GLM 5.2 Thinking Private",
59
68
  "upstreamModel": "glm-5-2",
60
69
  "billingModel": "TEE/glm-5-2:thinking",
70
+ "providerPricingModel": "TEE/glm-5.2:thinking",
61
71
  "teeTargetModel": "glm-5-2",
62
72
  "thinkingMode": "glm-5.2",
73
+ "maxInputTokens": 393216,
74
+ "maxOutputTokens": 131072,
63
75
  "created": 1781827200,
64
76
  "ownedBy": "nanogpt-private-mode",
65
77
  "aliases": ["private/glm-5.2:thinking", "TEE/glm-5-2:thinking", "TEE/glm-5.2:thinking"]
@@ -69,6 +81,7 @@
69
81
  "name": "Gemma 4 31B Private",
70
82
  "upstreamModel": "gemma4-31b",
71
83
  "billingModel": "TEE/gemma4-31b",
84
+ "providerPricingModel": "TEE/gemma4-31b",
72
85
  "teeTargetModel": "gemma4-31b",
73
86
  "thinkingMode": "gemma",
74
87
  "created": 1764547200,
@@ -80,6 +93,7 @@
80
93
  "name": "Gemma 4 31B Thinking Private",
81
94
  "upstreamModel": "gemma4-31b",
82
95
  "billingModel": "TEE/gemma4-31b:thinking",
96
+ "providerPricingModel": "TEE/gemma4-31b:thinking",
83
97
  "teeTargetModel": "gemma4-31b",
84
98
  "thinkingMode": "gemma",
85
99
  "created": 1764547200,
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@nanogpt/private-mode",
3
- "version": "0.2.8",
3
+ "version": "0.2.11",
4
4
  "description": "OpenAI-compatible localhost proxy for NanoGPT Private Mode.",
5
5
  "type": "module",
6
6
  "publishConfig": {
@@ -13,11 +13,15 @@
13
13
  "files": [
14
14
  "bin",
15
15
  "lib/cacheScope.js",
16
+ "lib/catalogCompatibility.js",
17
+ "lib/constants.js",
16
18
  "lib/originPolicy.js",
19
+ "lib/packageInfo.js",
17
20
  "lib/requestTransforms.js",
18
21
  "lib/secureClientLifecycle.js",
19
22
  "lib/server.js",
20
23
  "lib/serverErrorNormalization.js",
24
+ "lib/sseResponsePump.js",
21
25
  "lib/statusContract.js",
22
26
  "lib/verifyReceipt.js",
23
27
  "models",