@nanogpt/private-mode 0.2.8 → 0.2.11
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +25 -7
- package/bin/nanogpt-private-mode.js +8 -0
- package/lib/catalogCompatibility.js +76 -0
- package/lib/constants.js +6 -0
- package/lib/packageInfo.js +8 -0
- package/lib/requestTransforms.js +132 -8
- package/lib/secureClientLifecycle.js +114 -4
- package/lib/server.js +71 -56
- package/lib/sseResponsePump.js +163 -0
- package/lib/statusContract.js +79 -1
- package/lib/verifyReceipt.js +39 -1
- package/models/private-tee.json +16 -2
- package/package.json +5 -1
package/README.md
CHANGED
|
@@ -5,7 +5,7 @@ OpenAI-compatible localhost proxy for NanoGPT Private Mode with supported TEE mo
|
|
|
5
5
|
Requires Node.js 22 or later.
|
|
6
6
|
|
|
7
7
|
```bash
|
|
8
|
-
NANOGPT_API_KEY=sk-your-key npx @nanogpt/private-mode
|
|
8
|
+
NANOGPT_API_KEY=sk-your-key npx --yes @nanogpt/private-mode@latest
|
|
9
9
|
```
|
|
10
10
|
|
|
11
11
|
Then point any OpenAI-compatible client at:
|
|
@@ -32,6 +32,14 @@ const response = await client.chat.completions.create({
|
|
|
32
32
|
|
|
33
33
|
The local proxy verifies TEE attestation, encrypts request bodies with EHBP, sends ciphertext through NanoGPT, decrypts encrypted responses locally, and returns normal OpenAI JSON to the calling app.
|
|
34
34
|
|
|
35
|
+
The model list is bundled with the installed proxy package. Before the proxy starts listening, it checks that list against NanoGPT's hosted Private Mode catalog and stops with an upgrade command if models were added or retired. This prevents an old package from advertising enclave targets that are no longer available. The running proxy reports its exact version at `GET http://127.0.0.1:8787/v1/private-mode/status`. To print the latest published package version, run:
|
|
36
|
+
|
|
37
|
+
```bash
|
|
38
|
+
npx --yes @nanogpt/private-mode@latest --version
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
If an older proxy is already running, stop it before starting the command above. API clients must use the localhost base URL shown below; sending `private/*` model IDs directly to NanoGPT's standard `/api/v1/chat/completions` endpoint does not enable Private Mode.
|
|
42
|
+
|
|
35
43
|
After a long idle period (five minutes by default), the proxy re-verifies attestation and creates a fresh encrypted transport before the next request. This makes laptop sleep/resume safe without requiring a proxy restart. Set `NANOGPT_PRIVATE_CLIENT_IDLE_RESET_MS` to a positive millisecond value to adjust the idle threshold, or `0` to disable idle resets.
|
|
36
44
|
|
|
37
45
|
NanoGPT can see account identity, selected private model, selected TEE target metadata, timing, sizes, status, and usage metadata. NanoGPT cannot read the prompt or completion body for supported private models.
|
|
@@ -44,6 +52,12 @@ In the hosted web app, decrypted Private Mode turns remain in local browser hist
|
|
|
44
52
|
|
|
45
53
|
Private Mode also requires enough NanoGPT balance and API-key spend-limit headroom before dispatch. For streaming calls, NanoGPT may precharge a conservative reserve before dispatch and refund unused balance after verified usage metadata is available.
|
|
46
54
|
|
|
55
|
+
Because the request body is encrypted before NanoGPT receives it, the service cannot inspect the request's `max_tokens` value during the balance precheck. The default streaming reserve therefore includes up to 32,768 output tokens, even when the encrypted request asks for fewer. This is a temporary reserve, not the final charge: unused balance is refunded when verified usage metadata arrives. The encrypted `max_tokens` value still controls generation inside the verified TEE.
|
|
56
|
+
|
|
57
|
+
Private Mode streaming has a 29-minute soft deadline so billing can settle before the route's 30-minute hard runtime limit. If the decrypted upstream stream ends without both a non-null `finish_reason` and the OpenAI `[DONE]` marker, the local proxy emits an SSE error with code `private_stream_incomplete`. Any text received before that error may be partial.
|
|
58
|
+
|
|
59
|
+
Private Mode currently supports Chat Completions only. The Responses API and Batch API are not available through the encrypted Private Mode endpoint. The standard Responses API outside Private Mode supports background requests within its configured 30-minute runtime limit; work that must reliably continue longer needs a durable job system.
|
|
60
|
+
|
|
47
61
|
Useful local checks:
|
|
48
62
|
|
|
49
63
|
```text
|
|
@@ -54,17 +68,21 @@ GET http://127.0.0.1:8787/v1/private-mode/attestation
|
|
|
54
68
|
|
|
55
69
|
Supported private model IDs include:
|
|
56
70
|
|
|
57
|
-
- `private/deepseek-v4-flash`
|
|
71
|
+
- `private/deepseek-v4-flash` - DeepSeek V4 Flash 0731
|
|
58
72
|
- `private/kimi-k3`
|
|
59
|
-
- `private/
|
|
60
|
-
- `private/
|
|
73
|
+
- `private/gpt-oss-120b`
|
|
74
|
+
- `private/llama3-3-70b`
|
|
61
75
|
- `private/glm-5-2`
|
|
62
76
|
- `private/glm-5-2:thinking`
|
|
77
|
+
- `private/gemma4-31b`
|
|
78
|
+
- `private/gemma4-31b:thinking`
|
|
79
|
+
|
|
80
|
+
Both GLM 5.2 variants are deployed with a 393,216-token total context limit, with prompt and output sharing that window. Kimi K3 Private has a 256,000-token context limit. DeepSeek V4 Flash 0731 Private has a 1,048,576-token total context limit. The model-list extension `context_length` is the combined prompt-and-output window, while `max_output_tokens` is the output ceiling within that same window. These values are not additive.
|
|
63
81
|
|
|
64
82
|
Browser requests are locked down by default. The proxy only accepts same-machine clients and rejects browser `Origin` headers that are not explicitly allowed, so a random website or LAN client cannot spend the local `NANOGPT_API_KEY` while the proxy is running. If a local browser app needs to call the proxy directly, allow that exact origin:
|
|
65
83
|
|
|
66
84
|
```bash
|
|
67
|
-
NANOGPT_API_KEY=sk-your-key npx @nanogpt/private-mode --allow-origin http://localhost:3000
|
|
85
|
+
NANOGPT_API_KEY=sk-your-key npx --yes @nanogpt/private-mode@latest --allow-origin http://localhost:3000
|
|
68
86
|
```
|
|
69
87
|
|
|
70
88
|
You can also set `NANOGPT_PRIVATE_ALLOWED_ORIGINS=http://localhost:3000`. Wildcard browser origins are not supported.
|
|
@@ -86,13 +104,13 @@ NanoGPT's hosted web app shows a Private Mode receipt on completed Private Mode
|
|
|
86
104
|
To independently verify a copied receipt:
|
|
87
105
|
|
|
88
106
|
```bash
|
|
89
|
-
npx @nanogpt/private-mode verify receipt.json
|
|
107
|
+
npx --yes @nanogpt/private-mode@latest verify receipt.json
|
|
90
108
|
```
|
|
91
109
|
|
|
92
110
|
You can also pipe JSON on stdin:
|
|
93
111
|
|
|
94
112
|
```bash
|
|
95
|
-
cat receipt.json | npx @nanogpt/private-mode verify
|
|
113
|
+
cat receipt.json | npx --yes @nanogpt/private-mode@latest verify
|
|
96
114
|
```
|
|
97
115
|
|
|
98
116
|
The verifier re-fetches attestation material for the receipt's enclave, verifies it locally, and checks the resulting measurements, release digest, and HPKE key against the copied receipt. If an optional attestation bundle hash is present but the freshly fetched bundle serializes differently, the verifier prints a warning instead of failing the core verification.
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
2
|
|
|
3
3
|
import { parseArgs } from 'node:util';
|
|
4
|
+
import { PRIVATE_MODE_PROXY_VERSION } from '../lib/packageInfo.js';
|
|
4
5
|
import { startPrivateModeProxy } from '../lib/server.js';
|
|
5
6
|
import { verifyPrivateModeReceiptCli } from '../lib/verifyReceipt.js';
|
|
6
7
|
|
|
@@ -14,10 +15,16 @@ const { values, positionals } = parseArgs({
|
|
|
14
15
|
},
|
|
15
16
|
'allow-origin': { type: 'string', multiple: true },
|
|
16
17
|
help: { type: 'boolean', short: 'h', default: false },
|
|
18
|
+
version: { type: 'boolean', short: 'v', default: false },
|
|
17
19
|
},
|
|
18
20
|
allowPositionals: true,
|
|
19
21
|
});
|
|
20
22
|
|
|
23
|
+
if (values.version) {
|
|
24
|
+
process.stdout.write(`${PRIVATE_MODE_PROXY_VERSION}\n`);
|
|
25
|
+
process.exit(0);
|
|
26
|
+
}
|
|
27
|
+
|
|
21
28
|
if (values.help) {
|
|
22
29
|
process.stdout.write(`NanoGPT Private Mode Proxy
|
|
23
30
|
|
|
@@ -32,6 +39,7 @@ Options:
|
|
|
32
39
|
--api-base <url> NanoGPT API base. Default: https://nano-gpt.com
|
|
33
40
|
--allow-origin <origin>
|
|
34
41
|
Browser origin allowed to call the proxy, for example http://localhost:3000
|
|
42
|
+
--version, -v Print the installed proxy version
|
|
35
43
|
`);
|
|
36
44
|
process.exit(0);
|
|
37
45
|
}
|
|
@@ -0,0 +1,76 @@
|
|
|
1
|
+
const DEFAULT_CATALOG_CHECK_TIMEOUT_MS = 10_000;
|
|
2
|
+
|
|
3
|
+
function normalizeModelIds(models) {
|
|
4
|
+
if (!Array.isArray(models)) return null;
|
|
5
|
+
|
|
6
|
+
const ids = models.map((model) => (
|
|
7
|
+
typeof model?.id === 'string' ? model.id.trim().toLowerCase() : ''
|
|
8
|
+
));
|
|
9
|
+
if (ids.some((id) => !id)) return null;
|
|
10
|
+
|
|
11
|
+
return new Set(ids);
|
|
12
|
+
}
|
|
13
|
+
|
|
14
|
+
export function comparePrivateModeCatalogs(localModels, hostedCatalog) {
|
|
15
|
+
const localIds = normalizeModelIds(localModels);
|
|
16
|
+
const hostedIds = normalizeModelIds(hostedCatalog?.data);
|
|
17
|
+
if (!localIds || !hostedIds) {
|
|
18
|
+
throw new Error('Private Mode model catalog response was malformed.');
|
|
19
|
+
}
|
|
20
|
+
|
|
21
|
+
return {
|
|
22
|
+
status: 'checked',
|
|
23
|
+
retiredLocalModels: [...localIds].filter((id) => !hostedIds.has(id)).sort(),
|
|
24
|
+
missingLocalModels: [...hostedIds].filter((id) => !localIds.has(id)).sort(),
|
|
25
|
+
};
|
|
26
|
+
}
|
|
27
|
+
|
|
28
|
+
export async function checkHostedPrivateModeCatalog({
|
|
29
|
+
apiBase,
|
|
30
|
+
localModels,
|
|
31
|
+
fetchImpl = fetch,
|
|
32
|
+
timeoutMs = DEFAULT_CATALOG_CHECK_TIMEOUT_MS,
|
|
33
|
+
}) {
|
|
34
|
+
const controller = new AbortController();
|
|
35
|
+
const timeout = setTimeout(() => controller.abort(), timeoutMs);
|
|
36
|
+
|
|
37
|
+
try {
|
|
38
|
+
const response = await fetchImpl(`${apiBase}/api/v1/private/tinfoil/models`, {
|
|
39
|
+
headers: { accept: 'application/json' },
|
|
40
|
+
signal: controller.signal,
|
|
41
|
+
});
|
|
42
|
+
if (!response.ok) {
|
|
43
|
+
return { status: 'unavailable', reason: `HTTP ${response.status}` };
|
|
44
|
+
}
|
|
45
|
+
|
|
46
|
+
return comparePrivateModeCatalogs(localModels, await response.json());
|
|
47
|
+
} catch (error) {
|
|
48
|
+
return {
|
|
49
|
+
status: 'unavailable',
|
|
50
|
+
reason: error instanceof Error ? error.message : String(error),
|
|
51
|
+
};
|
|
52
|
+
} finally {
|
|
53
|
+
clearTimeout(timeout);
|
|
54
|
+
}
|
|
55
|
+
}
|
|
56
|
+
|
|
57
|
+
export function hasPrivateModeCatalogMismatch(result) {
|
|
58
|
+
return result.status === 'checked'
|
|
59
|
+
&& (result.retiredLocalModels.length > 0 || result.missingLocalModels.length > 0);
|
|
60
|
+
}
|
|
61
|
+
|
|
62
|
+
export function buildPrivateModeCatalogUnavailableMessage(result, apiBase) {
|
|
63
|
+
return `Could not verify the bundled Private Mode model catalog against ${apiBase}: ${result.reason}. The proxy did not start because catalog compatibility could not be confirmed. Check your network connection and NanoGPT service availability, then retry.`;
|
|
64
|
+
}
|
|
65
|
+
|
|
66
|
+
export function buildPrivateModeCatalogMismatchMessage(result, proxyVersion) {
|
|
67
|
+
const details = [];
|
|
68
|
+
if (result.retiredLocalModels.length > 0) {
|
|
69
|
+
details.push(`retired local models: ${result.retiredLocalModels.join(', ')}`);
|
|
70
|
+
}
|
|
71
|
+
if (result.missingLocalModels.length > 0) {
|
|
72
|
+
details.push(`hosted models missing locally: ${result.missingLocalModels.join(', ')}`);
|
|
73
|
+
}
|
|
74
|
+
|
|
75
|
+
return `The bundled Private Mode model catalog in @nanogpt/private-mode v${proxyVersion} is out of date (${details.join('; ')}). Stop this proxy and restart with: NANOGPT_API_KEY=... npx --yes @nanogpt/private-mode@latest`;
|
|
76
|
+
}
|
package/lib/constants.js
ADDED
|
@@ -0,0 +1,6 @@
|
|
|
1
|
+
export const PRIVATE_MODE_STREAM_ROUTE_MAX_DURATION_MS = 30 * 60 * 1000;
|
|
2
|
+
export const PRIVATE_MODE_STREAM_SOFT_DEADLINE_LEAD_MS = 60 * 1000;
|
|
3
|
+
export const PRIVATE_MODE_STREAM_SETTLEMENT_SAFETY_MS = 30 * 1000;
|
|
4
|
+
export const PRIVATE_MODE_STREAM_SOFT_DEADLINE_MS =
|
|
5
|
+
PRIVATE_MODE_STREAM_ROUTE_MAX_DURATION_MS - PRIVATE_MODE_STREAM_SOFT_DEADLINE_LEAD_MS;
|
|
6
|
+
export const PRIVATE_MODE_DEFAULT_RESERVE_MAX_OUTPUT_TOKENS = 32_768;
|
|
@@ -0,0 +1,8 @@
|
|
|
1
|
+
import { readFileSync } from 'node:fs';
|
|
2
|
+
|
|
3
|
+
const packageJson = JSON.parse(
|
|
4
|
+
readFileSync(new URL('../package.json', import.meta.url), 'utf8'),
|
|
5
|
+
);
|
|
6
|
+
|
|
7
|
+
export const PRIVATE_MODE_PROXY_VERSION = String(packageJson.version);
|
|
8
|
+
export const PRIVATE_MODE_PROXY_VERSION_HEADER = 'x-nanogpt-private-proxy-version';
|
package/lib/requestTransforms.js
CHANGED
|
@@ -14,8 +14,7 @@ const MAX_TOKEN_ALIAS_FIELDS = [
|
|
|
14
14
|
];
|
|
15
15
|
const KIMI_K3_STANDARD_DEFAULT_MAX_COMPLETION_TOKENS = 16_384;
|
|
16
16
|
const KIMI_K3_DEFAULT_MAX_COMPLETION_TOKENS = 65_536;
|
|
17
|
-
const
|
|
18
|
-
const KIMI_K3_CONTEXT_WINDOW_TOKENS = 1_048_576;
|
|
17
|
+
const KIMI_K3_PRIVATE_CONTEXT_WINDOW_TOKENS = 256_000;
|
|
19
18
|
const KIMI_K3_CONTEXT_SAFETY_MARGIN_TOKENS = 10;
|
|
20
19
|
const KIMI_K3_ESTIMATED_TEXT_BYTES_PER_TOKEN = 3;
|
|
21
20
|
const KIMI_K3_HIGH_ENTROPY_TEXT_MIN_LENGTH = 2_048;
|
|
@@ -549,8 +548,18 @@ function resolvePrivateModeKimiK3ReasoningEffort(body) {
|
|
|
549
548
|
return excludesReasoning ? normalizedEffort ?? 'low' : normalizedEffort ?? 'max';
|
|
550
549
|
}
|
|
551
550
|
|
|
552
|
-
function applyPrivateModeKimiK3RequestParams(body) {
|
|
551
|
+
function applyPrivateModeKimiK3RequestParams(body, model) {
|
|
553
552
|
const reasoningEffort = resolvePrivateModeKimiK3ReasoningEffort(body);
|
|
553
|
+
const contextWindowTokens = typeof model.maxInputTokens === 'number'
|
|
554
|
+
&& Number.isFinite(model.maxInputTokens)
|
|
555
|
+
&& model.maxInputTokens > 0
|
|
556
|
+
? Math.floor(model.maxInputTokens)
|
|
557
|
+
: KIMI_K3_PRIVATE_CONTEXT_WINDOW_TOKENS;
|
|
558
|
+
const maxCompletionTokens = typeof model.maxOutputTokens === 'number'
|
|
559
|
+
&& Number.isFinite(model.maxOutputTokens)
|
|
560
|
+
&& model.maxOutputTokens > 0
|
|
561
|
+
? Math.floor(model.maxOutputTokens)
|
|
562
|
+
: contextWindowTokens;
|
|
554
563
|
const requestedMaxTokens = typeof body.max_tokens === 'number' && Number.isFinite(body.max_tokens)
|
|
555
564
|
? body.max_tokens
|
|
556
565
|
: undefined;
|
|
@@ -559,13 +568,13 @@ function applyPrivateModeKimiK3RequestParams(body) {
|
|
|
559
568
|
? KIMI_K3_DEFAULT_MAX_COMPLETION_TOKENS
|
|
560
569
|
: KIMI_K3_STANDARD_DEFAULT_MAX_COMPLETION_TOKENS
|
|
561
570
|
: requestedMaxTokens < 0
|
|
562
|
-
?
|
|
571
|
+
? maxCompletionTokens
|
|
563
572
|
: requestedMaxTokens;
|
|
564
573
|
normalizePrivateModeKimiK3NamedToolChoice(body);
|
|
565
574
|
const promptTokenEstimate = estimatePrivateModeKimiK3PromptTokens(body);
|
|
566
575
|
const remainingContext = Math.max(
|
|
567
576
|
1,
|
|
568
|
-
|
|
577
|
+
contextWindowTokens -
|
|
569
578
|
promptTokenEstimate -
|
|
570
579
|
KIMI_K3_CONTEXT_SAFETY_MARGIN_TOKENS,
|
|
571
580
|
);
|
|
@@ -670,13 +679,128 @@ export function createPrivateModeReasoningContentSuppressor() {
|
|
|
670
679
|
}
|
|
671
680
|
|
|
672
681
|
export function splitCompleteSseFrames(buffer) {
|
|
673
|
-
const parts = buffer.split(
|
|
682
|
+
const parts = buffer.split(/(?:\r\n|\r|\n){2}/);
|
|
674
683
|
return {
|
|
675
|
-
frames: parts.slice(0, -1).map((frame) => frame.replace(/\r\n/g, '\n')),
|
|
684
|
+
frames: parts.slice(0, -1).map((frame) => frame.replace(/\r\n|\r/g, '\n')),
|
|
676
685
|
remainder: parts.at(-1) || '',
|
|
677
686
|
};
|
|
678
687
|
}
|
|
679
688
|
|
|
689
|
+
export function createPrivateModeSseTerminalState(expectedChoiceCount = 1) {
|
|
690
|
+
return {
|
|
691
|
+
expectedChoiceCount: Number.isSafeInteger(expectedChoiceCount) && expectedChoiceCount > 0
|
|
692
|
+
? expectedChoiceCount
|
|
693
|
+
: 1,
|
|
694
|
+
finishedChoiceIndexes: new Set(),
|
|
695
|
+
sawDone: false,
|
|
696
|
+
};
|
|
697
|
+
}
|
|
698
|
+
|
|
699
|
+
function readPrivateModeSseDataPayload(frame) {
|
|
700
|
+
if (typeof frame !== 'string') return '';
|
|
701
|
+
return frame
|
|
702
|
+
.split('\n')
|
|
703
|
+
.filter((line) => line.startsWith('data:'))
|
|
704
|
+
.map((line) => line.slice('data:'.length).trim())
|
|
705
|
+
.join('\n')
|
|
706
|
+
.trim();
|
|
707
|
+
}
|
|
708
|
+
|
|
709
|
+
export function observePrivateModeSseTerminalFrame(state, frame) {
|
|
710
|
+
if (!isPlainObject(state) || typeof frame !== 'string') return state;
|
|
711
|
+
const payload = readPrivateModeSseDataPayload(frame);
|
|
712
|
+
if (!payload) return state;
|
|
713
|
+
if (payload === '[DONE]') {
|
|
714
|
+
state.sawDone = true;
|
|
715
|
+
return state;
|
|
716
|
+
}
|
|
717
|
+
|
|
718
|
+
try {
|
|
719
|
+
const parsed = JSON.parse(payload);
|
|
720
|
+
if (isPlainObject(parsed) && Array.isArray(parsed.choices)) {
|
|
721
|
+
for (const choice of parsed.choices) {
|
|
722
|
+
if (
|
|
723
|
+
!isPlainObject(choice) ||
|
|
724
|
+
choice.finish_reason === undefined ||
|
|
725
|
+
choice.finish_reason === null
|
|
726
|
+
) continue;
|
|
727
|
+
const choiceIndex = Number.isSafeInteger(choice.index)
|
|
728
|
+
? choice.index
|
|
729
|
+
: state.expectedChoiceCount === 1
|
|
730
|
+
? 0
|
|
731
|
+
: null;
|
|
732
|
+
if (choiceIndex !== null && choiceIndex >= 0 && choiceIndex < state.expectedChoiceCount) {
|
|
733
|
+
state.finishedChoiceIndexes.add(choiceIndex);
|
|
734
|
+
}
|
|
735
|
+
}
|
|
736
|
+
}
|
|
737
|
+
} catch {
|
|
738
|
+
// Other SSE payloads do not contribute to the OpenAI terminal sequence.
|
|
739
|
+
}
|
|
740
|
+
return state;
|
|
741
|
+
}
|
|
742
|
+
|
|
743
|
+
export function isPrivateModeSseTerminalStateComplete(state) {
|
|
744
|
+
return state?.sawDone === true &&
|
|
745
|
+
state?.finishedChoiceIndexes instanceof Set &&
|
|
746
|
+
state.finishedChoiceIndexes.size === state.expectedChoiceCount;
|
|
747
|
+
}
|
|
748
|
+
|
|
749
|
+
export function createPrivateModeSseTerminalGuard(expectedChoiceCount = 1) {
|
|
750
|
+
const state = createPrivateModeSseTerminalState(expectedChoiceCount);
|
|
751
|
+
let emittedDone = false;
|
|
752
|
+
return {
|
|
753
|
+
state,
|
|
754
|
+
observe(frame, forwardedFrame = frame) {
|
|
755
|
+
const doneAlreadyEmitted = emittedDone;
|
|
756
|
+
const containsDone = readPrivateModeSseDataPayload(frame) === '[DONE]';
|
|
757
|
+
observePrivateModeSseTerminalFrame(state, frame);
|
|
758
|
+
// The pump drains transport bytes after client-visible completion so the
|
|
759
|
+
// hosted request can finish settlement. Nothing from that drain belongs
|
|
760
|
+
// on the client side of the canonical terminal marker.
|
|
761
|
+
if (doneAlreadyEmitted) return '';
|
|
762
|
+
// Drop every raw DONE marker so duplicates collapse to one canonical
|
|
763
|
+
// marker. Reasoning suppression may prepend a synthetic visible-content
|
|
764
|
+
// delta, which must remain ahead of that marker.
|
|
765
|
+
const forwardedWithoutDone = containsDone
|
|
766
|
+
? forwardedFrame
|
|
767
|
+
.split(/(?:\r\n|\r|\n){2}/)
|
|
768
|
+
.filter((forwarded) => readPrivateModeSseDataPayload(forwarded) !== '[DONE]')
|
|
769
|
+
.join('\n\n')
|
|
770
|
+
: forwardedFrame;
|
|
771
|
+
|
|
772
|
+
// A hosted stream can keep transport EOF open while billing settles.
|
|
773
|
+
// Release successful completion as soon as the full SSE terminal
|
|
774
|
+
// sequence is observed, then let the response pump drain the transport.
|
|
775
|
+
if (!emittedDone && isPrivateModeSseTerminalStateComplete(state)) {
|
|
776
|
+
emittedDone = true;
|
|
777
|
+
return [forwardedWithoutDone, 'data: [DONE]'].filter(Boolean).join('\n\n');
|
|
778
|
+
}
|
|
779
|
+
return forwardedWithoutDone;
|
|
780
|
+
},
|
|
781
|
+
hasEmittedDone() {
|
|
782
|
+
return emittedDone;
|
|
783
|
+
},
|
|
784
|
+
finish() {
|
|
785
|
+
if (isPrivateModeSseTerminalStateComplete(state)) {
|
|
786
|
+
if (emittedDone) return { complete: true, frame: '' };
|
|
787
|
+
emittedDone = true;
|
|
788
|
+
return { complete: true, frame: 'data: [DONE]' };
|
|
789
|
+
}
|
|
790
|
+
return {
|
|
791
|
+
complete: false,
|
|
792
|
+
frame: `data: ${JSON.stringify({
|
|
793
|
+
error: {
|
|
794
|
+
message: 'Private Mode stream ended before a complete terminal sequence. The response may be partial; retry the request.',
|
|
795
|
+
type: 'api_error',
|
|
796
|
+
code: 'private_stream_incomplete',
|
|
797
|
+
},
|
|
798
|
+
})}`,
|
|
799
|
+
};
|
|
800
|
+
},
|
|
801
|
+
};
|
|
802
|
+
}
|
|
803
|
+
|
|
680
804
|
const PRIVATE_MODE_SSE_CHUNK_METADATA_FIELDS = [
|
|
681
805
|
'id',
|
|
682
806
|
'object',
|
|
@@ -899,7 +1023,7 @@ export function applyPrivateModelRequestMutations(body, model) {
|
|
|
899
1023
|
normalizeStreamOptions(body);
|
|
900
1024
|
applyTinfoilCompatibilityMutations(body, model);
|
|
901
1025
|
if (isPrivateModeKimiK3Model(model)) {
|
|
902
|
-
applyPrivateModeKimiK3RequestParams(body);
|
|
1026
|
+
applyPrivateModeKimiK3RequestParams(body, model);
|
|
903
1027
|
} else if (model.thinkingMode === 'deepseek-v4') {
|
|
904
1028
|
clampPrivateModeDeepSeekV4Output(body);
|
|
905
1029
|
}
|
|
@@ -1,5 +1,40 @@
|
|
|
1
1
|
const DEFAULT_SECURE_CLIENT_IDLE_RESET_MS = 5 * 60 * 1000;
|
|
2
|
+
const DEFAULT_VERIFICATION_MAX_AGE_MS = 5 * 60 * 1000;
|
|
2
3
|
const EHBP_RESPONSE_NONCE_HEADER = 'ehbp-response-nonce';
|
|
4
|
+
const REQUIRED_VERIFICATION_STEPS = [
|
|
5
|
+
'fetchDigest',
|
|
6
|
+
'verifyCode',
|
|
7
|
+
'verifyEnclave',
|
|
8
|
+
'compareMeasurements',
|
|
9
|
+
];
|
|
10
|
+
|
|
11
|
+
function isNonEmptyString(value) {
|
|
12
|
+
return typeof value === 'string' && value.trim().length > 0;
|
|
13
|
+
}
|
|
14
|
+
|
|
15
|
+
function hasCompleteVerificationEvidence(document) {
|
|
16
|
+
return document?.securityVerified === true
|
|
17
|
+
&& isNonEmptyString(document.codeFingerprint)
|
|
18
|
+
&& isNonEmptyString(document.enclaveFingerprint)
|
|
19
|
+
&& isNonEmptyString(document.releaseDigest)
|
|
20
|
+
&& isNonEmptyString(document.hpkePublicKey)
|
|
21
|
+
&& REQUIRED_VERIFICATION_STEPS.every(
|
|
22
|
+
(step) => document.steps?.[step]?.status === 'success',
|
|
23
|
+
)
|
|
24
|
+
&& !Object.values(document.steps || {}).some((step) => step?.status === 'failed');
|
|
25
|
+
}
|
|
26
|
+
|
|
27
|
+
function isMeasurementMismatchError(error) {
|
|
28
|
+
let current = error;
|
|
29
|
+
for (let depth = 0; depth < 5 && current; depth += 1) {
|
|
30
|
+
const message = String(current.message || current);
|
|
31
|
+
if (/\b(?:code\s+)?measurement mismatch\b|\bcannot compare measurements\b/i.test(message)) {
|
|
32
|
+
return true;
|
|
33
|
+
}
|
|
34
|
+
current = current.cause;
|
|
35
|
+
}
|
|
36
|
+
return false;
|
|
37
|
+
}
|
|
3
38
|
|
|
4
39
|
function readSecureClientIdleResetMs() {
|
|
5
40
|
const configured = process.env.NANOGPT_PRIVATE_CLIENT_IDLE_RESET_MS?.trim();
|
|
@@ -14,9 +49,14 @@ export function createSecureState(apiBase, options = {}) {
|
|
|
14
49
|
: readSecureClientIdleResetMs();
|
|
15
50
|
const loadTinfoil = options.loadTinfoil || (() => import('tinfoil'));
|
|
16
51
|
const now = options.now || Date.now;
|
|
52
|
+
const verificationMaxAgeMs = Number.isFinite(options.verificationMaxAgeMs)
|
|
53
|
+
&& options.verificationMaxAgeMs >= 0
|
|
54
|
+
? Math.floor(options.verificationMaxAgeMs)
|
|
55
|
+
: DEFAULT_VERIFICATION_MAX_AGE_MS;
|
|
17
56
|
let clientState = null;
|
|
18
57
|
let verificationDocument = null;
|
|
19
58
|
let verificationError = null;
|
|
59
|
+
let verifiedAtMs = null;
|
|
20
60
|
|
|
21
61
|
async function getClient(userCacheSecret = clientState?.userCacheSecret) {
|
|
22
62
|
if (!userCacheSecret) {
|
|
@@ -24,12 +64,18 @@ export function createSecureState(apiBase, options = {}) {
|
|
|
24
64
|
}
|
|
25
65
|
const nowMs = now();
|
|
26
66
|
const idleMs = clientState ? nowMs - clientState.lastUsedAtMs : 0;
|
|
67
|
+
const verificationAgeMs = verifiedAtMs === null ? 0 : nowMs - verifiedAtMs;
|
|
27
68
|
if (
|
|
28
69
|
clientState?.userCacheSecret === userCacheSecret
|
|
29
70
|
&& (
|
|
30
71
|
idleResetMs === 0
|
|
31
72
|
|| (idleMs >= 0 && idleMs < idleResetMs)
|
|
32
73
|
)
|
|
74
|
+
&& (
|
|
75
|
+
verifiedAtMs === null
|
|
76
|
+
|| verificationMaxAgeMs === 0
|
|
77
|
+
|| (verificationAgeMs >= 0 && verificationAgeMs < verificationMaxAgeMs)
|
|
78
|
+
)
|
|
33
79
|
) {
|
|
34
80
|
clientState.lastUsedAtMs = nowMs;
|
|
35
81
|
return clientState.promise;
|
|
@@ -40,6 +86,7 @@ export function createSecureState(apiBase, options = {}) {
|
|
|
40
86
|
// still be finishing an older in-flight request.
|
|
41
87
|
verificationDocument = null;
|
|
42
88
|
verificationError = null;
|
|
89
|
+
verifiedAtMs = null;
|
|
43
90
|
const nextState = {
|
|
44
91
|
client: null,
|
|
45
92
|
lastUsedAtMs: nowMs,
|
|
@@ -54,10 +101,19 @@ export function createSecureState(apiBase, options = {}) {
|
|
|
54
101
|
transport: 'ehbp',
|
|
55
102
|
userCacheSecret,
|
|
56
103
|
});
|
|
57
|
-
await client.ready();
|
|
58
104
|
nextState.client = client;
|
|
105
|
+
try {
|
|
106
|
+
await client.ready();
|
|
107
|
+
} finally {
|
|
108
|
+
if (clientState === nextState) {
|
|
109
|
+
verificationDocument = client.getVerificationDocument();
|
|
110
|
+
}
|
|
111
|
+
}
|
|
112
|
+
if (!hasCompleteVerificationEvidence(verificationDocument)) {
|
|
113
|
+
throw new Error('Private Mode attestation evidence was incomplete.');
|
|
114
|
+
}
|
|
59
115
|
if (clientState === nextState) {
|
|
60
|
-
|
|
116
|
+
verifiedAtMs = now();
|
|
61
117
|
verificationError = null;
|
|
62
118
|
}
|
|
63
119
|
return client;
|
|
@@ -80,7 +136,23 @@ export function createSecureState(apiBase, options = {}) {
|
|
|
80
136
|
getClient,
|
|
81
137
|
markClientUsed(client) {
|
|
82
138
|
if (!clientState || clientState.client !== client) return false;
|
|
83
|
-
|
|
139
|
+
const nowMs = now();
|
|
140
|
+
const currentDocument = client.getVerificationDocument();
|
|
141
|
+
if (currentDocument !== verificationDocument) {
|
|
142
|
+
verificationDocument = currentDocument;
|
|
143
|
+
if (hasCompleteVerificationEvidence(currentDocument)) {
|
|
144
|
+
verifiedAtMs = nowMs;
|
|
145
|
+
verificationError = null;
|
|
146
|
+
} else {
|
|
147
|
+
verifiedAtMs = null;
|
|
148
|
+
// Preserve the incomplete document for status visibility, but never
|
|
149
|
+
// reuse a transport after the SDK has invalidated its evidence.
|
|
150
|
+
clientState.lastUsedAtMs = nowMs;
|
|
151
|
+
clientState = null;
|
|
152
|
+
return true;
|
|
153
|
+
}
|
|
154
|
+
}
|
|
155
|
+
clientState.lastUsedAtMs = nowMs;
|
|
84
156
|
return true;
|
|
85
157
|
},
|
|
86
158
|
invalidateClient(client) {
|
|
@@ -89,11 +161,49 @@ export function createSecureState(apiBase, options = {}) {
|
|
|
89
161
|
clientState = null;
|
|
90
162
|
verificationDocument = null;
|
|
91
163
|
verificationError = null;
|
|
164
|
+
verifiedAtMs = null;
|
|
92
165
|
return true;
|
|
93
166
|
},
|
|
94
167
|
getVerificationState() {
|
|
168
|
+
const nowMs = now();
|
|
169
|
+
// Tinfoil 1.1.12 resets its document to "pending" after an exhausted
|
|
170
|
+
// AttestationError retry, so retain the SDK's explicit mismatch signal.
|
|
171
|
+
const mismatch = verificationDocument?.steps?.compareMeasurements?.status === 'failed'
|
|
172
|
+
|| isMeasurementMismatchError(verificationError);
|
|
173
|
+
const verified = hasCompleteVerificationEvidence(verificationDocument)
|
|
174
|
+
&& !mismatch
|
|
175
|
+
&& verifiedAtMs !== null;
|
|
176
|
+
const verificationAgeMs = verifiedAtMs === null ? null : nowMs - verifiedAtMs;
|
|
177
|
+
const clockRolledBack = verificationAgeMs !== null && verificationAgeMs < 0;
|
|
178
|
+
const expiresAtMs = verifiedAtMs === null || verificationMaxAgeMs === 0
|
|
179
|
+
? null
|
|
180
|
+
: verifiedAtMs + verificationMaxAgeMs;
|
|
181
|
+
const isStale = verified && (
|
|
182
|
+
clockRolledBack
|
|
183
|
+
|| (expiresAtMs !== null && nowMs >= expiresAtMs)
|
|
184
|
+
);
|
|
185
|
+
const freshnessSeconds = expiresAtMs === null
|
|
186
|
+
? null
|
|
187
|
+
: clockRolledBack
|
|
188
|
+
? 0
|
|
189
|
+
: Math.max(0, Math.floor((expiresAtMs - nowMs) / 1000));
|
|
190
|
+
const verificationStatus = mismatch
|
|
191
|
+
? 'measurement_mismatch'
|
|
192
|
+
: verified && verifiedAtMs !== null
|
|
193
|
+
? isStale
|
|
194
|
+
? 'stale'
|
|
195
|
+
: 'verified'
|
|
196
|
+
: verificationError || verificationDocument
|
|
197
|
+
? 'verification_failed'
|
|
198
|
+
: 'not_checked';
|
|
95
199
|
return {
|
|
96
|
-
verified
|
|
200
|
+
verified,
|
|
201
|
+
verificationStatus,
|
|
202
|
+
verifiedAt: verifiedAtMs === null ? null : new Date(verifiedAtMs).toISOString(),
|
|
203
|
+
expiresAt: expiresAtMs === null
|
|
204
|
+
? null
|
|
205
|
+
: new Date(expiresAtMs).toISOString(),
|
|
206
|
+
freshnessSeconds,
|
|
97
207
|
verificationDocument,
|
|
98
208
|
error: verificationError ? String(verificationError.message || verificationError) : null,
|
|
99
209
|
};
|
package/lib/server.js
CHANGED
|
@@ -2,28 +2,45 @@ import { createServer } from 'node:http';
|
|
|
2
2
|
import { readFileSync } from 'node:fs';
|
|
3
3
|
|
|
4
4
|
import { buildPrivateModeCacheScopeProof } from './cacheScope.js';
|
|
5
|
+
import {
|
|
6
|
+
buildPrivateModeCatalogMismatchMessage,
|
|
7
|
+
buildPrivateModeCatalogUnavailableMessage,
|
|
8
|
+
checkHostedPrivateModeCatalog,
|
|
9
|
+
hasPrivateModeCatalogMismatch,
|
|
10
|
+
} from './catalogCompatibility.js';
|
|
11
|
+
import {
|
|
12
|
+
PRIVATE_MODE_DEFAULT_RESERVE_MAX_OUTPUT_TOKENS,
|
|
13
|
+
PRIVATE_MODE_STREAM_SOFT_DEADLINE_MS,
|
|
14
|
+
} from './constants.js';
|
|
5
15
|
import {
|
|
6
16
|
buildPrivateModeOriginPolicy,
|
|
7
17
|
getCorsHeadersForRequest,
|
|
8
18
|
} from './originPolicy.js';
|
|
9
19
|
import {
|
|
10
20
|
applyPrivateModelRequestMutations,
|
|
11
|
-
buildPrivateModeSseContentDelta,
|
|
12
|
-
createPrivateModeReasoningContentSuppressor,
|
|
13
21
|
shouldSuppressPrivateModelReasoning,
|
|
14
|
-
splitCompleteSseFrames,
|
|
15
|
-
suppressPrivateModeReasoningFromSseFrame,
|
|
16
22
|
suppressPrivateModeReasoningFromJsonText,
|
|
17
23
|
} from './requestTransforms.js';
|
|
18
24
|
import {
|
|
19
25
|
normalizePrivateModeUpstreamErrorMessage,
|
|
20
26
|
readErrorMessage,
|
|
21
27
|
} from './serverErrorNormalization.js';
|
|
28
|
+
import {
|
|
29
|
+
PRIVATE_MODE_PROXY_VERSION,
|
|
30
|
+
PRIVATE_MODE_PROXY_VERSION_HEADER,
|
|
31
|
+
} from './packageInfo.js';
|
|
22
32
|
import {
|
|
23
33
|
createSecureState,
|
|
24
34
|
fetchWithSecureClientRecovery,
|
|
25
35
|
} from './secureClientLifecycle.js';
|
|
26
|
-
import {
|
|
36
|
+
import {
|
|
37
|
+
buildPrivateModeStatusContract,
|
|
38
|
+
buildPublicAttestationSummary,
|
|
39
|
+
} from './statusContract.js';
|
|
40
|
+
import {
|
|
41
|
+
isPrivateModeSseResponse,
|
|
42
|
+
pipePrivateModeSseResponse,
|
|
43
|
+
} from './sseResponsePump.js';
|
|
27
44
|
|
|
28
45
|
const MODELS = JSON.parse(
|
|
29
46
|
readFileSync(new URL('../models/private-tee.json', import.meta.url), 'utf8'),
|
|
@@ -76,15 +93,18 @@ function openAIModelList() {
|
|
|
76
93
|
object: 'model',
|
|
77
94
|
created: model.created,
|
|
78
95
|
owned_by: model.ownedBy,
|
|
96
|
+
...(model.maxInputTokens ? { context_length: model.maxInputTokens } : {}),
|
|
79
97
|
...(model.maxOutputTokens ? { max_output_tokens: model.maxOutputTokens } : {}),
|
|
80
98
|
})),
|
|
81
99
|
};
|
|
82
100
|
}
|
|
83
101
|
|
|
84
|
-
export function privateModeStatus(apiBase, secureState, localBase, originPolicy) {
|
|
102
|
+
export function privateModeStatus(apiBase, secureState, localBase, originPolicy, catalogCompatibility) {
|
|
103
|
+
const verificationState = secureState.getVerificationState();
|
|
85
104
|
return {
|
|
86
105
|
ok: true,
|
|
87
106
|
mode: 'private_tee',
|
|
107
|
+
proxy_version: PRIVATE_MODE_PROXY_VERSION,
|
|
88
108
|
apiBase,
|
|
89
109
|
local_base_url: localBase,
|
|
90
110
|
models_path: '/v1/models',
|
|
@@ -97,12 +117,19 @@ export function privateModeStatus(apiBase, secureState, localBase, originPolicy)
|
|
|
97
117
|
response_body_encrypted: true,
|
|
98
118
|
streaming: true,
|
|
99
119
|
streaming_billing: 'precharged_reserve_with_verified_usage_refund',
|
|
120
|
+
// This deadline is enforced by the hosted route, not this process. A
|
|
121
|
+
// local environment override cannot configure the remote service.
|
|
122
|
+
streaming_soft_deadline_ms: PRIVATE_MODE_STREAM_SOFT_DEADLINE_MS,
|
|
123
|
+
default_stream_reserve_max_output_tokens: PRIVATE_MODE_DEFAULT_RESERVE_MAX_OUTPUT_TOKENS,
|
|
124
|
+
responses_api_supported: false,
|
|
125
|
+
batch_api_supported: false,
|
|
100
126
|
api_local_proxy_required: true,
|
|
101
127
|
browser_frontend_local_proxy_required: false,
|
|
102
|
-
...buildPrivateModeStatusContract(),
|
|
128
|
+
...buildPrivateModeStatusContract(verificationState),
|
|
103
129
|
browser_origins_allowed: originPolicy.allowedOrigins,
|
|
104
130
|
models: openAIModelList().data,
|
|
105
|
-
|
|
131
|
+
catalog_compatibility: catalogCompatibility,
|
|
132
|
+
attestation: buildPublicAttestationSummary(verificationState),
|
|
106
133
|
};
|
|
107
134
|
}
|
|
108
135
|
|
|
@@ -174,6 +201,7 @@ async function runPreflight({ apiBase, apiKey, model, req, requestBodyBytes }) {
|
|
|
174
201
|
headers: {
|
|
175
202
|
authorization: `Bearer ${apiKey}`,
|
|
176
203
|
'content-type': 'application/json',
|
|
204
|
+
[PRIVATE_MODE_PROXY_VERSION_HEADER]: PRIVATE_MODE_PROXY_VERSION,
|
|
177
205
|
...copyLocalHeaders(req),
|
|
178
206
|
},
|
|
179
207
|
body: JSON.stringify(preflightBody),
|
|
@@ -265,7 +293,7 @@ async function handleChatCompletion({ apiBase, apiKey, secureState, req, res, co
|
|
|
265
293
|
if (!model) {
|
|
266
294
|
jsonResponse(res, 400, {
|
|
267
295
|
error: {
|
|
268
|
-
message: `Unsupported private model "${body.model || ''}"
|
|
296
|
+
message: `Unsupported private model "${body.model || ''}" in @nanogpt/private-mode v${PRIVATE_MODE_PROXY_VERSION}. Check this proxy's /v1/models endpoint or restart with: NANOGPT_API_KEY=... npx --yes @nanogpt/private-mode@latest`,
|
|
269
297
|
type: 'invalid_request_error',
|
|
270
298
|
code: 'model_not_supported',
|
|
271
299
|
},
|
|
@@ -276,6 +304,7 @@ async function handleChatCompletion({ apiBase, apiKey, secureState, req, res, co
|
|
|
276
304
|
const suppressReasoning = shouldSuppressPrivateModelReasoning(body, model);
|
|
277
305
|
applyPrivateModelRequestMutations(body, model);
|
|
278
306
|
const privateStreamRequested = body.stream === true;
|
|
307
|
+
const expectedChoiceCount = Number.isSafeInteger(body.n) && body.n > 0 ? body.n : 1;
|
|
279
308
|
|
|
280
309
|
const privateRequestBody = JSON.stringify(body);
|
|
281
310
|
const preflight = await runPreflight({
|
|
@@ -312,6 +341,7 @@ async function handleChatCompletion({ apiBase, apiKey, secureState, req, res, co
|
|
|
312
341
|
'x-nanogpt-private-model': model.id,
|
|
313
342
|
'x-nanogpt-private-stream': privateStreamRequested ? 'true' : 'false',
|
|
314
343
|
'x-nanogpt-private-cache-scope': buildPrivateModeCacheScopeProof(preflight.cacheScope),
|
|
344
|
+
[PRIVATE_MODE_PROXY_VERSION_HEADER]: PRIVATE_MODE_PROXY_VERSION,
|
|
315
345
|
'x-query-source': 'api',
|
|
316
346
|
...copyLocalHeaders(req),
|
|
317
347
|
},
|
|
@@ -360,59 +390,30 @@ async function handleChatCompletion({ apiBase, apiKey, secureState, req, res, co
|
|
|
360
390
|
return;
|
|
361
391
|
}
|
|
362
392
|
|
|
363
|
-
if (privateStreamRequested
|
|
393
|
+
if (isPrivateModeSseResponse(privateStreamRequested, response)) {
|
|
364
394
|
res.writeHead(response.status, headers);
|
|
365
|
-
const reader = response.body.getReader();
|
|
366
|
-
const decoder = suppressReasoning ? new TextDecoder() : null;
|
|
367
|
-
const contentSuppressor = suppressReasoning
|
|
368
|
-
? createPrivateModeReasoningContentSuppressor()
|
|
369
|
-
: null;
|
|
370
|
-
const sseChunkMetadata = {};
|
|
371
|
-
let sseBuffer = '';
|
|
372
|
-
let streamFailed = false;
|
|
373
395
|
try {
|
|
374
|
-
|
|
375
|
-
|
|
376
|
-
|
|
377
|
-
|
|
378
|
-
|
|
379
|
-
|
|
380
|
-
|
|
381
|
-
|
|
382
|
-
|
|
383
|
-
|
|
384
|
-
|
|
385
|
-
|
|
386
|
-
}
|
|
387
|
-
sseBuffer += decoder.decode(value, { stream: true });
|
|
388
|
-
const { frames, remainder } = splitCompleteSseFrames(sseBuffer);
|
|
389
|
-
sseBuffer = remainder;
|
|
390
|
-
for (const frame of frames) {
|
|
391
|
-
res.write(`${suppressPrivateModeReasoningFromSseFrame(frame, contentSuppressor, sseChunkMetadata)}\n\n`);
|
|
392
|
-
}
|
|
393
|
-
}
|
|
394
|
-
if (suppressReasoning && decoder && contentSuppressor) {
|
|
395
|
-
sseBuffer += decoder.decode();
|
|
396
|
-
if (sseBuffer) {
|
|
397
|
-
res.write(`${suppressPrivateModeReasoningFromSseFrame(sseBuffer, contentSuppressor, sseChunkMetadata)}\n\n`);
|
|
398
|
-
}
|
|
399
|
-
const flushedContent = contentSuppressor.flush();
|
|
400
|
-
if (flushedContent) {
|
|
401
|
-
const finalDelta = buildPrivateModeSseContentDelta(flushedContent, sseChunkMetadata);
|
|
402
|
-
res.write(`data: ${finalDelta}\n\n`);
|
|
403
|
-
}
|
|
396
|
+
const outcome = await pipePrivateModeSseResponse({
|
|
397
|
+
body: response.body,
|
|
398
|
+
res,
|
|
399
|
+
suppressReasoning,
|
|
400
|
+
upstreamAbortController,
|
|
401
|
+
expectedChoiceCount,
|
|
402
|
+
});
|
|
403
|
+
// Successful completion and downstream disconnect still count as
|
|
404
|
+
// activity. Truncated or transport-failed streams keep the prior failure
|
|
405
|
+
// behavior and do not refresh the secure-client lifecycle.
|
|
406
|
+
if (!outcome.transportFailed && !outcome.streamFailed) {
|
|
407
|
+
secureState.markClientUsed(responseClient);
|
|
404
408
|
}
|
|
405
409
|
} catch (error) {
|
|
406
|
-
streamFailed = true;
|
|
407
410
|
upstreamAbortController.abort();
|
|
408
411
|
if (!res.destroyed) {
|
|
409
|
-
|
|
412
|
+
process.stderr.write(`Private Mode stream pump failed: ${error instanceof Error ? error.message : String(error)}\n`);
|
|
410
413
|
}
|
|
411
|
-
return;
|
|
412
414
|
} finally {
|
|
413
415
|
responseComplete = true;
|
|
414
|
-
if (!
|
|
415
|
-
if (!streamFailed && !res.writableEnded && !res.destroyed) res.end();
|
|
416
|
+
if (!res.writableEnded && !res.destroyed) res.end();
|
|
416
417
|
}
|
|
417
418
|
return;
|
|
418
419
|
}
|
|
@@ -468,6 +469,19 @@ function handleOptions(req, res, corsHeaders) {
|
|
|
468
469
|
|
|
469
470
|
export async function startPrivateModeProxy(options) {
|
|
470
471
|
const apiBase = normalizeApiBase(options.apiBase);
|
|
472
|
+
const catalogCompatibility = await checkHostedPrivateModeCatalog({
|
|
473
|
+
apiBase,
|
|
474
|
+
localModels: MODELS,
|
|
475
|
+
});
|
|
476
|
+
if (catalogCompatibility.status === 'unavailable') {
|
|
477
|
+
throw new Error(buildPrivateModeCatalogUnavailableMessage(catalogCompatibility, apiBase));
|
|
478
|
+
}
|
|
479
|
+
if (hasPrivateModeCatalogMismatch(catalogCompatibility)) {
|
|
480
|
+
throw new Error(buildPrivateModeCatalogMismatchMessage(
|
|
481
|
+
catalogCompatibility,
|
|
482
|
+
PRIVATE_MODE_PROXY_VERSION,
|
|
483
|
+
));
|
|
484
|
+
}
|
|
471
485
|
const secureState = createSecureState(apiBase);
|
|
472
486
|
const localBase = `http://${options.host}:${options.port}/v1`;
|
|
473
487
|
const originPolicy = buildPrivateModeOriginPolicy(options);
|
|
@@ -493,7 +507,7 @@ export async function startPrivateModeProxy(options) {
|
|
|
493
507
|
}
|
|
494
508
|
|
|
495
509
|
if (req.method === 'GET' && url.pathname === '/health') {
|
|
496
|
-
jsonResponse(res, 200, privateModeStatus(apiBase, secureState, localBase, originPolicy), corsHeaders);
|
|
510
|
+
jsonResponse(res, 200, privateModeStatus(apiBase, secureState, localBase, originPolicy, catalogCompatibility), corsHeaders);
|
|
497
511
|
return;
|
|
498
512
|
}
|
|
499
513
|
|
|
@@ -503,7 +517,7 @@ export async function startPrivateModeProxy(options) {
|
|
|
503
517
|
}
|
|
504
518
|
|
|
505
519
|
if (req.method === 'GET' && url.pathname === '/v1/private-mode/status') {
|
|
506
|
-
jsonResponse(res, 200, privateModeStatus(apiBase, secureState, localBase, originPolicy), corsHeaders);
|
|
520
|
+
jsonResponse(res, 200, privateModeStatus(apiBase, secureState, localBase, originPolicy, catalogCompatibility), corsHeaders);
|
|
507
521
|
return;
|
|
508
522
|
}
|
|
509
523
|
|
|
@@ -539,7 +553,7 @@ export async function startPrivateModeProxy(options) {
|
|
|
539
553
|
}
|
|
540
554
|
|
|
541
555
|
if (req.method === 'GET' && url.pathname === '/') {
|
|
542
|
-
textResponse(res, 200, `NanoGPT Private Mode Proxy\nOpenAI base URL: ${localBase}\nStatus: /v1/private-mode/status\n`, corsHeaders);
|
|
556
|
+
textResponse(res, 200, `NanoGPT Private Mode Proxy v${PRIVATE_MODE_PROXY_VERSION}\nOpenAI base URL: ${localBase}\nStatus: /v1/private-mode/status\n`, corsHeaders);
|
|
543
557
|
return;
|
|
544
558
|
}
|
|
545
559
|
|
|
@@ -567,6 +581,7 @@ export async function startPrivateModeProxy(options) {
|
|
|
567
581
|
|
|
568
582
|
if (options.quiet !== true) {
|
|
569
583
|
process.stdout.write(`NanoGPT Private Mode Proxy
|
|
584
|
+
Version: ${PRIVATE_MODE_PROXY_VERSION}
|
|
570
585
|
Local base URL: ${localBase}
|
|
571
586
|
NanoGPT API: ${apiBase}
|
|
572
587
|
Models: ${MODELS.map((model) => model.id).join(', ')}
|
|
@@ -0,0 +1,163 @@
|
|
|
1
|
+
import {
|
|
2
|
+
buildPrivateModeSseContentDelta,
|
|
3
|
+
createPrivateModeReasoningContentSuppressor,
|
|
4
|
+
createPrivateModeSseTerminalGuard,
|
|
5
|
+
splitCompleteSseFrames,
|
|
6
|
+
suppressPrivateModeReasoningFromSseFrame,
|
|
7
|
+
} from './requestTransforms.js';
|
|
8
|
+
|
|
9
|
+
const MAX_SSE_FRAME_BYTES = 2 * 1024 * 1024;
|
|
10
|
+
const DEFAULT_DOWNSTREAM_DRAIN_TIMEOUT_MS = 60 * 1000;
|
|
11
|
+
|
|
12
|
+
function assertSseFrameWithinLimit(frame) {
|
|
13
|
+
if (Buffer.byteLength(frame, 'utf8') > MAX_SSE_FRAME_BYTES) {
|
|
14
|
+
throw new Error('Private Mode upstream SSE frame exceeded the proxy buffer limit.');
|
|
15
|
+
}
|
|
16
|
+
}
|
|
17
|
+
|
|
18
|
+
export function isPrivateModeSseResponse(streamRequested, response) {
|
|
19
|
+
const contentType = response?.headers?.get?.('content-type') || '';
|
|
20
|
+
return streamRequested === true
|
|
21
|
+
&& Boolean(response?.body)
|
|
22
|
+
&& contentType.toLowerCase().includes('text/event-stream');
|
|
23
|
+
}
|
|
24
|
+
|
|
25
|
+
function writeWithBackpressure(res, chunk, timeoutMs) {
|
|
26
|
+
if (res.destroyed || res.writableEnded) return Promise.resolve(false);
|
|
27
|
+
try {
|
|
28
|
+
if (res.write(chunk)) return Promise.resolve(true);
|
|
29
|
+
} catch {
|
|
30
|
+
return Promise.resolve(false);
|
|
31
|
+
}
|
|
32
|
+
|
|
33
|
+
return new Promise((resolve) => {
|
|
34
|
+
let drainTimer;
|
|
35
|
+
const cleanup = () => {
|
|
36
|
+
if (drainTimer) clearTimeout(drainTimer);
|
|
37
|
+
res.off('drain', onDrain);
|
|
38
|
+
res.off('close', onClose);
|
|
39
|
+
res.off('error', onError);
|
|
40
|
+
};
|
|
41
|
+
const settle = (writable) => {
|
|
42
|
+
cleanup();
|
|
43
|
+
resolve(writable);
|
|
44
|
+
};
|
|
45
|
+
const onDrain = () => settle(!res.destroyed && !res.writableEnded);
|
|
46
|
+
const onClose = () => settle(false);
|
|
47
|
+
const onError = () => settle(false);
|
|
48
|
+
res.once('drain', onDrain);
|
|
49
|
+
res.once('close', onClose);
|
|
50
|
+
res.once('error', onError);
|
|
51
|
+
drainTimer = setTimeout(() => settle(false), timeoutMs);
|
|
52
|
+
drainTimer.unref?.();
|
|
53
|
+
});
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
export async function pipePrivateModeSseResponse({
|
|
57
|
+
body,
|
|
58
|
+
res,
|
|
59
|
+
suppressReasoning,
|
|
60
|
+
upstreamAbortController,
|
|
61
|
+
expectedChoiceCount = 1,
|
|
62
|
+
downstreamDrainTimeoutMs = DEFAULT_DOWNSTREAM_DRAIN_TIMEOUT_MS,
|
|
63
|
+
}) {
|
|
64
|
+
let reader;
|
|
65
|
+
const decoder = new TextDecoder();
|
|
66
|
+
const contentSuppressor = suppressReasoning
|
|
67
|
+
? createPrivateModeReasoningContentSuppressor()
|
|
68
|
+
: null;
|
|
69
|
+
const terminalGuard = createPrivateModeSseTerminalGuard(expectedChoiceCount);
|
|
70
|
+
const sseChunkMetadata = {};
|
|
71
|
+
let sseBuffer = '';
|
|
72
|
+
let transportFailed = false;
|
|
73
|
+
|
|
74
|
+
const writeFrame = async (frame) => {
|
|
75
|
+
if (!frame) return true;
|
|
76
|
+
return writeWithBackpressure(res, `${frame}\n\n`, downstreamDrainTimeoutMs);
|
|
77
|
+
};
|
|
78
|
+
const forwardFrame = async (frame) => {
|
|
79
|
+
const forwarded = suppressReasoning && contentSuppressor
|
|
80
|
+
? suppressPrivateModeReasoningFromSseFrame(frame, contentSuppressor, sseChunkMetadata)
|
|
81
|
+
: frame;
|
|
82
|
+
return writeFrame(terminalGuard.observe(frame, forwarded));
|
|
83
|
+
};
|
|
84
|
+
const finishStream = async () => {
|
|
85
|
+
// A complete terminal sequence may be released before transport EOF.
|
|
86
|
+
// Never append a defensive suppressor flush after that marker.
|
|
87
|
+
if (contentSuppressor && !terminalGuard.hasEmittedDone()) {
|
|
88
|
+
const flushedContent = contentSuppressor.flush();
|
|
89
|
+
if (flushedContent) {
|
|
90
|
+
const delta = buildPrivateModeSseContentDelta(flushedContent, sseChunkMetadata);
|
|
91
|
+
if (!await writeFrame(`data: ${delta}`)) return null;
|
|
92
|
+
}
|
|
93
|
+
}
|
|
94
|
+
const terminal = terminalGuard.finish();
|
|
95
|
+
if (!await writeFrame(terminal.frame)) return null;
|
|
96
|
+
return terminal;
|
|
97
|
+
};
|
|
98
|
+
|
|
99
|
+
try {
|
|
100
|
+
reader = body.getReader();
|
|
101
|
+
while (true) {
|
|
102
|
+
const { done, value } = await reader.read();
|
|
103
|
+
if (done) break;
|
|
104
|
+
if (res.destroyed || res.writableEnded) {
|
|
105
|
+
await reader.cancel('client disconnected').catch(() => {});
|
|
106
|
+
upstreamAbortController.abort();
|
|
107
|
+
return { streamFailed: false, clientDisconnected: true, transportFailed: false };
|
|
108
|
+
}
|
|
109
|
+
if (!value) continue;
|
|
110
|
+
sseBuffer += decoder.decode(value, { stream: true });
|
|
111
|
+
const { frames, remainder } = splitCompleteSseFrames(sseBuffer);
|
|
112
|
+
sseBuffer = remainder;
|
|
113
|
+
assertSseFrameWithinLimit(sseBuffer);
|
|
114
|
+
for (const frame of frames) {
|
|
115
|
+
assertSseFrameWithinLimit(frame);
|
|
116
|
+
if (!await forwardFrame(frame)) {
|
|
117
|
+
await reader.cancel('client disconnected').catch(() => {});
|
|
118
|
+
upstreamAbortController.abort();
|
|
119
|
+
return { streamFailed: false, clientDisconnected: true, transportFailed: false };
|
|
120
|
+
}
|
|
121
|
+
}
|
|
122
|
+
}
|
|
123
|
+
sseBuffer += decoder.decode();
|
|
124
|
+
if (sseBuffer && !await forwardFrame(sseBuffer)) {
|
|
125
|
+
await reader.cancel('client disconnected').catch(() => {});
|
|
126
|
+
upstreamAbortController.abort();
|
|
127
|
+
return { streamFailed: false, clientDisconnected: true, transportFailed: false };
|
|
128
|
+
}
|
|
129
|
+
} catch (error) {
|
|
130
|
+
transportFailed = true;
|
|
131
|
+
upstreamAbortController.abort();
|
|
132
|
+
process.stderr.write(`Private Mode upstream stream failed: ${error instanceof Error ? error.message : String(error)}\n`);
|
|
133
|
+
}
|
|
134
|
+
|
|
135
|
+
if (res.destroyed || res.writableEnded) {
|
|
136
|
+
upstreamAbortController.abort();
|
|
137
|
+
return { streamFailed: false, clientDisconnected: true, transportFailed };
|
|
138
|
+
}
|
|
139
|
+
|
|
140
|
+
let terminal;
|
|
141
|
+
try {
|
|
142
|
+
terminal = await finishStream();
|
|
143
|
+
} catch (error) {
|
|
144
|
+
transportFailed = true;
|
|
145
|
+
upstreamAbortController.abort();
|
|
146
|
+
process.stderr.write(`Private Mode stream finalization failed: ${error instanceof Error ? error.message : String(error)}\n`);
|
|
147
|
+
if (!terminalGuard.hasEmittedDone()) {
|
|
148
|
+
await writeFrame(`data: ${JSON.stringify({
|
|
149
|
+
error: {
|
|
150
|
+
message: 'Private Mode stream ended before a complete terminal sequence. The response may be partial; retry the request.',
|
|
151
|
+
type: 'api_error',
|
|
152
|
+
code: 'private_stream_incomplete',
|
|
153
|
+
},
|
|
154
|
+
})}`);
|
|
155
|
+
}
|
|
156
|
+
return { streamFailed: true, clientDisconnected: false, transportFailed };
|
|
157
|
+
}
|
|
158
|
+
if (!terminal) {
|
|
159
|
+
upstreamAbortController.abort();
|
|
160
|
+
return { streamFailed: false, clientDisconnected: true, transportFailed };
|
|
161
|
+
}
|
|
162
|
+
return { streamFailed: !terminal.complete, clientDisconnected: false, transportFailed };
|
|
163
|
+
}
|
package/lib/statusContract.js
CHANGED
|
@@ -1,3 +1,5 @@
|
|
|
1
|
+
import { createHash } from 'node:crypto';
|
|
2
|
+
|
|
1
3
|
export const PRIVATE_MODE_FRONTEND_SUPPORTED_FEATURES = Object.freeze([
|
|
2
4
|
'text_chat',
|
|
3
5
|
'streaming',
|
|
@@ -43,12 +45,88 @@ export const PRIVATE_MODE_PLAINTEXT_VISIBLE_TO = Object.freeze([
|
|
|
43
45
|
'verified_tee_target',
|
|
44
46
|
]);
|
|
45
47
|
|
|
46
|
-
export
|
|
48
|
+
export const PRIVATE_MODE_KNOWN_LIMITATIONS = Object.freeze([
|
|
49
|
+
'NanoGPT still observes account, routing, timing, size, status, and usage metadata.',
|
|
50
|
+
'Provider-reported usage metrics are trusted for billing and are not end-to-end signed.',
|
|
51
|
+
'A verification receipt does not prove usage correctness, prompt non-retention, or response content authenticity on its own.',
|
|
52
|
+
'Verification receipts are browser-generated and unsigned; the hashed request identifier is for opaque correlation, not proof of server issuance.',
|
|
53
|
+
'NanoGPT does not implement attestation-gated KMS release of upstream credentials; the inference enclave is provider-operated.',
|
|
54
|
+
]);
|
|
55
|
+
|
|
56
|
+
const PRIVATE_MODE_VERIFICATION_STATUSES = new Set([
|
|
57
|
+
'verified',
|
|
58
|
+
'unavailable',
|
|
59
|
+
'stale',
|
|
60
|
+
'measurement_mismatch',
|
|
61
|
+
'verification_failed',
|
|
62
|
+
'unsupported',
|
|
63
|
+
'not_checked',
|
|
64
|
+
]);
|
|
65
|
+
|
|
66
|
+
function stringIdentifier(value) {
|
|
67
|
+
return typeof value === 'string' && value.trim() ? value.trim().slice(0, 512) : undefined;
|
|
68
|
+
}
|
|
69
|
+
|
|
70
|
+
function publicKeyFingerprint(value) {
|
|
71
|
+
return typeof value === 'string' && value
|
|
72
|
+
? createHash('sha256').update(value).digest('hex')
|
|
73
|
+
: undefined;
|
|
74
|
+
}
|
|
75
|
+
|
|
76
|
+
export function buildPublicAttestationSummary(verificationState = {}) {
|
|
77
|
+
return {
|
|
78
|
+
verified: verificationState.verified === true,
|
|
79
|
+
verificationStatus: verificationState.verificationStatus,
|
|
80
|
+
verifiedAt: verificationState.verifiedAt,
|
|
81
|
+
expiresAt: verificationState.expiresAt,
|
|
82
|
+
freshnessSeconds: verificationState.freshnessSeconds,
|
|
83
|
+
error: verificationState.error ? 'Private Mode attestation failed.' : null,
|
|
84
|
+
};
|
|
85
|
+
}
|
|
86
|
+
|
|
87
|
+
export function buildPrivateModeStatusContract(verificationState = {}) {
|
|
88
|
+
const verificationStatus = PRIVATE_MODE_VERIFICATION_STATUSES.has(verificationState.verificationStatus)
|
|
89
|
+
? verificationState.verificationStatus
|
|
90
|
+
: 'not_checked';
|
|
91
|
+
const document = verificationState.verificationDocument || {};
|
|
92
|
+
const keyFingerprint = publicKeyFingerprint(document.hpkePublicKey);
|
|
47
93
|
return {
|
|
48
94
|
frontend_supported_features: [...PRIVATE_MODE_FRONTEND_SUPPORTED_FEATURES],
|
|
49
95
|
frontend_disabled_features: [...PRIVATE_MODE_FRONTEND_DISABLED_FEATURES],
|
|
50
96
|
nanogpt_visible: [...PRIVATE_MODE_NANOGPT_VISIBLE],
|
|
51
97
|
encrypted_from_nanogpt: [...PRIVATE_MODE_ENCRYPTED_FROM_NANOGPT],
|
|
52
98
|
plaintext_visible_to: [...PRIVATE_MODE_PLAINTEXT_VISIBLE_TO],
|
|
99
|
+
assurance: {
|
|
100
|
+
contract_version: 1,
|
|
101
|
+
assurance_level: 'private_mode',
|
|
102
|
+
transport: 'ehbp',
|
|
103
|
+
verification_status: verificationStatus,
|
|
104
|
+
eligible: verificationStatus === 'verified',
|
|
105
|
+
verified_by: verificationState.verifiedAt || verificationStatus !== 'not_checked'
|
|
106
|
+
? ['local_proxy:tinfoil-js']
|
|
107
|
+
: [],
|
|
108
|
+
...(verificationState.verifiedAt ? { verified_at: verificationState.verifiedAt } : {}),
|
|
109
|
+
...(verificationState.expiresAt ? { expires_at: verificationState.expiresAt } : {}),
|
|
110
|
+
...(Number.isFinite(verificationState.freshnessSeconds)
|
|
111
|
+
? { freshness_seconds: Math.max(0, Math.floor(verificationState.freshnessSeconds)) }
|
|
112
|
+
: {}),
|
|
113
|
+
...(stringIdentifier(document.codeFingerprint)
|
|
114
|
+
? { expected_measurement: stringIdentifier(document.codeFingerprint) }
|
|
115
|
+
: {}),
|
|
116
|
+
...(stringIdentifier(document.enclaveFingerprint)
|
|
117
|
+
? { observed_measurement: stringIdentifier(document.enclaveFingerprint) }
|
|
118
|
+
: {}),
|
|
119
|
+
...(stringIdentifier(document.releaseDigest)
|
|
120
|
+
? { release_digest: stringIdentifier(document.releaseDigest) }
|
|
121
|
+
: {}),
|
|
122
|
+
...(keyFingerprint ? { public_key_fingerprint: keyFingerprint } : {}),
|
|
123
|
+
receipt_available: true,
|
|
124
|
+
verifier: { name: 'tinfoil-js', version: '1.1.12' },
|
|
125
|
+
documentation_url: 'https://nano-gpt.com/private-mode-verification',
|
|
126
|
+
known_limitations: [...PRIVATE_MODE_KNOWN_LIMITATIONS],
|
|
127
|
+
...(verificationStatus !== 'verified' && verificationStatus !== 'not_checked'
|
|
128
|
+
? { error_code: verificationStatus }
|
|
129
|
+
: {}),
|
|
130
|
+
},
|
|
53
131
|
};
|
|
54
132
|
}
|
package/lib/verifyReceipt.js
CHANGED
|
@@ -2,6 +2,9 @@ import { createHash } from 'node:crypto';
|
|
|
2
2
|
import { readFile } from 'node:fs/promises';
|
|
3
3
|
import { fetchAttestationBundle, Verifier } from 'tinfoil';
|
|
4
4
|
|
|
5
|
+
const PRIVATE_MODE_ATTESTATION_MAX_AGE_MS = 5 * 60 * 1000;
|
|
6
|
+
const PRIVATE_MODE_RECEIPT_CLOCK_SKEW_MS = 60 * 1000;
|
|
7
|
+
|
|
5
8
|
function isRecord(value) {
|
|
6
9
|
return Boolean(value) && typeof value === 'object' && !Array.isArray(value);
|
|
7
10
|
}
|
|
@@ -84,7 +87,7 @@ function assertReceiptFlag(failures, label, value) {
|
|
|
84
87
|
}
|
|
85
88
|
|
|
86
89
|
export async function verifyPrivateModeReceipt(receipt) {
|
|
87
|
-
if (!isRecord(receipt) || receipt.schemaVersion !== 1) {
|
|
90
|
+
if (!isRecord(receipt) || (receipt.schemaVersion !== 1 && receipt.schemaVersion !== 2)) {
|
|
88
91
|
throw new Error('Unsupported or invalid Private Mode receipt.');
|
|
89
92
|
}
|
|
90
93
|
|
|
@@ -113,6 +116,29 @@ export async function verifyPrivateModeReceipt(receipt) {
|
|
|
113
116
|
if (receipt.status !== 'verified') {
|
|
114
117
|
failures.push(`Receipt status is ${receipt.status || 'missing'}; expected verified.`);
|
|
115
118
|
}
|
|
119
|
+
if (receipt.schemaVersion === 2) {
|
|
120
|
+
if (receipt.assuranceLevel !== 'private_mode') {
|
|
121
|
+
failures.push('Receipt assurance level is not private_mode.');
|
|
122
|
+
}
|
|
123
|
+
const verifiedAt = Date.parse(receipt.verifiedAt);
|
|
124
|
+
const expiresAt = Date.parse(receipt.expiresAt);
|
|
125
|
+
const now = Date.now();
|
|
126
|
+
if (
|
|
127
|
+
!Number.isFinite(verifiedAt)
|
|
128
|
+
|| !Number.isFinite(expiresAt)
|
|
129
|
+
|| expiresAt <= verifiedAt
|
|
130
|
+
|| expiresAt - verifiedAt > PRIVATE_MODE_ATTESTATION_MAX_AGE_MS
|
|
131
|
+
|| verifiedAt > now + PRIVATE_MODE_RECEIPT_CLOCK_SKEW_MS
|
|
132
|
+
) {
|
|
133
|
+
failures.push('Receipt freshness timestamps are invalid.');
|
|
134
|
+
} else if (expiresAt <= now) {
|
|
135
|
+
warnings.push('Receipt attestation freshness window has expired; fresh evidence was verified again now.');
|
|
136
|
+
}
|
|
137
|
+
const requestIdSha256 = receipt.response?.requestIdSha256;
|
|
138
|
+
if (requestIdSha256 !== undefined && !/^[a-f0-9]{64}$/i.test(requestIdSha256)) {
|
|
139
|
+
failures.push('Request identifier binding is not a SHA-256 digest.');
|
|
140
|
+
}
|
|
141
|
+
}
|
|
116
142
|
|
|
117
143
|
const encryption = isRecord(receipt.encryption) ? receipt.encryption : {};
|
|
118
144
|
if (encryption.transport !== 'EHBP') {
|
|
@@ -126,6 +152,16 @@ export async function verifyPrivateModeReceipt(receipt) {
|
|
|
126
152
|
assertRequiredEqual(failures, 'Runtime measurement', doc?.enclaveFingerprint, receipt.verifier?.enclaveFingerprint);
|
|
127
153
|
assertRequiredEqual(failures, 'Release digest', doc?.releaseDigest, receipt.verifier?.releaseDigest);
|
|
128
154
|
assertRequiredEqual(failures, 'HPKE public key', doc?.hpkePublicKey, receipt.verifier?.hpkePublicKey);
|
|
155
|
+
if (receipt.schemaVersion === 2) {
|
|
156
|
+
assertRequiredEqual(
|
|
157
|
+
failures,
|
|
158
|
+
'HPKE public-key fingerprint',
|
|
159
|
+
isNonEmptyString(doc?.hpkePublicKey)
|
|
160
|
+
? createHash('sha256').update(doc.hpkePublicKey).digest('hex')
|
|
161
|
+
: undefined,
|
|
162
|
+
receipt.verifier?.hpkePublicKeySha256,
|
|
163
|
+
);
|
|
164
|
+
}
|
|
129
165
|
assertOptionalEqual(warnings, 'Attestation bundle SHA-256', attestationBundleSha256, receipt.verifier?.attestationBundleSha256);
|
|
130
166
|
|
|
131
167
|
return {
|
|
@@ -137,6 +173,7 @@ export async function verifyPrivateModeReceipt(receipt) {
|
|
|
137
173
|
enclaveFingerprint: doc?.enclaveFingerprint,
|
|
138
174
|
attestationBundleSha256,
|
|
139
175
|
verificationDocumentSha256,
|
|
176
|
+
schemaVersion: receipt.schemaVersion,
|
|
140
177
|
};
|
|
141
178
|
}
|
|
142
179
|
|
|
@@ -155,6 +192,7 @@ export async function verifyPrivateModeReceiptCli(positionals, io = process) {
|
|
|
155
192
|
}
|
|
156
193
|
|
|
157
194
|
io.stdout.write('Private Mode receipt verified.\n');
|
|
195
|
+
io.stdout.write(`Receipt schema: v${result.schemaVersion}\n`);
|
|
158
196
|
io.stdout.write(`Enclave: ${result.enclaveURL}\n`);
|
|
159
197
|
io.stdout.write(`Expected measurement: ${result.codeFingerprint}\n`);
|
|
160
198
|
io.stdout.write(`Runtime measurement: ${result.enclaveFingerprint}\n`);
|
package/models/private-tee.json
CHANGED
|
@@ -1,11 +1,13 @@
|
|
|
1
1
|
[
|
|
2
2
|
{
|
|
3
3
|
"id": "private/deepseek-v4-flash",
|
|
4
|
-
"name": "DeepSeek V4 Flash Private",
|
|
4
|
+
"name": "DeepSeek V4 Flash 0731 Private",
|
|
5
5
|
"upstreamModel": "deepseek-v4-flash",
|
|
6
6
|
"billingModel": "private/deepseek-v4-flash",
|
|
7
|
+
"providerPricingModel": "TEE/deepseek-v4-flash",
|
|
7
8
|
"teeTargetModel": "deepseek-v4-flash",
|
|
8
9
|
"thinkingMode": "deepseek-v4",
|
|
10
|
+
"maxInputTokens": 1048576,
|
|
9
11
|
"maxOutputTokens": 1048576,
|
|
10
12
|
"created": 1786406400,
|
|
11
13
|
"ownedBy": "nanogpt-private-mode",
|
|
@@ -16,8 +18,10 @@
|
|
|
16
18
|
"name": "Kimi K3 Private",
|
|
17
19
|
"upstreamModel": "kimi-k3",
|
|
18
20
|
"billingModel": "TEE/kimi-k3",
|
|
21
|
+
"providerPricingModel": "TEE/kimi-k3",
|
|
19
22
|
"teeTargetModel": "kimi-k3",
|
|
20
|
-
"
|
|
23
|
+
"maxInputTokens": 256000,
|
|
24
|
+
"maxOutputTokens": 256000,
|
|
21
25
|
"created": 1786147200,
|
|
22
26
|
"ownedBy": "nanogpt-private-mode",
|
|
23
27
|
"aliases": ["TEE/kimi-k3"]
|
|
@@ -27,6 +31,7 @@
|
|
|
27
31
|
"name": "GPT OSS 120B Private",
|
|
28
32
|
"upstreamModel": "gpt-oss-120b",
|
|
29
33
|
"billingModel": "TEE/gpt-oss-120b",
|
|
34
|
+
"providerPricingModel": "openai/gpt-oss-120b",
|
|
30
35
|
"teeTargetModel": "gpt-oss-120b",
|
|
31
36
|
"created": 1764547200,
|
|
32
37
|
"ownedBy": "nanogpt-private-mode",
|
|
@@ -37,6 +42,7 @@
|
|
|
37
42
|
"name": "Llama 3.3 70B Private",
|
|
38
43
|
"upstreamModel": "llama3-3-70b",
|
|
39
44
|
"billingModel": "TEE/llama3-3-70b",
|
|
45
|
+
"providerPricingModel": "meta-llama/llama-3.3-70b-instruct",
|
|
40
46
|
"teeTargetModel": "llama3-3-70b",
|
|
41
47
|
"created": 1764547200,
|
|
42
48
|
"ownedBy": "nanogpt-private-mode",
|
|
@@ -47,8 +53,11 @@
|
|
|
47
53
|
"name": "GLM 5.2 Private",
|
|
48
54
|
"upstreamModel": "glm-5-2",
|
|
49
55
|
"billingModel": "TEE/glm-5-2",
|
|
56
|
+
"providerPricingModel": "TEE/glm-5.2",
|
|
50
57
|
"teeTargetModel": "glm-5-2",
|
|
51
58
|
"thinkingMode": "glm-5.2",
|
|
59
|
+
"maxInputTokens": 393216,
|
|
60
|
+
"maxOutputTokens": 131072,
|
|
52
61
|
"created": 1781827200,
|
|
53
62
|
"ownedBy": "nanogpt-private-mode",
|
|
54
63
|
"aliases": ["private/glm-5.2", "TEE/glm-5-2", "TEE/glm-5.2"]
|
|
@@ -58,8 +67,11 @@
|
|
|
58
67
|
"name": "GLM 5.2 Thinking Private",
|
|
59
68
|
"upstreamModel": "glm-5-2",
|
|
60
69
|
"billingModel": "TEE/glm-5-2:thinking",
|
|
70
|
+
"providerPricingModel": "TEE/glm-5.2:thinking",
|
|
61
71
|
"teeTargetModel": "glm-5-2",
|
|
62
72
|
"thinkingMode": "glm-5.2",
|
|
73
|
+
"maxInputTokens": 393216,
|
|
74
|
+
"maxOutputTokens": 131072,
|
|
63
75
|
"created": 1781827200,
|
|
64
76
|
"ownedBy": "nanogpt-private-mode",
|
|
65
77
|
"aliases": ["private/glm-5.2:thinking", "TEE/glm-5-2:thinking", "TEE/glm-5.2:thinking"]
|
|
@@ -69,6 +81,7 @@
|
|
|
69
81
|
"name": "Gemma 4 31B Private",
|
|
70
82
|
"upstreamModel": "gemma4-31b",
|
|
71
83
|
"billingModel": "TEE/gemma4-31b",
|
|
84
|
+
"providerPricingModel": "TEE/gemma4-31b",
|
|
72
85
|
"teeTargetModel": "gemma4-31b",
|
|
73
86
|
"thinkingMode": "gemma",
|
|
74
87
|
"created": 1764547200,
|
|
@@ -80,6 +93,7 @@
|
|
|
80
93
|
"name": "Gemma 4 31B Thinking Private",
|
|
81
94
|
"upstreamModel": "gemma4-31b",
|
|
82
95
|
"billingModel": "TEE/gemma4-31b:thinking",
|
|
96
|
+
"providerPricingModel": "TEE/gemma4-31b:thinking",
|
|
83
97
|
"teeTargetModel": "gemma4-31b",
|
|
84
98
|
"thinkingMode": "gemma",
|
|
85
99
|
"created": 1764547200,
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@nanogpt/private-mode",
|
|
3
|
-
"version": "0.2.
|
|
3
|
+
"version": "0.2.11",
|
|
4
4
|
"description": "OpenAI-compatible localhost proxy for NanoGPT Private Mode.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"publishConfig": {
|
|
@@ -13,11 +13,15 @@
|
|
|
13
13
|
"files": [
|
|
14
14
|
"bin",
|
|
15
15
|
"lib/cacheScope.js",
|
|
16
|
+
"lib/catalogCompatibility.js",
|
|
17
|
+
"lib/constants.js",
|
|
16
18
|
"lib/originPolicy.js",
|
|
19
|
+
"lib/packageInfo.js",
|
|
17
20
|
"lib/requestTransforms.js",
|
|
18
21
|
"lib/secureClientLifecycle.js",
|
|
19
22
|
"lib/server.js",
|
|
20
23
|
"lib/serverErrorNormalization.js",
|
|
24
|
+
"lib/sseResponsePump.js",
|
|
21
25
|
"lib/statusContract.js",
|
|
22
26
|
"lib/verifyReceipt.js",
|
|
23
27
|
"models",
|