@nanogpt/private-mode 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +85 -0
- package/bin/nanogpt-private-mode.js +73 -0
- package/lib/originPolicy.js +115 -0
- package/lib/requestTransforms.js +227 -0
- package/lib/server.js +503 -0
- package/lib/statusContract.js +54 -0
- package/lib/verifyReceipt.js +170 -0
- package/models/tinfoil.json +96 -0
- package/package.json +32 -0
package/README.md
ADDED
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
# NanoGPT Private Mode
|
|
2
|
+
|
|
3
|
+
OpenAI-compatible localhost proxy for NanoGPT Private Mode with Tinfoil-backed models.
|
|
4
|
+
|
|
5
|
+
```bash
|
|
6
|
+
NANOGPT_API_KEY=sk-your-key npx @nanogpt/private-mode
|
|
7
|
+
```
|
|
8
|
+
|
|
9
|
+
Then point any OpenAI-compatible client at:
|
|
10
|
+
|
|
11
|
+
```text
|
|
12
|
+
http://127.0.0.1:8787/v1
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
Example:
|
|
16
|
+
|
|
17
|
+
```ts
|
|
18
|
+
import OpenAI from "openai";
|
|
19
|
+
|
|
20
|
+
const client = new OpenAI({
|
|
21
|
+
baseURL: "http://127.0.0.1:8787/v1",
|
|
22
|
+
apiKey: "unused",
|
|
23
|
+
});
|
|
24
|
+
|
|
25
|
+
const response = await client.chat.completions.create({
|
|
26
|
+
model: "private/kimi-k2-6",
|
|
27
|
+
messages: [{ role: "user", content: "Hello" }],
|
|
28
|
+
});
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
The local proxy verifies Tinfoil attestation, encrypts request bodies with EHBP, sends ciphertext through NanoGPT, decrypts encrypted responses locally, and returns normal OpenAI JSON to the calling app.
|
|
32
|
+
|
|
33
|
+
NanoGPT can see account identity, selected private model, selected Tinfoil enclave metadata, timing, sizes, status, and usage metadata. NanoGPT cannot read the prompt or completion body for supported private models.
|
|
34
|
+
|
|
35
|
+
NanoGPT's web app can also use these models without running this local proxy. Select an eligible Tinfoil-backed `TEE/*` model and use the Private Mode control in the model picker. This package is for API clients, CLIs, agents, and other OpenAI-compatible tools.
|
|
36
|
+
|
|
37
|
+
The web-app toggle is narrower than the proxy path in v1. It supports text chat and model settings, and disables attachments, web search, URL-scraped content, project tools, multi-model chat, Context Memory injection, quick replies, and automatic title generation for private turns.
|
|
38
|
+
|
|
39
|
+
In the hosted web app, decrypted Private Mode turns remain in local browser history. Cloud conversation sync is blocked for Private Mode chats unless password-based end-to-end sync is enabled; the recoverable default sync mode is not used for those chats.
|
|
40
|
+
|
|
41
|
+
Private Mode also requires enough NanoGPT balance and API-key spend-limit headroom before dispatch. NanoGPT cannot read encrypted prompts to count tokens, so the reserve uses the encrypted request size when available and otherwise falls back to a conservative model estimate capped at 32,768 output tokens by default (`NANOGPT_PRIVATE_TINFOIL_RESERVE_MAX_OUTPUT_TOKENS`). The final charge is still based on Tinfoil usage metrics.
|
|
42
|
+
|
|
43
|
+
Useful local checks:
|
|
44
|
+
|
|
45
|
+
```text
|
|
46
|
+
GET http://127.0.0.1:8787/v1/models
|
|
47
|
+
GET http://127.0.0.1:8787/v1/private-mode/status
|
|
48
|
+
GET http://127.0.0.1:8787/v1/private-mode/attestation
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
Browser requests are locked down by default. The proxy only accepts same-machine clients and rejects browser `Origin` headers that are not explicitly allowed, so a random website or LAN client cannot spend the local `NANOGPT_API_KEY` while the proxy is running. If a local browser app needs to call the proxy directly, allow that exact origin:
|
|
52
|
+
|
|
53
|
+
```bash
|
|
54
|
+
NANOGPT_API_KEY=sk-your-key npx @nanogpt/private-mode --allow-origin http://localhost:3000
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
You can also set `NANOGPT_PRIVATE_ALLOWED_ORIGINS=http://localhost:3000`. Wildcard browser origins are not supported.
|
|
58
|
+
|
|
59
|
+
The hosted web app streams Private Mode chat by default. The local proxy preserves OpenAI API semantics: omitted `stream` returns a single JSON response, and `stream: true` returns server-sent events. For streaming calls, NanoGPT precharges a reserve before dispatch so interrupted streams remain billable, then refunds unused reserve when verified usage metadata is available at the end of the stream.
|
|
60
|
+
|
|
61
|
+
## Verification receipts
|
|
62
|
+
|
|
63
|
+
NanoGPT's hosted web app shows a Private Mode receipt on completed Private Mode responses. The receipt includes:
|
|
64
|
+
|
|
65
|
+
- browser attestation result
|
|
66
|
+
- TEE target and enclave host
|
|
67
|
+
- verifier step statuses
|
|
68
|
+
- expected and runtime measurement fingerprints
|
|
69
|
+
- attestation bundle SHA-256 when available
|
|
70
|
+
- verification document SHA-256
|
|
71
|
+
- request/response encryption flags
|
|
72
|
+
|
|
73
|
+
To independently verify a copied receipt:
|
|
74
|
+
|
|
75
|
+
```bash
|
|
76
|
+
npx @nanogpt/private-mode verify receipt.json
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
You can also pipe JSON on stdin:
|
|
80
|
+
|
|
81
|
+
```bash
|
|
82
|
+
cat receipt.json | npx @nanogpt/private-mode verify
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
The verifier re-fetches attestation material for the receipt's enclave, verifies it locally, and checks the resulting measurements, release digest, and HPKE key against the copied receipt. If an optional attestation bundle hash is present but the freshly fetched bundle serializes differently, the verifier prints a warning instead of failing the core verification.
|
|
@@ -0,0 +1,73 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
|
|
3
|
+
import { parseArgs } from 'node:util';
|
|
4
|
+
import { startPrivateModeProxy } from '../lib/server.js';
|
|
5
|
+
import { verifyPrivateModeReceiptCli } from '../lib/verifyReceipt.js';
|
|
6
|
+
|
|
7
|
+
const { values, positionals } = parseArgs({
|
|
8
|
+
options: {
|
|
9
|
+
host: { type: 'string', default: process.env.HOST || '127.0.0.1' },
|
|
10
|
+
port: { type: 'string', short: 'p', default: process.env.PORT || '8787' },
|
|
11
|
+
'api-base': {
|
|
12
|
+
type: 'string',
|
|
13
|
+
default: process.env.NANOGPT_API_BASE || 'https://nano-gpt.com',
|
|
14
|
+
},
|
|
15
|
+
'allow-origin': { type: 'string', multiple: true },
|
|
16
|
+
help: { type: 'boolean', short: 'h', default: false },
|
|
17
|
+
},
|
|
18
|
+
allowPositionals: true,
|
|
19
|
+
});
|
|
20
|
+
|
|
21
|
+
if (values.help) {
|
|
22
|
+
process.stdout.write(`NanoGPT Private Mode Proxy
|
|
23
|
+
|
|
24
|
+
Usage:
|
|
25
|
+
NANOGPT_API_KEY=sk-... nanogpt-private-mode
|
|
26
|
+
nanogpt-private-mode verify receipt.json
|
|
27
|
+
cat receipt.json | nanogpt-private-mode verify
|
|
28
|
+
|
|
29
|
+
Options:
|
|
30
|
+
--host <host> Local bind host. Default: 127.0.0.1
|
|
31
|
+
--port, -p <port> Local port. Default: 8787
|
|
32
|
+
--api-base <url> NanoGPT API base. Default: https://nano-gpt.com
|
|
33
|
+
--allow-origin <origin>
|
|
34
|
+
Browser origin allowed to call the proxy, for example http://localhost:3000
|
|
35
|
+
`);
|
|
36
|
+
process.exit(0);
|
|
37
|
+
}
|
|
38
|
+
|
|
39
|
+
if (positionals[0] === 'verify') {
|
|
40
|
+
try {
|
|
41
|
+
const exitCode = await verifyPrivateModeReceiptCli(positionals.slice(1));
|
|
42
|
+
process.exit(exitCode);
|
|
43
|
+
} catch (error) {
|
|
44
|
+
process.stderr.write(`${error instanceof Error ? error.message : String(error)}\n`);
|
|
45
|
+
process.exit(1);
|
|
46
|
+
}
|
|
47
|
+
} else if (positionals.length > 0) {
|
|
48
|
+
process.stderr.write(`Unknown command: ${positionals[0]}\n`);
|
|
49
|
+
process.exit(1);
|
|
50
|
+
}
|
|
51
|
+
|
|
52
|
+
const port = Number.parseInt(String(values.port), 10);
|
|
53
|
+
if (!Number.isFinite(port) || port <= 0 || port > 65535) {
|
|
54
|
+
process.stderr.write('Invalid --port value.\n');
|
|
55
|
+
process.exit(1);
|
|
56
|
+
}
|
|
57
|
+
|
|
58
|
+
const apiKey = process.env.NANOGPT_API_KEY?.trim();
|
|
59
|
+
if (!apiKey) {
|
|
60
|
+
process.stderr.write('Set NANOGPT_API_KEY before starting the proxy.\n');
|
|
61
|
+
process.exit(1);
|
|
62
|
+
}
|
|
63
|
+
|
|
64
|
+
startPrivateModeProxy({
|
|
65
|
+
apiBase: String(values['api-base']),
|
|
66
|
+
apiKey,
|
|
67
|
+
host: String(values.host),
|
|
68
|
+
port,
|
|
69
|
+
allowedOrigins: values['allow-origin'],
|
|
70
|
+
}).catch((error) => {
|
|
71
|
+
process.stderr.write(`${error instanceof Error ? error.message : String(error)}\n`);
|
|
72
|
+
process.exit(1);
|
|
73
|
+
});
|
|
@@ -0,0 +1,115 @@
|
|
|
1
|
+
const DEFAULT_LOCAL_HOSTS = ['127.0.0.1', 'localhost', '[::1]'];
|
|
2
|
+
|
|
3
|
+
export function normalizePrivateModeOrigin(value) {
|
|
4
|
+
const origin = String(value || '').trim();
|
|
5
|
+
if (!origin || origin === 'null' || origin === '*') return null;
|
|
6
|
+
try {
|
|
7
|
+
const parsed = new URL(origin);
|
|
8
|
+
if (parsed.protocol !== 'http:' && parsed.protocol !== 'https:') return null;
|
|
9
|
+
return parsed.origin;
|
|
10
|
+
} catch {
|
|
11
|
+
return null;
|
|
12
|
+
}
|
|
13
|
+
}
|
|
14
|
+
|
|
15
|
+
function splitOriginValues(values) {
|
|
16
|
+
if (!values) return [];
|
|
17
|
+
const list = Array.isArray(values) ? values : [values];
|
|
18
|
+
return list
|
|
19
|
+
.flatMap((value) => String(value || '').split(','))
|
|
20
|
+
.map((value) => value.trim())
|
|
21
|
+
.filter(Boolean);
|
|
22
|
+
}
|
|
23
|
+
|
|
24
|
+
function normalizeHostForOrigin(host) {
|
|
25
|
+
const trimmed = String(host || '').trim();
|
|
26
|
+
if (!trimmed || trimmed === '0.0.0.0' || trimmed === '::' || trimmed === '[::]') return null;
|
|
27
|
+
if (trimmed.includes(':') && !trimmed.startsWith('[')) return `[${trimmed}]`;
|
|
28
|
+
return trimmed;
|
|
29
|
+
}
|
|
30
|
+
|
|
31
|
+
export function buildPrivateModeOriginPolicy(options = {}) {
|
|
32
|
+
const port = Number.parseInt(String(options.port || '8787'), 10);
|
|
33
|
+
const allowedOrigins = new Set();
|
|
34
|
+
|
|
35
|
+
if (Number.isFinite(port) && port > 0) {
|
|
36
|
+
for (const host of DEFAULT_LOCAL_HOSTS) {
|
|
37
|
+
allowedOrigins.add(`http://${host}:${port}`);
|
|
38
|
+
}
|
|
39
|
+
|
|
40
|
+
const normalizedHost = normalizeHostForOrigin(options.host);
|
|
41
|
+
if (normalizedHost) {
|
|
42
|
+
const origin = normalizePrivateModeOrigin(`http://${normalizedHost}:${port}`);
|
|
43
|
+
if (origin) allowedOrigins.add(origin);
|
|
44
|
+
}
|
|
45
|
+
}
|
|
46
|
+
|
|
47
|
+
const configuredOrigins = [
|
|
48
|
+
...splitOriginValues(process.env.NANOGPT_PRIVATE_ALLOWED_ORIGINS),
|
|
49
|
+
...splitOriginValues(options.allowedOrigins),
|
|
50
|
+
];
|
|
51
|
+
|
|
52
|
+
for (const configuredOrigin of configuredOrigins) {
|
|
53
|
+
const origin = normalizePrivateModeOrigin(configuredOrigin);
|
|
54
|
+
if (!origin) {
|
|
55
|
+
throw new Error(`Invalid Private Mode browser origin "${configuredOrigin}". Use an http(s) origin, not a wildcard.`);
|
|
56
|
+
}
|
|
57
|
+
allowedOrigins.add(origin);
|
|
58
|
+
}
|
|
59
|
+
|
|
60
|
+
return {
|
|
61
|
+
allowedOrigins: [...allowedOrigins],
|
|
62
|
+
};
|
|
63
|
+
}
|
|
64
|
+
|
|
65
|
+
export function isPrivateModeOriginAllowed(originPolicy, origin) {
|
|
66
|
+
if (!origin) return false;
|
|
67
|
+
return getAllowedPrivateModeOrigin(originPolicy, origin) !== null;
|
|
68
|
+
}
|
|
69
|
+
|
|
70
|
+
function getAllowedPrivateModeOrigin(originPolicy, origin) {
|
|
71
|
+
const normalizedOrigin = normalizePrivateModeOrigin(origin);
|
|
72
|
+
if (!normalizedOrigin || !originPolicy.allowedOrigins.includes(normalizedOrigin)) return null;
|
|
73
|
+
return normalizedOrigin;
|
|
74
|
+
}
|
|
75
|
+
|
|
76
|
+
export function isLoopbackRemoteAddress(remoteAddress) {
|
|
77
|
+
const normalized = String(remoteAddress || '').trim().toLowerCase();
|
|
78
|
+
if (!normalized) return false;
|
|
79
|
+
if (normalized === '::1' || normalized === '0:0:0:0:0:0:0:1') return true;
|
|
80
|
+
|
|
81
|
+
const ipv4 = normalized.startsWith('::ffff:')
|
|
82
|
+
? normalized.slice('::ffff:'.length)
|
|
83
|
+
: normalized;
|
|
84
|
+
const parts = ipv4.split('.');
|
|
85
|
+
if (parts.length !== 4) return false;
|
|
86
|
+
const octets = parts.map((part) => Number(part));
|
|
87
|
+
if (!octets.every((octet, index) => (
|
|
88
|
+
Number.isInteger(octet) &&
|
|
89
|
+
octet >= 0 &&
|
|
90
|
+
octet <= 255 &&
|
|
91
|
+
String(octet) === parts[index]
|
|
92
|
+
))) {
|
|
93
|
+
return false;
|
|
94
|
+
}
|
|
95
|
+
|
|
96
|
+
return octets[0] === 127;
|
|
97
|
+
}
|
|
98
|
+
|
|
99
|
+
export function getCorsHeadersForOrigin(originPolicy, origin) {
|
|
100
|
+
if (!origin) return {};
|
|
101
|
+
|
|
102
|
+
const normalizedOrigin = getAllowedPrivateModeOrigin(originPolicy, origin);
|
|
103
|
+
if (!normalizedOrigin) return null;
|
|
104
|
+
|
|
105
|
+
return {
|
|
106
|
+
'access-control-allow-origin': normalizedOrigin,
|
|
107
|
+
vary: 'Origin',
|
|
108
|
+
};
|
|
109
|
+
}
|
|
110
|
+
|
|
111
|
+
export function getCorsHeadersForRequest(req, originPolicy) {
|
|
112
|
+
if (!isLoopbackRemoteAddress(req.socket?.remoteAddress)) return null;
|
|
113
|
+
const origin = typeof req.headers.origin === 'string' ? req.headers.origin : '';
|
|
114
|
+
return getCorsHeadersForOrigin(originPolicy, origin);
|
|
115
|
+
}
|
|
@@ -0,0 +1,227 @@
|
|
|
1
|
+
const MODELS_REQUIRING_TOOL_CHOICE_REQUIRED = new Set([
|
|
2
|
+
'phala/qwen-2.5-7b-instruct',
|
|
3
|
+
]);
|
|
4
|
+
const ASSISTANT_REASONING_MESSAGE_FIELDS = [
|
|
5
|
+
'reasoning',
|
|
6
|
+
'reasoning_content',
|
|
7
|
+
'reasoning_details',
|
|
8
|
+
];
|
|
9
|
+
|
|
10
|
+
// Keep request mutation behavior in sync with lib/privateMode/tinfoilBrowserClient.ts
|
|
11
|
+
// for explicit fields. The local proxy preserves OpenAI's non-streaming default.
|
|
12
|
+
function isPlainObject(value) {
|
|
13
|
+
return Boolean(value) && typeof value === 'object' && !Array.isArray(value);
|
|
14
|
+
}
|
|
15
|
+
|
|
16
|
+
function isThinkingEnabled(thinking) {
|
|
17
|
+
if (typeof thinking === 'boolean') return thinking;
|
|
18
|
+
if (!isPlainObject(thinking)) return undefined;
|
|
19
|
+
|
|
20
|
+
const type = typeof thinking.type === 'string'
|
|
21
|
+
? thinking.type.toLowerCase()
|
|
22
|
+
: undefined;
|
|
23
|
+
if (type === 'enabled') return true;
|
|
24
|
+
if (type === 'disabled') return false;
|
|
25
|
+
|
|
26
|
+
if (typeof thinking.enabled === 'boolean') {
|
|
27
|
+
return thinking.enabled;
|
|
28
|
+
}
|
|
29
|
+
|
|
30
|
+
return undefined;
|
|
31
|
+
}
|
|
32
|
+
|
|
33
|
+
function isReasoningEnabled(reasoning) {
|
|
34
|
+
if (typeof reasoning === 'boolean') return reasoning;
|
|
35
|
+
if (!isPlainObject(reasoning)) return undefined;
|
|
36
|
+
|
|
37
|
+
if (reasoning.exclude === true || reasoning.enabled === false) return false;
|
|
38
|
+
if (reasoning.enabled === true) return true;
|
|
39
|
+
if (typeof reasoning.effort === 'string') {
|
|
40
|
+
return reasoning.effort.toLowerCase() !== 'none';
|
|
41
|
+
}
|
|
42
|
+
|
|
43
|
+
return undefined;
|
|
44
|
+
}
|
|
45
|
+
|
|
46
|
+
function normalizeDeepSeekV4ReasoningEffort(value) {
|
|
47
|
+
const normalized = typeof value === 'string' ? value.trim().toLowerCase() : '';
|
|
48
|
+
if (normalized === 'max' || normalized === 'xhigh') return 'max';
|
|
49
|
+
if (normalized === 'medium') return 'medium';
|
|
50
|
+
if (normalized === 'low' || normalized === 'minimal') return 'low';
|
|
51
|
+
return 'high';
|
|
52
|
+
}
|
|
53
|
+
|
|
54
|
+
function getPrivateModeModelSignals(body, model) {
|
|
55
|
+
return [
|
|
56
|
+
body.model,
|
|
57
|
+
model.id,
|
|
58
|
+
model.billingModel,
|
|
59
|
+
...(model.aliases || []),
|
|
60
|
+
].map((value) => String(value || '').toLowerCase());
|
|
61
|
+
}
|
|
62
|
+
|
|
63
|
+
function shouldEnableThinking(body, model) {
|
|
64
|
+
const explicitThinking = isThinkingEnabled(body.thinking);
|
|
65
|
+
const explicitReasoning = isReasoningEnabled(body.reasoning);
|
|
66
|
+
const reasoningEffort = String(body.reasoning_effort || '').toLowerCase();
|
|
67
|
+
const modelSignals = getPrivateModeModelSignals(body, model);
|
|
68
|
+
const hasThinkingSuffix = modelSignals.some((value) => (
|
|
69
|
+
value.includes(':thinking') || value.endsWith('-thinking')
|
|
70
|
+
));
|
|
71
|
+
|
|
72
|
+
return explicitThinking
|
|
73
|
+
?? explicitReasoning
|
|
74
|
+
?? (body.reasoning_effort !== undefined ? reasoningEffort !== 'none' : undefined)
|
|
75
|
+
?? hasThinkingSuffix;
|
|
76
|
+
}
|
|
77
|
+
|
|
78
|
+
function mergeChatTemplateKwargs(body) {
|
|
79
|
+
return isPlainObject(body.chat_template_kwargs) ? body.chat_template_kwargs : {};
|
|
80
|
+
}
|
|
81
|
+
|
|
82
|
+
function stripPrivateModeReasoningBlocks(value) {
|
|
83
|
+
return value
|
|
84
|
+
.replace(/◁think▷/g, '<think>')
|
|
85
|
+
.replace(/◁\/think▷/g, '</think>')
|
|
86
|
+
.replace(/<think>[\s\S]*?<\/think>\s*/gi, '')
|
|
87
|
+
.replace(/<previous_reasoning>[\s\S]*?<\/previous_reasoning>\s*/gi, '')
|
|
88
|
+
.trimStart();
|
|
89
|
+
}
|
|
90
|
+
|
|
91
|
+
function stripPrivateModeReasoningFromContent(value) {
|
|
92
|
+
if (typeof value === 'string') return stripPrivateModeReasoningBlocks(value);
|
|
93
|
+
if (!Array.isArray(value)) return value;
|
|
94
|
+
|
|
95
|
+
return value.map((part) => {
|
|
96
|
+
if (!isPlainObject(part)) return part;
|
|
97
|
+
if (part.type === 'text' && typeof part.text === 'string') {
|
|
98
|
+
return {
|
|
99
|
+
...part,
|
|
100
|
+
text: stripPrivateModeReasoningBlocks(part.text),
|
|
101
|
+
};
|
|
102
|
+
}
|
|
103
|
+
return part;
|
|
104
|
+
});
|
|
105
|
+
}
|
|
106
|
+
|
|
107
|
+
function stripPrivateModeReasoningFromMessages(body) {
|
|
108
|
+
if (!Array.isArray(body.messages)) return;
|
|
109
|
+
|
|
110
|
+
body.messages = body.messages.map((message) => {
|
|
111
|
+
if (!isPlainObject(message) || message.role !== 'assistant') return message;
|
|
112
|
+
|
|
113
|
+
const sanitized = { ...message };
|
|
114
|
+
for (const field of ASSISTANT_REASONING_MESSAGE_FIELDS) {
|
|
115
|
+
delete sanitized[field];
|
|
116
|
+
}
|
|
117
|
+
if ('content' in sanitized) {
|
|
118
|
+
sanitized.content = stripPrivateModeReasoningFromContent(sanitized.content);
|
|
119
|
+
}
|
|
120
|
+
if ('prompt' in sanitized) {
|
|
121
|
+
sanitized.prompt = stripPrivateModeReasoningFromContent(sanitized.prompt);
|
|
122
|
+
}
|
|
123
|
+
return sanitized;
|
|
124
|
+
});
|
|
125
|
+
}
|
|
126
|
+
|
|
127
|
+
function buildJsonSchemaPromptSuffix(responseFormat) {
|
|
128
|
+
if (!isPlainObject(responseFormat?.json_schema)) return null;
|
|
129
|
+
const schemaWrapper = responseFormat.json_schema;
|
|
130
|
+
const schemaName = schemaWrapper.name || 'response';
|
|
131
|
+
const schema = schemaWrapper.schema || schemaWrapper;
|
|
132
|
+
return [
|
|
133
|
+
'',
|
|
134
|
+
'',
|
|
135
|
+
'<json_output_instructions>',
|
|
136
|
+
'CRITICAL: Respond with ONLY valid JSON. Do not use markdown code fences.',
|
|
137
|
+
`Required JSON Schema ("${schemaName}"):\n${JSON.stringify(schema, null, 2)}`,
|
|
138
|
+
'</json_output_instructions>',
|
|
139
|
+
].join('\n');
|
|
140
|
+
}
|
|
141
|
+
|
|
142
|
+
function appendSystemInstruction(body, instruction) {
|
|
143
|
+
if (typeof instruction !== 'string' || !instruction.trim()) return;
|
|
144
|
+
const messages = Array.isArray(body.messages) ? body.messages : [];
|
|
145
|
+
const firstMessage = messages[0];
|
|
146
|
+
if (
|
|
147
|
+
isPlainObject(firstMessage) &&
|
|
148
|
+
firstMessage.role === 'system' &&
|
|
149
|
+
typeof firstMessage.content === 'string'
|
|
150
|
+
) {
|
|
151
|
+
firstMessage.content = `${firstMessage.content}${instruction}`;
|
|
152
|
+
body.messages = messages;
|
|
153
|
+
return;
|
|
154
|
+
}
|
|
155
|
+
|
|
156
|
+
body.messages = [
|
|
157
|
+
{ role: 'system', content: instruction.trim() },
|
|
158
|
+
...messages,
|
|
159
|
+
];
|
|
160
|
+
}
|
|
161
|
+
|
|
162
|
+
function applyTinfoilCompatibilityMutations(body, model) {
|
|
163
|
+
if (body.response_format?.type === 'json_schema') {
|
|
164
|
+
appendSystemInstruction(body, buildJsonSchemaPromptSuffix(body.response_format));
|
|
165
|
+
body.response_format = { type: 'json_object' };
|
|
166
|
+
}
|
|
167
|
+
|
|
168
|
+
if (
|
|
169
|
+
Array.isArray(body.tools) &&
|
|
170
|
+
body.tools.length > 0 &&
|
|
171
|
+
MODELS_REQUIRING_TOOL_CHOICE_REQUIRED.has(model.upstreamModel) &&
|
|
172
|
+
(!body.tool_choice || body.tool_choice === 'auto')
|
|
173
|
+
) {
|
|
174
|
+
body.tool_choice = 'required';
|
|
175
|
+
}
|
|
176
|
+
}
|
|
177
|
+
|
|
178
|
+
export function applyPrivateModelRequestMutations(body, model) {
|
|
179
|
+
stripPrivateModeReasoningFromMessages(body);
|
|
180
|
+
|
|
181
|
+
if (body.max_tokens === undefined && body.maxTokens !== undefined) {
|
|
182
|
+
body.max_tokens = body.maxTokens;
|
|
183
|
+
}
|
|
184
|
+
delete body.maxTokens;
|
|
185
|
+
|
|
186
|
+
body.model = model.upstreamModel;
|
|
187
|
+
if (body.stream !== undefined) {
|
|
188
|
+
body.stream = body.stream === true;
|
|
189
|
+
}
|
|
190
|
+
if (body.stream === true) {
|
|
191
|
+
body.stream_options = {
|
|
192
|
+
...(body.stream_options && typeof body.stream_options === 'object' ? body.stream_options : {}),
|
|
193
|
+
include_usage: true,
|
|
194
|
+
};
|
|
195
|
+
}
|
|
196
|
+
applyTinfoilCompatibilityMutations(body, model);
|
|
197
|
+
|
|
198
|
+
if (model.thinkingMode === 'gemma') {
|
|
199
|
+
body.chat_template_kwargs = {
|
|
200
|
+
...mergeChatTemplateKwargs(body),
|
|
201
|
+
enable_thinking: shouldEnableThinking(body, model),
|
|
202
|
+
};
|
|
203
|
+
delete body.thinking;
|
|
204
|
+
delete body.reasoning_effort;
|
|
205
|
+
return body;
|
|
206
|
+
}
|
|
207
|
+
|
|
208
|
+
if (model.thinkingMode === 'deepseek-v4-pro') {
|
|
209
|
+
const thinkingEnabled = shouldEnableThinking(body, model);
|
|
210
|
+
body.chat_template_kwargs = {
|
|
211
|
+
...mergeChatTemplateKwargs(body),
|
|
212
|
+
thinking: thinkingEnabled,
|
|
213
|
+
};
|
|
214
|
+
|
|
215
|
+
if (thinkingEnabled) {
|
|
216
|
+
body.chat_template_kwargs.reasoning_effort =
|
|
217
|
+
normalizeDeepSeekV4ReasoningEffort(body.reasoning_effort);
|
|
218
|
+
} else {
|
|
219
|
+
delete body.chat_template_kwargs.reasoning_effort;
|
|
220
|
+
}
|
|
221
|
+
|
|
222
|
+
delete body.thinking;
|
|
223
|
+
delete body.reasoning_effort;
|
|
224
|
+
}
|
|
225
|
+
|
|
226
|
+
return body;
|
|
227
|
+
}
|
package/lib/server.js
ADDED
|
@@ -0,0 +1,503 @@
|
|
|
1
|
+
import { createServer } from 'node:http';
|
|
2
|
+
import { readFileSync } from 'node:fs';
|
|
3
|
+
|
|
4
|
+
import {
|
|
5
|
+
buildPrivateModeOriginPolicy,
|
|
6
|
+
getCorsHeadersForRequest,
|
|
7
|
+
} from './originPolicy.js';
|
|
8
|
+
import { applyPrivateModelRequestMutations } from './requestTransforms.js';
|
|
9
|
+
import { buildPrivateModeStatusContract } from './statusContract.js';
|
|
10
|
+
|
|
11
|
+
const MODELS = JSON.parse(
|
|
12
|
+
readFileSync(new URL('../models/tinfoil.json', import.meta.url), 'utf8'),
|
|
13
|
+
);
|
|
14
|
+
|
|
15
|
+
function readMaxBodyBytes() {
|
|
16
|
+
const parsed = Number(process.env.NANOGPT_PRIVATE_MAX_BODY_BYTES || '');
|
|
17
|
+
if (Number.isFinite(parsed) && parsed > 0) return Math.floor(parsed);
|
|
18
|
+
return 25 * 1024 * 1024;
|
|
19
|
+
}
|
|
20
|
+
|
|
21
|
+
const MAX_BODY_BYTES = readMaxBodyBytes();
|
|
22
|
+
|
|
23
|
+
function normalizeApiBase(apiBase) {
|
|
24
|
+
const parsed = new URL(apiBase);
|
|
25
|
+
if (parsed.protocol !== 'http:' && parsed.protocol !== 'https:') {
|
|
26
|
+
throw new Error('NANOGPT_API_BASE must be an http(s) URL.');
|
|
27
|
+
}
|
|
28
|
+
return parsed.toString().replace(/\/+$/, '');
|
|
29
|
+
}
|
|
30
|
+
|
|
31
|
+
function normalizeModelId(value) {
|
|
32
|
+
return String(value || '').trim().toLowerCase();
|
|
33
|
+
}
|
|
34
|
+
|
|
35
|
+
const MODEL_BY_ID = new Map();
|
|
36
|
+
for (const model of MODELS) {
|
|
37
|
+
const candidates = [
|
|
38
|
+
model.id,
|
|
39
|
+
model.id.replace(/^private\//, ''),
|
|
40
|
+
model.upstreamModel,
|
|
41
|
+
model.billingModel,
|
|
42
|
+
...(model.aliases || []),
|
|
43
|
+
];
|
|
44
|
+
for (const candidate of candidates) {
|
|
45
|
+
const normalized = normalizeModelId(candidate);
|
|
46
|
+
if (normalized && !MODEL_BY_ID.has(normalized)) MODEL_BY_ID.set(normalized, model);
|
|
47
|
+
}
|
|
48
|
+
}
|
|
49
|
+
|
|
50
|
+
export function resolveModel(modelId) {
|
|
51
|
+
return MODEL_BY_ID.get(normalizeModelId(modelId)) || null;
|
|
52
|
+
}
|
|
53
|
+
|
|
54
|
+
function openAIModelList() {
|
|
55
|
+
return {
|
|
56
|
+
object: 'list',
|
|
57
|
+
data: MODELS.map((model) => ({
|
|
58
|
+
id: model.id,
|
|
59
|
+
object: 'model',
|
|
60
|
+
created: model.created,
|
|
61
|
+
owned_by: model.ownedBy,
|
|
62
|
+
})),
|
|
63
|
+
};
|
|
64
|
+
}
|
|
65
|
+
|
|
66
|
+
export function privateModeStatus(apiBase, secureState, localBase, originPolicy) {
|
|
67
|
+
return {
|
|
68
|
+
ok: true,
|
|
69
|
+
mode: 'tinfoil',
|
|
70
|
+
apiBase,
|
|
71
|
+
local_base_url: localBase,
|
|
72
|
+
models_path: '/v1/models',
|
|
73
|
+
status_path: '/v1/private-mode/status',
|
|
74
|
+
attestation_path: '/v1/private-mode/attestation',
|
|
75
|
+
chat_completions_path: '/v1/chat/completions',
|
|
76
|
+
max_body_bytes: MAX_BODY_BYTES,
|
|
77
|
+
transport: 'ehbp',
|
|
78
|
+
request_body_encrypted: true,
|
|
79
|
+
response_body_encrypted: true,
|
|
80
|
+
streaming: true,
|
|
81
|
+
streaming_billing: 'precharged_reserve_with_verified_usage_refund',
|
|
82
|
+
api_local_proxy_required: true,
|
|
83
|
+
browser_frontend_local_proxy_required: false,
|
|
84
|
+
...buildPrivateModeStatusContract(),
|
|
85
|
+
browser_origins_allowed: originPolicy.allowedOrigins,
|
|
86
|
+
models: openAIModelList().data,
|
|
87
|
+
attestation: secureState.getVerificationState(),
|
|
88
|
+
};
|
|
89
|
+
}
|
|
90
|
+
|
|
91
|
+
function jsonResponse(res, status, body, headers = {}) {
|
|
92
|
+
const payload = Buffer.from(JSON.stringify(body));
|
|
93
|
+
res.writeHead(status, {
|
|
94
|
+
'content-type': 'application/json',
|
|
95
|
+
'content-length': String(payload.length),
|
|
96
|
+
...headers,
|
|
97
|
+
});
|
|
98
|
+
res.end(payload);
|
|
99
|
+
}
|
|
100
|
+
|
|
101
|
+
function textResponse(res, status, body, headers = {}) {
|
|
102
|
+
const payload = Buffer.from(body);
|
|
103
|
+
res.writeHead(status, {
|
|
104
|
+
'content-type': 'text/plain; charset=utf-8',
|
|
105
|
+
'content-length': String(payload.length),
|
|
106
|
+
...headers,
|
|
107
|
+
});
|
|
108
|
+
res.end(payload);
|
|
109
|
+
}
|
|
110
|
+
|
|
111
|
+
function notFound(res, corsHeaders) {
|
|
112
|
+
jsonResponse(res, 404, {
|
|
113
|
+
error: {
|
|
114
|
+
message: 'Not found',
|
|
115
|
+
type: 'invalid_request_error',
|
|
116
|
+
code: 'not_found',
|
|
117
|
+
},
|
|
118
|
+
}, corsHeaders);
|
|
119
|
+
}
|
|
120
|
+
|
|
121
|
+
function readBody(req) {
|
|
122
|
+
return new Promise((resolve, reject) => {
|
|
123
|
+
const chunks = [];
|
|
124
|
+
let size = 0;
|
|
125
|
+
req.on('data', (chunk) => {
|
|
126
|
+
if (size > MAX_BODY_BYTES) return;
|
|
127
|
+
size += chunk.length;
|
|
128
|
+
if (size > MAX_BODY_BYTES) {
|
|
129
|
+
reject(new Error('Request body is too large.'));
|
|
130
|
+
return;
|
|
131
|
+
}
|
|
132
|
+
chunks.push(chunk);
|
|
133
|
+
});
|
|
134
|
+
req.on('end', () => resolve(Buffer.concat(chunks)));
|
|
135
|
+
req.on('error', reject);
|
|
136
|
+
});
|
|
137
|
+
}
|
|
138
|
+
|
|
139
|
+
function copyLocalHeaders(req) {
|
|
140
|
+
const headers = {};
|
|
141
|
+
const teamId = req.headers['x-team-id'];
|
|
142
|
+
if (typeof teamId === 'string' && teamId.trim()) {
|
|
143
|
+
headers['x-team-id'] = teamId.trim();
|
|
144
|
+
}
|
|
145
|
+
return headers;
|
|
146
|
+
}
|
|
147
|
+
|
|
148
|
+
function createSecureState(apiBase) {
|
|
149
|
+
let clientPromise = null;
|
|
150
|
+
let verificationDocument = null;
|
|
151
|
+
let verificationError = null;
|
|
152
|
+
|
|
153
|
+
async function getClient() {
|
|
154
|
+
if (!clientPromise) {
|
|
155
|
+
clientPromise = import('tinfoil')
|
|
156
|
+
.then(async ({ SecureClient }) => {
|
|
157
|
+
const baseURL = `${apiBase}/api/v1/private/tinfoil/`;
|
|
158
|
+
const client = new SecureClient({
|
|
159
|
+
baseURL,
|
|
160
|
+
attestationBundleURL: `${apiBase}/api/v1/private/tinfoil`,
|
|
161
|
+
transport: 'ehbp',
|
|
162
|
+
});
|
|
163
|
+
await client.ready();
|
|
164
|
+
verificationDocument = client.getVerificationDocument();
|
|
165
|
+
verificationError = null;
|
|
166
|
+
return client;
|
|
167
|
+
})
|
|
168
|
+
.catch((error) => {
|
|
169
|
+
clientPromise = null;
|
|
170
|
+
verificationError = error;
|
|
171
|
+
throw error;
|
|
172
|
+
});
|
|
173
|
+
}
|
|
174
|
+
return clientPromise;
|
|
175
|
+
}
|
|
176
|
+
|
|
177
|
+
return {
|
|
178
|
+
getClient,
|
|
179
|
+
getVerificationState() {
|
|
180
|
+
return {
|
|
181
|
+
verified: verificationDocument?.securityVerified === true,
|
|
182
|
+
verificationDocument,
|
|
183
|
+
error: verificationError ? String(verificationError.message || verificationError) : null,
|
|
184
|
+
};
|
|
185
|
+
},
|
|
186
|
+
};
|
|
187
|
+
}
|
|
188
|
+
|
|
189
|
+
async function runPreflight({ apiBase, apiKey, model, req, requestBodyBytes }) {
|
|
190
|
+
const preflightBody = { model: model.id };
|
|
191
|
+
if (Number.isSafeInteger(requestBodyBytes) && requestBodyBytes >= 0) {
|
|
192
|
+
preflightBody.requestBodyBytes = requestBodyBytes;
|
|
193
|
+
}
|
|
194
|
+
|
|
195
|
+
const response = await fetch(`${apiBase}/api/v1/private/tinfoil/preflight`, {
|
|
196
|
+
method: 'POST',
|
|
197
|
+
headers: {
|
|
198
|
+
authorization: `Bearer ${apiKey}`,
|
|
199
|
+
'content-type': 'application/json',
|
|
200
|
+
...copyLocalHeaders(req),
|
|
201
|
+
},
|
|
202
|
+
body: JSON.stringify(preflightBody),
|
|
203
|
+
});
|
|
204
|
+
|
|
205
|
+
if (response.ok) return { ok: true };
|
|
206
|
+
|
|
207
|
+
let errorBody = null;
|
|
208
|
+
try {
|
|
209
|
+
errorBody = await response.json();
|
|
210
|
+
} catch {
|
|
211
|
+
errorBody = {
|
|
212
|
+
error: {
|
|
213
|
+
message: `NanoGPT preflight failed with HTTP ${response.status}`,
|
|
214
|
+
type: 'invalid_request_error',
|
|
215
|
+
code: 'preflight_failed',
|
|
216
|
+
},
|
|
217
|
+
};
|
|
218
|
+
}
|
|
219
|
+
return { ok: false, status: response.status, body: errorBody };
|
|
220
|
+
}
|
|
221
|
+
|
|
222
|
+
async function handleChatCompletion({ apiBase, apiKey, secureState, req, res, corsHeaders }) {
|
|
223
|
+
let rawBody;
|
|
224
|
+
try {
|
|
225
|
+
rawBody = await readBody(req);
|
|
226
|
+
} catch (error) {
|
|
227
|
+
jsonResponse(res, 413, {
|
|
228
|
+
error: {
|
|
229
|
+
message: error instanceof Error ? error.message : String(error),
|
|
230
|
+
type: 'invalid_request_error',
|
|
231
|
+
code: 'body_too_large',
|
|
232
|
+
},
|
|
233
|
+
}, corsHeaders);
|
|
234
|
+
return;
|
|
235
|
+
}
|
|
236
|
+
|
|
237
|
+
let body;
|
|
238
|
+
try {
|
|
239
|
+
body = JSON.parse(rawBody.toString('utf8'));
|
|
240
|
+
} catch {
|
|
241
|
+
jsonResponse(res, 400, {
|
|
242
|
+
error: {
|
|
243
|
+
message: 'Request body must be JSON.',
|
|
244
|
+
type: 'invalid_request_error',
|
|
245
|
+
code: 'invalid_json',
|
|
246
|
+
},
|
|
247
|
+
}, corsHeaders);
|
|
248
|
+
return;
|
|
249
|
+
}
|
|
250
|
+
|
|
251
|
+
const model = resolveModel(body.model);
|
|
252
|
+
if (!model) {
|
|
253
|
+
jsonResponse(res, 400, {
|
|
254
|
+
error: {
|
|
255
|
+
message: `Unsupported private model "${body.model || ''}".`,
|
|
256
|
+
type: 'invalid_request_error',
|
|
257
|
+
code: 'model_not_supported',
|
|
258
|
+
},
|
|
259
|
+
}, corsHeaders);
|
|
260
|
+
return;
|
|
261
|
+
}
|
|
262
|
+
|
|
263
|
+
applyPrivateModelRequestMutations(body, model);
|
|
264
|
+
const privateStreamRequested = body.stream === true;
|
|
265
|
+
|
|
266
|
+
const privateRequestBody = JSON.stringify(body);
|
|
267
|
+
const preflight = await runPreflight({
|
|
268
|
+
apiBase,
|
|
269
|
+
apiKey,
|
|
270
|
+
model,
|
|
271
|
+
req,
|
|
272
|
+
requestBodyBytes: Buffer.byteLength(privateRequestBody, 'utf8'),
|
|
273
|
+
});
|
|
274
|
+
if (!preflight.ok) {
|
|
275
|
+
jsonResponse(res, preflight.status, preflight.body, corsHeaders);
|
|
276
|
+
return;
|
|
277
|
+
}
|
|
278
|
+
|
|
279
|
+
let response;
|
|
280
|
+
const upstreamAbortController = new AbortController();
|
|
281
|
+
let responseComplete = false;
|
|
282
|
+
res.on('close', () => {
|
|
283
|
+
if (!responseComplete) upstreamAbortController.abort();
|
|
284
|
+
});
|
|
285
|
+
|
|
286
|
+
try {
|
|
287
|
+
const client = await secureState.getClient();
|
|
288
|
+
response = await client.fetch(`${apiBase}/api/v1/private/tinfoil/v1/chat/completions`, {
|
|
289
|
+
method: 'POST',
|
|
290
|
+
headers: {
|
|
291
|
+
authorization: `Bearer ${apiKey}`,
|
|
292
|
+
'content-type': 'application/json',
|
|
293
|
+
accept: privateStreamRequested ? 'text/event-stream' : 'application/json',
|
|
294
|
+
'x-nanogpt-private-model': model.id,
|
|
295
|
+
'x-nanogpt-private-stream': privateStreamRequested ? 'true' : 'false',
|
|
296
|
+
'x-query-source': 'api',
|
|
297
|
+
...copyLocalHeaders(req),
|
|
298
|
+
},
|
|
299
|
+
body: privateRequestBody,
|
|
300
|
+
signal: upstreamAbortController.signal,
|
|
301
|
+
});
|
|
302
|
+
} catch (error) {
|
|
303
|
+
responseComplete = true;
|
|
304
|
+
if (res.destroyed) return;
|
|
305
|
+
jsonResponse(res, 502, {
|
|
306
|
+
error: {
|
|
307
|
+
message: `Private Mode request failed: ${error instanceof Error ? error.message : String(error)}`,
|
|
308
|
+
type: 'api_error',
|
|
309
|
+
code: 'private_mode_request_failed',
|
|
310
|
+
},
|
|
311
|
+
}, corsHeaders);
|
|
312
|
+
return;
|
|
313
|
+
}
|
|
314
|
+
|
|
315
|
+
const headers = {
|
|
316
|
+
...corsHeaders,
|
|
317
|
+
'content-type': response.headers.get('content-type') || 'application/json',
|
|
318
|
+
};
|
|
319
|
+
const requestId = response.headers.get('x-request-id');
|
|
320
|
+
if (requestId) headers['x-request-id'] = requestId;
|
|
321
|
+
const privateMode = response.headers.get('x-nanogpt-private-mode');
|
|
322
|
+
if (privateMode) headers['x-nanogpt-private-mode'] = privateMode;
|
|
323
|
+
|
|
324
|
+
if (privateStreamRequested && response.body) {
|
|
325
|
+
res.writeHead(response.status, headers);
|
|
326
|
+
const reader = response.body.getReader();
|
|
327
|
+
let streamFailed = false;
|
|
328
|
+
try {
|
|
329
|
+
while (true) {
|
|
330
|
+
const { done, value } = await reader.read();
|
|
331
|
+
if (done) break;
|
|
332
|
+
if (res.destroyed) {
|
|
333
|
+
await reader.cancel('client disconnected');
|
|
334
|
+
upstreamAbortController.abort();
|
|
335
|
+
return;
|
|
336
|
+
}
|
|
337
|
+
if (value) res.write(Buffer.from(value));
|
|
338
|
+
}
|
|
339
|
+
} catch (error) {
|
|
340
|
+
streamFailed = true;
|
|
341
|
+
upstreamAbortController.abort();
|
|
342
|
+
if (!res.destroyed) {
|
|
343
|
+
res.destroy(error instanceof Error ? error : new Error(String(error)));
|
|
344
|
+
}
|
|
345
|
+
return;
|
|
346
|
+
} finally {
|
|
347
|
+
responseComplete = true;
|
|
348
|
+
if (!streamFailed && !res.writableEnded && !res.destroyed) res.end();
|
|
349
|
+
}
|
|
350
|
+
return;
|
|
351
|
+
}
|
|
352
|
+
|
|
353
|
+
const responseBody = Buffer.from(await response.arrayBuffer());
|
|
354
|
+
headers['content-length'] = String(responseBody.length);
|
|
355
|
+
responseComplete = true;
|
|
356
|
+
res.writeHead(response.status, headers);
|
|
357
|
+
res.end(responseBody);
|
|
358
|
+
}
|
|
359
|
+
|
|
360
|
+
function resolveAccessControlAllowHeaders(req) {
|
|
361
|
+
const requestedHeaders = req.headers['access-control-request-headers'];
|
|
362
|
+
if (typeof requestedHeaders === 'string' && requestedHeaders.trim()) {
|
|
363
|
+
return requestedHeaders
|
|
364
|
+
.split(',')
|
|
365
|
+
.map((header) => header.trim().toLowerCase())
|
|
366
|
+
.filter(Boolean)
|
|
367
|
+
.join(', ');
|
|
368
|
+
}
|
|
369
|
+
|
|
370
|
+
return 'authorization, content-type, x-team-id';
|
|
371
|
+
}
|
|
372
|
+
|
|
373
|
+
function handleOptions(req, res, corsHeaders) {
|
|
374
|
+
res.writeHead(204, {
|
|
375
|
+
...corsHeaders,
|
|
376
|
+
'access-control-allow-methods': 'GET,POST,OPTIONS',
|
|
377
|
+
'access-control-allow-headers': resolveAccessControlAllowHeaders(req),
|
|
378
|
+
'access-control-max-age': '86400',
|
|
379
|
+
});
|
|
380
|
+
res.end();
|
|
381
|
+
}
|
|
382
|
+
|
|
383
|
+
export async function startPrivateModeProxy(options) {
|
|
384
|
+
const apiBase = normalizeApiBase(options.apiBase);
|
|
385
|
+
const secureState = createSecureState(apiBase);
|
|
386
|
+
const localBase = `http://${options.host}:${options.port}/v1`;
|
|
387
|
+
const originPolicy = buildPrivateModeOriginPolicy(options);
|
|
388
|
+
|
|
389
|
+
const server = createServer(async (req, res) => {
|
|
390
|
+
try {
|
|
391
|
+
const url = new URL(req.url || '/', `http://${req.headers.host || '127.0.0.1'}`);
|
|
392
|
+
const corsHeaders = getCorsHeadersForRequest(req, originPolicy);
|
|
393
|
+
if (corsHeaders === null) {
|
|
394
|
+
jsonResponse(res, 403, {
|
|
395
|
+
error: {
|
|
396
|
+
message: 'Request origin is not allowed to use this Private Mode proxy.',
|
|
397
|
+
type: 'invalid_request_error',
|
|
398
|
+
code: 'origin_not_allowed',
|
|
399
|
+
},
|
|
400
|
+
});
|
|
401
|
+
return;
|
|
402
|
+
}
|
|
403
|
+
|
|
404
|
+
if (req.method === 'OPTIONS') {
|
|
405
|
+
handleOptions(req, res, corsHeaders);
|
|
406
|
+
return;
|
|
407
|
+
}
|
|
408
|
+
|
|
409
|
+
if (req.method === 'GET' && url.pathname === '/health') {
|
|
410
|
+
jsonResponse(res, 200, privateModeStatus(apiBase, secureState, localBase, originPolicy), corsHeaders);
|
|
411
|
+
return;
|
|
412
|
+
}
|
|
413
|
+
|
|
414
|
+
if (req.method === 'GET' && url.pathname === '/v1/models') {
|
|
415
|
+
jsonResponse(res, 200, openAIModelList(), corsHeaders);
|
|
416
|
+
return;
|
|
417
|
+
}
|
|
418
|
+
|
|
419
|
+
if (req.method === 'GET' && url.pathname === '/v1/private-mode/status') {
|
|
420
|
+
jsonResponse(res, 200, privateModeStatus(apiBase, secureState, localBase, originPolicy), corsHeaders);
|
|
421
|
+
return;
|
|
422
|
+
}
|
|
423
|
+
|
|
424
|
+
if (req.method === 'GET' && url.pathname === '/v1/private-mode/attestation') {
|
|
425
|
+
try {
|
|
426
|
+
await secureState.getClient();
|
|
427
|
+
jsonResponse(res, 200, secureState.getVerificationState(), corsHeaders);
|
|
428
|
+
} catch (error) {
|
|
429
|
+
jsonResponse(res, 502, {
|
|
430
|
+
verified: false,
|
|
431
|
+
verificationDocument: null,
|
|
432
|
+
error: error instanceof Error ? error.message : String(error),
|
|
433
|
+
}, corsHeaders);
|
|
434
|
+
}
|
|
435
|
+
return;
|
|
436
|
+
}
|
|
437
|
+
|
|
438
|
+
if (req.method === 'POST' && url.pathname === '/v1/chat/completions') {
|
|
439
|
+
await handleChatCompletion({
|
|
440
|
+
apiBase,
|
|
441
|
+
apiKey: options.apiKey,
|
|
442
|
+
secureState,
|
|
443
|
+
req,
|
|
444
|
+
res,
|
|
445
|
+
corsHeaders,
|
|
446
|
+
});
|
|
447
|
+
return;
|
|
448
|
+
}
|
|
449
|
+
|
|
450
|
+
if (req.method === 'GET' && url.pathname === '/') {
|
|
451
|
+
textResponse(res, 200, `NanoGPT Private Mode Proxy\nOpenAI base URL: ${localBase}\nStatus: /v1/private-mode/status\n`, corsHeaders);
|
|
452
|
+
return;
|
|
453
|
+
}
|
|
454
|
+
|
|
455
|
+
notFound(res, corsHeaders);
|
|
456
|
+
} catch (error) {
|
|
457
|
+
if (res.destroyed) return;
|
|
458
|
+
if (res.headersSent) {
|
|
459
|
+
res.destroy(error instanceof Error ? error : new Error(String(error)));
|
|
460
|
+
return;
|
|
461
|
+
}
|
|
462
|
+
jsonResponse(res, 500, {
|
|
463
|
+
error: {
|
|
464
|
+
message: error instanceof Error ? error.message : String(error),
|
|
465
|
+
type: 'api_error',
|
|
466
|
+
code: 'internal_error',
|
|
467
|
+
},
|
|
468
|
+
});
|
|
469
|
+
}
|
|
470
|
+
});
|
|
471
|
+
|
|
472
|
+
await new Promise((resolve, reject) => {
|
|
473
|
+
server.once('error', reject);
|
|
474
|
+
server.listen(options.port, options.host, resolve);
|
|
475
|
+
});
|
|
476
|
+
|
|
477
|
+
if (options.quiet !== true) {
|
|
478
|
+
process.stdout.write(`NanoGPT Private Mode Proxy
|
|
479
|
+
Local base URL: ${localBase}
|
|
480
|
+
NanoGPT API: ${apiBase}
|
|
481
|
+
Models: ${MODELS.map((model) => model.id).join(', ')}
|
|
482
|
+
Status: http://${options.host}:${options.port}/v1/private-mode/status
|
|
483
|
+
Allowed browser origins: ${originPolicy.allowedOrigins.join(', ')}
|
|
484
|
+
|
|
485
|
+
Request and response bodies are encrypted after they leave this local proxy.
|
|
486
|
+
NanoGPT can see your account, private model header, timing, sizes, status, and usage metadata.
|
|
487
|
+
The verified Tinfoil enclave and this local proxy can see plaintext.
|
|
488
|
+
`);
|
|
489
|
+
}
|
|
490
|
+
|
|
491
|
+
if (options.warmAttestation !== false) {
|
|
492
|
+
secureState.getClient()
|
|
493
|
+
.then((client) => {
|
|
494
|
+
const doc = client.getVerificationDocument();
|
|
495
|
+
process.stdout.write(`Attestation: verified=${doc.securityVerified === true}; enclave=${doc.enclaveHost || client.getEnclaveURL() || 'unknown'}\n`);
|
|
496
|
+
})
|
|
497
|
+
.catch((error) => {
|
|
498
|
+
process.stderr.write(`Attestation failed: ${error instanceof Error ? error.message : String(error)}\n`);
|
|
499
|
+
});
|
|
500
|
+
}
|
|
501
|
+
|
|
502
|
+
return server;
|
|
503
|
+
}
|
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
export const PRIVATE_MODE_FRONTEND_SUPPORTED_FEATURES = Object.freeze([
|
|
2
|
+
'text_chat',
|
|
3
|
+
'streaming',
|
|
4
|
+
'conversation_history',
|
|
5
|
+
'model_settings',
|
|
6
|
+
'reasoning_modes',
|
|
7
|
+
]);
|
|
8
|
+
|
|
9
|
+
export const PRIVATE_MODE_FRONTEND_DISABLED_FEATURES = Object.freeze([
|
|
10
|
+
'attachments',
|
|
11
|
+
'web_search',
|
|
12
|
+
'url_scraped_content',
|
|
13
|
+
'project_chats',
|
|
14
|
+
'project_tools',
|
|
15
|
+
'multi_model',
|
|
16
|
+
'context_memory',
|
|
17
|
+
'recoverable_cloud_sync',
|
|
18
|
+
'quick_replies',
|
|
19
|
+
'auto_title_generation',
|
|
20
|
+
]);
|
|
21
|
+
|
|
22
|
+
export const PRIVATE_MODE_NANOGPT_VISIBLE = Object.freeze([
|
|
23
|
+
'account',
|
|
24
|
+
'selected_private_model',
|
|
25
|
+
'selected_tinfoil_enclave',
|
|
26
|
+
'request_timing',
|
|
27
|
+
'request_size',
|
|
28
|
+
'response_size',
|
|
29
|
+
'status',
|
|
30
|
+
'usage_metadata',
|
|
31
|
+
]);
|
|
32
|
+
|
|
33
|
+
export const PRIVATE_MODE_ENCRYPTED_FROM_NANOGPT = Object.freeze([
|
|
34
|
+
'messages',
|
|
35
|
+
'system_prompt',
|
|
36
|
+
'tools',
|
|
37
|
+
'attachments',
|
|
38
|
+
'model_response',
|
|
39
|
+
]);
|
|
40
|
+
|
|
41
|
+
export const PRIVATE_MODE_PLAINTEXT_VISIBLE_TO = Object.freeze([
|
|
42
|
+
'local_proxy',
|
|
43
|
+
'verified_tinfoil_enclave',
|
|
44
|
+
]);
|
|
45
|
+
|
|
46
|
+
export function buildPrivateModeStatusContract() {
|
|
47
|
+
return {
|
|
48
|
+
frontend_supported_features: [...PRIVATE_MODE_FRONTEND_SUPPORTED_FEATURES],
|
|
49
|
+
frontend_disabled_features: [...PRIVATE_MODE_FRONTEND_DISABLED_FEATURES],
|
|
50
|
+
nanogpt_visible: [...PRIVATE_MODE_NANOGPT_VISIBLE],
|
|
51
|
+
encrypted_from_nanogpt: [...PRIVATE_MODE_ENCRYPTED_FROM_NANOGPT],
|
|
52
|
+
plaintext_visible_to: [...PRIVATE_MODE_PLAINTEXT_VISIBLE_TO],
|
|
53
|
+
};
|
|
54
|
+
}
|
|
@@ -0,0 +1,170 @@
|
|
|
1
|
+
import { createHash } from 'node:crypto';
|
|
2
|
+
import { readFile } from 'node:fs/promises';
|
|
3
|
+
import { fetchAttestationBundle, Verifier } from 'tinfoil';
|
|
4
|
+
|
|
5
|
+
function isRecord(value) {
|
|
6
|
+
return Boolean(value) && typeof value === 'object' && !Array.isArray(value);
|
|
7
|
+
}
|
|
8
|
+
|
|
9
|
+
function stableStringify(value) {
|
|
10
|
+
if (Array.isArray(value)) {
|
|
11
|
+
return `[${value.map((item) => stableStringify(item)).join(',')}]`;
|
|
12
|
+
}
|
|
13
|
+
if (isRecord(value)) {
|
|
14
|
+
return `{${Object.keys(value)
|
|
15
|
+
.sort()
|
|
16
|
+
.map((key) => `${JSON.stringify(key)}:${stableStringify(value[key])}`)
|
|
17
|
+
.join(',')}}`;
|
|
18
|
+
}
|
|
19
|
+
return JSON.stringify(value);
|
|
20
|
+
}
|
|
21
|
+
|
|
22
|
+
function sha256Json(value) {
|
|
23
|
+
return createHash('sha256').update(stableStringify(value)).digest('hex');
|
|
24
|
+
}
|
|
25
|
+
|
|
26
|
+
async function readStdin() {
|
|
27
|
+
const chunks = [];
|
|
28
|
+
for await (const chunk of process.stdin) {
|
|
29
|
+
chunks.push(Buffer.from(chunk));
|
|
30
|
+
}
|
|
31
|
+
return Buffer.concat(chunks).toString('utf8');
|
|
32
|
+
}
|
|
33
|
+
|
|
34
|
+
async function readReceiptInput(path) {
|
|
35
|
+
if (path) return readFile(path, 'utf8');
|
|
36
|
+
return readStdin();
|
|
37
|
+
}
|
|
38
|
+
|
|
39
|
+
function normalizeEnclaveUrl(receipt) {
|
|
40
|
+
const raw = receipt?.verifier?.selectedRouterEndpoint
|
|
41
|
+
|| receipt?.verifier?.enclaveHost
|
|
42
|
+
|| receipt?.response?.enclave;
|
|
43
|
+
if (typeof raw !== 'string' || !raw.trim()) {
|
|
44
|
+
throw new Error('Receipt does not include an enclave host or router endpoint.');
|
|
45
|
+
}
|
|
46
|
+
|
|
47
|
+
const trimmed = raw.trim();
|
|
48
|
+
return /^https:\/\//i.test(trimmed) ? trimmed : `https://${trimmed}`;
|
|
49
|
+
}
|
|
50
|
+
|
|
51
|
+
function isNonEmptyString(value) {
|
|
52
|
+
return typeof value === 'string' && value.trim().length > 0;
|
|
53
|
+
}
|
|
54
|
+
|
|
55
|
+
function assertRequiredEqual(failures, label, actual, expected) {
|
|
56
|
+
if (!isNonEmptyString(expected)) {
|
|
57
|
+
failures.push(`${label} missing from receipt.`);
|
|
58
|
+
return;
|
|
59
|
+
}
|
|
60
|
+
if (!isNonEmptyString(actual)) {
|
|
61
|
+
failures.push(`${label} missing from fresh verifier result.`);
|
|
62
|
+
return;
|
|
63
|
+
}
|
|
64
|
+
if (actual !== expected) {
|
|
65
|
+
failures.push(`${label} mismatch: expected ${expected}, got ${actual || 'missing'}`);
|
|
66
|
+
}
|
|
67
|
+
}
|
|
68
|
+
|
|
69
|
+
function assertOptionalEqual(warnings, label, actual, expected) {
|
|
70
|
+
if (!isNonEmptyString(expected)) return;
|
|
71
|
+
if (!isNonEmptyString(actual)) {
|
|
72
|
+
warnings.push(`${label} missing from fresh verifier result.`);
|
|
73
|
+
return;
|
|
74
|
+
}
|
|
75
|
+
if (actual !== expected) {
|
|
76
|
+
warnings.push(`${label} mismatch: expected ${expected}, got ${actual || 'missing'}`);
|
|
77
|
+
}
|
|
78
|
+
}
|
|
79
|
+
|
|
80
|
+
function assertReceiptFlag(failures, label, value) {
|
|
81
|
+
if (value !== true) {
|
|
82
|
+
failures.push(`${label} was not true in receipt.`);
|
|
83
|
+
}
|
|
84
|
+
}
|
|
85
|
+
|
|
86
|
+
export async function verifyPrivateModeReceipt(receipt) {
|
|
87
|
+
if (!isRecord(receipt) || receipt.schemaVersion !== 1) {
|
|
88
|
+
throw new Error('Unsupported or invalid Private Mode receipt.');
|
|
89
|
+
}
|
|
90
|
+
|
|
91
|
+
const configRepo = receipt.verifier?.configRepo;
|
|
92
|
+
if (typeof configRepo !== 'string' || !configRepo.trim()) {
|
|
93
|
+
throw new Error('Receipt does not include a verifier policy repo.');
|
|
94
|
+
}
|
|
95
|
+
|
|
96
|
+
const enclaveURL = normalizeEnclaveUrl(receipt);
|
|
97
|
+
const bundle = await fetchAttestationBundle({
|
|
98
|
+
enclaveURL,
|
|
99
|
+
configRepo,
|
|
100
|
+
});
|
|
101
|
+
const attestationBundleSha256 = sha256Json(bundle);
|
|
102
|
+
|
|
103
|
+
const verifier = new Verifier({ configRepo });
|
|
104
|
+
await verifier.verifyBundle(bundle);
|
|
105
|
+
const doc = verifier.getVerificationDocument();
|
|
106
|
+
const verificationDocumentSha256 = sha256Json(doc);
|
|
107
|
+
|
|
108
|
+
const failures = [];
|
|
109
|
+
const warnings = [];
|
|
110
|
+
if (doc?.securityVerified !== true) {
|
|
111
|
+
failures.push('Fresh verifier result is not securityVerified=true.');
|
|
112
|
+
}
|
|
113
|
+
if (receipt.status !== 'verified') {
|
|
114
|
+
failures.push(`Receipt status is ${receipt.status || 'missing'}; expected verified.`);
|
|
115
|
+
}
|
|
116
|
+
|
|
117
|
+
const encryption = isRecord(receipt.encryption) ? receipt.encryption : {};
|
|
118
|
+
if (encryption.transport !== 'EHBP') {
|
|
119
|
+
failures.push(`Receipt transport is ${encryption.transport || 'missing'}; expected EHBP.`);
|
|
120
|
+
}
|
|
121
|
+
assertReceiptFlag(failures, 'Request body encrypted', encryption.requestBodyEncrypted);
|
|
122
|
+
assertReceiptFlag(failures, 'Response body encrypted', encryption.responseBodyEncrypted);
|
|
123
|
+
assertReceiptFlag(failures, 'NanoGPT proxy confirmation', encryption.nanoGptProxyConfirmed);
|
|
124
|
+
|
|
125
|
+
assertRequiredEqual(failures, 'Expected measurement', doc?.codeFingerprint, receipt.verifier?.codeFingerprint);
|
|
126
|
+
assertRequiredEqual(failures, 'Runtime measurement', doc?.enclaveFingerprint, receipt.verifier?.enclaveFingerprint);
|
|
127
|
+
assertRequiredEqual(failures, 'Release digest', doc?.releaseDigest, receipt.verifier?.releaseDigest);
|
|
128
|
+
assertRequiredEqual(failures, 'HPKE public key', doc?.hpkePublicKey, receipt.verifier?.hpkePublicKey);
|
|
129
|
+
assertOptionalEqual(warnings, 'Attestation bundle SHA-256', attestationBundleSha256, receipt.verifier?.attestationBundleSha256);
|
|
130
|
+
|
|
131
|
+
return {
|
|
132
|
+
ok: failures.length === 0,
|
|
133
|
+
failures,
|
|
134
|
+
warnings,
|
|
135
|
+
enclaveURL,
|
|
136
|
+
codeFingerprint: doc?.codeFingerprint,
|
|
137
|
+
enclaveFingerprint: doc?.enclaveFingerprint,
|
|
138
|
+
attestationBundleSha256,
|
|
139
|
+
verificationDocumentSha256,
|
|
140
|
+
};
|
|
141
|
+
}
|
|
142
|
+
|
|
143
|
+
export async function verifyPrivateModeReceiptCli(positionals, io = process) {
|
|
144
|
+
const receiptPath = positionals[0];
|
|
145
|
+
const input = await readReceiptInput(receiptPath);
|
|
146
|
+
const receipt = JSON.parse(input);
|
|
147
|
+
const result = await verifyPrivateModeReceipt(receipt);
|
|
148
|
+
|
|
149
|
+
if (!result.ok) {
|
|
150
|
+
io.stderr.write('Private Mode receipt verification failed.\n');
|
|
151
|
+
for (const failure of result.failures) {
|
|
152
|
+
io.stderr.write(`- ${failure}\n`);
|
|
153
|
+
}
|
|
154
|
+
return 2;
|
|
155
|
+
}
|
|
156
|
+
|
|
157
|
+
io.stdout.write('Private Mode receipt verified.\n');
|
|
158
|
+
io.stdout.write(`Enclave: ${result.enclaveURL}\n`);
|
|
159
|
+
io.stdout.write(`Expected measurement: ${result.codeFingerprint}\n`);
|
|
160
|
+
io.stdout.write(`Runtime measurement: ${result.enclaveFingerprint}\n`);
|
|
161
|
+
io.stdout.write(`Attestation bundle SHA-256: ${result.attestationBundleSha256}\n`);
|
|
162
|
+
io.stdout.write(`Verification document SHA-256: ${result.verificationDocumentSha256}\n`);
|
|
163
|
+
if (result.warnings.length > 0) {
|
|
164
|
+
io.stderr.write('Private Mode receipt verification warnings:\n');
|
|
165
|
+
for (const warning of result.warnings) {
|
|
166
|
+
io.stderr.write(`- ${warning}\n`);
|
|
167
|
+
}
|
|
168
|
+
}
|
|
169
|
+
return 0;
|
|
170
|
+
}
|
|
@@ -0,0 +1,96 @@
|
|
|
1
|
+
[
|
|
2
|
+
{
|
|
3
|
+
"id": "private/kimi-k2-6",
|
|
4
|
+
"name": "Kimi K2.6 Private",
|
|
5
|
+
"upstreamModel": "kimi-k2-6",
|
|
6
|
+
"billingModel": "TEE/kimi-k2-6",
|
|
7
|
+
"tinfoilEnclaveModel": "kimi-k2-6",
|
|
8
|
+
"created": 1764547200,
|
|
9
|
+
"ownedBy": "nanogpt-tinfoil",
|
|
10
|
+
"aliases": ["private/kimi-k2.6", "TEE/kimi-k2-6", "TEE/kimi-k2.6"]
|
|
11
|
+
},
|
|
12
|
+
{
|
|
13
|
+
"id": "private/gpt-oss-120b",
|
|
14
|
+
"name": "GPT OSS 120B Private",
|
|
15
|
+
"upstreamModel": "gpt-oss-120b",
|
|
16
|
+
"billingModel": "TEE/gpt-oss-120b",
|
|
17
|
+
"tinfoilEnclaveModel": "gpt-oss-120b",
|
|
18
|
+
"created": 1764547200,
|
|
19
|
+
"ownedBy": "nanogpt-tinfoil",
|
|
20
|
+
"aliases": ["TEE/gpt-oss-120b", "phala/gpt-oss-120b"]
|
|
21
|
+
},
|
|
22
|
+
{
|
|
23
|
+
"id": "private/llama3-3-70b",
|
|
24
|
+
"name": "Llama 3.3 70B Private",
|
|
25
|
+
"upstreamModel": "llama3-3-70b",
|
|
26
|
+
"billingModel": "TEE/llama3-3-70b",
|
|
27
|
+
"tinfoilEnclaveModel": "llama3-3-70b",
|
|
28
|
+
"created": 1764547200,
|
|
29
|
+
"ownedBy": "nanogpt-tinfoil",
|
|
30
|
+
"aliases": ["private/llama-3.3-70b", "TEE/llama3-3-70b"]
|
|
31
|
+
},
|
|
32
|
+
{
|
|
33
|
+
"id": "private/glm-5-1",
|
|
34
|
+
"name": "GLM 5.1 Private",
|
|
35
|
+
"upstreamModel": "glm-5-1",
|
|
36
|
+
"billingModel": "TEE/glm-5-1",
|
|
37
|
+
"tinfoilEnclaveModel": "glm-5-1",
|
|
38
|
+
"created": 1764547200,
|
|
39
|
+
"ownedBy": "nanogpt-tinfoil",
|
|
40
|
+
"aliases": ["private/glm-5.1", "TEE/glm-5-1", "TEE/glm-5.1"]
|
|
41
|
+
},
|
|
42
|
+
{
|
|
43
|
+
"id": "private/glm-5-1-thinking",
|
|
44
|
+
"name": "GLM 5.1 Thinking Private",
|
|
45
|
+
"upstreamModel": "glm-5-1",
|
|
46
|
+
"billingModel": "TEE/glm-5-1-thinking",
|
|
47
|
+
"tinfoilEnclaveModel": "glm-5-1",
|
|
48
|
+
"created": 1764547200,
|
|
49
|
+
"ownedBy": "nanogpt-tinfoil",
|
|
50
|
+
"aliases": ["private/glm-5.1-thinking", "TEE/glm-5-1-thinking", "TEE/glm-5.1-thinking"]
|
|
51
|
+
},
|
|
52
|
+
{
|
|
53
|
+
"id": "private/gemma4-31b",
|
|
54
|
+
"name": "Gemma 4 31B Private",
|
|
55
|
+
"upstreamModel": "gemma4-31b",
|
|
56
|
+
"billingModel": "TEE/gemma4-31b",
|
|
57
|
+
"tinfoilEnclaveModel": "gemma4-31b",
|
|
58
|
+
"thinkingMode": "gemma",
|
|
59
|
+
"created": 1764547200,
|
|
60
|
+
"ownedBy": "nanogpt-tinfoil",
|
|
61
|
+
"aliases": ["TEE/gemma4-31b", "gemma4-31b"]
|
|
62
|
+
},
|
|
63
|
+
{
|
|
64
|
+
"id": "private/gemma4-31b:thinking",
|
|
65
|
+
"name": "Gemma 4 31B Thinking Private",
|
|
66
|
+
"upstreamModel": "gemma4-31b",
|
|
67
|
+
"billingModel": "TEE/gemma4-31b:thinking",
|
|
68
|
+
"tinfoilEnclaveModel": "gemma4-31b",
|
|
69
|
+
"thinkingMode": "gemma",
|
|
70
|
+
"created": 1764547200,
|
|
71
|
+
"ownedBy": "nanogpt-tinfoil",
|
|
72
|
+
"aliases": ["TEE/gemma4-31b:thinking", "gemma4-31b:thinking"]
|
|
73
|
+
},
|
|
74
|
+
{
|
|
75
|
+
"id": "private/deepseek-v4-pro",
|
|
76
|
+
"name": "DeepSeek V4 Pro Private",
|
|
77
|
+
"upstreamModel": "deepseek-v4-pro",
|
|
78
|
+
"billingModel": "TEE/deepseek-v4-pro",
|
|
79
|
+
"tinfoilEnclaveModel": "deepseek-v4-pro",
|
|
80
|
+
"thinkingMode": "deepseek-v4-pro",
|
|
81
|
+
"created": 1764547200,
|
|
82
|
+
"ownedBy": "nanogpt-tinfoil",
|
|
83
|
+
"aliases": ["TEE/deepseek-v4-pro", "deepseek-v4-pro"]
|
|
84
|
+
},
|
|
85
|
+
{
|
|
86
|
+
"id": "private/deepseek-v4-pro:thinking",
|
|
87
|
+
"name": "DeepSeek V4 Pro Thinking Private",
|
|
88
|
+
"upstreamModel": "deepseek-v4-pro",
|
|
89
|
+
"billingModel": "TEE/deepseek-v4-pro:thinking",
|
|
90
|
+
"tinfoilEnclaveModel": "deepseek-v4-pro",
|
|
91
|
+
"thinkingMode": "deepseek-v4-pro",
|
|
92
|
+
"created": 1764547200,
|
|
93
|
+
"ownedBy": "nanogpt-tinfoil",
|
|
94
|
+
"aliases": ["TEE/deepseek-v4-pro:thinking", "deepseek-v4-pro:thinking"]
|
|
95
|
+
}
|
|
96
|
+
]
|
package/package.json
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "@nanogpt/private-mode",
|
|
3
|
+
"version": "0.1.0",
|
|
4
|
+
"description": "OpenAI-compatible localhost proxy for NanoGPT Private Mode.",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"publishConfig": {
|
|
7
|
+
"access": "public"
|
|
8
|
+
},
|
|
9
|
+
"bin": {
|
|
10
|
+
"nanogpt-private-mode": "bin/nanogpt-private-mode.js"
|
|
11
|
+
},
|
|
12
|
+
"files": [
|
|
13
|
+
"bin",
|
|
14
|
+
"lib/originPolicy.js",
|
|
15
|
+
"lib/requestTransforms.js",
|
|
16
|
+
"lib/server.js",
|
|
17
|
+
"lib/statusContract.js",
|
|
18
|
+
"lib/verifyReceipt.js",
|
|
19
|
+
"models",
|
|
20
|
+
"README.md"
|
|
21
|
+
],
|
|
22
|
+
"scripts": {
|
|
23
|
+
"start": "node ./bin/nanogpt-private-mode.js"
|
|
24
|
+
},
|
|
25
|
+
"dependencies": {
|
|
26
|
+
"tinfoil": "1.1.3"
|
|
27
|
+
},
|
|
28
|
+
"engines": {
|
|
29
|
+
"node": ">=20"
|
|
30
|
+
},
|
|
31
|
+
"license": "MIT"
|
|
32
|
+
}
|