@nanogpt/private-mode 0.2.16 → 0.2.19
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +16 -6
- package/lib/documentConversion.js +90 -0
- package/lib/originPolicy.js +5 -0
- package/lib/requestTransforms.js +24 -0
- package/lib/server.js +253 -7
- package/lib/transportTimeouts.js +4 -1
- package/models/private-tee.json +0 -14
- package/package.json +2 -1
package/README.md
CHANGED
|
@@ -46,7 +46,17 @@ NanoGPT can see account identity, selected private model, selected TEE target me
|
|
|
46
46
|
|
|
47
47
|
NanoGPT's web app can also use these models without running this local proxy. Select an eligible Private Mode model and use the Private Mode control in the model picker. This package is for API clients, CLIs, agents, and other OpenAI-compatible tools.
|
|
48
48
|
|
|
49
|
-
The web app supports local image attachments for Private Mode vision models (Kimi K3 and Gemma 4 31B). Image bytes are converted to base64 in the browser and included inside the EHBP-encrypted request; they are not uploaded to NanoGPT object storage. Other attachment types, web search, URL-scraped content, project tools, multi-model chat, Context Memory injection, quick replies, and automatic title generation remain disabled for private turns.
|
|
49
|
+
The web app supports local image attachments for Private Mode vision models (Kimi K3 and Gemma 4 31B). Image bytes are converted to base64 in the browser and included inside the EHBP-encrypted request; they are not uploaded to NanoGPT object storage. It also supports encrypted text extraction from one local PDF up to 3 MiB. Other attachment types, web search, URL-scraped content, project tools, multi-model chat, Context Memory injection, quick replies, and automatic title generation remain disabled for private turns.
|
|
50
|
+
|
|
51
|
+
The localhost proxy exposes the same encrypted PDF conversion path:
|
|
52
|
+
|
|
53
|
+
```bash
|
|
54
|
+
curl --fail-with-body http://127.0.0.1:8787/v1/convert/file \
|
|
55
|
+
-H 'X-NanoGPT-Private-Model: private/glm-5-3' \
|
|
56
|
+
-F 'files=@document.pdf;type=application/pdf'
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
Send exactly one PDF in the `files` field and select any model returned by the proxy's `/v1/models` endpoint with `X-NanoGPT-Private-Model`. The proxy accepts PDFs up to 3 MiB, replaces the local filename before encryption, and returns extracted Markdown in `document.md_content`. Conversion is metered separately from model token usage; the successful response's `X-NanoGPT-Document-Cost-USD` header reports the settled conversion charge. A model catalog value of `capabilities.pdf_upload: false` means the model does not accept a PDF natively in its prompt. It does not disable this separate encrypted text-extraction step.
|
|
50
60
|
|
|
51
61
|
In the hosted web app, decrypted Private Mode turns remain in local browser history. Cloud conversation sync is blocked for Private Mode chats unless password-based end-to-end sync is enabled; the recoverable default sync mode is not used for those chats.
|
|
52
62
|
|
|
@@ -56,7 +66,7 @@ Because the request body is encrypted before NanoGPT receives it, the service ca
|
|
|
56
66
|
|
|
57
67
|
Private Mode streaming has a 29-minute soft deadline so billing can settle before the route's 30-minute hard runtime limit. If the decrypted upstream stream ends without both a non-null `finish_reason` and the OpenAI `[DONE]` marker, the local proxy emits an SSE error with code `private_stream_incomplete`. Any text received before that error may be partial.
|
|
58
68
|
|
|
59
|
-
Private Mode
|
|
69
|
+
Private Mode supports Chat Completions for text generation and `/v1/convert/file` for encrypted PDF-to-Markdown conversion. The Responses API and Batch API are not available through the encrypted Private Mode endpoint. The standard Responses API outside Private Mode supports background requests within its configured 30-minute runtime limit; work that must reliably continue longer needs a durable job system.
|
|
60
70
|
|
|
61
71
|
Useful local checks:
|
|
62
72
|
|
|
@@ -64,12 +74,12 @@ Useful local checks:
|
|
|
64
74
|
GET http://127.0.0.1:8787/v1/models
|
|
65
75
|
GET http://127.0.0.1:8787/v1/private-mode/status
|
|
66
76
|
GET http://127.0.0.1:8787/v1/private-mode/attestation
|
|
77
|
+
POST http://127.0.0.1:8787/v1/convert/file
|
|
67
78
|
```
|
|
68
79
|
|
|
69
80
|
Supported private model IDs include:
|
|
70
81
|
|
|
71
82
|
- `private/deepseek-v4-1-flash` - DeepSeek V4.1 Flash
|
|
72
|
-
- `private/deepseek-v4-flash` - DeepSeek V4 Flash 0731
|
|
73
83
|
- `private/kimi-k3`
|
|
74
84
|
- `private/gpt-oss-120b`
|
|
75
85
|
- `private/llama3-3-70b`
|
|
@@ -78,7 +88,7 @@ Supported private model IDs include:
|
|
|
78
88
|
- `private/gemma4-31b`
|
|
79
89
|
- `private/gemma4-31b:thinking`
|
|
80
90
|
|
|
81
|
-
GLM 5.3 Private and GLM 5.3 Flash Private each have a 1,048,576-token total context limit and a 131,072-token output ceiling. Kimi K3 Private has a 256,000-token context limit. DeepSeek V4 Flash
|
|
91
|
+
GLM 5.3 Private and GLM 5.3 Flash Private each have a 1,048,576-token total context limit and a 131,072-token output ceiling. Kimi K3 Private has a 256,000-token context limit. DeepSeek V4.1 Flash Private has a 1,048,576-token total context limit and a 384,000-token output ceiling. The model-list extension `context_length` is the combined prompt-and-output window, while `max_output_tokens` is the output ceiling within that same window. These values are not additive.
|
|
82
92
|
|
|
83
93
|
### Thinking without leaving Private Mode
|
|
84
94
|
|
|
@@ -86,14 +96,14 @@ Keep the same private model ID when enabling thinking. For example:
|
|
|
86
96
|
|
|
87
97
|
```json
|
|
88
98
|
{
|
|
89
|
-
"model": "private/deepseek-v4-flash",
|
|
99
|
+
"model": "private/deepseek-v4-1-flash",
|
|
90
100
|
"messages": [{ "role": "user", "content": "What is 17 times 19?" }],
|
|
91
101
|
"reasoning_effort": "high",
|
|
92
102
|
"stream": true
|
|
93
103
|
}
|
|
94
104
|
```
|
|
95
105
|
|
|
96
|
-
DeepSeek V4 Flash
|
|
106
|
+
DeepSeek V4.1 Flash and Gemma 4 support optional thinking. Use `thinking: true` or `enable_thinking: true` to enable it; `thinking: false` disables it for these optional-thinking models. Kimi K3, GLM 5.3, and GLM 5.3 Flash always generate reasoning. GLM 5.3 models accept `low`, `high`, or `max` effort. `reasoning: { "exclude": true }` controls response visibility and does not disable thinking.
|
|
97
107
|
|
|
98
108
|
Non-streaming reasoning requests may take more than five minutes. The CLI allows the encrypted request to wait for the hosted route's response, with a 30-minute transport limit. The hosted route stops non-streaming generation after at most 29 minutes to leave time before its platform deadline. Use `stream: true` for earlier output, and configure your API client's timeout to allow long generations. Upgrade the local proxy as well as the hosted service to receive both timeout fixes.
|
|
99
109
|
|
|
@@ -0,0 +1,90 @@
|
|
|
1
|
+
export const MAX_PRIVATE_DOCUMENT_BYTES = 3 * 1024 * 1024;
|
|
2
|
+
export const MAX_PRIVATE_DOCUMENT_MULTIPART_BYTES = 4 * 1024 * 1024;
|
|
3
|
+
export const MAX_PRIVATE_DOCUMENT_RESPONSE_BYTES = 4 * 1024 * 1024;
|
|
4
|
+
export const PRIVATE_DOCUMENT_CONVERSION_PATH = '/v1/convert/file';
|
|
5
|
+
export const HOSTED_PRIVATE_DOCUMENT_CONVERSION_PATH = '/api/v1/private/tinfoil/v1/convert/file';
|
|
6
|
+
|
|
7
|
+
export function resolvePrivateDocumentMultipartLimit(maxBodyBytes) {
|
|
8
|
+
return Math.min(maxBodyBytes, MAX_PRIVATE_DOCUMENT_MULTIPART_BYTES);
|
|
9
|
+
}
|
|
10
|
+
|
|
11
|
+
export function createPrivateDocumentRequestLifecycle(res) {
|
|
12
|
+
const controller = new AbortController();
|
|
13
|
+
let complete = false;
|
|
14
|
+
res.once('close', () => {
|
|
15
|
+
if (!complete) controller.abort();
|
|
16
|
+
});
|
|
17
|
+
|
|
18
|
+
return {
|
|
19
|
+
signal: controller.signal,
|
|
20
|
+
isCancelled: () => Boolean(res.destroyed || controller.signal.aborted),
|
|
21
|
+
complete: () => {
|
|
22
|
+
complete = true;
|
|
23
|
+
},
|
|
24
|
+
};
|
|
25
|
+
}
|
|
26
|
+
|
|
27
|
+
export function isPrivateDocumentMultipartContentType(value) {
|
|
28
|
+
return typeof value === 'string'
|
|
29
|
+
&& /^multipart\/form-data\s*;[^\r\n]*\bboundary=(?:"[^"]+"|[^;\s]+)/i.test(value);
|
|
30
|
+
}
|
|
31
|
+
|
|
32
|
+
export function hasPrivateDocumentConversionResult(value) {
|
|
33
|
+
return Boolean(
|
|
34
|
+
value
|
|
35
|
+
&& typeof value === 'object'
|
|
36
|
+
&& value.document
|
|
37
|
+
&& typeof value.document === 'object'
|
|
38
|
+
&& typeof value.document.md_content === 'string'
|
|
39
|
+
&& value.document.md_content.trim(),
|
|
40
|
+
);
|
|
41
|
+
}
|
|
42
|
+
|
|
43
|
+
export async function preparePrivateDocumentMultipart(rawBody, contentType) {
|
|
44
|
+
let form;
|
|
45
|
+
try {
|
|
46
|
+
form = await new Request('http://127.0.0.1/', {
|
|
47
|
+
method: 'POST',
|
|
48
|
+
headers: { 'content-type': contentType },
|
|
49
|
+
body: rawBody,
|
|
50
|
+
}).formData();
|
|
51
|
+
} catch {
|
|
52
|
+
return {
|
|
53
|
+
ok: false,
|
|
54
|
+
status: 400,
|
|
55
|
+
code: 'invalid_document_multipart',
|
|
56
|
+
message: 'Private PDF conversion requires valid multipart form data.',
|
|
57
|
+
};
|
|
58
|
+
}
|
|
59
|
+
|
|
60
|
+
const entries = [...form.entries()];
|
|
61
|
+
const file = entries.length === 1 && entries[0][0] === 'files' ? entries[0][1] : null;
|
|
62
|
+
if (!file || typeof file === 'string' || typeof file.arrayBuffer !== 'function') {
|
|
63
|
+
return {
|
|
64
|
+
ok: false,
|
|
65
|
+
status: 400,
|
|
66
|
+
code: 'invalid_document_fields',
|
|
67
|
+
message: 'Private PDF conversion requires exactly one PDF in the files field.',
|
|
68
|
+
};
|
|
69
|
+
}
|
|
70
|
+
if (file.size > MAX_PRIVATE_DOCUMENT_BYTES) {
|
|
71
|
+
return {
|
|
72
|
+
ok: false,
|
|
73
|
+
status: 413,
|
|
74
|
+
code: 'document_too_large',
|
|
75
|
+
message: 'Private PDF conversion accepts a PDF up to 3 MiB.',
|
|
76
|
+
};
|
|
77
|
+
}
|
|
78
|
+
if (file.size === 0 || await file.slice(0, 5).text() !== '%PDF-') {
|
|
79
|
+
return {
|
|
80
|
+
ok: false,
|
|
81
|
+
status: 400,
|
|
82
|
+
code: 'invalid_document',
|
|
83
|
+
message: 'Private PDF conversion requires a non-empty file with a valid PDF signature.',
|
|
84
|
+
};
|
|
85
|
+
}
|
|
86
|
+
|
|
87
|
+
const forwardedForm = new FormData();
|
|
88
|
+
forwardedForm.append('files', new Blob([await file.arrayBuffer()], { type: 'application/pdf' }), 'document.pdf');
|
|
89
|
+
return { ok: true, body: forwardedForm, fileSize: file.size };
|
|
90
|
+
}
|
package/lib/originPolicy.js
CHANGED
|
@@ -111,6 +111,11 @@ export function getCorsHeadersForOrigin(originPolicy, origin) {
|
|
|
111
111
|
|
|
112
112
|
return {
|
|
113
113
|
'access-control-allow-origin': normalizedOrigin,
|
|
114
|
+
'access-control-expose-headers': [
|
|
115
|
+
'x-request-id',
|
|
116
|
+
'x-nanogpt-private-mode',
|
|
117
|
+
'x-nanogpt-document-cost-usd',
|
|
118
|
+
].join(', '),
|
|
114
119
|
vary: 'Origin',
|
|
115
120
|
};
|
|
116
121
|
}
|
package/lib/requestTransforms.js
CHANGED
|
@@ -1021,6 +1021,29 @@ function applyTinfoilCompatibilityMutations(body, model) {
|
|
|
1021
1021
|
}
|
|
1022
1022
|
}
|
|
1023
1023
|
|
|
1024
|
+
function stripEmptyPrivateModeToolDeclarations(body) {
|
|
1025
|
+
// Some OpenAI-compatible clients send inactive tool support as empty arrays.
|
|
1026
|
+
// Tinfoil rejects those arrays, so omit them and their dependent controls.
|
|
1027
|
+
if (Array.isArray(body.tools) && body.tools.length === 0) {
|
|
1028
|
+
const hasMessageTools = Array.isArray(body.messages) && body.messages.some(
|
|
1029
|
+
(message) => isPlainObject(message) &&
|
|
1030
|
+
message.role === 'system' &&
|
|
1031
|
+
Array.isArray(message.tools) &&
|
|
1032
|
+
message.tools.length > 0,
|
|
1033
|
+
);
|
|
1034
|
+
delete body.tools;
|
|
1035
|
+
if (!hasMessageTools) {
|
|
1036
|
+
delete body.tool_choice;
|
|
1037
|
+
delete body.parallel_tool_calls;
|
|
1038
|
+
}
|
|
1039
|
+
}
|
|
1040
|
+
|
|
1041
|
+
if (Array.isArray(body.functions) && body.functions.length === 0) {
|
|
1042
|
+
delete body.functions;
|
|
1043
|
+
delete body.function_call;
|
|
1044
|
+
}
|
|
1045
|
+
}
|
|
1046
|
+
|
|
1024
1047
|
function normalizeMaxTokenAliases(body) {
|
|
1025
1048
|
for (const key of MAX_TOKEN_ALIAS_FIELDS) {
|
|
1026
1049
|
if (body.max_tokens === undefined && body[key] !== undefined) {
|
|
@@ -1115,6 +1138,7 @@ export function applyPrivateModelRequestMutations(body, model) {
|
|
|
1115
1138
|
if (model.thinkingMode !== 'glm-5.3') delete body.reasoning_effort;
|
|
1116
1139
|
}
|
|
1117
1140
|
|
|
1141
|
+
stripEmptyPrivateModeToolDeclarations(body);
|
|
1118
1142
|
stripUnsupportedPrivateTinfoilFields(body);
|
|
1119
1143
|
return body;
|
|
1120
1144
|
}
|
package/lib/server.js
CHANGED
|
@@ -12,6 +12,17 @@ import {
|
|
|
12
12
|
PRIVATE_MODE_DEFAULT_RESERVE_MAX_OUTPUT_TOKENS,
|
|
13
13
|
PRIVATE_MODE_STREAM_SOFT_DEADLINE_MS,
|
|
14
14
|
} from './constants.js';
|
|
15
|
+
import {
|
|
16
|
+
createPrivateDocumentRequestLifecycle,
|
|
17
|
+
hasPrivateDocumentConversionResult,
|
|
18
|
+
HOSTED_PRIVATE_DOCUMENT_CONVERSION_PATH,
|
|
19
|
+
isPrivateDocumentMultipartContentType,
|
|
20
|
+
MAX_PRIVATE_DOCUMENT_BYTES,
|
|
21
|
+
MAX_PRIVATE_DOCUMENT_RESPONSE_BYTES,
|
|
22
|
+
preparePrivateDocumentMultipart,
|
|
23
|
+
PRIVATE_DOCUMENT_CONVERSION_PATH,
|
|
24
|
+
resolvePrivateDocumentMultipartLimit,
|
|
25
|
+
} from './documentConversion.js';
|
|
15
26
|
import {
|
|
16
27
|
buildPrivateModeOriginPolicy,
|
|
17
28
|
getCorsHeadersForRequest,
|
|
@@ -53,6 +64,7 @@ function readMaxBodyBytes() {
|
|
|
53
64
|
}
|
|
54
65
|
|
|
55
66
|
const MAX_BODY_BYTES = readMaxBodyBytes();
|
|
67
|
+
const MAX_DOCUMENT_MULTIPART_BODY_BYTES = resolvePrivateDocumentMultipartLimit(MAX_BODY_BYTES);
|
|
56
68
|
|
|
57
69
|
function normalizeApiBase(apiBase) {
|
|
58
70
|
const parsed = new URL(apiBase);
|
|
@@ -111,6 +123,9 @@ export function privateModeStatus(apiBase, secureState, localBase, originPolicy,
|
|
|
111
123
|
status_path: '/v1/private-mode/status',
|
|
112
124
|
attestation_path: '/v1/private-mode/attestation',
|
|
113
125
|
chat_completions_path: '/v1/chat/completions',
|
|
126
|
+
document_conversion_path: PRIVATE_DOCUMENT_CONVERSION_PATH,
|
|
127
|
+
document_conversion_max_file_bytes: MAX_PRIVATE_DOCUMENT_BYTES,
|
|
128
|
+
document_conversion_max_multipart_bytes: MAX_DOCUMENT_MULTIPART_BODY_BYTES,
|
|
114
129
|
max_body_bytes: MAX_BODY_BYTES,
|
|
115
130
|
transport: 'ehbp',
|
|
116
131
|
request_body_encrypted: true,
|
|
@@ -163,14 +178,14 @@ function notFound(res, corsHeaders) {
|
|
|
163
178
|
}, corsHeaders);
|
|
164
179
|
}
|
|
165
180
|
|
|
166
|
-
function readBody(req) {
|
|
181
|
+
function readBody(req, maxBytes = MAX_BODY_BYTES) {
|
|
167
182
|
return new Promise((resolve, reject) => {
|
|
168
183
|
const chunks = [];
|
|
169
184
|
let size = 0;
|
|
170
185
|
req.on('data', (chunk) => {
|
|
171
|
-
if (size >
|
|
186
|
+
if (size > maxBytes) return;
|
|
172
187
|
size += chunk.length;
|
|
173
|
-
if (size >
|
|
188
|
+
if (size > maxBytes) {
|
|
174
189
|
reject(new Error('Request body is too large.'));
|
|
175
190
|
return;
|
|
176
191
|
}
|
|
@@ -190,8 +205,11 @@ function copyLocalHeaders(req) {
|
|
|
190
205
|
return headers;
|
|
191
206
|
}
|
|
192
207
|
|
|
193
|
-
async function runPreflight({ apiBase, apiKey, model, req, requestBodyBytes }) {
|
|
194
|
-
const preflightBody = {
|
|
208
|
+
async function runPreflight({ apiBase, apiKey, model, req, requestBodyBytes, operation, signal }) {
|
|
209
|
+
const preflightBody = {
|
|
210
|
+
model: model.id,
|
|
211
|
+
...(operation ? { operation } : {}),
|
|
212
|
+
};
|
|
195
213
|
if (Number.isSafeInteger(requestBodyBytes) && requestBodyBytes >= 0) {
|
|
196
214
|
preflightBody.requestBodyBytes = requestBodyBytes;
|
|
197
215
|
}
|
|
@@ -205,6 +223,7 @@ async function runPreflight({ apiBase, apiKey, model, req, requestBodyBytes }) {
|
|
|
205
223
|
...copyLocalHeaders(req),
|
|
206
224
|
},
|
|
207
225
|
body: JSON.stringify(preflightBody),
|
|
226
|
+
signal,
|
|
208
227
|
});
|
|
209
228
|
|
|
210
229
|
if (response.ok) {
|
|
@@ -244,6 +263,221 @@ async function runPreflight({ apiBase, apiKey, model, req, requestBodyBytes }) {
|
|
|
244
263
|
return { ok: false, status: response.status, body: errorBody };
|
|
245
264
|
}
|
|
246
265
|
|
|
266
|
+
async function readResponseBodyWithLimit(response, maxBytes) {
|
|
267
|
+
const reader = response.body?.getReader();
|
|
268
|
+
if (!reader) return Buffer.alloc(0);
|
|
269
|
+
const chunks = [];
|
|
270
|
+
let size = 0;
|
|
271
|
+
try {
|
|
272
|
+
while (true) {
|
|
273
|
+
const { done, value } = await reader.read();
|
|
274
|
+
if (done) break;
|
|
275
|
+
size += value.byteLength;
|
|
276
|
+
if (size > maxBytes) {
|
|
277
|
+
await reader.cancel();
|
|
278
|
+
throw new Error('Response body is too large.');
|
|
279
|
+
}
|
|
280
|
+
chunks.push(Buffer.from(value));
|
|
281
|
+
}
|
|
282
|
+
} finally {
|
|
283
|
+
reader.releaseLock();
|
|
284
|
+
}
|
|
285
|
+
return Buffer.concat(chunks);
|
|
286
|
+
}
|
|
287
|
+
|
|
288
|
+
async function handleDocumentConversion({ apiBase, apiKey, secureState, req, res, corsHeaders }) {
|
|
289
|
+
const contentType = typeof req.headers['content-type'] === 'string'
|
|
290
|
+
? req.headers['content-type']
|
|
291
|
+
: '';
|
|
292
|
+
if (!isPrivateDocumentMultipartContentType(contentType)) {
|
|
293
|
+
jsonResponse(res, 415, {
|
|
294
|
+
error: {
|
|
295
|
+
message: 'Private PDF conversion requires multipart/form-data with one files field.',
|
|
296
|
+
type: 'invalid_request_error',
|
|
297
|
+
code: 'invalid_document_content_type',
|
|
298
|
+
},
|
|
299
|
+
}, corsHeaders);
|
|
300
|
+
return;
|
|
301
|
+
}
|
|
302
|
+
|
|
303
|
+
const model = resolveModel(req.headers['x-nanogpt-private-model']);
|
|
304
|
+
if (!model) {
|
|
305
|
+
jsonResponse(res, 400, {
|
|
306
|
+
error: {
|
|
307
|
+
message: 'Set X-NanoGPT-Private-Model to a model returned by this proxy\'s /v1/models endpoint.',
|
|
308
|
+
type: 'invalid_request_error',
|
|
309
|
+
code: 'model_not_supported',
|
|
310
|
+
},
|
|
311
|
+
}, corsHeaders);
|
|
312
|
+
return;
|
|
313
|
+
}
|
|
314
|
+
|
|
315
|
+
let rawBody;
|
|
316
|
+
try {
|
|
317
|
+
rawBody = await readBody(req, MAX_DOCUMENT_MULTIPART_BODY_BYTES);
|
|
318
|
+
} catch {
|
|
319
|
+
jsonResponse(res, 413, {
|
|
320
|
+
error: {
|
|
321
|
+
message: 'Multipart request is too large. Private PDF conversion accepts a PDF up to 3 MiB.',
|
|
322
|
+
type: 'invalid_request_error',
|
|
323
|
+
code: 'body_too_large',
|
|
324
|
+
},
|
|
325
|
+
}, corsHeaders);
|
|
326
|
+
return;
|
|
327
|
+
}
|
|
328
|
+
if (rawBody.length === 0) {
|
|
329
|
+
jsonResponse(res, 400, {
|
|
330
|
+
error: {
|
|
331
|
+
message: 'Private PDF conversion requires one PDF in the files field.',
|
|
332
|
+
type: 'invalid_request_error',
|
|
333
|
+
code: 'missing_document',
|
|
334
|
+
},
|
|
335
|
+
}, corsHeaders);
|
|
336
|
+
return;
|
|
337
|
+
}
|
|
338
|
+
|
|
339
|
+
const prepared = await preparePrivateDocumentMultipart(rawBody, contentType);
|
|
340
|
+
if (!prepared.ok) {
|
|
341
|
+
jsonResponse(res, prepared.status, {
|
|
342
|
+
error: {
|
|
343
|
+
message: prepared.message,
|
|
344
|
+
type: 'invalid_request_error',
|
|
345
|
+
code: prepared.code,
|
|
346
|
+
},
|
|
347
|
+
}, corsHeaders);
|
|
348
|
+
return;
|
|
349
|
+
}
|
|
350
|
+
|
|
351
|
+
const requestLifecycle = createPrivateDocumentRequestLifecycle(res);
|
|
352
|
+
|
|
353
|
+
let preflight;
|
|
354
|
+
try {
|
|
355
|
+
preflight = await runPreflight({
|
|
356
|
+
apiBase,
|
|
357
|
+
apiKey,
|
|
358
|
+
model,
|
|
359
|
+
req,
|
|
360
|
+
requestBodyBytes: prepared.fileSize,
|
|
361
|
+
operation: 'document_conversion',
|
|
362
|
+
signal: requestLifecycle.signal,
|
|
363
|
+
});
|
|
364
|
+
} catch (error) {
|
|
365
|
+
requestLifecycle.complete();
|
|
366
|
+
if (requestLifecycle.isCancelled()) return;
|
|
367
|
+
throw error;
|
|
368
|
+
}
|
|
369
|
+
if (requestLifecycle.isCancelled()) {
|
|
370
|
+
requestLifecycle.complete();
|
|
371
|
+
return;
|
|
372
|
+
}
|
|
373
|
+
if (!preflight.ok) {
|
|
374
|
+
requestLifecycle.complete();
|
|
375
|
+
jsonResponse(res, preflight.status, preflight.body, corsHeaders);
|
|
376
|
+
return;
|
|
377
|
+
}
|
|
378
|
+
|
|
379
|
+
let responseClient;
|
|
380
|
+
|
|
381
|
+
let response;
|
|
382
|
+
try {
|
|
383
|
+
response = await fetchWithSecureClientRecovery({
|
|
384
|
+
secureState,
|
|
385
|
+
userCacheSecret: preflight.cacheScope,
|
|
386
|
+
shouldInvalidateError: () => !requestLifecycle.signal.aborted,
|
|
387
|
+
fetchWithClient: (client) => client.fetch(`${apiBase}${HOSTED_PRIVATE_DOCUMENT_CONVERSION_PATH}`, {
|
|
388
|
+
method: 'POST',
|
|
389
|
+
headers: {
|
|
390
|
+
authorization: `Bearer ${apiKey}`,
|
|
391
|
+
accept: 'application/json',
|
|
392
|
+
'x-nanogpt-private-model': model.id,
|
|
393
|
+
'x-nanogpt-private-cache-scope': buildPrivateModeCacheScopeProof(preflight.cacheScope),
|
|
394
|
+
[PRIVATE_MODE_PROXY_VERSION_HEADER]: PRIVATE_MODE_PROXY_VERSION,
|
|
395
|
+
'x-query-source': 'api',
|
|
396
|
+
...copyLocalHeaders(req),
|
|
397
|
+
},
|
|
398
|
+
body: prepared.body,
|
|
399
|
+
signal: requestLifecycle.signal,
|
|
400
|
+
}),
|
|
401
|
+
onResponseClient: (client) => {
|
|
402
|
+
responseClient = client;
|
|
403
|
+
},
|
|
404
|
+
});
|
|
405
|
+
} catch (error) {
|
|
406
|
+
requestLifecycle.complete();
|
|
407
|
+
if (requestLifecycle.isCancelled()) return;
|
|
408
|
+
jsonResponse(res, 502, {
|
|
409
|
+
error: {
|
|
410
|
+
message: normalizePrivateModeUpstreamErrorMessage(
|
|
411
|
+
error instanceof Error ? error.message : String(error),
|
|
412
|
+
'Private PDF conversion failed.',
|
|
413
|
+
),
|
|
414
|
+
type: 'api_error',
|
|
415
|
+
code: 'private_document_processing_failed',
|
|
416
|
+
},
|
|
417
|
+
}, corsHeaders);
|
|
418
|
+
return;
|
|
419
|
+
}
|
|
420
|
+
|
|
421
|
+
const responseHeaders = { ...corsHeaders };
|
|
422
|
+
for (const name of ['content-type', 'x-request-id', 'x-nanogpt-private-mode', 'x-nanogpt-document-cost-usd']) {
|
|
423
|
+
const value = response.headers.get(name);
|
|
424
|
+
if (value) responseHeaders[name] = value;
|
|
425
|
+
}
|
|
426
|
+
if (!response.ok) {
|
|
427
|
+
requestLifecycle.complete();
|
|
428
|
+
const errorHeaders = { ...responseHeaders };
|
|
429
|
+
delete errorHeaders['content-type'];
|
|
430
|
+
jsonResponse(res, response.status, {
|
|
431
|
+
error: {
|
|
432
|
+
message: await readErrorMessage(response, `Private PDF conversion failed with HTTP ${response.status}`),
|
|
433
|
+
type: 'api_error',
|
|
434
|
+
code: 'private_document_processing_failed',
|
|
435
|
+
},
|
|
436
|
+
}, errorHeaders);
|
|
437
|
+
secureState.markClientUsed(responseClient);
|
|
438
|
+
return;
|
|
439
|
+
}
|
|
440
|
+
|
|
441
|
+
if (response.headers.get('x-nanogpt-private-mode') !== 'tinfoil'
|
|
442
|
+
|| !response.headers.has('ehbp-response-nonce')) {
|
|
443
|
+
requestLifecycle.complete();
|
|
444
|
+
await response.body?.cancel().catch(() => undefined);
|
|
445
|
+
secureState.invalidateClient(responseClient);
|
|
446
|
+
jsonResponse(res, 502, {
|
|
447
|
+
error: {
|
|
448
|
+
message: 'Private PDF conversion returned an unverified response.',
|
|
449
|
+
type: 'api_error',
|
|
450
|
+
code: 'private_document_response_unverified',
|
|
451
|
+
},
|
|
452
|
+
}, corsHeaders);
|
|
453
|
+
return;
|
|
454
|
+
}
|
|
455
|
+
|
|
456
|
+
let responseBody;
|
|
457
|
+
try {
|
|
458
|
+
responseBody = await readResponseBodyWithLimit(response, MAX_PRIVATE_DOCUMENT_RESPONSE_BYTES);
|
|
459
|
+
const parsed = JSON.parse(responseBody.toString('utf8'));
|
|
460
|
+
if (!hasPrivateDocumentConversionResult(parsed)) throw new Error('missing_document_content');
|
|
461
|
+
} catch {
|
|
462
|
+
requestLifecycle.complete();
|
|
463
|
+
secureState.markClientUsed(responseClient);
|
|
464
|
+
jsonResponse(res, 502, {
|
|
465
|
+
error: {
|
|
466
|
+
message: 'Private PDF conversion returned an invalid response.',
|
|
467
|
+
type: 'api_error',
|
|
468
|
+
code: 'private_document_response_invalid',
|
|
469
|
+
},
|
|
470
|
+
}, corsHeaders);
|
|
471
|
+
return;
|
|
472
|
+
}
|
|
473
|
+
|
|
474
|
+
secureState.markClientUsed(responseClient);
|
|
475
|
+
responseHeaders['content-length'] = String(responseBody.length);
|
|
476
|
+
requestLifecycle.complete();
|
|
477
|
+
res.writeHead(response.status, responseHeaders);
|
|
478
|
+
res.end(responseBody);
|
|
479
|
+
}
|
|
480
|
+
|
|
247
481
|
async function getPreflightedClient({ apiBase, apiKey, secureState, req, model = MODELS[0] }) {
|
|
248
482
|
const preflight = await runPreflight({
|
|
249
483
|
apiBase,
|
|
@@ -454,7 +688,7 @@ function resolveAccessControlAllowHeaders(req) {
|
|
|
454
688
|
.join(', ');
|
|
455
689
|
}
|
|
456
690
|
|
|
457
|
-
return 'authorization, content-type, x-team-id';
|
|
691
|
+
return 'authorization, content-type, x-team-id, x-nanogpt-private-model';
|
|
458
692
|
}
|
|
459
693
|
|
|
460
694
|
function handleOptions(req, res, corsHeaders) {
|
|
@@ -552,8 +786,20 @@ export async function startPrivateModeProxy(options) {
|
|
|
552
786
|
return;
|
|
553
787
|
}
|
|
554
788
|
|
|
789
|
+
if (req.method === 'POST' && url.pathname === PRIVATE_DOCUMENT_CONVERSION_PATH) {
|
|
790
|
+
await handleDocumentConversion({
|
|
791
|
+
apiBase,
|
|
792
|
+
apiKey: options.apiKey,
|
|
793
|
+
secureState,
|
|
794
|
+
req,
|
|
795
|
+
res,
|
|
796
|
+
corsHeaders,
|
|
797
|
+
});
|
|
798
|
+
return;
|
|
799
|
+
}
|
|
800
|
+
|
|
555
801
|
if (req.method === 'GET' && url.pathname === '/') {
|
|
556
|
-
textResponse(res, 200, `NanoGPT Private Mode Proxy v${PRIVATE_MODE_PROXY_VERSION}\nOpenAI base URL: ${localBase}\nStatus: /v1/private-mode/status\n`, corsHeaders);
|
|
802
|
+
textResponse(res, 200, `NanoGPT Private Mode Proxy v${PRIVATE_MODE_PROXY_VERSION}\nOpenAI base URL: ${localBase}\nStatus: /v1/private-mode/status\nPDF conversion: ${PRIVATE_DOCUMENT_CONVERSION_PATH}\n`, corsHeaders);
|
|
557
803
|
return;
|
|
558
804
|
}
|
|
559
805
|
|
package/lib/transportTimeouts.js
CHANGED
|
@@ -5,6 +5,7 @@ export class PrivateModeDispatcher extends Agent {
|
|
|
5
5
|
constructor(apiBase) {
|
|
6
6
|
super();
|
|
7
7
|
this.chatUrl = new URL(`${apiBase.replace(/\/+$/, '')}/api/v1/private/tinfoil/v1/chat/completions`);
|
|
8
|
+
this.documentUrl = new URL(`${apiBase.replace(/\/+$/, '')}/api/v1/private/tinfoil/v1/convert/file`);
|
|
8
9
|
}
|
|
9
10
|
|
|
10
11
|
dispatch(options, handler) {
|
|
@@ -12,7 +13,9 @@ export class PrivateModeDispatcher extends Agent {
|
|
|
12
13
|
// dispatcher. Scope the CLI's override to its configured encrypted endpoint.
|
|
13
14
|
// Override dispatch options, not Agent defaults: fetch supplies its own
|
|
14
15
|
// five-minute timeout values on every dispatch.
|
|
15
|
-
|
|
16
|
+
const isPrivateModeRoute = String(options.origin) === this.chatUrl.origin
|
|
17
|
+
&& (options.path === this.chatUrl.pathname || options.path === this.documentUrl.pathname);
|
|
18
|
+
if (isPrivateModeRoute) {
|
|
16
19
|
options = {
|
|
17
20
|
...options,
|
|
18
21
|
headersTimeout: PRIVATE_MODE_STREAM_ROUTE_MAX_DURATION_MS,
|
package/models/private-tee.json
CHANGED
|
@@ -13,20 +13,6 @@
|
|
|
13
13
|
"ownedBy": "nanogpt-private-mode",
|
|
14
14
|
"aliases": ["private/deepseek-v4.1-flash", "TEE/deepseek-v4.1-flash"]
|
|
15
15
|
},
|
|
16
|
-
{
|
|
17
|
-
"id": "private/deepseek-v4-flash",
|
|
18
|
-
"name": "DeepSeek V4 Flash 0731 Private",
|
|
19
|
-
"upstreamModel": "deepseek-v4-flash",
|
|
20
|
-
"billingModel": "private/deepseek-v4-flash",
|
|
21
|
-
"providerPricingModel": "TEE/deepseek-v4-flash",
|
|
22
|
-
"teeTargetModel": "deepseek-v4-flash",
|
|
23
|
-
"thinkingMode": "deepseek-v4",
|
|
24
|
-
"maxInputTokens": 1048576,
|
|
25
|
-
"maxOutputTokens": 1048576,
|
|
26
|
-
"created": 1786406400,
|
|
27
|
-
"ownedBy": "nanogpt-private-mode",
|
|
28
|
-
"aliases": ["TEE/deepseek-v4-flash"]
|
|
29
|
-
},
|
|
30
16
|
{
|
|
31
17
|
"id": "private/kimi-k3",
|
|
32
18
|
"name": "Kimi K3 Private",
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@nanogpt/private-mode",
|
|
3
|
-
"version": "0.2.
|
|
3
|
+
"version": "0.2.19",
|
|
4
4
|
"description": "OpenAI-compatible localhost proxy for NanoGPT Private Mode.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"publishConfig": {
|
|
@@ -15,6 +15,7 @@
|
|
|
15
15
|
"lib/cacheScope.js",
|
|
16
16
|
"lib/catalogCompatibility.js",
|
|
17
17
|
"lib/constants.js",
|
|
18
|
+
"lib/documentConversion.js",
|
|
18
19
|
"lib/originPolicy.js",
|
|
19
20
|
"lib/packageInfo.js",
|
|
20
21
|
"lib/requestTransforms.js",
|