@offrouter/mcp 0.0.0-stage → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +110 -2
- package/dist/attest.js +74 -0
- package/dist/client.js +218 -0
- package/dist/config.js +32 -0
- package/dist/e2ee.js +194 -0
- package/dist/index.js +18 -0
- package/dist/pay.js +49 -0
- package/dist/server.js +243 -0
- package/package.json +58 -4
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 OffRouter
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,111 @@
|
|
|
1
|
-
#
|
|
1
|
+
# OffRouter MCP
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
An MCP server for [OffRouter](https://offrouter.org), private inference for agents.
|
|
4
|
+
|
|
5
|
+
Every model on OffRouter runs inside a GPU TEE (Intel TDX plus an NVIDIA confidential GPU). Nothing is cached or logged, and every answer comes back with a signed receipt. This server goes one step further: it encrypts your prompt on your own machine, to a key the enclave proves it holds, before anything goes over the network. OffRouter's gateway sits in the middle and only ever sees ciphertext.
|
|
6
|
+
|
|
7
|
+
That is the main reason to run this locally instead of using the hosted endpoint at `https://offrouter.org/mcp`. A remote MCP server has to read your prompt to build the request. This one never sends it in the clear.
|
|
8
|
+
|
|
9
|
+
## Quick start
|
|
10
|
+
|
|
11
|
+
You don't need an account to try it. With no configuration it uses OffRouter's free demo (one model, a few calls a day).
|
|
12
|
+
|
|
13
|
+
**Claude Code**
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
claude mcp add offrouter -- npx -y @offrouter/mcp
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
**Claude Desktop, Cursor, Windsurf and other clients**
|
|
20
|
+
|
|
21
|
+
```json
|
|
22
|
+
{
|
|
23
|
+
"mcpServers": {
|
|
24
|
+
"offrouter": {
|
|
25
|
+
"command": "npx",
|
|
26
|
+
"args": ["-y", "@offrouter/mcp"],
|
|
27
|
+
"env": {
|
|
28
|
+
"OFFROUTER_API_KEY": "off_sk_..."
|
|
29
|
+
}
|
|
30
|
+
}
|
|
31
|
+
}
|
|
32
|
+
}
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
Then ask your agent something like "use offrouter to verify the enclave" or "ask deepseek-v4-flash on offrouter to review this contract".
|
|
36
|
+
|
|
37
|
+
**Nothing to install.** If you'd rather not run anything locally, add the hosted server at `https://offrouter.org/mcp` with your key as an `Authorization: Bearer off_sk_...` header. It has the same models, but your prompt reaches the gateway in the clear, and it can't pay per call from a wallet.
|
|
38
|
+
|
|
39
|
+
## Three ways to pay
|
|
40
|
+
|
|
41
|
+
Pick one by setting an environment variable. If you set both, the key is used.
|
|
42
|
+
|
|
43
|
+
| Setup | How it pays | Good for |
|
|
44
|
+
| --- | --- | --- |
|
|
45
|
+
| nothing | Free demo, limited to one model and 5 calls a day | Trying it out |
|
|
46
|
+
| `OFFROUTER_API_KEY` | A prepaid key, billed per token | Everyday use |
|
|
47
|
+
| `OFFROUTER_WALLET_KEY` | Pay per call with x402 from your own wallet | Agents that hold their own funds |
|
|
48
|
+
|
|
49
|
+
**Prepaid key.** Sign in with a wallet at [offrouter.org/dashboard](https://offrouter.org/dashboard), fund a key, and copy the `off_sk_...` value. Each call is charged the exact token cost.
|
|
50
|
+
|
|
51
|
+
**Wallet.** Every call is paid on its own with a signed USDC transfer on Base, or USDG on Robinhood Chain. The private key stays in this process and is only used to sign the payment. OffRouter quotes each call as a ceiling (your whole input plus `max_tokens` of output) and refunds the unused part to the same wallet afterwards. This server won't sign anything above `OFFROUTER_MAX_USD`, whatever the server asks for. Use a dedicated wallet with a small balance, not your main one.
|
|
52
|
+
|
|
53
|
+
## Configuration
|
|
54
|
+
|
|
55
|
+
| Variable | Default | What it does |
|
|
56
|
+
| --- | --- | --- |
|
|
57
|
+
| `OFFROUTER_API_KEY` | | Prepaid key, starts with `off_sk_` |
|
|
58
|
+
| `OFFROUTER_WALLET_KEY` | | Private key of a wallet to pay per call from |
|
|
59
|
+
| `OFFROUTER_MAX_USD` | `0.10` | Most a single wallet payment can be, in USD |
|
|
60
|
+
| `OFFROUTER_NETWORK` | both | Only pay on `base` or only on `robinhood` |
|
|
61
|
+
| `OFFROUTER_MAX_TOKENS` | `2048` | Output budget when the agent doesn't set one |
|
|
62
|
+
| `OFFROUTER_E2EE` | `on` | Set to `off` to send prompts to the gateway in the clear |
|
|
63
|
+
| `OFFROUTER_BASE_URL` | `https://offrouter.org` | API to talk to |
|
|
64
|
+
|
|
65
|
+
## Tools
|
|
66
|
+
|
|
67
|
+
- **list_models** lists the models with context length and price per million tokens.
|
|
68
|
+
- **ask** sends a prompt to one model and returns the answer, what it cost, and a link to its signed receipt.
|
|
69
|
+
- **compare** runs the same prompt on 2 to 4 models in parallel, so you can see the answers and costs side by side.
|
|
70
|
+
- **account** shows the key balance and recent spend, the wallet address and cap, or what's left on the demo.
|
|
71
|
+
- **verify_enclave** checks the enclave from your machine right now and shows the key your prompts are encrypted to.
|
|
72
|
+
- **list_agents** lists the agents published on [OffRouter Studio](https://offrouter.org/studio): what each does, its model, how its runs have gone and whether it has a ready standing. Filter by a search term or to ready agents only.
|
|
73
|
+
- **get_agent** shows one Studio agent in full: its example tasks, latest evaluation, wallet and config hash.
|
|
74
|
+
|
|
75
|
+
The same calls are on the client, `listAgents()` and `getAgent(id)`, and need no key or wallet: the Studio directory is public.
|
|
76
|
+
|
|
77
|
+
## How the encryption works
|
|
78
|
+
|
|
79
|
+
Before the first call, the server fetches an attestation report for a random nonce it just generated and checks it with the enclave gateway's reference verifier:
|
|
80
|
+
|
|
81
|
+
1. The Intel TDX quote verifies all the way up to Intel's root certificate.
|
|
82
|
+
2. The report covers our nonce, so it was made just now and is not a replay, and it binds the enclave's key set.
|
|
83
|
+
3. The boot log replays to the measured launch file, which pins the open source [enclave gateway](https://github.com/Dstack-TEE/private-ai-gateway) at a specific commit.
|
|
84
|
+
|
|
85
|
+
If any of these fail, nothing is sent. If they pass, each message is encrypted to the enclave's X25519 key (HKDF-SHA256, then AES-256-GCM, with a fresh ephemeral key for every field). Each ciphertext is bound to the request's nonce and timestamp and to the model, so it can't be replayed or redirected to a different model. The answer comes back encrypted to a one-time key this server made for that request, and it's decrypted here. If the answer doesn't decrypt, it wasn't made by the enclave for your request, and the server throws it away.
|
|
86
|
+
|
|
87
|
+
The verified key is cached until shortly before it expires. If the enclave rejects a request, usually because its key rotated, the server checks the enclave again and retries once.
|
|
88
|
+
|
|
89
|
+
## What stays visible
|
|
90
|
+
|
|
91
|
+
To be clear about what this does and doesn't hide:
|
|
92
|
+
|
|
93
|
+
- The gateway can't read your messages or the answers.
|
|
94
|
+
- The gateway still sees which model you picked, settings like `max_tokens` and `temperature`, the size of the ciphertext, the token counts the enclave reports, timing, your IP address, and who paid.
|
|
95
|
+
- With a wallet, the payment itself is a normal onchain transfer, so it's public.
|
|
96
|
+
- Your MCP client (Claude, Cursor and so on) sees the prompt and the answer, because that's where they come from and go to. This server protects the hop from your machine to the model.
|
|
97
|
+
|
|
98
|
+
## Development
|
|
99
|
+
|
|
100
|
+
```bash
|
|
101
|
+
npm install
|
|
102
|
+
npm test
|
|
103
|
+
npm run build
|
|
104
|
+
OFFROUTER_API_KEY=off_sk_... npm run dev
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
Requires Node 20 or newer.
|
|
108
|
+
|
|
109
|
+
## License
|
|
110
|
+
|
|
111
|
+
MIT
|
package/dist/attest.js
ADDED
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
// Finds the enclave's encryption key without taking the gateway's word for it.
|
|
2
|
+
//
|
|
3
|
+
// We ask for an attestation report that covers a random nonce we just made up,
|
|
4
|
+
// then check it with the enclave gateway's reference verifier: the Intel TDX
|
|
5
|
+
// quote has to verify up to Intel's root, the report has to cover our nonce
|
|
6
|
+
// and the key set, and the boot log has to replay to the measured launch file.
|
|
7
|
+
// Only then do we use a key from it. A key from a report that fails any of
|
|
8
|
+
// those could belong to anyone, including the gateway.
|
|
9
|
+
import { verifyComposeMeasurement, verifyQuote, verifyReportBinding } from '@phala/aci-verifier';
|
|
10
|
+
import { randomBytes } from 'node:crypto';
|
|
11
|
+
import { chooseKey } from './e2ee.js';
|
|
12
|
+
const failed = (checks) => checks
|
|
13
|
+
.filter((c) => !c.ok)
|
|
14
|
+
.map((c) => c.detail || c.name)
|
|
15
|
+
.join('; ');
|
|
16
|
+
export class AttestationError extends Error {
|
|
17
|
+
}
|
|
18
|
+
export async function verifyEnclave(baseUrl) {
|
|
19
|
+
const nonce = randomBytes(32).toString('hex');
|
|
20
|
+
const res = await fetch(`${baseUrl}/v1/private/attestation?nonce=${nonce}`, { headers: { accept: 'application/json' } });
|
|
21
|
+
if (!res.ok)
|
|
22
|
+
throw new AttestationError(`the attestation endpoint answered ${res.status}`);
|
|
23
|
+
const report = (await res.json());
|
|
24
|
+
const quote = await verifyQuote(report);
|
|
25
|
+
if (!quote.ok)
|
|
26
|
+
throw new AttestationError(`the hardware quote did not verify: ${quote.detail || 'no reason given'}`);
|
|
27
|
+
const binding = await verifyReportBinding(report, nonce);
|
|
28
|
+
if (!binding.ok || !binding.keyset || !binding.workloadKeysetDigest) {
|
|
29
|
+
throw new AttestationError(`the report does not cover our nonce and key set: ${failed(binding.checks) || 'no key set bound'}`);
|
|
30
|
+
}
|
|
31
|
+
const code = await verifyComposeMeasurement(report);
|
|
32
|
+
if (!code.ok)
|
|
33
|
+
throw new AttestationError(`the boot measurement does not match: ${failed(code.checks) || 'no reason given'}`);
|
|
34
|
+
const keyset = binding.keyset;
|
|
35
|
+
if (keyset.not_after * 1000 < Date.now())
|
|
36
|
+
throw new AttestationError('the key set has expired');
|
|
37
|
+
const key = chooseKey(keyset.e2ee_public_keys);
|
|
38
|
+
if (!key)
|
|
39
|
+
throw new AttestationError('the enclave offers no encryption key this client supports');
|
|
40
|
+
const src = report.attestation.source_provenance;
|
|
41
|
+
const commit = typeof src?.repo_commit === 'string' && /^[0-9a-f]{40}$/.test(src.repo_commit) ? src.repo_commit : null;
|
|
42
|
+
const compose = String(report.attestation.evidence?.app_compose ?? '');
|
|
43
|
+
const pinned = commit && compose.includes(commit) ? commit : null;
|
|
44
|
+
const repo = typeof src?.repo_url === 'string' ? (/^https:\/\/github\.com\/[\w.-]+\/[\w.-]+?(?=\.git$|$)/.exec(src.repo_url)?.[0] ?? null) : null;
|
|
45
|
+
return { key, keyset: binding.workloadKeysetDigest, notAfter: keyset.not_after, commit: pinned, repo, verifiedAt: Date.now() };
|
|
46
|
+
}
|
|
47
|
+
/**
|
|
48
|
+
* Keeps one verified enclave key around so we don't re-run the full check on
|
|
49
|
+
* every call. It is dropped when the key set is close to expiring, or when the
|
|
50
|
+
* enclave rejects a request (which usually means the key rotated).
|
|
51
|
+
*/
|
|
52
|
+
export class EnclaveCache {
|
|
53
|
+
baseUrl;
|
|
54
|
+
current = null;
|
|
55
|
+
constructor(baseUrl) {
|
|
56
|
+
this.baseUrl = baseUrl;
|
|
57
|
+
}
|
|
58
|
+
get() {
|
|
59
|
+
if (this.current) {
|
|
60
|
+
return this.current.then((e) => (e.notAfter * 1000 - Date.now() > 60_000 ? e : this.refresh()));
|
|
61
|
+
}
|
|
62
|
+
return this.refresh();
|
|
63
|
+
}
|
|
64
|
+
refresh() {
|
|
65
|
+
const next = verifyEnclave(this.baseUrl);
|
|
66
|
+
this.current = next;
|
|
67
|
+
// A failed check should not stick around and poison later calls.
|
|
68
|
+
next.catch(() => {
|
|
69
|
+
if (this.current === next)
|
|
70
|
+
this.current = null;
|
|
71
|
+
});
|
|
72
|
+
return next;
|
|
73
|
+
}
|
|
74
|
+
}
|
package/dist/client.js
ADDED
|
@@ -0,0 +1,218 @@
|
|
|
1
|
+
// Talks to the OffRouter API. One chat() call handles all three ways to pay
|
|
2
|
+
// (prepaid key, wallet, free demo) and, when encryption is on, seals the
|
|
3
|
+
// messages to the enclave before anything goes over the network.
|
|
4
|
+
import { EnclaveCache } from './attest.js';
|
|
5
|
+
import { encryptRequest } from './e2ee.js';
|
|
6
|
+
import { makeWallet, settlement } from './pay.js';
|
|
7
|
+
export class ApiError extends Error {
|
|
8
|
+
status;
|
|
9
|
+
code;
|
|
10
|
+
constructor(message, status, code) {
|
|
11
|
+
super(message);
|
|
12
|
+
this.status = status;
|
|
13
|
+
this.code = code;
|
|
14
|
+
}
|
|
15
|
+
}
|
|
16
|
+
const MODELS_TTL_MS = 5 * 60_000;
|
|
17
|
+
export class OffRouter {
|
|
18
|
+
config;
|
|
19
|
+
enclave;
|
|
20
|
+
wallet;
|
|
21
|
+
models = null;
|
|
22
|
+
constructor(config) {
|
|
23
|
+
this.config = config;
|
|
24
|
+
this.enclave = new EnclaveCache(config.baseUrl);
|
|
25
|
+
this.wallet = config.mode === 'wallet' ? makeWallet(config) : null;
|
|
26
|
+
}
|
|
27
|
+
async listModels() {
|
|
28
|
+
if (!this.models || Date.now() - this.models.at > MODELS_TTL_MS) {
|
|
29
|
+
const list = this.getJson('/v1/models').then((b) => (b.data ?? []).filter((m) => m.type !== 'image'));
|
|
30
|
+
this.models = { at: Date.now(), list };
|
|
31
|
+
list.catch(() => (this.models = null));
|
|
32
|
+
}
|
|
33
|
+
return this.models.list;
|
|
34
|
+
}
|
|
35
|
+
/** Balance and recent usage of the prepaid key. */
|
|
36
|
+
keyInfo() {
|
|
37
|
+
return this.getJson('/v1/keys/me', { authorization: `Bearer ${this.config.apiKey}` });
|
|
38
|
+
}
|
|
39
|
+
/** What the free demo allows and what this IP has left today. */
|
|
40
|
+
demoInfo() {
|
|
41
|
+
return this.getJson('/v1/demo');
|
|
42
|
+
}
|
|
43
|
+
/** The Studio directory: the newest agents first, closed ones left out. */
|
|
44
|
+
async listAgents() {
|
|
45
|
+
const b = await this.getJson('/v1/studio/agents');
|
|
46
|
+
return (b.agents ?? []);
|
|
47
|
+
}
|
|
48
|
+
/** One Studio agent's public view, with its examples and last evaluation. */
|
|
49
|
+
async getAgent(id) {
|
|
50
|
+
if (!/^0x[0-9a-fA-F]{32}$/.test(id))
|
|
51
|
+
throw new ApiError('An agent id is 0x followed by 32 hex characters.', 400, 'invalid_agent_id');
|
|
52
|
+
return this.getJson(`/v1/studio/agents/${id}/public`);
|
|
53
|
+
}
|
|
54
|
+
async chat(input) {
|
|
55
|
+
const messages = [];
|
|
56
|
+
if (input.system)
|
|
57
|
+
messages.push({ role: 'system', content: input.system });
|
|
58
|
+
messages.push({ role: 'user', content: input.prompt });
|
|
59
|
+
const { mode } = this.config;
|
|
60
|
+
const body = { model: input.model, messages };
|
|
61
|
+
if (mode !== 'demo') {
|
|
62
|
+
// Pay per call needs max_tokens, since the price is quoted as a ceiling. With a key it is optional,
|
|
63
|
+
// but we send our default anyway so a runaway answer can't drain the balance.
|
|
64
|
+
body.max_tokens = input.max_tokens ?? this.config.defaultMaxTokens;
|
|
65
|
+
if (typeof input.temperature === 'number')
|
|
66
|
+
body.temperature = input.temperature;
|
|
67
|
+
}
|
|
68
|
+
try {
|
|
69
|
+
return await this.send(input.model, body);
|
|
70
|
+
}
|
|
71
|
+
catch (e) {
|
|
72
|
+
// The enclave takes each nonce once and rejects ciphertext for a key it no longer holds. The usual
|
|
73
|
+
// cause is a rotated key, so check the enclave again and retry once with a fresh encryption.
|
|
74
|
+
if (e instanceof ApiError && e.code === 'e2ee_rejected' && this.config.e2ee) {
|
|
75
|
+
await this.enclave.refresh();
|
|
76
|
+
return this.send(input.model, body);
|
|
77
|
+
}
|
|
78
|
+
throw e;
|
|
79
|
+
}
|
|
80
|
+
}
|
|
81
|
+
async send(model, body) {
|
|
82
|
+
const { mode } = this.config;
|
|
83
|
+
const headers = { 'content-type': 'application/json' };
|
|
84
|
+
if (mode === 'key')
|
|
85
|
+
headers.authorization = `Bearer ${this.config.apiKey}`;
|
|
86
|
+
let session = null;
|
|
87
|
+
let payload = body;
|
|
88
|
+
if (this.config.e2ee) {
|
|
89
|
+
const entry = (await this.listModels()).find((m) => m.id === model);
|
|
90
|
+
if (!entry)
|
|
91
|
+
throw new ApiError(`There is no model called "${model}". Call list_models to see what is available.`, 404, 'model_not_found');
|
|
92
|
+
if (!entry.e2ee?.model)
|
|
93
|
+
throw new ApiError(`${model} does not support end to end encryption. Set OFFROUTER_E2EE=off to call it in the clear.`, 400);
|
|
94
|
+
const enclave = await this.enclave.get();
|
|
95
|
+
session = await encryptRequest(body, { key: enclave.key, model: entry.e2ee.model });
|
|
96
|
+
// Belt and braces: never send a request that still has the prompt in it.
|
|
97
|
+
const plain = body.messages.map((m) => m.content).filter((c) => c.length >= 8);
|
|
98
|
+
const wire = JSON.stringify(session.body);
|
|
99
|
+
if (plain.some((c) => wire.includes(c)))
|
|
100
|
+
throw new Error('Encryption did not apply, so the request was not sent.');
|
|
101
|
+
payload = session.body;
|
|
102
|
+
Object.assign(headers, session.headers);
|
|
103
|
+
}
|
|
104
|
+
const path = mode === 'demo' ? '/v1/demo/chat' : '/v1/chat/completions';
|
|
105
|
+
const doFetch = mode === 'wallet' ? this.wallet.fetch : fetch;
|
|
106
|
+
let res;
|
|
107
|
+
try {
|
|
108
|
+
res = await doFetch(`${this.config.baseUrl}${path}`, { method: 'POST', headers, body: JSON.stringify(payload) });
|
|
109
|
+
}
|
|
110
|
+
catch (e) {
|
|
111
|
+
const msg = e.message || String(e);
|
|
112
|
+
if (mode === 'wallet' && /spendControls|maxAmountPerPayment/.test(msg)) {
|
|
113
|
+
throw new Error(`Not paid: this call is quoted above the $${this.config.maxUsd} cap per call. Lower max_tokens or raise OFFROUTER_MAX_USD.`);
|
|
114
|
+
}
|
|
115
|
+
if (mode === 'wallet')
|
|
116
|
+
throw new Error(`Payment failed: ${msg.replace(/\.+$/, '')}.`);
|
|
117
|
+
throw e;
|
|
118
|
+
}
|
|
119
|
+
if (!res.ok) {
|
|
120
|
+
const err = await apiError(res);
|
|
121
|
+
if (mode === 'wallet' && res.status === 402) {
|
|
122
|
+
const why = /insufficient_balance/.test(err.message) ? 'it does not hold enough' : err.message.replace(/\.+$/, '');
|
|
123
|
+
throw new ApiError(`The wallet ${this.wallet.address} could not pay: ${why}. Fund it with USDC on Base or USDG on Robinhood Chain.`, 402, err.code);
|
|
124
|
+
}
|
|
125
|
+
throw err;
|
|
126
|
+
}
|
|
127
|
+
if (session && !session.applied(res.headers))
|
|
128
|
+
throw new Error('The enclave did not confirm it encrypted the answer, so it was not read.');
|
|
129
|
+
let text = '';
|
|
130
|
+
let promptTokens = 0;
|
|
131
|
+
let completionTokens = 0;
|
|
132
|
+
if (mode === 'demo') {
|
|
133
|
+
// The demo always streams.
|
|
134
|
+
const out = await readStream(res, session);
|
|
135
|
+
text = out.text;
|
|
136
|
+
promptTokens = out.usage?.prompt_tokens ?? 0;
|
|
137
|
+
completionTokens = out.usage?.completion_tokens ?? 0;
|
|
138
|
+
}
|
|
139
|
+
else {
|
|
140
|
+
let completion = await res.json();
|
|
141
|
+
if (session)
|
|
142
|
+
completion = await session.decryptCompletion(completion);
|
|
143
|
+
text = String(completion.choices?.[0]?.message?.content ?? '');
|
|
144
|
+
promptTokens = completion.usage?.prompt_tokens ?? 0;
|
|
145
|
+
completionTokens = completion.usage?.completion_tokens ?? 0;
|
|
146
|
+
}
|
|
147
|
+
const charge = res.headers.get('x-offrouter-charge-usd');
|
|
148
|
+
const remaining = res.headers.get('x-offrouter-demo-remaining');
|
|
149
|
+
return {
|
|
150
|
+
model,
|
|
151
|
+
text: text.trim(),
|
|
152
|
+
promptTokens,
|
|
153
|
+
completionTokens,
|
|
154
|
+
chargeUsd: charge == null ? null : Number(charge),
|
|
155
|
+
receiptUrl: res.headers.get('x-offrouter-receipt-url'),
|
|
156
|
+
encrypted: !!session,
|
|
157
|
+
paidWith: mode,
|
|
158
|
+
transaction: mode === 'wallet' ? settlement(res.headers)?.transaction : undefined,
|
|
159
|
+
demoRemaining: remaining == null ? undefined : Number(remaining),
|
|
160
|
+
};
|
|
161
|
+
}
|
|
162
|
+
async getJson(path, headers = {}) {
|
|
163
|
+
const res = await fetch(`${this.config.baseUrl}${path}`, { headers: { accept: 'application/json', ...headers } });
|
|
164
|
+
if (!res.ok)
|
|
165
|
+
throw await apiError(res);
|
|
166
|
+
return res.json();
|
|
167
|
+
}
|
|
168
|
+
}
|
|
169
|
+
async function apiError(res) {
|
|
170
|
+
const body = await res.json().catch(() => null);
|
|
171
|
+
const err = body?.error;
|
|
172
|
+
const message = (typeof err === 'string' ? err : err?.message) || `OffRouter answered ${res.status}.`;
|
|
173
|
+
return new ApiError(message, res.status, typeof err === 'object' ? err?.code : undefined);
|
|
174
|
+
}
|
|
175
|
+
/** Reads a server sent event stream of chat chunks, decrypting each one when the request was encrypted. */
|
|
176
|
+
async function readStream(res, session) {
|
|
177
|
+
let text = '';
|
|
178
|
+
let usage = null;
|
|
179
|
+
let event = '';
|
|
180
|
+
let buf = '';
|
|
181
|
+
const decoder = new TextDecoder();
|
|
182
|
+
const handle = async (line) => {
|
|
183
|
+
if (line.startsWith('event:')) {
|
|
184
|
+
event = line.slice(6).trim();
|
|
185
|
+
return;
|
|
186
|
+
}
|
|
187
|
+
if (!line.startsWith('data:'))
|
|
188
|
+
return;
|
|
189
|
+
const data = line.slice(5).trim();
|
|
190
|
+
if (!data || data === '[DONE]')
|
|
191
|
+
return;
|
|
192
|
+
// Named events (offrouter.forwarded, offrouter.stored) are the gateway's own notes, not model output.
|
|
193
|
+
if (event) {
|
|
194
|
+
event = '';
|
|
195
|
+
return;
|
|
196
|
+
}
|
|
197
|
+
let chunk = JSON.parse(data);
|
|
198
|
+
if (chunk.error)
|
|
199
|
+
throw new ApiError(chunk.error.message ?? 'The stream ended with an error.', 502, chunk.error.code);
|
|
200
|
+
if (session)
|
|
201
|
+
chunk = await session.decryptChunk(chunk);
|
|
202
|
+
text += chunk.choices?.[0]?.delta?.content ?? '';
|
|
203
|
+
if (chunk.usage)
|
|
204
|
+
usage = chunk.usage;
|
|
205
|
+
};
|
|
206
|
+
for await (const part of res.body) {
|
|
207
|
+
buf += decoder.decode(part, { stream: true });
|
|
208
|
+
let nl;
|
|
209
|
+
while ((nl = buf.indexOf('\n')) !== -1) {
|
|
210
|
+
const line = buf.slice(0, nl).trim();
|
|
211
|
+
buf = buf.slice(nl + 1);
|
|
212
|
+
await handle(line);
|
|
213
|
+
}
|
|
214
|
+
}
|
|
215
|
+
if (buf.trim())
|
|
216
|
+
await handle(buf.trim());
|
|
217
|
+
return { text, usage };
|
|
218
|
+
}
|
package/dist/config.js
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
// Everything is configured through environment variables, so it fits in the
|
|
2
|
+
// "env" block of any MCP client config.
|
|
3
|
+
const off = (v) => !!v && ['0', 'false', 'off', 'no'].includes(v.trim().toLowerCase());
|
|
4
|
+
export function loadConfig(env = process.env) {
|
|
5
|
+
const baseUrl = (env.OFFROUTER_BASE_URL || 'https://offrouter.org').replace(/\/+$/, '');
|
|
6
|
+
const apiKey = env.OFFROUTER_API_KEY?.trim() || undefined;
|
|
7
|
+
const walletKey = env.OFFROUTER_WALLET_KEY?.trim() || undefined;
|
|
8
|
+
if (apiKey && !apiKey.startsWith('off_sk_'))
|
|
9
|
+
throw new Error('OFFROUTER_API_KEY should start with off_sk_');
|
|
10
|
+
if (walletKey && !/^0x[0-9a-fA-F]{64}$/.test(walletKey))
|
|
11
|
+
throw new Error('OFFROUTER_WALLET_KEY should be a 0x prefixed 32 byte hex private key');
|
|
12
|
+
const maxUsd = env.OFFROUTER_MAX_USD ? Number(env.OFFROUTER_MAX_USD) : 0.1;
|
|
13
|
+
if (!Number.isFinite(maxUsd) || maxUsd <= 0)
|
|
14
|
+
throw new Error('OFFROUTER_MAX_USD should be a positive number, like 0.10');
|
|
15
|
+
const network = env.OFFROUTER_NETWORK?.trim().toLowerCase() || undefined;
|
|
16
|
+
if (network && network !== 'base' && network !== 'robinhood')
|
|
17
|
+
throw new Error('OFFROUTER_NETWORK should be base or robinhood');
|
|
18
|
+
const defaultMaxTokens = env.OFFROUTER_MAX_TOKENS ? Number(env.OFFROUTER_MAX_TOKENS) : 2048;
|
|
19
|
+
if (!Number.isInteger(defaultMaxTokens) || defaultMaxTokens < 1)
|
|
20
|
+
throw new Error('OFFROUTER_MAX_TOKENS should be a whole number');
|
|
21
|
+
return {
|
|
22
|
+
baseUrl,
|
|
23
|
+
// A key wins over a wallet if both are set: it is cheaper per call and never needs a signature.
|
|
24
|
+
mode: apiKey ? 'key' : walletKey ? 'wallet' : 'demo',
|
|
25
|
+
apiKey,
|
|
26
|
+
walletKey: walletKey,
|
|
27
|
+
maxUsd,
|
|
28
|
+
network: network,
|
|
29
|
+
defaultMaxTokens,
|
|
30
|
+
e2ee: !off(env.OFFROUTER_E2EE),
|
|
31
|
+
};
|
|
32
|
+
}
|
package/dist/e2ee.js
ADDED
|
@@ -0,0 +1,194 @@
|
|
|
1
|
+
// Client side of end-to-end encryption to the enclave. It encrypts the content
|
|
2
|
+
// of each message to a key that the enclave's attestation report vouches for,
|
|
3
|
+
// and decrypts the answer with a throwaway key made for that one request. The
|
|
4
|
+
// OffRouter gateway in the middle only ever sees ciphertext.
|
|
5
|
+
//
|
|
6
|
+
// This file is just the cryptography. It trusts whatever key it is given, so
|
|
7
|
+
// only hand it a key from a report you verified yourself (see attest.ts).
|
|
8
|
+
//
|
|
9
|
+
// Wire format is "E2EE v2" from the enclave gateway. For every encrypted
|
|
10
|
+
// field: a fresh ephemeral key, ECDH with the recipient's key, HKDF-SHA256 to
|
|
11
|
+
// an AES-256-GCM key, and the field's position in the request bound in as
|
|
12
|
+
// associated data:
|
|
13
|
+
// hex( ephemeral_public_key || nonce (12) || ciphertext || tag (16) )
|
|
14
|
+
import { x25519 } from '@noble/curves/ed25519';
|
|
15
|
+
import { secp256k1 } from '@noble/curves/secp256k1';
|
|
16
|
+
import { hkdf } from '@noble/hashes/hkdf';
|
|
17
|
+
import { sha256 } from '@noble/hashes/sha2';
|
|
18
|
+
export class E2eeError extends Error {
|
|
19
|
+
}
|
|
20
|
+
const hex = (bytes) => [...bytes].map((b) => b.toString(16).padStart(2, '0')).join('');
|
|
21
|
+
function unhex(text) {
|
|
22
|
+
const s = text.replace(/^0x/, '');
|
|
23
|
+
if (s.length % 2 || !/^[0-9a-fA-F]*$/.test(s))
|
|
24
|
+
throw new E2eeError('not hex');
|
|
25
|
+
return Uint8Array.from(s.match(/../g) ?? [], (b) => parseInt(b, 16));
|
|
26
|
+
}
|
|
27
|
+
const utf8 = new TextEncoder();
|
|
28
|
+
const text = new TextDecoder('utf-8', { fatal: true, ignoreBOM: false });
|
|
29
|
+
const SUITES = [
|
|
30
|
+
{
|
|
31
|
+
algo: 'x25519-aes-256-gcm-hkdf-sha256',
|
|
32
|
+
info: 'aci.e2ee.v2.x25519',
|
|
33
|
+
publicLength: 32,
|
|
34
|
+
keypair() {
|
|
35
|
+
const secret = x25519.utils.randomPrivateKey();
|
|
36
|
+
return { secret, public: x25519.getPublicKey(secret) };
|
|
37
|
+
},
|
|
38
|
+
shared: (secret, peer) => x25519.getSharedSecret(secret, peer),
|
|
39
|
+
parse(publicKey) {
|
|
40
|
+
const bytes = unhex(publicKey);
|
|
41
|
+
if (bytes.length !== 32)
|
|
42
|
+
throw new E2eeError('an X25519 public key is 32 bytes');
|
|
43
|
+
return bytes;
|
|
44
|
+
},
|
|
45
|
+
},
|
|
46
|
+
{
|
|
47
|
+
algo: 'secp256k1-aes-256-gcm-hkdf-sha256',
|
|
48
|
+
info: 'aci.e2ee.v2.secp256k1',
|
|
49
|
+
publicLength: 65,
|
|
50
|
+
keypair() {
|
|
51
|
+
const secret = secp256k1.utils.randomPrivateKey();
|
|
52
|
+
return { secret, public: secp256k1.getPublicKey(secret, false) };
|
|
53
|
+
},
|
|
54
|
+
// The x coordinate of the shared point.
|
|
55
|
+
shared: (secret, peer) => secp256k1.getSharedSecret(secret, peer, false).slice(1, 33),
|
|
56
|
+
parse(publicKey) {
|
|
57
|
+
const bytes = unhex(publicKey);
|
|
58
|
+
// 64 bytes is the same key without its 0x04 prefix.
|
|
59
|
+
if (bytes.length === 64)
|
|
60
|
+
return Uint8Array.from([4, ...bytes]);
|
|
61
|
+
if (bytes.length !== 65 || bytes[0] !== 4)
|
|
62
|
+
throw new E2eeError('a secp256k1 public key is 65 bytes, uncompressed');
|
|
63
|
+
return bytes;
|
|
64
|
+
},
|
|
65
|
+
},
|
|
66
|
+
];
|
|
67
|
+
/** The key to encrypt to, out of the ones a report offers: X25519 when it is there. Null when none is usable. */
|
|
68
|
+
export function chooseKey(keys) {
|
|
69
|
+
for (const suite of SUITES) {
|
|
70
|
+
const key = keys?.find((k) => k?.algo === suite.algo && typeof k.public_key === 'string');
|
|
71
|
+
if (key)
|
|
72
|
+
return key;
|
|
73
|
+
}
|
|
74
|
+
return null;
|
|
75
|
+
}
|
|
76
|
+
const aesKey = (suite, shared, usage) => crypto.subtle.importKey('raw', Uint8Array.from(hkdf(sha256, shared, undefined, suite.info, 32)), 'AES-GCM', false, [usage]);
|
|
77
|
+
async function seal(suite, recipient, plaintext, aad) {
|
|
78
|
+
const ephemeral = suite.keypair();
|
|
79
|
+
const nonce = crypto.getRandomValues(new Uint8Array(12));
|
|
80
|
+
const key = await aesKey(suite, suite.shared(ephemeral.secret, recipient), 'encrypt');
|
|
81
|
+
const sealed = new Uint8Array(await crypto.subtle.encrypt({ name: 'AES-GCM', iv: nonce, additionalData: utf8.encode(aad) }, key, utf8.encode(plaintext)));
|
|
82
|
+
return hex(Uint8Array.from([...ephemeral.public, ...nonce, ...sealed]));
|
|
83
|
+
}
|
|
84
|
+
async function open(suite, secret, ciphertext, aad) {
|
|
85
|
+
const bytes = unhex(ciphertext);
|
|
86
|
+
if (bytes.length < suite.publicLength + 12 + 16)
|
|
87
|
+
throw new E2eeError('too short to be a ciphertext');
|
|
88
|
+
const ephemeral = bytes.slice(0, suite.publicLength);
|
|
89
|
+
const nonce = bytes.slice(suite.publicLength, suite.publicLength + 12);
|
|
90
|
+
const key = await aesKey(suite, suite.shared(secret, ephemeral), 'decrypt');
|
|
91
|
+
return text.decode(await crypto.subtle.decrypt({ name: 'AES-GCM', iv: nonce, additionalData: utf8.encode(aad) }, key, bytes.slice(suite.publicLength + 12)));
|
|
92
|
+
}
|
|
93
|
+
// The associated data is canonical JSON (RFC 8785). For these values that is JSON.stringify with the members in
|
|
94
|
+
// alphabetical order, which is the order they are written in here.
|
|
95
|
+
const requestAad = (algo, field, model, nonce, ts) => JSON.stringify({ algo, field, model, nonce, purpose: 'aci.e2ee.request.v2', ts });
|
|
96
|
+
const responseAad = (algo, field, id, model, nonce, ts) => JSON.stringify({ algo, field, id, model, nonce, purpose: 'aci.e2ee.response.v2', ts });
|
|
97
|
+
// What the enclave encrypts in an answer, under `choices.{i}.message` or `choices.{i}.delta`.
|
|
98
|
+
const ANSWER_FIELDS = ['content', 'reasoning', 'reasoning_content'];
|
|
99
|
+
/**
|
|
100
|
+
* Encrypt a chat request to the enclave.
|
|
101
|
+
*
|
|
102
|
+
* `key` is the enclave's key, from a verified attestation report. `model` is
|
|
103
|
+
* the model's name as the enclave knows it (`e2ee.model` in GET /v1/models):
|
|
104
|
+
* every ciphertext is bound to it, so a request encrypted for one model cannot
|
|
105
|
+
* be served by another. `body.model` stays the id you would normally send.
|
|
106
|
+
*
|
|
107
|
+
* What is encrypted: the `content` of every message, as a string or as a
|
|
108
|
+
* whole array of parts. What is not: the model, `max_tokens` and the other
|
|
109
|
+
* settings, tool definitions, and the arguments of tool calls.
|
|
110
|
+
*
|
|
111
|
+
* A session is for one request. To retry, encrypt again: the enclave takes
|
|
112
|
+
* each nonce once.
|
|
113
|
+
*/
|
|
114
|
+
export async function encryptRequest(body, opts) {
|
|
115
|
+
const suite = SUITES.find((s) => s.algo === opts.key?.algo);
|
|
116
|
+
if (!suite)
|
|
117
|
+
throw new E2eeError(`no support for the key type ${JSON.stringify(opts.key?.algo)}`);
|
|
118
|
+
if (!body || !Array.isArray(body.messages))
|
|
119
|
+
throw new E2eeError('`messages` is required');
|
|
120
|
+
if (typeof opts.model !== 'string' || !opts.model)
|
|
121
|
+
throw new E2eeError('the model\'s `e2ee.model` name is required');
|
|
122
|
+
const service = suite.parse(opts.key.public_key);
|
|
123
|
+
const client = suite.keypair();
|
|
124
|
+
const nonce = hex(crypto.getRandomValues(new Uint8Array(32)));
|
|
125
|
+
const ts = Math.floor((opts.now ?? Date.now()) / 1000);
|
|
126
|
+
let sealedAny = false;
|
|
127
|
+
const messages = await Promise.all(body.messages.map(async (m, i) => {
|
|
128
|
+
if (!m || m.content == null)
|
|
129
|
+
return m;
|
|
130
|
+
// An array of parts is encrypted whole, as its JSON.
|
|
131
|
+
const plaintext = typeof m.content === 'string' ? m.content : JSON.stringify(m.content);
|
|
132
|
+
sealedAny = true;
|
|
133
|
+
return { ...m, content: await seal(suite, service, plaintext, requestAad(suite.algo, `messages.${i}.content`, opts.model, nonce, ts)) };
|
|
134
|
+
}));
|
|
135
|
+
if (!sealedAny)
|
|
136
|
+
throw new E2eeError('no message has any content to encrypt');
|
|
137
|
+
const openField = async (value, field, id) => {
|
|
138
|
+
if (typeof value !== 'string' || value === '')
|
|
139
|
+
return value;
|
|
140
|
+
try {
|
|
141
|
+
return await open(suite, client.secret, value, responseAad(suite.algo, field, typeof id === 'string' ? id : '', opts.model, nonce, ts));
|
|
142
|
+
}
|
|
143
|
+
catch {
|
|
144
|
+
throw new E2eeError(`the answer's ${field} did not decrypt with this request's key: it was not made by the enclave for this request`);
|
|
145
|
+
}
|
|
146
|
+
};
|
|
147
|
+
const openChoices = async (response, part) => {
|
|
148
|
+
if (!response || !Array.isArray(response.choices))
|
|
149
|
+
return response;
|
|
150
|
+
const choices = await Promise.all(response.choices.map(async (choice) => {
|
|
151
|
+
const inner = choice?.[part];
|
|
152
|
+
if (!inner || typeof inner !== 'object')
|
|
153
|
+
return choice;
|
|
154
|
+
const opened = { ...inner };
|
|
155
|
+
// The field path names the choice by its own index, which a stream carries on each chunk.
|
|
156
|
+
const index = Number.isInteger(choice.index) ? choice.index : response.choices.indexOf(choice);
|
|
157
|
+
for (const name of ANSWER_FIELDS)
|
|
158
|
+
if (name in inner)
|
|
159
|
+
opened[name] = await openField(inner[name], `choices.${index}.${part}.${name}`, response.id);
|
|
160
|
+
return { ...choice, [part]: opened };
|
|
161
|
+
}));
|
|
162
|
+
return { ...response, choices };
|
|
163
|
+
};
|
|
164
|
+
return {
|
|
165
|
+
body: { ...body, messages },
|
|
166
|
+
headers: {
|
|
167
|
+
'X-E2EE-Version': '2',
|
|
168
|
+
'X-Client-Pub-Key': hex(client.public),
|
|
169
|
+
'X-Model-Pub-Key': hex(service),
|
|
170
|
+
'X-E2EE-Nonce': nonce,
|
|
171
|
+
'X-E2EE-Timestamp': String(ts),
|
|
172
|
+
},
|
|
173
|
+
applied: (headers) => headers.get('x-e2ee-applied') === 'true',
|
|
174
|
+
decryptCompletion: (completion) => openChoices(completion, 'message'),
|
|
175
|
+
decryptChunk: (chunk) => openChoices(chunk, 'delta'),
|
|
176
|
+
};
|
|
177
|
+
}
|
|
178
|
+
/** The enclave's side, for tests and for anything that plays the enclave: open a request field, seal an answer field. */
|
|
179
|
+
export const enclave = {
|
|
180
|
+
openRequestField(algo, secret, ciphertext, field, model, nonce, ts) {
|
|
181
|
+
const suite = SUITES.find((s) => s.algo === algo);
|
|
182
|
+
return open(suite, secret, ciphertext, requestAad(algo, field, model, nonce, ts));
|
|
183
|
+
},
|
|
184
|
+
sealAnswerField(algo, clientPublic, plaintext, field, id, model, nonce, ts) {
|
|
185
|
+
const suite = SUITES.find((s) => s.algo === algo);
|
|
186
|
+
return seal(suite, suite.parse(clientPublic), plaintext, responseAad(algo, field, id, model, nonce, ts));
|
|
187
|
+
},
|
|
188
|
+
keypair(algo) {
|
|
189
|
+
const pair = SUITES.find((s) => s.algo === algo).keypair();
|
|
190
|
+
return { secret: pair.secret, public: hex(pair.public) };
|
|
191
|
+
},
|
|
192
|
+
requestAad,
|
|
193
|
+
responseAad,
|
|
194
|
+
};
|
package/dist/index.js
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js';
|
|
3
|
+
import { OffRouter } from './client.js';
|
|
4
|
+
import { loadConfig } from './config.js';
|
|
5
|
+
import { createServer } from './server.js';
|
|
6
|
+
async function main() {
|
|
7
|
+
const config = loadConfig();
|
|
8
|
+
const api = new OffRouter(config);
|
|
9
|
+
const server = createServer(api);
|
|
10
|
+
await server.connect(new StdioServerTransport());
|
|
11
|
+
// stdout belongs to the protocol, so anything for humans goes to stderr.
|
|
12
|
+
const how = config.mode === 'wallet' ? `wallet ${api.wallet.address}` : config.mode === 'key' ? 'prepaid key' : 'free demo';
|
|
13
|
+
console.error(`offrouter mcp ${config.baseUrl} · paying with ${how} · e2ee ${config.e2ee ? 'on' : 'off'}`);
|
|
14
|
+
}
|
|
15
|
+
main().catch((e) => {
|
|
16
|
+
console.error(`offrouter mcp: ${e.message}`);
|
|
17
|
+
process.exit(1);
|
|
18
|
+
});
|
package/dist/pay.js
ADDED
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
// Pay per call with x402, signed by your own wallet on this machine. The
|
|
2
|
+
// private key never leaves the process: it signs an EIP-3009 transfer for the
|
|
3
|
+
// exact amount the server quotes, and we refuse to sign anything above
|
|
4
|
+
// OFFROUTER_MAX_USD no matter what the server asks for.
|
|
5
|
+
//
|
|
6
|
+
// OffRouter quotes a ceiling (your full input plus max_tokens of output) and
|
|
7
|
+
// refunds the unused part to the paying wallet after the call.
|
|
8
|
+
import { x402Client, wrapFetchWithPayment } from '@x402/fetch';
|
|
9
|
+
import { ExactEvmScheme } from '@x402/evm/exact/client';
|
|
10
|
+
import { privateKeyToAccount } from 'viem/accounts';
|
|
11
|
+
// Chains where a plain EIP-3009 signature settles. Arc is left out on purpose:
|
|
12
|
+
// OffRouter takes Arc payments through Circle Gateway, which needs a Gateway
|
|
13
|
+
// balance rather than a wallet signature. The server only ever offers its own
|
|
14
|
+
// environment, so listing testnets here is harmless on mainnet and vice versa.
|
|
15
|
+
const CHAINS = [
|
|
16
|
+
{ id: 'base', caip2: 'eip155:8453', asset: '0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913' },
|
|
17
|
+
{ id: 'base', caip2: 'eip155:84532', asset: '0x036CbD53842c5426634e7929541eC2318f3dCF7e' },
|
|
18
|
+
{ id: 'robinhood', caip2: 'eip155:4663', asset: '0x5fc5360D0400a0Fd4f2af552ADD042D716F1d168' },
|
|
19
|
+
{ id: 'robinhood', caip2: 'eip155:46630', asset: '0x7E955252E15c84f5768B83c41a71F9eba181802F' },
|
|
20
|
+
];
|
|
21
|
+
export function makeWallet(config) {
|
|
22
|
+
const account = privateKeyToAccount(config.walletKey);
|
|
23
|
+
const chains = CHAINS.filter((c) => !config.network || c.id === config.network);
|
|
24
|
+
const cap = String(Math.round(config.maxUsd * 1e6));
|
|
25
|
+
const client = x402Client.fromConfig({
|
|
26
|
+
schemes: chains.map((c) => ({ network: c.caip2, client: new ExactEvmScheme(account) })),
|
|
27
|
+
spendControls: {
|
|
28
|
+
maxAmountPerPayment: `$${config.maxUsd}`,
|
|
29
|
+
allowedAssets: chains.map((c) => ({ network: c.caip2, asset: c.asset, maxAmountPerPayment: cap })),
|
|
30
|
+
},
|
|
31
|
+
});
|
|
32
|
+
return {
|
|
33
|
+
address: account.address,
|
|
34
|
+
fetch: wrapFetchWithPayment(fetch, client),
|
|
35
|
+
chains: [...new Set(chains.map((c) => c.id))],
|
|
36
|
+
};
|
|
37
|
+
}
|
|
38
|
+
/** The settlement transaction from a PAYMENT-RESPONSE header, if there is one. */
|
|
39
|
+
export function settlement(headers) {
|
|
40
|
+
const raw = headers.get('payment-response');
|
|
41
|
+
if (!raw)
|
|
42
|
+
return null;
|
|
43
|
+
try {
|
|
44
|
+
return JSON.parse(Buffer.from(raw, 'base64').toString('utf8'));
|
|
45
|
+
}
|
|
46
|
+
catch {
|
|
47
|
+
return null;
|
|
48
|
+
}
|
|
49
|
+
}
|
package/dist/server.js
ADDED
|
@@ -0,0 +1,243 @@
|
|
|
1
|
+
import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js';
|
|
2
|
+
import { z } from 'zod';
|
|
3
|
+
export const VERSION = '0.1.0';
|
|
4
|
+
const MAX_COMPARE = 4;
|
|
5
|
+
const text = (t) => ({ content: [{ type: 'text', text: t }] });
|
|
6
|
+
const fail = (t) => ({ content: [{ type: 'text', text: t }], isError: true });
|
|
7
|
+
// Enough digits for sub-cent charges, without the trailing zeros.
|
|
8
|
+
const usd = (n) => '$' + ((Number(n) || 0).toFixed(6).replace(/\.?0+$/, '') || '0');
|
|
9
|
+
function footer(r) {
|
|
10
|
+
const parts = [r.model, `${r.promptTokens} in / ${r.completionTokens} out tokens`];
|
|
11
|
+
if (r.paidWith === 'demo')
|
|
12
|
+
parts.push(`free demo${r.demoRemaining != null ? `, ${r.demoRemaining} left today` : ''}`);
|
|
13
|
+
else if (r.chargeUsd != null)
|
|
14
|
+
parts.push(usd(r.chargeUsd));
|
|
15
|
+
if (r.transaction && !r.transaction.startsWith('dryrun_'))
|
|
16
|
+
parts.push(`paid in tx ${r.transaction}`);
|
|
17
|
+
parts.push(r.encrypted ? 'encrypted end to end' : 'not encrypted');
|
|
18
|
+
if (r.receiptUrl)
|
|
19
|
+
parts.push(`receipt ${r.receiptUrl}`);
|
|
20
|
+
return parts.join(' · ');
|
|
21
|
+
}
|
|
22
|
+
const date = (sec) => (sec ? new Date(sec * 1000).toISOString().slice(0, 16).replace('T', ' ') : 'never');
|
|
23
|
+
/** Where it stands: ready once it passed its evaluation and has run well for a day. */
|
|
24
|
+
function standing(a) {
|
|
25
|
+
const s = a.standing;
|
|
26
|
+
if (!s)
|
|
27
|
+
return a.status;
|
|
28
|
+
if (s.ready)
|
|
29
|
+
return 'ready';
|
|
30
|
+
const why = [];
|
|
31
|
+
if (!s.working?.passed)
|
|
32
|
+
why.push(s.working ? `evaluation ${s.working.score}/100, not passed` : 'not evaluated');
|
|
33
|
+
if (s.alive && !s.alive.ready)
|
|
34
|
+
why.push(`${s.alive.good_runs}/${s.alive.needs_runs} good runs, live ${Math.floor(s.alive.live_for_sec / 3600)}h of ${s.alive.needs_sec / 3600}h`);
|
|
35
|
+
return `not ready yet (${why.join('; ')})`;
|
|
36
|
+
}
|
|
37
|
+
function agentLine(a, base, full = false) {
|
|
38
|
+
const purpose = !full && a.purpose.length > 160 ? a.purpose.slice(0, 157).trimEnd() + '...' : a.purpose;
|
|
39
|
+
return [
|
|
40
|
+
`## ${a.name}`,
|
|
41
|
+
`id ${a.id} | ${a.status}${a.status === 'active' ? `, ${standing(a)}` : ''} | model ${a.model} | tools ${a.tools.join(', ') || 'none'}`,
|
|
42
|
+
`${a.stats.good_runs} good / ${a.stats.failed_runs} failed runs, last ${date(a.last_run_at ?? a.stats.last_good_run_at)}`,
|
|
43
|
+
purpose,
|
|
44
|
+
`${base}/agents/${a.id}`,
|
|
45
|
+
].join('\n');
|
|
46
|
+
}
|
|
47
|
+
function instructions(api) {
|
|
48
|
+
const how = {
|
|
49
|
+
key: 'Calls are billed per token to the connected prepaid key.',
|
|
50
|
+
wallet: `Calls are paid one at a time from the connected wallet, capped at ${usd(api.config.maxUsd)} each. Unused budget is refunded to the wallet.`,
|
|
51
|
+
demo: 'No key or wallet is set up, so calls go through the free demo: one small model and a few calls per day.',
|
|
52
|
+
}[api.config.mode];
|
|
53
|
+
return [
|
|
54
|
+
'OffRouter runs open models privately. Every call is served inside a GPU TEE, nothing is logged, and each answer comes with a signed receipt.',
|
|
55
|
+
api.config.e2ee ? 'Prompts are encrypted on this machine to a key the enclave proves it holds, so the gateway only sees ciphertext.' : '',
|
|
56
|
+
how,
|
|
57
|
+
'Use list_models to pick a model, then ask for one answer or compare to run the same prompt on several models.',
|
|
58
|
+
'list_agents and get_agent show the private agents builders have published on OffRouter Studio.',
|
|
59
|
+
]
|
|
60
|
+
.filter(Boolean)
|
|
61
|
+
.join(' ');
|
|
62
|
+
}
|
|
63
|
+
export function createServer(api) {
|
|
64
|
+
const server = new McpServer({ name: 'offrouter', title: 'OffRouter', version: VERSION, websiteUrl: 'https://offrouter.org' }, { instructions: instructions(api) });
|
|
65
|
+
server.registerTool('list_models', {
|
|
66
|
+
title: 'List models',
|
|
67
|
+
description: 'List the private models on OffRouter with their context length and price in USD per million tokens. Use it to find a model id for ask or compare.',
|
|
68
|
+
inputSchema: { search: z.string().optional().describe('Filter on the model id, like "qwen" or "deepseek".') },
|
|
69
|
+
annotations: { readOnlyHint: true, openWorldHint: true },
|
|
70
|
+
}, async ({ search }) => {
|
|
71
|
+
const models = await api.listModels();
|
|
72
|
+
if (api.config.mode === 'demo') {
|
|
73
|
+
const demo = await api.demoInfo().catch(() => null);
|
|
74
|
+
const allowed = demo?.models ?? [];
|
|
75
|
+
const lines = models.filter((m) => allowed.includes(m.id)).map((m) => `${m.id} | ${Math.round(m.context_length / 1000)}k ctx`);
|
|
76
|
+
return text(`Running on the free demo, which allows: ${lines.length ? lines.join(', ') : 'nothing right now'}.` +
|
|
77
|
+
(demo ? ` ${demo.remaining} of ${demo.daily_per_ip} calls left today, up to ${demo.max_tokens} output tokens each.` : '') +
|
|
78
|
+
`\n\nThe full catalog has ${models.length} models. Set OFFROUTER_API_KEY or OFFROUTER_WALLET_KEY to use any of them.`);
|
|
79
|
+
}
|
|
80
|
+
const q = search?.toLowerCase() ?? '';
|
|
81
|
+
const rows = models.filter((m) => !q || m.id.toLowerCase().includes(q));
|
|
82
|
+
if (!rows.length)
|
|
83
|
+
return text(`No models match "${search}".`);
|
|
84
|
+
const lines = rows.map((m) => `${m.id} | ${Math.round(m.context_length / 1000)}k ctx | ${usd(m.input_usdc_per_mtok)} in / ${usd(m.output_usdc_per_mtok)} out per M tokens` +
|
|
85
|
+
((m.input_modalities ?? []).length > 1 ? ` | reads ${m.input_modalities.slice(1).join(' and ')}` : '') +
|
|
86
|
+
(m.discount_pct ? ` | ${m.discount_pct}% off` : '') +
|
|
87
|
+
(api.config.e2ee && !m.e2ee?.model ? ' | no e2ee' : ''));
|
|
88
|
+
return text(`${rows.length} private models, all served in a GPU TEE:\n\n${lines.join('\n')}`);
|
|
89
|
+
});
|
|
90
|
+
const chatShape = {
|
|
91
|
+
prompt: z.string().min(1).describe('The user message.'),
|
|
92
|
+
system: z.string().optional().describe('Optional system prompt.'),
|
|
93
|
+
max_tokens: z.number().int().min(1).optional().describe('Cap on output tokens. Reasoning models need a generous budget.'),
|
|
94
|
+
};
|
|
95
|
+
server.registerTool('ask', {
|
|
96
|
+
title: 'Ask a private model',
|
|
97
|
+
description: 'Send a prompt to an OffRouter model and get the answer back. It runs inside a GPU TEE and is not logged, so it is a good fit for sensitive material, a second opinion, or cheap bulk work.',
|
|
98
|
+
inputSchema: {
|
|
99
|
+
model: z.string().min(1).describe('Model id from list_models, like "gemma-4-31b".'),
|
|
100
|
+
...chatShape,
|
|
101
|
+
temperature: z.number().min(0).max(2).optional(),
|
|
102
|
+
},
|
|
103
|
+
annotations: { readOnlyHint: false, openWorldHint: true },
|
|
104
|
+
}, async (args) => {
|
|
105
|
+
try {
|
|
106
|
+
const r = await api.chat(args);
|
|
107
|
+
return text(`${r.text || '(the model returned no text)'}\n\n${footer(r)}`);
|
|
108
|
+
}
|
|
109
|
+
catch (e) {
|
|
110
|
+
return fail(e.message);
|
|
111
|
+
}
|
|
112
|
+
});
|
|
113
|
+
server.registerTool('compare', {
|
|
114
|
+
title: 'Compare models',
|
|
115
|
+
description: `Run the same prompt on 2 to ${MAX_COMPARE} models in parallel and get every answer with its cost side by side.`,
|
|
116
|
+
inputSchema: {
|
|
117
|
+
models: z.array(z.string().min(1)).min(2).max(MAX_COMPARE).describe('Model ids from list_models.'),
|
|
118
|
+
...chatShape,
|
|
119
|
+
},
|
|
120
|
+
annotations: { readOnlyHint: false, openWorldHint: true },
|
|
121
|
+
}, async ({ models, ...rest }) => {
|
|
122
|
+
const ids = [...new Set(models)];
|
|
123
|
+
if (ids.length < 2)
|
|
124
|
+
return fail('Pass at least two different model ids.');
|
|
125
|
+
if (api.config.mode === 'demo')
|
|
126
|
+
return fail('compare needs a key or a wallet. The free demo only has one model.');
|
|
127
|
+
const results = await Promise.allSettled(ids.map((model) => api.chat({ model, ...rest })));
|
|
128
|
+
const sections = results.map((r, i) => r.status === 'fulfilled' ? `## ${ids[i]}\n\n${r.value.text || '(no text)'}\n\n${footer(r.value)}` : `## ${ids[i]}\n\nError: ${r.reason.message}`);
|
|
129
|
+
return { content: [{ type: 'text', text: sections.join('\n\n---\n\n') }], isError: results.every((r) => r.status === 'rejected') };
|
|
130
|
+
});
|
|
131
|
+
server.registerTool('account', {
|
|
132
|
+
title: 'Account',
|
|
133
|
+
description: 'Show how this server pays for calls: the prepaid key balance and recent spend, the wallet address and per call cap, or what is left on the free demo.',
|
|
134
|
+
inputSchema: {},
|
|
135
|
+
annotations: { readOnlyHint: true, openWorldHint: true },
|
|
136
|
+
}, async () => {
|
|
137
|
+
try {
|
|
138
|
+
const { mode } = api.config;
|
|
139
|
+
if (mode === 'key') {
|
|
140
|
+
const k = await api.keyInfo();
|
|
141
|
+
const recent = (k.recent_usage ?? [])
|
|
142
|
+
.slice(0, 5)
|
|
143
|
+
.map((u) => ` ${String(u.at).slice(0, 16).replace('T', ' ')} ${u.model} ${u.prompt_tokens + u.completion_tokens} tokens ${usd(u.charge_usdc)}`);
|
|
144
|
+
return text([
|
|
145
|
+
`Prepaid key ${k.key_prefix}...`,
|
|
146
|
+
`Balance: ${usd(k.balance_usdc)}`,
|
|
147
|
+
`Spent: ${usd(k.spent_usdc)} over ${k.totals?.requests ?? 0} requests`,
|
|
148
|
+
recent.length ? `Recent:\n${recent.join('\n')}` : '',
|
|
149
|
+
'Top up at https://offrouter.org/dashboard',
|
|
150
|
+
]
|
|
151
|
+
.filter(Boolean)
|
|
152
|
+
.join('\n'));
|
|
153
|
+
}
|
|
154
|
+
if (mode === 'wallet') {
|
|
155
|
+
const w = api.wallet;
|
|
156
|
+
return text([
|
|
157
|
+
`Paying per call from wallet ${w.address}`,
|
|
158
|
+
`Chains: ${w.chains.join(', ')} (USDC on Base, USDG on Robinhood Chain)`,
|
|
159
|
+
`Cap per call: ${usd(api.config.maxUsd)}`,
|
|
160
|
+
'Each call is quoted as a ceiling and the unused part is refunded to this wallet.',
|
|
161
|
+
].join('\n'));
|
|
162
|
+
}
|
|
163
|
+
const d = await api.demoInfo();
|
|
164
|
+
return text(`Free demo: ${d.remaining} of ${d.daily_per_ip} calls left today (resets ${d.resets_at}). Models: ${(d.models ?? []).join(', ')}.\n` +
|
|
165
|
+
'For the full catalog, set OFFROUTER_API_KEY (get one at https://offrouter.org/dashboard) or OFFROUTER_WALLET_KEY to pay per call.');
|
|
166
|
+
}
|
|
167
|
+
catch (e) {
|
|
168
|
+
return fail(e.message);
|
|
169
|
+
}
|
|
170
|
+
});
|
|
171
|
+
server.registerTool('verify_enclave', {
|
|
172
|
+
title: 'Verify the enclave',
|
|
173
|
+
description: 'Check the enclave right now from this machine: the Intel TDX quote, that the report is fresh, and that it booted the published code. Returns the key prompts are encrypted to.',
|
|
174
|
+
inputSchema: {},
|
|
175
|
+
annotations: { readOnlyHint: true, openWorldHint: true },
|
|
176
|
+
}, async () => {
|
|
177
|
+
try {
|
|
178
|
+
const e = await api.enclave.refresh();
|
|
179
|
+
return text([
|
|
180
|
+
'The enclave checks out.',
|
|
181
|
+
'- Intel TDX quote verified up to Intel\'s root certificate',
|
|
182
|
+
'- The report covers a random nonce made here just now, so it is not a replay',
|
|
183
|
+
'- The boot log replays to the measured launch file',
|
|
184
|
+
`Key set: ${e.keyset}`,
|
|
185
|
+
`Encryption key: ${e.key.algo} ${e.key.public_key.slice(0, 16)}...`,
|
|
186
|
+
`Valid until: ${new Date(e.notAfter * 1000).toISOString()}`,
|
|
187
|
+
e.commit ? `Enclave gateway code: ${e.repo ? `${e.repo}/tree/${e.commit}` : e.commit}` : '',
|
|
188
|
+
api.config.e2ee ? 'Prompts from this server are encrypted to that key.' : 'Encryption is off (OFFROUTER_E2EE=off), so prompts are sent in the clear to the gateway.',
|
|
189
|
+
]
|
|
190
|
+
.filter(Boolean)
|
|
191
|
+
.join('\n'));
|
|
192
|
+
}
|
|
193
|
+
catch (e) {
|
|
194
|
+
return fail(`The enclave did not verify: ${e.message}`);
|
|
195
|
+
}
|
|
196
|
+
});
|
|
197
|
+
server.registerTool('list_agents', {
|
|
198
|
+
title: 'List Studio agents',
|
|
199
|
+
description: 'List the agents published on OffRouter Studio, newest first: what each does, its model and tools, how its runs have gone, and whether it has earned a ready standing. Each runs privately in a TEE with its own wallet.',
|
|
200
|
+
inputSchema: {
|
|
201
|
+
search: z.string().optional().describe('Filter on the name or purpose, like "trading" or "news".'),
|
|
202
|
+
ready_only: z.boolean().optional().describe('Only agents that passed their evaluation and have run well for a day.'),
|
|
203
|
+
},
|
|
204
|
+
annotations: { readOnlyHint: true, openWorldHint: true },
|
|
205
|
+
}, async ({ search, ready_only }) => {
|
|
206
|
+
try {
|
|
207
|
+
const q = search?.toLowerCase() ?? '';
|
|
208
|
+
const agents = (await api.listAgents()).filter((a) => (!q || `${a.name} ${a.purpose}`.toLowerCase().includes(q)) && (!ready_only || a.standing?.ready));
|
|
209
|
+
if (!agents.length)
|
|
210
|
+
return text(search || ready_only ? 'No Studio agents match.' : 'No agents are published on Studio yet.');
|
|
211
|
+
const base = api.config.baseUrl;
|
|
212
|
+
return text(`${agents.length} Studio agent${agents.length === 1 ? '' : 's'}:\n\n${agents.map((a) => agentLine(a, base)).join('\n\n')}\n\nUse get_agent with an id for its examples and evaluation.`);
|
|
213
|
+
}
|
|
214
|
+
catch (e) {
|
|
215
|
+
return fail(e.message);
|
|
216
|
+
}
|
|
217
|
+
});
|
|
218
|
+
server.registerTool('get_agent', {
|
|
219
|
+
title: 'Get a Studio agent',
|
|
220
|
+
description: 'Show one Studio agent in full: its purpose, example tasks, latest evaluation, standing, wallet and config hash.',
|
|
221
|
+
inputSchema: { id: z.string().describe('The agent id from list_agents, 0x followed by 32 hex characters.') },
|
|
222
|
+
annotations: { readOnlyHint: true, openWorldHint: true },
|
|
223
|
+
}, async ({ id }) => {
|
|
224
|
+
try {
|
|
225
|
+
const a = await api.getAgent(id.trim());
|
|
226
|
+
const ev = a.evaluation;
|
|
227
|
+
return text([
|
|
228
|
+
agentLine(a, api.config.baseUrl, true),
|
|
229
|
+
`Builder ${a.builder} · agent wallet ${a.address}`,
|
|
230
|
+
`Config hash ${a.config_hash}`,
|
|
231
|
+
`Created ${date(a.created_at)} · ${a.scheduled ? 'runs on a schedule' : 'runs on request'} · ${usd(Number(a.stats.credits_spent) / 1e6)} of credits spent`,
|
|
232
|
+
ev ? `Last evaluation ${date(ev.at)}: ${ev.score}/100, ${ev.passed ? 'passed' : 'not passed'}${ev.leaked ? ', leaked its instructions' : ''} (${ev.judge} judge)` : 'Not evaluated yet.',
|
|
233
|
+
a.examples?.length ? `Examples:\n${a.examples.map((x) => `- ${x.task}\n Expect: ${x.expect}`).join('\n')}` : '',
|
|
234
|
+
]
|
|
235
|
+
.filter(Boolean)
|
|
236
|
+
.join('\n'));
|
|
237
|
+
}
|
|
238
|
+
catch (e) {
|
|
239
|
+
return fail(e.message);
|
|
240
|
+
}
|
|
241
|
+
});
|
|
242
|
+
return server;
|
|
243
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,60 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@offrouter/mcp",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"
|
|
5
|
-
"
|
|
6
|
-
|
|
3
|
+
"version": "0.1.0",
|
|
4
|
+
"description": "MCP server for OffRouter private inference. Every model runs in a GPU TEE, and your prompts are encrypted to the enclave before they leave your machine.",
|
|
5
|
+
"license": "MIT",
|
|
6
|
+
"author": "OffRouter",
|
|
7
|
+
"homepage": "https://offrouter.org",
|
|
8
|
+
"repository": {
|
|
9
|
+
"type": "git",
|
|
10
|
+
"url": "git+https://github.com/offrouter/mcp.git"
|
|
11
|
+
},
|
|
12
|
+
"bugs": "https://github.com/offrouter/mcp/issues",
|
|
13
|
+
"keywords": [
|
|
14
|
+
"mcp",
|
|
15
|
+
"model-context-protocol",
|
|
16
|
+
"private-inference",
|
|
17
|
+
"tee",
|
|
18
|
+
"confidential-computing",
|
|
19
|
+
"x402",
|
|
20
|
+
"llm"
|
|
21
|
+
],
|
|
22
|
+
"type": "module",
|
|
23
|
+
"bin": {
|
|
24
|
+
"offrouter-mcp": "dist/index.js"
|
|
25
|
+
},
|
|
26
|
+
"files": [
|
|
27
|
+
"dist",
|
|
28
|
+
"README.md",
|
|
29
|
+
"LICENSE"
|
|
30
|
+
],
|
|
31
|
+
"engines": {
|
|
32
|
+
"node": ">=20"
|
|
33
|
+
},
|
|
34
|
+
"publishConfig": {
|
|
35
|
+
"access": "public"
|
|
36
|
+
},
|
|
37
|
+
"scripts": {
|
|
38
|
+
"build": "tsc -p tsconfig.build.json",
|
|
39
|
+
"dev": "tsx src/index.ts",
|
|
40
|
+
"typecheck": "tsc --noEmit",
|
|
41
|
+
"test": "vitest run",
|
|
42
|
+
"prepare": "npm run build"
|
|
43
|
+
},
|
|
44
|
+
"dependencies": {
|
|
45
|
+
"@modelcontextprotocol/sdk": "^1.32.1",
|
|
46
|
+
"@noble/curves": "^1.9.7",
|
|
47
|
+
"@noble/hashes": "^1.8.0",
|
|
48
|
+
"@phala/aci-verifier": "^0.8.0",
|
|
49
|
+
"@x402/evm": "^2.28.0",
|
|
50
|
+
"@x402/fetch": "^2.28.0",
|
|
51
|
+
"viem": "^2.55.17",
|
|
52
|
+
"zod": "^4.6.7"
|
|
53
|
+
},
|
|
54
|
+
"devDependencies": {
|
|
55
|
+
"@types/node": "^22.20.5",
|
|
56
|
+
"tsx": "^4.23.12",
|
|
57
|
+
"typescript": "^5.9.3",
|
|
58
|
+
"vitest": "^3.2.4"
|
|
59
|
+
}
|
|
60
|
+
}
|