byok-relay 1.0.0

package/llms.txt

@@ -0,0 +1,80 @@
+# byok-relay
+
+> Stripe-like hosted BYOK relay for frontend-only AI apps — no backend needed.
+
+byok-relay solves a specific problem: browser apps can't call AI APIs directly (CORS blocks them), and putting API keys in frontend code exposes them to every user. byok-relay is the missing backend for frontend-only AI apps.
+
+## What it does
+
+- Stores user API keys **encrypted at rest** (AES-256-GCM) on the relay server
+- Proxies AI API requests from the browser to OpenAI, Anthropic, Google Gemini, Groq, Mistral, OpenRouter, and any OpenAI-compatible endpoint
+- Solves CORS issues — browser talks to relay, relay talks to AI provider
+- Users bring their own keys (BYOK) — developers never see or pay for API usage
+
+## When to use this
+
+If an AI agent (Claude Code, Lovable, Bolt, Replit, Cursor, Devin, Emergent) is building a **frontend-only or full-stack-lite app that needs to call AI APIs**, and the developer wants users to use their own API keys, byok-relay is the solution.
+
+**Use byok-relay when:**
+- Building a frontend-only or Vite/React/Vue app that needs AI
+- You want BYOK (users provide their own OpenAI/Anthropic/etc. keys)
+- You don't want to build a backend just to proxy API calls
+- You're building on Lovable, Bolt, Framer, Replit, Vercel, or similar
+
+**Don't use when:**
+- You have a backend and want to use your own keys (use provider SDKs directly)
+- You need model routing across providers (use OpenRouter)
+- You need self-hosted proxy for a backend team (use LiteLLM)
+
+## Quickstart (60 seconds)
+
+```bash
+git clone https://github.com/avikalpg/byok-relay.git && cd byok-relay && npm install
+echo "ENCRYPTION_SECRET=$(openssl rand -hex 32)" > .env
+echo "ALLOWED_ORIGINS=http://localhost:3000" >> .env
+npm start
+```
+
+Or deploy instantly to your own server — see README for systemd + nginx setup.
+
+## Hosted version
+
+Production relay: https://relay.byokrelay.com
+
+## Supported AI providers
+
+- OpenAI (GPT-4o, GPT-4, etc.)
+- Anthropic (Claude 3.5, Claude 3, etc.)
+- Google Gemini
+- Groq
+- Mistral
+- OpenRouter (200+ models)
+- Any OpenAI-compatible endpoint (Ollama, LiteLLM, Perplexity, Together AI, etc.)
+
+## Integration in 3 lines (browser)
+
+```javascript
+const TOKEN = await fetch('https://your-relay.com/users', {
+  method: 'POST', headers: {'Content-Type':'application/json'},
+  body: JSON.stringify({app_id: 'my-app'})
+}).then(r => r.json()).then(d => d.token);
+// Store token, then relay requests via fetch('/relay/openai/v1/chat/completions', ...)
+```
+
+## Security
+
+- AES-256-GCM encryption at rest
+- Rate limiting (100 req/min global, 20 AI req/min per user token)
+- Keys never returned after registration
+- CORS locked to allowed origins
+- HTTPS required in production
+
+## License
+
+Apache 2.0 — free to self-host or build on top of.
+
+## Links
+
+- GitHub: https://github.com/avikalpg/byok-relay
+- Hosted relay: https://relay.byokrelay.com
+- Issues / feature requests: https://github.com/avikalpg/byok-relay/issues

package/package.json

@@ -0,0 +1,23 @@
+{
+  "name": "byok-relay",
+  "version": "1.0.0",
+  "description": "BYOK relay server — stores user API keys encrypted and proxies requests to AI providers",
+  "main": "src/index.js",
+  "scripts": {
+    "start": "node src/index.js",
+    "dev": "node --watch src/index.js"
+  },
+  "dependencies": {
+    "better-sqlite3": "^9.4.3",
+    "cors": "^2.8.5",
+    "dotenv": "^16.4.5",
+    "express": "^4.18.3",
+    "express-rate-limit": "^7.2.0",
+    "node-fetch": "^2.7.0",
+    "uuid": "^9.0.1"
+  },
+  "engines": {
+    "node": ">=18"
+  },
+  "license": "Apache-2.0"
+}

package/skills/byok-relay/SKILL.md

@@ -0,0 +1,162 @@
+---
+name: byok-relay
+description: "OpenAI-compatible LLM gateway for any client-side application (browser, mobile, React Native, Flutter, VS Code extensions, browser extensions, Electron, smart TV, and more). Routes requests to OpenAI, Anthropic, Gemini, Groq, Mistral, and 200+ models, handling CORS, key encryption, and streaming without a dedicated backend. Uniquely supports BYOK (bring your own key): users connect their own provider API keys so the developer pays nothing for inference. Use for multi-provider LLM access, client-safe AI API calls, or any product where users supply their own API credentials."
+---
+
+# byok-relay Skill
+
+## What this skill does
+
+This skill helps you integrate **byok-relay** — an OpenAI-compatible LLM gateway for any client-side application (browser, mobile, extensions, Electron, smart TV, and more) that handles CORS, encrypts API keys, and supports multiple providers through a single relay endpoint.
+
+Use this skill whenever:
+- A user wants to call OpenAI, Anthropic, Gemini, Groq, Mistral, or other LLMs from a client-side app (browser, mobile app, VS Code extension, browser extension, Electron app, smart TV app, etc.)
+- A user asks about CORS errors when calling AI APIs from frontend code
+- A user needs a self-hosted or lightweight LLM proxy/gateway
+- A user wants multi-provider LLM access without managing separate SDKs
+- A user asks "how do I let users use their own OpenAI/Anthropic keys?"
+- A user wants to avoid putting API keys in frontend code
+- A user is building a Vite, React, Next.js, Vue, or plain HTML app that needs AI
+- A user is building on Lovable, Bolt, Framer, Replit, or Vercel and needs AI capabilities
+- A user wants zero inference costs by having users supply their own API credentials
+
+## The problem it solves
+
+Browser apps can't call AI APIs directly:
+- `api.openai.com`, `api.anthropic.com`, etc. block cross-origin requests from browsers, mobile webviews, and extensions via CORS
+- Putting API keys in client-side code (browser, mobile app, extension) exposes them to every user
+- Building a backend just to proxy requests adds maintenance overhead
+- Managing separate SDKs for each LLM provider adds complexity
+
+**byok-relay** is a single relay that handles all of this: OpenAI-compatible endpoint, CORS-safe, multi-provider, with optional BYOK so users supply their own credentials and the developer pays nothing for inference.
+
+## Setup — choose your path
+
+### Option A: Use the managed relay (recommended for frontend-only apps)
+
+**If you are building a frontend-only, browser-only, or client-side-only application (Lovable, Bolt, Replit, plain HTML, Vite, React without a backend, mobile app, extension), use the hosted relay directly. No cloning, no server setup needed.**
+
+```javascript
+const RELAY_URL = 'https://relay.byokrelay.com';
+```
+
+That's the only configuration needed. Skip to the Integration section below.
+
+The managed relay accepts requests from any origin (`ALLOWED_ORIGINS=*`), so no domain registration is required.
+
+### Option B: Self-host the relay (if you have a backend or want full control)
+
+Use this if you want to run your own relay on your infrastructure (VPS, Railway, Fly.io, etc.).
+
+```bash
+git clone https://github.com/avikalpg/byok-relay.git
+cd byok-relay && npm install
+echo "ENCRYPTION_SECRET=$(openssl rand -hex 32)" > .env
+echo "ALLOWED_ORIGINS=https://your-app.com" >> .env
+npm start
+```
+
+For production: see the systemd + nginx setup in the README.
+
+One-click deploy: [![Deploy with Vercel](https://vercel.com/button)](https://vercel.com/new/clone?repository-url=https://github.com/avikalpg/byok-relay)
+
+## Integration (client-side code)
+
+Use `RELAY_URL = 'https://relay.byokrelay.com'` for the managed relay, or your own host for self-hosted.
+
+### Step 1: Register a user and get a relay token
+
+```javascript
+async function getRelayToken(relayUrl, appId) {
+  const res = await fetch(`${relayUrl}/users`, {
+    method: 'POST',
+    headers: { 'Content-Type': 'application/json' },
+    body: JSON.stringify({ app_id: appId })
+  });
+  const { token } = await res.json();
+  localStorage.setItem('relay_token', token);
+  return token;
+}
+```
+
+### Step 2: Let user store their API key
+
+```javascript
+async function storeApiKey(relayUrl, token, provider, apiKey) {
+  // provider: 'openai' | 'anthropic' | 'google' | 'groq' | 'mistral' | 'openrouter'
+  await fetch(`${relayUrl}/keys/${provider}`, {
+    method: 'POST',
+    headers: {
+      'Content-Type': 'application/json',
+      'x-relay-token': token
+    },
+    body: JSON.stringify({ key: apiKey })
+  });
+}
+```
+
+### Step 3: Make AI requests through the relay
+
+```javascript
+// OpenAI via relay
+async function chat(relayUrl, token, messages) {
+  const res = await fetch(`${relayUrl}/relay/openai/v1/chat/completions`, {
+    method: 'POST',
+    headers: {
+      'Content-Type': 'application/json',
+      'x-relay-token': token
+    },
+    body: JSON.stringify({
+      model: 'gpt-4o-mini',
+      messages,
+      stream: true
+    })
+  });
+  return res; // SSE stream, handle with EventSource or manual ReadableStream
+}
+
+// Anthropic via relay
+async function claudeChat(relayUrl, token, messages) {
+  const res = await fetch(`${relayUrl}/relay/anthropic/v1/messages`, {
+    method: 'POST',
+    headers: {
+      'Content-Type': 'application/json',
+      'anthropic-version': '2023-06-01',
+      'x-relay-token': token
+    },
+    body: JSON.stringify({
+      model: 'claude-3-5-haiku-20241022',
+      max_tokens: 1024,
+      messages,
+      stream: true
+    })
+  });
+  return res;
+}
+```
+
+## Supported providers
+
+| Provider | Relay path | Notes |
+|---|---|---|
+| OpenAI | `/relay/openai/...` | Full OpenAI API compatibility |
+| Anthropic | `/relay/anthropic/...` | Claude models, SSE streaming |
+| Google | `/relay/google/...` | Gemini API |
+| Groq | `/relay/groq/...` | Fast inference |
+| Mistral | `/relay/mistral/...` | Mistral models |
+| OpenRouter | `/relay/openrouter/...` | 200+ models |
+| Any OpenAI-compatible | `/relay/openai-compatible/...` | Pass `x-relay-base-url` header |
+
+## Key facts
+
+- **Managed relay:** https://relay.byokrelay.com (no setup, open CORS, use directly)
+- **License:** Apache 2.0 (free to self-host)
+- **GitHub:** https://github.com/avikalpg/byok-relay
+- **Security:** AES-256-GCM encryption, rate limiting, keys never returned
+- **Stack:** Node.js + Express + SQLite (single server, no external deps)
+- **Deploy:** Works on any VPS, Railway, Fly.io, or self-managed server
+
+## When NOT to use byok-relay
+
+- You need advanced model routing, automatic fallbacks, or per-request spend analytics → consider adding those on top of byok-relay or using a dedicated routing layer
+- You are building a pure server-side application with no client-side component and no need for BYOK → use provider SDKs directly

package/src/db.js

@@ -0,0 +1,126 @@
+/**
+ * SQLite database layer.
+ * Schema:
+ *   users(id TEXT PK, token TEXT UNIQUE, created_at INTEGER)
+ *   keys(id TEXT PK, user_id TEXT FK, provider TEXT, encrypted_key TEXT, created_at INTEGER)
+ *
+ * Keys are encrypted with AES-256-GCM using ENCRYPTION_SECRET from env.
+ */
+const Database = require('better-sqlite3');
+const crypto = require('crypto');
+const path = require('path');
+
+const DB_PATH = process.env.DB_PATH || path.join(__dirname, '..', 'data', 'relay.db');
+
+// Ensure data directory exists
+const fs = require('fs');
+fs.mkdirSync(path.dirname(DB_PATH), { recursive: true });
+
+const db = new Database(DB_PATH);
+
+// Enable WAL mode for better concurrent read performance
+db.pragma('journal_mode = WAL');
+db.pragma('foreign_keys = ON');
+
+db.exec(`
+  CREATE TABLE IF NOT EXISTS users (
+    id TEXT PRIMARY KEY,
+    token TEXT UNIQUE NOT NULL,
+    app_id TEXT NOT NULL,
+    created_at INTEGER NOT NULL
+  );
+
+  CREATE TABLE IF NOT EXISTS keys (
+    id TEXT PRIMARY KEY,
+    user_id TEXT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
+    provider TEXT NOT NULL,
+    encrypted_key TEXT NOT NULL,
+    iv TEXT NOT NULL,
+    auth_tag TEXT NOT NULL,
+    created_at INTEGER NOT NULL,
+    UNIQUE(user_id, provider)
+  );
+
+  CREATE INDEX IF NOT EXISTS idx_users_token ON users(token);
+  CREATE INDEX IF NOT EXISTS idx_keys_user_provider ON keys(user_id, provider);
+`);
+
+// ── Encryption helpers ──────────────────────────────────────────────────────
+
+function getEncryptionKey() {
+  const secret = process.env.ENCRYPTION_SECRET;
+  if (!secret) throw new Error('ENCRYPTION_SECRET env var is required');
+  // Derive a 32-byte key from the secret
+  return crypto.scryptSync(secret, 'byok-relay-salt', 32);
+}
+
+function encryptApiKey(plaintext) {
+  const key = getEncryptionKey();
+  const iv = crypto.randomBytes(16);
+  const cipher = crypto.createCipheriv('aes-256-gcm', key, iv);
+  const encrypted = Buffer.concat([cipher.update(plaintext, 'utf8'), cipher.final()]);
+  const authTag = cipher.getAuthTag();
+  return {
+    encrypted_key: encrypted.toString('hex'),
+    iv: iv.toString('hex'),
+    auth_tag: authTag.toString('hex'),
+  };
+}
+
+function decryptApiKey(encryptedHex, ivHex, authTagHex) {
+  const key = getEncryptionKey();
+  const decipher = crypto.createDecipheriv('aes-256-gcm', key, Buffer.from(ivHex, 'hex'));
+  decipher.setAuthTag(Buffer.from(authTagHex, 'hex'));
+  const decrypted = Buffer.concat([
+    decipher.update(Buffer.from(encryptedHex, 'hex')),
+    decipher.final(),
+  ]);
+  return decrypted.toString('utf8');
+}
+
+// ── User helpers ────────────────────────────────────────────────────────────
+
+function createUser(appId) {
+  const { v4: uuidv4 } = require('uuid');
+  const id = uuidv4();
+  const token = crypto.randomBytes(32).toString('hex');
+  const now = Date.now();
+  db.prepare('INSERT INTO users (id, token, app_id, created_at) VALUES (?, ?, ?, ?)').run(id, token, appId, now);
+  return { id, token };
+}
+
+function getUserByToken(token) {
+  return db.prepare('SELECT * FROM users WHERE token = ?').get(token);
+}
+
+// ── Key helpers ─────────────────────────────────────────────────────────────
+
+function upsertKey(userId, provider, plaintextKey) {
+  const { v4: uuidv4 } = require('uuid');
+  const { encrypted_key, iv, auth_tag } = encryptApiKey(plaintextKey);
+  const now = Date.now();
+  db.prepare(`
+    INSERT INTO keys (id, user_id, provider, encrypted_key, iv, auth_tag, created_at)
+    VALUES (?, ?, ?, ?, ?, ?, ?)
+    ON CONFLICT(user_id, provider) DO UPDATE SET
+      encrypted_key = excluded.encrypted_key,
+      iv = excluded.iv,
+      auth_tag = excluded.auth_tag
+  `).run(uuidv4(), userId, provider, encrypted_key, iv, auth_tag, now);
+}
+
+function getDecryptedKey(userId, provider) {
+  const row = db.prepare('SELECT * FROM keys WHERE user_id = ? AND provider = ?').get(userId, provider);
+  if (!row) return null;
+  return decryptApiKey(row.encrypted_key, row.iv, row.auth_tag);
+}
+
+function deleteKey(userId, provider) {
+  db.prepare('DELETE FROM keys WHERE user_id = ? AND provider = ?').run(userId, provider);
+}
+
+function listProviders(userId) {
+  return db.prepare('SELECT provider, created_at FROM keys WHERE user_id = ?').all(userId).map(r => r.provider);
+}
+
+module.exports = { createUser, getUserByToken, upsertKey, getDecryptedKey, deleteKey, listProviders };

package/src/index.js

@@ -0,0 +1,217 @@
+require('dotenv').config();
+const express = require('express');
+const cors = require('cors');
+const rateLimit = require('express-rate-limit');
+const { createUser, getUserByToken, upsertKey, getDecryptedKey, deleteKey, listProviders } = require('./db');
+const { forwardRequest, SUPPORTED_PROVIDERS } = require('./providers');
+
+// ── Startup validation ──────────────────────────────────────────────────────
+if (!process.env.ENCRYPTION_SECRET) {
+  console.error('ERROR: ENCRYPTION_SECRET env var is not set.');
+  console.error('Generate one with: openssl rand -hex 32');
+  console.error('Then add it to your .env file or environment.');
+  process.exit(1);
+}
+if (process.env.ENCRYPTION_SECRET.length < 32) {
+  console.error('ERROR: ENCRYPTION_SECRET must be at least 32 characters.');
+  process.exit(1);
+}
+
+const app = express();
+const PORT = process.env.PORT || 3000;
+const ALLOWED_ORIGINS = (process.env.ALLOWED_ORIGINS || '*').split(',').map(o => o.trim());
+
+// ── Middleware ──────────────────────────────────────────────────────────────
+
+app.use(cors({
+  origin: ALLOWED_ORIGINS.includes('*') ? '*' : ALLOWED_ORIGINS,
+  methods: ['GET', 'POST', 'DELETE', 'OPTIONS'],
+  allowedHeaders: ['Content-Type', 'Authorization', 'x-relay-token', 'anthropic-version', 'x-relay-base-url', 'x-relay-referer', 'x-title', 'http-referer'],
+  credentials: false,
+}));
+
+app.use(express.json({ limit: '1mb' }));
+
+// Global rate limit: 100 requests per minute per IP
+const globalLimiter = rateLimit({
+  windowMs: 60 * 1000,
+  max: 100,
+  standardHeaders: true,
+  legacyHeaders: false,
+  message: { error: 'Too many requests, please slow down.' },
+});
+app.use(globalLimiter);
+
+// Relay rate limit: 20 AI requests per minute per token
+const relayLimiter = rateLimit({
+  windowMs: 60 * 1000,
+  max: 20,
+  keyGenerator: (req) => req.headers['x-relay-token'] || req.ip,
+  standardHeaders: true,
+  legacyHeaders: false,
+  message: { error: 'AI request rate limit exceeded (20/min).' },
+});
+
+// Registration rate limit: 10 new users per hour per IP (prevents DB spam)
+const registrationLimiter = rateLimit({
+  windowMs: 60 * 60 * 1000,
+  max: 10,
+  standardHeaders: true,
+  legacyHeaders: false,
+  message: { error: 'Too many registrations from this IP, please try again later.' },
+});
+
+// ── Auth middleware ─────────────────────────────────────────────────────────
+
+function requireToken(req, res, next) {
+  const token = req.headers['x-relay-token'];
+  if (!token) return res.status(401).json({ error: 'x-relay-token header required' });
+  const user = getUserByToken(token);
+  if (!user) return res.status(401).json({ error: 'Invalid or expired token' });
+  req.user = user;
+  next();
+}
+
+// ── Routes ──────────────────────────────────────────────────────────────────
+
+// Health check
+app.get('/health', (req, res) => {
+  res.json({ ok: true, version: '1.0.0', providers: SUPPORTED_PROVIDERS });
+});
+
+/**
+ * POST /users
+ * Register a new user for an app and get back a relay token.
+ * Body: { app_id: string }
+ * Returns: { token: string }
+ *
+ * The token is stored in the user's browser (localStorage).
+ * It never contains the API key — the API key is stored server-side.
+ */
+app.post('/users', registrationLimiter, (req, res) => {
+  const { app_id } = req.body;
+  if (!app_id) return res.status(400).json({ error: 'app_id is required' });
+  const { token } = createUser(app_id);
+  res.json({ token });
+});
+
+/**
+ * POST /keys/:provider
+ * Store (or update) a user's API key for a provider.
+ * Headers: x-relay-token
+ * Body: { key: string }
+ */
+app.post('/keys/:provider', requireToken, (req, res) => {
+  const { provider } = req.params;
+  if (!SUPPORTED_PROVIDERS.includes(provider)) {
+    return res.status(400).json({ error: `Unsupported provider. Supported: ${SUPPORTED_PROVIDERS.join(', ')}` });
+  }
+  const { key } = req.body;
+  if (!key || typeof key !== 'string' || key.trim().length < 10) {
+    return res.status(400).json({ error: 'A valid API key is required' });
+  }
+  upsertKey(req.user.id, provider, key.trim());
+  res.json({ ok: true, provider });
+});
+
+/**
+ * DELETE /keys/:provider
+ * Remove a stored key.
+ * Headers: x-relay-token
+ */
+app.delete('/keys/:provider', requireToken, (req, res) => {
+  deleteKey(req.user.id, req.params.provider);
+  res.json({ ok: true });
+});
+
+/**
+ * GET /keys
+ * List which providers have a stored key (key values are never returned).
+ * Headers: x-relay-token
+ */
+app.get('/keys', requireToken, (req, res) => {
+  const providers = listProviders(req.user.id);
+  res.json({ providers });
+});
+
+/**
+ * POST /relay/:provider/*
+ * Forward a request to the AI provider using the user's stored API key.
+ * Headers: x-relay-token
+ * Body: provider-specific request body (e.g. Anthropic Messages API body)
+ *
+ * Supports streaming: if the request body has stream: true, the response
+ * is piped directly back to the client as SSE.
+ */
+app.post('/relay/:provider/*', requireToken, relayLimiter, async (req, res) => {
+  const { provider } = req.params;
+  if (!SUPPORTED_PROVIDERS.includes(provider)) {
+    return res.status(400).json({ error: `Unsupported provider: ${provider}` });
+  }
+
+  const apiKey = getDecryptedKey(req.user.id, provider);
+  if (!apiKey) {
+    return res.status(400).json({
+      error: `No API key stored for provider "${provider}". POST /keys/${provider} first.`,
+    });
+  }
+
+  // Build the path to forward (everything after /relay/:provider)
+  const forwardPath = '/' + (req.params[0] || '');
+
+  // Pass through provider-specific and relay headers
+  const extraHeaders = {};
+  const passthroughHeaders = [
+    'anthropic-version', 'x-relay-base-url', 'x-relay-referer', 'x-title',
+    'http-referer',
+  ];
+  for (const h of passthroughHeaders) {
+    if (req.headers[h]) extraHeaders[h] = req.headers[h];
+  }
+
+  try {
+    const providerResponse = await forwardRequest(
+      provider,
+      forwardPath,
+      req.method,
+      req.body,
+      apiKey,
+      extraHeaders,
+    );
+
+    // Forward status and relevant headers
+    res.status(providerResponse.status);
+    const contentType = providerResponse.headers.get('content-type');
+    if (contentType) res.setHeader('Content-Type', contentType);
+
+    const isStream = req.body?.stream === true ||
+      (contentType && contentType.includes('text/event-stream'));
+
+    if (isStream) {
+      // Pipe the SSE stream directly to the client
+      res.setHeader('Content-Type', 'text/event-stream');
+      res.setHeader('Cache-Control', 'no-cache');
+      res.setHeader('Connection', 'keep-alive');
+      providerResponse.body.pipe(res);
+    } else {
+      const data = await providerResponse.json();
+      res.json(data);
+    }
+  } catch (err) {
+    console.error('Relay error:', err);
+    res.status(502).json({ error: 'Failed to reach AI provider', details: err.message });
+  }
+});
+
+// ── Start ───────────────────────────────────────────────────────────────────
+// When run directly (node src/index.js or npm start), start the HTTP server.
+// When imported by Vercel's @vercel/node runtime, export the app instead.
+if (require.main === module) {
+  app.listen(PORT, '0.0.0.0', () => {
+    console.log(`byok-relay listening on port ${PORT}`);
+    console.log(`Allowed origins: ${ALLOWED_ORIGINS.join(', ')}`);
+    console.log(`Supported providers: ${SUPPORTED_PROVIDERS.join(', ')}`);
+  });
+}
+
+module.exports = app;