freellm 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (55) hide show
  1. package/README.md +89 -0
  2. package/bin/cli.js +25 -0
  3. package/client-dist/assets/index-aUFet1Mp.js +84 -0
  4. package/client-dist/index.html +16 -0
  5. package/dist/cli.d.ts +1 -0
  6. package/dist/cli.js +38 -0
  7. package/dist/crypto.d.ts +6 -0
  8. package/dist/crypto.js +56 -0
  9. package/dist/db/index.d.ts +62 -0
  10. package/dist/db/index.js +368 -0
  11. package/dist/index.d.ts +10 -0
  12. package/dist/index.js +100 -0
  13. package/dist/providers/cloudflare.d.ts +10 -0
  14. package/dist/providers/cloudflare.js +140 -0
  15. package/dist/providers/google.d.ts +8 -0
  16. package/dist/providers/google.js +181 -0
  17. package/dist/providers/index.d.ts +14 -0
  18. package/dist/providers/index.js +60 -0
  19. package/dist/providers/ollama.d.ts +10 -0
  20. package/dist/providers/ollama.js +99 -0
  21. package/dist/providers/openai-compat.d.ts +11 -0
  22. package/dist/providers/openai-compat.js +115 -0
  23. package/dist/providers/openai-image-compat.d.ts +10 -0
  24. package/dist/providers/openai-image-compat.js +40 -0
  25. package/dist/providers/openai-tts-compat.d.ts +10 -0
  26. package/dist/providers/openai-tts-compat.js +40 -0
  27. package/dist/providers/pollinations-image.d.ts +7 -0
  28. package/dist/providers/pollinations-image.js +50 -0
  29. package/dist/providers/pollinations.d.ts +9 -0
  30. package/dist/providers/pollinations.js +76 -0
  31. package/dist/providers/types.d.ts +132 -0
  32. package/dist/providers/types.js +66 -0
  33. package/dist/routes/api/analytics.d.ts +1 -0
  34. package/dist/routes/api/analytics.js +10 -0
  35. package/dist/routes/api/auth.d.ts +3 -0
  36. package/dist/routes/api/auth.js +60 -0
  37. package/dist/routes/api/keys.d.ts +1 -0
  38. package/dist/routes/api/keys.js +65 -0
  39. package/dist/routes/api/models.d.ts +1 -0
  40. package/dist/routes/api/models.js +26 -0
  41. package/dist/routes/v1/audio.d.ts +1 -0
  42. package/dist/routes/v1/audio.js +94 -0
  43. package/dist/routes/v1/chat.d.ts +1 -0
  44. package/dist/routes/v1/chat.js +131 -0
  45. package/dist/routes/v1/images.d.ts +1 -0
  46. package/dist/routes/v1/images.js +85 -0
  47. package/dist/routes/v1/models.d.ts +1 -0
  48. package/dist/routes/v1/models.js +59 -0
  49. package/dist/services/health.d.ts +2 -0
  50. package/dist/services/health.js +80 -0
  51. package/dist/services/ratelimit.d.ts +4 -0
  52. package/dist/services/ratelimit.js +88 -0
  53. package/dist/services/router.d.ts +14 -0
  54. package/dist/services/router.js +72 -0
  55. package/package.json +57 -0
package/README.md ADDED
@@ -0,0 +1,89 @@
1
+ # UniLLM
2
+
3
+ **One OpenAI-compatible endpoint. 15+ free LLM providers. No billing.**
4
+
5
+ Aggregate the free tiers from Google, Groq, Cerebras, Mistral, OpenRouter, GitHub Models, Cohere, Cloudflare Workers AI, HuggingFace, Ollama, Pollinations, LLM7, OVH AI Endpoints — behind a single `/v1` API. Smart routing, automatic failover, encrypted key storage, web dashboard. All features free.
6
+
7
+ ## Quick start
8
+
9
+ ```bash
10
+ npx unillm
11
+ ```
12
+
13
+ That's it. Opens a dashboard at `http://localhost:3000`.
14
+
15
+ ## What it does
16
+
17
+ 1. **Add your free API keys** on the Keys page (Google, Groq, etc.)
18
+ 2. **Copy your unified key** from the dashboard
19
+ 3. **Point any OpenAI client** at `http://localhost:3000/v1`:
20
+
21
+ ```python
22
+ from openai import OpenAI
23
+
24
+ client = OpenAI(
25
+ base_url="http://localhost:3000/v1",
26
+ api_key="unillm-your-unified-key",
27
+ )
28
+
29
+ resp = client.chat.completions.create(
30
+ model="auto", # smart routing picks the best available provider
31
+ messages=[{"role": "user", "content": "Hello!"}],
32
+ )
33
+ print(resp.choices[0].message.content)
34
+ print("Routed via:", resp.headers.get("x-routed-via"))
35
+ ```
36
+
37
+ ```bash
38
+ curl http://localhost:3000/v1/chat/completions \
39
+ -H "Authorization: Bearer unillm-your-unified-key" \
40
+ -H "Content-Type: application/json" \
41
+ -d '{"model": "auto", "messages": [{"role": "user", "content": "hi"}]}'
42
+ ```
43
+
44
+ ## Supported providers (all free tiers)
45
+
46
+ | Provider | Free models | Key needed |
47
+ |---|---|---|
48
+ | Google Gemini | Gemini 2.5 Flash, 2.0 Flash, 1.5 Flash | Yes (aistudio.google.com) |
49
+ | Groq | Llama 3.3 70B, 3.1 8B, Gemma 2 9B | Yes (console.groq.com) |
50
+ | Cerebras | Llama 3.3 70B, 3.1 8B | Yes (cloud.cerebras.ai) |
51
+ | Mistral | Mistral Small, Nemo, Codestral | Yes (console.mistral.ai) |
52
+ | OpenRouter | 20+ free models (:free suffix) | Yes (openrouter.ai) |
53
+ | GitHub Models | GPT-4o, GPT-4o Mini | Yes (github.com) |
54
+ | Cohere | Command R+, Command R | Yes (dashboard.cohere.com) |
55
+ | Cloudflare | Llama 3.1 8B, Gemma 7B | Yes (ACCOUNT_ID:TOKEN) |
56
+ | HuggingFace | Llama 3 8B, Mistral 7B | Yes (huggingface.co) |
57
+ | Ollama | Any local model | No |
58
+ | Pollinations | GPT-OSS, Mistral | No |
59
+ | LLM7 | GPT-4o Mini, Llama 70B | No |
60
+ | OVH AI Endpoints | Llama 3.3 70B, Mistral 7B | No |
61
+
62
+ ## Features
63
+
64
+ - **OpenAI-compatible** — `/v1/chat/completions`, `/v1/models`, streaming, tool calling, vision
65
+ - **Smart routing** — picks highest-priority healthy model for each request
66
+ - **Automatic failover** — on 429/5xx, skips to next provider in chain (up to 20 attempts)
67
+ - **Rate limit tracking** — in-memory per-provider counters, no key is sent over its limits
68
+ - **Encrypted storage** — API keys are AES-256-GCM encrypted at rest in SQLite
69
+ - **Web dashboard** — manage keys, reorder fallback chain, playground, analytics
70
+ - **No billing** — all features free, forever
71
+
72
+ ## Options
73
+
74
+ ```bash
75
+ npx unillm --port 8080 # custom port (default: 3000)
76
+ npx unillm --no-open # don't open browser automatically
77
+ ```
78
+
79
+ ## Data storage
80
+
81
+ All data (database + encryption key) is stored in `~/.unillm/`.
82
+
83
+ ## Requirements
84
+
85
+ - Node.js 22.5+ (uses built-in `node:sqlite`)
86
+
87
+ ## License
88
+
89
+ MIT
package/bin/cli.js ADDED
@@ -0,0 +1,25 @@
1
+ #!/usr/bin/env node
2
+ 'use strict'
3
+
4
+ // Redirect to the compiled CLI if dist/ exists, else run with tsx
5
+ const path = require('path')
6
+ const fs = require('fs')
7
+
8
+ const distCli = path.join(__dirname, '..', 'dist', 'cli.js')
9
+ const srcCli = path.join(__dirname, '..', 'src', 'cli.ts')
10
+
11
+ if (fs.existsSync(distCli)) {
12
+ require(distCli)
13
+ } else if (fs.existsSync(srcCli)) {
14
+ // Dev mode — use tsx
15
+ const { execFileSync } = require('child_process')
16
+ const tsx = path.join(__dirname, '..', 'node_modules', '.bin', 'tsx')
17
+ try {
18
+ execFileSync(tsx, [srcCli, ...process.argv.slice(2)], { stdio: 'inherit' })
19
+ } catch (e) {
20
+ process.exit(e.status || 1)
21
+ }
22
+ } else {
23
+ console.error('UniLLM: Could not find compiled CLI. Run `npm run build` first.')
24
+ process.exit(1)
25
+ }