freellm 1.0.1 → 1.0.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +89 -89
- package/bin/cli.js +25 -25
- package/client-dist/assets/index-Danc4PW1.js +96 -0
- package/client-dist/index.html +13 -13
- package/dist/db/index.d.ts +1 -0
- package/dist/db/index.js +63 -59
- package/dist/index.js +4 -4
- package/dist/routes/api/keys.js +9 -0
- package/dist/routes/api/models.js +10 -3
- package/dist/routes/v1/models.js +8 -3
- package/package.json +57 -57
- package/client-dist/assets/index-aUFet1Mp.js +0 -84
package/README.md
CHANGED
|
@@ -1,89 +1,89 @@
|
|
|
1
|
-
# FreeLLM
|
|
2
|
-
|
|
3
|
-
**One OpenAI-compatible endpoint. 15+ free LLM providers. No billing.**
|
|
4
|
-
|
|
5
|
-
Aggregate the free tiers from Google, Groq, Cerebras, Mistral, OpenRouter, GitHub Models, Cohere, Cloudflare Workers AI, HuggingFace, Ollama, Pollinations, LLM7, OVH AI Endpoints — behind a single `/v1` API. Smart routing, automatic failover, encrypted key storage, web dashboard. All features free.
|
|
6
|
-
|
|
7
|
-
## Quick start
|
|
8
|
-
|
|
9
|
-
```bash
|
|
10
|
-
npx freellm
|
|
11
|
-
```
|
|
12
|
-
|
|
13
|
-
That's it. Opens a dashboard at `http://localhost:3000`.
|
|
14
|
-
|
|
15
|
-
## What it does
|
|
16
|
-
|
|
17
|
-
1. **Add your free API keys** on the Keys page (Google, Groq, etc.)
|
|
18
|
-
2. **Copy your unified key** from the dashboard
|
|
19
|
-
3. **Point any OpenAI client** at `http://localhost:3000/v1`:
|
|
20
|
-
|
|
21
|
-
```python
|
|
22
|
-
from openai import OpenAI
|
|
23
|
-
|
|
24
|
-
client = OpenAI(
|
|
25
|
-
base_url="http://localhost:3000/v1",
|
|
26
|
-
api_key="freellm-your-unified-key",
|
|
27
|
-
)
|
|
28
|
-
|
|
29
|
-
resp = client.chat.completions.create(
|
|
30
|
-
model="auto", # smart routing picks the best available provider
|
|
31
|
-
messages=[{"role": "user", "content": "Hello!"}],
|
|
32
|
-
)
|
|
33
|
-
print(resp.choices[0].message.content)
|
|
34
|
-
print("Routed via:", resp.headers.get("x-routed-via"))
|
|
35
|
-
```
|
|
36
|
-
|
|
37
|
-
```bash
|
|
38
|
-
curl http://localhost:3000/v1/chat/completions \
|
|
39
|
-
-H "Authorization: Bearer freellm-your-unified-key" \
|
|
40
|
-
-H "Content-Type: application/json" \
|
|
41
|
-
-d '{"model": "auto", "messages": [{"role": "user", "content": "hi"}]}'
|
|
42
|
-
```
|
|
43
|
-
|
|
44
|
-
## Supported providers (all free tiers)
|
|
45
|
-
|
|
46
|
-
| Provider | Free models | Key needed |
|
|
47
|
-
|---|---|---|
|
|
48
|
-
| Google Gemini | Gemini 2.5 Flash, 2.0 Flash, 1.5 Flash | Yes (aistudio.google.com) |
|
|
49
|
-
| Groq | Llama 3.3 70B, 3.1 8B, Gemma 2 9B | Yes (console.groq.com) |
|
|
50
|
-
| Cerebras | Llama 3.3 70B, 3.1 8B | Yes (cloud.cerebras.ai) |
|
|
51
|
-
| Mistral | Mistral Small, Nemo, Codestral | Yes (console.mistral.ai) |
|
|
52
|
-
| OpenRouter | 20+ free models (:free suffix) | Yes (openrouter.ai) |
|
|
53
|
-
| GitHub Models | GPT-4o, GPT-4o Mini | Yes (github.com) |
|
|
54
|
-
| Cohere | Command R+, Command R | Yes (dashboard.cohere.com) |
|
|
55
|
-
| Cloudflare | Llama 3.1 8B, Gemma 7B | Yes (ACCOUNT_ID:TOKEN) |
|
|
56
|
-
| HuggingFace | Llama 3 8B, Mistral 7B | Yes (huggingface.co) |
|
|
57
|
-
| Ollama | Any local model | No |
|
|
58
|
-
| Pollinations | GPT-OSS, Mistral | No |
|
|
59
|
-
| LLM7 | GPT-4o Mini, Llama 70B | No |
|
|
60
|
-
| OVH AI Endpoints | Llama 3.3 70B, Mistral 7B | No |
|
|
61
|
-
|
|
62
|
-
## Features
|
|
63
|
-
|
|
64
|
-
- **OpenAI-compatible** — `/v1/chat/completions`, `/v1/models`, streaming, tool calling, vision
|
|
65
|
-
- **Smart routing** — picks highest-priority healthy model for each request
|
|
66
|
-
- **Automatic failover** — on 429/5xx, skips to next provider in chain (up to 20 attempts)
|
|
67
|
-
- **Rate limit tracking** — in-memory per-provider counters, no key is sent over its limits
|
|
68
|
-
- **Encrypted storage** — API keys are AES-256-GCM encrypted at rest in SQLite
|
|
69
|
-
- **Web dashboard** — manage keys, reorder fallback chain, playground, analytics
|
|
70
|
-
- **No billing** — all features free, forever
|
|
71
|
-
|
|
72
|
-
## Options
|
|
73
|
-
|
|
74
|
-
```bash
|
|
75
|
-
npx freellm --port 8080 # custom port (default: 3000)
|
|
76
|
-
npx freellm --no-open # don't open browser automatically
|
|
77
|
-
```
|
|
78
|
-
|
|
79
|
-
## Data storage
|
|
80
|
-
|
|
81
|
-
All data (database + encryption key) is stored in `~/.freellm/`.
|
|
82
|
-
|
|
83
|
-
## Requirements
|
|
84
|
-
|
|
85
|
-
- Node.js 22.5+ (uses built-in `node:sqlite`)
|
|
86
|
-
|
|
87
|
-
## License
|
|
88
|
-
|
|
89
|
-
MIT
|
|
1
|
+
# FreeLLM
|
|
2
|
+
|
|
3
|
+
**One OpenAI-compatible endpoint. 15+ free LLM providers. No billing.**
|
|
4
|
+
|
|
5
|
+
Aggregate the free tiers from Google, Groq, Cerebras, Mistral, OpenRouter, GitHub Models, Cohere, Cloudflare Workers AI, HuggingFace, Ollama, Pollinations, LLM7, OVH AI Endpoints — behind a single `/v1` API. Smart routing, automatic failover, encrypted key storage, web dashboard. All features free.
|
|
6
|
+
|
|
7
|
+
## Quick start
|
|
8
|
+
|
|
9
|
+
```bash
|
|
10
|
+
npx freellm
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
That's it. Opens a dashboard at `http://localhost:3000`.
|
|
14
|
+
|
|
15
|
+
## What it does
|
|
16
|
+
|
|
17
|
+
1. **Add your free API keys** on the Keys page (Google, Groq, etc.)
|
|
18
|
+
2. **Copy your unified key** from the dashboard
|
|
19
|
+
3. **Point any OpenAI client** at `http://localhost:3000/v1`:
|
|
20
|
+
|
|
21
|
+
```python
|
|
22
|
+
from openai import OpenAI
|
|
23
|
+
|
|
24
|
+
client = OpenAI(
|
|
25
|
+
base_url="http://localhost:3000/v1",
|
|
26
|
+
api_key="freellm-your-unified-key",
|
|
27
|
+
)
|
|
28
|
+
|
|
29
|
+
resp = client.chat.completions.create(
|
|
30
|
+
model="auto", # smart routing picks the best available provider
|
|
31
|
+
messages=[{"role": "user", "content": "Hello!"}],
|
|
32
|
+
)
|
|
33
|
+
print(resp.choices[0].message.content)
|
|
34
|
+
print("Routed via:", resp.headers.get("x-routed-via"))
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
```bash
|
|
38
|
+
curl http://localhost:3000/v1/chat/completions \
|
|
39
|
+
-H "Authorization: Bearer freellm-your-unified-key" \
|
|
40
|
+
-H "Content-Type: application/json" \
|
|
41
|
+
-d '{"model": "auto", "messages": [{"role": "user", "content": "hi"}]}'
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
## Supported providers (all free tiers)
|
|
45
|
+
|
|
46
|
+
| Provider | Free models | Key needed |
|
|
47
|
+
|---|---|---|
|
|
48
|
+
| Google Gemini | Gemini 2.5 Flash, 2.0 Flash, 1.5 Flash | Yes (aistudio.google.com) |
|
|
49
|
+
| Groq | Llama 3.3 70B, 3.1 8B, Gemma 2 9B | Yes (console.groq.com) |
|
|
50
|
+
| Cerebras | Llama 3.3 70B, 3.1 8B | Yes (cloud.cerebras.ai) |
|
|
51
|
+
| Mistral | Mistral Small, Nemo, Codestral | Yes (console.mistral.ai) |
|
|
52
|
+
| OpenRouter | 20+ free models (:free suffix) | Yes (openrouter.ai) |
|
|
53
|
+
| GitHub Models | GPT-4o, GPT-4o Mini | Yes (github.com) |
|
|
54
|
+
| Cohere | Command R+, Command R | Yes (dashboard.cohere.com) |
|
|
55
|
+
| Cloudflare | Llama 3.1 8B, Gemma 7B | Yes (ACCOUNT_ID:TOKEN) |
|
|
56
|
+
| HuggingFace | Llama 3 8B, Mistral 7B | Yes (huggingface.co) |
|
|
57
|
+
| Ollama | Any local model | No |
|
|
58
|
+
| Pollinations | GPT-OSS, Mistral | No |
|
|
59
|
+
| LLM7 | GPT-4o Mini, Llama 70B | No |
|
|
60
|
+
| OVH AI Endpoints | Llama 3.3 70B, Mistral 7B | No |
|
|
61
|
+
|
|
62
|
+
## Features
|
|
63
|
+
|
|
64
|
+
- **OpenAI-compatible** — `/v1/chat/completions`, `/v1/models`, streaming, tool calling, vision
|
|
65
|
+
- **Smart routing** — picks highest-priority healthy model for each request
|
|
66
|
+
- **Automatic failover** — on 429/5xx, skips to next provider in chain (up to 20 attempts)
|
|
67
|
+
- **Rate limit tracking** — in-memory per-provider counters, no key is sent over its limits
|
|
68
|
+
- **Encrypted storage** — API keys are AES-256-GCM encrypted at rest in SQLite
|
|
69
|
+
- **Web dashboard** — manage keys, reorder fallback chain, playground, analytics
|
|
70
|
+
- **No billing** — all features free, forever
|
|
71
|
+
|
|
72
|
+
## Options
|
|
73
|
+
|
|
74
|
+
```bash
|
|
75
|
+
npx freellm --port 8080 # custom port (default: 3000)
|
|
76
|
+
npx freellm --no-open # don't open browser automatically
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
## Data storage
|
|
80
|
+
|
|
81
|
+
All data (database + encryption key) is stored in `~/.freellm/`.
|
|
82
|
+
|
|
83
|
+
## Requirements
|
|
84
|
+
|
|
85
|
+
- Node.js 22.5+ (uses built-in `node:sqlite`)
|
|
86
|
+
|
|
87
|
+
## License
|
|
88
|
+
|
|
89
|
+
MIT
|
package/bin/cli.js
CHANGED
|
@@ -1,25 +1,25 @@
|
|
|
1
|
-
#!/usr/bin/env node
|
|
2
|
-
'use strict'
|
|
3
|
-
|
|
4
|
-
// Redirect to the compiled CLI if dist/ exists, else run with tsx
|
|
5
|
-
const path = require('path')
|
|
6
|
-
const fs = require('fs')
|
|
7
|
-
|
|
8
|
-
const distCli = path.join(__dirname, '..', 'dist', 'cli.js')
|
|
9
|
-
const srcCli = path.join(__dirname, '..', 'src', 'cli.ts')
|
|
10
|
-
|
|
11
|
-
if (fs.existsSync(distCli)) {
|
|
12
|
-
require(distCli)
|
|
13
|
-
} else if (fs.existsSync(srcCli)) {
|
|
14
|
-
// Dev mode — use tsx
|
|
15
|
-
const { execFileSync } = require('child_process')
|
|
16
|
-
const tsx = path.join(__dirname, '..', 'node_modules', '.bin', 'tsx')
|
|
17
|
-
try {
|
|
18
|
-
execFileSync(tsx, [srcCli, ...process.argv.slice(2)], { stdio: 'inherit' })
|
|
19
|
-
} catch (e) {
|
|
20
|
-
process.exit(e.status || 1)
|
|
21
|
-
}
|
|
22
|
-
} else {
|
|
23
|
-
console.error('UniLLM: Could not find compiled CLI. Run `npm run build` first.')
|
|
24
|
-
process.exit(1)
|
|
25
|
-
}
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
'use strict'
|
|
3
|
+
|
|
4
|
+
// Redirect to the compiled CLI if dist/ exists, else run with tsx
|
|
5
|
+
const path = require('path')
|
|
6
|
+
const fs = require('fs')
|
|
7
|
+
|
|
8
|
+
const distCli = path.join(__dirname, '..', 'dist', 'cli.js')
|
|
9
|
+
const srcCli = path.join(__dirname, '..', 'src', 'cli.ts')
|
|
10
|
+
|
|
11
|
+
if (fs.existsSync(distCli)) {
|
|
12
|
+
require(distCli)
|
|
13
|
+
} else if (fs.existsSync(srcCli)) {
|
|
14
|
+
// Dev mode — use tsx
|
|
15
|
+
const { execFileSync } = require('child_process')
|
|
16
|
+
const tsx = path.join(__dirname, '..', 'node_modules', '.bin', 'tsx')
|
|
17
|
+
try {
|
|
18
|
+
execFileSync(tsx, [srcCli, ...process.argv.slice(2)], { stdio: 'inherit' })
|
|
19
|
+
} catch (e) {
|
|
20
|
+
process.exit(e.status || 1)
|
|
21
|
+
}
|
|
22
|
+
} else {
|
|
23
|
+
console.error('UniLLM: Could not find compiled CLI. Run `npm run build` first.')
|
|
24
|
+
process.exit(1)
|
|
25
|
+
}
|