billion-context 0.1.0 → 0.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +218 -27
- package/dist/index.js +2186 -355
- package/dist/index.js.map +1 -1
- package/package.json +8 -3
package/README.md
CHANGED
|
@@ -36,64 +36,255 @@ The proxy injects four context-management tools (`compress`, `decompress`, `sear
|
|
|
36
36
|
npm install -g billion-context
|
|
37
37
|
```
|
|
38
38
|
|
|
39
|
+
This installs the `bili` command (`bili-proxy` is kept as an alias).
|
|
40
|
+
|
|
39
41
|
## Usage
|
|
40
42
|
|
|
41
|
-
Start the proxy
|
|
43
|
+
### Start the proxy
|
|
44
|
+
|
|
45
|
+
```bash
|
|
46
|
+
bili
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
That's it. The proxy reads its config from `~/.config/billion-context/billion-context.json` (XDG) and listens on `127.0.0.1:8787`. If no config file exists yet, it uses sensible defaults and logs where it expects the file.
|
|
50
|
+
|
|
51
|
+
### Quick overrides (flags)
|
|
42
52
|
|
|
43
53
|
```bash
|
|
44
|
-
bili
|
|
45
|
-
#
|
|
54
|
+
bili --port 9000 # change listen port
|
|
55
|
+
bili --host 0.0.0.0 # listen on all interfaces
|
|
56
|
+
bili --debug # verbose logging (also: set "debug": true in config)
|
|
57
|
+
bili --passthrough # forward without compression (smoke-test mode)
|
|
58
|
+
bili --config ~/my-bili.json # use a different config file
|
|
46
59
|
```
|
|
47
60
|
|
|
48
|
-
|
|
61
|
+
Flags override the config file and env vars. `bili --help` lists them all.
|
|
62
|
+
|
|
63
|
+
### Point your agent at the proxy
|
|
64
|
+
|
|
65
|
+
The proxy routes by a **provider name in the URL path**. Set your agent's base URL to `http://localhost:8787/<provider>/...` and the proxy forwards to that provider (see [Configuration](#configuration) for how providers are declared).
|
|
49
66
|
|
|
50
|
-
|
|
67
|
+
#### Claude Code (Anthropic)
|
|
51
68
|
|
|
52
69
|
```bash
|
|
53
|
-
export ANTHROPIC_BASE_URL=http://localhost:8787
|
|
54
|
-
export ANTHROPIC_API_KEY=sk-ant-...
|
|
70
|
+
export ANTHROPIC_BASE_URL=http://localhost:8787/anthropic
|
|
71
|
+
export ANTHROPIC_API_KEY=sk-ant-... # real key — passed through as-is
|
|
55
72
|
claude
|
|
56
73
|
```
|
|
57
74
|
|
|
58
|
-
|
|
75
|
+
#### Codex / any OpenAI-compatible agent (zhipu / openai / deepseek)
|
|
59
76
|
|
|
60
77
|
```bash
|
|
61
|
-
export OPENAI_BASE_URL=http://localhost:8787/
|
|
62
|
-
export OPENAI_API_KEY
|
|
78
|
+
export OPENAI_BASE_URL=http://localhost:8787/zhipu/api/coding/paas/v4
|
|
79
|
+
export OPENAI_API_KEY=<your real glm key> # passed through as-is
|
|
63
80
|
codex
|
|
64
81
|
```
|
|
65
82
|
|
|
66
|
-
|
|
83
|
+
The `/zhipu/...` prefix tells the proxy to route to the `zhipu` provider; the
|
|
84
|
+
remaining path is preserved.
|
|
67
85
|
|
|
68
|
-
|
|
86
|
+
#### Cursor / Aider / others
|
|
87
|
+
|
|
88
|
+
Set the base URL to `http://localhost:8787/<provider>` in the agent's settings.
|
|
89
|
+
|
|
90
|
+
### Debugging
|
|
91
|
+
|
|
92
|
+
Three ways to enable verbose logging (priority: flag > env > config):
|
|
93
|
+
|
|
94
|
+
1. **CLI flag** (quickest): `bili --debug`
|
|
95
|
+
2. **Env var**: `ACP_DEBUG=1 bili`
|
|
96
|
+
3. **Config file**: `"debug": true` in `billion-context.json`
|
|
97
|
+
|
|
98
|
+
Verbose mode logs every `processTurn` (tag counts, token usage), the nudge
|
|
99
|
+
decision (growth/usage/pendingT1/shouldInject), client headers, and SSE
|
|
100
|
+
rewrites.
|
|
69
101
|
|
|
70
102
|
## Configuration
|
|
71
103
|
|
|
72
|
-
|
|
104
|
+
Configuration is read from a JSON file with env-var overrides. Priority:
|
|
105
|
+
**env var > config file > built-in default**.
|
|
106
|
+
|
|
107
|
+
### Config file
|
|
108
|
+
|
|
109
|
+
Location (XDG Base Directory):
|
|
110
|
+
|
|
111
|
+
- **Linux:** `~/.config/billion-context/billion-context.json`
|
|
112
|
+
- Override with `XDG_CONFIG_HOME` or `BILI_CONFIG_FILE`
|
|
113
|
+
|
|
114
|
+
The config file is a single JSON object. Example:
|
|
115
|
+
|
|
116
|
+
```json
|
|
117
|
+
{
|
|
118
|
+
"port": 8787,
|
|
119
|
+
"host": "127.0.0.1",
|
|
120
|
+
"providers": {
|
|
121
|
+
"zhipu": {
|
|
122
|
+
"url": "https://open.bigmodel.cn",
|
|
123
|
+
"models": {
|
|
124
|
+
"glm-5.2": { "context": 1000000, "output": 131072 },
|
|
125
|
+
"glm-5.1": { "context": 200000, "output": 131072 }
|
|
126
|
+
}
|
|
127
|
+
},
|
|
128
|
+
"anthropic": "https://api.anthropic.com",
|
|
129
|
+
"deepseek": "https://api.deepseek.com"
|
|
130
|
+
}
|
|
131
|
+
}
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
### Top-level keys
|
|
135
|
+
|
|
136
|
+
| Key | Default | Description |
|
|
137
|
+
|------|---------|-------------|
|
|
138
|
+
| `port` | `8787` | Proxy listen port |
|
|
139
|
+
| `host` | `127.0.0.1` | Proxy listen host |
|
|
140
|
+
| `upstream` | `https://api.anthropic.com` | Default upstream when no route matches |
|
|
141
|
+
| `sessionHeader` | `x-acp-session` | Header name clients may send to identify a conversation |
|
|
142
|
+
| `log` | `true` | Enable request logging |
|
|
143
|
+
| `debug` | `false` | Verbose logging (same as `ACP_DEBUG=1`) |
|
|
144
|
+
| `passthrough` | `false` | Forward without compression (same as `ACP_PASSTHROUGH=1`) |
|
|
145
|
+
| `providers` | *(none)* | Provider routes — see below |
|
|
146
|
+
| `condense` | *(see defaults)* | Tool-result condensing: `{ enabled, keepRecentToolResults, minCharsToCondense, maxKeptChars }` |
|
|
147
|
+
| `compress` | *(see defaults)* | `{ injectTool, injectNudge }` |
|
|
148
|
+
|
|
149
|
+
### Providers (URL routing + per-model context)
|
|
150
|
+
|
|
151
|
+
`providers` maps a route name to either a bare URL string (simple) or an
|
|
152
|
+
object with `url` + optional per-model context window (recommended).
|
|
153
|
+
|
|
154
|
+
**Simple form** — provider name → URL:
|
|
155
|
+
```json
|
|
156
|
+
{ "deepseek": "https://api.deepseek.com" }
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
**Full form** — provider name → `{ url, models }`:
|
|
160
|
+
```json
|
|
161
|
+
{
|
|
162
|
+
"zhipu": {
|
|
163
|
+
"url": "https://open.bigmodel.cn",
|
|
164
|
+
"models": {
|
|
165
|
+
"glm-5.2": { "context": 1000000, "output": 131072 },
|
|
166
|
+
"glm-5.1": { "context": 200000 }
|
|
167
|
+
}
|
|
168
|
+
}
|
|
169
|
+
}
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
The same model can have a different context window behind different providers
|
|
173
|
+
(e.g. relay wraps a model with a larger window). `context` is the **input
|
|
174
|
+
context limit** (used by the compressor to decide when to nudge); `output` is
|
|
175
|
+
the max output tokens. Both are optional; missing values fall back to the
|
|
176
|
+
built-in model table, then to `modelContextLimit`.
|
|
73
177
|
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
| `MODEL_CONTEXT_LIMIT` | `200000` | Context window for compression triggering |
|
|
80
|
-
| `ACP_DEBUG` | `0` | Set `1` to enable debug logging |
|
|
81
|
-
| `ACP_LOG_FILE` | `stderr` | Log file path |
|
|
82
|
-
| `ACP_PASSTHROUGH` | `0` | Set `1` to forward without compression (for debugging) |
|
|
178
|
+
> **Why declare context at all?** The LLM `/models` API does **not** return
|
|
179
|
+
> context windows (verified across OpenAI, Anthropic, 智谱, comfly). They are
|
|
180
|
+
> document-level information. A wrong value (e.g. GLM-5.2 guessed as 128K
|
|
181
|
+
> instead of 1M) causes spurious frequent compression. Declaring it per
|
|
182
|
+
> provider + model makes the proxy match the registry the client itself uses.
|
|
83
183
|
|
|
84
|
-
|
|
184
|
+
**API keys are never stored in the proxy** — whatever key the agent sends is
|
|
185
|
+
passed through untouched to the upstream.
|
|
85
186
|
|
|
86
|
-
|
|
187
|
+
### Routing
|
|
188
|
+
|
|
189
|
+
Point any agent at the proxy using a provider name as a path segment. The
|
|
190
|
+
proxy strips the name and forwards to that provider's root URL.
|
|
191
|
+
|
|
192
|
+
```
|
|
193
|
+
agent baseURL: http://localhost:8787/zhipu/api/coding/paas/v4
|
|
194
|
+
└──────────┬──────────┘└────────┬────────┘
|
|
195
|
+
proxy host remaining path
|
|
196
|
+
+ provider name (forwarded as-is)
|
|
197
|
+
```
|
|
198
|
+
|
|
199
|
+
#### Claude Code (Anthropic)
|
|
87
200
|
|
|
88
201
|
```bash
|
|
89
|
-
|
|
202
|
+
export ANTHROPIC_BASE_URL=http://localhost:8787/anthropic
|
|
203
|
+
export ANTHROPIC_API_KEY=sk-ant-... # real key — passed through as-is
|
|
204
|
+
claude
|
|
205
|
+
```
|
|
206
|
+
|
|
207
|
+
#### Codex / any OpenAI-compatible agent (zhipu / openai / deepseek)
|
|
208
|
+
|
|
209
|
+
```bash
|
|
210
|
+
export OPENAI_BASE_URL=http://localhost:8787/zhipu/api/coding/paas/v4
|
|
211
|
+
export OPENAI_API_KEY=<your real glm key> # passed through as-is
|
|
212
|
+
codex
|
|
90
213
|
```
|
|
91
214
|
|
|
92
|
-
The
|
|
215
|
+
The `/zhipu/...` prefix tells the proxy to route to the `zhipu` provider; the
|
|
216
|
+
remaining `/api/coding/paas/v4/...` path is preserved.
|
|
217
|
+
|
|
218
|
+
### Environment variables (override the config file)
|
|
219
|
+
|
|
220
|
+
Every config key has an env-var override. Set to override the file value.
|
|
221
|
+
|
|
222
|
+
| Env | Default | Description |
|
|
223
|
+
|-----|---------|-------------|
|
|
224
|
+
| `ACP_PORT` / `PORT` | `8787` | Listen port |
|
|
225
|
+
| `ACP_HOST` | `127.0.0.1` | Listen host |
|
|
226
|
+
| `ACP_UPSTREAM` | `https://api.anthropic.com` | Default upstream |
|
|
227
|
+
| `ACP_PROVIDERS` | *(none)* | Path to a legacy providers JSON file (overrides `providers` in config) |
|
|
228
|
+
| `ACP_MODEL_CONTEXT_LIMIT` | `200000` | Global fallback context window (only used when no provider/model match) |
|
|
229
|
+
| `ACP_SESSION_HEADER` | `x-acp-session` | Conversation-id header name |
|
|
230
|
+
| `ACP_COMPRESS_TOOL` | `1` | Set `0` to disable injecting the compress tool |
|
|
231
|
+
| `ACP_COMPRESS_NUDGE` | `1` | Set `0` to disable compression nudges |
|
|
232
|
+
| `ACP_CONDENSE_ENABLED` | `1` | Set `0` to disable tool-result condensing |
|
|
233
|
+
| `ACP_KEEP_RECENT_TOOL_RESULTS` | `6` | Tool results kept verbatim before condensing |
|
|
234
|
+
| `ACP_MIN_CHARS_TO_CONDENSE` | `1500` | Condense tool results longer than this |
|
|
235
|
+
| `ACP_MAX_KEPT_CHARS` | `400` | Max chars kept when condensing a tool result |
|
|
236
|
+
| `ACP_DEBUG` | `0` | Set `1` for verbose logging |
|
|
237
|
+
| `ACP_PASSTHROUGH` | `0` | Set `1` to forward without compression |
|
|
238
|
+
| `ACP_DUMP_SSE` | *(none)* | Directory to dump SSE for debugging |
|
|
239
|
+
| `BILI_PERSIST` | `1` | Set `0` to disable session persistence (in-memory only, lost on restart) |
|
|
240
|
+
| `BILI_PERSIST_DEBOUNCE_MS` | `500` | Debounce window for writes to disk (ms) |
|
|
241
|
+
| `BILI_MAX_SESSIONS` | `256` | Max sessions held in memory (LRU eviction; disk is source of truth) |
|
|
242
|
+
| `BILI_SESSIONS_DIR` | *(XDG data dir)* | Directory for persisted session state |
|
|
243
|
+
|
|
244
|
+
### Notes on provider names
|
|
245
|
+
|
|
246
|
+
- Must start with a letter, contain only letters/digits/`-`/`_`.
|
|
247
|
+
- Reserved words (`v1`, `chat`, `completions`, `messages`, `models`, `api`)
|
|
248
|
+
are rejected to avoid colliding with real API path segments.
|
|
249
|
+
- The provider name can appear anywhere in the path; the longest match wins.
|
|
250
|
+
|
|
251
|
+
### Session identity
|
|
252
|
+
|
|
253
|
+
The proxy needs a stable per-conversation identifier to isolate compression
|
|
254
|
+
state across concurrent users/accounts. It derives one from four dimensions
|
|
255
|
+
(see `src/session-id.ts`): **protocol × upstream origin × API key ×
|
|
256
|
+
conversation**. The first three prevent cross-account / cross-provider
|
|
257
|
+
bleeding; the conversation dimension comes from whatever the client sends.
|
|
258
|
+
|
|
259
|
+
Clients differ in what they send:
|
|
260
|
+
|
|
261
|
+
| Client | Sends conversation id? | Source | Safety |
|
|
262
|
+
|---|---|---|---|
|
|
263
|
+
| **Codex** (0.147+) | ✅ yes | `body.session_id` (per-conversation UUID) | ✅ safe |
|
|
264
|
+
| **OpenCode** | ✅ yes | `x-session-affinity` header (`ses_…`) | ✅ safe |
|
|
265
|
+
| **pi** | ❌ **no** | nothing | ⚠️ **collision risk** |
|
|
266
|
+
|
|
267
|
+
When the client sends an explicit id, the proxy uses it directly. When it
|
|
268
|
+
does not (pi), the proxy falls back to hashing the first user message — so
|
|
269
|
+
two conversations that start with the same opener collapse onto the same
|
|
270
|
+
session. This does **not** corrupt data (per-message refs use a separate
|
|
271
|
+
content fingerprint that stays stable), but it can skew nudge/compression
|
|
272
|
+
timing and occasionally over-eagerly reap a block. It is self-healing: the
|
|
273
|
+
worst case is reduced compression efficiency, never data loss.
|
|
274
|
+
|
|
275
|
+
For upstream sticky-routing, when the client sends no session header the
|
|
276
|
+
proxy synthesizes one (`x-session-id: ses_<hash>`) so cache pools / load
|
|
277
|
+
balancers still get a stable key.
|
|
278
|
+
|
|
279
|
+
**Recommendation:** Codex and OpenCode are safe to run many concurrent
|
|
280
|
+
conversations through the proxy. pi is fine for a single agent, but is **not
|
|
281
|
+
recommended** for many concurrent conversations because of the collision
|
|
282
|
+
risk — until pi grows its own session-id signal. For pi multi-agent use,
|
|
283
|
+
pass an explicit `x-acp-session` header per conversation to avoid collisions.
|
|
93
284
|
|
|
94
285
|
## Status
|
|
95
286
|
|
|
96
|
-
Early. Protocol handling and compression work against mock tests (
|
|
287
|
+
Early. Protocol handling and compression work against mock tests (141 passing). Real-model integration testing is the next milestone. Expect rough edges.
|
|
97
288
|
|
|
98
289
|
See [billion-context-pi](https://github.com/ranxianglei/billion-context-pi) for the pi-extension mode (in-process, tighter integration, the reference implementation).
|
|
99
290
|
|