billion-context 0.1.1 → 0.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +186 -46
- package/dist/index.js +2134 -354
- package/dist/index.js.map +1 -1
- package/package.json +8 -3
package/README.md
CHANGED
|
@@ -36,84 +36,165 @@ The proxy injects four context-management tools (`compress`, `decompress`, `sear
|
|
|
36
36
|
npm install -g billion-context
|
|
37
37
|
```
|
|
38
38
|
|
|
39
|
+
This installs the `bili` command (`bili-proxy` is kept as an alias).
|
|
40
|
+
|
|
39
41
|
## Usage
|
|
40
42
|
|
|
41
|
-
###
|
|
43
|
+
### Start the proxy
|
|
44
|
+
|
|
45
|
+
```bash
|
|
46
|
+
bili
|
|
47
|
+
```
|
|
42
48
|
|
|
43
|
-
|
|
49
|
+
That's it. The proxy reads its config from `~/.config/billion-context/billion-context.json` (XDG) and listens on `127.0.0.1:8787`. If no config file exists yet, it uses sensible defaults and logs where it expects the file.
|
|
50
|
+
|
|
51
|
+
### Quick overrides (flags)
|
|
44
52
|
|
|
45
53
|
```bash
|
|
46
|
-
|
|
54
|
+
bili --port 9000 # change listen port
|
|
55
|
+
bili --host 0.0.0.0 # listen on all interfaces
|
|
56
|
+
bili --debug # verbose logging (also: set "debug": true in config)
|
|
57
|
+
bili --passthrough # forward without compression (smoke-test mode)
|
|
58
|
+
bili --config ~/my-bili.json # use a different config file
|
|
47
59
|
```
|
|
48
60
|
|
|
49
|
-
|
|
61
|
+
Flags override the config file and env vars. `bili --help` lists them all.
|
|
50
62
|
|
|
51
|
-
###
|
|
63
|
+
### Point your agent at the proxy
|
|
52
64
|
|
|
53
|
-
|
|
54
|
-
providers by URL path — most users will want this.
|
|
65
|
+
The proxy routes by a **provider name in the URL path**. Set your agent's base URL to `http://localhost:8787/<provider>/...` and the proxy forwards to that provider (see [Configuration](#configuration) for how providers are declared).
|
|
55
66
|
|
|
56
|
-
|
|
67
|
+
#### Claude Code (Anthropic)
|
|
57
68
|
|
|
58
69
|
```bash
|
|
59
|
-
export ANTHROPIC_BASE_URL=http://localhost:8787
|
|
60
|
-
export ANTHROPIC_API_KEY=sk-ant-...
|
|
70
|
+
export ANTHROPIC_BASE_URL=http://localhost:8787/anthropic
|
|
71
|
+
export ANTHROPIC_API_KEY=sk-ant-... # real key — passed through as-is
|
|
61
72
|
claude
|
|
62
73
|
```
|
|
63
74
|
|
|
64
|
-
|
|
75
|
+
#### Codex / any OpenAI-compatible agent (zhipu / openai / deepseek)
|
|
65
76
|
|
|
66
77
|
```bash
|
|
67
|
-
export OPENAI_BASE_URL=http://localhost:8787/
|
|
68
|
-
export OPENAI_API_KEY
|
|
78
|
+
export OPENAI_BASE_URL=http://localhost:8787/zhipu/api/coding/paas/v4
|
|
79
|
+
export OPENAI_API_KEY=<your real glm key> # passed through as-is
|
|
69
80
|
codex
|
|
70
81
|
```
|
|
71
82
|
|
|
72
|
-
|
|
83
|
+
The `/zhipu/...` prefix tells the proxy to route to the `zhipu` provider; the
|
|
84
|
+
remaining path is preserved.
|
|
85
|
+
|
|
86
|
+
#### Cursor / Aider / others
|
|
87
|
+
|
|
88
|
+
Set the base URL to `http://localhost:8787/<provider>` in the agent's settings.
|
|
89
|
+
|
|
90
|
+
### Debugging
|
|
73
91
|
|
|
74
|
-
|
|
92
|
+
Three ways to enable verbose logging (priority: flag > env > config):
|
|
93
|
+
|
|
94
|
+
1. **CLI flag** (quickest): `bili --debug`
|
|
95
|
+
2. **Env var**: `ACP_DEBUG=1 bili`
|
|
96
|
+
3. **Config file**: `"debug": true` in `billion-context.json`
|
|
97
|
+
|
|
98
|
+
Verbose mode logs every `processTurn` (tag counts, token usage), the nudge
|
|
99
|
+
decision (growth/usage/pendingT1/shouldInject), client headers, and SSE
|
|
100
|
+
rewrites.
|
|
75
101
|
|
|
76
102
|
## Configuration
|
|
77
103
|
|
|
78
|
-
|
|
104
|
+
Configuration is read from a JSON file with env-var overrides. Priority:
|
|
105
|
+
**env var > config file > built-in default**.
|
|
79
106
|
|
|
80
|
-
|
|
81
|
-
|----------|---------|-------------|
|
|
82
|
-
| `PORT` | `8787` | Proxy listen port |
|
|
83
|
-
| `HOST` | `127.0.0.1` | Proxy listen host |
|
|
84
|
-
| `UPSTREAM` | `https://api.anthropic.com` | Default upstream when no route matches |
|
|
85
|
-
| `ACP_PROVIDERS` | *(none)* | Path to a JSON file mapping provider names to root URLs (see below) |
|
|
86
|
-
| `ACP_COMPRESS_TOOL` | `1` | Set `0` to disable injecting the compress tool |
|
|
87
|
-
| `ACP_DEBUG` | `0` | Set `1` for verbose logging |
|
|
88
|
-
| `ACP_PASSTHROUGH` | `0` | Set `1` to forward without compression |
|
|
107
|
+
### Config file
|
|
89
108
|
|
|
90
|
-
|
|
109
|
+
Location (XDG Base Directory):
|
|
91
110
|
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
are never stored in the proxy** — whatever key the agent sends is passed
|
|
95
|
-
through untouched to the upstream.
|
|
111
|
+
- **Linux:** `~/.config/billion-context/billion-context.json`
|
|
112
|
+
- Override with `XDG_CONFIG_HOME` or `BILI_CONFIG_FILE`
|
|
96
113
|
|
|
97
|
-
|
|
114
|
+
The config file is a single JSON object. Example:
|
|
98
115
|
|
|
99
116
|
```json
|
|
100
117
|
{
|
|
101
|
-
"
|
|
102
|
-
"
|
|
103
|
-
"
|
|
104
|
-
|
|
118
|
+
"port": 8787,
|
|
119
|
+
"host": "127.0.0.1",
|
|
120
|
+
"providers": {
|
|
121
|
+
"zhipu": {
|
|
122
|
+
"url": "https://open.bigmodel.cn",
|
|
123
|
+
"models": {
|
|
124
|
+
"glm-5.2": { "context": 1000000, "output": 131072 },
|
|
125
|
+
"glm-5.1": { "context": 200000, "output": 131072 }
|
|
126
|
+
}
|
|
127
|
+
},
|
|
128
|
+
"anthropic": "https://api.anthropic.com",
|
|
129
|
+
"deepseek": "https://api.deepseek.com"
|
|
130
|
+
}
|
|
105
131
|
}
|
|
106
132
|
```
|
|
107
133
|
|
|
108
|
-
|
|
134
|
+
### Top-level keys
|
|
109
135
|
|
|
110
|
-
|
|
111
|
-
|
|
136
|
+
| Key | Default | Description |
|
|
137
|
+
|------|---------|-------------|
|
|
138
|
+
| `port` | `8787` | Proxy listen port |
|
|
139
|
+
| `host` | `127.0.0.1` | Proxy listen host |
|
|
140
|
+
| `upstream` | `https://api.anthropic.com` | Default upstream when no route matches |
|
|
141
|
+
| `sessionHeader` | `x-acp-session` | Header name clients may send to identify a conversation |
|
|
142
|
+
| `log` | `true` | Enable request logging |
|
|
143
|
+
| `debug` | `false` | Verbose logging (same as `ACP_DEBUG=1`) |
|
|
144
|
+
| `passthrough` | `false` | Forward without compression (same as `ACP_PASSTHROUGH=1`) |
|
|
145
|
+
| `providers` | *(none)* | Provider routes — see below |
|
|
146
|
+
| `condense` | *(see defaults)* | Tool-result condensing: `{ enabled, keepRecentToolResults, minCharsToCondense, maxKeptChars }` |
|
|
147
|
+
| `compress` | *(see defaults)* | `{ injectTool, injectNudge }` |
|
|
148
|
+
|
|
149
|
+
### Providers (URL routing + per-model context)
|
|
150
|
+
|
|
151
|
+
`providers` maps a route name to either a bare URL string (simple) or an
|
|
152
|
+
object with `url` + optional per-model context window (recommended).
|
|
153
|
+
|
|
154
|
+
**Simple form** — provider name → URL:
|
|
155
|
+
```json
|
|
156
|
+
{ "deepseek": "https://api.deepseek.com" }
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
**Full form** — provider name → `{ url, models }`:
|
|
160
|
+
```json
|
|
161
|
+
{
|
|
162
|
+
"zhipu": {
|
|
163
|
+
"url": "https://open.bigmodel.cn",
|
|
164
|
+
"models": {
|
|
165
|
+
"glm-5.2": { "context": 1000000, "output": 131072 },
|
|
166
|
+
"glm-5.1": { "context": 200000 }
|
|
167
|
+
}
|
|
168
|
+
}
|
|
169
|
+
}
|
|
112
170
|
```
|
|
113
171
|
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
172
|
+
The same model can have a different context window behind different providers
|
|
173
|
+
(e.g. relay wraps a model with a larger window). `context` is the **input
|
|
174
|
+
context limit** (used by the compressor to decide when to nudge); `output` is
|
|
175
|
+
the max output tokens. Both are optional; missing values fall back to the
|
|
176
|
+
built-in model table, then to `modelContextLimit`.
|
|
177
|
+
|
|
178
|
+
> **Why declare context at all?** The LLM `/models` API does **not** return
|
|
179
|
+
> context windows (verified across OpenAI, Anthropic, 智谱, comfly). They are
|
|
180
|
+
> document-level information. A wrong value (e.g. GLM-5.2 guessed as 128K
|
|
181
|
+
> instead of 1M) causes spurious frequent compression. Declaring it per
|
|
182
|
+
> provider + model makes the proxy match the registry the client itself uses.
|
|
183
|
+
|
|
184
|
+
**API keys are never stored in the proxy** — whatever key the agent sends is
|
|
185
|
+
passed through untouched to the upstream.
|
|
186
|
+
|
|
187
|
+
### Routing
|
|
188
|
+
|
|
189
|
+
Point any agent at the proxy using a provider name as a path segment. The
|
|
190
|
+
proxy strips the name and forwards to that provider's root URL.
|
|
191
|
+
|
|
192
|
+
```
|
|
193
|
+
agent baseURL: http://localhost:8787/zhipu/api/coding/paas/v4
|
|
194
|
+
└──────────┬──────────┘└────────┬────────┘
|
|
195
|
+
proxy host remaining path
|
|
196
|
+
+ provider name (forwarded as-is)
|
|
197
|
+
```
|
|
117
198
|
|
|
118
199
|
#### Claude Code (Anthropic)
|
|
119
200
|
|
|
@@ -126,14 +207,39 @@ claude
|
|
|
126
207
|
#### Codex / any OpenAI-compatible agent (zhipu / openai / deepseek)
|
|
127
208
|
|
|
128
209
|
```bash
|
|
129
|
-
export OPENAI_BASE_URL=http://localhost:8787/
|
|
210
|
+
export OPENAI_BASE_URL=http://localhost:8787/zhipu/api/coding/paas/v4
|
|
130
211
|
export OPENAI_API_KEY=<your real glm key> # passed through as-is
|
|
131
212
|
codex
|
|
132
213
|
```
|
|
133
214
|
|
|
134
|
-
The `/
|
|
135
|
-
remaining `/
|
|
136
|
-
|
|
215
|
+
The `/zhipu/...` prefix tells the proxy to route to the `zhipu` provider; the
|
|
216
|
+
remaining `/api/coding/paas/v4/...` path is preserved.
|
|
217
|
+
|
|
218
|
+
### Environment variables (override the config file)
|
|
219
|
+
|
|
220
|
+
Every config key has an env-var override. Set to override the file value.
|
|
221
|
+
|
|
222
|
+
| Env | Default | Description |
|
|
223
|
+
|-----|---------|-------------|
|
|
224
|
+
| `ACP_PORT` / `PORT` | `8787` | Listen port |
|
|
225
|
+
| `ACP_HOST` | `127.0.0.1` | Listen host |
|
|
226
|
+
| `ACP_UPSTREAM` | `https://api.anthropic.com` | Default upstream |
|
|
227
|
+
| `ACP_PROVIDERS` | *(none)* | Path to a legacy providers JSON file (overrides `providers` in config) |
|
|
228
|
+
| `ACP_MODEL_CONTEXT_LIMIT` | `200000` | Global fallback context window (only used when no provider/model match) |
|
|
229
|
+
| `ACP_SESSION_HEADER` | `x-acp-session` | Conversation-id header name |
|
|
230
|
+
| `ACP_COMPRESS_TOOL` | `1` | Set `0` to disable injecting the compress tool |
|
|
231
|
+
| `ACP_COMPRESS_NUDGE` | `1` | Set `0` to disable compression nudges |
|
|
232
|
+
| `ACP_CONDENSE_ENABLED` | `1` | Set `0` to disable tool-result condensing |
|
|
233
|
+
| `ACP_KEEP_RECENT_TOOL_RESULTS` | `6` | Tool results kept verbatim before condensing |
|
|
234
|
+
| `ACP_MIN_CHARS_TO_CONDENSE` | `1500` | Condense tool results longer than this |
|
|
235
|
+
| `ACP_MAX_KEPT_CHARS` | `400` | Max chars kept when condensing a tool result |
|
|
236
|
+
| `ACP_DEBUG` | `0` | Set `1` for verbose logging |
|
|
237
|
+
| `ACP_PASSTHROUGH` | `0` | Set `1` to forward without compression |
|
|
238
|
+
| `ACP_DUMP_SSE` | *(none)* | Directory to dump SSE for debugging |
|
|
239
|
+
| `BILI_PERSIST` | `1` | Set `0` to disable session persistence (in-memory only, lost on restart) |
|
|
240
|
+
| `BILI_PERSIST_DEBOUNCE_MS` | `500` | Debounce window for writes to disk (ms) |
|
|
241
|
+
| `BILI_MAX_SESSIONS` | `256` | Max sessions held in memory (LRU eviction; disk is source of truth) |
|
|
242
|
+
| `BILI_SESSIONS_DIR` | *(XDG data dir)* | Directory for persisted session state |
|
|
137
243
|
|
|
138
244
|
### Notes on provider names
|
|
139
245
|
|
|
@@ -142,9 +248,43 @@ provider key — the proxy never reads or stores it.
|
|
|
142
248
|
are rejected to avoid colliding with real API path segments.
|
|
143
249
|
- The provider name can appear anywhere in the path; the longest match wins.
|
|
144
250
|
|
|
251
|
+
### Session identity
|
|
252
|
+
|
|
253
|
+
The proxy needs a stable per-conversation identifier to isolate compression
|
|
254
|
+
state across concurrent users/accounts. It derives one from four dimensions
|
|
255
|
+
(see `src/session-id.ts`): **protocol × upstream origin × API key ×
|
|
256
|
+
conversation**. The first three prevent cross-account / cross-provider
|
|
257
|
+
bleeding; the conversation dimension comes from whatever the client sends.
|
|
258
|
+
|
|
259
|
+
Clients differ in what they send:
|
|
260
|
+
|
|
261
|
+
| Client | Sends conversation id? | Source | Safety |
|
|
262
|
+
|---|---|---|---|
|
|
263
|
+
| **Codex** (0.147+) | ✅ yes | `body.session_id` (per-conversation UUID) | ✅ safe |
|
|
264
|
+
| **OpenCode** | ✅ yes | `x-session-affinity` header (`ses_…`) | ✅ safe |
|
|
265
|
+
| **pi** | ❌ **no** | nothing | ⚠️ **collision risk** |
|
|
266
|
+
|
|
267
|
+
When the client sends an explicit id, the proxy uses it directly. When it
|
|
268
|
+
does not (pi), the proxy falls back to hashing the first user message — so
|
|
269
|
+
two conversations that start with the same opener collapse onto the same
|
|
270
|
+
session. This does **not** corrupt data (per-message refs use a separate
|
|
271
|
+
content fingerprint that stays stable), but it can skew nudge/compression
|
|
272
|
+
timing and occasionally over-eagerly reap a block. It is self-healing: the
|
|
273
|
+
worst case is reduced compression efficiency, never data loss.
|
|
274
|
+
|
|
275
|
+
For upstream sticky-routing, when the client sends no session header the
|
|
276
|
+
proxy synthesizes one (`x-session-id: ses_<hash>`) so cache pools / load
|
|
277
|
+
balancers still get a stable key.
|
|
278
|
+
|
|
279
|
+
**Recommendation:** Codex and OpenCode are safe to run many concurrent
|
|
280
|
+
conversations through the proxy. pi is fine for a single agent, but is **not
|
|
281
|
+
recommended** for many concurrent conversations because of the collision
|
|
282
|
+
risk — until pi grows its own session-id signal. For pi multi-agent use,
|
|
283
|
+
pass an explicit `x-acp-session` header per conversation to avoid collisions.
|
|
284
|
+
|
|
145
285
|
## Status
|
|
146
286
|
|
|
147
|
-
Early. Protocol handling and compression work against mock tests (
|
|
287
|
+
Early. Protocol handling and compression work against mock tests (141 passing). Real-model integration testing is the next milestone. Expect rough edges.
|
|
148
288
|
|
|
149
289
|
See [billion-context-pi](https://github.com/ranxianglei/billion-context-pi) for the pi-extension mode (in-process, tighter integration, the reference implementation).
|
|
150
290
|
|