jev-gateway 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.env.example +12 -0
- package/CHANGELOG.md +36 -0
- package/README.md +220 -119
- package/bin/jev-claude.mjs +1 -1
- package/bin/jev-codex.mjs +1 -1
- package/bin/launcher.mjs +96 -26
- package/dist/app.js +33 -6
- package/dist/config.js +5 -1
- package/dist/dashboard.html +500 -0
- package/dist/dashboard.js +57 -0
- package/dist/events.js +101 -0
- package/dist/index.js +4 -1
- package/dist/usage.js +86 -0
- package/package.json +14 -5
package/.env.example
CHANGED
|
@@ -14,6 +14,8 @@ ARGS_MODEL=
|
|
|
14
14
|
|
|
15
15
|
# --- Routing ----------------------------------------------------------------
|
|
16
16
|
PORT=8787
|
|
17
|
+
# Interface to listen on. Keep loopback unless other machines must reach the gateway (then set ROUTER_API_KEY).
|
|
18
|
+
HOST=127.0.0.1
|
|
17
19
|
JEV_MODEL=jev-latest
|
|
18
20
|
JEV_TIMEOUT_MS=4000
|
|
19
21
|
# Below this confidence the request is forwarded untouched (the LLM decides).
|
|
@@ -24,6 +26,9 @@ JEV_ARG_MIN_CERTAINTY=0.8
|
|
|
24
26
|
JEV_ON_NONE=force_none
|
|
25
27
|
# Answer directly (no LLM call) when every argument is an enum/boolean.
|
|
26
28
|
JEV_DIRECT_CALLS=true
|
|
29
|
+
# off starts the gateway as a plain metering proxy (baseline for token comparisons).
|
|
30
|
+
# Can be flipped at runtime from the dashboard or with `jev-codex --routing on|off`.
|
|
31
|
+
JEV_ROUTING=on
|
|
27
32
|
JEV_MAX_STATE_CHARS=60000
|
|
28
33
|
JEV_MAX_MESSAGE_CHARS=4000
|
|
29
34
|
|
|
@@ -31,3 +36,10 @@ JEV_MAX_MESSAGE_CHARS=4000
|
|
|
31
36
|
# Opt-in wire dumps: every routed request (decoded JSON body, credentials redacted), what the
|
|
32
37
|
# router changed, and the reply's usage / error body. Dumps contain the whole conversation.
|
|
33
38
|
JEV_DEBUG_DUMP_DIR=
|
|
39
|
+
|
|
40
|
+
# --- Dashboard (GET /dashboard) ---------------------------------------------
|
|
41
|
+
# Label for this router's traffic; the jev-codex / jev-claude launchers set it.
|
|
42
|
+
JEV_CLIENT=standalone
|
|
43
|
+
# JSON-lines file this server's stdout is appended to (e.g. `pnpm start >> router.log`): read once
|
|
44
|
+
# at startup so the dashboard's history survives restarts. The launchers set it.
|
|
45
|
+
JEV_LOG_FILE=
|
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## 0.2.0
|
|
4
|
+
|
|
5
|
+
### Added
|
|
6
|
+
- **Monitoring dashboard.** `jev-codex --dashboard` (or `jev-claude --dashboard`) opens a local page
|
|
7
|
+
that shows whether each gateway is routing, passing traffic through, idle, or failing, and why.
|
|
8
|
+
One page covers both gateways and updates live. It shows request metadata only, never prompts,
|
|
9
|
+
tool arguments, or credentials.
|
|
10
|
+
- **Token metering.** Every forwarded request records what the provider says it cost: input tokens
|
|
11
|
+
(and how many came from the prompt cache), output tokens (and how many were reasoning), and
|
|
12
|
+
duration. Works for Chat Completions, the Responses API, and Anthropic Messages.
|
|
13
|
+
- **Baseline mode.** `--routing off` stops asking Jev but keeps metering, so the dashboard can show
|
|
14
|
+
token use with and without Jev side by side. Also a button on the dashboard and `JEV_ROUTING=off`.
|
|
15
|
+
- `HOST` setting for the interface to listen on.
|
|
16
|
+
|
|
17
|
+
### Changed
|
|
18
|
+
- **The gateway now listens on `127.0.0.1` only.** 0.1.0 listened on every interface, which made a
|
|
19
|
+
gateway that forwards your credentials reachable from the local network. Upgrade.
|
|
20
|
+
- Launcher flags lost their `jev-` prefix: `--dashboard`, `--routing`, `--status`, `--logs`,
|
|
21
|
+
`--start`, `--stop`. The gateway's own help and config are `--gateway-help` and `--print-config`,
|
|
22
|
+
because `--help` and `--config` belong to Codex and Claude Code. The old `--jev-*` spellings still
|
|
23
|
+
work.
|
|
24
|
+
- Requests are logged when their reply ends instead of when it starts, since that is when token
|
|
25
|
+
counts are known.
|
|
26
|
+
- Requires Node.js 22.15 or newer.
|
|
27
|
+
- README rewritten around getting started.
|
|
28
|
+
|
|
29
|
+
### Fixed
|
|
30
|
+
- `--stop` now stops whichever gateway answers on the port, including one started by an older
|
|
31
|
+
version that recorded its pid elsewhere.
|
|
32
|
+
|
|
33
|
+
## 0.1.0
|
|
34
|
+
|
|
35
|
+
First release: tool selection routed to Jev for Chat Completions, the Responses API, and Anthropic
|
|
36
|
+
Messages, with the `jev-codex` and `jev-claude` launchers.
|
package/README.md
CHANGED
|
@@ -1,85 +1,178 @@
|
|
|
1
1
|
# jev-gateway
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
3
|
+
A local LLM gateway for coding agents. When your agent is about to decide **which tool to call**,
|
|
4
|
+
the gateway asks [Jev](https://docs.typesafe.ai/introduction), TypeSafe's fast decision model,
|
|
5
|
+
instead of leaving that choice to the expensive reasoning model. Everything else goes to your usual
|
|
6
|
+
LLM untouched.
|
|
5
7
|
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
request is really asking **"which tool should I call?"**, the router hands that decision to
|
|
9
|
-
[Jev](https://docs.typesafe.ai/introduction) — TypeSafe's System One model — instead of paying a
|
|
10
|
-
reasoning LLM to make it.
|
|
8
|
+
It works with **Codex** and **Claude Code** out of the box, including on ChatGPT and claude.ai
|
|
9
|
+
subscriptions, and with any client that speaks the OpenAI or Anthropic APIs.
|
|
11
10
|
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
exactly that kind of question, so the split is:
|
|
11
|
+
> Independent project, not affiliated with or endorsed by TypeSafe. "Jev" is TypeSafe's model and
|
|
12
|
+
> this gateway is a client of its public API.
|
|
15
13
|
|
|
16
|
-
|
|
17
|
-
| --- | --- |
|
|
18
|
-
| Which tool, or no tool at all | **Jev** (Choice over the tool list + a Noul cross-check) |
|
|
19
|
-
| Arguments that are enums / booleans / consts | **Jev** (fanned out in the same call) |
|
|
20
|
-
| Open-ended arguments (free text, numbers, dates) | Upstream LLM, with `tool_choice` forced to Jev's pick |
|
|
21
|
-
| Plain text replies, everything without `tools` | Upstream LLM, untouched |
|
|
14
|
+
## Quick start
|
|
22
15
|
|
|
23
|
-
|
|
16
|
+
You need Node.js 22.15 or newer, a [TypeSafe API key](https://docs.typesafe.ai/introduction), and
|
|
17
|
+
Codex and/or Claude Code already installed and logged in.
|
|
24
18
|
|
|
25
|
-
|
|
26
|
-
Jev call. The conversation becomes the state; the questions are:
|
|
19
|
+
**1. Install**
|
|
27
20
|
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
argument (and "was it stated?" for optional ones), asked speculatively since extra questions are
|
|
32
|
-
nearly free
|
|
21
|
+
```bash
|
|
22
|
+
npm install -g jev-gateway
|
|
23
|
+
```
|
|
33
24
|
|
|
34
|
-
|
|
25
|
+
**2. Save your TypeSafe key**
|
|
35
26
|
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
| `hint` | Tool is confident, but `tool_choice` can't be rewritten (Anthropic: extended thinking on, or the conversation is prompt-cached) | Forwarded with a one-line suggestion appended *after* the client's last block, so cached prefixes stay intact. The LLM may disagree |
|
|
41
|
-
| `none` | Jev is confident no tool is needed | Forwarded with `tool_choice: "none"` (or untouched with `JEV_ON_NONE=passthrough`) |
|
|
42
|
-
| `passthrough` | Low confidence, the two questions disagree, Jev errored/timed out, no tools, caller already chose | Forwarded byte-for-byte; `x-jev-gateway-reason` says why |
|
|
27
|
+
```bash
|
|
28
|
+
mkdir -p ~/.jev-gateway
|
|
29
|
+
echo "TYPESAFE_API_KEY=your-key-here" > ~/.jev-gateway/.env
|
|
30
|
+
```
|
|
43
31
|
|
|
44
|
-
|
|
45
|
-
calls: every shard of the roster is ranked in one call, and the top 3 of each go on to the decision
|
|
46
|
-
above with full-length descriptions.
|
|
32
|
+
**3. Run your agent through the gateway**
|
|
47
33
|
|
|
48
|
-
|
|
49
|
-
|
|
34
|
+
```bash
|
|
35
|
+
jev-codex # use it exactly like `codex`
|
|
36
|
+
jev-claude # use it exactly like `claude`
|
|
37
|
+
```
|
|
50
38
|
|
|
51
|
-
|
|
39
|
+
**4. Watch it work**
|
|
52
40
|
|
|
53
41
|
```bash
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
42
|
+
jev-codex --dashboard
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
That's it. Your existing login keeps working, nothing in `~/.codex` or `~/.claude` is changed, and
|
|
46
|
+
plain `codex` and `claude` still behave as before. Only sessions started with the `jev-` commands go
|
|
47
|
+
through the gateway.
|
|
48
|
+
|
|
49
|
+
## What to expect
|
|
50
|
+
|
|
51
|
+
- The first `jev-codex` or `jev-claude` starts a small gateway in the background and then opens your
|
|
52
|
+
agent. Every argument is passed through, so `jev-codex exec "fix the failing test"` works like
|
|
53
|
+
`codex exec "fix the failing test"`.
|
|
54
|
+
- The gateway keeps running after you close the agent, so the next session starts instantly. Stop it
|
|
55
|
+
with `--stop`.
|
|
56
|
+
- Each turn, the gateway asks Jev which tool fits. When Jev is confident, the gateway steers the LLM
|
|
57
|
+
to that tool. When it is not, the request goes through unchanged.
|
|
58
|
+
- If Jev is down, slow, or your key is wrong, every request simply goes straight to the LLM. The
|
|
59
|
+
gateway never makes a request fail.
|
|
60
|
+
- It listens on `127.0.0.1` only.
|
|
61
|
+
|
|
62
|
+
## Commands
|
|
63
|
+
|
|
64
|
+
All of these work with both `jev-codex` and `jev-claude`.
|
|
65
|
+
|
|
66
|
+
| Command | What it does |
|
|
67
|
+
| --- | --- |
|
|
68
|
+
| `jev-codex [args]` | Start the gateway if needed, then run Codex through it |
|
|
69
|
+
| `jev-codex --dashboard` | Open the monitoring dashboard in your browser |
|
|
70
|
+
| `jev-codex --routing off` | Baseline mode: stop asking Jev, keep counting tokens |
|
|
71
|
+
| `jev-codex --routing on` | Let Jev decide again |
|
|
72
|
+
| `jev-codex --status` | Is the gateway running, and where does it forward to? |
|
|
73
|
+
| `jev-codex --logs` | Follow routing decisions live (use a second terminal) |
|
|
74
|
+
| `jev-codex --start` | Start the gateway without opening the agent |
|
|
75
|
+
| `jev-codex --stop` | Stop the background gateway (close your sessions first) |
|
|
76
|
+
| `jev-codex --print-config` | Print settings to point plain `codex` at the gateway permanently |
|
|
77
|
+
| `jev-codex --gateway-help` | List all of the above |
|
|
78
|
+
|
|
79
|
+
Codex uses port 8790 and Claude Code uses port 8789. Change them with `JEV_CODEX_PORT` and
|
|
80
|
+
`JEV_CLAUDE_PORT`.
|
|
81
|
+
|
|
82
|
+
## Dashboard
|
|
83
|
+
|
|
84
|
+
```bash
|
|
85
|
+
jev-codex --dashboard # or: jev-claude --dashboard
|
|
57
86
|
```
|
|
58
87
|
|
|
59
|
-
|
|
60
|
-
your
|
|
88
|
+
This opens `http://localhost:8790/dashboard`. If no browser window appears, paste that address into
|
|
89
|
+
your browser. One page shows both gateways (Codex and Claude) and refreshes every 2 seconds.
|
|
90
|
+
|
|
91
|
+
You will see:
|
|
92
|
+
|
|
93
|
+
- **A status per gateway:** Routing, Passthrough only, Jev is failing, Idle, Baseline, or Offline,
|
|
94
|
+
with a one-line explanation.
|
|
95
|
+
- **Why requests were not routed,** with each reason explained in plain English.
|
|
96
|
+
- **Jev's numbers:** calls, latency, confidence, and what it cost.
|
|
97
|
+
- **LLM tokens:** input (and how much came from the prompt cache), output (and how much was hidden
|
|
98
|
+
reasoning), and seconds per request.
|
|
99
|
+
- **A live table** of recent requests. A request appears when its reply finishes, because that is
|
|
100
|
+
when the provider reports its tokens.
|
|
61
101
|
|
|
62
|
-
|
|
102
|
+
The dashboard only shows request metadata. Prompts, tool arguments, and credentials never reach it.
|
|
103
|
+
|
|
104
|
+
### Is it worth it? Compare with a baseline
|
|
105
|
+
|
|
106
|
+
Switch routing off to measure the same work without Jev. The gateway keeps forwarding and counting
|
|
107
|
+
tokens, but never asks Jev and rewrites nothing.
|
|
108
|
+
|
|
109
|
+
```bash
|
|
110
|
+
jev-codex --routing off # do a task
|
|
111
|
+
jev-codex --routing on # do a similar task
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
The same switch is a button on each gateway card. The "Token use" card then shows both states side
|
|
115
|
+
by side: tokens in and out per request, cache share, reasoning tokens, and seconds. The comparison
|
|
116
|
+
is only meaningful if you do similar work in both states.
|
|
117
|
+
|
|
118
|
+
## Using it with Codex
|
|
119
|
+
|
|
120
|
+
`jev-codex` reuses your existing Codex login. With a ChatGPT subscription the gateway forwards to
|
|
121
|
+
`https://chatgpt.com/backend-api/codex`. With an API key it forwards to `https://api.openai.com/v1`.
|
|
122
|
+
Override either with `JEV_CODEX_UPSTREAM_BASE_URL`.
|
|
123
|
+
|
|
124
|
+
Codex speaks the Responses API, so the gateway handles `POST /v1/responses`, including Codex's
|
|
125
|
+
free-form tools such as `apply_patch`, tools declared inside the conversation, and compressed
|
|
126
|
+
request bodies. If the backend rejects a rewritten request, the gateway resends the original, so
|
|
127
|
+
Codex never sees an error caused by the gateway.
|
|
128
|
+
|
|
129
|
+
## Using it with Claude Code
|
|
130
|
+
|
|
131
|
+
`jev-claude` runs `claude` with only `ANTHROPIC_BASE_URL` set. Claude Code keeps using its saved
|
|
132
|
+
login, so a claude.ai subscription keeps working and its usual limits apply.
|
|
133
|
+
|
|
134
|
+
Jev can do less here than with Codex, because of how the Anthropic API works. Claude Code runs with
|
|
135
|
+
extended thinking, and the API rejects a forced tool while thinking is on. It also rereads a cached
|
|
136
|
+
conversation on every turn, and changing `tool_choice` would invalidate that cache. So for Claude
|
|
137
|
+
Code the gateway adds a short suggestion to the request instead (`hint` mode), which the model is
|
|
138
|
+
free to ignore. Expect better tool picks on large tool lists, not lower cost or latency.
|
|
139
|
+
|
|
140
|
+
## Running it as a server for your own app
|
|
141
|
+
|
|
142
|
+
Work from a checkout:
|
|
63
143
|
|
|
64
144
|
```bash
|
|
65
145
|
pnpm install
|
|
66
|
-
cp .env.example .env # set TYPESAFE_API_KEY, and UPSTREAM_BASE_URL if
|
|
67
|
-
pnpm dev
|
|
146
|
+
cp .env.example .env # set TYPESAFE_API_KEY, and UPSTREAM_BASE_URL if you don't use OpenAI
|
|
147
|
+
pnpm dev # listens on http://localhost:8787
|
|
68
148
|
```
|
|
69
149
|
|
|
150
|
+
Then point your client at it:
|
|
151
|
+
|
|
70
152
|
```python
|
|
71
153
|
from openai import OpenAI
|
|
72
154
|
client = OpenAI(base_url="http://localhost:8787/v1") # your usual provider key still works
|
|
73
155
|
```
|
|
74
156
|
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
157
|
+
The gateway routes three endpoints and proxies every other `/v1/*` path unchanged:
|
|
158
|
+
|
|
159
|
+
| Endpoint | API |
|
|
160
|
+
| --- | --- |
|
|
161
|
+
| `POST /v1/chat/completions` | OpenAI Chat Completions |
|
|
162
|
+
| `POST /v1/responses` | OpenAI Responses |
|
|
163
|
+
| `POST /v1/messages` | Anthropic Messages |
|
|
164
|
+
|
|
165
|
+
By default your client's own `Authorization` header is forwarded to the provider. Set
|
|
166
|
+
`UPSTREAM_API_KEY` to have the gateway hold the provider key instead, and `ROUTER_API_KEY` to
|
|
167
|
+
require a gateway key from clients. Any OpenAI-compatible provider works, for example OpenAI,
|
|
168
|
+
OpenRouter, vLLM, Ollama, or LiteLLM.
|
|
78
169
|
|
|
79
|
-
|
|
170
|
+
To skip Jev for a single request, send the header `x-jev-gateway: off`.
|
|
80
171
|
|
|
81
|
-
|
|
82
|
-
|
|
172
|
+
### Try a decision without calling any LLM
|
|
173
|
+
|
|
174
|
+
`POST /router/decide` takes a request body, asks Jev, and returns the decision: the mode, the tool,
|
|
175
|
+
the arguments, Jev's confidence, and its latency.
|
|
83
176
|
|
|
84
177
|
```bash
|
|
85
178
|
curl -s localhost:8787/router/decide -H 'content-type: application/json' -d '{
|
|
@@ -93,87 +186,95 @@ curl -s localhost:8787/router/decide -H 'content-type: application/json' -d '{
|
|
|
93
186
|
}'
|
|
94
187
|
```
|
|
95
188
|
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
## Use it with Codex (local)
|
|
99
|
-
|
|
100
|
-
Codex only speaks the Responses API, so the router handles `POST /v1/responses` the same way as
|
|
101
|
-
chat completions — including Codex's free-form tools (`apply_patch`), provider-run tools
|
|
102
|
-
(`web_search`, offered to Jev but never forced) and zstd-compressed request bodies.
|
|
103
|
-
|
|
104
|
-
```bash
|
|
105
|
-
jev-codex # instead of `codex`; every codex argument still works
|
|
106
|
-
jev-codex exec "fix the failing test"
|
|
107
|
-
jev-codex --jev-logs # second terminal: watch each routing decision live
|
|
108
|
-
```
|
|
189
|
+
## How it works
|
|
109
190
|
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
(`requires_openai_auth = true`): with a ChatGPT subscription the router forwards to
|
|
114
|
-
`https://chatgpt.com/backend-api/codex`, with an API key to `https://api.openai.com/v1`
|
|
115
|
-
(override with `JEV_CODEX_UPSTREAM_BASE_URL`). `TYPESAFE_API_KEY` is read from the environment,
|
|
116
|
-
`~/.jev-gateway/.env`, or a checkout's own `.env`. `jev-codex --jev-help` lists the rest (`--jev-status`, `--jev-stop`,
|
|
117
|
-
`--jev-config` for a permanent `codex --profile jev`).
|
|
191
|
+
Jev does not generate text. It answers typed questions about a piece of state (pick one option,
|
|
192
|
+
give a score, or yes/no) and returns calibrated probabilities with a confidence, in one fast call.
|
|
193
|
+
Choosing a tool is exactly that kind of question, so the work is split like this:
|
|
118
194
|
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
195
|
+
| Decision | Who makes it |
|
|
196
|
+
| --- | --- |
|
|
197
|
+
| Which tool, or no tool at all | **Jev** |
|
|
198
|
+
| Arguments that are enums, booleans, or constants | **Jev**, in the same call |
|
|
199
|
+
| Open-ended arguments such as free text, numbers, and dates | The LLM, already pointed at Jev's tool |
|
|
200
|
+
| Plain text replies, and any request without tools | The LLM, untouched |
|
|
123
201
|
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
```
|
|
202
|
+
A request that carries tools triggers one Jev call. The conversation becomes the state, and the
|
|
203
|
+
questions are: which tool (or none), whether a tool is needed at all (an independent cross-check),
|
|
204
|
+
and the value of every closed-set argument. The answer selects a mode, which is reported in the
|
|
205
|
+
`x-jev-gateway-mode` response header:
|
|
129
206
|
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
207
|
+
| Mode | When | What happens |
|
|
208
|
+
| --- | --- | --- |
|
|
209
|
+
| `direct` | Jev is confident about the tool and every argument is an enum, boolean, or constant | The gateway builds the tool call itself, streaming included. **No LLM call.** |
|
|
210
|
+
| `forced` | Jev is confident about the tool, but some arguments are open-ended | Forwarded with `tool_choice` set to that tool, so the LLM only fills in arguments. `ARGS_MODEL` can send these to a cheaper model |
|
|
211
|
+
| `hint` | Jev is confident, but `tool_choice` cannot be changed (Anthropic with thinking on, or a cached conversation) | Forwarded with a one-line suggestion added after the client's last block, so cached prefixes stay valid |
|
|
212
|
+
| `none` | Jev is confident that no tool is needed | Forwarded with `tool_choice: "none"` |
|
|
213
|
+
| `passthrough` | Low confidence, the two checks disagree, Jev failed, there are no tools, or the caller already chose | Forwarded byte for byte. `x-jev-gateway-reason` says why |
|
|
134
214
|
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
conversation every turn (any `tool_choice` change would invalidate it). So Claude Code requests are
|
|
138
|
-
steered with `hint` mode, `none` is never applied, and `direct` still answers without the LLM when
|
|
139
|
-
a tool's arguments are all closed-set. API callers without thinking or message caching get `forced`.
|
|
215
|
+
Tool lists longer than 120 entries (Claude Code sends about 280) take two Jev calls. The first ranks
|
|
216
|
+
the list in groups. The second decides among the top 3 of each group, using full descriptions.
|
|
140
217
|
|
|
141
218
|
## Configuration
|
|
142
219
|
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
- `JEV_ARG_MIN_CERTAINTY` (0.8) — the weakest argument must clear this for a `direct` answer;
|
|
147
|
-
otherwise the request degrades to `forced`.
|
|
148
|
-
- `ARGS_MODEL` — a cheap model for argument filling in `forced` mode.
|
|
149
|
-
- `JEV_DIRECT_CALLS=false` — never answer without the LLM; Jev only picks the tool.
|
|
220
|
+
Settings are environment variables. The launchers read them from your shell,
|
|
221
|
+
`~/.jev-gateway/.env`, or a checkout's own `.env`. See [.env.example](.env.example) for the full
|
|
222
|
+
list. The ones worth knowing:
|
|
150
223
|
|
|
151
|
-
|
|
224
|
+
| Variable | Default | Meaning |
|
|
225
|
+
| --- | --- | --- |
|
|
226
|
+
| `TYPESAFE_API_KEY` | required | Your TypeSafe key |
|
|
227
|
+
| `JEV_MIN_CONFIDENCE` | `0.7` | Below this confidence, the LLM decides. Lower it to route more, raise it to be more careful |
|
|
228
|
+
| `JEV_ARG_MIN_CERTAINTY` | `0.8` | Every argument must reach this for a `direct` answer |
|
|
229
|
+
| `JEV_DIRECT_CALLS` | `true` | Set to `false` so the gateway never answers without the LLM |
|
|
230
|
+
| `JEV_ROUTING` | `on` | Set to `off` to start in baseline mode |
|
|
231
|
+
| `JEV_TIMEOUT_MS` | `4000` | How long to wait for Jev before letting the LLM decide |
|
|
232
|
+
| `ARGS_MODEL` | unset | A cheaper model for filling arguments in `forced` mode |
|
|
233
|
+
| `HOST` | `127.0.0.1` | Interface to listen on. Set `ROUTER_API_KEY` before exposing it |
|
|
234
|
+
| `JEV_DEBUG_DUMP_DIR` | unset | Write requests and response summaries to this folder, with credentials redacted |
|
|
235
|
+
|
|
236
|
+
Each request also logs one JSON line to stdout, or to `~/.jev-gateway/<client>.log` under a launcher.
|
|
152
237
|
|
|
153
238
|
## Known trade-offs
|
|
154
239
|
|
|
240
|
+
- Jev adds a network call to every turn that carries tools. Expect roughly half a second to a second.
|
|
155
241
|
- Jev picks **one** tool per turn. In `forced` mode the LLM can still call that tool several times
|
|
156
|
-
in parallel, but
|
|
157
|
-
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
##
|
|
242
|
+
in parallel, but it cannot mix different tools in the same turn.
|
|
243
|
+
- A wrong forced tool can derail a turn. If the model had nothing left to do and is forced to call
|
|
244
|
+
a tool anyway, it may produce an incomplete reply and the agent will retry. Raise
|
|
245
|
+
`JEV_MIN_CONFIDENCE` if you see this.
|
|
246
|
+
- In `hint` mode the LLM still does its own reasoning, so the gain is accuracy, not cost.
|
|
247
|
+
- Jev reads text only and has a 32k-token window. Images become placeholders and long conversations
|
|
248
|
+
keep their newest turns. It is most accurate in English.
|
|
249
|
+
- The default confidence thresholds are starting points. Use the dashboard and baseline mode to tune
|
|
250
|
+
them for your own work.
|
|
251
|
+
|
|
252
|
+
## Development
|
|
167
253
|
|
|
254
|
+
```bash
|
|
255
|
+
pnpm install
|
|
256
|
+
pnpm test # runs against fake Jev and provider transports, no keys needed
|
|
257
|
+
pnpm typecheck
|
|
258
|
+
pnpm build
|
|
168
259
|
```
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
src/
|
|
175
|
-
|
|
176
|
-
|
|
260
|
+
|
|
261
|
+
`scripts/mock-jev.mjs` is a local stand-in for Jev. Point `TYPESAFE_BASE_URL` at it to drive a real
|
|
262
|
+
agent end to end without a TypeSafe key.
|
|
263
|
+
|
|
264
|
+
```
|
|
265
|
+
src/adapters/ request formats: chat.ts, responses.ts (Codex), messages.ts (Claude Code)
|
|
266
|
+
src/state.ts turns a conversation into Jev state
|
|
267
|
+
src/questions.ts turns tools into Jev questions and finds closed-set arguments
|
|
268
|
+
src/decide.ts the Jev call and the mode decision
|
|
269
|
+
src/upstream.ts streaming reverse proxy
|
|
270
|
+
src/usage.ts token usage read from a reply, normalised across providers
|
|
271
|
+
src/app.ts routes, auth, headers, and the resend-on-rejection fallback
|
|
272
|
+
src/events.ts recent request metadata kept in memory and restored from the log
|
|
273
|
+
src/dashboard.ts serves /dashboard (dashboard.html is the whole page, no build step)
|
|
274
|
+
bin/ jev-codex and jev-claude launchers (shared logic in launcher.mjs)
|
|
275
|
+
scripts/mock-jev.mjs local stand-in for Jev
|
|
177
276
|
```
|
|
178
277
|
|
|
179
|
-
|
|
278
|
+
## License
|
|
279
|
+
|
|
280
|
+
MIT
|
package/bin/jev-claude.mjs
CHANGED
|
@@ -13,7 +13,7 @@ await runLauncher({
|
|
|
13
13
|
// saved claude.ai login, so a Pro/Max subscription (or an existing API key) keeps working as is.
|
|
14
14
|
env: (origin) => ({ ANTHROPIC_BASE_URL: origin }),
|
|
15
15
|
configHelp: (origin) =>
|
|
16
|
-
`# Keep the router running (jev-claude --
|
|
16
|
+
`# Keep the router running (jev-claude --start), then either:\n` +
|
|
17
17
|
`# ANTHROPIC_BASE_URL=${origin} claude\n` +
|
|
18
18
|
`# or add to ~/.claude/settings.json:\n` +
|
|
19
19
|
JSON.stringify({ env: { ANTHROPIC_BASE_URL: origin } }, null, 2),
|
package/bin/jev-codex.mjs
CHANGED
|
@@ -44,7 +44,7 @@ await runLauncher({
|
|
|
44
44
|
...Object.entries(provider(origin)).flatMap(([key, value]) => ["-c", `model_providers.jev-gateway.${key}=${value}`]),
|
|
45
45
|
],
|
|
46
46
|
configHelp: (origin) =>
|
|
47
|
-
`# Save as ~/.codex/jev.config.toml, keep the router running (jev-codex --
|
|
47
|
+
`# Save as ~/.codex/jev.config.toml, keep the router running (jev-codex --start),\n` +
|
|
48
48
|
`# then use: codex --profile jev\n` +
|
|
49
49
|
`model_provider = "jev-gateway"\n\n[model_providers.jev-gateway]\n` +
|
|
50
50
|
Object.entries(provider(origin))
|