jev-gateway 0.1.0 → 0.2.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/.env.example CHANGED
@@ -14,6 +14,8 @@ ARGS_MODEL=
14
14
 
15
15
  # --- Routing ----------------------------------------------------------------
16
16
  PORT=8787
17
+ # Interface to listen on. Keep loopback unless other machines must reach the gateway (then set ROUTER_API_KEY).
18
+ HOST=127.0.0.1
17
19
  JEV_MODEL=jev-latest
18
20
  JEV_TIMEOUT_MS=4000
19
21
  # Below this confidence the request is forwarded untouched (the LLM decides).
@@ -24,6 +26,9 @@ JEV_ARG_MIN_CERTAINTY=0.8
24
26
  JEV_ON_NONE=force_none
25
27
  # Answer directly (no LLM call) when every argument is an enum/boolean.
26
28
  JEV_DIRECT_CALLS=true
29
+ # off starts the gateway as a plain metering proxy (baseline for token comparisons).
30
+ # Can be flipped at runtime from the dashboard or with `jev-codex --routing on|off`.
31
+ JEV_ROUTING=on
27
32
  JEV_MAX_STATE_CHARS=60000
28
33
  JEV_MAX_MESSAGE_CHARS=4000
29
34
 
@@ -31,3 +36,10 @@ JEV_MAX_MESSAGE_CHARS=4000
31
36
  # Opt-in wire dumps: every routed request (decoded JSON body, credentials redacted), what the
32
37
  # router changed, and the reply's usage / error body. Dumps contain the whole conversation.
33
38
  JEV_DEBUG_DUMP_DIR=
39
+
40
+ # --- Dashboard (GET /dashboard) ---------------------------------------------
41
+ # Label for this router's traffic; the jev-codex / jev-claude launchers set it.
42
+ JEV_CLIENT=standalone
43
+ # JSON-lines file this server's stdout is appended to (e.g. `pnpm start >> router.log`): read once
44
+ # at startup so the dashboard's history survives restarts. The launchers set it.
45
+ JEV_LOG_FILE=
package/CHANGELOG.md ADDED
@@ -0,0 +1,42 @@
1
+ # Changelog
2
+
3
+ ## 0.2.1
4
+
5
+ ### Fixed
6
+ - Token metering now works with Codex on a ChatGPT subscription. That backend streams its replies
7
+ without a content-type header, so 0.2.0 read them as plain JSON and recorded no tokens at all.
8
+
9
+ ## 0.2.0
10
+
11
+ ### Added
12
+ - **Monitoring dashboard.** `jev-codex --dashboard` (or `jev-claude --dashboard`) opens a local page
13
+ that shows whether each gateway is routing, passing traffic through, idle, or failing, and why.
14
+ One page covers both gateways and updates live. It shows request metadata only, never prompts,
15
+ tool arguments, or credentials.
16
+ - **Token metering.** Every forwarded request records what the provider says it cost: input tokens
17
+ (and how many came from the prompt cache), output tokens (and how many were reasoning), and
18
+ duration. Works for Chat Completions, the Responses API, and Anthropic Messages.
19
+ - **Baseline mode.** `--routing off` stops asking Jev but keeps metering, so the dashboard can show
20
+ token use with and without Jev side by side. Also a button on the dashboard and `JEV_ROUTING=off`.
21
+ - `HOST` setting for the interface to listen on.
22
+
23
+ ### Changed
24
+ - **The gateway now listens on `127.0.0.1` only.** 0.1.0 listened on every interface, which made a
25
+ gateway that forwards your credentials reachable from the local network. Upgrade.
26
+ - Launcher flags lost their `jev-` prefix: `--dashboard`, `--routing`, `--status`, `--logs`,
27
+ `--start`, `--stop`. The gateway's own help and config are `--gateway-help` and `--print-config`,
28
+ because `--help` and `--config` belong to Codex and Claude Code. The old `--jev-*` spellings still
29
+ work.
30
+ - Requests are logged when their reply ends instead of when it starts, since that is when token
31
+ counts are known.
32
+ - Requires Node.js 22.15 or newer.
33
+ - README rewritten around getting started.
34
+
35
+ ### Fixed
36
+ - `--stop` now stops whichever gateway answers on the port, including one started by an older
37
+ version that recorded its pid elsewhere.
38
+
39
+ ## 0.1.0
40
+
41
+ First release: tool selection routed to Jev for Chat Completions, the Responses API, and Anthropic
42
+ Messages, with the `jev-codex` and `jev-claude` launchers.
package/README.md CHANGED
@@ -1,85 +1,178 @@
1
1
  # jev-gateway
2
2
 
3
- > Independent project not affiliated with or endorsed by TypeSafe. "Jev" is TypeSafe's model; this
4
- > gateway is a client of its public API.
3
+ A local LLM gateway for coding agents. When your agent is about to decide **which tool to call**,
4
+ the gateway asks [Jev](https://docs.typesafe.ai/introduction), TypeSafe's fast decision model,
5
+ instead of leaving that choice to the expensive reasoning model. Everything else goes to your usual
6
+ LLM untouched.
5
7
 
6
- An LLM gateway that speaks Chat Completions, the Responses API and the Anthropic Messages API.
7
- Point your client — or Codex, or Claude Code at it instead of your LLM provider; whenever a
8
- request is really asking **"which tool should I call?"**, the router hands that decision to
9
- [Jev](https://docs.typesafe.ai/introduction) — TypeSafe's System One model — instead of paying a
10
- reasoning LLM to make it.
8
+ It works with **Codex** and **Claude Code** out of the box, including on ChatGPT and claude.ai
9
+ subscriptions, and with any client that speaks the OpenAI or Anthropic APIs.
11
10
 
12
- Jev doesn't generate text. It answers typed questions (Choice / Score / Noul) about a piece of
13
- state and returns calibrated probabilities plus a confidence, in one fast call. Tool selection is
14
- exactly that kind of question, so the split is:
11
+ > Independent project, not affiliated with or endorsed by TypeSafe. "Jev" is TypeSafe's model and
12
+ > this gateway is a client of its public API.
15
13
 
16
- | Decision | Who makes it |
17
- | --- | --- |
18
- | Which tool, or no tool at all | **Jev** (Choice over the tool list + a Noul cross-check) |
19
- | Arguments that are enums / booleans / consts | **Jev** (fanned out in the same call) |
20
- | Open-ended arguments (free text, numbers, dates) | Upstream LLM, with `tool_choice` forced to Jev's pick |
21
- | Plain text replies, everything without `tools` | Upstream LLM, untouched |
14
+ ## Quick start
22
15
 
23
- ## How a request is routed
16
+ You need Node.js 22.15 or newer, a [TypeSafe API key](https://docs.typesafe.ai/introduction), and
17
+ Codex and/or Claude Code already installed and logged in.
24
18
 
25
- `POST /v1/chat/completions` with `tools` and `tool_choice` of `auto`/`required` triggers **one**
26
- Jev call. The conversation becomes the state; the questions are:
19
+ **1. Install**
27
20
 
28
- - `tool` — Choice: every tool name → its description, plus `no_tool_needed` (omitted for `required`)
29
- - `needs_tool` Noul: an independent "does the assistant need a tool now?" check
30
- - `arg:*` / `stated:*` — for each tool whose parameters are *all* closed-set, one question per
31
- argument (and "was it stated?" for optional ones), asked speculatively since extra questions are
32
- nearly free
21
+ ```bash
22
+ npm install -g jev-gateway
23
+ ```
33
24
 
34
- The answer picks one of five modes, reported in the `x-jev-gateway-mode` response header:
25
+ **2. Save your TypeSafe key**
35
26
 
36
- | Mode | When | What happens |
37
- | --- | --- | --- |
38
- | `direct` | Tool is confident and every argument is closed-set and certain | The router synthesizes the `tool_calls` response itself (streaming included). **No LLM call.** |
39
- | `forced` | Tool is confident, arguments need an LLM | Forwarded with `tool_choice` set to that function — and to `ARGS_MODEL` if configured, since the hard part is already done |
40
- | `hint` | Tool is confident, but `tool_choice` can't be rewritten (Anthropic: extended thinking on, or the conversation is prompt-cached) | Forwarded with a one-line suggestion appended *after* the client's last block, so cached prefixes stay intact. The LLM may disagree |
41
- | `none` | Jev is confident no tool is needed | Forwarded with `tool_choice: "none"` (or untouched with `JEV_ON_NONE=passthrough`) |
42
- | `passthrough` | Low confidence, the two questions disagree, Jev errored/timed out, no tools, caller already chose | Forwarded byte-for-byte; `x-jev-gateway-reason` says why |
27
+ ```bash
28
+ mkdir -p ~/.jev-gateway
29
+ echo "TYPESAFE_API_KEY=your-key-here" > ~/.jev-gateway/.env
30
+ ```
43
31
 
44
- Rosters over 120 tools (Claude Code sends ~280) don't fit one good question, so they take two Jev
45
- calls: every shard of the roster is ranked in one call, and the top 3 of each go on to the decision
46
- above with full-length descriptions.
32
+ **3. Run your agent through the gateway**
47
33
 
48
- The router always **fails open**: any Jev problem means the LLM decides, as if the gateway weren't
49
- there. All other `/v1/*` routes (models, embeddings, …) are proxied unchanged.
34
+ ```bash
35
+ jev-codex # use it exactly like `codex`
36
+ jev-claude # use it exactly like `claude`
37
+ ```
50
38
 
51
- ## Install
39
+ **4. Watch it work**
52
40
 
53
41
  ```bash
54
- npm install -g jev-gateway
55
- mkdir -p ~/.jev-gateway && echo "TYPESAFE_API_KEY=…" > ~/.jev-gateway/.env
56
- jev-codex # or: jev-claude
42
+ jev-codex --dashboard
43
+ ```
44
+
45
+ That's it. Your existing login keeps working, nothing in `~/.codex` or `~/.claude` is changed, and
46
+ plain `codex` and `claude` still behave as before. Only sessions started with the `jev-` commands go
47
+ through the gateway.
48
+
49
+ ## What to expect
50
+
51
+ - The first `jev-codex` or `jev-claude` starts a small gateway in the background and then opens your
52
+ agent. Every argument is passed through, so `jev-codex exec "fix the failing test"` works like
53
+ `codex exec "fix the failing test"`.
54
+ - The gateway keeps running after you close the agent, so the next session starts instantly. Stop it
55
+ with `--stop`.
56
+ - Each turn, the gateway asks Jev which tool fits. When Jev is confident, the gateway steers the LLM
57
+ to that tool. When it is not, the request goes through unchanged.
58
+ - If Jev is down, slow, or your key is wrong, every request simply goes straight to the LLM. The
59
+ gateway never makes a request fail.
60
+ - It listens on `127.0.0.1` only.
61
+
62
+ ## Commands
63
+
64
+ All of these work with both `jev-codex` and `jev-claude`.
65
+
66
+ | Command | What it does |
67
+ | --- | --- |
68
+ | `jev-codex [args]` | Start the gateway if needed, then run Codex through it |
69
+ | `jev-codex --dashboard` | Open the monitoring dashboard in your browser |
70
+ | `jev-codex --routing off` | Baseline mode: stop asking Jev, keep counting tokens |
71
+ | `jev-codex --routing on` | Let Jev decide again |
72
+ | `jev-codex --status` | Is the gateway running, and where does it forward to? |
73
+ | `jev-codex --logs` | Follow routing decisions live (use a second terminal) |
74
+ | `jev-codex --start` | Start the gateway without opening the agent |
75
+ | `jev-codex --stop` | Stop the background gateway (close your sessions first) |
76
+ | `jev-codex --print-config` | Print settings to point plain `codex` at the gateway permanently |
77
+ | `jev-codex --gateway-help` | List all of the above |
78
+
79
+ Codex uses port 8790 and Claude Code uses port 8789. Change them with `JEV_CODEX_PORT` and
80
+ `JEV_CLAUDE_PORT`.
81
+
82
+ ## Dashboard
83
+
84
+ ```bash
85
+ jev-codex --dashboard # or: jev-claude --dashboard
57
86
  ```
58
87
 
59
- That is all the two launchers need (details below). To run the gateway as a standalone server for
60
- your own clients, work from a checkout:
88
+ This opens `http://localhost:8790/dashboard`. If no browser window appears, paste that address into
89
+ your browser. One page shows both gateways (Codex and Claude) and refreshes every 2 seconds.
90
+
91
+ You will see:
92
+
93
+ - **A status per gateway:** Routing, Passthrough only, Jev is failing, Idle, Baseline, or Offline,
94
+ with a one-line explanation.
95
+ - **Why requests were not routed,** with each reason explained in plain English.
96
+ - **Jev's numbers:** calls, latency, confidence, and what it cost.
97
+ - **LLM tokens:** input (and how much came from the prompt cache), output (and how much was hidden
98
+ reasoning), and seconds per request.
99
+ - **A live table** of recent requests. A request appears when its reply finishes, because that is
100
+ when the provider reports its tokens.
61
101
 
62
- ## Run it
102
+ The dashboard only shows request metadata. Prompts, tool arguments, and credentials never reach it.
103
+
104
+ ### Is it worth it? Compare with a baseline
105
+
106
+ Switch routing off to measure the same work without Jev. The gateway keeps forwarding and counting
107
+ tokens, but never asks Jev and rewrites nothing.
108
+
109
+ ```bash
110
+ jev-codex --routing off # do a task
111
+ jev-codex --routing on # do a similar task
112
+ ```
113
+
114
+ The same switch is a button on each gateway card. The "Token use" card then shows both states side
115
+ by side: tokens in and out per request, cache share, reasoning tokens, and seconds. The comparison
116
+ is only meaningful if you do similar work in both states.
117
+
118
+ ## Using it with Codex
119
+
120
+ `jev-codex` reuses your existing Codex login. With a ChatGPT subscription the gateway forwards to
121
+ `https://chatgpt.com/backend-api/codex`. With an API key it forwards to `https://api.openai.com/v1`.
122
+ Override either with `JEV_CODEX_UPSTREAM_BASE_URL`.
123
+
124
+ Codex speaks the Responses API, so the gateway handles `POST /v1/responses`, including Codex's
125
+ free-form tools such as `apply_patch`, tools declared inside the conversation, and compressed
126
+ request bodies. If the backend rejects a rewritten request, the gateway resends the original, so
127
+ Codex never sees an error caused by the gateway.
128
+
129
+ ## Using it with Claude Code
130
+
131
+ `jev-claude` runs `claude` with only `ANTHROPIC_BASE_URL` set. Claude Code keeps using its saved
132
+ login, so a claude.ai subscription keeps working and its usual limits apply.
133
+
134
+ Jev can do less here than with Codex, because of how the Anthropic API works. Claude Code runs with
135
+ extended thinking, and the API rejects a forced tool while thinking is on. It also rereads a cached
136
+ conversation on every turn, and changing `tool_choice` would invalidate that cache. So for Claude
137
+ Code the gateway adds a short suggestion to the request instead (`hint` mode), which the model is
138
+ free to ignore. Expect better tool picks on large tool lists, not lower cost or latency.
139
+
140
+ ## Running it as a server for your own app
141
+
142
+ Work from a checkout:
63
143
 
64
144
  ```bash
65
145
  pnpm install
66
- cp .env.example .env # set TYPESAFE_API_KEY, and UPSTREAM_BASE_URL if not OpenAI
67
- pnpm dev
146
+ cp .env.example .env # set TYPESAFE_API_KEY, and UPSTREAM_BASE_URL if you don't use OpenAI
147
+ pnpm dev # listens on http://localhost:8787
68
148
  ```
69
149
 
150
+ Then point your client at it:
151
+
70
152
  ```python
71
153
  from openai import OpenAI
72
154
  client = OpenAI(base_url="http://localhost:8787/v1") # your usual provider key still works
73
155
  ```
74
156
 
75
- By default the client's own `Authorization` header is forwarded upstream. Set `UPSTREAM_API_KEY`
76
- to have the gateway hold the provider key, and `ROUTER_API_KEY` to require a gateway key from
77
- clients. Any OpenAI-compatible upstream works (OpenAI, OpenRouter, vLLM, Ollama, LiteLLM, …).
157
+ The gateway routes three endpoints and proxies every other `/v1/*` path unchanged:
158
+
159
+ | Endpoint | API |
160
+ | --- | --- |
161
+ | `POST /v1/chat/completions` | OpenAI Chat Completions |
162
+ | `POST /v1/responses` | OpenAI Responses |
163
+ | `POST /v1/messages` | Anthropic Messages |
164
+
165
+ By default your client's own `Authorization` header is forwarded to the provider. Set
166
+ `UPSTREAM_API_KEY` to have the gateway hold the provider key instead, and `ROUTER_API_KEY` to
167
+ require a gateway key from clients. Any OpenAI-compatible provider works, for example OpenAI,
168
+ OpenRouter, vLLM, Ollama, or LiteLLM.
78
169
 
79
- ### Try a decision without an upstream
170
+ To skip Jev for a single request, send the header `x-jev-gateway: off`.
80
171
 
81
- `POST /router/decide` takes a chat.completions body, calls Jev, and returns the decision — mode,
82
- tool, arguments, Jev's confidence, top probabilities, tokens and latency — without calling the LLM:
172
+ ### Try a decision without calling any LLM
173
+
174
+ `POST /router/decide` takes a request body, asks Jev, and returns the decision: the mode, the tool,
175
+ the arguments, Jev's confidence, and its latency.
83
176
 
84
177
  ```bash
85
178
  curl -s localhost:8787/router/decide -H 'content-type: application/json' -d '{
@@ -93,87 +186,95 @@ curl -s localhost:8787/router/decide -H 'content-type: application/json' -d '{
93
186
  }'
94
187
  ```
95
188
 
96
- Send `x-jev-gateway: off` on any request to bypass Jev for that call.
97
-
98
- ## Use it with Codex (local)
99
-
100
- Codex only speaks the Responses API, so the router handles `POST /v1/responses` the same way as
101
- chat completions — including Codex's free-form tools (`apply_patch`), provider-run tools
102
- (`web_search`, offered to Jev but never forced) and zstd-compressed request bodies.
103
-
104
- ```bash
105
- jev-codex # instead of `codex`; every codex argument still works
106
- jev-codex exec "fix the failing test"
107
- jev-codex --jev-logs # second terminal: watch each routing decision live
108
- ```
189
+ ## How it works
109
190
 
110
- `jev-codex` starts a background router on `127.0.0.1:8790` if one isn't running, then launches
111
- `codex` with a `-c model_providers.jev-gateway…` override. **Nothing in `~/.codex` is modified**, and
112
- plain `codex` keeps working as before. It reuses your existing Codex login
113
- (`requires_openai_auth = true`): with a ChatGPT subscription the router forwards to
114
- `https://chatgpt.com/backend-api/codex`, with an API key to `https://api.openai.com/v1`
115
- (override with `JEV_CODEX_UPSTREAM_BASE_URL`). `TYPESAFE_API_KEY` is read from the environment,
116
- `~/.jev-gateway/.env`, or a checkout's own `.env`. `jev-codex --jev-help` lists the rest (`--jev-status`, `--jev-stop`,
117
- `--jev-config` for a permanent `codex --profile jev`).
191
+ Jev does not generate text. It answers typed questions about a piece of state (pick one option,
192
+ give a score, or yes/no) and returns calibrated probabilities with a confidence, in one fast call.
193
+ Choosing a tool is exactly that kind of question, so the work is split like this:
118
194
 
119
- If the upstream rejects a rewritten request (HTTP 400/422 — some backends only accept
120
- `tool_choice: "auto"`), the router replays the original, so Codex never sees a router-caused error.
121
-
122
- ## Use it with Claude Code (local)
195
+ | Decision | Who makes it |
196
+ | --- | --- |
197
+ | Which tool, or no tool at all | **Jev** |
198
+ | Arguments that are enums, booleans, or constants | **Jev**, in the same call |
199
+ | Open-ended arguments such as free text, numbers, and dates | The LLM, already pointed at Jev's tool |
200
+ | Plain text replies, and any request without tools | The LLM, untouched |
123
201
 
124
- ```bash
125
- jev-claude # instead of `claude`; every claude argument still works
126
- jev-claude -p "summarise this repo"
127
- jev-claude --jev-logs # second terminal: watch each routing decision live
128
- ```
202
+ A request that carries tools triggers one Jev call. The conversation becomes the state, and the
203
+ questions are: which tool (or none), whether a tool is needed at all (an independent cross-check),
204
+ and the value of every closed-set argument. The answer selects a mode, which is reported in the
205
+ `x-jev-gateway-mode` response header:
129
206
 
130
- `jev-claude` starts a background router on `127.0.0.1:8789` forwarding to `https://api.anthropic.com/v1`
131
- and runs `claude` with only `ANTHROPIC_BASE_URL` set. With no gateway credential alongside it, Claude
132
- Code keeps using its saved login, so a **claude.ai subscription keeps working** and its limits apply
133
- as usual; **nothing in `~/.claude` is modified**. The same `--jev-*` flags as `jev-codex` apply.
207
+ | Mode | When | What happens |
208
+ | --- | --- | --- |
209
+ | `direct` | Jev is confident about the tool and every argument is an enum, boolean, or constant | The gateway builds the tool call itself, streaming included. **No LLM call.** |
210
+ | `forced` | Jev is confident about the tool, but some arguments are open-ended | Forwarded with `tool_choice` set to that tool, so the LLM only fills in arguments. `ARGS_MODEL` can send these to a cheaper model |
211
+ | `hint` | Jev is confident, but `tool_choice` cannot be changed (Anthropic with thinking on, or a cached conversation) | Forwarded with a one-line suggestion added after the client's last block, so cached prefixes stay valid |
212
+ | `none` | Jev is confident that no tool is needed | Forwarded with `tool_choice: "none"` |
213
+ | `passthrough` | Low confidence, the two checks disagree, Jev failed, there are no tools, or the caller already chose | Forwarded byte for byte. `x-jev-gateway-reason` says why |
134
214
 
135
- What Jev can do here is narrower than with Codex, by design of the API rather than the router:
136
- Claude Code runs with adaptive thinking (a forced `tool_choice` is rejected) and re-reads a cached
137
- conversation every turn (any `tool_choice` change would invalidate it). So Claude Code requests are
138
- steered with `hint` mode, `none` is never applied, and `direct` still answers without the LLM when
139
- a tool's arguments are all closed-set. API callers without thinking or message caching get `forced`.
215
+ Tool lists longer than 120 entries (Claude Code sends about 280) take two Jev calls. The first ranks
216
+ the list in groups. The second decides among the top 3 of each group, using full descriptions.
140
217
 
141
218
  ## Configuration
142
219
 
143
- See [.env.example](.env.example). The ones worth tuning:
144
-
145
- - `JEV_MIN_CONFIDENCE` (0.7) below this the LLM decides. Raise it to be more conservative.
146
- - `JEV_ARG_MIN_CERTAINTY` (0.8) — the weakest argument must clear this for a `direct` answer;
147
- otherwise the request degrades to `forced`.
148
- - `ARGS_MODEL` — a cheap model for argument filling in `forced` mode.
149
- - `JEV_DIRECT_CALLS=false` — never answer without the LLM; Jev only picks the tool.
220
+ Settings are environment variables. The launchers read them from your shell,
221
+ `~/.jev-gateway/.env`, or a checkout's own `.env`. See [.env.example](.env.example) for the full
222
+ list. The ones worth knowing:
150
223
 
151
- Each routed request logs one JSON line (mode, reason, Jev's choice/confidence/latency) to stdout.
224
+ | Variable | Default | Meaning |
225
+ | --- | --- | --- |
226
+ | `TYPESAFE_API_KEY` | required | Your TypeSafe key |
227
+ | `JEV_MIN_CONFIDENCE` | `0.7` | Below this confidence, the LLM decides. Lower it to route more, raise it to be more careful |
228
+ | `JEV_ARG_MIN_CERTAINTY` | `0.8` | Every argument must reach this for a `direct` answer |
229
+ | `JEV_DIRECT_CALLS` | `true` | Set to `false` so the gateway never answers without the LLM |
230
+ | `JEV_ROUTING` | `on` | Set to `off` to start in baseline mode |
231
+ | `JEV_TIMEOUT_MS` | `4000` | How long to wait for Jev before letting the LLM decide |
232
+ | `ARGS_MODEL` | unset | A cheaper model for filling arguments in `forced` mode |
233
+ | `HOST` | `127.0.0.1` | Interface to listen on. Set `ROUTER_API_KEY` before exposing it |
234
+ | `JEV_DEBUG_DUMP_DIR` | unset | Write requests and response summaries to this folder, with credentials redacted |
235
+
236
+ Each request also logs one JSON line to stdout, or to `~/.jev-gateway/<client>.log` under a launcher.
152
237
 
153
238
  ## Known trade-offs
154
239
 
240
+ - Jev adds a network call to every turn that carries tools. Expect roughly half a second to a second.
155
241
  - Jev picks **one** tool per turn. In `forced` mode the LLM can still call that tool several times
156
- in parallel, but not mix different tools in one turn; `direct` mode emits exactly one call.
157
- - Jev is text-only with a 32k-token state budget: images become `[image_url]` placeholders and long
158
- conversations keep their newest turns (`JEV_MAX_STATE_CHARS`). It is most accurate in English.
159
- - A hint is a suggestion, not a decision: in `hint` mode the LLM still spends its own reasoning on
160
- the choice, so the gain is accuracy on large rosters, not latency or cost.
161
- - Validated end to end on subscriptions (Codex 0.154 on ChatGPT, Claude Code 2.1 on claude.ai) with
162
- `scripts/mock-jev.mjs` standing in for Jev: `forced`/`none` are accepted by the ChatGPT Codex
163
- backend, `hint` by Anthropic. Jev's real accuracy on these rosters, and the confidence thresholds,
164
- still need tuning against a real `TYPESAFE_API_KEY`.
165
-
166
- ## Layout
242
+ in parallel, but it cannot mix different tools in the same turn.
243
+ - A wrong forced tool can derail a turn. If the model had nothing left to do and is forced to call
244
+ a tool anyway, it may produce an incomplete reply and the agent will retry. Raise
245
+ `JEV_MIN_CONFIDENCE` if you see this.
246
+ - In `hint` mode the LLM still does its own reasoning, so the gain is accuracy, not cost.
247
+ - Jev reads text only and has a 32k-token window. Images become placeholders and long conversations
248
+ keep their newest turns. It is most accurate in English.
249
+ - The default confidence thresholds are starting points. Use the dashboard and baseline mode to tune
250
+ them for your own work.
251
+
252
+ ## Development
167
253
 
254
+ ```bash
255
+ pnpm install
256
+ pnpm test # runs against fake Jev and provider transports, no keys needed
257
+ pnpm typecheck
258
+ pnpm build
168
259
  ```
169
- src/adapters/ wire formats ↔ neutral shapes: chat.ts, responses.ts (Codex), messages.ts (Claude Code)
170
- src/state.ts conversation Jev state (truncation, newest-turns budget)
171
- src/questions.ts tools Jev questions; detects closed-set parameters
172
- src/decide.ts the Jev call and the mode decision
173
- src/upstream.ts streaming reverse proxy
174
- src/app.ts Hono app: routes, auth, headers, fail-open replay
175
- bin/ jev-codex / jev-claude launchers (shared logic in launcher.mjs)
176
- scripts/mock-jev.mjs local stand-in for Jev, for end-to-end runs without a TypeSafe key
260
+
261
+ `scripts/mock-jev.mjs` is a local stand-in for Jev. Point `TYPESAFE_BASE_URL` at it to drive a real
262
+ agent end to end without a TypeSafe key.
263
+
264
+ ```
265
+ src/adapters/ request formats: chat.ts, responses.ts (Codex), messages.ts (Claude Code)
266
+ src/state.ts turns a conversation into Jev state
267
+ src/questions.ts turns tools into Jev questions and finds closed-set arguments
268
+ src/decide.ts the Jev call and the mode decision
269
+ src/upstream.ts streaming reverse proxy
270
+ src/usage.ts token usage read from a reply, normalised across providers
271
+ src/app.ts routes, auth, headers, and the resend-on-rejection fallback
272
+ src/events.ts recent request metadata kept in memory and restored from the log
273
+ src/dashboard.ts serves /dashboard (dashboard.html is the whole page, no build step)
274
+ bin/ jev-codex and jev-claude launchers (shared logic in launcher.mjs)
275
+ scripts/mock-jev.mjs local stand-in for Jev
177
276
  ```
178
277
 
179
- `pnpm test` runs the suite against fake Jev and upstream transports; no keys needed.
278
+ ## License
279
+
280
+ MIT
@@ -13,7 +13,7 @@ await runLauncher({
13
13
  // saved claude.ai login, so a Pro/Max subscription (or an existing API key) keeps working as is.
14
14
  env: (origin) => ({ ANTHROPIC_BASE_URL: origin }),
15
15
  configHelp: (origin) =>
16
- `# Keep the router running (jev-claude --jev-start), then either:\n` +
16
+ `# Keep the router running (jev-claude --start), then either:\n` +
17
17
  `# ANTHROPIC_BASE_URL=${origin} claude\n` +
18
18
  `# or add to ~/.claude/settings.json:\n` +
19
19
  JSON.stringify({ env: { ANTHROPIC_BASE_URL: origin } }, null, 2),
package/bin/jev-codex.mjs CHANGED
@@ -44,7 +44,7 @@ await runLauncher({
44
44
  ...Object.entries(provider(origin)).flatMap(([key, value]) => ["-c", `model_providers.jev-gateway.${key}=${value}`]),
45
45
  ],
46
46
  configHelp: (origin) =>
47
- `# Save as ~/.codex/jev.config.toml, keep the router running (jev-codex --jev-start),\n` +
47
+ `# Save as ~/.codex/jev.config.toml, keep the router running (jev-codex --start),\n` +
48
48
  `# then use: codex --profile jev\n` +
49
49
  `model_provider = "jev-gateway"\n\n[model_providers.jev-gateway]\n` +
50
50
  Object.entries(provider(origin))