llmspy 0.0.0-stage → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +369 -2
- package/llms.txt +189 -0
- package/package.json +30 -5
- package/proxy/dist/capture.js +230 -0
- package/proxy/dist/cli.js +42 -0
- package/proxy/dist/command.js +50 -0
- package/proxy/dist/conversation.js +102 -0
- package/proxy/dist/envfile.js +22 -0
- package/proxy/dist/failed.js +11 -0
- package/proxy/dist/guard.js +45 -0
- package/proxy/dist/index.js +81 -0
- package/proxy/dist/parsers/anthropic.js +197 -0
- package/proxy/dist/parsers/error.js +17 -0
- package/proxy/dist/parsers/index.js +98 -0
- package/proxy/dist/parsers/json.js +31 -0
- package/proxy/dist/parsers/openai-chat.js +222 -0
- package/proxy/dist/parsers/openai-common.js +18 -0
- package/proxy/dist/parsers/openai-responses.js +279 -0
- package/proxy/dist/parsers/sse.js +68 -0
- package/proxy/dist/parsers/types.js +1 -0
- package/proxy/dist/pidfile.js +57 -0
- package/proxy/dist/proxy.js +337 -0
- package/proxy/dist/query.js +111 -0
- package/proxy/dist/static.js +61 -0
- package/proxy/dist/stats.js +71 -0
- package/proxy/dist/stop.js +68 -0
- package/proxy/dist/store.js +201 -0
- package/proxy/dist/summary.js +50 -0
- package/proxy/dist/types.js +1 -0
- package/proxy/dist/upstreams.js +85 -0
- package/proxy/dist/version.js +14 -0
- package/proxy/package.json +6 -0
- package/ui/dist/assets/index-CyJmqpbn.js +23 -0
- package/ui/dist/assets/index-moM-ONoJ.css +1 -0
- package/ui/dist/index.html +21 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 piyook
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,370 @@
|
|
|
1
|
-
#
|
|
1
|
+
# LLMSpy
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
[](https://github.com/piyook/llmspy/releases)
|
|
4
|
+
[](https://www.npmjs.com/package/llmspy)
|
|
5
|
+
[](https://github.com/piyook/llmspy/actions/workflows/checks.yaml)
|
|
6
|
+
[](LICENSE)
|
|
7
|
+
[](https://nodejs.org/)
|
|
8
|
+
[](https://www.typescriptlang.org/)
|
|
9
|
+
|
|
10
|
+
A lightweight local developer tool that shows LLM API traffic in real time, with next to no setup: to you in a web UI, and to coding agents through a small local API.
|
|
11
|
+
|
|
12
|
+
- **Live capture**: every call to Anthropic, OpenAI or any other upstream you add shows up as soon as it is sent and is filled in when the reply ends; streamed replies still stream to the app
|
|
13
|
+
- **Conversation view**: system prompt, tools, messages, tool calls matched to their results, and the reply with its stop reason and token use
|
|
14
|
+
- **Raw view**: headers and bodies exactly as sent and received, with API keys masked
|
|
15
|
+
- **Compare**: diff a call against the one before it, or any other, to see what an agent changed between turns
|
|
16
|
+
- **Projects**: tag each app's base URL to give its calls their own tab
|
|
17
|
+
- **Three formats**: Anthropic Messages, OpenAI Chat Completions and OpenAI Responses, streamed or not
|
|
18
|
+
- **Kept across restarts**: the last 7 days of calls, in one file on your machine, listed by day with older days folded away; one click narrows today's calls to the last 5 to 90 minutes, another to the failed ones
|
|
19
|
+
- **Agent friendly**: coding agents and scripts can list, search and read the captured calls as JSON, or one call as Markdown, count their tokens, failures and latency, and wait for a running call to end, all on the same port; [llms.txt](llms.txt) tells them how
|
|
20
|
+
- **Lightweight**: one command to start, one base URL to set in your app; no certificates or accounts
|
|
21
|
+
- **No runtime dependencies**: the npm package installs nothing else; the proxy uses only Node.js's own modules, and the UI's two libraries (Svelte and jsdiff) are built into its files
|
|
22
|
+
|
|
23
|
+

|
|
24
|
+
|
|
25
|
+
## Contents
|
|
26
|
+
|
|
27
|
+
- [Why](#why)
|
|
28
|
+
- [What you see](#what-you-see)
|
|
29
|
+
- [Quick start](#quick-start)
|
|
30
|
+
- [Start LLMSpy](#start-llmspy)
|
|
31
|
+
- [Point an app at it](#point-an-app-at-it)
|
|
32
|
+
- [Keep apps apart with projects](#keep-apps-apart-with-projects)
|
|
33
|
+
- [Commands](#commands)
|
|
34
|
+
- [Upstreams](#upstreams)
|
|
35
|
+
- [Adding an upstream](#adding-an-upstream)
|
|
36
|
+
- [Setting upstreams for a local run](#setting-upstreams-for-a-local-run)
|
|
37
|
+
- [Setting upstreams in Docker](#setting-upstreams-in-docker)
|
|
38
|
+
- [Checking your upstreams](#checking-your-upstreams)
|
|
39
|
+
- [Apps without a base-URL setting](#apps-without-a-base-url-setting)
|
|
40
|
+
- [Configuration](#configuration)
|
|
41
|
+
- [Storage](#storage)
|
|
42
|
+
- [Testing with mock data](#testing-with-mock-data)
|
|
43
|
+
- [How it works](#how-it-works)
|
|
44
|
+
- [Troubleshooting](#troubleshooting)
|
|
45
|
+
- [License](#license)
|
|
46
|
+
|
|
47
|
+
## Why
|
|
48
|
+
|
|
49
|
+
Apps and agents built on LLMs hide most of what they send. SDKs, frameworks and coding agents add their own system prompts, tool definitions, retries and parameters, and the actual request and response are rarely logged anywhere you can read them. When an agent misbehaves, burns tokens or gives a strange answer, the first question is "what did it actually send to the model, and what came back?", and that is usually the hardest one to answer.
|
|
50
|
+
|
|
51
|
+
LLMSpy answers it without changing any code. It runs on your machine as a small reverse proxy: point an app's base URL at it (most LLM SDKs and tools read one from an environment variable, such as `ANTHROPIC_BASE_URL` or `OPENAI_BASE_URL`), and every call shows up live in a web UI with the full headers, prompt, parameters, response, status and timing. Streamed responses still stream to the app; LLMSpy keeps a copy as they pass.
|
|
52
|
+
|
|
53
|
+
Use it to:
|
|
54
|
+
|
|
55
|
+
- see the real prompts, system messages and tool schemas a library or agent sends
|
|
56
|
+
- debug failing or odd requests: errors, status codes, latency
|
|
57
|
+
- understand token-heavy calls and why they cost what they do
|
|
58
|
+
- compare what different tools send for the same task
|
|
59
|
+
- let a coding agent check what an app it is building or debugging really sent and got back, without you copying anything across
|
|
60
|
+
|
|
61
|
+
It is deliberately small, and quick to set up:
|
|
62
|
+
|
|
63
|
+
- one command starts it, and one environment variable in your app points the app at it
|
|
64
|
+
- no certificates to install, no accounts, no API keys of its own and no config file
|
|
65
|
+
- the proxy has no runtime dependencies beyond Node.js (it stores calls with the SQLite built into Node), and the UI is plain static files it serves itself, on the same port
|
|
66
|
+
|
|
67
|
+
It is for local development only. It listens on `127.0.0.1`, masks API keys in what it records, and keeps the last 7 days of calls in a file on your machine (see [Storage](#storage)).
|
|
68
|
+
|
|
69
|
+
## What you see
|
|
70
|
+
|
|
71
|
+
Every call appears in the list on the left as soon as it is sent, newest first, with its status, time taken and token counts. A call still running is marked `waiting` or `replying` with a running timer; its reply shows once it ends.
|
|
72
|
+
|
|
73
|
+
The list is grouped by day, with older days folded away. The chips under Today show only the calls from the last 5 to 90 minutes, such as one run of an agent, and the Today heading counts how many are shown.
|
|
74
|
+
|
|
75
|
+
<img src="docs/screenshots/days.png" width="300" alt="The call list narrowed to the last 15 minutes: Today shows 3 of 15 calls, with Yesterday and Thu 12 Nov folded below">
|
|
76
|
+
|
|
77
|
+
When a project has calls that went wrong, a red **Failed** chip above the list counts them; press it to see only those, on every day. A call has failed when the upstream could not be reached or its reply broke off, when its status is 400 or above, or when the provider sent an error inside the reply. Agents get the same calls from `GET /api/requests?failed=1`.
|
|
78
|
+
|
|
79
|
+
<img src="docs/screenshots/failed.png" width="300" alt="The Failed chip pressed: the list shows only the failed calls, two 429s, two errors in the stream and two dropped streams">
|
|
80
|
+
|
|
81
|
+
Cards above the panes sum up the calls in the list as it is narrowed (by project, search, the Failed chip and the minutes chips): how many there are and how many are running, how many failed, their tokens in, out and cached, and their p50 and p95 latency. They update live, and agree with `GET /api/stats`.
|
|
82
|
+
|
|
83
|
+
<img src="docs/screenshots/cards.png" alt="The cards above the panes for the general project: 54 calls, 18 failed (33%, in red), 990 tokens in and 996 out, latency p50 16ms and p95 801ms">
|
|
84
|
+
|
|
85
|
+
Select a call to look at it in three ways.
|
|
86
|
+
|
|
87
|
+
**Conversation** (shown at the top) reads the call as the model sees it: the system prompt and tools, then each message by role, with tool calls matched to their results, and the reply at the end with its stop reason and token use. Streamed replies are put back together.
|
|
88
|
+
|
|
89
|
+
**Raw** shows the request and response exactly as they went over the wire: method and URL, headers (with API keys masked) and both bodies.
|
|
90
|
+
|
|
91
|
+

|
|
92
|
+
|
|
93
|
+
**Compare** sets the selected call (green) against another (red): by default the previous call to the same upstream, which in an agent loop is the turn before; "Compare with another call" picks any other. It sums up what changed (model, stop reason, system prompt, tools, messages and tokens) and shows only the messages that are new.
|
|
94
|
+
|
|
95
|
+

|
|
96
|
+
|
|
97
|
+
Under the summary, a line diff of the two request bodies shows everything else that changed, such as `max_tokens`, `temperature` or a tool's schema, with unchanged stretches folded away.
|
|
98
|
+
|
|
99
|
+

|
|
100
|
+
|
|
101
|
+
The traffic in these screenshots is made up, sent to the mock servers described under [Testing with mock data](#testing-with-mock-data).
|
|
102
|
+
|
|
103
|
+
## Quick start
|
|
104
|
+
|
|
105
|
+
### Start LLMSpy
|
|
106
|
+
|
|
107
|
+
With **Node.js 26 or later** (`node --version`), from any folder:
|
|
108
|
+
|
|
109
|
+
```bash
|
|
110
|
+
npx llmspy
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
This serves the proxy and the UI on `http://127.0.0.1:8080`, in the foreground. Stop it with Ctrl+C, or with `npx llmspy stop` from any terminal. To keep the `llmspy` command to hand, install it once with `npm install -g llmspy`; `llmspy --help` lists its commands and settings. `npx` runs the version it has cached, and `npx llmspy@latest` the newest.
|
|
114
|
+
|
|
115
|
+
With Docker, from a clone of the repo:
|
|
116
|
+
|
|
117
|
+
```bash
|
|
118
|
+
npm run start:docker
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
LLMSpy then runs in the background, restarts with Docker, and serves the same proxy and UI on `http://127.0.0.1:8080`. `npm run stop:docker` stops it. Captured calls are kept in the `llmspy-data` volume, so they are there again at the next start.
|
|
122
|
+
|
|
123
|
+
From source, in a clone of the repo, with Node.js 26 or later:
|
|
124
|
+
|
|
125
|
+
```bash
|
|
126
|
+
npm install
|
|
127
|
+
npm run build
|
|
128
|
+
npm start
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
Stop it with Ctrl+C, or with `npm run stop` from any terminal. Use one way at a time, as they all use port 8080.
|
|
132
|
+
|
|
133
|
+
Whichever way you run it, the port is open on the loopback interface only, so nothing outside your machine can reach it. Captured traffic (`/api/*`, `/events`, `/ping`) is only served to requests addressed to `localhost`, `127.0.0.1` or `[::1]`, and not to other web sites open in your browser.
|
|
134
|
+
|
|
135
|
+
### Point an app at it
|
|
136
|
+
|
|
137
|
+
Set the app's base URL to LLMSpy plus the provider's path prefix:
|
|
138
|
+
|
|
139
|
+
| Prefix | Forwards to | Example |
|
|
140
|
+
| ------------ | --------------------------- | ---------------------------------------------------- |
|
|
141
|
+
| `/anthropic` | `https://api.anthropic.com` | `ANTHROPIC_BASE_URL=http://127.0.0.1:8080/anthropic` |
|
|
142
|
+
| `/openai` | `https://api.openai.com` | `OPENAI_BASE_URL=http://127.0.0.1:8080/openai/v1` |
|
|
143
|
+
|
|
144
|
+
For example, to watch Claude Code, in PowerShell:
|
|
145
|
+
|
|
146
|
+
```powershell
|
|
147
|
+
$env:ANTHROPIC_BASE_URL = "http://127.0.0.1:8080/anthropic"
|
|
148
|
+
claude
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
Each request then appears in the UI at `http://127.0.0.1:8080`. The app keeps its own API key, which LLMSpy passes on to the provider.
|
|
152
|
+
|
|
153
|
+
For coding agents and scripts, [llms.txt](llms.txt) is a compact reference: how to run LLMSpy, point an app at it, read the captured traffic as JSON or one call as Markdown, and wait for a call that is still running.
|
|
154
|
+
|
|
155
|
+
### Keep apps apart with projects
|
|
156
|
+
|
|
157
|
+
Put `/@<project>` in front of the prefix to give an app's calls their own tab in the UI. The tag is not forwarded:
|
|
158
|
+
|
|
159
|
+
```powershell
|
|
160
|
+
$env:ANTHROPIC_BASE_URL = "http://127.0.0.1:8080/@my-app/anthropic"
|
|
161
|
+
```
|
|
162
|
+
|
|
163
|
+
Calls without a tag go to `general`. A project name is lowercase letters, digits and dashes. The tabs appear once there is a project besides `general`, and a tab you are not viewing counts its new calls. A search looks in every project; while it is on, each tab counts its matches instead. Clear All empties only the project you are viewing.
|
|
164
|
+
|
|
165
|
+

|
|
166
|
+
|
|
167
|
+
## Commands
|
|
168
|
+
|
|
169
|
+
The `llmspy` command, from npm (`npx llmspy`, or `llmspy` once installed with `npm install -g llmspy`), works in any folder:
|
|
170
|
+
|
|
171
|
+
| Command | What it does |
|
|
172
|
+
| ------------------ | ------------------------------------------------------------------- |
|
|
173
|
+
| `llmspy` | Start LLMSpy in the foreground |
|
|
174
|
+
| `llmspy stop` | Stop LLMSpy started with `llmspy` or `npm start`, from any terminal |
|
|
175
|
+
| `llmspy --help` | List the commands and the settings |
|
|
176
|
+
| `llmspy --version` | Print the version |
|
|
177
|
+
|
|
178
|
+
The rest run from the root of a clone of the repo:
|
|
179
|
+
|
|
180
|
+
| Command | What it does |
|
|
181
|
+
| ------------------------------- | -------------------------------------------------------------------------- |
|
|
182
|
+
| `npm run start:docker` | Build the image and start LLMSpy in Docker, in the background |
|
|
183
|
+
| `npm run stop:docker` | Stop and remove the container; captured calls stay in the `llmspy-data` volume |
|
|
184
|
+
| `docker compose logs -f` | Follow the container's output: its upstreams, storage and errors |
|
|
185
|
+
| `docker compose down -v` | Stop the container and delete its captured calls |
|
|
186
|
+
| `npm install` | Install the packages, for running without Docker |
|
|
187
|
+
| `npm run build` | Build the proxy and the UI (again after each update) |
|
|
188
|
+
| `npm start` | Start LLMSpy in the foreground |
|
|
189
|
+
| `npm run stop` | Stop LLMSpy started with `npm start`, from any terminal |
|
|
190
|
+
| `npm run mock` | Start the mock Anthropic and OpenAI APIs (see [Testing with mock data](#testing-with-mock-data)) |
|
|
191
|
+
| `node scripts/mock-traffic.mjs` | Send a set of sample calls through LLMSpy to the mocks |
|
|
192
|
+
|
|
193
|
+
## Upstreams
|
|
194
|
+
|
|
195
|
+
An upstream is an API that LLMSpy forwards to, reached through its own path prefix. The real Anthropic and OpenAI APIs are built in (`/anthropic` and `/openai`, see [Point an app at it](#point-an-app-at-it)) and need no setup. The ChatGPT app itself can't be pointed anywhere; `/openai` is for code and tools that call the OpenAI API.
|
|
196
|
+
|
|
197
|
+
### Adding an upstream
|
|
198
|
+
|
|
199
|
+
Set `LLMSPY_UPSTREAMS` to a comma-separated list of `name=url` before starting LLMSpy. Each entry adds the prefix `/<name>`, and a name that is already there is replaced. A path in the URL is kept and the rest of the request path is appended to it, so the app's base URL is LLMSpy's address plus the prefix:
|
|
200
|
+
|
|
201
|
+
```
|
|
202
|
+
LLMSPY_UPSTREAMS=openrouter=https://openrouter.ai/api/v1
|
|
203
|
+
app's base URL: http://127.0.0.1:8080/openrouter
|
|
204
|
+
app calls: http://127.0.0.1:8080/openrouter/chat/completions
|
|
205
|
+
LLMSpy sends to: https://openrouter.ai/api/v1/chat/completions
|
|
206
|
+
```
|
|
207
|
+
|
|
208
|
+
Names are lowercase letters, digits and dashes; `api`, `assets`, `events` and `ping` are taken by LLMSpy itself. LLMSpy reads the conversation from any upstream that speaks the Anthropic Messages, OpenAI Chat Completions or OpenAI Responses API, whatever its name; calls to other APIs are still captured and shown in Raw.
|
|
209
|
+
|
|
210
|
+
Examples, with the base URL to give the app:
|
|
211
|
+
|
|
212
|
+
| `LLMSPY_UPSTREAMS` entry | App's base URL |
|
|
213
|
+
| ---------------------------------------------------------------- | ------------------------------------------------------------- |
|
|
214
|
+
| `openrouter=https://openrouter.ai/api/v1` | `http://127.0.0.1:8080/openrouter` |
|
|
215
|
+
| `gemini=https://generativelanguage.googleapis.com/v1beta/openai` | `http://127.0.0.1:8080/gemini` (OpenAI SDK) |
|
|
216
|
+
| `ollama=http://localhost:11434` | `http://127.0.0.1:8080/ollama/v1` (OpenAI SDK) |
|
|
217
|
+
| `anthropic=https://llm-gateway.example.com/anthropic` | `http://127.0.0.1:8080/anthropic` (replaces the built-in one) |
|
|
218
|
+
|
|
219
|
+
### Setting upstreams for a local run
|
|
220
|
+
|
|
221
|
+
Set the variable in the shell that starts LLMSpy. In PowerShell:
|
|
222
|
+
|
|
223
|
+
```powershell
|
|
224
|
+
$env:LLMSPY_UPSTREAMS = "openrouter=https://openrouter.ai/api/v1,ollama=http://localhost:11434"
|
|
225
|
+
npx llmspy
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
In bash:
|
|
229
|
+
|
|
230
|
+
```bash
|
|
231
|
+
LLMSPY_UPSTREAMS=openrouter=https://openrouter.ai/api/v1,ollama=http://localhost:11434 npx llmspy
|
|
232
|
+
```
|
|
233
|
+
|
|
234
|
+
To keep the same settings every time, put them in `~/.llmspy/.env` (the folder that also holds the captured calls) instead, one per line, and `llmspy` reads them whenever it starts:
|
|
235
|
+
|
|
236
|
+
```bash
|
|
237
|
+
# ~/.llmspy/.env
|
|
238
|
+
LLMSPY_UPSTREAMS=openrouter=https://openrouter.ai/api/v1,ollama=http://localhost:11434
|
|
239
|
+
```
|
|
240
|
+
|
|
241
|
+
A variable set in the shell wins over the file, so a one-off value doesn't need the file changed. LLMSpy says `Settings read from …` when it starts if it found the file. It doesn't read a `.env` in the folder you start it from, which is usually an app's own, with its API keys.
|
|
242
|
+
|
|
243
|
+
In a clone of the repo, `npm start` reads a `.env` file at the root of the repo instead (copy `.env.example` to start), and here too the shell wins.
|
|
244
|
+
|
|
245
|
+
### Setting upstreams in Docker
|
|
246
|
+
|
|
247
|
+
Put the variable in a `.env` file next to `compose.yaml` (copy `.env.example` to start). Compose reads it and passes `LLMSPY_UPSTREAMS` and `LLMSPY_RETENTION_DAYS`, and nothing else from it, into the container:
|
|
248
|
+
|
|
249
|
+
```bash
|
|
250
|
+
# .env
|
|
251
|
+
LLMSPY_UPSTREAMS=openrouter=https://openrouter.ai/api/v1,ollama=http://host.docker.internal:11434
|
|
252
|
+
```
|
|
253
|
+
|
|
254
|
+
Then run `npm run start:docker`, which recreates the container if the value has changed.
|
|
255
|
+
|
|
256
|
+
Inside the container, `localhost` is the container itself, so an upstream running on your machine, such as Ollama or the [mocks](#testing-with-mock-data), is at `host.docker.internal`. External URLs are the same as for a local run.
|
|
257
|
+
|
|
258
|
+
The same goes for an app that runs in a container: its `localhost` is its own container, so point it at `http://host.docker.internal:8080/<name>`. An app running directly on your machine uses `http://127.0.0.1:8080/<name>`. If the upstream also runs in a container (a mock server, say), LLMSpy reaches it at `http://host.docker.internal:<published port>`.
|
|
259
|
+
|
|
260
|
+
This works as it is with Docker Desktop. On Linux, `host.docker.internal` doesn't reach a port bound only to `127.0.0.1`, which includes LLMSpy's own port and Ollama's default. Put the containers on one Docker network instead, and address them by container name: `http://llmspy:8080/<name>` from the app, and the upstream's name and its container port in `LLMSPY_UPSTREAMS`.
|
|
261
|
+
|
|
262
|
+
The repo's `.env` serves both `npm start` and Docker, except for upstreams on your machine: a local run reaches them at `localhost`, Docker at `host.docker.internal`. If you switch between the two, override the variable in the shell for local runs. The `llmspy` command never reads this file, only `~/.llmspy/.env`.
|
|
263
|
+
|
|
264
|
+
### Checking your upstreams
|
|
265
|
+
|
|
266
|
+
LLMSpy prints every upstream and its base URL when it starts (`docker compose logs` with Docker), lists them under **Base URLs** in the UI's header, and returns them at `GET /api/upstreams`. A name that isn't allowed or a URL that isn't valid stops it at startup, with the reason.
|
|
267
|
+
|
|
268
|
+
### Apps without a base-URL setting
|
|
269
|
+
|
|
270
|
+
LLMSpy only sees apps that let you set their base URL. For one that doesn't, an intercepting proxy such as [mitmproxy](https://mitmproxy.org) can catch its calls and hand them to LLMSpy, so you still get LLMSpy's views. A small mitmproxy add-on is enough as a starting point:
|
|
271
|
+
|
|
272
|
+
```python
|
|
273
|
+
# llmspy.py: send intercepted LLM API calls on to LLMSpy
|
|
274
|
+
from mitmproxy import http
|
|
275
|
+
|
|
276
|
+
LLMSPY = "http://127.0.0.1:8080"
|
|
277
|
+
ROUTES = {
|
|
278
|
+
"api.anthropic.com": "anthropic",
|
|
279
|
+
"api.openai.com": "openai",
|
|
280
|
+
}
|
|
281
|
+
|
|
282
|
+
|
|
283
|
+
def request(flow: http.HTTPFlow) -> None:
|
|
284
|
+
name = ROUTES.get(flow.request.pretty_host)
|
|
285
|
+
if name:
|
|
286
|
+
flow.request.url = f"{LLMSPY}/{name}{flow.request.path}"
|
|
287
|
+
```
|
|
288
|
+
|
|
289
|
+
Run it on another port, as mitmproxy also defaults to 8080: `mitmdump --listen-port 8081 -s llmspy.py`. Then start the app with `HTTPS_PROXY=http://127.0.0.1:8081` and with mitmproxy's CA certificate trusted (see mitmproxy's docs on certificates). Calls to other hosts pass through mitmproxy untouched. Put `@<project>/` before the name in the URL to give the app its own [project](#keep-apps-apart-with-projects). Don't set `HTTPS_PROXY` for LLMSpy itself.
|
|
290
|
+
|
|
291
|
+
mitmproxy's CA can stand in for any web site, so trust it only in the app you are watching (for Node, `NODE_EXTRA_CA_CERTS`), not in your system's certificate store. Only intercept apps on your own machine, or with the informed consent of whoever uses them.
|
|
292
|
+
|
|
293
|
+
## Configuration
|
|
294
|
+
|
|
295
|
+
LLMSpy reads these environment variables when it starts: from the shell, and from a `.env` file, which is `~/.llmspy/.env` for the `llmspy` command and the one at the root of the repo for `npm start` and Docker (see [Setting upstreams for a local run](#setting-upstreams-for-a-local-run)).
|
|
296
|
+
|
|
297
|
+
| Variable | Default | Meaning |
|
|
298
|
+
| ----------------------- | ----------- | ------------------------------------------------------------ |
|
|
299
|
+
| `PROXY_PORT` | `8080` | Port for the proxy and the UI |
|
|
300
|
+
| `LLMSPY_HOST` | `127.0.0.1` | Interface to listen on; the Docker image sets `0.0.0.0` |
|
|
301
|
+
| `LLMSPY_UPSTREAMS` | | Extra or replacement upstreams, see [Upstreams](#upstreams) |
|
|
302
|
+
| `LLMSPY_RETENTION_DAYS` | `7` | Days to keep captured calls; `0` keeps them until cleared |
|
|
303
|
+
|
|
304
|
+
### Storage
|
|
305
|
+
|
|
306
|
+
Captured calls are kept in `~/.llmspy/llmspy.db` (with Docker, in the `llmspy-data` volume), so they are still there after a restart. LLMSpy prints where, and for how long, when it starts. Calls older than `LLMSPY_RETENTION_DAYS` are removed at startup and every hour after. Clear All and the delete button remove calls from the file too. A call still running when LLMSpy stopped is shown as interrupted.
|
|
307
|
+
|
|
308
|
+
The file holds whole prompts and replies (API keys are masked), so treat it like the traffic itself: it stays on your machine, and deleting it while LLMSpy is stopped removes every call.
|
|
309
|
+
|
|
310
|
+
## Testing with mock data
|
|
311
|
+
|
|
312
|
+
To try LLMSpy without an API key or network, `npm run mock` starts two [llmock](https://www.npmjs.com/package/llmock) servers, with their config in `mock/.llmockrc.json`: the Anthropic Messages API on `http://127.0.0.1:8001` and OpenAI Chat Completions on `http://127.0.0.1:8002`. They reply with generated text. It needs `npm install` first, even if you run LLMSpy in Docker.
|
|
313
|
+
|
|
314
|
+
Start the mocks in one terminal and leave them running:
|
|
315
|
+
|
|
316
|
+
```bash
|
|
317
|
+
npm run mock
|
|
318
|
+
```
|
|
319
|
+
|
|
320
|
+
In a second terminal, start LLMSpy with the mocks as upstreams (after `npm run build`):
|
|
321
|
+
|
|
322
|
+
```bash
|
|
323
|
+
LLMSPY_UPSTREAMS=llmock=http://127.0.0.1:8001,llmock-openai=http://127.0.0.1:8002 npm start
|
|
324
|
+
```
|
|
325
|
+
|
|
326
|
+
With Docker, put `LLMSPY_UPSTREAMS=llmock=http://host.docker.internal:8001,llmock-openai=http://host.docker.internal:8002` in `.env` and run `npm run start:docker` instead.
|
|
327
|
+
|
|
328
|
+
Then, in a third terminal, send the sample calls:
|
|
329
|
+
|
|
330
|
+
```bash
|
|
331
|
+
node scripts/mock-traffic.mjs
|
|
332
|
+
```
|
|
333
|
+
|
|
334
|
+
The script sends one call of each kind below. To use the mocks from your own client, set its base URL to `http://127.0.0.1:8080/llmock` (Anthropic) or `http://127.0.0.1:8080/llmock-openai/v1` (OpenAI), with any non-empty key. A marker in the last message picks the reply:
|
|
335
|
+
|
|
336
|
+
| Marker | Reply |
|
|
337
|
+
| --------------- | --------------------------------------------------------------------------------------------- |
|
|
338
|
+
| `#tools` | A `get_weather` tool call (Anthropic); a tool result containing `sunny` gets the final answer |
|
|
339
|
+
| `#max-tokens` | A reply cut off at the token limit |
|
|
340
|
+
| `#refusal` | A refusal |
|
|
341
|
+
| `#rate-limit` | HTTP 429 |
|
|
342
|
+
| `#stream-error` | A stream that sends an error event part-way through |
|
|
343
|
+
| `#stream-drop` | A stream whose connection is cut part-way through |
|
|
344
|
+
|
|
345
|
+
Anything else gets random lorem ipsum. Both mocks stream when the request sets `"stream": true`.
|
|
346
|
+
|
|
347
|
+
## How it works
|
|
348
|
+
|
|
349
|
+
- `proxy/`: one Node HTTP server. `/<prefix>/...` (or `/@<project>/<prefix>/...`) is forwarded to the upstream with the bytes unchanged, chunk by chunk, while a copy of each body is kept, decompressed (gzip, deflate, br, zstd) and decoded for display. Calls are stored with `node:sqlite`; a running call is held in memory and written when it starts, when its response headers arrive and when it ends. Any path that is not an upstream or the API below is a file from the built UI (`ui/dist`).
|
|
350
|
+
- The UI and agents read the captured calls from a small API on the same port:
|
|
351
|
+
- `GET /events`: Server-Sent Events of call summaries
|
|
352
|
+
- `GET /api/requests`: summaries, filtered with `?project=`, `?q=` (search), `?since=`, `?model=`, `?status=`, `?failed=1` and `?limit=`; `?full=1` returns full records
|
|
353
|
+
- `GET /api/stats`: calls, failures, tokens and latency over the same filters, per model and project
|
|
354
|
+
- `GET` and `DELETE /api/requests/<id>`: one full record
|
|
355
|
+
- `GET /api/requests/<id>/conversation`: the call as Markdown (both reads of one call take `?wait=<seconds>` to answer once a running call ends)
|
|
356
|
+
- `GET /api/upstreams`: each prefix and where it forwards to
|
|
357
|
+
- `POST /api/clear`: Clear All
|
|
358
|
+
- `ui/`: Svelte and Vite.
|
|
359
|
+
|
|
360
|
+
## Troubleshooting
|
|
361
|
+
|
|
362
|
+
- **The app still calls the provider directly.** Check that it reads the base URL variable you set, and that it was started after you set it.
|
|
363
|
+
- **404 from LLMSpy.** The path prefix doesn't match an upstream; the response lists the known ones.
|
|
364
|
+
- **The app reports a connection error and nothing appears in LLMSpy.** The app can't reach LLMSpy. Check that LLMSpy is running; if the app runs in a container, `localhost` is that container (see [Setting upstreams in Docker](#setting-upstreams-in-docker)).
|
|
365
|
+
- **502 from LLMSpy.** The upstream could not be reached; the request in the UI shows the error. With Docker, an upstream on your machine or in another container is not at `localhost` (see [Setting upstreams in Docker](#setting-upstreams-in-docker)).
|
|
366
|
+
- **LLMSpy won't start: `EADDRINUSE`, or Docker's "port is already allocated".** Something else has port 8080, often LLMSpy started another way. Stop it with `llmspy stop` (or `npm run stop`), or `npm run stop:docker`.
|
|
367
|
+
|
|
368
|
+
## License
|
|
369
|
+
|
|
370
|
+
MIT, see [LICENSE](LICENSE).
|
package/llms.txt
ADDED
|
@@ -0,0 +1,189 @@
|
|
|
1
|
+
# LLMSpy
|
|
2
|
+
|
|
3
|
+
> A local reverse proxy that records the LLM API calls an app makes and shows each one in a web UI: the prompt, the system message, the tools, the reply and the token counts. An app is pointed at it by its base URL setting (`ANTHROPIC_BASE_URL`, `OPENAI_BASE_URL`); requests are forwarded unchanged and a copy is kept. This file is the reference for AI agents and scripts; it ships in the repository and in the npm package `llmspy`, and `README.md` is the human guide.
|
|
4
|
+
|
|
5
|
+
## What it does and does not do
|
|
6
|
+
|
|
7
|
+
- One HTTP server, on `http://127.0.0.1:8080` by default, serves three things: `/<upstream>/...` is forwarded to an LLM API, `/ping`, `/events` and `/api/*` serve the captured traffic, and anything else is a file of the built web UI.
|
|
8
|
+
- Forwarding is a byte-for-byte passthrough, chunk by chunk, so streamed replies still stream to the app. LLMSpy never changes a request or a response.
|
|
9
|
+
- It keeps calls in a SQLite file, `~/.llmspy/llmspy.db` (Docker: the `llmspy-data` volume), so they survive a restart. Calls older than `LLMSPY_RETENTION_DAYS` (default 7) are removed at startup and every hour. A running call is written when it starts, when its response headers arrive and when it ends; the updates in between are held in memory. A call still running when LLMSpy stopped gets `error: "interrupted: LLMSpy stopped before the call ended"` and `timings.end` set to when its last data arrived.
|
|
10
|
+
- `authorization` and `x-api-key` headers are masked in what it records (`Bearer ••••abcd`: the scheme and the last 4 characters of a long key). Prompts, replies and every other header are recorded as sent.
|
|
11
|
+
- A call is recorded as soon as its request body has arrived, updated when the response headers arrive, and completed when the exchange ends: when the response has been sent in full, the upstream failed, or the client hung up. Until then `timings.end` is absent. A streamed reply (`text/event-stream`, not compressed) is also updated at most every 500 ms with what has arrived so far; any other reply body stays empty until the exchange ends.
|
|
12
|
+
- It does not intercept traffic (no MITM, no CA certificate). An app that has no base URL setting can not be watched.
|
|
13
|
+
- It is for local development. It is not authenticated and must not be reachable from other machines.
|
|
14
|
+
|
|
15
|
+
## Run and stop
|
|
16
|
+
|
|
17
|
+
Running without Docker needs Node.js 26 or later (`engines` in `package.json`); the Docker image brings its own. The npm package runs from any folder; the other ways run from the root of a clone of the repository (`npm install` once first, except for Docker).
|
|
18
|
+
|
|
19
|
+
| How | Start | Stop |
|
|
20
|
+
|---|---|---|
|
|
21
|
+
| npm package | `npx --yes llmspy` (or `llmspy`, after `npm install -g llmspy`) | `npx --yes llmspy stop` (`llmspy stop`), from any terminal |
|
|
22
|
+
| Docker (runs in the background, restarts with Docker) | `npm run start:docker` (`docker compose up -d --build`) | `npm run stop:docker` (`docker compose down`) |
|
|
23
|
+
| Built, local | `npm run build`, then `npm start` | `npm run stop`, from any terminal |
|
|
24
|
+
| Development, with reload | `npm run dev` (UI also on `http://localhost:3000`) | Ctrl+C in its terminal |
|
|
25
|
+
|
|
26
|
+
- Ready when `GET http://127.0.0.1:8080/ping` answers `200`. `llmspy`, `npm start` and `npm run dev` stay in the foreground: run them in the background and poll `/ping`. Pass `--yes` to `npx`, or the first run waits for an answer to "Ok to proceed?" before installing the package, which blocks a run in the background.
|
|
27
|
+
- All of them use port 8080. Stop one before starting another; a second start on a taken port fails with `EADDRINUSE` and exits `1`.
|
|
28
|
+
- `llmspy`, `npm start` and `npm run dev` write `~/.llmspy/llmspy.pid` (`{"pid", "url"}`) once they listen. `llmspy stop` (in a clone, `npm run stop` runs the same) stops that process only if `/ping` at that URL reports the same pid, so a stale or unreadable file is removed and nothing else is killed. It exits `0` when LLMSpy is stopped or was not running, and `1` when it did not stop within 5 seconds or the pid file could not be read. It does not stop Docker. For `npm run dev` it stops the proxy, but the watcher starts it again on the next file change: stop `npm run dev` with Ctrl+C.
|
|
29
|
+
- `llmspy --help` lists the commands and settings, `llmspy --version` prints the version. Any other argument exits `1`; on Node.js older than 26 `llmspy` says so and exits `1`.
|
|
30
|
+
- On start LLMSpy prints the database file and how long calls are kept (`Calls kept 7 days in …`), then the base URL of each upstream. A bad `LLMSPY_UPSTREAMS` or `LLMSPY_RETENTION_DAYS` prints the reason and exits `1`.
|
|
31
|
+
|
|
32
|
+
## Point an app at it
|
|
33
|
+
|
|
34
|
+
Set the app's base URL to LLMSpy plus the upstream's path prefix, then start the app (most read the variable only at startup):
|
|
35
|
+
|
|
36
|
+
| Prefix | Forwards to | Base URL setting |
|
|
37
|
+
|---|---|---|
|
|
38
|
+
| `/anthropic` | `https://api.anthropic.com` | `ANTHROPIC_BASE_URL=http://127.0.0.1:8080/anthropic` |
|
|
39
|
+
| `/openai` | `https://api.openai.com` | `OPENAI_BASE_URL=http://127.0.0.1:8080/openai/v1` |
|
|
40
|
+
|
|
41
|
+
- The rest of the path and the query are appended to the upstream's URL: `/anthropic/v1/messages` goes to `https://api.anthropic.com/v1/messages`. The OpenAI SDKs expect `/v1` in their base URL; the Anthropic SDK does not.
|
|
42
|
+
- The app's API key is sent through to the provider as usual. LLMSpy needs no key.
|
|
43
|
+
- An app in another container reaches the proxy at `http://host.docker.internal:8080/<prefix>`, or by service name in the same compose project. Forwarding answers any `Host`; only the traffic endpoints below are limited to loopback.
|
|
44
|
+
- To group an app's calls into a project, put `/@<project>` in front of the prefix: `ANTHROPIC_BASE_URL=http://127.0.0.1:8080/@my-app/anthropic`. It is not forwarded, and sets the record's `project`; without it the project is `general`. Project names follow the same rule as upstream names (below); any other name gets the `404` for an unknown prefix.
|
|
45
|
+
- An app with no base-URL setting can't be pointed at LLMSpy directly. LLMSpy has no intercepting mode; the README section "Apps without a base-URL setting" has a mitmproxy add-on (an untested starting point) that rewrites `https://api.anthropic.com/...` to `http://127.0.0.1:8080/anthropic/...`. Run mitmproxy on another port (`--listen-port 8081`), as LLMSpy uses 8080. Ask the user before installing mitmproxy or trusting its CA.
|
|
46
|
+
|
|
47
|
+
## Configuration
|
|
48
|
+
|
|
49
|
+
Environment variables, read once at startup:
|
|
50
|
+
|
|
51
|
+
| Variable | Default | Meaning |
|
|
52
|
+
|---|---|---|
|
|
53
|
+
| `PROXY_PORT` | `8080` | Port for everything. Not passed into Docker: there it is always `8080` |
|
|
54
|
+
| `LLMSPY_HOST` | `127.0.0.1` | Interface to listen on. The Docker image sets `0.0.0.0` and compose publishes the port on `127.0.0.1` only |
|
|
55
|
+
| `LLMSPY_UPSTREAMS` | empty | Extra or replacement upstreams, as comma-separated `name=url` |
|
|
56
|
+
| `LLMSPY_RETENTION_DAYS` | `7` | Days to keep calls, a whole number; `0` keeps them until cleared. Passed into Docker from `.env` |
|
|
57
|
+
|
|
58
|
+
`LLMSPY_UPSTREAMS`:
|
|
59
|
+
|
|
60
|
+
- Each entry adds the prefix `/<name>`. A name that is already there (`anthropic`, `openai`) is replaced.
|
|
61
|
+
- Names are lowercase letters, digits and dashes, starting with a letter or digit. `api`, `assets`, `events` and `ping` are refused.
|
|
62
|
+
- URLs must be `http` or `https`. A path in the URL is kept and the request path is appended: `gw=https://gateway.example/v1` sends `/gw/messages` to `https://gateway.example/v1/messages`.
|
|
63
|
+
- Set it in the shell, or in a settings file. For `llmspy` (the npm package) that file is `~/.llmspy/.env`: `LLMSPY_UPSTREAMS=llmock=http://127.0.0.1:8001 npx --yes llmspy` for one run, or a line `LLMSPY_UPSTREAMS=...` in `~/.llmspy/.env` to keep it. `llmspy` never reads a `.env` in the current folder: that is usually the app's own, with its API keys, so do not write LLMSpy settings there. The shell wins over the file; `llmspy` prints `Settings read from <file>` when the file exists, and exits `1` if it can not be read.
|
|
64
|
+
- In a clone of the repository, `npm start`, `npm run dev` and `docker compose up` read `.env` at the repo root (next to `compose.yaml`) instead; in local runs a variable already set in the shell wins. Settings files are read at start: restart LLMSpy (or rerun `npm run start:docker`) after changing one.
|
|
65
|
+
- Inside the container `localhost` is the container: an upstream on the host is `http://host.docker.internal:<port>`. Locally it is `localhost`, so a `.env` with `host.docker.internal` upstreams is for Docker only.
|
|
66
|
+
|
|
67
|
+
```bash
|
|
68
|
+
LLMSPY_UPSTREAMS=ollama=http://localhost:11434,llmock=http://127.0.0.1:8001 npm start
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
## Reading captured traffic
|
|
72
|
+
|
|
73
|
+
| Route | Returns |
|
|
74
|
+
|---|---|
|
|
75
|
+
| `GET /ping` | `200 {"status":"running","message","version","port","pid","timestamp"}`; `version` is LLMSpy's, e.g. `0.1.0`; `port` is the port really bound |
|
|
76
|
+
| `GET /api/requests` | `200`, a JSON array of the summary of every record, newest first (below). `?project=<name>` keeps that project's; `?q=<text>` keeps those whose `host`, `path`, request body or response body holds the text, ignoring case; `?since=<time>` keeps those whose `timings.start` is at or after the time, given as an ISO time (`2026-11-15T10:00:00Z`; without a zone it is local time) or epoch milliseconds; `?model=<text>` keeps those whose `llm.model` holds the text, ignoring case (`model=sonnet`); `?status=<code>` keeps those whose `response.status` is that code (`429`) or in that class (`5xx`), so never a call with no response; `?failed=1` keeps the failed calls: those with an `error`, a `response.status` of 400 or above, or an `llm.error` (the same calls the UI's Failed chip shows); `?limit=<n>` keeps only the newest `n` of those left; `?full=1` returns the full records instead of summaries, in the same order (running calls as they are so far). Any of them together keep calls that match all. A value that can not be read (a project name that can not exist, a `since` that is not a time, an empty `model`, a `status` that is not three digits or a digit and `xx`, a `limit` that is not a whole number above 0, a `failed` or `full` other than `1`) gets `400` with a plain-text message |
|
|
77
|
+
| `GET /api/stats` | `200`, counts over the calls `GET /api/requests` would list with the same `project`, `q`, `since`, `model`, `status` and `failed` (read and refused the same way; `limit` and `full` get `400`): `{"calls","running","failed","tokens":{"input","output","cacheRead","cacheWrite"},"latencyMs":{"p50","p95","max"},"byModel":{...},"byProject":{...}}`. `running` counts calls with no `timings.end`; `failed` as `?failed=1`. `tokens` adds up `llm.usage` (a count a call did not report adds 0; a running call adds what it has so far); Anthropic's `input` leaves out cached tokens and OpenAI's includes them, so mixing providers mixes the two. `latencyMs` is over ended calls only, by nearest rank, and `null` when none has ended. `byModel` (keyed by `llm.model`, `none` for calls with no model) and `byProject` give the same fields for each, without their own `byModel`/`byProject` |
|
|
78
|
+
| `GET /api/requests/<id>` | `200`, the full record; `404` when no record has that `id`. `?wait=<seconds>` (a whole number from 1 to 300, else `400`) holds the answer while the call is running: it comes when the call ends, or when the seconds run out with the call so far (no `timings.end`; still `200`). An ended call, or an unknown `id`, is answered at once; a call deleted or cleared while you wait gets `404` |
|
|
79
|
+
| `GET /api/requests/<id>/conversation` | `200 text/markdown`: the call as you would read it, in full. A header line (time, project, status or `running`, latency, tokens, stop reason), then `## System`, `## Tools`, one section per message (`## User`, `## Assistant`, `## Tool`), `## Assistant (reply)` and `## Error` when there is one (a body that could not be read is named there, and what could be read is still shown). Thinking is quoted, tool calls and results are in code fences, images are `[image]`. `404` when no record has that `id`, or when the call was not read as an LLM API call (read the record instead). Takes `?wait=<seconds>` as the record does; on timeout the header says `running` |
|
|
80
|
+
| `GET /api/upstreams` | `200`, a JSON array of `{"name","target"}` in configuration order; any user name or password in `target` is removed |
|
|
81
|
+
| `POST /api/clear` | `200 {"success":true}`: every record is deleted (from the database file too) and every `/events` client is sent a `clear` event. No body needed. With `?project=<name>` only that project's records are deleted; `400` when the name can not be a project |
|
|
82
|
+
| `DELETE /api/requests/<id>` | `204`: that record is deleted and every `/events` client is sent a `delete` event; `404` when no record has that `id`. A call still running is stored again when it next updates |
|
|
83
|
+
| `GET /events` | Server-Sent Events: on connect the summary of every stored record, newest first, then a record's summary each time it is captured or updated. Each summary is one `data: <summary JSON>` line with no `event:` name; read the full record from `GET /api/requests/<id>`. When the records are cleared it sends `event: clear` with `data: {}`: drop every record you hold; `data: {"project":"<name>"}` when only one project was: drop that project's records. When one record is deleted it sends `event: delete` with `data: {"id":"<id>"}`: drop that record. A call arrives several times with the same `id` (more often while a reply streams): replace by `id`, and treat it as finished once `timings.end` is set |
|
|
84
|
+
|
|
85
|
+
- A query string is ignored, except `project` on `/api/clear`, `wait` on the two reads of one call, and the parameters above on `/api/requests` and `/api/stats`; an unknown parameter is ignored too (`/api/requests?max=1` returns every summary). Otherwise paths must match exactly; `/api/requests/` (trailing slash) is answered `404`. There is no paging: narrow with `since`, `project` and `limit`.
|
|
86
|
+
- These routes answer only requests whose `Host` is `localhost`, `127.0.0.1` or `[::1]` (with any port), or the address in `LLMSPY_HOST` when that is not `0.0.0.0`/`::`. A request with an `Origin` header must also come from the same origin as its `Host`. Anything else gets `403` with a plain-text message. curl, scripts and SDKs send no `Origin` and pass when they call `127.0.0.1`; a call from another container (`Host: llmspy` or `host.docker.internal`) gets `403`.
|
|
87
|
+
- A record can be large (a coding agent's request is often around 1 MB), which is why the list routes send summaries: read only the records you need, and use `full=1` with `since` or `limit`, not on its own.
|
|
88
|
+
|
|
89
|
+
## Record shape
|
|
90
|
+
|
|
91
|
+
Each record from `/api/requests/<id>`:
|
|
92
|
+
|
|
93
|
+
| Field | Meaning |
|
|
94
|
+
|---|---|
|
|
95
|
+
| `id` | Random UUID |
|
|
96
|
+
| `timestamp` | ISO time the request arrived |
|
|
97
|
+
| `project` | From `/@<project>` in front of the prefix, else `general` |
|
|
98
|
+
| `provider` | The upstream name, from the path prefix |
|
|
99
|
+
| `host` | The upstream host it was forwarded to |
|
|
100
|
+
| `method`, `path` | As sent upstream (`path` includes the query) |
|
|
101
|
+
| `headers` | Request headers, keys masked, hop-by-hop headers removed |
|
|
102
|
+
| `body` | Request body, decompressed and decoded as UTF-8 text (JSON is a string: parse it) |
|
|
103
|
+
| `response` | `{"status","headers","body"}`, body decompressed and decoded. Absent until the response headers arrive, and when the upstream never answered; `body` is `""` until the call ends, except for a streamed reply, which holds what has arrived so far |
|
|
104
|
+
| `timings` | `{"start","firstByte"?,"lastData"?,"end"?}`, epoch ms; `firstByte` is when the response headers arrived, `lastData` when the latest chunk of a streamed reply arrived (only for the uncompressed streams updated while they arrive); no `end` while the call is still running. On a running stream, `lastData` shows how long it has been quiet |
|
|
105
|
+
| `latency` | `end - start`, ms; absent while the call is still running |
|
|
106
|
+
| `error` | Set when the exchange failed: `Upstream request failed: …` (never reached; the app got `502`), `Upstream response failed: …` (the reply broke off), `Client closed the connection before the response finished`, `interrupted: LLMSpy stopped before the call ended` |
|
|
107
|
+
| `llm` | The call read as an LLM API call, below. Absent when the path is not one LLMSpy knows |
|
|
108
|
+
|
|
109
|
+
A streamed response body is the raw SSE text. A body that can not be decompressed is replaced by `[LLMSpy could not decode this body: …]`.
|
|
110
|
+
|
|
111
|
+
A summary, in `/api/requests` and `/events`, has the record's `id`, `timestamp`, `project`, `provider`, `host`, `method`, `path`, `timings`, `latency` and `error`, plus `response: {"status"}` and `llm: {"model","usage","error"}` when the record has them. It has no headers or bodies.
|
|
112
|
+
|
|
113
|
+
### `llm`
|
|
114
|
+
|
|
115
|
+
Present for `POST` requests whose path ends in `/v1/messages` (`format: "anthropic-messages"`), `/chat/completions` (`"openai-chat"`) or `/responses` (`"openai-responses"`), whatever the upstream's name. While the call is running it holds what the request says (model, system, tools, messages), plus, for a streamed reply, the reply read so far; the reply is complete when the call ends. The same shape for every format:
|
|
116
|
+
|
|
117
|
+
| Field | Meaning |
|
|
118
|
+
|---|---|
|
|
119
|
+
| `format` | One of the three above |
|
|
120
|
+
| `model` | The model that answered, else the one asked for |
|
|
121
|
+
| `stream` | Whether the reply was streamed |
|
|
122
|
+
| `system` | The system prompt or instructions, as one string |
|
|
123
|
+
| `messages` | The conversation sent, oldest first: `[{"role","parts"}]`, `role` is `user`, `assistant` or `tool` |
|
|
124
|
+
| `tools` | `[{"name","description"?}]`; the schemas are only in `body` |
|
|
125
|
+
| `output` | The reply as one message (`role: "assistant"`), reassembled if streamed |
|
|
126
|
+
| `stopReason` | As the provider sent it (`end_turn`, `tool_use`, `stop`, `length`, …) |
|
|
127
|
+
| `usage` | `{"input"?,"output"?,"cacheRead"?,"cacheWrite"?,"reasoning"?}` as the provider counts them; Anthropic's `input` excludes cached tokens, OpenAI's includes them |
|
|
128
|
+
| `error` | `{"type"?,"message"}`: an error the provider sent inside a `2xx` reply, such as an `error` event part-way through a stream |
|
|
129
|
+
| `parseError` | Set when a body could not be read; the fields that could be read are still there |
|
|
130
|
+
|
|
131
|
+
A part is one of:
|
|
132
|
+
|
|
133
|
+
- `{"type":"text","text"}`
|
|
134
|
+
- `{"type":"thinking","text"}`
|
|
135
|
+
- `{"type":"tool_call","id","name","input"}`, where `input` is parsed JSON when possible
|
|
136
|
+
- `{"type":"tool_result","id","content","isError"?}`, matched to its call by `id`
|
|
137
|
+
- `{"type":"image"}`, with the data left out
|
|
138
|
+
- `{"type":"other","raw"}`, a block LLMSpy has no view for, as sent
|
|
139
|
+
|
|
140
|
+
Only `2xx` responses are parsed. For a `4xx`/`5xx` call, `llm` has the request fields but no `output`, `stopReason` or `usage`: read `response.status` and `response.body`.
|
|
141
|
+
|
|
142
|
+
## Errors
|
|
143
|
+
|
|
144
|
+
| What the app gets | Why | In the record |
|
|
145
|
+
|---|---|---|
|
|
146
|
+
| `404 text/plain` `LLMSpy: no upstream for <path>. Known prefixes: …` | The first path segment is not an upstream | Nothing recorded |
|
|
147
|
+
| `502 text/plain` `LLMSpy could not reach <origin>: …` | DNS, connection or TLS failure to the upstream | `error`, no `response` |
|
|
148
|
+
| The connection is cut | The upstream's reply broke off part-way | `error`, `response` with what arrived |
|
|
149
|
+
| The provider's own status and body | Anything the provider answered, `4xx`/`5xx` included | `response` as sent |
|
|
150
|
+
| `403 text/plain` | A traffic endpoint called with another `Host` or `Origin` (above) | Nothing recorded |
|
|
151
|
+
|
|
152
|
+
## Recipes
|
|
153
|
+
|
|
154
|
+
The last call's prompt and reply, without `jq`:
|
|
155
|
+
|
|
156
|
+
```bash
|
|
157
|
+
node -e "
|
|
158
|
+
const base = 'http://127.0.0.1:8080/api/requests';
|
|
159
|
+
fetch(base).then((res) => res.json()).then(async ([s]) => {
|
|
160
|
+
const r = s && (await (await fetch(base + '/' + s.id)).json());
|
|
161
|
+
console.log(JSON.stringify({ status: r?.response?.status, error: r?.error, llm: r?.llm }, null, 2));
|
|
162
|
+
});"
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
With `jq`: `id=$(curl -s http://127.0.0.1:8080/api/requests | jq -r '.[0].id'); curl -s "http://127.0.0.1:8080/api/requests/$id" | jq '.llm | {model, system, messages, output, usage}'`.
|
|
166
|
+
|
|
167
|
+
Read the last call as text, the shortest way to see what an agent sent and got back: `id=$(curl -s 'http://127.0.0.1:8080/api/requests?limit=1' | jq -r '.[0].id'); curl -s "http://127.0.0.1:8080/api/requests/$id/conversation"`.
|
|
168
|
+
|
|
169
|
+
Find the calls that mention something: `curl -s 'http://127.0.0.1:8080/api/requests?q=get_weather'`, then read each by `id`.
|
|
170
|
+
|
|
171
|
+
Find what went wrong: `curl -s 'http://127.0.0.1:8080/api/requests?failed=1&limit=10'` lists the newest failed calls; read one's `error`, `response.status` and `llm.error` in its summary, then the record or `/conversation` for the detail. `?status=429` finds the rate-limited ones, `?model=haiku&failed=1` one model's failures.
|
|
172
|
+
|
|
173
|
+
How much a task cost and how slow it was: note `since` as below, run the app, then `curl -s "http://127.0.0.1:8080/api/stats?since=$since"` for the calls, failures, tokens and p50/p95 latency, in all and per model. Read `/api/requests` with the same parameters to see which calls they were.
|
|
174
|
+
|
|
175
|
+
The last 3 calls in full: `curl -s 'http://127.0.0.1:8080/api/requests?full=1&limit=3'`.
|
|
176
|
+
|
|
177
|
+
Check what an app sends for one task, without clearing anyone's calls: note the time (`since=$(date +%s%3N)`, or `Date.now()`), run the app, then read `curl -s "http://127.0.0.1:8080/api/requests?since=$since&full=1"` (newest first, so reverse it for the order the calls were made in). Add `&project=<name>` when other apps send traffic at the same time.
|
|
178
|
+
|
|
179
|
+
Wait for a call to end: find its `id` (`GET /api/requests?since=<when you started>&limit=1`, or the first summary on `/events`), then `curl -s "http://127.0.0.1:8080/api/requests/$id/conversation?wait=120"` answers once it ends; use `/api/requests/$id?wait=120` for the record. If the answer still has no `timings.end` (or the Markdown header says `running`), the wait ran out: ask again. Waiting for a call that has not been sent yet is not built in: poll `GET /api/requests?since=<when you started>` until a summary shows up.
|
|
180
|
+
|
|
181
|
+
Without an API key, in a clone of the repository: `npm run mock` starts mock LLM APIs (Anthropic Messages on `127.0.0.1:8001`, OpenAI Chat Completions on `127.0.0.1:8002`). Start LLMSpy with `LLMSPY_UPSTREAMS=llmock=http://127.0.0.1:8001,llmock-openai=http://127.0.0.1:8002`, then `node scripts/mock-traffic.mjs` sends one call of each kind (tool call, `max_tokens`, refusal, `429`, stream error, dropped stream). The README lists the prompt markers that pick each reply.
|
|
182
|
+
|
|
183
|
+
## Do not
|
|
184
|
+
|
|
185
|
+
- Do not set `LLMSPY_HOST` to `0.0.0.0` outside Docker, or publish the Docker port as anything but `127.0.0.1:8080:8080`: forwarding and the UI would be open to the network.
|
|
186
|
+
- Do not add mitmproxy's CA (or any intercepting proxy's) to the system certificate store, or intercept an app on someone else's machine: trust it only in the app being watched (for Node, `NODE_EXTRA_CA_CERTS`).
|
|
187
|
+
- Do not save or commit captured records: they hold whole prompts and replies, and the masked key's last 4 characters.
|
|
188
|
+
- Do not rely on records older than `LLMSPY_RETENTION_DAYS`.
|
|
189
|
+
- Do not read or change `llmspy.db` directly: use the routes above. Its layout is not part of this reference.
|