@prompt-helm/sdk 1.4.0 → 1.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +103 -14
- package/dist/cli.js +5 -1
- package/dist/index.cjs +339 -22
- package/dist/index.d.cts +172 -10
- package/dist/index.d.ts +172 -10
- package/dist/index.js +330 -21
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -16,6 +16,8 @@ refresh, 60 seconds by default.
|
|
|
16
16
|
their values, your provider keys or your LLM traffic.
|
|
17
17
|
- **Model settings travel with the text.** A version can carry its model, temperature and
|
|
18
18
|
output limit, so moving to a cheaper model is a publish you can roll back or A/B test.
|
|
19
|
+
- **System and user messages.** A prompt can hold the user message too, with its own variables,
|
|
20
|
+
and ask for an answer in JSON that matches a schema.
|
|
19
21
|
- **No dependencies.** ESM and CommonJS, Node.js 18 and later.
|
|
20
22
|
|
|
21
23
|
## Install
|
|
@@ -50,6 +52,30 @@ The stored prompt marks each variable in double braces, such as `{{customer_name
|
|
|
50
52
|
version uses a variable the call doesn't pass, `get` doesn't send a half-filled prompt: it
|
|
51
53
|
returns the fallback (or, without one, throws `missing_variables`).
|
|
52
54
|
|
|
55
|
+
## System and user messages
|
|
56
|
+
|
|
57
|
+
A prompt's text is its system message. It can also hold the user message, for a request your app
|
|
58
|
+
sends the same way each time with different values. `resolve` fills both in:
|
|
59
|
+
|
|
60
|
+
```ts
|
|
61
|
+
const r = await ph.resolve("classify-ticket", {
|
|
62
|
+
vars: { product: "Acme", ticket: ticket.text },
|
|
63
|
+
fallback: { system: CLASSIFY_TICKET, user: "Ticket: {{ticket}}" },
|
|
64
|
+
});
|
|
65
|
+
|
|
66
|
+
const reply = await openai.responses.create({
|
|
67
|
+
model: r.config.model ?? "gpt-5",
|
|
68
|
+
instructions: r.system,
|
|
69
|
+
input: r.user ?? ticket.text,
|
|
70
|
+
});
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
`user` is `null` when the live version has none: your app sends its own, such as what the person
|
|
74
|
+
typed. `get` returns the system message only and never needs the user message's variables, so an
|
|
75
|
+
app that sends its own user message keeps working when one is added in the dashboard. If the user
|
|
76
|
+
message uses a variable the call doesn't pass, `resolve` returns the fallback, as it does for the
|
|
77
|
+
system message. SDK versions before 1.5 receive the system message only.
|
|
78
|
+
|
|
53
79
|
## Options
|
|
54
80
|
|
|
55
81
|
All optional when the environment variables are set.
|
|
@@ -63,6 +89,7 @@ All optional when the environment variables are set.
|
|
|
63
89
|
| `onError` | `console.warn` | Called with the error and the slug when a fallback or a cached prompt was served instead of failing. |
|
|
64
90
|
| `reportFallbacks` | `true` | Tells PromptHelm, by error code only, when a fallback or a cached prompt was served, so the prompt's page shows which app fell back and why. |
|
|
65
91
|
| `reportResults` | `reportFallbacks` | During an [A/B test](#ab-tests), sends PromptHelm totals only: calls served per version, and the outcomes and usage you report. |
|
|
92
|
+
| `reportFailures` | `reportFallbacks` | Sends what [`failed`](#refused-calls) learns: the provider's HTTP status and error codes, by version, never a message. |
|
|
66
93
|
| `onResolve` | none | Called on every `get` and `resolve` with the version served, for your analytics or traces. |
|
|
67
94
|
| `fetch` | global `fetch` | Your own fetch, for tests or a proxy. |
|
|
68
95
|
|
|
@@ -78,6 +105,7 @@ Without a `fallback`, `get` throws them; with one, `onError` receives them.
|
|
|
78
105
|
| `missing_variables` | The live version uses a variable the call doesn't pass. |
|
|
79
106
|
| `timeout` | No response within the timeout. |
|
|
80
107
|
| `network` | PromptHelm couldn't be reached. |
|
|
108
|
+
| `api_mismatch` | The live version's model can't be called with the call's `api`. |
|
|
81
109
|
| `server` | Anything else, such as a 5xx response. |
|
|
82
110
|
|
|
83
111
|
After the first successful fetch, a failed background refresh keeps serving the cached prompt
|
|
@@ -106,35 +134,96 @@ const system = await ph.get("support-reply", {
|
|
|
106
134
|
});
|
|
107
135
|
```
|
|
108
136
|
|
|
137
|
+
A prompt with a user message is written as `{ system, user }`, the shape `fallback` takes.
|
|
138
|
+
|
|
109
139
|
Run it in CI before the build, so every deploy carries the text that was live then. It reads
|
|
110
140
|
`PROMPTHELM_API_KEY` (and `PROMPTHELM_BASE_URL`), and writes JSON for a `.json` path or with
|
|
111
141
|
`--format json`. A prompt appears in the file once it's published.
|
|
112
142
|
|
|
113
143
|
## Model settings
|
|
114
144
|
|
|
115
|
-
A version can carry the model to call it with
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
145
|
+
A version can carry the model to call it with and how: its temperature and output limit, stop
|
|
146
|
+
sequences, an answer in JSON, the provider and how to reach it, a timeout, retries and parameters
|
|
147
|
+
of your own, all set next to the text in the dashboard. Moving to a cheaper model is then a publish
|
|
148
|
+
like any other: one click rolls it back, and an A/B test can tell whether it does as well.
|
|
149
|
+
|
|
150
|
+
Each API wants them in its own place, so the SDK has a request builder for each:
|
|
151
|
+
`toOpenAIResponses`, `toChatCompletions` (OpenAI's and every compatible host's), `toAnthropic`,
|
|
152
|
+
`toGemini` (`@google/genai`) and `toBedrockConverse`. Give it what `resolve` returned and your own
|
|
153
|
+
request; it returns that request with the version's messages and settings in, and the options for
|
|
154
|
+
`create()`:
|
|
119
155
|
|
|
120
156
|
```ts
|
|
157
|
+
import Anthropic from "@anthropic-ai/sdk";
|
|
158
|
+
import { PromptHelm, toAnthropic } from "@prompt-helm/sdk";
|
|
159
|
+
|
|
121
160
|
const r = await ph.resolve("support-reply", {
|
|
122
161
|
vars: { company: "Acme", customer_name: user.name },
|
|
123
162
|
fallback: SUPPORT_REPLY,
|
|
163
|
+
api: "anthropic-messages",
|
|
124
164
|
});
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
max_output_tokens: r.config.maxTokens,
|
|
130
|
-
input: r.text,
|
|
165
|
+
const { params, options } = toAnthropic(r, {
|
|
166
|
+
model: "claude-sonnet-5-5", // when the dashboard sets none, and for the fallback
|
|
167
|
+
max_tokens: 1024,
|
|
168
|
+
messages: history, // your conversation so far; the version's user message follows it
|
|
131
169
|
});
|
|
170
|
+
try {
|
|
171
|
+
const reply = await anthropic.messages.create(params, options);
|
|
172
|
+
} catch (error) {
|
|
173
|
+
r.failed(error);
|
|
174
|
+
throw error;
|
|
175
|
+
}
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
What the dashboard leaves unset stays yours; what it sets wins. The builders import no provider
|
|
179
|
+
package and call nothing. When the fallback is served, `config` is empty and your request goes out
|
|
180
|
+
as you wrote it, with the fallback's messages. `r.config` holds the same settings (`model`,
|
|
181
|
+
`temperature`, `maxTokens`, `stop`, `response`, `provider`, `apis`, `connection`, `timeoutMs`,
|
|
182
|
+
`maxRetries`, `extra`) if you'd rather map them yourself. PromptHelm doesn't check a model name
|
|
183
|
+
with your provider, and a name your provider doesn't know fails your calls: try a new model in an
|
|
184
|
+
A/B test first. SDK versions before 1.4 ignore the settings, and before 1.6 receive only the first
|
|
185
|
+
four.
|
|
186
|
+
|
|
187
|
+
`api` names the API your app calls the model with: `openai-responses`, `chat-completions`,
|
|
188
|
+
`anthropic-messages`, `gemini` or `bedrock-converse`. `config.apis` lists the ones the version's
|
|
189
|
+
model takes where it's called (OpenAI's models take Responses and Chat Completions); a version
|
|
190
|
+
whose model your API can't reach isn't served, and the call gets its fallback (`api_mismatch`).
|
|
191
|
+
PromptHelm learns which API each key's app uses, so the prompt's page warns before such a version
|
|
192
|
+
is published.
|
|
193
|
+
|
|
194
|
+
### Reaching the provider
|
|
195
|
+
|
|
196
|
+
`clientOptions(r, yours)` adds what the dashboard set for reaching the provider (an address, a
|
|
197
|
+
Region or location, a timeout and retries) to your client's options, in the names the official SDK
|
|
198
|
+
of the API you named in `resolve` uses. Keys stay in your code:
|
|
199
|
+
|
|
200
|
+
```ts
|
|
201
|
+
const bedrock = new BedrockRuntimeClient(clientOptions(r, { credentials }));
|
|
132
202
|
```
|
|
133
203
|
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
204
|
+
Build a client again when `r.config.connection` changes; most apps keep one per connection. A
|
|
205
|
+
Microsoft Foundry resource's address becomes its `/openai/v1/` or `/anthropic/` URL and a
|
|
206
|
+
Databricks workspace its `/serving-endpoints`; for anything else, the dashboard's Base URL is used
|
|
207
|
+
as it is.
|
|
208
|
+
|
|
209
|
+
### Answers in JSON
|
|
210
|
+
|
|
211
|
+
A version can ask for an answer in JSON with the fields set in the dashboard. `config.response` is
|
|
212
|
+
then `{ type: "json", name, schema }`: a JSON Schema kept inside what OpenAI, Anthropic, Gemini and
|
|
213
|
+
Amazon Bedrock all accept (and, for GPT on Microsoft Foundry, what Azure accepts), so the same
|
|
214
|
+
schema works with each. The request builders put it where each API wants it: `text.format` on
|
|
215
|
+
Responses, `response_format` on Chat Completions, `output_config.format` on Anthropic,
|
|
216
|
+
`responseJsonSchema` on Gemini and `outputConfig.textFormat` on Bedrock, as text. PromptHelm never
|
|
217
|
+
sees the answer, so check it in your code. Without a response format the model answers in text;
|
|
218
|
+
SDK versions before 1.5 ignore it.
|
|
219
|
+
|
|
220
|
+
### Refused calls
|
|
221
|
+
|
|
222
|
+
`r.failed(error)` tells PromptHelm that your provider refused the call: the HTTP status, the
|
|
223
|
+
error's type and code, and the parameter it names (OpenAI's SDK gives one), never its message.
|
|
224
|
+
The prompt's page shows them per version, so a setting your provider rejects shows up while it's
|
|
225
|
+
still one publish to roll back. Anything without an HTTP status, such as a network error, is
|
|
226
|
+
ignored. `reportFailures: false` turns it off.
|
|
138
227
|
|
|
139
228
|
## A/B tests
|
|
140
229
|
|
package/dist/cli.js
CHANGED
|
@@ -94,7 +94,11 @@ async function fetchLivePrompts(options) {
|
|
|
94
94
|
}
|
|
95
95
|
function renderFallbacks(prompts, format) {
|
|
96
96
|
const sorted = [...prompts].sort((a, b) => a.slug < b.slug ? -1 : a.slug > b.slug ? 1 : 0);
|
|
97
|
-
const
|
|
97
|
+
const entries = sorted.map((p) => [
|
|
98
|
+
p.slug,
|
|
99
|
+
p.user ? { system: p.content, user: p.user.content } : p.content
|
|
100
|
+
]);
|
|
101
|
+
const json = JSON.stringify(Object.fromEntries(entries), null, 2);
|
|
98
102
|
return format === "json" ? `${json}
|
|
99
103
|
` : `${TS_HEADER}export const fallbacks = ${json} as const;
|
|
100
104
|
`;
|